close AI-186
#### PR Dependency Tree
* **PR #12811** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added a new tool enabling users to read document content and metadata
within a workspace, with enforced access control.
- **Improvements**
- Updated tool interfaces and outputs to support the document reading
functionality seamlessly.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced support for the Morph provider in the copilot module,
enabling integration with the Morph LLM API.
- Added a new document editing tool that allows users to propose and
apply edits to existing documents via AI assistance.
- **Configuration**
- Added configuration options for the Morph provider in both backend and
admin interfaces.
- **Enhancements**
- Expanded tool support to include document editing capabilities within
the copilot feature set.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Improved document reranking with a more streamlined and accurate
scoring system.
* Enhanced support for binary ("yes"/"no") document relevance judgments.
* **Improvements**
* Simplified user prompts and output formats for reranking tasks, making
results easier to interpret.
* Increased reliability and consistency in document ranking results.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<img width="775" alt="截屏2025-06-26 16 17 05"
src="https://github.com/user-attachments/assets/ed6bcae3-94af-4eb1-81e8-710f36ef5e46"
/>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced a tool to crawl web pages and extract key information.
* Added a visual component to display tool call failures in the AI
interface.
* Enhanced error reporting for document and web search tools with
structured error messages.
* **Improvements**
* Updated error handling across AI tools and components for more
consistent and informative feedback.
* Default values added for tool card components to improve reliability
and display.
* **Bug Fixes**
* Improved handling of error and empty states in web crawl and web
search result displays.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for the "AI Playground" experimental feature, including
new settings and localization entries.
* **Refactor**
* Renamed configuration and service references from "Model Switch" to
"Playground" across the AI chat and playground interfaces.
* Updated feature flag from "enable_ai_model_switch" to
"enable_ai_playground" for consistency.
* **Bug Fixes**
* The "Model" submenu in AI chat preferences is now always visible,
simplifying menu options.
* **Chores**
* Removed outdated Claude model options from the chat prompt.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-231
#### PR Dependency Tree
* **PR #12893** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Refactor**
- Updated background processing for document indexing and deletion to
use a job queue system instead of event-based triggers.
- **Bug Fixes**
- Improved reliability of embedding updates and deletions by ensuring
tasks are properly queued and processed.
- **Tests**
- Adjusted tests to verify that document operations correctly trigger
job queue actions.
No changes to user-facing features or interface.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close AI-185
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a keyword-based document search tool, allowing users to
search for relevant documents within their workspace using keywords.
- Search results include document titles, summaries, and direct links,
enhancing document discovery and navigation.
- **Bug Fixes**
- None.
- **Tests**
- Added new tests to verify document search by IDs and by keywords,
ensuring accurate and reliable search functionality.
- **Documentation**
- None.
- **Chores**
- Updated configuration file organization for improved clarity; no
changes to functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #12867**
* **PR #12863**
* **PR #12837** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
#### PR Dependency Tree
* **PR #12871** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Refactor**
- Updated internal logging mechanism to use a standard logger, resulting
in a simplified and more consistent logging approach. No changes to
user-facing features or functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #12871** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
#### PR Dependency Tree
* **PR #12857** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved error handling in document embedding jobs to log and skip
errors instead of interrupting the process.
- Enhanced logging with clearer messages about embedding outcomes and
skipped operations.
- Added warnings when documents or summaries are missing during
embedding.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Close [AI-193](https://linear.app/affine-design/issue/AI-193)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added support for streaming structured AI chat responses as objects,
enabling richer and more interactive chat experiences.
- Chat messages now include a new field displaying structured stream
objects, such as reasoning steps, text deltas, tool calls, and tool
results.
- GraphQL APIs and queries updated to expose these structured streaming
objects in chat histories.
- Introduced a new streaming chat endpoint for object-based responses.
- **Bug Fixes**
- Improved error handling for streaming responses to ensure more robust
and informative error reporting.
- **Refactor**
- Centralized and streamlined session preparation and streaming logic
for AI chat providers.
- Unified streaming setup across multiple AI model providers.
- **Tests**
- Extended test coverage for streaming object responses to ensure
reliability and correctness.
- **Documentation**
- Updated type definitions and schemas to reflect new streaming object
capabilities in both backend and frontend code.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Co-authored-by: DarkSky <25152247+darkskygit@users.noreply.github.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Updated Gemini model identifiers from preview versions to their latest
stable versions across prompts and provider configurations. This ensures
continued compatibility and access to the latest Gemini 2.5 Flash and
Pro models. No changes to user-facing features or prompt content.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Updated document-related tools in chat prompts, including new options
for document keyword and semantic search.
- **Refactor**
- Renamed and reorganized document search tools for improved clarity in
tool selection and results display.
- **Bug Fixes**
- Ensured tool identifiers and results are consistent across chat and
search features.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Refactor**
- Improved the handling of attachments in chat messages for more efficient processing of images and files without impacting user experience.
- **Chores**
- Added internal logging to enhance monitoring of AI model interactions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Support [AI-59](https://linear.app/affine-design/issue/AI-59)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Updated the default AI model for chat prompts to use Claude Sonnet 4.
- **Bug Fixes**
- Improved model selection logic to better support reasoning features across more AI models.
- **Tests**
- Enhanced test cases with consistent instructions for response length.
- Skipped certain chat-related tests to refine test suite stability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix AI-163
fix AI-164
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Tests**
- Added a new test to verify multi-turn chat interactions, ensuring accurate handling of chat history and correct responses for translation and explanation requests.
- **Bug Fixes**
- Improved chat session logic to better merge user messages and attachments, enhancing the accuracy and continuity of multi-step conversations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Chores**
- Improved internal monitoring for AI embedding operations to enhance reliability and performance tracking. No changes to user-facing features.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved handling of workspace embedding events to ensure they are only triggered for workspace-type spaces.
- **Chores**
- Added additional debug logging for document embedding jobs to aid in monitoring and troubleshooting.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Documentation**
- Clarified citation formatting rules, specifying that multiple citations should not be grouped within a single bracket.
- Added support and examples for citing web URLs in the allowed citation formats.
- Improved formatting in the "About AFFiNE" section for better readability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Close [AI-86](https://linear.app/affine-design/issue/AI-86)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Improved chat session forking to allow creating a fork without specifying the latest message, enabling more flexible session management.
- **Bug Fixes**
- Forking a chat session with an invalid latest message ID now correctly returns an error.
- **Tests**
- Added and updated test cases to cover session forking with missing or invalid latest message IDs, ensuring robust behavior in these scenarios.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Improvements**
- Increased the default concurrency for background tasks, enhancing processing efficiency.
- Improved handling of empty or unsupported documents to ensure consistent processing.
- Optimized document filtering to exclude certain documents from processing, improving performance.
- **Bug Fixes**
- Enhanced detection of empty document summaries, reducing errors during processing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix AI-14
fix AI-17
fix AI-39
fix AI-112
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Expanded and reorganized prompt options for text and image actions, adding new prompts for image generation, style conversions, upscaling, background removal, and sticker creation.
- Enhanced image editing capabilities with direct support for image attachments in prompts.
- **Improvements**
- Updated prompt names and descriptions to be more user-friendly and descriptive.
- Simplified and clarified prompt selection and image processing workflows with improved default behaviors.
- Better organization of prompts through clear grouping and categorization.
- **Bug Fixes**
- Improved validation and handling of image attachments during editing requests.
- **Refactor**
- Internal code restructuring of prompts and provider logic for clarity and maintainability without affecting user workflows.
- Refined message handling and content merging logic to ensure consistent prompt processing.
- Adjusted image attachment rendering logic for improved display consistency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Support [AI-82](https://linear.app/affine-design/issue/AI-82).
Added a `TextStreamParser` class to standardize formatting of different types of AI stream chunks across providers.
### What changed?
- Created a new `TextStreamParser` class in `utils.ts` that handles formatting of various chunk types (text-delta, reasoning, tool-call, tool-result, error)
- Refactored the Anthropic, Gemini, and OpenAI providers to use this shared parser instead of duplicating formatting logic
- Added comprehensive tests for the new `TextStreamParser` class, including tests for individual chunk types and sequences of chunks
- Defined a common `AITools` type to standardize tool interfaces across providers
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Enhanced formatting and structure for streamed AI responses, including improved handling of callouts, web search, and web crawl results.
- **Refactor**
- Streamlined and unified the processing of streamed AI response chunks across providers for more consistent output.
- **Bug Fixes**
- Improved error handling and display for streamed responses.
- **Tests**
- Added comprehensive tests to ensure correct formatting and handling of various streamed message types.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix AI-63
fix PD-2567
fix AI-150
fix AI-149
fix AI-148
fix AI-147
fix AI-146
fix AI-145
fix AI-144
fix AI-143
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Enhanced AI-generated responses for writing and code-related prompts with detailed, structured instructions for improved clarity and consistency.
- Updated AI model for multiple prompts to deliver faster and more accurate results.
- **Bug Fixes**
- Improved test validation for code explanation and error-checking prompts to better recognize correct outputs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Support [AI-59](https://linear.app/affine-design/issue/AI-59)
Deprecated use of the perplexity model. Makes chat and search use the same prompt content.
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
- **New Features**
- Added support for two new AI models: Claude Opus 4 and Claude Sonnet 4, enabling enhanced text and image input capabilities.
- Introduced a new chat prompt with detailed instructions for specialized AI interactions within AFFiNE.
- **Refactor**
- Improved prompt management for AI chat and search features by centralizing shared settings for better consistency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Close [AI-125](https://linear.app/affine-design/issue/AI-125)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added new provider configurations `geminiVertex` and `anthropicVertex` for Google Vertex AI in backend schema, provider classes, and admin config.
- Introduced `GeminiVertexProvider` and `AnthropicVertexProvider` classes supporting Vertex AI models with specific capabilities.
- Expanded model options for transcription prompts with newer Gemini models.
- Re-exported provider modules to include Vertex AI variants.
- **Improvements**
- Extended provider architecture to support separate Vertex AI configurations and models.
- Updated test setup to replace deprecated provider references with new Vertex variants.
- Consolidated environment variables for server testing with a single `SERVER_CONFIG`.
- **Bug Fixes**
- Updated mock models and import references in tests to align with new provider classes.
- **Chores**
- Added `@ai-sdk/google-vertex` dependency for Vertex AI support.
- Updated dependency list to include `@ai-sdk/google-vertex`.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved reliability by ensuring certain features are only enabled when required support and configuration are present, reducing the risk of runtime errors.
- Enhanced platform detection logic for better accuracy across different environments, including macOS and Windows systems.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix AI-10
fix AI-109
fix PD-2484
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added a method to check if a document requires embedding, improving embedding efficiency.
- Enhanced document embeddings with enriched metadata, including title, summary, creation/update dates, and author information.
- Introduced a new type for document fragments with extended metadata fields.
- **Improvements**
- Embedding logic now conditionally processes only documents needing updates.
- Embedding content now includes document metadata for more informative context.
- Expanded and improved test coverage for embedding scenarios and workspace behaviors.
- Event emission added for workspace embedding updates on client version mismatch.
- Job queueing enhanced with prioritization and explicit job IDs for better management.
- Job queue calls updated to include priority and context identifiers in a structured format.
- **Bug Fixes**
- Improved handling of ignored documents in embedding matches.
- Fixed incorrect document ID assignment in embedding job queueing.
- **Tests**
- Added and updated snapshot and behavioral tests for embedding and workspace document handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
DarkSky