Close [AI-193](https://linear.app/affine-design/issue/AI-193)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added support for streaming structured AI chat responses as objects,
enabling richer and more interactive chat experiences.
- Chat messages now include a new field displaying structured stream
objects, such as reasoning steps, text deltas, tool calls, and tool
results.
- GraphQL APIs and queries updated to expose these structured streaming
objects in chat histories.
- Introduced a new streaming chat endpoint for object-based responses.
- **Bug Fixes**
- Improved error handling for streaming responses to ensure more robust
and informative error reporting.
- **Refactor**
- Centralized and streamlined session preparation and streaming logic
for AI chat providers.
- Unified streaming setup across multiple AI model providers.
- **Tests**
- Extended test coverage for streaming object responses to ensure
reliability and correctness.
- **Documentation**
- Updated type definitions and schemas to reflect new streaming object
capabilities in both backend and frontend code.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Co-authored-by: DarkSky <25152247+darkskygit@users.noreply.github.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Chores**
- Updated Gemini model identifiers from preview versions to their latest
stable versions across prompts and provider configurations. This ensures
continued compatibility and access to the latest Gemini 2.5 Flash and
Pro models. No changes to user-facing features or prompt content.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Updated document-related tools in chat prompts, including new options
for document keyword and semantic search.
- **Refactor**
- Renamed and reorganized document search tools for improved clarity in
tool selection and results display.
- **Bug Fixes**
- Ensured tool identifiers and results are consistent across chat and
search features.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Refactor**
- Improved the handling of attachments in chat messages for more efficient processing of images and files without impacting user experience.
- **Chores**
- Added internal logging to enhance monitoring of AI model interactions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Support [AI-59](https://linear.app/affine-design/issue/AI-59)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Updated the default AI model for chat prompts to use Claude Sonnet 4.
- **Bug Fixes**
- Improved model selection logic to better support reasoning features across more AI models.
- **Tests**
- Enhanced test cases with consistent instructions for response length.
- Skipped certain chat-related tests to refine test suite stability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix AI-163
fix AI-164
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Tests**
- Added a new test to verify multi-turn chat interactions, ensuring accurate handling of chat history and correct responses for translation and explanation requests.
- **Bug Fixes**
- Improved chat session logic to better merge user messages and attachments, enhancing the accuracy and continuity of multi-step conversations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Chores**
- Improved internal monitoring for AI embedding operations to enhance reliability and performance tracking. No changes to user-facing features.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved handling of workspace embedding events to ensure they are only triggered for workspace-type spaces.
- **Chores**
- Added additional debug logging for document embedding jobs to aid in monitoring and troubleshooting.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Documentation**
- Clarified citation formatting rules, specifying that multiple citations should not be grouped within a single bracket.
- Added support and examples for citing web URLs in the allowed citation formats.
- Improved formatting in the "About AFFiNE" section for better readability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Close [AI-86](https://linear.app/affine-design/issue/AI-86)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Improved chat session forking to allow creating a fork without specifying the latest message, enabling more flexible session management.
- **Bug Fixes**
- Forking a chat session with an invalid latest message ID now correctly returns an error.
- **Tests**
- Added and updated test cases to cover session forking with missing or invalid latest message IDs, ensuring robust behavior in these scenarios.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Improvements**
- Increased the default concurrency for background tasks, enhancing processing efficiency.
- Improved handling of empty or unsupported documents to ensure consistent processing.
- Optimized document filtering to exclude certain documents from processing, improving performance.
- **Bug Fixes**
- Enhanced detection of empty document summaries, reducing errors during processing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix AI-14
fix AI-17
fix AI-39
fix AI-112
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Expanded and reorganized prompt options for text and image actions, adding new prompts for image generation, style conversions, upscaling, background removal, and sticker creation.
- Enhanced image editing capabilities with direct support for image attachments in prompts.
- **Improvements**
- Updated prompt names and descriptions to be more user-friendly and descriptive.
- Simplified and clarified prompt selection and image processing workflows with improved default behaviors.
- Better organization of prompts through clear grouping and categorization.
- **Bug Fixes**
- Improved validation and handling of image attachments during editing requests.
- **Refactor**
- Internal code restructuring of prompts and provider logic for clarity and maintainability without affecting user workflows.
- Refined message handling and content merging logic to ensure consistent prompt processing.
- Adjusted image attachment rendering logic for improved display consistency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Support [AI-82](https://linear.app/affine-design/issue/AI-82).
Added a `TextStreamParser` class to standardize formatting of different types of AI stream chunks across providers.
### What changed?
- Created a new `TextStreamParser` class in `utils.ts` that handles formatting of various chunk types (text-delta, reasoning, tool-call, tool-result, error)
- Refactored the Anthropic, Gemini, and OpenAI providers to use this shared parser instead of duplicating formatting logic
- Added comprehensive tests for the new `TextStreamParser` class, including tests for individual chunk types and sequences of chunks
- Defined a common `AITools` type to standardize tool interfaces across providers
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Enhanced formatting and structure for streamed AI responses, including improved handling of callouts, web search, and web crawl results.
- **Refactor**
- Streamlined and unified the processing of streamed AI response chunks across providers for more consistent output.
- **Bug Fixes**
- Improved error handling and display for streamed responses.
- **Tests**
- Added comprehensive tests to ensure correct formatting and handling of various streamed message types.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix AI-63
fix PD-2567
fix AI-150
fix AI-149
fix AI-148
fix AI-147
fix AI-146
fix AI-145
fix AI-144
fix AI-143
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Enhanced AI-generated responses for writing and code-related prompts with detailed, structured instructions for improved clarity and consistency.
- Updated AI model for multiple prompts to deliver faster and more accurate results.
- **Bug Fixes**
- Improved test validation for code explanation and error-checking prompts to better recognize correct outputs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Support [AI-59](https://linear.app/affine-design/issue/AI-59)
Deprecated use of the perplexity model. Makes chat and search use the same prompt content.
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
- **New Features**
- Added support for two new AI models: Claude Opus 4 and Claude Sonnet 4, enabling enhanced text and image input capabilities.
- Introduced a new chat prompt with detailed instructions for specialized AI interactions within AFFiNE.
- **Refactor**
- Improved prompt management for AI chat and search features by centralizing shared settings for better consistency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Close [AI-125](https://linear.app/affine-design/issue/AI-125)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added new provider configurations `geminiVertex` and `anthropicVertex` for Google Vertex AI in backend schema, provider classes, and admin config.
- Introduced `GeminiVertexProvider` and `AnthropicVertexProvider` classes supporting Vertex AI models with specific capabilities.
- Expanded model options for transcription prompts with newer Gemini models.
- Re-exported provider modules to include Vertex AI variants.
- **Improvements**
- Extended provider architecture to support separate Vertex AI configurations and models.
- Updated test setup to replace deprecated provider references with new Vertex variants.
- Consolidated environment variables for server testing with a single `SERVER_CONFIG`.
- **Bug Fixes**
- Updated mock models and import references in tests to align with new provider classes.
- **Chores**
- Added `@ai-sdk/google-vertex` dependency for Vertex AI support.
- Updated dependency list to include `@ai-sdk/google-vertex`.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved reliability by ensuring certain features are only enabled when required support and configuration are present, reducing the risk of runtime errors.
- Enhanced platform detection logic for better accuracy across different environments, including macOS and Windows systems.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix AI-10
fix AI-109
fix PD-2484
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added a method to check if a document requires embedding, improving embedding efficiency.
- Enhanced document embeddings with enriched metadata, including title, summary, creation/update dates, and author information.
- Introduced a new type for document fragments with extended metadata fields.
- **Improvements**
- Embedding logic now conditionally processes only documents needing updates.
- Embedding content now includes document metadata for more informative context.
- Expanded and improved test coverage for embedding scenarios and workspace behaviors.
- Event emission added for workspace embedding updates on client version mismatch.
- Job queueing enhanced with prioritization and explicit job IDs for better management.
- Job queue calls updated to include priority and context identifiers in a structured format.
- **Bug Fixes**
- Improved handling of ignored documents in embedding matches.
- Fixed incorrect document ID assignment in embedding job queueing.
- **Tests**
- Added and updated snapshot and behavioral tests for embedding and workspace document handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added support for a timezone parameter in chat prompts, allowing for more personalized AI responses.
- **Documentation**
- Updated the system prompt for "Chat With AFFiNE AI" to a clearer, sectioned format with detailed instructions and improved privacy emphasis.
- Introduced a structured markup format to guide AI response formatting, citations, and interaction rules.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Close [AI-125](https://linear.app/affine-design/issue/AI-125)
What Changed?
- Add `gemini-2.5-flash-preview-04-17` model
- Add `thinkingConfig` provider options
- Add callout syntax highlighting
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added support for the "Gemini 2.5 Flash" model and updated the "Gemini 2.5 Pro" model to a newer version.
- Enhanced streaming responses to better format reasoning outputs and provide clearer callouts in AI-generated content.
- **Bug Fixes**
- Improved audio transcription prompts in test cases for more accurate and explicit testing.
- **Documentation**
- Expanded citation instructions for AI chat responses, including examples for multiple citations.
- **Chores**
- Updated the "@ai-sdk/google" dependency to a newer version.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Improved input validation and error reporting for chat messages, attachments, and embeddings, with clearer error messages for invalid inputs.
- Enhanced support for multimodal messages, including attachments such as images or audio.
- **Refactor**
- Unified and streamlined parameter validation across AI providers, resulting in more consistent behavior and error handling.
- Centralized parameter checks into a common provider layer, removing duplicate validation code from individual AI providers.
- **Tests**
- Simplified and consolidated audio transcription test stubs for better maintainability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix AI-4
fix AI-18
better provider/model choose to allow fallback to similar models (e.g., self-hosted) when the provider is not fully configured
split functions of different output types
fix AI-131
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Refactor**
- Updated database schema to consolidate unique constraints into composite primary keys for embedding-related data, improving consistency.
- Changed the relation in the Snapshot model to allow multiple embeddings.
- Improved filtering logic for documents and snapshots based on embedding existence.
- Reformatted SQL queries and schema attributes for improved readability; no changes to functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix CLOUD-219
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Bug Fixes**
- Ensured that documents without content now receive a placeholder embedding, improving consistency in document processing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix AI-127
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added automated event handling for workspace updates and document embedding, streamlining document embedding workflows.
- Introduced detection and queuing of documents needing embedding, excluding ignored documents.
- **Improvements**
- Enhanced performance of embedding-related searches by filtering results at the database level.
- Increased concurrency for embedding job processing to improve throughput.
- **Bug Fixes**
- Improved error handling and fallback for missing document titles during embedding.
- Added safeguards to skip invalid embedding jobs based on document identifiers.
- **Tests**
- Expanded test coverage for document embedding and ignored document filtering.
- Updated end-to-end tests to use dynamic content for improved reliability.
- Added synchronization waits in document creation utilities to improve test stability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Close [AI-86](https://linear.app/affine-design/issue/AI-86)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Introduced AI model switching in chat, allowing users to select from multiple AI models during conversations.
- Added a floating menu for easy AI model selection within the chat interface.
- Enabled visibility of the AI model switcher through a new experimental feature flag, configurable in workspace settings (canary builds only).
- **Enhancements**
- Improved session management in the chat panel for smoother model switching and state handling.
- Updated localization to support the new AI model switch feature in settings.
- **Bug Fixes**
- None.
- **Chores**
- Registered new components and services to support AI model switching functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Close [AI-126](https://linear.app/affine-design/issue/AI-126)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Introduced a new web crawling tool, allowing users to extract live content from specific web pages in addition to traditional web search.
- **Improvements**
- Enhanced error handling for web search and web crawl operations, providing clearer failure messages.
- Updated terminology in AI prompts and user-facing messages to reflect the new web search/crawl capabilities.
- Improved formatting of web search and crawl results for better readability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Close [AI-116](https://linear.app/affine-design/issue/AI-116)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added the ability to retrieve detailed information for a specific Copilot session by its ID, including model metadata and optional models, via the user interface and API.
- Session data now includes additional fields such as the model used and a list of optional models.
- Enhanced GraphQL queries and UI components to support fetching and displaying these new session details.
- **Improvements**
- Session lists now provide richer information, including model details, for each session.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Enhanced AI-powered summary, translation, writing improvement, and spelling correction actions with more detailed and tailored instructions for improved output quality and user experience.
- **Improvements**
- Refined language detection, output formatting, and context-sensitive handling across AI actions for more accurate and relevant results.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix AI-20
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Enhanced file metadata with MIME type, blob ID, and file name across context and workspace, now visible in UI and API.
- Added workspace-level matching for files and documents with configurable thresholds and workspace scoping in search queries.
- Introduced a new error type and user-friendly messaging for global workspace context matching failures.
- **Bug Fixes**
- Improved consistent handling of file MIME types and nullable context IDs for accurate metadata.
- **Documentation**
- Updated GraphQL schema, queries, and mutations to include new metadata fields, optional parameters, and error types.
- **Style**
- Added new localization strings for global context matching error messages.
- **Tests**
- Extended test coverage with new and updated snapshot tests for metadata and matching logic.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Refactor**
- Updated the user message in the brainstorming workflow to clarify that provided data should be treated as raw input, not as a command.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Close [AI-116](https://linear.app/affine-design/issue/AI-116)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added support for specifying alternative AI models in chat prompts, enabling users to select from multiple available models.
- Expanded AI model options with new additions: 'gpt-4.1', 'o3', and 'claude-3-5-sonnet-20241022'.
- **Enhancements**
- Users can now optionally choose a specific AI model during chat interactions.
- Prompts and chat sessions reflect and support selection of alternative models where applicable.
- **Bug Fixes**
- Improved handling of prompt configuration defaults for better reliability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
DarkSky
