Commit Graph

1039 Commits

Author SHA1 Message Date
Brooooooklyn
e1ce42a6fc feat(native): upgrade NAPI-RS to 3.0.0 beta (#12652)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
	- Added a default export for the native binding in the frontend native module, allowing easier imports.

- **Refactor**
	- Streamlined and updated Rust-to-JavaScript type conversions and lifetime handling for improved safety and consistency.
	- Improved object and array construction in Rust modules for more idiomatic usage.
	- Simplified boolean and null value handling in JavaScript interop layers.

- **Chores**
	- Upgraded several dependencies and development tools to newer versions across backend, frontend, and common packages.
	- Updated build scripts for the frontend native package to simplify commands.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-29 16:09:32 +00:00
renovate
2a7f0162cf chore: bump up nestjs-cls version to v6 (#12648)
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [nestjs-cls](https://papooch.github.io/nestjs-cls/) ([source](https://redirect.github.com/Papooch/nestjs-cls)) | [`^5.0.0` -> `^6.0.0`](https://renovatebot.com/diffs/npm/nestjs-cls/5.4.3/6.0.0) | [![age](https://developer.mend.io/api/mc/badges/age/npm/nestjs-cls/6.0.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/npm/nestjs-cls/6.0.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/npm/nestjs-cls/5.4.3/6.0.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/nestjs-cls/5.4.3/6.0.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) |

---

### Release Notes

<details>
<summary>Papooch/nestjs-cls (nestjs-cls)</summary>

### [`v6.0.0`](https://redirect.github.com/Papooch/nestjs-cls/releases/tag/nestjs-cls%406.0.0)

[Compare Source](https://redirect.github.com/Papooch/nestjs-cls/compare/nestjs-cls@5.4.3...nestjs-cls@6.0.0)

##### Breaking Changes

-   The experimental Plugin API has been changed ([4623607](https://redirect.github.com/Papooch/nestjs-cls/commits/4623607))
-   Access to Proxy providers moved to a dedicated `proxy` property on the ClsService ([82cdeef](https://redirect.github.com/Papooch/nestjs-cls/commits/82cdeef))

##### Features

-   **core**: introduce hooks for the Plugin API ([#&#8203;283](https://redirect.github.com/Papooch/nestjs-cls/issues/283)) ([4623607](https://redirect.github.com/Papooch/nestjs-cls/commits/4623607))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/toeverything/AFFiNE).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MC4zMy42IiwidXBkYXRlZEluVmVyIjoiNDAuMzMuNiIsInRhcmdldEJyYW5jaCI6ImNhbmFyeSIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiXX0=-->
2025-05-29 15:45:33 +00:00
fengmk2
31a1841e25 chore(server): log removed job id (#12646)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **Chores**
	- Improved log messages to include job IDs when jobs are removed from the queue, enhancing traceability for users monitoring job activity.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-29 11:48:16 +00:00
fengmk2
39cb1afedb fix(server): limit rootDoc snapshot size (#12625)
close CLOUD-225

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **New Features**
  - Added support for reading document blocks without requiring a workspace or root document snapshot.
- **Bug Fixes**
  - Improved handling of large workspace snapshots by skipping them when they exceed 10MB.
- **Tests**
  - Introduced new test cases to cover scenarios where root or workspace snapshots are absent.
  - Expanded snapshot tests for document block reading.
- **Refactor**
  - Updated several function signatures to make root and workspace snapshot parameters optional for greater flexibility.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-29 05:32:31 +00:00
darkskygit
acf92aa3da fix(server): handle edge case of empty docs (#12608)
fix AI-130
2025-05-28 11:25:53 +00:00
fengmk2
85def83f5e chore(server): set log level to debug on canary (#12612)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **Refactor**
  - Adjusted logging verbosity to be more detailed in the 'canary' environment, providing debug-level logs, while maintaining info-level logs elsewhere.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-28 10:23:16 +00:00
fengmk2
9e5d132bd0 chore(server): log job start and finish (#12610)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **Refactor**
  - Improved visibility of job start and finish events by updating logging level, making these events more prominent in logs.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-28 09:51:37 +00:00
darkskygit
7ae564238d fix(server): link format in chat (#12606)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **Documentation**
  - Clarified citation formatting rules, specifying that multiple citations should not be grouped within a single bracket.
  - Added support and examples for citing web URLs in the allowed citation formats.
  - Improved formatting in the "About AFFiNE" section for better readability.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-28 09:35:45 +00:00
fengmk2
9abbfa3ab4 chore(server): print jobId (#12593)
Need to query the payload through job id for debugging

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **Tests**
  - Updated job metrics test to include an explicit job ID during execution.

- **Refactor**
  - Enhanced job execution to support an optional job ID, improving job tracking and logging.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-28 08:48:38 +00:00
fengmk2
274319dd6c fix(server): 4xx error property is optional (#12595)
close CLOUD-223

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **Bug Fixes**
  - Improved error handling for search requests to prevent issues when error details are missing, ensuring clearer fallback messages for unknown errors.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-28 07:50:05 +00:00
akumatus
eb49ffaedb feat(core): support fork session without latestMessageId (#12587)
Close [AI-86](https://linear.app/affine-design/issue/AI-86)

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **New Features**
  - Improved chat session forking to allow creating a fork without specifying the latest message, enabling more flexible session management.

- **Bug Fixes**
  - Forking a chat session with an invalid latest message ID now correctly returns an error.

- **Tests**
  - Added and updated test cases to cover session forking with missing or invalid latest message IDs, ensuring robust behavior in these scenarios.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-28 07:34:23 +00:00
pengx17
d5dd680855 fix(core): update favicon (#12581)
not changing the favicon.ico file to make sure the change will be updated on the user's browser to get rid of caching

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **Style**
  - Updated the favicon URL across the application and link previews to include a version query parameter (`?v=2`) for better cache control.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-28 05:52:16 +00:00
darkskygit
f4e7595f4b feat(server): add copilot embedding feature (#12590)
fix AI-154
2025-05-28 04:36:37 +00:00
darkskygit
9220b973c7 feat(server): increase embedding jobs concurrency & handle empty content after trim (#12574)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **Improvements**
  - Increased the default concurrency for background tasks, enhancing processing efficiency.
  - Improved handling of empty or unsupported documents to ensure consistent processing.
  - Optimized document filtering to exclude certain documents from processing, improving performance.

- **Bug Fixes**
  - Enhanced detection of empty document summaries, reducing errors during processing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-27 14:28:34 +00:00
forehalo
dc7cd0487b refactor(server): decrypt license with provided aes key (#12570)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Added support for a new AES key for license management, improving license encryption and decryption processes.

- **Bug Fixes**
  - Improved error messages and handling when activating expired or invalid licenses.

- **Refactor**
  - Updated license decryption logic to use a fixed AES key instead of deriving one from the workspace ID.
  - Added validation for environment variable values to prevent invalid configurations.

- **Tests**
  - Enhanced license-related tests to cover new key usage and updated error messages.
  - Updated environment variable validation tests with clearer error messages.

- **Chores**
  - Updated environment variable handling for improved consistency.
  - Set production environment variable explicitly in build configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-27 11:54:28 +00:00
darkskygit
3c0fa429c5 feat(server): switch i2i to gpt (#12238)
fix AI-14
fix AI-17
fix AI-39
fix AI-112

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Expanded and reorganized prompt options for text and image actions, adding new prompts for image generation, style conversions, upscaling, background removal, and sticker creation.
  - Enhanced image editing capabilities with direct support for image attachments in prompts.

- **Improvements**
  - Updated prompt names and descriptions to be more user-friendly and descriptive.
  - Simplified and clarified prompt selection and image processing workflows with improved default behaviors.
  - Better organization of prompts through clear grouping and categorization.

- **Bug Fixes**
  - Improved validation and handling of image attachments during editing requests.

- **Refactor**
  - Internal code restructuring of prompts and provider logic for clarity and maintainability without affecting user workflows.
  - Refined message handling and content merging logic to ensure consistent prompt processing.
  - Adjusted image attachment rendering logic for improved display consistency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-27 11:36:47 +00:00
darkskygit
1e9cbdb65d feat(server): use generative ai api for transcript (#12569)
fix AI-151
2025-05-27 11:36:47 +00:00
akumatus
f4cba7d6ee refactor(core): add text stream parser (#12459)
Support [AI-82](https://linear.app/affine-design/issue/AI-82).

Added a `TextStreamParser` class to standardize formatting of different types of AI stream chunks across providers.

### What changed?

- Created a new `TextStreamParser` class in `utils.ts` that handles formatting of various chunk types (text-delta, reasoning, tool-call, tool-result, error)
- Refactored the Anthropic, Gemini, and OpenAI providers to use this shared parser instead of duplicating formatting logic
- Added comprehensive tests for the new `TextStreamParser` class, including tests for individual chunk types and sequences of chunks
- Defined a common `AITools` type to standardize tool interfaces across providers

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
	- Enhanced formatting and structure for streamed AI responses, including improved handling of callouts, web search, and web crawl results.
- **Refactor**
	- Streamlined and unified the processing of streamed AI response chunks across providers for more consistent output.
- **Bug Fixes**
	- Improved error handling and display for streamed responses.
- **Tests**
	- Added comprehensive tests to ensure correct formatting and handling of various streamed message types.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-27 07:14:17 +00:00
fengmk2
409e71ff8b fix(server): use /_bulk endpoint instead (#12542)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **Bug Fixes**
	- Improved reliability of batch write operations to search providers, ensuring documents are correctly indexed and retrievable.
- **Tests**
	- Added new test cases and snapshots to verify batch write functionality and confirm resolution of prior batch processing issues.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-27 06:43:26 +00:00
forehalo
2f139bd02c chore(admin): remove useless config diff (#12545)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Added a GraphQL mutation to validate multiple app configuration updates, returning detailed validation results for each item.
  - Extended the API schema to support validation feedback, enabling client-side checks before applying changes.
  - Introduced a detailed, parameterized error message system for configuration validation errors.
  - Enabled validation of configuration inputs via the admin UI with clear, descriptive error messages.

- **Improvements**
  - Enhanced error reporting with specific, context-rich messages for invalid app configurations.
  - Simplified admin settings UI by removing the confirmation dialog and streamlining save actions.
  - Improved clarity and maintainability of validation logic and error handling components.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-27 06:07:26 +00:00
fengmk2
3e6384604c chore(server): remove request success log (#12550)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **Chores**
  - Removed detailed verbose logging for Elasticsearch requests to reduce log noise.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-27 05:26:04 +00:00
fengmk2
382c237dac fix(server): return empty summary field value (#12517)
close AF-2658

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **Tests**
  - Added new test cases and snapshots to enhance coverage for search results involving empty or missing fields like summary, title, and ref_doc_id.
  - Verified consistent handling of empty string values and absence of fields across different search providers.

- **Bug Fixes**
  - Improved handling of empty string values for specific fields by converting them to null to ensure consistent search result formatting.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-27 04:42:54 +00:00
darkskygit
8952ce4fb3 feat(server): update prompts (#12539)
fix AI-63
fix PD-2567
fix AI-150
fix AI-149
fix AI-148
fix AI-147
fix AI-146
fix AI-145
fix AI-144
fix AI-143

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Enhanced AI-generated responses for writing and code-related prompts with detailed, structured instructions for improved clarity and consistency.
  - Updated AI model for multiple prompts to deliver faster and more accurate results.

- **Bug Fixes**
  - Improved test validation for code explanation and error-checking prompts to better recognize correct outputs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-26 19:35:56 +00:00
akumatus
8b76644fc1 feat(core): use the same prompt for Search With AFFiNE AI (#12496)
Support [AI-59](https://linear.app/affine-design/issue/AI-59)

Deprecated use of the perplexity model. Makes chat and search use the same prompt content.

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

- **New Features**
  - Added support for two new AI models: Claude Opus 4 and Claude Sonnet 4, enabling enhanced text and image input capabilities.
  - Introduced a new chat prompt with detailed instructions for specialized AI interactions within AFFiNE.

- **Refactor**
  - Improved prompt management for AI chat and search features by centralizing shared settings for better consistency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-26 22:42:49 +08:00
akumatus
5fcdad46eb feat(core): add google vertex ai (#12423)
Close [AI-125](https://linear.app/affine-design/issue/AI-125)

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Added new provider configurations `geminiVertex` and `anthropicVertex` for Google Vertex AI in backend schema, provider classes, and admin config.
  - Introduced `GeminiVertexProvider` and `AnthropicVertexProvider` classes supporting Vertex AI models with specific capabilities.
  - Expanded model options for transcription prompts with newer Gemini models.
  - Re-exported provider modules to include Vertex AI variants.

- **Improvements**
  - Extended provider architecture to support separate Vertex AI configurations and models.
  - Updated test setup to replace deprecated provider references with new Vertex variants.
  - Consolidated environment variables for server testing with a single `SERVER_CONFIG`.

- **Bug Fixes**
  - Updated mock models and import references in tests to align with new provider classes.

- **Chores**
  - Added `@ai-sdk/google-vertex` dependency for Vertex AI support.
  - Updated dependency list to include `@ai-sdk/google-vertex`.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-26 13:09:29 +00:00
darkskygit
eb26e99ecd fix(server): skip embedding when not configured (#12544)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **Bug Fixes**
  - Improved reliability by ensuring certain features are only enabled when required support and configuration are present, reducing the risk of runtime errors.
  - Enhanced platform detection logic for better accuracy across different environments, including macOS and Windows systems.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-26 12:52:15 +00:00
fengmk2
e3d63896bf chore(server): add job backoff strategies (#12499)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Implemented an exponential backoff retry strategy for job queues, resulting in progressively longer wait times between retry attempts.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-26 02:33:58 +00:00
fengmk2
dfa62f7683 chore(server): support dynamic disable indexer (#12498)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **Bug Fixes**
  - Improved system stability by ensuring that indexing jobs do not run when the indexer feature is disabled.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-24 12:27:53 +00:00
darkskygit
2a80fbb993 feat(server): workspace embedding improve (#12022)
fix AI-10
fix AI-109
fix PD-2484

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Added a method to check if a document requires embedding, improving embedding efficiency.
  - Enhanced document embeddings with enriched metadata, including title, summary, creation/update dates, and author information.
  - Introduced a new type for document fragments with extended metadata fields.

- **Improvements**
  - Embedding logic now conditionally processes only documents needing updates.
  - Embedding content now includes document metadata for more informative context.
  - Expanded and improved test coverage for embedding scenarios and workspace behaviors.
  - Event emission added for workspace embedding updates on client version mismatch.
  - Job queueing enhanced with prioritization and explicit job IDs for better management.
  - Job queue calls updated to include priority and context identifiers in a structured format.

- **Bug Fixes**
  - Improved handling of ignored documents in embedding matches.
  - Fixed incorrect document ID assignment in embedding job queueing.

- **Tests**
  - Added and updated snapshot and behavioral tests for embedding and workspace document handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-23 10:16:15 +00:00
liuyi
41781902f6 feat(core): support apple sign in (#12424) 2025-05-23 15:27:27 +08:00
darkskygit
3d9b13c53c feat(server): better guard for embedding not support env (#12472) 2025-05-23 05:43:14 +00:00
darkskygit
36a764ccc4 feat(server): update prompts (#12471)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Added support for a timezone parameter in chat prompts, allowing for more personalized AI responses.

- **Documentation**
  - Updated the system prompt for "Chat With AFFiNE AI" to a clearer, sectioned format with detailed instructions and improved privacy emphasis.
  - Introduced a structured markup format to guide AI response formatting, citations, and interaction rules.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-23 05:43:13 +00:00
fengmk2
aea45f451d fix(server): avoid infinite loop in manticoresearch (#12460)
close CLOUD-221

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **Bug Fixes**
  - Resolved an issue that could cause the document listing process to enter an infinite loop under certain conditions.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-23 05:04:13 +00:00
forehalo
7978a2545f fix(server): should direct allocate seat if workspace is not team (#12469)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **New Features**
  - Added the ability for workspace owners to approve members under review, with different approval processes for team and non-team workspaces.
- **Bug Fixes**
  - Improved accuracy of workspace seat quota calculations for member management.
- **Tests**
  - Enhanced test coverage and consistency for workspace member actions, including approval and revocation scenarios.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-23 04:25:53 +00:00
forehalo
f38b8fef4d feat(server): handle account deleting properly (#12399)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Users are now prevented from deleting their account if they own one or more team workspaces. A clear error message instructs users to transfer ownership or delete those workspaces first.
  - Disabled (banned) users are explicitly prevented from signing in or re-registering.
  - Added new error messages and translations to improve clarity around account deletion restrictions.

- **Bug Fixes**
  - Disabled users are now explicitly handled to prevent sign-in attempts.

- **Tests**
  - Introduced comprehensive end-to-end tests covering account deletion, banning, and re-registration scenarios.

- **Chores**
  - Improved event handling for user deletion and subscription cancellation.
  - Updated localization resources with new error messages.
  - Renamed payment event handler class for clarity.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-23 03:57:29 +00:00
akumatus
119cf9442b feat(core): add gemini callout syntax highlighting (#12413)
Close [AI-125](https://linear.app/affine-design/issue/AI-125)

What Changed?
- Add `gemini-2.5-flash-preview-04-17` model
- Add `thinkingConfig` provider options
- Add callout syntax highlighting

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Added support for the "Gemini 2.5 Flash" model and updated the "Gemini 2.5 Pro" model to a newer version.
  - Enhanced streaming responses to better format reasoning outputs and provide clearer callouts in AI-generated content.

- **Bug Fixes**
  - Improved audio transcription prompts in test cases for more accurate and explicit testing.

- **Documentation**
  - Expanded citation instructions for AI chat responses, including examples for multiple citations.

- **Chores**
  - Updated the "@ai-sdk/google" dependency to a newer version.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-23 01:58:01 +00:00
darkskygit
477250f1b8 feat(server): extract check params (#12187)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Improved input validation and error reporting for chat messages, attachments, and embeddings, with clearer error messages for invalid inputs.
  - Enhanced support for multimodal messages, including attachments such as images or audio.

- **Refactor**
  - Unified and streamlined parameter validation across AI providers, resulting in more consistent behavior and error handling.
  - Centralized parameter checks into a common provider layer, removing duplicate validation code from individual AI providers.

- **Tests**
  - Simplified and consolidated audio transcription test stubs for better maintainability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-22 13:43:59 +00:00
darkskygit
b388f92c96 feat(server): refactor provider interface (#11665)
fix AI-4
fix AI-18

better provider/model choose to allow fallback to similar models (e.g., self-hosted) when the provider is not fully configured
split functions of different output types
2025-05-22 06:28:20 +00:00
fengmk2
c525ca24fd test(server): use mocker (#12435)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **Tests**
  - Refactored test setup for the database-backed document reader to improve modularity and clarity.
  - Simplified database initialization and cleanup in tests.
  - Updated mocks and assertions to align with the new test structure.
  - Maintained existing test coverage and logic.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-22 04:25:20 +00:00
fengmk2
346c0df800 chore(server): support disable indexer plugin (#12408)
close CLOUD-220

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **New Features**
  - Introduced a new service to handle indexing-related events and scheduled tasks, improving the management of document and workspace indexing.
  - Added support for configuring the indexer feature via the AFFINE_INDEXER_ENABLED environment variable.

- **Bug Fixes**
  - Ensured that indexing and deletion jobs are only enqueued when the indexer feature is enabled.

- **Tests**
  - Added comprehensive tests for the new indexing event service, covering various configuration scenarios.
  - Removed obsolete test related to auto-indexing scheduling.

- **Chores**
  - Updated configuration descriptions and mappings to improve clarity and environment variable support.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-21 13:19:02 +00:00
darkskygit
84667b3440 feat(server): workspace embedding status count with files (#12420)
fix AI-32
fix AI-132
2025-05-21 10:51:35 +00:00
darkskygit
7fd3ee957f fix(server): embedding chunks primary key (#12416)
fix AI-131

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **Refactor**
  - Updated database schema to consolidate unique constraints into composite primary keys for embedding-related data, improving consistency.
  - Changed the relation in the Snapshot model to allow multiple embeddings.
  - Improved filtering logic for documents and snapshots based on embedding existence.
  - Reformatted SQL queries and schema attributes for improved readability; no changes to functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-21 10:51:35 +00:00
darkskygit
c9b296c896 fix(server): process empty doc embedding (#12417)
fix CLOUD-219

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **Bug Fixes**
  - Ensured that documents without content now receive a placeholder embedding, improving consistency in document processing.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-21 09:37:22 +00:00
fengmk2
abfc994180 chore(server): support elasticsearch apiKey (#12405) 2025-05-21 08:35:51 +00:00
fengmk2
ff15779208 chore(server): add auto index batch size (#12391)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Added a configurable batch size setting for automatic workspace indexing. Users can now adjust how many workspaces are indexed per batch, with a default value of 10 and support for values between 1 and 1000.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-21 08:35:51 +00:00
darkskygit
afbda482de fix(server): skip empty docs (#12396)
fix AI-129
fix CLOUD-129
2025-05-20 10:52:00 +00:00
darkskygit
6f9361caee feat(server): trigger workspace embedding (#12328)
fix AI-127

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Added automated event handling for workspace updates and document embedding, streamlining document embedding workflows.
  - Introduced detection and queuing of documents needing embedding, excluding ignored documents.
- **Improvements**
  - Enhanced performance of embedding-related searches by filtering results at the database level.
  - Increased concurrency for embedding job processing to improve throughput.
- **Bug Fixes**
  - Improved error handling and fallback for missing document titles during embedding.
  - Added safeguards to skip invalid embedding jobs based on document identifiers.
- **Tests**
  - Expanded test coverage for document embedding and ignored document filtering.
  - Updated end-to-end tests to use dynamic content for improved reliability.
  - Added synchronization waits in document creation utilities to improve test stability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-20 05:16:45 +00:00
fengmk2
8e9d50bfe9 chore(server): add prefix to indexer jobId (#12369) 2025-05-20 03:09:23 +00:00
fengmk2
9ad76fe2f6 chore(server): support elasticsearch alias (#12363)
https://www.elastic.co/docs/manage-data/data-store/aliases
2025-05-20 03:09:22 +00:00
fengmk2
cce66f6107 fix(server): add stemmer filter (#12358)
CLOUD-214

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **New Features**
  - Improved search functionality to support stemming, allowing searches for variations of words (e.g., "window", "windows", "design") to return relevant results.
- **Tests**
  - Added new tests to verify that search results correctly highlight and match stemmed word variations in document titles.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-20 03:09:22 +00:00