#### PR Dependency Tree
* **PR #15282** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added automated document cleanup to reconcile missing workspace docs,
delete related stored data, and recover if the doc returns.
* Added effect-based follow-up reconciliation for search indexing,
Copilot embeddings, and comment attachment cleanup with explicit
acknowledgements.
* **Bug Fixes**
* Deleted-document references now persist as dangling references rather
than disappearing.
* Improved document deletion flow to enforce permissions and ensure
authorized deletions succeed.
* **Tests**
* Expanded coverage for cleanup recovery, indexing/embedding
reconciliation, permissions, and reference semantics.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #14996** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Admin mutations to grant/revoke commercial entitlements.
* New Doc comment-update permission.
* Realtime user/workspace quota-state endpoints and live-update rooms.
* **Bug Fixes**
* More accurate readable-doc filtering and permission evaluation.
* **Refactor**
* Workspace feature management moved to entitlement-based model;
permission and quota pipelines redesigned.
* Admin workspace UI now edits flags only (feature toggles removed).
* **Tests**
* Extensive new and updated tests for permissions, entitlements, quota,
projection, and backfills.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/toeverything/AFFiNE/pull/14996?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #14770** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Implemented batch processing for calendar synchronization to improve
performance and resource utilization.
* Added distributed locking to prevent concurrent operations in
multi-instance environments.
* **Bug Fixes**
* Improved reliability by preventing duplicate synchronization attempts.
* **Tests**
* Enhanced test coverage for batch processing and locking mechanisms.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix#13862
#### PR Dependency Tree
* **PR #14698** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Enhanced search support for Chinese, Japanese, and Korean languages
with improved text segmentation and character matching.
* Added index management capabilities with table recreation
functionality.
* **Bug Fixes**
* Improved search accuracy for non-Latin scripts through updated
morphology and n-gram configuration.
* **Chores**
* Added database migration for search index optimization.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Upgraded TypeScript toolchain to v5.9.3 across packages and tooling.
* Removed legacy ts-node and migrated developer tooling to newer
runtimes (tsx/SWC) where applicable.
* **Documentation**
* Updated developer CLI docs and runtime behavior notes to reflect the
new loader/runtime for running TypeScript files; no changes to public
APIs or end-user behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #14251** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Current user profile now exposes access tokens, revealed tokens, and
detailed calendar accounts/subscriptions.
* Workspace now exposes permissions, calendars, calendar events, and a
workspace-scoped blob upload part URL.
* New document-update mutation for applying doc updates.
* **API Changes**
* validateAppConfig is now a query (mutation deprecated).
* Several legacy top-level calendar/blob endpoints deprecated in favor
of user/workspace fields.
* **Refactor**
* Calendar, blob-upload and access-token surfaces reorganized to use
user/workspace-centric fields.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #14250** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Backend Optimization**
* Faster document retrieval via a native binary fetch path.
* Native-accelerated merging of document updates for improved
performance and consistency.
* **Indexing & Reliability**
* Indexing now only proceeds on valid parse results, with clearer
warnings and richer metadata on failures.
* More consistent sync behavior and enhanced diagnostic logging for
indexing operations.
* **Tests**
* Expanded tests to cover native binary retrieval error handling.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix AI-408
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a new field to workspaces to track the last time embeddings were
checked.
* Cleanup jobs for workspace embeddings now skip workspaces that haven't
changed in over 30 days or have no embeddings, improving efficiency.
* Cleanup jobs are now automatically triggered when a workspace is
updated.
* **Improvements**
* Enhanced workspace selection for cleanup and indexing tasks to use
more precise filters and batching.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close AF-2736
#### PR Dependency Tree
* **PR #13162** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved error handling for Elasticsearch operations by allowing
certain conflict errors to be ignored, resulting in more robust and
tolerant behavior during data deletion processes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Sessions associated with a deleted document are now automatically
updated to remove the document reference.
* **Improvements**
* Enhanced session management to better handle documents that have been
deleted.
No visible changes to the user interface; these updates improve backend
handling of session and document relationships.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-231
#### PR Dependency Tree
* **PR #12893** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Refactor**
- Updated background processing for document indexing and deletion to
use a job queue system instead of event-based triggers.
- **Bug Fixes**
- Improved reliability of embedding updates and deletions by ensuring
tasks are properly queued and processed.
- **Tests**
- Adjusted tests to verify that document operations correctly trigger
job queue actions.
No changes to user-facing features or interface.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close AI-220
#### PR Dependency Tree
* **PR #12866** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new document search capability, allowing users to search
for documents by keyword within a workspace.
- Search results include document details such as title, highlights,
creation and update timestamps, and creator/updater information.
- Added support for limiting the number of search results returned.
- **Tests**
- Added comprehensive end-to-end and snapshot tests to ensure accuracy
and access control for the new search functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #12867**
* **PR #12863** 👈
* **PR #12837**
* **PR #12866**
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
#### PR Dependency Tree
* **PR #12867** 👈
* **PR #12863**
* **PR #12837**
* **PR #12866**
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Date fields in search results are now returned as JavaScript Date
objects instead of strings or numeric timestamps.
- **Bug Fixes**
- Improved consistency and correctness of date field types across
different search providers.
- **Tests**
- Added tests to verify that date fields are correctly returned as Date
objects.
- Updated existing test snapshots to reflect the new Date object format
for date fields.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #12822** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added the ability to search for blob names by their IDs within a
workspace.
- **Tests**
- Introduced new test cases and snapshot tests to validate searching
blob names and reading filenames from document snapshots.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Chores**
- Updated the default setting for the indexer feature to be disabled by default.
- Added a sample environment variable for enabling the indexer in the example configuration file.
- Introduced a new environment variable for the indexer in the CI workflow configuration.
- **Tests**
- Adjusted test configurations to explicitly enable the indexer feature during test execution.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-225
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added support for reading document blocks without requiring a workspace or root document snapshot.
- **Bug Fixes**
- Improved handling of large workspace snapshots by skipping them when they exceed 10MB.
- **Tests**
- Introduced new test cases to cover scenarios where root or workspace snapshots are absent.
- Expanded snapshot tests for document block reading.
- **Refactor**
- Updated several function signatures to make root and workspace snapshot parameters optional for greater flexibility.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-223
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved error handling for search requests to prevent issues when error details are missing, ensuring clearer fallback messages for unknown errors.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved reliability of batch write operations to search providers, ensuring documents are correctly indexed and retrievable.
- **Tests**
- Added new test cases and snapshots to verify batch write functionality and confirm resolution of prior batch processing issues.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Chores**
- Removed detailed verbose logging for Elasticsearch requests to reduce log noise.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close AF-2658
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Tests**
- Added new test cases and snapshots to enhance coverage for search results involving empty or missing fields like summary, title, and ref_doc_id.
- Verified consistent handling of empty string values and absence of fields across different search providers.
- **Bug Fixes**
- Improved handling of empty string values for specific fields by converting them to null to ensure consistent search result formatting.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved system stability by ensuring that indexing jobs do not run when the indexer feature is disabled.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-221
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Bug Fixes**
- Resolved an issue that could cause the document listing process to enter an infinite loop under certain conditions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-220
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Introduced a new service to handle indexing-related events and scheduled tasks, improving the management of document and workspace indexing.
- Added support for configuring the indexer feature via the AFFINE_INDEXER_ENABLED environment variable.
- **Bug Fixes**
- Ensured that indexing and deletion jobs are only enqueued when the indexer feature is enabled.
- **Tests**
- Added comprehensive tests for the new indexing event service, covering various configuration scenarios.
- Removed obsolete test related to auto-indexing scheduling.
- **Chores**
- Updated configuration descriptions and mappings to improve clarity and environment variable support.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added a configurable batch size setting for automatic workspace indexing. Users can now adjust how many workspaces are indexed per batch, with a default value of 10 and support for values between 1 and 1000.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
CLOUD-214
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Improved search functionality to support stemming, allowing searches for variations of words (e.g., "window", "windows", "design") to return relevant results.
- **Tests**
- Added new tests to verify that search results correctly highlight and match stemmed word variations in document titles.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-207
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Introduced automated periodic indexing of workspaces with a new job type and a scheduled cron job running every 30 seconds.
- Added a unique sequential identifier (`sid`) and an "indexed" flag to workspaces to track indexing status.
- **Improvements**
- Enhanced workspace indexing to handle missing workspaces and snapshots distinctly and selectively index documents.
- Added ability to query workspaces after a given identifier with result limits.
- **Bug Fixes**
- Improved error handling and logging during workspace indexing operations.
- **Tests**
- Expanded test coverage for workspace indexing and auto-indexing, including scheduling and edge cases.
- **Chores**
- Updated data models and schema to support new workspace fields and indexing features.
- Enhanced mock data utilities to allow custom timestamps.
- Improved type safety and flexibility in document snapshot retrieval.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-208
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Enhanced document access control with batch permission checks, enabling efficient filtering of documents based on user roles and permissions.
- Added detailed document-level role and permission management for workspace users.
- **Bug Fixes**
- Improved accuracy in filtering search results to only display documents users have permission to read.
- **Tests**
- Added comprehensive tests for document-level permission filtering and search result accuracy.
- Introduced new mock utilities to support permission-related test scenarios.
- **Refactor**
- Simplified and optimized permission logic for determining user roles and document access.
- **Documentation**
- Updated type definitions for improved clarity in permission handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-217
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Introduced a dedicated test job for Elasticsearch, running only Elasticsearch-specific tests during CI.
- **Chores**
- Enhanced server test workflows with explicit setup steps and automated coverage uploads.
- Improved test suite structure for Elasticsearch provider to enable conditional and asynchronous test execution based on environment variables.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-137
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Introduced advanced workspace-scoped search and aggregation capabilities with support for complex queries, highlights, and pagination.
- Added pluggable search providers: Elasticsearch and Manticoresearch.
- New GraphQL queries, schema types, and resolver support for search and aggregation.
- Enhanced configuration options for search providers in self-hosted and cloud deployments.
- Added Docker Compose services and environment variables for Elasticsearch and Manticoresearch.
- Integrated indexer service into deployment and CI workflows.
- **Bug Fixes**
- Improved error handling with new user-friendly error messages for search provider and indexer issues.
- **Documentation**
- Updated configuration examples and environment variable references for indexer and search providers.
- **Tests**
- Added extensive end-to-end and provider-specific tests covering indexing, searching, aggregation, deletion, and error cases.
- Included snapshot tests and test fixtures for search providers.
- **Chores**
- Updated deployment scripts, Helm charts, and Kubernetes manifests to include indexer-related environment variables and secrets.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
DarkSky
