close AF-2723
#### PR Dependency Tree
* **PR #13113** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Comment attachments now track and display the user who uploaded them.
* **Tests**
* Updated tests to verify that the uploader’s information is correctly
stored and retrieved with comment attachments.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #13109** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Updated internal handling of comment attachments to improve processing
logic. No visible changes to end-user features or workflows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Before 11m
After 7m
#### PR Dependency Tree
* **PR #13103** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Expanded cloud end-to-end test coverage by increasing test shards from
6 to 10 for improved parallelization.
* Added a new suite of end-to-end tests focused on page sharing,
including scenarios for sharing links, table of contents, edgeless mode,
and image previews.
* Removed several redundant or relocated sharing-related tests,
retaining only the reference link verification in the affected suite.
* Updated test output format to use the "list" reporter for clearer test
results.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
keep compatibility
close AF-2720
#### PR Dependency Tree
* **PR #13091** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Summary by CodeRabbit
* **New Features**
* Added a new `LocalWorkspace` feature flag to server configuration,
enabling more flexible feature management.
* **Deprecations**
* The `allowGuestDemoWorkspace` flag is now deprecated and will be
removed in version 0.25.0. Please use the `features` array for feature
checks instead.
* **Bug Fixes**
* Updated UI and logic throughout the app to rely on the new
`LocalWorkspace` feature flag rather than the deprecated boolean flag.
* **Chores**
* Removed references to `allowGuestDemoWorkspace` from configuration,
queries, and type definitions for improved consistency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #13091** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
https://github.com/toeverything/AFFiNE/pull/12816
#### PR Dependency Tree
* **PR #12935** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated the default Manticore Search version in development
environment configurations from 9.3.2 to 10.1.0.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close AF-2719
#### PR Dependency Tree
* **PR #13083** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added the ability to mark all notifications as read with a single
action.
* **Bug Fixes**
* Ensured notifications marked as read are no longer shown as unread.
* **Tests**
* Introduced new tests to verify the functionality of marking all
notifications as read.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #13076** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Improved test isolation by using unique prompt names and randomized
user emails for each test run.
* Optimized test setup by initializing the testing database once
globally.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
disable on selfhosted
close AF-2715
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added an Apple iTunes app association meta tag to the document share
page for improved app integration, shown only in non-self-hosted
environments.
* **Tests**
* Introduced end-to-end tests to verify the conditional presence of the
Apple iTunes app meta tag based on the deployment environment.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #13065** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
#### PR Dependency Tree
* **PR #13074** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated deployment configuration to remove use of the jemalloc
library.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #13064** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Increased the number of test shards for end-to-end tests from 8 to 10,
improving test parallelization and potentially reducing testing time.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close AF-2714
#### PR Dependency Tree
* **PR #13063** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added the ability to list all replies for a specific comment.
* **Bug Fixes**
* Improved notification delivery for comment replies, ensuring all
relevant users (comment author, document owner, and all repliers) are
notified appropriately.
* **Tests**
* Added and updated tests to verify correct notification behavior and
the new reply listing functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #13037** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Improved test reliability by generating unique user accounts and
prompt names for each test run.
* Updated test setup to streamline database initialization.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close AF-2711
#### PR Dependency Tree
* **PR #13033** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Added an end-to-end test to verify that mention notifications are
correctly sent when replying to a comment authored by the document
owner.
* **Refactor**
* Improved the notification logic to streamline how mention and owner
notifications are sent, reducing redundancy and ensuring correct
recipients.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #13029** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added an option in user settings to enable or disable email
notifications for comments on your documents.
* Updated the user interface to include a toggle for comment email
notifications.
* Extended GraphQL queries and schema to support the new comment email
notification setting.
* **Localization**
* Added new English translations for comment email notification
settings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close AF-2708
#### PR Dependency Tree
* **PR #13018** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added notifications for users when their comment receives a reply,
marking them with a mention.
* **Tests**
* Introduced an end-to-end test to verify that replying to a comment
sends a mention notification to the original comment author.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
reduce dnsmasq service
#### PR Dependency Tree
* **PR #13012** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Updated instructions to use the domain `affine.localhost` instead of
`dev.affine.fail`.
* Simplified setup steps by removing references to DNS service
configuration and custom DNS server entries.
* **Chores**
* Removed DNS server configuration and related files from the
development environment setup.
* Updated example descriptions to reflect the new domain.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-236
#### PR Dependency Tree
* **PR #13001** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated system configuration to preload jemalloc for improved memory
management in the Node.js application.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced comprehensive management for workspace blobs, including
creating, updating, soft and permanent deletion, and retrieval of blobs.
- Added the ability to list all blobs, list deleted blobs, and calculate
the total size of blobs within a workspace.
- **Tests**
- Added extensive automated tests to validate blob creation, updating,
deletion, retrieval, and aggregation functionalities.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #12894** 👈
* **PR #12897**
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
#### PR Dependency Tree
* **PR #12994** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated the timestamp format in canary version strings to remove the
hyphen between hour and minute components.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
https://github.com/toeverything/AFFiNE/pull/12989#discussion_r2180167232
#### PR Dependency Tree
* **PR #12993** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* The "Comment" feature is now recognized as a server feature and is
enabled by default.
* The server configuration and API schema have been updated to include
the "Comment" feature.
* **Tests**
* Added a new end-to-end test to verify that the "Comment" feature is
enabled by default.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #12988** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Removed the configuration option that allowed guests to access demo
workspaces. This affects related settings in the system configuration
and admin interface.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close AF-2705
#### PR Dependency Tree
* **PR #12984** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* The Apple OAuth provider is now available only for clients version
0.22.0 or higher.
* Client version detection has been improved by extracting version
information from request headers.
* **Bug Fixes**
* Ensured that the Apple OAuth provider is hidden for clients below
version 0.22.0.
* **Tests**
* Added comprehensive end-to-end and utility tests for OAuth provider
selection and client version extraction.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #12924** 👈
* **PR #12925**
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced comment and comment mention notifications, including email
notifications when users are mentioned or receive comments on documents.
* Added new email templates for comment and comment mention
notifications.
* Users can now control whether they receive comment-related emails via
a new user setting.
* **Bug Fixes**
* None.
* **Documentation**
* Updated GraphQL schema documentation to reflect new notification types
and user settings.
* **Refactor**
* Streamlined and enhanced test coverage for notification and user
settings, including comment notifications.
* **Chores**
* Improved test setup and snapshot coverage for user settings and
notifications.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-227
close CLOUD-230
#### PR Dependency Tree
* **PR #12761** 👈
* **PR #12924**
* **PR #12925**
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced a comprehensive commenting system, enabling users to
create, update, resolve, and delete comments and replies on documents.
* Added support for uploading attachments to comments, with clear error
messaging if size limits are exceeded.
* Implemented role-based permissions for comment actions, including a
new "Commenter" role.
* Enabled paginated listing and change tracking of comments and replies
via GraphQL queries.
* Provided full localization and error handling for comment-related
actions.
* **Bug Fixes**
* Improved uniqueness handling when fetching user data for comments and
replies.
* **Documentation**
* Extended GraphQL schema and frontend localization to document and
support new comment features.
* **Tests**
* Added extensive backend test suites covering all comment and reply
functionalities, permissions, and attachment uploads.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-230
#### PR Dependency Tree
* **PR #12911** 👈
* **PR #12761**
* **PR #12924**
* **PR #12925**
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for uploading and retrieving comment attachments in
workspace documents via a new API endpoint.
* Introduced a service for managing comment attachments, including
storage, retrieval, deletion, and URL generation.
* Implemented localized error messages and improved error handling for
missing comment attachments.
* **Bug Fixes**
* Improved error feedback when comment attachments are not found.
* **Tests**
* Added comprehensive tests for comment attachment storage, retrieval,
deletion, API endpoint behavior, and permission checks.
* **Documentation**
* Updated GraphQL schema and localization files to include new error
types for comment attachments.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close AI-186
#### PR Dependency Tree
* **PR #12811** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added a new tool enabling users to read document content and metadata
within a workspace, with enforced access control.
- **Improvements**
- Updated tool interfaces and outputs to support the document reading
functionality seamlessly.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced support for comment attachments, allowing users to add,
view, and manage attachments linked to comments within documents.
* **Tests**
* Added comprehensive tests to ensure correct behavior for adding,
updating, deleting, listing, and retrieving comment attachments.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #12909** 👈
* **PR #12911**
* **PR #12761**
* **PR #12924**
* **PR #12925**
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
close CLOUD-226
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced support for comments and replies within workspaces and
documents, enabling users to create, update, delete, and resolve
comments, as well as manage threaded replies.
- **Bug Fixes**
- Added user-friendly error messages and handling for situations where
comments or replies are not found.
- **Tests**
- Added comprehensive tests to ensure correct behavior of comment and
reply operations.
- **Localization**
- Added English translations for new comment and reply error messages.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #12760** 👈
* **PR #12909**
* **PR #12911**
* **PR #12761**
* **PR #12924**
* **PR #12925**
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
close CLOUD-233
#### PR Dependency Tree
* **PR #12950** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for configuring multiple server hosts across backend and
frontend settings.
* Enhanced deployment and Helm chart configuration to allow specifying
multiple ingress hosts.
* Updated admin and configuration interfaces to display and manage
multiple server hosts.
* **Improvements**
* Improved URL generation, OAuth, and worker service logic to
dynamically handle requests from multiple hosts.
* Enhanced captcha verification to support multiple allowed hostnames.
* Updated frontend logic for platform-specific server base URLs and
allowed origins, including Apple app domains.
* Expanded test coverage for multi-host scenarios.
* **Bug Fixes**
* Corrected backend logic to consistently use dynamic base URLs and
origins based on request host context.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-232
#### PR Dependency Tree
* **PR #12913** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- The visibility status of documents now accurately reflects their
public status instead of always showing as private.
- **Tests**
- Added an end-to-end test to verify correct handling of the public
attribute for documents without snapshots.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-231
#### PR Dependency Tree
* **PR #12893** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Refactor**
- Updated background processing for document indexing and deletion to
use a job queue system instead of event-based triggers.
- **Bug Fixes**
- Improved reliability of embedding updates and deletions by ensuring
tasks are properly queued and processed.
- **Tests**
- Adjusted tests to verify that document operations correctly trigger
job queue actions.
No changes to user-facing features or interface.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close AI-220
#### PR Dependency Tree
* **PR #12866** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a new document search capability, allowing users to search
for documents by keyword within a workspace.
- Search results include document details such as title, highlights,
creation and update timestamps, and creator/updater information.
- Added support for limiting the number of search results returned.
- **Tests**
- Added comprehensive end-to-end and snapshot tests to ensure accuracy
and access control for the new search functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close AI-185
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced a keyword-based document search tool, allowing users to
search for relevant documents within their workspace using keywords.
- Search results include document titles, summaries, and direct links,
enhancing document discovery and navigation.
- **Bug Fixes**
- None.
- **Tests**
- Added new tests to verify document search by IDs and by keywords,
ensuring accurate and reliable search functionality.
- **Documentation**
- None.
- **Chores**
- Updated configuration file organization for improved clarity; no
changes to functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #12867**
* **PR #12863**
* **PR #12837** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
#### PR Dependency Tree
* **PR #12867**
* **PR #12863** 👈
* **PR #12837**
* **PR #12866**
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
#### PR Dependency Tree
* **PR #12867** 👈
* **PR #12863**
* **PR #12837**
* **PR #12866**
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Date fields in search results are now returned as JavaScript Date
objects instead of strings or numeric timestamps.
- **Bug Fixes**
- Improved consistency and correctness of date field types across
different search providers.
- **Tests**
- Added tests to verify that date fields are correctly returned as Date
objects.
- Updated existing test snapshots to reflect the new Date object format
for date fields.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-229
#### PR Dependency Tree
* **PR #12856** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Refactor**
- Improved the internal handling of document updates and snapshots to
enhance reliability and maintainability. No changes to user-facing
features or functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #12871** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Refactor**
- Updated internal logging mechanism to use a standard logger, resulting
in a simplified and more consistent logging approach. No changes to
user-facing features or functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #12871** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
close AI-218
#### PR Dependency Tree
* **PR #12861** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added a "Recently Updated Documents" feature, allowing users to view a
paginated list of the most recently updated documents within a
workspace.
- Document metadata now includes a "title" field for easier
identification.
- **Tests**
- Introduced new end-to-end tests to verify the recently updated
documents query and its pagination behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #12857** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved error handling in document embedding jobs to log and skip
errors instead of interrupting the process.
- Enhanced logging with clearer messages about embedding outcomes and
skipped operations.
- Added warnings when documents or summaries are missing during
embedding.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close AI-213
#### PR Dependency Tree
* **PR #12846** 👈
* **PR #12811**
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced support for AI-editable blocks in document parsing,
allowing blocks to include metadata for AI-based editing.
- Added rendering for todo list items with markdown checkbox syntax.
- Unsupported block types are now marked with placeholders in the parsed
output.
- **Tests**
- Added new test cases and snapshots to verify parsing behavior with
AI-editable content enabled.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close AI-190
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Introduced an endpoint to retrieve a document's markdown content and
title.
- Added backend support for parsing document snapshots directly into
markdown format.
- **Tests**
- Added comprehensive tests and snapshot files for markdown retrieval,
including success and error scenarios.
- Improved test coverage for content type validation and markdown
parsing utilities.
- **Documentation**
- Enhanced internal documentation through detailed test cases and
snapshot references for new markdown features.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#### PR Dependency Tree
* **PR #12812** 👈
* **PR #12846**
* **PR #12811**
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
#### PR Dependency Tree
* **PR #12822** 👈
This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added the ability to search for blob names by their IDs within a
workspace.
- **Tests**
- Introduced new test cases and snapshot tests to validate searching
blob names and reading filenames from document snapshots.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close#12692
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved error handling during migration rollbacks to better recognize and safely skip specific migration errors.
- Enhanced logging for migration rollback failures to provide clearer information without interrupting the process.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Chores**
- Updated the default setting for the indexer feature to be disabled by default.
- Added a sample environment variable for enabling the indexer in the example configuration file.
- Introduced a new environment variable for the indexer in the CI workflow configuration.
- **Tests**
- Adjusted test configurations to explicitly enable the indexer feature during test execution.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
closes#12701
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved error handling during migration rollbacks to prevent unnecessary errors when migrations were never applied.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Chores**
- Improved log messages to include job IDs when jobs are removed from the queue, enhancing traceability for users monitoring job activity.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-225
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added support for reading document blocks without requiring a workspace or root document snapshot.
- **Bug Fixes**
- Improved handling of large workspace snapshots by skipping them when they exceed 10MB.
- **Tests**
- Introduced new test cases to cover scenarios where root or workspace snapshots are absent.
- Expanded snapshot tests for document block reading.
- **Refactor**
- Updated several function signatures to make root and workspace snapshot parameters optional for greater flexibility.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Refactor**
- Adjusted logging verbosity to be more detailed in the 'canary' environment, providing debug-level logs, while maintaining info-level logs elsewhere.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Refactor**
- Improved visibility of job start and finish events by updating logging level, making these events more prominent in logs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Need to query the payload through job id for debugging
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Tests**
- Updated job metrics test to include an explicit job ID during execution.
- **Refactor**
- Enhanced job execution to support an optional job ID, improving job tracking and logging.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-223
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved error handling for search requests to prevent issues when error details are missing, ensuring clearer fallback messages for unknown errors.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved reliability of batch write operations to search providers, ensuring documents are correctly indexed and retrievable.
- **Tests**
- Added new test cases and snapshots to verify batch write functionality and confirm resolution of prior batch processing issues.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Chores**
- Removed detailed verbose logging for Elasticsearch requests to reduce log noise.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close AF-2658
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Tests**
- Added new test cases and snapshots to enhance coverage for search results involving empty or missing fields like summary, title, and ref_doc_id.
- Verified consistent handling of empty string values and absence of fields across different search providers.
- **Bug Fixes**
- Improved handling of empty string values for specific fields by converting them to null to ensure consistent search result formatting.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added '@affine/admin' to the list of selectable packages for development commands.
- **Enhancements**
- Improved package selection prompt by displaying up to 10 choices at a time for easier navigation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Implemented an exponential backoff retry strategy for job queues, resulting in progressively longer wait times between retry attempts.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved system stability by ensuring that indexing jobs do not run when the indexer feature is disabled.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
enable indexer using `compose.indexer.yml` on self-host:
```bash
docker compose -f compose.yml -f compose.indexer.yml up
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Chores**
- Updated the build process in the development container to include building the reader package.
- Added and configured a Manticore Search indexer service in the development container.
- Removed the indexer service and related environment variables from the self-hosted Docker Compose setup and environment example file.
- **Documentation**
- Improved documentation formatting for better readability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Chores**
- Updated workflow to revert changes to a specific i18n file after running the i18n build step during automated checks. No impact on user-facing features.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-221
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Bug Fixes**
- Resolved an issue that could cause the document listing process to enter an infinite loop under certain conditions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Tests**
- Refactored test setup for the database-backed document reader to improve modularity and clarity.
- Simplified database initialization and cleanup in tests.
- Updated mocks and assertions to align with the new test structure.
- Maintained existing test coverage and logic.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-220
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Introduced a new service to handle indexing-related events and scheduled tasks, improving the management of document and workspace indexing.
- Added support for configuring the indexer feature via the AFFINE_INDEXER_ENABLED environment variable.
- **Bug Fixes**
- Ensured that indexing and deletion jobs are only enqueued when the indexer feature is enabled.
- **Tests**
- Added comprehensive tests for the new indexing event service, covering various configuration scenarios.
- Removed obsolete test related to auto-indexing scheduling.
- **Chores**
- Updated configuration descriptions and mappings to improve clarity and environment variable support.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added a configurable batch size setting for automatic workspace indexing. Users can now adjust how many workspaces are indexed per batch, with a default value of 10 and support for values between 1 and 1000.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
CLOUD-214
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Improved search functionality to support stemming, allowing searches for variations of words (e.g., "window", "windows", "design") to return relevant results.
- **Tests**
- Added new tests to verify that search results correctly highlight and match stemmed word variations in document titles.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-207
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Introduced automated periodic indexing of workspaces with a new job type and a scheduled cron job running every 30 seconds.
- Added a unique sequential identifier (`sid`) and an "indexed" flag to workspaces to track indexing status.
- **Improvements**
- Enhanced workspace indexing to handle missing workspaces and snapshots distinctly and selectively index documents.
- Added ability to query workspaces after a given identifier with result limits.
- **Bug Fixes**
- Improved error handling and logging during workspace indexing operations.
- **Tests**
- Expanded test coverage for workspace indexing and auto-indexing, including scheduling and edge cases.
- **Chores**
- Updated data models and schema to support new workspace fields and indexing features.
- Enhanced mock data utilities to allow custom timestamps.
- Improved type safety and flexibility in document snapshot retrieval.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-208
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Enhanced document access control with batch permission checks, enabling efficient filtering of documents based on user roles and permissions.
- Added detailed document-level role and permission management for workspace users.
- **Bug Fixes**
- Improved accuracy in filtering search results to only display documents users have permission to read.
- **Tests**
- Added comprehensive tests for document-level permission filtering and search result accuracy.
- Introduced new mock utilities to support permission-related test scenarios.
- **Refactor**
- Simplified and optimized permission logic for determining user roles and document access.
- **Documentation**
- Updated type definitions for improved clarity in permission handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Chores**
- Added a `.gitignore` file to prevent the `.env` file in the self-hosted Docker directory from being tracked by Git.
- **New Features**
- Introduced a new environment variable for the search endpoint in the Docker Compose configuration for improved service connectivity.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-217
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Introduced a dedicated test job for Elasticsearch, running only Elasticsearch-specific tests during CI.
- **Chores**
- Enhanced server test workflows with explicit setup steps and automated coverage uploads.
- Improved test suite structure for Elasticsearch provider to enable conditional and asynchronous test execution based on environment variables.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Chores**
- Standardized the naming of the search service to "indexer" in automated test workflows.
- Ensured the "indexer" service is available in additional test jobs for improved consistency across workflows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-137
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Introduced advanced workspace-scoped search and aggregation capabilities with support for complex queries, highlights, and pagination.
- Added pluggable search providers: Elasticsearch and Manticoresearch.
- New GraphQL queries, schema types, and resolver support for search and aggregation.
- Enhanced configuration options for search providers in self-hosted and cloud deployments.
- Added Docker Compose services and environment variables for Elasticsearch and Manticoresearch.
- Integrated indexer service into deployment and CI workflows.
- **Bug Fixes**
- Improved error handling with new user-friendly error messages for search provider and indexer issues.
- **Documentation**
- Updated configuration examples and environment variable references for indexer and search providers.
- **Tests**
- Added extensive end-to-end and provider-specific tests covering indexing, searching, aggregation, deletion, and error cases.
- Included snapshot tests and test fixtures for search providers.
- **Chores**
- Updated deployment scripts, Helm charts, and Kubernetes manifests to include indexer-related environment variables and secrets.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Bug Fixes**
- Corrected CPU resource allocation for the documentation service to ensure accurate deployment configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Chores**
- Updated a backend dependency to the latest version to ensure continued compatibility and improvements. No user-facing changes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fengmk2
