fix AI-10
fix AI-109
fix PD-2484
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added a method to check if a document requires embedding, improving embedding efficiency.
- Enhanced document embeddings with enriched metadata, including title, summary, creation/update dates, and author information.
- Introduced a new type for document fragments with extended metadata fields.
- **Improvements**
- Embedding logic now conditionally processes only documents needing updates.
- Embedding content now includes document metadata for more informative context.
- Expanded and improved test coverage for embedding scenarios and workspace behaviors.
- Event emission added for workspace embedding updates on client version mismatch.
- Job queueing enhanced with prioritization and explicit job IDs for better management.
- Job queue calls updated to include priority and context identifiers in a structured format.
- **Bug Fixes**
- Improved handling of ignored documents in embedding matches.
- Fixed incorrect document ID assignment in embedding job queueing.
- **Tests**
- Added and updated snapshot and behavioral tests for embedding and workspace document handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Users are now prevented from deleting their account if they own one or more team workspaces. A clear error message instructs users to transfer ownership or delete those workspaces first.
- Disabled (banned) users are explicitly prevented from signing in or re-registering.
- Added new error messages and translations to improve clarity around account deletion restrictions.
- **Bug Fixes**
- Disabled users are now explicitly handled to prevent sign-in attempts.
- **Tests**
- Introduced comprehensive end-to-end tests covering account deletion, banning, and re-registration scenarios.
- **Chores**
- Improved event handling for user deletion and subscription cancellation.
- Updated localization resources with new error messages.
- Renamed payment event handler class for clarity.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-220
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Introduced a new service to handle indexing-related events and scheduled tasks, improving the management of document and workspace indexing.
- Added support for configuring the indexer feature via the AFFINE_INDEXER_ENABLED environment variable.
- **Bug Fixes**
- Ensured that indexing and deletion jobs are only enqueued when the indexer feature is enabled.
- **Tests**
- Added comprehensive tests for the new indexing event service, covering various configuration scenarios.
- Removed obsolete test related to auto-indexing scheduling.
- **Chores**
- Updated configuration descriptions and mappings to improve clarity and environment variable support.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix AI-131
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Refactor**
- Updated database schema to consolidate unique constraints into composite primary keys for embedding-related data, improving consistency.
- Changed the relation in the Snapshot model to allow multiple embeddings.
- Improved filtering logic for documents and snapshots based on embedding existence.
- Reformatted SQL queries and schema attributes for improved readability; no changes to functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix AI-127
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added automated event handling for workspace updates and document embedding, streamlining document embedding workflows.
- Introduced detection and queuing of documents needing embedding, excluding ignored documents.
- **Improvements**
- Enhanced performance of embedding-related searches by filtering results at the database level.
- Increased concurrency for embedding job processing to improve throughput.
- **Bug Fixes**
- Improved error handling and fallback for missing document titles during embedding.
- Added safeguards to skip invalid embedding jobs based on document identifiers.
- **Tests**
- Expanded test coverage for document embedding and ignored document filtering.
- Updated end-to-end tests to use dynamic content for improved reliability.
- Added synchronization waits in document creation utilities to improve test stability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved the accuracy of workspace member overcapacity calculation.
- Corrected seat allocation handling for one-time license variants, ensuring proper event emission and bypassing unnecessary updates.
- **Refactor**
- Streamlined internal logic by removing an obsolete method related to seat count checks.
- **Tests**
- Removed a workspace member list pagination test to streamline end-to-end testing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-207
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Introduced automated periodic indexing of workspaces with a new job type and a scheduled cron job running every 30 seconds.
- Added a unique sequential identifier (`sid`) and an "indexed" flag to workspaces to track indexing status.
- **Improvements**
- Enhanced workspace indexing to handle missing workspaces and snapshots distinctly and selectively index documents.
- Added ability to query workspaces after a given identifier with result limits.
- **Bug Fixes**
- Improved error handling and logging during workspace indexing operations.
- **Tests**
- Expanded test coverage for workspace indexing and auto-indexing, including scheduling and edge cases.
- **Chores**
- Updated data models and schema to support new workspace fields and indexing features.
- Enhanced mock data utilities to allow custom timestamps.
- Improved type safety and flexibility in document snapshot retrieval.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-208
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Enhanced document access control with batch permission checks, enabling efficient filtering of documents based on user roles and permissions.
- Added detailed document-level role and permission management for workspace users.
- **Bug Fixes**
- Improved accuracy in filtering search results to only display documents users have permission to read.
- **Tests**
- Added comprehensive tests for document-level permission filtering and search result accuracy.
- Introduced new mock utilities to support permission-related test scenarios.
- **Refactor**
- Simplified and optimized permission logic for determining user roles and document access.
- **Documentation**
- Updated type definitions for improved clarity in permission handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved the accuracy of document matching by excluding ignored documents from search results.
- **Chores**
- Updated internal handling of ignored document lists for better consistency and reliability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix AI-20
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Enhanced file metadata with MIME type, blob ID, and file name across context and workspace, now visible in UI and API.
- Added workspace-level matching for files and documents with configurable thresholds and workspace scoping in search queries.
- Introduced a new error type and user-friendly messaging for global workspace context matching failures.
- **Bug Fixes**
- Improved consistent handling of file MIME types and nullable context IDs for accurate metadata.
- **Documentation**
- Updated GraphQL schema, queries, and mutations to include new metadata fields, optional parameters, and error types.
- **Style**
- Added new localization strings for global context matching error messages.
- **Tests**
- Extended test coverage with new and updated snapshot tests for metadata and matching logic.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added support for installing self-hosted team licenses via encrypted license files.
- Introduced a new "Onetime" license variant for self-hosted environments.
- Added a GraphQL mutation to upload and install license files.
- License details now display the license variant.
- **Bug Fixes**
- Improved error messages for license activation and expiration, including dynamic reasons.
- **Localization**
- Updated and improved license-related error messages for better clarity.
- **Tests**
- Added comprehensive end-to-end tests for license installation scenarios.
- **Chores**
- Enhanced environment variable handling and public key management for license verification.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix AI-20
fix AI-77
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Enhanced relevance-based re-ranking for embedding results, improving the accuracy of content suggestions.
- Added prioritization for workspace content that matches specific document IDs in search results.
- Introduced a new scoped threshold parameter to refine workspace document matching.
- **Improvements**
- Increased default similarity threshold for file chunk matching, resulting in more precise matches.
- Doubled candidate retrieval for file and workspace chunk matching to improve result quality.
- Updated sorting to prioritize context-relevant documents in workspace matches.
- Explicitly included original input content in re-ranking calls for better relevance assessment.
- **Bug Fixes**
- Adjusted re-ranking logic to return only highly relevant results based on confidence scores.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added paginated document listing for workspaces, allowing users to browse documents with pagination controls.
- Enhanced document details to display creation and update timestamps, as well as information about the creator and last updater.
- **Bug Fixes**
- Updated deprecation notice for workspace document metadata fields to guide users to the latest recommended field.
- **Tests**
- Added new tests to verify document info retrieval and pagination functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Workspace quota information now includes a new metric: the count of members exceeding seat capacity ("overcapacityMemberCount"), available in both numeric and human-readable formats.
- The workspace quota display and related queries have been updated to show this new overcapacity member count alongside existing quota metrics.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Ignored documents in workspace embedding now display additional metadata, including document title, creation and update timestamps, and the names and avatars of users who created or updated the document.
- **Enhancements**
- The list of ignored documents provides richer information for easier identification and management within the workspace.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Expanded file chunk matching to include both context and workspace file embeddings, providing broader and more relevant search results.
- **Improvements**
- Enhanced result ranking by introducing a re-ranking step for combined embedding matches, improving the relevance of returned file chunks.
- Adjusted file count reporting to reflect the total number of workspace files instead of ignored documents for more accurate workspace file statistics.
- Renamed and streamlined workspace file management methods for clearer and more consistent API usage.
- **Bug Fixes**
- Prevented embedding similarity queries when embedding is disabled for a workspace, improving system behavior consistency.
- **Tests**
- Added comprehensive tests to verify workspace embedding management, including enabling, matching, and disabling embedding functionality.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
darkskygit