close CLOUD-220
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Introduced a new service to handle indexing-related events and scheduled tasks, improving the management of document and workspace indexing.
- Added support for configuring the indexer feature via the AFFINE_INDEXER_ENABLED environment variable.
- **Bug Fixes**
- Ensured that indexing and deletion jobs are only enqueued when the indexer feature is enabled.
- **Tests**
- Added comprehensive tests for the new indexing event service, covering various configuration scenarios.
- Removed obsolete test related to auto-indexing scheduling.
- **Chores**
- Updated configuration descriptions and mappings to improve clarity and environment variable support.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added a configurable batch size setting for automatic workspace indexing. Users can now adjust how many workspaces are indexed per batch, with a default value of 10 and support for values between 1 and 1000.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fix AI-127
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added automated event handling for workspace updates and document embedding, streamlining document embedding workflows.
- Introduced detection and queuing of documents needing embedding, excluding ignored documents.
- **Improvements**
- Enhanced performance of embedding-related searches by filtering results at the database level.
- Increased concurrency for embedding job processing to improve throughput.
- **Bug Fixes**
- Improved error handling and fallback for missing document titles during embedding.
- Added safeguards to skip invalid embedding jobs based on document identifiers.
- **Tests**
- Expanded test coverage for document embedding and ignored document filtering.
- Updated end-to-end tests to use dynamic content for improved reliability.
- Added synchronization waits in document creation utilities to improve test stability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Chores**
- Added a `.gitignore` file to prevent the `.env` file in the self-hosted Docker directory from being tracked by Git.
- **New Features**
- Introduced a new environment variable for the search endpoint in the Docker Compose configuration for improved service connectivity.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **Chores**
- Updated the Docker Compose configuration to use a different image for the Postgres service.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-137
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Introduced advanced workspace-scoped search and aggregation capabilities with support for complex queries, highlights, and pagination.
- Added pluggable search providers: Elasticsearch and Manticoresearch.
- New GraphQL queries, schema types, and resolver support for search and aggregation.
- Enhanced configuration options for search providers in self-hosted and cloud deployments.
- Added Docker Compose services and environment variables for Elasticsearch and Manticoresearch.
- Integrated indexer service into deployment and CI workflows.
- **Bug Fixes**
- Improved error handling with new user-friendly error messages for search provider and indexer issues.
- **Documentation**
- Updated configuration examples and environment variable references for indexer and search providers.
- **Tests**
- Added extensive end-to-end and provider-specific tests covering indexing, searching, aggregation, deletion, and error cases.
- Included snapshot tests and test fixtures for search providers.
- **Chores**
- Updated deployment scripts, Helm charts, and Kubernetes manifests to include indexer-related environment variables and secrets.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
