fix AI-127
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Added automated event handling for workspace updates and document embedding, streamlining document embedding workflows.
- Introduced detection and queuing of documents needing embedding, excluding ignored documents.
- **Improvements**
- Enhanced performance of embedding-related searches by filtering results at the database level.
- Increased concurrency for embedding job processing to improve throughput.
- **Bug Fixes**
- Improved error handling and fallback for missing document titles during embedding.
- Added safeguards to skip invalid embedding jobs based on document identifiers.
- **Tests**
- Expanded test coverage for document embedding and ignored document filtering.
- Updated end-to-end tests to use dynamic content for improved reliability.
- Added synchronization waits in document creation utilities to improve test stability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
close CLOUD-207
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
- **New Features**
- Introduced automated periodic indexing of workspaces with a new job type and a scheduled cron job running every 30 seconds.
- Added a unique sequential identifier (`sid`) and an "indexed" flag to workspaces to track indexing status.
- **Improvements**
- Enhanced workspace indexing to handle missing workspaces and snapshots distinctly and selectively index documents.
- Added ability to query workspaces after a given identifier with result limits.
- **Bug Fixes**
- Improved error handling and logging during workspace indexing operations.
- **Tests**
- Expanded test coverage for workspace indexing and auto-indexing, including scheduling and edge cases.
- **Chores**
- Updated data models and schema to support new workspace fields and indexing features.
- Enhanced mock data utilities to allow custom timestamps.
- Improved type safety and flexibility in document snapshot retrieval.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
darkskygit