feat(server): auto index all workspaces to indexer (#12205)

close CLOUD-207

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Introduced automated periodic indexing of workspaces with a new job type and a scheduled cron job running every 30 seconds.
  - Added a unique sequential identifier (`sid`) and an "indexed" flag to workspaces to track indexing status.
- **Improvements**
  - Enhanced workspace indexing to handle missing workspaces and snapshots distinctly and selectively index documents.
  - Added ability to query workspaces after a given identifier with result limits.
- **Bug Fixes**
  - Improved error handling and logging during workspace indexing operations.
- **Tests**
  - Expanded test coverage for workspace indexing and auto-indexing, including scheduling and edge cases.
- **Chores**
  - Updated data models and schema to support new workspace fields and indexing features.
  - Enhanced mock data utilities to allow custom timestamps.
  - Improved type safety and flexibility in document snapshot retrieval.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

packages/backend/server/schema.prisma

@@ -106,6 +106,8 @@ model VerificationToken {
 }
 
 model Workspace {
+  // NOTE: manually set this column type to identity in migration file
+  sid                Int      @unique @default(autoincrement())
   id                 String   @id @default(uuid()) @db.VarChar
   public             Boolean
   createdAt          DateTime @default(now()) @map("created_at") @db.Timestamptz(3)
@@ -115,6 +117,7 @@ model Workspace {
   enableDocEmbedding Boolean  @default(true) @map("enable_doc_embedding")
   name               String?  @db.VarChar
   avatarKey          String?  @map("avatar_key") @db.VarChar
+  indexed            Boolean  @default(false)
 
   features       WorkspaceFeature[]
   docs           WorkspaceDoc[]