feat(server): improve indexer perf (#15512)

#### PR Dependency Tree


* **PR #15512** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Search now supports generation-based indexing with embedded and remote
providers.
* Added automatic search reconciliation and improved handling of
document, workspace, and permission changes.
* Added clearer search status errors for unavailable, syncing, unready,
or failed indexes.
* Added Manticore Search end-to-end support and provider-specific search
behavior.

* **Improvements**
* Search and aggregate pagination now report returned results and
continuation status more accurately.
* Improved permission filtering to prevent inaccessible documents from
appearing in results.
  * Admin provider selection now consistently enables indexing.

* **Documentation**
* Clarified search pagination, aggregation counts, provider
configuration, and end-to-end setup.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

packages/backend/server/e2e/manticore-provider/README.md

@@ -0,0 +1,45 @@
+# Manticore Search provider E2E
+
+This fixture validates the shared immutable projection contract against a real
+Manticore Search RT table. Aggregate queries remain unsupported and are validated
+through the typed fallback boundary instead.
+
+Start the external dependency:
+
+```bash
+docker compose -f packages/backend/server/e2e/manticore-provider/compose.yml up -d --wait
+```
+
+Use a disposable PostgreSQL database and run the provider-gated E2E:
+
+```bash
+DATABASE_URL=postgresql://ds:ds@localhost:55433/affine_rfc6_manticore_e2e \
+yarn workspace @affine/server prisma migrate deploy
+```
+
+Configure `packages/backend/server/config.json` with:
+
+```json
+{
+  "indexer": {
+    "enabled": true,
+    "provider": {
+      "type": "manticoresearch",
+      "endpoint": "http://127.0.0.1:9308"
+    }
+  }
+}
+```
+
+Then run:
+
+```bash
+DATABASE_URL=postgresql://ds:ds@localhost:55433/affine_rfc6_manticore_e2e \
+yarn af server e2e src/__tests__/e2e/indexer/manticore-provider.spec.ts
+```
+
+Stop only this disposable dependency when finished:
+
+```bash
+docker compose -f packages/backend/server/e2e/manticore-provider/compose.yml down -v
+```

packages/backend/server/e2e/manticore-provider/compose.yml

@@ -0,0 +1,33 @@
+services:
+  postgres:
+    image: pgvector/pgvector:pg16
+    environment:
+      POSTGRES_DB: affine_rfc6_manticore_e2e
+      POSTGRES_USER: ds
+      POSTGRES_PASSWORD: ds
+    ports:
+      - '55433:5432'
+    healthcheck:
+      test: ['CMD-SHELL', 'pg_isready -U ds -d affine_rfc6_manticore_e2e']
+      interval: 2s
+      timeout: 2s
+      retries: 30
+      start_period: 5s
+
+  manticore:
+    image: manticoresearch/manticore:29.0.2
+    environment:
+      searchd.listen: 'http:9308'
+      searchd.network_timeout: '30'
+    ports:
+      - '9308:9308'
+    healthcheck:
+      test:
+        [
+          'CMD-SHELL',
+          "wget -qO- --post-data='SHOW TABLES' 'http://127.0.0.1:9308/sql?mode=raw' >/dev/null",
+        ]
+      interval: 2s
+      timeout: 2s
+      retries: 30
+      start_period: 5s