feat(server): improve indexer perf (#15512)

#### PR Dependency Tree


* **PR #15512** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Search now supports generation-based indexing with embedded and remote
providers.
* Added automatic search reconciliation and improved handling of
document, workspace, and permission changes.
* Added clearer search status errors for unavailable, syncing, unready,
or failed indexes.
* Added Manticore Search end-to-end support and provider-specific search
behavior.

* **Improvements**
* Search and aggregate pagination now report returned results and
continuation status more accurately.
* Improved permission filtering to prevent inaccessible documents from
appearing in results.
  * Admin provider selection now consistently enables indexing.

* **Documentation**
* Clarified search pagination, aggregation counts, provider
configuration, and end-to-end setup.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

.docker/dev/.env.example

@@ -10,6 +10,3 @@ DB_DATABASE_NAME=affine
 # enable for arm64, e.g.: macOS M1+
 # ELASTIC_VERSION_ARM64=-arm64
 # ELASTIC_PLATFORM=linux/arm64
-
-# manticoresearch
-MANTICORE_VERSION=10.1.0

.docker/dev/compose.yml.example

@@ -32,22 +32,6 @@ services:
     volumes:
       - mailpit_data:/data
 
-  # https://manual.manticoresearch.com/Starting_the_server/Docker
-  manticoresearch:
-    image: manticoresearch/manticore:${MANTICORE_VERSION:-10.1.0}
-    ports:
-      - 9308:9308
-    ulimits:
-      nproc: 65535
-      nofile:
-        soft: 65535
-        hard: 65535
-      memlock:
-        soft: -1
-        hard: -1
-    volumes:
-      - manticoresearch_data:/var/lib/manticore
-  
   # elasticsearch:
   #   image: docker.elastic.co/elasticsearch/elasticsearch:${ELASTIC_VERSION:-9.0.1}${ELASTIC_VERSION_ARM64}
   #   platform: ${ELASTIC_PLATFORM}
@@ -94,6 +78,5 @@ networks:
 
 volumes:
   postgres_data:
-  manticoresearch_data:
   mailpit_data:
   elasticsearch_data: