Commit Graph

12 Commits

Author SHA1 Message Date
DarkSky
f47ee2bc8a feat(server): improve indexer (#14698)
fix #13862 


#### PR Dependency Tree


* **PR #14698** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Enhanced search support for Chinese, Japanese, and Korean languages
with improved text segmentation and character matching.
* Added index management capabilities with table recreation
functionality.

* **Bug Fixes**
* Improved search accuracy for non-Latin scripts through updated
morphology and n-gram configuration.

* **Chores**
  * Added database migration for search index optimization.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-03-22 02:50:59 +08:00
DarkSky
7c24b2521a feat: reduce backend (#14251)
#### PR Dependency Tree


* **PR #14251** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Current user profile now exposes access tokens, revealed tokens, and
detailed calendar accounts/subscriptions.
* Workspace now exposes permissions, calendars, calendar events, and a
workspace-scoped blob upload part URL.
  * New document-update mutation for applying doc updates.

* **API Changes**
  * validateAppConfig is now a query (mutation deprecated).
* Several legacy top-level calendar/blob endpoints deprecated in favor
of user/workspace fields.

* **Refactor**
* Calendar, blob-upload and access-token surfaces reorganized to use
user/workspace-centric fields.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-01-14 00:01:07 +08:00
DarkSky
b331a08744 feat: native update merge (#14250)
#### PR Dependency Tree


* **PR #14250** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Backend Optimization**
  * Faster document retrieval via a native binary fetch path.
* Native-accelerated merging of document updates for improved
performance and consistency.
* **Indexing & Reliability**
* Indexing now only proceeds on valid parse results, with clearer
warnings and richer metadata on failures.
* More consistent sync behavior and enhanced diagnostic logging for
indexing operations.
* **Tests**
  * Expanded tests to cover native binary retrieval error handling.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-01-13 22:03:55 +08:00
DarkSky
844b9d9592 feat(server): impl native reader for server (#14100) 2025-12-14 00:28:43 +08:00
DarkSky
b9c4d7230e feat(server): update session after doc deletion (#13028)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Sessions associated with a deleted document are now automatically
updated to remove the document reference.

* **Improvements**
* Enhanced session management to better handle documents that have been
deleted.

No visible changes to the user interface; these updates improve backend
handling of session and document relationships.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-07-04 07:54:19 +00:00
fengmk2
862a50c982 fix(server): use job queue instead event on doc indexing changes (#12893)
close CLOUD-231



#### PR Dependency Tree


* **PR #12893** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Refactor**
- Updated background processing for document indexing and deletion to
use a job queue system instead of event-based triggers.
- **Bug Fixes**
- Improved reliability of embedding updates and deletions by ensuring
tasks are properly queued and processed.
- **Tests**
- Adjusted tests to verify that document operations correctly trigger
job queue actions.
  
No changes to user-facing features or interface.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-06-23 06:56:04 +00:00
fengmk2
62d74de810 feat(server): search docs by keywork from indexer (#12863)
#### PR Dependency Tree


* **PR #12867**
  * **PR #12863** 👈
    * **PR #12837**
    * **PR #12866**

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
2025-06-20 09:48:30 +00:00
fengmk2
c0c813edfd feat(server): search blob names from indexer (#12822)
#### PR Dependency Tree


* **PR #12822** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Added the ability to search for blob names by their IDs within a
workspace.
- **Tests**
- Introduced new test cases and snapshot tests to validate searching
blob names and reading filenames from document snapshots.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-06-16 16:34:23 +08:00
fengmk2
39cb1afedb fix(server): limit rootDoc snapshot size (#12625)
close CLOUD-225

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **New Features**
  - Added support for reading document blocks without requiring a workspace or root document snapshot.
- **Bug Fixes**
  - Improved handling of large workspace snapshots by skipping them when they exceed 10MB.
- **Tests**
  - Introduced new test cases to cover scenarios where root or workspace snapshots are absent.
  - Expanded snapshot tests for document block reading.
- **Refactor**
  - Updated several function signatures to make root and workspace snapshot parameters optional for greater flexibility.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-29 05:32:31 +00:00
fengmk2
aea45f451d fix(server): avoid infinite loop in manticoresearch (#12460)
close CLOUD-221

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **Bug Fixes**
  - Resolved an issue that could cause the document listing process to enter an infinite loop under certain conditions.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-23 05:04:13 +00:00
fengmk2
afa984da54 feat(server): handle workspace doc updates (#11937) 2025-05-14 14:52:41 +00:00
fengmk2
a1bcf77447 feat(server): add cloud indexer with Elasticsearch and Manticoresearch providers (#11835)
close CLOUD-137

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Introduced advanced workspace-scoped search and aggregation capabilities with support for complex queries, highlights, and pagination.
  - Added pluggable search providers: Elasticsearch and Manticoresearch.
  - New GraphQL queries, schema types, and resolver support for search and aggregation.
  - Enhanced configuration options for search providers in self-hosted and cloud deployments.
  - Added Docker Compose services and environment variables for Elasticsearch and Manticoresearch.
  - Integrated indexer service into deployment and CI workflows.

- **Bug Fixes**
  - Improved error handling with new user-friendly error messages for search provider and indexer issues.

- **Documentation**
  - Updated configuration examples and environment variable references for indexer and search providers.

- **Tests**
  - Added extensive end-to-end and provider-specific tests covering indexing, searching, aggregation, deletion, and error cases.
  - Included snapshot tests and test fixtures for search providers.

- **Chores**
  - Updated deployment scripts, Helm charts, and Kubernetes manifests to include indexer-related environment variables and secrets.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-14 14:52:41 +00:00