Commit Graph

7 Commits

Author SHA1 Message Date
DarkSky cc1d5b497a feat(server): cleanup trashed doc's embedding (#13201)
fix AI-359

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added automated cleanup of embeddings for documents deleted or trashed
from workspaces.
* Introduced a new job to schedule and perform this cleanup per
workspace daily and on demand.
  * Added new GraphQL mutation to manually trigger the cleanup process.
* Added the ability to list workspaces with flexible filtering and
selection options.

* **Improvements**
* Enhanced document status handling to more accurately reflect embedding
presence.
* Refined internal methods for managing and checking document
embeddings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-07-15 12:00:33 +00:00
DarkSky 8e374f5517 feat(server): skip embedding for deprecated doc ids & empty docs (#13211)
fix AI-367

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved document filtering to exclude settings documents and empty
blobs from embedding and status calculations.
* Enhanced embedding jobs to skip processing deprecated documents if a
newer version exists, ensuring only up-to-date documents are embedded.
* **New Features**
* Added a mutation to trigger the cron job for generating missing
titles.
* **Tests**
* Added test to verify exclusion of documents with empty content from
embedding.
* Updated embedding-related tests to toggle embedding state during
attachment upload under simulated network conditions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-07-15 08:50:48 +00:00
fengmk2 862a50c982 fix(server): use job queue instead event on doc indexing changes (#12893)
close CLOUD-231



#### PR Dependency Tree


* **PR #12893** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Refactor**
- Updated background processing for document indexing and deletion to
use a job queue system instead of event-based triggers.
- **Bug Fixes**
- Improved reliability of embedding updates and deletions by ensuring
tasks are properly queued and processed.
- **Tests**
- Adjusted tests to verify that document operations correctly trigger
job queue actions.
  
No changes to user-facing features or interface.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-06-23 06:56:04 +00:00
fengmk2 5a87d3d9f6 fix(server): don't use global AFFiNELogger (#12871)
![image](https://github.com/user-attachments/assets/0e5d313f-e72b-436b-9f13-9f0a1cff8685)

#### PR Dependency Tree


* **PR #12871** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Refactor**
- Updated internal logging mechanism to use a standard logger, resulting
in a simplified and more consistent logging approach. No changes to
user-facing features or functionality.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->


#### PR Dependency Tree


* **PR #12871** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)
2025-06-20 05:02:42 +00:00
fengmk2 5623d808bf fix(server): set empty embedding on empty doc (#12857)
#### PR Dependency Tree


* **PR #12857** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Bug Fixes**
- Improved error handling in document embedding jobs to log and skip
errors instead of interrupting the process.
- Enhanced logging with clearer messages about embedding outcomes and
skipped operations.
- Added warnings when documents or summaries are missing during
embedding.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-06-20 10:30:53 +08:00
DarkSky cdaaa52845 feat(server): embedding search tool (#12810) 2025-06-17 09:22:56 +08:00
DarkSky e98f035f97 feat(server): split embedding client (#12809) 2025-06-13 12:37:05 +08:00