feat(server): increase embedding jobs concurrency & handle empty content after trim (#12574)

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **Improvements**
  - Increased the default concurrency for background tasks, enhancing processing efficiency.
  - Improved handling of empty or unsupported documents to ensure consistent processing.
  - Optimized document filtering to exclude certain documents from processing, improving performance.

- **Bug Fixes**
  - Enhanced detection of empty document summaries, reducing errors during processing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
darkskygit
2025-05-27 14:28:34 +00:00
parent 7eb6b268a6
commit 9220b973c7
6 changed files with 54 additions and 23 deletions

View File

@@ -52,14 +52,14 @@
},
"queues.copilot": {
"type": "object",
"description": "The config for copilot job queue\n@default {\"concurrency\":5}",
"description": "The config for copilot job queue\n@default {\"concurrency\":10}",
"properties": {
"concurrency": {
"type": "number"
}
},
"default": {
"concurrency": 5
"concurrency": 10
}
},
"queues.doc": {