feat(server): increase embedding jobs concurrency & handle empty content after trim (#12574)

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **Improvements**
  - Increased the default concurrency for background tasks, enhancing processing efficiency.
  - Improved handling of empty or unsupported documents to ensure consistent processing.
  - Optimized document filtering to exclude certain documents from processing, improving performance.

- **Bug Fixes**
  - Enhanced detection of empty document summaries, reducing errors during processing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
darkskygit
2025-05-27 14:28:34 +00:00
parent 7eb6b268a6
commit 9220b973c7
6 changed files with 54 additions and 23 deletions
@@ -50,7 +50,7 @@ defineModuleConfig('job', {
'queues.copilot': {
desc: 'The config for copilot job queue',
default: {
concurrency: 5,
concurrency: 10,
},
schema,
},