Commit Graph

58 Commits

Author SHA1 Message Date
DarkSky cc1d5b497a feat(server): cleanup trashed doc's embedding (#13201)
fix AI-359

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added automated cleanup of embeddings for documents deleted or trashed
from workspaces.
* Introduced a new job to schedule and perform this cleanup per
workspace daily and on demand.
  * Added new GraphQL mutation to manually trigger the cleanup process.
* Added the ability to list workspaces with flexible filtering and
selection options.

* **Improvements**
* Enhanced document status handling to more accurately reflect embedding
presence.
* Refined internal methods for managing and checking document
embeddings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-07-15 12:00:33 +00:00
DarkSky 8e374f5517 feat(server): skip embedding for deprecated doc ids & empty docs (#13211)
fix AI-367

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved document filtering to exclude settings documents and empty
blobs from embedding and status calculations.
* Enhanced embedding jobs to skip processing deprecated documents if a
newer version exists, ensuring only up-to-date documents are embedded.
* **New Features**
* Added a mutation to trigger the cron job for generating missing
titles.
* **Tests**
* Added test to verify exclusion of documents with empty content from
embedding.
* Updated embedding-related tests to toggle embedding state during
attachment upload under simulated network conditions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-07-15 08:50:48 +00:00
德布劳外 · 贾贵 812c199b45 feat: split individual semantic change (#13155)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Introduced a new AI-powered document update feature, allowing users to
apply multiple independent block-level edits to Markdown documents.
* Added support for applying document updates via a new GraphQL query,
enabling seamless integration with the frontend.

* **Enhancements**
* Improved the document editing tool to handle and display multiple
simultaneous edit operations with better UI feedback and state
management.
* Expanded model support with new "morph-v3-fast" and "morph-v3-large"
options for document update operations.
* Enhanced frontend components and services to support asynchronous
application and acceptance of multiple document edits independently.

* **Bug Fixes**
* Enhanced error handling and user notifications for failed document
update operations.

* **Documentation**
* Updated tool descriptions and examples to clarify the new multi-edit
workflow and expected input/output formats.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

> CLOSE AI-337
2025-07-15 02:34:01 +00:00
DarkSky 9a3e44c6d6 feat(server): add generate title cron resolver (#13189)
fix AI-350

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added a new option to manually trigger the generation of missing
session titles via a GraphQL query.

* **Improvements**
* The process for generating missing session titles now considers all
eligible sessions, without limiting the number processed at a time.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-07-14 09:19:21 +00:00
DarkSky 6dac94d90a feat(server): paginated list endpoint (#13026)
fix AI-323
2025-07-08 17:11:58 +08:00
DarkSky 2ea3c3da9d feat(server): expose reuse latest chat (#12999)
fix AI-289
2025-07-03 04:46:23 +00:00
DarkSky 6e034185cf feat: title of session (#12971)
fix AI-253
2025-07-01 05:24:42 +00:00
DarkSky 06f27e8d6a feat(server): allow multiple session attach to doc (#12933)
fix AI-236
2025-06-26 02:15:31 +00:00
DarkSky e32c9a814a feat(server): improve session modify (#12928)
fix AI-248
2025-06-25 12:02:21 +00:00
DarkSky 07ec427021 fix(server): copilot permission check (#12923)
fix AI-176
2025-06-25 17:01:58 +08:00
DarkSky c7113b0195 feat(server): add pinned & action filter for session query (#12876)
fix AI-222
2025-06-20 08:31:04 +00:00
DarkSky bd04930560 feat(server): allow chat session dangling & pin session support (#12849)
fix AI-181
fix AI-179
fix AI-178
fix PD-2682
fix PD-2683
2025-06-19 13:17:01 +08:00
Wu Yue 6169cdab3a feat(core): add stream object api (#12841)
Close [AI-193](https://linear.app/affine-design/issue/AI-193)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Added support for streaming structured AI chat responses as objects,
enabling richer and more interactive chat experiences.
- Chat messages now include a new field displaying structured stream
objects, such as reasoning steps, text deltas, tool calls, and tool
results.
- GraphQL APIs and queries updated to expose these structured streaming
objects in chat histories.
  - Introduced a new streaming chat endpoint for object-based responses.

- **Bug Fixes**
- Improved error handling for streaming responses to ensure more robust
and informative error reporting.

- **Refactor**
- Centralized and streamlined session preparation and streaming logic
for AI chat providers.
  - Unified streaming setup across multiple AI model providers.

- **Tests**
- Extended test coverage for streaming object responses to ensure
reliability and correctness.

- **Documentation**
- Updated type definitions and schemas to reflect new streaming object
capabilities in both backend and frontend code.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: DarkSky <25152247+darkskygit@users.noreply.github.com>
2025-06-19 09:13:18 +08:00
akumatus eb49ffaedb feat(core): support fork session without latestMessageId (#12587)
Close [AI-86](https://linear.app/affine-design/issue/AI-86)

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **New Features**
  - Improved chat session forking to allow creating a fork without specifying the latest message, enabling more flexible session management.

- **Bug Fixes**
  - Forking a chat session with an invalid latest message ID now correctly returns an error.

- **Tests**
  - Added and updated test cases to cover session forking with missing or invalid latest message IDs, ensuring robust behavior in these scenarios.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-28 07:34:23 +00:00
darkskygit b388f92c96 feat(server): refactor provider interface (#11665)
fix AI-4
fix AI-18

better provider/model choose to allow fallback to similar models (e.g., self-hosted) when the provider is not fully configured
split functions of different output types
2025-05-22 06:28:20 +00:00
akumatus fcc9b31da9 feat(core): add get session graphql api (#12237)
Close [AI-116](https://linear.app/affine-design/issue/AI-116)

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Added the ability to retrieve detailed information for a specific Copilot session by its ID, including model metadata and optional models, via the user interface and API.
  - Session data now includes additional fields such as the model used and a list of optional models.
  - Enhanced GraphQL queries and UI components to support fetching and displaying these new session details.

- **Improvements**
  - Session lists now provide richer information, including model details, for each session.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-05-15 04:55:50 +00:00
darkskygit de8c65f4e6 fix(server): empty mimetype attachments fallback (#11869) 2025-04-22 15:36:35 +00:00
darkskygit b4c643e8bc chore(server): improve gql types (#11441) 2025-04-03 07:30:51 +00:00
darkskygit 10605b3793 fix(server): nullable value for parent id (#10725) 2025-03-12 03:53:33 +00:00
darkskygit fbb6df3da8 chore(server): return parent id of sessions (#10638) 2025-03-05 14:03:36 +00:00
liuyi 61162c59fc refactor(server): permission (#10449) 2025-03-05 15:57:00 +08:00
darkskygit 985906aa13 feat(server): copilot session prompt query (#10479) 2025-02-27 11:25:52 +00:00
darkskygit 42e0563d2e feat: filter out session create request for root doc (#10187) 2025-02-14 11:14:57 +00:00
akumatus 1bf1832211 refactor(core): get copilot sessions api (#10168)
Fix issue [BS-2575](https://linear.app/affine-design/issue/BS-2575).

### What Changed?
- Refactor `getCopilotSessions` api.
  - Add `docId` parameter.
  - Add `action` parameter.
2025-02-14 06:57:58 +00:00
akumatus 79d5f55471 feat(core): move context docs to user prompt (#10162)
Fix issue [BS-2522](https://linear.app/affine-design/issue/BS-2522).

### Why make this change?
If the user data contains illegal content, carrying the user data in the system prompt will run the risk of having the account banned.

### What Changed?
- Move the `Context Documents` to the user prompt.
- Add `withPrompt` in `QueryChatHistoriesInput` options.
- Get chat histories without prompt messages.
- Omit document context when saving messages to the `aiSessionMessage` db.
2025-02-13 12:08:13 +00:00
darkskygit a725df6ebe feat(server): basic context api (#10056)
fix CLOUD-97
fix CLOUD-98
2025-02-11 10:45:01 +00:00
forehalo d3843d8f11 refactor(server): role actions definition (#9962) 2025-02-06 04:54:34 +00:00
Brooooooklyn abeff8bb1a feat(server): doc level permission (#9760)
close CLOUD-89 CLOUD-90 CLOUD-91 CLOUD-92
2025-02-05 07:06:57 +00:00
akumatus 95cf2e047f feat(core): cite source documents in the AI answer (#9863)
Support issue [BS-2424](https://linear.app/affine-design/issue/BS-2424).

### What changed?
- Add relevant document prompt templates.
- Add citation rules in system prompts.
- Change message `params` type to `Record<string, any>`
- Add unit test.

<div class='graphite__hidden'>
          <div>🎥 Video uploaded on Graphite:</div>
            <a href="https://app.graphite.dev/media/video/sJGviKxfE3Ap685cl5bj/ec24e664-039e-4fab-bd26-b3312f011daf.mov">
              <img src="https://app.graphite.dev/api/v1/graphite/video/thumbnail/sJGviKxfE3Ap685cl5bj/ec24e664-039e-4fab-bd26-b3312f011daf.mov">
            </a>
          </div>
<video src="https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/sJGviKxfE3Ap685cl5bj/ec24e664-039e-4fab-bd26-b3312f011daf.mov">录屏2025-01-23 10.40.38.mov</video>
2025-01-24 04:04:01 +00:00
akumatus 58ce86533e feat(core): support ai network search (#9357)
### What Changed?
- Add `PerplexityProvider` in backend.
- Update session prompt name if user toggle network search mode in chat panel.
- Add experimental flag for AI network search feature.
- Add unit tests and e2e tests.

Search results are streamed and appear word for word:

<div class='graphite__hidden'>
          <div>🎥 Video uploaded on Graphite:</div>
            <a href="https://app.graphite.dev/media/video/sJGviKxfE3Ap685cl5bj/56f6ec7b-4b21-405f-9612-43e083f6fb84.mov">
              <img src="https://app.graphite.dev/api/v1/graphite/video/thumbnail/sJGviKxfE3Ap685cl5bj/56f6ec7b-4b21-405f-9612-43e083f6fb84.mov">
            </a>
          </div>
<video src="https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/sJGviKxfE3Ap685cl5bj/56f6ec7b-4b21-405f-9612-43e083f6fb84.mov">录屏2024-12-27 18.58.40.mov</video>

Click the little globe icon to manually turn on/off Internet search:
<div class='graphite__hidden'>
          <div>🎥 Video uploaded on Graphite:</div>
            <a href="https://app.graphite.dev/media/video/sJGviKxfE3Ap685cl5bj/778f1406-bf29-498e-a90d-7dad813392d1.mov">
              <img src="https://app.graphite.dev/api/v1/graphite/video/thumbnail/sJGviKxfE3Ap685cl5bj/778f1406-bf29-498e-a90d-7dad813392d1.mov">
            </a>
          </div>
<video src="https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/sJGviKxfE3Ap685cl5bj/778f1406-bf29-498e-a90d-7dad813392d1.mov">录屏2024-12-27 19.01.16.mov</video>

When there is an image, it will automatically switch to the openai model:

<div class='graphite__hidden'>
          <div>🎥 Video uploaded on Graphite:</div>
            <a href="https://app.graphite.dev/media/video/sJGviKxfE3Ap685cl5bj/56431d8e-75e1-4d84-ab4a-b6636042cc6a.mov">
              <img src="https://app.graphite.dev/api/v1/graphite/video/thumbnail/sJGviKxfE3Ap685cl5bj/56431d8e-75e1-4d84-ab4a-b6636042cc6a.mov">
            </a>
          </div>
<video src="https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/sJGviKxfE3Ap685cl5bj/56431d8e-75e1-4d84-ab4a-b6636042cc6a.mov">录屏2024-12-27 19.02.13.mov</video>
2025-01-09 04:00:59 +00:00
forehalo 0e73737407 refactor(server): make redis required module (#9121) 2024-12-13 06:27:15 +00:00
forehalo 4c23991047 chore: rename fundamentals to base (#9119) 2024-12-13 06:27:13 +00:00
darkskygit fa554b1054 feat: add copilot metrics (#8455)
fix CLOUD-73
2024-10-18 03:30:02 +00:00
forehalo 682a01e441 feat(server): make a singleton global mutex service (#7900) 2024-08-21 05:30:19 +00:00
forehalo 624f3514fc feat(server): make permission a standalone module (#7880) 2024-08-15 10:01:52 +00:00
darkskygit 339c39c1ec feat: improve prompt management (#7853) 2024-08-14 08:38:36 +00:00
JimmFly b214003968 feat(admin): add prompt management page (#7611)
close AF-907

Supports online modification of prompt, but does not support custom ai key yet

![CleanShot 2024-07-29 at 22 12 39@2x](https://github.com/user-attachments/assets/c67ad0d0-3e5b-44ff-b7db-d07dd11c19e2)
2024-08-13 14:51:54 +08:00
DarkSky dcb9d75db7 feat: allow sort and filter forked session (#7519) 2024-07-18 11:08:47 +08:00
darkskygit bf6c9a5955 feat: add prompt level config (#7445) 2024-07-08 08:11:22 +00:00
darkskygit e8285289fe feat: improve histories query for forked session (#7414) 2024-07-03 04:49:19 +00:00
darkskygit 61870c04d0 feat: fork session support (#7367) 2024-07-03 03:10:09 +00:00
liuyi 54fc1197ad feat(server): introduce user friendly server errors (#7111) 2024-06-17 11:30:58 +08:00
darkskygit a5f9bfdda9 fix: gql result check for clean history (#7154) 2024-06-05 07:49:40 +00:00
forehalo 4b30fbc1e2 feat(server): add copilot prompts management api (#7082) 2024-05-28 13:14:17 +00:00
darkskygit 937b8bf166 feat: history cleanup (#7007)
fix AFF-1069
2024-05-24 08:00:05 +00:00
darkskygit 850bbee629 test: copilot unit & e2e test (#6649)
fix CLOUD-31
2024-04-26 09:43:35 +00:00
darkskygit 15c1e46680 chore: use default throttler replace strict throttler (#6698) 2024-04-26 02:18:20 +00:00
forehalo 94de6f5853 fix(server): always return created timestamp of chat messages (#6658) 2024-04-22 08:39:42 +00:00
darkskygit 657a5250ad feat: update chat prompt (#6639)
fix AFF-878
fix AFF-852 (maybe)
2024-04-19 12:34:02 +00:00
darkskygit 28f2ff24b9 fix: copilot blob process (#6612) 2024-04-18 15:36:30 +00:00