Compare commits

...

16 Commits

Author SHA1 Message Date
akumatus
512a908fd4 fix(core): generate the image cannot enter text prompt (#12717)
Close [AI-167](https://linear.app/affine-design/issue/AI-167)

![截屏2025-06-05 12.15.49.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/sJGviKxfE3Ap685cl5bj/c1afff0e-1197-46dc-ae43-ff7257039509.png)

![截屏2025-06-05 12.14.07.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/sJGviKxfE3Ap685cl5bj/1439b0a7-1cca-4848-aea2-84bc73c536c5.png)

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Improved AI panel behavior with explicit modes for input and answer generation, providing a more intuitive user experience when interacting with AI features.

- **Refactor**
  - Streamlined AI panel toggling logic for more consistent and predictable panel states during different actions.

- **Tests**
  - Enhanced AI image generation test to simulate user input and send actions for more accurate end-to-end validation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-06-05 09:17:28 +00:00
liuyi
71be1d424a fix(server): oidc registration (#12723) 2025-06-05 09:16:21 +00:00
renovate
d6a26b8093 chore: bump up multer version to v2.0.1 [SECURITY] (#12716)
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [multer](https://redirect.github.com/expressjs/multer) | [`2.0.0` -> `2.0.1`](https://renovatebot.com/diffs/npm/multer/2.0.0/2.0.1) | [![age](https://developer.mend.io/api/mc/badges/age/npm/multer/2.0.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/npm/multer/2.0.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/npm/multer/2.0.0/2.0.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/multer/2.0.0/2.0.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) |

### GitHub Vulnerability Alerts

#### [CVE-2025-48997](https://redirect.github.com/expressjs/multer/security/advisories/GHSA-g5hg-p3ph-g8qg)

### Impact

A vulnerability in Multer versions >=1.4.4-lts.1, <2.0.1 allows an attacker to trigger a Denial of Service (DoS) by sending an upload file request with an empty string field name. This request causes an unhandled exception, leading to a crash of the process.

### Patches

Users should upgrade to `2.0.1`

### Workarounds

None

### References

35a3272b61
[https://github.com/expressjs/multer/issues/1233](https://redirect.github.com/expressjs/multer/issues/1233)
[https://github.com/expressjs/multer/pull/1256](https://redirect.github.com/expressjs/multer/pull/1256)

---

### Release Notes

<details>
<summary>expressjs/multer (multer)</summary>

### [`v2.0.1`](https://redirect.github.com/expressjs/multer/blob/HEAD/CHANGELOG.md#201)

[Compare Source](https://redirect.github.com/expressjs/multer/compare/v2.0.0...v2.0.1)

-   Fix [CVE-2025-48997](https://www.cve.org/CVERecord?id=CVE-2025-48997) ([GHSA-g5hg-p3ph-g8qg](https://redirect.github.com/expressjs/multer/security/advisories/GHSA-g5hg-p3ph-g8qg))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "" (UTC), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/toeverything/AFFiNE).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MC40MC4zIiwidXBkYXRlZEluVmVyIjoiNDAuNDAuMyIsInRhcmdldEJyYW5jaCI6ImNhbmFyeSIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiXX0=-->
2025-06-05 07:39:03 +00:00
EYHN
5e05952f6e feat(core): optimize tag performance (#12719)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **Refactor**
  - Improved tag options management to use a reactive, real-time approach, ensuring tag options are always up to date throughout the application.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-06-05 07:13:31 +00:00
JimmFly
c1930c5937 chore: adjust general access button styles (#12718)
close AF-2685

When the button is disabled, the frontmost icon is not positioned correctly. This commit is to fix the icon position.

![CleanShot 2025-06-05 at 12 38 55@2x](https://github.com/user-attachments/assets/af2f80bc-69a0-4e33-bc8f-e5e169f769fc)

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **Style**
  - Improved the layout of the share menu trigger text by aligning its content vertically and adding spacing between elements for a cleaner appearance.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-06-05 06:59:18 +00:00
fengmk2
b7ebd33389 chore(server): ignore rolled back error on the first time (#12714)
close #12692

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **Bug Fixes**
  - Improved error handling during migration rollbacks to better recognize and safely skip specific migration errors.
  - Enhanced logging for migration rollback failures to provide clearer information without interrupting the process.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-06-05 04:00:21 +00:00
Brooooooklyn
de9a3e1428 ci: fix missing environment in build-server-native (#12712)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **Chores**
  - Updated workflow configuration to set the environment for the build process based on input parameters.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-06-05 03:44:35 +00:00
fengmk2
374eee9196 chore(server): disable indexer by default (#12710)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **Chores**
	- Updated the default setting for the indexer feature to be disabled by default.
	- Added a sample environment variable for enabling the indexer in the example configuration file.
	- Introduced a new environment variable for the indexer in the CI workflow configuration.
- **Tests**
	- Adjusted test configurations to explicitly enable the indexer feature during test execution.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-06-05 03:28:12 +00:00
donteatfriedrice
1bdccdbd57 feat(editor): track citation events (#12664)
Closes: [BS-3551](https://linear.app/affine-design/issue/BS-3551/citation埋点)

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **New Features**
  - Enhanced citation tracking across attachments, bookmarks, embedded documents, paragraphs, footnotes, rename modals, and toolbars for actions like editing, deleting, expanding, and hovering on citations.
  - Introduced a centralized citation service to unify citation detection and telemetry event management.
- **Chores**
  - Updated service exports and telemetry modules to include the new citation service and citation-related event types.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-06-05 03:09:24 +00:00
donteatfriedrice
053efb61f0 fix(editor): should set event dispatcher active as false when document is hidden (#12559)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **New Features**
  - The application now automatically becomes inactive when the document is hidden, improving resource management and responsiveness to visibility changes.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-06-05 02:28:32 +00:00
pengx17
c7aebd0412 fix(electron): revert back electron to v35 (#12704)
v36 breaks worker loading in Electron's renderer
this use to work by turning off "PlzDedicatedWorker"
related to https://github.com/electron/electron/issues/43556

Before we know the root cause, revert back the electron version.

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **Chores**
  - Updated the version of Electron used in the application.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-06-04 15:17:51 +00:00
renovate
01aa6979eb chore: bump up @nestjs-cls/transactional-adapter-prisma version to v1.2.23 (#12680)
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [@nestjs-cls/transactional-adapter-prisma](https://papooch.github.io/nestjs-cls/) ([source](https://redirect.github.com/Papooch/nestjs-cls)) | [`1.2.21` -> `1.2.23`](https://renovatebot.com/diffs/npm/@nestjs-cls%2ftransactional-adapter-prisma/1.2.21/1.2.23) | [![age](https://developer.mend.io/api/mc/badges/age/npm/@nestjs-cls%2ftransactional-adapter-prisma/1.2.23?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/npm/@nestjs-cls%2ftransactional-adapter-prisma/1.2.23?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/npm/@nestjs-cls%2ftransactional-adapter-prisma/1.2.21/1.2.23?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/@nestjs-cls%2ftransactional-adapter-prisma/1.2.21/1.2.23?slim=true)](https://docs.renovatebot.com/merge-confidence/) |

---

### Release Notes

<details>
<summary>Papooch/nestjs-cls (@&#8203;nestjs-cls/transactional-adapter-prisma)</summary>

### [`v1.2.23`](https://redirect.github.com/Papooch/nestjs-cls/compare/@nestjs-cls/transactional-adapter-prisma@1.2.22...@nestjs-cls/transactional-adapter-prisma@1.2.23)

[Compare Source](https://redirect.github.com/Papooch/nestjs-cls/compare/@nestjs-cls/transactional-adapter-prisma@1.2.22...@nestjs-cls/transactional-adapter-prisma@1.2.23)

### [`v1.2.22`](https://redirect.github.com/Papooch/nestjs-cls/compare/@nestjs-cls/transactional-adapter-prisma@1.2.21...@nestjs-cls/transactional-adapter-prisma@1.2.22)

[Compare Source](https://redirect.github.com/Papooch/nestjs-cls/compare/@nestjs-cls/transactional-adapter-prisma@1.2.21...@nestjs-cls/transactional-adapter-prisma@1.2.22)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/toeverything/AFFiNE).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MC4zMy42IiwidXBkYXRlZEluVmVyIjoiNDAuNDAuMyIsInRhcmdldEJyYW5jaCI6ImNhbmFyeSIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiXX0=-->
2025-06-04 13:00:41 +00:00
akumatus
c32f7c7964 fix(core): read-only editor does not support code preview (#12700)
Close [AI-160](https://linear.app/affine-design/issue/AI-160)

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **New Features**
  - Improved preview state management for code blocks, ensuring consistent behavior in both editable and readonly modes.

- **Refactor**
  - Streamlined the way preview state is toggled and displayed for code blocks, resulting in a more reliable and maintainable user experience.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-06-04 12:41:10 +00:00
fengmk2
d219c92e98 chore(server): ignore never applied rolled back error (#12703)
closes #12701

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

- **Bug Fixes**
  - Improved error handling during migration rollbacks to prevent unnecessary errors when migrations were never applied.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-06-04 11:37:32 +00:00
darkskygit
063072457c fix(server): chat with image (#12699)
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

- **Refactor**
  - Improved the handling of attachments in chat messages for more efficient processing of images and files without impacting user experience.
- **Chores**
  - Added internal logging to enhance monitoring of AI model interactions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-06-04 08:51:02 +00:00
darkskygit
13fa4f922a fix(server): token calculate (#12667) 2025-06-04 07:09:33 +00:00
47 changed files with 533 additions and 284 deletions

View File

@@ -886,8 +886,8 @@
"properties": {
"enabled": {
"type": "boolean",
"description": "Enable indexer plugin\n@default true\n@environment `AFFINE_INDEXER_ENABLED`",
"default": true
"description": "Enable indexer plugin\n@default false\n@environment `AFFINE_INDEXER_ENABLED`",
"default": false
},
"provider.type": {
"type": "string",

View File

@@ -113,6 +113,7 @@ jobs:
build-server-native:
name: Build Server native - ${{ matrix.targets.name }}
runs-on: ubuntu-latest
environment: ${{ github.event.inputs.flavor }}
strategy:
fail-fast: false
matrix:

View File

@@ -20,6 +20,7 @@ env:
COVERAGE: true
MACOSX_DEPLOYMENT_TARGET: '10.13'
DEPLOYMENT_TYPE: affine
AFFINE_INDEXER_ENABLED: true
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}

235
Cargo.lock generated
View File

@@ -453,12 +453,6 @@ dependencies = [
"windows-targets 0.52.6",
]
[[package]]
name = "base64"
version = "0.21.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9d297deb1925b89f2ccc13d7635fa0714f12c87adce1c75356b39ca9b7178567"
[[package]]
name = "base64"
version = "0.22.1"
@@ -1263,7 +1257,7 @@ name = "docx-parser"
version = "0.1.1"
source = "git+https://github.com/toeverything/docx-parser#278ba3eeb29bbf1ee7958b02436e4402af61859b"
dependencies = [
"base64 0.22.1",
"base64",
"clap",
"docx-rust",
"serde",
@@ -1826,7 +1820,7 @@ dependencies = [
"js-sys",
"log",
"wasm-bindgen",
"windows-core 0.57.0",
"windows-core 0.61.2",
]
[[package]]
@@ -1838,18 +1832,6 @@ dependencies = [
"cc",
]
[[package]]
name = "icu_collections"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db2fa452206ebee18c4b5c2274dbf1de17008e874b4dc4f0aea9d01ca79e4526"
dependencies = [
"displaydoc",
"yoke 0.7.5",
"zerofrom",
"zerovec 0.10.4",
]
[[package]]
name = "icu_collections"
version = "2.0.0"
@@ -1858,9 +1840,25 @@ checksum = "200072f5d0e3614556f94a9930d5dc3e0662a652823904c3a75dc3b0af7fee47"
dependencies = [
"displaydoc",
"potential_utf",
"yoke 0.8.0",
"yoke",
"zerofrom",
"zerovec 0.11.2",
"zerovec",
]
[[package]]
name = "icu_locale"
version = "2.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ae5921528335e91da1b6c695dbf1ec37df5ac13faa3f91e5640be93aa2fbefd"
dependencies = [
"displaydoc",
"icu_collections",
"icu_locale_core",
"icu_locale_data",
"icu_provider",
"potential_utf",
"tinystr",
"zerovec",
]
[[package]]
@@ -1870,23 +1868,17 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0cde2700ccaed3872079a65fb1a78f6c0a36c91570f28755dda67bc8f7d9f00a"
dependencies = [
"displaydoc",
"litemap 0.8.0",
"tinystr 0.8.1",
"writeable 0.6.1",
"zerovec 0.11.2",
"litemap",
"tinystr",
"writeable",
"zerovec",
]
[[package]]
name = "icu_locid"
version = "1.5.0"
name = "icu_locale_data"
version = "2.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "13acbb8371917fc971be86fc8057c41a64b521c184808a698c02acc242dbf637"
dependencies = [
"displaydoc",
"litemap 0.7.5",
"tinystr 0.7.6",
"writeable 0.5.5",
]
checksum = "4fdef0c124749d06a743c69e938350816554eb63ac979166590e2b4ee4252765"
[[package]]
name = "icu_normalizer"
@@ -1895,12 +1887,12 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "436880e8e18df4d7bbc06d58432329d6458cc84531f7ac5f024e93deadb37979"
dependencies = [
"displaydoc",
"icu_collections 2.0.0",
"icu_collections",
"icu_normalizer_data",
"icu_properties",
"icu_provider 2.0.0",
"icu_provider",
"smallvec",
"zerovec 0.11.2",
"zerovec",
]
[[package]]
@@ -1916,13 +1908,13 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "016c619c1eeb94efb86809b015c58f479963de65bdb6253345c1a1276f22e32b"
dependencies = [
"displaydoc",
"icu_collections 2.0.0",
"icu_collections",
"icu_locale_core",
"icu_properties_data",
"icu_provider 2.0.0",
"icu_provider",
"potential_utf",
"zerotrie",
"zerovec 0.11.2",
"zerovec",
]
[[package]]
@@ -1931,23 +1923,6 @@ version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "298459143998310acd25ffe6810ed544932242d3f07083eee1084d83a71bd632"
[[package]]
name = "icu_provider"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ed421c8a8ef78d3e2dbc98a973be2f3770cb42b606e3ab18d6237c4dfde68d9"
dependencies = [
"displaydoc",
"icu_locid",
"icu_provider_macros",
"stable_deref_trait",
"tinystr 0.7.6",
"writeable 0.5.5",
"yoke 0.7.5",
"zerofrom",
"zerovec 0.10.4",
]
[[package]]
name = "icu_provider"
version = "2.0.0"
@@ -1957,46 +1932,37 @@ dependencies = [
"displaydoc",
"icu_locale_core",
"stable_deref_trait",
"tinystr 0.8.1",
"writeable 0.6.1",
"yoke 0.8.0",
"tinystr",
"writeable",
"yoke",
"zerofrom",
"zerotrie",
"zerovec 0.11.2",
]
[[package]]
name = "icu_provider_macros"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ec89e9337638ecdc08744df490b221a7399bf8d164eb52a665454e60e075ad6"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.101",
"zerovec",
]
[[package]]
name = "icu_segmenter"
version = "1.5.0"
version = "2.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a717725612346ffc2d7b42c94b820db6908048f39434504cb130e8b46256b0de"
checksum = "e185fc13b6401c138cf40db12b863b35f5edf31b88192a545857b41aeaf7d3d3"
dependencies = [
"core_maths",
"displaydoc",
"icu_collections 1.5.0",
"icu_locid",
"icu_provider 1.5.0",
"icu_collections",
"icu_locale",
"icu_locale_core",
"icu_provider",
"icu_segmenter_data",
"potential_utf",
"utf8_iter",
"zerovec 0.10.4",
"zerovec",
]
[[package]]
name = "icu_segmenter_data"
version = "1.5.1"
version = "2.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a1e52775179941363cc594e49ce99284d13d6948928d8e72c755f55e98caa1eb"
checksum = "5360a2fbe97f617c4f8b944356dedb36d423f7da7f13c070995cf89e59f01220"
[[package]]
name = "idna"
@@ -2193,7 +2159,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "07033963ba89ebaf1584d767badaa2e8fcec21aedea6b8c0346d487d49c28667"
dependencies = [
"cfg-if",
"windows-targets 0.48.5",
"windows-targets 0.52.6",
]
[[package]]
@@ -2229,12 +2195,6 @@ version = "0.9.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cd945864f07fe9f5371a27ad7b52a172b4b499999f1d97574c9fa68373937e12"
[[package]]
name = "litemap"
version = "0.7.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "23fb14cb19457329c82206317a5663005a4d404783dc74f4252769b0d5f42856"
[[package]]
name = "litemap"
version = "0.8.0"
@@ -2963,7 +2923,8 @@ version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5a7c30837279ca13e7c867e9e40053bc68740f988cb07f7ca6df43cc734b585"
dependencies = [
"zerovec 0.11.2",
"serde",
"zerovec",
]
[[package]]
@@ -3770,7 +3731,7 @@ version = "0.8.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ee6798b1838b6a0f69c007c133b8df5866302197e404e8b6ee8ed3e3a5e68dc6"
dependencies = [
"base64 0.22.1",
"base64",
"bytes",
"chrono",
"crc",
@@ -3846,7 +3807,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aa003f0038df784eb8fecbbac13affe3da23b45194bd57dba231c8f48199c526"
dependencies = [
"atoi",
"base64 0.22.1",
"base64",
"bitflags 2.9.1",
"byteorder",
"bytes",
@@ -3889,7 +3850,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db58fcd5a53cf07c184b154801ff91347e4c30d17a3562a635ff028ad5deda46"
dependencies = [
"atoi",
"base64 0.22.1",
"base64",
"bitflags 2.9.1",
"byteorder",
"chrono",
@@ -4300,14 +4261,14 @@ dependencies = [
[[package]]
name = "text-splitter"
version = "0.25.1"
version = "0.27.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d8130aecc3b7938ce3ea387d7615eca92bd4f702a5adc0548ba930a9c039dda4"
checksum = "9f7e97f5863248f7d07896a1816bd4110cb1b0b122741f157d702121a270bf33"
dependencies = [
"ahash",
"auto_enums",
"either",
"icu_provider 1.5.0",
"icu_provider",
"icu_segmenter",
"itertools 0.14.0",
"memchr",
@@ -4378,29 +4339,19 @@ dependencies = [
[[package]]
name = "tiktoken-rs"
version = "0.6.0"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "44075987ee2486402f0808505dd65692163d243a337fc54363d49afac41087f6"
checksum = "25563eeba904d770acf527e8b370fe9a5547bacd20ff84a0b6c3bc41288e5625"
dependencies = [
"anyhow",
"base64 0.21.7",
"base64",
"bstr",
"fancy-regex",
"lazy_static",
"parking_lot",
"regex",
"rustc-hash 1.1.0",
]
[[package]]
name = "tinystr"
version = "0.7.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9117f5d4db391c1cf6927e7bea3db74b9a1c1add8f7eda9ffd5364f40f57b82f"
dependencies = [
"displaydoc",
]
[[package]]
name = "tinystr"
version = "0.8.1"
@@ -4408,7 +4359,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5d4f6d1145dcb577acf783d4e601bc1d76a13337bb54e6233add580b07344c8b"
dependencies = [
"displaydoc",
"zerovec 0.11.2",
"zerovec",
]
[[package]]
@@ -4590,9 +4541,9 @@ dependencies = [
[[package]]
name = "tree-sitter-c"
version = "0.23.4"
version = "0.24.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "afd2b1bf1585dc2ef6d69e87d01db8adb059006649dd5f96f31aa789ee6e9c71"
checksum = "1a3aad8f0129083a59fe8596157552d2bb7148c492d44c21558d68ca1c722707"
dependencies = [
"cc",
"tree-sitter-language",
@@ -5165,7 +5116,7 @@ version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf221c93e13a30d793f7645a0e7762c55d169dbb0a49671918a2319d289b10bb"
dependencies = [
"windows-sys 0.48.0",
"windows-sys 0.59.0",
]
[[package]]
@@ -5504,12 +5455,6 @@ dependencies = [
"bitflags 2.9.1",
]
[[package]]
name = "writeable"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e9df38ee2d2c3c5948ea468a8406ff0db0b29ae1ffde1bcf20ef305bcc95c51"
[[package]]
name = "writeable"
version = "0.6.1"
@@ -5614,18 +5559,6 @@ dependencies = [
"yrs 0.17.4",
]
[[package]]
name = "yoke"
version = "0.7.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "120e6aef9aa629e3d4f52dc8cc43a015c7724194c97dfaf45180d2daf2b77f40"
dependencies = [
"serde",
"stable_deref_trait",
"yoke-derive 0.7.5",
"zerofrom",
]
[[package]]
name = "yoke"
version = "0.8.0"
@@ -5634,22 +5567,10 @@ checksum = "5f41bb01b8226ef4bfd589436a297c53d118f65921786300e427be8d487695cc"
dependencies = [
"serde",
"stable_deref_trait",
"yoke-derive 0.8.0",
"yoke-derive",
"zerofrom",
]
[[package]]
name = "yoke-derive"
version = "0.7.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2380878cad4ac9aac1e2435f3eb4020e8374b5f13c296cb75b4620ff8e229154"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.101",
"synstructure",
]
[[package]]
name = "yoke-derive"
version = "0.8.0"
@@ -5749,41 +5670,19 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "36f0bbd478583f79edad978b407914f61b2972f5af6fa089686016be8f9af595"
dependencies = [
"displaydoc",
"yoke 0.8.0",
"yoke",
"zerofrom",
]
[[package]]
name = "zerovec"
version = "0.10.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aa2b893d79df23bfb12d5461018d408ea19dfafe76c2c7ef6d4eba614f8ff079"
dependencies = [
"yoke 0.7.5",
"zerofrom",
"zerovec-derive 0.10.3",
]
[[package]]
name = "zerovec"
version = "0.11.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4a05eb080e015ba39cc9e23bbe5e7fb04d5fb040350f99f34e338d5fdd294428"
dependencies = [
"yoke 0.8.0",
"yoke",
"zerofrom",
"zerovec-derive 0.11.1",
]
[[package]]
name = "zerovec-derive"
version = "0.10.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6eafa6dfb17584ea3e2bd6e76e0cc15ad7af12b09abdd1ca55961bed9b1063c6"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.101",
"zerovec-derive",
]
[[package]]

View File

@@ -77,12 +77,12 @@ smol_str = "0.3"
sqlx = { version = "0.8", default-features = false, features = ["chrono", "macros", "migrate", "runtime-tokio", "sqlite", "tls-rustls"] }
strum_macros = "0.27.0"
symphonia = { version = "0.5", features = ["all", "opt-simd"] }
text-splitter = "0.25"
text-splitter = "0.27"
thiserror = "2"
tiktoken-rs = "0.6"
tiktoken-rs = "0.7"
tokio = "1.45"
tree-sitter = { version = "0.25" }
tree-sitter-c = { version = "0.23" }
tree-sitter-c = { version = "0.24" }
tree-sitter-c-sharp = { version = "0.23" }
tree-sitter-cpp = { version = "0.23" }
tree-sitter-go = { version = "0.23" }

View File

@@ -17,6 +17,7 @@ import {
AttachmentBlockStyles,
} from '@blocksuite/affine-model';
import {
CitationProvider,
DocModeProvider,
FileSizeLimitProvider,
TelemetryProvider,
@@ -37,6 +38,7 @@ import { type ClassInfo, classMap } from 'lit/directives/class-map.js';
import { guard } from 'lit/directives/guard.js';
import { styleMap } from 'lit/directives/style-map.js';
import { when } from 'lit/directives/when.js';
import { filter } from 'rxjs/operators';
import { AttachmentEmbedProvider } from './embed';
import { styles } from './styles';
@@ -79,8 +81,12 @@ export class AttachmentBlockComponent extends CaptionedBlockComponent<Attachment
return this.std.get(FileSizeLimitProvider).maxFileSize;
}
get citationService() {
return this.std.get(CitationProvider);
}
get isCitation() {
return !!this.model.props.footnoteIdentifier;
return this.citationService.isCitationModel(this.model);
}
convertTo = () => {
@@ -139,6 +145,34 @@ export class AttachmentBlockComponent extends CaptionedBlockComponent<Attachment
selectionManager.setGroup('note', [blockSelection]);
}
private readonly _trackCitationDeleteEvent = () => {
// Check citation delete event
this._disposables.add(
this.std.store.slots.blockUpdated
.pipe(
filter(payload => {
if (!payload.isLocal) return false;
const { flavour, id, type } = payload;
if (
type !== 'delete' ||
flavour !== this.model.flavour ||
id !== this.model.id
)
return false;
const { model } = payload;
if (!this.citationService.isCitationModel(model)) return false;
return true;
})
)
.subscribe(() => {
this.citationService.trackEvent('Delete');
})
);
};
override connectedCallback() {
super.connectedCallback();
@@ -162,6 +196,8 @@ export class AttachmentBlockComponent extends CaptionedBlockComponent<Attachment
});
});
}
this._trackCitationDeleteEvent();
}
override firstUpdated() {

View File

@@ -1,6 +1,7 @@
import { ConfirmIcon } from '@blocksuite/affine-components/icons';
import { toast } from '@blocksuite/affine-components/toast';
import type { AttachmentBlockModel } from '@blocksuite/affine-model';
import { CitationProvider } from '@blocksuite/affine-shared/services';
import type { EditorHost } from '@blocksuite/std';
import { html } from 'lit';
import { createRef, ref } from 'lit/directives/ref.js';
@@ -33,6 +34,7 @@ export const RenameModal = ({
let fileName = includeExtension ? nameWithoutExtension : originalName;
const extension = includeExtension ? originalExtension : '';
const citationService = editorHost.std.get(CitationProvider);
const abort = () => abortController.abort();
const onConfirm = () => {
@@ -44,6 +46,9 @@ export const RenameModal = ({
model.store.updateBlock(model, {
name: newFileName,
});
if (citationService.isCitationModel(model)) {
citationService.trackEvent('Edit');
}
abort();
};
const onInput = (e: InputEvent) => {

View File

@@ -8,6 +8,7 @@ import type {
} from '@blocksuite/affine-model';
import { ImageProxyService } from '@blocksuite/affine-shared/adapters';
import {
CitationProvider,
DocModeProvider,
LinkPreviewServiceIdentifier,
} from '@blocksuite/affine-shared/services';
@@ -18,6 +19,7 @@ import { html } from 'lit';
import { property, query } from 'lit/decorators.js';
import { type ClassInfo, classMap } from 'lit/directives/class-map.js';
import { type StyleInfo, styleMap } from 'lit/directives/style-map.js';
import { filter } from 'rxjs/operators';
import { refreshBookmarkUrlData } from './utils.js';
@@ -114,11 +116,12 @@ export class BookmarkBlockComponent extends CaptionedBlockComponent<BookmarkBloc
);
};
get citationService() {
return this.std.get(CitationProvider);
}
get isCitation() {
return (
!!this.model.props.footnoteIdentifier &&
this.model.props.style === 'citation'
);
return this.citationService.isCitationModel(this.model);
}
get imageProxyService() {
@@ -166,6 +169,31 @@ export class BookmarkBlockComponent extends CaptionedBlockComponent<BookmarkBloc
></bookmark-card>`;
};
private readonly _trackCitationDeleteEvent = () => {
// Check citation delete event
this._disposables.add(
this.std.store.slots.blockUpdated
.pipe(
filter(payload => {
if (!payload.isLocal) return false;
const { flavour, id, type } = payload;
if (
type !== 'delete' ||
flavour !== this.model.flavour ||
id !== this.model.id
)
return false;
const { model } = payload;
if (!this.citationService.isCitationModel(model)) return false;
return true;
})
)
.subscribe(() => {
this.citationService.trackEvent('Delete');
})
);
};
override connectedCallback() {
super.connectedCallback();
@@ -203,6 +231,8 @@ export class BookmarkBlockComponent extends CaptionedBlockComponent<BookmarkBloc
}
})
);
this._trackCitationDeleteEvent();
}
override disconnectedCallback(): void {

View File

@@ -40,6 +40,16 @@ export class CodeBlockComponent extends CaptionedBlockComponent<CodeBlockModel>
private _inlineRangeProvider: InlineRangeProvider | null = null;
private readonly _localPreview$ = signal<boolean | null>(null);
preview$: Signal<boolean> = computed(() => {
const modelPreview = !!this.model.props.preview$.value;
if (this.store.readonly) {
return this._localPreview$.value ?? modelPreview;
}
return modelPreview;
});
highlightTokens$: Signal<ThemedToken[][]> = signal([]);
languageName$: Signal<string> = computed(() => {
@@ -393,7 +403,7 @@ export class CodeBlockComponent extends CaptionedBlockComponent<CodeBlockModel>
true) &&
(this.model.props.lineNumber ?? true);
const preview = !!this.model.props.preview;
const preview = this.preview$.value;
const previewContext = this.std.getOptional(
CodeBlockPreviewIdentifier(this.model.props.language ?? '')
);
@@ -461,6 +471,14 @@ export class CodeBlockComponent extends CaptionedBlockComponent<CodeBlockModel>
override accessor useCaptionEditor = true;
override accessor useZeroWidth = true;
setPreviewState(preview: boolean) {
if (this.store.readonly) {
this._localPreview$.value = preview;
} else {
this.store.updateBlock(this.model, { preview });
}
}
}
declare global {

View File

@@ -58,11 +58,7 @@ export class PreviewButton extends WithDisposable(SignalWatcher(LitElement)) {
`;
private readonly _toggle = (value: boolean) => {
if (this.blockComponent.store.readonly) return;
this.blockComponent.store.updateBlock(this.blockComponent.model, {
preview: value,
});
this.blockComponent.setPreviewState(value);
const std = this.blockComponent.std;
const mode = std.getOptional(DocModeProvider)?.getEditorMode() ?? 'page';
@@ -77,7 +73,7 @@ export class PreviewButton extends WithDisposable(SignalWatcher(LitElement)) {
};
get preview() {
return !!this.blockComponent.model.props.preview$.value;
return this.blockComponent.preview$.value;
}
override render() {

View File

@@ -17,6 +17,7 @@ import {
REFERENCE_NODE,
} from '@blocksuite/affine-shared/consts';
import {
CitationProvider,
DocDisplayMetaProvider,
DocModeProvider,
OpenDocExtensionIdentifier,
@@ -43,6 +44,7 @@ import { repeat } from 'lit/directives/repeat.js';
import { styleMap } from 'lit/directives/style-map.js';
import { when } from 'lit/directives/when.js';
import throttle from 'lodash-es/throttle';
import { filter } from 'rxjs/operators';
import * as Y from 'yjs';
import { renderLinkedDocInCard } from '../common/render-linked-doc';
@@ -254,11 +256,12 @@ export class EmbedLinkedDocBlockComponent extends EmbedBlockComponent<EmbedLinke
return this.store.readonly;
}
get citationService() {
return this.std.get(CitationProvider);
}
get isCitation() {
return (
!!this.model.props.footnoteIdentifier &&
this.model.props.style === 'citation'
);
return this.citationService.isCitationModel(this.model);
}
private readonly _handleDoubleClick = (event: MouseEvent) => {
@@ -454,6 +457,31 @@ export class EmbedLinkedDocBlockComponent extends EmbedBlockComponent<EmbedLinke
);
};
private readonly _trackCitationDeleteEvent = () => {
// Check citation delete event
this._disposables.add(
this.std.store.slots.blockUpdated
.pipe(
filter(payload => {
if (!payload.isLocal) return false;
const { flavour, id, type } = payload;
if (
type !== 'delete' ||
flavour !== this.model.flavour ||
id !== this.model.id
)
return false;
const { model } = payload;
if (!this.citationService.isCitationModel(model)) return false;
return true;
})
)
.subscribe(() => {
this.citationService.trackEvent('Delete');
})
);
};
override connectedCallback() {
super.connectedCallback();
@@ -532,6 +560,8 @@ export class EmbedLinkedDocBlockComponent extends EmbedBlockComponent<EmbedLinke
}
})
);
this._trackCitationDeleteEvent();
}
getInitialState(): {

View File

@@ -7,7 +7,10 @@ import {
BLOCK_CHILDREN_CONTAINER_PADDING_LEFT,
EDGELESS_TOP_CONTENTEDITABLE_SELECTOR,
} from '@blocksuite/affine-shared/consts';
import { DocModeProvider } from '@blocksuite/affine-shared/services';
import {
CitationProvider,
DocModeProvider,
} from '@blocksuite/affine-shared/services';
import {
calculateCollapsedSiblings,
getNearestHeadingBefore,
@@ -63,6 +66,10 @@ export class ParagraphBlockComponent extends CaptionedBlockComponent<ParagraphBl
?.getPlaceholder(this.model);
}
get citationService() {
return this.std.get(CitationProvider);
}
get attributeRenderer() {
return this.inlineManager.getRenderer();
}
@@ -94,6 +101,12 @@ export class ParagraphBlockComponent extends CaptionedBlockComponent<ParagraphBl
return this.std.get(DefaultInlineManagerExtension.identifier);
}
get hasCitationSiblings() {
return this.collapsedSiblings.some(sibling =>
this.citationService.isCitationModel(sibling)
);
}
override get topContenteditableElement() {
if (this.std.get(DocModeProvider).getEditorMode() === 'edgeless') {
return this.closest<BlockComponent>(
@@ -286,6 +299,13 @@ export class ParagraphBlockComponent extends CaptionedBlockComponent<ParagraphBl
collapsed: value,
});
}
if (this.hasCitationSiblings) {
this.citationService.trackEvent('Expand', {
control: 'Source Button',
type: value ? 'Hide' : 'Show',
});
}
}}
></blocksuite-toggle-button>
`

View File

@@ -9,6 +9,7 @@ import {
} from '@blocksuite/affine-ext-loader';
import {
AutoClearSelectionService,
CitationService,
DefaultOpenDocExtension,
DNDAPIExtension,
DocDisplayMetaService,
@@ -76,6 +77,7 @@ export class FoundationViewExtension extends ViewExtensionProvider<FoundationVie
FileSizeLimitService,
LinkPreviewCache,
LinkPreviewService,
CitationService,
]);
context.register(clipboardConfigs);
if (this.isEdgeless(context.scope)) {

View File

@@ -1,6 +1,7 @@
import { HoverController } from '@blocksuite/affine-components/hover';
import { PeekViewProvider } from '@blocksuite/affine-components/peek';
import type { FootNote } from '@blocksuite/affine-model';
import { CitationProvider } from '@blocksuite/affine-shared/services';
import { unsafeCSSVarV2 } from '@blocksuite/affine-shared/theme';
import type { AffineTextAttributes } from '@blocksuite/affine-shared/types';
import { WithDisposable } from '@blocksuite/global/lit';
@@ -117,6 +118,10 @@ export class AffineFootnoteNode extends WithDisposable(ShadowlessElement) {
return this.std.store.readonly;
}
get citationService() {
return this.std.get(CitationProvider);
}
onFootnoteClick = () => {
if (!this.footnote) {
return;
@@ -215,6 +220,10 @@ export class AffineFootnoteNode extends WithDisposable(ShadowlessElement) {
return null;
}
this.citationService.trackEvent('Hover', {
control: 'Source Footnote',
});
return {
template: this._FootNotePopup(footnote, abortController),
container: this.std.host,

View File

@@ -0,0 +1,84 @@
import { type Container, createIdentifier } from '@blocksuite/global/di';
import { type BlockStdScope, StdIdentifier } from '@blocksuite/std';
import { type BlockModel, Extension } from '@blocksuite/store';
import { DocModeProvider } from '../doc-mode-service';
import type {
CitationEvents,
CitationEventType,
} from '../telemetry-service/citation';
import { TelemetryProvider } from '../telemetry-service/telemetry-service';
const CitationEventTypeMap = {
Hover: 'AICitationHoverSource',
Expand: 'AICitationExpandSource',
Delete: 'AICitationDelete',
Edit: 'AICitationEdit',
} as const;
type EventType = keyof typeof CitationEventTypeMap;
type EventTypeMapping = {
[K in EventType]: CitationEventType;
};
export interface CitationViewService {
/**
* Tracks citation-related events
* @param type - The type of citation event to track
* @param properties - The properties of the event
*/
trackEvent<T extends EventType>(
type: T,
properties?: CitationEvents[EventTypeMapping[T]]
): void;
/**
* Checks if the model is a citation model
* @param model - The model to check
* @returns True if the model is a citation model, false otherwise
*/
isCitationModel(model: BlockModel): boolean;
}
export const CitationProvider =
createIdentifier<CitationViewService>('CitationService');
export class CitationService extends Extension implements CitationViewService {
constructor(private readonly std: BlockStdScope) {
super();
}
static override setup(di: Container) {
di.addImpl(CitationProvider, CitationService, [StdIdentifier]);
}
get docModeService() {
return this.std.getOptional(DocModeProvider);
}
get telemetryService() {
return this.std.getOptional(TelemetryProvider);
}
isCitationModel = (model: BlockModel) => {
return (
'footnoteIdentifier' in model.props &&
!!model.props.footnoteIdentifier &&
'style' in model.props &&
model.props.style === 'citation'
);
};
trackEvent<T extends EventType>(
type: T,
properties?: CitationEvents[EventTypeMapping[T]]
) {
const editorMode = this.docModeService?.getEditorMode() ?? 'page';
this.telemetryService?.track(CitationEventTypeMap[type], {
page: editorMode === 'page' ? 'doc editor' : 'whiteboard editor',
module: 'AI Result',
control: 'Source',
...properties,
});
}
}

View File

@@ -0,0 +1 @@
export * from './citation-service';

View File

@@ -1,5 +1,6 @@
export * from './auto-clear-selection-service';
export * from './block-meta-service';
export * from './citation-service';
export * from './doc-display-meta-service';
export * from './doc-mode-service';
export * from './drag-handle-config';

View File

@@ -0,0 +1,8 @@
import type { TelemetryEvent } from './types';
export type CitationEventType =
| 'AICitationHoverSource'
| 'AICitationExpandSource'
| 'AICitationDelete'
| 'AICitationEdit';
export type CitationEvents = Record<CitationEventType, TelemetryEvent>;

View File

@@ -1,3 +1,4 @@
export * from './citation.js';
export * from './database.js';
export * from './link.js';
export * from './telemetry-service.js';

View File

@@ -1,6 +1,7 @@
import { createIdentifier } from '@blocksuite/global/di';
import type { ExtensionType } from '@blocksuite/store';
import type { CitationEvents } from './citation.js';
import type { CodeBlockEvents } from './code-block.js';
import type { OutDatabaseAllEvents } from './database.js';
import type { LinkToolbarEvents } from './link.js';
@@ -28,7 +29,8 @@ export type TelemetryEventMap = OutDatabaseAllEvents &
LinkToolbarEvents &
SlashMenuEvents &
CodeBlockEvents &
NoteEvents & {
NoteEvents &
CitationEvents & {
DocCreated: DocCreatedEvent;
Link: TelemetryEvent;
LinkedDocCreated: LinkedDocCreatedEvent;

View File

@@ -220,6 +220,12 @@ export class UIEventDispatcher extends LifeCycleWatcher {
this._setActive(false);
});
// When the document is hidden, the event dispatcher should be inactive
this.disposables.addFromEvent(document, 'visibilitychange', () => {
if (document.visibilityState === 'hidden') {
this._setActive(false);
}
});
}
private _buildEventScopeBySelection(name: EventName) {

View File

@@ -68,7 +68,7 @@
"@vitest/coverage-istanbul": "3.1.3",
"@vitest/ui": "3.1.3",
"cross-env": "^7.0.3",
"electron": "^36.0.0",
"electron": "^35.0.0",
"eslint": "^9.16.0",
"eslint-config-prettier": "^10.0.0",
"eslint-import-resolver-typescript": "^4.0.0",

View File

@@ -15,16 +15,25 @@ pub fn from_model_name(model_name: String) -> Option<Tokenizer> {
impl Tokenizer {
#[napi]
pub fn count(&self, content: String, allowed_special: Option<Vec<String>>) -> u32 {
self
.inner
.encode(
&content,
if let Some(allowed_special) = &allowed_special {
HashSet::from_iter(allowed_special.iter().map(|s| s.as_str()))
} else {
Default::default()
},
)
.len() as u32
let allowed_special = if let Some(allowed_special) = &allowed_special {
HashSet::from_iter(allowed_special.iter().map(|s| s.as_str()))
} else {
Default::default()
};
self.inner.encode(&content, &allowed_special).0.len() as u32
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_tokenizer() {
let tokenizer = from_model_name("gpt-4.1".to_string()).unwrap();
let content = "Hello, world!";
let count = tokenizer.count(content.to_string(), None);
assert!(count > 0);
}
}

View File

@@ -9,4 +9,6 @@
# MAILER_SENDER="noreply@toeverything.info"
# MAILER_USER="noreply@toeverything.info"
# MAILER_PASSWORD="affine"
# MAILER_SECURE=false
# MAILER_SECURE=false
# AFFINE_INDEXER_ENABLED=true

View File

@@ -64,12 +64,21 @@ function fixFailedMigrations() {
if (
err.message.includes(
'cannot be rolled back because it is not in a failed state'
) ||
err.message.includes(
'cannot be rolled back because it was never applied'
) ||
err.message.includes(
'called markMigrationRolledBack on a database without migrations table'
)
) {
// migration has been rolled back, skip it
continue;
}
throw err;
// ignore other errors
console.log(
`migration [${migration}] rolled back failed. ${err.message}`
);
}
}
}

View File

@@ -57,7 +57,6 @@ export abstract class AnthropicProvider<T> extends CopilotProvider<T> {
try {
metrics.ai.counter('chat_text_calls').add(1, { model: model.id });
const [system, msgs] = await chatToGPTMessage(messages, true, true);
const modelInstance = this.instance(model.id);
@@ -94,7 +93,8 @@ export abstract class AnthropicProvider<T> extends CopilotProvider<T> {
try {
metrics.ai.counter('chat_text_stream_calls').add(1, { model: model.id });
const [system, msgs] = await chatToGPTMessage(messages);
const [system, msgs] = await chatToGPTMessage(messages, true, true);
const { fullStream } = streamText({
model: this.instance(model.id),
system,

View File

@@ -101,13 +101,13 @@ export async function chatToGPTMessage(
({ attachment, mimeType } = attachment);
}
if (SIMPLE_IMAGE_URL_REGEX.test(attachment)) {
const data =
attachment.startsWith('data:') || useBase64Attachment
? await fetch(attachment).then(r => r.arrayBuffer())
: new URL(attachment);
if (mimeType.startsWith('image/')) {
contents.push({ type: 'image', image: attachment, mimeType });
contents.push({ type: 'image', image: data, mimeType });
} else {
const data =
attachment.startsWith('data:') || useBase64Attachment
? await fetch(attachment).then(r => r.arrayBuffer())
: new URL(attachment);
contents.push({ type: 'file' as const, data, mimeType });
}
}

View File

@@ -19,7 +19,6 @@ import { ChatMessageCache } from './message';
import { PromptService } from './prompt';
import { PromptMessage, PromptParams } from './providers';
import {
AvailableModel,
ChatHistory,
ChatMessage,
ChatMessageSchema,
@@ -38,7 +37,7 @@ export class ChatSession implements AsyncDisposable {
private readonly messageCache: ChatMessageCache,
private readonly state: ChatSessionState,
private readonly dispose?: (state: ChatSessionState) => Promise<void>,
private readonly maxTokenSize = 3840
private readonly maxTokenSize = state.prompt.config?.maxTokens || 128 * 1024
) {}
get model() {
@@ -297,8 +296,8 @@ export class ChatSessionService {
messageCost: { increment: userMessages.length },
tokenCost: {
increment: this.calculateTokenSize(
userMessages,
state.prompt.model as AvailableModel
state.messages,
state.prompt.model
),
},
},
@@ -402,10 +401,7 @@ export class ChatSessionService {
});
}
private calculateTokenSize(
messages: PromptMessage[],
model: AvailableModel
): number {
private calculateTokenSize(messages: PromptMessage[], model: string): number {
const encoder = getTokenEncoder(model);
return messages
.map(m => encoder?.count(m.content) ?? 0)

View File

@@ -47,26 +47,24 @@ export enum AvailableModels {
Gpt41 = 'gpt-4.1',
Gpt410414 = 'gpt-4.1-2025-04-14',
Gpt41Mini = 'gpt-4.1-mini',
Gpt41Nano = 'gpt-4.1-nano',
// embeddings
TextEmbedding3Large = 'text-embedding-3-large',
TextEmbedding3Small = 'text-embedding-3-small',
TextEmbeddingAda002 = 'text-embedding-ada-002',
// moderation
TextModerationLatest = 'text-moderation-latest',
TextModerationStable = 'text-moderation-stable',
// text to image
DallE3 = 'dall-e-3',
GptImage = 'gpt-image-1',
}
export type AvailableModel = keyof typeof AvailableModels;
const availableModels = Object.values(AvailableModels);
export function getTokenEncoder(model?: string | null): Tokenizer | null {
if (!model) return null;
const modelStr = AvailableModels[model as AvailableModel];
if (!modelStr) return null;
if (modelStr.startsWith('gpt')) {
return fromModelName(modelStr);
} else if (modelStr.startsWith('dall')) {
if (!availableModels.includes(model as AvailableModels)) return null;
if (model.startsWith('gpt')) {
return fromModelName(model);
} else if (model.startsWith('dall')) {
// dalle don't need to calc the token
return null;
} else {

View File

@@ -3,11 +3,19 @@ import Sinon from 'sinon';
import { createModule } from '../../../__tests__/create-module';
import { Config } from '../../../base';
import { ConfigModule } from '../../../base/config';
import { IndexerModule } from '..';
import { IndexerEvent } from '../event';
const module = await createModule({
imports: [IndexerModule],
imports: [
IndexerModule,
ConfigModule.override({
indexer: {
enabled: true,
},
}),
],
});
const indexerEvent = module.get(IndexerEvent);
const config = module.get(Config);

View File

@@ -7,6 +7,7 @@ import Sinon from 'sinon';
import { createModule } from '../../../__tests__/create-module';
import { Mockers } from '../../../__tests__/mocks';
import { JOB_SIGNAL } from '../../../base';
import { ConfigModule } from '../../../base/config';
import { ServerConfigModule } from '../../../core/config';
import { Models } from '../../../models';
import { IndexerModule, IndexerService } from '..';
@@ -15,7 +16,15 @@ import { IndexerJob } from '../job';
import { ManticoresearchProvider } from '../providers';
const module = await createModule({
imports: [IndexerModule, ServerConfigModule],
imports: [
IndexerModule,
ServerConfigModule,
ConfigModule.override({
indexer: {
enabled: true,
},
}),
],
providers: [IndexerService],
});
const indexerService = module.get(IndexerService);

View File

@@ -33,6 +33,7 @@ _test.before(async () => {
IndexerModule,
ConfigModule.override({
indexer: {
enabled: true,
provider: {
type: SearchProviderType.Elasticsearch,
endpoint: 'http://localhost:9200',

View File

@@ -18,6 +18,7 @@ const module = await createModule({
IndexerModule,
ConfigModule.override({
indexer: {
enabled: true,
provider: {
type: SearchProviderType.Manticoresearch,
endpoint: 'http://localhost:9308',

View File

@@ -6,6 +6,7 @@ import { omit, pick } from 'lodash-es';
import { createModule } from '../../../__tests__/create-module';
import { Mockers } from '../../../__tests__/mocks';
import { ConfigModule } from '../../../base/config';
import { ServerConfigModule } from '../../../core/config';
import { IndexerModule, IndexerService } from '..';
import { SearchProviderFactory } from '../factory';
@@ -20,7 +21,15 @@ import {
} from '../types';
const module = await createModule({
imports: [IndexerModule, ServerConfigModule],
imports: [
IndexerModule,
ServerConfigModule,
ConfigModule.override({
indexer: {
enabled: true,
},
}),
],
providers: [IndexerService],
});
const indexerService = module.get(IndexerService);

View File

@@ -30,7 +30,7 @@ declare global {
defineModuleConfig('indexer', {
enabled: {
desc: 'Enable indexer plugin',
default: true,
default: false,
env: ['AFFINE_INDEXER_ENABLED', 'boolean'],
},
'provider.type': {

View File

@@ -60,7 +60,7 @@ export class OIDCProvider extends OAuthProvider {
const validate = async () => {
this.#endpoints = null;
if (this.configured) {
if (super.configured) {
const config = this.config as OAuthOIDCProviderConfig;
try {
const res = await fetch(
@@ -73,7 +73,6 @@ export class OIDCProvider extends OAuthProvider {
if (res.ok) {
this.#endpoints = OIDCConfigurationSchema.parse(await res.json());
super.setup();
} else {
this.logger.error(`Invalid OIDC issuer ${config.issuer}`);
}
@@ -81,6 +80,8 @@ export class OIDCProvider extends OAuthProvider {
this.logger.error('Failed to validate OIDC configuration', e);
}
}
super.setup();
};
validate().catch(() => {

View File

@@ -262,12 +262,12 @@ enum CopilotModels {
Gpt4OmniMini0718
Gpt41
Gpt41Mini
Gpt41Nano
Gpt410414
GptImage
TextEmbedding3Large
TextEmbedding3Small
TextEmbeddingAda002
TextModerationLatest
TextModerationStable
}
input CopilotPromptConfigInput {

View File

@@ -356,12 +356,12 @@ export enum CopilotModels {
Gpt4OmniMini0718 = 'Gpt4OmniMini0718',
Gpt41 = 'Gpt41',
Gpt41Mini = 'Gpt41Mini',
Gpt41Nano = 'Gpt41Nano',
Gpt410414 = 'Gpt410414',
GptImage = 'GptImage',
TextEmbedding3Large = 'TextEmbedding3Large',
TextEmbedding3Small = 'TextEmbedding3Small',
TextEmbeddingAda002 = 'TextEmbeddingAda002',
TextModerationLatest = 'TextModerationLatest',
TextModerationStable = 'TextModerationStable',
}
export interface CopilotPromptConfigInput {

View File

@@ -57,7 +57,7 @@
"builder-util-runtime": "^9.2.10",
"cross-env": "^7.0.3",
"debug": "^4.4.0",
"electron": "^36.0.0",
"electron": "^35.0.0",
"electron-log": "^5.2.4",
"electron-squirrel-startup": "1.0.1",
"electron-window-state": "^5.0.3",

View File

@@ -208,7 +208,18 @@ export function actionToHandler<T extends keyof BlockSuitePresets.AIActions>(
if (!blocks || blocks.length === 0) return;
const block = blocks.at(-1);
if (!block) return;
aiPanel.toggle(block, '');
if (
blocks.length === 1 &&
block.model.flavour === 'affine:image' &&
id === 'createImage'
) {
// if only one image block is selected, and the action is createImage
// toggle panel to allow user to enter text prompt
aiPanel.toggle(block, 'input');
} else {
// generate the answer
aiPanel.toggle(block, 'generate');
}
};
}
@@ -232,7 +243,7 @@ export function handleInlineAskAIAction(
});
if (!actionGroups) {
panel.toggle(block);
panel.toggle(block, 'input');
return;
}
@@ -252,7 +263,7 @@ export function handleInlineAskAIAction(
clear();
};
panel.toggle(block);
panel.toggle(block, 'input');
setTimeout(() => {
abortController = new AbortController();

View File

@@ -457,7 +457,7 @@ export function actionToHandler<T extends keyof BlockSuitePresets.AIActions>(
togglePanel()
.then(isEmpty => {
aiPanel.toggle(referenceElement, isEmpty ? undefined : '', false);
aiPanel.toggle(referenceElement, isEmpty ? 'input' : 'generate');
})
.catch(console.error);
};

View File

@@ -333,20 +333,14 @@ export class AffineAIPanelWidget extends WidgetComponent {
}
};
toggle = (
reference: Element,
input?: string,
shouldTriggerCallback?: boolean
) => {
if (typeof input === 'string') {
this._inputText = input;
toggle = (reference: Element, type: 'input' | 'generate') => {
if (type === 'generate') {
this._inputText = '';
this.generate();
} else {
// reset state
this.hide(shouldTriggerCallback);
} else if (type === 'input') {
this.hide();
this.state = 'input';
}
this._autoUpdatePosition(reference);
};

View File

@@ -51,6 +51,7 @@ import { getSelectedModelsCommand } from '@blocksuite/affine/shared/commands';
import { ImageSelection } from '@blocksuite/affine/shared/selection';
import {
ActionPlacement,
CitationProvider,
GenerateDocUrlProvider,
isRemovedUserInfo,
OpenDocExtensionIdentifier,
@@ -462,6 +463,10 @@ function createExternalLinkableToolbarConfig(
(_std, _component, props) => {
ctx.store.updateBlock(model, props);
block.requestUpdate();
const citationService = ctx.std.get(CitationProvider);
if (citationService.isCitationModel(model)) {
citationService.trackEvent('Edit');
}
},
abortController
);
@@ -805,6 +810,10 @@ const embedLinkedDocToolbarConfig = {
(_std, _component, props) => {
ctx.store.updateBlock(model, props);
block.requestUpdate();
const citationService = ctx.std.get(CitationProvider);
if (citationService.isCitationModel(model)) {
citationService.trackEvent('Edit');
}
},
abortController
);

View File

@@ -21,6 +21,9 @@ export const menuTriggerStyle = style({
});
export const menuTriggerText = style({
margin: '0px 4px',
display: 'flex',
alignItems: 'center',
gap: '4px',
});
export const suffixClassName = style({
width: '20px',

View File

@@ -3,10 +3,11 @@ import {
LiveData,
Store,
yjsGetPath,
yjsObserveDeep,
yjsObservePath,
} from '@toeverything/infra';
import { nanoid } from 'nanoid';
import { map, Observable, switchMap } from 'rxjs';
import { map, switchMap } from 'rxjs';
import { Array as YArray } from 'yjs';
import type { WorkspaceService } from '../../workspace';
@@ -24,15 +25,22 @@ export class TagStore extends Store {
get properties() {
return this.workspaceService.workspace.docCollection.meta.properties;
}
get tagOptions() {
return this.properties.tags?.options ?? [];
}
tagOptions$ = LiveData.from(
new Observable<Tag[]>(sub => {
return this.subscribe(() => sub.next(this.tagOptions));
}),
this.tagOptions
yjsGetPath(
this.workspaceService.workspace.rootYDoc.getMap('meta'),
'properties.tags.options'
).pipe(
switchMap(yjsObserveDeep),
map(tagOptions => {
if (tagOptions instanceof YArray) {
return tagOptions.toJSON();
} else {
return [];
}
})
),
[]
);
subscribe(cb: () => void) {
@@ -82,12 +90,14 @@ export class TagStore extends Store {
};
updateTagOption = (id: string, option: Tag) => {
this.updateTagOptions(this.tagOptions.map(o => (o.id === id ? option : o)));
this.updateTagOptions(
this.tagOptions$.value.map(o => (o.id === id ? option : o))
);
};
removeTagOption = (id: string) => {
this.workspaceService.workspace.docCollection.doc.transact(() => {
this.updateTagOptions(this.tagOptions.filter(o => o.id !== id));
this.updateTagOptions(this.tagOptions$.value.filter(o => o.id !== id));
// need to remove tag from all pages
this.workspaceService.workspace.docCollection.docs.forEach(doc => {
const tags = doc.meta?.tags ?? [];

View File

@@ -551,9 +551,11 @@ export class EditorUtils {
explainImage: this.createAction(page, () =>
page.getByTestId('action-explain-image').click()
),
generateImage: this.createAction(page, () =>
page.getByTestId('action-generate-image').click()
),
generateImage: this.createAction(page, async () => {
await page.getByTestId('action-generate-image').click();
await page.keyboard.type('generate an image');
await page.getByTestId('ai-panel-input-send').click();
}),
generateCaption: this.createAction(page, () =>
page.getByTestId('action-generate-caption').click()
),

View File

@@ -573,7 +573,7 @@ __metadata:
builder-util-runtime: "npm:^9.2.10"
cross-env: "npm:^7.0.3"
debug: "npm:^4.4.0"
electron: "npm:^36.0.0"
electron: "npm:^35.0.0"
electron-log: "npm:^5.2.4"
electron-squirrel-startup: "npm:1.0.1"
electron-updater: "npm:^6.3.9"
@@ -776,7 +776,7 @@ __metadata:
"@vitest/coverage-istanbul": "npm:3.1.3"
"@vitest/ui": "npm:3.1.3"
cross-env: "npm:^7.0.3"
electron: "npm:^36.0.0"
electron: "npm:^35.0.0"
eslint: "npm:^9.16.0"
eslint-config-prettier: "npm:^10.0.0"
eslint-import-resolver-typescript: "npm:^4.0.0"
@@ -8968,14 +8968,14 @@ __metadata:
linkType: hard
"@nestjs-cls/transactional-adapter-prisma@npm:^1.2.19":
version: 1.2.21
resolution: "@nestjs-cls/transactional-adapter-prisma@npm:1.2.21"
version: 1.2.23
resolution: "@nestjs-cls/transactional-adapter-prisma@npm:1.2.23"
peerDependencies:
"@nestjs-cls/transactional": ^3.0.0
"@nestjs-cls/transactional": ^3.0.2
"@prisma/client": "> 4 < 7"
nestjs-cls: ^6.0.0
nestjs-cls: ^6.0.1
prisma: "> 4 < 7"
checksum: 10/3a0e6ed43ab97b6f94a859bcb9c71d79034f5c1ded1038204e374f170f81ea198ec93793c05b70d556342e4cec3d9fbd4e792f544f229572474bc10231b6a0ae
checksum: 10/98228a8fe132951095082b1657d73889ecbf564c4426bfc1b1daf6b0cdcfd29a9f243a77f1c9021edff5484e44664bc224b4337b6db3da188a6a969640a65a15
languageName: node
linkType: hard
@@ -19148,6 +19148,18 @@ __metadata:
languageName: node
linkType: hard
"concat-stream@npm:^2.0.0":
version: 2.0.0
resolution: "concat-stream@npm:2.0.0"
dependencies:
buffer-from: "npm:^1.0.0"
inherits: "npm:^2.0.3"
readable-stream: "npm:^3.0.2"
typedarray: "npm:^0.0.6"
checksum: 10/250e576d0617e7c58e1c4b2dd6fe69560f316d2c962a409f9f3aac794018499ddb31948b1e4296f217008e124cd5d526432097745157fe504b5d9f3dc469eadb
languageName: node
linkType: hard
"concordance@npm:^5.0.4":
version: 5.0.4
resolution: "concordance@npm:5.0.4"
@@ -20595,16 +20607,16 @@ __metadata:
languageName: node
linkType: hard
"electron@npm:^36.0.0":
version: 36.2.1
resolution: "electron@npm:36.2.1"
"electron@npm:^35.0.0":
version: 35.5.1
resolution: "electron@npm:35.5.1"
dependencies:
"@electron/get": "npm:^2.0.0"
"@types/node": "npm:^22.7.7"
extract-zip: "npm:^2.0.1"
bin:
electron: cli.js
checksum: 10/2dbcac9208c6a513b75ef92a969441407e44fb11377a3d831d3e043d3c5e1bf927badc475a490b554e6007995f1a28db9d833044202d21d629bb84587745dbf7
checksum: 10/765752c94c3bea799f62fb0a75049a10452ff97e791a96111f36b97293ffed9240b27dbc083c01f7a2c7172e84b6b394b07c9aaf91d57783958b2adb4e570579
languageName: node
linkType: hard
@@ -27077,7 +27089,7 @@ __metadata:
languageName: node
linkType: hard
"mkdirp@npm:^0.5.1, mkdirp@npm:^0.5.4":
"mkdirp@npm:^0.5.1, mkdirp@npm:^0.5.4, mkdirp@npm:^0.5.6":
version: 0.5.6
resolution: "mkdirp@npm:0.5.6"
dependencies:
@@ -27253,7 +27265,7 @@ __metadata:
languageName: node
linkType: hard
"multer@npm:2.0.0, multer@npm:^2.0.0":
"multer@npm:2.0.0":
version: 2.0.0
resolution: "multer@npm:2.0.0"
dependencies:
@@ -27268,6 +27280,21 @@ __metadata:
languageName: node
linkType: hard
"multer@npm:^2.0.0":
version: 2.0.1
resolution: "multer@npm:2.0.1"
dependencies:
append-field: "npm:^1.0.0"
busboy: "npm:^1.6.0"
concat-stream: "npm:^2.0.0"
mkdirp: "npm:^0.5.6"
object-assign: "npm:^4.1.1"
type-is: "npm:^1.6.18"
xtend: "npm:^4.0.2"
checksum: 10/cb0dda65ae37be40968fc1f9ea492bdb4c20bd189ce427e11e95d333837193544606b82ef6431f2acd3cd11156164f215bdeb46f47847d29b6bf3a36ac736a8f
languageName: node
linkType: hard
"multicast-dns@npm:^7.2.5":
version: 7.2.5
resolution: "multicast-dns@npm:7.2.5"
@@ -30208,7 +30235,7 @@ __metadata:
languageName: node
linkType: hard
"readable-stream@npm:3, readable-stream@npm:^3.0.6, readable-stream@npm:^3.4.0, readable-stream@npm:^3.6.2":
"readable-stream@npm:3, readable-stream@npm:^3.0.2, readable-stream@npm:^3.0.6, readable-stream@npm:^3.4.0, readable-stream@npm:^3.6.2":
version: 3.6.2
resolution: "readable-stream@npm:3.6.2"
dependencies:
@@ -33358,7 +33385,7 @@ __metadata:
languageName: node
linkType: hard
"type-is@npm:^1.6.4, type-is@npm:~1.6.18":
"type-is@npm:^1.6.18, type-is@npm:^1.6.4, type-is@npm:~1.6.18":
version: 1.6.18
resolution: "type-is@npm:1.6.18"
dependencies:
@@ -34902,7 +34929,7 @@ __metadata:
languageName: node
linkType: hard
"xtend@npm:^4.0.0":
"xtend@npm:^4.0.0, xtend@npm:^4.0.2":
version: 4.0.2
resolution: "xtend@npm:4.0.2"
checksum: 10/ac5dfa738b21f6e7f0dd6e65e1b3155036d68104e67e5d5d1bde74892e327d7e5636a076f625599dc394330a731861e87343ff184b0047fef1360a7ec0a5a36a