Compare commits

...

17 Commits

Author SHA1 Message Date
DarkSky
2414aa5848 feat: improve admin build (#14485)
#### PR Dependency Tree


* **PR #14485** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
  * Admin static assets now served under /admin for self-hosted installs
  * CLI is directly executable from the command line
  * Build tooling supports a configurable self-hosted public path
  * Updated admin package script for adding UI components
* Added a PostCSS dependency and plugin to the build toolchain for admin
builds

* **Style**
* Switched queue module to a local queuedash stylesheet, added queuedash
Tailwind layer, and scoped queuedash styles for the admin UI

* **Bug Fixes**
  * Improved error propagation in the Electron renderer
* Migration compatibility to repair a legacy checksum during native
storage upgrades

* **Tests**
  * Added tests covering the migration repair flow
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-02-21 23:25:05 +08:00
DarkSky
0de1bd0da8 feat: bump deps (#14484) 2026-02-21 08:04:18 +08:00
DarkSky
186ec5431d fix: android build 2026-02-21 05:12:12 +08:00
DarkSky
da57bfe8e7 fix: enhance MCP token handling (#14483)
fix #14475 

#### PR Dependency Tree


* **PR #14483** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

## Release Notes

* **New Features**
* Enhanced MCP server token management with improved security—tokens now
display only once with redaction support.
* Updated token creation and deletion workflows with clearer UI state
controls.
* Added tooltip guidance when copying configuration with redacted
tokens.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-02-21 04:14:14 +08:00
DarkSky
c9bffc13b5 feat: improve mobile native impl (#14481)
fix #13529 

#### PR Dependency Tree


* **PR #14481** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Mobile blob caching with file-backed storage for faster loads and
reduced network usage
* Blob decoding with lazy refresh on token-read failures for improved
reliability
  * Full-text search/indexing exposed to mobile apps
* Document sync APIs and peer clock management for robust cross-device
sync

* **Tests**
* Added unit tests covering payload decoding, cache safety, and
concurrency

* **Dependencies**
* Added an LRU cache dependency and a new mobile-shared package for
shared mobile logic
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-02-21 04:13:24 +08:00
DarkSky
d8cc0acdd0 chore: update flags 2026-02-19 10:18:43 +08:00
Neo
35e1411407 fix: docTitle unexpectedly translated (#14467)
fix #14465

In Chinese mode, the document with the specified name may not be
displayed correctly in the sidebar, and it may be mistaken for the
translation of the content that needs to be translated.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Fixed document title display in navigation panels on desktop and
mobile to properly render without additional processing steps.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-02-17 13:45:31 +00:00
DarkSky
8f833388eb feat: improve admin panel design (#14464) 2026-02-17 17:40:29 +08:00
DarkSky
850e646ab9 fix: electon rendering on windows (#14456)
fix #14450
fix #14401
fix #13983
fix #12766
fix #14404
fix #12019

#### PR Dependency Tree


* **PR #14456** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added new tab navigation functions: `switchTab`, `switchToNextTab`,
and `switchToPreviousTab`.

* **Bug Fixes**
  * Improved bounds validation for tab view resizing.
  * Enhanced tab lifecycle management during navigation events.
  * Refined background throttling behavior for active tabs.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-02-16 14:08:26 +08:00
DarkSky
728e02cab7 feat: bump eslint & oxlint (#14452)
#### PR Dependency Tree


* **PR #14452** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved null-safety, dependency tracking, upload validation, and
error logging for more reliable uploads, clipboard, calendar linking,
telemetry, PDF/theme printing, and preview/zoom behavior.
* Tightened handling of all-day calendar events (missing date now
reported).

* **Deprecations**
  * Removed deprecated RadioButton and RadioButtonGroup; use RadioGroup.

* **Chores**
* Unified and upgraded linting/config, reorganized imports, and
standardized binary handling for more consistent builds and tooling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-02-16 13:52:08 +08:00
DarkSky
792164edd1 fix: sign 2026-02-16 12:23:26 +08:00
DarkSky
e3177e6837 feat: normalize search text (#14449)
#### PR Dependency Tree


* **PR #14449** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Improvements**
* Search text normalization now applied consistently across doc titles,
search results, and highlights for uniform display formatting.

* **Tests**
* Added comprehensive test coverage for search text normalization
utility.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-02-16 08:07:04 +08:00
DarkSky
42f2d2b337 feat: support markdown preview (#14447) 2026-02-15 21:05:52 +08:00
DarkSky
9d7f4acaf1 fix: s3 upload compatibility (#14445)
fix #14432 

#### PR Dependency Tree


* **PR #14445** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Refactor**
* Improved file upload handling to ensure consistent support for
different data formats during object and multipart uploads.
* Enhanced type safety throughout storage and workflow components by
removing unnecessary type assertions.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-02-15 19:16:36 +08:00
DarkSky
9a1f600fc9 chore: update i18n status 2026-02-15 14:59:52 +08:00
steffenrapp
0f906ad623 feat(i18n): update German translation (#14444)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Chat panel: session management, history loading, embedding progress,
and deletion flow
* Document analytics: views, unique visitors, guest metrics, charts,
viewers and paywall messaging
* Calendar integration: expanded account/provider states, errors and
flow copy; DOCX import tooltip
  * Appearance: image antialiasing option and window-behavior toggles
  * Workspace sharing: visibility controls and related tooltips

* **Improvements**
* Expanded error and empty-state wording, subscription/payment
description, and experimental feature labels
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-02-15 14:57:47 +08:00
DarkSky
09aa65c52a feat: improve ci 2026-02-15 14:53:35 +08:00
313 changed files with 9621 additions and 6048 deletions

View File

@@ -222,7 +222,7 @@
}, },
"SMTP.sender": { "SMTP.sender": {
"type": "string", "type": "string",
"description": "Sender of all the emails (e.g. \"AFFiNE Self Hosted <noreply@example.com>\")\n@default \"AFFiNE Self Hosted <noreply@example.com>\"\n@environment `MAILER_SENDER`", "description": "Sender of all the emails (e.g. \"AFFiNE Self Hosted &lt;noreply@example.com&gt;\")\n@default \"AFFiNE Self Hosted <noreply@example.com>\"\n@environment `MAILER_SENDER`",
"default": "AFFiNE Self Hosted <noreply@example.com>" "default": "AFFiNE Self Hosted <noreply@example.com>"
}, },
"SMTP.ignoreTLS": { "SMTP.ignoreTLS": {
@@ -262,7 +262,7 @@
}, },
"fallbackSMTP.sender": { "fallbackSMTP.sender": {
"type": "string", "type": "string",
"description": "Sender of all the emails (e.g. \"AFFiNE Self Hosted <noreply@example.com>\")\n@default \"\"", "description": "Sender of all the emails (e.g. \"AFFiNE Self Hosted &lt;noreply@example.com&gt;\")\n@default \"\"",
"default": "" "default": ""
}, },
"fallbackSMTP.ignoreTLS": { "fallbackSMTP.ignoreTLS": {

View File

@@ -201,13 +201,44 @@ jobs:
nmHoistingLimits: workspaces nmHoistingLimits: workspaces
env: env:
npm_config_arch: ${{ matrix.spec.arch }} npm_config_arch: ${{ matrix.spec.arch }}
- name: Download and overwrite packaged artifacts - name: Download packaged artifacts
uses: actions/download-artifact@v4
with:
name: packaged-${{ matrix.spec.platform }}-${{ matrix.spec.arch }}
path: packaged-unsigned
- name: unzip packaged artifacts
run: Expand-Archive -Path packaged-unsigned/archive.zip -DestinationPath packages/frontend/apps/electron/out
- name: Download signed packaged file diff
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
name: signed-packaged-${{ matrix.spec.platform }}-${{ matrix.spec.arch }} name: signed-packaged-${{ matrix.spec.platform }}-${{ matrix.spec.arch }}
path: . path: signed-packaged-diff
- name: unzip file - name: Apply signed packaged file diff
run: Expand-Archive -Path signed.zip -DestinationPath packages/frontend/apps/electron/out shell: pwsh
run: |
$DiffRoot = 'signed-packaged-diff/files'
$TargetRoot = 'packages/frontend/apps/electron/out'
if (!(Test-Path -LiteralPath $DiffRoot)) {
throw "Signed diff directory not found: $DiffRoot"
}
Copy-Item -Path (Join-Path $DiffRoot '*') -Destination $TargetRoot -Recurse -Force
$ManifestPath = 'signed-packaged-diff/manifest.json'
if (Test-Path -LiteralPath $ManifestPath) {
$ManifestEntries = @(Get-Content -LiteralPath $ManifestPath | ConvertFrom-Json)
foreach ($Entry in $ManifestEntries) {
$TargetPath = Join-Path $TargetRoot $Entry.path
if (!(Test-Path -LiteralPath $TargetPath -PathType Leaf)) {
throw "Applied signed file not found: $($Entry.path)"
}
$TargetHash = (Get-FileHash -Algorithm SHA256 -LiteralPath $TargetPath).Hash
if ($TargetHash -ne $Entry.sha256) {
throw "Signed file hash mismatch: $($Entry.path)"
}
}
}
- name: Make squirrel.windows installer - name: Make squirrel.windows installer
run: yarn affine @affine/electron make-squirrel --platform=${{ matrix.spec.platform }} --arch=${{ matrix.spec.arch }} run: yarn affine @affine/electron make-squirrel --platform=${{ matrix.spec.platform }} --arch=${{ matrix.spec.arch }}
@@ -267,13 +298,44 @@ jobs:
arch: arm64 arch: arm64
runs-on: ${{ matrix.spec.runner }} runs-on: ${{ matrix.spec.runner }}
steps: steps:
- name: Download and overwrite installer artifacts - name: Download installer artifacts
uses: actions/download-artifact@v4
with:
name: installer-${{ matrix.spec.platform }}-${{ matrix.spec.arch }}
path: installer-unsigned
- name: unzip installer artifacts
run: Expand-Archive -Path installer-unsigned/archive.zip -DestinationPath packages/frontend/apps/electron/out/${{ env.BUILD_TYPE }}/make
- name: Download signed installer file diff
uses: actions/download-artifact@v4 uses: actions/download-artifact@v4
with: with:
name: signed-installer-${{ matrix.spec.platform }}-${{ matrix.spec.arch }} name: signed-installer-${{ matrix.spec.platform }}-${{ matrix.spec.arch }}
path: . path: signed-installer-diff
- name: unzip file - name: Apply signed installer file diff
run: Expand-Archive -Path signed.zip -DestinationPath packages/frontend/apps/electron/out/${{ env.BUILD_TYPE }}/make shell: pwsh
run: |
$DiffRoot = 'signed-installer-diff/files'
$TargetRoot = 'packages/frontend/apps/electron/out/${{ env.BUILD_TYPE }}/make'
if (!(Test-Path -LiteralPath $DiffRoot)) {
throw "Signed diff directory not found: $DiffRoot"
}
Copy-Item -Path (Join-Path $DiffRoot '*') -Destination $TargetRoot -Recurse -Force
$ManifestPath = 'signed-installer-diff/manifest.json'
if (Test-Path -LiteralPath $ManifestPath) {
$ManifestEntries = @(Get-Content -LiteralPath $ManifestPath | ConvertFrom-Json)
foreach ($Entry in $ManifestEntries) {
$TargetPath = Join-Path $TargetRoot $Entry.path
if (!(Test-Path -LiteralPath $TargetPath -PathType Leaf)) {
throw "Applied signed file not found: $($Entry.path)"
}
$TargetHash = (Get-FileHash -Algorithm SHA256 -LiteralPath $TargetPath).Hash
if ($TargetHash -ne $Entry.sha256) {
throw "Signed file hash mismatch: $($Entry.path)"
}
}
}
- name: Save artifacts - name: Save artifacts
run: | run: |

View File

@@ -148,7 +148,7 @@ jobs:
name: Wait for approval name: Wait for approval
with: with:
secret: ${{ secrets.GITHUB_TOKEN }} secret: ${{ secrets.GITHUB_TOKEN }}
approvers: darkskygit,pengx17,L-Sun,EYHN approvers: darkskygit
minimum-approvals: 1 minimum-approvals: 1
fail-on-denial: true fail-on-denial: true
issue-title: Please confirm to release docker image issue-title: Please confirm to release docker image

View File

@@ -30,13 +30,43 @@ jobs:
run: | run: |
cd ${{ env.ARCHIVE_DIR }}/out cd ${{ env.ARCHIVE_DIR }}/out
signtool sign /tr http://timestamp.globalsign.com/tsa/r6advanced1 /td sha256 /fd sha256 /a ${{ inputs.files }} signtool sign /tr http://timestamp.globalsign.com/tsa/r6advanced1 /td sha256 /fd sha256 /a ${{ inputs.files }}
- name: zip file - name: collect signed file diff
shell: cmd shell: powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -File {0}
run: | run: |
cd ${{ env.ARCHIVE_DIR }} $OutDir = Join-Path '${{ env.ARCHIVE_DIR }}' 'out'
7za a signed.zip .\out\* $DiffDir = Join-Path '${{ env.ARCHIVE_DIR }}' 'signed-diff'
$FilesDir = Join-Path $DiffDir 'files'
New-Item -ItemType Directory -Path $FilesDir -Force | Out-Null
$SignedFiles = [regex]::Matches('${{ inputs.files }}', '"([^"]+)"') | ForEach-Object { $_.Groups[1].Value }
if ($SignedFiles.Count -eq 0) {
throw 'No files to sign were provided.'
}
$Manifest = @()
foreach ($RelativePath in $SignedFiles) {
$SourcePath = Join-Path $OutDir $RelativePath
if (!(Test-Path -LiteralPath $SourcePath -PathType Leaf)) {
throw "Signed file not found: $RelativePath"
}
$TargetPath = Join-Path $FilesDir $RelativePath
$TargetDir = Split-Path -Parent $TargetPath
if ($TargetDir) {
New-Item -ItemType Directory -Path $TargetDir -Force | Out-Null
}
Copy-Item -LiteralPath $SourcePath -Destination $TargetPath -Force
$Manifest += [PSCustomObject]@{
path = $RelativePath
sha256 = (Get-FileHash -Algorithm SHA256 -LiteralPath $TargetPath).Hash
}
}
$Manifest | ConvertTo-Json -Depth 4 | Out-File -FilePath (Join-Path $DiffDir 'manifest.json') -Encoding utf8
Write-Host "Collected $($SignedFiles.Count) signed files."
- name: upload - name: upload
uses: actions/upload-artifact@v4 uses: actions/upload-artifact@v4
with: with:
name: signed-${{ inputs.artifact-name }} name: signed-${{ inputs.artifact-name }}
path: ${{ env.ARCHIVE_DIR }}/signed.zip path: ${{ env.ARCHIVE_DIR }}/signed-diff

View File

@@ -5,6 +5,10 @@
"correctness": "error", "correctness": "error",
"perf": "error" "perf": "error"
}, },
"env": {
"builtin": true,
"es2026": true
},
"ignorePatterns": [ "ignorePatterns": [
"**/node_modules", "**/node_modules",
".yarn", ".yarn",
@@ -44,6 +48,34 @@
"**/test-blocks.json" "**/test-blocks.json"
], ],
"rules": { "rules": {
"no-empty-static-block": "error",
"no-misleading-character-class": "error",
"no-new-native-nonconstructor": "error",
"no-unused-private-class-members": "error",
"no-useless-backreference": "error",
"react/display-name": "error",
"react/rules-of-hooks": "error",
"react/exhaustive-deps": "warn",
"@typescript-eslint/prefer-for-of": "error",
"@typescript-eslint/no-unsafe-function-type": "error",
"@typescript-eslint/no-wrapper-object-types": "error",
"no-restricted-imports": [
"error",
{
"patterns": [
{
"group": ["**/dist"],
"message": "Don't import from dist",
"allowTypeImports": false
},
{
"group": ["**/src"],
"message": "Don't import from src",
"allowTypeImports": false
}
]
}
],
"no-await-in-loop": "allow", "no-await-in-loop": "allow",
"no-redeclare": "allow", "no-redeclare": "allow",
"promise/no-callback-in-promise": "allow", "promise/no-callback-in-promise": "allow",
@@ -70,6 +102,14 @@
"no-func-assign": "error", "no-func-assign": "error",
"no-global-assign": "error", "no-global-assign": "error",
"no-unused-vars": "error", "no-unused-vars": "error",
"no-unused-expressions": [
"error",
{
"allowShortCircuit": true,
"allowTernary": true,
"allowTaggedTemplates": true
}
],
"no-ex-assign": "error", "no-ex-assign": "error",
"no-loss-of-precision": "error", "no-loss-of-precision": "error",
"no-fallthrough": "error", "no-fallthrough": "error",
@@ -126,6 +166,7 @@
"react/no-render-return-value": "error", "react/no-render-return-value": "error",
"react/jsx-no-target-blank": "error", "react/jsx-no-target-blank": "error",
"react/jsx-no-comment-textnodes": "error", "react/jsx-no-comment-textnodes": "error",
"react/no-array-index-key": "off",
"typescript/consistent-type-imports": "error", "typescript/consistent-type-imports": "error",
"typescript/no-non-null-assertion": "error", "typescript/no-non-null-assertion": "error",
"typescript/triple-slash-reference": "error", "typescript/triple-slash-reference": "error",
@@ -241,6 +282,42 @@
"typescript/consistent-type-imports": "off", "typescript/consistent-type-imports": "off",
"import/no-cycle": "off" "import/no-cycle": "off"
} }
},
{
"files": [
"packages/**/*.{ts,tsx}",
"tools/**/*.{ts,tsx}",
"blocksuite/**/*.{ts,tsx}"
],
"rules": {
"react/exhaustive-deps": [
"warn",
{
"additionalHooks": "(useAsyncCallback|useCatchEventCallback|useDraggable|useDropTarget|useRefEffect)"
}
]
}
},
{
"files": [
"**/__tests__/**/*",
"**/*.stories.tsx",
"**/*.spec.ts",
"**/tests/**/*",
"scripts/**/*",
"**/benchmark/**/*",
"**/__debug__/**/*",
"**/e2e/**/*"
],
"rules": {
"no-restricted-imports": "off"
}
},
{
"files": ["**/*.{ts,js,mjs}"],
"rules": {
"react/rules-of-hooks": "off"
}
} }
] ]
} }

View File

@@ -17,7 +17,7 @@
"explorer.fileNesting.enabled": true, "explorer.fileNesting.enabled": true,
"explorer.fileNesting.patterns": { "explorer.fileNesting.patterns": {
"*.js": "${capture}.js.map, ${capture}.min.js, ${capture}.d.ts, ${capture}.d.ts.map", "*.js": "${capture}.js.map, ${capture}.min.js, ${capture}.d.ts, ${capture}.d.ts.map",
"package.json": ".browserslist*, .circleci*, .codecov, .commitlint*, .cz-config.js, .czrc, .dlint.json, .dprint.json, .editorconfig, .eslint*, eslint.*, .firebase*, .flowconfig, .github*, .gitlab*, .gitpod*, .huskyrc*, .jslint*, .lighthouserc.*, .lintstagedrc*, .markdownlint*, .mocha*, .node-version, .nodemon*, .npm*, .nvmrc, .pm2*, .pnp.*, .pnpm*, .prettier*, .releaserc*, .sentry*, .stackblitz*, .styleci*, .stylelint*, .tazerc*, .textlint*, .tool-versions, .travis*, .versionrc*, .vscode*, .watchman*, .xo-config*, .yamllint*, .yarnrc*, Procfile, api-extractor.json, apollo.config.*, appveyor*, ava.config.*, azure-pipelines*, bower.json, build.config.*, commitlint*, dangerfile*, dlint.json, dprint.json, firebase.json, grunt*, gulp*, histoire.config.*, jasmine.*, jenkins*, jest.config.*, jsconfig.*, karma*, lerna*, lighthouserc.*, lint-staged*, nest-cli.*, netlify*, nodemon*, nx.*, package-lock.json, package.nls*.json, phpcs.xml, playwright.config.*, pm2.*, pnpm*, prettier*, pullapprove*, puppeteer.config.*, pyrightconfig.json, release-tasks.sh, renovate*, rollup.config.*, stylelint*, tsconfig.*, tsdoc.*, tslint*, tsup.config.*, turbo*, typedoc*, unlighthouse*, vercel*, vetur.config.*, vitest.*, webpack*, workspace.json, xo.config.*, yarn*, babel.*, .babelrc, project.json, oxlint.json, nyc.config.*", "package.json": ".browserslist*, .circleci*, .codecov, .commitlint*, .cz-config.js, .czrc, .dlint.json, .dprint.json, .editorconfig, .eslint*, eslint.*, .firebase*, .flowconfig, .github*, .gitlab*, .gitpod*, .huskyrc*, .jslint*, .lighthouserc.*, .lintstagedrc*, .markdownlint*, .mocha*, .node-version, .nodemon*, .npm*, .nvmrc, .pm2*, .pnp.*, .pnpm*, .prettier*, .releaserc*, .sentry*, .stackblitz*, .styleci*, .stylelint*, .tazerc*, .textlint*, .tool-versions, .travis*, .versionrc*, .vscode*, .watchman*, .xo-config*, .yamllint*, .yarnrc*, Procfile, api-extractor.json, apollo.config.*, appveyor*, ava.config.*, azure-pipelines*, bower.json, build.config.*, commitlint*, dangerfile*, dlint.json, dprint.json, firebase.json, grunt*, gulp*, histoire.config.*, jasmine.*, jenkins*, jest.config.*, jsconfig.*, karma*, lerna*, lighthouserc.*, lint-staged*, nest-cli.*, netlify*, nodemon*, nx.*, package-lock.json, package.nls*.json, phpcs.xml, playwright.config.*, pm2.*, pnpm*, prettier*, pullapprove*, puppeteer.config.*, pyrightconfig.json, release-tasks.sh, renovate*, rollup.config.*, stylelint*, tsconfig.*, tsdoc.*, tslint*, tsup.config.*, turbo*, typedoc*, unlighthouse*, vercel*, vetur.config.*, vitest.*, webpack*, workspace.json, xo.config.*, yarn*, babel.*, .babelrc, project.json, .oxlintrc.json, oxlint.json, nyc.config.*",
"Cargo.toml": "Cargo.lock, rust-toolchain*, rustfmt.toml, .taplo.toml", "Cargo.toml": "Cargo.lock, rust-toolchain*, rustfmt.toml, .taplo.toml",
"README.md": "LICENSE*, CHANGELOG.md, CODE_OF_CONDUCT.md, CONTRIBUTING.md, SECURITY.md, README.*", "README.md": "LICENSE*, CHANGELOG.md, CODE_OF_CONDUCT.md, CONTRIBUTING.md, SECURITY.md, README.*",
".gitignore": ".gitattributes, .dockerignore, .eslintignore, .prettierignore, .stylelintignore, .tslintignore, .yarnignore" ".gitignore": ".gitattributes, .dockerignore, .eslintignore, .prettierignore, .stylelintignore, .tslintignore, .yarnignore"

11
Cargo.lock generated
View File

@@ -111,10 +111,12 @@ dependencies = [
"base64-simd", "base64-simd",
"chrono", "chrono",
"homedir", "homedir",
"lru",
"objc2", "objc2",
"objc2-foundation", "objc2-foundation",
"sqlx", "sqlx",
"thiserror 2.0.17", "thiserror 2.0.17",
"tokio",
"uniffi", "uniffi",
] ]
@@ -2572,6 +2574,15 @@ dependencies = [
"weezl", "weezl",
] ]
[[package]]
name = "lru"
version = "0.16.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a1dc47f592c06f33f8e3aea9591776ec7c9f9e4124778ff8a3c3b87159f7e593"
dependencies = [
"hashbrown 0.16.1",
]
[[package]] [[package]]
name = "mac" name = "mac"
version = "0.1.1" version = "0.1.1"

View File

@@ -46,6 +46,7 @@ resolver = "3"
libc = "0.2" libc = "0.2"
log = "0.4" log = "0.4"
loom = { version = "0.7", features = ["checkpoint"] } loom = { version = "0.7", features = ["checkpoint"] }
lru = "0.16"
memory-indexer = "0.3.0" memory-indexer = "0.3.0"
mimalloc = "0.1" mimalloc = "0.1"
mp4parse = "0.17" mp4parse = "0.17"

View File

@@ -67,7 +67,7 @@ export const autoScrollOnBoundary = (
}; };
const cancelBoxListen = effect(() => { const cancelBoxListen = effect(() => {
box.value; void box.value;
startUpdate(); startUpdate();
}); });

View File

@@ -24,12 +24,12 @@ import {
DataViewUIBase, DataViewUIBase,
DataViewUILogicBase, DataViewUILogicBase,
} from '../../../core/view/data-view-base.js'; } from '../../../core/view/data-view-base.js';
import { LEFT_TOOL_BAR_WIDTH } from '../consts.js';
import { import {
type TableSingleView,
TableViewRowSelection, TableViewRowSelection,
type TableViewSelectionWithType, type TableViewSelectionWithType,
} from '../../index.js'; } from '../selection.js';
import { LEFT_TOOL_BAR_WIDTH } from '../consts.js'; import type { TableSingleView } from '../table-view-manager.js';
import { TableClipboardController } from './controller/clipboard.js'; import { TableClipboardController } from './controller/clipboard.js';
import { TableDragController } from './controller/drag.js'; import { TableDragController } from './controller/drag.js';
import { TableHotkeysController } from './controller/hotkeys.js'; import { TableHotkeysController } from './controller/hotkeys.js';

View File

@@ -60,10 +60,9 @@ export class BaseExtensionProvider<
* @param context - The context object containing scope and registration function * @param context - The context object containing scope and registration function
* @param option - Optional configuration options for the provider * @param option - Optional configuration options for the provider
*/ */
setup(context: Context<Scope>, option?: Options) { setup(_context: Context<Scope>, option?: Options) {
if (option) { if (option) {
this.schema.parse(option); this.schema.parse(option);
} }
context;
} }
} }

View File

@@ -884,7 +884,7 @@ export class ConnectionOverlay extends Overlay {
private _setupThemeListener(): void { private _setupThemeListener(): void {
const themeService = this.gfx.std.get(ThemeProvider); const themeService = this.gfx.std.get(ThemeProvider);
this._themeDisposer = effect(() => { this._themeDisposer = effect(() => {
themeService.theme$; void themeService.theme$.value;
this._emphasisColor = this._getEmphasisColor(); this._emphasisColor = this._getEmphasisColor();
}); });
} }

View File

@@ -1,3 +1,4 @@
/* eslint-disable @typescript-eslint/await-thenable */
import type { import type {
Template, Template,
TemplateCategory, TemplateCategory,

View File

@@ -9,7 +9,7 @@ import rehypeParse from 'rehype-parse';
import { unified } from 'unified'; import { unified } from 'unified';
import type { AffineTextAttributes } from '../../types/index.js'; import type { AffineTextAttributes } from '../../types/index.js';
import { HtmlDeltaConverter } from '../html/delta-converter.js'; import type { HtmlDeltaConverter } from '../html/delta-converter.js';
import { import {
rehypeInlineToBlock, rehypeInlineToBlock,
rehypeWrapInlineElements, rehypeWrapInlineElements,

View File

@@ -873,7 +873,7 @@ export class PdfAdapter extends BaseAdapter<PdfAdapterFile> {
return { return {
table: { table: {
headerRows: 0, headerRows: 0,
widths: Array(sortedColumns.length).fill('*'), widths: Array.from({ length: sortedColumns.length }, () => '*'),
body: tableBody, body: tableBody,
}, },
margin: [0, 5, 0, 5], margin: [0, 5, 0, 5],

View File

@@ -115,12 +115,9 @@ export async function printToPdf(
) as HTMLDivElement; ) as HTMLDivElement;
// force light theme in print iframe // force light theme in print iframe
iframe.contentWindow.document.documentElement.setAttribute( iframe.contentWindow.document.documentElement.dataset.theme = 'light';
'data-theme', iframe.contentWindow.document.body.dataset.theme = 'light';
'light' importedRoot.dataset.theme = 'light';
);
iframe.contentWindow.document.body.setAttribute('data-theme', 'light');
importedRoot.setAttribute('data-theme', 'light');
// draw saved canvas image to canvas // draw saved canvas image to canvas
const allImportedCanvas = importedRoot.getElementsByTagName('canvas'); const allImportedCanvas = importedRoot.getElementsByTagName('canvas');

View File

@@ -126,7 +126,7 @@ export class EdgelessZoomToolbar extends WithDisposable(LitElement) {
this.disposables.add( this.disposables.add(
effect(() => { effect(() => {
this.gfx.tool.currentToolName$.value; void this.gfx.tool.currentToolName$.value;
this.requestUpdate(); this.requestUpdate();
}) })
); );

View File

@@ -289,7 +289,7 @@ export class AffineKeyboardToolbar extends SignalWatcher(
this.disposables.add( this.disposables.add(
effect(() => { effect(() => {
const std = this.rootComponent.std; const std = this.rootComponent.std;
std.selection.value; void std.selection.value;
// wait cursor updated // wait cursor updated
requestAnimationFrame(() => { requestAnimationFrame(() => {
this._scrollCurrentBlockIntoView(); this._scrollCurrentBlockIntoView();

View File

@@ -1,5 +1,5 @@
import type { ExtensionType, Schema, Workspace } from '@blocksuite/store'; import type { ExtensionType, Schema, Workspace } from '@blocksuite/store';
// @ts-ignore // @ts-expect-error -- mammoth.browser has no compatible type declaration for this subpath.
import { convertToHtml } from 'mammoth/mammoth.browser'; import { convertToHtml } from 'mammoth/mammoth.browser';
import { HtmlTransformer } from './html'; import { HtmlTransformer } from './html';

View File

@@ -10,12 +10,12 @@ import { Container } from '@blocksuite/global/di';
import { BlockSuiteError, ErrorCode } from '@blocksuite/global/exceptions'; import { BlockSuiteError, ErrorCode } from '@blocksuite/global/exceptions';
import { sha } from '@blocksuite/global/utils'; import { sha } from '@blocksuite/global/utils';
import type { import type {
DocMeta,
ExtensionType, ExtensionType,
Schema, Schema,
Store, Store,
Workspace, Workspace,
} from '@blocksuite/store'; } from '@blocksuite/store';
import type { DocMeta } from '@blocksuite/store';
import { extMimeMap, Transformer } from '@blocksuite/store'; import { extMimeMap, Transformer } from '@blocksuite/store';
import type { AssetMap, ImportedFileEntry, PathBlobIdMap } from './type.js'; import type { AssetMap, ImportedFileEntry, PathBlobIdMap } from './type.js';

View File

@@ -171,9 +171,11 @@ export class Unzip {
const fileExt = const fileExt =
fileName.lastIndexOf('.') === -1 ? '' : fileName.split('.').at(-1); fileName.lastIndexOf('.') === -1 ? '' : fileName.split('.').at(-1);
const mime = extMimeMap.get(fileExt ?? ''); const mime = extMimeMap.get(fileExt ?? '');
const content = new File([this.unzipped![path]], fileName, { const content = new File(
type: mime ?? '', [new Uint8Array(this.unzipped![path]).buffer],
}) as Blob; fileName,
mime ? { type: mime } : undefined
) as Blob;
const fixedPath = this.fixFileNameEncoding(path); const fixedPath = this.fixFileNameEncoding(path);

View File

@@ -27,10 +27,10 @@ async function exportDocs(
titleMiddleware(collection.meta.docMetas), titleMiddleware(collection.meta.docMetas),
], ],
}); });
const snapshots = await Promise.all(docs.map(job.docToSnapshot));
await Promise.all( await Promise.all(
snapshots docs
.map(job.docToSnapshot)
.filter((snapshot): snapshot is DocSnapshot => !!snapshot) .filter((snapshot): snapshot is DocSnapshot => !!snapshot)
.map(async snapshot => { .map(async snapshot => {
// Use the title and id as the snapshot file name // Use the title and id as the snapshot file name

View File

@@ -190,7 +190,7 @@ export class Clipboard extends LifeCycleWatcher {
); );
} }
return slice; return slice;
} catch (error) { } catch {
const getDataByType = this._getDataByType(data); const getDataByType = this._getDataByType(data);
const slice = await this._getSnapshotByPriority( const slice = await this._getSnapshotByPriority(
type => getDataByType(type), type => getDataByType(type),

View File

@@ -1,5 +1,5 @@
import { LifeCycleWatcher } from '../extension/index.js';
import { BlockServiceIdentifier } from '../identifier.js'; import { BlockServiceIdentifier } from '../identifier.js';
import { LifeCycleWatcher } from './lifecycle-watcher.js';
export class ServiceManager extends LifeCycleWatcher { export class ServiceManager extends LifeCycleWatcher {
static override readonly key = 'serviceManager'; static override readonly key = 'serviceManager';

View File

@@ -87,6 +87,7 @@ export function batchRemoveChildren(
} }
uniqueElements.forEach(element => { uniqueElements.forEach(element => {
// oxlint-disable-next-line unicorn/prefer-dom-node-remove
container.removeChild(element); container.removeChild(element);
}); });
} }
@@ -114,7 +115,9 @@ function traverse(
}); });
} }
postCallBack && postCallBack(element); if (postCallBack) {
postCallBack(element);
}
}; };
innerTraverse(element); innerTraverse(element);

View File

@@ -170,10 +170,10 @@ export class EditorHost extends SignalWatcher(
...Object.values(widgetTags), ...Object.values(widgetTags),
]; ];
await Promise.all( await Promise.all(
elementsTags.map(tag => { elementsTags.map(async tag => {
const element = this.renderRoot.querySelector(tag._$litStatic$); const element = this.renderRoot.querySelector(tag._$litStatic$);
if (element instanceof LitElement) { if (element instanceof LitElement) {
return element.updateComplete; return await element.updateComplete;
} }
return null; return null;
}) })

View File

@@ -382,6 +382,7 @@ describe('addBlock', () => {
const doc0 = collection.createDoc('doc:home'); const doc0 = collection.createDoc('doc:home');
const doc1 = collection.createDoc('space:doc1'); const doc1 = collection.createDoc('space:doc1');
// eslint-disable-next-line @typescript-eslint/await-thenable
await Promise.all([doc0.load(), doc1.load()]); await Promise.all([doc0.load(), doc1.load()]);
assert.equal(collection.docs.size, 2); assert.equal(collection.docs.size, 2);
const store0 = doc0.getStore({ const store0 = doc0.getStore({

View File

@@ -1,7 +1,7 @@
import { minimatch } from 'minimatch'; import { minimatch } from 'minimatch';
import { SCHEMA_NOT_FOUND_MESSAGE } from '../consts.js'; import { SCHEMA_NOT_FOUND_MESSAGE } from '../consts.js';
import { BlockSchema, type BlockSchemaType } from '../model/index.js'; import { BlockSchema, type BlockSchemaType } from '../model/block/zod.js';
import { SchemaValidateError } from './error.js'; import { SchemaValidateError } from './error.js';
/** /**

View File

@@ -1,9 +1,6 @@
import { import { BlockModel } from '../model/block/block-model.js';
BlockModel, import { type DraftModel, toDraftModel } from '../model/block/draft.js';
type DraftModel, import type { Store } from '../model/store/store.js';
type Store,
toDraftModel,
} from '../model/index';
type SliceData = { type SliceData = {
content: DraftModel[]; content: DraftModel[];

View File

@@ -3,14 +3,11 @@ import { BlockSuiteError, ErrorCode } from '@blocksuite/global/exceptions';
import { nextTick } from '@blocksuite/global/utils'; import { nextTick } from '@blocksuite/global/utils';
import { Subject } from 'rxjs'; import { Subject } from 'rxjs';
import { import { BlockModel } from '../model/block/block-model.js';
BlockModel, import { type DraftModel, toDraftModel } from '../model/block/draft.js';
type BlockSchemaType, import type { BlockSchemaType } from '../model/block/zod.js';
type DraftModel, import type { Store } from '../model/store/store.js';
type Store, import type { Schema } from '../schema/schema.js';
toDraftModel,
} from '../model/index.js';
import type { Schema } from '../schema/index.js';
import { AssetsManager } from './assets.js'; import { AssetsManager } from './assets.js';
import { BaseBlockTransformer } from './base.js'; import { BaseBlockTransformer } from './base.js';
import type { import type {

View File

@@ -5,6 +5,7 @@ import eslint from '@eslint/js';
import tsParser from '@typescript-eslint/parser'; import tsParser from '@typescript-eslint/parser';
import eslintConfigPrettier from 'eslint-config-prettier'; import eslintConfigPrettier from 'eslint-config-prettier';
import importX from 'eslint-plugin-import-x'; import importX from 'eslint-plugin-import-x';
import oxlint from 'eslint-plugin-oxlint';
import react from 'eslint-plugin-react'; import react from 'eslint-plugin-react';
import reactHooks from 'eslint-plugin-react-hooks'; import reactHooks from 'eslint-plugin-react-hooks';
import simpleImportSort from 'eslint-plugin-simple-import-sort'; import simpleImportSort from 'eslint-plugin-simple-import-sort';
@@ -16,7 +17,10 @@ const __require = createRequire(import.meta.url);
const rxjs = __require('@smarttools/eslint-plugin-rxjs'); const rxjs = __require('@smarttools/eslint-plugin-rxjs');
const ignoreList = readFileSync('.prettierignore', 'utf-8') const ignoreList = readFileSync(
new URL('.prettierignore', import.meta.url),
'utf-8'
)
.split('\n') .split('\n')
.filter(line => line.trim() && !line.startsWith('#')); .filter(line => line.trim() && !line.startsWith('#'));
@@ -60,105 +64,51 @@ export default tseslint.config(
'simple-import-sort': simpleImportSort, 'simple-import-sort': simpleImportSort,
rxjs, rxjs,
unicorn, unicorn,
oxlint,
}, },
rules: { rules: {
...eslint.configs.recommended.rules, ...eslint.configs.recommended.rules,
...react.configs.recommended.rules, ...react.configs.recommended.rules,
...react.configs['jsx-runtime'].rules, ...react.configs['jsx-runtime'].rules,
...reactHooks.configs.recommended.rules, ...reactHooks.configs.recommended.rules,
...oxlint.configs.recommended.rules,
// covered by TypeScript // covered by TypeScript
'no-dupe-args': 'off', 'no-dupe-args': 'off',
// the following rules are disabled because they are covered by oxlint // the following rules are disabled because they are covered by oxlint
'array-callback-return': 'off', 'array-callback-return': 'off',
'constructor-super': 'off',
eqeqeq: 'off', eqeqeq: 'off',
'getter-return': 'off', 'getter-return': 'off',
'for-direction': 'off',
'require-yield': 'off',
'use-isnan': 'off',
'valid-typeof': 'off',
'no-self-compare': 'off', 'no-self-compare': 'off',
'no-empty': 'off', 'no-empty': 'off',
'no-constant-binary-expression': 'off',
'no-constructor-return': 'off', 'no-constructor-return': 'off',
'no-func-assign': 'off',
'no-global-assign': 'off',
'no-ex-assign': 'off',
'no-fallthrough': 'off', 'no-fallthrough': 'off',
'no-irregular-whitespace': 'off',
'no-control-regex': 'off',
'no-with': 'off',
'no-debugger': 'off',
'no-const-assign': 'off',
'no-import-assign': 'off',
'no-setter-return': 'off',
'no-obj-calls': 'off',
'no-unsafe-negation': 'off',
'no-dupe-class-members': 'off',
'no-dupe-keys': 'off',
'no-this-before-super': 'off',
'no-empty-character-class': 'off',
'no-useless-catch': 'off',
'no-async-promise-executor': 'off',
'no-unreachable': 'off', 'no-unreachable': 'off',
'no-duplicate-case': 'off',
'no-empty-pattern': 'off',
'no-unused-labels': 'off',
'no-sparse-arrays': 'off',
'no-delete-var': 'off',
'no-compare-neg-zero': 'off',
'no-redeclare': 'off', 'no-redeclare': 'off',
'no-case-declarations': 'off', 'no-case-declarations': 'off',
'no-class-assign': 'off',
'no-var': 'off', 'no-var': 'off',
'no-self-assign': 'off',
'no-inner-declarations': 'off', 'no-inner-declarations': 'off',
'no-dupe-else-if': 'off',
'no-invalid-regexp': 'off',
'no-unsafe-finally': 'off',
'no-prototype-builtins': 'off', 'no-prototype-builtins': 'off',
'no-shadow-restricted-names': 'off',
'no-nonoctal-decimal-escape': 'off',
'no-constant-condition': 'off',
'no-useless-escape': 'off',
'no-unsafe-optional-chaining': 'off',
'no-extra-boolean-cast': 'off',
'no-regex-spaces': 'off', 'no-regex-spaces': 'off',
'no-unused-vars': 'off', 'no-unused-vars': 'off',
'no-undef': 'off', 'no-undef': 'off',
'no-cond-assign': 'off',
'react/jsx-no-useless-fragment': 'off', 'react/jsx-no-useless-fragment': 'off',
'react/no-unknown-property': 'off', 'react/no-unknown-property': 'off',
'react/no-string-refs': 'off',
'react/no-direct-mutation-state': 'off',
'react/require-render-return': 'off', 'react/require-render-return': 'off',
'react/jsx-no-undef': 'off',
'react/jsx-no-duplicate-props': 'off',
'react/jsx-key': 'off',
'react/no-danger-with-children': 'off',
'react/no-unescaped-entities': 'off', 'react/no-unescaped-entities': 'off',
'react/no-is-mounted': 'off',
'react/no-find-dom-node': 'off',
'react/no-children-prop': 'off',
'react/no-render-return-value': 'off',
'react/jsx-no-target-blank': 'off', 'react/jsx-no-target-blank': 'off',
'react/jsx-no-comment-textnodes': 'off', 'react/jsx-no-comment-textnodes': 'off',
'react/prop-types': 'off', 'react/prop-types': 'off',
'react-hooks/immutability': 'off',
'react-hooks/refs': 'off',
'react-hooks/set-state-in-effect': 'off',
'react-hooks/static-components': 'off',
'react-hooks/use-memo': 'off',
'sonarjs/no-useless-catch': 'off', 'sonarjs/no-useless-catch': 'off',
'@typescript-eslint/consistent-type-imports': 'off', '@typescript-eslint/consistent-type-imports': 'off',
'@typescript-eslint/no-unused-vars': 'off', '@typescript-eslint/no-unused-vars': 'off',
'@typescript-eslint/no-non-null-assertion': 'off', '@typescript-eslint/no-non-null-assertion': 'off',
'@typescript-eslint/no-loss-of-precision': 'off',
'@typescript-eslint/ban-ts-comment': 'off', '@typescript-eslint/ban-ts-comment': 'off',
'@typescript-eslint/triple-slash-reference': 'off',
'@typescript-eslint/no-non-null-asserted-optional-chain': 'off',
'@typescript-eslint/no-duplicate-enum-values': 'off',
'@typescript-eslint/no-extra-non-null-assertion': 'off',
'@typescript-eslint/no-misused-new': 'off',
'@typescript-eslint/prefer-for-of': 'error', '@typescript-eslint/prefer-for-of': 'error',
'@typescript-eslint/no-unsafe-declaration-merging': 'off',
'@typescript-eslint/no-this-alias': 'off',
'@typescript-eslint/prefer-as-const': 'off',
'@typescript-eslint/no-var-requires': 'off', '@typescript-eslint/no-var-requires': 'off',
'@typescript-eslint/no-namespace': 'off', '@typescript-eslint/no-namespace': 'off',
'@typescript-eslint/no-unnecessary-type-constraint': 'off', '@typescript-eslint/no-unnecessary-type-constraint': 'off',
@@ -167,30 +117,13 @@ export default tseslint.config(
'@typescript-eslint/no-empty-function': 'off', '@typescript-eslint/no-empty-function': 'off',
// rules that are not supported by oxlint // rules that are not supported by oxlint
'no-unreachable-loop': 'error',
'@typescript-eslint/no-unsafe-function-type': 'error', '@typescript-eslint/no-unsafe-function-type': 'error',
'@typescript-eslint/no-wrapper-object-types': 'error',
'@typescript-eslint/unified-signatures': 'error', '@typescript-eslint/unified-signatures': 'error',
'@typescript-eslint/return-await': [ '@typescript-eslint/return-await': [
'error', 'error',
'error-handling-correctness-only', 'error-handling-correctness-only',
], ],
'@typescript-eslint/no-restricted-imports': [
'error',
{
patterns: [
{
group: ['**/dist'],
message: "Don't import from dist",
allowTypeImports: false,
},
{
group: ['**/src'],
message: "Don't import from src",
allowTypeImports: false,
},
],
},
],
'sonarjs/no-all-duplicated-branches': 'error', 'sonarjs/no-all-duplicated-branches': 'error',
'sonarjs/no-element-overwrite': 'error', 'sonarjs/no-element-overwrite': 'error',
'sonarjs/no-empty-collection': 'error', 'sonarjs/no-empty-collection': 'error',
@@ -198,7 +131,6 @@ export default tseslint.config(
'sonarjs/no-identical-conditions': 'error', 'sonarjs/no-identical-conditions': 'error',
'sonarjs/no-identical-expressions': 'error', 'sonarjs/no-identical-expressions': 'error',
'sonarjs/no-ignored-return': 'error', 'sonarjs/no-ignored-return': 'error',
'sonarjs/no-one-iteration-loop': 'error',
'sonarjs/no-use-of-empty-return-value': 'error', 'sonarjs/no-use-of-empty-return-value': 'error',
'sonarjs/non-existent-operator': 'error', 'sonarjs/non-existent-operator': 'error',
'sonarjs/no-collapsible-if': 'error', 'sonarjs/no-collapsible-if': 'error',
@@ -234,13 +166,6 @@ export default tseslint.config(
'error', 'error',
{ includeInternal: true }, { includeInternal: true },
], ],
'react-hooks/exhaustive-deps': [
'warn',
{
additionalHooks:
'(useAsyncCallback|useCatchEventCallback|useDraggable|useDropTarget|useRefEffect)',
},
],
'rxjs/finnish': [ 'rxjs/finnish': [
'error', 'error',
{ {
@@ -304,7 +229,6 @@ export default tseslint.config(
{ ignoreVoid: true }, { ignoreVoid: true },
], ],
'@typescript-eslint/no-misused-promises': 0, '@typescript-eslint/no-misused-promises': 0,
'@typescript-eslint/no-restricted-imports': 0,
}, },
}, },
{ {

View File

@@ -26,9 +26,10 @@
"lint:eslint:fix": "yarn lint:eslint --fix --fix-type problem,suggestion,layout", "lint:eslint:fix": "yarn lint:eslint --fix --fix-type problem,suggestion,layout",
"lint:prettier": "prettier --ignore-unknown --cache --check .", "lint:prettier": "prettier --ignore-unknown --cache --check .",
"lint:prettier:fix": "prettier --ignore-unknown --cache --write .", "lint:prettier:fix": "prettier --ignore-unknown --cache --write .",
"lint:ox": "oxlint -c oxlint.json --deny-warnings", "lint:ox": "oxlint --deny-warnings",
"lint": "yarn lint:eslint && yarn lint:prettier", "lint:ox:fix": "yarn lint:ox --fix",
"lint:fix": "yarn lint:eslint:fix && yarn lint:prettier:fix", "lint": "yarn lint:ox && yarn lint:eslint && yarn lint:prettier",
"lint:fix": "yarn lint:ox:fix && yarn lint:eslint:fix && yarn lint:prettier:fix",
"test": "vitest --run", "test": "vitest --run",
"test:ui": "vitest --ui", "test:ui": "vitest --ui",
"test:coverage": "vitest run --coverage", "test:coverage": "vitest run --coverage",
@@ -51,7 +52,7 @@
"devDependencies": { "devDependencies": {
"@affine-tools/cli": "workspace:*", "@affine-tools/cli": "workspace:*",
"@capacitor/cli": "^7.0.0", "@capacitor/cli": "^7.0.0",
"@eslint/js": "^9.16.0", "@eslint/js": "^9.39.2",
"@faker-js/faker": "^10.1.0", "@faker-js/faker": "^10.1.0",
"@istanbuljs/schema": "^0.1.3", "@istanbuljs/schema": "^0.1.3",
"@magic-works/i18n-codegen": "^0.6.1", "@magic-works/i18n-codegen": "^0.6.1",
@@ -61,32 +62,33 @@
"@toeverything/infra": "workspace:*", "@toeverything/infra": "workspace:*",
"@types/eslint": "^9.6.1", "@types/eslint": "^9.6.1",
"@types/node": "^22.0.0", "@types/node": "^22.0.0",
"@typescript-eslint/parser": "^8.18.0", "@typescript-eslint/parser": "^8.55.0",
"@vanilla-extract/vite-plugin": "^5.0.0", "@vanilla-extract/vite-plugin": "^5.0.0",
"@vitest/browser": "^3.2.4", "@vitest/browser": "^3.2.4",
"@vitest/coverage-istanbul": "^3.2.4", "@vitest/coverage-istanbul": "^3.2.4",
"@vitest/ui": "^3.2.4", "@vitest/ui": "^3.2.4",
"cross-env": "^10.1.0", "cross-env": "^10.1.0",
"electron": "^39.0.0", "electron": "^39.0.0",
"eslint": "^9.16.0", "eslint": "^9.39.2",
"eslint-config-prettier": "^10.0.0", "eslint-config-prettier": "^10.1.8",
"eslint-import-resolver-typescript": "^4.0.0", "eslint-import-resolver-typescript": "^4.4.4",
"eslint-plugin-import-x": "^4.5.0", "eslint-plugin-import-x": "^4.16.1",
"eslint-plugin-react": "^7.37.2", "eslint-plugin-oxlint": "^1.46.0",
"eslint-plugin-react-hooks": "^5.1.0", "eslint-plugin-react": "^7.37.5",
"eslint-plugin-react-hooks": "^7.0.1",
"eslint-plugin-simple-import-sort": "^12.1.1", "eslint-plugin-simple-import-sort": "^12.1.1",
"eslint-plugin-sonarjs": "^3.0.1", "eslint-plugin-sonarjs": "^3.0.7",
"eslint-plugin-unicorn": "^59.0.0", "eslint-plugin-unicorn": "^63.0.0",
"happy-dom": "^20.0.0", "happy-dom": "^20.0.0",
"husky": "^9.1.7", "husky": "^9.1.7",
"lint-staged": "^16.0.0", "lint-staged": "^16.0.0",
"msw": "^2.12.4", "msw": "^2.12.4",
"oxlint": "~1.18.0", "oxlint": "^1.47.0",
"prettier": "^3.7.4", "prettier": "^3.7.4",
"semver": "^7.7.3", "semver": "^7.7.3",
"serve": "^14.2.4", "serve": "^14.2.4",
"typescript": "^5.7.2", "typescript": "^5.7.2",
"typescript-eslint": "^8.18.0", "typescript-eslint": "^8.55.0",
"unplugin-swc": "^1.5.9", "unplugin-swc": "^1.5.9",
"vite": "^7.2.7", "vite": "^7.2.7",
"vitest": "^3.2.4" "vitest": "^3.2.4"

View File

@@ -31,7 +31,7 @@ assert.strictEqual(
bench bench
.add('tiktoken', () => { .add('tiktoken', () => {
const encoder = encoding_for_model('gpt-4o'); const encoder = encoding_for_model('gpt-4o');
encoder.encode_ordinary(FIXTURE).length; void encoder.encode_ordinary(FIXTURE).length;
}) })
.add('native', () => { .add('native', () => {
fromModelName('gpt-4o').count(FIXTURE); fromModelName('gpt-4o').count(FIXTURE);

View File

@@ -39,18 +39,18 @@
"@google-cloud/opentelemetry-cloud-trace-exporter": "^3.0.0", "@google-cloud/opentelemetry-cloud-trace-exporter": "^3.0.0",
"@google-cloud/opentelemetry-resource-util": "^3.0.0", "@google-cloud/opentelemetry-resource-util": "^3.0.0",
"@modelcontextprotocol/sdk": "^1.26.0", "@modelcontextprotocol/sdk": "^1.26.0",
"@nestjs-cls/transactional": "^2.6.1", "@nestjs-cls/transactional": "^2.7.0",
"@nestjs-cls/transactional-adapter-prisma": "^1.2.19", "@nestjs-cls/transactional-adapter-prisma": "^1.2.24",
"@nestjs/apollo": "^13.0.4", "@nestjs/apollo": "^13.0.4",
"@nestjs/bullmq": "^11.0.2", "@nestjs/bullmq": "^11.0.4",
"@nestjs/common": "^11.0.12", "@nestjs/common": "^11.0.21",
"@nestjs/core": "^11.0.12", "@nestjs/core": "^11.1.14",
"@nestjs/graphql": "^13.0.4", "@nestjs/graphql": "^13.0.4",
"@nestjs/platform-express": "^11.0.12", "@nestjs/platform-express": "^11.1.14",
"@nestjs/platform-socket.io": "^11.1.9", "@nestjs/platform-socket.io": "^11.1.14",
"@nestjs/schedule": "^6.0.0", "@nestjs/schedule": "^6.1.1",
"@nestjs/throttler": "^6.4.0", "@nestjs/throttler": "^6.5.0",
"@nestjs/websockets": "^11.0.12", "@nestjs/websockets": "^11.1.14",
"@node-rs/argon2": "^2.0.2", "@node-rs/argon2": "^2.0.2",
"@node-rs/crc32": "^1.10.6", "@node-rs/crc32": "^1.10.6",
"@opentelemetry/api": "^1.9.0", "@opentelemetry/api": "^1.9.0",
@@ -71,7 +71,7 @@
"@opentelemetry/semantic-conventions": "^1.38.0", "@opentelemetry/semantic-conventions": "^1.38.0",
"@prisma/client": "^6.6.0", "@prisma/client": "^6.6.0",
"@prisma/instrumentation": "^6.7.0", "@prisma/instrumentation": "^6.7.0",
"@queuedash/api": "^3.14.0", "@queuedash/api": "^3.16.0",
"@react-email/components": "0.0.38", "@react-email/components": "0.0.38",
"@socket.io/redis-adapter": "^8.3.0", "@socket.io/redis-adapter": "^8.3.0",
"ai": "^5.0.118", "ai": "^5.0.118",
@@ -81,7 +81,7 @@
"date-fns": "^4.0.0", "date-fns": "^4.0.0",
"dotenv": "^16.4.7", "dotenv": "^16.4.7",
"eventemitter2": "^6.4.9", "eventemitter2": "^6.4.9",
"exa-js": "^1.6.13", "exa-js": "^2.4.0",
"express": "^5.0.1", "express": "^5.0.1",
"fast-xml-parser": "^5.3.4", "fast-xml-parser": "^5.3.4",
"get-stream": "^9.0.1", "get-stream": "^9.0.1",

View File

@@ -97,7 +97,7 @@ test('should always return static asset files', async t => {
t.is(res.text, "const name = 'affine'"); t.is(res.text, "const name = 'affine'");
res = await request(t.context.app.getHttpServer()) res = await request(t.context.app.getHttpServer())
.get('/main.b.js') .get('/admin/main.b.js')
.expect(200); .expect(200);
t.is(res.text, "const name = 'affine-admin'"); t.is(res.text, "const name = 'affine-admin'");
@@ -119,7 +119,7 @@ test('should always return static asset files', async t => {
t.is(res.text, "const name = 'affine'"); t.is(res.text, "const name = 'affine'");
res = await request(t.context.app.getHttpServer()) res = await request(t.context.app.getHttpServer())
.get('/main.b.js') .get('/admin/main.b.js')
.expect(200); .expect(200);
t.is(res.text, "const name = 'affine-admin'"); t.is(res.text, "const name = 'affine-admin'");

View File

@@ -43,7 +43,6 @@ class MockR2Provider extends R2StorageProvider {
destroy() {} destroy() {}
// @ts-ignore expect override
override async proxyPutObject( override async proxyPutObject(
key: string, key: string,
body: any, body: any,

View File

@@ -1,6 +1,7 @@
import { LookupAddress } from 'node:dns';
import type { ExecutionContext, TestFn } from 'ava'; import type { ExecutionContext, TestFn } from 'ava';
import ava from 'ava'; import ava from 'ava';
import { LookupAddress } from 'dns';
import Sinon from 'sinon'; import Sinon from 'sinon';
import type { Response } from 'supertest'; import type { Response } from 'supertest';
@@ -14,7 +15,6 @@ import { createTestingApp, TestingApp } from './utils';
type TestContext = { type TestContext = {
app: TestingApp; app: TestingApp;
}; };
const test = ava as TestFn<TestContext>; const test = ava as TestFn<TestContext>;
const LookupAddressStub = (async (_hostname, options) => { const LookupAddressStub = (async (_hostname, options) => {

View File

@@ -51,10 +51,10 @@ function parseKey(privateKey: string) {
let priv: KeyObject; let priv: KeyObject;
try { try {
priv = createPrivateKey({ key: keyBuf, format: 'pem', type: 'pkcs8' }); priv = createPrivateKey({ key: keyBuf, format: 'pem', type: 'pkcs8' });
} catch (e1) { } catch {
try { try {
priv = createPrivateKey({ key: keyBuf, format: 'pem', type: 'sec1' }); priv = createPrivateKey({ key: keyBuf, format: 'pem', type: 'sec1' });
} catch (e2) { } catch {
// As a last resort rely on auto-detection // As a last resort rely on auto-detection
priv = createPrivateKey(keyBuf); priv = createPrivateKey(keyBuf);
} }

View File

@@ -175,7 +175,7 @@ export class R2StorageProvider extends S3StorageProvider {
body: Readable | Buffer | Uint8Array | string, body: Readable | Buffer | Uint8Array | string,
options: { contentType?: string; contentLength?: number } = {} options: { contentType?: string; contentLength?: number } = {}
) { ) {
return this.client.putObject(key, body as any, { return this.client.putObject(key, this.normalizeBody(body), {
contentType: options.contentType, contentType: options.contentType,
contentLength: options.contentLength, contentLength: options.contentLength,
}); });
@@ -192,13 +192,24 @@ export class R2StorageProvider extends S3StorageProvider {
key, key,
uploadId, uploadId,
partNumber, partNumber,
body as any, this.normalizeBody(body),
{ contentLength: options.contentLength } { contentLength: options.contentLength }
); );
return result.etag; return result.etag;
} }
private normalizeBody(body: Readable | Buffer | Uint8Array | string) {
// s3mini does not accept Node.js Readable directly.
// Convert it to Web ReadableStream for compatibility.
if (body instanceof Readable) {
return Readable.toWeb(body);
} else if (typeof body === 'string') {
return this.encoder.encode(body);
}
return body;
}
override async get( override async get(
key: string, key: string,
signedUrl?: boolean signedUrl?: boolean

View File

@@ -281,7 +281,7 @@ export class S3StorageProvider implements StorageProvider {
this.logger.verbose(`Read object \`${key}\``); this.logger.verbose(`Read object \`${key}\``);
return { return {
body: Readable.fromWeb(obj.body as any), body: Readable.fromWeb(obj.body),
metadata: { metadata: {
contentType: contentType ?? 'application/octet-stream', contentType: contentType ?? 'application/octet-stream',
contentLength: contentLength ?? 0, contentLength: contentLength ?? 0,

View File

@@ -22,12 +22,14 @@ function firstNonEmpty(...values: Array<string | undefined>) {
} }
export function getRequestClientIp(req: Request) { export function getRequestClientIp(req: Request) {
return firstNonEmpty( return (
req.get('CF-Connecting-IP'), firstNonEmpty(
firstForwardedForIp(req.get('X-Forwarded-For')), req.get('CF-Connecting-IP'),
req.get('X-Real-IP'), firstForwardedForIp(req.get('X-Forwarded-For')),
req.ip req.get('X-Real-IP'),
)!; req.ip
) ?? ''
);
} }
export function getRequestTrackerId(req: Request) { export function getRequestTrackerId(req: Request) {
@@ -39,6 +41,7 @@ export function getRequestTrackerId(req: Request) {
req.get('X-Real-IP'), req.get('X-Real-IP'),
req.get('CF-Ray'), req.get('CF-Ray'),
req.ip req.ip
)! ) ??
''
); );
} }

View File

@@ -180,7 +180,7 @@ export async function assertSsrFSafeUrl(
let addresses: string[]; let addresses: string[];
try { try {
addresses = await resolveHostAddresses(hostname); addresses = await resolveHostAddresses(hostname);
} catch (error) { } catch {
throw createSsrfBlockedError('unresolvable_hostname', { throw createSsrfBlockedError('unresolvable_hostname', {
url: url.toString(), url: url.toString(),
hostname, hostname,

View File

@@ -109,3 +109,45 @@ test('should record page view when rendering shared page', async t => {
docContent.restore(); docContent.restore();
record.restore(); record.restore();
}); });
test('should return markdown content and skip page view when accept is text/markdown', async t => {
const docId = randomUUID();
const { app, adapter, models, docReader } = t.context;
const doc = new YDoc();
const text = doc.getText('content');
const updates: Buffer[] = [];
doc.on('update', update => {
updates.push(Buffer.from(update));
});
text.insert(0, 'markdown');
await adapter.pushDocUpdates(workspace.id, docId, updates, user.id);
await models.doc.publish(workspace.id, docId);
const markdown = Sinon.stub(docReader, 'getDocMarkdown').resolves({
title: 'markdown-doc',
markdown: '# markdown-doc',
});
const docContent = Sinon.stub(docReader, 'getDocContent');
const record = Sinon.stub(
models.workspaceAnalytics,
'recordDocView'
).resolves();
const res = await app
.GET(`/workspace/${workspace.id}/${docId}`)
.set('accept', 'text/markdown')
.expect(200);
t.true(markdown.calledOnceWithExactly(workspace.id, docId, false));
t.is(res.text, '# markdown-doc');
t.true((res.headers['content-type'] as string).startsWith('text/markdown'));
t.true(docContent.notCalled);
t.true(record.notCalled);
markdown.restore();
docContent.restore();
record.restore();
});

View File

@@ -44,6 +44,12 @@ const staticPaths = new Set([
'trash', 'trash',
]); ]);
const markdownType = new Set([
'text/markdown',
'application/markdown',
'text/x-markdown',
]);
@Controller('/workspace') @Controller('/workspace')
export class DocRendererController { export class DocRendererController {
private readonly logger = new Logger(DocRendererController.name); private readonly logger = new Logger(DocRendererController.name);
@@ -68,6 +74,21 @@ export class DocRendererController {
.digest('hex'); .digest('hex');
} }
private async allowDocPreview(workspaceId: string, docId: string) {
const allowSharing = await this.models.workspace.allowSharing(workspaceId);
if (!allowSharing) return false;
let allowUrlPreview = await this.models.doc.isPublic(workspaceId, docId);
if (!allowUrlPreview) {
// if page is private, but workspace url preview is on
allowUrlPreview =
await this.models.workspace.allowUrlPreview(workspaceId);
}
return allowUrlPreview;
}
@Public() @Public()
@Get('/*path') @Get('/*path')
async render(@Req() req: Request, @Res() res: Response) { async render(@Req() req: Request, @Res() res: Response) {
@@ -81,28 +102,55 @@ export class DocRendererController {
let opts: RenderOptions | null = null; let opts: RenderOptions | null = null;
// /workspace/:workspaceId/{:docId | staticPaths} // /workspace/:workspaceId/{:docId | staticPaths}
const [, , workspaceId, subPath, ...restPaths] = req.path.split('/'); const [, , workspaceId, sub, ...rest] = req.path.split('/');
const isWorkspace =
workspaceId && sub && !staticPaths.has(sub) && rest.length === 0;
const isDocPath = isWorkspace && workspaceId !== sub;
if (
isDocPath &&
req.accepts().some(t => markdownType.has(t.toLowerCase()))
) {
try {
const allowPreview = await this.allowDocPreview(workspaceId, sub);
if (!allowPreview) {
res.status(404).end();
return;
}
const markdown = await this.doc.getDocMarkdown(workspaceId, sub, false);
if (markdown) {
res.setHeader('Content-Type', 'text/markdown; charset=utf-8');
res.send(markdown.markdown);
return;
}
} catch (e) {
this.logger.error('failed to render markdown page', e);
}
res.status(404).end();
return;
}
// /:workspaceId/:docId // /:workspaceId/:docId
if (workspaceId && !staticPaths.has(subPath) && restPaths.length === 0) { if (isWorkspace) {
try { try {
opts = opts = isDocPath
workspaceId === subPath ? await this.getPageContent(workspaceId, sub)
? await this.getWorkspaceContent(workspaceId) : await this.getWorkspaceContent(workspaceId);
: await this.getPageContent(workspaceId, subPath);
metrics.doc.counter('render').add(1); metrics.doc.counter('render').add(1);
if (opts && workspaceId !== subPath) { if (opts && isDocPath) {
void this.models.workspaceAnalytics void this.models.workspaceAnalytics
.recordDocView({ .recordDocView({
workspaceId, workspaceId,
docId: subPath, docId: sub,
visitorId: this.buildVisitorId(req, workspaceId, subPath), visitorId: this.buildVisitorId(req, workspaceId, sub),
isGuest: true, isGuest: true,
}) })
.catch(error => { .catch(error => {
this.logger.warn( this.logger.warn(
`Failed to record shared page view: ${workspaceId}/${subPath}`, `Failed to record shared page view: ${workspaceId}/${sub}`,
error as Error error as Error
); );
}); });
@@ -124,20 +172,7 @@ export class DocRendererController {
workspaceId: string, workspaceId: string,
docId: string docId: string
): Promise<RenderOptions | null> { ): Promise<RenderOptions | null> {
const allowSharing = await this.models.workspace.allowSharing(workspaceId); if (await this.allowDocPreview(workspaceId, docId)) {
if (!allowSharing) {
return null;
}
let allowUrlPreview = await this.models.doc.isPublic(workspaceId, docId);
if (!allowUrlPreview) {
// if page is private, but workspace url preview is on
allowUrlPreview =
await this.models.workspace.allowUrlPreview(workspaceId);
}
if (allowUrlPreview) {
return this.doc.getDocContent(workspaceId, docId); return this.doc.getDocContent(workspaceId, docId);
} }

View File

@@ -56,7 +56,7 @@ defineModuleConfig('mailer', {
env: 'MAILER_PASSWORD', env: 'MAILER_PASSWORD',
}, },
'SMTP.sender': { 'SMTP.sender': {
desc: 'Sender of all the emails (e.g. "AFFiNE Self Hosted \<noreply@example.com\>")', desc: 'Sender of all the emails (e.g. "AFFiNE Self Hosted &lt;noreply@example.com&gt;")',
default: 'AFFiNE Self Hosted <noreply@example.com>', default: 'AFFiNE Self Hosted <noreply@example.com>',
env: 'MAILER_SENDER', env: 'MAILER_SENDER',
}, },
@@ -92,7 +92,7 @@ defineModuleConfig('mailer', {
default: '', default: '',
}, },
'fallbackSMTP.sender': { 'fallbackSMTP.sender': {
desc: 'Sender of all the emails (e.g. "AFFiNE Self Hosted \<noreply@example.com\>")', desc: 'Sender of all the emails (e.g. "AFFiNE Self Hosted &lt;noreply@example.com&gt;")',
default: '', default: '',
}, },
'fallbackSMTP.ignoreTLS': { 'fallbackSMTP.ignoreTLS': {

View File

@@ -52,7 +52,7 @@ export class StaticFilesResolver implements OnModuleInit {
// serve all static files // serve all static files
app.use( app.use(
basePath, basePath + '/admin',
serveStatic(join(staticPath, 'admin'), { serveStatic(join(staticPath, 'admin'), {
redirect: false, redirect: false,
index: false, index: false,

View File

@@ -2,9 +2,11 @@ import { Body, Controller, Options, Post, Req, Res } from '@nestjs/common';
import type { Request, Response } from 'express'; import type { Request, Response } from 'express';
import { BadRequest, Throttle, UseNamedGuard } from '../../base'; import { BadRequest, Throttle, UseNamedGuard } from '../../base';
import type { CurrentUser as CurrentUserType } from '../auth'; import {
import { Public } from '../auth'; CurrentUser,
import { CurrentUser } from '../auth'; type CurrentUser as CurrentUserType,
Public,
} from '../auth';
import { TelemetryService } from './service'; import { TelemetryService } from './service';
import { TelemetryAck, type TelemetryBatch } from './types'; import { TelemetryAck, type TelemetryBatch } from './types';

View File

@@ -2,6 +2,7 @@ import { Injectable, NotFoundException } from '@nestjs/common';
import { import {
Args, Args,
Field, Field,
Info,
InputType, InputType,
Int, Int,
Mutation, Mutation,
@@ -14,6 +15,12 @@ import {
ResolveField, ResolveField,
Resolver, Resolver,
} from '@nestjs/graphql'; } from '@nestjs/graphql';
import {
type FragmentDefinitionNode,
type GraphQLResolveInfo,
Kind,
type SelectionNode,
} from 'graphql';
import { SafeIntResolver } from 'graphql-scalars'; import { SafeIntResolver } from 'graphql-scalars';
import { PaginationInput, URLHelper } from '../../../base'; import { PaginationInput, URLHelper } from '../../../base';
@@ -53,6 +60,44 @@ registerEnumType(AdminSharedLinksOrder, {
name: 'AdminSharedLinksOrder', name: 'AdminSharedLinksOrder',
}); });
function hasSelectedField(
selections: readonly SelectionNode[],
fieldName: string,
fragments: Record<string, FragmentDefinitionNode>
): boolean {
for (const selection of selections) {
if (selection.kind === Kind.FIELD) {
if (selection.name.value === fieldName) {
return true;
}
continue;
}
if (selection.kind === Kind.INLINE_FRAGMENT) {
if (
hasSelectedField(
selection.selectionSet.selections,
fieldName,
fragments
)
) {
return true;
}
continue;
}
const fragment = fragments[selection.name.value];
if (
fragment &&
hasSelectedField(fragment.selectionSet.selections, fieldName, fragments)
) {
return true;
}
}
return false;
}
@InputType() @InputType()
class ListWorkspaceInput { class ListWorkspaceInput {
@Field(() => Int, { defaultValue: 20 }) @Field(() => Int, { defaultValue: 20 })
@@ -471,22 +516,40 @@ export class AdminWorkspaceResolver {
}) })
async adminDashboard( async adminDashboard(
@Args('input', { nullable: true, type: () => AdminDashboardInput }) @Args('input', { nullable: true, type: () => AdminDashboardInput })
input?: AdminDashboardInput input?: AdminDashboardInput,
@Info() info?: GraphQLResolveInfo
) { ) {
this.assertCloudOnly(); this.assertCloudOnly();
const includeTopSharedLinks = Boolean(
info?.fieldNodes.some(
node =>
node.selectionSet &&
hasSelectedField(
node.selectionSet.selections,
'topSharedLinks',
info.fragments
)
)
);
const dashboard = await this.models.workspaceAnalytics.adminGetDashboard({ const dashboard = await this.models.workspaceAnalytics.adminGetDashboard({
timezone: input?.timezone, timezone: input?.timezone,
storageHistoryDays: input?.storageHistoryDays, storageHistoryDays: input?.storageHistoryDays,
syncHistoryHours: input?.syncHistoryHours, syncHistoryHours: input?.syncHistoryHours,
sharedLinkWindowDays: input?.sharedLinkWindowDays, sharedLinkWindowDays: input?.sharedLinkWindowDays,
includeTopSharedLinks,
}); });
return { return {
...dashboard, ...dashboard,
topSharedLinks: dashboard.topSharedLinks.map(link => ({ topSharedLinks: includeTopSharedLinks
...link, ? dashboard.topSharedLinks.map(link => ({
shareUrl: this.url.link(`/workspace/${link.workspaceId}/${link.docId}`), ...link,
})), shareUrl: this.url.link(
`/workspace/${link.workspaceId}/${link.docId}`
),
}))
: [],
}; };
} }

View File

@@ -5,6 +5,7 @@ import { fileURLToPath } from 'node:url';
import pkg from '../package.json' with { type: 'json' }; import pkg from '../package.json' with { type: 'json' };
declare global { declare global {
// oxlint-disable-next-line no-shadow-restricted-names
namespace globalThis { namespace globalThis {
// oxlint-disable-next-line no-var // oxlint-disable-next-line no-var
var env: Readonly<Env>; var env: Readonly<Env>;

View File

@@ -110,10 +110,10 @@ export class CalendarAccountModel extends BaseModel {
refreshIntervalMinutes: data.refreshIntervalMinutes, refreshIntervalMinutes: data.refreshIntervalMinutes,
}; };
if (!!accessToken) { if (accessToken) {
updateData.accessToken = accessToken; updateData.accessToken = accessToken;
} }
if (!!refreshToken) { if (refreshToken) {
updateData.refreshToken = refreshToken; updateData.refreshToken = refreshToken;
} }

View File

@@ -117,7 +117,7 @@ export class CopilotSessionModel extends BaseModel {
if (typeof value !== 'string') { if (typeof value !== 'string') {
return value; return value;
} }
return value.replace(/\u0000/g, '') as T; return value.replaceAll('\0', '') as T;
} }
private sanitizeJsonValue<T>(value: T): T { private sanitizeJsonValue<T>(value: T): T {

View File

@@ -51,6 +51,7 @@ export type AdminDashboardOptions = {
storageHistoryDays?: number; storageHistoryDays?: number;
syncHistoryHours?: number; syncHistoryHours?: number;
sharedLinkWindowDays?: number; sharedLinkWindowDays?: number;
includeTopSharedLinks?: boolean;
}; };
export type AdminAllSharedLinksOptions = { export type AdminAllSharedLinksOptions = {
@@ -262,6 +263,7 @@ export class WorkspaceAnalyticsModel extends BaseModel {
90, 90,
DEFAULT_SHARED_LINK_WINDOW_DAYS DEFAULT_SHARED_LINK_WINDOW_DAYS
); );
const includeTopSharedLinks = options.includeTopSharedLinks ?? true;
const now = new Date(); const now = new Date();
@@ -274,6 +276,66 @@ export class WorkspaceAnalyticsModel extends BaseModel {
const storageFrom = addUtcDays(currentDay, -(storageHistoryDays - 1)); const storageFrom = addUtcDays(currentDay, -(storageHistoryDays - 1));
const sharedFrom = addUtcDays(currentDay, -(sharedLinkWindowDays - 1)); const sharedFrom = addUtcDays(currentDay, -(sharedLinkWindowDays - 1));
const topSharedLinksPromise = includeTopSharedLinks
? this.db.$queryRaw<
{
workspaceId: string;
docId: string;
title: string | null;
publishedAt: Date | null;
docUpdatedAt: Date | null;
workspaceOwnerId: string | null;
lastUpdaterId: string | null;
views: bigint | number;
uniqueViews: bigint | number;
guestViews: bigint | number;
lastAccessedAt: Date | null;
}[]
>`
WITH view_agg AS (
SELECT
workspace_id,
doc_id,
COALESCE(SUM(total_views), 0) AS views,
COALESCE(SUM(unique_views), 0) AS unique_views,
COALESCE(SUM(guest_views), 0) AS guest_views,
MAX(last_accessed_at) AS last_accessed_at
FROM workspace_doc_view_daily
WHERE date BETWEEN ${sharedFrom}::date AND ${currentDay}::date
GROUP BY workspace_id, doc_id
)
SELECT
wp.workspace_id AS "workspaceId",
wp.page_id AS "docId",
wp.title AS title,
wp.published_at AS "publishedAt",
sn.updated_at AS "docUpdatedAt",
owner.user_id AS "workspaceOwnerId",
sn.updated_by AS "lastUpdaterId",
COALESCE(v.views, 0) AS views,
COALESCE(v.unique_views, 0) AS "uniqueViews",
COALESCE(v.guest_views, 0) AS "guestViews",
v.last_accessed_at AS "lastAccessedAt"
FROM workspace_pages wp
LEFT JOIN snapshots sn
ON sn.workspace_id = wp.workspace_id AND sn.guid = wp.page_id
LEFT JOIN view_agg v
ON v.workspace_id = wp.workspace_id AND v.doc_id = wp.page_id
LEFT JOIN LATERAL (
SELECT user_id
FROM workspace_user_permissions
WHERE workspace_id = wp.workspace_id
AND type = ${WorkspaceRole.Owner}
AND status = 'Accepted'::"WorkspaceMemberStatus"
ORDER BY created_at ASC
LIMIT 1
) owner ON TRUE
WHERE wp.public = TRUE
ORDER BY views DESC, "uniqueViews" DESC, "workspaceId" ASC, "docId" ASC
LIMIT 10
`
: Promise.resolve([]);
const [ const [
syncCurrent, syncCurrent,
syncTimeline, syncTimeline,
@@ -350,63 +412,7 @@ export class WorkspaceAnalyticsModel extends BaseModel {
AND created_at >= ${sharedFrom} AND created_at >= ${sharedFrom}
AND created_at <= ${now} AND created_at <= ${now}
`, `,
this.db.$queryRaw< topSharedLinksPromise,
{
workspaceId: string;
docId: string;
title: string | null;
publishedAt: Date | null;
docUpdatedAt: Date | null;
workspaceOwnerId: string | null;
lastUpdaterId: string | null;
views: bigint | number;
uniqueViews: bigint | number;
guestViews: bigint | number;
lastAccessedAt: Date | null;
}[]
>`
WITH view_agg AS (
SELECT
workspace_id,
doc_id,
COALESCE(SUM(total_views), 0) AS views,
COALESCE(SUM(unique_views), 0) AS unique_views,
COALESCE(SUM(guest_views), 0) AS guest_views,
MAX(last_accessed_at) AS last_accessed_at
FROM workspace_doc_view_daily
WHERE date BETWEEN ${sharedFrom}::date AND ${currentDay}::date
GROUP BY workspace_id, doc_id
)
SELECT
wp.workspace_id AS "workspaceId",
wp.page_id AS "docId",
wp.title AS title,
wp.published_at AS "publishedAt",
sn.updated_at AS "docUpdatedAt",
owner.user_id AS "workspaceOwnerId",
sn.updated_by AS "lastUpdaterId",
COALESCE(v.views, 0) AS views,
COALESCE(v.unique_views, 0) AS "uniqueViews",
COALESCE(v.guest_views, 0) AS "guestViews",
v.last_accessed_at AS "lastAccessedAt"
FROM workspace_pages wp
LEFT JOIN snapshots sn
ON sn.workspace_id = wp.workspace_id AND sn.guid = wp.page_id
LEFT JOIN view_agg v
ON v.workspace_id = wp.workspace_id AND v.doc_id = wp.page_id
LEFT JOIN LATERAL (
SELECT user_id
FROM workspace_user_permissions
WHERE workspace_id = wp.workspace_id
AND type = ${WorkspaceRole.Owner}
AND status = 'Accepted'::"WorkspaceMemberStatus"
ORDER BY created_at ASC
LIMIT 1
) owner ON TRUE
WHERE wp.public = TRUE
ORDER BY views DESC, "uniqueViews" DESC, "workspaceId" ASC, "docId" ASC
LIMIT 10
`,
]); ]);
const storageHistorySeries = storageHistory.map(row => ({ const storageHistorySeries = storageHistory.map(row => ({

View File

@@ -22,8 +22,8 @@ import {
CalendarProviderListCalendarsParams, CalendarProviderListCalendarsParams,
CalendarProviderListEventsParams, CalendarProviderListEventsParams,
CalendarProviderListEventsResult, CalendarProviderListEventsResult,
CalendarProviderName,
} from './def'; } from './def';
import { CalendarProviderName } from './factory';
import { CalendarSyncTokenInvalid } from './google'; import { CalendarSyncTokenInvalid } from './google';
const XML_PARSER = new XMLParser({ const XML_PARSER = new XMLParser({
@@ -113,7 +113,7 @@ const isRedirectStatus = (status: number) =>
const splitHeaderTokens = (value: string) => const splitHeaderTokens = (value: string) =>
value value
.split(/,(?=(?:[^\"]*\"[^\"]*\")*[^\"]*$)/) .split(/,(?=(?:[^"]*"[^"]*")*[^"]*$)/)
.map(token => token.trim()) .map(token => token.trim())
.filter(Boolean); .filter(Boolean);

View File

@@ -2,12 +2,7 @@ import { Inject, Injectable, Logger } from '@nestjs/common';
import type { CalendarAccount } from '@prisma/client'; import type { CalendarAccount } from '@prisma/client';
import { CalendarProviderRequestError, Config, OnEvent } from '../../../base'; import { CalendarProviderRequestError, Config, OnEvent } from '../../../base';
import { CalendarProviderFactory } from './factory'; import { CalendarProviderFactory, CalendarProviderName } from './factory';
export enum CalendarProviderName {
Google = 'google',
CalDAV = 'caldav',
}
export interface CalendarProviderTokens { export interface CalendarProviderTokens {
accessToken: string; accessToken: string;

View File

@@ -1,12 +1,20 @@
import { Injectable, Logger } from '@nestjs/common'; import { Injectable, Logger } from '@nestjs/common';
import type { CalendarProvider } from './def'; export enum CalendarProviderName {
import { CalendarProviderName } from './def'; Google = 'google',
CalDAV = 'caldav',
}
export interface CalendarProviderRef {
provider: CalendarProviderName;
}
@Injectable() @Injectable()
export class CalendarProviderFactory { export class CalendarProviderFactory<
TProvider extends CalendarProviderRef = CalendarProviderRef,
> {
private readonly logger = new Logger(CalendarProviderFactory.name); private readonly logger = new Logger(CalendarProviderFactory.name);
readonly #providers = new Map<CalendarProviderName, CalendarProvider>(); readonly #providers = new Map<CalendarProviderName, TProvider>();
get providers() { get providers() {
return Array.from(this.#providers.keys()); return Array.from(this.#providers.keys());
@@ -16,12 +24,12 @@ export class CalendarProviderFactory {
return this.#providers.get(name); return this.#providers.get(name);
} }
register(provider: CalendarProvider) { register(provider: TProvider) {
this.#providers.set(provider.provider, provider); this.#providers.set(provider.provider, provider);
this.logger.log(`Calendar provider [${provider.provider}] registered.`); this.logger.log(`Calendar provider [${provider.provider}] registered.`);
} }
unregister(provider: CalendarProvider) { unregister(provider: TProvider) {
this.#providers.delete(provider.provider); this.#providers.delete(provider.provider);
this.logger.log(`Calendar provider [${provider.provider}] unregistered.`); this.logger.log(`Calendar provider [${provider.provider}] unregistered.`);
} }

View File

@@ -1,17 +1,17 @@
import { Injectable } from '@nestjs/common'; import { Injectable } from '@nestjs/common';
import { CalendarProviderRequestError } from '../../../base'; import { CalendarProviderRequestError } from '../../../base';
import { CalendarProvider } from './def';
import { import {
CalendarProvider,
CalendarProviderEvent, CalendarProviderEvent,
CalendarProviderListCalendarsParams, CalendarProviderListCalendarsParams,
CalendarProviderListEventsParams, CalendarProviderListEventsParams,
CalendarProviderListEventsResult, CalendarProviderListEventsResult,
CalendarProviderName,
CalendarProviderTokens, CalendarProviderTokens,
CalendarProviderWatchParams, CalendarProviderWatchParams,
CalendarProviderWatchResult, CalendarProviderWatchResult,
} from './def'; } from './def';
import { CalendarProviderName } from './factory';
export class CalendarSyncTokenInvalid extends Error { export class CalendarSyncTokenInvalid extends Error {
readonly code = 'calendar_sync_token_invalid'; readonly code = 'calendar_sync_token_invalid';

View File

@@ -14,9 +14,8 @@ export type {
CalendarProviderWatchParams, CalendarProviderWatchParams,
CalendarProviderWatchResult, CalendarProviderWatchResult,
} from './def'; } from './def';
export { CalendarProviderName } from './def';
export { CalendarProvider } from './def'; export { CalendarProvider } from './def';
export { CalendarProviderFactory } from './factory'; export { CalendarProviderFactory, CalendarProviderName } from './factory';
export { CalendarSyncTokenInvalid, GoogleCalendarProvider } from './google'; export { CalendarSyncTokenInvalid, GoogleCalendarProvider } from './google';
export const CalendarProviders = [GoogleCalendarProvider, CalDAVProvider]; export const CalendarProviders = [GoogleCalendarProvider, CalDAVProvider];

View File

@@ -18,10 +18,10 @@ import {
CalendarProvider, CalendarProvider,
CalendarProviderEvent, CalendarProviderEvent,
CalendarProviderEventTime, CalendarProviderEventTime,
CalendarProviderFactory,
CalendarProviderName, CalendarProviderName,
CalendarSyncTokenInvalid, CalendarSyncTokenInvalid,
} from './providers'; } from './providers';
import { CalendarProviderFactory } from './providers';
import type { LinkCalDAVAccountInput } from './types'; import type { LinkCalDAVAccountInput } from './types';
const TOKEN_REFRESH_SKEW_MS = 60 * 1000; const TOKEN_REFRESH_SKEW_MS = 60 * 1000;
@@ -35,7 +35,7 @@ export class CalendarService {
constructor( constructor(
private readonly models: Models, private readonly models: Models,
private readonly providerFactory: CalendarProviderFactory, private readonly providerFactory: CalendarProviderFactory<CalendarProvider>,
private readonly mutex: Mutex, private readonly mutex: Mutex,
private readonly config: Config, private readonly config: Config,
private readonly url: URLHelper private readonly url: URLHelper
@@ -105,11 +105,11 @@ export class CalendarService {
const accessToken = accountTokens.accessToken; const accessToken = accountTokens.accessToken;
if (accessToken) { if (accessToken) {
await Promise.allSettled( await Promise.allSettled(
needToStopChannel.map(s => { needToStopChannel.map(async s => {
if (!s.customChannelId || !s.customResourceId) { if (!s.customChannelId || !s.customResourceId) {
return Promise.resolve(); return;
} }
return provider.stopChannel?.({ return await provider.stopChannel?.({
accessToken, accessToken,
channelId: s.customChannelId, channelId: s.customChannelId,
resourceId: s.customResourceId, resourceId: s.customResourceId,
@@ -654,8 +654,11 @@ export class CalendarService {
} }
const zone = time.timeZone ?? fallbackTimezone ?? 'UTC'; const zone = time.timeZone ?? fallbackTimezone ?? 'UTC';
if (!time.date) {
throw new Error('Calendar provider returned all-day event without date');
}
return { return {
date: this.convertDateToUtc(time.date!, zone), date: this.convertDateToUtc(time.date, zone),
allDay: true, allDay: true,
}; };
} }

View File

@@ -49,7 +49,7 @@ import {
FileChunkSimilarity, FileChunkSimilarity,
Models, Models,
} from '../../../models'; } from '../../../models';
import { CopilotEmbeddingJob } from '../embedding'; import { CopilotEmbeddingJob } from '../embedding/job';
import { COPILOT_LOCKER, CopilotType } from '../resolver'; import { COPILOT_LOCKER, CopilotType } from '../resolver';
import { ChatSessionService } from '../session'; import { ChatSessionService } from '../session';
import { CopilotStorage } from '../storage'; import { CopilotStorage } from '../storage';

View File

@@ -15,7 +15,8 @@ import {
ContextFile, ContextFile,
Models, Models,
} from '../../../models'; } from '../../../models';
import { type EmbeddingClient, getEmbeddingClient } from '../embedding'; import { getEmbeddingClient } from '../embedding/client';
import type { EmbeddingClient } from '../embedding/types';
import { ContextSession } from './session'; import { ContextSession } from './session';
const CONTEXT_SESSION_KEY = 'context-session'; const CONTEXT_SESSION_KEY = 'context-session';

View File

@@ -11,7 +11,7 @@ import {
FileChunkSimilarity, FileChunkSimilarity,
Models, Models,
} from '../../../models'; } from '../../../models';
import { EmbeddingClient } from '../embedding'; import { EmbeddingClient } from '../embedding/types';
export class ContextSession implements AsyncDisposable { export class ContextSession implements AsyncDisposable {
constructor( constructor(

View File

@@ -47,14 +47,14 @@ import {
} from '../../base'; } from '../../base';
import { ServerFeature, ServerService } from '../../core'; import { ServerFeature, ServerService } from '../../core';
import { CurrentUser, Public } from '../../core/auth'; import { CurrentUser, Public } from '../../core/auth';
import { CopilotContextService } from './context'; import { CopilotContextService } from './context/service';
import { CopilotProviderFactory } from './providers/factory';
import type { CopilotProvider } from './providers/provider';
import { import {
CopilotProvider,
CopilotProviderFactory,
ModelInputType, ModelInputType,
ModelOutputType, ModelOutputType,
StreamObject, type StreamObject,
} from './providers'; } from './providers/types';
import { StreamObjectParser } from './providers/utils'; import { StreamObjectParser } from './providers/utils';
import { ChatSession, ChatSessionService } from './session'; import { ChatSession, ChatSessionService } from './session';
import { CopilotStorage } from './storage'; import { CopilotStorage } from './storage';
@@ -560,7 +560,7 @@ export class CopilotController implements BeforeApplicationShutdown {
status: data.status, status: data.status,
id: data.node.id, id: data.node.id,
type: data.node.config.nodeType, type: data.node.config.nodeType,
} as any, },
}; };
} }
}) })

View File

@@ -12,14 +12,14 @@ import {
Embedding, Embedding,
EMBEDDING_DIMENSIONS, EMBEDDING_DIMENSIONS,
} from '../../../models'; } from '../../../models';
import { PromptService } from '../prompt'; import { PromptService } from '../prompt/service';
import { CopilotProviderFactory } from '../providers/factory';
import type { CopilotProvider } from '../providers/provider';
import { import {
type CopilotProvider,
CopilotProviderFactory,
type ModelFullConditions, type ModelFullConditions,
ModelInputType, ModelInputType,
ModelOutputType, ModelOutputType,
} from '../providers'; } from '../providers/types';
import { EmbeddingClient, type ReRankResult } from './types'; import { EmbeddingClient, type ReRankResult } from './types';
const EMBEDDING_MODEL = 'gemini-embedding-001'; const EMBEDDING_MODEL = 'gemini-embedding-001';

View File

@@ -8,7 +8,7 @@ import { DocReader, DocWriter } from '../../../core/doc';
import { AccessController } from '../../../core/permission'; import { AccessController } from '../../../core/permission';
import { clearEmbeddingChunk } from '../../../models'; import { clearEmbeddingChunk } from '../../../models';
import { IndexerService } from '../../indexer'; import { IndexerService } from '../../indexer';
import { CopilotContextService } from '../context'; import { CopilotContextService } from '../context/service';
@Injectable() @Injectable()
export class WorkspaceMcpProvider { export class WorkspaceMcpProvider {

View File

@@ -4,7 +4,11 @@ import { AiPrompt } from '@prisma/client';
import Mustache from 'mustache'; import Mustache from 'mustache';
import { getTokenEncoder } from '../../../native'; import { getTokenEncoder } from '../../../native';
import { PromptConfig, PromptMessage, PromptParams } from '../providers'; import type {
PromptConfig,
PromptMessage,
PromptParams,
} from '../providers/types';
// disable escaping // disable escaping
Mustache.escape = (text: string) => text; Mustache.escape = (text: string) => text;

View File

@@ -1,7 +1,7 @@
import { Logger } from '@nestjs/common'; import { Logger } from '@nestjs/common';
import { AiPrompt, PrismaClient } from '@prisma/client'; import { AiPrompt, PrismaClient } from '@prisma/client';
import { PromptConfig, PromptMessage } from '../providers'; import type { PromptConfig, PromptMessage } from '../providers/types';
type Prompt = Omit< type Prompt = Omit<
AiPrompt, AiPrompt,

View File

@@ -8,7 +8,7 @@ import {
PromptConfigSchema, PromptConfigSchema,
PromptMessage, PromptMessage,
PromptMessageSchema, PromptMessageSchema,
} from '../providers'; } from '../providers/types';
import { ChatPrompt } from './chat-prompt'; import { ChatPrompt } from './chat-prompt';
import { import {
CopilotPromptScenario, CopilotPromptScenario,

View File

@@ -13,8 +13,8 @@ import { DocReader, DocWriter } from '../../../core/doc';
import { AccessController } from '../../../core/permission'; import { AccessController } from '../../../core/permission';
import { Models } from '../../../models'; import { Models } from '../../../models';
import { IndexerService } from '../../indexer'; import { IndexerService } from '../../indexer';
import { CopilotContextService } from '../context'; import { CopilotContextService } from '../context/service';
import { PromptService } from '../prompt'; import { PromptService } from '../prompt/service';
import { import {
buildBlobContentGetter, buildBlobContentGetter,
buildContentGetter, buildContentGetter,

View File

@@ -42,9 +42,9 @@ import { AccessController, DocAction } from '../../core/permission';
import { UserType } from '../../core/user'; import { UserType } from '../../core/user';
import type { ListSessionOptions, UpdateChatSession } from '../../models'; import type { ListSessionOptions, UpdateChatSession } from '../../models';
import { CopilotCronJobs } from './cron'; import { CopilotCronJobs } from './cron';
import { PromptService } from './prompt'; import { PromptService } from './prompt/service';
import { PromptMessage, StreamObject } from './providers';
import { CopilotProviderFactory } from './providers/factory'; import { CopilotProviderFactory } from './providers/factory';
import type { PromptMessage, StreamObject } from './providers/types';
import { ChatSessionService } from './session'; import { ChatSessionService } from './session';
import { CopilotStorage } from './storage'; import { CopilotStorage } from './storage';
import { type ChatHistory, type ChatMessage, SubmittedMessage } from './types'; import { type ChatHistory, type ChatMessage, SubmittedMessage } from './types';

View File

@@ -28,13 +28,14 @@ import {
import { SubscriptionService } from '../payment/service'; import { SubscriptionService } from '../payment/service';
import { SubscriptionPlan, SubscriptionStatus } from '../payment/types'; import { SubscriptionPlan, SubscriptionStatus } from '../payment/types';
import { ChatMessageCache } from './message'; import { ChatMessageCache } from './message';
import { ChatPrompt, PromptService } from './prompt'; import { ChatPrompt } from './prompt/chat-prompt';
import { PromptService } from './prompt/service';
import { CopilotProviderFactory } from './providers/factory';
import { import {
CopilotProviderFactory,
ModelOutputType, ModelOutputType,
PromptMessage, type PromptMessage,
PromptParams, type PromptParams,
} from './providers'; } from './providers/types';
import { import {
type ChatHistory, type ChatHistory,
type ChatMessage, type ChatMessage,
@@ -322,7 +323,7 @@ export class ChatSessionService {
private stripNullBytes(value?: string | null): string { private stripNullBytes(value?: string | null): string {
if (!value) return ''; if (!value) return '';
return value.replace(/\u0000/g, ''); return value.replaceAll('\0', '');
} }
private isNullByteError(error: unknown): boolean { private isNullByteError(error: unknown): boolean {

View File

@@ -3,9 +3,8 @@ import { tool } from 'ai';
import { z } from 'zod'; import { z } from 'zod';
import { AccessController } from '../../../core/permission'; import { AccessController } from '../../../core/permission';
import type { ContextSession } from '../context/session';
import type { CopilotChatOptions } from '../providers';
import { toolError } from './error'; import { toolError } from './error';
import type { ContextSession, CopilotChatOptions } from './types';
const logger = new Logger('ContextBlobReadTool'); const logger = new Logger('ContextBlobReadTool');

View File

@@ -2,9 +2,9 @@ import { Logger } from '@nestjs/common';
import { tool } from 'ai'; import { tool } from 'ai';
import { z } from 'zod'; import { z } from 'zod';
import type { PromptService } from '../prompt';
import type { CopilotProviderFactory } from '../providers';
import { toolError } from './error'; import { toolError } from './error';
import type { CopilotProviderFactory, PromptService } from './types';
const logger = new Logger('CodeArtifactTool'); const logger = new Logger('CodeArtifactTool');
/** /**
* A copilot tool that produces a completely self-contained HTML artifact. * A copilot tool that produces a completely self-contained HTML artifact.

View File

@@ -2,9 +2,8 @@ import { Logger } from '@nestjs/common';
import { tool } from 'ai'; import { tool } from 'ai';
import { z } from 'zod'; import { z } from 'zod';
import type { PromptService } from '../prompt';
import type { CopilotProviderFactory } from '../providers';
import { toolError } from './error'; import { toolError } from './error';
import type { CopilotProviderFactory, PromptService } from './types';
const logger = new Logger('ConversationSummaryTool'); const logger = new Logger('ConversationSummaryTool');

View File

@@ -2,9 +2,8 @@ import { Logger } from '@nestjs/common';
import { tool } from 'ai'; import { tool } from 'ai';
import { z } from 'zod'; import { z } from 'zod';
import type { PromptService } from '../prompt';
import type { CopilotProviderFactory } from '../providers';
import { toolError } from './error'; import { toolError } from './error';
import type { CopilotProviderFactory, PromptService } from './types';
const logger = new Logger('DocComposeTool'); const logger = new Logger('DocComposeTool');

View File

@@ -3,8 +3,11 @@ import { z } from 'zod';
import { DocReader } from '../../../core/doc'; import { DocReader } from '../../../core/doc';
import { AccessController } from '../../../core/permission'; import { AccessController } from '../../../core/permission';
import { type PromptService } from '../prompt'; import type {
import type { CopilotChatOptions, CopilotProviderFactory } from '../providers'; CopilotChatOptions,
CopilotProviderFactory,
PromptService,
} from './types';
const CodeEditSchema = z const CodeEditSchema = z
.array( .array(

View File

@@ -3,8 +3,8 @@ import { z } from 'zod';
import type { AccessController } from '../../../core/permission'; import type { AccessController } from '../../../core/permission';
import type { IndexerService, SearchDoc } from '../../indexer'; import type { IndexerService, SearchDoc } from '../../indexer';
import type { CopilotChatOptions } from '../providers';
import { toolError } from './error'; import { toolError } from './error';
import type { CopilotChatOptions } from './types';
export const buildDocKeywordSearchGetter = ( export const buildDocKeywordSearchGetter = (
ac: AccessController, ac: AccessController,

View File

@@ -5,8 +5,8 @@ import { z } from 'zod';
import { DocReader } from '../../../core/doc'; import { DocReader } from '../../../core/doc';
import { AccessController } from '../../../core/permission'; import { AccessController } from '../../../core/permission';
import { Models, publicUserSelect } from '../../../models'; import { Models, publicUserSelect } from '../../../models';
import type { CopilotChatOptions } from '../providers';
import { toolError } from './error'; import { toolError } from './error';
import type { CopilotChatOptions } from './types';
const logger = new Logger('DocReadTool'); const logger = new Logger('DocReadTool');

View File

@@ -8,10 +8,12 @@ import {
clearEmbeddingChunk, clearEmbeddingChunk,
type Models, type Models,
} from '../../../models'; } from '../../../models';
import type { CopilotContextService } from '../context';
import type { ContextSession } from '../context/session';
import type { CopilotChatOptions } from '../providers';
import { toolError } from './error'; import { toolError } from './error';
import type {
ContextSession,
CopilotChatOptions,
CopilotContextService,
} from './types';
export const buildDocSearchGetter = ( export const buildDocSearchGetter = (
ac: AccessController, ac: AccessController,

View File

@@ -4,8 +4,8 @@ import { z } from 'zod';
import { DocWriter } from '../../../core/doc'; import { DocWriter } from '../../../core/doc';
import { AccessController } from '../../../core/permission'; import { AccessController } from '../../../core/permission';
import type { CopilotChatOptions } from '../providers';
import { toolError } from './error'; import { toolError } from './error';
import type { CopilotChatOptions } from './types';
const logger = new Logger('DocWriteTool'); const logger = new Logger('DocWriteTool');

View File

@@ -19,9 +19,7 @@ export const createExaCrawlTool = (config: Config) => {
const exa = new Exa(key); const exa = new Exa(key);
const result = await exa.getContents([url], { const result = await exa.getContents([url], {
livecrawl: 'always', livecrawl: 'always',
text: { text: { maxCharacters: 100000 },
maxCharacters: 100000,
},
}); });
return result.results.map(data => ({ return result.results.map(data => ({
title: data.title, title: data.title,

View File

@@ -18,10 +18,12 @@ export const createExaSearchTool = (config: Config) => {
try { try {
const { key } = config.copilot.exa; const { key } = config.copilot.exa;
const exa = new Exa(key); const exa = new Exa(key);
const result = await exa.searchAndContents(query, { const result = await exa.search(query, {
contents: {
summary: true,
livecrawl: mode === 'MUST' ? 'always' : undefined,
},
numResults: 10, numResults: 10,
summary: true,
livecrawl: mode === 'MUST' ? 'always' : undefined,
}); });
return result.results.map(data => ({ return result.results.map(data => ({
title: data.title, title: data.title,

View File

@@ -2,9 +2,8 @@ import { Logger } from '@nestjs/common';
import { tool } from 'ai'; import { tool } from 'ai';
import { z } from 'zod'; import { z } from 'zod';
import type { PromptService } from '../prompt';
import type { CopilotProviderFactory } from '../providers';
import { toolError } from './error'; import { toolError } from './error';
import type { CopilotProviderFactory, PromptService } from './types';
const logger = new Logger('SectionEditTool'); const logger = new Logger('SectionEditTool');

View File

@@ -0,0 +1,5 @@
export type { CopilotContextService } from '../context/service';
export type { ContextSession } from '../context/session';
export type { PromptService } from '../prompt/service';
export type { CopilotProviderFactory } from '../providers/factory';
export type { CopilotChatOptions } from '../providers/types';

View File

@@ -125,6 +125,7 @@ export class CopilotTranscriptionResolver {
user.id, user.id,
workspaceId, workspaceId,
blobId, blobId,
// eslint-disable-next-line @typescript-eslint/await-thenable
await Promise.all(allBlobs) await Promise.all(allBlobs)
); );

View File

@@ -15,14 +15,10 @@ import {
sniffMime, sniffMime,
} from '../../../base'; } from '../../../base';
import { Models } from '../../../models'; import { Models } from '../../../models';
import { PromptService } from '../prompt'; import { PromptService } from '../prompt/service';
import { import type { CopilotProvider, PromptMessage } from '../providers';
CopilotProvider, import { CopilotProviderFactory } from '../providers/factory';
CopilotProviderFactory, import { CopilotProviderType, ModelOutputType } from '../providers/types';
CopilotProviderType,
ModelOutputType,
PromptMessage,
} from '../providers';
import { CopilotStorage } from '../storage'; import { CopilotStorage } from '../storage';
import { import {
AudioBlobInfos, AudioBlobInfos,
@@ -171,7 +167,7 @@ export class CopilotTranscriptionService {
if (payload.success) { if (payload.success) {
let { url, mimeType, infos } = payload.data; let { url, mimeType, infos } = payload.data;
infos = infos || []; infos = infos || [];
if (url && mimeType && !infos.find(i => i.url === url)) { if (url && mimeType && !infos.some(i => i.url === url)) {
infos.push({ url, mimeType }); infos.push({ url, mimeType });
} }

View File

@@ -1,7 +1,7 @@
import { z } from 'zod'; import { z } from 'zod';
import type { ChatPrompt } from './prompt'; import type { ChatPrompt } from './prompt/chat-prompt';
import { PromptMessageSchema, PureMessageSchema } from './providers'; import { PromptMessageSchema, PureMessageSchema } from './providers/types';
const takeFirst = (v: unknown) => (Array.isArray(v) ? v[0] : v); const takeFirst = (v: unknown) => (Array.isArray(v) ? v[0] : v);

View File

@@ -3,7 +3,7 @@ import { Readable } from 'node:stream';
import type { Request } from 'express'; import type { Request } from 'express';
import { OneMB, readBufferWithLimit } from '../../base'; import { OneMB, readBufferWithLimit } from '../../base';
import type { PromptTools } from './providers'; import type { PromptTools } from './providers/types';
import type { ToolsConfig } from './types'; import type { ToolsConfig } from './types';
export const MAX_EMBEDDABLE_SIZE = 50 * OneMB; export const MAX_EMBEDDABLE_SIZE = 50 * OneMB;

View File

@@ -4,7 +4,7 @@ import { fileURLToPath } from 'node:url';
import { Logger } from '@nestjs/common'; import { Logger } from '@nestjs/common';
import Piscina from 'piscina'; import Piscina from 'piscina';
import { CopilotChatOptions } from '../providers'; import type { CopilotChatOptions } from '../providers/types';
import type { NodeExecuteResult, NodeExecutor } from './executor'; import type { NodeExecuteResult, NodeExecutor } from './executor';
import { getWorkflowExecutor, NodeExecuteState } from './executor'; import { getWorkflowExecutor, NodeExecuteState } from './executor';
import type { import type {

View File

@@ -1,6 +1,6 @@
import { Injectable, Logger } from '@nestjs/common'; import { Injectable, Logger } from '@nestjs/common';
import { CopilotChatOptions } from '../providers'; import type { CopilotChatOptions } from '../providers/types';
import { WorkflowGraphList } from './graph'; import { WorkflowGraphList } from './graph';
import { WorkflowNode } from './node'; import { WorkflowNode } from './node';
import type { WorkflowGraph, WorkflowGraphInstances } from './types'; import type { WorkflowGraph, WorkflowGraphInstances } from './types';

View File

@@ -1,6 +1,6 @@
import { Logger } from '@nestjs/common'; import { Logger } from '@nestjs/common';
import { CopilotChatOptions } from '../providers'; import type { CopilotChatOptions } from '../providers/types';
import { NodeExecuteState } from './executor'; import { NodeExecuteState } from './executor';
import { WorkflowNode } from './node'; import { WorkflowNode } from './node';
import type { WorkflowGraphInstances, WorkflowNodeState } from './types'; import type { WorkflowGraphInstances, WorkflowNodeState } from './types';

View File

@@ -132,6 +132,10 @@ export class IndexerJob {
indexed: true, indexed: true,
}); });
} }
if (!missingDocIds.length && !deletedDocIds.length) {
this.logger.verbose(`workspace ${workspaceId} is already indexed`);
return;
}
this.logger.log( this.logger.log(
`indexed workspace ${workspaceId} with ${missingDocIds.length} missing docs and ${deletedDocIds.length} deleted docs` `indexed workspace ${workspaceId} with ${missingDocIds.length} missing docs and ${deletedDocIds.length} deleted docs`
); );

View File

@@ -132,10 +132,11 @@ export class AppleOAuthProvider extends OAuthProvider {
{ method: 'GET' }, { method: 'GET' },
{ treatServerErrorAsInvalid: true } { treatServerErrorAsInvalid: true }
); );
const idToken = tokens.idToken;
const payload = await new Promise<JwtPayload>((resolve, reject) => { const payload = await new Promise<JwtPayload>((resolve, reject) => {
jwt.verify( jwt.verify(
tokens.idToken!, idToken,
(header, callback) => { (header, callback) => {
const key = keys.find(key => key.kid === header.kid); const key = keys.find(key => key.kid === header.kid);
if (!key) { if (!key) {

View File

@@ -29,6 +29,36 @@ const SHOULD_MANUAL_REDIRECT =
BUILD_CONFIG.isAndroid || BUILD_CONFIG.isIOS || BUILD_CONFIG.isElectron; BUILD_CONFIG.isAndroid || BUILD_CONFIG.isIOS || BUILD_CONFIG.isElectron;
const UPLOAD_REQUEST_TIMEOUT = 0; const UPLOAD_REQUEST_TIMEOUT = 0;
function toStrictArrayBuffer(
data: ArrayBuffer | ArrayBufferLike | ArrayBufferView
): ArrayBuffer {
if (data instanceof ArrayBuffer) {
return data;
}
if (ArrayBuffer.isView(data)) {
if (data.buffer instanceof ArrayBuffer) {
if (data.byteOffset === 0 && data.byteLength === data.buffer.byteLength) {
return data.buffer;
}
return data.buffer.slice(
data.byteOffset,
data.byteOffset + data.byteLength
);
}
const bytes = new Uint8Array(data.buffer, data.byteOffset, data.byteLength);
const copy = new Uint8Array(bytes.byteLength);
copy.set(bytes);
return copy.buffer;
}
const bytes = new Uint8Array(data);
const copy = new Uint8Array(bytes.byteLength);
copy.set(bytes);
return copy.buffer;
}
export class CloudBlobStorage extends BlobStorageBase { export class CloudBlobStorage extends BlobStorageBase {
static readonly identifier = 'CloudBlobStorage'; static readonly identifier = 'CloudBlobStorage';
override readonly isReadonly = false; override readonly isReadonly = false;
@@ -127,8 +157,11 @@ export class CloudBlobStorage extends BlobStorageBase {
if (upload.method === BlobUploadMethod.PRESIGNED) { if (upload.method === BlobUploadMethod.PRESIGNED) {
try { try {
if (!upload.uploadUrl) {
throw new Error('Missing upload URL for presigned upload.');
}
await this.uploadViaPresigned( await this.uploadViaPresigned(
upload.uploadUrl!, upload.uploadUrl,
upload.headers, upload.headers,
blob.data, blob.data,
signal signal
@@ -143,15 +176,20 @@ export class CloudBlobStorage extends BlobStorageBase {
if (upload.method === BlobUploadMethod.MULTIPART) { if (upload.method === BlobUploadMethod.MULTIPART) {
try { try {
if (!upload.uploadId || !upload.partSize) {
throw new Error(
'Missing upload ID or part size for multipart upload.'
);
}
const parts = await this.uploadViaMultipart( const parts = await this.uploadViaMultipart(
blob.key, blob.key,
upload.uploadId!, upload.uploadId,
upload.partSize!, upload.partSize,
blob.data, blob.data,
upload.uploadedParts, upload.uploadedParts,
signal signal
); );
await this.completeUpload(blob.key, upload.uploadId!, parts, signal); await this.completeUpload(blob.key, upload.uploadId, parts, signal);
return; return;
} catch { } catch {
if (upload.uploadId) { if (upload.uploadId) {
@@ -216,7 +254,9 @@ export class CloudBlobStorage extends BlobStorageBase {
query: setBlobMutation, query: setBlobMutation,
variables: { variables: {
workspaceId: this.options.id, workspaceId: this.options.id,
blob: new File([blob.data], blob.key, { type: blob.mime }), blob: new File([toStrictArrayBuffer(blob.data)], blob.key, {
type: blob.mime,
}),
}, },
context: { signal }, context: { signal },
timeout: UPLOAD_REQUEST_TIMEOUT, timeout: UPLOAD_REQUEST_TIMEOUT,
@@ -232,7 +272,7 @@ export class CloudBlobStorage extends BlobStorageBase {
const res = await this.fetchWithTimeout(uploadUrl, { const res = await this.fetchWithTimeout(uploadUrl, {
method: 'PUT', method: 'PUT',
headers: headers ?? undefined, headers: headers ?? undefined,
body: data, body: toStrictArrayBuffer(data),
signal, signal,
timeout: UPLOAD_REQUEST_TIMEOUT, timeout: UPLOAD_REQUEST_TIMEOUT,
}); });
@@ -275,7 +315,7 @@ export class CloudBlobStorage extends BlobStorageBase {
{ {
method: 'PUT', method: 'PUT',
headers: part.workspace.blobUploadPartUrl.headers ?? undefined, headers: part.workspace.blobUploadPartUrl.headers ?? undefined,
body: chunk, body: toStrictArrayBuffer(chunk),
signal, signal,
timeout: UPLOAD_REQUEST_TIMEOUT, timeout: UPLOAD_REQUEST_TIMEOUT,
} }

View File

@@ -141,10 +141,10 @@ export class CloudIndexerStorage extends IndexerStorageBase {
} }
override async refreshIfNeed(): Promise<void> { override async refreshIfNeed(): Promise<void> {
return Promise.resolve(); return;
} }
override async indexVersion(): Promise<number> { override async indexVersion(): Promise<number> {
return Promise.resolve(1); return 1;
} }
} }

View File

@@ -222,6 +222,6 @@ export class IndexedDBIndexerStorage extends IndexerStorageBase {
// Get the current indexer version // Get the current indexer version
// increase this number to re-index all docs // increase this number to re-index all docs
async indexVersion(): Promise<number> { async indexVersion(): Promise<number> {
return Promise.resolve(1); return 1;
} }
} }

View File

@@ -1,4 +1,5 @@
import { merge, Observable, of, Subject } from 'rxjs'; import type { Observable } from 'rxjs';
import { merge, of, Subject } from 'rxjs';
import { filter, throttleTime } from 'rxjs/operators'; import { filter, throttleTime } from 'rxjs/operators';
import { share } from '../../../connection'; import { share } from '../../../connection';
@@ -194,9 +195,9 @@ export class SqliteIndexerStorage extends IndexerStorageBase {
const schema = IndexerSchema[table]; const schema = IndexerSchema[table];
for (const [field, values] of document.fields) { for (const [field, values] of document.fields) {
const fieldSchema = schema[field]; const fieldSchema = schema[field];
// @ts-expect-error // @ts-expect-error -- IndexerSchema uses runtime-keyed fields from each table schema.
const shouldIndex = fieldSchema.index !== false; const shouldIndex = fieldSchema.index !== false;
// @ts-expect-error // @ts-expect-error -- IndexerSchema uses runtime-keyed fields from each table schema.
const shouldStore = fieldSchema.store !== false; const shouldStore = fieldSchema.store !== false;
if (!shouldStore && !shouldIndex) continue; if (!shouldStore && !shouldIndex) continue;

View File

@@ -86,9 +86,9 @@ export class DummyIndexerStorage extends IndexerStorageBase {
return Promise.resolve(); return Promise.resolve();
} }
override async refreshIfNeed(): Promise<void> { override async refreshIfNeed(): Promise<void> {
return Promise.resolve(); return;
} }
override async indexVersion(): Promise<number> { override async indexVersion(): Promise<number> {
return Promise.resolve(0); return 0;
} }
} }

View File

@@ -190,6 +190,7 @@ export class BlobSyncImpl implements BlobSync {
): Promise<void> { ): Promise<void> {
return Promise.race([ return Promise.race([
Promise.all( Promise.all(
// eslint-disable-next-line @typescript-eslint/await-thenable
peerId peerId
? [this.fullDownloadPeer(peerId)] ? [this.fullDownloadPeer(peerId)]
: this.peers.map(p => this.fullDownloadPeer(p.peerId)) : this.peers.map(p => this.fullDownloadPeer(p.peerId))

View File

@@ -125,8 +125,8 @@ export class TelemetryManager {
private mergeContext(event: TelemetryEvent): TelemetryEvent { private mergeContext(event: TelemetryEvent): TelemetryEvent {
const mergedUserProps = { const mergedUserProps = {
...(this.context.userProperties ?? {}), ...this.context.userProperties,
...(event.userProperties ?? {}), ...event.userProperties,
}; };
const mergedContext = { const mergedContext = {

View File

@@ -1,6 +1,6 @@
import { Buffer } from 'node:buffer'; import type { Buffer } from 'node:buffer';
import { stringify as stringifyQuery } from 'node:querystring'; import { stringify as stringifyQuery } from 'node:querystring';
import { Readable } from 'node:stream'; import type { Readable } from 'node:stream';
import aws4 from 'aws4'; import aws4 from 'aws4';
import { XMLParser } from 'fast-xml-parser'; import { XMLParser } from 'fast-xml-parser';
@@ -180,16 +180,16 @@ export function parseListPartsXml(xml: string): ParsedListParts {
function buildEndpoint(config: S3CompatConfig) { function buildEndpoint(config: S3CompatConfig) {
const url = new URL(config.endpoint); const url = new URL(config.endpoint);
if (config.forcePathStyle) { if (config.forcePathStyle) {
const segments = url.pathname.split('/').filter(Boolean); const firstSegment = url.pathname.split('/').find(Boolean);
if (segments[0] !== config.bucket) { if (firstSegment !== config.bucket) {
url.pathname = joinPath(url.pathname, config.bucket); url.pathname = joinPath(url.pathname, config.bucket);
} }
return url; return url;
} }
const pathSegments = url.pathname.split('/').filter(Boolean); const firstSegment = url.pathname.split('/').find(Boolean);
const hostHasBucket = url.hostname.startsWith(`${config.bucket}.`); const hostHasBucket = url.hostname.startsWith(`${config.bucket}.`);
const pathHasBucket = pathSegments[0] === config.bucket; const pathHasBucket = firstSegment === config.bucket;
if (!hostHasBucket && !pathHasBucket) { if (!hostHasBucket && !pathHasBucket) {
url.hostname = `${config.bucket}.${url.hostname}`; url.hostname = `${config.bucket}.${url.hostname}`;
} }
@@ -297,7 +297,7 @@ export class S3Compat implements S3CompatClient {
const expiresInSeconds = this.presignConfig.expiresInSeconds; const expiresInSeconds = this.presignConfig.expiresInSeconds;
const path = this.buildObjectPath(key); const path = this.buildObjectPath(key);
const queryString = buildQuery({ const queryString = buildQuery({
...(query ?? {}), ...query,
'X-Amz-Expires': expiresInSeconds, 'X-Amz-Expires': expiresInSeconds,
}); });
const requestPath = queryString ? `${path}?${queryString}` : path; const requestPath = queryString ? `${path}?${queryString}` : path;

Some files were not shown because too many files have changed in this diff Show More