Compare commits

...

4 Commits

Author SHA1 Message Date
renovate[bot]
4a89007448 chore: bump up all non-major npm dependencies 2026-03-22 01:08:51 +00:00
renovate[bot]
8ba02ed6fb chore: bump up RevenueCat/purchases-ios-spm version to from: "5.66.0" (#14699)
This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
|
[RevenueCat/purchases-ios-spm](https://redirect.github.com/RevenueCat/purchases-ios-spm)
| minor | `from: "5.60.0"` → `from: "5.66.0"` |

---

### Release Notes

<details>
<summary>RevenueCat/purchases-ios-spm
(RevenueCat/purchases-ios-spm)</summary>

###
[`v5.66.0`](https://redirect.github.com/RevenueCat/purchases-ios-spm/compare/5.65.0...5.66.0)

[Compare
Source](https://redirect.github.com/RevenueCat/purchases-ios-spm/compare/5.65.0...5.66.0)

###
[`v5.65.0`](https://redirect.github.com/RevenueCat/purchases-ios-spm/blob/HEAD/CHANGELOG.md#5650)

[Compare
Source](https://redirect.github.com/RevenueCat/purchases-ios-spm/compare/5.64.0...5.65.0)

#### 5.65.0

###
[`v5.64.0`](https://redirect.github.com/RevenueCat/purchases-ios-spm/blob/HEAD/CHANGELOG.md#5640)

[Compare
Source](https://redirect.github.com/RevenueCat/purchases-ios-spm/compare/5.63.0...5.64.0)

#### 5.64.0

###
[`v5.63.0`](https://redirect.github.com/RevenueCat/purchases-ios-spm/blob/HEAD/CHANGELOG.md#5630)

[Compare
Source](https://redirect.github.com/RevenueCat/purchases-ios-spm/compare/5.62.0...5.63.0)

#### 5.63.0

###
[`v5.62.0`](https://redirect.github.com/RevenueCat/purchases-ios-spm/blob/HEAD/CHANGELOG.md#5620)

[Compare
Source](https://redirect.github.com/RevenueCat/purchases-ios-spm/compare/5.61.0...5.62.0)

#### 5.62.0

###
[`v5.61.0`](https://redirect.github.com/RevenueCat/purchases-ios-spm/blob/HEAD/CHANGELOG.md#5610)

[Compare
Source](https://redirect.github.com/RevenueCat/purchases-ios-spm/compare/5.60.0...5.61.0)

#### 5.61.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/toeverything/AFFiNE).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My42Ni40IiwidXBkYXRlZEluVmVyIjoiNDMuNjYuNCIsInRhcmdldEJyYW5jaCI6ImNhbmFyeSIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiXX0=-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-22 04:03:11 +08:00
renovate[bot]
ffa3ff9d7f chore: bump up apple/swift-collections version to from: "1.4.1" (#14697)
This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
|
[apple/swift-collections](https://redirect.github.com/apple/swift-collections)
| patch | `from: "1.4.0"` → `from: "1.4.1"` |

---

### Release Notes

<details>
<summary>apple/swift-collections (apple/swift-collections)</summary>

###
[`v1.4.1`](https://redirect.github.com/apple/swift-collections/releases/tag/1.4.1):
Swift Collections 1.4.1

[Compare
Source](https://redirect.github.com/apple/swift-collections/compare/1.4.0...1.4.1)

This patch release is mostly focusing on evolving the package traits
`UnstableContainersPreview` and `UnstableHashedContainers`, with the
following notable fixes and improvements to the stable parts of the
package:

- Make the package documentation build successfully on the DocC that
ships in Swift 6.2.
- Avoid using floating point arithmetic to size collection storage in
the `DequeModule` and `OrderedCollections` modules.

#### Changes to experimental package traits

The new set and dictionary types enabled by the
`UnstableHashedContainers` trait have now resolved several correctness
issues in their implementation of insertions. They have also gained some
low-hanging performance optimizations. Like before, these types are in
"working prototype" phase, and while they have working implementations
of basic primitive operations, we haven't done much work validating
their performance yet. Feedback from intrepid early adopters would be
very welcome.

The `UnstableContainersPreview` trait has gained several new protocols
and algorithm implementations, working towards one possible working
model of a coherent, ownership-aware container/iteration model.

- [`BidirectionalContainer`][BidirectionalContainer] defines a container
that allows iterating over spans backwards, and provides decrement
operations on indices -- an analogue of the classic
`BidirectionalCollection` protocol.
- [`RandomAccessContainer`][RandomAccessContainer] models containers
that allow constant-time repositioning of their indices, like
`RandomAccessCollection`.
- [`MutableContainer`][MutableContainer] is the ownership-aware analogue
of `MutableCollection` -- it models a container type that allows its
elements to be arbitrarily reordered and mutated/reassigned without
changing the shape of the data structure (that is to say, without
invalidating any indices).
- [`PermutableContainer`][PermutableContainer] is an experimental new
spinoff of `MutableContainer`, focusing on reordering items without
allowing arbitrary mutations.
- [`RangeReplaceableContainer`][RangeReplaceableContainer] is a partial,
ownership-aware analogue of `RangeReplaceableCollection`, providing a
full set of insertion/append/removal/consumption operations, with
support for fixed-capacity conforming types.
- [`DynamicContainer`][DynamicContainer] rounds out the
range-replacement operations with initializer and capacity reservation
requirements that can only be implemented by dynamically sized
containers.

[BidirectionalContainer]:
https://redirect.github.com/apple/swift-collections/blob/main/Sources/ContainersPreview/Protocols/Container/BidirectionalContainer.swift

[RandomAccessContainer]:
https://redirect.github.com/apple/swift-collections/blob/main/Sources/ContainersPreview/Protocols/Container/RandomAccessContainer.swift

[MutableContainer]:
https://redirect.github.com/apple/swift-collections/blob/main/Sources/ContainersPreview/Protocols/Container/MutableContainer.swift

[PermutableContainer]:
https://redirect.github.com/apple/swift-collections/blob/main/Sources/ContainersPreview/Protocols/Container/PermutableContainer.swift

[RangeReplaceableContainer]:
https://redirect.github.com/apple/swift-collections/blob/main/Sources/ContainersPreview/Protocols/Container/RangeReplaceableContainer.swift

[DynamicContainer]:
https://redirect.github.com/apple/swift-collections/blob/main/Sources/ContainersPreview/Protocols/Container/DynamicContainer.swift

- We now have [working reference
implementations](https://redirect.github.com/apple/swift-collections/tree/main/Sources/ContainersPreview/Protocols)
of lazy `map`, `reduce` and `filter` operations on borrowing iterators,
producers and drains, as well a `collect(into:)` family of methods to
supply "greedy" variants, generating items into a container of the
user's choice. Importantly, the algorithms tend to be defined on the
iterator types, rather than directly on some sequence/container -- going
this way has some interesting benefits (explicitness, no confusion
between the various flavors or the existing `Sequence` algorithms), but
they also have notable drawbacks (minor design issues with the borrowing
iterator protocol, unknowns on how the pattern would apply to container
algorithms, etc.).

```swift
    let items: RigidArray<Int> = ...
    let transformed = 
      items.makeBorrowingIterator() // obviously we'd want a better name here, like `borrow()`
      .map { 2 * $0 }
      .collect(into: UniqueArray.self)
    // `transformed` is a UniqueArray instance holding all values in `items`, doubled up 
```

```swift
    let items: RigidArray = ...
    let transformed = 
       items.makeBorrowingIterator()
      .filter { !$0.isMultiple(of: 7) }
      .copy()
      .collect(into: UniqueArray.self)
    // `transformed` holds a copy of all values in `items` that aren't a multiple of 7
```

```swift
    let items: RigidArray = ...
    let transformed = 
       items.consumeAll()
      .filter { !$0.isMultiple(of: 7) }
      .collect(into: UniqueArray.self)
    // `transformed` holds all values that were previously in `items` that aren't a multiple of 7. `items` is now empty.
```

Like before, these are highly experimental, and they will definitely
change in dramatic/radical ways on the way to stabilization. Note that
there is no project- or team-wide consensus on any of these constructs.
I'm publishing them primarily as a crucial reference point, and to gain
a level of shared understanding of the actual problems that need to be
resolved, and the consequences of the design path we are on.

#### What's Changed

- Add some decorative badges in the README by
[@&#8203;lorentey](https://redirect.github.com/lorentey) in
[#&#8203;591](https://redirect.github.com/apple/swift-collections/pull/591)
- \[Dequemodule, OrderedCollections] Avoid using floating point
arithmetic by [@&#8203;lorentey](https://redirect.github.com/lorentey)
in
[#&#8203;592](https://redirect.github.com/apple/swift-collections/pull/592)
- Enforce dress code for license headers by
[@&#8203;lorentey](https://redirect.github.com/lorentey) in
[#&#8203;593](https://redirect.github.com/apple/swift-collections/pull/593)
- Bump swiftlang/github-workflows/.github/workflows/soundness.yml from
0.0.7 to 0.0.8 by
[@&#8203;dependabot](https://redirect.github.com/dependabot)\[bot] in
[#&#8203;595](https://redirect.github.com/apple/swift-collections/pull/595)
- Documentation updates for latest DocC by
[@&#8203;lorentey](https://redirect.github.com/lorentey) in
[#&#8203;596](https://redirect.github.com/apple/swift-collections/pull/596)
- \[BasicContainers] Allow standalone use of the
UnstableHashedContainers trait by
[@&#8203;lorentey](https://redirect.github.com/lorentey) in
[#&#8203;597](https://redirect.github.com/apple/swift-collections/pull/597)
- Bump
swiftlang/github-workflows/.github/workflows/swift\_package\_test.yml
from 0.0.7 to 0.0.8 by
[@&#8203;dependabot](https://redirect.github.com/dependabot)\[bot] in
[#&#8203;594](https://redirect.github.com/apple/swift-collections/pull/594)
- \[ContainersPreview] Rename Producer.generateNext() to next() by
[@&#8203;lorentey](https://redirect.github.com/lorentey) in
[#&#8203;599](https://redirect.github.com/apple/swift-collections/pull/599)
- \[ContainersPreview] Remove BorrowingSequence.first by
[@&#8203;lorentey](https://redirect.github.com/lorentey) in
[#&#8203;598](https://redirect.github.com/apple/swift-collections/pull/598)
- \[CI] Enable Android testing by
[@&#8203;marcprux](https://redirect.github.com/marcprux) in
[#&#8203;558](https://redirect.github.com/apple/swift-collections/pull/558)
- \[BasicContainers] Assorted hashed container fixes and improvements by
[@&#8203;lorentey](https://redirect.github.com/lorentey) in
[#&#8203;601](https://redirect.github.com/apple/swift-collections/pull/601)
- Flesh out BorrowingSequence/Container/Producer model a little more by
[@&#8203;lorentey](https://redirect.github.com/lorentey) in
[#&#8203;603](https://redirect.github.com/apple/swift-collections/pull/603)
- More exploration of ownership-aware container/iterator algorithms by
[@&#8203;lorentey](https://redirect.github.com/lorentey) in
[#&#8203;605](https://redirect.github.com/apple/swift-collections/pull/605)

#### New Contributors

- [@&#8203;marcprux](https://redirect.github.com/marcprux) made their
first contribution in
[#&#8203;558](https://redirect.github.com/apple/swift-collections/pull/558)

**Full Changelog**:
<https://github.com/apple/swift-collections/compare/1.4.0...1.4.1>

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/toeverything/AFFiNE).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My42Ni40IiwidXBkYXRlZEluVmVyIjoiNDMuNjYuNCIsInRhcmdldEJyYW5jaCI6ImNhbmFyeSIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiXX0=-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-03-22 02:52:07 +08:00
DarkSky
f47ee2bc8a feat(server): improve indexer (#14698)
fix #13862 


#### PR Dependency Tree


* **PR #14698** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Enhanced search support for Chinese, Japanese, and Korean languages
with improved text segmentation and character matching.
* Added index management capabilities with table recreation
functionality.

* **Bug Fixes**
* Improved search accuracy for non-Latin scripts through updated
morphology and n-gram configuration.

* **Chores**
  * Added database migration for search index optimization.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-03-22 02:50:59 +08:00
34 changed files with 3391 additions and 2507 deletions

View File

@@ -1,4 +1,4 @@
# syntax=docker/dockerfile:1.7
# syntax=docker/dockerfile:1.22
FROM node:22-bookworm-slim AS assets
WORKDIR /app

View File

@@ -182,7 +182,7 @@ jobs:
run: yarn workspace @affine/android cap sync
- uses: actions/setup-python@v6
with:
python-version: '3.13'
python-version: '3.14'
- name: Auth gcloud
id: auth
uses: google-github-actions/auth@v2

695
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -30,9 +30,9 @@ resolver = "3"
chrono = "0.4"
clap = { version = "4.4", features = ["derive"] }
core-foundation = "0.10"
coreaudio-rs = "0.12"
cpal = "0.15"
criterion = { version = "0.5", features = ["html_reports"] }
coreaudio-rs = "0.14"
cpal = "0.17"
criterion = { version = "0.8", features = ["html_reports"] }
criterion2 = { version = "3", default-features = false }
crossbeam-channel = "0.5"
dispatch2 = "0.3"
@@ -82,18 +82,18 @@ resolver = "3"
parking_lot = "0.12"
path-ext = "0.1.2"
pdf-extract = { git = "https://github.com/toeverything/pdf-extract", branch = "darksky/improve-font-decoding" }
phf = { version = "0.11", features = ["macros"] }
phf = { version = "0.13", features = ["macros"] }
proptest = "1.3"
proptest-derive = "0.5"
proptest-derive = "0.8"
pulldown-cmark = "0.13"
rand = "0.9"
rand_chacha = "0.9"
rand_distr = "0.5"
rand = "0.10"
rand_chacha = "0.10"
rand_distr = "0.6"
rayon = "1.10"
readability = { version = "0.3.0", default-features = false }
regex = "1.10"
rubato = "0.16"
screencapturekit = "0.3"
screencapturekit = "0.4"
serde = "1"
serde_json = "1"
sha3 = "0.10"
@@ -106,23 +106,23 @@ resolver = "3"
"sqlite",
"tls-rustls",
] }
strum_macros = "0.27.0"
strum_macros = "0.28.0"
symphonia = { version = "0.5", features = ["all", "opt-simd"] }
text-splitter = "0.27"
text-splitter = "0.29"
thiserror = "2"
tiktoken-rs = "0.7"
tiktoken-rs = "0.9"
tokio = "1.45"
tree-sitter = { version = "0.25" }
tree-sitter = { version = "0.26" }
tree-sitter-c = { version = "0.24" }
tree-sitter-c-sharp = { version = "0.23" }
tree-sitter-cpp = { version = "0.23" }
tree-sitter-go = { version = "0.23" }
tree-sitter-go = { version = "0.25" }
tree-sitter-java = { version = "0.23" }
tree-sitter-javascript = { version = "0.23" }
tree-sitter-javascript = { version = "0.25" }
tree-sitter-kotlin-ng = { version = "1.1" }
tree-sitter-python = { version = "0.23" }
tree-sitter-python = { version = "0.25" }
tree-sitter-rust = { version = "0.24" }
tree-sitter-scala = { version = "0.24" }
tree-sitter-scala = { version = "0.25" }
tree-sitter-typescript = { version = "0.23" }
typst = "0.14.2"
typst-as-lib = { version = "0.15.4", default-features = false, features = [
@@ -132,11 +132,11 @@ resolver = "3"
"ureq",
] }
typst-svg = "0.14.2"
uniffi = "0.29"
uniffi = "0.31"
url = { version = "2.5" }
uuid = "1.8"
v_htmlescape = "0.15"
windows = { version = "0.61", features = [
windows = { version = "0.62", features = [
"Win32_Devices_FunctionDiscovery",
"Win32_Foundation",
"Win32_Media_Audio",
@@ -148,10 +148,10 @@ resolver = "3"
"Win32_System_Variant",
"Win32_UI_Shell_PropertiesSystem",
] }
windows-core = { version = "0.61" }
windows-core = { version = "0.62" }
y-octo = { path = "./packages/common/y-octo/core" }
y-sync = { version = "0.4" }
yrs = "0.23.0"
yrs = "0.25.0"
[profile.dev.package.sqlx-macros]
opt-level = 3

View File

@@ -39,7 +39,7 @@
"micromark-extension-gfm-table": "^2.1.0",
"micromark-extension-gfm-task-list-item": "^2.1.0",
"micromark-util-combine-extensions": "^2.0.0",
"pdfmake": "^0.2.20",
"pdfmake": "^0.3.0",
"quick-lru": "^7.3.0",
"rehype-parse": "^9.0.0",
"rehype-stringify": "^10.0.0",
@@ -73,7 +73,7 @@
"!dist/__tests__"
],
"devDependencies": {
"@types/pdfmake": "^0.2.12",
"@types/pdfmake": "^0.3.0",
"vitest": "^4.0.18"
},
"version": "0.26.3"

View File

@@ -22,7 +22,7 @@
"@toeverything/pdfium": "^0.1.1",
"@toeverything/y-indexeddb": "0.10.0-canary.9",
"@types/katex": "^0.16.7",
"browser-fs-access": "^0.37.0",
"browser-fs-access": "^0.38.0",
"jszip": "^3.10.1",
"katex": "^0.16.27",
"lit": "^3.2.0",

View File

@@ -32,7 +32,7 @@
"build:debug": "napi build"
},
"devDependencies": {
"@napi-rs/cli": "3.5.0",
"@napi-rs/cli": "3.5.1",
"tiktoken": "^1.0.17"
}
}

View File

@@ -79,7 +79,7 @@
"graphql-scalars": "^1.24.0",
"graphql-upload": "^17.0.0",
"html-validate": "^9.0.0",
"htmlrewriter": "^0.0.12",
"htmlrewriter": "^0.0.13",
"http-errors": "^2.0.0",
"ioredis": "^5.8.2",
"is-mobile": "^5.0.0",
@@ -96,7 +96,7 @@
"piscina": "^5.1.4",
"prisma": "^6.6.0",
"react": "^19.2.1",
"react-dom": "19.2.1",
"react-dom": "19.2.4",
"reflect-metadata": "^0.2.2",
"rxjs": "^7.8.2",
"semver": "^7.7.3",

View File

@@ -0,0 +1,12 @@
import { ModuleRef } from '@nestjs/core';
import { PrismaClient } from '@prisma/client';
import { IndexerService } from '../../plugins/indexer';
export class RebuildManticoreMixedScriptIndexes1763800000000 {
static async up(_db: PrismaClient, ref: ModuleRef) {
await ref.get(IndexerService, { strict: false }).rebuildManticoreIndexes();
}
static async down(_db: PrismaClient) {}
}

View File

@@ -3,3 +3,4 @@ export * from './1703756315970-unamed-account';
export * from './1721299086340-refresh-unnamed-user';
export * from './1745211351719-create-indexer-tables';
export * from './1751966744168-correct-session-update-time';
export * from './1763800000000-rebuild-manticore-mixed-script-indexes';

View File

@@ -4,6 +4,75 @@ The actual snapshot is saved in `manticoresearch.spec.ts.snap`.
Generated by [AVA](https://avajs.dev).
## should search doc title match chinese word segmentation
> Snapshot 1
[
{
_id: '5373363211628325828',
_source: {
doc_id: 'doc-chinese',
workspace_id: 'workspace-test-doc-title-chinese',
},
fields: {
doc_id: [
'doc-chinese',
],
title: [
'AFFiNE 是一个基于云端的笔记应用',
],
},
highlights: undefined,
},
]
## should search block content match korean ngram
> Snapshot 1
[
{
_id: '1227635764506850985',
_source: {
doc_id: 'doc-korean',
workspace_id: 'workspace-test-block-content-korean',
},
fields: {
block_id: [
'block-korean',
],
content: [
'다람쥐 헌 쳇바퀴에 타고파',
],
},
highlights: undefined,
},
]
## should search block content match japanese kana ngram
> Snapshot 1
[
{
_id: '381498385699454292',
_source: {
doc_id: 'doc-japanese',
workspace_id: 'workspace-test-block-content-japanese',
},
fields: {
block_id: [
'block-japanese',
],
content: [
'いろはにほへと ちりぬるを',
],
},
highlights: undefined,
},
]
## should write document work
> Snapshot 1
@@ -889,7 +958,7 @@ Generated by [AVA](https://avajs.dev).
> Snapshot 1
{
term: {
equals: {
workspace_id: 'workspaceId1',
},
}
@@ -897,7 +966,7 @@ Generated by [AVA](https://avajs.dev).
> Snapshot 2
{
term: {
equals: {
workspace_id: 'workspaceId1',
},
}

View File

@@ -33,8 +33,8 @@ const user = await module.create(Mockers.User);
const workspace = await module.create(Mockers.Workspace);
test.before(async () => {
await searchProvider.createTable(SearchTable.block, blockSQL);
await searchProvider.createTable(SearchTable.doc, docSQL);
await searchProvider.recreateTable(SearchTable.block, blockSQL);
await searchProvider.recreateTable(SearchTable.doc, docSQL);
await searchProvider.write(
SearchTable.block,
@@ -163,6 +163,135 @@ test('should provider is manticoresearch', t => {
t.is(searchProvider.type, SearchProviderType.Manticoresearch);
});
test('should search doc title match chinese word segmentation', async t => {
const workspaceId = 'workspace-test-doc-title-chinese';
const docId = 'doc-chinese';
const title = 'AFFiNE 是一个基于云端的笔记应用';
await searchProvider.write(
SearchTable.doc,
[
{
workspace_id: workspaceId,
doc_id: docId,
title,
},
],
{
refresh: true,
}
);
const result = await searchProvider.search(SearchTable.doc, {
_source: ['workspace_id', 'doc_id'],
query: {
bool: {
must: [
{ term: { workspace_id: { value: workspaceId } } },
{ match: { title: '笔记' } },
],
},
},
fields: ['doc_id', 'title'],
sort: ['_score'],
});
t.true(result.total >= 1);
t.snapshot(
result.nodes
.filter(node => node._source.doc_id === docId)
.map(node => omit(node, ['_score']))
);
});
test('should search block content match korean ngram', async t => {
const workspaceId = 'workspace-test-block-content-korean';
const docId = 'doc-korean';
const blockId = 'block-korean';
const content = '다람쥐 헌 쳇바퀴에 타고파';
await searchProvider.write(
SearchTable.block,
[
{
workspace_id: workspaceId,
doc_id: docId,
block_id: blockId,
content,
flavour: 'affine:paragraph',
},
],
{
refresh: true,
}
);
const result = await searchProvider.search(SearchTable.block, {
_source: ['workspace_id', 'doc_id'],
query: {
bool: {
must: [
{ term: { workspace_id: { value: workspaceId } } },
{ match: { content: '쥐' } },
],
},
},
fields: ['block_id', 'content'],
sort: ['_score'],
});
t.true(result.total >= 1);
t.snapshot(
result.nodes
.filter(node => node.fields.block_id?.[0] === blockId)
.map(node => omit(node, ['_score']))
);
});
test('should search block content match japanese kana ngram', async t => {
const workspaceId = 'workspace-test-block-content-japanese';
const docId = 'doc-japanese';
const blockId = 'block-japanese';
const content = 'いろはにほへと ちりぬるを';
await searchProvider.write(
SearchTable.block,
[
{
workspace_id: workspaceId,
doc_id: docId,
block_id: blockId,
content,
flavour: 'affine:paragraph',
},
],
{
refresh: true,
}
);
const result = await searchProvider.search(SearchTable.block, {
_source: ['workspace_id', 'doc_id'],
query: {
bool: {
must: [
{ term: { workspace_id: { value: workspaceId } } },
{ match: { content: 'へ' } },
],
},
},
fields: ['block_id', 'content'],
sort: ['_score'],
});
t.true(result.total >= 1);
t.snapshot(
result.nodes
.filter(node => node.fields.block_id?.[0] === blockId)
.map(node => omit(node, ['_score']))
);
});
// #region write
test('should write document work', async t => {
@@ -189,7 +318,7 @@ test('should write document work', async t => {
let result = await searchProvider.search(SearchTable.block, {
_source: ['workspace_id', 'doc_id'],
query: { match: { doc_id: docId } },
query: { term: { doc_id: { value: docId } } },
fields: [
'flavour',
'flavour_indexed',
@@ -232,7 +361,7 @@ test('should write document work', async t => {
result = await searchProvider.search(SearchTable.block, {
_source: ['workspace_id', 'doc_id'],
query: { match: { doc_id: docId } },
query: { term: { doc_id: { value: docId } } },
fields: ['flavour', 'block_id', 'content', 'ref_doc_id'],
sort: ['_score'],
});
@@ -263,7 +392,7 @@ test('should write document work', async t => {
result = await searchProvider.search(SearchTable.block, {
_source: ['workspace_id', 'doc_id'],
query: { match: { doc_id: docId } },
query: { term: { doc_id: { value: docId } } },
fields: ['flavour', 'block_id', 'content', 'ref_doc_id'],
sort: ['_score'],
});
@@ -319,8 +448,8 @@ test('should handle ref_doc_id as string[]', async t => {
query: {
bool: {
must: [
{ match: { workspace_id: workspaceId } },
{ match: { doc_id: docId } },
{ term: { workspace_id: { value: workspaceId } } },
{ term: { doc_id: { value: docId } } },
],
},
},
@@ -371,8 +500,8 @@ test('should handle ref_doc_id as string[]', async t => {
query: {
bool: {
must: [
{ match: { workspace_id: workspaceId } },
{ match: { doc_id: docId } },
{ term: { workspace_id: { value: workspaceId } } },
{ term: { doc_id: { value: docId } } },
],
},
},
@@ -416,8 +545,8 @@ test('should handle content as string[]', async t => {
query: {
bool: {
must: [
{ match: { workspace_id: workspaceId } },
{ match: { doc_id: docId } },
{ term: { workspace_id: { value: workspaceId } } },
{ term: { doc_id: { value: docId } } },
],
},
},
@@ -455,8 +584,8 @@ test('should handle content as string[]', async t => {
query: {
bool: {
must: [
{ match: { workspace_id: workspaceId } },
{ match: { doc_id: docId } },
{ term: { workspace_id: { value: workspaceId } } },
{ term: { doc_id: { value: docId } } },
],
},
},
@@ -497,8 +626,8 @@ test('should handle blob as string[]', async t => {
query: {
bool: {
must: [
{ match: { workspace_id: workspaceId } },
{ match: { doc_id: docId } },
{ term: { workspace_id: { value: workspaceId } } },
{ term: { doc_id: { value: docId } } },
],
},
},
@@ -534,8 +663,8 @@ test('should handle blob as string[]', async t => {
query: {
bool: {
must: [
{ match: { workspace_id: workspaceId } },
{ match: { doc_id: docId } },
{ term: { workspace_id: { value: workspaceId } } },
{ term: { doc_id: { value: docId } } },
],
},
},
@@ -571,8 +700,8 @@ test('should handle blob as string[]', async t => {
query: {
bool: {
must: [
{ match: { workspace_id: workspaceId } },
{ match: { doc_id: docId } },
{ term: { workspace_id: { value: workspaceId } } },
{ term: { doc_id: { value: docId } } },
],
},
},
@@ -682,8 +811,10 @@ test('should search query all and get next cursor work', async t => {
'id',
],
query: {
match: {
workspace_id: workspaceId,
term: {
workspace_id: {
value: workspaceId,
},
},
},
fields: ['flavour', 'workspace_id', 'doc_id', 'block_id'],
@@ -708,8 +839,10 @@ test('should search query all and get next cursor work', async t => {
'id',
],
query: {
match: {
workspace_id: workspaceId,
term: {
workspace_id: {
value: workspaceId,
},
},
},
fields: ['flavour', 'workspace_id', 'doc_id', 'block_id'],
@@ -734,8 +867,10 @@ test('should search query all and get next cursor work', async t => {
'id',
],
query: {
match: {
workspace_id: workspaceId,
term: {
workspace_id: {
value: workspaceId,
},
},
},
fields: ['flavour', 'workspace_id', 'doc_id', 'block_id'],
@@ -780,16 +915,20 @@ test('should filter by workspace_id work', async t => {
bool: {
must: [
{
match: {
workspace_id: workspaceId,
term: {
workspace_id: {
value: workspaceId,
},
},
},
{
bool: {
must: [
{
match: {
doc_id: docId,
term: {
doc_id: {
value: docId,
},
},
},
],

View File

@@ -8,11 +8,12 @@ import { createModule } from '../../../__tests__/create-module';
import { Mockers } from '../../../__tests__/mocks';
import { ConfigModule } from '../../../base/config';
import { ServerConfigModule } from '../../../core/config';
import { Models } from '../../../models';
import { SearchProviderFactory } from '../factory';
import { IndexerModule, IndexerService } from '../index';
import { ManticoresearchProvider } from '../providers';
import { UpsertDoc } from '../service';
import { SearchTable } from '../tables';
import { blockSQL, docSQL, SearchTable } from '../tables';
import {
AggregateInput,
SearchInput,
@@ -35,6 +36,7 @@ const module = await createModule({
const indexerService = module.get(IndexerService);
const searchProviderFactory = module.get(SearchProviderFactory);
const manticoresearch = module.get(ManticoresearchProvider);
const models = module.get(Models);
const user = await module.create(Mockers.User);
const workspace = await module.create(Mockers.Workspace, {
snapshot: true,
@@ -50,7 +52,8 @@ test.after.always(async () => {
});
test.before(async () => {
await indexerService.createTables();
await manticoresearch.recreateTable(SearchTable.block, blockSQL);
await manticoresearch.recreateTable(SearchTable.doc, docSQL);
});
test.afterEach.always(async () => {
@@ -2311,3 +2314,29 @@ test('should search docs by keyword work', async t => {
});
// #endregion
test('should rebuild manticore indexes and requeue workspaces', async t => {
const workspace1 = await module.create(Mockers.Workspace, {
indexed: true,
});
const workspace2 = await module.create(Mockers.Workspace, {
indexed: true,
});
const queueCount = module.queue.count('indexer.indexWorkspace');
await indexerService.rebuildManticoreIndexes();
const queuedWorkspaceIds = new Set(
module.queue.add
.getCalls()
.filter(call => call.args[0] === 'indexer.indexWorkspace')
.slice(queueCount)
.map(call => call.args[1].workspaceId)
);
t.true(queuedWorkspaceIds.has(workspace1.id));
t.true(queuedWorkspaceIds.has(workspace2.id));
t.is((await models.workspace.get(workspace1.id))?.indexed, false);
t.is((await models.workspace.get(workspace2.id))?.indexed, false);
});

View File

@@ -38,6 +38,17 @@ const SupportIndexedAttributes = [
'parent_block_id',
];
const SupportExactTermFields = new Set([
'workspace_id',
'doc_id',
'block_id',
'flavour',
'parent_flavour',
'parent_block_id',
'created_by_user_id',
'updated_by_user_id',
]);
const ConvertEmptyStringToNullValueFields = new Set([
'ref_doc_id',
'ref',
@@ -55,23 +66,20 @@ export class ManticoresearchProvider extends ElasticsearchProvider {
table: SearchTable,
mapping: string
): Promise<void> {
const url = `${this.config.provider.endpoint}/cli`;
const response = await fetch(url, {
method: 'POST',
body: mapping,
headers: {
'Content-Type': 'text/plain',
},
});
// manticoresearch cli response is not json, so we need to handle it manually
const text = (await response.text()).trim();
if (!response.ok) {
this.logger.error(`failed to create table ${table}, response: ${text}`);
throw new InternalServerError();
}
const text = await this.#executeSQL(mapping);
this.logger.log(`created table ${table}, response: ${text}`);
}
async dropTable(table: SearchTable): Promise<void> {
const text = await this.#executeSQL(`DROP TABLE IF EXISTS ${table}`);
this.logger.log(`dropped table ${table}, response: ${text}`);
}
async recreateTable(table: SearchTable, mapping: string): Promise<void> {
await this.dropTable(table);
await this.createTable(table, mapping);
}
override async write(
table: SearchTable,
documents: Record<string, unknown>[],
@@ -252,6 +260,12 @@ export class ManticoresearchProvider extends ElasticsearchProvider {
// 1750389254 => new Date(1750389254 * 1000)
return new Date(value * 1000);
}
if (value && typeof value === 'string') {
const timestamp = Date.parse(value);
if (!Number.isNaN(timestamp)) {
return new Date(timestamp);
}
}
return value;
}
@@ -302,8 +316,10 @@ export class ManticoresearchProvider extends ElasticsearchProvider {
// workspace_id: 'workspaceId1'
// }
// }
let termField = options?.termMappingField ?? 'term';
let field = Object.keys(query.term)[0];
let termField =
options?.termMappingField ??
(SupportExactTermFields.has(field) ? 'equals' : 'term');
let value = query.term[field];
if (typeof value === 'object' && 'value' in value) {
if ('boost' in value) {
@@ -432,4 +448,28 @@ export class ManticoresearchProvider extends ElasticsearchProvider {
}
return value;
}
async #executeSQL(sql: string) {
const url = `${this.config.provider.endpoint}/cli`;
const headers: Record<string, string> = {
'Content-Type': 'text/plain',
};
if (this.config.provider.apiKey) {
headers.Authorization = `ApiKey ${this.config.provider.apiKey}`;
} else if (this.config.provider.password) {
headers.Authorization = `Basic ${Buffer.from(`${this.config.provider.username}:${this.config.provider.password}`).toString('base64')}`;
}
const response = await fetch(url, {
method: 'POST',
body: sql,
headers,
});
const text = (await response.text()).trim();
if (!response.ok) {
this.logger.error(`failed to execute SQL "${sql}", response: ${text}`);
throw new InternalServerError();
}
return text;
}
}

View File

@@ -14,6 +14,7 @@ import {
AggregateQueryDSL,
BaseQueryDSL,
HighlightDSL,
ManticoresearchProvider,
OperationOptions,
SearchNode,
SearchProvider,
@@ -130,6 +131,63 @@ export class IndexerService {
}
}
async rebuildManticoreIndexes() {
let searchProvider: SearchProvider | undefined;
try {
searchProvider = this.factory.get();
} catch (err) {
if (err instanceof SearchProviderNotFound) {
this.logger.debug('No search provider found, skip rebuilding tables');
return;
}
throw err;
}
if (!(searchProvider instanceof ManticoresearchProvider)) {
this.logger.debug(
`Search provider ${searchProvider.type} does not need manticore rebuild`
);
return;
}
const mappings = SearchTableMappingStrings[searchProvider.type];
for (const table of Object.keys(mappings) as SearchTable[]) {
await searchProvider.recreateTable(table, mappings[table]);
}
let lastWorkspaceSid = 0;
while (true) {
const workspaces = await this.models.workspace.list(
{ sid: { gt: lastWorkspaceSid } },
{ id: true, sid: true },
100
);
if (!workspaces.length) {
break;
}
for (const workspace of workspaces) {
await this.models.workspace.update(
workspace.id,
{ indexed: false },
false
);
await this.queue.add(
'indexer.indexWorkspace',
{
workspaceId: workspace.id,
},
{
jobId: `indexWorkspace/${workspace.id}`,
priority: 100,
}
);
}
lastWorkspaceSid = workspaces[workspaces.length - 1].sid;
}
}
async write<T extends SearchTable>(
table: T,
documents: UpsertTypeByTable<T>[],

View File

@@ -150,6 +150,8 @@ CREATE TABLE IF NOT EXISTS block (
updated_at timestamp
)
morphology = 'jieba_chinese, lemmatize_en_all, lemmatize_de_all, lemmatize_ru_all, libstemmer_ar, libstemmer_ca, stem_cz, libstemmer_da, libstemmer_nl, libstemmer_fi, libstemmer_fr, libstemmer_el, libstemmer_hi, libstemmer_hu, libstemmer_id, libstemmer_ga, libstemmer_it, libstemmer_lt, libstemmer_ne, libstemmer_no, libstemmer_pt, libstemmer_ro, libstemmer_es, libstemmer_sv, libstemmer_ta, libstemmer_tr'
charset_table = 'non_cjk, cjk'
charset_table = 'non_cjk, chinese'
ngram_len = '1'
ngram_chars = 'U+1100..U+11FF, U+3130..U+318F, U+A960..U+A97F, U+AC00..U+D7AF, U+D7B0..U+D7FF, U+3040..U+30FF, U+0E00..U+0E7F'
index_field_lengths = '1'
`;

View File

@@ -109,6 +109,8 @@ CREATE TABLE IF NOT EXISTS doc (
updated_at timestamp
)
morphology = 'jieba_chinese, lemmatize_en_all, lemmatize_de_all, lemmatize_ru_all, libstemmer_ar, libstemmer_ca, stem_cz, libstemmer_da, libstemmer_nl, libstemmer_fi, libstemmer_fr, libstemmer_el, libstemmer_hi, libstemmer_hu, libstemmer_id, libstemmer_ga, libstemmer_it, libstemmer_lt, libstemmer_ne, libstemmer_no, libstemmer_pt, libstemmer_ro, libstemmer_es, libstemmer_sv, libstemmer_ta, libstemmer_tr'
charset_table = 'non_cjk, cjk'
charset_table = 'non_cjk, chinese'
ngram_len = '1'
ngram_chars = 'U+1100..U+11FF, U+3130..U+318F, U+A960..U+A97F, U+AC00..U+D7AF, U+D7B0..U+D7FF, U+3040..U+30FF, U+0E00..U+0E7F'
index_field_lengths = '1'
`;

View File

@@ -18,7 +18,7 @@
"@affine/templates": "workspace:*",
"@preact/signals-core": "^1.8.0",
"eventemitter2": "^6.4.9",
"foxact": "^0.2.49",
"foxact": "^0.3.0",
"fractional-indexing": "^3.2.0",
"fuse.js": "^7.0.0",
"graphemer": "^1.4.0",

View File

@@ -74,6 +74,12 @@ dependencies = [
"windows-sys",
]
[[package]]
name = "anyhow"
version = "1.0.102"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c"
[[package]]
name = "arbitrary"
version = "1.4.1"
@@ -168,6 +174,17 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "613afe47fcd5fac7ccf1db93babcb082c5994d996f20b8b159f2ad1658eb5724"
[[package]]
name = "chacha20"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6f8d983286843e49675a4b7a2d174efe136dc93a18d69130dd18198a6c167601"
dependencies = [
"cfg-if",
"cpufeatures",
"rand_core 0.10.0",
]
[[package]]
name = "clap"
version = "4.5.38"
@@ -224,6 +241,15 @@ dependencies = [
"loom",
]
[[package]]
name = "cpufeatures"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b2a41393f66f16b0823bb79094d54ac5fbd34ab292ddafb9a0456ac9f87d201"
dependencies = [
"libc",
]
[[package]]
name = "crossbeam-utils"
version = "0.8.21"
@@ -238,7 +264,7 @@ checksum = "5041cc499144891f3790297212f32a74fb938e5136a14943f338ef9e0ae276cf"
dependencies = [
"cfg-if",
"crossbeam-utils",
"hashbrown",
"hashbrown 0.14.5",
"lock_api",
"once_cell",
"parking_lot_core",
@@ -255,6 +281,12 @@ dependencies = [
"syn",
]
[[package]]
name = "equivalent"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]]
name = "event-listener"
version = "5.4.0"
@@ -286,6 +318,12 @@ dependencies = [
"getrandom 0.2.16",
]
[[package]]
name = "foldhash"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2"
[[package]]
name = "generator"
version = "0.8.4"
@@ -320,22 +358,69 @@ checksum = "26145e563e54f2cadc477553f1ec5ee650b00862f0a58bcd12cbdc5f0ea2d2f4"
dependencies = [
"cfg-if",
"libc",
"r-efi",
"r-efi 5.2.0",
"wasi 0.14.2+wasi-0.2.4",
]
[[package]]
name = "getrandom"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0de51e6874e94e7bf76d726fc5d13ba782deca734ff60d5bb2fb2607c7406555"
dependencies = [
"cfg-if",
"libc",
"r-efi 6.0.0",
"rand_core 0.10.0",
"wasip2",
"wasip3",
]
[[package]]
name = "hashbrown"
version = "0.14.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1"
[[package]]
name = "hashbrown"
version = "0.15.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1"
dependencies = [
"foldhash",
]
[[package]]
name = "hashbrown"
version = "0.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "841d1cc9bed7f9236f321df977030373f4a4163ae1a7dbfe1a51a2c1a51d9100"
[[package]]
name = "heck"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
[[package]]
name = "id-arena"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954"
[[package]]
name = "indexmap"
version = "2.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7714e70437a7dc3ac8eb7e6f8df75fd8eb422675fc7678aff7364301092b1017"
dependencies = [
"equivalent",
"hashbrown 0.16.1",
"serde",
"serde_core",
]
[[package]]
name = "is_terminal_polyfill"
version = "1.70.1"
@@ -374,6 +459,12 @@ version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe"
[[package]]
name = "leb128fmt"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09edd9e8b54e49e587e4f6295a7d29c3ea94d469cb40ab8ca70b288248a81db2"
[[package]]
name = "lib0"
version = "0.16.10"
@@ -393,9 +484,9 @@ checksum = "d750af042f7ef4f724306de029d18836c26c1765a54a6a3f094cbd23a7267ffa"
[[package]]
name = "libfuzzer-sys"
version = "0.4.9"
version = "0.4.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf78f52d400cf2d84a3a973a78a592b4adc535739e0a5597a0da6f0c357adc75"
checksum = "f12a681b7dd8ce12bff52488013ba614b869148d54dd79836ab85aafdd53f08d"
dependencies = [
"arbitrary",
"cc",
@@ -530,29 +621,30 @@ dependencies = [
[[package]]
name = "phf"
version = "0.11.3"
version = "0.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fd6780a80ae0c52cc120a26a1a42c1ae51b247a253e4e06113d23d2c2edd078"
checksum = "c1562dc717473dbaa4c1f85a36410e03c047b2e7df7f45ee938fbef64ae7fadf"
dependencies = [
"phf_macros",
"phf_shared",
"serde",
]
[[package]]
name = "phf_generator"
version = "0.11.3"
version = "0.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3c80231409c20246a13fddb31776fb942c38553c51e871f8cbd687a4cfb5843d"
checksum = "135ace3a761e564ec88c03a77317a7c6b80bb7f7135ef2544dbe054243b89737"
dependencies = [
"fastrand",
"phf_shared",
"rand 0.8.5",
]
[[package]]
name = "phf_macros"
version = "0.11.3"
version = "0.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f84ac04429c13a7ff43785d75ad27569f2951ce0ffd30a3321230db2fc727216"
checksum = "812f032b54b1e759ccd5f8b6677695d5268c588701effba24601f6932f8269ef"
dependencies = [
"phf_generator",
"phf_shared",
@@ -563,9 +655,9 @@ dependencies = [
[[package]]
name = "phf_shared"
version = "0.11.3"
version = "0.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "67eabc2ef2a60eb7faa00097bd1ffdb5bd28e62bf39990626a582201b7a754e5"
checksum = "e57fef6bc5981e38c2ce2d63bfa546861309f875b8a75f092d1d54ae2d64f266"
dependencies = [
"siphasher",
]
@@ -585,6 +677,16 @@ dependencies = [
"zerocopy",
]
[[package]]
name = "prettyplease"
version = "0.2.37"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b"
dependencies = [
"proc-macro2",
"syn",
]
[[package]]
name = "proc-macro2"
version = "1.0.95"
@@ -609,6 +711,12 @@ version = "5.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "74765f6d916ee2faa39bc8e68e4f3ed8949b48cccdac59983d287a7cb71ce9c5"
[[package]]
name = "r-efi"
version = "6.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf"
[[package]]
name = "rand"
version = "0.8.5"
@@ -622,12 +730,13 @@ dependencies = [
[[package]]
name = "rand"
version = "0.9.1"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9fbfd9d094a40bf3ae768db9361049ace4c0e04a4fd6b359518bd7b73a73dd97"
checksum = "bc266eb313df6c5c09c1c7b1fbe2510961e5bcd3add930c1e31f7ed9da0feff8"
dependencies = [
"rand_chacha 0.9.0",
"rand_core 0.9.3",
"chacha20",
"getrandom 0.4.2",
"rand_core 0.10.0",
]
[[package]]
@@ -642,12 +751,12 @@ dependencies = [
[[package]]
name = "rand_chacha"
version = "0.9.0"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d3022b5f1df60f26e1ffddd6c66e8aa15de382ae63b3a0c1bfc0e4d3e3f325cb"
checksum = "3e6af7f3e25ded52c41df4e0b1af2d047e45896c2f3281792ed68a1c243daedb"
dependencies = [
"ppv-lite86",
"rand_core 0.9.3",
"rand_core 0.10.0",
]
[[package]]
@@ -661,21 +770,18 @@ dependencies = [
[[package]]
name = "rand_core"
version = "0.9.3"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "99d9a13982dcf210057a8a78572b2217b667c3beacbf3a0d8b454f6f82837d38"
dependencies = [
"getrandom 0.3.3",
]
checksum = "0c8d0fd677905edcbeedbf2edb6494d676f0e98d54d5cf9bda0b061cb8fb8aba"
[[package]]
name = "rand_distr"
version = "0.5.1"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6a8615d50dcf34fa31f7ab52692afec947c4dd0ab803cc87cb3b0b4570ff7463"
checksum = "4d431c2703ccf129de4d45253c03f49ebb22b97d6ad79ee3ecfc7e3f4862c1d8"
dependencies = [
"num-traits",
"rand 0.9.1",
"rand 0.10.0",
]
[[package]]
@@ -729,19 +835,35 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49"
[[package]]
name = "serde"
version = "1.0.219"
name = "semver"
version = "1.0.27"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5f0e2c6ed6606019b4e29e69dbaba95b11854410e5347d525002456dbbb786b6"
checksum = "d767eb0aabc880b29956c35734170f26ed551a859dbd361d140cdbeca61ab1e2"
[[package]]
name = "serde"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e"
dependencies = [
"serde_core",
"serde_derive",
]
[[package]]
name = "serde_core"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.219"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5b0276cf7f2c73365f7157c8123c21cd9a50fbbd844757af28ca1f5925fc2a00"
checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79"
dependencies = [
"proc-macro2",
"quote",
@@ -814,9 +936,9 @@ checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f"
[[package]]
name = "syn"
version = "2.0.101"
version = "2.0.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8ce2b7fc941b3a24138a0a7cf8e858bfc6a992e7978a068a5c760deb0ed43caf"
checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99"
dependencies = [
"proc-macro2",
"quote",
@@ -928,6 +1050,12 @@ version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a5f39404a5da50712a4c1eecf25e90dd62b613502b7e925fd4e4d19b5c96512"
[[package]]
name = "unicode-xid"
version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853"
[[package]]
name = "utf8parse"
version = "0.2.2"
@@ -961,6 +1089,24 @@ dependencies = [
"wit-bindgen-rt",
]
[[package]]
name = "wasip2"
version = "1.0.2+wasi-0.2.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9517f9239f02c069db75e65f174b3da828fe5f5b945c4dd26bd25d89c03ebcf5"
dependencies = [
"wit-bindgen",
]
[[package]]
name = "wasip3"
version = "0.4.0+wasi-0.3.0-rc-2026-01-06"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5428f8bf88ea5ddc08faddef2ac4a67e390b88186c703ce6dbd955e1c145aca5"
dependencies = [
"wit-bindgen",
]
[[package]]
name = "wasm-bindgen"
version = "0.2.100"
@@ -1018,6 +1164,40 @@ dependencies = [
"unicode-ident",
]
[[package]]
name = "wasm-encoder"
version = "0.244.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "990065f2fe63003fe337b932cfb5e3b80e0b4d0f5ff650e6985b1048f62c8319"
dependencies = [
"leb128fmt",
"wasmparser",
]
[[package]]
name = "wasm-metadata"
version = "0.244.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bb0e353e6a2fbdc176932bbaab493762eb1255a7900fe0fea1a2f96c296cc909"
dependencies = [
"anyhow",
"indexmap",
"wasm-encoder",
"wasmparser",
]
[[package]]
name = "wasmparser"
version = "0.244.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "47b807c72e1bac69382b3a6fb3dbe8ea4c0ed87ff5629b8685ae6b9a611028fe"
dependencies = [
"bitflags",
"hashbrown 0.15.5",
"indexmap",
"semver",
]
[[package]]
name = "windows"
version = "0.58.0"
@@ -1155,6 +1335,26 @@ version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec"
[[package]]
name = "wit-bindgen"
version = "0.51.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d7249219f66ced02969388cf2bb044a09756a083d0fab1e566056b04d9fbcaa5"
dependencies = [
"wit-bindgen-rust-macro",
]
[[package]]
name = "wit-bindgen-core"
version = "0.51.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ea61de684c3ea68cb082b7a88508a8b27fcc8b797d738bfc99a82facf1d752dc"
dependencies = [
"anyhow",
"heck",
"wit-parser",
]
[[package]]
name = "wit-bindgen-rt"
version = "0.39.0"
@@ -1164,6 +1364,74 @@ dependencies = [
"bitflags",
]
[[package]]
name = "wit-bindgen-rust"
version = "0.51.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b7c566e0f4b284dd6561c786d9cb0142da491f46a9fbed79ea69cdad5db17f21"
dependencies = [
"anyhow",
"heck",
"indexmap",
"prettyplease",
"syn",
"wasm-metadata",
"wit-bindgen-core",
"wit-component",
]
[[package]]
name = "wit-bindgen-rust-macro"
version = "0.51.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c0f9bfd77e6a48eccf51359e3ae77140a7f50b1e2ebfe62422d8afdaffab17a"
dependencies = [
"anyhow",
"prettyplease",
"proc-macro2",
"quote",
"syn",
"wit-bindgen-core",
"wit-bindgen-rust",
]
[[package]]
name = "wit-component"
version = "0.244.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9d66ea20e9553b30172b5e831994e35fbde2d165325bec84fc43dbf6f4eb9cb2"
dependencies = [
"anyhow",
"bitflags",
"indexmap",
"log",
"serde",
"serde_derive",
"serde_json",
"wasm-encoder",
"wasm-metadata",
"wasmparser",
"wit-parser",
]
[[package]]
name = "wit-parser"
version = "0.244.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ecc8ac4bc1dc3381b7f59c34f00b67e18f910c2c0f50015669dde7def656a736"
dependencies = [
"anyhow",
"id-arena",
"indexmap",
"log",
"semver",
"serde",
"serde_derive",
"serde_json",
"unicode-xid",
"wasmparser",
]
[[package]]
name = "y-octo"
version = "0.0.2"
@@ -1177,8 +1445,8 @@ dependencies = [
"nanoid",
"nom",
"ordered-float",
"rand 0.9.1",
"rand_chacha 0.9.0",
"rand 0.10.0",
"rand_chacha 0.10.0",
"rand_distr",
"serde",
"serde_json",
@@ -1192,8 +1460,8 @@ version = "0.0.0"
dependencies = [
"lib0",
"libfuzzer-sys",
"rand 0.9.1",
"rand_chacha 0.9.0",
"rand 0.10.0",
"rand_chacha 0.10.0",
"y-octo",
"y-octo-utils",
"yrs",
@@ -1207,17 +1475,17 @@ dependencies = [
"clap",
"lib0",
"phf",
"rand 0.9.1",
"rand_chacha 0.9.0",
"rand 0.10.0",
"rand_chacha 0.10.0",
"y-octo",
"yrs",
]
[[package]]
name = "yrs"
version = "0.23.1"
version = "0.25.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4a7cab84724ae7f361a8c92465f5160922cbb941a499e1a8cacd103351ab9c78"
checksum = "f6893d39bc55d014e4a1d0e71d06c0c41590d5cdeac35c126be44998bc320cff"
dependencies = [
"arc-swap",
"async-lock",
@@ -1228,7 +1496,7 @@ dependencies = [
"serde_json",
"smallstr",
"smallvec",
"thiserror 1.0.69",
"thiserror 2.0.12",
]
[[package]]

View File

@@ -10,9 +10,9 @@ version = "0.0.0"
[dependencies]
lib0 = "=0.16.10"
libfuzzer-sys = "0.4"
rand = "0.9"
rand_chacha = "0.9"
yrs = "=0.23.1"
rand = "0.10"
rand_chacha = "0.10"
yrs = "=0.25.0"
y-octo-utils = { path = "..", features = ["fuzz"] }

View File

@@ -45,7 +45,7 @@
"embla-carousel-react": "^8.5.1",
"input-otp": "^1.4.1",
"lodash-es": "^4.17.23",
"lucide-react": "^0.508.0",
"lucide-react": "^0.577.0",
"next-themes": "^0.4.4",
"react": "^19.2.1",
"react-day-picker": "^9.4.3",

View File

@@ -9,7 +9,7 @@ buildscript {
mavenCentral()
}
dependencies {
classpath 'com.android.tools.build:gradle:8.10.0'
classpath 'com.android.tools.build:gradle:8.13.2'
}
}

View File

@@ -1,44 +1,44 @@
[versions]
android-gradle-plugin = "8.10.0"
androidx-activity-compose = "1.10.1"
androidx-appcompat = "1.7.0"
androidx-browser = "1.8.0"
androidx-compose-bom = "2025.05.00"
android-gradle-plugin = "8.13.2"
androidx-activity-compose = "1.13.0"
androidx-appcompat = "1.7.1"
androidx-browser = "1.9.0"
androidx-compose-bom = "2025.12.01"
androidx-coordinatorlayout = "1.3.0"
androidx-core-ktx = "1.16.0"
androidx-core-splashscreen = "1.0.1"
androidx-datastore-preferences = "1.2.0-alpha02"
androidx-espresso-core = "3.6.1"
androidx-junit = "1.2.1"
androidx-lifecycle-compose = "2.9.0"
androidx-core-ktx = "1.18.0"
androidx-core-splashscreen = "1.2.0"
androidx-datastore-preferences = "1.2.1"
androidx-espresso-core = "3.7.0"
androidx-junit = "1.3.0"
androidx-lifecycle-compose = "2.10.0"
androidx-material3 = "1.3.1"
androidx-navigation = "2.9.0"
apollo = "4.2.0"
apollo-kotlin-adapters = "0.0.6"
androidx-navigation = "2.9.7"
apollo = "4.4.2"
apollo-kotlin-adapters = "0.7.0"
# @keep
compileSdk = "36"
firebase-bom = "33.13.0"
firebase-crashlytics = "3.0.3"
google-services = "4.4.2"
gradle-versions = "0.52.0"
hilt = "2.56.2"
hilt-ext = "1.2.0"
jna = "5.17.0"
firebase-bom = "33.16.0"
firebase-crashlytics = "3.0.6"
google-services = "4.4.4"
gradle-versions = "0.53.0"
hilt = "2.59.2"
hilt-ext = "1.3.0"
jna = "5.18.1"
junit = "4.13.2"
kotlin = "2.1.20"
kotlin = "2.3.20"
kotlinx-coroutines = "1.10.2"
kotlinx-datetime = "0.6.2"
kotlinx-serialization-json = "1.8.1"
ksp = "2.1.20-2.0.1"
kotlinx-datetime = "0.7.1-0.6.x-compat"
kotlinx-serialization-json = "1.10.0"
ksp = "2.3.6"
# @keep
minSdk = "23"
mozilla-rust-android = "0.9.6"
okhttp-bom = "5.0.0-alpha.14"
richtext = "1.0.0-alpha02"
okhttp-bom = "5.3.2"
richtext = "1.0.0-alpha03"
# @keep
targetSdk = "35"
timber = "5.0.1"
version-catalog-update = "1.0.0"
version-catalog-update = "1.1.0"
[libraries]
android-gradle-plugin = { module = "com.android.tools.build:gradle", version.ref = "android-gradle-plugin" }

View File

@@ -1,6 +1,6 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-8.14-all.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-8.14.4-all.zip
networkTimeout=10000
validateDistributionUrl=true
zipStoreBase=GRADLE_USER_HOME

View File

@@ -62,7 +62,7 @@
"electron-log": "^5.4.3",
"electron-squirrel-startup": "1.0.1",
"electron-window-state": "^5.0.3",
"esbuild": "^0.25.0",
"esbuild": "^0.27.0",
"fs-extra": "^11.2.0",
"glob": "^11.0.0",
"lodash-es": "^4.17.23",

View File

@@ -77,8 +77,8 @@
"kind" : "remoteSourceControl",
"location" : "https://github.com/RevenueCat/purchases-ios-spm.git",
"state" : {
"revision" : "2913a336eb37dc06795cdbaa5b5de330b6707669",
"version" : "5.65.0"
"revision" : "abb0d68c3e7ba97b16ab51c38fcaca16b0e358c8",
"version" : "5.66.0"
}
},
{
@@ -113,8 +113,8 @@
"kind" : "remoteSourceControl",
"location" : "https://github.com/apple/swift-collections",
"state" : {
"revision" : "8d9834a6189db730f6264db7556a7ffb751e99ee",
"version" : "1.4.0"
"revision" : "6675bc0ff86e61436e615df6fc5174e043e57924",
"version" : "1.4.1"
}
},
{

View File

@@ -17,7 +17,7 @@ let package = Package(
],
dependencies: [
.package(path: "../AffineResources"),
.package(url: "https://github.com/RevenueCat/purchases-ios-spm.git", from: "5.60.0"),
.package(url: "https://github.com/RevenueCat/purchases-ios-spm.git", from: "5.66.0"),
],
targets: [
.target(

View File

@@ -17,7 +17,7 @@ let package = Package(
.package(path: "../AffineGraphQL"),
.package(path: "../AffineResources"),
.package(url: "https://github.com/apollographql/apollo-ios.git", from: "1.23.0"),
.package(url: "https://github.com/apple/swift-collections.git", from: "1.4.0"),
.package(url: "https://github.com/apple/swift-collections.git", from: "1.4.1"),
.package(url: "https://github.com/SnapKit/SnapKit.git", from: "5.7.1"),
.package(url: "https://github.com/SwifterSwift/SwifterSwift.git", from: "6.2.0"),
.package(url: "https://github.com/Recouse/EventSource.git", from: "0.1.7"),

View File

@@ -53,7 +53,7 @@
"dayjs": "^1.11.13",
"emoji-mart": "^5.6.0",
"emojibase-data": "^16.0.3",
"foxact": "^0.2.49",
"foxact": "^0.3.0",
"jotai": "^2.10.3",
"lit": "^3.2.1",
"lodash-es": "^4.17.23",
@@ -62,7 +62,7 @@
"nanoid": "^5.1.6",
"next-themes": "^0.4.4",
"react": "^19.2.1",
"react-dom": "19.2.1",
"react-dom": "19.2.4",
"react-paginate": "^8.3.0",
"react-router-dom": "^6.30.3",
"react-transition-state": "^2.2.0",

View File

@@ -62,7 +62,7 @@
"eventemitter2": "^6.4.9",
"file-type": "^21.0.0",
"filesize": "^10.1.6",
"foxact": "^0.2.49",
"foxact": "^0.3.0",
"fuse.js": "^7.0.0",
"graphemer": "^1.4.0",
"graphql": "^16.9.0",
@@ -72,7 +72,7 @@
"image-blob-reduce": "^4.1.0",
"is-svg": "^6.1.0",
"jotai": "^2.10.3",
"jotai-scope": "^0.7.2",
"jotai-scope": "^0.10.0",
"katex": "^0.16.27",
"lib0": "^0.2.114",
"lit": "^3.2.1",

View File

@@ -25,7 +25,7 @@
]
},
"devDependencies": {
"@napi-rs/cli": "3.5.0",
"@napi-rs/cli": "3.5.1",
"@napi-rs/whisper": "^0.0.4",
"@types/node": "^22.0.0",
"ava": "^7.0.0",

View File

@@ -38,7 +38,7 @@
"postcss-loader": "^8.1.1",
"postcss-selector-parser": "^7.1.0",
"prettier": "^3.7.4",
"react-refresh": "^0.17.0",
"react-refresh": "^0.18.0",
"source-map-loader": "^5.0.0",
"style-loader": "^4.0.0",
"swc-loader": "^0.2.6",

4241
yarn.lock

File diff suppressed because it is too large Load Diff