Commit Graph

1121 Commits

Author SHA1 Message Date
DarkSky
3b4b0bad22 feat: improve admin panel styles (#14318) 2026-01-27 04:44:21 +08:00
JustAnDK
ecc98573eb fix: safe cookie parsing (#14292)
# Summary
This PR fixes a server-side cookie parsing edge case where malformed
cookie values throw `URI malformed`, causing socket.io auth to fail and
clients to get stuck in infinite workspace loading/syncing.

# Observed Behavior
- User creates a cloud-backed workspace and invites another user to it.
- Second user accepts the invite, awaits approval, and attempts to load
the workspace, getting stuck in infinite loading state.
- `api/workspaces/<id>/docs/<id>` return 404 for those users, as the
workspace they are trying to access was not synced to the server.
- Server logs show socket.io `CONNECT_ERROR` with `URI malformed`, then
connection closed.

# Confirmed Trigger
An externally-managed `auth_session` cookie containing a raw `%` symbol
causes `decodeURIComponent` to throw. This matches the observed
socket.io `CONNECT_ERROR`, explaining why some users were affected while
the rest were not.

# Root Cause
The `parseCookies` function calls `decodeURIComponent` on every cookie
key/value without guard, so when a malformed percent-encoded value is
encountered, `decodeURIComponent` throws, which bubbles into the
socket.io auth middleware, aborting the connection.

# Fix
Wrap `decodeURIComponent` calls in `try/catch`, on failure falling back
to the raw key/value.

# Testing
- Manually regenerating the bad cookie until no malformed parts are
present resolves the issue.
- With the guard in place, affected users can open shared workspaces
with sync successfully completing.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved cookie parsing robustness so malformed cookie values no
longer cause errors; the system now preserves raw cookie values when
decoding fails.

* **Tests**
* Added test coverage to ensure cookie parsing handles invalid/malformed
cookie values without throwing.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-01-26 17:24:11 +08:00
DarkSky
f373e08583 feat: refactor doc write in native (#14272) 2026-01-18 16:31:12 +08:00
realies
0da91e406e feat(server): add document write tools for mcp (#14245)
## Summary

This PR adds write capabilities to AFFiNE's MCP (Model Context Protocol)
integration, enabling external tools (Claude, GPT, etc.) to create and
modify documents programmatically.

**New MCP Tools:**
- `create_document` - Create new documents from markdown content
- `update_document` - Update document content using structural diffing
for minimal changes (preserves document history and enables real-time
collaboration)

**Implementation:**
- `markdown_to_ydoc.rs` - Converts markdown to AFFiNE-compatible y-octo
binary format
- `markdown_utils.rs` - Shared markdown parsing utilities (used by both
ydoc-to-md and md-to-ydoc)
- `update_ydoc.rs` - Structural diffing implementation for updating
existing documents
- `DocWriter` service - TypeScript service for document operations
- Exposes `markdownToDocBinary` and `updateDocBinary` via napi bindings

**Supported Markdown Elements:**
- Headings (H1-H6)
- Paragraphs
- Bullet lists and numbered lists
- Code blocks (with language detection)
- Blockquotes
- Horizontal dividers
- Todo items (checkboxes)

**y-octo Changes:**
This PR reverts the y-octo sync (ca2462f, a5b60cf) which introduced a
concurrency bug causing hangs when creating documents with many nested
block structures. It also ports the improved `get_node_index` binary
search fix from upstream that prevents divide-by-zero panics when
decoding documents.

## Test Results 

### Unit Tests (47/47 passing)

| Test Suite | Tests | Status |
|------------|-------|--------|
| markdown_to_ydoc | 16/16 |  Pass |
| markdown_utils | 11/11 |  Pass |
| update_ydoc | 13/13 |  Pass |
| delta_markdown | 2/2 |  Pass |
| affine (doc parser) | 5/5 |  Pass |

### End-to-End MCP Testing 

Tested against local AFFiNE server with real MCP client requests:

| Tool | Result | Notes |
|------|--------|-------|
| `tools/list` |  Pass | Returns all 5 tools with correct schemas |
| `create_document` |  Pass | Successfully created test documents |
| `update_document` |  Pass | Successfully updated documents with
structural diffing |
| `read_document` |  Pass | Existing tool, works correctly |
| `keyword_search` |  Pass | Existing tool, works correctly |

**E2E Test Details:**
- Started local AFFiNE server with PostgreSQL, Redis, and Manticore
- Created test user and workspace via seed/GraphQL
- Verified MCP endpoint at `/api/workspaces/:workspaceId/mcp`
- Tested JSON-RPC calls with proper SSE streaming
- Confirmed documents are stored and indexed correctly (verified via
server logs)

## Test Plan
- [x] All Rust unit tests pass (47 tests)
- [x] Native bindings build successfully (release mode)
- [x] Document creation via MCP works end-to-end
- [x] Document update via MCP works end-to-end
- [x] CodeRabbit feedback addressed
- [ ] Integration testing with Claude/GPT MCP clients

Closes #14161

---

**Requested by:** @realies  
**Key guidance from:** @darkskygit (use y-octo instead of yjs for memory
efficiency)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Create documents from Markdown: generate new documents directly from
Markdown content with automatic title extraction
* Update documents with Markdown: modify existing documents using
Markdown as the source with automatic diff calculation for efficient
updates
* Copilot integration: new tools for document creation and updates
through Copilot's interface

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-01-16 20:57:24 +08:00
DarkSky
2c5559ed0b fix: missing session fields 2026-01-16 17:26:05 +08:00
DarkSky
924d58603f chore: improve event flow (#14266) 2026-01-16 16:07:27 +08:00
DarkSky
7c24b2521a feat: reduce backend (#14251)
#### PR Dependency Tree


* **PR #14251** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Current user profile now exposes access tokens, revealed tokens, and
detailed calendar accounts/subscriptions.
* Workspace now exposes permissions, calendars, calendar events, and a
workspace-scoped blob upload part URL.
  * New document-update mutation for applying doc updates.

* **API Changes**
  * validateAppConfig is now a query (mutation deprecated).
* Several legacy top-level calendar/blob endpoints deprecated in favor
of user/workspace fields.

* **Refactor**
* Calendar, blob-upload and access-token surfaces reorganized to use
user/workspace-centric fields.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-01-14 00:01:07 +08:00
DarkSky
b331a08744 feat: native update merge (#14250)
#### PR Dependency Tree


* **PR #14250** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Backend Optimization**
  * Faster document retrieval via a native binary fetch path.
* Native-accelerated merging of document updates for improved
performance and consistency.
* **Indexing & Reliability**
* Indexing now only proceeds on valid parse results, with clearer
warnings and richer metadata on failures.
* More consistent sync behavior and enhanced diagnostic logging for
indexing operations.
* **Tests**
  * Expanded tests to cover native binary retrieval error handling.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-01-13 22:03:55 +08:00
DarkSky
0bd8160ed4 feat: init cloud calendar support (#14247)
#### PR Dependency Tree


* **PR #14247** 👈
  * **PR #14248**

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Google Calendar integration (disabled by default): link/unlink
accounts, OAuth flow, webhooks, real-time push, background sync,
workspace calendars with customizable items and date-range event
viewing.
* **GraphQL / Client**
* New queries & mutations for accounts, subscriptions, events,
providers, and workspace calendar management.
* **Localization**
* Added localized error message for calendar provider request failures.
* **Tests**
* Backend tests covering sync, webhook renewal, and error/error-recovery
scenarios.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-01-12 23:17:43 +08:00
DarkSky
e4dc82ee35 chore: bump deps (#14227)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Updated backend service dependencies to the latest stable versions for
improved performance and security.
* Upgraded UI component library dependencies to the latest minor
releases.

* **Improvements**
* Enhanced web search functionality for better search results on
standard AI models.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-01-07 13:15:17 +08:00
Yiding Jia
a11e9fe8ca feat(server): add LISTEN_ADDR env var for allowing server to listen on ipv6 (#14211)
The old code hardcoded 0.0.0.0 which means the server only listened for
ipv4 connections, making it not work on ipv6-only networks.

This change adds a LISTEN_ADDR env var which allows the server to bind
to ipv6 as well.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Server listen address is now configurable via the LISTEN_ADDR
environment variable (default: 0.0.0.0), enabling IPv4/IPv6 or
interface-specific binding.
* Configuration schemas and admin UI now expose the listen address
option so deployments can view and override it.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-01-05 09:31:47 +00:00
DarkSky
3633c75c6f feat: cleanup tables (#14203)
#### PR Dependency Tree


* **PR #14203** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* Removed deprecated database tables, enums and schema fields (cleanup
of legacy subscription, invoice, runtime settings and session expiry
data). This includes irreversible data removal for those legacy
elements.
* **Tests**
* Updated tests and test data to align with the cleaned-up schema and
removed fields.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-01-03 03:50:14 +08:00
DarkSky
9a7f8e7d4d feat: workspace level share settings (#14201)
fix #13698
2026-01-03 01:13:27 +08:00
DarkSky
60de882a30 feat: shared link list (#14200)
#### PR Dependency Tree


* **PR #14200** 👈

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

## Release Notes

* **New Features**
* Added a "Shared Links" panel to workspace management, enabling admins
to view all published documents within a workspace
* Added publication date tracking for published documents, now displayed
alongside shared links

* **Chores**
  * Removed deprecated `publicPages` field; use `publicDocs` instead

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-01-02 21:07:41 +08:00
DarkSky
0b0ae5ea0a feat: add queue management for admin panel 2026-01-01 06:13:50 +08:00
DarkSky
f745f7b669 feat: pre-aggregation workspace stats 2026-01-01 05:01:52 +08:00
DarkSky
97507e7043 fix: query type cast 2026-01-01 02:41:03 +08:00
DarkSky
91e6f3c45c feat: improve workspace index 2026-01-01 01:39:40 +08:00
DarkSky
c7b74384a4 fix: types 2025-12-31 23:55:51 +08:00
DarkSky
20c4951847 feat: improve workspace list perf 2025-12-31 22:59:52 +08:00
DarkSky
bc03fab649 chore: cleanup logs 2025-12-31 12:31:58 +08:00
DarkSky
e2adab7805 fix: table compatibility (#14185) 2025-12-30 12:01:27 +08:00
DarkSky
ff2e96d847 feat: database indexing support (#14181) 2025-12-30 05:23:09 +08:00
DarkSky
95a5e941e7 feat: improve admin panel (#14180) 2025-12-30 05:22:54 +08:00
DarkSky
e12fe9c12b fix: message handle (#14178)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Robustly sanitize session titles, messages, attachments, and embedded
data to remove invalid/null characters and prevent corrupt persistence.
* Improve chat title generation to skip or recover from invalid input
and log contextual errors without crashing.
* Add more detailed storage and workspace logs and reduce repetitive
checks to aid troubleshooting and stability.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-12-29 21:47:53 +08:00
DarkSky
6951f1002f feat: improve event handle (#14177) 2025-12-29 18:54:59 +08:00
DarkSky
4b721dffe0 feat: set admin name when self hosted init (#14146)
fix #14134
2025-12-23 23:38:34 +08:00
DarkSky
76524084d1 feat: multipart blob sync support (#14138)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Flexible blob uploads: GRAPHQL, presigned, and multipart flows with
per‑part URLs, abort/complete operations, presigned proxy endpoints, and
nightly cleanup of expired pending uploads.

* **API / Schema**
* GraphQL additions: new types, mutations, enum and error to manage
upload lifecycle (create, complete, abort, get part URL).

* **Database**
* New blob status enum and columns (status, upload_id); listing now
defaults to completed blobs.

* **Localization**
  * Added user-facing message: "Blob is invalid."

* **Tests**
* Expanded unit and end‑to‑end coverage for upload flows, proxy
behavior, multipart and provider integrations.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-12-23 22:09:21 +08:00
DarkSky
a9937e18b6 fix: cleanup expired records (#14140) 2025-12-23 22:08:57 +08:00
DarkSky
321965a424 feat: impl text delta support (#14132) 2025-12-22 03:16:16 +08:00
DarkSky
844b9d9592 feat(server): impl native reader for server (#14100) 2025-12-14 00:28:43 +08:00
Richard Lora
f832b28dac feat(editor): add date grouping configurations (#12679)
https://github.com/user-attachments/assets/d5578060-2c8c-47a5-ba65-ef2e9430518b

This PR adds the ability to group-by date with configuration which an
example is shown in the image below:


![image](https://github.com/user-attachments/assets/8762342a-999e-444e-afa2-5cfbf7e24907)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Date-based grouping modes (relative, day, week Sun/Mon, month, year),
a date group renderer, and quick lookup for group-by configs by name.

* **Improvements**
* Enhanced group settings: date sub‑modes, week‑start, per‑group
visibility, Hide All/Show All, date sort order, improved drag/drop and
reorder.
* Consistent popup placement/middleware, nested popup positioning,
per‑item close-on-select, and enforced minimum menu heights.
* UI: empty groups now display "No <property>"; views defensively handle
null/hidden groups.

* **Tests**
  * Added unit tests for date-key sorting and comparison.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Norkz <richardlora557@gmail.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-12-11 22:32:21 +00:00
DarkSky
cb0ff04efa feat: bump more deps (#14079) 2025-12-10 16:02:28 +08:00
DarkSky
40f3337d45 feat: bump deps (#14076)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* Updated core dependencies, developer tooling and Rust toolchain to
newer stable versions across the repo
* Upgraded Storybook to v10 and improved ESM path resolution for
storybook tooling
* Broadened native binding platform/architecture support and
strengthened native module version validation, loading and WASI handling

* **New Features**
* Exposed an additional native text export for consumers (enhanced
JS/native surface)

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-12-10 03:52:14 +08:00
DarkSky
027f741ed6 chore: bump deps (#14065)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* Updated dependency versions across the monorepo (notably zod →
^3.25.76 and vitest-related packages → ^3.2.4), plus minor package bumps
to align tooling and libraries. These are manifest/test-tooling updates
only; no public API, behavior, or end-user features were changed.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-12-08 21:47:25 +08:00
DarkSky
f29e47e9d2 feat: improve oauth (#14061)
fix #13730
fix #12901
fix #14025
2025-12-08 10:44:41 +08:00
DarkSky
5cac8971eb fix: apple sign (#14053) 2025-12-06 15:36:50 +08:00
DarkSky
33a014977a chore: ignore empty key 2025-11-19 13:46:11 +08:00
Altamir Benkenstein
221c493c56 feat(server): add Brazilian Portuguese translation support (#13725)
* Added 'Brazilian Portuguese' to the list of supported translation
languages in both backend and frontend.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added Brazilian Portuguese as a supported translation language across
the app.
* Brazilian Portuguese now appears in language selection for translation
actions.
* AI translation prompts now include Brazilian Portuguese as a valid
target option.
  * No other translation behaviors or controls were modified.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: DarkSky <25152247+darkskygit@users.noreply.github.com>
2025-11-18 15:09:48 +08:00
DarkSky
8544e58c01 fix: transaction timeout 2025-11-18 13:41:25 +08:00
DarkSky
62fe6982fb chore: cleanup logs 2025-11-16 11:13:56 +08:00
DarkSky
1654d8efe4 feat: improve sub sync (#13932) 2025-11-15 15:52:35 +08:00
Kevin
525b65c6ca fix(server): config typo (#13913)
Fix a typo on the SMTP configuration page
2025-11-15 14:41:03 +08:00
DarkSky
c302425a05 fix(server): doc squash timeout (#13939) 2025-11-15 03:19:51 +08:00
DarkSky
abe73e9996 fix: config escape error (#13936)
fix #13702
2025-11-14 23:24:44 +08:00
DarkSky
dd676611ce test: fix mock api 2025-11-01 01:36:11 +08:00
DarkSky
8535b3dc41 feat(server): refresh subscription if event is from anonymous account (#13840) 2025-10-31 22:53:20 +08:00
DarkSky
89cc9b072b feat(server): early subscription for iap (#13826)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added a mutation to request/apply a subscription by transaction ID
(client mutation and server operation), returning subscription details.
* **Bug Fixes / Improvements**
* More robust external subscription sync with safer conflict detection,
optional short-lived confirmation, improved parsing and error logging.
* **Chores**
  * Standardized time constants for clarity.
* **Tests**
* Updated subscription test data (expiration date) to reflect new
lifecycle expectations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-10-29 14:25:43 +00:00
DarkSky
9904f50e0b feat(server): improve crypto (#13822)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
  * Resolved cryptographic compatibility issues with OpenSSL 3.x.

* **New Features**
  * Broader support for multiple key types for signing and verification.
  * OTP generation adjusted (affects digit distribution/values).

* **Refactor**
* Internal cryptographic key handling reworked for improved stability,
compatibility, and multi-key-type support.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-10-29 18:12:23 +08:00
DarkSky
b7ac7caab4 chore(server): improve transcript stability (#13821)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Enhanced audio/video detection for MP4 files to better distinguish
audio-only vs. video.

* **Dependencies**
* Added MP4 parsing dependency and updated AI provider libraries
(Anthropic, Google, OpenAI, etc.).

* **Bug Fixes**
  * Tightened authentication state validation for magic-link/OTP flows.
* Stricter space-join validation to reject invalid client
types/versions.
  * Improved transcript entry deduplication and data handling.

* **API**
* Transcript submit payload now requires infos and removes deprecated
url/mimeType fields.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-10-29 17:48:15 +08:00