feat(server): parse ydoc to ai editable markdown format (#12846)

close AI-213









#### PR Dependency Tree


* **PR #12846** 👈
  * **PR #12811**

This tree was auto-generated by
[Charcoal](https://github.com/danerwilliams/charcoal)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Introduced support for AI-editable blocks in document parsing,
allowing blocks to include metadata for AI-based editing.
  - Added rendering for todo list items with markdown checkbox syntax.
- Unsupported block types are now marked with placeholders in the parsed
output.

- **Tests**
- Added new test cases and snapshots to verify parsing behavior with
AI-editable content enabled.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

packages/backend/server/src/core/utils/__tests__/__snapshots__/blocksute.spec.ts.md

@@ -1467,3 +1467,122 @@ Generated by [AVA](https://avajs.dev).
       `,
       title: 'Write, Draw, Plan all at Once.',
     }
+
+## can parse doc to markdown from doc snapshot with ai editable
+
+> Snapshot 1
+
+    {
+      markdown: `<!-- block_id=FoPQcAyV_m flavour=affine:paragraph -->␊
+      AFFiNE is an open source all in one workspace, an operating system for all the building blocks of your team wiki, knowledge management and digital assets and a better alternative to Notion and Miro.␊
+      ␊
+      ␊
+      <!-- block_id=oz48nn_zp8 flavour=affine:paragraph -->␊
+      ␊
+      ␊
+      ␊
+      <!-- block_id=g8a-D9-jXS flavour=affine:paragraph -->␊
+      # You own your data, with no compromises␊
+      ␊
+      ␊
+      <!-- block_id=J8lHN1GR_5 flavour=affine:paragraph -->␊
+      ## Local-first & Real-time collaborative␊
+      ␊
+      ␊
+      <!-- block_id=xCuWdM0VLz flavour=affine:paragraph -->␊
+      We love the idea proposed by Ink & Switch in the famous article about you owning your data, despite the cloud. Furthermore, AFFiNE is the first all-in-one workspace that keeps your data ownership with no compromises on real-time collaboration and editing experience.␊
+      ␊
+      ␊
+      <!-- block_id=zElMi0tViK flavour=affine:paragraph -->␊
+      AFFiNE is a local-first application upon CRDTs with real-time collaboration support. Your data is always stored locally while multiple nodes remain synced in real-time.␊
+      ␊
+      ␊
+      <!-- block_id=Z4rK0OF9Wk flavour=affine:paragraph -->␊
+      ␊
+      ␊
+      ␊
+      <!-- block_id=DQ0Ryb-SpW flavour=affine:paragraph -->␊
+      ### Blocks that assemble your next docs, tasks kanban or whiteboard␊
+      ␊
+      ␊
+      <!-- block_id=HAZC3URZp_ flavour=affine:paragraph -->␊
+      There is a large overlap of their atomic "building blocks" between these apps. They are neither open source nor have a plugin system like VS Code for contributors to customize. We want to have something that contains all the features we love and goes one step further.␊
+      ␊
+      ␊
+      <!-- block_id=0H87ypiuv8 flavour=affine:paragraph -->␊
+      We are building AFFiNE to be a fundamental open source platform that contains all the building blocks for docs, task management and visual collaboration, hoping you can shape your next workflow with us that can make your life better and also connect others, too.␊
+      ␊
+      ␊
+      <!-- block_id=Sp4G1KD0Wn flavour=affine:paragraph -->␊
+      If you want to learn more about the product design of AFFiNE, here goes the concepts:␊
+      ␊
+      ␊
+      <!-- block_id=RsUhDuEqXa flavour=affine:paragraph -->␊
+      To Shape, not to adapt. AFFiNE is built for individuals & teams who care about their data, who refuse vendor lock-in, and who want to have control over their essential tools.␊
+      ␊
+      ␊
+      <!-- block_id=Z2HibKzAr- flavour=affine:paragraph -->␊
+      ## A true canvas for blocks in any form␊
+      ␊
+      ␊
+      <!-- block_id=UwvWddamzM flavour=affine:paragraph -->␊
+      [Many editor apps](http://notion.so) claimed to be a canvas for productivity. Since _the Mother of All Demos,_ Douglas Engelbart, a creative and programable digital workspace has been a pursuit and an ultimate mission for generations of tool makers.␊
+      ␊
+      ␊
+      <!-- block_id=g9xKUjhJj1 flavour=affine:paragraph -->␊
+      ␊
+      ␊
+      ␊
+      <!-- block_id=wDTn4YJ4pm flavour=affine:paragraph -->␊
+      "We shape our tools and thereafter our tools shape us”. A lot of pioneers have inspired us a long the way, e.g.:␊
+      ␊
+      ␊
+      <!-- block_id=xFrrdiP3-V flavour=affine:list -->␊
+      * Quip & Notion with their great concept of "everything is a block"␊
+      ␊
+      ␊
+      <!-- block_id=Tp9xyN4Okl flavour=affine:list -->␊
+      * Trello with their Kanban␊
+      ␊
+      ␊
+      <!-- block_id=K_4hUzKZFQ flavour=affine:list -->␊
+      * Airtable & Miro with their no-code programable datasheets␊
+      ␊
+      ␊
+      <!-- block_id=QwMzON2s7x flavour=affine:list -->␊
+      * Miro & Whimiscal with their edgeless visual whiteboard␊
+      ␊
+      ␊
+      <!-- block_id=FFVmit6u1T flavour=affine:list -->␊
+      * Remnote & Capacities with their object-based tag system␊
+      ␊
+      ␊
+      <!-- block_id=YqnG5O6AE6 flavour=affine:paragraph -->␊
+      For more details, please refer to our [RoadMap](https://docs.affine.pro/docs/core-concepts/roadmap)␊
+      ␊
+      ␊
+      <!-- block_id=sbDTmZMZcq flavour=affine:paragraph -->␊
+      ## Self Host␊
+      ␊
+      ␊
+      <!-- block_id=QVvitesfbj flavour=affine:paragraph -->␊
+      Self host AFFiNE␊
+      ␊
+      ␊
+      <!-- block_id=U_GoHFD9At flavour=affine:database placeholder -->␊
+      ␊
+      <!-- block_id=NyHXrMX3R1 flavour=affine:paragraph -->␊
+      ## Affine Development␊
+      ␊
+      ␊
+      <!-- block_id=9-K49otbCv flavour=affine:paragraph -->␊
+      For developer or installation guides, please go to [AFFiNE Development](https://docs.affine.pro/docs/development/quick-start)␊
+      ␊
+      ␊
+      <!-- block_id=faFteK9eG- flavour=affine:paragraph -->␊
+      ␊
+      ␊
+      ␊
+      `,
+      title: 'Write, Draw, Plan all at Once.',
+    }

packages/backend/server/src/core/utils/__tests__/blocksute.spec.ts

@@ -99,3 +99,14 @@ test('can parse doc to markdown from doc snapshot', async t => {
 
   t.snapshot(result);
 });
+
+test('can parse doc to markdown from doc snapshot with ai editable', async t => {
+  const result = parseDocToMarkdownFromDocSnapshot(
+    workspace.id,
+    docSnapshot.id,
+    docSnapshot.blob,
+    true
+  );
+
+  t.snapshot(result);
+});

packages/backend/server/src/core/utils/blocksuite.ts

@@ -201,7 +201,8 @@ export async function readAllBlocksFromDocSnapshot(
 export function parseDocToMarkdownFromDocSnapshot(
   workspaceId: string,
   docId: string,
-  docSnapshot: Uint8Array
+  docSnapshot: Uint8Array,
+  aiEditable = false
 ) {
   const ydoc = new YDoc({
     guid: docId,
@@ -217,6 +218,7 @@ export function parseDocToMarkdownFromDocSnapshot(
     buildDocUrl: (docId: string) => {
       return `/workspace/${workspaceId}/${docId}`;
     },
+    aiEditable,
   });
 
   return {