Compare commits

..

21 Commits

Author SHA1 Message Date
Joooye_34
00c940f7df chore: bump affine version to 0.10.2 (#4959) 2023-11-16 15:48:37 +08:00
Flrande
931b459fbd chore: bump blocksuite (#4958) 2023-11-16 14:27:39 +08:00
LongYinan
51e71f4a0a ci: prevent error if rust build is cached by nx (#4951)
If Rust build was cached by nx, only the output file will be presented. The chmod command will be failed in this case like: https://github.com/toeverything/AFFiNE/actions/runs/6874496337/job/18697360212
2023-11-16 10:31:51 +08:00
Peng Xiao
9b631f2328 fix(infra): page id compat fix for page ids in workspace.meta (#4950)
since we strip `page:` in keys of workspacedoc.spaces, we should also strip the prefix in meta.pages as well.
2023-11-15 17:36:08 +08:00
LongYinan
01f481a9b6 ci: only disable postinstall on macOS in nightly desktop build (#4938) 2023-11-14 23:00:30 +08:00
Joooye_34
0177ab5c87 fix(infra): workspace migration without blockVersions (#4936) 2023-11-14 14:38:11 +01:00
Peng Xiao
4db35d341c perf(component): use png instead of svg for rendering noise svg (#4935) 2023-11-14 11:52:51 +00:00
DarkSky
3c4a803c97 fix: change password token check (#4934) (#4932) 2023-11-14 11:15:54 +00:00
LongYinan
05154dc7ca ci: disable postinstall in nightly desktop build (#4930)
Should be part of https://github.com/toeverything/AFFiNE/pull/4885
2023-11-14 14:13:55 +08:00
Peng Xiao
c90b477f60 fix(core): change server url of stable to insider (#4902) (#4926) 2023-11-14 12:05:52 +08:00
李华桥
6f18ddbe85 v0.10.1 2023-11-13 19:49:26 +08:00
LongYinan
dde779a71d test(e2e): add subdoc migration test (#4921)
test(e2e): add subdoc migration test

fix: remove .only
2023-11-13 18:00:40 +08:00
Peng Xiao
bd9f66fbc7 fix(infra): compatibility fix for space prefix (#4912)
It seems there are some cases that [this upstream PR](https://github.com/toeverything/blocksuite/pull/4747) will cause data loss.

Because of some historical reasons, the page id could be different with its doc id.
It might be caused by subdoc migration in the following (not 100% sure if all white screen issue is caused by it) 0714c12703/packages/common/infra/src/blocksuite/index.ts (L538-L540)

In version 0.10, page id in spaces no longer has prefix "space:"
The data flow for fetching a doc's updates is:
- page id in `meta.pages` -> find `${page-id}` in `doc.spaces` -> `doc` -> `doc.guid`
if `doc` is not found in `doc.spaces`, a new doc will be created and its `doc.guid` is the same with its pageId
- because of guid logic change, the doc that previously prefixed with `space:` will not be found in `doc.spaces`
- when fetching the rows of this doc using the doc id === page id,
  it will return EMPTY since there is no updates associated with the page id

The provided fix in the PR will patch the `spaces` field of the root doc so that after 0.10 the page doc can still be found in the `spaces` map. It shall apply to both of the idb & sqlite datasources.

Special thanks to @lawvs 's db file for investigation!
2023-11-13 17:57:56 +08:00
liuyi
92f1f40bfa fix(server): wrap updates applying in a transaction (#4922) 2023-11-13 08:49:30 +00:00
LongYinan
48dc1049b3 Merge pull request #4913 from toeverything/darksky/cleanup-depolyment
chore: cleanup deployment
2023-11-12 11:20:02 +08:00
DarkSky
9add530370 chore: cleanup deployment 2023-11-12 11:03:25 +08:00
LongYinan
b77460d871 Merge pull request #4908 from toeverything/61/hotfix-websocket-payload
fix(server): increase server acceptable websocket payload size
2023-11-10 22:01:48 +08:00
forehalo
42db41776b fix(server): increase server acceptable websocket payload size 2023-11-10 21:31:45 +08:00
李华桥
075439c74f fix(core): change server url of stable to insider 2023-11-10 18:32:53 +08:00
Yifeng Wang
fc6c553ece chore: bump theme (#4904)
Co-authored-by: 李华桥 <joooye1991@gmail.com>
2023-11-10 15:40:38 +08:00
Joooye_34
59cb3d5df1 fix(core): change server url of stable to insider (#4902) 2023-11-10 14:50:57 +08:00
7074 changed files with 128117 additions and 670810 deletions

View File

@@ -1,8 +1,2 @@
[target.x86_64-pc-windows-msvc]
rustflags = ["-C", "target-feature=+crt-static"]
[target.aarch64-pc-windows-msvc]
rustflags = ["-C", "target-feature=+crt-static"]
[target.'cfg(target_os = "linux")']
rustflags = ["-C", "link-args=-Wl,--warn-unresolved-symbols"]
[target.'cfg(target_os = "macos")']
rustflags = ["-C", "link-args=-all_load"]

33
.commitlintrc.json Normal file
View File

@@ -0,0 +1,33 @@
{
"extends": ["@commitlint/config-conventional"],
"rules": {
"scope-enum": [
2,
"always",
[
"electron",
"server",
"core",
"docs",
"storybook",
"component",
"workspace",
"env",
"graphql",
"cli",
"hooks",
"i18n",
"native",
"templates",
"y-indexeddb",
"y-provider",
"debug",
"storage",
"infra",
"plugin-cli",
"sdk",
"plugin"
]
]
}
}

View File

@@ -1,9 +0,0 @@
FROM mcr.microsoft.com/devcontainers/base:bookworm
# Install Homebrew For Linux
RUN /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" && \
eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)" && \
echo "eval \"\$($(brew --prefix)/bin/brew shellenv)\"" >> /home/vscode/.zshrc && \
echo "eval \"\$($(brew --prefix)/bin/brew shellenv)\"" >> /home/vscode/.bashrc && \
# Install Graphite
brew install withgraphite/tap/graphite && gt --version

View File

@@ -1,15 +0,0 @@
#!/bin/bash
# This is a script used by the devcontainer to build the project
#Enable yarn
corepack enable
corepack prepare yarn@stable --activate
# install dependencies
yarn install
# Build Server Dependencies
yarn affine @affine/server-native build
# Create database
yarn affine @affine/server prisma db push

View File

@@ -1,25 +0,0 @@
// For format details, see https://aka.ms/devcontainer.json.
{
"name": "Debian",
"dockerComposeFile": "docker-compose.yml",
"service": "app",
"workspaceFolder": "/workspaces/${localWorkspaceFolderBasename}",
"features": {
"ghcr.io/devcontainers/features/node:1": {
"version": "18"
},
"ghcr.io/devcontainers/features/rust:1": {}
},
// Configure tool-specific properties.
"customizations": {
"vscode": {
"extensions": [
"ms-playwright.playwright",
"esbenp.prettier-vscode",
"streetsidesoftware.code-spell-checker"
]
}
},
"updateContentCommand": "bash ./.devcontainer/build.sh",
"postCreateCommand": "bash ./.devcontainer/setup-user.sh"
}

View File

@@ -1,31 +0,0 @@
version: '3.8'
services:
app:
build:
context: .
dockerfile: Dockerfile
volumes:
- ../..:/workspaces:cached
command: sleep infinity
network_mode: service:db
environment:
DATABASE_URL: postgresql://affine:affine@db:5432/affine
REDIS_SERVER_HOST: redis
db:
image: postgres:latest
restart: unless-stopped
volumes:
- postgres-data:/var/lib/postgresql/data
environment:
POSTGRES_PASSWORD: affine
POSTGRES_USER: affine
POSTGRES_DB: affine
redis:
image: redis
ports:
- 6379:6379
volumes:
postgres-data:

View File

@@ -1,9 +0,0 @@
set -e
if [ -v GRAPHITE_TOKEN ];then
gt auth --token $GRAPHITE_TOKEN
fi
git fetch origin canary:canary --depth=1
git branch canary -t origin/canary
gt init --trunk canary

View File

@@ -1,4 +0,0 @@
DATABASE_LOCATION=./postgres
DB_PASSWORD=affine
DB_USERNAME=affine
DB_DATABASE_NAME=affine

View File

@@ -1,3 +0,0 @@
postgres
.env
compose.yml

View File

@@ -1,31 +0,0 @@
name: affine_dev_services
services:
postgres:
env_file:
- .env
image: postgres:16
ports:
- 5432:5432
environment:
POSTGRES_PASSWORD: ${DB_PASSWORD}
POSTGRES_USER: ${DB_USERNAME}
POSTGRES_DB: ${DB_DATABASE_NAME}
volumes:
- postgres_data:/var/lib/postgresql/data
redis:
image: redis:latest
ports:
- 6379:6379
mailhog:
image: mailhog/mailhog:latest
ports:
- 1025:1025
- 8025:8025
networks:
dev:
volumes:
postgres_data:

View File

@@ -1,23 +0,0 @@
# select a revision to deploy, available values: stable, beta, canary
AFFINE_REVISION=stable
# set the port for the server container it will expose the server on
PORT=3010
# set the host for the server for outgoing links
# AFFINE_SERVER_HTTPS=true
# AFFINE_SERVER_HOST=affine.yourdomain.com
# or
# AFFINE_SERVER_EXTERNAL_URL=https://affine.yourdomain.com
# position of the database data to persist
DB_DATA_LOCATION=~/.affine/self-host/postgres/pgdata
# position of the upload data(images, files, etc.) to persist
UPLOAD_LOCATION=~/.affine/self-host/storage
# position of the configuration files to persist
CONFIG_LOCATION=~/.affine/self-host/config
# database credentials
DB_USERNAME=affine
DB_PASSWORD=
DB_DATABASE=affine

View File

@@ -1,74 +0,0 @@
name: affine
services:
affine:
image: ghcr.io/toeverything/affine-graphql:${AFFINE_REVISION:-stable}
container_name: affine_server
ports:
- '${PORT:-3010}:3010'
depends_on:
redis:
condition: service_healthy
postgres:
condition: service_healthy
affine_migration:
condition: service_completed_successfully
volumes:
# custom configurations
- ${UPLOAD_LOCATION}:/root/.affine/storage
- ${CONFIG_LOCATION}:/root/.affine/config
env_file:
- .env
environment:
- REDIS_SERVER_HOST=redis
- DATABASE_URL=postgresql://${DB_USERNAME}:${DB_PASSWORD}@postgres:5432/${DB_DATABASE:-affine}
restart: unless-stopped
affine_migration:
image: ghcr.io/toeverything/affine-graphql:${AFFINE_REVISION:-stable}
container_name: affine_migration_job
volumes:
# custom configurations
- ${UPLOAD_LOCATION}:/root/.affine/storage
- ${CONFIG_LOCATION}:/root/.affine/config
command: ['sh', '-c', 'node ./scripts/self-host-predeploy.js']
env_file:
- .env
environment:
- REDIS_SERVER_HOST=redis
- DATABASE_URL=postgresql://${DB_USERNAME}:${DB_PASSWORD}@postgres:5432/${DB_DATABASE:-affine}
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
redis:
image: redis
container_name: redis
healthcheck:
test: ['CMD', 'redis-cli', '--raw', 'incr', 'ping']
interval: 10s
timeout: 5s
retries: 5
restart: unless-stopped
postgres:
image: postgres:16
container_name: postgres
volumes:
- ${DB_DATA_LOCATION}:/var/lib/postgresql/data
environment:
POSTGRES_USER: ${DB_USERNAME}
POSTGRES_PASSWORD: ${DB_PASSWORD}
POSTGRES_DB: ${DB_DATABASE:-affine}
POSTGRES_INITDB_ARGS: '--data-checksums'
# you better set a password for you database
# or you may add 'POSTGRES_HOST_AUTH_METHOD=trust' to ignore postgres security policy
POSTGRES_HOST_AUTH_METHOD: trust
healthcheck:
test:
['CMD', 'pg_isready', '-U', "${DB_USERNAME}", '-d', "${DB_DATABASE:-affine}"]
interval: 10s
timeout: 5s
retries: 5
restart: unless-stopped

14
.env.template Normal file
View File

@@ -0,0 +1,14 @@
ENABLE_PLUGIN=
ENABLE_TEST_PROPERTIES=
ENABLE_BC_PROVIDER=
CHANGELOG_URL=
ENABLE_PRELOADING=
ENABLE_NEW_SETTING_MODAL=
ENABLE_SQLITE_PROVIDER=
ENABLE_NEW_SETTING_UNSTABLE_API=
ENABLE_NOTIFICATION_CENTER=
ENABLE_CLOUD=
ENABLE_MOVE_DATABASE=
SHOULD_REPORT_TRACE=
TRACE_REPORT_ENDPOINT=
CAPTCHA_SITE_KEY=

16
.eslintignore Normal file
View File

@@ -0,0 +1,16 @@
node_modules
dist
.next
out
storybook-static
affine-out
_next
lib
.eslintrc.js
e2e-dist-*
static
web-static
public
packages/common/sdk/src/*.d.ts
packages/common/sdk/src/*.js
packages/frontend/i18n/src/i18n-generated.ts

290
.eslintrc.js Normal file
View File

@@ -0,0 +1,290 @@
const { resolve } = require('node:path');
const createPattern = packageName => [
{
group: ['**/dist', '**/dist/**'],
message: 'Do not import from dist',
allowTypeImports: false,
},
{
group: ['**/src', '**/src/**'],
message: 'Do not import from src',
allowTypeImports: false,
},
{
group: [`@affine/${packageName}`],
message: 'Do not import package itself',
allowTypeImports: false,
},
{
group: [`@toeverything/${packageName}`],
message: 'Do not import package itself',
allowTypeImports: false,
},
{
group: ['@blocksuite/store'],
message: "Import from '@blocksuite/global/utils'",
importNames: ['assertExists', 'assertEquals'],
},
{
group: ['react-router-dom'],
message: 'Use `useNavigateHelper` instead',
importNames: ['useNavigate'],
},
{
group: ['next-auth/react'],
message: "Import hooks from 'use-current-user.tsx'",
// useSession is type unsafe
importNames: ['useSession'],
},
{
group: ['next-auth/react'],
message: "Import hooks from 'cloud-utils.ts'",
importNames: ['signIn', 'signOut'],
},
{
group: ['yjs'],
message: 'Do not use this API because it has a bug',
importNames: ['mergeUpdates'],
},
{
group: ['@affine/env/constant'],
message:
'Do not import from @affine/env/constant. Use `environment.isDesktop` instead',
importNames: ['isDesktop'],
},
];
const allPackages = [
'packages/backend/server',
'packages/frontend/component',
'packages/frontend/web',
'packages/frontend/electron',
'packages/frontend/graphql',
'packages/frontend/hooks',
'packages/frontend/i18n',
'packages/frontend/native',
'packages/frontend/templates',
'packages/frontend/workspace',
'packages/common/debug',
'packages/common/env',
'packages/common/infra',
'packages/common/sdk',
'packages/common/theme',
'packages/common/y-indexeddb',
'packages/plugins/copilot',
'tools/cli',
'tests/storybook',
];
/**
* @type {import('eslint').Linter.Config}
*/
const config = {
root: true,
settings: {
react: {
version: 'detect',
},
next: {
rootDir: 'packages/frontend/core',
},
},
extends: [
'eslint:recommended',
'plugin:react-hooks/recommended',
'plugin:react/recommended',
'plugin:react/jsx-runtime',
'plugin:@typescript-eslint/recommended',
'prettier',
],
parser: '@typescript-eslint/parser',
parserOptions: {
ecmaFeatures: {
globalReturn: false,
impliedStrict: true,
jsx: true,
},
ecmaVersion: 'latest',
sourceType: 'module',
project: resolve(__dirname, './tsconfig.eslint.json'),
},
plugins: [
'react',
'@typescript-eslint',
'simple-import-sort',
'sonarjs',
'i',
'unused-imports',
'unicorn',
],
rules: {
'array-callback-return': 'error',
'no-undef': 'off',
'no-empty': 'off',
'no-func-assign': 'off',
'no-cond-assign': 'off',
'no-constant-binary-expression': 'error',
'no-constructor-return': 'error',
'react/prop-types': 'off',
'react/jsx-no-useless-fragment': 'error',
'@typescript-eslint/consistent-type-imports': 'error',
'@typescript-eslint/no-non-null-assertion': 'error',
'@typescript-eslint/no-explicit-any': 'off',
'@typescript-eslint/no-empty-function': 'off',
'@typescript-eslint/await-thenable': 'error',
'@typescript-eslint/no-unused-vars': [
'error',
{
varsIgnorePattern: '^_',
argsIgnorePattern: '^_',
caughtErrorsIgnorePattern: '^_',
},
],
'unused-imports/no-unused-imports': 'error',
'simple-import-sort/imports': 'error',
'simple-import-sort/exports': 'error',
'@typescript-eslint/ban-ts-comment': [
'error',
{
'ts-expect-error': 'allow-with-description',
'ts-ignore': true,
'ts-nocheck': true,
'ts-check': false,
},
],
'@typescript-eslint/no-restricted-imports': [
'error',
{
patterns: [
{
group: ['**/dist'],
message: "Don't import from dist",
allowTypeImports: false,
},
{
group: ['**/src'],
message: "Don't import from src",
allowTypeImports: false,
},
{
group: ['@blocksuite/store'],
message: "Import from '@blocksuite/global/utils'",
importNames: ['assertExists', 'assertEquals'],
},
{
group: ['react-router-dom'],
message: 'Use `useNavigateHelper` instead',
importNames: ['useNavigate'],
},
{
group: ['next-auth/react'],
message: "Import hooks from 'use-current-user.tsx'",
// useSession is type unsafe
importNames: ['useSession'],
},
{
group: ['next-auth/react'],
message: "Import hooks from 'cloud-utils.ts'",
importNames: ['signIn', 'signOut'],
},
{
group: ['yjs'],
message: 'Do not use this API because it has a bug',
importNames: ['mergeUpdates'],
},
],
},
],
'unicorn/filename-case': [
'error',
{
case: 'kebabCase',
ignore: ['^\\[[a-zA-Z0-9-_]+\\]\\.tsx$'],
},
],
'unicorn/no-unnecessary-await': 'error',
'sonarjs/no-all-duplicated-branches': 'error',
'sonarjs/no-element-overwrite': 'error',
'sonarjs/no-empty-collection': 'error',
'sonarjs/no-extra-arguments': 'error',
'sonarjs/no-identical-conditions': 'error',
'sonarjs/no-identical-expressions': 'error',
'sonarjs/no-ignored-return': 'error',
'sonarjs/no-one-iteration-loop': 'error',
'sonarjs/no-use-of-empty-return-value': 'error',
'sonarjs/non-existent-operator': 'error',
'sonarjs/no-collapsible-if': 'error',
'sonarjs/no-same-line-conditional': 'error',
'sonarjs/no-duplicated-branches': 'error',
'sonarjs/no-collection-size-mischeck': 'error',
'sonarjs/no-useless-catch': 'error',
'sonarjs/no-identical-functions': 'error',
},
overrides: [
{
files: 'packages/backend/server/**/*.ts',
rules: {
'@typescript-eslint/consistent-type-imports': 0,
},
},
{
files: '*.cjs',
rules: {
'@typescript-eslint/no-var-requires': 0,
},
},
...allPackages.map(pkg => ({
files: [`${pkg}/src/**/*.ts`, `${pkg}/src/**/*.tsx`],
parserOptions: {
project: resolve(__dirname, './tsconfig.eslint.json'),
},
rules: {
'@typescript-eslint/no-restricted-imports': [
'error',
{
patterns: createPattern(pkg),
},
],
'@typescript-eslint/no-floating-promises': [
'error',
{
ignoreVoid: false,
ignoreIIFE: false,
},
],
'@typescript-eslint/no-misused-promises': ['error'],
'i/no-extraneous-dependencies': ['error'],
},
})),
{
files: [
'**/__tests__/**/*',
'**/*.stories.tsx',
'**/*.spec.ts',
'**/tests/**/*',
'scripts/**/*',
'**/benchmark/**/*',
'**/__debug__/**/*',
'**/e2e/**/*',
],
rules: {
'@typescript-eslint/no-non-null-assertion': 0,
'@typescript-eslint/ban-ts-comment': [
'error',
{
'ts-expect-error': false,
'ts-ignore': true,
'ts-nocheck': true,
'ts-check': false,
},
],
'@typescript-eslint/no-floating-promises': 0,
'@typescript-eslint/no-misused-promises': 0,
'@typescript-eslint/no-restricted-imports': 0,
},
},
],
};
module.exports = config;

30
.github/CLA.md vendored
View File

@@ -33,6 +33,32 @@ You accept and agree to the following terms and conditions for your past, presen
9. This Agreement will be governed by the laws of Republic of Singapore without reference to conflict of laws principles.
## How To Sign
## List of Contributors
Visit https://cla-assistant.io/toeverything/AFFiNE and sign it.
The below-signed are contributors to a code repository that is part of the project named "AFFiNE". Each below-signed contributor has read, understand and agrees to the terms above in the section within this document entitled "AFFiNE Contributor License Agreement" as of the date beside their real name (or entity name) and GitHub account name.
---
<!--
Example:
- Dark Sky, @darkskygit, 2022/07/22
-->
- Dark Sky, @darkskygit, 2022/07/22
- Lin Onetwo, @linonetwo, 2022/02/14
- zqran, @zqran, 2023/02/17
- Alessio Gravili, @AlessioGr, 2023/03/04
- Victor Nanka, @victornanka, 2023/03/09
- Aditya Sharma, @adityash1, 2023/03/21
- Fangdun Tsai, @fundon, 2023/03/21
- Zhilin Liu, @lzlme, 2023/04/09
- Skye Sun, @skyesun, 2023/04/14
- Jordy Delgado, @Jdelgad8, 2023/04/17
- Howard Do, @howarddo2208, 2023/04/20
- 三咲智子 Kevin Deng, @sxzz, 2023/04/21
- Moeyua, @moeyua, 2023/04/22
- Shishu, @shishudesu, 2023/05/19
- Kushagra Singh, @kush002, 2023/06/28
- Sarvesh Kumar, @sarvesh521 2023/08/25
- 微扰理论 Qinghao Huang, @wfnuser 2023/09/29

1
.github/CODEOWNERS vendored
View File

@@ -1 +0,0 @@
/blocksuite/ @toeverything/blocksuite-core

View File

@@ -7,8 +7,6 @@ body:
attributes:
value: |
Thanks for taking the time to fill out this bug report!
Check out this [link](https://github.com/toeverything/AFFiNE/blob/canary/docs/issue-triaging.md)
to learn how we manage issues and when your issue will be processed.
- type: textarea
id: what-happened
attributes:
@@ -43,14 +41,6 @@ body:
- Firefox
- Safari
- Other
- type: checkboxes
id: selfhost
attributes:
label: Are you self-hosting?
description: >
If you are self-hosting, please check the box and provide information about your setup.
options:
- label: 'Yes'
- type: textarea
id: logs
attributes:
@@ -63,3 +53,11 @@ body:
description: |
Links? References? Anything that will give us more context about the issue you are encountering!
Tip: You can attach images here
- type: checkboxes
attributes:
label: Are you willing to submit a PR?
description: >
(Optional) We encourage you to submit a [Pull Request](https://github.com/toeverything/affine/pulls) (PR) to help improve AFFiNE for everyone, especially if you have a good understanding of how to implement a fix or feature.
See the AFFiNE [Contributing Guide](https://github.com/toeverything/affine/blob/master/CONTRIBUTING.md) to get started.
options:
- label: Yes I'd like to help by submitting a PR!

View File

@@ -31,6 +31,6 @@ body:
label: Are you willing to submit a PR?
description: >
(Optional) We encourage you to submit a [Pull Request](https://github.com/toeverything/affine/pulls) (PR) to help improve AFFiNE for everyone, especially if you have a good understanding of how to implement a fix or feature.
See the AFFiNE [Contributing Guide](https://github.com/toeverything/affine/blob/canary/CONTRIBUTING.md) to get started.
See the AFFiNE [Contributing Guide](https://github.com/toeverything/affine/blob/master/CONTRIBUTING.md) to get started.
options:
- label: Yes I'd like to help by submitting a PR!

View File

@@ -7,83 +7,69 @@ inputs:
package:
description: 'Package to build'
required: true
no-build:
description: 'Whether to skip building'
nx_token:
description: 'Nx Cloud access token'
required: false
default: 'false'
runs:
using: 'composite'
steps:
- name: Print rustup toolchain version
shell: bash
id: rustup-version
working-directory: ${{ env.DEV_DRIVE_WORKSPACE || github.workspace }}
run: |
export RUST_TOOLCHAIN_VERSION="$(grep 'channel' rust-toolchain.toml | head -1 | awk -F '"' '{print $2}')"
echo "Rust toolchain version: $RUST_TOOLCHAIN_VERSION"
echo "RUST_TOOLCHAIN_VERSION=$RUST_TOOLCHAIN_VERSION" >> "$GITHUB_OUTPUT"
- name: Setup Rust
uses: dtolnay/rust-toolchain@stable
if: ${{ runner.os != 'Windows' }}
with:
toolchain: '${{ steps.rustup-version.outputs.RUST_TOOLCHAIN_VERSION }}'
toolchain: stable
targets: ${{ inputs.target }}
env:
CARGO_INCREMENTAL: '1'
- name: Setup Rust
uses: dtolnay/rust-toolchain@stable
if: ${{ runner.os == 'Windows' }}
- name: Cache cargo
uses: actions/cache@v3
with:
toolchain: '${{ steps.rustup-version.outputs.RUST_TOOLCHAIN_VERSION }}'
targets: ${{ inputs.target }}
env:
CARGO_INCREMENTAL: '1'
CARGO_HOME: ${{ env.DEV_DRIVE }}/.cargo
RUSTUP_HOME: ${{ env.DEV_DRIVE }}/.rustup
- name: Set CC
if: ${{ contains(inputs.target, 'linux') && inputs.package != '@affine/native' && inputs.no-build != 'true' }}
working-directory: ${{ env.DEV_DRIVE_WORKSPACE || github.workspace }}
path: |
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
.cargo-cache
target/${{ inputs.target }}
key: stable-${{ inputs.target }}-cargo-cache
- name: Build
if: ${{ inputs.target != 'x86_64-unknown-linux-gnu' && inputs.target != 'aarch64-unknown-linux-gnu' }}
shell: bash
run: |
echo "CC=clang" >> "$GITHUB_ENV"
echo "TARGET_CC=clang" >> "$GITHUB_ENV"
- name: Cache cargo
uses: Swatinem/rust-cache@v2
if: ${{ runner.os == 'Windows' }}
with:
workspaces: ${{ env.DEV_DRIVE_WORKSPACE }}
save-if: ${{ github.ref_name == 'canary' }}
shared-key: ${{ inputs.target }}-${{ inputs.package }}
yarn workspace ${{ inputs.package }} nx build ${{ inputs.package }} --target ${{ inputs.target }}
env:
CARGO_HOME: ${{ env.DEV_DRIVE }}/.cargo
RUSTUP_HOME: ${{ env.DEV_DRIVE }}/.rustup
- name: Cache cargo
uses: Swatinem/rust-cache@v2
if: ${{ runner.os != 'Windows' }}
with:
save-if: ${{ github.ref_name == 'canary' }}
shared-key: ${{ inputs.target }}-${{ inputs.package }}
NX_CLOUD_ACCESS_TOKEN: ${{ inputs.nx_token }}
- name: Build
shell: bash
if: ${{ runner.os != 'Windows' && inputs.no-build != 'true' }}
run: |
yarn workspace ${{ inputs.package }} build --target ${{ inputs.target }} --use-napi-cross
env:
DEBUG: 'napi:*'
if: ${{ inputs.target == 'x86_64-unknown-linux-gnu' }}
uses: addnab/docker-run-action@v3
with:
image: ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-debian
options: --user 0:0 -v ${{ github.workspace }}/.cargo-cache/git/db:/usr/local/cargo/git/db -v ${{ github.workspace }}/.cargo/registry/cache:/usr/local/cargo/registry/cache -v ${{ github.workspace }}/.cargo/registry/index:/usr/local/cargo/registry/index -v ${{ github.workspace }}:/build -w /build -e NX_CLOUD_ACCESS_TOKEN=${{ inputs.nx_token }}
run: |
export CC=x86_64-unknown-linux-gnu-gcc
export CC_x86_64_unknown_linux_gnu=x86_64-unknown-linux-gnu-gcc
export RUSTFLAGS="-C debuginfo=1"
yarn workspace ${{ inputs.package }} nx build ${{ inputs.package }} --target ${{ inputs.target }}
if [ -d "node_modules/.cache" ]; then
chmod -R 777 node_modules/.cache
fi
if [ -d "target" ]; then
chmod -R 777 target;
fi
- name: Build
working-directory: ${{ env.DEV_DRIVE_WORKSPACE || github.workspace }}
shell: bash
if: ${{ runner.os == 'Windows' && inputs.no-build != 'true' }}
run: |
yarn workspace ${{ inputs.package }} build --target ${{ inputs.target }} --use-napi-cross
env:
DEBUG: 'napi:*'
CARGO_HOME: ${{ env.DEV_DRIVE }}/.cargo
RUSTUP_HOME: ${{ env.DEV_DRIVE }}/.rustup
if: ${{ inputs.target == 'aarch64-unknown-linux-gnu' }}
uses: addnab/docker-run-action@v3
with:
image: ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-debian-aarch64
options: --user 0:0 -v ${{ github.workspace }}/.cargo-cache/git/db:/usr/local/cargo/git/db -v ${{ github.workspace }}/.cargo/registry/cache:/usr/local/cargo/registry/cache -v ${{ github.workspace }}/.cargo/registry/index:/usr/local/cargo/registry/index -v ${{ github.workspace }}:/build -w /build -e NX_CLOUD_ACCESS_TOKEN=${{ inputs.nx_token }}
run: |
export RUSTFLAGS="-C debuginfo=1"
yarn workspace ${{ inputs.package }} nx build ${{ inputs.package }} --target ${{ inputs.target }}
if [ -d "node_modules/.cache" ]; then
chmod -R 777 node_modules/.cache
fi
if [ -d "target" ]; then
chmod -R 777 target;
fi

View File

@@ -1,36 +0,0 @@
name: 'Auth to Cluster'
description: 'Auth to the GCP Cluster'
inputs:
gcp-project-number:
description: 'GCP project number'
required: true
gcp-project-id:
description: 'GCP project id'
required: true
service-account:
description: 'Service account'
cluster-name:
description: 'Cluster name'
cluster-location:
description: 'Cluster location'
runs:
using: 'composite'
steps:
- id: auth
uses: google-github-actions/auth@v2
with:
workload_identity_provider: 'projects/${{ inputs.gcp-project-number }}/locations/global/workloadIdentityPools/github-actions/providers/github-actions-helm-deploy'
service_account: '${{ inputs.service-account }}'
token_format: 'access_token'
project_id: '${{ inputs.gcp-project-id }}'
- name: 'Setup gcloud cli'
uses: 'google-github-actions/setup-gcloud@v2'
with:
install_components: 'gke-gcloud-auth-plugin'
- id: get-gke-credentials
shell: bash
run: |
gcloud container clusters get-credentials ${{ inputs.cluster-name }} --region ${{ inputs.cluster-location }} --project ${{ inputs.gcp-project-id }}

View File

@@ -1,36 +0,0 @@
name: 'Run Copilot E2E Test'
description: 'Run Copilot E2E Test'
inputs:
script:
description: 'Script to run'
default: 'yarn affine @affine-test/affine-cloud-copilot e2e --forbid-only'
required: false
openai-key:
description: 'OpenAI secret key'
required: true
fal-key:
description: 'Fal secret key'
required: true
runs:
using: 'composite'
steps:
- name: Prepare Server Test Environment
uses: ./.github/actions/server-test-env
- name: Server Copilot E2E Test
shell: bash
run: ${{ inputs.script }}
env:
COPILOT: true
DEV_SERVER_URL: http://localhost:8080
COPILOT_OPENAI_API_KEY: ${{ inputs.openai-key }}
COPILOT_FAL_API_KEY: ${{ inputs.fal-key }}
- name: Upload test results
if: ${{ failure() }}
uses: actions/upload-artifact@v4
with:
name: test-results-e2e-server-copilot
path: ./test-results
if-no-files-found: ignore

View File

@@ -24,14 +24,24 @@ runs:
shell: bash
run: |
echo "GIT_SHORT_HASH=$(git rev-parse --short HEAD)" >> "$GITHUB_ENV"
- name: 'Auth to cluster'
uses: './.github/actions/cluster-auth'
- uses: azure/setup-helm@v3
- id: auth
uses: google-github-actions/auth@v1
with:
gcp-project-number: '${{ inputs.gcp-project-number }}'
gcp-project-id: '${{ inputs.gcp-project-id }}'
service-account: '${{ inputs.service-account }}'
cluster-name: '${{ inputs.cluster-name }}'
cluster-location: '${{ inputs.cluster-location }}'
workload_identity_provider: 'projects/${{ inputs.gcp-project-number }}/locations/global/workloadIdentityPools/github-actions/providers/github-actions-helm-deploy'
service_account: '${{ inputs.service-account }}'
token_format: 'access_token'
project_id: '${{ inputs.gcp-project-id }}'
- name: 'Setup gcloud cli'
uses: 'google-github-actions/setup-gcloud@v1'
with:
install_components: 'gke-gcloud-auth-plugin'
- id: get-gke-credentials
shell: bash
run: |
gcloud container clusters get-credentials ${{ inputs.cluster-name }} --region ${{ inputs.cluster-location }} --project ${{ inputs.gcp-project-id }}
- name: Deploy
shell: bash

View File

@@ -1,7 +1,6 @@
import { execSync } from 'node:child_process';
const {
APP_VERSION,
BUILD_TYPE,
DEPLOY_HOST,
CANARY_DEPLOY_HOST,
@@ -13,14 +12,12 @@ const {
R2_ACCOUNT_ID,
R2_ACCESS_KEY_ID,
R2_SECRET_ACCESS_KEY,
R2_BUCKET,
ENABLE_CAPTCHA,
CAPTCHA_TURNSTILE_SECRET,
METRICS_CUSTOMER_IO_TOKEN,
COPILOT_OPENAI_API_KEY,
COPILOT_FAL_API_KEY,
COPILOT_UNSPLASH_API_KEY,
MAILER_SENDER,
MAILER_USER,
MAILER_PASSWORD,
OAUTH_EMAIL_SENDER,
OAUTH_EMAIL_LOGIN,
OAUTH_EMAIL_PASSWORD,
AFFINE_GOOGLE_CLIENT_ID,
AFFINE_GOOGLE_CLIENT_SECRET,
CLOUD_SQL_IAM_ACCOUNT,
@@ -30,7 +27,6 @@ const {
REDIS_PASSWORD,
STRIPE_API_KEY,
STRIPE_WEBHOOK_KEY,
STATIC_IP_NAME,
} = process.env;
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion
@@ -38,49 +34,17 @@ const buildType = BUILD_TYPE || 'canary';
const isProduction = buildType === 'stable';
const isBeta = buildType === 'beta';
const isInternal = buildType === 'internal';
const replicaConfig = {
stable: {
web: 3,
graphql: Number(process.env.PRODUCTION_GRAPHQL_REPLICA) || 3,
sync: Number(process.env.PRODUCTION_SYNC_REPLICA) || 3,
renderer: Number(process.env.PRODUCTION_RENDERER_REPLICA) || 3,
},
beta: {
web: 2,
graphql: Number(process.env.BETA_GRAPHQL_REPLICA) || 2,
sync: Number(process.env.BETA_SYNC_REPLICA) || 2,
renderer: Number(process.env.BETA_RENDERER_REPLICA) || 2,
},
canary: {
web: 2,
graphql: 2,
sync: 2,
renderer: 2,
},
};
const cpuConfig = {
beta: {
web: '300m',
graphql: '1',
sync: '1',
renderer: '300m',
},
canary: {
web: '300m',
graphql: '1',
sync: '1',
renderer: '300m',
},
};
const createHelmCommand = ({ isDryRun }) => {
const flag = isDryRun ? '--dry-run' : '--atomic';
const imageTag = `${buildType}-${GIT_SHORT_HASH}`;
const staticIpName = isProduction
? 'affine-cluster-production'
: isBeta
? 'affine-cluster-beta'
: 'affine-cluster-dev';
const redisAndPostgres =
isProduction || isBeta || isInternal
isProduction || isBeta
? [
`--set-string global.database.url=${DATABASE_URL}`,
`--set-string global.database.user=${DATABASE_USERNAME}`,
@@ -94,81 +58,54 @@ const createHelmCommand = ({ isDryRun }) => {
]
: [];
const serviceAnnotations =
isProduction || isBeta || isInternal
isProduction || isBeta
? [
`--set-json web.service.annotations=\"{ \\"cloud.google.com/neg\\": \\"{\\\\\\"ingress\\\\\\": true}\\" }\"`,
`--set-json graphql.serviceAccount.annotations=\"{ \\"iam.gke.io/gcp-service-account\\": \\"${CLOUD_SQL_IAM_ACCOUNT}\\" }\"`,
`--set-json graphql.service.annotations=\"{ \\"cloud.google.com/neg\\": \\"{\\\\\\"ingress\\\\\\": true}\\" }\"`,
`--set-json sync.serviceAccount.annotations=\"{ \\"iam.gke.io/gcp-service-account\\": \\"${CLOUD_SQL_IAM_ACCOUNT}\\" }\"`,
`--set-json sync.service.annotations=\"{ \\"cloud.google.com/neg\\": \\"{\\\\\\"ingress\\\\\\": true}\\" }\"`,
`--set-json cloud-sql-proxy.serviceAccount.annotations=\"{ \\"iam.gke.io/gcp-service-account\\": \\"${CLOUD_SQL_IAM_ACCOUNT}\\" }\"`,
`--set-json cloud-sql-proxy.nodeSelector=\"{ \\"iam.gke.io/gke-metadata-server-enabled\\": \\"true\\" }\"`,
]
: [];
const cpu = cpuConfig[buildType];
const resources = cpu
? [
`--set web.resources.requests.cpu="${cpu.web}"`,
`--set graphql.resources.requests.cpu="${cpu.graphql}"`,
`--set sync.resources.requests.cpu="${cpu.sync}"`,
]
: [];
const replica = replicaConfig[buildType] || replicaConfig.canary;
const namespace = isProduction
? 'production'
: isBeta
? 'beta'
: isInternal
? 'internal'
: 'dev';
const webReplicaCount = isProduction ? 3 : isBeta ? 2 : 2;
const graphqlReplicaCount = isProduction ? 10 : isBeta ? 10 : 2;
const syncReplicaCount = isProduction ? 10 : isBeta ? 10 : 2;
const namespace = isProduction ? 'production' : isBeta ? 'beta' : 'dev';
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion
const host = DEPLOY_HOST || CANARY_DEPLOY_HOST;
const deployCommand = [
`helm upgrade --install affine .github/helm/affine`,
`--namespace ${namespace}`,
`--set-string global.app.buildType="${buildType}"`,
`--set global.ingress.enabled=true`,
`--set-json global.ingress.annotations=\"{ \\"kubernetes.io/ingress.class\\": \\"gce\\", \\"kubernetes.io/ingress.allow-http\\": \\"true\\", \\"kubernetes.io/ingress.global-static-ip-name\\": \\"${STATIC_IP_NAME}\\" }\"`,
`--set-json global.ingress.annotations=\"{ \\"kubernetes.io/ingress.class\\": \\"gce\\", \\"kubernetes.io/ingress.allow-http\\": \\"true\\", \\"kubernetes.io/ingress.global-static-ip-name\\": \\"${staticIpName}\\" }\"`,
`--set-string global.ingress.host="${host}"`,
`--set global.objectStorage.r2.enabled=true`,
`--set-string global.objectStorage.r2.accountId="${R2_ACCOUNT_ID}"`,
`--set-string global.objectStorage.r2.accessKeyId="${R2_ACCESS_KEY_ID}"`,
`--set-string global.objectStorage.r2.secretAccessKey="${R2_SECRET_ACCESS_KEY}"`,
`--set-string global.version="${APP_VERSION}"`,
...redisAndPostgres,
`--set web.replicaCount=${replica.web}`,
`--set web.replicaCount=${webReplicaCount}`,
`--set-string web.image.tag="${imageTag}"`,
`--set graphql.replicaCount=${replica.graphql}`,
`--set graphql.replicaCount=${graphqlReplicaCount}`,
`--set-string graphql.image.tag="${imageTag}"`,
`--set graphql.app.host=${host}`,
`--set graphql.app.captcha.enabled=true`,
`--set graphql.app.captcha.enabled=${ENABLE_CAPTCHA}`,
`--set-string graphql.app.captcha.turnstile.secret="${CAPTCHA_TURNSTILE_SECRET}"`,
`--set graphql.app.copilot.enabled=true`,
`--set-string graphql.app.copilot.openai.key="${COPILOT_OPENAI_API_KEY}"`,
`--set-string graphql.app.copilot.fal.key="${COPILOT_FAL_API_KEY}"`,
`--set-string graphql.app.copilot.unsplash.key="${COPILOT_UNSPLASH_API_KEY}"`,
`--set-string graphql.app.mailer.sender="${MAILER_SENDER}"`,
`--set-string graphql.app.mailer.user="${MAILER_USER}"`,
`--set-string graphql.app.mailer.password="${MAILER_PASSWORD}"`,
`--set graphql.app.objectStorage.r2.enabled=true`,
`--set-string graphql.app.objectStorage.r2.accountId="${R2_ACCOUNT_ID}"`,
`--set-string graphql.app.objectStorage.r2.accessKeyId="${R2_ACCESS_KEY_ID}"`,
`--set-string graphql.app.objectStorage.r2.secretAccessKey="${R2_SECRET_ACCESS_KEY}"`,
`--set-string graphql.app.objectStorage.r2.bucket="${R2_BUCKET}"`,
`--set-string graphql.app.oauth.email.sender="${OAUTH_EMAIL_SENDER}"`,
`--set-string graphql.app.oauth.email.login="${OAUTH_EMAIL_LOGIN}"`,
`--set-string graphql.app.oauth.email.password="${OAUTH_EMAIL_PASSWORD}"`,
`--set-string graphql.app.oauth.google.enabled=true`,
`--set-string graphql.app.oauth.google.clientId="${AFFINE_GOOGLE_CLIENT_ID}"`,
`--set-string graphql.app.oauth.google.clientSecret="${AFFINE_GOOGLE_CLIENT_SECRET}"`,
`--set-string graphql.app.payment.stripe.apiKey="${STRIPE_API_KEY}"`,
`--set-string graphql.app.payment.stripe.webhookKey="${STRIPE_WEBHOOK_KEY}"`,
`--set graphql.app.metrics.enabled=true`,
`--set-string graphql.app.metrics.customerIo.token="${METRICS_CUSTOMER_IO_TOKEN}"`,
`--set graphql.app.experimental.enableJwstCodec=${namespace === 'dev'}`,
`--set graphql.app.experimental.enableJwstCodec=true`,
`--set graphql.app.features.earlyAccessPreview=false`,
`--set graphql.app.features.syncClientVersionCheck=true`,
`--set sync.replicaCount=${replica.sync}`,
`--set sync.replicaCount=${syncReplicaCount}`,
`--set-string sync.image.tag="${imageTag}"`,
`--set-string renderer.image.tag="${imageTag}"`,
`--set renderer.app.host=${host}`,
`--set renderer.replicaCount=${replica.renderer}`,
...serviceAnnotations,
...resources,
`--timeout 10m`,
`--version "0.0.0-${buildType}.${GIT_SHORT_HASH}" --timeout 10m`,
flag,
].join(' ');
return deployCommand;

View File

@@ -1,22 +0,0 @@
name: 'Download core artifacts'
description: 'Download core artifacts and extract to dist'
inputs:
path:
description: 'Path to extract'
required: true
runs:
using: 'composite'
steps:
- name: Download tar.gz
uses: actions/download-artifact@v4
with:
name: web
path: .
- name: Extract core artifacts
shell: bash
run: |
mkdir -p ${{ inputs.path }}
tar -xvf dist.tar.gz --directory ${{ inputs.path }}
rm dist.tar.gz

View File

@@ -1,23 +0,0 @@
name: 'Prepare Server Test Environment'
description: 'Prepare Server Test Environment'
runs:
using: 'composite'
steps:
- name: Initialize database
shell: bash
run: |
psql -h localhost -U postgres -c "CREATE DATABASE affine;"
psql -h localhost -U postgres -c "CREATE USER affine WITH PASSWORD 'affine';"
psql -h localhost -U postgres -c "ALTER USER affine WITH SUPERUSER;"
env:
PGPASSWORD: affine
- name: Run init-db script
shell: bash
env:
NODE_ENV: test
run: |
yarn affine @affine/server prisma generate
yarn affine @affine/server prisma db push
yarn affine @affine/server data-migration run

View File

@@ -17,14 +17,18 @@ inputs:
description: 'Download the Electron binary'
required: false
default: 'true'
corepack-install:
description: 'Install CorePack'
required: false
default: 'false'
hard-link-nm:
description: 'set nmMode to hardlinks-local in .yarnrc.yml'
required: false
default: 'true'
build-infra:
description: 'Build infra'
required: false
default: 'true'
build-plugins:
description: 'Build plugins'
required: false
default: 'true'
nmHoistingLimits:
description: 'Set nmHoistingLimits in .yarnrc.yml'
required: false
@@ -32,104 +36,48 @@ inputs:
description: 'Set enableScripts in .yarnrc.yml'
required: false
default: 'true'
full-cache:
description: 'Full installation cache'
required: false
runs:
using: 'composite'
steps:
- name: Output workspace path
id: workspace-path
shell: bash
run: |
if [ -n "${{ env.DEV_DRIVE_WORKSPACE }}" ]; then
echo "workspace_path=${{ env.DEV_DRIVE_WORKSPACE }}" >> $GITHUB_OUTPUT
else
echo "workspace_path=${{ github.workspace }}" >> $GITHUB_OUTPUT
fi
- name: Setup Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v3
with:
node-version-file: '.nvmrc'
registry-url: https://npm.pkg.github.com
scope: '@toeverything'
- uses: kenchan0130/actions-system-info@master
id: system-info
- name: Init CorePack
if: ${{ inputs.corepack-install == 'true' }}
shell: bash
working-directory: ${{ steps.workspace-path.outputs.workspace_path }}
run: corepack enable
cache: 'yarn'
- name: Set nmMode
if: ${{ inputs.hard-link-nm == 'false' }}
if: ${{ inputs.hard-link-nm == 'true' }}
shell: bash
working-directory: ${{ steps.workspace-path.outputs.workspace_path }}
run: yarn config set nmMode classic
run: yarn config set nmMode hardlinks-local
- name: Set nmHoistingLimits
if: ${{ inputs.nmHoistingLimits }}
shell: bash
working-directory: ${{ steps.workspace-path.outputs.workspace_path }}
run: yarn config set nmHoistingLimits ${{ inputs.nmHoistingLimits }}
- name: Set enableScripts
if: ${{ inputs.enableScripts == 'false' }}
shell: bash
working-directory: ${{ steps.workspace-path.outputs.workspace_path }}
run: yarn config set enableScripts false
- name: Set yarn global cache path
shell: bash
id: yarn-cache
working-directory: ${{ steps.workspace-path.outputs.workspace_path }}
run: node -e "const p = $(yarn config cacheFolder --json).effective; console.log('yarn_global_cache=' + p)" >> $GITHUB_OUTPUT
- name: Cache non-full yarn cache on Linux
uses: actions/cache@v4
if: ${{ inputs.full-cache != 'true' && runner.os == 'Linux' }}
with:
path: |
${{ steps.workspace-path.outputs.workspace_path }}/node_modules
${{ steps.yarn-cache.outputs.yarn_global_cache }}
key: node_modules-cache-${{ github.job }}-${{ runner.os }}-${{ runner.arch }}-${{ steps.system-info.outputs.name }}-${{ steps.system-info.outputs.release }}-${{ steps.system-info.outputs.version }}
# The network performance on macOS is very poor
# and the decompression performance on Windows is very terrible
# so we reduce the number of cached files on non-Linux systems by remove node_modules from cache path.
- name: Cache non-full yarn cache on non-Linux
uses: actions/cache@v4
if: ${{ inputs.full-cache != 'true' && runner.os != 'Linux' }}
with:
path: |
${{ steps.yarn-cache.outputs.yarn_global_cache }}
key: node_modules-cache-${{ github.job }}-${{ runner.os }}-${{ runner.arch }}-${{ steps.system-info.outputs.name }}-${{ steps.system-info.outputs.release }}-${{ steps.system-info.outputs.version }}
- name: Cache full yarn cache on Linux
uses: actions/cache@v4
if: ${{ inputs.full-cache == 'true' && runner.os == 'Linux' }}
with:
path: |
node_modules
${{ steps.yarn-cache.outputs.yarn_global_cache }}
key: node_modules-cache-full-${{ runner.os }}-${{ runner.arch }}-${{ steps.system-info.outputs.name }}-${{ steps.system-info.outputs.release }}-${{ steps.system-info.outputs.version }}
- name: Cache full yarn cache on non-Linux
uses: actions/cache@v4
if: ${{ inputs.full-cache == 'true' && runner.os != 'Linux' }}
with:
path: |
${{ steps.yarn-cache.outputs.yarn_global_cache }}
key: node_modules-cache-full-${{ runner.os }}-${{ runner.arch }}-${{ steps.system-info.outputs.name }}-${{ steps.system-info.outputs.release }}-${{ steps.system-info.outputs.version }}
- name: yarn install
if: ${{ inputs.package-install == 'true' }}
continue-on-error: true
shell: bash
working-directory: ${{ steps.workspace-path.outputs.workspace_path }}
run: yarn ${{ inputs.extra-flags }}
env:
HUSKY: '0'
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: '1'
ELECTRON_SKIP_BINARY_DOWNLOAD: '1'
SENTRYCLI_SKIP_DOWNLOAD: '1'
DEBUG: '*'
- name: yarn install (try again)
if: ${{ steps.install.outcome == 'failure' }}
shell: bash
run: yarn ${{ inputs.extra-flags }}
env:
HUSKY: '0'
@@ -142,7 +90,6 @@ runs:
id: playwright-version
if: ${{ inputs.playwright-install == 'true' }}
shell: bash
working-directory: ${{ steps.workspace-path.outputs.workspace_path }}
run: echo "version=$(yarn why --json @playwright/test | grep -h 'workspace:.' | jq --raw-output '.children[].locator' | sed -e 's/@playwright\/test@.*://' | head -n 1)" >> $GITHUB_OUTPUT
# Attempt to restore the correct Playwright browser binaries based on the
@@ -151,12 +98,12 @@ runs:
# Note: Playwright's cache directory is hard coded because that's what it
# says to do in the docs. There doesn't appear to be a command that prints
# it out for us.
- uses: actions/cache@v4
- uses: actions/cache@v3
id: playwright-cache
if: ${{ inputs.playwright-install == 'true' }}
with:
path: ${{ steps.workspace-path.outputs.workspace_path }}/node_modules/.cache/ms-playwright
key: '${{ runner.os }}-${{ runner.arch }}-${{ steps.system-info.outputs.name }}-${{ steps.system-info.outputs.release }}-${{ steps.system-info.outputs.version }}-playwright-${{ steps.playwright-version.outputs.version }}'
path: '~/.cache/ms-playwright'
key: '${{ runner.os }}-${{ runner.arch }}-playwright-${{ steps.playwright-version.outputs.version }}'
# As a fallback, if the Playwright version has changed, try use the
# most recently cached version. There's a good chance that at least one
# of the browser binary versions haven't been updated, so Playwright can
@@ -166,44 +113,44 @@ runs:
# date cache, but still let Playwright decide if it needs to download
# new binaries or not.
restore-keys: |
${{ runner.os }}-${{ runner.arch }}-${{ steps.system-info.outputs.name }}-${{ steps.system-info.outputs.release }}-${{ steps.system-info.outputs.version }}-playwright-
${{ runner.os }}-${{ runner.arch }}-playwright-
# If the Playwright browser binaries weren't able to be restored, we tell
# playwright to install everything for us.
- name: Install Playwright's dependencies
shell: bash
if: inputs.playwright-install == 'true'
run: yarn playwright install --with-deps chromium webkit
working-directory: ${{ steps.workspace-path.outputs.workspace_path }}
env:
PLAYWRIGHT_BROWSERS_PATH: ${{ steps.workspace-path.outputs.workspace_path }}/node_modules/.cache/ms-playwright
run: yarn playwright install --with-deps chromium
- name: Get installed Electron version
id: electron-version
if: ${{ inputs.electron-install == 'true' }}
working-directory: ${{ steps.workspace-path.outputs.workspace_path }}
shell: bash
run: |
echo "version=$(yarn why --json electron | grep -h 'workspace:.' | jq --raw-output '.children[].locator' | sed -e 's/@playwright\/test@.*://' | head -n 1)" >> $GITHUB_OUTPUT
- uses: actions/cache@v4
- uses: actions/cache@v3
id: electron-cache
if: ${{ inputs.electron-install == 'true' }}
with:
path: ${{ steps.workspace-path.outputs.workspace_path }}/node_modules/.cache/electron
key: '${{ runner.os }}-${{ runner.arch }}-${{ steps.system-info.outputs.name }}-${{ steps.system-info.outputs.release }}-${{ steps.system-info.outputs.version }}-electron-${{ steps.electron-version.outputs.version }}'
path: 'node_modules/.cache/electron'
key: '${{ runner.os }}-{{ runner.arch }}-electron-${{ steps.electron-version.outputs.version }}'
restore-keys: |
${{ runner.os }}-${{ runner.arch }}-${{ steps.system-info.outputs.name }}-${{ steps.system-info.outputs.release }}-${{ steps.system-info.outputs.version }}-electron-
${{ runner.os }}-{{ runner.arch }}-electron-
- name: Install Electron binary
shell: bash
if: inputs.electron-install == 'true'
run: node ./node_modules/electron/install.js
working-directory: ${{ steps.workspace-path.outputs.workspace_path }}
env:
electron_config_cache: ${{ steps.workspace-path.outputs.workspace_path }}/node_modules/.cache/electron
ELECTRON_OVERRIDE_DIST_PATH: ./node_modules/.cache/electron
- name: Write PLAYWRIGHT_BROWSERS_PATH env
- name: Build Infra
shell: bash
run: |
echo "PLAYWRIGHT_BROWSERS_PATH=${{ steps.workspace-path.outputs.workspace_path }}/node_modules/.cache/ms-playwright" >> $GITHUB_ENV
if: inputs.build-infra == 'true'
run: yarn run build:infra
- name: Build Plugins
if: inputs.build-plugins == 'true'
shell: bash
run: yarn run build:plugins

View File

@@ -1,24 +0,0 @@
name: Setup Version
description: 'Setup Version'
outputs:
APP_VERSION:
description: 'App Version'
value: ${{ steps.version.outputs.APP_VERSION }}
runs:
using: 'composite'
steps:
- name: 'Write Version'
id: version
shell: bash
run: |
if [ "${{ github.ref_type }}" == "tag" ]; then
APP_VERSION=$(echo "${{ github.ref_name }}" | sed 's/^v//')
else
PACKAGE_VERSION=$(node -p "require('./package.json').version")
TIME_VERSION=$(date +%Y%m%d%H%M)
GIT_SHORT_HASH=$(git rev-parse --short HEAD)
APP_VERSION=$PACKAGE_VERSION-nightly-$GIT_SHORT_HASH
fi
echo $APP_VERSION
echo "APP_VERSION=$APP_VERSION" >> "$GITHUB_OUTPUT"
./scripts/set-version.sh $APP_VERSION

31
.github/dependabot.yml vendored Normal file
View File

@@ -0,0 +1,31 @@
version: 2
updates:
- package-ecosystem: 'npm'
directory: '/'
groups:
all-npm-dependencies:
patterns:
- '*'
schedule:
interval: 'weekly'
versioning-strategy: increase
commit-message:
prefix: 'chore'
- package-ecosystem: 'cargo'
directory: '/'
schedule:
interval: 'weekly'
versioning-strategy: auto
commit-message:
prefix: 'chore'
groups:
all-cargo-dependencies:
patterns:
- '*'
- package-ecosystem: 'github-actions'
directory: '/'
schedule:
interval: 'daily'
commit-message:
prefix: 'ci'

View File

@@ -1,8 +1,6 @@
FROM openresty/openresty:1.27.1.1-0-buster
FROM openresty/openresty:1.21.4.1-0-buster
WORKDIR /app
COPY ./packages/frontend/apps/web/dist ./dist
COPY ./packages/frontend/admin/dist ./admin
COPY ./packages/frontend/apps/mobile/dist ./mobile
COPY ./packages/frontend/core/dist ./dist
COPY ./.github/deployment/front/nginx.conf /usr/local/openresty/nginx/conf/nginx.conf
COPY ./.github/deployment/front/affine.nginx.conf /etc/nginx/conf.d/affine.nginx.conf

View File

@@ -1,42 +1,13 @@
server {
listen 8080;
location /admin {
root /app/;
index index.html;
try_files $uri/index.html $uri/ $uri /admin/index.html;
}
listen 8080;
root /app/dist;
set $app_root_path /app/dist/;
set $mobile_root /app/dist/;
set_by_lua $affine_env 'return os.getenv("AFFINE_ENV")';
location / {
try_files $uri $uri/ /index.html;
}
if ($affine_env = "dev") {
set $mobile_root /app/mobile/;
}
# https://gist.github.com/mariusom/6683dc52b1cad1a1f372e908bdb209d0
if ($http_user_agent ~* "(android|bb\d+|meego).+mobile|avantgo|bada\/|blackberry|blazer|compal|elaine|fennec|hiptop|iemobile|ip(hone|od)|iris|kindle|lge |maemo|midp|mmp|mobile.+firefox|netfront|opera m(ob|in)i|palm( os)?|phone|p(ixi|re)\/|plucker|pocket|psp|series(4|6)0|symbian|treo|up\.(browser|link)|vodafone|wap|windows ce|xda|xiino") {
set $app_root_path $mobile_root;
}
if ($http_user_agent ~* "^(1207|6310|6590|3gso|4thp|50[1-6]i|770s|802s|a wa|abac|ac(er|oo|s\-)|ai(ko|rn)|al(av|ca|co)|amoi|an(ex|ny|yw)|aptu|ar(ch|go)|as(te|us)|attw|au(di|\-m|r |s )|avan|be(ck|ll|nq)|bi(lb|rd)|bl(ac|az)|br(e|v)w|bumb|bw\-(n|u)|c55\/|capi|ccwa|cdm\-|cell|chtm|cldc|cmd\-|co(mp|nd)|craw|da(it|ll|ng)|dbte|dc\-s|devi|dica|dmob|do(c|p)o|ds(12|\-d)|el(49|ai)|em(l2|ul)|er(ic|k0)|esl8|ez([4-7]0|os|wa|ze)|fetc|fly(\-|_)|g1 u|g560|gene|gf\-5|g\-mo|go(\.w|od)|gr(ad|un)|haie|hcit|hd\-(m|p|t)|hei\-|hi(pt|ta)|hp( i|ip)|hs\-c|ht(c(\-| |_|a|g|p|s|t)|tp)|hu(aw|tc)|i\-(20|go|ma)|i230|iac( |\-|\/)|ibro|idea|ig01|ikom|im1k|inno|ipaq|iris|ja(t|v)a|jbro|jemu|jigs|kddi|keji|kgt( |\/)|klon|kpt |kwc\-|kyo(c|k)|le(no|xi)|lg( g|\/(k|l|u)|50|54|\-[a-w])|libw|lynx|m1\-w|m3ga|m50\/|ma(te|ui|xo)|mc(01|21|ca)|m\-cr|me(rc|ri)|mi(o8|oa|ts)|mmef|mo(01|02|bi|de|do|t(\-| |o|v)|zz)|mt(50|p1|v )|mwbp|mywa|n10[0-2]|n20[2-3]|n30(0|2)|n50(0|2|5)|n7(0(0|1)|10)|ne((c|m)\-|on|tf|wf|wg|wt)|nok(6|i)|nzph|o2im|op(ti|wv)|oran|owg1|p800|pan(a|d|t)|pdxg|pg(13|\-([1-8]|c))|phil|pire|pl(ay|uc)|pn\-2|po(ck|rt|se)|prox|psio|pt\-g|qa\-a|qc(07|12|21|32|60|\-[2-7]|i\-)|qtek|r380|r600|raks|rim9|ro(ve|zo)|s55\/|sa(ge|ma|mm|ms|ny|va)|sc(01|h\-|oo|p\-)|sdk\/|se(c(\-|0|1)|47|mc|nd|ri)|sgh\-|shar|sie(\-|m)|sk\-0|sl(45|id)|sm(al|ar|b3|it|t5)|so(ft|ny)|sp(01|h\-|v\-|v )|sy(01|mb)|t2(18|50)|t6(00|10|18)|ta(gt|lk)|tcl\-|tdg\-|tel(i|m)|tim\-|t\-mo|to(pl|sh)|ts(70|m\-|m3|m5)|tx\-9|up(\.b|g1|si)|utst|v400|v750|veri|vi(rg|te)|vk(40|5[0-3]|\-v)|vm40|voda|vulc|vx(52|53|60|61|70|80|81|83|85|98)|w3c(\-| )|webc|whit|wi(g |nc|nw)|wmlb|wonu|x700|yas\-|your|zeto|zte\-)") {
set $app_root_path $mobile_root;
}
location ~ ^/(_plugin|assets|imgs|js|plugins|static)/ {
root $app_root_path;
try_files $uri $uri/ =404;
}
location / {
root $app_root_path;
index index.html;
try_files $uri $uri/ /index.html;
add_header Cache-Control "private, no-cache, no-store, max-age=0, must-revalidate";
}
error_page 404 /404.html;
location = /404.html {
internal;
}
error_page 404 /404.html;
location = /404.html {
internal;
}
}

View File

@@ -1,15 +1,14 @@
worker_processes 4;
worker_processes 4;
error_log /var/log/nginx/error.log warn;
pcre_jit on;
env AFFINE_ENV;
events {
worker_connections 1024;
worker_connections 1024;
}
http {
include mime.types;
log_format main '$remote_addr [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
include /etc/nginx/conf.d/*.conf;
include mime.types;
log_format main '$remote_addr [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
include /etc/nginx/conf.d/*.conf;
}

View File

@@ -1,13 +1,10 @@
FROM node:20-bookworm-slim
FROM node:18-bookworm-slim
COPY ./packages/backend/server /app
COPY ./packages/frontend/apps/web/dist /app/static
COPY ./packages/frontend/admin/dist /app/static/admin
COPY ./packages/frontend/apps/mobile/dist /app/static/mobile
WORKDIR /app
RUN apt-get update && \
apt-get install -y --no-install-recommends openssl && \
rm -rf /var/lib/apt/lists/*
CMD ["node", "--import", "./scripts/register.js", "./dist/index.js"]
CMD ["node", "--es-module-specifier-resolution=node", "./dist/index.js"]

View File

@@ -1,60 +0,0 @@
services:
affine:
image: ghcr.io/toeverything/affine-graphql:stable
container_name: affine_selfhosted
command:
['sh', '-c', 'node ./scripts/self-host-predeploy && node ./dist/index.js']
ports:
- '3010:3010'
- '5555:5555'
depends_on:
redis:
condition: service_healthy
postgres:
condition: service_healthy
volumes:
# custom configurations
- ~/.affine/self-host/config:/root/.affine/config
# blob storage
- ~/.affine/self-host/storage:/root/.affine/storage
logging:
driver: 'json-file'
options:
max-size: '1000m'
restart: unless-stopped
environment:
- NODE_OPTIONS="--import=./scripts/register.js"
- AFFINE_CONFIG_PATH=/root/.affine/config
- REDIS_SERVER_HOST=redis
- DATABASE_URL=postgres://affine:affine@postgres:5432/affine
- NODE_ENV=production
# Telemetry allows us to collect data on how you use the affine. This data will helps us improve the app and provide better features.
# Uncomment next line if you wish to quit telemetry.
# - TELEMETRY_ENABLE=false
redis:
image: redis
container_name: affine_redis
restart: unless-stopped
volumes:
- ~/.affine/self-host/redis:/data
healthcheck:
test: ['CMD', 'redis-cli', '--raw', 'incr', 'ping']
interval: 10s
timeout: 5s
retries: 5
postgres:
image: postgres:16
container_name: affine_postgres
restart: unless-stopped
volumes:
- ~/.affine/self-host/postgres:/var/lib/postgresql/data
healthcheck:
test: ['CMD-SHELL', 'pg_isready -U affine']
interval: 10s
timeout: 5s
retries: 5
environment:
POSTGRES_USER: affine
POSTGRES_PASSWORD: affine
POSTGRES_DB: affine
PGDATA: /var/lib/postgresql/data/pgdata

1
.github/helm/affine-cloud/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
charts/

View File

@@ -20,4 +20,4 @@
.project
.idea/
*.tmproj
.vscode/
.vscode/

6
.github/helm/affine-cloud/Chart.lock vendored Normal file
View File

@@ -0,0 +1,6 @@
dependencies:
- name: postgresql
repository: https://charts.bitnami.com/bitnami
version: 12.5.8
digest: sha256:c91c0dc1370e879538dc9d6e435e731a726ef99d6a3b081372318483792b48a7
generated: "2023-06-27T18:34:12.683806+08:00"

12
.github/helm/affine-cloud/Chart.yaml vendored Normal file
View File

@@ -0,0 +1,12 @@
apiVersion: v2
name: affine-cloud
description: A Helm chart for AFFiNE Cloud
type: application
version: 0.6.1
appVersion: '0.6.1'
dependencies:
- name: postgresql
version: 12.5.8
repository: https://charts.bitnami.com/bitnami

30
.github/helm/affine-cloud/readme.md vendored Normal file
View File

@@ -0,0 +1,30 @@
# Helm Chart Configuration
The following table lists the configurable parameters of this Helm chart and their default values.
## AFFiNE Cloud Server parameters
| Parameter | Description | Default |
| ------------------------------ | -------------------------------------------------- | ------------------ |
| `affineCloud.tag` | The Docker tag of the AffineCloud image to be used | `'nightly-latest'` |
| `affineCloud.resources.cpu` | The CPU resources allocated for AffineCloud | `'250m'` |
| `affineCloud.resources.memory` | The memory resources allocated for AffineCloud | `'0.5Gi'` |
| `affineCloud.signKey` | The key used to sign the JWT tokens | `'c2VjcmV0'` |
| `affineCloud.service.type` | The type of the Kubernetes service | `'ClusterIP'` |
| `affineCloud.service.port` | The port of the Kubernetes service | `'http'` |
| `affineCloud.mail.account` | The email account used to send emails | `''` |
| `affineCloud.mail.password` | The password of the email account | `''` |
## PostgreSQL parameters
| Parameter | Description | Default |
| -------------------------------------------- | ------------------------------------------------------------------------------------- | ------------ |
| `postgresql.auth.username` | Username for the PostgreSQL database | `'affine'` |
| `postgresql.auth.password` | Password for the PostgreSQL database. Please change this for production environments. | `'password'` |
| `postgresql.auth.database` | The name of the default database that will be created on image startup | `'affine'` |
| `postgresql.primary.resources.limits.cpu` | The CPU resources allocated for the PostgreSQL primary node | `'500m'` |
| `postgresql.primary.resources.limits.memory` | The memory resources allocated for the PostgreSQL primary node | `'0.5Gi'` |
For more postgres parameters, please refer to: https://artifacthub.io/packages/helm/bitnami/postgresql
Please note that for the `postgresql.auth.password`, you should provide your own password for production environments. The default value is provided only for demonstration purposes.

View File

@@ -0,0 +1,51 @@
{{/*
Expand the name of the chart.
*/}}
{{- define "affine-cloud.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "affine-cloud.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "affine-cloud.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Common labels
*/}}
{{- define "affine-cloud.labels" -}}
helm.sh/chart: {{ include "affine-cloud.chart" . }}
{{ include "affine-cloud.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}
{{/*
Selector labels
*/}}
{{- define "affine-cloud.selectorLabels" -}}
app.kubernetes.io/name: {{ include "affine-cloud.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}

View File

@@ -0,0 +1,51 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: "{{ include "affine-cloud.fullname" . }}"
labels:
{{- include "affine-cloud.labels" . | nindent 4 }}
spec:
replicas: 1
selector:
matchLabels:
{{- include "affine-cloud.selectorLabels" . | nindent 6 }}
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 2
template:
metadata:
labels:
{{- include "affine-cloud.selectorLabels" . | nindent 8 }}
spec:
restartPolicy: Always
containers:
- name: affine-cloud
image: "ghcr.io/toeverything/cloud-self-hosted:{{ .Values.affineCloud.tag | default .Chart.AppVersion }}"
env:
- name: PG_USER
value: "{{ .Values.postgresql.auth.username }}"
- name: PG_PASS
value: "{{ .Values.postgresql.auth.password }}"
- name: PG_DATABASE
value: "{{ .Values.postgresql.auth.database }}"
- name: PG_HOST
value: "{{ .Values.postgresql.fullnameOverride | default (printf "%s-postgresql" .Release.Name) }}"
- name: DATABASE_URL
value: "{{ .Values.affineCloud.databaseUrl | default "postgresql://$(PG_USER):$(PG_PASS)@$(PG_HOST)/$(PG_DATABASE)" }}"
envFrom:
- secretRef:
name: affine-cloud-secret
ports:
- containerPort: 3000
livenessProbe:
httpGet:
path: /api/healthz
port: 3000
failureThreshold: 1
initialDelaySeconds: 10
periodSeconds: 10
resources:
limits:
cpu: "{{ .Values.affineCloud.resources.cpu }}"
memory: "{{ .Values.affineCloud.resources.memory }}"

View File

@@ -0,0 +1,9 @@
apiVersion: v1
kind: Secret
metadata:
name: affine-cloud-secret
type: Opaque
data:
SIGN_KEY: "{{ .Values.affineCloud.signKey }}"
MAIL_ACCOUNT: "{{ .Values.affineCloud.mail.account }}"
MAIL_PASSWORD: "{{ .Values.affineCloud.mail.password }}"

View File

@@ -0,0 +1,15 @@
apiVersion: v1
kind: Service
metadata:
name: "{{ include "affine-cloud.fullname" . }}"
labels:
{{- include "affine-cloud.labels" . | nindent 4 }}
spec:
type: "{{ .Values.affineCloud.service.type }}"
ports:
- name: http
protocol: TCP
port: {{ .Values.affineCloud.service.port }}
targetPort: 3000
selector:
{{- include "affine-cloud.selectorLabels" . | nindent 4 }}

30
.github/helm/affine-cloud/values.yaml vendored Normal file
View File

@@ -0,0 +1,30 @@
affineCloud:
tag: 'canary-5e0d5e0cc65ea46f326fdde12658bfac59b38c9f-0949'
# databaseUrl: 'postgresql://affine:password@affine-cloud-postgresql:5432/affine'
signKey: TUFtdFdzQTJhdGJuem01TA==
mail:
account: ''
password: ''
service:
type: ClusterIP
port: 80
resources:
cpu: '250m'
memory: 0.5Gi
postgresql:
fullnameOverride: tcp-postgresql
auth:
# only for demo, please modify it at prod env
username: affine
password: password
database: affine
primary:
initdb:
scripts:
01-init.sql: |
CREATE DATABASE affine_binary;
GRANT ALL PRIVILEGES ON DATABASE affine_binary TO affine;
resources:
limits:
cpu: '500m'
memory: 0.5Gi

View File

@@ -3,4 +3,4 @@ name: affine
description: AFFiNE cloud chart
type: application
version: 0.0.0
appVersion: "0.19.0"
appVersion: '0.7.0-canary.18'

View File

@@ -1,6 +0,0 @@
apiVersion: v2
name: cloud-sql-proxy
description: Google Cloud SQL Proxy
type: application
version: 0.0.0
appVersion: "2.8.1"

View File

@@ -1,18 +0,0 @@
{{- if .Values.global.database.gcloud.enabled -}}
1. Get the application URL by running these commands:
{{- if contains "NodePort" .Values.service.type }}
export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "gcloud-sql-proxy.fullname" . }})
export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
echo http://$NODE_IP:$NODE_PORT
{{- else if contains "LoadBalancer" .Values.service.type }}
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status of by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "gcloud-sql-proxy.fullname" . }}'
export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include "gcloud-sql-proxy.fullname" . }} --template "{{"{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}"}}")
echo http://$SERVICE_IP:{{ .Values.service.port }}
{{- else if contains "ClusterIP" .Values.service.type }}
export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "gcloud-sql-proxy.name" . }},app.kubernetes.io/instance={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
export CONTAINER_PORT=$(kubectl get pod --namespace {{ .Release.Namespace }} $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
echo "Visit http://127.0.0.1:8080 to use your application"
kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 8080:$CONTAINER_PORT
{{- end }}
{{- end }}

View File

@@ -1,62 +0,0 @@
{{/*
Expand the name of the chart.
*/}}
{{- define "gcloud-sql-proxy.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "gcloud-sql-proxy.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "gcloud-sql-proxy.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Common labels
*/}}
{{- define "gcloud-sql-proxy.labels" -}}
helm.sh/chart: {{ include "gcloud-sql-proxy.chart" . }}
{{ include "gcloud-sql-proxy.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}
{{/*
Selector labels
*/}}
{{- define "gcloud-sql-proxy.selectorLabels" -}}
app.kubernetes.io/name: {{ include "gcloud-sql-proxy.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
{{/*
Create the name of the service account to use
*/}}
{{- define "gcloud-sql-proxy.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "gcloud-sql-proxy.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}

View File

@@ -1,132 +0,0 @@
{{- if .Values.global.database.gcloud.enabled -}}
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "gcloud-sql-proxy.fullname" . }}
labels:
{{- include "gcloud-sql-proxy.labels" . | nindent 4 }}
spec:
replicas: {{ .Values.replicaCount }}
selector:
matchLabels:
{{- include "gcloud-sql-proxy.selectorLabels" . | nindent 6 }}
template:
metadata:
{{- with .Values.podAnnotations }}
annotations:
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "gcloud-sql-proxy.labels" . | nindent 8 }}
{{- with .Values.podLabels }}
{{- toYaml . | nindent 8 }}
{{- end }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ include "gcloud-sql-proxy.serviceAccountName" . }}
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
containers:
- name: {{ .Chart.Name }}
securityContext:
{{- toYaml .Values.securityContext | nindent 12 }}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
image: "{{ .Values.image.repository }}:{{ .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
args:
- "--address"
- "0.0.0.0"
- "--structured-logs"
- "--auto-iam-authn"
- "{{ .Values.global.database.gcloud.connectionName }}"
env:
# Enable HTTP healthchecks on port 9801. This enables /liveness,
# /readiness and /startup health check endpoints. Allow connections
# listen for connections on any interface (0.0.0.0) so that the
# k8s management components can reach these endpoints.
- name: CSQL_PROXY_HEALTH_CHECK
value: "true"
- name: CSQL_PROXY_HTTP_PORT
value: "9801"
- name: CSQL_PROXY_HTTP_ADDRESS
value: 0.0.0.0
ports:
- name: cloud-sql-proxy
containerPort: {{ .Values.global.database.gcloud.proxyPort }}
protocol: TCP
- containerPort: 9801
protocol: TCP
# The /startup probe returns OK when the proxy is ready to receive
# connections from the application. In this example, k8s will check
# once a second for 60 seconds.
startupProbe:
failureThreshold: 60
httpGet:
path: /startup
port: 9801
scheme: HTTP
periodSeconds: 1
successThreshold: 1
timeoutSeconds: 10
# The /liveness probe returns OK as soon as the proxy application has
# begun its startup process and continues to return OK until the
# process stops.
livenessProbe:
failureThreshold: 3
httpGet:
path: /liveness
port: 9801
scheme: HTTP
# The probe will be checked every 10 seconds.
periodSeconds: 10
# Number of times the probe is allowed to fail before the transition
# from healthy to failure state.
#
# If periodSeconds = 60, 5 tries will result in five minutes of
# checks. The proxy starts to refresh a certificate five minutes
# before its expiration. If those five minutes lapse without a
# successful refresh, the liveness probe will fail and the pod will be
# restarted.
successThreshold: 1
# The probe will fail if it does not respond in 10 seconds
timeoutSeconds: 10
readinessProbe:
# The /readiness probe returns OK when the proxy can establish
# a new connections to its databases.
httpGet:
path: /readiness
port: 9801
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 10
# Number of times the probe must report success to transition from failure to healthy state.
# Defaults to 1 for readiness probe.
successThreshold: 1
failureThreshold: 6
resources:
{{- toYaml .Values.resources | nindent 12 }}
{{- with .Values.volumeMounts }}
volumeMounts:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- with .Values.volumes }}
volumes:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}

View File

@@ -1,17 +0,0 @@
{{- if .Values.global.database.gcloud.enabled -}}
apiVersion: v1
kind: Service
metadata:
name: {{ include "gcloud-sql-proxy.fullname" . }}
labels:
{{- include "gcloud-sql-proxy.labels" . | nindent 4 }}
spec:
type: {{ .Values.service.type }}
ports:
- port: {{ .Values.global.database.port }}
targetPort: cloud-sql-proxy
protocol: TCP
name: cloud-sql-proxy
selector:
{{- include "gcloud-sql-proxy.selectorLabels" . | nindent 4 }}
{{- end }}

View File

@@ -1,15 +0,0 @@
{{- if .Values.global.database.gcloud.enabled -}}
{{- if .Values.serviceAccount.create -}}
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ include "gcloud-sql-proxy.serviceAccountName" . }}
labels:
{{- include "gcloud-sql-proxy.labels" . | nindent 4 }}
{{- with .Values.serviceAccount.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
automountServiceAccountToken: {{ .Values.serviceAccount.automount }}
{{- end }}
{{- end }}

View File

@@ -1,17 +0,0 @@
{{- if .Values.global.database.gcloud.enabled -}}
apiVersion: v1
kind: Pod
metadata:
name: "{{ include "gcloud-sql-proxy.fullname" . }}-test-connection"
labels:
{{- include "gcloud-sql-proxy.labels" . | nindent 4 }}
annotations:
"helm.sh/hook": test
spec:
containers:
- name: wget
image: busybox
command: ['wget']
args: ['{{ include "gcloud-sql-proxy.fullname" . }}:{{ .Values.service.port }}']
restartPolicy: Never
{{- end }}

View File

@@ -1,40 +0,0 @@
replicaCount: 3
image:
# the tag is defined as chart appVersion.
repository: gcr.io/cloud-sql-connectors/cloud-sql-proxy
pullPolicy: IfNotPresent
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
serviceAccount:
create: true
automount: true
annotations: {}
name: ""
podAnnotations: {}
podLabels: {}
podSecurityContext:
fsGroup: 2000
securityContext:
runAsNonRoot: true
service:
type: ClusterIP
port: 5432
resources:
limits:
memory: "4Gi"
cpu: "2"
volumes: []
volumeMounts: []
nodeSelector: {}
tolerations: []
affinity: {}

View File

@@ -3,9 +3,4 @@ name: graphql
description: AFFiNE GraphQL server
type: application
version: 0.0.0
appVersion: "0.19.0"
dependencies:
- name: gcloud-sql-proxy
version: 0.0.0
repository: "file://../gcloud-sql-proxy"
condition: .global.database.gcloud.enabled
appVersion: '0.7.0-canary.18'

View File

@@ -61,3 +61,18 @@ Create the name of the service account to use
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}
{{- define "jwt.key" -}}
{{- $secret := lookup "v1" "Secret" .Release.Namespace .Values.app.jwt.secretName -}}
{{- if and $secret $secret.data.private -}}
{{/*
Reusing existing secret data
*/}}
key: {{ $secret.data.private }}
{{- else -}}
{{/*
Generate new data
*/}}
key: {{ genPrivateKey "ecdsa" | b64enc }}
{{- end -}}
{{- end -}}

View File

@@ -1,11 +0,0 @@
{{- if .Values.app.copilot.enabled -}}
apiVersion: v1
kind: Secret
metadata:
name: "{{ .Values.app.copilot.secretName }}"
type: Opaque
data:
openaiSecret: {{ .Values.app.copilot.openai.key | b64enc }}
falSecret: {{ .Values.app.copilot.fal.key | b64enc }}
unsplashSecret: {{ .Values.app.copilot.unsplash.key | b64enc }}
{{- end }}

View File

@@ -28,23 +28,21 @@ spec:
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
env:
- name: AFFINE_PRIVATE_KEY
- name: AUTH_PRIVATE_KEY
valueFrom:
secretKeyRef:
name: "{{ .Values.global.secret.secretName }}"
name: "{{ .Values.app.jwt.secretName }}"
key: key
- name: NODE_ENV
value: "{{ .Values.env }}"
- name: NODE_OPTIONS
value: "--max-old-space-size=4096"
- name: NO_COLOR
value: "1"
- name: DEPLOYMENT_TYPE
value: "affine"
- name: SERVER_FLAVOR
value: "graphql"
- name: AFFINE_ENV
value: "{{ .Release.Namespace }}"
- name: NEXTAUTH_URL
value: "{{ .Values.global.ingress.host }}"
- name: DATABASE_PASSWORD
valueFrom:
secretKeyRef:
@@ -73,39 +71,37 @@ spec:
value: "{{ .Values.app.path }}"
- name: AFFINE_SERVER_HOST
value: "{{ .Values.app.host }}"
- name: AFFINE_SERVER_HTTPS
value: "{{ .Values.app.https }}"
- name: ENABLE_R2_OBJECT_STORAGE
value: "{{ .Values.global.objectStorage.r2.enabled }}"
value: "{{ .Values.app.objectStorage.r2.enabled }}"
- name: ENABLE_CAPTCHA
value: "{{ .Values.app.captcha.enabled }}"
- name: FEATURES_EARLY_ACCESS_PREVIEW
value: "{{ .Values.app.features.earlyAccessPreview }}"
- name: FEATURES_SYNC_CLIENT_VERSION_CHECK
value: "{{ .Values.app.features.syncClientVersionCheck }}"
- name: MAILER_HOST
- name: OAUTH_EMAIL_SENDER
valueFrom:
secretKeyRef:
name: "{{ .Values.app.mailer.secretName }}"
key: host
- name: MAILER_PORT
valueFrom:
secretKeyRef:
name: "{{ .Values.app.mailer.secretName }}"
key: port
- name: MAILER_USER
valueFrom:
secretKeyRef:
name: "{{ .Values.app.mailer.secretName }}"
key: user
- name: MAILER_PASSWORD
valueFrom:
secretKeyRef:
name: "{{ .Values.app.mailer.secretName }}"
key: password
- name: MAILER_SENDER
valueFrom:
secretKeyRef:
name: "{{ .Values.app.mailer.secretName }}"
name: "{{ .Values.app.oauth.email.secretName }}"
key: sender
- name: OAUTH_EMAIL_LOGIN
valueFrom:
secretKeyRef:
name: "{{ .Values.app.oauth.email.secretName }}"
key: login
- name: OAUTH_EMAIL_SERVER
valueFrom:
secretKeyRef:
name: "{{ .Values.app.oauth.email.secretName }}"
key: server
- name: OAUTH_EMAIL_PORT
valueFrom:
secretKeyRef:
name: "{{ .Values.app.oauth.email.secretName }}"
key: port
- name: OAUTH_EMAIL_PASSWORD
valueFrom:
secretKeyRef:
name: "{{ .Values.app.oauth.email.secretName }}"
key: password
- name: STRIPE_API_KEY
valueFrom:
secretKeyRef:
@@ -122,22 +118,27 @@ spec:
- name: DOC_MERGE_USE_JWST_CODEC
value: "true"
{{ end }}
{{ if .Values.global.objectStorage.r2.enabled }}
{{ if .Values.app.objectStorage.r2.enabled }}
- name: R2_OBJECT_STORAGE_ACCOUNT_ID
valueFrom:
secretKeyRef:
name: "{{ .Values.global.objectStorage.r2.secretName }}"
name: "{{ .Values.app.objectStorage.r2.secretName }}"
key: accountId
- name: R2_OBJECT_STORAGE_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: "{{ .Values.global.objectStorage.r2.secretName }}"
name: "{{ .Values.app.objectStorage.r2.secretName }}"
key: accessKeyId
- name: R2_OBJECT_STORAGE_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: "{{ .Values.global.objectStorage.r2.secretName }}"
name: "{{ .Values.app.objectStorage.r2.secretName }}"
key: secretAccessKey
- name: R2_OBJECT_STORAGE_BUCKET
valueFrom:
secretKeyRef:
name: "{{ .Values.app.objectStorage.r2.secretName }}"
key: bucket
{{ end }}
{{ if .Values.app.captcha.enabled }}
- name: CAPTCHA_TURNSTILE_SECRET
@@ -146,26 +147,7 @@ spec:
name: "{{ .Values.app.captcha.secretName }}"
key: turnstileSecret
{{ end }}
{{ if .Values.app.copilot.enabled }}
- name: COPILOT_OPENAI_API_KEY
valueFrom:
secretKeyRef:
name: "{{ .Values.app.copilot.secretName }}"
key: openaiSecret
- name: COPILOT_FAL_API_KEY
valueFrom:
secretKeyRef:
name: "{{ .Values.app.copilot.secretName }}"
key: falSecret
- name: COPILOT_UNSPLASH_API_KEY
valueFrom:
secretKeyRef:
name: "{{ .Values.app.copilot.secretName }}"
key: unsplashSecret
{{ end }}
{{ if .Values.app.oauth.google.enabled }}
- name: OAUTH_GOOGLE_ENABLED
value: "true"
- name: OAUTH_GOOGLE_CLIENT_ID
valueFrom:
secretKeyRef:
@@ -189,29 +171,36 @@ spec:
name: "{{ .Values.app.oauth.github.secretName }}"
key: clientSecret
{{ end }}
{{ if .Values.app.metrics.enabled }}
- name: METRICS_CUSTOMER_IO_TOKEN
valueFrom:
secretKeyRef:
name: "{{ .Values.app.metrics.secretName }}"
key: customerIoSecret
{{ end }}
ports:
- name: http
containerPort: {{ .Values.service.port }}
protocol: TCP
livenessProbe:
httpGet:
path: /info
path: /
port: http
initialDelaySeconds: {{ .Values.probe.initialDelaySeconds }}
readinessProbe:
httpGet:
path: /info
path: /
port: http
initialDelaySeconds: {{ .Values.probe.initialDelaySeconds }}
resources:
{{- toYaml .Values.resources | nindent 12 }}
{{ if .Values.global.database.gcloud.enabled }}
- name: cloud-sql-proxy
image: gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.6.0
args:
- "--structured-logs"
- "--auto-iam-authn"
- "{{ .Values.global.database.gcloud.connectionName }}"
securityContext:
runAsNonRoot: true
resources:
requests:
memory: "2Gi"
cpu: "1"
{{ end }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}

View File

@@ -0,0 +1,7 @@
apiVersion: v1
kind: Secret
metadata:
name: "{{ .Values.app.jwt.secretName }}"
type: Opaque
data:
{{- ( include "jwt.key" . ) | indent 2 -}}

View File

@@ -1,13 +0,0 @@
{{- if .Values.app.mailer.secretName -}}
apiVersion: v1
kind: Secret
metadata:
name: "{{ .Values.app.mailer.secretName }}"
type: Opaque
data:
host: "{{ .Values.app.mailer.host | b64enc }}"
port: "{{ .Values.app.mailer.port | b64enc }}"
user: "{{ .Values.app.mailer.user | b64enc }}"
password: "{{ .Values.app.mailer.password | b64enc }}"
sender: "{{ .Values.app.mailer.sender | b64enc }}"
{{- end }}

View File

@@ -1,9 +0,0 @@
{{- if .Values.app.metrics.enabled -}}
apiVersion: v1
kind: Secret
metadata:
name: "{{ .Values.app.metrics.secretName }}"
type: Opaque
data:
customerIoSecret: {{ .Values.app.metrics.customerIo.token | b64enc }}
{{- end }}

View File

@@ -22,8 +22,6 @@ spec:
value: "{{ .Values.env }}"
- name: AFFINE_ENV
value: "{{ .Release.Namespace }}"
- name: DEPLOYMENT_TYPE
value: "affine"
- name: DATABASE_PASSWORD
valueFrom:
secretKeyRef:
@@ -37,23 +35,6 @@ spec:
- name: DATABASE_URL
value: postgres://{{ .Values.global.database.user }}:$(DATABASE_PASSWORD)@{{ .Values.global.database.gcloud.cloudSqlInternal }}:{{ .Values.global.database.port }}/{{ .Values.global.database.name }}
{{ end }}
{{ if .Values.global.objectStorage.r2.enabled }}
- name: R2_OBJECT_STORAGE_ACCOUNT_ID
valueFrom:
secretKeyRef:
name: "{{ .Values.global.objectStorage.r2.secretName }}"
key: accountId
- name: R2_OBJECT_STORAGE_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: "{{ .Values.global.objectStorage.r2.secretName }}"
key: accessKeyId
- name: R2_OBJECT_STORAGE_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: "{{ .Values.global.objectStorage.r2.secretName }}"
key: secretAccessKey
{{ end }}
resources:
requests:
cpu: '100m'

View File

@@ -0,0 +1,13 @@
{{- if .Values.global.gke.enabled -}}
apiVersion: monitoring.googleapis.com/v1
kind: PodMonitoring
metadata:
name: "{{ .Chart.Name }}-monitoring"
spec:
selector:
matchLabels:
app.kubernetes.io/name: "{{ include "graphql.name" . }}"
endpoints:
- port: {{ .Values.service.port }}
interval: 30s
{{- end }}

View File

@@ -1,3 +1,15 @@
apiVersion: v1
kind: Secret
metadata:
name: "{{ .Values.app.oauth.email.secretName }}"
type: Opaque
data:
sender: "{{ .Values.app.oauth.email.sender | b64enc }}"
login: "{{ .Values.app.oauth.email.login | b64enc }}"
password: "{{ .Values.app.oauth.email.password | b64enc }}"
server: "{{ .Values.app.oauth.email.server | b64enc }}"
port: "{{ .Values.app.oauth.email.port | b64enc }}"
---
{{- if .Values.app.oauth.google.enabled -}}
apiVersion: v1
kind: Secret

View File

@@ -0,0 +1,12 @@
{{- if .Values.app.objectStorage.r2.enabled -}}
apiVersion: v1
kind: Secret
metadata:
name: "{{ .Values.app.objectStorage.r2.secretName }}"
type: Opaque
data:
accountId: {{ .Values.app.objectStorage.r2.accountId | b64enc }}
accessKeyId: {{ .Values.app.objectStorage.r2.accessKeyId | b64enc }}
secretAccessKey: {{ .Values.app.objectStorage.r2.secretAccessKey | b64enc }}
bucket: {{ .Values.app.objectStorage.r2.bucket | b64enc }}
{{- end }}

View File

@@ -1,18 +0,0 @@
{{- $privateKey := default (genPrivateKey "ecdsa") .Values.global.secret.privateKey | b64enc | quote }}
{{- if not .Values.global.secret.privateKey }}
{{- $existingKey := (lookup "v1" "Secret" .Release.Namespace .Values.global.secret.secretName) }}
{{- if $existingKey }}
{{- $privateKey = index $existingKey.data "key" }}
{{- end -}}
{{- end -}}
apiVersion: v1
kind: Secret
metadata:
name: {{ .Values.global.secret.secretName }}
annotations:
"helm.sh/resource-policy": "keep"
type: Opaque
data:
key: {{ $privateKey }}

View File

@@ -4,10 +4,6 @@ metadata:
name: {{ include "graphql.fullname" . }}
labels:
{{- include "graphql.labels" . | nindent 4 }}
{{- with .Values.service.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
type: {{ .Values.service.type }}
ports:

View File

@@ -16,20 +16,33 @@ app:
path: ''
# AFFINE_SERVER_HOST
host: '0.0.0.0'
https: true
doc:
mergeInterval: "3000"
jwt:
secretName: jwt-private-key
# base64 encoded ecdsa private key
privateKey: ''
captcha:
enabled: false
enable: false
secretName: captcha
turnstile:
secret: ''
copilot:
enabled: false
secretName: copilot
openai:
key: ''
objectStorage:
r2:
enabled: false
secretName: r2
accountId: ''
accessKeyId: ''
secretAccessKey: ''
bucket: ''
oauth:
email:
secretName: 'oauth-email'
sender: 'noreply@toeverything.info'
login: ''
password: ''
server: 'smtp.gmail.com'
port: '465'
google:
enabled: false
secretName: oauth-google
@@ -40,18 +53,6 @@ app:
secretName: oauth-github
clientId: ''
clientSecret: ''
mailer:
secretName: 'mailer'
host: 'smtp.gmail.com'
port: '465'
user: ''
password: ''
sender: 'noreply@toeverything.info'
metrics:
enabled: false
secretName: 'metrics'
customerIo:
token: ''
payment:
stripe:
secretName: 'stripe'
@@ -59,7 +60,6 @@ app:
webhookKey: ''
features:
earlyAccessPreview: false
syncClientVersionCheck: false
serviceAccount:
create: true
@@ -72,8 +72,11 @@ podSecurityContext:
fsGroup: 2000
resources:
requests:
limits:
cpu: '4'
memory: 8Gi
requests:
cpu: '2'
memory: 4Gi
probe:

View File

@@ -1,11 +0,0 @@
apiVersion: v2
name: renderer
description: AFFiNE renderer server
type: application
version: 0.0.0
appVersion: "0.16.0"
dependencies:
- name: gcloud-sql-proxy
version: 0.0.0
repository: "file://../gcloud-sql-proxy"
condition: .global.database.gcloud.enabled

View File

@@ -1,16 +0,0 @@
1. Get the application URL by running these commands:
{{- if contains "NodePort" .Values.service.type }}
export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "renderer.fullname" . }})
export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
echo http://$NODE_IP:$NODE_PORT
{{- else if contains "LoadBalancer" .Values.service.type }}
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status of by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "renderer.fullname" . }}'
export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include "renderer.fullname" . }} --template "{{"{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}"}}")
echo http://$SERVICE_IP:{{ .Values.service.port }}
{{- else if contains "ClusterIP" .Values.service.type }}
export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "renderer.name" . }},app.kubernetes.io/instance={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
export CONTAINER_PORT=$(kubectl get pod --namespace {{ .Release.Namespace }} $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
echo "Visit http://127.0.0.1:8080 to use your application"
kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 8080:$CONTAINER_PORT
{{- end }}

View File

@@ -1,63 +0,0 @@
{{/*
Expand the name of the chart.
*/}}
{{- define "renderer.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "renderer.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "renderer.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Common labels
*/}}
{{- define "renderer.labels" -}}
helm.sh/chart: {{ include "renderer.chart" . }}
{{ include "renderer.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
monitoring: enabled
{{- end }}
{{/*
Selector labels
*/}}
{{- define "renderer.selectorLabels" -}}
app.kubernetes.io/name: {{ include "renderer.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
{{/*
Create the name of the service account to use
*/}}
{{- define "renderer.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "renderer.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}

View File

@@ -1,124 +0,0 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "renderer.fullname" . }}
labels:
{{- include "renderer.labels" . | nindent 4 }}
spec:
replicas: {{ .Values.replicaCount }}
selector:
matchLabels:
{{- include "renderer.selectorLabels" . | nindent 6 }}
template:
metadata:
{{- with .Values.podAnnotations }}
annotations:
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "renderer.selectorLabels" . | nindent 8 }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ include "renderer.serviceAccountName" . }}
containers:
- name: {{ .Chart.Name }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
env:
- name: AFFINE_PRIVATE_KEY
valueFrom:
secretKeyRef:
name: "{{ .Values.global.secret.secretName }}"
key: key
- name: NODE_ENV
value: "{{ .Values.env }}"
- name: NODE_OPTIONS
value: "--max-old-space-size=4096"
- name: NO_COLOR
value: "1"
- name: DEPLOYMENT_TYPE
value: "affine"
- name: SERVER_FLAVOR
value: "renderer"
- name: AFFINE_ENV
value: "{{ .Release.Namespace }}"
- name: DATABASE_PASSWORD
valueFrom:
secretKeyRef:
name: pg-postgresql
key: postgres-password
- name: DATABASE_URL
value: postgres://{{ .Values.global.database.user }}:$(DATABASE_PASSWORD)@{{ .Values.global.database.url }}:{{ .Values.global.database.port }}/{{ .Values.global.database.name }}
- name: REDIS_SERVER_ENABLED
value: "true"
- name: REDIS_SERVER_HOST
value: "{{ .Values.global.redis.host }}"
- name: REDIS_SERVER_PORT
value: "{{ .Values.global.redis.port }}"
- name: REDIS_SERVER_USER
value: "{{ .Values.global.redis.username }}"
- name: REDIS_SERVER_PASSWORD
valueFrom:
secretKeyRef:
name: redis
key: redis-password
- name: REDIS_SERVER_DATABASE
value: "{{ .Values.global.redis.database }}"
- name: AFFINE_SERVER_PORT
value: "{{ .Values.service.port }}"
- name: AFFINE_SERVER_SUB_PATH
value: "{{ .Values.app.path }}"
- name: AFFINE_SERVER_HOST
value: "{{ .Values.app.host }}"
- name: AFFINE_SERVER_HTTPS
value: "{{ .Values.app.https }}"
- name: ENABLE_R2_OBJECT_STORAGE
value: "{{ .Values.global.objectStorage.r2.enabled }}"
{{ if .Values.global.objectStorage.r2.enabled }}
- name: R2_OBJECT_STORAGE_ACCOUNT_ID
valueFrom:
secretKeyRef:
name: "{{ .Values.global.objectStorage.r2.secretName }}"
key: accountId
- name: R2_OBJECT_STORAGE_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: "{{ .Values.global.objectStorage.r2.secretName }}"
key: accessKeyId
- name: R2_OBJECT_STORAGE_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: "{{ .Values.global.objectStorage.r2.secretName }}"
key: secretAccessKey
{{ end }}
ports:
- name: http
containerPort: {{ .Values.service.port }}
protocol: TCP
livenessProbe:
httpGet:
path: /info
port: http
initialDelaySeconds: {{ .Values.probe.initialDelaySeconds }}
readinessProbe:
httpGet:
path: /info
port: http
initialDelaySeconds: {{ .Values.probe.initialDelaySeconds }}
resources:
{{- toYaml .Values.resources | nindent 12 }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}

View File

@@ -1,19 +0,0 @@
apiVersion: v1
kind: Service
metadata:
name: {{ include "graphql.fullname" . }}
labels:
{{- include "graphql.labels" . | nindent 4 }}
{{- with .Values.service.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
type: {{ .Values.service.type }}
ports:
- port: {{ .Values.service.port }}
targetPort: http
protocol: TCP
name: http
selector:
{{- include "graphql.selectorLabels" . | nindent 4 }}

View File

@@ -1,12 +0,0 @@
{{- if .Values.serviceAccount.create -}}
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ include "graphql.serviceAccountName" . }}
labels:
{{- include "graphql.labels" . | nindent 4 }}
{{- with .Values.serviceAccount.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
{{- end }}

View File

@@ -1,15 +0,0 @@
apiVersion: v1
kind: Pod
metadata:
name: "{{ include "renderer.fullname" . }}-test-connection"
labels:
{{- include "renderer.labels" . | nindent 4 }}
annotations:
"helm.sh/hook": test
spec:
containers:
- name: wget
image: busybox
command: ['wget']
args: ['{{ include "renderer.fullname" . }}:{{ .Values.service.port }}']
restartPolicy: Never

View File

@@ -1,38 +0,0 @@
replicaCount: 1
image:
repository: ghcr.io/toeverything/affine-graphql
pullPolicy: IfNotPresent
tag: ''
imagePullSecrets: []
nameOverride: ''
fullnameOverride: ''
# map to NODE_ENV environment variable
env: 'production'
app:
# AFFINE_SERVER_SUB_PATH
path: ''
# AFFINE_SERVER_HOST
host: '0.0.0.0'
https: true
serviceAccount:
create: true
annotations: {}
name: 'affine-renderer'
podAnnotations: {}
podSecurityContext:
fsGroup: 2000
resources:
requests:
cpu: '4'
memory: 4Gi
probe:
initialDelaySeconds: 20
nodeSelector: {}
tolerations: []
affinity: {}

View File

@@ -1,11 +1,6 @@
apiVersion: v2
name: sync
description: AFFiNE Sync Server
description: A Helm chart for Kubernetes
type: application
version: 0.0.0
appVersion: "0.19.0"
dependencies:
- name: gcloud-sql-proxy
version: 0.0.0
repository: "file://../gcloud-sql-proxy"
condition: .global.database.gcloud.enabled
appVersion: "0.7.0-canary.18"

View File

@@ -32,19 +32,14 @@ spec:
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
env:
- name: AFFINE_PRIVATE_KEY
valueFrom:
secretKeyRef:
name: "{{ .Values.global.secret.secretName }}"
key: key
- name: NODE_ENV
value: "{{ .Values.env }}"
- name: NO_COLOR
value: "1"
- name: DEPLOYMENT_TYPE
value: "affine"
- name: SERVER_FLAVOR
value: "sync"
- name: NEXTAUTH_URL
value: "{{ .Values.global.ingress.host }}"
- name: AFFINE_ENV
value: "{{ .Release.Namespace }}"
- name: DATABASE_PASSWORD
@@ -87,6 +82,20 @@ spec:
initialDelaySeconds: {{ .Values.probe.initialDelaySeconds }}
resources:
{{- toYaml .Values.resources | nindent 12 }}
{{ if .Values.global.database.gcloud.enabled }}
- name: cloud-sql-proxy
image: gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.6.0
args:
- "--structured-logs"
- "--auto-iam-authn"
- "{{ .Values.global.database.gcloud.connectionName }}"
securityContext:
runAsNonRoot: true
resources:
requests:
memory: "2Gi"
cpu: "1"
{{ end }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}

View File

@@ -0,0 +1,13 @@
{{- if .Values.global.gke.enabled -}}
apiVersion: monitoring.googleapis.com/v1
kind: PodMonitoring
metadata:
name: "{{ .Chart.Name }}-monitoring"
spec:
selector:
matchLabels:
app.kubernetes.io/name: "{{ include "sync.name" . }}"
endpoints:
- port: {{ .Values.service.port }}
interval: 30s
{{- end }}

View File

@@ -12,6 +12,7 @@ env: 'production'
app:
# AFFINE_SERVER_HOST
host: '0.0.0.0'
serviceAccount:
create: true
annotations: {}

View File

@@ -27,9 +27,6 @@ spec:
- name: {{ .Chart.Name }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
env:
- name: AFFINE_ENV
value: "{{ .Release.Namespace }}"
ports:
- name: http
containerPort: {{ .Values.service.port }}

View File

@@ -1,9 +0,0 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Release.Name }}-runtime-config
data:
web-assets-manifest: |-
{{ .Files.Get "web-assets-manifest.json" | nindent 4 }}
mobile-assets-manifest: |-
{{ .Files.Get "mobile-assets-manifest.json" | nindent 4 }}

View File

@@ -60,13 +60,6 @@ spec:
name: affine-graphql
port:
number: {{ .Values.graphql.service.port }}
- path: /workspace
pathType: Prefix
backend:
service:
name: affine-renderer
port:
number: {{ .Values.renderer.service.port }}
- path: /
pathType: Prefix
backend:

View File

@@ -1,11 +0,0 @@
{{- if .Values.global.objectStorage.r2.enabled -}}
apiVersion: v1
kind: Secret
metadata:
name: "{{ .Values.global.objectStorage.r2.secretName }}"
type: Opaque
data:
accountId: {{ .Values.global.objectStorage.r2.accountId | b64enc }}
accessKeyId: {{ .Values.global.objectStorage.r2.accessKeyId | b64enc }}
secretAccessKey: {{ .Values.global.objectStorage.r2.secretAccessKey | b64enc }}
{{- end }}

View File

@@ -1,14 +1,9 @@
global:
app:
buildType: 'stable'
ingress:
enabled: false
className: ''
host: affine.pro
tls: []
secret:
secretName: 'server-private-key'
privateKey: ''
database:
user: 'postgres'
url: 'pg-postgresql'
@@ -21,8 +16,6 @@ global:
cloudSqlInternal: ''
connectionName: ''
serviceAccount: ''
cloudProxyReplicas: 3
proxyPort: '5432'
redis:
enabled: true
host: 'redis-master'
@@ -30,13 +23,6 @@ global:
username: ''
password: ''
database: 0
objectStorage:
r2:
enabled: false
secretName: r2
accountId: ''
accessKeyId: ''
secretAccessKey: ''
gke:
enabled: true
@@ -44,22 +30,13 @@ graphql:
service:
type: ClusterIP
port: 3000
annotations:
cloud.google.com/backend-config: '{"default": "affine-api-backendconfig"}'
sync:
service:
type: ClusterIP
port: 3010
annotations:
cloud.google.com/backend-config: '{"default": "affine-api-backendconfig"}'
renderer:
service:
type: ClusterIP
port: 3000
annotations:
cloud.google.com/backend-config: '{"default": "affine-api-backendconfig"}'
cloud.google.com/backend-config: '{"default": "affine-backendconfig"}'
web:
service:

60
.github/helm/deployment_guide.md vendored Normal file
View File

@@ -0,0 +1,60 @@
# Cluster Deployment Guide
This document provides a step-by-step guide for developers on how to deploy services in a Kubernetes cluster. The following content assumes that the reader already has a basic understanding of Kubernetes concepts and operations.
### 1. Configure Service Mesh (Optional)
In the Kubernetes cluster, we optionally use Service Mesh (like Istio and Anthos Service Mesh) to manage the network interactions of microservices. If Service Mesh is already deployed on your cluster or do not need to use the service network, you can skip this step. In this step, we assume that you are using Google Kubernetes Engine (GKE) and have already installed Anthos Service Mesh on your cluster, if you wish to use another Ingress Controller, please refer to the relevant documentation.
To configure your kubectl context to interact with your Kubernetes cluster using the gcloud tool, you need to execute the following commands:
```sh
export CLUSTER_NAME=your_cluster_name
export REGION=your_cluster_region
export PROJECT=your_project_id
gcloud container clusters get-credentials $CLUSTER_NAME --region $REGION --project $PROJECT
```
In this command, you should replace `CLUSTER_NAME`, `REGION` and `PROJECT` with the actual name, region and project id of your Kubernetes cluster. This command retrieves the access credentials for your Kubernetes cluster and automatically configures kubectl to use these credentials.
Now, to inject Service Mesh for a specific Namespace, first, set the environment variable `NAMESPACE` that should correspond to your target Kubernetes Namespace. In this example, we use `prod` as the target Namespace:
```sh
export NAMESPACE=prod
```
Then, we label the Namespace which will enable Istio to automatically inject the sidecar container for all new Pods under this Namespace:
```sh
kubectl label namespace $NAMESPACE istio-injection- istio.io/rev=asm-managed --overwrite
```
Finally, we trigger the Kubernetes Deployment restart mechanism to allow existing Pods to also obtain sidecar container injection:
```sh
kubectl rollout restart deployment -n $NAMESPACE
```
### 2. Deploying the Application
Next, we will deploy our application in the Kubernetes cluster through Helm. First, set relevant environment variables:
```sh
export NAMESPACE=prod
export RELEASE=affine-cloud-prod
export PATH=.github/helm/affine-cloud
```
- `NAMESPACE` should be consistent with the first step, indicating your target Kubernetes Namespace.
- `RELEASE` is the name of your Helm release.
- `PATH` is the location of your Helm chart in your file system.
Finally, use the `helm upgrade --install` command to deploy or upgrade your application:
```sh
helm upgrade --namespace $NAMESPACE --create-namespace --install $RELEASE $PATH
```
This command creates (if it doesn't already exist) and deploys your Helm chart in the specified Namespace. If the release already exists, it will be upgraded.
The above are the complete steps for deploying an application in a Kubernetes cluster. Make sure all prerequisites are met before deploying, and also ensure that you have the correct permissions for operations in Kubernetes.

View File

@@ -1,10 +0,0 @@
apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
name: "affine-api-backendconfig"
spec:
healthCheck:
timeoutSec: 1
type: HTTP
requestPath: /info

105
.github/labeler.yml vendored
View File

@@ -1,85 +1,62 @@
docs:
- changed-files:
- any-glob-to-any-file:
- 'docs/**/*'
- '**/README.md'
- 'packages/frontend/templates/**/*'
- 'docs/**/*'
- '**/README.md'
- 'packages/frontend/templates/**/*'
test:
- changed-files:
- any-glob-to-any-file:
- 'tests/**/*'
- '**/tests/**/*'
- '**/__tests__/**/*'
- 'tests/**/*'
- '**/tests/**/*'
- '**/__tests__/**/*'
mod:dev:
- changed-files:
- any-glob-to-any-file:
- 'scripts/**/*'
- 'tools/cli/**/*'
- 'packages/common/debug/**/*'
- 'scripts/**/*'
- 'tools/cli/**/*'
- 'packages/common/debug/**/*'
mod:plugin:
- 'packages/plugins/**/*'
plugin:copilot:
- 'packages/plugins/copilot/**/*'
mod:infra:
- changed-files:
- any-glob-to-any-file:
- 'packages/common/infra/**/*'
- 'packages/common/infra/**/*'
mod:sdk:
- 'packages/common/sdk/**/*'
mod:plugin-cli:
- changed-files:
- any-glob-to-any-file:
- 'tools/plugin-cli/**/*'
- 'tools/plugin-cli/**/*'
mod:i18n:
- changed-files:
- any-glob-to-any-file:
- 'packages/frontend/i18n/**/*'
mod:workspace: 'packages/frontend/workspace/**/*'
mod:env:
- changed-files:
- any-glob-to-any-file:
- 'packages/common/env/**/*'
mod:i18n: 'packages/frontend/i18n/**/*'
mod:component:
- changed-files:
- any-glob-to-any-file:
- 'packages/frontend/component/**/*'
mod:env: 'packages/common/env/**/*'
mod:server-native:
- changed-files:
- any-glob-to-any-file:
- 'packages/backend/native/**/*'
mod:hooks: 'packages/frontend/hooks/**/*'
mod:native:
- changed-files:
- any-glob-to-any-file:
- 'packages/frontend/native/**/*'
mod:component: 'packages/frontend/component/**/*'
mod:storage: 'packages/backend/storage/**/*'
mod:native: 'packages/frontend/native/**/*'
mod:store:
- changed-files:
- any-glob-to-any-file:
- '**/atoms/**/*'
- '**/atoms/**/*'
rust:
- changed-files:
- any-glob-to-any-file:
- '**/*.rs'
- '**/Cargo.toml'
- '**/Cargo.lock'
- '**/rust-toolchain'
- '**/rust-toolchain.toml'
- '**/rustfmt.toml'
- '**/*.rs'
- '**/Cargo.toml'
- '**/Cargo.lock'
- '**/rust-toolchain'
- '**/rust-toolchain.toml'
- '**/rustfmt.toml'
app:core:
- changed-files:
- any-glob-to-any-file:
- 'packages/frontend/core/**/*'
package:y-indexeddb: 'packages/common/y-indexeddb/**/*'
app:electron:
- changed-files:
- any-glob-to-any-file:
- 'packages/frontend/apps/electron/**/*'
app:core: 'packages/frontend/core/**/*'
app:server:
- changed-files:
- any-glob-to-any-file:
- 'packages/backend/server/**/*'
app:electron: 'packages/frontend/electron/**/*'
app:server: 'packages/backend/server/**/*'

66
.github/renovate.json vendored
View File

@@ -1,66 +0,0 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": ["config:recommended", ":disablePeerDependencies"],
"labels": ["dependencies"],
"ignorePaths": [
"**/node_modules/**",
"**/bower_components/**",
"**/vendor/**",
"**/examples/**",
"**/__tests__/**"
],
"packageRules": [
{
"rangeStrategy": "replace",
"groupName": "linter",
"matchPackageNames": ["/^eslint/", "/^@typescript-eslint/"]
},
{
"matchDepNames": ["oxlint"],
"rangeStrategy": "replace",
"groupName": "oxlint"
},
{
"groupName": "blocksuite",
"rangeStrategy": "replace",
"changelogUrl": "https://github.com/toeverything/blocksuite/blob/master/packages/blocks/CHANGELOG.md",
"matchPackageNames": ["/^@blocksuite/", "!@blocksuite/icons"]
},
{
"groupName": "all non-major dependencies",
"groupSlug": "all-minor-patch",
"matchUpdateTypes": ["minor", "patch"],
"matchPackageNames": ["*", "!/^@blocksuite//", "!/oxlint/"]
},
{
"groupName": "rust toolchain",
"matchManagers": ["custom.regex"],
"matchDepNames": ["rustc"]
},
{
"groupName": "nestjs",
"matchPackageNames": ["/^@nestjs/"]
}
],
"commitMessagePrefix": "chore: ",
"commitMessageAction": "bump up",
"commitMessageTopic": "{{depName}} version",
"ignoreDeps": [],
"postUpdateOptions": ["yarnDedupeHighest"],
"lockFileMaintenance": {
"enabled": true,
"extends": ["schedule:weekly"]
},
"customManagers": [
{
"customType": "regex",
"fileMatch": ["^rust-toolchain\\.toml?$"],
"matchStrings": [
"channel\\s*=\\s*\"(?<currentValue>\\d+\\.\\d+(\\.\\d+)?)\""
],
"depNameTemplate": "rustc",
"packageNameTemplate": "rust-lang/rust",
"datasourceTemplate": "github-releases"
}
]
}

View File

@@ -9,5 +9,4 @@ jobs:
pull-requests: write
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/labeler@v5
- uses: actions/labeler@v4

189
.github/workflows/build-desktop.yml vendored Normal file
View File

@@ -0,0 +1,189 @@
name: Build(Desktop) & Test
on:
push:
branches:
- master
- v[0-9]+.[0-9]+.x-staging
- v[0-9]+.[0-9]+.x
paths-ignore:
- README.md
- .github/**
- '!.github/workflows/build-desktop.yml'
- '!.github/actions/build-rust/action.yml'
- '!.github/actions/setup-node/action.yml'
pull_request:
merge_group:
branches:
- master
- v[0-9]+.[0-9]+.x-staging
- v[0-9]+.[0-9]+.x
paths-ignore:
- README.md
- .github/**
- '!.github/workflows/build-desktop.yml'
- '!.github/actions/build-rust/action.yml'
- '!.github/actions/setup-node/action.yml'
env:
DEBUG: napi:*
BUILD_TYPE: canary
APP_NAME: affine
COVERAGE: true
DISTRIBUTION: desktop
MACOSX_DEPLOYMENT_TARGET: '10.13'
NX_CLOUD_ACCESS_TOKEN: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}
jobs:
build-core:
name: Build @affine/core
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
electron-install: false
- name: Build Core
run: yarn nx build @affine/core
- name: Upload core artifact
uses: actions/upload-artifact@v3
with:
name: core
path: ./packages/frontend/core/dist
if-no-files-found: error
build-native:
name: Build Native
runs-on: ubuntu-latest
needs: build-core
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: ./.github/actions/setup-node
- name: Build AFFiNE native
uses: ./.github/actions/build-rust
with:
target: x86_64-unknown-linux-gnu
package: '@affine/native'
nx_token: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}
- name: Run tests
run: yarn test
working-directory: ./packages/frontend/native
desktop-test:
name: Desktop Test
runs-on: ${{ matrix.spec.os }}
strategy:
fail-fast: false
# all combinations: macos-latest x64, macos-latest arm64, windows-latest x64, ubuntu-latest x64
matrix:
spec:
- {
os: macos-latest,
platform: macos,
arch: x64,
target: x86_64-apple-darwin,
test: true,
}
- {
os: macos-latest,
platform: macos,
arch: arm64,
target: aarch64-apple-darwin,
test: false,
}
- {
os: ubuntu-latest,
platform: linux,
arch: x64,
target: x86_64-unknown-linux-gnu,
test: true,
}
- {
os: windows-latest,
platform: windows,
arch: x64,
target: x86_64-pc-windows-msvc,
test: true,
}
needs: build-core
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: ./.github/actions/setup-node
timeout-minutes: 10
with:
extra-flags: workspaces focus @affine/electron @affine/monorepo @affine-test/affine-desktop
playwright-install: true
hard-link-nm: false
enableScripts: false
- name: Build AFFiNE native
uses: ./.github/actions/build-rust
with:
target: ${{ matrix.spec.target }}
package: '@affine/native'
nx_token: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}
- name: Run unit tests
if: ${{ matrix.spec.test }}
shell: bash
run: yarn vitest
working-directory: packages/frontend/electron
- name: Download core artifact
uses: actions/download-artifact@v3
with:
name: core
path: packages/frontend/electron/resources/web-static
- name: Build Desktop Layers
run: yarn workspace @affine/electron build
- name: Run desktop tests
if: ${{ matrix.spec.test && matrix.spec.os == 'ubuntu-latest' }}
run: xvfb-run --auto-servernum --server-args="-screen 0 1280x960x24" -- yarn workspace @affine-test/affine-desktop e2e
env:
COVERAGE: true
- name: Run desktop tests
if: ${{ matrix.spec.test && matrix.spec.os != 'ubuntu-latest' }}
run: yarn workspace @affine-test/affine-desktop e2e
env:
COVERAGE: true
- name: Make bundle
if: ${{ matrix.spec.os == 'macos-latest' && matrix.spec.arch == 'arm64' }}
env:
SKIP_BUNDLE: true
SKIP_WEB_BUILD: true
run: yarn workspace @affine/electron make --platform=darwin --arch=arm64
- name: Output check
if: ${{ matrix.spec.os == 'macos-latest' && matrix.spec.arch == 'arm64' }}
run: |
yarn workspace @affine/electron ts-node ./scripts/macos-arm64-output-check.ts
- name: Collect code coverage report
if: ${{ matrix.spec.test }}
run: yarn exec nyc report -t .nyc_output --report-dir .coverage --reporter=lcov
- name: Upload e2e test coverage results
if: ${{ matrix.spec.test }}
uses: codecov/codecov-action@v3
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./.coverage/lcov.info
flags: e2etest-${{ matrix.spec.os }}-${{ matrix.spec.arch }}
name: affine
fail_ci_if_error: false
- name: Upload test results
if: ${{ failure() }}
uses: actions/upload-artifact@v3
with:
name: test-results-e2e-${{ matrix.spec.os }}-${{ matrix.spec.arch }}
path: ./test-results
if-no-files-found: ignore

View File

@@ -1,287 +0,0 @@
name: Build Images
on:
workflow_call:
inputs:
flavor:
type: string
required: true
permissions:
contents: 'write'
id-token: 'write'
packages: 'write'
jobs:
build-server:
name: Build Server
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Version
id: version
uses: ./.github/actions/setup-version
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
electron-install: false
extra-flags: workspaces focus @affine/server
- name: Build Server
run: yarn workspace @affine/server build
- name: Upload server dist
uses: actions/upload-artifact@v4
with:
name: server-dist
path: ./packages/backend/server/dist
if-no-files-found: error
build-web:
name: Build @affine/web
runs-on: ubuntu-latest
environment: ${{ github.event.inputs.flavor }}
steps:
- uses: actions/checkout@v4
- name: Setup Version
id: version
uses: ./.github/actions/setup-version
- name: Setup Node.js
uses: ./.github/actions/setup-node
- name: Build Core
run: yarn affine @affine/web build
env:
R2_ACCOUNT_ID: ${{ secrets.R2_ACCOUNT_ID }}
R2_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
BUILD_TYPE: ${{ github.event.inputs.flavor }}
CAPTCHA_SITE_KEY: ${{ secrets.CAPTCHA_SITE_KEY }}
SENTRY_ORG: ${{ secrets.SENTRY_ORG }}
SENTRY_PROJECT: 'affine-web'
SENTRY_RELEASE: ${{ steps.version.outputs.APP_VERSION }}
SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
SENTRY_DSN: ${{ secrets.SENTRY_DSN }}
PERFSEE_TOKEN: ${{ secrets.PERFSEE_TOKEN }}
MIXPANEL_TOKEN: ${{ secrets.MIXPANEL_TOKEN }}
- name: Upload web artifact
uses: actions/upload-artifact@v4
with:
name: web
path: ./packages/frontend/apps/web/dist
if-no-files-found: error
build-admin:
name: Build @affine/admin
runs-on: ubuntu-latest
environment: ${{ github.event.inputs.flavor }}
steps:
- uses: actions/checkout@v4
- name: Setup Version
id: version
uses: ./.github/actions/setup-version
- name: Setup Node.js
uses: ./.github/actions/setup-node
- name: Build Admin
run: yarn affine @affine/admin build
env:
R2_ACCOUNT_ID: ${{ secrets.R2_ACCOUNT_ID }}
R2_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
BUILD_TYPE: ${{ github.event.inputs.flavor }}
CAPTCHA_SITE_KEY: ${{ secrets.CAPTCHA_SITE_KEY }}
SENTRY_ORG: ${{ secrets.SENTRY_ORG }}
SENTRY_PROJECT: 'affine-admin'
SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
SENTRY_DSN: ${{ secrets.SENTRY_DSN }}
PERFSEE_TOKEN: ${{ secrets.PERFSEE_TOKEN }}
MIXPANEL_TOKEN: ${{ secrets.MIXPANEL_TOKEN }}
- name: Upload admin artifact
uses: actions/upload-artifact@v4
with:
name: admin
path: ./packages/frontend/admin/dist
if-no-files-found: error
build-mobile:
name: Build @affine/mobile
runs-on: ubuntu-latest
environment: ${{ github.event.inputs.flavor }}
steps:
- uses: actions/checkout@v4
- name: Setup Version
id: version
uses: ./.github/actions/setup-version
- name: Setup Node.js
uses: ./.github/actions/setup-node
- name: Build Mobile
run: yarn affine @affine/mobile build
env:
R2_ACCOUNT_ID: ${{ secrets.R2_ACCOUNT_ID }}
R2_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
BUILD_TYPE: ${{ github.event.inputs.flavor }}
CAPTCHA_SITE_KEY: ${{ secrets.CAPTCHA_SITE_KEY }}
SENTRY_ORG: ${{ secrets.SENTRY_ORG }}
SENTRY_PROJECT: 'affine-mobile'
SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
SENTRY_DSN: ${{ secrets.SENTRY_DSN }}
PERFSEE_TOKEN: ${{ secrets.PERFSEE_TOKEN }}
MIXPANEL_TOKEN: ${{ secrets.MIXPANEL_TOKEN }}
- name: Upload mobile artifact
uses: actions/upload-artifact@v4
with:
name: mobile
path: ./packages/frontend/apps/mobile/dist
if-no-files-found: error
build-server-native:
name: Build Server native - ${{ matrix.targets.name }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
targets:
- name: x86_64-unknown-linux-gnu
file: server-native.node
- name: aarch64-unknown-linux-gnu
file: server-native.arm64.node
- name: armv7-unknown-linux-gnueabihf
file: server-native.armv7.node
steps:
- uses: actions/checkout@v4
- name: Setup Version
id: version
uses: ./.github/actions/setup-version
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
electron-install: false
extra-flags: workspaces focus @affine/server-native
- name: Build Rust
uses: ./.github/actions/build-rust
with:
target: ${{ matrix.targets.name }}
package: '@affine/server-native'
- name: Upload ${{ matrix.targets.file }}
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.targets.file }}
path: ./packages/backend/native/server-native.node
if-no-files-found: error
build-images:
name: Build Images
runs-on: ubuntu-latest
needs:
- build-server
- build-web
- build-mobile
- build-admin
- build-server-native
steps:
- uses: actions/checkout@v4
- name: Download server dist
uses: actions/download-artifact@v4
with:
name: server-dist
path: ./packages/backend/server/dist
- name: Download server-native.node
uses: actions/download-artifact@v4
with:
name: server-native.node
path: ./packages/backend/server
- name: Download server-native.node arm64
uses: actions/download-artifact@v4
with:
name: server-native.arm64.node
path: ./packages/backend/native
- name: Download server-native.node arm64
uses: actions/download-artifact@v4
with:
name: server-native.armv7.node
path: .
- name: move server-native files
run: |
mv ./packages/backend/native/server-native.node ./packages/backend/server/server-native.arm64.node
mv server-native.node ./packages/backend/server/server-native.armv7.node
- name: Setup env
run: |
echo "GIT_SHORT_HASH=$(git rev-parse --short HEAD)" >> "$GITHUB_ENV"
if [ -z "${{ inputs.flavor }}" ]
then
echo "RELEASE_FLAVOR=canary" >> "$GITHUB_ENV"
else
echo "RELEASE_FLAVOR=${{ inputs.flavor }}" >> "$GITHUB_ENV"
fi
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
logout: false
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
# setup node without cache configuration
# Prisma cache is not compatible with docker build cache
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version-file: '.nvmrc'
registry-url: https://npm.pkg.github.com
scope: '@toeverything'
- name: Download web artifact
uses: actions/download-artifact@v4
with:
name: web
path: ./packages/frontend/apps/web/dist
- name: Download mobile artifact
uses: actions/download-artifact@v4
with:
name: mobile
path: ./packages/frontend/apps/mobile/dist
- name: Download admin artifact
uses: actions/download-artifact@v4
with:
name: admin
path: ./packages/frontend/admin/dist
- name: Install Node.js dependencies
run: |
yarn config set --json supportedArchitectures.cpu '["x64", "arm64", "arm"]'
yarn config set --json supportedArchitectures.libc '["glibc"]'
yarn workspaces focus @affine/server --production
- name: Generate Prisma client
run: yarn workspace @affine/server prisma generate
- name: Setup Version
id: version
uses: ./.github/actions/setup-version
- name: Build front Dockerfile
uses: docker/build-push-action@v6
with:
context: .
push: true
pull: true
platforms: linux/amd64,linux/arm64
provenance: true
file: .github/deployment/front/Dockerfile
tags: ghcr.io/toeverything/affine-front:${{env.RELEASE_FLAVOR}}-${{ env.GIT_SHORT_HASH }},ghcr.io/toeverything/affine-front:${{env.RELEASE_FLAVOR}}
- name: Build graphql Dockerfile
uses: docker/build-push-action@v6
with:
context: .
push: true
pull: true
platforms: linux/amd64,linux/arm64,linux/arm/v7
provenance: true
file: .github/deployment/node/Dockerfile
tags: ghcr.io/toeverything/affine-graphql:${{env.RELEASE_FLAVOR}}-${{ env.GIT_SHORT_HASH }},ghcr.io/toeverything/affine-graphql:${{env.RELEASE_FLAVOR}}

View File

@@ -1,25 +0,0 @@
name: Build Selfhost Image
on:
workflow_dispatch:
inputs:
flavor:
description: 'Select distribution to build'
type: choice
default: canary
options:
- canary
- beta
- stable
permissions:
contents: 'write'
id-token: 'write'
packages: 'write'
jobs:
build-image:
name: Build Image
uses: ./.github/workflows/build-images.yml
with:
flavor: ${{ github.event.inputs.flavor }}

311
.github/workflows/build-server.yml vendored Normal file
View File

@@ -0,0 +1,311 @@
name: Build(Server) & Test
on:
push:
branches:
- master
- v[0-9]+.[0-9]+.x-staging
- v[0-9]+.[0-9]+.x
paths-ignore:
- README.md
- .github/**
- '!.github/workflows/build-server.yml'
- '!.github/actions/build-rust/action.yml'
- '!.github/actions/setup-node/action.yml'
pull_request:
merge_group:
branches:
- master
- v[0-9]+.[0-9]+.x-staging
- v[0-9]+.[0-9]+.x
paths-ignore:
- README.md
- .github/**
- '!.github/workflows/build-server.yml'
- '!.github/actions/build-rust/action.yml'
- '!.github/actions/setup-node/action.yml'
env:
DEBUG: napi:*
BUILD_TYPE: canary
APP_NAME: affine
COVERAGE: true
DISTRIBUTION: browser
NX_CLOUD_ACCESS_TOKEN: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}
jobs:
build-storage:
name: Build Storage
runs-on: ubuntu-latest
env:
RUSTFLAGS: '-C debuginfo=1'
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
extra-flags: workspaces focus @affine/storage
electron-install: false
build-infra: false
build-plugins: false
- name: Build Rust
uses: ./.github/actions/build-rust
with:
target: 'x86_64-unknown-linux-gnu'
package: '@affine/storage'
nx_token: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}
- name: Upload storage.node
uses: actions/upload-artifact@v3
with:
name: storage.node
path: ./packages/backend/storage/storage.node
if-no-files-found: error
server-test:
name: Server Test
runs-on: ubuntu-latest
needs: build-storage
services:
postgres:
image: postgres
env:
POSTGRES_PASSWORD: affine
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
mailer:
image: mailhog/mailhog
ports:
- 1025:1025
- 8025:8025
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
electron-install: false
- name: Initialize database
run: |
psql -h localhost -U postgres -c "CREATE DATABASE affine;"
psql -h localhost -U postgres -c "CREATE USER affine WITH PASSWORD 'affine';"
psql -h localhost -U postgres -c "ALTER USER affine WITH SUPERUSER;"
env:
PGPASSWORD: affine
- name: Generate prisma client
run: |
yarn workspace @affine/server exec prisma generate
yarn workspace @affine/server exec prisma db push
env:
DATABASE_URL: postgresql://affine:affine@localhost:5432/affine
- name: Run init-db script
run: yarn workspace @affine/server exec ts-node ./scripts/init-db.ts
env:
DATABASE_URL: postgresql://affine:affine@localhost:5432/affine
- name: Download storage.node
uses: actions/download-artifact@v3
with:
name: storage.node
path: ./packages/backend/server
- name: Run server tests
run: yarn workspace @affine/server test:coverage
env:
CARGO_TARGET_DIR: '${{ github.workspace }}/target'
DATABASE_URL: postgresql://affine:affine@localhost:5432/affine
- name: Upload server test coverage results
uses: codecov/codecov-action@v3
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./packages/backend/server/.coverage/lcov.info
flags: server-test
name: affine
fail_ci_if_error: false
server-e2e-test:
name: Server E2E Test
runs-on: ubuntu-latest
needs: build-storage
services:
postgres:
image: postgres
env:
POSTGRES_PASSWORD: affine
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
mailer:
image: mailhog/mailhog
ports:
- 1025:1025
- 8025:8025
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
playwright-install: true
- name: Initialize database
run: |
psql -h localhost -U postgres -c "CREATE DATABASE affine;"
psql -h localhost -U postgres -c "CREATE USER affine WITH PASSWORD 'affine';"
psql -h localhost -U postgres -c "ALTER USER affine WITH SUPERUSER;"
env:
PGPASSWORD: affine
- name: Generate prisma client
run: |
yarn workspace @affine/server exec prisma generate
yarn workspace @affine/server exec prisma db push
env:
DATABASE_URL: postgresql://affine:affine@localhost:5432/affine
- name: Run init-db script
run: yarn workspace @affine/server exec ts-node ./scripts/init-db.ts
env:
DATABASE_URL: postgresql://affine:affine@localhost:5432/affine
- name: Download storage.node
uses: actions/download-artifact@v3
with:
name: storage.node
path: ./packages/backend/server
- name: Run playwright tests
run: xvfb-run --auto-servernum --server-args="-screen 0 1280x960x24" -- yarn workspace @affine-test/affine-cloud e2e --forbid-only
env:
COVERAGE: true
DATABASE_URL: postgresql://affine:affine@localhost:5432/affine
- name: Collect code coverage report
run: yarn exec nyc report -t .nyc_output --report-dir .coverage --reporter=lcov
- name: Upload e2e test coverage results
uses: codecov/codecov-action@v3
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./.coverage/lcov.info
flags: server-e2etest
name: affine
fail_ci_if_error: false
- name: Upload test results
if: ${{ failure() }}
uses: actions/upload-artifact@v3
with:
name: test-results-e2e-server
path: ./tests/affine-cloud/test-results
if-no-files-found: ignore
server-desktop-e2e-test:
name: Server Desktop E2E Test
runs-on: ubuntu-latest
needs: build-storage
services:
postgres:
image: postgres
env:
POSTGRES_PASSWORD: affine
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
mailer:
image: mailhog/mailhog
ports:
- 1025:1025
- 8025:8025
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
playwright-install: true
hard-link-nm: false
- name: Build AFFiNE native
uses: ./.github/actions/build-rust
with:
target: x86_64-unknown-linux-gnu
package: '@affine/native'
nx_token: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}
- name: Initialize database
run: |
psql -h localhost -U postgres -c "CREATE DATABASE affine;"
psql -h localhost -U postgres -c "CREATE USER affine WITH PASSWORD 'affine';"
psql -h localhost -U postgres -c "ALTER USER affine WITH SUPERUSER;"
env:
PGPASSWORD: affine
- name: Generate prisma client
run: |
yarn workspace @affine/server exec prisma generate
yarn workspace @affine/server prisma db push
env:
DATABASE_URL: postgresql://affine:affine@localhost:5432/affine
- name: Run init-db script
run: yarn workspace @affine/server exec ts-node ./scripts/init-db.ts
env:
DATABASE_URL: postgresql://affine:affine@localhost:5432/affine
- name: Download storage.node
uses: actions/download-artifact@v3
with:
name: storage.node
path: ./packages/backend/server
- name: Build Plugins
run: yarn run build:plugins
- name: Build Desktop Layers
run: yarn workspace @affine/electron build:dev
- name: Run playwright tests
run: xvfb-run --auto-servernum --server-args="-screen 0 1280x960x24" yarn workspace @affine-test/affine-desktop-cloud e2e
env:
COVERAGE: true
DEV_SERVER_URL: http://localhost:8080
DATABASE_URL: postgresql://affine:affine@localhost:5432/affine
ENABLE_LOCAL_EMAIL: true
- name: Collect code coverage report
run: yarn exec nyc report -t .nyc_output --report-dir .coverage --reporter=lcov
- name: Upload e2e test coverage results
uses: codecov/codecov-action@v3
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./.coverage/lcov.info
flags: server-e2etest
name: affine
fail_ci_if_error: false
- name: Upload test results
if: ${{ failure() }}
uses: actions/upload-artifact@v3
with:
name: test-results-e2e-server
path: ./tests/affine-cloud/test-results
if-no-files-found: ignore

View File

@@ -1,876 +0,0 @@
name: Build & Test
on:
push:
branches:
- canary
- beta
- stable
- v[0-9]+.[0-9]+.x-staging
- v[0-9]+.[0-9]+.x
paths-ignore:
- README.md
pull_request:
env:
DEBUG: napi:*
BUILD_TYPE: canary
APP_NAME: affine
AFFINE_ENV: dev
COVERAGE: true
MACOSX_DEPLOYMENT_TARGET: '10.13'
DEPLOYMENT_TYPE: affine
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
optimize_ci:
name: Optimize CI
runs-on: ubuntu-latest
outputs:
skip: ${{ steps.check_skip.outputs.skip }}
steps:
- uses: actions/checkout@v4
- name: Graphite CI Optimizer
uses: withgraphite/graphite-ci-action@main
id: check_skip
with:
graphite_token: ${{ secrets.GRAPHITE_CI_OPTIMIZER_TOKEN }}
analyze:
name: Analyze
runs-on: ubuntu-latest
needs: optimize_ci
if: needs.optimize_ci.outputs.skip == 'false'
permissions:
actions: read
contents: read
security-events: write
strategy:
fail-fast: false
matrix:
language: ['javascript', 'typescript']
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support
steps:
- name: Checkout repository
uses: actions/checkout@v4
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# Details on CodeQL's query packs refer to : https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
# queries: security-extended,security-and-quality
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v3
# Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
# If the Autobuild fails above, remove it and uncomment the following three lines.
# modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance.
# - run: |
# echo "Run, Build Application using script"
# ./location_of_script_within_repo/buildscript.sh
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
lint:
name: Lint
runs-on: ubuntu-latest
needs: optimize_ci
if: needs.optimize_ci.outputs.skip == 'false'
steps:
- uses: actions/checkout@v4
- name: Run oxlint
# oxlint is fast, so wrong code will fail quickly
run: yarn dlx $(node -e "console.log(require('./package.json').scripts['lint:ox'].replace('oxlint', 'oxlint@' + require('./package.json').devDependencies.oxlint))")
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
electron-install: false
full-cache: true
- name: Run i18n codegen
run: yarn affine @affine/i18n build
- name: Run ESLint
run: yarn lint:eslint --max-warnings=0
- name: Run Prettier
# Set nmMode in `actions/setup-node` will modify the .yarnrc.yml
run: |
git checkout .yarnrc.yml
yarn lint:prettier
- name: Yarn Dedupe
run: yarn dedupe --check
- name: Run Type Check
run: yarn typecheck
check-yarn-binary:
name: Check yarn binary
runs-on: ubuntu-latest
needs: optimize_ci
if: needs.optimize_ci.outputs.skip == 'false'
steps:
- uses: actions/checkout@v4
- name: Run check
run: |
yarn set version $(node -e "console.log(require('./package.json').packageManager.split('@')[1])")
git diff --exit-code
e2e-legacy-blocksuite-test:
name: Legacy Blocksuite E2E Test
runs-on: ubuntu-latest
needs: optimize_ci
if: needs.optimize_ci.outputs.skip == 'false'
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
playwright-install: true
electron-install: false
full-cache: true
- name: Run playground build
run: yarn workspace @blocksuite/playground build
- name: Run playwright tests
run: yarn workspace @blocksuite/legacy-e2e test --forbid-only --shard=${{ matrix.shard }}/${{ strategy.job-total }}
- name: Upload test results
if: ${{ failure() }}
uses: actions/upload-artifact@v4
with:
name: test-results-e2e-legacy-bs-${{ matrix.shard }}
path: ./test-results
if-no-files-found: ignore
e2e-test:
name: E2E Test
runs-on: ubuntu-latest
needs: optimize_ci
if: needs.optimize_ci.outputs.skip == 'false'
env:
DISTRIBUTION: web
IN_CI_TEST: true
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3, 4, 5]
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
playwright-install: true
electron-install: false
full-cache: true
- name: Run playwright tests
run: yarn affine @affine-test/affine-local e2e --forbid-only --shard=${{ matrix.shard }}/${{ strategy.job-total }}
- name: Upload test results
if: ${{ failure() }}
uses: actions/upload-artifact@v4
with:
name: test-results-e2e-${{ matrix.shard }}
path: ./test-results
if-no-files-found: ignore
e2e-mobile-test:
name: E2E Mobile Test
runs-on: ubuntu-latest
needs: optimize_ci
if: needs.optimize_ci.outputs.skip == 'false'
env:
DISTRIBUTION: mobile
IN_CI_TEST: true
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3, 4, 5]
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
playwright-install: true
electron-install: false
full-cache: true
- name: Run playwright tests
run: yarn affine @affine-test/affine-mobile e2e --forbid-only --shard=${{ matrix.shard }}/${{ strategy.job-total }}
- name: Upload test results
if: ${{ failure() }}
uses: actions/upload-artifact@v4
with:
name: test-results-e2e-mobile-${{ matrix.shard }}
path: ./test-results
if-no-files-found: ignore
unit-test:
name: Unit Test
runs-on: ubuntu-latest
needs:
- optimize_ci
- build-native
if: needs.optimize_ci.outputs.skip == 'false'
env:
DISTRIBUTION: web
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3, 4, 5]
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
electron-install: true
playwright-install: true
full-cache: true
- name: Download affine.linux-x64-gnu.node
uses: actions/download-artifact@v4
with:
name: affine.linux-x64-gnu.node
path: ./packages/frontend/native
- name: Unit Test
run: yarn test:coverage --shard=${{ matrix.shard }}/${{ strategy.job-total }}
- name: Upload unit test coverage results
uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./.coverage/store/lcov.info
flags: unittest
name: affine
fail_ci_if_error: false
build-native:
name: Build AFFiNE native (${{ matrix.spec.target }})
runs-on: ${{ matrix.spec.os }}
needs: optimize_ci
if: needs.optimize_ci.outputs.skip == 'false'
env:
CARGO_PROFILE_RELEASE_DEBUG: '1'
strategy:
fail-fast: false
matrix:
spec:
- { os: ubuntu-latest, target: x86_64-unknown-linux-gnu }
- { os: windows-latest, target: x86_64-pc-windows-msvc }
- { os: macos-latest, target: x86_64-apple-darwin }
- { os: macos-latest, target: aarch64-apple-darwin }
steps:
- uses: actions/checkout@v4
- uses: samypr100/setup-dev-drive@v3
if: ${{ matrix.spec.os == 'windows-latest' }}
with:
workspace-copy: true
drive-size: 8GB
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
extra-flags: workspaces focus @affine/native
electron-install: false
- name: Setup filename
id: filename
working-directory: ${{ env.DEV_DRIVE_WORKSPACE || github.workspace }}
shell: bash
run: |
export PLATFORM_ARCH_ABI=$(node -e "console.log(require('@napi-rs/cli').parseTriple('${{ matrix.spec.target }}').platformArchABI)")
echo "filename=affine.$PLATFORM_ARCH_ABI.node" >> "$GITHUB_OUTPUT"
- name: Build AFFiNE native
uses: ./.github/actions/build-rust
with:
target: ${{ matrix.spec.target }}
package: '@affine/native'
- name: Upload ${{ steps.filename.outputs.filename }}
uses: actions/upload-artifact@v4
with:
name: ${{ steps.filename.outputs.filename }}
path: ${{ env.DEV_DRIVE_WORKSPACE || github.workspace }}/packages/frontend/native/${{ steps.filename.outputs.filename }}
if-no-files-found: error
build-server-native:
name: Build Server native
runs-on: ubuntu-latest
needs: optimize_ci
if: needs.optimize_ci.outputs.skip == 'false'
env:
CARGO_PROFILE_RELEASE_DEBUG: '1'
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
extra-flags: workspaces focus @affine/server-native
electron-install: false
- name: Build Rust
uses: ./.github/actions/build-rust
with:
target: 'x86_64-unknown-linux-gnu'
package: '@affine/server-native'
- name: Upload server-native.node
uses: actions/upload-artifact@v4
with:
name: server-native.node
path: ./packages/backend/native/server-native.node
if-no-files-found: error
build-electron-renderer:
name: Build @affine/electron renderer
runs-on: ubuntu-latest
needs: optimize_ci
if: needs.optimize_ci.outputs.skip == 'false'
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
electron-install: false
full-cache: true
- name: Build Electron renderer
run: yarn affine @affine/electron-renderer build
env:
DISTRIBUTION: desktop
- name: zip web
run: tar -czf dist.tar.gz --directory=packages/frontend/apps/electron-renderer/dist .
- name: Upload web artifact
uses: actions/upload-artifact@v4
with:
name: web
path: dist.tar.gz
if-no-files-found: error
server-test:
name: Server Test
runs-on: ubuntu-latest
needs:
- optimize_ci
- build-server-native
if: needs.optimize_ci.outputs.skip == 'false'
env:
NODE_ENV: test
DISTRIBUTION: web
DATABASE_URL: postgresql://affine:affine@localhost:5432/affine
REDIS_SERVER_HOST: localhost
services:
postgres:
image: postgres
env:
POSTGRES_PASSWORD: affine
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
redis:
image: redis
ports:
- 6379:6379
mailer:
image: mailhog/mailhog
ports:
- 1025:1025
- 8025:8025
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
electron-install: false
full-cache: true
- name: Download server-native.node
uses: actions/download-artifact@v4
with:
name: server-native.node
path: ./packages/backend/server
- name: Prepare Server Test Environment
uses: ./.github/actions/server-test-env
- name: Run server tests
run: yarn affine @affine/server test:coverage --forbid-only
env:
CARGO_TARGET_DIR: '${{ github.workspace }}/target'
COPILOT_OPENAI_API_KEY: 'use_fake_openai_api_key'
- name: Upload server test coverage results
uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./packages/backend/server/.coverage/lcov.info
flags: server-test
name: affine
fail_ci_if_error: false
rust-test:
name: Run native tests
runs-on: ubuntu-latest
needs: optimize_ci
if: needs.optimize_ci.outputs.skip == 'false'
env:
CARGO_TERM_COLOR: always
steps:
- uses: actions/checkout@v4
- name: Setup Rust
uses: ./.github/actions/build-rust
with:
package: 'affine'
no-build: 'true'
- name: Install latest nextest release
uses: taiki-e/install-action@nextest
- name: Run tests
run: cargo nextest run --release --no-fail-fast
copilot-api-test:
name: Server Copilot Api Test
runs-on: ubuntu-latest
needs:
- optimize_ci
- build-server-native
if: needs.optimize_ci.outputs.skip == 'false'
env:
NODE_ENV: test
DISTRIBUTION: web
DATABASE_URL: postgresql://affine:affine@localhost:5432/affine
REDIS_SERVER_HOST: localhost
services:
postgres:
image: postgres
env:
POSTGRES_PASSWORD: affine
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
redis:
image: redis
ports:
- 6379:6379
mailer:
image: mailhog/mailhog
ports:
- 1025:1025
- 8025:8025
steps:
- uses: actions/checkout@v4
- name: Check blocksuite update
id: check-blocksuite-update
env:
BASE_REF: ${{ github.base_ref }}
run: |
if node ./scripts/detect-blocksuite-update.mjs "$BASE_REF"; then
echo "skip=false" >> $GITHUB_OUTPUT
else
echo "skip=true" >> $GITHUB_OUTPUT
fi
- uses: dorny/paths-filter@v3
id: apifilter
with:
filters: |
changed:
- 'packages/backend/server/src/plugins/copilot/**'
- 'packages/backend/server/tests/copilot.*'
- name: Setup Node.js
if: ${{ steps.check-blocksuite-update.outputs.skip != 'true' || steps.apifilter.outputs.changed == 'true' }}
uses: ./.github/actions/setup-node
with:
electron-install: false
full-cache: true
- name: Download server-native.node
if: ${{ steps.check-blocksuite-update.outputs.skip != 'true' || steps.apifilter.outputs.changed == 'true' }}
uses: actions/download-artifact@v4
with:
name: server-native.node
path: ./packages/backend/server
- name: Prepare Server Test Environment
if: ${{ steps.check-blocksuite-update.outputs.skip != 'true' || steps.apifilter.outputs.changed == 'true' }}
uses: ./.github/actions/server-test-env
- name: Run server tests
if: ${{ steps.check-blocksuite-update.outputs.skip != 'true' || steps.apifilter.outputs.changed == 'true' }}
run: yarn affine @affine/server test:copilot:coverage --forbid-only
env:
CARGO_TARGET_DIR: '${{ github.workspace }}/target'
COPILOT_OPENAI_API_KEY: ${{ secrets.COPILOT_OPENAI_API_KEY }}
COPILOT_FAL_API_KEY: ${{ secrets.COPILOT_FAL_API_KEY }}
- name: Upload server test coverage results
if: ${{ steps.check-blocksuite-update.outputs.skip != 'true' || steps.apifilter.outputs.changed == 'true' }}
uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./packages/backend/server/.coverage/lcov.info
flags: server-test
name: affine
fail_ci_if_error: false
copilot-e2e-test:
name: Server Copilot E2E Test
runs-on: ubuntu-latest
env:
DISTRIBUTION: web
DATABASE_URL: postgresql://affine:affine@localhost:5432/affine
IN_CI_TEST: true
REDIS_SERVER_HOST: localhost
DEPLOYMENT_TYPE: affine
strategy:
fail-fast: false
matrix:
shardIndex: [1, 2, 3]
shardTotal: [3]
needs:
- build-server-native
services:
postgres:
image: postgres
env:
POSTGRES_PASSWORD: affine
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
redis:
image: redis
ports:
- 6379:6379
steps:
- uses: actions/checkout@v4
- name: Check blocksuite update
id: check-blocksuite-update
env:
BASE_REF: ${{ github.base_ref }}
run: |
if node ./scripts/detect-blocksuite-update.mjs "$BASE_REF"; then
echo "skip=false" >> $GITHUB_OUTPUT
else
echo "skip=true" >> $GITHUB_OUTPUT
fi
- uses: dorny/paths-filter@v3
id: e2efilter
with:
filters: |
changed:
- 'packages/frontend/core/src/blocksuite/presets/ai/**'
- 'packages/frontend/core/src/components/blocksuite/block-suite-editor/ai/**'
- 'tests/affine-cloud-copilot/**'
- name: Setup Node.js
if: ${{ steps.check-blocksuite-update.outputs.skip != 'true' || steps.e2efilter.outputs.changed == 'true' }}
uses: ./.github/actions/setup-node
with:
playwright-install: true
electron-install: false
hard-link-nm: false
- name: Download server-native.node
if: ${{ steps.check-blocksuite-update.outputs.skip != 'true' || steps.e2efilter.outputs.changed == 'true' }}
uses: actions/download-artifact@v4
with:
name: server-native.node
path: ./packages/backend/server
- name: Run Copilot E2E Test ${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
if: ${{ steps.check-blocksuite-update.outputs.skip != 'true' || steps.e2efilter.outputs.changed == 'true' }}
uses: ./.github/actions/copilot-test
with:
script: yarn affine @affine-test/affine-cloud-copilot e2e --forbid-only --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
openai-key: ${{ secrets.COPILOT_OPENAI_API_KEY }}
fal-key: ${{ secrets.COPILOT_FAL_API_KEY }}
server-e2e-test:
name: ${{ matrix.tests.name }}
runs-on: ubuntu-latest
needs:
- optimize_ci
- build-server-native
- build-native
if: needs.optimize_ci.outputs.skip == 'false'
env:
DISTRIBUTION: web
DATABASE_URL: postgresql://affine:affine@localhost:5432/affine
REDIS_SERVER_HOST: localhost
IN_CI_TEST: true
strategy:
fail-fast: false
matrix:
tests:
- name: 'Server E2E Test 1/3'
script: yarn affine @affine-test/affine-cloud e2e --forbid-only --shard=1/3
- name: 'Server E2E Test 2/3'
script: yarn affine @affine-test/affine-cloud e2e --forbid-only --shard=2/3
- name: 'Server E2E Test 3/3'
script: yarn affine @affine-test/affine-cloud e2e --forbid-only --shard=3/3
- name: 'Server Desktop E2E Test'
script: |
yarn affine @affine/electron build:dev
# Workaround for Electron apps failing to initialize on Ubuntu 24.04 due to AppArmor restrictions
# Disables unprivileged user namespaces restriction to allow Electron apps to run
# Reference: https://github.com/electron/electron/issues/42510
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
xvfb-run --auto-servernum --server-args="-screen 0 1280x960x24" -- yarn affine @affine-test/affine-desktop-cloud e2e
services:
postgres:
image: postgres
env:
POSTGRES_PASSWORD: affine
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
redis:
image: redis
ports:
- 6379:6379
mailer:
image: mailhog/mailhog
ports:
- 1025:1025
- 8025:8025
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
playwright-install: true
hard-link-nm: false
- name: Download server-native.node
uses: actions/download-artifact@v4
with:
name: server-native.node
path: ./packages/backend/server
- name: Download affine.linux-x64-gnu.node
uses: actions/download-artifact@v4
with:
name: affine.linux-x64-gnu.node
path: ./packages/frontend/native
- name: Prepare Server Test Environment
uses: ./.github/actions/server-test-env
- name: ${{ matrix.tests.name }}
run: |
${{ matrix.tests.script }}
env:
DEV_SERVER_URL: http://localhost:8080
COPILOT_OPENAI_API_KEY: 1
COPILOT_FAL_API_KEY: 1
- name: Upload test results
if: ${{ failure() }}
uses: actions/upload-artifact@v4
with:
name: test-results-e2e-server
path: ./test-results
if-no-files-found: ignore
desktop-test:
name: Desktop Test (${{ matrix.spec.os }}, ${{ matrix.spec.platform }}, ${{ matrix.spec.arch }}, ${{ matrix.spec.target }}, ${{ matrix.spec.test }})
runs-on: ${{ matrix.spec.os }}
needs:
- optimize_ci
- build-electron-renderer
- build-native
if: needs.optimize_ci.outputs.skip == 'false'
strategy:
fail-fast: false
matrix:
spec:
- {
os: macos-latest,
platform: macos,
arch: x64,
target: x86_64-apple-darwin,
test: false,
}
- {
os: macos-latest,
platform: macos,
arch: arm64,
target: aarch64-apple-darwin,
test: true,
}
- {
os: ubuntu-latest,
platform: linux,
arch: x64,
target: x86_64-unknown-linux-gnu,
test: true,
}
- {
os: windows-latest,
platform: windows,
arch: x64,
target: x86_64-pc-windows-msvc,
test: true,
}
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: ./.github/actions/setup-node
timeout-minutes: 10
with:
extra-flags: workspaces focus @affine/electron @affine/monorepo @affine-test/affine-desktop @affine/nbstore @toeverything/infra
playwright-install: true
hard-link-nm: false
enableScripts: false
- name: Setup filename
id: filename
shell: bash
run: |
export PLATFORM_ARCH_ABI=$(node -e "console.log(require('@napi-rs/cli').parseTriple('${{ matrix.spec.target }}').platformArchABI)")
echo "filename=affine.$PLATFORM_ARCH_ABI.node" >> "$GITHUB_OUTPUT"
- name: Download ${{ steps.filename.outputs.filename }}
uses: actions/download-artifact@v4
with:
name: ${{ steps.filename.outputs.filename }}
path: ./packages/frontend/native
- name: Run unit tests
if: ${{ matrix.spec.test }}
shell: bash
run: yarn affine @affine/electron vitest
- name: Download web artifact
uses: ./.github/actions/download-web
with:
path: packages/frontend/apps/electron/resources/web-static
- name: Build Desktop Layers
run: yarn affine @affine/electron build
- name: Run desktop tests
if: ${{ matrix.spec.os == 'ubuntu-latest' }}
run: |
# Workaround for Electron apps failing to initialize on Ubuntu 24.04 due to AppArmor restrictions
# Disables unprivileged user namespaces restriction to allow Electron apps to run
# Reference: https://github.com/electron/electron/issues/42510
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
xvfb-run --auto-servernum --server-args="-screen 0 1280x960x24" -- yarn affine @affine-test/affine-desktop e2e
- name: Run desktop tests
if: ${{ matrix.spec.test && matrix.spec.os != 'ubuntu-latest' }}
run: yarn affine @affine-test/affine-desktop e2e
- name: Make bundle (macOS)
if: ${{ matrix.spec.target == 'aarch64-apple-darwin' }}
env:
SKIP_BUNDLE: true
SKIP_WEB_BUILD: true
HOIST_NODE_MODULES: 1
run: yarn affine @affine/electron package --platform=darwin --arch=arm64
- name: Make Bundle (Linux)
run: |
sudo add-apt-repository universe
sudo apt install -y libfuse2 elfutils flatpak flatpak-builder
flatpak remote-add --user --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
flatpak update
# some flatpak deps need git protocol.file.allow
git config --global protocol.file.allow always
yarn affine @affine/electron make --platform=linux --arch=x64
if: ${{ matrix.spec.target == 'x86_64-unknown-linux-gnu' }}
env:
SKIP_WEB_BUILD: 1
HOIST_NODE_MODULES: 1
- name: Output check
if: ${{ matrix.spec.os == 'macos-14' && matrix.spec.arch == 'arm64' }}
run: |
yarn affine @affine/electron node ./scripts/macos-arm64-output-check.ts
- name: Upload test results
if: ${{ failure() }}
uses: actions/upload-artifact@v4
with:
name: test-results-e2e-${{ matrix.spec.os }}-${{ matrix.spec.arch }}
path: ./test-results
if-no-files-found: ignore
test-build-mobile-app:
uses: ./.github/workflows/release-mobile.yml
needs: optimize_ci
if: needs.optimize_ci.outputs.skip == 'false'
with:
build-type: canary
build-target: development
secrets: inherit
permissions:
id-token: 'write'
test-done:
needs:
- analyze
- lint
- check-yarn-binary
- e2e-test
- e2e-legacy-blocksuite-test
- e2e-mobile-test
- unit-test
- build-native
- build-server-native
- build-electron-renderer
- server-test
- rust-test
- copilot-api-test
- copilot-e2e-test
- server-e2e-test
- desktop-test
- test-build-mobile-app
if: always()
runs-on: ubuntu-latest
name: 3, 2, 1 Launch
steps:
- run: exit 1
# Thank you, next https://github.com/vercel/next.js/blob/canary/.github/workflows/build_and_test.yml#L379
if: ${{ always() && (contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled')) }}

201
.github/workflows/build.yml vendored Normal file
View File

@@ -0,0 +1,201 @@
name: Build & Test
on:
push:
branches:
- master
- v[0-9]+.[0-9]+.x-staging
- v[0-9]+.[0-9]+.x
paths-ignore:
- README.md
- .github/**
- '!.github/workflows/build.yml'
- '!.github/actions/build-rust/action.yml'
- '!.github/actions/setup-node/action.yml'
pull_request:
merge_group:
branches:
- master
- v[0-9]+.[0-9]+.x-staging
- v[0-9]+.[0-9]+.x
paths-ignore:
- README.md
- .github/**
- '!.github/workflows/build.yml'
- '!.github/actions/build-rust/action.yml'
- '!.github/actions/setup-node/action.yml'
env:
DEBUG: napi:*
BUILD_TYPE: canary
APP_NAME: affine
AFFINE_ENV: dev
COVERAGE: true
DISTRIBUTION: browser
MACOSX_DEPLOYMENT_TARGET: '10.13'
NX_CLOUD_ACCESS_TOKEN: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}
jobs:
lint:
name: Lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run oxlint
# oxlint is fast, so wrong code will fail quickly
run: yarn dlx oxlint@latest .
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
electron-install: false
- name: Run i18n codegen
run: yarn i18n-codegen gen
- name: Run ESLint
run: yarn lint:eslint --max-warnings=0
- name: Run Prettier
# Set nmMode in `actions/setup-node` will modify the .yarnrc.yml
run: |
git checkout .yarnrc.yml
yarn lint:prettier
- name: Run circular
run: yarn circular
- name: Run Type Check
run: yarn typecheck
check-yarn-binary:
name: Check yarn binary
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run check
run: |
yarn set version $(node -e "console.log(require('./package.json').packageManager.split('@')[1])")
git diff --exit-code
e2e-plugin-test:
name: E2E Plugin Test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
playwright-install: true
electron-install: false
- name: Run playwright tests
run: yarn e2e --forbid-only
working-directory: tests/affine-plugin
env:
COVERAGE: true
- name: Collect code coverage report
run: yarn exec nyc report -t .nyc_output --report-dir .coverage --reporter=lcov
- name: Upload e2e test coverage results
uses: codecov/codecov-action@v3
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./.coverage/lcov.info
flags: e2e-plugin-test
name: affine
fail_ci_if_error: false
- name: Upload test results
if: ${{ failure() }}
uses: actions/upload-artifact@v3
with:
name: test-results-e2e-plugin
path: ./test-results
if-no-files-found: ignore
e2e-test:
name: E2E Test
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3, 4, 5]
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
playwright-install: true
electron-install: false
- name: Run playwright tests
run: yarn e2e --forbid-only --shard=${{ matrix.shard }}/${{ strategy.job-total }}
working-directory: tests/affine-local
env:
COVERAGE: true
- name: Collect code coverage report
run: yarn exec nyc report -t .nyc_output --report-dir .coverage --reporter=lcov
- name: Upload e2e test coverage results
uses: codecov/codecov-action@v3
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./.coverage/lcov.info
flags: e2etest
name: affine
fail_ci_if_error: false
- name: Upload test results
if: ${{ failure() }}
uses: actions/upload-artifact@v3
with:
name: test-results-e2e-${{ matrix.shard }}
path: ./test-results
if-no-files-found: ignore
e2e-migration-test:
name: E2E Migration Test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
playwright-install: true
electron-install: false
- name: Run playwright tests
run: yarn workspace @affine-test/affine-migration e2e --forbid-only
- name: Upload test results
if: ${{ failure() }}
uses: actions/upload-artifact@v3
with:
name: test-results-e2e-migration
path: ./tests/affine-migration/test-results
if-no-files-found: ignore
unit-test:
name: Unit Test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: ./.github/actions/setup-node
with:
electron-install: false
- name: Build AFFiNE native
uses: ./.github/actions/build-rust
with:
target: x86_64-unknown-linux-gnu
package: '@affine/native'
nx_token: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}
- name: Unit Test
run: yarn nx test:coverage @affine/monorepo
- name: Upload unit test coverage results
uses: codecov/codecov-action@v3
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./.coverage/store/lcov.info
flags: unittest
name: affine
fail_ci_if_error: false

36
.github/workflows/cache-cleanup.yml vendored Normal file
View File

@@ -0,0 +1,36 @@
# https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#force-deleting-cache-entries
name: Cleanup caches for closed branches
on:
pull_request:
types:
- closed
workflow_dispatch:
jobs:
cleanup:
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v4
- name: Cleanup
run: |
gh extension install actions/gh-actions-cache
REPO=${{ github.repository }}
BRANCH="refs/pull/${{ github.event.pull_request.number }}/merge"
echo "Fetching list of cache key"
cacheKeysForPR=$(gh actions-cache list -R $REPO -B $BRANCH | cut -f 1 )
## Setting this to not fail the workflow while deleting cache keys.
set +e
echo "Deleting caches..."
for cacheKey in $cacheKeysForPR
do
gh actions-cache delete $cacheKey -R $REPO -B $BRANCH --confirm
done
echo "Done"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

18
.github/workflows/cancel.yml vendored Normal file
View File

@@ -0,0 +1,18 @@
name: Cancel
on:
pull_request_target:
types:
- edited
- synchronize
jobs:
cancel:
name: 'Cancel Previous Runs'
runs-on: ubuntu-latest
timeout-minutes: 2
steps:
- uses: styfle/cancel-workflow-action@0.12.0
with:
# See https://api.github.com/repos/toeverything/AFFiNE/actions/workflows
workflow_id: 44038251, 61883931, 65188160, 66789140
access_token: ${{ github.token }}

70
.github/workflows/codeql.yml vendored Normal file
View File

@@ -0,0 +1,70 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: 'CodeQL'
on:
push:
branches: [master]
pull_request:
merge_group:
# The branches below must be a subset of the branches above
branches: [master]
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write
strategy:
fail-fast: false
matrix:
language: ['javascript']
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support
steps:
- name: Checkout repository
uses: actions/checkout@v4
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# Details on CodeQL's query packs refer to : https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
# queries: security-extended,security-and-quality
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v2
# Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
# If the Autobuild fails above, remove it and uncomment the following three lines.
# modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance.
# - run: |
# echo "Run, Build Application using script"
# ./location_of_script_within_repo/buildscript.sh
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2

Some files were not shown because too many files have changed in this diff Show More