Compare commits

..

3 Commits

Author SHA1 Message Date
DarkSky c4c9e3c36d fix: retry 2026-05-30 18:05:35 +08:00
DarkSky 1a8d884f8e chore(server): add logs 2026-05-30 16:53:16 +08:00
DarkSky 91acb88a2d fix(server): mail test & retry 2026-05-30 16:30:51 +08:00
22 changed files with 686 additions and 99 deletions
+1
View File
@@ -48,6 +48,7 @@ testem.log
/typings
tsconfig.tsbuildinfo
.context
*.md
# System Files
.DS_Store
@@ -858,7 +858,7 @@ Generated by [AVA](https://avajs.dev).
<!--/$-->␊
`
> test@test.com invited you to join Test Workspace
> You were invited to join a workspace on AFFiNE
`<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">␊
<link␊
@@ -973,7 +973,7 @@ Generated by [AVA](https://avajs.dev).
<!--/$-->␊
`
> test@test.com accepted your invitation
> Your workspace invitation was accepted
`<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">␊
<link␊
@@ -1072,7 +1072,7 @@ Generated by [AVA](https://avajs.dev).
<!--/$-->␊
`
> test@test.com left Test Workspace
> A workspace member left
`<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">␊
<link␊
@@ -1132,7 +1132,7 @@ Generated by [AVA](https://avajs.dev).
<!--/$-->␊
`
> New request to join Test Workspace
> New request to join a workspace
`<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">␊
<link␊
@@ -1239,7 +1239,7 @@ Generated by [AVA](https://avajs.dev).
<!--/$-->␊
`
> Your request to join Test Workspace has been approved
> Your request to join a workspace has been approved
`<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">␊
<link␊
@@ -1294,7 +1294,7 @@ Generated by [AVA](https://avajs.dev).
<!--/$-->␊
`
> Your request to join Test Workspace was declined
> Your request to join a workspace was declined
`<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">␊
<link␊
@@ -1348,7 +1348,7 @@ Generated by [AVA](https://avajs.dev).
<!--/$-->␊
`
> You have been removed from Test Workspace
> You have been removed from a workspace
`<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">␊
<link␊
@@ -1401,7 +1401,7 @@ Generated by [AVA](https://avajs.dev).
<!--/$-->␊
`
> Your ownership of Test Workspace has been transferred
> Your workspace ownership has been transferred
`<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">␊
<link␊
@@ -1454,7 +1454,7 @@ Generated by [AVA](https://avajs.dev).
<!--/$-->␊
`
> You are now the owner of Test Workspace
> You are now the owner of a workspace
`<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">␊
<link␊
@@ -1506,7 +1506,7 @@ Generated by [AVA](https://avajs.dev).
<!--/$-->␊
`
> test@test.com mentioned you in Test Doc
> You were mentioned in AFFiNE
`<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">␊
<!--$-->␊
@@ -1601,7 +1601,7 @@ Generated by [AVA](https://avajs.dev).
<!--/$-->␊
`
> test@test.com commented on Test Doc
> New comment in AFFiNE
`<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">␊
<!--$-->␊
@@ -1695,7 +1695,7 @@ Generated by [AVA](https://avajs.dev).
<!--/$-->␊
`
> test@test.com mentioned you in a comment on Test Doc
> You were mentioned in a comment
`<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">␊
<!--$-->␊
@@ -1894,7 +1894,7 @@ Generated by [AVA](https://avajs.dev).
<!--/$-->␊
`
> You are now an admin of Test Workspace
> You are now a workspace admin
`<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">␊
<link␊
@@ -1993,7 +1993,7 @@ Generated by [AVA](https://avajs.dev).
<!--/$-->␊
`
> Your role has been changed in Test Workspace
> Your workspace role has been changed
`<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">␊
<link␊
@@ -2094,7 +2094,7 @@ Generated by [AVA](https://avajs.dev).
<!--/$-->␊
`
> [Action Required] Final warning: Your workspace Test Workspace will be deleted in 24 hours
> [Action Required] Final warning: Your workspace will be deleted in 24 hours
`<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">␊
<link␊
@@ -2208,7 +2208,7 @@ Generated by [AVA](https://avajs.dev).
<!--/$-->␊
`
> [Action Required] Important: Your workspace Test Workspace will be deleted soon
> [Action Required] Important: Your workspace will be deleted soon
`<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">␊
<link␊
@@ -2324,7 +2324,7 @@ Generated by [AVA](https://avajs.dev).
<!--/$-->␊
`
> Your workspace Test Workspace has been deleted
> Your workspace has been deleted
`<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">␊
<link␊
@@ -2408,7 +2408,7 @@ Generated by [AVA](https://avajs.dev).
<!--/$-->␊
`
> [Action Required] Your Test Workspace team workspace will expire soon
> [Action Required] Your team workspace will expire soon
`<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">␊
<link␊
@@ -2511,7 +2511,7 @@ Generated by [AVA](https://avajs.dev).
<!--/$-->␊
`
> Your Test Workspace team workspace has expired
> Your team workspace has expired
`<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">␊
<link␊
@@ -2689,7 +2689,7 @@ Generated by [AVA](https://avajs.dev).
## should render mention email with empty doc title
> test@test.com mentioned you in
> You were mentioned in AFFiNE
`<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">␊
<!--$-->␊
@@ -1,5 +1,6 @@
import test from 'ava';
import { normalizeSMTPHeloHostname } from '../core/mail/utils';
import { Renderers } from '../mails';
import { TEST_DOC, TEST_USER } from '../mails/common';
@@ -21,3 +22,23 @@ test('should render mention email with empty doc title', async t => {
});
t.snapshot(content.html, content.subject);
});
test('should normalize valid SMTP HELO hostnames', t => {
t.is(normalizeSMTPHeloHostname('mail.example.com'), 'mail.example.com');
t.is(normalizeSMTPHeloHostname(' localhost '), 'localhost');
t.is(normalizeSMTPHeloHostname('[127.0.0.1]'), '[127.0.0.1]');
t.is(normalizeSMTPHeloHostname('[IPv6:2001:db8::1]'), '[IPv6:2001:db8::1]');
});
test('should reject invalid SMTP HELO hostnames', t => {
t.is(normalizeSMTPHeloHostname(), undefined);
t.is(normalizeSMTPHeloHostname(''), undefined);
t.is(normalizeSMTPHeloHostname(' '), undefined);
t.is(normalizeSMTPHeloHostname('AFFiNE Server'), undefined);
t.is(normalizeSMTPHeloHostname('-example.com'), undefined);
t.is(normalizeSMTPHeloHostname('example-.com'), undefined);
t.is(normalizeSMTPHeloHostname('example..com'), undefined);
t.is(normalizeSMTPHeloHostname('[bad host]'), undefined);
t.is(normalizeSMTPHeloHostname('[foo]'), undefined);
t.is(normalizeSMTPHeloHostname('[IPv6:foo]'), undefined);
});
@@ -6,6 +6,7 @@ import { JobQueue } from '../../base';
export class MockJobQueue {
add = Sinon.createStubInstance(JobQueue).add.resolves();
remove = Sinon.createStubInstance(JobQueue).remove.resolves();
removeWhere = Sinon.createStubInstance(JobQueue).removeWhere.resolves([]);
last<Job extends JobName>(name: Job): { name: Job; payload: Jobs[Job] } {
const addJobName = this.add.lastCall?.args[0];
@@ -437,6 +437,37 @@ test('should throw if user has subscription already', async t => {
);
});
test('should allow checkout after local subscription period ended', async t => {
const { service, u1, db, stripe } = t.context;
await db.subscription.create({
data: {
targetId: u1.id,
stripeSubscriptionId: 'sub_expired_ai',
plan: SubscriptionPlan.AI,
recurring: SubscriptionRecurring.Yearly,
status: SubscriptionStatus.Active,
start: new Date('2026-05-04T13:11:45.000Z'),
end: new Date('2026-05-11T13:11:45.000Z'),
},
});
await service.checkout(
{
plan: SubscriptionPlan.AI,
recurring: SubscriptionRecurring.Yearly,
successCallbackLink: '',
},
{ user: u1 }
);
t.true(stripe.checkout.sessions.create.calledOnce);
t.deepEqual(getLastCheckoutPrice(stripe.checkout.sessions.create), {
price: AI_YEARLY,
coupon: undefined,
});
});
test('should get correct pro plan price for checking out', async t => {
const { app, service, u1, stripe, feature } = t.context;
// non-ea user
@@ -117,6 +117,22 @@ test('should remove job from queue', async t => {
t.is(nullData, undefined);
t.is(nullJob, undefined);
});
test('should remove jobs by payload predicate', async t => {
const keep = await queue.add('nightly.__test__job', { name: 'keep' });
const remove = await queue.add('nightly.__test__job', { name: 'remove' });
const other = await queue.add('nightly.__test__job2', { name: 'remove' });
const removed = await queue.removeWhere(
'nightly.__test__job',
job => job.name === 'remove'
);
t.deepEqual(removed, [{ name: 'remove' }]);
t.truthy(await queue.get(keep.id!, 'nightly.__test__job'));
t.is(await queue.get(remove.id!, 'nightly.__test__job'), undefined);
t.truthy(await queue.get(other.id!, 'nightly.__test__job2'));
});
// #endregion
// #region executor
@@ -55,6 +55,39 @@ export class JobQueue {
return undefined;
}
async removeWhere<T extends JobName>(
jobName: T,
predicate: (payload: Jobs[T]) => boolean | Promise<boolean>
): Promise<Jobs[T][]> {
const ns = namespace(jobName);
const queue = this.getQueue(ns);
const jobs = (await queue.getJobs([
'waiting',
'delayed',
'prioritized',
'paused',
'waiting-children',
])) as Job<JobData<T>>[];
const removed: Jobs[T][] = [];
for (const job of jobs) {
if (job.name !== jobName) {
continue;
}
const payload = job.data.payload;
if (!(await predicate(payload))) {
continue;
}
await job.remove();
this.logger.log(`Job ${jobName}(id=${job.id}) removed from queue ${ns}`);
removed.push(payload);
}
return removed;
}
async get<T extends JobName>(jobId: string, jobName: T) {
const ns = namespace(jobName);
const queue = this.getQueue(ns);
@@ -0,0 +1,243 @@
import test from 'ava';
import Sinon from 'sinon';
import { Mockers } from '../../../__tests__/mocks';
import { createTestingModule } from '../../../__tests__/utils';
import { Cache } from '../../../base';
import { Models } from '../../../models';
import { MailJob } from '../job';
import { MailSender } from '../sender';
let module: Awaited<ReturnType<typeof createTestingModule>>;
let cache: Cache;
let mailJob: MailJob;
let sender: MailSender;
let models: Models;
test.before(async () => {
module = await createTestingModule();
cache = module.get(Cache);
mailJob = module.get(MailJob);
sender = module.get(MailSender);
models = module.get(Models);
});
test.after.always(async () => {
await module.close();
});
test.afterEach(() => {
Sinon.restore();
});
test('should clear pending mail records when user is deleted', async t => {
const user = await module.create(Mockers.User);
const another = await module.create(Mockers.User);
const sendMailKey = 'mailjob:sendMail';
const retryMailKey = 'mailjob:sendMail:retry';
const userKey = `${sendMailKey}:SignIn:${user.email}`;
const userRetryKey = `${sendMailKey}:VerifyEmail:${user.email}`;
const anotherKey = `${sendMailKey}:SignIn:${another.email}`;
await cache.mapSet(sendMailKey, userKey, 1);
await cache.mapSet(sendMailKey, anotherKey, 1);
await cache.mapSet(
retryMailKey,
userRetryKey,
JSON.stringify({
startTime: Date.now(),
name: 'VerifyEmail',
to: user.email,
props: { url: 'https://affine.pro/verify' },
})
);
await mailJob.onUserDeleted({ ...user, ownedWorkspaces: [] });
t.true(module.queue.removeWhere.calledOnce);
t.is(module.queue.removeWhere.firstCall.args[0], 'notification.sendMail');
const shouldRemove = module.queue.removeWhere.firstCall.args[1];
t.true(
await shouldRemove({
to: user.email,
} as Jobs['notification.sendMail'])
);
t.false(
await shouldRemove({
to: another.email,
} as Jobs['notification.sendMail'])
);
t.is(await cache.mapGet(sendMailKey, userKey), undefined);
t.is(await cache.mapGet(retryMailKey, userRetryKey), undefined);
t.is(await cache.mapGet(sendMailKey, anotherKey), 1);
});
test('should skip queued mail for disabled recipient', async t => {
const user = await module.create(Mockers.User, { disabled: true });
const send = Sinon.stub(sender, 'send').resolves(true);
await mailJob.sendMail({
startTime: Date.now(),
name: 'SignIn',
to: user.email,
props: {
url: 'https://affine.pro/sign-in',
otp: '123456',
},
});
t.false(send.called);
t.truthy(await models.user.get(user.id, { withDisabled: true }));
});
test('should drop expired mail retry', async t => {
const send = Sinon.stub(sender, 'send').resolves(true);
await mailJob.sendMail({
startTime: Date.now() - 25 * 60 * 60 * 1000,
name: 'SignIn',
to: 'expired-retry@example.com',
props: {
url: 'https://affine.pro/sign-in',
otp: '123456',
},
});
t.false(send.called);
});
test('should drop time-sensitive mail after its business expiration', async t => {
const send = Sinon.stub(sender, 'send').resolves(true);
await mailJob.sendMail({
startTime: Date.now() - 31 * 60 * 1000,
name: 'SignIn',
to: 'expired-sign-in@example.com',
props: {
url: 'https://affine.pro/sign-in',
otp: '123456',
},
});
t.false(send.called);
});
test('should use explicit mail expiration when provided', async t => {
const send = Sinon.stub(sender, 'send').resolves(true);
await mailJob.sendMail({
startTime: Date.now(),
expiresAt: Date.now() - 1,
name: 'MemberInvitation',
to: 'expired-invitation@example.com',
props: {
user: {
$$userId: 'owner-id',
},
workspace: {
$$workspaceId: 'workspace-id',
},
url: 'https://affine.pro/invite/test',
},
});
t.false(send.called);
});
test('should drop mail retry after max attempts', async t => {
const send = Sinon.stub(sender, 'send').resolves(true);
await mailJob.sendMail({
startTime: Date.now(),
retryCount: 12,
name: 'SignIn',
to: 'max-retry@example.com',
props: {
url: 'https://affine.pro/sign-in',
otp: '123456',
},
});
t.false(send.called);
});
test('should requeue legacy stringified retry mail', async t => {
const retryMailKey = 'mailjob:sendMail:retry';
const job: Jobs['notification.sendMail'] = {
startTime: Date.now(),
name: 'SignIn',
to: 'legacy-retry@example.com',
props: {
url: 'https://affine.pro/sign-in',
otp: '123456',
},
};
const cacheKey = `${retryMailKey}:SignIn:${job.to}`;
Sinon.stub(cache, 'mapRandomKey')
.onFirstCall()
.resolves(cacheKey)
.onSecondCall()
.resolves(undefined);
await cache.mapSet(retryMailKey, cacheKey, JSON.stringify(job));
await mailJob.sendRetryMails();
t.true(module.queue.add.calledWith('notification.sendMail', job));
t.is(await cache.mapGet(retryMailKey, cacheKey), undefined);
});
test('should skip member invitation mail when rendered workspace name contains domain', async t => {
const owner = await module.create(Mockers.User);
const member = await module.create(Mockers.User);
const workspace = await module.create(Mockers.Workspace, {
owner: { id: owner.id },
name: 'BTC https://spam.example',
});
const send = Sinon.stub(sender, 'send').resolves(true);
await mailJob.sendMail({
startTime: Date.now(),
name: 'MemberInvitation',
to: member.email,
props: {
user: {
$$userId: owner.id,
},
workspace: {
$$workspaceId: workspace.id,
},
url: 'https://affine.pro/invite/test',
},
});
t.false(send.called);
});
test('should keep dynamic mail props untouched for retry', async t => {
const owner = await module.create(Mockers.User);
const member = await module.create(Mockers.User);
const workspace = await module.create(Mockers.Workspace, {
owner: { id: owner.id },
name: 'Safe Workspace',
});
Sinon.stub(sender, 'send').resolves(false);
const job: Jobs['notification.sendMail'] = {
startTime: Date.now(),
name: 'MemberInvitation',
to: member.email,
props: {
user: {
$$userId: owner.id,
},
workspace: {
$$workspaceId: workspace.id,
},
url: 'https://affine.pro/invite/test',
},
};
await mailJob.sendMail(job);
t.deepEqual(job.props.user, { $$userId: owner.id });
t.deepEqual(job.props.workspace, { $$workspaceId: workspace.id });
});
+130 -12
View File
@@ -2,12 +2,13 @@ import { Injectable, Logger } from '@nestjs/common';
import { Cron, CronExpression } from '@nestjs/schedule';
import { getStreamAsBuffer } from 'get-stream';
import { Cache, JOB_SIGNAL, JobQueue, OnJob, sleep } from '../../base';
import { Cache, JOB_SIGNAL, JobQueue, OnEvent, OnJob, sleep } from '../../base';
import { type MailName, MailProps, Renderers } from '../../mails';
import { UserProps, WorkspaceProps } from '../../mails/components';
import { Models } from '../../models';
import { DocReader } from '../doc/reader';
import { WorkspaceBlobStorage } from '../storage';
import { containsUrlOrDomain } from '../workspaces/abuse';
import { MailSender, SendOptions } from './sender';
type DynamicallyFetchedProps<Props> = {
@@ -35,7 +36,11 @@ type SendMailJob<Mail extends MailName = MailName, Props = MailProps<Mail>> = {
declare global {
interface Jobs {
'notification.sendMail': { startTime: number } & {
'notification.sendMail': {
startTime: number;
retryCount?: number;
expiresAt?: number;
} & {
[K in MailName]: SendMailJob<K>;
}[MailName];
}
@@ -47,6 +52,19 @@ const sendMailCacheKey = (name: string, to: string) =>
`${sendMailKey}:${name}:${to}`;
const retryMaxPerTick = 20;
const retryFirstTime = 3;
const retryMaxAttempts = 12;
const retryMaxAge = 24 * 60 * 60 * 1000;
const magicLinkExpiresIn = 30 * 60 * 1000;
const mailExpiresIn: Partial<Record<MailName, number>> = {
SignIn: magicLinkExpiresIn,
SignUp: magicLinkExpiresIn,
SetPassword: magicLinkExpiresIn,
ChangePassword: magicLinkExpiresIn,
VerifyEmail: magicLinkExpiresIn,
ChangeEmail: magicLinkExpiresIn,
VerifyChangeEmail: magicLinkExpiresIn,
};
@Injectable()
export class MailJob {
@@ -66,17 +84,65 @@ export class MailJob {
return Math.min(30 * 1000, Math.round(elapsed / 2000) * 1000);
}
private getRetryExhaustedReason({
startTime,
retryCount,
expiresAt,
name,
}: Jobs['notification.sendMail']) {
const expiredAt =
expiresAt ?? startTime + (mailExpiresIn[name] ?? retryMaxAge);
if (Date.now() > expiredAt) {
return 'expired';
}
if ((retryCount ?? 0) > retryMaxAttempts) {
return 'max attempts reached';
}
return;
}
private async shouldSkipRecipient(to: string) {
const user = await this.models.user.getUserByEmail(to, {
withDisabled: true,
});
return user?.disabled === true;
}
private async deleteRecipientMailCache(to: string) {
const suffix = `:${to}`;
await Promise.all(
[sendMailKey, retryMailKey].map(async map => {
const keys = await this.cache.mapKeys(map);
await Promise.all(
keys
.filter(key => key.endsWith(suffix))
.map(key => this.cache.mapDelete(map, key))
);
})
);
}
private async sendMailInternal({
startTime,
name,
to,
props,
}: Jobs['notification.sendMail']) {
let options: Partial<SendOptions> = {};
if (await this.shouldSkipRecipient(to)) {
this.logger.debug(`Skip mail [${name}] to disabled user [${to}]`);
return;
}
for (const key in props) {
let options: Partial<SendOptions> = {};
const renderedProps = { ...props };
for (const key in renderedProps) {
// @ts-expect-error allow
const val = props[key];
const val = renderedProps[key];
if (val && typeof val === 'object') {
if ('$$workspaceId' in val) {
const workspaceProps = await this.fetchWorkspaceProps(
@@ -87,6 +153,16 @@ export class MailJob {
return;
}
if (
name === 'MemberInvitation' &&
containsUrlOrDomain(workspaceProps.name)
) {
this.logger.warn(
`Skip mail [${name}] to [${to}], reason=workspace name contains url or domain`
);
return;
}
if (workspaceProps.avatar) {
options.attachments = [
{
@@ -99,7 +175,7 @@ export class MailJob {
workspaceProps.avatar = 'cid:workspaceAvatar';
}
// @ts-expect-error replacement
props[key] = workspaceProps;
renderedProps[key] = workspaceProps;
} else if ('$$userId' in val) {
const userProps = await this.fetchUserProps(val.$$userId);
@@ -108,17 +184,30 @@ export class MailJob {
}
// @ts-expect-error replacement
props[key] = userProps;
renderedProps[key] = userProps;
}
}
}
if (
name === 'MemberInvitation' &&
'workspace' in renderedProps &&
containsUrlOrDomain(
(renderedProps.workspace as WorkspaceProps | undefined)?.name
)
) {
this.logger.warn(
`Skip mail [${name}] to [${to}], reason=workspace name contains url or domain`
);
return;
}
try {
const result = await this.sender.send(name, {
to,
...(await Renderers[name](
// @ts-expect-error the job trigger part has been typechecked
props
renderedProps
)),
...options,
});
@@ -130,7 +219,7 @@ export class MailJob {
}
return undefined;
} catch (e) {
this.logger.error(`Failed to send mail [${name}] to [${to}]`, e);
this.logger.error(`Failed to send mail [${name}] to [${to}]`, e, props);
// wait for a while before retrying
const retryDelay = this.calculateRetryDelay(startTime);
await sleep(retryDelay);
@@ -177,17 +266,41 @@ export class MailJob {
@OnJob('notification.sendMail')
async sendMail(job: Jobs['notification.sendMail']) {
const cacheKey = sendMailCacheKey(job.name, job.to);
job.retryCount = (job.retryCount ?? 0) + 1;
const exhaustedReason = this.getRetryExhaustedReason(job);
if (exhaustedReason) {
this.logger.warn(
`Drop mail [${job.name}] to [${job.to}], reason=${exhaustedReason}`
);
await Promise.all([
this.cache.mapDelete(sendMailKey, cacheKey),
this.cache.mapDelete(retryMailKey, cacheKey),
]);
return;
}
const retried = await this.cache.mapIncrease(sendMailKey, cacheKey, 1);
if (retried <= retryFirstTime) {
const ret = await this.sendMailInternal(job);
if (!ret) await this.cache.mapDelete(sendMailKey, cacheKey);
return ret;
}
await this.cache.mapSet(retryMailKey, cacheKey, JSON.stringify(job));
await this.cache.mapSet(retryMailKey, cacheKey, job);
await this.cache.mapDelete(sendMailKey, cacheKey);
return undefined;
}
@OnEvent('user.deleted')
async onUserDeleted(user: Events['user.deleted']) {
await Promise.all([
this.deleteRecipientMailCache(user.email),
this.queue.removeWhere(
'notification.sendMail',
job => job.to === user.email
),
]);
}
@Cron(CronExpression.EVERY_MINUTE)
async sendRetryMails() {
// pick random one from the retry map
@@ -195,9 +308,14 @@ export class MailJob {
let key = await this.cache.mapRandomKey(retryMailKey);
while (key && processed < retryMaxPerTick) {
try {
const job = await this.cache.mapGet<string>(retryMailKey, key);
const job = await this.cache.mapGet<
Jobs['notification.sendMail'] | string
>(retryMailKey, key);
if (job) {
const jobData = JSON.parse(job) as Jobs['notification.sendMail'];
const jobData =
typeof job === 'string'
? (JSON.parse(job) as Jobs['notification.sendMail'])
: job;
await this.queue.add('notification.sendMail', jobData);
// wait for a while before retrying
const retryDelay = this.calculateRetryDelay(jobData.startTime);
@@ -140,7 +140,11 @@ export class MailSender {
return true;
} catch (e) {
metrics.mail.counter('failed_total').add(1, { name });
this.logger.error(`Failed to send mail [${name}].`, e);
this.logger.error(`Failed to send mail [${name}].`, e, {
subject: options.subject,
from: options.from,
to: options.to,
});
return false;
}
}
@@ -0,0 +1,55 @@
import { isIP } from 'node:net';
import { hostname as getHostname } from 'node:os';
const hostnameLabelRegexp = /^[A-Za-z0-9-]+$/;
function isValidSMTPAddressLiteral(hostname: string) {
if (!hostname.startsWith('[') || !hostname.endsWith(']')) return false;
const literal = hostname.slice(1, -1);
if (!literal || literal.includes(' ')) return false;
if (isIP(literal) === 4) return true;
if (literal.startsWith('IPv6:')) {
return isIP(literal.slice('IPv6:'.length)) === 6;
}
return false;
}
export function normalizeSMTPHeloHostname(hostname?: string) {
if (!hostname) return undefined;
const normalized = hostname.trim().replace(/\.$/, '');
if (!normalized) return undefined;
if (isValidSMTPAddressLiteral(normalized)) return normalized;
if (normalized.length > 253) return undefined;
const labels = normalized.split('.');
for (const label of labels) {
if (!label || label.length > 63) return undefined;
if (
!hostnameLabelRegexp.test(label) ||
label.startsWith('-') ||
label.endsWith('-')
) {
return undefined;
}
}
return normalized;
}
function readSystemHostname() {
try {
return getHostname();
} catch {
return '';
}
}
export function resolveSMTPHeloHostname(configuredName: string) {
const normalizedConfiguredName = normalizeSMTPHeloHostname(configuredName);
if (normalizedConfiguredName) return normalizedConfiguredName;
return normalizeSMTPHeloHostname(readSystemHostname());
}
@@ -87,6 +87,29 @@ test('should create invitation notification and email', async t => {
t.is(invitationMail.payload.name, 'MemberInvitation');
});
test('should not send invitation email when workspace name contains domain', async t => {
const spamWorkspace = await module.create(Mockers.Workspace, {
owner: {
id: owner.id,
},
name: 'BTC https://spam.example',
});
const inviteId = randomUUID();
const invitationMailCount = module.queue.count('notification.sendMail');
const notification = await notificationService.createInvitation({
userId: member.id,
body: {
workspaceId: spamWorkspace.id,
createdByUserId: owner.id,
inviteId,
},
});
t.truthy(notification);
t.is(module.queue.count('notification.sendMail'), invitationMailCount);
});
test('should not send invitation email if user setting is not to receive invitation email', async t => {
const inviteId = randomUUID();
await module.create(Mockers.UserSettings, {
@@ -22,6 +22,7 @@ import {
generateWorkspaceSettingsPath,
WorkspaceSettingsTab,
} from '../utils/workspace';
import { containsUrlOrDomain } from '../workspaces/abuse';
@Injectable()
export class NotificationService {
@@ -151,6 +152,16 @@ export class NotificationService {
}
private async sendInvitationEmail(input: InvitationNotificationCreate) {
const workspace = await this.docReader.getWorkspaceContent(
input.body.workspaceId
);
if (containsUrlOrDomain(workspace?.name)) {
this.logger.warn(
`Skip invitation email for workspace ${input.body.workspaceId}, reason=workspace name contains url or domain`
);
return;
}
const inviteUrl = this.url.link(`/invite/${input.body.inviteId}`);
if (env.dev) {
// make it easier to test in dev mode
@@ -0,0 +1,12 @@
export const SHARE_ACTION_ACCOUNT_AGE_MS = 24 * 60 * 60 * 1000;
const URL_OR_DOMAIN_PATTERN =
/(?:https?:\/\/|www\.|(?<![@\w-])(?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.)+[a-z]{2,63}(?:[/?#:]|$))/i;
export function containsUrlOrDomain(value: string | null | undefined) {
return URL_OR_DOMAIN_PATTERN.test(value ?? '');
}
export function isUserOldEnoughForShareActions(user: { createdAt: Date }) {
return Date.now() - user.createdAt.getTime() >= SHARE_ACTION_ACCOUNT_AGE_MS;
}
@@ -15,6 +15,7 @@ import { PrismaClient } from '@prisma/client';
import { SafeIntResolver } from 'graphql-scalars';
import {
ActionForbidden,
Cache,
DocActionDenied,
DocDefaultRoleCanNotBeOwner,
@@ -40,6 +41,7 @@ import {
DocRole,
} from '../../permission';
import { PublicUserType, WorkspaceUserType } from '../../user';
import { isUserOldEnoughForShareActions } from '../abuse';
import { WorkspaceType } from '../types';
import { TimeBucket, TimeWindow } from './analytics-types';
import {
@@ -299,6 +301,15 @@ export class WorkspaceDocResolver {
private readonly cache: Cache
) {}
private async assertCanShare(userId: string) {
const user = await this.models.user.get(userId);
if (!user || !isUserOldEnoughForShareActions(user)) {
throw new ActionForbidden(
'Sharing links is unavailable during the first 24 hours after signup.'
);
}
}
@ResolveField(() => WorkspaceDocMeta, {
description: 'Cloud page metadata of workspace',
complexity: 2,
@@ -413,6 +424,7 @@ export class WorkspaceDocResolver {
}
await this.ac.user(user.id).doc(workspaceId, docId).assert('Doc.Publish');
await this.assertCanShare(user.id);
const doc = await this.models.doc.publish(workspaceId, docId, mode);
@@ -15,6 +15,7 @@ import {
import { nanoid } from 'nanoid';
import {
ActionForbidden,
ActionForbiddenOnNonTeamWorkspace,
AlreadyInSpace,
AuthenticationRequired,
@@ -40,6 +41,7 @@ import { AccessController, WorkspaceRole } from '../../permission';
import { QuotaService } from '../../quota';
import { UserType } from '../../user';
import { validators } from '../../utils/validators';
import { containsUrlOrDomain, isUserOldEnoughForShareActions } from '../abuse';
import { WorkspaceService } from '../service';
import {
InvitationType,
@@ -68,6 +70,24 @@ export class WorkspaceMemberResolver {
private readonly quota: QuotaService
) {}
private async assertCanInviteOrShare(userId: string) {
const user = await this.models.user.get(userId);
if (!user || !isUserOldEnoughForShareActions(user)) {
throw new ActionForbidden(
'Inviting members and creating share links are unavailable during the first 24 hours after signup.'
);
}
}
private async assertWorkspaceNameCanInvite(workspaceId: string) {
const workspace = await this.workspaceService.getWorkspaceInfo(workspaceId);
if (containsUrlOrDomain(workspace.name)) {
throw new ActionForbidden(
'Workspace names containing links or domains cannot be used to invite members.'
);
}
}
@ResolveField(() => UserType, {
description: 'Owner of workspace',
complexity: 2,
@@ -141,6 +161,8 @@ export class WorkspaceMemberResolver {
.user(me.id)
.workspace(workspaceId)
.assert('Workspace.Users.Manage');
await this.assertCanInviteOrShare(me.id);
await this.assertWorkspaceNameCanInvite(workspaceId);
if (emails.length > 512) {
throw new TooManyRequest();
@@ -272,6 +294,8 @@ export class WorkspaceMemberResolver {
.user(user.id)
.workspace(workspaceId)
.assert('Workspace.Users.Manage');
await this.assertCanInviteOrShare(user.id);
await this.assertWorkspaceNameCanInvite(workspaceId);
const cacheWorkspaceId = `workspace:inviteLink:${workspaceId}`;
const invite = await this.cache.get<{ inviteId: string }>(cacheWorkspaceId);
+20 -45
View File
@@ -83,95 +83,70 @@ export const Renderers = {
//#region Workspace
MemberInvitation: make(
Invitation,
props => `${props.user.email} invited you to join ${props.workspace.name}`
'You were invited to join a workspace on AFFiNE'
),
MemberAccepted: make(
InvitationAccepted,
props => `${props.user.email} accepted your invitation`
),
MemberLeave: make(
MemberLeave,
props => `${props.user.email} left ${props.workspace.name}`
'Your workspace invitation was accepted'
),
MemberLeave: make(MemberLeave, 'A workspace member left'),
LinkInvitationReviewRequest: make(
LinkInvitationReviewRequest,
props => `New request to join ${props.workspace.name}`
'New request to join a workspace'
),
LinkInvitationApprove: make(
LinkInvitationApproved,
props => `Your request to join ${props.workspace.name} has been approved`
'Your request to join a workspace has been approved'
),
LinkInvitationDecline: make(
LinkInvitationReviewDeclined,
props => `Your request to join ${props.workspace.name} was declined`
),
MemberRemoved: make(
MemberRemoved,
props => `You have been removed from ${props.workspace.name}`
'Your request to join a workspace was declined'
),
MemberRemoved: make(MemberRemoved, 'You have been removed from a workspace'),
OwnershipTransferred: make(
OwnershipTransferred,
props => `Your ownership of ${props.workspace.name} has been transferred`
'Your workspace ownership has been transferred'
),
OwnershipReceived: make(
OwnershipReceived,
props => `You are now the owner of ${props.workspace.name}`
'You are now the owner of a workspace'
),
//#endregion
//#region Doc
Mention: make(
Mention,
props => `${props.user.email} mentioned you in ${props.doc.title}`
),
Comment: make(
Comment,
props => `${props.user.email} commented on ${props.doc.title}`
),
CommentMention: make(
CommentMention,
props =>
`${props.user.email} mentioned you in a comment on ${props.doc.title}`
),
Mention: make(Mention, 'You were mentioned in AFFiNE'),
Comment: make(Comment, 'New comment in AFFiNE'),
CommentMention: make(CommentMention, 'You were mentioned in a comment'),
//#endregion
//#region Team
TeamWorkspaceUpgraded: make(TeamWorkspaceUpgraded, props =>
props.isOwner
? 'Your workspace has been upgraded to team workspace! 🎉'
: `${props.workspace.name} has been upgraded to team workspace! 🎉`
),
TeamBecomeAdmin: make(
TeamBecomeAdmin,
props => `You are now an admin of ${props.workspace.name}`
: 'A workspace has been upgraded to team workspace! 🎉'
),
TeamBecomeAdmin: make(TeamBecomeAdmin, 'You are now a workspace admin'),
TeamBecomeCollaborator: make(
TeamBecomeCollaborator,
props => `Your role has been changed in ${props.workspace.name}`
'Your workspace role has been changed'
),
TeamDeleteIn24Hours: make(
TeamDeleteIn24Hours,
props =>
`[Action Required] Final warning: Your workspace ${props.workspace.name} will be deleted in 24 hours`
'[Action Required] Final warning: Your workspace will be deleted in 24 hours'
),
TeamDeleteInOneMonth: make(
TeamDeleteInOneMonth,
props =>
`[Action Required] Important: Your workspace ${props.workspace.name} will be deleted soon`
'[Action Required] Important: Your workspace will be deleted soon'
),
TeamWorkspaceDeleted: make(
TeamWorkspaceDeleted,
props => `Your workspace ${props.workspace.name} has been deleted`
'Your workspace has been deleted'
),
TeamWorkspaceExpireSoon: make(
TeamExpireSoon,
props =>
`[Action Required] Your ${props.workspace.name} team workspace will expire soon`
),
TeamWorkspaceExpired: make(
TeamExpired,
props => `Your ${props.workspace.name} team workspace has expired`
'[Action Required] Your team workspace will expire soon'
),
TeamWorkspaceExpired: make(TeamExpired, 'Your team workspace has expired'),
//#endregion
//#region License
@@ -114,22 +114,17 @@ export class UserSubscriptionManager extends SubscriptionManager {
throw new ManagedByAppStoreOrPlay();
}
const subscription = await this.getSubscription({
plan: lookupKey.plan,
userId: user.id,
});
if (
subscription &&
active &&
// do not allow to re-subscribe unless
!(
/* current subscription is a onetime subscription and so as the one that's checking out */
(
(subscription.variant === SubscriptionVariant.Onetime &&
(active.variant === SubscriptionVariant.Onetime &&
lookupKey.variant === SubscriptionVariant.Onetime) ||
/* current subscription is normal subscription and is checking-out a lifetime subscription */
(subscription.recurring !== SubscriptionRecurring.Lifetime &&
subscription.variant !== SubscriptionVariant.Onetime &&
(active.recurring !== SubscriptionRecurring.Lifetime &&
active.variant !== SubscriptionVariant.Onetime &&
lookupKey.recurring === SubscriptionRecurring.Lifetime)
)
)
@@ -1,10 +1,21 @@
mutation sendTestEmail($host: String!, $port: Int!, $sender: String!, $username: String!, $password: String!, $ignoreTLS: Boolean!) {
sendTestEmail(config: {
host: $host,
port: $port,
sender: $sender,
username: $username,
password: $password,
ignoreTLS: $ignoreTLS,
})
}
mutation sendTestEmail(
$name: String!
$host: String!
$port: Int!
$sender: String!
$username: String!
$password: String!
$ignoreTLS: Boolean!
) {
sendTestEmail(
config: {
name: $name
host: $host
port: $port
sender: $sender
username: $username
password: $password
ignoreTLS: $ignoreTLS
}
)
}
+2 -2
View File
@@ -549,9 +549,9 @@ export const listUsersQuery = {
export const sendTestEmailMutation = {
id: 'sendTestEmailMutation' as const,
op: 'sendTestEmail',
query: `mutation sendTestEmail($host: String!, $port: Int!, $sender: String!, $username: String!, $password: String!, $ignoreTLS: Boolean!) {
query: `mutation sendTestEmail($name: String!, $host: String!, $port: Int!, $sender: String!, $username: String!, $password: String!, $ignoreTLS: Boolean!) {
sendTestEmail(
config: {host: $host, port: $port, sender: $sender, username: $username, password: $password, ignoreTLS: $ignoreTLS}
config: {name: $name, host: $host, port: $port, sender: $sender, username: $username, password: $password, ignoreTLS: $ignoreTLS}
)
}`,
};
+1
View File
@@ -4027,6 +4027,7 @@ export type ListUsersQuery = {
};
export type SendTestEmailMutationVariables = Exact<{
name: Scalars['String']['input'];
host: Scalars['String']['input'];
port: Scalars['Int']['input'];
sender: Scalars['String']['input'];