perf(federation): Ed25519署名に対応する (#13464)

* 1. ed25519キーペアを発行・Personとして公開鍵を送受信

* validate additionalPublicKeys

* getAuthUserFromApIdはmainを選ぶ

* :v:

* fix

* signatureAlgorithm

* set publicKeyCache lifetime

* refresh

* httpMessageSignatureAcceptable

* ED25519_SIGNED_ALGORITHM

* ED25519_PUBLIC_KEY_SIGNATURE_ALGORITHM

* remove sign additionalPublicKeys signature requirements

* httpMessageSignaturesSupported

* httpMessageSignaturesImplementationLevel

* httpMessageSignaturesImplementationLevel: '01'

* perf(federation): Use hint for getAuthUserFromApId (#13470)

* Hint for getAuthUserFromApId

* とどのつまりこれでいいのか？

* use @misskey-dev/node-http-message-signatures

* fix

* signedPost, signedGet

* ap-request.tsを復活させる

* remove digest prerender

* fix test?

* fix test

* add httpMessageSignaturesImplementationLevel to FederationInstance

* ManyToOne

* fetchPersonWithRenewal

* exactKey

* :v:

* use const

* use gen-key-pair fn. from  '@misskey-dev/node-http-message-signatures'

* update node-http-message-signatures

* fix

* @misskey-dev/node-http-message-signatures@0.0.0-alpha.11

* getAuthUserFromApIdでupdatePersonの頻度を増やす

* cacheRaw.date

* use requiredInputs
https://github.com/misskey-dev/misskey/pull/13464#discussion_r1509964359

* update @misskey-dev/node-http-message-signatures

* clean up

* err msg

* fix(backend): fetchInstanceMetadataのLockが永遠に解除されない問題を修正

Co-authored-by: まっちゃとーにゅ <17376330+u1-liquid@users.noreply.github.com>

* fix httpMessageSignaturesImplementationLevel validation

* fix test

* fix

* comment

* comment

* improve test

* fix

* use Promise.all in genRSAAndEd25519KeyPair

* refreshAndprepareEd25519KeyPair

* refreshAndfindKey

* commetn

* refactor public keys add

* digestプリレンダを復活させる

RFC実装時にどうするか考える

* fix, async

* fix

* !== true

* use save

* Deliver update person when new key generated (not tested)
https://github.com/misskey-dev/misskey/pull/13464#issuecomment-1977049061

* 循環参照で落ちるのを解消？

* fix?

* Revert "fix?"

This reverts commit 0082f6f8e8c5d5febd14933ba9a1ac643f70ca92.

* a

* logger

* log

* change logger

* 秘密鍵の変更は、フラグではなく鍵を引き回すようにする

* addAllKnowingSharedInboxRecipe

* nanka meccha kaeta

* delivre

* キャッシュ有効チェックはロック取得前に行う

* @misskey-dev/node-http-message-signatures@0.0.3

* PrivateKeyPem

* getLocalUserPrivateKey

* fix test

* if

* fix ap-request

* update node-http-message-signatures

* fix type error

* update package

* fix type

* update package

* retry no key

* @misskey-dev/node-http-message-signatures@0.0.8

* fix type error

* log keyid

* logger

* db-resolver

* JSON.stringify

* HTTP Signatureがなかったり使えなかったりしそうな場合にLD Signatureを活用するように

* inbox-delayed use actor if no signature

* ユーザーとキーの同一性チェックはhostの一致にする

* log signature parse err

* save array

* とりあえずtryで囲っておく

* fetchPersonWithRenewalでエラーが起きたら古いデータを返す

* use transactionalEntityManager

* fix spdx

* @misskey-dev/node-http-message-signatures@0.0.10

* add comment

* fix

* publicKeyに配列が入ってもいいようにする
https://github.com/misskey-dev/misskey/pull/13950

* define additionalPublicKeys

* fix

* merge fix

* refreshAndprepareEd25519KeyPair → refreshAndPrepareEd25519KeyPair

* remove gen-key-pair.ts

* defaultMaxListeners = 512

* Revert "defaultMaxListeners = 512"

This reverts commit f2c412c18057a9300540794ccbe4dfbf6d259ed6.

* genRSAAndEd25519KeyPairではキーを直列に生成する?

* maxConcurrency: 8

* maxConcurrency: 16

* maxConcurrency: 8

* Revert "genRSAAndEd25519KeyPairではキーを直列に生成する?"

This reverts commit d0aada55c1ed5aa98f18731ec82f3ac5eb5a6c16.

* maxWorkers: '90%'

* Revert "maxWorkers: '90%'"

This reverts commit 9e0a93f110456320d6485a871f014f7cdab29b33.

* e2e/timelines.tsで個々のテストに対するtimeoutを削除, maxConcurrency: 32

* better error handling of this.userPublickeysRepository.delete

* better comment

* set result to keypairEntityCache

* deliverJobConcurrency: 16, deliverJobPerSec: 1024, inboxJobConcurrency: 4

* inboxJobPerSec: 64

* delete request.headers['host'];

* fix

* // node-fetch will generate this for us. if we keep 'Host', it won't change with redirects!

* move delete host

* modify comment

* modify comment

* fix correct → collect

* refreshAndfindKey → refreshAndFindKey

* modify comment

* modify attachLdSignature

* getApId, InboxProcessorService

* TODO

* [skip ci] add CHANGELOG

---------

Co-authored-by: MeiMei <30769358+mei23@users.noreply.github.com>
Co-authored-by: まっちゃとーにゅ <17376330+u1-liquid@users.noreply.github.com>

1 files changed, 149 insertions, 6 deletions

diff --git a/packages/backend/src/core/UserKeypairService.ts b/packages/backend/src/core/UserKeypairService.ts
index 51ac99179a..aa90f1e209 100644
--- a/packages/backend/src/core/UserKeypairService.ts
+++ b/packages/backend/src/core/UserKeypairService.ts
@@ -5,41 +5,184 @@
 
 import { Inject, Injectable, OnApplicationShutdown } from '@nestjs/common';
 import * as Redis from 'ioredis';
+import { genEd25519KeyPair, importPrivateKey, PrivateKey, PrivateKeyWithPem } from '@misskey-dev/node-http-message-signatures';
 import type { MiUser } from '@/models/User.js';
 import type { UserKeypairsRepository } from '@/models/_.js';
-import { RedisKVCache } from '@/misc/cache.js';
+import { RedisKVCache, MemoryKVCache } from '@/misc/cache.js';
 import type { MiUserKeypair } from '@/models/UserKeypair.js';
 import { DI } from '@/di-symbols.js';
 import { bindThis } from '@/decorators.js';
+import { GlobalEventService, GlobalEvents } from '@/core/GlobalEventService.js';
+import { UserEntityService } from '@/core/entities/UserEntityService.js';
+import type { webcrypto } from 'node:crypto';
 
 @Injectable()
 export class UserKeypairService implements OnApplicationShutdown {
-	private cache: RedisKVCache<MiUserKeypair>;
+	private keypairEntityCache: RedisKVCache<MiUserKeypair>;
+	private privateKeyObjectCache: MemoryKVCache<webcrypto.CryptoKey>;
 
 	constructor(
 		@Inject(DI.redis)
 		private redisClient: Redis.Redis,
-
+		@Inject(DI.redisForSub)
+		private redisForSub: Redis.Redis,
 		@Inject(DI.userKeypairsRepository)
 		private userKeypairsRepository: UserKeypairsRepository,
+
+		private globalEventService: GlobalEventService,
+		private userEntityService: UserEntityService,
 	) {
-		this.cache = new RedisKVCache<MiUserKeypair>(this.redisClient, 'userKeypair', {
+		this.keypairEntityCache = new RedisKVCache<MiUserKeypair>(this.redisClient, 'userKeypair', {
 			lifetime: 1000 * 60 * 60 * 24, // 24h
 			memoryCacheLifetime: Infinity,
 			fetcher: (key) => this.userKeypairsRepository.findOneByOrFail({ userId: key }),
 			toRedisConverter: (value) => JSON.stringify(value),
 			fromRedisConverter: (value) => JSON.parse(value),
 		});
+		this.privateKeyObjectCache = new MemoryKVCache<webcrypto.CryptoKey>(1000 * 60 * 60 * 1);
+
+		this.redisForSub.on('message', this.onMessage);
 	}
 
 	@bindThis
 	public async getUserKeypair(userId: MiUser['id']): Promise<MiUserKeypair> {
-		return await this.cache.fetch(userId);
+		return await this.keypairEntityCache.fetch(userId);
+	}
+
+	/**
+	 * Get private key [Only PrivateKeyWithPem for queue data etc.]
+	 * @param userIdOrHint user id or MiUserKeypair
+	 * @param preferType
+	 *		If ed25519-like(`ed25519`, `01`, `11`) is specified, ed25519 keypair will be returned if exists.
+	 *		Otherwise, main keypair will be returned.
+	 * @returns
+	 */
+	@bindThis
+	public async getLocalUserPrivateKeyPem(
+		userIdOrHint: MiUser['id'] | MiUserKeypair,
+		preferType?: string,
+	): Promise<PrivateKeyWithPem> {
+		const keypair = typeof userIdOrHint === 'string' ? await this.getUserKeypair(userIdOrHint) : userIdOrHint;
+		if (
+			preferType && ['01', '11', 'ed25519'].includes(preferType.toLowerCase()) &&
+			keypair.ed25519PublicKey != null && keypair.ed25519PrivateKey != null
+		) {
+			return {
+				keyId: `${this.userEntityService.genLocalUserUri(keypair.userId)}#ed25519-key`,
+				privateKeyPem: keypair.ed25519PrivateKey,
+			};
+		}
+		return {
+			keyId: `${this.userEntityService.genLocalUserUri(keypair.userId)}#main-key`,
+			privateKeyPem: keypair.privateKey,
+		};
+	}
+
+	/**
+	 * Get private key [Only PrivateKey for ap request]
+	 * Using cache due to performance reasons of `crypto.subtle.importKey`
+	 * @param userIdOrHint user id, MiUserKeypair, or PrivateKeyWithPem
+	 * @param preferType
+	 * 		If ed25519-like(`ed25519`, `01`, `11`) is specified, ed25519 keypair will be returned if exists.
+	 *		Otherwise, main keypair will be returned. (ignored if userIdOrHint is PrivateKeyWithPem)
+	 * @returns
+	 */
+	@bindThis
+	public async getLocalUserPrivateKey(
+		userIdOrHint: MiUser['id'] | MiUserKeypair | PrivateKeyWithPem,
+		preferType?: string,
+	): Promise<PrivateKey> {
+		if (typeof userIdOrHint === 'object' && 'privateKeyPem' in userIdOrHint) {
+			// userIdOrHint is PrivateKeyWithPem
+			return {
+				keyId: userIdOrHint.keyId,
+				privateKey: await this.privateKeyObjectCache.fetch(userIdOrHint.keyId, async () => {
+					return await importPrivateKey(userIdOrHint.privateKeyPem);
+				}),
+			};
+		}
+
+		const userId = typeof userIdOrHint === 'string' ? userIdOrHint : userIdOrHint.userId;
+		const getKeypair = () => typeof userIdOrHint === 'string' ? this.getUserKeypair(userId) : userIdOrHint;
+
+		if (preferType && ['01', '11', 'ed25519'].includes(preferType.toLowerCase())) {
+			const keyId = `${this.userEntityService.genLocalUserUri(userId)}#ed25519-key`;
+			const fetched = await this.privateKeyObjectCache.fetchMaybe(keyId, async () => {
+				const keypair = await getKeypair();
+				if (keypair.ed25519PublicKey != null && keypair.ed25519PrivateKey != null) {
+					return await importPrivateKey(keypair.ed25519PrivateKey);
+				}
+				return;
+			});
+			if (fetched) {
+				return {
+					keyId,
+					privateKey: fetched,
+				};
+			}
+		}
+
+		const keyId = `${this.userEntityService.genLocalUserUri(userId)}#main-key`;
+		return {
+			keyId,
+			privateKey: await this.privateKeyObjectCache.fetch(keyId, async () => {
+				const keypair = await getKeypair();
+				return await importPrivateKey(keypair.privateKey);
+			}),
+		};
 	}
 
 	@bindThis
+	public async refresh(userId: MiUser['id']): Promise<void> {
+		return await this.keypairEntityCache.refresh(userId);
+	}
+
+	/**
+	 * If DB has ed25519 keypair, refresh cache and return it.
+	 * If not, create, save and return ed25519 keypair.
+	 * @param userId user id
+	 * @returns MiUserKeypair if keypair is created, void if keypair is already exists
+	 */
+	@bindThis
+	public async refreshAndPrepareEd25519KeyPair(userId: MiUser['id']): Promise<MiUserKeypair | void> {
+		await this.refresh(userId);
+		const keypair = await this.keypairEntityCache.fetch(userId);
+		if (keypair.ed25519PublicKey != null) {
+			return;
+		}
+
+		const ed25519 = await genEd25519KeyPair();
+		await this.userKeypairsRepository.update({ userId }, {
+			ed25519PublicKey: ed25519.publicKey,
+			ed25519PrivateKey: ed25519.privateKey,
+		});
+		this.globalEventService.publishInternalEvent('userKeypairUpdated', { userId });
+		const result = {
+			...keypair,
+			ed25519PublicKey: ed25519.publicKey,
+			ed25519PrivateKey: ed25519.privateKey,
+		};
+		this.keypairEntityCache.set(userId, result);
+		return result;
+	}
+
+	@bindThis
+	private async onMessage(_: string, data: string): Promise<void> {
+		const obj = JSON.parse(data);
+
+		if (obj.channel === 'internal') {
+			const { type, body } = obj.message as GlobalEvents['internal']['payload'];
+			switch (type) {
+				case 'userKeypairUpdated': {
+					this.refresh(body.userId);
+					break;
+				}
+			}
+		}
+	}
+	@bindThis
 	public dispose(): void {
-		this.cache.dispose();
+		this.keypairEntityCache.dispose();
 	}
 
 	@bindThis