diff options
| author | Julia <julia@insertdomain.name> | 2025-06-19 21:35:18 +0000 |
|---|---|---|
| committer | Julia <julia@insertdomain.name> | 2025-06-19 21:35:18 +0000 |
| commit | a77c32b17da63d3932b219f74152cce023a30f4a (patch) | |
| tree | d2a05796e942c8f250bbd01369eab0cbe5a14531 | |
| parent | merge: release 2025.4.2 (!1051) (diff) | |
| parent | Merge branch 'develop' into release/2025.4.3 (diff) | |
| download | sharkey-a77c32b17da63d3932b219f74152cce023a30f4a.tar.gz sharkey-a77c32b17da63d3932b219f74152cce023a30f4a.tar.bz2 sharkey-a77c32b17da63d3932b219f74152cce023a30f4a.zip | |
merge: prepare release 2025.4.3 (!1125)
View MR for information: https://activitypub.software/TransFem-org/Sharkey/-/merge_requests/1125
Approved-by: Marie <github@yuugi.dev>
Approved-by: Julia <julia@insertdomain.name>
396 files changed, 10456 insertions, 5046 deletions
diff --git a/.config/ci.yml b/.config/ci.yml index 5657b8beae..5fcf78b737 100644 --- a/.config/ci.yml +++ b/.config/ci.yml @@ -115,8 +115,14 @@ db: user: postgres pass: ci - # Whether disable Caching queries - #disableCache: true + ## Log a warning to the server console if any query takes longer than this to complete. + ## Measured in milliseconds; set to 0 to disable. (default: 300) + #slowQueryThreshold: 300 + + # If false, then query results will be cached in redis. + # If true (default), then queries will not be cached. + # This will reduce database load at the cost of increased Redis traffic and risk of bugs and unpredictable behavior. + #disableCache: false # Extra Connection options #extra: diff --git a/.config/cypress-devcontainer.yml b/.config/cypress-devcontainer.yml index ca62616462..97263da68f 100644 --- a/.config/cypress-devcontainer.yml +++ b/.config/cypress-devcontainer.yml @@ -57,8 +57,14 @@ db: user: postgres pass: postgres - # Whether disable Caching queries - #disableCache: true + ## Log a warning to the server console if any query takes longer than this to complete. + ## Measured in milliseconds; set to 0 to disable. (default: 300) + #slowQueryThreshold: 300 + + # If false, then query results will be cached in redis. + # If true (default), then queries will not be cached. + # This will reduce database load at the cost of increased Redis traffic and risk of bugs and unpredictable behavior. + #disableCache: false # Extra Connection options #extra: diff --git a/.config/docker_example.yml b/.config/docker_example.yml index df5d77a97f..3aaa56e333 100644 --- a/.config/docker_example.yml +++ b/.config/docker_example.yml @@ -118,8 +118,14 @@ db: user: example-misskey-user pass: example-misskey-pass - # Whether disable Caching queries - #disableCache: true + ## Log a warning to the server console if any query takes longer than this to complete. + ## Measured in milliseconds; set to 0 to disable. (default: 300) + #slowQueryThreshold: 300 + + # If false, then query results will be cached in redis. + # If true (default), then queries will not be cached. + # This will reduce database load at the cost of increased Redis traffic and risk of bugs and unpredictable behavior. + #disableCache: false # Extra Connection options #extra: diff --git a/.config/example.yml b/.config/example.yml index 9e2b6d6da5..8cac42c050 100644 --- a/.config/example.yml +++ b/.config/example.yml @@ -121,8 +121,14 @@ db: user: sharkey pass: example-misskey-pass - # Whether disable Caching queries - #disableCache: true + ## Log a warning to the server console if any query takes longer than this to complete. + ## Measured in milliseconds; set to 0 to disable. (default: 300) + #slowQueryThreshold: 300 + + # If false, then query results will be cached in redis. + # If true (default), then queries will not be cached. + # This will reduce database load at the cost of increased Redis traffic and risk of bugs and unpredictable behavior. + #disableCache: false # Extra Connection options #extra: diff --git a/.gitignore b/.gitignore index b07d195a3f..ea761882da 100644 --- a/.gitignore +++ b/.gitignore @@ -87,3 +87,7 @@ vite.config.local-dev.ts.timestamp-* # Sharkey /packages/megalodon/lib + +# TypeScript +.tsbuildinfo +*.tsbuildinfo diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c8a58fadab..e1a12926cc 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -622,6 +622,35 @@ marginはそのコンポーネントを使う側が設定する ### indexというファイル名を使うな ESMではディレクトリインポートは廃止されているのと、ディレクトリインポートせずともファイル名が index だと何故か一部のライブラリ？でディレクトリインポートだと見做されてエラーになる +### Memory Caches + +Sharkey offers multiple memory cache implementations, each meant for a different use case. +The following table compares the available options: + +| Cache | Type | Consistency | Persistence | Data Source | Cardinality | Eviction | Description | +|---------------------|-----------|-------------|-------------|-------------|-------------|----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `MemoryKVCache` | Key-Value | None | None | Caller | Single | Lifetime | Implements a basic in-memory Key-Value store. The implementation is entirely synchronous, except for user-provided data sources. | +| `MemorySingleCache` | Single | None | None | Caller | Single | Lifetime | Implements a basic in-memory Single Value store. The implementation is entirely synchronous, except for user-provided data sources. | +| `RedisKVCache` | Key-Value | Eventual | Redis | Callback | Single | Lifetime | Extends `MemoryKVCache` with Redis-backed persistence and a pre-defined callback data source. This provides eventual consistency guarantees based on the memory cache lifetime. | +| `RedisSingleCache` | Single | Eventual | Redis | Callback | Single | Lifetime | Extends `MemorySingleCache` with Redis-backed persistence and a pre-defined callback data source. This provides eventual consistency guarantees based on the memory cache lifetime. | +| `QuantumKVCache` | Key-Value | Immediate | None | Callback | Multiple | Lifetime | Combines `MemoryKVCache` with a pre-defined callback data source and immediate consistency via Redis sync events. The implementation offers multi-item batch overloads for efficient bulk operations. **This is the recommended cache implementation for most use cases.** | + +Key-Value caches store multiple entries per cache, while Single caches store a single value that can be accessed directly. +Consistency refers to the consistency of cached data between different processes in the instance cluster: "None" means no consistency guarantees, "Eventual" caches will gradually become consistent after some unknown time, and "Immediate" consistency ensures accurate data ASAP after the update. +Caches with persistence can retain their data after a reboot through an external service such as Redis. +If a data source is supported, then this allows the cache to directly load missing data in response to a fetch. +"Caller" data sources are passed into the fetch method(s) directly, while "Callback" sources are passed in as a function when the cache is first initialized. +The cardinality of a cache refers to the number of items that can be updated in a single operation, and eviction, finally, is the method that the cache uses to evict stale data. + +#### Selecting a cache implementation + +For most cache uses, `QuantumKVCache` should be considered first. +It offers strong consistency guarantees, multiple cardinality, and a cleaner API surface than the older caches. +An alternate cache implementation should be considered if any of the following apply: +* The data is particularly slow to calculate or difficult to access. In these cases, either `RedisKVCache` or `RedisSingleCache` should be considered. +* If stale data is acceptable, then consider `MemoryKVCache` or `MemorySingleCache`. These synchronous implementations have much less overhead than the other options. +* There is only one data item, or all data items must be fetched together. Using `MemorySingleCache` or `RedisSingleCache` could provide a cleaner implementation without resorting to hacks like a fixed key. + ## CSS Recipe ### Lighten CSS vars @@ -690,7 +719,7 @@ seems to do a decent job) * re-generate locales (`pnpm run build-assets`) and commit * build the frontend: `rm -rf built/; NODE_ENV=development pnpm --filter=frontend --filter=frontend-embed --filter=frontend-shared build` (the `development` tells it to keep some of the original filenames in the built files) * make sure there aren't any new `ti-*` classes (Tabler Icons), and replace them with appropriate `ph-*` ones (Phosphor Icons) in [`vite.replaceicons.ts`](packages/frontend/vite.replaceIcons.ts). - * This command should show you want to change: `grep -ohrP '(?<=["'\'']ti )(ti-(?!fw)[\w\-]+)' --exclude \*.map -- built/ | sort -u`. + * This command should show you want to change: `grep -ohrP '(?<=["'\''](ti )?)(ti-(?!fw)[\w\-]+)' --exclude \*.map -- built/ | sort -u`. * NOTE: `ti-fw` is a special class that's defined by Misskey, leave it alone. * After every change, re-build the frontend and check again, until there are no more `ti-*` classes in the built files. * Commit! diff --git a/Dockerfile b/Dockerfile index 72f934b3ce..e6e60992e3 100644 --- a/Dockerfile +++ b/Dockerfile @@ -38,7 +38,7 @@ ARG UID="991" ARG GID="991" ENV COREPACK_DEFAULT_TO_LATEST=0 -RUN apk add ffmpeg tini jemalloc pixman pango cairo libpng \ +RUN apk add ffmpeg tini jemalloc pixman pango cairo libpng librsvg font-noto font-noto-cjk font-noto-thai \ && corepack enable \ && addgroup -g "${GID}" sharkey \ && adduser -D -u "${UID}" -G sharkey -h /sharkey sharkey \ diff --git a/cypress/tsconfig.json b/cypress/tsconfig.json index 6fe7f32cc4..4435a4fda8 100644 --- a/cypress/tsconfig.json +++ b/cypress/tsconfig.json @@ -2,7 +2,8 @@ "compilerOptions": { "lib": ["dom", "es5"], "target": "es5", - "types": ["cypress", "node"] + "types": ["cypress", "node"], + "incremental": true }, "include": ["./**/*.ts"] } diff --git a/locales/index.d.ts b/locales/index.d.ts index 69c63cc714..a22a8e893e 100644 --- a/locales/index.d.ts +++ b/locales/index.d.ts @@ -12493,6 +12493,14 @@ export interface Locale extends ILocale { */ "centerDescription": string; /** + * Unix Time + */ + "unixtime": string; + /** + * Displays a timestamp in the viewer's current timezone. + */ + "unixtimeDescription": string; + /** * Code (Inline) */ "inlineCode": string; @@ -13070,6 +13078,26 @@ export interface Locale extends ILocale { */ "popularUsersLocal": ParameterizedString<"name">; /** + * Polls trending on {name} + */ + "pollsOnLocal": ParameterizedString<"name">; + /** + * Polls trending on the global network + */ + "pollsOnRemote": string; + /** + * Polls that have ended recently + */ + "pollsExpired": string; + /** + * Trending polls are disabled on this instance. + */ + "trendingPollsDisabled": string; + /** + * Please log in to view trending polls. + */ + "trendingPollsDisabledLogIn": string; + /** * Silenced */ "silenced": string; @@ -13129,6 +13157,110 @@ export interface Locale extends ILocale { * Timeout in milliseconds for translation API requests. */ "translationTimeoutCaption": string; + /** + * Staff notes + */ + "staffNotes": string; + /** + * Icon of {name} + */ + "instanceIconAlt": ParameterizedString<"name">; + /** + * Attribution Domains + */ + "attributionDomains": string; + /** + * A list of domains whose content can be attributed to you on link previews, separated by new-line. Any subdomain will also be valid. The following needs to be on the webpage: + */ + "attributionDomainsDescription": string; + /** + * Written by {user} + */ + "writtenBy": ParameterizedString<"user">; + /** + * Following (Pub) + */ + "followingPub": string; + /** + * Followers (Sub) + */ + "followersSub": string; + /** + * Well-known resources + */ + "wellKnownResources": string; + /** + * Last posted: {at} + */ + "lastPosted": ParameterizedString<"at">; + /** + * NSFW + */ + "nsfw": string; + /** + * Raw + */ + "raw": string; + /** + * CW + */ + "cw": string; + /** + * Media Silenced + */ + "mediaSilenced": string; + /** + * Bubble + */ + "bubble": string; + /** + * Verified + */ + "verified": string; + /** + * Not Verified + */ + "notVerified": string; + /** + * Hibernated + */ + "hibernated": string; + /** + * When replying to a post with a Content Warning, automatically use the same CW for the reply. + */ + "keepCwDescription": string; + /** + * Disabled (do not copy CWs) + */ + "keepCwDisabled": string; + /** + * Enabled (copy CWs verbatim) + */ + "keepCwEnabled": string; + /** + * Enabled (copy CW and prepend "RE:") + */ + "keepCwPrependRe": string; + /** + * Note controls + */ + "noteFooterLabel": string; + /** + * Packed user data in its raw form. Most of these fields are public and visible to all users. + */ + "rawUserDescription": string; + /** + * Extended user data in its raw form. These fields are private and can only be accessed by moderators. + */ + "rawInfoDescription": string; + /** + * ActivityPub user data in its raw form. These fields are public and accessible to other instances. + */ + "rawApDescription": string; + /** + * Signup Reason + */ + "signupReason": string; } declare const locales: { [lang: string]: Locale; diff --git a/package.json b/package.json index f088b31b52..24777d9d87 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "sharkey", - "version": "2025.4.2", + "version": "2025.4.3", "codename": "shonk", "repository": { "type": "git", @@ -54,17 +54,7 @@ "lodash": "4.17.21" }, "dependencies": { - "cssnano": "7.0.6", - "esbuild": "0.25.3", - "execa": "9.5.2", - "fast-glob": "3.3.3", - "glob": "11.0.2", - "ignore-walk": "7.0.0", - "js-yaml": "4.1.0", - "postcss": "8.5.3", - "tar": "7.4.3", - "terser": "5.39.0", - "typescript": "5.8.3" + "js-yaml": "4.1.0" }, "optionalDependencies": { "cypress": "14.3.2" @@ -75,10 +65,20 @@ "@typescript-eslint/eslint-plugin": "8.31.0", "@typescript-eslint/parser": "8.31.0", "cross-env": "7.0.3", + "cssnano": "7.0.6", + "esbuild": "0.25.3", "eslint": "9.25.1", - "globals": "16.0.0", + "execa": "9.5.2", + "fast-glob": "3.3.3", + "glob": "11.0.2", + "globals": "16.1.0", "ncp": "2.0.0", - "pnpm": "10.10.0", - "start-server-and-test": "2.0.11" + "pnpm": "9.6.0", + "ignore-walk": "7.0.0", + "postcss": "8.5.3", + "start-server-and-test": "2.0.11", + "tar": "7.4.3", + "terser": "5.39.0", + "typescript": "5.8.3" } } diff --git a/packages/backend/migration/1747938628395-add-missing-indexes.js b/packages/backend/migration/1747938628395-add-missing-indexes.js index 745b39c855..0229a6c898 100644 --- a/packages/backend/migration/1747938628395-add-missing-indexes.js +++ b/packages/backend/migration/1747938628395-add-missing-indexes.js @@ -4,19 +4,35 @@ */ export class AddMissingIndexes1747938628395 { - name = 'AddMissingIndexes1747938628395' + name = 'AddMissingIndexes1747938628395' - async up(queryRunner) { - await queryRunner.query(`CREATE INDEX "IDX_58699f75b9cf904f5f007909cb" ON "user_profile" ("birthday") `); - await queryRunner.query(`CREATE INDEX "IDX_021015e6683570ae9f6b0c62be" ON "user_list_membership" ("userId") `); - await queryRunner.query(`CREATE INDEX "IDX_cddcaf418dc4d392ecfcca842a" ON "user_list_membership" ("userListId") `); - await queryRunner.query(`CREATE UNIQUE INDEX "IDX_e4f3094c43f2d665e6030b0337" ON "user_list_membership" ("userId", "userListId") `); - } + async up(queryRunner) { + // Some instances have duplicate list entries + await queryRunner.query(` + DELETE FROM "user_list_membership" + WHERE "id" NOT IN ( + SELECT MIN("id") + FROM "user_list_membership" + GROUP BY "userId", "userListId" + )`); - async down(queryRunner) { - await queryRunner.query(`DROP INDEX "public"."IDX_e4f3094c43f2d665e6030b0337"`); - await queryRunner.query(`DROP INDEX "public"."IDX_cddcaf418dc4d392ecfcca842a"`); - await queryRunner.query(`DROP INDEX "public"."IDX_021015e6683570ae9f6b0c62be"`); - await queryRunner.query(`DROP INDEX "public"."IDX_58699f75b9cf904f5f007909cb"`); - } + // Some instances already have these indexes, for an unknown reason + await queryRunner.query(`DROP INDEX IF EXISTS "public"."IDX_e4f3094c43f2d665e6030b0337"`); + await queryRunner.query(`DROP INDEX IF EXISTS "public"."IDX_cddcaf418dc4d392ecfcca842a"`); + await queryRunner.query(`DROP INDEX IF EXISTS "public"."IDX_021015e6683570ae9f6b0c62be"`); + await queryRunner.query(`DROP INDEX IF EXISTS "public"."IDX_58699f75b9cf904f5f007909cb"`); + + // Now the actual migration + await queryRunner.query(`CREATE INDEX "IDX_58699f75b9cf904f5f007909cb" ON "user_profile" ("birthday") `); + await queryRunner.query(`CREATE INDEX "IDX_021015e6683570ae9f6b0c62be" ON "user_list_membership" ("userId") `); + await queryRunner.query(`CREATE INDEX "IDX_cddcaf418dc4d392ecfcca842a" ON "user_list_membership" ("userListId") `); + await queryRunner.query(`CREATE UNIQUE INDEX "IDX_e4f3094c43f2d665e6030b0337" ON "user_list_membership" ("userId", "userListId") `); + } + + async down(queryRunner) { + await queryRunner.query(`DROP INDEX "public"."IDX_e4f3094c43f2d665e6030b0337"`); + await queryRunner.query(`DROP INDEX "public"."IDX_cddcaf418dc4d392ecfcca842a"`); + await queryRunner.query(`DROP INDEX "public"."IDX_021015e6683570ae9f6b0c62be"`); + await queryRunner.query(`DROP INDEX "public"."IDX_58699f75b9cf904f5f007909cb"`); + } } diff --git a/packages/backend/migration/1748096357260-AddAttributionDomains.js b/packages/backend/migration/1748096357260-AddAttributionDomains.js new file mode 100644 index 0000000000..0a9679bccd --- /dev/null +++ b/packages/backend/migration/1748096357260-AddAttributionDomains.js @@ -0,0 +1,16 @@ +/* + * SPDX-FileCopyrightText: piuvas and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +export class AddAttributionDomains1748096357260 { + name = 'AddAttributionDomains1748096357260' + + async up(queryRunner) { + await queryRunner.query(`ALTER TABLE "user" ADD "attributionDomains" text array NOT NULL DEFAULT '{}'`); + } + + async down(queryRunner) { + await queryRunner.query(`ALTER TABLE "user" DROP COLUMN "attributionDomains"`); + } +} diff --git a/packages/backend/migration/1748104955717-index_IDX_instance_host_key.js b/packages/backend/migration/1748104955717-index_IDX_instance_host_key.js new file mode 100644 index 0000000000..139eae740f --- /dev/null +++ b/packages/backend/migration/1748104955717-index_IDX_instance_host_key.js @@ -0,0 +1,16 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +export class IndexIDXInstanceHostKey1748104955717 { + name = 'IndexIDXInstanceHostKey1748104955717' + + async up(queryRunner) { + await queryRunner.query(`CREATE UNIQUE INDEX "IDX_instance_host_key" ON "instance" (((lower(reverse("host")) || '.')::text) text_pattern_ops)`); + } + + async down(queryRunner) { + await queryRunner.query(`DROP INDEX "IDX_instance_host_key"`); + } +} diff --git a/packages/backend/migration/1748105111513-add_instance_block_columns.js b/packages/backend/migration/1748105111513-add_instance_block_columns.js new file mode 100644 index 0000000000..6e3d78d5e8 --- /dev/null +++ b/packages/backend/migration/1748105111513-add_instance_block_columns.js @@ -0,0 +1,85 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +/** + * @typedef {import('typeorm').MigrationInterface} MigrationInterface + * @typedef {{ blockedHosts: string[], silencedHosts: string[], mediaSilencedHosts: string[], federationHosts: string[], bubbleInstances: string[] }} Meta + */ + +/** + * @class + * @implements {MigrationInterface} + */ +export class AddInstanceBlockColumns1748105111513 { + name = 'AddInstanceBlockColumns1748105111513' + + async up(queryRunner) { + // Schema migration + await queryRunner.query(`ALTER TABLE "instance" ADD "isBlocked" boolean NOT NULL DEFAULT false`); + await queryRunner.query(`COMMENT ON COLUMN "instance"."isBlocked" IS 'True if this instance is blocked from federation.'`); + await queryRunner.query(`ALTER TABLE "instance" ADD "isAllowListed" boolean NOT NULL DEFAULT false`); + await queryRunner.query(`COMMENT ON COLUMN "instance"."isAllowListed" IS 'True if this instance is allow-listed.'`); + await queryRunner.query(`ALTER TABLE "instance" ADD "isBubbled" boolean NOT NULL DEFAULT false`); + await queryRunner.query(`COMMENT ON COLUMN "instance"."isBubbled" IS 'True if this instance is part of the local bubble.'`); + await queryRunner.query(`ALTER TABLE "instance" ADD "isSilenced" boolean NOT NULL DEFAULT false`); + await queryRunner.query(`COMMENT ON COLUMN "instance"."isSilenced" IS 'True if this instance is silenced.'`); + await queryRunner.query(`ALTER TABLE "instance" ADD "isMediaSilenced" boolean NOT NULL DEFAULT false`); + await queryRunner.query(`COMMENT ON COLUMN "instance"."isMediaSilenced" IS 'True if this instance is media-silenced.'`); + + // Data migration + /** @type {Meta[]} */ + const metas = await queryRunner.query(`SELECT "blockedHosts", "silencedHosts", "mediaSilencedHosts", "federationHosts", "bubbleInstances" FROM "meta"`); + if (metas.length > 0) { + /** @type {Meta} */ + const meta = metas[0]; + + // Blocked hosts + if (meta.blockedHosts.length > 0) { + const patterns = buildPatterns(meta.blockedHosts); + await queryRunner.query(`UPDATE "instance" SET "isBlocked" = true WHERE ((lower(reverse("host")) || '.')::text) LIKE ANY ($1)`, [ patterns ]); + } + + // Silenced hosts + if (meta.silencedHosts.length > 0) { + const patterns = buildPatterns(meta.silencedHosts); + await queryRunner.query(`UPDATE "instance" SET "isSilenced" = true WHERE ((lower(reverse("host")) || '.')::text) LIKE ANY ($1)`, [ patterns ]); + } + + // Media silenced hosts + if (meta.mediaSilencedHosts.length > 0) { + const patterns = buildPatterns(meta.mediaSilencedHosts); + await queryRunner.query(`UPDATE "instance" SET "isMediaSilenced" = true WHERE ((lower(reverse("host")) || '.')::text) LIKE ANY ($1)`, [ patterns ]); + } + + // Allow-listed hosts + if (meta.federationHosts.length > 0) { + const patterns = buildPatterns(meta.federationHosts); + await queryRunner.query(`UPDATE "instance" SET "isAllowListed" = true WHERE ((lower(reverse("host")) || '.')::text) LIKE ANY ($1)`, [ patterns ]); + } + + // Bubbled hosts + if (meta.bubbleInstances.length > 0) { + const patterns = buildPatterns(meta.bubbleInstances); + await queryRunner.query(`UPDATE "instance" SET "isBubbled" = true WHERE ((lower(reverse("host")) || '.')::text) LIKE ANY ($1)`, [ patterns ]); + } + } + } + + async down(queryRunner) { + await queryRunner.query(`ALTER TABLE "instance" DROP COLUMN "isMediaSilenced"`); + await queryRunner.query(`ALTER TABLE "instance" DROP COLUMN "isSilenced"`); + await queryRunner.query(`ALTER TABLE "instance" DROP COLUMN "isBubbled"`); + await queryRunner.query(`ALTER TABLE "instance" DROP COLUMN "isAllowListed"`); + await queryRunner.query(`ALTER TABLE "instance" DROP COLUMN "isBlocked"`); + } +} + +/** + * @param {string[]} input + * @returns {string[]} + */ +function buildPatterns(input) { + return input.map(i => i.toLowerCase().split('').reverse().join('') + '.%'); +} diff --git a/packages/backend/migration/1748128176881-add_instance_foreign_keys.js b/packages/backend/migration/1748128176881-add_instance_foreign_keys.js new file mode 100644 index 0000000000..2c2383c50f --- /dev/null +++ b/packages/backend/migration/1748128176881-add_instance_foreign_keys.js @@ -0,0 +1,44 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +/** + * @typedef {import('typeorm').MigrationInterface} MigrationInterface + */ + +/** + * @class + * @implements {MigrationInterface} + */ +export class AddInstanceForeignKeys1748128176881 { + name = 'AddInstanceForeignKeys1748128176881' + + async up(queryRunner) { + // Fix-up: Some older instances have users without a matching instance entry + await queryRunner.query(` + INSERT INTO "instance" ("id", "host", "firstRetrievedAt") + SELECT + MIN("id"), + "host", + COALESCE(MIN("lastFetchedAt"), CURRENT_TIMESTAMP) + FROM "user" + WHERE + "host" IS NOT NULL AND + NOT EXISTS (select 1 from "instance" where "instance"."host" = "user"."host") + GROUP BY "host" + `); + + await queryRunner.query(`ALTER TABLE "user" ADD CONSTRAINT "FK_user_host" FOREIGN KEY ("host") REFERENCES "instance"("host") ON DELETE CASCADE ON UPDATE NO ACTION`); + await queryRunner.query(`ALTER TABLE "note" ADD CONSTRAINT "FK_note_userHost" FOREIGN KEY ("userHost") REFERENCES "instance"("host") ON DELETE CASCADE ON UPDATE NO ACTION`); + await queryRunner.query(`ALTER TABLE "note" ADD CONSTRAINT "FK_note_replyUserHost" FOREIGN KEY ("replyUserHost") REFERENCES "instance"("host") ON DELETE CASCADE ON UPDATE NO ACTION`); + await queryRunner.query(`ALTER TABLE "note" ADD CONSTRAINT "FK_note_renoteUserHost" FOREIGN KEY ("renoteUserHost") REFERENCES "instance"("host") ON DELETE CASCADE ON UPDATE NO ACTION`); + } + + async down(queryRunner) { + await queryRunner.query(`ALTER TABLE "note" DROP CONSTRAINT "FK_note_renoteUserHost"`); + await queryRunner.query(`ALTER TABLE "note" DROP CONSTRAINT "FK_note_replyUserHost"`); + await queryRunner.query(`ALTER TABLE "note" DROP CONSTRAINT "FK_note_userHost"`); + await queryRunner.query(`ALTER TABLE "user" DROP CONSTRAINT "FK_user_host"`); + } +} diff --git a/packages/backend/migration/1748137683887-add_instance_foreign_keys_to_following.js b/packages/backend/migration/1748137683887-add_instance_foreign_keys_to_following.js new file mode 100644 index 0000000000..8f4a977ff5 --- /dev/null +++ b/packages/backend/migration/1748137683887-add_instance_foreign_keys_to_following.js @@ -0,0 +1,26 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +/** + * @typedef {import('typeorm').MigrationInterface} MigrationInterface + */ + +/** + * @class + * @implements {MigrationInterface} + */ +export class AddInstanceForeignKeysToFollowing1748137683887 { + name = 'AddInstanceForeignKeysToFollowing1748137683887' + + async up(queryRunner) { + await queryRunner.query(`ALTER TABLE "following" ADD CONSTRAINT "FK_following_followerHost" FOREIGN KEY ("followerHost") REFERENCES "instance"("host") ON DELETE CASCADE ON UPDATE NO ACTION`); + await queryRunner.query(`ALTER TABLE "following" ADD CONSTRAINT "FK_following_followeeHost" FOREIGN KEY ("followeeHost") REFERENCES "instance"("host") ON DELETE CASCADE ON UPDATE NO ACTION`); + } + + async down(queryRunner) { + await queryRunner.query(`ALTER TABLE "following" DROP CONSTRAINT "FK_following_followeeHost"`); + await queryRunner.query(`ALTER TABLE "following" DROP CONSTRAINT "FK_following_followerHost"`); + } +} diff --git a/packages/backend/migration/1748191631151-analyze_instance-user-note-following.js b/packages/backend/migration/1748191631151-analyze_instance-user-note-following.js new file mode 100644 index 0000000000..f03a60980b --- /dev/null +++ b/packages/backend/migration/1748191631151-analyze_instance-user-note-following.js @@ -0,0 +1,25 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +/** + * @typedef {import('typeorm').MigrationInterface} MigrationInterface + */ + +/** + * @class + * @implements {MigrationInterface} + */ +export class AnalyzeInstanceUserNoteFollowing1748191631151 { + name = 'AnalyzeInstanceUserNoteFollowing1748191631151' + + async up(queryRunner) { + // Refresh statistics for tables impacted by new indexes. + // This helps the query planner to efficiently use them without waiting for the next full vacuum. + await queryRunner.query(`ANALYZE "instance", "user", "following", "note"`); + } + + async down(queryRunner) { + } +} diff --git a/packages/backend/migration/1748990452958-replace_note-userHost_index.js b/packages/backend/migration/1748990452958-replace_note-userHost_index.js new file mode 100644 index 0000000000..55aadd8136 --- /dev/null +++ b/packages/backend/migration/1748990452958-replace_note-userHost_index.js @@ -0,0 +1,22 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +export class ReplaceNoteUserHostIndex1748990452958 { + name = 'ReplaceNoteUserHostIndex1748990452958' + + async up(queryRunner) { + await queryRunner.query(`DROP INDEX "public"."IDX_7125a826ab192eb27e11d358a5"`); + await queryRunner.query(` + create index "IDX_note_userHost_id" + on "note" ("userHost", "id" desc) + nulls not distinct`); + await queryRunner.query(`comment on index "IDX_note_userHost_id" is 'User host with ID included'`); + } + + async down(queryRunner) { + await queryRunner.query(`drop index if exists "IDX_note_userHost_id"`); + await queryRunner.query(`CREATE INDEX "IDX_7125a826ab192eb27e11d358a5" ON "note" ("userHost") `); + } +} diff --git a/packages/backend/migration/1748990662839-fix-IDX_instance_host_key.js b/packages/backend/migration/1748990662839-fix-IDX_instance_host_key.js new file mode 100644 index 0000000000..fc6d303743 --- /dev/null +++ b/packages/backend/migration/1748990662839-fix-IDX_instance_host_key.js @@ -0,0 +1,22 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +export class FixIDXInstanceHostKey1748990662839 { + async up(queryRunner) { + // must include host for index-only scans: https://www.postgresql.org/docs/current/indexes-index-only-scans.html + await queryRunner.query(`DROP INDEX "public"."IDX_instance_host_key"`); + await queryRunner.query(` + create index "IDX_instance_host_key" + on "instance" ((lower(reverse("host"::text)) || '.'::text) text_pattern_ops) + include ("host") + `); + await queryRunner.query(`comment on index "IDX_instance_host_key" is 'Expression index for finding instances by base domain'`); + } + + async down(queryRunner) { + await queryRunner.query(`DROP INDEX "public"."IDX_instance_host_key"`); + await queryRunner.query(`CREATE UNIQUE INDEX "IDX_instance_host_key" ON "instance" (((lower(reverse("host")) || '.')::text) text_pattern_ops)`); + } +} diff --git a/packages/backend/migration/1748991828473-create-IDX_note_for_timelines.js b/packages/backend/migration/1748991828473-create-IDX_note_for_timelines.js new file mode 100644 index 0000000000..2ea7fe95d2 --- /dev/null +++ b/packages/backend/migration/1748991828473-create-IDX_note_for_timelines.js @@ -0,0 +1,19 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +export class CreateIDXNoteForTimelines1748991828473 { + async up(queryRunner) { + await queryRunner.query(` + create index "IDX_note_for_timelines" + on "note" ("id" desc, "channelId", "visibility", "userHost") + include ("userId", "userHost", "replyId", "replyUserId", "replyUserHost", "renoteId", "renoteUserId", "renoteUserHost") + NULLS NOT DISTINCT`); + await queryRunner.query(`comment on index "IDX_note_for_timelines" is 'Covering index for timeline queries'`); + } + + async down(queryRunner) { + await queryRunner.query(`DROP INDEX "IDX_note_for_timelines"`); + } +} diff --git a/packages/backend/migration/1748992017688-create-IDX_instance_host_filters.js b/packages/backend/migration/1748992017688-create-IDX_instance_host_filters.js new file mode 100644 index 0000000000..76cf16a6de --- /dev/null +++ b/packages/backend/migration/1748992017688-create-IDX_instance_host_filters.js @@ -0,0 +1,17 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +export class CreateIDXInstanceHostFilters1748992017688 { + async up(queryRunner) { + await queryRunner.query(` + create index "IDX_instance_host_filters" + on "instance" ("host", "isBlocked", "isSilenced", "isMediaSilenced", "isAllowListed", "isBubbled", "suspensionState")`); + await queryRunner.query(`comment on index "IDX_instance_host_filters" is 'Covering index for host filter queries'`); + } + + async down(queryRunner) { + await queryRunner.query(`DROP INDEX "IDX_instance_host_filters"`); + } +} diff --git a/packages/backend/migration/1748992128683-create-statistics.js b/packages/backend/migration/1748992128683-create-statistics.js new file mode 100644 index 0000000000..5d08868536 --- /dev/null +++ b/packages/backend/migration/1748992128683-create-statistics.js @@ -0,0 +1,28 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +export class CreateStatistics1748992128683 { + async up(queryRunner) { + await queryRunner.query(`CREATE STATISTICS "STTS_instance_isBlocked_isBubbled" (mcv) ON "isBlocked", "isBubbled" FROM "instance"`); + await queryRunner.query(`CREATE STATISTICS "STTS_instance_isBlocked_isSilenced" (mcv) ON "isBlocked", "isSilenced" FROM "instance"`); + await queryRunner.query(`CREATE STATISTICS "STTS_note_replyId_replyUserId_replyUserHost" (dependencies) ON "replyId", "replyUserId", "replyUserHost" FROM "note"`) + await queryRunner.query(`CREATE STATISTICS "STTS_note_renoteId_renoteUserId_renoteUserHost" (dependencies) ON "renoteId", "renoteUserId", "renoteUserHost" FROM "note"`); + await queryRunner.query(`CREATE STATISTICS "STTS_note_userId_userHost" (mcv) ON "userId", "userHost" FROM "note"`); + await queryRunner.query(`CREATE STATISTICS "STTS_note_replyUserId_replyUserHost" (mcv) ON "replyUserId", "replyUserHost" FROM "note"`); + await queryRunner.query(`CREATE STATISTICS "STTS_note_renoteUserId_renoteUserHost" (mcv) ON "renoteUserId", "renoteUserHost" FROM "note"`); + await queryRunner.query(`ANALYZE "note", "instance"`); + } + + async down(queryRunner) { + await queryRunner.query(`DROP STATISTICS "STTS_instance_isBlocked_isBubbled"`); + await queryRunner.query(`DROP STATISTICS "STTS_instance_isBlocked_isSilenced"`); + await queryRunner.query(`DROP STATISTICS "STTS_note_replyId_replyUserId_replyUserHost"`); + await queryRunner.query(`DROP STATISTICS "STTS_note_renoteId_renoteUserId_renoteUserHost"`); + await queryRunner.query(`DROP STATISTICS "STTS_note_userId_userHost"`); + await queryRunner.query(`DROP STATISTICS "STTS_note_replyUserId_replyUserHost"`); + await queryRunner.query(`DROP STATISTICS "STTS_note_renoteUserId_renoteUserHost"`); + await queryRunner.query(`ANALYZE "note", "instance"`); + } +} diff --git a/packages/backend/migration/1749097536193-fix-IDX_note_for_timeline.js b/packages/backend/migration/1749097536193-fix-IDX_note_for_timeline.js new file mode 100644 index 0000000000..9a651e5871 --- /dev/null +++ b/packages/backend/migration/1749097536193-fix-IDX_note_for_timeline.js @@ -0,0 +1,28 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +export class FixIDXNoteForTimeline1749097536193 { + async up(queryRunner) { + await queryRunner.query('drop index "IDX_note_for_timelines"'); + await queryRunner.query(` + create index "IDX_note_for_timelines" + on "note" ("id" desc, "channelId", "visibility", "userHost") + include ("userId", "replyId", "replyUserId", "replyUserHost", "renoteId", "renoteUserId", "renoteUserHost", "threadId") + NULLS NOT DISTINCT + `); + await queryRunner.query(`comment on index "IDX_note_for_timelines" is 'Covering index for timeline queries'`); + } + + async down(queryRunner) { + await queryRunner.query('drop index "IDX_note_for_timelines"'); + await queryRunner.query(` + create index "IDX_note_for_timelines" + on "note" ("id" desc, "channelId", "visibility", "userHost") + include ("userId", "userHost", "replyId", "replyUserId", "replyUserHost", "renoteId", "renoteUserId", "renoteUserHost") + NULLS NOT DISTINCT + `); + await queryRunner.query(`comment on index "IDX_note_for_timelines" is 'Covering index for timeline queries'`); + } +} diff --git a/packages/backend/migration/1749229288946-create-IDX_note_url.js b/packages/backend/migration/1749229288946-create-IDX_note_url.js new file mode 100644 index 0000000000..4b2fc25cf7 --- /dev/null +++ b/packages/backend/migration/1749229288946-create-IDX_note_url.js @@ -0,0 +1,16 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +export class CreateIDXNoteUrl1749229288946 { + name = 'CreateIDXNoteUrl1749229288946' + + async up(queryRunner) { + await queryRunner.query(`CREATE INDEX IF NOT EXISTS "IDX_note_url" ON "note" ("url") `); + } + + async down(queryRunner) { + await queryRunner.query(`DROP INDEX "public"."IDX_note_url"`); + } +} diff --git a/packages/backend/migration/1749267016885-remove-IDX_instance_host_filters.js b/packages/backend/migration/1749267016885-remove-IDX_instance_host_filters.js new file mode 100644 index 0000000000..d0a4e4f91e --- /dev/null +++ b/packages/backend/migration/1749267016885-remove-IDX_instance_host_filters.js @@ -0,0 +1,17 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +export class RemoveIDXInstanceHostFilters1749267016885 { + async up(queryRunner) { + await queryRunner.query(`DROP INDEX IF EXISTS "IDX_instance_host_filters"`); + } + + async down(queryRunner) { + await queryRunner.query(` + create index "IDX_instance_host_filters" + on "instance" ("host", "isBlocked", "isSilenced", "isMediaSilenced", "isAllowListed", "isBubbled", "suspensionState")`); + await queryRunner.query(`comment on index "IDX_instance_host_filters" is 'Covering index for host filter queries'`); + } +} diff --git a/packages/backend/package.json b/packages/backend/package.json index b9cb0002ab..5ec6ededba 100644 --- a/packages/backend/package.json +++ b/packages/backend/package.json @@ -10,6 +10,9 @@ "start": "node ./built/boot/entry.js", "start:test": "cross-env NODE_ENV=test node ./built/boot/entry.js", "migrate": "pnpm typeorm migration:run -d ormconfig.js", + "migrate:revert": "pnpm typeorm migration:revert -d ormconfig.js", + "migrate:generate": "pnpm typeorm migration:generate -d ormconfig.js", + "migrate:create": "pnpm typeorm migration:create", "revert": "pnpm typeorm migration:revert -d ormconfig.js", "check:connect": "node ./scripts/check_connect.js", "build": "swc src -d built -D --strip-leading-paths", @@ -77,7 +80,7 @@ "@fastify/static": "8.1.1", "@fastify/view": "10.0.2", "@misskey-dev/sharp-read-bmp": "1.3.0", - "@misskey-dev/summaly": "5.2.1", + "@misskey-dev/summaly": "npm:@transfem-org/summaly@5.2.2", "@nestjs/common": "11.1.0", "@nestjs/core": "11.1.0", "@nestjs/testing": "11.1.0", @@ -87,33 +90,30 @@ "@simplewebauthn/server": "12.0.0", "@sinonjs/fake-timers": "11.3.1", "@smithy/node-http-handler": "2.5.0", - "@swc/cli": "0.7.3", - "@swc/core": "1.11.24", - "@transfem-org/sfm-js": "0.24.6", + "mfm-js": "npm:@transfem-org/sfm-js@0.24.6", "@twemoji/parser": "15.1.1", "accepts": "1.3.8", "ajv": "8.17.1", "archiver": "7.0.1", - "argon2": "^0.40.1", + "argon2": "0.43.0", "axios": "1.7.4", - "async-mutex": "0.5.0", "bcryptjs": "2.4.3", "blurhash": "2.0.5", - "body-parser": "1.20.3", "bullmq": "5.51.1", "cacheable-lookup": "7.0.0", - "canvas": "^3.1.0", + "canvas": "3.1.0", "cbor": "9.0.2", "chalk": "5.4.1", "chalk-template": "1.1.0", "cheerio": "1.0.0", - "chokidar": "3.6.0", - "cli-highlight": "2.1.11", + "cli-highlight": "npm:@transfem-org/cli-highlight@2.1.12", "color-convert": "2.0.1", "content-disposition": "0.5.4", "date-fns": "2.30.0", "deep-email-validator": "0.1.21", - "fast-xml-parser": "4.4.1", + "dom-serializer": "2.0.0", + "domhandler": "5.0.3", + "domutils": "3.2.2", "fastify": "5.3.2", "fastify-raw-body": "5.0.0", "feed": "4.2.2", @@ -122,10 +122,9 @@ "form-data": "4.0.2", "glob": "11.0.0", "got": "14.4.7", - "happy-dom": "16.8.1", "hpagent": "1.2.0", "htmlescape": "1.1.1", - "http-link-header": "1.1.3", + "htmlparser2": "9.1.0", "ioredis": "5.6.1", "ip-cidr": "4.0.2", "ipaddr.js": "2.2.0", @@ -133,49 +132,39 @@ "js-yaml": "4.1.0", "json5": "2.2.3", "jsonld": "8.3.3", - "jsrsasign": "11.1.0", "juice": "11.0.1", "megalodon": "workspace:*", "meilisearch": "0.50.0", - "microformats-parser": "2.0.2", "mime-types": "2.1.35", "misskey-js": "workspace:*", "misskey-reversi": "workspace:*", - "moment": "^2.30.1", + "moment": "2.30.1", "ms": "3.0.0-canary.1", "nanoid": "5.1.5", "nested-property": "4.0.0", "node-fetch": "3.3.2", "nodemailer": "6.10.1", - "oauth": "0.10.2", - "oauth2orize": "1.12.0", - "oauth2orize-pkce": "0.1.2", "os-utils": "0.0.14", "otpauth": "9.4.0", - "parse5": "7.3.0", "pg": "8.15.6", "pkce-challenge": "4.1.0", "probe-image-size": "7.2.3", "promise-limit": "2.7.0", - "proxy-addr": "^2.0.7", - "psl": "^1.13.0", + "proxy-addr": "2.0.7", + "psl": "1.15.0", "pug": "3.0.3", "qrcode": "1.5.4", "random-seed": "0.3.0", - "ratelimiter": "3.4.1", "re2": "1.21.4", "redis-info": "3.1.0", "redis-lock": "0.1.4", "reflect-metadata": "0.2.2", "rename": "1.0.4", - "rss-parser": "3.13.0", - "rxjs": "7.8.2", "sanitize-html": "2.16.0", "secure-json-parse": "3.0.2", "sharp": "0.34.1", "slacc": "0.0.10", "strict-event-emitter-types": "2.0.0", - "stringz": "2.1.0", "systeminformation": "5.25.11", "tinycolor2": "1.6.0", "tmp": "0.2.3", @@ -184,7 +173,7 @@ "typeorm": "0.3.22", "typescript": "5.8.3", "ulid": "2.4.0", - "uuid": "^9.0.1", + "uuid": "11.1.0", "vary": "1.1.2", "web-push": "3.6.7", "ws": "8.18.1", @@ -195,16 +184,16 @@ "@nestjs/platform-express": "11.1.0", "@sentry/vue": "9.14.0", "@simplewebauthn/types": "12.0.0", + "@swc/cli": "0.7.3", + "@swc/core": "1.11.24", "@swc/jest": "0.2.38", "@types/accepts": "1.3.7", "@types/archiver": "6.0.3", "@types/bcryptjs": "2.4.6", - "@types/body-parser": "1.19.5", "@types/color-convert": "2.0.4", "@types/content-disposition": "0.5.8", "@types/fluent-ffmpeg": "2.1.27", "@types/htmlescape": "1.1.3", - "@types/http-link-header": "1.0.7", "@types/jest": "29.5.14", "@types/js-yaml": "4.0.9", "@types/jsonld": "1.5.15", @@ -217,12 +206,11 @@ "@types/oauth2orize": "1.11.5", "@types/oauth2orize-pkce": "0.1.2", "@types/pg": "8.11.14", - "@types/proxy-addr": "^2.0.3", - "@types/psl": "^1.1.3", + "@types/proxy-addr": "2.0.3", + "@types/psl": "1.1.3", "@types/pug": "2.0.10", "@types/qrcode": "1.5.5", "@types/random-seed": "0.3.5", - "@types/ratelimiter": "3.4.6", "@types/redis-info": "3.0.3", "@types/rename": "1.0.7", "@types/sanitize-html": "2.15.0", @@ -232,7 +220,6 @@ "@types/supertest": "6.0.3", "@types/tinycolor2": "1.4.6", "@types/tmp": "0.2.6", - "@types/uuid": "^9.0.4", "@types/vary": "1.1.3", "@types/web-push": "3.6.4", "@types/ws": "8.18.1", @@ -241,7 +228,7 @@ "aws-sdk-client-mock": "4.1.0", "cross-env": "7.0.3", "eslint-plugin-import": "2.31.0", - "execa": "8.0.1", + "execa": "9.5.2", "fkill": "9.0.0", "jest": "29.7.0", "jest-mock": "29.7.0", diff --git a/packages/backend/src/boot/entry.ts b/packages/backend/src/boot/entry.ts index 735a0f4666..afb48e526c 100644 --- a/packages/backend/src/boot/entry.ts +++ b/packages/backend/src/boot/entry.ts @@ -9,6 +9,7 @@ import cluster from 'node:cluster'; import { EventEmitter } from 'node:events'; +import { inspect } from 'node:util'; import chalk from 'chalk'; import Xev from 'xev'; import Logger from '@/logger.js'; @@ -53,20 +54,45 @@ async function main() { // Display detail of unhandled promise rejection if (!envOption.quiet) { - process.on('unhandledRejection', console.dir); + process.on('unhandledRejection', e => { + try { + logger.error('Unhandled rejection:', inspect(e)); + } catch { + console.error('Unhandled rejection:', inspect(e)); + } + }); } // Display detail of uncaught exception - process.on('uncaughtException', err => { + process.on('uncaughtExceptionMonitor', ((err, origin) => { try { - logger.error(err); - console.trace(err); - } catch { } - }); + logger.error(`Uncaught exception (${origin}):`, err); + } catch { + console.error(`Uncaught exception (${origin}):`, err); + } + })); // Dying away... + process.on('disconnect', () => { + try { + logger.warn('IPC channel disconnected! The process may soon die.'); + } catch { + console.warn('IPC channel disconnected! The process may soon die.'); + } + }); + process.on('beforeExit', code => { + try { + logger.warn(`Event loop died! Process will exit with code ${code}.`); + } catch { + console.warn(`Event loop died! Process will exit with code ${code}.`); + } + }); process.on('exit', code => { - logger.info(`The process is going to exit with code ${code}`); + try { + logger.info(`The process is going to exit with code ${code}`); + } catch { + console.info(`The process is going to exit with code ${code}`); + } }); //#endregion diff --git a/packages/backend/src/boot/master.ts b/packages/backend/src/boot/master.ts index 538c529106..a90228eabc 100644 --- a/packages/backend/src/boot/master.ts +++ b/packages/backend/src/boot/master.ts @@ -74,7 +74,7 @@ export async function masterMain() { process.exit(1); } - bootLogger.succ('Sharkey initialized'); + bootLogger.info('Sharkey initialized'); if (config.sentryForBackend) { Sentry.init({ @@ -140,10 +140,10 @@ export async function masterMain() { } if (envOption.onlyQueue) { - bootLogger.succ('Queue started', null, true); + bootLogger.info('Queue started', null, true); } else { const addressString = net.isIPv6(config.address) ? `[${config.address}]` : config.address; - bootLogger.succ(config.socket ? `Now listening on socket ${config.socket} on ${config.url}` : `Now listening on ${addressString}:${config.port} on ${config.url}`, null, true); + bootLogger.info(config.socket ? `Now listening on socket ${config.socket} on ${config.url}` : `Now listening on ${addressString}:${config.port} on ${config.url}`, null, true); } } @@ -172,7 +172,7 @@ function loadConfigBoot(): Config { config = loadConfig(); } catch (exception) { if (typeof exception === 'string') { - configLogger.error(exception); + configLogger.error('Exception loading config:', exception); process.exit(1); } else if ((exception as any).code === 'ENOENT') { configLogger.error('Configuration file not found', null, true); @@ -181,7 +181,7 @@ function loadConfigBoot(): Config { throw exception; } - configLogger.succ('Loaded'); + configLogger.info('Loaded'); return config; } @@ -195,7 +195,7 @@ async function connectDb(): Promise<void> { dbLogger.info('Connecting...'); await initDb(); const v = await db.query('SHOW server_version').then(x => x[0].server_version); - dbLogger.succ(`Connected: v${v}`); + dbLogger.info(`Connected: v${v}`); } catch (err) { dbLogger.error('Cannot connect', null, true); dbLogger.error(err); @@ -211,7 +211,7 @@ async function spawnWorkers(limit = 1) { bootLogger.info(`Starting ${workers} worker${workers === 1 ? '' : 's'}...`); await Promise.all([...Array(workers)].map(spawnWorker)); - bootLogger.succ('All workers started'); + bootLogger.info('All workers started'); } function spawnWorker(): Promise<void> { diff --git a/packages/backend/src/config.ts b/packages/backend/src/config.ts index 8507420839..c2e7efd456 100644 --- a/packages/backend/src/config.ts +++ b/packages/backend/src/config.ts @@ -9,6 +9,7 @@ import { dirname, resolve } from 'node:path'; import * as yaml from 'js-yaml'; import { globSync } from 'glob'; import ipaddr from 'ipaddr.js'; +import Logger from './logger.js'; import type * as Sentry from '@sentry/node'; import type * as SentryVue from '@sentry/vue'; import type { RedisOptions } from 'ioredis'; @@ -40,6 +41,7 @@ type Source = { db?: string; user?: string; pass?: string; + slowQueryThreshold?: number; disableCache?: boolean; extra?: { [x: string]: string }; }; @@ -155,6 +157,8 @@ type Source = { } }; +const configLogger = new Logger('config'); + export type PrivateNetworkSource = string | { network?: string, ports?: number[] }; export type PrivateNetwork = { @@ -192,7 +196,7 @@ export function parsePrivateNetworks(patterns: PrivateNetworkSource[] | undefine } } - console.warn('[config] Skipping invalid entry in allowedPrivateNetworks: ', e); + configLogger.warn('Skipping invalid entry in allowedPrivateNetworks: ', e); return null; }) .filter(p => p != null); @@ -222,6 +226,7 @@ export type Config = { db: string; user: string; pass: string; + slowQueryThreshold?: number; disableCache?: boolean; extra?: { [x: string]: string }; }; @@ -375,11 +380,14 @@ export function loadConfig(): Config { if (configFiles.length === 0 && !process.env['MK_WARNED_ABOUT_CONFIG']) { - console.log('No config files loaded, check if this is intentional'); + configLogger.warn('No config files loaded, check if this is intentional'); process.env['MK_WARNED_ABOUT_CONFIG'] = '1'; } - const config = configFiles.map(path => fs.readFileSync(path, 'utf-8')) + const config = configFiles.map(path => { + configLogger.info(`Reading configuration from ${path}`); + return fs.readFileSync(path, 'utf-8'); + }) .map(contents => yaml.load(contents) as Source) .reduce( (acc: Source, cur: Source) => Object.assign(acc, cur), @@ -405,6 +413,10 @@ export function loadConfig(): Config { const internalMediaProxy = `${scheme}://${host}/proxy`; const redis = convertRedisOptions(config.redis, host); + // nullish => 300 (default) + // 0 => undefined (disabled) + const slowQueryThreshold = (config.db.slowQueryThreshold ?? 300) || undefined; + return { version, publishTarballInsteadOfProvideRepositoryUrl: !!config.publishTarballInsteadOfProvideRepositoryUrl, @@ -423,7 +435,7 @@ export function loadConfig(): Config { apiUrl: `${scheme}://${host}/api`, authUrl: `${scheme}://${host}/auth`, driveUrl: `${scheme}://${host}/files`, - db: { ...config.db, db: dbDb, user: dbUser, pass: dbPass }, + db: { ...config.db, db: dbDb, user: dbUser, pass: dbPass, slowQueryThreshold }, dbReplications: config.dbReplications, dbSlaves: config.dbSlaves, fulltextSearch: config.fulltextSearch, @@ -496,6 +508,10 @@ export function loadConfig(): Config { } function tryCreateUrl(url: string) { + if (!url) { + throw new Error('Failed to load: no "url" property found in config. Please check the value of "MISSKEY_CONFIG_DIR" and "MISSKEY_CONFIG_YML", and verify that all configuration files are correct.'); + } + try { return new URL(url); } catch (e) { @@ -627,7 +643,7 @@ function applyEnvOverrides(config: Source) { // these are all the settings that can be overridden _apply_top([['url', 'port', 'address', 'socket', 'chmodSocket', 'disableHsts', 'id', 'dbReplications', 'websocketCompression']]); - _apply_top(['db', ['host', 'port', 'db', 'user', 'pass', 'disableCache']]); + _apply_top(['db', ['host', 'port', 'db', 'user', 'pass', 'slowQueryThreshold', 'disableCache']]); _apply_top(['dbSlaves', Array.from((config.dbSlaves ?? []).keys()), ['host', 'port', 'db', 'user', 'pass']]); _apply_top([ ['redis', 'redisForPubsub', 'redisForJobQueue', 'redisForTimelines', 'redisForReactions', 'redisForRateLimit'], diff --git a/packages/backend/src/core/AbuseReportNotificationService.ts b/packages/backend/src/core/AbuseReportNotificationService.ts index 9bca795479..307f22586e 100644 --- a/packages/backend/src/core/AbuseReportNotificationService.ts +++ b/packages/backend/src/core/AbuseReportNotificationService.ts @@ -83,6 +83,28 @@ export class AbuseReportNotificationService implements OnApplicationShutdown { } /** + * Collects all email addresses that a abuse report should be sent to. + */ + @bindThis + public async getRecipientEMailAddresses(): Promise<string[]> { + const recipientEMailAddresses = await this.fetchEMailRecipients().then(it => it + .filter(it => it.isActive && it.userProfile?.emailVerified) + .map(it => it.userProfile?.email) + .filter(x => x != null), + ); + + if (this.meta.email) { + recipientEMailAddresses.push(this.meta.email); + } + + if (this.meta.maintainerEmail) { + recipientEMailAddresses.push(this.meta.maintainerEmail); + } + + return recipientEMailAddresses; + } + + /** * Mailを用いて{@link abuseReports}の内容を管理者各位に通知する. * メールアドレスの送信先は以下の通り. * - モデレータ権限所有者ユーザ(設定画面からメールアドレスの設定を行っているユーザに限る) @@ -96,15 +118,7 @@ export class AbuseReportNotificationService implements OnApplicationShutdown { return; } - const recipientEMailAddresses = await this.fetchEMailRecipients().then(it => it - .filter(it => it.isActive && it.userProfile?.emailVerified) - .map(it => it.userProfile?.email) - .filter(x => x != null), - ); - - recipientEMailAddresses.push( - ...(this.meta.email ? [this.meta.email] : []), - ); + const recipientEMailAddresses = await this.getRecipientEMailAddresses(); if (recipientEMailAddresses.length <= 0) { return; diff --git a/packages/backend/src/core/AbuseReportService.ts b/packages/backend/src/core/AbuseReportService.ts index 846d2c8ebd..bccb9f86f6 100644 --- a/packages/backend/src/core/AbuseReportService.ts +++ b/packages/backend/src/core/AbuseReportService.ts @@ -13,6 +13,7 @@ import { QueueService } from '@/core/QueueService.js'; import { ApRendererService } from '@/core/activitypub/ApRendererService.js'; import { ModerationLogService } from '@/core/ModerationLogService.js'; import { SystemAccountService } from '@/core/SystemAccountService.js'; +import { IdentifiableError } from '@/misc/identifiable-error.js'; import { IdService } from './IdService.js'; @Injectable() @@ -125,11 +126,11 @@ export class AbuseReportService { const report = await this.abuseUserReportsRepository.findOneByOrFail({ id: reportId }); if (report.targetUserHost == null) { - throw new Error('The target user host is null.'); + throw new IdentifiableError('0b1ce202-b2c1-4ee4-8af4-2742a51b383d', 'The target user host is null.'); } if (report.forwarded) { - throw new Error('The report has already been forwarded.'); + throw new IdentifiableError('5c008bdf-f0e8-4154-9f34-804e114516d7', 'The report has already been forwarded.'); } await this.abuseUserReportsRepository.update(report.id, { diff --git a/packages/backend/src/core/AccountMoveService.ts b/packages/backend/src/core/AccountMoveService.ts index 738026f753..e107f02796 100644 --- a/packages/backend/src/core/AccountMoveService.ts +++ b/packages/backend/src/core/AccountMoveService.ts @@ -26,6 +26,7 @@ import PerUserFollowingChart from '@/core/chart/charts/per-user-following.js'; import { SystemAccountService } from '@/core/SystemAccountService.js'; import { RoleService } from '@/core/RoleService.js'; import { AntennaService } from '@/core/AntennaService.js'; +import { CacheService } from '@/core/CacheService.js'; @Injectable() export class AccountMoveService { @@ -68,6 +69,7 @@ export class AccountMoveService { private systemAccountService: SystemAccountService, private roleService: RoleService, private antennaService: AntennaService, + private readonly cacheService: CacheService, ) { } @@ -107,12 +109,10 @@ export class AccountMoveService { this.globalEventService.publishMainStream(src.id, 'meUpdated', iObj); // Unfollow after 24 hours - const followings = await this.followingsRepository.findBy({ - followerId: src.id, - }); - this.queueService.createDelayedUnfollowJob(followings.map(following => ({ + const followings = await this.cacheService.userFollowingsCache.fetch(src.id); + this.queueService.createDelayedUnfollowJob(Array.from(followings.keys()).map(followeeId => ({ from: { id: src.id }, - to: { id: following.followeeId }, + to: { id: followeeId }, })), process.env.NODE_ENV === 'test' ? 10000 : 1000 * 60 * 60 * 24); await this.postMoveProcess(src, dst); @@ -138,11 +138,9 @@ export class AccountMoveService { // follow the new account const proxy = await this.systemAccountService.fetch('proxy'); - const followings = await this.followingsRepository.findBy({ - followeeId: src.id, - followerHost: IsNull(), // follower is local - followerId: Not(proxy.id), - }); + const followings = await this.cacheService.userFollowersCache.fetch(src.id) + .then(fs => Array.from(fs.values()) + .filter(f => f.followerHost == null && f.followerId !== proxy.id)); const followJobs = followings.map(following => ({ from: { id: following.followerId }, to: { id: dst.id }, @@ -318,9 +316,9 @@ export class AccountMoveService { await this.usersRepository.decrement({ id: In(localFollowerIds) }, 'followingCount', 1); // Decrease follower counts of local followees by 1. - const oldFollowings = await this.followingsRepository.findBy({ followerId: oldAccount.id }); - if (oldFollowings.length > 0) { - await this.usersRepository.decrement({ id: In(oldFollowings.map(following => following.followeeId)) }, 'followersCount', 1); + const oldFollowings = await this.cacheService.userFollowingsCache.fetch(oldAccount.id); + if (oldFollowings.size > 0) { + await this.usersRepository.decrement({ id: In(Array.from(oldFollowings.keys())) }, 'followersCount', 1); } // Update instance stats by decreasing remote followers count by the number of local followers who were following the old account. diff --git a/packages/backend/src/core/AntennaService.ts b/packages/backend/src/core/AntennaService.ts index cf696e3599..667df57943 100644 --- a/packages/backend/src/core/AntennaService.ts +++ b/packages/backend/src/core/AntennaService.ts @@ -130,7 +130,8 @@ export class AntennaService implements OnApplicationShutdown { } if (note.visibility === 'followers') { - const isFollowing = Object.hasOwn(await this.cacheService.userFollowingsCache.fetch(antenna.userId), note.userId); + const followings = await this.cacheService.userFollowingsCache.fetch(antenna.userId); + const isFollowing = followings.has(note.userId); if (!isFollowing && antenna.userId !== note.userId) return false; } diff --git a/packages/backend/src/core/BunnyService.ts b/packages/backend/src/core/BunnyService.ts index bdb5bba3dd..c9f8a427f5 100644 --- a/packages/backend/src/core/BunnyService.ts +++ b/packages/backend/src/core/BunnyService.ts @@ -80,15 +80,15 @@ export class BunnyService { }); req.on('error', (error) => { - this.bunnyCdnLogger.error(error); + this.bunnyCdnLogger.error('Unhandled error', error); data.destroy(); - throw new IdentifiableError('689ee33f-f97c-479a-ac49-1b9f8140bf91', 'An error has occured during the connectiong to BunnyCDN'); + throw new IdentifiableError('689ee33f-f97c-479a-ac49-1b9f8140bf91', 'An error has occurred while connecting to BunnyCDN', true, error); }); data.pipe(req).on('finish', () => { data.destroy(); }); - + // wait till stream gets destroyed upon finish of piping to prevent the UI from showing the upload as success way too early await finished(data); } diff --git a/packages/backend/src/core/CacheService.ts b/packages/backend/src/core/CacheService.ts index 1cf63221f9..2d37cd6bab 100644 --- a/packages/backend/src/core/CacheService.ts +++ b/packages/backend/src/core/CacheService.ts @@ -5,14 +5,16 @@ import { Inject, Injectable } from '@nestjs/common'; import * as Redis from 'ioredis'; -import { IsNull } from 'typeorm'; -import type { BlockingsRepository, FollowingsRepository, MutingsRepository, RenoteMutingsRepository, MiUserProfile, UserProfilesRepository, UsersRepository, MiFollowing, MiNote } from '@/models/_.js'; +import { In, IsNull } from 'typeorm'; +import type { BlockingsRepository, FollowingsRepository, MutingsRepository, RenoteMutingsRepository, MiUserProfile, UserProfilesRepository, UsersRepository, MiNote, MiFollowing } from '@/models/_.js'; import { MemoryKVCache, RedisKVCache } from '@/misc/cache.js'; +import { QuantumKVCache } from '@/misc/QuantumKVCache.js'; import type { MiLocalUser, MiUser } from '@/models/User.js'; import { DI } from '@/di-symbols.js'; import { UserEntityService } from '@/core/entities/UserEntityService.js'; import { bindThis } from '@/decorators.js'; -import type { GlobalEvents } from '@/core/GlobalEventService.js'; +import type { InternalEventTypes } from '@/core/GlobalEventService.js'; +import { InternalEventService } from '@/core/InternalEventService.js'; import type { OnApplicationShutdown } from '@nestjs/common'; export interface FollowStats { @@ -27,7 +29,7 @@ export interface CachedTranslation { text: string | undefined; } -interface CachedTranslationEntity { +export interface CachedTranslationEntity { l?: string; t?: string; u?: number; @@ -39,14 +41,16 @@ export class CacheService implements OnApplicationShutdown { public localUserByNativeTokenCache: MemoryKVCache<MiLocalUser | null>; public localUserByIdCache: MemoryKVCache<MiLocalUser>; public uriPersonCache: MemoryKVCache<MiUser | null>; - public userProfileCache: RedisKVCache<MiUserProfile>; - public userMutingsCache: RedisKVCache<Set<string>>; - public userBlockingCache: RedisKVCache<Set<string>>; - public userBlockedCache: RedisKVCache<Set<string>>; // NOTE: 「被」Blockキャッシュ - public renoteMutingsCache: RedisKVCache<Set<string>>; - public userFollowingsCache: RedisKVCache<Record<string, Pick<MiFollowing, 'withReplies'> | undefined>>; - private readonly userFollowStatsCache = new MemoryKVCache<FollowStats>(1000 * 60 * 10); // 10 minutes - private readonly translationsCache: RedisKVCache<CachedTranslationEntity>; + public userProfileCache: QuantumKVCache<MiUserProfile>; + public userMutingsCache: QuantumKVCache<Set<string>>; + public userBlockingCache: QuantumKVCache<Set<string>>; + public userBlockedCache: QuantumKVCache<Set<string>>; // NOTE: 「被」Blockキャッシュ + public renoteMutingsCache: QuantumKVCache<Set<string>>; + public userFollowingsCache: QuantumKVCache<Map<string, Omit<MiFollowing, 'isFollowerHibernated'>>>; + public userFollowersCache: QuantumKVCache<Map<string, Omit<MiFollowing, 'isFollowerHibernated'>>>; + public hibernatedUserCache: QuantumKVCache<boolean>; + protected userFollowStatsCache = new MemoryKVCache<FollowStats>(1000 * 60 * 10); // 10 minutes + protected translationsCache: RedisKVCache<CachedTranslationEntity>; constructor( @Inject(DI.redis) @@ -74,6 +78,7 @@ export class CacheService implements OnApplicationShutdown { private followingsRepository: FollowingsRepository, private userEntityService: UserEntityService, + private readonly internalEventService: InternalEventService, ) { //this.onMessage = this.onMessage.bind(this); @@ -82,58 +87,148 @@ export class CacheService implements OnApplicationShutdown { this.localUserByIdCache = new MemoryKVCache<MiLocalUser>(1000 * 60 * 5); // 5m this.uriPersonCache = new MemoryKVCache<MiUser | null>(1000 * 60 * 5); // 5m - this.userProfileCache = new RedisKVCache<MiUserProfile>(this.redisClient, 'userProfile', { + this.userProfileCache = new QuantumKVCache(this.internalEventService, 'userProfile', { lifetime: 1000 * 60 * 30, // 30m - memoryCacheLifetime: 1000 * 60, // 1m fetcher: (key) => this.userProfilesRepository.findOneByOrFail({ userId: key }), - toRedisConverter: (value) => JSON.stringify(value), - fromRedisConverter: (value) => JSON.parse(value), // TODO: date型の考慮 + bulkFetcher: userIds => this.userProfilesRepository.findBy({ userId: In(userIds) }).then(ps => ps.map(p => [p.userId, p])), }); - this.userMutingsCache = new RedisKVCache<Set<string>>(this.redisClient, 'userMutings', { + this.userMutingsCache = new QuantumKVCache<Set<string>>(this.internalEventService, 'userMutings', { lifetime: 1000 * 60 * 30, // 30m - memoryCacheLifetime: 1000 * 60, // 1m fetcher: (key) => this.mutingsRepository.find({ where: { muterId: key }, select: ['muteeId'] }).then(xs => new Set(xs.map(x => x.muteeId))), - toRedisConverter: (value) => JSON.stringify(Array.from(value)), - fromRedisConverter: (value) => new Set(JSON.parse(value)), + bulkFetcher: muterIds => this.mutingsRepository + .createQueryBuilder('muting') + .select('"muting"."muterId"', 'muterId') + .addSelect('array_agg("muting"."muteeId")', 'muteeIds') + .where({ muterId: In(muterIds) }) + .groupBy('muting.muterId') + .getRawMany<{ muterId: string, muteeIds: string[] }>() + .then(ms => ms.map(m => [m.muterId, new Set(m.muteeIds)])), }); - this.userBlockingCache = new RedisKVCache<Set<string>>(this.redisClient, 'userBlocking', { + this.userBlockingCache = new QuantumKVCache<Set<string>>(this.internalEventService, 'userBlocking', { lifetime: 1000 * 60 * 30, // 30m - memoryCacheLifetime: 1000 * 60, // 1m fetcher: (key) => this.blockingsRepository.find({ where: { blockerId: key }, select: ['blockeeId'] }).then(xs => new Set(xs.map(x => x.blockeeId))), - toRedisConverter: (value) => JSON.stringify(Array.from(value)), - fromRedisConverter: (value) => new Set(JSON.parse(value)), + bulkFetcher: blockerIds => this.blockingsRepository + .createQueryBuilder('blocking') + .select('"blocking"."blockerId"', 'blockerId') + .addSelect('array_agg("blocking"."blockeeId")', 'blockeeIds') + .where({ blockerId: In(blockerIds) }) + .groupBy('blocking.blockerId') + .getRawMany<{ blockerId: string, blockeeIds: string[] }>() + .then(ms => ms.map(m => [m.blockerId, new Set(m.blockeeIds)])), }); - this.userBlockedCache = new RedisKVCache<Set<string>>(this.redisClient, 'userBlocked', { + this.userBlockedCache = new QuantumKVCache<Set<string>>(this.internalEventService, 'userBlocked', { lifetime: 1000 * 60 * 30, // 30m - memoryCacheLifetime: 1000 * 60, // 1m fetcher: (key) => this.blockingsRepository.find({ where: { blockeeId: key }, select: ['blockerId'] }).then(xs => new Set(xs.map(x => x.blockerId))), - toRedisConverter: (value) => JSON.stringify(Array.from(value)), - fromRedisConverter: (value) => new Set(JSON.parse(value)), + bulkFetcher: blockeeIds => this.blockingsRepository + .createQueryBuilder('blocking') + .select('"blocking"."blockeeId"', 'blockeeId') + .addSelect('array_agg("blocking"."blockeeId")', 'blockeeIds') + .where({ blockeeId: In(blockeeIds) }) + .groupBy('blocking.blockeeId') + .getRawMany<{ blockeeId: string, blockerIds: string[] }>() + .then(ms => ms.map(m => [m.blockeeId, new Set(m.blockerIds)])), }); - this.renoteMutingsCache = new RedisKVCache<Set<string>>(this.redisClient, 'renoteMutings', { + this.renoteMutingsCache = new QuantumKVCache<Set<string>>(this.internalEventService, 'renoteMutings', { lifetime: 1000 * 60 * 30, // 30m - memoryCacheLifetime: 1000 * 60, // 1m fetcher: (key) => this.renoteMutingsRepository.find({ where: { muterId: key }, select: ['muteeId'] }).then(xs => new Set(xs.map(x => x.muteeId))), - toRedisConverter: (value) => JSON.stringify(Array.from(value)), - fromRedisConverter: (value) => new Set(JSON.parse(value)), + bulkFetcher: muterIds => this.renoteMutingsRepository + .createQueryBuilder('muting') + .select('"muting"."muterId"', 'muterId') + .addSelect('array_agg("muting"."muteeId")', 'muteeIds') + .where({ muterId: In(muterIds) }) + .groupBy('muting.muterId') + .getRawMany<{ muterId: string, muteeIds: string[] }>() + .then(ms => ms.map(m => [m.muterId, new Set(m.muteeIds)])), }); - this.userFollowingsCache = new RedisKVCache<Record<string, Pick<MiFollowing, 'withReplies'> | undefined>>(this.redisClient, 'userFollowings', { + this.userFollowingsCache = new QuantumKVCache<Map<string, Omit<MiFollowing, 'isFollowerHibernated'>>>(this.internalEventService, 'userFollowings', { lifetime: 1000 * 60 * 30, // 30m - memoryCacheLifetime: 1000 * 60, // 1m - fetcher: (key) => this.followingsRepository.find({ where: { followerId: key }, select: ['followeeId', 'withReplies'] }).then(xs => { - const obj: Record<string, Pick<MiFollowing, 'withReplies'> | undefined> = {}; - for (const x of xs) { - obj[x.followeeId] = { withReplies: x.withReplies }; + fetcher: (key) => this.followingsRepository.findBy({ followerId: key }).then(xs => new Map(xs.map(f => [f.followeeId, f]))), + bulkFetcher: followerIds => this.followingsRepository + .findBy({ followerId: In(followerIds) }) + .then(fs => fs + .reduce((groups, f) => { + let group = groups.get(f.followerId); + if (!group) { + group = new Map(); + groups.set(f.followerId, group); + } + group.set(f.followeeId, f); + return groups; + }, new Map<string, Map<string, Omit<MiFollowing, 'isFollowerHibernated'>>>)), + }); + + this.userFollowersCache = new QuantumKVCache<Map<string, Omit<MiFollowing, 'isFollowerHibernated'>>>(this.internalEventService, 'userFollowers', { + lifetime: 1000 * 60 * 30, // 30m + fetcher: followeeId => this.followingsRepository.findBy({ followeeId: followeeId }).then(xs => new Map(xs.map(x => [x.followerId, x]))), + bulkFetcher: followeeIds => this.followingsRepository + .findBy({ followeeId: In(followeeIds) }) + .then(fs => fs + .reduce((groups, f) => { + let group = groups.get(f.followeeId); + if (!group) { + group = new Map(); + groups.set(f.followeeId, group); + } + group.set(f.followerId, f); + return groups; + }, new Map<string, Map<string, Omit<MiFollowing, 'isFollowerHibernated'>>>)), + }); + + this.hibernatedUserCache = new QuantumKVCache<boolean>(this.internalEventService, 'hibernatedUsers', { + lifetime: 1000 * 60 * 30, // 30m + fetcher: async userId => { + const { isHibernated } = await this.usersRepository.findOneOrFail({ + where: { id: userId }, + select: { isHibernated: true }, + }); + return isHibernated; + }, + bulkFetcher: async userIds => { + const results = await this.usersRepository.find({ + where: { id: In(userIds) }, + select: { id: true, isHibernated: true }, + }); + return results.map(({ id, isHibernated }) => [id, isHibernated]); + }, + onChanged: async userIds => { + // We only update local copies since each process will get this event, but we can have user objects in multiple different caches. + // Before doing anything else we must "find" all the objects to update. + const userObjects = new Map<string, MiUser[]>(); + const toUpdate: string[] = []; + for (const uid of userIds) { + const toAdd: MiUser[] = []; + + const localUserById = this.localUserByIdCache.get(uid); + if (localUserById) toAdd.push(localUserById); + + const userById = this.userByIdCache.get(uid); + if (userById) toAdd.push(userById); + + if (toAdd.length > 0) { + toUpdate.push(uid); + userObjects.set(uid, toAdd); + } + } + + // In many cases, we won't have to do anything. + // Skipping the DB fetch ensures that this remains a single-step synchronous process. + if (toUpdate.length > 0) { + const hibernations = await this.usersRepository.find({ where: { id: In(toUpdate) }, select: { id: true, isHibernated: true } }); + for (const { id, isHibernated } of hibernations) { + const users = userObjects.get(id); + if (users) { + for (const u of users) { + u.isHibernated = isHibernated; + } + } + } } - return obj; - }), - toRedisConverter: (value) => JSON.stringify(value), - fromRedisConverter: (value) => JSON.parse(value), + }, }); this.translationsCache = new RedisKVCache<CachedTranslationEntity>(this.redisClient, 'translations', { @@ -143,20 +238,21 @@ export class CacheService implements OnApplicationShutdown { // NOTE: チャンネルのフォロー状況キャッシュはChannelFollowingServiceで行っている - this.redisForSub.on('message', this.onMessage); + this.internalEventService.on('userChangeSuspendedState', this.onUserEvent); + this.internalEventService.on('userChangeDeletedState', this.onUserEvent); + this.internalEventService.on('remoteUserUpdated', this.onUserEvent); + this.internalEventService.on('localUserUpdated', this.onUserEvent); + this.internalEventService.on('userChangeSuspendedState', this.onUserEvent); + this.internalEventService.on('userTokenRegenerated', this.onTokenEvent); + this.internalEventService.on('follow', this.onFollowEvent); + this.internalEventService.on('unfollow', this.onFollowEvent); } @bindThis - private async onMessage(_: string, data: string): Promise<void> { - const obj = JSON.parse(data); - - if (obj.channel === 'internal') { - const { type, body } = obj.message as GlobalEvents['internal']['payload']; - switch (type) { - case 'userChangeSuspendedState': - case 'userChangeDeletedState': - case 'remoteUserUpdated': - case 'localUserUpdated': { + private async onUserEvent<E extends 'userChangeSuspendedState' | 'userChangeDeletedState' | 'remoteUserUpdated' | 'localUserUpdated'>(body: InternalEventTypes[E], _: E, isLocal: boolean): Promise<void> { + { + { + { const user = await this.usersRepository.findOneBy({ id: body.id }); if (user == null) { this.userByIdCache.delete(body.id); @@ -166,6 +262,18 @@ export class CacheService implements OnApplicationShutdown { this.uriPersonCache.delete(k); } } + if (isLocal) { + await Promise.all([ + this.userProfileCache.delete(body.id), + this.userMutingsCache.delete(body.id), + this.userBlockingCache.delete(body.id), + this.userBlockedCache.delete(body.id), + this.renoteMutingsCache.delete(body.id), + this.userFollowingsCache.delete(body.id), + this.userFollowersCache.delete(body.id), + this.hibernatedUserCache.delete(body.id), + ]); + } } else { this.userByIdCache.set(user.id, user); for (const [k, v] of this.uriPersonCache.entries) { @@ -178,20 +286,37 @@ export class CacheService implements OnApplicationShutdown { this.localUserByIdCache.set(user.id, user); } } - break; } - case 'userTokenRegenerated': { + } + } + } + + @bindThis + private async onTokenEvent<E extends 'userTokenRegenerated'>(body: InternalEventTypes[E]): Promise<void> { + { + { + { const user = await this.usersRepository.findOneByOrFail({ id: body.id }) as MiLocalUser; this.localUserByNativeTokenCache.delete(body.oldToken); this.localUserByNativeTokenCache.set(body.newToken, user); - break; } + } + } + } + + @bindThis + private async onFollowEvent<E extends 'follow' | 'unfollow'>(body: InternalEventTypes[E], type: E): Promise<void> { + { + switch (type) { case 'follow': { const follower = this.userByIdCache.get(body.followerId); if (follower) follower.followingCount++; const followee = this.userByIdCache.get(body.followeeId); if (followee) followee.followersCount++; - this.userFollowingsCache.delete(body.followerId); + await Promise.all([ + this.userFollowingsCache.delete(body.followerId), + this.userFollowersCache.delete(body.followeeId), + ]); this.userFollowStatsCache.delete(body.followerId); this.userFollowStatsCache.delete(body.followeeId); break; @@ -201,13 +326,14 @@ export class CacheService implements OnApplicationShutdown { if (follower) follower.followingCount--; const followee = this.userByIdCache.get(body.followeeId); if (followee) followee.followersCount--; - this.userFollowingsCache.delete(body.followerId); + await Promise.all([ + this.userFollowingsCache.delete(body.followerId), + this.userFollowersCache.delete(body.followeeId), + ]); this.userFollowStatsCache.delete(body.followerId); this.userFollowStatsCache.delete(body.followeeId); break; } - default: - break; } } } @@ -299,8 +425,114 @@ export class CacheService implements OnApplicationShutdown { } @bindThis + public async getUsers(userIds: Iterable<string>): Promise<Map<string, MiUser>> { + const users = new Map<string, MiUser>; + + const toFetch: string[] = []; + for (const userId of userIds) { + const fromCache = this.userByIdCache.get(userId); + if (fromCache) { + users.set(userId, fromCache); + } else { + toFetch.push(userId); + } + } + + if (toFetch.length > 0) { + const fetched = await this.usersRepository.findBy({ + id: In(toFetch), + }); + + for (const user of fetched) { + users.set(user.id, user); + this.userByIdCache.set(user.id, user); + } + } + + return users; + } + + @bindThis + public async isFollowing(follower: string | { id: string }, followee: string | { id: string }): Promise<boolean> { + const followerId = typeof(follower) === 'string' ? follower : follower.id; + const followeeId = typeof(followee) === 'string' ? followee : followee.id; + + // This lets us use whichever one is in memory, falling back to DB fetch via userFollowingsCache. + return this.userFollowersCache.get(followeeId)?.has(followerId) + ?? (await this.userFollowingsCache.fetch(followerId)).has(followeeId); + } + + /** + * Returns all hibernated followers. + */ + @bindThis + public async getHibernatedFollowers(followeeId: string): Promise<MiFollowing[]> { + const followers = await this.getFollowersWithHibernation(followeeId); + return followers.filter(f => f.isFollowerHibernated); + } + + /** + * Returns all non-hibernated followers. + */ + @bindThis + public async getNonHibernatedFollowers(followeeId: string): Promise<MiFollowing[]> { + const followers = await this.getFollowersWithHibernation(followeeId); + return followers.filter(f => !f.isFollowerHibernated); + } + + /** + * Returns follower relations with populated isFollowerHibernated. + * If you don't need this field, then please use userFollowersCache directly for reduced overhead. + */ + @bindThis + public async getFollowersWithHibernation(followeeId: string): Promise<MiFollowing[]> { + const followers = await this.userFollowersCache.fetch(followeeId); + const hibernations = await this.hibernatedUserCache.fetchMany(followers.keys()).then(fs => fs.reduce((map, f) => { + map.set(f[0], f[1]); + return map; + }, new Map<string, boolean>)); + return Array.from(followers.values()).map(following => ({ + ...following, + isFollowerHibernated: hibernations.get(following.followerId) ?? false, + })); + } + + /** + * Refreshes follower and following relations for the given user. + */ + @bindThis + public async refreshFollowRelationsFor(userId: string): Promise<void> { + const followings = await this.userFollowingsCache.refresh(userId); + const followees = Array.from(followings.values()).map(f => f.followeeId); + await this.userFollowersCache.deleteMany(followees); + } + + @bindThis + public clear(): void { + this.userByIdCache.clear(); + this.localUserByNativeTokenCache.clear(); + this.localUserByIdCache.clear(); + this.uriPersonCache.clear(); + this.userProfileCache.clear(); + this.userMutingsCache.clear(); + this.userBlockingCache.clear(); + this.userBlockedCache.clear(); + this.renoteMutingsCache.clear(); + this.userFollowingsCache.clear(); + this.userFollowStatsCache.clear(); + this.translationsCache.clear(); + } + + @bindThis public dispose(): void { - this.redisForSub.off('message', this.onMessage); + this.internalEventService.off('userChangeSuspendedState', this.onUserEvent); + this.internalEventService.off('userChangeDeletedState', this.onUserEvent); + this.internalEventService.off('remoteUserUpdated', this.onUserEvent); + this.internalEventService.off('localUserUpdated', this.onUserEvent); + this.internalEventService.off('userChangeSuspendedState', this.onUserEvent); + this.internalEventService.off('userTokenRegenerated', this.onTokenEvent); + this.internalEventService.off('follow', this.onFollowEvent); + this.internalEventService.off('unfollow', this.onFollowEvent); this.userByIdCache.dispose(); this.localUserByNativeTokenCache.dispose(); this.localUserByIdCache.dispose(); diff --git a/packages/backend/src/core/CaptchaService.ts b/packages/backend/src/core/CaptchaService.ts index 13200bf7b3..c526a80aeb 100644 --- a/packages/backend/src/core/CaptchaService.ts +++ b/packages/backend/src/core/CaptchaService.ts @@ -54,7 +54,7 @@ export class CaptchaError extends Error { public readonly cause?: unknown; constructor(code: CaptchaErrorCode, message: string, cause?: unknown) { - super(message); + super(message, cause ? { cause } : undefined); this.code = code; this.cause = cause; this.name = 'CaptchaError'; @@ -117,7 +117,7 @@ export class CaptchaService { } const result = await this.getCaptchaResponse('https://www.recaptcha.net/recaptcha/api/siteverify', secret, response).catch(err => { - throw new CaptchaError(captchaErrorCodes.requestFailed, `recaptcha-request-failed: ${err}`); + throw new CaptchaError(captchaErrorCodes.requestFailed, `recaptcha-request-failed: ${err}`, err); }); if (result.success !== true) { @@ -133,7 +133,7 @@ export class CaptchaService { } const result = await this.getCaptchaResponse('https://hcaptcha.com/siteverify', secret, response).catch(err => { - throw new CaptchaError(captchaErrorCodes.requestFailed, `hcaptcha-request-failed: ${err}`); + throw new CaptchaError(captchaErrorCodes.requestFailed, `hcaptcha-request-failed: ${err}`, err); }); if (result.success !== true) { @@ -209,7 +209,7 @@ export class CaptchaService { } const result = await this.getCaptchaResponse('https://challenges.cloudflare.com/turnstile/v0/siteverify', secret, response).catch(err => { - throw new CaptchaError(captchaErrorCodes.requestFailed, `turnstile-request-failed: ${err}`); + throw new CaptchaError(captchaErrorCodes.requestFailed, `turnstile-request-failed: ${err}`, err); }); if (result.success !== true) { @@ -386,7 +386,7 @@ export class CaptchaService { this.logger.info(err); const error = err instanceof CaptchaError ? err - : new CaptchaError(captchaErrorCodes.unknown, `unknown error: ${err}`); + : new CaptchaError(captchaErrorCodes.unknown, `unknown error: ${err}`, err); return { success: false, error, diff --git a/packages/backend/src/core/ChannelFollowingService.ts b/packages/backend/src/core/ChannelFollowingService.ts index 12251595e2..430711fef1 100644 --- a/packages/backend/src/core/ChannelFollowingService.ts +++ b/packages/backend/src/core/ChannelFollowingService.ts @@ -9,14 +9,15 @@ import { DI } from '@/di-symbols.js'; import type { ChannelFollowingsRepository } from '@/models/_.js'; import { MiChannel } from '@/models/_.js'; import { IdService } from '@/core/IdService.js'; -import { GlobalEvents, GlobalEventService } from '@/core/GlobalEventService.js'; +import { GlobalEvents, GlobalEventService, InternalEventTypes } from '@/core/GlobalEventService.js'; import { bindThis } from '@/decorators.js'; import type { MiLocalUser } from '@/models/User.js'; -import { RedisKVCache } from '@/misc/cache.js'; +import { QuantumKVCache } from '@/misc/QuantumKVCache.js'; +import { InternalEventService } from './InternalEventService.js'; @Injectable() export class ChannelFollowingService implements OnModuleInit { - public userFollowingChannelsCache: RedisKVCache<Set<string>>; + public userFollowingChannelsCache: QuantumKVCache<Set<string>>; constructor( @Inject(DI.redis) @@ -27,19 +28,18 @@ export class ChannelFollowingService implements OnModuleInit { private channelFollowingsRepository: ChannelFollowingsRepository, private idService: IdService, private globalEventService: GlobalEventService, + private readonly internalEventService: InternalEventService, ) { - this.userFollowingChannelsCache = new RedisKVCache<Set<string>>(this.redisClient, 'userFollowingChannels', { + this.userFollowingChannelsCache = new QuantumKVCache<Set<string>>(this.internalEventService, 'userFollowingChannels', { lifetime: 1000 * 60 * 30, // 30m - memoryCacheLifetime: 1000 * 60, // 1m fetcher: (key) => this.channelFollowingsRepository.find({ where: { followerId: key }, select: ['followeeId'], }).then(xs => new Set(xs.map(x => x.followeeId))), - toRedisConverter: (value) => JSON.stringify(Array.from(value)), - fromRedisConverter: (value) => new Set(JSON.parse(value)), }); - this.redisForSub.on('message', this.onMessage); + this.internalEventService.on('followChannel', this.onMessage); + this.internalEventService.on('unfollowChannel', this.onMessage); } onModuleInit() { @@ -79,18 +79,15 @@ export class ChannelFollowingService implements OnModuleInit { } @bindThis - private async onMessage(_: string, data: string): Promise<void> { - const obj = JSON.parse(data); - - if (obj.channel === 'internal') { - const { type, body } = obj.message as GlobalEvents['internal']['payload']; + private async onMessage<E extends 'followChannel' | 'unfollowChannel'>(body: InternalEventTypes[E], type: E): Promise<void> { + { switch (type) { case 'followChannel': { - this.userFollowingChannelsCache.refresh(body.userId); + await this.userFollowingChannelsCache.delete(body.userId); break; } case 'unfollowChannel': { - this.userFollowingChannelsCache.delete(body.userId); + await this.userFollowingChannelsCache.delete(body.userId); break; } } @@ -99,6 +96,8 @@ export class ChannelFollowingService implements OnModuleInit { @bindThis public dispose(): void { + this.internalEventService.off('followChannel', this.onMessage); + this.internalEventService.off('unfollowChannel', this.onMessage); this.userFollowingChannelsCache.dispose(); } diff --git a/packages/backend/src/core/CoreModule.ts b/packages/backend/src/core/CoreModule.ts index dd8e61d322..6839ba0159 100644 --- a/packages/backend/src/core/CoreModule.ts +++ b/packages/backend/src/core/CoreModule.ts @@ -41,6 +41,7 @@ import { HttpRequestService } from './HttpRequestService.js'; import { IdService } from './IdService.js'; import { ImageProcessingService } from './ImageProcessingService.js'; import { SystemAccountService } from './SystemAccountService.js'; +import { InternalEventService } from './InternalEventService.js'; import { InternalStorageService } from './InternalStorageService.js'; import { MetaService } from './MetaService.js'; import { MfmService } from './MfmService.js'; @@ -186,6 +187,7 @@ const $HashtagService: Provider = { provide: 'HashtagService', useExisting: Hash const $HttpRequestService: Provider = { provide: 'HttpRequestService', useExisting: HttpRequestService }; const $IdService: Provider = { provide: 'IdService', useExisting: IdService }; const $ImageProcessingService: Provider = { provide: 'ImageProcessingService', useExisting: ImageProcessingService }; +const $InternalEventService: Provider = { provide: 'InternalEventService', useExisting: InternalEventService }; const $InternalStorageService: Provider = { provide: 'InternalStorageService', useExisting: InternalStorageService }; const $MetaService: Provider = { provide: 'MetaService', useExisting: MetaService }; const $MfmService: Provider = { provide: 'MfmService', useExisting: MfmService }; @@ -345,6 +347,7 @@ const $SponsorsService: Provider = { provide: 'SponsorsService', useExisting: Sp HttpRequestService, IdService, ImageProcessingService, + InternalEventService, InternalStorageService, MetaService, MfmService, @@ -500,6 +503,7 @@ const $SponsorsService: Provider = { provide: 'SponsorsService', useExisting: Sp $HttpRequestService, $IdService, $ImageProcessingService, + $InternalEventService, $InternalStorageService, $MetaService, $MfmService, @@ -656,6 +660,7 @@ const $SponsorsService: Provider = { provide: 'SponsorsService', useExisting: Sp HttpRequestService, IdService, ImageProcessingService, + InternalEventService, InternalStorageService, MetaService, MfmService, @@ -810,6 +815,7 @@ const $SponsorsService: Provider = { provide: 'SponsorsService', useExisting: Sp $HttpRequestService, $IdService, $ImageProcessingService, + $InternalEventService, $InternalStorageService, $MetaService, $MfmService, diff --git a/packages/backend/src/core/DownloadService.ts b/packages/backend/src/core/DownloadService.ts index 26e60e00b3..cb5bdb6cb7 100644 --- a/packages/backend/src/core/DownloadService.ts +++ b/packages/backend/src/core/DownloadService.ts @@ -18,6 +18,7 @@ import { LoggerService } from '@/core/LoggerService.js'; import type Logger from '@/logger.js'; import { bindThis } from '@/decorators.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; @Injectable() export class DownloadService { @@ -37,7 +38,7 @@ export class DownloadService { public async downloadUrl(url: string, path: string, options: { timeout?: number, operationTimeout?: number, maxSize?: number } = {} ): Promise<{ filename: string; }> { - this.logger.info(`Downloading ${chalk.cyan(url)} to ${chalk.cyanBright(path)} ...`); + this.logger.debug(`Downloading ${chalk.cyan(url)} to ${chalk.cyanBright(path)} ...`); const timeout = options.timeout ?? 30 * 1000; const operationTimeout = options.operationTimeout ?? 60 * 1000; @@ -86,7 +87,7 @@ export class DownloadService { filename = parsed.parameters.filename; } } catch (e) { - this.logger.warn(`Failed to parse content-disposition: ${contentDisposition}`, { stack: e }); + this.logger.warn(`Failed to parse content-disposition ${contentDisposition}: ${renderInlineError(e)}`); } } }).on('downloadProgress', (progress: Got.Progress) => { @@ -100,13 +101,17 @@ export class DownloadService { await stream.pipeline(req, fs.createWriteStream(path)); } catch (e) { if (e instanceof Got.HTTPError) { - throw new StatusError(`${e.response.statusCode} ${e.response.statusMessage}`, e.response.statusCode, e.response.statusMessage); - } else { + throw new StatusError(`download error from ${url}`, e.response.statusCode, e.response.statusMessage, e); + } else if (e instanceof Got.RequestError || e instanceof Got.AbortError) { + throw new Error(String(e), { cause: e }); + } else if (e instanceof Error) { throw e; + } else { + throw new Error(String(e), { cause: e }); } } - this.logger.succ(`Download finished: ${chalk.cyan(url)}`); + this.logger.info(`Download finished: ${chalk.cyan(url)}`); return { filename, @@ -118,7 +123,7 @@ export class DownloadService { // Create temp file const [path, cleanup] = await createTemp(); - this.logger.info(`text file: Temp file is ${path}`); + this.logger.debug(`text file: Temp file is ${path}`); try { // write content at URL to temp file diff --git a/packages/backend/src/core/DriveService.ts b/packages/backend/src/core/DriveService.ts index 82c447baaa..b9be4e3039 100644 --- a/packages/backend/src/core/DriveService.ts +++ b/packages/backend/src/core/DriveService.ts @@ -45,6 +45,7 @@ import { isMimeImage } from '@/misc/is-mime-image.js'; import { ModerationLogService } from '@/core/ModerationLogService.js'; import { UtilityService } from '@/core/UtilityService.js'; import { BunnyService } from '@/core/BunnyService.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; import { LoggerService } from './LoggerService.js'; type AddFileArgs = { @@ -159,6 +160,14 @@ export class DriveService { // thunbnail, webpublic を必要なら生成 const alts = await this.generateAlts(path, type, !file.uri); + if (type && type.startsWith('video/')) { + try { + await this.videoProcessingService.webOptimizeVideo(path, type); + } catch (err) { + this.registerLogger.warn(`Video optimization failed: ${renderInlineError(err)}`); + } + } + if (this.meta.useObjectStorage) { //#region ObjectStorage params let [ext] = (name.match(/\.([a-zA-Z0-9_-]+)$/) ?? ['']); @@ -194,7 +203,7 @@ export class DriveService { //#endregion //#region Uploads - this.registerLogger.info(`uploading original: ${key}`); + this.registerLogger.debug(`uploading original: ${key}`); const uploads = [ this.upload(key, fs.createReadStream(path), type, null, name), ]; @@ -203,7 +212,7 @@ export class DriveService { webpublicKey = `${prefix}webpublic-${randomUUID()}.${alts.webpublic.ext}`; webpublicUrl = `${ baseUrl }/${ webpublicKey }`; - this.registerLogger.info(`uploading webpublic: ${webpublicKey}`); + this.registerLogger.debug(`uploading webpublic: ${webpublicKey}`); uploads.push(this.upload(webpublicKey, alts.webpublic.data, alts.webpublic.type, alts.webpublic.ext, name)); } @@ -211,7 +220,7 @@ export class DriveService { thumbnailKey = `${prefix}thumbnail-${randomUUID()}.${alts.thumbnail.ext}`; thumbnailUrl = `${ baseUrl }/${ thumbnailKey }`; - this.registerLogger.info(`uploading thumbnail: ${thumbnailKey}`); + this.registerLogger.debug(`uploading thumbnail: ${thumbnailKey}`); uploads.push(this.upload(thumbnailKey, alts.thumbnail.data, alts.thumbnail.type, alts.thumbnail.ext, `${name}.thumbnail`)); } @@ -255,11 +264,11 @@ export class DriveService { const [url, thumbnailUrl, webpublicUrl] = await Promise.all(promises); if (thumbnailUrl) { - this.registerLogger.info(`thumbnail stored: ${thumbnailAccessKey}`); + this.registerLogger.debug(`thumbnail stored: ${thumbnailAccessKey}`); } if (webpublicUrl) { - this.registerLogger.info(`web stored: ${webpublicAccessKey}`); + this.registerLogger.debug(`web stored: ${webpublicAccessKey}`); } file.storedInternal = true; @@ -303,7 +312,7 @@ export class DriveService { thumbnail, }; } catch (err) { - this.registerLogger.warn(`GenerateVideoThumbnail failed: ${err}`); + this.registerLogger.warn(`GenerateVideoThumbnail failed: ${renderInlineError(err)}`); return { webpublic: null, thumbnail: null, @@ -336,7 +345,7 @@ export class DriveService { metadata.height && metadata.height <= 2048 ); } catch (err) { - this.registerLogger.warn(`sharp failed: ${err}`); + this.registerLogger.warn(`sharp failed: ${renderInlineError(err)}`); return { webpublic: null, thumbnail: null, @@ -347,7 +356,7 @@ export class DriveService { let webpublic: IImage | null = null; if (generateWeb && !satisfyWebpublic && !isAnimated) { - this.registerLogger.info('creating web image'); + this.registerLogger.debug('creating web image'); try { if (['image/jpeg', 'image/webp', 'image/avif'].includes(type)) { @@ -358,12 +367,12 @@ export class DriveService { this.registerLogger.debug('web image not created (not an required image)'); } } catch (err) { - this.registerLogger.warn('web image not created (an error occurred)', err as Error); + this.registerLogger.warn(`web image not created: ${renderInlineError(err)}`); } } else { - if (satisfyWebpublic) this.registerLogger.info('web image not created (original satisfies webpublic)'); - else if (isAnimated) this.registerLogger.info('web image not created (animated image)'); - else this.registerLogger.info('web image not created (from remote)'); + if (satisfyWebpublic) this.registerLogger.debug('web image not created (original satisfies webpublic)'); + else if (isAnimated) this.registerLogger.debug('web image not created (animated image)'); + else this.registerLogger.debug('web image not created (from remote)'); } // #endregion webpublic @@ -377,7 +386,7 @@ export class DriveService { thumbnail = await this.imageProcessingService.convertSharpToWebp(img, 498, 422); } } catch (err) { - this.registerLogger.warn('thumbnail not created (an error occurred)', err as Error); + this.registerLogger.warn(`Error creating thumbnail: ${renderInlineError(err)}`); } // #endregion thumbnail @@ -411,27 +420,21 @@ export class DriveService { ); if (this.meta.objectStorageSetPublicRead) params.ACL = 'public-read'; - if (this.bunnyService.usingBunnyCDN(this.meta)) { - await this.bunnyService.upload(this.meta, key, stream).catch( - err => { - this.registerLogger.error(`Upload Failed: key = ${key}, filename = ${filename}`, err); - }, - ); - } else { - await this.s3Service.upload(this.meta, params) - .then( - result => { - if ('Bucket' in result) { // CompleteMultipartUploadCommandOutput - this.registerLogger.debug(`Uploaded: ${result.Bucket}/${result.Key} => ${result.Location}`); - } else { // AbortMultipartUploadCommandOutput - this.registerLogger.error(`Upload Result Aborted: key = ${key}, filename = ${filename}`); - } - }) - .catch( - err => { - this.registerLogger.error(`Upload Failed: key = ${key}, filename = ${filename}`, err); - }, - ); + try { + if (this.bunnyService.usingBunnyCDN(this.meta)) { + await this.bunnyService.upload(this.meta, key, stream); + } else { + const result = await this.s3Service.upload(this.meta, params); + if ('Bucket' in result) { // CompleteMultipartUploadCommandOutput + this.registerLogger.debug(`Uploaded: ${result.Bucket}/${result.Key} => ${result.Location}`); + } else { // AbortMultipartUploadCommandOutput + this.registerLogger.error(`Upload Result Aborted: key = ${key}, filename = ${filename}`); + throw new Error('S3 upload aborted'); + } + } + } catch (err) { + this.registerLogger.error(`Upload Failed: key = ${key}, filename = ${filename}: ${renderInlineError(err)}`); + throw err; } } @@ -490,7 +493,6 @@ export class DriveService { }: AddFileArgs): Promise<MiDriveFile> { const userRoleNSFW = user && (await this.roleService.getUserPolicies(user.id)).alwaysMarkNsfw; const info = await this.fileInfoService.getFileInfo(path); - this.registerLogger.info(`${JSON.stringify(info)}`); // detect name const detectedName = correctFilename( @@ -500,6 +502,8 @@ export class DriveService { ext ?? info.type.ext, ); + this.registerLogger.debug(`Detected file info: ${JSON.stringify(info)}`); + if (user && !force) { // Check if there is a file with the same hash const matched = await this.driveFilesRepository.findOneBy({ @@ -508,7 +512,7 @@ export class DriveService { }); if (matched) { - this.registerLogger.info(`file with same hash is found: ${matched.id}`); + this.registerLogger.debug(`file with same hash is found: ${matched.id}`); if (sensitive && !matched.isSensitive) { // The file is federated as sensitive for this time, but was federated as non-sensitive before. // Therefore, update the file to sensitive. @@ -636,14 +640,14 @@ export class DriveService { } catch (err) { // duplicate key error (when already registered) if (isDuplicateKeyValueError(err)) { - this.registerLogger.info(`already registered ${file.uri}`); + this.registerLogger.debug(`already registered ${file.uri}`); file = await this.driveFilesRepository.findOneBy({ uri: file.uri!, userId: user ? user.id : IsNull(), }) as MiDriveFile; } else { - this.registerLogger.error(err as Error); + this.registerLogger.error('Error in drive register', err as Error); throw err; } } @@ -651,7 +655,7 @@ export class DriveService { file = await (this.save(file, path, detectedName, info)); } - this.registerLogger.succ(`drive file has been created ${file.id}`); + this.registerLogger.info(`Created file ${file.id} (${detectedName}) of type ${info.type.mime} for user ${user?.id ?? '<none>'}`); if (user) { this.driveFileEntityService.pack(file, { self: true }).then(packedFile => { @@ -847,7 +851,7 @@ export class DriveService { } } catch (err: any) { if (err.name === 'NoSuchKey') { - this.deleteLogger.warn(`The object storage had no such key to delete: ${key}. Skipping this.`, err as Error); + this.deleteLogger.warn(`The object storage had no such key to delete: ${key}. Skipping this.`); return; } else { throw new Error(`Failed to delete the file from the object storage with the given key: ${key}`, { @@ -884,13 +888,10 @@ export class DriveService { } const driveFile = await this.addFile({ user, path, name, comment, folderId, force, isLink, url, uri, sensitive, requestIp, requestHeaders }); - this.downloaderLogger.succ(`Got: ${driveFile.id}`); + this.downloaderLogger.debug(`Upload succeeded: created file ${driveFile.id}`); return driveFile!; } catch (err) { - this.downloaderLogger.error(`Failed to create drive file: ${err}`, { - url: url, - e: err, - }); + this.downloaderLogger.error(`Failed to create drive file from ${url}: ${renderInlineError(err)}`); throw err; } finally { cleanup(); diff --git a/packages/backend/src/core/FanoutTimelineEndpointService.ts b/packages/backend/src/core/FanoutTimelineEndpointService.ts index af2723e99d..f9cf41e854 100644 --- a/packages/backend/src/core/FanoutTimelineEndpointService.ts +++ b/packages/backend/src/core/FanoutTimelineEndpointService.ts @@ -136,10 +136,10 @@ export class FanoutTimelineEndpointService { const parentFilter = filter; filter = (note) => { if (!ps.ignoreAuthorFromInstanceBlock) { - if (this.utilityService.isBlockedHost(this.meta.blockedHosts, note.userHost)) return false; + if (note.userInstance?.isBlocked) return false; } - if (note.userId !== note.renoteUserId && this.utilityService.isBlockedHost(this.meta.blockedHosts, note.renoteUserHost)) return false; - if (note.userId !== note.replyUserId && this.utilityService.isBlockedHost(this.meta.blockedHosts, note.replyUserHost)) return false; + if (note.userId !== note.renoteUserId && note.renoteUserInstance?.isBlocked) return false; + if (note.userId !== note.replyUserId && note.replyUserInstance?.isBlocked) return false; return parentFilter(note); }; @@ -194,7 +194,10 @@ export class FanoutTimelineEndpointService { .leftJoinAndSelect('note.renote', 'renote') .leftJoinAndSelect('reply.user', 'replyUser') .leftJoinAndSelect('renote.user', 'renoteUser') - .leftJoinAndSelect('note.channel', 'channel'); + .leftJoinAndSelect('note.channel', 'channel') + .leftJoinAndSelect('note.userInstance', 'userInstance') + .leftJoinAndSelect('note.replyUserInstance', 'replyUserInstance') + .leftJoinAndSelect('note.renoteUserInstance', 'renoteUserInstance'); const notes = (await query.getMany()).filter(noteFilter); diff --git a/packages/backend/src/core/FederatedInstanceService.ts b/packages/backend/src/core/FederatedInstanceService.ts index 3f7ed99348..34df10f0ff 100644 --- a/packages/backend/src/core/FederatedInstanceService.ts +++ b/packages/backend/src/core/FederatedInstanceService.ts @@ -5,23 +5,24 @@ import { Inject, Injectable, OnApplicationShutdown } from '@nestjs/common'; import * as Redis from 'ioredis'; -import { QueryFailedError } from 'typeorm'; -import type { InstancesRepository } from '@/models/_.js'; +import type { InstancesRepository, MiMeta } from '@/models/_.js'; import type { MiInstance } from '@/models/Instance.js'; -import { MemoryKVCache, RedisKVCache } from '@/misc/cache.js'; +import { MemoryKVCache } from '@/misc/cache.js'; import { IdService } from '@/core/IdService.js'; import { DI } from '@/di-symbols.js'; import { UtilityService } from '@/core/UtilityService.js'; import { bindThis } from '@/decorators.js'; -import { isDuplicateKeyValueError } from '@/misc/is-duplicate-key-value-error.js'; +import type { GlobalEvents } from '@/core/GlobalEventService.js'; +import { Serialized } from '@/types.js'; +import { diffArrays, diffArraysSimple } from '@/misc/diff-arrays.js'; @Injectable() export class FederatedInstanceService implements OnApplicationShutdown { - public federatedInstanceCache: RedisKVCache<MiInstance | null>; + private readonly federatedInstanceCache: MemoryKVCache<MiInstance | null>; constructor( - @Inject(DI.redis) - private redisClient: Redis.Redis, + @Inject(DI.redisForSub) + private redisForSub: Redis.Redis, @Inject(DI.instancesRepository) private instancesRepository: InstancesRepository, @@ -29,67 +30,46 @@ export class FederatedInstanceService implements OnApplicationShutdown { private utilityService: UtilityService, private idService: IdService, ) { - this.federatedInstanceCache = new RedisKVCache<MiInstance | null>(this.redisClient, 'federatedInstance', { - lifetime: 1000 * 60 * 30, // 30m - memoryCacheLifetime: 1000 * 60 * 3, // 3m - fetcher: (key) => this.instancesRepository.findOneBy({ host: key }), - toRedisConverter: (value) => JSON.stringify(value), - fromRedisConverter: (value) => { - const parsed = JSON.parse(value); - if (parsed == null) return null; - return { - ...parsed, - firstRetrievedAt: new Date(parsed.firstRetrievedAt), - latestRequestReceivedAt: parsed.latestRequestReceivedAt ? new Date(parsed.latestRequestReceivedAt) : null, - infoUpdatedAt: parsed.infoUpdatedAt ? new Date(parsed.infoUpdatedAt) : null, - notRespondingSince: parsed.notRespondingSince ? new Date(parsed.notRespondingSince) : null, - }; - }, - }); + this.federatedInstanceCache = new MemoryKVCache(1000 * 60 * 3); // 3m + this.redisForSub.on('message', this.onMessage); } @bindThis public async fetchOrRegister(host: string): Promise<MiInstance> { host = this.utilityService.toPuny(host); - const cached = await this.federatedInstanceCache.get(host); + const cached = this.federatedInstanceCache.get(host); if (cached) return cached; - const index = await this.instancesRepository.findOneBy({ host }); - + let index = await this.instancesRepository.findOneBy({ host }); if (index == null) { - let i; - try { - i = await this.instancesRepository.insertOne({ + await this.instancesRepository.createQueryBuilder('instance') + .insert() + .values({ id: this.idService.gen(), host, firstRetrievedAt: new Date(), - }); - } catch (e: unknown) { - if (e instanceof QueryFailedError) { - if (isDuplicateKeyValueError(e)) { - i = await this.instancesRepository.findOneBy({ host }); - } - } + isBlocked: this.utilityService.isBlockedHost(host), + isSilenced: this.utilityService.isSilencedHost(host), + isMediaSilenced: this.utilityService.isMediaSilencedHost(host), + isAllowListed: this.utilityService.isAllowListedHost(host), + isBubbled: this.utilityService.isBubbledHost(host), + }) + .orIgnore() + .execute(); - if (i == null) { - throw e; - } - } - - this.federatedInstanceCache.set(host, i); - return i; - } else { - this.federatedInstanceCache.set(host, index); - return index; + index = await this.instancesRepository.findOneByOrFail({ host }); } + + this.federatedInstanceCache.set(host, index); + return index; } @bindThis public async fetch(host: string): Promise<MiInstance | null> { host = this.utilityService.toPuny(host); - const cached = await this.federatedInstanceCache.get(host); + const cached = this.federatedInstanceCache.get(host); if (cached !== undefined) return cached; const index = await this.instancesRepository.findOneBy({ host }); @@ -117,8 +97,35 @@ export class FederatedInstanceService implements OnApplicationShutdown { this.federatedInstanceCache.set(result.host, result); } + private syncCache(before: Serialized<MiMeta | undefined>, after: Serialized<MiMeta>): void { + const changed = + diffArraysSimple(before?.blockedHosts, after.blockedHosts) || + diffArraysSimple(before?.silencedHosts, after.silencedHosts) || + diffArraysSimple(before?.mediaSilencedHosts, after.mediaSilencedHosts) || + diffArraysSimple(before?.federationHosts, after.federationHosts) || + diffArraysSimple(before?.bubbleInstances, after.bubbleInstances); + + if (changed) { + // We have to clear the whole thing, otherwise subdomains won't be synced. + this.federatedInstanceCache.clear(); + } + } + + @bindThis + private async onMessage(_: string, data: string): Promise<void> { + const obj = JSON.parse(data); + + if (obj.channel === 'internal') { + const { type, body } = obj.message as GlobalEvents['internal']['payload']; + if (type === 'metaUpdated') { + this.syncCache(body.before, body.after); + } + } + } + @bindThis public dispose(): void { + this.redisForSub.off('message', this.onMessage); this.federatedInstanceCache.dispose(); } diff --git a/packages/backend/src/core/FetchInstanceMetadataService.ts b/packages/backend/src/core/FetchInstanceMetadataService.ts index 980f1fcacf..6fcfdfb596 100644 --- a/packages/backend/src/core/FetchInstanceMetadataService.ts +++ b/packages/backend/src/core/FetchInstanceMetadataService.ts @@ -7,7 +7,7 @@ import { URL } from 'node:url'; import { Inject, Injectable } from '@nestjs/common'; import tinycolor from 'tinycolor2'; import * as Redis from 'ioredis'; -import { load as cheerio } from 'cheerio'; +import { load as cheerio } from 'cheerio/slim'; import type { MiInstance } from '@/models/Instance.js'; import type Logger from '@/logger.js'; import { DI } from '@/di-symbols.js'; @@ -15,7 +15,8 @@ import { LoggerService } from '@/core/LoggerService.js'; import { HttpRequestService } from '@/core/HttpRequestService.js'; import { bindThis } from '@/decorators.js'; import { FederatedInstanceService } from '@/core/FederatedInstanceService.js'; -import type { CheerioAPI } from 'cheerio'; +import { renderInlineError } from '@/misc/render-inline-error.js'; +import type { CheerioAPI } from 'cheerio/slim'; type NodeInfo = { openRegistrations?: unknown; @@ -90,7 +91,7 @@ export class FetchInstanceMetadataService { } } - this.logger.info(`Fetching metadata of ${instance.host} ...`); + this.logger.debug(`Fetching metadata of ${instance.host} ...`); const [info, dom, manifest] = await Promise.all([ this.fetchNodeinfo(instance).catch(() => null), @@ -106,7 +107,7 @@ export class FetchInstanceMetadataService { this.getDescription(info, dom, manifest).catch(() => null), ]); - this.logger.succ(`Successfuly fetched metadata of ${instance.host}`); + this.logger.debug(`Successfuly fetched metadata of ${instance.host}`); const updates = { infoUpdatedAt: new Date(), @@ -128,9 +129,9 @@ export class FetchInstanceMetadataService { await this.federatedInstanceService.update(instance.id, updates); - this.logger.succ(`Successfuly updated metadata of ${instance.host}`); + this.logger.info(`Successfully updated metadata of ${instance.host}`); } catch (e) { - this.logger.error(`Failed to update metadata of ${instance.host}: ${e}`); + this.logger.error(`Failed to update metadata of ${instance.host}: ${renderInlineError(e)}`); } finally { await this.unlock(host); } @@ -138,7 +139,7 @@ export class FetchInstanceMetadataService { @bindThis private async fetchNodeinfo(instance: MiInstance): Promise<NodeInfo> { - this.logger.info(`Fetching nodeinfo of ${instance.host} ...`); + this.logger.debug(`Fetching nodeinfo of ${instance.host} ...`); try { const wellknown = await this.httpRequestService.getJson('https://' + instance.host + '/.well-known/nodeinfo') @@ -170,11 +171,11 @@ export class FetchInstanceMetadataService { throw err.statusCode ?? err.message; }); - this.logger.succ(`Successfuly fetched nodeinfo of ${instance.host}`); + this.logger.debug(`Successfuly fetched nodeinfo of ${instance.host}`); return info as NodeInfo; } catch (err) { - this.logger.error(`Failed to fetch nodeinfo of ${instance.host}: ${err}`); + this.logger.warn(`Failed to fetch nodeinfo of ${instance.host}: ${renderInlineError(err)}`); throw err; } @@ -182,7 +183,7 @@ export class FetchInstanceMetadataService { @bindThis private async fetchDom(instance: MiInstance): Promise<CheerioAPI> { - this.logger.info(`Fetching HTML of ${instance.host} ...`); + this.logger.debug(`Fetching HTML of ${instance.host} ...`); const url = 'https://' + instance.host; diff --git a/packages/backend/src/core/FileInfoService.ts b/packages/backend/src/core/FileInfoService.ts index cc66e9fe3a..98fbfe5f23 100644 --- a/packages/backend/src/core/FileInfoService.ts +++ b/packages/backend/src/core/FileInfoService.ts @@ -46,11 +46,13 @@ const TYPE_SVG = { @Injectable() export class FileInfoService { private logger: Logger; + private ffprobeLogger: Logger; constructor( private loggerService: LoggerService, ) { this.logger = this.loggerService.getLogger('file-info'); + this.ffprobeLogger = this.logger.createSubLogger('ffprobe'); } /** @@ -162,20 +164,19 @@ export class FileInfoService { */ @bindThis private hasVideoTrackOnVideoFile(path: string): Promise<boolean> { - const sublogger = this.logger.createSubLogger('ffprobe'); - sublogger.info(`Checking the video file. File path: ${path}`); + this.ffprobeLogger.debug(`Checking the video file. File path: ${path}`); return new Promise((resolve) => { try { FFmpeg.ffprobe(path, (err, metadata) => { if (err) { - sublogger.warn(`Could not check the video file. Returns true. File path: ${path}`, err); + this.ffprobeLogger.warn(`Could not check the video file. Returns true. File path: ${path}`, err); resolve(true); return; } resolve(metadata.streams.some((stream) => stream.codec_type === 'video')); }); } catch (err) { - sublogger.warn(`Could not check the video file. Returns true. File path: ${path}`, err as Error); + this.ffprobeLogger.warn(`Could not check the video file. Returns true. File path: ${path}`, err as Error); resolve(true); } }); diff --git a/packages/backend/src/core/GlobalEventService.ts b/packages/backend/src/core/GlobalEventService.ts index c0027ae129..c146811331 100644 --- a/packages/backend/src/core/GlobalEventService.ts +++ b/packages/backend/src/core/GlobalEventService.ts @@ -265,6 +265,7 @@ export interface InternalEventTypes { unmute: { muterId: MiUser['id']; muteeId: MiUser['id']; }; userListMemberAdded: { userListId: MiUserList['id']; memberId: MiUser['id']; }; userListMemberRemoved: { userListId: MiUserList['id']; memberId: MiUser['id']; }; + quantumCacheUpdated: { name: string, keys: string[] }; } type EventTypesToEventPayload<T> = EventUnionFromDictionary<UndefinedAsNullAll<SerializedAll<T>>>; @@ -353,12 +354,12 @@ export class GlobalEventService { } @bindThis - private publish(channel: StreamChannels, type: string | null, value?: any): void { + private async publish(channel: StreamChannels, type: string | null, value?: any): Promise<void> { const message = type == null ? value : value == null ? { type: type, body: null } : { type: type, body: value }; - this.redisForPub.publish(this.config.host, JSON.stringify({ + await this.redisForPub.publish(this.config.host, JSON.stringify({ channel: channel, message: message, })); @@ -370,6 +371,11 @@ export class GlobalEventService { } @bindThis + public async publishInternalEventAsync<K extends keyof InternalEventTypes>(type: K, value?: InternalEventTypes[K]): Promise<void> { + await this.publish('internal', type, typeof value === 'undefined' ? null : value); + } + + @bindThis public publishBroadcastStream<K extends keyof BroadcastTypes>(type: K, value?: BroadcastTypes[K]): void { this.publish('broadcast', type, typeof value === 'undefined' ? null : value); } diff --git a/packages/backend/src/core/HttpRequestService.ts b/packages/backend/src/core/HttpRequestService.ts index 5c271b81e3..151097095d 100644 --- a/packages/backend/src/core/HttpRequestService.ts +++ b/packages/backend/src/core/HttpRequestService.ts @@ -235,7 +235,7 @@ export class HttpRequestService { } @bindThis - public async getActivityJson(url: string, isLocalAddressAllowed = false): Promise<IObjectWithId> { + public async getActivityJson(url: string, isLocalAddressAllowed = false, allowAnonymous = false): Promise<IObjectWithId> { this.apUtilityService.assertApUrl(url); const res = await this.send(url, { @@ -255,7 +255,11 @@ export class HttpRequestService { // Make sure the object ID matches the final URL (which is where it actually exists). // The caller (ApResolverService) will verify the ID against the original / entry URL, which ensures that all three match. - this.apUtilityService.assertIdMatchesUrlAuthority(activity, res.url); + if (allowAnonymous && activity.id == null) { + activity.id = res.url; + } else { + this.apUtilityService.assertIdMatchesUrlAuthority(activity, res.url); + } return activity as IObjectWithId; } @@ -327,7 +331,7 @@ export class HttpRequestService { }); if (!res.ok && extra.throwErrorWhenResponseNotOk) { - throw new StatusError(`${res.status} ${res.statusText}`, res.status, res.statusText); + throw new StatusError(`request error from ${url}`, res.status, res.statusText); } if (res.ok) { diff --git a/packages/backend/src/core/InternalEventService.ts b/packages/backend/src/core/InternalEventService.ts new file mode 100644 index 0000000000..5b164b605e --- /dev/null +++ b/packages/backend/src/core/InternalEventService.ts @@ -0,0 +1,103 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +import { Inject, Injectable, OnApplicationShutdown } from '@nestjs/common'; +import Redis from 'ioredis'; +import { DI } from '@/di-symbols.js'; +import { GlobalEventService } from '@/core/GlobalEventService.js'; +import type { GlobalEvents, InternalEventTypes } from '@/core/GlobalEventService.js'; +import { bindThis } from '@/decorators.js'; + +export type Listener<K extends keyof InternalEventTypes> = (value: InternalEventTypes[K], key: K, isLocal: boolean) => void | Promise<void>; + +export interface ListenerProps { + ignoreLocal?: boolean, + ignoreRemote?: boolean, +} + +@Injectable() +export class InternalEventService implements OnApplicationShutdown { + private readonly listeners = new Map<keyof InternalEventTypes, Map<Listener<keyof InternalEventTypes>, ListenerProps>>(); + + constructor( + @Inject(DI.redisForSub) + private readonly redisForSub: Redis.Redis, + + private readonly globalEventService: GlobalEventService, + ) { + this.redisForSub.on('message', this.onMessage); + } + + @bindThis + public on<K extends keyof InternalEventTypes>(type: K, listener: Listener<K>, props?: ListenerProps): void { + let set = this.listeners.get(type); + if (!set) { + set = new Map(); + this.listeners.set(type, set); + } + + // Functionally, this is just a set with metadata on the values. + set.set(listener as Listener<keyof InternalEventTypes>, props ?? {}); + } + + @bindThis + public off<K extends keyof InternalEventTypes>(type: K, listener: Listener<K>): void { + this.listeners.get(type)?.delete(listener as Listener<keyof InternalEventTypes>); + } + + @bindThis + public async emit<K extends keyof InternalEventTypes>(type: K, value: InternalEventTypes[K]): Promise<void> { + await this.emitInternal(type, value, true); + await this.globalEventService.publishInternalEventAsync(type, { ...value, _pid: process.pid }); + } + + @bindThis + private async emitInternal<K extends keyof InternalEventTypes>(type: K, value: InternalEventTypes[K], isLocal: boolean): Promise<void> { + const listeners = this.listeners.get(type); + if (!listeners) { + return; + } + + const promises: Promise<void>[] = []; + for (const [listener, props] of listeners) { + if ((isLocal && !props.ignoreLocal) || (!isLocal && !props.ignoreRemote)) { + const promise = Promise.resolve(listener(value, type, isLocal)); + promises.push(promise); + } + } + await Promise.all(promises); + } + + @bindThis + private async onMessage(_: string, data: string): Promise<void> { + const obj = JSON.parse(data); + + if (obj.channel === 'internal') { + const { type, body } = obj.message as GlobalEvents['internal']['payload']; + if (!isLocalInternalEvent(body) || body._pid !== process.pid) { + await this.emitInternal(type, body as InternalEventTypes[keyof InternalEventTypes], false); + } + } + } + + @bindThis + public dispose(): void { + this.redisForSub.off('message', this.onMessage); + this.listeners.clear(); + } + + @bindThis + public onApplicationShutdown(): void { + this.dispose(); + } +} + +interface LocalInternalEvent { + _pid: number; +} + +function isLocalInternalEvent(body: object): body is LocalInternalEvent { + return '_pid' in body && typeof(body._pid) === 'number'; +} diff --git a/packages/backend/src/core/LatestNoteService.ts b/packages/backend/src/core/LatestNoteService.ts index c379805506..63f973c6c6 100644 --- a/packages/backend/src/core/LatestNoteService.ts +++ b/packages/backend/src/core/LatestNoteService.ts @@ -7,6 +7,7 @@ import { DI } from '@/di-symbols.js'; import type { LatestNotesRepository, NotesRepository } from '@/models/_.js'; import { LoggerService } from '@/core/LoggerService.js'; import Logger from '@/logger.js'; +import { QueryService } from './QueryService.js'; @Injectable() export class LatestNoteService { @@ -14,11 +15,12 @@ export class LatestNoteService { constructor( @Inject(DI.notesRepository) - private notesRepository: NotesRepository, + private readonly notesRepository: NotesRepository, @Inject(DI.latestNotesRepository) - private latestNotesRepository: LatestNotesRepository, + private readonly latestNotesRepository: LatestNotesRepository, + private readonly queryService: QueryService, loggerService: LoggerService, ) { this.logger = loggerService.getLogger('LatestNoteService'); @@ -91,7 +93,7 @@ export class LatestNoteService { // Find the newest remaining note for the user. // We exclude DMs and pure renotes. - const nextLatest = await this.notesRepository + const query = this.notesRepository .createQueryBuilder('note') .select() .where({ @@ -106,18 +108,11 @@ export class LatestNoteService { ? Not(null) : null, }) - .andWhere(` - ( - note."renoteId" IS NULL - OR note.text IS NOT NULL - OR note.cw IS NOT NULL - OR note."replyId" IS NOT NULL - OR note."hasPoll" - OR note."fileIds" != '{}' - ) - `) - .orderBy({ id: 'DESC' }) - .getOne(); + .orderBy({ id: 'DESC' }); + + this.queryService.andIsNotRenote(query, 'note'); + + const nextLatest = await query.getOne(); if (!nextLatest) return; // Record it as the latest diff --git a/packages/backend/src/core/MetaService.ts b/packages/backend/src/core/MetaService.ts index 40e7439f5f..07f82dc23e 100644 --- a/packages/backend/src/core/MetaService.ts +++ b/packages/backend/src/core/MetaService.ts @@ -4,7 +4,7 @@ */ import { Inject, Injectable } from '@nestjs/common'; -import { DataSource } from 'typeorm'; +import { DataSource, EntityManager } from 'typeorm'; import * as Redis from 'ioredis'; import { DI } from '@/di-symbols.js'; import { MiMeta } from '@/models/Meta.js'; @@ -12,6 +12,9 @@ import { GlobalEventService } from '@/core/GlobalEventService.js'; import { bindThis } from '@/decorators.js'; import type { GlobalEvents } from '@/core/GlobalEventService.js'; import { FeaturedService } from '@/core/FeaturedService.js'; +import { MiInstance } from '@/models/Instance.js'; +import { diffArrays } from '@/misc/diff-arrays.js'; +import type { MetasRepository } from '@/models/_.js'; import type { OnApplicationShutdown } from '@nestjs/common'; @Injectable() @@ -26,6 +29,9 @@ export class MetaService implements OnApplicationShutdown { @Inject(DI.db) private db: DataSource, + @Inject(DI.metasRepository) + private readonly metasRepository: MetasRepository, + private featuredService: FeaturedService, private globalEventService: GlobalEventService, ) { @@ -67,35 +73,35 @@ export class MetaService implements OnApplicationShutdown { public async fetch(noCache = false): Promise<MiMeta> { if (!noCache && this.cache) return this.cache; - return await this.db.transaction(async transactionalEntityManager => { - // 過去のバグでレコードが複数出来てしまっている可能性があるので新しいIDを優先する - const metas = await transactionalEntityManager.find(MiMeta, { - order: { - id: 'DESC', - }, - }); + // 過去のバグでレコードが複数出来てしまっている可能性があるので新しいIDを優先する + let meta = await this.metasRepository.createQueryBuilder('meta') + .select() + .orderBy({ + id: 'DESC', + }) + .limit(1) + .getOne(); - const meta = metas[0]; + if (!meta) { + await this.metasRepository.createQueryBuilder('meta') + .insert() + .values({ + id: 'x', + }) + .orIgnore() + .execute(); - if (meta) { - this.cache = meta; - return meta; - } else { - // metaが空のときfetchMetaが同時に呼ばれるとここが同時に呼ばれてしまうことがあるのでフェイルセーフなupsertを使う - const saved = await transactionalEntityManager - .upsert( - MiMeta, - { - id: 'x', - }, - ['id'], - ) - .then((x) => transactionalEntityManager.findOneByOrFail(MiMeta, x.identifiers[0])); + meta = await this.metasRepository.createQueryBuilder('meta') + .select() + .orderBy({ + id: 'DESC', + }) + .limit(1) + .getOneOrFail(); + } - this.cache = saved; - return saved; - } - }); + this.cache = meta; + return meta; } @bindThis @@ -103,7 +109,7 @@ export class MetaService implements OnApplicationShutdown { let before: MiMeta | undefined; const updated = await this.db.transaction(async transactionalEntityManager => { - const metas = await transactionalEntityManager.find(MiMeta, { + const metas: (MiMeta | undefined)[] = await transactionalEntityManager.find(MiMeta, { order: { id: 'DESC', }, @@ -126,6 +132,10 @@ export class MetaService implements OnApplicationShutdown { }, }); + // Propagate changes to blockedHosts, silencedHosts, mediaSilencedHosts, federationInstances, and bubbleInstances to the relevant instance rows + // Do this inside the transaction to avoid potential race condition (when an instance gets registered while we're updating). + await this.persistBlocks(transactionalEntityManager, before ?? {}, afters[0]); + return afters[0]; }); @@ -159,4 +169,49 @@ export class MetaService implements OnApplicationShutdown { public onApplicationShutdown(signal?: string | undefined): void { this.dispose(); } + + private async persistBlocks(tem: EntityManager, before: Partial<MiMeta>, after: Partial<MiMeta>): Promise<void> { + await this.persistBlock(tem, before.blockedHosts, after.blockedHosts, 'isBlocked'); + await this.persistBlock(tem, before.silencedHosts, after.silencedHosts, 'isSilenced'); + await this.persistBlock(tem, before.mediaSilencedHosts, after.mediaSilencedHosts, 'isMediaSilenced'); + await this.persistBlock(tem, before.federationHosts, after.federationHosts, 'isAllowListed'); + await this.persistBlock(tem, before.bubbleInstances, after.bubbleInstances, 'isBubbled'); + } + + private async persistBlock(tem: EntityManager, before: string[] | undefined, after: string[] | undefined, field: keyof MiInstance): Promise<void> { + const { added, removed } = diffArrays(before, after); + + if (removed.length > 0) { + await this.updateInstancesByHost(tem, field, false, removed); + } + + if (added.length > 0) { + await this.updateInstancesByHost(tem, field, true, added); + } + } + + private async updateInstancesByHost(tem: EntityManager, field: keyof MiInstance, value: boolean, hosts: string[]): Promise<void> { + // Use non-array queries when possible, as they are indexed and can be much faster. + if (hosts.length === 1) { + const pattern = genHostPattern(hosts[0]); + await tem + .createQueryBuilder(MiInstance, 'instance') + .update() + .set({ [field]: value }) + .where('(lower(reverse("host")) || \'.\') LIKE :pattern', { pattern }) + .execute(); + } else if (hosts.length > 1) { + const patterns = hosts.map(host => genHostPattern(host)); + await tem + .createQueryBuilder(MiInstance, 'instance') + .update() + .set({ [field]: value }) + .where('(lower(reverse("host")) || \'.\') LIKE ANY (:patterns)', { patterns }) + .execute(); + } + } +} + +function genHostPattern(host: string): string { + return host.toLowerCase().split('').reverse().join('') + '.%'; } diff --git a/packages/backend/src/core/MfmService.ts b/packages/backend/src/core/MfmService.ts index 1ee3bd2275..839cdf534c 100644 --- a/packages/backend/src/core/MfmService.ts +++ b/packages/backend/src/core/MfmService.ts @@ -5,25 +5,22 @@ import { URL } from 'node:url'; import { Inject, Injectable } from '@nestjs/common'; -import * as parse5 from 'parse5'; -import { type Document, type HTMLParagraphElement, Window } from 'happy-dom'; +import { isText, isTag, Text } from 'domhandler'; +import * as htmlparser2 from 'htmlparser2'; +import { Node, Document, ChildNode, Element, ParentNode } from 'domhandler'; +import * as domserializer from 'dom-serializer'; import { DI } from '@/di-symbols.js'; import type { Config } from '@/config.js'; import { intersperse } from '@/misc/prelude/array.js'; import { normalizeForSearch } from '@/misc/normalize-for-search.js'; import type { IMentionedRemoteUsers } from '@/models/Note.js'; import { bindThis } from '@/decorators.js'; -import type { DefaultTreeAdapterMap } from 'parse5'; -import type * as mfm from '@transfem-org/sfm-js'; - -const treeAdapter = parse5.defaultTreeAdapter; -type Node = DefaultTreeAdapterMap['node']; -type ChildNode = DefaultTreeAdapterMap['childNode']; +import type * as mfm from 'mfm-js'; const urlRegex = /^https?:\/\/[\w\/:%#@$&?!()\[\]~.,=+\-]+/; const urlRegexFull = /^https?:\/\/[\w\/:%#@$&?!()\[\]~.,=+\-]+$/; -export type Appender = (document: Document, body: HTMLParagraphElement) => void; +export type Appender = (document: Document, body: Element) => void; @Injectable() export class MfmService { @@ -40,7 +37,7 @@ export class MfmService { const normalizedHashtagNames = hashtagNames == null ? undefined : new Set<string>(hashtagNames.map(x => normalizeForSearch(x))); - const dom = parse5.parseFragment(html); + const dom = htmlparser2.parseDocument(html); let text = ''; @@ -51,57 +48,50 @@ export class MfmService { return text.trim(); function getText(node: Node): string { - if (treeAdapter.isTextNode(node)) return node.value; - if (!treeAdapter.isElementNode(node)) return ''; - if (node.nodeName === 'br') return '\n'; - - if (node.childNodes) { - return node.childNodes.map(n => getText(n)).join(''); - } + if (isText(node)) return node.data; + if (!isTag(node)) return ''; + if (node.tagName === 'br') return '\n'; - return ''; + return node.childNodes.map(n => getText(n)).join(''); } function appendChildren(childNodes: ChildNode[]): void { - if (childNodes) { - for (const n of childNodes) { - analyze(n); - } + for (const n of childNodes) { + analyze(n); } } function analyze(node: Node) { - if (treeAdapter.isTextNode(node)) { - text += node.value; + if (isText(node)) { + text += node.data; return; } // Skip comment or document type node - if (!treeAdapter.isElementNode(node)) { + if (!isTag(node)) { return; } - switch (node.nodeName) { + switch (node.tagName) { case 'br': { text += '\n'; - break; + return; } - case 'a': { const txt = getText(node); - const rel = node.attrs.find(x => x.name === 'rel'); - const href = node.attrs.find(x => x.name === 'href'); + const rel = node.attribs.rel; + const href = node.attribs.href; // ハッシュタグ if (normalizedHashtagNames && href && normalizedHashtagNames.has(normalizeForSearch(txt))) { text += txt; // メンション - } else if (txt.startsWith('@') && !(rel && rel.value.startsWith('me '))) { + } else if (txt.startsWith('@') && !(rel && rel.startsWith('me '))) { const part = txt.split('@'); if (part.length === 2 && href) { //#region ホスト名部分が省略されているので復元する - const acct = `${txt}@${(new URL(href.value)).hostname}`; + const acct = `${txt}@${(new URL(href)).hostname}`; text += acct; //#endregion } else if (part.length === 3) { @@ -116,25 +106,32 @@ export class MfmService { if (!href) { return txt; } - if (!txt || txt === href.value) { // #6383: Missing text node - if (href.value.match(urlRegexFull)) { - return href.value; + if (!txt || txt === href) { // #6383: Missing text node + if (href.match(urlRegexFull)) { + return href; } else { - return `<${href.value}>`; + return `<${href}>`; } } - if (href.value.match(urlRegex) && !href.value.match(urlRegexFull)) { - return `[${txt}](<${href.value}>)`; // #6846 + if (href.match(urlRegex) && !href.match(urlRegexFull)) { + return `[${txt}](<${href}>)`; // #6846 } else { - return `[${txt}](${href.value})`; + return `[${txt}](${href})`; } }; text += generateLink(); } - break; + return; } + } + + // Don't produce invalid empty MFM + if (node.childNodes.length < 1) { + return; + } + switch (node.tagName) { case 'h1': { text += '**【'; appendChildren(node.childNodes); @@ -185,14 +182,17 @@ export class MfmService { case 'ruby--': { let ruby: [string, string][] = []; for (const child of node.childNodes) { - if (child.nodeName === 'rp') { + if (isText(child) && !/\s|\[|\]/.test(child.data)) { + ruby.push([child.data, '']); + continue; + } + if (!isTag(child)) { continue; } - if (treeAdapter.isTextNode(child) && !/\s|\[|\]/.test(child.value)) { - ruby.push([child.value, '']); + if (child.tagName === 'rp') { continue; } - if (child.nodeName === 'rt' && ruby.length > 0) { + if (child.tagName === 'rt' && ruby.length > 0) { const rt = getText(child); if (/\s|\[|\]/.test(rt)) { // If any space is included in rt, it is treated as a normal text @@ -217,7 +217,7 @@ export class MfmService { // block code (<pre><code>) case 'pre': { - if (node.childNodes.length === 1 && node.childNodes[0].nodeName === 'code') { + if (node.childNodes.length === 1 && isTag(node.childNodes[0]) && node.childNodes[0].tagName === 'code') { text += '\n```\n'; text += getText(node.childNodes[0]); text += '\n```\n'; @@ -302,17 +302,17 @@ export class MfmService { let nonRtNodes = []; // scan children, ignore `rp`, split on `rt` for (const child of node.childNodes) { - if (treeAdapter.isTextNode(child)) { + if (isText(child)) { nonRtNodes.push(child); continue; } - if (!treeAdapter.isElementNode(child)) { + if (!isTag(child)) { continue; } - if (child.nodeName === 'rp') { + if (child.tagName === 'rp') { continue; } - if (child.nodeName === 'rt') { + if (child.tagName === 'rt') { // the only case in which we don't need a `$[group ]` // is when both sides of the ruby are simple words const needsGroup = nonRtNodes.length > 1 || @@ -350,45 +350,44 @@ export class MfmService { return null; } - const { happyDOM, window } = new Window(); + const doc = new Document([]); - const doc = window.document; + const body = new Element('p', {}); + doc.childNodes.push(body); - const body = doc.createElement('p'); - - function appendChildren(children: mfm.MfmNode[], targetElement: any): void { - if (children) { - for (const child of children.map(x => (handlers as any)[x.type](x))) targetElement.appendChild(child); + function appendChildren(children: mfm.MfmNode[], targetElement: ParentNode): void { + for (const child of children.map(x => handle(x))) { + targetElement.childNodes.push(child); } } function fnDefault(node: mfm.MfmFn) { - const el = doc.createElement('i'); + const el = new Element('i', {}); appendChildren(node.children, el); return el; } - const handlers: { [K in mfm.MfmNode['type']]: (node: mfm.NodeType<K>) => any } = { + const handlers: { [K in mfm.MfmNode['type']]: (node: mfm.NodeType<K>) => ChildNode } = { bold: (node) => { - const el = doc.createElement('b'); + const el = new Element('b', {}); appendChildren(node.children, el); return el; }, small: (node) => { - const el = doc.createElement('small'); + const el = new Element('small', {}); appendChildren(node.children, el); return el; }, strike: (node) => { - const el = doc.createElement('del'); + const el = new Element('del', {}); appendChildren(node.children, el); return el; }, italic: (node) => { - const el = doc.createElement('i'); + const el = new Element('i', {}); appendChildren(node.children, el); return el; }, @@ -399,11 +398,12 @@ export class MfmService { const text = node.children[0].type === 'text' ? node.children[0].props.text : ''; try { const date = new Date(parseInt(text, 10) * 1000); - const el = doc.createElement('time'); - el.setAttribute('datetime', date.toISOString()); - el.textContent = date.toISOString(); + const el = new Element('time', { + datetime: date.toISOString(), + }); + el.childNodes.push(new Text(date.toISOString())); return el; - } catch (err) { + } catch { return fnDefault(node); } } @@ -412,20 +412,20 @@ export class MfmService { if (node.children.length === 1) { const child = node.children[0]; const text = child.type === 'text' ? child.props.text : ''; - const rubyEl = doc.createElement('ruby'); - const rtEl = doc.createElement('rt'); + const rubyEl = new Element('ruby', {}); + const rtEl = new Element('rt', {}); // ruby未対応のHTMLサニタイザーを通したときにルビが「劉備（りゅうび）」となるようにする - const rpStartEl = doc.createElement('rp'); - rpStartEl.appendChild(doc.createTextNode('(')); - const rpEndEl = doc.createElement('rp'); - rpEndEl.appendChild(doc.createTextNode(')')); + const rpStartEl = new Element('rp', {}); + rpStartEl.childNodes.push(new Text('(')); + const rpEndEl = new Element('rp', {}); + rpEndEl.childNodes.push(new Text(')')); - rubyEl.appendChild(doc.createTextNode(text.split(' ')[0])); - rtEl.appendChild(doc.createTextNode(text.split(' ')[1])); - rubyEl.appendChild(rpStartEl); - rubyEl.appendChild(rtEl); - rubyEl.appendChild(rpEndEl); + rubyEl.childNodes.push(new Text(text.split(' ')[0])); + rtEl.childNodes.push(new Text(text.split(' ')[1])); + rubyEl.childNodes.push(rpStartEl); + rubyEl.childNodes.push(rtEl); + rubyEl.childNodes.push(rpEndEl); return rubyEl; } else { const rt = node.children.at(-1); @@ -435,20 +435,20 @@ export class MfmService { } const text = rt.type === 'text' ? rt.props.text : ''; - const rubyEl = doc.createElement('ruby'); - const rtEl = doc.createElement('rt'); + const rubyEl = new Element('ruby', {}); + const rtEl = new Element('rt', {}); // ruby未対応のHTMLサニタイザーを通したときにルビが「劉備（りゅうび）」となるようにする - const rpStartEl = doc.createElement('rp'); - rpStartEl.appendChild(doc.createTextNode('(')); - const rpEndEl = doc.createElement('rp'); - rpEndEl.appendChild(doc.createTextNode(')')); + const rpStartEl = new Element('rp', {}); + rpStartEl.childNodes.push(new Text('(')); + const rpEndEl = new Element('rp', {}); + rpEndEl.childNodes.push(new Text(')')); appendChildren(node.children.slice(0, node.children.length - 1), rubyEl); - rtEl.appendChild(doc.createTextNode(text.trim())); - rubyEl.appendChild(rpStartEl); - rubyEl.appendChild(rtEl); - rubyEl.appendChild(rpEndEl); + rtEl.childNodes.push(new Text(text.trim())); + rubyEl.childNodes.push(rpStartEl); + rubyEl.childNodes.push(rtEl); + rubyEl.childNodes.push(rpEndEl); return rubyEl; } } @@ -456,7 +456,7 @@ export class MfmService { // hack for ruby, should never be needed because we should // never send this out to other instances case 'group': { - const el = doc.createElement('span'); + const el = new Element('span', {}); appendChildren(node.children, el); return el; } @@ -468,125 +468,135 @@ export class MfmService { }, blockCode: (node) => { - const pre = doc.createElement('pre'); - const inner = doc.createElement('code'); - inner.textContent = node.props.code; - pre.appendChild(inner); + const pre = new Element('pre', {}); + const inner = new Element('code', {}); + inner.childNodes.push(new Text(node.props.code)); + pre.childNodes.push(inner); return pre; }, center: (node) => { - const el = doc.createElement('div'); + const el = new Element('div', {}); appendChildren(node.children, el); return el; }, emojiCode: (node) => { - return doc.createTextNode(`\u200B:${node.props.name}:\u200B`); + return new Text(`\u200B:${node.props.name}:\u200B`); }, unicodeEmoji: (node) => { - return doc.createTextNode(node.props.emoji); + return new Text(node.props.emoji); }, hashtag: (node) => { - const a = doc.createElement('a'); - a.setAttribute('href', `${this.config.url}/tags/${node.props.hashtag}`); - a.textContent = `#${node.props.hashtag}`; - a.setAttribute('rel', 'tag'); + const a = new Element('a', { + href: `${this.config.url}/tags/${node.props.hashtag}`, + rel: 'tag', + }); + a.childNodes.push(new Text(`#${node.props.hashtag}`)); return a; }, inlineCode: (node) => { - const el = doc.createElement('code'); - el.textContent = node.props.code; + const el = new Element('code', {}); + el.childNodes.push(new Text(node.props.code)); return el; }, mathInline: (node) => { - const el = doc.createElement('code'); - el.textContent = node.props.formula; + const el = new Element('code', {}); + el.childNodes.push(new Text(node.props.formula)); return el; }, mathBlock: (node) => { - const el = doc.createElement('code'); - el.textContent = node.props.formula; + const el = new Element('code', {}); + el.childNodes.push(new Text(node.props.formula)); return el; }, link: (node) => { - const a = doc.createElement('a'); - a.setAttribute('href', node.props.url); + const a = new Element('a', { + href: node.props.url, + }); appendChildren(node.children, a); return a; }, mention: (node) => { - const a = doc.createElement('a'); const { username, host, acct } = node.props; const remoteUserInfo = mentionedRemoteUsers.find(remoteUser => remoteUser.username.toLowerCase() === username.toLowerCase() && remoteUser.host?.toLowerCase() === host?.toLowerCase()); - a.setAttribute('href', remoteUserInfo - ? (remoteUserInfo.url ? remoteUserInfo.url : remoteUserInfo.uri) - : `${this.config.url}/${acct.endsWith(`@${this.config.url}`) ? acct.substring(0, acct.length - this.config.url.length - 1) : acct}`); - a.className = 'u-url mention'; - a.textContent = acct; + + const a = new Element('a', { + href: remoteUserInfo + ? (remoteUserInfo.url ? remoteUserInfo.url : remoteUserInfo.uri) + : `${this.config.url}/${acct.endsWith(`@${this.config.url}`) ? acct.substring(0, acct.length - this.config.url.length - 1) : acct}`, + class: 'u-url mention', + }); + a.childNodes.push(new Text(acct)); return a; }, quote: (node) => { - const el = doc.createElement('blockquote'); + const el = new Element('blockquote', {}); appendChildren(node.children, el); return el; }, text: (node) => { if (!node.props.text.match(/[\r\n]/)) { - return doc.createTextNode(node.props.text); + return new Text(node.props.text); } - const el = doc.createElement('span'); - const nodes = node.props.text.split(/\r\n|\r|\n/).map(x => doc.createTextNode(x)); + const el = new Element('span', {}); + const nodes = node.props.text.split(/\r\n|\r|\n/).map(x => new Text(x)); for (const x of intersperse<FIXME | 'br'>('br', nodes)) { - el.appendChild(x === 'br' ? doc.createElement('br') : x); + el.childNodes.push(x === 'br' ? new Element('br', {}) : x); } return el; }, url: (node) => { - const a = doc.createElement('a'); - a.setAttribute('href', node.props.url); - a.textContent = node.props.url; + const a = new Element('a', { + href: node.props.url, + }); + a.childNodes.push(new Text(node.props.url)); return a; }, search: (node) => { - const a = doc.createElement('a'); - a.setAttribute('href', `https://www.google.com/search?q=${node.props.query}`); - a.textContent = node.props.content; + const a = new Element('a', { + href: `https://www.google.com/search?q=${node.props.query}`, + }); + a.childNodes.push(new Text(node.props.content)); return a; }, plain: (node) => { - const el = doc.createElement('span'); + const el = new Element('span', {}); appendChildren(node.children, el); return el; }, }; + // Utility function to make TypeScript behave + function handle<T extends mfm.MfmNode>(node: T): ChildNode { + const handler = handlers[node.type] as (node: T) => ChildNode; + return handler(node); + } + appendChildren(nodes, body); for (const additionalAppender of additionalAppenders) { additionalAppender(doc, body); } - const serialized = body.outerHTML; - - happyDOM.close().catch(err => {}); - - return serialized; + return domserializer.render(body, { + encodeEntities: 'utf8' + }); } // the toMastoApiHtml function was taken from Iceshrimp and written by zotan and modified by marie to work with the current MK version @@ -598,55 +608,55 @@ export class MfmService { return null; } - const { happyDOM, window } = new Window(); - - const doc = window.document; + const doc = new Document([]); - const body = doc.createElement('p'); + const body = new Element('p', {}); + doc.childNodes.push(body); - function appendChildren(children: mfm.MfmNode[], targetElement: any): void { - if (children) { - for (const child of children.map((x) => (handlers as any)[x.type](x))) targetElement.appendChild(child); + function appendChildren(children: mfm.MfmNode[], targetElement: ParentNode): void { + for (const child of children) { + const result = handle(child); + targetElement.childNodes.push(result); } } const handlers: { - [K in mfm.MfmNode['type']]: (node: mfm.NodeType<K>) => any; + [K in mfm.MfmNode['type']]: (node: mfm.NodeType<K>) => ChildNode; } = { bold(node) { - const el = doc.createElement('span'); - el.textContent = '**'; + const el = new Element('span', {}); + el.childNodes.push(new Text('**')); appendChildren(node.children, el); - el.textContent += '**'; + el.childNodes.push(new Text('**')); return el; }, small(node) { - const el = doc.createElement('small'); + const el = new Element('small', {}); appendChildren(node.children, el); return el; }, strike(node) { - const el = doc.createElement('span'); - el.textContent = '~~'; + const el = new Element('span', {}); + el.childNodes.push(new Text('~~')); appendChildren(node.children, el); - el.textContent += '~~'; + el.childNodes.push(new Text('~~')); return el; }, italic(node) { - const el = doc.createElement('span'); - el.textContent = '*'; + const el = new Element('span', {}); + el.childNodes.push(new Text('*')); appendChildren(node.children, el); - el.textContent += '*'; + el.childNodes.push(new Text('*')); return el; }, fn(node) { switch (node.props.name) { case 'group': { // hack for ruby - const el = doc.createElement('span'); + const el = new Element('span', {}); appendChildren(node.children, el); return el; } @@ -654,119 +664,121 @@ export class MfmService { if (node.children.length === 1) { const child = node.children[0]; const text = child.type === 'text' ? child.props.text : ''; - const rubyEl = doc.createElement('ruby'); - const rtEl = doc.createElement('rt'); + const rubyEl = new Element('ruby', {}); + const rtEl = new Element('rt', {}); - const rpStartEl = doc.createElement('rp'); - rpStartEl.appendChild(doc.createTextNode('(')); - const rpEndEl = doc.createElement('rp'); - rpEndEl.appendChild(doc.createTextNode(')')); + const rpStartEl = new Element('rp', {}); + rpStartEl.childNodes.push(new Text('(')); + const rpEndEl = new Element('rp', {}); + rpEndEl.childNodes.push(new Text(')')); - rubyEl.appendChild(doc.createTextNode(text.split(' ')[0])); - rtEl.appendChild(doc.createTextNode(text.split(' ')[1])); - rubyEl.appendChild(rpStartEl); - rubyEl.appendChild(rtEl); - rubyEl.appendChild(rpEndEl); + rubyEl.childNodes.push(new Text(text.split(' ')[0])); + rtEl.childNodes.push(new Text(text.split(' ')[1])); + rubyEl.childNodes.push(rpStartEl); + rubyEl.childNodes.push(rtEl); + rubyEl.childNodes.push(rpEndEl); return rubyEl; } else { const rt = node.children.at(-1); if (!rt) { - const el = doc.createElement('span'); + const el = new Element('span', {}); appendChildren(node.children, el); return el; } const text = rt.type === 'text' ? rt.props.text : ''; - const rubyEl = doc.createElement('ruby'); - const rtEl = doc.createElement('rt'); + const rubyEl = new Element('ruby', {}); + const rtEl = new Element('rt', {}); - const rpStartEl = doc.createElement('rp'); - rpStartEl.appendChild(doc.createTextNode('(')); - const rpEndEl = doc.createElement('rp'); - rpEndEl.appendChild(doc.createTextNode(')')); + const rpStartEl = new Element('rp', {}); + rpStartEl.childNodes.push(new Text('(')); + const rpEndEl = new Element('rp', {}); + rpEndEl.childNodes.push(new Text(')')); appendChildren(node.children.slice(0, node.children.length - 1), rubyEl); - rtEl.appendChild(doc.createTextNode(text.trim())); - rubyEl.appendChild(rpStartEl); - rubyEl.appendChild(rtEl); - rubyEl.appendChild(rpEndEl); + rtEl.childNodes.push(new Text(text.trim())); + rubyEl.childNodes.push(rpStartEl); + rubyEl.childNodes.push(rtEl); + rubyEl.childNodes.push(rpEndEl); return rubyEl; } } default: { - const el = doc.createElement('span'); - el.textContent = '*'; + const el = new Element('span', {}); + el.childNodes.push(new Text('*')); appendChildren(node.children, el); - el.textContent += '*'; + el.childNodes.push(new Text('*')); return el; } } }, blockCode(node) { - const pre = doc.createElement('pre'); - const inner = doc.createElement('code'); + const pre = new Element('pre', {}); + const inner = new Element('code', {}); const nodes = node.props.code .split(/\r\n|\r|\n/) - .map((x) => doc.createTextNode(x)); + .map((x) => new Text(x)); for (const x of intersperse<FIXME | 'br'>('br', nodes)) { - inner.appendChild(x === 'br' ? doc.createElement('br') : x); + inner.childNodes.push(x === 'br' ? new Element('br', {}) : x); } - pre.appendChild(inner); + pre.childNodes.push(inner); return pre; }, center(node) { - const el = doc.createElement('div'); + const el = new Element('div', {}); appendChildren(node.children, el); return el; }, emojiCode(node) { - return doc.createTextNode(`\u200B:${node.props.name}:\u200B`); + return new Text(`\u200B:${node.props.name}:\u200B`); }, unicodeEmoji(node) { - return doc.createTextNode(node.props.emoji); + return new Text(node.props.emoji); }, hashtag: (node) => { - const a = doc.createElement('a'); - a.setAttribute('href', `${this.config.url}/tags/${node.props.hashtag}`); - a.textContent = `#${node.props.hashtag}`; - a.setAttribute('rel', 'tag'); - a.setAttribute('class', 'hashtag'); + const a = new Element('a', { + href: `${this.config.url}/tags/${node.props.hashtag}`, + rel: 'tag', + class: 'hashtag', + }); + a.childNodes.push(new Text(`#${node.props.hashtag}`)); return a; }, inlineCode(node) { - const el = doc.createElement('code'); - el.textContent = node.props.code; + const el = new Element('code', {}); + el.childNodes.push(new Text(node.props.code)); return el; }, mathInline(node) { - const el = doc.createElement('code'); - el.textContent = node.props.formula; + const el = new Element('code', {}); + el.childNodes.push(new Text(node.props.formula)); return el; }, mathBlock(node) { - const el = doc.createElement('code'); - el.textContent = node.props.formula; + const el = new Element('code', {}); + el.childNodes.push(new Text(node.props.formula)); return el; }, link(node) { - const a = doc.createElement('a'); - a.setAttribute('rel', 'nofollow noopener noreferrer'); - a.setAttribute('target', '_blank'); - a.setAttribute('href', node.props.url); + const a = new Element('a', { + rel: 'nofollow noopener noreferrer', + target: '_blank', + href: node.props.url, + }); appendChildren(node.children, a); return a; }, @@ -775,92 +787,107 @@ export class MfmService { const { username, host, acct } = node.props; const resolved = mentionedRemoteUsers.find(remoteUser => remoteUser.username === username && remoteUser.host === host); - const el = doc.createElement('span'); + const el = new Element('span', {}); if (!resolved) { - el.textContent = acct; + el.childNodes.push(new Text(acct)); } else { - el.setAttribute('class', 'h-card'); - el.setAttribute('translate', 'no'); - const a = doc.createElement('a'); - a.setAttribute('href', resolved.url ? resolved.url : resolved.uri); - a.className = 'u-url mention'; - const span = doc.createElement('span'); - span.textContent = resolved.username || username; - a.textContent = '@'; - a.appendChild(span); - el.appendChild(a); + el.attribs.class = 'h-card'; + el.attribs.translate = 'no'; + const a = new Element('a', { + href: resolved.url ? resolved.url : resolved.uri, + class: 'u-url mention', + }); + const span = new Element('span', {}); + span.childNodes.push(new Text(resolved.username || username)); + a.childNodes.push(new Text('@')); + a.childNodes.push(span); + el.childNodes.push(a); } return el; }, quote(node) { - const el = doc.createElement('blockquote'); + const el = new Element('blockquote', {}); appendChildren(node.children, el); return el; }, text(node) { - const el = doc.createElement('span'); + if (!node.props.text.match(/[\r\n]/)) { + return new Text(node.props.text); + } + + const el = new Element('span', {}); const nodes = node.props.text .split(/\r\n|\r|\n/) - .map((x) => doc.createTextNode(x)); + .map((x) => new Text(x)); for (const x of intersperse<FIXME | 'br'>('br', nodes)) { - el.appendChild(x === 'br' ? doc.createElement('br') : x); + el.childNodes.push(x === 'br' ? new Element('br', {}) : x); } return el; }, url(node) { - const a = doc.createElement('a'); - a.setAttribute('rel', 'nofollow noopener noreferrer'); - a.setAttribute('target', '_blank'); - a.setAttribute('href', node.props.url); - a.textContent = node.props.url.replace(/^https?:\/\//, ''); + const a = new Element('a', { + rel: 'nofollow noopener noreferrer', + target: '_blank', + href: node.props.url, + }); + a.childNodes.push(new Text(node.props.url.replace(/^https?:\/\//, ''))); return a; }, search: (node) => { - const a = doc.createElement('a'); - a.setAttribute('href', `https://www.google.com/search?q=${node.props.query}`); - a.textContent = node.props.content; + const a = new Element('a', { + href: `https://www.google.com/search?q=${node.props.query}`, + }); + a.childNodes.push(new Text(node.props.content)); return a; }, plain(node) { - const el = doc.createElement('span'); + const el = new Element('span', {}); appendChildren(node.children, el); return el; }, }; + // Utility function to make TypeScript behave + function handle<T extends mfm.MfmNode>(node: T): ChildNode { + const handler = handlers[node.type] as (node: T) => ChildNode; + return handler(node); + } + appendChildren(nodes, body); if (quoteUri !== null) { - const a = doc.createElement('a'); - a.setAttribute('href', quoteUri); - a.textContent = quoteUri.replace(/^https?:\/\//, ''); + const a = new Element('a', { + href: quoteUri, + }); + a.childNodes.push(new Text(quoteUri.replace(/^https?:\/\//, ''))); - const quote = doc.createElement('span'); - quote.setAttribute('class', 'quote-inline'); - quote.appendChild(doc.createElement('br')); - quote.appendChild(doc.createElement('br')); - quote.innerHTML += 'RE: '; - quote.appendChild(a); + const quote = new Element('span', { + class: 'quote-inline', + }); + quote.childNodes.push(new Element('br', {})); + quote.childNodes.push(new Element('br', {})); + quote.childNodes.push(new Text('RE: ')); + quote.childNodes.push(a); - body.appendChild(quote); + body.childNodes.push(quote); } - let result = body.outerHTML; + let result = domserializer.render(body, { + encodeEntities: 'utf8' + }); if (inline) { result = result.replace(/^<p>/, '').replace(/<\/p>$/, ''); } - happyDOM.close().catch(() => {}); - return result; } } diff --git a/packages/backend/src/core/NoteCreateService.ts b/packages/backend/src/core/NoteCreateService.ts index 097d657ba3..f4159facc3 100644 --- a/packages/backend/src/core/NoteCreateService.ts +++ b/packages/backend/src/core/NoteCreateService.ts @@ -4,7 +4,7 @@ */ import { setImmediate } from 'node:timers/promises'; -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; import { In, DataSource, IsNull, LessThan } from 'typeorm'; import * as Redis from 'ioredis'; import { Inject, Injectable, OnApplicationShutdown } from '@nestjs/common'; @@ -296,7 +296,7 @@ export class NoteCreateService implements OnApplicationShutdown { case 'followers': // 他人のfollowers noteはreject if (data.renote.userId !== user.id) { - throw new Error('Renote target is not public or home'); + throw new IdentifiableError('b6352a84-e5cd-4b05-a26c-63437a6b98ba', 'Renote target is not public or home'); } // Renote対象がfollowersならfollowersにする @@ -304,7 +304,7 @@ export class NoteCreateService implements OnApplicationShutdown { break; case 'specified': // specified / direct noteはreject - throw new Error('Renote target is not public or home'); + throw new IdentifiableError('b6352a84-e5cd-4b05-a26c-63437a6b98ba', 'Renote target is not public or home'); } } @@ -317,7 +317,7 @@ export class NoteCreateService implements OnApplicationShutdown { if (data.renote.userId !== user.id) { const blocked = await this.userBlockingService.checkBlocked(data.renote.userId, user.id); if (blocked) { - throw new Error('blocked'); + throw new IdentifiableError('b6352a84-e5cd-4b05-a26c-63437a6b98ba', 'Renote target is blocked'); } } } @@ -489,10 +489,10 @@ export class NoteCreateService implements OnApplicationShutdown { // should really not happen, but better safe than sorry if (data.reply?.id === insert.id) { - throw new Error('A note can\'t reply to itself'); + throw new IdentifiableError('ea93b7c2-3d6c-4e10-946b-00d50b1a75cb', 'A note can\'t reply to itself'); } if (data.renote?.id === insert.id) { - throw new Error('A note can\'t renote itself'); + throw new IdentifiableError('ea93b7c2-3d6c-4e10-946b-00d50b1a75cb', 'A note can\'t renote itself'); } if (data.uri != null) insert.uri = data.uri; @@ -549,8 +549,6 @@ export class NoteCreateService implements OnApplicationShutdown { throw err; } - console.error(e); - throw e; } } @@ -608,11 +606,11 @@ export class NoteCreateService implements OnApplicationShutdown { } if (data.reply == null) { - // TODO: キャッシュ - this.followingsRepository.findBy({ - followeeId: user.id, - notify: 'normal', - }).then(async followings => { + this.cacheService.userFollowersCache.fetch(user.id).then(async followingsMap => { + const followings = Array + .from(followingsMap.values()) + .filter(f => f.notify === 'normal'); + if (note.visibility !== 'specified') { const isPureRenote = this.isRenote(data) && !this.isQuote(data) ? true : false; for (const following of followings) { @@ -733,7 +731,7 @@ export class NoteCreateService implements OnApplicationShutdown { //#region AP deliver if (!data.localOnly && this.userEntityService.isLocalUser(user)) { trackTask(async () => { - const noteActivity = await this.renderNoteOrRenoteActivity(data, note, user); + const noteActivity = await this.apRendererService.renderNoteOrRenoteActivity(note, user, { renote: data.renote }); const dm = this.apDeliverManagerService.createDeliverManager(user, noteActivity); // メンションされたリモートユーザーに配送 @@ -877,17 +875,6 @@ export class NoteCreateService implements OnApplicationShutdown { } @bindThis - private async renderNoteOrRenoteActivity(data: Option, note: MiNote, user: MiUser) { - if (data.localOnly) return null; - - const content = this.isRenote(data) && !this.isQuote(data) - ? this.apRendererService.renderAnnounce(data.renote.uri ? data.renote.uri : `${this.config.url}/notes/${data.renote.id}`, note) - : this.apRendererService.renderCreate(await this.apRendererService.renderNote(note, user, false), note); - - return this.apRendererService.addContext(content); - } - - @bindThis private index(note: MiNote) { if (note.text == null && note.cw == null) return; @@ -950,14 +937,7 @@ export class NoteCreateService implements OnApplicationShutdown { // TODO: キャッシュ？ // eslint-disable-next-line prefer-const let [followings, userListMemberships] = await Promise.all([ - this.followingsRepository.find({ - where: { - followeeId: user.id, - followerHost: IsNull(), - isFollowerHibernated: false, - }, - select: ['followerId', 'withReplies'], - }), + this.cacheService.getNonHibernatedFollowers(user.id), this.userListMembershipsRepository.find({ where: { userId: user.id, @@ -973,6 +953,7 @@ export class NoteCreateService implements OnApplicationShutdown { // TODO: あまりにも数が多いと redisPipeline.exec に失敗する(理由は不明)ため、3万件程度を目安に分割して実行するようにする for (const following of followings) { + if (following.followerHost !== null) continue; // 基本的にvisibleUserIdsには自身のidが含まれている前提であること if (note.visibility === 'specified' && !note.visibleUserIds.some(v => v === following.followerId)) continue; @@ -1074,17 +1055,19 @@ export class NoteCreateService implements OnApplicationShutdown { }); if (hibernatedUsers.length > 0) { - this.usersRepository.update({ - id: In(hibernatedUsers.map(x => x.id)), - }, { - isHibernated: true, - }); - - this.followingsRepository.update({ - followerId: In(hibernatedUsers.map(x => x.id)), - }, { - isFollowerHibernated: true, - }); + await Promise.all([ + this.usersRepository.update({ + id: In(hibernatedUsers.map(x => x.id)), + }, { + isHibernated: true, + }), + this.followingsRepository.update({ + followerId: In(hibernatedUsers.map(x => x.id)), + }, { + isFollowerHibernated: true, + }), + this.cacheService.hibernatedUserCache.setMany(hibernatedUsers.map(x => [x.id, true])), + ]); } } diff --git a/packages/backend/src/core/NoteEditService.ts b/packages/backend/src/core/NoteEditService.ts index 58233b90ee..4be097465d 100644 --- a/packages/backend/src/core/NoteEditService.ts +++ b/packages/backend/src/core/NoteEditService.ts @@ -4,7 +4,7 @@ */ import { setImmediate } from 'node:timers/promises'; -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; import { DataSource, In, IsNull, LessThan } from 'typeorm'; import * as Redis from 'ioredis'; import { Inject, Injectable, OnApplicationShutdown } from '@nestjs/common'; @@ -309,7 +309,7 @@ export class NoteEditService implements OnApplicationShutdown { if (this.isRenote(data)) { if (data.renote.id === oldnote.id) { - throw new UnrecoverableError(`edit failed for ${oldnote.id}: cannot renote itself`); + throw new IdentifiableError('ea93b7c2-3d6c-4e10-946b-00d50b1a75cb', `edit failed for ${oldnote.id}: cannot renote itself`); } switch (data.renote.visibility) { @@ -325,7 +325,7 @@ export class NoteEditService implements OnApplicationShutdown { case 'followers': // 他人のfollowers noteはreject if (data.renote.userId !== user.id) { - throw new Error('Renote target is not public or home'); + throw new IdentifiableError('b6352a84-e5cd-4b05-a26c-63437a6b98ba', 'Renote target is not public or home'); } // Renote対象がfollowersならfollowersにする @@ -333,7 +333,7 @@ export class NoteEditService implements OnApplicationShutdown { break; case 'specified': // specified / direct noteはreject - throw new Error('Renote target is not public or home'); + throw new IdentifiableError('b6352a84-e5cd-4b05-a26c-63437a6b98ba', 'Renote target is not public or home'); } } @@ -675,7 +675,7 @@ export class NoteEditService implements OnApplicationShutdown { //#region AP deliver if (!data.localOnly && this.userEntityService.isLocalUser(user)) { trackTask(async () => { - const noteActivity = await this.renderNoteOrRenoteActivity(data, note, user); + const noteActivity = await this.apRendererService.renderNoteOrRenoteActivity(note, user, { renote: data.renote }); const dm = this.apDeliverManagerService.createDeliverManager(user, noteActivity); // メンションされたリモートユーザーに配送 @@ -771,17 +771,6 @@ export class NoteEditService implements OnApplicationShutdown { } @bindThis - private async renderNoteOrRenoteActivity(data: Option, note: MiNote, user: MiUser) { - if (data.localOnly) return null; - - const content = this.isRenote(data) && !this.isQuote(data) - ? this.apRendererService.renderAnnounce(data.renote.uri ? data.renote.uri : `${this.config.url}/notes/${data.renote.id}`, note) - : this.apRendererService.renderUpdate(await this.apRendererService.renderUpNote(note, user, false), user); - - return this.apRendererService.addContext(content); - } - - @bindThis private index(note: MiNote) { if (note.text == null && note.cw == null) return; @@ -833,14 +822,7 @@ export class NoteEditService implements OnApplicationShutdown { // TODO: キャッシュ？ // eslint-disable-next-line prefer-const let [followings, userListMemberships] = await Promise.all([ - this.followingsRepository.find({ - where: { - followeeId: user.id, - followerHost: IsNull(), - isFollowerHibernated: false, - }, - select: ['followerId', 'withReplies'], - }), + this.cacheService.getNonHibernatedFollowers(user.id), this.userListMembershipsRepository.find({ where: { userId: user.id, @@ -856,6 +838,7 @@ export class NoteEditService implements OnApplicationShutdown { // TODO: あまりにも数が多いと redisPipeline.exec に失敗する(理由は不明)ため、3万件程度を目安に分割して実行するようにする for (const following of followings) { + if (following.followerHost !== null) continue; // 基本的にvisibleUserIdsには自身のidが含まれている前提であること if (note.visibility === 'specified' && !note.visibleUserIds.some(v => v === following.followerId)) continue; @@ -957,17 +940,19 @@ export class NoteEditService implements OnApplicationShutdown { }); if (hibernatedUsers.length > 0) { - this.usersRepository.update({ - id: In(hibernatedUsers.map(x => x.id)), - }, { - isHibernated: true, - }); - - this.followingsRepository.update({ - followerId: In(hibernatedUsers.map(x => x.id)), - }, { - isFollowerHibernated: true, - }); + await Promise.all([ + this.usersRepository.update({ + id: In(hibernatedUsers.map(x => x.id)), + }, { + isHibernated: true, + }), + this.followingsRepository.update({ + followerId: In(hibernatedUsers.map(x => x.id)), + }, { + isFollowerHibernated: true, + }), + this.cacheService.hibernatedUserCache.setMany(hibernatedUsers.map(x => [x.id, true])), + ]); } } diff --git a/packages/backend/src/core/NotePiningService.ts b/packages/backend/src/core/NotePiningService.ts index 86f1a62d4a..a678108189 100644 --- a/packages/backend/src/core/NotePiningService.ts +++ b/packages/backend/src/core/NotePiningService.ts @@ -61,7 +61,7 @@ export class NotePiningService { }); if (note == null) { - throw new IdentifiableError('70c4e51f-5bea-449c-a030-53bee3cce202', 'No such note.'); + throw new IdentifiableError('70c4e51f-5bea-449c-a030-53bee3cce202', `Note ${noteId} does not exist`); } await this.db.transaction(async tem => { @@ -102,7 +102,7 @@ export class NotePiningService { }); if (note == null) { - throw new IdentifiableError('b302d4cf-c050-400a-bbb3-be208681f40c', 'No such note.'); + throw new IdentifiableError('b302d4cf-c050-400a-bbb3-be208681f40c', `Note ${noteId} does not exist`); } this.userNotePiningsRepository.delete({ diff --git a/packages/backend/src/core/NotificationService.ts b/packages/backend/src/core/NotificationService.ts index 0f05f5425d..2ce7bdb5a9 100644 --- a/packages/backend/src/core/NotificationService.ts +++ b/packages/backend/src/core/NotificationService.ts @@ -113,27 +113,27 @@ export class NotificationService implements OnApplicationShutdown { } if (recieveConfig?.type === 'following') { - const isFollowing = await this.cacheService.userFollowingsCache.fetch(notifieeId).then(followings => Object.hasOwn(followings, notifierId)); + const isFollowing = await this.cacheService.userFollowingsCache.fetch(notifieeId).then(followings => followings.has(notifierId)); if (!isFollowing) { return null; } } else if (recieveConfig?.type === 'follower') { - const isFollower = await this.cacheService.userFollowingsCache.fetch(notifierId).then(followings => Object.hasOwn(followings, notifieeId)); + const isFollower = await this.cacheService.userFollowingsCache.fetch(notifierId).then(followings => followings.has(notifieeId)); if (!isFollower) { return null; } } else if (recieveConfig?.type === 'mutualFollow') { const [isFollowing, isFollower] = await Promise.all([ - this.cacheService.userFollowingsCache.fetch(notifieeId).then(followings => Object.hasOwn(followings, notifierId)), - this.cacheService.userFollowingsCache.fetch(notifierId).then(followings => Object.hasOwn(followings, notifieeId)), + this.cacheService.userFollowingsCache.fetch(notifieeId).then(followings => followings.has(notifierId)), + this.cacheService.userFollowingsCache.fetch(notifierId).then(followings => followings.has(notifieeId)), ]); if (!(isFollowing && isFollower)) { return null; } } else if (recieveConfig?.type === 'followingOrFollower') { const [isFollowing, isFollower] = await Promise.all([ - this.cacheService.userFollowingsCache.fetch(notifieeId).then(followings => Object.hasOwn(followings, notifierId)), - this.cacheService.userFollowingsCache.fetch(notifierId).then(followings => Object.hasOwn(followings, notifieeId)), + this.cacheService.userFollowingsCache.fetch(notifieeId).then(followings => followings.has(notifierId)), + this.cacheService.userFollowingsCache.fetch(notifierId).then(followings => followings.has(notifieeId)), ]); if (!isFollowing && !isFollower) { return null; diff --git a/packages/backend/src/core/PushNotificationService.ts b/packages/backend/src/core/PushNotificationService.ts index 9333c1ebc5..e3f10d4504 100644 --- a/packages/backend/src/core/PushNotificationService.ts +++ b/packages/backend/src/core/PushNotificationService.ts @@ -12,7 +12,8 @@ import type { Packed } from '@/misc/json-schema.js'; import { getNoteSummary } from '@/misc/get-note-summary.js'; import type { MiMeta, MiSwSubscription, SwSubscriptionsRepository } from '@/models/_.js'; import { bindThis } from '@/decorators.js'; -import { RedisKVCache } from '@/misc/cache.js'; +import { QuantumKVCache } from '@/misc/QuantumKVCache.js'; +import { InternalEventService } from '@/core/InternalEventService.js'; // Defined also packages/sw/types.ts#L13 type PushNotificationsTypes = { @@ -48,7 +49,7 @@ function truncateBody<T extends keyof PushNotificationsTypes>(type: T, body: Pus @Injectable() export class PushNotificationService implements OnApplicationShutdown { - private subscriptionsCache: RedisKVCache<MiSwSubscription[]>; + private subscriptionsCache: QuantumKVCache<MiSwSubscription[]>; constructor( @Inject(DI.config) @@ -62,13 +63,11 @@ export class PushNotificationService implements OnApplicationShutdown { @Inject(DI.swSubscriptionsRepository) private swSubscriptionsRepository: SwSubscriptionsRepository, + private readonly internalEventService: InternalEventService, ) { - this.subscriptionsCache = new RedisKVCache<MiSwSubscription[]>(this.redisClient, 'userSwSubscriptions', { + this.subscriptionsCache = new QuantumKVCache<MiSwSubscription[]>(this.internalEventService, 'userSwSubscriptions', { lifetime: 1000 * 60 * 60 * 1, // 1h - memoryCacheLifetime: 1000 * 60 * 3, // 3m fetcher: (key) => this.swSubscriptionsRepository.findBy({ userId: key }), - toRedisConverter: (value) => JSON.stringify(value), - fromRedisConverter: (value) => JSON.parse(value), }); } @@ -114,8 +113,8 @@ export class PushNotificationService implements OnApplicationShutdown { endpoint: subscription.endpoint, auth: subscription.auth, publickey: subscription.publickey, - }).then(() => { - this.refreshCache(userId); + }).then(async () => { + await this.refreshCache(userId); }); } }); @@ -123,8 +122,8 @@ export class PushNotificationService implements OnApplicationShutdown { } @bindThis - public refreshCache(userId: string): void { - this.subscriptionsCache.refresh(userId); + public async refreshCache(userId: string): Promise<void> { + await this.subscriptionsCache.refresh(userId); } @bindThis diff --git a/packages/backend/src/core/QueryService.ts b/packages/backend/src/core/QueryService.ts index 50a72e8aa6..d0e281e20c 100644 --- a/packages/backend/src/core/QueryService.ts +++ b/packages/backend/src/core/QueryService.ts @@ -4,13 +4,14 @@ */ import { Inject, Injectable } from '@nestjs/common'; -import { Brackets, ObjectLiteral } from 'typeorm'; +import { Brackets, Not, WhereExpressionBuilder } from 'typeorm'; import { DI } from '@/di-symbols.js'; import type { MiUser } from '@/models/User.js'; -import type { UserProfilesRepository, FollowingsRepository, ChannelFollowingsRepository, BlockingsRepository, NoteThreadMutingsRepository, MutingsRepository, RenoteMutingsRepository, MiMeta } from '@/models/_.js'; +import { MiInstance } from '@/models/Instance.js'; +import type { UserProfilesRepository, FollowingsRepository, ChannelFollowingsRepository, BlockingsRepository, NoteThreadMutingsRepository, MutingsRepository, RenoteMutingsRepository, MiMeta, InstancesRepository } from '@/models/_.js'; import { bindThis } from '@/decorators.js'; import { IdService } from '@/core/IdService.js'; -import type { SelectQueryBuilder } from 'typeorm'; +import type { SelectQueryBuilder, ObjectLiteral } from 'typeorm'; @Injectable() export class QueryService { @@ -36,6 +37,9 @@ export class QueryService { @Inject(DI.renoteMutingsRepository) private renoteMutingsRepository: RenoteMutingsRepository, + @Inject(DI.instancesRepository) + private readonly instancesRepository: InstancesRepository, + @Inject(DI.meta) private meta: MiMeta, @@ -72,218 +76,485 @@ export class QueryService { // ここでいうBlockedは被Blockedの意 @bindThis - public generateBlockedUserQueryForNotes(q: SelectQueryBuilder<any>, me: { id: MiUser['id'] }): void { - const blockingQuery = this.blockingsRepository.createQueryBuilder('blocking') - .select('blocking.blockerId') - .where('blocking.blockeeId = :blockeeId', { blockeeId: me.id }); - + public generateBlockedUserQueryForNotes<E extends ObjectLiteral>(q: SelectQueryBuilder<E>, me: { id: MiUser['id'] }): SelectQueryBuilder<E> { // 投稿の作者にブロックされていない かつ // 投稿の返信先の作者にブロックされていない かつ // 投稿の引用元の作者にブロックされていない - q - .andWhere(`note.userId NOT IN (${ blockingQuery.getQuery() })`) - .andWhere(new Brackets(qb => { + return this + .andNotBlockingUser(q, 'note.userId', ':meId') + .andWhere(new Brackets(qb => this + .orNotBlockingUser(qb, 'note.replyUserId', ':meId') + .orWhere('note.replyUserId IS NULL'))) + .andWhere(new Brackets(qb => this + .orNotBlockingUser(qb, 'note.renoteUserId', ':meId') + .orWhere('note.renoteUserId IS NULL'))) + .setParameters({ meId: me.id }); + } + + @bindThis + public generateBlockQueryForUsers<E extends ObjectLiteral>(q: SelectQueryBuilder<E>, me: { id: MiUser['id'] }): SelectQueryBuilder<E> { + this.andNotBlockingUser(q, ':meId', 'user.id'); + this.andNotBlockingUser(q, 'user.id', ':meId'); + return q.setParameters({ meId: me.id }); + } + + @bindThis + public generateMutedNoteThreadQuery<E extends ObjectLiteral>(q: SelectQueryBuilder<E>, me: { id: MiUser['id'] }): SelectQueryBuilder<E> { + return this + .andNotMutingThread(q, ':meId', 'note.id') + .andWhere(new Brackets(qb => this + .orNotMutingThread(qb, ':meId', 'note.threadId') + .orWhere('note.threadId IS NULL'))) + .setParameters({ meId: me.id }); + } + + @bindThis + public generateMutedUserQueryForNotes<E extends ObjectLiteral>(q: SelectQueryBuilder<E>, me: { id: MiUser['id'] }, exclude?: { id: MiUser['id'] }): SelectQueryBuilder<E> { + // 投稿の作者をミュートしていない かつ + // 投稿の返信先の作者をミュートしていない かつ + // 投稿の引用元の作者をミュートしていない + return this + .andNotMutingUser(q, ':meId', 'note.userId', exclude) + .andWhere(new Brackets(qb => this + .orNotMutingUser(qb, ':meId', 'note.replyUserId', exclude) + .orWhere('note.replyUserId IS NULL'))) + .andWhere(new Brackets(qb => this + .orNotMutingUser(qb, ':meId', 'note.renoteUserId', exclude) + .orWhere('note.renoteUserId IS NULL'))) + // TODO exclude should also pass a host to skip these instances + // mute instances + .andWhere(new Brackets(qb => this + .andNotMutingInstance(qb, ':meId', 'note.userHost') + .orWhere('note.userHost IS NULL'))) + .andWhere(new Brackets(qb => this + .orNotMutingInstance(qb, ':meId', 'note.replyUserHost') + .orWhere('note.replyUserHost IS NULL'))) + .andWhere(new Brackets(qb => this + .orNotMutingInstance(qb, ':meId', 'note.renoteUserHost') + .orWhere('note.renoteUserHost IS NULL'))) + .setParameters({ meId: me.id }); + } + + @bindThis + public generateMutedUserQueryForUsers<E extends ObjectLiteral>(q: SelectQueryBuilder<E>, me: { id: MiUser['id'] }): SelectQueryBuilder<E> { + return this + .andNotMutingUser(q, ':meId', 'user.id') + .setParameters({ meId: me.id }); + } + + // This intentionally skips isSuspended, isDeleted, makeNotesFollowersOnlyBefore, makeNotesHiddenBefore, and requireSigninToViewContents. + // NoteEntityService checks these automatically and calls hideNote() to hide them without breaking threads. + // For moderation purposes, you can set isSilenced to forcibly hide existing posts by a user. + @bindThis + public generateVisibilityQuery<E extends ObjectLiteral>(q: SelectQueryBuilder<E>, me?: { id: MiUser['id'] } | null): SelectQueryBuilder<E> { + // This code must always be synchronized with the checks in Notes.isVisibleForMe. + return q.andWhere(new Brackets(qb => { + // Public post + qb.orWhere('note.visibility = \'public\'') + .orWhere('note.visibility = \'home\''); + + if (me != null) { qb - .where('note.replyUserId IS NULL') - .orWhere(`note.replyUserId NOT IN (${ blockingQuery.getQuery() })`); - })) + // My post + .orWhere(':meId = note.userId') + // Visible to me + .orWhere(':meIdAsList <@ note.visibleUserIds') + // Followers-only post + .orWhere(new Brackets(qb => qb + .andWhere(new Brackets(qbb => this + // Following author + .orFollowingUser(qbb, ':meId', 'note.userId') + // Mentions me + .orWhere(':meIdAsList <@ note.mentions') + // Reply to me + .orWhere(':meId = note.replyUserId'))) + .andWhere('note.visibility = \'followers\''))); + + q.setParameters({ meId: me.id, meIdAsList: [me.id] }); + } + })); + } + + @bindThis + public generateMutedUserRenotesQueryForNotes<E extends ObjectLiteral>(q: SelectQueryBuilder<E>, me: { id: MiUser['id'] }): SelectQueryBuilder<E> { + return q + .andWhere(new Brackets(qb => this + .orNotMutingRenote(qb, ':meId', 'note.userId') + .orWhere('note.renoteId IS NULL') + .orWhere('note.text IS NOT NULL') + .orWhere('note.cw IS NOT NULL') + .orWhere('note.replyId IS NOT NULL') + .orWhere('note.hasPoll = true') + .orWhere('note.fileIds != \'{}\''))) + .setParameters({ meId: me.id }); + } + + @bindThis + public generateExcludedRenotesQueryForNotes<Q extends WhereExpressionBuilder>(q: Q): Q { + return this.andIsNotRenote(q, 'note'); + } + + @bindThis + public generateBlockedHostQueryForNote<E extends ObjectLiteral>(q: SelectQueryBuilder<E>, excludeAuthor?: boolean): SelectQueryBuilder<E> { + const checkFor = (key: 'user' | 'replyUser' | 'renoteUser') => this + .leftJoinInstance(q, `note.${key}Instance`, `${key}Instance`) .andWhere(new Brackets(qb => { qb - .where('note.renoteUserId IS NULL') - .orWhere(`note.renoteUserId NOT IN (${ blockingQuery.getQuery() })`); + .orWhere(`"${key}Instance" IS NULL`) // local + .orWhere(`"${key}Instance"."isBlocked" = false`); // not blocked + + if (excludeAuthor) { + qb.orWhere(`note.userId = note.${key}Id`); // author + } })); - q.setParameters(blockingQuery.getParameters()); + if (!excludeAuthor) { + checkFor('user'); + } + checkFor('replyUser'); + checkFor('renoteUser'); + + return q; } @bindThis - public generateBlockQueryForUsers(q: SelectQueryBuilder<any>, me: { id: MiUser['id'] }): void { - const blockingQuery = this.blockingsRepository.createQueryBuilder('blocking') - .select('blocking.blockeeId') - .where('blocking.blockerId = :blockerId', { blockerId: me.id }); + public generateSilencedUserQueryForNotes<E extends ObjectLiteral>(q: SelectQueryBuilder<E>, me?: { id: MiUser['id'] } | null): SelectQueryBuilder<E> { + if (!me) { + return q.andWhere('user.isSilenced = false'); + } - const blockedQuery = this.blockingsRepository.createQueryBuilder('blocking') - .select('blocking.blockerId') - .where('blocking.blockeeId = :blockeeId', { blockeeId: me.id }); + return this + .leftJoinInstance(q, 'note.userInstance', 'userInstance') + .andWhere(new Brackets(qb => this + // case 1: we are following the user + .orFollowingUser(qb, ':meId', 'note.userId') + // case 2: user not silenced AND instance not silenced + .orWhere(new Brackets(qbb => qbb + .andWhere(new Brackets(qbbb => qbbb + .orWhere('"userInstance"."isSilenced" = false') + .orWhere('"userInstance" IS NULL'))) + .andWhere('user.isSilenced = false'))))) + .setParameters({ meId: me.id }); + } - q.andWhere(`user.id NOT IN (${ blockingQuery.getQuery() })`); - q.setParameters(blockingQuery.getParameters()); + /** + * Left-joins an instance in to the query with a given alias and optional condition. + * These calls are de-duplicated - multiple uses of the same alias are skipped. + */ + @bindThis + public leftJoinInstance<E extends ObjectLiteral>(q: SelectQueryBuilder<E>, relation: string | typeof MiInstance, alias: string, condition?: string): SelectQueryBuilder<E> { + // Skip if it's already joined, otherwise we'll get an error + if (!q.expressionMap.joinAttributes.some(j => j.alias.name === alias)) { + q.leftJoin(relation, alias, condition); + } - q.andWhere(`user.id NOT IN (${ blockedQuery.getQuery() })`); - q.setParameters(blockedQuery.getParameters()); + return q; } + /** + * Adds OR condition that noteProp (note ID) refers to a quote. + * The prop should be an expression, not a raw value. + */ @bindThis - public generateMutedNoteThreadQuery(q: SelectQueryBuilder<any>, me: { id: MiUser['id'] }): void { - const mutedQuery = this.noteThreadMutingsRepository.createQueryBuilder('threadMuted') - .select('threadMuted.threadId') - .where('threadMuted.userId = :userId', { userId: me.id }); + public orIsQuote<Q extends WhereExpressionBuilder>(q: Q, noteProp: string): Q { + return this.addIsQuote(q, noteProp, 'orWhere'); + } - q.andWhere(`note.id NOT IN (${ mutedQuery.getQuery() })`); - q.andWhere(new Brackets(qb => { - qb - .where('note.threadId IS NULL') - .orWhere(`note.threadId NOT IN (${ mutedQuery.getQuery() })`); - })); + /** + * Adds AND condition that noteProp (note ID) refers to a quote. + * The prop should be an expression, not a raw value. + */ + @bindThis + public andIsQuote<Q extends WhereExpressionBuilder>(q: Q, noteProp: string): Q { + return this.addIsQuote(q, noteProp, 'andWhere'); + } - q.setParameters(mutedQuery.getParameters()); + private addIsQuote<Q extends WhereExpressionBuilder>(q: Q, noteProp: string, join: 'andWhere' | 'orWhere'): Q { + return q[join](new Brackets(qb => qb + .andWhere(`${noteProp}.renoteId IS NOT NULL`) + .andWhere(new Brackets(qbb => qbb + .orWhere(`${noteProp}.text IS NOT NULL`) + .orWhere(`${noteProp}.cw IS NOT NULL`) + .orWhere(`${noteProp}.replyId IS NOT NULL`) + .orWhere(`${noteProp}.hasPoll = true`) + .orWhere(`${noteProp}.fileIds != '{}'`))))); } + /** + * Adds OR condition that noteProp (note ID) does not refer to a quote. + * The prop should be an expression, not a raw value. + */ @bindThis - public generateMutedUserQueryForNotes(q: SelectQueryBuilder<any>, me: { id: MiUser['id'] }, exclude?: { id: MiUser['id'] }): void { - const mutingQuery = this.mutingsRepository.createQueryBuilder('muting') - .select('muting.muteeId') - .where('muting.muterId = :muterId', { muterId: me.id }); + public orIsNotQuote<Q extends WhereExpressionBuilder>(q: Q, noteProp: string): Q { + return this.addIsNotQuote(q, noteProp, 'orWhere'); + } - if (exclude) { - mutingQuery.andWhere('muting.muteeId != :excludeId', { excludeId: exclude.id }); - } + /** + * Adds AND condition that noteProp (note ID) does not refer to a quote. + * The prop should be an expression, not a raw value. + */ + @bindThis + public andIsNotQuote<Q extends WhereExpressionBuilder>(q: Q, noteProp: string): Q { + return this.addIsNotQuote(q, noteProp, 'andWhere'); + } - const mutingInstanceQuery = this.userProfilesRepository.createQueryBuilder('user_profile') - .select('user_profile.mutedInstances') - .where('user_profile.userId = :muterId', { muterId: me.id }); + private addIsNotQuote<Q extends WhereExpressionBuilder>(q: Q, noteProp: string, join: 'andWhere' | 'orWhere'): Q { + return q[join](new Brackets(qb => qb + .orWhere(`${noteProp}.renoteId IS NULL`) + .orWhere(new Brackets(qb => qb + .andWhere(`${noteProp}.text IS NULL`) + .andWhere(`${noteProp}.cw IS NULL`) + .andWhere(`${noteProp}.replyId IS NULL`) + .andWhere(`${noteProp}.hasPoll = false`) + .andWhere(`${noteProp}.fileIds = '{}'`))))); + } - // 投稿の作者をミュートしていない かつ - // 投稿の返信先の作者をミュートしていない かつ - // 投稿の引用元の作者をミュートしていない - q - .andWhere(`note.userId NOT IN (${ mutingQuery.getQuery() })`) - .andWhere(new Brackets(qb => { - qb - .where('note.replyUserId IS NULL') - .orWhere(`note.replyUserId NOT IN (${ mutingQuery.getQuery() })`); - })) - .andWhere(new Brackets(qb => { - qb - .where('note.renoteUserId IS NULL') - .orWhere(`note.renoteUserId NOT IN (${ mutingQuery.getQuery() })`); - })) - // mute instances - .andWhere(new Brackets(qb => { - qb - .andWhere('note.userHost IS NULL') - .orWhere(`NOT ((${ mutingInstanceQuery.getQuery() })::jsonb ? note.userHost)`); - })) - .andWhere(new Brackets(qb => { - qb - .where('note.replyUserHost IS NULL') - .orWhere(`NOT ((${ mutingInstanceQuery.getQuery() })::jsonb ? note.replyUserHost)`); - })) - .andWhere(new Brackets(qb => { - qb - .where('note.renoteUserHost IS NULL') - .orWhere(`NOT ((${ mutingInstanceQuery.getQuery() })::jsonb ? note.renoteUserHost)`); - })); + /** + * Adds OR condition that noteProp (note ID) refers to a renote. + * The prop should be an expression, not a raw value. + */ + @bindThis + public orIsRenote<Q extends WhereExpressionBuilder>(q: Q, noteProp: string): Q { + return this.addIsRenote(q, noteProp, 'orWhere'); + } + + /** + * Adds AND condition that noteProp (note ID) refers to a renote. + * The prop should be an expression, not a raw value. + */ + @bindThis + public andIsRenote<Q extends WhereExpressionBuilder>(q: Q, noteProp: string): Q { + return this.addIsRenote(q, noteProp, 'andWhere'); + } - q.setParameters(mutingQuery.getParameters()); - q.setParameters(mutingInstanceQuery.getParameters()); + private addIsRenote<Q extends WhereExpressionBuilder>(q: Q, noteProp: string, join: 'andWhere' | 'orWhere'): Q { + return q[join](new Brackets(qb => qb + .andWhere(`${noteProp}.renoteId IS NOT NULL`) + .andWhere(`${noteProp}.text IS NULL`) + .andWhere(`${noteProp}.cw IS NULL`) + .andWhere(`${noteProp}.replyId IS NULL`) + .andWhere(`${noteProp}.hasPoll = false`) + .andWhere(`${noteProp}.fileIds = '{}'`))); } + /** + * Adds OR condition that noteProp (note ID) does not refer to a renote. + * The prop should be an expression, not a raw value. + */ @bindThis - public generateMutedUserQueryForUsers(q: SelectQueryBuilder<any>, me: { id: MiUser['id'] }): void { - const mutingQuery = this.mutingsRepository.createQueryBuilder('muting') - .select('muting.muteeId') - .where('muting.muterId = :muterId', { muterId: me.id }); + public orIsNotRenote<Q extends WhereExpressionBuilder>(q: Q, noteProp: string): Q { + return this.addIsNotRenote(q, noteProp, 'orWhere'); + } - q.andWhere(`user.id NOT IN (${ mutingQuery.getQuery() })`); + /** + * Adds AND condition that noteProp (note ID) does not refer to a renote. + * The prop should be an expression, not a raw value. + */ + @bindThis + public andIsNotRenote<Q extends WhereExpressionBuilder>(q: Q, noteProp: string): Q { + return this.addIsNotRenote(q, noteProp, 'andWhere'); + } - q.setParameters(mutingQuery.getParameters()); + private addIsNotRenote<Q extends WhereExpressionBuilder>(q: Q, noteProp: string, join: 'andWhere' | 'orWhere'): Q { + return q[join](new Brackets(qb => qb + .orWhere(`${noteProp}.renoteId IS NULL`) + .orWhere(`${noteProp}.text IS NOT NULL`) + .orWhere(`${noteProp}.cw IS NOT NULL`) + .orWhere(`${noteProp}.replyId IS NOT NULL`) + .orWhere(`${noteProp}.hasPoll = true`) + .orWhere(`${noteProp}.fileIds != '{}'`))); } + /** + * Adds OR condition that followerProp (user ID) is following followeeProp (user ID). + * Both props should be expressions, not raw values. + */ @bindThis - public generateVisibilityQuery(q: SelectQueryBuilder<any>, me?: { id: MiUser['id'] } | null): void { - // This code must always be synchronized with the checks in Notes.isVisibleForMe. - if (me == null) { - q.andWhere(new Brackets(qb => { - qb - .where('note.visibility = \'public\'') - .orWhere('note.visibility = \'home\''); - })); - } else { - const followingQuery = this.followingsRepository.createQueryBuilder('following') - .select('following.followeeId') - .where('following.followerId = :meId'); + public orFollowingUser<Q extends WhereExpressionBuilder>(q: Q, followerProp: string, followeeProp: string): Q { + return this.addFollowingUser(q, followerProp, followeeProp, 'orWhere'); + } - q.andWhere(new Brackets(qb => { - qb - // 公開投稿である - .where(new Brackets(qb => { - qb - .where('note.visibility = \'public\'') - .orWhere('note.visibility = \'home\''); - })) - // または 自分自身 - .orWhere('note.userId = :meId') - // または 自分宛て - .orWhere(':meIdAsList <@ note.visibleUserIds') - .orWhere(new Brackets(qb => { - qb - // または フォロワー宛ての投稿であり、 - .where('note.visibility = \'followers\'') - .andWhere(new Brackets(qb => { - qb - // 自分がフォロワーである - .where(`note.userId IN (${ followingQuery.getQuery() })`) - // または 自分の投稿へのリプライ - .orWhere('note.replyUserId = :meId') - .orWhere(':meIdAsList <@ note.mentions'); - })); - })); - })); + /** + * Adds AND condition that followerProp (user ID) is following followeeProp (user ID). + * Both props should be expressions, not raw values. + */ + @bindThis + public andFollowingUser<Q extends WhereExpressionBuilder>(q: Q, followerProp: string, followeeProp: string): Q { + return this.addFollowingUser(q, followerProp, followeeProp, 'andWhere'); + } + + private addFollowingUser<Q extends WhereExpressionBuilder>(q: Q, followerProp: string, followeeProp: string, join: 'andWhere' | 'orWhere'): Q { + const followingQuery = this.followingsRepository.createQueryBuilder('following') + .select('1') + .andWhere(`following.followerId = ${followerProp}`) + .andWhere(`following.followeeId = ${followeeProp}`); + + return q[join](`EXISTS (${followingQuery.getQuery()})`, followingQuery.getParameters()); + }; + + /** + * Adds OR condition that followerProp (user ID) is following followeeProp (channel ID). + * Both props should be expressions, not raw values. + */ + @bindThis + public orFollowingChannel<Q extends WhereExpressionBuilder>(q: Q, followerProp: string, followeeProp: string): Q { + return this.addFollowingChannel(q, followerProp, followeeProp, 'orWhere'); + } + + /** + * Adds AND condition that followerProp (user ID) is following followeeProp (channel ID). + * Both props should be expressions, not raw values. + */ + @bindThis + public andFollowingChannel<Q extends WhereExpressionBuilder>(q: Q, followerProp: string, followeeProp: string): Q { + return this.addFollowingChannel(q, followerProp, followeeProp, 'andWhere'); + } + + private addFollowingChannel<Q extends WhereExpressionBuilder>(q: Q, followerProp: string, followeeProp: string, join: 'andWhere' | 'orWhere'): Q { + const followingQuery = this.channelFollowingsRepository.createQueryBuilder('following') + .select('1') + .andWhere(`following.followerId = ${followerProp}`) + .andWhere(`following.followeeId = ${followeeProp}`); - q.setParameters({ meId: me.id, meIdAsList: [me.id] }); + return q[join](`EXISTS (${followingQuery.getQuery()})`, followingQuery.getParameters()); + } + + /** + * Adds OR condition that blockerProp (user ID) is not blocking blockeeProp (user ID). + * Both props should be expressions, not raw values. + */ + @bindThis + public orNotBlockingUser<Q extends WhereExpressionBuilder>(q: Q, blockerProp: string, blockeeProp: string): Q { + return this.excludeBlockingUser(q, blockerProp, blockeeProp, 'orWhere'); + } + + /** + * Adds AND condition that blockerProp (user ID) is not blocking blockeeProp (user ID). + * Both props should be expressions, not raw values. + */ + @bindThis + public andNotBlockingUser<Q extends WhereExpressionBuilder>(q: Q, blockerProp: string, blockeeProp: string): Q { + return this.excludeBlockingUser(q, blockerProp, blockeeProp, 'andWhere'); + } + + private excludeBlockingUser<Q extends WhereExpressionBuilder>(q: Q, blockerProp: string, blockeeProp: string, join: 'andWhere' | 'orWhere'): Q { + const blockingQuery = this.blockingsRepository.createQueryBuilder('blocking') + .select('1') + .andWhere(`blocking.blockerId = ${blockerProp}`) + .andWhere(`blocking.blockeeId = ${blockeeProp}`); + + return q[join](`NOT EXISTS (${blockingQuery.getQuery()})`, blockingQuery.getParameters()); + }; + + /** + * Adds OR condition that muterProp (user ID) is not muting muteeProp (user ID). + * Both props should be expressions, not raw values. + */ + @bindThis + public orNotMutingUser<Q extends WhereExpressionBuilder>(q: Q, muterProp: string, muteeProp: string, exclude?: { id: MiUser['id'] }): Q { + return this.excludeMutingUser(q, muterProp, muteeProp, 'orWhere', exclude); + } + + /** + * Adds AND condition that muterProp (user ID) is not muting muteeProp (user ID). + * Both props should be expressions, not raw values. + */ + @bindThis + public andNotMutingUser<Q extends WhereExpressionBuilder>(q: Q, muterProp: string, muteeProp: string, exclude?: { id: MiUser['id'] }): Q { + return this.excludeMutingUser(q, muterProp, muteeProp, 'andWhere', exclude); + } + + private excludeMutingUser<Q extends WhereExpressionBuilder>(q: Q, muterProp: string, muteeProp: string, join: 'andWhere' | 'orWhere', exclude?: { id: MiUser['id'] }): Q { + const mutingQuery = this.mutingsRepository.createQueryBuilder('muting') + .select('1') + .andWhere(`muting.muterId = ${muterProp}`) + .andWhere(`muting.muteeId = ${muteeProp}`); + + if (exclude) { + mutingQuery.andWhere({ muteeId: Not(exclude.id) }); } + + return q[join](`NOT EXISTS (${mutingQuery.getQuery()})`, mutingQuery.getParameters()); } + /** + * Adds OR condition that muterProp (user ID) is not muting renotes by muteeProp (user ID). + * Both props should be expressions, not raw values. + */ @bindThis - public generateMutedUserRenotesQueryForNotes(q: SelectQueryBuilder<any>, me: { id: MiUser['id'] }): void { + public orNotMutingRenote<Q extends WhereExpressionBuilder>(q: Q, muterProp: string, muteeProp: string): Q { + return this.excludeMutingRenote(q, muterProp, muteeProp, 'orWhere'); + } + + /** + * Adds AND condition that muterProp (user ID) is not muting renotes by muteeProp (user ID). + * Both props should be expressions, not raw values. + */ + @bindThis + public andNotMutingRenote<Q extends WhereExpressionBuilder>(q: Q, muterProp: string, muteeProp: string): Q { + return this.excludeMutingRenote(q, muterProp, muteeProp, 'andWhere'); + } + + private excludeMutingRenote<Q extends WhereExpressionBuilder>(q: Q, muterProp: string, muteeProp: string, join: 'andWhere' | 'orWhere'): Q { const mutingQuery = this.renoteMutingsRepository.createQueryBuilder('renote_muting') - .select('renote_muting.muteeId') - .where('renote_muting.muterId = :muterId', { muterId: me.id }); + .select('1') + .andWhere(`renote_muting.muterId = ${muterProp}`) + .andWhere(`renote_muting.muteeId = ${muteeProp}`); - q.andWhere(new Brackets(qb => { - qb - .where(new Brackets(qb => { - qb.where('note.renoteId IS NOT NULL'); - qb.andWhere('note.text IS NULL'); - qb.andWhere(`note.userId NOT IN (${ mutingQuery.getQuery() })`); - })) - .orWhere('note.renoteId IS NULL') - .orWhere('note.text IS NOT NULL'); - })); + return q[join](`NOT EXISTS (${mutingQuery.getQuery()})`, mutingQuery.getParameters()); + }; - q.setParameters(mutingQuery.getParameters()); + /** + * Adds OR condition that muterProp (user ID) is not muting muteeProp (instance host). + * Both props should be expressions, not raw values. + */ + @bindThis + public orNotMutingInstance<Q extends WhereExpressionBuilder>(q: Q, muterProp: string, muteeProp: string): Q { + return this.excludeMutingInstance(q, muterProp, muteeProp, 'orWhere'); } + /** + * Adds AND condition that muterProp (user ID) is not muting muteeProp (instance host). + * Both props should be expressions, not raw values. + */ @bindThis - public generateBlockedHostQueryForNote(q: SelectQueryBuilder<any>, excludeAuthor?: boolean): void { - let nonBlockedHostQuery: (part: string) => string; - if (this.meta.blockedHosts.length === 0) { - nonBlockedHostQuery = () => '1=1'; - } else { - nonBlockedHostQuery = (match: string) => `('.' || ${match}) NOT ILIKE ALL(select '%.' || x from (select unnest("blockedHosts") as x from "meta") t)`; - } + public andNotMutingInstance<Q extends WhereExpressionBuilder>(q: Q, muterProp: string, muteeProp: string): Q { + return this.excludeMutingInstance(q, muterProp, muteeProp, 'andWhere'); + } - if (excludeAuthor) { - const instanceSuspension = (user: string) => new Brackets(qb => qb - .where(`note.${user}Id IS NULL`) // no corresponding user - .orWhere(`note.userId = note.${user}Id`) - .orWhere(`note.${user}Host IS NULL`) // local - .orWhere(nonBlockedHostQuery(`note.${user}Host`))); + private excludeMutingInstance<Q extends WhereExpressionBuilder>(q: Q, muterProp: string, muteeProp: string, join: 'andWhere' | 'orWhere'): Q { + const mutingInstanceQuery = this.userProfilesRepository.createQueryBuilder('user_profile') + .select('1') + .andWhere(`user_profile.userId = ${muterProp}`) + .andWhere(`"user_profile"."mutedInstances"::jsonb ? ${muteeProp}`); - q - .andWhere(instanceSuspension('replyUser')) - .andWhere(instanceSuspension('renoteUser')); - } else { - const instanceSuspension = (user: string) => new Brackets(qb => qb - .where(`note.${user}Id IS NULL`) // no corresponding user - .orWhere(`note.${user}Host IS NULL`) // local - .orWhere(nonBlockedHostQuery(`note.${user}Host`))); + return q[join](`NOT EXISTS (${mutingInstanceQuery.getQuery()})`, mutingInstanceQuery.getParameters()); + } - q - .andWhere(instanceSuspension('user')) - .andWhere(instanceSuspension('replyUser')) - .andWhere(instanceSuspension('renoteUser')); - } + /** + * Adds OR condition that muterProp (user ID) is not muting muteeProp (note ID). + * Both props should be expressions, not raw values. + */ + @bindThis + public orNotMutingThread<Q extends WhereExpressionBuilder>(q: Q, muterProp: string, muteeProp: string): Q { + return this.excludeMutingThread(q, muterProp, muteeProp, 'orWhere'); + } + + /** + * Adds AND condition that muterProp (user ID) is not muting muteeProp (note ID). + * Both props should be expressions, not raw values. + */ + @bindThis + public andNotMutingThread<Q extends WhereExpressionBuilder>(q: Q, muterProp: string, muteeProp: string): Q { + return this.excludeMutingThread(q, muterProp, muteeProp, 'andWhere'); + } + + private excludeMutingThread<Q extends WhereExpressionBuilder>(q: Q, muterProp: string, muteeProp: string, join: 'andWhere' | 'orWhere'): Q { + const threadMutedQuery = this.noteThreadMutingsRepository.createQueryBuilder('threadMuted') + .select('1') + .andWhere(`threadMuted.userId = ${muterProp}`) + .andWhere(`threadMuted.threadId = ${muteeProp}`); + + return q[join](`NOT EXISTS (${threadMutedQuery.getQuery()})`, threadMutedQuery.getParameters()); } } diff --git a/packages/backend/src/core/ReactionService.ts b/packages/backend/src/core/ReactionService.ts index f05ee2ee73..8d2dc7d4e8 100644 --- a/packages/backend/src/core/ReactionService.ts +++ b/packages/backend/src/core/ReactionService.ts @@ -10,7 +10,7 @@ import { IdentifiableError } from '@/misc/identifiable-error.js'; import type { MiRemoteUser, MiUser } from '@/models/User.js'; import type { MiNote } from '@/models/Note.js'; import { IdService } from '@/core/IdService.js'; -import type { MiNoteReaction } from '@/models/NoteReaction.js'; +import { MiNoteReaction } from '@/models/NoteReaction.js'; import { isDuplicateKeyValueError } from '@/misc/is-duplicate-key-value-error.js'; import { GlobalEventService } from '@/core/GlobalEventService.js'; import { NotificationService } from '@/core/NotificationService.js'; @@ -31,6 +31,7 @@ import { isQuote, isRenote } from '@/misc/is-renote.js'; import { ReactionsBufferingService } from '@/core/ReactionsBufferingService.js'; import { PER_NOTE_REACTION_USER_PAIR_CACHE_MAX } from '@/const.js'; import { CacheService } from '@/core/CacheService.js'; +import type { DataSource } from 'typeorm'; const FALLBACK = '\u2764'; @@ -89,6 +90,9 @@ export class ReactionService { @Inject(DI.emojisRepository) private emojisRepository: EmojisRepository, + @Inject(DI.db) + private readonly db: DataSource, + private utilityService: UtilityService, private customEmojiService: CustomEmojiService, private roleService: RoleService, @@ -113,12 +117,12 @@ export class ReactionService { if (note.userId !== user.id) { const blocked = await this.userBlockingService.checkBlocked(note.userId, user.id); if (blocked) { - throw new IdentifiableError('e70412a4-7197-4726-8e74-f3e0deb92aa7'); + throw new IdentifiableError('e70412a4-7197-4726-8e74-f3e0deb92aa7', 'Note not accessible for you.'); } } // check visibility - if (!await this.noteEntityService.isVisibleForMe(note, user.id)) { + if (!await this.noteEntityService.isVisibleForMe(note, user.id, { me: user })) { throw new IdentifiableError('68e9d2d1-48bf-42c2-b90a-b20e09fd3d48', 'Note not accessible for you.'); } @@ -176,26 +180,28 @@ export class ReactionService { reaction, }; - try { - await this.noteReactionsRepository.insert(record); - } catch (e) { - if (isDuplicateKeyValueError(e)) { - const exists = await this.noteReactionsRepository.findOneByOrFail({ - noteId: note.id, - userId: user.id, - }); + const result = await this.db.transaction(async tem => { + await tem.createQueryBuilder(MiNoteReaction, 'noteReaction') + .insert() + .values(record) + .orIgnore() + .execute(); - if (exists.reaction !== reaction) { - // 別のリアクションがすでにされていたら置き換える - await this.delete(user, note); - await this.noteReactionsRepository.insert(record); - } else { - // 同じリアクションがすでにされていたらエラー - throw new IdentifiableError('51c42bb4-931a-456b-bff7-e5a8a70dd298'); - } - } else { - throw e; + return await tem.createQueryBuilder(MiNoteReaction, 'noteReaction') + .select() + .where({ noteId: note.id, userId: user.id }) + .getOneOrFail(); + }); + + if (result.id !== record.id) { + // Conflict with the same ID => nothing to do. + if (result.reaction === record.reaction) { + return; } + + // 別のリアクションがすでにされていたら置き換える + await this.delete(user, note); + await this.noteReactionsRepository.insert(record); } // Increment reactions count @@ -316,14 +322,14 @@ export class ReactionService { }); if (exist == null) { - throw new IdentifiableError('60527ec9-b4cb-4a88-a6bd-32d3ad26817d', 'not reacted'); + throw new IdentifiableError('60527ec9-b4cb-4a88-a6bd-32d3ad26817d', 'reaction does not exist'); } // Delete reaction const result = await this.noteReactionsRepository.delete(exist.id); if (result.affected !== 1) { - throw new IdentifiableError('60527ec9-b4cb-4a88-a6bd-32d3ad26817d', 'not reacted'); + throw new IdentifiableError('60527ec9-b4cb-4a88-a6bd-32d3ad26817d', 'reaction does not exist'); } // Decrement reactions count diff --git a/packages/backend/src/core/RemoteUserResolveService.ts b/packages/backend/src/core/RemoteUserResolveService.ts index a2f1b73cdb..4dbc9d6a36 100644 --- a/packages/backend/src/core/RemoteUserResolveService.ts +++ b/packages/backend/src/core/RemoteUserResolveService.ts @@ -3,7 +3,6 @@ * SPDX-License-Identifier: AGPL-3.0-only */ -import { URL } from 'node:url'; import { Inject, Injectable } from '@nestjs/common'; import chalk from 'chalk'; import { IsNull } from 'typeorm'; @@ -18,6 +17,7 @@ import { RemoteLoggerService } from '@/core/RemoteLoggerService.js'; import { ApDbResolverService } from '@/core/activitypub/ApDbResolverService.js'; import { ApPersonService } from '@/core/activitypub/models/ApPersonService.js'; import { bindThis } from '@/decorators.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; @Injectable() export class RemoteUserResolveService { @@ -44,27 +44,13 @@ export class RemoteUserResolveService { const usernameLower = username.toLowerCase(); if (host == null) { - this.logger.info(`return local user: ${usernameLower}`); - return await this.usersRepository.findOneBy({ usernameLower, host: IsNull() }).then(u => { - if (u == null) { - throw new Error('user not found'); - } else { - return u; - } - }) as MiLocalUser; + return await this.usersRepository.findOneByOrFail({ usernameLower, host: IsNull() }) as MiLocalUser; } host = this.utilityService.toPuny(host); if (host === this.utilityService.toPuny(this.config.host)) { - this.logger.info(`return local user: ${usernameLower}`); - return await this.usersRepository.findOneBy({ usernameLower, host: IsNull() }).then(u => { - if (u == null) { - throw new Error('user not found'); - } else { - return u; - } - }) as MiLocalUser; + return await this.usersRepository.findOneByOrFail({ usernameLower, host: IsNull() }) as MiLocalUser; } const user = await this.usersRepository.findOneBy({ usernameLower, host }) as MiRemoteUser | null; @@ -82,7 +68,7 @@ export class RemoteUserResolveService { .getUserFromApId(self.href) .then((u) => { if (u == null) { - throw new Error('local user not found'); + throw new Error(`local user not found: ${self.href}`); } else { return u; } @@ -90,7 +76,7 @@ export class RemoteUserResolveService { } } - this.logger.succ(`return new remote user: ${chalk.magenta(acctLower)}`); + this.logger.info(`Fetching new remote user ${chalk.magenta(acctLower)} from ${self.href}`); return await this.apPersonService.createPerson(self.href); } @@ -101,18 +87,16 @@ export class RemoteUserResolveService { lastFetchedAt: new Date(), }); - this.logger.info(`try resync: ${acctLower}`); const self = await this.resolveSelf(acctLower); if (user.uri !== self.href) { // if uri mismatch, Fix (user@host <=> AP's Person id(RemoteUser.uri)) mapping. - this.logger.info(`uri missmatch: ${acctLower}`); - this.logger.info(`recovery missmatch uri for (username=${username}, host=${host}) from ${user.uri} to ${self.href}`); + this.logger.warn(`Detected URI mismatch for ${acctLower}`); // validate uri - const uri = new URL(self.href); - if (uri.hostname !== host) { - throw new Error('Invalid uri'); + const uriHost = this.utilityService.extractDbHost(self.href); + if (uriHost !== host) { + throw new Error(`Failed to correct URI for ${acctLower}: new URI ${self.href} has different host from previous URI ${user.uri}`); } await this.usersRepository.update({ @@ -121,37 +105,28 @@ export class RemoteUserResolveService { }, { uri: self.href, }); - } else { - this.logger.info(`uri is fine: ${acctLower}`); } + this.logger.info(`Corrected URI for ${acctLower} from ${user.uri} to ${self.href}`); + await this.apPersonService.updatePerson(self.href); - this.logger.info(`return resynced remote user: ${acctLower}`); - return await this.usersRepository.findOneBy({ uri: self.href }).then(u => { - if (u == null) { - throw new Error('user not found'); - } else { - return u as MiLocalUser | MiRemoteUser; - } - }); + return await this.usersRepository.findOneByOrFail({ uri: self.href }) as MiLocalUser | MiRemoteUser; } - this.logger.info(`return existing remote user: ${acctLower}`); return user; } @bindThis private async resolveSelf(acctLower: string): Promise<ILink> { - this.logger.info(`WebFinger for ${chalk.yellow(acctLower)}`); const finger = await this.webfingerService.webfinger(acctLower).catch(err => { - this.logger.error(`Failed to WebFinger for ${chalk.yellow(acctLower)}: ${ err.statusCode ?? err.message }`); - throw new Error(`Failed to WebFinger for ${acctLower}: ${ err.statusCode ?? err.message }`); + this.logger.error(`Failed to WebFinger for ${chalk.yellow(acctLower)}: ${renderInlineError(err)}`); + throw new Error(`Failed to WebFinger for ${acctLower}: error thrown`, { cause: err }); }); const self = finger.links.find(link => link.rel != null && link.rel.toLowerCase() === 'self'); if (!self) { this.logger.error(`Failed to WebFinger for ${chalk.yellow(acctLower)}: self link not found`); - throw new Error('self link not found'); + throw new Error(`Failed to WebFinger for ${acctLower}: self link not found`); } return self; } diff --git a/packages/backend/src/core/ReversiService.ts b/packages/backend/src/core/ReversiService.ts index 8c0a8f6cc7..b57ab6d9cb 100644 --- a/packages/backend/src/core/ReversiService.ts +++ b/packages/backend/src/core/ReversiService.ts @@ -587,6 +587,8 @@ export class ReversiService implements OnApplicationShutdown, OnModuleInit { lastActiveDate: parsed.user1.lastActiveDate != null ? new Date(parsed.user1.lastActiveDate) : null, lastFetchedAt: parsed.user1.lastFetchedAt != null ? new Date(parsed.user1.lastFetchedAt) : null, movedAt: parsed.user1.movedAt != null ? new Date(parsed.user1.movedAt) : null, + instance: null, + userProfile: null, } : null, user2: parsed.user2 != null ? { ...parsed.user2, @@ -597,6 +599,8 @@ export class ReversiService implements OnApplicationShutdown, OnModuleInit { lastActiveDate: parsed.user2.lastActiveDate != null ? new Date(parsed.user2.lastActiveDate) : null, lastFetchedAt: parsed.user2.lastFetchedAt != null ? new Date(parsed.user2.lastFetchedAt) : null, movedAt: parsed.user2.movedAt != null ? new Date(parsed.user2.movedAt) : null, + instance: null, + userProfile: null, } : null, }; } else { diff --git a/packages/backend/src/core/RoleService.ts b/packages/backend/src/core/RoleService.ts index d3c458eec7..b250eeee21 100644 --- a/packages/backend/src/core/RoleService.ts +++ b/packages/backend/src/core/RoleService.ts @@ -86,10 +86,10 @@ export const DEFAULT_POLICIES: RolePolicies = { canManageCustomEmojis: false, canManageAvatarDecorations: false, canSearchNotes: false, - canUseTranslator: true, + canUseTranslator: false, canHideAds: false, driveCapacityMb: 100, - maxFileSizeMb: 10, + maxFileSizeMb: 25, alwaysMarkNsfw: false, canUpdateBioMedia: true, pinLimit: 5, diff --git a/packages/backend/src/core/UserBlockingService.ts b/packages/backend/src/core/UserBlockingService.ts index 8da1bb2092..1a1e7c4778 100644 --- a/packages/backend/src/core/UserBlockingService.ts +++ b/packages/backend/src/core/UserBlockingService.ts @@ -77,8 +77,10 @@ export class UserBlockingService implements OnModuleInit { await this.blockingsRepository.insert(blocking); - this.cacheService.userBlockingCache.refresh(blocker.id); - this.cacheService.userBlockedCache.refresh(blockee.id); + await Promise.all([ + this.cacheService.userBlockingCache.delete(blocker.id), + this.cacheService.userBlockedCache.delete(blockee.id), + ]); this.globalEventService.publishInternalEvent('blockingCreated', { blockerId: blocker.id, @@ -168,8 +170,10 @@ export class UserBlockingService implements OnModuleInit { await this.blockingsRepository.delete(blocking.id); - this.cacheService.userBlockingCache.refresh(blocker.id); - this.cacheService.userBlockedCache.refresh(blockee.id); + await Promise.all([ + this.cacheService.userBlockingCache.delete(blocker.id), + this.cacheService.userBlockedCache.delete(blockee.id), + ]); this.globalEventService.publishInternalEvent('blockingDeleted', { blockerId: blocker.id, diff --git a/packages/backend/src/core/UserFollowingService.ts b/packages/backend/src/core/UserFollowingService.ts index 897b950022..8470872eac 100644 --- a/packages/backend/src/core/UserFollowingService.ts +++ b/packages/backend/src/core/UserFollowingService.ts @@ -29,6 +29,7 @@ import { AccountMoveService } from '@/core/AccountMoveService.js'; import { UtilityService } from '@/core/UtilityService.js'; import type { ThinUser } from '@/queue/types.js'; import { LoggerService } from '@/core/LoggerService.js'; +import { InternalEventService } from '@/core/InternalEventService.js'; import type Logger from '../logger.js'; type Local = MiLocalUser | { @@ -86,6 +87,7 @@ export class UserFollowingService implements OnModuleInit { private accountMoveService: AccountMoveService, private perUserFollowingChart: PerUserFollowingChart, private instanceChart: InstanceChart, + private readonly internalEventService: InternalEventService, loggerService: LoggerService, ) { @@ -145,12 +147,7 @@ export class UserFollowingService implements OnModuleInit { if (blocked) throw new IdentifiableError('3338392a-f764-498d-8855-db939dcf8c48', 'blocked'); } - if (await this.followingsRepository.exists({ - where: { - followerId: follower.id, - followeeId: followee.id, - }, - })) { + if (await this.cacheService.isFollowing(follower, followee)) { // すでにフォロー関係が存在している場合 if (this.userEntityService.isRemoteUser(follower) && this.userEntityService.isLocalUser(followee)) { // リモート → ローカル: acceptを送り返しておしまい @@ -178,24 +175,14 @@ export class UserFollowingService implements OnModuleInit { let autoAccept = false; // 鍵アカウントであっても、既にフォローされていた場合はスルー - const isFollowing = await this.followingsRepository.exists({ - where: { - followerId: follower.id, - followeeId: followee.id, - }, - }); + const isFollowing = await this.cacheService.isFollowing(follower, followee); if (isFollowing) { autoAccept = true; } // フォローしているユーザーは自動承認オプション if (!autoAccept && (this.userEntityService.isLocalUser(followee) && followeeProfile.autoAcceptFollowed)) { - const isFollowed = await this.followingsRepository.exists({ - where: { - followerId: followee.id, - followeeId: follower.id, - }, - }); + const isFollowed = await this.cacheService.isFollowing(followee, follower); // intentionally reversed parameters if (isFollowed) autoAccept = true; } @@ -204,12 +191,7 @@ export class UserFollowingService implements OnModuleInit { if (followee.isLocked && !autoAccept) { autoAccept = !!(await this.accountMoveService.validateAlsoKnownAs( follower, - (oldSrc, newSrc) => this.followingsRepository.exists({ - where: { - followeeId: followee.id, - followerId: newSrc.id, - }, - }), + (oldSrc, newSrc) => this.cacheService.isFollowing(newSrc, followee), true, )); } @@ -264,7 +246,8 @@ export class UserFollowingService implements OnModuleInit { } }); - this.cacheService.userFollowingsCache.refresh(follower.id); + // Handled by CacheService + //this.cacheService.userFollowingsCache.refresh(follower.id); const requestExist = await this.followRequestsRepository.exists({ where: { @@ -291,7 +274,7 @@ export class UserFollowingService implements OnModuleInit { }, followee.id); } - this.globalEventService.publishInternalEvent('follow', { followerId: follower.id, followeeId: followee.id }); + await this.internalEventService.emit('follow', { followerId: follower.id, followeeId: followee.id }); const [followeeUser, followerUser] = await Promise.all([ this.usersRepository.findOneByOrFail({ id: followee.id }), @@ -363,31 +346,29 @@ export class UserFollowingService implements OnModuleInit { }, silent = false, ): Promise<void> { - const following = await this.followingsRepository.findOne({ - relations: { - follower: true, - followee: true, - }, - where: { - followerId: follower.id, - followeeId: followee.id, - }, - }); + const [ + followerUser, + followeeUser, + following, + ] = await Promise.all([ + this.cacheService.findUserById(follower.id), + this.cacheService.findUserById(followee.id), + this.cacheService.userFollowingsCache.fetch(follower.id).then(fs => fs.get(followee.id)), + ]); - if (following === null || !following.follower || !following.followee) { + if (following == null) { this.logger.warn('フォロー解除がリクエストされましたがフォローしていませんでした'); return; } await this.followingsRepository.delete(following.id); + await this.internalEventService.emit('unfollow', { followerId: follower.id, followeeId: followee.id }); - this.cacheService.userFollowingsCache.refresh(follower.id); - - this.decrementFollowing(following.follower, following.followee); + this.decrementFollowing(followerUser, followeeUser); if (!silent && this.userEntityService.isLocalUser(follower)) { // Publish unfollow event - this.userEntityService.pack(followee.id, follower, { + this.userEntityService.pack(followeeUser, follower, { schema: 'UserDetailedNotMe', }).then(async packed => { this.globalEventService.publishMainStream(follower.id, 'unfollow', packed); @@ -412,8 +393,6 @@ export class UserFollowingService implements OnModuleInit { follower: MiUser, followee: MiUser, ): Promise<void> { - this.globalEventService.publishInternalEvent('unfollow', { followerId: follower.id, followeeId: followee.id }); - // Neither followee nor follower has moved. if (!follower.movedToUri && !followee.movedToUri) { //#region Decrement following / followers counts @@ -687,22 +666,22 @@ export class UserFollowingService implements OnModuleInit { */ @bindThis private async removeFollow(followee: Both, follower: Both): Promise<void> { - const following = await this.followingsRepository.findOne({ - relations: { - followee: true, - follower: true, - }, - where: { - followeeId: followee.id, - followerId: follower.id, - }, - }); + const [ + followerUser, + followeeUser, + following, + ] = await Promise.all([ + this.cacheService.findUserById(follower.id), + this.cacheService.findUserById(followee.id), + this.cacheService.userFollowingsCache.fetch(follower.id).then(fs => fs.get(followee.id)), + ]); - if (!following || !following.followee || !following.follower) return; + if (!following) return; await this.followingsRepository.delete(following.id); + await this.internalEventService.emit('unfollow', { followerId: follower.id, followeeId: followee.id }); - this.decrementFollowing(following.follower, following.followee); + this.decrementFollowing(followerUser, followeeUser); } /** @@ -733,36 +712,26 @@ export class UserFollowingService implements OnModuleInit { } @bindThis - public getFollowees(userId: MiUser['id']) { - return this.followingsRepository.createQueryBuilder('following') - .select('following.followeeId') - .where('following.followerId = :followerId', { followerId: userId }) - .getMany(); + public async getFollowees(userId: MiUser['id']) { + const followings = await this.cacheService.userFollowingsCache.fetch(userId); + return Array.from(followings.values()); } @bindThis - public isFollowing(followerId: MiUser['id'], followeeId: MiUser['id']) { - return this.followingsRepository.exists({ - where: { - followerId, - followeeId, - }, - }); + public async isFollowing(followerId: MiUser['id'], followeeId: MiUser['id']) { + return this.cacheService.isFollowing(followerId, followeeId); } @bindThis public async isMutual(aUserId: MiUser['id'], bUserId: MiUser['id']) { - const count = await this.followingsRepository.createQueryBuilder('following') - .where(new Brackets(qb => { - qb.where('following.followerId = :aUserId', { aUserId }) - .andWhere('following.followeeId = :bUserId', { bUserId }); - })) - .orWhere(new Brackets(qb => { - qb.where('following.followerId = :bUserId', { bUserId }) - .andWhere('following.followeeId = :aUserId', { aUserId }); - })) - .getCount(); + const [ + isFollowing, + isFollowed, + ] = await Promise.all([ + this.isFollowing(aUserId, bUserId), + this.isFollowing(bUserId, aUserId), + ]); - return count === 2; + return isFollowing && isFollowed; } } diff --git a/packages/backend/src/core/UserKeypairService.ts b/packages/backend/src/core/UserKeypairService.ts index 92d61cd103..d8a67d273b 100644 --- a/packages/backend/src/core/UserKeypairService.ts +++ b/packages/backend/src/core/UserKeypairService.ts @@ -7,14 +7,14 @@ import { Inject, Injectable, OnApplicationShutdown } from '@nestjs/common'; import * as Redis from 'ioredis'; import type { MiUser } from '@/models/User.js'; import type { UserKeypairsRepository } from '@/models/_.js'; -import { RedisKVCache } from '@/misc/cache.js'; +import { MemoryKVCache, RedisKVCache } from '@/misc/cache.js'; import type { MiUserKeypair } from '@/models/UserKeypair.js'; import { DI } from '@/di-symbols.js'; import { bindThis } from '@/decorators.js'; @Injectable() export class UserKeypairService implements OnApplicationShutdown { - private cache: RedisKVCache<MiUserKeypair>; + private cache: MemoryKVCache<MiUserKeypair>; constructor( @Inject(DI.redis) @@ -23,18 +23,12 @@ export class UserKeypairService implements OnApplicationShutdown { @Inject(DI.userKeypairsRepository) private userKeypairsRepository: UserKeypairsRepository, ) { - this.cache = new RedisKVCache<MiUserKeypair>(this.redisClient, 'userKeypair', { - lifetime: 1000 * 60 * 60 * 24, // 24h - memoryCacheLifetime: 1000 * 60 * 60, // 1h - fetcher: (key) => this.userKeypairsRepository.findOneByOrFail({ userId: key }), - toRedisConverter: (value) => JSON.stringify(value), - fromRedisConverter: (value) => JSON.parse(value), - }); + this.cache = new MemoryKVCache<MiUserKeypair>(1000 * 60 * 60 * 24); // 24h } @bindThis public async getUserKeypair(userId: MiUser['id']): Promise<MiUserKeypair> { - return await this.cache.fetch(userId); + return await this.cache.fetch(userId, () => this.userKeypairsRepository.findOneByOrFail({ userId })); } @bindThis diff --git a/packages/backend/src/core/UserListService.ts b/packages/backend/src/core/UserListService.ts index e7200ab1bf..b4486b9808 100644 --- a/packages/backend/src/core/UserListService.ts +++ b/packages/backend/src/core/UserListService.ts @@ -11,21 +11,22 @@ import type { MiUser } from '@/models/User.js'; import type { MiUserList } from '@/models/UserList.js'; import type { MiUserListMembership } from '@/models/UserListMembership.js'; import { IdService } from '@/core/IdService.js'; -import type { GlobalEvents } from '@/core/GlobalEventService.js'; +import type { GlobalEvents, InternalEventTypes } from '@/core/GlobalEventService.js'; import { GlobalEventService } from '@/core/GlobalEventService.js'; import { DI } from '@/di-symbols.js'; import { UserEntityService } from '@/core/entities/UserEntityService.js'; import { bindThis } from '@/decorators.js'; import { QueueService } from '@/core/QueueService.js'; -import { RedisKVCache } from '@/misc/cache.js'; +import { QuantumKVCache } from '@/misc/QuantumKVCache.js'; import { RoleService } from '@/core/RoleService.js'; import { SystemAccountService } from '@/core/SystemAccountService.js'; +import { InternalEventService } from '@/core/InternalEventService.js'; @Injectable() export class UserListService implements OnApplicationShutdown, OnModuleInit { public static TooManyUsersError = class extends Error {}; - public membersCache: RedisKVCache<Set<string>>; + public membersCache: QuantumKVCache<Set<string>>; private roleService: RoleService; constructor( @@ -48,16 +49,15 @@ export class UserListService implements OnApplicationShutdown, OnModuleInit { private globalEventService: GlobalEventService, private queueService: QueueService, private systemAccountService: SystemAccountService, + private readonly internalEventService: InternalEventService, ) { - this.membersCache = new RedisKVCache<Set<string>>(this.redisClient, 'userListMembers', { + this.membersCache = new QuantumKVCache<Set<string>>(this.internalEventService, 'userListMembers', { lifetime: 1000 * 60 * 30, // 30m - memoryCacheLifetime: 1000 * 60, // 1m fetcher: (key) => this.userListMembershipsRepository.find({ where: { userListId: key }, select: ['userId'] }).then(xs => new Set(xs.map(x => x.userId))), - toRedisConverter: (value) => JSON.stringify(Array.from(value)), - fromRedisConverter: (value) => new Set(JSON.parse(value)), }); - this.redisForSub.on('message', this.onMessage); + this.internalEventService.on('userListMemberAdded', this.onMessage); + this.internalEventService.on('userListMemberRemoved', this.onMessage); } async onModuleInit() { @@ -65,15 +65,12 @@ export class UserListService implements OnApplicationShutdown, OnModuleInit { } @bindThis - private async onMessage(_: string, data: string): Promise<void> { - const obj = JSON.parse(data); - - if (obj.channel === 'internal') { - const { type, body } = obj.message as GlobalEvents['internal']['payload']; + private async onMessage<E extends 'userListMemberAdded' | 'userListMemberRemoved'>(body: InternalEventTypes[E], type: E): Promise<void> { + { switch (type) { case 'userListMemberAdded': { const { userListId, memberId } = body; - const members = await this.membersCache.get(userListId); + const members = this.membersCache.get(userListId); if (members) { members.add(memberId); } @@ -81,7 +78,7 @@ export class UserListService implements OnApplicationShutdown, OnModuleInit { } case 'userListMemberRemoved': { const { userListId, memberId } = body; - const members = await this.membersCache.get(userListId); + const members = this.membersCache.get(userListId); if (members) { members.delete(memberId); } @@ -150,7 +147,8 @@ export class UserListService implements OnApplicationShutdown, OnModuleInit { @bindThis public dispose(): void { - this.redisForSub.off('message', this.onMessage); + this.internalEventService.off('userListMemberAdded', this.onMessage); + this.internalEventService.off('userListMemberRemoved', this.onMessage); this.membersCache.dispose(); } diff --git a/packages/backend/src/core/UserMutingService.ts b/packages/backend/src/core/UserMutingService.ts index 06643be5fb..c15a979d0f 100644 --- a/packages/backend/src/core/UserMutingService.ts +++ b/packages/backend/src/core/UserMutingService.ts @@ -32,7 +32,7 @@ export class UserMutingService { muteeId: target.id, }); - this.cacheService.userMutingsCache.refresh(user.id); + await this.cacheService.userMutingsCache.delete(user.id); } @bindThis @@ -43,9 +43,6 @@ export class UserMutingService { id: In(mutings.map(m => m.id)), }); - const muterIds = [...new Set(mutings.map(m => m.muterId))]; - for (const muterId of muterIds) { - this.cacheService.userMutingsCache.refresh(muterId); - } + await this.cacheService.userMutingsCache.deleteMany(mutings.map(m => m.muterId)); } } diff --git a/packages/backend/src/core/UserRenoteMutingService.ts b/packages/backend/src/core/UserRenoteMutingService.ts index bdc5e23f4b..7c0693f216 100644 --- a/packages/backend/src/core/UserRenoteMutingService.ts +++ b/packages/backend/src/core/UserRenoteMutingService.ts @@ -33,7 +33,7 @@ export class UserRenoteMutingService { muteeId: target.id, }); - await this.cacheService.renoteMutingsCache.refresh(user.id); + await this.cacheService.renoteMutingsCache.delete(user.id); } @bindThis @@ -44,9 +44,6 @@ export class UserRenoteMutingService { id: In(mutings.map(m => m.id)), }); - const muterIds = [...new Set(mutings.map(m => m.muterId))]; - for (const muterId of muterIds) { - await this.cacheService.renoteMutingsCache.refresh(muterId); - } + await this.cacheService.renoteMutingsCache.deleteMany(mutings.map(m => m.muterId)); } } diff --git a/packages/backend/src/core/UserService.ts b/packages/backend/src/core/UserService.ts index 1f471513f3..4a04910105 100644 --- a/packages/backend/src/core/UserService.ts +++ b/packages/backend/src/core/UserService.ts @@ -10,6 +10,7 @@ import { DI } from '@/di-symbols.js'; import { bindThis } from '@/decorators.js'; import { SystemWebhookService } from '@/core/SystemWebhookService.js'; import { UserEntityService } from '@/core/entities/UserEntityService.js'; +import { CacheService } from '@/core/CacheService.js'; @Injectable() export class UserService { @@ -20,6 +21,7 @@ export class UserService { private followingsRepository: FollowingsRepository, private systemWebhookService: SystemWebhookService, private userEntityService: UserEntityService, + private readonly cacheService: CacheService, ) { } @@ -38,14 +40,17 @@ export class UserService { }); const wokeUp = result.isHibernated; if (wokeUp) { - this.usersRepository.update(user.id, { - isHibernated: false, - }); - this.followingsRepository.update({ - followerId: user.id, - }, { - isFollowerHibernated: false, - }); + await Promise.all([ + this.usersRepository.update(user.id, { + isHibernated: false, + }), + this.followingsRepository.update({ + followerId: user.id, + }, { + isFollowerHibernated: false, + }), + this.cacheService.hibernatedUserCache.set(user.id, false), + ]); } } else { this.usersRepository.update(user.id, { diff --git a/packages/backend/src/core/UserSuspendService.ts b/packages/backend/src/core/UserSuspendService.ts index 30dcaa6f7d..ddadab7022 100644 --- a/packages/backend/src/core/UserSuspendService.ts +++ b/packages/backend/src/core/UserSuspendService.ts @@ -6,7 +6,7 @@ import { Inject, Injectable } from '@nestjs/common'; import { Not, IsNull } from 'typeorm'; import type { FollowingsRepository, FollowRequestsRepository, UsersRepository } from '@/models/_.js'; -import type { MiUser } from '@/models/User.js'; +import { MiUser } from '@/models/User.js'; import { QueueService } from '@/core/QueueService.js'; import { GlobalEventService } from '@/core/GlobalEventService.js'; import { DI } from '@/di-symbols.js'; @@ -16,9 +16,16 @@ import { bindThis } from '@/decorators.js'; import { RelationshipJobData } from '@/queue/types.js'; import { ModerationLogService } from '@/core/ModerationLogService.js'; import { isSystemAccount } from '@/misc/is-system-account.js'; +import { CacheService } from '@/core/CacheService.js'; +import { LoggerService } from '@/core/LoggerService.js'; +import type Logger from '@/logger.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; +import { trackPromise } from '@/misc/promise-tracker.js'; @Injectable() export class UserSuspendService { + private readonly logger: Logger; + constructor( @Inject(DI.usersRepository) private usersRepository: UsersRepository, @@ -34,7 +41,11 @@ export class UserSuspendService { private globalEventService: GlobalEventService, private apRendererService: ApRendererService, private moderationLogService: ModerationLogService, + private readonly cacheService: CacheService, + + loggerService: LoggerService, ) { + this.logger = loggerService.getLogger('user-suspend'); } @bindThis @@ -45,16 +56,16 @@ export class UserSuspendService { isSuspended: true, }); - this.moderationLogService.log(moderator, 'suspend', { + await this.moderationLogService.log(moderator, 'suspend', { userId: user.id, userUsername: user.username, userHost: user.host, }); - (async () => { - await this.postSuspend(user).catch(e => {}); - await this.unFollowAll(user).catch(e => {}); - })(); + trackPromise((async () => { + await this.postSuspend(user); + await this.freezeAll(user); + })().catch(e => this.logger.error(`Error suspending user ${user.id}: ${renderInlineError(e)}`))); } @bindThis @@ -63,33 +74,36 @@ export class UserSuspendService { isSuspended: false, }); - this.moderationLogService.log(moderator, 'unsuspend', { + await this.moderationLogService.log(moderator, 'unsuspend', { userId: user.id, userUsername: user.username, userHost: user.host, }); - (async () => { - await this.postUnsuspend(user).catch(e => {}); - })(); + trackPromise((async () => { + await this.postUnsuspend(user); + await this.unFreezeAll(user); + })().catch(e => this.logger.error(`Error un-suspending for user ${user.id}: ${renderInlineError(e)}`))); } @bindThis private async postSuspend(user: { id: MiUser['id']; host: MiUser['host'] }): Promise<void> { this.globalEventService.publishInternalEvent('userChangeSuspendedState', { id: user.id, isSuspended: true }); + /* this.followRequestsRepository.delete({ followeeId: user.id, }); this.followRequestsRepository.delete({ followerId: user.id, }); + */ if (this.userEntityService.isLocalUser(user)) { // 知り得る全SharedInboxにDelete配信 const content = this.apRendererService.addContext(this.apRendererService.renderDelete(this.userEntityService.genLocalUserUri(user.id), user)); - const queue: string[] = []; + const queue = new Map<string, boolean>(); const followings = await this.followingsRepository.find({ where: [ @@ -102,12 +116,12 @@ export class UserSuspendService { const inboxes = followings.map(x => x.followerSharedInbox ?? x.followeeSharedInbox); for (const inbox of inboxes) { - if (inbox != null && !queue.includes(inbox)) queue.push(inbox); + if (inbox != null) { + queue.set(inbox, true); + } } - for (const inbox of queue) { - this.queueService.deliver(user, content, inbox, true); - } + await this.queueService.deliverMany(user, content, queue); } } @@ -119,7 +133,7 @@ export class UserSuspendService { // 知り得る全SharedInboxにUndo Delete配信 const content = this.apRendererService.addContext(this.apRendererService.renderUndo(this.apRendererService.renderDelete(this.userEntityService.genLocalUserUri(user.id), user), user)); - const queue: string[] = []; + const queue = new Map<string, boolean>(); const followings = await this.followingsRepository.find({ where: [ @@ -132,23 +146,19 @@ export class UserSuspendService { const inboxes = followings.map(x => x.followerSharedInbox ?? x.followeeSharedInbox); for (const inbox of inboxes) { - if (inbox != null && !queue.includes(inbox)) queue.push(inbox); + if (inbox != null) { + queue.set(inbox, true); + } } - for (const inbox of queue) { - this.queueService.deliver(user as any, content, inbox, true); - } + await this.queueService.deliverMany(user, content, queue); } } @bindThis private async unFollowAll(follower: MiUser) { - const followings = await this.followingsRepository.find({ - where: { - followerId: follower.id, - followeeId: Not(IsNull()), - }, - }); + const followings = await this.cacheService.userFollowingsCache.fetch(follower.id) + .then(fs => Array.from(fs.values()).filter(f => f.followeeHost != null)); const jobs: RelationshipJobData[] = []; for (const following of followings) { @@ -162,4 +172,36 @@ export class UserSuspendService { } this.queueService.createUnfollowJob(jobs); } + + @bindThis + private async freezeAll(user: MiUser): Promise<void> { + // Freeze follow relations with all remote users + await this.followingsRepository + .createQueryBuilder('following') + .orWhere({ + followeeId: user.id, + followerHost: Not(IsNull()), + }) + .update({ + isFollowerHibernated: true, + }) + .execute(); + } + + @bindThis + private async unFreezeAll(user: MiUser): Promise<void> { + // Restore follow relations with all remote users + await this.followingsRepository + .createQueryBuilder('following') + .innerJoin(MiUser, 'follower', 'user.id = following.followerId') + .andWhere('follower.isHibernated = false') // Don't unfreeze if the follower is *actually* frozen + .andWhere({ + followeeId: user.id, + followerHost: Not(IsNull()), + }) + .update({ + isFollowerHibernated: false, + }) + .execute(); + } } diff --git a/packages/backend/src/core/UtilityService.ts b/packages/backend/src/core/UtilityService.ts index 170afc72dc..3098367392 100644 --- a/packages/backend/src/core/UtilityService.ts +++ b/packages/backend/src/core/UtilityService.ts @@ -49,22 +49,49 @@ export class UtilityService { return regexp.test(email); } + public isBlockedHost(host: string | null): boolean; + public isBlockedHost(blockedHosts: string[], host: string | null): boolean; @bindThis - public isBlockedHost(blockedHosts: string[], host: string | null): boolean { + public isBlockedHost(blockedHostsOrHost: string[] | string | null, host?: string | null): boolean { + const blockedHosts = Array.isArray(blockedHostsOrHost) ? blockedHostsOrHost : this.meta.blockedHosts; + host = Array.isArray(blockedHostsOrHost) ? host : blockedHostsOrHost; + if (host == null) return false; return blockedHosts.some(x => `.${host.toLowerCase()}`.endsWith(`.${x}`)); } + public isSilencedHost(host: string | null): boolean; + public isSilencedHost(silencedHosts: string[], host: string | null): boolean; @bindThis - public isSilencedHost(silencedHosts: string[] | undefined, host: string | null): boolean { - if (!silencedHosts || host == null) return false; + public isSilencedHost(silencedHostsOrHost: string[] | string | null, host?: string | null): boolean { + const silencedHosts = Array.isArray(silencedHostsOrHost) ? silencedHostsOrHost : this.meta.silencedHosts; + host = Array.isArray(silencedHostsOrHost) ? host : silencedHostsOrHost; + + if (host == null) return false; return silencedHosts.some(x => `.${host.toLowerCase()}`.endsWith(`.${x}`)); } + public isMediaSilencedHost(host: string | null): boolean; + public isMediaSilencedHost(silencedHosts: string[], host: string | null): boolean; @bindThis - public isMediaSilencedHost(silencedHosts: string[] | undefined, host: string | null): boolean { - if (!silencedHosts || host == null) return false; - return silencedHosts.some(x => `.${host.toLowerCase()}`.endsWith(`.${x}`)); + public isMediaSilencedHost(mediaSilencedHostsOrHost: string[] | string | null, host?: string | null): boolean { + const mediaSilencedHosts = Array.isArray(mediaSilencedHostsOrHost) ? mediaSilencedHostsOrHost : this.meta.mediaSilencedHosts; + host = Array.isArray(mediaSilencedHostsOrHost) ? host : mediaSilencedHostsOrHost; + + if (host == null) return false; + return mediaSilencedHosts.some(x => `.${host.toLowerCase()}`.endsWith(`.${x}`)); + } + + @bindThis + public isAllowListedHost(host: string | null): boolean { + if (host == null) return false; + return this.meta.federationHosts.some(x => `.${host.toLowerCase()}`.endsWith(`.${x}`)); + } + + @bindThis + public isBubbledHost(host: string | null): boolean { + if (host == null) return false; + return this.meta.bubbleInstances.some(x => `.${host.toLowerCase()}`.endsWith(`.${x}`)); } @bindThis diff --git a/packages/backend/src/core/VideoProcessingService.ts b/packages/backend/src/core/VideoProcessingService.ts index 747fe4fc7e..3e4fd6a4b0 100644 --- a/packages/backend/src/core/VideoProcessingService.ts +++ b/packages/backend/src/core/VideoProcessingService.ts @@ -3,24 +3,41 @@ * SPDX-License-Identifier: AGPL-3.0-only */ +import fs from 'node:fs/promises'; import { Inject, Injectable } from '@nestjs/common'; import FFmpeg from 'fluent-ffmpeg'; import { DI } from '@/di-symbols.js'; import type { Config } from '@/config.js'; import { ImageProcessingService } from '@/core/ImageProcessingService.js'; import type { IImage } from '@/core/ImageProcessingService.js'; -import { createTempDir } from '@/misc/create-temp.js'; +import { createTemp, createTempDir } from '@/misc/create-temp.js'; import { bindThis } from '@/decorators.js'; import { appendQuery, query } from '@/misc/prelude/url.js'; +import { LoggerService } from '@/core/LoggerService.js'; +import type Logger from '@/logger.js'; + +// faststart is only supported for MP4, M4A, M4W and MOV files (the MOV family). +// WebM (and Matroska) files always support faststart-like behavior. +const supportedMimeTypes = new Map([ + ['video/mp4', 'mp4'], + ['video/m4a', 'mp4'], + ['video/m4v', 'mp4'], + ['video/quicktime', 'mov'], +]); @Injectable() export class VideoProcessingService { + private readonly logger: Logger; + constructor( @Inject(DI.config) private config: Config, private imageProcessingService: ImageProcessingService, + + private loggerService: LoggerService, ) { + this.logger = this.loggerService.getLogger('video-processing'); } @bindThis @@ -60,5 +77,50 @@ export class VideoProcessingService { }), ); } + + /** + * Optimize video for web playback by adding faststart flag. + * This allows the video to start playing before it is fully downloaded. + * The original file is modified in-place. + * @param source Path to the video file + * @param mimeType The MIME type of the video + * @returns Promise that resolves when optimization is complete + */ + @bindThis + public async webOptimizeVideo(source: string, mimeType: string): Promise<void> { + const outputFormat = supportedMimeTypes.get(mimeType); + if (!outputFormat) { + this.logger.debug(`Skipping web optimization for unsupported MIME type: ${mimeType}`); + return; + } + + const [tempPath, cleanup] = await createTemp(); + + try { + await new Promise<void>((resolve, reject) => { + FFmpeg(source) + .format(outputFormat) // Specify output format + .addOutputOptions('-c copy') // Copy streams without re-encoding + .addOutputOptions('-movflags +faststart') + .on('error', reject) + .on('end', async () => { + try { + // Replace original file with optimized version + await fs.copyFile(tempPath, source); + this.logger.info(`Web-optimized video: ${source}`); + resolve(); + } catch (copyError) { + reject(copyError); + } + }) + .save(tempPath); + }); + } catch (error) { + this.logger.warn(`Failed to web-optimize video: ${source}`, { error }); + throw error; + } finally { + cleanup(); + } + } } diff --git a/packages/backend/src/core/WebAuthnService.ts b/packages/backend/src/core/WebAuthnService.ts index 372e1e2ab7..afd1d68ce4 100644 --- a/packages/backend/src/core/WebAuthnService.ts +++ b/packages/backend/src/core/WebAuthnService.ts @@ -17,6 +17,8 @@ import type { Config } from '@/config.js'; import { bindThis } from '@/decorators.js'; import { MiUser } from '@/models/_.js'; import { IdentifiableError } from '@/misc/identifiable-error.js'; +import { LoggerService } from '@/core/LoggerService.js'; +import Logger from '@/logger.js'; import type { AuthenticationResponseJSON, AuthenticatorTransportFuture, @@ -28,6 +30,8 @@ import type { @Injectable() export class WebAuthnService { + private readonly logger: Logger; + constructor( @Inject(DI.config) private config: Config, @@ -40,7 +44,9 @@ export class WebAuthnService { @Inject(DI.userSecurityKeysRepository) private userSecurityKeysRepository: UserSecurityKeysRepository, + loggerService: LoggerService, ) { + this.logger = loggerService.getLogger('web-authn'); } @bindThis @@ -114,8 +120,8 @@ export class WebAuthnService { requireUserVerification: true, }); } catch (error) { - console.error(error); - throw new IdentifiableError('5c1446f8-8ca7-4d31-9f39-656afe9c5d87', 'verification failed'); + this.logger.error(error as Error, 'Error authenticating webauthn'); + throw new IdentifiableError('5c1446f8-8ca7-4d31-9f39-656afe9c5d87', 'verification failed', true, error); } const { verified } = verification; @@ -221,7 +227,7 @@ export class WebAuthnService { requireUserVerification: true, }); } catch (error) { - throw new IdentifiableError('b18c89a7-5b5e-4cec-bb5b-0419f332d430', `verification failed: ${error}`); + throw new IdentifiableError('b18c89a7-5b5e-4cec-bb5b-0419f332d430', `verification failed`, true, error); } const { verified, authenticationInfo } = verification; @@ -301,8 +307,8 @@ export class WebAuthnService { requireUserVerification: true, }); } catch (error) { - console.error(error); - throw new IdentifiableError('b18c89a7-5b5e-4cec-bb5b-0419f332d430', 'verification failed'); + this.logger.error(error as Error, 'Error authenticating webauthn'); + throw new IdentifiableError('b18c89a7-5b5e-4cec-bb5b-0419f332d430', 'verification failed', true, error); } const { verified, authenticationInfo } = verification; diff --git a/packages/backend/src/core/WebfingerService.ts b/packages/backend/src/core/WebfingerService.ts index f57e7a2c1f..bb9f0be4c6 100644 --- a/packages/backend/src/core/WebfingerService.ts +++ b/packages/backend/src/core/WebfingerService.ts @@ -5,10 +5,11 @@ import { URL } from 'node:url'; import { Injectable } from '@nestjs/common'; -import { XMLParser } from 'fast-xml-parser'; +import { load as cheerio } from 'cheerio/slim'; import { HttpRequestService } from '@/core/HttpRequestService.js'; import { bindThis } from '@/decorators.js'; import type Logger from '@/logger.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; import { RemoteLoggerService } from './RemoteLoggerService.js'; export type ILink = { @@ -100,16 +101,14 @@ export class WebfingerService { private async fetchWebFingerTemplateFromHostMeta(url: string): Promise<string | null> { try { const res = await this.httpRequestService.getHtml(url, 'application/xrd+xml'); - const options = { - ignoreAttributes: false, - isArray: (_name: string, jpath: string) => jpath === 'XRD.Link', - }; - const parser = new XMLParser(options); - const hostMeta = parser.parse(res); - const template = (hostMeta['XRD']['Link'] as Array<any>).filter(p => p['@_rel'] === 'lrdd')[0]['@_template']; - return template.indexOf('{uri}') < 0 ? null : template; + const hostMeta = cheerio(res, { + xml: true, + }); + + const template = hostMeta('XRD > Link[rel="lrdd"][template*="{uri}"]').attr('template'); + return template ?? null; } catch (err) { - this.logger.error(`error while request host-meta for ${url}: ${err}`); + this.logger.error(`error while request host-meta for ${url}: ${renderInlineError(err)}`); return null; } } diff --git a/packages/backend/src/core/WebhookTestService.ts b/packages/backend/src/core/WebhookTestService.ts index 2f8cfea7f7..8dc42e45c0 100644 --- a/packages/backend/src/core/WebhookTestService.ts +++ b/packages/backend/src/core/WebhookTestService.ts @@ -13,6 +13,7 @@ import { type WebhookEventTypes } from '@/models/Webhook.js'; import { CustomEmojiService } from '@/core/CustomEmojiService.js'; import { type UserWebhookPayload, UserWebhookService } from '@/core/UserWebhookService.js'; import { QueueService } from '@/core/QueueService.js'; +import { IdService } from '@/core/IdService.js'; import { ModeratorInactivityRemainingTime } from '@/queue/processors/CheckModeratorsActivityProcessorService.js'; const oneDayMillis = 24 * 60 * 60 * 1000; @@ -63,6 +64,7 @@ function generateDummyUser(override?: Partial<MiUser>): MiUser { emojis: [], score: 0, host: null, + instance: null, inbox: null, sharedInbox: null, featured: null, @@ -76,6 +78,8 @@ function generateDummyUser(override?: Partial<MiUser>): MiUser { mandatoryCW: null, rejectQuotes: false, allowUnsignedFetch: 'staff', + userProfile: null, + attributionDomains: [], ...override, }; } @@ -114,10 +118,13 @@ function generateDummyNote(override?: Partial<MiNote>): MiNote { channelId: null, channel: null, userHost: null, + userInstance: null, replyUserId: null, replyUserHost: null, + replyUserInstance: null, renoteUserId: null, renoteUserHost: null, + renoteUserInstance: null, updatedAt: null, processErrors: [], ...override, @@ -160,6 +167,7 @@ export class WebhookTestService { private userWebhookService: UserWebhookService, private systemWebhookService: SystemWebhookService, private queueService: QueueService, + private readonly idService: IdService, ) { } @@ -358,8 +366,10 @@ export class WebhookTestService { id: 'dummy-abuse-report1', targetUserId: 'dummy-target-user', targetUser: null, + targetUserInstance: null, reporterId: 'dummy-reporter-user', reporter: null, + reporterInstance: null, assigneeId: null, assignee: null, resolved: false, @@ -441,6 +451,8 @@ export class WebhookTestService { offsetX: it.offsetX, offsetY: it.offsetY, })), + createdAt: this.idService.parse(user.id).date.toISOString(), + description: '', isBot: user.isBot, isCat: user.isCat, emojis: await this.customEmojiService.populateEmojis(user.emojis, user.host), @@ -449,6 +461,7 @@ export class WebhookTestService { isAdmin: false, isModerator: false, isSystem: false, + instance: undefined, ...override, }; } diff --git a/packages/backend/src/core/activitypub/ApDbResolverService.ts b/packages/backend/src/core/activitypub/ApDbResolverService.ts index d8aa80f5b7..e9e0dde9cd 100644 --- a/packages/backend/src/core/activitypub/ApDbResolverService.ts +++ b/packages/backend/src/core/activitypub/ApDbResolverService.ts @@ -165,18 +165,23 @@ export class ApDbResolverService implements OnApplicationShutdown { */ @bindThis public async refetchPublicKeyForApId(user: MiRemoteUser): Promise<MiUserPublickey | null> { - this.apLoggerService.logger.debug('Re-fetching public key for user', { userId: user.id, uri: user.uri }); - await this.apPersonService.updatePerson(user.uri); + this.apLoggerService.logger.debug(`Updating public key for user ${user.id} (${user.uri})`); - const key = await this.apPersonService.findPublicKeyByUserId(user.id); + const oldKey = await this.apPersonService.findPublicKeyByUserId(user.id); + await this.apPersonService.updatePerson(user.uri); + const newKey = await this.apPersonService.findPublicKeyByUserId(user.id); - if (key) { - this.apLoggerService.logger.info('Re-fetched public key for user', { userId: user.id, uri: user.uri }); + if (newKey) { + if (oldKey && newKey.keyPem === oldKey.keyPem) { + this.apLoggerService.logger.debug(`Public key is up-to-date for user ${user.id} (${user.uri})`); + } else { + this.apLoggerService.logger.info(`Updated public key for user ${user.id} (${user.uri})`); + } } else { - this.apLoggerService.logger.warn('Failed to re-fetch key for user', { userId: user.id, uri: user.uri }); + this.apLoggerService.logger.warn(`Failed to update public key for user ${user.id} (${user.uri})`); } - return key; + return newKey ?? oldKey; } @bindThis diff --git a/packages/backend/src/core/activitypub/ApDeliverManagerService.ts b/packages/backend/src/core/activitypub/ApDeliverManagerService.ts index eaa592b9e0..91f6f2d9fc 100644 --- a/packages/backend/src/core/activitypub/ApDeliverManagerService.ts +++ b/packages/backend/src/core/activitypub/ApDeliverManagerService.ts @@ -5,7 +5,6 @@ import { Inject, Injectable } from '@nestjs/common'; import { IsNull, Not } from 'typeorm'; -import { UnrecoverableError } from 'bullmq'; import { DI } from '@/di-symbols.js'; import type { FollowingsRepository } from '@/models/_.js'; import type { MiLocalUser, MiRemoteUser, MiUser } from '@/models/User.js'; @@ -14,6 +13,7 @@ import { UserEntityService } from '@/core/entities/UserEntityService.js'; import { bindThis } from '@/decorators.js'; import type { IActivity } from '@/core/activitypub/type.js'; import { ThinUser } from '@/queue/types.js'; +import { CacheService } from '@/core/CacheService.js'; interface IRecipe { type: string; @@ -41,23 +41,21 @@ class DeliverManager { /** * Constructor - * @param userEntityService - * @param followingsRepository * @param queueService + * @param cacheService * @param actor Actor * @param activity Activity to deliver */ constructor( - private userEntityService: UserEntityService, - private followingsRepository: FollowingsRepository, private queueService: QueueService, + private readonly cacheService: CacheService, actor: { id: MiUser['id']; host: null; }, activity: IActivity | null, ) { // 型で弾いてはいるが一応ローカルユーザーかチェック // eslint-disable-next-line @typescript-eslint/no-unnecessary-condition - if (actor.host != null) throw new Error('actor.host must be null'); + if (actor.host != null) throw new Error(`deliver failed for ${actor.id}: host is not null`); // パフォーマンス向上のためキューに突っ込むのはidのみに絞る this.actor = { @@ -114,23 +112,23 @@ class DeliverManager { // Process follower recipes first to avoid duplication when processing direct recipes later. if (this.recipes.some(r => isFollowers(r))) { // followers deliver - // TODO: SELECT DISTINCT ON ("followerSharedInbox") "followerSharedInbox" みたいな問い合わせにすればよりパフォーマンス向上できそう // ただ、sharedInboxがnullなリモートユーザーも稀におり、その対応ができなさそう？ - const followers = await this.followingsRepository.find({ - where: { - followeeId: this.actor.id, - followerHost: Not(IsNull()), - }, - select: { - followerSharedInbox: true, - followerInbox: true, - }, - }); + const followers = await this.cacheService.userFollowersCache + .fetch(this.actor.id) + .then(f => Array + .from(f.values()) + .filter(f => f.followerHost != null) + .map(f => ({ + followerInbox: f.followerInbox, + followerSharedInbox: f.followerSharedInbox, + }))); for (const following of followers) { - const inbox = following.followerSharedInbox ?? following.followerInbox; - if (inbox === null) throw new UnrecoverableError(`inbox is null: following ${following.id}`); - inboxes.set(inbox, following.followerSharedInbox != null); + if (following.followerSharedInbox) { + inboxes.set(following.followerSharedInbox, true); + } else if (following.followerInbox) { + inboxes.set(following.followerInbox, false); + } } } @@ -152,11 +150,8 @@ class DeliverManager { @Injectable() export class ApDeliverManagerService { constructor( - @Inject(DI.followingsRepository) - private followingsRepository: FollowingsRepository, - - private userEntityService: UserEntityService, private queueService: QueueService, + private readonly cacheService: CacheService, ) { } @@ -168,9 +163,8 @@ export class ApDeliverManagerService { @bindThis public async deliverToFollowers(actor: { id: MiLocalUser['id']; host: null; }, activity: IActivity): Promise<void> { const manager = new DeliverManager( - this.userEntityService, - this.followingsRepository, this.queueService, + this.cacheService, actor, activity, ); @@ -187,9 +181,8 @@ export class ApDeliverManagerService { @bindThis public async deliverToUser(actor: { id: MiLocalUser['id']; host: null; }, activity: IActivity, to: MiRemoteUser): Promise<void> { const manager = new DeliverManager( - this.userEntityService, - this.followingsRepository, this.queueService, + this.cacheService, actor, activity, ); @@ -206,9 +199,8 @@ export class ApDeliverManagerService { @bindThis public async deliverToUsers(actor: { id: MiLocalUser['id']; host: null; }, activity: IActivity, targets: MiRemoteUser[]): Promise<void> { const manager = new DeliverManager( - this.userEntityService, - this.followingsRepository, this.queueService, + this.cacheService, actor, activity, ); @@ -219,9 +211,8 @@ export class ApDeliverManagerService { @bindThis public createDeliverManager(actor: { id: MiUser['id']; host: null; }, activity: IActivity | null): DeliverManager { return new DeliverManager( - this.userEntityService, - this.followingsRepository, this.queueService, + this.cacheService, actor, activity, diff --git a/packages/backend/src/core/activitypub/ApInboxService.ts b/packages/backend/src/core/activitypub/ApInboxService.ts index b8526a972c..009d4cbd39 100644 --- a/packages/backend/src/core/activitypub/ApInboxService.ts +++ b/packages/backend/src/core/activitypub/ApInboxService.ts @@ -32,11 +32,13 @@ import { AbuseReportService } from '@/core/AbuseReportService.js'; import { FederatedInstanceService } from '@/core/FederatedInstanceService.js'; import { fromTuple } from '@/misc/from-tuple.js'; import { IdentifiableError } from '@/misc/identifiable-error.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; import InstanceChart from '@/core/chart/charts/instance.js'; import FederationChart from '@/core/chart/charts/federation.js'; import { FetchInstanceMetadataService } from '@/core/FetchInstanceMetadataService.js'; import { UpdateInstanceQueue } from '@/core/UpdateInstanceQueue.js'; -import { getApHrefNullable, getApId, getApIds, getApType, getNullableApId, isAccept, isActor, isAdd, isAnnounce, isApObject, isBlock, isCollection, isCollectionOrOrderedCollection, isCreate, isDelete, isFlag, isFollow, isLike, isDislike, isMove, isPost, isReject, isRemove, isTombstone, isUndo, isUpdate, validActor, validPost, isActivity, IObjectWithId } from './type.js'; +import { CacheService } from '@/core/CacheService.js'; +import { getApHrefNullable, getApId, getApIds, getApType, getNullableApId, isAccept, isActor, isAdd, isAnnounce, isApObject, isBlock, isCollectionOrOrderedCollection, isCreate, isDelete, isFlag, isFollow, isLike, isDislike, isMove, isPost, isReject, isRemove, isTombstone, isUndo, isUpdate, validActor, validPost, isActivity, IObjectWithId } from './type.js'; import { ApNoteService } from './models/ApNoteService.js'; import { ApLoggerService } from './ApLoggerService.js'; import { ApDbResolverService } from './ApDbResolverService.js'; @@ -97,6 +99,7 @@ export class ApInboxService { private readonly instanceChart: InstanceChart, private readonly federationChart: FederationChart, private readonly updateInstanceQueue: UpdateInstanceQueue, + private readonly cacheService: CacheService, ) { this.logger = this.apLoggerService.logger; } @@ -106,25 +109,29 @@ export class ApInboxService { let result = undefined as string | void; if (isCollectionOrOrderedCollection(activity)) { const results = [] as [string, string | void][]; - // eslint-disable-next-line no-param-reassign resolver ??= this.apResolverService.createResolver(); - const items = toArray(isCollection(activity) ? activity.items : activity.orderedItems); - if (items.length >= resolver.getRecursionLimit()) { - throw new Error(`skipping activity: collection would surpass recursion limit: ${this.utilityService.extractDbHost(actor.uri)}`); - } - - for (const item of items) { - const act = await resolver.resolve(item); - if (act.id == null || this.utilityService.extractDbHost(act.id) !== this.utilityService.extractDbHost(actor.uri)) { - this.logger.debug('skipping activity: activity id is null or mismatching'); - continue; + const items = await resolver.resolveCollectionItems(activity); + for (let i = 0; i < items.length; i++) { + const act = items[i]; + if (act.id != null) { + if (this.utilityService.extractDbHost(act.id) !== this.utilityService.extractDbHost(actor.uri)) { + this.logger.warn('skipping activity: activity id mismatch'); + continue; + } + } else { + // Activity ID should only be string or undefined. + act.id = undefined; } + + const id = getNullableApId(act) ?? `${getNullableApId(activity)}#${i}`; + try { - results.push([getApId(item), await this.performOneActivity(actor, act, resolver)]); + const result = await this.performOneActivity(actor, act, resolver); + results.push([id, result]); } catch (err) { if (err instanceof Error || typeof err === 'string') { - this.logger.error(err); + this.logger.error(`Unhandled error in activity ${id}:`, err); } else { throw err; } @@ -144,7 +151,8 @@ export class ApInboxService { if (actor.lastFetchedAt == null || Date.now() - actor.lastFetchedAt.getTime() > 1000 * 60 * 60 * 24) { setImmediate(() => { // 同一ユーザーの情報を再度処理するので、使用済みのresolverを再利用してはいけない - this.apPersonService.updatePerson(actor.uri); + this.apPersonService.updatePerson(actor.uri) + .catch(err => this.logger.error(`Failed to update person: ${renderInlineError(err)}`)); }); } } @@ -217,6 +225,10 @@ export class ApInboxService { const note = await this.apNoteService.resolveNote(object, { resolver }); if (!note) return `skip: target note not found ${targetUri}`; + if (note.userHost == null && note.localOnly) { + throw new IdentifiableError('12e23cec-edd9-442b-aa48-9c21f0c3b215', 'Cannot react to local-only note'); + } + await this.apNoteService.extractEmojis(activity.tag ?? [], actor.host).catch(() => null); try { @@ -246,7 +258,7 @@ export class ApInboxService { resolver ??= this.apResolverService.createResolver(); const object = await resolver.resolve(activity.object).catch(err => { - this.logger.error(`Resolution failed: ${err}`); + this.logger.error(`Resolution failed: ${renderInlineError(err)}`); throw err; }); @@ -319,7 +331,7 @@ export class ApInboxService { if (targetUri.startsWith('bear:')) return 'skip: bearcaps url not supported.'; const target = await resolver.secureResolve(activityObject, uri).catch(e => { - this.logger.error(`Resolution failed: ${e}`); + this.logger.error(`Resolution failed: ${renderInlineError(e)}`); throw e; }); @@ -350,27 +362,19 @@ export class ApInboxService { } // Announce対象をresolve - let renote; - try { - // The target ID is verified by secureResolve, so we know it shares host authority with the actor who sent it. - // This means we can pass that ID to resolveNote and avoid an extra fetch, which will fail if the note is private. - renote = await this.apNoteService.resolveNote(target, { resolver, sentFrom: getApId(target) }); - if (renote == null) return 'announce target is null'; - } catch (err) { - // 対象が4xxならスキップ - if (err instanceof StatusError) { - if (!err.isRetryable) { - return `skip: ignored announce target ${target.id} - ${err.statusCode}`; - } - return `Error in announce target ${target.id} - ${err.statusCode}`; - } - throw err; - } + // The target ID is verified by secureResolve, so we know it shares host authority with the actor who sent it. + // This means we can pass that ID to resolveNote and avoid an extra fetch, which will fail if the note is private. + const renote = await this.apNoteService.resolveNote(target, { resolver, sentFrom: getApId(target) }); + if (renote == null) return 'announce target is null'; - if (!await this.noteEntityService.isVisibleForMe(renote, actor.id)) { + if (!await this.noteEntityService.isVisibleForMe(renote, actor.id, { me: actor })) { return 'skip: invalid actor for this activity'; } + if (renote.userHost == null && renote.localOnly) { + throw new IdentifiableError('12e23cec-edd9-442b-aa48-9c21f0c3b215', 'Cannot renote a local-only note'); + } + this.logger.info(`Creating the (Re)Note: ${uri}`); const activityAudience = await this.apAudienceService.parseAudience(actor, activity.to, activity.cc, resolver); @@ -443,9 +447,11 @@ export class ApInboxService { setImmediate(() => { // Don't re-use the resolver, or it may throw recursion errors. // Instead, create a new resolver with an appropriately-reduced recursion limit. - this.apPersonService.updatePerson(actor.uri, this.apResolverService.createResolver({ + const subResolver = this.apResolverService.createResolver({ recursionLimit: resolver.getRecursionLimit() - resolver.getHistory().length, - })); + }); + this.apPersonService.updatePerson(actor.uri, subResolver) + .catch(err => this.logger.error(`Failed to update person: ${renderInlineError(err)}`)); }); } }); @@ -500,7 +506,7 @@ export class ApInboxService { resolver ??= this.apResolverService.createResolver(); const object = await resolver.resolve(activityObject).catch(e => { - this.logger.error(`Resolution failed: ${e}`); + this.logger.error(`Resolution failed: ${renderInlineError(e)}`); throw e; }); @@ -537,12 +543,6 @@ export class ApInboxService { await this.apNoteService.createNote(note, actor, resolver, silent); return 'ok'; - } catch (err) { - if (err instanceof StatusError && !err.isRetryable) { - return `skip: ${err.statusCode}`; - } else { - throw err; - } } finally { unlock(); } @@ -675,7 +675,7 @@ export class ApInboxService { resolver ??= this.apResolverService.createResolver(); const object = await resolver.resolve(activity.object).catch(e => { - this.logger.error(`Resolution failed: ${e}`); + this.logger.error(`Resolution failed: ${renderInlineError(e)}`); throw e; }); @@ -747,7 +747,7 @@ export class ApInboxService { resolver ??= this.apResolverService.createResolver(); const object = await resolver.resolve(activity.object).catch(e => { - this.logger.error(`Resolution failed: ${e}`); + this.logger.error(`Resolution failed: ${renderInlineError(e)}`); throw e; }); @@ -768,12 +768,7 @@ export class ApInboxService { return 'skip: follower not found'; } - const isFollowing = await this.followingsRepository.exists({ - where: { - followerId: follower.id, - followeeId: actor.id, - }, - }); + const isFollowing = await this.cacheService.userFollowingsCache.fetch(follower.id).then(f => f.has(actor.id)); if (isFollowing) { await this.userFollowingService.unfollow(follower, actor); @@ -832,12 +827,7 @@ export class ApInboxService { }, }); - const isFollowing = await this.followingsRepository.exists({ - where: { - followerId: actor.id, - followeeId: followee.id, - }, - }); + const isFollowing = await this.cacheService.userFollowingsCache.fetch(actor.id).then(f => f.has(followee.id)); if (requestExist) { await this.userFollowingService.cancelFollowRequest(followee, actor); @@ -879,7 +869,7 @@ export class ApInboxService { resolver ??= this.apResolverService.createResolver(); const object = await resolver.resolve(activity.object).catch(e => { - this.logger.error(`Resolution failed: ${e}`); + this.logger.error(`Resolution failed: ${renderInlineError(e)}`); throw e; }); diff --git a/packages/backend/src/core/activitypub/ApMfmService.ts b/packages/backend/src/core/activitypub/ApMfmService.ts index c4a948429a..ddb6461746 100644 --- a/packages/backend/src/core/activitypub/ApMfmService.ts +++ b/packages/backend/src/core/activitypub/ApMfmService.ts @@ -4,7 +4,7 @@ */ import { Injectable } from '@nestjs/common'; -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; import { MfmService, Appender } from '@/core/MfmService.js'; import type { MiNote } from '@/models/Note.js'; import { bindThis } from '@/decorators.js'; diff --git a/packages/backend/src/core/activitypub/ApRendererService.ts b/packages/backend/src/core/activitypub/ApRendererService.ts index f41eeba39f..623e7002cd 100644 --- a/packages/backend/src/core/activitypub/ApRendererService.ts +++ b/packages/backend/src/core/activitypub/ApRendererService.ts @@ -6,8 +6,9 @@ import { createPublicKey, randomUUID } from 'node:crypto'; import { Inject, Injectable } from '@nestjs/common'; import { In } from 'typeorm'; -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; import { UnrecoverableError } from 'bullmq'; +import { Element, Text } from 'domhandler'; import { DI } from '@/di-symbols.js'; import type { Config } from '@/config.js'; import type { MiPartialLocalUser, MiLocalUser, MiPartialRemoteUser, MiRemoteUser, MiUser } from '@/models/User.js'; @@ -31,10 +32,12 @@ import { IdService } from '@/core/IdService.js'; import { appendContentWarning } from '@/misc/append-content-warning.js'; import { QueryService } from '@/core/QueryService.js'; import { UtilityService } from '@/core/UtilityService.js'; +import { CacheService } from '@/core/CacheService.js'; +import { isPureRenote, isQuote, isRenote } from '@/misc/is-renote.js'; import { JsonLdService } from './JsonLdService.js'; import { ApMfmService } from './ApMfmService.js'; import { CONTEXT } from './misc/contexts.js'; -import { getApId, IOrderedCollection, IOrderedCollectionPage } from './type.js'; +import { getApId, ILink, IOrderedCollection, IOrderedCollectionPage } from './type.js'; import type { IAccept, IActivity, IAdd, IAnnounce, IApDocument, IApEmoji, IApHashtag, IApImage, IApMention, IBlock, ICreate, IDelete, IFlag, IFollow, IKey, ILike, IMove, IObject, IPost, IQuestion, IReject, IRemove, ITombstone, IUndo, IUpdate } from './type.js'; @Injectable() @@ -74,6 +77,7 @@ export class ApRendererService { private idService: IdService, private readonly queryService: QueryService, private utilityService: UtilityService, + private readonly cacheService: CacheService, ) { } @@ -231,7 +235,7 @@ export class ApRendererService { */ @bindThis public async renderFollowUser(id: MiUser['id']): Promise<string> { - const user = await this.usersRepository.findOneByOrFail({ id: id }) as MiPartialLocalUser | MiPartialRemoteUser; + const user = await this.cacheService.findUserById(id) as MiPartialLocalUser | MiPartialRemoteUser; return this.userEntityService.getUserUri(user); } @@ -401,7 +405,7 @@ export class ApRendererService { inReplyToNote = await this.notesRepository.findOneBy({ id: note.replyId }); if (inReplyToNote != null) { - const inReplyToUser = await this.usersRepository.findOneBy({ id: inReplyToNote.userId }); + const inReplyToUser = await this.cacheService.findUserById(inReplyToNote.userId); if (inReplyToUser) { if (inReplyToNote.uri) { @@ -419,9 +423,9 @@ export class ApRendererService { inReplyTo = null; } - let quote; + let quote: string | undefined = undefined; - if (note.renoteId) { + if (isRenote(note) && isQuote(note)) { const renote = await this.notesRepository.findOneBy({ id: note.renoteId }); if (renote) { @@ -475,16 +479,18 @@ export class ApRendererService { // the claas name `quote-inline` is used in non-misskey clients for styling quote notes. // For compatibility, the span part should be kept as possible. apAppend.push((doc, body) => { - body.appendChild(doc.createElement('br')); - body.appendChild(doc.createElement('br')); - const span = doc.createElement('span'); - span.className = 'quote-inline'; - span.appendChild(doc.createTextNode('RE: ')); - const link = doc.createElement('a'); - link.setAttribute('href', quote); - link.textContent = quote; - span.appendChild(link); - body.appendChild(span); + body.childNodes.push(new Element('br', {})); + body.childNodes.push(new Element('br', {})); + const span = new Element('span', { + class: 'quote-inline', + }); + span.childNodes.push(new Text('RE: ')); + const link = new Element('a', { + href: quote, + }); + link.childNodes.push(new Text(quote)); + span.childNodes.push(link); + body.childNodes.push(span); }); } @@ -500,12 +506,22 @@ export class ApRendererService { const emojis = await this.getEmojis(note.emojis); const apemojis = emojis.filter(emoji => !emoji.localOnly).map(emoji => this.renderEmoji(emoji)); - const tag = [ + const tag: IObject[] = [ ...hashtagTags, ...mentionTags, ...apemojis, ]; + // https://codeberg.org/fediverse/fep/src/branch/main/fep/e232/fep-e232.md + if (quote) { + tag.push({ + type: 'Link', + mediaType: 'application/ld+json; profile="https://www.w3.org/ns/activitystreams"', + rel: 'https://misskey-hub.net/ns#_misskey_quote', + href: quote, + } satisfies ILink); + } + const asPoll = poll ? { type: 'Question', [poll.expiresAt && poll.expiresAt < new Date() ? 'closed' : 'endTime']: poll.expiresAt, @@ -529,6 +545,7 @@ export class ApRendererService { attributedTo, summary: summary ?? undefined, content: content ?? undefined, + updated: note.updatedAt?.toISOString() ?? undefined, _misskey_content: text, source: { content: text, @@ -537,6 +554,8 @@ export class ApRendererService { _misskey_quote: quote, quoteUrl: quote, quoteUri: quote, + // https://codeberg.org/fediverse/fep/src/branch/main/fep/044f/fep-044f.md + quote: quote, published: this.idService.parse(note.id).date.toISOString(), to, cc, @@ -613,6 +632,7 @@ export class ApRendererService { enableRss: user.enableRss, speakAsCat: user.speakAsCat, attachment: attachment.length ? attachment : undefined, + attributionDomains: user.attributionDomains, }; if (user.movedToUri) { @@ -741,162 +761,6 @@ export class ApRendererService { } @bindThis - public async renderUpNote(note: MiNote, author: MiUser, dive = true): Promise<IPost> { - const getPromisedFiles = async (ids: string[]): Promise<MiDriveFile[]> => { - if (ids.length === 0) return []; - const items = await this.driveFilesRepository.findBy({ id: In(ids) }); - return ids.map(id => items.find(item => item.id === id)).filter((item): item is MiDriveFile => item != null); - }; - - let inReplyTo; - let inReplyToNote: MiNote | null; - - if (note.replyId) { - inReplyToNote = await this.notesRepository.findOneBy({ id: note.replyId }); - - if (inReplyToNote != null) { - const inReplyToUser = await this.usersRepository.findOneBy({ id: inReplyToNote.userId }); - - if (inReplyToUser) { - if (inReplyToNote.uri) { - inReplyTo = inReplyToNote.uri; - } else { - if (dive) { - inReplyTo = await this.renderUpNote(inReplyToNote, inReplyToUser, false); - } else { - inReplyTo = `${this.config.url}/notes/${inReplyToNote.id}`; - } - } - } - } - } else { - inReplyTo = null; - } - - let quote; - - if (note.renoteId) { - const renote = await this.notesRepository.findOneBy({ id: note.renoteId }); - - if (renote) { - quote = renote.uri ? renote.uri : `${this.config.url}/notes/${renote.id}`; - } - } - - const attributedTo = this.userEntityService.genLocalUserUri(note.userId); - - const mentions = note.mentionedRemoteUsers ? (JSON.parse(note.mentionedRemoteUsers) as IMentionedRemoteUsers).map(x => x.uri) : []; - - let to: string[] = []; - let cc: string[] = []; - - if (note.visibility === 'public') { - to = ['https://www.w3.org/ns/activitystreams#Public']; - cc = [`${attributedTo}/followers`].concat(mentions); - } else if (note.visibility === 'home') { - to = [`${attributedTo}/followers`]; - cc = ['https://www.w3.org/ns/activitystreams#Public'].concat(mentions); - } else if (note.visibility === 'followers') { - to = [`${attributedTo}/followers`]; - cc = mentions; - } else { - to = mentions; - } - - const mentionedUsers = note.mentions && note.mentions.length > 0 ? await this.usersRepository.findBy({ - id: In(note.mentions), - }) : []; - - const hashtagTags = note.tags.map(tag => this.renderHashtag(tag)); - const mentionTags = mentionedUsers.map(u => this.renderMention(u as MiLocalUser | MiRemoteUser)); - - const files = await getPromisedFiles(note.fileIds); - - const text = note.text ?? ''; - let poll: MiPoll | null = null; - - if (note.hasPoll) { - poll = await this.pollsRepository.findOneBy({ noteId: note.id }); - } - - const apAppend: Appender[] = []; - - if (quote) { - // Append quote link as `<br><br><span class="quote-inline">RE: <a href="...">...</a></span>` - // the claas name `quote-inline` is used in non-misskey clients for styling quote notes. - // For compatibility, the span part should be kept as possible. - apAppend.push((doc, body) => { - body.appendChild(doc.createElement('br')); - body.appendChild(doc.createElement('br')); - const span = doc.createElement('span'); - span.className = 'quote-inline'; - span.appendChild(doc.createTextNode('RE: ')); - const link = doc.createElement('a'); - link.setAttribute('href', quote); - link.textContent = quote; - span.appendChild(link); - body.appendChild(span); - }); - } - - let summary = note.cw === '' ? String.fromCharCode(0x200B) : note.cw; - - // Apply mandatory CW, if applicable - if (author.mandatoryCW) { - summary = appendContentWarning(summary, author.mandatoryCW); - } - - const { content } = this.apMfmService.getNoteHtml(note, apAppend); - - const emojis = await this.getEmojis(note.emojis); - const apemojis = emojis.filter(emoji => !emoji.localOnly).map(emoji => this.renderEmoji(emoji)); - - const tag = [ - ...hashtagTags, - ...mentionTags, - ...apemojis, - ]; - - const asPoll = poll ? { - type: 'Question', - [poll.expiresAt && poll.expiresAt < new Date() ? 'closed' : 'endTime']: poll.expiresAt, - [poll.multiple ? 'anyOf' : 'oneOf']: poll.choices.map((text, i) => ({ - type: 'Note', - name: text, - replies: { - type: 'Collection', - totalItems: poll!.votes[i], - }, - })), - } as const : {}; - - return { - id: `${this.config.url}/notes/${note.id}`, - type: 'Note', - attributedTo, - summary: summary ?? undefined, - content: content ?? undefined, - updated: note.updatedAt?.toISOString(), - _misskey_content: text, - source: { - content: text, - mediaType: 'text/x.misskeymarkdown', - }, - _misskey_quote: quote, - quoteUrl: quote, - quoteUri: quote, - published: this.idService.parse(note.id).date.toISOString(), - to, - cc, - inReplyTo, - attachment: files.map(x => this.renderDocument(x)), - sensitive: note.cw != null || files.some(file => file.isSensitive), - tag, - ...asPoll, - }; - } - - @bindThis public renderVote(user: { id: MiUser['id'] }, vote: MiPollVote, note: MiNote, poll: MiPoll, pollOwner: MiRemoteUser): ICreate { return { id: `${this.config.url}/users/${user.id}#votes/${vote.id}/activity`, @@ -935,9 +799,7 @@ export class ApRendererService { const keypair = await this.userKeypairService.getUserKeypair(user.id); - const jsonLd = this.jsonLdService.use(); - jsonLd.debug = false; - activity = await jsonLd.signRsaSignature2017(activity, keypair.privateKey, `${this.config.url}/users/${user.id}#main-key`); + activity = await this.jsonLdService.signRsaSignature2017(activity, keypair.privateKey, `${this.config.url}/users/${user.id}#main-key`); return activity; } @@ -1052,6 +914,27 @@ export class ApRendererService { } @bindThis + public async renderNoteOrRenoteActivity(note: MiNote, user: MiUser, hint?: { renote?: MiNote | null }) { + if (note.localOnly) return null; + + if (isPureRenote(note)) { + const renote = hint?.renote ?? note.renote ?? await this.notesRepository.findOneByOrFail({ id: note.renoteId }); + const apAnnounce = this.renderAnnounce(renote.uri ?? `${this.config.url}/notes/${renote.id}`, note); + return this.addContext(apAnnounce); + } + + const apNote = await this.renderNote(note, user, false); + + if (note.updatedAt != null) { + const apUpdate = this.renderUpdate(apNote, user); + return this.addContext(apUpdate); + } else { + const apCreate = this.renderCreate(apNote, note); + return this.addContext(apCreate); + } + } + + @bindThis private async getEmojis(names: string[]): Promise<MiEmoji[]> { if (names.length === 0) return []; diff --git a/packages/backend/src/core/activitypub/ApRequestService.ts b/packages/backend/src/core/activitypub/ApRequestService.ts index b665b51700..e4db9b237c 100644 --- a/packages/backend/src/core/activitypub/ApRequestService.ts +++ b/packages/backend/src/core/activitypub/ApRequestService.ts @@ -6,7 +6,7 @@ import * as crypto from 'node:crypto'; import { URL } from 'node:url'; import { Inject, Injectable } from '@nestjs/common'; -import { Window } from 'happy-dom'; +import { load as cheerio } from 'cheerio/slim'; import { DI } from '@/di-symbols.js'; import type { Config } from '@/config.js'; import type { MiUser } from '@/models/User.js'; @@ -18,6 +18,8 @@ import { bindThis } from '@/decorators.js'; import type Logger from '@/logger.js'; import { validateContentTypeSetAsActivityPub } from '@/core/activitypub/misc/validator.js'; import type { IObject, IObjectWithId } from './type.js'; +import type { Cheerio, CheerioAPI } from 'cheerio/slim'; +import type { AnyNode } from 'domhandler'; type Request = { url: string; @@ -184,10 +186,11 @@ export class ApRequestService { * Get AP object with http-signature * @param user http-signature user * @param url URL to fetch - * @param followAlternate + * @param allowAnonymous If a fetched object lacks an ID, then it will be auto-generated from the final URL. (default: false) + * @param followAlternate Whether to resolve HTML responses to their referenced canonical AP endpoint. (default: true) */ @bindThis - public async signedGet(url: string, user: { id: MiUser['id'] }, followAlternate?: boolean): Promise<IObjectWithId> { + public async signedGet(url: string, user: { id: MiUser['id'] }, allowAnonymous = false, followAlternate?: boolean): Promise<IObjectWithId> { this.apUtilityService.assertApUrl(url); const _followAlternate = followAlternate ?? true; @@ -218,53 +221,33 @@ export class ApRequestService { (contentType ?? '').split(';')[0].trimEnd().toLowerCase() === 'text/html' && _followAlternate === true ) { - const html = await res.text(); - const { window, happyDOM } = new Window({ - settings: { - disableJavaScriptEvaluation: true, - disableJavaScriptFileLoading: true, - disableCSSFileLoading: true, - disableComputedStyleRendering: true, - handleDisabledFileLoadingAsSuccess: true, - navigation: { - disableMainFrameNavigation: true, - disableChildFrameNavigation: true, - disableChildPageNavigation: true, - disableFallbackToSetURL: true, - }, - timer: { - maxTimeout: 0, - maxIntervalTime: 0, - maxIntervalIterations: 0, - }, - }, - }); - const document = window.document; + let alternate: Cheerio<AnyNode> | null; try { - document.documentElement.innerHTML = html; + const html = await res.text(); + const document = cheerio(html); // Search for any matching value in priority order: // 1. Type=AP > Type=none > Type=anything // 2. Alternate > Canonical // 3. Page order (fallback) - const alternate = - document.querySelector('head > link[href][rel="alternate"][type="application/activity+json"]') ?? - document.querySelector('head > link[href][rel="canonical"][type="application/activity+json"]') ?? - document.querySelector('head > link[href][rel="alternate"]:not([type])') ?? - document.querySelector('head > link[href][rel="canonical"]:not([type])') ?? - document.querySelector('head > link[href][rel="alternate"]') ?? - document.querySelector('head > link[href][rel="canonical"]'); - - if (alternate) { - const href = alternate.getAttribute('href'); - if (href && this.apUtilityService.haveSameAuthority(url, href)) { - return await this.signedGet(href, user, false); - } - } + alternate = selectFirst(document, [ + 'head > link[href][rel="alternate"][type="application/activity+json"]', + 'head > link[href][rel="canonical"][type="application/activity+json"]', + 'head > link[href][rel="alternate"]:not([type])', + 'head > link[href][rel="canonical"]:not([type])', + 'head > link[href][rel="alternate"]', + 'head > link[href][rel="canonical"]', + ]); } catch { // something went wrong parsing the HTML, ignore the whole thing - } finally { - happyDOM.close().catch(err => {}); + alternate = null; + } + + if (alternate) { + const href = alternate.attr('href'); + if (href && this.apUtilityService.haveSameAuthority(url, href)) { + return await this.signedGet(href, user, allowAnonymous, false); + } } } //#endregion @@ -275,8 +258,23 @@ export class ApRequestService { // Make sure the object ID matches the final URL (which is where it actually exists). // The caller (ApResolverService) will verify the ID against the original / entry URL, which ensures that all three match. - this.apUtilityService.assertIdMatchesUrlAuthority(activity, res.url); + if (allowAnonymous && activity.id == null) { + activity.id = res.url; + } else { + this.apUtilityService.assertIdMatchesUrlAuthority(activity, res.url); + } return activity as IObjectWithId; } } + +function selectFirst($: CheerioAPI, selectors: string[]): Cheerio<AnyNode> | null { + for (const selector of selectors) { + const selection = $(selector); + if (selection.length > 0) { + return selection; + } + } + + return null; +} diff --git a/packages/backend/src/core/activitypub/ApResolverService.ts b/packages/backend/src/core/activitypub/ApResolverService.ts index 5e58f848c0..d53e265d36 100644 --- a/packages/backend/src/core/activitypub/ApResolverService.ts +++ b/packages/backend/src/core/activitypub/ApResolverService.ts @@ -5,6 +5,7 @@ import { Inject, Injectable } from '@nestjs/common'; import { IsNull, Not } from 'typeorm'; +import promiseLimit from 'promise-limit'; import type { MiLocalUser, MiRemoteUser } from '@/models/User.js'; import type { NotesRepository, PollsRepository, NoteReactionsRepository, UsersRepository, FollowRequestsRepository, MiMeta, SkApFetchLog } from '@/models/_.js'; import type { Config } from '@/config.js'; @@ -19,11 +20,14 @@ import { ApLogService, calculateDurationSince, extractObjectContext } from '@/co import { ApUtilityService } from '@/core/activitypub/ApUtilityService.js'; import { SystemAccountService } from '@/core/SystemAccountService.js'; import { IdentifiableError } from '@/misc/identifiable-error.js'; -import { getApId, getNullableApId, IObjectWithId, isCollectionOrOrderedCollection } from './type.js'; +import { toArray } from '@/misc/prelude/array.js'; +import { isPureRenote } from '@/misc/is-renote.js'; +import { CacheService } from '@/core/CacheService.js'; +import { AnyCollection, getApId, getNullableApId, IObjectWithId, isCollection, isCollectionOrOrderedCollection, isCollectionPage, isOrderedCollection, isOrderedCollectionPage } from './type.js'; import { ApDbResolverService } from './ApDbResolverService.js'; import { ApRendererService } from './ApRendererService.js'; import { ApRequestService } from './ApRequestService.js'; -import type { IObject, ICollection, IOrderedCollection, ApObject } from './type.js'; +import type { IObject, ApObject, IAnonymousObject } from './type.js'; export class Resolver { private history: Set<string>; @@ -47,6 +51,7 @@ export class Resolver { private loggerService: LoggerService, private readonly apLogService: ApLogService, private readonly apUtilityService: ApUtilityService, + private readonly cacheService: CacheService, private recursionLimit = 256, ) { this.history = new Set(); @@ -63,34 +68,129 @@ export class Resolver { return this.recursionLimit; } + public async resolveCollection(value: string | IObjectWithId, allowAnonymous?: boolean, sentFromUri?: string): Promise<AnyCollection & IObjectWithId>; + public async resolveCollection(value: string | IObject, allowAnonymous: boolean | undefined, sentFromUri: string): Promise<AnyCollection & IObjectWithId>; + public async resolveCollection(value: string | IObject, allowAnonymous?: boolean, sentFromUri?: string): Promise<AnyCollection>; @bindThis - public async resolveCollection(value: string | IObject): Promise<ICollection | IOrderedCollection> { + public async resolveCollection(value: string | IObject, allowAnonymous?: boolean, sentFromUri?: string): Promise<AnyCollection> { const collection = typeof value === 'string' - ? await this.resolve(value) - : value; + ? sentFromUri + ? await this.secureResolve(value, sentFromUri, allowAnonymous) + : await this.resolve(value, allowAnonymous) + : value; // TODO try and remove this eventually, as it's a major security foot-gun if (isCollectionOrOrderedCollection(collection)) { return collection; } else { - throw new IdentifiableError('f100eccf-f347-43fb-9b45-96a0831fb635', `unrecognized collection type: ${collection.type}`); + throw new IdentifiableError('f100eccf-f347-43fb-9b45-96a0831fb635', `collection ${getApId(value)} has unsupported type: ${collection.type}`); } } + public async resolveCollectionItems(collection: IAnonymousObject, limit?: number | null, allowAnonymousItems?: true, concurrency?: number): Promise<IAnonymousObject[]>; + public async resolveCollectionItems(collection: string | IObjectWithId, limit?: number | null, allowAnonymousItems?: boolean, concurrency?: number): Promise<IObjectWithId[]>; + public async resolveCollectionItems(collection: string | IObject, limit?: number | null, allowAnonymousItems?: boolean, concurrency?: number): Promise<IObject[]>; + /** + * Recursively resolves items from a collection. + * Stops when reaching the resolution limit or an optional item limit - whichever is lower. + * This method supports Collection, OrderedCollection, and individual pages of either type. + * Malformed collections (mixing Ordered and un-Ordered types) are also supported. + * @param collection Collection to resolve from - can be a URL or object of any supported collection type. + * @param limit Maximum number of items to resolve. If null or undefined (default), then items will be resolved until reaching the recursion limit. + * @param allowAnonymousItems If true, collection items can be anonymous (lack an ID). If false (default), then an error is thrown when reaching an item without ID. + * @param concurrency Maximum number of items to resolve at once. (default: 4) + */ + @bindThis + public async resolveCollectionItems(collection: string | IObject, limit?: number | null, allowAnonymousItems?: boolean, concurrency = 4): Promise<IObject[]> { + const resolvedItems: IObject[] = []; + + // This is pulled up to avoid code duplication below + const iterate = async(items: ApObject, current: AnyCollection) => { + const sentFrom = current.id; + const itemArr = toArray(items); + const itemLimit = limit ?? Number.MAX_SAFE_INTEGER; + const allowAnonymous = allowAnonymousItems ?? false; + await this.resolveItemArray(itemArr, sentFrom, itemLimit, concurrency, allowAnonymous, resolvedItems); + }; + + let current: AnyCollection | null = await this.resolveCollection(collection); + do { + // Iterate all items in the current page + if (current.items) { + await iterate(current.items, current); + } + if (current.orderedItems) { + await iterate(current.orderedItems, current); + } + + if (this.history.size >= this.recursionLimit) { + // Stop when we reach the fetch limit + current = null; + } else if (limit != null && resolvedItems.length >= limit) { + // Stop when we reach the item limit + current = null; + } else if (isCollection(current) || isOrderedCollection(current)) { + // Continue to first page + current = current.first ? await this.resolveCollection(current.first, true, current.id) : null; + } else if (isCollectionPage(current) || isOrderedCollectionPage(current)) { + // Continue to next page + current = current.next ? await this.resolveCollection(current.next, true, current.id) : null; + } else { + // Stop in all other conditions + current = null; + } + } while (current != null); + + return resolvedItems; + } + + private async resolveItemArray(source: (string | IObject)[], sentFrom: undefined, itemLimit: number, concurrency: number, allowAnonymousItems: true, destination: IAnonymousObject[]): Promise<void>; + private async resolveItemArray(source: (string | IObject)[], sentFrom: string, itemLimit: number, concurrency: number, allowAnonymousItems: boolean, destination: IObjectWithId[]): Promise<void>; + private async resolveItemArray(source: (string | IObject)[], sentFrom: string | undefined, itemLimit: number, concurrency: number, allowAnonymousItems: boolean, destination: IObject[]): Promise<void>; + private async resolveItemArray(source: (string | IObject)[], sentFrom: string | undefined, itemLimit: number, concurrency: number, allowAnonymousItems: boolean, destination: IObject[]): Promise<void> { + const recursionLimit = this.recursionLimit - this.history.size; + const batchLimit = Math.min(source.length, recursionLimit, itemLimit); + + const limiter = promiseLimit<IObject>(concurrency); + const batch = await Promise.all(source + .slice(0, batchLimit) + .map(item => limiter(async () => { + if (sentFrom) { + // Use secureResolve to avoid re-fetching items that were included inline. + return await this.secureResolve(item, sentFrom, allowAnonymousItems); + } else if (allowAnonymousItems) { + return await this.resolveAnonymous(item); + } else { + // ID is required if we have neither sentFrom not allowAnonymousItems + const id = getApId(item); + return await this.resolve(id); + } + }))); + + destination.push(...batch); + }; + /** * Securely resolves an AP object or URL that has been sent from another instance. * An input object is trusted if and only if its ID matches the authority of sentFromUri. * In all other cases, the object is re-fetched from remote by input string or object ID. + * @param input The input object or URL to resolve + * @param sentFromUri The URL where this object originated. This MUST be accurate - all security checks depend on this value! + * @param allowAnonymous If true, anonymous objects are allowed and will have their ID set to sentFromUri. If false (default) then anonymous objects will be rejected with an error. */ @bindThis - public async secureResolve(input: ApObject, sentFromUri: string): Promise<IObjectWithId> { + public async secureResolve(input: string | IObject | [string | IObject], sentFromUri: string, allowAnonymous?: boolean): Promise<IObjectWithId> { // Unpack arrays to get the value element. const value = fromTuple(input); - if (value == null) { - throw new IdentifiableError('20058164-9de1-4573-8715-425753a21c1d', 'Cannot resolve null input'); + + // If anonymous input is allowed, then any object is automatically valid if we set the ID. + // We can short-circuit here and avoid un-necessary checks. + if (allowAnonymous && typeof(value) === 'object' && value.id == null) { + value.id = sentFromUri; + return value as IObjectWithId; } - // This will throw if the input has no ID, which is good because we can't verify an anonymous object anyway. - const id = getApId(value); + // This ensures the input has a string ID, protecting against type confusion and rejecting anonymous objects. + const id = getApId(value, sentFromUri); // Check if we can use the provided object as-is. // Our security requires that the object ID matches the host authority that sent it, otherwise it can't be trusted. @@ -100,28 +200,52 @@ export class Resolver { } // If the checks didn't pass, then we must fetch the object and use that. - return await this.resolve(id); + return await this.resolve(id, allowAnonymous); } - public async resolve(value: string | [string]): Promise<IObjectWithId>; - public async resolve(value: string | IObject | [string | IObject]): Promise<IObject>; + /** + * Resolves an anonymous object. + * The returned value will not have any ID present. + * If one is provided in the response, it will be removed automatically. + */ + @bindThis + public async resolveAnonymous(value: string | IObject | [string | IObject]): Promise<IAnonymousObject> { + value = fromTuple(value); + + const object = await this.resolve(value); + object.id = undefined; + + return object as IAnonymousObject; + } + + public async resolve(value: string | [string], allowAnonymous?: boolean): Promise<IObjectWithId>; + public async resolve(value: string | IObjectWithId | [string | IObjectWithId], allowAnonymous?: boolean): Promise<IObjectWithId>; + public async resolve(value: string | IObject | [string | IObject], allowAnonymous?: boolean): Promise<IObject>; + /** + * Resolves a URL or object to an AP object. + * Tuples are expanded to their first element before anything else, and non-string inputs are returned as-is. + * Otherwise, the string URL is fetched and validated to represent a valid ActivityPub object. + * @param value The input value to resolve + * @param allowAnonymous Determines what to do if a response object lacks an ID field. If false (default), then an exception is thrown. If true, then the ID is populated from the final response URL. + */ @bindThis - public async resolve(value: string | IObject | [string | IObject]): Promise<IObject> { + public async resolve(value: string | IObject | [string | IObject], allowAnonymous = false): Promise<IObject> { value = fromTuple(value); + // TODO try and remove this eventually, as it's a major security foot-gun if (typeof value !== 'string') { return value; } const host = this.utilityService.extractDbHost(value); if (this.config.activityLogging.enabled && !this.utilityService.isSelfHost(host)) { - return await this._resolveLogged(value, host); + return await this._resolveLogged(value, host, allowAnonymous); } else { - return await this._resolve(value, host); + return await this._resolve(value, host, allowAnonymous); } } - private async _resolveLogged(requestUri: string, host: string): Promise<IObjectWithId> { + private async _resolveLogged(requestUri: string, host: string, allowAnonymous: boolean): Promise<IObjectWithId> { const startTime = process.hrtime.bigint(); const log = await this.apLogService.createFetchLog({ @@ -130,7 +254,7 @@ export class Resolver { }); try { - const result = await this._resolve(requestUri, host, log); + const result = await this._resolve(requestUri, host, allowAnonymous, log); log.accepted = true; log.result = 'ok'; @@ -150,20 +274,20 @@ export class Resolver { } } - private async _resolve(value: string, host: string, log?: SkApFetchLog): Promise<IObjectWithId> { + private async _resolve(value: string, host: string, allowAnonymous: boolean, log?: SkApFetchLog): Promise<IObjectWithId> { if (value.includes('#')) { // URLs with fragment parts cannot be resolved correctly because // the fragment part does not get transmitted over HTTP(S). // Avoid strange behaviour by not trying to resolve these at all. - throw new IdentifiableError('b94fd5b1-0e3b-4678-9df2-dad4cd515ab2', `cannot resolve URL with fragment: ${value}`); + throw new IdentifiableError('b94fd5b1-0e3b-4678-9df2-dad4cd515ab2', `failed to resolve ${value}: URL contains fragment`); } if (this.history.has(value)) { - throw new IdentifiableError('0dc86cf6-7cd6-4e56-b1e6-5903d62d7ea5', `cannot resolve already resolved URL: ${value}`); + throw new IdentifiableError('0dc86cf6-7cd6-4e56-b1e6-5903d62d7ea5', `failed to resolve ${value}: recursive resolution blocked`); } if (this.history.size > this.recursionLimit) { - throw new IdentifiableError('d592da9f-822f-4d91-83d7-4ceefabcf3d2', `hit recursion limit: ${value}`); + throw new IdentifiableError('d592da9f-822f-4d91-83d7-4ceefabcf3d2', `failed to resolve ${value}: hit recursion limit`); } this.history.add(value); @@ -173,7 +297,7 @@ export class Resolver { } if (!this.utilityService.isFederationAllowedHost(host)) { - throw new IdentifiableError('09d79f9e-64f1-4316-9cfa-e75c4d091574', `cannot fetch AP object ${value}: blocked instance ${host}`); + throw new IdentifiableError('09d79f9e-64f1-4316-9cfa-e75c4d091574', `failed to resolve ${value}: instance ${host} is blocked`); } if (this.config.signToActivityPubGet && !this.user) { @@ -181,8 +305,8 @@ export class Resolver { } const object = (this.user - ? await this.apRequestService.signedGet(value, this.user) - : await this.httpRequestService.getActivityJson(value)); + ? await this.apRequestService.signedGet(value, this.user, allowAnonymous) + : await this.httpRequestService.getActivityJson(value, false, allowAnonymous)); if (log) { const { object: objectOnly, context, contextHash } = extractObjectContext(object); @@ -203,12 +327,12 @@ export class Resolver { !(object['@context'] as unknown[]).includes('https://www.w3.org/ns/activitystreams') : object['@context'] !== 'https://www.w3.org/ns/activitystreams' ) { - throw new IdentifiableError('72180409-793c-4973-868e-5a118eb5519b', `invalid AP object ${value}: does not have ActivityStreams context`); + throw new IdentifiableError('72180409-793c-4973-868e-5a118eb5519b', `failed to resolve ${value}: response does not have ActivityStreams context`); } // The object ID is already validated to match the final URL's authority by signedGet / getActivityJson. // We only need to validate that it also matches the original URL's authority, in case of redirects. - const objectId = getApId(object); + const objectId = getApId(object, value); // We allow some limited cross-domain redirects, which means the host may have changed during fetch. // Additional checks are needed to validate the scope of cross-domain redirects. @@ -219,64 +343,65 @@ export class Resolver { // Check if the redirect bounce from [allowed domain] to [blocked domain]. if (!this.utilityService.isFederationAllowedHost(finalHost)) { - throw new IdentifiableError('0a72bf24-2d9b-4f1d-886b-15aaa31adeda', `cannot fetch AP object ${value}: redirected to blocked instance ${finalHost}`); + throw new IdentifiableError('0a72bf24-2d9b-4f1d-886b-15aaa31adeda', `failed to resolve ${value}: redirected to blocked instance ${finalHost}`); } } return object; } + // TODO try to remove this, as it creates a large attack surface @bindThis private resolveLocal(url: string): Promise<IObjectWithId> { const parsed = this.apDbResolverService.parseUri(url); - if (!parsed.local) throw new IdentifiableError('02b40cd0-fa92-4b0c-acc9-fb2ada952ab8', `resolveLocal - not a local URL: ${url}`); + if (!parsed.local) throw new IdentifiableError('02b40cd0-fa92-4b0c-acc9-fb2ada952ab8', `failed to resolve local ${url}: not a local URL`); switch (parsed.type) { case 'notes': - return this.notesRepository.findOneByOrFail({ id: parsed.id }) + return this.notesRepository.findOneOrFail({ where: { id: parsed.id, userHost: IsNull() }, relations: { user: true, renote: true } }) .then(async note => { - const author = await this.usersRepository.findOneByOrFail({ id: note.userId }); + const author = note.user ?? await this.cacheService.findUserById(note.userId); if (parsed.rest === 'activity') { - // this refers to the create activity and not the note itself - return this.apRendererService.addContext(this.apRendererService.renderCreate(await this.apRendererService.renderNote(note, author), note)); + return await this.apRendererService.renderNoteOrRenoteActivity(note, author); + } else if (!isPureRenote(note)) { + const apNote = await this.apRendererService.renderNote(note, author); + return this.apRendererService.addContext(apNote); } else { - return this.apRendererService.renderNote(note, author); + throw new IdentifiableError('732c2633-3395-4d51-a9b7-c7084774e3e7', `Failed to resolve local ${url}: cannot resolve a boost as note`); } }) as Promise<IObjectWithId>; case 'users': - return this.usersRepository.findOneByOrFail({ id: parsed.id }) + return this.cacheService.findLocalUserById(parsed.id) .then(user => this.apRendererService.renderPerson(user as MiLocalUser)); case 'questions': // Polls are indexed by the note they are attached to. return Promise.all([ - this.notesRepository.findOneByOrFail({ id: parsed.id }), - this.pollsRepository.findOneByOrFail({ noteId: parsed.id }), + this.notesRepository.findOneByOrFail({ id: parsed.id, userHost: IsNull() }), + this.pollsRepository.findOneByOrFail({ noteId: parsed.id, userHost: IsNull() }), ]) .then(([note, poll]) => this.apRendererService.renderQuestion({ id: note.userId }, note, poll)) as Promise<IObjectWithId>; case 'likes': - return this.noteReactionsRepository.findOneByOrFail({ id: parsed.id }).then(async reaction => - this.apRendererService.addContext(await this.apRendererService.renderLike(reaction, { uri: null }))); + return this.noteReactionsRepository.findOneOrFail({ where: { id: parsed.id }, relations: { user: true } }).then(async reaction => { + if (reaction.user?.host != null) { + throw new IdentifiableError('02b40cd0-fa92-4b0c-acc9-fb2ada952ab8', `failed to resolve local ${url}: not a local reaction`); + } + return this.apRendererService.addContext(await this.apRendererService.renderLike(reaction, { uri: null })); + }); case 'follows': return this.followRequestsRepository.findOneBy({ id: parsed.id }) .then(async followRequest => { - if (followRequest == null) throw new IdentifiableError('a9d946e5-d276-47f8-95fb-f04230289bb0', `resolveLocal - invalid follow request ID ${parsed.id}: ${url}`); + if (followRequest == null) throw new IdentifiableError('a9d946e5-d276-47f8-95fb-f04230289bb0', `failed to resolve local ${url}: invalid follow request ID`); const [follower, followee] = await Promise.all([ - this.usersRepository.findOneBy({ - id: followRequest.followerId, - host: IsNull(), - }), - this.usersRepository.findOneBy({ - id: followRequest.followeeId, - host: Not(IsNull()), - }), + this.cacheService.findLocalUserById(followRequest.followerId), + this.cacheService.findLocalUserById(followRequest.followeeId), ]); if (follower == null || followee == null) { - throw new IdentifiableError('06ae3170-1796-4d93-a697-2611ea6d83b6', `resolveLocal - follower or followee does not exist: ${url}`); + throw new IdentifiableError('06ae3170-1796-4d93-a697-2611ea6d83b6', `failed to resolve local ${url}: follower or followee does not exist`); } return this.apRendererService.addContext(this.apRendererService.renderFollow(follower as MiLocalUser | MiRemoteUser, followee as MiLocalUser | MiRemoteUser, url)); }); default: - throw new IdentifiableError('7a5d2fc0-94bc-4db6-b8b8-1bf24a2e23d0', `resolveLocal: type ${parsed.type} unhandled: ${url}`); + throw new IdentifiableError('7a5d2fc0-94bc-4db6-b8b8-1bf24a2e23d0', `failed to resolve local ${url}: unsupported type ${parsed.type}`); } } } @@ -314,6 +439,7 @@ export class ApResolverService { private loggerService: LoggerService, private readonly apLogService: ApLogService, private readonly apUtilityService: ApUtilityService, + private readonly cacheService: CacheService, ) { } @@ -339,6 +465,7 @@ export class ApResolverService { this.loggerService, this.apLogService, this.apUtilityService, + this.cacheService, opts?.recursionLimit, ); } diff --git a/packages/backend/src/core/activitypub/ApUtilityService.ts b/packages/backend/src/core/activitypub/ApUtilityService.ts index c3958cdf42..227dc3b9b3 100644 --- a/packages/backend/src/core/activitypub/ApUtilityService.ts +++ b/packages/backend/src/core/activitypub/ApUtilityService.ts @@ -24,7 +24,7 @@ export class ApUtilityService { public assertIdMatchesUrlAuthority(object: IObject, url: string): void { // This throws if the ID is missing or invalid, but that's ok. // Anonymous objects are impossible to verify, so we don't allow fetching them. - const id = getApId(object); + const id = getApId(object, url); // Make sure the object ID matches the final URL (which is where it actually exists). // The caller (ApResolverService) will verify the ID against the original / entry URL, which ensures that all three match. @@ -80,7 +80,6 @@ export class ApUtilityService { /** * Verifies that a provided URL is in a format acceptable for federation. * @throws {IdentifiableError} If URL cannot be parsed - * @throws {IdentifiableError} If URL contains a fragment * @throws {IdentifiableError} If URL is not HTTPS */ public assertApUrl(url: string | URL): void { @@ -93,11 +92,6 @@ export class ApUtilityService { } } - // Hash component breaks federation - if (url.hash) { - throw new IdentifiableError('0bedd29b-e3bf-4604-af51-d3352e2518af', `invalid AP url ${url}: contains a fragment (#)`); - } - // Must be HTTPS if (!this.checkHttps(url)) { throw new IdentifiableError('0bedd29b-e3bf-4604-af51-d3352e2518af', `invalid AP url ${url}: unsupported protocol ${url.protocol}`); diff --git a/packages/backend/src/core/activitypub/JsonLdService.ts b/packages/backend/src/core/activitypub/JsonLdService.ts index 9d1e2e06cc..8f150ab201 100644 --- a/packages/backend/src/core/activitypub/JsonLdService.ts +++ b/packages/backend/src/core/activitypub/JsonLdService.ts @@ -8,25 +8,61 @@ import { Injectable } from '@nestjs/common'; import { UnrecoverableError } from 'bullmq'; import { HttpRequestService } from '@/core/HttpRequestService.js'; import { bindThis } from '@/decorators.js'; +import Logger from '@/logger.js'; +import { LoggerService } from '@/core/LoggerService.js'; +import { StatusError } from '@/misc/status-error.js'; import { CONTEXT, PRELOADED_CONTEXTS } from './misc/contexts.js'; import { validateContentTypeSetAsJsonLD } from './misc/validator.js'; -import type { JsonLdDocument } from 'jsonld'; +import type { ContextDefinition, JsonLdDocument } from 'jsonld'; import type { JsonLd as JsonLdObject, RemoteDocument } from 'jsonld/jsonld-spec.js'; +// https://stackoverflow.com/a/66252656 +type RemoveIndex<T> = { + [ K in keyof T as string extends K + ? never + : number extends K + ? never + : symbol extends K + ? never + : K + ] : T[K]; +}; + +export type Document = RemoveIndex<JsonLdDocument>; + +export type Signature = { + id?: string; + type: string; + creator: string; + domain?: string; + nonce: string; + created: string; + signatureValue: string; +}; + +export type Signed<T extends Document> = T & { + signature: Signature; +}; + +export function isSigned<T extends Document>(doc: T): doc is Signed<T> { + return 'signature' in doc && typeof(doc.signature) === 'object'; +} + // RsaSignature2017 implementation is based on https://github.com/transmute-industries/RsaSignature2017 -class JsonLd { - public debug = false; - public preLoad = true; - public loderTimeout = 5000; +@Injectable() +export class JsonLdService { + private readonly logger: Logger; constructor( private httpRequestService: HttpRequestService, + loggerService: LoggerService, ) { + this.logger = loggerService.getLogger('json-ld'); } @bindThis - public async signRsaSignature2017(data: any, privateKey: string, creator: string, domain?: string, created?: Date): Promise<any> { + public async signRsaSignature2017<T extends Document>(data: T, privateKey: string, creator: string, domain?: string, created?: Date): Promise<Signed<T>> { const options: { type: string; creator: string; @@ -62,7 +98,7 @@ class JsonLd { } @bindThis - public async verifyRsaSignature2017(data: any, publicKey: string): Promise<boolean> { + public async verifyRsaSignature2017(data: Signed<Document>, publicKey: string): Promise<boolean> { const toBeSigned = await this.createVerifyData(data, data.signature); const verifier = crypto.createVerify('sha256'); verifier.update(toBeSigned); @@ -70,7 +106,7 @@ class JsonLd { } @bindThis - public async createVerifyData(data: any, options: any): Promise<string> { + public async createVerifyData<T extends Document>(data: T, options: Partial<Signature>): Promise<string> { const transformedOptions = { ...options, '@context': 'https://w3id.org/identity/v1', @@ -80,17 +116,18 @@ class JsonLd { delete transformedOptions['signatureValue']; const canonizedOptions = await this.normalize(transformedOptions); const optionsHash = this.sha256(canonizedOptions.toString()); - const transformedData = { ...data }; + const transformedData = { ...data } as T & { signature?: unknown }; delete transformedData['signature']; const cannonidedData = await this.normalize(transformedData); - if (this.debug) console.debug(`cannonidedData: ${cannonidedData}`); + this.logger.debug('cannonidedData', cannonidedData); const documentHash = this.sha256(cannonidedData.toString()); const verifyData = `${optionsHash}${documentHash}`; return verifyData; } @bindThis - public async compact(data: any, context: any = CONTEXT): Promise<JsonLdDocument> { + // TODO our default CONTEXT isn't valid for the library, is this a bug? + public async compact(data: Document, context: ContextDefinition = CONTEXT as unknown as ContextDefinition): Promise<Document> { const customLoader = this.getLoader(); // XXX: Importing jsonld dynamically since Jest frequently fails to import it statically // https://github.com/misskey-dev/misskey/pull/9894#discussion_r1103753595 @@ -100,7 +137,7 @@ class JsonLd { } @bindThis - public async normalize(data: JsonLdDocument): Promise<string> { + public async normalize(data: Document): Promise<string> { const customLoader = this.getLoader(); return (await import('jsonld')).default.normalize(data, { documentLoader: customLoader, @@ -112,9 +149,9 @@ class JsonLd { return async (url: string): Promise<RemoteDocument> => { if (!/^https?:\/\//.test(url)) throw new UnrecoverableError(`Invalid URL: ${url}`); - if (this.preLoad) { + { if (url in PRELOADED_CONTEXTS) { - if (this.debug) console.debug(`HIT: ${url}`); + this.logger.debug(`Preload HIT: ${url}`); return { contextUrl: undefined, document: PRELOADED_CONTEXTS[url], @@ -123,7 +160,7 @@ class JsonLd { } } - if (this.debug) console.debug(`MISS: ${url}`); + this.logger.debug(`Preload MISS: ${url}`); const document = await this.fetchDocument(url); return { contextUrl: undefined, @@ -141,7 +178,6 @@ class JsonLd { headers: { Accept: 'application/ld+json, application/json', }, - timeout: this.loderTimeout, }, { throwErrorWhenResponseNotOk: false, @@ -149,7 +185,7 @@ class JsonLd { }, ).then(res => { if (!res.ok) { - throw new Error(`JSON-LD fetch failed with ${res.status} ${res.statusText}: ${url}`); + throw new StatusError(`failed to fetch JSON-LD from ${url}`, res.status, res.statusText); } else { return res.json(); } @@ -165,16 +201,3 @@ class JsonLd { return hash.digest('hex'); } } - -@Injectable() -export class JsonLdService { - constructor( - private httpRequestService: HttpRequestService, - ) { - } - - @bindThis - public use(): JsonLd { - return new JsonLd(this.httpRequestService); - } -} diff --git a/packages/backend/src/core/activitypub/misc/contexts.ts b/packages/backend/src/core/activitypub/misc/contexts.ts index 5c0b8ffcbb..fa003b1791 100644 --- a/packages/backend/src/core/activitypub/misc/contexts.ts +++ b/packages/backend/src/core/activitypub/misc/contexts.ts @@ -540,12 +540,20 @@ const extension_context_definition = { quoteUrl: 'as:quoteUrl', fedibird: 'http://fedibird.com/ns#', quoteUri: 'fedibird:quoteUri', + quote: { + '@id': 'https://w3id.org/fep/044f#quote', + '@type': '@id', + }, // Mastodon toot: 'http://joinmastodon.org/ns#', Emoji: 'toot:Emoji', featured: 'toot:featured', discoverable: 'toot:discoverable', indexable: 'toot:indexable', + attributionDomains: { + '@id': 'toot:attributionDomains', + '@type': '@id', + }, // schema schema: 'http://schema.org#', PropertyValue: 'schema:PropertyValue', diff --git a/packages/backend/src/core/activitypub/misc/validator.ts b/packages/backend/src/core/activitypub/misc/validator.ts index 0ff83659c1..5cd2ddf006 100644 --- a/packages/backend/src/core/activitypub/misc/validator.ts +++ b/packages/backend/src/core/activitypub/misc/validator.ts @@ -3,15 +3,14 @@ * SPDX-License-Identifier: AGPL-3.0-only */ +import { IdentifiableError } from '@/misc/identifiable-error.js'; import type { Response } from 'node-fetch'; -// TODO throw identifiable or unrecoverable errors - export function validateContentTypeSetAsActivityPub(response: Response): void { const contentType = (response.headers.get('content-type') ?? '').toLowerCase(); if (contentType === '') { - throw new Error(`invalid content type of AP response - no content-type header: ${response.url}`); + throw new IdentifiableError('d09dc850-b76c-4f45-875a-7389339d78b8', `invalid AP response from ${response.url}: no content-type header`, true); } if ( contentType.startsWith('application/activity+json') || @@ -19,7 +18,7 @@ export function validateContentTypeSetAsActivityPub(response: Response): void { ) { return; } - throw new Error(`invalid content type of AP response - content type is not application/activity+json or application/ld+json: ${response.url}`); + throw new IdentifiableError('dc110060-a5f2-461d-808b-39c62702ca64', `invalid AP response from ${response.url}: content type "${contentType}" is not application/activity+json or application/ld+json`); } const plusJsonSuffixRegex = /^\s*(application|text)\/[a-zA-Z0-9\.\-\+]+\+json\s*(;|$)/; @@ -28,7 +27,7 @@ export function validateContentTypeSetAsJsonLD(response: Response): void { const contentType = (response.headers.get('content-type') ?? '').toLowerCase(); if (contentType === '') { - throw new Error(`invalid content type of JSON LD - no content-type header: ${response.url}`); + throw new IdentifiableError('45793ab7-7648-4886-b503-429f8a0d0f73', `invalid AP response from ${response.url}: no content-type header`, true); } if ( contentType.startsWith('application/ld+json') || @@ -37,5 +36,5 @@ export function validateContentTypeSetAsJsonLD(response: Response): void { ) { return; } - throw new Error(`invalid content type of JSON LD - content type is not application/ld+json or application/json: ${response.url}`); + throw new IdentifiableError('4bf8f36b-4d33-4ac9-ad76-63fa11f354e9', `invalid AP response from ${response.url}: content type "${contentType}" is not application/ld+json or application/json`); } diff --git a/packages/backend/src/core/activitypub/models/ApImageService.ts b/packages/backend/src/core/activitypub/models/ApImageService.ts index 423044b985..7a16972ea4 100644 --- a/packages/backend/src/core/activitypub/models/ApImageService.ts +++ b/packages/backend/src/core/activitypub/models/ApImageService.ts @@ -18,7 +18,7 @@ import type { Config } from '@/config.js'; import { IdentifiableError } from '@/misc/identifiable-error.js'; import { ApResolverService } from '../ApResolverService.js'; import { ApLoggerService } from '../ApLoggerService.js'; -import { isDocument, type IObject } from '../type.js'; +import { getNullableApId, isDocument, type IObject } from '../type.js'; @Injectable() export class ApImageService { @@ -48,7 +48,7 @@ export class ApImageService { public async createImage(actor: MiRemoteUser, value: string | IObject): Promise<MiDriveFile | null> { // 投稿者が凍結されていたらスキップ if (actor.isSuspended) { - throw new IdentifiableError('85ab9bd7-3a41-4530-959d-f07073900109', `actor has been suspended: ${actor.uri}`); + throw new IdentifiableError('85ab9bd7-3a41-4530-959d-f07073900109', `failed to create image ${getNullableApId(value)}: actor ${actor.id} has been suspended`); } const image = await this.apResolverService.createResolver().resolve(value); diff --git a/packages/backend/src/core/activitypub/models/ApNoteService.ts b/packages/backend/src/core/activitypub/models/ApNoteService.ts index 7811b81795..2a28405121 100644 --- a/packages/backend/src/core/activitypub/models/ApNoteService.ts +++ b/packages/backend/src/core/activitypub/models/ApNoteService.ts @@ -26,7 +26,8 @@ import { bindThis } from '@/decorators.js'; import { checkHttps } from '@/misc/check-https.js'; import { IdentifiableError } from '@/misc/identifiable-error.js'; import { isRetryableError } from '@/misc/is-retryable-error.js'; -import { getOneApId, getApId, validPost, isEmoji, getApType, isApObject, isDocument, IApDocument } from '../type.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; +import { getOneApId, getApId, validPost, isEmoji, getApType, isApObject, isDocument, IApDocument, isLink } from '../type.js'; import { ApLoggerService } from '../ApLoggerService.js'; import { ApMfmService } from '../ApMfmService.js'; import { ApDbResolverService } from '../ApDbResolverService.js'; @@ -100,29 +101,29 @@ export class ApNoteService { const apType = getApType(object); if (apType == null || !validPost.includes(apType)) { - return new IdentifiableError('d450b8a9-48e4-4dab-ae36-f4db763fda7c', `invalid Note: invalid object type ${apType ?? 'undefined'}`); + return new IdentifiableError('d450b8a9-48e4-4dab-ae36-f4db763fda7c', `invalid Note from ${uri}: invalid object type ${apType ?? 'undefined'}`); } if (object.id && this.utilityService.extractDbHost(object.id) !== expectHost) { - return new IdentifiableError('d450b8a9-48e4-4dab-ae36-f4db763fda7c', `invalid Note: id has different host. expected: ${expectHost}, actual: ${this.utilityService.extractDbHost(object.id)}`); + return new IdentifiableError('d450b8a9-48e4-4dab-ae36-f4db763fda7c', `invalid Note from ${uri}: id has different host. expected: ${expectHost}, actual: ${this.utilityService.extractDbHost(object.id)}`); } const actualHost = object.attributedTo && this.utilityService.extractDbHost(getOneApId(object.attributedTo)); if (object.attributedTo && actualHost !== expectHost) { - return new IdentifiableError('d450b8a9-48e4-4dab-ae36-f4db763fda7c', `invalid Note: attributedTo has different host. expected: ${expectHost}, actual: ${actualHost}`); + return new IdentifiableError('d450b8a9-48e4-4dab-ae36-f4db763fda7c', `invalid Note from ${uri}: attributedTo has different host. expected: ${expectHost}, actual: ${actualHost}`); } if (object.published && !this.idService.isSafeT(new Date(object.published).valueOf())) { - return new IdentifiableError('d450b8a9-48e4-4dab-ae36-f4db763fda7c', 'invalid Note: published timestamp is malformed'); + return new IdentifiableError('d450b8a9-48e4-4dab-ae36-f4db763fda7c', 'invalid Note from ${uri}: published timestamp is malformed'); } if (actor) { const attribution = (object.attributedTo) ? getOneApId(object.attributedTo) : actor.uri; if (attribution !== actor.uri) { - return new IdentifiableError('d450b8a9-48e4-4dab-ae36-f4db763fda7c', `invalid Note: attribution does not match the actor that send it. attribution: ${attribution}, actor: ${actor.uri}`); + return new IdentifiableError('d450b8a9-48e4-4dab-ae36-f4db763fda7c', `invalid Note from ${uri}: attribution does not match the actor that send it. attribution: ${attribution}, actor: ${actor.uri}`); } if (user && attribution !== user.uri) { - return new IdentifiableError('d450b8a9-48e4-4dab-ae36-f4db763fda7c', `invalid Note: updated attribution does not match original attribution. updated attribution: ${user.uri}, original attribution: ${attribution}`); + return new IdentifiableError('d450b8a9-48e4-4dab-ae36-f4db763fda7c', `invalid Note from ${uri}: updated attribution does not match original attribution. updated attribution: ${user.uri}, original attribution: ${attribution}`); } } @@ -161,7 +162,7 @@ export class ApNoteService { const entryUri = getApId(value); const err = this.validateNote(object, entryUri, actor); if (err) { - this.logger.error(err.message, { + this.logger.error(`Error creating note: ${renderInlineError(err)}`, { resolver: { history: resolver.getHistory() }, value, object, @@ -174,11 +175,11 @@ export class ApNoteService { this.logger.debug(`Note fetched: ${JSON.stringify(note, null, 2)}`); if (note.id == null) { - throw new UnrecoverableError(`Refusing to create note without id: ${entryUri}`); + throw new UnrecoverableError(`failed to create note ${entryUri}: missing ID`); } if (!checkHttps(note.id)) { - throw new UnrecoverableError(`unexpected schema of note.id ${note.id} in ${entryUri}`); + throw new UnrecoverableError(`failed to create note ${entryUri}: unexpected schema`); } const url = this.apUtilityService.findBestObjectUrl(note); @@ -187,7 +188,7 @@ export class ApNoteService { // 投稿者をフェッチ if (note.attributedTo == null) { - throw new UnrecoverableError(`invalid note.attributedTo ${note.attributedTo} in ${entryUri}`); + throw new UnrecoverableError(`failed to create note: ${entryUri}: missing attributedTo`); } const uri = getOneApId(note.attributedTo); @@ -196,7 +197,7 @@ export class ApNoteService { // eslint-disable-next-line no-param-reassign actor ??= await this.apPersonService.fetchPerson(uri) as MiRemoteUser | undefined; if (actor && actor.isSuspended) { - throw new IdentifiableError('85ab9bd7-3a41-4530-959d-f07073900109', `actor ${uri} has been suspended: ${entryUri}`); + throw new IdentifiableError('85ab9bd7-3a41-4530-959d-f07073900109', `failed to create note ${entryUri}: actor ${uri} has been suspended`); } const apMentions = await this.apMentionService.extractApMentions(note.tag, resolver); @@ -223,7 +224,7 @@ export class ApNoteService { */ const hasProhibitedWords = this.noteCreateService.checkProhibitedWordsContain({ cw, text, pollChoices: poll?.choices }); if (hasProhibitedWords) { - throw new IdentifiableError('689ee33f-f97c-479a-ac49-1b9f8140af99', `Note contains prohibited words: ${entryUri}`); + throw new IdentifiableError('689ee33f-f97c-479a-ac49-1b9f8140af99', `failed to create note ${entryUri}: contains prohibited words`); } //#endregion @@ -232,7 +233,7 @@ export class ApNoteService { // 解決した投稿者が凍結されていたらスキップ if (actor.isSuspended) { - throw new IdentifiableError('85ab9bd7-3a41-4530-959d-f07073900109', `actor has been suspended: ${entryUri}`); + throw new IdentifiableError('85ab9bd7-3a41-4530-959d-f07073900109', `failed to create note ${entryUri}: actor ${actor.id} has been suspended`); } const noteAudience = await this.apAudienceService.parseAudience(actor, note.to, note.cc, resolver); @@ -269,15 +270,15 @@ export class ApNoteService { ? await this.resolveNote(note.inReplyTo, { resolver }) .then(x => { if (x == null) { - this.logger.warn('Specified inReplyTo, but not found'); - throw new Error(`could not fetch inReplyTo ${note.inReplyTo} for note ${entryUri}`); + this.logger.warn(`Specified inReplyTo "${note.inReplyTo}", but not found`); + throw new IdentifiableError('1ebf0a96-2769-4973-a6c2-3dcbad409dff', `failed to create note ${entryUri}: could not fetch inReplyTo ${note.inReplyTo}`, true); } return x; }) .catch(async err => { - this.logger.warn(`error ${err.statusCode ?? err} fetching inReplyTo ${note.inReplyTo} for note ${entryUri}`); - throw err; + this.logger.warn(`error ${renderInlineError(err)} fetching inReplyTo ${note.inReplyTo} for note ${entryUri}`); + throw new IdentifiableError('1ebf0a96-2769-4973-a6c2-3dcbad409dff', `failed to create note ${entryUri}: could not fetch inReplyTo ${note.inReplyTo}`, true, err); }) : null; @@ -285,6 +286,13 @@ export class ApNoteService { const quote = await this.getQuote(note, entryUri, resolver); const processErrors = quote === null ? ['quoteUnavailable'] : null; + if (reply && reply.userHost == null && reply.localOnly) { + throw new IdentifiableError('12e23cec-edd9-442b-aa48-9c21f0c3b215', 'Cannot reply to local-only note'); + } + if (quote && quote.userHost == null && quote.localOnly) { + throw new IdentifiableError('12e23cec-edd9-442b-aa48-9c21f0c3b215', 'Cannot quote a local-only note'); + } + // vote if (reply && reply.hasPoll) { const poll = await this.pollsRepository.findOneByOrFail({ noteId: reply.id }); @@ -341,7 +349,7 @@ export class ApNoteService { this.logger.info('The note is already inserted while creating itself, reading again'); const duplicate = await this.fetchNote(value); if (!duplicate) { - throw new Error(`The note creation failed with duplication error even when there is no duplication: ${entryUri}`); + throw new IdentifiableError('39c328e1-e829-458b-bfc9-65dcd513d1f8', `failed to create note ${entryUri}: the note creation failed with duplication error even when there is no duplication. This is likely a bug.`); } return duplicate; } @@ -355,45 +363,39 @@ export class ApNoteService { const noteUri = getApId(value); // URIがこのサーバーを指しているならスキップ - if (noteUri.startsWith(this.config.url + '/')) throw new UnrecoverableError(`uri points local: ${noteUri}`); + if (this.utilityService.isUriLocal(noteUri)) { + throw new UnrecoverableError(`failed to update note ${noteUri}: uri is local`); + } //#region このサーバーに既に登録されているか const updatedNote = await this.notesRepository.findOneBy({ uri: noteUri }); - if (updatedNote == null) throw new Error(`Note is not registered (no note): ${noteUri}`); + if (updatedNote == null) throw new UnrecoverableError(`failed to update note ${noteUri}: note does not exist`); const user = await this.usersRepository.findOneBy({ id: updatedNote.userId }) as MiRemoteUser | null; - if (user == null) throw new Error(`Note is not registered (no user): ${noteUri}`); + if (user == null) throw new UnrecoverableError(`failed to update note ${noteUri}: user does not exist`); - // eslint-disable-next-line no-param-reassign - if (resolver == null) resolver = this.apResolverService.createResolver(); + resolver ??= this.apResolverService.createResolver(); const object = await resolver.resolve(value); const entryUri = getApId(value); const err = this.validateNote(object, entryUri, actor, user); if (err) { - this.logger.error(err.message, { - resolver: { history: resolver.getHistory() }, - value, - object, - }); + this.logger.error(`Failed to update note ${noteUri}: ${renderInlineError(err)}`); throw err; } // `validateNote` checks that the actor and user are one and the same - // eslint-disable-next-line no-param-reassign actor ??= user; const note = object as IPost; - this.logger.debug(`Note fetched: ${JSON.stringify(note, null, 2)}`); - if (note.id == null) { - throw new UnrecoverableError(`Refusing to update note without id: ${noteUri}`); + throw new UnrecoverableError(`failed to update note ${entryUri}: missing ID`); } if (!checkHttps(note.id)) { - throw new UnrecoverableError(`unexpected schema of note.id ${note.id} in ${noteUri}`); + throw new UnrecoverableError(`failed to update note ${entryUri}: unexpected schema`); } const url = this.apUtilityService.findBestObjectUrl(note); @@ -401,7 +403,7 @@ export class ApNoteService { this.logger.info(`Creating the Note: ${note.id}`); if (actor.isSuspended) { - throw new IdentifiableError('85ab9bd7-3a41-4530-959d-f07073900109', `actor ${actor.id} has been suspended: ${noteUri}`); + throw new IdentifiableError('85ab9bd7-3a41-4530-959d-f07073900109', `failed to update note ${entryUri}: actor ${actor.id} has been suspended`); } const apMentions = await this.apMentionService.extractApMentions(note.tag, resolver); @@ -428,7 +430,7 @@ export class ApNoteService { */ const hasProhibitedWords = this.noteCreateService.checkProhibitedWordsContain({ cw, text, pollChoices: poll?.choices }); if (hasProhibitedWords) { - throw new IdentifiableError('689ee33f-f97c-479a-ac49-1b9f8140af99', `Note contains prohibited words: ${noteUri}`); + throw new IdentifiableError('689ee33f-f97c-479a-ac49-1b9f8140af99', `failed to update note ${noteUri}: contains prohibited words`); } //#endregion @@ -466,15 +468,15 @@ export class ApNoteService { ? await this.resolveNote(note.inReplyTo, { resolver }) .then(x => { if (x == null) { - this.logger.warn('Specified inReplyTo, but not found'); - throw new Error(`could not fetch inReplyTo ${note.inReplyTo} for note ${entryUri}`); + this.logger.warn(`Specified inReplyTo "${note.inReplyTo}", but not found`); + throw new IdentifiableError('1ebf0a96-2769-4973-a6c2-3dcbad409dff', `failed to update note ${entryUri}: could not fetch inReplyTo ${note.inReplyTo}`, true); } return x; }) .catch(async err => { - this.logger.warn(`error ${err.statusCode ?? err} fetching inReplyTo ${note.inReplyTo} for note ${entryUri}`); - throw err; + this.logger.warn(`error ${renderInlineError(err)} fetching inReplyTo ${note.inReplyTo} for note ${entryUri}`); + throw new IdentifiableError('1ebf0a96-2769-4973-a6c2-3dcbad409dff', `failed to update note ${entryUri}: could not fetch inReplyTo ${note.inReplyTo}`, true, err); }) : null; @@ -482,6 +484,10 @@ export class ApNoteService { const quote = await this.getQuote(note, entryUri, resolver); const processErrors = quote === null ? ['quoteUnavailable'] : null; + if (quote && quote.userHost == null && quote.localOnly) { + throw new IdentifiableError('12e23cec-edd9-442b-aa48-9c21f0c3b215', 'Cannot quote a local-only note'); + } + // vote if (reply && reply.hasPoll) { const poll = await this.pollsRepository.findOneByOrFail({ noteId: reply.id }); @@ -538,7 +544,7 @@ export class ApNoteService { this.logger.info('The note is already inserted while creating itself, reading again'); const duplicate = await this.fetchNote(value); if (!duplicate) { - throw new Error(`The note creation failed with duplication error even when there is no duplication: ${noteUri}`); + throw new IdentifiableError('39c328e1-e829-458b-bfc9-65dcd513d1f8', `failed to update note ${entryUri}: the note update failed with duplication error even when there is no duplication. This is likely a bug.`); } return duplicate; } @@ -555,8 +561,7 @@ export class ApNoteService { const uri = getApId(value); if (!this.utilityService.isFederationAllowedUri(uri)) { - // TODO convert to identifiable error - throw new StatusError(`blocked host: ${uri}`, 451, 'blocked host'); + throw new IdentifiableError('04620a7e-044e-45ce-b72c-10e1bdc22e69', `failed to resolve note ${uri}: host is blocked`); } //#region このサーバーに既に登録されていたらそれを返す @@ -566,8 +571,7 @@ export class ApNoteService { // Bail if local URI doesn't exist if (this.utilityService.isUriLocal(uri)) { - // TODO convert to identifiable error - throw new StatusError(`cannot resolve local note: ${uri}`, 400, 'cannot resolve local note'); + throw new IdentifiableError('cbac7358-23f2-4c70-833e-cffb4bf77913', `failed to resolve note ${uri}: URL is local and does not exist`); } const unlock = await this.appLockService.getApLock(uri); @@ -653,9 +657,29 @@ export class ApNoteService { */ private async getQuote(note: IPost, entryUri: string, resolver: Resolver): Promise<MiNote | null | undefined> { const quoteUris = new Set<string>(); - if (note._misskey_quote) quoteUris.add(note._misskey_quote); - if (note.quoteUrl) quoteUris.add(note.quoteUrl); - if (note.quoteUri) quoteUris.add(note.quoteUri); + if (note._misskey_quote && typeof(note._misskey_quote as unknown) === 'string') quoteUris.add(note._misskey_quote); + if (note.quoteUrl && typeof(note.quoteUrl as unknown) === 'string') quoteUris.add(note.quoteUrl); + if (note.quoteUri && typeof(note.quoteUri as unknown) === 'string') quoteUris.add(note.quoteUri); + + // https://codeberg.org/fediverse/fep/src/branch/main/fep/044f/fep-044f.md + if (note.quote && typeof(note.quote as unknown) === 'string') quoteUris.add(note.quote); + + // https://codeberg.org/fediverse/fep/src/branch/main/fep/e232/fep-e232.md + const tags = toArray(note.tag).filter(tag => typeof(tag) === 'object' && isLink(tag)); + for (const tag of tags) { + if (!tag.href || typeof (tag.href as unknown) !== 'string') continue; + + const mediaTypes = toArray(tag.mediaType); + if ( + !mediaTypes.includes('application/ld+json; profile="https://www.w3.org/ns/activitystreams"') && + !mediaTypes.includes('application/activity+json') + ) continue; + + const rels = toArray(tag.rel); + if (!rels.includes('https://misskey-hub.net/ns#_misskey_quote')) continue; + + quoteUris.add(tag.href); + } // No quote, return undefined if (quoteUris.size < 1) return undefined; @@ -674,18 +698,13 @@ export class ApNoteService { const quote = await this.resolveNote(uri, { resolver }); if (quote == null) { - this.logger.warn(`Failed to resolve quote "${uri}" for note "${entryUri}": request error`); + this.logger.warn(`Failed to resolve quote "${uri}" for note "${entryUri}": fetch failed`); return false; } return quote; } catch (e) { - if (e instanceof Error) { - this.logger.warn(`Failed to resolve quote "${uri}" for note "${entryUri}":`, e); - } else { - this.logger.warn(`Failed to resolve quote "${uri}" for note "${entryUri}": ${e}`); - } - + this.logger.warn(`Failed to resolve quote "${uri}" for note "${entryUri}": ${renderInlineError(e)}`); return isRetryableError(e); } }; diff --git a/packages/backend/src/core/activitypub/models/ApPersonService.ts b/packages/backend/src/core/activitypub/models/ApPersonService.ts index 5c6716a0b8..29f7459219 100644 --- a/packages/backend/src/core/activitypub/models/ApPersonService.ts +++ b/packages/backend/src/core/activitypub/models/ApPersonService.ts @@ -7,7 +7,6 @@ import { Inject, Injectable, OnApplicationShutdown } from '@nestjs/common'; import promiseLimit from 'promise-limit'; import { DataSource } from 'typeorm'; import { ModuleRef } from '@nestjs/core'; -import { AbortError } from 'node-fetch'; import { UnrecoverableError } from 'bullmq'; import { DI } from '@/di-symbols.js'; import type { FollowingsRepository, InstancesRepository, MiMeta, UserProfilesRepository, UserPublickeysRepository, UsersRepository } from '@/models/_.js'; @@ -44,6 +43,8 @@ import { AppLockService } from '@/core/AppLockService.js'; import { MemoryKVCache } from '@/misc/cache.js'; import { HttpRequestService } from '@/core/HttpRequestService.js'; import { verifyFieldLinks } from '@/misc/verify-field-link.js'; +import { isRetryableError } from '@/misc/is-retryable-error.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; import { getApId, getApType, isActor, isCollection, isCollectionOrOrderedCollection, isPropertyValue } from '../type.js'; import { extractApHashtags } from './tag.js'; import type { OnModuleInit } from '@nestjs/common'; @@ -54,6 +55,7 @@ import type { ApLoggerService } from '../ApLoggerService.js'; import type { ApImageService } from './ApImageService.js'; import type { IActor, ICollection, IObject, IOrderedCollection } from '../type.js'; +import { IdentifiableError } from '@/misc/identifiable-error.js'; const nameLength = 128; const summaryLength = 2048; @@ -157,21 +159,21 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { const expectHost = this.utilityService.punyHostPSLDomain(uri); if (!isActor(x)) { - throw new UnrecoverableError(`invalid Actor type '${x.type}' in ${uri}`); + throw new UnrecoverableError(`invalid Actor ${uri}: unknown type '${x.type}'`); } if (!(typeof x.id === 'string' && x.id.length > 0)) { - throw new UnrecoverableError(`invalid Actor ${uri} - wrong id type`); + throw new UnrecoverableError(`invalid Actor ${uri}: wrong id type`); } if (!(typeof x.inbox === 'string' && x.inbox.length > 0)) { - throw new UnrecoverableError(`invalid Actor ${uri} - wrong inbox type`); + throw new UnrecoverableError(`invalid Actor ${uri}: wrong inbox type`); } this.apUtilityService.assertApUrl(x.inbox); const inboxHost = this.utilityService.punyHostPSLDomain(x.inbox); if (inboxHost !== expectHost) { - throw new UnrecoverableError(`invalid Actor ${uri} - wrong inbox ${inboxHost}`); + throw new UnrecoverableError(`invalid Actor ${uri}: wrong inbox host ${inboxHost}`); } const sharedInboxObject = x.sharedInbox ?? (x.endpoints ? x.endpoints.sharedInbox : undefined); @@ -179,7 +181,7 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { const sharedInbox = getApId(sharedInboxObject); this.apUtilityService.assertApUrl(sharedInbox); if (!(typeof sharedInbox === 'string' && sharedInbox.length > 0 && this.utilityService.punyHostPSLDomain(sharedInbox) === expectHost)) { - throw new UnrecoverableError(`invalid Actor ${uri} - wrong shared inbox ${sharedInbox}`); + throw new UnrecoverableError(`invalid Actor ${uri}: wrong shared inbox ${sharedInbox}`); } } @@ -190,7 +192,7 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { if (typeof collectionUri === 'string' && collectionUri.length > 0) { this.apUtilityService.assertApUrl(collectionUri); if (this.utilityService.punyHostPSLDomain(collectionUri) !== expectHost) { - throw new UnrecoverableError(`invalid Actor ${uri} - wrong ${collection} ${collectionUri}`); + throw new UnrecoverableError(`invalid Actor ${uri}: wrong ${collection} host ${collectionUri}`); } } else if (collectionUri != null) { throw new UnrecoverableError(`invalid Actor ${uri}: wrong ${collection} type`); @@ -199,7 +201,7 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { } if (!(typeof x.preferredUsername === 'string' && x.preferredUsername.length > 0 && x.preferredUsername.length <= 128 && /^\w([\w-.]*\w)?$/.test(x.preferredUsername))) { - throw new UnrecoverableError(`invalid Actor ${uri} - wrong username`); + throw new UnrecoverableError(`invalid Actor ${uri}: wrong username`); } // These fields are only informational, and some AP software allows these @@ -207,7 +209,7 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { // we can at least see these users and their activities. if (x.name) { if (!(typeof x.name === 'string' && x.name.length > 0)) { - throw new UnrecoverableError(`invalid Actor ${uri} - wrong name`); + throw new UnrecoverableError(`invalid Actor ${uri}: wrong name`); } x.name = truncate(x.name, nameLength); } else if (x.name === '') { @@ -216,24 +218,24 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { } if (x.summary) { if (!(typeof x.summary === 'string' && x.summary.length > 0)) { - throw new UnrecoverableError(`invalid Actor ${uri} - wrong summary`); + throw new UnrecoverableError(`invalid Actor ${uri}: wrong summary`); } x.summary = truncate(x.summary, summaryLength); } const idHost = this.utilityService.punyHostPSLDomain(x.id); if (idHost !== expectHost) { - throw new UnrecoverableError(`invalid Actor ${uri} - wrong id ${x.id}`); + throw new UnrecoverableError(`invalid Actor ${uri}: wrong id ${x.id}`); } if (x.publicKey) { if (typeof x.publicKey.id !== 'string') { - throw new UnrecoverableError(`invalid Actor ${uri} - wrong publicKey.id type`); + throw new UnrecoverableError(`invalid Actor ${uri}: wrong publicKey.id type`); } const publicKeyIdHost = this.utilityService.punyHostPSLDomain(x.publicKey.id); if (publicKeyIdHost !== expectHost) { - throw new UnrecoverableError(`invalid Actor ${uri} - wrong publicKey.id ${x.publicKey.id}`); + throw new UnrecoverableError(`invalid Actor ${uri}: wrong publicKey.id ${x.publicKey.id}`); } } @@ -271,8 +273,6 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { } private async resolveAvatarAndBanner(user: MiRemoteUser, icon: any, image: any, bgimg: any): Promise<Partial<Pick<MiRemoteUser, 'avatarId' | 'bannerId' | 'backgroundId' | 'avatarUrl' | 'bannerUrl' | 'backgroundUrl' | 'avatarBlurhash' | 'bannerBlurhash' | 'backgroundBlurhash'>>> { - if (user == null) throw new Error('failed to create user: user is null'); - const [avatar, banner, background] = await Promise.all([icon, image, bgimg].map(img => { // icon and image may be arrays // see https://www.w3.org/TR/activitystreams-vocabulary/#dfn-icon @@ -325,12 +325,11 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { */ @bindThis public async createPerson(uri: string, resolver?: Resolver): Promise<MiRemoteUser> { - if (typeof uri !== 'string') throw new UnrecoverableError(`uri is not string: ${uri}`); + if (typeof uri !== 'string') throw new UnrecoverableError(`failed to create user ${uri}: input is not string`); const host = this.utilityService.punyHost(uri); if (host === this.utilityService.toPuny(this.config.host)) { - // TODO convert to unrecoverable error - throw new StatusError(`cannot resolve local user: ${uri}`, 400, 'cannot resolve local user'); + throw new UnrecoverableError(`failed to create user ${uri}: URI is local`); } return await this._createPerson(uri, resolver); @@ -340,8 +339,7 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { const uri = getApId(value); const host = this.utilityService.punyHost(uri); - // eslint-disable-next-line no-param-reassign - if (resolver == null) resolver = this.apResolverService.createResolver(); + resolver ??= this.apResolverService.createResolver(); const object = await resolver.resolve(value); const person = this.validateActor(object, uri); @@ -356,14 +354,16 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { const [followingVisibility, followersVisibility] = await Promise.all( [ - this.isPublicCollection(person.following, resolver), - this.isPublicCollection(person.followers, resolver), + this.isPublicCollection(person.following, resolver, uri), + this.isPublicCollection(person.followers, resolver, uri), ].map((p): Promise<'public' | 'private'> => p .then(isPublic => isPublic ? 'public' : 'private') .catch(err => { - if (!(err instanceof StatusError) || err.isRetryable) { - this.logger.error('error occurred while fetching following/followers collection', { stack: err }); + // Permanent error implies hidden or inaccessible, which is a normal thing. + if (isRetryableError(err)) { + this.logger.error(`error occurred while fetching following/followers collection: ${renderInlineError(err)}`); } + return 'private'; }), ), @@ -372,7 +372,7 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { const bday = person['vcard:bday']?.match(/^\d{4}-\d{2}-\d{2}/); if (person.id == null) { - throw new UnrecoverableError(`Refusing to create person without id: ${uri}`); + throw new UnrecoverableError(`failed to create user ${uri}: missing ID`); } const url = this.apUtilityService.findBestObjectUrl(person); @@ -387,16 +387,27 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { const emojis = await this.apNoteService.extractEmojis(person.tag ?? [], host) .then(_emojis => _emojis.map(emoji => emoji.name)) .catch(err => { - this.logger.error('error occurred while fetching user emojis', { stack: err }); + // Permanent error implies hidden or inaccessible, which is a normal thing. + if (isRetryableError(err)) { + this.logger.error(`error occurred while fetching user emojis: ${renderInlineError(err)}`); + } return []; }); //#endregion //#region resolve counts - const _resolver = resolver ?? this.apResolverService.createResolver(); - const outboxcollection = await _resolver.resolveCollection(person.outbox).catch(() => { return null; }); - const followerscollection = await _resolver.resolveCollection(person.followers!).catch(() => { return null; }); - const followingcollection = await _resolver.resolveCollection(person.following!).catch(() => { return null; }); + const outboxCollection = person.outbox + ? await resolver.resolveCollection(person.outbox, true, uri).catch(() => { return null; }) + : null; + const followersCollection = person.followers + ? await resolver.resolveCollection(person.followers, true, uri).catch(() => { return null; }) + : null; + const followingCollection = person.following + ? await resolver.resolveCollection(person.following, true, uri).catch(() => { return null; }) + : null; + + // Register the instance first, to avoid FK errors + await this.federatedInstanceService.fetchOrRegister(host); try { // Start transaction @@ -423,9 +434,9 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { host, inbox: person.inbox, sharedInbox: person.sharedInbox ?? person.endpoints?.sharedInbox ?? null, - notesCount: outboxcollection?.totalItems ?? 0, - followersCount: followerscollection?.totalItems ?? 0, - followingCount: followingcollection?.totalItems ?? 0, + notesCount: outboxCollection?.totalItems ?? 0, + followersCount: followersCollection?.totalItems ?? 0, + followingCount: followingCollection?.totalItems ?? 0, followersUri: person.followers ? getApId(person.followers) : undefined, featured: person.featured ? getApId(person.featured) : undefined, uri: person.id, @@ -437,6 +448,11 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { makeNotesFollowersOnlyBefore: (person as any).makeNotesFollowersOnlyBefore ?? null, makeNotesHiddenBefore: (person as any).makeNotesHiddenBefore ?? null, emojis, + attributionDomains: Array.isArray(person.attributionDomains) + ? person.attributionDomains + .filter((a: unknown) => typeof(a) === 'string' && a.length > 0 && a.length <= 128) + .slice(0, 32) + : [], })) as MiRemoteUser; let _description: string | null = null; @@ -480,7 +496,7 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { user = u as MiRemoteUser; publicKey = await this.userPublickeysRepository.findOneBy({ userId: user.id }); } else { - this.logger.error(e instanceof Error ? e : new Error(e as string)); + this.logger.error('Error creating Person:', e instanceof Error ? e : new Error(e as string)); throw e; } } @@ -520,11 +536,19 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { // Register to the cache this.cacheService.uriPersonCache.set(user.uri, user); } catch (err) { - this.logger.error('error occurred while fetching user avatar/banner', { stack: err }); + // Permanent error implies hidden or inaccessible, which is a normal thing. + if (isRetryableError(err)) { + this.logger.error(`error occurred while fetching user avatar/banner: ${renderInlineError(err)}`); + } } //#endregion - await this.updateFeatured(user.id, resolver).catch(err => this.logger.error(err)); + await this.updateFeatured(user.id, resolver).catch(err => { + // Permanent error implies hidden or inaccessible, which is a normal thing. + if (isRetryableError(err)) { + this.logger.error(`Error updating featured notes: ${renderInlineError(err)}`); + } + }); return user; } @@ -541,7 +565,7 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { */ @bindThis public async updatePerson(uri: string, resolver?: Resolver | null, hint?: IObject, movePreventUris: string[] = []): Promise<string | void> { - if (typeof uri !== 'string') throw new UnrecoverableError('uri is not string'); + if (typeof uri !== 'string') throw new UnrecoverableError(`failed to update user ${uri}: input is not string`); // URIがこのサーバーを指しているならスキップ if (this.utilityService.isUriLocal(uri)) return; @@ -561,8 +585,11 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { this.logger.info(`Updating the Person: ${person.id}`); // カスタム絵文字取得 - const emojis = await this.apNoteService.extractEmojis(person.tag ?? [], exist.host).catch(e => { - this.logger.info(`extractEmojis: ${e}`); + const emojis = await this.apNoteService.extractEmojis(person.tag ?? [], exist.host).catch(err => { + // Permanent error implies hidden or inaccessible, which is a normal thing. + if (isRetryableError(err)) { + this.logger.error(`error occurred while fetching user emojis: ${renderInlineError(err)}`); + } return []; }); @@ -574,16 +601,18 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { const [followingVisibility, followersVisibility] = await Promise.all( [ - this.isPublicCollection(person.following, resolver), - this.isPublicCollection(person.followers, resolver), + this.isPublicCollection(person.following, resolver, exist.uri), + this.isPublicCollection(person.followers, resolver, exist.uri), ].map((p): Promise<'public' | 'private' | undefined> => p .then(isPublic => isPublic ? 'public' : 'private') .catch(err => { - if (!(err instanceof StatusError) || err.isRetryable) { - this.logger.error('error occurred while fetching following/followers collection', { stack: err }); + // Permanent error implies hidden or inaccessible, which is a normal thing. + if (isRetryableError(err)) { + this.logger.error(`error occurred while fetching following/followers collection: ${renderInlineError(err)}`); // Do not update the visibility on transient errors. return undefined; } + return 'private'; }), ), @@ -592,7 +621,7 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { const bday = person['vcard:bday']?.match(/^\d{4}-\d{2}-\d{2}/); if (person.id == null) { - throw new UnrecoverableError(`Refusing to update person without id: ${uri}`); + throw new UnrecoverableError(`failed to update user ${uri}: missing ID`); } const url = this.apUtilityService.findBestObjectUrl(person); @@ -620,7 +649,20 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { // We use "!== false" to handle incorrect types, missing / null values, and "default to true" logic. hideOnlineStatus: person.hideOnlineStatus !== false, isExplorable: person.discoverable !== false, - ...(await this.resolveAvatarAndBanner(exist, person.icon, person.image, person.backgroundUrl).catch(() => ({}))), + attributionDomains: Array.isArray(person.attributionDomains) + ? person.attributionDomains + .filter((a: unknown) => typeof(a) === 'string' && a.length > 0 && a.length <= 128) + .slice(0, 32) + : [], + ...(await this.resolveAvatarAndBanner(exist, person.icon, person.image, person.backgroundUrl).catch(err => { + // Permanent error implies hidden or inaccessible, which is a normal thing. + if (isRetryableError(err)) { + this.logger.error(`error occurred while fetching user avatar/banner: ${renderInlineError(err)}`); + } + + // Can't return null or destructuring operator will break + return {}; + })), } as Partial<MiRemoteUser> & Pick<MiRemoteUser, 'isBot' | 'isCat' | 'speakAsCat' | 'isLocked' | 'movedToUri' | 'alsoKnownAs' | 'isExplorable'>; const moving = ((): boolean => { @@ -699,12 +741,24 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { this.hashtagService.updateUsertags(exist, tags); // 該当ユーザーが既にフォロワーになっていた場合はFollowingもアップデートする - await this.followingsRepository.update( - { followerId: exist.id }, - { followerSharedInbox: person.sharedInbox ?? person.endpoints?.sharedInbox ?? null }, - ); + if (exist.inbox !== person.inbox || exist.sharedInbox !== (person.sharedInbox ?? person.endpoints?.sharedInbox)) { + await this.followingsRepository.update( + { followerId: exist.id }, + { + followerInbox: person.inbox, + followerSharedInbox: person.sharedInbox ?? person.endpoints?.sharedInbox ?? null, + }, + ); - await this.updateFeatured(exist.id, resolver).catch(err => this.logger.error(err)); + await this.cacheService.refreshFollowRelationsFor(exist.id); + } + + await this.updateFeatured(exist.id, resolver).catch(err => { + // Permanent error implies hidden or inaccessible, which is a normal thing. + if (isRetryableError(err)) { + this.logger.error(`Error updating featured notes: ${renderInlineError(err)}`); + } + }); const updated = { ...exist, ...updates }; @@ -743,8 +797,7 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { const uri = getApId(value); if (!this.utilityService.isFederationAllowedUri(uri)) { - // TODO convert to identifiable error - throw new StatusError(`blocked host: ${uri}`, 451, 'blocked host'); + throw new IdentifiableError('590719b3-f51f-48a9-8e7d-6f559ad00e5d', `failed to resolve person ${uri}: host is blocked`); } //#region このサーバーに既に登録されていたらそれを返す @@ -754,8 +807,7 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { // Bail if local URI doesn't exist if (this.utilityService.isUriLocal(uri)) { - // TODO convert to identifiable error - throw new StatusError(`cannot resolve local person: ${uri}`, 400, 'cannot resolve local person'); + throw new IdentifiableError('efb573fd-6b9e-4912-9348-a02f5603df4f', `failed to resolve person ${uri}: URL is local and does not exist`); } const unlock = await this.appLockService.getApLock(uri); @@ -799,16 +851,17 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { const _resolver = resolver ?? this.apResolverService.createResolver(); // Resolve to (Ordered)Collection Object - const collection = await _resolver.resolveCollection(user.featured).catch(err => { - if (err instanceof AbortError || err instanceof StatusError) { - this.logger.warn(`Failed to update featured notes: ${err.name}: ${err.message}`); - } else { - this.logger.error('Failed to update featured notes:', err); + const collection = user.featured ? await _resolver.resolveCollection(user.featured, true, user.uri).catch(err => { + // Permanent error implies hidden or inaccessible, which is a normal thing. + if (isRetryableError(err)) { + this.logger.warn(`Failed to update featured notes: ${renderInlineError(err)}`); } - }); + + return null; + }) : null; if (!collection) return; - if (!isCollectionOrOrderedCollection(collection)) throw new UnrecoverableError(`featured ${user.featured} is not Collection or OrderedCollection in ${user.uri}`); + if (!isCollectionOrOrderedCollection(collection)) throw new UnrecoverableError(`failed to update user ${user.uri}: featured ${user.featured} is not Collection or OrderedCollection`); // Resolve to Object(may be Note) arrays const unresolvedItems = isCollection(collection) ? collection.items : collection.orderedItems; @@ -891,11 +944,13 @@ export class ApPersonService implements OnModuleInit, OnApplicationShutdown { } @bindThis - private async isPublicCollection(collection: string | ICollection | IOrderedCollection | undefined, resolver: Resolver): Promise<boolean> { + private async isPublicCollection(collection: string | ICollection | IOrderedCollection | undefined, resolver: Resolver, sentFrom: string): Promise<boolean> { if (collection) { - const resolved = await resolver.resolveCollection(collection); - if (resolved.first || (resolved as ICollection).items || (resolved as IOrderedCollection).orderedItems) { - return true; + const resolved = await resolver.resolveCollection(collection, true, sentFrom).catch(() => null); + if (resolved) { + if (resolved.first || (resolved as ICollection).items || (resolved as IOrderedCollection).orderedItems) { + return true; + } } } diff --git a/packages/backend/src/core/activitypub/models/ApQuestionService.ts b/packages/backend/src/core/activitypub/models/ApQuestionService.ts index 335ca189ec..80900be2dc 100644 --- a/packages/backend/src/core/activitypub/models/ApQuestionService.ts +++ b/packages/backend/src/core/activitypub/models/ApQuestionService.ts @@ -93,7 +93,6 @@ export class ApQuestionService { // eslint-disable-next-line no-param-reassign if (resolver == null) resolver = this.apResolverService.createResolver(); const question = await resolver.resolve(value); - this.logger.debug(`fetched question: ${JSON.stringify(question, null, 2)}`); if (!isQuestion(question)) throw new UnrecoverableError(`object ${getApType(question)} is not a Question: ${uri}`); diff --git a/packages/backend/src/core/activitypub/type.ts b/packages/backend/src/core/activitypub/type.ts index 281733d484..554420d670 100644 --- a/packages/backend/src/core/activitypub/type.ts +++ b/packages/backend/src/core/activitypub/type.ts @@ -35,6 +35,7 @@ export interface IObject { mediaType?: string; url?: ApObject | string; href?: string; + rel?: string | string[]; tag?: IObject | IObject[]; sensitive?: boolean; } @@ -43,6 +44,28 @@ export interface IObjectWithId extends IObject { id: string; } +export function isObjectWithId(object: IObject): object is IObjectWithId { + return typeof(object.id) === 'string'; +} + +export interface IAnonymousObject extends IObject { + id: undefined; +} + +export function isAnonymousObject(object: IObject): object is IAnonymousObject { + return object.id === undefined; +} + +export interface ILink extends IObject { + '@context'?: string | string[] | Obj | Obj[]; + type: 'Link' | 'Mention'; + href: string; +} + +export const isLink = (object: IObject): object is ILink => + (getApType(object) === 'Link' || getApType(object) === 'Link') && + typeof object.href === 'string'; + /** * Get array of ActivityStreams Objects id */ @@ -63,24 +86,34 @@ export function getOneApId(value: ApObject): string { /** * Get ActivityStreams Object id */ -export function getApId(value: string | IObject | [string | IObject]): string { - // eslint-disable-next-line no-param-reassign - value = fromTuple(value); +export function getApId(value: string | IObject | [string | IObject], sourceForLogs?: string): string { + const id = getNullableApId(value); - if (typeof value === 'string') return value; - if (typeof value.id === 'string') return value.id; - throw new IdentifiableError('ad2dc287-75c1-44c4-839d-3d2e64576675', `invalid AP object ${value}: missing id`); + if (id == null) { + const message = sourceForLogs + ? `invalid AP object ${value} (sent from ${sourceForLogs}): missing id` + : `invalid AP object ${value}: missing id`; + throw new IdentifiableError('ad2dc287-75c1-44c4-839d-3d2e64576675', message); + } + + return id; } /** * Get ActivityStreams Object id, or null if not present */ -export function getNullableApId(value: string | IObject | [string | IObject]): string | null { - // eslint-disable-next-line no-param-reassign - value = fromTuple(value); +export function getNullableApId(source: string | IObject | [string | IObject]): string | null { + const value: unknown = fromTuple(source); + + if (value != null) { + if (typeof value === 'string') { + return value; + } + if (typeof (value) === 'object' && 'id' in value && typeof (value.id) === 'string') { + return value.id; + } + } - if (typeof value === 'string') return value; - if (typeof value.id === 'string') return value.id; return null; } @@ -125,48 +158,46 @@ export interface IActivity extends IObject { }; } -export interface ICollection extends IObject { - type: 'Collection'; - totalItems: number; +export interface CollectionBase extends IObject { + totalItems?: number; first?: IObject | string; last?: IObject | string; current?: IObject | string; + partOf?: IObject | string; + next?: IObject | string; + prev?: IObject | string; + items?: ApObject; + orderedItems?: ApObject; +} + +export interface ICollection extends CollectionBase { + type: 'Collection'; + totalItems: number; items?: ApObject; + orderedItems?: undefined; } -export interface IOrderedCollection extends IObject { +export interface IOrderedCollection extends CollectionBase { type: 'OrderedCollection'; totalItems: number; - first?: IObject | string; - last?: IObject | string; - current?: IObject | string; + items?: undefined; orderedItems?: ApObject; } -export interface ICollectionPage extends IObject { +export interface ICollectionPage extends CollectionBase { type: 'CollectionPage'; - totalItems: number; - first?: IObject | string; - last?: IObject | string; - current?: IObject | string; - partOf?: IObject | string; - next?: IObject | string; - prev?: IObject | string; items?: ApObject; + orderedItems?: undefined; } -export interface IOrderedCollectionPage extends IObject { +export interface IOrderedCollectionPage extends CollectionBase { type: 'OrderedCollectionPage'; - totalItems: number; - first?: IObject | string; - last?: IObject | string; - current?: IObject | string; - partOf?: IObject | string; - next?: IObject | string; - prev?: IObject | string; + items?: undefined; orderedItems?: ApObject; } +export type AnyCollection = ICollection | IOrderedCollection | ICollectionPage | IOrderedCollectionPage; + export const validPost = ['Note', 'Question', 'Article', 'Audio', 'Document', 'Image', 'Page', 'Video', 'Event']; export const isPost = (object: IObject): object is IPost => { @@ -184,6 +215,7 @@ export interface IPost extends IObject { _misskey_content?: string; quoteUrl?: string; quoteUri?: string; + quote?: string; updated?: string; } @@ -255,6 +287,7 @@ export interface IActor extends IObject { enableRss?: boolean; listenbrainz?: string; backgroundUrl?: string; + attributionDomains?: string[]; } export const isCollection = (object: IObject): object is ICollection => @@ -269,7 +302,7 @@ export const isCollectionPage = (object: IObject): object is ICollectionPage => export const isOrderedCollectionPage = (object: IObject): object is IOrderedCollectionPage => getApType(object) === 'OrderedCollectionPage'; -export const isCollectionOrOrderedCollection = (object: IObject): object is ICollection | IOrderedCollection => +export const isCollectionOrOrderedCollection = (object: IObject): object is AnyCollection => isCollection(object) || isOrderedCollection(object) || isCollectionPage(object) || isOrderedCollectionPage(object); export interface IApPropertyValue extends IObject { @@ -285,9 +318,8 @@ export const isPropertyValue = (object: IObject): object is IApPropertyValue => 'value' in object && typeof object.value === 'string'; -export interface IApMention extends IObject { +export interface IApMention extends ILink { type: 'Mention'; - href: string; name: string; } diff --git a/packages/backend/src/core/chart/charts/federation.ts b/packages/backend/src/core/chart/charts/federation.ts index bf702884ca..4bbb5437cc 100644 --- a/packages/backend/src/core/chart/charts/federation.ts +++ b/packages/backend/src/core/chart/charts/federation.ts @@ -44,10 +44,7 @@ export default class FederationChart extends Chart<typeof schema> { // eslint-di } protected async tickMinor(): Promise<Partial<KVs<typeof schema>>> { - const suspendedInstancesQuery = this.instancesRepository.createQueryBuilder('instance') - .select('instance.host') - .where('instance.suspensionState != \'none\''); - + // TODO optimization: replace these with exists() const pubsubSubQuery = this.followingsRepository.createQueryBuilder('f') .select('f.followerHost') .where('f.followerHost IS NOT NULL'); @@ -64,22 +61,25 @@ export default class FederationChart extends Chart<typeof schema> { // eslint-di this.followingsRepository.createQueryBuilder('following') .select('COUNT(DISTINCT following.followeeHost)') .where('following.followeeHost IS NOT NULL') - .andWhere(this.meta.blockedHosts.length === 0 ? '1=1' : '(\'.\' || following.followeeHost) NOT ILIKE ALL(select \'%.\' || x from (select unnest("blockedHosts") as x from "meta") t)') - .andWhere(`following.followeeHost NOT IN (${ suspendedInstancesQuery.getQuery() })`) + .innerJoin('following.followeeInstance', 'followeeInstance') + .andWhere('followeeInstance.suspensionState = \'none\'') + .andWhere('followeeInstance.isBlocked = false') .getRawOne() .then(x => parseInt(x.count, 10)), this.followingsRepository.createQueryBuilder('following') .select('COUNT(DISTINCT following.followerHost)') .where('following.followerHost IS NOT NULL') - .andWhere(this.meta.blockedHosts.length === 0 ? '1=1' : '(\'.\' || following.followerHost) NOT ILIKE ALL(select \'%.\' || x from (select unnest("blockedHosts") as x from "meta") t)') - .andWhere(`following.followerHost NOT IN (${ suspendedInstancesQuery.getQuery() })`) + .innerJoin('following.followerInstance', 'followerInstance') + .andWhere('followerInstance.isBlocked = false') + .andWhere('followerInstance.suspensionState = \'none\'') .getRawOne() .then(x => parseInt(x.count, 10)), this.followingsRepository.createQueryBuilder('following') .select('COUNT(DISTINCT following.followeeHost)') .where('following.followeeHost IS NOT NULL') - .andWhere(this.meta.blockedHosts.length === 0 ? '1=1' : '(\'.\' || following.followeeHost) NOT ILIKE ALL(select \'%.\' || x from (select unnest("blockedHosts") as x from "meta") t)') - .andWhere(`following.followeeHost NOT IN (${ suspendedInstancesQuery.getQuery() })`) + .innerJoin('following.followeeInstance', 'followeeInstance') + .andWhere('followeeInstance.isBlocked = false') + .andWhere('followeeInstance.suspensionState = \'none\'') .andWhere(`following.followeeHost IN (${ pubsubSubQuery.getQuery() })`) .setParameters(pubsubSubQuery.getParameters()) .getRawOne() @@ -87,7 +87,7 @@ export default class FederationChart extends Chart<typeof schema> { // eslint-di this.instancesRepository.createQueryBuilder('instance') .select('COUNT(instance.id)') .where(`instance.host IN (${ subInstancesQuery.getQuery() })`) - .andWhere(this.meta.blockedHosts.length === 0 ? '1=1' : '(\'.\' || instance.host) NOT ILIKE ALL(select \'%.\' || x from (select unnest("blockedHosts") as x from "meta") t)') + .andWhere('instance.isBlocked = false') .andWhere('instance.suspensionState = \'none\'') .andWhere('instance.isNotResponding = false') .getRawOne() @@ -95,7 +95,7 @@ export default class FederationChart extends Chart<typeof schema> { // eslint-di this.instancesRepository.createQueryBuilder('instance') .select('COUNT(instance.id)') .where(`instance.host IN (${ pubInstancesQuery.getQuery() })`) - .andWhere(this.meta.blockedHosts.length === 0 ? '1=1' : '(\'.\' || instance.host) NOT ILIKE ALL(select \'%.\' || x from (select unnest("blockedHosts") as x from "meta") t)') + .andWhere('instance.isBlocked = false') .andWhere('instance.suspensionState = \'none\'') .andWhere('instance.isNotResponding = false') .getRawOne() diff --git a/packages/backend/src/core/chart/charts/per-user-following.ts b/packages/backend/src/core/chart/charts/per-user-following.ts index 588ac638de..8d75a30e9a 100644 --- a/packages/backend/src/core/chart/charts/per-user-following.ts +++ b/packages/backend/src/core/chart/charts/per-user-following.ts @@ -15,6 +15,7 @@ import Chart from '../core.js'; import { ChartLoggerService } from '../ChartLoggerService.js'; import { name, schema } from './entities/per-user-following.js'; import type { KVs } from '../core.js'; +import { CacheService } from '@/core/CacheService.js'; /** * ユーザーごとのフォローに関するチャート @@ -31,23 +32,25 @@ export default class PerUserFollowingChart extends Chart<typeof schema> { // esl private appLockService: AppLockService, private userEntityService: UserEntityService, private chartLoggerService: ChartLoggerService, + private readonly cacheService: CacheService, ) { super(db, (k) => appLockService.getChartInsertLock(k), chartLoggerService.logger, name, schema, true); } protected async tickMajor(group: string): Promise<Partial<KVs<typeof schema>>> { const [ - localFollowingsCount, - localFollowersCount, - remoteFollowingsCount, - remoteFollowersCount, + followees, + followers, ] = await Promise.all([ - this.followingsRepository.countBy({ followerId: group, followeeHost: IsNull() }), - this.followingsRepository.countBy({ followeeId: group, followerHost: IsNull() }), - this.followingsRepository.countBy({ followerId: group, followeeHost: Not(IsNull()) }), - this.followingsRepository.countBy({ followeeId: group, followerHost: Not(IsNull()) }), + this.cacheService.userFollowingsCache.fetch(group).then(fs => Array.from(fs.values())), + this.cacheService.userFollowersCache.fetch(group).then(fs => Array.from(fs.values())), ]); + const localFollowingsCount = followees.reduce((sum, f) => sum + (f.followeeHost == null ? 1 : 0), 0); + const localFollowersCount = followers.reduce((sum, f) => sum + (f.followerHost == null ? 1 : 0), 0); + const remoteFollowingsCount = followees.reduce((sum, f) => sum + (f.followeeHost == null ? 0 : 1), 0); + const remoteFollowersCount = followers.reduce((sum, f) => sum + (f.followerHost == null ? 0 : 1), 0); + return { 'local.followings.total': localFollowingsCount, 'local.followers.total': localFollowersCount, diff --git a/packages/backend/src/core/entities/AbuseUserReportEntityService.ts b/packages/backend/src/core/entities/AbuseUserReportEntityService.ts index 70ead890ab..c1d877aa12 100644 --- a/packages/backend/src/core/entities/AbuseUserReportEntityService.ts +++ b/packages/backend/src/core/entities/AbuseUserReportEntityService.ts @@ -5,13 +5,14 @@ import { Inject, Injectable } from '@nestjs/common'; import { DI } from '@/di-symbols.js'; -import type { AbuseUserReportsRepository } from '@/models/_.js'; +import type { AbuseUserReportsRepository, InstancesRepository, MiInstance, MiUser } from '@/models/_.js'; import { awaitAll } from '@/misc/prelude/await-all.js'; import type { MiAbuseUserReport } from '@/models/AbuseUserReport.js'; import { bindThis } from '@/decorators.js'; import { IdService } from '@/core/IdService.js'; import type { Packed } from '@/misc/json-schema.js'; import { UserEntityService } from './UserEntityService.js'; +import { InstanceEntityService } from './InstanceEntityService.js'; @Injectable() export class AbuseUserReportEntityService { @@ -19,6 +20,10 @@ export class AbuseUserReportEntityService { @Inject(DI.abuseUserReportsRepository) private abuseUserReportsRepository: AbuseUserReportsRepository, + @Inject(DI.instancesRepository) + private instancesRepository: InstancesRepository, + + private readonly instanceEntityService: InstanceEntityService, private userEntityService: UserEntityService, private idService: IdService, ) { @@ -30,11 +35,14 @@ export class AbuseUserReportEntityService { hint?: { packedReporter?: Packed<'UserDetailedNotMe'>, packedTargetUser?: Packed<'UserDetailedNotMe'>, + packedTargetInstance?: Packed<'FederationInstance'>, packedAssignee?: Packed<'UserDetailedNotMe'>, }, + me?: MiUser | null, ) { const report = typeof src === 'object' ? src : await this.abuseUserReportsRepository.findOneByOrFail({ id: src }); + // noinspection ES6MissingAwait return await awaitAll({ id: report.id, createdAt: this.idService.parse(report.id).date.toISOString(), @@ -43,13 +51,22 @@ export class AbuseUserReportEntityService { reporterId: report.reporterId, targetUserId: report.targetUserId, assigneeId: report.assigneeId, - reporter: hint?.packedReporter ?? this.userEntityService.pack(report.reporter ?? report.reporterId, null, { + reporter: hint?.packedReporter ?? this.userEntityService.pack(report.reporter ?? report.reporterId, me, { schema: 'UserDetailedNotMe', }), - targetUser: hint?.packedTargetUser ?? this.userEntityService.pack(report.targetUser ?? report.targetUserId, null, { + targetUser: hint?.packedTargetUser ?? this.userEntityService.pack(report.targetUser ?? report.targetUserId, me, { schema: 'UserDetailedNotMe', }), - assignee: report.assigneeId ? hint?.packedAssignee ?? this.userEntityService.pack(report.assignee ?? report.assigneeId, null, { + // return hint, or pack by relation, or fetch and pack by id, or null + targetInstance: hint?.packedTargetInstance ?? ( + report.targetUserInstance + ? this.instanceEntityService.pack(report.targetUserInstance, me) + : report.targetUserHost + ? this.instancesRepository.findOneBy({ host: report.targetUserHost }).then(instance => instance + ? this.instanceEntityService.pack(instance, me) + : null) + : null), + assignee: report.assigneeId ? hint?.packedAssignee ?? this.userEntityService.pack(report.assignee ?? report.assigneeId, me, { schema: 'UserDetailedNotMe', }) : null, forwarded: report.forwarded, @@ -61,21 +78,28 @@ export class AbuseUserReportEntityService { @bindThis public async packMany( reports: MiAbuseUserReport[], + me?: MiUser | null, ) { const _reporters = reports.map(({ reporter, reporterId }) => reporter ?? reporterId); const _targetUsers = reports.map(({ targetUser, targetUserId }) => targetUser ?? targetUserId); const _assignees = reports.map(({ assignee, assigneeId }) => assignee ?? assigneeId).filter(x => x != null); const _userMap = await this.userEntityService.packMany( [..._reporters, ..._targetUsers, ..._assignees], - null, + me, { schema: 'UserDetailedNotMe' }, ).then(users => new Map(users.map(u => [u.id, u]))); + const _targetInstances = reports + .map(({ targetUserInstance, targetUserHost }) => targetUserInstance ?? targetUserHost) + .filter((i): i is MiInstance | string => i != null); + const _instanceMap = await this.instanceEntityService.packMany(await this.instanceEntityService.fetchInstancesByHost(_targetInstances), me) + .then(instances => new Map(instances.map(i => [i.host, i]))); return Promise.all( reports.map(report => { const packedReporter = _userMap.get(report.reporterId); const packedTargetUser = _userMap.get(report.targetUserId); + const packedTargetInstance = report.targetUserHost ? _instanceMap.get(report.targetUserHost) : undefined; const packedAssignee = report.assigneeId != null ? _userMap.get(report.assigneeId) : undefined; - return this.pack(report, { packedReporter, packedTargetUser, packedAssignee }); + return this.pack(report, { packedReporter, packedTargetUser, packedAssignee, packedTargetInstance }, me); }), ); } diff --git a/packages/backend/src/core/entities/InstanceEntityService.ts b/packages/backend/src/core/entities/InstanceEntityService.ts index fcc9bed3bd..4ca4ff650b 100644 --- a/packages/backend/src/core/entities/InstanceEntityService.ts +++ b/packages/backend/src/core/entities/InstanceEntityService.ts @@ -4,6 +4,7 @@ */ import { Inject, Injectable } from '@nestjs/common'; +import { In } from 'typeorm'; import type { Packed } from '@/misc/json-schema.js'; import type { MiInstance } from '@/models/Instance.js'; import { bindThis } from '@/decorators.js'; @@ -11,7 +12,7 @@ import { UtilityService } from '@/core/UtilityService.js'; import { RoleService } from '@/core/RoleService.js'; import { MiUser } from '@/models/User.js'; import { DI } from '@/di-symbols.js'; -import { MiMeta } from '@/models/_.js'; +import type { InstancesRepository, MiMeta } from '@/models/_.js'; @Injectable() export class InstanceEntityService { @@ -19,6 +20,9 @@ export class InstanceEntityService { @Inject(DI.meta) private meta: MiMeta, + @Inject(DI.instancesRepository) + private readonly instancesRepository: InstancesRepository, + private roleService: RoleService, private utilityService: UtilityService, @@ -43,7 +47,7 @@ export class InstanceEntityService { isNotResponding: instance.isNotResponding, isSuspended: instance.suspensionState !== 'none', suspensionState: instance.suspensionState, - isBlocked: this.utilityService.isBlockedHost(this.meta.blockedHosts, instance.host), + isBlocked: instance.isBlocked, softwareName: instance.softwareName, softwareVersion: instance.softwareVersion, openRegistrations: instance.openRegistrations, @@ -51,8 +55,8 @@ export class InstanceEntityService { description: instance.description, maintainerName: instance.maintainerName, maintainerEmail: instance.maintainerEmail, - isSilenced: this.utilityService.isSilencedHost(this.meta.silencedHosts, instance.host), - isMediaSilenced: this.utilityService.isMediaSilencedHost(this.meta.mediaSilencedHosts, instance.host), + isSilenced: instance.isSilenced, + isMediaSilenced: instance.isMediaSilenced, iconUrl: instance.iconUrl, faviconUrl: instance.faviconUrl, themeColor: instance.themeColor, @@ -62,6 +66,7 @@ export class InstanceEntityService { rejectReports: instance.rejectReports, rejectQuotes: instance.rejectQuotes, moderationNote: iAmModerator ? instance.moderationNote : null, + isBubbled: this.utilityService.isBubbledHost(instance.host), }; } @@ -72,5 +77,28 @@ export class InstanceEntityService { ) { return Promise.all(instances.map(x => this.pack(x, me))); } + + @bindThis + public async fetchInstancesByHost(instances: (MiInstance | MiInstance['host'])[]): Promise<MiInstance[]> { + const result: MiInstance[] = []; + + const toFetch: string[] = []; + for (const instance of instances) { + if (typeof(instance) === 'string') { + toFetch.push(instance); + } else { + result.push(instance); + } + } + + if (toFetch.length > 0) { + const fetched = await this.instancesRepository.findBy({ + host: In(toFetch), + }); + result.push(...fetched); + } + + return result; + } } diff --git a/packages/backend/src/core/entities/NoteEntityService.ts b/packages/backend/src/core/entities/NoteEntityService.ts index 6ada5463a3..4248fde77f 100644 --- a/packages/backend/src/core/entities/NoteEntityService.ts +++ b/packages/backend/src/core/entities/NoteEntityService.ts @@ -11,12 +11,13 @@ import type { Packed } from '@/misc/json-schema.js'; import { awaitAll } from '@/misc/prelude/await-all.js'; import type { MiUser } from '@/models/User.js'; import type { MiNote } from '@/models/Note.js'; -import type { UsersRepository, NotesRepository, FollowingsRepository, PollsRepository, PollVotesRepository, NoteReactionsRepository, ChannelsRepository, MiMeta } from '@/models/_.js'; +import type { UsersRepository, NotesRepository, FollowingsRepository, PollsRepository, PollVotesRepository, NoteReactionsRepository, ChannelsRepository, MiMeta, MiPollVote, MiPoll, MiChannel, MiFollowing } from '@/models/_.js'; import { bindThis } from '@/decorators.js'; import { DebounceLoader } from '@/misc/loader.js'; import { IdService } from '@/core/IdService.js'; import { ReactionsBufferingService } from '@/core/ReactionsBufferingService.js'; import { isPackedPureRenote } from '@/misc/is-renote.js'; +import type { Config } from '@/config.js'; import type { OnModuleInit } from '@nestjs/common'; import type { CacheService } from '../CacheService.js'; import type { CustomEmojiService } from '../CustomEmojiService.js'; @@ -25,13 +26,13 @@ import type { UserEntityService } from './UserEntityService.js'; import type { DriveFileEntityService } from './DriveFileEntityService.js'; // is-renote.tsとよしなにリンク -function isPureRenote(note: MiNote): note is MiNote & { renoteId: MiNote['id']; renote: MiNote } { +function isPureRenote(note: MiNote): note is MiNote & { renoteId: MiNote['id'] } { return ( - note.renote != null && - note.reply == null && + note.renoteId != null && + note.replyId == null && note.text == null && note.cw == null && - (note.fileIds == null || note.fileIds.length === 0) && + note.fileIds.length === 0 && !note.hasPoll ); } @@ -92,6 +93,9 @@ export class NoteEntityService implements OnModuleInit { @Inject(DI.channelsRepository) private channelsRepository: ChannelsRepository, + @Inject(DI.config) + private readonly config: Config, + //private userEntityService: UserEntityService, //private driveFileEntityService: DriveFileEntityService, //private customEmojiService: CustomEmojiService, @@ -128,7 +132,10 @@ export class NoteEntityService implements OnModuleInit { } @bindThis - public async hideNote(packedNote: Packed<'Note'>, meId: MiUser['id'] | null): Promise<void> { + public async hideNote(packedNote: Packed<'Note'>, meId: MiUser['id'] | null, hint?: { + myFollowing?: ReadonlyMap<string, unknown>, + myBlockers?: ReadonlySet<string>, + }): Promise<void> { if (meId === packedNote.userId) return; // TODO: isVisibleForMe を使うようにしても良さそう(型違うけど) @@ -184,14 +191,9 @@ export class NoteEntityService implements OnModuleInit { } else if (packedNote.renote && (meId === packedNote.renote.userId)) { hide = false; } else { - // フォロワーかどうか - // TODO: 当関数呼び出しごとにクエリが走るのは重そうだからなんとかする - const isFollowing = await this.followingsRepository.exists({ - where: { - followeeId: packedNote.userId, - followerId: meId, - }, - }); + const isFollowing = hint?.myFollowing + ? hint.myFollowing.has(packedNote.userId) + : (await this.cacheService.userFollowingsCache.fetch(meId)).has(packedNote.userId); hide = !isFollowing; } @@ -207,7 +209,8 @@ export class NoteEntityService implements OnModuleInit { } if (!hide && meId && packedNote.userId !== meId) { - const isBlocked = (await this.cacheService.userBlockedCache.fetch(meId)).has(packedNote.userId); + const blockers = hint?.myBlockers ?? await this.cacheService.userBlockedCache.fetch(meId); + const isBlocked = blockers.has(packedNote.userId); if (isBlocked) hide = true; } @@ -231,8 +234,11 @@ export class NoteEntityService implements OnModuleInit { } @bindThis - private async populatePoll(note: MiNote, meId: MiUser['id'] | null) { - const poll = await this.pollsRepository.findOneByOrFail({ noteId: note.id }); + private async populatePoll(note: MiNote, meId: MiUser['id'] | null, hint?: { + poll?: MiPoll, + myVotes?: MiPollVote[], + }) { + const poll = hint?.poll ?? await this.pollsRepository.findOneByOrFail({ noteId: note.id }); const choices = poll.choices.map(c => ({ text: c, votes: poll.votes[poll.choices.indexOf(c)], @@ -241,7 +247,7 @@ export class NoteEntityService implements OnModuleInit { if (meId) { if (poll.multiple) { - const votes = await this.pollVotesRepository.findBy({ + const votes = hint?.myVotes ?? await this.pollVotesRepository.findBy({ userId: meId, noteId: note.id, }); @@ -251,7 +257,7 @@ export class NoteEntityService implements OnModuleInit { choices[myChoice].isVoted = true; } } else { - const vote = await this.pollVotesRepository.findOneBy({ + const vote = hint?.myVotes ? hint.myVotes[0] : await this.pollVotesRepository.findOneBy({ userId: meId, noteId: note.id, }); @@ -313,7 +319,12 @@ export class NoteEntityService implements OnModuleInit { } @bindThis - public async isVisibleForMe(note: MiNote, meId: MiUser['id'] | null): Promise<boolean> { + public async isVisibleForMe(note: MiNote, meId: MiUser['id'] | null, hint?: { + myFollowing?: ReadonlySet<string>, + myBlocking?: ReadonlySet<string>, + myBlockers?: ReadonlySet<string>, + me?: Pick<MiUser, 'host'> | null, + }): Promise<boolean> { // This code must always be synchronized with the checks in generateVisibilityQuery. // visibility が specified かつ自分が指定されていなかったら非表示 if (note.visibility === 'specified') { @@ -341,16 +352,16 @@ export class NoteEntityService implements OnModuleInit { return true; } else { // フォロワーかどうか - const [blocked, following, user] = await Promise.all([ - this.cacheService.userBlockingCache.fetch(meId).then((ids) => ids.has(note.userId)), - this.followingsRepository.count({ - where: { - followeeId: note.userId, - followerId: meId, - }, - take: 1, - }), - this.usersRepository.findOneByOrFail({ id: meId }), + const [blocked, following, userHost] = await Promise.all([ + hint?.myBlocking + ? hint.myBlocking.has(note.userId) + : this.cacheService.userBlockingCache.fetch(meId).then((ids) => ids.has(note.userId)), + hint?.myFollowing + ? hint.myFollowing.has(note.userId) + : this.cacheService.userFollowingsCache.fetch(meId).then(ids => ids.has(note.userId)), + hint?.me !== undefined + ? (hint.me?.host ?? null) + : this.cacheService.findUserById(meId).then(me => me.host), ]); if (blocked) return false; @@ -362,12 +373,13 @@ export class NoteEntityService implements OnModuleInit { in which case we can never know the following. Instead we have to assume that the users are following each other. */ - return following > 0 || (note.userHost != null && user.host != null); + return following || (note.userHost != null && userHost != null); } } if (meId != null) { - const isBlocked = (await this.cacheService.userBlockedCache.fetch(meId)).has(note.userId); + const blockers = hint?.myBlockers ?? await this.cacheService.userBlockedCache.fetch(meId); + const isBlocked = blockers.has(note.userId); if (isBlocked) return false; } @@ -404,6 +416,12 @@ export class NoteEntityService implements OnModuleInit { packedFiles: Map<MiNote['fileIds'][number], Packed<'DriveFile'> | null>; packedUsers: Map<MiUser['id'], Packed<'UserLite'>>; mentionHandles: Record<string, string | undefined>; + userFollowings: Map<string, Map<string, Omit<MiFollowing, 'isFollowerHibernated'>>>; + userBlockers: Map<string, Set<string>>; + polls: Map<string, MiPoll>; + pollVotes: Map<string, Map<string, MiPollVote[]>>; + channels: Map<string, MiChannel>; + notes: Map<string, MiNote>; }; }, ): Promise<Packed<'Note'>> { @@ -433,9 +451,7 @@ export class NoteEntityService implements OnModuleInit { } const channel = note.channelId - ? note.channel - ? note.channel - : await this.channelsRepository.findOneBy({ id: note.channelId }) + ? (opts._hint_?.channels.get(note.channelId) ?? note.channel ?? await this.channelsRepository.findOneBy({ id: note.channelId })) : null; const reactionEmojiNames = Object.keys(reactions) @@ -481,7 +497,10 @@ export class NoteEntityService implements OnModuleInit { mentionHandles: note.mentions.length > 0 ? this.getUserHandles(note.mentions, options?._hint_?.mentionHandles) : undefined, uri: note.uri ?? undefined, url: note.url ?? undefined, - poll: note.hasPoll ? this.populatePoll(note, meId) : undefined, + poll: note.hasPoll ? this.populatePoll(note, meId, { + poll: opts._hint_?.polls.get(note.id), + myVotes: opts._hint_?.pollVotes.get(note.id)?.get(note.userId), + }) : undefined, ...(meId && Object.keys(reactions).length > 0 ? { myReaction: this.populateMyReaction({ @@ -495,14 +514,14 @@ export class NoteEntityService implements OnModuleInit { clippedCount: note.clippedCount, processErrors: note.processErrors, - reply: note.replyId ? this.pack(note.reply ?? note.replyId, me, { + reply: note.replyId ? this.pack(note.reply ?? opts._hint_?.notes.get(note.replyId) ?? note.replyId, me, { detail: false, skipHide: opts.skipHide, withReactionAndUserPairCache: opts.withReactionAndUserPairCache, _hint_: options?._hint_, }) : undefined, - renote: note.renoteId ? this.pack(note.renote ?? note.renoteId, me, { + renote: note.renoteId ? this.pack(note.renote ?? opts._hint_?.notes.get(note.renoteId) ?? note.renoteId, me, { detail: true, skipHide: opts.skipHide, withReactionAndUserPairCache: opts.withReactionAndUserPairCache, @@ -514,7 +533,10 @@ export class NoteEntityService implements OnModuleInit { this.treatVisibility(packed); if (!opts.skipHide) { - await this.hideNote(packed, meId); + await this.hideNote(packed, meId, meId == null ? undefined : { + myFollowing: opts._hint_?.userFollowings.get(meId), + myBlockers: opts._hint_?.userBlockers.get(meId), + }); } return packed; @@ -531,79 +553,139 @@ export class NoteEntityService implements OnModuleInit { ) { if (notes.length === 0) return []; - const targetNotes: MiNote[] = []; + const targetNotesMap = new Map<string, MiNote>(); + const targetNotesToFetch : string[] = []; for (const note of notes) { if (isPureRenote(note)) { // we may need to fetch 'my reaction' for renote target. - targetNotes.push(note.renote); - if (note.renote.reply) { - // idem if the renote is also a reply. - targetNotes.push(note.renote.reply); + if (note.renote) { + targetNotesMap.set(note.renote.id, note.renote); + if (note.renote.reply) { + // idem if the renote is also a reply. + targetNotesMap.set(note.renote.reply.id, note.renote.reply); + } + } else if (options?.detail) { + targetNotesToFetch.push(note.renoteId); } } else { if (note.reply) { // idem for OP of a regular reply. - targetNotes.push(note.reply); + targetNotesMap.set(note.reply.id, note.reply); + } else if (note.replyId && options?.detail) { + targetNotesToFetch.push(note.replyId); } - targetNotes.push(note); + targetNotesMap.set(note.id, note); } } - const bufferedReactions = this.meta.enableReactionsBuffering ? await this.reactionsBufferingService.getMany([...getAppearNoteIds(notes)]) : null; + // Don't fetch notes that were added by ID and then found inline in another note. + for (let i = targetNotesToFetch.length - 1; i >= 0; i--) { + if (targetNotesMap.has(targetNotesToFetch[i])) { + targetNotesToFetch.splice(i, 1); + } + } - const meId = me ? me.id : null; - const myReactionsMap = new Map<MiNote['id'], string | null>(); - if (meId) { - const idsNeedFetchMyReaction = new Set<MiNote['id']>(); + // Populate any relations that weren't included in the source + if (targetNotesToFetch.length > 0) { + const newNotes = await this.notesRepository.find({ + where: { + id: In(targetNotesToFetch), + }, + relations: { + user: { + userProfile: true, + }, + reply: { + user: { + userProfile: true, + }, + }, + renote: { + user: { + userProfile: true, + }, + reply: { + user: { + userProfile: true, + }, + }, + }, + channel: true, + }, + }); - for (const note of targetNotes) { - const reactionsCount = Object.values(this.reactionsBufferingService.mergeReactions(note.reactions, bufferedReactions?.get(note.id)?.deltas ?? {})).reduce((a, b) => a + b, 0); - if (reactionsCount === 0) { - myReactionsMap.set(note.id, null); - } else if (reactionsCount <= note.reactionAndUserPairCache.length + (bufferedReactions?.get(note.id)?.pairs.length ?? 0)) { - const pairInBuffer = bufferedReactions?.get(note.id)?.pairs.find(p => p[0] === meId); - if (pairInBuffer) { - myReactionsMap.set(note.id, pairInBuffer[1]); - } else { - const pair = note.reactionAndUserPairCache.find(p => p.startsWith(meId)); - myReactionsMap.set(note.id, pair ? pair.split('/')[1] : null); - } - } else { - idsNeedFetchMyReaction.add(note.id); - } + for (const note of newNotes) { + targetNotesMap.set(note.id, note); } + } - const myReactions = idsNeedFetchMyReaction.size > 0 ? await this.noteReactionsRepository.findBy({ - userId: meId, - noteId: In(Array.from(idsNeedFetchMyReaction)), - }) : []; + const targetNotes = Array.from(targetNotesMap.values()); + const noteIds = Array.from(targetNotesMap.keys()); - for (const id of idsNeedFetchMyReaction) { - myReactionsMap.set(id, myReactions.find(reaction => reaction.noteId === id)?.reaction ?? null); + const usersMap = new Map<string, MiUser | string>(); + const allUsers = notes.flatMap(note => [ + note.user ?? note.userId, + note.reply?.user ?? note.replyUserId, + note.renote?.user ?? note.renoteUserId, + ]); + + for (const user of allUsers) { + if (!user) continue; + + if (typeof(user) === 'object') { + // ID -> Entity + usersMap.set(user.id, user); + } else if (!usersMap.has(user)) { + // ID -> ID + usersMap.set(user, user); } } - await this.customEmojiService.prefetchEmojis(this.aggregateNoteEmojis(notes)); - // TODO: 本当は renote とか reply がないのに renoteId とか replyId があったらここで解決しておく - const fileIds = notes.map(n => [n.fileIds, n.renote?.fileIds, n.reply?.fileIds]).flat(2).filter(x => x != null); - const packedFiles = fileIds.length > 0 ? await this.driveFileEntityService.packManyByIdsMap(fileIds) : new Map(); - const users = [ - ...notes.map(({ user, userId }) => user ?? userId), - ...notes.map(({ replyUserId }) => replyUserId).filter(x => x != null), - ...notes.map(({ renoteUserId }) => renoteUserId).filter(x => x != null), - ]; - const packedUsers = await this.userEntityService.packMany(users, me) - .then(users => new Map(users.map(u => [u.id, u]))); + const users = Array.from(usersMap.values()); + const userIds = Array.from(usersMap.keys()); - // Recursively add all mentioned users from all notes + replies + renotes - const allMentionedUsers = targetNotes.reduce((users, note) => { - for (const user of note.mentions) { - users.add(user); - } - return users; - }, new Set<string>()); - const mentionHandles = await this.getUserHandles(Array.from(allMentionedUsers)); + const fileIds = new Set(targetNotes.flatMap(n => n.fileIds)); + const mentionedUsers = new Set(targetNotes.flatMap(note => note.mentions)); + + const [{ bufferedReactions, myReactionsMap }, packedFiles, packedUsers, mentionHandles, userFollowings, userBlockers, polls, pollVotes, channels] = await Promise.all([ + // bufferedReactions & myReactionsMap + this.getReactions(targetNotes, me), + // packedFiles + this.driveFileEntityService.packManyByIdsMap(Array.from(fileIds)), + // packedUsers + this.userEntityService.packMany(users, me) + .then(users => new Map(users.map(u => [u.id, u]))), + // mentionHandles + this.getUserHandles(Array.from(mentionedUsers)), + // userFollowings + this.cacheService.userFollowingsCache.fetchMany(userIds).then(fs => new Map(fs)), + // userBlockers + this.cacheService.userBlockedCache.fetchMany(userIds).then(bs => new Map(bs)), + // polls + this.pollsRepository.findBy({ noteId: In(noteIds) }) + .then(polls => new Map(polls.map(p => [p.noteId, p]))), + // pollVotes + this.pollVotesRepository.findBy({ noteId: In(noteIds), userId: In(userIds) }) + .then(votes => votes.reduce((noteMap, vote) => { + let userMap = noteMap.get(vote.noteId); + if (!userMap) { + userMap = new Map<string, MiPollVote[]>(); + noteMap.set(vote.noteId, userMap); + } + let voteList = userMap.get(vote.userId); + if (!voteList) { + voteList = []; + userMap.set(vote.userId, voteList); + } + voteList.push(vote); + return noteMap; + }, new Map<string, Map<string, MiPollVote[]>>)), + // channels + this.getChannels(targetNotes), + // (not returned) + this.customEmojiService.prefetchEmojis(this.aggregateNoteEmojis(notes)), + ]); return await Promise.all(notes.map(n => this.pack(n, me, { ...options, @@ -613,6 +695,12 @@ export class NoteEntityService implements OnModuleInit { packedFiles, packedUsers, mentionHandles, + userFollowings, + userBlockers, + polls, + pollVotes, + channels, + notes: new Map(targetNotes.map(n => [n.id, n])), }, }))); } @@ -680,4 +768,71 @@ export class NoteEntityService implements OnModuleInit { return map; }, {} as Record<string, string | undefined>); } + + private async getChannels(notes: MiNote[]): Promise<Map<string, MiChannel>> { + const channels = new Map<string, MiChannel>(); + const channelsToFetch = new Set<string>(); + + for (const note of notes) { + if (note.channel) { + channels.set(note.channel.id, note.channel); + } else if (note.channelId) { + channelsToFetch.add(note.channelId); + } + } + + if (channelsToFetch.size > 0) { + const newChannels = await this.channelsRepository.findBy({ + id: In(Array.from(channelsToFetch)), + }); + for (const channel of newChannels) { + channels.set(channel.id, channel); + } + } + + return channels; + } + + private async getReactions(notes: MiNote[], me: { id: string } | null | undefined) { + const bufferedReactions = this.meta.enableReactionsBuffering ? await this.reactionsBufferingService.getMany([...getAppearNoteIds(notes)]) : null; + + const meId = me ? me.id : null; + const myReactionsMap = new Map<MiNote['id'], string | null>(); + if (meId) { + const idsNeedFetchMyReaction = new Set<MiNote['id']>(); + + for (const note of notes) { + const reactionsCount = Object.values(this.reactionsBufferingService.mergeReactions(note.reactions, bufferedReactions?.get(note.id)?.deltas ?? {})).reduce((a, b) => a + b, 0); + if (reactionsCount === 0) { + myReactionsMap.set(note.id, null); + } else if (reactionsCount <= note.reactionAndUserPairCache.length + (bufferedReactions?.get(note.id)?.pairs.length ?? 0)) { + const pairInBuffer = bufferedReactions?.get(note.id)?.pairs.find(p => p[0] === meId); + if (pairInBuffer) { + myReactionsMap.set(note.id, pairInBuffer[1]); + } else { + const pair = note.reactionAndUserPairCache.find(p => p.startsWith(meId)); + myReactionsMap.set(note.id, pair ? pair.split('/')[1] : null); + } + } else { + idsNeedFetchMyReaction.add(note.id); + } + } + + const myReactions = idsNeedFetchMyReaction.size > 0 ? await this.noteReactionsRepository.findBy({ + userId: meId, + noteId: In(Array.from(idsNeedFetchMyReaction)), + }) : []; + + for (const id of idsNeedFetchMyReaction) { + myReactionsMap.set(id, myReactions.find(reaction => reaction.noteId === id)?.reaction ?? null); + } + } + + return { bufferedReactions, myReactionsMap }; + } + + @bindThis + public genLocalNoteUri(noteId: string): string { + return `${this.config.url}/notes/${noteId}`; + } } diff --git a/packages/backend/src/core/entities/UserEntityService.ts b/packages/backend/src/core/entities/UserEntityService.ts index 56506a5fa4..638eaac16f 100644 --- a/packages/backend/src/core/entities/UserEntityService.ts +++ b/packages/backend/src/core/entities/UserEntityService.ts @@ -30,6 +30,7 @@ import type { FollowingsRepository, FollowRequestsRepository, MiFollowing, + MiInstance, MiMeta, MiUserNotePining, MiUserProfile, @@ -42,7 +43,7 @@ import type { UsersRepository, } from '@/models/_.js'; import { bindThis } from '@/decorators.js'; -import { RoleService } from '@/core/RoleService.js'; +import { RolePolicies, RoleService } from '@/core/RoleService.js'; import { ApPersonService } from '@/core/activitypub/models/ApPersonService.js'; import { FederatedInstanceService } from '@/core/FederatedInstanceService.js'; import { IdService } from '@/core/IdService.js'; @@ -52,6 +53,7 @@ import { AvatarDecorationService } from '@/core/AvatarDecorationService.js'; import { ChatService } from '@/core/ChatService.js'; import { isSystemAccount } from '@/misc/is-system-account.js'; import { DriveFileEntityService } from '@/core/entities/DriveFileEntityService.js'; +import type { CacheService } from '@/core/CacheService.js'; import type { OnModuleInit } from '@nestjs/common'; import type { NoteEntityService } from './NoteEntityService.js'; import type { PageEntityService } from './PageEntityService.js'; @@ -77,7 +79,7 @@ function isRemoteUser(user: MiUser | { host: MiUser['host'] }): boolean { export type UserRelation = { id: MiUser['id'] - following: MiFollowing | null, + following: Omit<MiFollowing, 'isFollowerHibernated'> | null, isFollowing: boolean isFollowed: boolean hasPendingFollowRequestFromYou: boolean @@ -103,6 +105,7 @@ export class UserEntityService implements OnModuleInit { private idService: IdService; private avatarDecorationService: AvatarDecorationService; private chatService: ChatService; + private cacheService: CacheService; constructor( private moduleRef: ModuleRef, @@ -163,6 +166,7 @@ export class UserEntityService implements OnModuleInit { this.idService = this.moduleRef.get('IdService'); this.avatarDecorationService = this.moduleRef.get('AvatarDecorationService'); this.chatService = this.moduleRef.get('ChatService'); + this.cacheService = this.moduleRef.get('CacheService'); } //#region Validators @@ -193,16 +197,8 @@ export class UserEntityService implements OnModuleInit { memo, mutedInstances, ] = await Promise.all([ - this.followingsRepository.findOneBy({ - followerId: me, - followeeId: target, - }), - this.followingsRepository.exists({ - where: { - followerId: target, - followeeId: me, - }, - }), + this.cacheService.userFollowingsCache.fetch(me).then(f => f.get(target) ?? null), + this.cacheService.userFollowingsCache.fetch(target).then(f => f.has(me)), this.followRequestsRepository.exists({ where: { followerId: me, @@ -215,45 +211,22 @@ export class UserEntityService implements OnModuleInit { followeeId: me, }, }), - this.blockingsRepository.exists({ - where: { - blockerId: me, - blockeeId: target, - }, - }), - this.blockingsRepository.exists({ - where: { - blockerId: target, - blockeeId: me, - }, - }), - this.mutingsRepository.exists({ - where: { - muterId: me, - muteeId: target, - }, - }), - this.renoteMutingsRepository.exists({ - where: { - muterId: me, - muteeId: target, - }, - }), - this.usersRepository.createQueryBuilder('u') - .select('u.host') - .where({ id: target }) - .getRawOne<{ u_host: string }>() - .then(it => it?.u_host ?? null), + this.cacheService.userBlockingCache.fetch(me) + .then(blockees => blockees.has(target)), + this.cacheService.userBlockedCache.fetch(me) + .then(blockers => blockers.has(target)), + this.cacheService.userMutingsCache.fetch(me) + .then(mutings => mutings.has(target)), + this.cacheService.renoteMutingsCache.fetch(me) + .then(mutings => mutings.has(target)), + this.cacheService.findUserById(target).then(u => u.host), this.userMemosRepository.createQueryBuilder('m') .select('m.memo') .where({ userId: me, targetUserId: target }) .getRawOne<{ m_memo: string | null }>() .then(it => it?.m_memo ?? null), - this.userProfilesRepository.createQueryBuilder('p') - .select('p.mutedInstances') - .where({ userId: me }) - .getRawOne<{ p_mutedInstances: string[] }>() - .then(it => it?.p_mutedInstances ?? []), + this.cacheService.userProfileCache.fetch(me) + .then(profile => profile.mutedInstances), ]); const isInstanceMuted = !!host && mutedInstances.includes(host); @@ -277,8 +250,8 @@ export class UserEntityService implements OnModuleInit { @bindThis public async getRelations(me: MiUser['id'], targets: MiUser['id'][]): Promise<Map<MiUser['id'], UserRelation>> { const [ - followers, - followees, + myFollowing, + myFollowers, followersRequests, followeesRequests, blockers, @@ -289,13 +262,8 @@ export class UserEntityService implements OnModuleInit { memos, mutedInstances, ] = await Promise.all([ - this.followingsRepository.findBy({ followerId: me }) - .then(f => new Map(f.map(it => [it.followeeId, it]))), - this.followingsRepository.createQueryBuilder('f') - .select('f.followerId') - .where('f.followeeId = :me', { me }) - .getRawMany<{ f_followerId: string }>() - .then(it => it.map(it => it.f_followerId)), + this.cacheService.userFollowingsCache.fetch(me), + this.cacheService.userFollowersCache.fetch(me), this.followRequestsRepository.createQueryBuilder('f') .select('f.followeeId') .where('f.followerId = :me', { me }) @@ -306,34 +274,18 @@ export class UserEntityService implements OnModuleInit { .where('f.followeeId = :me', { me }) .getRawMany<{ f_followerId: string }>() .then(it => it.map(it => it.f_followerId)), - this.blockingsRepository.createQueryBuilder('b') - .select('b.blockeeId') - .where('b.blockerId = :me', { me }) - .getRawMany<{ b_blockeeId: string }>() - .then(it => it.map(it => it.b_blockeeId)), - this.blockingsRepository.createQueryBuilder('b') - .select('b.blockerId') - .where('b.blockeeId = :me', { me }) - .getRawMany<{ b_blockerId: string }>() - .then(it => it.map(it => it.b_blockerId)), - this.mutingsRepository.createQueryBuilder('m') - .select('m.muteeId') - .where('m.muterId = :me', { me }) - .getRawMany<{ m_muteeId: string }>() - .then(it => it.map(it => it.m_muteeId)), - this.renoteMutingsRepository.createQueryBuilder('m') - .select('m.muteeId') - .where('m.muterId = :me', { me }) - .getRawMany<{ m_muteeId: string }>() - .then(it => it.map(it => it.m_muteeId)), - this.usersRepository.createQueryBuilder('u') - .select(['u.id', 'u.host']) - .where({ id: In(targets) } ) - .getRawMany<{ m_id: string, m_host: string }>() - .then(it => it.reduce((map, it) => { - map[it.m_id] = it.m_host; - return map; - }, {} as Record<string, string>)), + this.cacheService.userBlockedCache.fetch(me), + this.cacheService.userBlockingCache.fetch(me), + this.cacheService.userMutingsCache.fetch(me), + this.cacheService.renoteMutingsCache.fetch(me), + this.cacheService.getUsers(targets) + .then(users => { + const record: Record<string, string | null> = {}; + for (const [id, user] of users) { + record[id] = user.host; + } + return record; + }), this.userMemosRepository.createQueryBuilder('m') .select(['m.targetUserId', 'm.memo']) .where({ userId: me, targetUserId: In(targets) }) @@ -342,16 +294,13 @@ export class UserEntityService implements OnModuleInit { map[it.m_targetUserId] = it.m_memo; return map; }, {} as Record<string, string | null>)), - this.userProfilesRepository.createQueryBuilder('p') - .select('p.mutedInstances') - .where({ userId: me }) - .getRawOne<{ p_mutedInstances: string[] }>() - .then(it => it?.p_mutedInstances ?? []), + this.cacheService.userProfileCache.fetch(me) + .then(p => p.mutedInstances), ]); return new Map( targets.map(target => { - const following = followers.get(target) ?? null; + const following = myFollowing.get(target) ?? null; return [ target, @@ -359,14 +308,14 @@ export class UserEntityService implements OnModuleInit { id: target, following: following, isFollowing: following != null, - isFollowed: followees.includes(target), + isFollowed: myFollowers.has(target), hasPendingFollowRequestFromYou: followersRequests.includes(target), hasPendingFollowRequestToYou: followeesRequests.includes(target), - isBlocking: blockers.includes(target), - isBlocked: blockees.includes(target), - isMuted: muters.includes(target), - isRenoteMuted: renoteMuters.includes(target), - isInstanceMuted: mutedInstances.includes(hosts[target]), + isBlocking: blockees.has(target), + isBlocked: blockers.has(target), + isMuted: muters.has(target), + isRenoteMuted: renoteMuters.has(target), + isInstanceMuted: hosts[target] != null && mutedInstances.includes(hosts[target]), memo: memos[target] ?? null, }, ]; @@ -391,6 +340,7 @@ export class UserEntityService implements OnModuleInit { return false; // TODO } + // TODO optimization: make redis calls in MULTI @bindThis public async getNotificationsInfo(userId: MiUser['id']): Promise<{ hasUnread: boolean; @@ -424,16 +374,14 @@ export class UserEntityService implements OnModuleInit { @bindThis public async getHasPendingReceivedFollowRequest(userId: MiUser['id']): Promise<boolean> { - const count = await this.followRequestsRepository.countBy({ + return await this.followRequestsRepository.existsBy({ followeeId: userId, }); - - return count > 0; } @bindThis public async getHasPendingSentFollowRequest(userId: MiUser['id']): Promise<boolean> { - return this.followRequestsRepository.existsBy({ + return await this.followRequestsRepository.existsBy({ followerId: userId, }); } @@ -480,6 +428,10 @@ export class UserEntityService implements OnModuleInit { userRelations?: Map<MiUser['id'], UserRelation>, userMemos?: Map<MiUser['id'], string | null>, pinNotes?: Map<MiUser['id'], MiUserNotePining[]>, + iAmModerator?: boolean, + userIdsByUri?: Map<string, string>, + instances?: Map<string, MiInstance | null>, + securityKeyCounts?: Map<string, number>, }, ): Promise<Packed<S>> { const opts = Object.assign({ @@ -487,7 +439,10 @@ export class UserEntityService implements OnModuleInit { includeSecrets: false, }, options); - const user = typeof src === 'object' ? src : await this.usersRepository.findOneByOrFail({ id: src }); + const user = typeof src === 'object' ? src : await this.usersRepository.findOneOrFail({ + where: { id: src }, + relations: { userProfile: true }, + }); // migration if (user.avatarId != null && user.avatarUrl === null) { @@ -518,10 +473,10 @@ export class UserEntityService implements OnModuleInit { const isDetailed = opts.schema !== 'UserLite'; const meId = me ? me.id : null; const isMe = meId === user.id; - const iAmModerator = me ? await this.roleService.isModerator(me as MiUser) : false; + const iAmModerator = opts.iAmModerator ?? (me ? await this.roleService.isModerator(me as MiUser) : false); const profile = isDetailed - ? (opts.userProfile ?? await this.userProfilesRepository.findOneByOrFail({ userId: user.id })) + ? (opts.userProfile ?? user.userProfile ?? await this.userProfilesRepository.findOneByOrFail({ userId: user.id })) : null; let relation: UserRelation | null = null; @@ -556,7 +511,7 @@ export class UserEntityService implements OnModuleInit { } } - const mastoapi = !isDetailed ? opts.userProfile ?? await this.userProfilesRepository.findOneByOrFail({ userId: user.id }) : null; + const mastoapi = !isDetailed ? opts.userProfile ?? user.userProfile ?? await this.userProfilesRepository.findOneByOrFail({ userId: user.id }) : null; const followingCount = profile == null ? null : (profile.followingVisibility === 'public') || isMe || iAmModerator ? user.followingCount : @@ -579,6 +534,9 @@ export class UserEntityService implements OnModuleInit { const checkHost = user.host == null ? this.config.host : user.host; const notificationsInfo = isMe && isDetailed ? await this.getNotificationsInfo(user.id) : null; + let fetchPoliciesPromise: Promise<RolePolicies> | null = null; + const fetchPolicies = () => fetchPoliciesPromise ??= this.roleService.getUserPolicies(user); + const packed = { id: user.id, name: user.name, @@ -603,19 +561,21 @@ export class UserEntityService implements OnModuleInit { enableRss: user.enableRss, mandatoryCW: user.mandatoryCW, rejectQuotes: user.rejectQuotes, - isSilenced: user.isSilenced || this.roleService.getUserPolicies(user.id).then(r => !r.canPublicNote), + attributionDomains: user.attributionDomains, + isSilenced: user.isSilenced || fetchPolicies().then(r => !r.canPublicNote), speakAsCat: user.speakAsCat ?? false, approved: user.approved, requireSigninToViewContents: user.requireSigninToViewContents === false ? undefined : true, makeNotesFollowersOnlyBefore: user.makeNotesFollowersOnlyBefore ?? undefined, makeNotesHiddenBefore: user.makeNotesHiddenBefore ?? undefined, - instance: user.host ? this.federatedInstanceService.federatedInstanceCache.fetch(user.host).then(instance => instance ? { + instance: user.host ? Promise.resolve(opts.instances?.has(user.host) ? opts.instances.get(user.host) : this.federatedInstanceService.fetch(user.host)).then(instance => instance ? { name: instance.name, softwareName: instance.softwareName, softwareVersion: instance.softwareVersion, iconUrl: instance.iconUrl, faviconUrl: instance.faviconUrl, themeColor: instance.themeColor, + isSilenced: instance.isSilenced, } : undefined) : undefined, followersCount: followersCount ?? 0, followingCount: followingCount ?? 0, @@ -623,7 +583,7 @@ export class UserEntityService implements OnModuleInit { emojis: this.customEmojiService.populateEmojis(user.emojis, checkHost), onlineStatus: this.getOnlineStatus(user), // パフォーマンス上の理由でローカルユーザーのみ - badgeRoles: user.host == null ? this.roleService.getUserBadgeRoles(user.id).then((rs) => rs + badgeRoles: user.host == null ? this.roleService.getUserBadgeRoles(user).then((rs) => rs .filter((r) => r.isPublic || iAmModerator) .sort((a, b) => b.displayOrder - a.displayOrder) .map((r) => ({ @@ -636,9 +596,9 @@ export class UserEntityService implements OnModuleInit { ...(isDetailed ? { url: profile!.url, uri: user.uri, - movedTo: user.movedToUri ? this.apPersonService.resolvePerson(user.movedToUri).then(user => user.id).catch(() => null) : null, + movedTo: user.movedToUri ? Promise.resolve(opts.userIdsByUri?.get(user.movedToUri) ?? this.apPersonService.resolvePerson(user.movedToUri).then(user => user.id).catch(() => null)) : null, alsoKnownAs: user.alsoKnownAs - ? Promise.all(user.alsoKnownAs.map(uri => this.apPersonService.fetchPerson(uri).then(user => user?.id).catch(() => null))) + ? Promise.all(user.alsoKnownAs.map(uri => Promise.resolve(opts.userIdsByUri?.get(uri) ?? this.apPersonService.fetchPerson(uri).then(user => user?.id).catch(() => null)))) .then(xs => xs.length === 0 ? null : xs.filter(x => x != null)) : null, updatedAt: user.updatedAt ? user.updatedAt.toISOString() : null, @@ -665,8 +625,8 @@ export class UserEntityService implements OnModuleInit { followersVisibility: profile!.followersVisibility, followingVisibility: profile!.followingVisibility, chatScope: user.chatScope, - canChat: this.roleService.getUserPolicies(user.id).then(r => r.chatAvailability === 'available'), - roles: this.roleService.getUserRoles(user.id).then(roles => roles.filter(role => role.isPublic).sort((a, b) => b.displayOrder - a.displayOrder).map(role => ({ + canChat: fetchPolicies().then(r => r.chatAvailability === 'available'), + roles: this.roleService.getUserRoles(user).then(roles => roles.filter(role => role.isPublic).sort((a, b) => b.displayOrder - a.displayOrder).map(role => ({ id: role.id, name: role.name, color: role.color, @@ -684,7 +644,7 @@ export class UserEntityService implements OnModuleInit { twoFactorEnabled: profile!.twoFactorEnabled, usePasswordLessLogin: profile!.usePasswordLessLogin, securityKeys: profile!.twoFactorEnabled - ? this.userSecurityKeysRepository.countBy({ userId: user.id }).then(result => result >= 1) + ? Promise.resolve(opts.securityKeyCounts?.get(user.id) ?? this.userSecurityKeysRepository.countBy({ userId: user.id })).then(result => result >= 1) : false, } : {}), @@ -728,7 +688,7 @@ export class UserEntityService implements OnModuleInit { emailNotificationTypes: profile!.emailNotificationTypes, achievements: profile!.achievements, loggedInDays: profile!.loggedInDates.length, - policies: this.roleService.getUserPolicies(user.id), + policies: fetchPolicies(), defaultCW: profile!.defaultCW, defaultCWPriority: profile!.defaultCWPriority, allowUnsignedFetch: user.allowUnsignedFetch, @@ -778,57 +738,103 @@ export class UserEntityService implements OnModuleInit { includeSecrets?: boolean, }, ): Promise<Packed<S>[]> { + if (users.length === 0) return []; + // -- IDのみの要素を補完して完全なエンティティ一覧を作る const _users = users.filter((user): user is MiUser => typeof user !== 'string'); if (_users.length !== users.length) { _users.push( - ...await this.usersRepository.findBy({ - id: In(users.filter((user): user is string => typeof user === 'string')), + ...await this.usersRepository.find({ + where: { + id: In(users.filter((user): user is string => typeof user === 'string')), + }, + relations: { + userProfile: true, + }, }), ); } const _userIds = _users.map(u => u.id); - // -- 実行者の有無や指定スキーマの種別によって要否が異なる値群を取得 - - let profilesMap: Map<MiUser['id'], MiUserProfile> = new Map(); - let userRelations: Map<MiUser['id'], UserRelation> = new Map(); - let userMemos: Map<MiUser['id'], string | null> = new Map(); - let pinNotes: Map<MiUser['id'], MiUserNotePining[]> = new Map(); + const iAmModerator = await this.roleService.isModerator(me as MiUser); + const meId = me ? me.id : null; + const isDetailed = options && options.schema !== 'UserLite'; + const isDetailedAndMod = isDetailed && iAmModerator; - if (options?.schema !== 'UserLite') { - profilesMap = await this.userProfilesRepository.findBy({ userId: In(_userIds) }) - .then(profiles => new Map(profiles.map(p => [p.userId, p]))); + const userUris = new Set(_users + .flatMap(user => [user.uri, user.movedToUri]) + .filter((uri): uri is string => uri != null)); - const meId = me ? me.id : null; - if (meId) { - userMemos = await this.userMemosRepository.findBy({ userId: meId }) - .then(memos => new Map(memos.map(memo => [memo.targetUserId, memo.memo]))); + const userHosts = new Set(_users + .map(user => user.host) + .filter((host): host is string => host != null)); - if (_userIds.length > 0) { - userRelations = await this.getRelations(meId, _userIds); - pinNotes = await this.userNotePiningsRepository.createQueryBuilder('pin') - .where('pin.userId IN (:...userIds)', { userIds: _userIds }) - .innerJoinAndSelect('pin.note', 'note') - .getMany() - .then(pinsNotes => { - const map = new Map<MiUser['id'], MiUserNotePining[]>(); - for (const note of pinsNotes) { - const notes = map.get(note.userId) ?? []; - notes.push(note); - map.set(note.userId, notes); - } - for (const [, notes] of map.entries()) { - // pack側ではDESCで取得しているので、それに合わせて降順に並び替えておく - notes.sort((a, b) => b.id.localeCompare(a.id)); - } - return map; - }); - } + const _profilesFromUsers: [string, MiUserProfile][] = []; + const _profilesToFetch: string[] = []; + for (const user of _users) { + if (user.userProfile) { + _profilesFromUsers.push([user.id, user.userProfile]); + } else { + _profilesToFetch.push(user.id); } } + // -- 実行者の有無や指定スキーマの種別によって要否が異なる値群を取得 + + const [profilesMap, userMemos, userRelations, pinNotes, userIdsByUri, instances, securityKeyCounts] = await Promise.all([ + // profilesMap + this.cacheService.userProfileCache.fetchMany(_profilesToFetch).then(profiles => new Map(profiles.concat(_profilesFromUsers))), + // userMemos + isDetailed && meId ? this.userMemosRepository.findBy({ userId: meId }) + .then(memos => new Map(memos.map(memo => [memo.targetUserId, memo.memo]))) : new Map(), + // userRelations + isDetailed && meId ? this.getRelations(meId, _userIds) : new Map(), + // pinNotes + isDetailed ? this.userNotePiningsRepository.createQueryBuilder('pin') + .where('pin.userId IN (:...userIds)', { userIds: _userIds }) + .innerJoinAndSelect('pin.note', 'note') + .getMany() + .then(pinsNotes => { + const map = new Map<MiUser['id'], MiUserNotePining[]>(); + for (const note of pinsNotes) { + const notes = map.get(note.userId) ?? []; + notes.push(note); + map.set(note.userId, notes); + } + for (const [, notes] of map.entries()) { + // pack側ではDESCで取得しているので、それに合わせて降順に並び替えておく + notes.sort((a, b) => b.id.localeCompare(a.id)); + } + return map; + }) : new Map(), + // userIdsByUrl + isDetailed ? this.usersRepository.createQueryBuilder('user') + .select([ + 'user.id', + 'user.uri', + ]) + .where({ + uri: In(Array.from(userUris)), + }) + .getRawMany<{ user_uri: string, user_id: string }>() + .then(users => new Map(users.map(u => [u.user_uri, u.user_id]))) : new Map(), + // instances + Promise.all(Array.from(userHosts).map(async host => [host, await this.federatedInstanceService.fetch(host)] as const)) + .then(hosts => new Map(hosts)), + // securityKeyCounts + isDetailedAndMod ? this.userSecurityKeysRepository.createQueryBuilder('key') + .select('key.userId', 'userId') + .addSelect('count(key.id)', 'userCount') + .where({ + userId: In(_userIds), + }) + .groupBy('key.userId') + .getRawMany<{ userId: string, userCount: number }>() + .then(counts => new Map(counts.map(c => [c.userId, c.userCount]))) + : undefined, // .pack will fetch the keys for the requesting user if it's in the _userIds + ]); + return Promise.all( _users.map(u => this.pack( u, @@ -839,6 +845,10 @@ export class UserEntityService implements OnModuleInit { userRelations: userRelations, userMemos: userMemos, pinNotes: pinNotes, + iAmModerator, + userIdsByUri, + instances, + securityKeyCounts, }, )), ); diff --git a/packages/backend/src/env.ts b/packages/backend/src/env.ts index ba44cfa2e6..9a50eb8561 100644 --- a/packages/backend/src/env.ts +++ b/packages/backend/src/env.ts @@ -11,6 +11,7 @@ const envOption = { verbose: false, withLogTime: false, quiet: false, + hideWorkerId: false, }; for (const key of Object.keys(envOption) as (keyof typeof envOption)[]) { diff --git a/packages/backend/src/logger.ts b/packages/backend/src/logger.ts index b3735200eb..4bf45fc76b 100644 --- a/packages/backend/src/logger.ts +++ b/packages/backend/src/logger.ts @@ -23,6 +23,14 @@ export type DataElement = DataObject | Error | string | null; // https://stackoverflow.com/questions/61148466/typescript-type-that-matches-any-object-but-not-arrays export type DataObject = Record<string, unknown> | (object & { length?: never; }); +const levelFuncs = { + error: 'error', + warning: 'warn', + success: 'info', + info: 'log', + debug: 'debug', +} as const satisfies Record<Level, keyof typeof console>; + // eslint-disable-next-line import/no-default-export export default class Logger { private context: Context; @@ -71,7 +79,9 @@ export default class Logger { level === 'info' ? message : null; - let log = `${l} ${worker}\t[${contexts.join(' ')}]\t${m}`; + let log = envOption.hideWorkerId + ? `${l}\t[${contexts.join(' ')}]\t\t${m}` + : `${l} ${worker}\t[${contexts.join(' ')}]\t\t${m}`; if (envOption.withLogTime) log = chalk.gray(time) + ' ' + log; const args: unknown[] = [important ? chalk.bold(log) : log]; @@ -84,7 +94,7 @@ export default class Logger { } else if (data != null) { args.push(data); } - console.log(...args); + console[levelFuncs[level]](...args); } @bindThis diff --git a/packages/backend/src/misc/FileWriterStream.ts b/packages/backend/src/misc/FileWriterStream.ts index 27c67cb5df..a61d949ef4 100644 --- a/packages/backend/src/misc/FileWriterStream.ts +++ b/packages/backend/src/misc/FileWriterStream.ts @@ -21,7 +21,7 @@ export class FileWriterStream extends WritableStream<Uint8Array> { write: async (chunk, controller) => { if (file === null) { controller.error(); - throw new Error(); + throw new Error('file is null'); } await file.write(chunk); diff --git a/packages/backend/src/misc/QuantumKVCache.ts b/packages/backend/src/misc/QuantumKVCache.ts new file mode 100644 index 0000000000..b96937d6f2 --- /dev/null +++ b/packages/backend/src/misc/QuantumKVCache.ts @@ -0,0 +1,385 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +import { InternalEventService } from '@/core/InternalEventService.js'; +import { bindThis } from '@/decorators.js'; +import { InternalEventTypes } from '@/core/GlobalEventService.js'; +import { MemoryKVCache } from '@/misc/cache.js'; + +export interface QuantumKVOpts<T> { + /** + * Memory cache lifetime in milliseconds. + */ + lifetime: number; + + /** + * Callback to fetch the value for a key that wasn't found in the cache. + * May be synchronous or async. + */ + fetcher: (key: string, cache: QuantumKVCache<T>) => T | Promise<T>; + + /** + * Optional callback to fetch the value for multiple keys that weren't found in the cache. + * May be synchronous or async. + * If not provided, then the implementation will fall back on repeated calls to fetcher(). + */ + bulkFetcher?: (keys: string[], cache: QuantumKVCache<T>) => Iterable<[key: string, value: T]> | Promise<Iterable<[key: string, value: T]>>; + + /** + * Optional callback when one or more values are changed (created, updated, or deleted) in the cache, either locally or elsewhere in the cluster. + * This is called *after* the cache state is updated. + * Implementations may be synchronous or async. + */ + onChanged?: (keys: string[], cache: QuantumKVCache<T>) => void | Promise<void>; +} + +/** + * QuantumKVCache is a lifetime-bounded memory cache (like MemoryKVCache) with automatic cross-cluster synchronization via Redis. + * All nodes in the cluster are guaranteed to have a *subset* view of the current accurate state, though individual processes may have different items in their local cache. + * This ensures that a call to get() will never return stale data. + */ +export class QuantumKVCache<T> implements Iterable<[key: string, value: T]> { + private readonly memoryCache: MemoryKVCache<T>; + + public readonly fetcher: QuantumKVOpts<T>['fetcher']; + public readonly bulkFetcher: QuantumKVOpts<T>['bulkFetcher']; + public readonly onChanged: QuantumKVOpts<T>['onChanged']; + + /** + * @param internalEventService Service bus to synchronize events. + * @param name Unique name of the cache - must be the same in all processes. + * @param opts Cache options + */ + constructor( + private readonly internalEventService: InternalEventService, + private readonly name: string, + opts: QuantumKVOpts<T>, + ) { + this.memoryCache = new MemoryKVCache(opts.lifetime); + this.fetcher = opts.fetcher; + this.bulkFetcher = opts.bulkFetcher; + this.onChanged = opts.onChanged; + + this.internalEventService.on('quantumCacheUpdated', this.onQuantumCacheUpdated, { + // Ignore our own events, otherwise we'll immediately erase any set value. + ignoreLocal: true, + }); + } + + /** + * The number of items currently in memory. + * This applies to the local subset view, not the cross-cluster cache state. + */ + public get size() { + return this.memoryCache.size; + } + + /** + * Iterates all [key, value] pairs in memory. + * This applies to the local subset view, not the cross-cluster cache state. + */ + @bindThis + public *entries(): Generator<[key: string, value: T]> { + for (const entry of this.memoryCache.entries) { + yield [entry[0], entry[1].value]; + } + } + + /** + * Iterates all keys in memory. + * This applies to the local subset view, not the cross-cluster cache state. + */ + @bindThis + public *keys() { + for (const entry of this.memoryCache.entries) { + yield entry[0]; + } + } + + /** + * Iterates all values pairs in memory. + * This applies to the local subset view, not the cross-cluster cache state. + */ + @bindThis + public *values() { + for (const entry of this.memoryCache.entries) { + yield entry[1].value; + } + } + + /** + * Creates or updates a value in the cache, and erases any stale caches across the cluster. + * Fires an onSet event after the cache has been updated in all processes. + * Skips if the value is unchanged. + */ + @bindThis + public async set(key: string, value: T): Promise<void> { + if (this.memoryCache.get(key) === value) { + return; + } + + this.memoryCache.set(key, value); + + await this.internalEventService.emit('quantumCacheUpdated', { name: this.name, keys: [key] }); + + if (this.onChanged) { + await this.onChanged([key], this); + } + } + + /** + * Creates or updates multiple value in the cache, and erases any stale caches across the cluster. + * Fires an onSet for each changed item event after the cache has been updated in all processes. + * Skips if all values are unchanged. + */ + @bindThis + public async setMany(items: Iterable<[key: string, value: T]>): Promise<void> { + const changedKeys: string[] = []; + + for (const item of items) { + if (this.memoryCache.get(item[0]) !== item[1]) { + changedKeys.push(item[0]); + this.memoryCache.set(item[0], item[1]); + } + } + + if (changedKeys.length > 0) { + await this.internalEventService.emit('quantumCacheUpdated', { name: this.name, keys: changedKeys }); + + if (this.onChanged) { + await this.onChanged(changedKeys, this); + } + } + } + + /** + * Adds a value to the local memory cache without notifying other process. + * Neither a Redis event nor onSet callback will be fired, as the value has not actually changed. + * This should only be used when the value is known to be current, like after fetching from the database. + */ + @bindThis + public add(key: string, value: T): void { + this.memoryCache.set(key, value); + } + + /** + * Adds multiple values to the local memory cache without notifying other process. + * Neither a Redis event nor onSet callback will be fired, as the value has not actually changed. + * This should only be used when the value is known to be current, like after fetching from the database. + */ + @bindThis + public addMany(items: Iterable<[key: string, value: T]>): void { + for (const [key, value] of items) { + this.memoryCache.set(key, value); + } + } + + /** + * Gets a value from the local memory cache, or returns undefined if not found. + * Returns cached data only - does not make any fetches. + */ + @bindThis + public get(key: string): T | undefined { + return this.memoryCache.get(key); + } + + /** + * Gets multiple values from the local memory cache; returning undefined for any missing keys. + * Returns cached data only - does not make any fetches. + */ + @bindThis + public getMany(keys: Iterable<string>): [key: string, value: T | undefined][] { + const results: [key: string, value: T | undefined][] = []; + for (const key of keys) { + results.push([key, this.get(key)]); + } + return results; + } + + /** + * Gets or fetches a value from the cache. + * Fires an onSet event, but does not emit an update event to other processes. + */ + @bindThis + public async fetch(key: string): Promise<T> { + let value = this.memoryCache.get(key); + if (value === undefined) { + value = await this.fetcher(key, this); + this.memoryCache.set(key, value); + + if (this.onChanged) { + await this.onChanged([key], this); + } + } + return value; + } + + /** + * Gets or fetches multiple values from the cache. + * Fires onSet events, but does not emit any update events to other processes. + */ + @bindThis + public async fetchMany(keys: Iterable<string>): Promise<[key: string, value: T][]> { + const results: [key: string, value: T][] = []; + const toFetch: string[] = []; + + // Spliterate into cached results / uncached keys. + for (const key of keys) { + const fromCache = this.get(key); + if (fromCache) { + results.push([key, fromCache]); + } else { + toFetch.push(key); + } + } + + // Fetch any uncached keys + if (toFetch.length > 0) { + const fetched = await this.bulkFetch(toFetch); + + // Add to cache and return set + this.addMany(fetched); + results.push(...fetched); + + // Emit event + if (this.onChanged) { + await this.onChanged(toFetch, this); + } + } + + return results; + } + + /** + * Returns true is a key exists in memory. + * This applies to the local subset view, not the cross-cluster cache state. + */ + @bindThis + public has(key: string): boolean { + return this.memoryCache.get(key) !== undefined; + } + + /** + * Deletes a value from the cache, and erases any stale caches across the cluster. + * Fires an onDelete event after the cache has been updated in all processes. + */ + @bindThis + public async delete(key: string): Promise<void> { + this.memoryCache.delete(key); + + await this.internalEventService.emit('quantumCacheUpdated', { name: this.name, keys: [key] }); + + if (this.onChanged) { + await this.onChanged([key], this); + } + } + /** + * Deletes multiple values from the cache, and erases any stale caches across the cluster. + * Fires an onDelete event for each key after the cache has been updated in all processes. + * Skips if the input is empty. + */ + @bindThis + public async deleteMany(keys: Iterable<string>): Promise<void> { + const deleted: string[] = []; + + for (const key of keys) { + this.memoryCache.delete(key); + deleted.push(key); + } + + if (deleted.length === 0) { + return; + } + + await this.internalEventService.emit('quantumCacheUpdated', { name: this.name, keys: deleted }); + + if (this.onChanged) { + await this.onChanged(deleted, this); + } + } + + /** + * Refreshes the value of a key from the fetcher, and erases any stale caches across the cluster. + * Fires an onSet event after the cache has been updated in all processes. + */ + @bindThis + public async refresh(key: string): Promise<T> { + const value = await this.fetcher(key, this); + await this.set(key, value); + return value; + } + + @bindThis + public async refreshMany(keys: Iterable<string>): Promise<[key: string, value: T][]> { + const values = await this.bulkFetch(keys); + await this.setMany(values); + return values; + } + + /** + * Erases all entries from the local memory cache. + * Does not send any events or update other processes. + */ + @bindThis + public clear() { + this.memoryCache.clear(); + } + + /** + * Removes expired cache entries from the local view. + * Does not send any events or update other processes. + */ + @bindThis + public gc() { + this.memoryCache.gc(); + } + + /** + * Erases all data and disconnects from the cluster. + * This *must* be called when shutting down to prevent memory leaks! + */ + @bindThis + public dispose() { + this.internalEventService.off('quantumCacheUpdated', this.onQuantumCacheUpdated); + + this.memoryCache.dispose(); + } + + @bindThis + private async bulkFetch(keys: Iterable<string>): Promise<[key: string, value: T][]> { + if (this.bulkFetcher) { + const results = await this.bulkFetcher(Array.from(keys), this); + return Array.from(results); + } + + const results: [key: string, value: T][] = []; + for (const key of keys) { + const value = await this.fetcher(key, this); + results.push([key, value]); + } + return results; + } + + @bindThis + private async onQuantumCacheUpdated(data: InternalEventTypes['quantumCacheUpdated']): Promise<void> { + if (data.name === this.name) { + for (const key of data.keys) { + this.memoryCache.delete(key); + } + + if (this.onChanged) { + await this.onChanged(data.keys, this); + } + } + } + + /** + * Iterates all [key, value] pairs in memory. + * This applies to the local subset view, not the cross-cluster cache state. + */ + [Symbol.iterator](): Iterator<[key: string, value: T]> { + return this.entries(); + } +} diff --git a/packages/backend/src/misc/cache.ts b/packages/backend/src/misc/cache.ts index 48b8f43678..666e684c1c 100644 --- a/packages/backend/src/misc/cache.ts +++ b/packages/backend/src/misc/cache.ts @@ -9,9 +9,9 @@ import { bindThis } from '@/decorators.js'; export class RedisKVCache<T> { private readonly lifetime: number; private readonly memoryCache: MemoryKVCache<T>; - private readonly fetcher: (key: string) => Promise<T>; - private readonly toRedisConverter: (value: T) => string; - private readonly fromRedisConverter: (value: string) => T | undefined; + public readonly fetcher: (key: string) => Promise<T>; + public readonly toRedisConverter: (value: T) => string; + public readonly fromRedisConverter: (value: string) => T | undefined; constructor( private redisClient: Redis.Redis, @@ -100,6 +100,11 @@ export class RedisKVCache<T> { } @bindThis + public clear() { + this.memoryCache.clear(); + } + + @bindThis public gc() { this.memoryCache.gc(); } @@ -113,9 +118,9 @@ export class RedisKVCache<T> { export class RedisSingleCache<T> { private readonly lifetime: number; private readonly memoryCache: MemorySingleCache<T>; - private readonly fetcher: () => Promise<T>; - private readonly toRedisConverter: (value: T) => string; - private readonly fromRedisConverter: (value: string) => T | undefined; + public readonly fetcher: () => Promise<T>; + public readonly toRedisConverter: (value: T) => string; + public readonly fromRedisConverter: (value: string) => T | undefined; constructor( private redisClient: Redis.Redis, @@ -123,16 +128,17 @@ export class RedisSingleCache<T> { opts: { lifetime: number; memoryCacheLifetime: number; - fetcher: RedisSingleCache<T>['fetcher']; - toRedisConverter: RedisSingleCache<T>['toRedisConverter']; - fromRedisConverter: RedisSingleCache<T>['fromRedisConverter']; + fetcher?: RedisSingleCache<T>['fetcher']; + toRedisConverter?: RedisSingleCache<T>['toRedisConverter']; + fromRedisConverter?: RedisSingleCache<T>['fromRedisConverter']; }, ) { this.lifetime = opts.lifetime; this.memoryCache = new MemorySingleCache(opts.memoryCacheLifetime); - this.fetcher = opts.fetcher; - this.toRedisConverter = opts.toRedisConverter; - this.fromRedisConverter = opts.fromRedisConverter; + + this.fetcher = opts.fetcher ?? (() => { throw new Error('fetch not supported - use get/set directly'); }); + this.toRedisConverter = opts.toRedisConverter ?? ((value) => JSON.stringify(value)); + this.fromRedisConverter = opts.fromRedisConverter ?? ((value) => JSON.parse(value)); } @bindThis @@ -237,6 +243,16 @@ export class MemoryKVCache<T> { return cached.value; } + public has(key: string): boolean { + const cached = this.cache.get(key); + if (cached == null) return false; + if ((Date.now() - cached.date) > this.lifetime) { + this.cache.delete(key); + return false; + } + return true; + } + @bindThis public delete(key: string): void { this.cache.delete(key); @@ -308,11 +324,24 @@ export class MemoryKVCache<T> { } } + /** + * Removes all entries from the cache, but does not dispose it. + */ + @bindThis + public clear(): void { + this.cache.clear(); + } + @bindThis public dispose(): void { + this.clear(); clearInterval(this.gcIntervalHandle); } + public get size() { + return this.cache.size; + } + public get entries() { return this.cache.entries(); } diff --git a/packages/backend/src/misc/diff-arrays.ts b/packages/backend/src/misc/diff-arrays.ts new file mode 100644 index 0000000000..b50ca1d4f7 --- /dev/null +++ b/packages/backend/src/misc/diff-arrays.ts @@ -0,0 +1,102 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +export interface DiffResult<T> { + added: T[]; + removed: T[]; +} + +/** + * Calculates the difference between two snapshots of data. + * Null, undefined, and empty arrays are supported, and duplicate values are ignored. + * Result sets are de-duplicated, and will be empty if no data was added or removed (respectively). + * The inputs are treated as un-ordered, so a re-ordering of the same data will NOT be considered a change. + * @param dataBefore Array containing data before the change + * @param dataAfter Array containing data after the change + */ +export function diffArrays<T>(dataBefore: T[] | null | undefined, dataAfter: T[] | null | undefined): DiffResult<T> { + const before = dataBefore ? new Set(dataBefore) : null; + const after = dataAfter ? new Set(dataAfter) : null; + + // data before AND after => changed + if (before?.size && after?.size) { + const added: T[] = []; + const removed: T[] = []; + + for (const host of before) { + // before and NOT after => removed + // delete operation removes duplicates to speed up the "after" loop + if (!after.delete(host)) { + removed.push(host); + } + } + + for (const host of after) { + // after and NOT before => added + if (!before.has(host)) { + added.push(host); + } + } + + return { added, removed }; + } + + // data ONLY before => all removed + if (before?.size) { + return { added: [], removed: Array.from(before) }; + } + + // data ONLY after => all added + if (after?.size) { + return { added: Array.from(after), removed: [] }; + } + + // data NEITHER before nor after => no change + return { added: [], removed: [] }; +} + +/** + * Checks for any difference between two snapshots of data. + * Null, undefined, and empty arrays are supported, and duplicate values are ignored. + * The inputs are treated as un-ordered, so a re-ordering of the same data will NOT be considered a change. + * @param dataBefore Array containing data before the change + * @param dataAfter Array containing data after the change + */ +export function diffArraysSimple<T>(dataBefore: T[] | null | undefined, dataAfter: T[] | null | undefined): boolean { + const before = dataBefore ? new Set(dataBefore) : null; + const after = dataAfter ? new Set(dataAfter) : null; + + if (before?.size && after?.size) { + // different size => changed + if (before.size !== after.size) return true; + + // removed => changed + for (const host of before) { + // delete operation removes duplicates to speed up the "after" loop + if (!after.delete(host)) { + return true; + } + } + + // added => changed + for (const host of after) { + if (!before.has(host)) { + return true; + } + } + + // identical values => no change + return false; + } + + // before and NOT after => change + if (before?.size) return true; + + // after and NOT before => change + if (after?.size) return true; + + // NEITHER before nor after => no change + return false; +} diff --git a/packages/backend/src/misc/extract-custom-emojis-from-mfm.ts b/packages/backend/src/misc/extract-custom-emojis-from-mfm.ts index 36a9b8e1f4..73ae9abb54 100644 --- a/packages/backend/src/misc/extract-custom-emojis-from-mfm.ts +++ b/packages/backend/src/misc/extract-custom-emojis-from-mfm.ts @@ -3,7 +3,7 @@ * SPDX-License-Identifier: AGPL-3.0-only */ -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; import { unique } from '@/misc/prelude/array.js'; export function extractCustomEmojisFromMfm(nodes: mfm.MfmNode[]): string[] { diff --git a/packages/backend/src/misc/extract-hashtags.ts b/packages/backend/src/misc/extract-hashtags.ts index ed7606d995..d3d245d414 100644 --- a/packages/backend/src/misc/extract-hashtags.ts +++ b/packages/backend/src/misc/extract-hashtags.ts @@ -3,7 +3,7 @@ * SPDX-License-Identifier: AGPL-3.0-only */ -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; import { unique } from '@/misc/prelude/array.js'; export function extractHashtags(nodes: mfm.MfmNode[]): string[] { diff --git a/packages/backend/src/misc/extract-mentions.ts b/packages/backend/src/misc/extract-mentions.ts index bb21c32ffb..2ec9349718 100644 --- a/packages/backend/src/misc/extract-mentions.ts +++ b/packages/backend/src/misc/extract-mentions.ts @@ -5,7 +5,7 @@ // test is located in test/extract-mentions -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; export function extractMentions(nodes: mfm.MfmNode[]): mfm.MfmMention['props'][] { // TODO: 重複を削除 diff --git a/packages/backend/src/misc/fastify-reply-error.ts b/packages/backend/src/misc/fastify-reply-error.ts index e6c4e78d2f..03109e8b96 100644 --- a/packages/backend/src/misc/fastify-reply-error.ts +++ b/packages/backend/src/misc/fastify-reply-error.ts @@ -8,8 +8,8 @@ export class FastifyReplyError extends Error { public message: string; public statusCode: number; - constructor(statusCode: number, message: string) { - super(message); + constructor(statusCode: number, message: string, cause?: unknown) { + super(message, cause ? { cause } : undefined); this.message = message; this.statusCode = statusCode; } diff --git a/packages/backend/src/misc/id/aid.ts b/packages/backend/src/misc/id/aid.ts index c0e8478db5..f0eba2d99c 100644 --- a/packages/backend/src/misc/id/aid.ts +++ b/packages/backend/src/misc/id/aid.ts @@ -8,6 +8,7 @@ import * as crypto from 'node:crypto'; import { parseBigInt36 } from '@/misc/bigint.js'; +import { IdentifiableError } from '../identifiable-error.js'; export const aidRegExp = /^[0-9a-z]{10}$/; @@ -26,7 +27,7 @@ function getNoise(): string { } export function genAid(t: number): string { - if (isNaN(t)) throw new Error('Failed to create AID: Invalid Date'); + if (isNaN(t)) throw new IdentifiableError('6b73b7d5-9d2b-48b4-821c-ef955efe80ad', 'Failed to create AID: Invalid Date'); counter++; return getTime(t) + getNoise(); } diff --git a/packages/backend/src/misc/id/aidx.ts b/packages/backend/src/misc/id/aidx.ts index 006673a6d0..d2bb566e35 100644 --- a/packages/backend/src/misc/id/aidx.ts +++ b/packages/backend/src/misc/id/aidx.ts @@ -10,6 +10,7 @@ import { customAlphabet } from 'nanoid'; import { parseBigInt36 } from '@/misc/bigint.js'; +import { IdentifiableError } from '../identifiable-error.js'; export const aidxRegExp = /^[0-9a-z]{16}$/; @@ -34,7 +35,7 @@ function getNoise(): string { } export function genAidx(t: number): string { - if (isNaN(t)) throw new Error('Failed to create AIDX: Invalid Date'); + if (isNaN(t)) throw new IdentifiableError('6b73b7d5-9d2b-48b4-821c-ef955efe80ad', 'Failed to create AIDX: Invalid Date'); counter++; return getTime(t) + nodeId + getNoise(); } diff --git a/packages/backend/src/misc/identifiable-error.ts b/packages/backend/src/misc/identifiable-error.ts index f5c3fcd6cb..56e13f2622 100644 --- a/packages/backend/src/misc/identifiable-error.ts +++ b/packages/backend/src/misc/identifiable-error.ts @@ -15,8 +15,8 @@ export class IdentifiableError extends Error { */ public readonly isRetryable: boolean; - constructor(id: string, message?: string, isRetryable = false) { - super(message); + constructor(id: string, message?: string, isRetryable = false, cause?: unknown) { + super(message, cause ? { cause } : undefined); this.message = message ?? ''; this.id = id; this.isRetryable = isRetryable; diff --git a/packages/backend/src/misc/is-retryable-error.ts b/packages/backend/src/misc/is-retryable-error.ts index 9bb8700c7a..63b561b280 100644 --- a/packages/backend/src/misc/is-retryable-error.ts +++ b/packages/backend/src/misc/is-retryable-error.ts @@ -3,20 +3,34 @@ * SPDX-License-Identifier: AGPL-3.0-only */ -import { AbortError } from 'node-fetch'; +import { AbortError, FetchError } from 'node-fetch'; import { UnrecoverableError } from 'bullmq'; import { StatusError } from '@/misc/status-error.js'; import { IdentifiableError } from '@/misc/identifiable-error.js'; +import { CaptchaError, captchaErrorCodes } from '@/core/CaptchaService.js'; +import { FastifyReplyError } from '@/misc/fastify-reply-error.js'; +import { ConflictError } from '@/server/SkRateLimiterService.js'; /** * Returns false if the provided value represents a "permanent" error that cannot be retried. * Returns true if the error is retryable, unknown (as all errors are retryable by default), or not an error object. */ export function isRetryableError(e: unknown): boolean { + if (e instanceof AggregateError) return e.errors.every(inner => isRetryableError(inner)); if (e instanceof StatusError) return e.isRetryable; if (e instanceof IdentifiableError) return e.isRetryable; + if (e instanceof CaptchaError) { + if (e.code === captchaErrorCodes.verificationFailed) return false; + if (e.code === captchaErrorCodes.invalidParameters) return false; + if (e.code === captchaErrorCodes.invalidProvider) return false; + return true; + } + if (e instanceof FastifyReplyError) return false; + if (e instanceof ConflictError) return true; if (e instanceof UnrecoverableError) return false; if (e instanceof AbortError) return true; + if (e instanceof FetchError) return true; + if (e instanceof SyntaxError) return false; if (e instanceof Error) return e.name === 'AbortError'; return true; } diff --git a/packages/backend/src/misc/render-full-error.ts b/packages/backend/src/misc/render-full-error.ts new file mode 100644 index 0000000000..5f0a09bba9 --- /dev/null +++ b/packages/backend/src/misc/render-full-error.ts @@ -0,0 +1,60 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +import * as Bull from 'bullmq'; +import { AbortError, FetchError } from 'node-fetch'; +import { StatusError } from '@/misc/status-error.js'; +import { IdentifiableError } from '@/misc/identifiable-error.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; +import { CaptchaError, captchaErrorCodes } from '@/core/CaptchaService.js'; + +export function renderFullError(e?: unknown): unknown { + if (e === undefined) return 'undefined'; + if (e === null) return 'null'; + + if (e instanceof Error) { + if (isSimpleError(e)) { + return renderInlineError(e); + } + + const data: ErrorData = {}; + if (e.stack) data.stack = e.stack; + if (e.message) data.message = e.message; + if (e.name) data.name = e.name; + + // mix "cause" and "errors" + if (e instanceof AggregateError && e.errors.length > 0) { + const causes = e.errors.map(inner => renderFullError(inner)); + if (e.cause) { + causes.push(renderFullError(e.cause)); + } + data.cause = causes; + } else if (e.cause) { + data.cause = renderFullError(e.cause); + } + + return data; + } + + return e; +} + +function isSimpleError(e: Error): boolean { + if (e instanceof Bull.UnrecoverableError) return true; + if (e instanceof AbortError || e.name === 'AbortError') return true; + if (e instanceof FetchError || e.name === 'FetchError') return true; + if (e instanceof StatusError) return true; + if (e instanceof IdentifiableError) return true; + if (e instanceof FetchError) return true; + if (e instanceof CaptchaError && e.code !== captchaErrorCodes.unknown) return true; + return false; +} + +interface ErrorData { + stack?: Error['stack']; + message?: Error['message']; + name?: Error['name']; + cause?: Error['cause'] | Error['cause'][]; +} diff --git a/packages/backend/src/misc/render-inline-error.ts b/packages/backend/src/misc/render-inline-error.ts new file mode 100644 index 0000000000..07f9f3068e --- /dev/null +++ b/packages/backend/src/misc/render-inline-error.ts @@ -0,0 +1,75 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +import { IdentifiableError } from '@/misc/identifiable-error.js'; +import { StatusError } from '@/misc/status-error.js'; +import { CaptchaError } from '@/core/CaptchaService.js'; + +export function renderInlineError(err: unknown): string { + const parts: string[] = []; + renderTo(err, parts); + return parts.join(''); +} + +function renderTo(err: unknown, parts: string[]): void { + parts.push(printError(err)); + + if (err instanceof AggregateError) { + for (let i = 0; i < err.errors.length; i++) { + parts.push(` [${i + 1}/${err.errors.length}]: `); + renderTo(err.errors[i], parts); + } + } + + if (err instanceof Error) { + if (err.cause) { + parts.push(' [caused by]: '); + renderTo(err.cause, parts); + // const cause = renderInlineError(err.cause); + // parts.push(' [caused by]: ', cause); + } + } +} + +function printError(err: unknown): string { + if (err === undefined) return 'undefined'; + if (err === null) return 'null'; + + if (err instanceof IdentifiableError) { + if (err.message) { + return `${err.name} ${err.id}: ${err.message}`; + } else { + return `${err.name} ${err.id}`; + } + } + + if (err instanceof StatusError) { + if (err.message) { + return `${err.name} ${err.statusCode}: ${err.message}`; + } else if (err.statusMessage) { + return `${err.name} ${err.statusCode}: ${err.statusMessage}`; + } else { + return `${err.name} ${err.statusCode}`; + } + } + + if (err instanceof CaptchaError) { + if (err.code.description) { + return `${err.name} ${err.code.description}: ${err.message}`; + } else { + return `${err.name}: ${err.message}`; + } + } + + if (err instanceof Error) { + if (err.message) { + return `${err.name}: ${err.message}`; + } else { + return err.name; + } + } + + return String(err); +} diff --git a/packages/backend/src/misc/status-error.ts b/packages/backend/src/misc/status-error.ts index c3533db607..4fd3bfcafb 100644 --- a/packages/backend/src/misc/status-error.ts +++ b/packages/backend/src/misc/status-error.ts @@ -9,8 +9,8 @@ export class StatusError extends Error { public isClientError: boolean; public isRetryable: boolean; - constructor(message: string, statusCode: number, statusMessage?: string) { - super(message); + constructor(message: string, statusCode: number, statusMessage?: string, cause?: unknown) { + super(message, cause ? { cause } : undefined); this.name = 'StatusError'; this.statusCode = statusCode; this.statusMessage = statusMessage; diff --git a/packages/backend/src/misc/truncate.ts b/packages/backend/src/misc/truncate.ts index 1c8a274609..a313ab7854 100644 --- a/packages/backend/src/misc/truncate.ts +++ b/packages/backend/src/misc/truncate.ts @@ -3,14 +3,12 @@ * SPDX-License-Identifier: AGPL-3.0-only */ -import { substring } from 'stringz'; - export function truncate(input: string, size: number): string; export function truncate(input: string | undefined, size: number): string | undefined; export function truncate(input: string | undefined, size: number): string | undefined { if (!input) { return input; } else { - return substring(input, 0, size); + return input.slice(0, size); } } diff --git a/packages/backend/src/misc/verify-field-link.ts b/packages/backend/src/misc/verify-field-link.ts index 62542eaaa0..f9fc352806 100644 --- a/packages/backend/src/misc/verify-field-link.ts +++ b/packages/backend/src/misc/verify-field-link.ts @@ -3,7 +3,7 @@ * SPDX-License-Identifier: AGPL-3.0-only */ -import { load as cheerio } from 'cheerio'; +import { load as cheerio } from 'cheerio/slim'; import type { HttpRequestService } from '@/core/HttpRequestService.js'; type Field = { name: string, value: string }; diff --git a/packages/backend/src/models/AbuseUserReport.ts b/packages/backend/src/models/AbuseUserReport.ts index d43ebf9342..8f8d759004 100644 --- a/packages/backend/src/models/AbuseUserReport.ts +++ b/packages/backend/src/models/AbuseUserReport.ts @@ -4,6 +4,7 @@ */ import { PrimaryColumn, Entity, Index, JoinColumn, Column, ManyToOne } from 'typeorm'; +import { MiInstance } from '@/models/Instance.js'; import { id } from './util/id.js'; import { MiUser } from './User.js'; @@ -88,11 +89,31 @@ export class MiAbuseUserReport { }) public targetUserHost: string | null; + @ManyToOne(() => MiInstance, { + // TODO create a foreign key constraint after hazelnoot/labs/persisted-instance-blocks is merged + createForeignKeyConstraints: false, + }) + @JoinColumn({ + name: 'targetUserHost', + referencedColumnName: 'host', + }) + public targetUserInstance: MiInstance | null; + @Index() @Column('varchar', { length: 128, nullable: true, comment: '[Denormalized]', }) public reporterHost: string | null; + + @ManyToOne(() => MiInstance, { + // TODO create a foreign key constraint after hazelnoot/labs/persisted-instance-blocks is merged + createForeignKeyConstraints: false, + }) + @JoinColumn({ + name: 'reporterHost', + referencedColumnName: 'host', + }) + public reporterInstance: MiInstance | null; //#endregion } diff --git a/packages/backend/src/models/Following.ts b/packages/backend/src/models/Following.ts index 62cbc29f26..0aa1b13976 100644 --- a/packages/backend/src/models/Following.ts +++ b/packages/backend/src/models/Following.ts @@ -4,6 +4,7 @@ */ import { PrimaryColumn, Entity, Index, JoinColumn, Column, ManyToOne } from 'typeorm'; +import { MiInstance } from '@/models/Instance.js'; import { id } from './util/id.js'; import { MiUser } from './User.js'; @@ -66,6 +67,16 @@ export class MiFollowing { }) public followerHost: string | null; + @ManyToOne(() => MiInstance, { + onDelete: 'CASCADE', + }) + @JoinColumn({ + name: 'followerHost', + foreignKeyConstraintName: 'FK_following_followerHost', + referencedColumnName: 'host', + }) + public followerInstance: MiInstance | null; + @Column('varchar', { length: 512, nullable: true, comment: '[Denormalized]', @@ -85,6 +96,16 @@ export class MiFollowing { }) public followeeHost: string | null; + @ManyToOne(() => MiInstance, { + onDelete: 'CASCADE', + }) + @JoinColumn({ + name: 'followeeHost', + foreignKeyConstraintName: 'FK_following_followeeHost', + referencedColumnName: 'host', + }) + public followeeInstance: MiInstance | null; + @Column('varchar', { length: 512, nullable: true, comment: '[Denormalized]', diff --git a/packages/backend/src/models/Instance.ts b/packages/backend/src/models/Instance.ts index c64ebb1b3b..0cde4b75fc 100644 --- a/packages/backend/src/models/Instance.ts +++ b/packages/backend/src/models/Instance.ts @@ -6,6 +6,7 @@ import { Entity, PrimaryColumn, Index, Column } from 'typeorm'; import { id } from './util/id.js'; +@Index('IDX_instance_host_key', { synchronize: false }) // ((lower(reverse("host"::text)) || '.'::text) @Entity('instance') export class MiInstance { @PrimaryColumn(id()) @@ -98,6 +99,56 @@ export class MiInstance { }) public suspensionState: 'none' | 'manuallySuspended' | 'goneSuspended' | 'autoSuspendedForNotResponding'; + /** + * True if this instance is blocked from federation. + */ + @Column('boolean', { + nullable: false, + default: false, + comment: 'True if this instance is blocked from federation.', + }) + public isBlocked: boolean; + + /** + * True if this instance is allow-listed. + */ + @Column('boolean', { + nullable: false, + default: false, + comment: 'True if this instance is allow-listed.', + }) + public isAllowListed: boolean; + + /** + * True if this instance is part of the local bubble. + */ + @Column('boolean', { + nullable: false, + default: false, + comment: 'True if this instance is part of the local bubble.', + }) + public isBubbled: boolean; + + /** + * True if this instance is silenced. + */ + @Column('boolean', { + nullable: false, + default: false, + comment: 'True if this instance is silenced.', + }) + public isSilenced: boolean; + + /** + * True if this instance is media-silenced. + */ + @Column('boolean', { + nullable: false, + default: false, + comment: 'True if this instance is media-silenced.', + }) + public isMediaSilenced: boolean; + @Column('varchar', { length: 64, nullable: true, comment: 'The software of the Instance.', diff --git a/packages/backend/src/models/Note.ts b/packages/backend/src/models/Note.ts index ee2098216d..bbe183cfbb 100644 --- a/packages/backend/src/models/Note.ts +++ b/packages/backend/src/models/Note.ts @@ -5,12 +5,15 @@ import { Entity, Index, JoinColumn, Column, PrimaryColumn, ManyToOne } from 'typeorm'; import { noteVisibilities } from '@/types.js'; +import { MiInstance } from '@/models/Instance.js'; import { id } from './util/id.js'; import { MiUser } from './User.js'; import { MiChannel } from './Channel.js'; import type { MiDriveFile } from './DriveFile.js'; @Index('IDX_724b311e6f883751f261ebe378', ['userId', 'id']) +@Index('IDX_note_userHost_id', { synchronize: false }) // (userHost, id desc) +@Index('IDX_note_for_timelines', { synchronize: false }) // (id desc, channelId, visibility, userHost) @Entity('note') export class MiNote { @PrimaryColumn(id()) @@ -130,6 +133,7 @@ export class MiNote { }) public uri: string | null; + @Index('IDX_note_url') @Column('varchar', { length: 512, nullable: true, comment: 'The human readable url of a note. it will be null when the note is local.', @@ -215,13 +219,22 @@ export class MiNote { public processErrors: string[] | null; //#region Denormalized fields - @Index() @Column('varchar', { length: 128, nullable: true, comment: '[Denormalized]', }) public userHost: string | null; + @ManyToOne(() => MiInstance, { + onDelete: 'CASCADE', + }) + @JoinColumn({ + name: 'userHost', + foreignKeyConstraintName: 'FK_note_userHost', + referencedColumnName: 'host', + }) + public userInstance: MiInstance | null; + @Column({ ...id(), nullable: true, @@ -235,6 +248,16 @@ export class MiNote { }) public replyUserHost: string | null; + @ManyToOne(() => MiInstance, { + onDelete: 'CASCADE', + }) + @JoinColumn({ + name: 'replyUserHost', + foreignKeyConstraintName: 'FK_note_replyUserHost', + referencedColumnName: 'host', + }) + public replyUserInstance: MiInstance | null; + @Column({ ...id(), nullable: true, @@ -247,6 +270,16 @@ export class MiNote { comment: '[Denormalized]', }) public renoteUserHost: string | null; + + @ManyToOne(() => MiInstance, { + onDelete: 'CASCADE', + }) + @JoinColumn({ + name: 'renoteUserHost', + foreignKeyConstraintName: 'FK_note_renoteUserHost', + referencedColumnName: 'host', + }) + public renoteUserInstance: MiInstance | null; //#endregion constructor(data: Partial<MiNote>) { diff --git a/packages/backend/src/models/User.ts b/packages/backend/src/models/User.ts index 46f8e84a94..f40bb41a22 100644 --- a/packages/backend/src/models/User.ts +++ b/packages/backend/src/models/User.ts @@ -3,10 +3,12 @@ * SPDX-License-Identifier: AGPL-3.0-only */ -import { Entity, Column, Index, OneToOne, JoinColumn, PrimaryColumn } from 'typeorm'; +import { Entity, Column, Index, OneToOne, JoinColumn, PrimaryColumn, ManyToOne } from 'typeorm'; import { type UserUnsignedFetchOption, userUnsignedFetchOptions } from '@/const.js'; +import { MiInstance } from '@/models/Instance.js'; import { id } from './util/id.js'; import { MiDriveFile } from './DriveFile.js'; +import type { MiUserProfile } from './UserProfile.js'; @Entity('user') @Index(['usernameLower', 'host'], { unique: true }) @@ -292,6 +294,16 @@ export class MiUser { }) public host: string | null; + @ManyToOne(() => MiInstance, { + onDelete: 'CASCADE', + }) + @JoinColumn({ + name: 'host', + foreignKeyConstraintName: 'FK_user_host', + referencedColumnName: 'host', + }) + public instance: MiInstance | null; + @Column('varchar', { length: 512, nullable: true, comment: 'The inbox URL of the User. It will be null if the origin of the user is local.', @@ -378,6 +390,15 @@ export class MiUser { }) public allowUnsignedFetch: UserUnsignedFetchOption; + @Column('text', { + name: 'attributionDomains', + array: true, default: '{}', + }) + public attributionDomains: string[]; + + @OneToOne('user_profile', (profile: MiUserProfile) => profile.user) + public userProfile: MiUserProfile | null; + constructor(data: Partial<MiUser>) { if (data == null) return; diff --git a/packages/backend/src/models/UserProfile.ts b/packages/backend/src/models/UserProfile.ts index 29c453dd71..6ee72e6ddd 100644 --- a/packages/backend/src/models/UserProfile.ts +++ b/packages/backend/src/models/UserProfile.ts @@ -17,7 +17,7 @@ export class MiUserProfile { @PrimaryColumn(id()) public userId: MiUser['id']; - @OneToOne(type => MiUser, { + @OneToOne(() => MiUser, user => user.userProfile, { onDelete: 'CASCADE', }) @JoinColumn() diff --git a/packages/backend/src/models/json-schema/federation-instance.ts b/packages/backend/src/models/json-schema/federation-instance.ts index 57d4466ffa..fd6eddf594 100644 --- a/packages/backend/src/models/json-schema/federation-instance.ts +++ b/packages/backend/src/models/json-schema/federation-instance.ts @@ -135,5 +135,9 @@ export const packedFederationInstanceSchema = { type: 'string', optional: true, nullable: true, }, + isBubbled: { + type: 'boolean', + optional: false, nullable: false, + }, }, } as const; diff --git a/packages/backend/src/models/json-schema/user.ts b/packages/backend/src/models/json-schema/user.ts index 964a179244..1678cab067 100644 --- a/packages/backend/src/models/json-schema/user.ts +++ b/packages/backend/src/models/json-schema/user.ts @@ -73,6 +73,16 @@ export const packedUserLiteSchema = { type: 'string', nullable: true, optional: false, }, + description: { + type: 'string', + nullable: true, optional: false, + example: 'Hi masters, I am Ai!', + }, + createdAt: { + type: 'string', + nullable: false, optional: false, + format: 'date-time', + }, avatarDecorations: { type: 'array', nullable: false, optional: false, @@ -200,6 +210,10 @@ export const packedUserLiteSchema = { type: 'string', nullable: true, optional: false, }, + isSilenced: { + type: 'boolean', + nullable: false, optional: false, + }, }, }, emojis: { @@ -236,6 +250,14 @@ export const packedUserLiteSchema = { }, }, }, + attributionDomains: { + type: 'array', + nullable: false, optional: false, + items: { + type: 'string', + nullable: false, optional: false, + }, + }, }, } as const; @@ -266,11 +288,6 @@ export const packedUserDetailedNotMeOnlySchema = { nullable: false, optional: false, }, }, - createdAt: { - type: 'string', - nullable: false, optional: false, - format: 'date-time', - }, updatedAt: { type: 'string', nullable: true, optional: false, @@ -312,11 +329,6 @@ export const packedUserDetailedNotMeOnlySchema = { nullable: false, optional: false, example: false, }, - description: { - type: 'string', - nullable: true, optional: false, - example: 'Hi masters, I am Ai!', - }, location: { type: 'string', nullable: true, optional: false, diff --git a/packages/backend/src/postgres.ts b/packages/backend/src/postgres.ts index 632fd58927..45caec54ce 100644 --- a/packages/backend/src/postgres.ts +++ b/packages/backend/src/postgres.ts @@ -98,9 +98,12 @@ pg.types.setTypeParser(20, Number); export const dbLogger = new MisskeyLogger('db'); const sqlLogger = dbLogger.createSubLogger('sql', 'gray'); +const sqlMigrateLogger = sqlLogger.createSubLogger('migrate'); +const sqlSchemaLogger = sqlLogger.createSubLogger('schema'); export type LoggerProps = { disableQueryTruncation?: boolean; + enableQueryLogging?: boolean; enableQueryParamLogging?: boolean; printReplicationMode?: boolean, }; @@ -112,7 +115,7 @@ function highlightSql(sql: string) { } function truncateSql(sql: string) { - return sql.length > 100 ? `${sql.substring(0, 100)}...` : sql; + return sql.length > 100 ? `${sql.substring(0, 100)} [truncated]` : sql; } function stringifyParameter(param: any) { @@ -136,13 +139,16 @@ class MyCustomLogger implements Logger { modded = truncateSql(modded); } - return highlightSql(modded); + return this.props.enableQueryLogging ? highlightSql(modded) : modded; } @bindThis private transformParameters(parameters?: any[]) { if (this.props.enableQueryParamLogging && parameters && parameters.length > 0) { - return parameters.map(stringifyParameter); + return parameters.reduce((params, p, i) => { + params[`$${i + 1}`] = stringifyParameter(p); + return params; + }, {} as Record<string, string>); } return undefined; @@ -150,10 +156,13 @@ class MyCustomLogger implements Logger { @bindThis public logQuery(query: string, parameters?: any[], queryRunner?: QueryRunner) { + if (!this.props.enableQueryLogging) return; + const prefix = (this.props.printReplicationMode && queryRunner) ? `[${queryRunner.getReplicationMode()}] ` : undefined; - sqlLogger.info(this.transformQueryLog(query, { prefix }), this.transformParameters(parameters)); + const transformed = this.transformQueryLog(query, { prefix }); + sqlLogger.debug(`Query run: ${transformed}`, this.transformParameters(parameters)); } @bindThis @@ -161,7 +170,8 @@ class MyCustomLogger implements Logger { const prefix = (this.props.printReplicationMode && queryRunner) ? `[${queryRunner.getReplicationMode()}] ` : undefined; - sqlLogger.error(this.transformQueryLog(query, { prefix }), this.transformParameters(parameters)); + const transformed = this.transformQueryLog(query, { prefix }); + sqlLogger.error(`Query error (${error}): ${transformed}`, this.transformParameters(parameters)); } @bindThis @@ -169,22 +179,32 @@ class MyCustomLogger implements Logger { const prefix = (this.props.printReplicationMode && queryRunner) ? `[${queryRunner.getReplicationMode()}] ` : undefined; - sqlLogger.warn(this.transformQueryLog(query, { prefix }), this.transformParameters(parameters)); + const transformed = this.transformQueryLog(query, { prefix }); + sqlLogger.warn(`Query is slow (${time}ms): ${transformed}`, this.transformParameters(parameters)); } @bindThis public logSchemaBuild(message: string) { - sqlLogger.info(message); + sqlSchemaLogger.debug(message); } @bindThis - public log(message: string) { - sqlLogger.info(message); + public log(level: 'log' | 'info' | 'warn', message: string) { + switch (level) { + case 'log': + case 'info': { + sqlLogger.info(message); + break; + } + case 'warn': { + sqlLogger.warn(message); + } + } } @bindThis public logMigration(message: string) { - sqlLogger.info(message); + sqlMigrateLogger.debug(message); } } @@ -306,7 +326,7 @@ export function createPostgresDataSource(config: Config) { } : {}), synchronize: process.env.NODE_ENV === 'test', dropSchema: process.env.NODE_ENV === 'test', - cache: !config.db.disableCache && process.env.NODE_ENV !== 'test' ? { // dbをcloseしても何故かredisのコネクションが内部的に残り続けるようで、テストの際に支障が出るため無効にする(キャッシュも含めてテストしたいため本当は有効にしたいが...) + cache: config.db.disableCache === false && process.env.NODE_ENV !== 'test' ? { // dbをcloseしても何故かredisのコネクションが内部的に残り続けるようで、テストの際に支障が出るため無効にする(キャッシュも含めてテストしたいため本当は有効にしたいが...) type: 'ioredis', options: { ...config.redis, @@ -314,14 +334,13 @@ export function createPostgresDataSource(config: Config) { }, } : false, logging: log, - logger: log - ? new MyCustomLogger({ - disableQueryTruncation: config.logging?.sql?.disableQueryTruncation, - enableQueryParamLogging: config.logging?.sql?.enableQueryParamLogging, - printReplicationMode: !!config.dbReplications, - }) - : undefined, - maxQueryExecutionTime: 300, + logger: new MyCustomLogger({ + disableQueryTruncation: config.logging?.sql?.disableQueryTruncation, + enableQueryLogging: log, + enableQueryParamLogging: config.logging?.sql?.enableQueryParamLogging, + printReplicationMode: !!config.dbReplications, + }), + maxQueryExecutionTime: config.db.slowQueryThreshold, entities: entities, migrations: ['../../migration/*.js'], }); diff --git a/packages/backend/src/queue/QueueProcessorService.ts b/packages/backend/src/queue/QueueProcessorService.ts index 7f7ce2452c..4c1a6a1d9e 100644 --- a/packages/backend/src/queue/QueueProcessorService.ts +++ b/packages/backend/src/queue/QueueProcessorService.ts @@ -11,7 +11,7 @@ import { DI } from '@/di-symbols.js'; import type Logger from '@/logger.js'; import { bindThis } from '@/decorators.js'; import { CheckModeratorsActivityProcessorService } from '@/queue/processors/CheckModeratorsActivityProcessorService.js'; -import { StatusError } from '@/misc/status-error.js'; +import { renderFullError } from '@/misc/render-full-error.js'; import { UserWebhookDeliverProcessorService } from './processors/UserWebhookDeliverProcessorService.js'; import { SystemWebhookDeliverProcessorService } from './processors/SystemWebhookDeliverProcessorService.js'; import { EndedPollNotificationProcessorService } from './processors/EndedPollNotificationProcessorService.js'; @@ -73,7 +73,9 @@ function getJobInfo(job: Bull.Job | undefined, increment = false): string { const currentAttempts = job.attemptsMade + (increment ? 1 : 0); const maxAttempts = job.opts.attempts ?? 0; - return `id=${job.id} attempts=${currentAttempts}/${maxAttempts} age=${formated}`; + return job.name + ? `id=${job.id} attempts=${currentAttempts}/${maxAttempts} age=${formated} name=${job.name}` + : `id=${job.id} attempts=${currentAttempts}/${maxAttempts} age=${formated}`; } @Injectable() @@ -134,35 +136,6 @@ export class QueueProcessorService implements OnApplicationShutdown { ) { this.logger = this.queueLoggerService.logger; - function renderError(e?: Error) { - // 何故かeがundefinedで来ることがある - if (!e) return '?'; - - if (e instanceof Bull.UnrecoverableError || e.name === 'AbortError' || e instanceof StatusError) { - return `${e.name}: ${e.message}`; - } - - return { - stack: e.stack, - message: e.message, - name: e.name, - }; - } - - function renderJob(job?: Bull.Job) { - if (!job) return '?'; - - const info: Record<string, string> = { - info: getJobInfo(job), - data: job.data, - }; - - if (job.name) info.name = job.name; - if (job.failedReason) info.failedReason = job.failedReason; - - return info; - } - //#region system { const processer = (job: Bull.Job) => { @@ -196,7 +169,7 @@ export class QueueProcessorService implements OnApplicationShutdown { .on('active', (job) => logger.debug(`active id=${job.id}`)) .on('completed', (job, result) => logger.debug(`completed(${result}) id=${job.id}`)) .on('failed', (job, err: Error) => { - logger.error(`failed(${err.name}: ${err.message}) id=${job?.id ?? '?'}`, { job: renderJob(job), e: renderError(err) }); + this.logError(logger, err, job); if (config.sentryForBackend) { Sentry.captureMessage(`Queue: System: ${job?.name ?? '?'}: ${err.name}: ${err.message}`, { level: 'error', @@ -204,7 +177,7 @@ export class QueueProcessorService implements OnApplicationShutdown { }); } }) - .on('error', (err: Error) => logger.error(`error ${err.name}: ${err.message}`, { e: renderError(err) })) + .on('error', (err: Error) => this.logError(logger, err)) .on('stalled', (jobId) => logger.warn(`stalled id=${jobId}`)); } //#endregion @@ -261,7 +234,7 @@ export class QueueProcessorService implements OnApplicationShutdown { .on('active', (job) => logger.debug(`active id=${job.id}`)) .on('completed', (job, result) => logger.debug(`completed(${result}) id=${job.id}`)) .on('failed', (job, err) => { - logger.error(`failed(${err.name}: ${err.message}) id=${job?.id ?? '?'}`, { job: renderJob(job), e: renderError(err) }); + this.logError(logger, err, job); if (config.sentryForBackend) { Sentry.captureMessage(`Queue: DB: ${job?.name ?? '?'}: ${err.name}: ${err.message}`, { level: 'error', @@ -269,7 +242,7 @@ export class QueueProcessorService implements OnApplicationShutdown { }); } }) - .on('error', (err: Error) => logger.error(`error ${err.name}: ${err.message}`, { e: renderError(err) })) + .on('error', (err: Error) => this.logError(logger, err)) .on('stalled', (jobId) => logger.warn(`stalled id=${jobId}`)); } //#endregion @@ -301,7 +274,7 @@ export class QueueProcessorService implements OnApplicationShutdown { .on('active', (job) => logger.debug(`active ${getJobInfo(job, true)} to=${job.data.to}`)) .on('completed', (job, result) => logger.debug(`completed(${result}) ${getJobInfo(job, true)} to=${job.data.to}`)) .on('failed', (job, err) => { - logger.error(`failed(${err.name}: ${err.message}) ${getJobInfo(job)} to=${job ? job.data.to : '-'}`); + this.logError(logger, err, job); if (config.sentryForBackend) { Sentry.captureMessage(`Queue: Deliver: ${err.name}: ${err.message}`, { level: 'error', @@ -309,7 +282,7 @@ export class QueueProcessorService implements OnApplicationShutdown { }); } }) - .on('error', (err: Error) => logger.error(`error ${err.name}: ${err.message}`, { e: renderError(err) })) + .on('error', (err: Error) => this.logError(logger, err)) .on('stalled', (jobId) => logger.warn(`stalled id=${jobId}`)); } //#endregion @@ -341,7 +314,7 @@ export class QueueProcessorService implements OnApplicationShutdown { .on('active', (job) => logger.debug(`active ${getJobInfo(job, true)}`)) .on('completed', (job, result) => logger.debug(`completed(${result}) ${getJobInfo(job, true)}`)) .on('failed', (job, err) => { - logger.error(`failed(${err.name}: ${err.message}) ${getJobInfo(job)} activity=${job ? (job.data.activity ? job.data.activity.id : 'none') : '-'}`, { job: renderJob(job), e: renderError(err) }); + this.logError(logger, err, job); if (config.sentryForBackend) { Sentry.captureMessage(`Queue: Inbox: ${err.name}: ${err.message}`, { level: 'error', @@ -349,7 +322,7 @@ export class QueueProcessorService implements OnApplicationShutdown { }); } }) - .on('error', (err: Error) => logger.error('inbox error:', renderError(err))) + .on('error', (err: Error) => this.logError(logger, err)) .on('stalled', (jobId) => logger.warn(`stalled id=${jobId}`)); } //#endregion @@ -381,7 +354,7 @@ export class QueueProcessorService implements OnApplicationShutdown { .on('active', (job) => logger.debug(`active ${getJobInfo(job, true)} to=${job.data.to}`)) .on('completed', (job, result) => logger.debug(`completed(${result}) ${getJobInfo(job, true)} to=${job.data.to}`)) .on('failed', (job, err) => { - logger.error(`failed(${err.name}: ${err.message}) ${getJobInfo(job)} to=${job ? job.data.to : '-'}`); + this.logError(logger, err, job); if (config.sentryForBackend) { Sentry.captureMessage(`Queue: UserWebhookDeliver: ${err.name}: ${err.message}`, { level: 'error', @@ -389,7 +362,7 @@ export class QueueProcessorService implements OnApplicationShutdown { }); } }) - .on('error', (err: Error) => logger.error(`error ${err.name}: ${err.message}`, { e: renderError(err) })) + .on('error', (err: Error) => this.logError(logger, err)) .on('stalled', (jobId) => logger.warn(`stalled id=${jobId}`)); } //#endregion @@ -421,7 +394,7 @@ export class QueueProcessorService implements OnApplicationShutdown { .on('active', (job) => logger.debug(`active ${getJobInfo(job, true)} to=${job.data.to}`)) .on('completed', (job, result) => logger.debug(`completed(${result}) ${getJobInfo(job, true)} to=${job.data.to}`)) .on('failed', (job, err) => { - logger.error(`failed(${err.name}: ${err.message}) ${getJobInfo(job)} to=${job ? job.data.to : '-'}`); + this.logError(logger, err, job); if (config.sentryForBackend) { Sentry.captureMessage(`Queue: SystemWebhookDeliver: ${err.name}: ${err.message}`, { level: 'error', @@ -429,7 +402,7 @@ export class QueueProcessorService implements OnApplicationShutdown { }); } }) - .on('error', (err: Error) => logger.error(`error ${err.name}: ${err.message}`, { e: renderError(err) })) + .on('error', (err: Error) => this.logError(logger, err)) .on('stalled', (jobId) => logger.warn(`stalled id=${jobId}`)); } //#endregion @@ -468,7 +441,7 @@ export class QueueProcessorService implements OnApplicationShutdown { .on('active', (job) => logger.debug(`active id=${job.id}`)) .on('completed', (job, result) => logger.debug(`completed(${result}) id=${job.id}`)) .on('failed', (job, err) => { - logger.error(`failed(${err.name}: ${err.message}) id=${job?.id ?? '?'}`, { job: renderJob(job), e: renderError(err) }); + this.logError(logger, err, job); if (config.sentryForBackend) { Sentry.captureMessage(`Queue: Relationship: ${job?.name ?? '?'}: ${err.name}: ${err.message}`, { level: 'error', @@ -476,7 +449,7 @@ export class QueueProcessorService implements OnApplicationShutdown { }); } }) - .on('error', (err: Error) => logger.error(`error ${err.name}: ${err.message}`, { e: renderError(err) })) + .on('error', (err: Error) => this.logError(logger, err)) .on('stalled', (jobId) => logger.warn(`stalled id=${jobId}`)); } //#endregion @@ -509,7 +482,7 @@ export class QueueProcessorService implements OnApplicationShutdown { .on('active', (job) => logger.debug(`active id=${job.id}`)) .on('completed', (job, result) => logger.debug(`completed(${result}) id=${job.id}`)) .on('failed', (job, err) => { - logger.error(`failed(${err.name}: ${err.message}) id=${job?.id ?? '?'}`, { job: renderJob(job), e: renderError(err) }); + this.logError(logger, err, job); if (config.sentryForBackend) { Sentry.captureMessage(`Queue: ObjectStorage: ${job?.name ?? '?'}: ${err.name}: ${err.message}`, { level: 'error', @@ -517,13 +490,15 @@ export class QueueProcessorService implements OnApplicationShutdown { }); } }) - .on('error', (err: Error) => logger.error(`error ${err.name}: ${err.message}`, { e: renderError(err) })) + .on('error', (err: Error) => this.logError(logger, err)) .on('stalled', (jobId) => logger.warn(`stalled id=${jobId}`)); } //#endregion //#region ended poll notification { + const logger = this.logger.createSubLogger('endedPollNotification'); + this.endedPollNotificationQueueWorker = new Bull.Worker(QUEUE.ENDED_POLL_NOTIFICATION, (job) => { if (this.config.sentryForBackend) { return Sentry.startSpan({ name: 'Queue: EndedPollNotification' }, () => this.endedPollNotificationProcessorService.process(job)); @@ -534,19 +509,75 @@ export class QueueProcessorService implements OnApplicationShutdown { ...baseWorkerOptions(this.config, QUEUE.ENDED_POLL_NOTIFICATION), autorun: false, }); + this.endedPollNotificationQueueWorker + .on('active', (job) => logger.debug(`active id=${job.id}`)) + .on('completed', (job, result) => logger.debug(`completed(${result}) id=${job.id}`)) + .on('failed', (job, err) => { + this.logError(logger, err, job); + if (config.sentryForBackend) { + Sentry.captureMessage(`Queue: EndedPollNotification: ${job?.name ?? '?'}: ${err.name}: ${err.message}`, { + level: 'error', + extra: { job, err }, + }); + } + }) + .on('error', (err: Error) => this.logError(logger, err)) + .on('stalled', (jobId) => logger.warn(`stalled id=${jobId}`)); } //#endregion //#region schedule note post { + const logger = this.logger.createSubLogger('scheduleNotePost'); + this.schedulerNotePostQueueWorker = new Bull.Worker(QUEUE.SCHEDULE_NOTE_POST, (job) => this.scheduleNotePostProcessorService.process(job), { ...baseWorkerOptions(this.config, QUEUE.SCHEDULE_NOTE_POST), autorun: false, }); + this.schedulerNotePostQueueWorker + .on('active', (job) => logger.debug(`active id=${job.id}`)) + .on('completed', (job, result) => logger.debug(`completed(${result}) id=${job.id}`)) + .on('failed', (job, err) => { + this.logError(logger, err, job); + if (config.sentryForBackend) { + Sentry.captureMessage(`Queue: ${QUEUE.SCHEDULE_NOTE_POST}: ${job?.name ?? '?'}: ${err.name}: ${err.message}`, { + level: 'error', + extra: { job, err }, + }); + } + }) + .on('error', (err: Error) => this.logError(logger, err)) + .on('stalled', (jobId) => logger.warn(`stalled id=${jobId}`)); } //#endregion } + private logError(logger: Logger, err: unknown, job?: Bull.Job | null): void { + const parts: string[] = []; + + // Render job + if (job) { + parts.push('job ['); + parts.push(getJobInfo(job)); + parts.push('] failed: '); + } else { + parts.push('job failed: '); + } + + // Render error + const fullError = renderFullError(err); + const errorText = typeof(fullError) === 'string' ? fullError : undefined; + if (errorText) { + parts.push(errorText); + } else if (job?.failedReason) { + parts.push(job.failedReason); + } + + const message = parts.join(''); + const data = typeof(fullError) !== 'string' ? { err: fullError } : undefined; + logger.error(message, data); + } + @bindThis public async start(): Promise<void> { await Promise.all([ diff --git a/packages/backend/src/queue/processors/AggregateRetentionProcessorService.ts b/packages/backend/src/queue/processors/AggregateRetentionProcessorService.ts index 4769cccabf..30bdd6ccca 100644 --- a/packages/backend/src/queue/processors/AggregateRetentionProcessorService.ts +++ b/packages/backend/src/queue/processors/AggregateRetentionProcessorService.ts @@ -62,7 +62,7 @@ export class AggregateRetentionProcessorService { }); } catch (err) { if (isDuplicateKeyValueError(err)) { - this.logger.succ('Skip because it has already been processed by another worker.'); + this.logger.debug('Skip because it has already been processed by another worker.'); return; } throw err; @@ -87,6 +87,6 @@ export class AggregateRetentionProcessorService { }); } - this.logger.succ('Retention aggregated.'); + this.logger.info('Retention aggregated.'); } } diff --git a/packages/backend/src/queue/processors/BakeBufferedReactionsProcessorService.ts b/packages/backend/src/queue/processors/BakeBufferedReactionsProcessorService.ts index d49c99f694..83b375de3f 100644 --- a/packages/backend/src/queue/processors/BakeBufferedReactionsProcessorService.ts +++ b/packages/backend/src/queue/processors/BakeBufferedReactionsProcessorService.ts @@ -37,6 +37,6 @@ export class BakeBufferedReactionsProcessorService { await this.reactionsBufferingService.bake(); - this.logger.succ('All buffered reactions baked.'); + this.logger.info('All buffered reactions baked.'); } } diff --git a/packages/backend/src/queue/processors/CheckExpiredMutingsProcessorService.ts b/packages/backend/src/queue/processors/CheckExpiredMutingsProcessorService.ts index 448fc9c763..76d0cb4304 100644 --- a/packages/backend/src/queue/processors/CheckExpiredMutingsProcessorService.ts +++ b/packages/backend/src/queue/processors/CheckExpiredMutingsProcessorService.ts @@ -41,6 +41,6 @@ export class CheckExpiredMutingsProcessorService { await this.userMutingService.unmute(expired); } - this.logger.succ('All expired mutings checked.'); + this.logger.info('All expired mutings checked.'); } } diff --git a/packages/backend/src/queue/processors/CheckModeratorsActivityProcessorService.ts b/packages/backend/src/queue/processors/CheckModeratorsActivityProcessorService.ts index db8d2e789e..7821cd3d1d 100644 --- a/packages/backend/src/queue/processors/CheckModeratorsActivityProcessorService.ts +++ b/packages/backend/src/queue/processors/CheckModeratorsActivityProcessorService.ts @@ -98,16 +98,16 @@ export class CheckModeratorsActivityProcessorService { @bindThis public async process(): Promise<void> { - this.logger.info('start.'); + this.logger.debug('start.'); const meta = await this.metaService.fetch(false); if (!meta.disableRegistration) { await this.processImpl(); } else { - this.logger.info('is already invitation only.'); + this.logger.debug('is already invitation only.'); } - this.logger.succ('finish.'); + this.logger.debug('finish.'); } @bindThis diff --git a/packages/backend/src/queue/processors/CleanChartsProcessorService.ts b/packages/backend/src/queue/processors/CleanChartsProcessorService.ts index 8c5faa8d07..c11682b0fe 100644 --- a/packages/backend/src/queue/processors/CleanChartsProcessorService.ts +++ b/packages/backend/src/queue/processors/CleanChartsProcessorService.ts @@ -62,6 +62,6 @@ export class CleanChartsProcessorService { await this.perUserDriveChart.clean(); await this.apRequestChart.clean(); - this.logger.succ('All charts successfully cleaned.'); + this.logger.info('All charts successfully cleaned.'); } } diff --git a/packages/backend/src/queue/processors/CleanProcessorService.ts b/packages/backend/src/queue/processors/CleanProcessorService.ts index a26b69cd2b..104d19103f 100644 --- a/packages/backend/src/queue/processors/CleanProcessorService.ts +++ b/packages/backend/src/queue/processors/CleanProcessorService.ts @@ -69,6 +69,6 @@ export class CleanProcessorService { this.reversiService.cleanOutdatedGames(); - this.logger.succ('Cleaned.'); + this.logger.info('Cleaned.'); } } diff --git a/packages/backend/src/queue/processors/CleanRemoteFilesProcessorService.ts b/packages/backend/src/queue/processors/CleanRemoteFilesProcessorService.ts index 81842b221f..2eddae95c8 100644 --- a/packages/backend/src/queue/processors/CleanRemoteFilesProcessorService.ts +++ b/packages/backend/src/queue/processors/CleanRemoteFilesProcessorService.ts @@ -75,6 +75,6 @@ export class CleanRemoteFilesProcessorService { await job.updateProgress(100 / total * deletedCount); } - this.logger.succ(`All cached remote files processed. Total deleted: ${deletedCount}, Failed: ${errorCount}.`); + this.logger.info(`All cached remote files processed. Total deleted: ${deletedCount}, Failed: ${errorCount}.`); } } diff --git a/packages/backend/src/queue/processors/DeleteAccountProcessorService.ts b/packages/backend/src/queue/processors/DeleteAccountProcessorService.ts index 4e9779a41b..5bf64e4f04 100644 --- a/packages/backend/src/queue/processors/DeleteAccountProcessorService.ts +++ b/packages/backend/src/queue/processors/DeleteAccountProcessorService.ts @@ -18,6 +18,7 @@ import { SearchService } from '@/core/SearchService.js'; import { ApLogService } from '@/core/ApLogService.js'; import { ReactionService } from '@/core/ReactionService.js'; import { QueueService } from '@/core/QueueService.js'; +import { CacheService } from '@/core/CacheService.js'; import { QueueLoggerService } from '../QueueLoggerService.js'; import type * as Bull from 'bullmq'; import type { DbUserDeleteJobData } from '../types.js'; @@ -94,6 +95,7 @@ export class DeleteAccountProcessorService { private searchService: SearchService, private reactionService: ReactionService, private readonly apLogService: ApLogService, + private readonly cacheService: CacheService, ) { this.logger = this.queueLoggerService.logger.createSubLogger('delete-account'); } @@ -128,7 +130,7 @@ export class DeleteAccountProcessorService { userId: user.id, }); - this.logger.succ('All clips have been deleted.'); + this.logger.info('All clips have been deleted.'); } { // Delete favorites @@ -136,10 +138,26 @@ export class DeleteAccountProcessorService { userId: user.id, }); - this.logger.succ('All favorites have been deleted.'); + this.logger.info('All favorites have been deleted.'); } { // Delete user relations + await this.cacheService.refreshFollowRelationsFor(user.id); + await this.cacheService.userFollowingsCache.delete(user.id); + await this.cacheService.userFollowingsCache.delete(user.id); + await this.cacheService.userBlockingCache.delete(user.id); + await this.cacheService.userBlockedCache.delete(user.id); + await this.cacheService.userMutingsCache.delete(user.id); + await this.cacheService.userMutingsCache.delete(user.id); + await this.cacheService.hibernatedUserCache.delete(user.id); + await this.cacheService.renoteMutingsCache.delete(user.id); + await this.cacheService.userProfileCache.delete(user.id); + this.cacheService.userByIdCache.delete(user.id); + this.cacheService.localUserByIdCache.delete(user.id); + if (user.token) { + this.cacheService.localUserByNativeTokenCache.delete(user.token); + } + await this.followingsRepository.delete({ followerId: user.id, }); @@ -172,7 +190,7 @@ export class DeleteAccountProcessorService { muteeId: user.id, }); - this.logger.succ('All user relations have been deleted.'); + this.logger.info('All user relations have been deleted.'); } { // Delete reactions @@ -206,7 +224,7 @@ export class DeleteAccountProcessorService { } } - this.logger.succ('All reactions have been deleted'); + this.logger.info('All reactions have been deleted'); } { // Poll votes @@ -238,7 +256,7 @@ export class DeleteAccountProcessorService { }); } - this.logger.succ('All poll votes have been deleted'); + this.logger.info('All poll votes have been deleted'); } { // Delete scheduled notes @@ -254,7 +272,7 @@ export class DeleteAccountProcessorService { userId: user.id, }); - this.logger.succ('All scheduled notes deleted'); + this.logger.info('All scheduled notes deleted'); } { // Delete notes @@ -312,7 +330,7 @@ export class DeleteAccountProcessorService { } } - this.logger.succ('All of notes deleted'); + this.logger.info('All of notes deleted'); } { // Delete files @@ -341,7 +359,7 @@ export class DeleteAccountProcessorService { } } - this.logger.succ('All of files deleted'); + this.logger.info('All of files deleted'); } { // Delete actor logs @@ -353,7 +371,7 @@ export class DeleteAccountProcessorService { await this.apLogService.deleteInboxLogs(user.id) .catch(err => this.logger.error(err, `Failed to delete AP logs for user '${user.uri}'`)); - this.logger.succ('All AP logs deleted'); + this.logger.info('All AP logs deleted'); } // Do this BEFORE deleting the account! @@ -379,7 +397,7 @@ export class DeleteAccountProcessorService { await this.usersRepository.delete(user.id); } - this.logger.succ('Account data deleted'); + this.logger.info('Account data deleted'); } { // Send email notification diff --git a/packages/backend/src/queue/processors/DeleteDriveFilesProcessorService.ts b/packages/backend/src/queue/processors/DeleteDriveFilesProcessorService.ts index 291fa4a6d8..ac3cddbed0 100644 --- a/packages/backend/src/queue/processors/DeleteDriveFilesProcessorService.ts +++ b/packages/backend/src/queue/processors/DeleteDriveFilesProcessorService.ts @@ -74,6 +74,6 @@ export class DeleteDriveFilesProcessorService { job.updateProgress(deletedCount / total); } - this.logger.succ(`All drive files (${deletedCount}) of ${user.id} has been deleted.`); + this.logger.info(`All drive files (${deletedCount}) of ${user.id} has been deleted.`); } } diff --git a/packages/backend/src/queue/processors/DeliverProcessorService.ts b/packages/backend/src/queue/processors/DeliverProcessorService.ts index 5a16496011..fc4c8bb814 100644 --- a/packages/backend/src/queue/processors/DeliverProcessorService.ts +++ b/packages/backend/src/queue/processors/DeliverProcessorService.ts @@ -133,23 +133,18 @@ export class DeliverProcessorService { } }); - if (res instanceof StatusError) { + if (res instanceof StatusError && !res.isRetryable) { // 4xx - if (!res.isRetryable) { - // 相手が閉鎖していることを明示しているため、配送停止する - if (job.data.isSharedInbox && res.statusCode === 410) { - this.federatedInstanceService.fetchOrRegister(host).then(i => { - this.federatedInstanceService.update(i.id, { - suspensionState: 'goneSuspended', - }); + // 相手が閉鎖していることを明示しているため、配送停止する + if (job.data.isSharedInbox && res.statusCode === 410) { + this.federatedInstanceService.fetchOrRegister(host).then(i => { + this.federatedInstanceService.update(i.id, { + suspensionState: 'goneSuspended', }); - throw new Bull.UnrecoverableError(`${host} is gone`); - } - throw new Bull.UnrecoverableError(`${res.statusCode} ${res.statusMessage}`); + }); + throw new Bull.UnrecoverableError(`${host} is gone`); } - - // 5xx etc. - throw new Error(`${res.statusCode} ${res.statusMessage}`); + throw new Bull.UnrecoverableError(`${res.statusCode} ${res.statusMessage}`); } else { // DNS error, socket error, timeout ... throw res; diff --git a/packages/backend/src/queue/processors/ExportAccountDataProcessorService.ts b/packages/backend/src/queue/processors/ExportAccountDataProcessorService.ts index 33a2362c4a..58d542635f 100644 --- a/packages/backend/src/queue/processors/ExportAccountDataProcessorService.ts +++ b/packages/backend/src/queue/processors/ExportAccountDataProcessorService.ts @@ -22,6 +22,7 @@ import { Packed } from '@/misc/json-schema.js'; import { UtilityService } from '@/core/UtilityService.js'; import { DownloadService } from '@/core/DownloadService.js'; import { EmailService } from '@/core/EmailService.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; import { QueueLoggerService } from '../QueueLoggerService.js'; import type * as Bull from 'bullmq'; @@ -85,21 +86,23 @@ export class ExportAccountDataProcessorService { @bindThis public async process(job: Bull.Job): Promise<void> { - this.logger.info('Exporting Account Data...'); - const user = await this.usersRepository.findOneBy({ id: job.data.user.id }); if (user == null) { + this.logger.debug(`Skip: user ${job.data.user.id} does not exist`); return; } const profile = await this.userProfilesRepository.findOneBy({ userId: job.data.user.id }); if (profile == null) { + this.logger.debug(`Skip: user ${job.data.user.id} has no profile`); return; } + this.logger.info(`Exporting account data for ${job.data.user.id} ...`); + const [path, cleanup] = await createTempDir(); - this.logger.info(`Temp dir is ${path}`); + this.logger.debug(`Temp dir is ${path}`); // User Export @@ -113,7 +116,7 @@ export class ExportAccountDataProcessorService { return new Promise<void>((res, rej) => { userStream.write(text, err => { if (err) { - this.logger.error(err); + this.logger.error('Error writing user:', err); rej(err); } else { res(); @@ -145,7 +148,7 @@ export class ExportAccountDataProcessorService { return new Promise<void>((res, rej) => { profileStream.write(text, err => { if (err) { - this.logger.error(err); + this.logger.error('Error writing profile:', err); rej(err); } else { res(); @@ -179,7 +182,7 @@ export class ExportAccountDataProcessorService { return new Promise<void>((res, rej) => { ipStream.write(text, err => { if (err) { - this.logger.error(err); + this.logger.error('Error writing IPs:', err); rej(err); } else { res(); @@ -214,7 +217,7 @@ export class ExportAccountDataProcessorService { return new Promise<void>((res, rej) => { notesStream.write(text, err => { if (err) { - this.logger.error(err); + this.logger.error('Error writing notes:', err); rej(err); } else { res(); @@ -275,7 +278,7 @@ export class ExportAccountDataProcessorService { return new Promise<void>((res, rej) => { followingStream.write(text, err => { if (err) { - this.logger.error(err); + this.logger.error('Error writing following:', err); rej(err); } else { res(); @@ -345,7 +348,7 @@ export class ExportAccountDataProcessorService { return new Promise<void>((res, rej) => { followerStream.write(text, err => { if (err) { - this.logger.error(err); + this.logger.error('Error writing followers:', err); rej(err); } else { res(); @@ -406,7 +409,7 @@ export class ExportAccountDataProcessorService { return new Promise<void>((res, rej) => { filesStream.write(text, err => { if (err) { - this.logger.error(err); + this.logger.error('Error writing drive:', err); rej(err); } else { res(); @@ -432,7 +435,7 @@ export class ExportAccountDataProcessorService { await this.downloadService.downloadUrl(file.url, filePath); downloaded = true; } catch (e) { - this.logger.error(e instanceof Error ? e : new Error(e as string)); + this.logger.error(`Error writing drive file ${file.id} (${file.name}): ${renderInlineError(e)}`); } if (!downloaded) { @@ -464,7 +467,7 @@ export class ExportAccountDataProcessorService { return new Promise<void>((res, rej) => { mutingStream.write(text, err => { if (err) { - this.logger.error(err); + this.logger.error('Error writing mutings:', err); rej(err); } else { res(); @@ -527,7 +530,7 @@ export class ExportAccountDataProcessorService { return new Promise<void>((res, rej) => { blockingStream.write(text, err => { if (err) { - this.logger.error(err); + this.logger.error('Error writing blockings:', err); rej(err); } else { res(); @@ -589,7 +592,7 @@ export class ExportAccountDataProcessorService { return new Promise<void>((res, rej) => { favoriteStream.write(text, err => { if (err) { - this.logger.error(err); + this.logger.error('Error writing favorites:', err); rej(err); } else { res(); @@ -650,7 +653,7 @@ export class ExportAccountDataProcessorService { return new Promise<void>((res, rej) => { antennaStream.write(text, err => { if (err) { - this.logger.error(err); + this.logger.error('Error writing antennas:', err); rej(err); } else { res(); @@ -708,7 +711,7 @@ export class ExportAccountDataProcessorService { return new Promise<void>((res, rej) => { listStream.write(text, err => { if (err) { - this.logger.error(err); + this.logger.error('Error writing lists:', err); rej(err); } else { res(); @@ -744,12 +747,12 @@ export class ExportAccountDataProcessorService { zlib: { level: 0 }, }); archiveStream.on('close', async () => { - this.logger.succ(`Exported to: ${archivePath}`); + this.logger.debug(`Exported to path: ${archivePath}`); const fileName = 'data-request-' + dateFormat(new Date(), 'yyyy-MM-dd-HH-mm-ss') + '.zip'; const driveFile = await this.driveService.addFile({ user, path: archivePath, name: fileName, force: true }); - this.logger.succ(`Exported to: ${driveFile.id}`); + this.logger.debug(`Exported to drive: ${driveFile.id}`); cleanup(); archiveCleanup(); if (profile.email) { diff --git a/packages/backend/src/queue/processors/ExportAntennasProcessorService.ts b/packages/backend/src/queue/processors/ExportAntennasProcessorService.ts index b3111865ad..61d76da5ac 100644 --- a/packages/backend/src/queue/processors/ExportAntennasProcessorService.ts +++ b/packages/backend/src/queue/processors/ExportAntennasProcessorService.ts @@ -45,15 +45,19 @@ export class ExportAntennasProcessorService { public async process(job: Bull.Job<DBExportAntennasData>): Promise<void> { const user = await this.usersRepository.findOneBy({ id: job.data.user.id }); if (user == null) { + this.logger.debug(`Skip: user ${job.data.user.id} does not exist`); return; } + + this.logger.info(`Exporting antennas of ${job.data.user.id} ...`); + const [path, cleanup] = await createTemp(); const stream = fs.createWriteStream(path, { flags: 'a' }); const write = (input: string): Promise<void> => { return new Promise((resolve, reject) => { stream.write(input, err => { if (err) { - this.logger.error(err); + this.logger.error('Error exporting antennas:', err); reject(); } else { resolve(); @@ -96,7 +100,7 @@ export class ExportAntennasProcessorService { const fileName = 'antennas-' + DateFormat(new Date(), 'yyyy-MM-dd-HH-mm-ss') + '.json'; const driveFile = await this.driveService.addFile({ user, path, name: fileName, force: true, ext: 'json' }); - this.logger.succ('Exported to: ' + driveFile.id); + this.logger.debug('Exported to: ' + driveFile.id); this.notificationService.createNotification(user.id, 'exportCompleted', { exportedEntity: 'antenna', diff --git a/packages/backend/src/queue/processors/ExportBlockingProcessorService.ts b/packages/backend/src/queue/processors/ExportBlockingProcessorService.ts index ecc439db69..4c17c3f718 100644 --- a/packages/backend/src/queue/processors/ExportBlockingProcessorService.ts +++ b/packages/backend/src/queue/processors/ExportBlockingProcessorService.ts @@ -40,17 +40,18 @@ export class ExportBlockingProcessorService { @bindThis public async process(job: Bull.Job<DbJobDataWithUser>): Promise<void> { - this.logger.info(`Exporting blocking of ${job.data.user.id} ...`); - const user = await this.usersRepository.findOneBy({ id: job.data.user.id }); if (user == null) { + this.logger.debug(`Skip: user ${job.data.user.id} does not exist`); return; } + this.logger.info(`Exporting blocking of ${job.data.user.id} ...`); + // Create temp file const [path, cleanup] = await createTemp(); - this.logger.info(`Temp file is ${path}`); + this.logger.debug(`Temp file is ${path}`); try { const stream = fs.createWriteStream(path, { flags: 'a' }); @@ -87,7 +88,7 @@ export class ExportBlockingProcessorService { await new Promise<void>((res, rej) => { stream.write(content + '\n', err => { if (err) { - this.logger.error(err); + this.logger.error('Error exporting blocking:', err); rej(err); } else { res(); @@ -105,12 +106,12 @@ export class ExportBlockingProcessorService { } stream.end(); - this.logger.succ(`Exported to: ${path}`); + this.logger.debug(`Exported to: ${path}`); const fileName = 'blocking-' + dateFormat(new Date(), 'yyyy-MM-dd-HH-mm-ss') + '.csv'; const driveFile = await this.driveService.addFile({ user, path, name: fileName, force: true, ext: 'csv' }); - this.logger.succ(`Exported to: ${driveFile.id}`); + this.logger.debug(`Exported to: ${driveFile.id}`); this.notificationService.createNotification(user.id, 'exportCompleted', { exportedEntity: 'blocking', diff --git a/packages/backend/src/queue/processors/ExportClipsProcessorService.ts b/packages/backend/src/queue/processors/ExportClipsProcessorService.ts index 583ddbb745..1d34d2b4e6 100644 --- a/packages/backend/src/queue/processors/ExportClipsProcessorService.ts +++ b/packages/backend/src/queue/processors/ExportClipsProcessorService.ts @@ -51,17 +51,18 @@ export class ExportClipsProcessorService { @bindThis public async process(job: Bull.Job<DbJobDataWithUser>): Promise<void> { - this.logger.info(`Exporting clips of ${job.data.user.id} ...`); - const user = await this.usersRepository.findOneBy({ id: job.data.user.id }); if (user == null) { + this.logger.debug(`Skip: user ${job.data.user.id} does not exist`); return; } + this.logger.info(`Exporting clips of ${job.data.user.id} ...`); + // Create temp file const [path, cleanup] = await createTemp(); - this.logger.info(`Temp file is ${path}`); + this.logger.debug(`Temp file is ${path}`); try { const stream = Writable.toWeb(fs.createWriteStream(path, { flags: 'a' })); @@ -75,12 +76,12 @@ export class ExportClipsProcessorService { await writer.write(']'); await writer.close(); - this.logger.succ(`Exported to: ${path}`); + this.logger.debug(`Exported to: ${path}`); const fileName = 'clips-' + dateFormat(new Date(), 'yyyy-MM-dd-HH-mm-ss') + '.json'; const driveFile = await this.driveService.addFile({ user, path, name: fileName, force: true, ext: 'json' }); - this.logger.succ(`Exported to: ${driveFile.id}`); + this.logger.debug(`Exported to: ${driveFile.id}`); this.notificationService.createNotification(user.id, 'exportCompleted', { exportedEntity: 'clip', diff --git a/packages/backend/src/queue/processors/ExportCustomEmojisProcessorService.ts b/packages/backend/src/queue/processors/ExportCustomEmojisProcessorService.ts index 14d32e78b3..b8f208bbfc 100644 --- a/packages/backend/src/queue/processors/ExportCustomEmojisProcessorService.ts +++ b/packages/backend/src/queue/processors/ExportCustomEmojisProcessorService.ts @@ -45,16 +45,17 @@ export class ExportCustomEmojisProcessorService { @bindThis public async process(job: Bull.Job): Promise<void> { - this.logger.info('Exporting custom emojis ...'); - const user = await this.usersRepository.findOneBy({ id: job.data.user.id }); if (user == null) { + this.logger.debug(`Skip: user ${job.data.user.id} does not exist`); return; } + this.logger.info(`Exporting custom emojis of ${job.data.user.id} ...`); + const [path, cleanup] = await createTempDir(); - this.logger.info(`Temp dir is ${path}`); + this.logger.debug(`Temp dir is ${path}`); const metaPath = path + '/meta.json'; @@ -66,7 +67,7 @@ export class ExportCustomEmojisProcessorService { return new Promise<void>((res, rej) => { metaStream.write(text, err => { if (err) { - this.logger.error(err); + this.logger.error('Error exporting custom emojis:', err); rej(err); } else { res(); @@ -101,7 +102,7 @@ export class ExportCustomEmojisProcessorService { await this.downloadService.downloadUrl(emoji.originalUrl, emojiPath); downloaded = true; } catch (e) { // TODO: 何度か再試行 - this.logger.error(e instanceof Error ? e : new Error(e as string)); + this.logger.error('Error exporting custom emojis:', e as Error); } if (!downloaded) { @@ -130,12 +131,12 @@ export class ExportCustomEmojisProcessorService { zlib: { level: 0 }, }); archiveStream.on('close', async () => { - this.logger.succ(`Exported to: ${archivePath}`); + this.logger.debug(`Exported to: ${archivePath}`); const fileName = 'custom-emojis-' + dateFormat(new Date(), 'yyyy-MM-dd-HH-mm-ss') + '.zip'; const driveFile = await this.driveService.addFile({ user, path: archivePath, name: fileName, force: true }); - this.logger.succ(`Exported to: ${driveFile.id}`); + this.logger.debug(`Exported to: ${driveFile.id}`); this.notificationService.createNotification(user.id, 'exportCompleted', { exportedEntity: 'customEmoji', diff --git a/packages/backend/src/queue/processors/ExportFavoritesProcessorService.ts b/packages/backend/src/queue/processors/ExportFavoritesProcessorService.ts index b81feece01..b5716f2d49 100644 --- a/packages/backend/src/queue/processors/ExportFavoritesProcessorService.ts +++ b/packages/backend/src/queue/processors/ExportFavoritesProcessorService.ts @@ -45,17 +45,18 @@ export class ExportFavoritesProcessorService { @bindThis public async process(job: Bull.Job<DbJobDataWithUser>): Promise<void> { - this.logger.info(`Exporting favorites of ${job.data.user.id} ...`); - const user = await this.usersRepository.findOneBy({ id: job.data.user.id }); if (user == null) { + this.logger.debug(`Skip: user ${job.data.user.id} does not exist`); return; } + this.logger.info(`Exporting favorites of ${job.data.user.id} ...`); + // Create temp file const [path, cleanup] = await createTemp(); - this.logger.info(`Temp file is ${path}`); + this.logger.debug(`Temp file is ${path}`); try { const stream = fs.createWriteStream(path, { flags: 'a' }); @@ -64,7 +65,7 @@ export class ExportFavoritesProcessorService { return new Promise<void>((res, rej) => { stream.write(text, err => { if (err) { - this.logger.error(err); + this.logger.error('Error exporting favorites:', err); rej(err); } else { res(); @@ -119,12 +120,12 @@ export class ExportFavoritesProcessorService { await write(']'); stream.end(); - this.logger.succ(`Exported to: ${path}`); + this.logger.debug(`Exported to: ${path}`); const fileName = 'favorites-' + dateFormat(new Date(), 'yyyy-MM-dd-HH-mm-ss') + '.json'; const driveFile = await this.driveService.addFile({ user, path, name: fileName, force: true, ext: 'json' }); - this.logger.succ(`Exported to: ${driveFile.id}`); + this.logger.debug(`Exported to: ${driveFile.id}`); this.notificationService.createNotification(user.id, 'exportCompleted', { exportedEntity: 'favorite', diff --git a/packages/backend/src/queue/processors/ExportFollowingProcessorService.ts b/packages/backend/src/queue/processors/ExportFollowingProcessorService.ts index 903f962515..883f35e366 100644 --- a/packages/backend/src/queue/processors/ExportFollowingProcessorService.ts +++ b/packages/backend/src/queue/processors/ExportFollowingProcessorService.ts @@ -44,17 +44,18 @@ export class ExportFollowingProcessorService { @bindThis public async process(job: Bull.Job<DbExportFollowingData>): Promise<void> { - this.logger.info(`Exporting following of ${job.data.user.id} ...`); - const user = await this.usersRepository.findOneBy({ id: job.data.user.id }); if (user == null) { + this.logger.debug(`Skip: user ${job.data.user.id} does not exist`); return; } + this.logger.info(`Exporting following of ${job.data.user.id} ...`); + // Create temp file const [path, cleanup] = await createTemp(); - this.logger.info(`Temp file is ${path}`); + this.logger.debug(`Temp file is ${path}`); try { const stream = fs.createWriteStream(path, { flags: 'a' }); @@ -98,7 +99,7 @@ export class ExportFollowingProcessorService { await new Promise<void>((res, rej) => { stream.write(content + '\n', err => { if (err) { - this.logger.error(err); + this.logger.error('Error exporting following:', err); rej(err); } else { res(); @@ -109,12 +110,12 @@ export class ExportFollowingProcessorService { } stream.end(); - this.logger.succ(`Exported to: ${path}`); + this.logger.debug(`Exported to: ${path}`); const fileName = 'following-' + dateFormat(new Date(), 'yyyy-MM-dd-HH-mm-ss') + '.csv'; const driveFile = await this.driveService.addFile({ user, path, name: fileName, force: true, ext: 'csv' }); - this.logger.succ(`Exported to: ${driveFile.id}`); + this.logger.debug(`Exported to: ${driveFile.id}`); this.notificationService.createNotification(user.id, 'exportCompleted', { exportedEntity: 'following', diff --git a/packages/backend/src/queue/processors/ExportMutingProcessorService.ts b/packages/backend/src/queue/processors/ExportMutingProcessorService.ts index f9867ade29..9cdb94beaf 100644 --- a/packages/backend/src/queue/processors/ExportMutingProcessorService.ts +++ b/packages/backend/src/queue/processors/ExportMutingProcessorService.ts @@ -40,17 +40,18 @@ export class ExportMutingProcessorService { @bindThis public async process(job: Bull.Job<DbJobDataWithUser>): Promise<void> { - this.logger.info(`Exporting muting of ${job.data.user.id} ...`); - const user = await this.usersRepository.findOneBy({ id: job.data.user.id }); if (user == null) { + this.logger.debug(`Skip: user ${job.data.user.id} does not exist`); return; } + this.logger.debug(`Exporting muting of ${job.data.user.id} ...`); + // Create temp file const [path, cleanup] = await createTemp(); - this.logger.info(`Temp file is ${path}`); + this.logger.debug(`Temp file is ${path}`); try { const stream = fs.createWriteStream(path, { flags: 'a' }); @@ -88,7 +89,7 @@ export class ExportMutingProcessorService { await new Promise<void>((res, rej) => { stream.write(content + '\n', err => { if (err) { - this.logger.error(err); + this.logger.error('Error exporting mutings:', err); rej(err); } else { res(); @@ -106,12 +107,12 @@ export class ExportMutingProcessorService { } stream.end(); - this.logger.succ(`Exported to: ${path}`); + this.logger.debug(`Exported to: ${path}`); const fileName = 'mute-' + dateFormat(new Date(), 'yyyy-MM-dd-HH-mm-ss') + '.csv'; const driveFile = await this.driveService.addFile({ user, path, name: fileName, force: true, ext: 'csv' }); - this.logger.succ(`Exported to: ${driveFile.id}`); + this.logger.debug(`Exported to: ${driveFile.id}`); this.notificationService.createNotification(user.id, 'exportCompleted', { exportedEntity: 'muting', diff --git a/packages/backend/src/queue/processors/ExportNotesProcessorService.ts b/packages/backend/src/queue/processors/ExportNotesProcessorService.ts index 9e2b678219..7d49a8dab2 100644 --- a/packages/backend/src/queue/processors/ExportNotesProcessorService.ts +++ b/packages/backend/src/queue/processors/ExportNotesProcessorService.ts @@ -120,17 +120,18 @@ export class ExportNotesProcessorService { @bindThis public async process(job: Bull.Job<DbJobDataWithUser>): Promise<void> { - this.logger.info(`Exporting notes of ${job.data.user.id} ...`); - const user = await this.usersRepository.findOneBy({ id: job.data.user.id }); if (user == null) { + this.logger.debug(`Skip: user ${job.data.user.id} does not exist`); return; } + this.logger.info(`Exporting notes of ${job.data.user.id} ...`); + // Create temp file const [path, cleanup] = await createTemp(); - this.logger.info(`Temp file is ${path}`); + this.logger.debug(`Temp file is ${path}`); try { // メモリが足りなくならないようにストリームで処理する @@ -146,12 +147,12 @@ export class ExportNotesProcessorService { .pipeThrough(new TextEncoderStream()) .pipeTo(new FileWriterStream(path)); - this.logger.succ(`Exported to: ${path}`); + this.logger.debug(`Exported to: ${path}`); const fileName = 'notes-' + dateFormat(new Date(), 'yyyy-MM-dd-HH-mm-ss') + '.json'; const driveFile = await this.driveService.addFile({ user, path, name: fileName, force: true, ext: 'json' }); - this.logger.succ(`Exported to: ${driveFile.id}`); + this.logger.debug(`Exported to: ${driveFile.id}`); this.notificationService.createNotification(user.id, 'exportCompleted', { exportedEntity: 'note', diff --git a/packages/backend/src/queue/processors/ExportUserListsProcessorService.ts b/packages/backend/src/queue/processors/ExportUserListsProcessorService.ts index c483d79854..43043e3a26 100644 --- a/packages/backend/src/queue/processors/ExportUserListsProcessorService.ts +++ b/packages/backend/src/queue/processors/ExportUserListsProcessorService.ts @@ -43,13 +43,14 @@ export class ExportUserListsProcessorService { @bindThis public async process(job: Bull.Job<DbJobDataWithUser>): Promise<void> { - this.logger.info(`Exporting user lists of ${job.data.user.id} ...`); - const user = await this.usersRepository.findOneBy({ id: job.data.user.id }); if (user == null) { + this.logger.debug(`Skip: user ${job.data.user.id} does not exist`); return; } + this.logger.info(`Exporting user lists of ${job.data.user.id} ...`); + const lists = await this.userListsRepository.findBy({ userId: user.id, }); @@ -57,7 +58,7 @@ export class ExportUserListsProcessorService { // Create temp file const [path, cleanup] = await createTemp(); - this.logger.info(`Temp file is ${path}`); + this.logger.debug(`Temp file is ${path}`); try { const stream = fs.createWriteStream(path, { flags: 'a' }); @@ -74,7 +75,7 @@ export class ExportUserListsProcessorService { await new Promise<void>((res, rej) => { stream.write(content + '\n', err => { if (err) { - this.logger.error(err); + this.logger.error('Error exporting lists:', err); rej(err); } else { res(); @@ -85,12 +86,12 @@ export class ExportUserListsProcessorService { } stream.end(); - this.logger.succ(`Exported to: ${path}`); + this.logger.debug(`Exported to: ${path}`); const fileName = 'user-lists-' + dateFormat(new Date(), 'yyyy-MM-dd-HH-mm-ss') + '.csv'; const driveFile = await this.driveService.addFile({ user, path, name: fileName, force: true, ext: 'csv' }); - this.logger.succ(`Exported to: ${driveFile.id}`); + this.logger.debug(`Exported to: ${driveFile.id}`); this.notificationService.createNotification(user.id, 'exportCompleted', { exportedEntity: 'userList', diff --git a/packages/backend/src/queue/processors/ImportAntennasProcessorService.ts b/packages/backend/src/queue/processors/ImportAntennasProcessorService.ts index 9c033b73e2..f29a19ce66 100644 --- a/packages/backend/src/queue/processors/ImportAntennasProcessorService.ts +++ b/packages/backend/src/queue/processors/ImportAntennasProcessorService.ts @@ -8,7 +8,7 @@ import _Ajv from 'ajv'; import { IdService } from '@/core/IdService.js'; import { GlobalEventService } from '@/core/GlobalEventService.js'; import Logger from '@/logger.js'; -import type { AntennasRepository } from '@/models/_.js'; +import type { AntennasRepository, UsersRepository } from '@/models/_.js'; import { DI } from '@/di-symbols.js'; import { bindThis } from '@/decorators.js'; import { QueueLoggerService } from '../QueueLoggerService.js'; @@ -59,6 +59,9 @@ export class ImportAntennasProcessorService { @Inject(DI.antennasRepository) private antennasRepository: AntennasRepository, + @Inject(DI.usersRepository) + private usersRepository: UsersRepository, + private queueLoggerService: QueueLoggerService, private idService: IdService, private globalEventService: GlobalEventService, @@ -68,12 +71,20 @@ export class ImportAntennasProcessorService { @bindThis public async process(job: Bull.Job<DBAntennaImportJobData>): Promise<void> { + const user = await this.usersRepository.findOneBy({ id: job.data.user.id }); + if (user == null) { + this.logger.debug(`Skip: user ${job.data.user.id} does not exist`); + return; + } + + this.logger.debug(`Importing blocking of ${job.data.user.id} ...`); + const now = new Date(); try { for (const antenna of job.data.antenna) { if (antenna.keywords.length === 0 || antenna.keywords[0].every(x => x === '')) continue; if (!validate(antenna)) { - this.logger.warn('Validation Failed'); + this.logger.warn('Antenna validation failed'); continue; } const result = await this.antennasRepository.insertOne({ @@ -92,11 +103,11 @@ export class ImportAntennasProcessorService { withReplies: antenna.withReplies, withFile: antenna.withFile, }); - this.logger.succ('Antenna created: ' + result.id); + this.logger.debug('Antenna created: ' + result.id); this.globalEventService.publishInternalEvent('antennaCreated', result); } } catch (err: any) { - this.logger.error(err); + this.logger.error('Error importing antennas:', err); } } } diff --git a/packages/backend/src/queue/processors/ImportBlockingProcessorService.ts b/packages/backend/src/queue/processors/ImportBlockingProcessorService.ts index b78229c648..e2de9532eb 100644 --- a/packages/backend/src/queue/processors/ImportBlockingProcessorService.ts +++ b/packages/backend/src/queue/processors/ImportBlockingProcessorService.ts @@ -40,10 +40,9 @@ export class ImportBlockingProcessorService { @bindThis public async process(job: Bull.Job<DbUserImportJobData>): Promise<void> { - this.logger.info(`Importing blocking of ${job.data.user.id} ...`); - const user = await this.usersRepository.findOneBy({ id: job.data.user.id }); if (user == null) { + this.logger.debug(`Skip: user ${job.data.user.id} does not exist`); return; } @@ -51,14 +50,17 @@ export class ImportBlockingProcessorService { id: job.data.fileId, }); if (file == null) { + this.logger.debug(`Skip: file ${job.data.fileId} does not exist`); return; } + this.logger.debug(`Importing blocking of ${job.data.user.id} ...`); + const csv = await this.downloadService.downloadTextFile(file.url); const targets = csv.trim().split('\n'); this.queueService.createImportBlockingToDbJob({ id: user.id }, targets); - this.logger.succ('Import jobs created'); + this.logger.debug('Import jobs created'); } @bindThis @@ -93,11 +95,11 @@ export class ImportBlockingProcessorService { // skip myself if (target.id === job.data.user.id) return; - this.logger.info(`Block ${target.id} ...`); + this.logger.debug(`Block ${target.id} ...`); this.queueService.createBlockJob([{ from: { id: user.id }, to: { id: target.id }, silent: true }]); } catch (e) { - this.logger.warn(`Error: ${e}`); + this.logger.error('Error importing blockings:', e as Error); } } } diff --git a/packages/backend/src/queue/processors/ImportCustomEmojisProcessorService.ts b/packages/backend/src/queue/processors/ImportCustomEmojisProcessorService.ts index d08cadd378..4b909328cd 100644 --- a/packages/backend/src/queue/processors/ImportCustomEmojisProcessorService.ts +++ b/packages/backend/src/queue/processors/ImportCustomEmojisProcessorService.ts @@ -16,6 +16,7 @@ import { DriveService } from '@/core/DriveService.js'; import { DownloadService } from '@/core/DownloadService.js'; import { bindThis } from '@/decorators.js'; import type { Config } from '@/config.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; import { QueueLoggerService } from '../QueueLoggerService.js'; import type * as Bull from 'bullmq'; import type { DbUserImportJobData } from '../types.js'; @@ -45,18 +46,19 @@ export class ImportCustomEmojisProcessorService { @bindThis public async process(job: Bull.Job<DbUserImportJobData>): Promise<void> { - this.logger.info('Importing custom emojis ...'); - const file = await this.driveFilesRepository.findOneBy({ id: job.data.fileId, }); if (file == null) { + this.logger.debug(`Skip: file ${job.data.fileId} does not exist`); return; } + this.logger.info(`Importing custom emojis from ${file.id} (${file.name}) ...`); + const [path, cleanup] = await createTempDir(); - this.logger.info(`Temp dir is ${path}`); + this.logger.debug(`Temp dir is ${path}`); const destPath = path + '/emojis.zip'; @@ -65,14 +67,14 @@ export class ImportCustomEmojisProcessorService { await this.downloadService.downloadUrl(file.url, destPath, { operationTimeout: this.config.import?.downloadTimeout, maxSize: this.config.import?.maxFileSize }); } catch (e) { // TODO: 何度か再試行 if (e instanceof Error || typeof e === 'string') { - this.logger.error(e); + this.logger.error('Error importing custom emojis:', e as Error); } throw e; } const outputPath = path + '/emojis'; try { - this.logger.succ(`Unzipping to ${outputPath}`); + this.logger.debug(`Unzipping to ${outputPath}`); ZipReader.withDestinationPath(outputPath).viaBuffer(await fs.promises.readFile(destPath)); const metaRaw = fs.readFileSync(outputPath + '/meta.json', 'utf-8'); const meta = JSON.parse(metaRaw); @@ -117,7 +119,7 @@ export class ImportCustomEmojisProcessorService { }); } catch (e) { if (e instanceof Error || typeof e === 'string') { - this.logger.error(`couldn't import ${emojiPath} for ${emojiInfo.name}: ${e}`); + this.logger.error(`couldn't import ${emojiPath} for ${emojiInfo.name}: ${renderInlineError(e)}`); } continue; } @@ -125,11 +127,9 @@ export class ImportCustomEmojisProcessorService { cleanup(); - this.logger.succ('Imported'); + this.logger.debug('Imported'); } catch (e) { - if (e instanceof Error || typeof e === 'string') { - this.logger.error(e); - } + this.logger.error('Error importing custom emojis:', e as Error); cleanup(); throw e; } diff --git a/packages/backend/src/queue/processors/ImportFollowingProcessorService.ts b/packages/backend/src/queue/processors/ImportFollowingProcessorService.ts index 70c9f3a096..816d5cf65a 100644 --- a/packages/backend/src/queue/processors/ImportFollowingProcessorService.ts +++ b/packages/backend/src/queue/processors/ImportFollowingProcessorService.ts @@ -40,10 +40,9 @@ export class ImportFollowingProcessorService { @bindThis public async process(job: Bull.Job<DbUserImportJobData>): Promise<void> { - this.logger.info(`Importing following of ${job.data.user.id} ...`); - const user = await this.usersRepository.findOneBy({ id: job.data.user.id }); if (user == null) { + this.logger.debug(`Skip: user ${job.data.user.id} does not exist`); return; } @@ -51,14 +50,17 @@ export class ImportFollowingProcessorService { id: job.data.fileId, }); if (file == null) { + this.logger.debug(`Skip: file ${job.data.fileId} does not exist`); return; } + this.logger.info(`Importing following of ${job.data.user.id} ...`); + const csv = await this.downloadService.downloadTextFile(file.url); const targets = csv.trim().split('\n'); this.queueService.createImportFollowingToDbJob({ id: user.id }, targets, job.data.withReplies); - this.logger.succ('Import jobs created'); + this.logger.debug('Import jobs created'); } @bindThis @@ -93,11 +95,11 @@ export class ImportFollowingProcessorService { // skip myself if (target.id === job.data.user.id) return; - this.logger.info(`Follow ${target.id} ${job.data.withReplies ? 'with replies' : 'without replies'} ...`); + this.logger.debug(`Follow ${target.id} ${job.data.withReplies ? 'with replies' : 'without replies'} ...`); this.queueService.createFollowJob([{ from: user, to: { id: target.id }, silent: true, withReplies: job.data.withReplies }]); } catch (e) { - this.logger.warn(`Error: ${e}`); + this.logger.error('Error importing followings:', e as Error); } } } diff --git a/packages/backend/src/queue/processors/ImportMutingProcessorService.ts b/packages/backend/src/queue/processors/ImportMutingProcessorService.ts index ec9d2b6c4c..d3827b12fd 100644 --- a/packages/backend/src/queue/processors/ImportMutingProcessorService.ts +++ b/packages/backend/src/queue/processors/ImportMutingProcessorService.ts @@ -14,6 +14,7 @@ import { DownloadService } from '@/core/DownloadService.js'; import { UserMutingService } from '@/core/UserMutingService.js'; import { UtilityService } from '@/core/UtilityService.js'; import { bindThis } from '@/decorators.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; import { QueueLoggerService } from '../QueueLoggerService.js'; import type * as Bull from 'bullmq'; import type { DbUserImportJobData } from '../types.js'; @@ -40,10 +41,9 @@ export class ImportMutingProcessorService { @bindThis public async process(job: Bull.Job<DbUserImportJobData>): Promise<void> { - this.logger.info(`Importing muting of ${job.data.user.id} ...`); - const user = await this.usersRepository.findOneBy({ id: job.data.user.id }); if (user == null) { + this.logger.debug(`Skip: user ${job.data.user.id} does not exist`); return; } @@ -51,9 +51,12 @@ export class ImportMutingProcessorService { id: job.data.fileId, }); if (file == null) { + this.logger.debug(`Skip: file ${job.data.fileId} does not exist`); return; } + this.logger.info(`Importing muting of ${job.data.user.id} ...`); + const csv = await this.downloadService.downloadTextFile(file.url); let linenum = 0; @@ -88,14 +91,14 @@ export class ImportMutingProcessorService { // skip myself if (target.id === job.data.user.id) continue; - this.logger.info(`Mute[${linenum}] ${target.id} ...`); + this.logger.debug(`Mute[${linenum}] ${target.id} ...`); await this.userMutingService.mute(user, target); } catch (e) { - this.logger.warn(`Error in line:${linenum} ${e}`); + this.logger.warn(`Error in line:${linenum} ${renderInlineError(e)}`); } } - this.logger.succ('Imported'); + this.logger.debug('Imported'); } } diff --git a/packages/backend/src/queue/processors/ImportNotesProcessorService.ts b/packages/backend/src/queue/processors/ImportNotesProcessorService.ts index 5e660e8081..e209855720 100644 --- a/packages/backend/src/queue/processors/ImportNotesProcessorService.ts +++ b/packages/backend/src/queue/processors/ImportNotesProcessorService.ts @@ -159,10 +159,9 @@ export class ImportNotesProcessorService { @bindThis public async process(job: Bull.Job<DbNoteImportJobData>): Promise<void> { - this.logger.info(`Starting note import of ${job.data.user.id} ...`); - const user = await this.usersRepository.findOneBy({ id: job.data.user.id }); if (user == null) { + this.logger.debug(`Skip: user ${job.data.user.id} does not exist`); return; } @@ -170,9 +169,12 @@ export class ImportNotesProcessorService { id: job.data.fileId, }); if (file == null) { + this.logger.debug(`Skip: file ${job.data.fileId} does not exist`); return; } + this.logger.info(`Starting note import of ${job.data.user.id} ...`); + let folder = await this.driveFoldersRepository.findOneBy({ name: 'Imports', userId: job.data.user.id }); if (folder == null) { await this.driveFoldersRepository.insert({ id: this.idService.gen(), name: 'Imports', userId: job.data.user.id }); @@ -184,7 +186,7 @@ export class ImportNotesProcessorService { if (type === 'Twitter' || file.name.startsWith('twitter') && file.name.endsWith('.zip')) { const [path, cleanup] = await createTempDir(); - this.logger.info(`Temp dir is ${path}`); + this.logger.debug(`Temp dir is ${path}`); const destPath = path + '/twitter.zip'; @@ -192,15 +194,13 @@ export class ImportNotesProcessorService { await fsp.writeFile(destPath, '', 'binary'); await this.downloadUrl(file.url, destPath); } catch (e) { // TODO: 何度か再試行 - if (e instanceof Error || typeof e === 'string') { - this.logger.error(e); - } + this.logger.error('Error importing notes:', e as Error); throw e; } const outputPath = path + '/twitter'; try { - this.logger.succ(`Unzipping to ${outputPath}`); + this.logger.debug(`Unzipping to ${outputPath}`); ZipReader.withDestinationPath(outputPath).viaBuffer(await fsp.readFile(destPath)); const unprocessedTweets = this.parseTwitterFile(await fsp.readFile(outputPath + '/data/tweets.js', 'utf-8')); @@ -214,7 +214,7 @@ export class ImportNotesProcessorService { } else if (type === 'Facebook' || file.name.startsWith('facebook-') && file.name.endsWith('.zip')) { const [path, cleanup] = await createTempDir(); - this.logger.info(`Temp dir is ${path}`); + this.logger.debug(`Temp dir is ${path}`); const destPath = path + '/facebook.zip'; @@ -222,15 +222,13 @@ export class ImportNotesProcessorService { await fsp.writeFile(destPath, '', 'binary'); await this.downloadUrl(file.url, destPath); } catch (e) { // TODO: 何度か再試行 - if (e instanceof Error || typeof e === 'string') { - this.logger.error(e); - } + this.logger.error('Error importing notes:', e as Error); throw e; } const outputPath = path + '/facebook'; try { - this.logger.succ(`Unzipping to ${outputPath}`); + this.logger.debug(`Unzipping to ${outputPath}`); ZipReader.withDestinationPath(outputPath).viaBuffer(await fsp.readFile(destPath)); const postsJson = await fsp.readFile(outputPath + '/your_activity_across_facebook/posts/your_posts__check_ins__photos_and_videos_1.json', 'utf-8'); const posts = JSON.parse(postsJson); @@ -247,7 +245,7 @@ export class ImportNotesProcessorService { } else if (file.name.endsWith('.zip')) { const [path, cleanup] = await createTempDir(); - this.logger.info(`Temp dir is ${path}`); + this.logger.debug(`Temp dir is ${path}`); const destPath = path + '/unknown.zip'; @@ -255,15 +253,13 @@ export class ImportNotesProcessorService { await fsp.writeFile(destPath, '', 'binary'); await this.downloadUrl(file.url, destPath); } catch (e) { // TODO: 何度か再試行 - if (e instanceof Error || typeof e === 'string') { - this.logger.error(e); - } + this.logger.error('Error importing notes:', e as Error); throw e; } const outputPath = path + '/unknown'; try { - this.logger.succ(`Unzipping to ${outputPath}`); + this.logger.debug(`Unzipping to ${outputPath}`); ZipReader.withDestinationPath(outputPath).viaBuffer(await fsp.readFile(destPath)); const isInstagram = type === 'Instagram' || fs.existsSync(outputPath + '/instagram_live') || fs.existsSync(outputPath + '/instagram_ads_and_businesses'); const isOutbox = type === 'Mastodon' || fs.existsSync(outputPath + '/outbox.json'); @@ -307,15 +303,13 @@ export class ImportNotesProcessorService { } else if (job.data.type === 'Misskey' || file.name.startsWith('notes-') && file.name.endsWith('.json')) { const [path, cleanup] = await createTemp(); - this.logger.info(`Temp dir is ${path}`); + this.logger.debug(`Temp dir is ${path}`); try { await fsp.writeFile(path, '', 'utf-8'); await this.downloadUrl(file.url, path); } catch (e) { // TODO: 何度か再試行 - if (e instanceof Error || typeof e === 'string') { - this.logger.error(e); - } + this.logger.error('Error importing notes:', e as Error); throw e; } @@ -326,7 +320,7 @@ export class ImportNotesProcessorService { cleanup(); } - this.logger.succ('Import jobs created'); + this.logger.debug('Import jobs created'); } @bindThis @@ -365,7 +359,7 @@ export class ImportNotesProcessorService { try { await this.downloadUrl(file.url, filePath); } catch (e) { // TODO: 何度か再試行 - this.logger.error(e instanceof Error ? e : new Error(e as string)); + this.logger.error('Error importing notes:', e as Error); } const driveFile = await this.driveService.addFile({ user: user, @@ -504,7 +498,7 @@ export class ImportNotesProcessorService { try { await this.downloadUrl(file.url, filePath); } catch (e) { // TODO: 何度か再試行 - this.logger.error(e instanceof Error ? e : new Error(e as string)); + this.logger.error('Error importing notes:', e as Error); } const driveFile = await this.driveService.addFile({ user: user, @@ -628,7 +622,7 @@ export class ImportNotesProcessorService { try { await this.downloadUrl(videos[0].url, filePath); } catch (e) { // TODO: 何度か再試行 - this.logger.error(e instanceof Error ? e : new Error(e as string)); + this.logger.error('Error importing notes:', e as Error); } const driveFile = await this.driveService.addFile({ user: user, @@ -653,7 +647,7 @@ export class ImportNotesProcessorService { try { await this.downloadUrl(file.media_url_https, filePath); } catch (e) { // TODO: 何度か再試行 - this.logger.error(e instanceof Error ? e : new Error(e as string)); + this.logger.error('Error importing notes:', e as Error); } const driveFile = await this.driveService.addFile({ @@ -673,7 +667,7 @@ export class ImportNotesProcessorService { const createdNote = await this.noteCreateService.import(user, { createdAt: date, reply: parentNote, text: text, files: files }); if (tweet.childNotes) this.queueService.createImportTweetsToDbJob(user, tweet.childNotes, createdNote.id); } catch (e) { - this.logger.warn(`Error: ${e}`); + this.logger.error('Error importing notes:', e as Error); } } diff --git a/packages/backend/src/queue/processors/ImportUserListsProcessorService.ts b/packages/backend/src/queue/processors/ImportUserListsProcessorService.ts index db9255b35d..482054e52f 100644 --- a/packages/backend/src/queue/processors/ImportUserListsProcessorService.ts +++ b/packages/backend/src/queue/processors/ImportUserListsProcessorService.ts @@ -15,6 +15,7 @@ import { UserListService } from '@/core/UserListService.js'; import { IdService } from '@/core/IdService.js'; import { UtilityService } from '@/core/UtilityService.js'; import { bindThis } from '@/decorators.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; import { QueueLoggerService } from '../QueueLoggerService.js'; import type * as Bull from 'bullmq'; import type { DbUserImportJobData } from '../types.js'; @@ -48,10 +49,9 @@ export class ImportUserListsProcessorService { @bindThis public async process(job: Bull.Job<DbUserImportJobData>): Promise<void> { - this.logger.info(`Importing user lists of ${job.data.user.id} ...`); - const user = await this.usersRepository.findOneBy({ id: job.data.user.id }); if (user == null) { + this.logger.debug(`Skip: user ${job.data.user.id} does not exist`); return; } @@ -59,9 +59,12 @@ export class ImportUserListsProcessorService { id: job.data.fileId, }); if (file == null) { + this.logger.debug(`Skip: file ${job.data.fileId} does not exist`); return; } + this.logger.info(`Importing user lists of ${job.data.user.id} ...`); + const csv = await this.downloadService.downloadTextFile(file.url); let linenum = 0; @@ -102,10 +105,10 @@ export class ImportUserListsProcessorService { this.userListService.addMember(target, list!, user); } catch (e) { - this.logger.warn(`Error in line:${linenum} ${e}`); + this.logger.warn(`Error in line:${linenum} ${renderInlineError(e)}`); } } - this.logger.succ('Imported'); + this.logger.debug('Imported'); } } diff --git a/packages/backend/src/queue/processors/InboxProcessorService.ts b/packages/backend/src/queue/processors/InboxProcessorService.ts index 9564724c62..5f82d558b3 100644 --- a/packages/backend/src/queue/processors/InboxProcessorService.ts +++ b/packages/backend/src/queue/processors/InboxProcessorService.ts @@ -21,7 +21,7 @@ import { ApDbResolverService } from '@/core/activitypub/ApDbResolverService.js'; import { StatusError } from '@/misc/status-error.js'; import { UtilityService } from '@/core/UtilityService.js'; import { ApPersonService } from '@/core/activitypub/models/ApPersonService.js'; -import { JsonLdService } from '@/core/activitypub/JsonLdService.js'; +import { isSigned, JsonLdService } from '@/core/activitypub/JsonLdService.js'; import { ApInboxService } from '@/core/activitypub/ApInboxService.js'; import { bindThis } from '@/decorators.js'; import { IdentifiableError } from '@/misc/identifiable-error.js'; @@ -31,6 +31,8 @@ import { SkApInboxLog } from '@/models/_.js'; import type { Config } from '@/config.js'; import { ApLogService, calculateDurationSince } from '@/core/ApLogService.js'; import { UpdateInstanceQueue } from '@/core/UpdateInstanceQueue.js'; +import { isRetryableError } from '@/misc/is-retryable-error.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; import { QueueLoggerService } from '../QueueLoggerService.js'; import type { InboxJobData } from '../types.js'; @@ -125,6 +127,14 @@ export class InboxProcessorService implements OnApplicationShutdown { return `Old keyId is no longer supported. ${keyIdLower}`; } + if (activity.actor as unknown == null || (Array.isArray(activity.actor) && activity.actor.length < 1)) { + return 'skip: activity has no actor'; + } + if (typeof(activity.actor) !== 'string' && typeof(activity.actor) !== 'object') { + return `skip: activity actor has invalid type: ${typeof(activity.actor)}`; + } + const actorId = getApId(activity.actor); + // HTTP-Signature keyIdを元にDBから取得 let authUser: { user: MiRemoteUser; @@ -134,26 +144,25 @@ export class InboxProcessorService implements OnApplicationShutdown { // keyIdでわからなければ、activity.actorを元にDBから取得 || activity.actorを元にリモートから取得 if (authUser == null) { try { - authUser = await this.apDbResolverService.getAuthUserFromApId(getApId(activity.actor)); + authUser = await this.apDbResolverService.getAuthUserFromApId(actorId); } catch (err) { // 対象が4xxならスキップ - if (err instanceof StatusError) { - if (!err.isRetryable) { - throw new Bull.UnrecoverableError(`skip: Ignored deleted actors on both ends ${activity.actor} - ${err.statusCode}`); - } - throw new Error(`Error in actor ${activity.actor} - ${err.statusCode}`); + if (!isRetryableError(err)) { + throw new Bull.UnrecoverableError(`skip: Ignored deleted actors on both ends ${actorId}`); } + + throw err; } } // それでもわからなければ終了 if (authUser == null) { - throw new Bull.UnrecoverableError(`skip: failed to resolve user ${getApId(activity.actor)}`); + throw new Bull.UnrecoverableError(`skip: failed to resolve user ${actorId}`); } // publicKey がなくても終了 if (authUser.key == null) { - throw new Bull.UnrecoverableError(`skip: failed to resolve user publicKey ${getApId(activity.actor)}`); + throw new Bull.UnrecoverableError(`skip: failed to resolve user publicKey ${actorId}`); } // HTTP-Signatureの検証 @@ -168,10 +177,10 @@ export class InboxProcessorService implements OnApplicationShutdown { } // また、signatureのsignerは、activity.actorと一致する必要がある - if (!httpSignatureValidated || authUser.user.uri !== getApId(activity.actor)) { + if (!httpSignatureValidated || authUser.user.uri !== actorId) { // 一致しなくても、でもLD-Signatureがありそうならそっちも見る - const ldSignature = activity.signature; - if (ldSignature) { + if (isSigned(activity)) { + const ldSignature = activity.signature; if (ldSignature.type !== 'RsaSignature2017') { throw new Bull.UnrecoverableError(`skip: unsupported LD-signature type ${ldSignature.type}`); } @@ -193,33 +202,30 @@ export class InboxProcessorService implements OnApplicationShutdown { throw new Bull.UnrecoverableError('skip: LD-SignatureのユーザーはpublicKeyを持っていませんでした'); } - const jsonLd = this.jsonLdService.use(); - // LD-Signature検証 - const verified = await jsonLd.verifyRsaSignature2017(activity, authUser.key.keyPem).catch(() => false); + const verified = await this.jsonLdService.verifyRsaSignature2017(activity, authUser.key.keyPem).catch(() => false); if (!verified) { throw new Bull.UnrecoverableError('skip: LD-Signatureの検証に失敗しました'); } // アクティビティを正規化 - delete activity.signature; + const copy = { ...activity, signature: undefined }; try { - activity = await jsonLd.compact(activity) as IActivity; + activity = await this.jsonLdService.compact(copy) as IActivity; } catch (e) { throw new Bull.UnrecoverableError(`skip: failed to compact activity: ${e}`); } // TODO: 元のアクティビティと非互換な形に正規化される場合は転送をスキップする // https://github.com/mastodon/mastodon/blob/664b0ca/app/services/activitypub/process_collection_service.rb#L24-L29 - activity.signature = ldSignature; // もう一度actorチェック - if (authUser.user.uri !== activity.actor) { - throw new Bull.UnrecoverableError(`skip: LD-Signature user(${authUser.user.uri}) !== activity.actor(${activity.actor})`); + if (authUser.user.uri !== actorId) { + throw new Bull.UnrecoverableError(`skip: LD-Signature user(${authUser.user.uri}) !== activity.actor(${actorId})`); } const ldHost = this.utilityService.extractDbHost(authUser.user.uri); if (!this.utilityService.isFederationAllowedHost(ldHost)) { - throw new Bull.UnrecoverableError(`Blocked request: ${ldHost}`); + throw new Bull.UnrecoverableError(`skip: request host is blocked: ${ldHost}`); } } else { throw new Bull.UnrecoverableError(`skip: http-signature verification failed and no LD-Signature. keyId=${signature.keyId}`); @@ -292,16 +298,8 @@ export class InboxProcessorService implements OnApplicationShutdown { } } - if (e instanceof StatusError && !e.isRetryable) { - return `skip: permanent error ${e.statusCode}`; - } - - if (e instanceof IdentifiableError && !e.isRetryable) { - if (e.message) { - return `skip: permanent error ${e.id}: ${e.message}`; - } else { - return `skip: permanent error ${e.id}`; - } + if (!isRetryableError(e)) { + return `skip: permanent error ${renderInlineError(e)}`; } throw e; diff --git a/packages/backend/src/queue/processors/ResyncChartsProcessorService.ts b/packages/backend/src/queue/processors/ResyncChartsProcessorService.ts index 0c47fdedb3..5b7a871af9 100644 --- a/packages/backend/src/queue/processors/ResyncChartsProcessorService.ts +++ b/packages/backend/src/queue/processors/ResyncChartsProcessorService.ts @@ -36,6 +36,6 @@ export class ResyncChartsProcessorService { await this.notesChart.resync(); await this.usersChart.resync(); - this.logger.succ('All charts successfully resynced.'); + this.logger.info('All charts successfully resynced.'); } } diff --git a/packages/backend/src/queue/processors/ScheduleNotePostProcessorService.ts b/packages/backend/src/queue/processors/ScheduleNotePostProcessorService.ts index d823d98ef1..73088f3312 100644 --- a/packages/backend/src/queue/processors/ScheduleNotePostProcessorService.ts +++ b/packages/backend/src/queue/processors/ScheduleNotePostProcessorService.ts @@ -12,6 +12,7 @@ import { DI } from '@/di-symbols.js'; import { NotificationService } from '@/core/NotificationService.js'; import { IdentifiableError } from '@/misc/identifiable-error.js'; import type { MiScheduleNoteType } from '@/models/NoteSchedule.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; import { QueueLoggerService } from '../QueueLoggerService.js'; import type * as Bull from 'bullmq'; import type { ScheduleNotePostJobData } from '../types.js'; @@ -129,10 +130,11 @@ export class ScheduleNotePostProcessorService { channel, }).catch(async (err: IdentifiableError) => { this.notificationService.createNotification(me.id, 'scheduledNoteFailed', { - reason: err.message, + reason: renderInlineError(err), }); await this.noteScheduleRepository.remove(data); - throw this.logger.error(`Schedule Note Failed Reason: ${err.message}`); + this.logger.error(`Scheduled note failed: ${renderInlineError(err)}`); + throw err; }); await this.noteScheduleRepository.remove(data); this.notificationService.createNotification(me.id, 'scheduledNotePosted', { diff --git a/packages/backend/src/queue/processors/SystemWebhookDeliverProcessorService.ts b/packages/backend/src/queue/processors/SystemWebhookDeliverProcessorService.ts index f6bef52684..f9fcd1e928 100644 --- a/packages/backend/src/queue/processors/SystemWebhookDeliverProcessorService.ts +++ b/packages/backend/src/queue/processors/SystemWebhookDeliverProcessorService.ts @@ -12,6 +12,7 @@ import type Logger from '@/logger.js'; import { HttpRequestService } from '@/core/HttpRequestService.js'; import { StatusError } from '@/misc/status-error.js'; import { bindThis } from '@/decorators.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; import { QueueLoggerService } from '../QueueLoggerService.js'; import { SystemWebhookDeliverJobData } from '../types.js'; @@ -63,21 +64,16 @@ export class SystemWebhookDeliverProcessorService { return 'Success'; } catch (res) { - this.logger.error(res as Error); + this.logger.error(`Failed to send webhook: ${renderInlineError(res)}`); this.systemWebhooksRepository.update({ id: job.data.webhookId }, { latestSentAt: new Date(), latestStatus: res instanceof StatusError ? res.statusCode : 1, }); - if (res instanceof StatusError) { + if (res instanceof StatusError && !res.isRetryable) { // 4xx - if (!res.isRetryable) { - throw new Bull.UnrecoverableError(`${res.statusCode} ${res.statusMessage}`); - } - - // 5xx etc. - throw new Error(`${res.statusCode} ${res.statusMessage}`); + throw new Bull.UnrecoverableError(`${res.statusCode} ${res.statusMessage}`); } else { // DNS error, socket error, timeout ... throw res; diff --git a/packages/backend/src/queue/processors/TickChartsProcessorService.ts b/packages/backend/src/queue/processors/TickChartsProcessorService.ts index fc8856a271..b4b8b1f205 100644 --- a/packages/backend/src/queue/processors/TickChartsProcessorService.ts +++ b/packages/backend/src/queue/processors/TickChartsProcessorService.ts @@ -62,6 +62,6 @@ export class TickChartsProcessorService { await this.perUserDriveChart.tick(false); await this.apRequestChart.tick(false); - this.logger.succ('All charts successfully ticked.'); + this.logger.info('All charts successfully ticked.'); } } diff --git a/packages/backend/src/queue/processors/UserWebhookDeliverProcessorService.ts b/packages/backend/src/queue/processors/UserWebhookDeliverProcessorService.ts index 9ec630ef70..0208ce6038 100644 --- a/packages/backend/src/queue/processors/UserWebhookDeliverProcessorService.ts +++ b/packages/backend/src/queue/processors/UserWebhookDeliverProcessorService.ts @@ -69,14 +69,9 @@ export class UserWebhookDeliverProcessorService { latestStatus: res instanceof StatusError ? res.statusCode : 1, }); - if (res instanceof StatusError) { + if (res instanceof StatusError && !res.isRetryable) { // 4xx - if (!res.isRetryable) { - throw new Bull.UnrecoverableError(`${res.statusCode} ${res.statusMessage}`); - } - - // 5xx etc. - throw new Error(`${res.statusCode} ${res.statusMessage}`); + throw new Bull.UnrecoverableError(`${res.statusCode} ${res.statusMessage}`); } else { // DNS error, socket error, timeout ... throw res; diff --git a/packages/backend/src/server/ActivityPubServerService.ts b/packages/backend/src/server/ActivityPubServerService.ts index 41beadb56d..27d25d2152 100644 --- a/packages/backend/src/server/ActivityPubServerService.ts +++ b/packages/backend/src/server/ActivityPubServerService.ts @@ -33,7 +33,7 @@ import type Logger from '@/logger.js'; import { LoggerService } from '@/core/LoggerService.js'; import { bindThis } from '@/decorators.js'; import { IActivity, IAnnounce, ICreate } from '@/core/activitypub/type.js'; -import { isQuote, isRenote } from '@/misc/is-renote.js'; +import { isPureRenote, isQuote, isRenote } from '@/misc/is-renote.js'; import * as Acct from '@/misc/acct.js'; import { CacheService } from '@/core/CacheService.js'; import type { FastifyInstance, FastifyRequest, FastifyReply, FastifyPluginOptions, FastifyBodyParser } from 'fastify'; @@ -571,7 +571,7 @@ export class ActivityPubServerService { const pinnedNotes = (await Promise.all(pinings.map(pining => this.notesRepository.findOneByOrFail({ id: pining.noteId })))) - .filter(note => !note.localOnly && ['public', 'home'].includes(note.visibility)); + .filter(note => !note.localOnly && ['public', 'home'].includes(note.visibility) && !isPureRenote(note)); const renderedNotes = await Promise.all(pinnedNotes.map(note => this.apRendererService.renderNote(note, user))); @@ -791,6 +791,10 @@ export class ActivityPubServerService { reply.header('Access-Control-Allow-Origin', '*'); reply.header('Access-Control-Expose-Headers', 'Vary'); + // Tell crawlers not to index AP endpoints. + // https://developers.google.com/search/docs/crawling-indexing/block-indexing + reply.header('X-Robots-Tag', 'noindex'); + /* tell any caching proxy that they should not cache these responses: we wouldn't want the proxy to return a 403 to someone presenting a valid signature, or return a cached @@ -838,6 +842,11 @@ export class ActivityPubServerService { return; } + // Boosts don't federate directly - they should only be referenced as an activity + if (isPureRenote(note)) { + return 404; + } + this.setResponseType(request, reply); const author = await this.usersRepository.findOneByOrFail({ id: note.userId }); diff --git a/packages/backend/src/server/FileServerService.ts b/packages/backend/src/server/FileServerService.ts index 4ef5539cff..0910c0d36b 100644 --- a/packages/backend/src/server/FileServerService.ts +++ b/packages/backend/src/server/FileServerService.ts @@ -32,6 +32,7 @@ import { getIpHash } from '@/misc/get-ip-hash.js'; import { AuthenticateService } from '@/server/api/AuthenticateService.js'; import { SkRateLimiterService } from '@/server/SkRateLimiterService.js'; import { Keyed, RateLimit, sendRateLimitHeaders } from '@/misc/rate-limit-utils.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; import type { FastifyInstance, FastifyRequest, FastifyReply, FastifyPluginOptions } from 'fastify'; const _filename = fileURLToPath(import.meta.url); @@ -69,6 +70,10 @@ export class FileServerService { fastify.addHook('onRequest', (request, reply, done) => { reply.header('Content-Security-Policy', 'default-src \'none\'; img-src \'self\'; media-src \'self\'; style-src \'unsafe-inline\''); reply.header('Access-Control-Allow-Origin', '*'); + + // Tell crawlers not to index files endpoints. + // https://developers.google.com/search/docs/crawling-indexing/block-indexing + reply.header('X-Robots-Tag', 'noindex'); done(); }); @@ -120,7 +125,7 @@ export class FileServerService { @bindThis private async errorHandler(request: FastifyRequest<{ Params?: { [x: string]: any }; Querystring?: { [x: string]: any }; }>, reply: FastifyReply, err?: any) { - this.logger.error(`${err}`); + this.logger.error(`Unhandled error in file server: ${renderInlineError(err)}`); reply.header('Cache-Control', 'max-age=300'); @@ -353,7 +358,7 @@ export class FileServerService { if (!request.headers['user-agent']) { throw new StatusError('User-Agent is required', 400, 'User-Agent is required'); } else if (request.headers['user-agent'].toLowerCase().indexOf('misskey/') !== -1) { - throw new StatusError('Refusing to proxy a request from another proxy', 403, 'Proxy is recursive'); + throw new StatusError(`Refusing to proxy recursive request to ${url} (from user-agent ${request.headers['user-agent']})`, 403, 'Proxy is recursive'); } // Create temp file @@ -383,7 +388,7 @@ export class FileServerService { ) { if (!isConvertibleImage) { // 画像でないなら404でお茶を濁す - throw new StatusError('Unexpected mime', 404); + throw new StatusError(`Unexpected non-convertible mime: ${file.mime}`, 404, 'Unexpected mime'); } } @@ -447,7 +452,7 @@ export class FileServerService { } else if (file.mime === 'image/svg+xml') { image = this.imageProcessingService.convertToWebpStream(file.path, 2048, 2048); } else if (!file.mime.startsWith('image/') || !FILE_TYPE_BROWSERSAFE.includes(file.mime)) { - throw new StatusError('Rejected type', 403, 'Rejected type'); + throw new StatusError(`Blocked mime type: ${file.mime}`, 403, 'Blocked mime type'); } if (!image) { @@ -521,7 +526,7 @@ export class FileServerService { > { if (url.startsWith(`${this.config.url}/files/`)) { const key = url.replace(`${this.config.url}/files/`, '').split('/').shift(); - if (!key) throw new StatusError('Invalid File Key', 400, 'Invalid File Key'); + if (!key) throw new StatusError(`Invalid file URL ${url}`, 400, 'Invalid file url'); return await this.getFileFromKey(key); } diff --git a/packages/backend/src/server/ServerService.ts b/packages/backend/src/server/ServerService.ts index 2d20aa1222..77b4519570 100644 --- a/packages/backend/src/server/ServerService.ts +++ b/packages/backend/src/server/ServerService.ts @@ -21,6 +21,7 @@ import { genIdenticon } from '@/misc/gen-identicon.js'; import { UserEntityService } from '@/core/entities/UserEntityService.js'; import { LoggerService } from '@/core/LoggerService.js'; import { bindThis } from '@/decorators.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; import { ActivityPubServerService } from './ActivityPubServerService.js'; import { NodeinfoServerService } from './NodeinfoServerService.js'; import { ApiServerService } from './api/ApiServerService.js'; @@ -277,7 +278,7 @@ export class ServerService implements OnApplicationShutdown { this.logger.error(`Port ${this.config.port} is already in use by another process.`); break; default: - this.logger.error(err); + this.logger.error(`Unhandled error in server: ${renderInlineError(err)}`); break; } diff --git a/packages/backend/src/server/SkRateLimiterService.ts b/packages/backend/src/server/SkRateLimiterService.ts index 8978318045..35e87b0fe8 100644 --- a/packages/backend/src/server/SkRateLimiterService.ts +++ b/packages/backend/src/server/SkRateLimiterService.ts @@ -389,7 +389,7 @@ function createLimitKey(limit: ParsedLimit, actor: string, value: string): strin return `rl_${actor}_${limit.key}_${value}`; } -class ConflictError extends Error {} +export class ConflictError extends Error {} interface LimitCounter { timestamp: number; diff --git a/packages/backend/src/server/api/ApiCallService.ts b/packages/backend/src/server/api/ApiCallService.ts index 0d2dafd556..66d968224a 100644 --- a/packages/backend/src/server/api/ApiCallService.ts +++ b/packages/backend/src/server/api/ApiCallService.ts @@ -20,12 +20,14 @@ import { RoleService } from '@/core/RoleService.js'; import type { Config } from '@/config.js'; import { sendRateLimitHeaders } from '@/misc/rate-limit-utils.js'; import { SkRateLimiterService } from '@/server/SkRateLimiterService.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; import { ApiError } from './error.js'; import { ApiLoggerService } from './ApiLoggerService.js'; import { AuthenticateService, AuthenticationError } from './AuthenticateService.js'; import type { FastifyRequest, FastifyReply } from 'fastify'; import type { OnApplicationShutdown } from '@nestjs/common'; import type { IEndpointMeta, IEndpoint } from './endpoints.js'; +import { renderFullError } from '@/misc/render-full-error.js'; const accessDenied = { message: 'Access denied.', @@ -100,26 +102,26 @@ export class ApiCallService implements OnApplicationShutdown { throw err; } else { const errId = randomUUID(); - this.logger.error(`Internal error occurred in ${ep.name}: ${err.message}`, { - ep: ep.name, - ps: data, - e: { - message: err.message, - code: err.name, - stack: err.stack, - id: errId, - }, + const fullError = renderFullError(err); + const message = typeof(fullError) === 'string' + ? `Internal error id=${errId} occurred in ${ep.name}: ${fullError}` + : `Internal error id=${errId} occurred in ${ep.name}:`; + const data = typeof(fullError) === 'object' + ? { e: fullError } + : {}; + this.logger.error(message, { + user: userId ?? '<unauthenticated>', + ...data, }); if (this.config.sentryForBackend) { - Sentry.captureMessage(`Internal error occurred in ${ep.name}: ${err.message}`, { + Sentry.captureMessage(`Internal error occurred in ${ep.name}: ${renderInlineError(err)}`, { level: 'error', user: { id: userId, }, extra: { ep: ep.name, - ps: data, e: { message: err.message, code: err.name, @@ -146,6 +148,10 @@ export class ApiCallService implements OnApplicationShutdown { request: FastifyRequest<{ Body: Record<string, unknown> | undefined, Querystring: Record<string, unknown> }>, reply: FastifyReply, ): void { + // Tell crawlers not to index API endpoints. + // https://developers.google.com/search/docs/crawling-indexing/block-indexing + reply.header('X-Robots-Tag', 'noindex'); + const body = request.method === 'GET' ? request.query : request.body; @@ -344,14 +350,14 @@ export class ApiCallService implements OnApplicationShutdown { } if (ep.meta.requireCredential || ep.meta.requireModerator || ep.meta.requireAdmin) { - if (user == null) { + if (user == null && ep.meta.requireCredential !== 'optional') { throw new ApiError({ message: 'Credential required.', code: 'CREDENTIAL_REQUIRED', id: '1384574d-a912-4b81-8601-c7b1c4085df1', httpStatusCode: 401, }); - } else if (user!.isSuspended) { + } else if (user?.isSuspended) { throw new ApiError({ message: 'Your account has been suspended.', code: 'YOUR_ACCOUNT_SUSPENDED', @@ -372,8 +378,8 @@ export class ApiCallService implements OnApplicationShutdown { } } - if ((ep.meta.requireModerator || ep.meta.requireAdmin) && (this.meta.rootUserId !== user!.id)) { - const myRoles = await this.roleService.getUserRoles(user!.id); + if ((ep.meta.requireModerator || ep.meta.requireAdmin) && (this.meta.rootUserId !== user?.id)) { + const myRoles = user ? await this.roleService.getUserRoles(user) : []; if (ep.meta.requireModerator && !myRoles.some(r => r.isModerator || r.isAdministrator)) { throw new ApiError({ message: 'You are not assigned to a moderator role.', @@ -392,9 +398,9 @@ export class ApiCallService implements OnApplicationShutdown { } } - if (ep.meta.requiredRolePolicy != null && (this.meta.rootUserId !== user!.id)) { - const myRoles = await this.roleService.getUserRoles(user!.id); - const policies = await this.roleService.getUserPolicies(user!.id); + if (ep.meta.requiredRolePolicy != null && (this.meta.rootUserId !== user?.id)) { + const myRoles = user ? await this.roleService.getUserRoles(user) : []; + const policies = await this.roleService.getUserPolicies(user ?? null); if (!policies[ep.meta.requiredRolePolicy] && !myRoles.some(r => r.isAdministrator)) { throw new ApiError({ message: 'You are not assigned to a required role.', @@ -418,7 +424,7 @@ export class ApiCallService implements OnApplicationShutdown { // Cast non JSON input if ((ep.meta.requireFile || request.method === 'GET') && ep.params.properties) { for (const k of Object.keys(ep.params.properties)) { - const param = ep.params.properties![k]; + const param = ep.params.properties[k]; if (['boolean', 'number', 'integer'].includes(param.type ?? '') && typeof data[k] === 'string') { try { data[k] = JSON.parse(data[k]); diff --git a/packages/backend/src/server/api/GetterService.ts b/packages/backend/src/server/api/GetterService.ts index 419017aaf4..f2850e6258 100644 --- a/packages/backend/src/server/api/GetterService.ts +++ b/packages/backend/src/server/api/GetterService.ts @@ -36,7 +36,7 @@ export class GetterService { const note = await this.notesRepository.findOneBy({ id: noteId }); if (note == null) { - throw new IdentifiableError('9725d0ce-ba28-4dde-95a7-2cbb2c15de24', 'No such note.'); + throw new IdentifiableError('9725d0ce-ba28-4dde-95a7-2cbb2c15de24', `Note ${noteId} does not exist`); } return note; @@ -47,7 +47,7 @@ export class GetterService { const note = await this.notesRepository.findOne({ where: { id: noteId }, relations: ['user'] }); if (note == null) { - throw new IdentifiableError('9725d0ce-ba28-4dde-95a7-2cbb2c15de24', 'No such note.'); + throw new IdentifiableError('9725d0ce-ba28-4dde-95a7-2cbb2c15de24', `Note ${noteId} does not exist`); } return note; @@ -59,7 +59,7 @@ export class GetterService { @bindThis public async getEdits(noteId: MiNote['id']) { const edits = await this.noteEditRepository.findBy({ noteId: noteId }).catch(() => { - throw new IdentifiableError('9725d0ce-ba28-4dde-95a7-2cbb2c15de24', 'No such note.'); + throw new IdentifiableError('9725d0ce-ba28-4dde-95a7-2cbb2c15de24', `Note ${noteId} does not exist`); }); return edits; @@ -73,7 +73,7 @@ export class GetterService { const user = await this.usersRepository.findOneBy({ id: userId }); if (user == null) { - throw new IdentifiableError('15348ddd-432d-49c2-8a5a-8069753becff', 'No such user.'); + throw new IdentifiableError('15348ddd-432d-49c2-8a5a-8069753becff', `User ${userId} does not exist`); } return user as MiLocalUser | MiRemoteUser; diff --git a/packages/backend/src/server/api/SigninApiService.ts b/packages/backend/src/server/api/SigninApiService.ts index 7f371ea309..a53fec88d0 100644 --- a/packages/backend/src/server/api/SigninApiService.ts +++ b/packages/backend/src/server/api/SigninApiService.ts @@ -205,37 +205,37 @@ export class SigninApiService { if (process.env.NODE_ENV !== 'test') { if (this.meta.enableHcaptcha && this.meta.hcaptchaSecretKey) { await this.captchaService.verifyHcaptcha(this.meta.hcaptchaSecretKey, body['hcaptcha-response']).catch(err => { - throw new FastifyReplyError(400, err); + throw new FastifyReplyError(400, String(err), err); }); } if (this.meta.enableMcaptcha && this.meta.mcaptchaSecretKey && this.meta.mcaptchaSitekey && this.meta.mcaptchaInstanceUrl) { await this.captchaService.verifyMcaptcha(this.meta.mcaptchaSecretKey, this.meta.mcaptchaSitekey, this.meta.mcaptchaInstanceUrl, body['m-captcha-response']).catch(err => { - throw new FastifyReplyError(400, err); + throw new FastifyReplyError(400, String(err), err); }); } if (this.meta.enableFC && this.meta.fcSecretKey) { await this.captchaService.verifyFriendlyCaptcha(this.meta.fcSecretKey, body['frc-captcha-solution']).catch(err => { - throw new FastifyReplyError(400, err); + throw new FastifyReplyError(400, String(err), err); }); } if (this.meta.enableRecaptcha && this.meta.recaptchaSecretKey) { await this.captchaService.verifyRecaptcha(this.meta.recaptchaSecretKey, body['g-recaptcha-response']).catch(err => { - throw new FastifyReplyError(400, err); + throw new FastifyReplyError(400, String(err), err); }); } if (this.meta.enableTurnstile && this.meta.turnstileSecretKey) { await this.captchaService.verifyTurnstile(this.meta.turnstileSecretKey, body['turnstile-response']).catch(err => { - throw new FastifyReplyError(400, err); + throw new FastifyReplyError(400, String(err), err); }); } if (this.meta.enableTestcaptcha) { await this.captchaService.verifyTestcaptcha(body['testcaptcha-response']).catch(err => { - throw new FastifyReplyError(400, err); + throw new FastifyReplyError(400, String(err), err); }); } } diff --git a/packages/backend/src/server/api/SigninWithPasskeyApiService.ts b/packages/backend/src/server/api/SigninWithPasskeyApiService.ts index f84f50523b..38886f8876 100644 --- a/packages/backend/src/server/api/SigninWithPasskeyApiService.ts +++ b/packages/backend/src/server/api/SigninWithPasskeyApiService.ts @@ -128,7 +128,7 @@ export class SigninWithPasskeyApiService { try { authorizedUserId = await this.webAuthnService.verifySignInWithPasskeyAuthentication(context, credential); } catch (err) { - this.logger.warn(`Passkey challenge Verify error! : ${err}`); + this.logger.warn('Passkey challenge verify error:', err as Error); const errorId = (err as IdentifiableError).id; return error(403, { id: errorId, diff --git a/packages/backend/src/server/api/SignupApiService.ts b/packages/backend/src/server/api/SignupApiService.ts index cb71047a24..81e3a5b706 100644 --- a/packages/backend/src/server/api/SignupApiService.ts +++ b/packages/backend/src/server/api/SignupApiService.ts @@ -83,37 +83,37 @@ export class SignupApiService { if (process.env.NODE_ENV !== 'test') { if (this.meta.enableHcaptcha && this.meta.hcaptchaSecretKey) { await this.captchaService.verifyHcaptcha(this.meta.hcaptchaSecretKey, body['hcaptcha-response']).catch(err => { - throw new FastifyReplyError(400, err); + throw new FastifyReplyError(400, String(err), err); }); } if (this.meta.enableMcaptcha && this.meta.mcaptchaSecretKey && this.meta.mcaptchaSitekey && this.meta.mcaptchaInstanceUrl) { await this.captchaService.verifyMcaptcha(this.meta.mcaptchaSecretKey, this.meta.mcaptchaSitekey, this.meta.mcaptchaInstanceUrl, body['m-captcha-response']).catch(err => { - throw new FastifyReplyError(400, err); + throw new FastifyReplyError(400, String(err), err); }); } if (this.meta.enableRecaptcha && this.meta.recaptchaSecretKey) { await this.captchaService.verifyRecaptcha(this.meta.recaptchaSecretKey, body['g-recaptcha-response']).catch(err => { - throw new FastifyReplyError(400, err); + throw new FastifyReplyError(400, String(err), err); }); } if (this.meta.enableTurnstile && this.meta.turnstileSecretKey) { await this.captchaService.verifyTurnstile(this.meta.turnstileSecretKey, body['turnstile-response']).catch(err => { - throw new FastifyReplyError(400, err); + throw new FastifyReplyError(400, String(err), err); }); } if (this.meta.enableFC && this.meta.fcSecretKey) { await this.captchaService.verifyFriendlyCaptcha(this.meta.fcSecretKey, body['frc-captcha-solution']).catch(err => { - throw new FastifyReplyError(400, err); + throw new FastifyReplyError(400, String(err), err); }); } if (this.meta.enableTestcaptcha) { await this.captchaService.verifyTestcaptcha(body['testcaptcha-response']).catch(err => { - throw new FastifyReplyError(400, err); + throw new FastifyReplyError(400, String(err), err); }); } } @@ -287,7 +287,7 @@ export class SignupApiService { token: secret, }; } catch (err) { - throw new FastifyReplyError(400, typeof err === 'string' ? err : (err as Error).toString()); + throw new FastifyReplyError(400, String(err), err); } } } @@ -356,7 +356,7 @@ export class SignupApiService { return this.signinService.signin(request, reply, account as MiLocalUser); } catch (err) { - throw new FastifyReplyError(400, typeof err === 'string' ? err : (err as Error).toString()); + throw new FastifyReplyError(400, String(err), err); } } } diff --git a/packages/backend/src/server/api/endpoints.ts b/packages/backend/src/server/api/endpoints.ts index 0ba041c536..c7d884cce1 100644 --- a/packages/backend/src/server/api/endpoints.ts +++ b/packages/backend/src/server/api/endpoints.ts @@ -92,7 +92,7 @@ export type IEndpointMeta = (Omit<IEndpointMetaBase, 'requireCrential' | 'requir }) | (Omit<IEndpointMetaBase, 'secure'> & { secure: true, }) | (Omit<IEndpointMetaBase, 'requireCredential' | 'kind'> & { - requireCredential: true, + requireCredential: true | 'optional', kind: (typeof permissions)[number], }) | (Omit<IEndpointMetaBase, 'requireModerator' | 'kind'> & { requireModerator: true, diff --git a/packages/backend/src/server/api/endpoints/admin/abuse-user-reports.ts b/packages/backend/src/server/api/endpoints/admin/abuse-user-reports.ts index 0dbfaae054..b8200c09aa 100644 --- a/packages/backend/src/server/api/endpoints/admin/abuse-user-reports.ts +++ b/packages/backend/src/server/api/endpoints/admin/abuse-user-reports.ts @@ -69,6 +69,11 @@ export const meta = { nullable: false, optional: false, ref: 'UserDetailedNotMe', }, + targetInstance: { + type: 'object', + nullable: true, optional: false, + ref: 'FederationInstance', + }, assignee: { type: 'object', nullable: true, optional: false, @@ -115,7 +120,15 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- private queryService: QueryService, ) { super(meta, paramDef, async (ps, me) => { - const query = this.queryService.makePaginationQuery(this.abuseUserReportsRepository.createQueryBuilder('report'), ps.sinceId, ps.untilId); + const query = this.queryService.makePaginationQuery(this.abuseUserReportsRepository.createQueryBuilder('report'), ps.sinceId, ps.untilId) + .leftJoinAndSelect('report.targetUser', 'targetUser') + .leftJoinAndSelect('targetUser.userProfile', 'targetUserProfile') + .leftJoinAndSelect('report.targetUserInstance', 'targetUserInstance') + .leftJoinAndSelect('report.reporter', 'reporter') + .leftJoinAndSelect('reporter.userProfile', 'reporterProfile') + .leftJoinAndSelect('report.assignee', 'assignee') + .leftJoinAndSelect('assignee.userProfile', 'assigneeProfile') + ; switch (ps.state) { case 'resolved': query.andWhere('report.resolved = TRUE'); break; @@ -134,7 +147,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- const reports = await query.limit(ps.limit).getMany(); - return await this.abuseUserReportEntityService.packMany(reports); + return await this.abuseUserReportEntityService.packMany(reports, me); }); } } diff --git a/packages/backend/src/server/api/endpoints/admin/nsfw-user.ts b/packages/backend/src/server/api/endpoints/admin/nsfw-user.ts index 194e793eda..f6c4f0b635 100644 --- a/packages/backend/src/server/api/endpoints/admin/nsfw-user.ts +++ b/packages/backend/src/server/api/endpoints/admin/nsfw-user.ts @@ -47,7 +47,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- alwaysMarkNsfw: true, }); - await this.cacheService.userProfileCache.refresh(ps.userId); + await this.cacheService.userProfileCache.delete(ps.userId); }); } } diff --git a/packages/backend/src/server/api/endpoints/admin/relays/add.ts b/packages/backend/src/server/api/endpoints/admin/relays/add.ts index 129f69aca9..4644a069ee 100644 --- a/packages/backend/src/server/api/endpoints/admin/relays/add.ts +++ b/packages/backend/src/server/api/endpoints/admin/relays/add.ts @@ -68,11 +68,8 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- private readonly moderationLogService: ModerationLogService, ) { super(meta, paramDef, async (ps, me) => { - try { - if (new URL(ps.inbox).protocol !== 'https:') throw new Error('https only'); - } catch { - throw new ApiError(meta.errors.invalidUrl); - } + if (!URL.canParse(ps.inbox)) throw new ApiError(meta.errors.invalidUrl); + if (new URL(ps.inbox).protocol !== 'https:') throw new ApiError(meta.errors.invalidUrl); await this.moderationLogService.log(me, 'addRelay', { inbox: ps.inbox, diff --git a/packages/backend/src/server/api/endpoints/admin/show-user.ts b/packages/backend/src/server/api/endpoints/admin/show-user.ts index 1579719246..6f0081f1f7 100644 --- a/packages/backend/src/server/api/endpoints/admin/show-user.ts +++ b/packages/backend/src/server/api/endpoints/admin/show-user.ts @@ -122,6 +122,10 @@ export const meta = { type: 'boolean', optional: false, nullable: false, }, + isAdministrator: { + type: 'boolean', + optional: false, nullable: false, + }, isSystem: { type: 'boolean', optional: false, nullable: false, @@ -217,6 +221,10 @@ export const meta = { }, }, }, + signupReason: { + type: 'string', + optional: false, nullable: true, + }, }, }, } as const; @@ -257,6 +265,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- } const isModerator = await this.roleService.isModerator(user); + const isAdministrator = await this.roleService.isAdministrator(user); const isSilenced = user.isSilenced || !(await this.roleService.getUserPolicies(user.id)).canPublicNote; const _me = await this.usersRepository.findOneByOrFail({ id: me.id }); @@ -289,6 +298,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- mutedInstances: profile.mutedInstances, notificationRecieveConfig: profile.notificationRecieveConfig, isModerator: isModerator, + isAdministrator: isAdministrator, isSystem: isSystemAccount(user), isSilenced: isSilenced, isSuspended: user.isSuspended, diff --git a/packages/backend/src/server/api/endpoints/admin/update-meta.ts b/packages/backend/src/server/api/endpoints/admin/update-meta.ts index 7c3d485a0f..4970d28cfa 100644 --- a/packages/backend/src/server/api/endpoints/admin/update-meta.ts +++ b/packages/backend/src/server/api/endpoints/admin/update-meta.ts @@ -778,9 +778,29 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- const after = await this.metaService.fetch(true); this.moderationLogService.log(me, 'updateServerSettings', { - before, - after, + before: sanitize(before), + after: sanitize(after), }); }); } } + +function sanitize(meta: Partial<MiMeta>): Partial<MiMeta> { + return { + ...meta, + hcaptchaSecretKey: '<redacted>', + mcaptchaSecretKey: '<redacted>', + recaptchaSecretKey: '<redacted>', + turnstileSecretKey: '<redacted>', + fcSecretKey: '<redacted>', + smtpPass: '<redacted>', + swPrivateKey: '<redacted>', + objectStorageAccessKey: '<redacted>', + objectStorageSecretKey: '<redacted>', + deeplAuthKey: '<redacted>', + libreTranslateKey: '<redacted>', + verifymailAuthKey: '<redacted>', + truemailAuthKey: '<redacted>', + }; +} + diff --git a/packages/backend/src/server/api/endpoints/antennas/notes.ts b/packages/backend/src/server/api/endpoints/antennas/notes.ts index b90ba6aa0d..e975b9ad0f 100644 --- a/packages/backend/src/server/api/endpoints/antennas/notes.ts +++ b/packages/backend/src/server/api/endpoints/antennas/notes.ts @@ -14,6 +14,7 @@ import { IdService } from '@/core/IdService.js'; import { FanoutTimelineService } from '@/core/FanoutTimelineService.js'; import { GlobalEventService } from '@/core/GlobalEventService.js'; import { trackPromise } from '@/misc/promise-tracker.js'; +import ActiveUsersChart from '@/core/chart/charts/active-users.js'; import { ApiError } from '../../error.js'; export const meta = { @@ -75,6 +76,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- private queryService: QueryService, private fanoutTimelineService: FanoutTimelineService, private globalEventService: GlobalEventService, + private readonly activeUsersChart: ActiveUsersChart, ) { super(meta, paramDef, async (ps, me) => { const untilId = ps.untilId ?? (ps.untilDate ? this.idService.gen(ps.untilDate!) : null); @@ -106,7 +108,8 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- return []; } - const query = this.notesRepository.createQueryBuilder('note') + const query = this.queryService.makePaginationQuery(this.notesRepository.createQueryBuilder('note'), + ps.sinceId, ps.untilId) .where('note.id IN (:...noteIds)', { noteIds: noteIds }) .innerJoinAndSelect('note.user', 'user') .leftJoinAndSelect('note.reply', 'reply') @@ -121,13 +124,13 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- this.queryService.generateVisibilityQuery(query, me); this.queryService.generateMutedUserQueryForNotes(query, me); this.queryService.generateBlockedUserQueryForNotes(query, me); + this.queryService.generateMutedUserRenotesQueryForNotes(query, me); const notes = await query.getMany(); - if (sinceId != null && untilId == null) { - notes.sort((a, b) => a.id < b.id ? -1 : 1); - } else { - notes.sort((a, b) => a.id > b.id ? -1 : 1); - } + + process.nextTick(() => { + this.activeUsersChart.read(me); + }); return await this.noteEntityService.packMany(notes, me); }); diff --git a/packages/backend/src/server/api/endpoints/ap/get.ts b/packages/backend/src/server/api/endpoints/ap/get.ts index 14286bc23e..e3e68b50af 100644 --- a/packages/backend/src/server/api/endpoints/ap/get.ts +++ b/packages/backend/src/server/api/endpoints/ap/get.ts @@ -3,10 +3,17 @@ * SPDX-License-Identifier: AGPL-3.0-only */ -import { Injectable } from '@nestjs/common'; +import { Inject, Injectable } from '@nestjs/common'; import ms from 'ms'; import { Endpoint } from '@/server/api/endpoint-base.js'; import { ApResolverService } from '@/core/activitypub/ApResolverService.js'; +import { isCollectionOrOrderedCollection, isOrderedCollection, isOrderedCollectionPage } from '@/core/activitypub/type.js'; +import { ApiError } from '@/server/api/error.js'; +import { CacheService } from '@/core/CacheService.js'; +import { UserEntityService } from '@/core/entities/UserEntityService.js'; +import { DI } from '@/di-symbols.js'; +import type { NotesRepository } from '@/models/_.js'; +import { NoteEntityService } from '@/core/entities/NoteEntityService.js'; export const meta = { tags: ['federation'], @@ -21,6 +28,16 @@ export const meta = { }, errors: { + noInputSpecified: { + message: 'uri, userId, or noteId must be specified.', + code: 'NO_INPUT_SPECIFIED', + id: 'b43ff2a7-e7a2-4237-ad7f-7b079563c09e', + }, + multipleInputsSpecified: { + message: 'Only one of uri, userId, or noteId can be specified', + code: 'MULTIPLE_INPUTS_SPECIFIED', + id: 'f1aa27ed-8f20-44f3-a92a-fe073c8ca52b', + }, }, res: { @@ -32,19 +49,57 @@ export const meta = { export const paramDef = { type: 'object', properties: { - uri: { type: 'string' }, + uri: { type: 'string', nullable: true }, + userId: { type: 'string', format: 'misskey:id', nullable: true }, + noteId: { type: 'string', format: 'misskey:id', nullable: true }, + expandCollectionItems: { type: 'boolean' }, + expandCollectionLimit: { type: 'integer', nullable: true }, + allowAnonymous: { type: 'boolean' }, }, - required: ['uri'], } as const; @Injectable() export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-disable-line import/no-default-export constructor( + @Inject(DI.notesRepository) + private readonly notesRepository: NotesRepository, + + private readonly cacheService: CacheService, + private readonly userEntityService: UserEntityService, + private readonly noteEntityService: NoteEntityService, private apResolverService: ApResolverService, ) { - super(meta, paramDef, async (ps, me) => { + super(meta, paramDef, async (ps) => { + if (ps.uri && ps.userId && ps.noteId) { + throw new ApiError(meta.errors.multipleInputsSpecified); + } + + let uri: string; + if (ps.uri) { + uri = ps.uri; + } else if (ps.userId) { + const user = await this.cacheService.findUserById(ps.userId); + uri = user.uri ?? this.userEntityService.genLocalUserUri(ps.userId); + } else if (ps.noteId) { + const note = await this.notesRepository.findOneByOrFail({ id: ps.noteId }); + uri = note.uri ?? this.noteEntityService.genLocalNoteUri(ps.noteId); + } else { + throw new ApiError(meta.errors.noInputSpecified); + } + const resolver = this.apResolverService.createResolver(); - const object = await resolver.resolve(ps.uri); + const object = await resolver.resolve(uri, ps.allowAnonymous ?? false); + + if (ps.expandCollectionItems && isCollectionOrOrderedCollection(object)) { + const items = await resolver.resolveCollectionItems(object, ps.expandCollectionLimit, ps.allowAnonymous ?? false); + + if (isOrderedCollection(object) || isOrderedCollectionPage(object)) { + object.orderedItems = items; + } else { + object.items = items; + } + } + return object; }); } diff --git a/packages/backend/src/server/api/endpoints/ap/show.ts b/packages/backend/src/server/api/endpoints/ap/show.ts index d69850515c..d631b002cc 100644 --- a/packages/backend/src/server/api/endpoints/ap/show.ts +++ b/packages/backend/src/server/api/endpoints/ap/show.ts @@ -173,6 +173,10 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- case '09d79f9e-64f1-4316-9cfa-e75c4d091574': throw new ApiError(meta.errors.federationNotAllowed); case '72180409-793c-4973-868e-5a118eb5519b': + case 'd09dc850-b76c-4f45-875a-7389339d78b8': + case 'dc110060-a5f2-461d-808b-39c62702ca64': + case '45793ab7-7648-4886-b503-429f8a0d0f73': + case '4bf8f36b-4d33-4ac9-ad76-63fa11f354e9': throw new ApiError(meta.errors.responseInvalid); // resolveLocal diff --git a/packages/backend/src/server/api/endpoints/channels/timeline.ts b/packages/backend/src/server/api/endpoints/channels/timeline.ts index 6336f43e9f..fa5b948eca 100644 --- a/packages/backend/src/server/api/endpoints/channels/timeline.ts +++ b/packages/backend/src/server/api/endpoints/channels/timeline.ts @@ -96,7 +96,11 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- throw new ApiError(meta.errors.noSuchChannel); } - if (me) this.activeUsersChart.read(me); + if (me) { + process.nextTick(() => { + this.activeUsersChart.read(me); + }); + } if (!this.serverSettings.enableFanoutTimeline) { return await this.noteEntityService.packMany(await this.getFromDb({ untilId, sinceId, limit: ps.limit, channelId: channel.id, withFiles: ps.withFiles, withRenotes: ps.withRenotes }, me), me); @@ -135,29 +139,28 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- .leftJoinAndSelect('note.renote', 'renote') .leftJoinAndSelect('reply.user', 'replyUser') .leftJoinAndSelect('renote.user', 'renoteUser') - .leftJoinAndSelect('note.channel', 'channel'); + .leftJoinAndSelect('note.channel', 'channel') + .limit(ps.limit); + this.queryService.generateVisibilityQuery(query, me); this.queryService.generateBlockedHostQueryForNote(query); + this.queryService.generateSilencedUserQueryForNotes(query, me); if (me) { this.queryService.generateMutedUserQueryForNotes(query, me); this.queryService.generateBlockedUserQueryForNotes(query, me); } - if (ps.withRenotes === false) { - query.andWhere(new Brackets(qb => { - qb.orWhere('note.renoteId IS NULL'); - qb.orWhere(new Brackets(qb => { - qb.orWhere('note.text IS NOT NULL'); - qb.orWhere('note.fileIds != \'{}\''); - })); - })); - } - if (ps.withFiles) { query.andWhere('note.fileIds != \'{}\''); } + + if (!ps.withRenotes) { + this.queryService.generateExcludedRenotesQueryForNotes(query); + } else if (me) { + this.queryService.generateMutedUserRenotesQueryForNotes(query, me); + } //#endregion - return await query.limit(ps.limit).getMany(); + return await query.getMany(); } } diff --git a/packages/backend/src/server/api/endpoints/charts/active-users.ts b/packages/backend/src/server/api/endpoints/charts/active-users.ts index dcdcf46d0b..9f5064fe83 100644 --- a/packages/backend/src/server/api/endpoints/charts/active-users.ts +++ b/packages/backend/src/server/api/endpoints/charts/active-users.ts @@ -17,11 +17,11 @@ export const meta = { allowGet: true, cacheSec: 60 * 60, - // Burst up to 100, then 2/sec average + // Burst up to 200, then 5/sec average limit: { type: 'bucket', - size: 100, - dripRate: 500, + size: 200, + dripRate: 200, }, } as const; diff --git a/packages/backend/src/server/api/endpoints/charts/ap-request.ts b/packages/backend/src/server/api/endpoints/charts/ap-request.ts index 28c64229e7..68dc87546e 100644 --- a/packages/backend/src/server/api/endpoints/charts/ap-request.ts +++ b/packages/backend/src/server/api/endpoints/charts/ap-request.ts @@ -17,11 +17,11 @@ export const meta = { allowGet: true, cacheSec: 60 * 60, - // Burst up to 100, then 2/sec average + // Burst up to 200, then 5/sec average limit: { type: 'bucket', - size: 100, - dripRate: 500, + size: 200, + dripRate: 200, }, } as const; diff --git a/packages/backend/src/server/api/endpoints/charts/drive.ts b/packages/backend/src/server/api/endpoints/charts/drive.ts index 69ff3c5d7a..c0bfb00608 100644 --- a/packages/backend/src/server/api/endpoints/charts/drive.ts +++ b/packages/backend/src/server/api/endpoints/charts/drive.ts @@ -17,11 +17,11 @@ export const meta = { allowGet: true, cacheSec: 60 * 60, - // Burst up to 100, then 2/sec average + // Burst up to 200, then 5/sec average limit: { type: 'bucket', - size: 100, - dripRate: 500, + size: 200, + dripRate: 200, }, } as const; diff --git a/packages/backend/src/server/api/endpoints/charts/federation.ts b/packages/backend/src/server/api/endpoints/charts/federation.ts index bd870cc3d9..bd15700670 100644 --- a/packages/backend/src/server/api/endpoints/charts/federation.ts +++ b/packages/backend/src/server/api/endpoints/charts/federation.ts @@ -17,11 +17,11 @@ export const meta = { allowGet: true, cacheSec: 60 * 60, - // Burst up to 100, then 2/sec average + // Burst up to 200, then 5/sec average limit: { type: 'bucket', - size: 100, - dripRate: 500, + size: 200, + dripRate: 200, }, } as const; diff --git a/packages/backend/src/server/api/endpoints/charts/instance.ts b/packages/backend/src/server/api/endpoints/charts/instance.ts index 765bf024ee..e1053d05d8 100644 --- a/packages/backend/src/server/api/endpoints/charts/instance.ts +++ b/packages/backend/src/server/api/endpoints/charts/instance.ts @@ -17,11 +17,11 @@ export const meta = { allowGet: true, cacheSec: 60 * 60, - // Burst up to 100, then 2/sec average + // Burst up to 200, then 5/sec average limit: { type: 'bucket', - size: 100, - dripRate: 500, + size: 200, + dripRate: 200, }, } as const; diff --git a/packages/backend/src/server/api/endpoints/charts/notes.ts b/packages/backend/src/server/api/endpoints/charts/notes.ts index ecac436311..4550e2f17e 100644 --- a/packages/backend/src/server/api/endpoints/charts/notes.ts +++ b/packages/backend/src/server/api/endpoints/charts/notes.ts @@ -17,11 +17,11 @@ export const meta = { allowGet: true, cacheSec: 60 * 60, - // Burst up to 100, then 2/sec average + // Burst up to 200, then 5/sec average limit: { type: 'bucket', - size: 100, - dripRate: 500, + size: 200, + dripRate: 200, }, } as const; diff --git a/packages/backend/src/server/api/endpoints/charts/user/drive.ts b/packages/backend/src/server/api/endpoints/charts/user/drive.ts index 98ec40ade2..9475a8ab0a 100644 --- a/packages/backend/src/server/api/endpoints/charts/user/drive.ts +++ b/packages/backend/src/server/api/endpoints/charts/user/drive.ts @@ -17,11 +17,11 @@ export const meta = { allowGet: true, cacheSec: 60 * 60, - // Burst up to 100, then 2/sec average + // Burst up to 200, then 5/sec average limit: { type: 'bucket', - size: 100, - dripRate: 500, + size: 200, + dripRate: 200, }, } as const; diff --git a/packages/backend/src/server/api/endpoints/charts/user/following.ts b/packages/backend/src/server/api/endpoints/charts/user/following.ts index cb3dd36bab..1d333f9a9b 100644 --- a/packages/backend/src/server/api/endpoints/charts/user/following.ts +++ b/packages/backend/src/server/api/endpoints/charts/user/following.ts @@ -8,6 +8,8 @@ import { Endpoint } from '@/server/api/endpoint-base.js'; import { getJsonSchema } from '@/core/chart/core.js'; import PerUserFollowingChart from '@/core/chart/charts/per-user-following.js'; import { schema } from '@/core/chart/charts/entities/per-user-following.js'; +import { CacheService } from '@/core/CacheService.js'; +import { RoleService } from '@/core/RoleService.js'; export const meta = { tags: ['charts', 'users', 'following'], @@ -17,11 +19,11 @@ export const meta = { allowGet: true, cacheSec: 60 * 60, - // Burst up to 100, then 2/sec average + // Burst up to 200, then 5/sec average limit: { type: 'bucket', - size: 100, - dripRate: 500, + size: 200, + dripRate: 200, }, } as const; @@ -40,9 +42,84 @@ export const paramDef = { export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-disable-line import/no-default-export constructor( private perUserFollowingChart: PerUserFollowingChart, + private readonly cacheService: CacheService, + private readonly roleService: RoleService, ) { super(meta, paramDef, async (ps, me) => { - return await this.perUserFollowingChart.getChart(ps.span, ps.limit, ps.offset ? new Date(ps.offset) : null, ps.userId); + const profile = await this.cacheService.userProfileCache.fetch(ps.userId); + + // These are structured weird to avoid un-necessary calls to roleService and cacheService + const iAmModeratorOrTarget = me && (me.id === ps.userId || await this.roleService.isModerator(me)); + const iAmFollowingOrTarget = me && (me.id === ps.userId || await this.cacheService.isFollowing(me.id, ps.userId)); + + const canViewFollowing = + profile.followingVisibility === 'public' + || iAmModeratorOrTarget + || (profile.followingVisibility === 'followers' && iAmFollowingOrTarget); + + const canViewFollowers = + profile.followersVisibility === 'public' + || iAmModeratorOrTarget + || (profile.followersVisibility === 'followers' && iAmFollowingOrTarget); + + if (!canViewFollowing && !canViewFollowers) { + return { + local: { + followings: { + total: [], + inc: [], + dec: [], + }, + followers: { + total: [], + inc: [], + dec: [], + }, + }, + remote: { + followings: { + total: [], + inc: [], + dec: [], + }, + followers: { + total: [], + inc: [], + dec: [], + }, + }, + }; + } + + const chart = await this.perUserFollowingChart.getChart(ps.span, ps.limit, ps.offset ? new Date(ps.offset) : null, ps.userId); + + if (!canViewFollowers) { + chart.local.followers = { + total: [], + inc: [], + dec: [], + }; + chart.remote.followers = { + total: [], + inc: [], + dec: [], + }; + } + + if (!canViewFollowing) { + chart.local.followings = { + total: [], + inc: [], + dec: [], + }; + chart.remote.followings = { + total: [], + inc: [], + dec: [], + }; + } + + return chart; }); } } diff --git a/packages/backend/src/server/api/endpoints/charts/user/notes.ts b/packages/backend/src/server/api/endpoints/charts/user/notes.ts index 0742a21210..1d24dc2b77 100644 --- a/packages/backend/src/server/api/endpoints/charts/user/notes.ts +++ b/packages/backend/src/server/api/endpoints/charts/user/notes.ts @@ -17,11 +17,11 @@ export const meta = { allowGet: true, cacheSec: 60 * 60, - // Burst up to 100, then 2/sec average + // Burst up to 200, then 5/sec average limit: { type: 'bucket', - size: 100, - dripRate: 500, + size: 200, + dripRate: 200, }, } as const; diff --git a/packages/backend/src/server/api/endpoints/charts/user/pv.ts b/packages/backend/src/server/api/endpoints/charts/user/pv.ts index a220381b00..e0026d5ff3 100644 --- a/packages/backend/src/server/api/endpoints/charts/user/pv.ts +++ b/packages/backend/src/server/api/endpoints/charts/user/pv.ts @@ -17,11 +17,11 @@ export const meta = { allowGet: true, cacheSec: 60 * 60, - // Burst up to 100, then 2/sec average + // Burst up to 200, then 5/sec average limit: { type: 'bucket', - size: 100, - dripRate: 500, + size: 200, + dripRate: 200, }, } as const; diff --git a/packages/backend/src/server/api/endpoints/charts/user/reactions.ts b/packages/backend/src/server/api/endpoints/charts/user/reactions.ts index 3bb33622c2..c15056466f 100644 --- a/packages/backend/src/server/api/endpoints/charts/user/reactions.ts +++ b/packages/backend/src/server/api/endpoints/charts/user/reactions.ts @@ -17,11 +17,11 @@ export const meta = { allowGet: true, cacheSec: 60 * 60, - // Burst up to 100, then 2/sec average + // Burst up to 200, then 5/sec average limit: { type: 'bucket', - size: 100, - dripRate: 500, + size: 200, + dripRate: 200, }, } as const; diff --git a/packages/backend/src/server/api/endpoints/charts/users.ts b/packages/backend/src/server/api/endpoints/charts/users.ts index b5452517ab..0f96fae202 100644 --- a/packages/backend/src/server/api/endpoints/charts/users.ts +++ b/packages/backend/src/server/api/endpoints/charts/users.ts @@ -17,11 +17,11 @@ export const meta = { allowGet: true, cacheSec: 60 * 60, - // Burst up to 100, then 2/sec average + // Burst up to 200, then 5/sec average limit: { type: 'bucket', - size: 100, - dripRate: 500, + size: 200, + dripRate: 200, }, } as const; diff --git a/packages/backend/src/server/api/endpoints/clips/notes.ts b/packages/backend/src/server/api/endpoints/clips/notes.ts index 59513e530d..4758dbad00 100644 --- a/packages/backend/src/server/api/endpoints/clips/notes.ts +++ b/packages/backend/src/server/api/endpoints/clips/notes.ts @@ -92,10 +92,11 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- .andWhere('clipNote.clipId = :clipId', { clipId: clip.id }); this.queryService.generateBlockedHostQueryForNote(query); + this.queryService.generateVisibilityQuery(query, me); if (me) { - this.queryService.generateVisibilityQuery(query, me); this.queryService.generateMutedUserQueryForNotes(query, me); this.queryService.generateBlockedUserQueryForNotes(query, me); + this.queryService.generateMutedUserRenotesQueryForNotes(query, me); } const notes = await query diff --git a/packages/backend/src/server/api/endpoints/drive/files/attached-notes.ts b/packages/backend/src/server/api/endpoints/drive/files/attached-notes.ts index 32c2620915..9d70044db8 100644 --- a/packages/backend/src/server/api/endpoints/drive/files/attached-notes.ts +++ b/packages/backend/src/server/api/endpoints/drive/files/attached-notes.ts @@ -81,10 +81,22 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- throw new ApiError(meta.errors.noSuchFile); } - const query = this.queryService.makePaginationQuery(this.notesRepository.createQueryBuilder('note'), ps.sinceId, ps.untilId); - query.andWhere(':file <@ note.fileIds', { file: [file.id] }); + const query = this.queryService.makePaginationQuery(this.notesRepository.createQueryBuilder('note'), ps.sinceId, ps.untilId) + .andWhere(':file <@ note.fileIds', { file: [file.id] }) + .innerJoinAndSelect('note.user', 'user') + .leftJoinAndSelect('note.reply', 'reply') + .leftJoinAndSelect('note.renote', 'renote') + .leftJoinAndSelect('reply.user', 'replyUser') + .leftJoinAndSelect('renote.user', 'renoteUser') + .limit(ps.limit); - const notes = await query.limit(ps.limit).getMany(); + this.queryService.generateVisibilityQuery(query, me); + this.queryService.generateBlockedHostQueryForNote(query); + this.queryService.generateSilencedUserQueryForNotes(query, me); + this.queryService.generateMutedUserQueryForNotes(query, me); + this.queryService.generateBlockedUserQueryForNotes(query, me); + + const notes = await query.getMany(); return await this.noteEntityService.packMany(notes, me, { detail: true, diff --git a/packages/backend/src/server/api/endpoints/drive/files/create.ts b/packages/backend/src/server/api/endpoints/drive/files/create.ts index f4c47d71bf..939eadad9b 100644 --- a/packages/backend/src/server/api/endpoints/drive/files/create.ts +++ b/packages/backend/src/server/api/endpoints/drive/files/create.ts @@ -10,6 +10,8 @@ import { Endpoint } from '@/server/api/endpoint-base.js'; import { DriveFileEntityService } from '@/core/entities/DriveFileEntityService.js'; import { DriveService } from '@/core/DriveService.js'; import type { Config } from '@/config.js'; +import { ApiLoggerService } from '@/server/api/ApiLoggerService.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; import { ApiError } from '../../../error.js'; import { MiMeta } from '@/models/_.js'; import { DI } from '@/di-symbols.js'; @@ -95,6 +97,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- private driveFileEntityService: DriveFileEntityService, private driveService: DriveService, + private readonly apiLoggerService: ApiLoggerService, ) { super(meta, paramDef, async (ps, me, _, file, cleanup, ip, headers) => { // Get 'name' parameter @@ -130,14 +133,14 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- return await this.driveFileEntityService.pack(driveFile, { self: true }); } catch (err) { if (err instanceof Error || typeof err === 'string') { - console.error(err); + this.apiLoggerService.logger.error(`Error saving drive file: ${renderInlineError(err)}`); } if (err instanceof IdentifiableError) { if (err.id === '282f77bf-5816-4f72-9264-aa14d8261a21') throw new ApiError(meta.errors.inappropriate); if (err.id === 'c6244ed2-a39a-4e1c-bf93-f0fbd7764fa6') throw new ApiError(meta.errors.noFreeSpace); if (err.id === 'f9e4e5f3-4df4-40b5-b400-f236945f7073') throw new ApiError(meta.errors.maxFileSizeExceeded); } - throw new ApiError(); + throw err; } finally { cleanup!(); } diff --git a/packages/backend/src/server/api/endpoints/fetch-rss.ts b/packages/backend/src/server/api/endpoints/fetch-rss.ts index 03f35f16a5..11244b30f6 100644 --- a/packages/backend/src/server/api/endpoints/fetch-rss.ts +++ b/packages/backend/src/server/api/endpoints/fetch-rss.ts @@ -3,12 +3,12 @@ * SPDX-License-Identifier: AGPL-3.0-only */ -import Parser from 'rss-parser'; import { Injectable } from '@nestjs/common'; +import { parseFeed } from 'htmlparser2'; import { Endpoint } from '@/server/api/endpoint-base.js'; import { HttpRequestService } from '@/core/HttpRequestService.js'; - -const rssParser = new Parser(); +import { ApiError } from '../error.js'; +import type { FeedItem } from 'domutils'; export const meta = { tags: ['meta'], @@ -17,52 +17,32 @@ export const meta = { allowGet: true, cacheSec: 60 * 3, + errors: { + fetchFailed: { + id: '88f4356f-719d-4715-b4fc-703a10a812d2', + code: 'FETCH_FAILED', + message: 'Failed to fetch RSS feed', + }, + }, + res: { type: 'object', properties: { - image: { - type: 'object', + type: { + type: 'string', + optional: false, + }, + id: { + type: 'string', optional: true, - properties: { - link: { - type: 'string', - optional: true, - }, - url: { - type: 'string', - optional: false, - }, - title: { - type: 'string', - optional: true, - }, - }, }, - paginationLinks: { - type: 'object', + updated: { + type: 'string', + optional: true, + }, + author: { + type: 'string', optional: true, - properties: { - self: { - type: 'string', - optional: true, - }, - first: { - type: 'string', - optional: true, - }, - next: { - type: 'string', - optional: true, - }, - last: { - type: 'string', - optional: true, - }, - prev: { - type: 'string', - optional: true, - }, - }, }, link: { type: 'string', @@ -94,113 +74,42 @@ export const meta = { type: 'string', optional: true, }, - creator: { - type: 'string', - optional: true, - }, - summary: { - type: 'string', - optional: true, - }, - content: { - type: 'string', - optional: true, - }, - isoDate: { + description: { type: 'string', optional: true, }, - categories: { + media: { type: 'array', - optional: true, + optional: false, items: { - type: 'string', - }, - }, - contentSnippet: { - type: 'string', - optional: true, - }, - enclosure: { - type: 'object', - optional: true, - properties: { - url: { - type: 'string', - optional: false, - }, - length: { - type: 'number', - optional: true, - }, - type: { - type: 'string', - optional: true, + type: 'object', + properties: { + medium: { + type: 'string', + optional: true, + }, + url: { + type: 'string', + optional: true, + }, + type: { + type: 'string', + optional: true, + }, + lang: { + type: 'string', + optional: true, + }, }, }, }, }, }, }, - feedUrl: { - type: 'string', - optional: true, - }, description: { type: 'string', optional: true, }, - itunes: { - type: 'object', - optional: true, - additionalProperties: true, - properties: { - image: { - type: 'string', - optional: true, - }, - owner: { - type: 'object', - optional: true, - properties: { - name: { - type: 'string', - optional: true, - }, - email: { - type: 'string', - optional: true, - }, - }, - }, - author: { - type: 'string', - optional: true, - }, - summary: { - type: 'string', - optional: true, - }, - explicit: { - type: 'string', - optional: true, - }, - categories: { - type: 'array', - optional: true, - items: { - type: 'string', - }, - }, - keywords: { - type: 'array', - optional: true, - items: { - type: 'string', - }, - }, - }, - }, }, }, @@ -224,7 +133,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- constructor( private httpRequestService: HttpRequestService, ) { - super(meta, paramDef, async (ps, me) => { + super(meta, paramDef, async (ps) => { const res = await this.httpRequestService.send(ps.url, { method: 'GET', headers: { @@ -234,8 +143,38 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- }); const text = await res.text(); + const feed = parseFeed(text, { + xmlMode: true, + }); + + if (!feed) { + throw new ApiError(meta.errors.fetchFailed); + } - return rssParser.parseString(text); + return { + type: feed.type, + id: feed.id, + title: feed.title, + link: feed.link, + description: feed.description, + updated: feed.updated?.toISOString(), + author: feed.author, + items: feed.items + .filter((item): item is FeedItem & { link: string, title: string } => !!item.link && !!item.title) + .map(item => ({ + guid: item.id, + title: item.title, + link: item.link, + description: item.description, + pubDate: item.pubDate?.toISOString(), + media: item.media.map(media => ({ + medium: media.medium, + url: media.url, + type: media.type, + lang: media.lang, + })), + })), + }; }); } } diff --git a/packages/backend/src/server/api/endpoints/following/delete.ts b/packages/backend/src/server/api/endpoints/following/delete.ts index ba146b6703..442352a4d2 100644 --- a/packages/backend/src/server/api/endpoints/following/delete.ts +++ b/packages/backend/src/server/api/endpoints/following/delete.ts @@ -12,6 +12,7 @@ import { UserFollowingService } from '@/core/UserFollowingService.js'; import { DI } from '@/di-symbols.js'; import { GetterService } from '@/server/api/GetterService.js'; import { ApiError } from '../../error.js'; +import { CacheService } from '@/core/CacheService.js'; export const meta = { tags: ['following', 'users'], @@ -69,6 +70,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- private userEntityService: UserEntityService, private getterService: GetterService, private userFollowingService: UserFollowingService, + private readonly cacheService: CacheService, ) { super(meta, paramDef, async (ps, me) => { const follower = me; @@ -85,12 +87,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- }); // Check not following - const exist = await this.followingsRepository.exists({ - where: { - followerId: follower.id, - followeeId: followee.id, - }, - }); + const exist = await this.cacheService.userFollowingsCache.fetch(follower.id).then(f => f.has(followee.id)); if (!exist) { throw new ApiError(meta.errors.notFollowing); diff --git a/packages/backend/src/server/api/endpoints/following/invalidate.ts b/packages/backend/src/server/api/endpoints/following/invalidate.ts index b45d21410b..3809bf29b0 100644 --- a/packages/backend/src/server/api/endpoints/following/invalidate.ts +++ b/packages/backend/src/server/api/endpoints/following/invalidate.ts @@ -11,6 +11,7 @@ import { UserEntityService } from '@/core/entities/UserEntityService.js'; import { UserFollowingService } from '@/core/UserFollowingService.js'; import { DI } from '@/di-symbols.js'; import { GetterService } from '@/server/api/GetterService.js'; +import { CacheService } from '@/core/CacheService.js'; import { ApiError } from '../../error.js'; export const meta = { @@ -69,6 +70,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- private userEntityService: UserEntityService, private getterService: GetterService, private userFollowingService: UserFollowingService, + private readonly cacheService: CacheService, ) { super(meta, paramDef, async (ps, me) => { const followee = me; @@ -85,12 +87,9 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- }); // Check not following - const exist = await this.followingsRepository.findOneBy({ - followerId: follower.id, - followeeId: followee.id, - }); + const isFollowing = await this.cacheService.userFollowingsCache.fetch(follower.id).then(f => f.has(followee.id)); - if (exist == null) { + if (!isFollowing) { throw new ApiError(meta.errors.notFollowing); } diff --git a/packages/backend/src/server/api/endpoints/following/update-all.ts b/packages/backend/src/server/api/endpoints/following/update-all.ts index c953feb393..a02b51cc79 100644 --- a/packages/backend/src/server/api/endpoints/following/update-all.ts +++ b/packages/backend/src/server/api/endpoints/following/update-all.ts @@ -12,6 +12,7 @@ import { UserFollowingService } from '@/core/UserFollowingService.js'; import { DI } from '@/di-symbols.js'; import { GetterService } from '@/server/api/GetterService.js'; import { ApiError } from '../../error.js'; +import { CacheService } from '@/core/CacheService.js'; export const meta = { tags: ['following', 'users'], @@ -39,6 +40,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- constructor( @Inject(DI.followingsRepository) private followingsRepository: FollowingsRepository, + private readonly cacheService: CacheService, ) { super(meta, paramDef, async (ps, me) => { await this.followingsRepository.update({ @@ -48,6 +50,8 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- withReplies: ps.withReplies != null ? ps.withReplies : undefined, }); + await this.cacheService.refreshFollowRelationsFor(me.id); + return; }); } diff --git a/packages/backend/src/server/api/endpoints/following/update.ts b/packages/backend/src/server/api/endpoints/following/update.ts index d62cf210ed..f4ca21856f 100644 --- a/packages/backend/src/server/api/endpoints/following/update.ts +++ b/packages/backend/src/server/api/endpoints/following/update.ts @@ -11,6 +11,7 @@ import { UserEntityService } from '@/core/entities/UserEntityService.js'; import { UserFollowingService } from '@/core/UserFollowingService.js'; import { DI } from '@/di-symbols.js'; import { GetterService } from '@/server/api/GetterService.js'; +import { CacheService } from '@/core/CacheService.js'; import { ApiError } from '../../error.js'; export const meta = { @@ -71,6 +72,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- private userEntityService: UserEntityService, private getterService: GetterService, private userFollowingService: UserFollowingService, + private readonly cacheService: CacheService, ) { super(meta, paramDef, async (ps, me) => { const follower = me; @@ -87,10 +89,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- }); // Check not following - const exist = await this.followingsRepository.findOneBy({ - followerId: follower.id, - followeeId: followee.id, - }); + const exist = await this.cacheService.userFollowingsCache.fetch(follower.id).then(f => f.get(followee.id)); if (exist == null) { throw new ApiError(meta.errors.notFollowing); @@ -103,6 +102,8 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- withReplies: ps.withReplies != null ? ps.withReplies : undefined, }); + await this.cacheService.refreshFollowRelationsFor(follower.id); + return await this.userEntityService.pack(follower.id, me); }); } diff --git a/packages/backend/src/server/api/endpoints/gallery/posts/create.ts b/packages/backend/src/server/api/endpoints/gallery/posts/create.ts index 504a9c789e..08abd7fed5 100644 --- a/packages/backend/src/server/api/endpoints/gallery/posts/create.ts +++ b/packages/backend/src/server/api/endpoints/gallery/posts/create.ts @@ -72,7 +72,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- ))).filter(x => x != null); if (files.length === 0) { - throw new Error(); + throw new Error('no files specified'); } const post = await this.galleryPostsRepository.insertOne(new MiGalleryPost({ diff --git a/packages/backend/src/server/api/endpoints/gallery/posts/update.ts b/packages/backend/src/server/api/endpoints/gallery/posts/update.ts index 5243ee9603..d0f9b56863 100644 --- a/packages/backend/src/server/api/endpoints/gallery/posts/update.ts +++ b/packages/backend/src/server/api/endpoints/gallery/posts/update.ts @@ -73,7 +73,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- ))).filter(x => x != null); if (files.length === 0) { - throw new Error(); + throw new Error('no files'); } } diff --git a/packages/backend/src/server/api/endpoints/i/2fa/remove-key.ts b/packages/backend/src/server/api/endpoints/i/2fa/remove-key.ts index d4098458d7..931c8d69b0 100644 --- a/packages/backend/src/server/api/endpoints/i/2fa/remove-key.ts +++ b/packages/backend/src/server/api/endpoints/i/2fa/remove-key.ts @@ -70,7 +70,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- try { await this.userAuthService.twoFactorAuthenticate(profile, token); } catch (e) { - throw new Error('authentication failed'); + throw new Error('authentication failed', { cause: e }); } } diff --git a/packages/backend/src/server/api/endpoints/i/move.ts b/packages/backend/src/server/api/endpoints/i/move.ts index 7852b5a2e1..e2a14b61af 100644 --- a/packages/backend/src/server/api/endpoints/i/move.ts +++ b/packages/backend/src/server/api/endpoints/i/move.ts @@ -17,7 +17,7 @@ import { ApiLoggerService } from '@/server/api/ApiLoggerService.js'; import { GetterService } from '@/server/api/GetterService.js'; import { ApPersonService } from '@/core/activitypub/models/ApPersonService.js'; import { UserEntityService } from '@/core/entities/UserEntityService.js'; - +import { renderInlineError } from '@/misc/render-inline-error.js'; import * as Acct from '@/misc/acct.js'; import { DI } from '@/di-symbols.js'; import { MiMeta } from '@/models/_.js'; @@ -105,7 +105,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- const { username, host } = Acct.parse(ps.moveToAccount); // retrieve the destination account let moveTo = await this.remoteUserResolveService.resolveUser(username, host).catch((e) => { - this.apiLoggerService.logger.warn(`failed to resolve remote user: ${e}`); + this.apiLoggerService.logger.warn(`failed to resolve remote user: ${renderInlineError(e)}`); throw new ApiError(meta.errors.noSuchUser); }); const destination = await this.getterService.getUser(moveTo.id) as MiLocalUser | MiRemoteUser; diff --git a/packages/backend/src/server/api/endpoints/i/notifications-grouped.ts b/packages/backend/src/server/api/endpoints/i/notifications-grouped.ts index b9c41b057d..444734070f 100644 --- a/packages/backend/src/server/api/endpoints/i/notifications-grouped.ts +++ b/packages/backend/src/server/api/endpoints/i/notifications-grouped.ts @@ -104,53 +104,88 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- } // grouping - let groupedNotifications = [notifications[0]] as MiGroupedNotification[]; - for (let i = 1; i < notifications.length; i++) { + const groupedNotifications : MiGroupedNotification[] = []; + // keep track of where reaction / renote notifications are, by note id + const reactionIdxByNoteId = new Map<string, number>(); + const renoteIdxByNoteId = new Map<string, number>(); + + // group notifications by type+note; notice that we don't try to + // split groups if they span a long stretch of time, because + // it's probably overkill: if the user has very few + // notifications, there should be very little difference; if the + // user has many notifications, the pagination will break the + // groups + + // scan `notifications` newest-to-oldest + for (let i = 0; i < notifications.length; i++) { const notification = notifications[i]; - const prev = notifications[i - 1]; - let prevGroupedNotification = groupedNotifications.at(-1)!; - if (prev.type === 'reaction' && notification.type === 'reaction' && prev.noteId === notification.noteId) { - if (prevGroupedNotification.type !== 'reaction:grouped') { - groupedNotifications[groupedNotifications.length - 1] = { + if (notification.type === 'reaction') { + const reactionIdx = reactionIdxByNoteId.get(notification.noteId); + if (reactionIdx === undefined) { + // first reaction to this note that we see, add it as-is + // and remember where we put it + groupedNotifications.push(notification); + reactionIdxByNoteId.set(notification.noteId, groupedNotifications.length - 1); + continue; + } + + let prevReaction = groupedNotifications[reactionIdx] as FilterUnionByProperty<MiGroupedNotification, 'type', 'reaction:grouped'> | FilterUnionByProperty<MiGroupedNotification, 'type', 'reaction'>; + // if the previous reaction is not a group, make it into one + if (prevReaction.type !== 'reaction:grouped') { + prevReaction = groupedNotifications[reactionIdx] = { type: 'reaction:grouped', - id: '', - createdAt: prev.createdAt, - noteId: prev.noteId!, + id: prevReaction.id, // this will be the newest id in this group + createdAt: prevReaction.createdAt, + noteId: prevReaction.noteId!, reactions: [{ - userId: prev.notifierId!, - reaction: prev.reaction!, + userId: prevReaction.notifierId!, + reaction: prevReaction.reaction!, }], }; - prevGroupedNotification = groupedNotifications.at(-1)!; } - (prevGroupedNotification as FilterUnionByProperty<MiGroupedNotification, 'type', 'reaction:grouped'>).reactions.push({ + // add this new reaction to the existing group + (prevReaction as FilterUnionByProperty<MiGroupedNotification, 'type', 'reaction:grouped'>).reactions.push({ userId: notification.notifierId!, reaction: notification.reaction!, }); - prevGroupedNotification.id = notification.id; continue; } - if (prev.type === 'renote' && notification.type === 'renote' && prev.targetNoteId === notification.targetNoteId) { - if (prevGroupedNotification.type !== 'renote:grouped') { - groupedNotifications[groupedNotifications.length - 1] = { + + if (notification.type === 'renote') { + const renoteIdx = renoteIdxByNoteId.get(notification.targetNoteId); + if (renoteIdx === undefined) { + // first renote of this note that we see, add it as-is and + // remember where we put it + groupedNotifications.push(notification); + renoteIdxByNoteId.set(notification.targetNoteId, groupedNotifications.length - 1); + continue; + } + + let prevRenote = groupedNotifications[renoteIdx] as FilterUnionByProperty<MiGroupedNotification, 'type', 'renote:grouped'> | FilterUnionByProperty<MiGroupedNotification, 'type', 'renote'>; + // if the previous renote is not a group, make it into one + if (prevRenote.type !== 'renote:grouped') { + prevRenote = groupedNotifications[renoteIdx] = { type: 'renote:grouped', - id: '', - createdAt: notification.createdAt, - noteId: prev.noteId!, - userIds: [prev.notifierId!], + id: prevRenote.id, // this will be the newest id in this group + createdAt: prevRenote.createdAt, + noteId: prevRenote.noteId!, + userIds: [prevRenote.notifierId!], }; - prevGroupedNotification = groupedNotifications.at(-1)!; } - (prevGroupedNotification as FilterUnionByProperty<MiGroupedNotification, 'type', 'renote:grouped'>).userIds.push(notification.notifierId!); - prevGroupedNotification.id = notification.id; + // add this new renote to the existing group + (prevRenote as FilterUnionByProperty<MiGroupedNotification, 'type', 'renote:grouped'>).userIds.push(notification.notifierId!); continue; } + // not a groupable notification, just push it groupedNotifications.push(notification); } - groupedNotifications = groupedNotifications.slice(0, ps.limit); + // sort the groups by their id, newest first + groupedNotifications.sort( + (a, b) => a.id < b.id ? 1 : a.id > b.id ? -1 : 0, + ); return await this.notificationEntityService.packGroupedMany(groupedNotifications, me.id); }); diff --git a/packages/backend/src/server/api/endpoints/i/registry/get.ts b/packages/backend/src/server/api/endpoints/i/registry/get.ts index dff33016e0..d284334834 100644 --- a/packages/backend/src/server/api/endpoints/i/registry/get.ts +++ b/packages/backend/src/server/api/endpoints/i/registry/get.ts @@ -20,9 +20,7 @@ export const meta = { }, }, - res: { - type: 'object', - }, + res: {}, // 10 calls per 5 seconds limit: { diff --git a/packages/backend/src/server/api/endpoints/i/update.ts b/packages/backend/src/server/api/endpoints/i/update.ts index f35e395841..5767880531 100644 --- a/packages/backend/src/server/api/endpoints/i/update.ts +++ b/packages/backend/src/server/api/endpoints/i/update.ts @@ -3,7 +3,7 @@ * SPDX-License-Identifier: AGPL-3.0-only */ -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; import { Inject, Injectable } from '@nestjs/common'; import ms from 'ms'; import { extractCustomEmojisFromMfm } from '@/misc/extract-custom-emojis-from-mfm.js'; @@ -34,6 +34,7 @@ import { verifyFieldLinks } from '@/misc/verify-field-link.js'; import { AvatarDecorationService } from '@/core/AvatarDecorationService.js'; import { notificationRecieveConfig } from '@/models/json-schema/user.js'; import { userUnsignedFetchOptions } from '@/const.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; import { ApiLoggerService } from '../../ApiLoggerService.js'; import { ApiError } from '../../error.js'; @@ -263,6 +264,15 @@ export const paramDef = { enum: userUnsignedFetchOptions, nullable: false, }, + attributionDomains: { + type: 'array', + items: { + type: 'string', + minLength: 1, + maxLength: 128, + }, + maxItems: 32, + }, }, } as const; @@ -373,6 +383,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- } if (ps.mutedInstances !== undefined) profileUpdates.mutedInstances = ps.mutedInstances; if (ps.notificationRecieveConfig !== undefined) profileUpdates.notificationRecieveConfig = ps.notificationRecieveConfig; + if (ps.attributionDomains !== undefined) updates.attributionDomains = ps.attributionDomains; if (typeof ps.isLocked === 'boolean') updates.isLocked = ps.isLocked; if (typeof ps.isExplorable === 'boolean') updates.isExplorable = ps.isExplorable; if (typeof ps.hideOnlineStatus === 'boolean') updates.hideOnlineStatus = ps.hideOnlineStatus; @@ -506,7 +517,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- // Retrieve the old account const knownAs = await this.remoteUserResolveService.resolveUser(username, host).catch((e) => { - this.apiLoggerService.logger.warn(`failed to resolve dstination user: ${e}`); + this.apiLoggerService.logger.warn(`failed to resolve destination user: ${renderInlineError(e)}`); throw new ApiError(meta.errors.noSuchUser); }); if (knownAs.id === _user.id) throw new ApiError(meta.errors.forbiddenToSetYourself); @@ -606,7 +617,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- const updatedProfile = await this.userProfilesRepository.findOneByOrFail({ userId: user.id }); - this.cacheService.userProfileCache.set(user.id, updatedProfile); + await this.cacheService.userProfileCache.set(user.id, updatedProfile); // Publish meUpdated event this.globalEventService.publishMainStream(user.id, 'meUpdated', iObj); @@ -663,7 +674,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- // these two methods need to be kept in sync with // `ApRendererService.renderPerson` private userNeedsPublishing(oldUser: MiLocalUser, newUser: Partial<MiUser>): boolean { - const basicFields: (keyof MiUser)[] = ['avatarId', 'bannerId', 'backgroundId', 'isBot', 'username', 'name', 'isLocked', 'isExplorable', 'isCat', 'noindex', 'speakAsCat', 'movedToUri', 'alsoKnownAs', 'hideOnlineStatus', 'enableRss', 'requireSigninToViewContents', 'makeNotesFollowersOnlyBefore', 'makeNotesHiddenBefore']; + const basicFields: (keyof MiUser)[] = ['avatarId', 'bannerId', 'backgroundId', 'isBot', 'username', 'name', 'isLocked', 'isExplorable', 'isCat', 'noindex', 'speakAsCat', 'movedToUri', 'alsoKnownAs', 'hideOnlineStatus', 'enableRss', 'requireSigninToViewContents', 'makeNotesFollowersOnlyBefore', 'makeNotesHiddenBefore', 'attributionDomains']; for (const field of basicFields) { if ((field in newUser) && oldUser[field] !== newUser[field]) { return true; diff --git a/packages/backend/src/server/api/endpoints/notes.ts b/packages/backend/src/server/api/endpoints/notes.ts index f6c37023e1..00a88521fd 100644 --- a/packages/backend/src/server/api/endpoints/notes.ts +++ b/packages/backend/src/server/api/endpoints/notes.ts @@ -64,7 +64,16 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- .leftJoinAndSelect('note.reply', 'reply') .leftJoinAndSelect('note.renote', 'renote') .leftJoinAndSelect('reply.user', 'replyUser') - .leftJoinAndSelect('renote.user', 'renoteUser'); + .leftJoinAndSelect('renote.user', 'renoteUser') + .limit(ps.limit); + + this.queryService.generateVisibilityQuery(query, me); + this.queryService.generateBlockedHostQueryForNote(query); + if (me) { + this.queryService.generateSilencedUserQueryForNotes(query, me); + this.queryService.generateMutedUserQueryForNotes(query, me); + this.queryService.generateBlockedUserQueryForNotes(query, me); + } if (ps.local) { query.andWhere('note.userHost IS NULL'); @@ -75,7 +84,15 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- } if (ps.renote !== undefined) { - query.andWhere(ps.renote ? 'note.renoteId IS NOT NULL' : 'note.renoteId IS NULL'); + if (ps.renote) { + this.queryService.andIsRenote(query, 'note'); + + if (me) { + this.queryService.generateMutedUserRenotesQueryForNotes(query, me); + } + } else { + this.queryService.andIsNotRenote(query, 'note'); + } } if (ps.withFiles !== undefined) { @@ -91,7 +108,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- // query.isBot = bot; //} - const notes = await query.limit(ps.limit).getMany(); + const notes = await query.getMany(); return await this.noteEntityService.packMany(notes); }); diff --git a/packages/backend/src/server/api/endpoints/notes/bubble-timeline.ts b/packages/backend/src/server/api/endpoints/notes/bubble-timeline.ts index df030d90aa..84d6aa0dc7 100644 --- a/packages/backend/src/server/api/endpoints/notes/bubble-timeline.ts +++ b/packages/backend/src/server/api/endpoints/notes/bubble-timeline.ts @@ -1,13 +1,16 @@ +/* + * SPDX-FileCopyrightText: Marie and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + import { Inject, Injectable } from '@nestjs/common'; -import { Brackets } from 'typeorm'; -import type { NotesRepository, MiMeta } from '@/models/_.js'; +import type { NotesRepository } from '@/models/_.js'; import { Endpoint } from '@/server/api/endpoint-base.js'; import { QueryService } from '@/core/QueryService.js'; import { NoteEntityService } from '@/core/entities/NoteEntityService.js'; import ActiveUsersChart from '@/core/chart/charts/active-users.js'; import { DI } from '@/di-symbols.js'; import { RoleService } from '@/core/RoleService.js'; -import { CacheService } from '@/core/CacheService.js'; import { ApiError } from '../../error.js'; export const meta = { @@ -56,9 +59,6 @@ export const paramDef = { @Injectable() export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-disable-line import/no-default-export constructor( - @Inject(DI.meta) - private serverSettings: MiMeta, - @Inject(DI.notesRepository) private notesRepository: NotesRepository, @@ -66,7 +66,6 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- private queryService: QueryService, private roleService: RoleService, private activeUsersChart: ActiveUsersChart, - private cacheService: CacheService, ) { super(meta, paramDef, async (ps, me) => { const policies = await this.roleService.getUserPolicies(me ? me.id : null); @@ -74,29 +73,34 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- throw new ApiError(meta.errors.btlDisabled); } - const [ - followings, - ] = me ? await Promise.all([ - this.cacheService.userFollowingsCache.fetch(me.id), - ]) : [undefined]; - //#region Construct query const query = this.queryService.makePaginationQuery(this.notesRepository.createQueryBuilder('note'), ps.sinceId, ps.untilId, ps.sinceDate, ps.untilDate) .andWhere('note.visibility = \'public\'') .andWhere('note.channelId IS NULL') - .andWhere('note.userHost IN (:...hosts)', { hosts: this.serverSettings.bubbleInstances }) + .andWhere('note.userHost IS NOT NULL') .innerJoinAndSelect('note.user', 'user') .leftJoinAndSelect('note.reply', 'reply') .leftJoinAndSelect('note.renote', 'renote') .leftJoinAndSelect('reply.user', 'replyUser') - .leftJoinAndSelect('renote.user', 'renoteUser'); + .leftJoinAndSelect('renote.user', 'renoteUser') + .limit(ps.limit); + + // This subquery mess teaches postgres how to use the right indexes. + // Using WHERE or ON conditions causes a fallback to full sequence scan, which times out. + // Important: don't use a query builder here or TypeORM will get confused and stop quoting column names! (known, unfixed bug apparently) + query + .leftJoin('(select "host" from "instance" where "isBubbled" = true)', 'bubbleInstance', '"bubbleInstance"."host" = "note"."userHost"') + .andWhere('"bubbleInstance" IS NOT NULL'); + this.queryService + .leftJoinInstance(query, 'note.userInstance', 'userInstance', '"userInstance"."host" = "bubbleInstance"."host"'); - this.queryService.generateVisibilityQuery(query, me); this.queryService.generateBlockedHostQueryForNote(query); - if (me) this.queryService.generateMutedUserQueryForNotes(query, me); - if (me) this.queryService.generateBlockedUserQueryForNotes(query, me); - if (me) this.queryService.generateMutedUserRenotesQueryForNotes(query, me); + this.queryService.generateSilencedUserQueryForNotes(query, me); + if (me) { + this.queryService.generateMutedUserQueryForNotes(query, me); + this.queryService.generateBlockedUserQueryForNotes(query, me); + } if (ps.withFiles) { query.andWhere('note.fileIds != \'{}\''); @@ -104,29 +108,20 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- if (!ps.withBots) query.andWhere('user.isBot = FALSE'); - if (ps.withRenotes === false) { - query.andWhere(new Brackets(qb => { - qb.where('note.renoteId IS NULL'); - qb.orWhere(new Brackets(qb => { - qb.where('note.text IS NOT NULL'); - qb.orWhere('note.fileIds != \'{}\''); - })); - })); + if (!ps.withRenotes) { + this.queryService.generateExcludedRenotesQueryForNotes(query); + } else if (me) { + this.queryService.generateMutedUserRenotesQueryForNotes(query, me); } //#endregion - let timeline = await query.limit(ps.limit).getMany(); - - timeline = timeline.filter(note => { - if (note.user?.isSilenced && me && followings && note.userId !== me.id && !followings[note.userId]) return false; - return true; - }); + const timeline = await query.getMany(); - process.nextTick(() => { - if (me) { + if (me) { + process.nextTick(() => { this.activeUsersChart.read(me); - } - }); + }); + } return await this.noteEntityService.packMany(timeline, me); }); diff --git a/packages/backend/src/server/api/endpoints/notes/children.ts b/packages/backend/src/server/api/endpoints/notes/children.ts index 8f19d534d4..cf8b11ccb5 100644 --- a/packages/backend/src/server/api/endpoints/notes/children.ts +++ b/packages/backend/src/server/api/endpoints/notes/children.ts @@ -57,26 +57,22 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- super(meta, paramDef, async (ps, me) => { const query = this.queryService.makePaginationQuery(this.notesRepository.createQueryBuilder('note'), ps.sinceId, ps.untilId) .andWhere(new Brackets(qb => { - qb - .where('note.replyId = :noteId', { noteId: ps.noteId }); - if (ps.showQuotes) { - qb.orWhere(new Brackets(qb => { - qb - .where('note.renoteId = :noteId', { noteId: ps.noteId }) - .andWhere(new Brackets(qb => { - qb - .where('note.text IS NOT NULL') - .orWhere('note.fileIds != \'{}\'') - .orWhere('note.hasPoll = TRUE'); - })); - })); - } + qb.orWhere('note.replyId = :noteId'); + + if (ps.showQuotes) { + qb.orWhere(new Brackets(qbb => this.queryService + .andIsQuote(qbb, 'note') + .andWhere('note.renoteId = :noteId'), + )); + } })) .innerJoinAndSelect('note.user', 'user') .leftJoinAndSelect('note.reply', 'reply') .leftJoinAndSelect('note.renote', 'renote') .leftJoinAndSelect('reply.user', 'replyUser') - .leftJoinAndSelect('renote.user', 'renoteUser'); + .leftJoinAndSelect('renote.user', 'renoteUser') + .setParameters({ noteId: ps.noteId }) + .limit(ps.limit); this.queryService.generateVisibilityQuery(query, me); this.queryService.generateBlockedHostQueryForNote(query); @@ -85,7 +81,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- this.queryService.generateBlockedUserQueryForNotes(query, me); } - const notes = await query.limit(ps.limit).getMany(); + const notes = await query.getMany(); return await this.noteEntityService.packMany(notes, me); }); diff --git a/packages/backend/src/server/api/endpoints/notes/create.ts b/packages/backend/src/server/api/endpoints/notes/create.ts index 3dd90c3dca..461910543f 100644 --- a/packages/backend/src/server/api/endpoints/notes/create.ts +++ b/packages/backend/src/server/api/endpoints/notes/create.ts @@ -348,7 +348,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- throw new ApiError(meta.errors.noSuchReplyTarget); } else if (isRenote(reply) && !isQuote(reply)) { throw new ApiError(meta.errors.cannotReplyToPureRenote); - } else if (!await this.noteEntityService.isVisibleForMe(reply, me.id)) { + } else if (!await this.noteEntityService.isVisibleForMe(reply, me.id, { me })) { throw new ApiError(meta.errors.cannotReplyToInvisibleNote); } else if (reply.visibility === 'specified' && ps.visibility !== 'specified') { throw new ApiError(meta.errors.cannotReplyToSpecifiedVisibilityNoteWithExtendedVisibility); diff --git a/packages/backend/src/server/api/endpoints/notes/edit.ts b/packages/backend/src/server/api/endpoints/notes/edit.ts index 2c01b26584..bd70cb7835 100644 --- a/packages/backend/src/server/api/endpoints/notes/edit.ts +++ b/packages/backend/src/server/api/endpoints/notes/edit.ts @@ -402,7 +402,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- throw new ApiError(meta.errors.noSuchReplyTarget); } else if (isRenote(reply) && !isQuote(reply)) { throw new ApiError(meta.errors.cannotReplyToPureRenote); - } else if (!await this.noteEntityService.isVisibleForMe(reply, me.id)) { + } else if (!await this.noteEntityService.isVisibleForMe(reply, me.id, { me })) { throw new ApiError(meta.errors.cannotReplyToInvisibleNote); } else if (reply.visibility === 'specified' && ps.visibility !== 'specified') { throw new ApiError(meta.errors.cannotReplyToSpecifiedVisibilityNoteWithExtendedVisibility); diff --git a/packages/backend/src/server/api/endpoints/notes/following.ts b/packages/backend/src/server/api/endpoints/notes/following.ts index 5f6ee9f903..0f8c61ab3e 100644 --- a/packages/backend/src/server/api/endpoints/notes/following.ts +++ b/packages/backend/src/server/api/endpoints/notes/following.ts @@ -4,7 +4,7 @@ */ import { Inject, Injectable } from '@nestjs/common'; -import { ObjectLiteral, SelectQueryBuilder } from 'typeorm'; +import { IsNull, ObjectLiteral, SelectQueryBuilder } from 'typeorm'; import { SkLatestNote, MiFollowing } from '@/models/_.js'; import type { NotesRepository } from '@/models/_.js'; import { Endpoint } from '@/server/api/endpoint-base.js'; @@ -12,6 +12,7 @@ import { NoteEntityService } from '@/core/entities/NoteEntityService.js'; import { DI } from '@/di-symbols.js'; import { QueryService } from '@/core/QueryService.js'; import { ApiError } from '@/server/api/error.js'; +import ActiveUsersChart from '@/core/chart/charts/active-users.js'; export const meta = { tags: ['notes'], @@ -76,8 +77,9 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- @Inject(DI.notesRepository) private notesRepository: NotesRepository, - private noteEntityService: NoteEntityService, - private queryService: QueryService, + private readonly noteEntityService: NoteEntityService, + private readonly queryService: QueryService, + private readonly activeUsersChart: ActiveUsersChart, ) { super(meta, paramDef, async (ps, me) => { if (ps.includeReplies && ps.filesOnly) throw new ApiError(meta.errors.bothWithRepliesAndWithFiles); @@ -85,7 +87,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- const query = this.notesRepository .createQueryBuilder('note') - .setParameter('me', me.id) + .setParameters({ meId: me.id }) // Limit to latest notes .innerJoin( @@ -130,7 +132,9 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- .leftJoinAndSelect('note.renote', 'renote') .leftJoinAndSelect('reply.user', 'replyUser') .leftJoinAndSelect('renote.user', 'renoteUser') - .leftJoinAndSelect('note.channel', 'channel') + + // Exclude channel notes + .andWhere({ channelId: IsNull() }) ; // Limit to files, if requested @@ -145,23 +149,26 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- // Hide blocked users / instances query.andWhere('"user"."isSuspended" = false'); - query.andWhere('("replyUser" IS NULL OR "replyUser"."isSuspended" = false)'); - query.andWhere('("renoteUser" IS NULL OR "renoteUser"."isSuspended" = false)'); this.queryService.generateBlockedHostQueryForNote(query); - // Respect blocks and mutes + // Respect blocks, mutes, and privacy + this.queryService.generateVisibilityQuery(query, me); this.queryService.generateBlockedUserQueryForNotes(query, me); this.queryService.generateMutedUserQueryForNotes(query, me); // Support pagination this.queryService .makePaginationQuery(query, ps.sinceId, ps.untilId, ps.sinceDate, ps.untilDate) - .orderBy('note.id', 'DESC') .take(ps.limit); // Query and return the next page const notes = await query.getMany(); - return await this.noteEntityService.packMany(notes, me); + + process.nextTick(() => { + this.activeUsersChart.read(me); + }); + + return await this.noteEntityService.packMany(notes, me, { skipHide: true }); }); } } @@ -170,14 +177,14 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- * Limit to followers (they follow us) */ function addFollower<T extends SelectQueryBuilder<ObjectLiteral>>(query: T): T { - return query.innerJoin(MiFollowing, 'follower', 'follower."followerId" = latest.user_id AND follower."followeeId" = :me'); + return query.innerJoin(MiFollowing, 'follower', 'follower."followerId" = latest.user_id AND follower."followeeId" = :meId'); } /** * Limit to followees (we follow them) */ function addFollowee<T extends SelectQueryBuilder<ObjectLiteral>>(query: T): T { - return query.innerJoin(MiFollowing, 'followee', 'followee."followerId" = :me AND followee."followeeId" = latest.user_id'); + return query.innerJoin(MiFollowing, 'followee', 'followee."followerId" = :meId AND followee."followeeId" = latest.user_id'); } /** diff --git a/packages/backend/src/server/api/endpoints/notes/global-timeline.ts b/packages/backend/src/server/api/endpoints/notes/global-timeline.ts index e82d9ca7af..506ea6fcda 100644 --- a/packages/backend/src/server/api/endpoints/notes/global-timeline.ts +++ b/packages/backend/src/server/api/endpoints/notes/global-timeline.ts @@ -12,7 +12,6 @@ import { NoteEntityService } from '@/core/entities/NoteEntityService.js'; import ActiveUsersChart from '@/core/chart/charts/active-users.js'; import { DI } from '@/di-symbols.js'; import { RoleService } from '@/core/RoleService.js'; -import { CacheService } from '@/core/CacheService.js'; import { ApiError } from '../../error.js'; export const meta = { @@ -68,7 +67,6 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- private queryService: QueryService, private roleService: RoleService, private activeUsersChart: ActiveUsersChart, - private cacheService: CacheService, ) { super(meta, paramDef, async (ps, me) => { const policies = await this.roleService.getUserPolicies(me ? me.id : null); @@ -76,8 +74,6 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- throw new ApiError(meta.errors.gtlDisabled); } - const followings = me ? await this.cacheService.userFollowingsCache.fetch(me.id) : {}; - //#region Construct query const query = this.queryService.makePaginationQuery(this.notesRepository.createQueryBuilder('note'), ps.sinceId, ps.untilId, ps.sinceDate, ps.untilDate) @@ -90,11 +86,10 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- .leftJoinAndSelect('renote.user', 'renoteUser'); this.queryService.generateBlockedHostQueryForNote(query); - + this.queryService.generateSilencedUserQueryForNotes(query, me); if (me) { this.queryService.generateMutedUserQueryForNotes(query, me); this.queryService.generateBlockedUserQueryForNotes(query, me); - this.queryService.generateMutedUserRenotesQueryForNotes(query, me); } if (ps.withFiles) { @@ -103,29 +98,20 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- if (!ps.withBots) query.andWhere('user.isBot = FALSE'); - if (ps.withRenotes === false) { - query.andWhere(new Brackets(qb => { - qb.where('note.renoteId IS NULL'); - qb.orWhere(new Brackets(qb => { - qb.where('note.text IS NOT NULL'); - qb.orWhere('note.fileIds != \'{}\''); - })); - })); + if (!ps.withRenotes) { + this.queryService.generateExcludedRenotesQueryForNotes(query); + } else if (me) { + this.queryService.generateMutedUserRenotesQueryForNotes(query, me); } //#endregion - let timeline = await query.limit(ps.limit).getMany(); + const timeline = await query.limit(ps.limit).getMany(); - timeline = timeline.filter(note => { - if (note.user?.isSilenced && me && followings && note.userId !== me.id && !followings[note.userId]) return false; - return true; - }); - - process.nextTick(() => { - if (me) { + if (me) { + process.nextTick(() => { this.activeUsersChart.read(me); - } - }); + }); + } return await this.noteEntityService.packMany(timeline, me); }); diff --git a/packages/backend/src/server/api/endpoints/notes/hybrid-timeline.ts b/packages/backend/src/server/api/endpoints/notes/hybrid-timeline.ts index 6461a2e33f..a5623d1f03 100644 --- a/packages/backend/src/server/api/endpoints/notes/hybrid-timeline.ts +++ b/packages/backend/src/server/api/endpoints/notes/hybrid-timeline.ts @@ -66,9 +66,6 @@ export const paramDef = { sinceDate: { type: 'integer' }, untilDate: { type: 'integer' }, allowPartial: { type: 'boolean', default: false }, // true is recommended but for compatibility false by default - includeMyRenotes: { type: 'boolean', default: true }, - includeRenotedMyNotes: { type: 'boolean', default: true }, - includeLocalRenotes: { type: 'boolean', default: true }, withFiles: { type: 'boolean', default: false }, withRenotes: { type: 'boolean', default: true }, withReplies: { type: 'boolean', default: false }, @@ -114,12 +111,10 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- untilId, sinceId, limit: ps.limit, - includeMyRenotes: ps.includeMyRenotes, - includeRenotedMyNotes: ps.includeRenotedMyNotes, - includeLocalRenotes: ps.includeLocalRenotes, withFiles: ps.withFiles, withReplies: ps.withReplies, withBots: ps.withBots, + withRenotes: ps.withRenotes, }, me); process.nextTick(() => { @@ -169,7 +164,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- excludeBots: !ps.withBots, noteFilter: note => { if (note.reply && note.reply.visibility === 'followers') { - if (!Object.hasOwn(followings, note.reply.userId) && note.reply.userId !== me.id) return false; + if (!followings.has(note.reply.userId) && note.reply.userId !== me.id) return false; } return true; @@ -178,12 +173,10 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- untilId, sinceId, limit, - includeMyRenotes: ps.includeMyRenotes, - includeRenotedMyNotes: ps.includeRenotedMyNotes, - includeLocalRenotes: ps.includeLocalRenotes, withFiles: ps.withFiles, withReplies: ps.withReplies, withBots: ps.withBots, + withRenotes: ps.withRenotes, }, me), }); @@ -199,103 +192,58 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- untilId: string | null, sinceId: string | null, limit: number, - includeMyRenotes: boolean, - includeRenotedMyNotes: boolean, - includeLocalRenotes: boolean, withFiles: boolean, withReplies: boolean, withBots: boolean, + withRenotes: boolean, }, me: MiLocalUser) { - const followees = await this.userFollowingService.getFollowees(me.id); - const followingChannels = await this.channelFollowingsRepository.find({ - where: { - followerId: me.id, - }, - }); - const query = this.queryService.makePaginationQuery(this.notesRepository.createQueryBuilder('note'), ps.sinceId, ps.untilId) - .andWhere(new Brackets(qb => { - if (followees.length > 0) { - const meOrFolloweeIds = [me.id, ...followees.map(f => f.followeeId)]; - qb.where('note.userId IN (:...meOrFolloweeIds)', { meOrFolloweeIds: meOrFolloweeIds }); - qb.orWhere('(note.visibility = \'public\') AND (note.userHost IS NULL)'); - } else { - qb.where('note.userId = :meId', { meId: me.id }); - qb.orWhere('(note.visibility = \'public\') AND (note.userHost IS NULL)'); - } - })) + // 1. by a user I follow, 2. a public local post, 3. my own post + .andWhere(new Brackets(qb => this.queryService + .orFollowingUser(qb, ':meId', 'note.userId') + .orWhere(new Brackets(qbb => qbb + .andWhere('note.visibility = \'public\'') + .andWhere('note.userHost IS NULL'))) + .orWhere(':meId = note.userId'))) + // 1. in a channel I follow, 2. not in a channel + .andWhere(new Brackets(qb => this.queryService + .orFollowingChannel(qb, ':meId', 'note.channelId') + .orWhere('note.channelId IS NULL'))) + .setParameters({ meId: me.id }) .innerJoinAndSelect('note.user', 'user') .leftJoinAndSelect('note.reply', 'reply') .leftJoinAndSelect('note.renote', 'renote') .leftJoinAndSelect('reply.user', 'replyUser') - .leftJoinAndSelect('renote.user', 'renoteUser'); - - if (followingChannels.length > 0) { - const followingChannelIds = followingChannels.map(x => x.followeeId); - - query.andWhere(new Brackets(qb => { - qb.where('note.channelId IN (:...followingChannelIds)', { followingChannelIds }); - qb.orWhere('note.channelId IS NULL'); - })); - } else { - query.andWhere('note.channelId IS NULL'); - } + .leftJoinAndSelect('renote.user', 'renoteUser') + .limit(ps.limit); if (!ps.withReplies) { - query.andWhere(new Brackets(qb => { - qb - .where('note.replyId IS NULL') // 返信ではない - .orWhere(new Brackets(qb => { - qb // 返信だけど投稿者自身への返信 - .where('note.replyId IS NOT NULL') - .andWhere('note.replyUserId = note.userId'); - })); - })); + query + // 1. Not a reply, 2. a self-reply + .andWhere(new Brackets(qb => qb + .orWhere('note.replyId IS NULL') // 返信ではない + .orWhere('note.replyUserId = note.userId'))); } this.queryService.generateVisibilityQuery(query, me); this.queryService.generateBlockedHostQueryForNote(query); + this.queryService.generateSilencedUserQueryForNotes(query, me); this.queryService.generateMutedUserQueryForNotes(query, me); this.queryService.generateBlockedUserQueryForNotes(query, me); - this.queryService.generateMutedUserRenotesQueryForNotes(query, me); - - if (ps.includeMyRenotes === false) { - query.andWhere(new Brackets(qb => { - qb.orWhere('note.userId != :meId', { meId: me.id }); - qb.orWhere('note.renoteId IS NULL'); - qb.orWhere('note.text IS NOT NULL'); - qb.orWhere('note.fileIds != \'{}\''); - qb.orWhere('0 < (SELECT COUNT(*) FROM poll WHERE poll."noteId" = note.id)'); - })); - } - - if (ps.includeRenotedMyNotes === false) { - query.andWhere(new Brackets(qb => { - qb.orWhere('note.renoteUserId != :meId', { meId: me.id }); - qb.orWhere('note.renoteId IS NULL'); - qb.orWhere('note.text IS NOT NULL'); - qb.orWhere('note.fileIds != \'{}\''); - qb.orWhere('0 < (SELECT COUNT(*) FROM poll WHERE poll."noteId" = note.id)'); - })); - } - - if (ps.includeLocalRenotes === false) { - query.andWhere(new Brackets(qb => { - qb.orWhere('note.renoteUserHost IS NOT NULL'); - qb.orWhere('note.renoteId IS NULL'); - qb.orWhere('note.text IS NOT NULL'); - qb.orWhere('note.fileIds != \'{}\''); - qb.orWhere('0 < (SELECT COUNT(*) FROM poll WHERE poll."noteId" = note.id)'); - })); - } if (ps.withFiles) { query.andWhere('note.fileIds != \'{}\''); } if (!ps.withBots) query.andWhere('user.isBot = FALSE'); + + if (!ps.withRenotes) { + this.queryService.generateExcludedRenotesQueryForNotes(query); + } else { + this.queryService.generateMutedUserRenotesQueryForNotes(query, me); + } //#endregion - return await query.limit(ps.limit).getMany(); + return await query.getMany(); } } diff --git a/packages/backend/src/server/api/endpoints/notes/local-timeline.ts b/packages/backend/src/server/api/endpoints/notes/local-timeline.ts index f55853f3f3..41b1ee1086 100644 --- a/packages/backend/src/server/api/endpoints/notes/local-timeline.ts +++ b/packages/backend/src/server/api/endpoints/notes/local-timeline.ts @@ -103,13 +103,14 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- withFiles: ps.withFiles, withReplies: ps.withReplies, withBots: ps.withBots, + withRenotes: ps.withRenotes, }, me); - process.nextTick(() => { - if (me) { + if (me) { + process.nextTick(() => { this.activeUsersChart.read(me); - } - }); + }); + } return await this.noteEntityService.packMany(timeline, me); } @@ -136,14 +137,15 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- withFiles: ps.withFiles, withReplies: ps.withReplies, withBots: ps.withBots, + withRenotes: ps.withRenotes, }, me), }); - process.nextTick(() => { - if (me) { + if (me) { + process.nextTick(() => { this.activeUsersChart.read(me); - } - }); + }); + } return timeline; }); @@ -156,40 +158,47 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- withFiles: boolean, withReplies: boolean, withBots: boolean, + withRenotes: boolean, }, me: MiLocalUser | null) { const query = this.queryService.makePaginationQuery(this.notesRepository.createQueryBuilder('note'), ps.sinceId, ps.untilId) - .andWhere('(note.visibility = \'public\') AND (note.userHost IS NULL) AND (note.channelId IS NULL)') + .andWhere('note.visibility = \'public\'') + .andWhere('note.channelId IS NULL') + .andWhere('note.userHost IS NULL') .innerJoinAndSelect('note.user', 'user') .leftJoinAndSelect('note.reply', 'reply') .leftJoinAndSelect('note.renote', 'renote') .leftJoinAndSelect('reply.user', 'replyUser') - .leftJoinAndSelect('renote.user', 'renoteUser'); + .leftJoinAndSelect('renote.user', 'renoteUser') + .limit(ps.limit); + + if (!ps.withReplies) { + query + // 1. Not a reply, 2. a self-reply + .andWhere(new Brackets(qb => qb + .orWhere('note.replyId IS NULL') // 返信ではない + .orWhere('note.replyUserId = note.userId'))); + } - this.queryService.generateVisibilityQuery(query, me); this.queryService.generateBlockedHostQueryForNote(query); - if (me) this.queryService.generateMutedUserQueryForNotes(query, me); - if (me) this.queryService.generateBlockedUserQueryForNotes(query, me); - if (me) this.queryService.generateMutedUserRenotesQueryForNotes(query, me); + this.queryService.generateSilencedUserQueryForNotes(query, me); + if (me) { + this.queryService.generateMutedUserQueryForNotes(query, me); + this.queryService.generateBlockedUserQueryForNotes(query, me); + } if (ps.withFiles) { query.andWhere('note.fileIds != \'{}\''); } - if (!ps.withReplies) { - query.andWhere(new Brackets(qb => { - qb - .where('note.replyId IS NULL') // 返信ではない - .orWhere(new Brackets(qb => { - qb // 返信だけど投稿者自身への返信 - .where('note.replyId IS NOT NULL') - .andWhere('note.replyUserId = note.userId'); - })); - })); - } - if (!ps.withBots) query.andWhere('user.isBot = FALSE'); - return await query.limit(ps.limit).getMany(); + if (!ps.withRenotes) { + this.queryService.generateExcludedRenotesQueryForNotes(query); + } else if (me) { + this.queryService.generateMutedUserRenotesQueryForNotes(query, me); + } + + return await query.getMany(); } } diff --git a/packages/backend/src/server/api/endpoints/notes/mentions.ts b/packages/backend/src/server/api/endpoints/notes/mentions.ts index 269b57366c..f30e5a583f 100644 --- a/packages/backend/src/server/api/endpoints/notes/mentions.ts +++ b/packages/backend/src/server/api/endpoints/notes/mentions.ts @@ -6,10 +6,12 @@ import { Brackets } from 'typeorm'; import { Inject, Injectable } from '@nestjs/common'; import type { NotesRepository, FollowingsRepository } from '@/models/_.js'; +import { MiNote } from '@/models/_.js'; import { Endpoint } from '@/server/api/endpoint-base.js'; import { QueryService } from '@/core/QueryService.js'; import { NoteEntityService } from '@/core/entities/NoteEntityService.js'; import { DI } from '@/di-symbols.js'; +import ActiveUsersChart from '@/core/chart/charts/active-users.js'; export const meta = { tags: ['notes'], @@ -57,42 +59,58 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- private noteEntityService: NoteEntityService, private queryService: QueryService, + private readonly activeUsersChart: ActiveUsersChart, ) { super(meta, paramDef, async (ps, me) => { - const followingQuery = this.followingsRepository.createQueryBuilder('following') - .select('following.followeeId') - .where('following.followerId = :followerId', { followerId: me.id }); - const query = this.queryService.makePaginationQuery(this.notesRepository.createQueryBuilder('note'), ps.sinceId, ps.untilId) - .andWhere(new Brackets(qb => { - qb // このmeIdAsListパラメータはqueryServiceのgenerateVisibilityQueryでセットされる - .where(':meIdAsList <@ note.mentions') - .orWhere(':meIdAsList <@ note.visibleUserIds'); - })) - // Avoid scanning primary key index - .orderBy('CONCAT(note.id)', 'DESC') + .innerJoin(qb => { + qb + .select('note.id', 'id') + .from(qbb => qbb + .select('note.id', 'id') + .from(MiNote, 'note') + .where(new Brackets(qbbb => qbbb + // DM to me + .orWhere(':meIdAsList <@ note.visibleUserIds') + // Mentions me + .orWhere(':meIdAsList <@ note.mentions'), + )) + .setParameters({ meIdAsList: [me.id] }) + , 'source') + .innerJoin(MiNote, 'note', 'note.id = source.id'); + + this.queryService.generateVisibilityQuery(qb, me); + this.queryService.generateBlockedHostQueryForNote(qb); + this.queryService.generateMutedUserQueryForNotes(qb, me); + this.queryService.generateMutedNoteThreadQuery(qb, me); + this.queryService.generateBlockedUserQueryForNotes(qb, me); + // A renote can't mention a user, so it will never appear here anyway. + //this.queryService.generateMutedUserRenotesQueryForNotes(qb, me); + + if (ps.visibility) { + qb.andWhere('note.visibility = :visibility', { visibility: ps.visibility }); + } + + if (ps.following) { + this.queryService + .andFollowingUser(qb, ':meId', 'note.userId') + .setParameters({ meId: me.id }); + } + + return qb; + }, 'source', 'source.id = note.id') .innerJoinAndSelect('note.user', 'user') .leftJoinAndSelect('note.reply', 'reply') .leftJoinAndSelect('note.renote', 'renote') .leftJoinAndSelect('reply.user', 'replyUser') - .leftJoinAndSelect('renote.user', 'renoteUser'); - - this.queryService.generateVisibilityQuery(query, me); - this.queryService.generateBlockedHostQueryForNote(query); - this.queryService.generateMutedUserQueryForNotes(query, me); - this.queryService.generateMutedNoteThreadQuery(query, me); - this.queryService.generateBlockedUserQueryForNotes(query, me); - - if (ps.visibility) { - query.andWhere('note.visibility = :visibility', { visibility: ps.visibility }); - } + .leftJoinAndSelect('renote.user', 'renoteUser') + .limit(ps.limit); - if (ps.following) { - query.andWhere(`((note.userId IN (${ followingQuery.getQuery() })) OR (note.userId = :meId))`, { meId: me.id }); - query.setParameters(followingQuery.getParameters()); - } + const mentions = await query.getMany(); - const mentions = await query.limit(ps.limit).getMany(); + process.nextTick(() => { + this.activeUsersChart.read(me); + }); return await this.noteEntityService.packMany(mentions, me); }); diff --git a/packages/backend/src/server/api/endpoints/notes/polls/recommendation.ts b/packages/backend/src/server/api/endpoints/notes/polls/recommendation.ts index 33a9c281b3..6f96821a63 100644 --- a/packages/backend/src/server/api/endpoints/notes/polls/recommendation.ts +++ b/packages/backend/src/server/api/endpoints/notes/polls/recommendation.ts @@ -9,13 +9,13 @@ import type { NotesRepository, MutingsRepository, PollsRepository, PollVotesRepo import { Endpoint } from '@/server/api/endpoint-base.js'; import { NoteEntityService } from '@/core/entities/NoteEntityService.js'; import { DI } from '@/di-symbols.js'; +import { QueryService } from '@/core/QueryService.js'; +import { RoleService } from '@/core/RoleService.js'; +import { ApiError } from '@/server/api/error.js'; export const meta = { tags: ['notes'], - requireCredential: true, - kind: 'read:account', - res: { type: 'array', optional: false, nullable: false, @@ -26,10 +26,24 @@ export const meta = { }, }, - // 2 calls per second + errors: { + ltlDisabled: { + message: 'Local timeline has been disabled.', + code: 'LTL_DISABLED', + id: '45a6eb02-7695-4393-b023-dd3be9aaaefd', + }, + gtlDisabled: { + message: 'Global timeline has been disabled.', + code: 'GTL_DISABLED', + id: '0332fc13-6ab2-4427-ae80-a9fadffd1a6b', + }, + }, + + // Up to 10 calls, then 2 per second limit: { - duration: 1000, - max: 2, + type: 'bucket', + size: 10, + dripRate: 500, }, } as const; @@ -39,6 +53,8 @@ export const paramDef = { limit: { type: 'integer', minimum: 1, maximum: 100, default: 10 }, offset: { type: 'integer', default: 0 }, excludeChannels: { type: 'boolean', default: false }, + local: { type: 'boolean', nullable: true, default: null }, + expired: { type: 'boolean', default: false }, }, required: [], } as const; @@ -59,18 +75,54 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- private mutingsRepository: MutingsRepository, private noteEntityService: NoteEntityService, + private readonly queryService: QueryService, + private readonly roleService: RoleService, ) { super(meta, paramDef, async (ps, me) => { const query = this.pollsRepository.createQueryBuilder('poll') - .where('poll.userHost IS NULL') - .andWhere('poll.userId != :meId', { meId: me.id }) - .andWhere('poll.noteVisibility = \'public\'') - .andWhere(new Brackets(qb => { + .innerJoinAndSelect('poll.note', 'note') + .innerJoinAndSelect('note.user', 'user') + .leftJoinAndSelect('note.renote', 'renote') + .leftJoinAndSelect('note.reply', 'reply') + .leftJoinAndSelect('renote.user', 'renoteUser') + .leftJoinAndSelect('reply.user', 'replyUser') + .andWhere('user.isExplorable = TRUE') + ; + + if (me) { + query.andWhere('poll.userId != :meId', { meId: me.id }); + } + + if (ps.expired) { + query.andWhere('poll.expiresAt IS NOT NULL'); + query.andWhere('poll.expiresAt <= :expiresMax', { + expiresMax: new Date(), + }); + query.andWhere('poll.expiresAt >= :expiresMin', { + expiresMin: new Date(Date.now() - (1000 * 60 * 60 * 24 * 7)), + }); + } else { + query.andWhere(new Brackets(qb => { qb .where('poll.expiresAt IS NULL') .orWhere('poll.expiresAt > :now', { now: new Date() }); })); + } + + const policies = await this.roleService.getUserPolicies(me?.id ?? null); + if (ps.local != null) { + if (ps.local) { + if (!policies.ltlAvailable) throw new ApiError(meta.errors.ltlDisabled); + query.andWhere('poll.userHost IS NULL'); + } else { + if (!policies.gtlAvailable) throw new ApiError(meta.errors.gtlDisabled); + query.andWhere('poll.userHost IS NOT NULL'); + } + } else { + if (!policies.gtlAvailable) throw new ApiError(meta.errors.gtlDisabled); + } + /* //#region exclude arleady voted polls const votedQuery = this.pollVotesRepository.createQueryBuilder('vote') .select('vote.noteId') @@ -81,16 +133,15 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- query.setParameters(votedQuery.getParameters()); //#endregion + */ - //#region mute - const mutingQuery = this.mutingsRepository.createQueryBuilder('muting') - .select('muting.muteeId') - .where('muting.muterId = :muterId', { muterId: me.id }); - - query - .andWhere(`poll.userId NOT IN (${ mutingQuery.getQuery() })`); - - query.setParameters(mutingQuery.getParameters()); + //#region block/mute/vis + this.queryService.generateVisibilityQuery(query, me); + this.queryService.generateBlockedHostQueryForNote(query); + if (me) { + this.queryService.generateBlockedUserQueryForNotes(query, me); + this.queryService.generateMutedUserQueryForNotes(query, me); + } //#endregion //#region exclude channels @@ -107,6 +158,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- if (polls.length === 0) return []; + /* const notes = await this.notesRepository.find({ where: { id: In(polls.map(poll => poll.noteId)), @@ -115,6 +167,10 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- id: 'DESC', }, }); + */ + + // eslint-disable-next-line @typescript-eslint/no-non-null-assertion + const notes = polls.map(poll => poll.note!); return await this.noteEntityService.packMany(notes, me, { detail: true, diff --git a/packages/backend/src/server/api/endpoints/notes/renotes.ts b/packages/backend/src/server/api/endpoints/notes/renotes.ts index 0f08cc9cf2..be7cb0320f 100644 --- a/packages/backend/src/server/api/endpoints/notes/renotes.ts +++ b/packages/backend/src/server/api/endpoints/notes/renotes.ts @@ -47,7 +47,7 @@ export const paramDef = { type: 'object', properties: { noteId: { type: 'string', format: 'misskey:id' }, - userId: { type: "string", format: "misskey:id" }, + userId: { type: 'string', format: 'misskey:id' }, limit: { type: 'integer', minimum: 1, maximum: 100, default: 10 }, sinceId: { type: 'string', format: 'misskey:id' }, untilId: { type: 'string', format: 'misskey:id' }, @@ -81,19 +81,21 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- .leftJoinAndSelect('renote.user', 'renoteUser'); if (ps.userId) { - query.andWhere("user.id = :userId", { userId: ps.userId }); + query.andWhere('user.id = :userId', { userId: ps.userId }); } if (ps.quote) { - query.andWhere("note.text IS NOT NULL"); + this.queryService.andIsQuote(query, 'note'); } else { - query.andWhere("note.text IS NULL"); + this.queryService.andIsRenote(query, 'note'); } this.queryService.generateVisibilityQuery(query, me); this.queryService.generateBlockedHostQueryForNote(query); - if (me) this.queryService.generateMutedUserQueryForNotes(query, me); - if (me) this.queryService.generateBlockedUserQueryForNotes(query, me); + if (me) { + this.queryService.generateMutedUserQueryForNotes(query, me); + this.queryService.generateBlockedUserQueryForNotes(query, me); + } const renotes = await query.limit(ps.limit).getMany(); diff --git a/packages/backend/src/server/api/endpoints/notes/replies.ts b/packages/backend/src/server/api/endpoints/notes/replies.ts index 0882e19182..f79bfaa7df 100644 --- a/packages/backend/src/server/api/endpoints/notes/replies.ts +++ b/packages/backend/src/server/api/endpoints/notes/replies.ts @@ -59,14 +59,17 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- .leftJoinAndSelect('note.reply', 'reply') .leftJoinAndSelect('note.renote', 'renote') .leftJoinAndSelect('reply.user', 'replyUser') - .leftJoinAndSelect('renote.user', 'renoteUser'); + .leftJoinAndSelect('renote.user', 'renoteUser') + .limit(ps.limit); this.queryService.generateVisibilityQuery(query, me); this.queryService.generateBlockedHostQueryForNote(query); - if (me) this.queryService.generateMutedUserQueryForNotes(query, me); - if (me) this.queryService.generateBlockedUserQueryForNotes(query, me); + if (me) { + this.queryService.generateMutedUserQueryForNotes(query, me); + this.queryService.generateBlockedUserQueryForNotes(query, me); + } - const timeline = await query.limit(ps.limit).getMany(); + const timeline = await query.getMany(); return await this.noteEntityService.packMany(timeline, me); }); diff --git a/packages/backend/src/server/api/endpoints/notes/search-by-tag.ts b/packages/backend/src/server/api/endpoints/notes/search-by-tag.ts index 91874a8195..5064144d9c 100644 --- a/packages/backend/src/server/api/endpoints/notes/search-by-tag.ts +++ b/packages/backend/src/server/api/endpoints/notes/search-by-tag.ts @@ -12,8 +12,6 @@ import { Endpoint } from '@/server/api/endpoint-base.js'; import { QueryService } from '@/core/QueryService.js'; import { NoteEntityService } from '@/core/entities/NoteEntityService.js'; import { DI } from '@/di-symbols.js'; -import { CacheService } from '@/core/CacheService.js'; -import { UtilityService } from '@/core/UtilityService.js'; export const meta = { tags: ['notes', 'hashtags'], @@ -82,26 +80,26 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- private noteEntityService: NoteEntityService, private queryService: QueryService, - private cacheService: CacheService, - private utilityService: UtilityService, ) { super(meta, paramDef, async (ps, me) => { const query = this.queryService.makePaginationQuery(this.notesRepository.createQueryBuilder('note'), ps.sinceId, ps.untilId) - .andWhere("note.visibility IN ('public', 'home')") // keep in sync with NoteCreateService call to `hashtagService.updateHashtags()` + .andWhere(new Brackets(qb => qb + .orWhere('note.visibility = \'public\'') + .orWhere('note.visibility = \'home\''))) // keep in sync with NoteCreateService call to `hashtagService.updateHashtags()` .innerJoinAndSelect('note.user', 'user') .leftJoinAndSelect('note.reply', 'reply') .leftJoinAndSelect('note.renote', 'renote') .leftJoinAndSelect('reply.user', 'replyUser') - .leftJoinAndSelect('renote.user', 'renoteUser'); + .leftJoinAndSelect('renote.user', 'renoteUser') + .limit(ps.limit); - if (!this.serverSettings.enableBotTrending) query.andWhere('user.isBot = FALSE'); - - this.queryService.generateVisibilityQuery(query, me); this.queryService.generateBlockedHostQueryForNote(query); + this.queryService.generateSilencedUserQueryForNotes(query, me); if (me) this.queryService.generateMutedUserQueryForNotes(query, me); if (me) this.queryService.generateBlockedUserQueryForNotes(query, me); + if (me) this.queryService.generateMutedUserRenotesQueryForNotes(query, me); - const followings = me ? await this.cacheService.userFollowingsCache.fetch(me.id) : {}; + if (!this.serverSettings.enableBotTrending) query.andWhere('user.isBot = FALSE'); try { if (ps.tag) { @@ -134,9 +132,9 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- if (ps.renote != null) { if (ps.renote) { - query.andWhere('note.renoteId IS NOT NULL'); + this.queryService.andIsRenote(query, 'note'); } else { - query.andWhere('note.renoteId IS NULL'); + this.queryService.andIsNotRenote(query, 'note'); } } @@ -153,17 +151,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- } // Search notes - let notes = await query.limit(ps.limit).getMany(); - - notes = notes.filter(note => { - if (note.user?.isSilenced && me && followings && note.userId !== me.id && !followings[note.userId]) return false; - if (note.user?.isSuspended) return false; - if (note.userHost) { - if (!this.utilityService.isFederationAllowedHost(note.userHost)) return false; - if (this.utilityService.isSilencedHost(this.serverSettings.silencedHosts, note.userHost)) return false; - } - return true; - }); + const notes = await query.getMany(); return await this.noteEntityService.packMany(notes, me); }); diff --git a/packages/backend/src/server/api/endpoints/notes/timeline.ts b/packages/backend/src/server/api/endpoints/notes/timeline.ts index a2dfa7fdac..44c539eaad 100644 --- a/packages/backend/src/server/api/endpoints/notes/timeline.ts +++ b/packages/backend/src/server/api/endpoints/notes/timeline.ts @@ -49,9 +49,6 @@ export const paramDef = { sinceDate: { type: 'integer' }, untilDate: { type: 'integer' }, allowPartial: { type: 'boolean', default: false }, // true is recommended but for compatibility false by default - includeMyRenotes: { type: 'boolean', default: true }, - includeRenotedMyNotes: { type: 'boolean', default: true }, - includeLocalRenotes: { type: 'boolean', default: true }, withFiles: { type: 'boolean', default: false }, withRenotes: { type: 'boolean', default: true }, withBots: { type: 'boolean', default: true }, @@ -88,9 +85,6 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- untilId, sinceId, limit: ps.limit, - includeMyRenotes: ps.includeMyRenotes, - includeRenotedMyNotes: ps.includeRenotedMyNotes, - includeLocalRenotes: ps.includeLocalRenotes, withFiles: ps.withFiles, withRenotes: ps.withRenotes, withBots: ps.withBots, @@ -121,7 +115,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- excludePureRenotes: !ps.withRenotes, noteFilter: note => { if (note.reply && note.reply.visibility === 'followers') { - if (!Object.hasOwn(followings, note.reply.userId) && note.reply.userId !== me.id) return false; + if (!followings.has(note.reply.userId) && note.reply.userId !== me.id) return false; } if (!ps.withBots && note.user?.isBot) return false; @@ -131,9 +125,6 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- untilId, sinceId, limit, - includeMyRenotes: ps.includeMyRenotes, - includeRenotedMyNotes: ps.includeRenotedMyNotes, - includeLocalRenotes: ps.includeLocalRenotes, withFiles: ps.withFiles, withRenotes: ps.withRenotes, withBots: ps.withBots, @@ -148,113 +139,48 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- }); } - private async getFromDb(ps: { untilId: string | null; sinceId: string | null; limit: number; includeMyRenotes: boolean; includeRenotedMyNotes: boolean; includeLocalRenotes: boolean; withFiles: boolean; withRenotes: boolean; withBots: boolean; }, me: MiLocalUser) { - const followees = await this.userFollowingService.getFollowees(me.id); - const followingChannels = await this.channelFollowingsRepository.find({ - where: { - followerId: me.id, - }, - }); - + private async getFromDb(ps: { untilId: string | null; sinceId: string | null; limit: number; withFiles: boolean; withRenotes: boolean; withBots: boolean; }, me: MiLocalUser) { //#region Construct query const query = this.queryService.makePaginationQuery(this.notesRepository.createQueryBuilder('note'), ps.sinceId, ps.untilId) + // 1. in a channel I follow, 2. my own post, 3. by a user I follow + .andWhere(new Brackets(qb => this.queryService + .orFollowingChannel(qb, ':meId', 'note.channelId') + .orWhere(':meId = note.userId') + .orWhere(new Brackets(qb2 => this.queryService + .andFollowingUser(qb2, ':meId', 'note.userId') + .andWhere('note.channelId IS NULL'))), + )) + // 1. Not a reply, 2. a self-reply + .andWhere(new Brackets(qb => qb + .orWhere('note.replyId IS NULL') // 返信ではない + .orWhere('note.replyUserId = note.userId'))) + .setParameters({ meId: me.id }) .innerJoinAndSelect('note.user', 'user') .leftJoinAndSelect('note.reply', 'reply') .leftJoinAndSelect('note.renote', 'renote') .leftJoinAndSelect('reply.user', 'replyUser') - .leftJoinAndSelect('renote.user', 'renoteUser'); - - if (followees.length > 0 && followingChannels.length > 0) { - // ユーザー・チャンネルともにフォローあり - const meOrFolloweeIds = [me.id, ...followees.map(f => f.followeeId)]; - const followingChannelIds = followingChannels.map(x => x.followeeId); - query.andWhere(new Brackets(qb => { - qb - .where(new Brackets(qb2 => { - qb2 - .where('note.userId IN (:...meOrFolloweeIds)', { meOrFolloweeIds: meOrFolloweeIds }) - .andWhere('note.channelId IS NULL'); - })) - .orWhere('note.channelId IN (:...followingChannelIds)', { followingChannelIds }); - })); - } else if (followees.length > 0) { - // ユーザーフォローのみ（チャンネルフォローなし） - const meOrFolloweeIds = [me.id, ...followees.map(f => f.followeeId)]; - query - .andWhere('note.channelId IS NULL') - .andWhere('note.userId IN (:...meOrFolloweeIds)', { meOrFolloweeIds: meOrFolloweeIds }); - } else if (followingChannels.length > 0) { - // チャンネルフォローのみ（ユーザーフォローなし） - const followingChannelIds = followingChannels.map(x => x.followeeId); - query.andWhere(new Brackets(qb => { - qb - .where('note.channelId IN (:...followingChannelIds)', { followingChannelIds }) - .orWhere('note.userId = :meId', { meId: me.id }); - })); - } else { - // フォローなし - query - .andWhere('note.channelId IS NULL') - .andWhere('note.userId = :meId', { meId: me.id }); - } - - query.andWhere(new Brackets(qb => { - qb - .where('note.replyId IS NULL') // 返信ではない - .orWhere(new Brackets(qb => { - qb // 返信だけど投稿者自身への返信 - .where('note.replyId IS NOT NULL') - .andWhere('note.replyUserId = note.userId'); - })); - })); + .leftJoinAndSelect('renote.user', 'renoteUser') + .limit(ps.limit); this.queryService.generateVisibilityQuery(query, me); this.queryService.generateBlockedHostQueryForNote(query); + this.queryService.generateSilencedUserQueryForNotes(query, me); this.queryService.generateMutedUserQueryForNotes(query, me); this.queryService.generateBlockedUserQueryForNotes(query, me); - this.queryService.generateMutedUserRenotesQueryForNotes(query, me); - - if (ps.includeMyRenotes === false) { - query.andWhere(new Brackets(qb => { - qb.orWhere('note.userId != :meId', { meId: me.id }); - qb.orWhere('note.renoteId IS NULL'); - qb.orWhere('note.text IS NOT NULL'); - qb.orWhere('note.fileIds != \'{}\''); - qb.orWhere('0 < (SELECT COUNT(*) FROM poll WHERE poll."noteId" = note.id)'); - })); - } - - if (ps.includeRenotedMyNotes === false) { - query.andWhere(new Brackets(qb => { - qb.orWhere('note.renoteUserId != :meId', { meId: me.id }); - qb.orWhere('note.renoteId IS NULL'); - qb.orWhere('note.text IS NOT NULL'); - qb.orWhere('note.fileIds != \'{}\''); - qb.orWhere('0 < (SELECT COUNT(*) FROM poll WHERE poll."noteId" = note.id)'); - })); - } - - if (ps.includeLocalRenotes === false) { - query.andWhere(new Brackets(qb => { - qb.orWhere('note.renoteUserHost IS NOT NULL'); - qb.orWhere('note.renoteId IS NULL'); - qb.orWhere('note.text IS NOT NULL'); - qb.orWhere('note.fileIds != \'{}\''); - qb.orWhere('0 < (SELECT COUNT(*) FROM poll WHERE poll."noteId" = note.id)'); - })); - } if (ps.withFiles) { query.andWhere('note.fileIds != \'{}\''); } - if (ps.withRenotes === false) { - query.andWhere('note.renoteId IS NULL'); - } - if (!ps.withBots) query.andWhere('user.isBot = FALSE'); + + if (!ps.withRenotes) { + this.queryService.generateExcludedRenotesQueryForNotes(query); + } else { + this.queryService.generateMutedUserRenotesQueryForNotes(query, me); + } //#endregion - return await query.limit(ps.limit).getMany(); + return await query.getMany(); } } diff --git a/packages/backend/src/server/api/endpoints/notes/translate.ts b/packages/backend/src/server/api/endpoints/notes/translate.ts index a97542c063..5ebd5ef362 100644 --- a/packages/backend/src/server/api/endpoints/notes/translate.ts +++ b/packages/backend/src/server/api/endpoints/notes/translate.ts @@ -20,11 +20,9 @@ import { ApiError } from '../../error.js'; export const meta = { tags: ['notes'], - // TODO allow unauthenticated if default template allows? - // Maybe a value 'optional' that allows unauthenticated OR a token w/ appropriate role. - // This will allow unauthenticated requests without leaking post data to restricted clients. - requireCredential: true, + requireCredential: 'optional', kind: 'read:account', + requiredRolePolicy: 'canUseTranslator', res: { type: 'object', @@ -88,17 +86,12 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- private readonly loggerService: ApiLoggerService, ) { super(meta, paramDef, async (ps, me) => { - const policies = await this.roleService.getUserPolicies(me.id); - if (!policies.canUseTranslator) { - throw new ApiError(meta.errors.unavailable); - } - const note = await this.getterService.getNote(ps.noteId).catch(err => { if (err.id === '9725d0ce-ba28-4dde-95a7-2cbb2c15de24') throw new ApiError(meta.errors.noSuchNote); throw err; }); - if (!(await this.noteEntityService.isVisibleForMe(note, me.id))) { + if (!(await this.noteEntityService.isVisibleForMe(note, me?.id ?? null, { me }))) { throw new ApiError(meta.errors.cannotTranslateInvisibleNote); } @@ -140,7 +133,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- if (this.serverSettings.deeplAuthKey) params.append('auth_key', this.serverSettings.deeplAuthKey); params.append('text', note.text); params.append('target_lang', targetLang); - const endpoint = deeplFreeInstance ?? this.serverSettings.deeplIsPro ? 'https://api.deepl.com/v2/translate' : 'https://api-free.deepl.com/v2/translate'; + const endpoint = deeplFreeInstance ?? ( this.serverSettings.deeplIsPro ? 'https://api.deepl.com/v2/translate' : 'https://api-free.deepl.com/v2/translate' ); const res = await this.httpRequestService.send(endpoint, { method: 'POST', diff --git a/packages/backend/src/server/api/endpoints/notes/user-list-timeline.ts b/packages/backend/src/server/api/endpoints/notes/user-list-timeline.ts index 60f18a09b0..0f038e5541 100644 --- a/packages/backend/src/server/api/endpoints/notes/user-list-timeline.ts +++ b/packages/backend/src/server/api/endpoints/notes/user-list-timeline.ts @@ -57,9 +57,6 @@ export const paramDef = { sinceDate: { type: 'integer' }, untilDate: { type: 'integer' }, allowPartial: { type: 'boolean', default: false }, // true is recommended but for compatibility false by default - includeMyRenotes: { type: 'boolean', default: true }, - includeRenotedMyNotes: { type: 'boolean', default: true }, - includeLocalRenotes: { type: 'boolean', default: true }, withRenotes: { type: 'boolean', default: true }, withFiles: { type: 'boolean', @@ -109,14 +106,13 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- untilId, sinceId, limit: ps.limit, - includeMyRenotes: ps.includeMyRenotes, - includeRenotedMyNotes: ps.includeRenotedMyNotes, - includeLocalRenotes: ps.includeLocalRenotes, withFiles: ps.withFiles, withRenotes: ps.withRenotes, }, me); - this.activeUsersChart.read(me); + process.nextTick(() => { + this.activeUsersChart.read(me); + }); return await this.noteEntityService.packMany(timeline, me); } @@ -135,15 +131,14 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- untilId, sinceId, limit, - includeMyRenotes: ps.includeMyRenotes, - includeRenotedMyNotes: ps.includeRenotedMyNotes, - includeLocalRenotes: ps.includeLocalRenotes, withFiles: ps.withFiles, withRenotes: ps.withRenotes, }, me), }); - this.activeUsersChart.read(me); + process.nextTick(() => { + this.activeUsersChart.read(me); + }); return timeline; }); @@ -153,93 +148,49 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- untilId: string | null, sinceId: string | null, limit: number, - includeMyRenotes: boolean, - includeRenotedMyNotes: boolean, - includeLocalRenotes: boolean, withFiles: boolean, withRenotes: boolean, }, me: MiLocalUser) { //#region Construct query const query = this.queryService.makePaginationQuery(this.notesRepository.createQueryBuilder('note'), ps.sinceId, ps.untilId) .innerJoin(this.userListMembershipsRepository.metadata.targetName, 'userListMemberships', 'userListMemberships.userId = note.userId') + .andWhere('userListMemberships.userListId = :userListId', { userListId: list.id }) + .andWhere('note.channelId IS NULL') // チャンネルノートではない + .andWhere(new Brackets(qb => qb + // 返信ではない + .orWhere('note.replyId IS NULL') + // 返信だけど投稿者自身への返信 + .orWhere('note.replyUserId = note.userId') + // 返信だけど自分宛ての返信 + .orWhere('note.replyUserId = :meId') + // 返信だけどwithRepliesがtrueの場合 + .orWhere('userListMemberships.withReplies = true'), + )) + .setParameters({ meId: me.id }) .innerJoinAndSelect('note.user', 'user') .leftJoinAndSelect('note.reply', 'reply') .leftJoinAndSelect('note.renote', 'renote') .leftJoinAndSelect('reply.user', 'replyUser') .leftJoinAndSelect('renote.user', 'renoteUser') - .andWhere('userListMemberships.userListId = :userListId', { userListId: list.id }) - .andWhere('note.channelId IS NULL') // チャンネルノートではない - .andWhere(new Brackets(qb => { - qb - .where('note.replyId IS NULL') // 返信ではない - .orWhere(new Brackets(qb => { - qb // 返信だけど投稿者自身への返信 - .where('note.replyId IS NOT NULL') - .andWhere('note.replyUserId = note.userId'); - })) - .orWhere(new Brackets(qb => { - qb // 返信だけど自分宛ての返信 - .where('note.replyId IS NOT NULL') - .andWhere('note.replyUserId = :meId', { meId: me.id }); - })) - .orWhere(new Brackets(qb => { - qb // 返信だけどwithRepliesがtrueの場合 - .where('note.replyId IS NOT NULL') - .andWhere('userListMemberships.withReplies = true'); - })); - })); + .limit(ps.limit); this.queryService.generateVisibilityQuery(query, me); this.queryService.generateBlockedHostQueryForNote(query); this.queryService.generateMutedUserQueryForNotes(query, me); this.queryService.generateBlockedUserQueryForNotes(query, me); - this.queryService.generateMutedUserRenotesQueryForNotes(query, me); - - if (ps.includeMyRenotes === false) { - query.andWhere(new Brackets(qb => { - qb.orWhere('note.userId != :meId', { meId: me.id }); - qb.orWhere('note.renoteId IS NULL'); - qb.orWhere('note.text IS NOT NULL'); - qb.orWhere('note.fileIds != \'{}\''); - qb.orWhere('0 < (SELECT COUNT(*) FROM poll WHERE poll."noteId" = note.id)'); - })); - } - - if (ps.includeRenotedMyNotes === false) { - query.andWhere(new Brackets(qb => { - qb.orWhere('note.renoteUserId != :meId', { meId: me.id }); - qb.orWhere('note.renoteId IS NULL'); - qb.orWhere('note.text IS NOT NULL'); - qb.orWhere('note.fileIds != \'{}\''); - qb.orWhere('0 < (SELECT COUNT(*) FROM poll WHERE poll."noteId" = note.id)'); - })); - } - if (ps.includeLocalRenotes === false) { - query.andWhere(new Brackets(qb => { - qb.orWhere('note.renoteUserHost IS NOT NULL'); - qb.orWhere('note.renoteId IS NULL'); - qb.orWhere('note.text IS NOT NULL'); - qb.orWhere('note.fileIds != \'{}\''); - qb.orWhere('0 < (SELECT COUNT(*) FROM poll WHERE poll."noteId" = note.id)'); - })); + if (ps.withFiles) { + query.andWhere('note.fileIds != \'{}\''); } - if (ps.withRenotes === false) { - query.andWhere(new Brackets(qb => { - qb.orWhere('note.renoteId IS NULL'); - qb.orWhere(new Brackets(qb => { - qb.orWhere('note.text IS NOT NULL'); - qb.orWhere('note.fileIds != \'{}\''); - })); - })); + if (!ps.withRenotes) { + this.queryService.generateExcludedRenotesQueryForNotes(query); + } else { + this.queryService.generateMutedUserRenotesQueryForNotes(query, me); } - if (ps.withFiles) { - query.andWhere('note.fileIds != \'{}\''); - } //#endregion - return await query.limit(ps.limit).getMany(); + return await query.getMany(); } } diff --git a/packages/backend/src/server/api/endpoints/roles/notes.ts b/packages/backend/src/server/api/endpoints/roles/notes.ts index d1c2e4b686..741bd819ba 100644 --- a/packages/backend/src/server/api/endpoints/roles/notes.ts +++ b/packages/backend/src/server/api/endpoints/roles/notes.ts @@ -12,6 +12,7 @@ import { DI } from '@/di-symbols.js'; import { NoteEntityService } from '@/core/entities/NoteEntityService.js'; import { IdService } from '@/core/IdService.js'; import { FanoutTimelineService } from '@/core/FanoutTimelineService.js'; +import ActiveUsersChart from '@/core/chart/charts/active-users.js'; import { ApiError } from '../../error.js'; export const meta = { @@ -74,6 +75,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- private noteEntityService: NoteEntityService, private queryService: QueryService, private fanoutTimelineService: FanoutTimelineService, + private readonly activeUsersChart: ActiveUsersChart, ) { super(meta, paramDef, async (ps, me) => { const untilId = ps.untilId ?? (ps.untilDate ? this.idService.gen(ps.untilDate!) : null); @@ -101,19 +103,24 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- const query = this.notesRepository.createQueryBuilder('note') .where('note.id IN (:...noteIds)', { noteIds: noteIds }) .andWhere('(note.visibility = \'public\')') + .orderBy('note.id', 'DESC') .innerJoinAndSelect('note.user', 'user') .leftJoinAndSelect('note.reply', 'reply') .leftJoinAndSelect('note.renote', 'renote') .leftJoinAndSelect('reply.user', 'replyUser') .leftJoinAndSelect('renote.user', 'renoteUser'); - this.queryService.generateVisibilityQuery(query, me); this.queryService.generateBlockedHostQueryForNote(query); + this.queryService.generateSilencedUserQueryForNotes(query, me); this.queryService.generateMutedUserQueryForNotes(query, me); this.queryService.generateBlockedUserQueryForNotes(query, me); + this.queryService.generateMutedUserRenotesQueryForNotes(query, me); const notes = await query.getMany(); - notes.sort((a, b) => a.id > b.id ? -1 : 1); + + process.nextTick(() => { + this.activeUsersChart.read(me); + }); return await this.noteEntityService.packMany(notes, me); }); diff --git a/packages/backend/src/server/api/endpoints/sw/register.ts b/packages/backend/src/server/api/endpoints/sw/register.ts index f447b5598b..2f72e6ce1d 100644 --- a/packages/backend/src/server/api/endpoints/sw/register.ts +++ b/packages/backend/src/server/api/endpoints/sw/register.ts @@ -104,7 +104,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- sendReadMessage: ps.sendReadMessage, }); - this.pushNotificationService.refreshCache(me.id); + await this.pushNotificationService.refreshCache(me.id); return { state: 'subscribed' as const, diff --git a/packages/backend/src/server/api/endpoints/sw/unregister.ts b/packages/backend/src/server/api/endpoints/sw/unregister.ts index aa7e03dceb..f43a2cce28 100644 --- a/packages/backend/src/server/api/endpoints/sw/unregister.ts +++ b/packages/backend/src/server/api/endpoints/sw/unregister.ts @@ -46,7 +46,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- }); if (me) { - this.pushNotificationService.refreshCache(me.id); + await this.pushNotificationService.refreshCache(me.id); } }); } diff --git a/packages/backend/src/server/api/endpoints/sw/update-registration.ts b/packages/backend/src/server/api/endpoints/sw/update-registration.ts index 78b9323b7b..0cbed273e8 100644 --- a/packages/backend/src/server/api/endpoints/sw/update-registration.ts +++ b/packages/backend/src/server/api/endpoints/sw/update-registration.ts @@ -86,7 +86,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- sendReadMessage: swSubscription.sendReadMessage, }); - this.pushNotificationService.refreshCache(me.id); + await this.pushNotificationService.refreshCache(me.id); return { userId: swSubscription.userId, diff --git a/packages/backend/src/server/api/endpoints/users/followers.ts b/packages/backend/src/server/api/endpoints/users/followers.ts index c1617e14e5..82ce282bfc 100644 --- a/packages/backend/src/server/api/endpoints/users/followers.ts +++ b/packages/backend/src/server/api/endpoints/users/followers.ts @@ -12,6 +12,7 @@ import { FollowingEntityService } from '@/core/entities/FollowingEntityService.j import { UtilityService } from '@/core/UtilityService.js'; import { DI } from '@/di-symbols.js'; import { RoleService } from '@/core/RoleService.js'; +import { CacheService } from '@/core/CacheService.js'; import { ApiError } from '../../error.js'; export const meta = { @@ -89,6 +90,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- private followingEntityService: FollowingEntityService, private queryService: QueryService, private roleService: RoleService, + private readonly cacheService: CacheService, ) { super(meta, paramDef, async (ps, me) => { const user = await this.usersRepository.findOneBy(ps.userId != null @@ -110,12 +112,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- if (me == null) { throw new ApiError(meta.errors.forbidden); } else if (me.id !== user.id) { - const isFollowing = await this.followingsRepository.exists({ - where: { - followeeId: user.id, - followerId: me.id, - }, - }); + const isFollowing = await this.cacheService.userFollowingsCache.fetch(me.id).then(f => f.has(user.id)); if (!isFollowing) { throw new ApiError(meta.errors.forbidden); } diff --git a/packages/backend/src/server/api/endpoints/users/following.ts b/packages/backend/src/server/api/endpoints/users/following.ts index c292c6d6a3..80f0b0c484 100644 --- a/packages/backend/src/server/api/endpoints/users/following.ts +++ b/packages/backend/src/server/api/endpoints/users/following.ts @@ -13,6 +13,7 @@ import { FollowingEntityService } from '@/core/entities/FollowingEntityService.j import { UtilityService } from '@/core/UtilityService.js'; import { DI } from '@/di-symbols.js'; import { RoleService } from '@/core/RoleService.js'; +import { CacheService } from '@/core/CacheService.js'; import { ApiError } from '../../error.js'; export const meta = { @@ -98,6 +99,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- private followingEntityService: FollowingEntityService, private queryService: QueryService, private roleService: RoleService, + private readonly cacheService: CacheService, ) { super(meta, paramDef, async (ps, me) => { const user = await this.usersRepository.findOneBy(ps.userId != null @@ -119,12 +121,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- if (me == null) { throw new ApiError(meta.errors.forbidden); } else if (me.id !== user.id) { - const isFollowing = await this.followingsRepository.exists({ - where: { - followeeId: user.id, - followerId: me.id, - }, - }); + const isFollowing = await this.cacheService.userFollowingsCache.fetch(me.id).then(f => f.has(user.id)); if (!isFollowing) { throw new ApiError(meta.errors.forbidden); } diff --git a/packages/backend/src/server/api/endpoints/users/notes.ts b/packages/backend/src/server/api/endpoints/users/notes.ts index 965baa859a..4602709067 100644 --- a/packages/backend/src/server/api/endpoints/users/notes.ts +++ b/packages/backend/src/server/api/endpoints/users/notes.ts @@ -134,7 +134,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- if (ps.withReplies) redisTimelines.push(`userTimelineWithReplies:${ps.userId}`); if (ps.withChannelNotes) redisTimelines.push(`userTimelineWithChannel:${ps.userId}`); - const isFollowing = me && Object.hasOwn(await this.cacheService.userFollowingsCache.fetch(me.id), ps.userId); + const isFollowing = me && (await this.cacheService.userFollowingsCache.fetch(me.id)).has(ps.userId); const timeline = await this.fanoutTimelineEndpointService.timeline({ untilId, @@ -205,7 +205,8 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- .leftJoinAndSelect('note.renote', 'renote') .leftJoinAndSelect('note.channel', 'channel') .leftJoinAndSelect('reply.user', 'replyUser') - .leftJoinAndSelect('renote.user', 'renoteUser'); + .leftJoinAndSelect('renote.user', 'renoteUser') + .limit(ps.limit); if (ps.withChannelNotes) { if (!isSelf) query.andWhere(new Brackets(qb => { @@ -230,26 +231,9 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- if (!ps.withRenotes && !ps.withQuotes) { query.andWhere('note.renoteId IS NULL'); } else if (!ps.withRenotes) { - query.andWhere(new Brackets(qb => { - qb.orWhere('note.userId != :userId', { userId: ps.userId }); - qb.orWhere('note.renoteId IS NULL'); - qb.orWhere('note.text IS NOT NULL'); - qb.orWhere('note.fileIds != \'{}\''); - qb.orWhere('0 < (SELECT COUNT(*) FROM poll WHERE poll."noteId" = note.id)'); - })); + this.queryService.andIsNotRenote(query, 'note'); } else if (!ps.withQuotes) { - query.andWhere(` - ( - note."renoteId" IS NULL - OR ( - note.text IS NULL - AND note.cw IS NULL - AND note."replyId" IS NULL - AND note."hasPoll" IS FALSE - AND note."fileIds" = '{}' - ) - ) - `); + this.queryService.andIsNotQuote(query, 'note'); } if (!ps.withRepliesToOthers && !ps.withRepliesToSelf) { @@ -268,6 +252,6 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- query.andWhere('"user"."isBot" = false'); } - return await query.limit(ps.limit).getMany(); + return await query.getMany(); } } diff --git a/packages/backend/src/server/api/endpoints/users/reactions.ts b/packages/backend/src/server/api/endpoints/users/reactions.ts index 56f59bd285..553787ad58 100644 --- a/packages/backend/src/server/api/endpoints/users/reactions.ts +++ b/packages/backend/src/server/api/endpoints/users/reactions.ts @@ -105,10 +105,15 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- const query = this.queryService.makePaginationQuery(this.noteReactionsRepository.createQueryBuilder('reaction'), ps.sinceId, ps.untilId, ps.sinceDate, ps.untilDate) .andWhere('reaction.userId = :userId', { userId: ps.userId }) - .leftJoinAndSelect('reaction.note', 'note'); + .innerJoinAndSelect('reaction.note', 'note'); this.queryService.generateVisibilityQuery(query, me); this.queryService.generateBlockedHostQueryForNote(query); + if (me) { + this.queryService.generateMutedUserQueryForNotes(query, me); + this.queryService.generateBlockedUserQueryForNotes(query, me); + this.queryService.generateMutedUserRenotesQueryForNotes(query, me); + } const reactions = (await query .limit(ps.limit) diff --git a/packages/backend/src/server/api/endpoints/users/recommendation.ts b/packages/backend/src/server/api/endpoints/users/recommendation.ts index 642d788459..52dd2197b2 100644 --- a/packages/backend/src/server/api/endpoints/users/recommendation.ts +++ b/packages/backend/src/server/api/endpoints/users/recommendation.ts @@ -71,6 +71,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- this.queryService.generateBlockQueryForUsers(query, me); this.queryService.generateBlockedUserQueryForNotes(query, me); + // TODO optimization: replace with exists() const followingQuery = this.followingsRepository.createQueryBuilder('following') .select('following.followeeId') .where('following.followerId = :followerId', { followerId: me.id }); diff --git a/packages/backend/src/server/api/endpoints/users/show.ts b/packages/backend/src/server/api/endpoints/users/show.ts index 7b1c8adfb8..84eb661742 100644 --- a/packages/backend/src/server/api/endpoints/users/show.ts +++ b/packages/backend/src/server/api/endpoints/users/show.ts @@ -13,6 +13,7 @@ import { RemoteUserResolveService } from '@/core/RemoteUserResolveService.js'; import { DI } from '@/di-symbols.js'; import PerUserPvChart from '@/core/chart/charts/per-user-pv.js'; import { RoleService } from '@/core/RoleService.js'; +import { renderInlineError } from '@/misc/render-inline-error.js'; import { ApiError } from '../../error.js'; import { ApiLoggerService } from '../../ApiLoggerService.js'; import type { FindOptionsWhere } from 'typeorm'; @@ -131,7 +132,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint- // Lookup user if (typeof ps.host === 'string' && typeof ps.username === 'string') { user = await this.remoteUserResolveService.resolveUser(ps.username, ps.host).catch(err => { - this.apiLoggerService.logger.warn(`failed to resolve remote user: ${err}`); + this.apiLoggerService.logger.warn(`failed to resolve remote user: ${renderInlineError(err)}`); throw new ApiError(meta.errors.failedToResolveRemoteUser); }); } else { diff --git a/packages/backend/src/server/api/mastodon/MastodonApiServerService.ts b/packages/backend/src/server/api/mastodon/MastodonApiServerService.ts index 74fd9d7d59..072dacf708 100644 --- a/packages/backend/src/server/api/mastodon/MastodonApiServerService.ts +++ b/packages/backend/src/server/api/mastodon/MastodonApiServerService.ts @@ -71,6 +71,13 @@ export class MastodonApiServerService { done(); }); + // Tell crawlers not to index API endpoints. + // https://developers.google.com/search/docs/crawling-indexing/block-indexing + fastify.addHook('onRequest', (request, reply, done) => { + reply.header('X-Robots-Tag', 'noindex'); + done(); + }); + // External endpoints this.apiAccountMastodon.register(fastify); this.apiAppsMastodon.register(fastify); diff --git a/packages/backend/src/server/api/mastodon/MastodonConverters.ts b/packages/backend/src/server/api/mastodon/MastodonConverters.ts index 375ea1ef08..df8d68042a 100644 --- a/packages/backend/src/server/api/mastodon/MastodonConverters.ts +++ b/packages/backend/src/server/api/mastodon/MastodonConverters.ts @@ -5,7 +5,7 @@ import { Inject, Injectable } from '@nestjs/common'; import { Entity, MastodonEntity, MisskeyEntity } from 'megalodon'; -import mfm from '@transfem-org/sfm-js'; +import mfm from 'mfm-js'; import { MastodonNotificationType } from 'megalodon/lib/src/mastodon/notification.js'; import { NotificationType } from 'megalodon/lib/src/notification.js'; import { DI } from '@/di-symbols.js'; @@ -252,10 +252,10 @@ export class MastodonConverters { return await this.convertStatus(status, me); } - public async convertStatus(status: Entity.Status, me: MiLocalUser | null): Promise<MastodonEntity.Status> { + public async convertStatus(status: Entity.Status, me: MiLocalUser | null, hints?: { note?: MiNote, user?: MiUser }): Promise<MastodonEntity.Status> { const convertedAccount = this.convertAccount(status.account); - const note = await this.mastodonDataService.requireNote(status.id, me); - const noteUser = await this.getUser(status.account.id); + const note = hints?.note ?? await this.mastodonDataService.requireNote(status.id, me); + const noteUser = hints?.user ?? note.user ?? await this.getUser(status.account.id); const mentionedRemoteUsers = JSON.parse(note.mentionedRemoteUsers); const emojis = await this.customEmojiService.populateEmojis(note.emojis, noteUser.host ? noteUser.host : this.config.host); diff --git a/packages/backend/src/server/api/mastodon/MastodonDataService.ts b/packages/backend/src/server/api/mastodon/MastodonDataService.ts index db257756de..e080cb10bd 100644 --- a/packages/backend/src/server/api/mastodon/MastodonDataService.ts +++ b/packages/backend/src/server/api/mastodon/MastodonDataService.ts @@ -7,8 +7,8 @@ import { Inject, Injectable } from '@nestjs/common'; import { IsNull } from 'typeorm'; import { DI } from '@/di-symbols.js'; import { QueryService } from '@/core/QueryService.js'; -import type { MiNote, NotesRepository } from '@/models/_.js'; -import type { MiLocalUser } from '@/models/User.js'; +import type { MiChannel, MiNote, NotesRepository } from '@/models/_.js'; +import type { MiLocalUser, MiUser } from '@/models/User.js'; import { ApiError } from '../error.js'; /** @@ -27,8 +27,8 @@ export class MastodonDataService { /** * Fetches a note in the context of the current user, and throws an exception if not found. */ - public async requireNote(noteId: string, me?: MiLocalUser | null): Promise<MiNote> { - const note = await this.getNote(noteId, me); + public async requireNote<Rel extends NoteRelations = NoteRelations>(noteId: string, me: MiLocalUser | null | undefined, relations?: Rel): Promise<NoteWithRelations<Rel>> { + const note = await this.getNote(noteId, me, relations); if (!note) { throw new ApiError({ @@ -46,12 +46,39 @@ export class MastodonDataService { /** * Fetches a note in the context of the current user. */ - public async getNote(noteId: string, me?: MiLocalUser | null): Promise<MiNote | null> { + public async getNote<Rel extends NoteRelations = NoteRelations>(noteId: string, me: MiLocalUser | null | undefined, relations?: Rel): Promise<NoteWithRelations<Rel> | null> { // Root query: note + required dependencies const query = this.notesRepository .createQueryBuilder('note') - .where('note.id = :noteId', { noteId }) - .innerJoinAndSelect('note.user', 'user'); + .where('note.id = :noteId', { noteId }); + + // Load relations + if (relations) { + if (relations.reply) { + query.leftJoinAndSelect('note.reply', 'reply'); + if (typeof(relations.reply) === 'object') { + if (relations.reply.reply) query.leftJoinAndSelect('reply.reply', 'replyReply'); + if (relations.reply.renote) query.leftJoinAndSelect('reply.renote', 'replyRenote'); + if (relations.reply.user) query.innerJoinAndSelect('reply.user', 'replyUser'); + if (relations.reply.channel) query.leftJoinAndSelect('reply.channel', 'replyChannel'); + } + } + if (relations.renote) { + query.leftJoinAndSelect('note.renote', 'renote'); + if (typeof(relations.renote) === 'object') { + if (relations.renote.reply) query.leftJoinAndSelect('renote.reply', 'renoteReply'); + if (relations.renote.renote) query.leftJoinAndSelect('renote.renote', 'renoteRenote'); + if (relations.renote.user) query.innerJoinAndSelect('renote.user', 'renoteUser'); + if (relations.renote.channel) query.leftJoinAndSelect('renote.channel', 'renoteChannel'); + } + } + if (relations.user) { + query.innerJoinAndSelect('note.user', 'user'); + } + if (relations.channel) { + query.leftJoinAndSelect('note.channel', 'channel'); + } + } // Restrict visibility this.queryService.generateVisibilityQuery(query, me); @@ -59,7 +86,7 @@ export class MastodonDataService { this.queryService.generateBlockedUserQueryForNotes(query, me); } - return await query.getOne(); + return await query.getOne() as NoteWithRelations<Rel> | null; } /** @@ -82,3 +109,41 @@ export class MastodonDataService { }); } } + +interface NoteRelations { + reply?: boolean | { + reply?: boolean; + renote?: boolean; + user?: boolean; + channel?: boolean; + }; + renote?: boolean | { + reply?: boolean; + renote?: boolean; + user?: boolean; + channel?: boolean; + }; + user?: boolean; + channel?: boolean; +} + +type NoteWithRelations<Rel extends NoteRelations> = MiNote & { + reply: Rel extends { reply: false } + ? null + : null | (MiNote & { + reply: Rel['reply'] extends { reply: true } ? MiNote | null : null; + renote: Rel['reply'] extends { renote: true } ? MiNote | null : null; + user: Rel['reply'] extends { user: true } ? MiUser : null; + channel: Rel['reply'] extends { channel: true } ? MiChannel | null : null; + }); + renote: Rel extends { renote: false } + ? null + : null | (MiNote & { + reply: Rel['renote'] extends { reply: true } ? MiNote | null : null; + renote: Rel['renote'] extends { renote: true } ? MiNote | null : null; + user: Rel['renote'] extends { user: true } ? MiUser : null; + channel: Rel['renote'] extends { channel: true } ? MiChannel | null : null; + }); + user: Rel extends { user: true } ? MiUser : null; + channel: Rel extends { channel: true } ? MiChannel | null : null; +}; diff --git a/packages/backend/src/server/api/mastodon/endpoints/status.ts b/packages/backend/src/server/api/mastodon/endpoints/status.ts index 22b8a911ca..7a058a0ed9 100644 --- a/packages/backend/src/server/api/mastodon/endpoints/status.ts +++ b/packages/backend/src/server/api/mastodon/endpoints/status.ts @@ -8,6 +8,10 @@ import { Injectable } from '@nestjs/common'; import { emojiRegexAtStartToEnd } from '@/misc/emoji-regex.js'; import { parseTimelineArgs, TimelineArgs, toBoolean, toInt } from '@/server/api/mastodon/argsUtils.js'; import { MastodonClientService } from '@/server/api/mastodon/MastodonClientService.js'; +import { MastodonDataService } from '@/server/api/mastodon/MastodonDataService.js'; +import { getNoteSummary } from '@/misc/get-note-summary.js'; +import type { Packed } from '@/misc/json-schema.js'; +import { isPureRenote } from '@/misc/is-renote.js'; import { convertAttachment, convertPoll, MastodonConverters } from '../MastodonConverters.js'; import type { Entity } from 'megalodon'; import type { FastifyInstance } from 'fastify'; @@ -22,6 +26,7 @@ export class ApiStatusMastodon { constructor( private readonly mastoConverters: MastodonConverters, private readonly clientService: MastodonClientService, + private readonly mastodonDataService: MastodonDataService, ) {} public register(fastify: FastifyInstance): void { @@ -29,13 +34,24 @@ export class ApiStatusMastodon { if (!_request.params.id) return reply.code(400).send({ error: 'BAD_REQUEST', error_description: 'Missing required parameter "id"' }); const { client, me } = await this.clientService.getAuthClient(_request); - const data = await client.getStatus(_request.params.id); - const response = await this.mastoConverters.convertStatus(data.data, me); + const note = await this.mastodonDataService.requireNote(_request.params.id, me, { user: true, renote: { user: true } }); + + // Unpack renote for Discord, otherwise the preview breaks + const appearNote = (isPureRenote(note) && _request.headers['user-agent']?.match(/\bDiscordbot\//)) + ? note.renote as NonNullable<typeof note.renote> + : note; + + const data = await client.getStatus(appearNote.id); + const response = await this.mastoConverters.convertStatus(data.data, me, { note: appearNote, user: appearNote.user }); // Fixup - Discord ignores CWs and renders the entire post. if (response.sensitive && _request.headers['user-agent']?.match(/\bDiscordbot\//)) { - response.content = '(preview disabled for sensitive content)'; + response.content = getNoteSummary(data.data satisfies Packed<'Note'>); response.media_attachments = []; + response.in_reply_to_id = null; + response.in_reply_to_account_id = null; + response.reblog = null; + response.quote = null; } return reply.send(response); @@ -170,7 +186,7 @@ export class ApiStatusMastodon { const data = await client.deleteEmojiReaction(id, react); return reply.send(data.data); } - if (!body.media_ids) body.media_ids = undefined; + body.media_ids ??= undefined; if (body.media_ids && !body.media_ids.length) body.media_ids = undefined; if (body.poll && !body.poll.options) { diff --git a/packages/backend/src/server/api/stream/Connection.ts b/packages/backend/src/server/api/stream/Connection.ts index e0535a2f14..0ee7078eb2 100644 --- a/packages/backend/src/server/api/stream/Connection.ts +++ b/packages/backend/src/server/api/stream/Connection.ts @@ -36,7 +36,7 @@ export default class Connection { private channels = new Map<string, Channel>(); private subscribingNotes = new Map<string, number>(); public userProfile: MiUserProfile | null = null; - public following: Record<string, Pick<MiFollowing, 'withReplies'> | undefined> = {}; + public following: Map<string, Omit<MiFollowing, 'isFollowerHibernated'>> = new Map(); public followingChannels: Set<string> = new Set(); public userIdsWhoMeMuting: Set<string> = new Set(); public userIdsWhoBlockingMe: Set<string> = new Set(); diff --git a/packages/backend/src/server/api/stream/channel.ts b/packages/backend/src/server/api/stream/channel.ts index 9af816dfbb..40ad454adb 100644 --- a/packages/backend/src/server/api/stream/channel.ts +++ b/packages/backend/src/server/api/stream/channel.ts @@ -61,12 +61,30 @@ export default abstract class Channel { return this.connection.subscriber; } + /** + * Checks if a note is visible to the current user *excluding* blocks and mutes. + */ + protected isNoteVisibleToMe(note: Packed<'Note'>): boolean { + if (note.visibility === 'public') return true; + if (note.visibility === 'home') return true; + if (!this.user) return false; + if (this.user.id === note.userId) return true; + if (note.visibility === 'followers') { + return this.following.has(note.userId); + } + if (!note.visibleUserIds) return false; + return note.visibleUserIds.includes(this.user.id); + } + /* * ミュートとブロックされてるを処理する */ protected isNoteMutedOrBlocked(note: Packed<'Note'>): boolean { + // Ignore notes that require sign-in + if (note.user.requireSigninToViewContents && !this.user) return true; + // 流れてきたNoteがインスタンスミュートしたインスタンスが関わる - if (isInstanceMuted(note, new Set<string>(this.userProfile?.mutedInstances ?? [])) && !this.following[note.userId]) return true; + if (isInstanceMuted(note, this.userMutedInstances) && !this.following.has(note.userId)) return true; // 流れてきたNoteがミュートしているユーザーが関わる if (isUserRelated(note, this.userIdsWhoMeMuting)) return true; @@ -79,6 +97,15 @@ export default abstract class Channel { // If it's a boost (pure renote) then we need to check the target as well if (isPackedPureRenote(note) && note.renote && this.isNoteMutedOrBlocked(note.renote)) return true; + // Hide silenced notes + if (note.user.isSilenced || note.user.instance?.isSilenced) { + if (this.user == null) return true; + if (this.user.id === note.userId) return false; + if (!this.following.has(note.userId)) return true; + } + + // TODO muted threads + return false; } diff --git a/packages/backend/src/server/api/stream/channels/bubble-timeline.ts b/packages/backend/src/server/api/stream/channels/bubble-timeline.ts index d29101cbc5..72f719b411 100644 --- a/packages/backend/src/server/api/stream/channels/bubble-timeline.ts +++ b/packages/backend/src/server/api/stream/channels/bubble-timeline.ts @@ -5,13 +5,12 @@ import { Injectable } from '@nestjs/common'; import type { Packed } from '@/misc/json-schema.js'; -import { MetaService } from '@/core/MetaService.js'; import { NoteEntityService } from '@/core/entities/NoteEntityService.js'; import { bindThis } from '@/decorators.js'; import { RoleService } from '@/core/RoleService.js'; -import type { MiMeta } from '@/models/Meta.js'; import { isRenotePacked, isQuotePacked } from '@/misc/is-renote.js'; import type { JsonObject } from '@/misc/json-value.js'; +import { UtilityService } from '@/core/UtilityService.js'; import Channel, { MiChannelService } from '../channel.js'; class BubbleTimelineChannel extends Channel { @@ -21,11 +20,10 @@ class BubbleTimelineChannel extends Channel { private withRenotes: boolean; private withFiles: boolean; private withBots: boolean; - private instance: MiMeta; constructor( - private metaService: MetaService, private roleService: RoleService, + private readonly utilityService: UtilityService, noteEntityService: NoteEntityService, id: string, @@ -42,7 +40,6 @@ class BubbleTimelineChannel extends Channel { this.withRenotes = !!(params.withRenotes ?? true); this.withFiles = !!(params.withFiles ?? false); this.withBots = !!(params.withBots ?? true); - this.instance = await this.metaService.fetch(); // Subscribe events this.subscriber.on('notesStream', this.onNote); @@ -50,20 +47,36 @@ class BubbleTimelineChannel extends Channel { @bindThis private async onNote(note: Packed<'Note'>) { + const isMe = this.user?.id === note.userId; + if (this.withFiles && (note.fileIds == null || note.fileIds.length === 0)) return; if (!this.withBots && note.user.isBot) return; if (note.visibility !== 'public') return; if (note.channelId != null) return; - if (note.user.host == null) return; - if (!this.instance.bubbleInstances.includes(note.user.host)) return; - if (note.user.requireSigninToViewContents && this.user == null) return; + if (!this.utilityService.isBubbledHost(note.user.host)) return; - if (isRenotePacked(note) && !isQuotePacked(note) && !this.withRenotes) return; + if (this.isNoteMutedOrBlocked(note)) return; - if (note.user.isSilenced && !this.following[note.userId] && note.userId !== this.user!.id) return; + if (note.reply) { + const reply = note.reply; + // 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く + if (!this.isNoteVisibleToMe(reply)) return; + if (!this.following.get(note.userId)?.withReplies) { + // 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合 + if (reply.userId !== this.user?.id && !isMe && reply.userId !== note.userId) return; + } + } - if (this.isNoteMutedOrBlocked(note)) return; + // 純粋なリノート（引用リノートでないリノート）の場合 + if (isRenotePacked(note) && !isQuotePacked(note) && note.renote) { + if (!this.withRenotes) return; + if (note.renote.reply) { + const reply = note.renote.reply; + // 自分のフォローしていないユーザーの visibility: followers な投稿への返信のリノートは弾く + if (!this.isNoteVisibleToMe(reply)) return; + } + } const clonedNote = await this.assignMyReaction(note); await this.hideNote(clonedNote); @@ -85,17 +98,17 @@ export class BubbleTimelineChannelService implements MiChannelService<false> { public readonly kind = BubbleTimelineChannel.kind; constructor( - private metaService: MetaService, private roleService: RoleService, private noteEntityService: NoteEntityService, + private readonly utilityService: UtilityService, ) { } @bindThis public create(id: string, connection: Channel['connection']): BubbleTimelineChannel { return new BubbleTimelineChannel( - this.metaService, this.roleService, + this.utilityService, this.noteEntityService, id, connection, diff --git a/packages/backend/src/server/api/stream/channels/global-timeline.ts b/packages/backend/src/server/api/stream/channels/global-timeline.ts index c899ad9490..5c73f637c7 100644 --- a/packages/backend/src/server/api/stream/channels/global-timeline.ts +++ b/packages/backend/src/server/api/stream/channels/global-timeline.ts @@ -48,20 +48,36 @@ class GlobalTimelineChannel extends Channel { @bindThis private async onNote(note: Packed<'Note'>) { + const isMe = this.user?.id === note.userId; + if (this.withFiles && (note.fileIds == null || note.fileIds.length === 0)) return; if (!this.withBots && note.user.isBot) return; if (note.visibility !== 'public') return; if (note.channelId != null) return; - if (note.user.requireSigninToViewContents && this.user == null) return; - if (note.renote && note.renote.user.requireSigninToViewContents && this.user == null) return; - if (note.reply && note.reply.user.requireSigninToViewContents && this.user == null) return; - if (isRenotePacked(note) && !isQuotePacked(note) && !this.withRenotes) return; + if (this.isNoteMutedOrBlocked(note)) return; + if (!this.isNoteVisibleToMe(note)) return; - if (note.user.isSilenced && !this.following[note.userId] && note.userId !== this.user!.id) return; + if (note.reply) { + const reply = note.reply; + // 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く + if (!this.isNoteVisibleToMe(reply)) return; + if (!this.following.get(note.userId)?.withReplies) { + // 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合 + if (reply.userId !== this.user?.id && !isMe && reply.userId !== note.userId) return; + } + } - if (this.isNoteMutedOrBlocked(note)) return; + // 純粋なリノート（引用リノートでないリノート）の場合 + if (isRenotePacked(note) && !isQuotePacked(note) && note.renote) { + if (!this.withRenotes) return; + if (note.renote.reply) { + const reply = note.renote.reply; + // 自分のフォローしていないユーザーの visibility: followers な投稿への返信のリノートは弾く + if (!this.isNoteVisibleToMe(reply)) return; + } + } const clonedNote = await this.assignMyReaction(note); await this.hideNote(clonedNote); diff --git a/packages/backend/src/server/api/stream/channels/home-timeline.ts b/packages/backend/src/server/api/stream/channels/home-timeline.ts index dfdb491113..c7062c0394 100644 --- a/packages/backend/src/server/api/stream/channels/home-timeline.ts +++ b/packages/backend/src/server/api/stream/channels/home-timeline.ts @@ -47,40 +47,32 @@ class HomeTimelineChannel extends Channel { if (!this.followingChannels.has(note.channelId)) return; } else { // その投稿のユーザーをフォローしていなかったら弾く - if (!isMe && !Object.hasOwn(this.following, note.userId)) return; + if (!isMe && !this.following.has(note.userId)) return; } - if (note.visibility === 'followers') { - if (!isMe && !Object.hasOwn(this.following, note.userId)) return; - } else if (note.visibility === 'specified') { - if (!isMe && !note.visibleUserIds!.includes(this.user!.id)) return; - } + if (this.isNoteMutedOrBlocked(note)) return; + if (!this.isNoteVisibleToMe(note)) return; if (note.reply) { const reply = note.reply; - if (this.following[note.userId]?.withReplies) { - // 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く - if (reply.visibility === 'followers' && !Object.hasOwn(this.following, reply.userId) && reply.userId !== this.user!.id) return; - } else { + // 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く + if (!this.isNoteVisibleToMe(reply)) return; + if (!this.following.get(note.userId)?.withReplies) { // 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合 if (reply.userId !== this.user!.id && !isMe && reply.userId !== note.userId) return; } } - if (note.user.isSilenced && !this.following[note.userId] && note.userId !== this.user!.id) return; - // 純粋なリノート（引用リノートでないリノート）の場合 if (isRenotePacked(note) && !isQuotePacked(note) && note.renote) { if (!this.withRenotes) return; if (note.renote.reply) { const reply = note.renote.reply; // 自分のフォローしていないユーザーの visibility: followers な投稿への返信のリノートは弾く - if (reply.visibility === 'followers' && !Object.hasOwn(this.following, reply.userId) && reply.userId !== this.user!.id) return; + if (!this.isNoteVisibleToMe(reply)) return; } } - if (this.isNoteMutedOrBlocked(note)) return; - const clonedNote = await this.assignMyReaction(note); await this.hideNote(clonedNote); diff --git a/packages/backend/src/server/api/stream/channels/hybrid-timeline.ts b/packages/backend/src/server/api/stream/channels/hybrid-timeline.ts index 6cb425ff81..7cb64c9f89 100644 --- a/packages/backend/src/server/api/stream/channels/hybrid-timeline.ts +++ b/packages/backend/src/server/api/stream/channels/hybrid-timeline.ts @@ -62,39 +62,31 @@ class HybridTimelineChannel extends Channel { // フォローしているチャンネルの投稿 の場合だけ if (!( (note.channelId == null && isMe) || - (note.channelId == null && Object.hasOwn(this.following, note.userId)) || + (note.channelId == null && this.following.has(note.userId)) || (note.channelId == null && (note.user.host == null && note.visibility === 'public')) || (note.channelId != null && this.followingChannels.has(note.channelId)) )) return; - if (note.visibility === 'followers') { - if (!isMe && !Object.hasOwn(this.following, note.userId)) return; - } else if (note.visibility === 'specified') { - if (!isMe && !note.visibleUserIds!.includes(this.user!.id)) return; - } - if (this.isNoteMutedOrBlocked(note)) return; + if (!this.isNoteVisibleToMe(note)) return; if (note.reply) { const reply = note.reply; - if ((this.following[note.userId]?.withReplies ?? false) || this.withReplies) { - // 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く - if (reply.visibility === 'followers' && !Object.hasOwn(this.following, reply.userId) && reply.userId !== this.user!.id) return; - } else { + // 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く + if (!this.isNoteVisibleToMe(reply)) return; + if (!this.following.get(note.userId)?.withReplies && !this.withReplies) { // 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合 if (reply.userId !== this.user!.id && !isMe && reply.userId !== note.userId) return; } } - if (note.user.isSilenced && !this.following[note.userId] && note.userId !== this.user!.id) return; - // 純粋なリノート（引用リノートでないリノート）の場合 if (isRenotePacked(note) && !isQuotePacked(note) && note.renote) { if (!this.withRenotes) return; if (note.renote.reply) { const reply = note.renote.reply; // 自分のフォローしていないユーザーの visibility: followers な投稿への返信のリノートは弾く - if (reply.visibility === 'followers' && !Object.hasOwn(this.following, reply.userId) && reply.userId !== this.user!.id) return; + if (!this.isNoteVisibleToMe(reply)) return; } } diff --git a/packages/backend/src/server/api/stream/channels/local-timeline.ts b/packages/backend/src/server/api/stream/channels/local-timeline.ts index 82b128eae0..4869d871d6 100644 --- a/packages/backend/src/server/api/stream/channels/local-timeline.ts +++ b/packages/backend/src/server/api/stream/channels/local-timeline.ts @@ -50,28 +50,37 @@ class LocalTimelineChannel extends Channel { @bindThis private async onNote(note: Packed<'Note'>) { + const isMe = this.user?.id === note.userId; + if (this.withFiles && (note.fileIds == null || note.fileIds.length === 0)) return; if (!this.withBots && note.user.isBot) return; if (note.user.host !== null) return; if (note.visibility !== 'public') return; if (note.channelId != null) return; - if (note.user.requireSigninToViewContents && this.user == null) return; - if (note.renote && note.renote.user.requireSigninToViewContents && this.user == null) return; - if (note.reply && note.reply.user.requireSigninToViewContents && this.user == null) return; + + if (this.isNoteMutedOrBlocked(note)) return; + if (!this.isNoteVisibleToMe(note)) return; // 関係ない返信は除外 - if (note.reply && this.user && !this.following[note.userId]?.withReplies && !this.withReplies) { + if (note.reply) { const reply = note.reply; - // 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合 - if (reply.userId !== this.user.id && note.userId !== this.user.id && reply.userId !== note.userId) return; + // 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く + if (!this.isNoteVisibleToMe(reply)) return; + if (!this.following.get(note.userId)?.withReplies) { + // 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合 + if (reply.userId !== this.user?.id && !isMe && reply.userId !== note.userId) return; + } } - if (note.user.isSilenced && !this.following[note.userId] && note.userId !== this.user!.id) return; - - if (isRenotePacked(note) && !isQuotePacked(note) && !this.withRenotes) return; - - if (this.isNoteMutedOrBlocked(note)) return; + if (isRenotePacked(note) && !isQuotePacked(note) && note.renote) { + if (!this.withRenotes) return; + if (note.renote.reply) { + const reply = note.renote.reply; + // 自分のフォローしていないユーザーの visibility: followers な投稿への返信のリノートは弾く + if (!this.isNoteVisibleToMe(reply)) return; + } + } const clonedNote = await this.assignMyReaction(note); await this.hideNote(clonedNote); diff --git a/packages/backend/src/server/api/stream/channels/main.ts b/packages/backend/src/server/api/stream/channels/main.ts index 6194bb78dd..193907504a 100644 --- a/packages/backend/src/server/api/stream/channels/main.ts +++ b/packages/backend/src/server/api/stream/channels/main.ts @@ -32,10 +32,12 @@ class MainChannel extends Channel { switch (data.type) { case 'notification': { // Ignore notifications from instances the user has muted - if (isUserFromMutedInstance(data.body, new Set<string>(this.userProfile?.mutedInstances ?? []))) return; + if (isUserFromMutedInstance(data.body, this.userMutedInstances)) return; if (data.body.userId && this.userIdsWhoMeMuting.has(data.body.userId)) return; if (data.body.note && data.body.note.isHidden) { + if (this.isNoteMutedOrBlocked(data.body.note)) return; + if (!this.isNoteVisibleToMe(data.body.id)) return; const note = await this.noteEntityService.pack(data.body.note.id, this.user, { detail: true, }); @@ -44,9 +46,7 @@ class MainChannel extends Channel { break; } case 'mention': { - if (isInstanceMuted(data.body, new Set<string>(this.userProfile?.mutedInstances ?? []))) return; - - if (this.userIdsWhoMeMuting.has(data.body.userId)) return; + if (this.isNoteMutedOrBlocked(data.body)) return; if (data.body.isHidden) { const note = await this.noteEntityService.pack(data.body.id, this.user, { detail: true, diff --git a/packages/backend/src/server/api/stream/channels/role-timeline.ts b/packages/backend/src/server/api/stream/channels/role-timeline.ts index 78cd9bf868..a3886618f1 100644 --- a/packages/backend/src/server/api/stream/channels/role-timeline.ts +++ b/packages/backend/src/server/api/stream/channels/role-timeline.ts @@ -9,6 +9,7 @@ import { bindThis } from '@/decorators.js'; import { RoleService } from '@/core/RoleService.js'; import type { GlobalEvents } from '@/core/GlobalEventService.js'; import type { JsonObject } from '@/misc/json-value.js'; +import { isQuotePacked, isRenotePacked } from '@/misc/is-renote.js'; import Channel, { type MiChannelService } from '../channel.js'; class RoleTimelineChannel extends Channel { @@ -40,7 +41,9 @@ class RoleTimelineChannel extends Channel { private async onEvent(data: GlobalEvents['roleTimeline']['payload']) { if (data.type === 'note') { const note = data.body; + const isMe = this.user?.id === note.userId; + // TODO this should be cached if (!(await this.roleservice.isExplorable({ id: this.roleId }))) { return; } @@ -48,6 +51,25 @@ class RoleTimelineChannel extends Channel { if (this.isNoteMutedOrBlocked(note)) return; + if (note.reply) { + const reply = note.reply; + // 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く + if (!this.isNoteVisibleToMe(reply)) return; + if (!this.following.get(note.userId)?.withReplies) { + // 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合 + if (reply.userId !== this.user?.id && !isMe && reply.userId !== note.userId) return; + } + } + + // 純粋なリノート（引用リノートでないリノート）の場合 + if (isRenotePacked(note) && !isQuotePacked(note) && note.renote) { + if (note.renote.reply) { + const reply = note.renote.reply; + // 自分のフォローしていないユーザーの visibility: followers な投稿への返信のリノートは弾く + if (!this.isNoteVisibleToMe(reply)) return; + } + } + const clonedNote = await this.assignMyReaction(note); await this.hideNote(clonedNote); diff --git a/packages/backend/src/server/api/stream/channels/user-list.ts b/packages/backend/src/server/api/stream/channels/user-list.ts index 8a7c2b2633..4dae24a696 100644 --- a/packages/backend/src/server/api/stream/channels/user-list.ts +++ b/packages/backend/src/server/api/stream/channels/user-list.ts @@ -16,7 +16,8 @@ import Channel, { type MiChannelService } from '../channel.js'; class UserListChannel extends Channel { public readonly chName = 'userList'; public static shouldShare = false; - public static requireCredential = false as const; + public static requireCredential = true as const; + public static kind = 'read:account'; private listId: string; private membershipsMap: Record<string, Pick<MiUserListMembership, 'withReplies'> | undefined> = {}; private listUsersClock: NodeJS.Timeout; @@ -81,7 +82,7 @@ class UserListChannel extends Channel { @bindThis private async onNote(note: Packed<'Note'>) { - const isMe = this.user!.id === note.userId; + const isMe = this.user?.id === note.userId; // チャンネル投稿は無視する if (note.channelId) return; @@ -90,26 +91,28 @@ class UserListChannel extends Channel { if (!Object.hasOwn(this.membershipsMap, note.userId)) return; - if (note.visibility === 'followers') { - if (!isMe && !Object.hasOwn(this.following, note.userId)) return; - } else if (note.visibility === 'specified') { - if (!note.visibleUserIds!.includes(this.user!.id)) return; - } + if (this.isNoteMutedOrBlocked(note)) return; + if (!this.isNoteVisibleToMe(note)) return; if (note.reply) { const reply = note.reply; - if (this.membershipsMap[note.userId]?.withReplies) { - // 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く - if (reply.visibility === 'followers' && !Object.hasOwn(this.following, reply.userId)) return; - } else { + // 自分のフォローしていないユーザーの visibility: followers な投稿への返信は弾く + if (!this.isNoteVisibleToMe(reply)) return; + if (!this.following.get(note.userId)?.withReplies) { // 「チャンネル接続主への返信」でもなければ、「チャンネル接続主が行った返信」でもなければ、「投稿者の投稿者自身への返信」でもない場合 if (reply.userId !== this.user!.id && !isMe && reply.userId !== note.userId) return; } } - if (isRenotePacked(note) && !isQuotePacked(note) && !this.withRenotes) return; - - if (this.isNoteMutedOrBlocked(note)) return; + // 純粋なリノート（引用リノートでないリノート）の場合 + if (isRenotePacked(note) && !isQuotePacked(note) && note.renote) { + if (!this.withRenotes) return; + if (note.renote.reply) { + const reply = note.renote.reply; + // 自分のフォローしていないユーザーの visibility: followers な投稿への返信のリノートは弾く + if (!this.isNoteVisibleToMe(reply)) return; + } + } const clonedNote = await this.assignMyReaction(note); await this.hideNote(clonedNote); @@ -128,7 +131,7 @@ class UserListChannel extends Channel { } @Injectable() -export class UserListChannelService implements MiChannelService<false> { +export class UserListChannelService implements MiChannelService<true> { public readonly shouldShare = UserListChannel.shouldShare; public readonly requireCredential = UserListChannel.requireCredential; public readonly kind = UserListChannel.kind; diff --git a/packages/backend/src/server/web/FeedService.ts b/packages/backend/src/server/web/FeedService.ts index dcd4d80303..a622ae7e34 100644 --- a/packages/backend/src/server/web/FeedService.ts +++ b/packages/backend/src/server/web/FeedService.ts @@ -15,7 +15,7 @@ import { DriveFileEntityService } from '@/core/entities/DriveFileEntityService.j import { bindThis } from '@/decorators.js'; import { IdService } from '@/core/IdService.js'; import { MfmService } from "@/core/MfmService.js"; -import { parse as mfmParse } from '@transfem-org/sfm-js'; +import { parse as mfmParse } from 'mfm-js'; @Injectable() export class FeedService { diff --git a/packages/backend/src/server/web/UrlPreviewService.ts b/packages/backend/src/server/web/UrlPreviewService.ts index 203bc908a8..71a142fc6f 100644 --- a/packages/backend/src/server/web/UrlPreviewService.ts +++ b/packages/backend/src/server/web/UrlPreviewService.ts @@ -20,6 +20,7 @@ import { RedisKVCache } from '@/misc/cache.js'; import { UtilityService } from '@/core/UtilityService.js'; import { ApDbResolverService } from '@/core/activitypub/ApDbResolverService.js'; import type { MiAccessToken, NotesRepository } from '@/models/_.js'; +import { RemoteUserResolveService } from '@/core/RemoteUserResolveService.js'; import { ApUtilityService } from '@/core/activitypub/ApUtilityService.js'; import { ApRequestService } from '@/core/activitypub/ApRequestService.js'; import { SystemAccountService } from '@/core/SystemAccountService.js'; @@ -30,14 +31,19 @@ import { BucketRateLimit, Keyed, sendRateLimitHeaders } from '@/misc/rate-limit- import type { MiLocalUser } from '@/models/User.js'; import { getIpHash } from '@/misc/get-ip-hash.js'; import { isRetryableError } from '@/misc/is-retryable-error.js'; +import * as Acct from '@/misc/acct.js'; +import { isNote } from '@/core/activitypub/type.js'; import type { FastifyRequest, FastifyReply } from 'fastify'; export type LocalSummalyResult = SummalyResult & { haveNoteLocally?: boolean; + linkAttribution?: { + userId: string, + } }; // Increment this to invalidate cached previews after a major change. -const cacheFormatVersion = 3; +const cacheFormatVersion = 4; type PreviewRoute = { Querystring: { @@ -82,6 +88,7 @@ export class UrlPreviewService { private readonly utilityService: UtilityService, private readonly apUtilityService: ApUtilityService, private readonly apDbResolverService: ApDbResolverService, + private readonly remoteUserResolveService: RemoteUserResolveService, private readonly apRequestService: ApRequestService, private readonly systemAccountService: SystemAccountService, private readonly apNoteService: ApNoteService, @@ -117,26 +124,49 @@ export class UrlPreviewService { request: FastifyRequest<PreviewRoute>, reply: FastifyReply, ): Promise<void> { + if (!this.meta.urlPreviewEnabled) { + // Tell crawlers not to index URL previews. + // https://developers.google.com/search/docs/crawling-indexing/block-indexing + reply.header('X-Robots-Tag', 'noindex'); + + return reply.code(403).send({ + error: { + message: 'URL preview is disabled', + code: 'URL_PREVIEW_DISABLED', + id: '58b36e13-d2f5-0323-b0c6-76aa9dabefb8', + }, + }); + } + const url = request.query.url; if (typeof url !== 'string' || !URL.canParse(url)) { reply.code(400); return; } + // Enforce HTTP(S) for input URLs + const urlScheme = this.utilityService.getUrlScheme(url); + if (urlScheme !== 'http:' && urlScheme !== 'https:') { + reply.code(400); + return; + } + const lang = request.query.lang; if (Array.isArray(lang)) { reply.code(400); return; } - if (!this.meta.urlPreviewEnabled) { - return reply.code(403).send({ - error: { - message: 'URL preview is disabled', - code: 'URL_PREVIEW_DISABLED', - id: '58b36e13-d2f5-0323-b0c6-76aa9dabefb8', - }, - }); + // Strip out hash (anchor) + const urlObj = new URL(url); + if (urlObj.hash) { + urlObj.hash = ''; + const params = new URLSearchParams({ url: urlObj.href }); + if (lang) params.set('lang', lang); + const newUrl = `/url?${params.toString()}`; + + reply.redirect(newUrl, 301); + return; } // Check rate limit @@ -145,7 +175,7 @@ export class UrlPreviewService { return; } - if (this.utilityService.isBlockedHost(this.meta.blockedHosts, new URL(url).host)) { + if (this.utilityService.isBlockedHost(this.meta.blockedHosts, urlObj.host)) { return reply.code(403).send({ error: { message: 'URL is blocked', @@ -160,7 +190,7 @@ export class UrlPreviewService { return; } - const cacheKey = `${url}@${lang}@${cacheFormatVersion}`; + const cacheKey = getCacheKey(url, lang); if (await this.sendCachedPreview(cacheKey, reply, fetch)) { return; } @@ -206,9 +236,23 @@ export class UrlPreviewService { } } + await this.validateLinkAttribution(summary); + // Await this to avoid hammering redis when a bunch of URLs are fetched at once await this.previewCache.set(cacheKey, summary); + // Also cache the response URL in case of redirects + if (summary.url !== url) { + const responseCacheKey = getCacheKey(summary.url, lang); + await this.previewCache.set(responseCacheKey, summary); + } + + // Also cache the ActivityPub URL, if different from the others + if (summary.activityPub && summary.activityPub !== summary.url) { + const apCacheKey = getCacheKey(summary.activityPub, lang); + await this.previewCache.set(apCacheKey, summary); + } + // Cache 1 day (matching redis), but only once we finalize the result if (!summary.activityPub || summary.haveNoteLocally) { reply.header('Cache-Control', 'public, max-age=86400'); @@ -370,7 +414,7 @@ export class UrlPreviewService { // Finally, attempt a signed GET in case it's a direct link to an instance with authorized fetch. const instanceActor = await this.systemAccountService.getInstanceActor(); const remoteObject = await this.apRequestService.signedGet(summary.url, instanceActor).catch(() => null); - if (remoteObject && this.apUtilityService.haveSameAuthority(remoteObject.id, summary.url)) { + if (remoteObject && isNote(remoteObject) && this.apUtilityService.haveSameAuthority(remoteObject.id, summary.url)) { summary.activityPub = remoteObject.id; return; } @@ -426,6 +470,30 @@ export class UrlPreviewService { } } + private async validateLinkAttribution(summary: LocalSummalyResult) { + if (!summary.fediverseCreator) return; + if (!URL.canParse(summary.url)) return; + + const url = URL.parse(summary.url); + + const acct = Acct.parse(summary.fediverseCreator); + if (acct.host?.toLowerCase() === this.config.host) { + acct.host = null; + } + try { + const user = await this.remoteUserResolveService.resolveUser(acct.username, acct.host); + + const attributionDomains = user.attributionDomains; + if (attributionDomains.some(x => `.${url?.host.toLowerCase()}`.endsWith(`.${x}`))) { + summary.linkAttribution = { + userId: user.id, + }; + } + } catch { + this.logger.debug('User not found: ' + summary.fediverseCreator); + } + } + // Adapted from ApiCallService private async checkFetchPermissions(auth: AuthArray, reply: FastifyReply): Promise<boolean> { const [user, app] = auth; @@ -501,3 +569,7 @@ export class UrlPreviewService { return true; } } + +function getCacheKey(url: string, lang = 'none') { + return `${url}@${lang}@${cacheFormatVersion}`; +} diff --git a/packages/backend/test-federation/tsconfig.json b/packages/backend/test-federation/tsconfig.json index 3a1cb3b9f3..16b333f877 100644 --- a/packages/backend/test-federation/tsconfig.json +++ b/packages/backend/test-federation/tsconfig.json @@ -3,7 +3,7 @@ /* Visit https://aka.ms/tsconfig to read more about this file */ /* Projects */ - // "incremental": true, /* Save .tsbuildinfo files to allow for incremental compilation of projects. */ + "incremental": true, /* Save .tsbuildinfo files to allow for incremental compilation of projects. */ // "composite": true, /* Enable constraints that allow a TypeScript project to be used with project references. */ // "tsBuildInfoFile": "./.tsbuildinfo", /* Specify the path to .tsbuildinfo incremental compilation file. */ // "disableSourceOfProjectReferenceRedirect": true, /* Disable preferring source files instead of declaration files when referencing composite projects. */ diff --git a/packages/backend/test-server/tsconfig.json b/packages/backend/test-server/tsconfig.json index 10313699c2..cb394ecccd 100644 --- a/packages/backend/test-server/tsconfig.json +++ b/packages/backend/test-server/tsconfig.json @@ -23,6 +23,7 @@ "emitDecoratorMetadata": true, "resolveJsonModule": true, "isolatedModules": true, + "incremental": true, "rootDir": "../src", "baseUrl": "./", "paths": { diff --git a/packages/backend/test/e2e/oauth.ts b/packages/backend/test/e2e/oauth.ts index 47851e9474..1dc8d87593 100644 --- a/packages/backend/test/e2e/oauth.ts +++ b/packages/backend/test/e2e/oauth.ts @@ -19,7 +19,7 @@ import { ResourceOwnerPassword, } from 'simple-oauth2'; import pkceChallenge from 'pkce-challenge'; -import { load as cheerio } from 'cheerio'; +import { load as cheerio } from 'cheerio/slim'; import Fastify, { type FastifyInstance, type FastifyReply } from 'fastify'; import { api, port, sendEnvUpdateRequest, signup } from '../utils.js'; import type * as misskey from 'misskey-js'; diff --git a/packages/backend/test/misc/FakeInternalEventService.ts b/packages/backend/test/misc/FakeInternalEventService.ts new file mode 100644 index 0000000000..d18a080eaf --- /dev/null +++ b/packages/backend/test/misc/FakeInternalEventService.ts @@ -0,0 +1,92 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +import type { Listener, ListenerProps } from '@/core/InternalEventService.js'; +import type Redis from 'ioredis'; +import type { GlobalEventService, InternalEventTypes } from '@/core/GlobalEventService.js'; +import { InternalEventService } from '@/core/InternalEventService.js'; +import { bindThis } from '@/decorators.js'; + +type FakeCall<K extends keyof InternalEventService> = [K, Parameters<InternalEventService[K]>]; +type FakeListener<K extends keyof InternalEventTypes> = [K, Listener<K>, ListenerProps]; + +/** + * Minimal implementation of InternalEventService meant for use in unit tests. + * There is no redis connection, and metadata is tracked in the public _calls and _listeners arrays. + * The on/off/emit methods are fully functional and can be called in tests to invoke any registered listeners. + */ +export class FakeInternalEventService extends InternalEventService { + /** + * List of calls to public methods, in chronological order. + */ + public _calls: FakeCall<keyof InternalEventService>[] = []; + + /** + * List of currently registered listeners. + */ + public _listeners: FakeListener<keyof InternalEventTypes>[] = []; + + /** + * Resets the mock. + * Clears all listeners and tracked calls. + */ + public _reset() { + this._calls = []; + this._listeners = []; + } + + /** + * Simulates a remote event sent from another process in the cluster via redis. + */ + @bindThis + public async _emitRedis<K extends keyof InternalEventTypes>(type: K, value: InternalEventTypes[K]): Promise<void> { + await this.emit(type, value, false); + } + + constructor() { + super( + { on: () => {} } as unknown as Redis.Redis, + {} as unknown as GlobalEventService, + ); + } + + @bindThis + public on<K extends keyof InternalEventTypes>(type: K, listener: Listener<K>, props?: ListenerProps): void { + if (!this._listeners.some(l => l[0] === type && l[1] === listener)) { + this._listeners.push([type, listener as Listener<keyof InternalEventTypes>, props ?? {}]); + } + this._calls.push(['on', [type, listener as Listener<keyof InternalEventTypes>, props]]); + } + + @bindThis + public off<K extends keyof InternalEventTypes>(type: K, listener: Listener<K>): void { + this._listeners = this._listeners.filter(l => l[0] !== type || l[1] !== listener); + this._calls.push(['off', [type, listener as Listener<keyof InternalEventTypes>]]); + } + + @bindThis + public async emit<K extends keyof InternalEventTypes>(type: K, value: InternalEventTypes[K], isLocal = true): Promise<void> { + for (const listener of this._listeners) { + if (listener[0] === type) { + if ((isLocal && !listener[2].ignoreLocal) || (!isLocal && !listener[2].ignoreRemote)) { + await listener[1](value, type, isLocal); + } + } + } + this._calls.push(['emit', [type, value]]); + } + + @bindThis + public dispose(): void { + this._listeners = []; + this._calls.push(['dispose', []]); + } + + @bindThis + public onApplicationShutdown(): void { + this._calls.push(['onApplicationShutdown', []]); + } +} + diff --git a/packages/backend/test/misc/mock-resolver.ts b/packages/backend/test/misc/mock-resolver.ts index 0bf85ef8eb..34241d13cb 100644 --- a/packages/backend/test/misc/mock-resolver.ts +++ b/packages/backend/test/misc/mock-resolver.ts @@ -19,6 +19,7 @@ import type { PollsRepository, UsersRepository, } from '@/models/_.js'; +import type { CacheService } from '@/core/CacheService.js'; import { ApLogService } from '@/core/ApLogService.js'; import { ApUtilityService } from '@/core/activitypub/ApUtilityService.js'; import { fromTuple } from '@/misc/from-tuple.js'; @@ -53,6 +54,7 @@ export class MockResolver extends Resolver { loggerService, {} as ApLogService, {} as ApUtilityService, + {} as CacheService, ); } diff --git a/packages/backend/test/misc/noOpCaches.ts b/packages/backend/test/misc/noOpCaches.ts new file mode 100644 index 0000000000..f3cc1e2ba2 --- /dev/null +++ b/packages/backend/test/misc/noOpCaches.ts @@ -0,0 +1,187 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +import * as Redis from 'ioredis'; +import { Inject } from '@nestjs/common'; +import { FakeInternalEventService } from './FakeInternalEventService.js'; +import type { BlockingsRepository, FollowingsRepository, MiUser, MutingsRepository, RenoteMutingsRepository, UserProfilesRepository, UsersRepository } from '@/models/_.js'; +import type { MiLocalUser } from '@/models/User.js'; +import { MemoryKVCache, MemorySingleCache, RedisKVCache, RedisSingleCache } from '@/misc/cache.js'; +import { QuantumKVCache, QuantumKVOpts } from '@/misc/QuantumKVCache.js'; +import { CacheService, FollowStats } from '@/core/CacheService.js'; +import { DI } from '@/di-symbols.js'; +import { UserEntityService } from '@/core/entities/UserEntityService.js'; +import { InternalEventService } from '@/core/InternalEventService.js'; + +export function noOpRedis() { + return { + set: () => Promise.resolve(), + get: () => Promise.resolve(null), + del: () => Promise.resolve(), + on: () => {}, + off: () => {}, + } as unknown as Redis.Redis; +} + +export class NoOpCacheService extends CacheService { + public readonly fakeRedis: { + [K in keyof Redis.Redis]: Redis.Redis[K]; + }; + public readonly fakeInternalEventService: FakeInternalEventService; + + constructor( + @Inject(DI.usersRepository) + usersRepository: UsersRepository, + + @Inject(DI.userProfilesRepository) + userProfilesRepository: UserProfilesRepository, + + @Inject(DI.mutingsRepository) + mutingsRepository: MutingsRepository, + + @Inject(DI.blockingsRepository) + blockingsRepository: BlockingsRepository, + + @Inject(DI.renoteMutingsRepository) + renoteMutingsRepository: RenoteMutingsRepository, + + @Inject(DI.followingsRepository) + followingsRepository: FollowingsRepository, + + @Inject(UserEntityService) + userEntityService: UserEntityService, + ) { + const fakeRedis = noOpRedis(); + const fakeInternalEventService = new FakeInternalEventService(); + + super( + fakeRedis, + fakeRedis, + usersRepository, + userProfilesRepository, + mutingsRepository, + blockingsRepository, + renoteMutingsRepository, + followingsRepository, + userEntityService, + fakeInternalEventService, + ); + + this.fakeRedis = fakeRedis; + this.fakeInternalEventService = fakeInternalEventService; + + // Override caches + this.userByIdCache = new NoOpMemoryKVCache<MiUser>(); + this.localUserByNativeTokenCache = new NoOpMemoryKVCache<MiLocalUser | null>(); + this.localUserByIdCache = new NoOpMemoryKVCache<MiLocalUser>(); + this.uriPersonCache = new NoOpMemoryKVCache<MiUser | null>(); + this.userProfileCache = NoOpQuantumKVCache.copy(this.userProfileCache, fakeInternalEventService); + this.userMutingsCache = NoOpQuantumKVCache.copy(this.userMutingsCache, fakeInternalEventService); + this.userBlockingCache = NoOpQuantumKVCache.copy(this.userBlockingCache, fakeInternalEventService); + this.userBlockedCache = NoOpQuantumKVCache.copy(this.userBlockedCache, fakeInternalEventService); + this.renoteMutingsCache = NoOpQuantumKVCache.copy(this.renoteMutingsCache, fakeInternalEventService); + this.userFollowingsCache = NoOpQuantumKVCache.copy(this.userFollowingsCache, fakeInternalEventService); + this.userFollowersCache = NoOpQuantumKVCache.copy(this.userFollowersCache, fakeInternalEventService); + this.hibernatedUserCache = NoOpQuantumKVCache.copy(this.hibernatedUserCache, fakeInternalEventService); + this.userFollowStatsCache = new NoOpMemoryKVCache<FollowStats>(); + this.translationsCache = NoOpRedisKVCache.copy(this.translationsCache, fakeRedis); + } +} + +export class NoOpMemoryKVCache<T> extends MemoryKVCache<T> { + constructor() { + super(-1); + } +} + +export class NoOpMemorySingleCache<T> extends MemorySingleCache<T> { + constructor() { + super(-1); + } +} + +export class NoOpRedisKVCache<T> extends RedisKVCache<T> { + constructor(opts?: { + redis?: Redis.Redis; + fetcher?: RedisKVCache<T>['fetcher']; + toRedisConverter?: RedisKVCache<T>['toRedisConverter']; + fromRedisConverter?: RedisKVCache<T>['fromRedisConverter']; + }) { + super( + opts?.redis ?? noOpRedis(), + 'no-op', + { + lifetime: -1, + memoryCacheLifetime: -1, + fetcher: opts?.fetcher, + toRedisConverter: opts?.toRedisConverter, + fromRedisConverter: opts?.fromRedisConverter, + }, + ); + } + + public static copy<T>(cache: RedisKVCache<T>, redis?: Redis.Redis): NoOpRedisKVCache<T> { + return new NoOpRedisKVCache<T>({ + redis, + fetcher: cache.fetcher, + toRedisConverter: cache.toRedisConverter, + fromRedisConverter: cache.fromRedisConverter, + }); + } +} + +export class NoOpRedisSingleCache<T> extends RedisSingleCache<T> { + constructor(opts?: { + redis?: Redis.Redis; + fetcher?: RedisSingleCache<T>['fetcher']; + toRedisConverter?: RedisSingleCache<T>['toRedisConverter']; + fromRedisConverter?: RedisSingleCache<T>['fromRedisConverter']; + }) { + super( + opts?.redis ?? noOpRedis(), + 'no-op', + { + lifetime: -1, + memoryCacheLifetime: -1, + fetcher: opts?.fetcher, + toRedisConverter: opts?.toRedisConverter, + fromRedisConverter: opts?.fromRedisConverter, + }, + ); + } + + public static copy<T>(cache: RedisSingleCache<T>, redis?: Redis.Redis): NoOpRedisSingleCache<T> { + return new NoOpRedisSingleCache<T>({ + redis, + fetcher: cache.fetcher, + toRedisConverter: cache.toRedisConverter, + fromRedisConverter: cache.fromRedisConverter, + }); + } +} + +export class NoOpQuantumKVCache<T> extends QuantumKVCache<T> { + constructor(opts: Omit<QuantumKVOpts<T>, 'lifetime'> & { + internalEventService?: InternalEventService, + }) { + super( + opts.internalEventService ?? new FakeInternalEventService(), + 'no-op', + { + ...opts, + lifetime: -1, + }, + ); + } + + public static copy<T>(cache: QuantumKVCache<T>, internalEventService?: InternalEventService): NoOpQuantumKVCache<T> { + return new NoOpQuantumKVCache<T>({ + internalEventService, + fetcher: cache.fetcher, + bulkFetcher: cache.bulkFetcher, + onChanged: cache.onChanged, + }); + } +} diff --git a/packages/backend/test/tsconfig.json b/packages/backend/test/tsconfig.json index 2b562acda8..f3b6a5108d 100644 --- a/packages/backend/test/tsconfig.json +++ b/packages/backend/test/tsconfig.json @@ -23,6 +23,7 @@ "emitDecoratorMetadata": true, "resolveJsonModule": true, "isolatedModules": true, + "incremental": true, "baseUrl": "./", "paths": { "@/*": ["../src/*"] diff --git a/packages/backend/test/unit/AbuseReportNotificationService.ts b/packages/backend/test/unit/AbuseReportNotificationService.ts index 6d555326fb..a67cb3664a 100644 --- a/packages/backend/test/unit/AbuseReportNotificationService.ts +++ b/packages/backend/test/unit/AbuseReportNotificationService.ts @@ -11,6 +11,7 @@ import { AbuseReportNotificationRecipientRepository, MiAbuseReportNotificationRecipient, MiAbuseUserReport, + MiMeta, MiSystemWebhook, MiUser, SystemWebhooksRepository, @@ -56,6 +57,15 @@ describe('AbuseReportNotificationService', () => { // -------------------------------------------------------------------------------------- + const meta = {} as MiMeta; + + function updateMeta(newMeta: Partial<MiMeta>): void { + for (const key in meta) { + delete (meta as any)[key]; + } + Object.assign(meta, newMeta); + } + async function createUser(data: Partial<MiUser> = {}) { const user = await usersRepository .insert({ @@ -66,6 +76,8 @@ describe('AbuseReportNotificationService', () => { await userProfilesRepository.insert({ userId: user.id, + email: user.username + '@example.com', + emailVerified: true, }); return user; @@ -130,6 +142,9 @@ describe('AbuseReportNotificationService', () => { { provide: GlobalEventService, useFactory: () => ({ publishAdminStream: jest.fn() }), }, + { + provide: DI.meta, useFactory: () => meta, + }, ], }) .compile(); @@ -156,6 +171,8 @@ describe('AbuseReportNotificationService', () => { systemWebhook2 = await createWebhook(); roleService.getModeratorIds.mockResolvedValue([root.id, alice.id, bob.id]); + + updateMeta({} as MiMeta); }); afterEach(async () => { @@ -367,8 +384,10 @@ describe('AbuseReportNotificationService', () => { id: idService.gen(), targetUserId: alice.id, targetUser: alice, + targetUserInstance: null, reporterId: bob.id, reporter: bob, + reporterInstance: null, assigneeId: null, assignee: null, resolved: false, @@ -390,4 +409,59 @@ describe('AbuseReportNotificationService', () => { expect(webhookService.enqueueSystemWebhook.mock.calls[0][2]).toEqual({ excludes: [systemWebhook2.id] }); }); }); + + describe('collection of recipient-mails', () => { + async function create() { + const recipient = await createRecipient({ + method: 'email', + userId: alice.id, + }); + + return recipient; + } + + test('with nothing set', async () => { + const mails = await service.getRecipientEMailAddresses(); + expect(mails).toEqual([]); + }); + + test('with maintainer mail set', async () => { + updateMeta({ maintainerEmail: 'maintainer_mail' }); + const mails = await service.getRecipientEMailAddresses(); + expect(mails).toEqual(['maintainer_mail']); + }); + + test('with smtp mail set', async () => { + updateMeta({ email: 'smtp_mail' }); + const mails = await service.getRecipientEMailAddresses(); + expect(mails).toEqual(['smtp_mail']); + }); + + test('with maintainer mail and smtp mail set', async () => { + updateMeta({ email: 'smtp_mail', maintainerEmail: 'maintainer_mail' }); + const mails = await service.getRecipientEMailAddresses(); + expect(mails).toEqual(['smtp_mail', 'maintainer_mail']); + }); + + test('with recipients', async () => { + await create(); + + const mails = await service.getRecipientEMailAddresses(); + expect(mails).toEqual([ + 'alice@example.com', + ]); + }); + + test('with recipients and maintainer mail set and smtp mail set', async () => { + await create(); + updateMeta({ maintainerEmail: 'maintainer_mail', email: 'smtp_mail' }); + + const mails = await service.getRecipientEMailAddresses(); + expect(mails).toEqual([ + 'alice@example.com', + 'smtp_mail', + 'maintainer_mail', + ]); + }); + }); }); diff --git a/packages/backend/test/unit/AnnouncementService.ts b/packages/backend/test/unit/AnnouncementService.ts index a79655c9aa..32d7df05bf 100644 --- a/packages/backend/test/unit/AnnouncementService.ts +++ b/packages/backend/test/unit/AnnouncementService.ts @@ -8,9 +8,12 @@ process.env.NODE_ENV = 'test'; import { jest } from '@jest/globals'; import { ModuleMocker } from 'jest-mock'; import { Test } from '@nestjs/testing'; +import { NoOpCacheService } from '../misc/noOpCaches.js'; +import { FakeInternalEventService } from '../misc/FakeInternalEventService.js'; import { GlobalModule } from '@/GlobalModule.js'; import { AnnouncementService } from '@/core/AnnouncementService.js'; import { AnnouncementEntityService } from '@/core/entities/AnnouncementEntityService.js'; +import { InternalEventService } from '@/core/InternalEventService.js'; import type { AnnouncementReadsRepository, AnnouncementsRepository, @@ -71,24 +74,27 @@ describe('AnnouncementService', () => { AnnouncementEntityService, CacheService, IdService, + InternalEventService, + GlobalEventService, + ModerationLogService, ], }) .useMocker((token) => { - if (token === GlobalEventService) { - return { - publishMainStream: jest.fn(), - publishBroadcastStream: jest.fn(), - }; - } else if (token === ModerationLogService) { - return { - log: jest.fn(), - }; - } else if (typeof token === 'function') { + if (typeof token === 'function') { const mockMetadata = moduleMocker.getMetadata(token) as MockFunctionMetadata<any, any>; const Mock = moduleMocker.generateFromMetadata(mockMetadata); return new Mock(); } }) + .overrideProvider(GlobalEventService).useValue({ + publishMainStream: jest.fn(), + publishBroadcastStream: jest.fn(), + } as unknown as GlobalEventService) + .overrideProvider(ModerationLogService).useValue({ + log: jest.fn(), + }) + .overrideProvider(InternalEventService).useClass(FakeInternalEventService) + .overrideProvider(CacheService).useClass(NoOpCacheService) .compile(); app.enableShutdownHooks(); diff --git a/packages/backend/test/unit/MetaService.ts b/packages/backend/test/unit/MetaService.ts index 19c98eab3d..056838e180 100644 --- a/packages/backend/test/unit/MetaService.ts +++ b/packages/backend/test/unit/MetaService.ts @@ -11,6 +11,7 @@ import { GlobalModule } from '@/GlobalModule.js'; import { DI } from '@/di-symbols.js'; import { MetaService } from '@/core/MetaService.js'; import { CoreModule } from '@/core/CoreModule.js'; +import { MetasRepository } from '@/models/_.js'; import type { TestingModule } from '@nestjs/testing'; import type { DataSource } from 'typeorm'; @@ -39,8 +40,8 @@ describe('MetaService', () => { }); test('fetch (cache)', async () => { - const db = app.get<DataSource>(DI.db); - const spy = jest.spyOn(db, 'transaction'); + const metasRepository = app.get<MetasRepository>(DI.metasRepository); + const spy = jest.spyOn(metasRepository, 'createQueryBuilder'); const result = await metaService.fetch(); @@ -49,12 +50,12 @@ describe('MetaService', () => { }); test('fetch (force)', async () => { - const db = app.get<DataSource>(DI.db); - const spy = jest.spyOn(db, 'transaction'); + const metasRepository = app.get<MetasRepository>(DI.metasRepository); + const spy = jest.spyOn(metasRepository, 'createQueryBuilder'); const result = await metaService.fetch(true); expect(result.id).toBe('x'); - expect(spy).toHaveBeenCalledTimes(1); + expect(spy).toHaveBeenCalled(); }); }); diff --git a/packages/backend/test/unit/MfmService.ts b/packages/backend/test/unit/MfmService.ts index e54c006a4f..af1fc4e132 100644 --- a/packages/backend/test/unit/MfmService.ts +++ b/packages/backend/test/unit/MfmService.ts @@ -4,7 +4,7 @@ */ import * as assert from 'assert'; -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; import { Test } from '@nestjs/testing'; import { CoreModule } from '@/core/CoreModule.js'; @@ -86,7 +86,7 @@ describe('MfmService', () => { test('ruby', async () => { const input = '$[ruby $[group *some* text] ignore me]'; - const output = '<p><ruby><span><span>*some*</span><span> text</span></span><rp>(</rp><rt>ignore me</rt><rp>)</rp></ruby></p>'; + const output = '<p><ruby><span><span>*some*</span> text</span><rp>(</rp><rt>ignore me</rt><rp>)</rp></ruby></p>'; assert.equal(await mfmService.toMastoApiHtml(mfm.parse(input)), output); }); }); diff --git a/packages/backend/test/unit/NoteCreateService.ts b/packages/backend/test/unit/NoteCreateService.ts index f4ecfef34d..63e3795a84 100644 --- a/packages/backend/test/unit/NoteCreateService.ts +++ b/packages/backend/test/unit/NoteCreateService.ts @@ -57,10 +57,13 @@ describe('NoteCreateService', () => { channelId: null, channel: null, userHost: null, + userInstance: null, replyUserId: null, replyUserHost: null, + replyUserInstance: null, renoteUserId: null, renoteUserHost: null, + renoteUserInstance: null, processErrors: [], }; diff --git a/packages/backend/test/unit/RoleService.ts b/packages/backend/test/unit/RoleService.ts index 553ff0982a..2afe22618d 100644 --- a/packages/backend/test/unit/RoleService.ts +++ b/packages/backend/test/unit/RoleService.ts @@ -10,11 +10,15 @@ import { jest } from '@jest/globals'; import { ModuleMocker } from 'jest-mock'; import { Test } from '@nestjs/testing'; import * as lolex from '@sinonjs/fake-timers'; +import { NoOpCacheService } from '../misc/noOpCaches.js'; +import { FakeInternalEventService } from '../misc/FakeInternalEventService.js'; import type { TestingModule } from '@nestjs/testing'; import type { MockFunctionMetadata } from 'jest-mock'; import { GlobalModule } from '@/GlobalModule.js'; import { RoleService } from '@/core/RoleService.js'; import { + InstancesRepository, + MetasRepository, MiMeta, MiRole, MiRoleAssignment, @@ -33,20 +37,36 @@ import { secureRndstr } from '@/misc/secure-rndstr.js'; import { NotificationService } from '@/core/NotificationService.js'; import { RoleCondFormulaValue } from '@/models/Role.js'; import { UserEntityService } from '@/core/entities/UserEntityService.js'; +import { InternalEventService } from '@/core/InternalEventService.js'; const moduleMocker = new ModuleMocker(global); describe('RoleService', () => { let app: TestingModule; let roleService: RoleService; + let instancesRepository: InstancesRepository; let usersRepository: UsersRepository; let rolesRepository: RolesRepository; let roleAssignmentsRepository: RoleAssignmentsRepository; let meta: jest.Mocked<MiMeta>; + let metasRepository: MetasRepository; let notificationService: jest.Mocked<NotificationService>; let clock: lolex.InstalledClock; async function createUser(data: Partial<MiUser> = {}) { + if (data.host != null) { + await instancesRepository + .createQueryBuilder('instance') + .insert() + .values({ + id: genAidx(Date.now()), + firstRetrievedAt: new Date(), + host: data.host, + }) + .orIgnore() + .execute(); + } + const un = secureRndstr(16); const x = await usersRepository.insert({ id: genAidx(Date.now()), @@ -128,26 +148,30 @@ describe('RoleService', () => { provide: NotificationService.name, useExisting: NotificationService, }, + MetaService, + InternalEventService, ], }) .useMocker((token) => { - if (token === MetaService) { - return { fetch: jest.fn() }; - } if (typeof token === 'function') { const mockMetadata = moduleMocker.getMetadata(token) as MockFunctionMetadata<any, any>; const Mock = moduleMocker.generateFromMetadata(mockMetadata); return new Mock(); } }) + .overrideProvider(MetaService).useValue({ fetch: jest.fn() }) + .overrideProvider(InternalEventService).useClass(FakeInternalEventService) + .overrideProvider(CacheService).useClass(NoOpCacheService) .compile(); app.enableShutdownHooks(); roleService = app.get<RoleService>(RoleService); + instancesRepository = app.get<InstancesRepository>(DI.instancesRepository); usersRepository = app.get<UsersRepository>(DI.usersRepository); rolesRepository = app.get<RolesRepository>(DI.rolesRepository); roleAssignmentsRepository = app.get<RoleAssignmentsRepository>(DI.roleAssignmentsRepository); + metasRepository = app.get<MetasRepository>(DI.metasRepository); meta = app.get<MiMeta>(DI.meta) as jest.Mocked<MiMeta>; notificationService = app.get<NotificationService>(NotificationService) as jest.Mocked<NotificationService>; @@ -159,7 +183,7 @@ describe('RoleService', () => { clock.uninstall(); await Promise.all([ - app.get(DI.metasRepository).delete({}), + metasRepository.delete({}), usersRepository.delete({}), rolesRepository.delete({}), roleAssignmentsRepository.delete({}), diff --git a/packages/backend/test/unit/UserSearchService.ts b/packages/backend/test/unit/UserSearchService.ts index 697425beb8..a6b331d1cb 100644 --- a/packages/backend/test/unit/UserSearchService.ts +++ b/packages/backend/test/unit/UserSearchService.ts @@ -7,16 +7,18 @@ import { Test, TestingModule } from '@nestjs/testing'; import { describe, jest, test } from '@jest/globals'; import { In } from 'typeorm'; import { UserSearchService } from '@/core/UserSearchService.js'; -import { FollowingsRepository, MiUser, UserProfilesRepository, UsersRepository } from '@/models/_.js'; +import { FollowingsRepository, InstancesRepository, MiUser, UserProfilesRepository, UsersRepository } from '@/models/_.js'; import { IdService } from '@/core/IdService.js'; import { GlobalModule } from '@/GlobalModule.js'; import { DI } from '@/di-symbols.js'; import { UserEntityService } from '@/core/entities/UserEntityService.js'; +import { genAidx } from '@/misc/id/aidx.js'; describe('UserSearchService', () => { let app: TestingModule; let service: UserSearchService; + let instancesRepository: InstancesRepository; let usersRepository: UsersRepository; let followingsRepository: FollowingsRepository; let idService: IdService; @@ -35,6 +37,19 @@ describe('UserSearchService', () => { let bobby: MiUser; async function createUser(data: Partial<MiUser> = {}) { + if (data.host != null) { + await instancesRepository + .createQueryBuilder('instance') + .insert() + .values({ + id: genAidx(Date.now()), + firstRetrievedAt: new Date(), + host: data.host, + }) + .orIgnore() + .execute(); + } + const user = await usersRepository .insert({ id: idService.gen(), @@ -104,6 +119,7 @@ describe('UserSearchService', () => { await app.init(); + instancesRepository = app.get<InstancesRepository>(DI.instancesRepository); usersRepository = app.get(DI.usersRepository); userProfilesRepository = app.get(DI.userProfilesRepository); followingsRepository = app.get(DI.followingsRepository); diff --git a/packages/backend/test/unit/activitypub.ts b/packages/backend/test/unit/activitypub.ts index 6f6d4c4121..ff93e1be07 100644 --- a/packages/backend/test/unit/activitypub.ts +++ b/packages/backend/test/unit/activitypub.ts @@ -9,8 +9,12 @@ import { generateKeyPair } from 'crypto'; import { Test } from '@nestjs/testing'; import { jest } from '@jest/globals'; +import { NoOpCacheService } from '../misc/noOpCaches.js'; +import { FakeInternalEventService } from '../misc/FakeInternalEventService.js'; import type { Config } from '@/config.js'; import type { MiLocalUser, MiRemoteUser } from '@/models/User.js'; +import { InternalEventService } from '@/core/InternalEventService.js'; +import { CacheService } from '@/core/CacheService.js'; import { ApImageService } from '@/core/activitypub/models/ApImageService.js'; import { ApNoteService } from '@/core/activitypub/models/ApNoteService.js'; import { ApPersonService } from '@/core/activitypub/models/ApPersonService.js'; @@ -30,7 +34,7 @@ import { genAidx } from '@/misc/id/aidx.js'; import { IdService } from '@/core/IdService.js'; import { MockResolver } from '../misc/mock-resolver.js'; import { UserKeypairService } from '@/core/UserKeypairService.js'; -import { MemoryKVCache, RedisKVCache } from '@/misc/cache.js'; +import { MemoryKVCache } from '@/misc/cache.js'; const host = 'https://host1.test'; @@ -103,6 +107,25 @@ describe('ActivityPub', () => { let config: Config; const metaInitial = { + id: 'x', + name: 'Test Instance', + shortName: 'Test Instance', + description: 'Test Instance', + langs: [] as string[], + pinnedUsers: [] as string[], + hiddenTags: [] as string[], + prohibitedWordsForNameOfUser: [] as string[], + silencedHosts: [] as string[], + mediaSilencedHosts: [] as string[], + policies: {}, + serverRules: [] as string[], + bannedEmailDomains: [] as string[], + preservedUsernames: [] as string[], + bubbleInstances: [] as string[], + trustedLinkUrlPatterns: [] as string[], + federation: 'all', + federationHosts: [] as string[], + allowUnsignedFetch: 'always', cacheRemoteFiles: true, cacheRemoteSensitiveFiles: true, enableFanoutTimeline: true, @@ -135,6 +158,8 @@ describe('ActivityPub', () => { }, }) .overrideProvider(DI.meta).useFactory({ factory: () => meta }) + .overrideProvider(CacheService).useClass(NoOpCacheService) + .overrideProvider(InternalEventService).useClass(FakeInternalEventService) .compile(); await app.init(); @@ -454,8 +479,6 @@ describe('ActivityPub', () => { describe('JSON-LD', () => { test('Compaction', async () => { - const jsonLd = jsonLdService.use(); - const object = { '@context': [ 'https://www.w3.org/ns/activitystreams', @@ -474,7 +497,7 @@ describe('ActivityPub', () => { unknown: 'test test bar', undefined: 'test test baz', }; - const compacted = await jsonLd.compact(object); + const compacted = await jsonLdService.compact(object); assert.deepStrictEqual(compacted, { '@context': CONTEXT, @@ -537,7 +560,7 @@ describe('ActivityPub', () => { publicKey, privateKey, }); - ((userKeypairService as unknown as { cache: RedisKVCache<MiUserKeypair> }).cache as unknown as { memoryCache: MemoryKVCache<MiUserKeypair> }).memoryCache.set(author.id, keypair); + (userKeypairService as unknown as { cache: MemoryKVCache<MiUserKeypair> }).cache.set(author.id, keypair); note = new MiNote({ id: idService.gen(), @@ -651,59 +674,6 @@ describe('ActivityPub', () => { }); }); - describe('renderUpnote', () => { - describe('summary', () => { - // I actually don't know why it does this, but the logic was already there so I've preserved it. - it('should be zero-width space when CW is empty string', async () => { - note.cw = ''; - - const result = await rendererService.renderUpNote(note, author, false); - - expect(result.summary).toBe(String.fromCharCode(0x200B)); - }); - - it('should be undefined when CW is null', async () => { - const result = await rendererService.renderUpNote(note, author, false); - - expect(result.summary).toBeUndefined(); - }); - - it('should be CW when present without mandatoryCW', async () => { - note.cw = 'original'; - - const result = await rendererService.renderUpNote(note, author, false); - - expect(result.summary).toBe('original'); - }); - - it('should be mandatoryCW when present without CW', async () => { - author.mandatoryCW = 'mandatory'; - - const result = await rendererService.renderUpNote(note, author, false); - - expect(result.summary).toBe('mandatory'); - }); - - it('should be merged when CW and mandatoryCW are both present', async () => { - note.cw = 'original'; - author.mandatoryCW = 'mandatory'; - - const result = await rendererService.renderUpNote(note, author, false); - - expect(result.summary).toBe('original, mandatory'); - }); - - it('should be CW when CW includes mandatoryCW', async () => { - note.cw = 'original and mandatory'; - author.mandatoryCW = 'mandatory'; - - const result = await rendererService.renderUpNote(note, author, false); - - expect(result.summary).toBe('original and mandatory'); - }); - }); - }); - describe('renderPersonRedacted', () => { it('should include minimal properties', async () => { const result = await rendererService.renderPersonRedacted(author); diff --git a/packages/backend/test/unit/entities/UserEntityService.ts b/packages/backend/test/unit/entities/UserEntityService.ts index ce3f931bb0..4f45f3216d 100644 --- a/packages/backend/test/unit/entities/UserEntityService.ts +++ b/packages/backend/test/unit/entities/UserEntityService.ts @@ -4,6 +4,8 @@ */ import { Test, TestingModule } from '@nestjs/testing'; +import { FakeInternalEventService } from '../../misc/FakeInternalEventService.js'; +import { NoOpCacheService } from '../../misc/noOpCaches.js'; import type { MiUser } from '@/models/User.js'; import { UserEntityService } from '@/core/entities/UserEntityService.js'; import { GlobalModule } from '@/GlobalModule.js'; @@ -51,6 +53,7 @@ import { ReactionService } from '@/core/ReactionService.js'; import { NotificationService } from '@/core/NotificationService.js'; import { ReactionsBufferingService } from '@/core/ReactionsBufferingService.js'; import { ChatService } from '@/core/ChatService.js'; +import { InternalEventService } from '@/core/InternalEventService.js'; process.env.NODE_ENV = 'test'; @@ -174,6 +177,7 @@ describe('UserEntityService', () => { ReactionsBufferingService, NotificationService, ChatService, + InternalEventService, ]; app = await Test.createTestingModule({ @@ -182,7 +186,10 @@ describe('UserEntityService', () => { ...services, ...services.map(x => ({ provide: x.name, useExisting: x })), ], - }).compile(); + }) + .overrideProvider(InternalEventService).useClass(FakeInternalEventService) + .overrideProvider(CacheService).useClass(NoOpCacheService) + .compile(); await app.init(); app.enableShutdownHooks(); diff --git a/packages/backend/test/unit/extract-mentions.ts b/packages/backend/test/unit/extract-mentions.ts index 2aad89d65b..3403387e30 100644 --- a/packages/backend/test/unit/extract-mentions.ts +++ b/packages/backend/test/unit/extract-mentions.ts @@ -5,7 +5,7 @@ import * as assert from 'assert'; -import { parse } from '@transfem-org/sfm-js'; +import { parse } from 'mfm-js'; import { extractMentions } from '@/misc/extract-mentions.js'; describe('Extract mentions', () => { diff --git a/packages/backend/test/unit/misc/QuantumKVCache.ts b/packages/backend/test/unit/misc/QuantumKVCache.ts new file mode 100644 index 0000000000..92792171be --- /dev/null +++ b/packages/backend/test/unit/misc/QuantumKVCache.ts @@ -0,0 +1,799 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +import { jest } from '@jest/globals'; +import { FakeInternalEventService } from '../../misc/FakeInternalEventService.js'; +import { QuantumKVCache, QuantumKVOpts } from '@/misc/QuantumKVCache.js'; + +describe(QuantumKVCache, () => { + let fakeInternalEventService: FakeInternalEventService; + let madeCaches: { dispose: () => void }[]; + + function makeCache<T>(opts?: Partial<QuantumKVOpts<T>> & { name?: string }): QuantumKVCache<T> { + const _opts = { + name: 'test', + lifetime: Infinity, + fetcher: () => { throw new Error('not implemented'); }, + } satisfies QuantumKVOpts<T> & { name: string }; + + if (opts) { + Object.assign(_opts, opts); + } + + const cache = new QuantumKVCache<T>(fakeInternalEventService, _opts.name, _opts); + madeCaches.push(cache); + return cache; + } + + beforeEach(() => { + madeCaches = []; + fakeInternalEventService = new FakeInternalEventService(); + }); + + afterEach(() => { + madeCaches.forEach(cache => { + cache.dispose(); + }); + }); + + it('should connect on construct', () => { + makeCache(); + + expect(fakeInternalEventService._calls).toContainEqual(['on', ['quantumCacheUpdated', expect.anything(), { ignoreLocal: true }]]); + }); + + it('should disconnect on dispose', () => { + const cache = makeCache(); + + cache.dispose(); + + const callback = fakeInternalEventService._calls + .find(c => c[0] === 'on' && c[1][0] === 'quantumCacheUpdated') + ?.[1][1]; + expect(fakeInternalEventService._calls).toContainEqual(['off', ['quantumCacheUpdated', callback]]); + }); + + it('should store in memory cache', async () => { + const cache = makeCache<string>(); + + await cache.set('foo', 'bar'); + await cache.set('alpha', 'omega'); + + const result1 = await cache.get('foo'); + const result2 = await cache.get('alpha'); + + expect(result1).toBe('bar'); + expect(result2).toBe('omega'); + }); + + it('should emit event when storing', async () => { + const cache = makeCache<string>({ name: 'fake' }); + + await cache.set('foo', 'bar'); + + expect(fakeInternalEventService._calls).toContainEqual(['emit', ['quantumCacheUpdated', { name: 'fake', keys: ['foo'] }]]); + }); + + it('should call onChanged when storing', async () => { + const fakeOnChanged = jest.fn(() => Promise.resolve()); + const cache = makeCache<string>({ + name: 'fake', + onChanged: fakeOnChanged, + }); + + await cache.set('foo', 'bar'); + + expect(fakeOnChanged).toHaveBeenCalledWith(['foo'], cache); + }); + + it('should not emit event when storing unchanged value', async () => { + const cache = makeCache<string>({ name: 'fake' }); + + await cache.set('foo', 'bar'); + await cache.set('foo', 'bar'); + + expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(1); + }); + + it('should not call onChanged when storing unchanged value', async () => { + const fakeOnChanged = jest.fn(() => Promise.resolve()); + const cache = makeCache<string>({ + name: 'fake', + onChanged: fakeOnChanged, + }); + + await cache.set('foo', 'bar'); + await cache.set('foo', 'bar'); + + expect(fakeOnChanged).toHaveBeenCalledTimes(1); + }); + + it('should fetch an unknown value', async () => { + const cache = makeCache<string>({ + name: 'fake', + fetcher: key => `value#${key}`, + }); + + const result = await cache.fetch('foo'); + + expect(result).toBe('value#foo'); + }); + + it('should store fetched value in memory cache', async () => { + const cache = makeCache<string>({ + name: 'fake', + fetcher: key => `value#${key}`, + }); + + await cache.fetch('foo'); + + const result = cache.has('foo'); + expect(result).toBe(true); + }); + + it('should call onChanged when fetching', async () => { + const fakeOnChanged = jest.fn(() => Promise.resolve()); + const cache = makeCache<string>({ + name: 'fake', + fetcher: key => `value#${key}`, + onChanged: fakeOnChanged, + }); + + await cache.fetch('foo'); + + expect(fakeOnChanged).toHaveBeenCalledWith(['foo'], cache); + }); + + it('should not emit event when fetching', async () => { + const cache = makeCache<string>({ + name: 'fake', + fetcher: key => `value#${key}`, + }); + + await cache.fetch('foo'); + + expect(fakeInternalEventService._calls).not.toContainEqual(['emit', ['quantumCacheUpdated', { name: 'fake', keys: ['foo'] }]]); + }); + + it('should delete from memory cache', async () => { + const cache = makeCache<string>(); + + await cache.set('foo', 'bar'); + await cache.delete('foo'); + + const result = cache.has('foo'); + expect(result).toBe(false); + }); + + it('should call onChanged when deleting', async () => { + const fakeOnChanged = jest.fn(() => Promise.resolve()); + const cache = makeCache<string>({ + name: 'fake', + onChanged: fakeOnChanged, + }); + + await cache.set('foo', 'bar'); + await cache.delete('foo'); + + expect(fakeOnChanged).toHaveBeenCalledWith(['foo'], cache); + }); + + it('should emit event when deleting', async () => { + const cache = makeCache<string>({ name: 'fake' }); + + await cache.set('foo', 'bar'); + await cache.delete('foo'); + + expect(fakeInternalEventService._calls).toContainEqual(['emit', ['quantumCacheUpdated', { name: 'fake', keys: ['foo'] }]]); + }); + + it('should delete when receiving set event', async () => { + const cache = makeCache<string>({ name: 'fake' }); + await cache.set('foo', 'bar'); + + await fakeInternalEventService._emitRedis('quantumCacheUpdated', { name: 'fake', keys: ['foo'] }); + + const result = cache.has('foo'); + expect(result).toBe(false); + }); + + it('should call onChanged when receiving set event', async () => { + const fakeOnChanged = jest.fn(() => Promise.resolve()); + const cache = makeCache<string>({ + name: 'fake', + onChanged: fakeOnChanged, + }); + + await fakeInternalEventService._emitRedis('quantumCacheUpdated', { name: 'fake', keys: ['foo'] }); + + expect(fakeOnChanged).toHaveBeenCalledWith(['foo'], cache); + }); + + it('should delete when receiving delete event', async () => { + const cache = makeCache<string>({ name: 'fake' }); + await cache.set('foo', 'bar'); + + await fakeInternalEventService._emitRedis('quantumCacheUpdated', { name: 'fake', keys: ['foo'] }); + + const result = cache.has('foo'); + expect(result).toBe(false); + }); + + it('should call onChanged when receiving delete event', async () => { + const fakeOnChanged = jest.fn(() => Promise.resolve()); + const cache = makeCache<string>({ + name: 'fake', + onChanged: fakeOnChanged, + }); + await cache.set('foo', 'bar'); + + await fakeInternalEventService._emitRedis('quantumCacheUpdated', { name: 'fake', keys: ['foo'] }); + + expect(fakeOnChanged).toHaveBeenCalledWith(['foo'], cache); + }); + + describe('get', () => { + it('should return value if present', async () => { + const cache = makeCache<string>(); + await cache.set('foo', 'bar'); + + const result = cache.get('foo'); + + expect(result).toBe('bar'); + }); + it('should return undefined if missing', () => { + const cache = makeCache<string>(); + + const result = cache.get('foo'); + + expect(result).toBe(undefined); + }); + }); + + describe('setMany', () => { + it('should populate all values', async () => { + const cache = makeCache<string>(); + + await cache.setMany([['foo', 'bar'], ['alpha', 'omega']]); + + expect(cache.has('foo')).toBe(true); + expect(cache.has('alpha')).toBe(true); + }); + + it('should emit one event', async () => { + const cache = makeCache<string>({ + name: 'fake', + }); + + await cache.setMany([['foo', 'bar'], ['alpha', 'omega']]); + + expect(fakeInternalEventService._calls).toContainEqual(['emit', ['quantumCacheUpdated', { name: 'fake', keys: ['foo', 'alpha'] }]]); + expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(1); + }); + + it('should call onChanged once with all items', async () => { + const fakeOnChanged = jest.fn(() => Promise.resolve()); + const cache = makeCache<string>({ + name: 'fake', + onChanged: fakeOnChanged, + }); + + await cache.setMany([['foo', 'bar'], ['alpha', 'omega']]); + + expect(fakeOnChanged).toHaveBeenCalledWith(['foo', 'alpha'], cache); + expect(fakeOnChanged).toHaveBeenCalledTimes(1); + }); + + it('should emit events only for changed items', async () => { + const fakeOnChanged = jest.fn(() => Promise.resolve()); + const cache = makeCache<string>({ + name: 'fake', + onChanged: fakeOnChanged, + }); + + await cache.set('foo', 'bar'); + fakeOnChanged.mockClear(); + fakeInternalEventService._reset(); + + await cache.setMany([['foo', 'bar'], ['alpha', 'omega']]); + + expect(fakeInternalEventService._calls).toContainEqual(['emit', ['quantumCacheUpdated', { name: 'fake', keys: ['alpha'] }]]); + expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(1); + expect(fakeOnChanged).toHaveBeenCalledWith(['alpha'], cache); + expect(fakeOnChanged).toHaveBeenCalledTimes(1); + }); + }); + + describe('getMany', () => { + it('should return empty for empty input', () => { + const cache = makeCache(); + const result = cache.getMany([]); + expect(result).toEqual([]); + }); + + it('should return the value for all keys', () => { + const cache = makeCache(); + cache.add('foo', 'bar'); + cache.add('alpha', 'omega'); + + const result = cache.getMany(['foo', 'alpha']); + + expect(result).toEqual([['foo', 'bar'], ['alpha', 'omega']]); + }); + + it('should return undefined for missing keys', () => { + const cache = makeCache(); + cache.add('foo', 'bar'); + + const result = cache.getMany(['foo', 'alpha']); + + expect(result).toEqual([['foo', 'bar'], ['alpha', undefined]]); + }); + }); + + describe('fetchMany', () => { + it('should do nothing for empty input', async () => { + const fakeOnChanged = jest.fn(() => Promise.resolve()); + const cache = makeCache({ + onChanged: fakeOnChanged, + }); + + await cache.fetchMany([]); + + expect(fakeOnChanged).not.toHaveBeenCalled(); + expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(0); + }); + + it('should return existing items', async () => { + const cache = makeCache(); + cache.add('foo', 'bar'); + cache.add('alpha', 'omega'); + + const result = await cache.fetchMany(['foo', 'alpha']); + + expect(result).toEqual([['foo', 'bar'], ['alpha', 'omega']]); + }); + + it('should return existing items without events', async () => { + const fakeOnChanged = jest.fn(() => Promise.resolve()); + const cache = makeCache({ + onChanged: fakeOnChanged, + }); + cache.add('foo', 'bar'); + cache.add('alpha', 'omega'); + + await cache.fetchMany(['foo', 'alpha']); + + expect(fakeOnChanged).not.toHaveBeenCalled(); + expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(0); + }); + + it('should call bulkFetcher for missing items', async () => { + const cache = makeCache({ + bulkFetcher: keys => keys.map(k => [k, `${k}#many`]), + fetcher: key => `${key}#single`, + }); + + const results = await cache.fetchMany(['foo', 'alpha']); + + expect(results).toEqual([['foo', 'foo#many'], ['alpha', 'alpha#many']]); + }); + + it('should call bulkFetcher only once', async () => { + const mockBulkFetcher = jest.fn((keys: string[]) => keys.map(k => [k, `${k}#value`] as [string, string])); + const cache = makeCache({ + bulkFetcher: mockBulkFetcher, + }); + + await cache.fetchMany(['foo', 'bar']); + + expect(mockBulkFetcher).toHaveBeenCalledTimes(1); + }); + + it('should call fetcher when fetchMany is undefined', async () => { + const cache = makeCache({ + fetcher: key => `${key}#single`, + }); + + const results = await cache.fetchMany(['foo', 'alpha']); + + expect(results).toEqual([['foo', 'foo#single'], ['alpha', 'alpha#single']]); + }); + + it('should call onChanged', async () => { + const fakeOnChanged = jest.fn(() => Promise.resolve()); + const cache = makeCache({ + onChanged: fakeOnChanged, + fetcher: k => k, + }); + + await cache.fetchMany(['foo', 'alpha']); + + expect(fakeOnChanged).toHaveBeenCalledWith(['foo', 'alpha'], cache); + expect(fakeOnChanged).toHaveBeenCalledTimes(1); + }); + + it('should call onChanged only for changed', async () => { + const fakeOnChanged = jest.fn(() => Promise.resolve()); + const cache = makeCache({ + onChanged: fakeOnChanged, + fetcher: k => k, + }); + cache.add('foo', 'bar'); + + await cache.fetchMany(['foo', 'alpha']); + + expect(fakeOnChanged).toHaveBeenCalledWith(['alpha'], cache); + expect(fakeOnChanged).toHaveBeenCalledTimes(1); + }); + + it('should not emit event', async () => { + const cache = makeCache({ + fetcher: k => k, + }); + + await cache.fetchMany(['foo', 'alpha']); + + expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(0); + }); + }); + + describe('refreshMany', () => { + it('should do nothing for empty input', async () => { + const fakeOnChanged = jest.fn(() => Promise.resolve()); + const cache = makeCache({ + onChanged: fakeOnChanged, + }); + + const result = await cache.refreshMany([]); + + expect(result).toEqual([]); + expect(fakeOnChanged).not.toHaveBeenCalled(); + expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(0); + }); + + it('should call bulkFetcher for all keys', async () => { + const mockBulkFetcher = jest.fn((keys: string[]) => keys.map(k => [k, `${k}#value`] as [string, string])); + const cache = makeCache({ + bulkFetcher: mockBulkFetcher, + }); + + const result = await cache.refreshMany(['foo', 'alpha']); + + expect(result).toEqual([['foo', 'foo#value'], ['alpha', 'alpha#value']]); + expect(mockBulkFetcher).toHaveBeenCalledWith(['foo', 'alpha'], cache); + expect(mockBulkFetcher).toHaveBeenCalledTimes(1); + }); + + it('should replace any existing keys', async () => { + const mockBulkFetcher = jest.fn((keys: string[]) => keys.map(k => [k, `${k}#value`] as [string, string])); + const cache = makeCache({ + bulkFetcher: mockBulkFetcher, + }); + cache.add('foo', 'bar'); + + const result = await cache.refreshMany(['foo', 'alpha']); + + expect(result).toEqual([['foo', 'foo#value'], ['alpha', 'alpha#value']]); + expect(mockBulkFetcher).toHaveBeenCalledWith(['foo', 'alpha'], cache); + expect(mockBulkFetcher).toHaveBeenCalledTimes(1); + }); + + it('should call onChanged for all keys', async () => { + const fakeOnChanged = jest.fn(() => Promise.resolve()); + const cache = makeCache({ + bulkFetcher: keys => keys.map(k => [k, `${k}#value`]), + onChanged: fakeOnChanged, + }); + cache.add('foo', 'bar'); + + await cache.refreshMany(['foo', 'alpha']); + + expect(fakeOnChanged).toHaveBeenCalledWith(['foo', 'alpha'], cache); + expect(fakeOnChanged).toHaveBeenCalledTimes(1); + }); + + it('should emit event for all keys', async () => { + const cache = makeCache({ + name: 'fake', + bulkFetcher: keys => keys.map(k => [k, `${k}#value`]), + }); + cache.add('foo', 'bar'); + + await cache.refreshMany(['foo', 'alpha']); + + expect(fakeInternalEventService._calls).toContainEqual(['emit', ['quantumCacheUpdated', { name: 'fake', keys: ['foo', 'alpha'] }]]); + expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(1); + }); + }); + + describe('deleteMany', () => { + it('should remove keys from memory cache', async () => { + const cache = makeCache<string>(); + + await cache.set('foo', 'bar'); + await cache.set('alpha', 'omega'); + await cache.deleteMany(['foo', 'alpha']); + + expect(cache.has('foo')).toBe(false); + expect(cache.has('alpha')).toBe(false); + }); + + it('should emit only one event', async () => { + const cache = makeCache<string>({ + name: 'fake', + }); + + await cache.deleteMany(['foo', 'alpha']); + + expect(fakeInternalEventService._calls).toContainEqual(['emit', ['quantumCacheUpdated', { name: 'fake', keys: ['foo', 'alpha'] }]]); + expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(1); + }); + + it('should call onChanged once with all items', async () => { + const fakeOnChanged = jest.fn(() => Promise.resolve()); + const cache = makeCache<string>({ + name: 'fake', + onChanged: fakeOnChanged, + }); + + await cache.deleteMany(['foo', 'alpha']); + + expect(fakeOnChanged).toHaveBeenCalledWith(['foo', 'alpha'], cache); + expect(fakeOnChanged).toHaveBeenCalledTimes(1); + }); + + it('should do nothing if no keys are provided', async () => { + const fakeOnChanged = jest.fn(() => Promise.resolve()); + const cache = makeCache<string>({ + name: 'fake', + onChanged: fakeOnChanged, + }); + + await cache.deleteMany([]); + + expect(fakeOnChanged).not.toHaveBeenCalled(); + expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(0); + }); + }); + + describe('refresh', () => { + it('should populate the value', async () => { + const cache = makeCache<string>({ + name: 'fake', + fetcher: key => `value#${key}`, + }); + + await cache.refresh('foo'); + + const result = cache.has('foo'); + expect(result).toBe(true); + }); + + it('should return the value', async () => { + const cache = makeCache<string>({ + name: 'fake', + fetcher: key => `value#${key}`, + }); + + const result = await cache.refresh('foo'); + + expect(result).toBe('value#foo'); + }); + + it('should replace the value if it exists', async () => { + const cache = makeCache<string>({ + name: 'fake', + fetcher: key => `value#${key}`, + }); + + await cache.set('foo', 'bar'); + const result = await cache.refresh('foo'); + + expect(result).toBe('value#foo'); + }); + + it('should call onChanged', async () => { + const fakeOnChanged = jest.fn(() => Promise.resolve()); + const cache = makeCache<string>({ + name: 'fake', + fetcher: key => `value#${key}`, + onChanged: fakeOnChanged, + }); + + await cache.refresh('foo'); + + expect(fakeOnChanged).toHaveBeenCalledWith(['foo'], cache); + }); + + it('should emit event', async () => { + const cache = makeCache<string>({ + name: 'fake', + fetcher: key => `value#${key}`, + }); + + await cache.refresh('foo'); + + expect(fakeInternalEventService._calls).toContainEqual(['emit', ['quantumCacheUpdated', { name: 'fake', keys: ['foo'] }]]); + }); + }); + + describe('add', () => { + it('should add the item', () => { + const cache = makeCache(); + cache.add('foo', 'bar'); + expect(cache.has('foo')).toBe(true); + }); + + it('should not emit event', () => { + const cache = makeCache({ + name: 'fake', + }); + + cache.add('foo', 'bar'); + + expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(0); + }); + + it('should not call onChanged', () => { + const fakeOnChanged = jest.fn(() => Promise.resolve()); + const cache = makeCache({ + onChanged: fakeOnChanged, + }); + + cache.add('foo', 'bar'); + + expect(fakeOnChanged).not.toHaveBeenCalled(); + }); + }); + + describe('addMany', () => { + it('should add all items', () => { + const cache = makeCache(); + + cache.addMany([['foo', 'bar'], ['alpha', 'omega']]); + + expect(cache.has('foo')).toBe(true); + expect(cache.has('alpha')).toBe(true); + }); + + it('should not emit event', () => { + const cache = makeCache({ + name: 'fake', + }); + + cache.addMany([['foo', 'bar'], ['alpha', 'omega']]); + + expect(fakeInternalEventService._calls.filter(c => c[0] === 'emit')).toHaveLength(0); + }); + + it('should not call onChanged', () => { + const fakeOnChanged = jest.fn(() => Promise.resolve()); + const cache = makeCache({ + onChanged: fakeOnChanged, + }); + + cache.addMany([['foo', 'bar'], ['alpha', 'omega']]); + + expect(fakeOnChanged).not.toHaveBeenCalled(); + }); + }); + + describe('has', () => { + it('should return false when empty', () => { + const cache = makeCache(); + const result = cache.has('foo'); + expect(result).toBe(false); + }); + + it('should return false when value is not in memory', async () => { + const cache = makeCache<string>(); + await cache.set('foo', 'bar'); + + const result = cache.has('alpha'); + + expect(result).toBe(false); + }); + + it('should return true when value is in memory', async () => { + const cache = makeCache<string>(); + await cache.set('foo', 'bar'); + + const result = cache.has('foo'); + + expect(result).toBe(true); + }); + }); + + describe('size', () => { + it('should return 0 when empty', () => { + const cache = makeCache(); + expect(cache.size).toBe(0); + }); + + it('should return correct size when populated', async () => { + const cache = makeCache<string>(); + await cache.set('foo', 'bar'); + + expect(cache.size).toBe(1); + }); + }); + + describe('entries', () => { + it('should return empty when empty', () => { + const cache = makeCache(); + + const result = Array.from(cache.entries()); + + expect(result).toHaveLength(0); + }); + + it('should return all entries when populated', async () => { + const cache = makeCache<string>(); + await cache.set('foo', 'bar'); + + const result = Array.from(cache.entries()); + + expect(result).toEqual([['foo', 'bar']]); + }); + }); + + describe('keys', () => { + it('should return empty when empty', () => { + const cache = makeCache(); + + const result = Array.from(cache.keys()); + + expect(result).toHaveLength(0); + }); + + it('should return all keys when populated', async () => { + const cache = makeCache<string>(); + await cache.set('foo', 'bar'); + + const result = Array.from(cache.keys()); + + expect(result).toEqual(['foo']); + }); + }); + + describe('values', () => { + it('should return empty when empty', () => { + const cache = makeCache(); + + const result = Array.from(cache.values()); + + expect(result).toHaveLength(0); + }); + + it('should return all values when populated', async () => { + const cache = makeCache<string>(); + await cache.set('foo', 'bar'); + + const result = Array.from(cache.values()); + + expect(result).toEqual(['bar']); + }); + }); + + describe('[Symbol.iterator]', () => { + it('should return empty when empty', () => { + const cache = makeCache(); + + const result = Array.from(cache); + + expect(result).toHaveLength(0); + }); + + it('should return all entries when populated', async () => { + const cache = makeCache<string>(); + await cache.set('foo', 'bar'); + + const result = Array.from(cache); + + expect(result).toEqual([['foo', 'bar']]); + }); + }); +}); diff --git a/packages/backend/test/unit/misc/diff-arrays.ts b/packages/backend/test/unit/misc/diff-arrays.ts new file mode 100644 index 0000000000..b6db5e2eca --- /dev/null +++ b/packages/backend/test/unit/misc/diff-arrays.ts @@ -0,0 +1,91 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +import { diffArrays, diffArraysSimple } from '@/misc/diff-arrays.js'; + +describe(diffArrays, () => { + it('should return empty result when both inputs are null', () => { + const result = diffArrays(null, null); + expect(result.added).toHaveLength(0); + expect(result.removed).toHaveLength(0); + }); + + it('should return empty result when both inputs are empty', () => { + const result = diffArrays([], []); + expect(result.added).toHaveLength(0); + expect(result.removed).toHaveLength(0); + }); + + it('should remove before when after is empty', () => { + const result = diffArrays([1, 2, 3], []); + expect(result.added).toHaveLength(0); + expect(result.removed).toEqual([1, 2, 3]); + }); + + it('should deduplicate before when after is empty', () => { + const result = diffArrays([1, 1, 2, 2, 3], []); + expect(result.added).toHaveLength(0); + expect(result.removed).toEqual([1, 2, 3]); + }); + + it('should add after when before is empty', () => { + const result = diffArrays([], [1, 2, 3]); + expect(result.added).toEqual([1, 2, 3]); + expect(result.removed).toHaveLength(0); + }); + + it('should deduplicate after when before is empty', () => { + const result = diffArrays([], [1, 1, 2, 2, 3]); + expect(result.added).toEqual([1, 2, 3]); + expect(result.removed).toHaveLength(0); + }); + + it('should return diff when both have values', () => { + const result = diffArrays( + ['a', 'b', 'c', 'd'], + ['a', 'c', 'e', 'f'], + ); + expect(result.added).toEqual(['e', 'f']); + expect(result.removed).toEqual(['b', 'd']); + }); +}); + +describe(diffArraysSimple, () => { + it('should return false when both inputs are null', () => { + const result = diffArraysSimple(null, null); + expect(result).toBe(false); + }); + + it('should return false when both inputs are empty', () => { + const result = diffArraysSimple([], []); + expect(result).toBe(false); + }); + + it('should return true when before is populated and after is empty', () => { + const result = diffArraysSimple([1, 2, 3], []); + expect(result).toBe(true); + }); + + it('should return true when before is empty and after is populated', () => { + const result = diffArraysSimple([], [1, 2, 3]); + expect(result).toBe(true); + }); + + it('should return true when values have changed', () => { + const result = diffArraysSimple( + ['a', 'a', 'b', 'c'], + ['a', 'b', 'c', 'd'], + ); + expect(result).toBe(true); + }); + + it('should return false when values have not changed', () => { + const result = diffArraysSimple( + ['a', 'a', 'b', 'c'], + ['a', 'b', 'c', 'c'], + ); + expect(result).toBe(false); + }); +}); diff --git a/packages/backend/test/unit/misc/is-renote.ts b/packages/backend/test/unit/misc/is-renote.ts index 24cd2236bb..b6cfa53466 100644 --- a/packages/backend/test/unit/misc/is-renote.ts +++ b/packages/backend/test/unit/misc/is-renote.ts @@ -40,10 +40,13 @@ const base: MiNote = { channelId: null, channel: null, userHost: null, + userInstance: null, replyUserId: null, replyUserHost: null, + replyUserInstance: null, renoteUserId: null, renoteUserHost: null, + renoteUserInstance: null, processErrors: [], }; diff --git a/packages/backend/test/unit/misc/is-retryable-error.ts b/packages/backend/test/unit/misc/is-retryable-error.ts index 096bf64d4f..6d241066f7 100644 --- a/packages/backend/test/unit/misc/is-retryable-error.ts +++ b/packages/backend/test/unit/misc/is-retryable-error.ts @@ -8,6 +8,9 @@ import { AbortError } from 'node-fetch'; import { isRetryableError } from '@/misc/is-retryable-error.js'; import { StatusError } from '@/misc/status-error.js'; import { IdentifiableError } from '@/misc/identifiable-error.js'; +import { CaptchaError, captchaErrorCodes } from '@/core/CaptchaService.js'; +import { FastifyReplyError } from '@/misc/fastify-reply-error.js'; +import { ConflictError } from '@/server/SkRateLimiterService.js'; describe(isRetryableError, () => { it('should return true for retryable StatusError', () => { @@ -55,6 +58,78 @@ describe(isRetryableError, () => { expect(result).toBeTruthy(); }); + it('should return false for CaptchaError with verificationFailed', () => { + const error = new CaptchaError(captchaErrorCodes.verificationFailed, 'verificationFailed'); + const result = isRetryableError(error); + expect(result).toBeFalsy(); + }); + + it('should return false for CaptchaError with invalidProvider', () => { + const error = new CaptchaError(captchaErrorCodes.invalidProvider, 'invalidProvider'); + const result = isRetryableError(error); + expect(result).toBeFalsy(); + }); + + it('should return false for CaptchaError with invalidParameters', () => { + const error = new CaptchaError(captchaErrorCodes.invalidParameters, 'invalidParameters'); + const result = isRetryableError(error); + expect(result).toBeFalsy(); + }); + + it('should return true for CaptchaError with noResponseProvided', () => { + const error = new CaptchaError(captchaErrorCodes.noResponseProvided, 'noResponseProvided'); + const result = isRetryableError(error); + expect(result).toBeTruthy(); + }); + + it('should return true for CaptchaError with requestFailed', () => { + const error = new CaptchaError(captchaErrorCodes.requestFailed, 'requestFailed'); + const result = isRetryableError(error); + expect(result).toBeTruthy(); + }); + + it('should return true for CaptchaError with unknown', () => { + const error = new CaptchaError(captchaErrorCodes.unknown, 'unknown'); + const result = isRetryableError(error); + expect(result).toBeTruthy(); + }); + + it('should return true for CaptchaError with any other', () => { + const error = new CaptchaError(Symbol('temp'), 'unknown'); + const result = isRetryableError(error); + expect(result).toBeTruthy(); + }); + + it('should return false for FastifyReplyError', () => { + const error = new FastifyReplyError(400, 'test error'); + const result = isRetryableError(error); + expect(result).toBeFalsy(); + }); + + it('should return true for ConflictError', () => { + const error = new ConflictError('test error'); + const result = isRetryableError(error); + expect(result).toBeTruthy(); + }); + + it('should return true for AggregateError when all inners are retryable', () => { + const error = new AggregateError([ + new ConflictError(), + new ConflictError(), + ]); + const result = isRetryableError(error); + expect(result).toBeTruthy(); + }); + + it('should return true for AggregateError when any error is not retryable', () => { + const error = new AggregateError([ + new ConflictError(), + new StatusError('test err', 400), + ]); + const result = isRetryableError(error); + expect(result).toBeFalsy(); + }); + const nonErrorInputs = [ [null, 'null'], [undefined, 'undefined'], diff --git a/packages/backend/test/utils.ts b/packages/backend/test/utils.ts index 7f2768488f..52cfb8ac93 100644 --- a/packages/backend/test/utils.ts +++ b/packages/backend/test/utils.ts @@ -11,12 +11,12 @@ import { inspect } from 'node:util'; import WebSocket, { ClientOptions } from 'ws'; import fetch, { File, RequestInit, type Headers } from 'node-fetch'; import { DataSource } from 'typeorm'; -import { load as cheerio } from 'cheerio'; +import { load as cheerio } from 'cheerio/slim'; import { type Response } from 'node-fetch'; import Fastify from 'fastify'; import { entities } from '../src/postgres.js'; import { loadConfig } from '../src/config.js'; -import type { CheerioAPI } from 'cheerio'; +import type { CheerioAPI } from 'cheerio/slim'; import type * as misskey from 'misskey-js'; import { DEFAULT_POLICIES } from '@/core/RoleService.js'; import { validateContentTypeSetAsActivityPub } from '@/core/activitypub/misc/validator.js'; @@ -652,7 +652,7 @@ export async function sendEnvResetRequest() { // 与えられた値を強制的にエラーとみなす。この関数は型安全性を破壊するため、異常系のアサーション以外で用いられるべきではない。 // FIXME(misskey-js): misskey-jsがエラー情報を公開するようになったらこの関数を廃止する -export function castAsError(obj: Record<string, unknown>): { error: ApiError } { +export function castAsError(obj: object | null | undefined): { error: ApiError } { return obj as { error: ApiError }; } diff --git a/packages/backend/tsconfig.json b/packages/backend/tsconfig.json index 392da169ad..afed1f186c 100644 --- a/packages/backend/tsconfig.json +++ b/packages/backend/tsconfig.json @@ -23,6 +23,7 @@ "emitDecoratorMetadata": true, "resolveJsonModule": true, "isolatedModules": true, + "incremental": true, "rootDir": "./src", "baseUrl": "./", "paths": { diff --git a/packages/frontend-embed/package.json b/packages/frontend-embed/package.json index 1a851df49b..6cdfd8f3f1 100644 --- a/packages/frontend-embed/package.json +++ b/packages/frontend-embed/package.json @@ -11,35 +11,25 @@ }, "dependencies": { "@discordapp/twemoji": "15.1.0", - "@phosphor-icons/web": "^2.0.3", - "@rollup/plugin-json": "6.1.0", - "@rollup/plugin-replace": "6.0.2", - "@rollup/pluginutils": "5.1.4", - "@transfem-org/sfm-js": "0.24.5", - "@twemoji/parser": "15.1.1", - "@vitejs/plugin-vue": "5.2.3", - "@vue/compiler-sfc": "3.5.14", - "astring": "1.9.0", + "@phosphor-icons/web": "2.1.2", + "mfm-js": "npm:@transfem-org/sfm-js@0.24.6", "buraha": "0.0.1", - "estree-walker": "3.0.3", "frontend-shared": "workspace:*", "json5": "2.2.3", "misskey-js": "workspace:*", "punycode.js": "2.3.1", - "rollup": "4.40.0", - "sass": "1.87.0", "shiki": "3.3.0", "tinycolor2": "1.6.0", - "tsc-alias": "1.8.15", - "tsconfig-paths": "4.2.0", - "typescript": "5.8.3", "uuid": "11.1.0", - "vite": "6.3.3", "vue": "3.5.14" }, "devDependencies": { - "@misskey-dev/summaly": "5.2.1", + "@misskey-dev/summaly": "npm:@transfem-org/summaly@5.2.2", + "@rollup/plugin-json": "6.1.0", + "@rollup/plugin-replace": "6.0.2", + "@rollup/pluginutils": "5.1.4", "@testing-library/vue": "8.1.0", + "@twemoji/parser": "15.1.1", "@types/estree": "1.0.7", "@types/micromatch": "4.0.9", "@types/node": "22.15.2", @@ -48,12 +38,16 @@ "@types/ws": "8.18.1", "@typescript-eslint/eslint-plugin": "8.31.0", "@typescript-eslint/parser": "8.31.0", + "@vitejs/plugin-vue": "5.2.3", "@vitest/coverage-v8": "3.1.2", + "@vue/compiler-sfc": "3.5.14", "@vue/runtime-core": "3.5.14", "acorn": "8.14.1", + "astring": "1.9.0", "cross-env": "7.0.3", "eslint-plugin-import": "2.31.0", "eslint-plugin-vue": "10.0.0", + "estree-walker": "3.0.3", "fast-glob": "3.3.3", "happy-dom": "17.4.4", "intersection-observer": "0.12.2", @@ -61,7 +55,13 @@ "msw": "2.7.5", "nodemon": "3.1.10", "prettier": "3.5.3", + "rollup": "4.40.0", + "sass": "1.87.0", "start-server-and-test": "2.0.11", + "tsc-alias": "1.8.15", + "tsconfig-paths": "4.2.0", + "typescript": "5.8.3", + "vite": "6.3.3", "vite-plugin-turbosnap": "1.0.3", "vue-component-type-helpers": "2.2.10", "vue-eslint-parser": "10.1.3", diff --git a/packages/frontend-embed/src/boot.ts b/packages/frontend-embed/src/boot.ts index 7c8336ce3f..f11cfef8fd 100644 --- a/packages/frontend-embed/src/boot.ts +++ b/packages/frontend-embed/src/boot.ts @@ -28,7 +28,7 @@ console.log('Sharkey Embed'); //#region Embedパラメータの取得・パース const params = new URLSearchParams(location.search); const embedParams = parseEmbedParams(params); -if (_DEV_) console.log(embedParams); +if (_DEV_) console.debug(embedParams); //#endregion //#region テーマ diff --git a/packages/frontend-embed/src/components/EmMfm.ts b/packages/frontend-embed/src/components/EmMfm.ts index d377d492e0..74ae3373ef 100644 --- a/packages/frontend-embed/src/components/EmMfm.ts +++ b/packages/frontend-embed/src/components/EmMfm.ts @@ -5,7 +5,7 @@ import { h, provide } from 'vue'; import type { VNode, SetupContext } from 'vue'; -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; import * as Misskey from 'misskey-js'; import { host } from '@@/js/config.js'; import EmUrl from '@/components/EmUrl.vue'; diff --git a/packages/frontend-embed/src/components/EmNote.vue b/packages/frontend-embed/src/components/EmNote.vue index 666cbde72d..0dc77d09a7 100644 --- a/packages/frontend-embed/src/components/EmNote.vue +++ b/packages/frontend-embed/src/components/EmNote.vue @@ -105,7 +105,7 @@ SPDX-License-Identifier: AGPL-3.0-only <script lang="ts" setup> import { computed, inject, ref, shallowRef } from 'vue'; -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; import * as Misskey from 'misskey-js'; import { shouldCollapsed } from '@@/js/collapsed.js'; import { url } from '@@/js/config.js'; diff --git a/packages/frontend-embed/src/components/EmNoteDetailed.vue b/packages/frontend-embed/src/components/EmNoteDetailed.vue index 9f4be8c666..8a10778e8a 100644 --- a/packages/frontend-embed/src/components/EmNoteDetailed.vue +++ b/packages/frontend-embed/src/components/EmNoteDetailed.vue @@ -64,17 +64,17 @@ SPDX-License-Identifier: AGPL-3.0-only </p> <div v-show="mergedCW == null || showContent"> <span v-if="appearNote.isHidden" style="opacity: 0.5">({{ i18n.ts.private }})</span> - <EmA v-if="appearNote.replyId" :class="$style.noteReplyTarget" :to="`/notes/${appearNote.replyId}`"><i class="ti ti-arrow-back-up"></i></EmA> - <EmMfm - v-if="appearNote.text" - :parsedNodes="parsed" - :text="appearNote.text" - :author="appearNote.user" - :nyaize="'respect'" - :emojiUrls="appearNote.emojis" - :isBlock="true" - /> - <a v-if="appearNote.renote != null" :class="$style.rn">RN:</a> + <div> + <EmA v-if="appearNote.replyId" :class="$style.noteReplyTarget" :to="`/notes/${appearNote.replyId}`"><i class="ti ti-arrow-back-up"></i></EmA> + <EmMfm + v-if="appearNote.text" + :parsedNodes="parsed" + :text="appearNote.text" + :author="appearNote.user" + :nyaize="'respect'" + :emojiUrls="appearNote.emojis" + /> + </div> <div v-if="appearNote.files && appearNote.files.length > 0"> <EmMediaList :mediaList="appearNote.files" :originalEntityUrl="`${url}/notes/${appearNote.id}`"/> </div> @@ -128,7 +128,7 @@ SPDX-License-Identifier: AGPL-3.0-only <script lang="ts" setup> import { computed, inject, ref } from 'vue'; -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; import * as Misskey from 'misskey-js'; import { computeMergedCw } from '@@/js/compute-merged-cw.js'; import I18n from '@/components/I18n.vue'; diff --git a/packages/frontend-embed/src/post-message.ts b/packages/frontend-embed/src/post-message.ts index 93b57c380b..f44b58acd3 100644 --- a/packages/frontend-embed/src/post-message.ts +++ b/packages/frontend-embed/src/post-message.ts @@ -28,7 +28,7 @@ let defaultIframeId: string | null = null; export function setIframeId(id: string): void { if (defaultIframeId != null) return; - if (_DEV_) console.log('setIframeId', id); + if (_DEV_) console.debug('setIframeId', id); defaultIframeId = id; } @@ -40,7 +40,7 @@ export function postMessageToParentWindow<T extends PostMessageEventType = PostM if (_iframeId == null) { _iframeId = defaultIframeId; } - if (_DEV_) console.log('postMessageToParentWindow', type, _iframeId, payload); + if (_DEV_) console.debug('postMessageToParentWindow', type, _iframeId, payload); window.parent.postMessage({ type, iframeId: _iframeId, diff --git a/packages/frontend-embed/src/style.scss b/packages/frontend-embed/src/style.scss index ba3238cd4c..529bb606be 100644 --- a/packages/frontend-embed/src/style.scss +++ b/packages/frontend-embed/src/style.scss @@ -101,7 +101,7 @@ rt { } } -.ti { +.ti, ph-lg { width: 1.28em; vertical-align: -12%; line-height: 1em; diff --git a/packages/frontend-embed/src/ui.vue b/packages/frontend-embed/src/ui.vue index 4ba5968a91..24db238ca2 100644 --- a/packages/frontend-embed/src/ui.vue +++ b/packages/frontend-embed/src/ui.vue @@ -54,7 +54,7 @@ function safeURIDecode(str: string): string { const page = location.pathname.split('/')[2]; const contentId = safeURIDecode(location.pathname.split('/')[3]); -if (_DEV_) console.log(page, contentId); +if (_DEV_) console.debug(page, contentId); const embedParams = inject(DI.embedParams, defaultEmbedParams); diff --git a/packages/frontend-embed/src/workers/tsconfig.json b/packages/frontend-embed/src/workers/tsconfig.json index 8ee8930465..39ba45ddbb 100644 --- a/packages/frontend-embed/src/workers/tsconfig.json +++ b/packages/frontend-embed/src/workers/tsconfig.json @@ -1,5 +1,6 @@ { "compilerOptions": { "lib": ["esnext", "webworker"], + "incremental": true } } diff --git a/packages/frontend-embed/tsconfig.json b/packages/frontend-embed/tsconfig.json index e0ee08188d..8db5776c91 100644 --- a/packages/frontend-embed/tsconfig.json +++ b/packages/frontend-embed/tsconfig.json @@ -23,6 +23,7 @@ "useDefineForClassFields": true, "verbatimModuleSyntax": true, "skipLibCheck": true, + "incremental": true, "baseUrl": ".", "paths": { "@/*": ["./src/*"], diff --git a/packages/frontend-shared/build.js b/packages/frontend-shared/build.js index 9941114757..f3a94fe364 100644 --- a/packages/frontend-shared/build.js +++ b/packages/frontend-shared/build.js @@ -101,7 +101,7 @@ async function watchSrc() { process.on('SIGHUP', resolve); process.on('SIGINT', resolve); process.on('SIGTERM', resolve); - process.on('uncaughtException', reject); + process.on('uncaughtExceptionMonitor', reject); process.on('exit', resolve); }).finally(async () => { await context.dispose(); diff --git a/packages/frontend-shared/js/intl-const.ts b/packages/frontend-shared/js/intl-const.ts index 33b65b6e9b..9a77cd29ef 100644 --- a/packages/frontend-shared/js/intl-const.ts +++ b/packages/frontend-shared/js/intl-const.ts @@ -20,7 +20,7 @@ try { }); } catch (err) { console.warn(err); - if (_DEV_) console.log('[Intl] Fallback to en-US'); + if (_DEV_) console.debug('[Intl] Fallback to en-US'); // Fallback to en-US _dateTimeFormat = new Intl.DateTimeFormat('en-US', { @@ -43,7 +43,7 @@ try { _numberFormat = new Intl.NumberFormat(versatileLang); } catch (err) { console.warn(err); - if (_DEV_) console.log('[Intl] Fallback to en-US'); + if (_DEV_) console.debug('[Intl] Fallback to en-US'); // Fallback to en-US _numberFormat = new Intl.NumberFormat('en-US'); diff --git a/packages/frontend-shared/js/retry-on-throttled.ts b/packages/frontend-shared/js/retry-on-throttled.ts new file mode 100644 index 0000000000..f73e19b5c7 --- /dev/null +++ b/packages/frontend-shared/js/retry-on-throttled.ts @@ -0,0 +1,31 @@ +/* + * SPDX-FileCopyrightText: outvi and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only +*/ + +async function sleep(ms: number): Promise<void> { + return new Promise((resolve) => { + window.setTimeout(() => { + resolve(); + }, ms); + }); +} + +export async function retryOnThrottled<T>(f: () => Promise<T>, retryCount = 5): Promise<T> { + let lastError; + for (let i = 0; i < Math.min(retryCount, 1); i++) { + try { + return await f(); + } catch (err: any) { + // RATE_LIMIT_EXCEEDED + if (typeof err === 'object' && err?.id === 'd5826d14-3982-4d2e-8011-b9e9f02499ef') { + lastError = err; + await sleep(err?.info?.fullResetMs ?? 1000); + } else { + throw err; + } + } + } + + throw lastError; +} diff --git a/packages/frontend-shared/js/worker-multi-dispatch.ts b/packages/frontend-shared/js/worker-multi-dispatch.ts index 5d393ed1ed..808b568d66 100644 --- a/packages/frontend-shared/js/worker-multi-dispatch.ts +++ b/packages/frontend-shared/js/worker-multi-dispatch.ts @@ -28,13 +28,13 @@ export class WorkerMultiDispatch<POST = unknown, RETURN = unknown> { }); this.finalizationRegistry.register(this, this.symbol); - if (_DEV_) console.log('WorkerMultiDispatch: Created', this); + if (_DEV_) console.debug('WorkerMultiDispatch: Created', this); } public postMessage(message: POST, options?: Transferable[] | StructuredSerializeOptions, useWorkerNumber: WorkerNumberGetter = this.getUseWorkerNumber) { let workerNumber = useWorkerNumber(this.prevWorkerNumber, this.workers.length); workerNumber = Math.abs(Math.round(workerNumber)) % this.workers.length; - if (_DEV_) console.log('WorkerMultiDispatch: Posting message to worker', workerNumber, useWorkerNumber); + if (_DEV_) console.debug('WorkerMultiDispatch: Posting message to worker', workerNumber, useWorkerNumber); this.prevWorkerNumber = workerNumber; // 不毛だがunionをoverloadに突っ込めない @@ -64,7 +64,7 @@ export class WorkerMultiDispatch<POST = unknown, RETURN = unknown> { public terminate() { this.terminated = true; - if (_DEV_) console.log('WorkerMultiDispatch: Terminating', this); + if (_DEV_) console.debug('WorkerMultiDispatch: Terminating', this); this.workers.forEach(worker => { worker.terminate(); }); diff --git a/packages/frontend-shared/package.json b/packages/frontend-shared/package.json index f129121d19..b4a5dd89f5 100644 --- a/packages/frontend-shared/package.json +++ b/packages/frontend-shared/package.json @@ -35,7 +35,6 @@ ], "dependencies": { "misskey-js": "workspace:*", - "nodemon": "3.1.7", "vue": "3.5.13" } } diff --git a/packages/frontend-shared/tsconfig.json b/packages/frontend-shared/tsconfig.json index 8f76763e10..0512b50caf 100644 --- a/packages/frontend-shared/tsconfig.json +++ b/packages/frontend-shared/tsconfig.json @@ -18,6 +18,7 @@ "esModuleInterop": true, "verbatimModuleSyntax": true, "skipLibCheck": true, + "incremental": true, "baseUrl": ".", "paths": { "@/*": ["./*"], diff --git a/packages/frontend/.storybook/tsconfig.json b/packages/frontend/.storybook/tsconfig.json index f325114522..18baf516ba 100644 --- a/packages/frontend/.storybook/tsconfig.json +++ b/packages/frontend/.storybook/tsconfig.json @@ -18,6 +18,7 @@ "esModuleInterop": true, "skipLibCheck": true, "forceConsistentCasingInFileNames": true, + "incremental": true, "jsx": "react", "jsxFactory": "h" }, diff --git a/packages/frontend/package.json b/packages/frontend/package.json index f5c7bcf1b4..5d028ce142 100644 --- a/packages/frontend/package.json +++ b/packages/frontend/package.json @@ -20,19 +20,11 @@ "@github/webauthn-json": "2.1.1", "@mcaptcha/vanilla-glue": "0.1.0-alpha-3", "@misskey-dev/browser-image-resizer": "2024.1.0", - "@phosphor-icons/web": "^2.0.3", - "@rollup/plugin-json": "6.1.0", - "@rollup/plugin-replace": "6.0.2", - "@rollup/pluginutils": "5.1.4", + "@phosphor-icons/web": "2.1.2", "@ruffle-rs/ruffle": "0.1.0-nightly.2024.10.15", "@sentry/vue": "9.14.0", "@syuilo/aiscript": "0.19.0", - "@transfem-org/sfm-js": "0.24.6", - "@twemoji/parser": "15.1.1", - "@vitejs/plugin-vue": "5.2.3", - "@vue/compiler-sfc": "3.5.14", "aiscript-vscode": "github:aiscript-dev/aiscript-vscode#v0.1.15", - "astring": "1.9.0", "broadcast-channel": "7.1.0", "buraha": "0.0.1", "canvas-confetti": "1.9.3", @@ -45,37 +37,30 @@ "compare-versions": "6.1.1", "cropperjs": "2.0.0", "date-fns": "4.1.0", - "estree-walker": "3.0.3", "eventemitter3": "5.0.1", "frontend-shared": "workspace:*", "idb-keyval": "6.2.1", "insert-text-at-cursor": "0.3.0", "is-file-animated": "1.0.2", "json5": "2.2.3", - "katex": "0.16.10", - "magic-string": "0.30.17", + "katex": "0.16.22", "matter-js": "0.20.0", "misskey-bubble-game": "workspace:*", "misskey-js": "workspace:*", "misskey-reversi": "workspace:*", - "moment": "^2.30.1", + "moment": "2.30.1", "photoswipe": "5.4.4", + "promise-limit": "2.7.0", "punycode.js": "2.3.1", - "rollup": "4.40.0", "sanitize-html": "2.16.0", - "sass": "1.87.0", "shiki": "3.3.0", "strict-event-emitter-types": "2.0.0", "textarea-caret": "3.1.0", - "three": "0.176.0", "throttle-debounce": "5.0.2", "tinycolor2": "1.6.0", - "tsc-alias": "1.8.15", - "tsconfig-paths": "4.2.0", "typescript": "5.8.3", "uuid": "11.1.0", "v-code-diff": "1.13.1", - "vite": "6.3.3", "vue": "3.5.14", "vuedraggable": "next", "wanakana": "5.3.1" @@ -84,7 +69,10 @@ "cypress": "14.3.2" }, "devDependencies": { - "@misskey-dev/summaly": "5.2.1", + "@misskey-dev/summaly": "npm:@transfem-org/summaly@5.2.2", + "@rollup/plugin-json": "6.1.0", + "@rollup/plugin-replace": "6.0.2", + "@rollup/pluginutils": "5.1.4", "@storybook/addon-actions": "8.6.12", "@storybook/addon-essentials": "8.6.12", "@storybook/addon-interactions": "8.6.12", @@ -104,9 +92,10 @@ "@storybook/vue3": "8.6.12", "@storybook/vue3-vite": "8.6.12", "@testing-library/vue": "8.1.0", + "@twemoji/parser": "15.1.1", "@types/canvas-confetti": "1.9.0", "@types/estree": "1.0.7", - "@types/katex": "^0.16.7", + "@types/katex": "0.16.7", "@types/matter-js": "0.19.8", "@types/micromatch": "4.0.9", "@types/node": "22.15.2", @@ -118,16 +107,22 @@ "@types/ws": "8.18.1", "@typescript-eslint/eslint-plugin": "8.31.0", "@typescript-eslint/parser": "8.31.0", + "@vitejs/plugin-vue": "5.2.3", "@vitest/coverage-v8": "3.1.2", "@vue/compiler-core": "3.5.14", + "@vue/compiler-sfc": "3.5.14", "@vue/runtime-core": "3.5.14", + "mfm-js": "npm:@transfem-org/sfm-js@0.24.6", "acorn": "8.14.1", + "astring": "1.9.0", "cross-env": "7.0.3", "eslint-plugin-import": "2.31.0", "eslint-plugin-vue": "10.0.0", + "estree-walker": "3.0.3", "fast-glob": "3.3.3", "happy-dom": "17.4.4", "intersection-observer": "0.12.2", + "magic-string": "0.30.17", "micromatch": "4.0.8", "minimatch": "10.0.1", "msw": "2.7.5", @@ -136,10 +131,16 @@ "prettier": "3.5.3", "react": "19.1.0", "react-dom": "19.1.0", + "rollup": "4.40.0", + "sass": "1.87.0", "seedrandom": "3.0.5", "start-server-and-test": "2.0.11", "storybook": "8.6.12", "storybook-addon-misskey-theme": "github:misskey-dev/storybook-addon-misskey-theme", + "three": "0.176.0", + "tsc-alias": "1.8.15", + "tsconfig-paths": "4.2.0", + "vite": "6.3.3", "vite-plugin-turbosnap": "1.0.3", "vitest": "3.1.2", "vitest-fetch-mock": "0.4.5", diff --git a/packages/frontend/src/components/MkAbuseReport.vue b/packages/frontend/src/components/MkAbuseReport.vue index c52fdb898e..6025bc44f0 100644 --- a/packages/frontend/src/components/MkAbuseReport.vue +++ b/packages/frontend/src/components/MkAbuseReport.vue @@ -33,10 +33,28 @@ SPDX-License-Identifier: AGPL-3.0-only <MkFolder :withSpacer="false"> <template #icon><MkAvatar :user="report.targetUser" style="width: 18px; height: 18px;"/></template> <template #label>{{ i18n.ts.target }}: <MkAcct :user="report.targetUser"/></template> - <template #suffix>#{{ report.targetUserId.toUpperCase() }}</template> + <template #suffix>{{ i18n.ts.id }}# {{ report.targetUserId }}</template> - <div style="height: 300px; --MI-stickyTop: 0; --MI-stickyBottom: 0;"> - <RouterView :router="targetRouter"/> + <div style="--MI-stickyTop: 0; --MI-stickyBottom: 0;"> + <admin-user :userId="report.targetUserId" :userHint="report.targetUser"></admin-user> + </div> + </MkFolder> + + <MkFolder v-if="report.targetInstance" :withSpacer="false"> + <template #icon> + <img + v-if="targetInstanceIcon" + :src="targetInstanceIcon" + :alt="i18n.tsx.instanceIconAlt({ name: report.targetInstance.name || report.targetInstance.host })" + :class="$style.instanceIcon" + class="icon" + /> + </template> + <template #label>{{ i18n.ts.instance }}: {{ report.targetInstance.name || report.targetInstance.host }}</template> + <template #suffix>{{ i18n.ts.id }}# {{ report.targetInstance.id }}</template> + + <div style="--MI-stickyTop: 0; --MI-stickyBottom: 0;"> + <instance-info :host="report.targetInstance.host" :instanceHint="report.targetInstance" :metaHint="metaHint"></instance-info> </div> </MkFolder> @@ -44,23 +62,24 @@ SPDX-License-Identifier: AGPL-3.0-only <template #icon><i class="ti ti-message-2"></i></template> <template #label>{{ i18n.ts.details }}</template> <div class="_gaps_s"> - <Mfm :text="report.comment" :isBlock="true" :linkNavigationBehavior="'window'"/> + <Mfm :text="report.comment" :parsedNodes="parsedComment" :isBlock="true" :linkNavigationBehavior="'window'" :author="report.reporter" :nyaize="false" :isAnim="false"/> + <SkUrlPreviewGroup :sourceNodes="parsedComment" :compact="false" :detail="false" :showAsQuote="true"/> </div> </MkFolder> <MkFolder :withSpacer="false"> <template #icon><MkAvatar :user="report.reporter" style="width: 18px; height: 18px;"/></template> <template #label>{{ i18n.ts.reporter }}: <MkAcct :user="report.reporter"/></template> - <template #suffix>#{{ report.reporterId.toUpperCase() }}</template> + <template #suffix>{{ i18n.ts.id }}# {{ report.reporterId }}</template> - <div style="height: 300px; --MI-stickyTop: 0; --MI-stickyBottom: 0;"> - <RouterView :router="reporterRouter"/> + <div style="--MI-stickyTop: 0; --MI-stickyBottom: 0;"> + <admin-user :userId="report.reporterId" :userHint="report.reporter"></admin-user> </div> </MkFolder> <MkFolder :defaultOpen="false"> <template #icon><i class="ti ti-message-2"></i></template> - <template #label>{{ i18n.ts.moderationNote }}</template> + <template #label>{{ i18n.ts.staffNotes }}</template> <template #suffix>{{ moderationNote.length > 0 ? '...' : i18n.ts.none }}</template> <div class="_gaps_s"> <MkTextarea v-model="moderationNote" manualSave> @@ -78,8 +97,9 @@ SPDX-License-Identifier: AGPL-3.0-only </template> <script lang="ts" setup> -import { provide, ref, watch } from 'vue'; +import { computed, provide, ref, watch } from 'vue'; import * as Misskey from 'misskey-js'; +import * as mfm from 'mfm-js'; import MkButton from '@/components/MkButton.vue'; import MkSwitch from '@/components/MkSwitch.vue'; import MkKeyValue from '@/components/MkKeyValue.vue'; @@ -91,19 +111,38 @@ import RouterView from '@/components/global/RouterView.vue'; import MkTextarea from '@/components/MkTextarea.vue'; import { copyToClipboard } from '@/utility/copy-to-clipboard.js'; import { createRouter } from '@/router.js'; +import { getProxiedImageUrlNullable } from '@/utility/media-proxy'; +import InstanceInfo from '@/pages/instance-info.vue'; +import { iAmAdmin } from '@/i'; +import { misskeyApi } from '@/utility/misskey-api'; +import AdminUser from '@/pages/admin-user.vue'; +import SkUrlPreviewGroup from '@/components/SkUrlPreviewGroup.vue'; -const props = defineProps<{ +const props = withDefaults(defineProps<{ report: Misskey.entities.AdminAbuseUserReportsResponse[number]; -}>(); + metaHint?: Misskey.entities.AdminMetaResponse | undefined; +}>(), { + metaHint: undefined, +}); const emit = defineEmits<{ (ev: 'resolved', reportId: string): void; }>(); +/* const targetRouter = createRouter(`/admin/user/${props.report.targetUserId}`); targetRouter.init(); const reporterRouter = createRouter(`/admin/user/${props.report.reporterId}`); reporterRouter.init(); +*/ + +const parsedComment = computed(() => mfm.parse(props.report.comment)); + +const targetInstanceIcon = computed(() => props.report.targetInstance?.faviconUrl + ? getProxiedImageUrlNullable(props.report.targetInstance.faviconUrl, 'preview') + : props.report.targetInstance?.iconUrl + ? getProxiedImageUrlNullable(props.report.targetInstance.iconUrl, 'preview') + : null); const moderationNote = ref(props.report.moderationNote ?? ''); @@ -150,4 +189,8 @@ function showMenu(ev: MouseEvent) { </script> <style lang="scss" module> +.instanceIcon { + width: 18px; + height: 18px; +} </style> diff --git a/packages/frontend/src/components/MkCaptcha.vue b/packages/frontend/src/components/MkCaptcha.vue index 21f604aa43..e19c6435ef 100644 --- a/packages/frontend/src/components/MkCaptcha.vue +++ b/packages/frontend/src/components/MkCaptcha.vue @@ -142,7 +142,7 @@ function reset() { function remove() { if (captcha.value.remove && captchaWidgetId.value) { try { - if (_DEV_) console.log('remove', props.provider, captchaWidgetId.value); + if (_DEV_) console.debug('remove', props.provider, captchaWidgetId.value); captcha.value.remove(captchaWidgetId.value); } catch (error: unknown) { // ignore diff --git a/packages/frontend/src/components/MkCode.core.vue b/packages/frontend/src/components/MkCode.core.vue index 40f41f5d0f..36c08a8c64 100644 --- a/packages/frontend/src/components/MkCode.core.vue +++ b/packages/frontend/src/components/MkCode.core.vue @@ -52,7 +52,7 @@ async function fetchLanguage(to: string): Promise<void> { return bundle.id === language || bundle.aliases?.includes(language); }); if (bundles.length > 0) { - if (_DEV_) console.log(`Loading language: ${language}`); + if (_DEV_) console.debug(`Loading language: ${language}`); await highlighter.loadLanguage(bundles[0].import); codeLang.value = language; } else { diff --git a/packages/frontend/src/components/MkDateSeparatedList.vue b/packages/frontend/src/components/MkDateSeparatedList.vue index 63e6b74154..8cf4e5fa2d 100644 --- a/packages/frontend/src/components/MkDateSeparatedList.vue +++ b/packages/frontend/src/components/MkDateSeparatedList.vue @@ -16,6 +16,7 @@ import { instance } from '@/instance.js'; import { prefer } from '@/preferences.js'; import { getDateText } from '@/utility/timeline-date-separate.js'; import { $i } from '@/i.js'; +import SkTransitionGroup from '@/components/SkTransitionGroup.vue'; export default defineComponent({ props: { @@ -146,14 +147,12 @@ export default defineComponent({ [$style['direction-up']]: props.direction === 'up', }; - return () => prefer.s.animation ? h(TransitionGroup, { + return () => h(SkTransitionGroup, { class: classes, name: 'list', tag: 'div', onBeforeLeave, onLeaveCancelled, - }, { default: renderChildren }) : h('div', { - class: classes, }, { default: renderChildren }); }, }); diff --git a/packages/frontend/src/components/MkDonation.vue b/packages/frontend/src/components/MkDonation.vue index 43d2002204..dfdfc0a871 100644 --- a/packages/frontend/src/components/MkDonation.vue +++ b/packages/frontend/src/components/MkDonation.vue @@ -53,6 +53,7 @@ import { i18n } from '@/i18n.js'; import * as os from '@/os.js'; import { miLocalStorage } from '@/local-storage.js'; import { instance } from '@/instance.js'; +import { prefer } from '@/preferences.js'; const emit = defineEmits<{ (ev: 'closed'): void; @@ -66,6 +67,7 @@ function close() { } function neverShow() { + prefer.commit('neverShowDonationInfo', 'true'); miLocalStorage.setItem('neverShowDonationInfo', 'true'); close(); } diff --git a/packages/frontend/src/components/MkFolder.vue b/packages/frontend/src/components/MkFolder.vue index 2e5d0a3dea..4537bc9d82 100644 --- a/packages/frontend/src/components/MkFolder.vue +++ b/packages/frontend/src/components/MkFolder.vue @@ -5,7 +5,7 @@ SPDX-License-Identifier: AGPL-3.0-only <template> <div ref="rootEl" :class="$style.root" role="group" :aria-expanded="opened"> - <MkStickyContainer> + <MkStickyContainer :sticky="sticky"> <template #header> <button :class="[$style.header, { [$style.opened]: opened }]" class="_button" role="button" data-cy-folder-header @click="toggle"> <div :class="$style.headerIcon"><slot name="icon"></slot></div> @@ -34,7 +34,7 @@ SPDX-License-Identifier: AGPL-3.0-only > <KeepAlive> <div v-show="opened"> - <MkStickyContainer> + <MkStickyContainer :sticky="sticky"> <template #header> <div v-if="$slots.header" :class="$style.inBodyHeader"> <slot name="header"></slot> @@ -73,12 +73,14 @@ const props = withDefaults(defineProps<{ withSpacer?: boolean; spacerMin?: number; spacerMax?: number; + sticky?: boolean; }>(), { defaultOpen: false, maxHeight: null, withSpacer: true, spacerMin: 14, spacerMax: 22, + sticky: true, }); const rootEl = useTemplateRef('rootEl'); diff --git a/packages/frontend/src/components/MkNote.vue b/packages/frontend/src/components/MkNote.vue index 366321565d..56bfa5de94 100644 --- a/packages/frontend/src/components/MkNote.vue +++ b/packages/frontend/src/components/MkNote.vue @@ -72,20 +72,21 @@ SPDX-License-Identifier: AGPL-3.0-only <div v-show="mergedCW == null || showContent" :class="[{ [$style.contentCollapsed]: collapsed }]"> <div :class="$style.text"> <span v-if="appearNote.isHidden" style="opacity: 0.5">({{ i18n.ts.private }})</span> - <MkA v-if="appearNote.replyId" :class="$style.replyIcon" :to="`/notes/${appearNote.replyId}`"><i class="ph-arrow-bend-left-up ph-bold ph-lg"></i></MkA> - <Mfm - v-if="appearNote.text" - :parsedNodes="parsed" - :text="appearNote.text" - :author="appearNote.user" - :nyaize="'respect'" - :emojiUrls="appearNote.emojis" - :enableEmojiMenu="true" - :enableEmojiMenuReaction="true" - :isAnim="allowAnim" - :isBlock="true" - class="_selectable" - /> + <div> + <MkA v-if="appearNote.replyId" :class="$style.replyIcon" :to="`/notes/${appearNote.replyId}`"><i class="ph-arrow-bend-left-up ph-bold ph-lg"></i></MkA> + <Mfm + v-if="appearNote.text" + :parsedNodes="parsed" + :text="appearNote.text" + :author="appearNote.user" + :nyaize="'respect'" + :emojiUrls="appearNote.emojis" + :enableEmojiMenu="true" + :enableEmojiMenuReaction="true" + :isAnim="allowAnim" + class="_selectable" + /> + </div> <SkNoteTranslation :note="note" :translation="translation" :translating="translating"></SkNoteTranslation> <MkButton v-if="!allowAnim && animated" :class="$style.playMFMButton" :small="true" @click="animatedMFM()" @click.stop><i class="ph-play ph-bold ph-lg "></i> {{ i18n.ts._animatedMFM.play }}</MkButton> <MkButton v-else-if="!prefer.s.animatedMfm && allowAnim && animated" :class="$style.playMFMButton" :small="true" @click="animatedMFM()" @click.stop><i class="ph-stop ph-bold ph-lg "></i> {{ i18n.ts._animatedMFM.stop }}</MkButton> @@ -95,7 +96,7 @@ SPDX-License-Identifier: AGPL-3.0-only </div> <MkPoll v-if="appearNote.poll" :noteId="appearNote.id" :poll="appearNote.poll" :local="!appearNote.user.host" :author="appearNote.user" :emojiUrls="appearNote.emojis" :class="$style.poll" @click.stop/> <div v-if="isEnabledUrlPreview"> - <MkUrlPreview v-for="url in urls" :key="url" :url="url" :compact="true" :detail="false" :showAsQuote="!appearNote.user.rejectQuotes" :skipNoteIds="selfNoteIds" :class="$style.urlPreview" @click.stop/> + <SkUrlPreviewGroup :sourceUrls="urls" :sourceNote="appearNote" :compact="true" :detail="false" :showAsQuote="!appearNote.user.rejectQuotes" :skipNoteIds="selfNoteIds" :class="$style.urlPreview" @click.stop/> </div> <div v-if="appearNote.renote" :class="$style.quote"><MkNoteSimple :note="appearNote.renote" :class="$style.quoteNote"/></div> <button v-if="isLong && collapsed" :class="$style.collapsed" class="_button" @click.stop @click="collapsed = false"> @@ -113,7 +114,7 @@ SPDX-License-Identifier: AGPL-3.0-only <MkA :to="`/notes/${appearNote.id}/reactions`" :class="[$style.reactionOmitted]">{{ i18n.ts.more }}</MkA> </template> </MkReactionsViewer> - <footer :class="$style.footer"> + <footer :class="$style.footer" class="_gaps _h_gaps" tabindex="0" role="group" :aria-label="i18n.ts.noteFooterLabel"> <button :class="$style.footerButton" class="_button" @click.stop @click="reply()"> <i class="ti ti-arrow-back-up"></i> <p v-if="appearNote.repliesCount > 0" :class="$style.footerButtonCount">{{ number(appearNote.repliesCount) }}</p> @@ -157,7 +158,7 @@ SPDX-License-Identifier: AGPL-3.0-only <button v-if="prefer.s.showClipButtonInNoteFooter" ref="clipButton" :class="$style.footerButton" class="_button" @click.stop="clip()"> <i class="ti ti-paperclip"></i> </button> - <button v-if="prefer.s.showTranslationButtonInNoteFooter && $i?.policies.canUseTranslator && instance.translatorAvailable" ref="translationButton" class="_button" :class="$style.footerButton" :disabled="translating || !!translation" @click.stop="translate()"> + <button v-if="prefer.s.showTranslationButtonInNoteFooter && policies.canUseTranslator && instance.translatorAvailable" ref="translationButton" class="_button" :class="$style.footerButton" :disabled="translating || !!translation" @click.stop="translate()"> <i class="ti ti-language-hiragana"></i> </button> <button ref="menuButton" :class="$style.footerButton" class="_button" @click.stop="showMenu()"> @@ -180,7 +181,7 @@ SPDX-License-Identifier: AGPL-3.0-only <script lang="ts" setup> import { computed, inject, onMounted, ref, useTemplateRef, watch, provide } from 'vue'; -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; import * as Misskey from 'misskey-js'; import { isLink } from '@@/js/is-link.js'; import { shouldCollapsed } from '@@/js/collapsed.js'; @@ -226,7 +227,7 @@ import { getNoteSummary } from '@/utility/get-note-summary.js'; import MkRippleEffect from '@/components/MkRippleEffect.vue'; import { showMovedDialog } from '@/utility/show-moved-dialog.js'; import { boostMenuItems, computeRenoteTooltip } from '@/utility/boost-quote.js'; -import { instance, isEnabledUrlPreview } from '@/instance.js'; +import { instance, isEnabledUrlPreview, policies } from '@/instance.js'; import { focusPrev, focusNext } from '@/utility/focus.js'; import { getAppearNote } from '@/utility/get-appear-note.js'; import { prefer } from '@/preferences.js'; @@ -237,6 +238,7 @@ import SkMutedNote from '@/components/SkMutedNote.vue'; import SkNoteTranslation from '@/components/SkNoteTranslation.vue'; import { getSelfNoteIds } from '@/utility/get-self-note-ids.js'; import { extractPreviewUrls } from '@/utility/extract-preview-urls.js'; +import SkUrlPreviewGroup from '@/components/SkUrlPreviewGroup.vue'; const props = withDefaults(defineProps<{ note: Misskey.entities.Note; @@ -304,9 +306,9 @@ const galleryEl = useTemplateRef('galleryEl'); const isMyRenote = $i && ($i.id === note.value.userId); const showContent = ref(prefer.s.uncollapseCW); const parsed = computed(() => appearNote.value.text ? mfm.parse(appearNote.value.text) : null); -const urls = computed(() => parsed.value ? extractPreviewUrls(props.note, parsed.value) : null); +const urls = computed(() => parsed.value ? extractPreviewUrls(appearNote.value, parsed.value) : []); const selfNoteIds = computed(() => getSelfNoteIds(props.note)); -const isLong = shouldCollapsed(appearNote.value, urls.value ?? []); +const isLong = shouldCollapsed(appearNote.value, urls.value); const collapsed = ref(prefer.s.expandLongNote && appearNote.value.cw == null && isLong ? false : appearNote.value.cw == null && isLong); const isDeleted = ref(false); const renoted = ref(false); @@ -360,7 +362,7 @@ const keymap = { clip(); }, 't': () => { - if (prefer.s.showTranslationButtonInNoteFooter && $i?.policies.canUseTranslator && instance.translatorAvailable) { + if (prefer.s.showTranslationButtonInNoteFooter && policies.value.canUseTranslator && instance.translatorAvailable) { translate(); } }, @@ -913,11 +915,11 @@ function emitUpdReaction(emoji: string, delta: number) { .footer { display: flex; align-items: center; - justify-content: space-between; + justify-content: flex-start; position: relative; z-index: 1; margin-top: 0.4em; - max-width: 400px; + overflow-x: auto; } &:hover > .article > .main > .footer > .footerButton { @@ -1203,10 +1205,6 @@ function emitUpdReaction(emoji: string, delta: number) { padding: 8px; opacity: 0.7; - &:not(:last-child) { - margin-right: 1.5em; - } - &:hover { color: var(--MI_THEME-fgHighlighted); } @@ -1290,25 +1288,7 @@ function emitUpdReaction(emoji: string, delta: number) { } } -@container (max-width: 400px) { - .root:not(.showActionsOnlyHover) { - .footerButton { - &:not(:last-child) { - margin-right: 0.2em; - } - } - } -} - @container (max-width: 350px) { - .root:not(.showActionsOnlyHover) { - .footerButton { - &:not(:last-child) { - margin-right: 0.1em; - } - } - } - .colorBar { top: 6px; left: 6px; diff --git a/packages/frontend/src/components/MkNoteDetailed.vue b/packages/frontend/src/components/MkNoteDetailed.vue index 52cc836926..7f38b9ec02 100644 --- a/packages/frontend/src/components/MkNoteDetailed.vue +++ b/packages/frontend/src/components/MkNoteDetailed.vue @@ -89,21 +89,21 @@ SPDX-License-Identifier: AGPL-3.0-only </p> <div v-show="mergedCW == null || showContent"> <span v-if="appearNote.isHidden" style="opacity: 0.5">({{ i18n.ts.private }})</span> - <MkA v-if="appearNote.replyId" :class="$style.noteReplyTarget" :to="`/notes/${appearNote.replyId}`"><i class="ph-arrow-bend-left-up ph-bold ph-lg"></i></MkA> - <Mfm - v-if="appearNote.text" - :parsedNodes="parsed" - :text="appearNote.text" - :author="appearNote.user" - :nyaize="'respect'" - :emojiUrls="appearNote.emojis" - :enableEmojiMenu="true" - :enableEmojiMenuReaction="true" - :isAnim="allowAnim" - :isBlock="true" - class="_selectable" - /> - <a v-if="appearNote.renote != null" :class="$style.rn">RN:</a> + <div> + <MkA v-if="appearNote.replyId" :class="$style.noteReplyTarget" :to="`/notes/${appearNote.replyId}`"><i class="ph-arrow-bend-left-up ph-bold ph-lg"></i></MkA> + <Mfm + v-if="appearNote.text" + :parsedNodes="parsed" + :text="appearNote.text" + :author="appearNote.user" + :nyaize="'respect'" + :emojiUrls="appearNote.emojis" + :enableEmojiMenu="true" + :enableEmojiMenuReaction="true" + :isAnim="allowAnim" + class="_selectable" + /> + </div> <SkNoteTranslation :note="note" :translation="translation" :translating="translating"></SkNoteTranslation> <MkButton v-if="!allowAnim && animated" :class="$style.playMFMButton" :small="true" @click="animatedMFM()" @click.stop><i class="ph-play ph-bold ph-lg "></i> {{ i18n.ts._animatedMFM.play }}</MkButton> <MkButton v-else-if="!prefer.s.animatedMfm && allowAnim && animated" :class="$style.playMFMButton" :small="true" @click="animatedMFM()" @click.stop><i class="ph-stop ph-bold ph-lg "></i> {{ i18n.ts._animatedMFM.stop }}</MkButton> @@ -112,13 +112,13 @@ SPDX-License-Identifier: AGPL-3.0-only </div> <MkPoll v-if="appearNote.poll" ref="pollViewer" :noteId="appearNote.id" :poll="appearNote.poll" :local="!appearNote.user.host" :class="$style.poll" :author="appearNote.user" :emojiUrls="appearNote.emojis"/> <div v-if="isEnabledUrlPreview"> - <MkUrlPreview v-for="url in urls" :key="url" :url="url" :compact="true" :detail="true" :showAsQuote="!appearNote.user.rejectQuotes" :skipNoteIds="selfNoteIds" style="margin-top: 6px;"/> + <SkUrlPreviewGroup :sourceNodes="nodes" :sourceNote="appearNote" :compact="true" :detail="true" :showAsQuote="!appearNote.user.rejectQuotes" :skipNoteIds="selfNoteIds" style="margin-top: 6px;" @click.stop/> </div> <div v-if="appearNote.renote" :class="$style.quote"><MkNoteSimple :note="appearNote.renote" :class="$style.quoteNote" :expandAllCws="props.expandAllCws"/></div> </div> <MkA v-if="appearNote.channel && !inChannel" :class="$style.channel" :to="`/channels/${appearNote.channel.id}`"><i class="ti ti-device-tv"></i> {{ appearNote.channel.name }}</MkA> </div> - <footer :class="$style.footer"> + <footer :class="$style.footer" class="_gaps _h_gaps" tabindex="0" role="group" :aria-label="i18n.ts.noteFooterLabel"> <div :class="$style.noteFooterInfo"> <div v-if="appearNote.updatedAt"> {{ i18n.ts.edited }}: <MkTime :time="appearNote.updatedAt" mode="detail"/> @@ -169,7 +169,7 @@ SPDX-License-Identifier: AGPL-3.0-only <button v-if="prefer.s.showClipButtonInNoteFooter" ref="clipButton" class="_button" :class="$style.noteFooterButton" @click.stop="clip()"> <i class="ti ti-paperclip"></i> </button> - <button v-if="prefer.s.showTranslationButtonInNoteFooter && $i?.policies.canUseTranslator && instance.translatorAvailable" ref="translationButton" class="_button" :class="$style.noteFooterButton" :disabled="translating || !!translation" @click.stop="translate()"> + <button v-if="prefer.s.showTranslationButtonInNoteFooter && policies.canUseTranslator && instance.translatorAvailable" ref="translationButton" class="_button" :class="$style.noteFooterButton" :disabled="translating || !!translation" @click.stop="translate()"> <i class="ti ti-language-hiragana"></i> </button> <button ref="menuButton" class="_button" :class="$style.noteFooterButton" @click.stop="showMenu()"> @@ -233,7 +233,7 @@ SPDX-License-Identifier: AGPL-3.0-only <script lang="ts" setup> import { computed, inject, onMounted, provide, ref, useTemplateRef, watch } from 'vue'; -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; import * as Misskey from 'misskey-js'; import { isLink } from '@@/js/is-link.js'; import * as config from '@@/js/config.js'; @@ -278,7 +278,7 @@ import MkPagination from '@/components/MkPagination.vue'; import MkReactionIcon from '@/components/MkReactionIcon.vue'; import MkButton from '@/components/MkButton.vue'; import { boostMenuItems, computeRenoteTooltip } from '@/utility/boost-quote.js'; -import { instance, isEnabledUrlPreview } from '@/instance.js'; +import { instance, isEnabledUrlPreview, policies } from '@/instance.js'; import { getAppearNote } from '@/utility/get-appear-note.js'; import { prefer } from '@/preferences.js'; import { getPluginHandlers } from '@/plugin.js'; @@ -286,7 +286,7 @@ import { DI } from '@/di.js'; import SkMutedNote from '@/components/SkMutedNote.vue'; import SkNoteTranslation from '@/components/SkNoteTranslation.vue'; import { getSelfNoteIds } from '@/utility/get-self-note-ids.js'; -import { extractPreviewUrls } from '@/utility/extract-preview-urls.js'; +import SkUrlPreviewGroup from '@/components/SkUrlPreviewGroup.vue'; const props = withDefaults(defineProps<{ note: Misskey.entities.Note; @@ -339,8 +339,7 @@ const isDeleted = ref(false); const renoted = ref(false); const translation = ref<Misskey.entities.NotesTranslateResponse | false | null>(null); const translating = ref(false); -const parsed = computed(() => appearNote.value.text ? mfm.parse(appearNote.value.text) : null); -const urls = computed(() => parsed.value ? extractPreviewUrls(props.note, parsed.value) : null); +const parsed = computed(() => appearNote.value.text ? mfm.parse(appearNote.value.text) : []); const selfNoteIds = computed(() => getSelfNoteIds(props.note)); const animated = computed(() => parsed.value ? checkAnimationFromMfm(parsed.value) : null); const allowAnim = ref(prefer.s.advancedMfm && prefer.s.animatedMfm); @@ -388,7 +387,7 @@ const keymap = { clip(); }, 't': () => { - if (prefer.s.showTranslationButtonInNoteFooter && $i?.policies.canUseTranslator && instance.translatorAvailable) { + if (prefer.s.showTranslationButtonInNoteFooter && policies.value.canUseTranslator && instance.translatorAvailable) { translate(); } }, @@ -415,6 +414,11 @@ provide(DI.mfmEmojiReactCallback, (reaction) => { const tab = ref(props.initialTab); const reactionTabType = ref<string | null>(null); +// Auto-select the first page of reactions +watch(appearNote, n => { + reactionTabType.value ??= Object.keys(n.reactions)[0] ?? null; +}, { immediate: true }); + const renotesPagination = computed<Paging>(() => ({ endpoint: 'notes/renotes', limit: 10, @@ -886,12 +890,10 @@ function animatedMFM() { } .footer { - position: relative; - z-index: 1; - margin-top: 0.4em; - width: max-content; - min-width: min-content; - max-width: fit-content; + position: relative; + z-index: 1; + margin-top: 0.4em; + overflow-x: auto; } .replyTo { @@ -1083,10 +1085,6 @@ function animatedMFM() { padding: 8px; opacity: 0.7; - &:not(:last-child) { - margin-right: 1.5em; - } - &:hover { color: var(--MI_THEME-fgHighlighted); } @@ -1169,14 +1167,6 @@ function animatedMFM() { } } -@container (max-width: 350px) { - .noteFooterButton { - &:not(:last-child) { - margin-right: 0.1em; - } - } -} - @container (max-width: 300px) { .root { font-size: 0.825em; @@ -1186,12 +1176,6 @@ function animatedMFM() { width: 50px; height: 50px; } - - .noteFooterButton { - &:not(:last-child) { - margin-right: 0.1em; - } - } } .muted { diff --git a/packages/frontend/src/components/MkNoteSub.vue b/packages/frontend/src/components/MkNoteSub.vue index 282854c6a8..58de5bd5a7 100644 --- a/packages/frontend/src/components/MkNoteSub.vue +++ b/packages/frontend/src/components/MkNoteSub.vue @@ -19,7 +19,7 @@ SPDX-License-Identifier: AGPL-3.0-only <MkSubNoteContent :class="$style.text" :note="note" :translating="translating" :translation="translation" :expandAllCws="props.expandAllCws"/> </div> </div> - <footer :class="$style.footer"> + <footer :class="$style.footer" class="_gaps _h_gaps" tabindex="0" role="group" :aria-label="i18n.ts.noteFooterLabel"> <MkReactionsViewer ref="reactionsViewer" :note="note"/> <button class="_button" :class="$style.noteFooterButton" @click="reply()"> <i class="ph-arrow-u-up-left ph-bold ph-lg"></i> @@ -62,7 +62,7 @@ SPDX-License-Identifier: AGPL-3.0-only <button v-if="prefer.s.showClipButtonInNoteFooter" ref="clipButton" :class="$style.noteFooterButton" class="_button" @click.stop="clip()"> <i class="ti ti-paperclip"></i> </button> - <button v-if="prefer.s.showTranslationButtonInNoteFooter && $i?.policies.canUseTranslator && instance.translatorAvailable" ref="translationButton" class="_button" :class="$style.noteFooterButton" :disabled="translating || !!translation" @click.stop="translate()"> + <button v-if="prefer.s.showTranslationButtonInNoteFooter && policies.canUseTranslator && instance.translatorAvailable" ref="translationButton" class="_button" :class="$style.noteFooterButton" :disabled="translating || !!translation" @click.stop="translate()"> <i class="ti ti-language-hiragana"></i> </button> <button ref="menuButton" class="_button" :class="$style.noteFooterButton" @click.stop="menu()"> @@ -113,7 +113,8 @@ import { boostMenuItems, computeRenoteTooltip } from '@/utility/boost-quote.js'; import { prefer } from '@/preferences.js'; import { useNoteCapture } from '@/use/use-note-capture.js'; import SkMutedNote from '@/components/SkMutedNote.vue'; -import { instance } from '@/instance'; +import { instance, policies } from '@/instance'; +import { getAppearNote } from '@/utility/get-appear-note'; const props = withDefaults(defineProps<{ note: Misskey.entities.Note; @@ -128,7 +129,9 @@ const props = withDefaults(defineProps<{ onDeleteCallback: undefined, }); -const canRenote = computed(() => ['public', 'home'].includes(props.note.visibility) || props.note.userId === $i?.id); +const appearNote = computed(() => getAppearNote(props.note)); + +const canRenote = computed(() => ['public', 'home'].includes(appearNote.value.visibility) || appearNote.value.userId === $i?.id); const el = shallowRef<HTMLElement>(); const translation = ref<Misskey.entities.NotesTranslateResponse | false | null>(null); @@ -144,19 +147,11 @@ const likeButton = shallowRef<HTMLElement>(); const renoteTooltip = computeRenoteTooltip(renoted); -const appearNote = computed(() => isRenote ? props.note.renote as Misskey.entities.Note : props.note); const defaultLike = computed(() => prefer.s.like ? prefer.s.like : null); const replies = ref<Misskey.entities.Note[]>([]); const mergedCW = computed(() => computeMergedCw(appearNote.value)); -const isRenote = ( - props.note.renote != null && - props.note.text == null && - props.note.fileIds && props.note.fileIds.length === 0 && - props.note.poll == null -); - const pleaseLoginContext = computed<OpenOnRemoteOptions>(() => ({ type: 'lookup', url: appearNote.value.url ?? appearNote.value.uri ?? `${config.url}/notes/${appearNote.value.id}`, @@ -206,8 +201,8 @@ async function reply(viaKeyboard = false): Promise<void> { pleaseLogin({ openOnRemote: pleaseLoginContext.value }); showMovedDialog(); await os.post({ - reply: props.note, - channel: props.note.channel ?? undefined, + reply: appearNote.value, + channel: appearNote.value.channel ?? undefined, animation: !viaKeyboard, }); focus(); @@ -217,9 +212,9 @@ function react(): void { pleaseLogin({ openOnRemote: pleaseLoginContext.value }); showMovedDialog(); sound.playMisskeySfx('reaction'); - if (props.note.reactionAcceptance === 'likeOnly') { + if (appearNote.value.reactionAcceptance === 'likeOnly') { misskeyApi('notes/like', { - noteId: props.note.id, + noteId: appearNote.value.id, override: defaultLike.value, }); const el = reactButton.value as HTMLElement | null | undefined; @@ -233,12 +228,12 @@ function react(): void { } } else { blur(); - reactionPicker.show(reactButton.value ?? null, props.note, reaction => { + reactionPicker.show(reactButton.value ?? null, appearNote.value, reaction => { misskeyApi('notes/reactions/create', { - noteId: props.note.id, + noteId: appearNote.value.id, reaction: reaction, }); - if (props.note.text && props.note.text.length > 100 && (Date.now() - new Date(props.note.createdAt).getTime() < 1000 * 3)) { + if (appearNote.value.text && appearNote.value.text.length > 100 && (Date.now() - new Date(appearNote.value.createdAt).getTime() < 1000 * 3)) { claimAchievement('reactWithoutRead'); } }, () => { @@ -252,7 +247,7 @@ function like(): void { showMovedDialog(); sound.playMisskeySfx('reaction'); misskeyApi('notes/like', { - noteId: props.note.id, + noteId: appearNote.value.id, override: defaultLike.value, }); const el = likeButton.value as HTMLElement | null | undefined; @@ -361,7 +356,7 @@ function quote() { }).then((cancelled) => { if (cancelled) return; misskeyApi('notes/renotes', { - noteId: props.note.id, + noteId: appearNote.value.id, userId: $i?.id, limit: 1, quote: true, @@ -383,12 +378,12 @@ function quote() { } function menu(): void { - const { menu, cleanup } = getNoteMenu({ note: props.note, translating, translation, isDeleted }); + const { menu, cleanup } = getNoteMenu({ note: appearNote.value, translating, translation, isDeleted }); os.popupMenu(menu, menuButton.value).then(focus).finally(cleanup); } async function clip(): Promise<void> { - os.popupMenu(await getNoteClipMenu({ note: props.note, isDeleted, currentClip: currentClip?.value }), clipButton.value).then(focus); + os.popupMenu(await getNoteClipMenu({ note: appearNote.value, isDeleted, currentClip: currentClip?.value }), clipButton.value).then(focus); } async function translate() { @@ -397,7 +392,7 @@ async function translate() { if (props.detail) { misskeyApi('notes/children', { - noteId: props.note.id, + noteId: appearNote.value.id, limit: prefer.s.numberOfReplies, showQuotes: false, }).then(res => { @@ -419,12 +414,10 @@ if (props.detail) { } .footer { - position: relative; - z-index: 1; - margin-top: 0.4em; - width: max-content; - min-width: min-content; - max-width: fit-content; + position: relative; + z-index: 1; + margin-top: 0.4em; + overflow-x: auto; } .main { @@ -469,23 +462,11 @@ if (props.detail) { padding-top: 10px; opacity: 0.7; - &:not(:last-child) { - margin-right: 1.5em; - } - &:hover { color: var(--MI_THEME-fgHighlighted); } } -@container (max-width: 400px) { - .noteFooterButton { - &:not(:last-child) { - margin-right: 0.7em; - } - } -} - .noteFooterButtonCount { display: inline; margin: 0 0 0 8px; diff --git a/packages/frontend/src/components/MkNotes.vue b/packages/frontend/src/components/MkNotes.vue index efb481d01d..04dff2eb36 100644 --- a/packages/frontend/src/components/MkNotes.vue +++ b/packages/frontend/src/components/MkNotes.vue @@ -15,13 +15,8 @@ SPDX-License-Identifier: AGPL-3.0-only <template #default="{ items: notes }"> <div :class="[$style.root, { [$style.noGap]: noGap, '_gaps': !noGap, [$style.reverse]: pagination.reversed }]"> <template v-for="(note, i) in notes" :key="note.id"> - <div v-if="note._shouldInsertAd_" :class="[$style.noteWithAd, { '_gaps': !noGap }]" :data-scroll-anchor="note.id"> - <DynamicNote :class="$style.note" :note="note as Misskey.entities.Note" :withHardMute="true"/> - <div :class="$style.ad"> - <MkAd :preferForms="['horizontal', 'horizontal-big']"/> - </div> - </div> - <DynamicNote v-else :class="$style.note" :note="note as Misskey.entities.Note" :withHardMute="true" :data-scroll-anchor="note.id"/> + <DynamicNote :class="$style.note" :note="note as Misskey.entities.Note" :withHardMute="true" :data-scroll-anchor="note.id"/> + <MkAd v-if="note._shouldInsertAd_" :preferForms="['horizontal', 'horizontal-big']" :class="$style.ad"/> </template> </div> </template> @@ -62,7 +57,7 @@ defineExpose({ &.noGap { background: color-mix(in srgb, var(--MI_THEME-panel) 65%, transparent); - .note { + .note:not(:empty) { border-bottom: solid 0.5px var(--MI_THEME-divider); } diff --git a/packages/frontend/src/components/MkNotifications.vue b/packages/frontend/src/components/MkNotifications.vue index 54edf771ed..46e98462dc 100644 --- a/packages/frontend/src/components/MkNotifications.vue +++ b/packages/frontend/src/components/MkNotifications.vue @@ -14,8 +14,8 @@ SPDX-License-Identifier: AGPL-3.0-only </template> <template #default="{ items: notifications }"> - <component - :is="prefer.s.animation ? TransitionGroup : 'div'" :class="[$style.notifications]" + <SkTransitionGroup + :class="[$style.notifications]" :enterActiveClass="$style.transition_x_enterActive" :leaveActiveClass="$style.transition_x_leaveActive" :enterFromClass="$style.transition_x_enterFrom" @@ -27,7 +27,7 @@ SPDX-License-Identifier: AGPL-3.0-only <DynamicNote v-if="['reply', 'quote', 'mention'].includes(notification.type)" :class="$style.item" :note="notification.note" :withHardMute="true" :data-scroll-anchor="notification.id"/> <XNotification v-else :class="$style.item" :notification="notification" :withTime="true" :full="true" :data-scroll-anchor="notification.id"/> </div> - </component> + </SkTransitionGroup> </template> </MkPagination> </MkPullToRefresh> @@ -45,6 +45,7 @@ import { i18n } from '@/i18n.js'; import { infoImageUrl } from '@/instance.js'; import MkPullToRefresh from '@/components/MkPullToRefresh.vue'; import { prefer } from '@/preferences.js'; +import SkTransitionGroup from '@/components/SkTransitionGroup.vue'; const props = defineProps<{ excludeTypes?: typeof notificationTypes[number][]; diff --git a/packages/frontend/src/components/MkPageWindow.vue b/packages/frontend/src/components/MkPageWindow.vue index a9e4704b24..44112775dc 100644 --- a/packages/frontend/src/components/MkPageWindow.vue +++ b/packages/frontend/src/components/MkPageWindow.vue @@ -106,7 +106,7 @@ windowRouter.addListener('replace', ctx => { }); windowRouter.addListener('change', ctx => { - if (_DEV_) console.log('windowRouter: change', ctx.fullPath); + if (_DEV_) console.debug('windowRouter: change', ctx.fullPath); searchMarkerId.value = getSearchMarker(ctx.fullPath); }); diff --git a/packages/frontend/src/components/MkPagination.vue b/packages/frontend/src/components/MkPagination.vue index 79a268e8f6..b850c17be1 100644 --- a/packages/frontend/src/components/MkPagination.vue +++ b/packages/frontend/src/components/MkPagination.vue @@ -386,7 +386,7 @@ function prepend(item: MisskeyEntity): void { return; } - if (_DEV_) console.log(isHead(), isPausingUpdate); + if (_DEV_) console.debug(isHead(), isPausingUpdate); if (isHead() && !isPausingUpdate) unshiftItems([item]); else prependQueue(item); diff --git a/packages/frontend/src/components/MkPostForm.vue b/packages/frontend/src/components/MkPostForm.vue index 000ccf50bf..a650365a28 100644 --- a/packages/frontend/src/components/MkPostForm.vue +++ b/packages/frontend/src/components/MkPostForm.vue @@ -107,7 +107,7 @@ SPDX-License-Identifier: AGPL-3.0-only <script lang="ts" setup> import { inject, watch, nextTick, onMounted, defineAsyncComponent, provide, shallowRef, ref, computed, useTemplateRef, toRaw } from 'vue'; -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; import * as Misskey from 'misskey-js'; import insertTextAtCursor from 'insert-text-at-cursor'; import { toASCII } from 'punycode.js'; @@ -373,7 +373,9 @@ if (props.specified) { // keep cw when reply if (prefer.s.keepCw && props.reply && props.reply.cw) { useCw.value = true; - cw.value = props.reply.cw; + cw.value = (prefer.s.keepCw === 'prepend-re' && !props.reply.cw.toLowerCase().startsWith('re:')) + ? `RE: ${props.reply.cw}` + : props.reply.cw; } // apply default CW @@ -557,6 +559,7 @@ async function toggleLocalOnly() { if (confirm.result === 'no') return; if (confirm.result === 'neverShow') { + prefer.commit('neverShowLocalOnlyInfo', 'true'); miLocalStorage.setItem('neverShowLocalOnlyInfo', 'true'); } } @@ -1356,7 +1359,7 @@ defineExpose({ } &.danger { - color: #ff2a2a; + color: var(--MI_THEME-warn); } } diff --git a/packages/frontend/src/components/MkReactionsViewer.vue b/packages/frontend/src/components/MkReactionsViewer.vue index 945640ab41..88ac8c87c1 100644 --- a/packages/frontend/src/components/MkReactionsViewer.vue +++ b/packages/frontend/src/components/MkReactionsViewer.vue @@ -4,8 +4,7 @@ SPDX-License-Identifier: AGPL-3.0-only --> <template> -<component - :is="prefer.s.animation ? TransitionGroup : 'div'" +<SkTransitionGroup :enterActiveClass="$style.transition_x_enterActive" :leaveActiveClass="$style.transition_x_leaveActive" :enterFromClass="$style.transition_x_enterFrom" @@ -14,8 +13,10 @@ SPDX-License-Identifier: AGPL-3.0-only tag="div" :class="$style.root" > <XReaction v-for="[reaction, count] in reactions" :key="reaction" :reaction="reaction" :count="count" :isInitial="initialReactions.has(reaction)" :note="note" @reactionToggled="onMockToggleReaction"/> - <slot v-if="hasMoreReactions" :key="'$more'" name="more"/> -</component> + <div v-if="hasMoreReactions" :key="'$more'" :class="$style.moreReactions"> + <slot name="more"/> + </div> +</SkTransitionGroup> </template> <script lang="ts" setup> @@ -25,6 +26,7 @@ import { TransitionGroup } from 'vue'; import XReaction from '@/components/MkReactionsViewer.reaction.vue'; import { prefer } from '@/preferences.js'; import { DI } from '@/di.js'; +import SkTransitionGroup from '@/components/SkTransitionGroup.vue'; const props = withDefaults(defineProps<{ note: Misskey.entities.Note; @@ -102,7 +104,7 @@ watch([() => props.note.reactions, () => props.maxNumber], ([newSource, maxNumbe position: absolute; } -.root { +.root, .moreReactions { display: flex; flex-wrap: wrap; align-items: center; diff --git a/packages/frontend/src/components/MkSelect.vue b/packages/frontend/src/components/MkSelect.vue index cf4e4eda74..511a45c165 100644 --- a/packages/frontend/src/components/MkSelect.vue +++ b/packages/frontend/src/components/MkSelect.vue @@ -39,32 +39,34 @@ SPDX-License-Identifier: AGPL-3.0-only </div> </template> -<script lang="ts" setup> -import { onMounted, nextTick, ref, watch, computed, toRefs, useSlots } from 'vue'; -import { useInterval } from '@@/js/use-interval.js'; -import type { VNode, VNodeChild } from 'vue'; -import type { MenuItem } from '@/types/menu.js'; -import * as os from '@/os.js'; - -type ItemOption = { +<script lang="ts"> +type ItemOption<T extends string | number | null | boolean = string | number | null> = { type?: 'option'; - value: string | number | null; + value: T; label: string; }; -type ItemGroup = { +type ItemGroup<T extends string | number | null | boolean = string | number | null> = { type: 'group'; label: string; - items: ItemOption[]; + items: ItemOption<T>[]; }; -export type MkSelectItem = ItemOption | ItemGroup; +export type MkSelectItem<T extends string | number | null | boolean = string | number | null> = ItemOption<T> | ItemGroup<T>; +</script> + +<script lang="ts" setup generic="T extends string | number | null | boolean = string | number | null"> +import { onMounted, nextTick, ref, watch, computed, toRefs, useSlots } from 'vue'; +import { useInterval } from '@@/js/use-interval.js'; +import type { VNode, VNodeChild } from 'vue'; +import type { MenuItem } from '@/types/menu.js'; +import * as os from '@/os.js'; // TODO: itemsをslot内のoptionで指定する用法は廃止する(props.itemsを必須化する) // see: https://github.com/misskey-dev/misskey/issues/15558 const props = defineProps<{ - modelValue: string | number | null; + modelValue: T; required?: boolean; readonly?: boolean; disabled?: boolean; @@ -73,11 +75,11 @@ const props = defineProps<{ inline?: boolean; small?: boolean; large?: boolean; - items?: MkSelectItem[]; + items?: MkSelectItem<T>[]; }>(); const emit = defineEmits<{ - (ev: 'update:modelValue', value: string | number | null): void; + (ev: 'update:modelValue', value: T): void; }>(); const slots = useSlots(); diff --git a/packages/frontend/src/components/MkSignupDialog.form.vue b/packages/frontend/src/components/MkSignupDialog.form.vue index 365b23f4ce..003c68309d 100644 --- a/packages/frontend/src/components/MkSignupDialog.form.vue +++ b/packages/frontend/src/components/MkSignupDialog.form.vue @@ -307,7 +307,7 @@ async function onSubmit(): Promise<void> { emit('approvalPending'); } else { const resJson = (await res.json()) as Misskey.entities.SignupResponse; - if (_DEV_) console.log(resJson); + if (_DEV_) console.debug(resJson); emit('signup', resJson); diff --git a/packages/frontend/src/components/MkSubNoteContent.vue b/packages/frontend/src/components/MkSubNoteContent.vue index 0780f6c910..60d303f937 100644 --- a/packages/frontend/src/components/MkSubNoteContent.vue +++ b/packages/frontend/src/components/MkSubNoteContent.vue @@ -8,8 +8,10 @@ SPDX-License-Identifier: AGPL-3.0-only <div :class="{ [$style.clickToOpen]: prefer.s.clickToOpen }" @click.stop="prefer.s.clickToOpen ? noteclick(note.id) : undefined"> <span v-if="note.isHidden" style="opacity: 0.5">({{ i18n.ts.private }})</span> <span v-if="note.deletedAt" style="opacity: 0.5">({{ i18n.ts.deletedNote }})</span> - <MkA v-if="note.replyId" :class="$style.reply" :to="`/notes/${note.replyId}`" @click.stop><i class="ph-arrow-bend-left-up ph-bold ph-lg"></i></MkA> - <Mfm v-if="note.text" :text="note.text" :isBlock="true" :author="note.user" :nyaize="'respect'" :isAnim="allowAnim" :emojiUrls="note.emojis"/> + <div> + <MkA v-if="note.replyId" :class="$style.reply" :to="`/notes/${note.replyId}`" @click.stop><i class="ph-arrow-bend-left-up ph-bold ph-lg"></i></MkA> + <Mfm v-if="note.text" :text="note.text" :author="note.user" :nyaize="'respect'" :isAnim="allowAnim" :emojiUrls="note.emojis"/> + </div> <MkButton v-if="!allowAnim && animated && !hideFiles" :class="$style.playMFMButton" :small="true" @click="animatedMFM()" @click.stop><i class="ph-play ph-bold ph-lg "></i> {{ i18n.ts._animatedMFM.play }}</MkButton> <MkButton v-else-if="!prefer.s.animatedMfm && allowAnim && animated && !hideFiles" :class="$style.playMFMButton" :small="true" @click="animatedMFM()" @click.stop><i class="ph-stop ph-bold ph-lg "></i> {{ i18n.ts._animatedMFM.stop }}</MkButton> <SkNoteTranslation :note="note" :translation="translation" :translating="translating"></SkNoteTranslation> @@ -35,7 +37,7 @@ SPDX-License-Identifier: AGPL-3.0-only <script lang="ts" setup> import { ref, computed, watch } from 'vue'; import * as Misskey from 'misskey-js'; -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; import { shouldCollapsed } from '@@/js/collapsed.js'; import MkMediaList from '@/components/MkMediaList.vue'; import MkPoll from '@/components/MkPoll.vue'; diff --git a/packages/frontend/src/components/MkTimeline.vue b/packages/frontend/src/components/MkTimeline.vue index 48e8c7f377..61b34b561d 100644 --- a/packages/frontend/src/components/MkTimeline.vue +++ b/packages/frontend/src/components/MkTimeline.vue @@ -14,8 +14,7 @@ SPDX-License-Identifier: AGPL-3.0-only </template> <template #default="{ items: notes }"> - <component - :is="prefer.s.animation ? TransitionGroup : 'div'" + <SkTransitionGroup :class="[$style.root, { [$style.noGap]: noGap, '_gaps': !noGap, [$style.reverse]: paginationQuery.reversed }]" :enterActiveClass="$style.transition_x_enterActive" :leaveActiveClass="$style.transition_x_leaveActive" @@ -24,16 +23,11 @@ SPDX-License-Identifier: AGPL-3.0-only :moveClass=" $style.transition_x_move" tag="div" > - <div v-for="(note, i) in notes" :key="note.id"> - <div v-if="note._shouldInsertAd_" :class="[$style.noteWithAd, { '_gaps': !noGap }]" :data-scroll-anchor="note.id"> - <DynamicNote :class="$style.note" :note="note" :withHardMute="true"/> - <div :class="$style.ad"> - <MkAd :preferForms="['horizontal', 'horizontal-big']"/> - </div> - </div> - <DynamicNote v-else :class="$style.note" :note="note" :withHardMute="true" :data-scroll-anchor="note.id"/> + <div v-for="(note, i) in notes" :key="note.id" :class="{ '_gaps': !noGap }"> + <DynamicNote :class="$style.note" :note="note as Misskey.entities.Note" :withHardMute="true" :data-scroll-anchor="note.id"/> + <MkAd v-if="note._shouldInsertAd_" :preferForms="['horizontal', 'horizontal-big']" :class="$style.ad"/> </div> - </component> + </SkTransitionGroup> </template> </MkPagination> </MkPullToRefresh> @@ -54,6 +48,7 @@ import DynamicNote from '@/components/DynamicNote.vue'; import MkPagination from '@/components/MkPagination.vue'; import { i18n } from '@/i18n.js'; import { infoImageUrl } from '@/instance.js'; +import SkTransitionGroup from '@/components/SkTransitionGroup.vue'; const props = withDefaults(defineProps<{ src: BasicTimelineType | 'mentions' | 'directs' | 'list' | 'antenna' | 'channel' | 'role'; @@ -358,7 +353,7 @@ defineExpose({ &.noGap { background: var(--MI_THEME-panel); - .note { + .note:not(:empty) { border-bottom: solid 0.5px var(--MI_THEME-divider); } diff --git a/packages/frontend/src/components/MkUrlPreview.vue b/packages/frontend/src/components/MkUrlPreview.vue index a14c2ecef9..5d0e6e3df7 100644 --- a/packages/frontend/src/components/MkUrlPreview.vue +++ b/packages/frontend/src/components/MkUrlPreview.vue @@ -65,6 +65,17 @@ SPDX-License-Identifier: AGPL-3.0-only </footer> </article> </component> + + <I18n v-if="attributionUser" :src="i18n.ts.writtenBy" :class="$style.linkAttribution" tag="p"> + <template #user> + <MkA v-user-preview="attributionUser.id" :to="userPage(attributionUser)"> + <MkAvatar :class="$style.linkAttributionIcon" :user="attributionUser"/> + <MkUserName :user="attributionUser" style="color: var(--MI_THEME-accent)"/> + </MkA> + </template> + </I18n> + <p v-else-if="linkAttribution" :class="$style.linkAttribution"><MkEllipsis/></p> + <template v-if="showActions"> <div v-if="tweetId" :class="$style.action"> <MkButton :small="true" inline @click="tweetExpanded = true"> @@ -88,13 +99,24 @@ SPDX-License-Identifier: AGPL-3.0-only </div> </template> +<script lang="ts"> +// eslint-disable-next-line import/order +import type { summaly } from '@misskey-dev/summaly'; + +export type SummalyResult = Awaited<ReturnType<typeof summaly>> & { + haveNoteLocally?: boolean, + linkAttribution?: { + userId: string, + } +}; +</script> + <script lang="ts" setup> import { defineAsyncComponent, onDeactivated, onUnmounted, ref } from 'vue'; import { url as local } from '@@/js/config.js'; import { versatileLang } from '@@/js/intl-const.js'; import * as Misskey from 'misskey-js'; import { maybeMakeRelative } from '@@/js/url.js'; -import type { summaly } from '@misskey-dev/summaly'; import { i18n } from '@/i18n.js'; import * as os from '@/os.js'; import { deviceKind } from '@/utility/device-kind.js'; @@ -106,8 +128,7 @@ import { misskeyApi } from '@/utility/misskey-api.js'; import { warningExternalWebsite } from '@/utility/warning-external-website.js'; import DynamicNoteSimple from '@/components/DynamicNoteSimple.vue'; import { $i } from '@/i'; - -type SummalyResult = Awaited<ReturnType<typeof summaly>>; +import { userPage } from '@/filters/user.js'; const props = withDefaults(defineProps<{ url: string; @@ -116,12 +137,18 @@ const props = withDefaults(defineProps<{ showAsQuote?: boolean; showActions?: boolean; skipNoteIds?: (string | undefined)[]; + previewHint?: SummalyResult; + noteHint?: Misskey.entities.Note | null; + attributionHint?: Misskey.entities.User | null; }>(), { detail: false, compact: false, showAsQuote: false, showActions: true, skipNoteIds: undefined, + previewHint: undefined, + noteHint: undefined, + attributionHint: undefined, }); const MOBILE_THRESHOLD = 500; @@ -146,6 +173,10 @@ const player = ref<SummalyResult['player']>({ height: null, allow: [], }); +const linkAttribution = ref<{ + userId: string, +} | null>(null); +const attributionUser = ref<Misskey.entities.User | null>(null); const playerEnabled = ref(false); const tweetId = ref<string | null>(null); const tweetExpanded = ref(props.detail); @@ -154,12 +185,35 @@ const tweetHeight = ref(150); const unknownUrl = ref(false); const theNote = ref<Misskey.entities.Note | null>(null); const fetchingTheNote = ref(false); +const fetchingAttribution = ref<Promise<void> | null>(null); onDeactivated(() => { playerEnabled.value = false; }); -async function fetchNote() { +async function fetchAttribution(initial: boolean): Promise<void> { + if (!linkAttribution.value) return; + if (attributionUser.value) return; + if (fetchingAttribution.value) return fetchingAttribution.value; + + return fetchingAttribution.value ??= (async (userId: string): Promise<void> => { + try { + if (initial && props.attributionHint !== undefined) { + attributionUser.value = props.attributionHint; + } else { + attributionUser.value = await misskeyApi('users/show', { userId }); + } + } catch { + // makes the loading ellipsis vanish. + linkAttribution.value = null; + } finally { + // Reset promise to mark as done + fetchingAttribution.value = null; + } + })(linkAttribution.value.userId); +} + +async function fetchNote(initial: boolean) { if (!props.showAsQuote) return; if (!activityPub.value) return; if (theNote.value) return; @@ -167,8 +221,15 @@ async function fetchNote() { fetchingTheNote.value = true; try { - const response = await misskeyApi('ap/show', { uri: activityPub.value }); + const response = (initial && props.noteHint !== undefined) + ? { type: 'Note', object: props.noteHint } + : await misskeyApi('ap/show', { uri: activityPub.value }); if (response.type !== 'Note') return; + if (!response.object) { + activityPub.value = null; + theNote.value = null; + return; + } const theNoteId = response['object'].id; if (theNoteId && props.skipNoteIds && props.skipNoteIds.includes(theNoteId)) { hidePreview.value = true; @@ -194,13 +255,16 @@ if (requestUrl.hostname === 'twitter.com' || requestUrl.hostname === 'mobile.twi if (m) tweetId.value = m[1]; } +// This is now handled on the backend +/* if (requestUrl.hostname === 'music.youtube.com' && requestUrl.pathname.match('^/(?:watch|channel)')) { requestUrl.hostname = 'www.youtube.com'; } requestUrl.hash = ''; +*/ -function refresh(withFetch = false) { +function refresh(withFetch = false, initial = false) { const params = new URLSearchParams({ url: requestUrl.href, lang: versatileLang, @@ -210,18 +274,21 @@ function refresh(withFetch = false) { } const headers = $i ? { Authorization: `Bearer ${$i.token}` } : undefined; - return fetching.value ??= window.fetch(`/url?${params.toString()}`, { headers }) - .then(res => { - if (!res.ok) { - if (_DEV_) { - console.warn(`[HTTP${res.status}] Failed to fetch url preview`); + const fetchPromise: Promise<SummalyResult | null> = (initial && props.previewHint) + ? Promise.resolve(props.previewHint) + : window.fetch(`/url?${params.toString()}`, { headers }) + .then(res => { + if (!res.ok) { + if (_DEV_) { + console.warn(`[HTTP${res.status}] Failed to fetch url preview`); + } + return null; } - return null; - } - return res.json(); - }) - .then(async (info: SummalyResult & { haveNoteLocally?: boolean } | null) => { + return res.json(); + }); + return fetching.value ??= fetchPromise + .then(async (info: SummalyResult | null) => { unknownUrl.value = info == null; title.value = info?.title ?? null; description.value = info?.description ?? null; @@ -236,11 +303,16 @@ function refresh(withFetch = false) { }; sensitive.value = info?.sensitive ?? false; activityPub.value = info?.activityPub ?? null; + linkAttribution.value = info?.linkAttribution ?? null; + // These will be populated by the fetch* functions + attributionUser.value = null; theNote.value = null; - if (info?.haveNoteLocally) { - await fetchNote(); - } + + await Promise.all([ + fetchAttribution(initial), + fetchNote(initial), + ]); }) .finally(() => { fetching.value = null; @@ -273,7 +345,7 @@ onUnmounted(() => { }); // Load initial data -refresh(); +refresh(false, true); </script> <style lang="scss" module> @@ -357,7 +429,7 @@ refresh(); .body { position: relative; box-sizing: border-box; - padding: 16px; + padding: 16px !important; // Unfortunately needed to win a specificity race with MkNoteSimple / SkNoteSimple } .header { @@ -395,6 +467,28 @@ refresh(); vertical-align: top; } +.linkAttributionIcon { + display: inline-block; + width: 16px; + height: 16px; + margin-left: 0.25em; + margin-right: 0.25em; + vertical-align: middle; + border-radius: 50%; + * { + border-radius: 4px; + } +} + +.linkAttribution { + width: 100%; + font-size: 0.8em; + display: inline-block; + margin: auto; + padding-top: 0.5em; + text-align: right; +} + .action { display: flex; gap: 6px; diff --git a/packages/frontend/src/components/SkApprovalUser.vue b/packages/frontend/src/components/SkApprovalUser.vue index 310d044387..3f5bd345f2 100644 --- a/packages/frontend/src/components/SkApprovalUser.vue +++ b/packages/frontend/src/components/SkApprovalUser.vue @@ -19,7 +19,7 @@ SPDX-License-Identifier: AGPL-3.0-only <div>{{ email }}</div> </div> <div> - <div :class="$style.label">Reason</div> + <div :class="$style.label">{{ i18n.ts.signupReason }}</div> <div>{{ reason }}</div> </div> </div> diff --git a/packages/frontend/src/components/SkBadgeStrip.vue b/packages/frontend/src/components/SkBadgeStrip.vue new file mode 100644 index 0000000000..6611d35b07 --- /dev/null +++ b/packages/frontend/src/components/SkBadgeStrip.vue @@ -0,0 +1,84 @@ +<!-- +SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors +SPDX-License-Identifier: AGPL-3.0-only +--> + +<template> +<div :class="$style.badges"> + <div + v-for="badge of badges" + :key="badge.key" + :class="[$style.badge, semanticClass(badge)]" + > + {{ badge.label }} + </div> +</div> +</template> + +<script lang="ts"> +export interface Badge { + /** + * ID/key of this badge, must be unique within the strip. + */ + key: string; + + /** + * Label text to display. + * Should already be translated. + */ + label: string; + + /** + * Semantic style of the badge. + * Defaults to "neutral" if unset. + */ + style?: 'success' | 'neutral' | 'warning' | 'error'; +} +</script> + +<script setup lang="ts"> +import { useCssModule } from 'vue'; + +const $style = useCssModule(); + +defineProps<{ + badges: Badge[], +}>(); + +function semanticClass(badge: Badge): string { + const style = badge.style ?? 'neutral'; + return $style[`semantic_${style}`]; +} +</script> + +<style module lang="scss"> +.badges { + display: flex; + flex-direction: row; + flex-wrap: wrap; + gap: var(--MI-margin); +} + +.badge { + display: inline-block; + border: solid 1px; + border-radius: var(--MI-radius-sm); + padding: 2px 6px; + font-size: 85%; +} + +.semantic_error { + color: var(--MI_THEME-error); + border-color: var(--MI_THEME-error); +} + +.semantic_warning { + color: var(--MI_THEME-warn); + border-color: var(--MI_THEME-warn); +} + +.semantic_success { + color: var(--MI_THEME-success); + border-color: var(--MI_THEME-success); +} +</style> diff --git a/packages/frontend/src/components/SkDateSeparatedList.vue b/packages/frontend/src/components/SkDateSeparatedList.vue new file mode 100644 index 0000000000..239d0c1939 --- /dev/null +++ b/packages/frontend/src/components/SkDateSeparatedList.vue @@ -0,0 +1,55 @@ +<!-- +SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors +SPDX-License-Identifier: AGPL-3.0-only +--> + +<template> +<div class="_gaps"> + <template v-for="(item, index) in timeline" :key="item.id"> + <slot v-if="item.type === 'item'" :id="item.id" :index="index" :item="item.data"></slot> + <slot v-else-if="item.type === 'date'" :id="item.id" :index="index" :prev="item.prev" :prevText="item.prevText" :next="item.next" :nextText="item.nextText" name="date"> + <div :class="$style.dateDivider"> + <span><i class="ti ti-chevron-up"></i> {{ item.nextText }}</span> + <span :class="$style.dateSeparator"></span> + <span>{{ item.prevText }} <i class="ti ti-chevron-down"></i></span> + </div> + </slot> + </template> +</div> +</template> + +<script setup lang="ts" generic="T extends { id: string; createdAt: string; }"> +import { computed } from 'vue'; +import { makeDateSeparatedTimelineComputedRef } from '@/utility/timeline-date-separate'; + +const props = defineProps<{ + items: T[], +}>(); + +const itemsRef = computed(() => props.items); +const timeline = makeDateSeparatedTimelineComputedRef(itemsRef); +</script> + +<style module lang="scss"> +// From room.vue +.dateDivider { + display: flex; + font-size: 85%; + align-items: center; + justify-content: center; + gap: 0.5em; + opacity: 0.75; + border: solid 0.5px var(--MI_THEME-divider); + border-radius: 999px; + width: fit-content; + padding: 0.5em 1em; + margin: 0 auto; +} + +// From room.vue +.dateSeparator { + height: 1em; + width: 1px; + background: var(--MI_THEME-divider); +} +</style> diff --git a/packages/frontend/src/components/SkFollowingRecentNotes.vue b/packages/frontend/src/components/SkFollowingRecentNotes.vue index 3eb2ac8572..37f2f8833a 100644 --- a/packages/frontend/src/components/SkFollowingRecentNotes.vue +++ b/packages/frontend/src/components/SkFollowingRecentNotes.vue @@ -14,6 +14,7 @@ SPDX-License-Identifier: AGPL-3.0-only </template> <template #default="{ items: notes }"> + <!-- TODO replace with SkDateSeparatedList when merged --> <MkDateSeparatedList v-slot="{ item: note }" :items="notes" :class="$style.panel" :noGap="true"> <SkFollowingFeedEntry :note="note" :class="props.selectedUserId == note.userId && $style.selected" @select="u => selectUser(u.id)"/> </MkDateSeparatedList> diff --git a/packages/frontend/src/components/SkMfmWindow.vue b/packages/frontend/src/components/SkMfmWindow.vue index 14d309b7ba..c544bc528c 100644 --- a/packages/frontend/src/components/SkMfmWindow.vue +++ b/packages/frontend/src/components/SkMfmWindow.vue @@ -100,6 +100,16 @@ SPDX-License-Identifier: AGPL-3.0-only </div> </div> <div class="section _block"> + <div class="title">{{ i18n.ts._mfm.unixtime }}</div> + <div class="content"> + <p>{{ i18n.ts._mfm.unixtimeDescription }}</p> + <div class="preview"> + <Mfm :text="preview_unixtime"/> + <MkTextarea v-model="preview_unixtime"><template #label>MFM</template></MkTextarea> + </div> + </div> + </div> + <div class="section _block"> <div class="title">{{ i18n.ts._mfm.inlineCode }}</div> <div class="content"> <p>{{ i18n.ts._mfm.inlineCodeDescription }}</p> @@ -429,6 +439,9 @@ const preview_small = ref( const preview_center = ref( `<center>${i18n.ts._mfm.dummy}</center>`, ); +const preview_unixtime = ref( + `$[unixtime ${Math.floor(Date.now() / 1000)}]`, +); const preview_inlineCode = ref('`<: "Hello, world!"`'); const preview_blockCode = ref( '```ai\n~ (#i, 100) {\n\t<: ? ((i % 15) = 0) "FizzBuzz"\n\t\t.? ((i % 3) = 0) "Fizz"\n\t\t.? ((i % 5) = 0) "Buzz"\n\t\t. i\n}\n```', diff --git a/packages/frontend/src/components/SkMutedNote.vue b/packages/frontend/src/components/SkMutedNote.vue index 3c072fab3f..c9b3d768de 100644 --- a/packages/frontend/src/components/SkMutedNote.vue +++ b/packages/frontend/src/components/SkMutedNote.vue @@ -9,7 +9,7 @@ SPDX-License-Identifier: AGPL-3.0-only <MkUserName :user="note.user"/> </template> </I18n> -<I18n v-else-if="prefer.s.showSoftWordMutedWord" :src="i18n.ts.userSaysSomething" tag="small"> +<I18n v-else-if="!prefer.s.showSoftWordMutedWord" :src="i18n.ts.userSaysSomething" tag="small"> <template #name> <MkUserName :user="note.user"/> </template> diff --git a/packages/frontend/src/components/SkNote.vue b/packages/frontend/src/components/SkNote.vue index 5184cbd801..4d6d080ddf 100644 --- a/packages/frontend/src/components/SkNote.vue +++ b/packages/frontend/src/components/SkNote.vue @@ -97,7 +97,7 @@ SPDX-License-Identifier: AGPL-3.0-only </div> <MkPoll v-if="appearNote.poll" :noteId="appearNote.id" :poll="appearNote.poll" :local="!appearNote.user.host" :author="appearNote.user" :emojiUrls="appearNote.emojis" :class="$style.poll" @click.stop/> <div v-if="isEnabledUrlPreview"> - <MkUrlPreview v-for="url in urls" :key="url" :url="url" :compact="true" :detail="false" :showAsQuote="!appearNote.user.rejectQuotes" :skipNoteIds="selfNoteIds" :class="$style.urlPreview" @click.stop/> + <SkUrlPreviewGroup :sourceUrls="urls" :sourceNote="appearNote" :compact="true" :detail="false" :showAsQuote="!appearNote.user.rejectQuotes" :skipNoteIds="selfNoteIds" :class="$style.urlPreview" @click.stop/> </div> <div v-if="appearNote.renote" :class="$style.quote"><SkNoteSimple :note="appearNote.renote" :class="$style.quoteNote"/></div> <button v-if="isLong && collapsed" :class="$style.collapsed" class="_button" @click.stop @click="collapsed = false"> @@ -114,7 +114,7 @@ SPDX-License-Identifier: AGPL-3.0-only <MkA :to="`/notes/${appearNote.id}/reactions`" :class="[$style.reactionOmitted]">{{ i18n.ts.more }}</MkA> </template> </MkReactionsViewer> - <footer :class="$style.footer"> + <footer :class="$style.footer" class="_gaps _h_gaps" tabindex="0" role="group" :aria-label="i18n.ts.noteFooterLabel"> <button :class="$style.footerButton" class="_button" @click.stop @click="reply()"> <i class="ti ti-arrow-back-up"></i> <p v-if="appearNote.repliesCount > 0" :class="$style.footerButtonCount">{{ number(appearNote.repliesCount) }}</p> @@ -158,7 +158,7 @@ SPDX-License-Identifier: AGPL-3.0-only <button v-if="prefer.s.showClipButtonInNoteFooter" ref="clipButton" :class="$style.footerButton" class="_button" @click.stop="clip()"> <i class="ti ti-paperclip"></i> </button> - <button v-if="prefer.s.showTranslationButtonInNoteFooter && $i?.policies.canUseTranslator && instance.translatorAvailable" class="_button" :class="$style.footerButton" :disabled="translating || !!translation" @click.stop="translate()"> + <button v-if="prefer.s.showTranslationButtonInNoteFooter && policies.canUseTranslator && instance.translatorAvailable" class="_button" :class="$style.footerButton" :disabled="translating || !!translation" @click.stop="translate()"> <i class="ti ti-language-hiragana"></i> </button> <button ref="menuButton" :class="$style.footerButton" class="_button" @click.stop="showMenu()"> @@ -181,7 +181,7 @@ SPDX-License-Identifier: AGPL-3.0-only <script lang="ts" setup> import { computed, inject, onMounted, ref, useTemplateRef, watch, provide } from 'vue'; -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; import * as Misskey from 'misskey-js'; import { isLink } from '@@/js/is-link.js'; import { shouldCollapsed } from '@@/js/collapsed.js'; @@ -226,7 +226,7 @@ import { getNoteSummary } from '@/utility/get-note-summary.js'; import MkRippleEffect from '@/components/MkRippleEffect.vue'; import { showMovedDialog } from '@/utility/show-moved-dialog.js'; import { boostMenuItems, computeRenoteTooltip } from '@/utility/boost-quote.js'; -import { instance, isEnabledUrlPreview } from '@/instance.js'; +import { instance, isEnabledUrlPreview, policies } from '@/instance.js'; import { focusPrev, focusNext } from '@/utility/focus.js'; import { getAppearNote } from '@/utility/get-appear-note.js'; import { prefer } from '@/preferences.js'; @@ -237,6 +237,7 @@ import SkMutedNote from '@/components/SkMutedNote.vue'; import SkNoteTranslation from '@/components/SkNoteTranslation.vue'; import { getSelfNoteIds } from '@/utility/get-self-note-ids.js'; import { extractPreviewUrls } from '@/utility/extract-preview-urls.js'; +import SkUrlPreviewGroup from '@/components/SkUrlPreviewGroup.vue'; const props = withDefaults(defineProps<{ note: Misskey.entities.Note; @@ -304,9 +305,9 @@ const galleryEl = useTemplateRef('galleryEl'); const isMyRenote = $i && ($i.id === note.value.userId); const showContent = ref(prefer.s.uncollapseCW); const parsed = computed(() => appearNote.value.text ? mfm.parse(appearNote.value.text) : null); -const urls = computed(() => parsed.value ? extractPreviewUrls(props.note, parsed.value) : null); +const urls = computed(() => parsed.value ? extractPreviewUrls(appearNote.value, parsed.value) : []); const selfNoteIds = computed(() => getSelfNoteIds(props.note)); -const isLong = shouldCollapsed(appearNote.value, urls.value ?? []); +const isLong = shouldCollapsed(appearNote.value, urls.value); const collapsed = ref(prefer.s.expandLongNote && appearNote.value.cw == null && isLong ? false : appearNote.value.cw == null && isLong); const isDeleted = ref(false); const renoted = ref(false); @@ -360,7 +361,7 @@ const keymap = { clip(); }, 't': () => { - if (prefer.s.showTranslationButtonInNoteFooter && $i?.policies.canUseTranslator && instance.translatorAvailable) { + if (prefer.s.showTranslationButtonInNoteFooter && policies.value.canUseTranslator && instance.translatorAvailable) { translate(); } }, @@ -921,11 +922,11 @@ function emitUpdReaction(emoji: string, delta: number) { .footer { display: flex; align-items: center; - justify-content: space-between; + justify-content: flex-start; position: relative; z-index: 1; margin-top: 0.4em; - max-width: 400px; + overflow-x: auto; } &:hover > .article > .main > .footer > .footerButton { @@ -947,10 +948,6 @@ function emitUpdReaction(emoji: string, delta: number) { .footerButton { font-size: 90%; - - &:not(:last-child) { - margin-right: 0; - } } } @@ -1238,10 +1235,6 @@ function emitUpdReaction(emoji: string, delta: number) { padding: 8px; opacity: 0.7; - &:not(:last-child) { - margin-right: 1.5em; - } - &:hover { color: var(--MI_THEME-fgHighlighted); } @@ -1358,25 +1351,7 @@ function emitUpdReaction(emoji: string, delta: number) { } } -@container (max-width: 400px) { - .root:not(.showActionsOnlyHover) { - .footerButton { - &:not(:last-child) { - margin-right: 0.2em; - } - } - } -} - @container (max-width: 350px) { - .root:not(.showActionsOnlyHover) { - .footerButton { - &:not(:last-child) { - margin-right: 0.1em; - } - } - } - .colorBar { top: 6px; left: 6px; @@ -1385,16 +1360,6 @@ function emitUpdReaction(emoji: string, delta: number) { } } -@container (max-width: 300px) { - .root:not(.showActionsOnlyHover) { - .footerButton { - &:not(:last-child) { - margin-right: 0.1em; - } - } - } -} - @container (max-width: 250px) { .quoteNote { padding: 12px; diff --git a/packages/frontend/src/components/SkNoteDetailed.vue b/packages/frontend/src/components/SkNoteDetailed.vue index b165b95d40..f761029cfb 100644 --- a/packages/frontend/src/components/SkNoteDetailed.vue +++ b/packages/frontend/src/components/SkNoteDetailed.vue @@ -108,7 +108,6 @@ SPDX-License-Identifier: AGPL-3.0-only :isBlock="true" class="_selectable" /> - <a v-if="appearNote.renote != null" :class="$style.rn">RN:</a> <SkNoteTranslation :note="note" :translation="translation" :translating="translating"></SkNoteTranslation> <MkButton v-if="!allowAnim && animated" :class="$style.playMFMButton" :small="true" @click="animatedMFM()" @click.stop><i class="ph-play ph-bold ph-lg "></i> {{ i18n.ts._animatedMFM.play }}</MkButton> <MkButton v-else-if="!prefer.s.animatedMfm && allowAnim && animated" :class="$style.playMFMButton" :small="true" @click="animatedMFM()" @click.stop><i class="ph-stop ph-bold ph-lg "></i> {{ i18n.ts._animatedMFM.stop }}</MkButton> @@ -117,7 +116,7 @@ SPDX-License-Identifier: AGPL-3.0-only </div> <MkPoll v-if="appearNote.poll" ref="pollViewer" :noteId="appearNote.id" :poll="appearNote.poll" :local="!appearNote.user.host" :class="$style.poll" :author="appearNote.user" :emojiUrls="appearNote.emojis"/> <div v-if="isEnabledUrlPreview"> - <MkUrlPreview v-for="url in urls" :key="url" :url="url" :compact="true" :detail="true" :showAsQuote="!appearNote.user.rejectQuotes" :skipNoteIds="selfNoteIds" style="margin-top: 6px;"/> + <SkUrlPreviewGroup :sourceNodes="nodes" :sourceNote="appearNote" :compact="true" :detail="true" :showAsQuote="!appearNote.user.rejectQuotes" :skipNoteIds="selfNoteIds" style="margin-top: 6px;" @click.stop/> </div> <div v-if="appearNote.renote" :class="$style.quote"><SkNoteSimple :note="appearNote.renote" :class="$style.quoteNote" :expandAllCws="props.expandAllCws"/></div> </div> @@ -132,7 +131,7 @@ SPDX-License-Identifier: AGPL-3.0-only </MkA> </div> <MkReactionsViewer v-if="appearNote.reactionAcceptance !== 'likeOnly'" ref="reactionsViewer" style="margin-top: 6px;" :note="appearNote"/> - <footer :class="$style.footer"> + <footer :class="$style.footer" class="_gaps _h_gaps" tabindex="0" role="group" :aria-label="i18n.ts.noteFooterLabel"> <button class="_button" :class="$style.noteFooterButton" @click="reply()"> <i class="ti ti-arrow-back-up"></i> <p v-if="appearNote.repliesCount > 0" :class="$style.noteFooterButtonCount">{{ number(appearNote.repliesCount) }}</p> @@ -174,7 +173,7 @@ SPDX-License-Identifier: AGPL-3.0-only <button v-if="prefer.s.showClipButtonInNoteFooter" ref="clipButton" class="_button" :class="$style.noteFooterButton" @click.stop="clip()"> <i class="ti ti-paperclip"></i> </button> - <button v-if="prefer.s.showTranslationButtonInNoteFooter && $i?.policies.canUseTranslator && instance.translatorAvailable" ref="translationButton" class="_button" :class="$style.noteFooterButton" :disabled="translating || !!translation" @click.stop="translate()"> + <button v-if="prefer.s.showTranslationButtonInNoteFooter && policies.canUseTranslator && instance.translatorAvailable" ref="translationButton" class="_button" :class="$style.noteFooterButton" :disabled="translating || !!translation" @click.stop="translate()"> <i class="ti ti-language-hiragana"></i> </button> <button ref="menuButton" class="_button" :class="$style.noteFooterButton" @click.stop="showMenu()"> @@ -238,7 +237,7 @@ SPDX-License-Identifier: AGPL-3.0-only <script lang="ts" setup> import { computed, inject, onMounted, onUnmounted, onUpdated, provide, ref, useTemplateRef, watch } from 'vue'; -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; import * as Misskey from 'misskey-js'; import { isLink } from '@@/js/is-link.js'; import * as config from '@@/js/config.js'; @@ -283,7 +282,7 @@ import MkPagination from '@/components/MkPagination.vue'; import MkReactionIcon from '@/components/MkReactionIcon.vue'; import MkButton from '@/components/MkButton.vue'; import { boostMenuItems, computeRenoteTooltip } from '@/utility/boost-quote.js'; -import { instance, isEnabledUrlPreview } from '@/instance.js'; +import { instance, isEnabledUrlPreview, policies } from '@/instance.js'; import { getAppearNote } from '@/utility/get-appear-note.js'; import { prefer } from '@/preferences.js'; import { getPluginHandlers } from '@/plugin.js'; @@ -291,7 +290,7 @@ import { DI } from '@/di.js'; import SkMutedNote from '@/components/SkMutedNote.vue'; import SkNoteTranslation from '@/components/SkNoteTranslation.vue'; import { getSelfNoteIds } from '@/utility/get-self-note-ids.js'; -import { extractPreviewUrls } from '@/utility/extract-preview-urls'; +import SkUrlPreviewGroup from '@/components/SkUrlPreviewGroup.vue'; const props = withDefaults(defineProps<{ note: Misskey.entities.Note; @@ -345,8 +344,7 @@ const isDeleted = ref(false); const renoted = ref(false); const translation = ref<Misskey.entities.NotesTranslateResponse | false | null>(null); const translating = ref(false); -const parsed = computed(() => appearNote.value.text ? mfm.parse(appearNote.value.text) : null); -const urls = computed(() => parsed.value ? extractPreviewUrls(props.note, parsed.value) : null); +const parsed = computed(() => appearNote.value.text ? mfm.parse(appearNote.value.text) : []); const selfNoteIds = computed(() => getSelfNoteIds(props.note)); const animated = computed(() => parsed.value ? checkAnimationFromMfm(parsed.value) : null); const allowAnim = ref(prefer.s.advancedMfm && prefer.s.animatedMfm ? true : false); @@ -394,7 +392,7 @@ const keymap = { clip(); }, 't': () => { - if (prefer.s.showTranslationButtonInNoteFooter && $i?.policies.canUseTranslator && instance.translatorAvailable) { + if (prefer.s.showTranslationButtonInNoteFooter && policies.value.canUseTranslator && instance.translatorAvailable) { translate(); } }, @@ -421,6 +419,11 @@ provide(DI.mfmEmojiReactCallback, (reaction) => { const tab = ref(props.initialTab); const reactionTabType = ref<string | null>(null); +// Auto-select the first page of reactions +watch(appearNote, n => { + reactionTabType.value ??= Object.keys(n.reactions)[0] ?? null; +}, { immediate: true }); + const renotesPagination = computed<Paging>(() => ({ endpoint: 'notes/renotes', limit: 10, @@ -918,13 +921,13 @@ onUnmounted(() => { } .footer { - display: flex; - align-items: center; - justify-content: space-between; - position: relative; - z-index: 1; - margin-top: 0.4em; - max-width: 400px; + display: flex; + align-items: center; + justify-content: flex-start; + position: relative; + z-index: 1; + margin-top: 0.4em; + overflow-x: auto; } .replyTo { @@ -1141,10 +1144,6 @@ onUnmounted(() => { padding: 8px; opacity: 0.7; - &:not(:last-child) { - margin-right: 1.5em; - } - &:hover { color: var(--MI_THEME-fgHighlighted); } @@ -1234,14 +1233,6 @@ onUnmounted(() => { } } -@container (max-width: 350px) { - .noteFooterButton { - &:not(:last-child) { - margin-right: 0.1em; - } - } -} - @container (max-width: 300px) { .root { font-size: 0.825em; @@ -1251,12 +1242,6 @@ onUnmounted(() => { width: 50px; height: 50px; } - - .noteFooterButton { - &:not(:last-child) { - margin-right: 0.1em; - } - } } .avatar { diff --git a/packages/frontend/src/components/SkNoteSub.vue b/packages/frontend/src/components/SkNoteSub.vue index 775436cb0f..4e8a3147ad 100644 --- a/packages/frontend/src/components/SkNoteSub.vue +++ b/packages/frontend/src/components/SkNoteSub.vue @@ -28,7 +28,7 @@ SPDX-License-Identifier: AGPL-3.0-only </div> </div> <MkReactionsViewer ref="reactionsViewer" :note="note"/> - <footer :class="$style.footer"> + <footer :class="$style.footer" class="_gaps _h_gaps" tabindex="0" role="group" :aria-label="i18n.ts.noteFooterLabel"> <button class="_button" :class="$style.noteFooterButton" @click="reply()"> <i class="ph-arrow-u-up-left ph-bold ph-lg"></i> <p v-if="note.repliesCount > 0" :class="$style.noteFooterButtonCount">{{ note.repliesCount }}</p> @@ -70,7 +70,7 @@ SPDX-License-Identifier: AGPL-3.0-only <button v-if="prefer.s.showClipButtonInNoteFooter" ref="clipButton" :class="$style.noteFooterButton" class="_button" @click.stop="clip()"> <i class="ti ti-paperclip"></i> </button> - <button v-if="prefer.s.showTranslationButtonInNoteFooter && $i?.policies.canUseTranslator && instance.translatorAvailable" ref="translationButton" class="_button" :class="$style.noteFooterButton" :disabled="translating || !!translation" @click.stop="translate()"> + <button v-if="prefer.s.showTranslationButtonInNoteFooter && policies.canUseTranslator && instance.translatorAvailable" ref="translationButton" class="_button" :class="$style.noteFooterButton" :disabled="translating || !!translation" @click.stop="translate()"> <i class="ti ti-language-hiragana"></i> </button> <button ref="menuButton" class="_button" :class="$style.noteFooterButton" @click.stop="menu()"> @@ -121,7 +121,8 @@ import { boostMenuItems, computeRenoteTooltip } from '@/utility/boost-quote.js'; import { prefer } from '@/preferences.js'; import { useNoteCapture } from '@/use/use-note-capture.js'; import SkMutedNote from '@/components/SkMutedNote.vue'; -import { instance } from '@/instance'; +import { instance, policies } from '@/instance'; +import { getAppearNote } from '@/utility/get-appear-note'; const props = withDefaults(defineProps<{ note: Misskey.entities.Note; @@ -141,7 +142,9 @@ const props = withDefaults(defineProps<{ onDeleteCallback: undefined, }); -const canRenote = computed(() => ['public', 'home'].includes(props.note.visibility) || props.note.userId === $i?.id); +const appearNote = computed(() => getAppearNote(props.note)); + +const canRenote = computed(() => ['public', 'home'].includes(appearNote.value.visibility) || appearNote.value.userId === $i?.id); const hideLine = computed(() => props.detail); const el = shallowRef<HTMLElement>(); @@ -158,19 +161,11 @@ const likeButton = shallowRef<HTMLElement>(); const renoteTooltip = computeRenoteTooltip(renoted); -let appearNote = computed(() => isRenote ? props.note.renote as Misskey.entities.Note : props.note); const defaultLike = computed(() => prefer.s.like ? prefer.s.like : null); const replies = ref<Misskey.entities.Note[]>([]); const mergedCW = computed(() => computeMergedCw(appearNote.value)); -const isRenote = ( - props.note.renote != null && - props.note.text == null && - props.note.fileIds && props.note.fileIds.length === 0 && - props.note.poll == null -); - const pleaseLoginContext = computed<OpenOnRemoteOptions>(() => ({ type: 'lookup', url: appearNote.value.url ?? appearNote.value.uri ?? `${config.url}/notes/${appearNote.value.id}`, @@ -220,8 +215,8 @@ async function reply(viaKeyboard = false): Promise<void> { pleaseLogin({ openOnRemote: pleaseLoginContext.value }); showMovedDialog(); await os.post({ - reply: props.note, - channel: props.note.channel ?? undefined, + reply: appearNote.value, + channel: appearNote.value.channel ?? undefined, animation: !viaKeyboard, }); focus(); @@ -231,9 +226,9 @@ function react(): void { pleaseLogin({ openOnRemote: pleaseLoginContext.value }); showMovedDialog(); sound.playMisskeySfx('reaction'); - if (props.note.reactionAcceptance === 'likeOnly') { + if (appearNote.value.reactionAcceptance === 'likeOnly') { misskeyApi('notes/like', { - noteId: props.note.id, + noteId: appearNote.value.id, override: defaultLike.value, }); const el = reactButton.value as HTMLElement | null | undefined; @@ -247,12 +242,12 @@ function react(): void { } } else { blur(); - reactionPicker.show(reactButton.value ?? null, props.note, reaction => { + reactionPicker.show(reactButton.value ?? null, appearNote.value, reaction => { misskeyApi('notes/reactions/create', { - noteId: props.note.id, + noteId: appearNote.value.id, reaction: reaction, }); - if (props.note.text && props.note.text.length > 100 && (Date.now() - new Date(props.note.createdAt).getTime() < 1000 * 3)) { + if (appearNote.value.text && appearNote.value.text.length > 100 && (Date.now() - new Date(appearNote.value.createdAt).getTime() < 1000 * 3)) { claimAchievement('reactWithoutRead'); } }, () => { @@ -266,7 +261,7 @@ function like(): void { showMovedDialog(); sound.playMisskeySfx('reaction'); misskeyApi('notes/like', { - noteId: props.note.id, + noteId: appearNote.value.id, override: defaultLike.value, }); const el = likeButton.value as HTMLElement | null | undefined; @@ -375,7 +370,7 @@ function quote() { }).then((cancelled) => { if (cancelled) return; misskeyApi('notes/renotes', { - noteId: props.note.id, + noteId: appearNote.value.id, userId: $i?.id, limit: 1, quote: true, @@ -397,12 +392,12 @@ function quote() { } function menu(): void { - const { menu, cleanup } = getNoteMenu({ note: props.note, translating, translation, isDeleted }); + const { menu, cleanup } = getNoteMenu({ note: appearNote.value, translating, translation, isDeleted }); os.popupMenu(menu, menuButton.value).then(focus).finally(cleanup); } async function clip(): Promise<void> { - os.popupMenu(await getNoteClipMenu({ note: props.note, isDeleted, currentClip: currentClip?.value }), clipButton.value).then(focus); + os.popupMenu(await getNoteClipMenu({ note: appearNote.value, isDeleted, currentClip: currentClip?.value }), clipButton.value).then(focus); } async function translate() { @@ -411,7 +406,7 @@ async function translate() { if (props.detail) { misskeyApi('notes/children', { - noteId: props.note.id, + noteId: appearNote.value.id, limit: prefer.s.numberOfReplies, showQuotes: false, }).then(res => { @@ -449,11 +444,11 @@ if (props.detail) { .footer { display: flex; align-items: center; - justify-content: space-between; + justify-content: flex-start; position: relative; z-index: 1; margin-top: 0.4em; - max-width: 400px; + overflow-x: auto; } .main { @@ -559,14 +554,6 @@ if (props.detail) { } } -@container (max-width: 400px) { - .noteFooterButton { - &:not(:last-child) { - margin-right: 0.7em; - } - } -} - .noteFooterButtonCount { display: inline; margin: 0 0 0 8px; diff --git a/packages/frontend/src/components/SkNoteTranslation.vue b/packages/frontend/src/components/SkNoteTranslation.vue index 170eea80cf..406242f1d1 100644 --- a/packages/frontend/src/components/SkNoteTranslation.vue +++ b/packages/frontend/src/components/SkNoteTranslation.vue @@ -33,7 +33,6 @@ if (_DEV_) { watch( [() => props.translation, () => props.translating], ([translation, translating]) => console.debug('Translation status changed: ', { translation, translating }), - { immediate: true }, ); } </script> diff --git a/packages/frontend/src/components/SkOldNoteWindow.vue b/packages/frontend/src/components/SkOldNoteWindow.vue index b6dbec81c5..aa1da2d6e3 100644 --- a/packages/frontend/src/components/SkOldNoteWindow.vue +++ b/packages/frontend/src/components/SkOldNoteWindow.vue @@ -40,19 +40,19 @@ SPDX-License-Identifier: AGPL-3.0-only <div v-show="appearNote.cw == null || showContent"> <span v-if="appearNote.isHidden" style="opacity: 0.5">({{ i18n.ts.private }})</span> <MkA v-if="appearNote.replyId" :class="$style.noteReplyTarget" :to="`/notes/${appearNote.replyId}`"><i class="ph-arrow-bend-left-up ph-bold ph-lg"></i></MkA> - <Mfm v-if="appearNote.text" :text="appearNote.text" :isBlock="true" :author="appearNote.user" :nyaize="'respect'" :emojiUrls="appearNote.emojis"/> + <Mfm v-if="appearNote.text" :text="appearNote.text" :parsedNodes="parsed" :isBlock="true" :author="appearNote.user" :nyaize="'respect'" :emojiUrls="appearNote.emojis"/> <a v-if="appearNote.renote != null" :class="$style.rn">RN:</a> <SkNoteTranslation :note="note" :translation="translation" :translating="translating"></SkNoteTranslation> <div v-if="appearNote.files && appearNote.files.length > 0"> <MkMediaList :mediaList="appearNote.files"/> </div> <MkPoll v-if="appearNote.poll" :noteId="appearNote.id" :poll="appearNote.poll" :local="!appearNote.user.host" :author="appearNote.user" :emojiUrls="appearNote.emojis" :class="$style.poll"/> - <MkUrlPreview v-for="url in urls" :key="url" :url="url" :compact="true" :detail="true" :showAsQuote="!appearNote.user.rejectQuotes" :skipNoteIds="selfNoteIds" style="margin-top: 6px;"/> + <SkUrlPreviewGroup :sourceNodes="parsed" :sourceNote="appearNote" :compact="true" :detail="true" :showAsQuote="!appearNote.user.rejectQuotes" :skipNoteIds="selfNoteIds" style="margin-top: 6px;" @click.stop/> <div v-if="appearNote.renote" :class="$style.quote"><MkNoteSimple :note="appearNote.renote" :class="$style.quoteNote"/></div> </div> <MkA v-if="appearNote.channel && !inChannel" :class="$style.channel" :to="`/channels/${appearNote.channel.id}`"><i class="ph-television ph-bold ph-lg"></i> {{ appearNote.channel.name }}</MkA> </div> - <footer :class="$style.footer"> + <footer :class="$style.footer" class="_gaps _h_gaps" tabindex="0" role="group" :aria-label="i18n.ts.noteFooterLabel"> <div :class="$style.noteFooterInfo"> <MkTime :time="appearNote.createdAt" mode="detail"/> </div> @@ -76,14 +76,13 @@ SPDX-License-Identifier: AGPL-3.0-only <script lang="ts" setup> import { inject, onMounted, ref, shallowRef, computed } from 'vue'; -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; import * as Misskey from 'misskey-js'; import MkNoteSimple from '@/components/MkNoteSimple.vue'; import MkMediaList from '@/components/MkMediaList.vue'; import MkCwButton from '@/components/MkCwButton.vue'; import MkWindow from '@/components/MkWindow.vue'; import MkPoll from '@/components/MkPoll.vue'; -import MkUrlPreview from '@/components/MkUrlPreview.vue'; import MkInstanceTicker from '@/components/MkInstanceTicker.vue'; import { userPage } from '@/filters/user.js'; import { i18n } from '@/i18n.js'; @@ -93,7 +92,7 @@ import { prefer } from '@/preferences'; import { getPluginHandlers } from '@/plugin.js'; import SkNoteTranslation from '@/components/SkNoteTranslation.vue'; import { getSelfNoteIds } from '@/utility/get-self-note-ids'; -import { extractPreviewUrls } from '@/utility/extract-preview-urls.js'; +import SkUrlPreviewGroup from '@/components/SkUrlPreviewGroup.vue'; const props = defineProps<{ note: Misskey.entities.Note; @@ -143,12 +142,11 @@ const isRenote = ( const el = shallowRef<HTMLElement>(); const appearNote = computed(() => isRenote ? note.value.renote as Misskey.entities.Note : note.value); -const parsed = computed(() => appearNote.value.text ? mfm.parse(appearNote.value.text) : null); +const parsed = computed(() => appearNote.value.text ? mfm.parse(appearNote.value.text) : []); const showContent = ref(false); const translation = ref<Misskey.entities.NotesTranslateResponse | false | null>(null); const translating = ref(false); -const urls = computed(() => parsed.value ? extractPreviewUrls(props.note, parsed.value) : null); const selfNoteIds = computed(() => getSelfNoteIds(props.note)); const showTicker = (prefer.s.instanceTicker === 'always') || (prefer.s.instanceTicker === 'remote' && appearNote.value.user.instance); @@ -163,11 +161,12 @@ const showTicker = (prefer.s.instanceTicker === 'always') || (prefer.s.instanceT } .footer { - position: relative; - z-index: 1; - margin-top: 0.4em; - width: max-content; - min-width: max-content; + position: relative; + z-index: 1; + margin-top: 0.4em; + width: max-content; + min-width: max-content; + overflow-x: auto; } .note { @@ -280,23 +279,11 @@ const showTicker = (prefer.s.instanceTicker === 'always') || (prefer.s.instanceT padding: 8px; opacity: 0.7; - &:not(:last-child) { - margin-right: 1.5em; - } - &:hover { color: var(--MI_THEME-fgHighlighted); } } -@container (max-width: 350px) { - .noteFooterButton { - &:not(:last-child) { - margin-right: 0.1em; - } - } -} - @container (max-width: 500px) { .root { font-size: 0.9em; @@ -323,11 +310,5 @@ const showTicker = (prefer.s.instanceTicker === 'always') || (prefer.s.instanceT width: 50px; height: 50px; } - - .noteFooterButton { - &:not(:last-child) { - margin-right: 0.1em; - } - } } </style> diff --git a/packages/frontend/src/components/SkTransitionGroup.vue b/packages/frontend/src/components/SkTransitionGroup.vue new file mode 100644 index 0000000000..1c07186501 --- /dev/null +++ b/packages/frontend/src/components/SkTransitionGroup.vue @@ -0,0 +1,43 @@ +<!-- +SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors +SPDX-License-Identifier: AGPL-3.0-only +--> + +<template> +<TransitionGroup v-if="animate ?? prefer.s.animation" v-bind="props" :class="props.class"> + <slot></slot> +</TransitionGroup> +<component :is="tag" v-else :class="props.class"> + <slot></slot> +</component> +</template> + +<script setup lang="ts"> +import type { TransitionGroupProps } from 'vue'; +import { prefer } from '@/preferences'; + +// This is a "best guess" type. +// If any valid :class binding produces a type error here, then please change this to match. +type ClassBinding = string | Record<string, boolean | undefined>; + +// This can be an inline type, but pulling it out makes TS errors clearer. +interface SkTransitionGroupProps extends TransitionGroupProps { + /** + * Override CSS styles for the TransitionGroup or native element. + */ + class?: undefined | ClassBinding | ClassBinding[]; + + /** + * If true, will render a TransitionGroup. + * If false, will render a native element. + * If null or undefined (default), will respect the value of prefer.s.animation. + */ + animate?: boolean | undefined | null; +} + +const props = withDefaults(defineProps<SkTransitionGroupProps>(), { + tag: 'div', + class: undefined, + animate: undefined, +}); +</script> diff --git a/packages/frontend/src/components/SkUrlPreviewGroup.vue b/packages/frontend/src/components/SkUrlPreviewGroup.vue new file mode 100644 index 0000000000..dbd930248a --- /dev/null +++ b/packages/frontend/src/components/SkUrlPreviewGroup.vue @@ -0,0 +1,348 @@ +<!-- +SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors +SPDX-License-Identifier: AGPL-3.0-only +--> + +<template> +<div v-if="isRefreshing"> + <MkLoading :class="$style.loading"></MkLoading> +</div> +<template v-else> + <MkUrlPreview + v-for="preview of urlPreviews" + :key="preview.url" + :url="preview.url" + :previewHint="preview" + :noteHint="preview.note" + :attributionHint="preview.attributionUser" + :detail="detail" + :compact="compact" + :showAsQuote="showAsQuote" + :showActions="showActions" + :skipNoteIds="skipNoteIds" + ></MkUrlPreview> +</template> +</template> + +<script setup lang="ts"> +import * as Misskey from 'misskey-js'; +import * as mfm from 'mfm-js'; +import { computed, ref, watch } from 'vue'; +import { versatileLang } from '@@/js/intl-const'; +import promiseLimit from 'promise-limit'; +import type { SummalyResult } from '@/components/MkUrlPreview.vue'; +import { extractPreviewUrls } from '@/utility/extract-preview-urls'; +import { extractUrlFromMfm } from '@/utility/extract-url-from-mfm'; +import { $i } from '@/i'; +import { misskeyApi } from '@/utility/misskey-api'; +import MkUrlPreview from '@/components/MkUrlPreview.vue'; +import { getNoteUrls } from '@/utility/getNoteUrls'; + +type Summary = SummalyResult & { + note?: Misskey.entities.Note | null; + attributionUser?: Misskey.entities.User | null; +}; + +type Limiter<T> = ReturnType<typeof promiseLimit<T>>; + +const props = withDefaults(defineProps<{ + sourceUrls?: string[]; + sourceNodes?: mfm.MfmNode[]; + sourceText?: string; + sourceNote?: Misskey.entities.Note; + + detail?: boolean; + compact?: boolean; + showAsQuote?: boolean; + showActions?: boolean; + skipNoteIds?: string[]; +}>(), { + sourceUrls: undefined, + sourceText: undefined, + sourceNodes: undefined, + sourceNote: undefined, + + detail: undefined, + compact: undefined, + showAsQuote: undefined, + showActions: undefined, + skipNoteIds: () => [], +}); + +const urlPreviews = ref<Summary[]>([]); + +const urls = computed<string[]>(() => { + if (props.sourceUrls) { + return props.sourceUrls; + } + + // sourceNodes > sourceText > sourceNote + const source = + props.sourceNodes ?? + (props.sourceText ? mfm.parse(props.sourceText) : null) ?? + (props.sourceNote?.text ? mfm.parse(props.sourceNote.text) : null); + + if (source) { + if (props.sourceNote) { + return extractPreviewUrls(props.sourceNote, source); + } else { + return extractUrlFromMfm(source); + } + } + + return []; +}); + +// todo un-ref these +const isRefreshing = ref<Promise<void> | false>(false); +const cachedNotes = ref(new Map<string, Misskey.entities.Note | null>()); +const cachedPreviews = ref(new Map<string, Summary | null>()); +const cachedUsers = new Map<string, Misskey.entities.User | null>(); + +/** + * Refreshes the group. + * Calls are automatically de-duplicated. + */ +function refresh(): Promise<void> { + if (isRefreshing.value) return isRefreshing.value; + + const promise = doRefresh(); + promise.finally(() => isRefreshing.value = false); + isRefreshing.value = promise; + return promise; +} + +/** + * Refreshes the group. + * Don't call this directly - use refresh() instead! + */ +async function doRefresh(): Promise<void> { + let previews = await fetchPreviews(); + + // Remove duplicates + previews = deduplicatePreviews(previews); + + // Remove any with hidden notes + previews = previews.filter(preview => !preview.note || !props.skipNoteIds.includes(preview.note.id)); + + urlPreviews.value = previews; +} + +async function fetchPreviews(): Promise<Summary[]> { + const userLimiter = promiseLimit<Misskey.entities.User | null>(4); + const noteLimiter = promiseLimit<Misskey.entities.Note | null>(2); + const summaryLimiter = promiseLimit<Summary | null>(5); + + const summaries = await Promise.all(urls.value.map(url => + summaryLimiter(async () => { + return await fetchPreview(url); + }).then(async (summary) => { + if (summary) { + await Promise.all([ + attachNote(summary, noteLimiter), + attachAttribution(summary, userLimiter), + ]); + } + + return summary; + }))); + + return summaries.filter((preview): preview is Summary => preview != null); +} + +async function fetchPreview(url: string): Promise<Summary | null> { + const cached = cachedPreviews.value.get(url); + if (cached) { + return cached; + } + + const headers = $i ? { Authorization: `Bearer ${$i.token}` } : undefined; + const params = new URLSearchParams({ url, lang: versatileLang }); + const res = await window.fetch(`/url?${params.toString()}`, { headers }).catch(() => null); + + if (res?.ok) { + // Success - got the summary + const summary: Summary = await res.json(); + cachedPreviews.value.set(url, summary); + if (summary.url !== url) { + cachedPreviews.value.set(summary.url, summary); + } + return summary; + } + + // Failed, blocked, or not found + cachedPreviews.value.set(url, null); + return null; +} + +async function attachNote(summary: Summary, noteLimiter: Limiter<Misskey.entities.Note | null>): Promise<void> { + if (props.showAsQuote && summary.activityPub && summary.haveNoteLocally) { + // Have to pull this out to make TS happy + const noteUri = summary.activityPub; + + summary.note = await noteLimiter(async () => { + return await fetchNote(noteUri); + }); + } +} + +async function fetchNote(noteUri: string): Promise<Misskey.entities.Note | null> { + const cached = cachedNotes.value.get(noteUri); + if (cached) { + return cached; + } + + const response = await misskeyApi('ap/show', { uri: noteUri }).catch(() => null); + if (response && response.type === 'Note') { + const note = response['object']; + + // Success - got the note + cachedNotes.value.set(noteUri, note); + if (note.uri && note.uri !== noteUri) { + cachedNotes.value.set(note.uri, note); + } + return note; + } + + // Failed, blocked, or not found + cachedNotes.value.set(noteUri, null); + return null; +} + +async function attachAttribution(summary: Summary, userLimiter: Limiter<Misskey.entities.User | null>): Promise<void> { + if (summary.linkAttribution) { + // Have to pull this out to make TS happy + const userId = summary.linkAttribution.userId; + + summary.attributionUser = await userLimiter(async () => { + return await fetchUser(userId); + }); + } +} + +async function fetchUser(userId: string): Promise<Misskey.entities.User | null> { + const cached = cachedUsers.get(userId); + if (cached) { + return cached; + } + + const user = await misskeyApi('users/show', { userId }).catch(() => null); + + cachedUsers.set(userId, user); + return user; +} + +function deduplicatePreviews(previews: Summary[]): Summary[] { + // eslint-disable-next-line no-param-reassign + previews = previews + // Remove any previews with duplicate URL + .filter((preview, index) => !previews.some((p, i) => { + // Skip the current preview (don't count self as duplicate). + if (p === preview) return false; + + // Skip differing URLs (not duplicate). + if (p.url !== preview.url) return false; + + // Skip if we have AP and the other doesn't + if (preview.activityPub && !p.activityPub) return false; + + // Skip if we have a note and the other doesn't + if (preview.note && !p.note) return false; + + // Skip later previews (keep the earliest instance)... + // ...but only if we have AP or the later one doesn't... + // ...and only if we have note or the later one doesn't. + if (i > index && (preview.activityPub || !p.activityPub) && (preview.note || !p.note)) return false; + + // If we get here, then "preview" is a duplicate of "p" and should be skipped. + return true; + })); + + // eslint-disable-next-line no-param-reassign + previews = previews + // Remove any previews with duplicate AP + .filter((preview, index) => !previews.some((p, i) => { + // Skip the current preview (don't count self as duplicate). + if (p === preview) return false; + + // Skip if we don't have AP + if (!preview.activityPub) return false; + + // Skip if other does not have AP + if (!p.activityPub) return false; + + // Skip differing URLs (not duplicate). + if (p.activityPub !== preview.activityPub) return false; + + // Skip later previews (keep the earliest instance) + if (i > index) return false; + + // If we get here, then "preview" is a duplicate of "p" and should be skipped. + return true; + })); + + // eslint-disable-next-line no-param-reassign + previews = previews + // Remove any previews with duplicate note + .filter((preview, index) => !previews.some((p, i) => { + // Skip the current preview (don't count self as duplicate). + if (p === preview) return false; + + // Skip if we don't have a note + if (!preview.note) return false; + + // Skip if other does not have a note + if (!p.note) return false; + + // Skip differing notes (not duplicate). + if (p.note.id !== preview.note.id) return false; + + // Skip later previews (keep the earliest instance) + if (i > index) return false; + + // If we get here, then "preview" is a duplicate of "p" and should be skipped. + return true; + })); + + // eslint-disable-next-line no-param-reassign + previews = previews + // Remove any previews where the note duplicates url + .filter((preview, index) => !previews.some((p, i) => { + // Skip the current preview (don't count self as duplicate). + if (p === preview) return false; + + // Skip if we have a note + if (preview.note) return false; + + // Skip if other does not have a note + if (!p.note) return false; + + // Skip later previews (keep the earliest instance) + if (i > index) return false; + + const noteUrls = getNoteUrls(p.note); + + // Remove if other duplicates our AP URL + if (preview.activityPub && noteUrls.includes(preview.activityPub)) return true; + + // Remove if other duplicates our main URL + return noteUrls.includes(preview.url); + })); + + return previews; +} + +// Kick everything off, and watch for changes. +watch( + [urls, () => props.showAsQuote, () => props.skipNoteIds], + () => refresh(), + { immediate: true }, +); +</script> + +<style module lang="scss"> +.loading { + box-shadow: 0 0 0 1px var(--MI_THEME-divider); + border-radius: var(--MI-radius-sm); +} +</style> diff --git a/packages/frontend/src/components/global/MkMfm.ts b/packages/frontend/src/components/global/MkMfm.ts index dea486e66d..9f92c43b68 100644 --- a/packages/frontend/src/components/global/MkMfm.ts +++ b/packages/frontend/src/components/global/MkMfm.ts @@ -4,7 +4,7 @@ */ import { h, defineAsyncComponent } from 'vue'; -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; import * as Misskey from 'misskey-js'; import { host } from '@@/js/config.js'; import CkFollowMouse from '../CkFollowMouse.vue'; diff --git a/packages/frontend/src/components/global/MkStickyContainer.vue b/packages/frontend/src/components/global/MkStickyContainer.vue index 05245716c2..73ce393113 100644 --- a/packages/frontend/src/components/global/MkStickyContainer.vue +++ b/packages/frontend/src/components/global/MkStickyContainer.vue @@ -5,17 +5,17 @@ SPDX-License-Identifier: AGPL-3.0-only <template> <div ref="rootEl"> - <div ref="headerEl" :class="$style.header"> + <div ref="headerEl" :class="{ [$style.header]: sticky }"> <slot name="header"></slot> </div> <div - :class="$style.body" + :class="{ [$style.body]: sticky }" :data-sticky-container-header-height="headerHeight" :data-sticky-container-footer-height="footerHeight" > <slot></slot> </div> - <div ref="footerEl" :class="$style.footer"> + <div ref="footerEl" :class="{ [$style.footer]: sticky }"> <slot name="footer"></slot> </div> </div> @@ -25,6 +25,12 @@ SPDX-License-Identifier: AGPL-3.0-only import { onMounted, onUnmounted, provide, inject, ref, watch, useTemplateRef } from 'vue'; import { DI } from '@/di.js'; +withDefaults(defineProps<{ + sticky?: boolean, +}>(), { + sticky: true, +}); + const rootEl = useTemplateRef('rootEl'); const headerEl = useTemplateRef('headerEl'); const footerEl = useTemplateRef('footerEl'); diff --git a/packages/frontend/src/components/global/PageWithHeader.vue b/packages/frontend/src/components/global/PageWithHeader.vue index d2e59bf4ad..485ea687de 100644 --- a/packages/frontend/src/components/global/PageWithHeader.vue +++ b/packages/frontend/src/components/global/PageWithHeader.vue @@ -6,8 +6,8 @@ SPDX-License-Identifier: AGPL-3.0-only <template> <div ref="rootEl" :class="[$style.root, reversed ? '_pageScrollableReversed' : '_pageScrollable']"> <MkStickyContainer> - <template #header><MkPageHeader v-model:tab="tab" v-bind="pageHeaderProps"/></template> - <div :class="$style.body"> + <template #header><MkPageHeader v-model:tab="tab" v-bind="pageHeaderProps" :class="{ _spacer: spacer }"/></template> + <div :class="[ $style.body, { _spacer: spacer } ]"> <MkSwiper v-if="swipable && (props.tabs?.length ?? 1) > 1" v-model:tab="tab" :class="$style.swiper" :tabs="props.tabs" :page="props.page"> <slot></slot> </MkSwiper> @@ -30,13 +30,16 @@ const props = withDefaults(defineProps<PageHeaderProps & { reversed?: boolean; swipable?: boolean; page?: string; + spacer?: boolean; }>(), { reversed: false, swipable: true, + page: undefined, + spacer: false, }); const pageHeaderProps = computed(() => { - const { reversed, ...rest } = props; + const { reversed, spacer, ...rest } = props; return rest; }); diff --git a/packages/frontend/src/components/global/StackingRouterView.vue b/packages/frontend/src/components/global/StackingRouterView.vue index c95c74aef3..38a5c1ba23 100644 --- a/packages/frontend/src/components/global/StackingRouterView.vue +++ b/packages/frontend/src/components/global/StackingRouterView.vue @@ -4,12 +4,12 @@ SPDX-License-Identifier: AGPL-3.0-only --> <template> -<TransitionGroup - :enterActiveClass="prefer.s.animation ? $style.transition_x_enterActive : ''" - :leaveActiveClass="prefer.s.animation ? $style.transition_x_leaveActive : ''" - :enterFromClass="prefer.s.animation ? $style.transition_x_enterFrom : ''" - :leaveToClass="prefer.s.animation ? $style.transition_x_leaveTo : ''" - :moveClass="prefer.s.animation ? $style.transition_x_move : ''" +<SkTransitionGroup + :enterActiveClass="$style.transition_x_enterActive" + :leaveActiveClass="$style.transition_x_leaveActive" + :enterFromClass="$style.transition_x_enterFrom" + :leaveToClass="$style.transition_x_leaveTo" + :moveClass="$style.transition_x_move" :duration="200" tag="div" :class="$style.tabs" > @@ -37,7 +37,7 @@ SPDX-License-Identifier: AGPL-3.0-only </div> </div> </div> -</TransitionGroup> +</SkTransitionGroup> </template> <script lang="ts" setup> @@ -47,6 +47,7 @@ import { prefer } from '@/preferences.js'; import MkLoadingPage from '@/pages/_loading_.vue'; import { DI } from '@/di.js'; import { deepEqual } from '@/utility/deep-equal.js'; +import SkTransitionGroup from '@/components/SkTransitionGroup.vue'; const props = defineProps<{ router?: Router; diff --git a/packages/frontend/src/components/page/page.note.vue b/packages/frontend/src/components/page/page.note.vue index df26874c17..543e9afdaf 100644 --- a/packages/frontend/src/components/page/page.note.vue +++ b/packages/frontend/src/components/page/page.note.vue @@ -11,8 +11,9 @@ SPDX-License-Identifier: AGPL-3.0-only </template> <script lang="ts" setup> -import { onMounted, ref } from 'vue'; +import { onMounted, onUnmounted, ref } from 'vue'; import * as Misskey from 'misskey-js'; +import { retryOnThrottled } from '@@/js/retry-on-throttled.js'; import MkNote from '@/components/MkNote.vue'; import MkNoteDetailed from '@/components/MkNoteDetailed.vue'; import { misskeyApi } from '@/utility/misskey-api.js'; @@ -20,16 +21,25 @@ import { misskeyApi } from '@/utility/misskey-api.js'; const props = defineProps<{ block: Misskey.entities.PageBlock, page: Misskey.entities.Page, + index: number; }>(); const note = ref<Misskey.entities.Note | null>(null); +// eslint-disable-next-line id-denylist +let timeoutId: ReturnType<typeof window.setTimeout> | null = null; + onMounted(() => { if (props.block.note == null) return; - misskeyApi('notes/show', { noteId: props.block.note }) - .then(result => { - note.value = result; - }); + timeoutId = window.setTimeout(async () => { + note.value = await retryOnThrottled(() => misskeyApi('notes/show', { noteId: props.block.note })); + }, 500 * props.index); // rate limit is 2 reqs per sec +}); + +onUnmounted(() => { + if (timeoutId !== null) { + window.clearTimeout(timeoutId); + } }); </script> diff --git a/packages/frontend/src/components/page/page.text.vue b/packages/frontend/src/components/page/page.text.vue index ef3524fe7a..3891380dd0 100644 --- a/packages/frontend/src/components/page/page.text.vue +++ b/packages/frontend/src/components/page/page.text.vue @@ -7,29 +7,20 @@ SPDX-License-Identifier: AGPL-3.0-only <div class="_gaps" :class="$style.textRoot"> <Mfm :text="block.text ?? ''" :isBlock="true" :isNote="false"/> <div v-if="isEnabledUrlPreview" class="_gaps_s"> - <MkUrlPreview v-for="url in urls" :key="url" :url="url" :showAsQuote="!page.user.rejectQuotes"/> + <SkUrlPreviewGroup :sourceText="block.text" :showAsQuote="!page.user.rejectQuotes" @click.stop/> </div> </div> </template> <script lang="ts" setup> -import { defineAsyncComponent, computed } from 'vue'; -import * as mfm from '@transfem-org/sfm-js'; import * as Misskey from 'misskey-js'; -import { extractUrlFromMfm } from '@/utility/extract-url-from-mfm.js'; import { isEnabledUrlPreview } from '@/instance.js'; +import SkUrlPreviewGroup from '@/components/SkUrlPreviewGroup.vue'; -const MkUrlPreview = defineAsyncComponent(() => import('@/components/MkUrlPreview.vue')); - -const props = defineProps<{ +defineProps<{ block: Misskey.entities.PageBlock, page: Misskey.entities.Page, }>(); - -const urls = computed(() => { - if (!props.block.text) return []; - return extractUrlFromMfm(mfm.parse(props.block.text)); -}); </script> <style lang="scss" module> diff --git a/packages/frontend/src/components/page/page.vue b/packages/frontend/src/components/page/page.vue index a31c5eff28..9f9feeed49 100644 --- a/packages/frontend/src/components/page/page.vue +++ b/packages/frontend/src/components/page/page.vue @@ -5,7 +5,7 @@ SPDX-License-Identifier: AGPL-3.0-only <template> <div :class="{ [$style.center]: page.alignCenter, [$style.serif]: page.font === 'serif' }" class="_gaps"> - <XBlock v-for="child in page.content" :key="child.id" :page="page" :block="child" :h="2"/> + <XBlock v-for="(child, index) in page.content" :key="child.id" :index="index" :page="page" :block="child" :h="2"/> </div> </template> diff --git a/packages/frontend/src/i.ts b/packages/frontend/src/i.ts index a71ed1671f..0021298d63 100644 --- a/packages/frontend/src/i.ts +++ b/packages/frontend/src/i.ts @@ -3,7 +3,7 @@ * SPDX-License-Identifier: AGPL-3.0-only */ -import { reactive } from 'vue'; +import { computed, reactive } from 'vue'; import * as Misskey from 'misskey-js'; import { miLocalStorage } from '@/local-storage.js'; diff --git a/packages/frontend/src/instance.ts b/packages/frontend/src/instance.ts index e75e3dfd34..953af6333a 100644 --- a/packages/frontend/src/instance.ts +++ b/packages/frontend/src/instance.ts @@ -3,11 +3,12 @@ * SPDX-License-Identifier: AGPL-3.0-only */ -import { computed, reactive } from 'vue'; +import { computed, nextTick, reactive } from 'vue'; import * as Misskey from 'misskey-js'; import { misskeyApi } from '@/utility/misskey-api.js'; import { miLocalStorage } from '@/local-storage.js'; import { DEFAULT_INFO_IMAGE_URL, DEFAULT_NOT_FOUND_IMAGE_URL, DEFAULT_SERVER_ERROR_IMAGE_URL } from '@@/js/const.js'; +import { $i } from '@/i'; // TODO: 他のタブと永続化されたstateを同期 @@ -38,6 +39,8 @@ export const notFoundImageUrl = computed(() => instance.notFoundImageUrl ?? DEFA export const isEnabledUrlPreview = computed(() => instance.enableUrlPreview ?? true); +export const policies = computed<Misskey.entities.RolePolicies>(() => $i?.policies ?? instance.policies); + export async function fetchInstance(force = false): Promise<Misskey.entities.MetaDetailed> { if (!force) { const cachedAt = miLocalStorage.getItem('instanceCachedAt') ? parseInt(miLocalStorage.getItem('instanceCachedAt')!) : 0; @@ -60,3 +63,8 @@ export async function fetchInstance(force = false): Promise<Misskey.entities.Met return instance; } + +// instance export can be empty sometimes, which causes problems. +await fetchInstance().catch(err => { + console.warn('Initial meta fetch failed:', err); +}); diff --git a/packages/frontend/src/lib/nirax.ts b/packages/frontend/src/lib/nirax.ts index a166df9eb0..792dde7dd1 100644 --- a/packages/frontend/src/lib/nirax.ts +++ b/packages/frontend/src/lib/nirax.ts @@ -274,7 +274,7 @@ export class Nirax<DEF extends RouteDef[]> extends EventEmitter<RouterEvents> { } else { redirectPath = current.route.redirect + (current._parsedRoute.queryString ? '?' + current._parsedRoute.queryString : '') + (current._parsedRoute.hash ? '#' + current._parsedRoute.hash : ''); } - if (_DEV_) console.log('Redirecting to: ', redirectPath); + if (_DEV_) console.debug('Redirecting to: ', redirectPath); if (_redirected && this.redirectCount++ > 10) { throw new Error('redirect loop detected'); } diff --git a/packages/frontend/src/lib/pizzax.ts b/packages/frontend/src/lib/pizzax.ts index a232ced75e..4c55b1ffa5 100644 --- a/packages/frontend/src/lib/pizzax.ts +++ b/packages/frontend/src/lib/pizzax.ts @@ -97,7 +97,7 @@ export class Pizzax<T extends StateDef> { if (this.isPureObject(value) && this.isPureObject(def)) { const merged = deepMerge(value, def); - if (_DEV_) console.log('Merging state. Incoming: ', value, ' Default: ', def, ' Result: ', merged); + if (_DEV_) console.debug('Merging state. Incoming: ', value, ' Default: ', def, ' Result: ', merged); return merged as X; } diff --git a/packages/frontend/src/local-storage.ts b/packages/frontend/src/local-storage.ts index 5c795b1f9d..f1d660a45b 100644 --- a/packages/frontend/src/local-storage.ts +++ b/packages/frontend/src/local-storage.ts @@ -48,23 +48,23 @@ export type Keys = ( //const safeSessionStorage = new Map<Keys, string>(); export const miLocalStorage = { - getItem: (key: Keys): string | null => { - return window.localStorage.getItem(key); + getItem: <T extends string = string>(key: Keys): T | null => { + return window.localStorage.getItem(key) as T | null; }, - setItem: (key: Keys, value: string): void => { + setItem: <T extends string = string>(key: Keys, value: T): void => { window.localStorage.setItem(key, value); }, removeItem: (key: Keys): void => { window.localStorage.removeItem(key); }, - getItemAsJson: (key: Keys): any | undefined => { + getItemAsJson: <T = any>(key: Keys): T | undefined => { const item = miLocalStorage.getItem(key); if (item === null) { return undefined; } return JSON.parse(item); }, - setItemAsJson: (key: Keys, value: any): void => { + setItemAsJson: <T = any>(key: Keys, value: T): void => { miLocalStorage.setItem(key, JSON.stringify(value)); }, }; diff --git a/packages/frontend/src/os.ts b/packages/frontend/src/os.ts index 5a12e3ae6d..24eb81beca 100644 --- a/packages/frontend/src/os.ts +++ b/packages/frontend/src/os.ts @@ -101,7 +101,7 @@ export const apiWithDialog = (< }); export function promiseDialog<T extends Promise<any>>( - promise: T, + promise: T | (() => T), onSuccess?: ((res: Awaited<T>) => void) | null, onFailure?: ((err: Misskey.api.APIError) => void) | null, text?: string, @@ -109,6 +109,10 @@ export function promiseDialog<T extends Promise<any>>( const showing = ref(true); const success = ref(false); + if (typeof(promise) === 'function') { + promise = promise(); + } + promise.then(res => { if (onSuccess) { showing.value = false; diff --git a/packages/frontend/src/pages/admin-user.vue b/packages/frontend/src/pages/admin-user.vue index d3c0de3040..dc29ae2f80 100644 --- a/packages/frontend/src/pages/admin-user.vue +++ b/packages/frontend/src/pages/admin-user.vue @@ -4,11 +4,11 @@ SPDX-License-Identifier: AGPL-3.0-only --> <template> -<PageWithHeader v-model:tab="tab" :actions="headerActions" :tabs="headerTabs"> - <div class="_spacer" style="--MI_SPACER-w: 600px; --MI_SPACER-min: 16px; --MI_SPACER-max: 32px;"> - <FormSuspense :p="init"> +<PageWithHeader v-model:tab="tab" :actions="headerActions" :tabs="headerTabs" :spacer="true" style="--MI_SPACER-w: 700px; --MI_SPACER-min: 16px; --MI_SPACER-max: 32px;"> + <FormSuspense v-if="init" :p="init"> + <div v-if="user && info"> <div v-if="tab === 'overview'" class="_gaps"> - <div v-if="user" class="aeakzknw"> + <div class="aeakzknw"> <MkAvatar class="avatar" :user="user" indicator link preview/> <div class="body"> <span class="name"><MkUserName class="name" :user="user"/></span> @@ -20,19 +20,14 @@ SPDX-License-Identifier: AGPL-3.0-only <span class="_monospace">{{ user.id }}</span> <button v-tooltip="i18n.ts.copy" class="_textButton" style="margin-left: 0.5em;" @click="copyToClipboard(user.id)"><i class="ti ti-copy"></i></button> </span> - <span class="state"> - <span v-if="!approved" class="silenced">{{ i18n.ts.notApproved }}</span> - <span v-if="approved && !user.host" class="moderator">{{ i18n.ts.approved }}</span> - <span v-if="suspended" class="suspended">{{ i18n.ts.suspended }}</span> - <span v-if="silenced" class="silenced">{{ i18n.ts.silenced }}</span> - <span v-if="moderator" class="moderator">{{ i18n.ts.moderator }}</span> - </span> </div> </div> + <SkBadgeStrip v-if="badges.length > 0" :badges="badges"></SkBadgeStrip> + <MkInfo v-if="isSystem">{{ i18n.ts.isSystemAccount }}</MkInfo> - <MkFolder v-if="!isSystem"> + <MkFolder v-if="!isSystem" :sticky="false"> <template #icon><i class="ph-list-bullets ph-bold ph-lg"></i></template> <template #label>{{ i18n.ts.details }}</template> <div style="display: flex; flex-direction: column; gap: 1em;"> @@ -89,7 +84,7 @@ SPDX-License-Identifier: AGPL-3.0-only </div> </MkFolder> - <MkFolder v-if="info"> + <MkFolder v-if="info" :sticky="false"> <template #icon><i class="ph-scroll ph-bold ph-lg"></i></template> <template #label>{{ i18n.ts._role.policies }}</template> <div class="_gaps"> @@ -99,7 +94,7 @@ SPDX-License-Identifier: AGPL-3.0-only </div> </MkFolder> - <MkFolder v-if="iAmAdmin && ips && ips.length > 0"> + <MkFolder v-if="iAmAdmin && ips && ips.length > 0" :sticky="false"> <template #icon><i class="ph-network ph-bold ph-lg"></i></template> <template #label>{{ i18n.ts.ip }}</template> <MkInfo>{{ i18n.ts.ipTip }}</MkInfo> @@ -109,7 +104,7 @@ SPDX-License-Identifier: AGPL-3.0-only </div> </MkFolder> - <MkFolder v-if="iAmModerator" :defaultOpen="moderationNote.length > 0"> + <MkFolder v-if="iAmModerator" :defaultOpen="moderationNote.length > 0" :sticky="false"> <template #icon><i class="ph-stamp ph-bold ph-lg"></i></template> <template #label>{{ i18n.ts.moderationNote }}</template> <MkTextarea v-model="moderationNote" manualSave @update:modelValue="onModerationNoteChanged"> @@ -135,6 +130,11 @@ SPDX-License-Identifier: AGPL-3.0-only </div> </FormSection> + <FormSection v-else-if="info.signupReason"> + <template #label>{{ i18n.ts.signupReason }}</template> + {{ info.signupReason }} + </FormSection> + <FormSection v-if="!isSystem && user && iAmModerator"> <div class="_gaps"> <MkSwitch v-model="silenced" @update:modelValue="toggleSilence">{{ i18n.ts.silence }}</MkSwitch> @@ -233,14 +233,46 @@ SPDX-License-Identifier: AGPL-3.0-only </div> <div v-else-if="tab === 'raw'" class="_gaps_m"> - <MkObjectView v-if="info && $i.isAdmin" tall :value="info"> - </MkObjectView> + <MkFolder :sticky="false" :defaultOpen="true"> + <template #icon><i class="ph-user-circle ph-bold ph-lg"></i></template> + <template #label>{{ i18n.ts.user }}</template> + <template #header> + <div :class="$style.rawFolderHeader"> + <span>{{ i18n.ts.rawUserDescription }}</span> + <button class="_textButton" @click="copyToClipboard(JSON.stringify(user, null, 4))"><i class="ti ti-copy"></i> {{ i18n.ts.copy }}</button> + </div> + </template> + + <MkObjectView tall :value="user"/> + </MkFolder> - <MkObjectView tall :value="user"> - </MkObjectView> + <MkFolder :sticky="false"> + <template #icon><i class="ti ti-info-circle"></i></template> + <template #label>{{ i18n.ts.details }}</template> + <template #header> + <div :class="$style.rawFolderHeader"> + <span>{{ i18n.ts.rawInfoDescription }}</span> + <button class="_textButton" @click="copyToClipboard(JSON.stringify(info, null, 4))"><i class="ti ti-copy"></i> {{ i18n.ts.copy }}</button> + </div> + </template> + + <MkObjectView tall :value="info"/> + </MkFolder> + + <MkFolder v-if="ap" :sticky="false"> + <template #icon><i class="ph-globe ph-bold ph-lg"></i></template> + <template #label>{{ i18n.ts.activityPub }}</template> + <template #header> + <div :class="$style.rawFolderHeader"> + <span>{{ i18n.ts.rawApDescription }}</span> + <button class="_textButton" @click="copyToClipboard(JSON.stringify(ap, null, 4))"><i class="ti ti-copy"></i> {{ i18n.ts.copy }}</button> + </div> + </template> + <MkObjectView tall :value="ap"/> + </MkFolder> </div> - </FormSuspense> - </div> + </div> + </FormSuspense> </PageWithHeader> </template> @@ -248,6 +280,8 @@ SPDX-License-Identifier: AGPL-3.0-only import { computed, defineAsyncComponent, watch, ref } from 'vue'; import * as Misskey from 'misskey-js'; import { url } from '@@/js/config.js'; +import type { Badge } from '@/components/SkBadgeStrip.vue'; +import type { ChartSrc } from '@/components/MkChart.vue'; import MkChart from '@/components/MkChart.vue'; import MkObjectView from '@/components/MkObjectView.vue'; import MkTextarea from '@/components/MkTextarea.vue'; @@ -272,16 +306,25 @@ import MkPagination from '@/components/MkPagination.vue'; import MkInput from '@/components/MkInput.vue'; import MkNumber from '@/components/MkNumber.vue'; import { copyToClipboard } from '@/utility/copy-to-clipboard'; +import SkBadgeStrip from '@/components/SkBadgeStrip.vue'; const props = withDefaults(defineProps<{ userId: string; initialTab?: string; + userHint?: Misskey.entities.UserDetailed; + infoHint?: Misskey.entities.AdminShowUserResponse; + ipsHint?: Misskey.entities.AdminGetUserIpsResponse; + apHint?: Misskey.entities.ApGetResponse; }>(), { initialTab: 'overview', + userHint: undefined, + infoHint: undefined, + ipsHint: undefined, + apHint: undefined, }); const tab = ref(props.initialTab); -const chartSrc = ref('per-user-notes'); +const chartSrc = ref<ChartSrc>('per-user-notes'); const user = ref<null | Misskey.entities.UserDetailed>(); const init = ref<ReturnType<typeof createFetcher>>(); const info = ref<Misskey.entities.AdminShowUserResponse | null>(null); @@ -304,6 +347,98 @@ const filesPagination = { })), }; +const badges = computed(() => { + const arr: Badge[] = []; + if (info.value && user.value) { + if (info.value.isSuspended) { + arr.push({ + key: 'suspended', + label: i18n.ts.suspended, + style: 'error', + }); + } + + if (info.value.isSilenced) { + arr.push({ + key: 'silenced', + label: i18n.ts.silenced, + style: 'warning', + }); + } + + if (info.value.alwaysMarkNsfw) { + arr.push({ + key: 'nsfw', + label: i18n.ts.nsfw, + style: 'warning', + }); + } + + if (user.value.mandatoryCW) { + arr.push({ + key: 'cw', + label: i18n.ts.cw, + style: 'warning', + }); + } + + if (info.value.isHibernated) { + arr.push({ + key: 'hibernated', + label: i18n.ts.hibernated, + style: 'neutral', + }); + } + + if (info.value.isAdministrator) { + arr.push({ + key: 'admin', + label: i18n.ts.administrator, + style: 'success', + }); + } else if (info.value.isModerator) { + arr.push({ + key: 'mod', + label: i18n.ts.moderator, + style: 'success', + }); + } + + if (user.value.host == null) { + if (info.value.email) { + if (info.value.emailVerified) { + arr.push({ + key: 'verified', + label: i18n.ts.verified, + style: 'success', + }); + } else { + arr.push({ + key: 'not_verified', + label: i18n.ts.notVerified, + style: 'success', + }); + } + } + + if (info.value.approved) { + arr.push({ + key: 'approved', + label: i18n.ts.approved, + style: 'success', + }); + } else { + arr.push({ + key: 'not_approved', + label: i18n.ts.notApproved, + style: 'warning', + }); + } + } + } + return arr; +}); + const announcementsStatus = ref<'active' | 'archived'>('active'); const announcementsPagination = { @@ -314,47 +449,65 @@ const announcementsPagination = { status: announcementsStatus.value, })), }; -const expandedRoles = ref([]); +const expandedRoles = ref<string[]>([]); -function createFetcher() { - return () => Promise.all([misskeyApi('users/show', { - userId: props.userId, - }), misskeyApi('admin/show-user', { - userId: props.userId, - }), iAmAdmin ? misskeyApi('admin/get-user-ips', { - userId: props.userId, - }) : Promise.resolve(null)]).then(([_user, _info, _ips]) => { +function createFetcher(withHint = true) { + return () => Promise.all([ + (withHint && props.userHint) ? props.userHint : misskeyApi('users/show', { + userId: props.userId, + }), + (withHint && props.infoHint) ? props.infoHint : misskeyApi('admin/show-user', { + userId: props.userId, + }), + iAmAdmin + ? (withHint && props.ipsHint) ? props.ipsHint : misskeyApi('admin/get-user-ips', { + userId: props.userId, + }) + : null, + iAmAdmin + ? (withHint && props.apHint) ? props.apHint : misskeyApi('ap/get', { + userId: props.userId, + }).catch(() => null) : null], + ).then(async ([_user, _info, _ips, _ap]) => { user.value = _user; info.value = _info; ips.value = _ips; - moderator.value = info.value.isModerator; - silenced.value = info.value.isSilenced; - approved.value = info.value.approved; - markedAsNSFW.value = info.value.alwaysMarkNsfw; - suspended.value = info.value.isSuspended; - rejectQuotes.value = user.value.rejectQuotes ?? false; - moderationNote.value = info.value.moderationNote; - mandatoryCW.value = user.value.mandatoryCW; + ap.value = _ap; + moderator.value = _info.isModerator; + silenced.value = _info.isSilenced; + approved.value = _info.approved; + markedAsNSFW.value = _info.alwaysMarkNsfw; + suspended.value = _info.isSuspended; + rejectQuotes.value = _user.rejectQuotes ?? false; + moderationNote.value = _info.moderationNote; + mandatoryCW.value = _user.mandatoryCW; }); } -function refreshUser() { - init.value = createFetcher(); +async function refreshUser() { + // Not a typo - createFetcher() returns a function() + await createFetcher(false)(); } -async function onMandatoryCWChanged(value: string) { - await os.apiWithDialog('admin/cw-user', { userId: props.userId, cw: value }); - refreshUser(); +async function onMandatoryCWChanged(value: string | number) { + await os.promiseDialog(async () => { + await misskeyApi('admin/cw-user', { userId: props.userId, cw: String(value) }); + await refreshUser(); + }); } async function onModerationNoteChanged(value: string) { - await misskeyApi('admin/update-user-note', { userId: props.userId, text: value }); - refreshUser(); + await os.promiseDialog(async () => { + await misskeyApi('admin/update-user-note', { userId: props.userId, text: value }); + await refreshUser(); + }); } async function updateRemoteUser() { - await os.apiWithDialog('federation/update-remote-user', { userId: user.value.id }); - refreshUser(); + await os.promiseDialog(async () => { + await misskeyApi('federation/update-remote-user', { userId: props.userId }); + await refreshUser(); + }); } async function resetPassword() { @@ -366,9 +519,9 @@ async function resetPassword() { return; } else { const { password } = await misskeyApi('admin/reset-password', { - userId: user.value.id, + userId: props.userId, }); - os.alert({ + await os.alert({ type: 'success', text: i18n.tsx.newPasswordIs({ password }), }); @@ -383,7 +536,7 @@ async function toggleNSFW(v) { if (confirm.canceled) { markedAsNSFW.value = !v; } else { - await misskeyApi(v ? 'admin/nsfw-user' : 'admin/unnsfw-user', { userId: user.value.id }); + await misskeyApi(v ? 'admin/nsfw-user' : 'admin/unnsfw-user', { userId: props.userId }); await refreshUser(); } } @@ -396,8 +549,10 @@ async function toggleSilence(v) { if (confirm.canceled) { silenced.value = !v; } else { - await misskeyApi(v ? 'admin/silence-user' : 'admin/unsilence-user', { userId: user.value.id }); - await refreshUser(); + await os.promiseDialog(async () => { + await misskeyApi(v ? 'admin/silence-user' : 'admin/unsilence-user', { userId: props.userId }); + await refreshUser(); + }); } } @@ -409,8 +564,10 @@ async function toggleSuspend(v) { if (confirm.canceled) { suspended.value = !v; } else { - await misskeyApi(v ? 'admin/suspend-user' : 'admin/unsuspend-user', { userId: user.value.id }); - await refreshUser(); + await os.promiseDialog(async () => { + await misskeyApi(v ? 'admin/suspend-user' : 'admin/unsuspend-user', { userId: props.userId }); + await refreshUser(); + }); } } @@ -422,11 +579,13 @@ async function toggleRejectQuotes(v: boolean): Promise<void> { if (confirm.canceled) { rejectQuotes.value = !v; } else { - await misskeyApi('admin/reject-quotes', { - userId: props.userId, - rejectQuotes: v, + await os.promiseDialog(async () => { + await misskeyApi('admin/reject-quotes', { + userId: props.userId, + rejectQuotes: v, + }); + await refreshUser(); }); - await refreshUser(); } } @@ -436,17 +595,10 @@ async function unsetUserAvatar() { text: i18n.ts.unsetUserAvatarConfirm, }); if (confirm.canceled) return; - const process = async () => { - await misskeyApi('admin/unset-user-avatar', { userId: user.value.id }); - os.success(); - }; - await process().catch(err => { - os.alert({ - type: 'error', - text: err.toString(), - }); + await os.promiseDialog(async () => { + await misskeyApi('admin/unset-user-avatar', { userId: props.userId }); + await refreshUser(); }); - refreshUser(); } async function unsetUserBanner() { @@ -455,17 +607,10 @@ async function unsetUserBanner() { text: i18n.ts.unsetUserBannerConfirm, }); if (confirm.canceled) return; - const process = async () => { - await misskeyApi('admin/unset-user-banner', { userId: user.value.id }); - os.success(); - }; - await process().catch(err => { - os.alert({ - type: 'error', - text: err.toString(), - }); + await os.promiseDialog(async () => { + await misskeyApi('admin/unset-user-banner', { userId: props.userId }); + await refreshUser(); }); - refreshUser(); } async function deleteAllFiles() { @@ -474,17 +619,10 @@ async function deleteAllFiles() { text: i18n.ts.deleteAllFilesConfirm, }); if (confirm.canceled) return; - const process = async () => { - await misskeyApi('admin/delete-all-files-of-a-user', { userId: user.value.id }); - os.success(); - }; - await process().catch(err => { - os.alert({ - type: 'error', - text: err.toString(), - }); + await os.promiseDialog(async () => { + await misskeyApi('admin/delete-all-files-of-a-user', { userId: props.userId }); + await refreshUser(); }); - await refreshUser(); } async function deleteAccount() { @@ -493,18 +631,19 @@ async function deleteAccount() { text: i18n.ts.deleteThisAccountConfirm, }); if (confirm.canceled) return; + if (!user.value) return; const typed = await os.inputText({ - text: i18n.tsx.typeToConfirm({ x: user.value?.username }), + text: i18n.tsx.typeToConfirm({ x: user.value.username }), }); if (typed.canceled) return; - if (typed.result === user.value?.username) { + if (typed.result === user.value.username) { await os.apiWithDialog('admin/delete-account', { - userId: user.value.id, + userId: props.userId, }); } else { - os.alert({ + await os.alert({ type: 'error', text: 'input not match', }); @@ -544,23 +683,27 @@ async function assignRole() { : period === 'oneMonth' ? Date.now() + (1000 * 60 * 60 * 24 * 30) : null; - await os.apiWithDialog('admin/roles/assign', { roleId, userId: user.value.id, expiresAt }); - refreshUser(); + await os.promiseDialog(async () => { + await misskeyApi('admin/roles/assign', { roleId, userId: props.userId, expiresAt }); + await refreshUser(); + }); } async function unassignRole(role, ev) { - os.popupMenu([{ + await os.popupMenu([{ text: i18n.ts.unassign, icon: 'ti ti-x', danger: true, action: async () => { - await os.apiWithDialog('admin/roles/unassign', { roleId: role.id, userId: user.value.id }); - refreshUser(); + await os.promiseDialog(async () => { + await misskeyApi('admin/roles/unassign', { roleId: role.id, userId: props.userId }); + await refreshUser(); + }); }, }], ev.currentTarget ?? ev.target); } -function toggleRoleItem(role) { +function toggleRoleItem(role: Misskey.entities.Role) { if (expandedRoles.value.includes(role.id)) { expandedRoles.value = expandedRoles.value.filter(x => x !== role.id); } else { @@ -569,6 +712,7 @@ function toggleRoleItem(role) { } function createAnnouncement() { + if (!user.value) return; const { dispose } = os.popup(defineAsyncComponent(() => import('@/components/MkUserAnnouncementEditDialog.vue')), { user: user.value, }, { @@ -577,6 +721,7 @@ function createAnnouncement() { } function editAnnouncement(announcement) { + if (!user.value) return; const { dispose } = os.popup(defineAsyncComponent(() => import('@/components/MkUserAnnouncementEditDialog.vue')), { user: user.value, announcement, @@ -591,14 +736,6 @@ watch(() => props.userId, () => { immediate: true, }); -watch(user, () => { - misskeyApi('ap/get', { - uri: user.value.uri ?? `${url}/users/${user.value.id}`, - }).then(res => { - ap.value = res; - }); -}); - const headerActions = computed(() => []); const headerTabs = computed(() => isSystem.value ? [{ @@ -782,6 +919,7 @@ definePage(() => ({ cursor: pointer; } +// Sync with instance-info.vue .buttonStrip { margin: calc(var(--MI-margin) / 2 * -1); @@ -789,4 +927,13 @@ definePage(() => ({ margin: calc(var(--MI-margin) / 2); } } + +.rawFolderHeader { + display: flex; + flex-direction: row; + justify-content: space-between; + align-items: flex-start; + padding: var(--MI-marginHalf); + gap: var(--MI-marginHalf); +} </style> diff --git a/packages/frontend/src/pages/admin/abuses.vue b/packages/frontend/src/pages/admin/abuses.vue index 4ec4372492..a2343d7e76 100644 --- a/packages/frontend/src/pages/admin/abuses.vue +++ b/packages/frontend/src/pages/admin/abuses.vue @@ -48,9 +48,9 @@ SPDX-License-Identifier: AGPL-3.0-only --> <MkPagination v-slot="{items}" ref="reports" :pagination="pagination" :displayLimit="50"> - <div class="_gaps"> - <XAbuseReport v-for="report in items" :key="report.id" :report="report" @resolved="resolved"/> - </div> + <SkDateSeparatedList v-slot="{ item: report }" :items="items"> + <XAbuseReport :report="report" :metaHint="metaHint" @resolved="resolved"/> + </SkDateSeparatedList> </MkPagination> </div> </div> @@ -59,6 +59,7 @@ SPDX-License-Identifier: AGPL-3.0-only <script lang="ts" setup> import { computed, useTemplateRef, ref } from 'vue'; +import * as Misskey from 'misskey-js'; import MkSelect from '@/components/MkSelect.vue'; import MkPagination from '@/components/MkPagination.vue'; import XAbuseReport from '@/components/MkAbuseReport.vue'; @@ -67,6 +68,9 @@ import { definePage } from '@/page.js'; import MkButton from '@/components/MkButton.vue'; import MkInfo from '@/components/MkInfo.vue'; import { store } from '@/store.js'; +import SkDateSeparatedList from '@/components/SkDateSeparatedList.vue'; +import { iAmAdmin } from '@/i'; +import { misskeyApi } from '@/utility/misskey-api'; const reports = useTemplateRef('reports'); @@ -76,6 +80,14 @@ const targetUserOrigin = ref('combined'); const searchUsername = ref(''); const searchHost = ref(''); +const metaHint = ref<Misskey.entities.AdminMetaResponse | undefined>(undefined); + +if (iAmAdmin) { + misskeyApi('admin/meta') + .then(meta => metaHint.value = meta) + .catch(err => console.error('[MkAbuseReport] Error fetching meta:', err)); +} + const pagination = { endpoint: 'admin/abuse-user-reports' as const, limit: 10, diff --git a/packages/frontend/src/pages/chat/XMessage.vue b/packages/frontend/src/pages/chat/XMessage.vue index 1a80f6fef1..b72583214b 100644 --- a/packages/frontend/src/pages/chat/XMessage.vue +++ b/packages/frontend/src/pages/chat/XMessage.vue @@ -14,6 +14,7 @@ SPDX-License-Identifier: AGPL-3.0-only ref="text" class="_selectable" :text="message.text" + :parsedNotes="parsed" :i="$i" :nyaize="'respect'" :enableEmojiMenu="true" @@ -21,19 +22,19 @@ SPDX-License-Identifier: AGPL-3.0-only /> <MkMediaList v-if="message.file" :mediaList="[message.file]" :class="$style.file"/> </MkFukidashi> - <MkUrlPreview v-for="url in urls" :key="url" :url="url" :showAsQuote="!message.fromUser.rejectQuotes" style="margin: 8px 0;"/> + <SkUrlPreviewGroup :sourceNodes="parsed" :showAsQuote="!message.fromUser.rejectQuotes" style="margin: 8px 0;"/> <div :class="$style.footer"> <button class="_textButton" style="color: currentColor;" @click="showMenu"><i class="ti ti-dots-circle-horizontal"></i></button> <MkTime :class="$style.time" :time="message.createdAt"/> <MkA v-if="isSearchResult && 'toRoom' in message && message.toRoom != null" :to="`/chat/room/${message.toRoomId}`">{{ message.toRoom.name }}</MkA> <MkA v-if="isSearchResult && 'toUser' in message && message.toUser != null && isMe" :to="`/chat/user/${message.toUserId}`">@{{ message.toUser.username }}</MkA> </div> - <TransitionGroup - :enterActiveClass="prefer.s.animation ? $style.transition_reaction_enterActive : ''" - :leaveActiveClass="prefer.s.animation ? $style.transition_reaction_leaveActive : ''" - :enterFromClass="prefer.s.animation ? $style.transition_reaction_enterFrom : ''" - :leaveToClass="prefer.s.animation ? $style.transition_reaction_leaveTo : ''" - :moveClass="prefer.s.animation ? $style.transition_reaction_move : ''" + <SkTransitionGroup + :enterActiveClass="$style.transition_reaction_enterActive" + :leaveActiveClass="$style.transition_reaction_leaveActive" + :enterFromClass="$style.transition_reaction_enterFrom" + :leaveToClass="$style.transition_reaction_leaveTo" + :moveClass="$style.transition_reaction_move" tag="div" :class="$style.reactions" > <div v-for="record in message.reactions" :key="record.reaction + record.user.id" :class="[$style.reaction, record.user.id === $i.id ? $style.reactionMy : null]" @click="onReactionClick(record)"> @@ -45,21 +46,19 @@ SPDX-License-Identifier: AGPL-3.0-only :class="$style.reactionIcon" /> </div> - </TransitionGroup> + </SkTransitionGroup> </div> </div> </template> <script lang="ts" setup> import { computed, defineAsyncComponent, provide } from 'vue'; -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; import * as Misskey from 'misskey-js'; import { url } from '@@/js/config.js'; import { isLink } from '@@/js/is-link.js'; import type { MenuItem } from '@/types/menu.js'; import type { NormalizedChatMessage } from './room.vue'; -import { extractUrlFromMfm } from '@/utility/extract-url-from-mfm.js'; -import MkUrlPreview from '@/components/MkUrlPreview.vue'; import { ensureSignin } from '@/i.js'; import { misskeyApi } from '@/utility/misskey-api.js'; import { i18n } from '@/i18n.js'; @@ -73,6 +72,8 @@ import MkReactionIcon from '@/components/MkReactionIcon.vue'; import { prefer } from '@/preferences.js'; import { DI } from '@/di.js'; import { getHTMLElementOrNull } from '@/utility/get-dom-node-or-null.js'; +import SkTransitionGroup from '@/components/SkTransitionGroup.vue'; +import SkUrlPreviewGroup from '@/components/SkUrlPreviewGroup.vue'; const $i = ensureSignin(); @@ -82,7 +83,7 @@ const props = defineProps<{ }>(); const isMe = computed(() => props.message.fromUserId === $i.id); -const urls = computed(() => props.message.text ? extractUrlFromMfm(mfm.parse(props.message.text)) : []); +const parsed = computed(() => props.message.text ? mfm.parse(props.message.text) : []); provide(DI.mfmEmojiReactCallback, (reaction) => { if ($i.policies.chatAvailability !== 'available') return; diff --git a/packages/frontend/src/pages/chat/room.vue b/packages/frontend/src/pages/chat/room.vue index 5afda5682f..6505e172dd 100644 --- a/packages/frontend/src/pages/chat/room.vue +++ b/packages/frontend/src/pages/chat/room.vue @@ -31,12 +31,12 @@ SPDX-License-Identifier: AGPL-3.0-only <MkButton :class="$style.more" :wait="moreFetching" primary rounded @click="fetchMore">{{ i18n.ts.loadMore }}</MkButton> </div> - <TransitionGroup - :enterActiveClass="prefer.s.animation ? $style.transition_x_enterActive : ''" - :leaveActiveClass="prefer.s.animation ? $style.transition_x_leaveActive : ''" - :enterFromClass="prefer.s.animation ? $style.transition_x_enterFrom : ''" - :leaveToClass="prefer.s.animation ? $style.transition_x_leaveTo : ''" - :moveClass="prefer.s.animation ? $style.transition_x_move : ''" + <SkTransitionGroup + :enterActiveClass="$style.transition_x_enterActive" + :leaveActiveClass="$style.transition_x_leaveActive" + :enterFromClass="$style.transition_x_enterFrom" + :leaveToClass="$style.transition_x_leaveTo" + :moveClass="$style.transition_x_move" tag="div" class="_gaps" > <div v-for="item in timeline.toReversed()" :key="item.id"> @@ -47,7 +47,7 @@ SPDX-License-Identifier: AGPL-3.0-only <span>{{ item.prevText }} <i class="ti ti-chevron-down"></i></span> </div> </div> - </TransitionGroup> + </SkTransitionGroup> </div> <div v-if="user && (!user.canChat || user.host !== null)"> @@ -111,6 +111,7 @@ import { useRouter } from '@/router.js'; import { useMutationObserver } from '@/use/use-mutation-observer.js'; import MkInfo from '@/components/MkInfo.vue'; import { makeDateSeparatedTimelineComputedRef } from '@/utility/timeline-date-separate.js'; +import SkTransitionGroup from '@/components/SkTransitionGroup.vue'; const $i = ensureSignin(); const router = useRouter(); diff --git a/packages/frontend/src/pages/explore.featured.vue b/packages/frontend/src/pages/explore.featured.vue index a47e3efbc8..82badd40b3 100644 --- a/packages/frontend/src/pages/explore.featured.vue +++ b/packages/frontend/src/pages/explore.featured.vue @@ -10,27 +10,77 @@ SPDX-License-Identifier: AGPL-3.0-only <option value="polls">{{ i18n.ts.poll }}</option> </MkTab> <MkNotes v-if="tab === 'notes'" :pagination="paginationForNotes"/> - <MkNotes v-else-if="tab === 'polls'" :pagination="paginationForPolls"/> + <div v-else-if="tab === 'polls'"> + <template v-if="ltlAvailable || gtlAvailable"> + <MkFoldableSection v-if="ltlAvailable" class="_margin"> + <template #header><i class="ph-house ph-bold ph-lg" style="margin-right: 0.5em;"></i>{{ i18n.tsx.pollsOnLocal({ name: instance.name ?? host }) }}</template> + <MkNotes :pagination="paginationForPollsLocal" :disableAutoLoad="true"/> + </MkFoldableSection> + + <MkFoldableSection v-if="gtlAvailable" class="_margin"> + <template #header><i class="ph-globe ph-bold ph-lg" style="margin-right: 0.5em;"></i>{{ i18n.ts.pollsOnRemote }}</template> + <MkNotes :pagination="paginationForPollsRemote" :disableAutoLoad="true"/> + </MkFoldableSection> + + <MkFoldableSection v-if="gtlAvailable" class="_margin"> + <template #header><i class="ph-timer ph-bold ph-lg" style="margin-right: 0.5em;"></i>{{ i18n.ts.pollsExpired }}</template> + <MkNotes :pagination="paginationForPollsExpired" :disableAutoLoad="true"/> + </MkFoldableSection> + </template> + <template v-else> + <div v-if="$i"><i class="ti ti-alert-triangle"></i>{{ i18n.ts.trendingPollsDisabled }}</div> + <div v-else><i class="ti ti-alert-triangle"></i>{{ i18n.ts.trendingPollsDisabledLogIn }}</div> + </template> + </div> </div> </template> <script lang="ts" setup> -import { ref } from 'vue'; +import { computed, ref } from 'vue'; +import { host } from '@@/js/config.js'; import MkNotes from '@/components/MkNotes.vue'; import MkTab from '@/components/MkTab.vue'; import { i18n } from '@/i18n.js'; +import MkFoldableSection from '@/components/MkFoldableSection.vue'; +import { instance } from '@/instance.js'; +import { $i } from '@/i'; + +const ltlAvailable = computed(() => $i?.policies.ltlAvailable ?? instance.policies.ltlAvailable); +const gtlAvailable = computed(() => $i?.policies.gtlAvailable ?? instance.policies.gtlAvailable); const paginationForNotes = { endpoint: 'notes/featured' as const, limit: 10, }; -const paginationForPolls = { +const paginationForPollsLocal = { + endpoint: 'notes/polls/recommendation' as const, + limit: 10, + offsetMode: true, + params: { + excludeChannels: true, + local: true, + }, +}; + +const paginationForPollsRemote = { + endpoint: 'notes/polls/recommendation' as const, + limit: 10, + offsetMode: true, + params: { + excludeChannels: true, + local: false, + }, +}; + +const paginationForPollsExpired = { endpoint: 'notes/polls/recommendation' as const, limit: 10, offsetMode: true, params: { excludeChannels: true, + local: null, + expired: true, }, }; diff --git a/packages/frontend/src/pages/favorites.vue b/packages/frontend/src/pages/favorites.vue index aa18f44e88..4f74467871 100644 --- a/packages/frontend/src/pages/favorites.vue +++ b/packages/frontend/src/pages/favorites.vue @@ -15,6 +15,7 @@ SPDX-License-Identifier: AGPL-3.0-only </template> <template #default="{ items }"> + <!-- TODO replace with SkDateSeparatedList when merged --> <MkDateSeparatedList v-slot="{ item }" :items="items" :direction="'down'" :noGap="false" :ad="false"> <DynamicNote :key="item.id" :note="item.note" :class="$style.note"/> </MkDateSeparatedList> diff --git a/packages/frontend/src/pages/instance-info.vue b/packages/frontend/src/pages/instance-info.vue index 479774faef..28fd593893 100644 --- a/packages/frontend/src/pages/instance-info.vue +++ b/packages/frontend/src/pages/instance-info.vue @@ -4,99 +4,131 @@ SPDX-License-Identifier: AGPL-3.0-only --> <template> -<PageWithHeader v-model:tab="tab" :actions="headerActions" :tabs="headerTabs" :swipable="true"> - <div v-if="instance" class="_spacer" style="--MI_SPACER-w: 600px; --MI_SPACER-min: 16px; --MI_SPACER-max: 32px;"> - <MkSwiper v-model:tab="tab" :tabs="headerTabs"> - <div v-if="tab === 'overview'" class="_gaps_m"> +<PageWithHeader v-model:tab="tab" :actions="headerActions" :tabs="headerTabs" :spacer="true" style="--MI_SPACER-w: 700px; --MI_SPACER-min: 16px; --MI_SPACER-max: 32px;"> + <div v-if="instance"> + <!-- This empty div is preserved to avoid merge conflicts --> + <div> + <div v-if="tab === 'overview'" class="_gaps"> <div class="fnfelxur"> - <img :src="faviconUrl" alt="" class="icon"/> - <span class="name">{{ instance.name || `(${i18n.ts.unknown})` }}</span> - </div> - <div style="display: flex; flex-direction: column; gap: 1em;"> - <MkKeyValue :copy="host" oneline> - <template #key>Host</template> - <template #value><span class="_monospace"><MkLink :url="`https://${host}`">{{ host }}</MkLink></span></template> - </MkKeyValue> - <MkKeyValue oneline> - <template #key>{{ i18n.ts.software }}</template> - <template #value><span class="_monospace">{{ instance.softwareName || `(${i18n.ts.unknown})` }} / {{ instance.softwareVersion || `(${i18n.ts.unknown})` }}</span></template> - </MkKeyValue> - <MkKeyValue oneline> - <template #key>{{ i18n.ts.administrator }}</template> - <template #value>{{ instance.maintainerName || `(${i18n.ts.unknown})` }} ({{ instance.maintainerEmail || `(${i18n.ts.unknown})` }})</template> - </MkKeyValue> + <!-- TODO copy the alt text stuff from reports UI PR --> + <img v-if="faviconUrl" :src="faviconUrl" alt="" class="icon"/> + <div :class="$style.headerData"> + <span class="name">{{ instance.name || instance.host }}</span> + <span> + <span class="_monospace">{{ instance.host }}</span> + <button v-tooltip="i18n.ts.copy" class="_textButton" style="margin-left: 0.5em;" @click="copyToClipboard(instance.host)"><i class="ti ti-copy"></i></button> + </span> + <span> + <span class="_monospace">{{ instance.id }}</span> + <button v-tooltip="i18n.ts.copy" class="_textButton" style="margin-left: 0.5em;" @click="copyToClipboard(instance.id)"><i class="ti ti-copy"></i></button> + </span> + </div> </div> - <MkKeyValue> - <template #key>{{ i18n.ts.description }}</template> - <template #value>{{ instance.description }}</template> - </MkKeyValue> - <FormSection v-if="iAmModerator"> - <template #label>Moderation</template> - <div class="_gaps_s"> - <MkKeyValue> - <template #key> - {{ i18n.ts._delivery.status }} - </template> - <template #value> - {{ i18n.ts._delivery._type[suspensionState] }} - </template> + <SkBadgeStrip v-if="badges.length > 0" :badges="badges"></SkBadgeStrip> + + <MkFolder :sticky="false"> + <template #icon><i class="ph-list-bullets ph-bold ph-lg"></i></template> + <template #label>{{ i18n.ts.details }}</template> + <div style="display: flex; flex-direction: column; gap: 1em;"> + <MkKeyValue :copy="instance.id" oneline> + <template #key>{{ i18n.ts.id }}</template> + <template #value><span class="_monospace">{{ instance.id }}</span></template> </MkKeyValue> - <div class="_buttons"> - <MkButton inline :disabled="!instance" danger @click="deleteAllFiles">{{ i18n.ts.deleteAllFiles }}</MkButton> - <MkButton inline :disabled="!instance" danger @click="severAllFollowRelations">{{ i18n.ts.severAllFollowRelations }}</MkButton> - </div> + <MkKeyValue :copy="instance.name" oneline> + <template #key>{{ i18n.ts.name }}</template> + <template #value><span class="_monospace">{{ instance.name || `(${i18n.ts.unknown})` }}</span></template> + </MkKeyValue> + <MkKeyValue :copy="host" oneline> + <template #key>{{ i18n.ts.host }}</template> + <template #value><span class="_monospace"><MkLink :url="`https://${host}`">{{ host }}</MkLink></span></template> + </MkKeyValue> + <MkKeyValue :copy="instance.firstRetrievedAt" oneline> + <template #key>{{ i18n.ts.createdAt }}</template> + <template #value><span class="_monospace"><MkTime :time="instance.firstRetrievedAt" :mode="'detail'"/></span></template> + </MkKeyValue> + <MkKeyValue :copy="instance.infoUpdatedAt" oneline> + <template #key>{{ i18n.ts.updatedAt }}</template> + <template #value><span class="_monospace"><MkTime :time="instance.infoUpdatedAt" :mode="'detail'"/></span></template> + </MkKeyValue> + <MkKeyValue :copy="instance.latestRequestReceivedAt" oneline> + <template #key>{{ i18n.ts.lastActiveDate }}</template> + <template #value><span class="_monospace"><MkTime :time="instance.latestRequestReceivedAt" :mode="'detail'"/></span></template> + </MkKeyValue> + <MkKeyValue :copy="instance.softwareName" oneline> + <template #key>{{ i18n.ts.software }}</template> + <template #value><span class="_monospace">{{ instance.softwareName || `(${i18n.ts.unknown})` }} / {{ instance.softwareVersion || `(${i18n.ts.unknown})` }}</span></template> + </MkKeyValue> + <MkKeyValue :copy="instance.maintainerName" oneline> + <template #key>{{ i18n.ts.administrator }}</template> + <template #value><span class="_monospace">{{ instance.maintainerName || `(${i18n.ts.unknown})` }}</span></template> + </MkKeyValue> + <MkKeyValue :copy="instance.maintainerEmail" oneline> + <template #key>{{ i18n.ts.email }}</template> + <template #value><span class="_monospace">{{ instance.maintainerEmail || `(${i18n.ts.unknown})` }}</span></template> + </MkKeyValue> + <MkKeyValue oneline> + <template #key>{{ i18n.ts.followingPub }}</template> + <template #value><span class="_monospace"><MkNumber :value="instance.followingCount"/></span></template> + </MkKeyValue> + <MkKeyValue oneline> + <template #key>{{ i18n.ts.followersSub }}</template> + <template #value><span class="_monospace"><MkNumber :value="instance.followersCount"/></span></template> + </MkKeyValue> + <MkKeyValue oneline> + <template #key>{{ i18n.ts._delivery.status }}</template> + <template #value><span class="_monospace">{{ i18n.ts._delivery._type[suspensionState] }}</span></template> + </MkKeyValue> + </div> + </MkFolder> + + <MkFolder :sticky="false"> + <template #label>{{ i18n.ts.wellKnownResources }}</template> + <template #icon><i class="ph-network ph-bold ph-lg"></i></template> + <ul :class="$style.linksList" class="_gaps_s"> + <!-- TODO more links here --> + <li><MkLink :url="`https://${host}/.well-known/host-meta`" class="_monospace">/.well-known/host-meta</MkLink></li> + <li><MkLink :url="`https://${host}/.well-known/host-meta.json`" class="_monospace">/.well-known/host-meta.json</MkLink></li> + <li><MkLink :url="`https://${host}/.well-known/nodeinfo`" class="_monospace">/.well-known/nodeinfo</MkLink></li> + <li><MkLink :url="`https://${host}/robots.txt`" class="_monospace">/robots.txt</MkLink></li> + <li><MkLink :url="`https://${host}/manifest.json`" class="_monospace">/manifest.json</MkLink></li> + </ul> + </MkFolder> + + <MkFolder v-if="iAmModerator" :defaultOpen="moderationNote.length > 0" :sticky="false"> + <template #icon><i class="ph-stamp ph-bold ph-lg"></i></template> + <template #label>{{ i18n.ts.moderationNote }}</template> + <MkTextarea v-model="moderationNote" manualSave @update:modelValue="saveModerationNote"> + <template #label>{{ i18n.ts.moderationNote }}</template> + <template #caption>{{ i18n.ts.moderationNoteDescription }}</template> + </MkTextarea> + </MkFolder> + + <FormSection v-if="instance.description"> + <template #label>{{ i18n.ts.description }}</template> + {{ instance.description }} + </FormSection> + + <FormSection v-if="iAmModerator"> + <template #label>{{ i18n.ts.moderation }}</template> + <div class="_gaps"> + <MkInfo v-if="isBaseSilenced" warn>{{ i18n.ts.silencedByBase }}</MkInfo> + <MkSwitch v-model="isSilenced" :disabled="!meta || !instance || isBaseSilenced" @update:modelValue="toggleSilenced">{{ i18n.ts.silenceThisInstance }}</MkSwitch> <MkSwitch v-model="isSuspended" :disabled="!instance" @update:modelValue="toggleSuspended">{{ i18n.ts._delivery.stop }}</MkSwitch> <MkInfo v-if="isBaseBlocked" warn>{{ i18n.ts.blockedByBase }}</MkInfo> <MkSwitch v-model="isBlocked" :disabled="!meta || !instance || isBaseBlocked" @update:modelValue="toggleBlock">{{ i18n.ts.blockThisInstance }}</MkSwitch> - <MkInfo v-if="isBaseSilenced" warn>{{ i18n.ts.silencedByBase }}</MkInfo> - <MkSwitch v-model="isSilenced" :disabled="!meta || !instance || isBaseSilenced" @update:modelValue="toggleSilenced">{{ i18n.ts.silenceThisInstance }}</MkSwitch> - <MkSwitch v-model="isNSFW" :disabled="!instance" @update:modelValue="toggleNSFW">{{ i18n.ts.markInstanceAsNSFW }}</MkSwitch> <MkSwitch v-model="rejectQuotes" :disabled="!instance" @update:modelValue="toggleRejectQuotes">{{ i18n.ts.rejectQuotesInstance }}</MkSwitch> + <MkSwitch v-model="isNSFW" :disabled="!instance" @update:modelValue="toggleNSFW">{{ i18n.ts.markInstanceAsNSFW }}</MkSwitch> <MkSwitch v-model="rejectReports" :disabled="!instance" @update:modelValue="toggleRejectReports">{{ i18n.ts.rejectReports }}</MkSwitch> <MkInfo v-if="isBaseMediaSilenced" warn>{{ i18n.ts.mediaSilencedByBase }}</MkInfo> <MkSwitch v-model="isMediaSilenced" :disabled="!meta || !instance || isBaseMediaSilenced" @update:modelValue="toggleMediaSilenced">{{ i18n.ts.mediaSilenceThisInstance }}</MkSwitch> - <MkButton @click="refreshMetadata"><i class="ti ti-refresh"></i> Refresh metadata</MkButton> - <MkTextarea v-model="moderationNote" manualSave> - <template #label>{{ i18n.ts.moderationNote }}</template> - <template #caption>{{ i18n.ts.moderationNoteDescription }}</template> - </MkTextarea> - </div> - </FormSection> - - <FormSection> - <MkKeyValue oneline style="margin: 1em 0;"> - <template #key>{{ i18n.ts.registeredAt }}</template> - <template #value><MkTime mode="detail" :time="instance.firstRetrievedAt"/></template> - </MkKeyValue> - <MkKeyValue oneline style="margin: 1em 0;"> - <template #key>{{ i18n.ts.updatedAt }}</template> - <template #value><MkTime mode="detail" :time="instance.infoUpdatedAt"/></template> - </MkKeyValue> - <MkKeyValue oneline style="margin: 1em 0;"> - <template #key>{{ i18n.ts.latestRequestReceivedAt }}</template> - <template #value><MkTime v-if="instance.latestRequestReceivedAt" :time="instance.latestRequestReceivedAt"/><span v-else>N/A</span></template> - </MkKeyValue> - </FormSection> - - <FormSection> - <MkKeyValue oneline style="margin: 1em 0;"> - <template #key>Following (Pub)</template> - <template #value>{{ number(instance.followingCount) }}</template> - </MkKeyValue> - <MkKeyValue oneline style="margin: 1em 0;"> - <template #key>Followers (Sub)</template> - <template #value>{{ number(instance.followersCount) }}</template> - </MkKeyValue> - </FormSection> - <FormSection> - <template #label>Well-known resources</template> - <FormLink :to="`https://${host}/.well-known/host-meta`" external style="margin-bottom: 8px;">host-meta</FormLink> - <FormLink :to="`https://${host}/.well-known/host-meta.json`" external style="margin-bottom: 8px;">host-meta.json</FormLink> - <FormLink :to="`https://${host}/.well-known/nodeinfo`" external style="margin-bottom: 8px;">nodeinfo</FormLink> - <FormLink :to="`https://${host}/robots.txt`" external style="margin-bottom: 8px;">robots.txt</FormLink> - <FormLink :to="`https://${host}/manifest.json`" external style="margin-bottom: 8px;">manifest.json</FormLink> + <div :class="$style.buttonStrip"> + <MkButton inline :disabled="!instance" @click="refreshMetadata"><i class="ph-cloud-arrow-down ph-bold ph-lg"></i> {{ i18n.ts.updateRemoteUser }}</MkButton> + <MkButton inline :disabled="!instance" danger @click="deleteAllFiles"><i class="ph-trash ph-bold ph-lg"></i> {{ i18n.ts.deleteAllFiles }}</MkButton> + <MkButton inline :disabled="!instance" danger @click="severAllFollowRelations"><i class="ph-link-break ph-bold ph-lg"></i> {{ i18n.ts.severAllFollowRelations }}</MkButton> + </div> + </div> </FormSection> </div> <div v-else-if="tab === 'chart'" class="_gaps_m"> @@ -126,7 +158,7 @@ SPDX-License-Identifier: AGPL-3.0-only </div> <div v-else-if="tab === 'users'" class="_gaps_m"> <MkPagination v-slot="{items}" :pagination="usersPagination" style="display: grid; grid-template-columns: repeat(auto-fill,minmax(270px,1fr)); grid-gap: 12px;"> - <MkA v-for="user in items" :key="user.id" v-tooltip.mfm="`Last posted: ${dateString(user.updatedAt)}`" class="user" :to="`/admin/user/${user.id}`"> + <MkA v-for="user in items" :key="user.id" v-tooltip.mfm="i18n.tsx.lastPosted({ at: dateString(user.updatedAt) })" class="user" :to="`/admin/user/${user.id}`"> <MkUserCardMini :user="user"/> </MkA> </MkPagination> @@ -135,11 +167,11 @@ SPDX-License-Identifier: AGPL-3.0-only <MkPagination v-slot="{items}" :pagination="followingPagination"> <div class="follow-relations-list"> <div v-for="followRelationship in items" :key="followRelationship.id" class="follow-relation"> - <MkA v-tooltip.mfm="`Last posted: ${dateString(followRelationship.followee.updatedAt)}`" :to="`/admin/user/${followRelationship.followee.id}`" class="user"> + <MkA v-tooltip.mfm="i18n.tsx.lastPosted({ at: dateString(followRelationship.followee.updatedAt) })" :to="`/admin/user/${followRelationship.followee.id}`" class="user"> <MkUserCardMini :user="followRelationship.followee" :withChart="false"/> </MkA> <span class="arrow">→</span> - <MkA v-tooltip.mfm="`Last posted: ${dateString(followRelationship.follower.updatedAt)}`" :to="`/admin/user/${followRelationship.follower.id}`" class="user"> + <MkA v-tooltip.mfm="i18n.tsx.lastPosted({ at: dateString(followRelationship.follower.updatedAt) })" :to="`/admin/user/${followRelationship.follower.id}`" class="user"> <MkUserCardMini :user="followRelationship.follower" :withChart="false"/> </MkA> </div> @@ -150,11 +182,11 @@ SPDX-License-Identifier: AGPL-3.0-only <MkPagination v-slot="{items}" :pagination="followersPagination"> <div class="follow-relations-list"> <div v-for="followRelationship in items" :key="followRelationship.id" class="follow-relation"> - <MkA v-tooltip.mfm="`Last posted: ${dateString(followRelationship.followee.updatedAt)}`" :to="`/admin/user/${followRelationship.followee.id}`" class="user"> + <MkA v-tooltip.mfm="i18n.tsx.lastPosted({ at: dateString(followRelationship.followee.updatedAt) })" :to="`/admin/user/${followRelationship.followee.id}`" class="user"> <MkUserCardMini :user="followRelationship.followee" :withChart="false"/> </MkA> <span class="arrow">←</span> - <MkA v-tooltip.mfm="`Last posted: ${dateString(followRelationship.follower.updatedAt)}`" :to="`/admin/user/${followRelationship.follower.id}`" class="user"> + <MkA v-tooltip.mfm="i18n.tsx.lastPosted({ at: dateString(followRelationship.follower.updatedAt) })" :to="`/admin/user/${followRelationship.follower.id}`" class="user"> <MkUserCardMini :user="followRelationship.follower" :withChart="false"/> </MkA> </div> @@ -165,16 +197,17 @@ SPDX-License-Identifier: AGPL-3.0-only <MkObjectView tall :value="instance"> </MkObjectView> </div> - </MkSwiper> + </div> </div> </PageWithHeader> </template> <script lang="ts" setup> -import { ref, computed, watch } from 'vue'; +import { ref, computed } from 'vue'; import * as Misskey from 'misskey-js'; import type { ChartSrc } from '@/components/MkChart.vue'; import type { Paging } from '@/components/MkPagination.vue'; +import type { Badge } from '@/components/SkBadgeStrip.vue'; import MkChart from '@/components/MkChart.vue'; import MkObjectView from '@/components/MkObjectView.vue'; import FormLink from '@/components/form/link.vue'; @@ -197,10 +230,19 @@ import { dateString } from '@/filters/date.js'; import MkTextarea from '@/components/MkTextarea.vue'; import MkInfo from '@/components/MkInfo.vue'; import { $i } from '@/i.js'; +import { copyToClipboard } from '@/utility/copy-to-clipboard'; +import MkFolder from '@/components/MkFolder.vue'; +import MkNumber from '@/components/MkNumber.vue'; +import SkBadgeStrip from '@/components/SkBadgeStrip.vue'; -const props = defineProps<{ +const props = withDefaults(defineProps<{ host: string; -}>(); + metaHint?: Misskey.entities.AdminMetaResponse; + instanceHint?: Misskey.entities.FederationInstance; +}>(), { + metaHint: undefined, + instanceHint: undefined, +}); const tab = ref('overview'); @@ -233,6 +275,55 @@ const isBaseBlocked = computed(() => meta.value && baseDomains.value.some(d => m const isBaseSilenced = computed(() => meta.value && baseDomains.value.some(d => meta.value?.silencedHosts.includes(d))); const isBaseMediaSilenced = computed(() => meta.value && baseDomains.value.some(d => meta.value?.mediaSilencedHosts.includes(d))); +const badges = computed(() => { + const arr: Badge[] = []; + if (instance.value) { + if (instance.value.isBlocked) { + arr.push({ + key: 'blocked', + label: i18n.ts.blocked, + style: 'error', + }); + } + if (instance.value.isSuspended) { + arr.push({ + key: 'suspended', + label: i18n.ts.suspended, + style: 'error', + }); + } + if (instance.value.isSilenced) { + arr.push({ + key: 'silenced', + label: i18n.ts.silenced, + style: 'warning', + }); + } + if (instance.value.isMediaSilenced) { + arr.push({ + key: 'media_silenced', + label: i18n.ts.mediaSilenced, + style: 'warning', + }); + } + if (instance.value.isNSFW) { + arr.push({ + key: 'nsfw', + label: i18n.ts.nsfw, + style: 'warning', + }); + } + if (instance.value.isBubbled) { + arr.push({ + key: 'bubbled', + label: i18n.ts.bubble, + style: 'success', + }); + } + } + return arr; +}); + const usersPagination = { endpoint: iAmModerator ? 'admin/show-users' : 'users', limit: 10, @@ -264,20 +355,30 @@ const followersPagination = { offsetMode: false, }; -if (iAmModerator) { - watch(moderationNote, async () => { - if (instance.value == null) return; - await misskeyApi('admin/federation/update-instance', { host: instance.value.host, moderationNote: moderationNote.value }); - }); +async function saveModerationNote() { + if (iAmModerator) { + await os.promiseDialog(async () => { + if (instance.value == null) return; + await os.apiWithDialog('admin/federation/update-instance', { host: instance.value.host, moderationNote: moderationNote.value }); + await fetch(); + }); + } } -async function fetch(): Promise<void> { - if (iAmAdmin) { - meta.value = await misskeyApi('admin/meta'); - } - instance.value = await misskeyApi('federation/show-instance', { - host: props.host, - }); +async function fetch(withHint = false): Promise<void> { + const [m, i] = await Promise.all([ + (withHint && props.metaHint) + ? props.metaHint + : iAmAdmin ? misskeyApi('admin/meta') : null, + (withHint && props.instanceHint) + ? props.instanceHint + : misskeyApi('federation/show-instance', { + host: props.host, + }), + ]); + meta.value = m; + instance.value = i; + suspensionState.value = instance.value?.suspensionState ?? 'none'; isSuspended.value = suspensionState.value !== 'none'; isBlocked.value = instance.value?.isBlocked ?? false; @@ -292,80 +393,106 @@ async function fetch(): Promise<void> { async function toggleBlock(): Promise<void> { if (!iAmAdmin) return; - if (!meta.value) throw new Error('No meta?'); - if (!instance.value) throw new Error('No instance?'); - const { host } = instance.value; - await misskeyApi('admin/update-meta', { - blockedHosts: isBlocked.value ? meta.value.blockedHosts.concat([host]) : meta.value.blockedHosts.filter(x => x !== host), + await os.promiseDialog(async () => { + if (!meta.value) throw new Error('No meta?'); + if (!instance.value) throw new Error('No instance?'); + const { host } = instance.value; + await os.apiWithDialog('admin/update-meta', { + blockedHosts: isBlocked.value ? meta.value.blockedHosts.concat([host]) : meta.value.blockedHosts.filter(x => x !== host), + }); + await fetch(); }); } async function toggleSilenced(): Promise<void> { if (!iAmAdmin) return; - if (!meta.value) throw new Error('No meta?'); - if (!instance.value) throw new Error('No instance?'); - const { host } = instance.value; - const silencedHosts = meta.value.silencedHosts ?? []; - await misskeyApi('admin/update-meta', { - silencedHosts: isSilenced.value ? silencedHosts.concat([host]) : silencedHosts.filter(x => x !== host), + await os.promiseDialog(async () => { + if (!meta.value) throw new Error('No meta?'); + if (!instance.value) throw new Error('No instance?'); + const { host } = instance.value; + const silencedHosts = meta.value.silencedHosts ?? []; + await os.promiseDialog(async () => { + await misskeyApi('admin/update-meta', { + silencedHosts: isSilenced.value ? silencedHosts.concat([host]) : silencedHosts.filter(x => x !== host), + }); + await fetch(); + }); }); } async function toggleMediaSilenced(): Promise<void> { if (!iAmAdmin) return; - if (!meta.value) throw new Error('No meta?'); - if (!instance.value) throw new Error('No instance?'); - const { host } = instance.value; - const mediaSilencedHosts = meta.value.mediaSilencedHosts ?? []; - await misskeyApi('admin/update-meta', { - mediaSilencedHosts: isMediaSilenced.value ? mediaSilencedHosts.concat([host]) : mediaSilencedHosts.filter(x => x !== host), + await os.promiseDialog(async () => { + if (!meta.value) throw new Error('No meta?'); + if (!instance.value) throw new Error('No instance?'); + const { host } = instance.value; + const mediaSilencedHosts = meta.value.mediaSilencedHosts ?? []; + await misskeyApi('admin/update-meta', { + mediaSilencedHosts: isMediaSilenced.value ? mediaSilencedHosts.concat([host]) : mediaSilencedHosts.filter(x => x !== host), + }); + await fetch(); }); } async function toggleSuspended(): Promise<void> { if (!iAmModerator) return; - if (!instance.value) throw new Error('No instance?'); - suspensionState.value = isSuspended.value ? 'manuallySuspended' : 'none'; - await misskeyApi('admin/federation/update-instance', { - host: instance.value.host, - isSuspended: isSuspended.value, + await os.promiseDialog(async () => { + if (!instance.value) throw new Error('No instance?'); + suspensionState.value = isSuspended.value ? 'manuallySuspended' : 'none'; + await misskeyApi('admin/federation/update-instance', { + host: instance.value.host, + isSuspended: isSuspended.value, + }); + await fetch(); }); } async function toggleNSFW(): Promise<void> { if (!iAmModerator) return; - if (!instance.value) throw new Error('No instance?'); - await misskeyApi('admin/federation/update-instance', { - host: instance.value.host, - isNSFW: isNSFW.value, + await os.promiseDialog(async () => { + if (!instance.value) throw new Error('No instance?'); + await misskeyApi('admin/federation/update-instance', { + host: instance.value.host, + isNSFW: isNSFW.value, + }); + await fetch(); }); } async function toggleRejectReports(): Promise<void> { if (!iAmModerator) return; - if (!instance.value) throw new Error('No instance?'); - await misskeyApi('admin/federation/update-instance', { - host: instance.value.host, - rejectReports: rejectReports.value, + await os.promiseDialog(async () => { + if (!instance.value) throw new Error('No instance?'); + await misskeyApi('admin/federation/update-instance', { + host: instance.value.host, + rejectReports: rejectReports.value, + }); + await fetch(); }); } async function toggleRejectQuotes(): Promise<void> { if (!iAmModerator) return; - if (!instance.value) throw new Error('No instance?'); - await misskeyApi('admin/federation/update-instance', { - host: instance.value.host, - rejectQuotes: rejectQuotes.value, + await os.promiseDialog(async () => { + if (!instance.value) throw new Error('No instance?'); + await misskeyApi('admin/federation/update-instance', { + host: instance.value.host, + rejectQuotes: rejectQuotes.value, + }); + await fetch(); }); } -function refreshMetadata(): void { +async function refreshMetadata(): Promise<void> { if (!iAmModerator) return; - if (!instance.value) throw new Error('No instance?'); - misskeyApi('admin/federation/refresh-remote-instance-metadata', { - host: instance.value.host, + await os.promiseDialog(async () => { + if (!instance.value) throw new Error('No instance?'); + await misskeyApi('admin/federation/refresh-remote-instance-metadata', { + host: instance.value.host, + }); + await fetch(); }); - os.alert({ + await os.alert({ text: 'Refresh requested', }); } @@ -380,14 +507,12 @@ async function deleteAllFiles(): Promise<void> { }); if (confirm.canceled) return; - await Promise.all([ - misskeyApi('admin/federation/delete-all-files', { - host: instance.value.host, - }), - os.alert({ - text: i18n.ts.deleteAllFilesQueued, - }), - ]); + await os.apiWithDialog('admin/federation/delete-all-files', { + host: instance.value.host, + }); + await os.alert({ + text: i18n.ts.deleteAllFilesQueued, + }); } async function severAllFollowRelations(): Promise<void> { @@ -404,17 +529,15 @@ async function severAllFollowRelations(): Promise<void> { }); if (confirm.canceled) return; - await Promise.all([ - misskeyApi('admin/federation/remove-all-following', { - host: instance.value.host, - }), - os.alert({ - text: i18n.tsx.severAllFollowRelationsQueued({ host: instance.value.host }), - }), - ]); + await os.apiWithDialog('admin/federation/remove-all-following', { + host: instance.value.host, + }); + await os.alert({ + text: i18n.tsx.severAllFollowRelationsQueued({ host: instance.value.host }), + }); } -fetch(); +fetch(true); const headerActions = computed(() => [{ text: `https://${props.host}`, @@ -429,16 +552,16 @@ const headerTabs = computed(() => [{ title: i18n.ts.overview, icon: 'ti ti-info-circle', }, { - key: 'chart', - title: i18n.ts.charts, - icon: 'ti ti-chart-line', -}, { key: 'users', title: i18n.ts.users, icon: 'ti ti-users', }, ...getFollowingTabs(), { + key: 'chart', + title: i18n.ts.charts, + icon: 'ti ti-chart-line', +}, { key: 'raw', - title: 'Raw', + title: i18n.ts.raw, icon: 'ti ti-code', }]); @@ -522,3 +645,38 @@ definePage(() => ({ } } </style> + +<style lang="scss" module> +.headerData { + display: flex; + flex-direction: column; + + > * { + overflow: hidden; + text-overflow: ellipsis; + font-size: 85%; + opacity: 0.7; + } + + > :first-child { + text-overflow: initial; + word-break: break-all; + font-size: 100%; + opacity: 1.0; + } +} + +.linksList { + margin: 0; + padding-left: 1.5em; +} + +// Sync with admin-user.vue +.buttonStrip { + margin: calc(var(--MI-margin) / 2 * -1); + + >* { + margin: calc(var(--MI-margin) / 2); + } +} +</style> diff --git a/packages/frontend/src/pages/page-editor/els/page-editor.el.note.vue b/packages/frontend/src/pages/page-editor/els/page-editor.el.note.vue index f275ec9517..7d56743967 100644 --- a/packages/frontend/src/pages/page-editor/els/page-editor.el.note.vue +++ b/packages/frontend/src/pages/page-editor/els/page-editor.el.note.vue @@ -25,6 +25,7 @@ SPDX-License-Identifier: AGPL-3.0-only /* eslint-disable vue/no-mutating-props */ import { watch, ref } from 'vue'; import * as Misskey from 'misskey-js'; +import { retryOnThrottled } from '@@/js/retry-on-throttled.js'; import XContainer from '../page-editor.container.vue'; import MkInput from '@/components/MkInput.vue'; import MkSwitch from '@/components/MkSwitch.vue'; @@ -35,6 +36,7 @@ import { i18n } from '@/i18n.js'; const props = defineProps<{ modelValue: Misskey.entities.PageBlock & { type: 'note' }; + index: number; }>(); const emit = defineEmits<{ @@ -58,7 +60,13 @@ watch(id, async () => { ...props.modelValue, note: id.value, }); - note.value = await misskeyApi('notes/show', { noteId: id.value }); + const timeoutId = window.setTimeout(async () => { + note.value = await retryOnThrottled(() => misskeyApi('notes/show', { noteId: id.value })); + }, 500 * props.index); // rate limit is 2 reqs per sec + + return () => { + window.clearTimeout(timeoutId); + }; }, { immediate: true, }); diff --git a/packages/frontend/src/pages/page-editor/page-editor.blocks.vue b/packages/frontend/src/pages/page-editor/page-editor.blocks.vue index f191320180..8d7ba1a3ab 100644 --- a/packages/frontend/src/pages/page-editor/page-editor.blocks.vue +++ b/packages/frontend/src/pages/page-editor/page-editor.blocks.vue @@ -5,10 +5,16 @@ SPDX-License-Identifier: AGPL-3.0-only <template> <Sortable :modelValue="modelValue" tag="div" itemKey="id" handle=".drag-handle" :group="{ name: 'blocks' }" :animation="150" :swapThreshold="0.5" @update:modelValue="v => emit('update:modelValue', v)"> - <template #item="{element}"> + <template #item="{element, index}"> <div :class="$style.item"> <!-- divが無いとエラーになる https://github.com/SortableJS/vue.draggable.next/issues/189 --> - <component :is="getComponent(element.type)" :modelValue="element" @update:modelValue="updateItem" @remove="() => removeItem(element)"/> + <component + :is="getComponent(element.type)" + :modelValue="element" + :index="index" + @update:modelValue="updateItem" + @remove="() => removeItem(element)" + /> </div> </template> </Sortable> diff --git a/packages/frontend/src/pages/page.vue b/packages/frontend/src/pages/page.vue index 59b1a5a137..f4d0f25734 100644 --- a/packages/frontend/src/pages/page.vue +++ b/packages/frontend/src/pages/page.vue @@ -347,7 +347,7 @@ definePage(() => ({ text-align: center; border-radius: 99rem; - & :global(.ti) { + & :global(.ti), & :global(.ph-lg) { line-height: 2.5rem; } diff --git a/packages/frontend/src/pages/reversi/game.board.vue b/packages/frontend/src/pages/reversi/game.board.vue index c0c90cb993..1c1adaf687 100644 --- a/packages/frontend/src/pages/reversi/game.board.vue +++ b/packages/frontend/src/pages/reversi/game.board.vue @@ -243,13 +243,13 @@ if (game.value.isStarted && !game.value.isEnded) { useInterval(() => { if (game.value.isEnded) return; const crc32 = engine.value.calcCrc32(); - if (_DEV_) console.log('crc32', crc32); + if (_DEV_) console.debug('crc32', crc32); misskeyApi('reversi/verify', { gameId: game.value.id, crc32: crc32.toString(), }).then((res) => { if (res.desynced) { - if (_DEV_) console.log('resynced'); + if (_DEV_) console.debug('resynced'); restoreGame(res.game!); } }); diff --git a/packages/frontend/src/pages/settings/custom-css.vue b/packages/frontend/src/pages/settings/custom-css.vue index 9b0e04860e..5b9b0d897a 100644 --- a/packages/frontend/src/pages/settings/custom-css.vue +++ b/packages/frontend/src/pages/settings/custom-css.vue @@ -22,24 +22,9 @@ import { unisonReload } from '@/utility/unison-reload.js'; import { i18n } from '@/i18n.js'; import { definePage } from '@/page.js'; import { miLocalStorage } from '@/local-storage.js'; +import { prefer } from '@/preferences.js'; -const localCustomCss = ref(miLocalStorage.getItem('customCss') ?? ''); - -async function apply() { - miLocalStorage.setItem('customCss', localCustomCss.value); - - const { canceled } = await os.confirm({ - type: 'info', - text: i18n.ts.reloadToApplySetting, - }); - if (canceled) return; - - unisonReload(); -} - -watch(localCustomCss, async () => { - await apply(); -}); +const localCustomCss = prefer.model('customCss'); const headerActions = computed(() => []); diff --git a/packages/frontend/src/pages/settings/mute-block.instance-mute.vue b/packages/frontend/src/pages/settings/mute-block.instance-mute.vue index a0a40e4c72..164179d21c 100644 --- a/packages/frontend/src/pages/settings/mute-block.instance-mute.vue +++ b/packages/frontend/src/pages/settings/mute-block.instance-mute.vue @@ -15,36 +15,50 @@ SPDX-License-Identifier: AGPL-3.0-only </template> <script lang="ts" setup> -import { ref, watch } from 'vue'; +import { ref, watch, computed } from 'vue'; import MkTextarea from '@/components/MkTextarea.vue'; import MkInfo from '@/components/MkInfo.vue'; import MkButton from '@/components/MkButton.vue'; import { ensureSignin } from '@/i.js'; import { misskeyApi } from '@/utility/misskey-api.js'; import { i18n } from '@/i18n.js'; +import * as os from '@/os.js'; const $i = ensureSignin(); const instanceMutes = ref($i.mutedInstances.join('\n')); +const domainArray = computed(() => { + return instanceMutes.value + .trim().split('\n') + .map(el => el.trim().toLowerCase()) + .filter(el => el); +}); const changed = ref(false); async function save() { - let mutes = instanceMutes.value - .trim().split('\n') - .map(el => el.trim()) - .filter(el => el); + // checks for a full line without whitespace. + if (!domainArray.value.every(d => /^\S+$/.test(d))) { + os.alert({ + type: 'error', + title: i18n.ts.invalidValue, + }); + return; + } await misskeyApi('i/update', { - mutedInstances: mutes, + mutedInstances: domainArray.value, }); - changed.value = false; - // Refresh filtered list to signal to the user how they've been saved - instanceMutes.value = mutes.join('\n'); + instanceMutes.value = domainArray.value.join('\n'); + + changed.value = false; } -watch(instanceMutes, () => { - changed.value = true; +watch(domainArray, (newArray, oldArray) => { + // compare arrays + if (newArray.length !== oldArray.length || !newArray.every((a, i) => a === oldArray[i])) { + changed.value = true; + } }); </script> diff --git a/packages/frontend/src/pages/settings/mute-block.vue b/packages/frontend/src/pages/settings/mute-block.vue index 8cc3945df8..e19d7eff85 100644 --- a/packages/frontend/src/pages/settings/mute-block.vue +++ b/packages/frontend/src/pages/settings/mute-block.vue @@ -12,10 +12,11 @@ SPDX-License-Identifier: AGPL-3.0-only <div class="_gaps_s"> <SearchMarker + v-slot="slotProps" :label="i18n.ts.wordMute" :keywords="['note', 'word', 'soft', 'mute', 'hide']" > - <MkFolder> + <MkFolder :defaultOpen="slotProps.isParentOfTarget"> <template #icon><i class="ph-envelope ph-bold ph-lg"></i></template> <template #label>{{ i18n.ts.wordMute }}</template> @@ -37,10 +38,11 @@ SPDX-License-Identifier: AGPL-3.0-only </SearchMarker> <SearchMarker + v-slot="slotProps" :label="i18n.ts.hardWordMute" :keywords="['note', 'word', 'hard', 'mute', 'hide']" > - <MkFolder> + <MkFolder :defaultOpen="slotProps.isParentOfTarget"> <template #icon><i class="ph-x-square ph-bold ph-lg"></i></template> <template #label>{{ i18n.ts.hardWordMute }}</template> @@ -55,10 +57,11 @@ SPDX-License-Identifier: AGPL-3.0-only </SearchMarker> <SearchMarker + v-slot="slotProps" :label="i18n.ts.instanceMute" :keywords="['note', 'server', 'instance', 'host', 'federation', 'mute', 'hide']" > - <MkFolder v-if="instance.federation !== 'none'"> + <MkFolder v-if="instance.federation !== 'none'" :defaultOpen="slotProps.isParentOfTarget"> <template #icon><i class="ti ti-planet-off"></i></template> <template #label>{{ i18n.ts.instanceMute }}</template> @@ -67,9 +70,10 @@ SPDX-License-Identifier: AGPL-3.0-only </SearchMarker> <SearchMarker + v-slot="slotProps" :keywords="['renote', 'mute', 'hide', 'user']" > - <MkFolder> + <MkFolder :defaultOpen="slotProps.isParentOfTarget"> <template #icon><i class="ti ti-repeat-off"></i></template> <template #label><SearchLabel>{{ i18n.ts.mutedUsers }} ({{ i18n.ts.renote }})</SearchLabel></template> @@ -102,10 +106,11 @@ SPDX-License-Identifier: AGPL-3.0-only </SearchMarker> <SearchMarker + v-slot="slotProps" :label="i18n.ts.mutedUsers" :keywords="['note', 'mute', 'hide', 'user']" > - <MkFolder> + <MkFolder :defaultOpen="slotProps.isParentOfTarget"> <template #icon><i class="ti ti-eye-off"></i></template> <template #label>{{ i18n.ts.mutedUsers }}</template> @@ -140,10 +145,11 @@ SPDX-License-Identifier: AGPL-3.0-only </SearchMarker> <SearchMarker + v-slot="slotProps" :label="i18n.ts.blockedUsers" :keywords="['block', 'user']" > - <MkFolder> + <MkFolder :defaultOpen="slotProps.isParentOfTarget"> <template #icon><i class="ti ti-ban"></i></template> <template #label>{{ i18n.ts.blockedUsers }}</template> @@ -223,12 +229,6 @@ const expandedBlockItems = ref([]); const showSoftWordMutedWord = prefer.model('showSoftWordMutedWord'); -watch([ - showSoftWordMutedWord, -], async () => { - await reloadAsk({ reason: i18n.ts.reloadToApplySetting, unison: true }); -}); - async function unrenoteMute(user, ev) { os.popupMenu([{ text: i18n.ts.renoteUnmute, diff --git a/packages/frontend/src/pages/settings/preferences.vue b/packages/frontend/src/pages/settings/preferences.vue index b85f45884d..84c625b502 100644 --- a/packages/frontend/src/pages/settings/preferences.vue +++ b/packages/frontend/src/pages/settings/preferences.vue @@ -99,7 +99,7 @@ SPDX-License-Identifier: AGPL-3.0-only </SearchMarker> </div> - <SearchMarker :keywords="['emoji', 'style', 'native', 'system', 'fluent', 'twemoji']"> + <SearchMarker :keywords="['emoji', 'style', 'native', 'system', 'fluent', 'twemoji', 'tossface']"> <MkPreferenceContainer k="emojiStyle"> <div> <MkRadios v-model="emojiStyle"> @@ -107,6 +107,7 @@ SPDX-License-Identifier: AGPL-3.0-only <option value="native">{{ i18n.ts.native }}</option> <option value="fluentEmoji">Fluent Emoji</option> <option value="twemoji">Twemoji</option> + <option value="tossface">Tossface</option> </MkRadios> <div style="margin: 8px 0 0 0; font-size: 1.5em;"><Mfm :key="emojiStyle" text="🍮🍦🍭🍩🍰🍫🍬🥞🍪"/></div> </div> @@ -237,6 +238,13 @@ SPDX-License-Identifier: AGPL-3.0-only <!-- If one of the other options is selected show this as a blank other --> <option v-if="!useCustomSearchEngine" value="">{{ i18n.ts.searchEngineOther }}</option> </MkSelect> + + <div v-if="useCustomSearchEngine"> + <MkInput v-model="searchEngine" :max="300" :manualSave="true"> + <template #label>{{ i18n.ts.searchEngineCusomURI }}</template> + <template #caption>{{ i18n.ts.searchEngineCustomURIDescription }}</template> + </MkInput> + </div> </MkPreferenceContainer> </SearchMarker> @@ -395,9 +403,13 @@ SPDX-License-Identifier: AGPL-3.0-only <div class="_gaps_s"> <SearchMarker :keywords="['remember', 'keep', 'note', 'cw']"> <MkPreferenceContainer k="keepCw"> - <MkSwitch v-model="keepCw"> + <MkSelect v-model="keepCw"> <template #label><SearchLabel>{{ i18n.ts.keepCw }}</SearchLabel></template> - </MkSwitch> + <template #caption><SearchKeyword>{{ i18n.ts.keepCwDescription }}</SearchKeyword></template> + <option :value="false">{{ i18n.ts.keepCwDisabled }}</option>> + <option :value="true">{{ i18n.ts.keepCwEnabled }}</option>> + <option value="prepend-re">{{ i18n.ts.keepCwPrependRe }}</option> + </MkSelect> </MkPreferenceContainer> </SearchMarker> @@ -675,7 +687,7 @@ SPDX-License-Identifier: AGPL-3.0-only <SearchMarker :keywords="['font', 'size']"> <MkRadios v-model="fontSize"> <template #label><SearchLabel>{{ i18n.ts.fontSize }}</SearchLabel></template> - <option :value="null"><span style="font-size: 14px;">Aa</span></option> + <option value="0"><span style="font-size: 14px;">Aa</span></option> <option value="1"><span style="font-size: 15px;">Aa</span></option> <option value="2"><span style="font-size: 16px;">Aa</span></option> <option value="3"><span style="font-size: 17px;">Aa</span></option> @@ -787,7 +799,7 @@ SPDX-License-Identifier: AGPL-3.0-only <SearchMarker :keywords="['corner', 'radius']"> <MkRadios v-model="cornerRadius"> <template #label><SearchLabel>{{ i18n.ts.cornerRadius }}</SearchLabel></template> - <option :value="null"><i class="sk-icons sk-shark sk-icons-lg" style="top: 2px;position: relative;"></i> Sharkey</option> + <option value="sharkey"><i class="sk-icons sk-shark sk-icons-lg" style="top: 2px;position: relative;"></i> Sharkey</option> <option value="misskey"><i class="sk-icons sk-misskey sk-icons-lg" style="top: 2px;position: relative;"></i> Misskey</option> </MkRadios> </SearchMarker> @@ -966,7 +978,6 @@ import { worksOnInstance } from '@/utility/favicon-dot.js'; const $i = ensureSignin(); -const lang = ref(miLocalStorage.getItem('lang')); const dataSaver = ref(prefer.s.dataSaver); const overridedDeviceKind = prefer.model('overridedDeviceKind'); @@ -1026,9 +1037,6 @@ const contextMenu = prefer.model('contextMenu'); const menuStyle = prefer.model('menuStyle'); const makeEveryTextElementsSelectable = prefer.model('makeEveryTextElementsSelectable'); -const fontSize = ref(miLocalStorage.getItem('fontSize')); -const useSystemFont = ref(miLocalStorage.getItem('useSystemFont') != null); - // Sharkey options const collapseNotesRepliedTo = prefer.model('collapseNotesRepliedTo'); const showTickerOnReplies = prefer.model('showTickerOnReplies'); @@ -1044,7 +1052,6 @@ const notificationClickable = prefer.model('notificationClickable'); const warnExternalUrl = prefer.model('warnExternalUrl'); const showVisibilitySelectorOnBoost = prefer.model('showVisibilitySelectorOnBoost'); const visibilityOnBoost = prefer.model('visibilityOnBoost'); -const cornerRadius = ref(miLocalStorage.getItem('cornerRadius')); const oneko = prefer.model('oneko'); const numberOfReplies = prefer.model('numberOfReplies'); const autoloadConversation = prefer.model('autoloadConversation'); @@ -1052,40 +1059,13 @@ const clickToOpen = prefer.model('clickToOpen'); const useCustomSearchEngine = computed(() => !Object.keys(searchEngineMap).includes(searchEngine.value)); const defaultCW = ref($i.defaultCW); const defaultCWPriority = ref($i.defaultCWPriority); - -watch(lang, () => { - miLocalStorage.setItem('lang', lang.value as string); - miLocalStorage.removeItem('locale'); - miLocalStorage.removeItem('localeVersion'); -}); - -watch(fontSize, () => { - if (fontSize.value == null) { - miLocalStorage.removeItem('fontSize'); - } else { - miLocalStorage.setItem('fontSize', fontSize.value); - } -}); - -watch(useSystemFont, () => { - if (useSystemFont.value) { - miLocalStorage.setItem('useSystemFont', 't'); - } else { - miLocalStorage.removeItem('useSystemFont'); - } -}); - -watch(cornerRadius, () => { - if (cornerRadius.value == null) { - miLocalStorage.removeItem('cornerRadius'); - } else { - miLocalStorage.setItem('cornerRadius', cornerRadius.value); - } -}); +const lang = prefer.model('lang'); +const fontSize = prefer.model('fontSize'); +const useSystemFont = prefer.model('useSystemFont'); +const cornerRadius = prefer.model('cornerRadius'); watch([ hemisphere, - lang, enableInfiniteScroll, showNoteActionsOnlyHover, overridedDeviceKind, @@ -1107,8 +1087,6 @@ watch([ useStickyIcons, keepScreenOn, contextMenu, - fontSize, - useSystemFont, makeEveryTextElementsSelectable, noteDesign, ], async () => { diff --git a/packages/frontend/src/pages/settings/profile.attribution-domains-setting.vue b/packages/frontend/src/pages/settings/profile.attribution-domains-setting.vue new file mode 100644 index 0000000000..c77870f9d3 --- /dev/null +++ b/packages/frontend/src/pages/settings/profile.attribution-domains-setting.vue @@ -0,0 +1,67 @@ +<!-- +SPDX-FileCopyrightText: syuilo and misskey-project +SPDX-License-Identifier: AGPL-3.0-only +--> + +<template> +<MkTextarea v-model="attributionDomains"> + <template #label><SearchLabel>{{ i18n.ts.attributionDomains }}</SearchLabel></template> + <template #caption> + {{ i18n.ts.attributionDomainsDescription }} + <br/> + <Mfm :text="tutorialTag"/> + </template> +</MkTextarea> +<MkButton primary :disabled="!changed" @click="save()"><i class="ti ti-device-floppy"></i> {{ i18n.ts.save }}</MkButton> +</template> + +<script lang="ts" setup> +import { ref, watch, computed } from 'vue'; +import { host as hostRaw } from '@@/js/config.js'; +import { toUnicode } from 'punycode.js'; +import MkTextarea from '@/components/MkTextarea.vue'; +import MkButton from '@/components/MkButton.vue'; +import { ensureSignin } from '@/i.js'; +import { misskeyApi } from '@/utility/misskey-api.js'; +import { i18n } from '@/i18n.js'; +import * as os from '@/os.js'; + +const $i = ensureSignin(); + +const attributionDomains = ref($i.attributionDomains.join('\n')); +const domainArray = computed(() => { + return attributionDomains.value + .trim().split('\n') + .map(el => el.trim().toLowerCase()) + .filter(el => el); +}); +const changed = ref(false); +const tutorialTag = '`<meta name="fediverse:creator" content="' + $i.username + '@' + toUnicode(hostRaw) + '" />`'; + +async function save() { + // checks for a full line without whitespace. + if (!domainArray.value.every(d => /^\S+$/.test(d))) { + os.alert({ + type: 'error', + title: i18n.ts.invalidValue, + }); + return; + } + + await misskeyApi('i/update', { + attributionDomains: domainArray.value, + }); + + // Refresh filtered list to signal to the user how they've been saved + attributionDomains.value = domainArray.value.join('\n'); + + changed.value = false; +} + +watch(domainArray, (newArray, oldArray) => { + // compare arrays + if (newArray.length !== oldArray.length || !newArray.every((a, i) => a === oldArray[i])) { + changed.value = true; + } +}); +</script> diff --git a/packages/frontend/src/pages/settings/profile.vue b/packages/frontend/src/pages/settings/profile.vue index ee4dd1b65a..21bc74326a 100644 --- a/packages/frontend/src/pages/settings/profile.vue +++ b/packages/frontend/src/pages/settings/profile.vue @@ -163,6 +163,13 @@ SPDX-License-Identifier: AGPL-3.0-only <template #caption>{{ i18n.ts.flagAsBotDescription }}</template> </MkSwitch> </SearchMarker> + + <SearchMarker + :label="i18n.ts.attributionDomains" + :keywords="['attribution', 'domains', 'preview', 'url']" + > + <AttributionDomainsSettings/> + </SearchMarker> </div> </MkFolder> </SearchMarker> @@ -172,6 +179,7 @@ SPDX-License-Identifier: AGPL-3.0-only <script lang="ts" setup> import { computed, reactive, ref, watch, defineAsyncComponent } from 'vue'; +import AttributionDomainsSettings from './profile.attribution-domains-setting.vue'; import MkButton from '@/components/MkButton.vue'; import MkInput from '@/components/MkInput.vue'; import MkSwitch from '@/components/MkSwitch.vue'; diff --git a/packages/frontend/src/pages/welcome.timeline.note.vue b/packages/frontend/src/pages/welcome.timeline.note.vue index 5ca487a70b..b38946d64c 100644 --- a/packages/frontend/src/pages/welcome.timeline.note.vue +++ b/packages/frontend/src/pages/welcome.timeline.note.vue @@ -10,14 +10,18 @@ SPDX-License-Identifier: AGPL-3.0-only <div><Mfm :text="note.cw" :author="note.user"/></div> <MkCwButton v-model="showContent" :text="note.text" :renote="note.renote" :files="note.files" :poll="note.poll" style="margin: 4px 0;"/> <div v-if="showContent"> - <MkA v-if="note.replyId" class="reply" :to="`/notes/${note.replyId}`"><i class="ti ti-arrow-back-up"></i></MkA> - <Mfm v-if="note.text" :text="note.text" :isBlock="true" :author="note.user"/> + <div> + <MkA v-if="note.replyId" class="reply" :to="`/notes/${note.replyId}`"><i class="ti ti-arrow-back-up"></i></MkA> + <Mfm v-if="note.text" :text="note.text" :author="note.user"/> + </div> <MkA v-if="note.renoteId" class="rp" :to="`/notes/${note.renoteId}`">RN: ...</MkA> </div> </div> <div v-else ref="noteTextEl" :class="[$style.text, { [$style.collapsed]: shouldCollapse }]"> - <MkA v-if="note.replyId" class="reply" :to="`/notes/${note.replyId}`"><i class="ti ti-arrow-back-up"></i></MkA> - <Mfm v-if="note.text" :text="note.text" :isBlock="true" :author="note.user"/> + <div> + <MkA v-if="note.replyId" class="reply" :to="`/notes/${note.replyId}`"><i class="ti ti-arrow-back-up"></i></MkA> + <Mfm v-if="note.text" :text="note.text" :author="note.user"/> + </div> <MkA v-if="note.renoteId" class="rp" :to="`/notes/${note.renoteId}`">RN: ...</MkA> </div> <div v-if="note.files && note.files.length > 0" :class="$style.richcontent"> diff --git a/packages/frontend/src/pref-migrate.ts b/packages/frontend/src/pref-migrate.ts index a7bb82a5f0..ccd4472ca2 100644 --- a/packages/frontend/src/pref-migrate.ts +++ b/packages/frontend/src/pref-migrate.ts @@ -13,15 +13,18 @@ import { deckStore } from '@/ui/deck/deck-store.js'; import { unisonReload } from '@/utility/unison-reload.js'; import * as os from '@/os.js'; import { i18n } from '@/i18n.js'; +import { miLocalStorage } from '@/local-storage'; // TODO: そのうち消す export function migrateOldSettings() { os.waiting(i18n.ts.settingsMigrating); store.loaded.then(async () => { - misskeyApi('i/registry/get', { scope: ['client'], key: 'themes' }).catch(() => []).then((themes: Theme[]) => { - if (themes.length > 0) { - prefer.commit('themes', themes); + prefer.suppressReload(); + + await misskeyApi('i/registry/get', { scope: ['client'], key: 'themes' }).catch(() => []).then(themes => { + if (Array.isArray(themes) && themes.length > 0) { + prefer.commit('themes', themes as Theme[]); } }); @@ -33,7 +36,7 @@ export function migrateOldSettings() { }))); prefer.commit('deck.profile', deckStore.s.profile); - misskeyApi('i/registry/keys', { + await misskeyApi('i/registry/keys', { scope: ['client', 'deck', 'profiles'], }).then(async keys => { const profiles: DeckProfile[] = []; @@ -41,16 +44,18 @@ export function migrateOldSettings() { const deck = await misskeyApi('i/registry/get', { scope: ['client', 'deck', 'profiles'], key: key, - }); - profiles.push({ - id: uuid(), - name: key, - columns: deck.columns, - layout: deck.layout, - }); + }).catch(() => null); + if (deck) { + profiles.push({ + id: uuid(), + name: key, + columns: (deck as DeckProfile).columns, + layout: (deck as DeckProfile).layout, + }); + } } prefer.commit('deck.profiles', profiles); - }); + }).catch(() => null); prefer.commit('lightTheme', ColdDeviceStorage.get('lightTheme')); prefer.commit('darkTheme', ColdDeviceStorage.get('darkTheme')); @@ -164,8 +169,17 @@ export function migrateOldSettings() { prefer.commit('warnMissingAltText', store.s.warnMissingAltText); //#endregion - window.setTimeout(() => { - unisonReload(); - }, 10000); + //#region Hybrid migrations + prefer.commit('fontSize', miLocalStorage.getItem('fontSize') ?? '0'); + prefer.commit('useSystemFont', miLocalStorage.getItem('useSystemFont') != null); + prefer.commit('cornerRadius', miLocalStorage.getItem('cornerRadius') ?? 'sharkey'); + prefer.commit('lang', miLocalStorage.getItem('lang') ?? 'en-US'); + prefer.commit('customCss', miLocalStorage.getItem('customCss') ?? ''); + prefer.commit('neverShowDonationInfo', miLocalStorage.getItem('neverShowDonationInfo') != null); + prefer.commit('neverShowLocalOnlyInfo', miLocalStorage.getItem('neverShowLocalOnlyInfo') != null); + //#endregion + + prefer.allowReload(); + unisonReload(); }); } diff --git a/packages/frontend/src/preferences.ts b/packages/frontend/src/preferences.ts index 8d3cbae797..f8a6edcc98 100644 --- a/packages/frontend/src/preferences.ts +++ b/packages/frontend/src/preferences.ts @@ -130,7 +130,7 @@ function syncBetweenTabs() { latestSyncedAt = Date.now(); - if (_DEV_) console.log('prefer:synced'); + if (_DEV_) console.debug('prefer:synced'); } window.setInterval(syncBetweenTabs, 5000); diff --git a/packages/frontend/src/preferences/def.ts b/packages/frontend/src/preferences/def.ts index a4d52c8acb..f430c4573c 100644 --- a/packages/frontend/src/preferences/def.ts +++ b/packages/frontend/src/preferences/def.ts @@ -10,11 +10,12 @@ import type { SoundType } from '@/utility/sound.js'; import type { Plugin } from '@/plugin.js'; import type { DeviceKind } from '@/utility/device-kind.js'; import type { DeckProfile } from '@/deck.js'; -import type { PreferencesDefinition } from './manager.js'; +import type { Pref, PreferencesDefinition } from './manager.js'; import type { FollowingFeedState } from '@/types/following-feed.js'; import { DEFAULT_DEVICE_KIND } from '@/utility/device-kind.js'; import { searchEngineMap } from '@/utility/search-engine-map.js'; import { defaultFollowingFeedState } from '@/types/following-feed.js'; +import { miLocalStorage } from '@/local-storage'; /** サウンド設定 */ export type SoundStore = { @@ -120,7 +121,7 @@ export const PREF_DEF = { default: false, }, keepCw: { - default: true, + default: true as boolean | 'prepend-re', }, rememberNoteVisibility: { default: false, @@ -477,4 +478,84 @@ export const PREF_DEF = { default: true, }, //#endregion + + //#region hybrid options + // These exist in preferences, but may have a legacy value in local storage. + // Some parts of the system may still reference the legacy storage so both need to stay in sync! + // Null means "fall back to existing value from localStorage" + // For all of these preferences, "null" means fall back to existing value in localStorage. + fontSize: { + default: '0', + needsReload: true, + onSet: fontSize => { + if (fontSize !== '0') { + miLocalStorage.setItem('fontSize', fontSize); + } else { + miLocalStorage.removeItem('fontSize'); + } + }, + } as Pref<'0' | '1' | '2' | '3'>, + useSystemFont: { + default: false, + needsReload: true, + onSet: useSystemFont => { + if (useSystemFont) { + miLocalStorage.setItem('useSystemFont', 't'); + } else { + miLocalStorage.removeItem('useSystemFont'); + } + }, + } as Pref<boolean>, + cornerRadius: { + default: 'sharkey', + needsReload: true, + onSet: cornerRadius => { + if (cornerRadius === 'sharkey') { + miLocalStorage.removeItem('cornerRadius'); + } else { + miLocalStorage.setItem('cornerRadius', cornerRadius); + } + }, + } as Pref<'misskey' | 'sharkey'>, + lang: { + default: 'en-US', + needsReload: true, + onSet: lang => { + miLocalStorage.setItem('lang', lang); + miLocalStorage.removeItem('locale'); + miLocalStorage.removeItem('localeVersion'); + }, + } as Pref<string>, + customCss: { + default: '', + needsReload: true, + onSet: customCss => { + if (customCss) { + miLocalStorage.setItem('customCss', customCss); + } else { + miLocalStorage.removeItem('customCss'); + } + }, + } as Pref<string>, + neverShowDonationInfo: { + default: false, + onSet: neverShowDonationInfo => { + if (neverShowDonationInfo) { + miLocalStorage.setItem('neverShowDonationInfo', 'true'); + } else { + miLocalStorage.removeItem('neverShowDonationInfo'); + } + }, + } as Pref<boolean>, + neverShowLocalOnlyInfo: { + default: false, + onSet: neverShowLocalOnlyInfo => { + if (neverShowLocalOnlyInfo) { + miLocalStorage.setItem('neverShowLocalOnlyInfo', 'true'); + } else { + miLocalStorage.removeItem('neverShowLocalOnlyInfo'); + } + }, + } as Pref<boolean>, + //#endregion } satisfies PreferencesDefinition; diff --git a/packages/frontend/src/preferences/manager.ts b/packages/frontend/src/preferences/manager.ts index f96aa2f368..349040d98e 100644 --- a/packages/frontend/src/preferences/manager.ts +++ b/packages/frontend/src/preferences/manager.ts @@ -3,7 +3,7 @@ * SPDX-License-Identifier: AGPL-3.0-only */ -import { computed, onUnmounted, ref, watch } from 'vue'; +import { computed, nextTick, onUnmounted, ref, watch } from 'vue'; import { v4 as uuid } from 'uuid'; import { host, version } from '@@/js/config.js'; import { PREF_DEF } from './def.js'; @@ -14,6 +14,7 @@ import { copyToClipboard } from '@/utility/copy-to-clipboard.js'; import { i18n } from '@/i18n.js'; import * as os from '@/os.js'; import { deepEqual } from '@/utility/deep-equal.js'; +import { reloadAsk } from '@/utility/reload-ask'; // NOTE: 明示的な設定値のひとつとして null もあり得るため、設定が存在しないかどうかを判定する目的で null で比較したり ?? を使ってはいけない @@ -84,16 +85,29 @@ export type StorageProvider = { cloudSet: <K extends keyof PREF>(ctx: { key: K; scope: Scope; value: ValueOf<K>; }) => Promise<void>; }; -export type PreferencesDefinition = Record<string, { - default: any; +export type Pref<T> = { + default: T; accountDependent?: boolean; serverDependent?: boolean; -}>; + needsReload?: boolean; + onSet?: (value: T) => void; +}; + +export type PreferencesDefinition = Record<string, Pref<any> | undefined>; export class PreferencesManager { private storageProvider: StorageProvider; public profile: PreferencesProfile; public cloudReady: Promise<void>; + private enableReload = true; + + public suppressReload() { + this.enableReload = false; + } + + public allowReload() { + this.enableReload = true; + } /** * static / state の略 (static が予約語のため) @@ -126,11 +140,11 @@ export class PreferencesManager { } private isAccountDependentKey<K extends keyof PREF>(key: K): boolean { - return (PREF_DEF as PreferencesDefinition)[key].accountDependent === true; + return (PREF_DEF as PreferencesDefinition)[key]?.accountDependent === true; } private isServerDependentKey<K extends keyof PREF>(key: K): boolean { - return (PREF_DEF as PreferencesDefinition)[key].serverDependent === true; + return (PREF_DEF as PreferencesDefinition)[key]?.serverDependent === true; } private rewriteRawState<K extends keyof PREF>(key: K, value: ValueOf<K>) { @@ -142,14 +156,28 @@ export class PreferencesManager { const v = JSON.parse(JSON.stringify(value)); // deep copy 兼 vueのプロキシ解除 if (deepEqual(this.s[key], v)) { - if (_DEV_) console.log('(skip) prefer:commit', key, v); + if (_DEV_) console.debug('(skip) prefer:commit', key, v); return; } - if (_DEV_) console.log('prefer:commit', key, v); + if (_DEV_) console.debug('prefer:commit', key, v); this.rewriteRawState(key, v); + const pref = (PREF_DEF as PreferencesDefinition)[key]; + if (pref) { + // Call custom setter + if (pref.onSet) { + pref.onSet(v); + } + + // Prompt to reload the frontend + if (pref.needsReload && this.enableReload) { + // noinspection JSIgnoredPromiseFromCall + nextTick(() => reloadAsk({ unison: true })); + } + } + const record = this.getMatchedRecordOf(key); if (parseScope(record[0]).account == null && this.isAccountDependentKey(key)) { @@ -250,13 +278,13 @@ export class PreferencesManager { if (!deepEqual(cloudValue, record[1])) { this.rewriteRawState(key, cloudValue); record[1] = cloudValue; - if (_DEV_) console.log('cloud fetched', key, cloudValue); + if (_DEV_) console.debug('cloud fetched', key, cloudValue); } } } this.save(); - if (_DEV_) console.log('cloud fetch completed'); + if (_DEV_) console.debug('cloud fetch completed'); } public static newProfile(): PreferencesProfile { diff --git a/packages/frontend/src/preferences/utility.ts b/packages/frontend/src/preferences/utility.ts index adba908c3c..d8986ceb52 100644 --- a/packages/frontend/src/preferences/utility.ts +++ b/packages/frontend/src/preferences/utility.ts @@ -153,7 +153,7 @@ export async function restoreFromCloudBackup() { scope: ['client', 'preferences', 'backups'], }); - if (_DEV_) console.log(keys); + if (_DEV_) console.debug(keys); if (keys.length === 0) { os.alert({ @@ -179,7 +179,7 @@ export async function restoreFromCloudBackup() { key: select.result, }); - if (_DEV_) console.log(profile); + if (_DEV_) console.debug(profile); miLocalStorage.setItem('preferences', JSON.stringify(profile)); miLocalStorage.setItem('hidePreferencesRestoreSuggestion', 'true'); diff --git a/packages/frontend/src/style.scss b/packages/frontend/src/style.scss index 7b2638903b..1333b227f3 100644 --- a/packages/frontend/src/style.scss +++ b/packages/frontend/src/style.scss @@ -211,7 +211,9 @@ rt { max-width: min(var(--MI_SPACER-w, 100%), calc(100% - (var(--MI_SPACER-max, 24px) * 2))); margin: var(--MI_SPACER-max, 24px) auto; container-type: inline-size; +} +._spacer > * { /* 子に継承させない */ --MI_SPACER-w: initial; --MI_SPACER-min: initial; @@ -428,6 +430,14 @@ rt { gap: var(--MI-margin); } +/** + * Use with _gaps, _gaps_m, or _gaps_s. + * Place the other class *first*! + */ +._h_gaps { + flex-direction: row; +} + ._buttons { display: flex; gap: 8px; diff --git a/packages/frontend/src/tab-id.ts b/packages/frontend/src/tab-id.ts index 49b69f72d2..0178fc36be 100644 --- a/packages/frontend/src/tab-id.ts +++ b/packages/frontend/src/tab-id.ts @@ -8,4 +8,4 @@ import { v4 as uuid } from 'uuid'; // HMR有効時にバグか知らんけど複数回実行されるのでその対策 export const TAB_ID = window.sessionStorage.getItem('TAB_ID') ?? uuid(); window.sessionStorage.setItem('TAB_ID', TAB_ID); -if (_DEV_) console.log('TAB_ID', TAB_ID); +if (_DEV_) console.debug('TAB_ID', TAB_ID); diff --git a/packages/frontend/src/ui/_common_/common.vue b/packages/frontend/src/ui/_common_/common.vue index 7cfbd9df0a..d080f48696 100644 --- a/packages/frontend/src/ui/_common_/common.vue +++ b/packages/frontend/src/ui/_common_/common.vue @@ -67,8 +67,7 @@ SPDX-License-Identifier: AGPL-3.0-only <XUpload v-if="uploads.length > 0"/> -<component - :is="prefer.s.animation ? TransitionGroup : 'div'" +<SkTransitionGroup tag="div" :class="[$style.notifications, { [$style.notificationsPosition_leftTop]: prefer.s.notificationPosition === 'leftTop', @@ -87,7 +86,7 @@ SPDX-License-Identifier: AGPL-3.0-only <div v-for="notification in notifications" :key="notification.id" :class="$style.notification" :style="{ pointerEvents: getPointerEvents() }"> <XNotification :notification="notification"/> </div> -</component> +</SkTransitionGroup> <XStreamIndicator/> @@ -115,6 +114,7 @@ import { i18n } from '@/i18n.js'; import { prefer } from '@/preferences.js'; import { globalEvents } from '@/events.js'; import XDrawerMenu from '@/ui/_common_/navbar-for-mobile.vue'; +import SkTransitionGroup from '@/components/SkTransitionGroup.vue'; const XStreamIndicator = defineAsyncComponent(() => import('./stream-indicator.vue')); const XUpload = defineAsyncComponent(() => import('./upload.vue')); diff --git a/packages/frontend/src/ui/_common_/navbar.vue b/packages/frontend/src/ui/_common_/navbar.vue index 6bf0dfc17c..f0c62aa8e2 100644 --- a/packages/frontend/src/ui/_common_/navbar.vue +++ b/packages/frontend/src/ui/_common_/navbar.vue @@ -7,6 +7,7 @@ SPDX-License-Identifier: AGPL-3.0-only <div :class="[$style.root, { [$style.iconOnly]: iconOnly }]"> <div :class="$style.body"> <div :class="$style.top"> + <div :class="$style.banner" :style="{ backgroundImage: `url(${ instance.bannerUrl })` }"></div> <button v-tooltip.noDelay.right="instance.name ?? i18n.ts.instance" class="_button" :class="$style.instance" @click="openInstanceMenu"> <img :src="instance.sidebarLogoUrl && !iconOnly ? instance.sidebarLogoUrl : instance.iconUrl || '/favicon.ico'" alt="" :class="instance.sidebarLogoUrl && !iconOnly ? $style.wideInstanceIcon : $style.instanceIcon" style="viewTransitionName: navbar-serverIcon;"/> </button> @@ -299,6 +300,18 @@ function menuEdit() { backdrop-filter: var(--MI-blur, blur(8px)); } + .banner { + position: absolute; + top: 0; + left: 0; + width: 100%; + height: 100%; + background-size: cover; + background-position: center center; + -webkit-mask-image: linear-gradient(0deg, rgba(0,0,0,0) 15%, rgba(0,0,0,0.75) 100%); + mask-image: linear-gradient(0deg, rgba(0,0,0,0) 15%, rgba(0,0,0,0.75) 100%); + } + .instance { position: relative; display: block; diff --git a/packages/frontend/src/utility/check-animated-mfm.ts b/packages/frontend/src/utility/check-animated-mfm.ts index 2614dfb4f1..371a631af7 100644 --- a/packages/frontend/src/utility/check-animated-mfm.ts +++ b/packages/frontend/src/utility/check-animated-mfm.ts @@ -3,7 +3,7 @@ * SPDX-License-Identifier: AGPL-3.0-only */ -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; export function checkAnimationFromMfm(nodes: mfm.MfmNode[]): boolean { const animatedNodes = mfm.extract(nodes, (node) => { diff --git a/packages/frontend/src/utility/extract-mentions.ts b/packages/frontend/src/utility/extract-mentions.ts index 89a5ce1df8..d518562053 100644 --- a/packages/frontend/src/utility/extract-mentions.ts +++ b/packages/frontend/src/utility/extract-mentions.ts @@ -5,7 +5,7 @@ // test is located in test/extract-mentions -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; export function extractMentions(nodes: mfm.MfmNode[]): mfm.MfmMention['props'][] { // TODO: 重複を削除 diff --git a/packages/frontend/src/utility/extract-preview-urls.ts b/packages/frontend/src/utility/extract-preview-urls.ts index 5fc9c87a32..264359f179 100644 --- a/packages/frontend/src/utility/extract-preview-urls.ts +++ b/packages/frontend/src/utility/extract-preview-urls.ts @@ -3,35 +3,18 @@ * SPDX-License-Identifier: AGPL-3.0-only */ -import * as config from '@@/js/config.js'; import type * as Misskey from 'misskey-js'; -import type * as mfm from '@transfem-org/sfm-js'; +import type * as mfm from 'mfm-js'; import { extractUrlFromMfm } from '@/utility/extract-url-from-mfm.js'; +import { getNoteUrls } from '@/utility/getNoteUrls'; /** * Extracts all previewable URLs from a note. */ export function extractPreviewUrls(note: Misskey.entities.Note, contents: mfm.MfmNode[]): string[] { const links = extractUrlFromMfm(contents); - return links.filter(url => - // Remote note - url !== note.url && - url !== note.uri && - // Local note - url !== `${config.url}/notes/${note.id}` && - // Remote reply - url !== note.reply?.url && - url !== note.reply?.uri && - // Local reply - url !== `${config.url}/notes/${note.reply?.id}` && - // Remote renote or quote - url !== note.renote?.url && - url !== note.renote?.uri && - // Local renote or quote - url !== `${config.url}/notes/${note.renote?.id}` && - // Remote renote *of* a quote - url !== note.renote?.renote?.url && - url !== note.renote?.renote?.uri && - // Local renote *of* a quote - url !== `${config.url}/notes/${note.renote?.renote?.id}`); + if (links.length < 0) return []; + + const self = getNoteUrls(note); + return links.filter(url => !self.includes(url)); } diff --git a/packages/frontend/src/utility/extract-url-from-mfm.ts b/packages/frontend/src/utility/extract-url-from-mfm.ts index 260dba030e..99d80f3624 100644 --- a/packages/frontend/src/utility/extract-url-from-mfm.ts +++ b/packages/frontend/src/utility/extract-url-from-mfm.ts @@ -3,7 +3,7 @@ * SPDX-License-Identifier: AGPL-3.0-only */ -import * as mfm from '@transfem-org/sfm-js'; +import * as mfm from 'mfm-js'; // unique without hash // [ http://a/#1, http://a/#2, http://b/#3 ] => [ http://a/#1, http://b/#3 ] diff --git a/packages/frontend/src/utility/get-note-menu.ts b/packages/frontend/src/utility/get-note-menu.ts index f773149fac..056480a7ef 100644 --- a/packages/frontend/src/utility/get-note-menu.ts +++ b/packages/frontend/src/utility/get-note-menu.ts @@ -11,7 +11,7 @@ import type { Ref, ShallowRef } from 'vue'; import type { MenuItem } from '@/types/menu.js'; import { $i } from '@/i.js'; import { i18n } from '@/i18n.js'; -import { instance } from '@/instance.js'; +import { instance, policies } from '@/instance.js'; import * as os from '@/os.js'; import { misskeyApi } from '@/utility/misskey-api.js'; import { copyToClipboard } from '@/utility/copy-to-clipboard.js'; @@ -342,7 +342,7 @@ export function getNoteMenu(props: { }); } - if ($i.policies.canUseTranslator && instance.translatorAvailable) { + if (policies.value.canUseTranslator && instance.translatorAvailable) { menuItems.push({ icon: 'ti ti-language-hiragana', text: i18n.ts.translate, @@ -497,6 +497,14 @@ export function getNoteMenu(props: { } else { menuItems.push(getNoteEmbedCodeMenu(appearNote, i18n.ts.embed)); } + + if (policies.value.canUseTranslator && instance.translatorAvailable) { + menuItems.push({ + icon: 'ti ti-language-hiragana', + text: i18n.ts.translate, + action: () => translateNote(appearNote.id, props.translation, props.translating), + }); + } } const noteActions = getPluginHandlers('note_action'); @@ -523,7 +531,7 @@ export function getNoteMenu(props: { } const cleanup = () => { - if (_DEV_) console.log('note menu cleanup', cleanups); + if (_DEV_) console.debug('note menu cleanup', cleanups); for (const cl of cleanups) { cl(); } diff --git a/packages/frontend/src/utility/get-note-versions-menu.ts b/packages/frontend/src/utility/get-note-versions-menu.ts index aac0375640..ec830f3d3f 100644 --- a/packages/frontend/src/utility/get-note-versions-menu.ts +++ b/packages/frontend/src/utility/get-note-versions-menu.ts @@ -54,7 +54,7 @@ export async function getNoteVersionsMenu(props: { note: Misskey.entities.Note } }); const cleanup = () => { - if (_DEV_) console.log('note menu cleanup', cleanups); + if (_DEV_) console.debug('note menu cleanup', cleanups); for (const cl of cleanups) { cl(); } diff --git a/packages/frontend/src/utility/get-user-menu.ts b/packages/frontend/src/utility/get-user-menu.ts index 9b7320586a..fde390cece 100644 --- a/packages/frontend/src/utility/get-user-menu.ts +++ b/packages/frontend/src/utility/get-user-menu.ts @@ -443,7 +443,7 @@ export function getUserMenu(user: Misskey.entities.UserDetailed, router: Router return { menu: menuItems, cleanup: () => { - if (_DEV_) console.log('user menu cleanup', cleanups); + if (_DEV_) console.debug('user menu cleanup', cleanups); for (const cl of cleanups) { cl(); } diff --git a/packages/frontend/src/utility/getNoteUrls.ts b/packages/frontend/src/utility/getNoteUrls.ts new file mode 100644 index 0000000000..efd014cbf0 --- /dev/null +++ b/packages/frontend/src/utility/getNoteUrls.ts @@ -0,0 +1,44 @@ +/* + * SPDX-FileCopyrightText: hazelnoot and other Sharkey contributors + * SPDX-License-Identifier: AGPL-3.0-only + */ + +import * as config from '@@/js/config.js'; +import type * as Misskey from 'misskey-js'; + +export function getNoteUrls(note: Misskey.entities.Note): string[] { + const urls: string[] = [ + // Any note + `${config.url}/notes/${note.id}`, + ]; + + // Remote note + if (note.url) urls.push(note.url); + if (note.uri) urls.push(note.uri); + + if (note.reply) { + // Any Reply + urls.push(`${config.url}/notes/${note.reply.id}`); + // Remote Reply + if (note.reply.url) urls.push(note.reply.url); + if (note.reply.uri) urls.push(note.reply.uri); + } + + if (note.renote) { + // Any Renote + urls.push(`${config.url}/notes/${note.renote.id}`); + // Remote Renote + if (note.renote.url) urls.push(note.renote.url); + if (note.renote.uri) urls.push(note.renote.uri); + } + + if (note.renote?.renote) { + // Any Quote + urls.push(`${config.url}/notes/${note.renote.renote.id}`); + // Remote Quote + if (note.renote.renote.url) urls.push(note.renote.renote.url); + if (note.renote.renote.uri) urls.push(note.renote.renote.uri); + } + + return urls; +} diff --git a/packages/frontend/src/utility/intl-const.ts b/packages/frontend/src/utility/intl-const.ts index 385f59ec39..cb2bf7c70d 100644 --- a/packages/frontend/src/utility/intl-const.ts +++ b/packages/frontend/src/utility/intl-const.ts @@ -19,7 +19,7 @@ try { }); } catch (err) { console.warn(err); - if (_DEV_) console.log('[Intl] Fallback to en-US'); + if (_DEV_) console.debug('[Intl] Fallback to en-US'); // Fallback to en-US _dateTimeFormat = new Intl.DateTimeFormat('en-US', { @@ -42,7 +42,7 @@ try { _numberFormat = new Intl.NumberFormat(versatileLang); } catch (err) { console.warn(err); - if (_DEV_) console.log('[Intl] Fallback to en-US'); + if (_DEV_) console.debug('[Intl] Fallback to en-US'); // Fallback to en-US _numberFormat = new Intl.NumberFormat('en-US'); diff --git a/packages/frontend/src/utility/reload-ask.ts b/packages/frontend/src/utility/reload-ask.ts index 7c7ea113d4..f49de80231 100644 --- a/packages/frontend/src/utility/reload-ask.ts +++ b/packages/frontend/src/utility/reload-ask.ts @@ -12,6 +12,10 @@ let isReloadConfirming = false; export async function reloadAsk(opts: { unison?: boolean; reason?: string; + type?: 'error' | 'info' | 'success' | 'warning' | 'waiting' | 'question'; + title?: string; + okText?: string; + cancelText?: string; }) { if (isReloadConfirming) { return; @@ -19,13 +23,12 @@ export async function reloadAsk(opts: { isReloadConfirming = true; - const { canceled } = await os.confirm(opts.reason == null ? { - type: 'info', - text: i18n.ts.reloadConfirm, - } : { - type: 'info', - title: i18n.ts.reloadConfirm, - text: opts.reason, + const { canceled } = await os.confirm({ + type: opts.type ?? 'question', + title: opts.title ?? i18n.ts.reloadConfirm, + text: opts.reason ?? undefined, + okText: opts.okText ?? i18n.ts.yes, + cancelText: opts.cancelText ?? i18n.ts.no, }).finally(() => { isReloadConfirming = false; }); diff --git a/packages/frontend/src/utility/sound.ts b/packages/frontend/src/utility/sound.ts index d3f82a37f2..3eba4d3e20 100644 --- a/packages/frontend/src/utility/sound.ts +++ b/packages/frontend/src/utility/sound.ts @@ -134,7 +134,7 @@ export function playMisskeySfx(operationType: OperationType) { if (!succeed && sound.type === '_driveFile_') { // ドライブファイルが存在しない場合はデフォルトのサウンドを再生する const soundName = PREF_DEF[`sound_${operationType}`].default.type as Exclude<SoundType, '_driveFile_'>; - if (_DEV_) console.log(`Failed to play sound: ${sound.fileUrl}, so play default sound: ${soundName}`); + if (_DEV_) console.debug(`Failed to play sound: ${sound.fileUrl}, so play default sound: ${soundName}`); playMisskeySfxFileInternal({ type: soundName, volume: sound.volume, diff --git a/packages/frontend/src/utility/timeline-date-separate.ts b/packages/frontend/src/utility/timeline-date-separate.ts index e1bc9790b9..ef946b11d6 100644 --- a/packages/frontend/src/utility/timeline-date-separate.ts +++ b/packages/frontend/src/utility/timeline-date-separate.ts @@ -4,7 +4,7 @@ */ import { computed } from 'vue'; -import type { Ref } from 'vue'; +import type { Ref, ComputedRef } from 'vue'; export function getDateText(dateInstance: Date) { const date = dateInstance.getDate(); @@ -25,7 +25,7 @@ export type DateSeparetedTimelineItem<T> = { nextText: string; }; -export function makeDateSeparatedTimelineComputedRef<T extends { id: string; createdAt: string; }>(items: Ref<T[]>) { +export function makeDateSeparatedTimelineComputedRef<T extends { id: string; createdAt: string; }>(items: Ref<T[]> | ComputedRef<T[]>) { return computed<DateSeparetedTimelineItem<T>[]>(() => { const tl: DateSeparetedTimelineItem<T>[] = []; for (let i = 0; i < items.value.length; i++) { diff --git a/packages/frontend/src/widgets/WidgetFederation.vue b/packages/frontend/src/widgets/WidgetFederation.vue index d983376566..799225662b 100644 --- a/packages/frontend/src/widgets/WidgetFederation.vue +++ b/packages/frontend/src/widgets/WidgetFederation.vue @@ -10,7 +10,7 @@ SPDX-License-Identifier: AGPL-3.0-only <div class="wbrkwalb"> <MkLoading v-if="fetching"/> - <TransitionGroup v-else tag="div" :name="prefer.s.animation ? 'chart' : ''" class="instances"> + <SkTransitionGroup v-else tag="div" name="chart" class="instances"> <div v-for="(instance, i) in instances" :key="instance.id" class="instance"> <img :src="getInstanceIcon(instance)" alt=""/> <div class="body"> @@ -19,7 +19,7 @@ SPDX-License-Identifier: AGPL-3.0-only </div> <MkMiniChart class="chart" :src="charts[i].requests.received"/> </div> - </TransitionGroup> + </SkTransitionGroup> </div> </MkContainer> </template> @@ -37,6 +37,7 @@ import { misskeyApi, misskeyApiGet } from '@/utility/misskey-api.js'; import { i18n } from '@/i18n.js'; import { getProxiedImageUrlNullable } from '@/utility/media-proxy.js'; import { prefer } from '@/preferences.js'; +import SkTransitionGroup from '@/components/SkTransitionGroup.vue'; const name = 'federation'; diff --git a/packages/frontend/src/widgets/WidgetTrends.vue b/packages/frontend/src/widgets/WidgetTrends.vue index db09031c33..d5e2c930de 100644 --- a/packages/frontend/src/widgets/WidgetTrends.vue +++ b/packages/frontend/src/widgets/WidgetTrends.vue @@ -10,7 +10,7 @@ SPDX-License-Identifier: AGPL-3.0-only <div class="wbrkwala"> <MkLoading v-if="fetching"/> - <TransitionGroup v-else tag="div" :name="prefer.s.animation ? 'chart' : ''" class="tags"> + <SkTransitionGroup v-else tag="div" name="chart" class="tags"> <div v-for="stat in stats" :key="stat.tag"> <div class="tag"> <MkA class="a" :to="`/tags/${ encodeURIComponent(stat.tag) }`" :title="stat.tag">#{{ stat.tag }}</MkA> @@ -18,7 +18,7 @@ SPDX-License-Identifier: AGPL-3.0-only </div> <MkMiniChart class="chart" :src="stat.chart"/> </div> - </TransitionGroup> + </SkTransitionGroup> </div> </MkContainer> </template> @@ -35,6 +35,7 @@ import MkMiniChart from '@/components/MkMiniChart.vue'; import { misskeyApiGet } from '@/utility/misskey-api.js'; import { i18n } from '@/i18n.js'; import { prefer } from '@/preferences.js'; +import SkTransitionGroup from '@/components/SkTransitionGroup.vue'; const name = 'hashtags'; diff --git a/packages/frontend/src/workers/tsconfig.json b/packages/frontend/src/workers/tsconfig.json index 8ee8930465..39ba45ddbb 100644 --- a/packages/frontend/src/workers/tsconfig.json +++ b/packages/frontend/src/workers/tsconfig.json @@ -1,5 +1,6 @@ { "compilerOptions": { "lib": ["esnext", "webworker"], + "incremental": true } } diff --git a/packages/frontend/test/tsconfig.json b/packages/frontend/test/tsconfig.json index 98ac45211b..1490a66d20 100644 --- a/packages/frontend/test/tsconfig.json +++ b/packages/frontend/test/tsconfig.json @@ -22,6 +22,7 @@ "emitDecoratorMetadata": true, "resolveJsonModule": true, "isolatedModules": true, + "incremental": true, "baseUrl": "./", "paths": { "@/*": ["../src/*"] diff --git a/packages/frontend/tsconfig.json b/packages/frontend/tsconfig.json index 3c7e5e1da3..0616eee5be 100644 --- a/packages/frontend/tsconfig.json +++ b/packages/frontend/tsconfig.json @@ -23,6 +23,7 @@ "useDefineForClassFields": true, "verbatimModuleSyntax": true, "skipLibCheck": true, + "incremental": true, "baseUrl": ".", "paths": { "@/*": ["./src/*"], diff --git a/packages/frontend/vite.replaceIcons.ts b/packages/frontend/vite.replaceIcons.ts index ce05fc7e7b..36e6666925 100644 --- a/packages/frontend/vite.replaceIcons.ts +++ b/packages/frontend/vite.replaceIcons.ts @@ -224,6 +224,7 @@ export function pluginReplaceIcons() { 'ti ti-dice-5': 'ph ph-dice-five ph-bold ph-lg', 'ti ti-dots': 'ph-dots-three ph-bold ph-lg', 'ti ti-download': 'ph-download ph-bold ph-lg', + 'ti-download': 'ph-download ph-bold ph-lg', // in custom-emoji-manager.remote.list 'ti ti-edit': 'ph-pencil-simple-line ph-bold ph-lg', 'ti ti-equal-double': 'ph-equals ph-bold ph-lg', 'ti ti-equal-not': 'ph-prohibit ph-bold ph-lg', @@ -258,6 +259,7 @@ export function pluginReplaceIcons() { 'ti ti-home': 'ph-house ph-bold ph-lg', 'ti ti-hourglass-empty': 'ph-hourglass ph-bold ph-lg', 'ti ti-icons': 'ph-squares-four ph-bold ph-lg', + 'ti-icons': 'ph-squares-four ph-bold ph-lg', // in custom-emoji-manager.local.list 'ti ti-id': 'ph-identification-card ph-bold ph-lg', 'ti ti-info-circle': 'ph-info ph-bold ph-lg', 'ti ti-json': 'ph-brackets-curly ph-bold ph-lg', @@ -275,6 +277,7 @@ export function pluginReplaceIcons() { 'ti ti-lock-star': 'ph-shield-star ph-bold ph-lg', 'ti ti-login-2': 'ph-sign-in ph-bold ph-lg', 'ti ti-mail': 'ph-envelope ph-bold ph-lg', + 'ti-mail': 'ph-envelope ph-bold ph-lg', // in notification-recipient.item.vue 'ti ti-map-pin': 'ph-map-pin ph-bold ph-lg', 'ti ti-maximize': 'ph-frame-corners ph-bold ph-lg', 'ti ti-medal': 'ph-trophy ph-bold ph-lg', @@ -359,6 +362,7 @@ export function pluginReplaceIcons() { 'ti ti-text-caption': 'ph-text-indent ph-bold ph-lg', 'ti ti-tool': 'ph-wrench ph-bold ph-lg', 'ti ti-trash': 'ph-trash ph-bold ph-lg', + 'ti-trash': 'ph-trash ph-bold ph-lg', // in custom-emoji-manager.local.list 'ti ti-trophy': 'ph-trophy ph-bold ph-lg', 'ti ti-universe': 'ph-rocket-launch ph-bold ph-lg', 'ti ti-upload': 'ph-upload ph-bold ph-lg', @@ -379,6 +383,7 @@ export function pluginReplaceIcons() { 'ti ti-volume': 'ph-speaker-high ph-bold ph-lg', 'ti ti-volume-3': 'ph-speaker-x ph-bold ph-lg', 'ti ti-webhook': 'ph-webhooks-logo ph-bold ph-lg', + 'ti-webhook': 'ph-webhooks-logo ph-bold ph-lg', // in notification-recipient.item.vue 'ti ti-whirl': 'ph-globe-hemisphere-west ph-bold ph-lg', 'ti ti-window-maximize': 'ph-frame-corners ph-bold ph-lg', 'ti ti-world': 'ph-globe-hemisphere-west ph-bold ph-lg', @@ -389,6 +394,7 @@ export function pluginReplaceIcons() { 'ti ti-world-x': 'ph-planet ph-bold ph-lg', 'ti ti-x': 'ph-x ph-bold ph-lg', 'ti ti-help': 'ph-question ph-bold ph-lg', + 'ti-help': 'ph-question ph-bold ph-lg', // in notification-recipient.item.vue 'ti ti ti-caret-down': 'ph-caret-down ph-bold ph-lg', 'ti ti-chevron-down': 'ph-caret-down ph-bold ph-lg', 'ti ti-accessible': 'ph-person-simple-circle ph-bold ph-lg', diff --git a/packages/megalodon/package.json b/packages/megalodon/package.json index f10a9cf9dc..4c7f9750de 100644 --- a/packages/megalodon/package.json +++ b/packages/megalodon/package.json @@ -1,75 +1,69 @@ { - "name": "megalodon", - "version": "7.0.1", - "description": "Mastodon API client for node.js and browser", - "main": "./lib/src/index.js", - "typings": "./lib/src/index.d.ts", - "scripts": { - "build": "tsc -p ./", - "doc": "typedoc --out ../docs ./src", - "test": "cross-env NODE_ENV=test jest -u --maxWorkers=3" - }, - "engines": { - "node": "^22.0.0" - }, - "repository": { - "type": "git", - "url": "git+https://github.com/h3poteto/megalodon.git" - }, - "keywords": [ - "mastodon", - "client", - "api", - "streaming", - "rest", - "proxy" - ], - "author": "h3poteto", - "license": "MIT", - "bugs": { - "url": "https://github.com/h3poteto/megalodon/issues" - }, - "jest": { - "moduleFileExtensions": [ - "ts", - "js" - ], - "moduleNameMapper": { - "^@/(.+)": "<rootDir>/src/$1", - "^~/(.+)": "<rootDir>/$1" - }, - "testMatch": [ - "**/test/**/*.spec.ts" - ], - "preset": "ts-jest/presets/default", - "transform": { - "^.+\\.(ts|tsx)$": [ - "ts-jest", - { - "tsconfig": "tsconfig.json" - } - ] - }, - "testEnvironment": "node" - }, - "homepage": "https://github.com/h3poteto/megalodon#readme", - "dependencies": { - "@types/jest": "^29.5.10", - "@types/oauth": "^0.9.4", - "axios": "1.7.4", - "dayjs": "^1.11.10", - "form-data": "4.0.2", - "oauth": "0.10.2", + "name": "megalodon", + "version": "7.0.1", + "description": "Mastodon API client for node.js and browser", + "main": "./lib/src/index.js", + "typings": "./lib/src/index.d.ts", + "scripts": { + "build": "tsc -p ./", + "test": "cross-env NODE_ENV=test jest -u --maxWorkers=3" + }, + "engines": { + "node": "^22.0.0" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/h3poteto/megalodon.git" + }, + "keywords": [ + "mastodon", + "client", + "api", + "streaming", + "rest", + "proxy" + ], + "author": "h3poteto", + "license": "MIT", + "bugs": { + "url": "https://github.com/h3poteto/megalodon/issues" + }, + "jest": { + "moduleFileExtensions": [ + "ts", + "js" + ], + "moduleNameMapper": { + "^@/(.+)": "<rootDir>/src/$1", + "^~/(.+)": "<rootDir>/$1" + }, + "testMatch": [ + "**/test/**/*.spec.ts" + ], + "preset": "ts-jest/presets/default", + "transform": { + "^.+\\.(ts|tsx)$": [ + "ts-jest", + { + "tsconfig": "tsconfig.json" + } + ] + }, + "testEnvironment": "node" + }, + "homepage": "https://github.com/h3poteto/megalodon#readme", + "dependencies": { + "axios": "1.9.0", + "dayjs": "1.11.13", + "form-data": "4.0.2", + "oauth": "0.10.2", "typescript": "5.8.3" - }, - "devDependencies": { - "@typescript-eslint/eslint-plugin": "8.31.0", - "@typescript-eslint/parser": "8.31.0", - "eslint": "9.25.1", - "eslint-config-prettier": "^9.0.0", - "jest": "29.7.0", - "jest-worker": "29.7.0", - "prettier": "3.5.3", - "ts-jest": "^29.1.1" - } + }, + "devDependencies": { + "@types/jest": "29.5.14", + "@types/oauth": "0.9.6", + "jest": "29.7.0", + "jest-worker": "29.7.0", + "ts-jest": "29.3.4" + } } diff --git a/packages/megalodon/src/index.ts b/packages/megalodon/src/index.ts index bacd0574d4..e13e1a1faf 100644 --- a/packages/megalodon/src/index.ts +++ b/packages/megalodon/src/index.ts @@ -12,17 +12,17 @@ import MastodonEntity from './mastodon/entity'; import MisskeyEntity from './misskey/entity'; export { - Response, - OAuth, - RequestCanceledError, - isCancel, - detector, - MegalodonInterface, - NotificationType, - FilterContext, - Misskey, - Entity, - Converter, - MastodonEntity, - MisskeyEntity, + type Response, + OAuth, + RequestCanceledError, + isCancel, + detector, + type MegalodonInterface, + NotificationType, + FilterContext, + Misskey, + type Entity, + Converter, + type MastodonEntity, + type MisskeyEntity, } diff --git a/packages/megalodon/tsconfig.json b/packages/megalodon/tsconfig.json index 2b90738318..0a30b30cff 100644 --- a/packages/megalodon/tsconfig.json +++ b/packages/megalodon/tsconfig.json @@ -1,65 +1,105 @@ { - "compilerOptions": { - /* Basic Options */ - "target": "ES2022", /* Specify ECMAScript target version: 'ES3' (default), 'ES5', 'ES2015', 'ES2016', 'ES2017','ES2018' or 'ESNEXT'. */ - "module": "commonjs", /* Specify module code generation: 'none', 'commonjs', 'amd', 'system', 'umd', 'es2015', or 'ESNext'. */ - "lib": ["ES2022", "dom"], /* Specify library files to be included in the compilation. */ - // "allowJs": true, /* Allow javascript files to be compiled. */ - // "checkJs": true, /* Report errors in .js files. */ - // "jsx": "preserve", /* Specify JSX code generation: 'preserve', 'react-native', or 'react'. */ - "declaration": true, /* Generates corresponding '.d.ts' file. */ - // "declarationMap": true, /* Generates a sourcemap for each corresponding '.d.ts' file. */ - // "sourceMap": true, /* Generates corresponding '.map' file. */ - // "outFile": "./", /* Concatenate and emit output to single file. */ - "outDir": "./lib", /* Redirect output structure to the directory. */ - "rootDir": "./", /* Specify the root directory of input files. Use to control the output directory structure with --outDir. */ - // "composite": true, /* Enable project compilation */ - "removeComments": true, /* Do not emit comments to output. */ - // "noEmit": true, /* Do not emit outputs. */ - // "importHelpers": true, /* Import emit helpers from 'tslib'. */ - "downlevelIteration": true, /* Provide full support for iterables in 'for-of', spread, and destructuring when targeting 'ES5' or 'ES3'. */ - // "isolatedModules": true, /* Transpile each file as a separate module (similar to 'ts.transpileModule'). */ + "compilerOptions": { + /* Basic Options */ + "target": "ES2022", + /* Specify ECMAScript target version: 'ES3' (default), 'ES5', 'ES2015', 'ES2016', 'ES2017','ES2018' or 'ESNEXT'. */ + "module": "commonjs", + /* Specify module code generation: 'none', 'commonjs', 'amd', 'system', 'umd', 'es2015', or 'ESNext'. */ + "lib": [ + "ES2022", + "dom" + ], + /* Specify library files to be included in the compilation. */ + // "allowJs": true, /* Allow javascript files to be compiled. */ + // "checkJs": true, /* Report errors in .js files. */ + // "jsx": "preserve", /* Specify JSX code generation: 'preserve', 'react-native', or 'react'. */ + "declaration": true, + /* Generates corresponding '.d.ts' file. */ + "declarationMap": true, + /* Generates a sourcemap for each corresponding '.d.ts' file. */ + // "sourceMap": true, /* Generates corresponding '.map' file. */ + // "outFile": "./", /* Concatenate and emit output to single file. */ + "outDir": "./lib", + /* Redirect output structure to the directory. */ + "rootDir": "./", + /* Specify the root directory of input files. Use to control the output directory structure with --outDir. */ + // "composite": true, /* Enable project compilation */ + "removeComments": true, + /* Do not emit comments to output. */ + // "noEmit": true, /* Do not emit outputs. */ + // "importHelpers": true, /* Import emit helpers from 'tslib'. */ + "downlevelIteration": false, + /* Provide full support for iterables in 'for-of', spread, and destructuring when targeting 'ES5' or 'ES3'. */ + "isolatedModules": true, + /* Transpile each file as a separate module (similar to 'ts.transpileModule'). */ + "incremental": true, + /* Strict Type-Checking Options */ + "strict": true, + /* Enable all strict type-checking options. */ + "noImplicitAny": true, + /* Raise error on expressions and declarations with an implied 'any' type. */ + "strictNullChecks": true, + /* Enable strict null checks. */ + "strictFunctionTypes": true, + /* Enable strict checking of function types. */ + "strictPropertyInitialization": true, + /* Enable strict checking of property initialization in classes. */ + "noImplicitThis": true, + /* Raise error on 'this' expressions with an implied 'any' type. */ + "alwaysStrict": true, + /* Parse in strict mode and emit "use strict" for each source file. */ - /* Strict Type-Checking Options */ - "strict": true, /* Enable all strict type-checking options. */ - "noImplicitAny": true, /* Raise error on expressions and declarations with an implied 'any' type. */ - "strictNullChecks": true, /* Enable strict null checks. */ - "strictFunctionTypes": true, /* Enable strict checking of function types. */ - "strictPropertyInitialization": true, /* Enable strict checking of property initialization in classes. */ - "noImplicitThis": true, /* Raise error on 'this' expressions with an implied 'any' type. */ - "alwaysStrict": true, /* Parse in strict mode and emit "use strict" for each source file. */ - - /* Additional Checks */ - "noUnusedLocals": true, /* Report errors on unused locals. */ - "noUnusedParameters": true, /* Report errors on unused parameters. */ - "noImplicitReturns": true, /* Report error when not all code paths in function return a value. */ - "noFallthroughCasesInSwitch": true, /* Report errors for fallthrough cases in switch statement. */ + /* Additional Checks */ + "noUnusedLocals": true, + /* Report errors on unused locals. */ + "noUnusedParameters": true, + /* Report errors on unused parameters. */ + "noImplicitReturns": true, + /* Report error when not all code paths in function return a value. */ + "noFallthroughCasesInSwitch": true, + /* Report errors for fallthrough cases in switch statement. */ "skipLibCheck": true, + /* Module Resolution Options */ + "moduleResolution": "node", + /* Specify module resolution strategy: 'node' (Node.js) or 'classic' (TypeScript pre-1.6). */ + "baseUrl": "./", + /* Base directory to resolve non-absolute module names. */ + "paths": { + "@*": [ + "src*" + ], + "~*": [ + "./*" + ] + }, + /* A series of entries which re-map imports to lookup locations relative to the 'baseUrl'. */ + // "rootDirs": [], /* List of root folders whose combined content represents the structure of the project at runtime. */ + // "typeRoots": [], /* List of folders to include type definitions from. */ + // "types": [], /* Type declaration files to be included in compilation. */ + // "allowSyntheticDefaultImports": true, /* Allow default imports from modules with no default export. This does not affect code emit, just typechecking. */ + "esModuleInterop": true, + /* Enables emit interoperability between CommonJS and ES Modules via creation of namespace objects for all imports. Implies 'allowSyntheticDefaultImports'. */ + // "preserveSymlinks": true, /* Do not resolve the real path of symlinks. */ - /* Module Resolution Options */ - "moduleResolution": "node", /* Specify module resolution strategy: 'node' (Node.js) or 'classic' (TypeScript pre-1.6). */ - "baseUrl": "./", /* Base directory to resolve non-absolute module names. */ - "paths": { - "@*": ["src*"], - "~*": ["./*"] - }, /* A series of entries which re-map imports to lookup locations relative to the 'baseUrl'. */ - // "rootDirs": [], /* List of root folders whose combined content represents the structure of the project at runtime. */ - // "typeRoots": [], /* List of folders to include type definitions from. */ - // "types": [], /* Type declaration files to be included in compilation. */ - // "allowSyntheticDefaultImports": true, /* Allow default imports from modules with no default export. This does not affect code emit, just typechecking. */ - "esModuleInterop": true /* Enables emit interoperability between CommonJS and ES Modules via creation of namespace objects for all imports. Implies 'allowSyntheticDefaultImports'. */ - // "preserveSymlinks": true, /* Do not resolve the real path of symlinks. */ - - /* Source Map Options */ - // "sourceRoot": "./", /* Specify the location where debugger should locate TypeScript files instead of source locations. */ - // "mapRoot": "./", /* Specify the location where debugger should locate map files instead of generated locations. */ - // "inlineSourceMap": true, /* Emit a single file with source maps instead of having a separate file. */ - // "inlineSources": true, /* Emit the source alongside the sourcemaps within a single file; requires '--inlineSourceMap' or '--sourceMap' to be set. */ + /* Source Map Options */ + // "sourceRoot": "./", /* Specify the location where debugger should locate TypeScript files instead of source locations. */ + // "mapRoot": "./", /* Specify the location where debugger should locate map files instead of generated locations. */ + "inlineSourceMap": false, + /* Emit a single file with source maps instead of having a separate file. */ + "inlineSources": false, + /* Emit the source alongside the sourcemaps within a single file; requires '--inlineSourceMap' or '--sourceMap' to be set. */ - /* Experimental Options */ - // "experimentalDecorators": true, /* Enables experimental support for ES7 decorators. */ - // "emitDecoratorMetadata": true, /* Enables experimental support for emitting type metadata for decorators. */ - }, - "include": ["./src", "./test"], - "exclude": ["node_modules", "example"] + /* Experimental Options */ + "experimentalDecorators": true + /* Enables experimental support for ES7 decorators. */ + // "emitDecoratorMetadata": true, /* Enables experimental support for emitting type metadata for decorators. */ + }, + "include": [ + "./src", + "./test" + ], + "exclude": [ + "node_modules", + "example" + ] } diff --git a/packages/misskey-bubble-game/build.js b/packages/misskey-bubble-game/build.js index 5d534cc6fd..93a695e649 100644 --- a/packages/misskey-bubble-game/build.js +++ b/packages/misskey-bubble-game/build.js @@ -99,7 +99,7 @@ async function watchSrc() { process.on('SIGHUP', resolve); process.on('SIGINT', resolve); process.on('SIGTERM', resolve); - process.on('uncaughtException', reject); + process.on('uncaughtExceptionMonitor', reject); process.on('exit', resolve); }).finally(async () => { await context.dispose(); diff --git a/packages/misskey-bubble-game/tsconfig.json b/packages/misskey-bubble-game/tsconfig.json index f467951ef6..3cf8bb037f 100644 --- a/packages/misskey-bubble-game/tsconfig.json +++ b/packages/misskey-bubble-game/tsconfig.json @@ -16,6 +16,7 @@ "noImplicitReturns": true, "esModuleInterop": true, "skipLibCheck": true, + "incremental": true, "typeRoots": [ "./node_modules/@types" ], diff --git a/packages/misskey-js/build.js b/packages/misskey-js/build.js index b794592815..76a95fb3f5 100644 --- a/packages/misskey-js/build.js +++ b/packages/misskey-js/build.js @@ -100,7 +100,7 @@ async function watchSrc() { process.on('SIGHUP', resolve); process.on('SIGINT', resolve); process.on('SIGTERM', resolve); - process.on('uncaughtException', reject); + process.on('uncaughtExceptionMonitor', reject); process.on('exit', resolve); }).finally(async () => { await context.dispose(); diff --git a/packages/misskey-js/generator/tsconfig.json b/packages/misskey-js/generator/tsconfig.json index d65042dc6d..0de2a7fe77 100644 --- a/packages/misskey-js/generator/tsconfig.json +++ b/packages/misskey-js/generator/tsconfig.json @@ -8,6 +8,7 @@ "strictFunctionTypes": true, "strictNullChecks": true, "esModuleInterop": true, + "incremental": true, "lib": [ "esnext", ] diff --git a/packages/misskey-js/package.json b/packages/misskey-js/package.json index 3341baf441..48d5912a07 100644 --- a/packages/misskey-js/package.json +++ b/packages/misskey-js/package.json @@ -1,7 +1,7 @@ { "type": "module", "name": "misskey-js", - "version": "2025.4.2", + "version": "2025.4.3", "description": "Misskey SDK for JavaScript", "license": "MIT", "main": "./built/index.js", @@ -36,6 +36,7 @@ }, "devDependencies": { "@microsoft/api-extractor": "7.52.5", + "@simplewebauthn/types": "12.0.0", "@swc/jest": "0.2.38", "@types/jest": "29.5.14", "@types/node": "22.15.2", @@ -47,7 +48,7 @@ "mock-socket": "9.3.1", "ncp": "2.0.0", "nodemon": "3.1.10", - "execa": "8.0.1", + "execa": "9.5.2", "tsd": "0.32.0", "typescript": "5.8.3", "esbuild": "0.25.3", @@ -57,7 +58,6 @@ "built" ], "dependencies": { - "@simplewebauthn/types": "12.0.0", "eventemitter3": "5.0.1", "reconnecting-websocket": "4.4.0" } diff --git a/packages/misskey-js/src/autogen/apiClientJSDoc.ts b/packages/misskey-js/src/autogen/apiClientJSDoc.ts index 0dfe042811..8827fe9c39 100644 --- a/packages/misskey-js/src/autogen/apiClientJSDoc.ts +++ b/packages/misskey-js/src/autogen/apiClientJSDoc.ts @@ -3840,7 +3840,7 @@ declare module '../api.js' { /** * No description provided. * - * **Credential required**: *Yes* / **Permission**: *read:account* + * **Credential required**: *No* */ request<E extends 'notes/polls/recommendation', P extends Endpoints[E]['req']>( endpoint: E, diff --git a/packages/misskey-js/src/autogen/types.ts b/packages/misskey-js/src/autogen/types.ts index 55302960dc..8debdec9c1 100644 --- a/packages/misskey-js/src/autogen/types.ts +++ b/packages/misskey-js/src/autogen/types.ts @@ -3317,7 +3317,7 @@ export type paths = { * notes/polls/recommendation * @description No description provided. * - * **Credential required**: *Yes* / **Permission**: *read:account* + * **Credential required**: *No* */ post: operations['notes___polls___recommendation']; }; @@ -4246,6 +4246,10 @@ export type components = { /** Format: url */ avatarUrl: string | null; avatarBlurhash: string | null; + /** @example Hi masters, I am Ai! */ + description: string | null; + /** Format: date-time */ + createdAt: string; avatarDecorations: { /** Format: id */ id: string; @@ -4281,6 +4285,7 @@ export type components = { iconUrl: string | null; faviconUrl: string | null; themeColor: string | null; + isSilenced: boolean; }; emojis: { [key: string]: string; @@ -4292,6 +4297,7 @@ export type components = { iconUrl: string | null; displayOrder: number; })[]; + attributionDomains: string[]; }; UserDetailedNotMeOnly: { /** Format: url */ @@ -4302,8 +4308,6 @@ export type components = { movedTo: string | null; alsoKnownAs: string[] | null; /** Format: date-time */ - createdAt: string; - /** Format: date-time */ updatedAt: string | null; /** Format: date-time */ lastFetchedAt: string | null; @@ -4317,8 +4321,6 @@ export type components = { isSilenced: boolean; /** @example false */ isSuspended: boolean; - /** @example Hi masters, I am Ai! */ - description: string | null; location: string | null; /** @example 2018-03-12 */ birthday: string | null; @@ -5294,6 +5296,7 @@ export type components = { rejectReports: boolean; rejectQuotes: boolean; moderationNote?: string | null; + isBubbled: boolean; }; GalleryPost: { /** @@ -6195,6 +6198,7 @@ export type operations = { assigneeId: string | null; reporter: components['schemas']['UserDetailedNotMe']; targetUser: components['schemas']['UserDetailedNotMe']; + targetInstance: components['schemas']['FederationInstance'] | null; assignee: components['schemas']['UserDetailedNotMe'] | null; forwarded: boolean; /** @enum {string|null} */ @@ -11209,6 +11213,7 @@ export type operations = { }]>; }; isModerator: boolean; + isAdministrator: boolean; isSystem: boolean; isSilenced: boolean; isSuspended: boolean; @@ -11231,6 +11236,7 @@ export type operations = { remoteFollowing: number; remoteFollowers: number; }; + signupReason: string | null; }; }; }; @@ -12917,7 +12923,14 @@ export type operations = { requestBody: { content: { 'application/json': { - uri: string; + uri?: string | null; + /** Format: misskey:id */ + userId?: string | null; + /** Format: misskey:id */ + noteId?: string | null; + expandCollectionItems?: boolean; + expandCollectionLimit?: number | null; + allowAnonymous?: boolean; }; }; }; @@ -19569,18 +19582,10 @@ export type operations = { 200: { content: { 'application/json': { - image?: { - link?: string; - url: string; - title?: string; - }; - paginationLinks?: { - self?: string; - first?: string; - next?: string; - last?: string; - prev?: string; - }; + type: string; + id?: string; + updated?: string; + author?: string; link?: string; title?: string; items: { @@ -19588,33 +19593,15 @@ export type operations = { guid?: string; title?: string; pubDate?: string; - creator?: string; - summary?: string; - content?: string; - isoDate?: string; - categories?: string[]; - contentSnippet?: string; - enclosure?: { - url: string; - length?: number; - type?: string; - }; + description?: string; + media: { + medium?: string; + url?: string; + type?: string; + lang?: string; + }[]; }[]; - feedUrl?: string; description?: string; - itunes?: { - image?: string; - owner?: { - name?: string; - email?: string; - }; - author?: string; - summary?: string; - explicit?: string; - categories?: string[]; - keywords?: string[]; - [key: string]: unknown; - }; }; }; }; @@ -24255,7 +24242,7 @@ export type operations = { /** @description OK (with results) */ 200: { content: { - 'application/json': Record<string, never>; + 'application/json': unknown; }; }; /** @description Client error */ @@ -25180,6 +25167,7 @@ export type operations = { defaultCWPriority?: 'default' | 'parent' | 'defaultParent' | 'parentDefault'; /** @enum {string} */ allowUnsignedFetch?: 'never' | 'always' | 'essential' | 'staff'; + attributionDomains?: string[]; }; }; }; @@ -27225,12 +27213,6 @@ export type operations = { untilDate?: number; /** @default false */ allowPartial?: boolean; - /** @default true */ - includeMyRenotes?: boolean; - /** @default true */ - includeRenotedMyNotes?: boolean; - /** @default true */ - includeLocalRenotes?: boolean; /** @default false */ withFiles?: boolean; /** @default true */ @@ -27493,7 +27475,7 @@ export type operations = { * notes/polls/recommendation * @description No description provided. * - * **Credential required**: *Yes* / **Permission**: *read:account* + * **Credential required**: *No* */ notes___polls___recommendation: { requestBody: { @@ -27505,6 +27487,10 @@ export type operations = { offset?: number; /** @default false */ excludeChannels?: boolean; + /** @default null */ + local?: boolean | null; + /** @default false */ + expired?: boolean; }; }; }; @@ -28642,12 +28628,6 @@ export type operations = { untilDate?: number; /** @default false */ allowPartial?: boolean; - /** @default true */ - includeMyRenotes?: boolean; - /** @default true */ - includeRenotedMyNotes?: boolean; - /** @default true */ - includeLocalRenotes?: boolean; /** @default false */ withFiles?: boolean; /** @default true */ @@ -28849,12 +28829,6 @@ export type operations = { /** @default false */ allowPartial?: boolean; /** @default true */ - includeMyRenotes?: boolean; - /** @default true */ - includeRenotedMyNotes?: boolean; - /** @default true */ - includeLocalRenotes?: boolean; - /** @default true */ withRenotes?: boolean; /** * @description Only show notes that have attached files. diff --git a/packages/misskey-js/tsconfig.json b/packages/misskey-js/tsconfig.json index 95128b8fab..e0603832c7 100644 --- a/packages/misskey-js/tsconfig.json +++ b/packages/misskey-js/tsconfig.json @@ -17,6 +17,7 @@ "esModuleInterop": true, "exactOptionalPropertyTypes": true, "skipLibCheck": true, + "incremental": true, "typeRoots": [ "./node_modules/@types" ], diff --git a/packages/misskey-reversi/build.js b/packages/misskey-reversi/build.js index 5d534cc6fd..93a695e649 100644 --- a/packages/misskey-reversi/build.js +++ b/packages/misskey-reversi/build.js @@ -99,7 +99,7 @@ async function watchSrc() { process.on('SIGHUP', resolve); process.on('SIGINT', resolve); process.on('SIGTERM', resolve); - process.on('uncaughtException', reject); + process.on('uncaughtExceptionMonitor', reject); process.on('exit', resolve); }).finally(async () => { await context.dispose(); diff --git a/packages/misskey-reversi/tsconfig.json b/packages/misskey-reversi/tsconfig.json index f467951ef6..3cf8bb037f 100644 --- a/packages/misskey-reversi/tsconfig.json +++ b/packages/misskey-reversi/tsconfig.json @@ -16,6 +16,7 @@ "noImplicitReturns": true, "esModuleInterop": true, "skipLibCheck": true, + "incremental": true, "typeRoots": [ "./node_modules/@types" ], diff --git a/packages/sw/tsconfig.json b/packages/sw/tsconfig.json index 112a932e58..3a78106e46 100644 --- a/packages/sw/tsconfig.json +++ b/packages/sw/tsconfig.json @@ -20,6 +20,7 @@ "resolveJsonModule": true, "isolatedModules": true, "skipLibCheck": true, + "incremental": true, "baseUrl": ".", "paths": { "@/*": ["./src/*"], diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 75771bb969..a8b6784dc6 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -12,39 +12,9 @@ importers: .: dependencies: - cssnano: - specifier: 7.0.6 - version: 7.0.6(postcss@8.5.3) - esbuild: - specifier: 0.25.3 - version: 0.25.3 - execa: - specifier: 9.5.2 - version: 9.5.2 - fast-glob: - specifier: 3.3.3 - version: 3.3.3 - glob: - specifier: 11.0.2 - version: 11.0.2 - ignore-walk: - specifier: 7.0.0 - version: 7.0.0 js-yaml: specifier: 4.1.0 version: 4.1.0 - postcss: - specifier: 8.5.3 - version: 8.5.3 - tar: - specifier: 7.4.3 - version: 7.4.3 - terser: - specifier: 5.39.0 - version: 5.39.0 - typescript: - specifier: 5.8.3 - version: 5.8.3 optionalDependencies: cypress: specifier: 14.3.2 @@ -52,7 +22,7 @@ importers: devDependencies: '@misskey-dev/eslint-plugin': specifier: 2.1.0 - version: 2.1.0(@eslint/compat@1.1.1)(@stylistic/eslint-plugin@4.2.0(eslint@9.25.1)(typescript@5.8.3))(@typescript-eslint/eslint-plugin@8.31.0(@typescript-eslint/parser@8.31.0(eslint@9.25.1)(typescript@5.8.3))(eslint@9.25.1)(typescript@5.8.3))(@typescript-eslint/parser@8.31.0(eslint@9.25.1)(typescript@5.8.3))(eslint-plugin-import@2.31.0(@typescript-eslint/parser@8.31.0(eslint@9.25.1)(typescript@5.8.3))(eslint@9.25.1))(eslint@9.25.1)(globals@16.0.0) + version: 2.1.0(@eslint/compat@1.1.1)(@stylistic/eslint-plugin@4.2.0(eslint@9.25.1)(typescript@5.8.3))(@typescript-eslint/eslint-plugin@8.31.0(@typescript-eslint/parser@8.31.0(eslint@9.25.1)(typescript@5.8.3))(eslint@9.25.1)(typescript@5.8.3))(@typescript-eslint/parser@8.31.0(eslint@9.25.1)(typescript@5.8.3))(eslint-plugin-import@2.31.0(@typescript-eslint/parser@8.31.0(eslint@9.25.1)(typescript@5.8.3))(eslint@9.25.1))(eslint@9.25.1)(globals@16.1.0) '@types/node': specifier: 22.15.2 version: 22.15.2 @@ -65,21 +35,51 @@ importers: cross-env: specifier: 7.0.3 version: 7.0.3 + cssnano: + specifier: 7.0.6 + version: 7.0.6(postcss@8.5.3) + esbuild: + specifier: 0.25.3 + version: 0.25.3 eslint: specifier: 9.25.1 version: 9.25.1 + execa: + specifier: 9.5.2 + version: 9.5.2 + fast-glob: + specifier: 3.3.3 + version: 3.3.3 + glob: + specifier: 11.0.2 + version: 11.0.2 globals: - specifier: 16.0.0 - version: 16.0.0 + specifier: 16.1.0 + version: 16.1.0 + ignore-walk: + specifier: 7.0.0 + version: 7.0.0 ncp: specifier: 2.0.0 version: 2.0.0 pnpm: - specifier: 10.10.0 - version: 10.10.0 + specifier: 9.6.0 + version: 9.6.0 + postcss: + specifier: 8.5.3 + version: 8.5.3 start-server-and-test: specifier: 2.0.11 version: 2.0.11 + tar: + specifier: 7.4.3 + version: 7.4.3 + terser: + specifier: 5.39.0 + version: 5.39.0 + typescript: + specifier: 5.8.3 + version: 5.8.3 packages/backend: dependencies: @@ -120,8 +120,8 @@ importers: specifier: 1.3.0 version: 1.3.0 '@misskey-dev/summaly': - specifier: 5.2.1 - version: 5.2.1 + specifier: npm:@transfem-org/summaly@5.2.2 + version: '@transfem-org/summaly@5.2.2' '@nestjs/common': specifier: 11.1.0 version: 11.1.0(reflect-metadata@0.2.2)(rxjs@7.8.2) @@ -149,15 +149,6 @@ importers: '@smithy/node-http-handler': specifier: 2.5.0 version: 2.5.0 - '@swc/cli': - specifier: 0.7.3 - version: 0.7.3(@swc/core@1.11.24)(chokidar@4.0.3) - '@swc/core': - specifier: 1.11.24 - version: 1.11.24 - '@transfem-org/sfm-js': - specifier: 0.24.6 - version: 0.24.6 '@twemoji/parser': specifier: 15.1.1 version: 15.1.1 @@ -171,11 +162,8 @@ importers: specifier: 7.0.1 version: 7.0.1 argon2: - specifier: ^0.40.1 - version: 0.40.1 - async-mutex: - specifier: 0.5.0 - version: 0.5.0 + specifier: 0.43.0 + version: 0.43.0 axios: specifier: 1.7.4 version: 1.7.4 @@ -185,9 +173,6 @@ importers: blurhash: specifier: 2.0.5 version: 2.0.5 - body-parser: - specifier: 1.20.3 - version: 1.20.3 bullmq: specifier: 5.51.1 version: 5.51.1 @@ -195,7 +180,7 @@ importers: specifier: 7.0.0 version: 7.0.0 canvas: - specifier: ^3.1.0 + specifier: 3.1.0 version: 3.1.0 cbor: specifier: 9.0.2 @@ -209,12 +194,9 @@ importers: cheerio: specifier: 1.0.0 version: 1.0.0 - chokidar: - specifier: 4.0.3 - version: 4.0.3 cli-highlight: - specifier: 2.1.11 - version: 2.1.11 + specifier: npm:@transfem-org/cli-highlight@2.1.12 + version: '@transfem-org/cli-highlight@2.1.12' color-convert: specifier: 2.0.1 version: 2.0.1 @@ -227,9 +209,15 @@ importers: deep-email-validator: specifier: 0.1.21 version: 0.1.21 - fast-xml-parser: - specifier: 4.4.1 - version: 4.4.1 + dom-serializer: + specifier: 2.0.0 + version: 2.0.0 + domhandler: + specifier: 5.0.3 + version: 5.0.3 + domutils: + specifier: 3.2.2 + version: 3.2.2 fastify: specifier: 5.3.2 version: 5.3.2 @@ -254,18 +242,15 @@ importers: got: specifier: 14.4.7 version: 14.4.7 - happy-dom: - specifier: 16.8.1 - version: 16.8.1 hpagent: specifier: 1.2.0 version: 1.2.0 htmlescape: specifier: 1.1.1 version: 1.1.1 - http-link-header: - specifier: 1.1.3 - version: 1.1.3 + htmlparser2: + specifier: 9.1.0 + version: 9.1.0 ioredis: specifier: 5.6.1 version: 5.6.1 @@ -287,9 +272,6 @@ importers: jsonld: specifier: 8.3.3 version: 8.3.3(web-streams-polyfill@4.0.0) - jsrsasign: - specifier: 11.1.0 - version: 11.1.0 juice: specifier: 11.0.1 version: 11.0.1 @@ -299,9 +281,9 @@ importers: meilisearch: specifier: 0.50.0 version: 0.50.0 - microformats-parser: - specifier: 2.0.2 - version: 2.0.2 + mfm-js: + specifier: npm:@transfem-org/sfm-js@0.24.6 + version: '@transfem-org/sfm-js@0.24.6' mime-types: specifier: 2.1.35 version: 2.1.35 @@ -312,7 +294,7 @@ importers: specifier: workspace:* version: link:../misskey-reversi moment: - specifier: ^2.30.1 + specifier: 2.30.1 version: 2.30.1 ms: specifier: 3.0.0-canary.1 @@ -329,24 +311,12 @@ importers: nodemailer: specifier: 6.10.1 version: 6.10.1 - oauth: - specifier: 0.10.2 - version: 0.10.2 - oauth2orize: - specifier: 1.12.0 - version: 1.12.0 - oauth2orize-pkce: - specifier: 0.1.2 - version: 0.1.2 os-utils: specifier: 0.0.14 version: 0.0.14 otpauth: specifier: 9.4.0 version: 9.4.0 - parse5: - specifier: 7.3.0 - version: 7.3.0 pg: specifier: 8.15.6 version: 8.15.6 @@ -360,11 +330,11 @@ importers: specifier: 2.7.0 version: 2.7.0 proxy-addr: - specifier: ^2.0.7 + specifier: 2.0.7 version: 2.0.7 psl: - specifier: ^1.13.0 - version: 1.13.0 + specifier: 1.15.0 + version: 1.15.0 pug: specifier: 3.0.3 version: 3.0.3 @@ -374,9 +344,6 @@ importers: random-seed: specifier: 0.3.0 version: 0.3.0 - ratelimiter: - specifier: 3.4.1 - version: 3.4.1 re2: specifier: 1.21.4 version: 1.21.4 @@ -392,12 +359,6 @@ importers: rename: specifier: 1.0.4 version: 1.0.4 - rss-parser: - specifier: 3.13.0 - version: 3.13.0 - rxjs: - specifier: 7.8.2 - version: 7.8.2 sanitize-html: specifier: 2.16.0 version: 2.16.0 @@ -413,9 +374,6 @@ importers: strict-event-emitter-types: specifier: 2.0.0 version: 2.0.0 - stringz: - specifier: 2.1.0 - version: 2.1.0 systeminformation: specifier: 5.25.11 version: 5.25.11 @@ -441,8 +399,8 @@ importers: specifier: 2.4.0 version: 2.4.0 uuid: - specifier: ^9.0.1 - version: 9.0.1 + specifier: 11.1.0 + version: 11.1.0 vary: specifier: 1.1.2 version: 1.1.2 @@ -550,6 +508,12 @@ importers: '@simplewebauthn/types': specifier: 12.0.0 version: 12.0.0 + '@swc/cli': + specifier: 0.7.3 + version: 0.7.3(@swc/core@1.11.24)(chokidar@4.0.3) + '@swc/core': + specifier: 1.11.24 + version: 1.11.24 '@swc/jest': specifier: 0.2.38 version: 0.2.38(@swc/core@1.11.24) @@ -562,9 +526,6 @@ importers: '@types/bcryptjs': specifier: 2.4.6 version: 2.4.6 - '@types/body-parser': - specifier: 1.19.5 - version: 1.19.5 '@types/color-convert': specifier: 2.0.4 version: 2.0.4 @@ -577,9 +538,6 @@ importers: '@types/htmlescape': specifier: 1.1.3 version: 1.1.3 - '@types/http-link-header': - specifier: 1.0.7 - version: 1.0.7 '@types/jest': specifier: 29.5.14 version: 29.5.14 @@ -617,10 +575,10 @@ importers: specifier: 8.11.14 version: 8.11.14 '@types/proxy-addr': - specifier: ^2.0.3 + specifier: 2.0.3 version: 2.0.3 '@types/psl': - specifier: ^1.1.3 + specifier: 1.1.3 version: 1.1.3 '@types/pug': specifier: 2.0.10 @@ -631,9 +589,6 @@ importers: '@types/random-seed': specifier: 0.3.5 version: 0.3.5 - '@types/ratelimiter': - specifier: 3.4.6 - version: 3.4.6 '@types/redis-info': specifier: 3.0.3 version: 3.0.3 @@ -661,9 +616,6 @@ importers: '@types/tmp': specifier: 0.2.6 version: 0.2.6 - '@types/uuid': - specifier: ^9.0.4 - version: 9.0.8 '@types/vary': specifier: 1.1.3 version: 1.1.3 @@ -689,8 +641,8 @@ importers: specifier: 2.31.0 version: 2.31.0(@typescript-eslint/parser@8.31.0(eslint@9.25.1)(typescript@5.8.3))(eslint@9.25.1) execa: - specifier: 8.0.1 - version: 8.0.1 + specifier: 9.5.2 + version: 9.5.2 fkill: specifier: 9.0.0 version: 9.0.0 @@ -728,17 +680,8 @@ importers: specifier: 2024.1.0 version: 2024.1.0 '@phosphor-icons/web': - specifier: ^2.0.3 - version: 2.1.1 - '@rollup/plugin-json': - specifier: 6.1.0 - version: 6.1.0(rollup@4.40.0) - '@rollup/plugin-replace': - specifier: 6.0.2 - version: 6.0.2(rollup@4.40.0) - '@rollup/pluginutils': - specifier: 5.1.4 - version: 5.1.4(rollup@4.40.0) + specifier: 2.1.2 + version: 2.1.2 '@ruffle-rs/ruffle': specifier: 0.1.0-nightly.2024.10.15 version: 0.1.0-nightly.2024.10.15 @@ -748,24 +691,9 @@ importers: '@syuilo/aiscript': specifier: 0.19.0 version: 0.19.0 - '@transfem-org/sfm-js': - specifier: 0.24.6 - version: 0.24.6 - '@twemoji/parser': - specifier: 15.1.1 - version: 15.1.1 - '@vitejs/plugin-vue': - specifier: 5.2.3 - version: 5.2.3(vite@6.3.3(@types/node@22.15.2)(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3))(vue@3.5.14(typescript@5.8.3)) - '@vue/compiler-sfc': - specifier: 3.5.14 - version: 3.5.14 aiscript-vscode: specifier: github:aiscript-dev/aiscript-vscode#v0.1.15 version: https://codeload.github.com/aiscript-dev/aiscript-vscode/tar.gz/c3cde89e79a41d93540cf8a48cd619c3f2dcb1b7 - astring: - specifier: 1.9.0 - version: 1.9.0 broadcast-channel: specifier: 7.1.0 version: 7.1.0 @@ -802,9 +730,6 @@ importers: date-fns: specifier: 4.1.0 version: 4.1.0 - estree-walker: - specifier: 3.0.3 - version: 3.0.3 eventemitter3: specifier: 5.0.1 version: 5.0.1 @@ -824,11 +749,8 @@ importers: specifier: 2.2.3 version: 2.2.3 katex: - specifier: 0.16.10 - version: 0.16.10 - magic-string: - specifier: 0.30.17 - version: 0.30.17 + specifier: 0.16.22 + version: 0.16.22 matter-js: specifier: 0.20.0 version: 0.20.0 @@ -842,23 +764,20 @@ importers: specifier: workspace:* version: link:../misskey-reversi moment: - specifier: ^2.30.1 + specifier: 2.30.1 version: 2.30.1 photoswipe: specifier: 5.4.4 version: 5.4.4 + promise-limit: + specifier: 2.7.0 + version: 2.7.0 punycode.js: specifier: 2.3.1 version: 2.3.1 - rollup: - specifier: 4.40.0 - version: 4.40.0 sanitize-html: specifier: 2.16.0 version: 2.16.0 - sass: - specifier: 1.87.0 - version: 1.87.0 shiki: specifier: 3.3.0 version: 3.3.0 @@ -868,21 +787,12 @@ importers: textarea-caret: specifier: 3.1.0 version: 3.1.0 - three: - specifier: 0.176.0 - version: 0.176.0 throttle-debounce: specifier: 5.0.2 version: 5.0.2 tinycolor2: specifier: 1.6.0 version: 1.6.0 - tsc-alias: - specifier: 1.8.15 - version: 1.8.15 - tsconfig-paths: - specifier: 4.2.0 - version: 4.2.0 typescript: specifier: 5.8.3 version: 5.8.3 @@ -892,9 +802,6 @@ importers: v-code-diff: specifier: 1.13.1 version: 1.13.1(vue@3.5.14(typescript@5.8.3)) - vite: - specifier: 6.3.3 - version: 6.3.3(@types/node@22.15.2)(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3) vue: specifier: 3.5.14 version: 3.5.14(typescript@5.8.3) @@ -910,8 +817,17 @@ importers: version: 14.3.2 devDependencies: '@misskey-dev/summaly': - specifier: 5.2.1 - version: 5.2.1 + specifier: npm:@transfem-org/summaly@5.2.2 + version: '@transfem-org/summaly@5.2.2' + '@rollup/plugin-json': + specifier: 6.1.0 + version: 6.1.0(rollup@4.40.0) + '@rollup/plugin-replace': + specifier: 6.0.2 + version: 6.0.2(rollup@4.40.0) + '@rollup/pluginutils': + specifier: 5.1.4 + version: 5.1.4(rollup@4.40.0) '@storybook/addon-actions': specifier: 8.6.12 version: 8.6.12(storybook@8.6.12(bufferutil@4.0.9)(prettier@3.5.3)(utf-8-validate@6.0.5)) @@ -969,6 +885,9 @@ importers: '@testing-library/vue': specifier: 8.1.0 version: 8.1.0(@vue/compiler-sfc@3.5.14)(@vue/server-renderer@3.5.14(vue@3.5.14(typescript@5.8.3)))(vue@3.5.14(typescript@5.8.3)) + '@twemoji/parser': + specifier: 15.1.1 + version: 15.1.1 '@types/canvas-confetti': specifier: 1.9.0 version: 1.9.0 @@ -976,7 +895,7 @@ importers: specifier: 1.0.7 version: 1.0.7 '@types/katex': - specifier: ^0.16.7 + specifier: 0.16.7 version: 0.16.7 '@types/matter-js': specifier: 0.19.8 @@ -1011,18 +930,27 @@ importers: '@typescript-eslint/parser': specifier: 8.31.0 version: 8.31.0(eslint@9.25.1)(typescript@5.8.3) + '@vitejs/plugin-vue': + specifier: 5.2.3 + version: 5.2.3(vite@6.3.3(@types/node@22.15.2)(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3))(vue@3.5.14(typescript@5.8.3)) '@vitest/coverage-v8': specifier: 3.1.2 - version: 3.1.2(vitest@3.1.2(@types/debug@4.1.12)(@types/node@22.15.2)(happy-dom@17.4.4)(jsdom@26.1.0)(msw@2.7.5(@types/node@22.15.2)(typescript@5.8.3))(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3)) + version: 3.1.2(vitest@3.1.2(@types/debug@4.1.12)(@types/node@22.15.2)(happy-dom@17.4.4)(jsdom@26.1.0(bufferutil@4.0.9)(utf-8-validate@6.0.5))(msw@2.7.5(@types/node@22.15.2)(typescript@5.8.3))(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3)) '@vue/compiler-core': specifier: 3.5.14 version: 3.5.14 + '@vue/compiler-sfc': + specifier: 3.5.14 + version: 3.5.14 '@vue/runtime-core': specifier: 3.5.14 version: 3.5.14 acorn: specifier: 8.14.1 version: 8.14.1 + astring: + specifier: 1.9.0 + version: 1.9.0 cross-env: specifier: 7.0.3 version: 7.0.3 @@ -1032,6 +960,9 @@ importers: eslint-plugin-vue: specifier: 10.0.0 version: 10.0.0(eslint@9.25.1)(vue-eslint-parser@10.1.3(eslint@9.25.1)) + estree-walker: + specifier: 3.0.3 + version: 3.0.3 fast-glob: specifier: 3.3.3 version: 3.3.3 @@ -1041,6 +972,12 @@ importers: intersection-observer: specifier: 0.12.2 version: 0.12.2 + magic-string: + specifier: 0.30.17 + version: 0.30.17 + mfm-js: + specifier: npm:@transfem-org/sfm-js@0.24.6 + version: '@transfem-org/sfm-js@0.24.6' micromatch: specifier: 4.0.8 version: 4.0.8 @@ -1065,6 +1002,12 @@ importers: react-dom: specifier: 19.1.0 version: 19.1.0(react@19.1.0) + rollup: + specifier: 4.40.0 + version: 4.40.0 + sass: + specifier: 1.87.0 + version: 1.87.0 seedrandom: specifier: 3.0.5 version: 3.0.5 @@ -1077,12 +1020,24 @@ importers: storybook-addon-misskey-theme: specifier: github:misskey-dev/storybook-addon-misskey-theme version: https://codeload.github.com/misskey-dev/storybook-addon-misskey-theme/tar.gz/cf583db098365b2ccc81a82f63ca9c93bc32b640(@storybook/blocks@8.6.12(react-dom@19.1.0(react@19.1.0))(react@19.1.0)(storybook@8.6.12(bufferutil@4.0.9)(prettier@3.5.3)(utf-8-validate@6.0.5)))(@storybook/components@8.6.12(storybook@8.6.12(bufferutil@4.0.9)(prettier@3.5.3)(utf-8-validate@6.0.5)))(@storybook/core-events@8.6.12(storybook@8.6.12(bufferutil@4.0.9)(prettier@3.5.3)(utf-8-validate@6.0.5)))(@storybook/manager-api@8.6.12(storybook@8.6.12(bufferutil@4.0.9)(prettier@3.5.3)(utf-8-validate@6.0.5)))(@storybook/preview-api@8.6.12(storybook@8.6.12(bufferutil@4.0.9)(prettier@3.5.3)(utf-8-validate@6.0.5)))(@storybook/theming@8.6.12(storybook@8.6.12(bufferutil@4.0.9)(prettier@3.5.3)(utf-8-validate@6.0.5)))(@storybook/types@8.6.12(storybook@8.6.12(bufferutil@4.0.9)(prettier@3.5.3)(utf-8-validate@6.0.5)))(react-dom@19.1.0(react@19.1.0))(react@19.1.0) + three: + specifier: 0.176.0 + version: 0.176.0 + tsc-alias: + specifier: 1.8.15 + version: 1.8.15 + tsconfig-paths: + specifier: 4.2.0 + version: 4.2.0 + vite: + specifier: 6.3.3 + version: 6.3.3(@types/node@22.15.2)(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3) vite-plugin-turbosnap: specifier: 1.0.3 version: 1.0.3 vitest: specifier: 3.1.2 - version: 3.1.2(@types/debug@4.1.12)(@types/node@22.15.2)(happy-dom@17.4.4)(jsdom@26.1.0)(msw@2.7.5(@types/node@22.15.2)(typescript@5.8.3))(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3) + version: 3.1.2(@types/debug@4.1.12)(@types/node@22.15.2)(happy-dom@17.4.4)(jsdom@26.1.0(bufferutil@4.0.9)(utf-8-validate@6.0.5))(msw@2.7.5(@types/node@22.15.2)(typescript@5.8.3))(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3) vitest-fetch-mock: specifier: 0.4.5 version: 0.4.5(vitest@3.1.2(@types/debug@4.1.12)(@types/node@22.15.2)(happy-dom@17.4.4)(jsdom@26.1.0(bufferutil@4.0.9)(utf-8-validate@6.0.5))(msw@2.7.5(@types/node@22.15.2)(typescript@5.8.3))(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3)) @@ -1102,87 +1057,57 @@ importers: specifier: 15.1.0 version: 15.1.0 '@phosphor-icons/web': - specifier: ^2.0.3 - version: 2.1.1 - '@rollup/plugin-json': - specifier: 6.1.0 - version: 6.1.0(rollup@4.40.0) - '@rollup/plugin-replace': - specifier: 6.0.2 - version: 6.0.2(rollup@4.40.0) - '@rollup/pluginutils': - specifier: 5.1.4 - version: 5.1.4(rollup@4.40.0) - '@transfem-org/sfm-js': - specifier: 0.24.5 - version: 0.24.5 - '@twemoji/parser': - specifier: 15.1.1 - version: 15.1.1 - '@vitejs/plugin-vue': - specifier: 5.2.3 - version: 5.2.3(vite@6.3.3(@types/node@22.15.2)(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3))(vue@3.5.14(typescript@5.8.3)) - '@vue/compiler-sfc': - specifier: 3.5.14 - version: 3.5.14 - astring: - specifier: 1.9.0 - version: 1.9.0 + specifier: 2.1.2 + version: 2.1.2 buraha: specifier: 0.0.1 version: 0.0.1 - estree-walker: - specifier: 3.0.3 - version: 3.0.3 frontend-shared: specifier: workspace:* version: link:../frontend-shared json5: specifier: 2.2.3 version: 2.2.3 + mfm-js: + specifier: npm:@transfem-org/sfm-js@0.24.6 + version: '@transfem-org/sfm-js@0.24.6' misskey-js: specifier: workspace:* version: link:../misskey-js punycode.js: specifier: 2.3.1 version: 2.3.1 - rollup: - specifier: 4.40.0 - version: 4.40.0 - sass: - specifier: 1.87.0 - version: 1.87.0 shiki: specifier: 3.3.0 version: 3.3.0 tinycolor2: specifier: 1.6.0 version: 1.6.0 - tsc-alias: - specifier: 1.8.15 - version: 1.8.15 - tsconfig-paths: - specifier: 4.2.0 - version: 4.2.0 - typescript: - specifier: 5.8.3 - version: 5.8.3 uuid: specifier: 11.1.0 version: 11.1.0 - vite: - specifier: 6.3.3 - version: 6.3.3(@types/node@22.15.2)(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3) vue: specifier: 3.5.14 version: 3.5.14(typescript@5.8.3) devDependencies: '@misskey-dev/summaly': - specifier: 5.2.1 - version: 5.2.1 + specifier: npm:@transfem-org/summaly@5.2.2 + version: '@transfem-org/summaly@5.2.2' + '@rollup/plugin-json': + specifier: 6.1.0 + version: 6.1.0(rollup@4.40.0) + '@rollup/plugin-replace': + specifier: 6.0.2 + version: 6.0.2(rollup@4.40.0) + '@rollup/pluginutils': + specifier: 5.1.4 + version: 5.1.4(rollup@4.40.0) '@testing-library/vue': specifier: 8.1.0 version: 8.1.0(@vue/compiler-sfc@3.5.14)(@vue/server-renderer@3.5.14(vue@3.5.14(typescript@5.8.3)))(vue@3.5.14(typescript@5.8.3)) + '@twemoji/parser': + specifier: 15.1.1 + version: 15.1.1 '@types/estree': specifier: 1.0.7 version: 1.0.7 @@ -1207,15 +1132,24 @@ importers: '@typescript-eslint/parser': specifier: 8.31.0 version: 8.31.0(eslint@9.25.1)(typescript@5.8.3) + '@vitejs/plugin-vue': + specifier: 5.2.3 + version: 5.2.3(vite@6.3.3(@types/node@22.15.2)(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3))(vue@3.5.14(typescript@5.8.3)) '@vitest/coverage-v8': specifier: 3.1.2 - version: 3.1.2(vitest@3.1.2(@types/debug@4.1.12)(@types/node@22.15.2)(happy-dom@17.4.4)(jsdom@26.1.0)(msw@2.7.5(@types/node@22.15.2)(typescript@5.8.3))(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3)) + version: 3.1.2(vitest@3.1.2(@types/debug@4.1.12)(@types/node@22.15.2)(happy-dom@17.4.4)(jsdom@26.1.0(bufferutil@4.0.9)(utf-8-validate@6.0.5))(msw@2.7.5(@types/node@22.15.2)(typescript@5.8.3))(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3)) + '@vue/compiler-sfc': + specifier: 3.5.14 + version: 3.5.14 '@vue/runtime-core': specifier: 3.5.14 version: 3.5.14 acorn: specifier: 8.14.1 version: 8.14.1 + astring: + specifier: 1.9.0 + version: 1.9.0 cross-env: specifier: 7.0.3 version: 7.0.3 @@ -1225,6 +1159,9 @@ importers: eslint-plugin-vue: specifier: 10.0.0 version: 10.0.0(eslint@9.25.1)(vue-eslint-parser@10.1.3(eslint@9.25.1)) + estree-walker: + specifier: 3.0.3 + version: 3.0.3 fast-glob: specifier: 3.3.3 version: 3.3.3 @@ -1246,9 +1183,27 @@ importers: prettier: specifier: 3.5.3 version: 3.5.3 + rollup: + specifier: 4.40.0 + version: 4.40.0 + sass: + specifier: 1.87.0 + version: 1.87.0 start-server-and-test: specifier: 2.0.11 version: 2.0.11 + tsc-alias: + specifier: 1.8.15 + version: 1.8.15 + tsconfig-paths: + specifier: 4.2.0 + version: 4.2.0 + typescript: + specifier: 5.8.3 + version: 5.8.3 + vite: + specifier: 6.3.3 + version: 6.3.3(@types/node@22.15.2)(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3) vite-plugin-turbosnap: specifier: 1.0.3 version: 1.0.3 @@ -1267,9 +1222,6 @@ importers: misskey-js: specifier: workspace:* version: link:../misskey-js - nodemon: - specifier: 3.1.7 - version: 3.1.7 vue: specifier: 3.5.13 version: 3.5.13(typescript@5.8.3) @@ -1289,6 +1241,9 @@ importers: eslint-plugin-vue: specifier: 10.0.0 version: 10.0.0(eslint@9.25.1)(vue-eslint-parser@10.1.3(eslint@9.25.1)) + nodemon: + specifier: 3.1.10 + version: 3.1.10 typescript: specifier: 5.8.3 version: 5.8.3 @@ -1298,18 +1253,12 @@ importers: packages/megalodon: dependencies: - '@types/jest': - specifier: ^29.5.10 - version: 29.5.12 - '@types/oauth': - specifier: ^0.9.4 - version: 0.9.6 axios: - specifier: 1.7.4 - version: 1.7.4 + specifier: 1.9.0 + version: 1.9.0(debug@4.4.0) dayjs: - specifier: ^1.11.10 - version: 1.11.10 + specifier: 1.11.13 + version: 1.11.13 form-data: specifier: 4.0.2 version: 4.0.2 @@ -1320,30 +1269,21 @@ importers: specifier: 5.8.3 version: 5.8.3 devDependencies: - '@typescript-eslint/eslint-plugin': - specifier: 8.31.0 - version: 8.31.0(@typescript-eslint/parser@8.31.0(eslint@9.25.1)(typescript@5.8.3))(eslint@9.25.1)(typescript@5.8.3) - '@typescript-eslint/parser': - specifier: 8.31.0 - version: 8.31.0(eslint@9.25.1)(typescript@5.8.3) - eslint: - specifier: 9.25.1 - version: 9.25.1 - eslint-config-prettier: - specifier: ^9.0.0 - version: 9.1.0(eslint@9.25.1) + '@types/jest': + specifier: 29.5.14 + version: 29.5.14 + '@types/oauth': + specifier: 0.9.6 + version: 0.9.6 jest: specifier: 29.7.0 version: 29.7.0(@types/node@22.15.2) jest-worker: specifier: 29.7.0 version: 29.7.0 - prettier: - specifier: 3.5.3 - version: 3.5.3 ts-jest: - specifier: ^29.1.1 - version: 29.1.2(@babel/core@7.24.7)(@jest/types@29.6.3)(babel-jest@29.7.0(@babel/core@7.24.7))(esbuild@0.25.3)(jest@29.7.0(@types/node@22.15.2))(typescript@5.8.3) + specifier: 29.3.4 + version: 29.3.4(@babel/core@7.24.7)(@jest/transform@29.7.0)(@jest/types@29.6.3)(babel-jest@29.7.0(@babel/core@7.24.7))(esbuild@0.25.3)(jest@29.7.0(@types/node@22.15.2))(typescript@5.8.3) packages/misskey-bubble-game: dependencies: @@ -1390,9 +1330,6 @@ importers: packages/misskey-js: dependencies: - '@simplewebauthn/types': - specifier: 12.0.0 - version: 12.0.0 eventemitter3: specifier: 5.0.1 version: 5.0.1 @@ -1403,6 +1340,9 @@ importers: '@microsoft/api-extractor': specifier: 7.52.5 version: 7.52.5(@types/node@22.15.2) + '@simplewebauthn/types': + specifier: 12.0.0 + version: 12.0.0 '@swc/jest': specifier: 0.2.38 version: 0.2.38(@swc/core@1.11.24) @@ -1422,8 +1362,8 @@ importers: specifier: 0.25.3 version: 0.25.3 execa: - specifier: 8.0.1 - version: 8.0.1 + specifier: 9.5.2 + version: 9.5.2 glob: specifier: 11.0.2 version: 11.0.2 @@ -1845,11 +1785,6 @@ packages: resolution: {integrity: sha512-EStJpq4OuY8xYfhGVXngigBJRWxftKX9ksiGDnmlY3o7B/V7KIAc9X4oiK87uPJSc/vs5L869bem5fhZa8caZw==} engines: {node: '>=6.9.0'} - '@babel/parser@7.25.7': - resolution: {integrity: sha512-aZn7ETtQsjjGG5HruveUK06cU3Hljuhd9Iojm4M8WWv3wLE6OkE5PWbDUkItmMgegmccaITudyuW5RPYrYlgWw==} - engines: {node: '>=6.0.0'} - hasBin: true - '@babel/parser@7.27.2': resolution: {integrity: sha512-QYLs8299NA7WM/bZAdp+CviYYkVoYXlDW2rzliy3chxd1PQjej7JORuMJDJXJUb9g0TT+B99EwaVLKmX+sPXWw==} engines: {node: '>=6.0.0'} @@ -2690,9 +2625,6 @@ packages: '@misskey-dev/sharp-read-bmp@1.3.0': resolution: {integrity: sha512-18K95y0tXTtwl4BVfQb0JCr/9KHoHOfTKUUmZ7ibjzbS4bR/kGKoRkADsrdqBllF3nvu7PQN8zjUoM4SWoBLBg==} - '@misskey-dev/summaly@5.2.1': - resolution: {integrity: sha512-fcFd7ssHAghRntewRROOpRxv+VH18uz85Kzg6pZK1EFyqPOXxf39ErRA9HnJSzPYQT6KJTqBWuKHbCGoFlceXg==} - '@msgpackr-extract/msgpackr-extract-darwin-arm64@3.0.2': resolution: {integrity: sha512-9bfjwDxIDWmmOKusUcqdS4Rw+SETlp9Dy39Xui9BEGEk19dDwH0jhipwFzEff/pFg95NKymc6TOTbRKcWeRqyQ==} cpu: [arm64] @@ -3146,8 +3078,8 @@ packages: resolution: {integrity: sha512-m7X9U6BG2+J+R1lSOdCiITLLrxm+cWlNI3HUFA92oLO77ObGNzaKdh8pMLqdZcshtkKuV84olNNXDfMc4FezBQ==} engines: {node: '>=10'} - '@phosphor-icons/web@2.1.1': - resolution: {integrity: sha512-QjrfbItu5Rb2i37GzsKxmrRHfZPTVk3oXSPBnQ2+oACDbQRWGAeB0AsvZw263n1nFouQuff+khOCtRbrc6+k+A==} + '@phosphor-icons/web@2.1.2': + resolution: {integrity: sha512-rPAR9o/bEcp4Cw4DEeZHXf+nlGCMNGkNDRizYHM47NLxz9vvEHp/Tt6FMK1NcWadzw/pFDPnRBGi/ofRya958A==} '@pkgjs/parseargs@0.11.0': resolution: {integrity: sha512-+1VkjdD0QBLPodGrJUeqarH8VAIvQODIbwh9XpP5Syisf7YoQgsJKPNFoqqLQlu+VQ/tVSshMR6loPMn8U+dPg==} @@ -3164,7 +3096,6 @@ packages: '@readme/json-schema-ref-parser@1.2.0': resolution: {integrity: sha512-Bt3QVovFSua4QmHa65EHUmh2xS0XJ3rgTEUPH998f4OW4VVJke3BuS16f+kM0ZLOGdvIrzrPRqwihuv5BAjtrA==} - deprecated: This package is no longer maintained. Please use `@apidevtools/json-schema-ref-parser` instead. '@readme/openapi-parser@2.7.0': resolution: {integrity: sha512-P8WSr8WTOxilnT89tcCRKWYsG/II4sAwt1a/DIWub8xTtkrG9cCBBy/IUcvc5X8oGWN82MwcTA3uEkDrXZd/7A==} @@ -4077,12 +4008,16 @@ packages: '@tokenizer/token@0.3.0': resolution: {integrity: sha512-OvjF+z51L3ov0OyAU0duzsYuvO01PH7x4t6DJx+guahgTnBHkhJdG7soQeTSFLWN3efnHyibZ4Z8l2EuWwJN3A==} - '@transfem-org/sfm-js@0.24.5': - resolution: {integrity: sha1-c9qJO12lIG+kovDGKjZmK2qPqcw=, tarball: https://activitypub.software/api/v4/projects/2/packages/npm/@transfem-org/sfm-js/-/@transfem-org/sfm-js-0.24.5.tgz} + '@transfem-org/cli-highlight@2.1.12': + resolution: {integrity: sha1-LSVFMGgZU9oQlHSVb5XEzOG+yeQ=, tarball: https://activitypub.software/api/v4/projects/229/packages/npm/@transfem-org/cli-highlight/-/@transfem-org/cli-highlight-2.1.12.tgz} + engines: {node: ^22.0.0} '@transfem-org/sfm-js@0.24.6': resolution: {integrity: sha1-7t+TkCd3PZk+RbbrGbZ/iMs2y7o=, tarball: https://activitypub.software/api/v4/projects/2/packages/npm/@transfem-org/sfm-js/-/@transfem-org/sfm-js-0.24.6.tgz} + '@transfem-org/summaly@5.2.2': + resolution: {integrity: sha1-MO7cCppxE0luitQqz9A6RiWHpco=, tarball: https://activitypub.software/api/v4/projects/217/packages/npm/@transfem-org/summaly/-/@transfem-org/summaly-5.2.2.tgz} + '@trysound/sax@0.2.0': resolution: {integrity: sha512-L7z9BgrNEcYyUYtF+HaEfiS5ebkh9jXqbszz7pC0hRBPaatV0XjSD3+eHrpqFemQfgwiFF0QPIarnIihIDn7OA==} engines: {node: '>=10.13.0'} @@ -4196,9 +4131,6 @@ packages: '@types/http-cache-semantics@4.0.4': resolution: {integrity: sha512-1m0bIFVc7eJWyve9S0RnuRgcQqF/Xd5QsUZAZeQFr1Q3/p9JWoQQEqmVy+DPTNpGXwhgIetAoYF8JSc33q29QA==} - '@types/http-link-header@1.0.7': - resolution: {integrity: sha512-snm5oLckop0K3cTDAiBnZDy6ncx9DJ3mCRDvs42C884MbVYPP74Tiq2hFsSDRTyjK6RyDYDIulPiW23ge+g5Lw==} - '@types/istanbul-lib-coverage@2.0.4': resolution: {integrity: sha512-z/QT1XN4K4KYuslS23k62yDIDLwLFkzxOuMplDtObz0+y7VqJCaO2o+SPwHCvLFZh7xazvvoor2tA/hPz9ee7g==} @@ -4208,9 +4140,6 @@ packages: '@types/istanbul-reports@3.0.1': resolution: {integrity: sha512-c3mAZEuK0lvBp8tmuL74XRKn1+y2dcwOUpH7x4WrF6gk1GIgiluDRgMYQtw2OFcBvAJWlt6ASU3tSqxp0Uu0Aw==} - '@types/jest@29.5.12': - resolution: {integrity: sha512-eDC8bTvT/QhYdxJAulQikueigY5AsdBRH2yDKW3yveW7svY3+DzN84/2NUgkw10RTiJbWqZrTtoGVdYlvFJdLw==} - '@types/jest@29.5.14': resolution: {integrity: sha512-ZN+4sdnLUbo8EVvVc2ao0GFW6oVrQRPn4K2lglySj7APvSrgzxHiNNK99us4WDMi57xxA2yggblIAMNhXOotLQ==} @@ -4316,9 +4245,6 @@ packages: '@types/range-parser@1.2.4': resolution: {integrity: sha512-EEhsLsD6UsDM1yFhAvy0Cjr6VwmpMWqFBCb9w07wVugF7w9nfajxLuVmngTIpgS6svCnm6Vaw+MZhoDCKnOfsw==} - '@types/ratelimiter@3.4.6': - resolution: {integrity: sha512-Bv6WLSXPGLVsBjkizXtn+ef78R92e36/DFQo2wXPTHtp1cYXF6rCULMqf9WcZPAtyMZMvQAtIPeYMA1xAyxghw==} - '@types/react@18.0.28': resolution: {integrity: sha512-RD0ivG1kEztNBdoAK7lekI9M+azSnitIn85h4iOiaLjaTrMjzslhaqCGaI4IyCJ1RljWiLCEu4jyrLLgqxBTew==} @@ -4808,9 +4734,6 @@ packages: resolution: {integrity: sha512-0qWUglt9JEqLFr3w1I1pbrChn1grhaiAR2ocX1PP/flRmxgtwTzPFFFnfIlD6aMOLQZgSuCRlidD70lvx8yhzg==} engines: {node: '>=14'} - any-promise@1.3.0: - resolution: {integrity: sha512-7UvmKalWRt1wgjL1RrGxoSJW/0QZFIegpeGvZG9kjp8vrRu55XTHbwnqq2GpXm9uLbcuhxm3IqX9OB4MZR1b2A==} - anymatch@3.1.3: resolution: {integrity: sha512-KMReFUr0B4t+D+OBkjR3KYqvocp2XaSzO55UcB6mgQMd3KbcE+mWTyvVV7D/zsdEbNnV6acZUutkiHQXvTr1Rw==} engines: {node: '>= 8'} @@ -4839,8 +4762,8 @@ packages: arg@5.0.2: resolution: {integrity: sha512-PYjyFOLKQ9y57JvQ6QLo8dAgNqswh8M1RMJYdQduT6xbWSgK36P/Z/v+p888pM69jMMfS8Xd8F6I1kQ/I9HUGg==} - argon2@0.40.1: - resolution: {integrity: sha512-DjtHDwd7pm12qeWyfihHoM8Bn5vGcgH6sKwgPqwNYroRmxlrzadHEvMyuvQxN/V8YSyRRKD5x6ito09q1e9OyA==} + argon2@0.43.0: + resolution: {integrity: sha512-u/HKLcbWShVDhkfwI4hWyiUf3qyX8QhTfaIv2cWE18uqhXCmR5hb6Ed7oqYi2KCQegeAnRhiFzbjzm7i5yl1GA==} engines: {node: '>=16.17.0'} argparse@1.0.10: @@ -4926,9 +4849,6 @@ packages: resolution: {integrity: sha512-LElXdjswlqjWrPpJFg1Fx4wpkOCxj1TDHlSV4PlaRxHGWko024xICaa97ZkMfs6DRKlCguiAI+rbXv5GWwXIkg==} hasBin: true - async-mutex@0.5.0: - resolution: {integrity: sha512-1A94B18jkJ3DYq284ohPxoXbfTA5HsQ7/Mf4DEhcyLx3Bz27Rh59iScbB6EPiP+B+joue6YCxcMXSbFC1tZKwA==} - async@0.2.10: resolution: {integrity: sha512-eAkdoKxU6/LkKDBzLpT+t6Ff5EtfSF4wx1WfJiPEEV7WNLnDaRXk0oVysiEPm262roaachGexwUv94WhSgN5TQ==} @@ -4972,8 +4892,8 @@ packages: axios@1.7.4: resolution: {integrity: sha512-DukmaFRnY6AzAALSH4J2M3k6PkaC+MfaAGdEERRWcC9q3/TWQwLpHR8ZRLKTdQ3aBDL64EdluRDjJqKw+BPZEw==} - axios@1.8.4: - resolution: {integrity: sha512-eBSYY4Y68NNlHbHBMdeDmKNtDgXWhQsJcGqzO3iLUM0GraQFSS9cVgPX5I9b3lbdFKyYoAEGAZF1DwhTaljNAw==} + axios@1.9.0: + resolution: {integrity: sha512-re4CqKTJaURpzbLHtIi6XpDv20/CnpXOtjRY5/CU32L8gU8ek9UIivcfvSWvmKEngmVbrUtPpdDwWDWL7DNHvg==} b4a@1.6.4: resolution: {integrity: sha512-fpWrvyVHEKyeEvbKZTVOeZF3VSKKWtJxFIxX/jaVPf+cLbGUSitjb49pHLqPV2BUNNZ0LcoeEGfE/YCpyDYHIw==} @@ -5353,11 +5273,6 @@ packages: resolution: {integrity: sha512-I/zHAwsKf9FqGoXM4WWRACob9+SNukZTd94DWF57E4toouRulbCxcUh6RKUEOQlYTHJnzkPMySvPNaaSLNfLZw==} engines: {node: '>=8'} - cli-highlight@2.1.11: - resolution: {integrity: sha512-9KDcoEVwyUXrjcJNvHD0NFc/hiwe/WPVYIleQh2O1N2Zro5gWJZ/K+3DGn8w8P/F6FxOgzyC5bxDyHIgCSPhGg==} - engines: {node: '>=8.0.0', npm: '>=5.0.0'} - hasBin: true - cli-table3@0.6.5: resolution: {integrity: sha512-+W/5efTR7y5HRD7gACw9yQjqMVvEMLBHmboM/kPWam+H+Hmyrgjh6YncVKK122YZkXrLudzTuAukUw9FnMf7IQ==} engines: {node: 10.* || >= 12.*} @@ -5373,9 +5288,6 @@ packages: cliui@6.0.0: resolution: {integrity: sha512-t6wbgtoCXvAzst7QgXxJYqPt0usEfbgQdftEPbLL/cvv6HPE5VgvqCuAIDR0NgU52ds6rFwqrgakNLrHEjCbrQ==} - cliui@7.0.4: - resolution: {integrity: sha512-OcRE68cOsVMXp1Yvonl/fzkQOyjLSu/8bhPDfQt0e0/Eb283TKP20Fs2MqoPsr9SwA595rRCA+QMzYc9nBP+JQ==} - cliui@8.0.1: resolution: {integrity: sha512-BSeNnyus75C4//NQ9gQt1/csTXyo/8Sb+afLAkzAptFuMsod9HFokGNudZpi/oQV73hnVK+sR+5PVRMd+Dr7YQ==} engines: {node: '>=12'} @@ -5666,9 +5578,6 @@ packages: date-fns@4.1.0: resolution: {integrity: sha512-Ukq0owbQXxa/U3EGtsdVBkR1w7KOQ5gIBqdH2hkvknzZPYvBxb/aa6E8L7tmjFtkwZBu3UXBbjIgPo/Ez4xaNg==} - dayjs@1.11.10: - resolution: {integrity: sha512-vjAczensTgRcqDERK0SR2XMwsF/tSvnvlv6VcF2GIhg6Sx4yOIt/irsr1RDJsKiIyBzJDpCoXiWWq28MqH2cnQ==} - dayjs@1.11.13: resolution: {integrity: sha512-oaMBel6gjolK862uaPQOVTA7q3TZhuSvuMQAAglQDOWYO9A91IrAOUJEyKVlqJlHE0vq5p5UXxzdPfMH/x6xNg==} @@ -5898,11 +5807,8 @@ packages: domutils@2.8.0: resolution: {integrity: sha512-w96Cjofp72M5IIhpjgobBimYEfoPjx1Vx0BSX9P30WBdZW2WIKU0T1Bd0kz2eNZ9ikjKgHbEyKx8BB6H1L3h3A==} - domutils@3.0.1: - resolution: {integrity: sha512-z08c1l761iKhDFtfXO04C7kTdPBLi41zwOZl00WS8b5eiaebNpY00HKbztwBq+e3vyqWNwWF3mP9YLUeqIrF+Q==} - - domutils@3.1.0: - resolution: {integrity: sha512-H78uMmQtI2AhgDJjWeQmHwJJ2bLPD3GMmO7Zja/ZZh84wkm+4ut+IUnUdRa8uCGX88DiVx1j6FRe1XfxEgjEZA==} + domutils@3.2.2: + resolution: {integrity: sha512-6kZKyUajlDuqlHKVX1w7gyslj9MPIXzIFiz/rGu35uC1wMi+kMhQwGhl4lt9unC9Vb9INnY9Z3/ZA3+FhASLaw==} dotenv@16.5.0: resolution: {integrity: sha512-m/C+AwOAr9/W1UOIZUo232ejMNnJAJtYQjUbHoNTBNTJSvqzzDh7vnrei3o3r3m9blf6ZoDkvcw0VmozNRFJxg==} @@ -5932,6 +5838,11 @@ packages: ee-first@1.1.1: resolution: {integrity: sha512-WMwm9LhRUo+WUaRN+vRuETqG89IgZphVSNkdFgeb6sS/E4OrDIN7t48CAewSHXc6C8lefD8KKfr5vY61brQlow==} + ejs@3.1.10: + resolution: {integrity: sha512-UeJmFfOrAQS8OJWPZ4qtgHyWExa088/MtK5UEyoJGFH67cDEXkZSviOiKRCZ4Xij0zxI3JECgYs3oKx+AizQBA==} + engines: {node: '>=0.10.0'} + hasBin: true + electron-to-chromium@1.4.686: resolution: {integrity: sha512-3avY1B+vUzNxEgkBDpKOP8WarvUAEwpRaiCL0He5OKWEFxzaOFiq4WoZEZe7qh0ReS7DiWoHMnYoQCKxNZNzSg==} @@ -6082,12 +5993,6 @@ packages: resolution: {integrity: sha512-/veY75JbMK4j1yjvuUxuVsiS/hr/4iHs9FTT6cgTexxdE0Ly/glccBAkloH/DofkjRbZU3bnoj38mOmhkZ0lHw==} engines: {node: '>=12'} - eslint-config-prettier@9.1.0: - resolution: {integrity: sha512-NSWl5BFQWEPi1j4TjVNItzYV7dZXZ+wP6I6ZhrBGpChQhZRUaElihE9uRRkcbRnNb76UMKDF3r+WTmNcGPKsqw==} - hasBin: true - peerDependencies: - eslint: '>=7.0.0' - eslint-formatter-pretty@4.1.0: resolution: {integrity: sha512-IsUTtGxF1hrH6lMWiSl1WbGaiP01eT6kzywdY1U+zLc0MP+nwEnUiS9UI8IaOTUhTeQJLlCEWIbXINBH4YJbBQ==} engines: {node: '>=10'} @@ -6390,6 +6295,9 @@ packages: resolution: {integrity: sha512-hw9gNZXUfZ02Jo0uafWLaFVPter5/k2rfcrjFJJHX/77xtSDOfJuEFb6oKlFV86FLP1SuyHMW1PSk0U9M5tKkQ==} engines: {node: '>=18'} + filelist@1.0.4: + resolution: {integrity: sha512-w1cEuf3S+DrLCQL7ET6kz+gmlJdbq9J7yXCSjK/OZCPA+qEN1WyF4ZAf0YYJa4/shHJra2t/d/r8SV4Ji+x+8Q==} + filename-reserved-regex@3.0.0: resolution: {integrity: sha512-hn4cQfU6GOT/7cFHXBqeBg2TbrMBgdD0kcjLhvSQYYwm3s4B6cjvBfb7nBALJLAXqmU5xajSa7X2NnUud/VCdw==} engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0} @@ -6627,12 +6535,10 @@ packages: glob@7.2.3: resolution: {integrity: sha512-nFR0zLpU2YCaRxwoCJvL6UvCH2JFyFVIvwTLsIf21AuHlMskA1hhTdk+LlYJtOlYt9v6dvszD2BGRqBL+iQK9Q==} - deprecated: Glob versions prior to v9 are no longer supported glob@8.1.0: resolution: {integrity: sha512-r8hpEjiQEYlF2QU0df3dS+nxxSIreXQS1qRhMJM0Q5NDdR386C7jb7Hwwod8Fgiuex+k0GFjgft18yvxm5XoCQ==} engines: {node: '>=12'} - deprecated: Glob versions prior to v9 are no longer supported global-dirs@3.0.1: resolution: {integrity: sha512-NBcGGFbBA9s1VzD41QXDG+3++t9Mn5t1FpLdhESY6oKY4gYTFpX4wO3sqGUa0Srjtbfj3szX0RnemmrVRUdULA==} @@ -6646,8 +6552,8 @@ packages: resolution: {integrity: sha512-oahGvuMGQlPw/ivIYBjVSrWAfWLBeku5tpPE2fOPLi+WHffIWbuh2tCjhyQhTBPMf5E9jDEH4FOmTYgYwbKwtQ==} engines: {node: '>=18'} - globals@16.0.0: - resolution: {integrity: sha512-iInW14XItCXET01CQFqudPOWP2jYMl7T+QRQT+UNcR/iQncN/F0UNpgd76iFkBPgNQb4+X3LV9tLJYzwh+Gl3A==} + globals@16.1.0: + resolution: {integrity: sha512-aibexHNbb/jiUSObBgpHLj+sIuUmJnYcgXBlrfsiDZ9rt4aF2TFRbyLgZ2iFQuVZ1K5Mx3FVkbKRSgKrbK3K2g==} engines: {node: '>=18'} globalthis@1.0.3: @@ -6687,10 +6593,6 @@ packages: resolution: {integrity: sha512-tSQXBXS/MWQOn/RKckawJ61vvsDpCom87JgxiYdGwHdOa0ht0vzUWDlfioofFCRU0L+6NGDt6XzbgoJvZkMeRQ==} engines: {node: '>=0.8.0'} - happy-dom@16.8.1: - resolution: {integrity: sha512-n0QrmT9lD81rbpKsyhnlz3DgnMZlaOkJPpgi746doA+HvaMC79bdWkwjrNnGJRvDrWTI8iOcJiVTJ5CdT/AZRw==} - engines: {node: '>=18.0.0'} - happy-dom@17.4.4: resolution: {integrity: sha512-/Pb0ctk3HTZ5xEL3BZ0hK1AqDSAUuRQitOmROPHhfUYEWpmTImwfD8vFDGADmMAX0JYgbcgxWoLFKtsWhcpuVA==} engines: {node: '>=18.0.0'} @@ -6768,11 +6670,8 @@ packages: headers-polyfill@4.0.2: resolution: {integrity: sha512-EWGTfnTqAO2L/j5HZgoM/3z82L7necsJ0pO9Tp0X1wil3PDLrkypTBRgVO2ExehEEvUycejZD3FuRaXpZZc3kw==} - highlight.js@10.7.3: - resolution: {integrity: sha512-tzcUFauisWKNHaRkN4Wjl/ZA07gENAjFl3J/c480dprkGTg5EQstgaNFqBfUqCq54kZRIEcreTsAgF/m2quD7A==} - - highlight.js@11.10.0: - resolution: {integrity: sha512-SYVnVFswQER+zu1laSya563s+F8VDGt7o35d4utbamowvUNLLMovFqwCLSocpZTz3MgaSRA1IbqRWZv97dtErQ==} + highlight.js@11.11.1: + resolution: {integrity: sha512-Xwwo44whKBVCYoliBQwaPvtd/2tYFkRQtXDWj1nackaV2JPXx3L0+Jvd8/qCJ2p+ML0/XVkJ2q+Mr+UVdpJK5w==} engines: {node: '>=12.0.0'} hosted-git-info@2.8.9: @@ -6819,10 +6718,6 @@ packages: resolution: {integrity: sha512-FtwrG/euBzaEjYeRqOgly7G0qviiXoJWnvEH2Z1plBdXgbyjv34pHTSb9zoeHMyDy33+DWy5Wt9Wo+TURtOYSQ==} engines: {node: '>= 0.8'} - http-link-header@1.1.3: - resolution: {integrity: sha512-3cZ0SRL8fb9MUlU3mKM61FcQvPfXx2dBrZW3Vbg5CXa8jFlK8OaEpePenLe1oEXQduhz8b0QjsqfS59QP4AJDQ==} - engines: {node: '>=6.0.0'} - http-proxy-agent@7.0.2: resolution: {integrity: sha512-T1gkAiYYDWYx3V5Bmyu7HcfcvL7mUrTWiM6yOfa3PIphViJ/gFPbvidQ+veqSOHci/PxBcDabeUNCzpOODJZig==} engines: {node: '>= 14'} @@ -6929,7 +6824,6 @@ packages: inflight@1.0.6: resolution: {integrity: sha512-k92I/b08q4wvFscXCLvqfsHCrjrF7yiXsQuIVvVE7N82W3+aqpzuUdBbfhWcy/FZR3/4IgflMgKLOsvPDrGCJA==} - deprecated: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful. inherits@2.0.4: resolution: {integrity: sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==} @@ -7242,6 +7136,11 @@ packages: resolution: {integrity: sha512-cub8rahkh0Q/bw1+GxP7aeSe29hHHn2V4m29nnDlvCdlgU+3UGxkZp7Z53jLUdpX3jdTO0nJZUDl3xvbWc2Xog==} engines: {node: 20 || >=22} + jake@10.9.2: + resolution: {integrity: sha512-2P4SQ0HrLQ+fw6llpLnOaGAvN2Zu6778SJMrCUwns4fOoG9ayrTiZk3VV8sCPkVZF8ab0zksVpS8FDY5pRCNBA==} + engines: {node: '>=10'} + hasBin: true + jest-changed-files@29.7.0: resolution: {integrity: sha512-fEArFiwf1BpQ+4bXSprcDc3/x4HSzL4al2tozwVpDFpsxALjLYdyiIK4e5Vz66GQJIbXJ82+35PtysofptNX2w==} engines: {node: ^14.15.0 || ^16.10.0 || >=18.0.0} @@ -7494,9 +7393,6 @@ packages: resolution: {integrity: sha512-gqXddjPqQ6G40VdnI6T6yObEC+pDNvyP95wdQhkWkg7crHH3km5qP1FsOXEkzEQwnz6gz5qGTn1c2Y52wP3OyQ==} engines: {'0': node >=0.6.0} - jsrsasign@11.1.0: - resolution: {integrity: sha512-Ov74K9GihaK9/9WncTe1mPmvrO7Py665TUfUKvraXBpu+xcTWitrtuOwcjf4KMU9maPaYn0OuaWy0HOzy/GBXg==} - jstransformer@1.0.0: resolution: {integrity: sha512-C9YK3Rf8q6VAPDCCU9fnqo3mAfOH6vUGnMcP4AQAYIEpWtfGLpwOTmZ+igtdK5y+VvI2n3CyYSzy4Qh34eq24A==} @@ -7514,8 +7410,8 @@ packages: jws@4.0.0: resolution: {integrity: sha512-KDncfTmOZoOMTFG4mBlG0qUIOlc03fmzH+ru6RgYVZhPkyiy/92Owlt/8UEN+a4TXR1FQetfIpJE8ApdvdVxTg==} - katex@0.16.10: - resolution: {integrity: sha512-ZiqaC04tp2O5utMsl2TEZTXxa6WSC4yo0fv5ML++D3QZv/vx2Mct0mTlRx3O+uUkjfuAgOkzsCmq5MiUEsDDdA==} + katex@0.16.22: + resolution: {integrity: sha512-XCHRdUw4lf3SKBaJe4EvgqIuWwkPSo9XoeO8GjQW94Bp7TWv9hNhzZjZ+OH9yf1UmLygb7DIT5GSFQiyt16zYg==} hasBin: true keyv@4.5.4: @@ -7595,7 +7491,6 @@ packages: lodash.get@4.4.2: resolution: {integrity: sha512-z+Uw/vLuy6gQe8cfaFWD7p0wVv8fJl3mbzXh33RS+0oW2wvUqiRXiQ69gLWSLpgB5/6sU+r6BlQR0MBILadqTQ==} - deprecated: This package is deprecated. Use the optional chaining (?.) operator instead. lodash.isarguments@3.1.0: resolution: {integrity: sha512-chi4NHZlZqZD18a0imDHnZPrDeBbTtVN7GXMwuGdRH9qotxAjYs3aVLKc7zNOG9eddR5Ksd8rvFEBc9SsggPpg==} @@ -7795,10 +7690,6 @@ packages: resolution: {integrity: sha512-iclAHeNqNm68zFtnZ0e+1L2yUIdvzNoauKU4WBA3VvH/vPFieF7qfRlwUZU+DA9P9bPXIS90ulxoUoCH23sV2w==} engines: {node: '>= 0.6'} - microformats-parser@2.0.2: - resolution: {integrity: sha512-tUf9DmN4Jq/tGyp1YH2V6D/Cud+9Uc0WhjjUFirqVeHTRkkfLDacv6BQFT7h7HFsD0Z8wja5eKkRgzZU8bv0Fw==} - engines: {node: '>=18'} - micromark-core-commonmark@2.0.0: resolution: {integrity: sha512-jThOz/pVmAYUtkroV3D5c1osFXAMv9e0ypGDOIZuCeAe91/sD6BoE2Sjzt30yuXtwOYUmySOhMas/PVyh02itA==} @@ -8090,9 +7981,6 @@ packages: resolution: {integrity: sha512-+MrqnJRtxdF+xngFfUUkIMQrUUL0KsxbADUkn23Z/4ibGg192Q+z+CQyiYwvWTsYjJygmMR8+w3ZDa98Zh6ESg==} engines: {node: '>=12.0.0'} - mz@2.7.0: - resolution: {integrity: sha512-z81GNO7nnYMEhrGh9LeymoE4+Yr0Wn5McHIZMK5cfQCl+NDX08sCZgUc9/6MHni9IWuFLm1Z3HTCXu2z9fN62Q==} - nan@2.20.0: resolution: {integrity: sha512-bk3gXBZDGILuuo/6sKtr0DQmSThYHLtNCdSdXk9YkxD/jK6X2vmCyyXBBxyqZ4XcnzTyYEAThfX3DCEnLf6igw==} @@ -8157,10 +8045,13 @@ packages: resolution: {integrity: sha512-mNcltoe1R8o7STTegSOHdnJNN7s5EUvhoS7ShnTHDyOSd+8H+UdWODq6qSv67PjC8Zc5JRT8+oLAMCr0SIXw7g==} engines: {node: ^16 || ^18 || >= 20} + node-addon-api@8.3.1: + resolution: {integrity: sha512-lytcDEdxKjGJPTLEfW4mYMigRezMlyJY8W4wxJK8zE533Jlb8L8dRuObJFWg2P+AuOIxoCgKF+2Oq4d4Zd0OUA==} + engines: {node: ^18 || ^20 || >= 21} + node-domexception@1.0.0: resolution: {integrity: sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ==} engines: {node: '>=10.5.0'} - deprecated: Use your platform's native DOMException instead node-fetch@2.7.0: resolution: {integrity: sha512-c4FRfUm/dbcWZ7U+1Wq0AwCyFL+3nt2bEw05wfxSz+DWpWsitgmSgYmy2dQdWyKC1694ELPqMs/YzUSNozLt8A==} @@ -8183,6 +8074,10 @@ packages: resolution: {integrity: sha512-OSs33Z9yWr148JZcbZd5WiAXhh/n9z8TxQcdMhIOlpN9AhWpLfvVFO73+m77bBABQMaY9XSvIa+qk0jlI7Gcaw==} hasBin: true + node-gyp-build@4.8.4: + resolution: {integrity: sha512-LA4ZjwlnUblHVgq0oBF3Jl/6h/Nvs5fzBLwdEF4nuxnFdsfajde4WfxtJr3CaiH+F6ewcIB/q4jQ4UzPyid+CQ==} + hasBin: true + node-gyp@10.2.0: resolution: {integrity: sha512-sp3FonBAaFe4aYTcFdZUn2NYkbP7xroPGYvQmP4Nl5PxamznItBnNCgjrVTKrEfQynInMsJvZrdmqUnysCJ8rw==} engines: {node: ^16.14.0 || >=18.0.0} @@ -8206,11 +8101,6 @@ packages: engines: {node: '>=10'} hasBin: true - nodemon@3.1.7: - resolution: {integrity: sha512-hLj7fuMow6f0lbB0cD14Lz2xNjwsyruH251Pk4t/yIitCFJbmY1myuLlHm/q06aST4jg6EgAh74PIBBrRqpVAQ==} - engines: {node: '>=10'} - hasBin: true - nofilter@3.1.0: resolution: {integrity: sha512-l2NNj07e9afPnhAhvgVrCD/oy2Ai1yfLpuo3EpiO1jFTsB4sFz6oIfAfSZyQzVpkZQ9xS8ZS5g1jCBgq4Hwo0g==} engines: {node: '>=12.19'} @@ -8266,13 +8156,6 @@ packages: nwsapi@2.2.19: resolution: {integrity: sha512-94bcyI3RsqiZufXjkr3ltkI86iEl+I7uiHVDtcq9wJUTwYQJ5odHDeSzkkrRzi80jJ8MaeZgqKjH1bAWAFw9bA==} - oauth2orize-pkce@0.1.2: - resolution: {integrity: sha512-grto2UYhXHi9GLE3IBgBBbV87xci55+bCyjpVuxKyzol6I5Rg0K1MiTuXE+JZk54R86SG2wqXODMiZYHraPpxw==} - - oauth2orize@1.12.0: - resolution: {integrity: sha512-j4XtFDQUBsvUHPjUmvmNDUDMYed2MphMIJBhyxVVe8hGCjkuYnjIsW+D9qk8c5ciXRdnk6x6tEbiO6PLeOZdCQ==} - engines: {node: '>= 0.4.0'} - oauth@0.10.2: resolution: {integrity: sha512-JtFnB+8nxDEXgNyniwz573xxbKSOu3R8D40xQKqcjwJ2CDkYqUDI53o6IuzDJBx60Z8VKCm271+t8iFjakrl8Q==} @@ -8439,21 +8322,12 @@ packages: parse-srcset@1.0.2: resolution: {integrity: sha512-/2qh0lav6CmI15FzA3i/2Bzk2zCgQhGMkvhOhKNcBVQ1ldgpbfiNTVslmooUmWJcADi1f1kIeynbDRVzNlfR6Q==} - parse5-htmlparser2-tree-adapter@6.0.1: - resolution: {integrity: sha512-qPuWvbLgvDGilKc5BoicRovlT4MtYT6JfJyBOMDsKoiT+GiuP5qyrPCnR9HcPECIJJmZh5jRndyNThnhhb/vlA==} - parse5-htmlparser2-tree-adapter@7.0.0: resolution: {integrity: sha512-B77tOZrqqfUfnVcOrUvfdLbz4pu4RopLD/4vmu3HUPswwTA8OH0EMW9BlWR2B0RCoiZRAHEUu7IxeP1Pd1UU+g==} parse5-parser-stream@7.1.2: resolution: {integrity: sha512-JyeQc9iwFLn5TbvvqACIF/VXG6abODeB3Fwmv/TGdLk2LfbWkaySGY72at4+Ty7EkPZj854u4CrICqNk2qIbow==} - parse5@5.1.1: - resolution: {integrity: sha512-ugq4DFI0Ptb+WWjAdOK16+u/nHfiIrcE+sh8kZMaM0WllQKLI9rOUq6c2b7cwPkXdzfQESqvoqK6ug7U/Yyzug==} - - parse5@6.0.1: - resolution: {integrity: sha512-Ofn/CTFzRGTTxwpNEs9PP93gXShHcTq255nzRYSKe8AkVpZY7e1fpmTfOyoIvjP5HG7Z2ZM7VS9PPhQGW2pOpw==} - parse5@7.3.0: resolution: {integrity: sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw==} @@ -8633,8 +8507,8 @@ packages: resolution: {integrity: sha512-40QW5YalBNfQo5yRYmiw7Yz6TKKVr3h6970B2YE+3fQpsWcrbj1PzJgxeJ19DRQjhMbKPIuMY8rFaXc8moolVw==} engines: {node: '>=10.13.0'} - pnpm@10.10.0: - resolution: {integrity: sha512-1hXbJG/nDyXc/qbY1z3ueCziPiJF48T2+Igkn7VoFJMYY33Kc8LFyO8qTKDVZX+5VnGIv6tH9WbR7mzph4FcOQ==} + pnpm@9.6.0: + resolution: {integrity: sha512-ONxvuo26NbOTQLlwARLC/h4S8QsXE0cVpKqYzPe7A152/Zgc8Ls4TfqY+NavVIHCvvL0Jmokv6IMNOtxR84LXg==} engines: {node: '>=18.12'} hasBin: true @@ -8965,8 +8839,8 @@ packages: engines: {node: '>= 0.10'} hasBin: true - psl@1.13.0: - resolution: {integrity: sha512-BFwmFXiJoFqlUpZ5Qssolv15DMyc84gTBds1BjsV1BfXEo1UyyD7GsmN67n7J77uRhoSNW1AXtXKPLcBFQn9Aw==} + psl@1.15.0: + resolution: {integrity: sha512-JZd3gMVBAVQkSs6HdNZo9Sdo0LNcQeMNP3CozBJb3JYC/QUYZTnKxP+f8oWRX4rHP5EurWxqAHTSwUCjlNKa1w==} pstree.remy@1.1.8: resolution: {integrity: sha512-77DZwxQmxKnu3aR542U+X8FypNzbfJ+C5XQDk3uWjWxn6151aIMGthWYRXTqT1E5oJvg+ljaa2OJi+VfvCOQ8w==} @@ -9075,9 +8949,6 @@ packages: resolution: {integrity: sha512-Hrgsx+orqoygnmhFbKaHE6c296J+HTAQXoxEF6gNupROmmGJRoyzfG3ccAveqCBrwr/2yxQ5BVd/GTl5agOwSg==} engines: {node: '>= 0.6'} - ratelimiter@3.4.1: - resolution: {integrity: sha512-5FJbRW/Jkkdk29ksedAfWFkQkhbUrMx3QJGwMKAypeIiQf4yrLW+gtPKZiaWt4zPrtw1uGufOjGO7UGM6VllsQ==} - raw-body@2.5.2: resolution: {integrity: sha512-8zGqypfENjCIqGhgXToC8aB2r7YrBX+AQAfIPs/Mlk+BtPTztOvTS01NRW/3Eh60J+a48lt8qsCzirQ6loCVfA==} engines: {node: '>= 0.8'} @@ -9298,9 +9169,6 @@ packages: rrweb-cssom@0.8.0: resolution: {integrity: sha512-guoltQEx+9aMf2gDZ0s62EcV8lsXR+0w8915TC3ITdn2YueuNjdAYh/levpU9nFaoChh9RUS5ZdQMrKfVEN9tw==} - rss-parser@3.13.0: - resolution: {integrity: sha512-7jWUBV5yGN3rqMMj7CZufl/291QAhvrrGpDNE4k/02ZchL0npisiYYqULF71jCEKoIiHvK/Q2e6IkDwPziT7+w==} - run-parallel@1.2.0: resolution: {integrity: sha512-5l4VyZR86LZ/lDxZTR6jqL8AFE2S0IFLMP26AbjsLVADxHdhB/c0GUsH+y39UfCi3dzz8OlQuPmnaJOMoDHQBA==} @@ -9401,6 +9269,11 @@ packages: engines: {node: '>=10'} hasBin: true + semver@7.7.2: + resolution: {integrity: sha512-RF0Fw+rO5AMf9MAyaRXI4AV0Ulj5lMHqVxxdSgiVbixSCXoEmmX/jk0CuJw4+3SqroYO9VoUh+HcuJivvtJemA==} + engines: {node: '>=10'} + hasBin: true + send@0.19.0: resolution: {integrity: sha512-dW41u5VfLXu8SJh5bwRmyYUbAoSB3c9uQh6L8h/KtsFREPWpbX1lrljJo186Jc4nmci/sGUZ9a0a0J2zgfq2hw==} engines: {node: '>= 0.8.0'} @@ -9963,13 +9836,6 @@ packages: textarea-caret@3.1.0: resolution: {integrity: sha512-cXAvzO9pP5CGa6NKx0WYHl+8CHKZs8byMkt3PCJBCmq2a34YA9pO1NrQET5pzeqnBjBdToF5No4rrmkDUgQC2Q==} - thenify-all@1.6.0: - resolution: {integrity: sha512-RNxQH/qI8/t3thXJDwcstUO4zeqo64+Uy/+sNVRBx4Xn2OX+OZ9oP+iJnNFqplFra2ZUVeKCSa2oVWi3T4uVmA==} - engines: {node: '>=0.8'} - - thenify@3.3.1: - resolution: {integrity: sha512-RVZSIV5IG10Hk3enotrhvz0T9em6cyHBLkH/YAZuKqd8hRkKhSfCGIcP2KUY0EPxndzANBmNllzWPwak+bheSw==} - thread-stream@3.1.0: resolution: {integrity: sha512-OqyPZ9u96VohAyMfJykzmivOrY2wfMSf3C5TtFJVgN+Hm6aj+voFhlK+kZEIv2FBh1X6Xp3DlnCOfEQ3B2J86A==} @@ -10108,12 +9974,13 @@ packages: resolution: {integrity: sha512-q5W7tVM71e2xjHZTlgfTDoPF/SmqKG5hddq9SzR49CH2hayqRKJtQ4mtRlSxKaJlR/+9rEM+mnBHf7I2/BQcpQ==} engines: {node: '>=6.10'} - ts-jest@29.1.2: - resolution: {integrity: sha512-br6GJoH/WUX4pu7FbZXuWGKGNDuU7b8Uj77g/Sp7puZV6EXzuByl6JrECvm0MzVzSTkSHWTihsXt+5XYER5b+g==} - engines: {node: ^16.10.0 || ^18.0.0 || >=20.0.0} + ts-jest@29.3.4: + resolution: {integrity: sha512-Iqbrm8IXOmV+ggWHOTEbjwyCf2xZlUMv5npExksXohL+tk8va4Fjhb+X2+Rt9NBmgO7bJ8WpnMLOwih/DnMlFA==} + engines: {node: ^14.15.0 || ^16.10.0 || ^18.0.0 || >=20.0.0} hasBin: true peerDependencies: '@babel/core': '>=7.0.0-beta.0 <8' + '@jest/transform': ^29.0.0 '@jest/types': ^29.0.0 babel-jest: ^29.0.0 esbuild: '*' @@ -10122,6 +9989,8 @@ packages: peerDependenciesMeta: '@babel/core': optional: true + '@jest/transform': + optional: true '@jest/types': optional: true babel-jest: @@ -10201,6 +10070,10 @@ packages: resolution: {integrity: sha512-3IMSWgP7C5KSQqmo1wjhKrwsvXAtF33jO3QY+Uy++ia7hqvgSK6iXbbg5PbDBc1P2ZbNEDgejOrN4YooXvhwCw==} engines: {node: '>=16'} + type-fest@4.41.0: + resolution: {integrity: sha512-TeTSQ6H5YHvpqVwBRcnLDCBnDOHWYu7IvGbHT6N8AOymcr9PJGjc1GTtiWZTYg0NCgYwvnYWEkVChQAr9bjfwA==} + engines: {node: '>=16'} + type-is@1.6.18: resolution: {integrity: sha512-TkRKr9sUTxEH8MdfuCSP7VizJyzRNMjj2J2do2Jr3Kym598JVdEksuzPQCnlFPW4ky9Q+iA+ma9BGm06XQBy8g==} engines: {node: '>= 0.6'} @@ -10297,9 +10170,6 @@ packages: engines: {node: '>=14.17'} hasBin: true - uid2@0.0.4: - resolution: {integrity: sha512-IevTus0SbGwQzYh3+fRsAMTVVPOoIVufzacXcHPmdlle1jUpq7BRL+mw3dgeLanvGZdwwbWhRV6XrcFNdBmjWA==} - uid@2.0.2: resolution: {integrity: sha512-u3xV3X7uzvi5b1MncmZo3i2Aw222Zk1keqLA1YkHldREkAhAqi65wuPfe7lHx8H/Wzy+8CE7S7uS3jekIM5s8g==} engines: {node: '>=8'} @@ -10806,14 +10676,6 @@ packages: resolution: {integrity: sha512-EvGK8EJ3DhaHfbRlETOWAS5pO9MZITeauHKJyb8wyajUfQUenkIg2MvLDTZ4T/TgIcm3HU0TFBgWWboAZ30UHg==} engines: {node: '>=18'} - xml2js@0.5.0: - resolution: {integrity: sha512-drPFnkQJik/O+uPKpqSgr22mpuFHqKdbS835iAQrUC73L2F5WkboIRd63ai/2Yg6I1jzifPFKH2NTK+cfglkIA==} - engines: {node: '>=4.0.0'} - - xmlbuilder@11.0.1: - resolution: {integrity: sha512-fDlsI/kFEx7gLvbecc0/ohLG50fugQp8ryHzMTuW9vSa1GJ0XYWKnhsUx7oie3G98+r56aTQIUB4kht42R3JvA==} - engines: {node: '>=4.0'} - xmlchars@2.2.0: resolution: {integrity: sha512-JZnDKK8B0RCDw84FNdDAIpZK+JuJw+s7Lz8nksI7SIuU3UXJJslUthsi+uWBUYOwPFwW7W7PRLRfUKpxjtjFCw==} @@ -10854,10 +10716,6 @@ packages: resolution: {integrity: sha512-aePbxDmcYW++PaqBsJ+HYUFwCdv4LVvdnhBy78E57PIor8/OVvhMrADFFEDh8DHDFRv/O9i3lPhsENjO7QX0+A==} engines: {node: '>=8'} - yargs@16.2.0: - resolution: {integrity: sha512-D1mvvtDG0L5ft/jGWkLpG1+m0eQxOfaBvTNELraWj22wSVUMWxZUvYgJYcKh6jGGIkJFhH4IZPQhR4TKpc8mBw==} - engines: {node: '>=10'} - yargs@17.7.2: resolution: {integrity: sha512-7dSzzRQ++CKnNI/krKnYRV7JKKPUXMEh61soaHKg9mrWEhzFWhFnxPxGl+69cD1Ou63C13NUPCnmIcrvqCuM6w==} engines: {node: '>=12'} @@ -11560,10 +11418,6 @@ snapshots: js-tokens: 4.0.0 picocolors: 1.1.1 - '@babel/parser@7.25.7': - dependencies: - '@babel/types': 7.25.7 - '@babel/parser@7.27.2': dependencies: '@babel/types': 7.27.1 @@ -12593,7 +12447,7 @@ snapshots: '@misskey-dev/browser-image-resizer@2024.1.0': {} - '@misskey-dev/eslint-plugin@2.1.0(@eslint/compat@1.1.1)(@stylistic/eslint-plugin@4.2.0(eslint@9.25.1)(typescript@5.8.3))(@typescript-eslint/eslint-plugin@8.31.0(@typescript-eslint/parser@8.31.0(eslint@9.25.1)(typescript@5.8.3))(eslint@9.25.1)(typescript@5.8.3))(@typescript-eslint/parser@8.31.0(eslint@9.25.1)(typescript@5.8.3))(eslint-plugin-import@2.31.0(@typescript-eslint/parser@8.31.0(eslint@9.25.1)(typescript@5.8.3))(eslint@9.25.1))(eslint@9.25.1)(globals@16.0.0)': + '@misskey-dev/eslint-plugin@2.1.0(@eslint/compat@1.1.1)(@stylistic/eslint-plugin@4.2.0(eslint@9.25.1)(typescript@5.8.3))(@typescript-eslint/eslint-plugin@8.31.0(@typescript-eslint/parser@8.31.0(eslint@9.25.1)(typescript@5.8.3))(eslint@9.25.1)(typescript@5.8.3))(@typescript-eslint/parser@8.31.0(eslint@9.25.1)(typescript@5.8.3))(eslint-plugin-import@2.31.0(@typescript-eslint/parser@8.31.0(eslint@9.25.1)(typescript@5.8.3))(eslint@9.25.1))(eslint@9.25.1)(globals@16.1.0)': dependencies: '@eslint/compat': 1.1.1 '@stylistic/eslint-plugin': 4.2.0(eslint@9.25.1)(typescript@5.8.3) @@ -12601,7 +12455,7 @@ snapshots: '@typescript-eslint/parser': 8.31.0(eslint@9.25.1)(typescript@5.8.3) eslint: 9.25.1 eslint-plugin-import: 2.31.0(@typescript-eslint/parser@8.31.0(eslint@9.25.1)(typescript@5.8.3))(eslint@9.25.1) - globals: 16.0.0 + globals: 16.1.0 '@misskey-dev/sharp-read-bmp@1.3.0': dependencies: @@ -12609,16 +12463,6 @@ snapshots: decode-ico: 0.4.1 sharp: 0.34.1 - '@misskey-dev/summaly@5.2.1': - dependencies: - cheerio: 1.0.0 - escape-regexp: 0.0.1 - got: 14.4.7 - html-entities: 2.5.2 - iconv-lite: 0.6.3 - jschardet: 3.1.4 - private-ip: 3.0.2 - '@msgpackr-extract/msgpackr-extract-darwin-arm64@3.0.2': optional: true @@ -12840,7 +12684,7 @@ snapshots: '@opentelemetry/instrumentation': 0.57.1(@opentelemetry/api@1.9.0) '@opentelemetry/semantic-conventions': 1.28.0 forwarded-parse: 2.1.2 - semver: 7.6.3 + semver: 7.7.1 transitivePeerDependencies: - supports-color @@ -12973,7 +12817,7 @@ snapshots: '@types/shimmer': 1.2.0 import-in-the-middle: 1.11.2 require-in-the-middle: 7.3.0 - semver: 7.6.3 + semver: 7.7.1 shimmer: 1.2.1 transitivePeerDependencies: - supports-color @@ -12985,7 +12829,7 @@ snapshots: '@types/shimmer': 1.2.0 import-in-the-middle: 1.11.2 require-in-the-middle: 7.3.0 - semver: 7.6.3 + semver: 7.7.1 shimmer: 1.2.1 transitivePeerDependencies: - supports-color @@ -12997,7 +12841,7 @@ snapshots: '@types/shimmer': 1.2.0 import-in-the-middle: 1.11.2 require-in-the-middle: 7.3.0 - semver: 7.6.3 + semver: 7.7.1 shimmer: 1.2.1 transitivePeerDependencies: - supports-color @@ -13133,7 +12977,7 @@ snapshots: '@phc/format@1.0.0': {} - '@phosphor-icons/web@2.1.1': {} + '@phosphor-icons/web@2.1.2': {} '@pkgjs/parseargs@0.11.0': optional: true @@ -14308,14 +14152,27 @@ snapshots: '@tokenizer/token@0.3.0': {} - '@transfem-org/sfm-js@0.24.5': + '@transfem-org/cli-highlight@2.1.12': dependencies: - '@twemoji/parser': 15.0.0 + chalk: 5.4.1 + domhandler: 5.0.3 + highlight.js: 11.11.1 + htmlparser2: 9.1.0 '@transfem-org/sfm-js@0.24.6': dependencies: '@twemoji/parser': 15.0.0 + '@transfem-org/summaly@5.2.2': + dependencies: + cheerio: 1.0.0 + escape-regexp: 0.0.1 + got: 14.4.7 + html-entities: 2.5.2 + iconv-lite: 0.6.3 + jschardet: 3.1.4 + private-ip: 3.0.2 + '@trysound/sax@0.2.0': {} '@tsd/typescript@5.8.3': {} @@ -14436,10 +14293,6 @@ snapshots: '@types/http-cache-semantics@4.0.4': {} - '@types/http-link-header@1.0.7': - dependencies: - '@types/node': 22.15.2 - '@types/istanbul-lib-coverage@2.0.4': {} '@types/istanbul-lib-report@3.0.0': @@ -14450,11 +14303,6 @@ snapshots: dependencies: '@types/istanbul-lib-report': 3.0.0 - '@types/jest@29.5.12': - dependencies: - expect: 29.7.0 - pretty-format: 29.7.0 - '@types/jest@29.5.14': dependencies: expect: 29.7.0 @@ -14559,8 +14407,6 @@ snapshots: '@types/range-parser@1.2.4': {} - '@types/ratelimiter@3.4.6': {} - '@types/react@18.0.28': dependencies: '@types/prop-types': 15.7.14 @@ -14749,7 +14595,7 @@ snapshots: vite: 6.3.3(@types/node@22.15.2)(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3) vue: 3.5.14(typescript@5.8.3) - '@vitest/coverage-v8@3.1.2(vitest@3.1.2(@types/debug@4.1.12)(@types/node@22.15.2)(happy-dom@17.4.4)(jsdom@26.1.0)(msw@2.7.5(@types/node@22.15.2)(typescript@5.8.3))(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3))': + '@vitest/coverage-v8@3.1.2(vitest@3.1.2(@types/debug@4.1.12)(@types/node@22.15.2)(happy-dom@17.4.4)(jsdom@26.1.0(bufferutil@4.0.9)(utf-8-validate@6.0.5))(msw@2.7.5(@types/node@22.15.2)(typescript@5.8.3))(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3))': dependencies: '@ampproject/remapping': 2.3.0 '@bcoe/v8-coverage': 1.0.2 @@ -14763,7 +14609,7 @@ snapshots: std-env: 3.9.0 test-exclude: 7.0.1 tinyrainbow: 2.0.0 - vitest: 3.1.2(@types/debug@4.1.12)(@types/node@22.15.2)(happy-dom@17.4.4)(jsdom@26.1.0)(msw@2.7.5(@types/node@22.15.2)(typescript@5.8.3))(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3) + vitest: 3.1.2(@types/debug@4.1.12)(@types/node@22.15.2)(happy-dom@17.4.4)(jsdom@26.1.0(bufferutil@4.0.9)(utf-8-validate@6.0.5))(msw@2.7.5(@types/node@22.15.2)(typescript@5.8.3))(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3) transitivePeerDependencies: - supports-color @@ -15204,8 +15050,6 @@ snapshots: ansis@3.17.0: {} - any-promise@1.3.0: {} - anymatch@3.1.3: dependencies: normalize-path: 3.0.0 @@ -15242,11 +15086,11 @@ snapshots: arg@5.0.2: {} - argon2@0.40.1: + argon2@0.43.0: dependencies: '@phc/format': 1.0.0 - node-addon-api: 7.1.0 - node-gyp-build: 4.8.1 + node-addon-api: 8.3.1 + node-gyp-build: 4.8.4 argparse@1.0.10: dependencies: @@ -15350,10 +15194,6 @@ snapshots: astring@1.9.0: {} - async-mutex@0.5.0: - dependencies: - tslib: 2.6.2 - async@0.2.10: {} async@3.2.4: {} @@ -15402,7 +15242,7 @@ snapshots: transitivePeerDependencies: - debug - axios@1.8.4(debug@4.4.0): + axios@1.9.0(debug@4.4.0): dependencies: follow-redirects: 1.15.9(debug@4.4.0) form-data: 4.0.2 @@ -15859,14 +15699,14 @@ snapshots: css-what: 6.1.0 domelementtype: 2.3.0 domhandler: 5.0.3 - domutils: 3.1.0 + domutils: 3.2.2 cheerio@1.0.0: dependencies: cheerio-select: 2.1.0 dom-serializer: 2.0.0 domhandler: 5.0.3 - domutils: 3.1.0 + domutils: 3.2.2 encoding-sniffer: 0.2.0 htmlparser2: 9.1.0 parse5: 7.3.0 @@ -15905,15 +15745,6 @@ snapshots: restore-cursor: 3.1.0 optional: true - cli-highlight@2.1.11: - dependencies: - chalk: 4.1.2 - highlight.js: 10.7.3 - mz: 2.7.0 - parse5: 5.1.1 - parse5-htmlparser2-tree-adapter: 6.0.1 - yargs: 16.2.0 - cli-table3@0.6.5: dependencies: string-width: 4.2.3 @@ -15935,12 +15766,6 @@ snapshots: strip-ansi: 6.0.1 wrap-ansi: 6.2.0 - cliui@7.0.4: - dependencies: - string-width: 4.2.3 - strip-ansi: 6.0.1 - wrap-ansi: 7.0.0 - cliui@8.0.1: dependencies: string-width: 4.2.3 @@ -16139,7 +15964,7 @@ snapshots: boolbase: 1.0.0 css-what: 6.1.0 domhandler: 5.0.3 - domutils: 3.1.0 + domutils: 3.2.2 nth-check: 2.1.1 css-tree@2.2.1: @@ -16297,8 +16122,6 @@ snapshots: date-fns@4.1.0: {} - dayjs@1.11.10: {} - dayjs@1.11.13: {} de-indent@1.0.2: {} @@ -16317,11 +16140,9 @@ snapshots: dependencies: ms: 2.1.2 - debug@4.3.5(supports-color@5.5.0): + debug@4.3.5: dependencies: ms: 2.1.2 - optionalDependencies: - supports-color: 5.5.0 debug@4.4.0(supports-color@5.5.0): dependencies: @@ -16512,13 +16333,7 @@ snapshots: domelementtype: 2.3.0 domhandler: 4.3.1 - domutils@3.0.1: - dependencies: - dom-serializer: 2.0.0 - domelementtype: 2.3.0 - domhandler: 5.0.3 - - domutils@3.1.0: + domutils@3.2.2: dependencies: dom-serializer: 2.0.0 domelementtype: 2.3.0 @@ -16554,6 +16369,10 @@ snapshots: ee-first@1.1.1: {} + ejs@3.1.10: + dependencies: + jake: 10.9.2 + electron-to-chromium@1.4.686: {} electron-to-chromium@1.5.123: {} @@ -16758,10 +16577,6 @@ snapshots: escape-string-regexp@5.0.0: {} - eslint-config-prettier@9.1.0(eslint@9.25.1): - dependencies: - eslint: 9.25.1 - eslint-formatter-pretty@4.1.0: dependencies: '@types/eslint': 7.29.0 @@ -16946,7 +16761,7 @@ snapshots: execa@5.1.1: dependencies: - cross-spawn: 7.0.3 + cross-spawn: 7.0.6 get-stream: 6.0.1 human-signals: 2.1.0 is-stream: 2.0.1 @@ -16958,7 +16773,7 @@ snapshots: execa@6.1.0: dependencies: - cross-spawn: 7.0.3 + cross-spawn: 7.0.6 get-stream: 6.0.1 human-signals: 3.0.1 is-stream: 3.0.0 @@ -16983,7 +16798,7 @@ snapshots: execa@9.5.2: dependencies: '@sindresorhus/merge-streams': 4.0.0 - cross-spawn: 7.0.3 + cross-spawn: 7.0.6 figures: 6.1.0 get-stream: 9.0.1 human-signals: 8.0.0 @@ -17239,6 +17054,10 @@ snapshots: transitivePeerDependencies: - supports-color + filelist@1.0.4: + dependencies: + minimatch: 5.1.6 + filename-reserved-regex@3.0.0: {} filenamify@6.0.0: @@ -17325,7 +17144,7 @@ snapshots: foreground-child@3.1.1: dependencies: - cross-spawn: 7.0.3 + cross-spawn: 7.0.6 signal-exit: 4.1.0 forever-agent@0.6.1: @@ -17545,7 +17364,7 @@ snapshots: globals@14.0.0: {} - globals@16.0.0: {} + globals@16.1.0: {} globalthis@1.0.3: dependencies: @@ -17602,11 +17421,6 @@ snapshots: hammerjs@2.0.8: {} - happy-dom@16.8.1: - dependencies: - webidl-conversions: 7.0.0 - whatwg-mimetype: 3.0.0 - happy-dom@17.4.4: dependencies: webidl-conversions: 7.0.0 @@ -17680,9 +17494,7 @@ snapshots: headers-polyfill@4.0.2: {} - highlight.js@10.7.3: {} - - highlight.js@11.10.0: {} + highlight.js@11.11.1: {} hosted-git-info@2.8.9: {} @@ -17716,14 +17528,14 @@ snapshots: dependencies: domelementtype: 2.3.0 domhandler: 5.0.3 - domutils: 3.0.1 + domutils: 3.2.2 entities: 4.5.0 htmlparser2@9.1.0: dependencies: domelementtype: 2.3.0 domhandler: 5.0.3 - domutils: 3.1.0 + domutils: 3.2.2 entities: 4.5.0 http-cache-semantics@4.1.1: {} @@ -17736,8 +17548,6 @@ snapshots: statuses: 2.0.1 toidentifier: 1.0.1 - http-link-header@1.1.3: {} - http-proxy-agent@7.0.2: dependencies: agent-base: 7.1.3 @@ -18147,6 +17957,13 @@ snapshots: optionalDependencies: '@pkgjs/parseargs': 0.11.0 + jake@10.9.2: + dependencies: + async: 3.2.4 + chalk: 4.1.2 + filelist: 1.0.4 + minimatch: 3.1.2 + jest-changed-files@29.7.0: dependencies: execa: 5.1.1 @@ -18611,8 +18428,6 @@ snapshots: verror: 1.10.0 optional: true - jsrsasign@11.1.0: {} - jstransformer@1.0.0: dependencies: is-promise: 2.2.2 @@ -18640,7 +18455,7 @@ snapshots: jwa: 2.0.0 safe-buffer: 5.2.1 - katex@0.16.10: + katex@0.16.22: dependencies: commander: 8.3.0 @@ -18775,7 +18590,7 @@ snapshots: magicast@0.3.5: dependencies: - '@babel/parser': 7.25.7 + '@babel/parser': 7.27.2 '@babel/types': 7.25.7 source-map-js: 1.2.1 @@ -18975,10 +18790,6 @@ snapshots: methods@1.1.2: {} - microformats-parser@2.0.2: - dependencies: - parse5: 7.3.0 - micromark-core-commonmark@2.0.0: dependencies: decode-named-character-reference: 1.0.2 @@ -19369,12 +19180,6 @@ snapshots: mylas@2.1.13: {} - mz@2.7.0: - dependencies: - any-promise: 1.3.0 - object-assign: 4.1.1 - thenify-all: 1.6.0 - nan@2.20.0: {} nanoid@3.3.11: {} @@ -19428,6 +19233,8 @@ snapshots: node-addon-api@7.1.0: {} + node-addon-api@8.3.1: {} + node-domexception@1.0.0: {} node-fetch@2.7.0(encoding@0.1.13): @@ -19445,7 +19252,10 @@ snapshots: node-gyp-build-optional-packages@5.0.7: optional: true - node-gyp-build@4.8.1: {} + node-gyp-build@4.8.1: + optional: true + + node-gyp-build@4.8.4: {} node-gyp@10.2.0: dependencies: @@ -19483,19 +19293,6 @@ snapshots: touch: 3.1.0 undefsafe: 2.0.5 - nodemon@3.1.7: - dependencies: - chokidar: 4.0.3 - debug: 4.3.5(supports-color@5.5.0) - ignore-by-default: 1.0.1 - minimatch: 3.1.2 - pstree.remy: 1.1.8 - semver: 7.6.0 - simple-update-notifier: 2.0.0 - supports-color: 5.5.0 - touch: 3.1.0 - undefsafe: 2.0.5 - nofilter@3.1.0: {} nopt@1.0.10: @@ -19552,16 +19349,6 @@ snapshots: nwsapi@2.2.19: optional: true - oauth2orize-pkce@0.1.2: {} - - oauth2orize@1.12.0: - dependencies: - debug: 2.6.9 - uid2: 0.0.4 - utils-merge: 1.0.1 - transitivePeerDependencies: - - supports-color - oauth@0.10.2: {} object-assign@4.1.1: {} @@ -19733,10 +19520,6 @@ snapshots: parse-srcset@1.0.2: {} - parse5-htmlparser2-tree-adapter@6.0.1: - dependencies: - parse5: 6.0.1 - parse5-htmlparser2-tree-adapter@7.0.0: dependencies: domhandler: 5.0.3 @@ -19746,10 +19529,6 @@ snapshots: dependencies: parse5: 7.3.0 - parse5@5.1.1: {} - - parse5@6.0.1: {} - parse5@7.3.0: dependencies: entities: 6.0.0 @@ -19910,7 +19689,7 @@ snapshots: pngjs@5.0.0: {} - pnpm@10.10.0: {} + pnpm@9.6.0: {} polished@4.2.2: dependencies: @@ -20219,7 +19998,7 @@ snapshots: dependencies: event-stream: 3.3.4 - psl@1.13.0: + psl@1.15.0: dependencies: punycode: 2.3.1 @@ -20345,8 +20124,6 @@ snapshots: range-parser@1.2.1: {} - ratelimiter@3.4.1: {} - raw-body@2.5.2: dependencies: bytes: 3.1.2 @@ -20642,11 +20419,6 @@ snapshots: rrweb-cssom@0.8.0: optional: true - rss-parser@3.13.0: - dependencies: - entities: 2.2.0 - xml2js: 0.5.0 - run-parallel@1.2.0: dependencies: queue-microtask: 1.2.3 @@ -20740,6 +20512,8 @@ snapshots: semver@7.7.1: {} + semver@7.7.2: {} + send@0.19.0: dependencies: debug: 2.6.9 @@ -20920,7 +20694,7 @@ snapshots: dependencies: '@hapi/hoek': 11.0.4 '@hapi/wreck': 18.0.1 - debug: 4.3.5(supports-color@5.5.0) + debug: 4.3.5 joi: 17.11.0 transitivePeerDependencies: - supports-color @@ -20931,7 +20705,7 @@ snapshots: simple-update-notifier@2.0.0: dependencies: - semver: 7.6.3 + semver: 7.7.1 sinon@18.0.1: dependencies: @@ -21432,14 +21206,6 @@ snapshots: textarea-caret@3.1.0: {} - thenify-all@1.6.0: - dependencies: - thenify: 3.3.1 - - thenify@3.3.1: - dependencies: - any-promise: 1.3.0 - thread-stream@3.1.0: dependencies: real-require: 0.2.0 @@ -21511,7 +21277,7 @@ snapshots: tough-cookie@4.1.4: dependencies: - psl: 1.13.0 + psl: 1.15.0 punycode: 2.3.1 universalify: 0.2.0 url-parse: 1.5.10 @@ -21550,20 +21316,23 @@ snapshots: ts-dedent@2.2.0: {} - ts-jest@29.1.2(@babel/core@7.24.7)(@jest/types@29.6.3)(babel-jest@29.7.0(@babel/core@7.24.7))(esbuild@0.25.3)(jest@29.7.0(@types/node@22.15.2))(typescript@5.8.3): + ts-jest@29.3.4(@babel/core@7.24.7)(@jest/transform@29.7.0)(@jest/types@29.6.3)(babel-jest@29.7.0(@babel/core@7.24.7))(esbuild@0.25.3)(jest@29.7.0(@types/node@22.15.2))(typescript@5.8.3): dependencies: bs-logger: 0.2.6 + ejs: 3.1.10 fast-json-stable-stringify: 2.1.0 jest: 29.7.0(@types/node@22.15.2) jest-util: 29.7.0 json5: 2.2.3 lodash.memoize: 4.1.2 make-error: 1.3.6 - semver: 7.6.0 + semver: 7.7.2 + type-fest: 4.41.0 typescript: 5.8.3 yargs-parser: 21.1.1 optionalDependencies: '@babel/core': 7.24.7 + '@jest/transform': 29.7.0 '@jest/types': 29.6.3 babel-jest: 29.7.0(@babel/core@7.24.7) esbuild: 0.25.3 @@ -21640,6 +21409,8 @@ snapshots: type-fest@4.27.0: {} + type-fest@4.41.0: {} + type-is@1.6.18: dependencies: media-typer: 0.3.0 @@ -21711,8 +21482,6 @@ snapshots: typescript@5.8.3: {} - uid2@0.0.4: {} - uid@2.0.2: dependencies: '@lukeed/csprng': 1.0.1 @@ -21855,7 +21624,7 @@ snapshots: dependencies: diff: 5.2.0 diff-match-patch: 1.0.5 - highlight.js: 11.10.0 + highlight.js: 11.11.1 vue: 3.5.14(typescript@5.8.3) vue-demi: 0.14.7(vue@3.5.14(typescript@5.8.3)) @@ -21931,9 +21700,9 @@ snapshots: vitest-fetch-mock@0.4.5(vitest@3.1.2(@types/debug@4.1.12)(@types/node@22.15.2)(happy-dom@17.4.4)(jsdom@26.1.0(bufferutil@4.0.9)(utf-8-validate@6.0.5))(msw@2.7.5(@types/node@22.15.2)(typescript@5.8.3))(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3)): dependencies: - vitest: 3.1.2(@types/debug@4.1.12)(@types/node@22.15.2)(happy-dom@17.4.4)(jsdom@26.1.0)(msw@2.7.5(@types/node@22.15.2)(typescript@5.8.3))(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3) + vitest: 3.1.2(@types/debug@4.1.12)(@types/node@22.15.2)(happy-dom@17.4.4)(jsdom@26.1.0(bufferutil@4.0.9)(utf-8-validate@6.0.5))(msw@2.7.5(@types/node@22.15.2)(typescript@5.8.3))(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3) - vitest@3.1.2(@types/debug@4.1.12)(@types/node@22.15.2)(happy-dom@17.4.4)(jsdom@26.1.0)(msw@2.7.5(@types/node@22.15.2)(typescript@5.8.3))(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3): + vitest@3.1.2(@types/debug@4.1.12)(@types/node@22.15.2)(happy-dom@17.4.4)(jsdom@26.1.0(bufferutil@4.0.9)(utf-8-validate@6.0.5))(msw@2.7.5(@types/node@22.15.2)(typescript@5.8.3))(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3): dependencies: '@vitest/expect': 3.1.2 '@vitest/mocker': 3.1.2(msw@2.7.5(@types/node@22.15.2)(typescript@5.8.3))(vite@6.3.3(@types/node@22.15.2)(sass@1.87.0)(terser@5.39.0)(tsx@4.19.3)) @@ -22021,7 +21790,7 @@ snapshots: vue-docgen-api@4.75.1(vue@3.5.14(typescript@5.8.3)): dependencies: - '@babel/parser': 7.25.7 + '@babel/parser': 7.27.2 '@babel/types': 7.25.7 '@vue/compiler-dom': 3.5.14 '@vue/compiler-sfc': 3.5.14 @@ -22094,7 +21863,7 @@ snapshots: wait-on@8.0.3(debug@4.4.0): dependencies: - axios: 1.8.4(debug@4.4.0) + axios: 1.9.0(debug@4.4.0) joi: 17.13.3 lodash: 4.17.21 minimist: 1.2.8 @@ -22258,13 +22027,6 @@ snapshots: xml-name-validator@5.0.0: optional: true - xml2js@0.5.0: - dependencies: - sax: 1.2.4 - xmlbuilder: 11.0.1 - - xmlbuilder@11.0.1: {} - xmlchars@2.2.0: optional: true @@ -22303,16 +22065,6 @@ snapshots: y18n: 4.0.3 yargs-parser: 18.1.3 - yargs@16.2.0: - dependencies: - cliui: 7.0.4 - escalade: 3.1.1 - get-caller-file: 2.0.5 - require-directory: 2.1.1 - string-width: 4.2.3 - y18n: 5.0.8 - yargs-parser: 20.2.9 - yargs@17.7.2: dependencies: cliui: 8.0.1 diff --git a/sharkey-locales/en-US.yml b/sharkey-locales/en-US.yml index f1ad66fc8c..3f5c82f2dc 100644 --- a/sharkey-locales/en-US.yml +++ b/sharkey-locales/en-US.yml @@ -371,6 +371,8 @@ _mfm: smallDescription: "Displays content small and thin." center: "Center" centerDescription: "Displays content centered." + unixtime: "Unix Time" + unixtimeDescription: "Displays a timestamp in the viewer's current timezone." inlineCode: "Code (Inline)" inlineCodeDescription: "Displays inline syntax highlighting for (program) code." blockCode: "Code (Block)" @@ -572,6 +574,12 @@ bubbleTimelineMustBeEnabled: "Note: the bubble timeline is hidden by default, an popularUsersGlobal: "Users popular on the global network" popularUsersLocal: "Users popular on {name}" +pollsOnLocal: "Polls trending on {name}" +pollsOnRemote: "Polls trending on the global network" +pollsExpired: "Polls that have ended recently" +trendingPollsDisabled: "Trending polls are disabled on this instance." +trendingPollsDisabledLogIn: "Please log in to view trending polls." + silenced: "Silenced" totalFollowers: "Total followers" totalFollowing: "Total following" @@ -589,3 +597,36 @@ roleAutomatic: "automatic" translationTimeoutLabel: "Translation timeout" translationTimeoutCaption: "Timeout in milliseconds for translation API requests." + +staffNotes: "Staff notes" +instanceIconAlt: "Icon of {name}" + +attributionDomains: "Attribution Domains" +attributionDomainsDescription: "A list of domains whose content can be attributed to you on link previews, separated by new-line. Any subdomain will also be valid. The following needs to be on the webpage:" +writtenBy: "Written by {user}" + +followingPub: "Following (Pub)" +followersSub: "Followers (Sub)" +wellKnownResources: "Well-known resources" +lastPosted: "Last posted: {at}" +nsfw: "NSFW" +raw: "Raw" +cw: "CW" +mediaSilenced: "Media Silenced" +bubble: "Bubble" +verified: "Verified" +notVerified: "Not Verified" +hibernated: "Hibernated" + +keepCwDescription: "When replying to a post with a Content Warning, automatically use the same CW for the reply." +keepCwDisabled: "Disabled (do not copy CWs)" +keepCwEnabled: "Enabled (copy CWs verbatim)" +keepCwPrependRe: "Enabled (copy CW and prepend \"RE:\")" + +noteFooterLabel: "Note controls" + +rawUserDescription: "Packed user data in its raw form. Most of these fields are public and visible to all users." +rawInfoDescription: "Extended user data in its raw form. These fields are private and can only be accessed by moderators." +rawApDescription: "ActivityPub user data in its raw form. These fields are public and accessible to other instances." + +signupReason: "Signup Reason" diff --git a/sharkey-locales/pt-PT.yml b/sharkey-locales/pt-PT.yml index b3f7611ee3..7220cd2b59 100644 --- a/sharkey-locales/pt-PT.yml +++ b/sharkey-locales/pt-PT.yml @@ -7,3 +7,6 @@ openRemoteProfile: "Abrir perfil remoto" allowClickingNotifications: "Permitir clicar em notificações" pinnedOnly: "Fixado" blockingYou: "Bloqueando você" +attributionDomains: "Domínios de Atribuição" +attributionDomainsDescription: "Uma lista de domínios cujo conteúdo pode ser atribuído a você em prévias de link, separadas por linha. Qualquer subdomínio também será válido. O código seguinte precisa estar presente na página:" +writtenBy: "Escrito por {user}" diff --git a/sharkey-locales/th-TH.yml b/sharkey-locales/th-TH.yml index c4670d48bb..22d1eda8dd 100644 --- a/sharkey-locales/th-TH.yml +++ b/sharkey-locales/th-TH.yml @@ -52,3 +52,11 @@ _moderationLogTypes: acceptRemoteInstanceReports: "รายงานได้รับการยอมรับจากเซิร์ฟเวอร์ระยะไกล" rejectQuotesUser: "โพสต์คำพูดของผู้ใช้ถูกบล็อค/ลบ" allowQuotesUser: "อนุญาตให้อ้างอิงข้อความจากผู้ใช้" + clearUserFiles: "ล้างไฟล์ไดรฟ์ของผู้ใช้" + nsfwUser: "ผู้ใช้ถูกทำเครื่องหมายเป็น NSFW" + unNsfwUser: "ผู้ใช้ที่ไม่ได้ทำเครื่องหมายเป็น NSFW" + silenceUser: "ผู้ใช้ที่ถูกปิดเสียง" + unSilenceUser: "ผู้ใช้ที่ไม่ได้รับการปิดเสียง" + createAccount: "สร้างบัญชีแล้ว" + clearRemoteFiles: "ล้างไฟล์ไดรฟ์ระยะไกล" + clearOwnerlessFiles: "ล้างไฟล์ไดรฟ์ที่ไม่มีเจ้าของ" |