11.0.0-beta.13

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

.config/example.yml

@@ -65,10 +65,10 @@ db:
 #   ┌─────────────────────┐
 #───┘ Redis configuration └─────────────────────────────────────
 
-#redis:
-#  host: localhost
-#  port: 6379
-#  pass: example-pass
+redis:
+  host: localhost
+  port: 6379
+  #pass: example-pass
 
 #   ┌─────────────────────────────┐
 #───┘ Elasticsearch configuration └─────────────────────────────
@@ -127,28 +127,12 @@ drive:
 # change it according to your preferences.
 
 # Available methods:
-# aid1 ... Use AID for ID generation (with random 1 char)
-# aid2 ... Use AID for ID generation (with random 2 chars)
-# aid3 ... Use AID for ID generation (with random 3 chars)
-# aid4 ... Use AID for ID generation (with random 4 chars)
-# ulid ... Use ulid for ID generation
-# objectid ... This is left for backward compatibility.
+# aid ... Short, Millisecond accuracy
+# meid ... Similar to ObjectID, Millisecond accuracy
+# ulid ... Millisecond accuracy
+# objectid ... This is left for backward compatibility
 
-# AID(n) is the original ID generation method.
-# The trailing n represents the number of random characters that
-# will be suffixed.
-# The larger n is the safer. If n is small, the possibility of
-# collision at the same time increases, but there are also
-# advantages such as shortening of the URL.
-
-# ULID: Universally Unique Lexicographically Sortable Identifier.
-# for more details: https://github.com/ulid/spec
-# * Normally, AID should be sufficient.
-
-# ObjectID is the method used in previous versions of Misskey.
-# * Choose this if you are migrating from a previous Misskey.
-
-id: 'aid2'
+id: 'aid'
 
 #   ┌─────────────────────┐
 #───┘ Other configuration └─────────────────────────────────────

CHANGELOG.md

@@ -7,11 +7,37 @@ If you encounter any problems with updating, please try the following:
 
 11.0.0
 ----------
-* データベースがMongoDBからPostgreSQLに変更されました
+* **データベースがMongoDBからPostgreSQLに変更されました**
+* **Redisが必須に**
+* アカウントを完全に削除できるように
+* ミュート/ブロック時にそのユーザーの投稿のウォッチをすべて解除するように
+* フォロー申請数が実際より1すくなくなる問題を修正
+* リストからアカウント削除したユーザーを削除できない問題を修正
+* リストTLでフォローしていないユーザーの非公開投稿が流れる問題を修正
+* リストTLでダイレクト投稿が流れない問題を修正
+* ミュートしているユーザーの投稿がタイムラインに流れてくることがある問題を修正
 
 ### APIの破壊的変更
 * v10時点で deprecated だったパラメータなどを削除
 * ユーザーリストの title が name に
+* リバーシの対局の`settings`プロパティがなくなり、その中にあったプロパティがすべて上の階層に
+  * 例えば`game.settings.map`は`game.map`になる
+
+### 既知の問題
+* アプリが作成できない
+  * 依存ライブラリの問題と思わるため、対応が難しい
+
+### Migration
+coming soon...
+
+10.100.0
+----------
+* ユーザーリストでフォローボタンを表示するように
+* ドライブのファイルのサムネイルを修正
+* 投稿ウィジットでローカルのみの公開範囲で投稿できない問題を修正
+* TLを遡った時に抜けがある時がある問題を修正
+* ユーザータイムラインが投稿日時順ではなくなっているのを修正
+* 10.99.0 でチャートのレンダリングがおかしい問題を修正
 
 10.99.0
 ----------

CONTRIBUTING.md

@@ -130,6 +130,40 @@ const users = userIds.length > 0 ? await Users.find({
 }) : [];
 ```
 
+### 配列のインデックス in SQL
+SQLでは配列のインデックスは**1始まり**。
+`[a, b, c]`の `a`にアクセスしたいなら`[0]`ではなく`[1]`と書く
+
 ### `undefined`にご用心
 MongoDBの時とは違い、findOneでレコードを取得する時に対象レコードが存在しない場合 **`undefined`** が返ってくるので注意。
 MongoDBは`null`で返してきてたので、その感覚で`if (x === null)`とか書くとバグる。代わりに`if (x == null)`と書いてください
+
+### 簡素な`undefined`チェック
+データベースからレコードを取得するときに、プログラムの流れ的に(ほぼ)絶対`undefined`にはならない場合でも、`undefined`チェックしないとTypeScriptに怒られます。
+でもいちいち複数行を費やして、発生するはずのない`undefined`をチェックするのも面倒なので、`ensure`というユーティリティ関数を用意しています。
+例えば、
+``` ts
+const user = await Users.findOne(userId);
+// この時点で user の型は User | undefined
+if (user == null) {
+	throw 'missing user';
+}
+// この時点で user の型は User
+```
+という処理を`ensure`を使うと
+``` ts
+const user = await Users.findOne(userId).then(ensure);
+// この時点で user の型は User
+```
+という風に書けます。
+もちろん`ensure`内部でエラーを握りつぶすようなことはしておらず、万が一`undefined`だった場合はPromiseがRejectされ後続の処理は実行されません。
+``` ts
+const user = await Users.findOne(userId).then(ensure);
+// 万が一 Users.findOne の結果が undefined だったら、ensure でエラーが発生するので
+// この行に到達することは無い
+// なので、.then(ensure) は
+// if (user == null) {
+//	throw 'missing user';
+// }
+// の糖衣構文のような扱いです
+```

docs/setup.en.md

@@ -24,18 +24,13 @@ Please install and setup these softwares:
 #### Dependencies :package:
 * **[Node.js](https://nodejs.org/en/)** >= 11.7.0
 * **[PostgreSQL](https://www.postgresql.org/)** >= 10
+* **[Redis](https://redis.io/)**
 
 ##### Optional
-* [Redis](https://redis.io/)
-  * Redis is optional, but we strongly recommended to install it
 * [Elasticsearch](https://www.elastic.co/) - required to enable the search feature
 * [FFmpeg](https://www.ffmpeg.org/)
 
-*3.* Setup PostgreSQL
-----------------------------------------------------------------
-:)
-
-*4.* Install Misskey
+*3.* Install Misskey
 ----------------------------------------------------------------
 1. `su - misskey` Connect to misskey user.
 2. `git clone -b master git://github.com/syuilo/misskey.git` Clone the misskey repo from master branch.
@@ -43,12 +38,12 @@ Please install and setup these softwares:
 4. `git checkout $(git tag -l | grep -Ev -- '-(rc|alpha)\.[0-9]+$' | sort -V | tail -n 1)` Checkout to the [latest release](https://github.com/syuilo/misskey/releases/latest)
 5. `npm install` Install misskey dependencies.
 
-*5.* Configure Misskey
+*4.* Configure Misskey
 ----------------------------------------------------------------
 1. `cp .config/example.yml .config/default.yml` Copy the `.config/example.yml` and rename it to `default.yml`.
 2. Edit `default.yml`
 
-*6.* Build Misskey
+*5.* Build Misskey
 ----------------------------------------------------------------
 
 Build misskey with the following:
@@ -64,13 +59,13 @@ If you're still encountering errors about some modules, use node-gyp:
 3. `node-gyp build`
 4. `NODE_ENV=production npm run build`
 
-*7.* Init DB
+*6.* Init DB
 ----------------------------------------------------------------
 ``` shell
 npm run init
 ```
 
-*8.* That is it.
+*7.* That is it.
 ----------------------------------------------------------------
 Well done! Now, you have an environment that run to Misskey.
 

docs/setup.fr.md

@@ -24,18 +24,13 @@ Installez les paquets suivants :
 #### Dépendences :package:
 * **[Node.js](https://nodejs.org/en/)** >= 11.7.0
 * **[PostgreSQL](https://www.postgresql.org/)** >= 10
+* **[Redis](https://redis.io/)**
 
 ##### Optionnels
-* [Redis](https://redis.io/)
-  * Redis est optionnel mais nous vous recommandons vivement de l'installer
 * [Elasticsearch](https://www.elastic.co/) - requis pour pouvoir activer la fonctionnalité de recherche
 * [FFmpeg](https://www.ffmpeg.org/)
 
-*3.* Paramètrage de PostgreSQL
-----------------------------------------------------------------
-:)
-
-*4.* Installation de Misskey
+*3.* Installation de Misskey
 ----------------------------------------------------------------
 1. `su - misskey` Basculez vers l'utilisateur misskey.
 2. `git clone -b master git://github.com/syuilo/misskey.git` Clonez la branche master du dépôt misskey.
@@ -43,12 +38,12 @@ Installez les paquets suivants :
 4. `git checkout $(git tag -l | grep -Ev -- '-(rc|alpha)\.[0-9]+$' | sort -V | tail -n 1)` Checkout sur le tag de la [version la plus récente](https://github.com/syuilo/misskey/releases/latest)
 5. `npm install` Installez les dépendances de misskey.
 
-*5.* Création du fichier de configuration
+*4.* Création du fichier de configuration
 ----------------------------------------------------------------
 1. `cp .config/example.yml .config/default.yml` Copiez le fichier `.config/example.yml` et renommez-le`default.yml`.
 2. Editez le fichier `default.yml`
 
-*6.* Construction de Misskey
+*5.* Construction de Misskey
 ----------------------------------------------------------------
 
 Construisez Misskey comme ceci :
@@ -64,7 +59,7 @@ Si vous rencontrez des erreurs concernant certains modules, utilisez node-gyp:
 3. `node-gyp build`
 4. `NODE_ENV=production npm run build`
 
-*7.* C'est tout.
+*6.* C'est tout.
 ----------------------------------------------------------------
 Excellent ! Maintenant, vous avez un environnement prêt pour lancer Misskey
 

docs/setup.ja.md

@@ -24,25 +24,14 @@ adduser --disabled-password --disabled-login misskey
 #### 依存関係 :package:
 * **[Node.js](https://nodejs.org/en/)** (11.7.0以上)
 * **[PostgreSQL](https://www.postgresql.org/)** (10以上)
+* **[Redis](https://redis.io/)**
 
 ##### オプション
-* [Redis](https://redis.io/)
-	* Redisはオプションですが、インストールすることを強く推奨します。
-	* インストールしなくていいのは、あなたのインスタンスが自分専用のときだけとお考えください。
-	* 具体的には、Redisをインストールしないと、次の事が出来なくなります:
-		* Misskeyプロセスを複数起動しての負荷分散
-		* レートリミット
-		* ジョブキュー
-		* Twitter連携
 * [Elasticsearch](https://www.elastic.co/)
 	* 検索機能を有効にするためにはインストールが必要です。
 * [FFmpeg](https://www.ffmpeg.org/)
 
-*3.* PostgreSQLの設定
-----------------------------------------------------------------
-:)
-
-*4.* Misskeyのインストール
+*3.* Misskeyのインストール
 ----------------------------------------------------------------
 1. `su - misskey` misskeyユーザーを使用
 2. `git clone -b master git://github.com/syuilo/misskey.git` masterブランチからMisskeyレポジトリをクローン
@@ -50,12 +39,12 @@ adduser --disabled-password --disabled-login misskey
 4. `git checkout $(git tag -l | grep -Ev -- '-(rc|alpha)\.[0-9]+$' | sort -V | tail -n 1)` [最新のリリース](https://github.com/syuilo/misskey/releases/latest)を確認
 5. `npm install` Misskeyの依存パッケージをインストール
 
-*5.* 設定ファイルを作成する
+*4.* 設定ファイルを作成する
 ----------------------------------------------------------------
 1. `cp .config/example.yml .config/default.yml` `.config/example.yml`をコピーし名前を`default.yml`にする。
 2. `default.yml` を編集する。
 
-*6.* Misskeyのビルド
+*5.* Misskeyのビルド
 ----------------------------------------------------------------
 
 次のコマンドでMisskeyをビルドしてください:
@@ -70,13 +59,13 @@ Debianをお使いであれば、`build-essential`パッケージをインスト
 3. `node-gyp build`
 4. `NODE_ENV=production npm run build`
 
-*7.* データベースを初期化
+*6.* データベースを初期化
 ----------------------------------------------------------------
 ``` shell
 npm run init
 ```
 
-*8.* 以上です！
+*7.* 以上です！
 ----------------------------------------------------------------
 お疲れ様でした。これでMisskeyを動かす準備は整いました。
 

gulpfile.ts

@@ -120,7 +120,7 @@ gulp.task('copy:client', () =>
 		])
 			.pipe(isProduction ? (imagemin as any)() : gutil.noop())
 			.pipe(rename(path => {
-				path.dirname = path.dirname.replace('assets', '.');
+				path.dirname = path.dirname!.replace('assets', '.');
 			}))
 			.pipe(gulp.dest('./built/client/assets/'))
 );

package.json

@@ -1,7 +1,7 @@
 {
 	"name": "misskey",
 	"author": "syuilo <i@syuilo.com>",
-	"version": "11.0.0-beta.2",
+	"version": "11.0.0-beta.13",
 	"codename": "daybreak",
 	"repository": {
 		"type": "git",
@@ -106,7 +106,7 @@
 		"bcryptjs": "2.4.3",
 		"bootstrap-vue": "2.0.0-rc.13",
 		"bull": "3.7.0",
-		"cafy": "15.1.0",
+		"cafy": "15.1.1",
 		"chai": "4.2.0",
 		"chai-http": "4.2.1",
 		"chalk": "2.4.2",

src/boot/master.ts

@@ -44,7 +44,7 @@ function greet() {
 export async function masterMain() {
 	greet();
 
-	let config: Config;
+	let config!: Config;
 
 	try {
 		// initialize app

src/boot/worker.ts

@@ -15,6 +15,6 @@ export async function workerMain() {
 
 	if (cluster.isWorker) {
 		// Send a 'ready' message to parent process
-		process.send('ready');
+		process.send!('ready');
 	}
 }

src/client/app/common/views/deck/deck.user-column.vue

@@ -8,7 +8,7 @@
 		<div class="is-remote" v-if="user.host != null">
 			<details>
 				<summary><fa icon="exclamation-triangle"/> {{ $t('@.is-remote-user') }}</summary>
-				<a :href="user.url || user.uri" target="_blank">{{ $t('@.view-on-remote') }}</a>
+				<a :href="user.url" target="_blank">{{ $t('@.view-on-remote') }}</a>
 			</details>
 		</div>
 		<header :style="bannerStyle">

src/client/app/desktop/views/home/user/index.vue

@@ -4,7 +4,7 @@
 		<fa icon="exclamation-triangle"/> {{ $t('@.user-suspended') }}
 	</div>
 	<div class="is-remote" v-if="user.host != null" :class="{ shadow: $store.state.device.useShadow, round: $store.state.device.roundedCorners }">
-		<fa icon="exclamation-triangle"/> {{ $t('@.is-remote-user') }}<a :href="user.url || user.uri" target="_blank">{{ $t('@.view-on-remote') }}</a>
+		<fa icon="exclamation-triangle"/> {{ $t('@.is-remote-user') }}<a :href="user.url" target="_blank">{{ $t('@.view-on-remote') }}</a>
 	</div>
 	<div class="main">
 		<x-header class="header" :user="user"/>

src/client/app/mobile/views/pages/user/index.vue

@@ -5,7 +5,7 @@
 	</template>
 	<div class="wwtwuxyh" v-if="!fetching">
 		<div class="is-suspended" v-if="user.isSuspended"><p><fa icon="exclamation-triangle"/> {{ $t('@.user-suspended') }}</p></div>
-		<div class="is-remote" v-if="user.host != null"><p><fa icon="exclamation-triangle"/> {{ $t('@.is-remote-user') }}<a :href="user.url || user.uri" target="_blank">{{ $t('@.view-on-remote') }}</a></p></div>
+		<div class="is-remote" v-if="user.host != null"><p><fa icon="exclamation-triangle"/> {{ $t('@.is-remote-user') }}<a :href="user.url" target="_blank">{{ $t('@.view-on-remote') }}</a></p></div>
 		<header>
 			<div class="banner" :style="style"></div>
 			<div class="body">

src/client/themes/light.json5

@@ -7,7 +7,7 @@
 	kind: 'light',
 
 	vars: {
-		primary: '#fb4e4e',
+		primary: '#f18570',
 		secondary: '#fff',
 		text: '#666',
 	},

src/config/load.ts

@@ -29,7 +29,7 @@ export default function load() {
 
 	config.url = url.origin;
 
-	config.port = config.port || parseInt(process.env.PORT, 10);
+	config.port = config.port || parseInt(process.env.PORT || '', 10);
 
 	mixin.host = url.host;
 	mixin.hostname = url.hostname;

src/daemons/notes-stats-child.ts

@@ -19,7 +19,7 @@ initDb().then(() => {
 			all, local
 		};
 
-		process.send(stats);
+		process.send!(stats);
 	}
 
 	tick();

src/db/postgre.ts

@@ -76,7 +76,7 @@ class MyCustomLogger implements Logger {
 }
 
 export function initDb(justBorrow = false, sync = false, log = false) {
-	const enableLogging = log || !['production', 'test'].includes(process.env.NODE_ENV);
+	const enableLogging = log || !['production', 'test'].includes(process.env.NODE_ENV || '');
 
 	try {
 		const conn = getConnection();
@@ -93,7 +93,7 @@ export function initDb(justBorrow = false, sync = false, log = false) {
 		synchronize: process.env.NODE_ENV === 'test' || sync,
 		dropSchema: process.env.NODE_ENV === 'test' && !justBorrow,
 		logging: enableLogging,
-		logger: enableLogging ? new MyCustomLogger() : null,
+		logger: enableLogging ? new MyCustomLogger() : undefined,
 		entities: [
 			Meta,
 			Instance,

src/db/redis.ts

@@ -1,7 +1,7 @@
 import * as redis from 'redis';
 import config from '../config';
 
-export default config.redis ? redis.createClient(
+export default redis.createClient(
 	config.redis.port,
 	config.redis.host,
 	{
@@ -9,4 +9,4 @@ export default config.redis ? redis.createClient(
 		prefix: config.redis.prefix,
 		db: config.redis.db || 0
 	}
-) : null;
+);

src/games/reversi/core.ts

@@ -37,7 +37,7 @@ export type Undo = {
 	/**
 	 * ターン
 	 */
-	turn: Color;
+	turn: Color | null;
 };
 
 /**
@@ -47,12 +47,12 @@ export default class Reversi {
 	public map: MapPixel[];
 	public mapWidth: number;
 	public mapHeight: number;
-	public board: Color[];
-	public turn: Color = BLACK;
+	public board: (Color | null | undefined)[];
+	public turn: Color | null = BLACK;
 	public opts: Options;
 
 	public prevPos = -1;
-	public prevColor: Color = null;
+	public prevColor: Color | null = null;
 
 	private logs: Undo[] = [];
 
@@ -145,12 +145,12 @@ export default class Reversi {
 		// ターン計算
 		this.turn =
 			this.canPutSomewhere(!this.prevColor) ? !this.prevColor :
-			this.canPutSomewhere(this.prevColor) ? this.prevColor :
+			this.canPutSomewhere(this.prevColor!) ? this.prevColor :
 			null;
 	}
 
 	public undo() {
-		const undo = this.logs.pop();
+		const undo = this.logs.pop()!;
 		this.prevColor = undo.color;
 		this.prevPos = undo.pos;
 		this.board[undo.pos] = null;
@@ -254,10 +254,10 @@ export default class Reversi {
 	/**
 	 * ゲームの勝者 (null = 引き分け)
 	 */
-	public get winner(): Color {
+	public get winner(): Color | null {
 		return this.isEnded ?
 			this.blackCount == this.whiteCount ? null :
 			this.opts.isLlotheo === this.blackCount > this.whiteCount　? WHITE : BLACK :
-			undefined;
+			undefined as never;
 	}
 }

src/mfm/fromHtml.ts

@@ -3,8 +3,6 @@ import { URL } from 'url';
 import { urlRegex } from './prelude';
 
 export function fromHtml(html: string): string {
-	if (html == null) return null;
-
 	const dom = parseFragment(html) as DefaultTreeDocumentFragment;
 
 	let text = '';

src/mfm/parse.ts

@@ -2,7 +2,7 @@ import { mfmLanguage } from './language';
 import { MfmForest } from './prelude';
 import { normalize } from './normalize';
 
-export function parse(source: string): MfmForest {
+export function parse(source: string | null): MfmForest | null {
 	if (source == null || source == '') {
 		return null;
 	}
@@ -10,7 +10,7 @@ export function parse(source: string): MfmForest {
 	return normalize(mfmLanguage.root.tryParse(source));
 }
 
-export function parsePlain(source: string): MfmForest {
+export function parsePlain(source: string | null): MfmForest | null {
 	if (source == null || source == '') {
 		return null;
 	}

src/mfm/toHtml.ts

@@ -4,7 +4,7 @@ import { intersperse } from '../prelude/array';
 import { MfmForest, MfmTree } from './prelude';
 import { IMentionedRemoteUsers } from '../models/entities/note';
 
-export function toHtml(tokens: MfmForest, mentionedRemoteUsers: IMentionedRemoteUsers = []) {
+export function toHtml(tokens: MfmForest | null, mentionedRemoteUsers: IMentionedRemoteUsers = []) {
 	if (tokens == null) {
 		return null;
 	}

src/migrate.ts

@@ -16,7 +16,6 @@ import { InternalStorage } from './services/drive/internal-storage';
 import { createTemp } from './misc/create-temp';
 import { Note } from './models/entities/note';
 import { Following } from './models/entities/following';
-import { genId } from './misc/gen-id';
 import { Poll } from './models/entities/poll';
 import { PollVote } from './models/entities/poll-vote';
 import { NoteFavorite } from './models/entities/note-favorite';
@@ -25,9 +24,14 @@ import { UserPublickey } from './models/entities/user-publickey';
 import { UserKeypair } from './models/entities/user-keypair';
 import { extractPublic } from './crypto_key';
 import { Emoji } from './models/entities/emoji';
-import { toPuny } from './misc/convert-host';
+import { toPuny as _toPuny } from './misc/convert-host';
 import { UserProfile } from './models/entities/user-profile';
 
+function toPuny(x: string | null): string | null {
+	if (x == null) return null;
+	return _toPuny(x);
+}
+
 const u = (config as any).mongodb.user ? encodeURIComponent((config as any).mongodb.user) : null;
 const p = (config as any).mongodb.pass ? encodeURIComponent((config as any).mongodb.pass) : null;
 
@@ -36,6 +40,9 @@ const uri = `mongodb://${u && p ? `${u}:${p}@` : ''}${(config as any).mongodb.ho
 const db = monk(uri);
 let mdb: mongo.Db;
 
+const test = false;
+const limit = 500;
+
 const nativeDbConn = async (): Promise<mongo.Db> => {
 	if (mdb) return mdb;
 
@@ -92,14 +99,14 @@ async function main() {
 			usernameLower: user.username.toLowerCase(),
 			host: toPuny(user.host),
 			token: generateUserToken(),
-			isAdmin: user.isAdmin,
+			isAdmin: user.isAdmin || false,
 			name: user.name,
-			followersCount: user.followersCount,
-			followingCount: user.followingCount,
-			notesCount: user.notesCount,
-			isBot: user.isBot,
-			isCat: user.isCat,
-			isVerified: user.isVerified,
+			followersCount: user.followersCount || 0,
+			followingCount: user.followingCount || 0,
+			notesCount: user.notesCount || 0,
+			isBot: user.isBot || false,
+			isCat: user.isCat || false,
+			isVerified: user.isVerified || false,
 			inbox: user.inbox,
 			sharedInbox: user.sharedInbox,
 			uri: user.uri,
@@ -133,7 +140,7 @@ async function main() {
 	async function migrateFollowing(following: any) {
 		await Followings.save({
 			id: following._id.toHexString(),
-			createdAt: following.createdAt || new Date(),
+			createdAt: new Date(),
 			followerId: following.followerId.toHexString(),
 			followeeId: following.followeeId.toHexString(),
 
@@ -160,6 +167,7 @@ async function main() {
 		const user = await _User.findOne({
 			_id: file.metadata.userId
 		});
+		if (user == null) return;
 		if (file.metadata.storage && file.metadata.storage.key) { // when object storage
 			await DriveFiles.save({
 				id: file._id.toHexString(),
@@ -169,7 +177,7 @@ async function main() {
 				md5: file.md5,
 				name: file.filename,
 				type: file.contentType,
-				properties: file.metadata.properties,
+				properties: file.metadata.properties || {},
 				size: file.length,
 				url: file.metadata.url,
 				uri: file.metadata.uri,
@@ -255,7 +263,6 @@ async function main() {
 
 		if (note.poll) {
 			await Polls.save({
-				id: genId(),
 				noteId: note._id.toHexString(),
 				choices: note.poll.choices.map((x: any) => x.text),
 				expiresAt: note.poll.expiresAt,
@@ -301,13 +308,13 @@ async function main() {
 		const u = await _User.findOne({
 			_id: new mongo.ObjectId(user.id)
 		});
-		const avatar = await DriveFiles.findOne(u.avatarId.toHexString());
-		const banner = await DriveFiles.findOne(u.bannerId.toHexString());
+		const avatar = u.avatarId ? await DriveFiles.findOne(u.avatarId.toHexString()) : null;
+		const banner = u.bannerId ? await DriveFiles.findOne(u.bannerId.toHexString()) : null;
 		await Users.update(user.id, {
-			avatarId: avatar.id,
-			bannerId: banner.id,
-			avatarUrl: avatar.url,
-			bannerUrl: banner.url
+			avatarId: avatar ? avatar.id : null,
+			bannerId: banner ? banner.id : null,
+			avatarUrl: avatar ? avatar.url : null,
+			bannerUrl: banner ? banner.url : null
 		});
 	}
 
@@ -323,9 +330,14 @@ async function main() {
 		});
 	}
 
-	const allUsersCount = await _User.count();
+	let allUsersCount = await _User.count({
+		deletedAt: { $exists: false }
+	});
+	if (test && allUsersCount > limit) allUsersCount = limit;
 	for (let i = 0; i < allUsersCount; i++) {
-		const user = await _User.findOne({}, {
+		const user = await _User.findOne({
+			deletedAt: { $exists: false }
+		}, {
 			skip: i
 		});
 		try {
@@ -337,7 +349,8 @@ async function main() {
 		}
 	}
 
-	const allFollowingsCount = await _Following.count();
+	let allFollowingsCount = await _Following.count();
+	if (test && allFollowingsCount > limit) allFollowingsCount = limit;
 	for (let i = 0; i < allFollowingsCount; i++) {
 		const following = await _Following.findOne({}, {
 			skip: i
@@ -351,7 +364,8 @@ async function main() {
 		}
 	}
 
-	const allDriveFoldersCount = await _DriveFolder.count();
+	let allDriveFoldersCount = await _DriveFolder.count();
+	if (test && allDriveFoldersCount > limit) allDriveFoldersCount = limit;
 	for (let i = 0; i < allDriveFoldersCount; i++) {
 		const folder = await _DriveFolder.findOne({}, {
 			skip: i
@@ -365,9 +379,16 @@ async function main() {
 		}
 	}
 
-	const allDriveFilesCount = await _DriveFile.count();
+	let allDriveFilesCount = await _DriveFile.count({
+		'metadata._user.host': null,
+		'metadata.deletedAt': { $exists: false }
+	});
+	if (test && allDriveFilesCount > limit) allDriveFilesCount = limit;
 	for (let i = 0; i < allDriveFilesCount; i++) {
-		const file = await _DriveFile.findOne({}, {
+		const file = await _DriveFile.findOne({
+			'metadata._user.host': null,
+			'metadata.deletedAt': { $exists: false }
+		}, {
 			skip: i
 		});
 		try {
@@ -379,12 +400,15 @@ async function main() {
 		}
 	}
 
-	const allNotesCount = await _Note.count({
-		'_user.host': null
+	let allNotesCount = await _Note.count({
+		'_user.host': null,
+		'metadata.deletedAt': { $exists: false }
 	});
+	if (test && allNotesCount > limit) allNotesCount = limit;
 	for (let i = 0; i < allNotesCount; i++) {
 		const note = await _Note.findOne({
-			'_user.host': null
+			'_user.host': null,
+			'metadata.deletedAt': { $exists: false }
 		}, {
 			skip: i
 		});
@@ -397,7 +421,8 @@ async function main() {
 		}
 	}
 
-	const allPollVotesCount = await _PollVote.count();
+	let allPollVotesCount = await _PollVote.count();
+	if (test && allPollVotesCount > limit) allPollVotesCount = limit;
 	for (let i = 0; i < allPollVotesCount; i++) {
 		const vote = await _PollVote.findOne({}, {
 			skip: i
@@ -411,7 +436,8 @@ async function main() {
 		}
 	}
 
-	const allNoteFavoritesCount = await _Favorite.count();
+	let allNoteFavoritesCount = await _Favorite.count();
+	if (test && allNoteFavoritesCount > limit) allNoteFavoritesCount = limit;
 	for (let i = 0; i < allNoteFavoritesCount; i++) {
 		const favorite = await _Favorite.findOne({}, {
 			skip: i
@@ -425,7 +451,8 @@ async function main() {
 		}
 	}
 
-	const allNoteReactionsCount = await _NoteReaction.count();
+	let allNoteReactionsCount = await _NoteReaction.count();
+	if (test && allNoteReactionsCount > limit) allNoteReactionsCount = limit;
 	for (let i = 0; i < allNoteReactionsCount; i++) {
 		const reaction = await _NoteReaction.findOne({}, {
 			skip: i
@@ -439,7 +466,8 @@ async function main() {
 		}
 	}
 
-	const allActualUsersCount = await Users.count();
+	let allActualUsersCount = await Users.count();
+	if (test && allActualUsersCount > limit) allActualUsersCount = limit;
 	for (let i = 0; i < allActualUsersCount; i++) {
 		const [user] = await Users.find({
 			take: 1,

src/misc/acct/type.ts

@@ -1,6 +1,6 @@
 type Acct = {
 	username: string;
-	host: string;
+	host: string | null;
 };
 
 export default Acct;

src/misc/aid.ts

@@ -1,26 +0,0 @@
-// AID
-// 長さ8の[2000年1月1日からの経過ミリ秒をbase36でエンコードしたもの] + 長さnの[ランダムな文字列]
-
-const CHARS = '0123456789abcdefghijklmnopqrstuvwxyz';
-const TIME2000 = 946684800000;
-
-function getTime(time: number) {
-	time = time - TIME2000;
-	if (time < 0) time = 0;
-
-	return time.toString(36);
-}
-
-function getRandom(length: number) {
-	let str = '';
-
-	for (let i = 0; i < length; i++) {
-		str += CHARS[Math.floor(Math.random() * CHARS.length)];
-	}
-
-	return str;
-}
-
-export function genAid(date: Date, rand: number): string {
-	return getTime(date.getTime()).padStart(8, CHARS[0]) + getRandom(rand);
-}

src/misc/aidc.ts

@@ -1,26 +0,0 @@
-// AID(Cheep)
-// 長さ6の[2000年1月1日からの経過秒をbase36でエンコードしたもの] + 長さ3の[ランダムな文字列]
-
-const CHARS = '0123456789abcdefghijklmnopqrstuvwxyz';
-const TIME2000 = 946684800000;
-
-function getTime(time: number) {
-	time = time - TIME2000;
-	if (time < 0) time = 0;
-	time = Math.floor(time / 1000);
-	return time.toString(36);
-}
-
-function getRandom() {
-	let str = '';
-
-	for (let i = 0; i < 3; i++) {
-		str += CHARS[Math.floor(Math.random() * CHARS.length)];
-	}
-
-	return str;
-}
-
-export function genAidc(date: Date): string {
-	return getTime(date.getTime()).padStart(6, CHARS[0]) + getRandom();
-}

src/misc/convert-host.ts

@@ -2,7 +2,7 @@ import config from '../config';
 import { toASCII } from 'punycode';
 import { URL } from 'url';
 
-export function getFullApAccount(username: string, host: string) {
+export function getFullApAccount(username: string, host: string | null) {
 	return host ? `${username}@${toPuny(host)}` : `${username}@${toPuny(config.host)}`;
 }
 
@@ -17,6 +17,10 @@ export function extractDbHost(uri: string) {
 }
 
 export function toPuny(host: string) {
+	return toASCII(host.toLowerCase());
+}
+
+export function toPunyNullable(host: string | null | undefined): string | null {
 	if (host == null) return null;
 	return toASCII(host.toLowerCase());
 }

src/misc/detect-mine.ts

@@ -3,7 +3,7 @@ import fileType from 'file-type';
 import checkSvg from '../misc/check-svg';
 
 export async function detectMine(path: string) {
-	return new Promise<[string, string]>((res, rej) => {
+	return new Promise<[string, string | null]>((res, rej) => {
 		const readable = fs.createReadStream(path);
 		readable
 			.on('error', rej)

src/misc/donwload-url.ts

@@ -1,5 +1,4 @@
 import * as fs from 'fs';
-import * as URL from 'url';
 import * as request from 'request';
 import config from '../config';
 import chalk from 'chalk';
@@ -26,7 +25,7 @@ export async function downloadUrl(url: string, path: string) {
 			rej(error);
 		});
 
-		const requestUrl = URL.parse(url).pathname.match(/[^\u0021-\u00ff]/) ? encodeURI(url) : url;
+		const requestUrl = new URL(url).pathname.match(/[^\u0021-\u00ff]/) ? encodeURI(url) : url;
 
 		const req = request({
 			url: requestUrl,

src/misc/fetch-meta.ts

@@ -9,7 +9,6 @@ export default async function(): Promise<Meta> {
 	} else {
 		return Metas.save({
 			id: genId(),
-			hiddenTags: []
 		} as Meta);
 	}
 }

src/misc/fetch-proxy-account.ts

@@ -1,8 +1,9 @@
 import fetchMeta from './fetch-meta';
 import { ILocalUser } from '../models/entities/user';
 import { Users } from '../models';
+import { ensure } from '../prelude/ensure';
 
 export async function fetchProxyAccount(): Promise<ILocalUser> {
 	const meta = await fetchMeta();
-	return await Users.findOne({ username: meta.proxyAccount, host: null }) as ILocalUser;
+	return await Users.findOne({ username: meta.proxyAccount!, host: null }).then(ensure) as ILocalUser;
 }

src/misc/gen-id.ts

@@ -1,7 +1,7 @@
 import { ulid } from 'ulid';
-import { genAid } from './aid';
-import { genAidc } from './aidc';
-import { genObjectId } from './object-id';
+import { genAid } from './id/aid';
+import { genMeid } from './id/meid';
+import { genObjectId } from './id/object-id';
 import config from '../config';
 
 const metohd = config.id.toLowerCase();
@@ -10,11 +10,8 @@ export function genId(date?: Date): string {
 	if (!date || (date > new Date())) date = new Date();
 
 	switch (metohd) {
-		case 'aidc': return genAidc(date);
-		case 'aid1': return genAid(date, 1);
-		case 'aid2': return genAid(date, 2);
-		case 'aid3': return genAid(date, 3);
-		case 'aid4': return genAid(date, 4);
+		case 'aid': return genAid(date);
+		case 'meid': return genMeid(date);
 		case 'ulid': return ulid(date.getTime());
 		case 'objectid': return genObjectId(date);
 		default: throw 'unknown id generation method';

src/misc/id/aid.ts

@@ -0,0 +1,23 @@
+// AID
+// 長さ8の[2000年1月1日からの経過ミリ秒をbase36でエンコードしたもの] + 長さ2の[ノイズ文字列]
+
+import * as cluster from 'cluster';
+
+const TIME2000 = 946684800000;
+let counter = process.pid + (cluster.isMaster ? 0 : cluster.worker.id);
+
+function getTime(time: number) {
+	time = time - TIME2000;
+	if (time < 0) time = 0;
+
+	return time.toString(36).padStart(8, '0');
+}
+
+function getNoise() {
+	return counter.toString(36).padStart(2, '0').slice(-2);
+}
+
+export function genAid(date: Date): string {
+	counter++;
+	return getTime(date.getTime()) + getNoise();
+}

src/misc/id/meid.ts

@@ -0,0 +1,26 @@
+const CHARS = '0123456789abcdef';
+
+function getTime(time: number) {
+	if (time < 0) time = 0;
+	if (time === 0) {
+		return CHARS[0];
+	}
+
+	time += 0x800000000000;
+
+	return time.toString(16).padStart(12, CHARS[0]);
+}
+
+function getRandom() {
+	let str = '';
+
+	for (let i = 0; i < 12; i++) {
+		str += CHARS[Math.floor(Math.random() * CHARS.length)];
+	}
+
+	return str;
+}
+
+export function genMeid(date: Date): string {
+	return 'f' + getTime(date.getTime()) + getRandom();
+}

src/misc/id/object-id.ts

@@ -8,7 +8,7 @@ function getTime(time: number) {
 
 	time = Math.floor(time / 1000);
 
-	return time.toString(16);
+	return time.toString(16).padStart(8, CHARS[0]);
 }
 
 function getRandom() {

src/misc/identifiable-error.ts

@@ -7,7 +7,7 @@ export class IdentifiableError extends Error {
 
 	constructor(id: string, message?: string) {
 		super(message);
-		this.message = message;
+		this.message = message || '';
 		this.id = id;
 	}
 }

src/misc/nyaize.ts

@@ -3,7 +3,7 @@ export function nyaize(text: string): string {
 		// ja-JP
 		.replace(/な/g, 'にゃ').replace(/ナ/g, 'ニャ').replace(/ﾅ/g, 'ﾆｬ')
 		// ko-KR
-		.replace(/[나-낳]/g, (match: string) => String.fromCharCode(
-			match.codePointAt(0)  + '냐'.charCodeAt(0) - '나'.charCodeAt(0)
+		.replace(/[나-낳]/g, match => String.fromCharCode(
+			match.codePointAt(0)! + '냐'.charCodeAt(0) - '나'.charCodeAt(0)
 		));
 }

src/misc/reaction-lib.ts

@@ -20,7 +20,7 @@ export async function getFallbackReaction(): Promise<string> {
 	return  meta.useStarForReactionFallback ? 'star' : 'like';
 }
 
-export async function toDbReaction(reaction: string, enableEmoji = true): Promise<string> {
+export async function toDbReaction(reaction?: string | null, enableEmoji = true): Promise<string> {
 	if (reaction == null) return await getFallbackReaction();
 
 	// 既存の文字列リアクションはそのまま

src/misc/schema.ts

@@ -19,8 +19,8 @@ type MyType<T extends Schema> = {
 export type SchemaType<p extends Schema> =
 	p['type'] extends 'number' ? number :
 	p['type'] extends 'string' ? string :
-	p['type'] extends 'array' ? MyType<p['items']>[] :
-	p['type'] extends 'object' ? ObjType<p['properties']> :
+	p['type'] extends 'array' ? MyType<NonNullable<p['items']>>[] :
+	p['type'] extends 'object' ? ObjType<NonNullable<p['properties']>> :
 	any;
 
 export function convertOpenApiSchema(schema: Schema) {

src/models/entities/app.ts

@@ -53,7 +53,7 @@ export class App {
 	public permission: string[];
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: 'The callbackUrl of the App.'
 	})
 	public callbackUrl: string | null;

src/models/entities/auth-session.ts

@@ -19,11 +19,15 @@ export class AuthSession {
 	})
 	public token: string;
 
-	@Column(id())
+	@Column({
+		...id(),
+		nullable: true
+	})
 	public userId: User['id'];
 
 	@ManyToOne(type => User, {
-		onDelete: 'CASCADE'
+		onDelete: 'CASCADE',
+		nullable: true
 	})
 	@JoinColumn()
 	public user: User | null;

src/models/entities/emoji.ts

@@ -25,12 +25,12 @@ export class Emoji {
 	public host: string | null;
 
 	@Column('varchar', {
-		length: 256,
+		length: 512,
 	})
 	public url: string;
 
 	@Column('varchar', {
-		length: 256, nullable: true
+		length: 512, nullable: true
 	})
 	public uri: string | null;
 

src/models/entities/follow-request.ts

@@ -53,13 +53,13 @@ export class FollowRequest {
 	public followerHost: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: '[Denormalized]'
 	})
 	public followerInbox: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: '[Denormalized]'
 	})
 	public followerSharedInbox: string | null;
@@ -71,13 +71,13 @@ export class FollowRequest {
 	public followeeHost: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: '[Denormalized]'
 	})
 	public followeeInbox: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: '[Denormalized]'
 	})
 	public followeeSharedInbox: string | null;

src/models/entities/following.ts

@@ -48,13 +48,13 @@ export class Following {
 	public followerHost: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: '[Denormalized]'
 	})
 	public followerInbox: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: '[Denormalized]'
 	})
 	public followerSharedInbox: string | null;
@@ -66,13 +66,13 @@ export class Following {
 	public followeeHost: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: '[Denormalized]'
 	})
 	public followeeInbox: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: '[Denormalized]'
 	})
 	public followeeSharedInbox: string | null;

src/models/entities/meta.ts

@@ -78,27 +78,27 @@ export class Meta {
 	public blockedHosts: string[];
 
 	@Column('varchar', {
-		length: 256,
+		length: 512,
 		nullable: true,
 		default: '/assets/ai.png'
 	})
 	public mascotImageUrl: string | null;
 
 	@Column('varchar', {
-		length: 256,
+		length: 512,
 		nullable: true
 	})
 	public bannerUrl: string | null;
 
 	@Column('varchar', {
-		length: 256,
+		length: 512,
 		nullable: true,
 		default: 'https://ai.misskey.xyz/aiart/yubitun.png'
 	})
 	public errorImageUrl: string | null;
 
 	@Column('varchar', {
-		length: 256,
+		length: 512,
 		nullable: true
 	})
 	public iconUrl: string | null;

src/models/entities/note.ts

@@ -15,13 +15,6 @@ export class Note {
 	})
 	public createdAt: Date;
 
-	@Index()
-	@Column('timestamp with time zone', {
-		nullable: true,
-		comment: 'The updated date of the Note.'
-	})
-	public updatedAt: Date | null;
-
 	@Index()
 	@Column({
 		...id(),
@@ -126,7 +119,7 @@ export class Note {
 
 	@Index({ unique: true })
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: 'The URI of a note. it will be null when the note is local.'
 	})
 	public uri: string | null;
@@ -195,12 +188,6 @@ export class Note {
 	})
 	public userHost: string | null;
 
-	@Column('varchar', {
-		length: 128, nullable: true,
-		comment: '[Denormalized]'
-	})
-	public userInbox: string | null;
-
 	@Column({
 		...id(),
 		nullable: true,
@@ -227,6 +214,14 @@ export class Note {
 	})
 	public renoteUserHost: string | null;
 	//#endregion
+
+	constructor(data: Partial<Note>) {
+		if (data == null) return;
+
+		for (const [k, v] of Object.entries(data)) {
+			(this as any)[k] = v;
+		}
+	}
 }
 
 export type IMentionedRemoteUsers = {

src/models/entities/poll.ts

@@ -53,11 +53,19 @@ export class Poll {
 	})
 	public userHost: string | null;
 	//#endregion
+
+	constructor(data: Partial<Poll>) {
+		if (data == null) return;
+
+		for (const [k, v] of Object.entries(data)) {
+			(this as any)[k] = v;
+		}
+	}
 }
 
 export type IPoll = {
 	choices: string[];
 	votes?: number[];
 	multiple: boolean;
-	expiresAt: Date;
+	expiresAt: Date | null;
 };

src/models/entities/sw-subscription.ts

@@ -21,7 +21,7 @@ export class SwSubscription {
 	public user: User | null;
 
 	@Column('varchar', {
-		length: 256,
+		length: 512,
 	})
 	public endpoint: string;
 

src/models/entities/user-keypair.ts

@@ -22,4 +22,12 @@ export class UserKeypair {
 		length: 4096,
 	})
 	public privateKey: string;
+
+	constructor(data: Partial<UserKeypair>) {
+		if (data == null) return;
+
+		for (const [k, v] of Object.entries(data)) {
+			(this as any)[k] = v;
+		}
+	}
 }

src/models/entities/user-profile.ts

@@ -39,6 +39,12 @@ export class UserProfile {
 		value: string;
 	}[];
 
+	@Column('varchar', {
+		length: 512, nullable: true,
+		comment: 'Remote URL of the user.'
+	})
+	public url: string | null;
+
 	@Column('varchar', {
 		length: 128, nullable: true,
 		comment: 'The email address of the User.'
@@ -192,4 +198,12 @@ export class UserProfile {
 	})
 	public userHost: string | null;
 	//#endregion
+
+	constructor(data: Partial<UserProfile>) {
+		if (data == null) return;
+
+		for (const [k, v] of Object.entries(data)) {
+			(this as any)[k] = v;
+		}
+	}
 }

src/models/entities/user-publickey.ts

@@ -23,4 +23,12 @@ export class UserPublickey {
 		length: 4096,
 	})
 	public keyPem: string;
+
+	constructor(data: Partial<UserPublickey>) {
+		if (data == null) return;
+
+		for (const [k, v] of Object.entries(data)) {
+			(this as any)[k] = v;
+		}
+	}
 }

src/models/entities/user.ts

@@ -96,12 +96,12 @@ export class User {
 	public tags: string[];
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 	})
 	public avatarUrl: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 	})
 	public bannerUrl: string | null;
 
@@ -175,26 +175,26 @@ export class User {
 	public host: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
-		comment: 'The inbox of the User. It will be null if the origin of the user is local.'
+		length: 512, nullable: true,
+		comment: 'The inbox URL of the User. It will be null if the origin of the user is local.'
 	})
 	public inbox: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
-		comment: 'The sharedInbox of the User. It will be null if the origin of the user is local.'
+		length: 512, nullable: true,
+		comment: 'The sharedInbox URL of the User. It will be null if the origin of the user is local.'
 	})
 	public sharedInbox: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
-		comment: 'The featured of the User. It will be null if the origin of the user is local.'
+		length: 512, nullable: true,
+		comment: 'The featured URL of the User. It will be null if the origin of the user is local.'
 	})
 	public featured: string | null;
 
 	@Index()
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: 'The URI of the User. It will be null if the origin of the user is local.'
 	})
 	public uri: string | null;
@@ -205,6 +205,14 @@ export class User {
 		comment: 'The native access token of the User. It will be null if the origin of the user is local.'
 	})
 	public token: string | null;
+
+	constructor(data: Partial<User>) {
+		if (data == null) return;
+
+		for (const [k, v] of Object.entries(data)) {
+			(this as any)[k] = v;
+		}
+	}
 }
 
 export interface ILocalUser extends User {

src/models/repositories/abuse-user-report.ts

@@ -2,6 +2,7 @@ import { EntityRepository, Repository } from 'typeorm';
 import { Users } from '..';
 import rap from '@prezzemolo/rap';
 import { AbuseUserReport } from '../entities/abuse-user-report';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(AbuseUserReport)
 export class AbuseUserReportRepository extends Repository<AbuseUserReport> {
@@ -14,7 +15,7 @@ export class AbuseUserReportRepository extends Repository<AbuseUserReport> {
 	public async pack(
 		src: AbuseUserReport['id'] | AbuseUserReport,
 	) {
-		const report = typeof src === 'object' ? src : await this.findOne(src);
+		const report = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return await rap({
 			id: report.id,

src/models/repositories/app.ts

@@ -1,6 +1,7 @@
 import { EntityRepository, Repository } from 'typeorm';
 import { App } from '../entities/app';
 import { AccessTokens } from '..';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(App)
 export class AppRepository extends Repository<App> {
@@ -19,7 +20,7 @@ export class AppRepository extends Repository<App> {
 			includeProfileImageIds: false
 		}, options);
 
-		const app = typeof src === 'object' ? src : await this.findOne(src);
+		const app = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return {
 			id: app.id,

src/models/repositories/auth-session.ts

@@ -2,6 +2,7 @@ import { EntityRepository, Repository } from 'typeorm';
 import { Apps } from '..';
 import rap from '@prezzemolo/rap';
 import { AuthSession } from '../entities/auth-session';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(AuthSession)
 export class AuthSessionRepository extends Repository<AuthSession> {
@@ -9,7 +10,7 @@ export class AuthSessionRepository extends Repository<AuthSession> {
 		src: AuthSession['id'] | AuthSession,
 		me?: any
 	) {
-		const session = typeof src === 'object' ? src : await this.findOne(src);
+		const session = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return await rap({
 			id: session.id,

src/models/repositories/blocking.ts

@@ -2,6 +2,7 @@ import { EntityRepository, Repository } from 'typeorm';
 import { Users } from '..';
 import rap from '@prezzemolo/rap';
 import { Blocking } from '../entities/blocking';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(Blocking)
 export class BlockingRepository extends Repository<Blocking> {
@@ -16,7 +17,7 @@ export class BlockingRepository extends Repository<Blocking> {
 		src: Blocking['id'] | Blocking,
 		me?: any
 	) {
-		const blocking = typeof src === 'object' ? src : await this.findOne(src);
+		const blocking = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return await rap({
 			id: blocking.id,

src/models/repositories/drive-file.ts

@@ -4,6 +4,7 @@ import { Users, DriveFolders } from '..';
 import rap from '@prezzemolo/rap';
 import { User } from '../entities/user';
 import { toPuny } from '../../misc/convert-host';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(DriveFile)
 export class DriveFileRepository extends Repository<DriveFile> {
@@ -91,7 +92,7 @@ export class DriveFileRepository extends Repository<DriveFile> {
 			self: false
 		}, options);
 
-		const file = typeof src === 'object' ? src : await this.findOne(src);
+		const file = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return await rap({
 			id: file.id,
@@ -108,7 +109,7 @@ export class DriveFileRepository extends Repository<DriveFile> {
 			folder: opts.detail && file.folderId ? DriveFolders.pack(file.folderId, {
 				detail: true
 			}) : null,
-			user: opts.withUser ? Users.pack(file.userId) : null
+			user: opts.withUser ? Users.pack(file.userId!) : null
 		});
 	}
 }

src/models/repositories/drive-folder.ts

@@ -2,6 +2,7 @@ import { EntityRepository, Repository } from 'typeorm';
 import { DriveFolders, DriveFiles } from '..';
 import rap from '@prezzemolo/rap';
 import { DriveFolder } from '../entities/drive-folder';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(DriveFolder)
 export class DriveFolderRepository extends Repository<DriveFolder> {
@@ -22,7 +23,7 @@ export class DriveFolderRepository extends Repository<DriveFolder> {
 			detail: false
 		}, options);
 
-		const folder = typeof src === 'object' ? src : await this.findOne(src);
+		const folder = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return await rap({
 			id: folder.id,

src/models/repositories/follow-request.ts

@@ -1,6 +1,7 @@
 import { EntityRepository, Repository } from 'typeorm';
 import { FollowRequest } from '../entities/follow-request';
 import { Users } from '..';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(FollowRequest)
 export class FollowRequestRepository extends Repository<FollowRequest> {
@@ -8,7 +9,7 @@ export class FollowRequestRepository extends Repository<FollowRequest> {
 		src: FollowRequest['id'] | FollowRequest,
 		me?: any
 	) {
-		const request = typeof src === 'object' ? src : await this.findOne(src);
+		const request = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return {
 			id: request.id,

src/models/repositories/following.ts

@@ -2,9 +2,50 @@ import { EntityRepository, Repository } from 'typeorm';
 import { Users } from '..';
 import rap from '@prezzemolo/rap';
 import { Following } from '../entities/following';
+import { ensure } from '../../prelude/ensure';
+
+type LocalFollowerFollowing = Following & {
+	followerHost: null;
+	followerInbox: null;
+	followerSharedInbox: null;
+};
+
+type RemoteFollowerFollowing = Following & {
+	followerHost: string;
+	followerInbox: string;
+	followerSharedInbox: string;
+};
+
+type LocalFolloweeFollowing = Following & {
+	followeeHost: null;
+	followeeInbox: null;
+	followeeSharedInbox: null;
+};
+
+type RemoteFolloweeFollowing = Following & {
+	followeeHost: string;
+	followeeInbox: string;
+	followeeSharedInbox: string;
+};
 
 @EntityRepository(Following)
 export class FollowingRepository extends Repository<Following> {
+	public isLocalFollower(following: Following): following is LocalFollowerFollowing {
+		return following.followerHost == null;
+	}
+
+	public isRemoteFollower(following: Following): following is RemoteFollowerFollowing {
+		return following.followerHost != null;
+	}
+
+	public isLocalFollowee(following: Following): following is LocalFolloweeFollowing {
+		return following.followeeHost == null;
+	}
+
+	public isRemoteFollowee(following: Following): following is RemoteFolloweeFollowing {
+		return following.followeeHost != null;
+	}
+
 	public packMany(
 		followings: any[],
 		me?: any,
@@ -24,7 +65,7 @@ export class FollowingRepository extends Repository<Following> {
 			populateFollower?: boolean;
 		}
 	) {
-		const following = typeof src === 'object' ? src : await this.findOne(src);
+		const following = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		if (opts == null) opts = {};
 

src/models/repositories/games/reversi/game.ts

@@ -1,6 +1,7 @@
 import { EntityRepository, Repository } from 'typeorm';
 import { Users } from '../../..';
 import { ReversiGame } from '../../../entities/games/reversi/game';
+import { ensure } from '../../../../prelude/ensure';
 
 @EntityRepository(ReversiGame)
 export class ReversiGameRepository extends Repository<ReversiGame> {
@@ -15,7 +16,7 @@ export class ReversiGameRepository extends Repository<ReversiGame> {
 			detail: true
 		}, options);
 
-		const game = typeof src === 'object' ? src : await this.findOne(src);
+		const game = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 		const meId = me ? typeof me === 'string' ? me : me.id : null;
 
 		return {

src/models/repositories/games/reversi/matching.ts

@@ -2,6 +2,7 @@ import { EntityRepository, Repository } from 'typeorm';
 import rap from '@prezzemolo/rap';
 import { ReversiMatching } from '../../../entities/games/reversi/matching';
 import { Users } from '../../..';
+import { ensure } from '../../../../prelude/ensure';
 
 @EntityRepository(ReversiMatching)
 export class ReversiMatchingRepository extends Repository<ReversiMatching> {
@@ -9,7 +10,7 @@ export class ReversiMatchingRepository extends Repository<ReversiMatching> {
 		src: ReversiMatching['id'] | ReversiMatching,
 		me: any
 	) {
-		const matching = typeof src === 'object' ? src : await this.findOne(src);
+		const matching = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return await rap({
 			id: matching.id,

src/models/repositories/messaging-message.ts

@@ -1,6 +1,7 @@
 import { EntityRepository, Repository } from 'typeorm';
 import { MessagingMessage } from '../entities/messaging-message';
 import { Users, DriveFiles } from '..';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(MessagingMessage)
 export class MessagingMessageRepository extends Repository<MessagingMessage> {
@@ -19,7 +20,7 @@ export class MessagingMessageRepository extends Repository<MessagingMessage> {
 			populateRecipient: true
 		};
 
-		const message = typeof src === 'object' ? src : await this.findOne(src);
+		const message = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return {
 			id: message.id,

src/models/repositories/muting.ts

@@ -2,6 +2,7 @@ import { EntityRepository, Repository } from 'typeorm';
 import { Users } from '..';
 import rap from '@prezzemolo/rap';
 import { Muting } from '../entities/muting';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(Muting)
 export class MutingRepository extends Repository<Muting> {
@@ -16,7 +17,7 @@ export class MutingRepository extends Repository<Muting> {
 		src: Muting['id'] | Muting,
 		me?: any
 	) {
-		const muting = typeof src === 'object' ? src : await this.findOne(src);
+		const muting = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return await rap({
 			id: muting.id,

src/models/repositories/note-favorite.ts

@@ -1,6 +1,7 @@
 import { EntityRepository, Repository } from 'typeorm';
 import { NoteFavorite } from '../entities/note-favorite';
 import { Notes } from '..';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(NoteFavorite)
 export class NoteFavoriteRepository extends Repository<NoteFavorite> {
@@ -15,7 +16,7 @@ export class NoteFavoriteRepository extends Repository<NoteFavorite> {
 		src: NoteFavorite['id'] | NoteFavorite,
 		me?: any
 	) {
-		const favorite = typeof src === 'object' ? src : await this.findOne(src);
+		const favorite = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return {
 			id: favorite.id,

src/models/repositories/note-reaction.ts

@@ -1,6 +1,7 @@
 import { EntityRepository, Repository } from 'typeorm';
 import { NoteReaction } from '../entities/note-reaction';
 import { Users } from '..';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(NoteReaction)
 export class NoteReactionRepository extends Repository<NoteReaction> {
@@ -8,7 +9,7 @@ export class NoteReactionRepository extends Repository<NoteReaction> {
 		src: NoteReaction['id'] | NoteReaction,
 		me?: any
 	) {
-		const reaction = typeof src === 'object' ? src : await this.findOne(src);
+		const reaction = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return {
 			id: reaction.id,

src/models/repositories/note.ts

@@ -5,6 +5,7 @@ import { unique, concat } from '../../prelude/array';
 import { nyaize } from '../../misc/nyaize';
 import { Emojis, Users, Apps, PollVotes, DriveFiles, NoteReactions, Followings, Polls } from '..';
 import rap from '@prezzemolo/rap';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(Note)
 export class NoteRepository extends Repository<Note> {
@@ -12,7 +13,7 @@ export class NoteRepository extends Repository<Note> {
 		return x.trim().length <= 100;
 	}
 
-	private async hideNote(packedNote: any, meId: User['id']) {
+	private async hideNote(packedNote: any, meId: User['id'] | null) {
 		let hide = false;
 
 		// visibility が specified かつ自分が指定されていなかったら非表示
@@ -75,7 +76,7 @@ export class NoteRepository extends Repository<Note> {
 
 	public packMany(
 		notes: (Note['id'] | Note)[],
-		me?: User['id'] | User,
+		me?: User['id'] | User | null | undefined,
 		options?: {
 			detail?: boolean;
 			skipHide?: boolean;
@@ -86,7 +87,7 @@ export class NoteRepository extends Repository<Note> {
 
 	public async pack(
 		src: Note['id'] | Note,
-		me?: User['id'] | User,
+		me?: User['id'] | User | null | undefined,
 		options?: {
 			detail?: boolean;
 			skipHide?: boolean;
@@ -98,11 +99,11 @@ export class NoteRepository extends Repository<Note> {
 		}, options);
 
 		const meId = me ? typeof me === 'string' ? me : me.id : null;
-		const note = typeof src === 'object' ? src : await this.findOne(src);
+		const note = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 		const host = note.userHost;
 
 		async function populatePoll() {
-			const poll = await Polls.findOne({ noteId: note.id });
+			const poll = await Polls.findOne({ noteId: note.id }).then(ensure);
 			const choices = poll.choices.map(c => ({
 				text: c,
 				votes: poll.votes[poll.choices.indexOf(c)],
@@ -111,7 +112,7 @@ export class NoteRepository extends Repository<Note> {
 
 			if (poll.multiple) {
 				const votes = await PollVotes.find({
-					userId: meId,
+					userId: meId!,
 					noteId: note.id
 				});
 
@@ -121,7 +122,7 @@ export class NoteRepository extends Repository<Note> {
 				}
 			} else {
 				const vote = await PollVotes.findOne({
-					userId: meId,
+					userId: meId!,
 					noteId: note.id
 				});
 
@@ -139,7 +140,7 @@ export class NoteRepository extends Repository<Note> {
 
 		async function populateMyReaction() {
 			const reaction = await NoteReactions.findOne({
-				userId: meId,
+				userId: meId!,
 				noteId: note.id,
 			});
 
@@ -147,7 +148,7 @@ export class NoteRepository extends Repository<Note> {
 				return reaction.reaction;
 			}
 
-			return null;
+			return undefined;
 		}
 
 		let text = note.text;
@@ -161,15 +162,15 @@ export class NoteRepository extends Repository<Note> {
 		const packed = await rap({
 			id: note.id,
 			createdAt: note.createdAt,
-			app: note.appId ? Apps.pack(note.appId) : null,
+			app: note.appId ? Apps.pack(note.appId) : undefined,
 			userId: note.userId,
 			user: Users.pack(note.user || note.userId, meId),
 			text: text,
 			cw: note.cw,
 			visibility: note.visibility,
-			localOnly: note.localOnly,
-			visibleUserIds: note.visibleUserIds,
-			viaMobile: note.viaMobile,
+			localOnly: note.localOnly || undefined,
+			visibleUserIds: note.visibility === 'specified' ? note.visibleUserIds : undefined,
+			viaMobile: note.viaMobile || undefined,
 			renoteCount: note.renoteCount,
 			repliesCount: note.repliesCount,
 			reactions: note.reactions,
@@ -182,17 +183,18 @@ export class NoteRepository extends Repository<Note> {
 			files: DriveFiles.packMany(note.fileIds),
 			replyId: note.replyId,
 			renoteId: note.renoteId,
+			uri: note.uri,
 
 			...(opts.detail ? {
 				reply: note.replyId ? this.pack(note.replyId, meId, {
 					detail: false
-				}) : null,
+				}) : undefined,
 
 				renote: note.renoteId ? this.pack(note.renoteId, meId, {
 					detail: true
-				}) : null,
+				}) : undefined,
 
-				poll: note.hasPoll ? populatePoll() : null,
+				poll: note.hasPoll ? populatePoll() : undefined,
 
 				...(meId ? {
 					myReaction: populateMyReaction()

src/models/repositories/notification.ts

@@ -2,6 +2,7 @@ import { EntityRepository, Repository } from 'typeorm';
 import { Users, Notes } from '..';
 import rap from '@prezzemolo/rap';
 import { Notification } from '../entities/notification';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(Notification)
 export class NotificationRepository extends Repository<Notification> {
@@ -14,7 +15,7 @@ export class NotificationRepository extends Repository<Notification> {
 	public async pack(
 		src: Notification['id'] | Notification,
 	) {
-		const notification = typeof src === 'object' ? src : await this.findOne(src);
+		const notification = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return await rap({
 			id: notification.id,
@@ -23,23 +24,23 @@ export class NotificationRepository extends Repository<Notification> {
 			userId: notification.notifierId,
 			user: Users.pack(notification.notifier || notification.notifierId),
 			...(notification.type === 'mention' ? {
-				note: Notes.pack(notification.note || notification.noteId),
+				note: Notes.pack(notification.note || notification.noteId!),
 			} : {}),
 			...(notification.type === 'reply' ? {
-				note: Notes.pack(notification.note || notification.noteId),
+				note: Notes.pack(notification.note || notification.noteId!),
 			} : {}),
 			...(notification.type === 'renote' ? {
-				note: Notes.pack(notification.note || notification.noteId),
+				note: Notes.pack(notification.note || notification.noteId!),
 			} : {}),
 			...(notification.type === 'quote' ? {
-				note: Notes.pack(notification.note || notification.noteId),
+				note: Notes.pack(notification.note || notification.noteId!),
 			} : {}),
 			...(notification.type === 'reaction' ? {
-				note: Notes.pack(notification.note || notification.noteId),
+				note: Notes.pack(notification.note || notification.noteId!),
 				reaction: notification.reaction
 			} : {}),
 			...(notification.type === 'pollVote' ? {
-				note: Notes.pack(notification.note || notification.noteId),
+				note: Notes.pack(notification.note || notification.noteId!),
 				choice: notification.choice
 			} : {})
 		});

src/models/repositories/user-list.ts

@@ -1,16 +1,23 @@
 import { EntityRepository, Repository } from 'typeorm';
 import { UserList } from '../entities/user-list';
+import { ensure } from '../../prelude/ensure';
+import { UserListJoinings } from '..';
 
 @EntityRepository(UserList)
 export class UserListRepository extends Repository<UserList> {
 	public async pack(
-		src: any,
+		src: UserList['id'] | UserList,
 	) {
-		const userList = typeof src === 'object' ? src : await this.findOne(src);
+		const userList = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
+
+		const users = await UserListJoinings.find({
+			userListId: userList.id
+		});
 
 		return {
 			id: userList.id,
-			name: userList.name
+			name: userList.name,
+			userIds: users.map(x => x.userId)
 		};
 	}
 }

src/models/repositories/user.ts

@@ -2,6 +2,7 @@ import { EntityRepository, Repository, In } from 'typeorm';
 import { User, ILocalUser, IRemoteUser } from '../entities/user';
 import { Emojis, Notes, NoteUnreads, FollowRequests, Notifications, MessagingMessages, UserNotePinings, Followings, Blockings, Mutings, UserProfiles } from '..';
 import rap from '@prezzemolo/rap';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(User)
 export class UserRepository extends Repository<User> {
@@ -51,7 +52,7 @@ export class UserRepository extends Repository<User> {
 
 	public packMany(
 		users: (User['id'] | User)[],
-		me?: User['id'] | User,
+		me?: User['id'] | User | null | undefined,
 		options?: {
 			detail?: boolean,
 			includeSecrets?: boolean,
@@ -63,7 +64,7 @@ export class UserRepository extends Repository<User> {
 
 	public async pack(
 		src: User['id'] | User,
-		me?: User['id'] | User,
+		me?: User['id'] | User | null | undefined,
 		options?: {
 			detail?: boolean,
 			includeSecrets?: boolean,
@@ -75,26 +76,24 @@ export class UserRepository extends Repository<User> {
 			includeSecrets: false
 		}, options);
 
-		const user = typeof src === 'object' ? src : await this.findOne(src);
+		const user = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 		const meId = me ? typeof me === 'string' ? me : me.id : null;
 
 		const relation = meId && (meId !== user.id) && opts.detail ? await this.getRelation(meId, user.id) : null;
 		const pins = opts.detail ? await UserNotePinings.find({ userId: user.id }) : [];
-		const profile = opts.detail ? await UserProfiles.findOne({ userId: user.id }) : null;
+		const profile = opts.detail ? await UserProfiles.findOne({ userId: user.id }).then(ensure) : null;
 
 		return await rap({
 			id: user.id,
 			name: user.name,
 			username: user.username,
 			host: user.host,
-			uri: user.uri,
 			avatarUrl: user.avatarUrl,
-			bannerUrl: user.bannerUrl,
 			avatarColor: user.avatarColor,
-			bannerColor: user.bannerColor,
-			isAdmin: user.isAdmin,
-			isBot: user.isBot,
-			isCat: user.isCat,
+			isAdmin: user.isAdmin || undefined,
+			isBot: user.isBot || undefined,
+			isCat: user.isCat || undefined,
+			isVerified: user.isVerified || undefined,
 
 			// カスタム絵文字添付
 			emojis: user.emojis.length > 0 ? Emojis.find({
@@ -117,9 +116,14 @@ export class UserRepository extends Repository<User> {
 			} : {}),
 
 			...(opts.detail ? {
-				description: profile.description,
-				location: profile.location,
-				birthday: profile.birthday,
+				url: profile!.url,
+				createdAt: user.createdAt,
+				updatedAt: user.updatedAt,
+				bannerUrl: user.bannerUrl,
+				bannerColor: user.bannerColor,
+				description: profile!.description,
+				location: profile!.location,
+				birthday: profile!.birthday,
 				followersCount: user.followersCount,
 				followingCount: user.followingCount,
 				notesCount: user.notesCount,
@@ -132,9 +136,9 @@ export class UserRepository extends Repository<User> {
 			...(opts.detail && meId === user.id ? {
 				avatarId: user.avatarId,
 				bannerId: user.bannerId,
-				autoWatch: profile.autoWatch,
-				alwaysMarkNsfw: profile.alwaysMarkNsfw,
-				carefulBot: profile.carefulBot,
+				autoWatch: profile!.autoWatch,
+				alwaysMarkNsfw: profile!.alwaysMarkNsfw,
+				carefulBot: profile!.carefulBot,
 				hasUnreadMessagingMessage: MessagingMessages.count({
 					where: {
 						recipientId: user.id,
@@ -155,9 +159,9 @@ export class UserRepository extends Repository<User> {
 			} : {}),
 
 			...(opts.includeSecrets ? {
-				clientData: profile.clientData,
-				email: profile.email,
-				emailVerified: profile.emailVerified,
+				clientData: profile!.clientData,
+				email: profile!.email,
+				emailVerified: profile!.emailVerified,
 			} : {}),
 
 			...(relation ? {

src/prelude/ensure.ts

@@ -0,0 +1,10 @@
+/**
+ * 値が null または undefined の場合はエラーを発生させ、そうでない場合は値をそのまま返します
+ */
+export function ensure<T>(x: T): NonNullable<T> {
+	if (x == null) {
+		throw 'ぬるぽ';
+	} else {
+		return x!;
+	}
+}

src/queue/index.ts

@@ -12,7 +12,7 @@ import { queueLogger } from './logger';
 import { DriveFile } from '../models/entities/drive-file';
 
 function initializeQueue(name: string) {
-	return new Queue(name, config.redis != null ? {
+	return new Queue(name, {
 		redis: {
 			port: config.redis.port,
 			host: config.redis.host,
@@ -20,7 +20,15 @@ function initializeQueue(name: string) {
 			db: config.redis.db || 0,
 		},
 		prefix: config.redis.prefix ? `${config.redis.prefix}:queue` : 'queue'
-	} : null);
+	});
+}
+
+function renderError(e: Error): any {
+	return {
+		stack: e.stack,
+		message: e.message,
+		name: e.name
+	};
 }
 
 export const deliverQueue = initializeQueue('deliver');
@@ -34,16 +42,16 @@ deliverQueue
 	.on('waiting', (jobId) => deliverLogger.debug(`waiting id=${jobId}`))
 	.on('active', (job) => deliverLogger.debug(`active id=${job.id} to=${job.data.to}`))
 	.on('completed', (job, result) => deliverLogger.debug(`completed(${result}) id=${job.id} to=${job.data.to}`))
-	.on('failed', (job, err) => deliverLogger.warn(`failed(${err}) id=${job.id} to=${job.data.to}`))
-	.on('error', (error) => deliverLogger.error(`error ${error}`))
+	.on('failed', (job, err) => deliverLogger.warn(`failed(${err}) id=${job.id} to=${job.data.to}`, renderError(err)))
+	.on('error', (error) => deliverLogger.error(`error ${error}`, renderError(error)))
 	.on('stalled', (job) => deliverLogger.warn(`stalled id=${job.id} to=${job.data.to}`));
 
 inboxQueue
 	.on('waiting', (jobId) => inboxLogger.debug(`waiting id=${jobId}`))
 	.on('active', (job) => inboxLogger.debug(`active id=${job.id}`))
 	.on('completed', (job, result) => inboxLogger.debug(`completed(${result}) id=${job.id}`))
-	.on('failed', (job, err) => inboxLogger.warn(`failed(${err}) id=${job.id} activity=${job.data.activity ? job.data.activity.id : 'none'}`))
-	.on('error', (error) => inboxLogger.error(`error ${error}`))
+	.on('failed', (job, err) => inboxLogger.warn(`failed(${err}) id=${job.id} activity=${job.data.activity ? job.data.activity.id : 'none'}`, renderError(err)))
+	.on('error', (error) => inboxLogger.error(`error ${error}`, renderError(error)))
 	.on('stalled', (job) => inboxLogger.warn(`stalled id=${job.id} activity=${job.data.activity ? job.data.activity.id : 'none'}`));
 
 export function deliver(user: ILocalUser, content: any, to: any) {

src/queue/processors/db/delete-drive-files.ts

@@ -10,9 +10,11 @@ const logger = queueLogger.createSubLogger('delete-drive-files');
 export async function deleteDriveFiles(job: Bull.Job, done: any): Promise<void> {
 	logger.info(`Deleting drive files of ${job.data.user.id} ...`);
 
-	const user = await Users.findOne({
-		id: job.data.user.id
-	});
+	const user = await Users.findOne(job.data.user.id);
+	if (user == null) {
+		done();
+		return;
+	}
 
 	let deletedCount = 0;
 	let ended = false;

src/queue/processors/db/export-blocking.ts

@@ -14,9 +14,11 @@ const logger = queueLogger.createSubLogger('export-blocking');
 export async function exportBlocking(job: Bull.Job, done: any): Promise<void> {
 	logger.info(`Exporting blocking of ${job.data.user.id} ...`);
 
-	const user = await Users.findOne({
-		id: job.data.user.id
-	});
+	const user = await Users.findOne(job.data.user.id);
+	if (user == null) {
+		done();
+		return;
+	}
 
 	// Create temp file
 	const [path, cleanup] = await new Promise<[string, any]>((res, rej) => {
@@ -56,6 +58,10 @@ export async function exportBlocking(job: Bull.Job, done: any): Promise<void> {
 
 		for (const block of blockings) {
 			const u = await Users.findOne({ id: block.blockeeId });
+			if (u == null) {
+				exportedCount++; continue;
+			}
+
 			const content = getFullApAccount(u.username, u.host);
 			await new Promise((res, rej) => {
 				stream.write(content + '\n', err => {

src/queue/processors/db/export-following.ts

@@ -14,9 +14,11 @@ const logger = queueLogger.createSubLogger('export-following');
 export async function exportFollowing(job: Bull.Job, done: any): Promise<void> {
 	logger.info(`Exporting following of ${job.data.user.id} ...`);
 
-	const user = await Users.findOne({
-		id: job.data.user.id
-	});
+	const user = await Users.findOne(job.data.user.id);
+	if (user == null) {
+		done();
+		return;
+	}
 
 	// Create temp file
 	const [path, cleanup] = await new Promise<[string, any]>((res, rej) => {
@@ -56,6 +58,10 @@ export async function exportFollowing(job: Bull.Job, done: any): Promise<void> {
 
 		for (const following of followings) {
 			const u = await Users.findOne({ id: following.followeeId });
+			if (u == null) {
+				exportedCount++; continue;
+			}
+
 			const content = getFullApAccount(u.username, u.host);
 			await new Promise((res, rej) => {
 				stream.write(content + '\n', err => {

src/queue/processors/db/export-mute.ts

@@ -14,9 +14,11 @@ const logger = queueLogger.createSubLogger('export-mute');
 export async function exportMute(job: Bull.Job, done: any): Promise<void> {
 	logger.info(`Exporting mute of ${job.data.user.id} ...`);
 
-	const user = await Users.findOne({
-		id: job.data.user.id
-	});
+	const user = await Users.findOne(job.data.user.id);
+	if (user == null) {
+		done();
+		return;
+	}
 
 	// Create temp file
 	const [path, cleanup] = await new Promise<[string, any]>((res, rej) => {
@@ -56,6 +58,10 @@ export async function exportMute(job: Bull.Job, done: any): Promise<void> {
 
 		for (const mute of mutes) {
 			const u = await Users.findOne({ id: mute.muteeId });
+			if (u == null) {
+				exportedCount++; continue;
+			}
+
 			const content = getFullApAccount(u.username, u.host);
 			await new Promise((res, rej) => {
 				stream.write(content + '\n', err => {

src/queue/processors/db/export-notes.ts

@@ -9,15 +9,18 @@ import { Users, Notes, Polls } from '../../../models';
 import { MoreThan } from 'typeorm';
 import { Note } from '../../../models/entities/note';
 import { Poll } from '../../../models/entities/poll';
+import { ensure } from '../../../prelude/ensure';
 
 const logger = queueLogger.createSubLogger('export-notes');
 
 export async function exportNotes(job: Bull.Job, done: any): Promise<void> {
 	logger.info(`Exporting notes of ${job.data.user.id} ...`);
 
-	const user = await Users.findOne({
-		id: job.data.user.id
-	});
+	const user = await Users.findOne(job.data.user.id);
+	if (user == null) {
+		done();
+		return;
+	}
 
 	// Create temp file
 	const [path, cleanup] = await new Promise<[string, any]>((res, rej) => {
@@ -67,9 +70,9 @@ export async function exportNotes(job: Bull.Job, done: any): Promise<void> {
 		cursor = notes[notes.length - 1].id;
 
 		for (const note of notes) {
-			let poll: Poll;
+			let poll: Poll | undefined;
 			if (note.hasPoll) {
-				poll = await Polls.findOne({ noteId: note.id });
+				poll = await Polls.findOne({ noteId: note.id }).then(ensure);
 			}
 			const content = JSON.stringify(serialize(note, poll));
 			await new Promise((res, rej) => {
@@ -114,7 +117,7 @@ export async function exportNotes(job: Bull.Job, done: any): Promise<void> {
 	done();
 }
 
-function serialize(note: Note, poll: Poll): any {
+function serialize(note: Note, poll: Poll | null = null): any {
 	return {
 		id: note.id,
 		text: note.text,

src/queue/processors/db/export-user-lists.ts

@@ -14,9 +14,11 @@ const logger = queueLogger.createSubLogger('export-user-lists');
 export async function exportUserLists(job: Bull.Job, done: any): Promise<void> {
 	logger.info(`Exporting user lists of ${job.data.user.id} ...`);
 
-	const user = await Users.findOne({
-		id: job.data.user.id
-	});
+	const user = await Users.findOne(job.data.user.id);
+	if (user == null) {
+		done();
+		return;
+	}
 
 	const lists = await UserLists.find({
 		userId: user.id

src/queue/processors/db/import-following.ts

@@ -13,13 +13,19 @@ const logger = queueLogger.createSubLogger('import-following');
 export async function importFollowing(job: Bull.Job, done: any): Promise<void> {
 	logger.info(`Importing following of ${job.data.user.id} ...`);
 
-	const user = await Users.findOne({
-		id: job.data.user.id
-	});
+	const user = await Users.findOne(job.data.user.id);
+	if (user == null) {
+		done();
+		return;
+	}
 
 	const file = await DriveFiles.findOne({
 		id: job.data.fileId
 	});
+	if (file == null) {
+		done();
+		return;
+	}
 
 	const csv = await downloadTextFile(file.url);
 
@@ -31,11 +37,11 @@ export async function importFollowing(job: Bull.Job, done: any): Promise<void> {
 		try {
 			const { username, host } = parseAcct(line.trim());
 
-			let target = isSelfHost(host) ? await Users.findOne({
+			let target = isSelfHost(host!) ? await Users.findOne({
 				host: null,
 				usernameLower: username.toLowerCase()
 			}) : await Users.findOne({
-				host: toPuny(host),
+				host: toPuny(host!),
 				usernameLower: username.toLowerCase()
 			});
 

src/queue/processors/db/import-user-lists.ts

@@ -14,13 +14,19 @@ const logger = queueLogger.createSubLogger('import-user-lists');
 export async function importUserLists(job: Bull.Job, done: any): Promise<void> {
 	logger.info(`Importing user lists of ${job.data.user.id} ...`);
 
-	const user = await Users.findOne({
-		id: job.data.user.id
-	});
+	const user = await Users.findOne(job.data.user.id);
+	if (user == null) {
+		done();
+		return;
+	}
 
 	const file = await DriveFiles.findOne({
 		id: job.data.fileId
 	});
+	if (file == null) {
+		done();
+		return;
+	}
 
 	const csv = await downloadTextFile(file.url);
 
@@ -43,22 +49,20 @@ export async function importUserLists(job: Bull.Job, done: any): Promise<void> {
 			});
 		}
 
-		let target = isSelfHost(host) ? await Users.findOne({
+		let target = isSelfHost(host!) ? await Users.findOne({
 			host: null,
 			usernameLower: username.toLowerCase()
 		}) : await Users.findOne({
-			host: toPuny(host),
+			host: toPuny(host!),
 			usernameLower: username.toLowerCase()
 		});
 
-		if (host == null && target == null) continue;
-
-		if (await UserListJoinings.findOne({ userListId: list.id, userId: target.id }) != null) continue;
-
 		if (target == null) {
 			target = await resolveUser(username, host);
 		}
 
+		if (await UserListJoinings.findOne({ userListId: list.id, userId: target.id }) != null) continue;
+
 		pushUserToUserList(target, list);
 	}
 

src/queue/processors/deliver.ts

@@ -7,7 +7,7 @@ import { instanceChart } from '../../services/chart';
 
 const logger = new Logger('deliver');
 
-let latest: string = null;
+let latest: string | null = null;
 
 export default async (job: Bull.Job) => {
 	const { host } = new URL(job.data.to);

src/queue/processors/inbox.ts

@@ -12,7 +12,9 @@ import { Instances, Users, UserPublickeys } from '../../models';
 import { instanceChart } from '../../services/chart';
 import { UserPublickey } from '../../models/entities/user-publickey';
 import fetchMeta from '../../misc/fetch-meta';
-import { toPuny } from '../../misc/convert-host';
+import { toPuny, toPunyNullable } from '../../misc/convert-host';
+import { validActor } from '../../remote/activitypub/type';
+import { ensure } from '../../prelude/ensure';
 
 const logger = new Logger('inbox');
 
@@ -34,7 +36,7 @@ export default async (job: Bull.Job): Promise<void> => {
 
 	if (keyIdLower.startsWith('acct:')) {
 		const acct = parseAcct(keyIdLower.slice('acct:'.length));
-		const host = toPuny(acct.host);
+		const host = toPunyNullable(acct.host);
 		const username = toPuny(acct.username);
 
 		if (host === null) {
@@ -63,9 +65,7 @@ export default async (job: Bull.Job): Promise<void> => {
 			host: host
 		}) as IRemoteUser;
 
-		key = await UserPublickeys.findOne({
-			userId: user.id
-		});
+		key = await UserPublickeys.findOne(user.id).then(ensure);
 	} else {
 		// アクティビティ内のホストの検証
 		const host = toPuny(new URL(signature.keyId).hostname);
@@ -86,14 +86,14 @@ export default async (job: Bull.Job): Promise<void> => {
 
 		key = await UserPublickeys.findOne({
 			keyId: signature.keyId
-		});
+		}).then(ensure);
 
 		user = await Users.findOne(key.userId) as IRemoteUser;
 	}
 
 	// Update Person activityの場合は、ここで署名検証/更新処理まで実施して終了
 	if (activity.type === 'Update') {
-		if (activity.object && activity.object.type === 'Person') {
+		if (activity.object && validActor.includes(activity.object.type)) {
 			if (user == null) {
 				logger.warn('Update activity received, but user not registed.');
 			} else if (!httpSignature.verifySignature(signature, key.keyPem)) {

src/remote/activitypub/kernel/accept/follow.ts

@@ -6,9 +6,10 @@ import { Users } from '../../../../models';
 
 export default async (actor: IRemoteUser, activity: IFollow): Promise<void> => {
 	const id = typeof activity.actor == 'string' ? activity.actor : activity.actor.id;
+	if (id == null) throw 'missing id';
 
 	if (!id.startsWith(config.url + '/')) {
-		return null;
+		return;
 	}
 
 	const follower = await Users.findOne({

src/remote/activitypub/kernel/add/index.ts

@@ -14,6 +14,7 @@ export default async (actor: IRemoteUser, activity: IAdd): Promise<void> => {
 
 	if (activity.target === actor.featured) {
 		const note = await resolveNote(activity.object);
+		if (note == null) throw new Error('note not found');
 		await addPinned(actor, note.id);
 		return;
 	}

src/remote/activitypub/kernel/announce/note.ts

@@ -53,16 +53,16 @@ export default async function(resolver: Resolver, actor: IRemoteUser, activity:
 	logger.info(`Creating the (Re)Note: ${uri}`);
 
 	//#region Visibility
-	const visibility = getVisibility(activity.to, activity.cc, actor);
+	const visibility = getVisibility(activity.to || [], activity.cc || [], actor);
 
 	let visibleUsers: User[] = [];
 	if (visibility == 'specified') {
-		visibleUsers = await Promise.all(note.to.map(uri => resolvePerson(uri)));
+		visibleUsers = await Promise.all((note.to || []).map(uri => resolvePerson(uri)));
 	}
 	//#endergion
 
 	await post(actor, {
-		createdAt: new Date(activity.published),
+		createdAt: activity.published ? new Date(activity.published) : null,
 		renote,
 		visibility,
 		visibleUsers,
@@ -75,9 +75,6 @@ type visibility = 'public' | 'home' | 'followers' | 'specified';
 function getVisibility(to: string[], cc: string[], actor: IRemoteUser): visibility {
 	const PUBLIC = 'https://www.w3.org/ns/activitystreams#Public';
 
-	to = to || [];
-	cc = cc || [];
-
 	if (to.includes(PUBLIC)) {
 		return 'public';
 	} else if (cc.includes(PUBLIC)) {

src/remote/activitypub/kernel/block/index.ts

@@ -9,13 +9,14 @@ const logger = apLogger;
 
 export default async (actor: IRemoteUser, activity: IBlock): Promise<void> => {
 	const id = typeof activity.object == 'string' ? activity.object : activity.object.id;
+	if (id == null) throw 'missing id';
 
 	const uri = activity.id || activity;
 
 	logger.info(`Block: ${uri}`);
 
 	if (!id.startsWith(config.url + '/')) {
-		return null;
+		return;
 	}
 
 	const blockee = await Users.findOne(id.split('/').pop());

src/remote/activitypub/kernel/follow.ts

@@ -6,9 +6,10 @@ import { Users } from '../../../models';
 
 export default async (actor: IRemoteUser, activity: IFollow): Promise<void> => {
 	const id = typeof activity.object == 'string' ? activity.object : activity.object.id;
+	if (id == null) throw 'missing id';
 
 	if (!id.startsWith(config.url + '/')) {
-		return null;
+		return;
 	}
 
 	const followee = await Users.findOne(id.split('/').pop());

src/remote/activitypub/kernel/index.ts

@@ -71,7 +71,7 @@ const self = async (actor: IRemoteUser, activity: Object): Promise<void> => {
 
 	default:
 		apLogger.warn(`unknown activity type: ${(activity as any).type}`);
-		return null;
+		return;
 	}
 };
 

src/remote/activitypub/kernel/like.ts

@@ -5,6 +5,7 @@ import { Notes } from '../../../models';
 
 export default async (actor: IRemoteUser, activity: ILike) => {
 	const id = typeof activity.object == 'string' ? activity.object : activity.object.id;
+	if (id == null) throw 'missing id';
 
 	// Transform:
 	// https://misskey.ex/notes/xxxx to

src/remote/activitypub/kernel/reject/follow.ts

@@ -6,9 +6,10 @@ import { Users } from '../../../../models';
 
 export default async (actor: IRemoteUser, activity: IFollow): Promise<void> => {
 	const id = typeof activity.actor == 'string' ? activity.actor : activity.actor.id;
+	if (id == null) throw 'missing id';
 
 	if (!id.startsWith(config.url + '/')) {
-		return null;
+		return;
 	}
 
 	const follower = await Users.findOne(id.split('/').pop());

src/remote/activitypub/kernel/remove/index.ts

@@ -14,6 +14,7 @@ export default async (actor: IRemoteUser, activity: IRemove): Promise<void> => {
 
 	if (activity.target === actor.featured) {
 		const note = await resolveNote(activity.object);
+		if (note == null) throw new Error('note not found');
 		await removePinned(actor, note.id);
 		return;
 	}

src/remote/activitypub/kernel/undo/block.ts

@@ -9,13 +9,14 @@ const logger = apLogger;
 
 export default async (actor: IRemoteUser, activity: IBlock): Promise<void> => {
 	const id = typeof activity.object == 'string' ? activity.object : activity.object.id;
+	if (id == null) throw 'missing id';
 
 	const uri = activity.id || activity;
 
 	logger.info(`UnBlock: ${uri}`);
 
 	if (!id.startsWith(config.url + '/')) {
-		return null;
+		return;
 	}
 
 	const blockee = await Users.findOne(id.split('/').pop());

src/remote/activitypub/kernel/undo/follow.ts

@@ -7,9 +7,10 @@ import { Users, FollowRequests, Followings } from '../../../../models';
 
 export default async (actor: IRemoteUser, activity: IFollow): Promise<void> => {
 	const id = typeof activity.object == 'string' ? activity.object : activity.object.id;
+	if (id == null) throw 'missing id';
 
 	if (!id.startsWith(config.url + '/')) {
-		return null;
+		return;
 	}
 
 	const followee = await Users.findOne(id.split('/').pop());

src/remote/activitypub/kernel/undo/index.ts

@@ -39,6 +39,4 @@ export default async (actor: IRemoteUser, activity: IUndo): Promise<void> => {
 			undoLike(actor, object as ILike);
 			break;
 	}
-
-	return null;
 };

src/remote/activitypub/kernel/undo/like.ts

@@ -8,6 +8,7 @@ import { Notes } from '../../../../models';
  */
 export default async (actor: IRemoteUser, activity: ILike): Promise<void> => {
 	const id = typeof activity.object == 'string' ? activity.object : activity.object.id;
+	if (id == null) throw 'missing id';
 
 	const noteId = id.split('/').pop();
 

src/remote/activitypub/models/image.ts

@@ -5,6 +5,7 @@ import fetchMeta from '../../../misc/fetch-meta';
 import { apLogger } from '../logger';
 import { DriveFile } from '../../../models/entities/drive-file';
 import { DriveFiles } from '../../../models';
+import { ensure } from '../../../prelude/ensure';
 
 const logger = apLogger;
 
@@ -14,7 +15,7 @@ const logger = apLogger;
 export async function createImage(actor: IRemoteUser, value: any): Promise<DriveFile> {
 	// 投稿者が凍結されていたらスキップ
 	if (actor.isSuspended) {
-		return null;
+		throw new Error('actor has been suspended');
 	}
 
 	const image = await new Resolver().resolve(value) as any;
@@ -28,17 +29,7 @@ export async function createImage(actor: IRemoteUser, value: any): Promise<Drive
 	const instance = await fetchMeta();
 	const cache = instance.cacheRemoteFiles;
 
-	let file;
-	try {
-		file = await uploadFromUrl(image.url, actor, null, image.url, image.sensitive, false, !cache);
-	} catch (e) {
-		// 4xxの場合は添付されてなかったことにする
-		if (e >= 400 && e < 500) {
-			logger.warn(`Ignored image: ${image.url} - ${e}`);
-			return null;
-		}
-		throw e;
-	}
+	let file = await uploadFromUrl(image.url, actor, null, image.url, image.sensitive, false, !cache);
 
 	if (file.isLink) {
 		// URLが異なっている場合、同じ画像が以前に異なるURLで登録されていたということなので、
@@ -49,7 +40,7 @@ export async function createImage(actor: IRemoteUser, value: any): Promise<Drive
 				uri: image.url
 			});
 
-			file = DriveFiles.findOne(file.id);
+			file = await DriveFiles.findOne(file.id).then(ensure);
 		}
 	}
 

src/remote/activitypub/models/note.ts

@@ -21,6 +21,7 @@ import { IObject, INote } from '../type';
 import { Emoji } from '../../../models/entities/emoji';
 import { genId } from '../../../misc/gen-id';
 import fetchMeta from '../../../misc/fetch-meta';
+import { ensure } from '../../../prelude/ensure';
 
 const logger = apLogger;
 
@@ -29,13 +30,14 @@ const logger = apLogger;
  *
  * Misskeyに対象のNoteが登録されていればそれを返します。
  */
-export async function fetchNote(value: string | IObject, resolver?: Resolver): Promise<Note> {
+export async function fetchNote(value: string | IObject, resolver?: Resolver): Promise<Note | null> {
 	const uri = typeof value == 'string' ? value : value.id;
+	if (uri == null) throw 'missing uri';
 
 	// URIがこのサーバーを指しているならデータベースからフェッチ
 	if (uri.startsWith(config.url + '/')) {
 		const id = uri.split('/').pop();
-		return await Notes.findOne(id);
+		return await Notes.findOne(id).then(x => x || null);
 	}
 
 	//#region このサーバーに既に登録されていたらそれを返す
@@ -52,7 +54,7 @@ export async function fetchNote(value: string | IObject, resolver?: Resolver): P
 /**
  * Noteを作成します。
  */
-export async function createNote(value: any, resolver?: Resolver, silent = false): Promise<Note> {
+export async function createNote(value: any, resolver?: Resolver, silent = false): Promise<Note | null> {
 	if (resolver == null) resolver = new Resolver();
 
 	const object: any = await resolver.resolve(value);
@@ -65,7 +67,7 @@ export async function createNote(value: any, resolver?: Resolver, silent = false
 			value: value,
 			object: object
 		});
-		return null;
+		throw 'invalid note';
 	}
 
 	const note: INote = object;
@@ -75,11 +77,11 @@ export async function createNote(value: any, resolver?: Resolver, silent = false
 	logger.info(`Creating the Note: ${note.id}`);
 
 	// 投稿者をフェッチ
-	const actor = await resolvePerson(note.attributedTo, null, resolver) as IRemoteUser;
+	const actor = await resolvePerson(note.attributedTo, resolver) as IRemoteUser;
 
 	// 投稿者が凍結されていたらスキップ
 	if (actor.isSuspended) {
-		return null;
+		throw 'actor has been suspended';
 	}
 
 	//#region Visibility
@@ -95,9 +97,9 @@ export async function createNote(value: any, resolver?: Resolver, silent = false
 			visibility = 'followers';
 		} else {
 			visibility = 'specified';
-			visibleUsers = await Promise.all(note.to.map(uri => resolvePerson(uri, null, resolver)));
+			visibleUsers = await Promise.all(note.to.map(uri => resolvePerson(uri, resolver)));
 		}
-}
+	}
 	//#endergion
 
 	const apMentions = await extractMentionedUsers(actor, note.to, note.cc, resolver);
@@ -118,20 +120,22 @@ export async function createNote(value: any, resolver?: Resolver, silent = false
 		: [];
 
 	// リプライ
-	const reply: Note = note.inReplyTo
-		? await resolveNote(note.inReplyTo, resolver).catch(e => {
-			// 4xxの場合はリプライしてないことにする
-			if (e.statusCode >= 400 && e.statusCode < 500) {
-				logger.warn(`Ignored inReplyTo ${note.inReplyTo} - ${e.statusCode} `);
-				return null;
+	const reply: Note | null = note.inReplyTo
+		? await resolveNote(note.inReplyTo, resolver).then(x => {
+			if (x == null) {
+				logger.warn(`Specified inReplyTo, but nout found`);
+				throw 'inReplyTo not found';
+			} else {
+				return x;
 			}
+		}).catch(e => {
 			logger.warn(`Error in inReplyTo ${note.inReplyTo} - ${e.statusCode || e}`);
 			throw e;
 		})
 		: null;
 
 	// 引用
-	let quote: Note;
+	let quote: Note | undefined | null;
 
 	if (note._misskey_quote && typeof note._misskey_quote == 'string') {
 		quote = await resolveNote(note._misskey_quote).catch(e => {
@@ -148,11 +152,12 @@ export async function createNote(value: any, resolver?: Resolver, silent = false
 	const cw = note.summary === '' ? null : note.summary;
 
 	// テキストのパース
-	const text = note._misskey_content || fromHtml(note.content);
+	const text = note._misskey_content || (note.content ? fromHtml(note.content) : null);
 
 	// vote
 	if (reply && reply.hasPoll) {
-		const poll = await Polls.findOne({ noteId: reply.id });
+		const poll = await Polls.findOne(reply.id).then(ensure);
+
 		const tryCreateVote = async (name: string, index: number): Promise<null> => {
 			if (poll.expiresAt && Date.now() > new Date(poll.expiresAt).getTime()) {
 				logger.warn(`vote to expired poll from AP: actor=${actor.username}@${actor.host}, note=${note.id}, choice=${name}`);
@@ -180,7 +185,7 @@ export async function createNote(value: any, resolver?: Resolver, silent = false
 		}
 	}
 
-	const emojis = await extractEmojis(note.tag, actor.host).catch(e => {
+	const emojis = await extractEmojis(note.tag || [], actor.host).catch(e => {
 		logger.info(`extractEmojis: ${e}`);
 		return [] as Emoji[];
 	});
@@ -196,7 +201,7 @@ export async function createNote(value: any, resolver?: Resolver, silent = false
 	}
 
 	return await post(actor, {
-		createdAt: new Date(note.published),
+		createdAt: note.published ? new Date(note.published) : null,
 		files,
 		reply,
 		renote: quote,
@@ -223,8 +228,9 @@ export async function createNote(value: any, resolver?: Resolver, silent = false
  * Misskeyに対象のNoteが登録されていればそれを返し、そうでなければ
  * リモートサーバーからフェッチしてMisskeyに登録しそれを返します。
  */
-export async function resolveNote(value: string | IObject, resolver?: Resolver): Promise<Note> {
+export async function resolveNote(value: string | IObject, resolver?: Resolver): Promise<Note | null> {
 	const uri = typeof value == 'string' ? value : value.id;
+	if (uri == null) throw 'missing uri';
 
 	// ブロックしてたら中断
 	// TODO: いちいちデータベースにアクセスするのはコスト高そうなのでどっかにキャッシュしておく
@@ -242,71 +248,81 @@ export async function resolveNote(value: string | IObject, resolver?: Resolver):
 	// リモートサーバーからフェッチしてきて登録
 	// ここでuriの代わりに添付されてきたNote Objectが指定されていると、サーバーフェッチを経ずにノートが生成されるが
 	// 添付されてきたNote Objectは偽装されている可能性があるため、常にuriを指定してサーバーフェッチを行う。
-	return await createNote(uri, resolver);
+	return await createNote(uri, resolver).catch(e => {
+		if (e.name === 'duplicated') {
+			return fetchNote(uri).then(note => {
+				if (note == null) {
+					throw 'something happened';
+				} else {
+					return note;
+				}
+			});
+		} else {
+			throw e;
+		}
+	});
 }
 
-export async function extractEmojis(tags: ITag[], host: string) {
+export async function extractEmojis(tags: ITag[], host: string): Promise<Emoji[]> {
 	host = toPuny(host);
 
 	if (!tags) return [];
 
-	const eomjiTags = tags.filter(tag => tag.type === 'Emoji' && tag.icon && tag.icon.url);
+	const eomjiTags = tags.filter(tag => tag.type === 'Emoji' && tag.icon && tag.icon.url && tag.name);
 
-	return await Promise.all(
-		eomjiTags.map(async tag => {
-			const name = tag.name.replace(/^:/, '').replace(/:$/, '');
+	return await Promise.all(eomjiTags.map(async tag => {
+		const name = tag.name!.replace(/^:/, '').replace(/:$/, '');
 
-			const exists = await Emojis.findOne({
-				host,
-				name
-			});
+		const exists = await Emojis.findOne({
+			host,
+			name
+		});
 
-			if (exists) {
-				if ((tag.updated != null && exists.updatedAt == null)
-					|| (tag.id != null && exists.uri == null)
-					|| (tag.updated != null && exists.updatedAt != null && new Date(tag.updated) > exists.updatedAt)
-				) {
-					await Emojis.update({
-						host,
-						name,
-					}, {
-						uri: tag.id,
-						url: tag.icon.url,
-						updatedAt: new Date(tag.updated),
-					});
+		if (exists) {
+			if ((tag.updated != null && exists.updatedAt == null)
+				|| (tag.id != null && exists.uri == null)
+				|| (tag.updated != null && exists.updatedAt != null && new Date(tag.updated) > exists.updatedAt)
+			) {
+				await Emojis.update({
+					host,
+					name,
+				}, {
+					uri: tag.id,
+					url: tag.icon!.url,
+					updatedAt: new Date(tag.updated!),
+				});
 
-					return await Emojis.findOne({
-						host,
-						name
-					});
-				}
-
-				return exists;
+				return await Emojis.findOne({
+					host,
+					name
+				}) as Emoji;
 			}
 
-			logger.info(`register emoji host=${host}, name=${name}`);
+			return exists;
+		}
 
-			return await Emojis.save({
-				id: genId(),
-				host,
-				name,
-				uri: tag.id,
-				url: tag.icon.url,
-				updatedAt: tag.updated ? new Date(tag.updated) : undefined,
-				aliases: []
-			} as Emoji);
-		})
-	);
+		logger.info(`register emoji host=${host}, name=${name}`);
+
+		return await Emojis.save({
+			id: genId(),
+			host,
+			name,
+			uri: tag.id,
+			url: tag.icon!.url,
+			updatedAt: tag.updated ? new Date(tag.updated) : undefined,
+			aliases: []
+		} as Partial<Emoji>);
+	}));
 }
 
 async function extractMentionedUsers(actor: IRemoteUser, to: string[], cc: string[], resolver: Resolver) {
 	const ignoreUris = ['https://www.w3.org/ns/activitystreams#Public', `${actor.uri}/followers`];
 	const uris = difference(unique(concat([to || [], cc || []])), ignoreUris);
 
-	const limit = promiseLimit(2);
+	const limit = promiseLimit<User | null>(2);
 	const users = await Promise.all(
-		uris.map(uri => limit(() => resolvePerson(uri, null, resolver).catch(() => null)) as Promise<User>)
+		uris.map(uri => limit(() => resolvePerson(uri, resolver).catch(() => null)) as Promise<User | null>)
 	);
 
-	return users.filter(x => x != null);
+	return users.filter(x => x != null) as User[];
 }

src/remote/activitypub/models/person.ts

@@ -24,6 +24,9 @@ import { UserPublickey } from '../../../models/entities/user-publickey';
 import { isDuplicateKeyValueError } from '../../../misc/is-duplicate-key-value-error';
 import { toPuny } from '../../../misc/convert-host';
 import { UserProfile } from '../../../models/entities/user-profile';
+import { validActor } from '../../../remote/activitypub/type';
+import { getConnection } from 'typeorm';
+import { ensure } from '../../../prelude/ensure';
 const logger = apLogger;
 
 /**
@@ -38,7 +41,7 @@ function validatePerson(x: any, uri: string) {
 		return new Error('invalid person: object is null');
 	}
 
-	if (x.type != 'Person' && x.type != 'Service') {
+	if (!validActor.includes(x.type)) {
 		return new Error(`invalid person: object is not a person or service '${x.type}'`);
 	}
 
@@ -84,13 +87,13 @@ function validatePerson(x: any, uri: string) {
  *
  * Misskeyに対象のPersonが登録されていればそれを返します。
  */
-export async function fetchPerson(uri: string, resolver?: Resolver): Promise<User> {
+export async function fetchPerson(uri: string, resolver?: Resolver): Promise<User | null> {
 	if (typeof uri !== 'string') throw 'uri is not string';
 
 	// URIがこのサーバーを指しているならデータベースからフェッチ
 	if (uri.startsWith(config.url + '/')) {
 		const id = uri.split('/').pop();
-		return await Users.findOne(id);
+		return await Users.findOne(id).then(x => x || null);
 	}
 
 	//#region このサーバーに既に登録されていたらそれを返す
@@ -126,7 +129,7 @@ export async function createPerson(uri: string, resolver?: Resolver): Promise<Us
 
 	const host = toPuny(new URL(object.id).hostname);
 
-	const { fields } = analyzeAttachments(person.attachment);
+	const { fields } = analyzeAttachments(person.attachment || []);
 
 	const tags = extractHashtags(person.tag).map(tag => tag.toLowerCase());
 
@@ -135,27 +138,42 @@ export async function createPerson(uri: string, resolver?: Resolver): Promise<Us
 	// Create user
 	let user: IRemoteUser;
 	try {
-		user = await Users.save({
-			id: genId(),
-			avatarId: null,
-			bannerId: null,
-			createdAt: Date.parse(person.published) || new Date(),
-			lastFetchedAt: new Date(),
-			name: person.name,
-			isLocked: person.manuallyApprovesFollowers,
-			username: person.preferredUsername,
-			usernameLower: person.preferredUsername.toLowerCase(),
-			host,
-			inbox: person.inbox,
-			sharedInbox: person.sharedInbox || (person.endpoints ? person.endpoints.sharedInbox : undefined),
-			featured: person.featured,
-			endpoints: person.endpoints,
-			uri: person.id,
-			url: person.url,
-			tags,
-			isBot,
-			isCat: (person as any).isCat === true
-		} as Partial<User>) as IRemoteUser;
+		// Start transaction
+		await getConnection().transaction(async transactionalEntityManager => {
+			user = await transactionalEntityManager.save(new User({
+				id: genId(),
+				avatarId: null,
+				bannerId: null,
+				createdAt: new Date(),
+				lastFetchedAt: new Date(),
+				name: person.name,
+				isLocked: person.manuallyApprovesFollowers,
+				username: person.preferredUsername,
+				usernameLower: person.preferredUsername.toLowerCase(),
+				host,
+				inbox: person.inbox,
+				sharedInbox: person.sharedInbox || (person.endpoints ? person.endpoints.sharedInbox : undefined),
+				featured: person.featured,
+				uri: person.id,
+				tags,
+				isBot,
+				isCat: (person as any).isCat === true
+			})) as IRemoteUser;
+
+			await transactionalEntityManager.save(new UserProfile({
+				userId: user.id,
+				description: person.summary ? fromHtml(person.summary) : null,
+				url: person.url,
+				fields,
+				userHost: host
+			}));
+
+			await transactionalEntityManager.save(new UserPublickey({
+				userId: user.id,
+				keyId: person.publicKey.id,
+				keyPem: person.publicKey.publicKeyPem
+			}));
+		});
 	} catch (e) {
 		// duplicate key error
 		if (isDuplicateKeyValueError(e)) {
@@ -166,39 +184,26 @@ export async function createPerson(uri: string, resolver?: Resolver): Promise<Us
 		throw e;
 	}
 
-	await UserProfiles.save({
-		userId: user.id,
-		description: fromHtml(person.summary),
-		fields,
-		userHost: host
-	} as Partial<UserProfile>);
-
-	await UserPublickeys.save({
-		userId: user.id,
-		keyId: person.publicKey.id,
-		keyPem: person.publicKey.publicKeyPem
-	} as UserPublickey);
-
 	// Register host
 	registerOrFetchInstanceDoc(host).then(i => {
 		Instances.increment({ id: i.id }, 'usersCount', 1);
 		instanceChart.newUser(i.host);
 	});
 
-	usersChart.update(user, true);
+	usersChart.update(user!, true);
 
 	// ハッシュタグ更新
-	for (const tag of tags) updateHashtag(user, tag, true, true);
-	for (const tag of (user.tags || []).filter(x => !tags.includes(x))) updateHashtag(user, tag, true, false);
+	for (const tag of tags) updateHashtag(user!, tag, true, true);
+	for (const tag of (user!.tags || []).filter(x => !tags.includes(x))) updateHashtag(user!, tag, true, false);
 
 	//#region アイコンとヘッダー画像をフェッチ
-	const [avatar, banner] = (await Promise.all<DriveFile>([
+	const [avatar, banner] = (await Promise.all<DriveFile | null>([
 		person.icon,
 		person.image
 	].map(img =>
 		img == null
 			? Promise.resolve(null)
-			: resolveImage(user, img).catch(() => null)
+			: resolveImage(user!, img).catch(() => null)
 	)));
 
 	const avatarId = avatar ? avatar.id : null;
@@ -206,9 +211,9 @@ export async function createPerson(uri: string, resolver?: Resolver): Promise<Us
 	const avatarUrl = avatar ? DriveFiles.getPublicUrl(avatar) : null;
 	const bannerUrl = banner ? DriveFiles.getPublicUrl(banner) : null;
 	const avatarColor = avatar && avatar.properties.avgColor ? avatar.properties.avgColor : null;
-	const bannerColor = banner && avatar.properties.avgColor ? banner.properties.avgColor : null;
+	const bannerColor = banner && banner.properties.avgColor ? banner.properties.avgColor : null;
 
-	await Users.update(user.id, {
+	await Users.update(user!.id, {
 		avatarId,
 		bannerId,
 		avatarUrl,
@@ -217,30 +222,30 @@ export async function createPerson(uri: string, resolver?: Resolver): Promise<Us
 		bannerColor
 	});
 
-	user.avatarId = avatarId;
-	user.bannerId = bannerId;
-	user.avatarUrl = avatarUrl;
-	user.bannerUrl = bannerUrl;
-	user.avatarColor = avatarColor;
-	user.bannerColor = bannerColor;
+	user!.avatarId = avatarId;
+	user!.bannerId = bannerId;
+	user!.avatarUrl = avatarUrl;
+	user!.bannerUrl = bannerUrl;
+	user!.avatarColor = avatarColor;
+	user!.bannerColor = bannerColor;
 	//#endregion
 
 	//#region カスタム絵文字取得
-	const emojis = await extractEmojis(person.tag, host).catch(e => {
+	const emojis = await extractEmojis(person.tag || [], host).catch(e => {
 		logger.info(`extractEmojis: ${e}`);
 		return [] as Emoji[];
 	});
 
 	const emojiNames = emojis.map(emoji => emoji.name);
 
-	await Users.update(user.id, {
+	await Users.update(user!.id, {
 		emojis: emojiNames
 	});
 	//#endregion
 
-	await updateFeatured(user.id).catch(err => logger.error(err));
+	await updateFeatured(user!.id).catch(err => logger.error(err));
 
-	return user;
+	return user!;
 }
 
 /**
@@ -250,7 +255,7 @@ export async function createPerson(uri: string, resolver?: Resolver): Promise<Us
  * @param resolver Resolver
  * @param hint Hint of Person object (この値が正当なPersonの場合、Remote resolveをせずに更新に利用します)
  */
-export async function updatePerson(uri: string, resolver?: Resolver, hint?: object): Promise<void> {
+export async function updatePerson(uri: string, resolver?: Resolver | null, hint?: object): Promise<void> {
 	if (typeof uri !== 'string') throw 'uri is not string';
 
 	// URIがこのサーバーを指しているならスキップ
@@ -286,7 +291,7 @@ export async function updatePerson(uri: string, resolver?: Resolver, hint?: obje
 	logger.info(`Updating the Person: ${person.id}`);
 
 	// アイコンとヘッダー画像をフェッチ
-	const [avatar, banner] = (await Promise.all<DriveFile>([
+	const [avatar, banner] = (await Promise.all<DriveFile | null>([
 		person.icon,
 		person.image
 	].map(img =>
@@ -296,14 +301,14 @@ export async function updatePerson(uri: string, resolver?: Resolver, hint?: obje
 	)));
 
 	// カスタム絵文字取得
-	const emojis = await extractEmojis(person.tag, exist.host).catch(e => {
+	const emojis = await extractEmojis(person.tag || [], exist.host).catch(e => {
 		logger.info(`extractEmojis: ${e}`);
 		return [] as Emoji[];
 	});
 
 	const emojiNames = emojis.map(emoji => emoji.name);
 
-	const { fields, services } = analyzeAttachments(person.attachment);
+	const { fields, services } = analyzeAttachments(person.attachment || []);
 
 	const tags = extractHashtags(person.tag).map(tag => tag.toLowerCase());
 
@@ -313,7 +318,7 @@ export async function updatePerson(uri: string, resolver?: Resolver, hint?: obje
 		sharedInbox: person.sharedInbox || (person.endpoints ? person.endpoints.sharedInbox : undefined),
 		featured: person.featured,
 		emojis: emojiNames,
-		description: fromHtml(person.summary),
+		description: person.summary ? fromHtml(person.summary) : null,
 		name: person.name,
 		url: person.url,
 		endpoints: person.endpoints,
@@ -322,7 +327,6 @@ export async function updatePerson(uri: string, resolver?: Resolver, hint?: obje
 		isBot: object.type == 'Service',
 		isCat: (person as any).isCat === true,
 		isLocked: person.manuallyApprovesFollowers,
-		createdAt: new Date(Date.parse(person.published)) || null,
 	} as Partial<User>;
 
 	if (avatar) {
@@ -375,7 +379,7 @@ export async function updatePerson(uri: string, resolver?: Resolver, hint?: obje
  * Misskeyに対象のPersonが登録されていればそれを返し、そうでなければ
  * リモートサーバーからフェッチしてMisskeyに登録しそれを返します。
  */
-export async function resolvePerson(uri: string, verifier?: string, resolver?: Resolver): Promise<User> {
+export async function resolvePerson(uri: string, resolver?: Resolver): Promise<User> {
 	if (typeof uri !== 'string') throw 'uri is not string';
 
 	//#region このサーバーに既に登録されていたらそれを返す
@@ -435,21 +439,24 @@ export function analyzeAttachments(attachments: ITag[]) {
 	}[] = [];
 	const services: { [x: string]: any } = {};
 
-	if (Array.isArray(attachments))
-		for (const attachment of attachments.filter(isPropertyValue))
-			if (isPropertyValue(attachment.identifier))
-				addService(services, attachment.identifier);
-			else
+	if (Array.isArray(attachments)) {
+		for (const attachment of attachments.filter(isPropertyValue)) {
+			if (isPropertyValue(attachment.identifier!)) {
+				addService(services, attachment.identifier!);
+			} else {
 				fields.push({
-					name: attachment.name,
-					value: fromHtml(attachment.value)
+					name: attachment.name!,
+					value: fromHtml(attachment.value!)
 				});
+			}
+		}
+	}
 
 	return { fields, services };
 }
 
 export async function updateFeatured(userId: User['id']) {
-	const user = await Users.findOne(userId);
+	const user = await Users.findOne(userId).then(ensure);
 	if (!Users.isRemoteUser(user)) return;
 	if (!user.featured) return;
 
@@ -467,18 +474,18 @@ export async function updateFeatured(userId: User['id']) {
 	if (!Array.isArray(items)) throw new Error(`Collection items is not an array`);
 
 	// Resolve and regist Notes
-	const limit = promiseLimit(2);
+	const limit = promiseLimit<Note | null>(2);
 	const featuredNotes = await Promise.all(items
 		.filter(item => item.type === 'Note')
 		.slice(0, 5)
-		.map(item => limit(() => resolveNote(item, resolver)) as Promise<Note>));
+		.map(item => limit(() => resolveNote(item, resolver))));
 
 	for (const note of featuredNotes.filter(note => note != null)) {
 		UserNotePinings.save({
 			id: genId(),
 			createdAt: new Date(),
 			userId: user.id,
-			noteId: note.id
+			noteId: note!.id
 		} as UserNotePining);
 	}
 }

src/remote/activitypub/models/question.ts

@@ -14,10 +14,10 @@ export async function extractPollFromQuestion(source: string | IQuestion): Promi
 		throw 'invalid question';
 	}
 
-	const choices = question[multiple ? 'anyOf' : 'oneOf']
-		.map((x, i) => x.name);
+	const choices = question[multiple ? 'anyOf' : 'oneOf']!
+		.map((x, i) => x.name!);
 
-	const votes = question[multiple ? 'anyOf' : 'oneOf']
+	const votes = question[multiple ? 'anyOf' : 'oneOf']!
 		.map((x, i) => x.replies && x.replies.totalItems || x._misskey_votes || 0);
 
 	return {
@@ -60,7 +60,7 @@ export async function updateQuestion(value: any) {
 
 	for (const choice of poll.choices) {
 		const oldCount = poll.votes[poll.choices.indexOf(choice)];
-		const newCount = apChoices.filter(ap => ap.name === choice)[0].replies.totalItems;
+		const newCount = apChoices!.filter(ap => ap.name === choice)[0].replies!.totalItems;
 
 		if (oldCount != newCount) {
 			changed = true;
@@ -68,10 +68,6 @@ export async function updateQuestion(value: any) {
 		}
 	}
 
-	await Notes.update(note.id, {
-		updatedAt: new Date(),
-	});
-
 	await Polls.update({ noteId: note.id }, {
 		votes: poll.votes
 	});

src/remote/activitypub/models/tag.ts

@@ -14,13 +14,13 @@ export type ITag = {
 	identifier?: IIdentifier;
 };
 
-export function extractHashtags(tags: ITag[]) {
-	if (!tags) return [];
+export function extractHashtags(tags: ITag[] | null | undefined): string[] {
+	if (tags == null) return [];
 
 	const hashtags = tags.filter(tag => tag.type === 'Hashtag' && typeof tag.name == 'string');
 
 	return hashtags.map(tag => {
-		const m = tag.name.match(/^#(.+)/);
+		const m = tag.name ? tag.name.match(/^#(.+)/) : null;
 		return m ? m[1] : null;
-	}).filter(x => x != null);
+	}).filter(x => x != null) as string[];
 }