11.0.0-beta.13

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

.config/example.yml

@@ -65,10 +65,10 @@ db:
 #   ┌─────────────────────┐
 #───┘ Redis configuration └─────────────────────────────────────
 
-#redis:
-#  host: localhost
-#  port: 6379
-#  pass: example-pass
+redis:
+  host: localhost
+  port: 6379
+  #pass: example-pass
 
 #   ┌─────────────────────────────┐
 #───┘ Elasticsearch configuration └─────────────────────────────
@@ -127,28 +127,12 @@ drive:
 # change it according to your preferences.
 
 # Available methods:
-# aid1 ... Use AID for ID generation (with random 1 char)
-# aid2 ... Use AID for ID generation (with random 2 chars)
-# aid3 ... Use AID for ID generation (with random 3 chars)
-# aid4 ... Use AID for ID generation (with random 4 chars)
-# ulid ... Use ulid for ID generation
-# objectid ... This is left for backward compatibility.
+# aid ... Short, Millisecond accuracy
+# meid ... Similar to ObjectID, Millisecond accuracy
+# ulid ... Millisecond accuracy
+# objectid ... This is left for backward compatibility
 
-# AID(n) is the original ID generation method.
-# The trailing n represents the number of random characters that
-# will be suffixed.
-# The larger n is the safer. If n is small, the possibility of
-# collision at the same time increases, but there are also
-# advantages such as shortening of the URL.
-
-# ULID: Universally Unique Lexicographically Sortable Identifier.
-# for more details: https://github.com/ulid/spec
-# * Normally, AID should be sufficient.
-
-# ObjectID is the method used in previous versions of Misskey.
-# * Choose this if you are migrating from a previous Misskey.
-
-id: 'aid2'
+id: 'aid'
 
 #   ┌─────────────────────┐
 #───┘ Other configuration └─────────────────────────────────────

CHANGELOG.md

@@ -7,11 +7,37 @@ If you encounter any problems with updating, please try the following:
 
 11.0.0
 ----------
-* データベースがMongoDBからPostgreSQLに変更されました
+* **データベースがMongoDBからPostgreSQLに変更されました**
+* **Redisが必須に**
+* アカウントを完全に削除できるように
+* ミュート/ブロック時にそのユーザーの投稿のウォッチをすべて解除するように
+* フォロー申請数が実際より1すくなくなる問題を修正
+* リストからアカウント削除したユーザーを削除できない問題を修正
+* リストTLでフォローしていないユーザーの非公開投稿が流れる問題を修正
+* リストTLでダイレクト投稿が流れない問題を修正
+* ミュートしているユーザーの投稿がタイムラインに流れてくることがある問題を修正
 
 ### APIの破壊的変更
 * v10時点で deprecated だったパラメータなどを削除
 * ユーザーリストの title が name に
+* リバーシの対局の`settings`プロパティがなくなり、その中にあったプロパティがすべて上の階層に
+  * 例えば`game.settings.map`は`game.map`になる
+
+### 既知の問題
+* アプリが作成できない
+  * 依存ライブラリの問題と思わるため、対応が難しい
+
+### Migration
+coming soon...
+
+10.100.0
+----------
+* ユーザーリストでフォローボタンを表示するように
+* ドライブのファイルのサムネイルを修正
+* 投稿ウィジットでローカルのみの公開範囲で投稿できない問題を修正
+* TLを遡った時に抜けがある時がある問題を修正
+* ユーザータイムラインが投稿日時順ではなくなっているのを修正
+* 10.99.0 でチャートのレンダリングがおかしい問題を修正
 
 10.99.0
 ----------

CONTRIBUTING.md

@@ -130,6 +130,40 @@ const users = userIds.length > 0 ? await Users.find({
 }) : [];
 ```
 
+### 配列のインデックス in SQL
+SQLでは配列のインデックスは**1始まり**。
+`[a, b, c]`の `a`にアクセスしたいなら`[0]`ではなく`[1]`と書く
+
 ### `undefined`にご用心
 MongoDBの時とは違い、findOneでレコードを取得する時に対象レコードが存在しない場合 **`undefined`** が返ってくるので注意。
 MongoDBは`null`で返してきてたので、その感覚で`if (x === null)`とか書くとバグる。代わりに`if (x == null)`と書いてください
+
+### 簡素な`undefined`チェック
+データベースからレコードを取得するときに、プログラムの流れ的に(ほぼ)絶対`undefined`にはならない場合でも、`undefined`チェックしないとTypeScriptに怒られます。
+でもいちいち複数行を費やして、発生するはずのない`undefined`をチェックするのも面倒なので、`ensure`というユーティリティ関数を用意しています。
+例えば、
+``` ts
+const user = await Users.findOne(userId);
+// この時点で user の型は User | undefined
+if (user == null) {
+	throw 'missing user';
+}
+// この時点で user の型は User
+```
+という処理を`ensure`を使うと
+``` ts
+const user = await Users.findOne(userId).then(ensure);
+// この時点で user の型は User
+```
+という風に書けます。
+もちろん`ensure`内部でエラーを握りつぶすようなことはしておらず、万が一`undefined`だった場合はPromiseがRejectされ後続の処理は実行されません。
+``` ts
+const user = await Users.findOne(userId).then(ensure);
+// 万が一 Users.findOne の結果が undefined だったら、ensure でエラーが発生するので
+// この行に到達することは無い
+// なので、.then(ensure) は
+// if (user == null) {
+//	throw 'missing user';
+// }
+// の糖衣構文のような扱いです
+```

docs/setup.en.md

@@ -24,18 +24,13 @@ Please install and setup these softwares:
 #### Dependencies :package:
 * **[Node.js](https://nodejs.org/en/)** >= 11.7.0
 * **[PostgreSQL](https://www.postgresql.org/)** >= 10
+* **[Redis](https://redis.io/)**
 
 ##### Optional
-* [Redis](https://redis.io/)
-  * Redis is optional, but we strongly recommended to install it
 * [Elasticsearch](https://www.elastic.co/) - required to enable the search feature
 * [FFmpeg](https://www.ffmpeg.org/)
 
-*3.* Setup PostgreSQL
-----------------------------------------------------------------
-:)
-
-*4.* Install Misskey
+*3.* Install Misskey
 ----------------------------------------------------------------
 1. `su - misskey` Connect to misskey user.
 2. `git clone -b master git://github.com/syuilo/misskey.git` Clone the misskey repo from master branch.
@@ -43,12 +38,12 @@ Please install and setup these softwares:
 4. `git checkout $(git tag -l | grep -Ev -- '-(rc|alpha)\.[0-9]+$' | sort -V | tail -n 1)` Checkout to the [latest release](https://github.com/syuilo/misskey/releases/latest)
 5. `npm install` Install misskey dependencies.
 
-*5.* Configure Misskey
+*4.* Configure Misskey
 ----------------------------------------------------------------
 1. `cp .config/example.yml .config/default.yml` Copy the `.config/example.yml` and rename it to `default.yml`.
 2. Edit `default.yml`
 
-*6.* Build Misskey
+*5.* Build Misskey
 ----------------------------------------------------------------
 
 Build misskey with the following:
@@ -64,13 +59,13 @@ If you're still encountering errors about some modules, use node-gyp:
 3. `node-gyp build`
 4. `NODE_ENV=production npm run build`
 
-*7.* Init DB
+*6.* Init DB
 ----------------------------------------------------------------
 ``` shell
 npm run init
 ```
 
-*8.* That is it.
+*7.* That is it.
 ----------------------------------------------------------------
 Well done! Now, you have an environment that run to Misskey.
 

docs/setup.fr.md

@@ -24,18 +24,13 @@ Installez les paquets suivants :
 #### Dépendences :package:
 * **[Node.js](https://nodejs.org/en/)** >= 11.7.0
 * **[PostgreSQL](https://www.postgresql.org/)** >= 10
+* **[Redis](https://redis.io/)**
 
 ##### Optionnels
-* [Redis](https://redis.io/)
-  * Redis est optionnel mais nous vous recommandons vivement de l'installer
 * [Elasticsearch](https://www.elastic.co/) - requis pour pouvoir activer la fonctionnalité de recherche
 * [FFmpeg](https://www.ffmpeg.org/)
 
-*3.* Paramètrage de PostgreSQL
-----------------------------------------------------------------
-:)
-
-*4.* Installation de Misskey
+*3.* Installation de Misskey
 ----------------------------------------------------------------
 1. `su - misskey` Basculez vers l'utilisateur misskey.
 2. `git clone -b master git://github.com/syuilo/misskey.git` Clonez la branche master du dépôt misskey.
@@ -43,12 +38,12 @@ Installez les paquets suivants :
 4. `git checkout $(git tag -l | grep -Ev -- '-(rc|alpha)\.[0-9]+$' | sort -V | tail -n 1)` Checkout sur le tag de la [version la plus récente](https://github.com/syuilo/misskey/releases/latest)
 5. `npm install` Installez les dépendances de misskey.
 
-*5.* Création du fichier de configuration
+*4.* Création du fichier de configuration
 ----------------------------------------------------------------
 1. `cp .config/example.yml .config/default.yml` Copiez le fichier `.config/example.yml` et renommez-le`default.yml`.
 2. Editez le fichier `default.yml`
 
-*6.* Construction de Misskey
+*5.* Construction de Misskey
 ----------------------------------------------------------------
 
 Construisez Misskey comme ceci :
@@ -64,7 +59,7 @@ Si vous rencontrez des erreurs concernant certains modules, utilisez node-gyp:
 3. `node-gyp build`
 4. `NODE_ENV=production npm run build`
 
-*7.* C'est tout.
+*6.* C'est tout.
 ----------------------------------------------------------------
 Excellent ! Maintenant, vous avez un environnement prêt pour lancer Misskey
 

docs/setup.ja.md

@@ -24,25 +24,14 @@ adduser --disabled-password --disabled-login misskey
 #### 依存関係 :package:
 * **[Node.js](https://nodejs.org/en/)** (11.7.0以上)
 * **[PostgreSQL](https://www.postgresql.org/)** (10以上)
+* **[Redis](https://redis.io/)**
 
 ##### オプション
-* [Redis](https://redis.io/)
-	* Redisはオプションですが、インストールすることを強く推奨します。
-	* インストールしなくていいのは、あなたのインスタンスが自分専用のときだけとお考えください。
-	* 具体的には、Redisをインストールしないと、次の事が出来なくなります:
-		* Misskeyプロセスを複数起動しての負荷分散
-		* レートリミット
-		* ジョブキュー
-		* Twitter連携
 * [Elasticsearch](https://www.elastic.co/)
 	* 検索機能を有効にするためにはインストールが必要です。
 * [FFmpeg](https://www.ffmpeg.org/)
 
-*3.* PostgreSQLの設定
-----------------------------------------------------------------
-:)
-
-*4.* Misskeyのインストール
+*3.* Misskeyのインストール
 ----------------------------------------------------------------
 1. `su - misskey` misskeyユーザーを使用
 2. `git clone -b master git://github.com/syuilo/misskey.git` masterブランチからMisskeyレポジトリをクローン
@@ -50,12 +39,12 @@ adduser --disabled-password --disabled-login misskey
 4. `git checkout $(git tag -l | grep -Ev -- '-(rc|alpha)\.[0-9]+$' | sort -V | tail -n 1)` [最新のリリース](https://github.com/syuilo/misskey/releases/latest)を確認
 5. `npm install` Misskeyの依存パッケージをインストール
 
-*5.* 設定ファイルを作成する
+*4.* 設定ファイルを作成する
 ----------------------------------------------------------------
 1. `cp .config/example.yml .config/default.yml` `.config/example.yml`をコピーし名前を`default.yml`にする。
 2. `default.yml` を編集する。
 
-*6.* Misskeyのビルド
+*5.* Misskeyのビルド
 ----------------------------------------------------------------
 
 次のコマンドでMisskeyをビルドしてください:
@@ -70,13 +59,13 @@ Debianをお使いであれば、`build-essential`パッケージをインスト
 3. `node-gyp build`
 4. `NODE_ENV=production npm run build`
 
-*7.* データベースを初期化
+*6.* データベースを初期化
 ----------------------------------------------------------------
 ``` shell
 npm run init
 ```
 
-*8.* 以上です！
+*7.* 以上です！
 ----------------------------------------------------------------
 お疲れ様でした。これでMisskeyを動かす準備は整いました。
 

gulpfile.ts

@@ -120,7 +120,7 @@ gulp.task('copy:client', () =>
 		])
 			.pipe(isProduction ? (imagemin as any)() : gutil.noop())
 			.pipe(rename(path => {
-				path.dirname = path.dirname.replace('assets', '.');
+				path.dirname = path.dirname!.replace('assets', '.');
 			}))
 			.pipe(gulp.dest('./built/client/assets/'))
 );

package.json

@@ -1,7 +1,7 @@
 {
 	"name": "misskey",
 	"author": "syuilo <i@syuilo.com>",
-	"version": "11.0.0-alpha.6",
+	"version": "11.0.0-beta.13",
 	"codename": "daybreak",
 	"repository": {
 		"type": "git",
@@ -106,7 +106,7 @@
 		"bcryptjs": "2.4.3",
 		"bootstrap-vue": "2.0.0-rc.13",
 		"bull": "3.7.0",
-		"cafy": "15.1.0",
+		"cafy": "15.1.1",
 		"chai": "4.2.0",
 		"chai-http": "4.2.1",
 		"chalk": "2.4.2",
@@ -258,7 +258,6 @@
 		"vuex": "3.1.0",
 		"vuex-persistedstate": "2.5.4",
 		"web-push": "3.3.3",
-		"webfinger.js": "2.7.0",
 		"webpack": "4.28.4",
 		"webpack-cli": "3.2.3",
 		"websocket": "1.0.28",

src/@types/webfinger.js.d.ts

@@ -1,65 +0,0 @@
-declare module 'webfinger.js' {
-	interface IWebFingerConstructorConfig {
-		tls_only?: boolean;
-		webfist_fallback?: boolean;
-		uri_fallback?: boolean;
-		request_timeout?: number;
-	}
-
-	type JRDProperties = { [type: string]: string };
-
-	interface IJRDLink {
-		rel: string;
-		type?: string;
-		href?: string;
-		template?: string;
-		titles?: { [lang: string]: string };
-		properties?: JRDProperties;
-	}
-
-	interface IJRD {
-		subject?: string;
-		expires?: Date;
-		aliases?: string[];
-		properties?: JRDProperties;
-		links?: IJRDLink[];
-	}
-
-	interface IIDXLinks {
-		'avatar': IJRDLink[];
-		'remotestorage': IJRDLink[];
-		'blog': IJRDLink[];
-		'vcard': IJRDLink[];
-		'updates': IJRDLink[];
-		'share': IJRDLink[];
-		'profile': IJRDLink[];
-		'webfist': IJRDLink[];
-		'camlistore': IJRDLink[];
-		[type: string]: IJRDLink[];
-	}
-
-	interface IIDXProperties {
-		'name': string;
-		[type: string]: string;
-	}
-
-	interface IIDX {
-		links: IIDXLinks;
-		properties: IIDXProperties;
-	}
-
-	interface ILookupCallbackResult {
-		object: IJRD;
-		json: string;
-		idx: IIDX;
-	}
-
-	type LookupCallback = (err: Error | string, result?: ILookupCallbackResult) => void;
-
-	export class WebFinger {
-		constructor(config?: IWebFingerConstructorConfig);
-
-		public lookup(address: string, cb: LookupCallback): NodeJS.Timeout;
-		public lookupLink(address: string, rel: string, cb: IJRDLink): void;
-	}
-}

src/boot/master.ts

@@ -44,7 +44,7 @@ function greet() {
 export async function masterMain() {
 	greet();
 
-	let config: Config;
+	let config!: Config;
 
 	try {
 		// initialize app

src/boot/worker.ts

@@ -15,6 +15,6 @@ export async function workerMain() {
 
 	if (cluster.isWorker) {
 		// Send a 'ready' message to parent process
-		process.send('ready');
+		process.send!('ready');
 	}
 }

src/client/app/common/views/deck/deck.user-column.vue

@@ -8,7 +8,7 @@
 		<div class="is-remote" v-if="user.host != null">
 			<details>
 				<summary><fa icon="exclamation-triangle"/> {{ $t('@.is-remote-user') }}</summary>
-				<a :href="user.url || user.uri" target="_blank">{{ $t('@.view-on-remote') }}</a>
+				<a :href="user.url" target="_blank">{{ $t('@.view-on-remote') }}</a>
 			</details>
 		</div>
 		<header :style="bannerStyle">

src/client/app/desktop/views/home/user/index.vue

@@ -4,7 +4,7 @@
 		<fa icon="exclamation-triangle"/> {{ $t('@.user-suspended') }}
 	</div>
 	<div class="is-remote" v-if="user.host != null" :class="{ shadow: $store.state.device.useShadow, round: $store.state.device.roundedCorners }">
-		<fa icon="exclamation-triangle"/> {{ $t('@.is-remote-user') }}<a :href="user.url || user.uri" target="_blank">{{ $t('@.view-on-remote') }}</a>
+		<fa icon="exclamation-triangle"/> {{ $t('@.is-remote-user') }}<a :href="user.url" target="_blank">{{ $t('@.view-on-remote') }}</a>
 	</div>
 	<div class="main">
 		<x-header class="header" :user="user"/>

src/client/app/mobile/views/pages/user/index.vue

@@ -5,7 +5,7 @@
 	</template>
 	<div class="wwtwuxyh" v-if="!fetching">
 		<div class="is-suspended" v-if="user.isSuspended"><p><fa icon="exclamation-triangle"/> {{ $t('@.user-suspended') }}</p></div>
-		<div class="is-remote" v-if="user.host != null"><p><fa icon="exclamation-triangle"/> {{ $t('@.is-remote-user') }}<a :href="user.url || user.uri" target="_blank">{{ $t('@.view-on-remote') }}</a></p></div>
+		<div class="is-remote" v-if="user.host != null"><p><fa icon="exclamation-triangle"/> {{ $t('@.is-remote-user') }}<a :href="user.url" target="_blank">{{ $t('@.view-on-remote') }}</a></p></div>
 		<header>
 			<div class="banner" :style="style"></div>
 			<div class="body">

src/client/themes/light.json5

@@ -7,7 +7,7 @@
 	kind: 'light',
 
 	vars: {
-		primary: '#fb4e4e',
+		primary: '#f18570',
 		secondary: '#fff',
 		text: '#666',
 	},

src/config/load.ts

@@ -25,11 +25,11 @@ export default function load() {
 
 	const mixin = {} as Mixin;
 
-	const url = validateUrl(config.url);
+	const url = tryCreateUrl(config.url);
 
-	config.url = normalizeUrl(config.url);
+	config.url = url.origin;
 
-	config.port = config.port || parseInt(process.env.PORT, 10);
+	config.port = config.port || parseInt(process.env.PORT || '', 10);
 
 	mixin.host = url.host;
 	mixin.hostname = url.hostname;
@@ -53,14 +53,3 @@ function tryCreateUrl(url: string) {
 		throw `url="${url}" is not a valid URL.`;
 	}
 }
-
-function validateUrl(url: string) {
-	const result = tryCreateUrl(url);
-	if (result.pathname.replace('/', '').length) throw `url="${url}" is not a valid URL, has a pathname.`;
-	if (!url.includes(result.host)) throw `url="${url}" is not a valid URL, has an invalid hostname.`;
-	return result;
-}
-
-function normalizeUrl(url: string) {
-	return url.endsWith('/') ? url.substr(0, url.length - 1) : url;
-}

src/daemons/notes-stats-child.ts

@@ -19,7 +19,7 @@ initDb().then(() => {
 			all, local
 		};
 
-		process.send(stats);
+		process.send!(stats);
 	}
 
 	tick();

src/db/postgre.ts

@@ -36,10 +36,10 @@ import { Emoji } from '../models/entities/emoji';
 import { ReversiGame } from '../models/entities/games/reversi/game';
 import { ReversiMatching } from '../models/entities/games/reversi/matching';
 import { UserNotePining } from '../models/entities/user-note-pinings';
-import { UserServiceLinking } from '../models/entities/user-service-linking';
 import { Poll } from '../models/entities/poll';
 import { UserKeypair } from '../models/entities/user-keypair';
 import { UserPublickey } from '../models/entities/user-publickey';
+import { UserProfile } from '../models/entities/user-profile';
 
 const sqlLogger = dbLogger.createSubLogger('sql', 'white', false);
 
@@ -76,7 +76,7 @@ class MyCustomLogger implements Logger {
 }
 
 export function initDb(justBorrow = false, sync = false, log = false) {
-	const enableLogging = log || !['production', 'test'].includes(process.env.NODE_ENV);
+	const enableLogging = log || !['production', 'test'].includes(process.env.NODE_ENV || '');
 
 	try {
 		const conn = getConnection();
@@ -93,7 +93,7 @@ export function initDb(justBorrow = false, sync = false, log = false) {
 		synchronize: process.env.NODE_ENV === 'test' || sync,
 		dropSchema: process.env.NODE_ENV === 'test' && !justBorrow,
 		logging: enableLogging,
-		logger: enableLogging ? new MyCustomLogger() : null,
+		logger: enableLogging ? new MyCustomLogger() : undefined,
 		entities: [
 			Meta,
 			Instance,
@@ -101,12 +101,12 @@ export function initDb(justBorrow = false, sync = false, log = false) {
 			AuthSession,
 			AccessToken,
 			User,
+			UserProfile,
 			UserKeypair,
 			UserPublickey,
 			UserList,
 			UserListJoining,
 			UserNotePining,
-			UserServiceLinking,
 			Following,
 			FollowRequest,
 			Muting,

src/db/redis.ts

@@ -1,7 +1,7 @@
 import * as redis from 'redis';
 import config from '../config';
 
-export default config.redis ? redis.createClient(
+export default redis.createClient(
 	config.redis.port,
 	config.redis.host,
 	{
@@ -9,4 +9,4 @@ export default config.redis ? redis.createClient(
 		prefix: config.redis.prefix,
 		db: config.redis.db || 0
 	}
-) : null;
+);

src/games/reversi/core.ts

@@ -37,7 +37,7 @@ export type Undo = {
 	/**
 	 * ターン
 	 */
-	turn: Color;
+	turn: Color | null;
 };
 
 /**
@@ -47,12 +47,12 @@ export default class Reversi {
 	public map: MapPixel[];
 	public mapWidth: number;
 	public mapHeight: number;
-	public board: Color[];
-	public turn: Color = BLACK;
+	public board: (Color | null | undefined)[];
+	public turn: Color | null = BLACK;
 	public opts: Options;
 
 	public prevPos = -1;
-	public prevColor: Color = null;
+	public prevColor: Color | null = null;
 
 	private logs: Undo[] = [];
 
@@ -145,12 +145,12 @@ export default class Reversi {
 		// ターン計算
 		this.turn =
 			this.canPutSomewhere(!this.prevColor) ? !this.prevColor :
-			this.canPutSomewhere(this.prevColor) ? this.prevColor :
+			this.canPutSomewhere(this.prevColor!) ? this.prevColor :
 			null;
 	}
 
 	public undo() {
-		const undo = this.logs.pop();
+		const undo = this.logs.pop()!;
 		this.prevColor = undo.color;
 		this.prevPos = undo.pos;
 		this.board[undo.pos] = null;
@@ -254,10 +254,10 @@ export default class Reversi {
 	/**
 	 * ゲームの勝者 (null = 引き分け)
 	 */
-	public get winner(): Color {
+	public get winner(): Color | null {
 		return this.isEnded ?
 			this.blackCount == this.whiteCount ? null :
 			this.opts.isLlotheo === this.blackCount > this.whiteCount　? WHITE : BLACK :
-			undefined;
+			undefined as never;
 	}
 }

src/mfm/fromHtml.ts

@@ -3,8 +3,6 @@ import { URL } from 'url';
 import { urlRegex } from './prelude';
 
 export function fromHtml(html: string): string {
-	if (html == null) return null;
-
 	const dom = parseFragment(html) as DefaultTreeDocumentFragment;
 
 	let text = '';

src/mfm/parse.ts

@@ -2,7 +2,7 @@ import { mfmLanguage } from './language';
 import { MfmForest } from './prelude';
 import { normalize } from './normalize';
 
-export function parse(source: string): MfmForest {
+export function parse(source: string | null): MfmForest | null {
 	if (source == null || source == '') {
 		return null;
 	}
@@ -10,7 +10,7 @@ export function parse(source: string): MfmForest {
 	return normalize(mfmLanguage.root.tryParse(source));
 }
 
-export function parsePlain(source: string): MfmForest {
+export function parsePlain(source: string | null): MfmForest | null {
 	if (source == null || source == '') {
 		return null;
 	}

src/mfm/toHtml.ts

@@ -4,7 +4,7 @@ import { intersperse } from '../prelude/array';
 import { MfmForest, MfmTree } from './prelude';
 import { IMentionedRemoteUsers } from '../models/entities/note';
 
-export function toHtml(tokens: MfmForest, mentionedRemoteUsers: IMentionedRemoteUsers = []) {
+export function toHtml(tokens: MfmForest | null, mentionedRemoteUsers: IMentionedRemoteUsers = []) {
 	if (tokens == null) {
 		return null;
 	}

src/migrate.ts

@@ -16,7 +16,6 @@ import { InternalStorage } from './services/drive/internal-storage';
 import { createTemp } from './misc/create-temp';
 import { Note } from './models/entities/note';
 import { Following } from './models/entities/following';
-import { genId } from './misc/gen-id';
 import { Poll } from './models/entities/poll';
 import { PollVote } from './models/entities/poll-vote';
 import { NoteFavorite } from './models/entities/note-favorite';
@@ -25,6 +24,13 @@ import { UserPublickey } from './models/entities/user-publickey';
 import { UserKeypair } from './models/entities/user-keypair';
 import { extractPublic } from './crypto_key';
 import { Emoji } from './models/entities/emoji';
+import { toPuny as _toPuny } from './misc/convert-host';
+import { UserProfile } from './models/entities/user-profile';
+
+function toPuny(x: string | null): string | null {
+	if (x == null) return null;
+	return _toPuny(x);
+}
 
 const u = (config as any).mongodb.user ? encodeURIComponent((config as any).mongodb.user) : null;
 const p = (config as any).mongodb.pass ? encodeURIComponent((config as any).mongodb.pass) : null;
@@ -34,6 +40,9 @@ const uri = `mongodb://${u && p ? `${u}:${p}@` : ''}${(config as any).mongodb.ho
 const db = monk(uri);
 let mdb: mongo.Db;
 
+const test = false;
+const limit = 500;
+
 const nativeDbConn = async (): Promise<mongo.Db> => {
 	if (mdb) return mdb;
 
@@ -69,6 +78,7 @@ const getDriveFileBucket = async (): Promise<mongo.GridFSBucket> => {
 async function main() {
 	await initDb();
 	const Users = getRepository(User);
+	const UserProfiles = getRepository(UserProfile);
 	const DriveFiles = getRepository(DriveFile);
 	const DriveFolders = getRepository(DriveFolder);
 	const Notes = getRepository(Note);
@@ -87,29 +97,32 @@ async function main() {
 			createdAt: user.createdAt || new Date(),
 			username: user.username,
 			usernameLower: user.username.toLowerCase(),
-			host: user.host,
+			host: toPuny(user.host),
 			token: generateUserToken(),
-			password: user.password,
-			isAdmin: user.isAdmin,
-			autoAcceptFollowed: true,
-			autoWatch: false,
+			isAdmin: user.isAdmin || false,
 			name: user.name,
-			location: user.profile ? user.profile.location : null,
-			birthday: user.profile ? user.profile.birthday : null,
-			followersCount: user.followersCount,
-			followingCount: user.followingCount,
-			notesCount: user.notesCount,
-			description: user.description,
-			isBot: user.isBot,
-			isCat: user.isCat,
-			isVerified: user.isVerified,
+			followersCount: user.followersCount || 0,
+			followingCount: user.followingCount || 0,
+			notesCount: user.notesCount || 0,
+			isBot: user.isBot || false,
+			isCat: user.isCat || false,
+			isVerified: user.isVerified || false,
 			inbox: user.inbox,
 			sharedInbox: user.sharedInbox,
 			uri: user.uri,
 		});
+		await UserProfiles.save({
+			userId: user._id.toHexString(),
+			description: user.description,
+			userHost: toPuny(user.host),
+			autoAcceptFollowed: true,
+			autoWatch: false,
+			password: user.password,
+			location: user.profile ? user.profile.location : null,
+			birthday: user.profile ? user.profile.birthday : null,
+		});
 		if (user.publicKey) {
 			await UserPublickeys.save({
-				id: genId(),
 				userId: user._id.toHexString(),
 				keyId: user.publicKey.id,
 				keyPem: user.publicKey.publicKeyPem
@@ -117,7 +130,6 @@ async function main() {
 		}
 		if (user.keypair) {
 			await UserKeypairs.save({
-				id: genId(),
 				userId: user._id.toHexString(),
 				publicKey: extractPublic(user.keypair),
 				privateKey: user.keypair,
@@ -128,15 +140,15 @@ async function main() {
 	async function migrateFollowing(following: any) {
 		await Followings.save({
 			id: following._id.toHexString(),
-			createdAt: following.createdAt || new Date(),
+			createdAt: new Date(),
 			followerId: following.followerId.toHexString(),
 			followeeId: following.followeeId.toHexString(),
 
 			// 非正規化
-			followerHost: following._follower ? following._follower.host : null,
+			followerHost: following._follower ? toPuny(following._follower.host) : null,
 			followerInbox: following._follower ? following._follower.inbox : null,
 			followerSharedInbox: following._follower ? following._follower.sharedInbox : null,
-			followeeHost: following._followee ? following._followee.host : null,
+			followeeHost: following._followee ? toPuny(following._followee.host) : null,
 			followeeInbox: following._followee ? following._followee.inbox : null,
 			followeeSharedInbox: following._followee ? following._followee.sharedInbo : null
 		});
@@ -155,16 +167,17 @@ async function main() {
 		const user = await _User.findOne({
 			_id: file.metadata.userId
 		});
+		if (user == null) return;
 		if (file.metadata.storage && file.metadata.storage.key) { // when object storage
 			await DriveFiles.save({
 				id: file._id.toHexString(),
 				userId: user._id.toHexString(),
-				userHost: user.host,
+				userHost: toPuny(user.host),
 				createdAt: file.uploadDate || new Date(),
 				md5: file.md5,
 				name: file.filename,
 				type: file.contentType,
-				properties: file.metadata.properties,
+				properties: file.metadata.properties || {},
 				size: file.length,
 				url: file.metadata.url,
 				uri: file.metadata.uri,
@@ -191,7 +204,7 @@ async function main() {
 			await DriveFiles.save({
 				id: file._id.toHexString(),
 				userId: user._id.toHexString(),
-				userHost: user.host,
+				userHost: toPuny(user.host),
 				createdAt: file.uploadDate || new Date(),
 				md5: file.md5,
 				name: file.filename,
@@ -210,7 +223,7 @@ async function main() {
 			await DriveFiles.save({
 				id: file._id.toHexString(),
 				userId: user._id.toHexString(),
-				userHost: user.host,
+				userHost: toPuny(user.host),
 				createdAt: file.uploadDate || new Date(),
 				md5: file.md5,
 				name: file.filename,
@@ -250,7 +263,6 @@ async function main() {
 
 		if (note.poll) {
 			await Polls.save({
-				id: genId(),
 				noteId: note._id.toHexString(),
 				choices: note.poll.choices.map((x: any) => x.text),
 				expiresAt: note.poll.expiresAt,
@@ -296,13 +308,13 @@ async function main() {
 		const u = await _User.findOne({
 			_id: new mongo.ObjectId(user.id)
 		});
-		const avatar = await DriveFiles.findOne(u.avatarId.toHexString());
-		const banner = await DriveFiles.findOne(u.bannerId.toHexString());
+		const avatar = u.avatarId ? await DriveFiles.findOne(u.avatarId.toHexString()) : null;
+		const banner = u.bannerId ? await DriveFiles.findOne(u.bannerId.toHexString()) : null;
 		await Users.update(user.id, {
-			avatarId: avatar.id,
-			bannerId: banner.id,
-			avatarUrl: avatar.url,
-			bannerUrl: banner.url
+			avatarId: avatar ? avatar.id : null,
+			bannerId: banner ? banner.id : null,
+			avatarUrl: avatar ? avatar.url : null,
+			bannerUrl: banner ? banner.url : null
 		});
 	}
 
@@ -313,14 +325,19 @@ async function main() {
 			aliases: emoji.aliases,
 			url: emoji.url,
 			uri: emoji.uri,
-			host: emoji.host,
+			host: toPuny(emoji.host),
 			name: emoji.name
 		});
 	}
 
-	const allUsersCount = await _User.count();
+	let allUsersCount = await _User.count({
+		deletedAt: { $exists: false }
+	});
+	if (test && allUsersCount > limit) allUsersCount = limit;
 	for (let i = 0; i < allUsersCount; i++) {
-		const user = await _User.findOne({}, {
+		const user = await _User.findOne({
+			deletedAt: { $exists: false }
+		}, {
 			skip: i
 		});
 		try {
@@ -332,7 +349,8 @@ async function main() {
 		}
 	}
 
-	const allFollowingsCount = await _Following.count();
+	let allFollowingsCount = await _Following.count();
+	if (test && allFollowingsCount > limit) allFollowingsCount = limit;
 	for (let i = 0; i < allFollowingsCount; i++) {
 		const following = await _Following.findOne({}, {
 			skip: i
@@ -346,7 +364,8 @@ async function main() {
 		}
 	}
 
-	const allDriveFoldersCount = await _DriveFolder.count();
+	let allDriveFoldersCount = await _DriveFolder.count();
+	if (test && allDriveFoldersCount > limit) allDriveFoldersCount = limit;
 	for (let i = 0; i < allDriveFoldersCount; i++) {
 		const folder = await _DriveFolder.findOne({}, {
 			skip: i
@@ -360,9 +379,16 @@ async function main() {
 		}
 	}
 
-	const allDriveFilesCount = await _DriveFile.count();
+	let allDriveFilesCount = await _DriveFile.count({
+		'metadata._user.host': null,
+		'metadata.deletedAt': { $exists: false }
+	});
+	if (test && allDriveFilesCount > limit) allDriveFilesCount = limit;
 	for (let i = 0; i < allDriveFilesCount; i++) {
-		const file = await _DriveFile.findOne({}, {
+		const file = await _DriveFile.findOne({
+			'metadata._user.host': null,
+			'metadata.deletedAt': { $exists: false }
+		}, {
 			skip: i
 		});
 		try {
@@ -374,12 +400,15 @@ async function main() {
 		}
 	}
 
-	const allNotesCount = await _Note.count({
-		'_user.host': null
+	let allNotesCount = await _Note.count({
+		'_user.host': null,
+		'metadata.deletedAt': { $exists: false }
 	});
+	if (test && allNotesCount > limit) allNotesCount = limit;
 	for (let i = 0; i < allNotesCount; i++) {
 		const note = await _Note.findOne({
-			'_user.host': null
+			'_user.host': null,
+			'metadata.deletedAt': { $exists: false }
 		}, {
 			skip: i
 		});
@@ -392,7 +421,8 @@ async function main() {
 		}
 	}
 
-	const allPollVotesCount = await _PollVote.count();
+	let allPollVotesCount = await _PollVote.count();
+	if (test && allPollVotesCount > limit) allPollVotesCount = limit;
 	for (let i = 0; i < allPollVotesCount; i++) {
 		const vote = await _PollVote.findOne({}, {
 			skip: i
@@ -406,7 +436,8 @@ async function main() {
 		}
 	}
 
-	const allNoteFavoritesCount = await _Favorite.count();
+	let allNoteFavoritesCount = await _Favorite.count();
+	if (test && allNoteFavoritesCount > limit) allNoteFavoritesCount = limit;
 	for (let i = 0; i < allNoteFavoritesCount; i++) {
 		const favorite = await _Favorite.findOne({}, {
 			skip: i
@@ -420,7 +451,8 @@ async function main() {
 		}
 	}
 
-	const allNoteReactionsCount = await _NoteReaction.count();
+	let allNoteReactionsCount = await _NoteReaction.count();
+	if (test && allNoteReactionsCount > limit) allNoteReactionsCount = limit;
 	for (let i = 0; i < allNoteReactionsCount; i++) {
 		const reaction = await _NoteReaction.findOne({}, {
 			skip: i
@@ -434,7 +466,8 @@ async function main() {
 		}
 	}
 
-	const allActualUsersCount = await Users.count();
+	let allActualUsersCount = await Users.count();
+	if (test && allActualUsersCount > limit) allActualUsersCount = limit;
 	for (let i = 0; i < allActualUsersCount; i++) {
 		const [user] = await Users.find({
 			take: 1,

src/misc/acct/render.ts

@@ -1,5 +1,5 @@
 import Acct from './type';
 
 export default (user: Acct) => {
-	return user.host === null ? user.username : `${user.username}@${user.host}`;
+	return user.host == null ? user.username : `${user.username}@${user.host}`;
 };

src/misc/acct/type.ts

@@ -1,6 +1,6 @@
 type Acct = {
 	username: string;
-	host: string;
+	host: string | null;
 };
 
 export default Acct;

src/misc/aid.ts

@@ -1,26 +0,0 @@
-// AID
-// 長さ8の[2000年1月1日からの経過ミリ秒をbase36でエンコードしたもの] + 長さnの[ランダムな文字列]
-
-const CHARS = '0123456789abcdefghijklmnopqrstuvwxyz';
-const TIME2000 = 946684800000;
-
-function getTime(time: number) {
-	time = time - TIME2000;
-	if (time < 0) time = 0;
-
-	return time.toString(36);
-}
-
-function getRandom(length: number) {
-	let str = '';
-
-	for (let i = 0; i < length; i++) {
-		str += CHARS[Math.floor(Math.random() * CHARS.length)];
-	}
-
-	return str;
-}
-
-export function genAid(date: Date, rand: number): string {
-	return getTime(date.getTime()).padStart(8, CHARS[0]) + getRandom(rand);
-}

src/misc/aidc.ts

@@ -1,26 +0,0 @@
-// AID(Cheep)
-// 長さ6の[2000年1月1日からの経過秒をbase36でエンコードしたもの] + 長さ3の[ランダムな文字列]
-
-const CHARS = '0123456789abcdefghijklmnopqrstuvwxyz';
-const TIME2000 = 946684800000;
-
-function getTime(time: number) {
-	time = time - TIME2000;
-	if (time < 0) time = 0;
-	time = Math.floor(time / 1000);
-	return time.toString(36);
-}
-
-function getRandom() {
-	let str = '';
-
-	for (let i = 0; i < 3; i++) {
-		str += CHARS[Math.floor(Math.random() * CHARS.length)];
-	}
-
-	return str;
-}
-
-export function genAidc(date: Date): string {
-	return getTime(date.getTime()).padStart(6, CHARS[0]) + getRandom();
-}

src/misc/convert-host.ts

@@ -1,27 +1,26 @@
 import config from '../config';
-import { toUnicode, toASCII } from 'punycode';
+import { toASCII } from 'punycode';
 import { URL } from 'url';
 
-export function getFullApAccount(username: string, host: string) {
-	return host ? `${username}@${toApHost(host)}` : `${username}@${toApHost(config.host)}`;
+export function getFullApAccount(username: string, host: string | null) {
+	return host ? `${username}@${toPuny(host)}` : `${username}@${toPuny(config.host)}`;
 }
 
 export function isSelfHost(host: string) {
 	if (host == null) return true;
-	return toApHost(config.host) === toApHost(host);
+	return toPuny(config.host) === toPuny(host);
 }
 
 export function extractDbHost(uri: string) {
 	const url = new URL(uri);
-	return toDbHost(url.hostname);
+	return toPuny(url.hostname);
 }
 
-export function toDbHost(host: string) {
-	if (host == null) return null;
-	return toUnicode(host.toLowerCase());
+export function toPuny(host: string) {
+	return toASCII(host.toLowerCase());
 }
 
-export function toApHost(host: string) {
+export function toPunyNullable(host: string | null | undefined): string | null {
 	if (host == null) return null;
 	return toASCII(host.toLowerCase());
 }

src/misc/detect-mine.ts

@@ -3,7 +3,7 @@ import fileType from 'file-type';
 import checkSvg from '../misc/check-svg';
 
 export async function detectMine(path: string) {
-	return new Promise<[string, string]>((res, rej) => {
+	return new Promise<[string, string | null]>((res, rej) => {
 		const readable = fs.createReadStream(path);
 		readable
 			.on('error', rej)

src/misc/donwload-url.ts

@@ -1,5 +1,4 @@
 import * as fs from 'fs';
-import * as URL from 'url';
 import * as request from 'request';
 import config from '../config';
 import chalk from 'chalk';
@@ -26,7 +25,7 @@ export async function downloadUrl(url: string, path: string) {
 			rej(error);
 		});
 
-		const requestUrl = URL.parse(url).pathname.match(/[^\u0021-\u00ff]/) ? encodeURI(url) : url;
+		const requestUrl = new URL(url).pathname.match(/[^\u0021-\u00ff]/) ? encodeURI(url) : url;
 
 		const req = request({
 			url: requestUrl,

src/misc/fetch-meta.ts

@@ -9,7 +9,6 @@ export default async function(): Promise<Meta> {
 	} else {
 		return Metas.save({
 			id: genId(),
-			hiddenTags: []
 		} as Meta);
 	}
 }

src/misc/fetch-proxy-account.ts

@@ -1,8 +1,9 @@
 import fetchMeta from './fetch-meta';
 import { ILocalUser } from '../models/entities/user';
 import { Users } from '../models';
+import { ensure } from '../prelude/ensure';
 
 export async function fetchProxyAccount(): Promise<ILocalUser> {
 	const meta = await fetchMeta();
-	return await Users.findOne({ username: meta.proxyAccount, host: null }) as ILocalUser;
+	return await Users.findOne({ username: meta.proxyAccount!, host: null }).then(ensure) as ILocalUser;
 }

src/misc/gen-id.ts

@@ -1,7 +1,7 @@
 import { ulid } from 'ulid';
-import { genAid } from './aid';
-import { genAidc } from './aidc';
-import { genObjectId } from './object-id';
+import { genAid } from './id/aid';
+import { genMeid } from './id/meid';
+import { genObjectId } from './id/object-id';
 import config from '../config';
 
 const metohd = config.id.toLowerCase();
@@ -10,11 +10,8 @@ export function genId(date?: Date): string {
 	if (!date || (date > new Date())) date = new Date();
 
 	switch (metohd) {
-		case 'aidc': return genAidc(date);
-		case 'aid1': return genAid(date, 1);
-		case 'aid2': return genAid(date, 2);
-		case 'aid3': return genAid(date, 3);
-		case 'aid4': return genAid(date, 4);
+		case 'aid': return genAid(date);
+		case 'meid': return genMeid(date);
 		case 'ulid': return ulid(date.getTime());
 		case 'objectid': return genObjectId(date);
 		default: throw 'unknown id generation method';

src/misc/get-user-summary.ts

@@ -1,19 +0,0 @@
-import getAcct from './acct/render';
-import getUserName from './get-user-name';
-import { User } from '../models/entities/user';
-import { Users } from '../models';
-
-/**
- * ユーザーを表す文字列を取得します。
- * @param user ユーザー
- */
-export default function(user: User): string {
-	let string = `${getUserName(user)} (@${getAcct(user)})\n` +
-		`${user.notesCount}投稿、${user.followingCount}フォロー、${user.followersCount}フォロワー\n`;
-
-	if (Users.isLocalUser(user)) {
-		string += `場所: ${user.location}、誕生日: ${user.birthday}\n`;
-	}
-
-	return string + `「${user.description}」`;
-}

src/misc/id/aid.ts

@@ -0,0 +1,23 @@
+// AID
+// 長さ8の[2000年1月1日からの経過ミリ秒をbase36でエンコードしたもの] + 長さ2の[ノイズ文字列]
+
+import * as cluster from 'cluster';
+
+const TIME2000 = 946684800000;
+let counter = process.pid + (cluster.isMaster ? 0 : cluster.worker.id);
+
+function getTime(time: number) {
+	time = time - TIME2000;
+	if (time < 0) time = 0;
+
+	return time.toString(36).padStart(8, '0');
+}
+
+function getNoise() {
+	return counter.toString(36).padStart(2, '0').slice(-2);
+}
+
+export function genAid(date: Date): string {
+	counter++;
+	return getTime(date.getTime()) + getNoise();
+}

src/misc/id/meid.ts

@@ -0,0 +1,26 @@
+const CHARS = '0123456789abcdef';
+
+function getTime(time: number) {
+	if (time < 0) time = 0;
+	if (time === 0) {
+		return CHARS[0];
+	}
+
+	time += 0x800000000000;
+
+	return time.toString(16).padStart(12, CHARS[0]);
+}
+
+function getRandom() {
+	let str = '';
+
+	for (let i = 0; i < 12; i++) {
+		str += CHARS[Math.floor(Math.random() * CHARS.length)];
+	}
+
+	return str;
+}
+
+export function genMeid(date: Date): string {
+	return 'f' + getTime(date.getTime()) + getRandom();
+}

src/misc/id/object-id.ts

@@ -8,7 +8,7 @@ function getTime(time: number) {
 
 	time = Math.floor(time / 1000);
 
-	return time.toString(16);
+	return time.toString(16).padStart(8, CHARS[0]);
 }
 
 function getRandom() {

src/misc/identifiable-error.ts

@@ -7,7 +7,7 @@ export class IdentifiableError extends Error {
 
 	constructor(id: string, message?: string) {
 		super(message);
-		this.message = message;
+		this.message = message || '';
 		this.id = id;
 	}
 }

src/misc/nyaize.ts

@@ -3,7 +3,7 @@ export function nyaize(text: string): string {
 		// ja-JP
 		.replace(/な/g, 'にゃ').replace(/ナ/g, 'ニャ').replace(/ﾅ/g, 'ﾆｬ')
 		// ko-KR
-		.replace(/[나-낳]/g, (match: string) => String.fromCharCode(
-			match.codePointAt(0)  + '냐'.charCodeAt(0) - '나'.charCodeAt(0)
+		.replace(/[나-낳]/g, match => String.fromCharCode(
+			match.codePointAt(0)! + '냐'.charCodeAt(0) - '나'.charCodeAt(0)
 		));
 }

src/misc/reaction-lib.ts

@@ -20,7 +20,7 @@ export async function getFallbackReaction(): Promise<string> {
 	return  meta.useStarForReactionFallback ? 'star' : 'like';
 }
 
-export async function toDbReaction(reaction: string, enableEmoji = true): Promise<string> {
+export async function toDbReaction(reaction?: string | null, enableEmoji = true): Promise<string> {
 	if (reaction == null) return await getFallbackReaction();
 
 	// 既存の文字列リアクションはそのまま

src/misc/schema.ts

@@ -19,8 +19,8 @@ type MyType<T extends Schema> = {
 export type SchemaType<p extends Schema> =
 	p['type'] extends 'number' ? number :
 	p['type'] extends 'string' ? string :
-	p['type'] extends 'array' ? MyType<p['items']>[] :
-	p['type'] extends 'object' ? ObjType<p['properties']> :
+	p['type'] extends 'array' ? MyType<NonNullable<p['items']>>[] :
+	p['type'] extends 'object' ? ObjType<NonNullable<p['properties']>> :
 	any;
 
 export function convertOpenApiSchema(schema: Schema) {

src/models/entities/app.ts

@@ -53,7 +53,7 @@ export class App {
 	public permission: string[];
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: 'The callbackUrl of the App.'
 	})
 	public callbackUrl: string | null;

src/models/entities/auth-session.ts

@@ -19,11 +19,15 @@ export class AuthSession {
 	})
 	public token: string;
 
-	@Column(id())
+	@Column({
+		...id(),
+		nullable: true
+	})
 	public userId: User['id'];
 
 	@ManyToOne(type => User, {
-		onDelete: 'CASCADE'
+		onDelete: 'CASCADE',
+		nullable: true
 	})
 	@JoinColumn()
 	public user: User | null;

src/models/entities/emoji.ts

@@ -25,12 +25,12 @@ export class Emoji {
 	public host: string | null;
 
 	@Column('varchar', {
-		length: 256,
+		length: 512,
 	})
 	public url: string;
 
 	@Column('varchar', {
-		length: 256, nullable: true
+		length: 512, nullable: true
 	})
 	public uri: string | null;
 

src/models/entities/follow-request.ts

@@ -53,13 +53,13 @@ export class FollowRequest {
 	public followerHost: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: '[Denormalized]'
 	})
 	public followerInbox: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: '[Denormalized]'
 	})
 	public followerSharedInbox: string | null;
@@ -71,13 +71,13 @@ export class FollowRequest {
 	public followeeHost: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: '[Denormalized]'
 	})
 	public followeeInbox: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: '[Denormalized]'
 	})
 	public followeeSharedInbox: string | null;

src/models/entities/following.ts

@@ -48,13 +48,13 @@ export class Following {
 	public followerHost: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: '[Denormalized]'
 	})
 	public followerInbox: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: '[Denormalized]'
 	})
 	public followerSharedInbox: string | null;
@@ -66,13 +66,13 @@ export class Following {
 	public followeeHost: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: '[Denormalized]'
 	})
 	public followeeInbox: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: '[Denormalized]'
 	})
 	public followeeSharedInbox: string | null;

src/models/entities/meta.ts

@@ -78,27 +78,27 @@ export class Meta {
 	public blockedHosts: string[];
 
 	@Column('varchar', {
-		length: 256,
+		length: 512,
 		nullable: true,
 		default: '/assets/ai.png'
 	})
 	public mascotImageUrl: string | null;
 
 	@Column('varchar', {
-		length: 256,
+		length: 512,
 		nullable: true
 	})
 	public bannerUrl: string | null;
 
 	@Column('varchar', {
-		length: 256,
+		length: 512,
 		nullable: true,
 		default: 'https://ai.misskey.xyz/aiart/yubitun.png'
 	})
 	public errorImageUrl: string | null;
 
 	@Column('varchar', {
-		length: 256,
+		length: 512,
 		nullable: true
 	})
 	public iconUrl: string | null;

src/models/entities/note.ts

@@ -15,13 +15,6 @@ export class Note {
 	})
 	public createdAt: Date;
 
-	@Index()
-	@Column('timestamp with time zone', {
-		nullable: true,
-		comment: 'The updated date of the Note.'
-	})
-	public updatedAt: Date | null;
-
 	@Index()
 	@Column({
 		...id(),
@@ -126,7 +119,7 @@ export class Note {
 
 	@Index({ unique: true })
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: 'The URI of a note. it will be null when the note is local.'
 	})
 	public uri: string | null;
@@ -195,12 +188,6 @@ export class Note {
 	})
 	public userHost: string | null;
 
-	@Column('varchar', {
-		length: 128, nullable: true,
-		comment: '[Denormalized]'
-	})
-	public userInbox: string | null;
-
 	@Column({
 		...id(),
 		nullable: true,
@@ -227,6 +214,14 @@ export class Note {
 	})
 	public renoteUserHost: string | null;
 	//#endregion
+
+	constructor(data: Partial<Note>) {
+		if (data == null) return;
+
+		for (const [k, v] of Object.entries(data)) {
+			(this as any)[k] = v;
+		}
+	}
 }
 
 export type IMentionedRemoteUsers = {

src/models/entities/poll.ts

@@ -6,10 +6,6 @@ import { User } from './user';
 @Entity()
 export class Poll {
 	@PrimaryColumn(id())
-	public id: string;
-
-	@Index({ unique: true })
-	@Column(id())
 	public noteId: Note['id'];
 
 	@OneToOne(type => Note, {
@@ -57,11 +53,19 @@ export class Poll {
 	})
 	public userHost: string | null;
 	//#endregion
+
+	constructor(data: Partial<Poll>) {
+		if (data == null) return;
+
+		for (const [k, v] of Object.entries(data)) {
+			(this as any)[k] = v;
+		}
+	}
 }
 
 export type IPoll = {
 	choices: string[];
 	votes?: number[];
 	multiple: boolean;
-	expiresAt: Date;
+	expiresAt: Date | null;
 };

src/models/entities/sw-subscription.ts

@@ -21,7 +21,7 @@ export class SwSubscription {
 	public user: User | null;
 
 	@Column('varchar', {
-		length: 256,
+		length: 512,
 	})
 	public endpoint: string;
 

src/models/entities/user-keypair.ts

@@ -1,14 +1,10 @@
-import { PrimaryColumn, Entity, Index, JoinColumn, Column, OneToOne } from 'typeorm';
+import { PrimaryColumn, Entity, JoinColumn, Column, OneToOne } from 'typeorm';
 import { User } from './user';
 import { id } from '../id';
 
 @Entity()
 export class UserKeypair {
 	@PrimaryColumn(id())
-	public id: string;
-
-	@Index({ unique: true })
-	@Column(id())
 	public userId: User['id'];
 
 	@OneToOne(type => User, {
@@ -26,4 +22,12 @@ export class UserKeypair {
 		length: 4096,
 	})
 	public privateKey: string;
+
+	constructor(data: Partial<UserKeypair>) {
+		if (data == null) return;
+
+		for (const [k, v] of Object.entries(data)) {
+			(this as any)[k] = v;
+		}
+	}
 }

src/models/entities/user-profile.ts

@@ -0,0 +1,209 @@
+import { Entity, Column, Index, OneToOne, JoinColumn, PrimaryColumn } from 'typeorm';
+import { id } from '../id';
+import { User } from './user';
+
+@Entity()
+export class UserProfile {
+	@PrimaryColumn(id())
+	public userId: User['id'];
+
+	@OneToOne(type => User, {
+		onDelete: 'CASCADE'
+	})
+	@JoinColumn()
+	public user: User | null;
+
+	@Column('varchar', {
+		length: 128, nullable: true,
+		comment: 'The location of the User.'
+	})
+	public location: string | null;
+
+	@Column('char', {
+		length: 10, nullable: true,
+		comment: 'The birthday (YYYY-MM-DD) of the User.'
+	})
+	public birthday: string | null;
+
+	@Column('varchar', {
+		length: 1024, nullable: true,
+		comment: 'The description (bio) of the User.'
+	})
+	public description: string | null;
+
+	@Column('jsonb', {
+		default: [],
+	})
+	public fields: {
+		name: string;
+		value: string;
+	}[];
+
+	@Column('varchar', {
+		length: 512, nullable: true,
+		comment: 'Remote URL of the user.'
+	})
+	public url: string | null;
+
+	@Column('varchar', {
+		length: 128, nullable: true,
+		comment: 'The email address of the User.'
+	})
+	public email: string | null;
+
+	@Column('varchar', {
+		length: 128, nullable: true,
+	})
+	public emailVerifyCode: string | null;
+
+	@Column('boolean', {
+		default: false,
+	})
+	public emailVerified: boolean;
+
+	@Column('varchar', {
+		length: 128, nullable: true,
+	})
+	public twoFactorTempSecret: string | null;
+
+	@Column('varchar', {
+		length: 128, nullable: true,
+	})
+	public twoFactorSecret: string | null;
+
+	@Column('boolean', {
+		default: false,
+	})
+	public twoFactorEnabled: boolean;
+
+	@Column('varchar', {
+		length: 128, nullable: true,
+		comment: 'The password hash of the User. It will be null if the origin of the user is local.'
+	})
+	public password: string | null;
+
+	@Column('jsonb', {
+		default: {},
+		comment: 'The client-specific data of the User.'
+	})
+	public clientData: Record<string, any>;
+
+	@Column('boolean', {
+		default: false,
+	})
+	public autoWatch: boolean;
+
+	@Column('boolean', {
+		default: false,
+	})
+	public autoAcceptFollowed: boolean;
+
+	@Column('boolean', {
+		default: false,
+	})
+	public alwaysMarkNsfw: boolean;
+
+	@Column('boolean', {
+		default: false,
+	})
+	public carefulBot: boolean;
+
+	//#region Linking
+	@Column('boolean', {
+		default: false,
+	})
+	public twitter: boolean;
+
+	@Column('varchar', {
+		length: 64, nullable: true, default: null,
+	})
+	public twitterAccessToken: string | null;
+
+	@Column('varchar', {
+		length: 64, nullable: true, default: null,
+	})
+	public twitterAccessTokenSecret: string | null;
+
+	@Column('varchar', {
+		length: 64, nullable: true, default: null,
+	})
+	public twitterUserId: string | null;
+
+	@Column('varchar', {
+		length: 64, nullable: true, default: null,
+	})
+	public twitterScreenName: string | null;
+
+	@Column('boolean', {
+		default: false,
+	})
+	public github: boolean;
+
+	@Column('varchar', {
+		length: 64, nullable: true, default: null,
+	})
+	public githubAccessToken: string | null;
+
+	@Column('integer', {
+		nullable: true, default: null,
+	})
+	public githubId: number | null;
+
+	@Column('varchar', {
+		length: 64, nullable: true, default: null,
+	})
+	public githubLogin: string | null;
+
+	@Column('boolean', {
+		default: false,
+	})
+	public discord: boolean;
+
+	@Column('varchar', {
+		length: 64, nullable: true, default: null,
+	})
+	public discordAccessToken: string | null;
+
+	@Column('varchar', {
+		length: 64, nullable: true, default: null,
+	})
+	public discordRefreshToken: string | null;
+
+	@Column('integer', {
+		nullable: true, default: null,
+	})
+	public discordExpiresDate: number | null;
+
+	@Column('varchar', {
+		length: 64, nullable: true, default: null,
+	})
+	public discordId: string | null;
+
+	@Column('varchar', {
+		length: 64, nullable: true, default: null,
+	})
+	public discordUsername: string | null;
+
+	@Column('varchar', {
+		length: 64, nullable: true, default: null,
+	})
+	public discordDiscriminator: string | null;
+	//#endregion
+
+	//#region Denormalized fields
+	@Index()
+	@Column('varchar', {
+		length: 128, nullable: true,
+		comment: '[Denormalized]'
+	})
+	public userHost: string | null;
+	//#endregion
+
+	constructor(data: Partial<UserProfile>) {
+		if (data == null) return;
+
+		for (const [k, v] of Object.entries(data)) {
+			(this as any)[k] = v;
+		}
+	}
+}

src/models/entities/user-publickey.ts

@@ -5,10 +5,6 @@ import { id } from '../id';
 @Entity()
 export class UserPublickey {
 	@PrimaryColumn(id())
-	public id: string;
-
-	@Index({ unique: true })
-	@Column(id())
 	public userId: User['id'];
 
 	@OneToOne(type => User, {
@@ -27,4 +23,12 @@ export class UserPublickey {
 		length: 4096,
 	})
 	public keyPem: string;
+
+	constructor(data: Partial<UserPublickey>) {
+		if (data == null) return;
+
+		for (const [k, v] of Object.entries(data)) {
+			(this as any)[k] = v;
+		}
+	}
 }

src/models/entities/user-service-linking.ts

@@ -1,108 +0,0 @@
-import { PrimaryColumn, Entity, Index, JoinColumn, Column, OneToOne } from 'typeorm';
-import { User } from './user';
-import { id } from '../id';
-
-@Entity()
-export class UserServiceLinking {
-	@PrimaryColumn(id())
-	public id: string;
-
-	@Index({ unique: true })
-	@Column(id())
-	public userId: User['id'];
-
-	@OneToOne(type => User, {
-		onDelete: 'CASCADE'
-	})
-	@JoinColumn()
-	public user: User | null;
-
-	@Column('boolean', {
-		default: false,
-	})
-	public twitter: boolean;
-
-	@Column('varchar', {
-		length: 64, nullable: true, default: null,
-	})
-	public twitterAccessToken: string | null;
-
-	@Column('varchar', {
-		length: 64, nullable: true, default: null,
-	})
-	public twitterAccessTokenSecret: string | null;
-
-	@Column('varchar', {
-		length: 64, nullable: true, default: null,
-	})
-	public twitterUserId: string | null;
-
-	@Column('varchar', {
-		length: 64, nullable: true, default: null,
-	})
-	public twitterScreenName: string | null;
-
-	@Column('boolean', {
-		default: false,
-	})
-	public github: boolean;
-
-	@Column('varchar', {
-		length: 64, nullable: true, default: null,
-	})
-	public githubAccessToken: string | null;
-
-	@Column('integer', {
-		nullable: true, default: null,
-	})
-	public githubId: number | null;
-
-	@Column('varchar', {
-		length: 64, nullable: true, default: null,
-	})
-	public githubLogin: string | null;
-
-	@Column('boolean', {
-		default: false,
-	})
-	public discord: boolean;
-
-	@Column('varchar', {
-		length: 64, nullable: true, default: null,
-	})
-	public discordAccessToken: string | null;
-
-	@Column('varchar', {
-		length: 64, nullable: true, default: null,
-	})
-	public discordRefreshToken: string | null;
-
-	@Column('integer', {
-		nullable: true, default: null,
-	})
-	public discordExpiresDate: number | null;
-
-	@Column('varchar', {
-		length: 64, nullable: true, default: null,
-	})
-	public discordId: string | null;
-
-	@Column('varchar', {
-		length: 64, nullable: true, default: null,
-	})
-	public discordUsername: string | null;
-
-	@Column('varchar', {
-		length: 64, nullable: true, default: null,
-	})
-	public discordDiscriminator: string | null;
-
-	//#region Denormalized fields
-	@Index()
-	@Column('varchar', {
-		length: 128, nullable: true,
-		comment: '[Denormalized]'
-	})
-	public userHost: string | null;
-	//#endregion
-}

src/models/entities/user.ts

@@ -45,18 +45,6 @@ export class User {
 	})
 	public name: string | null;
 
-	@Column('varchar', {
-		length: 128, nullable: true,
-		comment: 'The location of the User.'
-	})
-	public location: string | null;
-
-	@Column('char', {
-		length: 10, nullable: true,
-		comment: 'The birthday (YYYY-MM-DD) of the User.'
-	})
-	public birthday: string | null;
-
 	@Column('integer', {
 		default: 0,
 		comment: 'The count of followers.'
@@ -101,12 +89,6 @@ export class User {
 	@JoinColumn()
 	public banner: DriveFile | null;
 
-	@Column('varchar', {
-		length: 1024, nullable: true,
-		comment: 'The description (bio) of the User.'
-	})
-	public description: string | null;
-
 	@Index()
 	@Column('varchar', {
 		length: 128, array: true, default: '{}'
@@ -114,38 +96,12 @@ export class User {
 	public tags: string[];
 
 	@Column('varchar', {
-		length: 128, nullable: true,
-		comment: 'The email address of the User.'
-	})
-	public email: string | null;
-
-	@Column('varchar', {
-		length: 128, nullable: true,
-	})
-	public emailVerifyCode: string | null;
-
-	@Column('boolean', {
-		default: false,
-	})
-	public emailVerified: boolean;
-
-	@Column('varchar', {
-		length: 128, nullable: true,
-	})
-	public twoFactorTempSecret: string | null;
-
-	@Column('varchar', {
-		length: 128, nullable: true,
-	})
-	public twoFactorSecret: string | null;
-
-	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 	})
 	public avatarUrl: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 	})
 	public bannerUrl: string | null;
 
@@ -206,11 +162,6 @@ export class User {
 	})
 	public isVerified: boolean;
 
-	@Column('boolean', {
-		default: false,
-	})
-	public twoFactorEnabled: boolean;
-
 	@Column('varchar', {
 		length: 128, array: true, default: '{}'
 	})
@@ -224,36 +175,30 @@ export class User {
 	public host: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
-		comment: 'The inbox of the User. It will be null if the origin of the user is local.'
+		length: 512, nullable: true,
+		comment: 'The inbox URL of the User. It will be null if the origin of the user is local.'
 	})
 	public inbox: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
-		comment: 'The sharedInbox of the User. It will be null if the origin of the user is local.'
+		length: 512, nullable: true,
+		comment: 'The sharedInbox URL of the User. It will be null if the origin of the user is local.'
 	})
 	public sharedInbox: string | null;
 
 	@Column('varchar', {
-		length: 256, nullable: true,
-		comment: 'The featured of the User. It will be null if the origin of the user is local.'
+		length: 512, nullable: true,
+		comment: 'The featured URL of the User. It will be null if the origin of the user is local.'
 	})
 	public featured: string | null;
 
 	@Index()
 	@Column('varchar', {
-		length: 256, nullable: true,
+		length: 512, nullable: true,
 		comment: 'The URI of the User. It will be null if the origin of the user is local.'
 	})
 	public uri: string | null;
 
-	@Column('varchar', {
-		length: 128, nullable: true,
-		comment: 'The password hash of the User. It will be null if the origin of the user is local.'
-	})
-	public password: string | null;
-
 	@Index({ unique: true })
 	@Column('char', {
 		length: 16, nullable: true, unique: true,
@@ -261,31 +206,13 @@ export class User {
 	})
 	public token: string | null;
 
-	@Column('jsonb', {
-		default: {},
-		comment: 'The client-specific data of the User.'
-	})
-	public clientData: Record<string, any>;
+	constructor(data: Partial<User>) {
+		if (data == null) return;
 
-	@Column('boolean', {
-		default: false,
-	})
-	public autoWatch: boolean;
-
-	@Column('boolean', {
-		default: false,
-	})
-	public autoAcceptFollowed: boolean;
-
-	@Column('boolean', {
-		default: false,
-	})
-	public alwaysMarkNsfw: boolean;
-
-	@Column('boolean', {
-		default: false,
-	})
-	public carefulBot: boolean;
+		for (const [k, v] of Object.entries(data)) {
+			(this as any)[k] = v;
+		}
+	}
 }
 
 export interface ILocalUser extends User {

src/models/index.ts

@@ -25,7 +25,6 @@ import { FollowRequestRepository } from './repositories/follow-request';
 import { MutingRepository } from './repositories/muting';
 import { BlockingRepository } from './repositories/blocking';
 import { NoteReactionRepository } from './repositories/note-reaction';
-import { UserServiceLinking } from './entities/user-service-linking';
 import { NotificationRepository } from './repositories/notification';
 import { NoteFavoriteRepository } from './repositories/note-favorite';
 import { ReversiMatchingRepository } from './repositories/games/reversi/matching';
@@ -35,6 +34,7 @@ import { AppRepository } from './repositories/app';
 import { FollowingRepository } from './repositories/following';
 import { AbuseUserReportRepository } from './repositories/abuse-user-report';
 import { AuthSessionRepository } from './repositories/auth-session';
+import { UserProfile } from './entities/user-profile';
 
 export const Apps = getCustomRepository(AppRepository);
 export const Notes = getCustomRepository(NoteRepository);
@@ -45,12 +45,12 @@ export const NoteUnreads = getRepository(NoteUnread);
 export const Polls = getRepository(Poll);
 export const PollVotes = getRepository(PollVote);
 export const Users = getCustomRepository(UserRepository);
+export const UserProfiles = getRepository(UserProfile);
 export const UserKeypairs = getRepository(UserKeypair);
 export const UserPublickeys = getRepository(UserPublickey);
 export const UserLists = getCustomRepository(UserListRepository);
 export const UserListJoinings = getRepository(UserListJoining);
 export const UserNotePinings = getRepository(UserNotePining);
-export const UserServiceLinkings = getRepository(UserServiceLinking);
 export const Followings = getCustomRepository(FollowingRepository);
 export const FollowRequests = getCustomRepository(FollowRequestRepository);
 export const Instances = getRepository(Instance);

src/models/repositories/abuse-user-report.ts

@@ -2,6 +2,7 @@ import { EntityRepository, Repository } from 'typeorm';
 import { Users } from '..';
 import rap from '@prezzemolo/rap';
 import { AbuseUserReport } from '../entities/abuse-user-report';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(AbuseUserReport)
 export class AbuseUserReportRepository extends Repository<AbuseUserReport> {
@@ -14,7 +15,7 @@ export class AbuseUserReportRepository extends Repository<AbuseUserReport> {
 	public async pack(
 		src: AbuseUserReport['id'] | AbuseUserReport,
 	) {
-		const report = typeof src === 'object' ? src : await this.findOne(src);
+		const report = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return await rap({
 			id: report.id,

src/models/repositories/app.ts

@@ -1,6 +1,7 @@
 import { EntityRepository, Repository } from 'typeorm';
 import { App } from '../entities/app';
 import { AccessTokens } from '..';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(App)
 export class AppRepository extends Repository<App> {
@@ -19,7 +20,7 @@ export class AppRepository extends Repository<App> {
 			includeProfileImageIds: false
 		}, options);
 
-		const app = typeof src === 'object' ? src : await this.findOne(src);
+		const app = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return {
 			id: app.id,

src/models/repositories/auth-session.ts

@@ -2,6 +2,7 @@ import { EntityRepository, Repository } from 'typeorm';
 import { Apps } from '..';
 import rap from '@prezzemolo/rap';
 import { AuthSession } from '../entities/auth-session';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(AuthSession)
 export class AuthSessionRepository extends Repository<AuthSession> {
@@ -9,7 +10,7 @@ export class AuthSessionRepository extends Repository<AuthSession> {
 		src: AuthSession['id'] | AuthSession,
 		me?: any
 	) {
-		const session = typeof src === 'object' ? src : await this.findOne(src);
+		const session = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return await rap({
 			id: session.id,

src/models/repositories/blocking.ts

@@ -2,6 +2,7 @@ import { EntityRepository, Repository } from 'typeorm';
 import { Users } from '..';
 import rap from '@prezzemolo/rap';
 import { Blocking } from '../entities/blocking';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(Blocking)
 export class BlockingRepository extends Repository<Blocking> {
@@ -16,7 +17,7 @@ export class BlockingRepository extends Repository<Blocking> {
 		src: Blocking['id'] | Blocking,
 		me?: any
 	) {
-		const blocking = typeof src === 'object' ? src : await this.findOne(src);
+		const blocking = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return await rap({
 			id: blocking.id,

src/models/repositories/drive-file.ts

@@ -3,6 +3,8 @@ import { DriveFile } from '../entities/drive-file';
 import { Users, DriveFolders } from '..';
 import rap from '@prezzemolo/rap';
 import { User } from '../entities/user';
+import { toPuny } from '../../misc/convert-host';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(DriveFile)
 export class DriveFileRepository extends Repository<DriveFile> {
@@ -39,7 +41,7 @@ export class DriveFileRepository extends Repository<DriveFile> {
 	public async clacDriveUsageOfHost(host: string): Promise<number> {
 		const { sum } = await this
 			.createQueryBuilder('file')
-			.where('file.userHost = :host', { host: host })
+			.where('file.userHost = :host', { host: toPuny(host) })
 			.select('SUM(file.size)', 'sum')
 			.getRawOne();
 
@@ -90,7 +92,7 @@ export class DriveFileRepository extends Repository<DriveFile> {
 			self: false
 		}, options);
 
-		const file = typeof src === 'object' ? src : await this.findOne(src);
+		const file = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return await rap({
 			id: file.id,
@@ -107,7 +109,7 @@ export class DriveFileRepository extends Repository<DriveFile> {
 			folder: opts.detail && file.folderId ? DriveFolders.pack(file.folderId, {
 				detail: true
 			}) : null,
-			user: opts.withUser ? Users.pack(file.userId) : null
+			user: opts.withUser ? Users.pack(file.userId!) : null
 		});
 	}
 }

src/models/repositories/drive-folder.ts

@@ -2,6 +2,7 @@ import { EntityRepository, Repository } from 'typeorm';
 import { DriveFolders, DriveFiles } from '..';
 import rap from '@prezzemolo/rap';
 import { DriveFolder } from '../entities/drive-folder';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(DriveFolder)
 export class DriveFolderRepository extends Repository<DriveFolder> {
@@ -22,7 +23,7 @@ export class DriveFolderRepository extends Repository<DriveFolder> {
 			detail: false
 		}, options);
 
-		const folder = typeof src === 'object' ? src : await this.findOne(src);
+		const folder = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return await rap({
 			id: folder.id,

src/models/repositories/follow-request.ts

@@ -1,6 +1,7 @@
 import { EntityRepository, Repository } from 'typeorm';
 import { FollowRequest } from '../entities/follow-request';
 import { Users } from '..';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(FollowRequest)
 export class FollowRequestRepository extends Repository<FollowRequest> {
@@ -8,7 +9,7 @@ export class FollowRequestRepository extends Repository<FollowRequest> {
 		src: FollowRequest['id'] | FollowRequest,
 		me?: any
 	) {
-		const request = typeof src === 'object' ? src : await this.findOne(src);
+		const request = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return {
 			id: request.id,

src/models/repositories/following.ts

@@ -2,9 +2,50 @@ import { EntityRepository, Repository } from 'typeorm';
 import { Users } from '..';
 import rap from '@prezzemolo/rap';
 import { Following } from '../entities/following';
+import { ensure } from '../../prelude/ensure';
+
+type LocalFollowerFollowing = Following & {
+	followerHost: null;
+	followerInbox: null;
+	followerSharedInbox: null;
+};
+
+type RemoteFollowerFollowing = Following & {
+	followerHost: string;
+	followerInbox: string;
+	followerSharedInbox: string;
+};
+
+type LocalFolloweeFollowing = Following & {
+	followeeHost: null;
+	followeeInbox: null;
+	followeeSharedInbox: null;
+};
+
+type RemoteFolloweeFollowing = Following & {
+	followeeHost: string;
+	followeeInbox: string;
+	followeeSharedInbox: string;
+};
 
 @EntityRepository(Following)
 export class FollowingRepository extends Repository<Following> {
+	public isLocalFollower(following: Following): following is LocalFollowerFollowing {
+		return following.followerHost == null;
+	}
+
+	public isRemoteFollower(following: Following): following is RemoteFollowerFollowing {
+		return following.followerHost != null;
+	}
+
+	public isLocalFollowee(following: Following): following is LocalFolloweeFollowing {
+		return following.followeeHost == null;
+	}
+
+	public isRemoteFollowee(following: Following): following is RemoteFolloweeFollowing {
+		return following.followeeHost != null;
+	}
+
 	public packMany(
 		followings: any[],
 		me?: any,
@@ -24,7 +65,7 @@ export class FollowingRepository extends Repository<Following> {
 			populateFollower?: boolean;
 		}
 	) {
-		const following = typeof src === 'object' ? src : await this.findOne(src);
+		const following = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		if (opts == null) opts = {};
 

src/models/repositories/games/reversi/game.ts

@@ -1,6 +1,7 @@
 import { EntityRepository, Repository } from 'typeorm';
 import { Users } from '../../..';
 import { ReversiGame } from '../../../entities/games/reversi/game';
+import { ensure } from '../../../../prelude/ensure';
 
 @EntityRepository(ReversiGame)
 export class ReversiGameRepository extends Repository<ReversiGame> {
@@ -15,7 +16,7 @@ export class ReversiGameRepository extends Repository<ReversiGame> {
 			detail: true
 		}, options);
 
-		const game = typeof src === 'object' ? src : await this.findOne(src);
+		const game = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 		const meId = me ? typeof me === 'string' ? me : me.id : null;
 
 		return {

src/models/repositories/games/reversi/matching.ts

@@ -2,6 +2,7 @@ import { EntityRepository, Repository } from 'typeorm';
 import rap from '@prezzemolo/rap';
 import { ReversiMatching } from '../../../entities/games/reversi/matching';
 import { Users } from '../../..';
+import { ensure } from '../../../../prelude/ensure';
 
 @EntityRepository(ReversiMatching)
 export class ReversiMatchingRepository extends Repository<ReversiMatching> {
@@ -9,7 +10,7 @@ export class ReversiMatchingRepository extends Repository<ReversiMatching> {
 		src: ReversiMatching['id'] | ReversiMatching,
 		me: any
 	) {
-		const matching = typeof src === 'object' ? src : await this.findOne(src);
+		const matching = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return await rap({
 			id: matching.id,

src/models/repositories/messaging-message.ts

@@ -1,6 +1,7 @@
 import { EntityRepository, Repository } from 'typeorm';
 import { MessagingMessage } from '../entities/messaging-message';
 import { Users, DriveFiles } from '..';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(MessagingMessage)
 export class MessagingMessageRepository extends Repository<MessagingMessage> {
@@ -19,7 +20,7 @@ export class MessagingMessageRepository extends Repository<MessagingMessage> {
 			populateRecipient: true
 		};
 
-		const message = typeof src === 'object' ? src : await this.findOne(src);
+		const message = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return {
 			id: message.id,

src/models/repositories/muting.ts

@@ -2,6 +2,7 @@ import { EntityRepository, Repository } from 'typeorm';
 import { Users } from '..';
 import rap from '@prezzemolo/rap';
 import { Muting } from '../entities/muting';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(Muting)
 export class MutingRepository extends Repository<Muting> {
@@ -16,7 +17,7 @@ export class MutingRepository extends Repository<Muting> {
 		src: Muting['id'] | Muting,
 		me?: any
 	) {
-		const muting = typeof src === 'object' ? src : await this.findOne(src);
+		const muting = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return await rap({
 			id: muting.id,

src/models/repositories/note-favorite.ts

@@ -1,6 +1,7 @@
 import { EntityRepository, Repository } from 'typeorm';
 import { NoteFavorite } from '../entities/note-favorite';
 import { Notes } from '..';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(NoteFavorite)
 export class NoteFavoriteRepository extends Repository<NoteFavorite> {
@@ -15,7 +16,7 @@ export class NoteFavoriteRepository extends Repository<NoteFavorite> {
 		src: NoteFavorite['id'] | NoteFavorite,
 		me?: any
 	) {
-		const favorite = typeof src === 'object' ? src : await this.findOne(src);
+		const favorite = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return {
 			id: favorite.id,

src/models/repositories/note-reaction.ts

@@ -1,6 +1,7 @@
 import { EntityRepository, Repository } from 'typeorm';
 import { NoteReaction } from '../entities/note-reaction';
 import { Users } from '..';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(NoteReaction)
 export class NoteReactionRepository extends Repository<NoteReaction> {
@@ -8,7 +9,7 @@ export class NoteReactionRepository extends Repository<NoteReaction> {
 		src: NoteReaction['id'] | NoteReaction,
 		me?: any
 	) {
-		const reaction = typeof src === 'object' ? src : await this.findOne(src);
+		const reaction = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return {
 			id: reaction.id,

src/models/repositories/note.ts

@@ -5,6 +5,7 @@ import { unique, concat } from '../../prelude/array';
 import { nyaize } from '../../misc/nyaize';
 import { Emojis, Users, Apps, PollVotes, DriveFiles, NoteReactions, Followings, Polls } from '..';
 import rap from '@prezzemolo/rap';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(Note)
 export class NoteRepository extends Repository<Note> {
@@ -12,7 +13,7 @@ export class NoteRepository extends Repository<Note> {
 		return x.trim().length <= 100;
 	}
 
-	private async hideNote(packedNote: any, meId: User['id']) {
+	private async hideNote(packedNote: any, meId: User['id'] | null) {
 		let hide = false;
 
 		// visibility が specified かつ自分が指定されていなかったら非表示
@@ -75,7 +76,7 @@ export class NoteRepository extends Repository<Note> {
 
 	public packMany(
 		notes: (Note['id'] | Note)[],
-		me?: User['id'] | User,
+		me?: User['id'] | User | null | undefined,
 		options?: {
 			detail?: boolean;
 			skipHide?: boolean;
@@ -86,7 +87,7 @@ export class NoteRepository extends Repository<Note> {
 
 	public async pack(
 		src: Note['id'] | Note,
-		me?: User['id'] | User,
+		me?: User['id'] | User | null | undefined,
 		options?: {
 			detail?: boolean;
 			skipHide?: boolean;
@@ -98,11 +99,11 @@ export class NoteRepository extends Repository<Note> {
 		}, options);
 
 		const meId = me ? typeof me === 'string' ? me : me.id : null;
-		const note = typeof src === 'object' ? src : await this.findOne(src);
+		const note = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 		const host = note.userHost;
 
 		async function populatePoll() {
-			const poll = await Polls.findOne({ noteId: note.id });
+			const poll = await Polls.findOne({ noteId: note.id }).then(ensure);
 			const choices = poll.choices.map(c => ({
 				text: c,
 				votes: poll.votes[poll.choices.indexOf(c)],
@@ -111,7 +112,7 @@ export class NoteRepository extends Repository<Note> {
 
 			if (poll.multiple) {
 				const votes = await PollVotes.find({
-					userId: meId,
+					userId: meId!,
 					noteId: note.id
 				});
 
@@ -121,7 +122,7 @@ export class NoteRepository extends Repository<Note> {
 				}
 			} else {
 				const vote = await PollVotes.findOne({
-					userId: meId,
+					userId: meId!,
 					noteId: note.id
 				});
 
@@ -139,7 +140,7 @@ export class NoteRepository extends Repository<Note> {
 
 		async function populateMyReaction() {
 			const reaction = await NoteReactions.findOne({
-				userId: meId,
+				userId: meId!,
 				noteId: note.id,
 			});
 
@@ -147,7 +148,7 @@ export class NoteRepository extends Repository<Note> {
 				return reaction.reaction;
 			}
 
-			return null;
+			return undefined;
 		}
 
 		let text = note.text;
@@ -161,15 +162,15 @@ export class NoteRepository extends Repository<Note> {
 		const packed = await rap({
 			id: note.id,
 			createdAt: note.createdAt,
-			app: note.appId ? Apps.pack(note.appId) : null,
+			app: note.appId ? Apps.pack(note.appId) : undefined,
 			userId: note.userId,
 			user: Users.pack(note.user || note.userId, meId),
 			text: text,
 			cw: note.cw,
 			visibility: note.visibility,
-			localOnly: note.localOnly,
-			visibleUserIds: note.visibleUserIds,
-			viaMobile: note.viaMobile,
+			localOnly: note.localOnly || undefined,
+			visibleUserIds: note.visibility === 'specified' ? note.visibleUserIds : undefined,
+			viaMobile: note.viaMobile || undefined,
 			renoteCount: note.renoteCount,
 			repliesCount: note.repliesCount,
 			reactions: note.reactions,
@@ -182,17 +183,18 @@ export class NoteRepository extends Repository<Note> {
 			files: DriveFiles.packMany(note.fileIds),
 			replyId: note.replyId,
 			renoteId: note.renoteId,
+			uri: note.uri,
 
 			...(opts.detail ? {
 				reply: note.replyId ? this.pack(note.replyId, meId, {
 					detail: false
-				}) : null,
+				}) : undefined,
 
 				renote: note.renoteId ? this.pack(note.renoteId, meId, {
 					detail: true
-				}) : null,
+				}) : undefined,
 
-				poll: note.hasPoll ? populatePoll() : null,
+				poll: note.hasPoll ? populatePoll() : undefined,
 
 				...(meId ? {
 					myReaction: populateMyReaction()

src/models/repositories/notification.ts

@@ -2,6 +2,7 @@ import { EntityRepository, Repository } from 'typeorm';
 import { Users, Notes } from '..';
 import rap from '@prezzemolo/rap';
 import { Notification } from '../entities/notification';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(Notification)
 export class NotificationRepository extends Repository<Notification> {
@@ -14,7 +15,7 @@ export class NotificationRepository extends Repository<Notification> {
 	public async pack(
 		src: Notification['id'] | Notification,
 	) {
-		const notification = typeof src === 'object' ? src : await this.findOne(src);
+		const notification = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 
 		return await rap({
 			id: notification.id,
@@ -23,23 +24,23 @@ export class NotificationRepository extends Repository<Notification> {
 			userId: notification.notifierId,
 			user: Users.pack(notification.notifier || notification.notifierId),
 			...(notification.type === 'mention' ? {
-				note: Notes.pack(notification.note || notification.noteId),
+				note: Notes.pack(notification.note || notification.noteId!),
 			} : {}),
 			...(notification.type === 'reply' ? {
-				note: Notes.pack(notification.note || notification.noteId),
+				note: Notes.pack(notification.note || notification.noteId!),
 			} : {}),
 			...(notification.type === 'renote' ? {
-				note: Notes.pack(notification.note || notification.noteId),
+				note: Notes.pack(notification.note || notification.noteId!),
 			} : {}),
 			...(notification.type === 'quote' ? {
-				note: Notes.pack(notification.note || notification.noteId),
+				note: Notes.pack(notification.note || notification.noteId!),
 			} : {}),
 			...(notification.type === 'reaction' ? {
-				note: Notes.pack(notification.note || notification.noteId),
+				note: Notes.pack(notification.note || notification.noteId!),
 				reaction: notification.reaction
 			} : {}),
 			...(notification.type === 'pollVote' ? {
-				note: Notes.pack(notification.note || notification.noteId),
+				note: Notes.pack(notification.note || notification.noteId!),
 				choice: notification.choice
 			} : {})
 		});

src/models/repositories/user-list.ts

@@ -1,16 +1,23 @@
 import { EntityRepository, Repository } from 'typeorm';
 import { UserList } from '../entities/user-list';
+import { ensure } from '../../prelude/ensure';
+import { UserListJoinings } from '..';
 
 @EntityRepository(UserList)
 export class UserListRepository extends Repository<UserList> {
 	public async pack(
-		src: any,
+		src: UserList['id'] | UserList,
 	) {
-		const userList = typeof src === 'object' ? src : await this.findOne(src);
+		const userList = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
+
+		const users = await UserListJoinings.find({
+			userListId: userList.id
+		});
 
 		return {
 			id: userList.id,
-			name: userList.name
+			name: userList.name,
+			userIds: users.map(x => x.userId)
 		};
 	}
 }

src/models/repositories/user.ts

@@ -1,7 +1,8 @@
 import { EntityRepository, Repository, In } from 'typeorm';
 import { User, ILocalUser, IRemoteUser } from '../entities/user';
-import { Emojis, Notes, NoteUnreads, FollowRequests, Notifications, MessagingMessages, UserNotePinings, Followings, Blockings, Mutings } from '..';
+import { Emojis, Notes, NoteUnreads, FollowRequests, Notifications, MessagingMessages, UserNotePinings, Followings, Blockings, Mutings, UserProfiles } from '..';
 import rap from '@prezzemolo/rap';
+import { ensure } from '../../prelude/ensure';
 
 @EntityRepository(User)
 export class UserRepository extends Repository<User> {
@@ -51,7 +52,7 @@ export class UserRepository extends Repository<User> {
 
 	public packMany(
 		users: (User['id'] | User)[],
-		me?: User['id'] | User,
+		me?: User['id'] | User | null | undefined,
 		options?: {
 			detail?: boolean,
 			includeSecrets?: boolean,
@@ -63,7 +64,7 @@ export class UserRepository extends Repository<User> {
 
 	public async pack(
 		src: User['id'] | User,
-		me?: User['id'] | User,
+		me?: User['id'] | User | null | undefined,
 		options?: {
 			detail?: boolean,
 			includeSecrets?: boolean,
@@ -75,25 +76,24 @@ export class UserRepository extends Repository<User> {
 			includeSecrets: false
 		}, options);
 
-		const user = typeof src === 'object' ? src : await this.findOne(src);
+		const user = typeof src === 'object' ? src : await this.findOne(src).then(ensure);
 		const meId = me ? typeof me === 'string' ? me : me.id : null;
 
 		const relation = meId && (meId !== user.id) && opts.detail ? await this.getRelation(meId, user.id) : null;
 		const pins = opts.detail ? await UserNotePinings.find({ userId: user.id }) : [];
+		const profile = opts.detail ? await UserProfiles.findOne({ userId: user.id }).then(ensure) : null;
 
 		return await rap({
 			id: user.id,
 			name: user.name,
 			username: user.username,
 			host: user.host,
-			uri: user.uri,
 			avatarUrl: user.avatarUrl,
-			bannerUrl: user.bannerUrl,
 			avatarColor: user.avatarColor,
-			bannerColor: user.bannerColor,
-			isAdmin: user.isAdmin,
-			isBot: user.isBot,
-			isCat: user.isCat,
+			isAdmin: user.isAdmin || undefined,
+			isBot: user.isBot || undefined,
+			isCat: user.isCat || undefined,
+			isVerified: user.isVerified || undefined,
 
 			// カスタム絵文字添付
 			emojis: user.emojis.length > 0 ? Emojis.find({
@@ -116,9 +116,14 @@ export class UserRepository extends Repository<User> {
 			} : {}),
 
 			...(opts.detail ? {
-				description: user.description,
-				location: user.location,
-				birthday: user.birthday,
+				url: profile!.url,
+				createdAt: user.createdAt,
+				updatedAt: user.updatedAt,
+				bannerUrl: user.bannerUrl,
+				bannerColor: user.bannerColor,
+				description: profile!.description,
+				location: profile!.location,
+				birthday: profile!.birthday,
 				followersCount: user.followersCount,
 				followingCount: user.followingCount,
 				notesCount: user.notesCount,
@@ -131,9 +136,9 @@ export class UserRepository extends Repository<User> {
 			...(opts.detail && meId === user.id ? {
 				avatarId: user.avatarId,
 				bannerId: user.bannerId,
-				autoWatch: user.autoWatch,
-				alwaysMarkNsfw: user.alwaysMarkNsfw,
-				carefulBot: user.carefulBot,
+				autoWatch: profile!.autoWatch,
+				alwaysMarkNsfw: profile!.alwaysMarkNsfw,
+				carefulBot: profile!.carefulBot,
 				hasUnreadMessagingMessage: MessagingMessages.count({
 					where: {
 						recipientId: user.id,
@@ -153,6 +158,12 @@ export class UserRepository extends Repository<User> {
 				}),
 			} : {}),
 
+			...(opts.includeSecrets ? {
+				clientData: profile!.clientData,
+				email: profile!.email,
+				emailVerified: profile!.emailVerified,
+			} : {}),
+
 			...(relation ? {
 				isFollowing: relation.isFollowing,
 				isFollowed: relation.isFollowed,
@@ -166,7 +177,7 @@ export class UserRepository extends Repository<User> {
 	}
 
 	public isLocalUser(user: User): user is ILocalUser {
-		return user.host === null;
+		return user.host == null;
 	}
 
 	public isRemoteUser(user: User): user is IRemoteUser {

src/prelude/ensure.ts

@@ -0,0 +1,10 @@
+/**
+ * 値が null または undefined の場合はエラーを発生させ、そうでない場合は値をそのまま返します
+ */
+export function ensure<T>(x: T): NonNullable<T> {
+	if (x == null) {
+		throw 'ぬるぽ';
+	} else {
+		return x!;
+	}
+}

src/queue/index.ts

@@ -12,7 +12,7 @@ import { queueLogger } from './logger';
 import { DriveFile } from '../models/entities/drive-file';
 
 function initializeQueue(name: string) {
-	return new Queue(name, config.redis != null ? {
+	return new Queue(name, {
 		redis: {
 			port: config.redis.port,
 			host: config.redis.host,
@@ -20,7 +20,15 @@ function initializeQueue(name: string) {
 			db: config.redis.db || 0,
 		},
 		prefix: config.redis.prefix ? `${config.redis.prefix}:queue` : 'queue'
-	} : null);
+	});
+}
+
+function renderError(e: Error): any {
+	return {
+		stack: e.stack,
+		message: e.message,
+		name: e.name
+	};
 }
 
 export const deliverQueue = initializeQueue('deliver');
@@ -34,16 +42,16 @@ deliverQueue
 	.on('waiting', (jobId) => deliverLogger.debug(`waiting id=${jobId}`))
 	.on('active', (job) => deliverLogger.debug(`active id=${job.id} to=${job.data.to}`))
 	.on('completed', (job, result) => deliverLogger.debug(`completed(${result}) id=${job.id} to=${job.data.to}`))
-	.on('failed', (job, err) => deliverLogger.warn(`failed(${err}) id=${job.id} to=${job.data.to}`))
-	.on('error', (error) => deliverLogger.error(`error ${error}`))
+	.on('failed', (job, err) => deliverLogger.warn(`failed(${err}) id=${job.id} to=${job.data.to}`, renderError(err)))
+	.on('error', (error) => deliverLogger.error(`error ${error}`, renderError(error)))
 	.on('stalled', (job) => deliverLogger.warn(`stalled id=${job.id} to=${job.data.to}`));
 
 inboxQueue
 	.on('waiting', (jobId) => inboxLogger.debug(`waiting id=${jobId}`))
 	.on('active', (job) => inboxLogger.debug(`active id=${job.id}`))
 	.on('completed', (job, result) => inboxLogger.debug(`completed(${result}) id=${job.id}`))
-	.on('failed', (job, err) => inboxLogger.warn(`failed(${err}) id=${job.id} activity=${job.data.activity ? job.data.activity.id : 'none'}`))
-	.on('error', (error) => inboxLogger.error(`error ${error}`))
+	.on('failed', (job, err) => inboxLogger.warn(`failed(${err}) id=${job.id} activity=${job.data.activity ? job.data.activity.id : 'none'}`, renderError(err)))
+	.on('error', (error) => inboxLogger.error(`error ${error}`, renderError(error)))
 	.on('stalled', (job) => inboxLogger.warn(`stalled id=${job.id} activity=${job.data.activity ? job.data.activity.id : 'none'}`));
 
 export function deliver(user: ILocalUser, content: any, to: any) {

src/queue/processors/db/delete-drive-files.ts

@@ -10,9 +10,11 @@ const logger = queueLogger.createSubLogger('delete-drive-files');
 export async function deleteDriveFiles(job: Bull.Job, done: any): Promise<void> {
 	logger.info(`Deleting drive files of ${job.data.user.id} ...`);
 
-	const user = await Users.findOne({
-		id: job.data.user.id
-	});
+	const user = await Users.findOne(job.data.user.id);
+	if (user == null) {
+		done();
+		return;
+	}
 
 	let deletedCount = 0;
 	let ended = false;

src/queue/processors/db/export-blocking.ts

@@ -14,9 +14,11 @@ const logger = queueLogger.createSubLogger('export-blocking');
 export async function exportBlocking(job: Bull.Job, done: any): Promise<void> {
 	logger.info(`Exporting blocking of ${job.data.user.id} ...`);
 
-	const user = await Users.findOne({
-		id: job.data.user.id
-	});
+	const user = await Users.findOne(job.data.user.id);
+	if (user == null) {
+		done();
+		return;
+	}
 
 	// Create temp file
 	const [path, cleanup] = await new Promise<[string, any]>((res, rej) => {
@@ -56,6 +58,10 @@ export async function exportBlocking(job: Bull.Job, done: any): Promise<void> {
 
 		for (const block of blockings) {
 			const u = await Users.findOne({ id: block.blockeeId });
+			if (u == null) {
+				exportedCount++; continue;
+			}
+
 			const content = getFullApAccount(u.username, u.host);
 			await new Promise((res, rej) => {
 				stream.write(content + '\n', err => {

src/queue/processors/db/export-following.ts

@@ -14,9 +14,11 @@ const logger = queueLogger.createSubLogger('export-following');
 export async function exportFollowing(job: Bull.Job, done: any): Promise<void> {
 	logger.info(`Exporting following of ${job.data.user.id} ...`);
 
-	const user = await Users.findOne({
-		id: job.data.user.id
-	});
+	const user = await Users.findOne(job.data.user.id);
+	if (user == null) {
+		done();
+		return;
+	}
 
 	// Create temp file
 	const [path, cleanup] = await new Promise<[string, any]>((res, rej) => {
@@ -56,6 +58,10 @@ export async function exportFollowing(job: Bull.Job, done: any): Promise<void> {
 
 		for (const following of followings) {
 			const u = await Users.findOne({ id: following.followeeId });
+			if (u == null) {
+				exportedCount++; continue;
+			}
+
 			const content = getFullApAccount(u.username, u.host);
 			await new Promise((res, rej) => {
 				stream.write(content + '\n', err => {

src/queue/processors/db/export-mute.ts

@@ -14,9 +14,11 @@ const logger = queueLogger.createSubLogger('export-mute');
 export async function exportMute(job: Bull.Job, done: any): Promise<void> {
 	logger.info(`Exporting mute of ${job.data.user.id} ...`);
 
-	const user = await Users.findOne({
-		id: job.data.user.id
-	});
+	const user = await Users.findOne(job.data.user.id);
+	if (user == null) {
+		done();
+		return;
+	}
 
 	// Create temp file
 	const [path, cleanup] = await new Promise<[string, any]>((res, rej) => {
@@ -56,6 +58,10 @@ export async function exportMute(job: Bull.Job, done: any): Promise<void> {
 
 		for (const mute of mutes) {
 			const u = await Users.findOne({ id: mute.muteeId });
+			if (u == null) {
+				exportedCount++; continue;
+			}
+
 			const content = getFullApAccount(u.username, u.host);
 			await new Promise((res, rej) => {
 				stream.write(content + '\n', err => {

src/queue/processors/db/export-notes.ts

@@ -9,15 +9,18 @@ import { Users, Notes, Polls } from '../../../models';
 import { MoreThan } from 'typeorm';
 import { Note } from '../../../models/entities/note';
 import { Poll } from '../../../models/entities/poll';
+import { ensure } from '../../../prelude/ensure';
 
 const logger = queueLogger.createSubLogger('export-notes');
 
 export async function exportNotes(job: Bull.Job, done: any): Promise<void> {
 	logger.info(`Exporting notes of ${job.data.user.id} ...`);
 
-	const user = await Users.findOne({
-		id: job.data.user.id
-	});
+	const user = await Users.findOne(job.data.user.id);
+	if (user == null) {
+		done();
+		return;
+	}
 
 	// Create temp file
 	const [path, cleanup] = await new Promise<[string, any]>((res, rej) => {
@@ -67,9 +70,9 @@ export async function exportNotes(job: Bull.Job, done: any): Promise<void> {
 		cursor = notes[notes.length - 1].id;
 
 		for (const note of notes) {
-			let poll: Poll;
+			let poll: Poll | undefined;
 			if (note.hasPoll) {
-				poll = await Polls.findOne({ noteId: note.id });
+				poll = await Polls.findOne({ noteId: note.id }).then(ensure);
 			}
 			const content = JSON.stringify(serialize(note, poll));
 			await new Promise((res, rej) => {
@@ -114,7 +117,7 @@ export async function exportNotes(job: Bull.Job, done: any): Promise<void> {
 	done();
 }
 
-function serialize(note: Note, poll: Poll): any {
+function serialize(note: Note, poll: Poll | null = null): any {
 	return {
 		id: note.id,
 		text: note.text,

src/queue/processors/db/export-user-lists.ts

@@ -14,9 +14,11 @@ const logger = queueLogger.createSubLogger('export-user-lists');
 export async function exportUserLists(job: Bull.Job, done: any): Promise<void> {
 	logger.info(`Exporting user lists of ${job.data.user.id} ...`);
 
-	const user = await Users.findOne({
-		id: job.data.user.id
-	});
+	const user = await Users.findOne(job.data.user.id);
+	if (user == null) {
+		done();
+		return;
+	}
 
 	const lists = await UserLists.find({
 		userId: user.id

src/queue/processors/db/import-following.ts

@@ -5,7 +5,7 @@ import follow from '../../../services/following/create';
 import parseAcct from '../../../misc/acct/parse';
 import { resolveUser } from '../../../remote/resolve-user';
 import { downloadTextFile } from '../../../misc/download-text-file';
-import { isSelfHost, toDbHost } from '../../../misc/convert-host';
+import { isSelfHost, toPuny } from '../../../misc/convert-host';
 import { Users, DriveFiles } from '../../../models';
 
 const logger = queueLogger.createSubLogger('import-following');
@@ -13,13 +13,19 @@ const logger = queueLogger.createSubLogger('import-following');
 export async function importFollowing(job: Bull.Job, done: any): Promise<void> {
 	logger.info(`Importing following of ${job.data.user.id} ...`);
 
-	const user = await Users.findOne({
-		id: job.data.user.id
-	});
+	const user = await Users.findOne(job.data.user.id);
+	if (user == null) {
+		done();
+		return;
+	}
 
 	const file = await DriveFiles.findOne({
 		id: job.data.fileId
 	});
+	if (file == null) {
+		done();
+		return;
+	}
 
 	const csv = await downloadTextFile(file.url);
 
@@ -31,11 +37,11 @@ export async function importFollowing(job: Bull.Job, done: any): Promise<void> {
 		try {
 			const { username, host } = parseAcct(line.trim());
 
-			let target = isSelfHost(host) ? await Users.findOne({
+			let target = isSelfHost(host!) ? await Users.findOne({
 				host: null,
 				usernameLower: username.toLowerCase()
 			}) : await Users.findOne({
-				host: toDbHost(host),
+				host: toPuny(host!),
 				usernameLower: username.toLowerCase()
 			});
 

src/queue/processors/db/import-user-lists.ts

@@ -5,7 +5,7 @@ import parseAcct from '../../../misc/acct/parse';
 import { resolveUser } from '../../../remote/resolve-user';
 import { pushUserToUserList } from '../../../services/user-list/push';
 import { downloadTextFile } from '../../../misc/download-text-file';
-import { isSelfHost, toDbHost } from '../../../misc/convert-host';
+import { isSelfHost, toPuny } from '../../../misc/convert-host';
 import { DriveFiles, Users, UserLists, UserListJoinings } from '../../../models';
 import { genId } from '../../../misc/gen-id';
 
@@ -14,13 +14,19 @@ const logger = queueLogger.createSubLogger('import-user-lists');
 export async function importUserLists(job: Bull.Job, done: any): Promise<void> {
 	logger.info(`Importing user lists of ${job.data.user.id} ...`);
 
-	const user = await Users.findOne({
-		id: job.data.user.id
-	});
+	const user = await Users.findOne(job.data.user.id);
+	if (user == null) {
+		done();
+		return;
+	}
 
 	const file = await DriveFiles.findOne({
 		id: job.data.fileId
 	});
+	if (file == null) {
+		done();
+		return;
+	}
 
 	const csv = await downloadTextFile(file.url);
 
@@ -43,22 +49,20 @@ export async function importUserLists(job: Bull.Job, done: any): Promise<void> {
 			});
 		}
 
-		let target = isSelfHost(host) ? await Users.findOne({
+		let target = isSelfHost(host!) ? await Users.findOne({
 			host: null,
 			usernameLower: username.toLowerCase()
 		}) : await Users.findOne({
-			host: toDbHost(host),
+			host: toPuny(host!),
 			usernameLower: username.toLowerCase()
 		});
 
-		if (host == null && target == null) continue;
-
-		if (await UserListJoinings.findOne({ userListId: list.id, userId: target.id }) != null) continue;
-
 		if (target == null) {
 			target = await resolveUser(username, host);
 		}
 
+		if (await UserListJoinings.findOne({ userListId: list.id, userId: target.id }) != null) continue;
+
 		pushUserToUserList(target, list);
 	}
 

src/queue/processors/deliver.ts

@@ -7,7 +7,7 @@ import { instanceChart } from '../../services/chart';
 
 const logger = new Logger('deliver');
 
-let latest: string = null;
+let latest: string | null = null;
 
 export default async (job: Bull.Job) => {
 	const { host } = new URL(job.data.to);

src/queue/processors/inbox.ts

@@ -4,7 +4,6 @@ import parseAcct from '../../misc/acct/parse';
 import { IRemoteUser } from '../../models/entities/user';
 import perform from '../../remote/activitypub/perform';
 import { resolvePerson, updatePerson } from '../../remote/activitypub/models/person';
-import { toUnicode } from 'punycode';
 import { URL } from 'url';
 import { publishApLogStream } from '../../services/stream';
 import Logger from '../../services/logger';
@@ -13,6 +12,9 @@ import { Instances, Users, UserPublickeys } from '../../models';
 import { instanceChart } from '../../services/chart';
 import { UserPublickey } from '../../models/entities/user-publickey';
 import fetchMeta from '../../misc/fetch-meta';
+import { toPuny, toPunyNullable } from '../../misc/convert-host';
+import { validActor } from '../../remote/activitypub/type';
+import { ensure } from '../../prelude/ensure';
 
 const logger = new Logger('inbox');
 
@@ -33,7 +35,10 @@ export default async (job: Bull.Job): Promise<void> => {
 	let key: UserPublickey;
 
 	if (keyIdLower.startsWith('acct:')) {
-		const { username, host } = parseAcct(keyIdLower.slice('acct:'.length));
+		const acct = parseAcct(keyIdLower.slice('acct:'.length));
+		const host = toPunyNullable(acct.host);
+		const username = toPuny(acct.username);
+
 		if (host === null) {
 			logger.warn(`request was made by local user: @${username}`);
 			return;
@@ -50,19 +55,20 @@ export default async (job: Bull.Job): Promise<void> => {
 		// ブロックしてたら中断
 		// TODO: いちいちデータベースにアクセスするのはコスト高そうなのでどっかにキャッシュしておく
 		const meta = await fetchMeta();
-		if (meta.blockedHosts.includes(host.toLowerCase())) {
+		if (meta.blockedHosts.includes(host)) {
 			logger.info(`Blocked request: ${host}`);
 			return;
 		}
 
-		user = await Users.findOne({ usernameLower: username, host: host.toLowerCase() }) as IRemoteUser;
+		user = await Users.findOne({
+			usernameLower: username.toLowerCase(),
+			host: host
+		}) as IRemoteUser;
 
-		key = await UserPublickeys.findOne({
-			userId: user.id
-		});
+		key = await UserPublickeys.findOne(user.id).then(ensure);
 	} else {
 		// アクティビティ内のホストの検証
-		const host = toUnicode(new URL(signature.keyId).hostname.toLowerCase());
+		const host = toPuny(new URL(signature.keyId).hostname);
 		try {
 			ValidateActivity(activity, host);
 		} catch (e) {
@@ -73,21 +79,21 @@ export default async (job: Bull.Job): Promise<void> => {
 		// ブロックしてたら中断
 		// TODO: いちいちデータベースにアクセスするのはコスト高そうなのでどっかにキャッシュしておく
 		const meta = await fetchMeta();
-		if (meta.blockedHosts.includes(host.toLowerCase())) {
+		if (meta.blockedHosts.includes(host)) {
 			logger.info(`Blocked request: ${host}`);
 			return;
 		}
 
 		key = await UserPublickeys.findOne({
 			keyId: signature.keyId
-		});
+		}).then(ensure);
 
 		user = await Users.findOne(key.userId) as IRemoteUser;
 	}
 
 	// Update Person activityの場合は、ここで署名検証/更新処理まで実施して終了
 	if (activity.type === 'Update') {
-		if (activity.object && activity.object.type === 'Person') {
+		if (activity.object && validActor.includes(activity.object.type)) {
 			if (user == null) {
 				logger.warn('Update activity received, but user not registed.');
 			} else if (!httpSignature.verifySignature(signature, key.keyPem)) {
@@ -145,7 +151,7 @@ export default async (job: Bull.Job): Promise<void> => {
 function ValidateActivity(activity: any, host: string) {
 	// id (if exists)
 	if (typeof activity.id === 'string') {
-		const uriHost = toUnicode(new URL(activity.id).hostname.toLowerCase());
+		const uriHost = toPuny(new URL(activity.id).hostname);
 		if (host !== uriHost) {
 			const diag = activity.signature ? '. Has LD-Signature. Forwarded?' : '';
 			throw new Error(`activity.id(${activity.id}) has different host(${host})${diag}`);
@@ -154,7 +160,7 @@ function ValidateActivity(activity: any, host: string) {
 
 	// actor (if exists)
 	if (typeof activity.actor === 'string') {
-		const uriHost = toUnicode(new URL(activity.actor).hostname.toLowerCase());
+		const uriHost = toPuny(new URL(activity.actor).hostname);
 		if (host !== uriHost) throw new Error('activity.actor has different host');
 	}
 
@@ -162,13 +168,13 @@ function ValidateActivity(activity: any, host: string) {
 	if (activity.type === 'Create' && activity.object) {
 		// object.id (if exists)
 		if (typeof activity.object.id === 'string') {
-			const uriHost = toUnicode(new URL(activity.object.id).hostname.toLowerCase());
+			const uriHost = toPuny(new URL(activity.object.id).hostname);
 			if (host !== uriHost) throw new Error('activity.object.id has different host');
 		}
 
 		// object.attributedTo (if exists)
 		if (typeof activity.object.attributedTo === 'string') {
-			const uriHost = toUnicode(new URL(activity.object.attributedTo).hostname.toLowerCase());
+			const uriHost = toPuny(new URL(activity.object.attributedTo).hostname);
 			if (host !== uriHost) throw new Error('activity.object.attributedTo has different host');
 		}
 	}

src/remote/activitypub/kernel/accept/follow.ts

@@ -6,9 +6,10 @@ import { Users } from '../../../../models';
 
 export default async (actor: IRemoteUser, activity: IFollow): Promise<void> => {
 	const id = typeof activity.actor == 'string' ? activity.actor : activity.actor.id;
+	if (id == null) throw 'missing id';
 
 	if (!id.startsWith(config.url + '/')) {
-		return null;
+		return;
 	}
 
 	const follower = await Users.findOne({

src/remote/activitypub/kernel/add/index.ts

@@ -14,6 +14,7 @@ export default async (actor: IRemoteUser, activity: IAdd): Promise<void> => {
 
 	if (activity.target === actor.featured) {
 		const note = await resolveNote(activity.object);
+		if (note == null) throw new Error('note not found');
 		await addPinned(actor, note.id);
 		return;
 	}

src/remote/activitypub/kernel/announce/note.ts

@@ -53,16 +53,16 @@ export default async function(resolver: Resolver, actor: IRemoteUser, activity:
 	logger.info(`Creating the (Re)Note: ${uri}`);
 
 	//#region Visibility
-	const visibility = getVisibility(activity.to, activity.cc, actor);
+	const visibility = getVisibility(activity.to || [], activity.cc || [], actor);
 
 	let visibleUsers: User[] = [];
 	if (visibility == 'specified') {
-		visibleUsers = await Promise.all(note.to.map(uri => resolvePerson(uri)));
+		visibleUsers = await Promise.all((note.to || []).map(uri => resolvePerson(uri)));
 	}
 	//#endergion
 
 	await post(actor, {
-		createdAt: new Date(activity.published),
+		createdAt: activity.published ? new Date(activity.published) : null,
 		renote,
 		visibility,
 		visibleUsers,
@@ -75,9 +75,6 @@ type visibility = 'public' | 'home' | 'followers' | 'specified';
 function getVisibility(to: string[], cc: string[], actor: IRemoteUser): visibility {
 	const PUBLIC = 'https://www.w3.org/ns/activitystreams#Public';
 
-	to = to || [];
-	cc = cc || [];
-
 	if (to.includes(PUBLIC)) {
 		return 'public';
 	} else if (cc.includes(PUBLIC)) {

src/remote/activitypub/kernel/block/index.ts

@@ -9,13 +9,14 @@ const logger = apLogger;
 
 export default async (actor: IRemoteUser, activity: IBlock): Promise<void> => {
 	const id = typeof activity.object == 'string' ? activity.object : activity.object.id;
+	if (id == null) throw 'missing id';
 
 	const uri = activity.id || activity;
 
 	logger.info(`Block: ${uri}`);
 
 	if (!id.startsWith(config.url + '/')) {
-		return null;
+		return;
 	}
 
 	const blockee = await Users.findOne(id.split('/').pop());

src/remote/activitypub/kernel/follow.ts

@@ -6,9 +6,10 @@ import { Users } from '../../../models';
 
 export default async (actor: IRemoteUser, activity: IFollow): Promise<void> => {
 	const id = typeof activity.object == 'string' ? activity.object : activity.object.id;
+	if (id == null) throw 'missing id';
 
 	if (!id.startsWith(config.url + '/')) {
-		return null;
+		return;
 	}
 
 	const followee = await Users.findOne(id.split('/').pop());

src/remote/activitypub/kernel/index.ts

@@ -71,7 +71,7 @@ const self = async (actor: IRemoteUser, activity: Object): Promise<void> => {
 
 	default:
 		apLogger.warn(`unknown activity type: ${(activity as any).type}`);
-		return null;
+		return;
 	}
 };
 

src/remote/activitypub/kernel/like.ts

@@ -5,6 +5,7 @@ import { Notes } from '../../../models';
 
 export default async (actor: IRemoteUser, activity: ILike) => {
 	const id = typeof activity.object == 'string' ? activity.object : activity.object.id;
+	if (id == null) throw 'missing id';
 
 	// Transform:
 	// https://misskey.ex/notes/xxxx to

src/remote/activitypub/kernel/reject/follow.ts

@@ -6,9 +6,10 @@ import { Users } from '../../../../models';
 
 export default async (actor: IRemoteUser, activity: IFollow): Promise<void> => {
 	const id = typeof activity.actor == 'string' ? activity.actor : activity.actor.id;
+	if (id == null) throw 'missing id';
 
 	if (!id.startsWith(config.url + '/')) {
-		return null;
+		return;
 	}
 
 	const follower = await Users.findOne(id.split('/').pop());

src/remote/activitypub/kernel/remove/index.ts

@@ -14,6 +14,7 @@ export default async (actor: IRemoteUser, activity: IRemove): Promise<void> => {
 
 	if (activity.target === actor.featured) {
 		const note = await resolveNote(activity.object);
+		if (note == null) throw new Error('note not found');
 		await removePinned(actor, note.id);
 		return;
 	}

src/remote/activitypub/kernel/undo/block.ts

@@ -9,13 +9,14 @@ const logger = apLogger;
 
 export default async (actor: IRemoteUser, activity: IBlock): Promise<void> => {
 	const id = typeof activity.object == 'string' ? activity.object : activity.object.id;
+	if (id == null) throw 'missing id';
 
 	const uri = activity.id || activity;
 
 	logger.info(`UnBlock: ${uri}`);
 
 	if (!id.startsWith(config.url + '/')) {
-		return null;
+		return;
 	}
 
 	const blockee = await Users.findOne(id.split('/').pop());

src/remote/activitypub/kernel/undo/follow.ts

@@ -7,9 +7,10 @@ import { Users, FollowRequests, Followings } from '../../../../models';
 
 export default async (actor: IRemoteUser, activity: IFollow): Promise<void> => {
 	const id = typeof activity.object == 'string' ? activity.object : activity.object.id;
+	if (id == null) throw 'missing id';
 
 	if (!id.startsWith(config.url + '/')) {
-		return null;
+		return;
 	}
 
 	const followee = await Users.findOne(id.split('/').pop());

src/remote/activitypub/kernel/undo/index.ts

@@ -39,6 +39,4 @@ export default async (actor: IRemoteUser, activity: IUndo): Promise<void> => {
 			undoLike(actor, object as ILike);
 			break;
 	}
-
-	return null;
 };

src/remote/activitypub/kernel/undo/like.ts

@@ -8,6 +8,7 @@ import { Notes } from '../../../../models';
  */
 export default async (actor: IRemoteUser, activity: ILike): Promise<void> => {
 	const id = typeof activity.object == 'string' ? activity.object : activity.object.id;
+	if (id == null) throw 'missing id';
 
 	const noteId = id.split('/').pop();