2026-04-12 12:07:06 +02:00
2026-04-12 12:07:06 +02:00
2026-04-12 11:53:04 +02:00

Dendrite to Synapse Migration

Migrate a Matrix homeserver from Dendrite to Synapse, preserving users, rooms, messages, media, E2EE keys, and all associated metadata.

Tested against a real production Dendrite instance (bocken.org): 1194 users, 492 rooms, 51,474 events, 2,747 media files, 1,480 pending to-device messages — full migration completes in ~8 seconds. Users, rooms, messages, media, devices, sessions (access tokens), cross-signing, key backup, and the to-device inbox are all ported. Encrypted history clients can decrypt locally continues to decrypt; undelivered Megolm key shares are flushed to Synapse's device_inbox on next sync.

What gets migrated

Phase Data Dendrite source Synapse target
1 Users, profiles, devices, sessions userapi_accounts, userapi_profiles, userapi_devices users, profiles, devices, access_tokens
2 Rooms and aliases roomserver_rooms, roomserver_room_aliases rooms, room_aliases
3 Events (messages, state) roomserver_events, roomserver_event_json events, event_json, state_events, event_edges, event_auth
4 Room state snapshots syncapi_current_room_state current_state_events, state_groups, state_groups_state
5 Memberships syncapi_current_room_state (member events) room_memberships, local_current_membership
6 Media files + thumbnails mediaapi_media_repository, mediaapi_thumbnail local_media_repository, local_media_repository_thumbnails
7 Receipts, redactions, stats syncapi_receipts, roomserver_redactions receipts_linearized, redactions, room_stats_*
8 E2EE keys, to-device inbox, device-list stream userapi_key_backup_*, keyserver_*, syncapi_send_to_device e2e_room_keys*, e2e_device_keys_json, e2e_fallback_keys_json, e2e_cross_signing_*, device_inbox, device_lists_stream

Prerequisites

  • PostgreSQL for both databases
  • Python 3.10+ with psycopg2
  • Dendrite should be stopped (DB not actively written to)
  • Synapse must be initialized first (schema created)
pip install psycopg2-binary

Step-by-step guide

This documents the full process used to migrate bocken.org from Dendrite to Synapse.

1. Dump the Dendrite database

On the server running Dendrite, dump the PostgreSQL database:

# Stop Dendrite first to ensure a clean dump
sudo systemctl stop dendrite

# Dump in custom format (compressed, restorable)
pg_dump -Fc -f dendrite_dump.custom dendrite

Transfer the dump to your local machine:

scp server:dendrite_dump.custom .

2. Set up the local porting environment

Restore the Dendrite dump into a local PostgreSQL instance:

# Create a fresh database for the Dendrite data
createdb dendrite_from_dump

# Restore the dump
pg_restore -d dendrite_from_dump dendrite_dump.custom

Create an empty Synapse database and initialize the schema:

# Create the Synapse database
createdb synapse_from_dendrite

# Clone Synapse and set up a minimal config
git clone https://github.com/element-hq/synapse.git
cd synapse
pip install -e ".[postgres]"

# Generate a minimal homeserver.yaml
python -m synapse.app.homeserver \
    --server-name your-domain.com \
    --config-path homeserver.yaml \
    --generate-config \
    --report-stats no

# Edit homeserver.yaml — set the database to PostgreSQL:
#   database:
#     name: psycopg2
#     args:
#       database: synapse_from_dendrite

# Start Synapse once so it creates the schema, then stop it
python -m synapse.app.homeserver --config-path homeserver.yaml
# Wait for "Synapse now listening on..." then Ctrl+C

3. Run the migration

# Full migration (all 8 phases)
python3 migrate.py \
    --dendrite-db "dbname=dendrite_from_dump" \
    --synapse-db "dbname=synapse_from_dendrite" \
    --server-name your-domain.com

# With media file copying (if you have the media directory locally)
python3 migrate.py \
    --dendrite-db "dbname=dendrite_from_dump" \
    --synapse-db "dbname=synapse_from_dendrite" \
    --server-name your-domain.com \
    --dendrite-media-path ./media \
    --synapse-media-path ./media_store

Useful flags

# Dry run — inspect without committing any changes
python3 migrate.py ... --dry-run -v

# Run specific phases (e.g., re-run only media)
python3 migrate.py ... --phase 6

# Copy media files only (no database writes, no --synapse-db needed)
python3 migrate.py --media-only \
    --dendrite-db "dbname=dendrite_from_dump" \
    --server-name your-domain.com \
    --dendrite-media-path ./media \
    --synapse-media-path ./media_store

4. Verify locally

Start Synapse against the migrated database and check:

python -m synapse.app.homeserver --config-path homeserver.yaml

# In another terminal — verify via admin API
# (create an admin token first or use an existing one)
curl -s http://localhost:8008/_synapse/admin/v2/rooms | python3 -m json.tool | head
curl -s http://localhost:8008/_synapse/admin/v2/users | python3 -m json.tool | head

5. Deploy to the server

Dump the migrated Synapse database and transfer it:

# Dump the migrated Synapse DB
pg_dump -Fc -f synapse_from_dendrite.sql.custom synapse_from_dendrite

# Transfer to server
scp synapse_from_dendrite.sql.custom server:

On the server:

# Stop Dendrite
sudo systemctl stop dendrite

# Create the Synapse database and restore
sudo -u postgres createdb -O synapse synapse_bocken
pg_restore -d synapse_bocken synapse_from_dendrite.sql.custom

# If you ran --media-only locally or need to copy media on the server:
# Option A: copy the media_store directory you built locally
# Option B: run --media-only on the server pointing at Dendrite's media path
python3 migrate.py --media-only \
    --dendrite-db "dbname=dendrite" \
    --server-name your-domain.com \
    --dendrite-media-path /var/lib/dendrite/media \
    --synapse-media-path /var/lib/synapse/media_store

# Start Synapse
sudo systemctl start synapse

6. Post-migration: OIDC user linking

If you're switching to OIDC authentication and existing users already have accounts, Synapse will try to create new accounts (e.g., alexander1 instead of alexander). Link existing accounts to their OIDC identities directly in the database:

-- Check if a stale mapping was created from a failed login attempt
SELECT * FROM user_external_ids
WHERE auth_provider = 'oidc'
  AND external_id = '<your-oidc-sub-claim>';

-- If it points to the wrong user (e.g., alexander1), update it
UPDATE user_external_ids
SET user_id = '@alexander:your-domain.com'
WHERE auth_provider = 'oidc'
  AND external_id = '<your-oidc-sub-claim>';

-- Clean up the accidentally created user if needed
DELETE FROM user_external_ids WHERE user_id = '@alexander1:your-domain.com';

The sub claim is the unique identifier from your OIDC provider (not the username) — typically a UUID or hash. Find it in your provider's admin panel or decode an ID token:

echo '<id_token>' | cut -d. -f2 | base64 -d 2>/dev/null | jq .sub

Architecture notes

Key differences between Dendrite and Synapse

  • Dendrite uses numeric IDs (NIDs) for rooms, events, event types, state keys. Event JSON stored separately, types resolved via lookup tables.
  • Synapse uses text IDs directly. State managed via delta-chained state groups. Media uses a different filesystem layout.

Schema mapping

See TODO.md for the complete mapping table, findings log, and per-phase validation results.

Media file layout conversion

Dendrite:  {base}/{hash[0]}/{hash[1]}/{hash[2:]}/file
Synapse:   {base}/local_content/{id[0:2]}/{id[2:4]}/{id[4:]}

Dendrite:  {base}/{hash[0]}/{hash[1]}/{hash[2:]}/thumbnail-{W}x{H}-{method}
Synapse:   {base}/local_thumbnails/{id[0:2]}/{id[2:4]}/{id[4:]}/{W}-{H}-{type}-{subtype}-{method}

E2EE: what gets preserved, what depends on the client

The homeserver never sees Megolm message keys in the clear — those live in each client's key store. For a migration to preserve decryptable history end-to-end, three independent pieces all have to line up:

  1. Client device identity stays the same. The client's device_id, access_token, curve25519 identity key, and ed25519 signing key must all match what the client has cached locally. The migration copies all four: userapi_devicesdevices + access_tokens (so the client is not forced to re-login), and keyserver_device_keyse2e_device_keys_json (so other users and the client itself see the same identity keys). Without access_tokens, clients get 401s on their bearer token, log in again, and generate a fresh device — at which point the local Megolm store on many clients becomes unusable.

  2. Undelivered room-key shares survive. When a user shares a Megolm session, the key is wrapped in an m.room.encrypted Olm message and sent as a to-device message. If the recipient was offline, the message sits in Dendrite's syncapi_send_to_device until they next sync. The migration moves this queue verbatim into Synapse's device_inbox. Skipping this step is what caused "keys partially lost" in the first test run — offline recipients never got the room keys, and those rooms remained undecryptable until someone re-shared.

  3. Clients re-verify device lists on first sync. Synapse tells clients which users' device lists have changed via /sync's device_lists.changed. The migration seeds device_lists_stream with one entry per local device so every client treats every local device as "changed" once after migration, re-fetches e2e_device_keys_json, and avoids a stale-cache "device key mismatch → refuse to decrypt" failure.

What cannot be migrated server-side:

  • Client-side-only Megolm sessions. If a client chose not to enable server-side key backup, the Megolm session keys for messages it received live only in that client's local store. The migration preserves device identity so the client keeps and uses those local keys, but there is no server-side copy to restore from.
  • Olm session state (the double-ratchet chains between device pairs) is client-side only. Clients will transparently start new Olm sessions using the migrated e2e_one_time_keys_json / e2e_fallback_keys_json on the next message exchange — no user-visible effect.

Known issues

  • 2,262 rejected events in Dendrite are skipped (expected)
  • ~5,500 orphan event edges referencing federated events we don't have (normal for any homeserver)
  • Synapse runs background update tasks after first startup on the migrated DB — this is normal and may take a few minutes
  • Re-running Phase 8 appends duplicate e2e_cross_signing_signatures rows (no unique constraint on the target table). Either run Phase 8 once, or TRUNCATE e2e_cross_signing_signatures before re-running.
Description
a vibe-coded attempt to port a dendrite matrix server to synapse
Readme AGPL-3.0 120 KiB
Languages
Python 100%