Migrate Existing Mastodon from Masto.host to Linode

Migrate Existing Mastodon from Masto.host to Linode

Table of Contents

SKILL LEVEL: Advanced.

Info

This article was written to address a specific issue for a client. There is nothing inherently wrong with Masto.host for those organizations looking to have their own Mastodon instance without the headache involved in self-managed instances.

This guide addresses migrating a production Mastodon instance from Masto.host to a self-managed Linode VPS. There is nothing inherently wrong with Masto.host, but this tutorial serves a specific use case requiring independent infrastructure.

Existing Setup

For this tutorial, we’ll be moving a production instance of Mastodon from masto.host. This instance is running on the lowest tier of service from masto.host.

Because this tutorial used a test data, there are minimal accounts, posts, avatars, and cache involved. Real instances may be considerable larger.

Our DNS is via Cloudflare.

Things to confirm after migration

Here are a list of settngs configured on the existing instance that we want to ensure carry over to the new instance. Instance settings

  1. Server name
  2. Contact username
  3. Contact email
  4. Server description
  5. Server thumbnail
  6. Extended description
  7. Privacy policy

User content

  1. Bookmarks
  2. Toots
  3. Following
  4. Followers
  5. Direct Messages
  6. Cached content

Pre-Migration Tasks

Do this stuff a week or so before migration day.

Build out the future VPS in Linode

Now it’s time to provision the Virtual Private Server (VPS).

  1. Go to create a new Linode
  2. Select Debian 11
  3. Select a region (us-southeast)
  4. Select a Plan. Shared CPU on Linode 4GB
  5. Add a Linode Label
  6. Create a root user password
  7. Add SSH Keys
  8. Select the Backups Add-on
  9. Click “Create Linode” button.

Once the server has completed provisioning and booting, the status will turn to a green and state Running.

Setup and secure the VPS

Let’s do some configuration.

  1. Set a hostname
hostnamectl set-hostname YOUR.DOMAIN
  1. Set the timezone
timedatectl set-timezone 'TIME/ZONE'

Now that the VPS is up and running, we’ll want to make some adjustments fairly quickly.

apt update && apt upgrade

Create a new limited user

adduser USER

Add to sudo

usermod -aG sudo USER

Enable SSH and Multi-factor auth

  1. Generate a SSH Key via preferred method
  2. Create an .ssh directory
mkdir /home/USER/.ssh
  1. Create the auth keys file
touch /home/USER/.ssh/authorized_keys
  1. Change ownership
chown -R USER:USER /home/USER/.ssh
  1. Change permissions of auth file
chmod go-rw /home/USER/.ssh/authorized_keys
  1. Open auth file in nano
nano /home/USER/.ssh/authorized_keys
  1. Paste the public key and save

Install Google Authenticator

apt install -y libpam-google-authenticator

Edit sshd_config to be more

nano /etc/ssh/sshd_config
  1. Change PermitRootLogin yes to PermitRootLogin no
  2. Change PasswordAuthentication yes to PasswordAuthentication no
  3. Save

Restart SSHD

systemctl restart sshd

Exit and log back in as limited user.

Once logged in as the limited user, run Google Authenticator

google-authenticator
  1. Time-based tokens is yes
  2. Enroll using the code into authenticator app
  3. Save backup codes
  4. Last four quesitons are y,y,n,y

Edit sshd_config one more time for 2-factor login

sudo nano /etc/ssh/sshd_config
  1. Add ChallengeResponseAuthentication yes
  2. Add AuthenticationMethods publickey,keyboard-interactive
  3. Save

Edit Pam.d config

sudo nano /etc/pam.d/sshd
  1. Confirm @include common-auth exists
  2. Add auth required pam_google_authenticator.so

Restart SSH

sudo systemctl restart ssh

Exit and log back in to confirm.

Create object storage at Linode

  1. Go here: https://cloud.linode.com/object-storage/buckets
  2. Click ***Create Bucket## Sample .env.production file ##
LOCAL_DOMAIN=kilpen.net
SINGLE_USER_MODE=false
SECRET_KEY_BASE=[Shh...]
VAPID_PRIVATE_KEY=[Shh...]
VAPID_PUBLIC_KEY=[Shh...]
DB_HOST=/var/run/postgresql
DB_PORT=5432
DB_NAME=mastodon_production
DB_USER=mastodon
DB_PASS=[Shh...]
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_PASSWORD=
S3_ENABLED=true
S3_PROTOCOL=https
S3_BUCKET=hometowndemo
S3_ENDPOINT=https://kilpen-net-test.us-southeast-1.linodeobjects.com
S3_REGION=us-southeast-1
S3_HOSTNAME=kilpen-net-test.us-southeast-1.linodeobjects.com
AWS_ACCESS_KEY_ID=[Shh...]
AWS_SECRET_ACCESS_KEY=[Shh...]
S3_ALIAS_HOST=files.kilpen.net
SMTP_SERVER=smtp.mailgun.org
SMTP_PORT=587
SMTP_LOGIN=[email protected]
SMTP_PASSWORD=[Shh...]
SMTP_AUTH_METHOD=plain
SMTP_OPENSSL_VERIFY_MODE=none
SMTP_ENABLE_STARTTLS=auto
SMTP_FROM_ADDRESS='Mastodon <[email protected]>'
```***
3.  Give it a label and select a region
4.  Go here: https://cloud.linode.com/object-storage/access-keys
5.  Click ***Create Access Key***
6.  Give it a label and limit access to Read/Write only on the new bucket you just created.
7.  Save credentail and click ***I have my secret key***

## Create MailGun Credential ##
1. Create a MailGun account, or use an existing one.
2. Downgrade your plan to the Flex plan.   You'll need to Begin cancellation to do this.
3. Add your domain and verify using Cloudflare DNS TXT entry.
4. Adjust your domain's SPF record, DKIM record, and add MX if one doesn't already exist.
5. Once all verified, create SMTP credentials.
6. Save these credentials for later.

## Installing Mastodon and other packages ##
[Reference:](https://docs.joinmastodon.org/admin/install/)
Install some necessary stuff
```bash
sudo apt install -y curl wget gnupg apt-transport-https lsb-release ca-certificates

Install Node.js

sudo curl -sL https://deb.nodesource.com/setup_16.x | sudo bash -

Add PostgreSQL to source.list

sudo wget -O /usr/share/keyrings/postgresql.asc https://www.postgresql.org/media/keys/ACCC4CF8.asc
sudo echo "deb [signed-by=/usr/share/keyrings/postgresql.asc] http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > sudo /etc/apt/sources.list.d/postgresql.list

Install System Packages

sudo apt update
  sudo apt install -y \
  imagemagick ffmpeg libpq-dev libxml2-dev libxslt1-dev file git-core \
  g++ libprotobuf-dev protobuf-compiler pkg-config nodejs gcc autoconf \
  bison build-essential libssl-dev libyaml-dev libreadline6-dev \
  zlib1g-dev libncurses5-dev libffi-dev libgdbm-dev \
  nginx redis-server redis-tools postgresql postgresql-contrib \
  certbot python3-certbot-nginx libidn11-dev libicu-dev libjemalloc-dev

Install Yarn

sudo npm install --global yarn
yarn set version classic

Create the PostgreSQL database

Switch to the Postgres user

sudo su - postgres

Launch psql

psql

Create the database, user, and change ownership

CREATE DATABASE mastodon_production;
CREATE USER mastodon;
ALTER USER mastodon createdb;
ALTER USER mastodon WITH ENCRYPTED PASSWORD 'SET_A_PASSWORD';
ALTER DATABASE mastodon_production OWNER TO mastodon;

Now exit back to the limited user account

exit
exit

Create a Mastodon user

Add the new user

sudo adduser --disabled-login mastodon

Setup and secure the VPS

Switch accounts

sudo su - mastodon

Install Ruby

Install and Setup

git clone https://github.com/rbenv/rbenv.git ~/.rbenv
cd ~/.rbenv && src/configure && make -C src
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(rbenv init -)"' >> ~/.bashrc
exec bash
git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build
RUBY_CONFIGURE_OPTS=--with-jemalloc rbenv install 3.0.6
rbenv global 3.0.6
gem install bundler --no-document

Clone Mastodon Repo

As the mastodon user, ensure you’re in ~ and clone the repo

git clone https://github.com/mastodon/mastodon.git live && cd live

Configure nginx

As the limited user, prepare the .conf files.

sudo cp /home/mastodon/live/dist/nginx.conf /etc/nginx/sites-available/mastodon

Edit the configuration file for the primary webserver

sudo nano /etc/nginx/sites-available/mastodon

Update these fields:

  • server_name - replace example.com with your domain in both 80 and 443 sections
  • Add snake oil lines for certbot
ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;

Enable the mastodon site

sudo ln -s /etc/nginx/sites-available/mastodon /etc/nginx/sites-enabled/mastodon

Create another .conf to proxy cache and storage

sudo touch /etc/nginx/sites-available/files.DOMAIN.NAME

Edit the .conf

sudo nano /etc/nginx/sites-available/files.DOMAIN.NAME

Paste in the following config Source: https://docs.joinmastodon.org/admin/optional/object-storage-proxy/

server {
  listen 443 ssl http2;
  listen [::]:443 ssl http2;
  server_name files.example.com;
  root /var/www/html;

  keepalive_timeout 30;

  location = / {
    index index.html;
  }

  location / {
    try_files $uri @s3;
  }

  set $s3_backend 'https://YOUR_BUCKET_NAME.YOUR_S3_HOSTNAME';

  location @s3 {
    limit_except GET {
      deny all;
    }

    resolver 8.8.8.8;
    proxy_set_header Host YOUR_BUCKET_NAME.YOUR_S3_HOSTNAME;
    proxy_set_header Connection '';
    proxy_set_header Authorization '';
    proxy_hide_header Set-Cookie;
    proxy_hide_header 'Access-Control-Allow-Origin';
    proxy_hide_header 'Access-Control-Allow-Methods';
    proxy_hide_header 'Access-Control-Allow-Headers';
    proxy_hide_header x-amz-id-2;
    proxy_hide_header x-amz-request-id;
    proxy_hide_header x-amz-meta-server-side-encryption;
    proxy_hide_header x-amz-server-side-encryption;
    proxy_hide_header x-amz-bucket-region;
    proxy_hide_header x-amzn-requestid;
    proxy_ignore_headers Set-Cookie;
    proxy_pass $s3_backend$uri;
    proxy_intercept_errors off;

    proxy_cache CACHE;
    proxy_cache_valid 200 48h;
    proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
    proxy_cache_lock on;

    expires 1y;
    add_header Cache-Control public;
    add_header 'Access-Control-Allow-Origin' '*';
    add_header X-Cache-Status $upstream_cache_status;
    add_header X-Content-Type-Options nosniff;
    add_header Content-Security-Policy "default-src 'none'; form-action 'none'";
  }
}

Now make the following edits:

ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;

Lastly, activate this site in Nginx:

sudo ln -s /etc/nginx/sites-available/files.DOMAIN.NAME /etc/nginx/sites-enabled

Setting up Mastodon

As the Mastodon user from the ~/live directory

[TAKE A SNAPSHOT BACKUP BEFORE MOVING FORWARD!]

Select the version

git checkout v4.1.6

Run the following to configure and install Mastodon

bundle config deployment 'true'## Sample .env.production file ##

LOCAL_DOMAIN=kilpen.net SINGLE_USER_MODE=false SECRET_KEY_BASE=[Shh…] VAPID_PRIVATE_KEY=[Shh…] VAPID_PUBLIC_KEY=[Shh…] DB_HOST=/var/run/postgresql DB_PORT=5432 DB_NAME=mastodon_production DB_USER=mastodon DB_PASS=[Shh…] REDIS_HOST=localhost REDIS_PORT=6379 REDIS_PASSWORD= S3_ENABLED=true S3_PROTOCOL=https S3_BUCKET=hometowndemo S3_ENDPOINT=https://kilpen-net-test.us-southeast-1.linodeobjects.com S3_REGION=us-southeast-1 S3_HOSTNAME=kilpen-net-test.us-southeast-1.linodeobjects.com AWS_ACCESS_KEY_ID=[Shh…] AWS_SECRET_ACCESS_KEY=[Shh…] S3_ALIAS_HOST=files.kilpen.net SMTP_SERVER=smtp.mailgun.org SMTP_PORT=587 [email protected] SMTP_PASSWORD=[Shh…] SMTP_AUTH_METHOD=plain SMTP_OPENSSL_VERIFY_MODE=none SMTP_ENABLE_STARTTLS=auto SMTP_FROM_ADDRESS=‘Mastodon [email protected]’

bundle config without 'development test'
bundle install -j$(getconf _NPROCESSORS_ONLN)
yarn install --pure-lockfile

DNS Prep

Cloudflare should have three A records entries:

  • domain root
  • www
  • files

Migration Day

Retrieve data from MastoHost

Via the Masto.host interface, stop your production server and request a backup of the current data. Once you receive the email with the link, download the contents to your local machine. Unfortunately, MastoHost doesn’t seem to allow download using wget from their servers directly to your VPS.

Upload MastoHost database to the Linode VPS

Create a new SSH key just for this effort. Add the public key to ~/.ssh/authorized_keys

scp -i ~/.ssh/id_rsa.pub FILENAME kilpen@[IP]:/home/kilpen/FILENAME

Upload MastoHost files to the Linode Bucket

Certbot

Run Certbot

sudo certbot --nginx

After successful certificate issue, restart nginx

sudo systemctl reload nginx

Start Mastodon

Copy the services to systemd

sudo cp /home/mastodon/live/dist/mastodon-sidekiq.service /etc/systemd/system/ &&
sudo cp /home/mastodon/live/dist/mastodon-streaming.service /etc/systemd/system/ && sudo cp /home/mastodon/live/dist/mastodon-web.service /etc/systemd/system/

Reload and enable

sudo systemctl daemon-reload
sudo systemctl enable --now mastodon-web mastodon-sidekiq mastodon-streaming

Sample .env.production file

LOCAL_DOMAIN=kilpen.net
SINGLE_USER_MODE=false
SECRET_KEY_BASE=[Shh...]
VAPID_PRIVATE_KEY=[Shh...]
VAPID_PUBLIC_KEY=[Shh...]
DB_HOST=/var/run/postgresql
DB_PORT=5432
DB_NAME=mastodon_production
DB_USER=mastodon
DB_PASS=[Shh...]
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_PASSWORD=
S3_ENABLED=true
S3_PROTOCOL=https
S3_BUCKET=hometowndemo
S3_ENDPOINT=https://kilpen-net-test.us-southeast-1.linodeobjects.com
S3_REGION=us-southeast-1
S3_HOSTNAME=kilpen-net-test.us-southeast-1.linodeobjects.com
AWS_ACCESS_KEY_ID=[Shh...]
AWS_SECRET_ACCESS_KEY=[Shh...]
S3_ALIAS_HOST=files.kilpen.net
SMTP_SERVER=smtp.mailgun.org
SMTP_PORT=587
SMTP_LOGIN=[email protected]
SMTP_PASSWORD=[Shh...]
SMTP_AUTH_METHOD=plain
SMTP_OPENSSL_VERIFY_MODE=none
SMTP_ENABLE_STARTTLS=auto
SMTP_FROM_ADDRESS='Mastodon <[email protected]>'
Tags :
Share :

Related Posts

Migrate Existing Docker Peertube Instance to Linode, Enable Object Storage

Migrate Existing Docker Peertube Instance to Linode, Enable Object Storage

SKILL LEVEL: Advanced

Read More