Upgrading a docker-compose based Mastodon server to gain today's security fixes

My previous post detailing how to run a Mastodon server using docker-compose includes a section detailing how to upgrade an instance built using that approach

However, today's Mastodon releases (v4.1.3, v4.0.5 and v3.5.9) include important security fixes (especially the fix for CVE-2023-36460), so I thought it was worth a quick post detailing the process that I've followed to upgrade my instance (mastodon.bentasker.co.uk) to ensure that I get today's security fixes in place.

Backup

Every Mastodon upgrade should be preceded by a backup of the Postgres database:

docker exec postgres pg_dumpall -U postgres | gzip > postgres_backup_PREUPGRADE_`date +'%Y%m%d'`.sql.gz

Depending on the size of your database (and the speed of the storage you're writing too), this may take a little while to complete. The next section includes some prep that you can be doing whilst it runs.

Upgrading from Earlier Versions

Prior to today's release, the latest version of Mastodon was v4.1.2.

If you've fallen behind in applying updates, it's important that you check release notes for each of the versions between your current version and v4.1.2: upgrades sometimes include additional steps, so you need to identify whether there are any additional commands that you need to run.

For example, my instance was running v4.1.0, so I needed to check

• Release notes for v4.1.2 (no additional steps when upgrading from v4.1.1)
• Release notes for v4.1.1 (no additional steps when upgrading from v4.1.0)

However, had I instead been running v4.0.2, I would have needed to run a database migration.

If you're in that position, you'll need to work through from the earliest work-requiring version to the latest (you can skip over any version that doesn't need additional steps run).

Essentially you just need to follow the process below for each of the interim versions that you need to update to, run the migration, and then upgrade to the next work-requiring version until you reach today's version - as long as you're not miles behind, it's a much quicker process than it sounds.

Docker Image

The notification email sent to admins of larger instances notes that, when released, the docker image would be tagged nightly-2023-07-06-security.

Note that Nightlies go to the Github Container Registry not docker hub.

So, in preparation for the upgrade, we need to update the image in docker-compose.yml (it'll need updating for your web, streaming and sidekiq instances)

diff --git a/docker-compose.yml b/docker-compose.yml
index dd681e4..cfdc182 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -97,7 +97,7 @@ services:
             - /usr/local/share/meiko/files/mastodon/files/redis:/data

     web:
-        image: tootsuite/mastodon:v4.1.0
+        image: tootsuite/mastodon:v4.1.3
         restart: always
         container_name: web
         env_file: /usr/local/share/meiko/files/mastodon/files/.env.production
@@ -117,7 +117,7 @@ services:
             - /usr/local/share/meiko/files/mastodon/files/public/system:/mastodon/public/system

     streaming:
-        image: tootsuite/mastodon:v4.1.0
+        image: tootsuite/mastodon:v4.1.3
         restart: always
         container_name: streaming
         env_file: /usr/local/share/meiko/files/mastodon/files/.env.production
@@ -135,7 +135,7 @@ services:
             - redis

     sidekiq:
-        image: tootsuite/mastodon:v4.1.0
+        image: tootsuite/mastodon:v4.1.3
         restart: always
         container_name: sidekiq
         env_file: /usr/local/share/meiko/files/mastodon/files/.env.production

Note: Do not use latest - it'll drop you onto the 3.x branch

Run database migrations (don't forget this bit or you can end up looking a bit of a tit)

docker-compose run --rm web rails db:migrate

Pull the new image (not technically necessary, but will speed the next step up)

docker pull ghcr.io/mastodon/mastodon:nightly-2023-07-06-security

And then have services restart using the new image

docker-compose up -d

Update

The v4.1.3 tag has landed, so if you followed an earlier version of this post to move to nightlies you can now move back onto the main release track - just update the image to tootsuite/mastodon:v4.1.3 and run docker-compose up -d again.

Conclusion

With the fixes having been released as a tagged image, upgrading is quick and easy - at least once I realised I needed to watch ghcr.io for the release and not dovker hub.

In the unlikely event that something does go wrong, rollback is just a case of switching the docker image back in docker-compose.yml and (if needed) restoring the database backup.