Docs Upgrades and Rollbacks

Upgrades and Rollbacks

This guide shows how to move a Mockarty installation to a new release without
stopping it, and how to go back if the new release misbehaves.

What happens during an upgrade

A new release can bring database changes. The first node of the new release
applies them when it starts; nodes of the previous release that are still
running keep serving users until they are replaced. When several new nodes start
at once, they wait for each other, so the changes are applied exactly once.

Most database changes only add things — a new table, a new column with a
default. The previous release keeps working on such a database, so it can run
side by side with the new one during a rolling upgrade and can take over again
after a rollback. A few changes remove or reshape something the previous release
still uses. After such a change the previous release refuses to start on the
database and tells you why, instead of starting and failing on the first
request.

You can see where a database stands at any time:

mockarty --migrate version

On Kubernetes, run it inside an admin pod, which already has the database
settings:

kubectl exec deploy/mockarty -- /app/mockarty --migrate version

The output looks like this:

Current version: 438 (dirty: false)
Binary schema version: 438
Oldest binary schema version this database accepts: 435
  • Current version — the database changes applied so far.
  • Binary schema version — the newest change this Mockarty binary knows.
    Run the command with each release’s binary to learn its number.
  • Oldest binary schema version this database accepts — a release whose
    binary schema version is at least this number can run on the database.

Before you upgrade

  1. Take a backup of the database and of the file storage. See
    Backup and Disaster Recovery.
  2. Read the release notes of every release between yours and the target one.
  3. Note the binary schema version of the release you run now: you need it if
    you roll back.

Upgrade on Kubernetes (Helm)

The chart replaces admin pods one at a time: a new pod starts, becomes ready,
and only then an old pod stops. Pin the exact version so the upgrade can be
repeated:

helm upgrade mockarty ./mockarty -f my-values.yaml \
  --set admin.image.tag=1.5.0 \
  --set resolver.image.tag=1.5.0 \
  --set runner.image.tag=1.5.0
kubectl rollout status deployment/mockarty

Upgrade the admin nodes first, then resolvers and runners — runners one release
behind keep working meanwhile; see Runner Fleet Operations.

Upgrade with Docker Compose or a single binary

A single node restarts, so users see a short interruption:

docker compose pull
docker compose up -d

For a binary installation, replace the binary and restart the service. The new
binary applies the database changes on start.

Roll back

First find out whether the previous release can run on the database as it is.
Run mockarty --migrate version and compare Oldest binary schema version this
database accepts with the binary schema version you noted before the upgrade.

The previous release is accepted (its number is equal or higher): roll back
the deployment. No database step is needed.

helm rollback mockarty

The previous release is not accepted (its number is lower): the new release
changed the database in a way the previous one cannot use. Undo those changes
with the new release’s binary, then roll back:

  1. Stop traffic to Mockarty, or scale the admin nodes to zero, so nothing
    writes during the step.

  2. Run the new release’s binary once against the database, with the previous
    release’s binary schema version as the target:

    mockarty --migrate down-to 435

    On Kubernetes, run it as a one-off pod with the new image and the same
    database settings as the admin pods.

  3. Roll back the deployment (helm rollback mockarty) and start traffic again.

down-to refuses to go below the oldest version this installation can return
to in place. If it refuses, restore the backup you took before the upgrade
instead.

Hardened PostgreSQL mode

When the admin nodes run with a restricted database role (DB_RUNTIME_ROLE),
they never change the database themselves. Apply the new release’s changes as a
separate maintenance step with the schema-owner credential before you roll out
the new image, and run down-to the same way when you roll back. The
Administration Guide describes this mode.

Previous
Retail Platform