ShipOps

Backups

The backup that restores perfectly and contains nothing

Here is a backup restoring. Read it carefully, because on the face of it there is nothing wrong with it.

$ pg_restore -U app -d verify --exit-on-error backup.dump
$ echo $?
0

$ psql -U app -d verify -c "\dt"
        List of relations
 Schema |    Name     | Type  | Owner
--------+-------------+-------+-------
 public | users       | table | app
 public | orders      | table | app
 public | line_items  | table | app
(3 rows)

The archive is well-formed. pg_restore exits zero with --exit-on-error set. Every table is present, with the right names and the right schema.

There are no rows in any of them.

This backup would pass every check that most backup systems perform, and it is worth precisely nothing. If you restored it during an incident you would have a database shaped exactly like production, containing none of production.

How this happens

Not through corruption. Corruption is the easy case — a truncated file fails loudly and you find out immediately. This is the quiet one, and it arrives through ordinary mistakes:

Every one of those produces a file of a plausible size that restores without complaint. And because the failure is silent, it survives until the day you need it.

Why the usual checks miss it

There are three checks people normally run, and each one passes.

"The file exists and looks about the right size"

A schema-only dump of a real database is not small. Three tables with indexes, constraints and sequences is several kilobytes before any rows. Size thresholds catch empty files, not empty backups.

"The archive parses"

For Postgres, pg_restore --list reads the archive's table of contents. It is a genuinely good integrity check — it fails on a truncated file — and a schema-only dump passes it perfectly, because there is nothing wrong with the archive.

"The restore exits zero"

It does. There were no errors. Restoring nothing successfully is still success.

MongoDB is worse, because the recommended check is misleading

If you back up MongoDB with mongodump --archive --gzip, the natural way to verify the result is mongorestore --dryRun, which parses the archive without writing anything.

It reads the archive prelude — the header listing which collections are inside — and stops.

Here is --dryRun against an archive deliberately truncated to 400 bytes:

$ head -c 400 good.archive.gz > truncated.archive.gz
$ mongorestore --archive --gzip --dryRun < truncated.archive.gz
preparing collections to restore from
dry run completed
0 document(s) restored successfully. 0 document(s) failed to restore.

$ echo $?
0

Exit zero on an archive with 400 bytes of a multi-megabyte file. The only case it fails is a completely empty file. If your Mongo backup verification is built on --dryRun, it is not verification — it confirms that a header exists.

A real restore does catch it. Restoring the same truncated archive into a scratch namespace exits 1 and produces one collection where the header claimed two. But you have to actually restore it.

The check that works

Restore the backup somewhere disposable, then compare row counts against production. That is the whole idea, and it is the only check in this article that catches every case above.

It works because it tests the thing you actually care about — is the data there — rather than a proxy for it. A schema-only dump restores three tables and zero rows against a live database with 8,415. That comparison cannot be passed by a backup that does not contain your data.

The shape of it, whichever database you run:

  1. Fetch the most recent backup from wherever it is stored.
  2. Verify its checksum against one recorded at the time it was written.
  3. Start a throwaway database container — on the same image as production, see below.
  4. Restore into it. It touches nothing live.
  5. List the tables or collections, and count the rows in each.
  6. Compare against production, allowing some drift — the dump is older than now — but not allowing zero.
  7. Tear it down and report.

Run weekly, it takes seconds and it moves your backups from a hypothesis to a fact.

$ ./restore-test.sh
  PASS  backup freshness: 7h old
  PASS  checksum matches (51b103a05e7c…)
  PASS  restore completed without errors
  PASS  restored copy contains 3 tables
  FAIL  users: only 0 restored but live has 1,203
  FAIL  orders: only 0 restored but live has 8,415

  restore verification: 2 passed, 2 failed

One trap worth knowing about

Run the verification with client tools from inside the database container, never from the host.

A custom-format Postgres dump carries an archive version, and an older pg_restore cannot read a newer one:

$ pg_restore --list backup.dump
pg_restore: error: unsupported version (1.15) in file header

That is a host running Ubuntu's postgresql-client 14 against a postgres:16 container. The backup is perfect. The message is not about corruption at all, and if your verification is wired this way it will declare every healthy backup broken — which means you will either ignore the alert or lose a day to it. Using the binary inside the database container makes a version mismatch impossible. Full write-up of that error here.

Go and check yours

Not the file listing. Not the exit code. Restore your most recent backup somewhere disposable and count the rows.

If you got here because you built something with an assistant and are working out how to launch it, start here instead — this is one item on a longer list.

Most people have never done this once. Some of them find out that the backup they have been keeping for eight months restores an empty database — and they find out on a Tuesday afternoon rather than during an outage, which is the entire point.