Backup validation: integrity checks, copies and immutability

This article is for backup administrators and the managers who sign off on backup spending. It separates three questions that are usually blurred together: is the backup data intact, is it restorable, and could an attacker or an accident have destroyed it along with production? Each needs a different check.

Three properties, three kinds of check

A backup can fail you in three independent ways.

1. It is damaged. Bits changed in storage or in transit, an upload was truncated, or the job wrote a file that looks complete but is not.
2. It is unusable. The data is intact but cannot be turned back into a working system: the dump is inconsistent, the encryption key is gone, the software version that reads the format is no longer available, or a dependency was never captured.
3. It is reachable by whatever destroyed the original. Ransomware, a compromised administrator account, a misconfigured retention policy or a mistaken command can delete or encrypt backups as readily as production data if they sit within the same blast radius.

Integrity checks address the first. Test restores address the second (the mechanics are in restore testing). Isolation and immutability address the third. A programme that does only one of these is covered against a single failure mode and exposed to the other two.

Integrity checks

Use the tool’s own verification

Backup products and open-source tools usually include a verify or check operation that reads stored data and confirms it matches recorded hashes. Schedule it, and keep its output. Examples include restic check and borg check; for vendor products, look for a feature called backup verification, health check, or integrity check in the documentation for your version. Know what it covers: some verify only metadata or a sample, which is fast and misses silent corruption of unsampled data.

Do your own checksums where the tool cannot

For files you move yourself, such as database dumps copied to a remote location, record a cryptographic checksum at the source and verify at the destination:

# at the source, immediately after the dump completes
sha256sum orders-2026-10-01.dump > orders-2026-10-01.dump.sha256

# at the destination, after transfer
sha256sum -c orders-2026-10-01.dump.sha256

The check prints OK or FAILED per file, and the command exits non-zero on failure, so it can gate a job. Store the checksum file somewhere an attacker with write access to the dump cannot also quietly rewrite it, otherwise the checksum proves nothing about tampering. The Windows PowerShell equivalent is Get-FileHash -Algorithm SHA256.

Be precise about what a checksum shows. It shows that the file is byte-for-byte what it was when hashed. It does not show that the file was a valid backup at that moment.

Check the container too

Archive formats have their own validation:

gzip -t orders-2026-10-01.dump.gz
tar -tzf site-files-2026-10-01.tar.gz > /dev/null

Both commands only read. gzip -t tests the compressed stream; tar -tzf lists the archive and fails on a truncated or corrupt one. For a PostgreSQL custom-format dump, pg_restore --list orders-2026-10-01.dump reads the table of contents and fails on a damaged file, though it still does not prove every table’s data restores.

Watch for the quiet failures

Alert on facts that don’t produce an error message:

• Size anomalies. A backup that is suddenly 5% of its usual size has probably skipped something.
• Date anomalies. Compare the newest restore point with the schedule. A job “running successfully” against a repository whose newest snapshot is a week old is common.
• File-count anomalies. The same, for count.
• Silent exclusions. Open files, locked databases and mounted volumes are often skipped with a warning that nobody reads. Treat “completed with warnings” as a failure until the warnings are explained.

Test restores

Integrity and anomaly checks tell you the backup is probably healthy. Only a restore tells you it works. The practical approach is to restore into an isolated location and compare the result with a baseline, as laid out in the restore testing ladder. The validation programme should ensure that over a rotation every protected system gets at least a rung-4 or rung-5 test, and that the results (what was restored, from which restore point, how long it took, what failed) are recorded.

Copies, media and the 3-2-1 rule

A long-standing rule of thumb is 3-2-1: keep at least three copies of the data (the original plus two backups), on at least two different kinds of storage, with at least one copy held off-site. Its purpose is to make sure no single event takes out every copy. Some vendors describe an extended variant, often written 3-2-1-1-0, that adds one copy that is offline or immutable and zero unresolved errors after verification. Treat both as heuristics for reasoning about independence, not as compliance requirements.

Apply the rule by asking what single event destroys all my copies? for each of: a hardware failure, a site loss, a credential compromise, a software bug in the backup product, a mistaken retention change, a billing or account suspension at the storage provider. Where the answer is “everything”, you have found a common cause that needs a copy outside it.

Immutability and isolation

Production credentials are the usual route to destroying backups. If the account that administers the servers can also delete the backup repository, then an attacker who obtains that account, or an engineer who makes a mistake with it, can erase both at once. Two controls break that link.

Immutable storage prevents modification or deletion of data for a defined retention period, even by privileged users. Object storage services commonly offer this as write-once-read-many behaviour; Amazon S3 Object Lock is one well-documented example, and it works with bucket versioning. It has two modes that behave very differently: in governance mode, specially permissioned users can override the lock; in compliance mode, nobody, including the account’s root user, can shorten the retention or delete the object until it expires. Compliance mode is the stronger protection and also the harder to undo if you misconfigure the retention period, so test it with a small bucket and short retention first. Other platforms offer comparable features under different names; read your vendor’s documentation to see exactly what it promises and which credentials can bypass it.

Be careful with weaker substitutes. A filesystem immutable flag on Linux (chattr +i) stops casual modification, but anyone with root on that host can remove the flag. It protects against some accidents, not against an attacker who has administrative control of the machine.

Offline or logically separate copies keep a copy out of reach of the network or the credential domain. Examples are tapes or disks that are disconnected and stored elsewhere, or a repository in a separate cloud account whose credentials are not held by the production administrators and which accepts writes only through a narrowly scoped role. The test is concrete: sign in as the most privileged production administrator you have and try to find a way to delete the last copy. If you succeed, it is not isolated. Do this as a read-through of permissions and policies, or in a test environment; do not attempt deletion against your real last copy.

Isolation has operating costs. An immutable copy cannot be pruned early, so storage grows; an offline copy has to be rotated by a person; a separate-account copy needs its own access plan for the day you need it. Plan the retrieval path as carefully as the storage, and include it in a drill: during an incident the people who hold the isolated credentials must be reachable and must know the procedure.

Encryption and keys

Encrypted backups are sensible and add one more dependency: without the key, the data is noise. Record, for each encrypted backup set, where the key or passphrase is held, who can retrieve it, and whether retrieving it depends on any system that the backup is meant to restore. Test retrieval during a restore exercise. Keep a separate, controlled copy of key material in a location that survives the loss of the primary environment.

What to record

For each protected system keep a short validation record:

Field	Meaning
System and data set	What is protected
Copies	How many, on what media, in which locations
Isolated or immutable copy	Yes or no, mechanism, retention
Last integrity verification	Date and result
Last test restore	Date, rung reached, restore point, duration, result
Key location and retrieval path	Who, how, last tested
Known gaps	Anything unverified

“Unknown” is a valid entry, and a more useful one than an assumption.

Common mistakes

• Equating replication with backup. Replicated deletions and corruption arrive at the copy within seconds.
• Backing up to the same credential domain. The repository is a share mapped by the domain administrators.
• A verification job nobody reads. Failures email an address that no longer exists.
• Validating the newest backup only. Corruption or retention problems often affect older restore points that you will need after a late-discovered deletion.
• Assuming cloud means safe. Provider durability protects against hardware loss; it does not protect against your own deletion commands or a compromised account.
• Setting retention without checking it. Confirm that the oldest restore point you think you hold actually exists.

What to do next

Fill in the validation record for your three most important systems and note every “unknown”. Then use the drill planner, which asks about backup isolation and test-restore history and tells you what to address first; it only organises what you tell it and verifies nothing. For turning verified backups into a service that actually comes back in time, continue with designing a recovery drill.