Idempotency in Infrastructure as Code, Explained

Idempotency is the property that separates infrastructure code from infrastructure scripts. An operation is idempotent when running it once or running it fifty times gives the same result.

In practice, this is what makes automation safe to re-run. You can run it after a failure, on a schedule, or from CI, with no fear of duplicates or piled-up side effects.

It also sits at the center of how codenization works. Every tool in the codenize family is built around an idempotent apply.

What Idempotency Means

Here is the math version. A function f is idempotent when f(f(x)) = f(x). The first application may change many things. Every application after that changes nothing, as long as nothing else changed in between.

You already use idempotent operations. HTTP PUT is idempotent; POST is not. mkdir -p is idempotent; plain mkdir is not.

For infrastructure, the working definition is short. The same code, applied N times against the same system, leaves it in the state one apply would.

Two distinctions to keep straight

• Idempotent is not the same as deterministic. A tool can be idempotent even if its API calls run in a different order each time, as long as the end state matches.
• Idempotent is not the same as transactional. An apply can fail halfway and leave partial state. Idempotency is what rescues you: re-running finishes the rest instead of doubling the start.

Why It Matters

• Safe retries. Networks time out. API rate limits bite mid-run. With an idempotent tool, the recovery for any failure is the same: run it again. Without one, you first have to work out how far the last run got.
• Convergence, not sequence. You do not need a system's history to bring it to a desired state. The file says where to end up. The tool works out the path from wherever the system is now.
• Drift correction. Out-of-band edits, like a console hotfix or a manual GRANT, get reverted by the next apply. The file stays authoritative.
• No snowflakes. Apply the same file to staging and production, and you get identical config. Not two hand-raised environments that only look alike on a good day.

There is a quieter benefit too. Automation becomes boring. Once applying is guaranteed safe, you can hang it off a merge to main, a nightly cron, or an on-call runbook step. No human has to stand by and judge whether this particular run is risky.

Non-idempotent scripts never earn that trust, so they stay manual forever. And manual steps are where configuration drift is born. The full loop that relies on this safety is the export, review, apply workflow.

How Codenize Tools Achieve It

Every codenize tool runs the same three-phase apply:

1. Read. Fetch the current live state through the service's API. List the record sets, the users, the grants.
2. Evaluate. Run the Ruby DSL file to build the desired state as an in-memory model.
3. Diff and apply. Compare the two. Issue mutating API calls only for the delta: create what is missing, update what differs, delete what the file no longer declares.

If the diff is empty, the tool makes no write calls at all. That is the whole trick. Idempotency is not a property of each API call. It is a property of the diff step in front of them.

When Roadworker applies a Routefile, it lists the zone's records, compares them to the file, and calls ChangeResourceRecordSets only for records that actually differ.

Notice what is missing: a state file

Terraform reaches convergence by diffing code against a recorded state backend. Codenize tools diff code against the live API directly, on every run. There is no state to lock, migrate, or fall out of sync.

The trade-off is that the tool has no memory. Everything within its scope is treated as managed. It cannot tell an intentional resource from a forgotten one.

Declarative DSLs vs Imperative Scripts

The instinctive way to automate AWS is a shell script full of CLI calls:

aws route53 change-resource-record-sets \
  --hosted-zone-id Z1EXAMPLE \
  --change-batch '{"Changes":[{"Action":"CREATE",
    "ResourceRecordSet":{"Name":"www.example.com.",
    "Type":"A","TTL":300,
    "ResourceRecords":[{"Value":"192.0.2.10"}]}}]}'

Run it twice, and the second run fails with InvalidChangeBatch, because the record already exists. Switching CREATE to UPSERT makes the call idempotent per record. But only per record.

Nothing in the script will ever delete records that should no longer exist. So the zone quietly fills with garbage. The script encodes actions. Nothing encodes the desired whole.

Where scripts still fit

To be fair to imperative scripts, they are the right tool for genuinely one-shot jobs. A data backfill or an emergency block during an incident fits them well. The failure mode is using them for config that lives on, because a script only records what was true the day someone ran it.

Two Worked Examples

A Route 53 record

hosted_zone "example.com." do
  rrset "api.example.com.", "A" do
    ttl 60
    resource_records "192.0.2.20"
  end
end

The first roadwork -a creates the record. Run it again right away: no API writes, nothing reported as changed.

Now suppose someone bumps the TTL to 300 in the console during an incident. The next apply sets it back to 60. The file wins, every time. That is exactly the convergence you signed up for, and a communication problem to manage. See the pitfalls below.

A MySQL grant

user "app", "10.0.%.%" do
  on "orders.*" do
    grant "SELECT"
    grant "INSERT"
    grant "UPDATE"
  end
end

The imperative equivalent is GRANT statements pasted from a runbook. In practice it is additive only, because nobody ever writes the matching REVOKE. Privileges pile up for years.

Gratan computes both directions. It grants what the file declares and the database lacks. It revokes what the database holds but the file does not. Apply the same Grantfile twice, and the second run issues zero SQL. Posgra applies the same idea to PostgreSQL roles.

Common Pitfalls

• Out-of-band changes. Reverting console edits is correct, but it surprises people. An engineer's 2 a.m. hotfix silently disappears at the next scheduled apply. Fix it socially (the repo is the only sanctioned change path) and technically (run scheduled dry-runs so drift is visible before an apply erases it).
• Partial failures. An apply that dies halfway leaves partial state. Idempotency is the remedy: re-run, and the tool finishes the rest. But your automation has to actually re-run, not just mark the job red and move on.
• Normalized resources. Some APIs rewrite what you send. DNS names get case-folded. TXT values get re-escaped. Security group rules get reordered. If the exported form and the applied form differ, you get a perpetual diff: a "change" the tool applies on every run that never converges. This shows up with Piculet security group edge cases and with escaped characters in Route 53 records. Fix the file to match the API's canonical form, or you lose the empty-diff signal that everything is fine.
• Absence means deletion. Because the file declares the whole scope, removing a block deletes the real resource. This bites hardest right after adoption. Export, delete a "stray" entry from the file, apply, and you have deleted it in production. It also means idempotent is not reversible. Ridgepole will apply a Schemafile missing a column perfectly idempotently, and the dropped column's data is gone.

Dry-Run and Diff: Idempotency You Can See

Every codenize tool ships a --dry-run flag. It runs the full read-evaluate-diff pipeline and prints the delta without writing anything:

$ roadwork -a --dry-run
Update ResourceRecordSet: api.example.com. A
  ttl: 300 => 60 (dry-run)

Three habits fall out of this:

1. Dry-run before every apply. The printed delta is the change. If it surprises you, stop.
2. Dry-run after every apply. Empty output is a machine-checked proof of convergence. It is the definition of idempotency made visible.
3. Dry-run on a schedule. A nightly dry-run with non-empty output is a drift alarm. It catches out-of-band changes while they are still news.

Idempotency gives you a cheap, side-effect-free question you can ask any time: does reality still match the code? An empty diff is the only answer you ever want to see.

That last habit is a discipline of its own, covered in Dry-Run and Drift Detection.