Infrastructure as Code for ML

Why Does This Exist?

Every ML system beyond a laptop lives on infrastructure someone had to create: GPU machines for training, buckets for datasets, queues between pipeline steps, endpoints for serving. The default way to create all that is clicking through a cloud console. It works once. Then you need a second environment for staging, a third region for latency, or a rebuild after someone deletes the wrong thing, and nothing matches anymore. The training cluster has a driver the serving cluster lacks. The staging bucket has versioning on; production does not. You spend days diffing settings by eye.

Infrastructure as Code fixes this by writing down the desired cloud in files, under version control, and letting a tool do the creating. The file says what should exist. The tool compares that wish against reality, shows you the difference, and only then makes changes. Rebuilding an environment stops being archaeology and becomes a rerun.

This page covers provisioning the cloud around your code: machines, networks, buckets, permissions. Packing the code itself into a portable image is the neighbouring job of containerization for ML. Running tests and retraining on every change is CI/CD for ML. If the full lifecycle is new, start with the MLOps overview first.

Basics first: declarative means you describe the destination, not the route. You write "there is a bucket called ml-datasets with versioning on" instead of a script of clicks: open console, click create, tick the box. Because the wish is written down, the tool can compare it against the cloud as it actually is and tell you the gap before touching anything. That preview is the whole safety story, and it only works because you stated the end state.

Think of It Like This

How It Actually Works

Four pieces move in a loop: a template, a plan, an apply, and a state record.

Declare. You write resources as data: a bucket, four GPU nodes, a serving endpoint, two alarms. Each block names its type and its settings. Variables fill in the parts that differ per environment, like the region or the machine size, while everything else stays shared.

Plan. Before anything changes, the tool reads the cloud as it is and diffs it against the template. Call the wish $D$ (desired bindings) and reality $A$ (actual bindings); the plan is the gap $D - A$, reported as counts to add, change, and destroy. You read those three numbers before approving anything.

Apply. Approval runs the cloud APIs in dependency order: network before machines, bucket before the training job that reads it. Independent resources go in parallel. Templates get tested like any code before they run against production.

Record. Every created object is written into a state file that maps template blocks to real cloud IDs. Next plan starts from this record, so a change made by hand in the console shows up as an unexpected difference rather than silently passing.

Worked example

One template holds 5 resource blocks: 1 bucket, 2 GPU nodes, 1 serving endpoint, 1 alarm. You render it for 3 environments (dev, staging, prod):

The state record says 14 exist, because someone deleted the staging alarm by hand last Tuesday. The plan is therefore $15 - 14 = 1$ to add, 0 to change, 0 to destroy. You approve, the tool recreates exactly the missing alarm, and the next plan reads $15 - 15 = 0$ everywhere. One number told you the whole story before a single API call ran.

Code

The point of this snippet is the counting, not the cloud. It renders one template across three environments and diffs the wish against a state record, printing the same three numbers a real plan preview shows.

1template = {"s3_bucket": 1, "gpu_node": 2, "api_service": 1, "alarm": 1}2envs = ["dev", "staging", "prod"]3
4desired = sum(template.values()) * len(envs)  # (1+2+1+1)*35state_record = desired - 1  # someone deleted the staging alarm by hand6
7print(f"desired={desired} actual={state_record}")8# -> desired=15 actual=149print(f"plan: {desired - state_record} to add, 0 to change, 0 to destroy")10# -> plan: 1 to add, 0 to change, 0 to destroy

Fifteen bindings from five lines of template, and the gap of one names the exact repair. That is the daily experience: small integers, read before anything moves.

Watch Out For

The Quick Version

• IaC declares cloud resources in version-controlled files, so environments are rebuilt by rerunning instead of by remembering.
• The plan step diffs desired bindings against actual ones and reports counts to add, change, and destroy before anything moves.
• A state file maps template blocks to real cloud IDs, which is what makes hand-made console edits visible as drift.
• Five template blocks across three environments means fifteen bindings, and a gap of one names the exact repair.
• Secrets go in a manager and are referenced, never pasted, because templates live in git history.
• The console is read-only by discipline: every real change flows through the file, or the next apply reverts it.

What to Read Next

• Containerization for ML
• CI/CD in ML Pipelines
• ML Testing Strategies