You’ve spun up a new VM in AWS, Azure, or GCP. You’ve got your cloud-init user data script configured to install some packages, tweak a few config files, and crucially, start a service. Everything seems fine on the first boot. Then you restart the instance a week later for patching, or maybe it’s part of an auto-scaling group that gets replaced, and suddenly that service isn’t starting. Or worse, your setup commands run *again*, causing subtle configuration drift or outright errors. Most people treat cloud-init’s `runcmd` like a magic box where you dump commands and they just work, always. That’s a dangerous assumption.

The subtle art of cloud-init’s execution phases

`cloud-init` isn’t just one big script; it’s a series of modules that run in specific phases. `runcmd` typically executes during the ‘config’ stage, and by design, it’s meant for tasks that run *once* on the first boot of a system. Think initial setup: creating users, fetching an SSH key, perhaps some initial package installs. The system’s identity and basic configuration are pretty much set by this point. But if you’re trying to start a long-running daemon or ensure a persistent state, `runcmd` is the wrong tool. It simply won’t re-run after a subsequent reboot, unless you force it with specific directives, which most people don’t do, and honestly, shouldn’t for general service management.

Then there’s `bootcmd`. This runs even earlier, during the ‘init’ stage, before many other services are online, and definitely before network is fully configured in many cloud environments. It’s good for really fundamental system tweaks, like messing with `/etc/fstab` or setting kernel parameters. But try to fire up your shiny new web server here and you’re gonna have a bad time. Networking won’t be ready, dependencies won’t be in place. I’ve seen folks try to cram too much into `bootcmd` thinking ‘earlier is better,’ and it just leads to silent failures or services that refuse to start because their world isn’t ready yet.

sudo cloud-init status --wait
sudo cat /var/log/cloud-init.log | grep -E 'runcmd|bootcmd|Starting Cloud-init|cloud-init finished'
sudo cloud-init analyze show

The right hammer: `systemd` units (via `write_files`)

For anything that needs to run reliably across reboots, or should act as a proper service, you shouldn’t be starting it directly from `runcmd` or `bootcmd`. That’s `systemd`’s job. Cloud-init’s superpower, especially in a cloud-native context where you want immutable or consistently configured instances, is its ability to *prepare* the system. This means dropping proper `systemd` unit files into `/etc/systemd/system/`, enabling them, and then starting them. Cloud-init handles `write_files` gracefully, allowing you to define entire files inline in your user data, including full systemd service definitions. This approach clearly separates ‘provisioning’ (cloud-init) from ‘runtime service management’ (systemd), which is how it should be.

• `runcmd` is for first-boot, one-off configuration tasks, not persistent service initiation.
• `bootcmd` is for extremely early, low-level system changes, often before networking is ready.
• For services, define a `systemd` unit file using `write_files`, then `enable` and `start` it (if needed immediately) within `runcmd`.
• Always design your `cloud-init` scripts to be idempotent – running them multiple times shouldn’t break things, even if they aren’t meant to.
• Don’t just assume; always check `/var/log/cloud-init.log` and use `cloud-init analyze show` to understand the exact execution order and timing.

Understanding this distinction is fundamental if you’re building reliable cloud infrastructure. It’s not about memorizing commands, but understanding the lifecycle of an instance, especially when you’re leaning into concepts like immutable infrastructure or instances that get regularly replaced. The simplicity of `runcmd` is appealing, but its limitations will absolutely bite you in production when you least expect it.