Cloud Engineer

HOWTOs

Concrete operational procedures from the courses on this page.

• How to migrate a relational database to Amazon RDS or Aurora with DMS at near-zero downtime

  not executable: Needs a billed AWS account with a DMS replication instance, a source database that exposes change capture, and an RDS or Aurora target reachable from that instance. No runner in the fleet has that account. LocalStack does not implement a full-load-and-CDC DMS task a reader can trust as proof of these steps, so nothing here is executed.

  Moving a live relational database onto RDS or Aurora with an AWS DMS full-load-and-CDC task: the schema and index work around the load, the CDC latency gate that decides when a cutover is safe, and the reverse replication path that keeps rollback open.

  AWS Databases

• How to restore an Amazon RDS or Aurora database to a point in time after data corruption

  not executable: Needs a billed AWS account with an RDS or Aurora source that has automated backups enabled, because restore-db-instance-to-point-in-time and restore-db-cluster-to-point-in-time create a new billed instance beside the source. No runner in the fleet has that account. A local emulator cannot prove real PITR against AWS automated backups, so nothing here is executed.

  Recovering from a bad write by restoring RDS or Aurora point-in-time recovery into a new instance beside the untouched source, validating the restore time, and choosing between a full cutover and a selective repair.

  AWS Databases

• How to contain a leaked cloud access key or compromised cloud identity

  not executable: Needs a live cloud account (AWS, Microsoft Entra, or Google Cloud) where the leaked key or compromised identity still exists, with pre-incident audit logs and a break-glass responder principal that can disable credentials and revoke sessions. No runner in the fleet holds a tenant under an active credential incident, and a local emulator cannot prove real disable, session revoke, or persistence cleanup against a provider control plane, so nothing here is executed.

  Disabling the credential, revoking the temporary sessions it already started, and scoping the principals, identity providers, and resources an attacker may have created for persistence, across AWS, Microsoft Entra, and Google Cloud.

  Cloud Security

• How to scope and contain a compromised cloud compute instance

  not executable: Needs a live cloud account with a running compute instance under investigation (EC2, Azure VM, or Compute Engine), same-region forensics capacity for memory and disk capture, and authority to isolate the instance and revoke its cloud credentials without destroying evidence. No runner in the fleet hosts that incident context, and a local VM cannot prove provider network isolation or instance-role session revoke, so nothing here is executed.

  Isolating one compromised virtual machine without destroying evidence: the order of memory capture, disk snapshots, network isolation, and revoking the instance's own cloud credentials, then rebuilding from a known-good image rather than cleaning in place.

  Cloud Security

• How to rotate the Consul gossip encryption key

  verified on linux-amd64-proxmox, amd64, 2026-10-18, sha256:f635f01f2426dc7d8c5153a0bb556f1e585e8d27004bfa0b25cbac3dd61e4913

  Replacing the shared Serf gossip key on a running datacenter without partitioning any agent, and what changes when the datacenters are WAN-federated.

  HashiCorp Consul

• How to restore a Consul cluster from a Raft snapshot after quorum loss

  verified on linux-amd64-proxmox, amd64, 2026-10-18, sha256:f635f01f2426dc7d8c5153a0bb556f1e585e8d27004bfa0b25cbac3dd61e4913

  Recovering a Consul server cluster that has lost quorum, and where the peers.json Raft rebuild ends and a consul snapshot restore begins.

  HashiCorp Consul

• How to bootstrap ACLs on a running Consul cluster without an outage

  verified on linux-amd64-proxmox, amd64, 2026-10-18, sha256:f635f01f2426dc7d8c5153a0bb556f1e585e8d27004bfa0b25cbac3dd61e4913

  Turning on the Consul ACL system on a datacenter that has none without a service-discovery outage, and why the permissive-first, deny-last order matters.

  HashiCorp Consul

• How to upgrade a production Kubernetes cluster one minor version

  Moving a self-managed kubeadm cluster forward one Kubernetes minor version, including the version-skew rules and pre-flight checks that decide whether the upgrade can proceed at all.

  Kubernetes Operations

• How to drain and replace a worker node safely

  verified on k8s-kubeadm-1-37-proxmox, amd64, 2026-10-28, sha256:9eb3a768197140c54046d712b1e7d28449c3696c4fe88ff7df6c0ed26b0b733b

  Taking a worker node out of service for disruptive work or retirement without breaching any workload's availability guarantee, and returning it or removing it cleanly afterward.

  Kubernetes Operations

• How to renew expiring kubeadm control-plane certificates

  verified on k8s-kubeadm-1-37-proxmox, amd64, 2026-10-28, sha256:9eb3a768197140c54046d712b1e7d28449c3696c4fe88ff7df6c0ed26b0b733b

  Renewing the one-year kubeadm control-plane certificates before they expire, and the offline recovery path for a cluster whose certificates have already lapsed and whose API server is down.

  Kubernetes Operations

• How to rebuild a Kubernetes control plane from an etcd snapshot

  The last-resort recovery when etcd data is lost or corrupt beyond quorum: replacing entire cluster state from a snapshot, with the data-loss and reconciliation consequences that come with it.

  Kubernetes Operations

• How to migrate deprecated Kubernetes API usage before an upgrade

  verified on k8s-kubeadm-1-37-proxmox, amd64, 2026-10-28, sha256:9eb3a768197140c54046d712b1e7d28449c3696c4fe88ff7df6c0ed26b0b733b

  Clearing an upgrade blocker by locating every client, manifest, and controller still calling a Kubernetes API version the next minor removes, and proving the usage is gone before the upgrade.

  Kubernetes Operations