Operating Containers Safely & the Kubernetes Handoff

Table of Contents#

1. From Running to Operating
2. Capabilities: Dropping Root's Superpowers
3. Seccomp and AppArmor: Filtering the Kernel Surface
4. Rootless Docker
5. Read-Only Filesystems and Immutable Containers
6. A Hardened Baseline Configuration
7. Resource Governance at Scale
8. cgroups v1 vs. v2: Why Resource Limits Sometimes Behave Differently
9. Logging Drivers and Centralized Collection
10. Health Checks as an Operating Contract
11. Secrets and Runtime Configuration
12. Auditing a Host with Docker Bench for Security
13. Applying the Hardening Baseline in Compose
14. Graceful Shutdown and Connection Draining in Practice
15. Image Update Strategy in Production
16. Shared-Host Multi-Tenancy and Blast Radius
17. Runtime Anomaly Detection
18. Monitoring a Docker Host in Production
19. CVE Response for Already-Running Containers
20. Backup, Upgrade, and Host Maintenance
21. Disaster Recovery for a Single Host
22. Rolling Out Hardening Changes Safely
23. An Incident, End to End
24. The Limits of a Single Docker Host
25. Signals That Mean It's Time for an Orchestrator
26. What Actually Changes Moving to Kubernetes
27. A Note on Terminology Before Migrating
28. A Deliberate Migration Path
29. Common Mistakes and Interview Traps
30. Worked Practice Problems
31. Summary and Series Wrap-Up

From Running to Operating#

The first three parts of this series built the pieces: an image with a clear lifecycle, a reproducible and auditable build pipeline, and a multi-service stack wired together with networking, storage, and Compose. This chapter asks the question every one of those pieces eventually faces in production: what happens when something goes wrong, who notices, and how much damage can a single compromised or misbehaving container actually do?

The theme threading through this chapter is the same one from Part 1's incident workflow: a container is a process with kernel-enforced boundaries, and every hardening control here narrows those boundaries deliberately, on purpose, for a specific threat this workload actually faces — not as a checklist applied uniformly regardless of what the container does. A batch job with no network needs and a public-facing API with a large attack surface deserve genuinely different postures, and treating them identically either over-restricts the batch job for no benefit or under-restricts the API for a false sense of parity.

From the Trenches: A post-incident review of a container escape attempt found the compromised process ran as root inside the container, retained the full default capability set, had no seccomp restriction beyond Docker's already-permissive default profile, and ran on a writable root filesystem. None of those four conditions alone would have been fatal; together, they gave the attacker everything needed to attempt a kernel-level privilege escalation. The remediation wasn't one silver-bullet control — it was layering four independent, individually modest restrictions that each closed one part of the same door.

Capabilities: Dropping Root's Superpowers#

Linux capabilities split root's traditionally all-or-nothing privilege into roughly 40 discrete units — CAP_NET_BIND_SERVICE to bind a privileged port, CAP_SYS_ADMIN (a notorious catch-all worth treating with particular suspicion), CAP_CHOWN, CAP_NET_RAW, and dozens more. Docker grants a default subset to every container, tuned for broad compatibility rather than minimal privilege.

docker run --rm --cap-drop ALL --cap-add NET_BIND_SERVICE nginx:1.27

Dropping every capability and adding back only what's demonstrably needed inverts Docker's default posture from "permissive unless restricted" to "denied unless justified." For most application containers — anything that isn't managing its own network interfaces, mounting filesystems, or manipulating other processes — an empty or near-empty capability set works without any functional loss.

docker run --rm --cap-drop ALL alpine:3.21 sh -c 'id; cat /proc/self/status | grep Cap'

/proc/self/status's CapEff field is the fastest way to confirm exactly what a running container actually holds, rather than trusting the docker run flags alone — useful when a base image's entrypoint script itself adjusts capabilities before the application starts.

From the Trenches: A team requested CAP_SYS_ADMIN for a service because a dependency's installation instructions mentioned it as a workaround for a permissions error during a specific setup step, not because the running application actually needed it. Nobody had verified the claim before it became a standing production configuration. Testing with the capability dropped after the fact showed the application ran identically without it — the original error had actually been a misconfigured file ownership issue that CAP_CHOWN alone resolved, several orders of magnitude less risky than the catch-all capability that had been granted instead.

Seccomp and AppArmor: Filtering the Kernel Surface#

Capabilities control what a privileged action can do; seccomp controls which system calls a process can make at all, privileged or not. Docker's default seccomp profile already blocks around 44 rarely-needed and historically dangerous syscalls (like keyctl and various obscure namespace operations) without requiring any explicit configuration — a custom profile narrows this further for a specific workload's actual syscall footprint.

docker run --rm --security-opt seccomp=./api-seccomp.json catalog-api:2.6.0

{
  "defaultAction": "SCMP_ACT_ERRNO",
  "architectures": ["SCMP_ARCH_X86_64"],
  "syscalls": [
    { "names": ["read", "write", "openat", "close", "epoll_wait", "futex", "mmap", "brk"], "action": "SCMP_ACT_ALLOW" }
  ]
}

A hand-written allowlist like the abbreviated example above is fragile in practice — most real applications need dozens of syscalls, and hand-maintaining the list against every language runtime's actual behavior is a significant undertaking. The practical path is generating a profile from observed behavior (tracing a workload's actual syscalls under realistic load with a tool built for the purpose) rather than authoring one from a general syscall reference and hoping it's complete.

AppArmor (or SELinux, depending on the host distribution) works at a different layer: mandatory access control over filesystem paths, network operations, and capability use, enforced by the kernel regardless of what the container process believes its own privileges are.

docker run --rm --security-opt apparmor=docker-default nginx:1.27

The three layers are complementary specifically because none of them assume the others exist — a process root inside a container's user namespace can be seccomp-filtered even though it "looks like root," and can be AppArmor-confined even though it retains a capability that AppArmor's policy simply doesn't authorize for a specific path.

From the Trenches: A vulnerability researcher's proof-of-concept container escape relied on a syscall sequence blocked by Docker's default seccomp profile — the finding was real, but only reproducible on hosts where the default profile had been disabled entirely (--security-opt seccomp=unconfined) for an unrelated debugging session and never re-enabled. The lesson generalized: audit for security controls disabled "temporarily" for a debugging session that outlived the debugging need, not just for controls that were never configured in the first place.

Rootless Docker#

Every control so far restricts what a container process can do; rootless Docker restricts what the daemon itself can do, by running dockerd entirely under an unprivileged user account via Linux user namespaces, rather than as the traditional root-owned daemon.

dockerd-rootless-setuptool.sh install
systemctl --user start docker
docker context use rootless

In rootless mode, a container escape lands the attacker as the invoking unprivileged user on the host, not as root — a meaningfully smaller blast radius than the traditional model, where the daemon (and therefore anything that escapes a container it manages) has full host root by default.

From the Trenches: A platform team piloting rootless Docker on shared build infrastructure found their existing CI images assumed binding to port 80 directly — a pattern that had worked silently under root-mode Docker for years. Rather than treating this as a rootless limitation to work around, the team used it as forcing function to fix a design smell that predated rootless mode entirely: no application container should bind a privileged port directly in the first place, and a reverse proxy or load balancer in front of it (already a near-universal production pattern) makes the port number a proxy-layer concern, not an application one.

Read-Only Filesystems and Immutable Containers#

A container that cannot write to its own root filesystem cannot be used to persist a webshell, modify a binary in place, or plant a backdoor that survives a restart — because there's nowhere on that filesystem for such a change to land.

docker run -d --name catalog-api \
  --read-only \
  --tmpfs /tmp:rw,noexec,nosuid,size=64m \
  --tmpfs /var/run:rw,noexec,nosuid,size=16m \
  catalog-api:2.6.0

Most applications need some writable path — a cache directory, a PID file, a temporary upload buffer — which is exactly what the earlier tmpfs mounts provide: writable, but explicitly scoped, size-limited, and non-executable (noexec), so even a successful write can't be used to execute new code from that path.

Testing --read-only against a real workload surfaces every implicit write assumption the application makes — package manager caches, log files written to a local path instead of stdout, a framework's default temp-file location — each of which needs either a tmpfs mount or (better, per Part 1's observability guidance) a fix to write structured logs to stdout instead of a file at all.

From the Trenches: Enabling --read-only on a service in staging immediately broke it, with an error referencing a missing writable path the team hadn't known the application used — a bundled dependency wrote a lock file to its own installation directory on every startup, a detail invisible in the application's own documentation. Rather than reverting to a writable filesystem, the fix was a narrowly scoped tmpfs mount at exactly that path, preserving the read-only guarantee everywhere else while accommodating the one legitimate write the application actually needed.

A Hardened Baseline Configuration#

Combining every control from this chapter into one deliberate baseline, rather than treating them as an unordered checklist, produces a configuration a team can review, replicate, and, most importantly, articulate the reasoning behind rather than following a template blindly.

docker run -d \
  --name catalog-api \
  --network app-net \
  --read-only \
  --tmpfs /tmp:rw,noexec,nosuid,size=64m \
  --cap-drop ALL \
  --cap-add NET_BIND_SERVICE \
  --security-opt no-new-privileges:true \
  --security-opt apparmor=docker-default \
  --user 10001:10001 \
  --pids-limit 256 \
  --memory 512m --memory-reservation 384m --cpus 1.0 \
  --restart unless-stopped \
  catalog-api:2.6.0

services:
  api:
    image: catalog-api:2.6.0
    read_only: true
    tmpfs:
      - /tmp:rw,noexec,nosuid,size=64m
    cap_drop: [ALL]
    cap_add: [NET_BIND_SERVICE]
    security_opt:
      - no-new-privileges:true
      - apparmor=docker-default
    user: "10001:10001"
    pids_limit: 256
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 512M
    restart: unless-stopped

no-new-privileges:true is a control this chapter hasn't named yet but belongs in every baseline: it prevents a process from gaining additional privileges through a setuid binary or file capability, even if the container's own filesystem contains one — closing a specific escalation path that capability-dropping alone doesn't address, since a setuid binary's privilege comes from the filesystem, not from the container's granted capability set.

Every line in this baseline maps directly back to a specific threat from earlier in the chapter: --cap-drop ALL/--cap-add narrows privileged operations to the one genuinely needed; --read-only/tmpfs removes persistence for a filesystem-based attack; --user avoids running as container root at all; --pids-limit bounds a fork-bomb-style resource exhaustion; and the resource limits bound a memory or CPU runaway from starving neighbors on the same host.

Resource Governance at Scale#

Part 1 introduced per-container resource limits for a single service. Operating many containers on one host requires thinking about resource governance as a fleet-wide property, not a per-container afterthought.

docker run -d --name batch-worker --cpus 0.5 --cpu-shares 512 --memory 256m worker:1.4
docker run -d --name api --cpus 2.0 --cpu-shares 2048 --memory 1g catalog-api:2.6.0

--cpus sets a hard ceiling (this container can never use more than N CPUs' worth of time, even if the host is otherwise idle); --cpu-shares sets a relative weight used only when the host is under CPU contention — a higher-share container gets proportionally more CPU time during contention, but both containers can burst above their share when the host has spare capacity. Confusing the two is a common source of "why is this batch job throttled even though the host looks idle" (usually an overly tight --cpus ceiling) versus "why does this low-priority job starve the API during a spike" (usually a missing or too-low --cpu-shares weighting).

From the Trenches: A batch reporting job and a customer-facing API shared a host with no CPU shares configured on either — both defaulted to equal weight. A weekly report run, entirely CPU-bound and otherwise harmless, coincided with a traffic spike and measurably degraded API latency for the duration of the report, because the scheduler had no signal that one workload mattered more under contention than the other. Setting the API's --cpu-shares several times higher than the batch job's fixed the symptom without touching either container's hard --cpus ceiling — during contention, the scheduler now favored the API automatically; when the host was idle, both could still burst freely.

Disk I/O throttling#

CPU and memory limits don't bound disk I/O — a container performing heavy sequential writes (a log-shipping sidecar under load, a batch job writing large temporary files) can saturate the host's disk throughput and degrade every other container's I/O latency, even when its own CPU and memory usage look completely unremarkable.

docker run -d --name batch-export --device-write-bps /dev/sda:50mb --device-read-bps /dev/sda:100mb batch-export:1.0

--device-write-bps/--device-read-bps cap a container's I/O throughput against a specific block device, the same hard-ceiling model --cpus applies to CPU time. This control is used far less consistently than CPU and memory limits in practice, largely because disk I/O contention is less visible in routine monitoring than CPU or memory pressure — until a batch job's I/O pattern degrades an unrelated service's database write latency and the connection between the two takes real investigation to find.

From the Trenches: A nightly data-export job's disk writes were never rate-limited, on the reasoning that "it only runs for twenty minutes, it's not worth the configuration." A separate database container sharing the same underlying disk periodically showed elevated write latency during exactly that twenty-minute window, months before anyone connected the two — the export job's own metrics looked fine (it wasn't CPU- or memory-bound), so it was never the first suspect. Adding a conservative --device-write-bps ceiling to the export job, discovered only after correlating the database's latency graph against the export job's cron schedule, resolved it without slowing the export job in any way its own SLA cared about.

cgroups v1 vs. v2: Why Resource Limits Sometimes Behave Differently#

Every resource limit this chapter covers — --memory, --cpus, --pids-limit, device I/O throttling — is ultimately enforced through Linux control groups, and which cgroup version a host runs changes some of their exact enforcement behavior. Most current Linux distributions default to cgroups v2, but a host provisioned from an older image, or a specific container runtime configuration, can still be running v1 — a detail worth confirming explicitly rather than assuming, since diagnosing a resource-limit discrepancy against the wrong mental model wastes real incident time.

docker info --format '{{.CgroupVersion}}'
cat /sys/fs/cgroup/cgroup.controllers 2>/dev/null && echo "v2 unified hierarchy" || echo "likely v1"

The practical consequence for this chapter's guidance: a memory-pressure or throttling investigation that reads /sys/fs/cgroup/memory.pressure will find nothing on a v1 host, because that file simply doesn't exist under v1's separate-hierarchy model — the equivalent v1 signal is coarser and lives under a different, controller-specific path. Confirming the cgroup version first, before trusting a specific diagnostic path from a runbook or blog post, avoids a wasted investigation cycle chasing a file that was never going to exist on that host.

From the Trenches: An on-call engineer, following a runbook written against a cgroups v2 host, spent twenty minutes convinced that memory pressure reporting was "broken" on a production host that had simply never been migrated off cgroups v1 during a prior OS upgrade — the exact file the runbook referenced didn't exist, and no error explained why. The actual fix took two minutes once someone confirmed the cgroup version explicitly; the twenty minutes lost were entirely attributable to an unstated assumption baked into the runbook that every host in the fleet had already migrated, which turned out to be false for a subset provisioned earlier.

This same version awareness matters again during a Kubernetes migration later in this chapter: a cluster's nodes need a consistent, current cgroup configuration for the kubelet's own resource-management features to behave as documented, so auditing host cgroup versions is worth doing once, deliberately, before a migration rather than discovering the inconsistency node by node afterward.

Logging Drivers and Centralized Collection#

Part 1 established that containers should write to stdout/stderr, not a local log file, and that docker logs is a one-host diagnostic tool, not a production log store. This chapter covers the mechanism that turns stdout into a centrally collected, retained log stream.

{
  "log-driver": "local",
  "log-opts": {
    "max-size": "20m",
    "max-file": "5"
  }
}

Setting this in /etc/docker/daemon.json establishes a safe fleet-wide default — Docker's historical default, unbounded json-file, can genuinely fill a host's disk over time if nothing ever rotates it, which has caused real production outages having nothing to do with the application itself. The local driver is Docker's own recommended default going forward: a more compact binary format than json-file, with rotation enabled out of the box, while still supporting docker logs transparently.

docker run -d --name api --log-driver local --log-opts max-size=10m --log-opts max-file=3 catalog-api:2.6.0
docker run -d --name api --log-driver journald catalog-api:2.6.0
docker run -d --name api --log-driver fluentd --log-opt fluentd-address=localhost:24224 catalog-api:2.6.0

From the Trenches: A host ran out of disk overnight and every container on it began failing simultaneously, unrelated to any application code — the root cause traced back to the still-default json-file driver on a host provisioned before the team's daemon.json hardening baseline existed, with several long-running containers having accumulated tens of gigabytes of unrotated logs from a verbose debug-level logger nobody had noticed. The incident's actual fix took thirty seconds (setting max-size/max-file); the hour spent finding the root cause was entirely attributable to nobody having checked log driver configuration as part of the standard incident triage steps from Part 1.

Health Checks as an Operating Contract#

Part 3 used service_healthy to gate Compose startup ordering. In production, a health check is also the input that determines whether a container gets restarted, and, once an orchestrator is in the picture, whether it receives traffic at all — which means a poorly designed health check can cause exactly the outages it's meant to prevent.

HEALTHCHECK --interval=15s --timeout=3s --start-period=30s --retries=3 \
  CMD wget --spider --quiet http://127.0.0.1:8080/healthz || exit 1

The critical design decision is what /healthz actually verifies. A liveness-style check ("is the process able to respond at all") should stay cheap and dependency-free — it exists to catch a genuinely wedged process, not to fail every replica the moment a downstream dependency has a bad minute. A readiness-style check ("can this instance currently serve real traffic correctly") should verify the specific dependencies that request handling actually needs.

From the Trenches: A service's single HEALTHCHECK ran a full database query as its test. A database maintenance window that added a few seconds of latency to every query caused every replica of the service to simultaneously fail its health check and restart — converting a minor, expected database latency blip into a full service outage, because "the database is briefly slow" and "this process is dead and needs to be killed" were treated as the same signal. Splitting the check into a cheap liveness probe (process responds at all) and a separate, differently-consequenced readiness signal prevented the same maintenance window from causing an outage the next time it happened.

Secrets and Runtime Configuration#

Part 2 covered build-time secrets and Part 3 covered Compose's secrets: file mounts. At runtime on a single host, the same principle — a mounted file over an inspectable environment variable — still applies, with one addition worth naming explicitly for production: rotation.

docker run -d --name api \
  -v /run/secrets/db-credential:/run/secrets/db-credential:ro \
  catalog-api:2.6.0

An application that reads a credential once at startup and holds it in memory indefinitely can't benefit from a rotated file without a restart — which means a genuinely zero-downtime credential rotation needs the application to either re-read the file periodically or receive an explicit reload signal, a requirement that's easy to overlook until an incident forces an emergency credential rotation and the team discovers every replica needs a coordinated restart to pick it up.

Auditing a Host with Docker Bench for Security#

Rather than manually re-verifying every control in this chapter across every host, Docker Bench for Security runs a standardized set of checks derived from the CIS Docker Benchmark — daemon configuration, container runtime configuration, and image/build practices — and reports pass/fail against each.

docker run --rm --net host --pid host --userns host --cap-add audit_control \
  -v /etc:/etc:ro -v /var/lib/docker:/var/lib/docker:ro -v /var/run/docker.sock:/var/run/docker.sock:ro \
  docker/docker-bench-security

The tool itself needs unusually broad access to the host to inspect daemon and container configuration accurately — worth noting explicitly, since it's a legitimate exception to this chapter's "narrow every permission" theme, justified by the tool's specific, well-understood, read-mostly purpose rather than a general excuse to grant broad access elsewhere.

Running this as a recurring, scheduled audit — not a one-time check before a compliance review — catches configuration drift as new services and hosts are added, which is exactly how the disk-exhaustion and seccomp-disabled incidents earlier in this chapter each went undetected for as long as they did: a control that was correct once but never re-verified.

From the Trenches: A quarterly compliance audit consistently passed a Docker Bench scan run against a golden host image, while three production hosts provisioned outside the standard pipeline (created during an incident response, when time pressure led someone to skip the usual provisioning script) had drifted from that baseline in ways nobody had checked. Scheduling the scan to run against every host continuously, rather than against a single reference image before an audit, surfaced the drifted hosts within a day of the next scheduled run.

Applying the Hardening Baseline in Compose#

Part 3's Compose examples focused on service wiring, not the hardening posture from this chapter — applying both together to the same stack is what a real production Compose file actually looks like.

services:
  api:
    image: catalog-api:2.6.0
    read_only: true
    tmpfs:
      - /tmp:rw,noexec,nosuid,size=64m
    cap_drop: [ALL]
    cap_add: [NET_BIND_SERVICE]
    security_opt:
      - no-new-privileges:true
    user: "10001:10001"
    pids_limit: 256
    healthcheck:
      test: ["CMD", "wget", "--spider", "-q", "http://127.0.0.1:8080/healthz"]
      interval: 15s
      timeout: 3s
      retries: 3
      start_period: 30s
    logging:
      driver: local
      options:
        max-size: "20m"
        max-file: "5"
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 512M
    networks: [app-net]

  db:
    image: postgres:16
    read_only: true
    tmpfs:
      - /tmp:rw,noexec,nosuid,size=64m
      - /run/postgresql:rw,noexec,nosuid,size=16m
    volumes:
      - pgdata:/var/lib/postgresql/data
    cap_drop: [ALL]
    cap_add: [CHOWN, SETUID, SETGID, DAC_OVERRIDE, FOWNER]
    security_opt:
      - no-new-privileges:true
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d catalog"]
      interval: 5s
      timeout: 3s
      retries: 10
    networks: [app-net]

The db service needs a slightly larger capability set than api — Postgres's own startup process performs filesystem ownership operations (CAP_CHOWN, CAP_FOWNER) that a stateless application typically doesn't. This is the "threat-specific, not uniform" principle from this chapter's opening section made concrete: the two services share the same hardening pattern, but not an identical capability list, because they genuinely need different things — and arriving at that specific list required testing the container with --cap-drop ALL and adding back only what startup actually failed without, rather than guessing a Postgres-specific list from documentation alone.

Graceful Shutdown and Connection Draining in Practice#

Part 1 introduced SIGTERM/SIGKILL and the stop-timeout deadline. In production, whether that grace period is actually usable depends on details easy to get wrong even with the mechanism nominally in place.

docker stop --time 30 catalog-api

STOPSIGNAL SIGTERM

Two failure modes account for most "graceful shutdown didn't actually work" incidents. First, PID 1 signal handling: if a container's entrypoint is a shell script (CMD ["sh", "-c", "node server.js"]) rather than the application process directly, the shell — not the application — receives SIGTERM as PID 1, and many shells don't forward signals to child processes by default, so the application never learns it should start draining. Using exec in the entrypoint script (exec node server.js), or invoking the binary directly in exec form (CMD ["node", "server.js"]), ensures the application itself is PID 1 and receives the signal.

Second, the stop timeout must exceed the application's actual drain time, measured under realistic load, not assumed. A service that takes 45 seconds to drain in-flight requests under peak load but has a 10-second stop timeout gets SIGKILLed mid-drain every time, silently dropping the in-flight requests the graceful shutdown logic was specifically written to protect.

From the Trenches: A deployment tool reported "zero-downtime" rolling restarts for months based only on the absence of a visible error in its own logs, while a small but consistent trickle of client-reported dropped connections went uninvestigated as "occasional network noise." The actual cause was a CMD ["sh", "-c", "..."] entrypoint pattern across the whole fleet — SIGTERM never reached the application processes at all, and every "graceful" restart was in practice a hard kill after the full stop-timeout elapsed, with genuinely in-flight requests dropped every single time. Switching every service's entrypoint to exec form fixed the drop rate immediately, and the incident's lasting change was adding an explicit exec-form entrypoint check to the team's Dockerfile linting from Part 2.

Image Update Strategy in Production#

Part 2 covered scheduled base-image rebuilds for CVE hygiene at build time; this section covers the separate question of how a running host actually picks up a new image once one is pushed.

An automated watcher polling for "any new tag" and redeploying unconditionally reintroduces exactly the problem Part 2's promotion discipline was built to prevent — a floating tag being redeployed the moment it moves, with no guarantee the new digest passed scanning, signing, or staging validation. The CI-driven approach is the one that actually preserves this series' supply-chain guarantees end to end: nothing reaches a running host that didn't first pass through the pipeline that verified it.

From the Trenches: A team ran an automated image-watcher against their :latest tag "for convenience," reasoning that always running the newest build was inherently safer than running a stale one. A broken build that failed its test suite, but was pushed to a registry before the CI pipeline's failure was noticed, was picked up and deployed to production automatically within minutes — the watcher had no concept of "verified" versus "merely pushed." The fix was removing the watcher entirely and routing all deployments through the CI pipeline's own final apply step, so nothing reached production that hadn't passed the same gates a human-triggered deploy would have required.

Shared-Host Multi-Tenancy and Blast Radius#

A single Docker host running more than one application — common for a small team consolidating several low-traffic services onto one box for cost reasons — introduces a blast-radius question none of this chapter's per-container controls fully answer on their own: how much can one tenant's misbehaving or compromised container affect another's, when they share the same daemon, kernel, and physical resources.

docker run -d --name app-a-worker --cpus 1.0 --memory 512m --network app-a-net app-a:1.0
docker run -d --name app-b-api --cpus 1.0 --memory 512m --network app-b-net app-b:2.0

Per-container --cpus/--memory limits (from Resource Governance earlier) prevent one tenant from starving another's CPU or memory outright, and per-tenant user-defined networks (from Part 3's segmentation guidance) prevent one tenant's containers from reaching another's over the network. Neither control, by itself, addresses a kernel-level compromise — a container escape from tenant A's workload lands on the shared host kernel, at which point tenant B's containers are only protected by the same host-wide hardening (seccomp, AppArmor, non-root daemon via rootless mode) this chapter has already covered, not by any tenant-specific boundary.

For genuinely untrusted multi-tenant workloads — arbitrary user-submitted code, not simply "several internal teams' own services" — plain Docker's isolation model is not the right tool regardless of how thoroughly this chapter's controls are applied; that's a signal for a stronger isolation boundary (a gVisor/Kata-style sandboxed runtime, or genuinely separate hosts/VMs per tenant) rather than a more aggressive Docker hardening configuration.

From the Trenches: A team consolidated five internal services from five separate teams onto one Docker host to cut infrastructure cost, reasoning that per-container resource limits and networks provided "enough" isolation between them. A dependency vulnerability in one team's service, exploited in a low-severity way that would have been contained to that one service on a dedicated host, instead gave the attacker a foothold on the shared kernel — and the incident review's most uncomfortable finding was that the other four teams' services had never been threat-modeled against "what if a different team's container is compromised," because nobody had explicitly decided the host was a multi-tenant trust boundary in the first place. The corrective action split the services back across separate hosts for the two with materially different risk profiles, while leaving the three lowest-risk, internally-facing services consolidated with an explicit, documented acceptance of the shared-kernel risk.

Runtime Anomaly Detection#

Every control so far in this chapter is preventive — it stops a class of action before it happens. Runtime anomaly detection (Falco is the most widely deployed open-source example) is detective instead: it observes actual container behavior via kernel-level instrumentation and alerts when something deviates from an expected pattern, catching exactly the case where a preventive control was missing, misconfigured, or bypassed.

- rule: Unexpected shell spawned in container
  desc: A shell was spawned inside a container that shouldn't normally have one
  condition: >
    spawned_process and container and
    proc.name in (bash, sh, zsh) and
    not container.image.repository in (debug-allowed-images)
  output: >
    Shell spawned in container (user=%user.name container=%container.name
    image=%container.image.repository command=%proc.cmdline)
  priority: WARNING

This rule directly complements the distroless-base-image guidance from Part 2: a genuinely shell-less image should never trigger it at all, so an alert firing on a supposedly shell-less container is a strong, specific signal that something is either misconfigured or actively compromised — a shell was spawned somewhere it structurally shouldn't be possible for one to exist.

Detective controls don't replace preventive ones — they exist specifically to catch the gap between "we believe every control is correctly configured everywhere" and reality, which the Docker Bench and seccomp-disabled trenches stories earlier in this chapter both illustrate as a real, recurring gap rather than a hypothetical one.

External secret managers vs. mounted files#

The mounted-file approach above works well for a small, relatively static set of credentials on one host. A larger fleet, or a compliance requirement for centralized audit logging of every secret access, usually calls for an external secret manager (HashiCorp Vault, AWS Secrets Manager, or equivalent) instead.

docker run -d --name api \
  -v /run/secrets:/run/secrets:ro \
  --entrypoint /usr/local/bin/vault-agent-wrapper \
  catalog-api:2.6.0

A sidecar or init pattern (an agent process that authenticates to the secret manager, fetches the credential, and writes it to a location the main container reads, or injects it as an environment variable at process-start time) keeps the application itself unaware of which backend actually holds the secret — the same /run/secrets-style file path this chapter has used throughout, just populated by a fetching agent instead of a static Compose-mounted file.

Neither is universally correct — a small single-host deployment gains little from the operational overhead of running a secret manager, while a compliance-driven or larger fleet often can't satisfy its audit requirements without one.

Monitoring a Docker Host in Production#

Beyond per-container health checks, a production Docker host needs host-level and fleet-level visibility that no single container's health check can provide.

docker stats --no-stream --format 'table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}\t{{.PIDs}}'
docker system df
docker events --since 1h --filter event=die --filter event=oom

docker events filtered for die and oom surfaces exactly the signals an on-call engineer needs first during an incident — which containers exited, and whether the kernel OOM-killed any of them — without manually correlating timestamps across many containers' individual logs. A production setup exports these as metrics (container restart count, OOM-kill count, per-container CPU/memory against limit) into the same observability stack the rest of the organization already uses, rather than treating docker stats as anything beyond an ad-hoc debugging tool.

CVE Response for Already-Running Containers#

Part 2 covered answering "does any built image contain this vulnerable library" from stored SBOM attestations. Operating containers adds a second, time-sensitive question during a real CVE response: which of the containers currently running right now on this host are affected, before any rebuild or redeploy has happened.

docker container ls --format '{{.Names}}\t{{.Image}}' | while read -r name image; do
  digest=$(docker container inspect "$name" --format '{{.Image}}')
  echo "$name -> $image ($digest)"
done

trivy image --severity CRITICAL,HIGH "$(docker container inspect catalog-api --format '{{.Image}}')"

Enumerating every running container's exact image reference — not the tag it was started with, but the resolved digest it's actually running — and checking each against the new CVE is the fastest way to produce an accurate "here's what's affected right now" answer without waiting for a fleet-wide rebuild to even start. This is a different exercise from Part 2's build-time SBOM query: it answers "what's live," not "what was ever built," and it's the number an incident commander actually needs first.

From the Trenches: During a widely publicized CVE affecting a common base-image component, a team's first response was to grep their Dockerfiles for the affected package — which correctly identified every build definition using it, but missed that two long-running containers on a legacy host were still running images built from a Dockerfile version predating a change that had removed the dependency. Enumerating actually-running image digests directly, rather than trusting the current Dockerfile as a proxy for what's deployed, caught the two stale containers that the source-level grep had no way to see.

Backup, Upgrade, and Host Maintenance#

A Docker host itself needs a maintenance discipline distinct from any single container's lifecycle — Docker Engine version upgrades, kernel updates, and the underlying OS's own patch cycle all interact with running containers in ways worth planning for deliberately.

docker system prune -a --filter "until=168h"
apt list --upgradable | grep docker-ce

Engine upgrades occasionally introduce breaking changes to defaults (a new seccomp default, a changed log driver default) — reading the release notes before upgrading a production host, not just running the upgrade and checking whether things still start, catches this class of surprise before it becomes an incident. A scheduled docker system prune with an age filter (until=168h — a week) reclaims space from genuinely stale, unreferenced images and containers without the aggressive, undiscriminating cleanup of a bare prune -a run with no age filter at all.

From the Trenches: A routine Docker Engine minor-version upgrade, applied without reading the release notes because "it's just a minor version," changed a default that altered how a specific --log-opt was interpreted, silently disabling log rotation for every container that relied on the old default rather than an explicit override. The disk-exhaustion incident from earlier in this chapter happened a second time, on a different host, for a subtly different root cause — and the second incident's corrective action, finally, was making "read the release notes for anything touching logging, security defaults, or networking" a mandatory pre-upgrade checklist item rather than an assumed best practice nobody enforced.

Disaster Recovery for a Single Host#

A single Docker host is a legitimate architecture (per the next section's framing), but only if its failure mode is planned for deliberately, not discovered during an actual outage. Disaster recovery for one host has a narrower scope than a multi-host orchestrator's built-in failover, but it's still a real, testable process.

# Provisioning script, run on a fresh replacement host
docker network create app-net
docker volume create pgdata
aws s3 cp s3://backups/pgdata-latest.tar.gz .
docker run --rm -v pgdata:/data -v "$(pwd):/backup" alpine:3.21 \
  tar xzf /backup/pgdata-latest.tar.gz -C /data
docker compose up -d

The elements that make this actually work when it's needed, rather than merely existing as an aspirational script nobody has run: the provisioning steps are themselves version-controlled and tested regularly (not just written once and trusted), the backup referenced is one that's been through a real restore test per Part 3's backup guidance, and the Compose file being applied is the exact same one already running in production — not a separately maintained "disaster recovery" variant that has quietly drifted from what's actually deployed.

From the Trenches: A team's disaster recovery plan for their single-host production deployment was a wiki page describing manual steps last verified two years earlier, against a Compose file that had since gained three new services. When a host actually needed replacement, the on-call engineer discovered the plan referenced a volume name that had been renamed in a refactor eighteen months prior, and the actual recovery took four hours of live troubleshooting instead of the twenty minutes the (untested) plan implied. The corrective action was converting the wiki page into an actual script, checked into the same repository as the Compose file, exercised on a schedule against a disposable test host — not trusted as documentation alone.

Rolling Out Hardening Changes Safely#

Applying this chapter's controls to an already-running fleet is itself an operational change that deserves the same caution as any production deployment — a capability drop or read-only filesystem change that seems obviously safe in theory can still break an application that relies on undocumented behavior nobody remembered.

1. Test in a non-production environment first, ideally against real traffic patterns (a staging environment receiving mirrored or replayed production traffic, not just a synthetic smoke test) — the read-only filesystem trenches story earlier in this chapter is exactly the kind of surprise that only surfaces under a workload's actual behavior, not a cursory manual check.
2. Roll out to one replica or one host before the whole fleet. If the service runs multiple replicas behind a load balancer, apply the hardened configuration to a single replica, monitor its error rate and latency against its siblings for a meaningful window, and only proceed once it's confirmed equivalent.
3. Change one control at a time when introducing several at once for the first time. Applying --cap-drop ALL, --read-only, and a custom seccomp profile simultaneously and hitting a failure gives no signal about which control caused it — introduce them incrementally at least for the first rollout, even if the steady-state target configuration applies all of them together.
4. Keep a fast, well-understood rollback path. Because these are runtime flags (or Compose fields) rather than application code changes, rolling back is usually a redeploy with the previous configuration — but only if that previous configuration is still readily available and the redeploy process itself is fast, not a multi-step manual procedure that adds its own delay during an active incident.

From the Trenches: A team rolled out --cap-drop ALL --read-only --security-opt seccomp=custom.json to their entire fleet simultaneously during a single maintenance window, confident because each control had been individually tested weeks apart in staging. One service failed in production in a way none of the individual staging tests had caught — a specific combination where a library's fallback behavior under a denied syscall triggered a code path that then hit the read-only filesystem restriction, a failure mode that only manifested under the combination, not either control alone. Rolling incrementally, one replica and one control at a time, even for a "already individually validated" combined change, would have isolated the interaction immediately instead of causing a fleet-wide incident.

An Incident, End to End#

Putting this chapter's tools together against a realistic scenario: a production API's replicas are intermittently unresponsive, and users are reporting timeouts.

docker events --since 30m --filter event=oom --filter event=die
docker container ls -a --filter name=catalog-api
docker container inspect catalog-api --format '{{json .State}}'
docker container stats --no-stream catalog-api
docker container logs --since 30m --timestamps catalog-api

This decision tree is deliberately built from this series' own sections — the point isn't a novel diagnostic framework, it's that Parts 1 through 4 already gave you every tool this incident needs, in the order a real triage actually proceeds: confirm what the kernel/daemon observed first (docker events), then check the specific mechanism most likely responsible (resource limits, health checks, or application/dependency issues) in the order suggested by that first signal, rather than guessing.

The Limits of a Single Docker Host#

Everything in this series so far — networking, storage, Compose, and this chapter's hardening — operates within one fundamental constraint: it's all one host. Every mechanism this series covered assumes a single Docker daemon, a single kernel, and a single point of failure for the entire stack.

None of this is a flaw in Docker — it's a scope boundary. A single well-hardened host, correctly monitored and backed up, is a completely legitimate production architecture for a workload whose availability requirements tolerate a single point of failure, or where the operational cost of a multi-host orchestrator genuinely isn't justified yet.

Signals That Mean It's Time for an Orchestrator#

The honest answer to "when do I need Kubernetes" is not a fixed traffic number — it's a set of concrete operational pains that a single Docker host structurally cannot solve, no matter how well-hardened.

Adopting Kubernetes before any of these signals are real, concrete, and currently causing pain is a well-documented mistake in the other direction — the operational cost of running Kubernetes well (a team that understands its scheduling, networking, and failure modes) is itself a serious ongoing investment, and paying it before the corresponding problem exists is trading a real, current cost for a hypothetical future benefit.

From the Trenches: A ten-person startup adopted Kubernetes eighteen months before any of the signals in the table above were real problems, based on hiring-market perception ("engineers want Kubernetes on their resume") rather than an operational need. The result was a small team spending a disproportionate fraction of its engineering time on cluster upgrades, YAML sprawl, and debugging Kubernetes-specific failure modes (a misconfigured resource request causing eviction storms) instead of the product — problems a single well-hardened Docker host running the same application would never have presented, because the actual traffic and availability requirements at that stage didn't need multi-host scheduling at all.

What Actually Changes Moving to Kubernetes#

The Kubernetes Deep Dive series covers this in full depth; this section names the direct conceptual mapping so the transition, when it's genuinely warranted, extends this series' vocabulary rather than replacing it wholesale.

The hardening controls from this chapter — capability drops, seccomp, read-only root filesystems, no-new-privileges — carry over almost unchanged, expressed through a Pod's securityContext instead of docker run flags. Nothing in Parts 1-3's mental model about images, layers, and registries changes at all; Kubernetes schedules and networks the exact same container images this series has been building the whole time.

From flags to securityContext#

The hardened baseline from earlier in this chapter translates nearly one-to-one into a Kubernetes Pod's securityContext, which is worth seeing side by side once before the migration path below, since it's the clearest evidence that this chapter's judgment — not just its specific flags — is what actually carries forward.

securityContext:
  runAsNonRoot: true
  runAsUser: 10001
  readOnlyRootFilesystem: true
  allowPrivilegeEscalation: false
  capabilities:
    drop: ["ALL"]
    add: ["NET_BIND_SERVICE"]
  seccompProfile:
    type: RuntimeDefault

Kubernetes additionally formalizes this into cluster-wide Pod Security Standards (privileged, baseline, restricted) that a cluster administrator can enforce as an admission policy — a namespace-level guarantee that no pod can be scheduled without meeting a minimum hardening bar, closing the gap this chapter's single-host approach has no equivalent for: on plain Docker, nothing stops a teammate from running an unhardened container next to a hardened one, because there's no cluster-wide policy layer to enforce it. See the Kubernetes Deep Dive series' Architecture and Control Plane chapter for how admission control and Pod Security Standards are actually enforced.

A Note on Terminology Before Migrating#

One last piece of vocabulary worth being precise about before the migration path below: "container orchestration" and "container scheduling" are sometimes used loosely as synonyms, but they name distinct responsibilities that Kubernetes happens to bundle together. Scheduling is the narrower concern — deciding which node a given workload should run on, given its resource requirements and the cluster's current capacity. Orchestration is the broader concern — scheduling, plus health-driven rescheduling, rolling updates, service discovery, and the declarative reconciliation loop that continuously drives actual state toward desired state. A single Docker host has no scheduling problem at all (there's only one place anything can run), which is precisely why "do we need a scheduler" is the wrong framing for the decision in the previous section — the real question is whether the broader orchestration responsibilities (rescheduling on failure, rolling updates, multi-node service discovery) are needed, not whether placement decisions have become complex.

This distinction also explains why a team sometimes reaches for a narrower tool than Kubernetes and is satisfied by it: Docker Swarm, Nomad, or a managed platform-as-a-service each bundle a different subset of orchestration responsibilities, and a team whose actual pain point is "I need multi-host rescheduling on failure" without needing Kubernetes' full extensibility model (custom resources, a broad ecosystem of operators, fine-grained RBAC) may find a narrower orchestrator solves their concrete problem with meaningfully less operational overhead. The signals table earlier in this chapter names the problems worth solving deliberately; which specific tool solves them is a separate decision that deserves its own evaluation against the team's actual operational capacity, not an assumption that Kubernetes is the only orchestrator that exists.

A Deliberate Migration Path#

A migration driven by the signals above, rather than a wholesale rewrite, moves incrementally and validates each step against the working Docker/Compose baseline.

1. Inventory the Compose file's real contract. Every depends_on condition, resource limit, health check, volume, and network boundary in the existing compose.yaml is a requirement the Kubernetes manifests must also satisfy — treat the Compose file as the specification, not just a development convenience to discard.
2. Translate service by service, verifying parity. Convert one service to a Deployment/Service pair at a time, confirming its readiness/liveness behavior and resource limits match the Docker Compose baseline before moving to the next — not a single big-bang conversion of the whole stack.
3. Replace depends_on startup ordering with readiness-aware application logic. Since Kubernetes has no direct equivalent, any service that assumed a dependency was already ready at startup (per Part 3's service_healthy discussion) needs its own retry/backoff logic, tested independently of the migration itself.
4. Validate stateful services last and most carefully. A database or anything with a PersistentVolumeClaim carries real migration risk — test the failover and rescheduling behavior in a non-production cluster before trusting it with production data, and keep the Docker Compose fallback path available until confidence is established.
5. Keep the hardening baseline, don't relax it during migration. It's tempting to strip security controls "temporarily" to get something running under time pressure during a migration — the seccomp trenches story earlier in this chapter is exactly what happens when a temporary relaxation outlives its justification.

Common Mistakes and Interview Traps#

Worked Practice Problems#

1. A container's health check is passing, but users report the service is unresponsive. What are the first two things to check?#

First, confirm what the health check actually tests — per this chapter's liveness/readiness distinction, a check that only verifies the process can respond to a trivial endpoint can pass indefinitely while the specific code path users depend on is broken or blocked on a dependency the check never exercises. Second, check docker events and docker stats for resource pressure (CPU throttling, memory near its limit, PID count) that could make the process technically alive and responding to the health check while too resource-starved to serve real requests within an acceptable latency.

2. A security review asks why a service still runs as root inside its container despite the team's stated hardening policy. The team says "we tried --user, but the app crashed on startup." What's the most likely actual issue, and how do you fix it root cause, not by reverting to root?#

The most likely cause is a file or directory the non-root user doesn't have permission to write to or read from — commonly a directory the application expects to write logs, a cache, or a PID file to, still owned by root from the image build. Fix it at the source: use COPY --chown in the Dockerfile (from Part 2) to set correct ownership on any path the application legitimately needs to write, or mount a tmpfs/volume at that path with the correct ownership, rather than reverting to running as root to avoid diagnosing the specific permission error.

3. A team wants to migrate to Kubernetes primarily because their current Compose-based deployment causes a few seconds of downtime on every release. Is this justified, and is there a smaller fix worth trying first?#

The downtime-on-deploy symptom is a real, valid signal from this chapter's table — but before committing to a full Kubernetes migration, check whether a smaller intervention solves the immediate pain: running multiple replicas of the service behind a reverse proxy on the same host, with the proxy configured to only route to a replica that's passed its readiness check, can eliminate deploy-time downtime without a multi-host orchestrator at all. If the team's actual scale only needs single-host redundancy, this is a substantially smaller investment than a Kubernetes migration; if they also need multi-host resilience against a whole-host failure, that signal independently justifies the larger move, and it's worth confirming which motivation is actually driving the request before committing engineering time to either.

4. A service's Postgres container needs CAP_CHOWN and CAP_FOWNER beyond the default drop-all baseline. A reviewer asks how the team determined that specific set rather than granting a broader "database capability profile." What's the correct methodology, and why does it matter?#

The correct methodology is empirical, not documentation-driven: start from --cap-drop ALL, run the actual container through its real startup and operational paths, and add back only the specific capability that resolves an observed, reproducible failure — repeating until the container operates correctly, as this chapter's Compose hardening section did for Postgres. This matters because a "broader database capability profile" adopted from a generic guide risks including capabilities that particular database version or configuration doesn't actually need, silently carrying unnecessary privilege forward — the same failure mode as the earlier CAP_SYS_ADMIN trenches story, just with a more plausible-sounding justification.

5. A team's automated image-watcher was removed after the broken-:latest-deployed-automatically incident, but engineers now complain that deploying a security fix takes "too long" through the CI-driven pipeline. How do you reconcile fast incident response with the verification discipline this series has built?#

The fix is not reintroducing an unconditional watcher — it's making the verified pipeline itself fast enough for genuine urgency, and giving it an explicit expedited path for exactly that case. A well-designed CI pipeline can run its scan/sign/promote steps in minutes, not hours, if they're not needlessly serialized behind unrelated slow steps; and a documented "expedited security promotion" procedure — still going through scanning and signing, but skipping a normally-required staging soak period with an explicit sign-off — gives urgent fixes a fast, still-verified path rather than forcing a choice between speed and safety.

6. After moving a stateful database service to Kubernetes, the team observes a rolling update briefly runs two Postgres replicas writing to the same underlying volume, causing data corruption. What went wrong, and how does this connect back to this chapter's guidance?#

This is exactly the risk this chapter flagged under "Validate stateful services last and most carefully" in the migration path — a naive Kubernetes Deployment's rolling update strategy assumes replicas are interchangeable and safely concurrent, which is true for a stateless API but actively dangerous for a single-writer database attached to shared storage. The fix belongs to Kubernetes-specific StatefulSet semantics (ordered, one-at-a-time rollout, and typically a ReadWriteOnce volume that can't even be mounted by two pods simultaneously) — covered in the Kubernetes Deep Dive series — but the underlying lesson is this chapter's own: don't treat a stateful workload's migration as equivalent in risk to a stateless one, and validate its specific failure modes in a non-production cluster before trusting it with real data.

7. A small team is consolidating three low-traffic internal tools onto a single Docker host to reduce cost, and someone raises the multi-tenancy concerns from this chapter. How do you decide whether the consolidation is acceptable, rather than either blocking it outright or ignoring the concern?#

Make the trust-boundary decision explicit rather than implicit, the corrective action from this chapter's own consolidation trenches story. Assess each service's actual risk profile — its exposure (public internet-facing versus internal-only), the sensitivity of the data it touches, and its dependency surface (how many third-party packages, how frequently patched) — and document which services are judged acceptable to share a kernel-level trust boundary and which aren't. A reasonable outcome is often consolidating the genuinely low-risk, internal-only tools together while keeping anything with real external exposure or sensitive data on separate hosts, rather than a binary "consolidate everything" or "isolate everything" decision made without looking at the actual risk differences between the three services.

8. A Falco-style anomaly detection rule fires a WARNING for a shell being spawned inside what's supposed to be a distroless production container. On-call dismisses it as a false positive because "distroless images don't have a shell, so this must be a detection bug." How should this actually be triaged?#

Treat it as a high-priority signal, not a likely false positive — the entire premise of the alert rule is that a shell should be structurally impossible to spawn in a genuinely shell-less image, so a firing alert most likely means one of two things, both serious: either the running image isn't actually the distroless image it's believed to be (a build or deployment mismatch worth confirming immediately against the image digest actually running, per this chapter's CVE-response enumeration technique), or a compromise has introduced an executable shell into the container through some other means (a downloaded binary, a bind-mounted path). Confirm the exact running image digest and inspect the container's actual filesystem before accepting the "must be a detection bug" explanation, since dismissing this specific alert as noise is exactly the failure mode detective controls exist to catch operators making.

9. A finance team's compliance requirement mandates a full audit log of every time a production database credential is accessed, including by which process and when. The current setup uses a static Compose secrets: file mount. What has to change, and what doesn't?#

The credential delivery mechanism itself needs to change — a static file mount has no per-access audit trail beyond host-level file-open logging, which is unlikely to satisfy a compliance requirement asking specifically "who accessed this and when" at the granularity of individual reads. Introducing an external secret manager with a fetch-agent pattern, as covered in this chapter, provides that audit trail natively at the secret manager itself. What doesn't need to change is the application's own interface to the credential — if it already reads from a file path rather than an inspectable environment variable (this chapter's consistent recommendation throughout), the fetch agent can populate that same path, and the application code requires no modification at all.

10. A postmortem for a host-wide outage finds that a single container's runaway process count exhausted the host's total PID limit, starving every other container on the box, even though that specific container's own memory and CPU usage stayed within its configured limits the entire time. What single control from this chapter would have prevented this, and why didn't the existing memory and CPU limits catch it?#

--pids-limit on that container would have prevented it by bounding its process count directly, independent of memory or CPU consumption — a fork loop can exhaust the kernel's total PID space while each individual process consumes a trivially small amount of memory and CPU, so limits on those two resources provide no protection against this specific failure mode at all. This is a concrete instance of the same lesson the disk-I/O throttling and multi-tenancy sections both made independently: CPU and memory limits are necessary but not sufficient for full resource isolation between containers sharing a host, and a genuinely defensive baseline needs a limit on every resource dimension a runaway process could actually exhaust, not just the two most commonly configured ones.

Summary and Series Wrap-Up#

This chapter turned "the container runs" into "the container runs safely, observably, and recoverably" — capability drops, seccomp and AppArmor, rootless daemons, read-only filesystems, and a combined hardened baseline that treats each control as answering a specific threat rather than following an undifferentiated checklist. Resource governance, log rotation, and a liveness/readiness split in health checks turn a container from something that merely starts into something an on-call engineer can actually trust and diagnose at 3 AM. Graceful shutdown handled correctly at the PID 1 level, a deployment pipeline that never bypasses its own verification gates, and a disaster recovery plan that's actually been exercised rather than merely documented round out the operational half of running containers responsibly — and the honest answer to "when do we need Kubernetes" is a short list of concrete, currently-real operational pains, not a maturity milestone to chase preemptively.

Across all four parts, the series has followed one throughline: a container is a declared, auditable, kernel-bounded process, and every practice — from Part 1's image lifecycle, through Part 2's supply-chain controls, Part 3's networking and storage discipline, to this chapter's hardening and operational judgment — exists to keep that declaration trustworthy under real production conditions. None of it is complicated in isolation; what makes it hold up in practice is treating each control as answering a specific, nameable threat, testing changes incrementally rather than trusting theory alone, and being honest about what a single host's isolation model can and can't guarantee once more than one team or workload shares it.

A team that has internalized this series' habits doesn't need Kubernetes to run containers responsibly; it needs Kubernetes only once a real, specific multi-host problem shows up that a single well-operated Docker host genuinely cannot solve — at which point the Kubernetes Deep Dive series picks up exactly where this one ends, extending the same discipline (declared configuration, minimal privilege, tested recovery, honest risk boundaries) across a fleet instead of reinventing it from scratch.

The four parts of this series were deliberately sequenced to build on each other rather than stand alone: Part 1's image and lifecycle model is the vocabulary every later chapter assumes; Part 2's build pipeline is what makes the digest this chapter's CVE-response and image-update sections rely on actually trustworthy in the first place; Part 3's networking and storage discipline is the substrate this chapter's hardening baseline and multi-tenancy guidance sit on top of. Revisit an earlier part whenever a concept here feels underspecified — the answer is very likely already there, expressed in that chapter's own vocabulary before this one needed it.