Networking, Storage & Docker Compose

Table of Contents#

1. One Host, Many Services
2. Docker Network Drivers
3. User-Defined Bridges and Embedded DNS
4. Publishing Ports Correctly
5. Multi-Network Containers and Segmentation
6. Static IPs, Custom IPAM, and extra_hosts
7. Volumes vs. Bind Mounts vs. tmpfs
8. Volume Drivers and Backup
9. Docker Compose as a Local Control Plane
10. Service Dependencies That Actually Wait
11. Compose Networking and Multiple Environments
12. Environment Variable Substitution and .env Files
13. Secrets and Configuration in Compose
14. Compose Profiles and Development Workflow
15. Resource Limits and Restart Policies in Compose
16. Reusing Configuration: extends and Fragments
17. Running Compose Stacks in CI
18. One-Off Commands: run vs. exec
19. Cleaning Up Networks and Volumes
20. How This Maps to Kubernetes Networking
21. Debugging Network and Storage Failures
22. Common Mistakes and Interview Traps
23. Worked Practice Problems
24. Summary and What's Next

One Host, Many Services#

Parts 1 and 2 treated a container mostly in isolation: one image, one process, one lifecycle. Real applications are rarely one container. An API needs a database, a cache, and often a message broker; a batch job needs a shared filesystem for intermediate output; a local development environment needs all of that running together, reproducibly, on one engineer's laptop.

Docker's answer to "how do these containers find and reach each other" is its networking model; its answer to "where does data live when a container is replaced" is volumes; its answer to "how do I describe a multi-service application declaratively instead of a sequence of docker run commands" is Compose. This chapter treats all three as one coherent problem — a service that needs a database is not fully specified until its network reachability, DNS name, and persistent storage are all declared alongside its image.

From the Trenches: A team debugging "the API can't reach the database" spent an hour checking application configuration before realizing both containers were on Docker's default bridge network, which has no embedded DNS and no automatic service discovery by name — the connection string referenced db as a hostname that simply couldn't resolve. Moving both containers onto a user-defined network fixed it in one command; the deeper lesson was that the default bridge is a legacy compatibility network, not the starting point for a real multi-service application.

Docker Network Drivers#

Docker ships several network drivers, each solving a genuinely different topology problem — not stylistic variants of the same thing.

docker network ls
docker network create --driver bridge --subnet 172.28.0.0/16 orderapp
docker network inspect orderapp

ipvlan and macvlan solve closely related but distinct constraints — worth distinguishing before reaching for either, since choosing the wrong one usually surfaces as a confusing switch-port security policy violation rather than a Docker error.

The default bridge network is a legacy fallback#

Any container started without an explicit --network lands on Docker's default bridge (usually docker0). It provides basic outbound connectivity and port publishing, but no embedded DNS and no automatic name-based discovery between containers — the exact gap in the trenches story above. Docker's own guidance is to treat it as a compatibility fallback, not a starting point; every real multi-container deployment should create and use an explicit user-defined network.

User-Defined Bridges and Embedded DNS#

When Docker creates a user-defined bridge network, every container attached to it gets automatic access to an embedded DNS resolver listening at 127.0.0.11 inside the container's network namespace. That resolver maps container names (and Compose service names, covered later in this chapter) to their current IP address, and forwards anything it doesn't recognize to the host's configured upstream DNS.

docker network create app-net
docker run -d --name postgres-db --network app-net postgres:16
docker run --rm --network app-net alpine:3.21 sh -c 'getent hosts postgres-db'

This is what makes DATABASE_URL=postgres://postgres-db:5432/app a portable connection string across environments — the hostname resolves to whatever IP that container currently has, even after a restart reassigns it, because the embedded resolver tracks the live mapping rather than a static address.

Network aliases and multiple names#

A container can be reachable under more than one DNS name on the same network via --network-alias, useful when a service must answer to both a generic role name and a specific instance name.

docker run -d --name postgres-db-1 --network app-net --network-alias primary-db postgres:16

From the Trenches: A blue-green deployment script relied on stopping and restarting a container with the same name, assuming the DNS mapping would simply follow. It did — the embedded resolver re-resolves on every lookup, not just at connection setup — but a downstream service using a connection pool that cached resolved IPs at pool-creation time kept sending traffic to the old, now-dead IP until the pool was recycled. The fix wasn't a Docker networking change; it was recognizing that DNS names being dynamic doesn't help a client that resolved once and cached the result indefinitely.

Publishing Ports Correctly#

A container's network namespace is invisible from outside the Docker host unless a port is explicitly published. Part 1 introduced -p/--publish; this chapter treats the decision of which interface to bind as a real security control, not a formatting detail.

docker run -d --name api --network app-net -p 127.0.0.1:8080:8080 catalog-api:2.6.0
docker run -d --name public-web --network app-net -p 0.0.0.0:443:443 web:1.2.0

-p 127.0.0.1:8080:8080 binds only to the host's loopback interface — reachable from the host itself (and anything tunneling through it, like an SSH port-forward) but not from the wider network. -p 443:443 with no explicit host IP binds to all host interfaces, which is correct for something the internet is actually meant to reach and wrong for an internal-only dependency.

A container-to-container-only service (the middle row skipped, the last row used) is the most secure option whenever nothing outside the Docker host — not even the host itself — needs direct access; the database in the diagram earlier in this chapter is a canonical example.

From the Trenches: A Redis cache was published with a bare -p 6379:6379 "for local debugging convenience" and the change was never reverted before deployment to a shared internal network. Redis had no authentication configured, on the reasoning that it was "only reachable from the application" — that reasoning stopped being true the moment the port bound to all interfaces. The incident review's corrective action was a linting rule in CI that flags any -p/ports: entry without an explicit host IP for anything other than a designated public-facing service.

Multi-Network Containers and Segmentation#

A container is not limited to one network. Attaching a container to more than one user-defined network is the primary tool for segmenting a multi-tier application so that, for example, a reverse proxy can reach both the public-facing web tier and an internal API tier while the database tier remains reachable only from the API.

services:
  proxy:
    image: nginx:1.27
    ports:
      - "0.0.0.0:443:443"
    networks: [edge-net, app-net]

  api:
    build: ./api
    networks: [app-net, data-net]

  db:
    image: postgres:16
    networks: [data-net]

networks:
  edge-net:
  app-net:
  data-net:

proxy can reach api (both on app-net) but has no path to db, because db is only attached to data-net and proxy never joined it. This is real network-layer segmentation, not merely an organizational convention — a compromised proxy container has no network route to the database tier at all, independent of any application-layer access control.

From the Trenches: A post-incident review of a compromised web-facing container found that the attacker's lateral movement attempt to reach the database directly failed at the network layer, well before any database credential would have mattered, because the web tier and data tier had never been on a shared network in the first place. The team's own retrospective noted this was closer to luck than deliberate design — the segmentation had been set up for a different reason (reducing accidental cross-service dependencies during development) — and they subsequently documented it as an explicit, intentional security control instead of an accidental side effect.

Static IPs, Custom IPAM, and extra_hosts#

Most services should rely on DNS names, not static IPs — the embedded resolver already solves the "how do I address another container" problem robustly. Static addressing is occasionally still necessary for a service with an external dependency that only accepts a fixed source IP, or a legacy client that can't do DNS lookups.

docker network create --subnet 172.30.0.0/24 --gateway 172.30.0.1 fixed-net
docker run -d --name legacy-client --network fixed-net --ip 172.30.0.10 legacy-app:1.0

extra_hosts injects a static entry into a container's /etc/hosts without involving Docker's own DNS — useful for a container that needs to resolve a hostname Docker itself doesn't manage, such as a host-level service or an external endpoint pinned for testing.

services:
  api:
    extra_hosts:
      - "payments.internal:10.0.4.12"

From the Trenches: A team assigned static IPs to every container in a Compose file, reasoning it would make debugging "more predictable." It instead made the environment fragile: recreating any single container (a routine docker compose up --build after a code change) occasionally produced IP conflicts when Compose's IPAM allocator handed out an address the static assignment elsewhere still expected. Removing the static assignments and relying on DNS names — the same fix from this chapter's opening trenches story — removed an entire, self-inflicted class of startup failure.

Docker networks default to IPv4; IPv6 support exists (docker network create --ipv6 --subnet <v6-subnet>) but is opt-in per network and not universally exercised by every downstream tool in a typical stack. Treat IPv6 as a deliberate requirement to design for explicitly — verifying the application, any reverse proxy, and the host firewall all handle it correctly — rather than something that "just works" once enabled on the network object.

Volumes vs. Bind Mounts vs. tmpfs#

Part 1 established that a container's writable layer is not durable storage. Docker offers three real mechanisms for anything that needs to outlive or bypass that writable layer, and they solve different problems.

docker volume create pgdata
docker run -d --name postgres-db --network app-net \
  -v pgdata:/var/lib/postgresql/data \
  postgres:16

docker run -d --name api-dev --network app-net \
  -v "$(pwd)/src:/app/src:ro" \
  -p 127.0.0.1:8080:8080 \
  catalog-api:dev

docker run --rm --tmpfs /run/secrets:rw,noexec,nosuid,size=16m alpine:3.21

The postgres-db example uses a named volume because database durability must survive docker rm and container recreation — exactly the property Part 1's incident workflow depends on when correcting a database container "by replacement, not repair." The api-dev example uses a read-only bind mount so an engineer's local edits appear instantly inside the running container without a rebuild, deliberately marked :ro so the container itself can't corrupt the host's working tree.

Named volumes vs. bind mounts as a decision, not a preference#

The underlying rule is simple even though the mechanisms differ: identify what actually needs to survive a container's replacement, and choose the mechanism whose lifecycle matches that requirement exactly, rather than picking whichever mount type happens to be the most familiar to whoever is writing the Compose file that day.

From the Trenches: A production Postgres container was run with a bind mount to /data/postgres in the belief that a bind mount is "just as durable" as a named volume. It is, until the host is rebuilt or the container is redeployed to a different node — the bind mount path silently didn't exist on the replacement host, and Postgres initialized a brand-new empty database rather than failing loudly. A named volume, or better, an external managed database, would have made the missing data an obvious startup condition to investigate rather than a silent data-loss event.

Volume Drivers and Backup#

The default local volume driver stores data on the Docker host's own disk. That is fine for single-host development and small deployments, but it means the volume's durability is exactly as good as that one host's disk — no more.

docker volume create --driver local pgdata
docker volume inspect pgdata

Third-party volume drivers (cloud block storage plugins, NFS-backed drivers) extend this to networked or replicated storage, trading local-disk simplicity for the operational complexity of running or depending on that driver's infrastructure. For most single-host Docker deployments, the practical durability strategy is not a fancier volume driver — it's a disciplined backup routine.

docker run --rm \
  -v pgdata:/data:ro \
  -v "$(pwd)/backups:/backup" \
  alpine:3.21 \
  tar czf /backup/pgdata-$(date -u +%Y%m%dT%H%M%SZ).tar.gz -C /data .

Running a throwaway container that mounts the volume read-only alongside a host backup directory is a portable pattern that works identically regardless of which volume driver actually backs the data — it operates on the volume's mount point, not on driver-specific internals.

A stateful service usually needs at least two of these layered together — a fast, frequent snapshot mechanism for quick point-in-time recovery, and a slower, application-consistent dump retained for longer for the case where the fast mechanism captured a mid-write inconsistency.

From the Trenches: A team's disaster-recovery test was the first time anyone had actually run their restore procedure — the nightly volume tarball had been running successfully (as measured by "did the backup job exit 0") for over a year, but the underlying tar command silently excluded a directory that had been remounted with different permissions months earlier. "The backup job succeeds" and "the backup is restorable" are different claims; only a periodic real restore test verifies the second one.

Docker Compose as a Local Control Plane#

Compose describes a multi-service application declaratively in a compose.yaml file, replacing a sequence of manually ordered docker run, docker network create, and docker volume create commands with one file that a team can review, version, and reproduce identically.

services:
  api:
    build: ./api
    image: catalog-api:dev
    ports:
      - "127.0.0.1:8080:8080"
    environment:
      DATABASE_URL: postgres://app:app@db:5432/catalog
    depends_on:
      db:
        condition: service_healthy
    networks: [app-net]

  db:
    image: postgres:16
    environment:
      POSTGRES_USER: app
      POSTGRES_PASSWORD: app
      POSTGRES_DB: catalog
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d catalog"]
      interval: 5s
      timeout: 3s
      retries: 10
    networks: [app-net]

networks:
  app-net:

volumes:
  pgdata:

docker compose up -d
docker compose ps
docker compose logs -f api
docker compose down

Note there is no top-level version: key — that field is obsolete in the current Compose specification and should be omitted; the tooling infers behavior from the Compose CLI version and file structure directly, not from a declared schema version number.

The mental model: Compose builds what docker run would have built#

Every top-level Compose concept maps directly onto a primitive from Parts 1-3: services.<name>.build is a docker build (Part 2's multi-stage patterns apply unchanged inside it), networks: is docker network create, volumes: is docker volume create, and depends_on/healthcheck sequence what would otherwise be manually ordered docker run commands. Nothing about Compose is a separate runtime — docker compose up produces ordinary containers, networks, and volumes indistinguishable from ones created by hand, inspectable with the exact same docker container inspect/docker network inspect commands from earlier chapters.

Service Dependencies That Actually Wait#

A bare depends_on: [db] only waits for the db container to start — not for Postgres inside it to actually accept connections. This is the single most common source of "the app crashed on startup in CI but works fine when I run it manually a second later" reports.

services:
  api:
    depends_on:
      db:
        condition: service_healthy
      migrate:
        condition: service_completed_successfully

  migrate:
    build: ./migrate
    depends_on:
      db:
        condition: service_healthy
    networks: [app-net]

  db:
    image: postgres:16
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d catalog"]
      interval: 5s
      timeout: 3s
      retries: 10

• condition: service_healthy waits for the dependency's healthcheck to pass — the same HEALTHCHECK mechanism introduced for standalone containers in Part 1, now used as an explicit startup gate.
• condition: service_completed_successfully waits for a one-shot service (a migration job) to exit 0 before starting the dependent service — a clean way to express "run migrations, then start the API" without a hand-written wait script.

From the Trenches: A CI pipeline's integration test suite failed intermittently, roughly one run in ten, always with a database connection error in the first few seconds of the API's logs. depends_on was present but used its default form (start-order only), and Postgres's actual readiness time varied slightly under CI runner load — usually fast enough to beat the API's first connection attempt, occasionally not. Switching to condition: service_healthy eliminated the flake entirely, because the API's container literally does not start until Postgres's own health check reports success.

Compose Networking and Multiple Environments#

Compose creates one default network per project (named after the project directory) if none is declared explicitly, and every service joins it automatically — which is why service names resolve as DNS hostnames between services with no extra configuration in the example above.

docker compose -p orderapp-staging -f compose.yaml -f compose.staging.yaml up -d

-p/--project-name lets multiple independent copies of the same Compose file run side by side (a staging and a feature-branch environment on one host) without network or volume name collisions — Compose namespaces every resource it creates with the project name.

Layering override files per environment#

# compose.yaml — base definition, shared by every environment
services:
  api:
    build: ./api
    environment:
      LOG_LEVEL: info

# compose.override.yaml — applied automatically in local dev
services:
  api:
    volumes:
      - ./api/src:/app/src:ro
    environment:
      LOG_LEVEL: debug

# compose.prod.yaml — applied explicitly for a production-like run
services:
  api:
    deploy:
      resources:
        limits:
          memory: 512M

docker compose up automatically merges compose.yaml with compose.override.yaml if present — the mechanism local development relies on for a bind-mounted source tree without touching the base file every other environment shares. A production-like run instead specifies -f compose.yaml -f compose.prod.yaml explicitly, skipping the dev-only override.

Environment Variable Substitution and .env Files#

Compose automatically reads a .env file in the project directory and substitutes ${VAR} references throughout compose.yaml — distinct from a service's own environment:/env_file: keys, which control what the container sees at runtime rather than what Compose itself resolves while parsing the file.

services:
  api:
    ports:
      - "127.0.0.1:${API_PORT}:8080"
  db:
    image: postgres:${POSTGRES_VERSION}

docker compose config    # shows the fully substituted result
API_PORT=9090 docker compose up -d    # shell env overrides .env for this invocation

Precedence matters when the same variable could come from more than one place: a value set directly in the invoking shell overrides .env, which is itself just a convenience default. This is useful for a one-off override (API_PORT=9090 docker compose up) without editing the checked-in .env file, but it also means a stale shell-exported variable from an unrelated project can silently override what looks like the correct .env value — docker compose config is the reliable way to confirm which value actually won.

From the Trenches: A production deployment script relied on a .env file's POSTGRES_VERSION default and worked correctly for months, until a CI runner's build environment happened to have POSTGRES_VERSION set globally for an unrelated tool, silently overriding the project's .env value and deploying a different major Postgres version than the one the team had tested against. The fix wasn't a Compose feature — it was adding docker compose config as an explicit, logged pre-deployment step so the resolved configuration was visible and auditable before up ever ran, rather than trusted implicitly.

Secrets and Configuration in Compose#

The db service example above put a database password directly in environment: for brevity — acceptable for a disposable local Postgres, wrong for anything resembling a real credential. Compose supports a secrets: mechanism that mounts a file rather than exposing a value through docker inspect-visible environment variables.

services:
  api:
    secrets:
      - db_password
    environment:
      DB_PASSWORD_FILE: /run/secrets/db_password

secrets:
  db_password:
    file: ./secrets/db_password.txt

The application reads the credential from the mounted file path at /run/secrets/db_password rather than an environment variable — environment variables are visible to any process that can read /proc/<pid>/environ inside the container and are captured verbatim by docker inspect, while a secret file's mount is scoped more tightly and doesn't appear in inspect output at all.

From the Trenches: A security review flagged that every credential for a local development stack was visible in plaintext via docker compose config and docker inspect, because the team had put real (non-dummy) third-party API keys directly in environment: for convenience, reasoning "it's just local dev." The actual risk wasn't local — those same Compose files were being adapted with minimal changes for a staging deployment, carrying the plaintext-credential pattern along with them. The fix was standardizing on secrets: file mounts from the start, even in local dev, so the pattern that reached staging was already the secure one.

Compose Profiles and Development Workflow#

Profiles let a Compose file describe optional service groups that don't start by default — useful for keeping a heavyweight debugging tool, a seed-data job, or an alternate service variant out of the everyday docker compose up path.

services:
  api:
    build: ./api

  db:
    image: postgres:16

  adminer:
    image: adminer:latest
    profiles: ["debug"]
    ports:
      - "127.0.0.1:8081:8080"

  seed-data:
    build: ./seed
    profiles: ["seed"]
    depends_on:
      db:
        condition: service_healthy

docker compose up -d
docker compose --profile debug up -d adminer
docker compose --profile seed run --rm seed-data

Services with no profiles: key always start; adminer and seed-data only start when their profile is explicitly requested, keeping the default developer experience minimal while still making optional tooling one command away.

Live development with watch#

Compose's develop.watch configuration re-syncs or rebuilds a service automatically when source files change, replacing a hand-rolled bind-mount-plus-file-watcher setup for languages that need a rebuild step (compiled languages, bundlers) rather than a simple bind mount.

services:
  api:
    build: ./api
    develop:
      watch:
        - action: sync
          path: ./api/src
          target: /app/src
        - action: rebuild
          path: ./api/package.json

docker compose watch

A sync action pushes changed files into the running container without a restart — appropriate for interpreted languages or hot-reloading frontends. A rebuild action triggers a full image rebuild and container replacement — appropriate when the change affects something baked into the image (a dependency manifest) rather than application source alone.

Resource Limits and Restart Policies in Compose#

Part 1 covered --memory/--cpus/--pids-limit for a standalone docker run. Compose exposes the same controls declaratively under deploy.resources, plus a restart policy field matching the standalone --restart flag.

services:
  api:
    build: ./api
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 512M
        reservations:
          memory: 256M
    restart: unless-stopped

deploy.resources is honored by both plain docker compose up and Swarm-mode deployment, unlike some other deploy: sub-keys (replica count, placement constraints) that only take effect under Swarm — a common point of confusion for anyone assuming the entire deploy: block is Swarm-only. Reservations express a soft floor the scheduler tries to guarantee; limits express a hard ceiling enforced by the kernel cgroup the same way Part 1's --memory flag was.

restart: unless-stopped mirrors Part 1's guidance directly: appropriate for a deliberately host-resident local service, not a substitute for health-aware orchestration when the failure needs investigation rather than a blind restart loop.

Reusing Configuration: extends and Fragments#

A Compose file describing several structurally similar services (multiple workers differing only in a queue name, several environments sharing most of one service's definition) benefits from avoiding copy-pasted blocks that drift out of sync.

services:
  worker-email:
    extends:
      file: compose.base.yaml
      service: worker-base
    environment:
      QUEUE_NAME: email

  worker-billing:
    extends:
      file: compose.base.yaml
      service: worker-base
    environment:
      QUEUE_NAME: billing

extends merges a named service definition from another file as a starting point, then applies local overrides — the two workers above share their image, build context, and volumes from worker-base in compose.base.yaml, differing only in the one field that actually varies.

YAML anchors and aliases solve a related but distinct problem — reusing a fragment of configuration within a single file, including fragments that aren't a whole service (a common environment: block, a common healthcheck:).

x-common-healthcheck: &common-healthcheck
  interval: 5s
  timeout: 3s
  retries: 10

services:
  db:
    image: postgres:16
    healthcheck:
      <<: *common-healthcheck
      test: ["CMD-SHELL", "pg_isready -U app"]

  cache:
    image: redis:7
    healthcheck:
      <<: *common-healthcheck
      test: ["CMD", "redis-cli", "ping"]

The x- prefix on x-common-healthcheck is the Compose specification's convention for an extension field ignored by the schema validator but usable as an anchor target — the standard way to define a reusable fragment without Compose mistaking it for an actual top-level common-healthcheck service or setting.

From the Trenches: Five near-identical worker services in one team's compose.yaml had drifted over a year of individual edits — one had gained a resource limit the others lacked, another had an outdated health check nobody had backported. Consolidating them onto a shared x- anchor and extends base made the actual per-worker differences (queue name, concurrency) visible for the first time, and surfaced the drifted health check as an obvious, fixable inconsistency instead of an invisible one.

Running Compose Stacks in CI#

A Compose file written for local development is often the fastest path to a realistic integration-test environment in CI — the same service definitions, the same networking and health-check behavior, run non-interactively and torn down after the test run.

# .github/workflows/integration.yml
jobs:
  integration:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Start the stack
        run: docker compose -f compose.yaml -f compose.ci.yaml up -d --wait
      - name: Run integration tests against the running stack
        run: npm run test:integration
      - name: Collect logs on failure
        if: failure()
        run: docker compose logs
      - name: Tear down
        if: always()
        run: docker compose down --volumes

docker compose up -d --wait blocks until every service with a health check reports healthy (or fails the step if one doesn't within its configured retries), which removes the need for a hand-rolled polling script in the CI job — the same service_healthy mechanism from earlier in this chapter, now used as a CI gate rather than an inter-service dependency. compose.ci.yaml is a small override file (the same layering pattern covered earlier) that might disable a dev-only bind mount or point at a CI-specific fixture dataset, without touching the base file every environment shares.

Collecting logs specifically if: failure(), and tearing the stack down if: always(), are the two details that make this reliable in practice: a failed integration test with no captured logs from the dependent services is often undebuggable after the fact, and a stack left running (or its volumes left behind) on a shared CI runner leaks resources into the next job.

One-Off Commands: run vs. exec#

docker compose run and docker compose exec both execute a command against a Compose-managed service, but they solve different problems and mixing them up produces confusing results.

docker compose run --rm api npm run lint
docker compose exec api npm run lint

run starts a brand-new, temporary container for the service, honoring its dependencies (depends_on still applies), and is the right choice when the service isn't already running or when the command needs a fresh, isolated container. exec runs the command inside an already-running service container — the same distinction as docker exec versus docker run from Part 1, now applied through Compose's service names instead of container names.

--rm on run matters for the same reason it mattered in Part 1 — without it, every one-off run invocation leaves a stopped container behind, and a CI job or local dev loop that runs migrations dozens of times a day will otherwise accumulate a large number of exited containers.

From the Trenches: A migration script run via docker compose exec api npm run migrate intermittently applied migrations against stale application code, because exec runs inside whatever container is currently live — including one still running the previous deployment's image after a rebuild that hadn't been applied yet with up. Switching the migration step to compose run --rm migrate (a dedicated one-shot service built from the current image) removed the ambiguity about which code version was actually executing the migration.

Cleaning Up Networks and Volumes#

A Docker host that runs many short-lived Compose projects over time accumulates unused networks and volumes that docker compose down alone doesn't always fully reclaim — a network or volume still referenced by a stopped-but-not-removed container, or one from a project that was torn down with docker compose stop rather than down, lingers.

docker network prune
docker volume prune
docker system df

docker system df shows how much disk space images, containers, volumes, and the build cache are each consuming — the first place to look when a Docker host unexpectedly runs low on disk. docker volume prune and docker network prune remove only resources with no attached container, which makes them safe to run without first auditing every volume by hand — but always confirm this with docker volume ls afterward, especially on a shared development host where "no attached container right now" doesn't always mean "safe to lose."

From the Trenches: A shared CI runner's disk filled up over a period of weeks because every integration-test run created a new set of named volumes and none were ever pruned — docker compose down without --volumes was the correct choice to avoid the earlier trenches story about accidental data loss, but it meant volumes silently accumulated forever in an environment where, unlike production, nothing needed to persist between runs. The fix was environment-specific: CI's teardown step uses down --volumes deliberately, because a CI runner's volumes have no long-term value, while the same --volumes flag remains banned from any script touching a real environment.

How This Maps to Kubernetes Networking#

Every Docker networking and storage concept in this chapter has a direct, but not identical, Kubernetes counterpart — worth naming now, since Part 4 discusses when to make that jump and the companion Kubernetes Deep Dive series covers the Kubernetes side in full depth.

See the Kubernetes Deep Dive series' Networking and Storage chapter for the full depth on Services, Ingress, CNI plugins, and persistent volume provisioning — this table is a bridge between the two series' vocabulary, not a substitute for that chapter.

Debugging Network and Storage Failures#

docker network inspect app-net --format '{{json .Containers}}'
docker exec api-dev sh -c 'wget -qO- http://db:5432 || true; nc -zv db 5432'
docker compose config
docker compose ps --format 'table {{.Name}}\t{{.Status}}\t{{.Health}}'

docker compose config renders the fully merged, resolved configuration (after all -f overrides and .env substitutions) — often the fastest way to confirm what Compose actually thinks it's about to run, rather than debugging a mismatch between the source YAML files and the runtime result.

A network-debugging sidecar for distroless services#

Part 2 deliberately recommended distroless and minimal base images with no shell and no networking tools — excellent for production security, unhelpful the moment you need to run curl or dig from inside that container's exact network namespace to debug a connectivity issue. The fix is not adding tools back into the production image; it's attaching a purpose-built debugging container to the same namespaces temporarily.

docker run --rm -it \
  --network container:api \
  --pid container:api \
  nicolaka/netshoot \
  curl -v http://db:5432

--network container:api and --pid container:api join the debugging container to the exact network and process namespaces of the running api container, without modifying api itself in any way — it sees exactly what api sees on the network, including DNS resolution through the same embedded resolver, while carrying its own full toolkit of curl, dig, tcpdump, and netstat. This is the practical answer to "how do I debug a distroless container's networking" that doesn't compromise the production image's minimal attack surface to get there.

Common Mistakes and Interview Traps#

Worked Practice Problems#

1. Two Compose services can't reach each other even though both are defined in the same compose.yaml with no custom networks: section. What's the first thing to check?#

Confirm both services are actually part of the same Compose project — a second, independently started Compose stack (a different -p project name, or a separate compose.yaml in another directory) creates its own separate default network, and two projects' default networks are not the same network even if the service names look related. Run docker compose ps and docker network ls to confirm which project each container belongs to and which network each is attached to before suspecting Docker's DNS itself.

2. A team wants zero downtime for a database container upgrade. Compose alone doesn't provide rolling replacement. What do you tell them?#

Compose is a single-host declarative tool, not an orchestrator with rolling-update or placement logic — it will stop and recreate a service's container directly, causing a connection interruption for anything depending on it. For genuine zero-downtime stateful upgrades, the team needs either a managed database service with its own upgrade tooling, or an orchestrator (Part 4's Kubernetes handoff) with StatefulSet-aware rolling update support and a database that tolerates a rolling restart of its replicas. Compose is the right tool for describing the stack, not for zero-downtime stateful rollout.

3. A docker compose down accidentally deleted a named volume containing real data. How could this have been prevented, and what does it reveal about the command?#

docker compose down --volumes (or the shorthand -v) explicitly removes named volumes declared in the Compose file — this is not the default behavior of a bare docker compose down, which preserves named volumes. The incident reveals either a habitual alias/muscle-memory use of -v, or a script that always includes --volumes regardless of environment. The prevention is process, not a Docker feature: never include --volumes in any script or alias used against an environment with real data, and consider a separate, clearly named script for the intentional "tear down and wipe data" case used only in disposable environments.

4. A team asks whether they should just publish every service's port to 0.0.0.0 "to keep things simple" during initial development, planning to lock it down before production. Why is this a bad default even temporarily?#

Habits formed during development tend to survive into production, especially under deadline pressure — the earlier Redis trenches story in this chapter is exactly a case of a "temporary" broad binding that was never reverted. The safer default is the opposite: bind to 127.0.0.1 or omit -p/ports: entirely for anything internal from the start, and only widen a specific service's binding when there's a concrete, reviewed reason (it's the actual public entry point). Treating restrictive binding as the default state that requires a deliberate, reviewable exception — rather than the reverse — removes an entire class of "we meant to fix that before shipping" incidents.

5. A reverse proxy container needs to reach both a public web tier and an internal API tier, but the API tier's database must remain completely unreachable from the proxy even if the proxy is compromised. How do you express this with Compose networking alone?#

Attach the proxy to a network shared with the web/API tier, and attach the database to a separate network that the proxy never joins — the multi-network segmentation pattern covered earlier in this chapter. The API service joins both the proxy-facing network and the database-only network, acting as the sole bridge between the two, while the proxy has no route to the database network at all. This is enforced at the network layer by Docker itself, independent of any application-level authentication the database might also have, so a compromised proxy cannot reach the database even by IP address.

6. An integration-test CI job using docker compose up -d --wait occasionally times out waiting for a service to become healthy, even though the same stack starts reliably in local development. What should you check before assuming the health check itself is broken?#

Compare the CI runner's resource allocation against a developer's local machine — a health check's start_period, interval, and retries were likely tuned against local startup times, and a resource-constrained or noisy-neighbor CI runner can genuinely take longer for the same service to become ready. Before rewriting the health check itself, try widening start_period and retries for the CI environment specifically (via a CI-specific override file, the same layering pattern used elsewhere in this chapter) and confirm whether the timeout disappears — if it does, the health check logic was correct and the CI environment simply needed a more generous startup allowance, not a different readiness test.

7. A .env-driven API_PORT value works correctly for every teammate except one, who insists their .env file is correct. What's the most likely explanation?#

The most likely cause is a shell-exported environment variable on that one teammate's machine with the same name, left over from an unrelated project or a previous manual override — shell environment variables take precedence over .env file values during Compose's substitution, so a correct .env file can still be silently overridden. Have them run docker compose config to see the actually-resolved value, and check their shell profile (.bashrc/.zshrc) or current session for a stray export API_PORT=... before assuming the .env file itself is the problem.

8. A shared development host is running low on disk, and docker system df shows a large amount of space attributed to volumes. Before running docker volume prune, what should you confirm?#

Confirm that every volume the prune would remove genuinely has no container depending on it currently or in the near future — docker volume prune only removes volumes with zero attached containers, so it will not touch anything actively in use, but a volume belonging to a stopped-but-not-yet-removed project (one torn down with docker compose stop rather than down) can still be safe to lose from Docker's perspective while representing data someone intended to come back to. List the candidate volumes with docker volume ls -f dangling=true first and cross-check the list against any team member's in-progress work before running the prune, rather than trusting the disk-space pressure alone as justification.

9. A worker service's extends-based configuration in compose.base.yaml was recently changed, but one specific worker's behavior didn't update as expected. What's the first thing to check?#

Confirm the specific worker's own service block doesn't already override the exact field that changed in the base — extends merges the base definition first and then applies local overrides, so a local override always wins even if the base definition changes underneath it. Run docker compose config to see the fully resolved configuration for that specific worker and compare it against the base definition directly, rather than assuming the base change should have applied uniformly to every service that extends it.

Summary and What's Next#

Docker's networking model turns a set of independent containers into an application: user-defined bridge networks provide isolation and automatic DNS-based service discovery, deliberate port-publishing decisions control exactly what's reachable from where, and multi-network segmentation turns tier boundaries into an enforced network-layer control rather than a documentation-only convention. Volumes, bind mounts, and tmpfs each solve a distinct persistence problem, and choosing the wrong one for the environment — a bind mount standing in for a database's durability guarantee — is a common, avoidable source of data loss, one a disciplined, periodically-tested backup routine exists to catch before it becomes an incident.

Compose brings all of this together declaratively: service dependencies that actually wait on readiness, layered configuration for different environments, secrets handled as mounted files rather than inspectable environment variables, resource limits and restart policies expressed the same way a standalone docker run would, reusable configuration via extends and anchors, and profiles that keep optional tooling out of the everyday path. The same Compose file that describes a local development environment is, with a small CI-specific override, also the fastest path to a realistic integration-test environment — one more instance of this series' recurring theme that Docker's primitives compose (no pun intended) rather than requiring a separate toolchain per environment.

Every mechanism in this chapter still answers to the same questions Part 1 opened with: what exactly is reachable, from where, and what actually persists when a container is replaced. A Compose file that gets networking segmentation, volume durability, and readiness gating right is not just more convenient than a sequence of manual commands — it is the difference between a stack that fails loudly and specifically when something is genuinely wrong, and one that fails silently or intermittently in ways that cost hours to trace back to a missing DNS entry, a misdirected bind mount, or a health check that never actually tested readiness. None of these mechanisms require an orchestrator to get right on a single host; they are exactly the discipline that makes the eventual move to one, when it's warranted, a change in scale rather than a change in fundamentals.

Part 4 shifts from "how do I run this stack" to "how do I run it safely and know when it's time to stop running it on plain Docker at all" — resource limits, logging and health in production, a rootless and hardened default posture, and the concrete signals that mean an orchestrator like Kubernetes has become necessary. The Kubernetes-mapping table earlier in this chapter is worth revisiting once Part 4 reaches that decision, since it names exactly which Compose concepts need a genuinely different, multi-host answer once a single Docker host stops being enough.