Systems Manager & Fleet Operations

Assumes you're comfortable with EC2/Auto Scaling (Part 3), IAM roles (Part 2), and CloudWatch alarms (Part 10) — this part is the operational layer that sits on top of all three: how a fleet of instances actually gets managed, patched, and remediated at scale, day to day, long after the initial launch.

Table of Contents#

1. Why This Part Exists
2. Systems Manager, the Big Picture
3. The SSM Agent and Managed Nodes
4. Session Manager — Shell Access Without SSH Keys or Bastion Hosts
5. Session Manager Port Forwarding — A Bastion Replacement for More Than Shells
6. Session Manager Logging and Auditability
7. Run Command — Fleet-Wide Ad Hoc Execution
8. Rate Control: Concurrency and Error Thresholds
9. Automation — Runbooks as Executable Documents
10. A Worked Automation Runbook
11. Targeting at Scale: Tags, Resource Groups, and Rate Control Together
12. Change Manager — Approval Workflows for Risky Automation
13. Cancelling and Recovering From a Bad Automation Execution
14. State Manager — Enforcing Desired State Continuously
15. Patch Manager, Visually
16. Patch Manager — Automated Patching at Scale
17. Patch Baselines and Approval Rules
18. Maintenance Window Tasks Beyond Patching
19. Maintenance Windows — Controlling When Disruptive Work Happens
20. Patch Compliance Reporting
21. Patching Beyond Reboot-and-Done: Scan-Only and No-Reboot Options
22. Avoiding Plaintext Secrets in Run Command and Automation Parameters
23. Parameter Store, Revisited
24. Hybrid Activations — Managing Servers Outside AWS
25. Keeping the SSM Agent Itself Current
26. Distributor — Packaging and Distributing Software Fleet-Wide
27. Explorer — A Cross-Account, Cross-Region Operational Dashboard
28. Inventory — What's Actually Installed, Fleet-Wide
29. Fleet Manager — A Console for the Whole Fleet
30. Tying Fleet Health Back to Part 10's Observability Stack
31. Application Manager — Grouping Resources by Application, Not Just Type
32. OpsCenter Source Integrations, Precisely
33. OpsCenter — Centralizing Operational Work Items
34. EC2 Image Builder — Automating the Golden AMI Pipeline
35. EC2 Image Builder: Components and Testing, in Detail
36. Compute Optimizer — Right-Sizing Recommendations
37. The 2026 Hybrid/Multicloud Pricing Change
38. Reacting Automatically: EventBridge Integration
39. Systems Manager vs Ansible/Chef/Puppet — Where Each Fits
40. Systems Manager IAM: Least Privilege for Fleet Operations
41. Windows-Specific Fleet Operations
42. Cost of Systems Manager Itself
43. A Full Worked Example: Zero-SSH-Key Fleet Operations
44. Troubleshooting a Managed Node That Won't Register
45. Cross-Account Fleet Operations at Organization Scale
46. Fleet Operations Best Practices — The Consolidated Checklist
47. Part 15 CLI Cheat Sheet
48. Common Mistakes and Interview Traps
49. Worked Practice Problems
50. Summary and What's Next

Why This Part Exists#

Every part of this series so far has been about building something — a network, a compute fleet, a database, a pipeline. This part is about operating what's already been built, at fleet scale, which is exactly SOA-C02's center of gravity: its largest domains are monitoring/remediation and reliability, and its networking/troubleshooting domains assume the operator already has a safe, auditable way to reach every instance in a fleet without SSH keys scattered across laptops. Systems Manager is that operational layer — one console and API surface for patching, remote command execution, configuration drift enforcement, and inventory, across every EC2 instance (and, with the SSM Agent installed, on-premises servers too) in an account. Where earlier parts asked "how do I build this correctly," this part asks "how do I keep hundreds or thousands of already-built instances in that correct state, continuously, without a person manually touching each one."

Systems Manager, the Big Picture#

Every capability in this part shares the same foundation: an IAM-authenticated agent running on a managed node, communicating outbound to the Systems Manager service — no inbound network path required at all, which is the single idea that makes almost everything else in this part possible. Keep that one diagram in mind as an anchor while reading the rest of this part — every section below is really just one branch of this same tree, examined in depth.

The SSM Agent and Managed Nodes#

The SSM Agent runs as a background process on the instance itself (pre-installed on most current Amazon Linux, Ubuntu, and Windows AMIs; installable on others) and initiates outbound HTTPS connections to the Systems Manager service — the instance never needs an inbound security group rule for SSM to reach it at all. For the agent to register as a managed node, the instance needs network reachability to the Systems Manager service endpoints (either a public route, or interface VPC endpoints for ssm, ssmmessages, and ec2messages for a fully private subnet, tying directly back to Part 4's PrivateLink pattern) and an attached instance profile (Part 2) carrying, at minimum, the AmazonSSMManagedInstanceCore managed policy.

Session Manager — Shell Access Without SSH Keys or Bastion Hosts#

Session Manager provides an interactive shell to a managed node through the SSM Agent's outbound connection — no open inbound port 22/3389, no SSH key to distribute or rotate, and no bastion host to patch and secure as its own attack surface (Part 4's bastion pattern, effectively made unnecessary for instances that don't need it for anything else). Access is controlled entirely through IAM policy on the calling principal, optionally scoped further by document ARN conditions limiting exactly what a session can do (e.g., restricted to a specific port-forwarding session type, not a full shell) — the same least-privilege discipline from Part 2, applied to interactive access instead of API calls.

Session Manager Port Forwarding — A Bastion Replacement for More Than Shells#

Beyond an interactive shell, Session Manager supports port forwarding: tunneling a local port through the same agent-initiated, no-inbound-port channel to a remote port on the managed node — or, with the AWS-StartPortForwardingSessionToRemoteHost document, to an entirely different host reachable from that instance's network, such as a private RDS instance (Part 6) with no public endpoint at all. This is the direct, more auditable replacement for the classic "SSH tunnel through a bastion to reach a private database" pattern from Part 4 — a database client on an engineer's laptop connects to localhost:5432, which Session Manager tunnels to the actual private RDS endpoint, with the same IAM-scoped access control and CloudTrail logging as any other Session Manager session, and zero bastion host to provision or patch.

Session Manager Logging and Auditability#

Every session's input and output can be streamed to CloudWatch Logs and/or an S3 bucket, and every StartSession/TerminateSession API call is itself recorded in CloudTrail (Part 9) — meaning an organization gets a complete, centrally auditable record of who accessed which instance, when, and exactly what commands they ran, something a traditional SSH-key-based bastion setup essentially never provides without extensive extra tooling. This is precisely the kind of control an auditor or a SOC 2/PCI compliance review looks for, and it's a strong, concrete answer to "how do you control and audit production access" in an interview setting.

Run Command — Fleet-Wide Ad Hoc Execution#

Where Session Manager gives one engineer an interactive shell on one instance, Run Command executes a predefined command or script across any number of instances at once, targeted by instance ID, tag, or resource group — restarting a service fleet-wide, rotating a log file, or running a one-off diagnostic script without SSH-ing into each box individually. Commands run as SSM Documents (the same JSON/YAML document format Automation runbooks use below), either AWS-provided (AWS-RunShellScript, AWS-RunPowerShellScript) or custom-authored for a repeated internal task.

Rate Control: Concurrency and Error Thresholds#

Running a command against a thousand instances at once risks the exact "thundering herd" problem Part 3's Auto Scaling discussion warned about — Run Command (and Automation, and Patch Manager below) all support rate control: a concurrency limit (how many targets run simultaneously) and an error threshold (automatically stop the rollout if too many targets fail) applied together. This is the same blast-radius-limiting instinct from Part 14's canary deployments, applied to fleet-wide command execution instead of application traffic — a bad script caught after 5% of a fleet fails, not after all of it does.

Automation — Runbooks as Executable Documents#

Automation turns a written runbook — the kind of step-by-step incident/maintenance procedure Part 11 of the broader course's incident-management material already covers conceptually — into an actual executable document: a sequence of steps (each either an AWS API call, a Run Command invocation, an approval gate, or a branch) defined in YAML/JSON, versioned, and invokable on demand or on a schedule. AWS ships dozens of predefined runbooks (AWS-UpdateLinuxAmi, AWS-RestartEC2Instance, AWS-CreateImage) covering common operational tasks out of the box, and an organization can author its own for anything repeatable and error-prone enough to be worth turning into code — the direct, concrete implementation of "runbooks as code" rather than a wiki page a human has to follow by hand and might get wrong under 3am incident pressure.

A Worked Automation Runbook#

schemaVersion: "0.3"
description: "Restart a service and verify it came back healthy"
parameters:
  InstanceId:
    type: String
mainSteps:
  - name: RestartService
    action: aws:runCommand
    inputs:
      DocumentName: AWS-RunShellScript
      InstanceIds: ["{{ InstanceId }}"]
      Parameters:
        commands: ["systemctl restart myapp"]
  - name: VerifyHealthy
    action: aws:runCommand
    inputs:
      DocumentName: AWS-RunShellScript
      InstanceIds: ["{{ InstanceId }}"]
      Parameters:
        commands: ["curl -f localhost:8080/health"]

Two steps, each a plain Run Command invocation, sequenced and parameterized — this is genuinely the whole shape of most real runbooks: a handful of steps, each either an AWS API call or a Run Command execution, with the automation service handling sequencing, retry, and failure reporting so nobody has to run this by hand from a wiki page at 3 AM and risk a typo mid-incident.

Targeting at Scale: Tags, Resource Groups, and Rate Control Together#

Run Command, Automation, State Manager, and Patch Manager all share the same targeting model, and picking the right target scope matters as much as the automation logic itself: targeting by instance ID is precise but doesn't scale past a handful of instances named by hand; targeting by tag (the pattern used throughout this part's examples) scales naturally as a fleet grows, since a newly launched instance carrying the right tag is automatically in scope with zero extra configuration; targeting by Resource Group (a saved, reusable query — Part 1's tagging discipline, made queryable) is the right choice when the same complex targeting logic (e.g., "every EC2 instance tagged Tier=web AND Environment=production in this specific VPC") needs to be reused consistently across several different Automation documents rather than re-specified by hand each time. Whichever targeting mechanism is chosen, it composes with the rate control settings covered above — targeting scope decides which instances are eligible, rate control decides how fast the operation actually reaches them.

Change Manager — Approval Workflows for Risky Automation#

For a change too high-risk to run unattended, Change Manager layers an approval workflow on top of Automation: a requested change (an Automation runbook execution against production, say) requires sign-off from a designated approver — individual or via an SNS-notified approval group — before it's allowed to execute, with the full request, approval, and execution history retained for audit. This is Systems Manager's native implementation of the change-advisory-board pattern familiar from ITIL-style change management, expressed as an actual gated workflow instead of a meeting or an email thread, and it composes directly with the runbooks Automation and State Manager already define — the same document, just now requiring approval before Change Manager will let it run.

Cancelling and Recovering From a Bad Automation Execution#

An in-progress Automation execution can be stopped mid-run (aws ssm stop-automation-execution) — either a Cancel (stop cleanly after the current step finishes) or, for a document authored with rollback steps defined, an automatic transition into those rollback steps to undo whatever the execution had already done. This is why a well-authored runbook for anything destructive or hard-to-reverse should define explicit onFailure/rollback steps rather than assuming "it either fully succeeds or nothing happened" — a multi-step runbook that creates a resource in step 2 and fails in step 4 has real partial state to clean up, exactly the same "partial failure is a real state, not an edge case" lesson Part 11's CloudFormation rollback behavior already teaches for infrastructure deployments.

State Manager — Enforcing Desired State Continuously#

State Manager is what makes Automation continuous rather than one-shot: an association binds a runbook to a set of targets on a recurring schedule (or on every state change), re-applying it automatically — attaching a security patch baseline, enforcing a required tag, ensuring an EBS snapshot policy is present, or simply re-running an inventory-gathering document on every managed node every 30 minutes. This is the same reconciliation-loop idea already familiar from GitOps/Kubernetes controllers, scoped to EC2/on-prem fleet configuration instead of container manifests — drift gets automatically corrected on the next association run rather than silently persisting until someone notices.

Patch Manager, Visually#

Patch Manager — Automated Patching at Scale#

Patch Manager is the specific, most heavily-weighted-on-SOA-C02 application of Automation and State Manager together: scanning managed nodes for missing OS and application patches, then installing approved ones on a schedule, with a full compliance report at the end. This directly answers the exam-favorite question "how do you patch hundreds of EC2 instances without SSH-ing into each one" — the answer is never "a person runs apt upgrade on each box," it's Patch Manager driven by a patch baseline and a maintenance window.

Patch Baselines and Approval Rules#

A patch baseline defines which patches are actually approved for installation — AWS-provided default baselines exist per OS, but a custom baseline is the norm in any real environment: approval rules (e.g., "auto-approve any Critical or Important security patch 7 days after release, giving time to catch a bad patch reported elsewhere first") and an explicit rejected patches list for anything known to break a specific application. This 7-day-delay pattern is a genuinely common, exam-relevant practice — patching immediately on release trades safety for a small window of extra exposure; most production environments accept that small window in exchange for not being the first to discover a bad vendor patch.

Maintenance Window Tasks Beyond Patching#

A maintenance window isn't exclusively for Patch Manager — any Automation runbook, Run Command document, Lambda function, or Step Functions state machine (Part 13) can be registered as a task inside a window, each with its own targets and rate control, sharing the same schedule. A common real pattern: one weekly maintenance window running three tasks in sequence — Patch Manager's scan-and-install, then an Automation runbook that restarts a dependent caching layer, then a Run Command health-check script confirming the fleet came back healthy — all confined to the same communicated low-traffic window instead of three separately scheduled, uncoordinated operations that could otherwise land at unpredictable, potentially overlapping times.

Maintenance Windows — Controlling When Disruptive Work Happens#

A maintenance window schedules exactly when disruptive operational work (patching, a reboot, a Run Command script that briefly interrupts service) is allowed to execute — a defined recurring time block, targets, and the specific tasks (Automation runbooks, Run Command documents, Lambda functions) permitted to run inside it. Combining a maintenance window with Patch Manager is the standard shape: patches get scanned continuously, but installation and any required reboot only happen inside the approved window, keeping disruptive change confined to a predictable, communicated time rather than happening whenever a scan completes.

Patch Compliance Reporting#

After a patching operation, every managed node reports a compliance status (Compliant, Non-Compliant, or Unknown) against its assigned baseline, queryable via the console, API, or exported to S3 as a CSV for further analysis — and, combined with AWS Config (Part 1/Part 9), a non-compliant instance can trigger an automated remediation action rather than waiting for a human to notice a dashboard. This is the fleet-wide extension of the same compliance-as-code idea Part 9 already established for security configuration, applied specifically to patch currency.

Patching Beyond Reboot-and-Done: Scan-Only and No-Reboot Options#

Not every patching cycle should install and reboot immediately. Patch Manager supports a Scan operation type (report compliance without installing anything — useful for a pre-change compliance snapshot) separate from Scan and install, and, per-patch-baseline, an option to suppress the reboot even after installing patches that would normally require one (deferring the reboot to a separately scheduled, lower-traffic window). A stateful application fleet behind an ALB (Part 8) with connection draining configured can usually tolerate a rolling reboot across a maintenance window without customer impact, but a fleet fronting long-lived connections (a WebSocket API's backend, say) often deliberately decouples "patches installed" from "instance rebooted" for exactly this reason.

Avoiding Plaintext Secrets in Run Command and Automation Parameters#

The same discipline Part 14 established for application code applies equally to operational tooling: a Run Command or Automation parameter passed as a plain string is visible in the command's own history (queryable via describe-command-invocations) and, depending on document configuration, in the session's own logs — never an appropriate place for a database password or API key. The correct pattern is a SecureString Parameter Store reference ({{ssm-secure:/path/to/param}}) or a direct Secrets Manager lookup performed inside the script itself at execution time, so the actual secret value is fetched just-in-time by the running command and never appears in the command's own recorded parameters or logs at all — the fleet- operations equivalent of never baking a credential into a deploy-time environment variable.

Parameter Store, Revisited#

Part 9 introduced Systems Manager Parameter Store as a Secrets Manager alternative for non-rotating configuration values. Worth stating explicitly in this part's fleet-operations context: Parameter Store is also how Automation runbooks and Run Command documents typically receive shared configuration (an AMI ID to standardize on, a list of approved instance types) without hardcoding values into every document — a runbook referencing {{ssm:/fleet/standard-ami-id}} picks up a centrally-updated value automatically, the same "single source of truth, referenced not duplicated" principle already familiar from Part 11's infrastructure-as-code discipline.

Hybrid Activations — Managing Servers Outside AWS#

Systems Manager isn't limited to EC2. An activation generates a registration code and ID that, combined with the SSM Agent installed manually, lets an on-premises server or a VM in another cloud register as a managed node — appearing in Fleet Manager, receiving patches through Patch Manager, and reachable through Session Manager exactly like an EC2 instance, using an IAM role assumed via the activation rather than an instance profile (since there's no EC2 instance metadata service to source credentials from off-AWS). This is the concrete mechanism behind "single-pane-of-glass fleet management across a hybrid or multi-cloud estate" — a genuinely common real-world requirement for an organization mid-migration (Part 17) or deliberately running a permanent hybrid footprint, and the exact capability the 2026 pricing change above affects directly.

Keeping the SSM Agent Itself Current#

The SSM Agent is, itself, software running on every managed node — and an outdated agent can miss newer Session Manager/Run Command features or, rarely, carry a fixed bug from an older release. AWS publishes the agent's own update as a State Manager association target (AWS-UpdateSSMAgent), the same mechanism used for every other continuous-enforcement task in this part — meaning "keep the agent current fleet-wide" is itself just one more association, not a special-cased manual process, closing the loop on the one piece of this whole system that isn't otherwise self-updating.

Distributor — Packaging and Distributing Software Fleet-Wide#

Distributor packages arbitrary software (not just OS patches) — an internally built agent, a third-party monitoring tool, a licensed package — into a versioned artifact that can be installed and kept updated across a fleet via the same State Manager association mechanism already covered above. This closes a gap Patch Manager doesn't cover: Patch Manager handles OS-vendor and common-application patches from standard repositories, while Distributor handles anything else an organization needs installed and version-pinned consistently across every instance, without hand-rolling a bespoke deployment mechanism for it.

Explorer — A Cross-Account, Cross-Region Operational Dashboard#

Explorer aggregates OpsItems, patch compliance, and other operational data from every account and region in an Organization (using the same delegated-administrator pattern from Part 1) into one dashboard — the practical answer to "how does a central platform team see operational health across fifty accounts without opening fifty separate consoles." Combined with OpsCenter's per-item drill-down, Explorer is typically the first screen a platform on-call engineer checks at the start of a shift, and OpsCenter the tool they use once something specific needs investigating.

Inventory — What's Actually Installed, Fleet-Wide#

Inventory collects metadata from every managed node on a schedule (itself just a State Manager association running the AWS-GatherSoftwareInventory document) — installed applications, OS patch state, running services, network configuration, and (optionally) custom-defined data — aggregated centrally and queryable across the whole fleet. This answers the unglamorous but genuinely critical question "which of our thousand instances is still running the vulnerable version of this library," in minutes via a query, instead of a fleet-wide manual audit.

Fleet Manager — A Console for the Whole Fleet#

Fleet Manager is the consolidated console view over everything above: browse every managed node's health, connect via Session Manager directly from the same screen, view installed applications from Inventory, and manage Windows-specific operations (registry, services, event logs) that don't have a Linux-side equivalent — a single operational surface instead of jumping between the EC2 console, Session Manager, and Inventory separately for routine fleet work.

Tying Fleet Health Back to Part 10's Observability Stack#

Everything in this part produces signal that ultimately belongs in the same CloudWatch-centric observability picture Part 10 already established: patch compliance can be published as a custom metric and alarmed on ("alert if fleet-wide compliance drops below 95%"), OpsItem creation can drive the same SNS/on-call paging paths as any other alarm, and Inventory data can feed a CloudWatch dashboard tracking fleet-wide software currency over time. The point worth internalizing: Systems Manager isn't a separate monitoring silo running parallel to Part 10's tooling — it's a producer of operationally meaningful signal that plugs into the exact same alerting and dashboard infrastructure already built for application-level observability, so an on-call engineer has one unified view rather than a separate Systems Manager-specific dashboard to check independently.

Application Manager — Grouping Resources by Application, Not Just Type#

Most AWS consoles are organized by resource type — the EC2 console shows instances, the RDS console shows databases. Application Manager instead groups resources by application (using tags, a CloudFormation stack, or a Resource Groups query as the grouping key), so a team can see one application's EC2 instances, RDS databases, Lambda functions, and their current OpsItems/cost/compliance status together on one screen, rather than reconstructing that picture by cross-referencing five separate consoles. For an organization running dozens of applications across shared accounts, this is a meaningfully faster path to "is Application X healthy right now" than the resource-type-first default view most engineers reach for out of habit.

OpsCenter Source Integrations, Precisely#

OpsCenter doesn't just accept manually created OpsItems — it natively ingests findings from CloudWatch Alarms, AWS Config compliance changes (Part 9), Trusted Advisor checks (Part 1), and Systems Manager's own Automation/Patch Manager failures, normalizing all of them into the same OpsItem shape regardless of which service originally raised the issue. This normalization is the actual value: an operator triaging OpsItems doesn't need to separately know each source service's own alerting UI and severity conventions — a Config non-compliance finding and a CloudWatch alarm breach show up as equally structured, equally actionable OpsItems in the same queue.

OpsCenter — Centralizing Operational Work Items#

OpsCenter aggregates operational issues (a failed CloudWatch alarm, a Config non-compliance finding, a Trusted Advisor check) into a single, standardized work-item type — an OpsItem — with contextual diagnostic data and, critically, a suggested Automation runbook to resolve it attached directly to the item. This is Systems Manager's answer to "where does an operator actually go to see everything currently wrong across the account," reducing the mean-time-to-resolution problem Part 11's incident-response material addresses conceptually, by giving the responder a pre-diagnosed starting point instead of a raw alarm needing manual investigation from scratch.

EC2 Image Builder — Automating the Golden AMI Pipeline#

Part 3 covered AMI lifecycle management as a manual/CI-driven discipline. EC2 Image Builder automates that pipeline directly: a recipe (a base image plus a list of components — install packages, apply patches, run custom scripts) feeds a pipeline that builds, tests, and distributes a new AMI on a schedule or on a trigger (a new base-image release, for instance), with built-in testing (AWS-provided or custom validation tests) before an image is marked as ready for use, and automated distribution to multiple accounts/regions via Resource Access Manager (Part 1). This is the direct mechanism behind "golden AMI" pipelines: a security-patched, hardened, pre-configured base image rebuilt automatically on a cadence, rather than a stale image quietly drifting out of compliance for months.

EC2 Image Builder: Components and Testing, in Detail#

An Image Builder recipe is built from reusable components — each a small, versioned YAML document declaring a build phase (install a package, apply a patch, copy a file) and a validation/test phase (assert a service is running, run a vulnerability scanner, confirm a required file exists) — AWS-provided or custom, and shareable across recipes the same way an SAM policy template (Part 14) is shared across functions. A pipeline runs the full build, then automatically launches a temporary test instance from the new image and runs every attached component's test phase against it — an image only reaches the Available distribution stage after passing, meaning a broken image never silently reaches production Auto Scaling Groups pulling "the latest approved AMI." This test-before-distribute step is the concrete mechanism that makes an automated AMI pipeline trustworthy enough to point production launch templates at directly, rather than requiring a human to manually smoke-test every new image build first.

Compute Optimizer — Right-Sizing Recommendations#

Compute Optimizer analyzes actual CloudWatch utilization history (CPU, memory with the CloudWatch Agent installed, network, EBS throughput) for EC2 instances, Auto Scaling Groups, EBS volumes, Lambda functions, and ECS services on Fargate, and produces concrete right-sizing recommendations — "this m5.2xlarge has averaged 8% CPU utilization for 30 days; an m5.large would meet the same workload at a fraction of the cost." This is a genuinely low-effort, high-value first pass before any manual capacity review (Part 3's capacity-planning discipline) — it's free, requires no separate agent for CPU-only recommendations, and directly feeds Part 16's cost-optimization workflow.

Each finding carries a risk classification worth understanding before acting on it blindly:

Compute Optimizer also reports a performance risk score per recommendation — a downsizing suggestion with a nonzero performance risk is a genuine cost/reliability tradeoff to evaluate deliberately, not a blind auto-apply, which is exactly why Compute Optimizer produces recommendations, not automatic changes.

The 2026 Hybrid/Multicloud Pricing Change#

Worth a specific currency note: AWS removed Systems Manager's "advanced-instances" tier effective June 30, 2026, alongside its previous 1,000-instance limit for hybrid managed nodes — meaning there's no longer a hard cap forcing an organization onto a paid advanced tier once it exceeds 1,000 on-premises/hybrid nodes. A transition window (through September 30, 2026) offered free Session Manager and Run Command usage for hybrid/multicloud nodes; standard pay-as-you-go pricing for those specific capabilities applies from that date forward. Any older documentation describing a strict 1,000-node advanced-tier requirement is describing a since-removed constraint.

Reacting Automatically: EventBridge Integration#

Systems Manager publishes events onto the default EventBridge bus (Part 13) for state changes worth reacting to automatically rather than watching a dashboard for — a new OpsItem created, a State Manager association's compliance status changing, a Patch Manager operation completing. A common, genuinely useful pattern: an EventBridge rule matching "OpsItem created with severity Critical" targeting an SNS topic that pages on-call, or targeting a Step Functions workflow (Part 13) that automatically attempts a known-safe remediation runbook before ever paging a human at all — closing the loop from "something's wrong" to "already being fixed" without a person in the critical path for well-understood, low-risk failure modes.

Systems Manager vs Ansible/Chef/Puppet — Where Each Fits#

A reasonable question for a team already running Ansible, Chef, or Puppet: does Systems Manager replace that tooling? Mostly no — they solve overlapping but distinct problems:

In practice, many organizations run both: Systems Manager for the access/audit/patching layer (replacing bastion hosts and ad hoc SSH), and Ansible/Chef/Puppet for deep application-configuration management — they're not mutually exclusive, and Run Command can even trigger an Ansible playbook run as one of its targets rather than forcing an either/or choice.

Systems Manager IAM: Least Privilege for Fleet Operations#

Following Part 2's least-privilege discipline, an engineer's Session Manager access should rarely be the broad AmazonSSMFullAccess managed policy — a scoped custom policy restricting ssm:StartSession to specific tag-based resource conditions (e.g., only instances tagged Environment=staging), combined with a Session Manager document-level restriction limiting which session types are permitted, is the real production pattern. The managed node's own instance profile, separately, needs only AmazonSSMManagedInstanceCore (or a more scoped custom policy) — never broader permissions bundled onto it "just in case," since anything on that profile is reachable by anyone with shell access to the instance through Session Manager.

Windows-Specific Fleet Operations#

SOA-C02 tests Windows fleet management specifically, and it's worth calling out where Systems Manager diverges from the Linux-centric examples above. Fleet Manager exposes Windows-only management surfaces — browsing and editing the Windows Registry, managing Windows Services (start/stop/configure), viewing the Windows Event Log — none of which have a Linux equivalent, directly from the same console used for Session Manager access. Patch Manager's Windows baselines work against Windows Update classifications (Critical, Security, Definition Updates) rather than a Linux package manager's repository metadata, and AWS-RunPowerShellScript is Run Command's Windows-side equivalent of AWS-RunShellScript — the operational model (agent-initiated, IAM-authenticated, no inbound port) is identical across both operating systems; only the specific documents and management surfaces differ.

Cost of Systems Manager Itself#

Most of what's covered in this part — Session Manager, Run Command, Automation, State Manager, Patch Manager, Inventory, standard-tier managed nodes — is free, the same "no markup over the underlying resources" property already established for Elastic Beanstalk and SAM in Part 14. The costs that do apply: Parameter Store's advanced parameters (beyond the free-tier standard parameter limits), Change Manager's approval workflow at scale, and — per the pricing change noted above — hybrid/multicloud managed nodes using Session Manager or Run Command beyond the 2026 transition window. This near-zero direct cost is part of why Systems Manager adoption is rarely a budget conversation; the real cost, as with most of this part's tooling, is the engineering time to configure it well, not a line item on the AWS bill. Part 16 covers the tooling for tracking exactly where that near-zero-marginal-cost story stops holding — EC2 Image Builder's build-instance minutes and Distributor's package storage both still accrue ordinary resource charges even though the orchestration layer around them is free.

A Full Worked Example: Zero-SSH-Key Fleet Operations#

A platform team operating a 200-instance fleet with no SSH keys anywhere:

1. Every instance's instance profile carries AmazonSSMManagedInstanceCore; the VPC has interface endpoints for ssm, ssmmessages, and ec2messages (Part 4), so private-subnet instances with no internet route still register as managed nodes.
2. Engineers get IAM policies scoped to ssm:StartSession against specific tag conditions — no SSH key ever generated, distributed, or rotated for this fleet.
3. A custom patch baseline auto-approves Critical/Important patches after a 7-day delay; a maintenance window runs Patch Manager every Sunday at 02:00 local time, with rate control capped at 10% concurrent targets and a 2% error threshold that halts the rollout if exceeded.
4. A State Manager association re-runs AWS-GatherSoftwareInventory every 30 minutes, feeding a query used during incident response to instantly confirm which instances are running a since-patched vulnerable library version.
5. A failed CloudWatch alarm on any instance automatically creates an OpsItem in OpsCenter, pre-populated with a suggested AWS-RestartEC2Instance or AWS-RebootInstance remediation runbook, cutting the responder's first diagnostic step out entirely.
6. New AMIs are rebuilt weekly by an EC2 Image Builder pipeline layering the latest OS patches onto the team's hardened base recipe, distributed automatically to every account in the Organization via RAM.
7. Compute Optimizer's monthly recommendations feed directly into the team's Part 16 cost-review cadence, catching over-provisioned instances before they've accumulated months of wasted spend.
8. A database administrator needing occasional access to a private RDS instance uses Session Manager port forwarding to tunnel a local connection through, rather than a standing bastion host that would need its own patching and monitoring.
9. Any change to the fleet's core Automation runbooks (the one that restarts the primary application service, say) requires Change Manager approval from a second engineer before it's allowed to execute against production — routine read-only Run Command diagnostics don't require this gate, keeping the approval friction proportional to actual risk.
10. Explorer gives the on-call platform engineer a single cross-account view of every open OpsItem and patch-compliance gap at the start of a shift; OpsCenter is where they drill into any one item that needs actual investigation.

Troubleshooting a Managed Node That Won't Register#

A recurring, genuinely practical SOA-C02 scenario: a newly launched instance never shows up in Fleet Manager as a managed node. The diagnostic order that actually resolves this fastest: confirm the SSM Agent is installed and running (systemctl status amazon-ssm-agent on Linux) — most current AMIs include it, but a custom or older AMI might not; confirm the instance profile carries AmazonSSMManagedInstanceCore (or equivalent scoped permissions) — a missing or overly narrow instance profile is the single most common cause; confirm network reachability to the Systems Manager service endpoints — either a public route with the right security group egress rules, or, for a private subnet, that all three required interface VPC endpoints (ssm, ssmmessages, ec2messages) are actually provisioned and their security groups allow traffic from the instance; and finally check the agent's own logs (/var/log/amazon/ssm/amazon-ssm-agent.log) for the specific registration error, since the first three checks resolve the overwhelming majority of cases without ever needing to read agent logs at all.

Cross-Account Fleet Operations at Organization Scale#

Every capability in this part extends beyond a single account the same way Part 1's delegated administrator pattern works generally: a designated administrator account can create associations, run Automation, and view Explorer/OpsCenter data across every account in an Organization, without an engineer needing separate credentials per account. This is what makes "patch every instance in the Organization by Sunday" or "confirm no account has an instance more than 30 days out of patch compliance" an actual answerable, centrally-executed operation rather than fifty individual account-by-account checks — the same organization-wide visibility goal Part 9's centralized-security-tooling account already established, applied here to day-to-day fleet operations instead of security findings specifically.

Fleet Operations Best Practices — The Consolidated Checklist#

• Never provision an SSH key or open port 22/3389 for an instance that only needs interactive access via Session Manager — remove the inbound rule entirely once SSM access is confirmed working.
• Stream every Session Manager session's logs to CloudWatch Logs/S3 for audit — the whole value of removing SSH is undermined if the replacement access path isn't itself logged.
• Always set a concurrency limit and error threshold on Run Command/Automation/Patch Manager operations targeting more than a handful of instances.
• Use a custom patch baseline with a deliberate approval delay rather than auto-approving every patch the moment it's released.
• Confine disruptive patching/reboot work to an explicit maintenance window, communicated to the teams it affects.
• Scope Session Manager IAM access by tag/resource condition per engineer's actual responsibility, never a blanket AmazonSSMFullAccess grant.
• Rebuild golden AMIs on a real cadence via Image Builder rather than letting a "golden" image quietly go stale for months.
• Review Compute Optimizer recommendations on a regular cadence, not just once at initial launch.
• Prefer tag-based or Resource Group targeting over hand-listed instance IDs so newly launched instances are automatically in scope for existing automations.
• Gate any Automation runbook capable of production-impacting change behind Change Manager approval; leave routine read-only diagnostics ungated so the approval friction stays proportional to actual risk.
• Provision the ssm, ssmmessages, and ec2messages interface VPC endpoints for any private-subnet fleet before assuming Session Manager access will work — a missing endpoint is the most common reason a private instance never registers as a managed node.
• Route Systems Manager's own operational signal (patch compliance, OpsItems) into the same CloudWatch alerting infrastructure the rest of the fleet already uses, rather than a separate dashboard nobody checks on a routine basis.
• Keep the SSM Agent itself updated via a standing State Manager association, the same continuous- enforcement mechanism used for everything else in this part.
• Define explicit rollback steps in any Automation runbook capable of leaving partial, destructive state behind on failure — never assume a multi-step runbook either fully succeeds or does nothing at all.
• Extend cross-account visibility (Explorer, delegated Organization-wide patch reporting) before a fleet outgrows what a single-account operational view can meaningfully show.
• Confirm compliance data is actually consumed somewhere (a dashboard, an alarm, a recurring review) — a report nobody looks at provides no more real safety than not generating it at all.
• Treat Windows and Linux fleet management as the same operational model with different documents, not two entirely separate disciplines requiring separate tooling.
• Revisit patch baseline approval delays periodically rather than setting them once and forgetting them.
• Verify Session Manager port forwarding is actually the tool reached for before standing up a new bastion host for a "just this once" access need.
• Review which OpsCenter source integrations are actually enabled — a source that was never wired in (Config, Trusted Advisor) silently means its findings never surface as OpsItems at all.
• Confirm every Distributor-managed package has an owner responsible for keeping its content current, the same way a Docker base image or a golden AMI needs an owner rather than drifting unmaintained.

Part 15 CLI Cheat Sheet#

Common Mistakes and Interview Traps#

Worked Practice Problems#

Problem 1: A security team wants to eliminate all standing SSH access to a 500-instance fleet without losing the ability to audit exactly who accessed which instance and what they did. What's the fix, and what specifically provides the audit trail SSH access alone never did?

Answer: Migrate to Session Manager, remove inbound SSH security group rules once confirmed working, and scope access via IAM policy instead of distributed keys. The audit trail comes from two layers together: CloudTrail logging every StartSession API call (who, when, against which instance) and, separately, session log streaming to CloudWatch Logs/S3 capturing the actual command input/output within each session — neither of which a bare SSH key ever provided on its own.

Problem 2: A patching rollout across a 1,000-instance fleet needs to catch a bad patch early without risking the whole fleet. What Patch Manager/maintenance window configuration prevents a single bad patch from reaching every instance?

Answer: Rate control on the maintenance window task — a concurrency limit (e.g., 10% of targets at a time) paired with an error threshold (e.g., halt after 2% of attempted targets fail). The rollout progresses in controlled batches; if the error threshold trips early in the rollout, the remaining 90%+ of the fleet never receives the bad patch at all.

Problem 3: An operations team wants every EC2 instance's software inventory queryable within minutes during an incident, without manually SSH-ing into instances to check. What Systems Manager capability provides this, and how does it stay current without manual effort?

Answer: Systems Manager Inventory, kept current by a State Manager association re-running AWS-GatherSoftwareInventory on a recurring schedule (e.g., every 30 minutes) across the fleet — the same continuous-reconciliation mechanism State Manager uses for any desired-state enforcement, applied here to keep inventory data fresh rather than a one-time manual snapshot that goes stale immediately.

Problem 4: A database administrator needs occasional, auditable access to a private RDS instance with no public endpoint, without a standing bastion host to maintain. What Session Manager capability solves this, and what's the actual security improvement over a bastion?

Answer: Session Manager port forwarding to a remote host (AWS-StartPortForwardingSessionToRemoteHost), tunneling a local port on the DBA's machine through the SSM Agent's outbound channel to the RDS endpoint. The security improvement over a bastion: there's no standing EC2 instance to patch, no SSH key to manage, and every tunnel session is individually IAM-scoped and CloudTrail-logged — a bastion, by contrast, is itself a persistent piece of infrastructure requiring its own patching and monitoring, and typically offers weaker per-session audit granularity than Session Manager's native logging.

Summary and What's Next#

Systems Manager is the operational backbone SOA-C02 weights most heavily: Session Manager and Run Command remove SSH keys and bastion hosts from the access-control picture entirely, replaced by IAM-scoped, fully audited access. Automation and State Manager turn written runbooks into executable, continuously-enforced documents. Patch Manager applies that same machinery specifically to keeping a fleet's OS and application patches current, safely, inside controlled maintenance windows. And Inventory, Fleet Manager, OpsCenter, Image Builder, and Compute Optimizer round out the fleet-visibility and right-sizing layer — together, the concrete answer to "how does an SRE actually operate hundreds of instances" that this series has been building toward since Part 3. None of it requires new infrastructure to stand up separately — Session Manager, Automation, Patch Manager, and Inventory are all built directly into the same accounts and instances already covered across Parts 1 through 14, activated through IAM and an instance profile rather than a new service to provision and operate on top of everything else.

Part 16 shifts from operating the fleet to paying for it: Cost Explorer, Budgets, Savings Plans vs Reserved Instances, and the per-service cost levers that turn Compute Optimizer's recommendations (and everything else built across this series) into an actual, defensible cost story — the domain SAA-C03, SOA-C02, and SAP-C02 all weight explicitly, and this site's own currently-empty finops-cost content area.