Security & Compliance
A note on this part's depth: Like Part 4 (Networking), security is one of the most heavily tested areas in real SRE, DevOps, and Platform Engineering interviews, and one of the most consequential areas in real production operations. This part goes noticeably deeper than most, covering every major AWS security service, with heavy CLI usage and explicit best practices throughout.
Table of Contents#
- Why Security Gets This Much Depth
- The Defense-in-Depth Model, Applied to AWS
- KMS — Key Management Service, Core Concepts
- Envelope Encryption, Precisely
- KMS Key Policies vs IAM Policies
- Customer Managed Keys vs AWS Managed Keys
- KMS Key Rotation
- Secrets Manager — Managing Application Secrets
- Secrets Manager vs Systems Manager Parameter Store
- AWS WAF — Web Application Firewall
- AWS Shield — DDoS Protection
- GuardDuty — Managed Threat Detection
- Security Hub — Centralized Findings
- Amazon Inspector — Automated Vulnerability Scanning
- Amazon Macie — Sensitive Data Discovery
- CloudTrail — The Account's Audit Log
- CloudTrail Deep Dive: Management vs Data Events
- AWS Config Rules for Security — Recap and Extension
- Network Security Recap: Security Groups, NACLs, and Network Firewall
- Zero Trust Architecture on AWS
- Incident Response on AWS
- Compliance Frameworks and AWS Artifact — Recap and Extension
- A Layered Security Architecture, Fully Worked
- Amazon Detective — Root-Causing a Security Finding
- AWS Firewall Manager — Centralized Security Policy at Scale
- AWS Audit Manager — Continuous Compliance Evidence Collection
- IAM Access Analyzer — External Access, Revisited in Full
- Security Best Practices — The Consolidated Checklist
- Part 9 CLI Cheat Sheet
- A Worked Example: Responding to a Real GuardDuty Finding, End to End
- Common Mistakes
- Worked Practice Problems
- Summary and What's Next
Why Security Gets This Much Depth#
Every previous part of this series touched security in passing — IAM (Part 2), VPC isolation (Part 4), S3 Block Public Access (Part 5), database encryption (Part 6). This part is where those threads get pulled together into a coherent, defense-in-depth security architecture, and where the DEDICATED AWS security services — the ones whose entire job is detection, protection, and compliance — get the full treatment they deserve. This is the direct, concrete AWS implementation of the entire DevSecOps series, now expressed as specific, real AWS services with specific, real CLI commands.
The Defense-in-Depth Model, Applied to AWS#
Directly the AWS-specific realization of the "defense in depth" resilience pattern already covered generically in the Reliability & Architecture Patterns series — worth seeing the full stack in one place before diving into each layer.
Diagram
Why no single layer is ever treated as "the" security control, worth stating explicitly as the core philosophy behind this entire part: a real compromise typically requires an attacker to defeat MULTIPLE independent layers simultaneously — a leaked credential (Layer 1) is far less damaging if the resource it grants access to is also encrypted (Layer 4) and network-isolated (Layer 2), and even a successful breach is far more survivable if it's detected quickly (Layer 5) and fully reconstructable after the fact (Layer 6).
KMS — Key Management Service, Core Concepts#
AWS KMS is the foundational encryption service underlying nearly every other AWS service's encryption-at-rest capability already referenced throughout this series (S3 in Part 5, EBS in Part 5, RDS in Part 6).
Diagram
# Create a customer-managed KMS key aws kms create-key --description "app-encryption-key" \ --tags TagKey=Team,TagValue=payments aws kms create-alias --alias-name alias/app-key \ --target-key-id 1234abcd-12ab-34cd-56ef-1234567890ab # Encrypt/decrypt small pieces of data directly (for larger # data, envelope encryption — next section — is used instead) aws kms encrypt --key-id alias/app-key --plaintext "sensitive-value" \ --query CiphertextBlob --output text | base64 --decode > encrypted.bin aws kms decrypt --ciphertext-blob fileb://encrypted.bin --query Plaintext --output text | base64 --decode
Why "never leaves KMS in plaintext" is worth stating precisely, a genuinely important architectural fact: the actual master key material is generated and stored inside KMS's own hardware security modules (HSMs) and is never extractable, even by AWS itself, even with a support request — every encrypt/decrypt operation happens INSIDE the KMS service boundary, which is precisely what makes it a trustworthy foundation for every other service's encryption.
Envelope Encryption, Precisely#
Already introduced briefly in the DynamoDB deep dive (Databases series, Part 7) — worth a full, precise explanation here, since it's genuinely the mechanism underlying almost every AWS encryption-at-rest feature.
Diagram
Why this two-tier design exists at all, worth stating precisely, a genuinely strong interview answer: KMS itself is relatively slow and rate-limited (each call is a real network round-trip to a highly secure, audited service) — encrypting large volumes of actual DATA directly with KMS on every read/write would be both slow and expensive. Envelope encryption solves this: KMS is only called ONCE per data key (a fast, infrequent operation), while the actual bulk data encryption happens LOCALLY, at full hardware speed, using that data key — combining KMS's strong security guarantees with the performance of local encryption.
KMS Key Policies vs IAM Policies#
A genuinely important, frequently-tested nuance: KMS keys have their OWN resource-based policy (a "key policy"), and — unlike most AWS resources — an IAM policy alone is NEVER sufficient to grant access to a KMS key; the key policy must ALSO explicitly allow it.
Diagram
# A KMS key policy MUST explicitly allow the account's IAM # policies to even be considered — the default key policy # (created automatically) does exactly this via the # "enable IAM policies" root account statement aws kms get-key-policy --key-id alias/app-key --policy-name default
Why this "double gate" exists, worth stating precisely as the reasoning, not just the rule: KMS keys often protect an organization's MOST sensitive data — requiring BOTH an IAM grant AND an explicit key policy grant is a deliberate, extra layer of defense-in-depth specifically for the service most likely to be the last line of defense, directly reusing the "belt and suspenders" philosophy already seen for Block Public Access (Part 5) layered on top of bucket policies.
Customer Managed Keys vs AWS Managed Keys#
AWS Managed Key (aws/s3, etc.) | Customer Managed Key (CMK) | |
|---|---|---|
| Created by | AWS automatically, on first use of a service | You, explicitly |
| Key policy control | None — AWS controls it entirely | Full control — you write the key policy |
| Rotation | Automatic, AWS-controlled schedule | You control (automatic annual, or manual) |
| Cross-account sharing | Not possible | Possible, via key policy |
| Cost | Free | A small monthly charge per key |
| Best fit | Simple, single-account use cases with no special requirements | Cross-account sharing, custom rotation needs, or genuinely sensitive data requiring explicit, auditable key policy control |
Why choosing a Customer Managed Key matters specifically for compliance-sensitive data, worth stating explicitly, directly connecting to the DevSecOps series' compliance discussion: an auditor asking "who can decrypt this data, precisely" can be given a definitive, explicit answer only when the key policy itself is fully under your control — an AWS Managed Key's policy is opaque and not directly inspectable/customizable, which is a real limitation for strict compliance requirements.
KMS Key Rotation#
# Enable automatic annual rotation for a Customer Managed Key aws kms enable-key-rotation --key-id alias/app-key # Check rotation status aws kms get-key-rotation-status --key-id alias/app-key
Why rotation doesn't require re-encrypting existing data, worth stating precisely, a genuinely non-obvious but important fact: KMS transparently keeps OLD key material available for decrypting data that was encrypted under it, while NEW encrypt operations use the newly rotated key material — data encrypted years ago under an old key version remains decryptable without any migration or re-encryption step, since KMS manages this key-version history internally and transparently.
Secrets Manager — Managing Application Secrets#
Already referenced briefly in the DevSecOps series (Part 4) as a cloud-native alternative to HashiCorp Vault — worth the full, AWS-specific treatment here.
# Store a secret aws secretsmanager create-secret \ --name prod/app/db-credentials \ --secret-string '{"username":"app_user","password":"..."}' # Retrieve it at runtime (the application's IAM role, Part 2, # must have secretsmanager:GetSecretValue permission) aws secretsmanager get-secret-value --secret-id prod/app/db-credentials --query SecretString --output text # Enable AUTOMATIC ROTATION — Secrets Manager can invoke a # Lambda function (Part 7) on a schedule to actually CHANGE # the underlying credential (e.g. the database password itself) # AND update the stored secret, atomically aws secretsmanager rotate-secret \ --secret-id prod/app/db-credentials \ --rotation-lambda-arn arn:aws:lambda:us-east-1:123456789012:function:rotate-db-credential \ --rotation-rules AutomaticallyAfterDays=30
Why automatic rotation is Secrets Manager's single most valuable capability, worth stating explicitly, directly connecting to the secret-rotation discussion already introduced generically in the DevSecOps series (Part 4): rotation done RIGHT means actually changing the underlying credential (not just relabeling a stored value) — Secrets Manager's rotation Lambda pattern handles the full, correct sequence (create a new credential, update the actual database/service to accept it, update the stored secret, and only then invalidate the old credential), avoiding the classic rotation bug where the secret store and the actual credential drift out of sync.
Secrets Manager vs Systems Manager Parameter Store#
A genuinely common, real interview question worth having a precise, honest answer for.
| Secrets Manager | Parameter Store (SecureString) | |
|---|---|---|
| Automatic rotation | Yes, native Lambda-based rotation | No — must be built manually |
| Cost | Per-secret monthly charge | Free (standard tier) |
| Cross-account sharing | Native support via resource policies | More limited |
| Versioning | Full version history | Basic version history |
| Best fit | Database credentials, API keys needing rotation | Configuration values, feature flags, secrets where rotation isn't needed |
A genuinely honest, balanced answer worth having ready: "I'd use Secrets Manager for anything needing real rotation — database credentials being the classic case — and Parameter Store for configuration and secrets that don't need automatic rotation, purely to avoid Secrets Manager's per-secret cost for values that don't benefit from its extra capability. It's a real cost-vs-capability tradeoff, not a strict 'always use X' rule."
AWS WAF — Web Application Firewall#
AWS WAF inspects HTTP(S) requests at the ALB (Part 8) or CloudFront (Part 8) layer, filtering based on rules — directly the AWS-native implementation of application-layer defenses against the OWASP Top 10 already covered in the DevSecOps series (Part 2).
# Create a Web ACL with a managed rule group targeting # common attack patterns (SQL injection, XSS — directly # the OWASP Top 10 categories from the DevSecOps series) aws wafv2 create-web-acl \ --name app-protection --scope REGIONAL \ --default-action Allow={} \ --rules '[{"Name":"AWS-AWSManagedRulesCommonRuleSet","Priority":1,"Statement":{"ManagedRuleGroupStatement":{"VendorName":"AWS","Name":"AWSManagedRulesCommonRuleSet"}},"OverrideAction":{"None":{}},"VisibilityConfig":{"SampledRequestsEnabled":true,"CloudWatchMetricsEnabled":true,"MetricName":"CommonRuleSet"}}]' \ --visibility-config SampledRequestsEnabled=true,CloudWatchMetricsEnabled=true,MetricName=app-protection # Associate it with an ALB aws wafv2 associate-web-acl \ --web-acl-arn arn:aws:wafv2:...:regional/webacl/app-protection/abc123 \ --resource-arn arn:aws:elasticloadbalancing:...:loadbalancer/app/app-alb/xyz # A custom rate-based rule — rate limiting at the WAF layer, # directly the AWS-native implementation of the rate limiting # resilience pattern from the Reliability & Architecture # Patterns series aws wafv2 update-web-acl --name app-protection --scope REGIONAL --id abc123 --lock-token xyz \ --rules '[{"Name":"RateLimit","Priority":2,"Statement":{"RateBasedStatement":{"Limit":2000,"AggregateKeyType":"IP"}},"Action":{"Block":{}},"VisibilityConfig":{"SampledRequestsEnabled":true,"CloudWatchMetricsEnabled":true,"MetricName":"RateLimit"}}]'
Why WAF operates at a fundamentally different layer than the security groups/NACLs already covered in Part 4, worth stating precisely: security groups and NACLs understand only IP/port/protocol — WAF understands actual HTTP request CONTENT (headers, body, query strings), letting it detect and block application-layer attack patterns like SQL injection payloads or cross-site scripting attempts that are structurally invisible at the network layer.
AWS Shield — DDoS Protection#
Diagram
Why "cost protection during an attack" is a genuinely underrated, worth-knowing Shield Advanced benefit, worth stating explicitly: a large-scale DDoS attack can trigger legitimate-looking Auto Scaling (Part 3) or CloudFront/data-transfer charges as the infrastructure tries to absorb the flood — Shield Advanced includes a cost protection guarantee specifically covering these scaling-related charges incurred during a declared DDoS event, directly addressing a real, non-obvious financial risk beyond the availability risk itself.
GuardDuty — Managed Threat Detection#
GuardDuty continuously analyzes VPC Flow Logs (Part 4), CloudTrail logs (this part), and DNS logs using machine learning and threat-intelligence feeds, to detect genuinely malicious or anomalous activity — without you needing to write a single detection rule yourself.
aws guardduty create-detector --enable # List active findings aws guardduty list-findings --detector-id abc123 \ --finding-criteria '{"Criterion":{"severity":{"Gte":7}}}' aws guardduty get-findings --detector-id abc123 --finding-ids finding-id-1
| Example finding type | What it means |
|---|---|
UnauthorizedAccess:EC2/SSHBruteForce | An instance is being targeted by SSH brute-force attempts |
CryptoCurrency:EC2/BitcoinTool.B!DNS | An instance is communicating with a known cryptocurrency-mining domain — often a sign of compromise |
Recon:IAMUser/MaliciousIPCaller | An IAM credential is being used from a known-malicious IP address |
Exfiltration:S3/ObjectRead.Unusual | Anomalous, high-volume S3 read activity, possibly indicating data exfiltration |
Why GuardDuty is worth enabling as a near-zero-effort default, worth stating explicitly, directly connecting to the "detection" layer of this part's defense-in-depth model: it requires zero rule-writing (unlike a traditional SIEM you'd have to tune yourself) and continuously improves via AWS's own threat-intelligence updates — a genuinely strong, low-effort addition to any account's security posture, and specifically the kind of automated, always-on detection that a team without a dedicated security operations function can realistically maintain.
Security Hub — Centralized Findings#
As an organization adopts GuardDuty, Inspector, Macie, Config, and third-party security tools, each producing its OWN findings in its own format, aggregating them by hand becomes impractical. Security Hub centralizes findings from all of these into one normalized format and dashboard.
aws securityhub enable-security-hub # Enable a specific compliance standard — directly connecting # to the compliance frameworks already covered in depth in # the DevSecOps series (Part 6) aws securityhub batch-enable-standards \ --standards-subscription-requests '[{"StandardsArn":"arn:aws:securityhub:us-east-1::standards/aws-foundational-security-best-practices/v/1.0.0"}]' # Get an aggregated compliance score aws securityhub get-findings --filters '{"ComplianceStatus":[{"Value":"FAILED","Comparison":"EQUALS"}]}'
Why this is worth enabling org-wide (via delegated administration, already introduced in Part 1) once more than one detection service is in use, worth stating explicitly: it's the practical realization of the "centralized visibility" theme already carried through this entire series (Config aggregation in Part 1, CloudTrail centralization later in this part) — a single security team dashboard showing findings from GuardDuty, Inspector, Macie, and Config together, normalized and prioritized, instead of several disconnected consoles each requiring separate review.
Amazon Inspector — Automated Vulnerability Scanning#
Directly extending the ECR image-scanning discussion already introduced in Part 7 — Inspector is the underlying engine, and it scans more than just container images.
Diagram
aws inspector2 enable --resource-types EC2 ECR LAMBDA aws inspector2 list-findings \ --filter-criteria '{"severity":[{"comparison":"EQUALS","value":"CRITICAL"}]}'
Why continuous scanning matters more than a one-time scan, worth stating explicitly, directly connecting to the SCA (Software Composition Analysis) discussion already covered in the DevSecOps series (Part 2): a package with no known vulnerabilities today can have a new CVE disclosed against it tomorrow — Inspector continuously RE-EVALUATES already-deployed EC2 instances, container images, and Lambda functions against the latest vulnerability database, rather than only scanning once at build/deploy time, catching newly-disclosed vulnerabilities in resources that have been running unchanged for months.
Amazon Macie — Sensitive Data Discovery#
Macie uses machine learning to scan S3 (Part 5) for sensitive data — PII, credentials, financial information — and flags where it exists and how it's protected.
aws macie2 enable-macie # Create a classification job scanning specific buckets # for sensitive data patterns aws macie2 create-classification-job \ --job-type ONE_TIME \ --s3-job-definition '{"bucketDefinitions":[{"accountId":"123456789012","buckets":["customer-uploads"]}]}' \ --name pii-discovery-scan
Why this matters specifically for compliance requirements like GDPR (already covered in the DevSecOps series, Part 6), worth stating explicitly: "where is our customers' personal data actually stored" is a question many organizations genuinely cannot answer confidently by memory alone, especially in buckets accumulated over years by many different teams — Macie answers it empirically, by actually scanning content, rather than relying on institutional knowledge or documentation that may be outdated or incomplete.
CloudTrail — The Account's Audit Log#
CloudTrail records every API call made against an AWS account — genuinely the foundational audit capability underlying nearly every other security and compliance capability in this part.
# Create a trail, delivering logs to S3 (ideally in the # dedicated log-archive account from Part 1's landing zone) aws cloudtrail create-trail \ --name org-audit-trail \ --s3-bucket-name log-archive-cloudtrail \ --is-multi-region-trail \ --is-organization-trail \ --enable-log-file-validation aws cloudtrail start-logging --name org-audit-trail # Query recent activity for a specific user/role — genuinely # the first place to look during an incident investigation aws cloudtrail lookup-events \ --lookup-attributes AttributeKey=Username,AttributeValue=alice \ --max-results 20
Why --is-organization-trail combined with the dedicated log-archive account from Part 1 matters so much, worth stating explicitly, directly reinforcing that part's tamper-evident logging discussion: an organization trail automatically captures EVERY member account's API activity into ONE centralized, immutable log location — a compromised workload account's own local CloudTrail configuration cannot be disabled or tampered with to hide activity, since the actual trail and its destination bucket are controlled centrally, from a different account entirely.
CloudTrail Deep Dive: Management vs Data Events#
A genuinely important, precise distinction worth knowing, since it directly affects both cost and what's actually visible in an investigation.
| Event type | What it captures | Default logging |
|---|---|---|
| Management events | Control-plane operations — creating a bucket, launching an instance, changing an IAM policy | Logged by default, free |
| Data events | Data-plane operations — individual GetObject/PutObject calls on S3, individual Invoke calls on Lambda | NOT logged by default — must be explicitly enabled, and incurs a per-event cost |
# Enable data event logging for S3 — genuinely necessary if # you need to answer "who read THIS specific object" during # an investigation, since management events alone won't show it aws cloudtrail put-event-selectors \ --trail-name org-audit-trail \ --event-selectors '[{"ReadWriteType":"All","IncludeManagementEvents":true,"DataResources":[{"Type":"AWS::S3::Object","Values":["arn:aws:s3:::sensitive-data-bucket/"]}]}]'
Why this distinction is a genuinely common, costly gap discovered too late, worth stating explicitly: a team investigating "was this specific sensitive object in S3 ever accessed by someone unauthorized" often discovers, mid-investigation, that data events were never enabled — meaning individual object-level access simply isn't in the CloudTrail history at all, only the bucket's creation and policy changes are. Enabling data events specifically for sensitive, high-value resources BEFORE an incident, not after, is the only way to guarantee this visibility exists when it's actually needed.
AWS Config Rules for Security — Recap and Extension#
Already introduced generically in Part 1 — worth a specific, security-focused callout here, since Config Rules are genuinely one of the most practical, proactive security tools in this entire part.
# A few genuinely high-value, security-specific managed rules aws configservice put-config-rule --config-rule '{"ConfigRuleName":"restricted-ssh","Source":{"Owner":"AWS","SourceIdentifier":"INCOMING_SSH_DISABLED"}}' aws configservice put-config-rule --config-rule '{"ConfigRuleName":"iam-root-mfa","Source":{"Owner":"AWS","SourceIdentifier":"ROOT_ACCOUNT_MFA_ENABLED"}}' aws configservice put-config-rule --config-rule '{"ConfigRuleName":"encrypted-volumes","Source":{"Owner":"AWS","SourceIdentifier":"ENCRYPTED_VOLUMES"}}' # Config can also AUTO-REMEDIATE certain non-compliant findings aws configservice put-remediation-configurations \ --remediation-configurations '[{"ConfigRuleName":"restricted-ssh","TargetType":"SSM_DOCUMENT","TargetId":"AWS-DisablePublicAccessForSecurityGroup","Automatic":true}]'
Why auto-remediation is worth treating as a real, deliberate escalation from "detect" to "prevent," worth stating precisely: a Config rule alone only REPORTS non-compliance — pairing it with an auto-remediation action (via an SSM document, already covered in Part 3) closes the loop entirely, automatically fixing certain classes of drift (like an accidentally-opened SSH port) without waiting for a human to notice the finding and act on it.
Network Security Recap: Security Groups, NACLs, and Network Firewall#
Already covered in full depth in Part 4 — worth a brief, explicit connection here, since these ARE this part's "network" defense-in-depth layer, not a separate concern.
- Security groups (Part 4): stateful, resource-level allow rules — the primary, day-to-day network access control.
- NACLs (Part 4): stateless, subnet-level allow/deny rules — a coarser backstop.
- AWS Network Firewall (Part 4): deep packet inspection, domain filtering, IDS/IPS — for compliance or threat-detection needs security groups/NACLs structurally cannot express.
Zero Trust Architecture on AWS#
Worth pulling together everything from this part and Part 2/4 into the explicit "Zero Trust" framing, since it's a genuinely common, modern interview topic.
Diagram
Why this framing directly ties together several AWS services already covered across this series, worth stating explicitly as a genuinely strong, synthesizing interview answer: "AWS enables Zero Trust through a combination of IAM's identity-based, least-privilege access (Part 2) instead of network-location-based trust, PrivateLink's identity-and-policy-gated service access (Part 4) instead of broad network reachability, SSM Session Manager's IAM-authenticated instance access (Part 3) instead of network-perimeter-based SSH trust, and mTLS-based service mesh authentication (Kubernetes Deep Dive series, Part 4) for service-to-service calls — the consistent theme across every one of these is verifying WHO is making a request, not just WHERE it's coming from."
Incident Response on AWS#
Directly extending the Incident Response Process already covered in exhaustive depth in the Incident Management series — worth the AWS-specific tooling that supports it.
# During a suspected compromise: immediately isolate an # instance by swapping its security group to a "quarantine" # group with NO inbound/outbound rules, preserving it for # forensics without further network exposure aws ec2 modify-instance-attribute \ --instance-id i-0123456789abcdef0 --groups sg-quarantine123 # Snapshot the instance's EBS volumes (Part 5) BEFORE any # further action, preserving forensic evidence aws ec2 create-snapshot --volume-id vol-0123456789abcdef0 \ --description "forensic-snapshot-incident-2026-08-19" # Revoke potentially compromised temporary credentials # immediately, for a specific role aws iam put-role-policy --role-name CompromisedRole \ --policy-name DenyAll --policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Deny","Action":"*","Resource":"*"}]}'
Why "isolate, don't immediately terminate" is the correct first move, worth stating explicitly, directly connecting to the Incident Response Process series' emphasis on preserving evidence: terminating a compromised instance immediately destroys volatile forensic evidence (running processes, network connections, memory state) that could be critical to understanding HOW the compromise happened — quarantining via security group isolation stops further damage while preserving the instance for investigation, exactly the same "contain, then investigate, then remediate" sequencing already covered generically in the Incident Management series.
Compliance Frameworks and AWS Artifact — Recap and Extension#
Already covered generically in the DevSecOps series (Part 6) and AWS Artifact specifically in Part 1 — worth one additional, AWS-security-specific connection: Security Hub's compliance standards (shown earlier in this part) directly, continuously evaluate an account against frameworks like the AWS Foundational Security Best Practices, CIS AWS Foundations Benchmark, and PCI-DSS — turning what used to be a periodic, manual audit exercise into continuous, automated compliance monitoring, directly the "shift-left" philosophy from the DevSecOps series applied to compliance itself, not just application security.
A Layered Security Architecture, Fully Worked#
Bringing this entire part together into one concrete, complete design.
Diagram
Every layer in this diagram maps to a specific part or section already covered — worth narrating it end to end as a single, coherent answer to "design AWS security for a production web application," rather than listing services in isolation.
Amazon Detective — Root-Causing a Security Finding#
GuardDuty tells you SOMETHING suspicious happened; Amazon Detective helps you understand the FULL STORY around it — automatically building a visual, interconnected graph of resource behavior over time, so an investigator doesn't have to manually correlate CloudTrail, VPC Flow Logs, and GuardDuty findings by hand.
Diagram
aws detective create-graph aws detective list-graphs
Why this matters specifically for incident response speed, worth stating explicitly, directly connecting to the MTTR discussion from the Incident Management series: manually correlating CloudTrail events, VPC Flow Logs (Part 4), and GuardDuty findings by hand, across potentially months of history, is exactly the kind of slow, error-prone manual investigation work that directly extends an incident's time-to-resolution — Detective automates that correlation, turning what could be hours of manual log-diving into a visual graph an investigator can traverse in minutes.
AWS Firewall Manager — Centralized Security Policy at Scale#
For an organization with dozens or hundreds of accounts (the multi-account landing zone from Part 1), manually configuring WAF rules, security groups, and Shield Advanced protection consistently across every single account becomes impractical. Firewall Manager centrally manages and enforces these policies across an entire Organization.
aws fms put-policy --policy '{ "PolicyName": "org-wide-waf-baseline", "SecurityServicePolicyData": {"Type": "WAFV2", "ManagedServiceData": "{\"type\":\"WAFV2\",\"defaultAction\":{\"type\":\"ALLOW\"},\"preProcessRuleGroups\":[{\"managedRuleGroupIdentifier\":{\"vendorName\":\"AWS\",\"managedRuleGroupName\":\"AWSManagedRulesCommonRuleSet\"}}]}", "ResourceType": "AWS::ElasticLoadBalancingV2::LoadBalancer", "ExcludeResourceTags": false, "RemediationEnabled": true }'
Why RemediationEnabled: true matters, worth stating explicitly, directly extending the auto-remediation theme already introduced for Config earlier in this part: without it, Firewall Manager only REPORTS accounts/resources that don't comply with the org-wide policy; with it enabled, it automatically APPLIES the required WAF/security-group/Shield configuration to any new or drifted resource across the entire Organization — turning security baseline enforcement from a per-account manual task into a genuinely centralized, self-healing guarantee.
AWS Audit Manager — Continuous Compliance Evidence Collection#
Directly extending the compliance discussion from this part and the DevSecOps series (Part 6) — Audit Manager automates the collection of EVIDENCE for a compliance audit (SOC 2, PCI-DSS, HIPAA), rather than a team manually gathering screenshots and exports when an auditor asks.
aws auditmanager create-assessment \ --name "SOC2-2026-Assessment" \ --framework-id soc2-framework-id \ --scope '{"awsAccounts":[{"id":"123456789012"}]}'
Why this is worth distinguishing precisely from Security Hub's compliance standards (already covered earlier in this part): Security Hub continuously evaluates TECHNICAL compliance (is this S3 bucket encrypted, is MFA enabled) — Audit Manager collects and organizes the actual EVIDENCE ARTIFACTS an auditor needs to see (configuration snapshots, IAM policies, access logs) into a structured assessment report, directly reducing the manual "evidence gathering" burden that's traditionally one of the most time-consuming parts of a real compliance audit cycle, as already discussed generically in the DevSecOps series' compliance part.
IAM Access Analyzer — External Access, Revisited in Full#
Already introduced in Part 2 as an auditing tool — worth a fuller, security-focused treatment here, since it's genuinely one of the highest-signal, lowest-effort security tools available.
# Create an analyzer scoped to the whole ORGANIZATION, # not just one account aws accessanalyzer create-analyzer --analyzer-name org-analyzer --type ORGANIZATION # Findings specifically flag resources (S3 buckets, IAM roles, # KMS keys, Lambda functions, and more) accessible from # OUTSIDE the trusted zone (the account or organization) aws accessanalyzer list-findings --analyzer-arn arn:aws:access-analyzer:us-east-1:123456789012:analyzer/org-analyzer \ --filter '{"isPublic":{"eq":["true"]}}'
Why this is worth running continuously (not just once) as a standing, high-signal alert source, worth stating explicitly: unlike Config rules (which check specific, predefined conditions) or GuardDuty (which detects behavioral anomalies), Access Analyzer's specific job is answering exactly one question extremely well — "is ANYTHING in this account/organization reachable from OUTSIDE the trust boundary that shouldn't be" — directly catching the exact class of mistake (an accidentally public S3 bucket, an overly permissive cross-account IAM role) responsible for a large share of real-world cloud security incidents.
Security Best Practices — The Consolidated Checklist#
A dense, exam-and-interview-ready summary of every recommendation made across this part.
- Enable GuardDuty, Security Hub, Inspector, and IAM Access Analyzer as near-zero-effort defaults on every account — each requires no custom rule-writing and continuously improves via AWS-managed intelligence.
- Use Customer Managed KMS keys for anything requiring explicit, auditable key policy control — especially cross-account sharing or strict compliance requirements.
- Remember KMS's "double gate" — an IAM policy alone is never sufficient; the key policy must also explicitly allow the principal.
- Use Secrets Manager (not Parameter Store) for anything genuinely needing automatic rotation.
- Enable CloudTrail as an organization trail, delivering to a dedicated log-archive account (Part 1) — never rely on a per-account trail that a compromised account could tamper with.
- Explicitly enable CloudTrail data events for sensitive resources proactively, before they're needed during an investigation.
- Layer WAF managed rule groups with a custom rate-based rule for genuinely complete application-layer protection.
- Use Firewall Manager to enforce security baselines centrally once an organization has more than a handful of accounts.
- Isolate, snapshot, and investigate before terminating a suspected-compromised resource — preserve forensic evidence first.
- Treat Access Analyzer findings as high-priority — they specifically flag externally-reachable resources, a leading real-world cause of cloud security incidents.
Part 9 CLI Cheat Sheet#
| Area | Command | Purpose |
|---|---|---|
| KMS | aws kms create-key / create-alias | Create an encryption key |
| KMS | aws kms enable-key-rotation | Enable automatic annual rotation |
| Secrets | aws secretsmanager create-secret / get-secret-value | Store and retrieve a secret |
| Secrets | aws secretsmanager rotate-secret | Enable automatic rotation |
| WAF | aws wafv2 create-web-acl / associate-web-acl | Create and attach a Web ACL |
| GuardDuty | aws guardduty create-detector --enable | Enable threat detection |
| Security Hub | aws securityhub enable-security-hub | Enable centralized findings |
| Inspector | aws inspector2 enable | Enable continuous vulnerability scanning |
| Macie | aws macie2 create-classification-job | Scan S3 for sensitive data |
| CloudTrail | aws cloudtrail create-trail / start-logging | Enable audit logging |
| CloudTrail | aws cloudtrail lookup-events | Investigate recent account activity |
| Config | aws configservice put-config-rule | Add a compliance rule |
| Incident response | aws ec2 modify-instance-attribute --groups | Quarantine a compromised instance |
| Access review | aws accessanalyzer create-analyzer / list-findings | Flag externally-reachable resources |
| Policy at scale | aws fms put-policy | Enforce a security policy org-wide |
| Investigation | aws detective create-graph | Build a behavior graph for an investigation |
| Compliance evidence | aws auditmanager create-assessment | Automate audit evidence collection |
A Worked Example: Responding to a Real GuardDuty Finding, End to End#
Bringing this entire part together into one concrete, realistic incident-response walkthrough — genuinely worth internalizing as a repeatable playbook.
The finding: GuardDuty reports UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration.OutsideAWS — meaning an EC2 instance's temporary IAM credentials (Part 2's instance profile pattern) are being used from an IP address OUTSIDE AWS entirely, a strong signal those credentials were stolen.
Diagram
Walking through each step's explicit reasoning: isolation first (per this part's earlier incident-response section) stops ongoing damage without destroying evidence; revoking the IAM session immediately (via an explicit Deny policy attached to the role, since temporary credentials can't be individually "deleted" the way a static access key can) closes the actual exposed access path; Detective answers "what did the attacker actually DO with these credentials" far faster than manual CloudTrail review; and the final rotation step is scoped precisely by what the investigation actually found reachable — not a guess, but an evidence-based response.
Why this specific finding type is worth knowing by name, worth stating explicitly: "credentials used from outside AWS" is a particularly high-confidence, high-severity signal — legitimate AWS SDK/CLI usage of an instance's own temporary credentials should, by definition, always originate from within AWS's own network, making external usage a strong, low-false-positive indicator of actual credential theft, unlike some other finding types that may warrant more investigation before concluding malicious intent.
Common Mistakes#
| Mistake | Why It's Wrong | Fix |
|---|---|---|
| Assuming an IAM policy alone grants KMS access | KMS key policies are a separate, mandatory gate — an IAM Allow is never sufficient by itself | Always confirm the key policy also explicitly allows the intended principal |
| Storing frequently-rotated credentials (database passwords) in Parameter Store instead of Secrets Manager | Parameter Store has no native automatic rotation — someone has to build and maintain that themselves | Use Secrets Manager for anything genuinely needing rotation |
| Never enabling CloudTrail data events for sensitive S3 buckets | Individual object-level access ("who read this file") is invisible without them, discovered too late during an investigation | Enable data events proactively for sensitive resources, before an incident, not during one |
| Treating GuardDuty/Inspector/Macie findings as three separate, disconnected dashboards | Slows down and fragments security review as tooling grows | Enable Security Hub to centralize and normalize findings across all detection services |
| Terminating a suspected-compromised instance immediately | Destroys volatile forensic evidence needed to understand how the compromise happened | Isolate via security group quarantine and snapshot first, investigate, then remediate |
| Relying on WAF managed rules alone without a custom rate-based rule | Managed rules catch known attack signatures but not high-volume abuse from a single source | Layer a rate-based rule on top of managed rule groups for genuinely complete protection |
| Treating compliance as a periodic, manual audit exercise | Misses drift that occurs between audit cycles | Enable Security Hub's compliance standards for continuous, automated compliance monitoring |
Worked Practice Problems#
Problem 1: A security team grants an application's IAM role kms:Decrypt permission via an IAM policy, expecting this alone to let the application decrypt data protected by a specific Customer Managed Key. The application reports AccessDeniedException errors when attempting to decrypt. What's the most likely cause, and what's the fix?
Answer: The KMS key's own key policy almost certainly doesn't explicitly grant this principal access — unlike most AWS resources, KMS enforces a genuinely important "double gate": an IAM policy alone is never sufficient to grant access to a key; the key's resource-based key policy must ALSO explicitly allow the principal (or explicitly delegate that decision to IAM policies via the standard root-account statement). The fix is reviewing and updating the KMS key policy to explicitly include the application's IAM role as an allowed principal for the kms:Decrypt action, alongside the existing IAM policy grant — both gates need to agree before access is actually permitted.
Problem 2: Months after a data breach investigation, a security team discovers they cannot determine which specific objects in a sensitive S3 bucket were accessed by a compromised credential, because only bucket-level events (creation, policy changes) appear in CloudTrail — individual object read/write activity is completely absent from the logs. What configuration gap caused this, and how should it be addressed for the future?
Answer: CloudTrail data events for S3 were never enabled on this bucket — by default, CloudTrail only captures management (control-plane) events, and individual GetObject/PutObject calls (data-plane events) require explicit, separate configuration, at an additional per-event cost. This is exactly why enabling data events for genuinely sensitive resources needs to happen PROACTIVELY, as part of standard security baseline configuration, not reactively after an incident already occurred — going forward, any bucket holding sensitive data should have data event logging enabled by default, ideally enforced via a Config rule or landing zone baseline (Part 1) rather than left to individual team discretion.
Problem 3: During a suspected compromise, an on-call engineer's first instinct is to immediately terminate the affected EC2 instance to "stop the bleeding" as fast as possible. A more experienced colleague intervenes and recommends a different first step. What's the more experienced colleague's likely reasoning, and what should happen instead?
Answer: Immediately terminating the instance destroys volatile forensic evidence — running processes, active network connections, and in-memory state — that could be essential to understanding exactly how the compromise happened, what the attacker actually did, and what else might be affected. This directly mirrors the "contain, then investigate, then remediate" sequencing already established in the Incident Management series' incident response process. The better first step is isolating the instance (swapping its security group to a quarantine group with no inbound/outbound rules) to stop any further damage or lateral movement, while also snapshotting its EBS volumes to preserve a forensic copy — only after evidence is preserved and the immediate threat is contained should the instance actually be terminated and replaced.
Problem 4: A security team receives a GuardDuty finding indicating unusual API activity from a specific IAM role over the past several weeks, but manually reconstructing exactly what that role did — which resources it touched, what data it accessed, whether it interacted with other suspicious principals — by cross-referencing CloudTrail and VPC Flow Logs by hand is taking the investigating engineer most of a day. What AWS service would directly address this specific bottleneck, and how?
Answer: Amazon Detective, specifically built for this exact bottleneck. Rather than manually correlating CloudTrail events and VPC Flow Log entries across potentially months of history, Detective automatically constructs a visual behavior graph of everything the flagged IAM role (or any resource) has done over time — every API call, every network connection, every related resource — letting the investigator traverse that graph visually instead of reconstructing it by hand from raw logs. This directly reduces investigation time from what could be most of a day of manual log correlation down to minutes of graph exploration, a meaningful, concrete improvement to the incident's overall time-to-resolution (MTTR, Incident Management series).
Problem 5: An organization with 60 AWS accounts wants every account's public-facing load balancers to have a consistent, mandatory WAF baseline applied, but a review finds enforcement has been inconsistent — some teams configured WAF correctly, others never did, and there's no reliable way to know which accounts are compliant without manually checking each one. What AWS service directly solves both the enforcement and the ongoing-drift problem here?
Answer: AWS Firewall Manager, configured with a WAF policy applied at the Organization level with remediation enabled. Rather than relying on each of the 60 individual teams to correctly and consistently configure WAF themselves — the root cause of the inconsistency described — Firewall Manager centrally defines the required WAF baseline once and automatically applies it across every in-scope resource organization-wide, including new load balancers created in the future and any resource that drifts out of compliance later. This converts the problem from "hope every team remembers and does this correctly" into a structurally enforced, centrally managed guarantee — directly the same governance-by-construction philosophy already applied to CIDR planning via IPAM in Part 4 and tagging via Tag Policies in Part 1, now applied to security policy specifically.
Summary and What's Next#
- AWS security follows a genuine defense-in-depth model: identity (Part 2), network (Part 4), perimeter (WAF/Shield), data (KMS/Secrets Manager), detection (GuardDuty/Inspector/Macie), and audit (CloudTrail/Config) — no single layer is ever "the" control.
- KMS's envelope encryption pattern — a fast, local data key wrapped by a slower, highly secure master key — underlies nearly every AWS service's encryption-at-rest capability; KMS key policies are a mandatory, separate gate from IAM policies.
- Secrets Manager's automatic rotation correctly handles the full credential-change sequence; Parameter Store is the lighter-weight alternative for secrets that don't need rotation.
- WAF filters at the application layer (content-aware); Shield protects against network/transport-layer DDoS, with Shield Advanced adding cost protection during an attack.
- GuardDuty, Inspector, and Macie provide zero-rule-writing, continuously updated threat/vulnerability/sensitive-data detection; Security Hub centralizes their findings into one normalized view.
- CloudTrail is the foundational audit log — data events must be explicitly enabled for sensitive resources, ideally before they're ever needed, not after an incident.
- Zero Trust on AWS means verifying identity on every request (IAM, PrivateLink, SSM, mTLS) rather than trusting based on network location alone.
- Incident response on AWS follows the same contain-investigate-remediate sequence already covered generically in the Incident Management series, with specific AWS tooling (security group quarantine, EBS snapshots, credential revocation) supporting each step.
Continue to Part 10 (10-monitoring-logging-and-tracing.md) to see how AWS's observability services — CloudWatch, X-Ray, and centralized logging — apply the Observability series' concepts concretely, building directly on the security-relevant logging already introduced in this part.