CI/CD, IaC & Messaging

A note on scope: CI/CD fundamentals, deployment strategies, Terraform, and GitOps already received exhaustive treatment in the Automation, CI/CD & GitOps series. This part covers AWS's OWN native CI/CD and IaC tooling (CodePipeline, CodeBuild, CodeDeploy, CloudFormation, CDK) as concrete alternatives/complements to the vendor-neutral tools already covered there, plus AWS's messaging services.

Table of Contents#

1. AWS-Native CI/CD vs Third-Party Tools
2. CodeCommit — A Brief, Practical Note
3. CodeBuild — Managed Build Service
4. CodeDeploy — Managed Deployment Service
5. CodePipeline — Orchestrating the Full Pipeline
6. CloudFormation — AWS-Native IaC
7. CloudFormation StackSets — Multi-Account IaC
8. The AWS CDK — Infrastructure as Real Code
9. CloudFormation/CDK vs Terraform
10. SQS — Simple Queue Service
11. SQS Standard vs FIFO Queues
12. SQS Dead-Letter Queues and Visibility Timeout
13. SNS — Simple Notification Service
14. SNS Fan-Out Pattern
15. EventBridge — AWS's Event Bus
16. EventBridge Rules and Pattern Matching
17. Choosing Between SQS, SNS, and EventBridge
18. Kinesis — Real-Time Streaming Data
19. A Full Worked Example: A Complete AWS-Native Deployment Pipeline
20. Pipeline Security: IAM Roles and Cross-Account Deployment
21. CI/CD and Messaging Best Practices — The Consolidated Checklist
22. Part 11 CLI Cheat Sheet
23. Common Mistakes
24. Worked Practice Problems
25. Summary and What's Next

---

AWS-Native CI/CD vs Third-Party Tools#

Worth an honest framing before diving in: most real organizations use SOME combination of GitHub Actions/GitLab CI (already covered generically in the Automation series) alongside AWS-native services, rather than exclusively one or the other. AWS-native CI/CD tooling is worth knowing precisely because it appears constantly in AWS-centric job postings and interviews, and because it integrates with IAM (Part 2) more deeply and natively than most third-party alternatives.

---

CodeCommit — A Brief, Practical Note#

Worth knowing it EXISTED as AWS's own managed Git repository service — but worth an equally important, honest note: AWS stopped onboarding new CodeCommit customers in 2024, effectively deprecating it for new adoption. Most AWS-centric pipelines today source code from GitHub, GitLab, or Bitbucket instead, integrated directly into CodePipeline (covered shortly). This is worth knowing specifically so you don't recommend CodeCommit for a new project in an interview or real design.

---

CodeBuild — Managed Build Service#

CodeBuild runs build/test commands in a managed, ephemeral container — directly the AWS-native equivalent of a GitHub Actions runner or GitLab CI job already covered generically in the Automation series (Part 1).

Copy
cat <<'BUILDSPEC'
version: 0.2
phases:
  install:
    commands:
      - echo "Installing dependencies"
  build:
    commands:
      - docker build -t $ECR_REPO:$CODEBUILD_RESOLVED_SOURCE_VERSION .
      - docker push $ECR_REPO:$CODEBUILD_RESOLVED_SOURCE_VERSION
  post_build:
    commands:
      - echo "Build complete"
artifacts:
  files:
    - imagedefinitions.json
BUILDSPEC

aws codebuild create-project \
  --name app-build \
  --source '{"type":"GITHUB","location":"https://github.com/org/app.git","buildspec":"buildspec.yml"}' \
  --artifacts '{"type":"S3","location":"build-artifacts-bucket"}' \
  --environment '{"type":"LINUX_CONTAINER","image":"aws/codebuild/standard:7.0","computeType":"BUILD_GENERAL1_MEDIUM"}' \
  --service-role arn:aws:iam::123456789012:role/CodeBuildRole

Why the buildspec.yml file's structure directly mirrors the "anatomy of a pipeline" concept already covered generically in the Automation series (Part 1), worth stating explicitly: install/build/post_build are just AWS's specific vocabulary for the same universal pipeline STAGES concept already covered there — every CI system, regardless of vendor, breaks a build down into roughly the same conceptual phases.

---

CodeDeploy — Managed Deployment Service#

CodeDeploy automates the actual DEPLOYMENT step, supporting EC2/on-premises, ECS (Part 7), and Lambda (Part 7) targets — directly implementing the deployment strategies already covered in exhaustive depth in the Automation series (Part 1).

Copy
# An appspec.yml for an ECS blue-green deployment — directly
# the AWS-native implementation of the Blue-Green Deployment
# strategy already covered generically in the Automation series
cat <<'APPSPEC'
version: 0.0
Resources:
  - TargetService:
      Type: AWS::ECS::Service
      Properties:
        TaskDefinition: <TASK_DEFINITION>
        LoadBalancerInfo:
          ContainerName: "my-app"
          ContainerPort: 8080
APPSPEC

aws deploy create-deployment-group \
  --application-name app-ecs \
  --deployment-group-name production \
  --deployment-config-name CodeDeployDefault.ECSAllAtOnce \
  --ecs-services clusterName=production-cluster,serviceName=my-app-service \
  --blue-green-deployment-configuration '{"terminateBlueInstancesOnDeploymentSuccess":{"action":"TERMINATE","terminationWaitTimeInMinutes":10}}'

Why terminationWaitTimeInMinutes is worth stating precisely, directly connecting to the near-instant-rollback advantage of blue-green deployments already covered in the Automation series (Part 1): keeping the "blue" (old) environment running for a grace period after cutover means a rollback during that window is exactly as fast as blue-green promised — simply switching traffic back — rather than needing to redeploy the old version from scratch if a problem is discovered shortly after the switch.

---

CodePipeline — Orchestrating the Full Pipeline#

CodePipeline ties source, build (CodeBuild), and deploy (CodeDeploy) stages together into one orchestrated, visualized pipeline — the AWS-native equivalent of a full GitHub Actions/GitLab CI workflow.

Diagram

Copy
aws codepipeline create-pipeline --pipeline '{
  "name": "app-pipeline",
  "roleArn": "arn:aws:iam::123456789012:role/CodePipelineRole",
  "artifactStore": {"type": "S3", "location": "pipeline-artifacts-bucket"},
  "stages": [
    {"name": "Source", "actions": [{"name": "GitHub", "actionTypeId": {"category": "Source", "owner": "ThirdParty", "provider": "GitHub", "version": "1"}, "outputArtifacts": [{"name": "SourceOutput"}]}]},
    {"name": "Build", "actions": [{"name": "CodeBuild", "actionTypeId": {"category": "Build", "owner": "AWS", "provider": "CodeBuild", "version": "1"}, "inputArtifacts": [{"name": "SourceOutput"}], "outputArtifacts": [{"name": "BuildOutput"}]}]},
    {"name": "Deploy", "actions": [{"name": "CodeDeploy", "actionTypeId": {"category": "Deploy", "owner": "AWS", "provider": "CodeDeploy", "version": "1"}, "inputArtifacts": [{"name": "BuildOutput"}]}]}
  ]
}'

Why the "manual approval gate" stage type is worth calling out explicitly, directly connecting to Continuous Delivery vs Continuous Deployment already covered in the Automation series (Part 1): a pipeline with a manual approval step before production is precisely CONTINUOUS DELIVERY (every change is automatically build/test-verified and READY to deploy, but a human explicitly triggers the final production push) — removing that gate entirely would make it CONTINUOUS DEPLOYMENT, the same distinction already covered generically, just expressed as a real CodePipeline stage configuration.

---

CloudFormation — AWS-Native IaC#

CloudFormation is AWS's own, native Infrastructure as Code service — directly implementing the declarative IaC concepts already covered in exhaustive depth in the Automation series (Part 2), using AWS's own YAML/JSON template format instead of Terraform's HCL.

Copy
# A CloudFormation template — declarative, exactly like the
# Terraform concepts already covered generically
AWSTemplateFormatVersion: '2010-09-09'
Resources:
  AppBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: my-app-bucket
      VersioningConfiguration:
        Status: Enabled
Outputs:
  BucketArn:
    Value: !GetAtt AppBucket.Arn

Copy
aws cloudformation create-stack \
  --stack-name app-infra --template-body file://template.yaml

aws cloudformation describe-stacks --stack-name app-infra \
  --query 'Stacks[0].StackStatus'

# Update via a CHANGE SET first — see the diff BEFORE applying,
# directly the CloudFormation equivalent of `terraform plan`
aws cloudformation create-change-set \
  --stack-name app-infra --change-set-name update-1 \
  --template-body file://template-v2.yaml
aws cloudformation describe-change-set --stack-name app-infra --change-set-name update-1
aws cloudformation execute-change-set --stack-name app-infra --change-set-name update-1

Why Change Sets are worth stating as directly, precisely the same concept as terraform plan, worth stating explicitly: both let you preview EXACTLY what will change (resources created, modified, or destroyed) before actually applying it — directly the same "never apply blind" discipline already covered as a best practice in the Automation series' Terraform workflow discussion.

---

CloudFormation StackSets — Multi-Account IaC#

Directly solving the same "apply this consistently across many accounts" problem already covered for Firewall Manager (Part 9) and Tag Policies (Part 1) — StackSets deploy the SAME CloudFormation template across many accounts and regions from one central place.

Copy
aws cloudformation create-stack-set \
  --stack-set-name baseline-security-config \
  --template-body file://security-baseline.yaml \
  --permission-model SERVICE_MANAGED \
  --auto-deployment Enabled=true,RetainStacksOnAccountRemoval=false

aws cloudformation create-stack-instances \
  --stack-set-name baseline-security-config \
  --deployment-targets OrganizationalUnitIds=ou-abc123 \
  --regions us-east-1 eu-west-1

Why permission-model SERVICE_MANAGED combined with auto-deployment Enabled=true matters, worth stating explicitly, directly connecting to the landing zone automation theme from Part 1: this means a NEW account created anywhere under the specified OU AUTOMATICALLY receives this baseline stack the moment it's created — turning "every new account gets our standard security config" from a manual onboarding checklist item into a structurally guaranteed outcome, the same governance-by-construction philosophy already applied via IPAM (Part 4) and Tag Policies (Part 1).

---

The AWS CDK — Infrastructure as Real Code#

The Cloud Development Kit (CDK) lets you define infrastructure using a REAL programming language (TypeScript, Python, Java, Go) instead of YAML/JSON — it compiles DOWN to a CloudFormation template under the hood.

Copy
from aws_cdk import Stack, aws_s3 as s3
from constructs import Construct

class AppStack(Stack):
    def __init__(self, scope: Construct, id: str, **kwargs):
        super().__init__(scope, id, **kwargs)
        bucket = s3.Bucket(self, "AppBucket", versioned=True)

Copy
cdk synth   # generates the underlying CloudFormation template
cdk diff    # shows what would change — the CDK's own change-set preview
cdk deploy  # synthesizes AND deploys in one step

Why writing infrastructure in a real programming language is a genuinely meaningful capability upgrade over raw YAML, worth stating explicitly: it enables real loops, conditionals, functions, and unit tests around infrastructure definitions — something YAML/JSON-based CloudFormation (or HCL-based Terraform) can only approximate with more limited templating constructs — at the real cost of needing an actual programming language runtime and more sophisticated tooling than a plain declarative file.

---

CloudFormation/CDK vs Terraform#

A genuinely common, real architectural decision, directly extending the Terraform discussion from the Automation series (Part 2).

Why "no separate state file to manage" is a genuinely real CloudFormation advantage worth naming explicitly, directly connecting to the Terraform state-locking/drift discussion in the Automation series (Part 2): CloudFormation's state is inherently, natively tracked by AWS itself as part of the stack resource — there's no separate state file that can go missing, get corrupted, or need a locking mechanism the way Terraform's state file does, entirely removing one real category of Terraform operational risk already covered generically.

---

SQS — Simple Queue Service#

SQS is AWS's fully managed message queue — directly implementing the queue-based decoupling patterns worth knowing from general messaging/event-driven architecture principles.

Diagram

Copy
aws sqs create-queue --queue-name order-processing

# Send a message
aws sqs send-message \
  --queue-url https://sqs.us-east-1.amazonaws.com/123456789012/order-processing \
  --message-body '{"order_id": "12345"}'

# Consumer polls for messages (typically via long polling,
# reducing empty-response API calls and cost)
aws sqs receive-message \
  --queue-url https://sqs.us-east-1.amazonaws.com/123456789012/order-processing \
  --wait-time-seconds 20 --max-number-of-messages 10

# ONLY after successfully processing, delete the message —
# this is what makes SQS "at-least-once" delivery, not
# "exactly-once" by default
aws sqs delete-message \
  --queue-url https://sqs.us-east-1.amazonaws.com/123456789012/order-processing \
  --receipt-handle AQEBRXy...

Why explicitly deleting the message only AFTER successful processing is the entire mechanism behind SQS's reliability guarantee, worth stating precisely: if a consumer crashes or fails mid-processing WITHOUT deleting the message, it automatically becomes visible again for another consumer to pick up after the visibility timeout expires (next section) — this is "at-least-once" delivery, meaning a message could theoretically be processed more than once, which is exactly why consumer logic should be designed to be idempotent wherever possible.

---

SQS Standard vs FIFO Queues#

Copy
aws sqs create-queue --queue-name order-events.fifo \
  --attributes FifoQueue=true,ContentBasedDeduplication=true

Why choosing FIFO "just to be safe" when ordering genuinely doesn't matter is a real, worth-naming mistake, worth stating explicitly: FIFO's throughput ceiling and added complexity are a real cost paid for a guarantee many workloads don't actually need — Standard queues' effectively unlimited throughput and simpler operational model should be the default, reserving FIFO specifically for the genuinely order-sensitive minority of use cases.

---

SQS Dead-Letter Queues and Visibility Timeout#

Directly extending the Lambda Dead-Letter Queue concept already covered in Part 7 — SQS itself has its own, closely related DLQ mechanism.

Copy
# Configure a redrive policy — after N failed processing
# attempts, the message moves to a DLQ instead of retrying
# forever
aws sqs set-queue-attributes \
  --queue-url https://sqs.us-east-1.amazonaws.com/123456789012/order-processing \
  --attributes '{"RedrivePolicy":"{\"deadLetterTargetArn\":\"arn:aws:sqs:us-east-1:123456789012:order-processing-dlq\",\"maxReceiveCount\":5}"}'

# Visibility timeout — how long a message stays HIDDEN from
# other consumers after being received, giving the current
# consumer time to process it before it's considered failed
aws sqs set-queue-attributes \
  --queue-url https://sqs.us-east-1.amazonaws.com/123456789012/order-processing \
  --attributes VisibilityTimeout=60

Why setting the visibility timeout too SHORT is a genuinely common, real production bug worth naming explicitly: if a consumer legitimately takes 90 seconds to process a message but the visibility timeout is only 60 seconds, the message becomes visible to ANOTHER consumer at the 60-second mark, while the FIRST consumer is still working on it — resulting in the same message being processed twice concurrently, a real, avoidable race condition caused purely by a misconfigured timeout, not application logic.

---

SNS — Simple Notification Service#

SNS is AWS's pub/sub messaging service — one message published to a TOPIC is delivered to EVERY subscriber, directly the concrete AWS implementation of the pub/sub pattern already referenced for CloudWatch alarm notifications in Part 10.

Copy
aws sns create-topic --name order-events
aws sns subscribe --topic-arn arn:aws:sns:us-east-1:123456789012:order-events \
  --protocol sqs --notification-endpoint arn:aws:sqs:us-east-1:123456789012:inventory-queue
aws sns publish --topic-arn arn:aws:sns:us-east-1:123456789012:order-events \
  --message '{"order_id": "12345", "status": "placed"}'

---

SNS Fan-Out Pattern#

Combining SNS and SQS together produces a genuinely powerful, extremely common architectural pattern worth knowing precisely.

Diagram

Why this "fan-out" pattern is worth stating as a genuinely strong default for event-driven microservices, worth explaining precisely: ONE event (an order placed) needs to be reliably delivered to MULTIPLE, entirely independent downstream services — SNS handles the "deliver to everyone subscribed" fan-out, while EACH subscriber gets its OWN SQS queue, meaning one slow or failing consumer (e.g. shipping service having an outage) never blocks or loses messages for the OTHER consumers (inventory, notifications) — each queue buffers and retries completely independently.

---

EventBridge — AWS's Event Bus#

EventBridge is AWS's more modern, more sophisticated event routing service — genuinely a superset of what SNS alone provides, with native, structured event pattern matching and hundreds of built-in AWS service integrations.

Copy
# Create a custom event bus
aws events create-event-bus --name app-events

# A rule matching events by CONTENT PATTERN, not just topic —
# a meaningfully more expressive routing capability than SNS
aws events put-rule \
  --name high-value-orders \
  --event-bus-name app-events \
  --event-pattern '{"source":["app.orders"],"detail-type":["OrderPlaced"],"detail":{"amount":[{"numeric":[">",1000]}]}}'

aws events put-targets \
  --rule high-value-orders --event-bus-name app-events \
  --targets '[{"Id":"1","Arn":"arn:aws:lambda:us-east-1:123456789012:function:fraud-review"}]'

---

EventBridge Rules and Pattern Matching#

Worth stating the key differentiator from SNS precisely: EventBridge rules can match on the actual CONTENT of an event (a specific field's value, a numeric comparison, a prefix match), not just which topic it was published to — the example above routes ONLY orders over $1,000 to a fraud-review Lambda, something SNS's simpler topic-based subscription model cannot express without additional application-level filtering logic.

---

Choosing Between SQS, SNS, and EventBridge#

Diagram

---

Kinesis — Real-Time Streaming Data#

Worth knowing by name for completeness, and precisely how it differs from SQS: Kinesis Data Streams handles high-throughput, ORDERED, REPLAYABLE streams of data — genuinely different from SQS's queue model.

Copy
aws kinesis create-stream --stream-name clickstream --shard-count 4
aws kinesis put-record --stream-name clickstream --partition-key user-123 --data '{"event":"page_view"}'

Why "replayable" is the key distinguishing property from SQS, worth stating precisely: an SQS message is DELETED once successfully processed and gone forever — a Kinesis stream RETAINS data for a configurable window (up to 365 days), letting MULTIPLE independent consumers read the SAME data at their OWN pace, and even letting a NEW consumer added later re-process HISTORICAL data from earlier in the stream — a genuinely different use case (real-time analytics, clickstream processing, log aggregation) than SQS's "process once and discard" queue model.

---

A Full Worked Example: A Complete AWS-Native Deployment Pipeline#

Bringing this entire part together into one concrete, complete CI/CD and event-driven architecture.

Diagram

Why this design is worth narrating end to end as a coherent interview answer, directly synthesizing this entire series: infrastructure (Parts 3-8) is provisioned via CloudFormation/CDK; deployment (this part) uses CodePipeline/CodeBuild/CodeDeploy implementing the blue-green strategy from the Automation series; security (Part 9) governs every IAM role involved; observability (Part 10) monitors the whole pipeline and the running application; and EventBridge/SQS decouple the application's own downstream event processing — every AWS service covered across this entire series has a specific, named role in this one coherent picture.

---

Pipeline Security: IAM Roles and Cross-Account Deployment#

Directly extending the IAM discussion from Part 2 — worth a specific, pipeline-focused treatment, since a CI/CD pipeline is genuinely one of the highest-privilege, highest-value targets in an organization's entire AWS footprint (already flagged generically in the DevSecOps series' supply-chain security discussion).

Diagram

Copy
# The pipeline's OWN role only has permission to ASSUME
# specific, narrowly-scoped deployment roles in target
# accounts — it never holds broad permissions itself
aws iam create-role --role-name PipelineExecutionRole \
  --assume-role-policy-document file://pipeline-trust-policy.json

aws iam put-role-policy --role-name PipelineExecutionRole \
  --policy-name AssumeDeployRoles --policy-document '{
    "Version": "2012-10-17",
    "Statement": [{"Effect": "Allow", "Action": "sts:AssumeRole", "Resource": ["arn:aws:iam::STAGING_ACCOUNT:role/DeployRole", "arn:aws:iam::PROD_ACCOUNT:role/DeployRole"]}]
  }'

Why running the pipeline itself in a SEPARATE, dedicated "tooling" account (rather than directly in the production account) matters, worth stating explicitly, directly reusing the multi-account landing zone pattern from Part 1: a compromised pipeline (via a malicious dependency, a leaked GitHub token, or a supply-chain attack — all already covered generically in the DevSecOps series) has no standing access to production at all — it can only reach production by explicitly assuming a specific, narrowly-scoped, Permission-Boundary-limited role, exactly the same cross-account access pattern already covered in depth in Part 2, now applied specifically to protect the deployment pipeline itself as a high-value target.

---

CI/CD and Messaging Best Practices — The Consolidated Checklist#

• Run CI/CD pipelines from a dedicated tooling account, assuming narrowly-scoped, Permission-Boundary-limited roles into target accounts — never granting the pipeline standing production access directly.
• Always review a CloudFormation Change Set (or cdk diff) before executing a production infrastructure change.
• Use StackSets for baseline configuration that every account should automatically receive — governance by construction, not a manual onboarding checklist.
• Default to SQS Standard queues; reserve FIFO for genuinely order-sensitive workloads.
• Set SQS visibility timeout comfortably longer than worst-case processing time, not just the average.
• Configure a Dead-Letter Queue on every production SQS queue handling business-critical messages.
• Use EventBridge over SNS when routing decisions depend on event CONTENT, not just which topic something was published to.
• Choose Kinesis over SQS specifically when multiple independent consumers need to replay the same data — they solve genuinely different problems.

---

Part 11 CLI Cheat Sheet#

---

Common Mistakes#

---

Worked Practice Problems#

Problem 1: A team's SQS consumer occasionally processes the same order twice, causing duplicate charges. Investigation shows the consumer takes an average of 45 seconds to process a message, but the queue's visibility timeout is set to 30 seconds. What's causing the duplicate processing, and what's the fix?

Answer: Because the visibility timeout (30 seconds) is shorter than the actual processing time (45 seconds average, meaning some messages take even longer), a message becomes visible again to OTHER consumers at the 30-second mark while the original consumer is still actively processing it — a second consumer then picks up and begins processing the same message concurrently, resulting in duplicate processing and the observed duplicate charges. The fix is increasing the visibility timeout to comfortably exceed the worst-case processing time (not just the average), giving the original consumer enough time to finish and delete the message before it could ever become visible to another consumer.

Problem 2: An e-commerce platform needs an order-placed event to trigger three completely independent downstream actions: updating inventory, notifying the shipping service, and sending a customer confirmation email. The team wants a slow or failing shipping service to never block or delay inventory updates or customer notifications. What messaging architecture achieves this, and why?

Answer: An SNS topic fanning out to three separate SQS queues, one per downstream consumer (inventory, shipping, notifications) — the classic SNS fan-out pattern. Publishing the order-placed event once to the SNS topic reliably delivers it to all three subscribed queues; because each consumer has its OWN independent queue, a slow or failing shipping service only affects messages sitting in the SHIPPING queue — it has no effect whatsoever on the inventory or notification queues, which continue processing normally and independently, exactly satisfying the requirement that one consumer's problems never block the others.

Problem 3: A team wants to route "high-value order" events (orders over $5,000) to a special fraud-review process, while routing all other orders through the normal processing path — using a single event published by the order service. A team member suggests using SNS with two separate topics the order service would need to decide between at publish time. A colleague suggests EventBridge instead. Which approach is better, and why?

Answer: EventBridge is the better fit. SNS's topic-based model would require the order service itself to contain the "is this high-value" business logic at publish time, deciding which of two topics to publish to — coupling publishing logic with downstream routing decisions that arguably don't belong in the order service at all. EventBridge instead lets the order service publish ONE event type regardless of order value, with the content-based routing rule (matching on the order amount field) living entirely in EventBridge's own rule configuration — decoupling the publishing service from the routing logic entirely, and making it easy to add or change routing rules later without ever touching the order service's own code.

Problem 4: A security review flags that the organization's CodePipeline, running in the same AWS account as production workloads, has an execution role with broad, standing IAM permissions across the entire production environment — the pipeline can directly modify nearly any production resource at any time, not just during an actual deployment. What's the underlying architectural risk, and what's the recommended restructuring?

Answer: The underlying risk is that the pipeline itself becomes one of the highest-value targets in the entire AWS footprint — a compromise of the pipeline (via a malicious dependency, a leaked source-control token, or any other supply-chain attack vector already covered generically in the DevSecOps series) would grant an attacker the SAME broad, standing production access the pipeline itself holds, with no additional barrier to cross. The recommended restructuring is running the pipeline from a separate, dedicated tooling account (per Part 1's multi-account landing zone pattern), whose own execution role holds no direct production permissions at all — instead, it's only permitted to ASSUME specific, narrowly-scoped, Permission-Boundary-limited deployment roles in the target accounts, and only for the specific actions a deployment genuinely requires, meaning a pipeline compromise no longer grants an attacker standing, broad production access by default.

Problem 5: A team building an analytics feature needs to process the same stream of user clickstream events through three different, independently-evolving analytics jobs — one computing real-time dashboards, one for a daily batch aggregation job, and one for a machine learning feature pipeline that was added six months after the other two were already in production and needs to reprocess several weeks of historical clickstream data it never saw the first time. Would SQS or Kinesis be the better fit here, and why?

Answer: Kinesis Data Streams is the better fit, specifically because of the ML pipeline's requirement to reprocess HISTORICAL data it never originally consumed — this is exactly the "replayable" property that distinguishes Kinesis from SQS. An SQS queue permanently deletes a message once any consumer successfully processes it, meaning the two already-running consumers (dashboards, batch aggregation) would have already consumed and removed the historical data long before the ML pipeline was ever added, leaving nothing for it to replay. Kinesis instead retains data for a configurable retention window (up to 365 days) and allows multiple independent consumers to read the same stream at their own pace — including a consumer added much later, which can still read historical data from earlier in the stream's retention window, exactly satisfying the ML pipeline's need to reprocess weeks of clickstream data it wasn't originally built to consume.

---

Summary and What's Next#

• AWS-native CI/CD (CodeBuild, CodeDeploy, CodePipeline) directly implements the CI/CD fundamentals and deployment strategies already covered generically in the Automation series, with tight, native IAM integration as its main differentiator from third-party tools.
• CloudFormation Change Sets are the direct equivalent of terraform plan — never apply a production infrastructure change without reviewing one first.
• StackSets apply the same governance-by-construction philosophy already seen for Firewall Manager and Tag Policies, automatically deploying baseline infrastructure to every account in an OU.
• The CDK compiles real programming-language code down to CloudFormation, trading YAML's simplicity for genuine loops, conditionals, and testability.
• SQS (queue, at-least-once, optional FIFO ordering) and SNS (pub/sub fan-out) solve different problems; combining them (SNS→SQS fan-out) is a genuinely strong default pattern for decoupled microservices.
• EventBridge adds content-based routing SNS structurally cannot express, at the cost of additional configuration complexity.
• Kinesis provides replayable, multi-consumer streaming — a fundamentally different guarantee than SQS's delete-on-success queue model.

Continue to Part 12 (12-multi-region-dr-migration-and-cheatsheet.md) — the final part of this series, covering multi-region architecture, disaster recovery, migration strategies, and a full cross-service cheat sheet tying the entire series together.