Part 13 of 1937 min read · 4 diagramsAI-assisted

API Gateway, Cognito & Event Integration

Assumes you're comfortable with IAM roles and policies (Part 2), Lambda (Part 7), and SQS/SNS/EventBridge at the level Part 11 introduced them — this part goes much deeper on the pieces that sit in front of an application: who's allowed to call it, and how its internal services talk to each other.

Table of Contents#

  1. Why This Part Exists
  2. Two Different Problems Wearing Similar Names
  3. API Gateway — Three Products, One Name
  4. REST APIs — The Full-Featured Option
  5. HTTP APIs — Lightweight and Cheap
  6. REST vs HTTP APIs — Head to Head
  7. WebSocket APIs — Persistent, Two-Way Connections
  8. Integration Types — What's Actually Behind the Route
  9. Mapping Templates and Payload Transformation
  10. Request Validation and Models (REST APIs)
  11. Observability for APIs: Logs, Metrics, and Tracing
  12. Authorizers — IAM, Lambda, Cognito, and JWT
  13. Throttling, Usage Plans, and API Keys
  14. Caching at the API Gateway Layer
  15. Private APIs and VPC Links
  16. Endpoint Types: Edge-Optimized, Regional, and Private
  17. CORS — A Persistent Source of Confusion
  18. Canary Releases and Stage Variables
  19. Cognito — Identity for Customer-Facing Applications
  20. Cognito User Pools — Core Concepts
  21. User Pool Authentication Flows
  22. Hosted UI and Federation
  23. Identity Pools — From Token to AWS Credentials
  24. User Pools and Identity Pools, Wired Together
  25. Securing an API Gateway Route With Cognito, Worked
  26. Cognito Lambda Triggers — Customizing the Auth Lifecycle
  27. Advanced Security: Adaptive Authentication and Compromised Credentials
  28. Cognito vs IAM vs IAM Identity Center — Choosing the Right Identity System
  29. AppSync — GraphQL APIs on AWS
  30. AppSync Resolvers — Unit and Pipeline
  31. AppSync Real-Time Subscriptions
  32. AppSync Caching and Data Source Depth
  33. REST/HTTP APIs vs GraphQL — Choosing
  34. Step Functions, Revisited in Depth
  35. Standard vs Express Workflows
  36. Service Integrations and the Callback Pattern
  37. Error Handling: Retry and Catch
  38. The Map State and Distributed Map — Large-Scale Parallelism
  39. EventBridge, Revisited in Depth
  40. EventBridge Schema Registry
  41. EventBridge Pipes — Point-to-Point Integration
  42. Cross-Account and Cross-Region Event Buses
  43. Choosing Among SQS, SNS, EventBridge, Step Functions, and AppSync
  44. Amazon MQ — When Protocol Compatibility Matters
  45. Idempotency for Public APIs
  46. A Full Worked Example: A Public Order-Status API
  47. Security Checklist for Public APIs
  48. API and Event Integration Best Practices — The Consolidated Checklist
  49. Part 13 CLI Cheat Sheet
  50. Common Mistakes and Interview Traps
  51. Worked Practice Problems
  52. Summary and What's Next

Why This Part Exists#

Part 7 built Lambda functions and containers. Part 11 introduced SQS, SNS, and EventBridge as the plumbing that lets those services talk to each other asynchronously. What's been missing is the layer that sits in front of all of it — the thing an external client, a mobile app, or a partner integration actually calls, and the thing that decides whether the caller is who they claim to be. That's this part: API Gateway as the front door, Cognito as the identity system behind that door, AppSync as an alternative front door for GraphQL, and a much deeper pass on the event-driven orchestration tools Part 11 only introduced at a survey level.

This maps directly to the "API-first" and "identity as a first-class architectural concern" ideas that show up constantly in real platform work: an air-traffic control tower doesn't just route planes, it verifies every aircraft's clearance before it's allowed anywhere near the runway. API Gateway is that tower for a system's edge; Cognito is the clearance registry it checks against.

Two Different Problems Wearing Similar Names#

Before touching any service, separate two questions that get conflated constantly, including on both the DVA-C02 and SAP-C02 exams:

  • Authentication — "who is this caller?" Cognito user pools, IAM users/roles, federated identity providers all answer this.
  • Authorization — "what is this caller allowed to do?" IAM policies (Part 2), Cognito identity-pool role mappings, and API Gateway resource policies all answer this — separately, after authentication has already established an identity.

Note

Every identity system in this part produces the same end product for AWS API calls: a JSON Web Token (JWT) or a set of temporary IAM credentials. The system generating that token differs (Cognito, a corporate IdP via IAM Identity Center from Part 1, a Lambda authorizer's own logic), but what API Gateway and downstream AWS services actually check is always one of those two things.

API Gateway — Three Products, One Name#

"API Gateway" isn't one product with optional features — it's three genuinely different products sharing a console and a name: REST APIs, HTTP APIs, and WebSocket APIs. Picking the wrong one for a new project is a common, expensive-to-unwind mistake, because they don't share configuration and migrating between them means rebuilding routes and integrations from scratch.

Diagram

The original API Gateway product (sometimes called "REST APIs" or, in older docs, just "API Gateway"). Full feature set: request/response validation against a JSON Schema model, per-stage caching, API keys and usage plans, resource policies (including making an API private to a VPC), WAF integration, and mapping templates for arbitrary request/response transformation. This depth is also why it's the most expensive and most complex of the three, and why AWS's own current guidance is "use it only when you specifically need one of these features."

HTTP APIs — Lightweight and Cheap#

A newer, deliberately minimal product built for the common serverless case: a client calling a Lambda function or an HTTP backend, with JWT-based auth and CORS handling built in natively (no custom authorizer needed for the common case), and native integration with IAM Identity Center / OIDC / OAuth 2.0 providers. It does not support request validation, per-stage response caching, or API keys/usage plans as of this writing — if a project turns out to need those later, that's a genuine migration, not a config flag.

Tip

Default to HTTP APIs for anything new unless a specific REST-API-only feature is already a known requirement. Confirmed current pricing gap: HTTP APIs run roughly 70% cheaper per million requests than REST APIs for equivalent traffic, and the DVA-C02 exam explicitly expects "cheapest option that meets requirements" reasoning — reaching for REST APIs by habit when nothing in the requirements needs its extra feature set is a real, gradeable mistake on that exam, not just a cost nitpick in production.

REST vs HTTP APIs — Head to Head#

CapabilityREST APIHTTP API
Lambda/HTTP proxy integrationYesYes
Native JWT authorizerNo (needs a Lambda authorizer)Yes, built in
Request/response validationYesNo
Per-stage response cachingYesNo
API keys and usage plansYesNo
Private API (VPC-only)YesYes
WAF integrationYesNo
Mapping templates (VTL)YesLimited (parameter mapping only)
Relative cost per million requestsBaseline~70% cheaper

WebSocket APIs — Persistent, Two-Way Connections#

Unlike REST/HTTP APIs, which are strictly request-response, a WebSocket API holds a persistent connection open, so either side can push a message at any time without the client polling. Routes are keyed by message content rather than by URL path: $connect fires once when a client opens the connection, $disconnect fires when it closes, $default catches anything not matched by a custom route, and any number of custom routes match on a field inside the incoming message body. Connection state (who's connected, what data to associate with a connection ID) has to be stored somewhere durable — DynamoDB is the standard choice, since Lambda itself is stateless between invocations.

Diagram

A live chat feature, a real-time dashboard, or a stock-ticker-style price feed are the canonical fits — anywhere the server needs to initiate the message, not just respond to one.

Integration Types — What's Actually Behind the Route#

Every route in API Gateway maps to an integration type, and picking the right one avoids writing glue code that AWS Gateway would otherwise do for free:

Integration typeWhat it does
Lambda proxyPasses the entire request through to Lambda as an event object; Lambda's response shape (statusCode, headers, body) is passed back verbatim. The default choice for serverless backends.
Lambda custom (non-proxy)API Gateway transforms the request/response via mapping templates before/after Lambda runs. More setup, more control — mostly a REST-API-only pattern now.
HTTP proxyForwards the request essentially unchanged to an existing HTTP backend (an ALB-fronted service, a third-party API).
AWS service integrationCalls another AWS service's API directly (e.g., putting a message straight onto SQS, or invoking Step Functions) with zero compute in between.
MockReturns a canned response with no backend call at all — useful for CORS preflight OPTIONS responses or building out a contract before the backend exists.

Mapping Templates and Payload Transformation#

REST APIs (and, to a lesser degree, HTTP APIs) can rewrite a request or response body in flight using Velocity Template Language (VTL) mapping templates. The classic use case: exposing a clean, versioned public API contract while the actual backend (say, a legacy SOAP service, or a differently-shaped internal schema) doesn't match that contract at all. This is powerful and also genuinely painful to debug — VTL errors show up as unhelpful Execution failed due to configuration error messages with limited context, which is precisely why most new serverless-first designs prefer Lambda proxy integration and do the transformation in ordinary application code instead, reserving VTL for the cases (AWS service integration with no Lambda in the loop at all) where there's no other option.

Request Validation and Models (REST APIs)#

REST APIs can reject a malformed request before it ever reaches the backend, by validating the request body against a JSON Schema model, and/or validating required query-string parameters and headers are present — configured declaratively on the method, no application code involved. This moves a whole class of defensive "is this field present and the right type" checks out of every Lambda function and into the API layer itself, where a bad request fails fast with a 400 and never consumes any compute at all. HTTP APIs, true to their minimal design, don't support this — a Lambda-side validation library (or a shared validation layer, Part 7) is the equivalent there.

Observability for APIs: Logs, Metrics, and Tracing#

API Gateway integrates directly with the observability stack from Part 10 rather than inventing its own: execution logs (detailed per-request debug logging, useful during development, expensive to leave on in production at high volume), access logs (a configurable-format log line per request — method, path, status, latency — the one worth always leaving on), CloudWatch metrics (Count, 4XXError, 5XXError, Latency, IntegrationLatency — the last one specifically isolating backend latency from API Gateway's own overhead), and native X-Ray tracing support that stitches an API Gateway request into the same trace as the Lambda/backend work it triggers, so a single trace shows the full request lifecycle end to end rather than two disconnected fragments.

Authorizers — IAM, Lambda, Cognito, and JWT#

Authorizer typeHow it verifies the callerFits
IAM authorizationCaller signs the request with SigV4 using real IAM credentials; API Gateway checks the resulting identity against an IAM policy.Service-to-service calls already inside AWS, internal tooling.
Lambda authorizerA Lambda function receives the request (or just its token) and returns an IAM policy document deciding allow/deny, optionally with a caching TTL.Custom auth logic (a legacy session token, an API key stored outside Cognito, header-based tenant routing).
Cognito user pool authorizerAPI Gateway validates a Cognito-issued JWT directly, no Lambda in the loop.REST APIs backed by Cognito-authenticated users.
JWT authorizer (HTTP APIs)Native JWT validation against any OIDC-compliant issuer (Cognito, or a third-party IdP), configured declaratively.HTTP APIs — the built-in replacement for a Lambda authorizer's most common use case.

Important

A Lambda authorizer's response is cached by default (5 minutes unless configured otherwise), keyed on the caller-supplied identity source (usually the Authorization header value). Revoking a compromised token doesn't take effect until that cache entry expires unless the cache is deliberately invalidated — a real operational gap worth knowing about before an incident, not during one.

Throttling, Usage Plans, and API Keys#

API Gateway enforces two levels of throttling: an account-level steady-state/burst limit shared across every API in the account/region, and a per-method or per-stage limit set explicitly. REST APIs add usage plans: a named bundle of throttle and quota limits (e.g., "1000 requests/day, 10 requests/second burst") associated with one or more API keys, letting an operator hand different partners different tiers of access to the same API without deploying separate infrastructure. An API key by itself is not an authentication mechanism — it identifies which usage plan applies, nothing more; a key alone grants no authorization, which is a genuinely common point of confusion worth stating explicitly.

Caching at the API Gateway Layer#

REST APIs support an optional, per-stage response cache (sized from 0.5 GB to 237 GB), keyed by request parameters and TTL-controlled per method. This trades a small added cost for the cache itself against a real reduction in backend load and latency for read-heavy, slowly-changing responses (a product catalog listing, a configuration payload) — but it's a full response cache at the edge closest to the client, a different layer entirely from CloudFront's caching (Part 8) or DAX in front of DynamoDB (Part 6). All three can legitimately be layered on the same request path, each solving a different part of the latency budget.

A REST (or HTTP) API can be made private — reachable only from inside a VPC via an interface VPC endpoint (Part 4), never from the public internet — the standard shape for an internal platform API that should never be internet-facing at all. Going the other direction, a VPC Link lets a public-facing API Gateway route traffic into a private VPC resource (an internal ALB, an NLB fronting an ECS service) without that backend ever needing a public IP or an internet gateway route — the API Gateway layer is the only piece of the path exposed to the internet.

Endpoint Types: Edge-Optimized, Regional, and Private#

A REST API chooses one of three endpoint types, and the choice is about where traffic enters AWS's network, not about functionality: edge-optimized routes client requests through the nearest CloudFront edge location first (Part 8), reducing latency for geographically distributed clients at the cost of an extra network hop for a client that's already close to the API's home region; regional serves traffic directly from the API's own region, the better default when clients are concentrated near that region already, or when the API sits behind a separately managed CloudFront distribution (giving full control over caching/WAF/custom domains instead of the semi-managed edge-optimized setup); private, as covered above, is reachable only from inside a VPC. For a genuinely global user base needing true multi-region failover (Part 12's patterns), the real answer is usually regional APIs deployed in multiple regions, fronted by Route 53 latency-based or failover routing — edge-optimized alone doesn't provide multi-region backend failover, only faster network entry to a single backend region.

CORS — A Persistent Source of Confusion#

Cross-Origin Resource Sharing trips up more API Gateway builds than almost anything else, because the error surfaces in the browser console with no useful detail (No 'Access-Control-Allow-Origin' header is present) that points nowhere near the actual misconfiguration. What's actually required, for both HTTP and REST APIs, is two separate things working together: the browser's preflight OPTIONS request must get a response listing the allowed origin/methods/headers, and the actual GET/POST/etc. response itself must also carry an Access-Control-Allow-Origin header — configuring only one of the two is the single most common cause of "it works in Postman but not the browser," since Postman never sends a preflight request at all.

HTTP APIs support CORS as a native, declarative configuration block (allowed origins, methods, headers, credentials, max-age) with no extra integration required. REST APIs need either a manually configured OPTIONS Mock integration returning the right headers, or — for a Lambda proxy integration — the Lambda function itself must include the CORS headers on every response it returns, preflight and real alike, since a proxy integration passes the response through exactly as the function shaped it.

Warning

Setting Access-Control-Allow-Origin: * alongside Access-Control-Allow-Credentials: true is invalid per the CORS spec and will be silently rejected by browsers — a wildcard origin cannot be combined with credentialed requests (cookies, Authorization headers carrying session state). An API that needs both wildcard-style flexibility and credentialed calls has to enumerate allowed origins explicitly instead.

Canary Releases and Stage Variables#

A REST API stage supports a canary release: a configurable percentage of traffic routed to a newly deployed version while the rest continues hitting the previous one, with independent CloudWatch metrics per canary vs. baseline traffic — the same progressive-delivery idea from Part 7's ECS/Lambda deployment strategies, applied at the API Gateway layer itself rather than at the compute layer behind it. Stage variables (REST APIs) are named key-value pairs scoped to a single stage, commonly used to point the same API definition at different backend Lambda aliases or HTTP endpoints per environment (dev, staging, prod) without maintaining separate API definitions — the API Gateway equivalent of an environment variable, resolved at request time via ${stageVariables.lambdaAlias} inside an integration's configuration.

Cognito — Identity for Customer-Facing Applications#

Part 2 covered IAM in full depth — but IAM identities (users, roles) are fundamentally for AWS principals: engineers, CI/CD pipelines, EC2 instances. They were never meant to model millions of end-users signing up for a consumer app. Amazon Cognito is the purpose-built answer to that different problem: a managed identity system for the humans (or devices) using an application, entirely separate from the IAM identities of the people operating the AWS account itself.

Cognito User Pools — Core Concepts#

A user pool is a managed user directory: sign-up, sign-in, password policies, MFA, account recovery, and custom user attributes, all without running a database of password hashes yourself. Successful authentication against a user pool returns three JWTs: an ID token (claims about who the user is — name, email, custom attributes), an access token (scoped for calling your own backend APIs), and a refresh token (used to get new ID/access tokens without forcing the user to log in again). Each token has its own separate, configurable expiry — access and ID tokens are short-lived by design (default one hour), refresh tokens live far longer (default 30 days), a deliberate tradeoff between session convenience and the blast radius of a leaked token.

User Pool Authentication Flows#

FlowHow it worksFits
USER_SRP_AUTHSecure Remote Password protocol — the password itself never crosses the network, even encrypted.Default for first-party web/mobile clients — the recommended flow.
USER_PASSWORD_AUTHPlain username/password sent directly (over TLS).Server-side / trusted-backend flows where SRP's extra round trips aren't practical.
Custom auth flowA chain of Lambda triggers (DefineAuthChallenge, CreateAuthChallenge, VerifyAuthChallengeResponse) implementing arbitrary logic.Passwordless flows (magic link, OTP via SMS/email), CAPTCHA-gated login, adaptive/step-up auth.
Refresh token flowExchange a valid refresh token for new ID/access tokens.Keeping a session alive without re-prompting for credentials.

Hosted UI and Federation#

A user pool can host a fully-managed sign-up/sign-in web page (the Hosted UI) so an application never has to build its own login form or handle raw credentials at all — a real reduction in the app's PCI/PII handling surface. The same user pool can federate with external identity providers: social logins (Google, Facebook, Apple, Amazon), or enterprise SAML/OIDC IdPs — letting a B2B SaaS product accept "sign in with your company's Okta/Azure AD" without building that integration from scratch. Federated and native (username/password) users land in the same user pool and produce the same token shape downstream, so the rest of the application never needs to know which path a given user came in through.

Identity Pools — From Token to AWS Credentials#

A user pool proves who someone is. An identity pool (a genuinely different, commonly confused resource) is what turns that proof into temporary AWS credentials — a short-lived access key/secret key/session token pair scoped by an IAM role, obtained via STS, exactly like the role-assumption mechanics covered in Part 2. This is what lets a mobile app upload a file straight to a specific S3 prefix, or read straight from a DynamoDB table, without a backend server in the middle brokering that access at all. Identity pools also support unauthenticated (guest) identities — a distinct, separately-scoped IAM role for users who haven't signed in yet, useful for letting a mobile app do something minimal (log analytics events, browse public content) before requiring a login.

User Pools and Identity Pools, Wired Together#

Diagram

The identity pool's IAM role can use policy variables (Part 2's ABAC pattern) keyed on cognito-identity.amazonaws.com:sub — the user's unique Cognito identity ID — to scope S3/DynamoDB access to exactly that user's own data with one shared role definition, rather than provisioning a role per user.

Securing an API Gateway Route With Cognito, Worked#

  1. Create a user pool; enable the Hosted UI or a native client integration.
  2. Attach a Cognito user pool authorizer (REST API) or a JWT authorizer pointed at the pool's issuer URL (HTTP API) to the route.
  3. The client signs in against the user pool, gets an ID/access token.
  4. The client calls the API with Authorization: Bearer <token>.
  5. API Gateway validates the token's signature and expiry against the user pool's public keys — before the request ever reaches Lambda; an invalid or expired token never executes any application code.
  6. The Lambda function receives the decoded claims (user ID, custom attributes, scopes) in the event's requestContext.authorizer — enough to make per-user authorization decisions without a second call back to Cognito.

Cognito Lambda Triggers — Customizing the Auth Lifecycle#

A user pool can invoke a Lambda function at over a dozen points in the authentication lifecycle, letting an application inject custom logic without forking Cognito's own behavior:

TriggerFires whenCommon use
PreSignUpBefore a new account is createdAuto-confirm trusted domains, block disposable-email signups, validate an invite code
PostConfirmationRight after a user confirms their accountCreate a matching row in an application database, send a welcome event onto EventBridge
PreTokenGenerationBefore ID/access tokens are issued (including on refresh)Inject custom claims (a tenant ID, a subscription tier) into the token so downstream services never need a second lookup
PreAuthenticationBefore a sign-in attempt is validatedCustom risk checks, blocking known-bad IPs
CustomMessageBefore a verification/invite email or SMS is sentFully custom email templates and branding
DefineAuthChallenge / CreateAuthChallenge / VerifyAuthChallengeResponseDuring a custom auth flowPasswordless (OTP, magic link) login flows

PreTokenGeneration is worth calling out specifically: it's what makes Cognito viable for genuinely multi-tenant SaaS applications, since a tenant ID baked into the access token means every downstream Lambda/API can authorize purely from the token's claims, with zero additional database round trip per request.

Advanced Security: Adaptive Authentication and Compromised Credentials#

User pools with the Plus feature tier enabled add adaptive authentication: each sign-in attempt is scored for risk (unfamiliar device, impossible-travel location, known bad IP reputation) and can trigger step-up MFA or an outright block on high-risk attempts, without the application writing any custom risk logic itself. A separate, always-available check, compromised credentials detection, blocks sign-in or sign-up attempts using a username/password pair AWS has identified in a known public credential-leak dataset — the managed equivalent of the "have I been pwned" pattern, wired directly into the sign-in flow rather than bolted on separately.

Cognito vs IAM vs IAM Identity Center — Choosing the Right Identity System#

SystemIdentities it managesTypical caller
IAM users/roles (Part 2)AWS-account-scoped principalsCI/CD pipelines, service-to-service, break-glass access
IAM Identity Center (Part 1)Workforce (employees) across a whole OrganizationEngineers logging into the AWS Console/CLI across many accounts
CognitoEnd-users of an application your company builtCustomers of a mobile app, users of a SaaS product's web UI

A genuinely common architectural mistake: provisioning individual IAM users for an application's customers. It doesn't scale (IAM has hard account-level identity limits nowhere near consumer-app volume), it conflates two categorically different trust boundaries, and it's precisely the anti-pattern Cognito exists to replace.

AppSync — GraphQL APIs on AWS#

AppSync is API Gateway's GraphQL-shaped sibling: a managed GraphQL API that can resolve fields from DynamoDB, Lambda, RDS (via the RDS Data API), OpenSearch, or any HTTP endpoint — often several of them in a single client request. Where a REST/HTTP API typically means "one round trip per resource," a GraphQL client can request exactly the fields it needs, from however many underlying data sources those fields actually live in, in one round trip — a real advantage for mobile clients on constrained networks pulling data assembled from multiple services.

AppSync Resolvers — Unit and Pipeline#

A resolver is the piece of configuration that maps one GraphQL field to a data source and a request/response transformation, written in APPSYNC_JS (JavaScript) or the older VTL. A unit resolver talks to exactly one data source. A pipeline resolver chains several AppSync functions in sequence — validate input, call one data source, feed its result into a second data source — all within a single GraphQL field resolution, without needing a Step Functions state machine or an orchestrating Lambda just to sequence two internal calls.

AppSync Real-Time Subscriptions#

Beyond queries (read) and mutations (write), GraphQL defines subscriptions — a client subscribes to a field, and AppSync pushes an update over a managed WebSocket connection whenever a matching mutation happens, with no connection-management code required on either side (unlike a hand-rolled WebSocket API, where $connect/$disconnect bookkeeping is the caller's responsibility). This is the same "push, not poll" capability a raw WebSocket API provides, but scoped to GraphQL's type system and wired in automatically.

AppSync Caching and Data Source Depth#

AppSync supports its own server-side response cache, either full-query or per-resolver, backed transparently by a managed cluster the operator sizes but never patches — conceptually the same tradeoff as API Gateway's stage cache (Part 13's earlier section), applied to GraphQL field resolution instead of whole HTTP responses. Beyond DynamoDB and Lambda, AppSync data sources also include OpenSearch (full-text search fields inside a GraphQL schema), the RDS Data API (querying Aurora Serverless without managing a persistent connection pool from inside a resolver), and HTTP data sources for wrapping an existing REST API behind a GraphQL facade — a common modernization step when a GraphQL layer needs to sit in front of services that were never rebuilt as GraphQL-native.

REST/HTTP APIs vs GraphQL — Choosing#

ConsiderationREST/HTTP API (API Gateway)GraphQL (AppSync)
Client fetches exactly the fields it needsNo — fixed response shape per endpointYes — client specifies the shape
Aggregating multiple backend sources in one callRequires a BFF/orchestrating LambdaNative, via pipeline resolvers
CachingMature (per-stage cache, CloudFront)Possible but less mature tooling
Learning curve / ecosystem maturityLower — plain HTTP semanticsHigher — GraphQL schema design discipline
Real-time pushWebSocket API (separate product)Built in via subscriptions

Neither replaces the other outright — a public partner-facing API with a stable, documented contract still usually favors REST/HTTP; a mobile app assembling a dashboard from five different data types in one screen is the case GraphQL was built for.

Step Functions, Revisited in Depth#

Part 7 introduced Step Functions as "orchestrating multiple Lambdas." The full picture: Step Functions is a managed state machine, defined in Amazon States Language (ASL), that can orchestrate not just Lambda but any of over 220 AWS service API actions directly — starting a Glue job (Part 18), running an ECS task, publishing to SNS — with retry, error handling, and parallel/choice branching declared in the state machine definition itself instead of hand-written in application code.

Diagram

Standard vs Express Workflows#

StandardExpress
Max durationUp to 1 yearUp to 5 minutes
Execution semanticsExactly-onceAt-least-once (async) or at-most-once (sync)
Execution historyFull, visual, retained 90 days, freeCloudWatch Logs only, at extra cost
Pricing modelPer state transitionPer invocation + duration (GB-seconds)
FitsOrder fulfillment, approval workflows, long ETL jobs — anything needing a durable audit trailHigh-volume event processing, streaming transformation, IoT ingestion

The choice is rarely close once the durations and volumes are known: a workflow that must survive for days and be individually auditable belongs on Standard; a workflow firing thousands of times a second for sub-second tasks belongs on Express, where Standard's per-state-transition pricing would be far more expensive.

Service Integrations and the Callback Pattern#

Most AWS service integrations in Step Functions support one of three request-response patterns: Request Response (fire the API call, immediately continue), Run a Job (.sync) (wait for a service's own asynchronous job to finish — an ECS task, a Glue job, a Batch job — polling internally so the workflow doesn't need custom polling logic), and Wait for Callback (.waitForTaskToken) (pause the state machine entirely until an external system calls back SendTaskSuccess/SendTaskFailure with a token, the standard pattern for a human-approval step or a long-running third-party process with no native Step Functions integration).

Error Handling: Retry and Catch#

Every state can declare a Retry block (which errors to retry, backoff interval, max attempts, backoff rate) and a Catch block (which errors route to which fallback state) directly in the ASL definition — the same resilience patterns from Part 8/Part 4 of the broader course (circuit breakers, exponential backoff), expressed declaratively instead of hand-coded in every Lambda function that might fail.

The Map State and Distributed Map — Large-Scale Parallelism#

Step Functions' Map state iterates a state machine's sub-workflow over every item in an input array, running iterations concurrently up to a configurable concurrency limit — the declarative equivalent of a for loop with a worker pool, without writing the pool management. The standard Map state holds its entire input array in the state's own execution history, which caps it at roughly 40 items of meaningfully sized payload before hitting Step Functions' 256 KB state-size limit. Distributed Map, a newer mode built for genuinely large-scale fan-out, instead reads its input directly from an S3 object (a manifest file, or every object under a prefix) and can launch up to 10,000 parallel child workflow executions, each running independently with its own execution history — the pattern for processing every file dropped into an S3 bucket, or reprocessing millions of existing database rows, without hitting the standard Map state's size ceiling at all.

EventBridge, Revisited in Depth#

Part 11 covered EventBridge's core loop: producers publish to a bus, rules match events by pattern, and matched events fan out to up to five targets each. Two pieces worth going deeper on for real production use and for SAP-C02-level architecture questions: schema management, and the newer point-to-point Pipes product.

EventBridge Schema Registry#

The schema registry stores the shape of events flowing through a bus — either discovered automatically from real traffic, or defined manually as OpenAPI 3 or JSON Schema — and can generate strongly-typed code bindings for consuming applications in several languages. This solves a real, easy-to-underestimate problem in event-driven architectures: without a shared schema contract, a producer team can silently change an event's shape and break every consumer that assumed the old shape, with no compile-time warning anywhere. The registry itself is free; it's a governance tool, not a runtime dependency.

EventBridge Pipes — Point-to-Point Integration#

Pipes are a different shape than rules entirely: one source, one target, no bus and no many-to-many routing in between, but with built-in filtering, enrichment (calling a Lambda/Step Functions/API Destination to transform the payload before delivery), and support for source types rules don't have direct access to — an SQS queue, a Kinesis/DynamoDB stream, or an MSK topic (Part 18) feeding directly into a target with no polling Lambda required to bridge them. Where a rule answers "route this event to whichever of several interested targets," a pipe answers "reliably move and transform everything from this one specific source into this one specific target" — a narrower but often simpler and cheaper tool for that narrower job.

Cross-Account and Cross-Region Event Buses#

A single organization's event-driven architecture rarely lives in one account. EventBridge supports cross-account event delivery: a rule in the producing account's bus can target the default bus of a different account directly (with a resource-based policy on the receiving bus granting that specific sender permission), which is how a security-tooling account (Part 1/Part 9's log-archive-style pattern) can centrally receive GuardDuty/Config compliance events from every workload account without each one individually pushing to a shared queue. Cross-region delivery works the same way via a rule targeting a bus ARN in another region — useful for centralizing operational events from a multi-region deployment (Part 12) into one region's monitoring/alerting stack, though it's worth remembering this is a one-directional push, not a replicated bus — each region's bus still only sees what's been explicitly routed to it.

Choosing Among SQS, SNS, EventBridge, Step Functions, and AppSync#

NeedReach for
Decouple two services, one consumer processes each messageSQS
Fan out one message to many independent subscribersSNS (often with SQS behind each subscriber)
Route events by content/source across many possible consumers, with schema governanceEventBridge (rules)
Move a stream/queue's contents into exactly one target, with transformationEventBridge Pipes
Orchestrate a multi-step, stateful process with retries, branching, and an audit trailStep Functions
Serve a client that needs exactly-the-fields-it-asks-for from multiple sources, with real-time pushAppSync
Bridge an existing app that already speaks AMQP/JMS/MQTT, not worth rewritingAmazon MQ
Fan out processing across thousands of independent items from an S3 manifestStep Functions Distributed Map

Amazon MQ — When Protocol Compatibility Matters#

Amazon MQ is a managed broker running actual Apache ActiveMQ or RabbitMQ, speaking industry-standard protocols (AMQP 0-9-1, AMQP 1.0, MQTT, OpenWire, STOMP) rather than a proprietary AWS API. It exists for exactly one reason, and AWS's own guidance is explicit about it: migrating an existing application that already speaks one of those protocols, without rewriting producer/consumer code to target SQS/SNS's API. For anything greenfield, SQS/SNS/EventBridge are the default — they scale further, cost less at low-to- moderate throughput, and require zero broker capacity planning, none of which Amazon MQ removes (it still bills per broker-hour, not per message, so it can actually be cheaper only at sustained high, steady throughput). For availability, a broker can be deployed single-instance (cheapest, no built-in failover) or active/standby across two AZs, where the standby takes over automatically on a failure — the same Multi-AZ shape RDS uses (Part 6), applied to a message broker instead of a database.

Idempotency for Public APIs#

A mobile client on a flaky connection will retry a request it never got a response for — and a naive POST /orders handler that creates a new order on every invocation will happily create duplicates from that single user action. The standard fix, expected knowledge on DVA-C02's troubleshooting domain: require an idempotency key (a client-generated UUID) on the request, and have the backend check a fast-lookup store (DynamoDB, with a short TTL) for that key before doing any real work — if it's already been seen, return the original response instead of repeating the side effect. This is application-level logic, not something API Gateway provides natively, but it belongs in this part because the failure mode it prevents is specifically a public-API-and-retry problem, not a general backend concern: a well-behaved internal service call inside a VPC rarely needs it, while anything reachable from an unreliable client network almost always does.

Tip

Step Functions' Standard workflows are exactly-once by design (revisited earlier in this part) — which is one more reason a long-running, side-effect-heavy public API operation (charging a payment, for instance) is often better modeled as starting a Standard workflow idempotently (keyed the same way) than as a single Lambda doing everything inline.

A Full Worked Example: A Public Order-Status API#

A retail platform needs to let its mobile app show a customer their order status in real time, without exposing internal fulfillment systems directly.

  1. Identity: A Cognito user pool handles customer sign-up/sign-in via the Hosted UI, federated with Google and Apple sign-in for lower-friction onboarding.
  2. Read path: An HTTP API, secured with a JWT authorizer pointed at the user pool, exposes GET /orders/{id} — a Lambda proxy integration reads from a DynamoDB table (Part 6) scoped to the caller's own sub claim.
  3. Live updates: Rather than polling, the mobile app opens an AppSync subscription on onOrderStatusChanged — internally, a fulfillment-side Step Functions workflow publishes an EventBridge event on each status transition, which a small Lambda target turns into an AppSync mutation, fanning out to every subscribed client instantly.
  4. Internal orchestration: The fulfillment workflow itself (validate → charge → reserve inventory → notify) runs as the Standard Step Functions state machine diagrammed above — long-running by fulfillment standards (minutes to hours), needing the full audit trail for customer-support investigations.
  5. Direct upload: When a customer needs to upload a return-request photo, the app exchanges its Cognito ID token at the identity pool for temporary AWS credentials scoped (via a policy variable on sub) to only that customer's own S3 prefix, uploading directly with no backend server or Lambda in the path at all.

Security Checklist for Public APIs#

  • Every public route has an explicit authorizer — no route silently left open by omission.
  • Throttling limits set at both the account and per-method level, sized for real expected traffic, not left at defaults.
  • WAF attached (REST APIs) for anything internet-facing and handling sensitive data (Part 9).
  • Cognito user pool has MFA enabled for anything beyond a low-sensitivity consumer app.
  • Identity pool IAM roles use sub-scoped policy variables — never one broad shared role for every authenticated identity.
  • Lambda authorizer cache TTL is understood and acceptable for the token-revocation risk it implies.
  • CloudTrail (Part 9) is capturing API Gateway management-plane changes; access logging is enabled on every stage.
  • No secrets or long-lived credentials embedded in a mobile/web client — only short-lived tokens ever leave the identity system.

API and Event Integration Best Practices — The Consolidated Checklist#

  • Default new APIs to HTTP APIs; move to REST only when a specific feature actually requires it.
  • Never build a custom auth Lambda for the common JWT-validation case — HTTP APIs' native JWT authorizer and REST APIs' Cognito authorizer both cover it without custom code.
  • Keep IAM identities for AWS principals and Cognito identities for application end-users strictly separate — never provision IAM users to represent app customers.
  • Scope every identity-pool IAM role with sub-based policy variables rather than one shared broad role.
  • Treat API keys as a usage-plan selector only, never as an authentication mechanism on their own.
  • Pick Standard vs Express Step Functions workflows by duration and volume, not by habit — the wrong choice is either an unnecessary per-state cost or a hard 5-minute ceiling hit in production.
  • Reach for EventBridge Pipes only for genuine one-source-to-one-target plumbing; use rules the moment more than one consumer might ever care about an event type.
  • Attach WAF (Part 9) to any public REST API handling sensitive data; HTTP APIs need a CloudFront distribution in front for the same protection, since HTTP APIs don't support WAF directly.
  • Confirm CORS is configured on both the preflight response and the real response — testing only via a tool that skips preflight (like a bare curl or Postman call) hides the most common failure mode.

Part 13 CLI Cheat Sheet#

TaskCommand
Create an HTTP APIaws apigatewayv2 create-api --name my-api --protocol-type HTTP --target <lambda-arn>
Create a JWT authorizeraws apigatewayv2 create-authorizer --api-id <id> --authorizer-type JWT --identity-source '$request.header.Authorization' --jwt-configuration Audience=<client-id>,Issuer=<issuer-url>
Deploy a REST API stageaws apigateway create-deployment --rest-api-id <id> --stage-name prod
Create a Cognito user poolaws cognito-idp create-user-pool --pool-name my-pool
Create a user pool clientaws cognito-idp create-user-pool-client --user-pool-id <id> --client-name web-app
Create an identity poolaws cognito-identity create-identity-pool --identity-pool-name my-identities --allow-unauthenticated-identities
Start a Step Functions executionaws stepfunctions start-execution --state-machine-arn <arn> --input '{"orderId":"123"}'
Describe an execution's historyaws stepfunctions get-execution-history --execution-arn <arn>
Create an EventBridge pipeaws pipes create-pipe --name my-pipe --source <arn> --target <arn> --role-arn <role-arn>
Create an AppSync APIaws appsync create-graphql-api --name my-api --authentication-type AMAZON_COGNITO_USER_POOLS

Common Mistakes and Interview Traps#

MistakeWhy it's wrongCorrect approach
Defaulting to REST APIs for a new serverless projectPays ~3.5x more per request for features never usedDefault to HTTP APIs; move to REST only when a specific feature (caching, usage plans) is actually needed
Provisioning IAM users for application end-usersDoesn't scale, conflates workforce and customer identityUse Cognito user pools for application identities
Treating an API key as authenticationAn API key only selects a usage plan — it grants no identity or authorization on its ownPair a usage plan/API key with a real authorizer
Using Step Functions Standard for a sub-second, high-volume event pipelinePer-state-transition pricing gets expensive fast at that volumeUse Express workflows for high-throughput, short-duration work
Assuming a Lambda authorizer's deny takes effect instantlyResults are cached (5 min default); a revoked token can still pass until the cache entry expiresSet an appropriately short cache TTL for anything security-sensitive, or explicitly invalidate
Building a hand-rolled WebSocket connection table just to push GraphQL updatesAppSync subscriptions already do this nativelyUse AppSync subscriptions when the API is already GraphQL
Choosing Amazon MQ for a new, greenfield queuing needPays for broker-hour capacity even during idle periods; more operational surface than SQSDefault to SQS/SNS/EventBridge; reach for MQ only for protocol-compatible migrations

Worked Practice Problems#

Problem 1: A mobile app needs to let authenticated users upload profile photos directly to S3, and the team is currently routing every upload through a Lambda function that just proxies bytes from the client to S3. What's the architectural improvement, and what AWS service makes it possible?

Answer: Route the upload directly from the client to S3, bypassing the Lambda proxy entirely. A Cognito identity pool, fed by the user pool's ID token, issues temporary AWS credentials scoped (via a sub-based policy variable) to only that user's own S3 prefix — the client uploads with those credentials directly. This removes Lambda's payload-size limits and cold-start latency from the upload path, and removes the cost of running compute purely to relay bytes it never needed to touch.

Problem 2: An order-processing workflow needs to pause and wait for a human manager's approval on any order over $10,000, potentially for hours, before continuing. What Step Functions pattern fits, and why not just have the Lambda function poll a database for the approval?

Answer: The .waitForTaskToken (Wait for Callback) integration pattern. The state machine pauses entirely — consuming no compute, incurring no polling cost or complexity — until an external system (an approval UI's backend) calls SendTaskSuccess or SendTaskFailure with the token issued when the state paused. Polling from inside a Lambda would either need a slow, wasteful loop or an entirely separate scheduling mechanism to re-invoke itself, reinventing what the callback pattern already provides natively.

Problem 3: A partner integration needs webhook-style events whenever an order's status changes, but different partners care about different subsets of event types, and the list of partners changes frequently. Would EventBridge rules or EventBridge Pipes fit better, and why?

Answer: Rules. Pipes are one-source-to-one-target; adding or removing partners would mean creating or deleting individual pipes per partner with no shared routing logic. A bus with per-partner rules, each matching on the event types that partner cares about and targeting that partner's own SQS queue or API destination, handles an arbitrary and changing number of many-to-many subscribers without restructuring the pipeline — exactly the shape EventBridge rules were built for.

Summary and What's Next#

API Gateway is the front door — three different products depending on whether the API needs REST's full feature set, HTTP's low cost and native JWT auth, or WebSocket's persistent push. Cognito is the identity system behind that door for customer-facing applications: user pools authenticate, identity pools convert that authentication into real temporary AWS credentials. AppSync offers the same front-door role for GraphQL clients that need to assemble data from multiple sources in one round trip. And Step Functions/EventBridge, revisited in depth, round out the event-driven orchestration toolkit Part 11 only introduced — Standard vs Express workflows for stateful orchestration, schema registry and Pipes for governed and point-to-point event delivery.

Part 14 moves from the request path to the deployment path: Elastic Beanstalk as a simpler PaaS-style alternative to the container/serverless patterns already covered, the AWS SAM framework for building and deploying serverless applications with a tighter feedback loop than raw CloudFormation, and the developer productivity tools (CodeArtifact, CodeGuru, Cloud9/CloudShell) that round out the DVA-C02 deployment domain.