Incident Management — Fundamentals
3 questions — read through for prep, or practice this domain interactively.
What's the difference between MTTD, MTTA, and MTTR, and which one matters most to focus on improving?Technical
How to answer
Define all three quickly and precisely, then take a real position on which one is most worth investing in and why — interviewers are testing judgment here, not just recall of the acronyms.
Example answer
MTTD (mean time to detect) is how long between the problem actually starting and someone/something noticing it. MTTA (mean time to acknowledge) is how long between an alert firing and a human picking it up. MTTR (mean time to resolve) is the whole thing, start to finish. In practice I'd focus on MTTD first — a fast MTTA and MTTR don't help if the incident already ran for 40 minutes before anyone knew, and MTTD gaps are usually the cheapest to close (better SLO-based alerting, synthetic checks) compared to MTTR, which depends on the actual complexity of the fix. That said, if MTTD is already tight and most of the incident duration is time-to-resolve, then MTTR work — better runbooks, faster rollback tooling — becomes the higher-leverage investment.
What interviewers listen for
correct, precise definitions (not conflating detect/acknowledge); an actual opinion on where to invest, backed by reasoning, rather than "they're all important."
Tell me about a time you led an incident that didn't go well.Behavioral
How to answer
Use STAR, but the key thing interviewers actually want here is honesty about what went wrong and what you changed afterward — a story where everything you did was perfect isn't believable and doesn't answer the question that was asked. Pick a real failure, not a near-miss dressed up as a failure.
Situation
During a database migration, I was the incident commander when a schema change locked a high-traffic table for far longer than expected, causing a full checkout-flow outage.
Task
I needed to both stabilize the immediate outage and coordinate communication to stakeholders who were asking for updates every few minutes.
Action
I made the mistake of trying to diagnose the lock myself instead of immediately delegating — as IC I should have been coordinating, not heads-down in a terminal. It took me almost ten minutes to realize I'd become a bottleneck, at which point I handed diagnosis to another engineer and refocused on comms and the rollback decision. We ultimately rolled back the migration and re-planned it for a maintenance window with the lock behavior pre-tested on a replica.
Result
The outage lasted 34 minutes, about 15 of which were avoidable delay from me not delegating fast enough. In the postmortem I owned that specifically, and we added an explicit rule to our IC training afterward: the IC's job is coordination, not the first person diagnosing — if you catch yourself heads-down for more than a couple minutes, hand it off.
What interviewers listen for
a real failure, not a humble-brag; specific ownership of what they personally did wrong, not just "the system failed"; a concrete, durable change that came out of it — not just "we learned to communicate better."
You get paged at 3am for a P1: checkout is returning 500s for roughly 20% of requests. Walk me through your first ten minutes.Scenario
How to answer
Show a triage order that prioritizes stopping the bleeding over root-causing — most candidates over-index on diagnosis and under-index on mitigation. Also show you'd communicate early, not just work silently.
Approach
Acknowledge the page and post an initial "investigating" update immediately, even before you know anything, so stakeholders aren't left guessing. Check what changed recently — deploys, config, feature flags — since a 20%-of-requests failure rate (not 100%) often points at a partial rollout or one bad instance/AZ rather than a total outage. Pull up the error-rate and latency dashboards scoped to checkout to confirm blast radius and whether it's growing or stable. If a recent deploy correlates, roll it back first and investigate root cause after — don't debug in place while customers are actively affected unless rollback is clearly not the cause.
Example answer
"First thing, before I even look at anything, I acknowledge the page and post a short 'investigating checkout 500s' update — that stops five people from independently paging me asking what's going on. Then I check what changed in the last hour: deploys, feature flags, config pushes. A 20% failure rate rather than 100% is a strong signal it's partial — one bad pod, one AZ, one canary — not a total outage, so I'd check the error distribution across pods/AZs next. If there's a deploy in the window that lines up, I roll it back immediately rather than trying to root-cause live; restoring service comes before understanding why. Only once it's mitigated do I dig into why, with a live incident channel and a written timeline going the whole time so the postmortem doesn't rely on anyone's memory."
What interviewers listen for
communicates before diagnosing in silence; explicitly reasons about the 20%-not-100% detail instead of ignoring it; prioritizes mitigation (rollback) over full root-cause during the live incident; keeps a timeline/channel running for the postmortem rather than treating that as an afterthought.