What needs to be finished first
An established on-call rotation with escalation paths and SLO definitions exists, so the incident trigger has one clear recipient.
Runbooks for the most common failure modes of the affected systems exist and are linked from the alert context.
What needs to be ready before start
Incident channel template (Slack, MS Teams) with prebuilt commands; status-page access; paging tool (PagerDuty, Opsgenie, Grafana OnCall); conference bridge or video-call link; logging dashboard per service; escalation list.
Incident Commander (IC) coordinates; Operations Lead (OL) executes technical steps; Communications Lead (CL) keeps stakeholders informed; scribe records actions with timestamps; Subject Matter Experts (SMEs) on call.
Severity definition (SEV1-SEV3) with clear criteria; escalation paths; severity-based status-page templates; stakeholder list per severity; known service dependencies.
First IC handoff within 5 min, incident duration open, handoff every 2 h
Pager alert automatically opens an incident channel with topic, severity, IC slot. Bot posts role list and next steps. Conference bridge is attached to topic. Status-page flips to Investigating automatically for SEV1/SEV2.
The one question this method answers
Who is in each role, which effect should be checked next, and when is the next stakeholder update due?
Marker: Phase
| Step | Duration | Action | Hint |
|---|---|---|---|
1Phase 1: Activation | 0-5 min | First responder takes IC role and posts in the channel: I am IC, severity X, hypothesis Y. Update channel topic with IC name. Confirm pager alert. | If no one takes IC, pager escalates after 5 min. IC take over must be explicit, not implicit. No "someone will handle it". |
2Phase 2: Assign roles | 5-15 min | IC calls Operations Lead and Communications Lead. Scribe is named. SMEs are paged selectively, not everyone. Start conference bridge and keep channel as primary communication medium. | IC stays coordinator, not operator. If IC spends all time clicking, no one holds the overall view. For SEV1, assign OL immediately. |
3Phase 3: Investigation and mitigation | variable | OL runs hypothesis tests, scribe records timestamps. IC coordinates: what has been tested, next action, owner. CL posts status updates every 30 min to stakeholders. | Mitigation takes priority over root-cause identification. If rollback is available, OL runs rollback and causes analysis follows in postmortem. No parallel actions without IC confirmation. |
4Phase 4: Resolution and handoff | 15 min | IC confirms recovery based on SLO metrics, not gut feeling. Move channel topic to Resolved. Close status-page entry. For long-running incidents, perform handoff to new IC every 2 h. | Recovery needs two consecutive green measurement intervals. A single green tick is not enough. Handoff requires a written statement (status, open points, next actions). |
5Phase 5: Transition to postmortem | 15 min after resolution | IC creates postmortem ticket with incident ID, links channel log and timeline. Schedule postmortem within 5 working days. Collect action items in ticket before closure. | Anyone leaving channel before postmortem ticket exists risks losing lessons. Scribe is responsible for complete log export. |
What comes out at the end
Incident channel log with timestamps for all actions, role assignments, hypotheses, mitigations, status-page updates, and stakeholder communications, plus postmortem ticket with incident ID, severity, duration, and owner.
Use one unique ID per incident (INC-YYYY-NNNN). Archive channel log on closure. Postmortem document carries the same ID. Action item status remains current in the postmortem header.
Compact working template for Incident Command with context, input, output artifacts, and next step.
A role-based approach for coordinating larger incidents.
When and for what do we use this method?
Which data, observations, decisions, or materials are available?
Short notes along the runsheet.
Owner, date, and success signal.
Concrete filled scenario, fictional example
Concrete filled scenario, fictional example
Severity: SEV-2. Duration: 35 min. Status: Resolved at 09:47 UTC.
09:12 :alarm: PagerDuty SEV-2 order-api.error_rate >5%
09:13 @anna /ic-take — taking IC, hypothesis: deployment at 09:05?
09:14 @ben /role ops-lead — OL set, checking deployment revert option
09:15 @cara /role comms-lead — status-page on Investigating, first message to internal stakeholders
09:18 @ben: no deployment today, checking connection-pool
09:21 @ben: pool exhausted, correlates with batch-recon-202 at 09:08
09:23 @anna: OL stops batch, CL posts mitigation update
09:33 @ben: batch stopped, error rate dropping
09:42 @anna: first green interval, keep open
09:47 @anna: second green interval, /ic-resolve
09:48 @cara: status-page resolved, external comms sent
09:55 @anna: postmortem ticket POST-2026-0847 created, date 16.05.
Role handoff: none (incident under 2 h). Pre-postmortem action items: 3 (see ticket).
Recognize symptoms and steer against them
Incident Commander runs shell commands and loses track of actions by others.
Name OL and keep IC strictly on coordination. If IC must execute commands, hand over IC role first.
Multiple hypotheses run in parallel, nobody knows current tests, actions duplicate.
IC summarizes every 10 min: status, current action, next action. Do not run parallel tests without synchronization.
First external communication only appears after resolution, while customers escalate in parallel.
From SEV-2, CL posts every 30 min, even without progress. Silence creates more overhead than a factual interim update.
Incident is marked resolved after first green tick and fails again after 10 min.
Require at least two consecutive green intervals before resolution. For short intervals use three ticks.
IC remains active for 6 h, decision quality drops and fatigue increases.
Perform handoff every 2 h as a rule, even if IC wants to stay. A rested IC is a better decision-maker.
Channel is archived, lessons are lost, same incident class repeats.
Postmortem ticket is mandatory in the resolution checklist. Channel may be archived only after ticket link is posted.
Done signals checkable in under a minute
Go to the profile for purpose, similar methods, and sources or continue to the next method in the catalog.