What needs to be finished first
A baseline or target state is known and reproducible (for example a successful build, a comparable process, or a comparable day).
A chronological record of the failing process is available so changes can be placed in time.
What needs to be ready before start
Whiteboard or table with three columns (should, is, difference); access to version control, config states, deploy logs, staff shift plans; template for the difference list.
One lead investigator; 2-5 people who know the change areas (engineering, ops, data, supplier); one scribe; in safety-critical systems, one independent reviewer.
What worked last time, what no longer works now; time window of the deviation; list of known changes since baseline (deploys, configs, people, external systems); logs and diffs.
60-180 min, depending on complexity
Create a table with should, is, and difference columns. Use categories as rows: code, configuration, data, infrastructure, people, supplier, environment, process. Instruction: make should and is as concrete as possible per row, and note difference with source and timestamp.
The one question this method answers
Which differences between should and is explain the observed deviation, and which of those differences are causal, contributing, or random?
Marker: Phase
| Step | Duration | Action | Hint |
|---|---|---|---|
1Phase 1: Capture baseline and actual state | 20-30 min | Describe the baseline state (when it last worked) and the current state concretely per category. Version IDs, configuration hashes, staffing, environmental conditions. | If the baseline cannot be reconstructed concretely, Change Analysis will not work. Better spend time rebuilding the baseline than relying on surface-level guesses. |
2Phase 2: List differences | 30-60 min | List all differences between should and is per category. Capture even seemingly irrelevant changes (library update, new person on the team, changed supplier service). | Common mistake: only capture the obvious changes. Differences such as 'new security patch in the OS' or 'changed NTP server' often appear only with a systematic list. |
3Phase 3: Evaluate differences | 30-45 min | For each difference: causally plausible, contributing, or random. Provide a reason for each rating. If several causes are plausible, set the order of verification. | Avoid confirmation bias: the most obvious difference is not automatically the cause. For each difference, check when it would take effect and whether that fits the timeline. |
4Phase 4: Test the hypothesis | 30-90 min | Check the suspicious difference in isolation: rollback in staging, A/B comparison, targeted test. Document the result. | Test instead of guessing. If testing is not possible, at least check plausibility using timeline and logs. A hypothesis without a test remains a candidate, not a result. |
5Phase 5: Measures and lessons | 30-45 min | Define a corrective action for the causal difference. Structural lesson: why was this change possible without attention. Improve change management. | Often the change turns out to be unplanned or not communicated. Structural means not 'more reviews', but better detection of relevant differences, for example a change calendar or diff notifications. |
What comes out at the end
Difference table with should, is, difference, rating (causal, contributing, random), test result, and measure per relevant difference. Lessons learned about change visibility as an additional section.
Date, incident ID, investigator, and reviewer in the header. Later findings as an update section at the end. For follow-up incidents with a similar pattern, link back to the previous Change Analysis.
Compact working template for Change Analysis with context, input, output artifacts, and next step.
| Element | Description | Rating | Evidence | Owner | Next step |
|---|---|---|---|---|---|
| 1 | |||||
| 2 | |||||
| 3 |
What consequence follows from the matrix?
Concrete filled scenario, fictional example
Concrete filled scenario, fictional example
Should: Build succeeds, 4.2 min runtime, last successful run on 11.05.2026 17:23 UTC. Is: Build has been failing since 12.05.2026 09:14 UTC with error 'TLS handshake timeout' during artifact upload.
| Category | Should | Is | Rating |
|---|---|---|---|
| Code | Commit SHA a3f9 | Commit SHA b7c2 | Random (docs only) |
| CI image | golang:1.22.2 | golang:1.22.3 | Contributing (patch includes new CA certificates) |
| Artifact registry | registry.alpha.io | registry.alpha.io | Should = is |
| Network egress | Direct | Through new proxy since 12.05. 08:00 | Causal plausible |
| People | DevOps lead Tobias | Tobias on vacation since 12.05. | Contributing (no escalation path) |
Proxy configuration in the build container disabled: build succeeds (12.05. 11:40). Hypothesis confirmed.
Network changes were introduced by a separate team without notifying the build-pipeline owners. Structural gap: cross-team change visibility.
Recognize symptoms and steer against them
The should state is 'it worked before' without version, configuration, or timestamp.
Reconstruct the baseline with versioning, logs, and configuration snapshots. If that is not possible, pause the method until the baseline is clear.
The team lists three deploys, but other differences (OS patch, supplier service, personnel change) are missing.
Walk through a systematic category list (code, config, data, infrastructure, people, supplier, environment, process). Search actively per category instead of waiting to be told.
The team settles on the first plausible hypothesis and does not test alternatives.
Make at least two alternative hypotheses explicit. A devil's advocate checks the apparently confirmed hypothesis for gaps. Test in isolation, not in production.
The hypothesis is checked in the production system, so correction and test mix together and the insight blurs.
Test in staging or pre-prod. If that is not possible, use a controlled test with a rollback plan, a clear observation window, and an explicit hypothesis.
Measures fix symptoms, but the structural change-management gap is not addressed.
For each incident, define at least one structural lesson action: visibility, communication, reviews. Otherwise the pattern repeats.
Done signals checkable in under a minute
Go to the profile for purpose, similar methods, and sources or continue to the next method in the catalog.