Purpose
The method helps clarify funnel performance, user behavior, and experiments concretely. It makes user behavior measurable and derives experiments. The outcome is captured as experiment results, a decision log, and a learning summary.
Growth
A/B Testing
Turns funnel performance, user behavior, and experiments into a tangible result through defining the hypothesis, planning metrics and sample size, and interpreting results.
Does the variant deliver a statistically significant effect on the success metric without degrading the counter-metric?
HighAsync1-4 Wochen
How it works
The team follows the steps: formulate the hypothesis, plan metric and sample, build variants, run the experiment, and interpret the results. Each step is captured visibly. At the end, experiment results, a decision log, and a learning summary are available so decisions, tests, or actions can follow directly.
Visual orientation
Method sketch for a quick mental model.
Flow
- 1Formulate hypothesis
- 2Plan metric and sample
- 3Build variants
- 4Run experiment
- 5Interpret results
The runsheet guides execution with 5 phases, timeboxes, 6 pitfalls, and clear stop criteria.
Open runsheetIdeal for
- Conversion optimization
- Messaging tests
- Feature validation
Not good for
- Very small traffic
- Unclear hypotheses
Deep dive
In detail
A/B testing follows a clear working logic: formulate a hypothesis, plan metric and sample size, build variants, run the experiment, and interpret outcomes. This turns the method into a visible thinking process rather than only a conversation. Participants move step by step from raw material, observations, or options toward a shared structure. The result is experiment results, decision log, and learning summary that make decisions, learning, or further planning actionable.
Facilitation
Prepare a clear prompt, the right information, and a visible workspace. Plan about 1-4 weeks with 1-6 people and use the format asynchronously. The method is demanding and should be prepared carefully; short timeboxes, visible intermediate results, and a parking lot for open questions help.
Output artifacts
Experiment resultsDecision logLearning summary
Artifact templates
A/B Testing Working TemplateCompact working template for A/B Testing with context, input, output artifacts, and next step.markdown
A/B Testing Working TemplateCompact working template for A/B Testing with context, input, output artifacts, and next step.
markdownab-testing-working-template.md
Compact working template for A/B Testing with context, input, output artifacts, and next step.
A/B Testing Working Template
Goal
Compares two or more variants using defined success metrics.
Context
When and for what do we use this method?
Input
Which data, observations, decisions, or materials are available?
Execution
Short notes along the runsheet.
Output artifacts
- Experiment Result:
- Decision Log:
- Learning Summary:
Assumptions and open questions
- ...
Decision / Next step
Owner, date, and success signal.
When to choose differently
Short decision aid for existing alternatives.
Usability Testing
Statt A/B Testing, wenn ihr direktes Nutzungsverhalten sehen wollt, statt Annahmen im Team zu diskutieren.
Similar methods
All methodsGrowth
Pirate Metrics AARRR
Turns funnel behavior and experiments into a tangible result through defining funnel stages, selecting metrics per stage, and planning experiments.
Growth
Flywheel
Turns funnel, user behavior, and experiments into a tangible result by clarifying the growth goal and audience, sketching the flywheel loop, and prioritizing experiments or improvements.
Growth
Funnel Analysis
Turns funnel structure, user behavior, and experiments into a tangible result through defining the funnel, checking events, and deriving hypotheses.
Marketing
Growth Experiment
Turns hypotheses, target audience, channels, and measurement into a tangible result through defining the hypothesis, choosing audience and channel, and documenting learning.
Growth
North Star Metric
Turns funnel behavior, user behavior, and experiments into a tangible result through mapping customer value, collecting candidate metrics, and deriving input metrics.
Product Discovery
Smoke Test
Turns customer problem, solution idea, and evidence into a tangible result by defining the hypothesis and signal, building the minimal test, and deriving the decision.