Complete firstExperiment design
Write down audience, problem, promised value, requested action, traffic source, observation window, and decision threshold before building.
Choose an action that requires real effort or willingness to be contacted.
Method guideProduct Discovery
Smoke Test
A practical method guide: clarify prerequisites and materials, understand the complete process, and look up outputs, pitfalls, and stop criteria.
ComplexityLow
Time1-5 Tage
ParticipantsNutzertraffic
FormatAsync
MaturityEstablished
01
Prerequisite
What needs to be finished first
Complete firstExposure and evidence
Show only enough of the offer to enable a real behavioural choice. Test the CTA, counting rule, stop condition, and follow-up for interested people before launch.
Explain availability and purpose; follow up reliably with interested people.
02
Preparation
What needs to be ready before start
Materials
Hypothesis sheet, credible offer stub, one clear call to action, defined audience or traffic source, tracking plan, and privacy notice.
People / roles
Experiment owner · offer and copy owner · tracking and privacy owner · decision owner
Pre-read
Write down audience, problem, promised value, requested action, traffic source, observation window, and decision threshold before building.
Time needed
2–5 days for setup, exposure, and analysis; design work alone usually takes 45–90 minutes.
Setup
Show only enough of the offer to enable a real behavioural choice. Test the CTA, counting rule, stop condition, and follow-up for interested people before launch.
03
Core question
The one question this method answers
Does the defined audience choose the promised next action under realistic conditions?
04
Flow
Phases with goal, actions, output and pitfall
A Smoke Test offers a credible pre-step and measures whether a defined audience chooses the next action. It tests interest, not complete product quality.
Smoke test experiment card
Fictional example
Fictional example for a pickup service. Each row connects assumption, test, observable signal, and decision.
| Experiment | Hypothesis | Test and audience | Signal and threshold | Observation and decision |
|---|---|---|---|---|
| Tool pickup slotDoes the defined audience choose the promised next action under realistic conditions? | Weekend customers reserve a Saturday pickup slot before the service is built. | Offer page sent to 80 previous weekend customers; one 'Reserve a slot' CTA. | Qualified: email plus preferred date. Pre-set rule: investigate further from 8 reservations. | 5 reservations. Threshold missed; interview people about reasons and revise the offer. |
1. Hypothesis
- Tool pickup slot
- Weekend customers reserve a Saturday pickup slot before the service is built.
2. Test and audience
- Tool pickup slot
- Offer page sent to 80 previous weekend customers; one 'Reserve a slot' CTA.
3. Signal and threshold
- Tool pickup slot
- Qualified: email plus preferred date. Pre-set rule: investigate further from 8 reservations.
4. Observation and decision
- Tool pickup slot
- 5 reservations. Threshold missed; interview people about reasons and revise the offer.
The numbers are fictional and do not demonstrate real demand.
- 1
Hypothesis
Define assumption and audience
Goal
Testable demand hypothesis: State an observable demand assumption with audience, problem, and desired action.
Inputs for this step
- Write down audience, problem, promised value, requested action, traffic source, observation window, and decision threshold before building.
Actions
- State an observable demand assumption with audience, problem, and desired action.
Output
Testable demand hypothesis
Testable demand hypothesis
Quality check
Are audience, problem, value, action, and threshold explicit before launch?
Pitfall
The hypothesis merely says that people are interested and omits both action and decision threshold. Return to “Define assumption and audience” and repeat this method step: State an observable demand assumption with audience, problem, and desired action.
Timing: 30–45 minutes
- 2
Signal
Build minimal test
Goal
Testable offer stub: Provide a credible pre-step with exactly one next action.
Inputs for this step
- Testable demand hypothesis
Actions
- Provide a credible pre-step with exactly one next action.
Output
Testable offer stub
Testable offer stub
Quality check
Does the test promise only what the team can transparently fulfil or explain afterwards?
Pitfall
The offer stub has several CTAs or appears available although the service cannot yet be provided. Return to “Build minimal test” and repeat this method step: Provide a credible pre-step with exactly one next action.
Timing: 0.5–2 days
- 3
Traffic
Reach relevant people
Goal
Documented target-audience reach: Document source and inclusion criterion; do not treat arbitrary reach as evidence.
Inputs for this step
- Testable offer stub
Actions
- Document source and inclusion criterion; do not treat arbitrary reach as evidence.
Output
Documented target-audience reach
Documented target-audience reach
Quality check
Does exposure reach the defined audience, with a traceable source?
Pitfall
Reach is purchased without documenting source, audience criteria, or repeat contacts. Return to “Reach relevant people” and repeat this method step: Document source and inclusion criterion; do not treat arbitrary reach as evidence.
Timing: 1–3 days
- 4
Observation
Measure behavior
Goal
Clean behavioural data: Capture visits and qualified actions in the same period.
Inputs for this step
- Documented target-audience reach
Actions
- Capture visits and qualified actions in the same period.
Output
Clean behavioural data
Clean behavioural data
Quality check
Are visits and qualified actions defined for the same observation window and technically verified?
Pitfall
Clicks, sign-ups, and conversations are mixed or counted across different windows. Return to “Measure behavior” and repeat this method step: Capture visits and qualified actions in the same period.
Timing: During exposure; 20 minutes for data checks
- 5
Decision
Turn learning into action
Goal
Learning note with decision: Record signal strength, uncertainty, and next test separately.
Inputs for this step
- Clean behavioural data
Actions
- Record signal strength, uncertainty, and next test separately.
Output
Learning note with decision
Learning note with decision
Quality check
Does the analysis separate observed behaviour, possible bias, and the resulting decision?
Pitfall
A positive signal is reported as proven demand although sample and selection effects remain open. Return to “Turn learning into action” and repeat this method step: Record signal strength, uncertainty, and next test separately.
Timing: 30–45 minutes
05
Artifact
What comes out at the end
Sketch: what the artifact looks like
Form
Experiment report containing hypothesis, test setup, traffic source, observation window, visits, qualified actions, limitations, and next decision.
Versioning / ownership
Version the learning note with decision with date, owner, source state, and review date. Preserve earlier observations and append changes with their reason.
Tool alternatives
- Simple offer page, fake door, or form with one CTA
- Analytics with tested events and documented traffic source
- Contact and consent log for interested people
experiment-plan-markdown.md
Short plan for hypothesis, test design, success criteria, and decision.
Experiment Plan
Hypothesis
We believe that ...
Target audience
Who are we testing for?
Test design
What exactly will participants or users do?
Success criterion
We count the test as positive if ...
Risks and limits
What can the test not prove?
Decision afterward
If positive: ... If negative: ...
Owner and date
...
smoke-test-working-template.md
Plan the hypothesis, setup, real run, and analysis with clear ownership.
Smoke Test: Worksheet
Question
To be confirmed
Desired outcome
To be confirmed
Scope
Whole initiative Time: 2–5 days for setup, exposure, and analysis; design work alone usually takes 45–90 minutes.
Complete this worksheet on paper or in your own document during the work.
Preparation for this scope
Method setup
- Materials: Hypothesis sheet, credible offer stub, one clear call to action, defined audience or traffic source, tracking plan, and privacy notice.
- Roles: Experiment owner · offer and copy owner · tracking and privacy owner · decision owner
- Advance information: Write down audience, problem, promised value, requested action, traffic source, observation window, and decision threshold before building.
- Overall time needed: 2–5 days for setup, exposure, and analysis; design work alone usually takes 45–90 minutes.
- Setup: Show only enough of the offer to enable a real behavioural choice. Test the CTA, counting rule, stop condition, and follow-up for interested people before launch.
Prepare for the work steps
Hypothesis: Define assumption and audience
- Write down audience, problem, promised value, requested action, traffic source, observation window, and decision threshold before building.
Signal: Build minimal test
- Testable demand hypothesis
Traffic: Reach relevant people
- Testable offer stub
Observation: Measure behavior
- Documented target-audience reach
Decision: Turn learning into action
- Clean behavioural data
Smoke test experiment card
| Experiment | Hypothesis | Test and audience | Signal and threshold | Observation and decision |
|---|---|---|---|---|
| Tool pickup slot |
- Tool pickup slot: Does the defined audience choose the promised next action under realistic conditions?
Complete the hypothesis and decision threshold before exposure.
Strategyzer · original teaching example
Hypothesis: Define assumption and audience
Expected artifact: Testable demand hypothesis
Testable demand hypothesis
Entry:
...
- Are audience, problem, value, action, and threshold explicit before launch?
Signal: Build minimal test
Expected artifact: Testable offer stub
Testable offer stub
Entry:
...
- Does the test promise only what the team can transparently fulfil or explain afterwards?
Traffic: Reach relevant people
Expected artifact: Documented target-audience reach
Documented target-audience reach
Entry:
...
- Does exposure reach the defined audience, with a traceable source?
Observation: Measure behavior
Expected artifact: Clean behavioural data
Clean behavioural data
Entry:
...
- Are visits and qualified actions defined for the same observation window and technically verified?
Decision: Turn learning into action
Expected artifact: Learning note with decision
Learning note with decision
Entry:
...
- Does the analysis separate observed behaviour, possible bias, and the resulting decision?
Open questions and next steps
...
Method guide: https://methodatlas.meierhoff-systems.de/en/methods/smoke-test/run-sheet
06
Example output
Concrete filled scenario, fictional example
Fictional example for orientation. Counts and timings describe this case and are not universal requirements.
smoke-test-beispiel.md
Concrete filled scenario, fictional example
Fictional teaching example: Smoke test experiment card
Guiding question: Does the defined audience choose the promised next action under realistic conditions?
| Hypothesis | Test and audience | Signal and threshold | Observation and decision |
|---|---|---|---|
| Weekend customers reserve a Saturday pickup slot before the service is built. | Offer page sent to 80 previous weekend customers; one 'Reserve a slot' CTA. | Qualified: email plus preferred date. Pre-set rule: investigate further from 8 reservations. | 5 reservations. Threshold missed; interview people about reasons and revise the offer. |
The numbers are fictional and do not demonstrate real demand.
07
Pitfalls
Recognize symptoms and steer against them
Quick overview, the full text is in the matching phase in the flow
- Define assumption and audienceThe hypothesis merely says that people are interested and omits both action and decision threshold.View in phase
- Build minimal testThe offer stub has several CTAs or appears available although the service cannot yet be provided.View in phase
- Reach relevant peopleReach is purchased without documenting source, audience criteria, or repeat contacts.View in phase
- Measure behaviorClicks, sign-ups, and conversations are mixed or counted across different windows.View in phase
- Turn learning into actionA positive signal is reported as proven demand although sample and selection effects remain open.View in phase
Trap
Define assumption and audience
Symptom
The hypothesis merely says that people are interested and omits both action and decision threshold.
What to do
Return to “Define assumption and audience” and repeat this method step: State an observable demand assumption with audience, problem, and desired action.
Trap
Build minimal test
Symptom
The offer stub has several CTAs or appears available although the service cannot yet be provided.
What to do
Return to “Build minimal test” and repeat this method step: Provide a credible pre-step with exactly one next action.
Trap
Reach relevant people
Symptom
Reach is purchased without documenting source, audience criteria, or repeat contacts.
What to do
Return to “Reach relevant people” and repeat this method step: Document source and inclusion criterion; do not treat arbitrary reach as evidence.
Trap
Measure behavior
Symptom
Clicks, sign-ups, and conversations are mixed or counted across different windows.
What to do
Return to “Measure behavior” and repeat this method step: Capture visits and qualified actions in the same period.
Trap
Turn learning into action
Symptom
A positive signal is reported as proven demand although sample and selection effects remain open.
What to do
Return to “Turn learning into action” and repeat this method step: Record signal strength, uncertainty, and next test separately.
08
Stop criteria
When to pause, resolve prerequisites, or choose another approach
Pause or return to the previous step when this check is not satisfied: Are audience, problem, value, action, and threshold explicit before launch?
Pause or return to the previous step when this check is not satisfied: Does the test promise only what the team can transparently fulfil or explain afterwards?
Pause or return to the previous step when this check is not satisfied: Does exposure reach the defined audience, with a traceable source?
Pause or return to the previous step when this check is not satisfied: Are visits and qualified actions defined for the same observation window and technically verified?
Pause or return to the previous step when this check is not satisfied: Does the analysis separate observed behaviour, possible bias, and the resulting decision?
Finished the method guide?
Go to the profile for purpose, similar methods, and sources or continue to the next method in the catalog.