methodatlas
RunsheetProduct Discovery

Fake Door Test

ComplexityMedium
Time1-5 Tage
ParticipantsNutzertraffic
FormatAsync
MaturityEstablished
01

Prerequisite

What needs to be finished first

Complete firstAssumption Surfacing

The demand hypothesis to be tested is explicitly documented and linked to a measurable success threshold.

Without: Without a clear hypothesis and threshold, the outcome will be interpreted in both directions after the test.
02

Preparation

What needs to be ready before start

Materials

Visible entry point (menu item, button, card) in an existing product or marketing surface; tracking setup (analytics event); explanatory page or modal after click; definition of success.

People / roles

A Product Manager who sets the hypothesis and threshold; an engineer for tracking and explanatory page; an analyst for interpretation; a customer support owner for incoming questions.

Pre-read

Demand hypothesis as one sentence; target signal (for example, at least 5% click-through at 10,000 impressions); legal and ethical review (GDPR, consumer protection); plan for informing people who clicked.

Time needed

1-5 days runtime, plus 1-2 days setup

Setup

Place the entry point in the live product. Click leads to a transparent explanatory page ("This feature is still being developed, we are collecting interest"). Implement and test tracking events.

03

Core question

The one question this method answers

Does the measured demand at the fake-door entry clearly confirm the demand hypothesis above the predefined success threshold?

04

Flow

Marker: Phase

StepDurationActionHint
1Phase 1: Hypothesis and threshold
Setup, 2-4 hFormulate hypothesis ("At least 5% of active users in solo tax advisor dashboard click a new menu item 'Client switch service' within 5 days"). Write threshold and evaluation window up front.Do not define the threshold only after the test. Otherwise every outcome can be rationalized.
2Phase 2: Build fake door
Setup, 4-8 hPlace entry point with clear label. Prepare explanatory page that appears after click and manages expectations. Implement and test tracking events.The explanatory page is not optional. If users land in uncertainty, this is deceptive behavior and trust is damaged.
3Phase 3: Go live and observe
Run, 1-5 daysLaunch test live and distribute traffic. Check tracking daily. Collect support questions. Intervene immediately if anomalies occur (technical error, unclear wording).Very high click rates can indicate confusing wording rather than demand. Cross-check with time on explanatory page.
4Phase 4: Synthesize and learning decision
Synthesis, 2-4 hCompare tracking data with threshold. Include support questions and explanatory-page behavior in interpretation. Document the learning decision: build, build differently, or drop.Clear failure to reach threshold is also learning. Treating this as failure misunderstands the method.
05

Artifact

What comes out at the end

Form

Markdown experiment report with hypothesis, threshold, setup description, tracking data (impressions, clicks, click-through, time on page), support request sample, learning decision, and follow-up experiments.

Versioning / ownership

One entry per experiment with date and hypothesis. Link prior experiments. Record learning decision as status in header (Build/Pivot/Kill).

Tool alternatives
  • Amplitude or Mixpanel for tracking
  • Google Analytics 4 with custom events
  • Notion or Confluence for experiment report
  • Linear or Jira ticket with report attachment

experiment-plan-markdown.md

Short plan for hypothesis, test design, success criteria, and decision.

Experiment Plan

Hypothesis

We believe that ...

Target audience

Who are we testing for?

Test design

What exactly will participants or users do?

Success criterion

We count the test as positive if ...

Risks and limits

What can the test not prove?

Decision afterward

If positive: ... If negative: ...

Owner and date

...

06

Example output

Concrete filled scenario, fictional example

fake-door-test-beispiel.md

Concrete filled scenario, fictional example

Fake Door Test — Tenant Switch Service (2026-05-10 to 2026-05-15)

Hypothesis: At least 4% of active solo tax advisors click a new menu item "Tenant Switch Service" within 5 days.

Setup: Menu item added in sidebar between "Clients" and "Reports". Explanatory page with text: "We are building this offer right now, leave your email if you want to be informed first" plus micro-survey "What would be important to you?".

Results:

  • Impressions: 4,812 unique sessions
  • Clicks: 287 (5.96% click-through, threshold 4% reached)
  • Email captures: 142 (49.5% of clickers)
  • Survey answers: 89, 71% name liability as the top requirement
  • Support requests: 12, all polite, no complaints about explanation

Learning decision: Build. At least 142 qualified leads. Next experiment: Concierge MVP with 5 of these leads from 2026-05-25. Owner: @anna.

07

Pitfalls

Recognize symptoms and steer against them

Trap

No explanatory page

Symptom

Clicking users arrive at nothing or see an error page, and trust drops.

What to do

An explanatory page is mandatory. It must clearly state that the feature is planned. Optional email capture should be transparent.

Trap

Misleading label

Symptom

Very high click rate but short time on explanatory page. Users expected something else.

What to do

Validate wording with 2-3 users beforehand. If language is misleading, pause test and clarify text.

Trap

Threshold adjusted afterward

Symptom

Result is just below threshold, team shifts threshold.

What to do

Fix threshold before test and record it in report header. Changes are allowed only in follow-up experiment, not retroactively.

Trap

Sensitive promises without compliance

Symptom

Fake door promises something that would trigger regulatory constraints in production (health, finance).

What to do

Run legal and compliance review before each fake-door in regulated areas. In doubt, run a different experiment.

Trap

Too short runtime

Symptom

Data evaluated after 24 hours, weekly pattern not covered.

What to do

At least 3-5 days to smooth weekly variance. Run longer with low traffic.

08

Stop criteria

Done signals checkable in under a minute

No explanatory page prepared, ethical requirements not met.
Hypothesis is not formulated measurably.
Tracking does not work reliably, data not trustworthy.
Sensitive use case without legal review.
Expected traffic below 500 sessions in 5 days; statistics are not meaningful.
Team refuses to fix threshold upfront, and the test becomes a political instrument.

Finished the runsheet?

Go to the profile for purpose, similar methods, and sources or continue to the next method in the catalog.