Detectable Calculator for A/B Tests
Use this tool to estimate the minimum detectable effect (MDE) for conversion tests. In plain English: how big of a change your test can reliably detect.
Tip: Leave “Observed variant conversion rate” blank if you only want MDE. Fill it in to also check approximate significance.
What is a detectable calculator?
A detectable calculator helps you answer one of the most important experiment design questions: “How large does the real change need to be before my test is likely to detect it?” In optimization, product analytics, and growth marketing, this is often called the minimum detectable effect (MDE).
If your sample size is too small, a meaningful lift might look like random noise. If your sample is large enough, even smaller lifts become detectable. This calculator gives you an estimate before (or during) a test so you can make better planning decisions.
How this calculator works
Inputs used
- Baseline conversion rate: Your current conversion rate for control.
- Visitors per variant: Sample size in each group (A and B assumed equal).
- Confidence level: Statistical significance threshold (90%, 95%, 99%).
- Power: Your chance of detecting a real effect if it exists.
- Observed variant rate (optional): Lets you check an approximate two-sided z-test p-value.
Core idea
The calculator uses a normal approximation for two-proportion testing to estimate detectable absolute lift:
MDE ≈ (zalpha + zbeta) × √(2 × p × (1 - p) / n)
Where p is baseline rate as a decimal and n is visitors per variant. This gives a practical, quick estimate that is commonly used for planning.
How to interpret your result
- Absolute lift (percentage points): The smallest difference likely detectable under your assumptions.
- Relative lift (%): Absolute lift divided by baseline (useful for business framing).
- Detectable range: Baseline ± MDE gives a rough zone where effects may become statistically visible.
Example: if baseline is 8.5% and MDE is 1.0 percentage point, then changes smaller than ±1.0 pp may often fail to reach significance with your current setup.
Practical tips to improve detectability
1) Increase sample size
This is the most direct lever. Larger samples reduce standard error and lower MDE.
2) Reduce noise in your experiment
Keep tracking clean, randomize correctly, and avoid instrumentation drift. Better data quality improves trust in results.
3) Keep test conditions stable
Major traffic source shifts, campaign changes, or UX rollouts during a test can inflate variance and blur true effects.
4) Set realistic power/confidence targets
Higher confidence and higher power are stricter and require larger sample sizes. Choose settings aligned with risk tolerance and decision impact.
Common mistakes this tool helps avoid
- Stopping tests too early because early fluctuations look exciting.
- Declaring “no effect” when the test was underpowered from the start.
- Comparing tiny lifts against a setup that can only detect large movements.
- Ignoring baseline rate: low and high baselines behave differently in variance terms.
Final note
This detectable calculator is meant for fast, practical planning. For high-stakes experiments, consider a full power analysis tailored to your exact metric, traffic split, and testing design. Still, as a day-to-day decision aid, this gives you a strong reality check on what your experiment can actually “see.”