A/B Test MDE Calculator
Estimate the smallest conversion-rate change your experiment can reliably detect.
This planner uses a standard normal approximation for two-sample proportion tests with equal group sizes.
What is a minimum detectable effect (MDE)?
The minimum detectable effect is the smallest true change your test is likely to pick up as statistically significant, given your traffic, confidence level, and power target. In plain English: it tells you how big of a lift (or drop) must exist before your experiment has a good chance of finding it.
If your MDE is too large, small but valuable improvements may go unnoticed. If your MDE is tight enough, you can confidently evaluate subtle product, pricing, UX, and marketing changes.
Why MDE matters before you launch an experiment
- Sets realistic expectations: You know whether your test can detect a 2% lift, 10% lift, or only very large shifts.
- Prevents underpowered tests: Running low-traffic tests without planning often leads to inconclusive outcomes.
- Aligns stakeholders: Teams can agree in advance on what effect size is practically meaningful.
- Improves prioritization: Big, high-impact ideas are better candidates when traffic is limited.
How this calculator works
Inputs
- Baseline conversion rate: Your current conversion probability.
- Sample size per variant: Users in control and treatment each.
- Alpha (significance): Type I error rate, commonly 5%.
- Power: Probability of detecting a real effect, commonly 80% or 90%.
- One-sided or two-sided: Two-sided checks both increase and decrease; one-sided checks only one direction.
Outputs
- Absolute MDE (percentage points): e.g., +0.80 pp.
- Relative lift: e.g., +16% over baseline.
- Detectable variant rates: Upper and lower rates your test can likely distinguish from baseline.
Quick interpretation example
Suppose your baseline conversion is 5.0%, each variant gets 10,000 users, alpha is 5%, and power is 80%. If the calculator returns an MDE around 1.0 percentage point, your test is tuned to reliably detect something like 6.0% vs 5.0% (or 4.0% vs 5.0% for a decrease), but not tiny changes like 5.2% vs 5.0%.
How to reduce MDE
1) Increase sample size
More users shrink uncertainty and make smaller effects detectable.
2) Improve measurement quality
Better event tracking, cleaner data pipelines, and stable attribution reduce noise, which improves detection sensitivity.
3) Raise power target thoughtfully
Higher power improves sensitivity but generally requires more traffic. Choose power based on decision risk and test cost.
4) Use stronger hypotheses
Tests with a clear directional hypothesis and substantial product change are more likely to exceed the detectable threshold.
Common mistakes with MDE planning
- Confusing statistical significance with business significance.
- Ending tests early when p-values look promising.
- Ignoring baseline variability over time (seasonality, campaigns, outages).
- Assuming all segments behave the same way.
- Running many tests simultaneously without accounting for interaction effects.
Bottom line
A minimum detectable effect calculator is a planning tool, not a crystal ball. But using it up front dramatically improves experiment quality. Before launching your next A/B test, confirm your MDE is aligned with the smallest change worth shipping.