A/B Test Sample Size Calculator
Find how many visitors per variant your A/B test needs to detect a given lift at your chosen confidence and power.
How it works
n per group = (z_α⁄2 + z_β)² · [ p₁(1−p₁) + p₂(1−p₂) ] ÷ (p₂ − p₁)²
Sample size is driven by four levers: the baseline rate, the smallest lift you want to catch, your confidence level (which sets z_α⁄2) and your desired power (which sets z_β). Smaller lifts are exponentially more expensive to detect because the difference sits in the denominator squared — halving the lift you care about roughly quadruples the visitors needed. Decide these numbers before launching so you do not stop the test at a misleading moment.
Worked example
Baseline 5%, minimum detectable relative lift 20% → target 6%. At 95% confidence (z = 1.96) and 80% power (z = 0.84): n = (1.96 + 0.84)² · [0.05·0.95 + 0.06·0.94] ÷ (0.01)² ≈ 7.84 · 0.1039 ÷ 0.0001 ≈ 8,145 visitors per variant.
Frequently asked questions
Is the lift here absolute or relative?
Relative. A 20% lift on a 5% baseline means detecting a rise to 6% (5% × 1.20), not to 25%. Relative lift is how most experiment goals are stated ("increase signups by 20%").
What confidence and power should I pick?
95% confidence and 80% power are the conventional defaults, giving a 5% false-positive rate and a 20% chance of missing a real effect. Raise power to 90% for high-stakes decisions, at the cost of a larger sample.
Why is my required sample so large?
Small conversion rates and small lifts both inflate the number dramatically. If the sample is impractical, either aim to detect a bigger lift or accept a lower confidence level — there is no way around the statistics for a truly tiny effect.
Related calculators
This calculator is for educational purposes. Double-check important results before acting on them.