Calculate statistical significance for A/B test results. Input visitors, conversions, and confidence level to determine if results are reliable.
This tool runs the standard two-proportion z-test on your A/B results: it converts each variant's visitors and conversions into a conversion rate, measures how far apart the two rates are relative to the variation you would expect from pure chance, and returns a p-value against the confidence level you choose. The result most users get wrong: sample size matters as much as the rate gap. A jump from 5% to 9% with 200 visitors per variant can come back not significant, while 10.0% to 10.5% across 50,000 visitors per variant can be. The verdict tells you whether the difference is plausibly real, not whether it is worth shipping.
A concrete input and expected output from the current implementation.
Input
Variant A: 2,000 visitors, 200 conversions. Variant B: 2,000 visitors, 240 conversions. Confidence level: 95%.
Expected output
Variant A converts at 10.0%, variant B at 12.0%. Z-score: 2.02. P-value: 0.043. The difference is statistically significant at the 95% confidence level (p < 0.05).
The pooled conversion rate is 11%, giving a standard error of about 0.0099 for the difference; the 2-point rate gap divided by that error gives z = 2.02, whose two-tailed p-value is about 0.043, below the 0.05 threshold. At 99% confidence the same result would not be significant.
The tool pools both variants' conversions into one expected rate, computes the standard error of the difference from that pooled rate and both sample sizes, then divides the observed rate gap by that error to get a z-score: the larger the z-score, the less likely the gap is a chance artifact. Every conversion counts as one independent trial, so the inputs must be unique visitors and conversions, not pageviews or repeated events.
The z-score becomes a p-value: the probability of seeing a rate gap at least this large if both variants truly converted at the same rate. If the p-value is below the threshold implied by your confidence level (p < 0.05 for 95% confidence), the tool reports the difference as significant and rejects the no-difference assumption. Otherwise it reports that the evidence is insufficient to call a winner.
Because 200 visitors per variant gives the test very little power: with 10 of 200 versus 18 of 200 conversions, the z-score is only about 1.6 and the p-value about 0.12, far above 0.05. The gap is real in your data, but it is well within the range of random variation at that sample size. Let the test run longer, or accept that this much traffic cannot distinguish the two variants.
No. A significant result means the observed difference would be unlikely under pure chance, assuming the data were collected cleanly. It says nothing about how the variants will perform with different audiences, seasons, or traffic sources, and it does not mean there is a 95% chance the variant is better. For high-stakes changes, run longer or validate with a follow-up experiment.
The example and behavioral notes were checked against the browser implementation. Standards and primary references below define the relevant format, formula, or platform behavior.
Shorten URLs with local redirect tracking via localStorage. See click counts and generate a QR code for each link.
Build UTM-tagged URLs for campaign tracking with source, medium, campaign, term, and content parameters.
Enter 2+ headlines and see simulated previews in Google results, email subject lines, and social posts side-by-side.
Count characters for Twitter/X, LinkedIn, Facebook, and Instagram with platform-specific limits and warnings.
Calculate click-through rates from impressions and clicks with industry benchmarks and trend analysis.
Check email subject line length against best practices for desktop and mobile with preview text optimization.