Two-Proportion Z-Test
Compare the proportions of two independent groups, such as conversion or defect rates between a control and a variant. Get the z-statistic, p-value and confidence interval for the difference, with the sample-size conditions checked.
Run Two-Proportion Z-Test →What is a Two-Proportion Z-Test?
A two-proportion z-test compares the proportions of a binary outcome between two independent groups to determine whether they differ significantly. It is the standard test behind A/B testing of conversion rates and comparisons of defect or success rates between two processes.
Under the null hypothesis that the two proportions are equal, the test pools the successes from both groups to estimate a common proportion, then uses that to compute the standard error of the difference. The z-statistic measures the observed difference relative to this standard error.
As with the one-proportion test, the normal approximation requires adequate sample sizes: the expected numbers of successes and failures in each group should each be at least about 10. When groups are too small, an exact method such as Fisher's exact test is preferable.
In plain terms: You have two groups, say a control and a new version, each with a success rate, and you want to know if the difference between them is real. This test compares the two rates directly, accounting for the uncertainty in both, which is why it's the backbone of A/B testing.
Key Points
Pooled Estimate
Under the null of equal proportions, successes from both groups are pooled to estimate the common proportion for the standard error.
Difference & Interval
The confidence interval for the difference in proportions shows the size and direction of the effect, not just significance.
Sample-Size Conditions
Each group needs enough expected successes and failures (about 10 each) for the normal approximation to be valid.
Key Formulas
Interpreting the Result
A small p-value indicates the two group proportions differ by more than sampling variation would explain. The pooled standard error is used for the test because it assumes equal proportions under the null.
The confidence interval for the difference is the key practical output. If it excludes zero, the difference is significant, and its width and location show how large the difference is, which is what matters for a business or engineering decision.
Assumptions & Validation
Independent Groups
The two samples are independent of each other.
If violated: Paired binary data requires McNemar's test.
Random Samples
Each group is a random sample of independent binary outcomes.
If violated: Address dependence or sampling bias.
Adequate Sample Sizes
Expected successes and failures in each group are each at least about 10.
If violated: Use Fisher's exact test for small groups.
⚠️ Check assumptions first
The two-proportion z-test assumes the two groups are independent; for paired binary data, such as the same subjects measured twice, McNemar's test is required instead. Also confirm the normal-approximation conditions in both groups before trusting the p-value. When either group is too small, Fisher's exact test gives valid results where the z-test does not.
When NOT to Use Two-Proportion Z-Test
Paired Binary Data
For matched or before-and-after binary outcomes on the same subjects, use McNemar's test.
Small Samples
When expected counts are low, use Fisher's exact test.
More Than Two Groups
To compare proportions across three or more groups, use a chi-square test of independence.
Industry Applications
A/B Testing
Compare conversion rates between a control and a variant to decide which performs better.
Defect-Rate Comparison
Test whether two processes, lines, or suppliers have different defect rates.
Treatment vs Control
Compare success or response rates between a treatment and a control group.
Before-After Across Groups
Compare the proportion meeting a criterion between two independent groups.
Frequently Asked Questions
When should I use a two-proportion z-test?
Use it when you want to compare the proportion of a binary outcome, such as conversion or defect rate, between two independent groups. It is the standard test for A/B experiments and for comparing success rates between two processes or treatments. Both groups need to be large enough for the normal approximation, and the observations must be independent, not paired.
Why is the pooled proportion used in the test?
Under the null hypothesis, the two groups are assumed to share the same underlying proportion. Pooling the successes from both groups gives the best estimate of that common proportion, which is then used to compute the standard error of the difference. This pooled standard error is specific to the hypothesis test; the confidence interval for the difference uses the separate group proportions instead.
What sample size does the two-proportion z-test need?
Each group should have an expected number of successes and failures of at least about 10 for the normal approximation to the binomial to be reliable. When either group is smaller than this, or has proportions near zero or one, the approximation becomes poor and Fisher's exact test, which computes probabilities directly, is the appropriate alternative.
What is the difference between this test and McNemar's test?
The two-proportion z-test compares proportions between two independent groups. McNemar's test compares proportions in paired or matched binary data, such as the same subjects measured before and after, where the two measurements are related. Using the two-proportion z-test on paired data ignores the pairing and gives incorrect results, so the dependence structure determines which test is valid.
Is the two-proportion z-test the same as a chi-square test?
For comparing two proportions, the two-tailed two-proportion z-test and the chi-square test of a two-by-two table give equivalent p-values, since the z-statistic squared equals the chi-square statistic. The z-test has the advantage of supporting one-tailed, directional hypotheses and producing a confidence interval for the difference, which the chi-square test does not directly provide.
How do I interpret the confidence interval for the difference?
The confidence interval gives the plausible range for the true difference between the two proportions. If it excludes zero, the difference is statistically significant at the corresponding level. Its position and width reveal how large the difference is likely to be and in which direction, which is essential for deciding whether a statistically significant A/B result is large enough to act on.
Compare Two Proportions
Run a two-proportion z-test for A/B and rate comparisons. Free during Beta.
Run Two-Proportion Z-Test →