The Advora toolkit
Compare conversion rates in two randomized groups using a two-sided proportion test, effect size and confidence interval, with small-sample warnings.
Compare two independent randomized groups using a two-sided, pooled proportion z-test. Count people who converted, not total orders or repeat conversions.
Works in your browser · No account requiredMethod: NIST difference of proportions test. The interval uses an unpooled normal approximation.
Loading helpful votes…
02 / THE EXPLAINER
The method behind the result.
Enter unique visitors and the number of those visitors who converted in each independent randomized group. A person is either a converter or not; do not enter repeat transactions as converting visitors. The tool is for a binary outcome, not revenue per visitor.
The test compares two conversion proportions using a pooled standard error and a two-sided normal approximation. It reports the absolute percentage-point difference, relative lift, p-value and an approximate unpooled difference interval. A p-value is not the probability that a variant is the winner.
At least ten converters and ten non-converters in each group are required before this tool reports inference. Smaller samples need an exact method or more observations. Repeated peeking, multiple comparisons, overlapping users and non-random allocation can make the result misleading. Statistical significance does not establish commercial importance.
03 / A LITTLE MORE CLARITY
What to know before you put your result to work.
90%, 95% and 99%, with a two-sided test.
The normal approximation is unreliable with too few events or non-events.
Relative lift is undefined. The absolute difference is still shown, but inference may be withheld.
No. This calculation is for binary conversion outcomes only.
No. It means this test did not establish a difference at the selected threshold.
