Free Trial

Safari Books Online is a digital library providing on-demand subscription access to thousands of learning resources.

Share this Page URL

Confidence Interval around the Differenc... > Confidence Interval around the Diffe... - Pg. 89

Comparing Completion Rates, Conversion Rates, and A/B Testing 89 Confidence Interval around the Difference for Matched Pairs To estimate the likely magnitude of the difference between matched pairs of binary responses, we recommend the appropriate adjusted-Wald confidence interval (Agresti and Min, 2005). As described in Chapter 3 for confidence intervals around a single proportion, this adjustment uses the same concept as that for the between-subjects confidence interval around two proportions. When applied to a 2 × 2 table for a within-subjects setup (as shown in Table 5.14), the adjust- ment is to add one-eighth of a squared critical value from the normal distribution for the specified level of confidence to each cell in the 2 × 2 table. For a 95% level of confidence, this has the effect of adding two pseudo observations to the total number of trials (N). z 2 Using the same notation from the 2 × 2 table with the "adj" meaning to add 8 to each cell, we have the formula s ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ð^ 12adj + ^ 21adj Þ - ð^ 21adj - ^ 12adj Þ 2 p p p p p ð^ 2adj - ^ 1adj Þ ± z p N adj where m ^ 1adj = N adj p adj r adj ^ 2adj = N p