Table of Contents#### Download Safari Books Online apps: Apple iOS | Android | BlackBerry

Entire Site

Free Trial

Safari Books Online is a digital library providing on-demand subscription access to thousands of learning resources.

80 CHAPTER 5 Is There a Statistical Difference between Designs? Many readers may find this approach more intuitive for three reasons: 1. It is often easier to think in terms of completion rates or conversion rates (measured as proportions) rather than the number of users who pass or fail. 2. We use the more familiar and readily available normal distribution as the reference distribution for finding p-values and don't need to worry about degrees of freedom. 3. The confidence interval formula uses the difference between the two proportions and makes for an easier transition in computation and understanding. The N - 1 two-proportion test uses the standard large sample two-proportion formula (as shown q ffiffiffiffiffiffiffiffi in the previous section) except that it is adjusted by a factor of r ffiffiffiffiffiffiffiffiffiffiffi N - 1 ð^ 1 - ^ 2 Þ p p N z = s ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi PQ × 1 + 1 n 1 n 2 where N - 1 . N This adjustment is algebrai- cally equivalent to the N - 1 chi-square adjustment. The resulting formula is ^ 1 and ^ 2 are the sample proportions p p À x 1 + x 2 Á P = n + n , where x 1 and x 2 are the numbers completing or converting, and n 1 and n 2 are the 1 2 numbers attempting Q = 1 - P N is the total sample size in both groups Using the example data we have 11 out of 12 (91.7%) completing on Design A and 5 out of 10 (50%) completing on Design B, for a total sample size of 22. First we compute the values for P and Q and substitute them in the larger equation: 11 + 5 = 0:727 and Q = 1 - 0:727 = 0:273 P = 12 + 10 r ffiffiffiffiffiffiffiffiffiffiffiffiffi 22 - 1 ð0:917 - 0:5Þ 22 z = r ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 1 + 0:727 × 0:272 × 12 10 z = 2:135 We can use a normal (z) table to look up the two-sided p-value, or the Excel function =(1-NORMSDIST(2.135))*2, which generates a two-sided p-value of 0.0328--the same p-value we got from the N - 1 chi-square test, demonstrating their mathematical equivalence. Table 5.8 summarizes the p-values generated from the sample data for all approaches and our recommended strategy.