172 CHAPTER 7 What Sample Sizes Do We Need? Part 2 (Continued ) 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 FIGURE 7.5 Fit of regression equation for predicting p adj from p. OTHER STATISTICAL MODELS FOR PROBLEM DISCOVERY Criticisms of the Binomial Model for Problem Discovery In the early 2000s, there were a number of published criticisms of the use of the binomial model for problem discovery. For example, Woolrych and Cockton (2001) pointed out that a simple point esti- mate of p might not be sufficient for estimating the sample size required for the discovery of a specified percentage of usability problems in an interface. They criticized the formula 1 - (1 - p) n for failing to take into account individual differences among participants in problem discoverability and claimed that the typical values used for p (0.3) derived from Nielsen and Landauer (1993) tended to be too optimistic. Without citing a specific alternative distribution, they recommended the development of a formula that would replace a single value of p with a probability density function. In the same year, Caulton (2001) also criticized simple estimates of p as only applying given a strict homogeneity assumption: that all types of users have the same probability of encountering all usability problems. To address this, Caulton added to the standard cumulative binomial probability formula a parameter for the number of heterogeneous groups. He also introduced and modeled the concept of problems that heterogeneous groups share and those that are unique to a particular sub- group. His primary claims were (1) the more subgroups, the lower will be the expected value of p; and (2) the more distinct the subgroups are, the lower will be the expected value of p. Kanis (2011) recently evaluated four methods for estimating the number of usability problems from the results of initial participants in formative user research (usability studies and heuristic