Free Trial

Safari Books Online is a digital library providing on-demand subscription access to thousands of learning resources.

Share this Page URL

01. A/B Testing - Pg. 8

RESEARCH METHOD 01 A/B Testing Use A/B testing to compare two versions of the same design to see which one performs statistically better against a predetermined goal. 1 A/B testing is an optimization technique that allows you to compare two different versions of a design to see which one gets you closer to a business objective. 2 The tests are run by randomly assigning different people down two paths--the "A" test and the "B" test--until a statistically relevant sample size is reached. At the end of the test, you will be able to determine which design gets you closer to your goals. Take, for instance, the challenge of increasing the number of people who sign up for a free trial of your online service. There could be many explanations why people aren't registering: Is the sign-up form too long? Are people worried about their privacy and what you will do with their data? Do they want to know about pricing information before they register? You can find out the answer to each of these questions by making small modifications to the interface, and then run an A/B test to see which version prompts more people to register. For instance, given the scenario above, you can design and run several tests that compare: · different treatments of the page microcopy--the text that guides and reassures the user-- regarding the terms of the service (tone, length, font size); · the form elements (how many, layout, which are required); and · different treatments of the button or call to action (page placement, size, color, labeling). 1. A/B tests are adapted from the classic direct mail practice in which two different versions of the same mailing are sent out to different people in order to see which one gets the better response rates. 2. Nielsen, Jakob. "Putting A/B Testing in Its Place," 2005, 3. Kahavi, Ron, Randal M. Henne, and Dan Sommerfield. "Practical Guide to Controlled Experiments on the Web: Listen to Your Customer Not to the HiPPO." Proceedings of the 13th ACM SIGKDD, 2007.