Optimizing the process

Because their executable specifications were previously very technical, the automation layer was complicated and hard to maintain. Tests described technical components as parts of larger flows, so they had to be automated by faking portions of the user workflows. Tim Andersen says that the test results also weren’t reliable:

“We were able to show the test working, and then we weren’t able to show working software. Our tests were lying (false green bar). For example, a borrower can borrow money if he is less than 18 years old. We’d have a test that if they are less than 18 on that day, it would say, “You’re not allowed to borrow without a co-signer.” If you change the date of birth to be over 18 years, it would say “OK, you can borrow without a co-signer.” Our test was green, but when we actually opened a browser and tried it in development, it didn’t work. Even though we coded the validation rule, it wasn’t hooked up in the right place. Our test code was setting up a loan within a fantasy state.”