Chapter Ten: Testing Assumptions, Not Ideas

Armed with your “leap of faith” assumptions for three ideas, you might be tempted to rush into assumption testing. It’s exhilarating to get an experiment live and start collecting data. In my coaching program, teams learn to identify assumptions one week and then learn to test assumptions in the following week. Sometimes teams get so excited about testing assumptions that they come to their session during the identifying-assumptions week and report that they’ve already started collecting data on some of their top assumptions.

I admire these teams’ bias toward action. But more often than not, we find problems with their assumption tests. Sometimes their tests aren’t designed to test their “leap of faith” assumption but instead are designed to test the whole idea. Even after doing all the work to identify our riskiest assumptions, it’s easy to get distracted by our great ideas. Sometimes the team hasn’t agreed on what success looks like upfront, and they aren’t sure how to interpret their results. Sometimes they test with the wrong audience, or they get distracted by interesting, but not meaningful, data. These teams are all smart, capable, motivated product trios. However, it’s easy to rush into experimenting before we are ready.

When we rush our experiments, we tend to throw spaghetti at the wall, hoping something sticks. We try variations with abandon, instead of systematically searching for our best option. We run countless tests with little impact. We forget to clearly define what we are trying to learn and what success looks like, leaving us with ambiguous results.

If you’ve ever run an experiment and weren’t sure how to interpret the results, or if you’ve ever wondered if your prototype feedback was good enough, this chapter is for you. You’ll learn how to slow down (just a little bit) to make sure you get more value from each and every assumption test.