The Evolution of Modern Product Discovery

Product management is quickly evolving. Over the past 30 years, with the rise of the Internet, our industry has seen a rapid evolution in how we do both discovery and delivery. As a result, we see tremendous variation in our practice. Product management looks different everywhere. A brief history of the evolution of modern discovery can help us understand why this variation still exists. It can also give us a clear picture of how we can improve, wherever we are in the progression.

For many years, traditional discovery was not done by the product team. In the early days of software, business leaders owned discovery—they decided what to build. Discovery happened once a year in an annual budgeting process, where projects with fixed timelines were assigned to specific engineering teams. A project manager managed the work, budget, and schedule. Sometimes a product manager translated business needs to product requirements, but not always. There were (and still are) many challenges with this way of working. Software development is unpredictable. Projects were often delivered late and over budget. Business needs often trumped customer needs. Teams learned after the product shipped that customers weren’t excited about what they built. This way of working led to a lot of waste. Sadly, I still meet many teams and companies that work this way. Marty Cagan refers to these types of teams as delivery teams.²

Fortunately, in 2001, a group of engineers got fed up, went into the mountains, put their heads together, and came up with the Agile manifesto. This group of software engineers, influenced by the broader industry discussion about the pain points of developing software, proposed a number of principles to correct for what they saw. Projects were too big. Teams spent way too much time building the wrong stuff before they learned that customers didn’t want it. The authors of the Agile manifesto advocated for shorter cycles with more frequent customer feedback. Second, they proposed working at a pace that could be sustained continuously, rather than furiously scurrying from one milestone to another. Third, they advocated for maximum flexibility—having the ability to adapt to customer feedback quickly and easily. And fourth, they advocated for simplicity. They were concerned with how much of what they built was never used or offered limited value and instead advocated for teams to ruthlessly limit what they built. You’ll see these four principles infused throughout the methods in this book.

In the years following the Agile manifesto, teams worked to adopt these principles. We saw a rise in the adoption of Scrum and Kanban, two popular Agile frameworks, to manage delivery work. In parallel, we saw the growth of user-experience design and user research as means for collecting customer feedback. But this way of working also ran into challenges.

Leaders struggled to give up ownership of discovery. Even with shorter cycles and more customer feedback, business stakeholders still clung to their original ideas. Most teams weren’t very good at estimating unpredictable work (who is?), and their shorter cycles, aptly named sprints in Scrum, truly became biweekly sprints, killing any chance of finding a continuously sustainable pace. The rest of the business continued operating on an annual budgeting cycle, making true flexibility nearly impossible. When teams learned something wouldn’t work, they were still expected to deliver it on time and under budget. Usability testing was often done too late in the process, making it hard to address the substantial issues that were so often uncovered. User research was often outsourced to design agencies who did project-based research. And finally, teams continued to be measured by what they delivered, not whether anyone used it or if it created any value for the customer or the business.

However, it wasn’t all bad news. Teams did shorten their delivery cycles. Companies iterated from annual releases to quarterly releases to monthly releases. Today, many teams work on a weekly or even daily release schedule. More frequent releases meant we could measure the impact of what we were building sooner. We got better at instrumenting our products. We got better at usability testing our solutions. We got better at starting small and iterating to bigger solutions. These were giant steps in the right direction. But we still struggled with deciding what to build. We still learned, after shipping code, that we’d built the wrong stuff.

More instrumentation, however, did make us acutely aware of this problem. We could now measure when we released a feature that nobody used, when we redesigned our navigation and our metrics didn’t move, and when we added a product to our portfolio that nobody bought. These were hard lessons to learn. But the upside was that we started to question how we made decisions about what to build. We started with who should make those decisions. We started to push decision-making from business stakeholders to product managers and eventually to the whole product team. We started to question how we made discovery decisions. Instead of making them in conference rooms with just our own thoughts, we started engaging customers throughout the discovery process. Instead of just validating our ideas at the end of discovery, we started co-creating with customers from the very beginning. And our discovery cadence started to change. As our delivery cycles got shorter, so, too, did our discovery cycles. And this is where we are today.

Today, many teams are adopting, developing, and iterating on their own continuous discovery practices. They are engaging with customers on a regular basis. They are testing their assumptions. Rather than just validating their ideas, they are co-creating with customers—combining the team’s knowledge of what’s technically possible with the customer’s knowledge of their own needs, pain points, and desires to build better products. They are doing all of this on a continuous cadence, supporting the continuous development of their products. They are adapting to changes in the market, in customer needs, and in technology, in real time. So can you. This book will show you how.