Product manager's dilemma: data or intuition?
Do I follow my gut? Or the data?
Every product manager (and entrepreneur) seems to struggle with this question. And unfortunately, there is no hard-and-fast “right” answer about how to best design new products. As a result, product managers often take an “it depends” approach, varying the balance between intuition and data situation-by-situation.
This raises more fundamental questions about product development. When is data more reliable than intuition? How does a data- or intuition-driven culture perform in the long term? What approach leads to the most “successful”, “delightful” products?
Notwithstanding any empirical data (yes, a proverbial sin in HBS’s “Digital Innovation & Transformation” class), it’s instructive to consider a few examples of data- and intuition- driven product development.
Let’s take Google to start.
So the story goes, Marissa Mayer famously tested over 40 shades of blue to maximize click-through-rates. This is but one example of the rigorously analytical culture Google is known for, which is hardly surprising given its engineering-centric ethos. It also seems to fit with their product (at least search) where small percentage increases in key metrics drive millions of dollars in return.
Apple on the other hand was guided by Steve Job’s mantra that “It’s really hard to design products by focus groups… a lot of times, people don’t know what they want until you show it to them.” This mantra was underscored by a perspective that Apple products are designed for what Apple employees wanted for themselves. This also fits with their products: ones where it seems hard for consumers to truly articulate their needs, wants and preferences.
However, both approaches have their downfalls. Google has been much maligned for having poorly designed, emotionless user experiences contrived from a “by the numbers” approach. Whereas Apple has missed opportunities as a result of its wall-garden approach and has struggled to introduce a meaningful innovation since the iPad’s launch in 2010.
Neither of these examples solves our early questions, though. That’s why I personally keep coming back to: “it depends”.
It depends what you’re trying to accomplish. It depends what types of products you’re building. It depends what fits with your existing culture.
Google highlighted an approach that can be wildly successful for incremental product improvements, especially for web-based products that can be rapidly tested in real-time. Apple highlighted an approach that can be wildly successful for transformative, new-to-the-world products, especially where hardware and industrial design are critical elements.
Really, the major takeaway is that a balanced “data vs. intuition” product development culture should be built within your startup, company or organization.
In many ways, the New Zealand yachting team in the 1995 America’s Cup struck the perfect balance. Their engineering team used complex data models to inform design decisions, but ultimately deferred to the “crew” on how yacht configurations “felt” in the water. If the crew didn’t like a configuration, the engineering team never even looked at the data.
With all of that, I think there are three additional takeaways:
Always start product design from genuine insight: whether from observation or feedback
Create a product feedback-loop that integrates data, intuition and clear metrics
Make the tough call: it’s what the product manager is paid to do!
… and always remember, you can’t A/B test your way to a great product!