Data-driven decision-making: when using instinct and intuition makes more sense.
"The Answer to the Ultimate Question of Life, the Universe, and Everything is 42."
-Deep Thought
Not everyone believes in the power of 42.
Data-driven to a fault
For many, data is the (only) answer. We capture, process, and analyze data about everything: our products, users, customers, competitors, markets, trends, and, well, anything you can think of. The data’s volume is daunting. It comes flying off the screen at supersonic speed, and I can barely keep up.
It all becomes a bit much. After a while, it’s like trying to make sense of a 42-car pileup on a foggy highway.
Some of you might be saying, “Dude, this sounds like a ‘You’ problem,” and you wouldn’t be entirely wrong. Focusing on the vital data would help a lot. The trick is figuring out what is vital.
Having too much data works the same as having no data at all. There’s a limit to how much we can process and how deeply we can analyze. Over the years, I’ve learned that my ability to pay attention to data diminishes as the volume of data increases. That conundrum looks a bit like this.
Large datasets create cognitive complexity. It taxes our brains to parse, synthesize, and create intelligence from large datasets. Data that are unstructured or contradictory exacerbate the dissonance.
A couple of things happen when we deal with cognitive complexity. One, we fall back onto our biases, whether conscious or not. Two, it’s hard to fully digest and interpret large datasets, so we don’t consider the data.
What happens? We make poor decisions based on that data. Or, worse, we defer decisions, promising to dive into the data when we have more time. We never have more time.
We don’t understand whether the data represents something good or bad. Even when data are consistent and reliable, we mistake what that data tells us about risks and rewards. (There’s a fantastic TED Talk about risk illiteracy in the face of data here.)
I am not alone in this problem. There’s a lot of research about how large data volumes affect decision-making. Too much data is as challenging for board-level strategic decision-making as tactical day-to-day decisions.
When data is useful
So, when is data good? Data is best when it helps you answer concrete questions. Let’s say you’ve built and released a new feature. Are customers using it? If so, how? If not, why not? Is the feature poorly designed, poorly implemented, or just useless? Is it incomplete, missing key capabilities, and suffering from ‘MVP syndrome’? How does your data help you figure out the real reason?
Don’t neglect adding and enhancing your product’s analytics. Today’s tools make it stupidly easy to incorporate excellent usage analytics in your product that answer your most important questions. How long is your user’s time-to-value? When you added feature x, did it increase or decrease conversion? Which capabilities correlate to expanded usage? Which capabilities increase retention?
Early in my career, I rarely had answers to these questions. I emphasized feature development over non-functional infrastructure like usage analytics. At times, success correlated more closely to luck than features. I had no idea whether the revenue received justified the cost of developing something.
I had no hope of knowing how close I was to a profitable business.
The case for instinct and intuition
“How’s that working out for you? Being clever.”
-Tyler Durden
I used to say I am adept at making BAD decisions. “Ha ha,” I’d laugh, “I mean using the Best Available Data!” I hoped my cleverness excused what was, at the time, an overreliance on instinct and intuition to make critical, strategic decisions. Experienced product leaders, I knew, sometimes rely on instinct and intuition to make critical product decisions. You know a good idea when you see one, and your experience tells you the idea is achievable.
You’ll use instinct to make most decisions in early-stage companies when outcomes are uncertain, and goals are aspirational. The data you’ll benefit from later isn’t available to you. You can’t measure how well you are growing when you’re unsure if there’s even a market for your product. You are testing hypotheses.
It’s also true for later-stage companies when expanding a product line or entering a new market. Early in your product’s lifecycle, distinguishing whether features are customer-focused or market-driven is often tricky. Has your new product achieved product-market fit? When you examine your backlog, you see features clustered into themes, and delivering those features might predict success, but it’s hard to know.
Under these circumstances, you must trust your instincts and go with your gut.
When you need both
Great products result from good data and reliable instincts. Data tells you when you are on track; your instincts help you take risks.
Put yourself in a position to take those risks.