In honor of Thanksgiving, here is a re-post of an earlier entry on the dangers of historic data.
————————-
The City of New York is considering suing BP for losses to its pension investments. (Reuters, June 24, 2010). They claim that BP misled them regarding safety procedures that led to the current oil spill. Perhaps that’s the case. Or, perhaps, as with any investment, they took a risk. At the time the investments were made, the data predicted a positive return from BP. But then an unexpected event occurred which changed the results. Now they want their money back.
Despite all of the advances that have been made in analytics, we still cannot predict the future. We can do a pretty good job of understanding what’s probable, but that’s not the same as predicting. It’s important to understand the difference and the implications of that difference.
Many executives fall into the trap of thinking that if they just had the “right” data, they’d be able to make the right decision. There’s no such thing as the “right” data. There will always be supporting and contradicting data for any decision. And, the best data we have is only from the past or present. All we can do is model that and make our best guess as to whether the future will look the same. And, make no mistake – regardless of how much data we have, when talking about the future we are making a guess.
Often the future doesn’t behave in quite the same way as the past. For example, a company’s stock price can vary greatly despite the company turning in similar business performance. A positive national economic report might drive all stocks up. In other quarters, there might be an oil spill. While understanding a company’s business performance data is a good start toward understanding what it stock price might do, there are no guarantees. There is a broader context in which that performance plays out
Sometimes we forget that data don’t occur in a vacuum – the context surrounding it matters. For example, one company performed an extensive ROI analysis on one of its internal departments. They used the results of that analysis, which were quite positive, to justify and drive a new operating strategy. However, the new strategy fundamentally sought to change the very operating model that had driven the ROI data. Predictive models are good, but they are based on past conditions. Changing context will also change results.
In The Black Swan, Nicholas Taleb provides a striking illustration of this:
“Consider a turkey that is fed every day. Every single feeding will firm up the bird’s belief that it is the general rule of life to be fed every day by friendly members of the human race ‘looking out for its best interests,’ as a politician would say. On the afternoon before the Wednesday of Thanksgiving, something unexpected will happen to the turkey. It will incur a revision of belief.” (p. 41)
Taleb uses the term “learning backward” in describing this line of thinking. He argues that the thousand days of historic data (of the Turkey being safely fed) actually provide a negative value.
“Consider that the feeling of safety reaches its maximum when the risk is the highest. But the problem is more general than that; it strikes at the nature of empirical knowledge itself. Something has worked in the past until – well, it unexpectedly no longer does, and what we’ve learned from the past turns out to be at best irrelevant or false, at worst misleading” (p. 42)
In this case a simple understanding of Thanksgiving is much more useful than the thousand days of data. While our world is a bit more complex than a turkey’s, we often fall into the same trap. I’d bet that the New York Pension Fund’s feeling of safety (for their investment) increased proportionally with BP’s exploration and drilling activities.
I’m not suggesting that we abandon all data and pull out the Ouija board to make decisions. Having solid facts and data is the foundation to good decision making. However, it’s just a foundation. We still need to use our brains, our experience, and yes, even our gut. When we combine those things with data, we are more likely to make a good decision.
At the same time, we also have to acknowledge the limits of data. Too many executives get stuck in the illusion that if they just find the right set of data, they will make perfect decisions. In the midst of that search, they wind up making no decisions or are surprised and unprepared when their decisions don’t turn out as expected.
If the data are accurate and reasonable (not just to you but to others who understand the context of what that data is describing) they are probably sufficient. It’s then time to make some judgment calls and move forward.
————————————
Brad Kolar is the President of Kolar Associates, a leadership consulting and workforce productivity consulting firm. He can be reached at brad.kolar@kolarassociates.com.
