There is an old saying that what you measure is what you get.
The problem is that what you’ve got isn’t always what you measure.
In other words, there might be things falling below the radar simply because you aren’t measuring them.
There is a simple example of this in the book Jurassic Park.
The island (Jurassic Park) had elaborate tracking systems to keep track of the dinosaurs.
The scientists believed that the genetically re-created dinosaurs on their island couldn’t reproduce. They programmed the systems to stop searching and counting as soon as they reached the expected number of dinosaurs.
At one point, an outsider suggests that perhaps the dinosaurs are reproducing. They adjust the systems. Sure enough, they discover more dinosaurs than they had when they started. Cue ominous music as mayhem ensues.
Of course, that’s fiction. Yet, the very same problems occur in real life often with similarly unexpected and damaging results.
The Jurassic Park scientists’ bias and assumptions tainted what and how they measured. Sound familiar? What you measure is what you get. That also means that what you measure is what you are looking to get.
In his book, Why Smart Executives Fail, Sidney Finkelstein illustrates this problem with regard to Rubbermaid.
For a long period of time after the Great Depression, Rubbermaid was known for excellence in product innovation. By 1993, the company won Fortune Magazine’s “Most Admired Company” distinction. Yet, Rubbermaid crashed shortly after. The industry had shifted. Product design and innovation gave way to cost, availability, and efficiency. But Rubbermaid continued to measure innovation. They got what they measured, but they were measuring (or at least focusing on) the wrong thing.
Advances in measurement have also fueled the problem. Techniques such as Lean and Six Sigma have given rise to a new appreciation of data and measurement. Changes in technology allow us to gather millions of data points daily. We can generate statistics to greater level of precision than ever imaginable. But all of this creates an illusion of understanding, certainty, and control. Six Sigma and Lean are excellent management tools. However, by themselves, they are no more effective at driving your business as a ledger or time and motion study. They are just tools.
The problem is that we now equate precision with accuracy. In some cases we are even lulled into the belief than greater precision yields greater accuracy.
As a result, things that can’t be measured precisely are often not measured at all.
This problem isn’t new. Paul Krugman, an economist and columnist for the New York Times, describes the problem in his paper, The Fall And Rise Of Development Economics.
Krugman recalled a story about the evolution of maps. In the 15th century maps were not always completely accurate on distances and specific locations. However, they were quite accurate on what actually existed across the African continent. Yet, by the 18th century this changed significantly. Africa’s coastline had become meticulously and accurately represented on the maps. However, most of the interior of the continent had literally disappeared. The map makers of the time wouldn’t include things on their maps that hadn’t met their standards for data collection and documentation. Since the interior hadn’t been explored as extensively as the coastline, the data weren’t as robust. While what was shown on the map was extremely precise, the map as a whole no longer accurately represented Africa.
They became blind to those things that they couldn’t measure. Instead of having a rough idea about the location of a river or town, now they saw nothing.
Does that make sense? Isn’t a rough idea of the location and existence of river better than no knowledge of the river? You’d think so. Yet, I see this “if it can’t be measured accurately, let’s not measure at all” thinking all the time.
Leaders often dismiss measures that are subjective, qualitative, or anecdotal. Yet sometimes those measures can provide the best information.
Your measures should be statistically reliable and valid. But they also must be balanced with reality.
A precise, reliable, and valid measure of the wrong thing cannot substitute for a rough measure of the right thing.
Spending five minutes talking with your people will give you a much better sense for their level of engagement than any employee engagement study. Asking your customers if they are happy will yield much better information than detailed metrics about call center response time, product quality, or profit margin. Of course, those latter metrics are important too for the specific questions they answer. However, they don’t replace simple understanding.
Don’t get blinded by your measures. Use them to drive decisions and actions. But remember, metrics, in and of themselves, don’t provide value. They must be combined with your understanding of your business and current situation.
Some decisions don’t require precision as much as they require a simple understanding of the situation. Being told that it’s cold outside provides enough data know whether to put on a coat. Simple measures can be helpful.
———————
Brad Kolar is an executive consultant, speaker, and author with Avail Advisors. Avail can help your team close the gap between data and action. Contact Brad at brad.kolar@availadvisors.com.
