There is an old saying that what you measure is what you get. While generally true, this statement can be a bit misleading. It doesn’t always work the other way around. What you’ve got isn’t always what you measure. In other words, there might be things falling below the radar simply because you aren’t measuring them.
There is a classic example of this in the book Jurassic Park. The island (Jurassic Park) had elaborate tracking systems to keep track of the dinosaurs. For efficiency purposes, the scientists set the systems to stop counting as soon as they reached the expected number of dinosaurs. The scientists believed that the genetically re-created dinosaurs on their island couldn’t reproduce. Therefore, they thought that they only needed to worry about decreases in number.
The story’s complication occurs when an outsider suggests that they increase the number at which the system stops counting. The scientists quickly find out that their assumptions about the dinosaurs’ reproductive abilities were wrong. The dinosaurs are multiplying.
Of course, that’s fiction. Yet, the very same problems occur in real life often with similarly unexpected and damaging results. The Jurassic Park scientists’ bias and assumptions tainted what and how they measured. Sound familiar? What you measure is what you get. That also means that what you measure is what you are looking to get. In his book, Why Smart Executives Fail, Sidney Finkelstein illustrates this problem with regard to Rubbermaid.
For a long period of time after the Great Depression, Rubbermaid was known for excellence in product innovation. By 1993, the company won Fortune Magazine’s “Most Admired Company” distinction. Yet, Rubbermaid crashed shortly after. Product design and innovation gave way to cost, availability, and efficiency. As the industry shifted, they did not. They got what they measured, but they were measuring (or at least focusing on) the wrong thing.
Advances in measurement have also fueled the problem. Techniques such as Lean and Six Sigma have given rise to a new appreciation of data and measurement. Changes in technology allow us to gather millions of data points daily and generate statistics to greater level of precision than ever imaginable. But all of this creates an illusion of understanding, certainty, and control. Six Sigma and Lean are excellent management tools. However, by themselves, they are no more effective at driving your business as a ledger or time and motion study. They are just tools.
The problem is that we now equate precision measurement with valid measurement. The result is that things that can’t be measured precisely are often not measured at all. This problem isn’t new. Paul Krugman, an economist and columnist for the New York Times, describes the problem in his paper, The Fall And Rise Of Development Economics.
Krugman explained that 15th century maps, while not always accurate on distances and specific locations, were quite accurate on what actually existed across the African continent. Yet, by the 18th century while Africa’s coastline had become meticulously and accurately represented, most of the interior of the continent had literally disappeared from the map. The problem was that map makers of the day wouldn’t include things on their maps that hadn’t met their standards for data collection and documentation. Since the interior hadn’t been explored as extensively as the coastline, the data weren’t as robust. While what was shown on the map was extremely accurate, the map as a whole no longer adequately represented Africa.
In other words, Europeans became blind to those things that they couldn’t measure accurately. So instead of having a rough idea about the location of a river or town, analytical rigor caused them to show nothing at all. Does that make sense? Isn’t a rough idea of the location and existence of river better than no knowledge of the river? You’d think so. Yet, I see this “if it can’t be measured accurately, let’s not measure at all” thinking all the time.
Leaders often dismiss measures that are subjective, qualitative, or anecdotal. Yet sometimes those measures can provide the best information. Ideally most of your measures should be statistically reliable and valid. However, a highly precise, reliable, and valid measure of the wrong thing cannot substitute for a rough measure of the right thing. Spending five minutes talking with your people will give you a much better sense for their level of engagement than any employee engagement study. Asking your customers if they are happy will yield much better information than detailed metrics about call center response time, product quality, or profit margin. Of course, those latter metrics are important too for the specific questions they answer. However, they don’t replace simple understanding.
Don’t get blinded by your measures. Use them to drive decisions and actions. But remember, metrics, in and of themselves, don’t provide value. They must be combined with your understanding of your business and current situation. Some decisions don’t require precision as much as they require understanding of the situation. Being told that it’s cold outside provides enough data know whether to put on a coat. Simple measures can be helpful.
———————
Brad Kolar is the President of Kolar Associates, a leadership consulting and workforce productivity consulting firm. He can be reached at brad.kolar@kolarassociates.com
