Measurements and indicators – You will only get what you ask for!

Measures and indicators are important tools for every manager. They tell us how we have performed in the past (lagging indicators) or what to expect in future (leading indicators). It is common sense that business-related measures and indicators help managers to make better decision. However, the value of these measures largely depends on a wise choice of what is actually measured. Besides that, businesses can thrive and flourish without lots of measures too.

“If you can’t measure it, you can’t manage it.”

This famous adage, which is attributed to Dr. W. Edwards Deming, has made its way into managers’ minds all across the world. It probably has contributed to the rising popularity of management dashboards of all kind. Consultants and IT-solutions providers eagerly explain their clients why they would need such a dashboard (and help them implement it). In some industries, like in financial services, regulators even ask for a growing variety of measures to be reported on a regular basis.

As with every popular management trend, there are some downsides, pitfalls, and misunderstandings. This article will discuss some of these downsides that businesses should avoid in order to make most of their measures and indicators.

The myth that what is not measurable is not manageable

On the W. Edwards Deming Institute Blog, John Hunter writes

“One of the quotes you will see quite frequently “incorrectly” attributed to Dr. Deming is “if you can’t measure it, you can’t manage it.” … His quote is saying the opposite of what most people think it means when they use the quote without the extra words he used.”

The correct wording of the Deming-quote is

“It is wrong to suppose that if you can’t measure it, you can’t manage it – a costly myth.”

Without doubt, Deming emphasized the importance of data and measurements. However, as John states in his blogpost, Deming also knew that

“There are many things that cannot be measured and still must be managed. And there are many things that cannot be measured and managers must still make decisions about.”

Conclusion:

The over-reliance on measurements and indicators is a popular misconception. The popular adage easily misleads managers in two ways

  • Decision makers might demand data to back up every decision they have to take. When a manager calls for data strongly enough, someone in controlling, marketing or some other department will deliver data. If these data are relevant, reliable and suitable is rarely asked. Everybody is happy to have some data backup for a decision.
  • Managers might compile huge dashboards and reports full of measures in order to be well equipped for their decisions. Thus, the regular inflow of measures and indicators tends to lead management attention to those issues that are expressed by these data. What’s not in the data easily slips out of focus.

Decisions need the right kind of measures

It is undeniably a wise decision to back up management tasks with suitable data. The emphasis here is not on “data” and not on the “amount” of data or on its “timeliness” and “accuracy”. It is on “suitable” and “relevant” instead.

The most comprehensive dashboard of measures won’t improve decision making, if the data in there are not related to the decision. It is imperative to choose the right kind of measures for any decision.

Ben Cohen lists three principles for designing the most effective key performance indicators here. They can equally well be applied to all other sorts of decisions-backing measures and indicators:

  1. Begin at the beginning: Effective KPIs must originate from a specific strategic objective or desired outcome for your business
  2. Measure what you want to know instead of what you can measure right now: Effective KPIs must be designed to thoughtfully answer the right questions for your business
  3. Create a bias towards action: Effective KPIs must enable decision-making

Besides that, Henry Mintzberg reminds us that measurable data may not always be the most relevant data when he writes about the problems of managing by efficiency:

“When we hear the word efficiency we zero in―subconsciously―on the most measurable criteria, like speed of service or consumption of energy. Efficiency means measurable efficiency. That’s not neutral at all, since it favors what can best be measured.”

Conclusion:

The purpose of the decision determines what kind of measures you need – not the other way around! It is the wrong approach to take the data you have and attribute it to upcoming decisions.

You also have to look at relevant factors that are not (easily) measurable.

The purpose of the decision determines what kind of measures you need – not the other way around! Click To Tweet

There are many poor measures and indicators

During my professional career, I have seen many poor information systems that hardly support sound management decisions. Here are some typical manifestations:

  • (Almost) no information at all: Entrepreneurs and small business owners often are very hands-on and rely on their specialist knowledge in their area of expertise. Many of them never had a formal management education and they aren’t interested in much data beyond revenues, costs and profits.
  • Information overload: This is more common in large organizations. Top managers request information systems to stay on track with the performance of their business. Somewhere deep down in finance or some other department, some specialists with limited access to top management try to compile all relevant data. They don’t really know what sorts of data managers need for what types of decisions. On the other hand, they are proud of their huge data pool and embrace the chance to present what they are able to do. In their best attempt not to leave any relevant data out, they put in as much data as will fit on a given sheet of paper. I have seen columns of figures in font size 8!
  • We take what we have: Some businesses compile measures and indicators just for the sake of having them. They can impress investors, analysts, regulators and others with their charts but don’t really use them for decision making. This attitude tempts those who have to retrieve the data to use whatever they have readily at hand. It just has to be somehow related to the requested purpose.
  • Wrongly defined indicators: The last company I worked for had defined a set of key risk indicators. From just reading the list, they all made sense. However, when I looked at the definitions of the indicators, one stood out to me. It did not make any sense at all. It was a ratio with a built-in volatility. The denominator was a measure that would change rapidly from time to time (without any business risk) followed by periods of stability.

I have listed more problems with the use of early indicators in this article.

Reasons for poor measures and indicators

The reasons for these shortfalls are manifold. In my experience, there are two major underlying problems that foster poorly designed measures and indicators.

  • Suitable and relevant data is often hard to gather
  • Top managers do not care enough about the suitability and relevance of their measures and indicators

The problem of data gathering

The problem

“We don’t have this kind of data”

is one of the most frequent excuses I heard in my professional life. Normally, this statement is followed by a lengthy explanation why it is absolutely impossible to retrieve these data and why nobody ever bothered to have it.

This is nobody’s fault. Businesses design their pools of internal and external data to their needs at a given point in time. New questions arise, needs change, and other types of data are required. It would be both – inefficient and ineffective (i.e. close to impossible) to design a data pool with everything in there that might be needed in future.

Of course, it can be costly and time-consuming to gather a particular new measure or indicator. The principles of efficiency apply here as well. The request for a particular piece of information must not initiate a huge IT project that delays more customer orientated projects. If it would be necessary to buy some external data, businesses must weigh up the costs and benefits too.

The solution

Obviously, it is not a good solution not to use any data at all or to use what is readily at hand.

Instead, businesses have to

“… ask the difficult questions about your business that you want and need to have answered.”

as Ben Cohen suggests in his article. Following that, businesses should go through these four steps:

  1. Determine what question is to be answered
  2. Determine which measures, indicators and figures will answer that question
    – ideally
    – alternatively
  3. Assess if these data are available or at which cost they could be obtained (in terms of time, effort, and expenses)
  4. Decide which of these data provide the relevant information most effectively and efficiently

This article provides more practical advice for the use of early indicators in businesses.

The problem of limited management involvement

The problem

In my opinion, the most important reason for poor measures and indicators is that top managers do not care enough about the suitability and relevance of their measures and indicators. Here are some typical shortcomings:

  • They fail to relate their data requests to particular business objectives. Instead, they just give vague reasons such as “We want to track why our profitability is too low”.
  • Thus, they start with compiling data in the hope that some management insight would reveal from it.
  • They don’t communicate their intentions clearly enough.
  • They don’t engage in the discussion about which kind of information is needed, which kind of data could be obtained and how these will match together.
  • They don’t question the data that is delivered to them, i.e. they don’t check if the data really answers their question.
  • They want to compare what is not comparable. It is tempting to compare all products / business units / customers with an identical set of measures. Nevertheless, their success may depend on very different drivers and mechanisms. When forced into a one-size-fits-all measure, results may become close to meaningless.

The solution

Simply put, the solution is to do the opposite of the above shortcomings.

It is the responsibility of top management to determine which questions must be answered. They have to decide which measures and indicators will best provide this information. It is equally important to clearly explain this purpose to those who have to obtain the data.

To get the best result, managers have to engage in an open discussion with their data specialists:

  • Which data can be obtained at which costs
  • Potential limitations in the data (accuracy, timeliness, relatedness …)

In short: Managers will only get what they ask for.

The quality of the measures and indicators they get depends on how good they explain their intention and how wisely they question the results.

Efficiency is important. However, it cannot be the only measurement.


Our book recommendations on measurements and indicators

Facebookgoogle_plusredditmail

Trackbacks