Introduction
The IT revolution made it possible; management trends made it inevitable: the KPI mania that drives
businesses and institutions to set targets for everything. A huge number of people are busy pulling,
analysing and communication numbers, all in order to make the right decisions and change behaviour
of those who are subjected to the targets. But does it work? If so, does it work too well?
Keep on measuring but watch out for targets
Beware of your true costs of measuring (and of not measuring)
I love numbers; actually, I am sucker for facts and fully concur with those who say
“if you can’t
measure, you can’t manage”.
There is a cost of measuring of course. Don’t underestimate the
resources you spend pulling, analysing and communicating what you measure. Try to add up in your
organisation (if you dare). You will also have hidden costs when management and employees
misunderstand what the measurements actually tell you, and when you have endless discussions about
unclear definitions. So, altogether there is a fine balance between costs and benefits of measuring.
Still, I argue that measuring (the right things) is mostly a benefit. Setting targets is a different story,
however.
Targets
do
change behaviour (and that might be your problem)
You will need targets, let me make that clear. I am much too old to believe that people always know
about the company’s best, nor are they always motivated to achieve the company’s best when this is
in contrast to personal preferences and motivators. Most managers would agree that setting targets
will impact employee behaviour (after all, that’s their whole purpose). What some managers do not
pay sufficient attention to, or plainly deny, is that employees will often engage in counterproductive
behaviour in cases, where the targets can (most easily) be met by doing so. The lack of attention to
subordinates’ counterproductive behaviour also goes for managers’ managers, despite my
(unsubstantiated) assertion that the more senior the manager, the more he or she might be prepared
to do in order to deliver a green scorecard.
In the final part of this article, I will give examples of KPIs that can (and will) lead to counterproductive
behaviour, and I will give some suggestions to what to do instead. Many of the examples relate to
micro-management
that leaves little room for judgement and common sense.
Let me pre-empt one example from tables 1-4: It is always wise to consider what causes or
drives
a
high level KPI, and to measure accordingly (where possible). But when you start to set targets, you
risk getting what you ask for, and not what you want. For a customer service department, it is a
natural (and wise) thing to target customer satisfaction, and you might have a good idea about what
drives this satisfaction. One such thing would be that customers want fast responses, when they have
a question or a problem. Everyone can agree to that, and your company has probably set a target for
the response time. The target will work fine as long as your customer service team has the skills and
manpower to meet it with a good effort. If, for some reason, your team does not have sufficient skills
or manpower, the right thing to do is to fix these problems first. But management wants to meet the
KPI, which impacts bonuses and organisational advancement. Employees will feel pressed, and they
will ensure to deliver fast,
low quality
responses that lead to failure demand (communication loops
leading to more work for everyone in the end), dissatisfied customers and stressed and demotivated
employees. My key point is that measuring makes sense, but you should be careful about setting
targets, particularly when the KPI is not high level.
My last note of caution is that KPIs compete for resources. This leads to wrong prioritisation due to a
lack of clarity of business impact and due to the (typically) binary nature of KPIs (either you’re in, or
When you chase the thin rabbit
Your organisation probably has a lot of KPIs. Most of your KPIs may even be reasonable and measure
important parameters. Almost certainly, your KPIs will be binary. This means that if you set a target of
80% of something, you meet the target in full by achieving 80%, but you get zero reward for achieving
79%. It is very unlikely that management or employees are aware of the business impact of this one
per cent in terms of customer satisfaction, savings or similar. What happens is that you may spend too
much time and money achieving a small performance improvement just to get on the right side of the
bar. Instead your efforts could be directed to further improvements to another KPI which is “home
safe”, or maybe even to something else. KPIs always compete for your attention, and sometimes this
attention is plain counterproductive. What should you do then? Well, there is lots of literature on how
to find critical success factors and how to define KPIs. What I want to emphasize is the need for
quantifying the business impact of different levels of performance for these KPIs to make sure that
a)
Different performance levels reflect their relative business impact
b)
Rewards (points, scores) are graduated according to performance
Your management will probably find this approach “too academic” or just too cumbersome, but hey –
we’re only talking about the priorities of your organisation for the next year or so (how can that be so
important?).
KPIs that lead to bad behaviour (and what to do instead)
Unless you are fresh out of school (or work in a small business), you can easily find an example of
something stupid you or your colleagues (or management) have done because of a KPI. Maybe you
even wonder about things you’re instructed to do, or why your SOPs look as they do. If you are a
business consultant, you have almost certainly worked with a process, where you just couldn’t find a
good reason for some of the activities. If you’re looking for an answer, try looking at the KPIs that
have been defined for the process. And if you don’t find a link, try looking at the KPIs that were in
place during the last decade; then you are very likely to find the answer. Stupid KPIs lead to stupid
behaviour.
It’s not that all KPIs are plain stupid. Most KPIs make some sense, and aim at an honest business
objective. But you have to be aware of both the constructive and the unconstructive ways of achieving
a KPI. Here are some examples.
Table 1: Examples of KPIs in Service and Administration
Typical KPI
Possible counterproductive behaviour (and what to do)
≥ X% of all requests must be answered within Y hours
This KPI could ensure speedy replies, if you have enough employees, and if they have the skills to do their job as required. However, if that is not the case, e.g. if you have had high employee turnover, then employees might start sending unfinished, low quality answers to their customers just to meet the KPI. This introduces failure demand to your system. Note also that setting a target of X% means less priority to requests that are difficult or which have already surpassed the Y-hour threshold. Better measurements would be “variation (or standard deviation) of leadtime”, supplemented by a leading measurement on “maximum number of outstanding requests”. And remember, you can measure without setting a target.
≥ X% of all requests must be fixed within Z hours
This KPI focuses on fixes, which prevents answers that just “buy time”. We have the same problem about lack of focus on cases already delayed or which are just very difficult. Better measurements would be “variation (or standard deviation) of leadtime”, supplemented by a leading measurement on “maximum number of outstanding requests”. Be careful about setting a target. You will create an incentive to keep a high flow of easy cases, which puts a stop to (any) ambitions of reducing such requests by error proofing/mitigation or making customers self-propelled, e.g. through better IT solutions.
≥ X% of all calls answered in Y seconds
Answering calls says nothing about call satisfaction, so teams may nurse this KPI by adding staff to pick up phones, but who can only redirect or take messages (and introduce failure demand). You may be able to implement an instant satisfaction measurement, alternatively think about how to ensure call quality.
Average call time < X minutes
Some customer service people like to chit-chat, so management wants to improve efficiency by setting a limit to the call time. If you do so, be sure that you can measure quality, for instance if a longer phone call saves long e-mail exchanges
Manual cases per FTE ≥ X
Setting targets for this measurement creates an incentive to keep a high flow of easy cases, which puts a stop to (any) ambitions of reducing such requests by error proofing/mitigation or making customers self-propelled, e.g. through better IT solutions. If part of your cases are automated, you could set a target for the “total number of cases per FTE”, which provides an incentive for automation.
Invoices posted automatically ≥ X%
This KPI is an example of a downstream measurement, which may lead to squeezing in unproductive process steps before the measuring point. In some cases, purchase orders may be adjusted retroactively to match the invoice. In other cases, employees make changes to the electronically captured invoice data before the measuring point. The auto-posting could instead be measured without a target, as long as the whole process (all process steps) is measured on costs or productivity.
Table 2: Examples of KPIs in Sales
Typical KPI
Possible counterproductive behaviour (and remedies)
Sales ≥ budget for product A, and sales ≥ budget for product B
Sometimes sales people get different targets for each product in their portfolio. Sales bonuses typically constitute a large share of the total salary, and you can be sure that the sales person directs all focus to the product below target, provided the target is met for the other product. This means that the company may miss out on (easier) additional profit for the product which has already exceeded the target. Unless driven by market entry objectives, you should measure the aggregate sales instead, or even better: the profit margin. Whichever way, do not set binary targets for the month or the year, as sales agents might postpone or “lump” sales in order to trigger the target. Instead, make sure that any targets are graduated and do not encourage unproductive “calendar creativity”.
Sales ≥ revenue budget Optimising revenue is not the same as optimising profits. Be careful about not giving the sales
people the incentive to talk down prices or focus on products with high revenue and low profit margins. Instead you should set targets on the total profits made. Whichever way, do not set binary targets for the month or the year, as sales agents might postpone or “lump” sales in order to trigger the target. Instead, make sure that any targets are graduated and do not encourage unproductive “calendar creativity”.
On top of that, I want to make a general alert to sales budgeting: Most large organisations have the unfortunate habit of mixing financial and operational planning. Many years ago I took part in sales budgeting for a container shipping company and prepared a conservative revenue budget due to equipment constraints the previous years. Equipment planners used the budget to position what they thought was the necessary amount of equipment, which in the end meant a huge opportunity loss of profits to the company.
Forecast accuracy ≥ X% Forecasting is not what keeps the typical sales agent awake at night. Management needs to
plan, and sometimes gives the sales people a KPI on forecasting accuracy. Depending on the weight of the target, be careful that you are not making a self-fulfilling prophecy, as sales could be slowed down deliberately if sales targets are achieved.
Table 3: Examples of KPIs in Logistics and Production
Typical KPI
Possible counterproductive behaviour (and remedies)
Produce ≥ the productionagreement (e.g. as weighted average for all products)
Production agreements are normally not negotiated more than once or twice per year. Producing more than the target may be unnecessary if the inventory is loaded. You need to pay attention to production capacity utilisation, but also to stock costs and scrap. Sometimes the target may even be exceeded by several per cent, because the produced (or released) volume happens to be in the counter or denominator of other KPIs.
A better idea would be to target “unit costs” and “inventory DOH interval” for each product. One of the least ingenious KPI-inventions I came across was a “weighted production agreement”. As you can guess, when production of one product fell short (and maybe even
Inventory ≤ X days To achieve this KPI, Logistics might order very small batches with many costly transactions, or maybe even compromise safety stocks. If you’re building up stock to scale down capacity or to prepare for a launch, an inventory KPI could compromise such objectives. Sales agencies or affiliates with inventory KPIs sometimes hold back orders (particularly close to calendar year end, or whenever the KPI is measured) with a resulting order boom in the subsequent period. Instead you should put a target for “days on hand” and let the target be an interval for each product, reflecting safety stock and most economic order size (with exceptions for stock building due to launches, expected seasonal peaks, or cuts in capacity).
≥ X% of batches released
within Y days Setting a target of X% means less priority to batches that are difficult or which have already surpassed the Y-hour threshold before release. Note also that variation is crucial to determine safety stock levels. This means that companies that increase variation (by prioritising the easy batches) in most circumstances need larger safety stocks, despite a reduction of the average leadtime. Better measurements would be “variation (or standard deviation) of leadtime”, supplemented by a leading KPI on “maximum number of batches in process”.
OEE2 or OEE3 ≥ X% Introducing OEE KPIs will first of all induce production managers to show great creativity
when registering time. OEE2 (and OEE3) targets may be achieved by speeding up batch change overs, in some circumstances at higher costs than benefits. Overall production speed (and speed of change overs) can also impact scrap, maybe with a higher number of released units per hour (i.e. higher OEE2 and OEE3), but at excessive costs. Equipment already purchased is a sunk cost, but you may postpone buying new equipment by improving the equipment utilisation. What you should do in that case is to make a unit cost KPI and include the WACC of the invested production equipment.
Raw material deviation to
budget ≤ X Raw material deviations are calculated based on the bill of material. If you improve your yield in one year compared to the previous, the deviation will be negative (a saving compared to budget). This saving may be part of the unit cost calculation or the “operating profit” (which is sometimes calculated as capacity costs + raw material deviations). If, in the following year, you maintain the improvement without improving further, your “deviation” will be zero, and your (incorrectly calculated) unit costs or operating profit will increase. Managers may then wish to hold back on improving yield, or at least do so in very small steps. Further, this is a classic example of mixing financial and operational tools. Bills of material can be manipulated to show a high baseline in order to meet your raw material deviation target. This will in turn mess with your materials planning. You should skip this KPI altogether, and measure unit costs including the full raw material costs.
Production (or releases) per
FTE ≥ X Depending on how your investment targets are put together, this KPI could lead to investing (you embark on projects with negative NPV) in cases where the saving of FTE cannot over-offset the investment (IT) costs. The KPI could still be good to keep, but only if you have intelligent investment incentives.
Table 4: Examples of KPIs in Finance
Typical KPI
Possible counterproductive behaviour (and remedies)
Return on investment or
return on assets ≥ X A high current performance can lead to a high target being set. This means that you could miss out on good investments at a reasonable ROI, which is nevertheless below the target. In fact, raising the target could mean that you should skip currently planned, good investments which would have increased shareholder value. Set targets for your individual projects, but refrain from doing so overall.
Net working capital ≤ X One way of meeting this KPI is to reduce inventories (possibly at much higher order costs due
to more transactions and over-capacity to absorb process or demand variation). Other ways could be to extend payment terms indefinitely (days payables outstanding), which at the end of the day will end up in the price you pay for purchased goods and services. In fact, if your suppliers finance debt at a higher interest rate than you do, you will miss out on opportunities to get price discounts for fast cash. There are other ways of ensuring short term liquidity, but they don’t figure in the net working capital measure. Skip it, and make up a liquidity measure that includes all assets that can be readily converted to cash.
Investments / Investment
budget ≥ X% ≤ 100% Generally, investments should be undertaken if they are expected to yield a (sufficiently) high return. Keeping investment budgets at levels below business units will sometimes lead to skipping or postponing projects that would have led to more shareholder value. On the other hand, like magic, organisations spend very close to the budget if they have the time and resources to undertake these investments. That is because managers know that an unused budget in the current year is likely to lead to a smaller budget the following year. In addition,
with the argument of “cash flow management” controllers may introduce a KPI on minimum
investment spend, which will tempt managers to throw good money after bad money. Instead, “small stuff” could be kept as part of the capacity budget, and there should be no budget below business unit level for medium and large investments. In that case, investment spend could be approved by a “ROI committee” (which doesn’t hold a “must spend” budget either…). Controllers are better advised to manage cash flows by ensuring knowledge of specific major investments, and estimating the residual by means of statistics and best guesses.
Spend to budget ≤% In large corporations (as in public institutions) a lot of management resources are spent on
creative budget management. If you have many separate budgets, you may “mask” expenses to put them in another basket. Managers will also want to spend as close to the target as possible in order to keep the same budget next year. And if they have money in reserve, they may try to prepone some of next year’s expenses. If you peruse your company’s spend profile, you will probably find an unusually large spend of indirect materials around the end of the fiscal year. Moreover, if you mix up long term planning and daily running costs, managers may prioritize short term rewards at the expense of long term profitability. I cannot think of perfect ways of avoiding these kinds of misconduct, however. Some organisations have skipped internal budgets altogether, but I have not seen empirics of the long term financial results.
Final remarks
KPIs are risky business. Organisations are prepared to spend huge resources measuring, tracking,
analysing, interpreting, discussing, benchmarking etc. But they are not always prepared to spend time
on a good process to weed out most of the incentives that lead to counterproductive behaviour. Are
you?
If you have any feedback to this article, positive or negative, or if you have good supplementary
examples then please share on
http://carstenhansenblog.wordpress.com/