Performance measurements

Measuring performance is an integral part of doing business. For profit-making organizations the ultimate performance metric is whether the business is returning a profit; without profit the business cannot continue in the long run. But measuring profit alone is not sufficient; there are several other considerations which should be taken into account in the long run. Perhaps the organization could do better, perhaps the business is too focused on present profit at the cost of future gain and/or resources are being misused.

Measurements can be used to induce better performance; the challenge is to find the right metrics, implement these and to develop the best means for using the results. Measuring the wrong parameters and using the results in an incentive system can have disastrous effects as has been learned from the last economic crisis. In this context it is appropriate to quote William Bruce Cameron : “not everything that counts can be counted, and not everything that can be counted counts” (Cameron, 1963).

The types of measures that are commonly used in business today can be divided into three main categories (Anupindi, Chopra, Deshmukh, Van Mieghem, & Zemel, 2006):

1. Financial measures. These are the traditional accounting measurements which organizations use for the evaluation of their financial performance. For most organizations these measures can be found in annual reports and describe absolute performance (revenues, costs, net income, profit), performance relative to asset utilization (accounting ratios like Return on Assets and Return on Investment) and financial strength (cash flow).

2. External measures. These measures focus on the market (how the organization is performing compared to competitors) and the customers (how to attract and retain

customers). Performance metrics in this category could be for example market share, customer satisfaction and customer loyalty.

3. Internal measures. These are intended for measuring the operational performance, how well the systems of the organization are performing. The operational metrics need to be linked to the financial and external metrics. These metrics can be very different from one organization to another. An example of internal performance metrics could be number of stock outs per year, which influence both the financial metrics (lost sale) and the external metrics (customer dissatisfaction). System performance metrics can in certain cases be defined in order to compare performance between similar systems as is done in benchmarking.

Performance measurements are often linked to the strategy of organizations. A well-known framework for performance measurements is the Balanced Scorecards (Kaplan & Norton, 1992) which suggests that organizations should translate their strategic goals into performance measurements and the measurements should be balanced. Following a research Kaplan and Norton found that using only financial performance metrics was not enough; these metrics should be balanced with other metrics which are drivers of future financial performance. The metrics which are drivers of future gain are in their opinion operational metrics of customer satisfaction, internal processes and the organization’s innovation and improvement activities. In order to obtain balanced measurements Kaplan and Norton suggest that organizations use the following questions as guidance:

 How do we look to our shareholders (financial perspective)?

 What must we excel at (internal business perspective)?

 How do our customers see us (the customer perspective)?

 How can we continue to improve and create value (innovation and learning perspective)?

The Balanced Scorecard has been widely used but implementing a performance system of this type is not always straightforward and should be considered carefully. If it is to be successful several factors have to be considered. It is important that the metrics are carefully chosen, the implementation planned, the use of the measurements considered, etc. The main learning point from the Balanced Scorecard is that that you should not use financial metrics alone and that they should be balanced against other factors which are important to the success of the organization.

There are several studies and articles which give recommendation as to the design of performance measurements and performance frameworks. Neely, et. al. have in a literature review (Neely, Richards, Mills, Platts, & Bourne, 1997) gathered together some of the recommendations which have been given for the design of performance measurements, the recommendations are listed in table 3.1.

Table 3.1: A list of recommendation for the design of performance measurements

Furthermore Neely et. al. suggests that for each metric a data sheet is prepared containing the following information:

 Title (a title which explains what is measured)

 Purpose (the rationale behind the measure)

 Relates to (the business objectives which the measure relates to)

 Target (the goal for the measure and the time limit for achieving it)

 Formula (the data elements and how the metric is calculated)

 Frequency of measurement (how often the measure is performed and reported)

 Frequency of review (how often the metric is re-evaluated)

 Who measures? (the person who collects the data and reports the results)

 Source of data (specification of the raw data for each data element)

 Who owns the measure? (the person accountable for performance improvements)

 What do they do? (a description of how the measure will help to improve performance)

- be derived from strategy - be simple to understand

- provide timely and accurate feedback

- be based on quantities that can be influenced, or controlled, by the user alone or in co- operation with others

- reflect the “business process” – i.e. both the supplier and customer should be involved in the definition of the measure

- relate to specific goals (targets) - be relevant

- be part of a closed management loop - be clearly defined

- have visual impact - focus on improvement

- be consistent (in that they maintain their significance as time goes by) - provide fast feedback

- have an explicit purpose

- be based on an explicitly defined formula and source of data - employ ratios rather than absolute numbers

- use data which are automatically collected as part of a process whenever possible - be reported in a simple consistent format

- be based on trends rather than snapshots - provide information

- be precise – be exact about what is being measured - be objective – not based on opinion

Recommendations for the design of performance measurements (Neely, Richards, Mills, Platts, & Bourne, 1997).

 Who acts on the data? (the person/team who reacts to a positive or negative results)

 What do they do? (define how to react to positive or negative results of the measure)

 Notes and comments (any additional information)

Several additional suggestions have been made since this article was published. Phelps prepared in his book (Phelps, 2004) a metrics audit where the metrics are evaluated against four headings. Performance metrics should:

 “clarify strategy”

 “capture real performance drivers”

 “promote joined-up management” (ensure that metrics do not conflict and do not lead to managers playing the system)

 “be useful for performance management” (used for reward and appraisal systems) Under these headings Phelps has prepared 18 statements which the metrics should be measured against in order to evaluate their effectiveness.

Phelps emphasizes the need to distinguish between metrics which are drivers on one hand and outputs on the other hand and similarly between metrics which measure present value and metrics that measure future value.

Although it is very valuable information to know the profit of last year or last month, that information alone does not help organizations to increase its future profits. It is important that performance metrics provide information on the current status. However metrics which help the organization increase their performance next month and in the future should also be included. Managers should therefore ask themselves what factors will be driving future performance.

In order to address this issue, organizations need to find out what are the drivers behind the performance they wish to achieve. One way to do this is to use data mining, i.e. to line up several metrics which managers suspect will influence a certain output metric and to use regression or other data mining models to find which metrics are most significant. An example of this method can be found in Smart Business Metrics (Phelps, 2004, page 49-52). While the above recommendations on how to design performance metrics were probably made considering for-profit organizations, operating in a competitive market, some of them will also apply to companies in a market where there is regulated monopoly.

There are however several factors which differ for these different types of working environments. Looking for example at the Air Navigation Service Providers (ANSPs), such as Isavia, the customers are the airlines which in most cases cannot choose between different service providers. The airlines can only on rare occasions bypass certain air traffic control areas in order to avoid high charges.

It is however not like the ANSPs can exploit their monopolistic position as there are several stakeholders which affect their operation. For Isavia the stakeholders are: Airlines, Aircraft owners, Airline passengers, The Icelandic Civil Aviation Administration, International Civil

Aviation Organization (ICAO), the Icelandic government, Eurocontrol and other service providers, especially those adjacent to the Reykjavík Control Area. On the other hand the ANSP’s have obligations to service all aircraft in their control area and cannot choose to service only part of their customers or part of the control area. There are also several other obligations that are defined by laws and regulations which the ANSPs must adhere to and can be costly for smaller units to satisfy.