Performance measurement systems (PMS) design

Performance measurement systems (PMS) are tools to assess performance, to demonstrate accountability through performance reporting, and to improve the performance of an organisation (Goh, 2012). Current practice in measuring performance indicates that, regardless of the wide adoption of PMS by public organisations, its effectiveness in enhancing performance and accountability attracts criticism (Ammons et al., 2013; Berman & Wang, 2000; Sanger, 2008). Public organisations face considerable challenges to design effective performance measurement, ranging from the selection of measures, to their integration into key management practice, including strategic planning and budgeting (Berman & Wang, 2000). The following paragraphs summarise three aspects of performance measurement that contribute to improving performance and accountability: types of indicators, performance targets, and integration with key management practice.

Type of indicators

The quality of measures or indicators significantly influences the use of performance information (Ammons & Rivenbark, 2008; Van Dooren, 2005). Non-financial performance information contains outputs, workloads, outcomes, efficiency, and service quality measures (Hatry, 2006; Julnes & Holzer, 2001). Evidence from a North Carolina benchmarking project on the use of performance information suggests that good measures of efficiency and effectiveness are more likely to inspire managerial thinking about service improvement than output measures (Ammons & Rivenbark, 2008). Project coordinators of the participating cities declared that organisations that focused their attention on efficiency and effectiveness and did not mention workload measures, experienced service

improvement due to the application of performance measurement. In contrast, those that focused on workload measures reaped less benefit from performance measurement. Hence, selecting outcomes, rather than workload measures or outputs is likely to increase the effectiveness of performance measurement.

Apart from the importance of selecting outcomes, rather than outputs, performance information should also be valid, reliable and legitimate (Bouckaert, 1993a). Defining valid measures in public organisations is challenging compared to that of private organisations. The intangible forms of product or service provided by public organisations are difficult to measure, especially when the service relates to people, such as education, care for children, and assisting disadvantaged people. To ensure validity, performance measures have to be checked, improved and adapted to the different situations in which they are applied. Public organisations are challenged with developing valid data, specifically with regard to outcome measures (Bouckaert, 1993b; Moynihan & Landuyt, 2009). The need to provide relevant and useful information is not supported by existing systems to collect the information (Poister et al., 2014). As a result, managers rely on available data, regardless of its validity and legitimacy in preparing performance information. Bouckaert (1993b) ascertained that it is necessary to set up a new procedure for data collection and new data techniques of data processing to produced desired data, rather than to rely on available data. The lack of validity, reliability, timeliness, and prescriptive information may not generate useful information (Poister, Edwards, Pasha, & Edwards, 2013).

Aside from validity, another strategy to encourage the use of performance information is increasing the legitimacy of measures. Involving stakeholders, as well as all management levels of an organisation, in developing measures can encourage managers to make better use of their performance information. Bouckaert (1993a) maintained that the development of a performance measurement systems should adopt both a top-down and bottom-up approach, rather than just top- down. Incorporating a bottom-up approach requires all levels (bottom and middle management) to be involved in the system development. Developing performance measures using both top-down and bottom-up systems motivates and encourages managers to commit to using the measures in their decision-making processes. In addition, considering external stakeholders in developing measures also matters. Ammons (1995) underscored the importance of developing measures that can capture the interest of local media and the public. The experience of a few Iowa (US) cities and a survey of Iowa local officials suggested that citizen involvement has impacted the practice of performance measurement and increased the likelihood of elected officials integrating performance information in the decision-making and planning process. Involving citizens in defining measures opened up new opportunities to build a stronger partnership between the government and the citizenry (Ho, 2007).

Target setting

To enable assessment, targets should be set along with KPIs (ANAO, 1996; Kaplan & Norton, 2007; Mendes, Nunes, & Teixeira, 2014). Targets express quantifiable performance levels or changes of level to be attained at a future date, as opposed to the minimum level of performance (ANAO, 1996). The formulation of targets provides a basis for public managers to measure the achievement of their strategic objectives and goals. In turn, measuring the achievement of the targets allows managers to learn and identify factors that influence success and failure. Poister, Edwards, et al. (2013) asserted that the presence of targets enables public managers to make better decisions and manage programs and agencies more effectively. The presence of targets helps: to keep the focus of managers on performance; to motivate managers to achieve goals; provide insights regarding performance problems and potential solutions; and helps to hold lower level managers and organisational units accountable for maintaining or improving performance. Using data from 147 English local education authorities between 1998 and 2003, Boyne and Chen (2006) found that performance improvement was influenced positively by the presence of targets. Local authorities with a target on an indicator of educational performance achieved higher scores for exam results than those without a target.

However, despite its usefulness, target setting can also have negative effects on service delivery (Bevan & Hood, 2006b; Boyne & Chen, 2006; Radnor, 2008). Fryer, Antony, and Ogden (2009) classified these effects into the following: setting undemanding targets and working to only just achieve them; performance clustering around the target either through deliberately underperforming or manipulating the data; concentrating on meeting targets at the expense of other (unmeasured) factors; and choosing indicators and targets to influence the results/manipulating the measures. Bevan and Hood (2006a) consider these negative effects as gaming, which is defined as reactive subversion such as hitting the target and missing the point or reducing performance where targets do not apply. Bird, Coz, Farewell, CB, and Smith (2005) also warned that managers should set rational targets, since unrealistic standards may demoralize.

To enable improved performance and accountability, it is essential to be cautious in setting targets to minimise gaming. This motivates managers to achieve individual targets at the expense of overall performance (ANAO, 1996; Boyne, Gould-Williams, Law, & Walker, 2004). The ANAO (1996) pointed out that complementing targets with the quality of service performance is useful for discouraging ‘wrong’ or ‘inappropriate’ outcomes. Poister et al. (2014) argued that the intention to set targets should be ambitious and realistic in order to motivate managers and employees to improve performance. If performance targets are to be set, they need to have a sound basis and to take account of prior (and emerging) knowledge about essential variation (Bird et al., 2005).

Integration of key management activities

In order to gain benefits, performance information should be incorporated into key management activities, such as strategic planning, performance contracts, departmental or individual work plans, performance targets, performance audit, program evaluation, service improvement strategies, cost- benefit analysis, or budget proposals (Ammons & Rivenbark, 2008). A survey of US Midwest city mayors suggested that the impact of performance information on decision-making depends on whether the information is integrated into strategic planning and goal setting (Ho, 2005). Conversely, the failure to integrate the measures with strategic planning, budgeting, and reporting will undermine the intended outcomes of the approach (Marr, 2012). An online survey of agency managers in the public transit service industry in the US, conducted by Poister, Edwards, et al. (2013), showed that designing strategic planning, as part of performance measurement systems, has a positive effect on some dimensions of performance. Using data from a national survey of city and county administrators and budgeters, Melkers and Willoughby (2005) found that active integration of measures throughout the budget process is important in determining real budget and communication effects on local governments. Determining targets, which are incorporated with strategic planning as well as budgeting, will also motivate managers to use performance information. 2.3.3 Resource availability

The availability of resources, money, and staff, has an impact on the effective use of performance information (Ammons, 1995; Berman & Wang, 2000; Julnes & Holzer, 2001; Kim & Hoontis, 2012). The availability of funds enables public organisations to hire experts and other staff to analyse, interpret, and use performance information (Kim & Hoontis, 2012). By way of contrast, Ammons (1996) argued that the unavailability of resources prevented department officials from being involved in performance measurements. A case study of two New Jersey (US) municipalities conducted by Kim and Hoontis (2012) found that the allocation of resources was one of the antecedents for successful implementation of performance measurement that contribute to improving performance and enhancing accountability. Similarly, Berman and Wang (2000) highlighted that resources are important for increasing the capacity to develop performance goals and measures and to distinguish outcomes from outputs. The authors stressed that high capacity is a prerequisite for widespread use of performance measurement in the US counties.