Working Papers Management Sciences Gr 08-02
Behavioral factors influencing performance management systems’ use
Taco Elzinga
Shell International Exploration and Production Kesslerpark 1, 2288 GS Rijswijk, The Netherlands
Tel. +31 70 4472312; E-mail :[email protected]
Bé Albronda (corresponding author) Faculty of Management Sciences (MW) Open University of the Netherlands (OUNL) P.O. Box 2960, 6401 DL Heerlen, The Netherlands
Tel: +31 45 5762587; E-mail: [email protected]
Frits Kluijtmans
Faculty of Management Sciences (MW) Open University of the Netherlands (OUNL) P.O. Box 2960, 6401 DL Heerlen, The Netherlands
Tel: +31 45 5762387; E-mail: [email protected]
Abstract
The purpose of this paper is to substantiate conclusions from De Waal’s (2002) case study research about the role of behavioral factors in the use of performance management systems. De Waal’s exploratory research is replicated in four organizations. Data were collected through questionnaires, interviews and document research. Data of the additional case studies were combined with De Waal’s. Pattern matching was used to compare all case studies. De Waal’s initial research design was extended through the inclusion of a number of sensitivity analyses in the pattern matching exercise. The behavioral factors in the use of performance management systems De Waal deemed significant appear rather arbitrary, due to methodological choices made during the pattern matching and data analyses. Sensitivity analyses shows that behavioral factors can be ranked according to their relative importance. Although the addition of sensitivity analyses has shed some light on the relative importance of behavioral factors, there still remain unanswered questions with regard to the greater validity and reliability of De Waal’s original research design. Some assumptions made have to be studied in more detail to strenghten findings of the research.
Behavioral factors influencing performance management systems’ use
Introduction
In recent years, it has become clear that behavioral factors play an important role in the successful use of a performance management system (PMS) in organizations. Nevertheless, little research on the subject has been done to date. De Waal (2002, 2003, 2004) is one of few authors who has studied the subject in detail.
De Waal based his exploratory research, Quest for balance, on four case studies in three
Dutch companies. He identified 18 behavioral factors that seem to be important for the regular use of a PMS by the management of a company. To substantiate his conclusions, additional studies are required. Our research, in which we replicate De Waal’s study in four more organizations, is a first step to contribute to that goal.
The results from our additional case studies were not particularly consistent, neither between these studies themselves, nor (even more importantly) with the original findings of De Waal. This poses the question how these rather confusing results can be explained and what impact the research set-up could have had on the results.
In section 4 we compare the results of our case studies, and integrate these with De Waal’s results. Finally, in section 5, we try to give some explanations for our puzzling results.
De Waal’s research emphasizes the important, but underexposed role of behavioral factors for the effective use of performance management systems. Our aim was to extend his study and to substantiate its conclusions. However, our results compel us to reconsider the
methodology in use and to propose an alternative route to obtain better (read more valid and reliable) results.
Literature review
In recent years, an increasing number of companies have implemented performance management systems (PMS) that are based on critical success factors (CSF) and key performance indicators (KPI). A frequently used framework in this context is the balanced scorecard (BSC). Surveys among corporate executives conducted by the consulting firm Bain and Company revealed that by 2004, 64 percent of their respondents in North America and 57 percent of the respondents worldwide were using BSCs in their companies (Rigby and Bilodeau, 2005).
According to Kaplan and Norton (1992, 1993, 1996) senior executives should not rely on one set of measures to the exclusion of others, but they want a balanced set of financial and operational measures. They therefore proposed to base the BSC on four perspectives (financial, customer, internal business, and innovation and learning perspective) and defined a number of goals and measures for each perspective.
The above is supported by Lingle and Schiemann (1996), who describe the results of a study that confirms that companies who balance financial and non-financial measurements, but also link strategic measures to operational ones, update their strategic scorecard regularly and clearly communicate measures and progress to all employees, are better performers.
Since Kaplan and Norton introduced their concept of the BSC as a framework for translating a company’s strategic objectives and CSFs to financial and non-financial performance measures (KPIs), there have been many articles on this subject. Initially
research focused on how to design a BSC. In the late 1990s the attention shifted to the
implementation process, and since around 2000 the literature also addresses the use of PMSs, and to what extent organizations actually take advantage of their PMS and use the performance data to initiate management actions. However, despite the increase in experience with these systems, there is still a lot to be learned about the factors that influence their effective use.
The literature generally agrees that a good PMS consists of a balanced set of factors that are critical for the success of a company, and a limited number of indicators that are a measure for organizational performance. But what are the characteristics of a good KPI and how
should this measure of performance be defined? According to Neely et al. (1995) it has
A number of authors (e.g. Globerson, 1985; Maskell, 1989; Bauer, 2004) have provided guidelines to select the preferred set of performance criteria, but the development of PMSs
doesn’t stop with the selection and definition of measures. Bourne et al. (2000) emphasize
that the development process is not a simple linear progression from system design to the use of performance measures. Instead, the PMS requires developing and reviewing at a number of different levels as the situation changes. According to them a PMS should include a process or mechanism for 1) reviewing and revising targets and standards, 2) developing individual measures as performance and circumstances change, 3) periodically reviewing and revising the complete set of measures in use, and 4) challenging the strategic
assumptions. In their discussion and conclusions, Bourne et al. state that the task of
implementing and using a PMS is far from complete at the end of the design stage and that
specifically the development of the use of the measures is a problem which has hardly been
researched, and for which few tools and techniques are currently available.
In a review of the literature on performance measurement in manufacturing organizations
(operations), Lohman et al. (2004) concluded that most papers appear to deal with a “green
field” situation, in which PMSs are designed more or less in isolation, independent from
existing systems in the organization. Olsen et al. (2007) point to this as a normative
direction, focusing largely on how an integrated PMS should be designed, as opposed to
addressing the more realistic problem of how to design, implement and improve a PMS in
its organizational context. Both Lohman et al. and Olsen et al. propose a change in the
Analoguously, Franco and Bourne (2005) state that a PMS cannot be fully understood using a narrow analysis perspective that only focuses on the single process of designing and/or implementing the system. Following Pettigrew’s framework (Pettigrew, 1985), they propose a “contextualist approach”, in which the development of a PMS involves the identification of the contextual factors that influence or are being influenced by the system. Pettigrew’s framework shows the factors that should be taken into account during a change management process, and contains three basic components: context, process (here PSM design, implementation and use), and outcome.
Based on a literature review and interviews with a number of key professionals, Franco and
Bourne (2003, 2005) have investigated which critical factors play a role in enabling
organizations to effectively use their PMSs. They hereby focus their attention on what is called the “knowing-doing gap” (i.e. organizations struggle to transform performance information into effective improvement actions) and try to understand what differentiates companies that are able to “manage through measures” (i.e. using their performance management system effectively) from those that are not. In the end, they investigated 11 process factors and five context factors that facilitate a more effective use of a PMS (see table I). Based on the literature, among these 16 factors, the implementation ones seem to be crucial.
Table I. Critical factors (Franco and Bourne, 2005)
_________________________________________________________________________
Categories Factors Sub-factors
_________________________________________________________________________
Process Design PMS framework (like BSC) and strategy map
Measures and targets
Alignment and integration
Information infrastructure
Implementation Top management agreement and commitment
The three E’s: empower, enable and encourage
Communication
Use Review and update measures
Data analysis, interpretation, decion-making and action-taking
Rewards
Performance measurement helping tools and
management processes
Context Internal Firm strategy
Culture
Organizational structure and size
External Industry Environment
_________________________________________________________________________
What can be said about the benefits to the company after it has implemented a PMS?
Lawson et al. (2003) have investigated 150 organizations of which two-thirds agreed that
significant benefits had been realized from using a scorecard system. Feedback from these two-thirds indicates that implementing a scorecard system improves communicating the company strategy to the employees, that employees see the benefits of using a scorecard, that employees have become more aware of business plan goals and that the system has helped to align operational improvements with the overall strategy of the organization. They emphasize the need for a formal tie between scorecard and strategy and recommend
to implement this by assigning accountability for actions and placing appropriate measures on the scorecards of responsible people or groups of people.
Lawson’s research seems to confirm the outcomes of earlier studies on PMS benefits (e.g. Lingle and Schiemann, 1996; Hoque and James, 2000; Davis and Albright, 2004), in that companies perform better if they are managed using formalised, balanced and integrated
performance measures (Bititci et al., 2006). On the other hand, research conducted by Ittner
et al. (2003), Neely et al. (2004) and others, suggests that the use of PMSs does not make any difference to business performance. Still other authors (e.g. Braam and Nijssen, 2004) suggest that the impact of performance management is contingent upon the way it is used.
Despite the fact that research outcomes seem to point in different directions, one observation turns out to be rather common: providing information on performance is not sufficient to improve business performance results (Nudurupati and Bititci, 2005). The real
success of a PMS lies in how people use this performance information (e.g. Eccles, 1991;
Davenport, 1997; Prahalad and Krishnan, 2002; Nudurupati and Bititci, 2005). Many authors believe that the main reason performance measurement in organizations is often
short-lived is because of people’s behavior with the information (Marchand et al., 2000;
Bititci et al., 2002). However, the influence of user’s characteristics on the use of PMSs has
been underexposed in scientific and professional literature (Vagneur and Peiperl, 2000; Krause, 2000; De Waal, 2002).
In other words, behavioral factors are important for the successful use of a PMS (e.g. Lipe and Salterio, 2000; Malina and Selto, 2001). This subject is studied in detail by De Waal (2002), who turned away from performance management research with a focus on the technicalities of designing and implementing a PMS. His focuses on behavioral issues linked with the use of PMS. In his research, De Waal identified 18 behavioral factors that are important for the regular use of PMS. In a later article, De Waal (2004) has further developed his work by grouping behavioral factors that are important for the regular use of PMS into five areas (manager’s understanding, manager’s attitude, PMS alignment, PMS focus, and organizational culture), to which a company should pay special attention. There is no fundamental difference with conclusions from the 2002 studies, but rather a different way of presenting. De Waal’s work is based on four case studies in three companies. Additional studies are required to substantiate his conclusions. Our study is a first step to contribute to that goal.
Methodology: Quest for balance revisited
In many organizations performance management systems (PMS) are introduced to improve performance in today’s dynamic and turbulent environments. However, results are often disappointing. As we have seen in the previous section, it may not be the system itself that is the problem, but the actual use of it by management. The main focus of De Waal’s study
Quest for balance was on the human element in PMSs: which behavioral factors influence managers’ use of a PMS?
Based on a literature review, De Waal deduced behavioral factors that could have a positive or negative impact on the use of a PMS. All in all, he identified some 45 factors. Then he selected criteria which denote when the use of a PMS and its components (like critical success factors, key performance indicators, etc.) is valuable for management. This led to a
total of seven so-called criteria for regular use.
In order to determine the relationship between the behavioral factors and the degree of the use of a PMS, De Waal conducted a multiple case study, in which four Dutch
organizations, both profit and not-for-the profit, were researched in depth. The description of the data collection and analysis process below closely follows De Waal (2002).
Data were collected through the use of questionnaires, interviews and document research . The questionnaire focused on the purposes managers used the PMS for, and on their attitude towards the PMS. Interviews were held with key persons in the design and
implementation stages of the PMS, and with users. A structured review list was used for the research of various written sources, like management reports, project documentation, and minutes of management team meetings.
For each organization, the gathered information was integrated in a case study description. To determine whether or not an organization satisfied a particular behavioral factor, scores were awarded as follows. If the results for a behavioral factor from all data were positive, a
Next, a final score for each of the stages (start, development, and use) was determined. This was done by calculating the average of all behavioral factors in a particular stage. If the average was below -0.2, the end result was denoted a being (-). For an average above +0.2, the end result (+) was given. For an average between -0.2 and +0.2, the end result was (0).
A similar procedure was followed to evaluate whether or not the criteria for regular use were satisfied. Subsequently, final analyis of the data took place by comparing the case studies with each other. Pattern matching was used to identify those behavioral factors with the greatest impact on the use of PMSs.
Patterns between the case studies were identified through a detailed examination of matches between the scores on individual behavioral factors, the end scores for the three stages, and the scores for the criteria for regular use. In his analysis, De Waal distinguished
behavioral factors that show a complete match (a match between all four case study scores),
a partial match (a match between two or three scores), and a non-match (a match between none or one of the scores).
According to De Waal, behavioral factors that show a complete match seem to have a general similarity with a successful implementation and use of a PMS. Consequently, these
factors can be considered essential, whereas behavioral factors with a partial match may be
important for successful PMS implementation and use. All in all, De Waal identified 18 behavioral factors that play an important role in the use of performance management systems.
In an attempt to substantiate De Waal’s conclusions, we replicated his research in four more Dutch organizations. Like De Waal, we selected both profit and not-for-the-profit organizations. Generally speaking our research in each organization followed De Waal’s design. First a questionnaire was distributed to all or most of the managers who used the PMS. The first part of the questionnaire focused on the behavioral factors that could have played a role in the different stages (design, implemetation and use) of the system’s development. The second part addressed the degree of manager’s use of the PMS. Secondly, interviews were held with a few managers and with those people who were responsible for the development project. And finally document research should shed light on the reasons to introduce a PMS and on the course of the implementation process.
Table II. Main characteristics of case studies
A B C D Kind of Performance system in use since… IT service Management system (ITIL) since 2004 BSC since 2002 Kind of BSC since 2004 PMS since 2002 with 18 KPIs Number of respondents (managers) 9 23 16 10 Number of 9 6 7 3
In table 2 we give an overview of the main characteristics of each case study. For confidentiality reasons, we have indicated the organizations as case A to D respectively.
Organization A is the common management organization of ICT systems of a Dutch ministry. More than 100 people are employed. Clients are owners of computer systems and applications, and end-users.
Organization B is a business enterprise in home appliances. The company is the result of a recent merger of three separate organizations, till then competitors, and gives employ to 350 people.
Organization C is the common maintenance organization of a division of the Dutch defense organization. More than 2300 mostly technical civilians are employed in this part of the organization .
Organization D is an energy company and the case study is carried out in the business area that is responsible for oil and gas exploration and production.
The results of these additional case studies were compared with the original findings of De Waal. At first sight, the results did not seem to be very consistent. For example, in our studies the number of important behavioral factors varied between 13 and 28, whereas De Waal identified 18 factors. These varying results raised some serious questions. Were the additional case studies too different from the original ones? Was the research method used too unreliable to generate consistent results? Or where there other factors that played a role?
With these questions in mind, we decided to combine the data of our additional case studies with De Waal’s original ones, thus extending the data set from four to eight case studies. We then re-analyzed all data, following De Waal’s pattern matching technique. In this way we changed the scope of our research. Apart from gaining a deeper insight into the
relationship between behavioral factors and manager’s use of PMSs, we now also looked for an explanation of the variability of the earlier results.
Before discussing the details of our findings, it is important to also take note of a number of assumptions De Waal has made in his research, because, as will become clear later on, this has given additional insight into reasons for different results in different case studies. During the review of De Waal’s results, a number of shortcomings in his work have been identified and therefore the decision was made to first assess the impact of these.
Firstly, due to errors in his pattern matching analysis, not 18, but 20 behavioral factors should have been identified as important for the regular use of PMSs.
Secondly, De Waal has drawn a number of firm conclusions, based on the assumption that
behavioral factors that show a partial match have a 50-75% matching score (two or three
out of four scores). But one can argue whether a match of 50% should be considered as a
partial match. Why not a minimum of 75% (three out of four scores)? In that case, the
Therefore, we decided to include a number of sensitivity analyses in the pattern matching exercise, in which we varied the matching scores of behavioral factors. By so doing we extended De Waal’s initial research design. Our analysis will be further described in the next section.
Results
In the previous section it was described that the additional case studies did not show a consistent result at first glance. Although a direct one-to-one comparison between the results of these case studies and De Waal’s work could not always be done (due to the fact that some of the studies also introduced a number of new behavioral factors), one recurrent finding was that the number of important behavioral factors varied between studies.
The reason for this inconsistency becomes clear when a sensitivity analysis is carried out on the pattern matching exercise. In De Waal’s research a behavioral factor is called important for the regular use of PMS if a match of at least 50% (two out of four scores) is obtained. Based on this definition he concluded that there are 18 (20) important factors.
In our study we have carried out pattern matching analyses for a number of definitions of a
match and looked at the impact on the number of behavioral factors that are identified as important.
The new analyses were done based on the total of eight case studies (De Waal’s original ones and the ones described in the previous section). Pattern matching (following De Waal’s methodology) has been applied to identify the behavioral factors that seem to be
It turns out that the definition of a match (e.g. 50% or 75%) is of key importance for the number of important behavioral factors. The more stringent the definition, the less
behavioral factors are identified as being important. The relation between matching criteria and number of important behavioral factors appears to be linear. The results are shown in figure 1.
Number of important factors vs matching criteria
0 5 10 15 20 25 30 35 0 20 40 60 80
pattern matching criteria (%)
nu m b e r of i m por ta nt f a c tors
Figure 1. Number of important factors vs matching criteria
The 18 (20) factors De Waal concluded to be important for the regular use of performance management systems are, therefore, rather arbitrary because it is the consequence of methodological choices made during pattern matching and data analysis. One could find 20, 10 or even only six important factors, depending on the criteria selected. The conclusion from the analysis is that one cannot identify a single firm number of factors that are important for the use of PMSs.
A spin-off of the results can be obtained if we use the analyses to determine the relative importance of the behavioral factors. The results from the analyses with different matching criteria have been presented with increasing number of corresponding scores, i.e. from 25% to 75%. In this way one can clearly see which factors are important in all analyses, and which ones only in analyses with less strict matching criteria.
The behavioral factors have been ranked in order of satisfying more stringent matching criteria. Behavioral factors that satisfy even the most stringent ‘matching definition’ are obviously the ones most important for the regular use of performance management systems. Based on the eight case studies, the results are listed in table 3 below, ranked from most important for the regular use of PMSs on top to least important at the bottom.
Table III. Behavioral factors ranked in order of importance Behavioral Factor 25% match 37.5% match 50% match 62.5% match 75% match
Managers realize the importance of CSFs/KPIs/BSC to their performance
Managers accept the need for performance management Managers have earlier (positive) experiences with performance management
Managers' frames of reference contain similar KPIs Managers are involved in making analyses
Managers do not experience CSFs/KPIs/BSC as threatening Managers clearly see the promoter using the PMS Managers agree on the starting time
Managers understand the meaning of KPIs Managers can influence the KPIs assigned to them
Managers find the PMS relevant because it has a clear internal control purpose.
Managers trust good quality analyses
Managers and their controlling systems have mutual trust Managers agree on changes in the CSF/KPI set Managers are stimulated to improve their performance Managers are informed about the status of the PMS project Managers have insight into the relationship between business processes and CSFs/KPIs
Managers' KPI sets are aligned with their responsibility area Managers understand the CSF/KPI/BSC reporting Managers find the PMS relevant because only those
stakeholders' interests are incorporated that are important to the organisation's success.
Managers trust the performance information
Managers' results on CSFs/KPIs/BSC are openly communicated Managers have an active role during the development stage of the PMS project
Managers are involved in setting KPI targets
Managers accept the promoter
Managers do not get discouraged by the collection of performance data
Managers have insight into the relationship between cause and effect
Managers' activities are supported by KPIs
Managers' information processing capabilities are not exceeded by the number of CSFs/KPIs
Managers can use their CSFs/KPIs/BSC for managing their employees
The above results allow companies, who want to improve the use of their performance management system, to pay attention to the most important factors first before addressing the others.
Discussion
Despite the promising results presented in the previous section, there are still questions unanswered. There is, for example, a number of factors in the top 10 that one would not expect to be amongst the most important ones. In other words, some results are surprising and there is no clear explanation why certain factors are more important than others. Further analysis of the results in table 3 does not give any lead here. The top 10 most important factors come from all three stages (design, implementation and use) identified by Franco and Bourne (2005), and thus there does not seem to be a reason to assume that one phase is more important than another.
Instead of accepting the results at face value, it is important to carefully consider De Waal’s methodology. He has used a number of assumptions which have not been justified or verified. For example, De Waal assumes that all behavioral factors in the list are independent, which can be easily challenged.
Another important assumption relates to the criteria for regular use. De Waal lists seven criteria, without describing the basis where they originate from, or justifying their selection.
One can argue the relevance of some of these criteria, e.g. what does the criterion “Are their plans for follow up projects” tell you about the regular use of the PMS ?
Another point of attention is that there appears to be an overlap between some of the criteria for regular use and the behavioral factors. Take for example the criterion for regular use “Regular communication about KPI results”. This is very similar to the behavioral factor “Managers’ results on CSFs/KPIs/BSC are openly communicated”. One can even argue whether some of the behavioral factors shouldn’t be more appropriate as a criteria for
regular use, e.g. “Managers use the PMS regularly during the planning and control cycle”.
Concluding one can say that De Waal has set an important first step in raising the awareness that behavioral factors are important for the regular use of performance management systems. Based on his work a number of additional studies have been carried out and described in this article, which give interesting new insights in the relative importance of these factors. However a number of questions have been raised with respect to De Waal’s methodology.
References
Bauer, K. (2004), “The power of metrics; KPI: not all metrics are created equal”, DM
Review, Vol. 14 No. 12.
Bititci, U.S., Nudurupati, S., Turner, T.J. and Creighton, S. (2002), “Web enabled
performance measurement system : management implications“, International Journal of
Operations and Production Management, Vol. 22 No. 11, pp. 1273-1287.
Bititci, U.S., Mendibil, K., Nudurupati, S., Garengo, P. and Turner, T. (2006), “Dynamics
of performance measurement and organisational culture“, International Journal of
Operations and Production management, Vol. 26 No. 12, pp. 1325-1350.
Bourne, M., Mills, J.,Wilcox, M., Neely, A.D. and Platts, K. (2000), “Designing,
implementing and updating performance measurement systems“, International Journal of
Operations and Production Management. Vol.20 No. 7, pp. 754-771.
Braam, G.J.M. and Nijssen, E.J. (2004), “Performance effects of using the balanced
scorecard: a note on the Dutch experience“. Long Range Planning, Vol. 37 No. 4, pp.
335-349.
Davenport, T.H. (1997), Information ecology, Oxford University Press, Oxford.
Davis, S. and Albright, T. (2004), “An investigation of the effect of balanced scorecard
implementation in financial performance“, Management Accounting Research, Vol. 15 No.
2, pp. 135-53.
De Kruijff, R. (2006), De prikkel tot gebruik van de balanced score card, Thesis (dutch
text), Open University of the Netherlands, Faculty of Management Sciences, Heerlen.
De Waal, A.A. (2002), Quest for balance - the human element in performance management
systems, John Wiley and Sons, Inc., New York.
De Waal, A.A. (2003), “Behavioral factors important for the successful implementation and
use of performance management systems”, Management Decision, Vol. 41 No. 8, pp.
688-697.
De Waal, A.A. (2004), “Stimulating performance-driven behaviour to obtain better results”,
International Journal of Productivity and Performance Management, Vol. 53 No. 4, pp. 301-316.
Eccles, R.G. (1991), “The performance measurement manifesto“, Harvard Business
Elzinga, T. (2006), A further investigation into the importance of behavioral factors on the use of performance management systems, Thesis, Open University of the Netherlands, Faculty of Management Sciences, Heerlen.
Franco, M. and Bourne, M. (2003), “Factors that play a role in managing through
measures“, Management Decision, Vol. 41 No. 8, pp. 698-710.
Franco, M. and Bourne, M. (2005), “An examination of the literature relating to issues
affecting how companies manage through measures“, Production Planning and Control,
Vol. 16 No.2, pp. 114-124.
Globerson, S. (1985), “Issues in developing a performance criteria system for an
organization“, International Journal of Production Research, Vol. 23 No.4, pp. 639-46.
Hoque, Z. and James, W. (2000), “Linking balanced scorecard measures to size and market
factors: impact on organizational performance“, Journal of Management Accounting
Research, Vol. 12, pp. 1-17.
Ittner, D., Larcker, D.F. and Randall, T. (2003), “Performance implications of strategic
performance measurement in financial services firms“, Accounting, Organisation and
Society, Vol. 28 Nos 7/8, pp. 715-741.
Kaplan, R.S. and Norton, D.P. (1992), “The balanced scorecard: measures that drive
performance“, Harvard Business Review, Vol. 70 No. 1, pp. 71-79.
Kaplan, R.S. and Norton, D.P. (1993), “Putting the balanced scorecard to work“, Harvard
Business Review, Vol. 71 No. 5, pp. 134-142.
Kaplan, R.S. and Norton, D.P. (1996), “Linking the balanced scorecard to strategy“,
Harvard Business Review, Vol. 74 No. 1, pp. 53-79.
Krause, O. (2000), “Management knowledge engineering: a toolkit to engineer adaptive
management systems“. In Neely, A.D. (Ed.), Performance measurement: past, present and
future, Center for Business Performance, Cambridge, pp. 307-314.
Kwak, J.F. (2006), Prestatiemanagement en het INK-model, Thesis (dutch text), Open
University of the Netherlands, Faculty of Management Sciences, Heerlen.
Lawson, R., Stratton, W. and Hatch, T. (2003), “The benefits of a scorecard system“, CMA
Lohman, C., Fortuin, L. and Wouters, M. (2004), “Designing a performance measurement
system: a case study”, European Journal of Operational Research, Vol. 156, 267-286.
Marchand, D., Kettinger, W. and Rollins, J. (2000), “Company performance and IM: the
view from the top”, In Marchand, D., Davenport, T. and Dickson, T. (Ed.), Mastering
information management, Financial Times/Prentice Hall, London, pp. 10-16.
Malina, M.A. and Selto, F.M. (2001), “Communicating and controlling strategy: an
empirical study of the effectiveness of the balanced scorecard”, Journal of Management
Accounting Research, Vol. 13, pp. 47-90.
Maskell, B. (1989), “Performance measures of world class manufacturing“, Management
Accounting, Vol. 5, pp. 32-33.
Neely, A.D., Gregory, M.J. and Platts, K. (1995), “Performance measurement system
design: a literature review and research agenda”, International Journal of Operations and
Production Management, Vol.15 No. 4, pp. 80-116.
Neely, A.D., Kennerly, M. and Martinez, V. (2004), “Does the balanced scorecard work: an empiral investigation”, paper presented at Performance Measurement Association Conference, Edinburgh, July.
Nudurupati, S. and U.S. Bititci (2005), “Implementation and impact of IT-supported
performance measurement systems”, Production Planning and Control, Vol. 16 No. 2, pp.
152-162.
Olsen, E.O., Zhou, H., Lee, D.M.S. and Padunchwit, P. (2007), “Performance measurement
system and relationships with performance results”, International Journal of Productivity
and Performance management, Vol. 56 No. 7, pp. 559-582.
Pettigrew, A.M. (1985), The awakening giant: continuity and change in Imperial Chemical
Industries, Blackwell, Oxford.
Prahalad, C.K. and Krishnan, M.S. (2002), “The dynamic synchronisation of strategy and
information technology”, MIT Sloan Management Review, Summer, pp. 24-33.
Rigby, D. and Bilodeau, B. (2005), Management tools and trends. Bain and Company,
Boston.
Vagneur, K. and M. Peiperl (2000), “Reconsidering performance evaluative style”,
Accounting, Organizations and Society, Vol. 25., pp. 511-525.
Van den End, W. (2006), Gebruik performance management systeem bij de GBO, Thesis
(dutch text), Open University of the Netherlands, Faculty of Management Sciences, Heerlen.
Be havior a l Fa c tor A Z U E IS -FA EI S-CS K ad Org B O rg C O rg A O rg D 25 % ma tch eed f or pe rf or m anc e m anag em en t + + + + + + + + e st ar ting t im e 0 + + + + 0 + 0 v olved in dec is ion m ak ing ab out t he -+ + -+ 000 ( pos it iv e ) ex per ie nc es w it h per for m a nc e ++ ++ +++ + ab le , r ela tively tr a nquil envir on m en t -0 0 0 -0 -v e r o le d ur ing t he develop m en t s ta ge of -+ + -+ + 0 0 d abou t th e s tat us of t he PMS pr oj ec t N A + + -0 0 + c o m m unic at ing abo ut t he PMS pr oj ec t N A + + -+ 0 0 -he m ea ning of KPI s 0 + + N A + + 0 + inin g K P Is -+ + -+ 0 0 0 ight int o the r e latio ns hip bet w e en s tr ate gy - 0-0-0 into t he r e la tio ns hip bet w e en bus ine s s PIs 0+ + 0 -+ 0 0 et ting KP I t ar get s -+ + 0 + 0 + 0 e aligned with t heir r e s pons ibilit y ar ea -+ -0 + + 0 -a k ing th e CS F /KP I/ BSC r epo rt lay o ut -+ + -0 0 -he C S F /K P I/ BSC r ep or ting + + + 0 + + 0 + 37. 5% mat c h 50 % ma tc h 62. 5% mat c h 75% ma tc h s /KP Is /BSC th at m a tc h t heir r e s pons ibilit y NA NA NA 0 NA 0 0 0 e the KP Is as s igne d to t hem 0 + + N A + + + -rom ot er -+ + 0 -N A 0 + o m ot er s pe nds enou gh tim e on th e NA -0 -NA NA 0 elevant bec aus e it h as a c lea r in te rnal ++ ++ ++ 0 + elevant bec aus e on ly t hos e ar e in co rpor at ed th at ar e im por tan t t o the . N A ++ ++ 0 +
-Be ha v ior a l Fa ct or A ZU E IS -FA E IS -CS K a d Or g B O rg C O rg A O rg D U1 Ma nager s have in
sight into the
re lations hip betw een KPIs and fi nanc ial r e s u lt s ---0 + 0 0 0 U2 Ma nager s do n ot get dis c our aged by the c ollec tion of pe rf or m anc e da ta + N A N A +++ 0 -U3 Ma nager s have in
sight into the
re lations hip betw een c aus e a nd ef fec t ---+ 0 0 U4 Ma nager s ar e inv o lv ed in f o re c a s ting ----+ 0 --U5 Ma nager s tr us t goo d quality f o re ca sts N A N A N A N A + 0 -0 U6 Ma nager s ' ac tiv ities ar e sup por ted b y KPIs NA 0 -0 + 0 + -U7 Ma nager s ' f ram es of r ef er e nce c ontain s im ila r KPIs -0 -+++++ U8 Ma nager s tr us t the per fo rm an ce inf o rm ation 0 + -0 + 0 -0 U9 Ma nager s ar e inv olv ed in m ak ing an aly s es 0 + -+ + + 0 -U10 M a nager s tr us t goo d quality ana ly se s 0 0 -0 -+ 0 + U11 Ma nager s ' inf or m ation pr oc es sing c apab ilities ar e n ot ex c eeded by the num ber of CSF s /KPIs -N A N A-+ + 0 + U12 M a nager s have eno ugh tim e to w o rk w ith their C S F s /KPIs /BSC + 0 0 + + 0 0 0 U13 Ma nager s r ealiz e th e im por ta nc e of C S F s /KPI s/BSC to their pe rf or m anc e 0 + ++ 0 +++ U14 M a nager s do n ot ex per ienc e CSF s /KPIs /BSC as thr eaten ing + + ++++++ U15 Ma nager s c an us e t heir CSF s /K PIs /BSC f o r m a naging their em ploy ees ++ 0 + -0 0 -U16 M a nager s have s ole re sp ons ibility f or a KPI -+ + 0 + -0 -Use S tage U17 M a nager s c lear ly s ee the pr om o ter us ing the PM S -+ -0 -+ + + U18 M a nager s and their c on tro lling s y ste m s h av e m u tual t ru s t N A N A N A + NA + + + U19 M a nager s f ind th e PMS r elevant d ue to r eg ular evalua tions -N A N A -0 + 0 U20 Ma nager s us e th e PMS r egular ly dur ing
the planning and
co ntr ol cy c le 0 N A N A -+0 +0 U21 M a nager s agr ee on c hang es in th e CSF /KPI s et -+ -+ + + 0 U22 M a nager s ar e s tim u lated to im p ro v e th eir per for m anc e N A 0 -+ -+ 0 + U23 M a nager s ' r e su lts on CSF s /KPIs /BSC ar e o penly c o m m u n ic a ted + + 0 + -0 0 + U24 M a nager s ' us e of the PM S is s tim ulated by the r ew ar d s tr uc tur e --
---A Kad O rg B O rg C O rg C rit er ia fo r r e gul a r us e A ZU EIS-F A EIS-CS Or g D 1 Are th e r e s u lts of th e o rg anis a tion , ac co rd ing t o th e m a n a g e rs , im prov e d t h ro ug h t h e u s e of th e p erf o rm an c e m a na g em en t sy s te m ? -0 0+ -+ 0+ 2 Are th e r e s u lts of th e o rg anis a tion , ob je c tively , im pr oved th ro ugh th e u s e of th e p e rf o rm a n c e m a nag e m ent s y s tem ? 0 0000 N A 00 3 Has th e d e gree of p e rf o rm an ce m a n ag em e nt s y st em use b y m ana ge rs inc rea s e d ? 0+ -+ + + 0 -4 A re th er e p la n s f o r f o llo w -u p pr oj ec ts ? + -0 -+ + + 5 Is t he re a d if feren ce i n m a na ge r a tti tud e t o w ards pe rform a nce m ana ge m e nt, fr om pr oj ec t s tar t to c u rr en tly ? 0+0 + 0+ + -6 Is th er e r e g u lar c o m m un ic atio n a b ou t KPI r e s u lts ? 0 + -+ 0 + + + 7 Are th e Cr itic a l S u c c es s F a c tor s (CSF 's ), Key P e rf o rm anc e In dic a tor s (KPI' s ) an d Bala nc e d Sc or e Ca rd ( B SC) inc o rp o ra ted in t he re gula r m a n a g e m e nt re por ting ? ++ 0 + ++ 0 + F in a l S c o re crit eria f o r re gu la r us e 0 + -+ 0 ++ +
Working Papers Management Sciences
Groene en Gele reeks 2007-2008
GROEN
Code Auteur(s) en titel
gr07-01 Marjolein Caniels en Cees Gelderman
The safeguarding effect of legal, economic and social
control: The decisive role of mutual opportunism
gr07-02 Marjolein C.J. Caniëls, Adriaan Roeleveld, Janjaap
Semeijn:
Power and dependence perspectives on outsourcing decisions
gr07-03 Kesidou, Caniels en Romijn
Mechanisms of Local Knowledge Spillovers: Evidence from the Software Cluster in Uruguay
gr07-04 Caniels and Romijn
Actor networks in Strategic Niche Management: Insights from Social Network Theory
gr07-05 Savelsbergh, Van der Heijden en Poell
Explaining Differences in Team Performance. Does team learning behavior matter?
gr07-06 Janssen. Kusters en Heemstra
Clustering ERP implementation project activities
gr07-07 M.C.J. Caniels en H.A. Romijn
Does innovation matter for LDC’s? Discussions and New Agenda
2008
gr08-01 P.G. Daams, C.J. Gelderman en J.M.C. Schijns: The
impact of loyalty programs in a B-to-B context - results of an experimental design
GEEL
Code Auteur(s) en titel
ge07-01 Caniels, Van Eijck en Romijn: Development of
new supply chains:
Insights from Strategic Niche Management
ge07-02 Caniels, Kesidiou, Romijn: The software sector in
Uruguay: A Sectoral Systems of Innovation perspective
ge08-01 F. de Langen: Business cases in an electronic
environment - lessons for e-education?
ge08-02 G. Janssens, R. Kusters en F. Heemstra: A small
survey into the importance of and into a concept for estimating effort-related costs of ERP