Designing and validating an instrument to measure the degree of customization and cost of use and maintenance for a packaged ERP system

(1)

Eindhoven, February 2010

BSc Industrial Engineering and Management Science — TU/e 2007 Student identity number 0519968

in partial fulfilment of the requirements for the degree of Master of Science

in Operations Management and Logistics

Supervisors:

dr.ir. R.M. Dijkman, TU/e, IS prof.dr. R.J. Kusters, TU/e, IS

drs. D. Zijlstra, Accenture, SI&T Consulting

Designing and validating an

instrument to measure the degree

of customization and cost of use

and maintenance for a packaged

ERP system

By

(2)

ii TUE. Department Technology Management.

Series Master Theses Operations Management and Logistics

(3)

iii

Abstract

Standard Enterprise Resource Planning (ERP) packages are often customized by the adopting organizations in order to close the gap between the functionalities offered by the package and the specific needs of the organization. When customizing these packages, most organizations only take into account the extra costs during the implementation phase and neglect the ongoing use and maintenance costs. A reason for this is the lack of a good estimation tool which can be used to estimate the expected use and maintenance cost as a function of the degree of customization (DOC). To develop such a tool, data should be gathered from practice on degree of customization and cost of use and maintenance but in order to be able to do that first a validated measurement instrument is needed. This research focuses on developing and validating such an instrument which can be used to quantify the DOC and estimate the cost of use and maintenance. Two case studies are conducted: one to design the instrument and one to validate it.

(4)

iv

Summary

Standard Enterprise Resource Planning (ERP) packages developed by vendors such as SAP and Oracle are designed to support standard business processes. However, since every organization has some uniqueness in their processes at a certain level, there is always a gap between business and IT. This gap could either be closed by customizing the ERP package (IT) or changing the existing business processes. To avoid costly change management practices, most organizations choose to customize the ERP package and while doing that they only account for the extra costs during the implementation and neglect the costs of using and maintaining the package.

This research is carried out at Accenture Consulting and focuses on the relation between the cost of use and maintenance for a specific ERP package, PeopleSoft HRMS, and the degree to which the package is customized. The following problem statement is formulated:

Taking the above stated problem into account the following research goal is formulated:

To design the instrument, knowledge from the literature is combined with findings from a case study which is conducted at a Dutch Regional Education Centre (ROC) and validated by conducting an additional case study at an Academic Hospital in Leiden, Netherlands (LUMC). The measurement instrument consists of two frameworks: one for estimating the cost of use and maintenance and one for quantifying the degree of customization. Figure 1 represents the first framework. From the figure we can conclude that there are seven main use and maintenance activities which bring costs along three different dimensions (people, process and products&services) and that these costs can be estimated by using several sources of information within an organization.

There is a lack of insight into the relation between the degree of customization to the HRMS package and its use and maintenance costs after the implementation. Moreover, there are no tools which can be used to: (1) quantify the degree of customization (DOC) and (2) estimate the associated cost of using and maintaining the customized HRMS package based on the DOC.

Designing a measurement instrument which can be used to:

1. Quantify the extent to which the ERP package is customized and 2. To estimate the cost of using and maintaining such a package,

with the objective of gathering data from practice to do research on the exact relation between the degree of customization and the associated costs of use and maintenance so that the research results can be used to develop a tool, similar to existing tools at Accenture, which can be used by Accenture consultants to estimate the cost of use and maintenance for a certain level of customization.”

(5)

v

Figure 1: Measuring cost of use and maintenance

To quantify the degree of customization for a specific system, the amount of development time is taken as a measure. The total degree of customization is thus the total amount of development time spent on customizing the ERP package. The DOC can be estimated in two ways: either by simply summing up all the effort and calculating the total development time in hours or using the matrix of figure 2.

Customization types

Complexity

Simple Medium Complex Very complex

Reports 0 < dt ≤ 56 i = 31 56 < dt ≤ 84 i = 66 84 < dt ≤ 157 i = 121 157 < dt i = 292 Interfaces 0 < dt ≤ 44 i = 33 44 < dt ≤ 116 i = 66 166 < dt i = 213 Extensions 0 < dt ≤ 44 i = 21 44 < dt ≤ 103 i = 65 103 < dt ≤ 227 i = 179 227 < dt i = 281 Conversions 0 < dt ≤ 90 i = 56 90 < dt ≤ 212 i = 124 212 < dt i = 300 Workflows 0 < dt ≤ 57 i = 49 57 < dt ≤ 71 i = 64 71 < dt i = 77

(6)

vi

The matrix can be used to estimate the development time based on experts judgment on how complex a piece of customization is. First it should be evaluated to which of the five types the customization belongs (vertical axis of the matrix) and then how complex it is using the four categories: simple, medium, complex and very complex (horizontal axis of the matrix). To assess the complexity the assessor can also make use of the ranges in development time (dt) for each complexity level. Then, depending on which cell is pointed out in the matrix the complexity index (i) can be used to calculate the total degree of customization which is the sum of all the complexity indices:

Where CIi is the complexity index of the i-th customization.

Figure 3 presents the findings from the two case studies conducted at ROC and LUMC. As we can see the differences are large for people and products&services dimensions but there is almost no difference in costs along the process dimension.

Figure 3: Findings from the two case studies

0 1000 2000 3000 4000 5000 6000

ROC (DOC = 2120) LUMC (DOC = 3911) Process People 0 200000 400000 600000 800000

ROC (DOC = 2120) LUMC (DOC = 3911) Products & Services

(7)

vii

1 Introduction

Standard ERP systems are off-the-shelve software packages designed to support typical business processes within a whole field or industry. Unlike tailor-made information systems, ERP systems are based on best practice processes from the specific industry. Since these ERP systems are standard and organizations have some uniqueness in their processes on a certain level of detail, there is often a misfit between the ERP system and the adopting organization’s business processes. Academic literature provides two main kinds of solutions to resolve these misfits. Either the organization should adopt the business processes on which the ERP package is based or the ERP package should be tailored to meet the organizations demand. Tailoring the ERP package can vary between simply configuring the system in order to choose between different executions of processes and functions to changing the software code. While configuring the system is seen as normal by the ERP vendors, all the other changes or customizations are discouraged due to limitations and increasing costs concerning maintenance, future updates and migrations. Despite this, the occurrence of customizations in ERP implementation projects is very common. However, a large number of companies which customize their ERP, fall back to a new version without any customization at all (also called the vanilla system) because of cost overrun during the usage period of the previous version. One of the hypothetical reasons for this is that the costs of using and maintaining a customized system turns out to be much higher than most organizations anticipated during the implementation phase.

Taking the above in mind the conclusion can be drawn that there is a lack of insight into the use and maintenance cost of a customized ERP package and especially into the relation between the degree (or extent) of customization and the associated use and maintenance costs after the implementation. To explore this relation it is necessary to measure the cost of use and maintenance at one side and the degree of customization at the other side in practice and compare the cases with each other. But to be able to do that first it has to be defined what is exactly meant by customizations and the cost of use and maintenance and how these two can be measured. This research concentrates on developing a measurement instrument which can be used to measure the cost of use and maintenance and the degree of customization for an ERP package. The instrument is designed for a specific ERP package, namely PeopleSoft Human Resources Management System (HRMS) module of Oracle developed through a case study at a regional education centre and validated through another case study conducted at an Academic Hospital in Leiden.

This introduction chapter is organized as follows. First some background information is given on the topic in section 1.1. Within the research context, which is described in sections 1.2, the problem statement is formulated in section 1.3 and taken the problem statement in mind the research goal is stated in section 1.4.

1.1 Background

In this section some background information is given on ERP customization. First, it is explained what exactly a standard ERP system is and secondly what kinds of misfits could exist between

(11)

2

such a software system and the business processes of the organization adopting it. Finally, it is explained how a standard ERP software can be modified to resolve the misfits.

1.1.1 Standard ERP systems

ERP systems are process aware information systems which support the operational processes of the whole enterprise by integrating and optimizing the processes and managing transactions throughout the whole organization (Moon 2007). According to Moon (2007), ERP finds its origin in the manufacturing which needed an information system that was capable of calculating more efficiently materials required through the whole manufacturing process as well as enabling efficient use of manufacturing capacity and optimizing scheduling (hence the terms Resource and Planning). Soon after these kinds of information systems proved their value for manufacturing, companies began to realize that their functionality could be extended beyond manufacturing and encompassing finance, sales and distribution and human resources (Klaus et al. 2000).

To avoid major development costs, during the last decades, more and more companies choose to purchase packaged ERP software resulting into the rise of major ERP vendors such as SAP and Oracle. In order to be able to earn back the large development costs these vendors needed to sell as many software packages as possible and therefore standardize the software as much as possible. Although standard ERP packages are developed for specific processes (such as Human Resource and Supply Chain Management) and for specific industries (such as financial or manufacturing industry) and based on so called best practices within these fields, there still remains a certain misfit between these ERP packages and the business processes of the adopting organizations. This misfit is addresses in the next subsection.

1.1.2 Misfit between ERP and business

Misfit between ERP packages and organizations is primarily caused by the fact that standard ERP systems are designed to support standard processes in a certain field whereas each organization has its own unique way of working, which is influenced by a combination of company specific, sector specific and country/culture specific factors (Soh et al. 2000).

There are four types of misfits identified in the literature: Data, Functional, Output and Interface (Hong and Kim 2002; Soh et al. 2000):

1 Data misfit refers to incompatibilities between organizational requirements and ERP package in terms of data format and the relationships between entities as represented in the underlining data model.

2 Functional misfit can be divided into Access, Control and Operational misfit. Access misfits occur when the required access to perform certain tasks is not present, control misfits refer to missing validation procedures or checking routines and operational misfits refer to missing operational steps or the presence of certain steps which are inappropriate. 3 Output misfit refers to incompatibilities between requirements and ERP package in terms

(12)

3

4 Interface misfit occurs when there is a gap between the way the graphical user interface (look and feel) of the ERP package is designed and what the users within the adopting organization are accustomed to work with.

1.1.3 Customizing ERP to resolve misfits

Although standard ERP packages allow the adopting organization to configure the software to meet most of their specific needs, there always remain some misfits which cannot be resolved by simply configuring the package. In order to resolve these misfits, there are basically two things that an organization can do: either changing their processes to better match the software package or modifying the software to meet their specific needs. The latter is referred to as customizing the ERP package.

1.1.4 ERP lifecycle

There are four basic stages within an ERP systems lifecycle which can be distinguished from each other (Brehm and Markus 2000; Esteves and Pastor 1999; Markus and Tanis 1999):

‐ Acquisition: in this phase the decision whether or not to buy a packaged ERP system is analyzed as well as which package would be the most suitable one for the organization. ‐ Implementation: in this phase the package is actually configured and implemented. All

the decisions regarding whether, what and how much to customize are made in this phase.

‐ Use and maintenance: during this stage the system is actually used and maintained. This phase is or at least ought to be the biggest stage in the ERP systems lifecycle since in this phase the organization actually benefits (or should benefit) from the ERP system and there is return on investment.

‐ Evolution: after using the ERP package for a while, internal and external factors might cause the system to evolve either to a complete new package or an upgrade of the existing one. Internal factors could be anything from changing the processes within the organization or changes in the IT environment of the ERP package whereas external factors could be the opportunity to upgrade to a better system with more functionalities or the expiration of support from the vendor for the current version.

1.1.5 Definition of ERP maintenance

Maintenance of traditional software systems is defined by the IEEE as “the modification of a software product after delivery to correct faults, to improve performance or other attributes, or to adapt the product to a modified environment” (IEEE 1990). This definition points out that maintenance is a post-delivery activity and comprises all the tasks necessary to correct faults, improve performances and to adapt to changed environment. Correcting, improving and adapting could either be done to the software code, functionality or the hardware on which the software system is running.

Since there are significant differences between standard ERP packages and traditional tailor-made software systems (Klaus et al. 2000), this definition should be adjusted for ERP. Ng et al. (2002) state that ERP maintenance involves “all post-implementation activities related to the

(13)

4

packaged application software undertaken by the client organization from the time the system goes live (i.e. successfully implemented) until it is retired from an organization’s production system:

- To keep the system running,

- To adapt to a changed environment in order to operate well, - To provide help to the system users in using the system, - To realize benefits from the system, and

- To keep the system a supported-version and meet the vendor’s requirements for standard code.” (Ng et al. 2002)

1.2 Research context

This research is carried out at a company called Accenture. Accenture is a global management consulting, technology services and outsourcing company which among other things offers consulting services during ERP implementation projects. One of the various ERP modules for which Accenture offers consulting services is the PeopleSoft Human Resource Management System (HRMS).

The research is carried out within the Service Integration and Technology (SI&T) department of Accenture Netherlands. The sub department “Oracle” within SI&T is responsible for implementation projects of Oracle modules and the focus group “Oracle HR” focuses specifically on the PeopleSoft HRMS package and has implemented this package with and without customizations in various companies in the Netherlands. The HR focus group is mainly interested in a tool which enables their consultants to predict the expected cost of using and maintaining the HRMS module for client companies who are considering customizing the package to a certain extent.

PeopleSoft HRMS was originally developed by the company PeopleSoft Inc, which was specialized in human resource management, customer relationship management and student administration systems. In 2005 PeopleSoft was taken over by Oracle Corporation; the PeopleSoft name and product line are now marketed by Oracle, who also is responsible for developing maintenance patches and upgrades for the software. Table 1 provides an overview of HRMS versions released during the last decade along with the availability of upgrades and patches for each version. Since Oracle does not release any upgrades and patches for older versions than HRMS 8.9, it is likely that most companies currently use version 8.9 or higher.

(14)

5

Table 1: Overview of PeopleSoft HRMS versions

Version Release Updates,

Fixes, Security Alerts, and Upgrade Scripts Tax, Legal and Regulatory Extended Support Ends Sustaining Support Ends

HRMS 8.0 SP1 Dec 2000 Mar 2007 Mar 2007 Not Available Indefinite

HRMS 8.3 Nov 2001 Mar 2007 Mar 2008 Not Available Indefinite

HRMS 8.8 Dec 2002 Dec 2007 Dec 2008 Dec 2011 Indefinite

HRMS 8.9 ANZ Sep 2005 Sep 2010 Sep 2011 Sep 2013 Indefinite

1.3 Problem statement

Since standard ERP systems, like the HRMS package, do not fully meet the requirements of all the companies using it, there is always a gap between the ERP functionality and the business processes of the companies. To fill this gap, the adopting organizations have the option to adjust their processes and/or to tailor the ERP system (Ehsary 2009).

According to Accenture consultants, most organizations choose to customize the ERP system and this choice is often based on cost and benefit analysis concentrating on only the implementation phase. As long as the cost of customizations fit within the implementation budget, companies mostly choose to customize the ERP system in order to avoid difficult change management practices. But what often is neglected are the update and maintenance costs incurred after the implementation phase, during the rest of the ERP system’s lifecycle, as a result of these customizations. Figure 4 shows hypothetically the difference in total lifecycle costs between a vanilla- and customized system as it is expected to be according to Accenture consultants. The figure demonstrates that although these costs may not seem much at the implementation phase, during the systems usage period the costs could dramatically increase for a customized system because of implications during update and maintenance activities. Moreover, the more complex the customizations are, the more dramatically the costs are expected to increase at a certain point, which results into exceeding the allocated budget for the ERP package before the accounted depreciation period is reached.

(15)

6

Figure 4: expected total cost of vanilla system versus customized system

To be able to offer better services to the customers and to help them avoid major cost overrun during the lifecycle of the (customized) HRMS package, Accenture consultants need to gain more insights into the exact relation between the complexity of customization and the use and maintenance costs of the HRMS package. So to conclude, Accenture consultants face the following problem:

There is a lack of insight into the relation between the degree of customization to the HRMS package and its use and maintenance costs after the implementation. Moreover, there are no tools which can be used to: (1) quantify the degree of customization (DOC) and (2) estimate the associated cost of using and maintaining the customized HRMS package based on the DOC.

1.4 Research goal

Taking the problem statement in mind, the main goal of the project can be defined as: “Designing a measurement instrument which can be used to:

1. Quantify the extent to which the ERP package is customized and 2. To estimate the cost of using and maintaining such a package,

with the objective of gathering data from practice to do research on the exact relation between the degree of customization and the associated costs of use and maintenance so that the research results can be used to develop a tool, similar to existing tools at Accenture, which can be used by Accenture consultants to estimate the cost of use and maintenance for a certain level of customization.”

(16)

7

The above stated research goal is further divided into the following sub–goals:

1. Development of a framework which can be used to estimate the total cost of maintenance and use.

2. Validation of the framework for cost of maintenance and use through case study analysis

3. Development of a framework which can be used to quantify the degree of customization (DOC) for the PeopleSoft HRMS package.

4. Validation of the DOC framework through case study analysis

5. Analyzing the results of the case studies and drawing preliminary conclusions regarding the relation between DOC and cost of use and maintenance

1.5 Report overview

This report is structured as follows. After this introduction chapter, in chapter 2 the research approach is explained in detail. Chapters 3 and 4 focus on the frameworks for estimating the cost of maintenance and quantifying the degree of customization respectively. The validation process for both frameworks is explained in chapter 5. In chapter 6 the findings from the case studies are discussed and chapter 6 concludes the report.

(17)

8

2 Research

approach

In this chapter the research approach is explained in more detail. Figure 5 represents the research process on a high level. As it can be noted from the figure, the starting point is the research goal as defined in section 1.4. Taking the research goal in mind, a literature review (2.1) is conducted to find out how literature can contribute when designing the instrument. Next to literature, also findings from a case study analysis and an analysis of currently used methodology at Accenture are used to design the measurement instrument. The analysis of Accenture methodology and the case study analysis are more thoroughly explained in sections 2.2 and 2.3 respectively. In section 2.4 it is explained how exactly the design process is carried out. The instrument is validated through an additional case study. This validation process is explained in more detail in Section 2.5. Finally, section 2.6 explains how the findings of the case studies are analyzed to draw preliminary conclusions on how the relation between the DOC and cost of maintenance looks like.

Research goal

_{Literature review} (2.1)

Analysis of Accenture methodology (2.2)

_{Case study} analysis (2.3) Designing the measurement instrument (2.4) Validation through case study (2.5) Analysis of results (2.6)

Figure 5: Research approach

2.1 Literature review

In order to avoid ‘reinventing the wheel’ it is important to first study relevant literature on ERP packages to see what already has been researched and how it can contribute to the design of the measurement instrument. Therefore, the research goal is used to define the following set of research questions for the literature review:

Research questions on ERP maintenance

1. What is ERP maintenance and which types of ERP maintenance exist? 2. How can maintenance activities be organized?

3. What kinds of costs are involved with ERP maintenance?

Research questions on ERP customization

4. How can an ERP package be customized?

5. What are the implications of customizations on (cost of) maintenance? 6. How can the degree of customization be quantified?

(18)

9

To answer the above stated questions, academic journals and books as well as research papers published by large commercial research organizations were consulted. Table 2 presents an overview of the used resources and search terms.

Table 2: Overview of literature sources and search terms

Sources Search term(s)

Academic research libraries

ACM Digital Library Elsevier ScienceDirect Emerald

IEEE – IEE Electronic Library (IEL) IngentaConnect

JSTOR SpringerLink Wiley Interscience

Commercial research organizations

Gartner Research Burton Group Forrester Research

International Data Group (IDC)

ERP maintenance

ERP application management Information systems maintenance Information systems management ERP lifecycle cost

ERP maintenance cost Cost of IS maintenance ERP (mis)fit

ERP modification ERP customization Tailoring ERP systems

It should be noted here that these search terms were only used to get started and to find the most important research and surveys on these topics. A substantial part of the references used in this report are found by looking into the references used in the surveys.

2.2 Analysis of Accenture methodology

Accenture consultants already use a methodology, called Accenture Delivery Methods (ADM), to estimate the amount of effort needed for ERP implementation projects. To determine the estimated effort, ADM also take into account the amount of customization a client needs or asks for. ADM tools provide a good starting point for designing the instrument which can be used to quantify the DOC and therefore it is wise to study them to see how they can be used in the design process.

Another reason for first analyzing ADM tools is because of the terminology used in it. Since the goal of the measurement instrument is to collect data which can be used to develop a tool for Accenture consultants, it would be convenient to use the same terminology as in the already existing Accenture tools.

(19)

10 7. How is ADM developed and how does it work? 8. How are ADM tools used by Accenture consultants? 9. Which parts of ADM is relevant for this research?

10. How can ADM be used when designing the measurement instrument?

2.3 Case study research

Next to literature and Accenture methodology, a third important source of information is conducting field research in order to be able to find out what kind of information is available in practice and how and where exactly within an organization one could gather data on cost of maintenance and degree of customization. Therefore, the following research questions are addressed by the field research:

11.How do organizations organize and execute ERP maintenance activities? 12.How can one gather data on cost of maintenance activities in an organization?

13.How do organizations keep track of the number and type of customizations in their ERP package?

14.How can one gather data on the degree of customization of an ERP package used in an organization?

There are several ways to conduct field research such as experiments, surveys, workshops, focus groups, case study analysis etcetera; see (Cooper and Schindler 2001) for an extensive list. However in this research, we have chosen to conduct a case study because of two specific characteristics of case study research: (1) high complexity of the unit of analysis and (2) possibility to do in-depth contextual analysis (Cooper and Schindler 2001; Yin 2008). Since ERP packages are enterprise wide systems and people from different departments and with different expertise are maintaining and using the package, the unit of analysis, which is the organization using the ERP package, is very complex. Also, to answer the above defined how-questions, an in-depth contextual analysis is necessary and therefore a case study approach is used here (Yin 2008).

The case study analysis is carried out at a Dutch education institution (“ROC Eindhoven”) which has implemented the PeopleSoft HRMS package and has been using it for a few years. To conduct the case study, methodology of Yin (2008) is used. Within the organization first an intake interview with the head of Application Management was carried out. The intake interview was not structured and its goal was to present the problem statement and research goal in order to determine together with the respondent which people and roles within ROC Eindhoven are responsible for maintaining the PeopleSoft HRMS package. After the intake interview, three semi-structured interviews were conducted later on with (1) the change manager, (2) PeopleSoft developer and (3) PeopleSoft HRMS functional application manager to answer the research questions stated above. Next to the interviews, also some documentation and archival records were provided by the respondents in order to support their answers and increase construct validity (Yin 2008).

(20)

11

2.4 Designing the measurement instrument

The measurement instrument has been developed by answering the research questions defined in sections 2.1, 2.2 and 7; thus combining theory form academic literature and commercial papers with Accenture methodology and findings from a field research. The instrument basically consists of two frameworks which are elaborated in more detail in chapters 3 and 4:

1. Framework for measuring cost of use and maintenance of ERP packages (Chapter 3) 2. Framework for measuring the degree of customization (Chapter 4)

Figure 6 depicts how the three sources of information are combined to design the instrument. As it can be noted from the figure, the answers to research questions 1-3, 11 and 12 are used to design framework 1 and the answers to research questions 4-6, 13 and 14 to design framework 2. Research questions 7 through 10 focus on how Accenture methodology can contribute in the design of the whole instrument and thus the answers to those questions are used for both frameworks (this is illustrated by the dashed line in the centre column between framework 1 and 2.

Literature review Accenture methodology Field research

1. What is ERP maintenance and how are maintenance activities organized 2. What kinds of costs are involved with ERP maintenance 3. Which types of ERP

maintenance exist?

4. How can an ERP package be customized?

5. What are the implications of customizations on (cost of) maintenance?

6. How can the degree of customization be quantified?

11. How do organizations organize and execute ERP maintenance activities? 12. How can one gather data on cost of maintenance activities in an organization? 7. How is ADM developed

and how does it work? 8. How are ADM tools used by Accenture consultants? 9. Which parts of ADM is relevant for this research? 10. How can ADM be used when designing the

measurement instrument?

13. How do organizations keep track of the number and type of customizations in their ERP package?

14. How can one gather data on the degree of customization of an ERP package used in an organization?

Figure 6: Combining literature, Accenture methodology and field research findings to design the measurement instrument

2.5 Validation process

In order to ensure the quality of the measurement instrument, its validity should be tested first. Two types of instrument validity are taken into account here: external and internal validity (Carmines and Zeller 1994; Yin 2008). External validity refers to the extent to which the

(21)

12

instrument is applicable in other situations, or in other words the extent to which the instrument is generalizable and transferable whereas internal validity refers to how well the instrument does what it is designed to do. The latter can again be divided into construct and content validity (Figure 7).

Figure 7: Validity

Construct validity refers to the extent to which the instrument measures what it is supposed to measure. In this study the construct validity can be determined by evaluating after the case study whether that what has been measured matches with what was intended to measure. So the following questions are taken into consideration:

15.Did the measurement framework measure the costs of maintenance for the specific ERP package?

16.Did the measurement framework quantify the degree of customization of the specific ERP package?

Content validity ensures that the instrument measures everything that is relevant and leaves nothing out. To evaluate the instruments content validity, the findings are compared to that of similar researches in academic literature. The following questions are addressed here:

17.Are all relevant costs of maintenance for the specific ERP package taken into account? 18.Are all kinds of customization options taken into account?

External validity of the instrument is addressed by conducting an additional case study and evaluating for each of them whether the instrument consistently measures what it supposed to measure. Thus the research questions 15-18 are again evaluated, but now at a different organization using the same measurement instrument. To ensure the external validity we need to ask the following:

19.Are there any significant differences between the answers to the research questions 15-18 when we compare the ROC case study with the LUMC case study?

(22)

13

2.6 Analysis of results

Once the instrument is designed and validated, the data collected during the case studies can be analyzed to see whether we can draw any conclusions regarding the relation between the cost of maintenance and the degree of customization. Based on the data gathered we try in this step to answer the following questions:

20.Are there any differences in the magnitude and composition of the total cost of maintenance for the ERP package between the two cases?

21.Are there any differences in magnitude and composition of the degree of customization of the ERP package between the two cases?

22.Can we conclude on the basis of the findings that there is a positive relation between the degree of customization and the total cost of use and maintenance?

The answers to the above stated questions can be found in chapter 6 where the findings from the case studies are discussed.

(23)

14

3 Framework for measuring cost of use and maintenance

In this chapter the framework for measuring the cost of use and maintenance of a standard ERP package is presented. First the findings from the literature are discussed in section 3.1. Section 3.2 covers the findings from the case study analysis and in section 3.3 the framework for cost of maintenance is presented. Section 3.4 finally concludes this chapter.

3.1 Findings from literature

In sections 3.1.1 through 3.1.2 findings from the literature review are presented which cover the following research questions:

1. What is ERP maintenance and which types of ERP maintenance exist?  3.1.1 2. How can maintenance activities be organized according to literature? 3.1.2 3. What kinds of costs are involved with ERP maintenance? 3.1.3

3.1.1 Maintenance categories

With ERP systems being relatively new compared to traditional information systems, there is little scientific literature available about its use and maintenance stage. Therefore, in this subsection research on software maintenance in general together with that on maintenance of ERP systems in particular is reviewed to understand what kinds of activities typically are carried out in this phase. From the literature there are seven maintenance categories identified:

- Corrective: refers to fixing errors in the program, fixing performance issues and repair implementation failures (Abran and Nguyenkim 1991; Burch and Grupe 1993; Ng et al. 2002; Swanson 1976)

- Adaptive: refers to adapting the system to meet changing environment such as changes in software and hardware or business processes and regulations (Abran and Nguyenkim 1991; Burch and Grupe 1993; Ng et al. 2002; Swanson 1976)

- Perfective: eliminating the programs inefficiencies or other imperfections such as source code readability and poor use of computer operator time (Abran and Nguyenkim 1991; Burch and Grupe 1993; Ng et al. 2002; Swanson 1976)

- Preventive: proactive maintenance which consists of a periodic inspection and review of the system to uncover and anticipate problems (Burch and Grupe 1993)

- User-support and training: supporting and training the end users in using the system (Abran and Nguyenkim 1991; Ng et al. 2002)

- Upgrade: upgrading the software to a newer version which has been developed by the vendor with major functional and technical improvements (Ng et al. 2002)

- Patch maintenance: implementing small patches distributed by the vendor in order to fix, problems, adjust to changes in regulations or to realize small functional and/or technical improvements (Ng et al. 2002)

3.1.2 Organizing maintenance

In order to be able to measure the costs of ERP maintenance activities it is important to find out how these activities are typically managed within an organization. Since ERP is an information

(24)

15

system (IS) and maintaining such a system is something which is part of the IS management field, we approach ERP maintenance from this viewpoint.

IS management involves all the activities and tasks needed to design, implement and manage the various information systems within an organization. IS management can be approached from several different perspectives such as the nature of the tasks and the object(s) on which the task is performed, the states in which an IS can find itself, the specific processes carried out and the way IS management tasks are organized (Looijen 2004). The latter is relevant for our framework. Looijen (2004) links the state in which an IS can find itself (use, change and exploit) to task areas and assigns them to one of the three types of management: functional management, application management and infrastructure management as depicted in Figure 8 (Looijen 2004).

Figure 8: Linking IS states to management areas (Looijen 2004)

Since information systems, like ERP systems, are software systems (application) which run on hardware (infrastructure) and used to support business processes (functional) and since the software, hardware as well as the functionality of it has to be maintained, it can be concluded that ERP maintenance activities are performed in all these IT management areas (Figure 9).

(25)

16 Users and business processes Functional management Application management Infrastructure management ERP Maintenance activities Internal IT organization Business Organization

Figure 9: Organizing maintenance, adopted from (Meijer and Smalley 2006)

Combining the three IS management areas with the maintenance categories of section 3.1.1, Table 3 presents all the maintenance activities which are performed within an organization.

Table 3: Maintenance and use categories combined with IS management areas

Functional management Application management Infrastructure management

Corrective - All activities needed to

keep the ERP application running when problems occur

All activities needed to keep the infrastructure running when problems occur

Adaptive All activities needed to adjust the application to the business when changes occur in the business processes (new users, new procedures etcetera) or regulations

All activities needed to keep the application running when something has been changed

All activities needed to keep the infrastructure running when something has been changed

(26)

17 Perfective Changing the way ERP

is used to perform business operations in order to gain more benefits from it Changing the application parameters to improve performance Changing the infrastructure to improve performance

Preventive General periodic maintenance to

anticipate on changes in the business

environment

General periodic

maintenance to keep the application in good condition and anticipate on problems

General periodic

maintenance to keep the infrastructure in good condition and anticipate on problems

User support and training

Providing user support on functional level Training the users to work with the ERP module

Providing technical user support

-

Upgrade Activities needed to arrive at the functional design for the upgrade

Technical design and implementation of the upgrade Adjusting the infrastructure to support the upgrade Patch maintenance

All activities regarding the decision of implementing a vendor patch or parts of it Implementing a vendor patch or parts of it Adjusting the infrastructure to support the patches (if

necessary)

3.1.3 Cost dimensions

The costs of the use and maintenance activities, identified in the previous section, manifest themselves in different forms. When designing an instrument to measure the total costs of use and maintenance it is important to first identify how these costs manifest themselves. Literature on cost of ERP and packaged application maintenance, as well as literature on total cost of ownership (TCO) for such systems, is consulted to see how costs were measured in earlier studies. It was found out that there are basically three different cost dimensions, namely:

1. Products & services: all costs for which an organization receives an invoice and has to pay in monetary units (Gable et al. 1998; Somers and Nelson 2004)

2. People: cost of effort spent by the internal IT personnel on maintenance activities (Abts et al. 2000; Boehm 1981; David et al. 2002; Somers and Nelson 2004)

(27)

18

3. Processes: indirect costs at the business side as a result of downtime due to planned or unplanned ERP maintenance activities; this also includes time spent on training by end-users. (David et al. 2002)

3.2 Findings from ROC case study

The goal of the case study research was to find out:

11.How do organizations organize and execute ERP maintenance activities in practice? 12.How can one gather data on cost of maintenance activities in an organization?

The case study was conducted at the Regional Education Centrum (ROC) Eindhoven. First an intake interview took place with the head of application management to understand how maintenance is organized within ROC and to identify which people and roles were involved in conducting maintenance activities specifically for PeopleSoft HRMS. Then, depending on the specific function and responsibility within the organization, for each identified respondent a different list of questions was prepared to conduct the semi structured interviews. The maintenance activities as listed in Table 3 were taken as a basis to formulate the questions. Next to the above stated two research questions also two research questions regarding the degree of customization were built into the interviews but those findings are discussed in the next chapter.

3.2.1 ERP maintenance at ROC Eindhoven

HRMS maintenance activities are organized within ROC as depicted in Figure 10. All maintenance activities are divided over the three areas: functional management, technical application management and infrastructure management. The functional managers are dedicated to the HRMS package, within PeopleSoft technical management there is only one PeopleSoft developer which is responsible for changes in PeopleSoft HRMS as well as PeopleSoft Campus Solutions (CS) and the people within infrastructure management are not dedicated to any application at all. Since the latter makes it very difficult to measure the amount of effort spent on HRMS maintenance by Infrastructure management, it was decided to leave infrastructure (right column in Table 3) out of the framework.

The following roles within ROC were identified as people responsible for maintaining the HRMS package and thus interviewed during the case study:

- Change manager: responsible for managing all modifications to applications.

- PeopleSoft developer: responsible for technical design of the modifications to PeopleSoft HRMS and CS.

- PeopleSoft HRMS functional application mangers: responsible for providing functional support to HRMS users as well as performing functional maintenance activities such as training and functional design of the modifications to the HRMS package.

Maintenance activities to HRMS package are, with the exception of user support and training, performed through change requests. A change request can either by initiated by end users

(28)

19

because of changes at the business side or by the application management department because of changes at the IT side (corrective, adaptive, preventive and perfective maintenance from Table 3). For each change request first the functional managers make a functional design which is then built by the PeopleSoft developer and again tested by the functional managers before it is implemented by the database administrator (DBA).

Figure 10: HRMS maintenance at ROC Eindhoven

Table 4 presents an overview of the maintenance activities which are performed for the HRMS package. The table includes almost all the maintenance activities except patches since ROC does not actively implement official Oracle patches. Whenever necessary, parts from the official patches are implemented into the system through a change request.

(29)

20

Table 4: Overview of maintenance activities at ROC Eindhoven

PeopleSoft HRMS Functional application manager

PeopleSoft developer Database administrator (DBA) Modifications through change requests (adaptive, corrective, perfective and preventive maintenance)

Minor changes to the application Assessment of change requests Functional design of the modifications Testing the modifications built by the PeopleSoft developer Solving problems (bug fixes) Building the functional designs of the functional manager Testing the modifications Solving problems (bug fixes) Implementing the modifications

User support Providing user support whenever necessary

Training Training the users whenever new functionalities are added Upgrades and extension of used functionality

Functional design Technical design and testing

3.2.2 Collecting data on cost of maintenance activities

The first goal of the case study was to identify (or verify) which maintenance activities are performed from the ones mentioned in the literature and how these activities are organized. The second goal of the case study was to see how we can gather data on cost of maintenance.

(30)

21

During the case study analysis, it became clear that maintenance activities involving infrastructure were impossible to ascribe to a single application, such as HRMS package, since the hardware is mostly shared with other applications. Also the people responsible for infrastructure maintenance are not committed to a single application. Besides, the respondents agreed that infrastructure maintenance activities are unlikely to be affected by the amount of customization in the application software. Therefore, taking this in mind, infrastructure maintenance is left out of the framework for cost of use and maintenance.

For the remaining maintenance activities from Table 3 (the columns functional and application management), for each of the three cost dimensions products&services, people and processes it is identified which sources of information should be used. To gather data within an organization two kinds of sources can be distinguished: the knowledge available in the information systems (either or not automated) and the knowledge of people within the organization. Taking the maintenance activities and cost dimensions as a basis the following sources of information is identified in the case study analysis for measuring the cost of use and maintenance:

Information systems: Incident and Problem management system, change management, Time reporting system, procurement system, financial system.

People and roles within organization: Functional application manager, Technical application manager, application developer, Purchase/contract manger.

3.3 Measurement instrument for cost of use and maintenance

Combining the findings from literature and ROC case study and the three cost dimensions, Table 5 was constructed. The total cost for maintaining the PeopleSoft HRMS package can be obtained by evaluating for each maintenance category the cost along the dimensions Products & services, People and Process. The overview is basically a matrix with the use and maintenance activities which are taken into account along the vertical axis and the cost dimensions along its horizontal axis. To obtain a complete view of the total cost of use and maintenance of the ERP system, all of the boxes in the matrix should be evaluated.

(31)

22

Table 5: Costs of use and maintenance activities

Maintenance activity

Cost dimensions

Products & Services People Process Modifications Bug fixes User – support Training Upgrades Patch maintenance

Extension of used functionality

To measure the cost of maintenance activities along the three dimensions or in other words to fill in the matrix of Table 5, different sources of information might be used within an organization. On one hand there are information systems, such as a time reporting system or an incident or problem management system which can be used to extract the information from. On the other hand, at some organizations these information systems are not used in a proper way or not used at all. In those cases it might be more convenient to extract information from the people with the right role within the organization. Figure 11 depicts for each cost dimension, which sources of information can be used to gather data on cost of use and maintenance activities. The figure is further elaborated below in subsections 3.3.1 through 3.3.3.

(32)

23

Figure 11: Measuring cost of use and maintenance

3.3.1 Cost of people

The ‘people’ dimension measures the cost of time spent by the organizations own staff on maintenance and use activities. The information systems which can be used here are time reporting systems and incident or problem management system. In a time reporting system an organization can track employees time spent on activities and/or projects. Such a system can for example be consulted to find out how much time the application and functional managers have spent on maintenance activities such as patch maintenance, upgrade projects, training etcetera. An incident or problem management system on the other hand can be used to find out how much time is spent on bug fixes, user support and modifications which are applied to solve a critical problem (corrective maintenance). In case the above mentioned information systems do not provide the needed information, it can be extracted by interviewing functional and application managers.

3.3.2 Cost of products and services

To measure the cost of products and services one could look into the financial system or records of an organization since these costs always result into a bill or invoice which has to be paid by the organization and therefore there must be some kind of a record for it in those systems.

From the case studies, it was found out that, when abovementioned information systems are not sufficient, either the purchasing manager or someone from the finance department, who is

(33)

24

responsible for IT expenses, can also provide the needed information on cost of purchasing software, hardware and external services.

3.3.3 Cost of processes

The cost of processes or in other words cost of system downtime as a result of maintenance is compared to the previous two dimensions the most difficult one to estimate since it only concerns intangible and indirect costs. When looking at ERP systems in general, these costs are for example for a manufacturing system much easier to determine than for a human resource system. When a manufacturing system is down, for example the amount of units which cannot be produced due to downtime can be taken as a basis to estimate the cost of downtime. In case of an ERP system which is used to perform HR operations this is different. In contrast with a manufacturing system which is used to produce something with has a certain value, the value of the output of HR operations is difficult to quantify. Therefore in this research the cost of processes is defined as the cost of HR staff during the downtime periods. To measure this, the incident and problem management database can be analyzed for critical problems with the ERP package and the functional application manager can make an estimation of how many HR employees are affected by each problem.

3.4 Conclusion

In this chapter, findings from the literature review on ERP maintenance activities, how ERP maintenance typically is organized and how costs manifest themselves are combined with the findings from a case study to develop the framework for measuring the costs of ERP maintenance.

The framework basically answers two questions: what to measure and how to measure it? The what part is mainly answered by the literature review by focusing on which activities are important and which forms of cost dimensions we should take into account and the how part is answered by investigating within an actual organization how the answers to the what questions can be obtained.

(34)

25

4 Framework for measuring the degree of customization

(DOC)

In this chapter the framework for measuring the degree of customization (DOC) is presented. First, in section 4.1 the findings from the literature review are discussed. Then the design process which is followed to arrive at the DOC measurement framework is explained in detail in section 4.2. In section 4.3 it is explained how the instrument can be used in practice and section 4.4 concludes the chapter.

4.1 Findings from literature

The following research questions were formulated before conducting the literature study: 4. How can an ERP package be customized?  4.1.1

5. What are the implications of customizations on (cost of) maintenance?  4.1.2 6. How can the degree of customization be quantified?  4.1.3

Sections 4.1.1 through 4.1.3 provide the answers to the above stated research questions. 4.1.1 Customizing ERP packages

ERP package customization is a broad term which includes all modifications, not supported by the vendor, to the package or its functionality (Brehm et al. 2001; Gartner Research 2009; Light 2001). ‘Modifications to the package’ refers to changing the software itself whereas ‘modifications to its functionality’ also includes building software around the package, which changes the functionality of the package compared to how it was originally designed by the vendor. ERP customization can be classified in various ways. There are thus different ways to customize an ERP package. Several attempts are made to classify all customization options. Table 6 provides an overview of three different taxonomies: one from academic literature (Brehm et al. 2001), one from non-academic literature (Gartner Research 2009) and the taxonomy which is used by Accenture consultants in order to be able to systematically cover client’s requirements during implementation projects. For the exact definition of the customization classes we refer to the mentioned references and Table 7 for the Accenture methodology.

When looking at the three different taxonomies, similarities as well as differences can be noticed. For example, all three mention Reporting as a customization class, but the Gartner and Accenture taxonomy uses the term Extensions for what Brehm et al. (2001) refer to as Bolt-ons. Despite the fact that all three taxonomies are based on different terminology and reasoning, all three still cover all the available customization possibilities. We believe that there is no one correct way of classifying ERP customization, as long as the classes cover all the possible types of customization and leave nothing out and for each class it is clearly defined which types of customizations belong to that specific class. All three taxonomies meet these criteria and thus can be used to develop the DOC measurement framework. However, since the framework is designed to be used within the Accenture methodology, it was convenient to use Accenture’s customization taxonomy since both of them use the same terminology.

(35)

26

Table 6: ERP customization taxonomies

Brehm et al. (2001) Gartner Research (2009) Accenture Methodology Bolt-ons Screen masks Extended reporting Workflow programming User exits ERP programming Interface development Package code modification

Reporting extensions Templating Extensions Code Exits Vendor-permitted customization Full source-code customization Reports Interfaces Conversions Extensions Forms Workflow 4.1.2 Customization implications

Where the misfit between standard ERP packages and organizations needs is addressed extensively in the literature (Hong and Kim 2002; Soh et al. 2000; Wagner and Newell 2004; Wei et al. 2005), the implications of resolving the misfits by customizing the ERP package needs attention. Light (2001), for example, has investigated customization implications using two case studies. He found out that the customizations would require extra attention during upgrades and the associated effort can vary dramatically. Besides, since the customizations are custom made, ongoing maintenance would be necessary outside any upgrade implications (Light 2001).

4.1.3 Quantification of the degree of customization (DOC)

The goal of the third research question addressed by the literature review was to find out how the degree of customization can be quantified. Customizing an ERP package basically means modifying or building software to meet the specific needs of the adopting organization. Thus the DOC is nothing more than the degree of software complexity. There are numerous metrics suggested by the literature for software complexity such as lines of code, alternative paths loops, number of unique operators, development time etc; see Zuse (1990) for an extensive survey of software complexity metrics. To choose a metric to quantify the degree of customization, first we need to define which criteria are important and should be addressed by the metric. This is done in section 4.2, where the design process is explained in more detail.

4.2 Developing the DOC measurement framework

(36)

27 - Which metric can we use to quantify the DOC? - How can we estimate the DOC in practice?

The first question is addressed in section 4.2.1 and the second one in section 4.2.2, where the findings from the ROC case study are discussed. The actual design steps which are followed are thoroughly discussed in section 4.2.3.

4.2.1 Development time as a metric to quantify the DOC

To find a suitable metric to quantify the DOC we have taken the following three criteria in mind. First, the metric should be representative for the amount and complexity of the modifications taking in mind the effect of those modifications on use and maintenance effort. Second criterion was that the metric should be easy to measure in practice, meaning that it should be possible to measure it without having extensive knowledge of the system’s current configuration. Third and most important criterion was that it should fit within the existing Accenture methodology. This is important because the ultimate goal of the instrument is to gather data which can be used to develop a tool for estimating the use and maintenance cost related to a certain degree of customization. For Accenture consultants it would be convenient if this tool uses the same metric for customization complexity as the already existing methodology. This relation is illustrated in the figure below.

Figure 12: Relation between the existing methodology and the to-be developed tool

The Accenture methodology estimates the customization complexity in terms of expected amount of effort in development time to design and build the modifications based on systems specifications. To meet the third criterion, we need to use the amount of development time as the metric for customization complexity for the measurement instrument as well. Since software development time is directly related to expected maintenance effort (Hybertson et al. 1997; Vigder and Kark 2006) and can be measured in practice without having extensive knowledge of the system itself, also the first and second criteria are met. The total DOC would thus be the total sum of the development effort for all the modified objects in the system.

4.2.2 Findings from ROC case study

Taking the development time as a metric to quantify the DOC, we now need to know how we can measure this in practice. To find this out, the findings from the case study research at ROC Eindhoven is used. The following research questions were formulated:

13.How do organizations keep track of the number and type of customizations in their ERP package?

14.How can one gather data on the degree of customization of an ERP package used in an organization?

(37)

28

To find the answers on the above stated questions, the following three people were interviewed within ROC Eindhoven:

- Change manager

- PeopleSoft developer/ technical application manager - PeopleSoft functional application manager

All three were asked whether ROC keeps track of:

- the customizations in their PeopleSoft HRMS package, and

- the amount of effort spent on the development of those customizations.

The answer to both questions was no. There were no configuration databases or any other records of any kind, in which we could find what exactly was customized and how much effort that has cost ROC. This lack of documentation resulted into the decision to design the framework in such way that the DOC can be quantified based on the judgment of the people within the organization who are responsible for maintaining the specific ERP package, which was in our case PeopleSoft HRMS.

At ROC Eindhoven, the people who are responsible for maintaining the PeopleSoft HRMS package are the same people as the ones we have interviewed. To make clear what was meant by customizations, the taxonomy which is used within Accenture Methodology for ERP customization, as presented in Table 7, was discussed with the interviewees.

Table 7: Accenture’s ERP customization taxonomy

Customization type Definition

Reports A listing or output of information in the system. Although there are

typically standard reports delivered that come with the packaged software, the term report is used in this methodology to refer to custom developed reports. These reports are custom designed and built to fulfill specific requirements.

Interfaces An interface defines the data and operations of an application or

component that uses it to interact with internal or external applications/components.

Conversions The conversion of data from the current format to the structure required

by the new application. A conversion can be performed via an automated program or can be completed manually.

Extensions A modification to the functionality of the packaged software application,

(38)

29

modifying the code of the packaged software.

Forms A customized output that is developed to meet a specific requirement. An

example of a Form is a customized purchase order form that will be printed.

Workflow The sequential flow of tasks and information in a business process.

When customizing the PeopleSoft HRMS package, specific PeopleSoft object types are modified (Landres et al. 2000). Thus next to the ERP customization types of Table 7, also an overview of all specific PeopleSoft object types, as presented in Figure 13, was discussed with the interviewees to find out whether they are able to estimate the DOC by assessing for each PeopleSoft object types whether there are any which are customized. For the exact definition of each object type and how these are modified, specific literature on modifying PeopleSoft objects were consulted (Landres et al. 2000).

Figure 13: PeopleSoft objects

Table 8 summarizes the important findings from the case study analysis. In the table we can see how the respondents were able to estimate the DOC using Table 7 and Figure 13.