CHAPTER 5
ERP PROJECT LIFECYCLE –KEY PLANNING PHASE
ISSUES
5.1 INTRODUCTION
The research framework described in chapter 3, uses the Parr and Shanks (2000) PPM model in Figure 3.1(a) and identifies planning, implementation and enhancement as the three important phases in the lifecycle of an ERP project. This chapter identifies some of the critical issues involved in the planning phase and their influence on the final outcome of the project. The key issues involved during this phase are (a) scope finalization (b) ERP selection decision, (b) implementation partner selection (d) core team selection and (d) steering committee formation. This chapter addresses the first three of these which are the most crucial at this stage. Each of these add to complexity and impact eventual project outcome.
Complexity of ERP projects has its genesis in the dynamics of the implementation process. This is explored in this chapter using system dynamics (SD) tools. The core paradigm behind SD is that the eventual path a system takes is guided by the way in which elements of the system interact with each other. This interaction results in patterns of system behavior that may be counter intuitive and difficult to determine through a linear thought process. An attempt is made to explore some of the critical dynamics and draw inferences to guide project management practice.
This chapter illustrates the use of Analytical Hierarchy Process (AHP) for the ERP selection decision. An example of one such exercise and
problems faced are described. Suggestions for improving the process and extending it to other selection decisions are also given.
This chapter also suggests an improved methodology for scope development and goal setting. An approach called the “Single Goal Set” (SGS) method is explained. This method is expected to greatly improve scope development and thereby positively impact the final outcome of an ERP project.
5.2 THE DYNAMICS OF ERP IMPLEMENTATION
ERP implementation issues arise from the inherent complexity of the ERP artifact in the context of its implementation. In Chapter 1 the concept of “reciprocal dependencies” (Williams 2004) has been introduced. This refers to the complexity that arises due to each element of the process being an output or an input or both of another process. Such complexity makes it difficult to assess how changing one element would affect another element separated in time or geography. It becomes impossible to assess how the overall system would react to a change in one or the other of the causal factors because of the intertwined relationships between them. Sometimes the system response to an act could be contrary to what one would intuitively assume it to be.
An example of one such counter intuitive effect , in software development, is what is known Brooks law which states that adding manpower to a late software project actually delays the project rather than speed it up (Brooks 1975). ERP projects, on account of reciprocal dependencies, also exhibit such behaviour which runs counter to intuitive wisdom.
One method that helps in the understanding of these complex relationships is System Dynamics (SD) a way of thinking that was pioneered by Forester (1961) and developed to include all manners of business problems
by Sterman ( 2000) and others. System Dynamics is “concerned with the behaviour of complex systems …..(and).. is grounded in the theory of non linear dynamics and feedback control in mathematics, physics and engineering” (Sterman 2000 pp.4-5). Abdel- Hamid and Madnick (1991) model a real life software project, the NASA-DE-A project using SD tools and simulate the occurrence of the Brook’s phenomena. They show that adding manpower does not always cause a negative outcome and that it depends on the stage of the project.
System Dynamics offers the method of causal loop diagrams (CLDs) an important and useful tool for representing the feedback structure of systems. Positive feedback loops that move the situation forward are known as reinforcing loops and negative feedback loops that hold a situation from moving forward are known as balancing loops. The basic premise of SD is that all systems are made up of a large number of reinforcing and balancing loops that together impact the final direction of the system. Carefully conceptualizing these loops and tracing their direction helps in the understanding of complex systems. Such tools are needed as without these the human mind is incapable of judging the movement of a complex system beyond the second or third interconnected loop. SD methods help in modeling large systems with innumerable loops.
Akkermans and Van Helden (2002) identify a two such loops and illustrate vicious and virtuous cycles in case study of an ERP implementation (Figure 5.1). The first reinforcing loop identified is the communications – collaboration loop. Communication reinforces collaboration –poor communication therefore inhibits collaboration. They opine that in the first instance communications were restricted to the consultants and a “core group” which resulted in weaker interdepartmental communication leading to poor interdepartmental coordination. This is a vicious cycle as mistrust increases inhibiting communication. Left unaddressed this would spiral out of control resulting in a failed project.
This was halted through a countermeasure of a change in the composition of the implementing team. Consultants who were purely technical, were replaced by trained colleagues who were able to communicate more effectively with users. This resulted in better communication and consequently better collaboration. The reinforcing loop was thus turned from a vicious one to a virtuous one. The second reinforcing loop identified by Akkermans and Van Helden (2002) is the expectations management loop. The researchers do not elaborate on this loop in their paper. We believe this is an important loop as it has ramifications across the implementation process.
Figure 5.1 illustrates the CLD developed by Akkermans and Van Helden.
Figure 5.1 Reinforcing loops in ERP implementation (Source: Akkermans and Van Helden 2000)
There are several important intertwined loops in the process of an ERP system getting implemented in an organization. CSF literature captures these as events that occur at points of time. Causal loop representation helps us to see the manner in which these CSFs interact and influence each other and impact the outcome of the project.
Figure 5.2 gives an example of some of these interrelationships. We identify some important loops: the expectations confirmation loop: This is a balancing loop or negative loop. All negative loops have a goal –or the desired state of the system (Sterman 2000 pp.155). The goal of this loop in our case is the project scope. If it is higher expectations would be more, the perceived gap in performance would be greater leading to dissatisfaction. This impacts the Usage loop: If satisfaction is high usage is more and tendency to revert to legacy systems is reduced. If satisfaction is low tendency to revert to legacy systems is high and efforts to correct the system deficiencies is low. So one sees the impact of this in both the corrective actions as well as in the legacy systems influence loops. We also identify a cost loop. This is a balancing loop with the budgeted cost as the system goal. With increased top management support , additional resources could be sanctioned to close gaps . This leads to a cost overrun which negatively impacts top management support. Top management support also has influence on the use of legacy. With greater top management support and advocacy for the ERP project the tendency to revert to legacy systems is diminished.
In summary we see that SD offers us an effective tool to translate our mental maps into CLDs. This facilitates a better understanding of how interactions between factors, events and goals finally impact outcome. SD has another central idea called the stock and flow structure and a programming
methodology that allows us to simulate the effect of changes in parameters on system response. Stocks are accumulations and flows are what cause accumulation or attrition of stock.
In this case, Satisfaction, Top Management Support , etc., are stocks that keep changing on the basis of the flow of information regarding goals attainment, modules completion , cost overrun , etc. There are inflows as well as outflows- if gaps are reduced this information increases the stock of top management support for the project. If costs increases, this information depletes the stock of top management support. This powerful methodology helps us model the impact of changes on key stocks.
When we started out this research we hoped to be able to model the implementation process using SD and use the model to test the PVI scale. We did not succeed in our attempt as we were not able to resolve the issue of units of measure that could be used to model the implementation. SD requires that there is unity of measure across the system. For example if Satisfaction was the stock that we were tracking, what would be the flow – obviously Satisfaction /unit time. How do we then model each of the causal factors to yield the influence on this flow. We found this task daunting and restricted the use of SD tools to understand the dynamics of the implementation rather than model the whole process. We found that other researchers also followed the same – not modeling the whole process but using SD analysis to understand the dynamics . A case in point is the Akkermans and Van Helden study (2002) that studies the virtuous and vicious cycles in ERP implementations.
Our objective in this chapter is to highlight the essential complexity of an ERP implementation process.
111 Project deadline goal Actual Gap + Tendency to revert to legacy + Top Management support -legacy system influence
-Corrective actions loop
Legay Systems Influence Loop
Cost Overrun -Cost Loop Perceived gap Satisfaction -Expectations + Efforts to Close Gap -Additional Resources Sanctioned + + PROJECT SCOPE + + BUDGETED COST + Expectations - Confirmation Loop
+
-Usage Loop
5.3 ERP SCOPE FINALIZATION
We have seen that project scope is an important element that contributes to the dynamics of an implementation. One of the key activities during the planning stage of an ERP project is scope finalization. The eventual success of an implementation is to a large extent dependent on the effectiveness of this process.
A lower scope translates into an easier project deadline, which results in a quicker implementation, lower costs and resultant support from all. This impacts the expectations-confirmations loop, is one of the key balancing loops in an ERP implementation project. A lower gap between expectations and actual also reduces the tendency of users to resist the ERP and revert to the legacy processes. This in turn positively impacts the efforts to close whatever expectation gaps that may exist.
Scope finalization therefore becomes important. Right definition of the scope has been identified by several researchers as an important CSF for ERP success as is shown in Chapter 2 , Tables 2.2 (a) and (b). The case study conducted as part of this research and described in Chapter 4 also showed that the initial scope of the ERP project has a strong relationship to the success of the project. In the case of ERP 1 the scope was too ambitious ; in ERP 2 the scope was very simple and it led to a successful implementation.
The standard methodology for finalization of project scope begins with defining the business vision of the enterprise. Following this the key information needs for meeting this business goal are identified. A gap analysis is conducted to ascertain the current status of the MIS in the company and the existing gaps with reference to the information needs. Normally in organizations this process is followed by a detailed requirements gathering exercise.
5.3.1 Scope Finalization - Traditional Methods
Traditionally scope finalization in ERP projects draws its methods from requirements engineering (RE) used in software development. This involves identifying stakeholders, eliciting their needs and then documenting these in a form that is communicable for subsequent implementation. While the ERP scope finalization is similar, there are key differences that must be noted. A major difference in the case of ERP is that the objective of the process is to understand the business needs of the organization so as to “fit” the standard processes of an ERP rather than to create a module to meet these needs as in the case of software development.
Requirements engineering has some important sub-processes. These are: (1) requirements elicitation, involving the gathering of user/ process needs, understanding the current and proposed work flows and documenting the work/ tasks that need to be performed; (2) requirements agreements involving the process of consensus building when there are conflicting requirements and (3) requirements communications which involves wide communications of the finally agreed requirements or goal of the ERP being implemented (Nuseibeh and Easterbrook, 2000). The final project goal or scope document is the output of this process (Figure 5.3).
An important area of requirements elicitation is identifying MIS reports needed by users. In the case of an ERP the aim is to identify those reports that are available in the ERP package and ascertaining the acceptance of these by the users. The process also elicits new report requirements.
An area of concern in ERP implementations is the extent of customization that results from new work flows and new reports demanded by users (often the standard report writer in an ERP suite cannot accommodate all report formats and customization that may be needed). This often results in time and cost overruns and is often the cause for ERP project failures. One of the problems in the standard method of requirements elicitation is that when asked users tend to give a lot of requirements and reports –whether really needed or not. Users see this as an opportunity to ask for their complete “wish lists”. This results in a large requirements list, often not available in the standard list of modules /reports of the ERP, requiring several man months of work to deliver.
To illustrate a case in point: in an ERP implementation where the researcher was personally involved the RE phase identified 55 new reports for the materials module (note: 200 odd standard reports were already available in the module). The implementing team spent 6 additional man months to develop these. After three years during an audit (prior to a version change exercise) it was found that out of the 255 (200 standard reports plus 55 new reports made) that were made available only 10 to 15 reports were being used regularly, another 5 to 6 were used sparingly and the balance not used at all. This highlights the need to be very careful in agreeing to user requests –often these are of arguable utility and contribute to the phenomena of “scope creep”.
In conclusion the current system needs a relook. It is important to have some means of prioritizing requirements and ensuring that only the
most essential is taken up during implementation. Cost and time overruns are often the result of unrealistic over ambitious goals, cumbersome work arounds and unnecessary reports. The following section suggests a new paradigm for defining the scope of an ERP implementation.
5.3.2 Scope Finalization – A New Paradigm
In the earlier section we have identified that inappropriately defined scope can adversely affect the final outcome of an ERP project. We also showed that the current system of requirements gathering could result in a very large and often unachievable project scope which would lead to negative project outcome. Therefore there is a need to look at a different method of scope finalization.
We looked at the other mega projects to see if there could be lessons learnt. A typical mega project is a civil project like the construction of a bridge. Or consider project like a mountaineering expedition to conquer Mount Everest. These are mega projects with multiple linkages, complex tasks to be carried out and different stakeholder expectations. Yet a majority is considered successful – despite slippages.
Did Mount Everest get conquered? The answer is –yes despite major “overruns” on the way –of time, money and lives. Then why was this mega project involving a challenging goal, large teams, several intermediate milestones and unpredictable extraneous circumstances considered successful despite intermediate failures. We would like to postulate that this is because the super ordinate goal ofreaching the peak was achieved
Let us take another mega project - building the Concorde. Was there a slippage of time, cost and non-achievement of several expectations? Without doubt … volumes have been written about all of these. But was the
project “successful” again, without doubt - yes. The grace of the Concorde in flight removes all doubt from the mind.
What then has made these mega projects successful even though there have been slippages in deliverables along the way? The answer probably is that the achievement of the overarching goal of “scaling Everest” or the “Concorde flying” negates all else. It does not matter how much cost overrun there was, or what was not achieved, or even how many lives were lost in the attempt. The achievement of the single overarching goal gives the project the label ofsuccessful. Subsequent milestones are achieved easily.
This does not mean that cost or time overruns are not important. Once the key overarching goal is achieved, the project is successful in the minds of most stake holders. The team approaches the subsequent stages of the project with a positive success led mindset. All other subordinate goals then get achieved in the subsequent iterations. Thus all goals get achieved in time.
Our study of several ERP project implementations suggests that a major cause of project failures is “unmet expectations” due to unrealistic goals. This is vindicated by the findings of the original The Chaos Report of the Standish Corporation (Standish 2004). In this report, “Clear statement of requirements”, “Clear vision and objectives”, “Realistic expectations” and “Ownership” together account for nearly 30% of the factors of success of projects. All these factors, collectively, represent user expectations, which need to be met.
Based on this insight a new approach for goal finalization is suggested. The approach would focus on a single overarching goal for the ERP project –a goal that is understood and appreciated and “owned” by all the entities in an
organization. The suggested methodology is christened as the Single Goal Implementation Method (SGIM) (Venugopal 2005).
While ideally this methodology requires one overarching goal for the whole enterprise, in practice it may not be really be possible to have or obtain consensus for only one single goal. There are several systems and sub-systems in an organization. Therefore, while keeping to the spirit of the methodology, a refinement in the single goal concept may be necessary. In practice, therefore, the methodology would be deployed as follows:
The core process of the enterprise is identified. A Single goal (which could be called the Summit goal) for this process is determined. In addition, two or three more vital processes are also defined and goals for these are also identified. The Summit Goal and the two or three vital process goals become the Single Goal Set (SGS) for the ERP project implementation (Figure 5.4).
This set of goals, the single goal set (SGS), serves as the centering force for the project –like a gyroscope, always keeping the project from “spiraling” out of control. The implementation of the project would be iterative. The first iteration would be aimed at achieving the SGS. Subsequent iterations would seek to achieve the other subordinate goals. The process therefore incorporates the agility and simplicity of the “agile methods”, while ensuring an overall discipline of goal orientation as is the essence of the planned methodologies. It is agile in that it is simple and iterative. It is disciplined, in that, there is an overarching goal for the implementation understood and accepted by all. The steps in the implementation would consist of the following:
1. The core process of an organization is identified.
2. The other Key processes (ideally not more than two more) are identified.
3. The team searches for a Single Goal for the core process (The Summit Goal) and goals for each of key processes. (Ideally there should be not more than three goals for the entire implementation). These become the SGS for the implementation program.
4. Other Subordinate Goals are identified. (these are for the other processes and are not focused on in the first iteration of the implementation)
5. The project champion and the core team intensely work on building consensus around this set of goals for the project. This is the key change management task of the implementation.
6. The internal communication in the organization stresses that the IT project is only for achieving stated Singe Goal Set (SGS).
7. The implementation team focuses only on achievement of SGS goals – subordinate goals are not focused on at all in the first iteration.
8. When the SGS have been achieved the project is declared successful.
9. The gaps in the subordinate goals are identified and the organizational impact assessed.
10. The subsequent iterations are carried out to bridge gaps and achieve subordinate goals.
This revised approach to ERP goal setting is expected to improve the perception of the users regarding the success of the project as the publicized key goals of the project are achieved. As a consequence the biggest problem of implementations, i.e., “managing user expectations” is taken care of. This would result in an overall “feel good” factor about the project, which would give added impetus to the subsequent iterations to complete the other goals.
There could be an argument that the gains from such an approach are only psychological and that the organization would finally wake up to the fact that the only gains were a false “feel good” factor? This is not so because this methodology essentially optimizes resource allocation. Time, money, and efforts are all first spent on realizing the most important goals of the business. Only after these are achieved are any of the subordinate goals taken up for implementation. In the traditional methodology often, this sharp focus is lacking, with the result that by the end of the project the budget gets spent
with none of the goals being “fully” met leading to dissatisfaction. Any further budget allocated also gets drawn reluctantly into the sinking pit that is the project without any sign of the organization being able to come out of it. This is classic project “escalation” described in Chapter 1.
Is the approach too simplistic? Can there really be one Summit Goal for the entire organization. We believe it is possible to find such a goal. In fact most organizations do have one powerful driver (or goal) that led them to the ERP initiative in the first place as evidenced by our research. During the scoping process, deluged by the large set of benefits promised by the implementation partners and ERP vendors this goal gets diffused. The expectations of the company are raised to impossible levels which are generally not met leading to dissatisfaction.
To validate this a survey was carried out among five very diverse companies, who had recently implemented ERP projects. A few key users were contacted in each company and asked just one question “ What was the main driver for the company to go for the ERP?” We found that each company had a specific driver for ERP. In one case, it was to integrate with the parent company’s ERP after a merger (the parent was using SAP while the unit was on Oracle Applications) . In another case it was driven by the fact that the present legacy system’s RDBMS was going out of support (the legacy system was on Ingres and this was going out of support in a year’s time). In yet another case the prime driver was the need to conform to the new accounting standards of book closure within 2 days of the last day of every month (a consistent internal audit comment to the board was that the account finalization was taking too much time –sometimes as much as 3 months. The board had mandated that this be corrected). In another case the prime need was to integrate the remote offices to get real time inventory information. (a management audit had shown that the inventory turn-round ratio was adverse
- 6 versus the desired level of 10; unaccounted inventory in the remote locations was seen as the culprit).
It is clear from the above that companies had an overriding reason, or overarching goal, for venturing on a large IT project like an ERP implementation. The core of the suggested methodology is to capture this goal and keep it in the forefront of the implementation as a Summit Goal.
A criticism of this methodology could be that this would lead to sub-optimization. For example for meeting the SGS additional and redundant data capture may be needed; or it may be decided to continue with a legacy application for a sub-process. All these are sub optimizations. This criticism is justified as there is some sub optimization happening in this method. However we believe that these are planned sub-optimizations - something akin to planned redundancy creation in a normalized table to achieve higher retrieval efficiency. The crux is that the most important drivers for the implementation are met and therefore the business needs are not compromised. Any “local” sub optimization is acceptable if true global optimization is achieved by meeting the most pressing needs of the enterprise.
In an actual implementation situation all such “sub-optimizations” should be tabled and their impact on the key deliverables of cost, quality and delivery critically examined. If justified, from a cost-benefit perspective, they should be taken up for resolution through work-arounds or customizations as is appropriate.
5.4 ERP SELECTION
ERP selection is an important decision during the planning phase of an implementation. Several researchers have identified ERP selection as an important CSF (Al Mashari et al 2003, Hong and Kim 2002, Somers and Nelson 2001, Umble et al 2004). Various methods have been used for this the
most common being the scoring and ranking method (Lucas and Moore 1976). This is simple and easy to use. However it is fraught with error as weights are generally assigned arbitrarily often based on practitioner wisdom. There is the danger that, lacking a logical basis for assigning weights, these could be adjusted to “force” decisions to favour one or the other choice. The method is seemingly scientific when in reality it is not.
Mathematical optimization and multi-criteria decision analysis such as goal programming, and nonlinear programming have been applied for software selection (Santhanam and Kyparisis 1995, 1996). A combination of analytic network process (ANP) and a 0–1 goal-programming model has been tried for IS project selection (Lee and Kim 2000). However practical applicability of these methods outside of a research setting is doubtful. This is because it becomes difficult for line managers to appreciate the methods and take time out to participate in the process. Moreover several of the attributes are not easily quantifiable and attempts to do so sometimes only adds “noise” to the system.
In summary the simpler methods like scoring and ranking are too simplistic and mathematical optimization too complex to be of practical use. A technique that uses the intuitive simplicity of ranking with the rigor of mathematical optimization is the Analytical Hierarchy Process (AHP) introduced by Saaty (2008). In the following section this process is used to select between various ERP packages based on a systematic evaluation of decision criteria by experts.
5.4.1 The Analytic Hierarchy Process (AHP)
AHP is designed to solve complex multi criteria decision problems. It involves structuring multiple choice criteria into a hierarchy, assessing the relative importance of these criteria, comparing alternatives for each, and
determining an overall ranking of the alternatives. The output of AHP is a prioritized ranking of alternatives based on overall preferences expressed by decision makers. AHP helps capture both subjective and objective evaluation measures.
AHP is useful when decision-making is complex and unstructured. AHP splits the overall problem into smaller evaluations between multiple alternatives and finally through its methodology combines these to provide the global decision. It also provides a mechanism for checking consistency of the evaluation responses.
Since its introduction, AHP has been used in myriad situations and is one of the most widely used multiple criteria decision-making (MCDM) tools. These include applications of AHP in planning, selecting a best alternative, resource allocations, resolving conflict, optimization etc.
The AHP method involves three phases: decomposition, comparative judgments and synthesis of priorities (Wei et al 2005). Decomposition involves developing the fundamental-objective hierarchy which clearly outlines key objectives that decision makers seek to optimize and breaks the same down to its attributes. Comparative judgments involve assigning of priorities to attributes through the pair-wise comparison method of AHP using a nine point scale. The final stage involves use of the AHP algorithm to develop the priority matrix. Finally mathematically obtained priorities are checked for consistency using the consistency testing methodology developed by Saaty (1980).
5.4.2 Procedure for ERP Selection
The following stepwise procedure has already been referred to in Chapter 3 – we repeat the same in this chapter for ease of reference (Figure 5.5).
Figure 5.5 AHP Methodology
5.4.3 Application of AHP Methodology for ERP Selection
The first step in applying the AHP methodology is developing primary secondary and tertiary hierarchies of decision criteria. The AHP methodology is applied to arrive at priority matrices for each criteria set. These are then combined to obtain the priority for the final decision. Figure 5.6 gives a two level criteria set that could be used to choose between competing ERPs.
Figure 5.6 Objective hierarchies –ERP selection
This process is to be carried out iteratively with the relevant management team of the company for each set of criteria in the hierarchy. While the actual process to develop this can be complex, as part of this research, a simple adaptation this methodology was tried out in a unit proposing to implement an ERP. The results are presented in the following sections.
The firm chosen was a medium scale manufacturing unit proposing to implement an ERP system. The choice was between alternative ERP
packages, Oracle Applications SAP R/3, J.D. Edwards and RAMCO Marshal. The evaluation team consisted of the IT head, the finance head who was leading the ERP team, two functional users and an ERP consultant. The following criteria for ERP selection used by Wei et al (2005) was considered appropriate and adopted.
Table 5.1 Evaluation criteria
Criteria Label Factors to be considered by the respondent*
Price Maintenance cost Consultant expense Training cost Total cost TC Implantation cost 6 – 9 months Implementation time IT
Project management ability Module completion Function – fitness Functionality Fu Security Ease of operation User – friendliness UF Ease of learning Upgrade ability Ease of integration Flexibility Fl
Ease on in-house development Stability
Reliability R
Recovery ability
Customization C Minimum customization
*Note: The factors mentioned above were provided to the respondent to help them understand our definition of the criteria being compared. They are not secondary criteria.
In a focus group setting the evaluation team was asked to do pair wise comparisons between two of the criteria taken at a time. Criteria, relevant to the firm in question, to be considered for making the comparison were first arrived at by the group. Factors that define the criteria were also arrived at. This is to make sure that all respondents understand the criteria in the same manner. Comparison was made using the 9 point Saaty (1980) scale given in Table 5.2.
Table 5.2 The Saaty rating scale
Intensity of
importance Definition Explanation
1 Equal importance Two factors contribute equally to the objective 3 Somewhat more
important
Experience and judgment slightly favour one over the other.
5 Much more
important
Experience and judgment strongly favour one over the other.
7 Very much more important
Experience and judgment very strongly favour one over the other. Its importance is
demonstrated in practice. 9 Absolutely more
important.
The evidence favoring one over the other is of the highest possible validity.
2,4,6,8 Intermediate values
When compromise is needed
(Source: Saaty 2008)
Each criterion under evaluation is to be rated against every other criterion one pair taken at a time. The relative importance of one criterion over the other is marked indicating the degree of importance using the modified Saaty scale Table 5.3. For example if the criterion Total Cost is considered “somewhat more important” than say User Friendliness the number 3 on the left of the scale would be marked as shown in Table 5.3. If Implementation Time on the other hand is “equally important” as Total cost then 1 on the right of the scale would be marked on the relevant row. The scale thus makes the marking quite easy.
The rating was carried out collectively for all the seven selection criteria identified in Table 5.1. The process took time as the team struggled to ensure that there was logical consistency in rating. It is sometimes difficult to ensure this: for example if A >B and B> C then A has to be >C. We used a focus group approach to arrive at group consensus. Saaty suggests use of the geometric mean of individual ratings. However as suggested by an examiner of this thesis, group decision aggregation methods could have been attempted. The AHP method also gives a method to test this consistency – this is explained in the following sections.
Interdependence of criteria – a discussion
In MCDM techniques interdependence of variables is considered a problem. This is especially so in techniques like regression where the primary interest of the researcher is in knowing the effect of unit change in an independent variable on the dependent variable. In a selection technique like AHP the primary interest of the researcher is in using the criteria to choose between alternatives – so interdependence of criteria if any poses less of a problem as long as there are no complex interrelationships between alternatives and as long as AHP algorithm per se is not affected by the presence of some interdependence.
Furthermore in social sciences research (as opposed to the natural sciences) some amount of interdependence is inevitable. Tests like discriminant analysis help identify such interdependence and techniques like factor analysis can be used to identify and bring interdependent variables under one group.
In the case of AHP the issue is one of selection. If criteria are indeed interdependent, the Saaty scale would bring out this interdependence. For example if criteria A and criteria C are highly interdependent , then the respondent would rate them as of “ equal importance” and give it a value of 1 on the Saaty scale.
The issue of interdependence in selection can also be handled using another more general form of the AHP technique - the Analytical Network Process(ANP). In the AHP, each element in the hierarchy is considered to be independent of all the others—the decision criteria are considered to be independent of one another, and the alternatives are considered to be independent of the decision criteria and of each other. ANP
does not require independence among elements, so it can be used as an effective tool in these cases.
To illustrate this, consider the decision of choosing between ERP packages . The decision maker wants to base his decision on only three factors: Cost, Functionality and Customization needs. Both the AHP and ANP can be used to provide useful frameworks to use in making his decision. The AHP as we have seen assumes independence of these criteria. However it can be argued that Cost and Functionality are not truly independent. Also higher functionality may mean lesser customization hence these are also not independent. ANP would allow consideration of these interdependences.
While AHP considers the decision making to be a hierarchical process , top-down from the goal- ANP models it as one that is cyclical with each node having a relationship with another node.
The steps for gathering the responses are the same using the Saaty scale or any other appropriate means to obtain pair wise comparisons. The computation is slightly different. An illustration of the ANP method is provided in Appendix 7.
130
S.No. Factor Rating Factor
1 Total Cost 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Customization
2 Total Cost 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Functionality
3 Total Cost 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 User-friendliness
4 Total Cost 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Flexibility
5 Total Cost 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Reliability
6 Total Cost 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Imp. Time
7 Customization 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Functionality
8 Customization 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 User-friendliness
9 Customization 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Flexibility
10 Customization 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Reliability
11 Customization 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Imp. Time
12 Functionality 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 User-friendliness
13 Functionality 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Flexibility
14 Functionality 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Reliability
15 Functionality 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Imp. Time
16 User-friendliness 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Flexibility
17 User-friendliness 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Reliability
18 User-friendliness 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Implementation Time
19 Flexibility 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Reliability
20 Flexibility 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 Imp. Time
The next step in the AHP process is creation of the reciprocal matrix based on the ratings. For example in the cases above against Cost V/s User Friendliness a number 5 would be recorded and against User Friendliness versus Cost the reciprocal 1/5 would be recorded. In this manner the reciprocal matrix is created. The actual reciprocal matrix generated for the case under evaluation is given in Table 5.4 (detailed AHP workings are in Appendix 6).
Table 5.4 Reciprocal matrix
TC C Fu UF Fl R IT TC 1.0000 5.0000 5.0000 3.0000 2.0000 3.0000 1.0000 C 0.2000 1.0000 3.0000 0.1667 1.0000 0.2500 2.0000 Fu 0.2000 0.3333 1.0000 4.0000 2.0000 1.0000 3.0000 UF 0.3333 6.0000 0.2500 1.0000 3.0000 1.0000 3.0000 Fl 0.5000 1.0000 0.5000 0.3333 1.0000 1.0000 3.0000 R 0.3333 4.0000 1.0000 1.0000 1.0000 1.0000 2.0000 IT 1.0000 0.5000 0.3333 0.3333 0.3333 0.5000 1.0000 SUM 3.567 17.833 11.083 9.833 10.333 7.750 15.000
The next step in the AHP methodology is creation of the normalized pair wise comparison matrix. This is done by dividing each element in the matrix by its column sum. The eigen vector of each row is computed. A common method is to take the nth root of the product of items in a row where the number of items in a row is n.
Table 5.5 Normalized matrix
TC C F UF F R IT Eigen Vector* Priori
ty TC 0.280 0.280 0.451 0.305 0.194 0.387 0.067 0.246 0.31 C 0.056 0.056 0.271 0.017 0.097 0.032 0.133 0.067 0.08 Fu 0.056 0.019 0.090 0.407 0.194 0.129 0.200 0.110 0.14 UF 0.093 0.336 0.023 0.102 0.290 0.129 0.200 0.127 0.16 Fl 0.140 0.056 0.045 0.034 0.097 0.129 0.200 0.084 0.10 R 0.093 0.224 0.090 0.102 0.097 0.129 0.133 0.118 0.15 IT 0.280 0.028 0.030 0.034 0.032 0.065 0.067 0.053 0.07 Sum 1.000 1.000 1.000 1.000 1.000 1.000 1.000 0.805 1.00
*Note: The Eigen vector can be approximated by several methods – one of the methods is taking the geometric mean (nth root of the product of cell items) and normalized by dividing by sum of the roots (Coyle 2004) For example for the first row TC, the geometric mean is given by (0.280x 0.280 x 0.451 x 0.305 x 0.194 x0.387 x 0.067) 1/7 = 0.246. Similarly the means are computed for each of the rows. When normalized by dividing by the sum of roots 0.805 the first element of the relative Priority Vector, 0.31 is obtained.
The priority vector is the relative importance of the criteria in the ERP selection decision. To illustrate in the case above Total Cost (TC) the importance is 0.31 (31%) as compared to Functionality (FU) which has an importance of 0.132 (13%). The sum of all priorities would total to 1.
The final stage is to calculate a Consistency Ratio (CR) to measure how consistent the judgments are relative to a sample of purely random judgments (Coyle 2004). The process of this evaluation is explained in the following section.
Consistency Test
For calculating CR we first need to calculate max which is the
maximum Eigen value of the matrix. This is approximated by the weighted total of the reciprocal matrix column totals using the relevant priority vector as a multiplicand for assigning weights. The maxin this case is given by:
X = 8.840
CR is given by the equation:
C.R. = Consistency Ratio = (C.I.) / (R.I.) (5.1) where C.I. = Consistency Index = ( max –n) / (n –1) and R.I. = Random Index
which is described below. For each matrix of size n, Saaty’s team generated random matrices and computed their mean C.I. value and called it the Random Index. These values are shown in the Table 5.6.
Table 5.6 Random Index (RI) table
n 1 2 3 4 5 6 7 8 9 10
Random
Index(RI) 0 0 .58 .90 1.12 1.24 1.32 1.41 1.45 1.49
(Source: Saaty 1980)
A value of CI less than or equal to 10 % of the table value is acceptable. Larger values indicate that the judgements are no better than random and therefore not to be considered consistent. The decision maker is
advised to revisit the judgement process to attempt to reduce the inconsistencies.
We next evaluate the data in Table 5.5 for consistency:
The consistency check is carried out using an Excel AHP calculator. The outputs are as given below (See Appendix 6):
Table 5.7 AHP calculations-final outputs
Constructs max CI CR Priorities
TC 0.31 C 0.08 Fu 0.14 UF 0.16 Fl 0.10 R 0.15 IT 8.840 0.3066 0.2322 0.07
The consistency ratio comes to 0.23. For a matrix n=7 the CI should be < 10% of 1.32 or <0.132. In the present case the value is 0.3066 which is greater than the guideline given by Saaty for a consistent judgement. This indicates that there are logical inconsistencies in the responses given by the experts.
Table 5.7 gives the priorities of various decision as applicable for the firm in question. For example in this case Total Cost (TC) comes out as being significantly more important while Functionality (Fu) and Flexibility (Fi) are relatively lower down on priority. This priority matrix would be different for different decision environments and represents the collective judgement of the decision makers on the relative importance of decision criteria.
Checking against alternative choices
Next the decision alternatives are evaluated against the same decision criteria. This is done by comparing how well each of the alternative ERPs performs against each of the decision criteria. This is not easy to do as it requires a knowledge of multiple ERP functionalities. It is usually done by studying literature, discussing with vendors, discussing with implementation consultants and talking to other firms that have made similar evaluations.
As is evident this would result in seven different relative performance matrices one for each of the seven decision criteria used above. For each of the matrices AHP calculations are to be followed to arrive at the final Product v/s decision criteria performance matrix. For this matrix the eigen vector representing the priorities can be calculated and the consistency check worked out.
The AHP method requires that the decision criteria matrix must first meet the consistency criteria before it can be relied upon for the rest of the calculation. As our decision criteria judgements fail on this count the rest of the calculation is for academic interest only. However it is presented here for illustrative purposes.
Comparison between ERPs
The comparison is made between four ERP systems SAP, Oracle Applications, J.D. Edwards and RAMCO Marshal. In this case, three experts from ERP implementation organizations were requested to rate the performance of each of these ERPs against the seven decision criteria. Saaty recommends the use of the geometric mean of individual rater scores (Saaty 2008 pp 95). In this case however we used a consensus building approach where the experts discussed and came to a conclusion on the rating. Table 5.8
gives the resulting reciprocal matrix for Total Cost (TC) comparison between the four products. In the case of Total Cost a reverse scale is used with the lower number being the more preferred.
Table 5.8 ERP Comparison table –criteria: Total Cost (TC)
SAP Oracle Ramco J.D Edwards
SAP 1.00 1.00 0.11 0.20
Oracle 1.00 1.00 0.11 0.20
Ramco 9.00 9.00 1.00 9.00
J.D. Edwards 5.00 5.00 0.11 1.00
Sum 16.00 16.00 1.33 10.40
The rest of the AHP calculation proceeded as described earlier resulting in a final comparison matrix with CI and CR as given in Table 5.9.
Table 5.9AHP results –TC
ERPs A. max CI CR SAP 4.0443305 ORACLE 4.0443305 RAMCO 4.35772 0.11924 0.1325 5.1359431 J.D.EDWARDS 4.2062908
Similarly each of the decision criteria was rated by the experts and the final table of priorities for the ERPs was computed (Table 5.10).
Table 5.10 ERP selection: final priorities TC C F UF F R IT Overall priority SAP 0.057 0.055 0.664 0.380 0.063 0.665 0.047 0.292 Oracle 0.057 0.099 0.152 0.380 0.063 0.202 0.127 0.150 Ramco 0.685 0.577 0.049 0.178 0.618 0.041 0.502 0.386 J.D. Edwards 0.201 0.268 0.135 0.062 0.255 0.092 0.324 0.172 5.4.4 Discussions
The decision matrix 5.10 shows that RAMCO is the preferred ERP system for this firm followed by SAP. It must be pointed out that this judgment in no way reflects either a real evaluation between ERP products nor the researcher’s personal view of which ERP is better. It is just an illustrative case of a judgment made by a set of experts in a firm considering the priorities of that firm at that point of time.
Any judgment is contextual. At a point of time in a specific context one or the other decision criteria gains or falls in importance. The final ranking between alternatives would reflect this. In another context with a different priority a different ranking would emerge. In this case cost was of primary importance and this has translated into RAMCO coming out as the preferred choice. If another parameter, say “functionality” was a priority another choice would have emerged.
As is seen in our case the initial evaluation of the priorities fails on the consistency test. This would mean that subsequent calculations are suspect. We tried to analyze why inconsistency creeps in. Our conclusions are as follows:
1. It becomes difficult to avoid logical inconsistency when parameters under comparison are more than 3 or 4. As a test
case when we asked the experts to evaluate against just 3 criteria-Cost, User-friendliness and Customization, we found that CR was within the 0.1 stipulated by Saaty. As alternatives go up evaluation becomes less consistent.
2. When parameters are many, time needed to do detailed comparisons increases. By the time the rater reaches the end he/she is quite bored and the rating then becomes arbitrary. 3. Our choice of Cost as one of the evaluation criteria was
probably erroneous (as pointed out by one of the thesis examiners). Respondents would almost always give preference to Cost thus vitiating the exercise. A better approach would have been to use the Benefits (B), Opportunities (O), Cost (C), Risks ( R) framework suggested by Saaty.
4. The Saaty scale of 9 choices is also probably too complex to give consistent results. It becomes difficult to distinguish between “absolutely more important” and “very much more important”.
As a guideline for practice we would suggest the use of the AHP technique as one of the methods for guiding for the final decision. It does give a priority matrix which has a mathematical basis and can be better than a simple scoring sheet. To avoid the problem of the rater getting tired or bored, it may be worthwhile to do the rating two or three criteria at a time over two or three sittings. This would probable give better results.
As a guideline for research we would suggest future research experiments with a simpler 5 point scale. The Saaty algorithm would work just as well and the overall process would probably be better.
