A comprehensive framework for selecting an ERP system
Chun-Chin Wei, Mao-Jiun J. Wang
*
Department of Industrial Engineering and Engineering Management, National Tsing Hua University, Hsin Chu, Taiwan, 300, ROC Received 8 May 2002; received in revised form 28 May 2002; accepted 25 October 2002
Abstract
This paper presents a comprehensive framework for combining objective data obtained from external professional reports and subjective data obtained from internal interviews with vendors to select a suitable Enterprise Resource Planning (ERP) project. A hierarchical attribute structure is proposed to evaluate ERP projects systematically. In addition, fuzzy set theory is used to aggre-gate the linguistic evaluation descriptions and weights. An actual example in Taiwan demonstrates the feasibility of applying the proposed framework.
#2003 Elsevier Ltd and IPMA. All rights reserved.
Keywords:Enterprise Resource Planning; Decision making; Fuzzy set theory
1. Introduction
An Enterprise Resource Planning (ERP) system is an integrated enterprise computing system to automate the flow of material, information, and financial resources among all functions within an enterprise on a common database[1]. A successful ERP project involves selecting an ERP software system and vendor, implementing this system, managing business processes change (BPC), and examining the practicality of the system. However, a wrong ERP project selection would either fail the pro-ject or weaken the system to an adverse impact on company performance[2,3]. Due to limitations in avail-able resources, the complexity of ERP systems, and the diversity of alternatives, selecting an ERP project is a time-consuming task.
Several methods have been proposed for selecting a suitable ERP project or management information sys-tem [4–11]. The scoring method [5] is one of the most popular. Although it is intuitively simple, it does not ensure resource feasibility [9,10]. Teltumbde [4] sug-gested 10 criteria for evaluating ERP projects and con-structed a framework based on the Nominal Group Technique (NGT) and the analytic hierarchy process (AHP) to make the final choice. Santhanam and
Kyparisis [7,8] proposed a nonlinear programming
model to optimize resource allocation and the
inter-action of factors; their model considered
inter-dependencies of criteria in the information system selection process. Lee and Kim[9]combined the analy-tic network process (ANP) and a 0–1 goal-programming model to select an information system. However, these mathematical programming methods can not contain sufficient detailed attributes, above all, which are not easy to quantify, so that the attributes were restricted to some financial factors, such as costs and benefits. Fur-thermore, many of them involved only the consideration of internal managers, but do not offer a comprehensive process for combining evaluations of different data sources to select an ERP project objectively.
Reports made by professional organizations and information collected from interviews with ERP suppli-ers should be considered in evaluating information of ERP projects. Professional organizations, such as research institutes and consulting companies, employ many experts to analyze information about ERP, including market share, vendor size, system perfor-mance, and other data. Their professional studies are very helpful to managers to have an overview of ERP systems and vendors. Furthermore, decision-makers can extract important attributes from these reports. How-ever, the literature lacks studies on integrating the eval-uation of objective external professional data sources and subjective internal interview data sources. This
0263-7863/03/$30.00 # 2003 Elsevier Ltd and IPMA. All rights reserved. doi:10.1016/S0263-7863(02)00064-9
www.elsevier.com/locate/ijproman
*Corresponding author. Tel.: 5742655; fax: +886-3-5722685.
study aims to provide a new framework for integrating the two kinds of data evaluation for selecting a suitable ERP project.
In reality, selecting a suitable ERP project involves multiple factors. Some of the measures, for example, the risk of the project, the functional fitness, and the ability of a vendor may not be precisely defined. Evaluation ratings under various attributes and the weights of the attributes are frequently assessed in linguistic terms, ‘high’, ‘poor’, among others. A fuzzy multiple-criteria decision-making method (FMCDM) is very useful in integrating various linguistic assessments and weights to evaluate ERP alternatives.
This study proposes a comprehensive framework for selecting a suitable ERP project. Decision-makers can effectively integrate objective professional comments and subjective opinions of managers. A measure called, ‘‘fuzzy ERP suitability index’’ is used to account for the ambiguities involved in the evaluation of the appro-priateness of ERP alternatives and the importance weights of attributes. An actual case in Taiwan is descri-bed to demonstrate the proposed method in practice.
2. Procedure for selecting an ERP project
A systematic ERP selection algorithm, using two-dimensional analysis and fuzzy set theory, is presented. The first dimension involves objective ratings of ERP project data in accordance with external professional reports. The second dimension requires assigning sub-jective ratings to ERP projects on the basis of data acquired in interviews. The objective and subjective evaluations are combined to obtain the final fuzzy ERP suitability.
A stepwise procedure follows.
Step 1. Form a project team and conduct the business process re-engineering (BPR).
Step 2. Collect all possible information about ERP vendors and systems. Filter out unqualified vendors. Step 3. Establish the attribute hierarchy and assign weights to the attributes.
Step 4. Interview vendors and collect detailed information.
Step 5. Analyze the data obtained from the external professional reports to obtain the objective ERP suitability.
Step 6. Assign subjective ratings to the ERP projects on the basis of data acquired in interviews to calculate the subjective ERP suitability.
Step 7. Combine the evaluations of both data sources and aggregate the decision-making assessments to determine the final fuzzy ERP suitability.
Step 8. Utilize the fuzzy integral value ranking method to obtain the rank of each ERP project.
Step 9. Analyze the results of indices, l and k. Observe the change in the final ERP suitability and the final ranking value.
Step 10. Select the ERP project with the maximum ranking value.
Step 11. Implement the selected ERP project.
Fig. 1 shows the comprehensive framework of the method.
2.1. Form a project team and conduct BPR
The first step is to form a project team that consists of decision-makers, functional experts and senior representa-tives of user departments. In essence, an ERP project is not
only installing a new information technology system to replace the legacy system but also reshaping the business processes to overcome the challenges of dynamic market. BPR is necessary to be undertaken to rationalize and stan-dardize the workflows of all business processes in advance. The functional characteristics of ERP are developed during the BPR. The project team can decompose the business objectives, for example costs reducing, quality and effi-ciency improving, and performance enhancing. Structuring the objectives involves organizing them, so that the project team can describe in detail what the company wants to achieve, and then incorporating these objectives appro-priately into the decision model.
2.2. Collect information and eliminate unqualified alternatives
Collect as wide a range of information as possible concerning ERP vendors and systems from professional magazines, exhibitions, yearbooks, the Internet, and other sources. Ensure the search includes less widely known vendors to make sure that some more feasible projects are not overlooked.
The characteristics that reflect the system’s require-ments are transferred to a questionnaire or a checklist of the system specifications. The listed vendors are invi-ted to provide information in response to these specific questions. Eliminate the clearly unqualified vendors and thereby reduce the number of candidates.
2.3. Establish attribute hierarchy
Several researchers claim that both quantitative and qualitative attributes that can satisfy the routine oper-ation under the strategies and goals of the company should be involved [5]. The aspects companies usually consider when selecting ERP project include:
1. The strategy of system to meet the business strategy and goals
2. The ability of system to support the business process
3. The technical requirements on which the system operates
4. The ability of vendor to support the system implementation and maintenance
5. The methodologies of business processes change and project management
Thus, after organizing the factors addressed in prior studies[4–11], the attributes can be classified into three categories, as follows:
1. Project factors: attributes involved in project man-agement, such as total cost, time of implementation, benefits, and risks;
2. Software system factors: features of the software and system, including strategic fitness and the function of ERP;
3. Vendor factors: attributes that pertain to ven-dors, such as ability and reputation.
Fig. 2depicts the attribute hierarchy for selecting the ERP projects.
The main attributes xi(i=1, 2,. . ., 8) are summarized
from the attributes used in professional reports. They are typical but not sufficiently detailed to evaluate ERP projects. Therefore, each main attribute is divided into sub-attributes, xij (j=1, 2,. . ., n(i)), where n(i) denotes
the number of sub-attributes of main attribute xi. The
decision-makers evaluate only the professional data of the ERP projects under the main attributes. On the other hand, data are assessed from interviews under the sub-attributes and aggregate to the corresponding main attributes. Finally, the evaluations from both data sources are combined to obtain the conclusions.
The weights of each attribute can be determined by direct assignment or pairwise comparisons. Decision-makers use a set of five linguistic terms in weighting set, W, to describe the weights of each attribute, W={VL, L, M, H, VH}. In addition, assume that a set of lin-guistic terms, S={VP, P, F, G, VG}, is used to rate ERP projects to qualitative attributes.Table 1specifies the triangular fuzzy numbers for these linguistic weights and values.
2.4. Hold interview meetings
The vendors that remain on the list are asked to pro-vide their proposals. A series of interviews with these vendors is scheduled. The project team arranges the schedule, agenda, scenarios, and questions for the ven-dors before the interviews are held. The scenarios describe how the ERP system exchanges data in a transaction and performs particular functions. The real company data, which were arranged in advance, were used to ask for the detailed demonstrations. The repre-sentatives of different user departments in the project team should provide the knowledge of their special processes to examine the vendor’s demonstrations. Above all, decision-makers should direct the meetings and ensure that sufficient information about the ERP projects can be collected.
2.5. Aggregate external professional data
Among the attributes, quantitative attributes are those that can be numerically evaluated. The values of these quantitative attributes are collected from the data by the ERP vendor provided or the data, which nego-tiated with the vendor. The crisp values must be con-verted into dimensionless values to ensure that these
values are compatible with the linguistic ratings of the qualitative attributes. Define B and C to be the set of benefit attributes and cost attributes, respectively. That is, in Fig. 2, B={benefit (x3)} and C={total cost (x1),
implementation time (x2)}. Let Tti(t=1, 2,. . ., m, i=1,
2, 3) represent the values assigned to ERP project Pt
under main attribute xi. Then,
RTti¼ Tti Pm t¼1 Tti ; if i 2 B ð1Þ else, RTti¼ Tti1 Pm t¼1 T1 ti ; if i 2 C ð2Þ
The advantage of using the above converting equation is that it can prevent any extreme attribute value of con-scious or unconcon-scious negligence after transformation.
Assume that the value of a crisp rating is r, its triangular fuzzy number is (r, r, r). For different decision-makers the values of these quantitative data are the same. Then, let O~ti ¼RTti, i =1, 2, 3, where O~ti is the transferred fuzzy rating of ERP project Ptunder main attribute xi
from the quantitative data.
On the other hand, the attributes (from x4to x8) that
are difficult to quantify are reasonably treated as quali-tative attributes. Decision-makers evaluate the profes-sional data of the ERP projects under main qualitative
Fig. 2. ERP evaluation attribute hierarchy.
Table 1
Linguistic variables describing weights of attributes and values of ratings Very low (VL) Very poor (VP) (0, 0, 0.3)
Low (L) Poor (P) (0, 0.3, 0.5)
Medium (M) Fair (F) (0.2, 0.5, 0.8)
High (H) Good (G) (0.5, 0.7, 1.0)
attributes in linguistic terms set S. Assess O~tih (t=1, 2,. . ., m; i=4, 5,. . ., 8; h=1, 2,. . ., n), the linguistic rat-ing of ERP project Ptby decision-maker Dhunder main
attribute xi from the professional data evaluation.
Define W~ih(i=1, 2,. . ., 8; h=1, 2,. . ., n) as the linguistic weight assigned to main attribute xi by decision-maker
Dh. A mean operator is used to pool each rating by
decision-makers, since the fuzzy average operation is a commonly used method of aggregation and is easy to understand[12]. Define,
O~ti¼ð1=nÞ O~ti1O~ti2. . . O~tin
; t ¼1; 2; . . . ; m; i ¼4; 5; . . . ; 8 ð3Þ and W~i¼ð1=nÞ W~i1W~i2. . . W~in ; i ¼1; 2; . . . ; 8 ð4Þ
where O~ti is the average fuzzy rating of ERP project Pt
under main attribute xi from the professional data
evaluation and W~i is the average weight of main attri-bute xi. Then, combine the quantitative and qualitative
evaluations to obtain the fuzzy objective suitability, O~t, of ERP project Ptby the following equation:
O~t¼ 1=8 ð Þ O~t1W~1 O~t2W~2 . . . O~t8W~8 h i ; t ¼ 1; 2;. . . ; m ð5Þ 2.6. Aggregate interview data
Like the quantitative attributes transformation
method we mentioned above, let S~ti¼RTti, i =1, 2, 3, where S~ti is the transferred fuzzy rating of ERP project Pt under main attribute xi from the quantitative data.
After interviewing with ERP vendors, decision-makers assess the linguistic rating under the sub-attributes. S~tijh (t=1, 2,. . ., m; i=4, 5,. . ., 8; j=1, 2,. . ., n(i); h=1, 2,. . ., n) indicates the linguistic rating of ERP project Ptby
deci-sion-maker Dhfor sub-attribute xijfrom the evaluation of
interview data. Let W~ijhbe the linguistic weight assigned to sub-attribute xijby decision-maker Dh. Define,
S~tij¼ð1=nÞ S~tij1S~tij2. . . S~tijn
;
t ¼1; 2; . . . ; m; i ¼ 4; 5; . . . ; 8; j ¼ 1; 2; . . . ; nðiÞ ð6Þ and
W~ij¼ð1=nÞ W~ij1W~ij2. . . W~ijn
i ¼4; 5; . . . ; 8; j ¼1; 2; . . . ; nðiÞ ð7Þ
where S~tijis the average rating of ERP project Ptunder
sub-attribute xij from the evaluation of interview data,
and W~ijis the average weight of sub-attribute xij.
Then, aggregate these S~tijto the corresponding main attributes. The aggregated rating, S~ti, of ERP project Pt
under main qualitative attribute xi from the evaluation
of interview data can be obtained, as inEq. (8).
S~ti¼ S~ti1W~i1 S~ti2W~i2 . . . S~tinðiÞW~inðiÞ h i W~i1. . . W~inðiÞ ; t ¼1; 2; . . . ; m; i ¼4; 5; . . . ; 8 ð8Þ Thus, integrate the quantitative and qualitative eva-luations to obtain the fuzzy subjective suitability, S~t, of ERP project Ptby aggregating S~tiwith W~i. Then, S~t ¼ 1=8 ð Þ S~t1W~1 S~t2W~2 . . . S~t8W~8 h i ; t ¼ 1; 2;. . . ; m ð9Þ
2.7. Combine objective and subjective suitabilities Combine the evaluations of both data sources. The final fuzzy ERP suitability, R~t, of ERP project Ptcan be
obtained with an index l byEq. (10).
R~t¼lO~tð1 lÞS~t; 0 4 l 4 1;
t ¼1; 2; . . . ; m ð10Þ
The value l can be manipulated to reflect the deci-sion-makers’ attitude concerning the relative impor-tance of both data sources.
2.8. Rank final fuzzy ERP suitability
Selecting the appropriate ERP project depends on rank-ing the final fuzzy ERP suitability. Many fuzzy rankrank-ing methods have been proposed[13–15]. For simplicity and effectiveness in problem solving, the fuzzy ranking method with integral value proposed by Liou and Wang [13] is applied to rank the final fuzzy ERP suitability.
According to the ranking method, index k indicates the degree of optimism of the decision-makers. A larger k represents a higher degree of optimism. Rank R~t by the total integral value, respectively. Select the ERP project with the maximum total integral value.
2.9. Change indices,l and k, and make final decision A method that considers various trade-offs among projects is necessary for making the final decision, since
the preferences of decision-makers and their environ-ment are not always stable. Substitute various values of l and k into the model and analyze the changes in the final outcomes. Finally, the project team can choose the ERP project with the maximum total integral ranking value. 2.10. Implement the selected ERP project
With the suitable ERP project selected, the project team should prepare to make a contract with the selec-ted vendor. ERP project implementation is a complex exercise in technology innovation and business pro-cesses change management. A cautious, evolutionary, bureaucratic, and interactive implementation process based on change management and culture readiness can lead to successful ERP implementation.
3. An actual evaluation
The proposed framework was used to select an ERP project at an electronics company in Science-Based Industrial Park in Taiwan. This medium-sized company designs and manufactures a variety of modular micro-wave systems. The company seeks to maintain its com-petitive advantage by improving the effectiveness of its global logistics and the efficiency of its response to
cus-tomer demand. The stepwise procedure is presented in the following.
Step 1.The top managers announced the launching of a series of E-business projects, including BPR, ERP, and an information communication system to increase the competitive advantage of the business and replace the legacy system. Executing BPR and analyzing workflow are essential in determining ERP system requirements.
A steering committee of seven major managers was formed. It included the General Manager and managers of sales, manufacturing, R&D, MIS, finance, and pur-chasing to formulate the project plan and select an ERP system. Representatives of different user departments were also chosen to participate in the project team.
Step 2.Information on 20 ERP vendors and systems was initially collected. Unfavorable alternatives were eliminated by asking a few questions, which were for-mulated by the specifications.Table 2 lists some of the questions. After preliminary screening, four local ERP vendors, P1, P2, P3, and P4, remained under
considera-tion.
Step 3. The system’s requirements were translated into the corresponding attributes to formulate the archical attribute structure. On the basis of the
hier-archical attribute structure, the decision-makers
assigned weights to main attributes, W~i, and to sub-attributes, W~ij, from the linguistic description set W.
Table 2
Examples of screening questions
Item Question
Vendor size 1. Does the vendor’s size suit our company? Complexity 1. Is the ERP system too complex, or is it a good fit?
2. Does it fit our requirements, or is it overqualified? Cost vs. budget 1. What is the total cost of the project?
2. Can we accept the difference between the cost and budget? Domain knowledge 1. What is the provider’s target domain and market?
2. Does it match to our business needs? Flexibility 1. Is the technology flexible and durable?
Covering requirements 1. Does the system and its modules cover all our requirements? Fundamental 1. What database and hardware can be supported by the system?
Information technology 1. Does the vendor provide other information systems, such as SCM, MES, DW, CRM, and EC? 2. Does the vendor widely integrate its system with other partners’ information systems? Implementation methodology 1. What is the implementation methodology?
2. Is it feasible and simple?
Service maintenance 1. Who supports upgrades and maintenance? The software supplier or the reseller? 2. Does the vendor have any local service point or a branch company?
Consulting service 1. Does the vendor provide consulting services? 2. Does it cooperate with another consultant company?
Financial consideration 1. How did the vendor perform financially over the last two years? 2. What is its current financial forecast?
Step 4.Intensive interviews were scheduled with each of the four vendors. The team used a form to record data on the functionality, processes, local support, and finance. Most importantly, core processes and special operational features were assessed by considering demo scenarios and by examining each system’s capacity to fulfill key demands.
Step 5. The decision-makers evaluated the quantita-tive attributes (x1, x2, and x3), using the information
provided by the vendors. These attributes were rated for each ERP project by using Eqs. (1) and (2). On the other hand, the decision-makers evaluated the profes-sional reports of ERP projects with respect to the main qualitative attributes (from x4 to x8) by using the
lin-guistic ratings in scale set S. Then, aggregated the quantitative and qualitative evaluation with the corre-sponding weight to yield the objective ERP suitability O~t by Eqs. (3), (4), and (5). Table 3 gives the fuzzy objective suitabilities of all ERP projects.
Step 6. Let S~ti¼RTti, i=1,2,3. The linguistic rating of ERP project Pt by decision-maker Dh under
sub-attribute xijwas S~tijh (t=1, 2,. . ., m; i=4, 5,. . ., 8; j=1,
2,. . ., n(i); h=1, 2,. . ., n), assessed by evaluating the interview data. Then, the aggregated rating S~ti of ERP project Pt under main attribute xi for evaluating the
interview data can be obtained by conducting Eqs. (6), (7), and (8). Aggregate S~ti and W~i by averaging the corresponding products over all main attributes. The fuzzy subjective suitability S~t of ERP project Ptcan be
obtained by Eq. (9). Table 3 presents the fuzzy sub-jective suitabilities of all ERP projects.
Step 7.Combine the results of both data sources, as in
Eq. (10), with l=0.5. That is, the degrees of importance of both data source evaluations were equal. Table 3
summaries the fuzzy ERP suitability indices.
Step 8.The total integral values R~tof these final fuzzy ERP suitability indices were obtained by using the fuzzy integral value ranking method with k=0.5 (Table 3).
Step 9. The rank order of the ERP projects was P1,
P2, P3, and P4. The most suitable project was thus P1.
Fig. 3shows the change in ranking when k=0.5 and l was varied from 0 to 1. Project P1was the best choice
with any l. However, the order of P2and P3 changed
(P3> P2) when l=0.8 was changed to 1.0. Therefore, P3
Table 3 Evaluation results ERP project P1 P2 P3 P4 O~t (0.257, 0.515, 0.688) (0.216, 0.439, 0.681) (0.226, 0.455, 0.678) (0.167, 0.368, 0.640) S~t (0.223, 0.466, 0.678) (0.196, 0.414, 0.672) (0.178, 0.380, 0.656) (0.138, 0.353, 0.617) R~t (0.240, 0.491, 0.683) (0.206, 0.427, 0.677) (0.202, 0.418, 0.667) (0.153, 0.361, 0.628) I0:5 T ðR~TÞ 0.476 0.434 0.426 0.375
was preferred over P2 if the evaluation of the
profes-sional data source was considered to be more important than the evaluation of the interview data.
Next, l=0.5 was fixed and k was varied.Fig. 4shows P1 always remaining the first preference. However, the
values of P2and P3were very close to each other when
k was varied. That is, P2 and P3 were evaluated as
approximately equal.
Step 10.The project team finally recommended ERP
project P1 as the most suitable selection for the
company.
The managers were satisfied with the framework that we presented. They can integrate the knowledge of external professional experts and their judgments to choose an ERP project. With a proper ERP project selected using the proposed framework, the selection time, as well as the cost of ERP project implementation and maintenance were reduced. After the ERP system was implemented, the managers were able to acquire information from the USA, mainland China, and Tai-wan to respond to customers more efficiently and
effec-tively. The mean lead-time was reduced by
approximately 43% from 3 weeks to 12 days.
4. Conclusion
This study has proposed a comprehensive framework for selecting an ERP project that combines data obtained from professional studies with that surveyed from interviews with vendors. A hierarchical attribute
structure including project, software, and vendor factors has been provided for evaluating ERP projects. An integration model that uses the fuzzy average method and fuzzy integral ranking has been developed. The final decision is determined by the highest total integral value. The results of a real example indicate that the proposed framework is very useful for selecting a sui-table ERP system selection.
The proposed framework offers the following advan-tages in the ERP project.
1. It provides a comprehensive and systematic method. Decision-makers can easily select a sui-table ERP project by following the stepwise procedure.
2. It provides a simple and intuitive procedure for integrating the subjective opinions of decision-makers and the objective professional comments of external experts, thereby avoiding the use of a complex mathematical model.
3. The proposed algorithm considers not only quantitative data but also linguistic data. Man-agers can assess various attributes of a system, particularly in an ill-defined situation, by using linguistic or quantitative values. It can be refined since it flexibly accommodates additional con-siderations.
4. The values of l and k can be changed to deter-mine related changes in the prioritization of projects, with regard to the current business sit-uation, to solidify the final decision.
References
[1] Kumar V, Maheshwari B, Kumar U. Enterprise resource plan-ning systems adoption process: a survey of Canadian organi-zations. International Journal of Production Research 2002;40: 509–23.
[2] Wilson F, Desmond J, Roberts H. Success and failure of MRPII implementation. British Journal of Management 1994;5:221–40. [3] Hicks DA, Stecke KE. The ERP maze: enterprise resource
plan-ning and other production and inventory control software. IIE Solutions 1995;27(8):12–16.
[4] Teltumbde A. A framework of evaluating ERP projects. Inter-national Journal of Production Research 2000;28:4507–20. [5] Ptak CA. ERP tools, techniques, and applications for integrating
the supply chain. St. Lucie Press, 2000.
[6] Chen K, Gorla N. Information system project selection using fuzzy logic. IEEE transactions on systems, man, and cyber-netics—part A: Systems and Humans 1998;28:849–55.
[7] Santhanam R, Kyparisis GJ. A multiple criteria decision model for information system project selection. Computers & Oper-ations Research 1995;22(8):807–18.
[8] Santhanam R, Kyparisis GJ. A decision model for inter-dependent information system project selection. European Journal of Operational Research 1996;89:380–99.
[9] Lee JW, Kim SH. An integrated approach for interdependent information system project selection. International Journal of Project Management 2001;19:111–8.
[10] Badri MA, Davis D, Davis D. A comprehensive 0–1 goal pro-gramming model for project selection. International Journal of Project Management 2001;19:243–52.
[11] Lai VS, Robert TP, Wong BK. Software selection: a case study of the application of the analytical hierarchical process to the selec-tion of a multimedia authoring system. Informaselec-tion Management 1999;36:221–32.
[12] Lee DH, Park D. An efficient algorithm for fuzzy weighted aver-age. Fuzzy Sets and Systems 1997;87:39–45.
[13] Liou TS, Wang MJ. Ranking fuzzy numbers with integral value. Fuzzy Sets and Systems 1992;50:247–55.
[14] Chen SH. Ranking fuzzy numbers with maximizing set and minimizing set. Fuzzy Sets and Systems 1985;17:113–29. [15] Buckley JJ. Ranking alternatives using fuzzy numbers. Fuzzy Sets