6. Conclusions, Recommendations and Future Work
6.2 Recommendations and Future Work
3. To establish relative importance among all evaluated factors and to use DMs judgements according to their area of expertise.
4. To determinate alternatives innovation rate and projected functionalities. 5. To facilitate the evaluation of qualitative criteria by using fuzzy logic and by
detecting influences among criteria.
6. To detect important criteria for evaluation, that influences other criteria.
7. To perform post-evaluation revisions and anticipate technology trends and busi- ness needs changes.
From case study analysis, it was also detected that group judgements combination, expert consistency and the complexity in performing matrix calculation, are the most critical topics for getting the best results of methodology application.
6.2 Recommendations and Future Work
It is recommended further analysis of the CRM case study, such as the performing of sensitivity analysis and proofing different DMs and alternatives configurations.
It is also recommended, the application of the methodology to other IT selection cases, specially in complex cases when the application of one period methodologies is not enough to make decisions. The selection of IT with high innovation rate should be also considered in the exploration of new cases.
From managers who have not applied any decision method to IT selection yet, it is recommended to start with the gathering of requirement step of the approach by using the technology roadmapping technique. The application of this tool will allow manager to explore technology trends and business present and future needs, and will provide them a deep insight into the decision problem they are facing.
About the methodology, it is recommended for future work, more research on group decision making considering uncertainty and deepening on the requirements gathering using technology roadmapping. For future work, it is also recommended the construc- tion of a software tool for facilitating the selection process.
Interactive approaches where DMs negotiate between their decision and the result are also recommended to be explored as part of future work, as they can contribute to give DMs better knowledge of the decision problem, which will finally reflect in better
64 6. Conclusions, Recommendations and Future Work.
decisions.
Finally, the inclusion of other future technology assessment both quantitative and qual- itative is desired to support DM evaluation about future behaviours of the alternatives. The inclusion of economical and legal factors affecting the vendors can also contribute in better evaluating the technology providers.
References
Adamcsek, E. (2008). The analytic hierarchy process and its generalizations. Unpub- lished doctoral dissertation, Thesis, Eotvos Lor a n d Un iversit y.
Asgari, M., Allahverdiloo, M., & Samkhani, S. (2011). A comprehensive framework for selecting the ERP system in iran khodro company. European Journal of Economics, Finance and Administrative Sciences(38), 7–19.
Ayag, Z. (2010). A combined fuzzy AHP-simulation approach
to CAD software selection. International Journal of Gen-
eral Systems, 39 (7), 731–756. Retrieved 2012-10-14, from
http://www.tandfonline.com/doi/abs/10.1080/03081079.2010.495190 doi: 10.1080/03081079.2010.495190
Ayag, Z. (2011). Evaluating simulation software alternatives through ANP. In
Proceedings of the 2011 international conference on industrial engineering and operations management, kuala lumpur, malaysia. Retrieved 2013-01-23, from http://www.iieom.org/ieom2011/pdfs/IEOM079.pdf
Ayag, Z., & ¨Ozdemir, R. (2007). An intelligent approach to ERP software selection through fuzzy ANP. International Journal of Production Research, 45 (10), 2169– 2194.
Barreiros, M., Grilo, A., Cruz-Machado, V., & Cabrita, M. (2010). Applying fuzzy sets for ERP systems selection within the construction industry. In IEEM2010 - IEEE international conference on industrial engineering and engineering management (pp. 320–324).
Botella, P., Burgu´es, X., Carvallo, J. P., Franch, X., Grau, G., Marco, J., &
Quer, C. (2004). ISO/IEC 9126 in practice: what do we need to know?
In Proceedings of the 1st software measurement european forum. Retrieved
2013-01-29, from http://www.essi.upc.edu/ webgessi/publicacions/
SMEF’04-ISO-QualityModels.pdf
Bozdag, C., Kahraman, C., & Ruan, D. (2003). Fuzzy group decision making for selec- tion among computer integrated manufacturing systems. Computers in Industry, 51 (1), 13–29.
Burgelman, R. A., Maidique, M. A., & Wheelwright, S. C. (2004). Strategic manage- ment of technology and innovation. McGraw-Hill Irwin Boston.
66 6. Conclusions, Recommendations and Future Work.
with non-technical factors for COTS components selection. In Proceedings of the 2006 international workshop on software quality (p. 9–14). Retrieved 2013-01-29, from http://dl.acm.org/citation.cfm?id=1137706
Cebeci, U. (2009). Fuzzy AHP-based decision support system for selecting ERP
systems in textile industry by using balanced scorecard. Expert Systems with Applications, 36 (5), 8900–8909.
Chang, D.-Y. (1996). Applications of the extent analysis method on fuzzy AHP. Euro- pean journal of operational research, 95 (3), 649–655. Retrieved 2013-02-27, from http://www.sciencedirect.com/science/article/pii/0377221795003002 Changyun, L., Haiyan, C., Gexin, M., & Zhibing, W. (2012, January). A BPM soft-
ware evaluation method. In 2012 second international conference on intelligent system design and engineering application (ISDEA) (pp. 1 –4). doi: 10.1109/IS- dea.2012.681
Cil, I., Alpturk, O., & Yazgan, H. R. (2005). A new collaborative system framework based on a multiple perspective approach: InteliTeam. Decision Support Systems, 39 (4), 619––641.
Cochran, J. K., & Chen, H.-N. (2005, January). Fuzzy multi-criteria selection of object-oriented simulation software for production system analysis. Com- puters & Operations Research, 32 (1), 153–168. Retrieved 2013-01-23, from http://www.sciencedirect.com/science/article/pii/S0305054803002090 doi: 10.1016/S0305-0548(03)00209-0
Colombo, E., & Francalanci, C. (2004). Selecting CRM packages based on architec- tural, functional, and cost requirements: Empirical validation of a hierarchical ranking model. Requirements engineering, 9 (3), 186––203. Retrieved 2013-01-23, from http://www.springerlink.com/index/KAJD96HVWH39RBQP.pdf
Dur´an, O. (2011, October). Computer-aided maintenance manage-
ment systems selection based on a fuzzy AHP approach. Adv.
Eng. Softw., 42 (10), 821––829. Retrieved 2012-08-21, from
http://dx.doi.org/10.1016/j.advengsoft.2011.05.023 doi:
10.1016/j.advengsoft.2011.05.023
Eldrandaly, K. (2007). GIS software selection: A multi criteria decision making approach. Applied GIS , 3 (5).
Etaati, L., Sadi-Nezha, S., & Makue, A. (2011, January). Using fuzzy group analyt- ical network process and ISO 9126 quality model in software selection: A case study in e-learning systems. Journal of Applied Sciences, 11 (1), 96–103. doi: 10.3923/jas.2011.96.103
Far, B., Mudigonda, V., & Elamy, A.-H. (2009). A general purpose software evaluation system. Informatica (Ljubljana), 33 (3), 261–270.
Grau, G., Carvallo, J., Franch, X., & Quer, C. (2004). DesCOTS: a software system for selecting COTS components. In Conference proceedings of the EUROMICRO (Vol. 30, pp. 118–126).
References 67
Gregory, M. J. (1995, October). Technology management: A process approach. Pro- ceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineer- ing Manufacture, 209 (5), 347–356.
Gupta, P., Mehlawat, M. K., Mittal, G., & Verma, S. (2009, January). A hybrid ap- proach for selecting optimal COTS products. In O. Gervasi, D. Taniar, B. Mur- gante, A. Lagan, Y. Mun, & M. L. Gavrilova (Eds.), Computational science and its applications – ICCSA 2009 (pp. 949–962). Springer Berlin Heidelberg. Gupta, P., Mehlawat, M. K., & Verma, S. (2012, February). COTS selection using fuzzy
interactive approach. Optimization Letters, 6 (2), 273–289. doi: 10.1007/s11590- 010-0243-5
G¨urb¨uz, T., Alptekin, S., & Isiklar Alptekin, G. (2012). A hybrid MCDM methodology for ERP selection problem with interacting criteria.
Haghighi, H., & Mafi, O. (2010, January). Towards a systematic, cost-effective ap- proach for ERP selection. World Academy of Science, Engineering & Technology, 61 , 231–237.
Henningsson, K., & Wohlin, C. (2002). Understanding the relations between software quality attributes-a survey approach. In Proceedings 12th international confer- ence for software quality, ottawa, canada.
Hong, T., & Kim, E. (2007, October). The selection of CRM systems in financial institutes using the analytic hierarchy. In 2nd international conference on dig- ital information management, 2007. ICDIM ’07 (Vol. 1, pp. 399 –404). doi: 10.1109/ICDIM.2007.4444256
Hua, G., & Song, X. (2010). Fuzzy comprehensive appraisal of ERP selection. In Pro- ceedings - international conference on electrical and control engineering, ICECE 2010 (pp. 2780–2784).
Huiqun, H., & Guang, S. (2012). ERP software selection using the rough set and TPOSIS methods under fuzzy environment. Advances in Information Sciences and Service Sciences, 4 (3), 111–118.
Ishizaka, A., & Labib, A. (2011, October). Review of the main developments in the analytic hierarchy process. Expert Systems with Applications, 38 (11), 14336– 14345. doi: 10.1016/j.eswa.2011.04.143
Jadhav, A. S., & Sonar, R. M. (2011). Framework for evaluation
and selection of the software packages: A hybrid knowledge based
system approach. J. Syst. Softw., 84 (8), 1394–1407. Retrieved
2012-08-21, from http://dx.doi.org/10.1016/j.jss.2011.03.034 doi:
10.1016/j.jss.2011.03.034
Jankovic, D., & Milidragovic, R. (2012, May). Selecting the optimal opensource GIS software for local authorities by combining the ISO 9126 standard and AHP approach. In 2012 proceedings of the 35th international convention MIPRO (pp. 1661 –1665).
68 6. Conclusions, Recommendations and Future Work.
natives using a fuzzy multi-criteria decision making methodology. International Journal of Production Research, 48 (2), 547–566.
Kalakota, R., & Robinson, M. (2002). M-business: The race to mobility. McGraw-Hill. Karaarslan, N., & Gundogar, E. (2009, October). An application for modular
capability-based ERP software selection using AHP method. International Jour- nal of Advanced Manufacturing Technology, 42 (9/10), 1025–1033.
Karsak, E., & ¨Ozogul, C. (2009). An integrated decision making approach for ERP system selection. Expert Systems with Applications, 36 (1), 660–667.
Kaufmann, A., & Gupta, M. M. (1988). Fuzzy mathematical models in engineering and management science. New York, NY, USA: Elsevier Science Inc.
Lan, H., & Congbo, L. (2009). A method for selecting ERP system based on fuzzy set theory and analytical hierarchy process. In Proceedings of the 2009 WRI global congress on intelligent systems, GCIS 2009 (Vol. 1, pp. 329–332).
Lee, Y. M., Kim, J. B., Choi, I. W., & Rhew, S. Y. (2007, August). A study on selection process of open source software. In Sixth international conference on advanced language processing and web information technology, 2007. ALPIT 2007 (pp. 568 –571). doi: 10.1109/ALPIT.2007.83
Liao, X., Li, Y., & Lu, B. (2007). A model for selecting an ERP system based on linguistic information processing. Information Systems, 32 (7), 1005–1017.
Lin, C.-W., & Wang, C.-H. (2011). A selection model for auditing soft-
ware. Industrial Management & Data Systems, 111 (5), 776–790. doi:
10.1108/02635571111137304
Lin, H.-Y., Hsu, P.-Y., & Sheen, G.-J. (2007, April). A fuzzy-based decision-
making procedure for data warehouse system selection. Expert Sys-
tems with Applications, 32 (3), 939–953. Retrieved 2013-01-23, from
http://www.sciencedirect.com/science/article/pii/S0957417406000509 doi: 10.1016/j.eswa.2006.01.031
Lingyu, H., Bingwu, L., & Juntao, L. (2009). An ERP system selection model based on fuzzy grey TOPSIS for SMEs. In 6th international conference on fuzzy systems and knowledge discovery, FSKD 2009 (Vol. 3, pp. 244–248).
Lucheng, H., Xin, L., & Wenguang, L. (2010, April). Research on emerging technology selection and assessment by technology foresight and fuzzy consistent matrix. foresight , 12 (2), 77–89. doi: 10.1108/14636681011035762
Mamaghani, F. (2002, March). Evaluation and selection of an antivirus and content filtering software. Information Management & Computer Security, 10 (1), 28–32. doi: 10.1108/09685220210417481
March, S. T., & Smith, G. F. (1995, December). Design and natural science research on information technology. Decision Support Systems, 15 (4), 251–266. doi: 10.1016/0167-9236(94)00041-2
Mikhailov, L., & Masizana, A. (2004). Decision support for information systems selection. In Conference proceedings - IEEE international conference on systems,
References 69
man and cybernetics (Vol. 6, pp. 5044–5049).
Mikhailov, L., & Singh, M. G. (2003). Fuzzy analytic network process and its ap- plication to the development of decision support systems. Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on, 33 (1), 33–41.
Mohamed, A., Ruhe, G., & Eberlein, A. (2011). Mismatch handling for
COTS selection: a case study. Journal of Software Maintenance and Evo-
lution: Research and Practice, 23 (3), 145––178. Retrieved 2012-08-21, from
http://onlinelibrary.wiley.com/doi/10.1002/smr.493/abstract doi:
10.1002/smr.493
Mulebeke, J. A., & Zheng, L. (2006, December). Analytical network process for software selection in product development: A case study. Journal of Engineering and Technology Management , 23 (4), 337–352. Retrieved 2013-01-23, from http://www.sciencedirect.com/science/article/pii/S0923474806000439 doi: 10.1016/j.jengtecman.2006.08.004
Nazir, S., Khan, M., Anwar, S., Khan, H., & Nazir, M. (2012, May). A novel fuzzy logic based software component selection modeling. In 2012 international conference on information science and applications (ICISA) (pp. 1 –6). doi: 10.1109/ICISA.2012.6220925
Olson, D. L., Johansson, B., & Carvalho, R. A. d. (2012, January). A combined method for evaluating criteria when selecting ERP systems. In C. Moller & S. Chaudhry (Eds.), Re-conceptualizing enterprise information systems (pp. 64–74). Springer Berlin Heidelberg.
Onut, S., & Efendigil, T. (2010). A theorical model design for ERP software selection process under the constraints of cost and quality: A fuzzy approach. Journal of Intelligent and Fuzzy Systems, 21 (6), 365–378.
Per¸cin, S. (2008). Using the ANP approach in selecting and benchmarking ERP systems. Benchmarking, 15 (5), 630–649.
Phaal, R., Farrukh, C. J. P., & Probert, D. R. (2001, August). Technology manage- ment process assessment: a case study. International Journal of Operations & Production Management , 21 (8), 1116–1132. doi: 10.1108/EUM0000000005588 Phaal, R., Farrukh, C. J. P., & Probert, D. R. (2004a). Customizing roadmapping.
Research Technology Management , 47 (2), 26–37.
Phaal, R., Farrukh, C. J. P., & Probert, D. R. (2004b). Technology roadmapping?A planning framework for evolution and revolution. Technological forecasting and social change, 71 (1), 5–26.
Razmi, J., & Sangari, M. (2008). A hybrid multi-criteria decision making model for ERP system selection. In Proceedings of the 2008 4th international conference on information and automation for sustainability, ICIAFS 2008 (pp. 489–495). Ruhe, G., & Saliu, M. (2005, December). The art and science of software release
70 6. Conclusions, Recommendations and Future Work.
Saaty, T., & Ozdemir, M. (2003, August). Why the magic number seven plus or
minus two. Mathematical and Computer Modelling , 38 (3–4), 233–244. doi:
10.1016/S0895-7177(03)90083-5
Saaty, T., & Vargas, L. G. (2006). Decision making with the analytic network process: Economic, political, social and technological applications with benefits, opportu- nities, costs and risks (Vol. 95). Springer.
Sarkis, J., & Talluri, S. (2004). Evaluating and selecting e-commerce soft-
ware and communication systems for a supply chain. European Journal
of Operational Research, 159 (2), 318–329. Retrieved 2012-10-15, from http://www.sciencedirect.com/science/article/pii/S0377221703005204 doi: 10.1016/j.ejor.2003.08.018
Sen, C., Bara¸cli, H., Sen, S., & Basligil, H. (2009). An integrated decision support sys- tem dealing with qualitative and quantitative objectives for enterprise software selection. Expert Systems with Applications, 36 (3 PART 1), 5272–5283.
Shih, Y.-Y. (2010). A study of ERP systems selection via fuzzy AHP method. In 2010 2nd international symposium on information engineering and electronic commerce, IEEC 2010 (pp. 333–336).
Shyur, H.-J. (2006, June). COTS evaluation using modified TOPSIS and ANP. Applied Mathematics and Computation, 177 (1), 251–259. Retrieved 2012-10-15, from http://www.sciencedirect.com/science/article/pii/S0096300305009070 doi: 10.1016/j.amc.2005.11.006
Suomalainen, T., Salo, O., Abrahamsson, P., & Simil¨a, J. (2011, June). Software product roadmapping in a volatile business environment. Journal of Systems and Software, 84 (6), 958–975. doi: 10.1016/j.jss.2011.01.031
Tolga, A. C. (2011). Fuzzy multi-criteria method for revaluation of ERP system choices using real options. Lecture Notes in Engineering and Computer Science, 2191 . Tun, T. T., Trew, T., Jackson, M., Laney, R., & Nuseibeh, B. (2009). Specifying
features of an evolving software system. Software: Practice and Experience, 39 (11), 973––1002. doi: 10.1002/spe.923
Uysal, F., & Tosun, O. (2012). Fuzzy TOPSIS-based computerized
maintenance management system selection. Journal of Manufactur-
ing Technology Management , 23 (2), 212–228. Retrieved 2013-01-23,
from http://www.emeraldinsight.com/10.1108/17410381211202205 doi:
10.1108/17410381211202205
Vlahavas, I., Stamelos, I., Refanidis, I., & Tsoukias, A. (1999). ESSE: an expert system for software evaluation. Knowledge-based systems, 12 (4), 183––197. Wei, C.-C., Chien, C.-F., & Wang, M.-J. (2005). An AHP-based approach to ERP
system selection. International Journal of Production Economics, 96 (1), 47–62.
Wei, G., & Lin, R. (2010). Models for selecting an ERP sys-
tem with intuitionistic trapezoidal fuzzy information. Jour-
References 71
http://www.ojs.academypublisher.com/index.php/jsw/article/ viewArticle/0503275279
Xu, Z. (2008, March). On multi-period multi-attribute decision making. Knowledge- Based Systems, 21 (2), 164–171. doi: 10.1016/j.knosys.2007.05.007
Xu, Z. (2009). Multi-period multi-attribute group decision-making under linguistic assessments. International journal of general systems, 38 (8), 823–850.
Yang, W. Z., Ge, Y. H., He, J. J., & Liu, B. (2010). Designing a group decision support system under uncertainty using group fuzzy analytic network process (ANP). Afr. J. Bus. Manage, 4 (12), 2571––2585.
Yazgan, H., Boran, S., & Goztepe, K. (2009). An ERP software selection process with using artificial neural network based on analytic network process approach. Expert Systems with Applications, 36 (5), 9214–9222.
Yeoh, H., & Miller, J. (2004, September). COTS acquisition process: incorporating business factors in COTS vendor evaluation taxonomy. In 10th international sym- posium on software metrics, 2004. proceedings (pp. 84 – 95). doi: 10.1109/MET- RIC.2004.1357893
Zadeh, L. (1965, June). Fuzzy sets. Information and Control , 8 (3), 338–353. doi: 10.1016/S0019-9958(65)90241-X
Zulzalil, H., Ghani, A. A. A., Selamat, M. H., & Mahmod, R. (2008).
A case study to identify quality attributes relationships for web-
based applications. IJCSNS , 8 (11), 215. Retrieved 2013-01-09, from