Chapter 8 : Summary, Conclusion and Future Work
8.2 Future Work
Although this thesis carried out a comprehensive and rigorous study of the potential to use machine learning algorithms in the recognition of attacks in an imbalanced IDS related dataset, further research could have been conducted, time permitted. Below is a list of further directions of research that is recommended based on the research findings of this thesis:
It will be possible to fine-tune the feature selection algorithms utilised in this research by carrying out a sensitivity analysis of their parameters with the view to selecting the optimal set of parameters that will result eventually on optimisation of the classification accuracies.
The research conducted within the context of this thesis can be verified, further supported and enhanced through the application of deep learning.
The feature selection impact should be investigated via the adoption of other commonly used classifiers, such as random forest, for example.
Owing to a lack of Cloud-specific, attack datasets, there is a need to collect further useful data and making them publically available.
Chapter-7 conducted a comprehensive and rigorous study of the significance of attack features/parameters on the attack classification accuracy when different, popular classification approaches are used. For this purpose feature selection algorithms were used which provided the features in their rank order. The Rank order thus obtained provides vital features of an attack that could be used in understanding the unique characteristics of different kinds of attacks, leading to the possibility that this subject understanding can lead to interesting findings on how best to design a software system that will most efficient in the detection of network intrusions.
Page 114 of 146
References
[1] Cloud Security Alliance, “The Treacherous 12 - Top Threats to Cloud Computing + Industry Insights,” Cloud Secur. Alliance, p. 60, 2017.
[2] C. Modi, D. Patel, B. Borisaniya, A. Patel, and M. Rajarajan, “A survey on security issues and solutions at different layers of Cloud computing,” J. Supercomput., vol. 63, no. 2, pp. 561–592, 2013.
[3] Y. Chen and R. Sion, “On Securing Untrusted Clouds with Cryptography,” Science
(80-. )., pp. 109–114, 2010.
[4] V. Engen, “Machine Learning For Network Based Intrusion Detection,” Int. J., 2010. [5] M. Galar, A. Fern, E. Barrenechea, and H. Bustince, “Hybrid-Based Approaches,” vol.
42, no. 4, pp. 463–484, 2012.
[6] M. Galar, A. Fernandez, E. Barrenechea, H. Bustince, and F. Herrera, “A review on ensembles for the class imbalance problem: Bagging-, boosting-, and hybrid-based approaches,” IEEE Trans. Syst. Man Cybern. Part C Appl. Rev., vol. 42, no. 4, pp. 463–484, 2012.
[7] M. Vouk et al., “‘ Powered by VCL ’ - Using Virtual Computing Laboratory ( VCL ) Technology to Power Cloud Computing,” East, vol. 6, no. Vcl, pp. 1–10, 2008. [8] M. Armbrust et al., “A view of cloud computing,” Commun. ACM, vol. 53, no. 4, p.
50, 2010.
[9] M. Armbrust, a Fox, R. Griffith, A. Joseph, and Rh, “Above the clouds: A Berkeley view of cloud computing,” Univ. California, Berkeley, Tech. Rep. UCB , pp. 07–013, 2009.
[10] Q. Zhang, L. Cheng, and R. Boutaba, “Cloud computing: State-of-the-art and research challenges,” J. Internet Serv. Appl., vol. 1, no. 1, pp. 7–18, 2010.
[11] P. G. Tim Mell, “Draft NIST Working Definition of Cloud Computing,” Natl. Inst.
Stand. Technol., vol. 53, p. 50, 2009.
[12] R. Buyya, C. Vecchiola, and S. T. Selvi, Mastering Cloud Computing: Foundations
Page 115 of 146
[13] M. N. O. Sadiku, S. M. Musa, and O. D. Momoh, “Cloud computing: Opportunities and challenges,” IEEE Potentials, vol. 33, no. 1, pp. 34–36, 2014.
[14] Z. Erl, T., Puttini, R. & Mahmood, Cloud Computing: Concepts, Technology &
Architecture, vol. 1. Prentice Hall PTR., 2015.
[15] N. Leavitt, “Is cloud computing really ready for prime time?,” Comput. Soc. IEEE, vol. 42, no. 1, pp. 15–25, 2009.
[16] Microsoft TechNet, “Common Types of Network Attacks,” Microsoft Technet, vol. 959354. pp. 1–3, 2011.
[17] wikipedia, “Attack (computing) - Wikipedia, the free encyclopedia.” 2015.
[18] V. Cerf, H. Kong, Y. Dalal, C. Sunshine, and P. R. Net, “Denial-of-service attack - Wikipedia, the free encyclopedia,” North, 2009. [Online]. Available:
http://en.wikipedia.org/wiki/California_Eagle. [19] Wikipedia, “Denial-of-service attack - Wikipedia.” .
[20] “Article_ K14813 - Detecting and mitigating DoS_DDoS attacks (11.” .
[21] N. Gruschka and M. Jensen, “Attack surfaces: A taxonomy for attacks on cloud services,” Proc. - 2010 IEEE 3rd Int. Conf. Cloud Comput. CLOUD 2010, pp. 276– 279, 2010.
[22] D. D. I. Informatica, D. Di, R. In, and P. H. D. Thesis, “Cloud Computing Security , An Intrusion Detection System for Cloud Computing Systems Hesham Abdelazim Ismail Mohamed To the most precious inspiration of my life : My parents and my brothers and sisters,” 2013.
[23] C. N. Modi, D. R. Patel, A. Patel, and M. Rajarajan, “Integrating Signature Apriori based Network Intrusion Detection System (NIDS) in Cloud Computing,” Procedia
Technol., vol. 6, pp. 905–912, 2012.
[24] K. Patel and R. Srivastava, “Classification of Cloud Data using Bayesian Classification,” Int. J. Sci. Res., vol. 2, no. 6, pp. 2–7, 2013.
[25] R. Bace, “NIST special publication on intrusion detection systems,” Nist Spec. Publ., pp. 1–51, 2001.
Page 116 of 146
[26] T. W. Purboyo, B. Rahardjo, and Kuspriyanto, “Security metrics: A brief survey,”
2011 Int. Conf. Instrumentation, Commun. Inf. Technol. Biomed. Eng., no. November,
p. 4, 2011.
[27] R. M. Savola, “A Security Metrics Taxonomization Model for Software-Intensive Systems,” J. Inf. Process. Syst., vol. 5, no. 4, pp. 197–206, 2009.
[28] J. Arshad, P. Townend, and J. Xu, “A novel intrusion severity analysis approach for Clouds,” Futur. Gener. Comput. Syst., vol. 29, no. 1, pp. 416–428, 2013.
[29] P. a Porras and P. G. Neumann, “EMERALD: Event Monitoring Enabling Responses to Anomalous Live Disturbances,” Proc. 20th NIST-{NCSC} Natl. Inf. Syst. Secur.
Conf., pp. 353–365, 1997.
[30] H. a. Kholidy and F. Baiardi, “CIDS: A framework for intrusion detection in cloud systems,” Proc. 9th Int. Conf. Inf. Technol. ITNG 2012, pp. 379–385, 2012.
[31] S. Shamshirband, N. B. Anuar, M. L. M. Kiah, and A. Patel, “An appraisal and design of a multi-agent system based cooperative wireless intrusion detection computational intelligence technique,” Eng. Appl. Artif. Intell., vol. 26, no. 9, pp. 2105–2127, 2013. [32] H. T. Elshoush and I. M. Osman, “Alert correlation in collaborative intelligent
intrusion detection systems - A survey,” Appl. Soft Comput. J., vol. 11, no. 7, pp. 4349–4365, 2011.
[33] R. Lippmann, J. W. Haines, D. J. Fried, J. Korba, and K. Das, “The 1999 DARPA o € -line intrusion detection evaluation,” Comput. Networks, vol. 34, no. 4, pp. 579–595, 2000.
[34] S. Roschke, F. Cheng, and C. Meinel, “Intrusion detection in the cloud,” 8th IEEE Int.
Symp. Dependable, Auton. Secur. Comput. DASC 2009, pp. 729–734, 2009.
[35] K. Scarfone and P. Mell, “Guide to Intrusion Detection and Prevention Systems ( IDPS ) Recommendations of the National Institute of Standards and Technology,” Nist Spec.
Publ., vol. 800, p. 94, 2007.
[36] S. Roschke, F. Cheng, and C. Meinel, “An extensible and virtualization-compatible IDS management architecture,” 5th Int. Conf. Inf. Assur. Secur. IAS 2009, vol. 2, pp. 130–134, 2009.
Page 117 of 146
[37] B. Tjaden et al., “INBOUNDS: The Integrated Network-Based Ohio University Network Detective,” 2000.
[38] M. Dacier and A. Wespi, “Towards a taxonomy of intrusion-detection systems,” 1999. [39] P. Helman and G. Liepins, “Statistical Foundations of Audit Trail Analysis for the
Detection of Computer Misuse,” IEEE Trans. Softw. Eng., vol. 19, no. 9, pp. 886–901, 1993.
[40] H. Debar, M. Dacier, and A. Wespi, “Towards a taxonomy of intrusion-detection systems,” Comput. Networks, vol. 31, pp. 805–822, 1999.
[41] T. M. Mitchell, “The Discipline of Machine Learning,” Mach. Learn., vol. 17, no. July, pp. 1–7, 2006.
[42] R. Grossman, S. Kasif, R. Moore, D. Rocke, and J. Ullman, “Data Mining Research: Opportunities and Challenges,” vol. 1998, 1999.
[43] S. P. Portillo, “PhD Thesis Attacks Against Intrusion Detection Networks : Evasion , Reverse Engineering and Optimal Countermeasures,” no. June, 2014.
[44] L. Long, X. Wang, and X. Zhu, “Machine Learning in Network Intrusion Detection,” vol. 11, no. 2, p. 9941, 2015.
[45] R. Sommer and V. Paxson, “Outside the Closed World: On Using Machine Learning for Network Intrusion Detection,” pp. 305–316, 2010.
[46] S. N. S. Naiping and Z. G. Z. Genyuan, “A Study on Intrusion Detection Based on Data Mining,” Inf. Sci. Manag. Eng. ISME 2010 Int. Conf., vol. 1, pp. 8–15, 2010. [47] C. Thomas and N. Balakrishnan, “Performance enhancement of Intrusion Detection
Systems using advances in sensor fusion,” Inf. Fusion, 2008 11th Int. Conf., pp. 1–7, 2008.
[48] M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani, “A detailed analysis of the KDD CUP 99 data set,” IEEE Symp. Comput. Intell. Secur. Def. Appl. CISDA 2009, no. Cisda, pp. 1–6, 2009.
[49] J. McHugh, “Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory,”
Page 118 of 146
ACM Trans. Inf. Syst. Secur., vol. 3, no. 4, pp. 262–294, 2000.
[50] C. Thomas and N. Balakrishnan, “Performance enhancement of Intrusion Detection Systems using advances in sensor fusion,” 2008 11th Int. Conf. Inf. Fusion, pp. 1–7, 2008.
[51] M. Tavallaee, “An Adaptive Intrusion Detection System,” Sdstate.Edu, 2011.
[52] V. Engen, J. Vincent, and K. Phalp, “Exploring discrepancies in findings obtained with the KDD Cup ’99 data set,” Intell. Data Anal., vol. 15, no. 2, pp. 251–276, 2011. [53] M. M. Andreasen, C. T. Hansen, and P. Cash, “Good Design,” Concept. Des., vol. 45,
no. 21, pp. 369–389, 2015.
[54] M. Jouini, L. B. A. Rabai, and A. Ben Aissa, “Classification of security threats in information systems,” Procedia Comput. Sci., vol. 32, pp. 489–496, 2014.
[55] S. S. Kaushik and P. R. Deshmukh, “Detection of Attacks in an Intrusion Detection System,” Int. J. Comput. Sci. Inf. Technol., vol. 2, no. 3, pp. 982–986, 2011.
[56] J. Postel and J. Reynolds, “FILE TRANSFER PROTOCOL (FTP),” 2011. . [57] M. Rouse, “What is IMAP (Internet Message Access Protocol)? - Definition from
WhatIs.com.” 2015.
[58] D. Dey, A. DInda, P. P. Kundapur, and R. Smitha, “Warezmaster and Warezclient: An implementation of FTP based R2L attacks,” 8th Int. Conf. Comput. Commun. Netw.
Technol. ICCCNT 2017, pp. 6–11, 2017.
[59] S. Akasapu, “An Integrated Approach for detecting DDoS attacks in Cloud Computing,” no. June, pp. 258–261, 2017.
[60] A. S. Janusz S. Kowalik, Janusz Gorski, Cyberspace Security and Defense: Research
Issues. 2006.
[61] VERACODE, “Rootkit: What is a Rootkit and How to Detect It | Veracode.” 2017. [62] T. Tran, P. Tsai, T. Jan, and X. Kong, “Network Intrusion Detection using Machine
Learning and Voting techniques,” Mach. Learn., pp. 7–10, 2010.
Page 119 of 146
by cost-sensitive neural network methods,” Sci. Total Environ., vol. 407, no. 6, pp. 2124–2135, 2009.
[64] M. Troesch and I. Walsh, “Machine Learning for Network Intrusion Detection,” pp. 1– 5, 2014.
[65] S. Juma, Z. Muda, M. M.A., and W. Yassin, “Machine Learning Techniques for Intrusion Detection System: A Review,” J. Theor. Appl. Inf. Theory, vol. 72, no. 3, pp. 422–429, 2015.
[66] M. Panda, A. Abraham, S. Das, and M. R. Patra, “Network intrusion detection system: A machine learning approach,” Intell. Decis. Technol., vol. 5, no. 4, pp. 347–356, 2011.
[67] M. Kubat, “Neural networks: a comprehensive foundation by Simon Haykin,
Macmillan, 1994, ISBN 0-02-352781-7.,” The Knowledge Engineering Review, vol. 13, no. 4. pp. 409–412, 1999.
[68] Y. A. LeCun, L. Bottou, G. B. Orr, and K. R. M??ller, “Efficient backprop,” Lect.
Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 7700 LECTU, pp. 9–48, 2012.
[69] V. López, A. Fernández, S. García, V. Palade, and F. Herrera, “An insight into
classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics,” Inf. Sci. (Ny)., vol. 250, pp. 113–141, 2013.
[70] N. V Chawla, “Data Mining for Imbalanced Datasets: An Overview,” Data Min.
Knowl. Discov. Handb., pp. 853–867, 2005.
[71] G. M. Weiss, “Mining with Rarity: A Unifying Framework,” SIGKDD Explor., vol. 6, no. 1, pp. 7–19, 2004.
[72] S. Kotsiantis, D. Kanellopoulos, and P. Pintelas, “Handling imbalanced datasets : A review,” Science (80-. )., vol. 30, no. 1, pp. 25–36, 2006.
[73] G. M. Weiss and F. Provost, “Learning when training data are costly: The effect of class distribution on tree induction,” J. Artif. Intell. Res., vol. 19, pp. 315–354, 2003. [74] A. H. R. Ko, R. Sabourin, and A. S. Britto, “From dynamic classifier selection to
Page 120 of 146
dynamic ensemble selection,” Pattern Recognit., vol. 41, no. 5, pp. 1735–1748, 2008. [75] R. Batuwita and V. Palade, “microPred: Effective classification of pre-miRNAs for
human miRNA gene prediction,” Bioinformatics, vol. 25, no. 8, pp. 989–995, 2009. [76] H.-Y. Lo et al., “Learning to improve area-under-FROC for imbalanced medical data
classification using an ensemble method,” ACM SIGKDD Explor. Newsl., vol. 10, no. 2, p. 43, 2008.
[77] G. Cohen, M. Hilario, H. Sax, S. Hugonnet, and A. Geissbuhler, “Learning from imbalanced data in surveillance of nosocomial infection,” Artif. Intell. Med., vol. 37, no. 1, pp. 7–18, 2006.
[78] L. Mena and J. a Gonzalez, “Machine learning for imbalanced datasets: Application in medical diagnostic,” Breast, pp. 574–579, 2006.
[79] A. Al-Shahib, R. Breitling, and D. R. Gilbert, “Predicting protein function by machine learning on amino acid sequences--a critical evaluation.,” BMC Genomics, vol. 8, p. 78, 2007.
[80] Ł. Kobyliński and A. Przepiórkowski, “Definition extraction with balanced random forests,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect.
Notes Bioinformatics), vol. 5221 LNAI, pp. 237–247, 2008.
[81] K. Kermanidis, M. Maragoudakis, N. Fakotakis, and G. Kokkinakis, “Learning Greek Verb Complements : Addressing the Class Imbalance,” no. Laurikkala 2001, 2002. [82] E. Stamatatos, “Author identification: Using text sampling to handle the class
imbalance problem,” Inf. Process. Manag., vol. 44, no. 2, pp. 790–799, 2008. [83] D. A. Cieslak, N. V. Chawla, and A. Striegel, “Combating imbalance in network
intrusion datasets,” 2006 IEEE Int. Conf. Granul. Comput., pp. 732–737, 2006.
[84] R. Barandela, J. S. Sanchez, V. Garcia, and E. Rangel, “Strategies for learning in class imbalance problems.pdf,” Pattern Recog., vol. 36, pp. 849–851, 2003.
[85] P. Ducange, B. Lazzerini, and F. Marcelloni, “Multi-objective genetic fuzzy classifiers for imbalanced and cost-sensitive datasets,” Soft Comput., vol. 14, no. 7, pp. 713–728, 2010.
Page 121 of 146
[86] W. J. Lin and J. J. Chen, “Class-imbalanced classifiers for high-dimensional data,”
Brief. Bioinform., vol. 14, no. 1, pp. 13–26, 2013.
[87] J. Wang, J. You, Q. Li, and Y. Xu, “Extract minimum positive and maximum negative features for imbalanced binary classification,” Pattern Recognit., vol. 45, no. 3, pp. 1136–1145, 2012.
[88] R. Batuwita and V. Palade, “Class Imbalance Learning Methods for Support Vector,”
Imbalanced Learn. Found. Algorithms, Appl., pp. 83–100, 2013.
[89] N. García-Pedrajas, J. Pérez-Rodríguez, M. García-Pedrajas, D. Ortiz-Boyer, and C. Fyfe, “Class imbalance methods for translation initiation site recognition in DNA sequences,” Knowledge-Based Syst., vol. 25, no. 1, pp. 22–34, 2012.
[90] P. Domingos, “MetaCost: A General Method for Making Classifiers Cost-Sensitive,”
Proc. fifth ACM SIGKDD Int. Conf. Knowl. Discov. data Min., vol. 55, pp. 155–164,
1999.
[91] Z. H. Zhou and X. Y. Liu, “Training cost-sensitive neural networks with methods addressing the class imbalance problem,” IEEE Trans. Knowl. Data Eng., vol. 18, no. 1, pp. 63–77, 2006.
[92] J. Błaszczyński, M. Deckert, J. Stefanowski, and S. Wilk, “Integrating selective pre- processing of imbalanced data with Ivotes ensemble,” Lect. Notes Comput. Sci.
(including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 6086
LNAI, pp. 148–157, 2010.
[93] N. V Chawla, A. Lazarevic, L. O. Hall, and K. W. Bowyer, “SMOTEBoost : Improving Prediction,” Lect. Notes Comput. Sci., vol. 2838, pp. 107–119, 2003. [94] C. Seiffert, T. M. Khoshgoftaar, J. Van Hulse, and A. Napolitano, “RUSBoost: A
hybrid approach to alleviating class imbalance,” IEEE Trans. Syst. Man, Cybern. Part
ASystems Humans, vol. 40, no. 1, pp. 185–197, 2010.
[95] R. Batuwita and V. Palade, “Efficient resampling methods for training support vector machines with imbalanced datasets,” Proc. Int. Jt. Conf. Neural Networks, 2010. [96] A. Fernández, M. J. del Jesus, and F. Herrera, “On the 2-tuples based genetic tuning
Page 122 of 146
Sci. (Ny)., vol. 180, no. 8, pp. 1268–1291, 2010.
[97] A. Fernandez, S. Garc??a, M. J. del Jesus, and F. Herrera, “A study of the behaviour of linguistic fuzzy rule based classification systems in the framework of imbalanced data- sets,” Fuzzy Sets Syst., vol. 159, no. 18, pp. 2378–2398, 2008.
[98] N. Japkowicz, “The Class Imbalance Problem: Significance and Strategies,” Proc.
2000 Int. Conf. Artif. Intell., pp. 111--117, 2000.
[99] J. Van Hulse, “An Empirical Comparison of Repetitive Undersampling Techniques,” pp. 29–34, 2009.
[100] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: Synthetic minority over-sampling technique,” J. Artif. Intell. Res., vol. 16, pp. 321–357, 2002. [101] J. Burez and D. Van den Poel, “Handling class imbalance in customer churn
prediction,” Expert Syst. Appl., vol. 36, no. 3 PART 1, pp. 4626–4636, 2009.
[102] A. S. Nickerson, N. Japkowicz, and E. Milios, “Using Unsupervised Learning to Guide Resampling in Imbalanced Data Sets,” Proc. Eighth Int. Work. AI Statitsics, p. 5, 2001. [103] A. Estabrooks and N. Japkowicz, “A mixture-of-experts framework for learning from
imbalanced data sets,” Adv. Intell. Data Anal., pp. 34–43, 2001.
[104] K. Yoon and S. Kwek, “An unsupervised learning approach to resolving the data imbalanced issue in supervised learning problems in functional genomics,” Proc. - HIS
2005 Fifth Int. Conf. Hybrid Intell. Syst., vol. 2005, pp. 303–308, 2005.
[105] S. J. Yen and Y. S. Lee, “Under-sampling approaches for improving prediction of the minority class in an imbalanced dataset,” Lect. Notes Control Inf. Sci., vol. 344, pp. 731–740, 2006.
[106] Q. Wang, “A Hybrid Sampling SVM Approach to,” vol. 2014.
[107] N. V. Chawla, “C4. 5 and imbalanced data sets: investigating the effect of sampling method, probabilistic estimate, and decision tree structure,” Proc. Int. Conf. Mach.
Learn. Work. Learn. from Imbalanced Data Set II, 2003.
[108] S. Choudhury and A. Bhowal, “Comparative analysis of machine learning algorithms along with classifiers for network intrusion detection,” 2015 Int. Conf. Smart Technol.
Page 123 of 146
Manag. Comput. Commun. Control. Energy Mater., no. May, pp. 89–95, 2015.
[109] G. Weiss, K. McCarthy, and B. Zabar, “Cost-sensitive learning vs. sampling: Which is best for handling unbalanced classes with unequal error costs?,” Dmin, pp. 1–7, 2007. [110] D. Adamu Teshome and V. S. Rao, “A Cost Sensitive Machine Learning Approach for
Intrusion Detection,” Glob. J. Comput. Sci. Technol., vol. 14, no. 6, 2014.
[111] J. Liu, Q. Hu, and D. Yu, “A weighted rough set based method developed for class imbalance learning,” Inf. Sci. (Ny)., vol. 178, no. 4, pp. 1235–1256, 2008.
[112] N. V Chawla, “Data Mining for Imbalanced Datasets: An Overview,” Data Min.
Knowl. Discov. Handb., no. January 2005, pp. 853–867, 2005.
[113] H. He and E. A. Garcia, “Learning from imbalanced data,” IEEE Trans. Knowl. Data
Eng., vol. 21, no. 9, pp. 1263–1284, 2009.
[114] J. Van Hulse, T. M. Khoshgoftaar, and A. Napolitano, “Experimental perspectives on learning from imbalanced data,” Proc. 24th Int. Conf. Mach. Learn., pp. 935–942, 2007.
[115] X.-Y. Liu, J. Wu, and Z.-H. Zhou, “Exploratory Undersampling for Class Imbalance Learning,” IEEE Trans. Syst. Man Cybern., vol. 39, no. 2, pp. 539–550, 2009.
[116] S. Wang and X. Yao, “Relationships between diversity of classification ensembles and single-class performance measures,” IEEE Trans. Knowl. Data Eng., vol. 25, no. 1, pp. 206–219, 2013.
[117] C. Seiffert, T. M. Khoshgoftaar, J. Van Hulse, and A. Napolitano, “2010-IEEE
TSMCpartA-RUSBoost A Hybrid Approach to Alleviating Class Imbalance.pdf,” vol. 40, no. 1, p. 13, 2010.
[118] R. Barandela, J. S. Sánchez, and R. M. Valdovinos, “New Applications of Ensembles of Classifiers,” Pattern Anal. Appl., vol. 6, no. 3, pp. 245–256, 2003.
[119] F. Morstatter and Z. Zheng, “Advancing Feature Selection Research − ASU Feature Selection Repository,” 2010.
[120] B. L. Guedes, E. B. Orzolin, and K. D. Keller, “Selection of Relevant Features in Machine Learning,” AAAI Tech. Rep. ., pp. 127–131, 2014.
Page 124 of 146
[121] G. H. John, R. Kohavi, and Karl Pfleger, “Irrelevant Features and the Subset Selection Problem,” Icml, pp. 121–129, 1994.
[122] Z. L. Sun, D. S. Huang, and Y. M. Cheun, “Extracting nonlinear features for
multispectral images by FCMC and KPCA,” Digit. Signal Process. A Rev. J., vol. 15, no. 4, pp. 331–346, 2005.
[123] J. L. Crowley and A. C. Parker, “A Representation for Shape Based on Peaks and Ridges in the Difference of Low-Pass Transform,” IEEE Trans. Pattern Anal. Mach.
Intell., vol. PAMI-6, no. 2, pp. 156–170, 1984.
[124] Z. L. Sun, D. S. Huang, Y. M. Cheung, J. Liu, and G. Bin Huang, “Using FCMC, FVS, and PCA techniques for feature extraction of multispectral images,” IEEE
Geosci. Remote Sens. Lett., vol. 2, no. 2, pp. 108–112, 2005.
[125] A. Khotanzad and Y. H. Hong, “Rotation invariant image recognition using features selected via a systematic method,” Pattern Recognit., vol. 23, no. 10, pp. 1089–1101, 1990.
[126] N. Vasconcelos, “Feature selection by maximum marginal diversity: optimality and implications for visual recognition,” 2003 IEEE Comput. Soc. Conf. Comput. Vis.