Multi-Objective Feature Subset Selection using Non-dominated Sorting Genetic Algorithm

 

A. Khan*¹ and A. R. Baig²

 

¹ National University of Computers and Emerging Sciences Islamabad, Pakistan.

² College of Computer and Information Sciences, Al Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh, Saudi Arabia. *ayeshak1417@yahoo.com

 

ABSTRACT

This paper presents an evolutionary algorithm based technique to solve multi-objective feature subset selection problem. The data used for classification contains large number of features called attributes. Some of these attributes are not relevant and needs to be eliminated. In classification procedure, each feature has an effect on the accuracy, cost and learning time of the classifier. So, there is a strong requirement to select a subset of the features before building the classifier. This proposed technique treats feature subset selection as multi-objective optimization problem. This research uses one of the latest multi-objective genetic algorithms (NSGA - II). The fitness value of a particular feature subset is measured by using ID3. The testing accuracy acquired is then assigned to the fitness value. This technique is tested on several datasets taken from the UCI machine repository. The experiments demonstrate the feasibility of using NSGA-II for feature subset selection.

Keywords: Optimization, genetic algorithm, classification, Feature subset selection.

 

DESCARGAR ARTÍCULO EN FORMATO PDF

 

References

[1] A. P. Engelbrecht, "Computational Intelligence - An Introduction", John Wiley & Sons Inc., NJ, USA 2002.         [ Links ]

[2] C. A. Coello, "Handling multiple objectives with particle swarm optimization", IEEE Transactions on Evolutionary Computation, vol. 8, no. 3, Jun 2004.         [ Links ]

[3] D. W. Aha, and R. L. Banket, "A comparative evaluation of sequential feature selection algorithms" in Proceedings of the 5th International Workshop on Artificial Intelligence and Statistics, pp 1-7, Menlo Park, CA, USA. 1994.         [ Links ]

[4] G. H. John, R. Kohavi, and K. Pfleger, "Irrelevant features and the subset selection problem", in Proceedings of the Eleventh International Conference on Machine learning, pages 121-129, New Brunswick, NJ, 1994.         [ Links ]

[5] H. Vafaie, and K. D. Jong, "Genetic algorithms as a tool for feature selection in machine learning," in Center for Artificial Intelligence, George Mason University, 1992.         [ Links ]

[6] H. Vafaie, and K. D. Jong, "Genetic algorithms as a tool for restructuring feature space representations", Computer Science Department, George Mason University Fairfax, USA, 1995.         [ Links ]

[7] I. S Oh, J. S. Lee, and B. R. Moon, "Hybrid genetic algorithms for feature selection", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 11, Nov 2004.         [ Links ]

[8] J. D. Shaffer, "Multiple objective optimization with vector evaluated genetic algorithms", in Proceedings of an International Conference on Genetic Algorithms and their Applications, Pittsburgh, PA, Jul 1985.         [ Links ]

[9] J. Han and M. Kamber, "Data Mining Concept and Techniques", Morgan Kaufmann Publishers, San Francisco, CA, USA, 2001.         [ Links ]

[10] J. Horn and N. Nafpliotis, "Multi-objective using the niched pareto genetic algorithm" IlliGAL Report 93005, Illinois Genetic Algorithms Laboratory, University of Illinois, Urbana, Champaign, Jul 1993.         [ Links ]

[11] J. Horn, N. Nafpliotis, and D. E. Goldberg, "A niched pareto genetic algorithm for multi-objective optimization", in Proceedings of the First IEEE Conference on Evolutionary Computation, IEEE World Congress on Computational Computation, Piscataway, NJ, Jun 1994.         [ Links ]

[12] J. Yang, and V. Honavar, "Feature subset selection using a genetic algorithm" IEEE Intelligent Systems, vol. 13, no. 2, 1998.         [ Links ]

[13] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, "A fast elitist multi- objective genetic algorithm: NSGA-II", Evolutionary Computation, vol. 2, 1995.         [ Links ]

[14] K. Deb, "Multi-objective optimization using evolutionary algorithms", Reading, John Wiley & Sons, Ltd, Reprinted in April 2002.         [ Links ]

[15] M. L. Raymer, W. F. Punch, E. D. Goodman, L. A. Kuhn, and A.K. Jain, "Dimensionality reduction using genetic algorithms", in IEEE Transactions on Evolutionary Computation, vol. 4, no. 2, Jul 2000.         [ Links ]

[16] N. Srinivas, and K. Deb, "Multi-objective optimization using non-dominated sorting in genetic algorithms", in IEEE Transactions on Evolutionary Computation, 1994.         [ Links ]

[17] P. Hajela, and C.Y.Lin, "Genetic search strategies in multi-criterion optimal design," Structural Optimization, vol. 4, June 1992.         [ Links ]

[18] S. M. Weiss, and C.A. Kulikowski, "Computer Systems That Learn", Morgan Kaufmann, 1991.         [ Links ]

[19] Datasets from the "University of Irvine" http://archive.ics.uci.edu/ml/.         [ Links ]

[20] T. Takahama, S. Sakai, A. Hara and N. Iwane, "Predicting stock price using neural networks optimized by differential evolution with degeneration", International Journal of Innovative Computing, Information and Control, vol.5, no. 12(B), pp.5021-5032, 2009.         [ Links ]

[21] T. Uno, H. Katagiri and K. Kato, "An evolutionary multi-agent based search method for stackelberg solutions of bilevel facility location problems", International Journal of Innovative Computing, Information and Control, vol. 4, no. 5, pp. 1033-1042, 2008.         [ Links ]

[22] R. Kuo, T. Hu and Z. Chen, "Evolutionary algorithm-based RBF neural network for oil price forecasting", ICIC Express Letters, vol. 3, no. 3 (B), pp. 701-706, 2009.         [ Links ]

[23] C. Liu, "An evolutionary algorithm for solving dynamic non-linear constrained optimization", ICIC Express Letters, vol. 4, no. 3(B),pp. 1039-1044, 2010.         [ Links ]

[24] M. Nazir, A. M. Mirza, S. A. Khan, "PSO-GA Based Optimized Feature Selection Using Facial and Clothing Information for Gender Classification" Journal of Applied Research and Technology, vol. 12 , no. 1, pp. 145 - 152, 2014.         [ Links ]

[25] F. Yaman, A. E. Yilmaz, "Impacts of Genetic Algorithm Parameters on the Solution Performance for the Uniform Circular Antenna Array Pattern Synthesis Problem" Journal of Applied Research and Technology, vol. 8, no. 3, pp. 378394, 2010.         [ Links ]

[26] A. Belloufi, M. Assas, I. Rezgui, "Optimization of Turning Operations by Using a Hybrid Genetic Algorithm with Sequential Quadratic Programming" Journal of Applied Research and Technology, vol. 11, no. 1, pp. 88-94, 2013.         [ Links ]

[27] B. Bernabe-Loranca, C. A. Coello-Coello, M. OsorioLama, "A Multi-objective Approach for the Heuristic Optimization of Compactness and Homogeneity in the Optimal Zoning" Journal of Applied Research and Technology, vol. 10, no. 1, pp. 447-457, 2012.         [ Links ]