Improving the Multilayer Perceptron Learning by Using a Method to Calculate the Initial Weights with the Similarity Quality Measure Based on Fuzzy Sets and Particle Swarms

Lenniet Coello¹, Yumilka Fernandez¹, Yaima Filiberto¹, Rafael Bello²

¹ Universidad de Camagüey, Department of Computer Sciences, Cuba. lenniet.coello@reduc.edu.cu, yumilka.fernandez@reduc.edu.cu, yaima.filiberto@reduc.edu.cu

² Universidad Central de Las Villas, Department of Computer Sciences, Cuba. rbellop@uclv.edu.cu

Corresponding author is Lenniet Coello.

Article received on 23/02/2015.
Accepted on 05/04/2015.

Abstract

The most widely used neural network model is Multilayer Perceptron (MLP), in which training of the connection weights is normally completed by a Back Propagation learning algorithm. Good initial values of weights bear a fast convergence and a better generalization capability even with simple gradient-based error minimization techniques. This work presents a method to calculate the initial weights in order to train the Multilayer Perceptron Model. The method named PSO+RST+FUZZY is based on the similarity quality measure proposed within the framework of the extended Rough Set Theory that employs fuzzy sets to characterize the domain of similarity thresholds. Sensitivity of BP to initial weights with PSO+RST+FUZZY was studied experimentally, showing better performance than other methods used to calculate feature weights.

Keywords: Multilayer perceptron, weight initialization, similarity quality measure, fuzzy sets.

DESCARGAR ARTÍCULO EN FORMATO PDF

1. Filiberto, Y., Bello, R., Caballero, Y., & Larrua, R. (2010). A method to build similarity relations into extended Rough Set Theory. 10th International Conference on Intelligent Systems Design and Applications (ISDA2010), Cairo, Egipt. DOI: 10.1109/ISDA.2010.5687091        [ Links ]

2. Filiberto, Y., Bello, R., Caballero, Y., & Frias, M. (2013). An analysis about the measure quality of similarity and its applications in machine learning. 4th International Workshop on Knowledge Discovery, Knowledge Management and Decision Support (EUREKA 2013), Mexico. DOI: 10.2991/.2013.16.         [ Links ]

3. Filiberto, Y., Bello, R., Caballero, Y., & Larrua, R. (2010) . Using PSO and RST to Predict the Resistant Capacity of Connections in Composite Structures. González, J.R., Pelta, D.A., Cruz, C., Terrazas, G., Krasnogor, N. (eds.) NICSO 2010, SCI, Vol. 284, pp. 359-370, Springer, Heidelberg. DOI: 10.1007/978-3-642-12538-6_30        [ Links ]

4. Fernandez, Y., Coello, L., Filiberto, Y., Bello, R., & Falcon, R. (2014). Learning Similarity Measures from Data with Fuzzy Sets and Particle Swarms. Electrical Engineering, Computing Science and Automatic Control (CCE), 11th International Conference, pp. 1-6, DOI: 10.1109/ICEEE.2014.6978261        [ Links ]

5. Filiberto, Y., Bello, R., Caballero, Y., & Larrua, R. (2011) . A measure in the rough set theory to decision systems with continuo features. Revista de la Facultad de Ingeniería de la Universidad Antioquia, No. 60, pp. 141-152.         [ Links ]

6. Mosqueda, R. (2010). Fallibility of the Rough Set Method in the formulation of a failure prediction index model of dynamic risk. Journal of Economics, Finance and Administrative Science, México.

7. Pawlak, Z. & Skowron, A. (2007). Rough sets: Some Extensions. Information Sciences, Vol. 177, pp. 28-40. DOI: 10.1016/j.ins.2006.06.006        [ Links ]

8. Slowinski, R. & Vanderpooten, D. (2000). A generalized definition of rough approximations based on similarity. IEEE Transactions on Data and Knowledge Engineering, Vol. 12, No. 2, pp. 331-336. DOI: 10.1109/69.842271        [ Links ]

9. Filiberto, Y., Bello, R., Caballero, Y., & Ramos, G. (2011). Improving the MLP Learning by Using a Method to Calculate the Initial Weights of the Network Based on the Quality of Similarity Measure. MICAI 2011. DOI: 10.1007/978-3-642-25330-0_31        [ Links ]

10. Bello, M., García, M., & Bello, R. (2013). A method for building prototypes in the nearest prototype approach based on similarity relations for problems of function approximation. LNCS, Vol. 7629, pp. 39-50. DOI: 10.1007/978-3-642-37807-2_4        [ Links ]

11. Filiberto, Y., Bello, R., Caballero, Y., Frias, & M. (2011). Algoritmo para el aprendizaje de reglas de clasificación basado en la teoría de los conjuntos aproximados extendida. DYNA, 78, pp. 62-70.         [ Links ]

12. Bratton, D. & Kennedy, J. (2007). Defining a Standard for Particle Swarm Optimization. IEEE Swarm Intelligence Symposium (SIS 2007). DOI: 10.1109/SIS.2007.368035        [ Links ]

13. Hussain, M. (2010). Fuzzy Relation. Thesis for the degree Master of Science in Mathematical Modelling and Simulation. Blekinge Institute of Technology School of Engineering.         [ Links ]

14. Zadeh, L.A. (1971). Similarity relations and fuzzy orderings. Information Sciences, Vol. 3 No. 2, pp. 177-200. DOI: 10.1016/S0020-0255(71)80005-1        [ Links ]

15. Bodenhofer, U. (2000). A similarity-based generalization of fuzzy orderings preserving the classical axioms. International Journal on Uncertainty and Fuzziness Knowledge-Based Systems, Vol. 8, No. 5, pp. 593-610. DOI: 10.1142/S0218488500000411        [ Links ]

16. Yang, M.S & Shih, H.M. (2001). Cluster analysis based on fuzzy relations. Fuzzy Sets and Systems, Vol. 120, pp. 197-212. DOI: 10.1016/S0165-0114(99)00146-3        [ Links ]

17. Verdegay, J.L., Yager, R.R., & Bonissone, P.P. (2008). On heuristics as a fundamental constituent of soft computing. Fuzzy Sets and Systems, Vol. 159, pp. 846- 855. DOI: 10.1016/j.fss.2007.08.014        [ Links ]

18. Cortez, P., Rocha, M., & Neves, J. (2005). Simultaneous Evolution of Neural Network Topologies and Weights for Classification and Regression. IWANN 2005, LNCS, Vol. 3512, pp. 59-66.         [ Links ]

19. Hocenski, Z., Antunoviae, M. & Filko, D. (2008). Accelerated Gradient Learning Algorithm for Neural Network Weights Update. LNCS, Vol. 5177, pp. 49-56. DOI: 10.1007/s00521-009-0286-7        [ Links ]

20. Fu, X., Zhang, S., & Pang, Z. (2010). A Resource Limited Immune Approach for Evolving Architecture and Weights of Multilayer Neural Network. LNCS, Vol. 6145, pp. 328-337. DOI: 10.1007/978-3-64213495-1 41        [ Links ]

21. Stavros A., Karras, D.A. & Vrahatis, M.N. (2009). Revisiting the Problem of Weight Initialization for Multi-Layer Perceptrons Trained with Back Propagation. LNCS, Vol. 5507, pp. 308-315. DOI: 10.1007/978-3-642-03040-6_38        [ Links ]

22. Kolen, J.F., & Pollack, J.B. (1991). Back propagation is sensitive to initial conditions. Advances in Neural Information Processing Systems, 3, Denver.         [ Links ]

23. Asuncion, A., & Newman, D. (2007). UCI machine learning repository. A study of the behavior of several methods for balancing machine learning training data. SIGKDD Explorations, Vol. 6, No. 1, pp. 20-29.         [ Links ]