A constructive algorithm for unsupervised learning with incremental neural network

 

Jenq-Haur Wang, Hsin-Yang Wang, Yen-Lin Chen, Chuan-Ming Liu*

 

Department of Computer Science and Information Engineering, National Taipei University of Technology, Taipei, Taiwan. *Corresponding author. E-mail address: cmliu@csie.ntut.edu.tw

 

Abstract

Artificial neural network (ANN) has wide applications such as data processing and classification. However, comparing with other classification methods, ANN needs enormous memory space and training time to build the model. This makes ANN infeasible in practical applications. In this paper, we try to integrate the ideas of human learning mechanism with the existing models of ANN. We propose an incremental neural network construction framework for unsupervised learning. In this framework, a neural network is incrementally constructed by the corresponding subnets with individual instances. First, a subnet maps the relation between inputs and outputs for an observed instance. Then, when combining multiple subnets, the neural network keeps the corresponding abilities to generate the same outputs with the same inputs. This makes the learning process unsupervised and inherent in this framework.

In our experiment, Reuters-21578 was used as the dataset to show the effectiveness of the proposed method on text classification. The experimental results showed that our method can effectively classify texts with the best F1-measure of 92.5%. It also showed the learning algorithm can enhance the accuracy effectively and efficiently. This framework also validates scalability in terms of the network size, in which the training and testing times both showed a constant trend. This also validates the feasibility of the method for practical uses.

Keywords: Artificial neural network; Unsupervised learning; Text classification.

 

DESCARGAR ARTÍCULO EN FORMATO PDF

 

Acknowledgements

The authors would like to thank the support from National Science Council, Taiwan, under the grant number NSC102-2219-E-027-005.

 

References

Apte, C., Damerau, F., & Weiss, S.M. (1994). Automated learning of decision rules for text categorization. ACM T. Inform. Syst., 12, 233-251.         [ Links ]

Aran, O., & Alpaydin, E. (2003). An incremental neural network construction algorithm for training multilayer perceptrons. In Kaynak, O., Alpaydin, E., Oja, E., & Xu, L. (Eds.), Artificial Neural Networks and Neural Information Processing. Istanbul, Turkey: ICANN/ICONIP 2003.         [ Links ]

Bishop, C.M. (1996). Neural Networks for Pattern Recognition (1st ed.). New York: Oxford University Press.         [ Links ]

Cochocki, A., & Unbehauan, R. (1993). Neural networks for optimization and signal processing (1st ed.). New York: Wiley.         [ Links ]

Ghiassi, M., Olschimke, M., Moon, B., & Arnaudo, P. (2012). Automated text classification using a dynamic artificial neural network model. Expert Syst. Appl., 39, 10967-10976.         [ Links ]

Hebb, D.O. (1949). The Organization of Behavior. New York: Wiley & Sons.         [ Links ]

Herrero, J., Valencia, A., & Dopazo, J. (2001). A hierarchical unsupervised growing neural network for clustering gene expression patterns. Bioinformatics, 17, 126-136.         [ Links ]

Lee, H.M., Chen, C.M., & Hwang, C.W. (2000). A neural network document classifier with linguistic feature selection. In Loganantharaj, R., Palm, G., & Ali, M. (Eds.), Intelligent problem solving. methodologies and approaches (pp. 555-560). New York: Springer.         [ Links ]

Liu, Z.Q., & Zhang, Y. (2001). A competitive neural network approach to web-page categorization. Int. J. Uncertain. Fuzz., 9, 731-741.         [ Links ]

Rivera-Mejía, J., Léon-Rubio, A.G., & Arzabala-Contreras, E. (2012). PID based on a single artificial neural network algorithm for intelligent sensors. Journal of Applied Research and Technology, 10, 262-282.         [ Links ]

Rosenblatt, F. (1962). Principles of neurodynamics: perceptrons and the theory of brain mechanisms. Washington: Spartan Books.         [ Links ]

Rumelhart, D.E., Hinton, G.E., & McClelland, J.L. (1986). Learning representations by back-propagating errors. Nature, 323, 533-536.         [ Links ]

Shen, F., & Hasegawa, O. (2006). An incremental network for on-line unsupervised classification and topology learning. Neural Networks, 19, 90-106.         [ Links ]

Vukmirovic, S., Erdeljan, A., Imre, L., & Capko, D. (2012). Optimal workflow scheduling in critical infrastructure systems with neural networks. Journal of Applied Research and Technology, 10, 114-121.         [ Links ]

Zhang, M.L., & Zhou, Z.H. (2006). Multi-label neural networks with applications to functional genomics and text categorization. IEEE T. Knowl. Data En., 18, 1338-1351.         [ Links ]