Aprendiendo con detección de cambio online
Learning with Online Drift Detection
Isvani Frías Blanco1, José del Campo Ávila2, Gonzalo Ramos Jiménez2, Rafael Morales Bueno2, Agustín Ortiz Díaz3 y Yailé Caballero Mota4
1 Universidad de las Ciencias Informáticas, Cuba. Ifriasb@grm.uci.cu
]]> 2 Universidad de Málaga, España. jcampo@lcc.uma.es, ramos@lcc.uma.es, morales@lcc.uma.es3 Universidad de Granma, Cuba. aortizd@grm.uci.cu
4 Universidad de Camagüey, Cuba. yailec@yahoo.com
Resumen
En la actualidad, muchas fuentes generan grandes cantidades de datos en largos períodos de tiempo, requiriéndose su procesamiento incremental. Debido a la dimensión temporal de estos datos, un modelo de aprendizaje inducido previamente puede ser inconsistente con los datos actuales, problema comúnmente conocido como cambio de concepto. Una estrategia ampliamente usada para detectar cambio de concepto supervisa a lo largo del tiempo alguna medida de rendimiento del modelo. Si se estima un deterioro significativo del modelo mediante dicha medida se ejecutan algunas acciones para adaptar el aprendizaje. En este sentido, en el presente artículo se propone un nuevo método para detectar cambio de concepto no dependiente del algoritmo de aprendizaje. Se usa la inecuación de probabilidad de Hoeffding para ofrecer garantías probabilísticas de detección de cambios en la media de flujos de valores reales. Dicho método se basa en la comparación de medias correspondientes a dos muestras, mediante la identificación de un único punto de corte relevante en dicha secuencia de valores reales; manteniendo así un número fijo de contadores además con complejidad temporal constante. Evaluaciones empíricas preliminares considerando conocidos flujos de datos, diferentes detectores de cambio de concepto y algoritmos de aprendizaje muestran promisorio el método propuesto.
Palabras clave: Aprendizaje incremental, cambio de concepto, cota de Hoeffding, detección de cambio de concepto, flujos de datos.
Abstract
Learning in data streams is a problem of growing interest. The target function of data streams may change over time, so in such situations, a learning model induced with some previous data may be inconsistent with the current data. This problem is commonly known as concept drift. The strategy broadly used to handle concept drift is to continuously monitor a chosen performance measure of the model over time; if the model performance drops, adequate actions are executed to adapt the model. Taking this into account, our paper proposes a new method to detect drifting concepts, which is independent of the learning algorithm. We use a probability inequality (Hoeffding's inequality) to offer probabilistic guarantees for the detection of significant changes in the mean of real values. The detection is based on the comparison of averages corresponding to two samples by means of identification of a single relevant cut-point in this sequence of real values maintaining a fixed number of counters and with constant time complexity. As some previous approaches, our method is based on ideas of statistical process control. Preliminary empirical evaluations considering well-known data streams, change detectors and various classifiers reveal advantages of the proposed method.
]]> Keywords: Incremental learning, concept drift, concept drift detection, control chart, data stream, Hoeffding's bound.
DESCARGAR ARTÍCULO EN FORMATO PDF
Referencias
1. Agrawal, R., Imielinski, T., & Swami, A. (1993). Database mining: A performance perspective. IEEE Transaction on Knowledge and Data Engineering, 5(6), 914-925. [ Links ]
2. Aha, D.W., Kibler, D., & Albert, M.K. (1991). Instance-based learning algorithms. Machine Learning, 6(1), 37-66. [ Links ]
3. Babcock, B., Babu, S., Datar, M., Motwani, R., & Widom, J. (2002). Models and issues in data stream systems. Twenty-first ACM SIGMOD-SIGACT-SIGART Symposium on Principles of database systems (PODS02), Madison, Wisconsin, USA, 1-16. [ Links ]
4. Baena-García, M., del Campo-Ávila, J., Fidalgo, R., Bifet, A., Gavaldà, R., & Morales-Bueno, R. (2006). Early Drift Detection Method. Fourth International Workshop on Knowledge Discovery from Data Streams. [ Links ]
5. Basseville, M. & Nikiforov, I.V. (1993). Detection of Abrupt Changes: Theory and Application. Englewood Cliffs, NJ: Prentice-Hall. [ Links ]
6. Beringer, J. & Hüllermeier, E. (2007). Efficient instance-based learning on data streams. Intelligent Data Analysis, 11(6), 627-650. [ Links ]
7. Bifet, A. & Gavaldà, R. (2007). Learning from time-changing data with adaptive windowing. 2007 SIAM International Conference on Data Mining, Minneapolis, Minnesota, 443-448. [ Links ]
8. Bifet, A. & Gavaldá, R. (2009). Adaptive learning from evolving data streams. 8th International Symposium on Intelligent Data Analysis: Advances in Intelligent Data Analysis VIII (IDA '09), Lyon, France, 249-260. [ Links ]
9. Bifet, A., Holmes, G., Kirkby, R., & Pfahringer, B. (2010). MOA: Massive Online Analysis. Journal of Machine Learning Research, 11(2010), 1601-1604. [ Links ]
10. Bifet, A., Holmes, G., Pfahringer, B., & Frank, E. (2010). Fast perceptron decision tree learning from evolving data streams. 14th Pacific-Asia conference on Advances in Knowledge Discovery and Data Mining - Volume Part II (PAKDD'10), Hyderabad, India, 299-310. [ Links ]
11. Bifet, A., Holmes, G., Pfahringer, B., Kirkby, R., & Gavaldà, R. (2009). New ensemble methods for evolving data streams. 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '09), Paris, France, 139-148. [ Links ]
12. del Campo-Ávila, J., Ramos-Jiménez, G., Gama, J., & Morales-Bueno, R. (2008). Improving the performance of an incremental algorithm driven by error margins. Intelligent Data Analysis-Knowledge Discovery from Data Streams, 12(3), 305-318. [ Links ]
13. Chernoff, H. (1952). A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the Sum of Observations. Annals of Mathematical Statistics, 23(4), 493-507. [ Links ]
14. Cunningham, P., Nowlan, N., Delany, S.J., & Haahr, M. (2003). A case-based approach to spam filtering that can track concept drift. ICCBR'2003 Workshop on Long-Lived CBR Systems, Trondheim, Norway. [ Links ]
15. Datar, M., Gionis, A., Indyk, P., & Motwani, R. (2002). Maintaining stream statistics over sliding windows. SIAM Journal on Computing, 31(6), 1794-1813. [ Links ]
16. Domingos, P. & Hulten, G. (2000). Mining HighSpeed Data Streams. Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '00), Boston, MA, USA, 71-80. [ Links ]
17. Dredze, M., Oates, T., & Piatko, C. (2010). We're not in Kansas anymore: detecting domain changes in streams. 2010 Conference on Empirical Methods in Natural Language Processing (EMNLP'10), Massachusetts, USA, 585-595. [ Links ]
18. Dries, A. & Rückert, U. (2009). Adaptive concept drift detection. Statistical Analysis and Data Mining, 2(5-6), 311-327. [ Links ]
19. Ferrer-Troyano, F.J., Aguilar, J.S., & Riquelme, J.C. (2005). Incremental Rule Learning and Border Examples Selection from Numerical Data Streams. Journal of Universal Computer Science, 11(8), 1426-1439. [ Links ]
20. Frías, I., Ortiz, A., Ramos, G., Morales, R., & Caballero, Y. (2010). Clasificadores y multiclasificadores con cambio de concepto basados en árboles de decisión. Revista Iberoamericana de Inteligencia Artificial 14(45), 32-43. [ Links ]
21. Gama, J. (2010). Knowledge Discovery from Data Streams. Boca Raton, FL: Chapman and Hall/CRC. [ Links ]
22. Gama, J. & Gaber, M.M. (2007). Learning from Data Streams: Processing Techniques in Sensor Networks. Berlin; New York: Springer. [ Links ]
23. Gama, J. & Kosina, P. (2011). Learning decision rules from data streams. Twenty-Second International Joint Conference on Artificial Intelligence, Barcelona, Catalonia, Spain, 1255-1260. [ Links ]
24. Gama, J., Medas, P., Castillo, G., & Rodrigues, P. (2004). Learning with drift detection. Advances in Artificial Intelligence, SBIA 2004, Lecture Notes in Computer Science, 3171, 286-295. [ Links ]
25. Gama, J., Sebastião, R., & Rodrigues, P. (2009). Issues in Evaluation of Stream Learning Algorithms. 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Paris, France, 329-338. [ Links ]
26. Gretton, A., Borgwardt, K., Rasch, M., Schõlkopf, B., & Smola, A. (2006). A Kernel Method for the Two Sample Problem. Twentieth Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, 513-520. [ Links ]
27. Harries, M.B., Sammut, C., & Horn, K. (1998). Extracting hidden context. Machine Learning -Special issue on context sensitivity and concept drift, 32(2), 101-126. [ Links ]
28. Hawkins, D.M. & Deng, Q. (2010). A Nonparametric Change-Point Control Chart. Journal of Quality Technology, 42(2), 165-173. [ Links ]
29. Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables. Journal of American Statistical Association, 58(301), 13-30. [ Links ]
30. Ikonomovska, E., Gama, J., & Dzeroski, S. (2011). Learning model trees from evolving data streams. Data Mining and Knowledge Discovery, 23(1), 128-168. [ Links ]
31. Katakis, I., Tsoumakas, G., & Vlahavas, I. (2008). An Ensemble of Classifiers for coping with Recurring Contexts in Data Streams. 2008 conference on ECAI 2008: 18th European Conference on Artificial Intelligence, Patras, Greece, 763-764. [ Links ]
32. Kawahara, Y. & Sugiyama, M. (2009). Change-Point Detection in Time-Series Data by Direct Density-Ratio Estimation. SIAM International Conference on Data Mining (SDM 2009), Sparks, Nevada, USA, 389-400. [ Links ]
33. Kawahara, Y., Yairi, T., & Machida, K. (2007). Change-Point Detection in Time-Series Data Based on Subspace Identification. Seventh IEEE International Conference on Data Mining (ICDM '07), Omaha, NE, USA, 559-564. [ Links ]
34. Kifer, D., Ben-David, S., & Gehrke, J. (2004). Detecting Change in Data Streams. Thirtieth International Conference on Very Large Data Bases (VLDB '04), Toronto, Ontario, Canada, 30, 180-191. [ Links ]
35. Klinkenberg, R. (2004). Learning drifting concepts: example selection vs. example weighting. Intelligent Data Analysis, 8(3), 281-300. [ Links ]
36. Klinkenberg, R. & Joachims, T. (2000). Detecting Concept Drift with Support Vector Machines. Seventeenth International Conference on Machine Learning (ICML '00), Stanford, CA, USA, 487-494. [ Links ]
37. Kubat, M. & Widmer, G. (1994). Adapting to drift in continuous domains. Machine Learning: ECML-95, Lecture Notes in Computer Science, 912, 307-310. [ Links ]
38. Pettitt, A.N. (1979). A Non-Parametric Approach to the Change-Point Problem. Journal of the Royal Statistical Society, Series C (Applied Statistics), 28(2), 126-135. [ Links ]
39. Ross, G., Tasoulis, D.K., & Adams, N.M. (2011). Nonparametric Monitoring of Data Streams for Changes in Location and Scale. Technometrics, 53(4), 379-389. [ Links ]
40. Salganicoff, M. (1997). Tolerating Concept and Sampling Shift in Lazy Learning Using Prediction Error Context Switching. Artificial Intelligence Review, 11(1-5), 133-155. [ Links ]
41. Schlimmer, J.C. & Granger Jr., R.H. (1986). Incremental learning from noisy data. Machine Learning 1(3), 317-354. [ Links ]
42. Scholz, M. & Klinkenberg, R. (2007). Boosting classifiers for drifting concepts. Intelligent Data Analysis - Knowlegde Discovery from Data Streams, 11(1), 3-28. [ Links ]
43. Yamanishi, K. & Takeuchi, J.I. (2002). A Unifying Framework for Detecting Outliers and Change Points from Non-Stationary Time Series Data. Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '02), Edmonton, AB, Canada, 676-681. [ Links ]
44. Zhou, C., Zou, C., Zhang, Y., & Wang, Z. (2009). Nonparametric control chart based on change-point model. Statistical Papers, 50(1), 13-28. [ Links ]
45. Žliobaitė, I. (2009). Learning under Concept Drift: an Overview. Lithuania: Vilnius University. [ Links ]
]]>