A Feed–Forward Neural Networks–Based Nonlinear Autoregressive Model for Forecasting Time Series

Modelo auto regresivo no lineal basado en redes neuronales multicapa para pronóstico de series temporales

Julián A. Pucheta¹, Cristian M. Rodríguez Rivero¹, Martín R. Herrera², Carlos A. Salas², H. Daniel Patiño³ and Benjamín R. Kuchen³

¹ Mathematics Research Laboratory Applied to Control, Departments of Electrical and Electromechanical Engineering, Faculty of Exact, Physical and Natural Sciences, National University of Córdoba, Córdoba, Argentina. julian.pucheta@gmail.com, cristian.rodriguezrivero@gmail.com.

³ Institute of Automatics Faculty of Engineering–National University of San Juan, San Juan, Argentina. dpatino@unsj.edu.ar, bkuchen@unsj.edu.ar.

Article received on January 15, 2010
Accepted on October 08, 2010

Abstract

In this work a feed–forward NN based NAR model for forecasting time series is presented. The learning rule used to adjust the NN weights is based on the Levenberg–Marquardt method. In function of the long or short term stochastic dependence of the time series, we propose an online heuristic law to set the training process and to modify the NN topology. The approach is tested over five time series obtained from samples of the Mackey–Glass delay differential equations and from monthly cumulative rainfall. Three sets of parameters for MG solution were used, whereas the monthly cumulative rainfall belongs to two different sites and times period, La Perla 1962–1971 and Santa Francisca 200–2010, both located at Córdoba, Argentina. The approach performance presented is shown by forecasting the 18 future values from each time series simulated by a Monte Carlo of 500 trials with fractional Gaussian noise to specify the variance.

Keywords: Neural networks, time series forecast, Hurst's parameter, Mackey–Glass.

Se presenta un modelo auto–regresivo no lineal (ARN) basado en redes neuronales para el pronóstico de series temporales. La regla de aprendizaje para ajustar los parámetros de la red neuronal (RN) está basado en el método Levenberg–Marquardt en función de la dependencia estocástica de la serie temporal, proponemos una ley heurística que ajusta el proceso de aprendizaje y modifica la topología de la RN. Esta propuesta es experimentada sobre cinco series temporales. Tres son obtenidas de la ecuación de Mackey–Glass (MG) en un intervalo de tiempo. Las dos restantes son series históricas de lluvia acumulada mensualmente pertenecientes a dos lugares y tiempos diferentes, La Perla 1962–1971 y Santa Francisca 2000–2010, Córdoba, Argentina. El desempeño del esquema se muestra a través del pronóstico de 18 valores de cada serie temporal, donde el pronóstico fue simulado mediante Monte Carlo con de 500 realizaciones con ruido Gaussiano fraccionario para especificar la varianza.

Palabras Clave: Redes neuronales, pronóstico de series temporales, parámetro de Hurst, ecuación Mackey–Glass.

DESCARGAR ARTÍCULO EN FORMATO PDF

Acknowledgments

This paper was supported by National University of Córdoba (Secyt UNC 69/08), National University of San Juan (UNSJ), National Agency for Scientific and Technological Promotion (ANPCyT) under grant PAV 076, PICT/04 No. 21592 and PICT–2007–00526. The authors want to thank Carlos Bossio (Coop. Huinca Renancó), Ronald del Águila (LIADE) and Eduardo Carreño (Santa Francisca) for their help.

References

1. Abry, P., Flandrin, P., Taqqu, M.S. & Veitch, D. (2003). Self–similarity and long–range dependence through the wavelet lens In P. Doukhan, G. Oppenheim & M. Taqqu (Eds.), Theory and applications of long–range dependence (527–556). Boston: Birkhäuser.         [ Links ]

2. Bardet, J.M., Lang, G., Oppenheim, G., Philippe, A., Stoev, S. & Taqqu, M.S. (2003). Semi–parametric estimation of the long–range dependence parameter: a survey. In P. Doukhan, G. Oppenheim & M. Taqqu (Eds.), Theory and applications of long–range dependence. (557–577) Boston, MA: Birkhäuser.         [ Links ]

3. Bishop, C.M. (1995). Neural Networks for Pattern Recognition. New York: Oxford University Press.         [ Links ]

4. Bishop, C.M. (2006). Pattern Recognition and Machine Learning. New York: Springer.         [ Links ]

5. Dieker, T. (2004). Simulation of fractional Brownian motion. MSc thesis, University of Twente, Amsterdam, The Netherlands.         [ Links ]

6. Flandrin, P. (1992). Wavelet analysis and synthesis of fractional Brownian motion. IEEE Transactions on Information Theory, 38(2), 910–917.         [ Links ]

7. Glass L., & Mackey M.C. (1998). From Clocks to Chaos, The Rhythms of Life. Princeton, NJ: Princeton University Press.         [ Links ]

8. Guo, W.W., Li L.D. & Whymark G. (2009). Statistics and neural networks for approaching nonlinear relations between wheat plantation and production in queensland of Australia. IEEE International Conference on Industrial Technology, Gippsland, Australia, 1–4.         [ Links ]

9. Haykin, S. (1999). Neural Networks: A comprehensive Foundation (2nd Ed). Upper Saddle River, N.J.: Prentice Hall.         [ Links ]

10. Istas, J. & G. Lang. (1997). Quadratic variations and estimation of the local Hó'lder index of a Gaussian process. Annales de l'Institut Henri Poincaré (B) Probabilités et Statistiques, v 33 (4), 407–436.         [ Links ]

11. Liu, J.N.K. & Lee, R.S.T (1999). Rainfall forecasting from multiple point sources using neural networks. International Conference on Systems, Man, and Cybernetics, Tokyo, Japan, Vol 3, 429–434.         [ Links ]

12. Mandelbrot, B.B. (1983). The Fractal Geometry of Nature. New York: W.H. Freeman.         [ Links ]

13. Masulli, F., Baratta, D., Cicione, G. & Studer, L. (2001). Daily rainfall forecasting using an ensemble technique based on singular spectrum analysis. International Joint Conference on Neural Networks, Washington, D.C, USA, 263–268.         [ Links ]

14. Mozer, M.C. (1994). Neural net architectures for temporal sequence processing. In A. S. Weigend & N. A. Gershenfeld (Eds.), Time series prediction: Forecasting the future and understanding the past (243–264). Redwood Reading, MA: Addison–Wesley Publishing.         [ Links ]