Associative Memory in a Continuous Metric Space: A Theoretical Foundation
Memoria Asociativa en un Espacio Métrico Continuo: Fundamentos Teóricos
Enrique Carlos Segura
Department of Computer Science, University of Buenos Aires Ciudad Universitaria, Pab.I, (1428) Buenos Aires, Argentina. esegura@dc.uba.ar
]]> Article received on December 18, 2007
Abstract
We introduce a formal theoretical background, which includes theorems and their proofs, for a neural network model with associative memory and continuous topology, i.e. its processing units are elements of a continuous metric space and the state space is Euclidean and infinite dimensional. This approach is intended as a generalization of the previous ones due to Little and Hopfield. The main contribution of the present work is to integrate –and to provide a theoretical background that makes this integration consistent– two levels of continuity: i) continuous response processing units and ii) continuous topology of the neural system, obtaining a more biologically plausible model of associative memory. We present our analysis according to the following sequence of steps: general results concerning attractors and stationary solutions, including a variational approach for the derivation of the energy function; focus on the case of orthogonal memories, proving theorems on stability, size of attraction basins and spurious states; considerations on the problem of resolution, analyzing the more general case of memories that are not orthogonal, and with possible modifications to the synaptic operator; getting back to discrete models, i. e. considering new viewpoints arising from the present continuous approach and examine which of the new results are also valid for the discrete models; discussion about the generalization of the non deterministic, finite temperature dynamics.
Keywords: associative memory, continuous metric space, dynamical systems, Hopfield model, stability, Glauber dynamics, continuous topology.
Resumen
Presentamos bases teóricas formales, incluyendo teoremas y sus demostraciones, para un modelo de red neuronal con memoria asociativa y topología continua, i. e. sus unidades de procesamiento son elementos de un espacio métrico continuo y el espacio de estados es euclidiano y de dimensión infinita. El enfoque es concebido como una generalización de los precedentes debidos a Little y Hopfield. La principal contribución del presente trabajo es integrar –y proveer fundamentos teóricos que den consistencia a tal integración– dos niveles de continuidad: i) unidades de proceso de respuesta continua y ii) topología continua del sistema neuronal, obteniendo de esta manera un modelo mas biológicamente plausible de memoria asociativa. Nuestro análisis es presentado de acuerdo con la siguiente secuencia de pasos: resultados generales sobre atractores y soluciones estacionarias, que incluyen un enfoque variacional para derivar la función de energía; estudio detallado del caso de memorias ortogonales, demostrando teoremas sobre estabilidad, tamaño de cuencas de atracción y estados espurios; consideraciones sobre el problema de la resolución, analizando el caso más general de memorias no ortogonales, y con modificaciones posibles al operador sináptico; retorno a los modelos discretos, i.e. consideración de nuevos puntos de vista que surgen del presente esquema, y de cuales de los nuevos resultados son también válidos para los modelos discretos; discusión sobre la generalización de la dinámica no deterministica a temperatura finita.
Palabras clave: memoria asociativa, espacio métrico continuo, sistema dinámico, modelo de Hopfield, dinámica de Glauber, topología continua, estabilidad.
]]>DESCARGAR ARTÍCULO EN FORMATO PDF
Acknowledgements
This work was partially supported by grants X166 from the University of Buenos Aires and 26001 from the National Agency for Scientific Research, Argentina.
References
1. Amit, D. J., Modeling Brain Function. Cambridge: Cambridge University Press, 1989. [ Links ]
2. Dreyfus, G., Neural Networks. Methodology and Applications. Berlin: Springer, 2005. [ Links ]
3. Feynman, R. P. and Hibbs, A. R., Quantum Mechanics and Path Integrals. New York: McGraw–Hill, 1965. [ Links ]
4. Fodor, J. A., The Modularity of Mind. Cambridge: MIT Press, 1983. [ Links ]
5. Glauber R. J., "Time–dependent Statistics of the Ising Model". Journal of Math. Phys., 4:294–307 (1963). [ Links ]
6. Hagan, M. T., Demuth, H. B. and Beale, M. H., Neural Network Design. Boston: PWS Publishing Company, 2002. [ Links ]
7. Hebb, D. O., The Organization of Behavior: A Neuropsychological Theory. New York: Wiley, 1949. [ Links ]
8. Hopfield, J. J., "Neural networks and physical systems with emergent collective computational abilities". Proc. Natl. Acad Sci., 79:2554–2558 (1982). [ Links ]
9. Hopfield, J. J., "Neurons with graded response have collective computational properties like those of two–state neurons". Proc. Natl. Acad. Sci., 81:3088–3092 (1984). [ Links ]
10. Kampen, N. G. van, Stochastic Processes in Physics and Chemistry. Amsterdam: Elsevier, 1997. [ Links ]
11. Little, W. A., "The Existence of Persistent States in the Brain". Mathematical Biosciences, 19:101–120 (1974). [ Links ]
12. Little, W. A., "Analytic Study of the Memory Storage Capacity of a Neural Network". Mathematical Biosciences, 39:281–290 (1978). [ Links ]
13. MacLennan, B. J., "Field Computation in Natural and Artificial Intelligence". Information Sciences, 119:73–89 (1999). [ Links ]
14. MacLennan, B. J., "Continuous Information Representation and Processing in Natural and Artificial Neural Networks". JCIS 2002, Research Triangle Park, North Carolina, USA, 561–565 (2002). [ Links ]
15. Segura, E. C. and Perazzo, R. P. J., "Associative memories in infinite dimensional spaces". Neural Processing Letters, 12(2): 129–144 (2000). [ Links ]
16. Segura, E. C. and Perazzo, R. P. J., "Biologically Plausible Associative Memory: Continuous Unit Response + Stochastic Dynamics". Neural Processing Letters, 16(3):243–257 (2002). [ Links ]
17. Segura, E. C., "Infinite Dimensional Associative Memories: Foundations (with Theorems)". Res. Comp. Science, 28(3): 15–28 (2007). [ Links ] ]]>