A Reinforcement Learning Solution for Allocating Replicated Fragments in a Distributed Database

Una solución de Aprendizaje Reforzado para ubicar fragmentos replicados en Bases de Datos Distribuidas

Abel Rodríguez Morffi¹, Darien Rosa Paz¹, Marisela Mainegra Hing² and Luisa Manuela González González¹

¹Departamento de Ciencia de la Computación,
e–mail: arm@uclv.edu.cu , drosa@uclv.edu.cu , luisagon@uclv.edu.cu

² Departamento de Matemática Universidad Central "Marta Abreu" de Las Villas, Carretera a Camajuaní km. 5.5, C.P. 54830, Santa Clara, Cuba. Tel: +(53)(42) 281515
e–mail: marisela@uclv.edu.cu

Article received on April 23, 2007; accepted on October 18, 2007

Abstract

Due to the complexity of the data distribution problem in Distributed Database Systems, most of the proposed solutions divide the design process into two parts: the fragmentation and the allocation of fragments to the locations in the network. Here we consider the allocation problem with the possibility to replicate fragments, minimizing the total cost, which is in general NP–complete, and propose a method based on Q–learning to solve the allocation of fragments in the design of a distributed database. As a result we obtain for several cases, logical allocation of fragments in a practical time

Keywords: Distributed database design, allocation, replication, reinforcement learning, Q–Learning.

Resumen

Debido a la complejidad del problema de la distribución de los datos, la mayoría de las propuestas de solución presentadas hasta la fecha han coincidido en dividir el proceso de diseño de la distribución en dos fases seriadas: la fragmentación y la ubicación de los fragmentos en los sitios de la red. Este trabajo aborda el problema de ubicación de fragmentos partiendo de un modelo matemático que en su forma general es NP–Completo y propone un método metaheurístico basado en Q–Learning de Aprendizaje Reforzado que minimiza el costo total en un tiempo aceptable. Esta propuesta integra la replicación de fragmentos.

Palabras claves: Diseño de bases de datos distribuidas, ubicación, replicación, aprendizaje reforzado, Q–Learning.

DESACARGAR ARTÍCULO EN FORMATO PDF

Acknowledgements

This work is part of a research and development Project "Database Technologies applied to problems", Number 01700031 from the National Program of Information Technology supported by the Cuban Ministry of Science, Technology and Environment. The authors would like to thanks to the Flemish Interuniversity Council (Vlaamse InterUniversitaire Raad) for their support through the IUC VLIR–UCLV Program.

References

1. Abe, N.; Biermann, A. W. and Long, P. M. (2003). "Reinforcement Learning with Immediate Rewards and Linear Hypotheses." Algorithmica 37(4): 263–293.        [ Links ]

2. Ceri, S. and Pelagatti, G. (1984). Distributed Databases: Principles and Systems, McGraw–Hill Book Company.        [ Links ]

3. Choi, S. P. M.; Zhang, N. L. and Yeung, D.–Y. (2004). Reinforcement Learning in Episodic Non–stationary Markovian Environments. Proceedings of the International Conference on Artificial Intelligence, IC–AI '04. Proceedings of the International Conference on Machine Learning; Models, Technologies and Applications, MLMTA '04, Las Vegas, Nevada, USA, CSREA Press.        [ Links ]

4. Hababeh, I. O.; Bowring, N. and Ramachandran, M. (2004). A Method for Fragment Allocation in Distributed Object Oriented Database Systems. Proceedings of the 5th Annual PostGraduate Symposium on The Convergence of Telecommunications, Networking & Broadcasting (PGNet), Liverpool, UK.        [ Links ]

5. Huang, Y.–F. and Chen, J.–H. (2001). "Fragment Allocation in Distributed Database Design." Journal of Information Science and Engineering 17(3): 491–506.        [ Links ]

6. Iida, S.; Kuwayama, K.; Kanoh, M.; Kato, S. and Itoh, H. (2004). A Dynamic Allocation Method of Basis Functions in Reinforcement Learning. Proceedings of the 17th Australian Joint Conference on Artificial Intelligence AI 2004: Advances in Artificial Intelligence, Cairns, Australia, Springer.        [ Links ]

7. Kaelbling, L.; Littman, M. and Moore, A. (1996). "Reinforcement learning: A survey." Journal of Artificial Intelligence Research 4: 237–285.        [ Links ]

8. Lin, X. and Orlowska, M. E. (1995). "An Integer Linear Programming Approach to Data Allocation with the Minimum Total Communication Cost in Distributed Database Systems." Information Sciences 85(1–3): 1–10.        [ Links ]

9. Lin, X.; Orlowska, M. E. and Zhang, Y. (1993). On Data Allocation with the Minimum Overall Communication Costs in Distributed Database Design. Proceedings of the Fifth International Conference on Computing and Information – ICCI'93, Sudbury, Ontario, Canada, IEEE Computer Society.        [ Links ]

10. Ma, H.; Schewe, K.–D. and Wang, Q. (2006). A Heuristic Approach to Cost–Efficient Fragmentation and Allocation of Complex Value Databases. The 17th Australasian Database Conference (ADC2006) Hobart, Australia, ACM International Conference Archive Proceeding Series vol. 170. Australian Computer Society, Inc.        [ Links ]

11. March, S. T. and Rho, S. (1995). "Allocating Data and Operations to Nodes in Distributed Database Design." IEEE Trans. Knowl. Data Eng. 7(2): 305–317.        [ Links ]

12. Morales, E. F. and Sammut, C. (2004). Learning to fly by combining reinforcement learning with behavioural cloning, ACM.        [ Links ]

13. Özsu, M. T. and Valduriez, P. (1999). Principles of Distributed Database Systems, Second Edition, Prentice–Hall.        [ Links ]

14. Papadimitriou, C. H. (1997). NP–Completeness: A Retrospective. Proceedings of the 24th International Colloquium on Automata, Languages and Programming, ICALP'97, Bologna, Italy, Springer.        [ Links ]

15. Pérez, J.; Pazos, R. A.; Frausto–Solis, J.; Reyes, G.; Santaolaya, R.; Fraire, H. J. and Cruz, L. (2005). An approach for solving very large scale instances of the design distribution problem for distributed database systems. Proceedings of the 4th International School and Symposium on Advanced Distributed Systems (ISSADS2005). Lecture Notes in Computer Science, Springer.        [ Links ]

16. Pérez, J.; Pazos, R. A.; Mora, G.; Castilla, G.; Martínez, J. A.; Landero, V.; Fraire, H. J. and González, J. J. (2004). Dynamic Allocation of Data–Objects in the Web, Using Self–tuning Genetic Algorithms. Proceedings of the 17th Brazilian Symposium on Artificial Intelligence– SBIA 2004, Advances in Artificial Intelligence São Luis, Maranhão, Brazil, Springer.        [ Links ]

17. Pérez, J.; Pazos, R. A.; Romero, D.; Santaolaya–Salgado, R.; Rodríguez, G. and Sosa–Sosa, V. J. (2003). Adaptive and Scalable Allocation of Data–Objects in the Web. Proceedings of the International Conference on Computational Science and Its Applications – ICCSA 2003, Part I., Montreal, Canada, Springer.        [ Links ]

18. Pérez, J.; Pazos, R. A.; Santaolaya–Salgado, R.; Frausto–Solís, J.; Rodríguez, G.; Cruz, L. and Bravo, M. (2003). Data–Object Replication, Distribution, and Mobility in Network Environments. Revised Papers of the 5th International Andrei Ershov Memorial Conference, Perspectives of Systems Informatics, PSI 2003, Akademgorodok, Novosibirsk, Russia, Springer.        [ Links ]

19. Pérez, J.; Pazos, R. A.; Velez, L. and Rodríguez, G. (2002). Automatic Generation of Control Parameters for the Threshold Accepting Algorithm. MICAI 2002: Advances in Artificial Intelligence, Second Mexican International Conference on Artificial Intelligence, Merida, Yucatan, Mexico, Springer.        [ Links ]

20. Puterman, M. L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming. New York, NY John Wiley & Sons.        [ Links ]

21. Rosa, D. (2006). Aplicación de técnicas de Inteligencia Artificial en la solución del problema de ubicación en el diseño de bases de datos distribuidas. Departamento de Ciencia de la Computación. Santa Clara, Universidad Central "Marta Abreu" de Las Villas. BSc.Thesis.        [ Links ]

22. Sutton, R. and Barto, A. (1998). Reinforcement Learning: An Introduction. Cambridge, Massachussets, USA., MIT Press.        [ Links ]

23. Watkins, C. (1999). Learning from Delayed Rewards. Cambridge, UK, King's College.        [ Links ]

24. Wolfson, O. and Jajodia, S. (1995). "An Algorithm for Dynamic Data Allocation in Distributed Systems." Inf. Process. Lett. 53(2): 113–119.        [ Links ]