Using a Markov Random Field for Image Re–ranking Based on Visual and Textual Features
Utilizando un campo aleatorio de markov para el reordenamiento de imágenes basado en atributos visuales y textuales
R. Omar Chávez1, Manuel Montes2 and L. Enrique Sucar3
Coordinación de Ciencias Computacionales, Instituto Nacional de Astrofísica Óptica y Electrónica, Puebla, México. romarcg@ccc.inaoep.mx1, mmontesg@ccc.inaoep.mx2, esucar@ccc.inaoep.mx3.
]]>Article received on January 15, 2010
Accepted on June 29, 2010
Abstract
We propose a novel method to re–order the list of images returned by an image retrieval system (IRS). The method combines the original order obtained by the IRS, the similarity between images obtained with visual and textual features, and a relevance feedback approach, all of them with the purpose of separating relevant from irrelevant images, and thus, obtaining a more appropriate order. The method is based on a Markov random field (MRF) model, in which each image in the list is represented as a random variable that could be relevant or irrelevant. The energy function proposed for the MRF combines two factors: the similarity between the images in the list (internal similarity); and information obtained from the original order and the similarity of each image with the query (external similarity). Experiments were conducted with resources from the Image CLEF 2008 forum for the photo retrieval track, taking into account textual and visual features. The results show that the proposed method improves, according to the MAP measure, the order of the original list up to 63% (in the textual case) and up to 55% (in the visual case); and suggest future work using a combination of both kind of features.
Keywords: Image Re–ranking, Image Retrieval, Markov Random Field, Relevance Feedback.
Resumen
En este trabajo proponemos un método novedoso para re–ordenar una lista de imágenes recuperadas por un sistema de recuperación de imágenes (SRI). El método combina el orden original obtenido por el SRI, la similitud entre imágenes, obtenida con las características visuales y textuales, y un enfoque de retroalimentación de relevancia, todos ellos con el propósito de separar las imágenes relevantes de las irrelevantes, y así, obtener un orden más apropiado. El método está basado en el modelo de un campo aleatorio de Markov (CAM), en el que cada imagen en la lista fue representada como una variable aleatoria con dos posibles valores: relevante o irrelevante. La función de energía propuesta para el campo aleatorio de Markov combina dos factores: la similitud entre imágenes en la lista (similitud interna); y la información obtenida del orden original y la similitud de cada imagen con la consulta (similitud externa). Los experimentos fueron realizados con los recursos del foro Image CLEF 2008 para la tarea de recuperación de fotografías, tomando en cuenta los atributos textuales y visuales. Los resultados mostraron que el método propuesto mejora, de acuerdo con la medida MAP, el orden de la lista original hasta en un 63% (en el caso textual) y hasta un 55% (en el caso visual); y sugieren como trabajo a futuro el utilizar una combinación de ambos tipos de atributos.
]]> Palabras clave: Re–ordenamiento de Imágenes, Recuperación de Imágenes, Campos Aleatorios de Markov, Retroalimentación de Relevancia.
DESCARGAR ARTÍCULO EN FORMATO PDF
References
1. Arni, T., Clough, P., Sanderson, M., & Grubinger, M. (2008). Overview of the ImageCLEFphoto 2008 Photographic Retrieval Task. 9th Cross–language evaluation forum conference on Evaluating systems for multilingual and multimodal information access (CLEF 2008), Aarhus, Denmark, 500–511. [ Links ]
2. Awang, D. N. F., Pehcevski, J., Thom, J. A., & Tahaghoghi, S. M.M. (2006). Combining image and structured text retrieval. Advances in XML Information Retrieval and Evaluation. Lecture Notes in Computer Science, 3977, 525–539. [ Links ]
3. Bay, H., Tuytelaars, T., & Van Gool, L. (2006). Surf: Speeded up robust features. 9th European Conference on Computer Vision , Graz, Austria, 404–417. [ Links ]
4. Berg, T. L. (2009). Finding Iconic Images. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Miami, FL, USA, 2009, 1–8. [ Links ]
5. Bihong, G., Bo, P., & Xiaoming, L. (2007). A Personalized Re–ranking Algorithm Based on Relevance Feedback. Advances in Web and Network Technologies, and Information Management. Lecture Notes in Computer Science, 4537, 255–263. [ Links ]
6. Chellappa, R., & Jain, A. (1993). Markov Random Fields: Theory and Application. Boston: Academic Press. [ Links ]
7. Chong, T., Yanxiang, H., Donghong, J., Guimin, L., & Zhewei, M. (2009). A Study on Pseudo Labeled Document Constructed for Document Re–ranking. International Conference on Artificial Intelligence and Computational Intelligence, Shanghai, China, 377–380. [ Links ]
8. Choochaiwattana, W., & Spring, M. B. (2009). Applying Social Annotations to Retrieve and Re–rank Web Resources. International Conference on Information Management and Engineering, Kuala Lumpur, Malaysia, 215–219. [ Links ]
9. Clough, P., & Sanderson, M. (2004). Relevance Feedback for Cross Language Image Retrieval. Advances in Information Retrieval: 26th European Conference on IR Research, ECIR 2004. Lecture Notes in Computer Science, 2997, 238–252. [ Links ]
10. Cui, J., Wen, F., & Tang, X. (2008). Real time google and live image search re–ranking. 16th ACM international conference on multimedia MM '08, Vancouver, Canada, 729–732. [ Links ]
11. Datta, R., Joshi, D., Li, J., & Wang, J. Z. (2008). Image retrieval: Ideas, influences, and trends of the new age. ACM Computer Surveys , 40 (2), 1–60. [ Links ]
12. Deselaers, T., Paredes, R., Vidal E., & V, E. (2008). Learning Weighted Distances for Relevance Feedback in Image Retrieval. 19th International Conference on Pattern Recognition, Tampa, Florida, USA, 1–4. [ Links ]
13. Dietterich, T. G. (1998). Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms. Neural Computation, 10 (7), 1895–1923. [ Links ]
14. Escalante, H. J., Gonzalez, J. A., Hernandez, C., Lopez, A., Montes, M., Morales, E., et al. (2008). TIA–INAOE's Participation at ImageCLEF2008. Working Notes of the CLEF 2008 Workshop, Aarhus, Denmark. [ Links ]
15. Escalante, H. J., Hernandez, C. A., Sucar, L. E., & Montes, M. (2008). Late fusion of heterogeneous methods for multimedia image Retrieval. 1st ACM international conference on Multimedia information retrieval (MIR '08), Vancouver, Canada, 172–179. [ Links ]
16. Gong, Z., & Liu, Q. (2009). Improving keyword based web image search with visual feature distribution and term expansion. Knowledge and Information Systems, 21 (1), 113–132. [ Links ]
17. Jégou, H., Douze, M., & Schmid, C. (2010). Improving bag–of–features for large scale image search. International Journal of Computer Vision, 87 (3) , 316–336. [ Links ]
18. Jianjiang, L., Zhenghui, X., Ran, L., Yafei, Z., & Jiabao, W. (2009). A Framework of CBIR System Based on Relevance Feedback,. Third International Symposium on Intelligent Information Technology Application (IITA 2009), 175–178. [ Links ]
19. Kanji, G. K. (1993). 100 statistical tests. Newbury Park, California: Sage Publications. [ Links ]
20. Gao, K., Lin, S., Zhang, Y., & Tang, S. (2008). Object–based Image Retrieval with Attention Analysis and Spatial Re–ranking. IFIP Advances in Information and Communication Technology, 288, 118–128. [ Links ]
21. Li, S.Z. (1994). Markov random field models in computer vision. Computer Vision ECCV '94, Lecture Notes in Computer science, 801, 361–370. [ Links ]
22. Lin, W. H., Jin, R., & Hauptmann, A. (2003). Web Image Retrieval Re–Ranking with Relevance Model. IEEE/WIC International Conference on Web Intelligence (WI 03), Halifax, Canada, 242–248. [ Links ]
23. Lowe, D. G. (2004). Distinctive Image Features from Scale–Invariant Keypoints. International Journal of Computer Vision, 60 (2), 91–110. [ Links ]
24. Mani, I. (2001). Automatic Summarization . Amsterdam ; Philadelphia: John Benjamins Publishing Co. [ Links ]
25. Marakakis, A., Galatsanos, N., Likas, A., & Stafylopatis, A. (2008). Application of Relevance Feedback in Content Based Image Retrieval Using Gaussian Mixture Models: 20th IEEE International Conference on Tools with Artificial Intelligence (ICTAI '08), Dayton, Ohio, USA, 141–148. [ Links ]
26. Martin, B., Dirk, P., & Josiah, P. (2004). Application of Machine Learning Techniques to the Re–ranking of Search Results KI 2004: Advances in Artificial Intelligence. Lecture Notes in Computer Science, 3238, 67–81. [ Links ]
27. Richter, F., Romberg, S., Horster, E., & Lienhart, R. (2010). Multimodal ranking for image search on community databases. International conference on multimedia information retrieval (MIR '10), Philadelphia, Pennsylvania, USA, 63–72. [ Links ]
]]>