Binocular visual tracking and grasping of a moving object with a 3D trajectory predictor

 

J. Fuentes–Pacheco¹, J. Ruiz–Ascencio*¹, J. M. Rendón–Mancha²

 

¹ Centro Nacional de Investigación y Desarrollo Tecnológico, Interior Internado Palmira S/N, Col. Palmira, C.P. 62490, Cuernavaca, Morelos, México.

² Universidad Autónoma del Estado de Morelos. Av. Universidad 1001, Col. Chamilpa, C.P. 62209, Cuernavaca, Morelos, México. *josera@cenidet.edu.mx.

 

ABSTRACT

This paper presents a binocular eye–to–hand visual servoing system that is able to track and grasp a moving object in real time. Linear predictors are employed to estimate the object trajectory in three dimensions and are capable of predicting future positions even if the object is temporarily occluded. For its development we have used a CRS T475 manipulator robot with six degrees of freedom and two fixed cameras in a stereo pair configuration. The system has a client–server architecture and is composed of two main parts: the vision system and the control system. The vision system uses color detection to extract the object from the background and a tracking technique based on search windows and object moments. The control system uses the RobWork library to generate the movement instructions and to send them to a C550 controller by means of the serial port. Experimental results are presented to verify the validity and the efficacy of the proposed visual servoing system.

Keywords: linear prediction, visual servoing, tracking, grasping, stereo vision, camera calibration.

 

RESUMEN

En este trabajo se presenta un sistema servo control visual binocular eye–to–hand capaz de seguir y asir un objeto móvil en tiempo real. Se utilizaron predictores lineales para estimar la trayectoria del objeto en tres dimensiones, estos son capaces de predecir futuras posiciones incluso si el objeto es temporalmente ocluido. Para su desarrollo se utiliza un robot manipulador CRS T475 de seis grados de libertad y dos cámaras fijas con una configuración de par estéreo. El sistema tiene una arquitectura cliente–servidor y se compone de dos partes principales: el sistema de visión y el sistema de control. El sistema de visión utiliza una detección por color para extraer el objeto del fondo y una técnica de seguimiento basada en ventanas de búsqueda y momentos del objeto. El sistema de control emplea la librería RobWork para generar las instrucciones de movimiento y enviarlas a un controlador C550 por medio del puerto serial. Se presentan resultados experimentales para verificar la validez y eficacia del sistema servo control visual.

Palabras clave: Predicción lineal, servo control visual, seguimiento, visión estéreo, calibración de cámaras.

 

DESCARGAR ARTÍCULO EN FORMATO PDF

 

Acknowledgments

This paper has been made possible thanks to the generous support from the following institutions which we are pleased to acknowledge: CONACYT (Consejo Nacional de Ciencia y Tecnología) and CENIDET (Centro Nacional de Investigación y Desarrollo Tecnológico).

 

References

[1] Fu, K., González, R., Lee, C., ROBÓTICA: Control, Detección, Visión e Inteligencia, Editorial Mc Graw–Hill, (1988).         [ Links ]

[2] Hutchinson, S., Hager, G., Corke, P., A Tutorial on Visual Servo Control, In IEEE Transactions on Robotics and Automation, Vol. 12, No. 5, pp. 651–670, (1996).         [ Links ]

[3] Corke, P., VISUAL CONTROL OF ROBOTS high–performance visual servoing, Editorial Research Studies Press Ltd., (1996).         [ Links ]

[4] Allen, P., Timcenko, A., Yoshimi, B., et al., Automated Tracking and Grasping of a Moving Object with a Robotic Hand–Eye System, Proceedings of the IEEE Transactions on Robotics and Automation, Vol. 9, No. 2, (1993).         [ Links ]

[5] Nomura, H., Naito, T., Integrated visual servoing system to grasp industrial parts moving on conveyer by controlling 6DOF arm, Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, pp. 1768–1775, (2000).         [ Links ]

[6] Sen, G., Messom, C., Demidenko, S., et al., Identification and prediction of a moving object using real–time global vision sensing, Proceedings of the 20th IEEE Instrumentation and Measurement Technology Conference, Volume 2, pp.1402–1406, (2003).         [ Links ]

[7] Ge, L., Jie, Z., A Real–time Stereo Visual Servoing for Moving Object Grasping Based Parallel Algorithms, IEEE Conference on Industrial Electronics and Applications (ICIEA), pp. 2886–2891, (2007).         [ Links ]

[8] Balkenius, C., Johansson, B., Event Prediction and Object Motion Estimation in the Development of Visual Attention, Proceedings of the Fifth International Workshop on Epigenetic, (2005).         [ Links ]

[9] Makhoul, J., Linear prediction: A tutorial review, Proceedings of the IEEE, April, (1975).         [ Links ]

[10] Yeoh, P., Abu–Bakar, S., Accurate real–time object tracking with linear prediction method, Proceedings of International Conference on Image Processing, Volume 3, (2003).         [ Links ]

[11] Matas, J., Zimmermann, K., Learning efficient linear predictors for motion estimation, Proceedings of 5^th Indian Conference on Computer Vision, Graphics and Image Processing, Springer–Verlag, Madurai, India, (2006).         [ Links ]

[12] Ellis, L. Dowson, N., Linear Predictors for Fast Simultaneous Modeling and Tracking, Proceedings of IEEE 11^th International Conference on Computer Vision, Brazil, (2007).         [ Links ]

[13] Kragic, D., Christensen, H., A framework for visual servoing, Proceedings of the International Conference on Visual Systems, pp. 345–354, Austria, April, (2003).         [ Links ]

[14] Kragic, D., Christensen, H., Survey on Visual Servoing for Manipulation, Technical report, Computational Vision and Active Perception Laboratory, (2002).         [ Links ]

[15] Woods, A., Docherty, T., Koch, R., Image Distortions in Stereoscopic Video Systems, in Stereoscopic Displays and Applications IV, Proceedings of the SPIE, Volume 1915, San Jose, California, (1993).         [ Links ]

[16] Hartley, R., Zisserman A., Multiple View Geometry in computer vision, Cambridge University Press, Second Edition, (2004).         [ Links ]

[17] Scandaliaris, J., Villamizar, M., Robust Color Contour Object Detection Invariant to Shadows, 12^th Iberoamerican Congress on Pattern Recognition (CIARP), Valparaiso, (2007).         [ Links ]

[18] Fuentes, J., Ruíz, J., Rendón, J., Seguimiento y Asimiento de un Objeto en Movimiento por Medio de un Robot Manipulador y Visión Estéreo, Master thesis in Computer Science, Centro Nacional de Investigación y Desarrollo Tecnológico (CENIDET), México, (2009).         [ Links ]

[19] Faugeras, O., Quang–Tuan, L., The Geometry of Multiple Images: The Laws that Govern the Formation of Multiple Images of a Scene and Some of their Applications, Editorial MIT Press, London, England, (2001).         [ Links ]

[20] González R., Woods R., Tratamiento Digital de Imágenes, First Edition, Editorial Addison Wesley Iberoamericana, Wilmington, Delaware, U.S.A, (1996).         [ Links ]

[21] Hartley, R., Sturm, P., Triangulation, Computer Vision and Image Understanding, Vol. 68, No.2, November, (1997).         [ Links ]

[22] M. Lagrange, S. Marchand, M. Raspaud, Enhanced Partial Tracking Using Linear Prediction, Proceedings of the 6th Conference on Digital Audio Effects, London, (2003).         [ Links ]

[23] RobWork, Yet Another Robotics Library. A framework for simulation and control of robotics with emphasis on industrial robotics and their applications. http://www.robwork.dk/.         [ Links ]