3D–Facial Expression Synthesis and its Application to Face Recognition Systems

 

Leonel Ramírez–Valdez¹, Rogelio Hasimoto–Beltran²

 

^{1, 2} Centro de Investigación en Matemáticas (CIMAT) Jalisco s/n, Col. Mineral de Valenciana, Guanajuato, Gto., México 36240 lraval@cimat.mx, hasimoto@cimat.mx.

 

ABSTRACT

One of the main problems in Face Recognition systems is the recognition of an input face with a different expression than the available in the training database. In this work, we propose a new 3D–face expression synthesis approach for expression independent face recognition systems (FRS). Different than current schemes in the literature, all the steps involved in our approach (face denoising, registration, and expression synthesis) are performed in the 3D domain. Our final goal is to increase the flexibility of 3D–FRS by allowing them to artificially generate multiple face expressions from a neutral expression face. A generic 3D–range image is modeled by the Finite Element Method with three simplified layers representing the skin, fatty tissue and the cranium. The face muscular anatomy is superimposed to the 3D model for the synthesis of expressions. Our approach can be divided into three main steps: Denoising Algorithm, which is applied to remove long peaks present in the original 3D–face samples; Automatic Control Points Detection, to detect particular facial landmarks such as eye and mouth corners, nose tip, etc., helpful in the recognition process; Face Registration of a 3D–face model with each sample face with neutral expression in the training database in order to augment its training set (with 18 predefined expressions). Additional expressions can be learned from input faces or an unknown expression can be transformed to the closest known expression. Our results show that the 3D–face model resembles perfectly the neutral expression faces in the training database while providing a natural change of expression. Moreover, the inclusion of our expression synthesis approach in a simple 3D–FRS based on Fisherfaces increased significantly the recognition rate without requiring complex 3D–face recognition schemes.

Keywords: Facial expression synthesis, Finite Element Method, feature points detection, eigenfaces, fisherfaces.

 

RESUMEN

Uno de los problemas principales en los sistemas de reconocimiento de caras es el reconocer una cara con una expresión distinta a la presente en la base de datos, esto es, son dependientes de la expresión de la cara de entrada. Con el propósito de flexibilizar los sistemas de reconocimiento de caras, se propone un método nuevo y eficiente para la síntesis de expresiones faciales en 3D y su aplicación a los sistemas de reconocimiento de caras independiente de la expresión (FRS). A diferencia de los métodos actuales en la literatura, todos los pasos involucrados en la síntesis de expresión facial (eliminación de ruido, registro y síntesis de expresión) son realizados en 3D. Nuestra meta es darle mayor flexibilización a los sistemas 3D–FRS para generar múltiples expresiones a partir de una cara base neutral, la cual es modelada con una malla de elemento finito de 3 capas que representan la piel, el tejido adiposo y el cráneo. Para la realización de la síntesis de expresiones en 3D, el modelo base es complementado con los músculos mas importantes que intervienen en la generación de expresiones faciales. El modelo propuesto se puede dividir en tres pasos principales: Filtrado de Ruido, usado para eliminar los picos (prominentes) presentes en las imágenes de profundidad; Detección de Puntos de Control en la base de datos de caras en 3D, como por ejemplo, punta y grosor de la nariz, puntos en los extremos de los ojos y de la boca, etc.; Registro del modelo base con cada una de las imágenes muestra con cara neutral en la base de datos de entrenamiento, para la generación de expresiones faciales sintéticas y su posterior inclusión en la base de datos misma para incrementar el conjunto de entrenamiento (a 18 expresiones predefinidas). Expresiones adicionales pueden ser aprendidas de las imágenes de entrada o bien expresiones desconocidas pueden ser transformadas a la expresión más cercana en la base de datos. Para medir la eficiencia del 3D–FRS con síntesis de expresiones, utilizamos una técnica muy simple en el reconocimiento de caras conocida con el nombre de FisherFace. Los resultados muestran que el método propuesto representa fielmente la imagen neutral de la base de datos y además, la adición de expresiones faciales sintéticas para el reconocimiento de caras efectivamente incrementa la taza de reconocimiento sin requerir algoritmos complejos para el reconocimiento de caras en 3D.

Palabras clave: Síntesis de expresiones faciales, Método de los elementos finitos, detección de los puntos de control, eigenfaces, fisherfaces.

 

DESCARGAR ARTÍCULO EN FORMATO PDF

 

References

[1] L. Akarun, B. Gökberk, A. Salah. 3D Face Recognition for Biometric Applications. 13th European Signal Processing Conference (EUSIPCO), September 2005.         [ Links ]

[2] C. Beumier, M. Acheroy. Automatic 3D Face Authentication. Image and Vision Computing, 18(4):315–321, 2000.         [ Links ]

[3] P. Belhumeur, J. Hespanha, D. Kriegman. Eigenfaces vs. Fisherfaces: Recognition using Class Specific Linear Projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):711–720, 1997.         [ Links ]

[4] X. Lu, A. Jain. Deformation Modeling for Robust 3D Face Matching. IEEE Trans. Pattern Analysis and Machine Intelligence, 2007.         [ Links ]

[5] C. Chua, F. Han, T. Ho. 3D Human Face Recognition Using Point Signature. Proc. IEEE International Conference on Automatic Face and Gesture Recognition , 233–238, 2000.         [ Links ]

[6] S. Botello, H. Esqueda, F. Gómez, M. Moreles, E. Oñate. Finite Element Method and Applications. Monografía M–AUGTO–2, Guanajuato, Gto., Noviembre 2004.         [ Links ]

[7] K. Bowyer, K. Chang, P. Flynn. A Survey of Approaches and Challenges in 3D and Multi–modal 3D+2D Face Recognition. Notre Dame Computer Science and Engineering Technical Report, 2004.         [ Links ]

[8] J. Cartoux, J. LaPreste, M. Richetin. Face Authentication or Recognition by Profile Extraction from Image Ranges. Proceedings of the Workshop on Interpretation of 3D Scenes, 194–199, 1989.         [ Links ]

[9] D. Colbry, G. Stockman, A. Jain. Detection of Anchor Points for 3D Face Verification. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), 2005.         [ Links ]

[10] V. Contreras. Artnatomy. Anatomical Basis of Facial Expression Learning Tool. Spain, 2005. Available at http://www.artnatomia.net.         [ Links ]

[11] P. Ekman, W. Friesen. Manual for the Facial Action Coding System. Consulting Psychologists Press, Palo Alto, 1977.         [ Links ]

[12] S. Fleishman, I. Drori, D. Cohen–Or. Bilateral Mesh Denoising. International Conference on Computer Graphics and Interactive Techniques, ACM SIGGRAPH 2003, 22(3):950–953, 2003.         [ Links ]

[13] G. Gordon. Face Recognition based on Depth and Curvature Features. 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'92), 108–110, 1992.         [ Links ]

[14] F. Hendriks, D. Brokken, J. van Eemeren, C. Oomens, F. Baaijens, J. Horsten. A Numerical–Experimental Method to Characterize the Non–Linear Mechanical Behaviour of Human Skin. Skin Research and Technology 9(3):274–283, 2003.         [ Links ]

[15] T. Heseltine, N. Pears, J. Austin. Three–Dimensional Face Recognition: An Eigensurface Approach. International Conference on Image Processing (ICIP'04), 2:1421–1424, 2004.         [ Links ]

[16] T. Heseltine, N. Pears, J. Austin. Three–Dimensional Face Recognition: A Fishersurface Approach. Proceedings of the International Conference on Image Analysis and Recognition, LNCS 3212:684–691, 2004.         [ Links ]

[17] B. Horn. Closed–Form Solution of Absolute Orientation using Unit Quaternions. Journal of the Optical Society of America, 4(4):629–642 1987.         [ Links ]

[18] J. Huang, V. Blanz, and B. Heisele. Face Recognition with Support Vector Machines and 3D Head Models. International Workshop on Pattern Recognition with Support Vector Machines (SVM2002), Niagara Falls, Canada, 334–341, 2002.         [ Links ]

[19] H. Huang, U. Ascher. Fast Denoising of Surface Meshes with Intrinsic Texture. Preprint, 2006.         [ Links ]

[20] J. Lee, E. Milios. Matching Range Images of Human Faces. Third International Conference on Computer Vision, 722–726, 1990.         [ Links ]

[21] S. Li, A. Jain (Editors). Handbook of Face Recognition. Springer, 2005.         [ Links ]

[22] X. Lu, R. Hsu, A. Jain, B. Kamgar–Parsi, B. Kamgar–Parsi. Face Recognition with 3D Model–Based Synthesis. International Conference on Biometric Authentication (ICBA'04), LNCS 3072:139–146, 2004.         [ Links ]

[23] A. Moreno, A. Sánchez, J. Vélez, F. Díaz. Face Recognition using 3D Surface– Extracted Descriptors. Iris Machine Vision and Image Processing Conference (IMVIPC 2003), September 2003.         [ Links ]

[24] A. Moreno, A. Sanchez. GavabDB: A 3D Face Database. Proceedings of the Second COST275 Workshop on Biometrics on the Internet, Vigo (Spain), 2004.         [ Links ]

[25] H. Myler, A. Weeks. The pocket handbook of image processing algorithms in C, Prentice Hall, 1993.         [ Links ]

[26] Singular Inversions Inc., FaceGen Modeller. Available at http://www.facegen.com/downloads.htm.         [ Links ]

[27] J. Sobotta. Atlas of Human Anatomy. Volume 1: Head, Neck, Upper Limb. Lippincot Williams & Wilkins, 13th english/english edition, 2001.         [ Links ]

[28] D. Terzopoulos, K. Waters. Physically–Based Facial Modeling, Analysis and Animation. Journal of Visualization and Computer Animation, 1:73–80, 1990.         [ Links ]

[29] J. Vollmer, R. Mencl, H. Müller. Improved Laplacian Smoothing of Noisy Surface Meshes. Computer Graphics Forum, 18(3):131–138, 1999.         [ Links ]

[30] K.Waters. A Muscle Model for Animating Three–Dimensional Facial Expression. 14th Annual Conference on Computer Graphics and Interactive Techniques, 1724, 1987.         [ Links ]

[31] C. Xu, Y. Wang, T. Tan, L, Quan. A New Attempt to Face Recognition using 3D Eigenfaces. 6th. Asian Conference on Computer Vision, 2:884–889, 2004.         [ Links ]

[32] W. Zao, R. Chellappa, P. Phillips, A. Rosenfeld. Face Recognition: A Literature Survey. ACM Computing Surveys, 35(4):399–458, 2003.         [ Links ]

[33] O. Zienkiewicz, R. Taylor. The Finite Element Method. Volume 1: The Basis. Butterworth–Heinemann, 5th. Edition, 2000.         [ Links ]

[34] A. M. Bronstein, M. M. Bronstein, R. Kimmel. Three–Dimensional Face Recognition. International Journal of Computer Vision, 64(1):5–30, 2005.         [ Links ]

[35] F. I. Parke. "SIGGRAPH'89 Course Notes Vol 22: State of the Art Facial Animation", July, 1989.         [ Links ]

[36] Y. Zhang, E. C. Prakash, E. Sung. Modeling and Animation of Individualized Faces for 3D Facial Expression Synthesis. International Journal of Imaging Systems and Technology , 13(1):42–64, 2003.         [ Links ]