Les commentaires sont clos.

Workshop on 3D and 2D Face Analysis and Recognition

7 Janvier 2011

Catégorie : Journée d étude

Workshop on 3D and 2D Face Analysis and Recognition

Proposed by Pr. L.Chen (ECL Liris), Pr. Mohamed Daoudi (LIFL/Telecom Lille 1), Pr. J.L.Dugelay (Eurecom)

Jan. 28, 2011At Université de Lyon Ecole Centrale de Lyon, Amphithéâtre 202 36 avenue Guy de Collongue, 69134 Ecully Cedex Access plan : Free access with mandatory registration

Contact : Isabelle Dominique,, tél : +33 472 18 64 42 Pr.Liming Chen,, tél : +33 472 18 65 76


L'accès au workshop est gratuit mais l’inscription est obligatoire.

La fiche ci-dessous complétée est à retourner le plus rapidement possible à l’adresse suivante :

Nom :
Prénom :
Adresse électronique :
Téléphone :

Etablissement/Entreprise :
Laboratoire/Entité :
Adresse :

Votre centre d’intérêt :



9h00-9h25. Welcome 9h25-9H30. Opening address, By Pr. L.Chen

Session I – 3D face analysis and processing, chair : Pr. L. Chen

1)9h30-10h05. “Facial Surfaces Analyzing by Using Riemannian geometry”, by Prof. Mohamed Daoudi, LIFL, Telecom Lille 1

In this talk we explore the use of Riemannian geometry to analyze shapes of facial surfaces. That is, we define a differentiable manifold, with a suitable Riemannian metric, whose elements represent individual facial surfaces. In particular, we obtain algorithms for computing geodesics, computing statistical means, and stochastic clustering.We demonstrate these ideas in two application contexts: biometric applications and facial expression recognition.

Mohamed Daoudi is a Full Professor of Computer Science in the Institut TELECOM ; TELECOM Lille 1, LIFL (UMR 8022). He received the Ph.D. degree in Computer Engineering from the University of Lille 1 (USTL), France, in 1993 and Habilitation à Diriger des Recherches from the University of Littoral, France, in 2000. He was the founder of the MIIRE research group of LIFL (UMR 8022). His research interests include pattern recognition, image processing, three-dimensional analysis and retrieval and more recently 3D face recognition. He has published more than 80 papers in refereed journals and proceedings of international conferences. He is the coauthor of the book "3D processing : Compression, Indexing and Watermarking (Wiley, 2008)". He has served as a Program Commitee member for the International Conference on Pattern Recognition(ICPR) in 2004 and the International Conference on Multimedia and Expo (ICME) in 2004 and 2005. He was a co-organizer and co-chair of ACM Workkshop on 3D Retrieval 2010 and Eurographics 3D Object retrieval 2009. He has organized a special session on 3D Face Analysis and Recognition at ICME 2008.He was an associate editor of the Journal of Multimedia (2006-2009). He is a frequent reviewer for IEEE Transactions on Pattern Analysis and Machine Intelligence and for IJCV, JMIV. His research has been funded by ANR, RNRT and European Commission grants. He is a Senior Member of IEEE.

2)10h05-10h40. “3D face segmentation based on high curvature edge detection for harmonic map alignment”, by Przemyslaw Szeptycki, Liris, ECL

In this talk, we propose an automatic open mouth detection algorithm based on an analysis of one of the principal curvatures. While facial surface deformation during expression is assumed to be near-isometric, mouth opening significantly changes the surface topology and introduces anisometry, thus making Harmonic Mapping inconsistent. We show how removing the open mouth part and using an modified geodesic distance for expression invariant face segmentation provides more reliable data for further stages of the 3D Face processing pipeline, that require consistently segmented faces, including the computation of Harmonic Maps. The algorithm was evaluated on two 3D face datasets, thus testing performance both on noisy models (FRGCv2) and for large facial expressions (Bosphorpus). In order to evaluate the need for open mouth detection, the statistics of mouth opening in terms of position and size were evaluated for a number of common labeled facial actions.

Przemyslaw is a PhD student under the supervision of Dr.Mohsen Ardabilian and Prof. L.Chen. His current interests include 3D face preprocessing, 3D face landmarking and 3D face segmentation

10h40-10h50. Coffee break

Session II – 3D Face Analysis and Processing, chair : Pr. Mohamed Daoudi

3)10h50-11h25. “Occlusion in 3D face recognition”, by Dr.Alessandro Colombo, University of Milano-Bicocca

The greater part of state of the art work in face recognition does not consider the occlusion issue; only a few approaches which consider 2D images can be found in the literature. In this seminar it will be presented an innovative three dimensional face detection and face restoration strategy for the recognition of three dimensional faces which may be partially occluded by unforeseen, extraneous objects. No a-priori knowledge about the occluding objects is required. These may be glasses, hats, scarves and the like, and differ greatly in shape or size, introducing a high level of variability in appearance. The restoration strategy is independent of the method used to detect occlusions and can also be applied to restore faces in the presence of noise and missing pixels due to acquisition inaccuracies. Results obtained on artificial and real datasets will be presented.

Alessandro Colombo took his degree in Computer Science in 2004 and his Ph.D. in Computer Science in 2008 at DISCo, Department of Information Science, Systems Theory, and Communication at the University of Milano-Bicocca. He is currently a post-doc researcher at the Imaging and Vision Laboratory ( His research interests cover the processing, analysis and synthesis of 2D and 3D images. In particular, he focused his research on 2D/3D detection and recognition of objects and faces.

4)11h25-12h00. “3D face analysis”, by Prof.Wang Yunhong, Beihang University, China

As one of the most important organ of human beings, face conveys a significant amount of information, including the identity, emotion, gender, age, ethnicity etc., which plays an important role in face-to-face communications. Human can acquire this information easily, but it’s difficult for computers to possess the same ability. Enabling computers to understand various information that human face conveys is the target always pursued by the computer vision and pattern recognition communities. Due to its non-intrusive and user friendly advantages, automatic interpretation of face information has promising applications in public security, access control, human-computer interaction, image retrieval, computer animation, etc. Although studies have shown that facial attributes such as the aging, gender and ethnicity are revealed not only by the 2D textures but also by the 3D morphology of human faces, most of the proposed works on face analysis are 2D image based, which are sensitive to illumination and pose variations. In order to deal with this problem, and with the rapid development of 3D capturing equipments, 3D face analysis is attracting more and more attentions. This talk will present our ongoing research work on 3D facial feature point localization and registration, feature extraction and matching, multimodal gender and ethnicity classification, and face aging.

Yunhong Wang is Professor of Computer Science at Beihang University. She received her PhD in 1998 and worked in the National Lab of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences from1998 to 2004. She transferred to Beihang University at 2004. She has been working on pattern recognition and image processing. She served as program committee members of many important academic conferences such as International Conference on Pattern Recognition (ICPR), International Conference on Biometrics (ICB) etc. Her current research interests include digital watermarking, pattern recognition and image processing.

12h00-13h30. Buffet

Session III – 3D and 2D Face Analysis and Processing, chair : Pr. Yunhong Wang

5)13h30-14h05. “Asymmetric 3D/2D Face recognition”, By Huang Di, Liris, ECL

3D Face recognition has been considered as a major solution to deal with these unsolved issues for reliable 2D face recognition in the recent years, i.e. illumination and pose variations. However, 3D based technique is currently limited by its high registration and computation cost. In this talk, asymmetric 3D- 2D face recognition is presented, enrolling people in textured 3D environment but performing identification in 2D automatically. The goal is to limit the use of 3D data to where it really helps to improve recognition accuracy. The proposed method consists of two separate matching processes: Sparse Representation Classifier (SRC) is applied in 2D-2D matching, while Canonical Correlation Analysis (CCA) is utilized to learn the mapping between range LBP face (3D) and texture LBP face (2D). Both matching scores are further combined for final decision. Moreover, new designed preprocessing step enhances its robustness to illumination and pose effects. The proposed approach achieves much better experimental results in the FRGC v2.0 database than 2D algorithms do, while avoiding the high cost of data acquisition and computation in 3D approaches.

Huang Di is a PhD student at Liris ECL under the supervision of Dr.Mohsen Ardabilian, Prof. Yunhong Wang and Prof. Liming Chen. His current research interests include 2D/3D facial representations, 2D/3D asymmetric face recognition, lighting normalization

6)14h05-14h40. “Biometric Application for 3D face model”, by Dr Stéphane Gentric, Morpho, Safran Group, France

The limitation of considering a face as a 2D object appears in most of biometric applications. This talk will present how 3D face morphable models are used in two different products, what are the problems that are currently solved and what are the remaining issues. The first application is one of the tools proposed for face images enhancement. It allows an operator to simply drive the fitting of 3D face model on multiple bad quality images, in order to build a frontal synthetic face image, by merging textures. We will show how much it improves the biometric performances. The second application is Face-On-The-Fly, an automatic gate for border crossing system. From four video streams, it builds a frontal face of a subject, during the crossing the gate, without any stop and any cooperation. Fitting of a 3D face model, image synthesis, feature extraction, matching against passport image and door opening decision are done in real time. For most of the face recognition issues (pose, expression and illumination), the use of 3D models open new perspectives and more complex challenges.

Stéphane Gentric is Research Unit Manager at Morpho ( He receives his PhD in 1999, on Pattern Recognition at UPMC. From 1999 to 2002, he worked mainly on fingerprint algorithms. From 2002, he focused on Face Recognition, then Iris Recognition. He is now, team leader for both biometries, driving all algorithmic aspects, from Acquisition Device to Large Scale Matching System. He was involved in most of Morpho’s projects in biometrics of the past 10 years, such as Smartgate Australian border crossing System as well as NIST benchmarks, or the UIDAI project. His current research interests stay pattern recognition for improvement of biometric systems.

7)14h40-15h15 .“A unified Neural Scheme for facial Image Understanding”, by Prof. Christophe Garcia, Liris, INSA-Lyon

Over the last decade, facial image processing has become a very active research field due to the large number of possible applications, such as model-based video coding, image retrieval, surveillance and biometrics, visual speech understanding, virtual characters for e-learning, entertainment and intelligent human-computer interaction. With the introduction of new powerful machine learning techniques, statistical classification methods and complex deformable models, recent progress has been made on face detection and tracking, person identification, facial expression and emotion recognition, gender classification, face coding and virtual face synthesis. However, much progress is still to be made to provide more robust systems, in order to cope with the variability of facial image appearances caused by lighting conditions, poses and expressions, image noise and partial occlusions, in an unconstrained, real-world context. Among these machine learning approaches, Convolutional Neural Networks (ConvNets) are powerful models that tightly couple local feature detection, global model construction and classification in a single architecture where all parameters are conjointly learnt. They alleviate the limitations of the traditional hand-designed feature extraction and selection steps, by automatically learning optimal filters and classifiers that are very robust to noise. In this presentation, we will show that ConvNets are very effective for facial image processing by presenting different architectures and learning schemes, designed for face detection, facial feature detection, face alignment, gender classification and face recognition.

Christophe received his PhD degree in computer vision from the University of Lyon I, France, in 1994 and his "Habilitation à Diriger des Recherches" (HDR) from Insa-Lyon / University of Lyon I, in 2009. From 1992 to 1997, he has been involved in various computer vision and robotics research projects at the IBM Vision Automation Group, France, the Computer Vision Center of the Autonomous University of Barcelona, and the German National Research Center (now Fraunhofer Institute). From 1997 to 2000, he has been a researcher at the Foundation for Research and Technology Hellas (FORTH), Greece, where he was involved in several advanced EU projects in the field of video and image analysis. From 2000 to 2002, he was a visiting Professor at the Computer Science Department of the University of Crete, Greece, where he was teaching Artificial Neural Networks and Pattern Recognition. In 2003, he spent 10 months at IRISA- INRIA, Rennes, France, working in the field of automatic video structuring and indexing. From 2004 to 2010, Christophe has been working in France Telecom R&D / Orange Labs, as a Fellow Expert Researcher in Pattern Recognition, Neural Networks and Image Indexing and manager of the Multimedia Content Analysis Technologies group. Since November 2010, He is a Full Professor at INSA-LYON, working in the “Laboratoire d'InfoRmatique en Image et Systèmes d'information” (LIRIS). His current technical and research activities are in the areas of neural networks, pattern recognition, image and video indexing, and computer vision. He holds 22 industrial patents and has published more than 110 articles in international conferences and journals. He is currently associate editor of the Int. Journal of Visual Communication and Image Representation (Elsevier), Image and Video Processing (Hindawi), Pattern Analysis and Application (Springer-Verlag) and Pattern Recognition (Elsevier).

15h15-15h30. Coffee break

Session IV – 2D face recognition and biometrics, Chair: Pr. Christophe Garcia

8) 15h30-16h05. “Face Recognition with Patterns of Oriented Edge Magnitudes”, by Son VU & Prof. Alice Caplier

This talk addresses the question of computationally inexpensive yet discriminative and robust feature sets for real-world face recognition. The proposed descriptor named

Patterns of Oriented Edge Magnitudes (POEM) has desirable properties: POEM (1) is an oriented, spatial multi-resolution descriptor capturing rich information about the original image; (2) is a multi-scale self-similarity based structure that results in robustness to exterior variations; and (3) is of low complexity and is therefore practical for real-time applications. Briefly speaking, for every pixel, the POEM feature is built by applying a self- similarity based structure on oriented magnitudes, calculated by accumulating a local histogram of gradient orientations over all pixels of image cells, centered on the considered pixel. The robustness and discriminative power of the POEM descriptor is evaluated for face recognition on both constrained (FERET) and unconstrained (LFW) datasets. Experimental results show that our algorithm achieves better performance than the state-of-the-art representations. More impressively, the computational cost of extracting the POEM descriptor is so low that it runs around 20 times faster than just the first step of the methods based upon Gabor filters. Moreover, its data storage requirements are 13 and 27 times smaller than those of the LGBP (Local Gabor Binary Patterns) and HGPP (Histogram of Gabor Phase Patterns) descriptors respectively

9) 16h05-16h40. “Face recognition in the wild: verification and caption-based recognition”, by Dr. Jakob Verbeek, INRIA Rhône-Alpes

In this talk I will present our recent work on face verification and recognition in uncontrolled settings. We work with images taken from Yahoo News and the associated captions, from which we automatically extract faces and names respectively. The data was manually annotated, yielding over 30.000 faces of over 5.000 individuals. We compute high dimensional redundant face descriptors, anchored at facial "parts" such as the mouth, nose, and eyes that are automatically located using a constellation model. Metric learning is used to obtain a representation that is compact and robust to intra- person expression and pose changes, while being sensitive to inter-person appearance variations. Experimentally we evaluate this approach for face verification, and its impact on caption based recognition. In the former task, we are interested in determining whether two face images represent the same person or not. In the latter task the goal is to automatically associate faces in an image to names found in the caption.

Jakob Verbeek is a researcher in the LEAR (learning and recognition in vision) team at INRIA Rhone-Alpes. His research focuses on machine learning approaches to solve computer vision problems. Recent work includes methods for automatic image annotation, face recognition, and semi-supervised image categorization.

10) 16h40-17h15. “Person recognition using a bag of soft biometrics (BoSB)”, by Antitza DANTCHEVA and Prof. Jean-Luc DUGELAY. (

We introduce and examine the novel idea of using a bag of soft biometrics for person recognition. This novel tool inherits the non-intrusiveness and computational efficiency of soft biometrics, which allow for fast and enrollment-free biometric analysis, even in the absence of consent and cooperation of the surveillance subject. In this work we provide insight on general design aspects in soft-biometric systems, and different aspects regarding capabilities, challenges and efficient resource allocation. Moreover we propose a specific soft biometric system including traits like hair, eye and skin color, as well as the presence of beard, moustache and glasses. In conjunction with the system design and detection algorithms, we also proceed to shed some light on the statistical characterization of different parameters that are pertinent to the proposed system. b. Antitza Dantcheva is a Ph.D. student at EURECOM, Sophia Antipolis under the supervision of Prof. Jean-Luc Dugelay. Her current research interests are in soft biometrics: algorithms and methods. Recent work include studies on eye color as a soft biometric trait and on the reliability of soft biometric systems.

11) 17h15-17h30. “Conclusion & discussion”, Prof. L.Chen

Contact : Isabelle Dominique,, tél : +33 472 18 64 42 Pr.Liming Chen,, tél : +33 472 18 65 76