Active Appearance Models (AAM) are able to align known faces in an efficient manner when face pose and illumination are controlled. The AAM exploit a set of face examples in order to extract a statistical model. There is no difficulty to align a face with the same type (same morphology, illumination and pose) which constitute the example data set. Unfortunately, the AAM are less outstanding from the moment when the illumination, pose and face type changes. AAM robustness is link to the variability introduced in the learning base. The more the AAM will contain variability, the more it will be able to adapt itself to variable faces with the following drawback: the data represented in the reduced parameters space will then form different classes letting appear holes, regions without any data (see Fig. 1). It is therefore very difficult to make the AAM converge in this scattered space. We propose in this paper a robust Active Appearance Models allowing a real-time implementation. To increase the AAM robustness to illumination changes, we propose Oriented Map AAM (OM-AAM). Adapted AAM will be presented after to increase the AAM robustness to any other types of variability (in identity, pose, expression etc.). OM-AAM We propose a specific transformation of the active model texture in an orientation map, which change the normalization process of the AAM. First of all, we apply systematically an adaptive histogram equalization from CLAHE [Zuiderweld 1994] on the images. We then evaluate the horizontal and vertical gradient and simply use the gradient angle on each pixel instead of its gray level. This angle is evaluated on N-a values. In practice we quantify it on eight bits so N-a = 255. The new texture is then made up from an image representing the orientation of each pixel, that we call an oriented map. To overcome a discontinuity problem between 0 and 2Pi associated to similar edge orientations, we realize a mapping (Eq. 11) from [0..2 Pi] to [0..Pi/2]. In order to reduce the noise in uniform regions as illustrated in the background of Figure 3c, we emphasize the signal correlated with the region with high gradient information as it is proposed by [Cooste 2001] in using a non-linear function f (see Eq. 12). During the modelization, the oriented texture from image 14 (Eq. 13) will replace the texture usually used by the AAM. We compare the OM-AAM performance to those of the DM-AAM and classical AAM on two public databases. The first one is dedicated to illumination problems (CMU-PIE: 1386 images of 66 faces under 21 different illuminations [SIM 2002]) and the other one is composed of different faces with several expressions taken in different backgrounds (BIOID: 1521 images [Res01]) under variable light exposition (see Fig. 5). These comparisons will be made in a generalization context: the faces used to construct the model (18 persons from the M2VTS database [Pigeon 1996]) do not belong to the testing databases. We normalize the error found on four relevant points (gravity centers of the eyes, nose and mouth) by the distance between the eyes (see Eq. 15). Figure 6 represents the percentage of the images which have been aligned with the error e. On the CMU-PIE database, the OM-AAM are able to align 94% of the faces with a precision less or equal to 15% where the DM-AAM and classical ones are less efficient: their performances are of 88% and 79% respectively. But when the faces are acquired in real situations, our proposition overcome other methods: on BIOID database, the OM-AAM can align 52% of the faces with a precision less or equal to 15% which represents a gain of 27 and 42% in the performance relatively to classical AAM and DM ones respectively. Adapted AAM As said above, the AAM robustness is related to the face variability in the learning base. Instead of using a very generic model containing a lot of variability, we propose to use an initial model M-o which contains only a variability in identity and then use a specific model M-adapt containing variability in pose and expression. Let a general database contains three types of variability: in expression, identity and pose (see Fig. 7). It is made of several different faces, holding four distinct expressions. Each of the face presents each expression in five poses. The initial model Mo is realized from a database BDDo containing different frontal faces in neutral expression (see Fig. 8). This initial model will be used to perform a rough alignment on the unknown face. Let C-o be the appearance vector after the alignment of the model Mo on the unknown analyzed face. In the reduced space of the model parameters, we seek for the k nearest parameters vectors of Co belonging to the learning initial database BDDo. These k nearest neighbors correspond to the k nearest face of the analyzed one. For example in figure 9, the vector CP will identify the face number p as the most resemble to the analyzed one. From this set of k nearest identities, we generate an adapted database BDDadapt containing the corresponding faces in different expressions and poses, BDDadapt is a subset of the general database (Fig. 7) from which we generate the adapted model M-adapt. When k = 1, 2 or 3 it is possible to evaluate beforehand the adapted model. When we need to align an unknown face in a static image, we then simply align the face with the initial model Mo and apply the pre-computed model which corresponds to the k nearest faces. We test the adapted AAM on the static images of the general database B D Do (Fig. 7). A test sequence is then constituted of one unknown person showing four expressions under five different poses, the learning base associated to this testing base being constituted of all the other persons. A cross-validation of type Leave-one-out is used: every face is tested separately, using all the other one for the learning base. All the faces of the database have been tested, representing at the end a set of 580 images. We compare the performance of our system when k = 2 ("Adapted AAM") to the two other different AAM. The first one ("AAM 28") get identity as the only variability and is constructed from the 28 faces (the twenty-ninth being tested) in frontal view and neutral expression. The second one ("AAM 560") is full variability since it is based on 560 images representing 28 faces showing four expressions under five different poses. Figure 11 shows the superiority of the "Adapted AAM" on the two other models. If we look at the performances at the reference error of 15% our proposition is ten times more rapid and 16% more effective than a model constructed from a more rich database ("AAM 560"). If now we compare the "Adapted AAM" to the "AAM 28" which has the same computational complexity, it is more effective in 55% of the cases (94% versus 49%). As a conclusion our model is more rapid and effective compared to the other models because it focuses on a relevant database relatively to the testing face.
