Static Segmentation of the Lips for Follow-Up

: Problem statement: The objective of this study particularly aims the static segmentation of the lips. Approach: For that, we proposed algorithms for the determination of certain characteristic points of the lips, as well as a method of optimal slope allowing to trace the cubic ones forming external contours of the lips. Results: Finally to make sure of the performance of our algorithms, we also used a manual determination of the characteristic points. Conclusion: The results obtained showed the rigor of our algorithms and that by comparing the results obtained by the manual determination with those of the semi-automatic determination.


INTRODUCTION
The lips play a big role in the visual perception of the speech. The shape and the motion of labial contours give visual information making it possible to improve comprehension of the speech in a disturbed environment [1,2] . A growing number of studies relates to the automatic processing of the visual speech, in particular on the labial reading.
These last years, of many methods were proposed to solve the problem of the segmentation of the lips.
Delmas [3] presented an algorithm of detection of contours of the lips integrating a preliminary detection of the corners and horizontal extrema of the mouth.
Eveno [4] proposed to use cubic parametric curves to describe contours by imposing the conditions and the limits, with derived from the curves on the levels of the projecting points.
In its study Gacon [5] separated total appearance in two quantities: A static appearance, characteristic of a given speaker and a dynamic appearance which corresponds to the fluctuations induced by the movement (and thus more particularly speech).
Bouvier [6] presented an algorithm for the extraction of external contour of the mouth, where it used two types of methods, a deterministic and another statistic.
In this article, we propose an algorithm of contours extraction of the lips, robust with respect to the change of the speaker and lighting. This algorithm is based on the static segmentation of lips.
In the static segmentation, we choose a semi automatic determination in order to seek the Characteristic Points (CP) of the lips, thereafter we trace contour by a method which we proposed, by using the optimal slope. Moreover, we have includes with the static segmentation a segmentation based on the manual and rigorous site of the Characteristic Points, in order to compare the two methods, manual and semi-automatic.

MATERIALS AND METHODS
Choice of the model: The cubic model [7] which we chose is composed of ( Fig. 1 Static study: The purpose of the static segmentation of the lips is initialization and the extraction of the lips external contours, for that we propose a semi-automatic method for the determination of the characteristic points. We use an active contour called "Jumping snake", in order to determine the cupid arc [8] . For the layout of the lips contours we propose the method of optimal slope for the determination of the cubic ones.

Characteristics of the lips:
To determine the characteristics of the lips, the starting point is to choose a space color. Many authors choose the space RGB, which is best space to characterize the lips compared to the skin [7] . We define thereafter sizes to characterize and segment the lips.

Pseudo-hue:
The pseudo-hue h (x,y) is defined in the pixel (x,y ) of image I [9] by: where, R (x,y) and G(x,y) are respectively the Red and Green components with the pixel (x,y). The value of the pseudo hue is understood between 0 and 1, this value is large for the lips that for the skin.

Gradient hydride:
The hybrid gradient R top (x,y) at the pixel (x,y) of image I [7] is defined by: Such as: The operator gradient hn(x,y ) = Pseudo hue standardized at the point (x, y) Ln(x,y) = The luminance standardized at the point (x, y) The hybrid gradient allows a good detection of the lips higher edge compared to luminance and the pseudo hue.

Determination of the characteristic points:
To determine the characteristic points, we propose to select a rectangle which surrounds the area of the mouth and then we select the P3 point manually in the good position ( Fig. 2). We determine thereafter the remainder of the points using various methods, while starting initially with the high points which constitute the Cupid arc.

Determination of the points P2 and P4:
The high points of the Cupid arc can be given by the method of jumping snake [8] (Fig. 3). The selected points ending will be represented by M i+1 and M -i-1 . The latter are the points which have a maximum flow of the hybrid gradient through the two segments M i M i+1 and M -i M -i-1 .
Two average flows are defined [8] by: where, dn is the normal at the segment. In order that the jumping snake converges on the higher Cupid arc of the lip, at the first time (iteration 0), we force jumping snake to go up to the top, this is done by the reduction in the range of search [θ inf , θ sup ]. We admit for the range of search only two angles: (θ inf = π /9, θ sup = 2π/9) In iteration 1, we divide also the range [0, 2π/9)] in four equal angles. The range of search thus becomes: [π/18, 2π/18, 3π/18, 4π/18] Thereafter, in the other iterations we admit the propagation upwards and downwards.
In this case, the range of search becomes 8 equal values distributed in the following interval: Fig. 4, we represent the right propagation of snake.

Determination of the points of commissures P1 and P5:
The darkest zones of the luminance plan thus appear on the level of the mouth commissures inside the mouth, whether it is opened or closed [3] . Figure 5 represents the minimum of luminance, calculated on all the image. To avoid search on all the columns of the zone selected at the beginning of the segmentation process, we proposed horizontally to divide this zone into 3 equal rectangles, to emphasize the interesting points P1 and P5 located in the rectangle of the medium in order to determine the zone which contains 2 commissures (Fig. 6). For better specifying the zone of commissures, we proposed to also divide rectangle 2 into three semirectangles: One on the right for P5 and on the left another for P1 and a third in the one of the middle. The width of each extreme rectangle is from 15-20 pixels approximately.
We calculate thereafter, the minimum of Lmin luminance on the two semi rectangles. Figure 7 represents the Minimum of Luminance in the rectangle on the Right L minR .
In our algorithm, to calculate P1 and P5 we were interested only in the curves located in top. Indeed, to find P5 (idem for P1), we traverse respectively the minimum of Right Luminance L minR and Left Luminance L minL . And each time we connect to each point of L minR (respectively L minL ) a higher curve connecting this point by the point P4 (respectively P2).
The point P5 (respectively P1) is the point which maximizes the average flow of the hybrid gradient through these curves [3] . Figure 8 represents unfolding to find the P5 point.  (Fig. 9).
In the image, we seek in R 0 line i, corresponding to the maximum value of the following sum: With: i, j = The line and the current column n = Number of points constituting the width of the R 0 rectangle tab = Table corresponding to the values of the pseudohue of image I The line which corresponds to the maximum value of the sum S, its position indicates the ordinate of P6 and the X-coordinate of P6 it is the same one as the Xcoordinate of P3 (Fig. 10).

Layout of the final contour of the lips:
After the determination of the six characteristic points, it thus remains to connect these points by the cubic ones which approach the contour of the lip well, for that we propose the implementation of the method of optimal slope for the calculation of the cubic ones.
We tested this slope on several images and the results obtained are good. Calculation of the optimal slope: That is to say P the intersection point of the vertical line which passes by the P2 point and the horizontal line which passes by P1 (Fig. 11).
To trace the cubic one between the point P1 and P2: that is to say the point M belonging to the segment [P1 P] such as: That is to say M1 the point of intersection between the vertical line passing by the point M and the horizontal line which passes by the P2 point.
β is the angle located between the line (P1M1) and line (P1M).
The slope corresponds to the tangent of this angle: Starting from this slope, we trace a cubic between the P1 point and the P2 point. We proceed in the same way for the lower cubic by using this equality: Fig. 12: Calculation of the CP and traced the cubic ones according to the method of optimal slope Figure 12 shows the layout of the final contour of the lips.

Calculation of the labial parameters:
In present study, we consider the three following labial parameters: • The stretching: Outdistance between the points P1 and P5 • The opening: Outdistance between the points P3 and P6 • Surface: The existing number of points extracted in contour lips

RESULTS
Our results were obtained by the two methods of segmentation namely manual and semi automatic in an environment of equipment and software such as a camera of resolution 5 Méga pixels to acquire the images, a microcomputer of processor INTEL dual core 1.46 GHZ with a random access memory of 1 Go, equipped with the following software: Windows XP service pack2 (32 bits) and the computer programming language is visual C# 2008.

Tests on images:
We test our application on several images containing of the various speakers, betweenothers those having of the barbs and the moustache, to see the effectiveness of our algorithms.
Manual method: We present in Fig. 13, the manual determination of the Characteristic Points (CP) and the layout of external contours of the lips for various speakers. Semi automatic method: The stages to follow for the lips contours layout are presented in Fig. 14.
We present the results of a variety of images developed by our application (Fig. 15) and through those, we note that the contours obtained for various speakers are realistic and that even for men with a moustache or bearded.     The results of the study are presented in Table 1 container the labial parameters (stretching, opening and surfaces), for the various images.

DISCUSSION
For the two methods, we note that the layouts of cubic give realistic segmentations of the lips (Fig. 13-15) and through the table, we notice that the difference between the parameters opening and stretching varies from 1-5 pixels. We can say that this difference is not significant for these parameters. This difference is due to the approximate site of the C.P. For a rigorous site this difference will be very reduced. While basing itself on the layouts of cubic given in Fig. 13-15 and the results of the various parameters. We can say that the application of our algorithms seems to show that the semi automatic method approaches reality better i.e. it is in conformity with the manual method ( Fig. 14 and 15).

CONCLUSION
In this article, we based ourselves on a variety of former researches [3][4][5][6][7][8] and we contributed in the following points: • The manual site of the P3 point in a rigorous way on the lip, for the determination of higher Cupid according to the principle suggested by Eveno [7] • Localization of the corners P1 and P5 by dividing our selection rectangle into portions • Determination of the P6 point on the lower lip, by reducing the zone of search to a R 0 rectangle centered by the line which passes by P3 • The layout of cubic by the method of optimal slope We tested our application on a very large variety of speakers, unconstrained of lighting or make-up and we obtain satisfactory results.
We note through our results, that our application makes the extraction of labial contours for varied images. We see that the shapes of the lips obtained are very realistic and suitable with labial contours. The method is robust even in the following cases such as a bearded speaker, or man with a moustache. Through the Fig. 13-15), we note also conformity between the results of the two methods: Manual and semi automatic.
Lastly, we estimate that the semi-automatic method is satisfactory to be able to exploit it with the follow-up of the lips movements.