Improvement of image matching by using the proximity criterion

The aim of this work is to develop a new algorithm for matching stereo images to the 3D reconstruction. We propose the use of a proximity criterion, to improve performance, and applying of the second chance algorithm. The similarity measures used are mutual information and correlation coefficient. The matching is done between neighborhoods of points of interest extracted from the images. We work in case which the sensor has a slight displacement between two images. The tests are performed on grayscale images.


INTRODUCTION
Approaching the human visual system is one of the major goals of computer vision. In this context, the stereoscopic image processing was the subject of much research during recent decades. The idea is to make a 3D reconstruction from at least two images of the same scene. These images can be taken by two different cameras or a single camera captures the scene at two very close moments. The matching is done between points of interest extracted from two images as it may apply to the contours. The outline approach consumes more time compared with the global issues of interest because of the considerable number of points that must match. For this reason, most research focuses on the method based on point of interest initiated by Moravec [10]. In this case, through the epipolar constraints, the correspondence between neighborhoods of points of the image left and right allows the determination of 3D coordinates. If the neighborhood size is reduced, the information available for matching is depleted by cons if the neighborhood size is large the information is more reliable statistically, but the probability of occultation is higher [11]. In [6] the authors proposed to vary the window size depending on the texture. The epipolar constraint comes directly from the geometry of the stereoscopic sensor. It greatly reduces the search space corresponding to the entire image on the epipolar line ( Fig. 1). It can apply in cases where the system is first calibrated or not [8] [20]. It is also applicable to other primitives that point [17], ie segments [15] [2] or regions [5] [19]. However, the calculated epipolar segment of each item consumes a considerable time and it must know in advance the parameters of the sensor.
Among the applications of stereovision found the construction of map, navigation for robotics [10], reconstruction of objects, face recognition [13] and others.
We propose a new matching method that we introduced a criterion of proximity to improve performance. We compare the results of matching with and without use of the criterion introduced. We also applied the second chance algorithm. The similarity measures used are mutual information and correlation coefficient.
In the following we present the similarity measures used, then we present the proposed method, and finish with a conclusion and perspectives.

A. Mutual information
The mutual information (MI) between two random variables measures the amount of information that knowledge of one variable can make on another. The mutual information Such that H is the entropy function and is equal to: Mutual information is a positive quantity, symmetric and is cancelled if the random variables are independent. It follows the principle of no information creation (or Data Processing Theorem): If 1 g and 2 g are measurable functions then: (5) means that no processing on raw data can reveal information. The MI is a universal similarity measure [23][3] [13] which is used in stereo matching [21], image registration [1], parameter selection [22].

B. Correlation coefficient
The correlation coefficient (CC) between two random variables calculates the degree of linear dependence between them. It is equal to the ratio of their covariance and nonzero product of their standard deviations (equation 6) y x xy σ σ σ ρ = ( 6 ) where: ρ : correlation coefficient; xy σ : covariance between X and Y; x σ : standard deviation of variable X; The CC is symmetrical and can vary from -1 to 1, values where linearity between two variables is perfect. If two variables are totally independent, then their correlation is zero. However, the converse is not necessarily true, because there may be a nonlinear relationship between the two variables. The difference between correlation coefficient and mutual information is that MI allows measurement of linear and nonlinear dependencies between random variables whereas CC calculates only the degree of linear dependence between variables.

III. PROPOSED METHOD
We chose to match the points of interest extracted from two images.

A. Detection of points of interest
To choose a detection algorithm, we must take into account two criteria: quality and detection time. Depending on the type of application, one is led to focus on one criterion at the expense of another.
There are two main families of interest point detectors: -Detectors based on mathematical operators: Harris [4], Shi and Tomasi [14], Lindeberg [7], Harris-Laplace [9] -Detector based on the change of appearance: Moravec [10], SUSAN [16], FAST [12] We chose the use of Harris detector because it is stable, invariant to rotation and has good repeatability. We can also control the number of points detected for each image by changing the parameters of the Harris detector for the purpose of alleviating the calculations.

B. Followed algorithm
First, we introduce a criterion of proximity taking into account the condition that the images are supposed to be taken at times very close. We consider the distance between the point P in the left image and the point Q in the right image as: seek the corresponding points in the right image. For this, we calculate the similarity (MID and CCD) between the neighborhood of the point P and the respective neighborhoods of interest points extracted in the image on the right (Fig 2). The point Q that maximizes the similarity measure is a candidate to be correspondent if it is above an empirical threshold. We redo the same work for the point Q with the points extracted from the left image (Fig. 3). If the corresponding point Q is the point P, then it is decided that P and Q are related, else we introduce the notion of second chance by using a confidence level. It measures the relative difference between the maximum similarity measure and the next smallest. It is very useful in cases where the maximum similarity measure is achieved by two points or if the values are very close.
We consider R, the second point chosen from right image, using the second chance. If the corresponding point on left image is P, then it is decided that P and R are related, else the point P has no correspondent in the right image.

IV. EXPERIMENTAL RESULTS AND DISCUSSION
We have made tests on eight pairs of stereoscopic images of size 187x250 pixels (Fig 5). We chose various images, which contain different structures to better evaluate our method. The number of points detected in the left images is 113. The neighborhood size used is 9x9. We used this size of neighborhood to have a rich sample (81 items) for the calculation of probabilities in order to have a correct measure of similarity. The threshold used varies with the type of images.
For a given point in left image, a good correspondence is realised if the corresponding actual is found or decide that it does not correspondent if it did not.
In the case in Figure 4, the algorithm without proximity criterion chooses a very distant point (R) from the true corresponding (Q) of point P, but by introducing the criterion the algorithm selects the correct corresponding.
Taking into account the condition that the images are supposed to be taken at times very close, the introduction of proximity criterion improves the results of matching by 15.9% in case of mutual information and 14.3% in case of coefficient correlation. This is due to the fact that the nearest points are more likely to be chosen for correspondence (Table1 and Graph1).
The same work was done on noised images by Gaussian noise multiplied by 25, the result of improvement is 32.1% in case of mutual information and 20.0% in case of correlation coefficient (Table 2). In this paper we presented a new matching method based on a criterion of proximity. The similarity measures used are mutual information and correlation coefficient. Considering that the images are supposed to be taken at times very close, the introduction of the proximity criterion improved significantly the results.
The results are promising, which encourages us to apply our method to the mobile robotics after compared it with existing methods.