Improvement of Image Matching by using the Proximity Criterion: Application to Omnidirectional and Perspective Images

Problem statement: In computer vision, matching is an important phase for several applications (object reconstruction, robot navigati on ...). The similarity measures used provided results which could be improved. Approach: This research proposed to improve image matching by using the proximity criterion. The similarity me asures used mutual information and correlation coefficient. The matching was done between neighbor hoods of points of interest extracted from the images. The second chance algorithm was also applie d. We have worked in case which the sensor had a slight displacement between two images . The tests were performed on omnidirectional and perspective grayscale images. Results: The improvement by introducing the proximity criterion reached 15.9% for non-noise d perspective images, 32.1% for noised perspective images, 47.69% for non-noised omnidirec tional images and 58.5% for noised omnidirectional images. Conclusion/Recommendations: The introduction of the proximity criterion has significantly improved the performanc e of the matching. The method is recommended in mobile robotics, knowing that a good matching leads to a better location and better movement of the robot.


INTRODUCTION
Approaching the human visual system is one of the major goals of computer vision. In this context, the stereoscopic image processing was the subject of much research during recent decades (Hofmann and Gavrila, 2011).
The idea is to make a 3D reconstruction from at least two images of the same scene. These images can be taken by two different cameras or a single mobile camera that captures the scene at two very close moments.
The matching is done between points of interest extracted from two images (Dutta et al., 2011) as it may apply to the graphs (Leordeanu et al., 2011), shapes (Shu and Wu. 2011), contours (Prasad et al., 2006) or regions (Zhang et al., 2010). The outline approach consumes more time compared with the global issues of interest because of the considerable number of points that must match. For this reason, most research focuses on the method based on point of interest initiated by Moravec. In this case, through the epipolar constraints, the correspondence between neighborhoods of points of the image left and right allows the determination of 3D coordinates. If the neighborhood size is reduced, the information available for matching is depleted by cons if the neighborhood size is large the information is more reliable statistically, but the probability of occultation is higher (Keck and Davis, 2011). Other authors have proposed to vary the window size depending on the texture (Aydin and Akgul 2010).
The epipolar constraint comes directly from the geometry of the stereoscopic sensor. It greatly reduces the search space corresponding to the entire image on the epipolar line (Fig. 1). It can apply in cases where the system is first calibrated or not. It is also applicable to other primitives that point i.e., segments (Reisner-Kollmann et al., 2010) or regions (Xiong et al., 2010).
However, the calculated epipolar segment of each item consumes a considerable time and it must know in advance the parameters of the sensor.
Among the applications of stereovision found the construction of map, 3D reconstruction (Khalil et al., 2010), face recognition, 3D motion (Wedel et al., 2010); robot navigation where omnidirectional images is often used (Maohai et al., 2011). The omnidirectional vision is a vision process that provides a sphere of sight of the world observed from its center. It increases the vision fields to collect maximum of information. In the artificial systems, the omnidirectionnality is obtained by the association of a camera and a mirror of revolution which, by reflecting the luminous rays coming from all the directions, forms an omnidirectional image once projected on the sensor.
Several applications have been interested in this type of vision, however their approaches were rather empirical. Nayar and Baker (1997) studied the geometry of the different forms of panoramic mirrors and the formation of images obtained with the sensor. By respecting the laws of the reflection, they built the constraint from the single view point. The resolution of this constraint leads to two equations that represent all classes of catadioptric sensors of single view point. Special cases of these solutions give the shapes of mirrors and the conditions to be met. The mirror used can be plane (Nalwa, 1996), hyperboloid (Rees, 1970;Yamazawa et al., 1993), spherical (Hong et al., 1992, paraboloid (Nayar, 1997) or conical (Yagi and Kawato, 1990;Lin and Bajcsy, 2001). Sturm and Ramalingam (2004), Ramalingam et al. (2005) proposed a method of generic calibration applicable to both types of cameras: with or without a single view point.
In the omnidirectional vision, the assembling used: mirror(s) + camera(s) provides images with a non uniform resolution and involves the geometrical distortions. To apply a classic operator of image processing on the omnidirectional image, a previous adaptation of the neighborhood is needed (Strauss and Comby, 2007;Jacquey et al., 2007;Bazin et al., 2007).
However, in practice the universal neighborhood does not exist because it would be necessary to adapt it not only to the sensor, with a priori knowledge of its calibration, but also to the geometry of the scene (Demonceaux and Vasseur, 2006). Thus, several robotics applications use fixed neighborhoods (Radgui, 2010).
In the omnidirectional stereo vision, as in perspective stereo vision, matching can be used by a single camera (Yi and Ahuja, 2006) or more (Mouaddib et al., 2006). It can be between point of interest, contour (Hwang et al., 2007;Caron and Mouaddib, 2009) or region (Hwang et al., 2007). The epipolar constraints are also suitable for this type of image (Tang et al. 2010).
The aim of this study is to propose a new matching method that we introduced a criterion of proximity to improve performance in omnidirectionnal and perspective images. We compare the results of matching with and without use of the criterion introduced. We also applied the second chance algorithm. The similarity measures used are mutual information and correlation coefficient. The improvement of the matching is shown even on noised images.

MATERIALS AND METHODS
Similarity measure used: Mutual information: The Mutual Information (MI) between two random variables measures the amount of information that knowledge of one variable can make on another. The mutual information between two random variables X = {x 1 ,x 2 ,…x k } and Y = {y 1 ,y 2 ,…y n } is : Such that H is the entropy function and is equal to: With: Mutual information is a positive quantity, symmetric and is cancelled if the random variables are independent.
It follows the principle of no information creation (or Data Processing Theorem): If g 1 and g 2 are measurable functions then: The inequality (5) means that no processing on raw data can reveal information.

Correlation coefficient:
The Correlation Coefficient (CC) between two random variables calculates the degree of linear dependence between them. It is equal to the ratio of their covariance and nonzero product of their standard deviations (Eq. 6): Where: p = Correlation coefficient σ xy = Covariance between X and Y σ x = Standard deviation of variable X σ y = Standard deviation of variable Y The CC is symmetrical and can vary from -1 to 1, values where linearity between two variables is perfect.
If two variables are totally independent, then their correlation is zero. However, the converse is not necessarily true, because there may be a nonlinear relationship between the two variables.
The CC is a measure that has been adapted by several authors for various disciplines of science (Rafida et al., 2009)(Doros and Lew 2010) (Tahani and Abdelfattah 2008).
The difference between correlation coefficient and mutual information is that MI allows measurement of linear and nonlinear dependencies between random variables whereas CC calculates only the degree of linear dependence between variables.
In this study, we chose to match the points of interest extracted from two images. The neighborhood of fixed size is used, including in omnidirectional images.

Detection of points of interest:
The detection of points of interest is a fundamental phase, because it influences the treatment outcome of several applications: 3D reconstruction, image matching, Face Recognition (Yuen et al. 2009). To choose a detection algorithm, we must take into account two criteria: quality and detection time. Depending on the type of application, one is led to focus on one criterion at the expense of another.
There are two main families of interest point detectors: • Detectors based on mathematical operators: Harris and Stephens (1988), , Lindeberg (1998), Mikolajczyk and Schmid (2004) • Detector based on the change of appearance: Moravec (1982), SUSAN (Smith and Brady, 1997), FAST (Rosten, 2006) We chose the use of Harris detector because it is stable, invariant to rotation and has good repeatability.
Followed algorithm: First, we introduce a criterion of proximity taking into account the condition that the images are supposed to be taken at times very close.
We consider the distance between the point P in the left image and the point Q in the right image as: Such that (x P , y P ) are the coordinates of P in the left image and (x Q , y Q ) coordinates of Q in the right image.
The new similarity criterions are: Then, for a detected point of interest P in the left image, we seek the corresponding points in the right image. For this, we calculate the similarity (MID and CCD) between the neighborhood of the point P and the respective neighborhoods of interest points extracted in the image on the right (Fig. 2).
The point Q that maximizes the similarity measure is a candidate to be correspondent if it is above an empirical threshold. We redo the same work for the point Q with the points extracted from the left image (Fig.  3). If the corresponding point Q is the point P, then it is decided that P and Q are related, else we introduce the notion of second chance by using a confidence level. It measures the relative difference between the maximum similarity measure and the next smallest. It is very useful in cases where the maximum similarity measure is achieved by two points or if the values are very close.
We consider R, the second point chosen from right image, using the second chance. If the corresponding point on left image is P, then it is decided that P and R are related, else the point P has no correspondent in the right image.

RESULTS
We have made tests on real and synthetic grayscale images. We chose various images, which contain different structures to better evaluate our method.
The number of points detected in the left perspective images is 113 and in the omnidirectional images is 196.
For a given point in left image, a good correspondence is realised if the corresponding actual is found or decide that it does not correspondent if it did not (1 and Table 2).
Taking into account the condition that the images are supposed to be taken at times very close, the introduction of proximity criterion improves the results of matching in perspectives images by 15.9% in case of mutual information and 14.3% in case of coefficient correlation and in omnidirectional images, the results of matching in perspectives images by 47.6% in case of mutual information and 11.5% in case of coefficient correlation.
The same work was done on noised images by Gaussian noise multiplied by 25, the result of improvement in case of perspective images is 32.1% for mutual information and 20.0% for correlation coefficient (Table 3).
The result of improvement in case of omnidirectional images is 58.5 % for mutual information and 30.0% for correlation coefficient (Table 4).

DISCUSSION
The neighborhood size used is 9×9. We used this size of neighborhood to have a rich sample (81 items) for the calculation of probabilities in order to have a correct measure of similarity. Like Kanade and Lucas method, which is the most used method of estimating of optical flow in robotic applications (Radgui et al, 2011), we choose to browse our images by a neighborhood of fixed size, including the omnidirectional images. The threshold used varies with the type of images.
In the case in Fig. 4, the algorithm without proximity criterion chooses a very far point (R) from the true corresponding (Q) of point P, but by introducing the criterion the algorithm selects the correct corresponding. This is due to the fact that the nearest points are more likely to be chosen for correspondence. The improvement of matching is more important for noised images because the use of the similarity measure alone is not sufficient to determine the corresponding. So the contribution of the proximity criterion is more significant.

CONCLUSION
In this study we presented a new matching method based on a criterion of proximity. The similarity measures used are mutual information and correlation coefficient. Considering that the images are supposed to be taken at times very close, the introduction of the proximity criterion improved significantly the results either on perspective and omnidirectional images. We also tested the good behavior of the proposed method on noisy images. The neighborhood of fixed size is used, including in omnidirectional images.
The results are promising, which encourages us to apply our method to the mobile robotics, knowing that a good matching leads to a better location and better movement of the robot.

ACKNOLEDGEMENT
The researchers wish to thank Cédric DEMONCEAUX and Amina RADGUI for their cooperation.