MULTIMODAL RETINAL IMAGE REGISTRATION USING RANSAC MATCHING AND GRADIENT ICP ALGORITHM

Multimodal retinal imaging is an important facet of the diagnosis and therapy of retinal disorders like retinopathy, occlusion. Many imaging techniques for multimodal retinal images are developing in the recent years. These novel developments are subjected to a tradeoff between the computational time and effective registration. This study aims at developing a new algorithm based on the RANSAC matching and gradient iterative closest point technique which has proven to have less computational time with the best matched coordinates irrespective of the nature of the input retinal image. This study uses a new adaptive thresholding technique to extract the bifurcations from the target image and the control points are selected using the RANSAC matching algorithm. The registration is achieved by implementing gradient iterative closest point algorithm to minimize the mean square error between the target control points of the base and the reference images.


INTRODUCTION
Retinal image is the major perspective that is used to diagnose and detect the various eye related diseases. This is due to the fact that the retinal image stores the local and temporal information of the retina which can be viewed at any given instance. Human retina consists of many parts and layers out of which the visible parts which give the useful information about the pathological disorders are the optic disc, macula, fovea which is shown in Fig. 1.
Among the various imaging techniques available for retinal imaging fundus, intravenous fluorescein angiography and optical coherence tomography are highly emphasized because of the ample information of the retinal morphology and anatomy.
In Intravenous Fluorescein angiography the functional state of retina and the circulation of blood in the vascular structures are effectively imaged when a fluorescent dye is injected into the blood and the retinal image is captured with a fundus camera. This technique is proven to be more efficient than the fundus camera imaging which becomes more complicated of projecting the 3D retina over a 2D plane (Li et al., 2011).
The retinal image registration is very important for diagnosis as well as therapy planning of various retinal disease like retinopathy, occlusion as shown in Fig. 2. Any image registration technique falls under any one of the two categories: Intensity based registration and Feature based registration. The intensity based techniques register the images by determining the matching criteria and transforming the images accordingly. The matching criterion is chosen based on the grey scale values and finding the cross correlation or mutual information of the grey values. The later one is dependent on the features of the image such as the vessels, bifurcations. In both the cases the registration efficiency is highly dependent on the matching criteria or the extracted feature (Gavet et al., 2012;Jan et al., 2012).
This study presents a new multimodal retinal imaging technique which is based on the feature based registration. Multimodal images are the images of the same organ taken with different imaging technique or same organ imaged at different time.  Multimodal image registration aids the physician to diagnose the consequence of any disease in a course of time. The retinal images have different features out of which the blood vessels and the bifurcations hold the required data regarding the retinal diseases like retinopathy and vein occlusion. Hence this study attempts in extracting the bifurcations from the retinal image by using spectral adaptive thresholding. Once the bifurcations are extracted the control points are selected by RANSAC matching algorithm and the registration is performed by gradient iterative closest point method.

Existing Multimodal Retinal Imaging Registration Techniques
Early research on retinal image analysis is based on vessel segmentation of the fluroscien images carried out by (El Abbadi et al., 2013)With the development of extensive algorithms and innovations in imaging modalities the digital sliced images were replaced by OCT, Fundus photography. Zaart (2010) have suggested that the extraction of bifurcations is a difficult task because of the non uniform illumination of the retinal image. Gavet et al. (2012) have highlighted the techniques of vessel based and non vessel based of retinal images registration. Reel Parminder singh have discussed the fast multimodal image registration using principal component analysis. Yuping (2011) have explained about the registration of 2D from 3D retinal images. Shao Wen (Yang et al., 2010) have insisted the effectiveness of using RANSAC matching for segmentation and registration of bio medical images. Even though there are many available techniques either based on intensity or feature of the image the proposed method has proven to hold good for multimodal retinal images with effective registered output.

PROPOSED REGISTRATION METHODOLOGY
The retinal image obtained is initially preprocessed for the noise removal and to overcome the non uniform illumination of the fovea. The minimum mean square error filtering technique is preferred in retinal images because of its adaptive compatibility. Then the bifurcations are extracted by using the spectral adaptive thresholding algorithm. To provide robustness against the non uniform intensity illuminated retinal image is subjected to RANSAC matching and the registration is successfully accomplished by implementing gradient iterative closest point algorithm. The overall process can be diagrammatically represented as shown in Fig. 3.

Retinal Image Preprocessing and Segmentation
The obtained retinal images are subjected to imaging noise and the noise Amplified Spontaneous Emission (ASE) during image acquisition. Such noise has to be eliminated before registering. The retinal images present a non uniform illumination and the background texture differs from patient to patient and even the image from the same patient presents magnifying difference owing to the effect of disease. Hence rather than using a median or mean filter an adaptive filter is used which can function as both. This can be more clearly stated that the same function will act as a mean filter for some part of the image and median filter some other part of the same image.
The function is given by: where G is the noisy retinal image with a dimension of M×N.

JCS
W l,c is the n×n window function which is centered at (l,c) and n = 2t +1, t≤l≤M-t, t≤c≤N-t, σ 2 is the noise variance and 2 , l c σ is the local variance in the window W l,c , µ L,c is the local mean or average in the same window. The filter operation is such that when the noise variance is approximately equal to the local variance the MMSE works as a mean filter. This condition is achieved when the background area of the image is a constant value. When the retinal image has a local variance which is very much larger than the noise variance then the filter computes a value analogous to the unfiltered image which is closer to the pixel value. The magnitude is often modified by the values of the local and noise variance and the corresponding weighted value is calculated which makes the filter to work as an adaptive filter. This finally reduces the variance component of the noise.
The segmentation of the noise free retinal image helps to extract the bifurcation points of the affected image (Nguyen et al., 2013). The effective segmentation is done with the adaptive multi spectral thresholding algorithm. The high resolution retinal image is subjected to find the sum of squares of orthogonal derivatives. This gives a better output which is independent of the orientation of the image: The segmented vessels are now tested with the iterative closest point algorithm.
The iterative closest point algorithm is widely used technique for the geometrical alignment of 2D and 3D images. This study of retinal image registration is based on this technique in which.
The reference image and the floating image have to be geometrically aligned. This algorithm iteratively refines the transformation by generating corresponding points between the two images repeatedly. Then the distance between the set of points are determined and the error metric is minimized.

RANSAC Matching and Gradient ICP
The ICP algorithm is widely applied for the registration of 3D images. The algorithm essentially holds good for medical images because the available number of data points is relatively less and the algorithm considers all the points in the targeted image (Guo et al., 2011). The ICP algorithm is efficiently used to project the 2D image data on a 3D plane so that the medical diagnosis and treatment can be effectively planned. The retinal images (the floating image and the reference image) are preprocessed and the control points are selected by using RANSAC algorithm.
The initial control points required for the closest point algorithm is determined using RANSAC algorithm (Yang et al., 2010). Assume the segmented retinal image can have a total number of M control points. From this data set N number of control points is randomly selected and X is estimated the number of data that fits the esti ated odel ector X within the er issi le tolerance is calculated. Let this number be K. If K meets the set threshold then it may be accepted to be fit else the above steps are repeated L times. L can be determined by: The algorithm works such that the distance between the two sets of the control points is minimized after each iteration. Basically the floating image is the transformed version of the reference image. The difference between the two images is such that they may be linearly, rotationally transformed from each other. Hence we have the source control points and the target control points. The main objective of this algorithm is to move the source control points to the target control points and hence they are overlapped and finally registered as a single image. Hence this ICP algorithm uses the geometrical information of the retinal image rather than the intensity or color of the image. The source control points are initially randomly mis aligned and after a set of n iterations they get aligned rotationally and after a set of k iterations they get aligned transversely.
The original iterative closest point algorithm was proposed by Besl and McKey. This Gradient Iterative Closest Point algorithm is based on the gradient of the distance between the reference and the target control points and is also advantageous owing to the fact that it supports six degrees of freedom. The retinal image considered for registration is subjected to surface comparison without any deformation or modifications. But the transformation may introduce rotation scaling or translational changes (Oswald, 2010;Guo et al., 2011). This can be overcome by the following steps: Step 1: Determination of the Closest Points by using the Euclidean distance between the sensed image and the floating image.
Step 2: Finding the best rotation angle or translational distance that is required to align the sensed image and the floating image.
Step 3: Restoring the modified values after applying the transformation to make the sensed image and floating image to appear closer.
The practical implementation results in a 4×4 transformation matrix after each iteration. This consists of an inner 3×3 rotation matrix and the fourth column is the translational vector of 3×1: The above determined angles are decomposed into matrix form:

Rx cos Rx -sin Rx sin Rx cos Rx
cos Ry sin Ry Ry -sin Ry The above values are determined and finally all the three values of Rx, Ry and Rz are combined to a single matrix which exhibits the rotation in the entire three axes Science Publications JCS which is given by Rotation Matrix = Rx*Ry*Rz. The final 4×4 transformation matrix is obtained by adding the 3×3 rotation matrix and 3×1 translational vectors.

RESULTS
The retinal images are obtained from the STARE and DRIVE data bases. Few images collected from the Eye Foundation-Ooty were also tested with this algorithm. The results are tabulated such that the Table 1. shows the sample test images and the corresponding bifurcations extracted using adaptive multi spectral thresholding is also shown besides the image size.
The control points are selected from the extracted bifurcations. The number of control points, the number of points that meet the threshold, fits the scale and the success rate are shown in the graphical representation in Fig. 4.
The extracted bifurcations are subjected to RANSAC matching and Table 2 shows the match 1 of reference image, match 2 of floating image, the corresponding transform and the final registered state.
The computational time differs depending on the dimension of the images. The best angle of rotation, best matching coordinates and the computational time for the test images are tabulated in Table 3. The variation in the mean square error is compared when the algorithm is tested using ICP, GICP and a combined RANSAC and GICP. Figure 5 graphically plots the reduction in the mean square error when the gradient ICP is performed after RANSAC matching.

DISCUSSION
This novel method of registration has shown that the the accuracy of retinal image registration is increased when compared to the existing methods from the literature. The algorithm proves to be suitable for retinal images of various dimensions and the retinal image obtained from the patients with severe inflamation due to the retinal occlusion can also be processed to get better efficiency. The initial matching at variuos stages along with the transformation match can also be compared with the reference retinal images which helps in varying the run time parameters like the best matching coordinates which helps in reducing the computational time.

CONCLUSION
Thus the registration of multimodal retinal images is a tedious process because of its structure and non uniform illumination. This study presents a novel method of feature based registration to register multimodal retinal images. The images are preprocessed and segmented and finally registered using the RANSAC matching and gradient iterative closest point algorithm. The results has shown that registration of retinal images using iterative closest point algorithm consumes more computational time than gradient iterative closest point algorithm. Fast convergence is achieved by selecting the control points using RANSAC matching. This further reduces the computational time and the effective results with efficient registration are obtained. In future a hybrid method of registration can be developed by combining this extracted bifurcations and intensity information of those bifurcations.