Automatic Detection of Exudates in Retinal Images using Region-Based, Neighborhood and Block Operation

: Exudates detection is one of the research areas attracting great attention of physicians and scientists. A region-based, neighborhood, block operation and optimal global thresholding are proposed as new methods to exudates detection. The exudates are coarse and fine segmentation following preprocessing steps, i


Introduction
Exudates are a major cause of an increasing percentage of Diabetic Retinopathy (DR) in developing countries, especially in Thailand. It has shown that early detection and treatment are mostly effective for reducing the exudates. Currently, the digital retinal image is widely used in the eye and exudates diagnostic. Therefore, the method for segmentation and detection of exudates with digital retinal image has been proposed. Some research works have been done to segment and detect of exudates using a combination of the threshold technique and global grayscale image techniques (Sanchez et al., 2010). In another work, a region growing method uses to detect the exudates (Sinthanayothin et al., 2002). However, these methods require a long period of time in processing computationally. A third work is a Minimum Distance Discriminant (MDD) operator for classifying and detecting exudates (Goldbaum et al., 1989;Barrett et al., 2001), in this study a MDD operator was created on Red, Green and Blue (RGB) color components of the retinal images. To achieve the same goal, the application described in Goh et al. (2001), an automated detection of the exudates in the digital retinal images using a Principle Component Analysis (PCA) method in RGB color component is adopted. However, this method works only for detection of exudates that can be approximated by the high resolution images it takes into consideration. Sopharak et al. (2010) and Osareh et al. (2002) presented the Naive Bayesian (NB) classifier to indicate the boundary of the exudates. In , the author proposed Gaussian mixture models to categorize each pixel into exudates and non-exudates. The predicted segmentation of exudates was 96.8%.
Besides the method discussed above, many studies have examined exudates detection. Osareh et al. (2009) and Garcia et al. (2009) applied Neural Network (NN) algorithm and Wang et al. (2000) and Sopharak et al. (2008) used NB classifier. In addition, Zhang and Chutatape (2005) and Chutatape and Zhang (2004) used support vector machines (SVM) classifier. On the other hand, Wang et al. (2003) presented an automatic detection method by using a Fuzzy C-Mean Clustering (FCMC) method and then used SVM classifier to distinguish from exudates and background pixel. Walter et al. (2002) used mathematical morphology method to estimate the regions of exudates. These methods achieved an average sensitivity and predictive of 92.8 and 92.4% against a dataset comprising of 15 digital retinal images. Gardner et al. (1996) applied the Artificial Neural Network (ANN) method to categorize each pixel into exudates lesions.  Sopharak et al. (2008) NB and SVM 98.41 Osareh et al. (2002) FCMC 90. 10 Wang et al. (2000) RFB 88. 10 Gardner et al. (1996) ANN 93. 10 Xu and Luo (2009) SVM 84.00 Kittipol and Worawat (2017) FCMC and Morphological 97.45 Comparing ANN method result and Ground-Truth (GT) image, this method achieved accuracy for exudates which is detected as 93.1%. However, a drawback of this method is that it takes time in computation the low contrast of retinal images. Sanchez et al. (2004) relied on the combination of the statistical classification technique and edge detector for exudates detection from 20 color retinal images. Vijaya and Suriya (2010) realized exudates detection by using template matching technique followed by the MDD operator for the segmentation of exudates in the color retinal image. Some of the methods showing the best performance in detecting the exudates presented above are summarized in Table 1. From Table 1, the early work in automated detection of exudates was presented by Sopharak et al. (2008). The authors utilized NB and SVM classifiers to detect exudates and reported an overall Sensitivity (SE), Specificity (SP) and Accuracy (AC) for the detection of exudates are 92.28, 98.52 and 98.41%, respectively. However, the main problems of SVM classifiers are that it takes a time in the training process. Osareh et al. (2002) applied FCMC method to categorize each pixel into exudates or non-exudates class. The classification performance of this method was reported as 90.20% accuracy. However, FCMC method requires the number of suitable clusters beforehand otherwise false results may be returned. Wang et al. (2000) applied Radial Basis Function (RBF) classifier to distinguish exudates from the retinal background. The author reported an accuracy of 88.10% for detecting exudates against the small dataset.
From the above methods for detection of exudates, the quality of the digital retinal images has a great impact on segmentation and extraction stage. As a result, the process of searching the number of exudates is considered time-consuming and requires high performance computer power. Inspired by the previous studies, we have decided to propose a new method in the automatic detection of exudates. The newly proposed method combines image preprocessing methods and feature extraction based on region-based, neighborhood and block operation to analyze exudates in retinal images. Through this method, the exudates are automatically discriminated from non-exudates and an accurate assessment of exudates detection is obtained at pixel based evaluation.

Materials and Methods
The following are some stages of material preparing, which are very important; these stages help to understand about this study.

Image Acquisition
Automatic detection of exudates has been developed using the digital retinal images provided by local dataset and available public dataset. Benchmarking of retinal image analysis methods comprises three mandatory requirements: (1) Digital retinal images from local dataset, (2) a public available dataset (Kalesnykiene et al., 2014) and (3) digital retinal images from an expert ophthalmologist. Here, we have constructed an all dataset of our propose method. In local dataset consists of 412 retinal images (80 non-exudates and 322 exudates) taken with a forty-five-degree field of view from a screening program for diabetic retinopathy in Thailand Eye Hospital. Each image was resized at a resolution of 750×556 pixels. Of the 89 images in DIARETDB1 dataset, 84 images are of patients with exudates and the rest are non-exudates. Fifty-degree field of view images were used. Examples of such images from DIARETDB1 dataset are shown in Fig. 1. Figure  1a show a non-exudates, optic disc, retinal background and blood vessel pixels, but. the exudates image also has contain pathologies and artifacts caused by diabetic retinopathy disease (Fig. 1b).

Ground Truth Images Given by Experts
In exudates segmentation stage, the GT images obtained from experts are considered very useful in algorithms training process. Additionally, the basic image processing, such as color threshold, image segmentation and image recognition for viewing the images, is needed and provided for marking the exudates regions. Examples of ground truth images representing the non-exudates and exudates are shown in Fig. 1c and 1d.

Image Preprocessing
Typically, one main difficulty for detecting exudates from the original retinal images is due to the various colors, brightness and low resolution, which were taken from different patients and in different periods of time. These variations are strongly correlated to color and skin of exudates. Therefore, the non-exudates pixels can wrongly be segmented as the exudates pixels. Therefore, the first sub-method proposed a combination of image preprocessing stages, i.e., color mapping, image enhance the image contrast, fuzzy filtering and optic disc localization. The original image color will be mapped to the color of the image reference and we then applied the color mapping was performed by histogram equalization (Grundland and Dodgson, 2005;Gonzalez and Wood, 1992) to modify the original images values in the dataset though a histogram transformation operator which maps frequency histogram of the reference image. To demonstrate the color mapping effect, the image mapped version and its histogram are shown in Fig. 2. Figure 2 shows two further images in 2(a) and 2(c), which have been mapped image as shown in Fig. 2b and 2d respectively. This can clearly be seen from comparison of the mapped image with the reference image histogram. The color mapping process improves the quality of the different retinal image between individuals. However, the periphery contrast in the retina gradually decreases from the edge to the center of image. Therefore, the decrease of the color intensity will have an impact to detect the exudate's pixels. Consequently, in the second stage, the central part of the retinal image and the optic disc region are both enhanced. We applied global contrast enhancement (Changjiang and Xiaodong, 2006) to exaggerate the contribution of the RGB color information.
As expressed, the global contrast enhancement procedure enhances the image contrast by mapping the initial intensities to the new values which are covering all possible intensities within retinal region. Optic disc and exudates region are mostly showing high intensity values in the channel RGB and thus the global contrast enhancement method assign them the highest intensity values. While the global contrast improving the contrast of the area of interest, it may also enhance the other pixels in retinal image such as noise pixels. Therefore, these pixels can wrongly be detected as exudates pixels. Here, the fuzzy filtering was applied on the retinal images to reduce this effect of the global contrast enhancement technique. This stage, a fuzzy filters the image in the retinal with a 3by-3 filter containing equal weight. The image processed with a fuzzy filter has lessened visible noise. (c) ground truth image of (a); (d) ground truth image of (b) (white pixels represent exudates regions)

Optic Disc Localization
The optic disc shape, size and color may very significantly. The optic disc part located on the retinal is usually brighter than the surrounding area. In either case the optic disc appears as a smaller, brighter region within the retinal. However, the optic disc due to fuzzy boundaries, inconsistent image contrast and missing edge features . Therefore, the localization of the optic disc is surprisingly difficult.
As suggested in previous works, a segmentation of optic disc is very expensive in computer power requiring computation time. In this study, to reduce the computation time and thus increasing the accuracy, three different techniques are proposed including: Morphological transformations, extraction of connected component and feature measurement. The first stage provides an approximate region of the optic disc, while the second state estimates the optic disc region as a circle with connected component technique. Either of these methods are enough to remove the optic disc as candidate the exudates regions. However, the optic disc can sometimes have an elliptical shape and thus baseline plotting of exact the optic disc boundaries is important. Finally, a Sobel edge detector is also applied to generate an accurate the optic disc boundary. The overview of optic disc localization is shown in Fig. 3. The procedures of the optic disc localization are as follows: Step 1: Initialization Read an image Create disc object, size = 7, se Im_close = image_rgb se Finding threshold from Im_close + 0.21; level Convert Im_close → Im_threshold (OD binary image with level = 0.7) Step 2: Create new image Separate an image_rgb to 3 dimensions; OD2_1, OD2_2, OD2_3 Find (image_threshold = = 1) = 0; for each other Compute OD2 = cat(3, OD2_1, OD2_2, OD2_3) Create a structuring (disk, size = 5), se2 Step 3: Imdilate images OD3 = OD2 se2; If find (OD3 = = 0) > 0 OD3 se3; End 3.1 Finding threshold from OD3; graythresh(OD3); called se3 3.2 OD4 = image_threshold2; image_rgb -OD3, 0.01 For k = 1 to 3; OD4 = OD4 se3; End Step 4: Find the optic disc region C = zeros(1, Compactness); If (area > 3000) perimeter = 2*pi*(radius/2); cc = 4*pi*(area/( perimeter* perimeter)); C(i) = abs(area -cc); Else C(i) = inf; End

Finding the Area of Optic Disc
Normally, the optic disc area has the largest high contrast among the circular shape areas. However, to finding the area of optic disc is not an easy task as some parts of the area is obscured by the crossing blood vessels. Therefore, the supervised segmentation based on adaptive mathematical morphology operators are applied (Romain et al., 2005;Crespo et al., 1995). The concept of mathematical morphology operators is defined as Equation 1: In Equation 1, OD represents the real optic disc boundary, OD initail is the initial boundary from the optic disc definition stage. We could find out the interior boundary OD in lying inside the optic disc and the exterior boundary (OD out ) lying outside the optic disc, then we can limit searching area where the real optic disc boundary lies. The binary morphological operators are chosen to obtain OD in and OD out . While thinking each pixel inside the OD initial as optic disc and others as background, a binary image initail OD B is created. Where SE is the structure element. By property choosing the structuring element SE, we can guarantee that . The size and shape of the structuring element controls the tolerance of the displacement between the initial boundary and real optic disc boundary. In this stage, the structuring element is chosen as a square structuring element of size 3. However, the weakness of this step is that the white area does not correspond to the true optic disc regions. Therefore, the location of the optic disc is considered not corresponding with the true optic disc region and requires further processing to be isolated from the other regions. In this way, to tolerance the error between the initial boundary and real optic disc boundary, an adaptive morphological operator is proposed to accurate create the band where the optic disc boundary resides. The procedure of adaptive assignment of structure element should be designed such that the domain of structure element whose center coincides with each pixel belonging to the initial boundary includes the real optic disc boundary by using Equation 2: (2) Fig. 3: Overview of optic disc localization. We extract size, color and feature to perform optic segmentation and localization The closing grey level morphology operation is used to smooth the blood vessel structures while keeping the actual edge of the optic disc. This stage, closing operation by 8-by-8 square structure element is used and accomplished by closing first with a 5-by-5 structure element. This results in a theoretical speed improvement of a factor 5, although in practice the actual speed improvement is somewhat less. To provide the feature measure of the match between localized optic disc region and the optic disc ground truth provided by expert, a converting of grey level image to binary image and locate the optic disc region slightly to the nature of the optic disc by using global thresholding method. The regions of interest in the case are darker than the background, so any pixel was labeled black (optic disc) and any pixel was labeled white (background). The key objective is merely to generate a binary image, so the black-white relationship could be reversed. The result obtained using global thresholding in a value of 136 to segment the grey level image. Afterword, all pixels is used as a mask image and then use superimposed on the retinal image for estimate the shape and optic disc location. However, the weakness of this step is that the white area does not correspond to the true optic disc regions. Therefore, the location of the optic disc is considered not corresponding with the true optic disc region and requires further processing to be isolated from the other regions. In this way, the morphological transformations based on dilation operation is applied to expand and smooth the optic disc region. As dilation only eliminate image detail smaller than the structuring element used, it is convenient to set the structuring element big enough to cover all possible optic disc regions. Here, always used a 8×8 symmetrical disc structuring element, since the optic disc are not wider than 3,000 pixels in color retinal image. Then, we isolated the optic disc region-level to outperform the optic disc pixel-level by the global thresholding again. At this stage, to locate the optic disc region, the threshold value of 132 will be selected. As expected from the clear separation of modes in the thresholding, the segmentation between optic disc and background was very effective.

Extraction of Connected Component
Having segmented of retinal image into a set of optic disc regions, an extraction of connected component was followed to assign a unique label to optic disc region. The optic disc regions are automatically excluded from the background regions based on this technique. With the circle being the most compact region, compactness feature give a measure about how far an object is from a circular shape (Sopharak et al., 2010;Kittipol and Worawat, 2017;Dogra et al., 2017). This is defined by using the following Equation 3: To obtain the region optic disc, which was necessary for both derivations of compactness feature and regions edge feature measurement, an optic disc region tracking was developed on the labelled segmented regions and then overlaid on the original image. The compactness region size image thresholding more than 3,000 pixels is believed to be all possible optic disc regions. Then, small regions by compactness criterion would be removed. The results of locating the optic disc based on this method are illustrated in Fig. 4b. In this case, we can see the optic disc portion on the retinal background. However, a real candidate optic disc boundary has not been estimated. Thus, considered suitable for our proposed optic disc localization, the edge detector of Sobel is introduced to locate the boundary of the optic disc. Figure 4c shows a localized optic disc region and superimposed on the original image.

Segmentation of the Exudates
To construct learning datasets of region of interest pixels, an expert ophthalmologists manually labelled a retinal images and marked the exudates and non-exudates regions. Figure 5 shows a typical image of respective retinal image and its corresponding manually exudates result.
An almost balanced learning dataset of exudates and non-exudates was segmented to any eliminate possible toward either of the two classes. The exudates pixels were collected from 70 segmented abnormal images while the non-exudates pixels were segmented from 50 normal images. In this stage, the optic disc pixels were excluded from those 120 retinal images. There are general segmentation methods in the literature that are functional for a variety of dataset (Chutatape and Zhang, 2004;Kittipol and Worawat, 2017). However, the traditional methods such as FCMC method provide a mechanism to represent and allow pixels to belong to multiple classes with varying degrees of membership. Therefore, this segmentation method make hard decisions about exudates regions. In this study, we investigations are made to identify a robust method to segment the exudates in retinal images. The coarse segmentation of exudates is based on region-based processing. A coarse to fine exudates segmentation method, based upon neighborhood and block operation is developed. The segmentation details are presented in the following two sections.

Coarse Segmentation of Exudates using Regionbased Segmentation
At the coarse segmentation stage, yellow color pixels of interest can be extracted into exudates and non-exudates, which are easily segmented by using region-based processing in RGB color image. Once a color retinal images is segmented, its regions are the identified in term of exudates and background pixels. This paper is proposed, by region-based processing and extracting a set of feature from learning dataset stage and then classifying the exudates and non-exudates based on the feature vectors garnered.
Region and feature based selection is an important step in exudates analysis process since in the segmentation phase only pixel information is used, whereas in this step, region information is also available to be considered. In an RGB image, each color component is a value between 0 and 255, where the intensity 0 usually represents background and the intensity 255 usually represents full intensity or exudates region. A pixel whose color components are (0, 0, 0) displays as black and a pixel whose color components are (255, 255, 255) displays as white. The RGB color components for each pixel are stored along the third dimension of the data array. The red, green and blue color components of the retinal pixel are stored in RGB are shown in Fig. 6

Coarse Segmentation of Exudates using Neighborhood and Block Operations
A neighborhood operation can also be called a Sliding Neighborhood Operation (SNO) (Howard, 1997). A SNO is performed a pixel at a time, with the value of any given pixel in the output image being determined by applying some operation to the values of corresponding input pixels neighborhood. A pixels neighborhood is some set of pixels, defined by their locations relative to that pixel, which is called the center pixel or average pixel. The neighborhood is a rectangular block and as move from one element to the next element in an image matrix, the neighborhood block slides in the same direction. Figure 8 shows the neighborhood blocks for some of the element in 6 by 4 matrix with RGB color component sliding blocks.     Images  0001  205  169  74  0002  211  171  74  0002  214  112  73  0004  203  115  66  0005  201  110  65  0006  198  142  59  …  …  …  …  0089  205  120  68  Average pixel  212  129  Considering in Fig. 8, the average pixel is the actual pixel in the input image being processed by the neighborhood and block operation. For example, in a 2 by 3 neighborhood, the average or center pixel is ([2 3]+1)/2. The function might sum the values of the ten neighborhood pixels and then divide by 10. Therefore, the result and average values of exudates pixels in RGB color component are shown in Table 2.
To investigate the effect of the scale parameter in the RGB image on the coarse segmentation stage, we experimented with RGB values of 212, 129 and 64 respectively. The coarse segmentation results are illustrated in Fig. 9, where the yellow pixels refer to the candidates of exudates and non-exudates pixel respectively.

Finding the Exudates Area of Binary Images
In previous section, a coarse segmentation stages are suitable for exudates segmentation, but it has one problem is does not take into the match between the exudates region and the ground-truth image provided by expert ophthalmologist. Therefore, to select exudates in binary image through optimal global thresholding is proposed. Global thresholding is a method to convert a RGB image into binary image so that exudates of interests are separated from the non-exudates (Sang et al., 1990). Global thresholding using an appropriate threshold T is defined as Equation 4: The procedures of the global thresholding are as follows: Step 1. Select an initial estimate of T.
Step 2. Segment the retinal image using T. This will produce two classes of pixels (exudates and non-exudates). G1 consisting of all pixels brighter than T or exudates pixels and G2 consisting of pixels darker than or equal to T.
Step 3. Compute the average of intensities values mean1 and mean2 for the pixels in regions of exudates and non-exudates.
Step 5. If |T-T(new)|> ∆T, Repeat steps 2 through 4 until difference in T in successive iterations is smaller than a predefined parameter T(new).
The results of producing an exudates using different thresholds are shown in Fig. 10. It is observed from the experiments that, the coarse segmentation and optimal global threshold value T = 126 gives best results for exudates detection in retinal image.

Detection Result Fusion and Pixel based Evaluation
The proposed method is used to evaluate the effectiveness of exudates detection which is based on a pixel-based evaluation and GT image pixels. All new unseen images (412 from the local dataset and 89 from DIARETDB1 datasets) are considered. Each image is then assessed using the proposed method and a final decision was made as to whether the image has some evidence of exudates. The segmentation result for each image pixel is a real value, obtained based on mapping our results with GT image pixels. To have a fair comparison with literature, the performance of the exudates detection method is best described in terms of sensitivity and specificity. If the outcome of such diagnostic test is binary (e.g., exudates and nonexudates), there are four possible results of the test. These criteria quantify the performance according to the True Positive (TP) pixels, False Negative (FN) pixels, False Positive (FP) pixels and True Negative (TN) pixels. For an exudates pixels, the experiment result is True Positive if the detected pixels result is exudates while the result is False Negative if the detected pixels result is non-exudates. For the nonexudates pixels, the result is True Negative if detected result is non-exudates and False Positive if detected pixels result is exudates. An example diagnostic pixels result is shown in Table 3.
For a given set of exudates and non-exudates, the number of TP, TN, FP and FN can be used to determine the accuracy of the diagnostic test in the form of sensitivity and specificity. Sensitivity (SE) gives the percentage to detect the correct exudates pixels in the unseen images while Specificity (SP) is the percentage indicating the correct non-exudates cases. The expression of sensitivity and specificity values measured was mostly near 100%, can be represented as shown in Equation 5 and 6 (Howard, 1997).
The overall detection ability of the method is also measured by using following the Equation 7 In pixels-based evaluation, the retinal image analysis system takes an image as the input and outputs in order to find the score of each pixel. The objective of the pixels-based evaluation is to generate an SE and SP which describes the pixels-level success. Then, the SE and SP can be computed by comparing the output of classification to the pixel-wise ground-truth images.

Simulation Results of the Proposed Method
In the experimental results, we compare the proposed methods with ground-truth images in a pixel-based basis. As expressed earlier in the pixelbased evaluation section, the overall pixel classification accuracy of the exudates detection was obtained in terms of SE, SP and AC against a set of local and DIARETDB1 datasets. The experiments based on the quantitative pixel-based results for the RGB image are illustrated in Table 4. In the coarse segmentation stage in RGB color space, the results reveal that there is some losing of exudates region and sometimes there are extra segmented regions. This clearly affects the pixels-based performance when the parameter with R = 212, G = 129 and B = 64. Then, the optimal global thresholding with value T = 126 was used to indicate the presence of exudates region in the retinal images. This method was obtained an average accuracy are 98.62% by local datasets and 92.14% on images of the available public DIARETDB1 datasets.     In addition, our newly proposed method has performed clearly that the processing time has been much reduced (i.e., from 36s and 39s to 4s across the datasets) as opposed to the recent results in the literature. That is to say, the exudates detection time by Sopharak and Barman (2009) and Sanchez et al. (2004) require 36s and 39s respectively. Therefore, the new methods is recommended, since it produced better performance on a pixels-based basis and can achieve running performance on poor quality images. An example detection of exudates and superimposed in original images are shown in Fig. 11.

Conclusion and Future Work
The proposal of detecting the exudates was two-fold: (1) How to separate the optic disc regions from the retinal images to the candidate exudates regions in poor quality retinal images and (2) how to improve the accuracy of the exudates detection in retinal images as well as to locate the baseline of the exact boundaries of the exudates region. One novelty of the proposed methods is that segment exudates in the color retinal images and these images were preprocessed via normalizing the color of retinal images, contrast enhancement, noise removal and optic disc localization.

Input image RGB segmentation Binary image Sobel edge Overlaid image
We attempted to remove the optic disc pixels from the retinal images by using color-based search segmentation. This was mainly based on morphological transformations, extraction of connected component and feature measurement techniques. The accuracy of the localization was high for the optic disc. Consequently, two approaches for detecting the exudates were proposed. The regionbased, neighborhood and block operation could only provide an estimate location of the exudates region while the optimal global thresholding is used estimate the exudates as a pixel-level.
The final detected and ground-truth images were utilized to assess the performance of the proposed methods. The proposed methods could detect the exudates in local and publicly available DIARETDB1 datasets. The algorithms comprising true retinal images and ground-truth images are used to analyze algorithm performance. Hence, SE, SP and AC are considered frequently used tools for the segmentation performance of exudates in retinal images. The proposed effective methods (neighborhood and block operation with R = 212, G = 126, B = 64 and optimal global thresholding with value T = 126) demonstrated an overall pixelsbased evaluation accuracy of 98.62% including 98.12% SE and 98.23% SP in local datasets.
This showed that the proposed methods are suitable for the automated system of diabetic retinopathy, in particular for the exudates detection. Based on the above-mentioned conclusions, the performance of methods available in the literature, compared to the proposed-method results, which is shown in Table 5. For the overall detecting results, the newly proposed method has been considered effective for detecting exudates despite low quality images. Nonetheless, one of the limitations of this study is that a diagnosis of red lesions in diabetic retinopathy such as haemorrhages and microaneurysms has not been covered. This is due to the fact that the color of these regions and that of blood vessels on OD are considered much similar. Hence, these regions might have been assumed automatically by the algorithms as OD.