Segmentation of Exudates via Color-Based K-Means Clustering and Statistical-based Thresholding

: This paper provides a novel approach for the problem of detecting the yellowish lesions in the eye fundus images, such as hard and soft exudates, in a fully-automated manner. To solve this problem of segmenting exudates automatically, the fundus image was first converted into the L*a*b* color space to decouple the chromaticity information of the image. Next, the fundus image was partitioned into five disjoint clusters based on this information via the unsupervised k-means algorithm. Among the clustered images, the one having the brightest average intensity was chosen to be the best cluster containing all the bright yellowish pixels. Using this cluster, a threshold value was estimated via statistic-based metrics and subsequently applied to remove any non-bright clustered pixels and preserve only the relatively bright ones within the image. Finally, the optic disc was eliminated from the thresholded image, leaving out only the bright abnormalities. This approach was evaluated over a total of 1419 images retrieved from three heterogeneous datasets:


Introduction
Segmenting fundus abnormalities occurring in the interior surface of the eye shall assist ophthalmologists in diagnosing and treating serious eye diseases such as glaucoma, diabetic retinopathy and macular degeneration. Diabetic retinopathy particularly is a major sight-threatening complication caused by diabetes mellitus. Diabetes harmfully affects the retina (diabetic retinopathy), as well as the kidneys (diabetic nephropathy) and the nervous system (diabetic neuropathy) as stated by Patton et al. (2006) and Allam et al. (2015). Also, WHO (2016) reported that 2.6% of global blindness could be attributed to diabetic retinopathy, in which the number of people with diabetes had risen from 108 million in 1980 to 422 million in 2014.
Accordingly, the greatest emphasis in automated diagnosis had unsurprisingly been given to the detection of the symptoms of diabetic retinopathy. Ghoneim (2007) categorized these indicators into two classes: "Bright/yellowish abnormalities" which include hard exudates and cotton wool spots and the "dark/red abnormalities" which include hemorrhages and microaneurysms. Fig. 1 illustrates such bright and dark abnormalities within the diabetic fundus image. As noticed in Fig. 1, bright abnormalities such as hard exudates and cotton wool spots are mainly characterized by their yellowish color. This is also the characterizing feature of the circular optic disc which appears on the right of the fundus image illustrated in Fig. 1. But on the contrary to the optic disc, such bright abnormalities have varying sizes (i.e., tinytolarge), irregular shapes and unspecific random positions within the fundus image.

Problem
In large-scale screening tests of numerous number of patients, it is normally inefficient to rely solely on ophthalmologists to detect the exudates that may be found in the captured fundus images. Therefore, it is extremely necessary to rely on automated computer vision systems to detect such abnormalities that diabetic patients suffer from.

Challenges
Due to their similar color and appearance within the fundus image, exudates may be confused with the optic disc. Also, such abnormalities may be indiscernible (i.e., hard to detect) in low-quality images.

Objective
Accordingly, the main objective of this paper is to provide a computer-aided automated technique for detecting and segmenting the bright yellowish lesions, which significantly affect the normal appearance and semblance of the fundus.

Advantages
The proposed technique contributed in presenting a detection strategy that was fully automated, regardless of the extreme heterogeneity of the screened images (e.g., spatial size, field of view, image quality, etc.); as no image-dependent parameters were adjusted, nor training images were utilized, nor predefined templates were exploited for the sake of customizing the proposed algorithm over certain image datasets.
The remainder of this paper is organized as follows. First, it reviews the previous work pertaining to exudates segmentation, followed by a section that describes our proposed method for segmenting the bright abnormalities such as hard and soft exudates. Finally, the paper presents the results of the proposed technique, followed by a concluding section. Harangi et al. (2012) enhanced the visual appearance of the exudates by eliminating the optic disc as well as improving the contrast of the green channel via contrastlimited adaptive histogram equalization (CLAHE). Then, they identified the candidate regions containing exudates via morphological closing and reconstruction. Consequently, 56 descriptors were extracted for each candidate pixel to classify them, where the most relevant features were selected to train a boosted naïve Bayes classifier, achieving a sensitivity of 63% over the DIARETDB1 dataset.

Related Work
Similar to the previous preprocessing method, Jaafar et al. (2011) first enhanced the contrast of the green channel using CLAHE. Consequently, the hard exudates were detected by top-down image segmentation to partition the image into two homogeneous regions followed by Canny edge detection and region growing to segment the candidates of hard exudates. These candidates were classified into either "exudates" or "non-exudates" using a rule-based classifier. The proposed method was trained using the DIARETDB0 dataset and successfully detected the hard exudates in 93.2% of the 106 images picked from the DIARETDB1 and MESSIDOR datasets. Kumar et al. (2013) employed histogram analysis to detect exudates. First, the contrast of the image was changed via the nonlinear curve with brightness values of the HSV space, in which gamma correction was applied on each of the red and green components of the image. Then, the exudates candidates were detected using histogram analysis. Finally, multi-channel histogram analysis was performed to eliminate the false positive candidates in the optic disc. This algorithm was tested over a dataset of 158 images achieving a sensitivity and specificity of 88.45% and 95.5%, respectively. Sopharak et al. (2008) preprocessed the image to enhance its contrast and eliminate the optic disc. Then, they carried out a set of experiments on feature selection and exudate pixel classification using a naïve Bayes classifier, which proved that the best feature set is a combination of six features: (1) the preprocessed pixel intensity, (2) the standard deviation of the preprocessed intensities in a window around the pixel, (3) the pixel's hue, (4) the number of edge pixels in a window around the pixel, (5) the ratio between the size of the pixel's intensity cluster and the optic disc and (6) the response of the pixel to Gaussian filter derivative. Their approach correctly classified the exudates in 93.38% of a dataset containing 39 images.
Again, to prevent confusion with exudates, Sreng et al. (2013) first detected and eliminated the optic disc through image binarization, ROI-based segmentation and morphological reconstruction. Then, the exudates were detected by applying the maximum entropy thresholding to filter out the bright pixels and finally, exudates were extracted via morphological reconstruction. This exudates extraction method was tested over 100 fundus images achieving a sensitivity of 91%. Eadgahi and Pourreza (2012) detected and eliminated the blood vessels using bottom-hat transformation and then extracted the bright components of high intensities using the top-hat transformation. After extracting the bright components, their method consequently distinguished the exudates by locating and eliminating the optic disc via its distinctive properties such as its high intensity and its high density of blood vessels. Their Method successfully extracted exudates with an average sensitivity of 78.28% over the DIARETDB1 dataset. Franklin and Rajan (2014) preprocessed the fundus images in the LAB color space using CLAHE to enhance its contrast. After eliminating the optic disc, the bright pixels were detected and classified as either "exudates" or "non-exudates" based on their high grey-level variations via employing an artificial neural network that utilized features such as color, size, shape, edge strength and texture. Their algorithm was tested over DIARETDB1 achieving an average sensitivity and specificity of 96.3% and 99.8%, respectively. Singh et al. (2015) proposed two independent methods to detect exudates after eliminating the optic disc and then strategically combined both methods to reduce all possible false positives. The first method transformed the OD-free image into the LAB color space, in which the contrast of the luminance channel was adjusted and then thresholded to detect exudates. Whilst, the second method morphologically processed the green channel of the OD-free image by dilating it twice via two different structuring elements and consequently subtracted the two dilated images to detect exudates edges. Finally, both methods were combined using some morphological and logical operators to remove noisy false positives. Their algorithm achieved an average sensitivity of 94.87% over a dataset made up of 20 diabetic fundus images. Roychowdhury et al. (2014) measured the severity grade of diabetic retinopathy using a machine learning system that classified the bright and red lesions. Their algorithm went through three stages; initially the optic disc and blood vessels were detected and then excluded from further processing. Secondly, the bright lesions were detected and classified into exudates or cotton wool spots and the red lesions were classified into microaneurysms or hemorrhages. At the end the severity grade was estimated per the severity scale specified by MESSIDOR project. Among several classifiers, the Gaussian Mixture Model (GMM) classifier proved to be the best in classifying the bright lesions using a set of 30 features, achieving a sensitivity and specificity of 89% and 85%, respectively, over the DIARETDB1 dataset.
To detect the exudates, Rajan et al. (2016) utilized the orientation scores of the retinal image to form an orientation-enhanced image, from which a binary mask of exudates was obtained by thresholding. First, the 2D orientation-score framework mapped the position and orientation angle of each pixel to a complex scalar, where the information of the edges and oriented structures were given by the imaginary and real parts, respectively. Afterwards, an enhanced image was formed using the real and imaginary parts of the complex scores. Finally, the resultant intensity image was thresholded to obtain the exudates, achieving a sensitivity and specificity of 86.2% and 85%, respectively, over the DIARETDB1 dataset. Prentasic and Loncaric (2016) detected the exudates using deep convolutional neural networks. First, the anatomical structures of the eye fundus were detected. Accordingly, the probability maps of blood vessels, optic disc, parabola fitting and values for the bright border detection were combined all together to get one probability map via weighted summing. Afterwards, the outputs of those probability maps were combined with the output of the deep neural network of exudates in order to reduce the false positive detection and hence improve the detection accuracy. This algorithm achieved a sensitivity of 78% using 50 fundus images from the DRiDB database. Liu et al. (2017) presented a location-to-segmentation strategy for segmenting exudates via three stages. First, the fundus image was preprocessed by removing the main vessels and the optic disc to facilitate the detection of the exudate patches at the further stages. In the second stage, the random forest classifier was learned to classify, identify and locate the exudate patches, in which the histograms of completed local binary patterns were extracted to describe the texture structures of the patches. At the final stage, the local variance, the size prior about the exudate regions and the local contrast prior were used to segment the exudate regions out from patches which were classified as exudate patches in the location stage. The algorithm was tested on DIARETDB1, achieving a sensitivity and specificity of 83% and 75%, respectively.

Materials and Methods
This section presents and describes the proposed system architecture. The block diagram shown in Fig. 2 illustrates the proposed architecture of segmenting the bright abnormalities, showing the materials that were exploited and the methods that were utilized in manipulating the eye fundus images.

Input Datasets: Fundus Images & Ground Truths
The fundus images are the raw material to be manipulated by a screening system. These fundus images are also accompanied with the ground truth to evaluate the results achieved by a segmentation algorithm using the true results provided by ophthalmological experts.
Thus, in order to diagnose diabetic retinopathy and detect its abnormal clinical signs, our proposed system utilized a total of 1419 images retrieved from three datasets: DIARETDB0, DIARETDB1 and MESSIDOR, which are the only ones among publicly available datasets that provided the ground truths of abnormalities occurring in diabetic retinopathy, as shown in Table 1.

Bright Abnormalities Segmentation
The proposed detection system adopted a two-stage approach for segmenting the bright abnormalities. First, the fundus image was partitioned into clusters based on its chromaticity information extracted from the L*a*b* color space. At the second stage, the pixels within the brightest cluster were thresholded to remove the relatively non-bright pixels and preserve only the brightest ones within the cluster.
Besides, the circular yellowish optic disc is usually clustered together with the other bright abnormalities found in the fundus image. Therefore, the optic disc was removed from the fundus image to improve the detection of such bright abnormalities and thereby avoid false responses so that the bright lesions would not be misclassified or confused with the optic disc.

Convert RGB Image into L*A*B* Color Space
As mentioned before, hard and soft exudates are mainly characterized by their yellowish color. But on the other hand, such abnormalities have varying sizes (i.e., tiny-tomedium), irregular shapes and no specific locations within the fundus. Therefore, the proposed detection method relied mainly on the color properties and nothing more else, to detect and segment such lesions.
Compared to the RGB color space, the L*a*b* space can quantify and decouple the chromaticity of the fundus image, which is represented by the (a*) and (b*) layers. Accordingly, the RGB fundus image was initially transformed into the L*a*b* color space to facilitate the detection of the hard exudates and the cotton wool spots based on their yellowish color.

Perform Color-Based K-Means Clustering
After extracting the (a*) and (b*) layers that represent the chromaticity (color) values of the fundus image, the pixels in the image were partitioned and classified into five disjoint clusters (k = 5) via the unsupervised k-means clustering algorithm according to their color, such that the colors of the objects within each cluster are as close as possible to each other and as far as possible from colors within other clusters, as illustrated in Fig. 3.
Each cluster is defined by its centroid as well as its member objects. The centroid for each cluster is the value to which the sum of distances from all the objects in that cluster is minimized. Once the grouping is done, the process is repeated several times to calculate the new centroid of each cluster and subsequently reassign the member objects to the cluster having the closest distance to the new centroid.
Thus, this color-based clustering algorithm using the k-means method worked as follows: Step 1: Create five empty cluster images (i.e., k = 5) to which each image pixel will be furtherly assigned (step3) and randomly specify the centroid value (c k ) of each of the five clusters, using the chromaticity values of the image pixels p(x,y) Step 2: Based on the chromaticity value of each pixel p(x,y) in the fundus image, the algorithm measures the cosine distance (d) between the centroid value (c k ) and the chromaticity value of each pixel p(x,y), as follows: Step 3: Assign and group each pixel with the cluster having the closest centroid according to the distance (d) computed by Equation (1). Thus, the centroid of each cluster is the value to which the sum of distances from all the pixels in that cluster is minimized Step 4: For each of the five clusters, calculate the arithmetic mean of the new cluster centroid (c k ) to re-cluster the pixels based on the new centroid. The new cluster centroid is calculated using the equation given below: where, n and p(x,y) are the number and values of the member objects, respectively, within each cluster Step 5: Replicate steps (2 to 4) to avoid local minima clustering (i.e., avoid grouping pixels all together within the same or a few of the clusters) Step 6: Reshape the cluster pixels into the image, such that the colors of the pixels within each cluster are as close as possible to each other and as far as possible from colors within other clusters ( Fig. 3 and 4) The number of clusters (k = 5) was empirically proven to be the ideal value of clustering the pixels of the fundus image, as the pixels of all yellowish objects were "best grouped" together and, at the same time, "best isolated" from the pixels of all other objects. Experimentally, it was found that at (k = 2,3 or 4), the bright yellowish abnormalities were grouped together in the same cluster with some other pixels that are not yellowish enough, such as the bright orange pixels of the retina (i.e., less number of clusters leads to poor isolation). On the other hand, using more than five clusters (e.g., k = 6) resulted in having the yellowish pixels separated and dispersed over more than one cluster image (i.e., more number of clusters leads to poor grouping).

Select the Best Cluster Image
Among the five clusters that were created via the k-means algorithm at the preceding step, it was important to automatically find and retrieve the best cluster, the one having the highest average intensity of clustered pixels. The notion behind considering such cluster, is that the exudates as well as the optic disc usually fall within the same cluster due to their similar bright color and which in turn, leads to having the highest mean intensity among other clusters due to their bright color. Therefore, the cluster with the highest mean intensity was selected to be furtherly processed and manipulated, as shown in Fig. 4.

Estimate a Threshold Based on Statistical Metrics
After selecting the best cluster, the proposed detection system automatically computed a threshold value (θ t ) that was estimated via order-statistic properties of the selected cluster intensity image (I c ). This threshold value was computed as the difference between the highest non-zero values of I c and the median of the non-zero values of I c . Thus, the threshold value was estimated via statistic-based metrics, using the following equation: where, I is the foreground (non-zero) pixels of the cluster intensity image I c . The notion behind using the "median" metric is because of its appropriateness for skewed distributions, as well as its resistance to outliers (i.e., extreme/noisy values) which may be found in an image; which makes it slightly better than other statistical measures such as the mean or minimum metrics.
Consequently, the threshold (θ t ) was applied to the intensity cluster image to remove the relatively nonbright pixels and preserve only the bright ones within the cluster, as illustrated in Fig. 6.

Remove the Optic Disc and its Connected Pixels
Eliminating the optic disc would reduce the false positive responses of exudates. Moreover, the pixels connected to the optic disc are most likely part of the segmented optic disc region. Accordingly, the optic disc and the pixels connected to it were removed from the thresholded image, leaving out only the additional clustered pixels, as illustrated in Fig. 7(e). The optic disc shown in Fig. 7 (b) was localized and segmented using the method proposed by Allam et al. (2016).
Finally, the last step was inpainting the internal skeleton of those remaining pixels, after removing the optic disc and the objects connected to it, as exemplified in Fig. 7 (f):

Evaluation Metrics of Segmentation
In order to evaluate the proposed algorithm of segmenting the bright abnormalities, the produced segmentations are compared against the ground truths provided by the employed datasets. The evaluation of the segmentation algorithm was measured via two metrics: the "sensitivity" which is the rate of correctly detecting the presence of bright abnormalities, as well as the "specificity" which is the rate of correctly confirming the absence of such abnormalities within the exploited fundus images, given by the following equations:

Experimental Environment and Results
The proposed algorithm was implemented via MATLAB platform using the "Image Processing Toolbox" and the "Statistics Toolbox". The proposed system was tested in a unique environment composed of three miscellaneous datasets of a total of 1419 images having diverse properties (e.g., spatial size, image quality, FOV, format, etc.). The notion behind employing a huge set of heterogeneous images was to examine the effectiveness and robustness of the proposed algorithm over any image.
The proposed system correctly detected the bright abnormalities, such as hard exudates and cotton wool spots, achieving an average sensitivity and specificity of 85.08% and 56.77%, respectively. Table 2 aggregates and compares the results achieved by our proposed segmentation approach over the three different datasets, DIARETDB0, DIARETDB1 and MESSIDOR, mainly in terms of sensitivity and specificity. Figure 8 shows samples of the segmentations obtained by the proposed algorithm over the employed datasets: DIARETDB0, DIARETDB1 and MESSIDOR.  Table 3 shows a comparison of the proposed technique against the other related work presented for segmenting the bright abnormalities.

Results Discussion
It is practically difficult to conduct a thorough and concrete comparison of the proposed method against others due to the big differences in the experimental environment used by each. On the contrary of all previous studies, the proposed method was assessed in a unique and strict environment composed of three diverse datasets of a total of 1419 images. However, this challenging experimental environment, in turn, led sometimes to a slight deterioration in our results compared to some of the other presented methods. Table 3, the DIARETDB1 dataset was utilized by many of the previous studies for assessing their algorithms. By comparing the results of the presented methods over this dataset particularly, it is deduced that the result of the proposed method was comparable to those achieved by others. Although the proposed approach outperformed some of the presented methods, yet it was not the best among all the other ones. The reason behind that is because of the fullautomation of the proposed approach without any human interaction, either before, during, or after the segmentation process. Also, the proposed algorithm was indifferent against the extreme heterogeneity and huge size of the employed datasets, which means that it would have achieved better results if it was customized over only one specific dataset.

Conclusion
The proposed segmentation technique followed a strategy that relied mainly on the color properties and nothing more else, to detect the bright abnormalities, such as hard exudates and cotton wool spots. The presented method adopted a two-stage approach for segmenting the bright abnormalities occurring in pathological fundus images. Thus, the overall process of detecting and segmenting those abnormalities was affected by those two stages/factors.
The first stage partitioned the fundus image into several clusters, each of which contained the colors that were as close as possible to each other and as far as possible from colors within the other clusters. Subsequently, the cluster having the brightest objects was selected. Therefore, a wrongfully selected cluster leads to either a false negative response (i.e., deselecting the cluster of abnormal pixels), or a false positive response (i.e., misclassifying the normal pixels as abnormal).
Similarly, the second stage thresholded the retrieved cluster to remove the relatively non-bright pixels and preserve only the bright ones within the cluster. Therefore, an inaccurately estimated threshold may also lead to either a false negative response or a false positive response.
However, the segmentation algorithm proved its effectiveness and robustness by achieving an average sensitivity of 85.08% which was comparable to the results achieved by other approaches. But more importantly, the segmentation algorithm was fully automated, simple and straightforward regardless of the heterogeneity of the employed datasets; as no training images were exploited, nor predefined templates were used and no image-dependent parameters were tuned, for the sake of customizing the proposed algorithm over certain images.

Future Work
The system proposed for segmenting the "bright/yellowish abnormalities" and distinguishing it from the "optic disc" can be furtherly applied likewise on the akin pair of the "dark/red abnormalities" and "blood vessels". This inspiration has been deduced during the process of partitioning the fundus image into several color-based clusters within the presented algorithm. It was observed that the optic disc and the bright lesions were usually grouped within the same cluster and at the same time the blood vessels and red abnormalities were normally grouped together in another cluster, as noticed previously in the images of Fig. 3 (c) and 4 (e). So, this technique can be exploited and potentially achieve promising results for segmenting the "blood vessels", as well as detecting and distinguishing the "red lesions".