Multilevel Image Thresholding Using the Complement Feature

Email: salahameer@alumni.uwaterloo.ca Abstract: A novel Eigen formulation is proposed for image segmentation. Each pixel is represented by a unit vector having the x-component as the normalized gray value of the pixel. The axes of inertia are simply the Eigen vectors of the auto-correlation matrix. The largest Eigen vector is used as the point of split. Each sub-image is further split in the same way. The ratio of the smallest Eigen value to the sum of Eigen values represents the percentage of the minority and was used to control further splitting. The process continues until no sub-image is larger in size than this ratio. The results are very encouraging on a wide range of images.


Introduction
Image segmentation plays a vital role in almost all image processing and computer vision tasks. The ultimate goal is to delineate the image in such a way to obtain useful descriptions of the objects comprising the scene. To achieve this goal, many algorithms have been (and still being) developed (Goh et al., 2018). Details regarding categorization of these algorithms and the feature space used can be found in many traditional survey papers such as (Sezgin and Sankur, 2004). In fact, the field is so vast and diverse that there are survey papers on a single subcategory e.g., (Oliva et al., 2019;Lucchese and Mitra, 2001;Dey et al., 2010;Ilea and Whelan, 2011;Peng et al., 2013;Unnikrishnan and Hebert, 2005).
In this study, the image segmentation problem is considered as a multilevel thresholding task. A simple but effective Eigen structure is proposed as a solution.

Method
This research is an extension to the work of (Ameer, 2020) on image thresholding using the complement feature. Without loss of generality, the original image is normalized to the interval [0,1] and concatenated to produce a column vector of size N1, N is the number of pixels in the image. For the current work, each pixel is represented by one feature, namely intensity gi. Pixel representation is then extended to a 2D unit vector having the intensity value as the x-component. In other words, each element is now represented by the vector: (1) G is now an N2 array. An auto correlation matrix (AG) of size 22 are then constructed from G as: Solving the Eigen formula: The Eigen vectors of AG represent the axes of inertia for the data set. The largest vector Vmax (the one corresponding to the maximum Eigen value λmax) points toward the direction of maximum inertia. Since the ycomponent is not an independent component, the xcomponent represents a point of concentration of the original data, close to the mean. The smallest Eigen vector Vmin (the one corresponding to the minimum Eigen value λmin), on the other hand, is normal to Vmax and hence, is not guaranteed to have its x-component within the original data range.
The hypothesis adopted in this study can be stated in an abstract form as: The largest eigen value λmax should be associated with the major process in the data, while the smallest eigen value λmin should be associated with the minor process. However, no claim is made as whether the major process is the foreground or the background. In fact, it's a mixture of both as will be demonstrated shortly.
This motivates the author to use the x-component of Vmax as a threshold. Indeed, the image data is far from being uniform, hence, the previous splitting (thresholding) procedure is successively applied to the newly generated sub-images. In fact, using the mean instead of the x-component can be used with minor degradation in performance (mean square error).
Another feature of AG worth noting is that its Eigen values, more specifically their ratio, gives a good indication of the percentage of minorities and majorities. Minorities can relate to noise or small objects in reference to the dominant or major object in the image. For this purpose, let's define the Eigen ratio: The splitting procedure, described earlier, will continue as long as the relative size (compared to the original image) is larger than rλ, the Eigen ratio. For each resulting sub-image, Equation (3) is applied to extract the x-component of Vmax.
After completing the splitting procedure, a merging stage can be added to combine adjacent sub-images that are small enough. The merging process is left for a future work. Each resulting region can then be replaced by the x-component of Vmax or just the mean.
The full procedure is presented in sequence in Table 1. Hence, four schemes will be tested:  where, x and y stand for original and segmented images and ||x|| is the cardinality of the set. Adjustment should be placed when the range of images are different. In addition, it is unfair to compare performance between images having different number of segments.
Another evaluation scheme is the SSIM given by, (Wang et al., 2004): where, µ is the mean, σ 2 is the variance, c1 = 0.0001, c2 = 0.0009 and σxy is the covariance between x and y.

Experimental Results
The proposed algorithms are implemented on various images shown in Fig. 1.
Results for scheme PosVec, NegVec, PosMean and NegMean are demonstrated in Fig. 2 to 5 respectively. Table 2 to 3 lists the values of Equation (5) and (6) for the results obtained in these schemes. Append to create a unit vector. Equation (1)  4 Find autocorrelation matrix. Equation (2)  5 Find Eigen values and Eigen vectors of autocorrelation matrix. Equation (3)  6 Find rλ. Equation (4)  7 Split image data at the x-component of Vmax. 8 Repeat steps 4-7 for each sub-image occupying more than rλ of the original image. 9 Each resulting sub-image will be represented by a uniform intensity. 10 Small sub-images can be merged.   To further illustrate the effectiveness of the proposed schemes, some images from Berkley Segmentation Database (BSD) (Martin et al., 2001) are segmented in Fig. 6 and 7. As the values of RMSE and SSIM are not good discriminators for high quality segmentation, they will not be reported for these images. It is easily noticed from these images that using the normalization [-1,1] produces less number of regions compared to [0,1].  38092(15, 5, 15, 5), 42049(20, 9, 20, 8), 45077(54, 14, 54, 14), 89072(18, 5, 18, 5) and 101087 (7,5,8,4) However, the latter has better performance in preserving majority of the details without diminishing small regions that are of visual importance to the viewer. Despite the fact that mean can be used, the x-component of Vmax can be of better computational benefit as the Eigen values are already computed.

Conclusion and Future Work
A simple Eigen structure is proposed in this study to perform multi-level image segmentation using one feature, intensity. The algorithm is fully automatic and no adjustment to any sort of parameters is needed.
The proposed schemes are very effective as demonstrated by the values of RMSE and SSIM. However, better evaluation schemes are needed to better compare the performances.
Work is currently in progress to extend the algorithm to segment colored images. Other feature spaces, specifically ones incorporating local information, need to be investigated to improve segmentation quality.

Ethics
This article is original and contains unpublished material. The corresponding author confirms that all of the other authors have read and approved the manuscript and no ethical issues involved.