Detection of Rotten Fresh-Cut Cauliflowers Based on Machine Vision Technology and Watershed Segmentation Method

: In this study, machine vision technology was used to separate the samples and detect the rotting degrees of fresh-cut cauliflowers. First, the improved watershed algorithm was used for the segmentation of fresh-cut cauliflower samples and the extraction of single-sample. Then, three-color models, a gray co-occurrence matrix, and two feature extraction algorithms were used to extract the color, texture, and spectral feature parameters of the images. At the same time, the Partial Least Squares Discriminant Analysis (PLS-DA) and Extreme Learning Machines (ELM) discriminant models were established. The identification accuracy of PLS-DA and ELM discriminant models for rotting samples was 95 and 90.9%, respectively. Moreover, according to the size of rotten areas, the rotting grades were divided and the contours and feature areas of rotten cauliflower samples were identified by the region growth algorithm and the “Sobel” operator. Finally, the detection and identification of the rotting degree of cauliflower samples were realized. The results showed that machine vision technology can segment the cohesive fresh-cut cauliflower samples and can be used for qualitative and quantitative identification of the intact and rotten cauliflower


Introduction
Brassica vegetables are a healthy food that is rich in vitamin C, phenolic compounds, dietary fiber, glycosylates, and isothiocyanates . Cauliflower (Brassica oleracea L. botrytis) belongs to the genus Brassica in the Brassicaceae family and it is a common vegetable in the human diet (Vaishnav et al., 2015). Epidemiological studies showed that a diet rich in Brassica vegetables can reduce cancer rates (Giuffrida et al., 2018). Fresh-cut fruits and vegetables refer to the fresh fruits and vegetables that have been selected, cleaned, drained, trimmed, graded, packed, and sometimes even peeled and cut as needed (Ma et al., 2017). With the acceleration of people's lives and the improved food safety awareness, the detection of fresh-cut vegetables has important practice (Boumail et al., 2016). As consumers' preference for ready-to-use or ready-to-eat vegetables continues to increase, fresh-cut cauliflower is becoming more common in food services and retail markets than as a convenience product (Mashabela et al., 2019).
Computer vision technology can be used to check the fruits and vegetables in fresh-keeping packages. These fruits and vegetables are first separated, then classified, identified, and abandoned. During harvesting, transportation, and storage, brown spots and even diseases may appear on the cauliflower surface due to collision or natural causes. Manual identification may cause friction on the surface of cauliflowers and affect the identification accuracy of rotten cauliflowers. Computer vision technology uses a computer and an image acquisition system to simulate the information acquisition process of the human eyes and transmits the acquired image information to the computer terminals for detection and identification (Li et al., 2019a). Nowadays, it can replace the human eyes to acquire images for analysis and processing.
Image segmentation plays a vital role in image processing. When computer vision technology is applied, how correctly and completely segments images is the key to image processing. Traditional image segmentation methods include threshold-based segmentation (Xue et al., 2015), clustering (Shen et al., 2018), and the neural network method (Gao et al., 2019;Costa et al., 2021). The watershed transform has the advantages of small computational complexity, high accuracy, sensitivity to detail changes, continuous single-pixel segmentation lines, and enclosed segmentation regions (Ma et al., 2021).
The scabs adhered to the cotton leaves were segmented and detected . Combined with the H-minima function, the mathematical morphology was coupled to improve the watershed and segment the remote sensing images (He et al., 2020). Generally, the watershed algorithm is an image adhesion segmentation method with wide application and stable segmentation results. However, the traditional watershed algorithm would result in over-segmentation due to excessive local extremum (Li et al., 2019b). If the traditional watershed algorithm is improved for increasing segmentation efficiency, the over-segmentation reduction would be one of the keys to explore.
In addition, computer vision technology has also been widely applied in the classification, maturity, diseases, and pests of fruits and vegetables. For example, it was used to extract the shape, color, and texture characteristic parameters of barley sample images (Szczypiński et al., 2015). A variety identification model with an average identification rate of 80% was established to accurately identify the barley varieties. Combined with the Convolutional Neural Network (CNN), computer vision technology detected the intactness of wheat grains with an average recognition rate of 96.67% (Zhu et al., 2020). It was also used to identify and study the surface defects of carrots in HIV color space (Xie et al., 2020). The results showed that the shoulder, heads, bends, fractures, divarication, and surface cracks in carrot green could be detected and identified by computer vision technology and other image processing technologies. According to computer vision technology and deep learning, the shells, outer shells, and embryos of pecan fruits at different growth stages were monitored (Costa et al., 2021). Moreover, the Mask Region-CNN algorithm was used to segment and detect the targets and establish the growth curve, and prediction model. In this way, the average absolute error was controlled at 0.14-28.06%, and color ripe orange images were obtained under the lighting conditions of different fruit sizes. The adjusted red-blue image and tone images were used to identify these images with 83% accuracy (Zhao et al., 2016). Although computer vision technology can detect the qualities of fruits and vegetables and its stability should be improved. In addition, the more eigenvalue inputs required for testing, the longer the model takes to detect.
The fruit images captured from high-speed CMOS sensors and lightweight CNN architecture were used to realize the recognition of fresh/rotten apples, fresh/rotten oranges, and fresh/rotten bananas on different edge platforms (Ananthanarayana et al., 2020). Whereas, a single image can only be detected by a single sample and the detection efficiency needs to be improved. According to the deep learning and CNN model, a computer vision technology was designed to detect fruit freshness (Valentino et al., 2021). The classification of fruit freshness was realized, while the classification of grades was not. Based on the state-of-the-art deep learning techniques and stacking ensemble methods, an efficient and effective machine vision system was proposed to offer a non-destructive solution for visual inspection automation of fruit freshness and appearance (Ismail and Malik, 2021). Currently, there are few reports on the quality test of fresh-cut cauliflowers.
On this basis, computer vision technology was applied to segment and identify cauliflowers and detect the rotting degrees. First, the morphology technology incorporated watershed changes to segment adhesive images. Then, the segmented images with specific feature parameters were extracted and optimized and the rotting samples of fresh-cut cauliflowers were detected through different identification models. Finally, the region growing algorithm was used to extract the outline and feature regions of rotten cauliflower samples.

Experimental Materials
The test samples of cauliflowers were collected from the farmer's market in Taigu County, Shanxi Province, China, and shipped to the laboratory on the same day. The plastic gloves were worn throughout the process. After that, they were cut into small samples with blades and one stem was kept for each sample. These small processed samples were put in boxes and wrapped with a fresh-keeping membrane for storage. The sample images were collected in 5 days and they were edge-cut by watershed segmentation. Using the K-S algorithm, the samples were grouped into a calibration set and a prediction set in a proportion of 3:1. For the intact samples, 90 were used for the correction set and 30 for prediction. For the rotten samples, 72 were used for correction and 25 for prediction.

Image Processing System and Methods
The image acquisition system is composed of a darkroom, a camera, a ring light source, and a stage. The darkroom wrapping with black cloth could prevent interference from the outside. The brightness of the light source can be adjusted appropriately and the stage height was adjusted to 240 mm. The brown spots on the sample surfaces were focused for clearer image pixels.
First, the irrelevant edge information in the original images was removed by using the open-close-based reconstruction morphology technology. At the same time, the maximum and minimum values of the grayscale images were highlighted. Then, the local minimum was calculated to label the foreground image and the background image was obtained by threshold segmentation. The Euclidean matrix of the background image was calculated to obtain the watershed ridge graph. The background of the sample image was removed and the feature variables were extracted with Matlab R2016b. The Competitive Adaptive Reweighted Sampling (CARS) and Regression Coefficient (RC) methods were used to optimize the preferred eigenvalues. These values were taken as the outputs to establish the Partial Least Squares Discriminant Analysis (PLS-DA) and Extreme Learning Machines (ELM) identification models, respectively. Finally, the outline and feature region of the rotten cauliflower samples were identified by the region growing algorithm to identify and test the rotting degree. The flowchart of image processing is shown in Fig. 1.

Image Segmentation based on Morphology and Watershed Transform
The RGB image is converted into a gray image by RGB 2 grays. The gradient amplitude image of the original image is calculated as follows (Wang and Luo, 2021): Among them, f/y is the Sobel operator used to obtain the filtering modulus in the vertical direction of the image and f/y is the filtering modulus in the horizontal direction. The watershed of the original gradient amplitude image is calculated by the watered function, which is then filled into a color image.

Extraction and Optimization of Sample Color and Texture Features
After extracting RGB color features, it can be transformed into an HSI color model expressed by hue, saturation, and brightness (Guo, 2020). The conversion formula is as follows: When B ≤ G, H is 1-180. When B ≥ G, H = 180-H, the mean and variance of G, B, H, S, I, L*, a*, b* components are calculated by means and (std.) functions.
Texture features represent the important information on surface structure arrangement. Gray Level Co-occurrence Matrix (GLCM) is a common method to extract texture features. There are some differences in the surface tissue structure between intact and rotten cauliflowers, which can be used to identify their integrity of them. When the pixel value of the image is close, the energy value is high. Entropy is the level of chaos, which represents the linear correlation of image grayscale (Xue et al., 2015). The calculation formulas of texture eigenvalue energy (ASM), Entropy (ENT), second moment of inertia (CON), Correlation (COR), and Inverse Difference Moment (IDM) are as follows: where, p(i, j) is the pixel value of abscissa i and ordinate j.

Detection Model
ELM is a machine learning method based on the feed-forward neural network. It has higher learning efficiency and generalization ability than a support vector machine (Song et al., 2018). To save time, the number of hidden layer neurons is set as the number of input types and the activation function is the sigmoidal function. The calculation of the sigmoidal function is as follows: where tempH is the product of input weight and correction set input matrix. PLS-DA is a statistical method of stepwise multiple regression. It projects the predictive and observed variables into a new space to find a linear regression model. The PLS model is established by the unscramble x software and then the principal components were extracted for linear analysis to effectively predict the output results (Yin et al., 2021).

Image Segmentation by the Morphology Technology and Watershed Transformation Background Removal of Sample Images
During image acquisition, test-independent information can be collected from the background and errors would occur. At this time, the background needs to be removed to improve image quality. The grayscale images of samples are binarized and a proper threshold is selected by the gray thresh function. If the value is below the threshold, the pixel would be "0" and the pixel area will be darkened; on the contrary, the pixel would be "1" and the pixel area would be white. The boundary of sample images is extracted by using the mathematical morphology, i.e., the corrosion and expansion algorithms. To extract an intact outline boundary, the sample-related information is magnified with the expansion algorithm, and several pieces of irrelevant information are removed by the corrosion algorithm. The extracted boundary image is a result of the expanded image minus the corroded image. The expanded image might bring some irrelevant variables. Thereby these variables are removed by the "beware open" function and the filling is performed by the "imfill" function. For composition, the original image is operated by the "repmat" function and it had the same size as the background image. The last step is to combine the filled background image with the original image to obtain a background-removed image. Figure 2 is a process of removing the background of a small sample image. It can be seen that the watershed transform oversegments the original gradient amplitude and phase (Li et al., 2019b). This is mainly due to too many different minimum regions in the gradient amplitude of the original image and excessive reception basins are highlighted. Therefore, the watershed should be developed with several similar minimum regions to distinguish small samples along the limited reception basin edges (Ma et al., 2021).  Figure 4(D) is the foreground label overlaid on the original image. Figure 5 is the results of subdividing watershed ridge and modifying gradient amplitude image grouped based on skeleton influence. Figure 5(A) is a threshold graph of the reconstructed image based on an opening operation. Figure 5(B) finds the watershed ridgeline by setting the edge between reception basins to "0" and calculating the Euclid matrix of the threshold image. Figure 5(C) is a color-labeled watershed ridge image. Figure 5(D) is an image of the smallest region in the original gradient amplitude image, which refers to the watershed ridge image and foreground label image by using the imimposemin function. In this way, there are only local minima in the foreground and background regions and the background can be segmented and labeled.

Identification Model of Rotten Cauliflower Samples Based on Computer Vision Technology
Samples were segmented by the improved watershed algorithm. The color and texture features were extracted from these segmented single-sample images. A recognition model of rotten cauliflower samples was established using the extracted effective image information.

Extraction and Analysis of Image Eigenvalues Color Feature Extraction of Sample Images
Three RGB (Lalabadi et al., 2020), HSI (Abbasgholipour et al., 2011), and L*a*b* (Ayustaningwarno et al., 2021) models almost cover all colors and are widely used. This study proposes the mean and standard deviation of R, G, B, H, S, I, L*, a*, and b* color components to determine image colors. Figure 7 is the image of the three color models.
It can be seen from Larger differences among R, G, B, H, a*, and b* components are more likely to be a high proportion of rotten regions in the samples. Thereby, the three primary colors, the tone, and the two high-brightness colors are very different. L*, S and I components show close results. The reason is that the ambient brightness remains constant during the image acquisition.

Extraction of Texture Eigenvalues
Texture features represent the key information of the texture arrangement and surface structure of the subject. GLCM is a common method to extract texture features. Since the intact and rotten cauliflowers have different surface structures, the texture feature information is used to distinguish the integrity of cauliflowers. The closer the image pixel values are, the higher the energy value. The entropy represents the degree of chaos. Relevance represents the linearity of the grayscale image (Khojastehnazhand and Ramezani, 2020;Ansari et al., 2021). The texture feature of cauliflower energy, entropy, inertia moment, relevance, and inverse difference moment extracted at 0°, 45°, 90°, and 135°, respectively, is as shown in Table2.
It can be seen from Table 2 that rotten cauliflower samples have the average moment of inertia and the relevant features at 0°, 45°, 90°, and 135° are close to those of intact samples. The average of energy, entropy, and inverse difference moment at 0°, 45°, 90°, and 135° are different. The reason may be because the extracted texture features are mostly from the rotten parts, or the irrelevant information in the background of the original sample image is cropped during image preprocessing. As the image size changes, the surface texture of samples also changes. The pixel size will change during sharpening, which is one of the reasons why the texture information changes.

Extraction of Characteristic Parameters
The analysis shows that there is little difference between intact and rotten cauliflower samples for H, S, and I components. The identification accuracy of rotten cauliflowers may be affected by the mean and variance of components R, G, B, H, S, I, L*, a*, and b*. The energy, entropy, the moment of inertia, relevance, and inverse moment at 0°, 45°, 90°, and 135° are used to distinguish the intact and rotten cauliflowers. Thereby, the RC algorithm and the CARS algorithm are used to optimize the characteristic parameters and the input analysis results of the three models are compared.

Characteristic Parameters Optimized by the RC Algorithm
The RC algorithm (Huang et al., 2021) simplifies the original variables into multiple variables and the new feature variables contain effective information from the original spectral data, resulting in better prediction results than all-band variables. The mean and variance of components R, G and B, H, S and I and L*, a* and b*, as well as the energy, entropy, moment of inertia, relevance, inverse difference moment in the four directions (0°, 45°, 90°, and 135°) are denoted by 1-38. The PLSR model is used to select the partial extremum of the regression coefficient. The feature parameters extracted are shown in Fig. 8. The feature parameters for 5, 6, 7,13, 15, 16, 18, 23, 24, 25, 28, 29, and 30, i.e., the variance of components G and B, the means of H, L*, and b*, the variance of L* and b*, the entropy at 0°, 45° and 90° and the moment of inertia at 45°, 90° and 135° are selected.

Characteristic Parameters Optimized by the CARS Algorithm
The CARS algorithm selects a Partial Least Square (PLS) model for a smaller Root Mean Square Error of Cross-Validation (RMSECV) to obtain the extremum regression coefficient (Song et al., 2020). Figure 9 is the flowchart of feature parameters optimized by CARS. From Fig. 9, the total sampling is set to 50. Figure 9(A) shows the decrease of the image parameters at the beginning of 38 as the sampling progresses. Figure 9(B) shows that when the sample parameters are 18, the minimum value of RMSECV is 0.2776. Figure 9(C) reflects the relationship between sampling and regression coefficients. 14 feature parameters are retained when the sampling is 18. They are 7, 9, 14, 17, 18, 21, 22, 23, 25, 29, 32, 33, 35 and 36. The corresponding image parameters are the mean of H, I, and a*, the variance of a* and b*, the energy at 90° and 135°, the entropy at 0° and 90°, the moment of inertia at 90°, the relevance at 45° and 90° and the inverse moment at 0° and 45°.
The Partial Least Squares Regression (PLSR) prediction model is established using the feature parameters and original parameters optimized by CARS and RC algorithms, respectively. The prediction results are shown in Table 3.
It can be seen from Tab. 3 that the image parameters optimized by the RC and the coefficient of determination (R²c) of the prediction set in the PLSR model are smaller than that of the prediction set of the original parameters. This may be because the local extrema of the regression coefficient related to how to distinguish whether the cauliflowers are rotted or not are fully selected.

Establishment of the Discriminant Model The ELM Discriminant Model
The intact and rotten cauliflowers take labels 1 and 2 as the input parameters, respectively. The maximum deviation is set to 0.5. The feature parameters of 162 samples optimized by CARS are used as the correction set of ELM (Song et al., 2018) for regression operation. The neurons in the ELM hidden layer are set to 38. The label values of 55 samples are predicted by the Sigmoidal activation function and the distinguishing results are shown in Table 4.
The identification results show that there are 3 misjudgments for the intact cauliflower samples with an accuracy rate of 90% and 2 misjudgments for the rotten cauliflower samples with an accuracy rate of 92%. The overall accuracy rate is 90.9%. In the cauliflower samples, the predicted inputs for the three misjudgments in cauliflower samples are close to 1.5. Friction or corrosion on the surface of the cauliflower can cause browning and the rotten samples are misjudged. If a rotten sample is misjudged as intact, it may be the result of occasional errors in image processing.

The PLS-DA Discriminant Model
PLS-DA is a statistical method for stepwise multiple regression and it extracts the principal components for linear analysis and effectively predicts the results (Ansari et al., 2021). The absolute coefficient (R²c) of the calibration set model for the intact and rotten samples is 0.7576 and the Root Means Square Error of Calibration (RMSEC) is 0.2446, indicating that the PLS-DA model could effectively predict the variety of samples. The identification results are shown in Table 5.
It can be seen from the identification results that there is one misjudgment for the intact cauliflower samples and the accuracy is 97%. There are 2 misjudgments for the rotten cauliflower samples and the accuracy rate is 92%. The overall accuracy rate is 95%. The identification accuracy of the linear PLS-DA model is higher than that of the non-linear ELM prediction model. The reason for this misjudgment may also be the insufficient modeling sample numbers and limited prediction range.

Identification of Rotting Degree of Cauliflowers
Computer vision technology was used to identify the degree of rotten cauliflower samples. There were three levels based on the size of the rotting areas of the cauliflower samples. Level 0 indicates no rotting. Level 1 indicates the rotting area is 0-25% of the total. Level 2 indicates the rotting area is 25%-50% of the total (Lin et al., 2017;Zhang et al., 2016).    The intactness of the two groups of cauliflowers is first identified by computer vision technology. The edge and feature areas of rotten cauliflower samples are identified by the "Sobel" operator and the region growing algorithm (Xue et al., 2015). Figure 10 is the identification image of rotting samples at different levels. Figure 10 (2, 4, 6, 8, 10, and 12) could better identify the size of the rotting areas of the samples. There are 120 rotten cauliflower samples with Level 0, 64 rotten samples with Level 1, and 33 rotten samples with Level 2. Combined with the region-growing algorithm, the Sobel operator can accurately identify the size, location, and shape of the feature areas.

Conclusion
An improved watershed segmentation algorithm is used to realize the segmentation of interconnected cauliflowers. On this basis, the PLS-DA model is established by the three-color models, GLCM, and two feature extraction algorithms (RC and CARS), and the optimal discriminant model is determined. Finally, the "Sobel" operator and region growing algorithm is used to detect the size and location of the rotting areas of cauliflower samples. This study theoretically supports and establishes the isolation of adherent clean cauliflowers and the identification of rotten cauliflowers.
However, computer vision technology can only classify and detect the samples in the middle and late stages of infection. In future work, machine vision technology and spectral detection technology will be used to realize the early detection and classification of cauliflower samples in the early stage of infection. The color identification of rotting areas of cauliflower requires further research to determine the type of disease infection so that the nondestructive and rapid identification of rotting cauliflowers can be improved.