Edge Sensing Demosaicing Using Adaptive Weighted Interpolation

Demosaicing is a process of obtaining a full color image by interpolating the missing colors of an image captured from a digital still and video cameras that use a single-sensor array.The edges are not handled properly in the existing demosaicking methods. This study provides an effective demosaicing algorithm applying an adaptive weighted-edge interpolation to handle green pixels. Due to the introduction of adaptive threshold based on the high frequency information for the classification of edges and textures, the interpolation of edges is performed excellently. Adaptive weights depending on spatial and spectral correlation are assigned for interpolation on both edges and smooth region. A refinement scheme based on enhanced effective color interpolation is used to improve the red and blue interpolation to obtain better quality image. As compared with the latest demosaicing algorithms, the proposed algorithm produces the best average demosaicing performance both objectively and subjectively. This approach handles the edges in a proper way and reconstructs the color image with best peak signal to noise ratio


INTRODUCTION
In order to reduce the hardware cost, many digital still cameras (DSC) use a single sensor equipped with a color filter array (CFA) to capture any one of the three primary color components, R (Red), G (Green), or B (Blue), on each pixel location. Among the various suggested CFAs, the Bayer CFA pattern (Bayer, 1976) is the most prevalent one, where G pixels occupy half of all and R and B pixels share the others. A representation of a full-color image needs all the information from the three colors on each pixel location. As a result, the missing two colors on each pixel location should be interpolated back to get a fullcolor image. The process of interpolating the missing colors is called as demosaicing or color interpolation whose main objective aims to reconstruct the missing colors as closely to the original ones as possible while keeping the computational complexity as low as possible.
It is identified that most of the existing techniques (Menon et al., 2007;Chung and Chan, 2006;Su, 2006;Gunturk et al., 2002;Li, 2005;Zen et al., 1998) have not considered the level of smoothness of smoother region during interpolation. Poor quality on smoother region in demosaiced image has resulted due to this problem. Addressing the issue of improving the quality on smoother region was another focus of this research study. A weighted technique to address this problem is proposed on Demosaicing using Adaptive weighted interpolation to improve the quality on smoother region.
Demosaicing using Adaptive weighted interpolation (Hore and Ziou, 2011), in which interpolation for each missing green component in the CFA, the algorithm Science Publications AJAS performs a gradient test, to identify edge direction and then carries out an interpolation along the direction of a smaller gradient to determine the missing green component. For smooth region, the variance of color differences in vertical, horizontal and diagonal directions can be used as a supplementary criterion to determine the interpolation direction for the green components. Other color planes are estimated based on the interpolated value of the green plane.
Due to the introduction of adaptive threshold based on the high frequency information for the classification of edges and textures, the interpolation of edges is performed excellently. Adaptive weights depending on spatial and spectral correlation are assigned for interpolation on both edges as well as smooth region. A refinement scheme based on enhanced effective color interpolation is used to improve the red and blue interpolation to make better quality image. As compared with the latest demosaicing algorithms, the proposed algorithm produces the best average demosaicing performance objectively and subjectively.
The study describes how to demosaic an image captured through a single sensor array camera and simultaneously produce a better image at various edge levels.

Edge Sensing Demosaicking Using Adaptive Weighted Interpolation
The directional gradients for 5×5 windows are computed first, sharp and smooth regions are effectively classified using an adaptive technique that uses spatial and spectral correlation of respective local window. To interpolate the green component at each red and blue pixel positions on sharp region, adaptive weights are derived using the local window characteristics. On smooth region, variances of color differences are computed horizontally, vertically and block-wise to find the suitable interpolation direction in-order to give best performance. The green component on smooth region is also estimated by applying the weighting technique. Once the green plane is interpolated, then red and blue components can be interpolated accurately using the green component available or interpolated at the corresponding position with the color difference plane.
A refinement step is used finally to generate more consistent values for red and blue components as close as to original, which is based on exploiting the image spectral correlation on either green and red or green and blue color planes.

Computation of Directional Gradients
To classify the sharp and smooth regions on the mosaic image and to interpolate green component, the directional gradients L H (horizontal) and L V (vertical) are computed for each red pixel position. There is a variety of classifiers used in demosaicing algorithms. The simplest first order spatial classifier is the neighborhood difference in pixel values of green plane as |Gi,j-1-Gi,j+1|. The most commonly used classifier in the demosaicing algorithms is: |Gi,j-1 -Gi,j+1| + |2Ri,j -Ri,j -2 -Ri,j+2| which uses both spatial and spectral correlations. These classifiers are not good enough to classify a natural image into sharp and smooth regions.
To compute the directional gradients L H and L V at blue pixel position, Equation 1 and 2 should be modified by replacing R by B and B by R:

Interpolation of Green Plane for Sharp Edge Blocks
The leading edge direction is determined by computing the ratio of the above two directional gradients and as in Equation 3:

AJAS
To classify the regions, an edge threshold is derived based on the edge details of mosaic image. The edge threshold (ethresh) is calculated as the ratio of number of 5×5 blocks to number of 5×5 blocks for which e ∊ {1, 2} in the image. If the e (edge value) of a 5×5 block is greater than the ethresh (edge threshold i.e., (e >ethresh)), then the block is defined as a sharp edge block. If horizontal gradient value and vertical gradient value of a block are same, then the block is defined as perfectly smooth block and if e < ethresh for a block, then that block is classified as smooth block.

Determination of Weights for Sharp Edge Blocks
Weights for sharp edge blocks (W sharp ) are determined by local neighborhood as a ratio of color differences. For a reference neighborhood of 3×3 Bayer mosaic image with red pixel at center as shown in Fig. 1. weights are estimated using Equation 4: For a reference neighborhood of 3×3 Bayer mosaic image with blue pixel at center as shown in Fig. 2, weights are estimated using Equation 5:

Interpolation of Green Plane for Sharp Edge Blocks
For sharp edge blocks, green components at red pixel positions are estimated exploiting the neighborhood correlation by weight of sharp transition (W sharp ) as in Equation 6: Similarly green components at blue pixel positions for sharp edge blocks are estimated as in Equation 7:

Determination of Variances of Color Differences for Smooth Edge Blocks
The variances of the color differences for smooth edge blocks are computed at each red and blue pixel positions using Equation 8

Interpolation of Green Plane for Smooth Edge Blocks
The missing green value can be estimated for smooth region by using an experimentally found weight (W smooth ) constant throughout the smooth regions as in Equation 8-10. The weight value (W smooth = 0.87) is used in this method, which has given the performance close to optimal.
If 2

Interpolation of Red and Blue Components at Green Pixel Positions
The missing red and blue values at green CFA sampling positions are estimated using the available and interpolated green plane along with the information on the plane itself as Equation 11: where, C is R(red) or B(blue) color plane component and is the interpolated green plane component.

Interpolation of Red Components at Blue Pixel Position and Blue Components at Red Pixel Position
The missing red components at blue pixel positions and blue components at red pixel positions are estimated as in Equation 12: ( ) i m, j n i m, j n m 1 n 1 i, j i, j C G C G 4 where, C is R (red) or B (blue) color plane component and is the interpolated green plane component.

Refinement on Red and Blue Interpolation
To reduce the error on red and blue estimated values, a refinement scheme is used based on spectral and spatial correlation. This refinement step is focused on generating more consistent and close to optimal values for red and blue pixels using the values interpolated in initial step.
The weighted color difference value is calculated exploiting the image spectral correlation on two color planes around a pixel under consideration. The weights along the four adjacent directions for the reference The color difference value at the center location is estimated as in Equation 15 by assigning the weights of four adjacent color difference values: Similarly the color difference value is computed. Using the weighted color difference values, the red and blue pixel values are refined as in Equation 16: Thus the weighted color difference value has generated more consistent and optimal demosaicing values.
Most of the demosaicing techniques use a hard threshold to classify the regions and hard interpolation solely along the edge direction that results poor interpolation on edges. Also a hard decision is used for the interpolation of smooth region that results poor image quality on smooth regions. These problems are considered while deriving this Adaptive Demosaicing.
In this method, to identify the edge details an adaptive threshold is proposed. Due to the introduction of adaptive threshold based on the high frequency information for the classification of edges and textures, the interpolation of edges can be performed superbly.
Adaptive weights depending on spatial and spectral correlation are assigned for interpolation on both edges and smooth region. A refinement scheme based on enhanced effective color interpolation is used to improve the red and blue interpolation to make better quality image. Due to the introduction of adaptive threshold and weights, this demosaicing technique can achieve high performance.

RESULTS
Simulation was carried out to evaluate the performance of the proposed algorithm.
The 24 digital color images shown in Fig. 4 were used to generate a set of testing images. The Color Peak Signal-to-Noise Ratio (CPSNR) and Mean Square Error (MSE) were used as measures to quantify the performance of the demosaicing methods. Table 1-3 and Fig. 5 shows the performance of various algorithms like Bilinear Interpolation (BI) (Sakamoto et al., 1998), Effective Color Interpolation (ECI) (Pei and Tam, 2003), Enhanced ECI (EECI) (Chang and Tam, 2004) and Demosaicing using Variance of Color Difference (DVCD) (Chung and Chan, 2006). The proposed algorithm provided the best performance among the evaluated algorithm.   Different results prove the effectiveness of the study developed. We note that the proposed algorithm is developed based on the fact that the interpolation direction for each missing green sample is critical to the final demosaicing result.
Here due to the introduction of the adaptive weights based on edge levels this demosaicking achieves better performance.

CONCLUSION
In Edge Sensing Demosaicing using Adaptive weighted interpolation, an adaptive threshold exploiting the correlation among neighborhood pixels is employed for efficient classification of sharp edges and smooth region. To preserve the high frequency information, an adaptive weighted interpolation scheme is proposed. Also, to improve the performance of demosaicing technique on smooth region, an adaptive weighted interpolation scheme is employed. Simulation results show that this proposed algorithm is able to produce a Science Publications AJAS subjectively and objectively better demosaicing results as compared with a number of advanced algorithms widely used in the industry.
There are good scopes for future improvements in the proposed demosaicing techniques. For practical implementation, the computational complexities of the adaptive algorithms can be further reduced. An adaptive weighted interpolation by avoiding the classifiers can be used to achieve the demosaiced output without color artifacts at lower computational complexity.