Computer Machine Vision Inspection on Printed Circuit Boards Flux Defects

The new visual inspection systems techniques using real time machine vision replace the human visual manual inspection on PCB flux defects, which brings harmful effects on the board which may come in the form of corrosion and can cause harm to the assembly. In short, it brings improvement in Printed Circuit Boards (PCB) production quality, principally concerning the acceptance or rejection of the PCB boards. To develop new algorithm in image processing which detects flux defect at PCB board during re-flow process and achieve good accuracy of the PCB quality checking. The machine will be designed and fabricated with the total automation control system with mechanical PCB loader/un-loader, pneumatic system handler with vacuum cap, vision inspection station and final classification station (accept or reject). The image processing system is based on shape (pattern) and color image analysis techniques with Matrox Imaging Library. The shape/texture of the PCB pins is analyzed by using pattern matching technique to detect the PCB flux defect area. The color analysis of the flux defect in a PCB boards are processed based on their red color pixel percentage in Red, Green and Blue (RGB) model. The red color filter band mean value of histogram is measured and compared to the value threshold to determine the occurring on the PCB flux defects. The system was tested with PCB boards from factory production line and achieved PCB board flux defects sorting accuracy at 86.0% based on proposed pattern matching technique combined with red color filter band histogram.


INTRODUCTION
The main disadvantage of manual inspection of PCB defects are human errors, inconsistent grading and labour intensive. The inspection process can be automated by "PC-Based Machine Vision". To identify the fatal defects the system uses a connectivity approach, it finds any type of error like: PCB board printing and labeling, scratches, marking on components, components orientation and others. For this research, the main defect focus is on PCB flux defect, which is very difficult to visualize by human eye because it's liquid and transparent. The result, obtained based on the proposed technique is possibly be applied in automated PCB manufacturing process.
The machine will be designed and fabricated with the Total Automation Control System with mechanical PCB loader/un-loader, pneumatic system handler with vacuum cap, vision inspection station and final classification station (accept or reject).
Computer Vision techniques were used to develop an automatic visual inspection of PCB boards, which Science Publications AJEAS intends to evaluate the various PCB board defects. The fault detection strategy refers to the use of referential inspection methods, in which the reference is a board artwork or a manufactured board without errors. The PCB defects are normally grouped in two categories, the fatal defects (reject units) and no defects (accept units). The system identifies the fatal defects using an image comparison technique, subtracting the reference board image from the tested board image. This project has been designed to combine all aspects of engineering, including mechanical, electrical, electronic, communications and software engineering, into one development product. The machine will be able to inspect and separate the PCBs defects board from good ones.

Literature Review
Numerous PCB inspection algorithms have been proposed in the literature to date. Greenberg et al. (2006), this invention discloses a method for Printed Circuit Board (PCB) inspection, including providing a multiplicity of PCBs placed on an inspection panel, defingng each non-indentical PCB in terms of geometry and features which are to be inspected. Leta et al. (2005), This study uses some computer vision techniques to measure parts and discusses common difficulities of automated inspection. The parts conformity analysis using a non-contact measurement system has been adopted specially to small objects, where accurate instruments like Coordinate Measuring Machine (CMM) is used. Tsai and Tsai (2002), in this study a rotationinvariant pattern-mathcing scheme for detecting objects in complex color images is proposed. The complexity and computational load for matching colored objects in arbitary orientations are reduced significantly by the 1-D color ring-protection representation. Singh and Bharti (2012), The on-line or automatic visual inspection of PCB is basically a very first examination before its electronic testing. This inspection consists of mainly missing or wrongly placed components in the PCB. If there is any missing electronic component then it is not so damaging the PCB. Mashohor et al. (2004), this study presents the first prototype of automating a low-cost printed circuit (PCB) inspection on physical defects through the development of a technique for image detection using Genetic Algorithm (GA). Sundaraj (2009), during etching process, the anomalies occurring on bare PCB could be largely classified in two categories: the one with excessive copper and missing copper. The incomplete etching process leaves unwanted conductive materials and forms defects like short-circuit, extra hole, protrusion, island and small space. Excessive etching leads to open-circuit and thin pattern on PCB. In addition, some other defects may exist on bare PCB, i.e. missing holes, scratch and cracks. Mar et al. (2009) proposed a front-end system for the automatic detection, localization and segmentation of solder joint defects. An illumination normalization approach is applied to an image, which can effectively and efficiently eliminate the effect of uneven illumination while keeping the properties of the processed image the same as in the corresponding image under normal lighting conditions. Consequently special lighting and instrumental setup can be reduced in order to detect solder joints. In the segmentation approach, the PCB image is transformed from an RGB color space to a YIQ color space for the effective detection of solder joints from the background. Zeng et al. (2011) proposed an algorithm of allocating and identifying PCB board components based on color distribution of solder joints, the proposed approach analyzes the color distribution patterns of solders joints under three layers of ring-shaped LEDs. Chuhan and Bhardwaj (2011) used image subtraction method to detect PCB Bare defects which involves loading a reference image, buffering the reference image for subtraction operation and inspecting the PCB error; inspected image is XORed with reference image. The obtained defected area undergoes "particle analysis" which is used to analyze the defects in terms of area, size, orientation and percentage. Kaushik and Ashraf (2012) also tested on image subtraction method to detect PCB defects under different noise levels. Bhardwaj (2012) proposed an Automated Optical Inspection (AOI) algorithm for PCB inspection system. The system performs measurement and detection of holes defect that occur during PCB etching process during manufacturing. One image feature is measured from the examined image and is used for detection of defects. Different steps and methodology are used for defect detection. At first, an image is selected for inspection from the location where captured image are stored. To enhance the image for detection filters are used. Low pass filter and highlight detail filter are used and grayscale image converted into binary one by applying threshold. For the detection of defects algorithms are applied on the enhanced image. Advanced Morphology is used to remove the boarder object in Science Publications AJEAS the binary image. This process removes unwanted portion of image. On the resultant image Particle Analysis functions i.e., circle detection and Particle measurement are applied. Output of these functions gives the desired results in terms of specified parameters.

MATERIALS AND METHODS
This involves specifically computer vision that will be used to detect the PCB boards flux defects during "After Cleaning" process in real time. Firstly, the introduction and then followed by methodology use and specifically highlight the algorithms used, which involved pattern matching technique. This algorithm was applied also with red color filter band (RGB) segmentation after image pre-processing to remove noise/threshold/masking. The results obtained will be presented, thoroughly discussed and conclusion will be drawn.
The automated modules of the machine vision system for the detection of the PCB boards flux defects are SAMSUNG camera, Matrox frame grabber and software, Dell computer, Advantech DAQ and mechanical handler with electro-pneumatic system. The machine vision constructed with these componenets is shown in Fig. 1 along with block diagram of inspection system in Fig. 2.

Camera Specification
Crystal clear color images at 600 TV lines resolution, Ultra low light level of 0.05 Lux and 2D/3D Filtering noise reduction technology SSNRIII.

Frame Grabber Specification
Standard analog color/monochrome PCI frame grabber with two video decoders, analog composite (CVBS) or Y/C NTSC/PAL video input through BNC connectors, horizontal and/or vertical flip, sub sampling to 1/16th of a field or frame.

DAQ Specification
48 TTL digital I/O lines, Emulates mode 0 of 8255 PPI, Buffered circuits for higher driving capacity than 8255, Interrupt handling, Timer/Counter interrupt capability, supports both dry and wet contact and compatible with USB 2.0.
The PC-Based Machine technology also gives more advantage on control system and provides capability on the statistic database result on the product analysis compare to PLC control system. The machine will be designed and fabricated with the total automation control system with mechanical PCB loader/un-loader, pneumatic system handler with vacuum cap, vision inspection station and final electro-pneumatic classification station (accept or reject). The flowchart of PCB inspection system shown in Fig. 3 provides a clear understanding of the image processing. In this industry research, Auto Focus (AF) Lens was used where the brightness automatically detects environmental lighting to control the best light source of the illumination to the machine vision system on PCB boards inspection.
Basically, a PCB is picked from a stacker using a vacuum cup and placed on a platform. The platform moves until the PCB is directly underneath a camera, which captures an image of the PCB and sends it to the computer for processing. The results of the inspection are shown on a monitor. The inspected PCB is then placed at its respective position based on whether it is good or has errors.
The basic principle of machine vision system can be divided into few parts as below.

Image Acquisition
An optical system gathers an image, which is then converted to a digital format and stored into the computer memory.

Image Pre-Processing
GLPF and Morphology Analysis (Area Open) algorithms are used to enhance elements or remove noise off the image that are of specific importance to the process.

Feature Extraction (Image Processing)
The processor identifies and quantifies critical features in the image (e.g., PCB flux defects, pattern matching, red color filter band, threshold and histogram) and sends the data to a control program.

Decision and Control
The processor's control program makes decisions based upon the data. Is the flux area within specification?

Image Pre-Processing
The purpose of image pre-processing is to improve or to enhance the digital images. It is important for correction of geometric distortions, removal of noise, grey level correction and correction for blurring. In order to remove unwanted noise the GLPF and Morphology Analysis (Area Open) was used. As the image operations have been changed, so as the image pre-process flow chart shown in Fig. 4.

AJEAS
an area size never occurs, the peak will be clipped until all the pixels have same intensity value as the background pixel.
To increase the processing speed and reading better result of the PCB board rotated angle, the amount of information was reduced by converting color image to 8-bit unsigned grayscale, Fig. 5. In order to remove the image noise and reduce detail levels and to enhance the image structures the Gaussian Low-Pass Filter (GLPF) smoothing operator for two dimensions as in Equation 1 was applied: After image smoothening with GLPF, the image will then increase its brightness for the purpose of next image processing and analysis. The brightness operation of the GLPF image was done using the Equation 2:

Feature Extraction
In this study, the reference image ROI (regions of interest) with the size 869×469 pixels were chosen as shown in Fig. 6 from PCB image with size 1890×2332.
After the Pre-processing stage, the image will be process with feature extraction algorithm. The purpose of the feature extraction is to segmentation the flux defect area with reference to the 3 IC's pin. The flux defect areas always appear around the pin area instead of other parts in PCB. The red color band will be extracted from RGB color to find the flux coverage area.
The detail on the feature extraction and image processing based on pattern matching and red color filter band was shown on Fig. 7-Image Processing Algorithm.

The Pattern Matching
Function of the Pattern Matching module is to search for occurrences of a pattern in an image.
Normalized correlation operation can be seen as a form of convolution, where the pattern matching model is analogous to the convolution kernel. Ordinary correlation is exactly the same as a convolution Equation 3: The N pixels of the model are multiplied by the N underlying image pixels and these products are summed. Note, the model does not have to be rectangular because it can contain "don't care" pixels that are completely ignored during the calculation. When the correlation function is evaluated at every pixel in the sample image, the locations where the result is largest are those where the surrounding image is most similar to the model. The search algorithm then has to locate these peaks in the correlation result and return their positions (Where I image and M is model pixel).
Unfortunately, with ordinary correlation, the result increases if the image gets brighter. In fact, the function reaches a maximum when the image is uniformly white, even though at this point it no longer looks like the model. The solution is to use a more complex, normalized version of the correlation. In this case, negative values are not important, so results are clipped to 0. In addition the r 2 value is used instead of r to avoid the slow square-root operation. Finally, the result is converted to a percentage, where 100% represents a perfect match. So, the match score returned is actually: Note, some of the terms in the normalized correlation function depend only on the model and hence can be evaluated once and for all when the model is defined. The only terms that need to be calculated during the search are Equation 5: This amounts to two multiplications and three additions for each model pixel.
The sums used to compute the correlation function can be retrieved using MpatGetResult(). The retrievable sums are the following Eqation 6: The number of pixels N is also available using MpatGetResult(). The above sums are only available if they are saved in the result buffer using MpatSetSearchParameter() with M_SAVE_SUMS set to M_ENABLE. These sums can be used, for example, to assess the model or target intensity level and contrast. Note that the correlation function calculated with the retrieved sums, will probably differ from the score returned by MpatGetResult().
Although the score is derived from the correlation function, it also depends on the first and last subsampling levels chosen, optimization schemes and sub-pixel interpolation. On a typical computer, the multiplications alone account for most of the computation time.

Real Color Filter Band Algorithm
After the sample images PCB flux area are being captured by pattern matching. Then, extract the ROI to RED color filter band in a new window as shown at The Threshold value 100 is chosen because it produces the best result appearance to extract the flux out of 50 samples. It is supposed that the image f(x,y) consists of light objects on a dark background. One obvious way of extract the objects from the background is to select a threshold T; the threshold image F(x,y) is defined as (7): The images show the red color filter band with three different Threshold value (T).
For each image the histogram function is used to identify the mean value of RED intensities in the image. Only the mean value of the Red color filter band in histogram was considered to get the average on the red value pixel.  Repeating the same process for the sample image without the flux (pass). Note that from the Table 1 the mean value for the sample image 3 without the flux is 1.13 and the sample image 1 with the flux is 10.59. From this we conclude that if the image has a mean value over 9 then it's considered as Fail. The histogram for ROI (RED-Band) and mean value is given at Fig. 12.

RESULTS
In order to test the prototype system, the pictures were taken in the real condition lighting at site. In total, 50 PCB were collected and tested with the different feature extraction technique described previously. In this research, the fix threshold is used based on Red color filter band method on histogram.

DISSCUSSION
After applying pre-image processing to remove the image noise and reduce detail levels and to enhance the image structures with Gaussian low-pass filter (GLPF) for smoothing. The images are ready for features extraction with pattern matching and red color filter band for PCB flux detection. Before the image processing start on the production samples, the PCB reference image or model source image will be used with the pattern matching technique to recognize the 3 IC's pin lead for the flux area inspection. After the 3 pin lead recognition, all leads will be mask so that the balance area only contains the flux affected area for flux inspection.
In RGB representation, colors are described with three numbers for the red, green and blue components, Therefore, red filter band is applied, only the red portion in the image will be remain. Therefore, a more apparent trend of the histogram for each category due to flux defects area is given. The function extract only the R of RGB analysis to process the image because the PCB flux is more depend on red color only.  Table 2 and Fig. 13 show the result on the red color filter band mean value based on PCB board rotated angle. Some bigger rotated angle shows "undefined". The reason is the pattern matching techniques used may not able to detect the flux defect after certain bigger rotation angle on the PCB. 7 out of 50 sample images cannot be classified by using this algorithm. Figure 14 shows 86.0% accuracy with algorithm applied.

CONCLUSION
This research shows that machine vision system is able to implement the sorting on PCB boards flux defects based on shape (pattern) and color space with machine vision processing. The system was developed with various images processing techniques by using Matrox Imaging software. The algorithms shown below were carried out in this research to improve the PCB fluxes defects grade into accept or reject: • With features extraction color space based on red color filter band constraints manage to extract the PCB flux defect images • Fix threshold can select its desire and best threshold value 100 from the image and is able to take care of different category to make sure PCB fluxes occur for the rejection category • The feature extraction algorithm using pattern matching based on RGB color with red color filter band, having accuracy 86.0% were correctly classified • Additional algorithm may be used to resolve the bigger PCB rotated angle