International Journal of Emerging Technologies in Computational and Applied Sciences (IJETCAS)
(Open Access, Double Blind Peer-reviewed, Refereed and Indexed Journal)
www.iasir.net
Symbolic Pattern Analysis Method for PCB Defect Detection and Classification
Vaddin Prathiba1, Dr. P.K Srimani2
1Assistant Professor, 2Professor,
Department of Computer Science & Applications, Bangalore University, Karnataka, INDIA
______________________________________________________________________________________
Abstract: According to recent advancements in manufacturing industries, Printed Circuit Boards and other components require depth inspection for better quality of product manufacturing. Electronic components are manufactured massively which require better quality assurance. Various approaches have been developed in recent years to provide support for manufacturing industries with the help of data mining or computer vision techniques. During image acquisition, images suffer from various occlusions and orientations which causes performance degradation in image inspection resulting in quality product manufacturing degradation and defective production. To overcome this issue, here we propose a computer vision based approach for defect detection in PCB and classification of defects. Classification is a significant stage to identify defective components. Main aim of this approach is to provide an automated system for PCB defect detection and classification with better accuracy and lower complexities. Proposed automated approach is carried out with the help of symbolic pattern analysis methodology. This approach is implemented on a given PCB image where initially image is transformed into symbolized form and features are extracted by dividing image into sub-regions i.e. background, foreground and shadow of defective region. Finally, a binary classifier is constructed with the help of symbolic dynamics to provide improved classification performance. Experimental study shows improved performance of proposed model when compared with existing state of art algorithms.
Keywords: PCB defect; computer vision; symbolic pattern; finite automata; classification
__________________________________________________________________________________________
I. Introduction
Bare printed circuit boards are the type of electronic components which are used for placing other electronic components for any electronic component manufacturing process [1]. PCB is an important part for producing any electronic component. Now a days, technology has grown rapidly which results in higher production of electronic components and devices. During this mass manufacturing because of various issues some electronic components are produced defective which are not acceptable by consumers or doesn’t meet industry manufacturing criteria.
These issues increase the cost of manufacturing because a manual inspection is required for better quality assurance.
Since electronic components are produced in a massive number where manual inspection is not feasible to implement. To cope up with this issue, various techniques have been developed in recent years for automated PCB defect detection and classification. Ercal et al. [2] discussed about this issue of PCB defect detection where three categories of PCB inspection approaches are discussed which are given as: (a) reference based technique (b) non- reference based inspection technique and (c) hybrid technique for PCB defect inspection. Reference based inspection approaches utilizes comparison of two PCB images for defect detection where one image is considered as reference image or ground truth image and another image is considered as a target image or defective image. In contrast to this approach, non-referential images don’t consider any ground truth image and there is no comparison performed between two images. According to this approach, image quality is verified by comparing with design rules which are defied by industry design experts such as width, height, length of conductors and insulator etc.
Various known defects which are present in PCB, mainly 14 types of known defects are presented in the literature for single layer PCB which are tabulated in table 1. Another approach is known as hybrid approach which is a combination of both referential and non-referential techniques. These approaches are complex to implement and expensive for mass production of electronic goods and are capable only for defect detection whereas classification of defect is not addressed. Hence, classification of defect still remains a challenging task for researchers [3]. Defect detection approaches are not able to provide complete information about defective region such are foreground of defect, background and shadow which makes it more complex to analyze the PCB defects during inspection. For better quality products, an efficient classification technique is required. Based on computer vision and data mining technique various methods have been developed. Wang et al [4] developed a classification technique for PCB defect detection. This approach comprised pixel-wise processing technique for PCB defect detection. Overall approach
consists of two stages; detection of PCB defects and classification of defects. Chomsuwan et al. [5] developed an automated approach for PCB defect detection. This approach is based on the analysis of testing of eddy current and magneto-resistive sensors. In order to carry out the study, eddy current signals are obtained with the help of mounted sensors on the coil in the scanning direction. Later fast Fourier transformation is applied to make scanning speed faster.
Table 1. Known defects for single layer PCB
Defect Number Name of Defect
1 Hole missing
2 Over etching
3 Conductor missing
4 Under etching
5 Breakout
6 Open-circuit
7 Unnecessary Short
8 Shorting
9 Spur
10 Mouse-bite
11 Wrong hole size
12 Under Etching
13 Conductor closeness
14 Pin Hole
Conventional approaches for PCB inspection systems, require human operators for better quality maintenance which is not automated. This method again increases the cost of implementation. Based on data mining technique, PCB defect detection techniques also have been developed. Darwish et al. [6] presented PCB defect detection for real time applications. This method of defect detection requires rule based strategy for defect detection. In order to detect the defect in PCB, rule based approach is applied to analyze the visual pattern of given input data resulting in detection of defect.
In this work, we present an automated process for PCB defect detection and classification with the help of computer vision system approach. In order to detect defects in PCB, highlighting the shadow of defective region, detection of foreground and background is performed here. Shadow of defective region is a critical parameter to be detected for better analysis of defective region detection in PCB components. According to proposed approach, initially input is processed through pre-processing stages, next stage includes image transformation to obtain symbolized image for feature extraction, and feature extraction step includes detection of background, foreground and shadow.
Finally, a binary classifier is developed with the help of analysed patterns of the input image and to obtain classification results.
Main contributions of this work are summarized as follows:
(a) Development of symbolized pattern analysis scheme
(b) Implementation of this approach for 2D-signals
(c) Development of a new approach for feature extraction and
classification
Rest of the article is organized as follows: section II briefs about most recent studies in this field of PCB defect detection, proposed solution for PCB defect detection is presented in section III, section IV discusses about experimental study and finally concluding remarks are presented in section V.
II. Related Work
This section provides a brief discussion about most recent studies which are presented in this field of PCB defect detection. As we have discussed in previous section that based on data mining approach also, various techniques have been developed. Kusiak et al. [6], discussed about defect detection technique based on data mining technique for industrial manufacturing inspection. Guh et al. [7], discussed about the pattern analysis study about manufacturing process to improve the defect detection analysis for quality assessment in manufacturing industry.
Authors discussed about conventional approach for pattern recognition as neural network methodology. This work mainly aims on pattern recognition and classification by considering information such as slope, magnitude etc.
According to this article authors have discussed that the existing techniques have faced the problem when there are
two or more patterns mixed together. Considering these issues, authors have developed a neural –network based approach to identify abnormal patterns present in PCB datasets where mixed patterns can be analyzed efficiently.
Sequential Pattern Analysis designs strategy is adopted for complexity reduction by reducing complexity between patterns.
Zhang et al. [8] developed a clustering approach for detection of solder joint variations for quality control technique.
In order to carry out this work, a logistic regression model is incorporated along with latent variable models. This combination of models is capable to provide better information about solder joint pattern. For dimensionality reduction, maximum-likelihood principal component analysis (MLPCA) is applied for feature pattern analysis.
Taniguchi et al. [9], presented a computer vision based approach for wavelet-based image processing technique with the help eddy-current testing (ECT). Here in this work wavelet filtering is applied to remove undesired components. In the next stage of implementation, reference based image processing is applied to extract the defective region.
Accian et al. [10] presented a well-defined scheme for PCB defect detection with the help of computer vision approach. This work mainly aims on the detection of defective solder joints. In order to carry out this computer vision based technique for solder joint, neural network based method have been proposed here for surface mount technologies. According to this approach of solder joint detection, five type of defects are classified referring to higher classification performance to support manufacturing process. Feature extraction is carried out by considering region of interest with the help of wavelet transform. Finally a comparative study is presented for K-NN, and neural network classifiers.
Erdahl et. al. [11] also discussed about PCB and their connectivity inside the package. For connectivity analysis, conventional approaches have been used such as X-ray, ultrasonic models. During the analysis, these methods are not capable to provide solder mismatch between chips and PCB. In this work authors have addressed this issue by applying computer vision based analysis technique.
III. Proposed Model
In this section, we describe proposed symbolic pattern analysis approach for PCB defect detection using PCB texture image analysis with the help of computer vision techniques. Proposed scheme is divided in below mentioned stages:
1. Modelling of PCB defect data
2. Symbolic pattern analysis of PCB data 3. Geometrical partition of data
4. Modeling of neighboring pixels 5. Feature extraction
6. Classification
First of all we discuss about modeling of PCB data. According to proposed approach of symbolic pattern analysis, here PCB image is considered as a texture image which is acquired by varying illumination, pose and scales.
Complete modeling is presented in next sub-section.
A. Modelling of PCB defect data
This section deals with PCB data modeling. As discussed before, here PCB image data is considered as a texture data which is developed by applying computer vision technique such as illumination variation, pose variation etc.
In order to model the data, we determine the textures of image by evaluating input image pixel-wise processing.
Pixel variations are computed and stored in a pattern in the form of symbols where defective PCB region is denoted as 𝛼 and non-defective is given as 𝛽.
Based on this model, feature extraction and further processing on the given texture image dataset. In our model, texture image with defects is categorized into four main sections which are given as:
(a) Defective spot in image (𝐷𝑚), (b) Defective region shadow (𝐷𝑠),
(c) Defective region background (𝐷𝑏𝑚) and
(d) Shadow background near defective region (𝐷𝑏𝑠)
In some cases, defect is present in an isolated region of PCB image. This isolated defect can be characterize by highlighting the shadow of defective region or highlighting the dark pixels. Defective region length is responsible for size of defective region shadow. In this experimental analysis, small variation is considered in the shadow of defective region. In this work, a symbolic pattern analysis scheme is used for PCB defect analysis and detection.
Overall architecture of proposed work is presented in figure 1. According to proposed approach, we apply image pre-processing techniques, symbolic pattern initialization, pattern computation, feature extraction , building a classifier, performing classification for given image set with the help of pattern classification.
Figure 1: Overall flow chart of proposed model
B. Symbolic pattern analysis
This section deals with symbolic pattern analysis modeling to extract the various properties of PCB image. Existing work concentrates on time-series data, here we develop this model for two-dimensional image data for PCB defect detection purpose.
Main stages of this work are presented below:
First of all, a time series data is generated for a given data by encoding non-linear system dynamics
A probabilistic finite state automata is constructed with the help of symbol sequence.
Later, a pattern vector is generated for detection and classification of given PCB image set.
Here we present a brief discussion about feature extraction with the help of symbolic pattern analysis. According to symbolic dynamic theory, time series data is represented in the form of symbolic sequence. Let us consider a bounded region as Ω ∈ 𝑅𝑛 in 𝑛 dimensional phase which consist of the information about the trajectory of pixels.
This region is divided into Ω similar regions which is used to form the grid cells. Each visited cell by the pixel trajectory is denoted by taking random variables as symbol from constructed alphabets ∑. This dynamical system is denoted as {𝑥0, 𝑥1, … , 𝑥𝑘… } with 𝑥𝑖∈ Ω which follows each cell from grid. With the help of this symbolic pattern set, a symbolic sequence is created which is denoted as {𝜎1, 𝜎2, … , 𝜎𝑘, … } where each symbol of this sequence is belong to the set of alphabets Σ.
C. Geometrical partition of data
In this section, we present a geometrical modeling for data partition by considering each cell. In order to carry out this approach, two parameters need to target which are: size of alphabet construction and partition segmentation boundary. For PCB defect detection application, information of PCB need to be preserved such as highlighting are, edges etc. of images. For defective region, here we perform histogram analysis for defective region analysis based on the intensities of defective region, shadow of defective region and combined background region. Histogram analysis is performed to analyze the intensity values of image in a given geometric model where intensity varies from 0 to 255. From this observation it can be concluded that intensity of background and defective region background is almost similar which can be combined. According to this observation of pixel distribution, similar intensities of defective region and background are obtained which is combined in a single histogram where these histograms consist the intensity variation of image as depicted in figure 2. Hence, we construct a symbolic pattern by considering defective image, shadow of defective image and background of defective region which is denoted as Σ = {𝑎, 𝑏, 𝑐}.
D. Modeling of neighboring pixels
This section deals with neighborhood pixel modeling which is used for feature extraction process. As discussed in previous section that how symbolized image 𝐼Σ is generated by computing histogram model of each region of input image. In order to model the neighboring pixels, these stages are performed:
1. Labeling grayscale pixel,
2. Intensity representation of grayscale pixels 3. Region partition based on their classes 4. Construction of finite state automata
According to this model, input image is converted into grayscale image where each gray-scale pixel is labeled as 𝜎 ∈ ∑ = {𝑎, 𝑏, 𝑐} where a, b and c represents the intensity levels of a gray-scale image. In the next stage, regions are divided based on the class of image where 𝐵 ⊂ 𝐻 denotes a local region of image, here original image is denoted as 𝐴𝑚, shadow of this region is given as 𝐴𝑠, background is denoted as 𝐴𝑚𝑏 and background of defective region shadow is given as 𝐴𝑠𝑏. By considering these models, we construct a finite state automata which is applied here for feature extraction modeling in given neighborhood of pixels as (𝑖, 𝑗) ∈ 𝐵
In this process of state construction by using neighborhood modeling, each pixel is mapped in the states.
Two different locations of an image, given as (𝑖, 𝑗) and (𝑖1, 𝑗1) with the configuration symbol ℕ(𝑘−1)(𝑖, 𝑗) and ℕ(𝑘−1)(𝑖1, 𝑗1) respectively. These symbol represents the same state if the occurrence of pixel identities is similar.
Transition probability is computed as 𝑀𝑝𝑟𝑜𝑏(𝑠1
𝑠𝑛
) = 𝑁(𝑠1, 𝑠𝑛)
∑𝑛=1,2,…𝑛 𝑁(𝑠1′, 𝑠𝑛) (4)
𝑁(𝑠1, 𝑠𝑛) is the total number of occurrence E. Feature Extraction
In this section, feature extraction process is presented based on the transition probability as mentioned in Eq. (4).
For any given image, based on the pixel locations, geometric is constructed including all pixels which are present in defective region. For input PCB image, below mentioned features are extracted
1. 𝜌 is constructed in the image region where the location of pixel is in the center of image.
2. 𝜏 is constructed from the shadow of the image where the location of pixel is in the centre of image.
3. 𝜑 is constructed from the background of the image where the location of pixel is in the centre of image.
4. 𝜔 is constructed from the shadow of the background image where the location of pixel is in the centre of image
Thus each image is having neighborhood pixels based on the location of the regions.
IV. Results and Discussion
In this section we present results of proposed approach for PCB defect detection. Propose approach is applied by considering synthetic and publically available free dataset where defects are present. Proposed approach is implemented using MATLAB. In order to show robust performance of this approach, we consider experimental studies where PCB images are rotated, noise is added and different orientations are applied to the input image. In the first experiment, we consider a general PCB defective image where defect is present in the image, entire process for this image is presented in figure 3 to figure 12.
Complete process of this approach is mentioned here. Next stage to perform modelling on the dataset by using the proposed scheme, according to that main region, defect, shadow and background are identified for further processing. During the partition of image, each image is divided into exhaustive and exclusive section to create the symbols. This image is called symbolized image which is used for the construction of finite state automata. To construct the finite state automata model, neighborhood pixel modelling is introduced, according to this process each pixel and neighbouring pixels are analyzed for the state transitions, at this stage all the states are defined of the image and based on the intensity variations of image. After this feature extraction takes place, for a specific pixel location of a texture image four features are extracted which are defined as: image region for center location of image, image shadow features, image background features and shadow background features, these features are extracted by using finite state automata model. Finally classification is performed by using a threshold based method is used based on the pattern of the texture. Four scalar thresholds are used to determine the classification.
The classes are defied based on the available classes in the dataset. Scalar threshold values are denoted by 𝓉1, 𝓉2, 𝓉3𝑎𝑛𝑑 𝓉4. To classify the texture threshold values are matched with probabilistic model of finite automata and classified as the texture resemblance, the resemblance to the shadow of texture and background of texture is adjacent to the texture.
Figure2: Original Image Figure 3: Grayscale Image
Figure 4: Binary Conversion Figure 5: Dialeated image
Figure 6: Finite Automata image Figure 7: Finite set construction
Figure 8: FSA Classified background and Figure 9: Foreground Shadow
Figure 10: Shadow of background Figure 11: Feature extracted image
In above figures we present entire processing of feature extraction for defect detection with the help of finite automata scheme. Figure 2 shows input image, in figure 4 grayscale image is presented, later we perform binary conversion and dilation which is presented in figure 5 and 6 respectively. In figure 6 and 7, we show finite state automata initialization and construction. Finally background, foreground and shadows are classified as presented in figure 8, 9 and 10 respectively and figure 11 present feature extracted image.
V. Conclusion
For PCB defect detection, here we present a symbolic pattern analysis based methodology for detection and classification. In order to carry the research, here we develop a pattern of PCB input image sequence using symbolic pattern analysis. In this work, we present a modeling of finite state automata approach for two dimensional images where feature extraction and classification is performed. Finite state automata is used to design a classifier for PCB defect detection. A key aspect of this paper is construction of a geometric model for texture classification from texture images. The observed patterns corresponding to different regions of the geometric model are then fused in a classification scheme to make binary decisions. The algorithm has been tested on different images of 200X200.
The probability of correct classification of texture is found to be 99.12%.
The major advantages of the proposed pattern analysis algorithm are delineated below.
1) Performance of the statistical symbolic pattern analysis algorithm is robust with respect to pose, illumination and orientation.
2) The algorithm is computationally efficient in terms of execution time and memory requirements as a consequence of small alphabet size and a small number of states in probabilistic automata.
VI. References
[1] J. Hong, K. Park and K. Kim, Parallel processing machine vision system for bare PCB inspection, Proc. of the 24th Annual Conference of the IEEE, pp.1346-1350, 1998.
[2] M. Moganti, F. Ercal, C. H. Dagli and S. Tsunekawa, Automatic PCB inspection algorithms: A survey, Computer Vision and Image Understanding, vol.63, no.2, 1996
[3] Ismail Ibrahim, Zuwairie Ibrahim, Kamal Khalil, Musa Mohd Mokji Syed Abdul Rahman Syed Abu Bakar, Norrima Mokhtar and Wan Khairunizam Wan Ahmad, An improved defect classification algorithm for six Printing defects and its implementation on real Printed circuit board images, International Journal of Innovative Computing, Volume 8, Number 5(A), May 2012
[4] W. Y. Wu, M. J. Wang and C. M. Liu, Automated inspection of printed circuit boards through machine vision, Computers in Industry, vol.28, no.2, pp.103-111, 1996
[5] K. Chomsuwan, S. Yamada and M. Iwahara, "Improvement on Defect Detection Performance of PCB Inspection Based on ECT Technique With Multi-SV-GMR Sensor," in IEEE Transactions on Magnetics, vol. 43, no. 6, pp. 2394-2396, June 2007
[6] A. Kusiak and C. Kurasek, "Data mining of printed-circuit board defects," in IEEE Transactions on Robotics and Automation, vol.
17, no. 2, pp. 191-196, Apr 2001.
[7] R. S. Guh and J.D.T. Tannock, A neural network approach to characterize pattern parameters in process control charts, Journal of Intelligent Manufacturing, vol.10, no.5, pp.449-462, 1996
[8] F. Zhang and T. Luk, "A Data Mining Algorithm for Monitoring PCB Assembly Quality," in IEEE Transactions on Electronics Packaging Manufacturing, vol. 30, no. 4, pp. 299-305, Oct. 2007.
[9] T. Taniguchi, D. Kacprzak, S. Yamada and M. Iwahara, "Wavelet-based processing of ECT images for inspection of printed circuit board," in IEEE Transactions on Magnetics, vol. 37, no. 4, pp. 2790-2793, Jul 2001
[10] G. Acciani, G. Brunetti and G. Fornarelli, "Application of neural networks in optical inspection and classification of solder joints in surface mount technology," in IEEE Transactions on Industrial Informatics, vol. 2, no. 3, pp. 200-209, Aug. 2006.
[11] D. S. Erdahl, M. S. Allen, I. C. Ume and J. H. Ginsberg, "Structural Modal Analysis for Detecting Open Solder Bumps on Flip Chips," in IEEE Transactions on Advanced Packaging, vol. 31, no. 1, pp. 118-126, Feb. 2008.
Acknowledgements
One of the authors Vaddin Prathiba acknowledges the Rayalaseema University, Kurnool, Andhra Pradesh, India & M. S. Ramaiah College of Arts, Science and Commerce, Bangalore, Karnataka, India for providing the facilities to carry out the research work.
