This is the author’s version of a work that was submitted/accepted for
pub-lication in the following source:
Mar, Nang S.S.
,
Fookes, Clinton B.
, &
Yarlagadda, Prasad K.
(2012) Solder
joint defects classification using the Log-Gabor Filter, the Discrete Wavelet
Transform and the Discrete Cosine Transform. In International Conference
on Advances in Mechanical and Building, 9-11 January 2012, Tamil Nadu,
India.
This file was downloaded from:
http://eprints.qut.edu.au/49329/
c
Copyright 2012 [please consult the author]
Notice: Changes introduced as a result of publishing processes such as
copy-editing and formatting may not be reflected in this document. For a
definitive version of this work, please refer to the published source:
Solder Joint Defects Classification using
the Log-Gabor Filter, the Discrete Wavelet
Transform and the Discrete Cosine
Transform
N.S.S. Mar, C. Fookes, P.K.D.V. Yarlagadda
Abstract — Inspection of solder joints has been a critical process in the electronic manufacturing industry to reduce manufacturing cost, improve yield, and ensure project quality and reliability. This paper proposes the use of the Log-Gabor filter bank, Discrete Wavelet Transform and Discrete Cosine Transform for feature extraction of solder joint images on Printed Circuit Boards (PCBs). A distance based on the Mahalanobis Cosine metric is also presented for classification of five different types of solder joints. From the experimental results, this methodology achieved high accuracy and a well generalised performance. This can be an effective method to reduce cost and improve quality in the production of PCBs in the manufacturing industry.
Keywords— Classification of solder joint defects, Discrete Cosine Transform, Discrete Wavelet Transform, Log-Gabor filter
I. INTRODUCTION
SSEMBLY of Printed Circuit Boards (PCBs) using Surface Mount Technology (SMT) has been widely used in the electronics industry recently [1] and the quality of the solder joint is critical to the quality of that PCBs. Automatic Optical Inspection (AOI) of solder joints has become an important issue for quality control in PCB assembly as AOI has the enormous potential of completely automating human visual inspection procedures [2]-[6]. The aim of these inspection procedures is to detect and locate any potential solder joint defects which will break down the functions of the final PCB products. Common solder joint defects which are of interest include: excess solder, less solder, no solder, and bridge solder joints. Many research techniques have been developed to recognize faulty solder joints.
II. LITERATURE REVIEW
To complete the inspection system, classifiers have been developed using different approaches to select features
extracted from an image. The Bayesian and maximum likelihood classifier are a traditional statistical approach for classification. Bartlet et al. [7] used a minimum classifier to classify each solder joint into one of the two good types and seven defect types depending on five different features: basic gray level statistics, 3D gray level inertia features, faceted gray level surface area features, differential geometric gray level surface curvature features and binary image connected region features.
Although rule based classification system is easy to understand and interpretable, this method needs counter measures against the appearance of unexpected members of a class. Capson [8] proposed a tree classifier by using geometric descriptors which measure the shape of the contours and the colour level intensities of the solder joint images. Another rule based classification system was also employed by using fuzzy set theory [9],[10]. This can be especially useful when objects are not clearly members of one class or another. In addition, fuzzy techniques will specify to what degree the object belongs to each class, which is useful information in solder joint recognition.
Artificial Neural Network (ANN) approaches have also been applied to AOI systems due to their learning capability and nonlinear classification performance. Multilayer neural networks are one of the supervised neural networks and have been applied to solder joint inspection [11]. Kim [11] proposed the neural network with two following modules: a processing module and training module. The first module was designed to implement the calculation of the correlations in functional terms and the second module was designed to learn about solder joint classification as a human inspector. The advantage of a multilayered neural network is the ability of learning human experiences. However, the complexity of solder joint shapes causes the neural network’s convergence rate to become poor.
Kim [12] applied an adaptive learning mechanism to correct the LVQ classifier. The adaptive learning mechanism can select the optimal number of prototypes set to achieve
A
performance within an acceptable error rate. Again, Ong [13] proposed a new approach using a dual viewing angle imaging method with an ANN and learning vector quantization architecture. From the experimental results, this system had an improved recognition rate and resistance to noise. However, high accuracy on the relative geometrical position of the camera was required.
Ko et al. [14] combine a fuzzy logic scheme into the LVQ neural network. In the neural network module, three LVQ classifiers were used to cluster solder joints by using similarity between an input pattern and prototype patterns. In the fuzzy module, new classification criteria were generated using a pre-defined rule built by the inspectors. More accurate classification results can be achieved because expert knowledge is applied into the classification criteria.
Accianni [15],[16] extracted two feature vectors, the “geometric” feature vector and the “wavelet” feature vector, from the images and used a multiple neural network system to characterise solder joint defects on Printed Circuit Boards assembled in Surface Mounting Technology. Although many methods have been developed in this area, these methods need a complex system of image acquisition and do not give good results when the inconsistency for solder joint shapes is encountered. Better inspection technologies are needed to fill the gap between available inspection capabilities and industry systems.
III. METHODOLOGY
This paper introduces a new technique for solder joint defect classification using theLog-Gabor filter which has been demonstrated to achieve a high recognition rate and is resistant to misalignment [17]. This filter has also found favour in many image processing fields due to its desirable characteristics of spatial locality and orientation selectivity. Further testing demonstrates the advantage of Log-Gabor filter over Discrete Wavelet Transform and Discrete Cosine Transform.
A. Gabor Filter
In recent times, Gabor filters have emerged as one of the most commonly used techniques in the field of image recognition and analysis. An important property of Gabor filters is that it has optimal localisation properties in both spatial and frequency domains [18] and can be used to decompose images into components corresponding to different scales and orientations. The frequency response of Log-Gabor filter is defined as,
2 0 2 0 ) / ln( 2 ) / ln( exp ) (
, (1)where 0 is the centre frequency of the sinusoid and is a scaling factor of the bandwidth.
In this experiment, the Log Gabor filter bank is constructed with a total of 6 orientations and 5 scales. The shape parameter, / 0 was chosen such that each filter had a bandwidth of approximately 1.5 octaves. The total of 30 feature vectors representing different scales and orientations
are achieved. Different combinations of the Gabor representation are achieved from 30 feature vectors. However, the magnitude representation outperforms other representation.
B. Discrete Wavelet transform
The Discrete Wavelet transform is computed by successive low pass and high pass filtering of the discrete time-domain signal. Wavelet decomposition is the projection onto an orthonormal set of basis vectors which are generated by dilation and translation of a single “mother wavelet”. In Fig 1, the signal is denoted by the signal X of length of N, the DWT consists of log2 N levels at most. At each level, the high pass
filter produces detail coefficients d[n], while the low pass filter associated with scaling function produces approximation coefficients a[n]. These vectors are obtained by convolving X with the low-pass filter Lo_D for approximation, and with the high-pass filter Hi_D for detail, followed by downsampling. At each decomposition level, the half band filters produce signals spanning only half the frequency band. This doubles the frequency resolution as the uncertainty in frequency is reduced by half.
Fig. 1 Three level wavelet decomposition tree
The simplest Haar wavelet is ,
else
t
t
t
:
0
1
2
/
1
:
1
2
/
1
0
:
1
)
(
.C. Discrete Cosine Transform
Discrete Cosine Transform has been widely used in feature extraction and image compression because it has the effect of concentrating the energy in a signal in a relatively small number of coefficients [19]. The 2D- Discrete Cosine Transform is defined as,
.
2
)
1
2
(
cos
2
)
1
2
(
cos
)
,
(
)
(
)
(
)
,
(
1 0 1 0
N
y
M
u
x
y
x
f
u
u
C
M x N y
The Blocked DCT is used in transforming to the frequency domain. The block based nature of the DCT performs the same task as Fast Fourier Transform (FFT) in a more efficient
manner. Another advantage of the DCT is that its basis vectors are comprised of entirely real-valued components. Therefore all pixel values are represented by real numbers and the pixels themselves do not affect each other as values of the pixels come directly from the transform of the time domain values.
D. Distance Measure
The Mahalanobis Cosine Distance is used to compute the similarity measure between two filter banks. The Mahalanobis Cosine Distance measure is defined as,
,
cos
) , ( cosn
m
n
m
n
m
n
m
D
M ah ineuv m n
where m and n are two feature vectors transformed into Mahalanobis space.
Detail of theory background in this experiment can be found in [6].
IV. EXPERIMENTATION AND RESULT ANALYSIS
The resolution of each solder joint image after segmentation is 40 x 40 pixels. Fig 2 shows the five different types of solder joints after segmentation. For each type of solder joint, 50 images are used as the training samples and two hundred images are used as the test samples. The images used for training and testing have been chosen randomly from the original database for every individual experiment. During testing, experiments are performed 25 times and the value averaged to get one complete classification result. When the number of experiments reached 25, the mean value becomes stable and this value was used to plot the graph. The experimental result is plotted as a probability distribution and evaluated by Detection Error Trade-off (DET) plot. From 1000 test images, the solder joints are classified into five different groups with respect to the amount of solder paste: good solder joint, excess solder joint, less (insufficient) solder joint, bridge solder joint and no solder joints. All the experiments and results are performed on the existing database and the results may be varied when this method is applied to an actual production environment. Other types of defects can be added to the system.
(a) Good (b) Excess
(c) Less (d) No solder (e) Bridge Fig. 2 Different types of solder joint appearance
The verification results are normally presented using DET plot. The DET plot has a logarithmic scale on both axes which makes it easier to observe the system contrast since the curves tend to be close to linear.
Fig. 3 DET plot for classification performance of good solder joints and defect solder joints.
Fig. 4 DET plot for classification performance of good solder joints and individual defect solder joints.
Fig 3 shows the DET plot for classification performance of good joints and defect solder joints and Fig 4 shows the DET plot for classification performance of good joints and individual defect types by using Log-Gabor filters. The Equal Error Rate (EER) is the location on the DET curve where the false alarm probability and miss probability are equal. From Fig 4, the EER for the classification of good joints versus excess joints is 3% and the good joints versus less joints is
0.1 0.2 0.5 1 2 5 10 20 40 0.1 0.2 0.5 1 2 5 10 20 40
False Alarm probability (in %)
M is s p ro b a b ili ty ( in % )
DET plot for the Gabor filter
0.1 0.2 0.5 1 2 5 10 20 40 0.1 0.2 0.5 1 2 5 10 20 40
False Alarm probability (in %)
M is s p ro b a b ili ty ( in % )
DET plot for the Gabor filter
Excess Less Nojoint Bridge
4.5%. Again in Fig 4, the EER for the good joints versus combination of four defect joints is 3% i.e. the recognition rate for the good joint is 97%. Fig 5 and Fig 6 show classification performance of good solder joints across defect solder joints by using Discrete Wavelet Transform. The overall EER of good solder joints compared to defect solder joints is nearly 5% which is slightly higher than the EER rate of Gabor filter. 10% of excess solder joints, 5% of less solder joints, 1.5% of no solder joints and 0.5% of bridge solder joint are incorrectly classified when the Discrete Wavelet Transform is applied. The EER of good solder joints across defect solder joints by using Discrete Cosine Transform is shown in Fig 7 and Fig 8. 5% of EER is achieved when good solder joints are compared to defects solder joints. The mis-classification rates of excess solder joints, less solder joints, no solder joints and bridge solder joints are 7%, 8%, 1% and 1% respectively. Table 1 is the summarised result for all DET curves in this experiment. The first column represents the recognition rate for each type of solder joint and the rest present EER.
Fig. 5 DET plot for classification performance of good solder joints and defect solder joints.
Fig. 6 DET plot for classification performance of good solder joints and individual defect solder joints.
Fig. 7 DET plot for classification performance of good solder joints and defect solder joints.
Fig. 8 DET plot for classification performance of good solder joints and individual defect solder joints
Recognition rate and Equal Error Rate Good Excess Less No
Joint Bridge Log Gabor Filter 97 3 4.5 1 0.5 Discrete Wavelet Transform 96 10 5 1.5 0.5 Discrete Cosine Transform 95 7 8 1 1
Table 1: Summary result for classification performance across three classifiers
This research proposed three techniques for the classification of solder joint defects. From the experimental results, the Log-Gabor filter bank has been shown to outperform the Discrete Wavelet Transform and the Discrete
0.1 0.2 0.5 1 2 5 10 20 40 0.1 0.2 0.5 1 2 5 10 20 40
False Alarm probability (in %)
M is s p ro b a b ili ty ( in % )
DET Plot for Wavelet transform
0.1 0.2 0.5 1 2 5 10 20 40 0.1 0.2 0.5 1 2 5 10 20 40
False Alarm probability (in %)
M is s p ro b a b ili ty ( in % )
DET Plot for Wavelet transform Excess Less Nojoint Bridge 0.1 0.2 0.5 1 2 5 10 20 40 0.1 0.2 0.5 1 2 5 10 20 40
False Alarm probability (in %)
M is s p ro b a b ili ty ( in % )
DET plot for DCT
0.1 0.2 0.5 1 2 5 10 20 40 0.1 0.2 0.5 1 2 5 10 20 40
False Alarm probability (in %)
M is s p ro b a b ili ty ( in % )
DET plot for DCT
Excess Less Nojoint Bridge
Cosine Transform. There are some limitation of the Discrete Wavelet Transform and the Discrete Cosine Transform. The Haar wavelet transform performs an average and difference on a pair of values and then shifts over by two values and calculates another average and difference on the next pair. Thus, it cannot detect if a big change takes place from an odd index value to an even index value. The blockwise Discrete Cosine Transform destroys the invariance properties of the system, because the blockwise frequencies do not produce a simple relation to the frequencies achieved by just transforming the image into the frequency domain. Hence, any linear scaling factor from the time domain will not carry over into the frequency domain if blocking is used because linearity is no longer maintained. For this reason, the performance degradation is caused for certain types of images like less solder joints and no solder joints.
V. CONCLUSION
This paper aims to provide a solution that can overcome some of the limitations of current inspection techniques. This research proposes classification algorithms for an automatic solder joint classification system. This involves the classification of solder joints by using the Log Gabor filter, Discrete Wavelet Transform and Discrete Cosine Transform. Five different levels of solder quality with respect to the amount of solder paste have been defined. The Log Gabor filter has been demonstrated to achieve high recognition rates and is resistant to misalignment. Further testing demonstrates the advantage of the Log Gabor filter over both the Discrete Wavelet Transform and Discrete Cosine Transform. From the experimental results, the Log-Gabor filter bank has been shown to outperform the Discrete Wavelet Transform and the Discrete Cosine Transform. The performance of the preliminary inspection algorithms is encouraging however some improvement can be expected from the current technologies. More robust features could be developed by focusing on the errors of the current algorithm and the use of multiple views of solder joints could improve classification performance.
REFERENCES
[1] H.H. Loh and M.S. Lu, Printed circuit board inspection using image
analysis. IEEE Transactions on Industry Applications, vol 35(2): p.
426-432.
[2] F.C. Yang, et al., Reconstructing the 3D solder paste surface model
using image processing and artificial neural network, in Proc of the
IEEE International Conference on Systems, Man and Cybernetics. [3] S.C. Lin, C.H. Chou, and C.H. Su, A development of visual inspection
system for surface mounted devices on printed circuit board, in Proc of
the The 33rd Annual Conference of the IEEE Industrial Electronics Society.
[4] N.S.S. Mar, C. Fookes, and P.K.D.V. YarLagadda, Design of automatic
vision-based inspection system for solder joint segmentation.
Achievements in Materials and Manufacturing Engineering, vol 34(2): p. 145-151.
[5] N.S.S. Mar, C. Fookes, and P.K.D.V. YarLagadda, Automatic solder
joint defect classification using the Log-Gabor filter. Advanced
Materials Research, vol 97-101: p. 2940-2943.
[6] N.S.S. Mar, P.K.D.V. Yarlagadda, and C. Fookes, Design and
development of automatic visual inspection system for PCB manufacturing. Robotics and Computer-Integrated Manufacturing, vol
27(5): p. 949-962.
[7] S.L. Bartlett, et al., Automatic solder joint inspection. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol 10(1): p. 31-43. [8] D.W. Capson and S.K. Eng, A tiered-color illumination approach for
machine inspection of solder joints. IEEE Transactions on Pattern
Analysis and Machine Intelligence, vol 10(3): p. 387-393.
[9] Z.S. Lee and L.R. Chin, Application of vision image cooperated with
multi-light sources to recognition of solder joints for PCB.
[10] J.S. Park and J.T. Tou, A solder joint inspection system for automated
printed circuit board manufacturing, in Proc of the IEEE International
Conference on Robotics and Automation.
[11] J.H. Kim, H.S. Cho, and S. Kim, Pattern classification of solder joint
images using a correlation neural network. Engineering Applications of
Artificial Intelligence, vol 9(6): p. 655-669.
[12] J.H. Kim and H.S. Cho, Neural network-based inspection of solder joints
using a circular illumination. Image and Vision Computing, vol 13(6): p.
479-490.
[13] T. Ong, Z. Samad, and M. Ratnam, Solder joint inspection with
multi-angle imaging and an artificial neural network. The International
Journal of Advanced Manufacturing Technology, vol 38(5): p. 455-462. [14] K.W. Ko and H.S. Cho, Solder joints inspection using a neural network
and fuzzy rule-based classification method. IEEE Transactions on
Electronics Packaging Manufacturing, vol 23(2): p. 93-103.
[15] G. Acciani, G. Brunetti, and G. Fornarelli, Application of neural
networks in optical inspection and classification of solder joints in surface mount technology. IEEE Transactions on Industrial Informatics,
vol 2(3): p. 200-209.
[16] G. Acciani, G. Brunetti, and G. Fornarelli, A multiple neural network
system to classify solder joints on integrated circuits. International
Journal of Computational Intelligence Research, vol 2(4): p. 337-348. [17] J. Cook, et al., Combined 2D/3D Face Recognition Using Log-Gabor
Templates, in Proc of the IEEE International Conference on Video and
Signal Based Surveillance.
[18] J. Daugman, Two-dimensional spectral analysis of cortical receptive
field profiles. Vision Research, vol 20(10): p. 847-856.
[19] J.A. Cook, A Decompositional Investigation of 3D Face Recognition. 2007, Queensland University of Technology: Brisbane, Australia.
