inspection of PCBA 2
Min-Chie Chiu∗
Department of Automatic Control Engineering 4
Chungchou Institute of Technology 6, Lane 2, Sec. 3, Shanchiao Rd. 6 Yuanlin, Changhua 51003 Taiwan, R.O.C. 8 Long-Jye Yeh Che-Jung Hsu 10
Department of Mechanical Engineering Tatung University
12
Taiwan, R.O.C. Abstract 14
Because of the vigorous growth in the electronic industry, the quantity and variety of products has risen enormously. In order to pursue profits, a strategy of cost-reduction is
16
necessary.Automatic optical inspection(AOI) has been widely used in the inspection process ofprinted circuit board assembles(PCBAs); however, unrecognized deficiencies still occur.
18
In order to increase precision in recognizing a PCBA’s deficiencies by using current inspection techniques, a huge quantity of samples used in off-line training is obligatory.
20
Unfortunately, it is not suitable for an industry which produces a variety of products with a smaller quantity.
22
Traditional AOI methods have been investigated and substantially tested in this paper. Results reveal that many image errors which haven’t been identified may raise maintenance
24
cost of PCBAs. To overcome the above drawbacks, a new and efficient algorithm, animage division method(IDM), is proposed. Consequently, the experimental results using the IDM
26
reveal that recognition efficiency can be improved.
Keywords and phrases :Machine version, image division method, AOI, PCBA, SMT. 28
∗E-mail:[email protected] ——————————–
Journal of Information & Optimization Sciences Vol. ( ), No. , pp. 1–18
c
1. Introduction
To improve life, many electronic products have been developed. And 2
now, the electronic industry is playing an essential role in the world. The printed circuit board assembly (PCBA) is one of the most important 4
components installed inside an electronic product. To promote quality and increase electronic performance, many electronic elements have been 6
installed onto the shape-minimized board; therefore, the surface mount technology(SMT) used to fasten and assemble an element onto the printed 8
circuit board’s surface becomes important. The SMT locates the surface mounting device(SMD) on the plannedprinted circuit board(PCB) in which 10
the solder is sprayed. Thereafter, the solder flows between the SMD and PCB.
12
Traditional inspection of the PCBA is performed by visual examina-tion followed by an electrical instrumental test; however, the inspecexamina-tion 14
is time-consuming, resulting in eye fatigue and leading to an inferior product. Therefore, the automatic optical inspection (AOI), which may 16
reduce labor expenses, improve inspection levels, and increase product quality, has been developed and widely used in the inspection process. 18
To successfully identify the deficient elements inside a PCBA, a superior inspection algorithm is important. To improve the inspection 20
efficiency with respect to the deficiencies, the AOI will be equipped with a different algorithm.
22
Teoh proposed the histogram method [1] which can establish a related chart between pixel-frequency and gray-value for a specified 24
image zone; moreover, a parameter index in calculating all the values of gray pixels within the zone will be summed up. The situation of 26
misalignment or a missing element will then be judged by the above parameter index. In Lin’s report [2], the identification work will be 28
maintained even though there is a change of light. However, the selection of the gray level will be decided in advance by the background color 30
of the PCBA’s missing element before the off-line training is performed. When the background color is similar to that of the electronic element, the 32
recognition work by the histogram method will be inefficient.
The white point statistic method is mainly used to identify the printed 34
character. In Teoh’s research [1], the above image was classified as two values (black and white) by a threshold value which is decided by a 36
white points, the deficiency on the opposite side of the element can be picked up. Because of the white color on the opposite side [2], the white 2
point statistic method is superior in identifying the deficiency of that side. However, precision will be decreased when the ratios of the white point 4
in the testing image are similar to that of the standard image (qualified image). Similarly, if there is a missing element, accuracy will also decrease. 6
To overcome this drawback, manipulating the image zone is required. Loh [3] proposes a run-length encoding method in which the image 8
is classified as two values (black and white) in a horizontal direction. By comparing the value between the standard and the testing image, the 10
corresponding deficiency can be examined; however, it is not easy to recognize if there is a slightly shifted or misaligned condition.
12
The projection method is often used to identify an inferior solder such as a solder bridge phenomena. By using a threshold value, the 14
integrated gray value along the horizontal or vertical image is judged for the deficiency of the solder bridge. The threshold value plays an essential 16
role which will tremendously influence precise inspection. This method is suitable for the inspection of the connector of asmall outline J-lead(SOJ); 18
however, it not suitable for both the resister and capacitor.
Pern proposed the coefficient correlation method [4] to recognize the 20
deficiency in the PCBA’s image. By comparing the averaged gray value and variation between standard image and testing image, calculating 22
their relationship, and determining a threshold, the deficiency can be distinguished. The coefficient correlation method is easy to use without 24
presetting a threshold value; in addition, precision will not be influenced by various testing images. However, it will be highly influenced when 26
light intensity is changed or misalignment occurs. The total gray error index method is proposed by Lin [2]. The total error index is calculated 28
by subtracting all the gray values of the testing image from the standard image and then summing up the absolute variation. The deficiency can be 30
distinguished by using the above index. This method has the advantage of not influencing the various images of the PCAB too much; however, the 32
PCBA image will be highly influenced when the light intensity is slightly changed or misalignment occurs.
34
The gray zone division/statistic method proposed by Lin [2] divides the gray zones of the standard image and testing image into five regions 36
chart to analyze the number of pixels with respect to the five regions, the bar with the biggest deviation will be selected as the characterized zone. 2
A selected threshold value is taken to evaluate the deficiency of the testing image. The gray zone division/statistic method is superior when the light 4
intensity is slightly changed or the location of image is slightly shifted [4]. The high gray variation/pixel ratio method (T1 method) is similar to 6
the total gray error index method [5]. The threshold (T1) is calculated by dividing the total gray error by the total image points and multiplying by 8
1.5. By investigating the number (N1) of pixels in which the gray value is greater than T1 , the new indicator used to identify the deficiency is then 10
obtained by dividing N1 by the total image points.
As investigated above, not all the deficiencies can be recognized by a 12
single algorithm. To overcome the above drawbacks, a new and efficient algorithm — animage division method(IDM) — is proposed in this paper. 14
2. Nomenclature
This paper is constructed on the basis of the following notations: 16
f the maximal common factor of an image’s length(m)and width (n).
Is(x,y) the gray value of the pixel(x,y)in the standard image. It(x,y) the gray value of the pixel(x,y)in the testing image.
En the total gray value’s variation between standard and testing images at thenth zone.
Emax the maximum total gray value’s variation between standard
and testing images for all zones.
µs the mean gray value of the standard image after re-division. σ2
s the variance of the standard image after re-division. µT the mean gray value of the testing image after re-division. σ2
T the variance of the testing image after re-division.
3. Classification of the deficiency in a PCBA 18
Because of technical improvements for semi-conductors, many elec-tronic elements attached to the PCBA are miniaturized; therefore, several 20
deficiencies often exist in a completed PCBA. In order to assure product quality after the re-flow of a PCBA, the AOI is used to find deficiencies 22
circuit board. The manufacturing process of the PCBA illustrated in Figure 1 includes a PCB loader, a printing machine, a mounting machine, 2
a re-flow, and a PCB un-loader.
4
Figure 1
The manufacturing process of a PCBA
The general deficiencies which occasionally occurred in an AOI are 6
classified as the following:
A. Wrong element: The misplacement of an electronic element in the PCBA 8
is possible during an incorrect assembly process which will result in a tremendous rise in cost.
10
B. Missing element: A missing electronic element caused by collision and vibration can happen during the assembly process. This will ruin the 12
PCBA’s performance. The related deficient images after the graying process are shown in Figure 2.
14
Figure 2
The deficiency of missing elements in a PCBA 16
C. Misalignment: Incorrect placement of an element will happen when the machine operation is not precise. The related deficient images after the 2
graying process are shown in Figure 3.
4
Figure 3
The deficiency of misalignment in a PCBA
D. Reverse: The influence of reverse is huge for the directional electronic 6
elements such as capacitor and integrated circuit.
E. Opposite side: If the bottom of the electronic element is turned up, it will 8
result in an incorrect performance in the PCBA. The related deficient images are shown in Figure 4.
10
Figure 4
The opposite deficiency in a PCBA 12
F. No-solder: The solder for the electronic element is insufficient or termi-nated when the soldering process is incomplete. The related deficient 14
Figure 5
The deficiency of no solder in a PCBA 2
G. Bridge: Over-soldering will happen when the soldering process is imperfect. This will result in an extra connection between electronic 4
elements. The related deficiency images after the graying process are shown in Figure 6.
6
Figure 6
The bridge deficiency (solder overflow) in a PCBA 8
4. AOI index in a PCBA
In order to evaluate the availability of the AOI algorithm, four 10
kinds of AOI indexes including (1) false-alarm rate, (2) fault-miss rate, (3) incorrect-flaw-classification rate, and (4) inspection time are consid-12
ered.
The false-alarm rate is an incorrect judgment that occurs by identify-14
ing a qualified product as an unqualified product. A higher false-alarm rate will increase a product’s inspection and maintenance load.
A fault-miss rate is a misjudgment that occurs by identifying the un-qualified product as the un-qualified product. The higher fault-miss rate will 2
influence the quality of the product; in addition, the unqualified product which has not been picked up will be sent to the next manufacturing 4
process which may consequently cause the product to be voided. This will lead to an increase in the cost of the product.
6
The incorrect flaw-classification rate is the miss-classification of a deficiency. For example, a missing deficiency is regarded as a misalign-8
ment. A higher incorrect-flaw-classification rate which happens because of an inappropriate inspection algorithm will lead to misjudgments about 10
the products deficiencies and highly influence an improvement strategy during the manufacturing and soldering process.
12
The inspection time in an AOI system is essential. The maximum allowable time is no more than the operation time of the previous equip-14
ment.
In this paper, the above AOI indexes in conjunction with various 16
algorithms are programmed by JAVA. 5. Image division method
18
When the ratio of the image deficiency to the full image is small enough, the gray value which is lower than that of the threshold value 20
will lead to an incorrect recognition. In order to improve this drawback, the image division method is adopted by dividing the testing image into 22
several regions. Subsequently, the individual image comparison for each region will be carried out to identify the deficiency by using the specified 24
threshold value. Obviously, the required inspection time will be increased if the number of regions increases. The f, a maximum common factor of 26
the image’s length (m) and width (n), is adopted for dividing the full image. Here, the standard image and inspection image are divided as and 28 respectively. E1= f
∑
x=0 f∑
y=0 |Is(x,y)−It(x,y)|, (1a) 30 E2= 2f∑
x=f+1 f∑
y=0 |Is(x,y)−It(x,y)|, (1b) En−1= m−f−1∑
x=(m−2f) n−1∑
y=(n−f) |Is(x,y)−It(x,y)|, (1c) 32En = m−1
∑
x=(m−f) n−1∑
y=(n−f) Is(x,y)−It(x,y)|. (1d) To find the location of the primary deficiency, the maximum total variation 2is selected.
Emax=max(E1,E2,E3, . . . ,En−1,En). (2) 4
Because of the deficiency located along the edge of the region, and in order to shift the deficiency to the center of the region, information 6
from the deficient center is required in advance. With the coordinates of the deficiency at the down/left corner and the upper/right corner 8
[(xmin,ymin) and (xmax,ymax)], the center (xc,yc) of the deficiency can be obtained. 10 (xc,yc) = µ xmax+xmin 2 , ymax+ymin 2 ¶ , (3)
where (xmax,ymax) is the corresponding (x,y) that causes the maximum
12
variation of (Is(x,y)−It(x,y))at the zone with Emax and (xmin,ymin) is
the corresponding (x,y) that causes the minimum variation of (Is(x,y)− 14
It(x,y)) at the zone with Emax
After shifting the center of the specified region to the center of the 16
primary deficiency, the mean gray values (µs,µT) and variances(σs2,σT2) with respect to both the standard and inspection images are calculated as 18 µs = 1 f ·f yc+f/2
∑
y=yc−f/2 xc+f/2∑
x=xc−f/2 Is(x,y), (4) σs2= 1 f· f yc+f/2∑
y=yc−f/2 xc+f/2∑
x=xc−f/2 [Is(x,y)−µs]2, (5) 20 µT= 1 f ·f yc+f/2∑
y=yc−f/2 xc+f/2∑
x=xc−f/2 It(x,y), (6) σT2= 1 f ·f yc+f/2∑
y=yc−f/2 xc+f/2∑
x=xc−f/2 [It(x,y)−µT]2. (7) 22By using Eqs. (4)-(7), a new index (I) for the IDM (image division method) is defined as 24 I= 1 f·f ¯ ¯ ¯ ¯ ∑yc+f/2 y=yc−f/2∑ xc+f/2 x=xc−f/2[Is(x,y)−µs]·[It(x,y)−µT] q σ2 sσT2 ¯ ¯ ¯ ¯, 0≤r≤1 . (8)
6. Results and discussion 6.1 Results
2
In this paper, two hundred inspection pictures used in a practical PCBA’s inspection process have been adopted. One hundred and seventy-4
four pictures are qualified: ten pictures are missing a component, eight are misaligned, and eight are on the opposite side. The related images and 6
various AOI algorithms can be obtained and assigned by the interface window programmed by JAVA run on a notebook (INTEL PENTIUM
8
1.5GHz&768MB RAM). The selected range of an image is 640×48 pixels. The flow diagram of the AOI is shown in Figure 7.
10
Figure 7
The flow diagram of an AOI in a PCBA 12
As indicated in Figure 7, both the standard image and inspection image are captured. The AOI system is initialized by starting the JAVA’s 14
interface window shown in Figure 8.
16
Figure 8
The start up of JAVA’s interface window of an AOI system in a PCBA. As indicated in Figure 9, the standard image will be put inside the left window. The testing image will be put inside the middle window. The discrepancy between these images will be shown in the right window
Figure 9
Loading the images in the JAVA’s interface window. To avoid the influence of light intensity, the transformation of color to a gray image shown in Figures 10 and 11 is required
2
Figure 10
The transformation of color to a gray image in the JAVA’s interface window of an AOI system in a PCBA (before a gray transformation). Figure 11 The transformation of color to a gray image in the JAVA’s interface window of an AOI system in a PCBA (after a gray transformation)
4
Figure 12
The selection of algorithm of an AOI algorithm 6
Figure 13
The result of an AOI algorithm by the coefficient correlation method 2
6.2 Discussion
After using various AOI algorithms, the related results with respect 4
to each algorithm are listed in Tables 1-6 and Figures 14-19, respectively. Table 1
Recognition result for the coefficient correlation method 6
Item Coefficient Pass Missing Opposite Misalign-number number component side ment
number number number
1 1∼0.9990 36 0 0 6 2 0.9989∼0.9980 66 1 4 1 3 0.9979∼0.9970 69 5 1 1 4 0.9969∼0.9960 2 3 1 0 5 0.9959∼0.9950 1 1 1 0 6 <0.9949 0 0 1 0 Table 2
Recognition result for the total gray error index method 8
Item Coefficient Pass Missing Opposite Misalign-number number component side ment
number number number
1 1∼0.010 0 1 0 0 2 0.011∼0.020 0 0 0 0 3 0.021∼0.030 22 3 6 8 4 0.031∼0.040 59 3 2 0 5 0.041∼0.050 43 1 0 0 6 0.051∼0.060 14 0 0 0 7 0.061∼0.070 15 0 0 0 8 0.071∼0.080 14 2 0 0 9 0.081∼0.090 2 0 0 0 10 >0.091 5 0 0 0
Table 3
Recognition result for the gray zone division/statistic method
Item Coefficient Pass Missing Opposite Misalign-number number component side ment
number number number
1 1∼0.010 0 1 1 0 2 0.011∼0.020 0 0 0 0 3 0.021∼0.030 57 5 7 8 4 0.031∼0.040 47 1 0 0 5 0.041∼0.050 34 1 0 0 6 0.051∼0.060 13 0 0 0 7 0.061∼0.070 16 2 0 0 8 0.071∼0.080 1 0 0 0 9 0.081∼0.090 1 0 0 0 10 >0.091 5 0 0 0 2 Table 4
Recognition result for the white point statistic method
Item Coefficient Pass Missing Opposite Misalign-number number component side ment
number number number
1 <0.999 0 2 4 0 2 0.9991∼1 48 1 0 6 3 1∼1.001 42 3 4 2 4 1.0011∼1.002 9 1 0 0 5 1.0021∼1.003 46 2 0 0 6 1.0031∼1.004 27 1 0 0 7 1.0041∼1.005 1 0 0 0 8 >1.005 1 0 0 0 4
As indicated in Table 1 and Figure 14, most of the qualified images have index values which are larger than 0.9970 when the coefficient 6
correlation method is used; therefore, an assumption is made that the image will be qualified when the index is larger than 0.9970. It is obvious 8
that the coefficient correlation method has a good effect on the false-alarm rate. However, three kinds of deficiencies are diversely allocated. The 10
effect of an flaw-classification rate is insufficient.
To proceed with the AOI inspection, an algorithm selection is re-12
quired. As indicated in Figure 12, the coefficient correlation method is selected. The result and mean gray value are shown in Figure 13.
Table 5
Recognition result for the high gray variation/pixel ratio method (T1 method)
Item Coefficient Pass Missing Opposite Misalign-number number component side ment
number number number
1 <0.1 20 1 1 0 2 0.1∼0.11 8 5 3 3 3 0.11∼0.12 6 1 0 5 4 0.12∼0.13 13 0 0 0 5 0.13∼0.14 20 0 0 0 6 014∼0.15 50 1 0 0 7 0.15∼0.16 47 2 1 0 8 >0.16 10 0 3 0 2 Table 6
Recognition result for the IDM
Item Coefficient Pass Missing Opposite Misalign-number number component side ment
number number number
1 1∼0.99 56 0 1 0 2 0.989∼0.98 15 2 1 4 3 0.979∼0.97 81 0 0 1 4 0.969∼0.96 22 1 1 1 5 0.959∼0.95 0 1 1 0 6 0.949∼0.94 0 1 0 0 7 0.939∼0.93 0 0 0 0 8 0.929∼0.92 0 1 0 2 9 0.919∼0.91 0 1 0 0 10 <0.91 0 3 4 0 4
As indicated in Table 2 and Figure 15, both the qualified and un-qualified images are diversely distributed along the index axis when the 6
total gray error index method is applied on an AOI. It is possible that the number of samples is insufficient.
8
As indicated in Table 3 and Figure 16, the character of the qualified image can be roughly identified when the gray zone division/statistic 10
method is used. However, the ability of the flaw-classification rate is insufficient because of the diverse distribution of the deficiencies on the 12
Figure 14
The result of an AOI by the coefficient correlation method 2
Figure 15
The result of an AOI by the total gray error index method 4
Figure 16
The result of an AOI by the gray zone division/statistic method 6
Figure 17
The result of an AOI by the white point statistic method 2
Figure 18
The result of an AOI by the high gray variation/pixel ratio method (T1 method)
4
Figure 19
The result of an AOI by the image division method 6
As indicated in Table 4 and Figure 17, the distinction between the qualified and unqualified images is not very clear when using the white 2
point statistic method. Moreover, the ability of the flaw-classification rate is insufficient because of the diverse distribution of the deficiencies on the 4
index’s axis.
As indicated in Table 5 and Figure 18, the deficiency of misalignment 6
is grouped at the index of 0.1∼0.12 when using the high gray varia-tion/pixel ratio method. However, the distinction between the qualified 8
and unqualified images is not clear. Therefore, the effect of the false-alarm rate is insufficient.
10
As indicated in Table 6 and Figure 19, most of the qualified images have index values which are larger than 0.96 when the IDM is used. It is 12
obvious that the effect of the false-alarm rate in the IDM is superior to the coefficient correlation method. Consequently, the image division method 14
proposed in this paper promotes the effectiveness of the false-alarm rate during the AOI process.
16
7. Conclusion
Five kinds of traditional AOI methods – the coefficient correlation 18
method, the total gray error index method, the gray zone division/statistic method, the white point statistic method, and the high gray varia-20
tion/pixel ratio method – have been substantially applied in the AOI process. The results reveal that only the coefficient correlation method is 22
effective in the false – alarm rate. When a new algorithm (IDM) is adopted in the AOI process, the experiment proves that its effectiveness in the false-24
alarm rate is superior to the coefficient correlation method. Consequently, the IDM proposed in this paper promotes the efficiency of the false – alarm 26
rate in an AOI system. References
28
[1] E. K. Teoh, D. P. Mital, B. W. Lee and L.K. Wee, Automated visual inspection of surface mount PCBs,Industrial Electronics Society, 16th 30
Annual Conference of IEEE, Vol. 1 (1) (1990), pp. 576–580.
[2] H. H. Loh and M. S. Lu, Printed circuit board inspection using image 32
analysis,IEEE Transactions on Industry Applications, Vol. 35 (2) (1999), pp. 426–432.
[3] K. Y. Pern,The PCBA’s AOI on Surface Mount Element by Computer’s Vision Technique, Master thesis, Chao-Tung University, Taiwan, 2000. 2
[4] Y. S. Lin, The Evaluation and Comparison of Various Machine Vision Algorithm Used in AOI of PCBA’ Surface Mount Element, Master thesis, 4
Tsing Hua University, Taiwan, 2003.
[5] D. L. Tsai, The Inspection of PCBA’ Surface Mount Element by Using 6
Machine Vision, Master thesis, Tsing Hua University, Taiwan, 2004. Received August, 2008
