Nondestructive testing is one of the important applications of machine vision. Nondestructive testing using radiation beyond the visible spectrum, such as infrared, ultraviolet and X-ray radiation, is described in other volumes of the
NDT Handbook and in the introductory chapter of this volume. This chapter focuses on the use of visible light in a machine vision system. In nondestructive testing, the purpose of a machine vision system is to capture and characterize anomalies of the object under inspection, that is, to inspect for structural and surface quality.29
Three types of results may be obtained from a machine vision system. The first type is an enhanced image in which the discontinuities are highlighted or
intuitively presented so that the inspector can easily make a subjective assessment. One example is the edge-of-light surface inspection technique, which uses the edge of light to highlight the surface slope or deformation.30Figure 24 shows the result of edge-of-light inspection of an aircraft lap joint. Figure 24a shows the lap joint; in the corresponding edge-of-light scan (Fig. 24b), bright and dark regions present the surface deformation. Such
deformation implies the potential hidden corrosion between the two layers.
A similar technique, double-pass retroreflection surface inspection, has also been applied to the same application.31,32 Figure 25 also shows the inspection result of aircraft lap joints. Figure 25a shows a
picture of the specimen whereas Fig. 25b is the double-pass retroreflection image. These two techniques implement enhanced visual inspection through the design of a special machine vision system.
Image processing techniques can also achieve an enhanced image to facilitate the inspection. A three-dimensional stereoscopic visual system has been built to inspect aircraft skin.33Algorithms to enhance monoscopic and stereoscopic images were developed. A high frequency emphasis algorithm consists of two steps as illustrated in Fig. 26.27The live image passed through low pass and high pass filters. Then, a fraction of low frequency content was added back to the high pass filtered image. This algorithm emphasized the high frequency features while
attenuating the background low pass information. Therefore, the potential surface flaws or cracks were highlighted.
For stereoscopic image enhancement (Fig. 26), the high frequency emphasis algorithm was applied to the left and right images of the stereoscopic image. An augmented stereoscopic, high frequency emphasis algorithm was implemented:
173
Machine Vision for Visual Testing
FIGURE24. Enhanced surface inspection of aircraft lap joint: (a) aircraft lap joint; (b) image resulting from edge-of-light technique.
(a)
(b)
FIGURE25. Enhanced surface inspection of
aircraft lap joint: (a) aircraft lap joint; (b) image resulting from double-pass retroreflection technique.
(a)
(b)
FIGURE26. High frequency emphasis algorithm.27
Live image High pass filter Low pass filter Fraction High frequency emphasized image
1. High frequency emphasis algorithms are applied to the left and right images.
2. To identify the features of interest, images are dynamically threshold filtered.
3. The original left and right images are overlaid with the depth offset desired for identified features.
4. The processed images are displayed stereoscopically on the screen. The eyewear of the inspector or operator can help highlight features of interest. The second type of result is binary — that is, crack or noncrack. Binary results are useful for the inspection of a specific part, where a binary accept/reject decision may follow. As described in one study,34a crack detection algorithm shown in Fig. 27 was developed to identify the surface cracks on aircraft skin. Cracks frequently happen near rivets; therefore, the first step is to detect rivets by detecting the circular arcs in the image.
Once the edge maps of the rivets are detected, the region of interest can be determined with the centroid of the rivet.
Once the region of interest is
identified, the multiscale edge detection is applied to the region of interest to generate a list of edges at different scales. This technique will help discriminate cracks from noncracks according to the size of a typical crack in comparison to other objects such as scratches and repair plates appearing on the surface. A coarse-to-fine edge linking process traced an edge from the coarse resolution (high scale) to a fine resolution (low scale). The propagation depth of all edges presented at scale one was found. Here, the propagation depth means the number of scales in which the edge appears. A feature vector for each edge in scale was generated so that the edges of cracks could be discriminated from those of noncracks. The feature vector includes the following: average wavelet magnitude of active pixels, which belong to the edges; the propagation depth number; average wavelet magnitudes of any linked edges in scale two and scale four; spins of sum WX and sum WY, where WX, WYare the wavelet coefficients in the x and y direction of an active pixel at scale one; and the number of active pixels.34
A neural network as shown in Fig. 28 was trained to classify the inputs — feature vectors of edges in the region of interest — into cracks and noncracks. The feature vectors used for the training may represent the cracks that need immediate repair. In this case, the classification result indicating a crack calls for further
investigation of the corresponding region of interest for repairing. An accuracy rate of 71.5 percent and a false alarm rate 27 percent for the neural network based classification were reported.
The third type is more informative, which allows quantitative information
174 Visual Testing
FIGURE27. Surface crack detection algorithm.
Image Rivet detection and region-of-interest identification Multiscale edge detection Edge linking Feature vector calculation Classification
FIGURE28. Neural network used for crack classification.
Input
Hidden layer
Output
FIGURE29. Pillowing deformation: (a) on
aircraft lap joints; (b) on hidden, faying surface.
(a)
about the discontinuity to be derived. For the application of aircraft inspection, corrosion detection is crucial to the risk assessment of structural integrity. One type of corrosion occurs on the interior, hidden surface of aircraft lap joints if sealant and corrosion protection systems break down. Corroded product is of much higher volume than the original material and this will cause an expansion of the skins between rivets. This phenomenon is known as pillowing. An example is shown in Fig. 29. Figure 29a shows an example of pillowing on a lap joint whereas Fig. 29b shows the corroded area on the faying surface. Another type of corrosion happens to the surface, which can be detected by its suggestive texture captured by a machine vision system. In a
procedure for surface corrosion detection,34the image was first
decomposed into subimages with a discrete wavelet transform. Figure 30 shows a three-level decomposition, which consists of ten subimages. Let Wj(k,l) be the wavelet coefficient at (k,l) in the subimage Wj. The original image was divided into nonoverlapping blocks each of 8 × 8 pixels. For each block B(i), a ten-dimensional feature vector was created. The element is Ej(i)(j = 1, …, 10), the corresponding energy function in subimages and can be expressed:
(37)
Then, a nearest neighbor classifier was trained to classify the original image into corrosion and corrosion free regions. A detection rate of 95 percent of the test set was reported.34Once the original image is classified, postprocessing can be carried out to calculate the corrosion area. Therefore, the information about the size of the corroded area is available.
A more general procedure is shown in Fig. 31. The image is first preprocessed for enhancement and noise removal so that the features of targeted objects
(discontinuities) are highlighted. The discrimination of different objects will be achieved in a feature space. Extraction of image features can be done in the spatial domain and frequency domain. There are numerous approaches available for this purpose. Some of these techniques have
E ij w k lj k l B i
( )
=( )
( )
∈( )
∑
, , 2 175Machine Vision for Visual Testing
FIGURE30. Wavelet decomposition of image: (a) three-level decomposition into ten images; (b) procedure for classification.
(a) (b) LL HL LH HH Image Wavelet transform Feature extraction Classification Postprocessing Result 1 2 3 4 5 7 6 8 10 9 Feature vectors HH = high high HL = high low LH = low high LL = low low
FIGURE31. Wavelet decomposition of image and procedure for classification.
Image Image preprocessing: · enhancement; · denoising; · segmentation; · others Feature extraction: · spatial domain; · transform domain Classification Postprocessing Result
been described in the previous section of this chapter. Sometimes, postprocessing is needed to further characterize the classified results as described in the example above. The measurement results can also be compared with calibrated samples for quantitative analysis. Such comparison can also be done in the feature space.
Conclusion
This chapter provides a general
description of machine vision techniques
for nondestructive testing. Both the system architecture and algorithm implementation for machine vision are described. A good understanding of the application’s requirements is essential to the success of a machine vision system. The technical advances in machine vision make it applicable to varied
nondestructive test applications. The capability of a machine vision system can be further expanded and enhanced by incorporating multiple image modalities or other nondestructive test techniques, which may provide complementary information.
1. Batchelor, B.G. and P. F. Whelan. Intelligent Vision Systems for Industry. Bruce G. Batchelor, Cardiff, United Kingdom; Paul F. Whelan, Dublin, Republic of Ireland (2002). 2. Zuech, N. “Smart Cameras vs.
PC-Based Machine Vision Systems.” Machine Vision Online. Ann Arbor, MI: Automated Imaging Association (October 2002).
3. Zuech, N. “Optics in Machine Vision Applications.” Machine Vision Online. Ann Arbor, MI: Automated Imaging Association (August 2005).
4. Fales, G. “Ten Lens Specifications You Must Know for Machine-Vision Optics.” Test and Measurement World. Web page. Waltham, MA: Reed Elsevier (27 October 2003).
5. “What is Structured Light?” Web page. Salem, NH: StockerYale (2009). 6. Casasent, D.[P.], Y.F. Cheu and
D. Clark. Chapter 4:
Part 4,
“Machine Vision Technology.” Nondestructive Testing Handbook, second edition: Vol. 8, Visual and Optical Testing. Columbus, Ohio: American Society of Nondestructive Testing (1993): p 92-107.7. Forsyth, D.A. and J. Ponce. Computer Vision: A Modern Approach. Upper Saddle River, NJ: Prentice Hall (2002). 8. Martin, D. Practical Guide to Machine
Vision Lighting. Web pages. Austin, TX: National Instruments Corporation (November 2008).
9. Hainaut, O.R. “Basic Image Processing.” Web pages. Santiago, Chile: European Organisation for Astronomical Research in the Southern Hemisphere, European Southern Observatory (December 1996). 10. Users Manual MTD/PS-0218, Kodak
Image Sensors. Revision 2.0. Rochester, NY: Eastman Kodak (July 2008). 11. Peterson, C. “How It Works: The
Charged-Coupled Device, or CCD.” Journal of Young Investigators. Vol. 3, No. 1. Durham, NC: Journal of Young Investigators, Incorporated (March 2001).
12. Litwiller, D. “CCD vs. CMOS: Facts and Fiction.” Photonics Spectra. Vol. 35, No. 1. Pittsfield, MA: Laurin
Publishing (January 2001): p 154-158.
13. Charge Injection Device Research at RIT. Web site. Rochester, NY: Rochester Institute of Technology, Center for Imaging Science (2009).
14. Trucco, E. and A. Verri. Introductory Techniques for 3-D Computer Vision. Upper Saddle River, NJ :Prentice Hall (1998).
15. Bouguet, J.-Y. Camera Calibration Toolbox for Matlab. Web pages. Pasadena, CA: California Institute of Technology (2009).
16. Wang, J., F. Shi, J. Zhang and Y. Liu. “A New Calibration Model of Camera Lens Distortion.” Pattern Recognition. Vol. 41, No. 2. Amsterdam,
Netherlands: Elsevier, for Pattern Recognition Society (February 2008): p 607-615.
17. IEEE 1394, High-Performance Serial Bus. New York, NY: IEEE (2008).
18. Wilson, A. “Camera Connections.” Vision Systems Design. Tulsa, OK: PennWell Corporation (April 2008). 19. Specifications of the Camera Link
Interface Standard for Digital Cameras and Frame Grabbers. Ann Arbor, MI: Automated Imaging Association (Annex D, 2007).
20. “Universal Serial Bus,” Web site. Beaverton, OR: USB Implementers Forum (2008).
21. “GigE Vision®.” Web page. Machine Vision Online. Ann Arbor, MI: Automated Imaging Association (2009).
22. 1394 Technology. Web page. Southlake, TX: 1394 Trade Association (2008). 23. Sgro, J. “USB Advantages Offset Other
Interfaces.” Vision Systems Design. Tulsa, OK: PennWell Corporation (September 2003).
24. Jain, R., R. Kasturi and B.G. Schunck. Machine Vision. New York, NY: McGraw-Hill (1995).
25. Gonzalez, R.C. and R.E. Woods. Digital Image Processing: Upper Saddle River, NJ: Prentice Hall (2002).
26. Otsu, N. “A Threshold Selection Method from Gray-Level Histograms.” SMC-9, IEEE Transactions on Systems, Man, and Cybernetics. Vol. 9, No. 1. New York, NY: IEEE
(January 1979): p 62-66.
177
Machine Vision for Visual Testing
27. Wada, T., T. Fujii and T. Matsuyama. “γ - ω Hough Transform — Linearizing Voting Curves in an Unbiased ρ - θ Parameter Space” [in Japanese]. IEICE D-II. Vol. J75, No. 1. Minato-ku, Tokyo, Japan: Institute of Electronics, Information and Communication Engineers (1992): p 21-30.
28. T. Wada, M. Seki and T. Matsuyama, “High Precision γ - ω Hough Transform Algorithm to Detect Arbitrary Digital Lines” [in Japanese]. IEICE D-II. Vol. J77, No. 3. Minato-ku, Tokyo, Japan: Institute of Electronics, Information and Communication Engineers (1994): p 529-530.
29. Malamas, E.N., E.G.M. Petrakis, M.E. Zervakis, L. Petit and J.D. Legat. “A Survey on Industrial Vision Systems, Applications and Tools.” Image and Vision Computing. Vol. 21, No. 2. Amsterdam, Netherlands: Elsevier (February 2003): p 171-188. 30. Komorowski, J.P. and D.S. Forsyth.
“The Role of Enhanced Visual Inspection in the New Strategy for Corrosion Management.” Aircraft Engineering and Aerospace Technology. Vol. 72, No. 1. Bingley, United Kingdom: Emerald Group Publishing (2000): p 5-13.
31. Komorowski, J.P., N.C. Bellinger, R.W. Gould , A. Marincak and R. Reynolds. “Quantification of Corrosion in Aircraft Structures with Double Pass Retroreflection.” Canadian Aeronautics and Space Journal. Vol. 42, No. 2. Kanata, Ontario, Canada: Canadian Aeronautics and Space Institute (1996): p 76-82.
32. Komorowski, J.P., S. Krishnakumar, R.W. Gould, N.C. Bellinger, F. Karpala, O.L. Hageniers. “Double Pass
Retroreflection for Corrosion Detection in Aircraft Structures.” Materials Evaluation. Vol. 54, No. 1. Columbus, OH: American Society for Nondestructive Testing (January 1996): p 80-86.
33. Siegel, M. and P. Gunatilake. “Remote Inspection Technologies for Aircraft Skin Inspection.” ETVSIM’97, IEEE Workshop on Emergent Technologies and Virtual Systems for Instrumentation and Measurement [Niagara Falls, Ontario, Canada, May 1997]. New York, NY: IEEE (1997): p 1-10.
34. Gunatilake, P., M.W. Siegel,
A.G. Jordan and G.W. Podnar. “Image Understanding Algorithms for Remote Visual Inspection of Aircraft Surface.” Machine Vision Applications in Industrial Inspection V [Orlando, FL, April 1997]. Proceedings Vol. 3029. Bellingham, WA: International Society for Optical Engineering (Society of Photo-Optical Instrumentation Engineers)
(1997): p 2-13.
8
Portions of Part 6 are reprinted with permission from Donald J. Wulpi, Understanding How Components Fail, © 1985, ASM International,
Materials Park, OH. Reprinted with permission. Reference number9superscripted in headings indicates sections adapted. ASNT has revised the
text in 1993 and 2010, and deficiencies are not the responsibility of ASM International.