Summary

Figure 6.9 shows a flow diagram of the operations executed in the classification software. The input to the classification scheme is the camera data from the inspection system and the ruleset defined by the user. The output is a text file showing the results of the classification according to three different decision criteria.

The implementations discussed enabled the complete automation of the classification process. The source code for the main elements of the implementation are described in appendix C. The following section describes the definition of a complete ruleset.

6.2 Creating a classification scheme

This section discusses the definition of the ‘normal’ ruleset and the data used to construct it. The objective of the normal ruleset was to provide a benchmark from which the experimental rulesets (modified versions of the normal ruleset) could be analysed to test the performance of the uncertainty techniques. The purpose was not to construct a perfect ruleset for this application (or set o f training data), rather the aim was to define the kind of information used by an expert developing an on-line inspection system. The development process was deliberately stopped short of a perfect solution to the training data since one of the experiments was to try and improve the performance of the ruleset.

Chapter 6 - The classification system CLASSIFIER INSPECTION SYSTEM USER Antecedent Consequent Confidence in rule Normalise results Format results for user List of belief for

each rule Load

cam d a t a cam d a t a

Define a rule Add rule to

ruleset Add more rules to Rules [i] ? Decision Making using: Dempster Shafer Combine all beliefs into CeList Decision Making using: Wesley Generate frame of discernment (omega) Decision Making using: Smets

Output text file: Results.txt

Load ruleset

Using c a m _ d a ta fire each rule in

Rules[i]

Chapter 6 - The classification system

Section 6.2 works through the development of the normal ruleset and section 6.3 describes the experimental rulesets. As an introduction to both these sections the following paragraphs describe the sources o f information available for defining rules.

6.2.1 Sources of information

The normal ruleset was designed using two forms of information: training data and expert knowledge.

The ‘normal’ ruleset and the other experimental rulesets where then tested using a set of test data.

6.2.1.1 Training data

The training data was obtained from a batch of transparent plastic film with defect types marked by the suppliers. Five types o f defect and two types of contamination were selected. For brevity, defects and contamination shall be referred to as defect type. The difference between the two is that the former results in scrap product, since the defect is within the material, and the latter would not result in scrap product since the contamination is on top o f the material and could be removed without affecting the product^'. When discussing defects and not contamination the distinction shall be made clear.

Two types of contamination were chosen: dust and fibre. They were chosen since they are relatively easy to obtain and are representative of contamination types. They were not supplied by the manufacturers of the product being inspected, however, they were suitable for the experimental design since they resembled contamination which may have been found in a factory environment.

For each defect type four samples were scanned five times each, hence, 20 images per seven defect types were available as output from the inspection system. The four samples of each defect type were selected from the samples available and were representative of the defects in the material. Each sample was scanned five times to generate a natural variation in the same defect images available for classification. These images were used to construct the ruleset. Figure 6.10 shows an example data sheet obtained for defect type fish eye. The references (e.g. df_17_01) denote the sample used. For example, df_17_01 refers to the dark field image of defect type 17 (fish eye) sample number 1. Figure 6.11 shows a sample of the corresponding images. The data sheet shows the values calculated for each sample at the various thresholds (section 5 describes how the numbers were calculated).

6.2.1.2 Expert knowledge

The second source o f information used to define the normal ruleset was expert knowledge. For this application the information came fi"om two engineers with practical experience of installing inspection

Contam ination exists since it is not necessarily possible to rem ove it prior to inspection.

Chapter 6 - The classification system

systems. Knowledge about lighting configurations meant that properties of defects and contamination could be used to define rules.

h lsn in B right Meld

1 A 1 B C A B c A B C A B C A B

b f_ 1 7 _ 0 1 FISH

Area: 12 0 0 W idth 6 0 0 Length 3 0 0 %Area 0 .6 7 0 0 W /L 2 .0 0 0 Area: 15 0 0 W idth 8 0 0 Length 3 0 0 %Area 0 .6 3 0 0 W /L 2 .6 7 0 Area: 16 0 0 Width 9 0 0 Length 3 0 0 %Area 0 .5 9 0 0 W /L 3 .0 0 0 Area: 15 0 0 W idth 8 0 0 Length 3 0 0 %Area 0 .6 3 0 0 W /L 2 .6 7 0 Area: 12 0 0 Width 6 0 0 Length 3 0 0 %Area 0 .6 7 0 0 W /L 2 .0 0 0 b f_ 1 7 _ 0 2

Area: 2 0 0 W idth 1 0 0 Length 2 0 0 %Area 1 0 0 W /L 0 .5 0 Area: 2 0 0 W idth 1 0 0 Length 2 0 0 %Area 1 0 0 W /L 0 .5 0 Area: 1 0 0 Width 1 0 0 Length 1 0 0 %Area 1 0 0 W /L 1 0 Area: 2 0 0 Width 1 0 0 Length 2 0 0 %Area 1 0 0 W /L 0 .5 0 Area: 2 0 0 W idth 1 0 0 Length 2 0 0 %Area 1 0 0 W /L 0 .5 0 b f_ 1 7 _ 0 3

Area: 6 0 0 Width: 3 0 0 Length 2 0 0 %Area 1 0 0 W /L 1.5 0 Area: 7 0 0 Width: 3 0 0 Length 3 0 0 %Area 0 .7 7 8 0 0 W /L 1 0 Area: 8 0 0 Width: 3 0 0 Length 3 0 0 %Area 0 .8 8 9 0 0 W /L 1 0 Area: 8 0 0 Width: 3 0 0 Length 3 0 0 %Area 0 .8 8 9 0 0 W /L 1 0 Area: 7 0 0 Width: 3 0 0 Length 3 0 0 %Area 0 .7 7 8 0 0 W /L 1 0 b f _ 1 7 _ 0 4

Area: 5 0 0 Width: 2 0 0 Length 3 0 0 %Area 0 .8 3 3 0 0 W /L 0 .6 6 7 0 Area: 8 0 0 Width: 5 0 0 Length 3 0 0 %Area 0 .5 3 3 0 0 W /L 1 .6 6 7 0 Area: 8 0 0 Width: 5 0 0 Length 3 0 0 %Area 0 .5 3 3 0 0 W /L 1 .6 6 7 0 Area: 8 0 0 Width: 5 0 0 Length 3 0 0 %Area 0 .5 3 3 0 0 W /L 1 .6 6 7 0 Area: 7 0 0 Width: 4 0 0 Length 3 0 0 %Area 0 .5 8 3 0 0 W /L 1 .3 3 3 0

Fish in Dark Field

1 A 1 8 C A B c A B c A B c A B

d f_ 1 7 _ 0 1 FISH

Area: 4 0 10 0 Width: 7 5 0 Length 7 3 0 %Area 0 .8 1 6 0 .6 6 7 0 W /L 1 1 .6 6 7 Area: 41 9 0 Width: 7 5 0 Length 7 3 0 %Area 0 .8 3 7 0 .6 0 W /L 1 1 .6 6 7 Area: 4 2 8 0 Width: 7 5 0 Length 8 3 0 %Area 0 .7 5 0 .5 3 3 0 W /L 0 .8 7 5 1 .6 6 7 Area: 41 9 0 Width: 7 5 0 Length 8 4 0 %Area 0 .7 3 2 0 .4 5 0 W /L 0 .8 7 5 1 .2 5 Area: 4 2 8 0 Width: 7 5 0 Length 8 3 0 %Area 0 .7 5 0 .5 3 3 0 W /L 0 .8 7 5 1 .6 6 7 d f_ 1 7 _ 0 2

Area: 19 2 0 Width: 5 2 0 Length 5 1 0 %Area 0 .7 6 1 0 W /L 1 2 Area: 2 0 2 0 Width: 5 2 0 Length 5 1 0 %Area 0.8 1 0 W /L 1 2 Area: 21 2 0 Width: 5 2 0 Length 5 1 0 %Area 0 .8 4 1 0 W /L 1 2 Area: 2 3 2 0 Width: 5 2 0 Length 6 1 0 %Area 0 .7 6 7 1 0 W /L 0 .8 3 3 2 Area: 2 4 3 0 Width: 5 2 0 Length 6 2 0 %Area 0.8 0 .7 5 0 W /L 0 .8 3 3 1 d f_ 1 7 _ 0 3

Area: 1 4 0 0 Width: 4 0 0 Length 5 0 0 %Area 0 .7 0 0 W /L 0.8 0 Area: 1 4 0 0 Width: 4 0 0 Length 5 0 0 %Area 0 .7 0 0 W /L 0.8 0 Area: 12 0 0 Width: 4 0 0 Length 4 0 0 %Area 0 .7 5 0 0 W /L 1 0 Area: 13 0 0 Width: 4 0 0 Length 4 0 0 %Area 0 .8 1 3 0 0 W /L 1 0 Area: 14 0 0 Width: 4 0 0 Length 5 0 0 %Area 0 .7 0 0 W /L 0.8 0 d f _ 1 7 _ 0 4

Area: 3 3 3 0 Width: 17 1 0 Length 9 3 0 %Area 0 .2 1 6 1 0 W /L 1 .8 8 9 0 .3 3 3 Area: 3 4 3 0 Width: 17 1 0 Length 9 3 0 %Area 0 .2 2 2 1 0 W /L 1 .8 8 9 0 .3 3 3 Area: 31 2 0 Width: 13 1 0 Length 7 2 0 %Area 0 .341 1 0 W /L 1 .8 5 7 0 .5 Area: 3 0 2 0 Width: 5 1 0 Length 7 2 0 %Area 0 .8 5 7 1 0 W /L 0 .7 1 4 0 .5 Area: 34 3 0 Width: 14 1 0 Length 7 3 0 %Area 0 .3 4 7 1 0 W /L 2 0 .3 3 3

Chapter 6 - The classification system

F o r e x a m p le , th e tw o fo rm s o f lig h tin g c o n fig u ra tio n (d a rk fie ld an d b rig h t field , se c tio n 5 .2 .3 ) allo w th e e x tra c tio n o f sim p le ru les su ch as ‘i f light is b lo c k e d at all b rig h t field th re sh o ld s th en it is likely to be d u s t’. T h is k in d o f in fo rm atio n is in tu itiv e b e c a u se th e e x p e rt k n o w s th at d u st n o rm a lly b lo ck s light. E q u a lly th e e x p e rt m a y kn o w th at in th e d a rk field d u st m ig h t sc a tte r lig h t h o w e v e r it w ill n o t refract light in th e sa m e w a y as a d e fe c t w ith in th e m a te ria l, su ch as a g e l, m ig h t.

Fish 1701 Image 25*25 pixels Below: Bright Field A.B & C Left: contrast stretched raw image. (note: no B or C signal)

Fish 1701 Image 25*25 pixels Below: Dark Field A.B & C (note: no C slice signal)

Fig. 6.11 Thresholded images of fish eye defect from training data

T h e e x p e rt w ill also h av e h eu ristics w h ich c h a ra c te rise d e fe c t ty p es. F o r e x a m p le , lin es and fib res are u su a lly lo n g an d th in , g els are c o m p a c t w ith a ‘le n s ’ like stru c tu re , fish ey es are q u ite sim ila r in sh ap e h o w e v e r th e y te n d to h av e a d ark sp o t at th e ir ce n tre (h e n c e the ‘e y e ’ like a p p e a ra n c e ). A g ain th ese ru les o f th u m b sh o u ld be u sed to try an d c a p tu re w h a t th e e x p e rt k n o w s ab o u t th e d o m a in .

T h e e x p e rt w ill h a v e k n o w le d g e ab o u t th e p ro c e ss u sed to m a n u fa c tu re th e p ro d u ct. F acto ry e n v iro n m e n ts are su sc e p tib le to ce rta in k in d s o f c o n ta m in a tio n , so th e e x p e rt m a y be ab le to d escrib e ty p e s th a t o c c u r on a p a rtic u la r p ro d u c tio n line. F ib re s an d d u st are p a rtic u la rly c o m m o n since th ey are a re s u lt o f th e g e n e ra l e n v iro n m e n ta l c o n d itio n s. P eo p le are in v a ria b ly c o v e re d in fib re s fro m th e ir c lo th in g an d d u st can be fo u n d in even th e c lea n est facto ries. A n e n v iro n m e n ta l d e fe c t m a y o c c u r as a resu lt o f e q u ip m e n t fau lts. A n e x a m p le o f su ch a d e fe c t is a line. A g ain an e x p e rt m a y b e fa m ilia r w ith a ty p e o f d e fe c t w h ic h o c c u rs w h e n e v e r a p ie c e o f e q u ip m e n t n eed s re p la c in g . T h e y th e re fo re h av e h eu ristics ab o u t ty p e s o f d e fe c ts su ch as ‘a long thin re p e a tin g d e fe c t is p ro b a b ly a lin e ’.

Chapter 6 - The classification system

Finally the expert will also have knowledge about the product itself and data on causes of defects [such as lAL 1993a, b]. Most defects are caused by deviations from the normal in the production of the material. For example a gel is a local area o f resin with a different molecular weight and therefore different physical properties. These changes might mean the defect looks like a lens in the product which therefore shows up under the lighting configurations described previously. By knowing how the product deviates from the normal, the expert can express the information in heuristics.

Two sources of information have been described. The first was hard data which described what the camera could see. The second was the rule of thumb used by the expert to describe the defects. The two are connected in that they both describe the features of the defect. When constructing a classification scheme consideration must be given to both these sources since the expert’s heuristics will dictate what features should be measured (so that the expert’s knowledge can be expressed in the classification scheme) and the cameras capabilities (i.e. what features can be measured) will dictate how the experts express their heuristics.

6.2.1.3 Test data

The final source of information remaining is the test data. The test data was only used for testing the normal and experimental rulesets. It was taken from the same product as those for the training data and the same material and defect types were used. For the test data ten samples of each defect type were selected and each sample was scanned five times. Each o f the seven defect types therefore had fifty sample images. Each time a normal ruleset or experimental ruleset was executed a test set of 350 images were classified.