The ‘normal’ ruleset

This section looks at the construction of the normal ruleset. The objective in this process was to develop rules which described the defects whilst making full use o f the uncertainty techniques. Since the aim of the research was to analyse the combined techniques of fuzzy logic and Dempster Shafer, the normal ruleset was required to contain implementations of these formalisms. The antecedents were represented in both fuzzy and crisp numbers, and the consequents were both individual and set propositions. This provided a mix of rules which could be used as a benchmark against which the other experimental rulesets could be measured.

The method used followed a simple plan, do, check, action (PDCA) cycle. It took the following form: P... look for defining characteristics

D code them using UMTs as framework

C implement and test them against sample data

Chapter 6 - The classification system

It was an iterative process and took 4 cycles before an adequate ruleset was defined. During the process the expert must be conscious of not trying to provide a ‘fix’ for the sample data only. The rules being added or amended should take into account the general properties required for a robust ruleset. The main benefit o f the PDCA cycle was that as rules were written, executed, analysed and then improvements suggested, the ruleset was evolving naturally. One of the difficulties of defining a ruleset was seeing in advance the effect a rule would have on the ruleset so the testing process allowed a visualisation o f how the rules interacted. Many of the PDCA cycles were run to ‘balance’ the ruleset.

6,2.2.1 Constructing the ruleset

The following steps summarise the actual procedure used to construct the ‘normal’ ruleset.

1. Samples were scanned using the inspection system to obtain the image data summarised in chapter 5. This provided both numerical data about the samples (what the machine could ‘see’), and the form of the defect types under the different lighting configurations. The ability to visualise what the defects looked like to the naked eye and to the inspection system helped distinguish the vastly different levels o f information available to make a classification.

2. The visual images alone were not enough to start to differentiate classes. The data sheets helped to capture the information in an alternative form. The measurements taken were: area, maximum length, maximum width, percentage area (compactness), and widthilength ratio. All these five measures were recorded in bright and dark field.

3. Since there was no obvious starting point one defect type was selected and defining characteristics were recorded. For example, using the visual information and the data sheets it could be seen that gels had no bright field signal. Hence a first characteristic became ‘no bright field signal then defect type gel’.

4. Step 3 was repeated for each o f the seven defect types. A rather arbitrary number of two characteristics per defect type was chosen during this first round o f definitions. As this process continued some characteristics were repeated. For instance ‘no bright field then gel or line’. This process resulted in the definition of nine rules with various multiple outcomes.

5. In many o f the rules expressions such as ‘%area is high’ had been used. The next stage was to define the fuzzy regions which described these expressions. For the purpose of this research simple ramp, triangular and trapezoidal fuzzy regions were defined.

6. Finally a confidence in the rule being true was assigned. This was quite arbitrary at this stage since no real statistical data was available (as a result of the limited defect data). The measures were approximately based upon: always true = 100%, nearly always = 90%, usually = 75%, on average = 50%, and sometimes = 25%.

Chapter 6 - The classification system

This first round of rule definition was only partially successful. Three from 7 defects were classified correctly, the other 4 were either incorrect or not specific enough. The rules were then refined by looking at sample camera data to identify which characteristics could be used to separate the existing defect types further. In some cases rules were modified, in others rules were added.

The general approach to designing a ruleset has 5 key tasks:

1. Identify at a global level the defining characteristics o f the defect. 2. Specify the measurements which highlight these properties. 3. Using this knowledge define the rules which imply defect types

4. Define the fuzzy regions which capture the measurements from the sample data.

5. Return to the global view (to take into account the fact that the sample data doesn’t include all cases o f the defect) and update the rules or fuzzy regions if required.

This process should lead to a ruleset which is accurate enough to classify all defects and robust enough to handle extreme cases of defects.

6.2,2,2 Difficulties

There were various difficulties in defining the ‘normal’ ruleset. The first problem was knowing where to start. Although only a few simple measurements were taken, a lot o f numbers are generated on the data sheets. As a consequence it was never quite apparent whether the rules being defined were base rules or just tweaks to segregate sample types. As mentioned previously it was not possible to see the impact o f the rules until they had been implemented.

An important characteristic of using Dempster Shafer was the rules being defined were all positive in nature. The expert only captured knowledge which was known^^. In order to ensure a completeness across all defect types the expert would need to do some form o f rule balancing. For example, if the defect type gel was in every consequent in the ruleset, the classifier would naturally choose gel.

Finally, a small sample set means little statistical data is available to define the rule confidence, and it is difficult to define specific rules. The expert must be aware that whilst the rules are based upon a sample of data they will actually be used on a different set. This process o f selecting features which can extrapolate for new instances o f defect type is difficult. Fibres are typically hard to extrapolate for since

A rule w hich specifies a negative property is still view ed as a positive rule since a positive fact about a defect is being stated. For example, i f no bright fie ld signal then type gel. So if a positive rule is constructed using known inform ation, this implies it would be difficult to define a negative rule since it w ould be constructed from unknown information.

Chapter 6 - The classification system

they come in such a wide variety of shapes and sizes. Writing rules which will take this into account is quite difficult.

6.2.23 The Rules

The ‘normal’ ruleset is listed below. The rule confidence, the antecedent and the consequent are all highlighted. In figure 6.1 these items are represented in the Rules [i] array as ruleConfidence,

fuzzyArgument and possibleDefects respectively.

1 -> 90% sure that if no B F then possible defects are { line small_gel l a r g e g e l }

2 -> 50% sure that if no DF then possible defects are

{ small_white dust large_fibre small_fibre }

3 -> 90% sure that if BF_C then possible defects are { large_fibre dust }

4 -> 90% sure that if D F _ C then possible defects are { large_gel }

5 75% sure that if B F _ A Wid t h / Length is SQUARE then possible defects are

(small_fish large_fish oval dust small_white l a r g e w h i t e }

6 75% sure that if DF_B then possible defects are { large_fish l a r g e g e l }

7 -> 75% sure that if D F _ A & !DF_B then possible defects are

{ small_fish large_white oval small_gel line }

8 -> 75% sure that if D F _ A W i d t h / Length is SQUARE then possible defects are

{ dust large_white oval small_fish large_fish s m a l l g e l large_gel }

9 90% sure that if D F _ A W i dth / Length is LONG then possible defects are

{ line }

1 0 90% sure that if D F _ A %AREA LOW then possible defects are

{ oval }

1 1 90% sure that if D F _ A & BF_A then possible defects are

{ small_fish large_fish oval large_white }

1 2 -> 100% sure that if N O SIGNAL then possible defects are { no_defect }

Chapter 6 - The classification system

13 75% sure that if BF_A %AREA HIGH

then possible defects are

{small_fibre large_fibre dust small_fish large_fish small_white large_white}

14 ^ 50% sure that if BF_A Width / Length is SQUARE & BF_A %A LOW

then possible defects are

{ small_fibre large_fibre }

15 -> 50% sure that if !BF_A Width / Length is SQUARE

then possible defects are

{ small fibre large fibre }

T h e fu z z y re g io n s in the a n te c e d e n ts fo r ru les 5, 8-1 0 , an d 13-15 are d e sc rib e d b elo w . T h e o th ers h av e n o t b een sh o w n sin ce th ey re p re se n t ju s t tru e o r false states.

In ru le s 5 a n d 8 th e ex p re ssio n ‘w id th / len g th is s q u a r e ’ is u sed to re p re se n t o b je c ts w h ich look sq u are (i.e. th e ir w id th an d length h av e sim ila r v alu es). T h e fu zzy re g io n fo r rules 5 a n d 8 are re p re se n te d in fig u re 6 .1 2 an d fig u re 6.13 re s p ectiv ely . T h e sm all im ag e in th e rig h t o f fig u re 6 .1 2 illu strates a ‘s q u a re ’ o b je c t. F ig u re 6 .1 2 also sh o w s the fu zzy reg io n fo r ‘no t w id th / len g th is s q u a re ’ in rule 15.

NOT

SQUARE

Degree Membership

(DofM) SQUARE NOT

SQUARE 0 0.3 0.7 1 1.5 2 2.5 3 A square object (5 pixels by 5 pixels) Length ' Width " Typical image from camera.

width / length = I

Width / Length

Fig. 6.12 Fuzzy Region for Bright Field Width/Length ratio with example image

F ig u re 6 .1 3 also sh o w s th e fu zzy re g io n fo r ru le 9 ‘w id th / length is lo n g ’. T h e im a g e in th e rig h t o f th is fig u re re p re se n ts a lo n g o b ject. I f w e in p u t th e v a lu e 0.2 (th e w id th / length v a lu e ) into th e fu zzy reg io n w e see th a t w e g et a z e ro v alu e fo r th e sq u are re g io n an d a v a lu e o f 1.0 fo r th e lo n g reg io n .

Chapter 6 - The classification system Degree Membership (DotM) LONG SQUARE 0 0.3 0.6 1,0 A long object Length Width

Typical image from camera, width / length = 0.2 Width / Length

Fig. 6.13 Fuzzy Region for Dark Field Width/Length ratio

F igure 6 .1 4 sh o w s th e fu z z y reg io n fo r ru le 10 ‘d ark field p e rc e n ta g e a re a is lo w ’. T h e im age to th e rig h t illu strates th e c o n c e p t o f % A re a low .

Example o f %area calculation 1 0.73 Degree LOW Membership (DofM) 0 0 0.1 0.2 0.3 0.4 0.5 Length Percentage Area

Fig. 6.14 Fuzzy Region for Dark Field % Area with example image

' Width " Typical image from camera.

%Area = area / (width / length) = 7/25

= 0.28

F ig u re 6 .15 sh o w s th e b rig h t field p e rc e n ta g e a re a fu zzy re g io n s u se d in rules 13 an d 14.

1 Degree LOW HIGH Membership (DofM) 0 0 0 2 n . 0.8 1.0 Percentage Area Fig. 6.15 Fuzzy R egion for Bright Field % Area

Chapter 6 - The classification system

6.2.2Â A sample output from the ^normaV ruleset using test data

The output from the classifier for a fish eye defect is shown in figure 6.16. The first line provides an index to the result. The next section highlights the input values used from the inspection system (see section 5). The following segment lists the values calculated for each rule in the normal ruleset. (section 6.1 describes how the values are calculated). The normalisation coefficient is shown to highlight how much conflicting evidence exists. The coefficient shown is calculated as one minus the belief in the empty set. (The empty set is a sum o f all the combinations where the sets being combined were mutually exclusive, i.e. in conflict). The closer the coefficient gets to zero the higher the conflict in the evidence.

The third segment shows the results of combining all the evidence from the 15 rules using Dempster’s rule of combination. The b.p.a. is the amount o f evidence assigned exactly to the proposition in the curly brackets. The values in square brackets are the belief and plausibility of the solution respectively. The most likely defect type according to Dempster Shafer is reported below the list of evidence. When analysing the results, the actual values in the list were used in conjunction with the ‘most likely’ output from the Dempster Shafer decision algorithm. This was a result o f the low confidence in the output for borderline cases where multiple defect types are proposed. By analysing the complete set of results (the b.p.a., belief and plausibility) manually the user can get a better ‘feel’ for the most likely defect type. The lack of a suitable automated decision criteria was a weakness of the Dempster Shafer implementation and is discussed further in chapter 8.

The next segment shows Smets pignistic transformation where the evidence from multiple outcomes is apportioned to singletons using the insufficient reasoning principle. BetP highlights the pignistic probability assigned to the singleton defect type. The defect with the highest BetP is chosen as the most likely result. The last segment shows Wesley’s decision criteria which takes into account the belief and plausibility of a singleton. Dec(.) represents the value assigned to individual defect types according to Wesley’s decision criteria. Again the decision making algorithm selects the singleton with the greatest value. Further discussion of the decision making criteria can be found in section 6.1.3.

Since the results described above are generated 350 times per experiment, the results are summarised as follows. A data sheet is created for each of the seven defect types. On this sheet a classification is recorded for each of the three methods for making decisions (Dempster Shafer, Smets and Wesley) for each of the 50 sample defect types. The seven data sheets are then summarised into one summary sheet which shows all the data collected in an experiment. One extra set o f parameters has also been included on these sheets and is explained below.

Chapter 6 - The classification system

**Defect Type 50-00-02 (type - sangle - scan number) - Fisheye

slice -> A B C BF area 5 0 0 BF width 5 0 0 BF length 5 0 0 BF %area 0.2 0 0 BF w / 1 1 0 0 **Camera data DF area 38 10 0 DF width 13 4 0 DF length 9 4 0 DF %area 0.33 0.63 0 DF w / 1 1.44 1 0 **Belief in rules: Belief for rule 1 is 0 B elief for rule 2 is 0 B elief for rule 3 is 0 Belief for rule 4 is 0 Belief for rule 5 is 0.75 Belief for rule 6 is 0.75 Belief for rule 7 is 0 Belief for rule 8 is 0.75 Belief for rule 9 is 0 Belief for rule 10 is 0.495 Belief for rule 11 is 0.9

Belief for rule 12 is 0 **Factor by which values are n o r m alised

Belief for rule 13 is 0 * * (1-belief in empty set)

Belief for rule 14 is 0.475

Belief for rule 15 is 0 N o r malisation coefficient = 0 330469

**Using Dempster's Combination rule we get the evidence list below.

bpa = 0 .20 [0.20,0.40] ---{ oval }

bpa = 0 .01 [0.98,1.00] ---{ fish oval dust large_white } (The square brackets

bpa = 0 .00 [0.98,1.00] ---{ fish white oval dust } show the belief

bpa = 0 .59 [0.59,0.80] ---{ large_fish } and plaus i b i l i t y

bpa = 0 .02 [0.60,0.80] ---{ l a r g e g e l large_fish } respectively)

bpa = 0 .00 [0.99,1.00] ---{ fish gel large_white oval dust } bpa = 0 .18 [0.96,1.00] ----{ fish large _white oval }

bpa = 0 .00 [0.00,0.00] ----{ fibre }

bpa = 0 .00 [1.00,1.00] ---{ omega } The most likely defect type is large_fish **Smets' evidence list (BetP is the pignistic probability)

BetP =-0.00 -- { line } BetP =-0.00 -- { no_defect } BetP =-0.00 -- { small_fibre } BetP =-0.00 -- { large_fibre } BetP =-0.00 -- { small_gel } BetP =-0.01 -- { large_gel } BetP =-0.00 -- { small_white } BetP = 0.05 -- { large_white } BetP =-0.00 -- { dust } BetP =-0.25 -- { oval } BetP =-0.64 -- { large_fish } BetP = 0.05 -- { small_fish }

S m e t 's mos t likely singleton is: large_fish BetP(.) = 0.64 **Wesley's evidence list (DEC(.) is Wesley's decision D E C ( . ]1= 1 . 0 0--- { omega }

D E C ( . ]\ - - 1 . 0 0 --- { fibre }

D E C (.] 0 . 9 6--- [ fish large_white oval } D E C (.] - 0 . 4 1 — { oval }

D E C (.] 0 . 9 9 — { fish gel large_white oval dust } D E C (.) 0 . 4 0 --- { large_gel large_fish }

D E C (. ) 0 . 3 9 — { large_fish }

DEC ( . ) 0 . 9 8 --- { fish white oval dust }

D E C (.) 0 . 9 7 — { fish oval dust large_white } W esley's most likely singleton is: large_fish 0.39

Fig. 6.16 L isting o f resu lts for Fish eye d efect using norm al ruleset.

Chapter 6 - The classification system

A c c o rd in g to th e p re v io u s d e fin itio n o f the d e fe c t ty p e s th e re w e re five cla sse d as p ro d u c in g d e fe c tiv e m a te ria l an d tw o c la sse d as c o n ta m in a tio n (w h ich co u ld la te r b e re m o v e d th e re fo re n o t a ffe c tin g q u a lity o f m a te ria l). T h is leads to fo u r d iffe re n t en d c la ssific a tio n s w h ic h are ty p ic a lly u sed: correct, incorrect, escape, an d overkill. T h e se shall be ex p la in e d fu rth e r b y w a y o f ex am p le. It sh o u ld be n o te d th at th e o p tio n to classify m a te ria l as g o o d h as n o t been in c lu d e d in th e resu lts. It w as a ssu m e d th a t th e b lo b