Role of Expert Judgement in Analysing Large Complex Reliability Data Sets

(1)

Expert Judgement and Big Data in Reliability

Role of Expert Judgement in Analysing Large

Complex Reliability Data Sets

(2)

1

Background to the problem

2

Empirical analysis of observational condition test data

3

Engineering judgement reasoning of the problem example

4

_{Building a predictive model for inference about failure}

(3)

Expert Judgement and Big Data in Reliability Background to the problem

Motivating engineering problem

Predict the time to failure of vehicles given information about engine condition

Our problem of interest today

Mining observational condition monitoring data to understand patterns and correlation structures

Compare empirical insights with engineering judgement of causal relationships

Use relevant to inform predictive model development

Condition monitoring observational data

Relate to fleet of like vehicles with irregular usage profile Engine oil samples taken at regular intervals

(4)

(5)

Data initially in 3 spreadsheets by vehicle type.

Dimensions are (437

×

41), (286

×

41) and (70

×

41).

The covariates are the same in all sheets but there are many

missing values and entire missing covariates in some sheets.

The data were messy and required initial cleansing.

Specific variables:

Distances travelled since oil change for each vehicle at each observation: km and motor hours.

Number of indicator particles in oil: cutting, sliding, fatigue, total, “large”.

Content of specific elements in oil: chromium, lead, copper, nickel, etc.

Other measures: kinematic viscosity, flash point, total alkilinity number.

(6)

Expert Judgement and Big Data in Reliability Empirical analysis of observational condition test data

km.tot km.oil mh.tot mh.oil viscosity flash alkalinity

fuel water glycol oxid nitrate sulphate antioxid

soot ferrum chromium lead copper tin aluminium

nickel silver silicon borium natrium magnes calcium

barium phosphor zinc molybdenum titanium vanadium part.tot

(7)

0 20 40 60 −1.0 −0.5 0.0 0.5 1.0 Correlation count

Variable 1

Variable 2

Correlation

Total km

Total motor hours

0.90 Fuel content

Flash point

-0.90

Zinc content

Calcium content

0.88 Chromium content

Ferrum content

0.86 Sulphate content

Nitrate content

0.71

(8)

km.tot km.oil mh.tot mh.oil viscosity flash alkalinity fuel oxid nitrate sulphate antioxid soot ferrum chromium lead copper aluminium nickel silver silicon borium natrium magnes calcium barium phosphor zinc molybdenum vanadium part.tot cut.part slid.part fat.part large.part

km.tot km.oil mh.tot mh.oil viscosity flash alkalinity fuel o_xid nitr

ate

sulphate antio

xid

soot ferr

um

chromium lead copper aluminium nick

el silv er silicon bor ium natr ium

magnes calcium bar

ium

phosphor zinc molybden

um v anadium par t.tot cut.par t slid.par t fat.par t large .par t −1.0 −0.5 0.0 0.5 1.0

(9)

Two algorithms to determine Bayesian network structure from

data: Growth-Shrink algorithm and Hill-Climbing algorithm.

The Growth-Shrink algorithm uses Markov Blankets to detect

the structure of the network.

Definition

A Markov Blanket

BL

(

X

) for some variable

X

∈

U

is a set of

variables which, for any further variables

Y

∈

U

−

BL

(

X

)

−

X

, has

the property that

X

,

Y

are independent given

BL

(

X

).

The Hill-Climbing algorithm works as follows:

1 _{Start from an empty, full or random network.}

2 _{Attempting every possible single-edge addition, removal, or}

reversal.

3 _{Choose the move which increases the “score” by the most.}

4 _{The process stops when there is no single-edge change that}

increases the “score”.

The score in question can either be “log-likelihood”, “AIC”, or

“BIC”.

(10)

(11)

Grow−shrink nr km.tot mh.tot viscosity flash alkalinity fuel oxid nitrate sulphate antioxid soot ferrum chromium lead copper aluminium nickel silver silicon borium natrium magnes calcium barium phosphor zinc molybdenum vanadium part.tot cut.part slid.part fat.part large.part Hill−climbing nr km.tot mh.tot viscosity flash alkalinity fuel oxid nitrate sulphate antioxid soot ferrum chromium lead copper aluminium nickel silver silicon borium natrium magnes calcium barium phosphor zinc molybdenum vanadium part.tot cut.part slid.part fat.part large.part

(12)

0 5 10 15 20 25 30 35 0 2 4 6 8 Scree Plot Dimension Eigen v alue ● ● ● ●● ● ●● ●● ● ● ●● ● ● ● ●_{● ● ● ●} ● ● ● ● ● ● ● ● ● ● ● ●● Grow−shrink

f1

f2

f3

f4

f5

f6

Factor descriptions:

Factor 1: chemical elements that reflect the primary materials of key dynamic (as in moving) engine parts.

Factor 2: operational exposure over both degradation paths of oil and engine condition.

Factor 3: operational exposure over degradation in confounded degradation path of oil and engine condition.

Factor 4: measures of oil quality.

Factor 5: chemical elements arising from other static. Factor 6: measures of oil morphology.

(13)

km.tot km.oil mh.tot mh.oil viscosity flash alkalinity fuel oxid nitrate sulphate antioxid soot ferrum chromium lead copper aluminium nickel silver silicon borium natrium magnes calcium barium phosphor zinc molybdenum vanadium part.tot cut.part slid.part fat.part large.part Grow−shrink algorithm

(14)

Engineering judgement reasoning of the problem example

Expert

Highly qualified (PhD) mechanical engineer with operational experience of vehicles

Process

Motivate and condition to provide engineering reasoning from cause to effect to establish relationships between engine condition and observables

Structure around key questions to build causal maps of reasoning using why and how laddering

E.g. if condition of engine is X then observation on test is Y (HOW manifest)

E.g. if observe Y on test then state of engine condition is X (WHY might see)

Consistency checking the causal reasoning

Real-time reasoning verification by dual questions and preliminary quantification

Multiple (3) stages of cross-questioning as facilitator learns

Limitations

Only one expert with knowledge domain of the operation, test and maintenance of vehicles

(15)

(16)

Expert Judgement and Big Data in Reliability Building a predictive model for inference about failure

We can model the degredation of the oil morphology through

the numbers of Cut, Slide and Fatigue particles.

Y

t,j,k

=

t

X

i=1

X

i,j,k

,

for vehicle

j

, measure

k

and exposure time point

t

.

Define

Zt

,j,k

=

log

Yt

,j,k

Y

t−1,j,k

d

j,k

(

t

)

−

d

j,k

(

t

)

,

where

dj

,k

(

t

) is the exposure of vehicle

j

at time point

t

.

Assume parametric forms

Z

t,j,k

∼

G

(1

, µ

j,k

)

,

µ

j,k

∼

IG

(

α

k

, β

k

)

.

(17)

Expert Judgement and Big Data in Reliability Building a predictive model for inference about failure

(18)

Expert Judgement and Big Data in Reliability Summary and next steps

Our problem

is a typical general asset management problem although presented for specific data

Our observational data

are real and typical of condition monitoring data structures although our frequency of observation is less than some other assets with real-time applications (e.g. wind farms)

Our judgemental data process

challenges the engineer to think through causal reasoning from true condition state to observable measures of condition

Our analysis

reveals insights into life and condition of vehicles that align with engineering expectations about first order dependencies highlights more subtle dependency structures that should provide information about engine condition

exposes the usual limitations of data mining without contextual knowledge e.g. sample number in blind analysis!

Our next step is

To use the insights gained to develop a predictive model grounded in the observations and the judgements

(19)

Expert Judgement and Big Data in Reliability Summary and next steps