Interlaboratory studies

Interlaboratory studies

Vladimír Kocourek

Prague, 2012

Various titles:

 Interlaboratory proficiency test or studies

 Interlaboratory comparisons

 Round test, round robin tests

 Test of qualification

 Certification study

 Interlaboratory validation

P

1

2

3

4

5

6

P

1

2

3

4

5

6

„Non-destructive tests“

Sample circulates between

laboratories

„Destructive tests“

Parallel distribution of samples

Interlaboratory studies

Importance of interlaboratory comparisons:

Successful participation in an interlaboratory comparison is one of the

necessary requirements of a laboratory accreditation or authorisation !!!

Interlaboratory comparison:

Interlaboratory comparison often consists in inter-comparison of

measurement results of a laboratory and a reference (target) value or

reference laboratory which is the highest authority of particular country

for the measurement concerned. The goal of such interlaboratory

comparison is to verify the competence of accredited or non-accredited

laboratories, including verification of the reported measurement

Interlaboratory studies

Before you get started you should ask and answer the following questions:



What am I testing ?



Why do I test ?



How do I test ?



How will the workflow be organized ?



What happens with the results ?

What am I testing ?



What is the commodity/matrix I have to test?



What is the analyte / substance I have to test for?

Interlaboratory studies

quantitative exact concentration information only YES/NO answer at a certain level approximative concentration levels qualitative semi-quantitative What kind of result is needed?



Why do I test ?



How do I test ?

TYPES:

Proficiency

testing

Validation of

methods

Materials

certification

Assessing: Laboratory performance Analytical methods performance Reference quantity values

Number of Labs. not-specified 8 and more (min.5) not-specified

Experience of Labs. Various Only experienced Only experienced

Analytical method Various Strictly specified Various

Influence on lab. confidence

Significant Low Low

Tracebility Not required Required Extremely

important

Target quantity X, , (z) r, R X 

Protocols: ISO 17043:2010 ISO 5725-2 ISO Guide 35:2006 34:2000

Reference materials

o Certified

o In-house

o Proficiency Scheme samples Usage for:

Analytical quality control Method validation

Calibration

“Material or substance one or more of whose property values are sufficiently homogenous and well established to be used for calibration of an apparatus or assessment of a measurement methods” - ISO Guide No. 30 1992

-controls or standards used to check the quality and traceability of products - a reference standard for a unit of measurement is an artifact that embodies the quantity of interest in a way that ties its value to the reference base for calibration.

- the most of analytical instrumentation is comparative, it requires a sample of known composition (reference material) for accurate calibration.

- reference materials are produced under stringent manufacturing procedures and differ from laboratory reagents in their certification and the traceability of the data provided.

- Quality management systems involving laboratory accreditation under national and international accreditation/certification standards such as ISO 9000 and ISO 17025 require the use of Reference Materials.

- Whilst Certified Reference Materials are preferred, their availability is limited. The available Reference Materials generally differ only in the detail provided on the certificate.

Preparation of reference materials

Material is prepared with desired characteristic

Homogeneity Stability

Certified through a round robin study to establish

Reference (consensus) value

μ

uncertainty of values

u

or U or confidence interval (95 %)

• Usage of CRM’s improve the reliability lab by means of verifying the accuracy and precision of the lab/method

• Many matrix/analytes combinations available - does not cover every combination

• Expensive and limited supply

• Not relevant where either matrix or analytes are

• Caution must be exercised when interpreting the results

Reference materials – commercially available

Properties:

o Stable

o Homogenous - to allow sub-sampling without bias

o Large amounts allow use over a long period

o Over 100 producers of CRM’s worldwide

o LGC/Promochem in the UK

o BCR/IRRM at the EU level

o Coordinated exercise to prepare CRM’s o Certified o In-house

The primary purpose of proficiency testing is to help

laboratories detect and cure any unacceptably

large inaccuracy in their reported results.

THE INTERNATIONAL HARMONIZED PROTOCOL FOR THE PROFICIENCY TESTING OF ANALYTICAL CHEMISTRY LABORATORIES

(IUPAC Technical Report).

MICHAEL THOMPSON, STEPHEN L. R. ELLISON, AND ROGER WOOD: Pure Appl. Chem., Vol. 78, No. 1, pp. 145–196, 2006.

Purpose of proficiency tests

 evaluation of the performance of laboratories for specific tests and monitoring laboratory performance,

 identification of problems in laboratories and initiation of action for improvement of which,

 establishment of the effectiveness and comparability of test methods,  provision of conffidence to laboratory customers and lab management,  education of participating laboratories based on the outcomes of

comparisons,

 validation of uncertainty claims,

 external quality control activities required by accreditation bodies and/or authorities,

Assessment & Reporting

of results

Report distribution

identification of non-conformities (by labs)

Preparation & verification of

testing material

Distribution of testing

material to participants

Analysis of testing

material by participants

Treatment

of results

Proficiency Testing for Food Chemistry

Individual value for each result:

x = value (concentration) in testing material

reported by laboratory

X = assigned value (considered as true)



_p

= target value of standard deviation (reproducibility)

z-score

p

X

x

z



ˆ





Assessments of results

| z | ≤ 2 Satisfactory

2 < | z | ≤ 3

Questionable (95 %)

| z | > 3 Unsatisfactory (99 %)

Unsatisfactory – FP !

Unsatisfactory– FN !

GOOD !

GOOD !

Questionable – FN ?

Questionable – FP ?

-4

-3

-2

-1

0

1

2

3

4

z-Scores

Assessments of z-score

Proficiency Testing for Food Chemistry

FAPAS

®

_{- Food Analysis Performance Assessment Scheme}

Provider: FERA, UK (

http://www.fapas.com/

):

 Nutritional components

 Food ingredients

 Natural food contaminants

 Organic & inorganic contaminants

 Pesticides and Veterinary drug residues

 Food additives

FAPAS proficiency tests

FAPAS

®

_{- Food Analysis Performance Assessment}

Scheme,

www.fapas.com

Report:

-

Assigned value

-

Results of all laboratories

-

Z-score

of all laboratories

-

Used methods

PT results evaluation - statistics

1. Assessment of assigned value (X

_ref

)

known (reference) value – addition of std., CRM,…

median of all results

mean of results

mean value of reference (expert) laboratories

mean value of experienced laboratories using

standard testing method (ISO, EN, DIN…)

0 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9

Histogram with two appex, i.e. low frequencies inside:

Data population is non-homogeneous i.e.

affected mostly by different analytical methods with different bias.

Assessment: various metods

Fig.. 1 - MTZL V/6

z-score ARSENIC (Xref = 0.160 mg/kg)

-4 -2 0 2 4 7 2 8 13 11 14 5 9 1 12 10 4 3 6 Laboratory No. z-sc or e 0,227 0,160 0,093 Fig. 1 - MTZL V/6

z-score ARSENIC ( Xref = 0.116 mg/kg)

-4 -2 0 2 4 7 2 8 13 11 14 5 9 1 12 10 4 3 6 Laboratory No. z-s c o re 0,116

Mean = 0,116

Median = 0,160

Assessment of assigned value (X

_ref

)

Uncertainty of assigned value - possibilities

p…number of expert laboratories u_i …standard deviation of i-laboratory

1a.

X

_ref

is established of median of all results, when

each of laboratory is giving its own uncertainty

(ISO 13528:2005)

n

u

X rob





1b.

X

_ref

is established as mean of values, uncrtainties

are not taken into consideration

(



_p assigned value for PT)

ISO 13528:2005

Guidelines for limiting the standard uncertainty of the assigned value:

Than is uncertainty of results small and cannot be taken into

acconsideration.

In case:

u

X

 3

0

.





p

When the number of participants is smaller than about 15, even the statistical uncertainty on the consensus (as the standard deviation) will be undesirably high. (THOMPSON M., ELLISON S.L.R., WOOD R.: Pure Appl. Chem., Vol. 78, No. 1, 2006, p. 145)

General criteria – precision and trueness

AOAC/FAO/IAEA/IUPAC:

Guidelines for single-laboratory validation of analytical methods for

trace-level concentrations of organic chemicals.

Examples of target values



(as RSD %):

Analyte Conc.

level

 (%)

Source

Ochratoxin A

3 g/kg

38 Horwitz

Vitamin B

2

24 mg/kg

20

Standard

(ISO)

Nitrates 150

mg/kg

7,5

Horwitz

Cyclamate 195

mg/kg 3,1 Standard

(ISO)

pH 3.25

1,5

Standard

(CSN)

ethanol 11.48

%

0,87

Standard

(CSN)

Evaluation of results in PT

p

X

x

z



ˆ





„Individual“ z-score

z

 <-2;+2>

„Do not set up any „social ladder“ of laboratories based on z-scores within -1; +1“

Combined z-score: weighted sum

1) assign the index ω lZl to each z-score for individual analytes in this way:

2) Calculate WSZ as the average of all ωlZl values :

z

 <-2;+2>

n

z

RSZ

i i

/





2





i i

z

SSZ

R

escaled

S

um of

Z

-scores,

sensitive to systematic errors / trends

S

um of

S

quared

Z

-scores,

sensitive to random errors and outliers

Combined z-score

Participation of official control laboratories

ICT Prague

3rd best results in Europe

Proficiency tests organized by Community Reference Laboratories - EUPT (European Commission’s Proficiency Testing Program )

„Fit For Purpose“ assessment of uncertainty

2 2

ˆ

X

u

u

X

x

x









2 2

ˆ

X x n

U

U

X

x

E







zeta-score: z-score reflecting differences related to standard uncertainty reported by individual laboratory

L

X

x

z

L



ˆ





zL-score specific to fit individual laboratory criteria

(e.g.



_L = 20% as RSD)

E_n-score: zeta-score calculated using expanded uncertainty

z

_L

 <-2;+2>

zeta  <-2;+2>

FAPAS



_{Series 19, Round 37}

Matrix: lemons

Analyt: fenitrothion

z-skore: -0.3

FAPAS

 Series 19, Round 35

Matrix: lettuce Analyte: lambda-cyhalotrin z-skore: -2,1 Lab. code: 075

Eluční profil pesticidů na GPC (PL gel [60 x 0,2 cm, 10 um]) jímaná frakce 14.5- 31 ml -10% 10% 30% 50% 70% 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 frakce [ml] re la ti vn í o d ez va

difenylamin metalaxyl fenitrothion-NPD cyhalothrin-lambda koextrakty

FAPAS

 Series 19, Round 32

„Baby Food“ test

 identification

of pesticides on trace levels

MRL = 10

g/kg

Matrix: carrot puree

EU – PT 07: Incurred residues of pesticides in grapes

homogenate (organized by CRL for pesticides)

1. Individual z-skore (z-skore + FFP skore)

Laboratories assessment:

5. Weigh sum z-skore

3. Number of good identified analytes (TP)

2. Number of analytes in methods

Documents:

THE INTERNATIONAL HARMONIZED PROTOCOL FOR THE PROFICIENCY TESTING OF ANALYTICAL CHEMISTRY LABORATORIES (IUPAC Technical Report).

MICHAEL THOMPSON, STEPHEN L. R. ELLISON, AND ROGER WOOD: Pure Appl. Chem., Vol. 78, No. 1, pp. 145–196, 2006.

ISO/IEC 17043:2010

General requirements for preparation and distribution

suggestion, selection of materials, methods and procedures, Performance and evaluation,

reports, communication with participants, conffidence, security, Requirements for management,

Thank you for your

attention…