Third Party Data Validation: What you need to know and do!

(1)

Third Party Data Validation:

What you need to know and do!

Anand Mudambi & Pamela Wehrmann

2012 DoD EMDQ Workshop San Diego, CA

(2)

Course Objectives/Outline

• The course will provide an overview of third party data verification and

validation for EPA and DoD Environmental Projects

• It will cover many aspects including:

• Data requested from laboratories

• Stages of review that can be performed and their uses

• Communicating the level of data validation and its benefits

(3)

Third Party (External Organizations)

• Third Parties are defined as

organizations (including Governmental entities, contractors, or vendors) that conduct analytical data review,

verification, and validation activities

and that are not part of the immediate laboratory that generates the subject analytical data (but that are part of the overall project-specific data review

process).

(4)

Course Overview

• Why Verify/Validate Data?

• Project Planning & Documentation

• Validation scenarios (how much is enough?)

• Laboratory Analytical Data Packages

• Labeling Validated Data

• Benefits of Labeling & Examples

• However!!!

• Won’t teach the how of data validation!

(5)

Why Verify / Validate Data?

• Additional resources are used by DoD/EPA to review (i.e., verify and validate) laboratory analytical data packages

• Reviews are conducted to ensure adequate quality and usability for site decision making

• Often different procedures used to evaluate laboratory data quality across programs

• Goal for this course is to encourage use of

consistent terminology to describe the scope and content of validation for data users.

(6)

Verification & Validation - Definitions

• Analytical data verification generally consists of a completeness check to confirm that all data

requested from the laboratory have been received and comply with specified requirements.

• Analytical data validation generally consists of an analyte and sample specific process for evaluating compliance of the laboratory data received with methods, procedures or contract requirements.

• Data review – commonly used for any combination of the above. Not limited to analytical data but

should include items like sampling documentation and field analytical. (UFP-QAPP Manual section 5).

(7)

Verification & Validation – Where to Start

• Project Planning Process:

• DQOs;

• Assessment criteria (regulatory levels (MCLs);

screening criteria(RSLs)or eco risk assessment

• Sampling strategy

• Methods; DQIs; LOD/LOQ;

• Type of data needed from lab

• Level of data verification and/or validation

• Where to document data validation planning? Use UFP-QAPP WS# 34-36

(8)

UFP-QAPP Manual – Data Review Process

(9)

Project Planning

• Establish validation needs and procedures based on the project DQOs and phase of the project;

• Verification and validation procedures should address the following:

• Process used to verify sample collection, handling, field analysis etc.

• Validation scenario for project laboratory data.

• Must be documented in the QAPP to ensure that data are evaluated correctly, completely, and consistently;

• Should be used to determine analytical data needs from the lab (contract requirements).

(10)

Criteria to Guide Project Planning

• Level of risk associated with the target analytes/COCs at the site (not always known in the planning stage).

• Cost and schedule demands of the overall project

• The specific decisions the data is

supporting (initial screening (SI) or risk assessment (RA)

• Complexity of analysis (a more

comprehensive validation may be

appropriate for highly complex analyses).

March 2012 Third Party Data Validation - Training 10

(11)

• Ability to identify critical/significant) samples and focus data review on those samples.

• Project-specific audits suggest that data quality problems exist or alternatively the lab is performing high-quality

work (long term monitoring).

• Sampling events that include recurring samples (i.e., long-term monitoring of the same chemicals could reduce validation for these events).

• Proximity of results to action levels.

• levels that are close to action levels = may require a higher level of confidence (and a greater amount of validation)

• levels considerably above action levels = validation is not likely to show a difference in the presence or

absence of risk.

Criteria to Guide Project Planning

(12)

Project Budget…

or How Much Is Enough?

• Validation can take into account budget &

schedule efficiencies (examples):

• All data will be validated electronically.

• Validate a specific percentage (e.g., 10-20%) of all data sets (SDGs) unless a problem is identified.

• Validate a specific percentage of all

data sets and all critical decision samples (as identified in the UFP-QAPP) will have full sample validation.

(13)

Determining Data Sets for Validation

• Project-specific Data Quality Objectives (DQOs) defined in the QAPP

• Sample collection period of time within a Project, Site or Operable Unit (e.g., site investigation,

remedial action, semi-annual monitoring event)

• Laboratory (multiple project labs or specialty analysis)

• Laboratory determined sample groupings (e.g., Sample Delivery Group, laboratory reporting batch, preparation batch, method batch)

• Combination of the preceding factors

(14)

Data Validation Software

• Data review software - both data

verification and validation are sequentially performed with software such as ADR.NET

• Efficiencies in batch processing & input

error reduction result from use of EDDs and data review software.

• Compliance checks of electronic data (sampling and analytical) prior to import

into project data base, e.g. are there results for every sample collected?

(15)

Inputs To Review Process - Planning

Item Step I WS#34

Verification

Step IIa WS# 35/36 Compliance

Step IIb WS# 35/36 Comparison

Evidence of required approval of QAPP X Identification of personnel (those

involved in conducting verification steps) X

Laboratory identification X

Methods (sampling and analysis) X X

Performance requirements (including QC

criteria) X X X

Project quality objectives X X

Reporting forms X X

Field Sampling Plans: location, maps,

grids, and sample ID numbers X X

SOPs (sampling and analytical) X X

List of project-specific analytes X X

UFP-QAPP, Manual V1, March 2005, Data Review Elements

(16)

Lab Data Package Requirements

Data Package Item** Step I WS#34

Verification

Step IIa WS# 35/36 Compliance

Step IIb WS# 35/36 Comparison

Laboratory name / Case narrative / Signature Page X X

Chain of custody X X

Sample receipt / storage records X X

Communication logs X X

Sample chronology (lab receipt, extraction, analysis) X X

Laboratory sample identification numbers X X

Identification of QC samples (sampling and lab) X X

Copies of laboratory notebook, records, prep sheets X X

Sample results / reporting units / dilution factors X X X

Sample QC results (surrogate spike, internal standards) X X X

Method QC results (e.g., spike, duplicate, LCS) X X X

Associated PT sample results (batch or periodic) X X X

Instrument calibration reports X X X

Corrective action reports and results X X X

Definitions of laboratory qualifiers X X X

Documentation of any laboratory method deviations X X X

QC sample raw data X X X

Sample raw data X X X

Standards traceability records (standard source) X X X

Electronic data deliverables X X X

** DoD QSM data reporting requirements (version 4.2, Oct. 2010, section 5.10 and Appendix E)

(17)

Verification (Step I) Process - WS#34

• Step I (verification): is a completeness check of all of the sampling and

analytical data collected for the project.

• Who

• conducted by the environmental laboratory (for analytical data) and

• The prime contractor / DoD component PDT

• Should address both sample collection and analytical data.

(18)

Verification (Step I) Process - WS#34

• UFP-QAPP Manual Section 5.2.1

• Verify both data packages & Electronic Data Deliverables

• Verify other data information operations:

• field operations e.g. sample collection &

shipping documentation

• field analytical records and correlation studies

• database input

(19)

Example 1 – Data Verification (WS#34)

Verification

Input Description Internal/

External

Person(s) Responsible for Verification Chain-of-

Custody and Shipping Forms

Chain-of-Custody forms and shipping

documentation will be reviewed internally upon their completion and verified against the packed sample coolers they represent. The shipper’s signature on the Chain-of-Custody forms will be initialed by the reviewer, a copy of the Chain-of- Custody retained in the project file, and the original and remaining copies taped inside the cooler for shipment. See Field SOP-29 (Sample Handling and Custody) in Attachment 1 for further details.

Internal Contractor Field Team Leader

Field

Notebooks Field notes will be reviewed internally at the end of each working day and placed in the project file.

Internal Contractor Field Team Leader

Laboratory

Data Laboratory data packages will be verified internally by the laboratory performing the work for completeness and technical accuracy prior to submittal. Received data packages will be

validated internally by the Contractor Project Chemist or designee.

Internal/

External Laboratory QA Officer Contractor Chemist

Project

Reports Project reports will undergo a QA review by

Contractor senior staff with applicable expertise. Internal Contractor Project Team

(20)

Example 2 – Data Verification (WS#34)

Verification

Input Description Internal/

External Responsible for Verification

Sample Receipt and Chain of Custody

The condition of shipping coolers, temperature preservation and condition / sample ID of enclosed sample containers will be documented upon receipt at the analytical laboratory through use of a cooler receipt form. The completed cooler receipt form will be transmitted by e-mail upon receipt of samples in the lab.

The cooler receipt form will also be submitted with the final analytical results from the laboratory.

External Project Lab

Analytical Data Package and Electronic Data Deliverable

The lab will perform in-house analytical data reduction under the direction of the Laboratory QA Manager. Completeness of all requirements and submittals (Data Packages / EDDs) will be verified by the Laboratory PM.

External Project Lab QA Project Lab PM

Laboratory Data Packages and Electronic Data

Deliverables

Verify Data Package for completeness for the following:

 Laboratory Case Narrative

 All samples analyzed for requested methods

 Sample condition upon receipt

 Holding times

 Sample Results

 Blanks

 MS/MSDs

 LCSs

 Surrogate recovery

 Initial and Continuing Calibration summary and raw data

 Sample analysis raw data including sample and CCV standard chromatography

Verify the completeness and identify any errors in the EDD.

Internal Project Chemist

(21)

Validation (Steps IIa) Process (WS#35)

• UFP-QAPP Manual Section 5.2.2

• Step IIa Validation Activities address compliance with:

• published methods,

• procedures (lab SOPs etc.),

• contract requirements.

(22)

Validation (Steps IIb) Process (WS#35)

• Step IIb Validation activities address comparison with measurement

performance criteria

• DQIs documented in the QAPP

• LOD/LOQs for project

• DoD QSM method specific criteria (tables in Appendix G and F)

• Laboratory derived performance criteria (LCS criteria)

(23)

Example 1 – Data Validation (WS#35)

Step IIa/IIb Validation Input Description Person(s) Responsible for

Validation IIa Field SOPs Verify that the sampling SOPs were followed Contractor Field Team

Leader

IIa Analytical SOPs Verify that the analytical SOPs were followed Laboratory QA Officer

IIa Results Verify that the required QC samples were run and met required limits

Laboratory QA Officer Contractor Project Chemist

IIa/IIb Data Validation Validate 100 percent of the data to confirm quality as defined in Worksheet #14 (Summary of Project Tasks)

Contractor Project Chemist

IIa/IIb Data Usability Evaluation

Evaluate data based on precision, accuracy, representativeness, comparability and

completeness for project objectives

Contractor Project Chemist

IIb Field

Documentation Verify accuracy and completeness of field

notes Field Team Leader

IIb Sample Results Verify that the required field QC samples were

run and met required limits Contractor Project Chemist

IIb Quantification

Limits Verify that the sample results met the project

quantification limit specified in the QAPP. Contractor Project Chemist

(24)

Example 2 – Data Validation (WS#35)

Step Validation Input Description Responsible

for Validation IIa

sample receipt, preservation, holding times

Verify that all samples were analyzed for the methods requested.

Check holding times for compliance to DoD QSM Chemist

IIb Initial Calibration

Verify criteria in DoD QSM (Table 7-2) & WS #24 were met.

Recalculate the calibration factors and % RSD for at least one target compound and surrogate for each standard. Recalculate at least one target compound and/or surrogate in instances where average calibration factor was not used for quantitation (i.e., linear regression, or other).

Chemist

IIb ICV and CCV

Verify criteria in Worksheet #24 were met. Recalculate the calibration factors and % Difference for at least one target compound and surrogate (including confirmation column, if applicable) for each standard. Verify and recalculate at least one target and/or surrogate in instances where average calibration factor was not used for quantitation

Chemist

IIb Method and

Field Blanks Verify that criteria in Worksheet #28 are met for all blanks. Review raw data

for the presence of target compounds that are above the lab LOD. Chemist IIb MS/MSD Inspect the MS/MSD Recovery summary and verify that the recovery and

RPD results meet criteria specified in WS#24. Recalculate at least one target

compound and surrogate from the MS/MSD. Recalculate at least one RPD. Chemist IIb Surrogates Verify that the Surrogate recovery results are within criteria specified in

Worksheet #28. Recalculate at least one surrogate recovery from the raw

data in each field sample and QC sample. Chemist

IIb LCS Verify that the LCS is within criteria specified in QAPP Worksheet #28.

Recalculate at least one LCS recovery from the raw data. Chemist

(25)

Validation Summary (Steps IIa & IIb) WS#36

• Step IIa and Step IIb (validation): The

amount of data / type of information to be validated will vary on a project-by-project basis

• DQOs for the project data

• The project phase (SI data or confirmation data after RA?)

• Budget & schedule: usually always a factor for analytical data validation

• Ensure sufficient representativeness for data quality across methods and analytes of

concern.

(26)

Example 1 –

Validation Summary (WS#36)

QAPP Worksheet #36 – Analytical Data Validation (Steps IIa and IIb) Summary Table

Step

IIa/IIb Matrix Analytical

Group Validation Criteria Data Validator

IIa and IIb

Soil Explosives – 8330B

QC criteria specified in this document, DOD QSM requirements, EPA Methods, USACE 200-1-10, Guidance for Evaluation Performance Based Data (June 30, 2005), USACE Shell Document Tables and EPA Contract Laboratory Program, and National Functional Guidelines for Organic Data Review (EPA, 2008)

Contractor Project Chemist automated data review system or third party data validation company IIa

and IIb

Soil Metals – 6010C/

6020A

QC criteria specified in this document, DOD QSM requirements, EPA Methods, USACE 200-1-10, Guidance for Evaluation Performance Based Data (June 30, 2005), USACE Shell Document Tables and EPA Contract Laboratory Program, and National Functional Guidelines for Inorganic Data Review (EPA, 2004)

Notes: 1. Step IIa denotes compliance with methods, procedures, and contracts. Step IIb denotes comparison with measurement performance criteria in the UFP-QAPP.

(27)

Example 2 –

Validation Summary (WS#36)

Step

IIa/IIb Matrix Analytical

Group Validation Criteria Responsible for Validation

IIa & IIb soil PCB Aroclors

SW8082 WS# 24 and 28

Project chemist

ADR.NET ver. 1.4.0.111 IIa & IIb soil VOCs

SW8260C WS# 24 and 28

IIa & IIb soil Metals

6020A WS# 24 and 28

(28)

Documentation and Communication

• Typical Project flow 1. Planning

2. Data collection

3. Validation processes 4. Database

5. Data users 6. ?

(29)

Validation Checks – Why it is the way it is

• Checks are usually done in a sequence that gets the “biggest bang for your

buck” (to use efficiently use validation resources)

• Sequence of checks may change depending on what parts of the

validation process can be automated

• This is turn depends on what laboratories can provide electronically

(30)

How does this play out

when validating analytical results

• Critical factors affecting the results(sometimes legal not analytical - like holding times) are done first

• Followed by reviewing QC results that affect a large number of results (e.g., Calibration,

Laboratory control samples) within a sample batch

• Validation generally completed by

representative checks that affect fewer results but are needed for more assurance (e.g., analyte

identification & recalculations to confirm a

contaminant of concern and its concentrations)

(31)

Validation Process Types

• Manual

• Review of hard copy data package submitted by the laboratory

• Automated

• Review of an electronic data deliverable by review software

• Combination of the above

(32)

6. How are the validation checks

and process used communicated

with data results?

(33)

Labeling Validated Data:

The Black Box Opened !

(34)

OR

•Tell me what you did when you validated my data

AND don’t take all day

about it!!!

(35)

Typical Project Data Generation and Review Process

Data May Support Future Projects

QAPP ANALYTICAL REQUIREMENTS SENT TO LABORATORY

SAMPLES + CHAIN OF CUSTODY INFORMATION

4

Laboratory Analysis and Reporting

1

Project Scope Defined

2

Quality Assurance Project Plan (QAPP)

Requirements Identified

7

Data Repository e.g., Database (Project or

General)

6

Data Quality Assessment of Field and Laboratory Data

5

External Party Validation of the Laboratory Analytical

Data

3

Field Activities

ANALYTICAL DATA PACKAGE (with Laboratory Qualifiers)

VALIDATED ANALYTICAL DATA FIELD LOGS

INFORMATION

DATA USED IN PROJECT DECISION (S)

(36)

The Analytical Data Validation

“Black Box”

• Each year Federal Programs spend considerable resources to validate laboratory analytical data.

• Validation guidelines vary from program to program.

• No consistent mechanism to indicate what was actually checked during the

validation process.

(37)

Opening up the Black Box !

• Guidance for Labeling Externally

Validated Laboratory Analytical Data for Superfund Use

• Location on the web:

http://www.epa.gov/superfund/policy/

pdfs/EPA-540-R-08-005.pdf

• Length: 21 pages

• Main document: 7 pages

• 5 Appendices: 14 pages

(38)

Guidance Goals

• Improve communication within Superfund about scope and content of lab analytical data verification and validation.

• Encourage appropriate use of data for

• Task at hand

• Project phase

• Future decisions

Guidance is not limited to EPA Superfund!

(39)

Approach Taken

• Facilitate communication to data users using “labels” that summarize verification and validation checks.

• Checks are grouped into stages.

• Each stage builds on the checks from previous stage.

• Labels also describe nature of review process (manual and/or electronic)

• Based on the general guidelines used for data validation

(40)

Stages of Validation Checks

• Completeness

• Compliance

• Sample-related QC

• Instrument-related QC

• Recalculation

• Instrument output review

• Examples of checks in each stage are given in Appendix A of the Guidance Document.

(41)

Completeness Checks

• To make sure that the requested data deliverables are provided.

• To determine that data requested are actually present in the deliverables.

• To ensure consistency within the deliverable (e.g., between hardcopy and electronic

deliverables).

(42)

Compliance Checks

• To compare analytical Quality Control (QC) results with the acceptance criteria,

requirements or guidelines present in the regional data validation documents,

analytical methods or contract.

• Sample-Related QC (e.g., blank

contamination, surrogate recoveries)

• Instrument-Related QC (e.g., instrument calibrations, tunes)

(43)

Recalculation Checks

• The laboratory reported values (e.g.,

sample results, instrument calibration results) are verified by recalculation using

instrument output data reported by the laboratory.

• Confirms that correct formulae and values were used in calculation of results.

(44)

Instrument Output Checks

• Actual instrument outputs should be checked to ensure that the laboratory reported analytes have been correctly

identified and quantitated (e.g., are mass spectra properly identified? Are peak

integrations correct?).

(45)

Process Labels

• Electronic review only E

• Manual validation M

• Electronic and manual EM

(46)

Validation Checks as Stages

• Stage 2a Validation: completeness and compliance checks of sample receipt conditions and ONLY sample-related QC results.

• Stage 2b Validation: completeness and compliance checks of sample receipt conditions, BOTH sample-related AND instrument-related QC results.

• Stage 3 Validation: completeness and compliance checks of sample receipt conditions, BOTH sample-related AND instrument-related QC results, and recalculations of sample results from laboratory instrument responses.

• Stage 4 Validation: completeness and compliance checks of sample receipt conditions, BOTH sample-related AND

instrument-related QC results, recalculations of sample results from laboratory instrument responses, and instrument outputs

(47)

Validation Checks, Stage, and Labels

Checks Validation Stage Label

Completeness Stage 1 S1V

Completeness & Sample QC Stage 2a S2AV Completeness, Sample QC &

Instrument QC Stage 2b S2BV

Completeness, Sample QC,

Instrument QC & Recalculations

Stage 3 S3V

Completeness, Sample QC,

Instrument QC, Recalculations &

Instrument Outputs

Stage 4 S4V

(48)

Data Packages for Validation Stages

Type of Data Package Data Validation Stage for Data Package Type

Results (with sample receipt conditions) Stage

1 N/A N/A N/A N/A

Results with sample receipt conditions AND

sample related QC information Stage

1 Stage

2A N/A N/A N/A Results with sample receipt conditions and

BOTH sample related and instrument related QC information

Stage

1 Stage

2A Stage

2B N/A N/A Results with sample receipt conditions and

BOTH sample related and instrument related QC information, and summary instrument data used for calculating sample results.

Stage

1 Stage

2A Stage

2B Stage

3 N/A

Results with sample receipt conditions.

BOTH sample related and instrument related QC information, summary instrument data used for calculating

sample results, and instrument outputs (e.g., chromatograms, spectra)

Stage

1 Stage

2A Stage

2B Stage

3 Stage 4

(49)

Example Labels For Validated Data

• Stage 2b validation by electronic tools only:

• S2BVE

• Stage 3 validation by both electronic and manual processes:

• S3VEM

Note: Labels combining all validation stages and processes are given in

Appendix B of the Guidance

(50)

Software Requirements for Validation Labels

• Data review software must:

• automatically append the appropriate label to each analyte in the output electronic

deliverable

• have the capability for data reviewers to

manually change the label for an analyte or group of analytes based on additional

manual review of a data package or SDG.

• ensure that the label(s) transfer(s) with the data qualifier (flag) with each analyte into any database that will be maintained for the project

(51)

Data Storage Requirements

• End Database must have a field to store validation label

• For uniformity it is recommended that this field be called “Analytical Data Validation Stage”

• This will be addition to the Data Qualifier field

(52)

List of Data Validation Label Codes

LABEL CODE

Stage_1_Validation_Electronic_Only S1VE

Stage_1_Validation_Manual S1VM

Stage_1_Validation_Electronic_and_Manual S1VEM Stage_2a_Validation_Electronic_Only S2AVE

Stage_2a_Validation_Manual S2AVM

Stage_2a_Validation_Electronic_and_Manual S2AVEM Stage_2b_Validation_Electronic_Only S2BVE

Stage_2b_Validation_Manual S2BVM

Stage_2b_Validation_Electronic_and_Manual S2BVEM Stage_3_Validation_Electronic_Only S3VE

Stage_3_Validation_Electronic_and_Manual S3VEM Stage_4_Validation_Electronic_Only S4VE

Stage_4_Validation_Electronic_and_ Manual S4VEM

Not_Validated NV

(53)

Labeled EDD (ADR.NET)

FIELD SAMPLE ID SAMPLING

DATE MATRIX LAB SAMPLE ID QC TYPE METHOD PREP DATE ANALYSIS DATE VALIDATION

CODE METHOD TYPE DUPL-09092011 9/9/2011 AQ 230944-011 FD 6010C 9/15/2011 9/19/2011 S2AVE METALS

MW-78 9/8/2011 AQ 230944-007 6010C 9/15/2011 9/16/2011 S2AVE METALS

MW-81D 9/8/2011 AQ 230944-015 6010C 9/15/2011 9/19/2011 S2AVE METALS

MW-92-38 9/9/2011 AQ 230944-010 6010C 9/15/2011 9/19/2011 S2AVE METALS

MW-LF26-1 9/9/2011 AQ 230944-019 6010C 9/15/2011 9/19/2011 S2AVE METALS

MW-LF26-4MS 9/8/2011 AQ QC609298MS MS 6010C 9/15/2011 9/16/2011 S2AVE METALS MW-LF26-4MSD 9/8/2011 AQ QC609299MSD MSD 6010C 9/15/2011 9/16/2011 S2AVE METALS

PZ-09 9/9/2011 AQ 230944-009 6010C 9/15/2011 9/19/2011 S2AVE METALS

QC609295 9/15/2011 AQ QC609295 MB 6010C 9/15/2011 9/16/2011 COA METALS

QC609296 9/15/2011 AQ QC609296 LCS 6010C 9/15/2011 9/16/2011 COA METALS

QC609297 9/15/2011 AQ QC609297 LCSD 6010C 9/15/2011 9/16/2011 COA METALS

(54)

Data Review Sample Summary Report by Analysis Method

Lab Batch ID: 01_AW309546 Approved by: DN (3/15/2012) Laboratory: LDC Client Sample ID Lab Sample ID Matrix Sample Type Preparation

Method Collection

Date Validation Code

Method: 6010C (TOT)

MW-01 810095-01 AQ N 3010A 10/6/2008 S2AVE

MW-02 810095-02 AQ N 3010A 10/6/2008 S2AVE

MW-03 810095-03 AQ N 3010A 10/6/2008 S2AVE

MW-04 810095-04 AQ N 3010A 10/6/2008 S2AVE

Method: 8081A

MW-01 810095-01 AQ N 3520C 10/6/2008 S2AVE

MW-02 810095-02 AQ N 3520C 10/6/2008 S2AVE

MW-03 810095-03 AQ N 3520C 10/6/2008 S2AVE

MW-04 810095-04 AQ N 3520C 10/6/2008 S2AVE

Method: 8260C

MW-01 810095-01 AQ N 5030B 10/6/2008 S2AVE

MW-02 810095-02 AQ N 5030B 10/6/2008 S2AVE

MW-03 810095-03 AQ N 5030B 10/6/2008 S2AVE

MW-04 810095-04 AQ N 5030B 10/6/2008 S2AVE

Method: 8270D

MW-01 810095-01 AQ N 3520C 10/6/2008 S2AVE

MW-02 810095-02 AQ N 3520C 10/6/2008 S2AVE

MW-03 810095-03 AQ N 3520C 10/6/2008 S2AVE

MW-04 810095-04 AQ N 3520C 10/6/2008 S2AVE

(55)

Example of Labeled Project Report

Location ID / Analyte Lab Result Qualifier LOD LOQ Units Validation Code MW-L66-1

Diesel C10-C28 15 J 9.8 50 µg/L S2BVE

MW-L66-1

Motor Oil C28-C36 <300 UJ 98 300 µg/L S2BVEM

MW-081

Motor Oil C28-C36 <300 UJ 98 300 ug/L S2BVEM

MW-L66-3

MW-L66-6

MW-01

(56)

(57)

(58)

Desired Outcome

• Third party reviewers/validators associate the validated data with its validation stage as data is shared with decision makers.

• Data users quickly recognize the nature of review performed on data prior to use.

• Future use of data is facilitated by labels that travel with data (e.g., kept associated with data in repositories)

(59)

Benefits of Providing Validation Labels

• Opens up the “Validation Black Box” by telling data recipients and users in a short and succinct manner:

• Checks used to validate the data

• Process used to validate the data

• Supports future use of data

(60)

Benefits of Providing Validation Labels

• Allows for automation of some stages of the data validation process (e.g., use of electronic tools for compliance checks and recalculations)

• Helps integrate manual and electronic processes used for data validation

(61)

Summary & Things to Think About….

• Determine data sets and level of validation as part of the Project Planning Process (e.g.,

document in UFP-QAPP Worksheet)

• Ensure Data Review software can provide the labels and allows for manual editing of labels based on further validation (for EDDs)

• Retain Data Validation Labels by ensuring data repositories have fields for the labels which are linked to the validated

analytical data

(62)

Implementation Status

• US Army Corps of Engineers’ through use of ADR.NET software (puts validation label

after automated review)

• US Army Corps of Engineers FUDS CHEM Database (has a field for the validation label)

• EPA Regions (may need some prodding!)

(63)

FINAL STEP!!!

• UP TO YOU

• Repeat after me: “I will never use validated data that does not have a label

attached to it!!!”

(64)

Contact Information

• Pam Wehrmann

Phone: 916-557-6662

Email: pamela.a.wehrmann@usace.army.mil

• Anand Mudambi

Phone: 202-564-2817

Email: mudambi.anand@epa.gov

(65)

Questions?