How to Conduct the Method Validation with a New IPT (In-Process Testing) Method

How to Conduct the Method Validation with

a New IPT (In-Process Testing) Method

IVT Method Validation, San Francisco, July 30, 2010

Weifeng “Frank” Zhang QA Engineer

BMS

Ernest Lee

Senior Manager, Facilities & Engineering

Medarex, a fully owned subsidiary of BMS

Forward Looking Statements

Except for historical information, the matters contained in this slide presentation may constitute forward-looking statements that involve risks and uncertainties, including uncertainties related to product development and clinical trials, unforeseen safety issues resulting from the administration of antibody products in patients, uncertainties related to the need for regulatory and other government approvals, dependence on patents and proprietary technology, the need for additional capital, uncertainty of market acceptance of Medarex’s product candidates, the receipt of future payments, the continuation of business partnerships and other risks detailed from time to time in Medarex’s filings with the Securities and Exchange Commission (SEC).

All forward-looking statements included in this slide presentation are based on information available to us as of July 30, 2010. We do not assume any obligation to update any information contained in these materials. Our actual results may differ materially from the results discussed in the forward-looking statements.

Disclaimer:

ƒ

All data in this presentation is not from actual

tests. Virtual data is used in this presentation for

confidentiality reasons.

Weifeng “Frank” Zhang

B.Sc. (Materials Eng.), M.Eng. (Chem. Eng.),

12 years in pharmaceutical, biotech

Ernest Lee

AGENDA

ƒ

Overview of method validation

and IPT

ƒ

New automated method

ƒ

Validation approach

ƒ

Q&A, Open Discussion

OVERVIEW

ƒ

Method validation elements

– Accuracy

– Specificity

– Precision

– Suitability

– Linearity

– Repeatability

– Limit of Detection

– Limit of Quantitation

– …

Things to Consider in Method Validation

Parameters Applicable to Different Analytical Procedures:

ƒ From: Method Validation Guidelines, Sep 15, 2005

By: Alex D. Kanarek, Ph.D. BioPharm International

IPT in This Study

ƒ

Cedex

ƒ

Automated Method

ƒ

IPT (In-Process Test)

ƒ

The most important parameters

Total cell counts

Cedex Description

ƒ Photos from: http://www.innovatis.com

IPT in This Study – Cedex Description

ƒ

Counts cells and determines viability based on the

Trypan Blue Exclusion method

ƒ

Takes 20 digital images of the fluid as it passes through

a flow cell

ƒ

Evaluate light level differences between the background

and the discovered cells

Counts – Cells are included or excluded from the count based on

relative size

Viability – Shading differences between the cells are utilized to

assess viability

IPT in This Study – Dynamic Data

In bio-production, every day, every hour,

every minute counts!

Accounting for Human Factors Variance

ƒ

User-dependent

New Automated Method

ƒ

Manual vs. Automated (

human factors in here?)

Manual method uses a microscope and a hemacytometer

after staining the cells with Trypan blue

After study and practice, develop SOPs to minimum the

human variance in the manual measurement

Challenges

ƒ

Different testing instruments between two

methods

ƒ

No “Gold Standard”

Validation Acceptance Criteria

ƒ

IOQ and PQ for instruments have been

successfully performed

ƒ

Identify parameters

ƒ

No significant difference between new

automated method and existing manual method

Validation Approach

ƒ

Design validation experiments

We will have to consider

all factors!

Accounting for All Variance

Formula:

Where, σ

is total variance,

In manual method, σ

1

is sample variance, σ

2

is human factor variance;

In automated method, σ

1

is sample variance, σ

2

is instrument

variance;

We need to make sure sample variance is

minimized between methods!

...

2

2

2

1

2

=

σ

+

σ

+

σ

Sampling Matrix Design

Day #

Sample Taken

Bioreactor

Manual

Cedex

0

Two samples taken at the same time. Time______

#1 Operator 1

Cedex log 1

1

Two samples taken at the same time. Time______

#1 Operator 2

Cedex log 2

2

Two samples taken at the same time. Time______

#1 Operator 3

Cedex log 3

3

Two samples taken at the same time. Time______

#1 Operator 1

Cedex log 4

…

_…

#1

_…

_…

n

Two samples taken at the same time. Time______

#1 Operator 3

Cedex log n

Raw Data

Cedex\Method Validation\Cedex Method

Validation Analysis -- Final.xls

Data Overview

Comparison Between Two Methods

4.00 6.00 8.00 10.00 12.00 14.00 16.00 18.00 le C e ll C ount s ( x 1 0 6 c e ll/m L ) Manual Method Auotmated Method

Statistical

Since viable cell counts base changes on different days, suggesting the

differences are not directly comparable. So the data was transformed

to percent differences to exhibit constant variance over the entire study.

Viable Cell Count Difference Between Manual and Auotmated Methods -1.50 -1.00 -0.50 0.00 0.50 1.00 1.50 2.00 2.50 3.00 0 1 2 3 4 5 6 7 Days C e ll C o u n t D iffe re n ce (x 1 0 6 cel l/m L ) Day 0 Day 4 Day 7 Day 11 Day 14 Day 18

Viable Cell Count Percent Difference Between Manual and Automated Method -20.00% -10.00% 0.00% 10.00% 20.00% 30.00% 40.00% 0 1 2 3 4 5 6 7 Days P er cen t D if fer en ce Day 0 Day 4 Day 7 Day 11 Day 14 Day 18

Statistical

ƒ

Since there is no “Gold Standard” in this study, we have to test

whether the two methods are different

ƒ

null hypothesis: The measurements from two (2) methods have no

significant difference

ƒ

From testing data of two methods, we can calculate the mean and

standard of the percent differences. Then construct a 95%

confidence interval on the mean

ƒ

Whether the 95% confidence interval includes zero will decide if the

null hypothesis is rejected

ƒ

Remember, trade-offs associated with using higher confidence

levels

Statistical

Mean of data

Zero Upper 95% confidence level

Lower 95% confidence level

null hypothesis can beacceptedin this case.

Mean of data Zero Upper 95% confidence level

Lower 95% confidence level null hypothesis is rejected in this case

Statistical

Null Hypothesis

Actual Result

Reject Fail To Reject

True

Type I Error

α: Probability of Type I Error (significance level)

True

False True _(1-β) Type II Error : Probability of Type II Error

Data Analysis – Step by Step

ƒ

Step 1 – Calculate mean for each day across all

bioreactor runs

ƒ

Step 2 – Calculate the percent difference in cell count

ƒ

Step 3 – Calculate the mean and standard deviation for

the percent difference in cell count

ƒ

Step 4 – Calculate the upper and lower 95% confidence

intervals

ƒ

Step 5 – Verify that “0” falls within the upper and

lower 95% confidence intervals

Theory Behind Validation Approach

ƒ

ANOVA Test

A collection of statistical models. It provides a

statistical test of whether or not the means of

several groups are all equal.

SSTREATMENT SSERROR ANALYSIS OF VARIANCE BIOREACTOR

ANOVA Test

M

H0: μ1= μ2 H1: μ1≠μ2 M = f ( t, ε(t))

M :Measurand (Cell Concentrations) (Viable and Non-Viable)

AGITATOR GROWTH PLATEAU DEATH Cavg(r, z, φ, t) (P) (pH, T, PO2) (rpm)

+

SSTOTAL SSTREATMENT SSERROR ANALYSIS OF VARIANCE BIOREACTOR

ANOVA Test

M

M = f ( t, ε(t)) AGITATOR GROWTH PLATEAU DEATH Cavg(r, z, φ, t) (P) (pH, T, PO2) (rpm)

+

SSTOTAL XMEAN= 4 SS = ∑(Xi– XMEAN)2

MSTREATMENT

MSERROR

MICROSCOPE

ANALYSIS OF VARIANCE

HEMACYTOMETER / HUMAN SYSTEM

AUTOMATED CELL COUNTER SYSTEM LINEARITY REPEATABILITY LIMIT OF DETECTION LIMIT OF QUANTITATION BIOREACTOR

ANOVA Test

M

μP SENSOR

SAMPLING / DILUTION / DIVISION

H0: μ1= μ2

H1: μ1≠μ2

M = f ( t, ε(t))

M :Measurand (Cell Concentrations) (Viable and Non-Viable)

(TRYPAN BLUE) (TRYPAN BLUE) AGITATOR GROWTH PLATEAU DEATH Cavg(r, z, φ, t) (P) (pH, T, PO2) (rpm) ε0 MST= ∑∑(cij– x)2/(Degrees of Freedom) C11 C12 C13 C21 C22 C23 RANDOMIZATION REPLICATION F < FCRITICAL

Summary

ƒ

Resolved the challenge of testing in the absence

of a standard

ƒ

Designed experiments in validation protocol

ƒ

Used statistical method to perform data analysis

ƒ

Established acceptance criteria based on 95%

confidence interval

Special Thanks to:

David Lorah

Supervisor, Validation Department

Lance Marquardt

Senior Manager, Biopharm Production Department

John R. Mosack

Senior Director, Clinical Manufacturing & Validation

Interactive Exercise

Q &A

ƒ

Open item

707 State Road • Princeton, NJ 08540 T: +1-609-430-2880

F: +1-609-430-2850 www.medarex.com