SPREADSHEET VALIDATION 101: CREATE AND VALIDATE FDA-COMPLIANT MS EXCEL SPREADSHEETS

1

RA U L SO T O, M SC, C Q E I V T AP R I L 2 0 1 5 PH I L A D E L P H I A

S

PREADSHEET

V

ALIDATION

101:

CREATE AND VALIDATE FDA-COMPLIANT

A

BOUT THE AUTHOR

• 20+ years of experience in the medical devices, pharmaceutical,

biotechnology, and consumer electronic industries

• MS Biotechnology, emphasis in Biomedical Engineering • BS Mechanical Engineering

• ASQ Certified Quality Engineer (CQE)

• I have led validation / qualification efforts in multiple scenarios:

• High-speed, high-volume manufacturing and packaging equipment

• Laboratory : information systems and equipment / instruments

• Enterprise resource planning applications (i.e. SAP) • IT network infrastructure

• Software applications

• Product improvements, material changes, vendor changes

• Contact information:

T

ABLE OF

C

ONTENTS

1. R

P

2. S

V

3. B

C

S

S

COPE

SPREADSHEET

•

Microsoft Excel file that may contain:

• mathematical calculations

• Statistical analysis

• Graphing

•

Outside our scope:

• Macros or custom code

5

RE G U L A T I O N S GO V E R N I N G SP R E A D S H E E T S

P

ART

1: R

EGULATORY

W

HICH

S

PREADSHEETS NEED TO BE

V

ALIDATED

?

If the spreadsheet supports any activity governed by

FDA 483

S AND

W

ARNING

L

ETTERS

•

Companies have been

cited by FDA

for not

validating spreadsheets used for:

• In–process analytical calculations

• Finished Product analytical calculations • Tracking and trending quality metrics • Databases / list

FDA 483

FDA W

ARNING

L

ETTERS

http://www.contractormisconduct.org/ass/contractors/18/cases/305/221/f da-warning-letter.pdf

FDA W

ARNING

L

ETTERS

http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2013/ucm 369409.htm

FDA W

ARNING

L

ETTERS

http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2006/u cm075766.htm

FDA W

ARNING

L

ETTERS

http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2011/ucm2 56498.htm

FDA W

ARNING

L

ETTERS

http://www.ofnisystems.com/media/g3667d.pdf

FDA W

ARNING

L

ETTERS

http://www.ofnisystems.com/Resources/Warning_Letters/g4452d.pdf

FDA W

ARNING

L

ETTERS

http://www.ofnisystems.com/media/g1483d.pdf

M

EDICAL

D

EVICES

•

21 CFR 820.70(i): Automated processes

• When computers or automated data processing

systems are used as part of production or the

quality system, the manufacturer shall validate

computer software for its intended use according to an established protocol.

• All software changes shall be validated before

approval and issuance.

• These validation activities and results shall be

M

EDICAL

D

EVICES

•

Let’s take this apart:

• Applies to computer / data processing systems used for

production or quality

• The manufacturer is responsible for validation • Validation must be for the intended use(s) • Must have an established protocol

• Changes must be validated before implemented • Document all validation activities and results

M

EDICAL

D

EVICES

• Section 6.0

• “Many other commercial software

applications, such as word processors,

spreadsheets, databases, and flowcharting software are used to

implement the quality system. All of these applications are subject to the requirement for software validation, but the validation approach used for each application can vary widely.”

M

EDICAL

D

EVICES

• Section 6.1

• “Numerous commercial software

applications may be used as part of the quality system (e.g., a spreadsheet or

statistical package used for quality system calculations, a graphics package used for trend analysis, or a commercial database used for recording device history records or for complaint management). The extent of validation evidence needed for such

software depends on the device

manufacturer's documented intended use of that software.”

P

HARMACEUTICALS

•

21 CFR 211.165(e): Testing and Release

for Distribution

• (e) The accuracy, sensitivity, specificity, and

reproducibility of test methods employed by the firm shall be established and documented. Such validation and documentation may be

B

IOLOGICS

•

21 CFR 606.140: Laboratory Controls

• Laboratory control procedures shall include:

• (a) The establishment of scientifically sound and appropriate specifications, standards and test procedures to assure that blood and blood components are safe, pure, potent and effective.

• (b) Adequate provisions for monitoring the reliability,

accuracy, precision and performance of laboratory test procedures and instruments.

• (c) Adequate identification and handling of all test samples so that they are accurately related to the specific unit of product being tested, or to its donor, or to the specific recipient, where applicable.

ISO 13485

DESIGN AND MANUFACTURE OF MEDICAL DEVICES

•

Clause 7.5.2.1

• The organization shall establish documented

procedures for the validation of the application of

computer software (and changes to such software

and/or its application) for production and service provision that affect the ability of the product to conform to specified requirements.

• Such software applications shall be validated prior

ISO 13485 : 2015

DESIGN AND MANUFACTURE OF MEDICAL DEVICES

•

Will explicitly mandate:

•

That software used

as part of the quality

system

must be validated and subject to

change control.

•

That we must have procedures to govern

this

•

That validation must be documented and

FDA L

AB

M

ANUAL OF

Q

UALITY

P

ROCEDURES

• http://www.fda.gov/ScienceResearch/FieldScience

4.5.3 V

ALIDATION OF

S

PREADSHEETS

SUMMARY

• Lock all non-input cells

• Make spreadsheets read-only, with password protection • Automatically reject data outside valid ranges

• Positive and negative testing: Test by entering expected and extreme values, and nonsensical data

• Keep record of all cell formulas

• Document changes made to the spreadsheet • Change control with version numbers

• Periodic re-validation spreadsheets, including

• verification of cell formulas

FDA’

S

I

NTERNAL

G

UIDELINES

FOR

S

PREADSHEET

D

ESIGN AND

V

ALIDATION

• Spreadsheet Design and Validation for the

Multi-User Application for the Chemistry Laboratory: Part I

• www.spreadsheetvalidation.com/pdf/

LIB_Design_Multi-User2A.pdf

• Spreadsheet Design, Verification and Validation,

Use and Storage of Single-User Workbook Files in the US FFDA Laboratories: Part II

• www.spreadsheetvalidation.com/pdf/

C

OMMON

S

PREADSHEET

I

SSUES FOUND BY

FDA

IN

I

NDUSTRY

• Spreadsheet files not maintained, some deleted • Rounding errors

• Formulas don't match (Excel vs source, Excel vs manual calculation)

• Conversion factors issues

• Spreadsheet doesn't show specs or limits • Unclear documentation

• Units not labeled

• Weights described as volumes, areas as ratios • Security and data integrity issues

• Cells not locked

29

V A L I D A T I O N P L A N , E X E C U T I O N , R E P O R T I N G

P

ART

2: V

ALIDATING THE

V

ALIDATING YOUR

S

PREADSHEET

•

Validation

:

Provide documented

evidence that the

spreadsheet does what it

is intended to do,

T

YPICAL

V

ALIDATION

E

LEMENTS

-or-• System / User Acceptance

• Validation Report

• Procedures & Governance

• Templates

You can use

separate documents for each -OR-A single Validation Protocol that includes ALL validation elements

G

OVERNANCE

• Spreadsheet Validation Procedure

• Outlines HOW spreadsheets will be validated • What document(s) need to be produced

• Roles and responsibilities, Authors and Approvers, etc.

• Templates (i.e. Spreadsheet template)

• Change Control Procedure

• May be a separate procedure, or part of the main

Spreadsheets Validation Procedure

• Validated Spreadsheet Inventory

G

ENERAL

I

NFORMATION

• Spreadsheet name

• Description and Purpose

• What the spreadsheet will be used for • Process(es) supported

• Where it will be used

• References

R

EQUIREMENTS

• Describes, with detail, WHAT you want the

spreadsheet to do.

• Requirements should be numbered

• MS Excel version required

• Excel Add-Ins required, if any

• OS version in which it is developed

• For example: MS Windows 7 SP1, Mac OS X 10.10 • Include OS service pack, if applicable

R

EQUIREMENTS

• Input cell and formula cell requirements

• Formatting (colors, font size and type, etc.) • Data validation (field type, size, etc.)

• Descriptions for every formula, and source

• Page formatting

• Titles

• Headers • Footers

R

EQUIREMENTS

• Charts and Graphs

• Title and Description of each • Required labels for each axis

• Security requirements

• Password protection

• Output requirements

• Color printer, PDF-creating software

• Specific folder in a specific shared drive • Automatic eMail to specific people

D

ESIGN

S

PECIFICATION

• Describes, with details, HOW the spreadsheet will

comply with the Requirements

• Address all requirements, individually or by groups • Describes the actual spreadsheet as written

• Shows actual EXCEL version of formulas

• Describes algorithms used, if any

• Describe conditional cell formatting used, if any • Describe data validation used, if any

T

RACEABILITY

M

ATRIX

• Traces every requirement to its

corresponding design

element, and to the test

script(s) where the

requirement is challenged and tested.

D

EVELOPMENT

T

ESTING

& D

RY

R

UNS

• Recommended!

• Typically not documented formally

• Dev Testing helps discover bugs and fix them while

we’re not yet in formal validation.

• Dry Runs help discover issues with the validation test

scripts, and allows us to fix these before we move into formal document change control.

I

NSTALLATION

Q

UALIFICATION

(IQ)

•

IQ includes Installation and Configuration

• State all prerequisites, ensure they are met prior to

execution

• List all spreadsheet files that have to be installed • List location and path where spreadsheet files will

be installed

• Development environment – does not have to be validated • Test environment – may need to be validated

O

PERATIONAL

Q

UALIFICATION

(OQ)

• State all prerequisites, ensure they are met prior to

execution

• Verify that all requirements documented in the

Requirements section are met

• Ensure all outputs have the correct formatting

• Provide printouts of all formulas, input values

used, output results.

•

Positive

v

Negative

testing !!!

• Security testing

• Performance testing

O

PERATIONAL

Q

UALIFICATION

(OQ)

• Ensure all non-input cells are locked • Ensure conversion factors are correct

• Ensure spreadsheets are read-only, with password

protection

• Ensure the spreadsheet automatically rejects data

outside valid ranges

• Test for rounding errors

• Ensure Excel formulas match the source formula

• Ensure formula input / output cells are correct

• Compare formula results vs manual calculations

OQ: P

OSITIVE VS

N

EGATIVE

T

ESTING

• Positive testing : test and verify every single

formula using values that should produce correct results

• Ensure the spreadsheet produces correct results for

various input data sets

• i.e. at least two data sets

• Data set 1: known to result in a PASS decision • Data set 2: known to result in a FAIL decision • Include test cases with borderline data

• Verify spreadsheet results against results from

OQ: P

OSITIVE VS

N

EGATIVE

T

ESTING

•

OQ

• Negative testing :

• Use incorrect inputs to ensure

the spreadsheet is robust, able to handle them without crashing or producing false or incorrect outputs, and that it is not

OQ: P

OSITIVE VS

N

EGATIVE

T

ESTING

Negative testing examples:

• Required data entry : ensure input fields cannot be left

blank

• Field type test: ensure data entered is of the correct type

(i.e. dates, alphanumeric, etc.)

• Enter alpha or special (@%^#) characters on a numeric field • Enter a decimal number in an integer field

• Enter negative numbers in a positive numbers field

• Enter nonsensical inputs (i.e. ASCII CRTL codes)

• http://academic.evergreen.edu/projects/biophysics/technotes/program/ascii_ctr l.htm

• Field size test: ensure user can only enter the specified

OQ: P

OSITIVE VS

N

EGATIVE

T

ESTING

Negative testing examples:

• Date bounds test: test upper and lower bounds for dates. • Numeric bounds test: for example, if the number in a field

should be between 10 and 50, the system should not accept numbers lower than 10, and higher than 50.

• Numeric limits test: make sure the spreadsheet doesn’t

crash or produce incorrect output when entering a number outside of Excel’s limits:

• Excel 2013 limits:

•

OQ: P

OSITIVE VS

N

EGATIVE

T

ESTING

• For a good discussion, with

examples, of Positive vs

Negative software testing,

please refer to:

• http://www.softwaretestingcl

ass.com/positive-and- negative-testing-in-software-testing/

OQ S

ECURITY

T

ESTING

• Security testing:

• Ensure it is not possible to:

• Unlock locked cells / worksheets

• Alter formulas, output results,

output graphs /charts/ tables

OQ P

ERFORMANCE

T

ESTING

• Output issues : ensure the spreadsheet behaves as

expected when you

• disconnect printer

• close the PDF-rendering application

• Performance :

• Multiple users simultaneously

• Open multiple instances of the spreadsheet at the

same time

• Execute spreadsheet while various other

O

PERATIONAL

Q

UALIFICATION

• Formula Auditing

• Excel’s Formula Auditing tools allows us to demonstrate that

T

EST

E

XECUTION

• Ensure test scripts are

executed in the appropriate sequence.

• Document deviations when

actual results do not match expected results

• Take screenshots and printouts

T

EST

E

XECUTION

• Include all executed test scripts • Include all screenshots, printed

outputs, manual calculations

C

HANGE

C

ONTROL

A

FTER

V

ALIDATION

• Update Validation documentation

• Additional or changed requirements

• Corresponding design elements changes

• Update Testing

• New or updated test scripts

• Test new functionality (i.e. new/updated formula)

• Regression testing: ensure areas not changed still work

correctly

• Update traceability matrix

• Upon release, increase the Version number

V

ALIDATION

: H

OW MUCH IS TOO MUCH

?

57

DE S I G N A CO M P L I A N T SP R E A D S H E E T

P

ART

3:

C

HANGE

C

ONTROL

• Spreadsheets used for GxP processes should be

subjected to formal version control and validation.

• The organization’s change management system

should be used to document whenever we:

• Create a new spreadsheet for regulated / validated uses

• Make changes to a validated spreadsheet

• Retire/decommission a validated spreadsheet

• Move a validated spreadsheet to a different version of

Excel (i.e. from Excel 2007 to Excel 2010) from the one used in development and validation

• Move a validated spreadsheet to a different operating

C

REATING THE

S

PREADSHEET

• The following information (at minimum) should be

included on the main worksheet:

• Spreadsheet Name

• Version number

• Date of last modification

• Every printout should contain

C

REATING THE

S

PREADSHEET

• Verify formulas against sources

• Use correct units of measurement

• Lock all non-input cells

• Disable column/row add/remove

• Unlock input cells

• Distinguish input, calculation (formula) vs output

cells

• Cell background color

• Font : bold, italics, size, color

• Use conditional formatting on output cells

• Different cell background color and/or font color for results

CREATING THE SPREADSHEET

• Add a field where the user will enter his/her name

every time the spreadsheet is used.

• Use Page Setup to print the following information:

• Spreadsheet filename

• Correct date and time

• Page numbers and total pages

U

NITS

C

ONVERSION

• You can use the Excel CONVERT function to convert

values from one unit to another in your formulas

C

HANGE

H

ISTORY

• Each spreadsheet should have a Change History

worksheet (a.k.a. Revision History)

• Every time the spreadsheet is updated, an entry

should be made in this Change History:

• New version number • Date of change

• Change control number

• Name of person(s) who made the change

SECURITY LAYERS

• Lock all cells except user input cells.

• HOME tab/ CELLS group/ Format / Protection

• Password-protect the worksheet

• Prepare / Encrypt document

• Install the validated spreadsheet in a protected

folder on a networked shared drive

• Access to the folder should only be given to authorized

personnel

• OR save the validated spreadsheet as a controlled

I

MPORTANT

!!!

• Locking / unlocking cells

• Protect spreadsheets /worksheets

• Cell formatting:

• Data types

• Look and feel : Font, Colors

• Validation criteria definitions for input cells

• Define formulas

• Use of logical statements in formulas

66

H A N D S - O N S P R E A D S H E E T W O R K S H O P

S

PREADSHEET

C

REATION

Once you create your spreadsheet

• 1. ID cell types, by color

• 2. Conditional formatting for displaying results

• 3. Data validation

• 4. Formula auditing

• Trace Precedents and Dependents

• Show and evaluate formulas

• Error checking

• Preventing formulas from being displayed

S

PREADSHEET

S

ECURITY

• 6. Protect the Worksheet

• 7. Unprotect input cells

• 8. Protect the Workbook

• 9. Protect and SHARE the Workbook

1. ID C

ELL

T

YPES BY

C

OLOR

• Input Cells: Yellow background, red text

2. C

ONDITIONAL

F

ORMATTING

• Conditional formatting can be used to change the

formatting of a cell (background color, text color,

text properties, etc.) depending on its content, or on the content of another cell

2. C

ONDITIONAL

F

ORMATTING

• You can apply multiple formatting rules

to a single cell

• Use Manage Rules to keep track of the

2. C

ONDITIONAL

F

ORMATTING

• Example 1

• Minimum spec value is 14, maximum is 17 • We want cells to be:

• Lt Green fill and Dk Green text IF value is between min and max • Lt Red fill and Dk Red text IF value is outside min and max

1. Home / Styles / Conditional Formatting/ Highlight Cell Rules/ Less Than => dialog box

2. C

ONDITIONAL

F

ORMATTING

• Example 1 (cont.)

3. Home / Styles / Conditional Formatting/ Highlight Cell Rules/ Greater Than => dialog box

4. Select cell where maximum value is listed, and desired formatting:

5. Home / Styles / Conditional Formatting/ Highlight Cell Rules/

Between => dialog box

6. Select cells where maximum and minimum values are listed, and desired formatting:

2. C

ONDITIONAL

F

ORMATTING

• Example 2:

• Have a cell that shows PASS or FAIL depending on the value of

another cell

• We want

• PASS: Lt Green fill and Dk Green text • FAIL: Lt Red fill and Dk Red text

1. = IF ({cell} <= {min value}, “FAIL”,”PASS”)

2. = IF ({cell} >= {max value}, “FAIL”,”PASS”)

3. Home / Styles / Conditional Formatting/ Highlight Cell Rules/ Text that Contains => dialog box

4. Format Cells that contain: FAIL

with Lt Red Fill with Dk Red Text

5. Format Cells that contain: PASS

3. D

ATA

V

ALIDATION

• Use Excel’s Data Validation tools to enforce limits in

input fields.

• Examples:

• Integer input between 1 and 20

• Alphanumeric field, from 1 to 16 characters

• Date or time

• Decimal number greater, lower or than a specific value; or

between specific values

• Excel’s Data Validation also allows you to:

• Display custom messages on input cells

• Display custom error messages when input is outside

3. D

ATA

V

ALIDATION

- S

ETTINGS

• Define the types of

content that is allowed on the cell

• In this case the cell is

restricted to positive decimal values

3. D

ATA

V

ALIDATION

– I

NPUT

M

ESSAGE

• Enter the Title and Input

Message to be displayed when the mouse pointer hovers over the selected cell

3. D

ATA

V

ALIDATION

– E

RROR

A

LERT

• Enter the Error Alert

message to be displayed if the data entered does not meet the restrictions.

3. D

ATA

V

ALIDATION

– D

ATA

T

YPES

• restrict input to items from a specified list • List should be included in a protected

column in the spreadsheet

• Date • Time

• Text length • Custom

• Example: restrict field to specific length

• Example: restrict possible values of a cell to those in

drop-down list

• Example: prevent future dating

• Allow: Date

• Data: less than or equal to • End Date: =INT(NOW())

• If you want a date field to always auto-populate with the

date the spreadsheet is executed, put =INT(NOW()) as the field value for that cell, and LOCK the cell (don’t make it an input field). The user will not be able to modify that date.

3. D

ATA

V

ALIDATION

• Custom:

• Example: ensure the entry in a column is a number

1. Select the column

2. select Data, Validation

3. Select the Settings tab

4. Select Custom from the Allow list

5. In the Formula field, enter =ISNUMBER(c1) 6. Click OK

• The ISNUMBER function returns TRUE if the cell contains a

4. F

ORMULA

A

UDITING

4.1 Preventing formulas from being displayed 4.2 Trace Precedents and Dependents

4.3 Show and evaluate formulas 4.4 Error checking

4. F

ORMULA

A

UDITING

4. F

ORMULA

A

UDITING

4.1 How to prevent your formulas from being displayed in the formula bar:

• 1. Select formula cell

• 2. Right click and select Format Cell

• -OR- Home / Cells / Format / Format Cells • 3. Protection:

• [√] Locked • [√] Hidden

• 4. OK

• 5. Review / Changes / Protect Sheet

4. F

ORMULA

A

UDITING

4.2 How to trace Precedents and Dependents in your formulas:

• Precedents

• 1. Select formula cell • 2. Formulas / Formula

Auditing / Trace Precedents Excel will display arrows

showing which cells your formula inputs come from Remove arrows deletes the

4. F

ORMULA

A

UDITING

4.2 How to trace Precedents and

Dependents in your formulas:

• Dependents

• 1. Select formula cell • 2. Formulas / Formula

Auditing / Trace Dependents

Excel will display arrows showing which cells your formula sends outputs to Remove arrows deletes the

4. F

ORMULA

A

UDITING

4.3 How to show and evaluate your formulas:

Show your formulas:

4. F

ORMULA

A

UDITING

4.3 How to show and evaluate your formulas:

Evaluate your formulas:

• Select a formula cell

• Formulas / Formula Auditing / Evaluate Formula

• Keep pressing EVALUATE, and Excel will show you step-by-step, with all intermediate values, how it evaluates your formula and calculates a result

4. F

ORMULA

A

UDITING

• 4.4 Formula Error Checking

• If there are errors in a formula (for example, entering a letter in a numerical input cell), Excel will show a #VALUE! error message • Click on Formula / Formula

4. F

ORMULA

A

UDITING

• Error Checking will display the cause of the error (wrong data type in this example)

4. F

ORMULA

A

UDITING

• From this dialog box you can: • Get HELP on the error type

• Show calculation steps (same as Evaluate Formula).

• You can see the step-by-step calculations Excel is

performing to evaluate your formula • Ignore Error:

• error checking will continue checking the rest of your

spreadsheet and stop at all errors until it’s done • Edit in formula bar:

5. S

ET

P

RINT

H

EADERS

& F

OOTERS

1. File / Print / Page Setup / Header & Footer

2. Header: Custom header

Left: Page & [Page] of & [Pages] Page x of xx

Center: & [Tab] v1.0 Worksheet name and version

Right: & [Date] & [Time] Date and time 3. Footer: Custom footer

Left: & [Path] & [file] Path and filename

Center: CONFIDENTIAL (use [A] icon to format text)

Right: nothing add anything you want 4. OK

6. P

ROTECT YOUR

W

ORKSHEET

• 1. Review tab / Changes group / Protect Sheet

• 2. √ Select locked cells

• 3. √ Select unlocked cells

• 4. Enter and re-enter worksheet password

• This password-protects your worksheet cells from any changes

• This also disables Column & Row Add / Remove

• BUT someone can still delete, add, and rename worksheets from

the spreadsheet

7. U

NPROTECT

I

NPUT

C

ELLS

1. Review tab / Changes group / Unprotect Sheet

2. Enter password, press OK => worksheet is now unprotected 3. Click on Allow Users to Edit Ranges

4. When the “Allow Users to Edit Ranges” dialog box opens, click on

NEW

5. When the “New Range” dialog box opens:

Title: <any title you want>

Refers to cells: range of input cell(s)

Range Password: <don’t pwd protect>

OK, OK

6. Click on Protect Sheet : enter worksheet password, then OK

8. P

ROTECT YOUR

W

ORKBOOK

• 1. Review tab / Changes group / Protect Workbook

• 2. [√] Structure • 3. [ ] Windows

• 4. Password: enter and re-enter worksheet password

When the worksheet is protected, you can’t add, remove, or rename worksheets in the MS Excel file

To unprotect, click on Review / Changes / Protect Workbook and enter the password

9. P

ROTECT

A

ND

S

HARE

W

ORKBOOK

• Once you are ready to publish your spreadsheet,

use Protect and Share Workbook

• 1. Review / Changes / Protect and Share Workbook

• 2. [√] Sharing with Track Change

• 3. Enter and re-enter worksheet password

• 4. SAVE the file

 The file will be saved, and will be ready for sharing

10. T

RACKING

C

HANGES

• Excel’s Track Changes functionality displays:

• Cells changed • Date and time

• User who changed it • Old and new value

• Helpful when you are editing a spreadsheet

• However it should not be used as a Part

11-compliant audit trail

10. T

RACKING

C

HANGES

• When you turn on Track Changes, Excel will mark

every cell you edit/change.

• If you hover the mouse pointer over the cell it will

display the change information.

• Once you SAVE your worksheet, the change marks

10. T

RACKING

C

HANGES

• You can get a “report” that will show you all the changes made to the spreadsheet since the last save

10. T

RACKING

C

HANGES

• If you want to review and Accept / Reject changes: