Method Validation and Verification: An Overview - PowerPoint Presentation

Slide1

Method Validation and Verification: An Overview

Patricia Hanson, Biological Administrator IFlorida Department of Agriculture and Consumer Services, Food Safety Microbiology LaboratorySlide2

Introduction

17 years in the microbiology section of the Florida Department of Agriculture and Consumer Services, Food LaboratoryLaboratory moved from primarily cultural methods in 1998 to a leader in the implementation of new technology in 2015

Numerous partnerships with state, federal and industrial customers to validate and implement new methodsSlide3

Goals

Distinguish between method verification and method validationBased on our laboratory’s experience, identify the critical components of both a method verification and a method validationOffer insight to approaches we have found useful, specifically in the microbiology section

Share lessons learnedSlide4

Verification vs

ValidationVerification is a demonstration that your laboratory can perform a

Standard Method

or other well documented method and produce acceptable results

Validation is both a demonstration that your lab can run the method

and

that the method or change to a method is fit for purposeSlide5

Aspects of a Verification

Run entire method, start to finish, exactly as it is writtenInclude both positive and negative samples

Represent all matrix types

Be spiked at a level that is fit for the purpose of the method if spiked samples are usedSlide6

Verifications can be ongoing….

For the purpose of this discussion, “Verification” is being used to describe what the laboratory does prior to implementing a method in the laboratoryThe term “Verification” is often also used to describe the ongoing process of verifying a method with each run – in our lab this it the use of Process Controls, Cultural Controls, and Media ControlsSlide7

Validation

Laboratory developed methodMatrix ExtensionsANY technically significant change to a Standard Method or previously validated methodSlide8

Overview of an Existing Guideline

Level One: Urgent Use, Single Lab ValidationLevel Two: Single Lab Validation

Level Three: Multi-Lab Validation

Level Four: Full Collaborative StudySlide9

Level 2: Single Lab Validation Highlights

50 different strains of target organism, if available30 different strains of “nearest neighbor” organisms

At least two target

analyte

levels

20 replicates per food type

per

analyte

level

Perform side by side if possibleSlide10

Aspect of a Verification

Be extensive enough to show that the method is fit for the intended use and meets the customer’s needsSlide11

What is “Extensive Enough”

Items to be considered:Matrix: number and types

Target: source and levels

Method Evaluation: side by side or use of standard reference material

When considering these items during the development of the validation, keep in mind what is needed to show the method is fit for the intended use and meets the customer needs.Slide12

Matrices

Validation or verification needs to be performed on each food typeMeat (raw, smoked, cooked, etc)

Vegetable (whole, cut, sprout, etc)

Dairy (liquid, solid, etc)

Other (spices, candies, etc) Slide13

Replicates

Exact number depends on methodVerification, normally triplicateValidation, normally six replicates per levelImportant to do the same number of replicates of “un-spiked”Slide14

Naturally Contaminated Samples

Naturally contaminated pros and consBest test of the methodLevel of

analyte

is unknown

Little or no control of

analyte

levelSlide15

Spiked Samples

Spiked sample pros and consAbility to control organism levelMethods often perform differently on spiked samples than on naturally contaminated samplesSlide16

Aging of Spiked Cultures

Spiking samples 48-72 hours prior to initiation of testing can better represent naturally occurring contaminationSometimes the cells are so stressed, they die offConsider the method or part of method being validated and type of validationSlide17

Selection of Organisms

Select at least one organism for each target“Nearest Neighbor” organisms – not often used but good to considerSlide18

“Side by Side” Validations

Normally involves running the method being validated side by side with a Standard or established methodTwice the work but very valuable for determining if the “new” method is equivalent or better than the Standard or established method

Best choice when changing part of a method – for example a new enrichment broth but same screening methodSlide19

How Much Confirmation?

Screening Method Validations: Confirm everythingYou can get by with an abbreviated confirmation, for example just biochemical panel and/or serology but do enough to show you “got out what you put in.”Slide20

Why Do So Many Confirmations?

Ensure you can confirm samples that screen positiveEnsure samples you can confirm screen positive

Spiked samples that don’t screen positive and don’t confirm point to limitations of the method or problems with the spiking procedureSlide21

Data Evaluation

The measures used to evaluate data depend on the purpose and type of the method and the needs of the customer

Accuracy

Ruggedness

Limit of

Quantitation

Specificity

False Positive Rate

Precision

RangeSlide22

Qualitative Data Evaluation

Detection Limits – can the method detect the target at the levels required by the customer?Specificity – can the method detect the target in the matrix?Selectivity – can the method differentiate between the target and other

analytes

in the matrix?

Robustness – can the method be repeated on the same sample with the same results?Slide23

Quantitative Data Evaluation

Detection Limits, Specificity, Selectivity, and RobustnessPrecision – Does the method produce the same results as established method?Linearity – Does the method produce proportional results with in the customer’s target range?Slide24

Personnel Considerations

Personnel designing and approving verifications and validations should be knowledgeable about the technology as well as the needs of the labSlide25

Personnel Considerations

Personnel carrying out verification and validations should be competent in all techniques Work on verification and validation studies can be used to demonstrate competency but should not be used as trainingSlide26

Pit-falls of Method Validation

Repeating tests until you get the result you want Drifting from method validation into method developmentAdjusting criteria for acceptance AFTER

the data is obtainedSlide27

Overcoming Pit-falls

Have a well thought out, documented and approved plan and stick to itGo back and start over, incorporating any changes into new validation

All data should still be included in evaluationSlide28

Documentation

Approval of plan prior to runningRaw DataInterpretation of raw data Final approval from customer

Reference validation in SOP

If SOP is a modified Standard Method, list deviations from the Standard Method in a section in the SOPSlide29

References

U. S. Department of Health and Human Services, Food and Drug Administration (2014, August 29). Volume II – Methods, Method Verification and Validation ORA-LAB.5.4.5. Retrieved from

http://www.fda.gov/ScienceResearch/FieldScience/ucm171877.htm

U. S. Department of Health and Human Services, Food and Drug Administration (2011, September 8). Guidelines for the Validation of Analytical Methods for the Detection of Microbial Pathogens in Foods. Retrieved from

http://

www.fda.gov/downloads/ScienceResearch/FieldScience/UCM273418.pdf

Slide30

Questions