Raouf E Nakhleh MD Mayo Clinic Florida Disclosure Information Raouf E Nakhleh MD I have no financial relationships to disclose Objectives Discuss evidence based guideline development and their impact on the future from a QA perspective ID: 208898
Download Presentation The PPT/PDF document "What’s new in Anatomic Pathology Quali..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
Slide1
What’s new in Anatomic Pathology Quality Assurance?
Raouf E. Nakhleh, MD
Mayo Clinic FloridaSlide2
Disclosure InformationRaouf E Nakhleh, MD
I have no financial relationships to discloseSlide3
Objectives
Discuss evidence based guideline development and their impact on the future from a QA perspective
Discuss issues related to immunohistochemistry validation
Discuss ongoing assessment of practicing pathologistsSlide4
Why these topics
Not new
Subject to regulation (USA)
Formalization of issues not previously addressedSlide5
Evidence Based Guidelines
How does one judge good medical practice?
Much easier in retrospect, particularly when there is a negative outcome.
Literature is vast with conflicting knowledge
In the past, informal evolution of practice
Emphasis on local standards
Today national standardsSlide6
Evidence Based Guidelines
In particular situation:
Guidelines provide a road map on how to proceed.
Guidelines help us judge appropriateness of care
From a practitioner’s perspective, easier to know if you are meeting your obligations
From a quality assurance perspective, easier to create measures for comparisonSlide7
Evidence Based Guidelines – Examples
In pathology
Cancer protocols and checklists
Standard set by physicians
Reporting standards
Accreditation by Commission on Cancer
Clinical Laboratory Improvement Act
Standard set by legal decree
Procedural and quality standardsSlide8
Evidence Based Guidelines
Cancer reporting
1980 report
Breast, right, mastectomy:
Infiltrating ductal carcinoma
1 of 20 lymph nodes positive
Margins free Slide9
Evidence Based Guidelines
INVASIVE BREAST CANCER
Macroscopic
1.
Specimen
(partial breast, total breast including nipple and skin, etc)
2.
Procedure
(excision without wire‐guided localization, total mastectomy, etc)
3.
Lymph Node Sampling
(no lymph nodes present, sentinel lymph nodes, etc)
4.
Specimen Integrity
(single intact specimen, multiple designated specimens, etc)
5.
Specimen Size (greatest dimension in cm)6. Specimen Laterality (right, left, not specified)Microscopic7. Tumor Size (size of largest invasive cancer, may also be based on macroscopic)8. Tumor Focality (single focus, multiple foci, etc)9. Skin extent (not present, not involved, invades dermis, etc)*10. Nipple extent (not present, not involved, DICS, Paget’s disease)11. Skeletal muscle extent (not present, present, etc)12. Ductal Carcinoma In Situ (DCIS not present, present, etc)13. Lobular Carcinoma In Situ (LCIS not present, present, etc)14. Histologic Type of Invasive Carcinoma (ductal NOS, mucinous, papillary, etc)
15.
Nottingham Score Overall Grade
(1, 2, 3, not graded)
16.
Tubular Differentiation
(1, 2, 3, not graded)
17.
Nuclear Pleomorphism
(1, 2, 3, not graded)
18.
Mitotic Count
(1, 2, 3, not graded)
19.
Margins
(involved by invasive cancer, involved by DCIS, etc)
20.
Lymph Nodes
‐ total number examined*
21.
Lymph Nodes
‐ number with micrometastases*
22.
Lymph Nodes
‐ number with macrometastases*
23.
Lymph Nodes
0 size of largest metastatic deposit*
24.
Tumor pT Stage
(pTX, pT0, etc)
25.
Regional Lymph Nodes pN Stage
(pNX, pN0, pN1a, etc)
26.
Distant Metastasis M Stage
(not applicable, pM1, etc)
Ancillary Studies
27.
Estrogen Receptor Disposition
(performed, not performed, etc)
28.
Progesterone Receptor Disposition
(performed, not performed, etc)
29.
HER2 Disposition
(performed, not performed, etc)
* If not applicable to the specimen, code as if the element was present in the reportSlide10
Evidence Based Guidelines – Laws
Physician Quality Reporting System
Tax Relief and Health Care Act of 2006
Incentive pay (up to 2% of eligible Part B) for physician who report quality measures (pay-for-reporting)
Voluntary program
2015 becomes mandatory Slide11
Physician Quality Reporting System
Colon Cancer grade and stage (2008)
Breast Cancer grade and stage (2008)
Barrett’s esophagus (2012)
Prostate grade and stage (2012)
HER2 determination in breast cancer (2012)Slide12
Accountable Care Organization (ACO)
Payment of ACO’s rewards quality and efficiency rather than volume
Monitor practice patterns and use performance data to improve the quality of care
CMS establishes performance measures
If met, then eligible for shared savings
Over time - higher standards
Use evidence-based guidelinesSlide13
Overview of Potential Quality System
Evidence based medicine
Practice Guidelines
Addition of standards
Inspection, QA review and PT of an institutions
National or local QA review
Accreditation
Improvement
Clinical studiesSlide14
Evidence Based Guidelines
What’s in our present and future?Slide15
College of American Pathologists Pathology and Laboratory Quality Center
The Center develops evidence-based guidelines and consensus statements related to the practice of pathology and laboratory medicine. Through them, we continually improve the quality of diagnostic medicine and patient outcomes.Slide16
College of American Pathologists Pathology and Laboratory Quality Center
HER2, ER and PR
before official Center creation
CAP/ADASP Consensus Statements on Effective Communication of Urgent Diagnoses and Significant Unexpected Diagnoses in Surgical Pathology and Cytopathology
(published)
Validating Whole Slide Imaging Systems for Diagnostic Purposes in Digital Pathology
(complete, not published)
CAP/ASCCP Lower Anogenital Squamous Terminology (LAST) Standardization Project for HPV-associated Lesions
(complete, not published)
CAP/IASLC/AMP Molecular Testing Guidelines for Selection of Lung Cancer Patients for EGFR and ALK Tyrosine Kinase Inhibitors
(near completion)Slide17
Guidelines in the works
Immunohistochemistry (IHC) Assay Analytic Validation Principles
CAP/ASH Algorithm for Initial Work-up of Acute Leukemia
ASCP/CAP/AMP/ASCO Molecular Markers for the Evaluation of Colorectal Cancer
CAP-ADASP Interpretive Diagnostic Error Reduction in Surgical Pathology and Cytopathology
Bone Marrow Synoptic Reporting for Hematologic Neoplasms
CAP-NSH Uniform Labeling Requirements for Blocks and Slides in Surgical Pathology
ASCO/CAP Guideline Recommendations for HER2 Testing in Breast Cancer – Update to 2007 Edition
ASCO/CAP Guideline Recommendations for HER2 Testing in Gastric CancerSlide18
Selection Criteria for Center Topics
Patient risk, patient safety and quality; issues that affect patient care and quality outcomes
Performance characteristics of assay and ability to reach consensus. Do problems exist including high false positive/false negative rates and lack of uniformity in practice with significant ability to reach consensus?
Amount of available evidence to predict a clinical response to an agent or change a patient outcome
Guidelines are lacking but regulatory bodies or other professional organizations are trying to preempt specialty development of guidelines or approval/disapproval or show unusual interest
5. Adoption momentum of a particular assay. Consider the number of pathologists affected.
18Slide19
Selection Criteria for Center Topics
6. Feasibility of significantly changing practice of pathology or medicine (e.g., using Ultra Sound Guided FNA to change cytology practice).
7. Ability to collaborate with the right partner(s) in development of a Center product to facilitate development and acceptance, and improve overall probability of success.
8. The public's perception of the issue negatively affects the image of pathologists and the pathologists' role in medicine.
9. Evaluation of existing methods is obsolete or does not bring value in patient care
19Slide20
The Center Process for Developing Guidelines and Consensus StatementsSlide21Slide22
IHC Antibodies and FDA Classification
Antibodies are medical devices
Classification base on patient safety
Class I
Adjunctive diagnostic information
Class II
Prognostic or predictive data
Separately reported
More stringent quality assurance
Class III
Require FDA premarket approval Slide23
HER2 Guideline
Provide the appropriate HER2 assay validation procedure
Validation with FISH or with another IHC lab
Prescribe appropriate fixation time and documentation: 6 – 48 Hours
Modifies scoring criteria to enhance specificity
Ongoing competency assessment for pathologists
Enroll in HER2 proficiency testing serviceSlide24
HER2
Guideline introduced in late 2006
Mandated in 2008
LAP checklist standards related to HER2 effective in late 2008
CAP sponsored survey to determine how well labs are able to comply (late 2008)
Leads to insight of effectiveness of guideline (2010)Slide25
LAP HER2 Standards
Properly validate HER2 assays
Ensuring appropriate fixation
use the ASCO/CAP scoring criteria
Enforce HER2 proficiency testingSlide26
2008 HER2 Survey
Arch Pathol Lab Med 2010;134: 728-734
HER2 assay validation
Concordance with FISH
81% of Labs achieved 95% concordance for(-)
73% of Labs achieved 95% concordance for(+)
Concordance with another IHC LAB
72% of Labs achieved 95% concordance for(-)
68% of Labs achieved 95% concordance for(+)Slide27
2008 HER2 Survey
Fixation time and documentation
86% made changes to address fixation
60% process specimens on the weekend
Documentation is addressed in reports in 70% of laboratoriesSlide28
2008 HER2 Survey
ASCO/CAP scoring criteria
84% using criteria
Ongoing pathologists competency assessment
91% have a program in placeSlide29
2008 HER2 Survey – Summary
Most labs are meeting HER2 guideline
Gaps still exist, particularly validation
Deficiencies hopefully corrected over time with inspection cycle
Follow up survey conducted at the end of 2011, currently under analysisSlide30
Validation
What about other antibodies?
Gap in understanding of Validation in AP
Few documents address IHC antibody assay validation
Definition
Confirmation through a defined process that a method performs as intendedSlide31
31
Principles of Test Validation
Use manufacturer’s instructions
Use high-quality test materials with known target values
Test materials should be of similar type as patient samples
Complete documentation, including procedure and results
Lab director review/approve resultsSlide32
IHC Validation
For cell markers, (e.g. keratin, actin, Melan A etc.) how many cases should be performed in validation study?
Should you validate with another lab or another technique?
What level of concordance is acceptable?
Are there situations where validation is not necessary?
What about cytology material?Slide33
IHC Validation – Example
New antibody introduction – Prognostic
Example: MIB-1 (Ki-67)
Proliferation marker (estimate of mitotic count)
Antibody needs to work at spectrum of findings (0 – 100%)
Include tissues with various levels of proliferation
Carcinoid tumors of lung (0-10%)
Typical carcinoid (0-3%)
Atypical carcinoid (2-10%)
Small cell carcinoma (25-90%)
Validation cases should cover the spectrum of findingsSlide34
Immunohistochemistry Validation Procedures and Practice: A College of American Pathologists Survey of 727 Laboratories
Lindsay Hardy, MD (Boston, MA)
Raouf Nakhleh, MD (Jacksonville, FL) Jeffrey Goldsmith, MD (Boston, MA)
Patrick Fitzgibbons, MD (Fullerton, CA) Richard Eisen, MD (Greenwich, CT)
Mary Beth Beasley, MD (New York, NY) Rhona Souers, MS (Northfield, IL)Slide35
Methods
September 2010: Questionnaire
1064 surveys distributed with CAP proficiency test mailing (HER-2)
754 returned (Oct. 2010)
27 excluded
727 included
2 major sections
Non-FDA approved, non-predictive
Non-FDA approved, predictive other than HER-2Slide36
Results: Non-predictive
Validation Procedures
Percent
Laboratory has a procedure for validation of new antibodies?
68
Procedure specifies the number of cases to be used?
54Slide37
Results: Non-predictive
68% had a written procedure
86% validated the most recently introduced non-predictive marker (despite lack of procedures for some)Slide38
Results: Non-predictive
Validation Procedures
10th Pctl
Median
90th Pctl
Minimum
# Cases to validate new antibody:
Positive
2
8
20
Negative
0
5
15
Total Cases
4
13
30
Validation Procedures
10th Pctl
Median
90th Pctl
Minimum
# Cases to validate new antibody:
Positive
2
8
20
Negative
0
5
15
Total Cases
4
13
30
Acceptable concordance (%):
Positive
80
95
99
Negative
50
95
99
Validation Procedures
10th Pctl
Median
90th Pctl
Minimum
# Cases to validate new antibody:
Positive
2
8
20
Negative
0
5
15
Total Cases
4
13
30
Acceptable concordance (%):
Positive
80
95
99 Negative509599# Cases used in most recent validation: Positive2720 Negative1515Slide39
Results: Non-predictive
Most recent validation
Percent
Weakly or focally positive cases were included
40
Results confirmed by running parallel at another lab (same tissue)
18
Cases were tested on multiple days (between-run precision)
53Slide40
Results: Predictive, excluding HER2
Predictive Markers:
ER
PR
ER/PR
CD117/C-KIT
KI-67
P53
P63
PMS2
EGFR
Other
Validation Procedures
Non-Predictive
Percent
Percent
Laboratory has a procedure for validation of new antibodies?
46
68
Validation Procedures
Non-Predictive
Percent
Percent
Laboratory has a procedure for validation of new antibodies?
46
68
Procedure specifies the number of cases to be used?
65
Validation Procedures
Non-Predictive
Percent
Percent
Laboratory has a procedure for validation of new antibodies?
46
68
Procedure specifies the number of cases to be used?
65
54Slide41
Results: Predictive, excluding HER2
46% had a written procedure
75% validated the most recently introduced predictive marker other than HER-2Slide42
Results: Predictive, excluding HER2
Validation Procedures
10
th
Pctl
25
th
Pctl
Median
75
th
Pctl
90
th
Pctl
Minimum
# cases to validate new Ab:
Positive
3
5
11
20
25
Negative
1
5
10
15
25
Total Cases
5
10
25
35
50
Acceptable concordance (%):
Positive
90
90
95
95
99
Negative
50
90
95
95
99
# Cases
used
in most recent validation:
Positive
2
5
10
20
30
Negative
1
3
5
13
20Slide43
Revalidation
Validation Procedures
Non-Predictive
Percent
Percent
Procedure specifies repeat validation for:
New lot of antibody
64
66
Antigen retrieval
80
71
Detection system
81
74
Instrumentation
74
74
Fixative
78
65
Tissue processor
55
49
Validation Procedures
Non-Predictive
Percent
Percent
Procedure specifies repeat validation for:
New lot of antibody
64
66
Antigen retrieval
80
71
Detection system
81
74
Instrumentation
74
74
Fixative
78
65
Tissue processor
55
49
Procedure includes specifications for use with cytology?
42
Validation Procedures
Non-Predictive
Percent
Percent
Procedure specifies repeat validation for:
New lot of antibody
64
66
Antigen retrieval
80
71
Detection system
81
74
Instrumentation
74
74
Fixative
78
65
Tissue processor
55
49
Procedure includes specifications for use with cytology?
42
37Slide44
Summary
IHC validation: 727 labs
Validation was performed at higher rates than the availability of written procedures
Labs are generally following procedures
Most meet designated minimum number of cases
Trend to slightly fewer
Guideline is needed to assist in Predictive and non-predictive IHC
Uncertainty as to the # of cases needed
Uncertainty as to when to revalidate vs. verification
Uncertainty RE validation in cytologySlide45Slide46
Ongoing assessment of practicing pathologists
Why now?Slide47
American Board of Pathologists
Time limited certification (10 yrs)
The four part MOC process requires diplomates to submit documentation in the following areas:
Part I – Professional Standing
Part II – Lifelong Learning and Self-Assessment
Part III – Cognitive Expertise
Part IV – Practice Performance AssessmentSlide48
Joint Commission Standards
Ms.08.01.01 Monitoring Performance; The organized medical staff defines the circumstances requiring monitoring and evaluation of a practitioner's performance (FPPE)
Ms.08.01.03 Use of Monitoring Information: Ongoing professional practice evaluation information is factored into the decision to maintain existing privileges, to revise existing privileges, or to revoke an existing privilege prior to or at the time of renewalSlide49
Professional Practice Evaluation
Use of objective performance data in the granting and/or maintenance of practice privileges
Ongoing Professional Practice Evaluation – OPPE
On an ongoing basis for privileged practitioners
Focused Professional Practice Evaluation – FPPE
When a practitioner is first privileged
When new privileges are first granted to an already privileged practitioner
When insufficient activity or “performance issues” are identified for a privileged practitionerSlide50
Concepts
Professional competency assessment
Is part of the peer review process (protected)
Is not publicly reported or stored in credentialing files
Is a tool for evaluation, not an adverse action
Produces some data points for assessment
Thresholds are minimum bars, not goalsSlide51
OPPE – Six Core Competencies
Patient care
Medical/Clinical knowledge
Practice-based learning and improvement
Interpersonal and communication skills
Professionalism
Systems-based practiceSlide52
OPPE - Ongoing Professional Practice Evaluation
REQUIREMENTS
Evaluation must be ongoing – More frequently than annually
Process must be clearly defined
What data (“metrics”) will be monitored
Who will be responsible for review of data
How often data will be reviewed
How performance data will be used for decision making
How data will be archived and made availableSlide53
OPPE - Ongoing Professional Practice Evaluation
POSSIBLE OUTCOMES
Privilege being evaluated may be continued, limited, or revoked
POSSIBLE ACTIONS
Continue privilege – no action required
Suspend or revoke privilege because it’s no longer performed or required
Initiate FPPE – Due to insufficient activity or identified “performance issues” with privilegeSlide54
OPPE - Ongoing Professional Practice Evaluation
WHAT METRICS ARE APPROPRIATE FOR PATHOLOGISTS?
Metrics for core vs. delineated privileges
Core privileges – Privileges broadly applicable to the majority of practitioners in a discipline
e.g. anatomic pathologists, clinical pathologists, etc.
Delineated privileges – Privileges specific to a subset of practitioners within a discipline
e.g. surgical pathologists, cytopathologists, hematopathologists, microbiologists, etc,Slide55
OPPE - Ongoing Professional Practice Evaluation
EXAMPLES OF METRICS FOR CORE PRIVILEGES IN AP OR CP
Patient care – Turnaround time for key reports
Medical knowledge – General CME credits, successful MOC
Practice-based learning and improvement – Participation in SAMs
Interpersonal and communication skills – 360° evaluation provided by clinicians, lab staff, etc.
Professionalism – Attendance at medical staff meetings
System-based practice – Membership on hospital or other organizational committeesSlide56
OPPE - Ongoing Professional Practice Evaluation
EXAMPLES OF METRICS FOR DELINIATED PRIVILEGES IN SPECIFIC AREAS
Patient care
Surgical pathology
– Turnaround time for biopsy reports
Number or % amended reports
IOC vs. permanent dx discrepancies
Cytopathology
– FNA success rate
Clinical pathology subspecialties
– timeliness of administrative and clinical reports (transfusion reaction)
All subspecialties
– Peer review of new method validations in relevant lab section(s)
Medical knowledge
Each subspecialty
– CME credits relevant to subspecialtySlide57
FPPE - Focused Professional Practice Evaluation
DIFFERENCES FROM OPPE
More restricted in scope – “focused”
Episodic rather than continuous, with finite endpoint
SIMILARITIES TO OPPE
Basic process must be pre-defined and consistent
Similar monitoring methodologies (review of records, direct observation, monitoring practice techniques, discussion with others involved in care)Slide58
FPPE - Focused Professional Practice Evaluation
CIRCUMSTANCES WHEN NEEDED
When a practitioner is first privileged
When new privileges are first granted to an already privileged practitioner
When insufficient activity or “performance issues” are identified for a privileged practitionerSlide59
Thank You!