NAM Digital Learning Collaborative AI and the
Description: NAM Digital Learning Collaborative AI and the Future of Continuous Health Learning Improvement Workgroup Publication Introduction DLC AI Future of Continuous Health Learning and Improvement Workgroup Original Charter: to explore the
Related Topics
Download Presentation
"NAM Digital Learning Collaborative AI and the" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
slide1. NAM Digital Learning Collaborative AI and the Future of Continuous Health Learning & Improvement Workgroup – Publication Introduction<br>
slide2. DLC AI & Future of Continuous Health Learning and Improvement Workgroup Original Charter: to explore the fields of AI and their applications in health and health care
strategies to enhance data integration to advance healthcare AI
practical challenges to AI model development and implementation
opportunities for accelerating progress<br>
slide3. Initial Workgroup Membership Sonoo Thadaney, Stanford (Workgroup Co-chair)
Michael Matheny, Vanderbilt (Workgroup co-chair)
John Burch, JLB Associates
Wendy Chapman, University of Utah
Jonathan Chen, Stanford University
Len D’Avolio, Cyft
Sharam Ebadollahi, IBM Watson Health Group
Hossien Estiri, Harvard Medical School
Steve Fihn, University of Washington
Jim Fackler, John Hopkins School of Medicine
Seth Hain, Epic
Brigham Hyde, Precision Health Intelligence
Edmund Jackson, HCA
Hongfang Liu, Mayo Clinic
Doug McNair, Cerner
Eneida Mendonca, University of Wisconsin Madison
Sean Khozin, FDA
Matthew Quinn, HRSA
Robert E. Samuel, Aetna
Bob Tavares, Emmi Solutions
Howard Underwood, Anthem)
Daniel Yang, Moore Foundation Jonathan Perlin, CMO HCA, DLC Co-Chair
Reed Tuckson, Tuckson Health Con., DLC Co-Chair
Wendy Nilsen, NSF
Joachim Roski, Booz Allen Hamilton
Howard Underwood, Anthem
Daniel Yang, Moore Foundation
Doug Badzik, Department of Defense)
Carlos Blanco, National Institute on Drug Abuse
Paul Bleicher, OptumLabs
Carla Brodley, Northeastern University
Tim Estes, Digital Reasoning
Daniel Fabbri, Vanderbilt University Medical Center
Kenneth R. Gersing, NIH
Michael Howell, Google
Brigham Hyde , Precision Health Intelligence
Javier Jimenez, Sanofi
Jennifer MacDonald, VA
Nigam H. Shah, Stanford)
David Sontag, MIT
Noel Southall, NIH
Shawn Wang, Anthem
Maryan Zirkle, PCORI<br>
slide4. NAM Workgroup Publication Objectives & Scope Develop a reference document for model developers, clinical implementers, clinical users, and regulatory and policy makers to:
understand strengths and limitations of AI/ML
promote use of these methods and technologies within the healthcare system
Highlight areas of future work needed in research, implementation science, and regulatory bodies to facilitate broader use of AI/ML in healthcare<br>
slide5. NAM DLC AI Publication: Organization TOPIC Leads
NAM DLC Jonathan Perlin, Reed Tuckson
NAM Program Office Danielle Whicher, Mahnoor Ahmed
Publication Editors Sonoo Thadaney, Michael Matheny
Chapter 1: Introduction Sonoo Thadaney, Michael Matheny
Chapter 2: History of AI Edmund Jackson, Jim Fackler
Chapter 3: Promise/Opportunities for AI Joachim Roski, Wendy Chapman
Chapter 4: Pitfalls/Challenges for AI Eneida Mendonca, Jonathan Chen
Chapter 5: AI Development & Validation Hongfang Liu, Nigam Shah
Chapter 6: AI Deployment in Clinical Settings Steve Fihn, Andy Auerbach
Chapter 7: Regulatory & Policy Issues Doug McNair, Nicholson Price
Chapter 8: Conclusions & Key Needs Sonoo Thadaney, Michael Matheny<br>
slide6. AI: What Do We Mean? https://www.legaltechnology.com/latest-news/artificial-intelligence-in-law-the-state-of-play-in-2015/<br>
slide7. “Health & Healthcare” Settings Direct Encounter-Based Care
“Non-Traditional” Settings: CVS, Home
Population Health Management
“Back Office” Healthcare Administration
Patient/Consumer Facing Technologies<br>
slide8. Target Audiences Direct Care Providers
Patients and their Caregivers
Healthcare System Leadership & Admin
Data Scientists (Developers)
Clinical Informatics (Implementers)
Legislative & Regulatory Bodies
Third Party Payors<br>
slide9. Chapter 2: History & Current State of AI Discusses history of AI with examples from other industries
Summarize the growth, maturity, and adoption in healthcare as compared to other industries.
Target general audience<br>
slide10. Chapter 3: Promise & Potential Impact of AI Focus on the utility of AI for improving healthcare delivery
Discuss near-future opportunities and potential gains from the use of AI
Target General Audiences<br>
slide11. Chapter 4: Potential Unintended Consequences of AI Focus on the potential unintended consequences of AI on:
work processes
culture
equity / fairness
patient-provider relationship
workforce composition & skills
Target General Audiences<br>
slide12. Chapter 5: AI Modeling Development & Validation Most Technical Chapter
Topics
process for developing and validating models
choice of data, variables, model complexity
performance metrics, validation
Target Model Developers<br>
slide13. Chapter 6: Deploying AI in Clinical Settings Focus on implementing and maintaining AI within ‘production’ healthcare domains
Address issues of:
Software development
Integration into a Learning Healthcare System
Applications of Implementation Science
Model Maintenance & Surveillance over Time
Target Healthcare System Leaders & Implementers<br>
slide14. Chapter 7: Regulatory & Policy Considerations Summarize key legislative and regulatory considerations for the use of AI in health care
Identify strengths and weaknesses in current framework
Discuss legal liability concerns
Make recommendations to address gaps<br>
slide15. Chapter 8: Conclusions & Key Needs build on and summarize key & cross-cutting themes from previous chapters
Recommend key areas for:
Moving the field forward
Highlight over-arcing these from chapters<br>
slide16. Publication Timeline NAM Meeting 11/2017
Publication Workgroup Kick-Off 02/2018
Content Scope Established 05/2018
Chapter Outlines Completed 07/2018
Chapter Draft Versions 09-12/2018
NAM Meeting 01/2019
Publication Revisions 01-02/2019
NAM/External Reviews 03/2019
Tentative Release 04/2019<br>
slide17. Mental Framework for This Meeting Out of Scope: Discussion of Major Content Additions/Subtractions
In Scope: Changes to Framing / Addressing Imbalance / Voice of Chapters
In Scope: Focus on Recommendations
Identify and discuss modifications, additions, and subtractions as each chapter is discussed
Be mindful of a desired balance between stakeholder groups (patients, providers, administrators, regulatory bodies, etc.)
If you felt like the opportunity to discuss a point passed and major themes, please send it to us in an email, or write it down and give it to us during a break
mahmed@nas.edu, dwhicher@nas.edu<br>
slide18. Thank You NAM Leadership
Victor Zhau
Michael McGinnis
DLC Leadership
Jonathan Perlin
Reed Tuckson
NAM Staff Leads
Danielle Whicher
Mahnoor Ahmed
DLC Clinical AI Workgroup Members<br>
slide2. DLC AI & Future of Continuous Health Learning and Improvement Workgroup Original Charter: to explore the fields of AI and their applications in health and health care
strategies to enhance data integration to advance healthcare AI
practical challenges to AI model development and implementation
opportunities for accelerating progress<br>
slide3. Initial Workgroup Membership Sonoo Thadaney, Stanford (Workgroup Co-chair)
Michael Matheny, Vanderbilt (Workgroup co-chair)
John Burch, JLB Associates
Wendy Chapman, University of Utah
Jonathan Chen, Stanford University
Len D’Avolio, Cyft
Sharam Ebadollahi, IBM Watson Health Group
Hossien Estiri, Harvard Medical School
Steve Fihn, University of Washington
Jim Fackler, John Hopkins School of Medicine
Seth Hain, Epic
Brigham Hyde, Precision Health Intelligence
Edmund Jackson, HCA
Hongfang Liu, Mayo Clinic
Doug McNair, Cerner
Eneida Mendonca, University of Wisconsin Madison
Sean Khozin, FDA
Matthew Quinn, HRSA
Robert E. Samuel, Aetna
Bob Tavares, Emmi Solutions
Howard Underwood, Anthem)
Daniel Yang, Moore Foundation Jonathan Perlin, CMO HCA, DLC Co-Chair
Reed Tuckson, Tuckson Health Con., DLC Co-Chair
Wendy Nilsen, NSF
Joachim Roski, Booz Allen Hamilton
Howard Underwood, Anthem
Daniel Yang, Moore Foundation
Doug Badzik, Department of Defense)
Carlos Blanco, National Institute on Drug Abuse
Paul Bleicher, OptumLabs
Carla Brodley, Northeastern University
Tim Estes, Digital Reasoning
Daniel Fabbri, Vanderbilt University Medical Center
Kenneth R. Gersing, NIH
Michael Howell, Google
Brigham Hyde , Precision Health Intelligence
Javier Jimenez, Sanofi
Jennifer MacDonald, VA
Nigam H. Shah, Stanford)
David Sontag, MIT
Noel Southall, NIH
Shawn Wang, Anthem
Maryan Zirkle, PCORI<br>
slide4. NAM Workgroup Publication Objectives & Scope Develop a reference document for model developers, clinical implementers, clinical users, and regulatory and policy makers to:
understand strengths and limitations of AI/ML
promote use of these methods and technologies within the healthcare system
Highlight areas of future work needed in research, implementation science, and regulatory bodies to facilitate broader use of AI/ML in healthcare<br>
slide5. NAM DLC AI Publication: Organization TOPIC Leads
NAM DLC Jonathan Perlin, Reed Tuckson
NAM Program Office Danielle Whicher, Mahnoor Ahmed
Publication Editors Sonoo Thadaney, Michael Matheny
Chapter 1: Introduction Sonoo Thadaney, Michael Matheny
Chapter 2: History of AI Edmund Jackson, Jim Fackler
Chapter 3: Promise/Opportunities for AI Joachim Roski, Wendy Chapman
Chapter 4: Pitfalls/Challenges for AI Eneida Mendonca, Jonathan Chen
Chapter 5: AI Development & Validation Hongfang Liu, Nigam Shah
Chapter 6: AI Deployment in Clinical Settings Steve Fihn, Andy Auerbach
Chapter 7: Regulatory & Policy Issues Doug McNair, Nicholson Price
Chapter 8: Conclusions & Key Needs Sonoo Thadaney, Michael Matheny<br>
slide6. AI: What Do We Mean? https://www.legaltechnology.com/latest-news/artificial-intelligence-in-law-the-state-of-play-in-2015/<br>
slide7. “Health & Healthcare” Settings Direct Encounter-Based Care
“Non-Traditional” Settings: CVS, Home
Population Health Management
“Back Office” Healthcare Administration
Patient/Consumer Facing Technologies<br>
slide8. Target Audiences Direct Care Providers
Patients and their Caregivers
Healthcare System Leadership & Admin
Data Scientists (Developers)
Clinical Informatics (Implementers)
Legislative & Regulatory Bodies
Third Party Payors<br>
slide9. Chapter 2: History & Current State of AI Discusses history of AI with examples from other industries
Summarize the growth, maturity, and adoption in healthcare as compared to other industries.
Target general audience<br>
slide10. Chapter 3: Promise & Potential Impact of AI Focus on the utility of AI for improving healthcare delivery
Discuss near-future opportunities and potential gains from the use of AI
Target General Audiences<br>
slide11. Chapter 4: Potential Unintended Consequences of AI Focus on the potential unintended consequences of AI on:
work processes
culture
equity / fairness
patient-provider relationship
workforce composition & skills
Target General Audiences<br>
slide12. Chapter 5: AI Modeling Development & Validation Most Technical Chapter
Topics
process for developing and validating models
choice of data, variables, model complexity
performance metrics, validation
Target Model Developers<br>
slide13. Chapter 6: Deploying AI in Clinical Settings Focus on implementing and maintaining AI within ‘production’ healthcare domains
Address issues of:
Software development
Integration into a Learning Healthcare System
Applications of Implementation Science
Model Maintenance & Surveillance over Time
Target Healthcare System Leaders & Implementers<br>
slide14. Chapter 7: Regulatory & Policy Considerations Summarize key legislative and regulatory considerations for the use of AI in health care
Identify strengths and weaknesses in current framework
Discuss legal liability concerns
Make recommendations to address gaps<br>
slide15. Chapter 8: Conclusions & Key Needs build on and summarize key & cross-cutting themes from previous chapters
Recommend key areas for:
Moving the field forward
Highlight over-arcing these from chapters<br>
slide16. Publication Timeline NAM Meeting 11/2017
Publication Workgroup Kick-Off 02/2018
Content Scope Established 05/2018
Chapter Outlines Completed 07/2018
Chapter Draft Versions 09-12/2018
NAM Meeting 01/2019
Publication Revisions 01-02/2019
NAM/External Reviews 03/2019
Tentative Release 04/2019<br>
slide17. Mental Framework for This Meeting Out of Scope: Discussion of Major Content Additions/Subtractions
In Scope: Changes to Framing / Addressing Imbalance / Voice of Chapters
In Scope: Focus on Recommendations
Identify and discuss modifications, additions, and subtractions as each chapter is discussed
Be mindful of a desired balance between stakeholder groups (patients, providers, administrators, regulatory bodies, etc.)
If you felt like the opportunity to discuss a point passed and major themes, please send it to us in an email, or write it down and give it to us during a break
mahmed@nas.edu, dwhicher@nas.edu<br>
slide18. Thank You NAM Leadership
Victor Zhau
Michael McGinnis
DLC Leadership
Jonathan Perlin
Reed Tuckson
NAM Staff Leads
Danielle Whicher
Mahnoor Ahmed
DLC Clinical AI Workgroup Members<br>