FGAI4H-J-017-A03 E-meeting, 30 September – 2 October 2020 - PowerPoint Presentation

1. FGAI4H-J-017-A03E-meeting, 30 September – 2 October 2020Source:TG-Ophthalmo Topic DriverTitle:Att.3 – Presentation (TG-Ophthalmo)Purpose:Discussion | InformationContact:Arun ShroffE-mail: arun@xtend.aiAbstract:This PPT summarizes the content of the TDD for the TG on ophthalmology, for presentation and discussion during meeting J.

2. Meeting J Topic Group UpdateOphthalmology (TG-Ophthalmo )E-meeting, 30 September – 2 October 2020Arun Shroff, Topic Driver, TG-Ophthalmology

3. Topic Group – OphthalmologyStandardized benchmarking of artificial intelligence for Ophthalmology.Conditions/Sub-topics in this group:Diabetic Retinopathy (DR) Age Related Macular Degeneration (AMD)Glaucoma (GC) Pathological Myopia (PM)Red Eye (RE) – (Added Meeting G)

4. The Health ChallengeDiabetic Retinopathy (DR) At risk population: 422M worldwide with diabetes (2014)35%, 148M have DR 11%, 48M have Vision Threatening DR (64M by 2040) Leading cause of blindness among adults worldwideAge Related Macular Degeneration (AMD) Damages macula and impairs central vision 196M by 2020Third leading cause of vision loss overall, leading cause for those over 50

5. The Health ChallengeGlaucoma (GC) Damages optic nerve & leads to vision loss 80M by 2020Pathological Myopia (PM) Global Prevalence is 0.9% to 3.1%35% of people with myopia have High Myopia, which can develop into PMRed Eye [Added Meeting G] 2-3% visits to primary health centers & emergency facilities due to eye problems & majority are due to Red Eye. May denote more serious conditions like keratitis, iritis, glaucoma, which could lead to vision loss

6. Impact of AI Bridge acute shortage of healthcare professionals in LMICs, rural areas.Provide earlier detection and prevent vision loss for millions. Decrease healthcare costs via earlier interventions Increase overall efficiency and scalability of current screening methods.

7. Available Datasets - DREyePACS dataset: Approx 90,000 fundus images, 5 levels of severity Kaggle: (derived from EyePACS)2015 Challenge: Approx 35,000 images : 5 levels of severity Approx. 53.500 images : 5 levels of severity Aptos 2019 Challenge : 3664 images : 5 levels of severity (Aravind Eye Hospiital) MESSIDOR dataset: 1,200 images, 4 levels of severity DiaRetDB dataset: ~ 200 images marked with lesions etc

8. Available Datasets - AMD, GCAMD: AREDS dataset: (Age Related Eye Disease Study )Images from ~4700 patients : KORA dataset: (Cooperative Health Research in the Region of Augsburg (KORA) dataset,)Approx 2840 patient recordsGC: ORIGA, 650 fundus imagesRetinal fundus images for glaucoma analysis (RIGA, 760 images) ACHIKO-K (258 images)DRISHTI-GS (100 images)

9. Benchmarking: DR ClassificationsBinary: [0 (Nogradable Image)] (Optional) 1 (Non-referable Retinopathy = Normal or Mild) 2 (Referable Retinopathy = Moderate, Severe, PDR) Multi-class Classification: [0 (Nongradable Image) ] (Optional) 1(Normal) 2 (Mild) 3 (Moderate NPDR) 4 (Severe NPDR) 5 (PDR)

10. Benchmarking: AMD, GC, PM ClassificationsAMD: [0 (Image Nongradable)]1 (No/early stage AMD 2 (Intermediate/advanced stage AMD) GC: [0 (Image Nongradable.]1 (No GC)2 (GC)Optic Disk PM: [0 (Image Nongradable)]1 (No PM/HM)2 (HM: high myopia)3 (PM)

11. Benchmarking MetricsSensitivity:% of positive (disease) cases correctly classified True Positive/(True Positive + False Negative)Specificity: % of negative (normal) cases correctly classified True Negative/(True Negative + False Positive)AUC (Area Under ROC);Sensitivity Vs (1-Specificity) plotted at different points of the modelAccuracy, F1 Score Cohen’s Kappa / Quadratic Kappa Score:

12. TG-Ophthalmo Output DocumentsTDD: Topic Group Description Document (FGAI4H-J-017-A01) Topic Group Call for Participation (FGAI4H-J-017-A02)Topic Group Collaboration Site: https://extranet.itu.int/sites/itu-t/focusgroups/ai4h/tg/SitePages/TG-Ophthalmo.aspx​​

13. TG-Ophthalmo MembersCurrently 22 members – From 15 organizations & 8 countriesDedicated mailing list : fgai4htgophthalmo@lists.itu.int​Mailing List has 35 members​​

14. Topic Group HistoryMeeting B - New York (Nov 2018)AI for Ophthalmology Use Case submitted in response to the Call for ProposalsMeeting C – Lausanne, (Jan 2019)Topic Group “Ophthalmology” established (2 Members)Meeting D- Shanghai, (Apr 2019)First version of Topic Description Document (TDD) – Version 1.0 completed Meeting E to I: TDD Revised & Updated during each meetingNew members added

15. Edits and updates to TDD. Contribution received for TG-Ophthalmo from Tencent during last meeting. Working on it being integrated into the main TDD. 7 new members to the topic group: Daniel Ting MD (1st Hons) PhD, Consultant, Vitreo-retinal Service, Singapore National Eye Center, Head, AI and Digital Innovation, Singapore Eye Research InstituteDr. Karthik Srinivasan, Medical Officer, Vitreo retinal Services, Aravind Eye Hospital, Chennai.João Victor Dias, Lead Data Scientist, NTT Data Brazil, Artificial Intelligence for Health Tech and Financial Machine LearningJianrong Wu, Yanchun Zhu, Man Tat Alexander Ng and Yajun Zhang, Tencent Healthcare (Shenzhen), ChinaProgress Since Meeting I

16. Collaboration with WG-DAISAM (Data and AI solution assessment methods)​Objective to assess factors such as bias, robustness, explainability and uncertainty in AI models. Collaborated in contributing AI models, data, meta-data for testing for the Ophthalmology use case. Results of analysis to be included in a paper submitted to the ML4H workshop for NeurIPS conference. Progress Since Meeting I

17. Next StepsUpdates to TDD: Complete sections on ethics, benchmarking, reporting. Split into subtopics Incorporate new TDD Template Dataset Procurement: Undisclosed datasets needed for testing Possible sources: DR-Net, EyePACs, Aravind Eye Hospital, Verily (Google)Benchmarking:Establish methodology and implement.  Outreach / Community BuildingIncrease engagement from membersGet more experts on board and involved.

18. Thank you!