Source TGOutbreaks topic driver Title TDD new TGOutbreaks Outbreak detection Att2 Data amp Benchmarking challenges Purpose Discussion Contact Dr Stéphane Ghozzi Dr Martina Fischer ID: 933607
Download Presentation The PPT/PDF document "FGAI4H-G-013-A0 2 New Delhi, 13-15 Novem..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
FGAI4H-G-013-A02
New Delhi, 13-15 November 2019
Source:TG-Outbreaks topic driverTitle:TDD (new): TG-Outbreaks (Outbreak detection) - Att.2 - Data & Benchmarking challengesPurpose:Discussion
Contact:Dr. Stéphane GhozziDr. Martina FischerAuss AbboodRobert Koch-Institute, GermanyE-mail: GhozziS@rki.de
Abstract:
This PPT complements the content of G-013 with challenges on the use of AI for outbreak detection. This is an updated version of the presentation in document FG-AI4H-013-A01 (Zanzibar).
Slide2Topic Group: Disease Outbreak Detection
Data & Benchmarking challengesDr. Stéphane
Ghozzi, Dr. Martina Fischer, and Auss Abbood Robert Koch-Institute, Berlin, GermanyNew Delhi 14th Nov 2019
Slide3AI for Outbreak Detection - FG-AI4H
Background
Infectious disease outbreaks pose a major risk to public healthEarly detection of outbreaks can prompt fast interventionsCase data are collected by diverse surveillance systemsAI algorithms can be applied to detect aberrant case numbers based on these data collectionsAI algorithms have the potential to increase the timeliness and accuracy of outbreak detection3
Slide4AI for Outbreak Detection - FG-AI4H
Objective
Data(spatio-)temporal data (reported cases of infection, symptom counts, etc.)AI task
warning signal
hint on potential outbreak event
Detection of
aberrant
case numbers
( )
4
Slide5Objective
AI task
warning signal
hint on potential outbreak event
Test Data:
confirmed outbreak labels
Benchmarking
Detection of aberrant numbers
( )
5
Slide66
Notifiable pathogens
(Infection Protection Act)Surveillance of > 80 pathogens and > 400 countiesrecording ~ 500.000 cases/yeardetection ~ 20.000 outbreaks/yearAim:Early outbreak detection for fast
interventionlocal health agenciesdoctorslaboratories
state health agencies
Robert Koch Institute
ECDC / WHO
patient
German Reporting System
IfSG §7.1
Adenoviren
Ebolavirus
Lassavirus
Rickettsia prowazekii
Bacillus anthracis
EHEC
Legionella spp.
Rubellavirus
Bordetella parapertussis
Francisella tularensis
Leptospira interrogans
Salmonella Paratyphi
Bordetella pertussis
FSME-Virus
Listeria monocytogenes
Salmonella Typhi
Borrelia recurrentis
Gelbfiebervirus
Marburgvirus
Salmonella, sonstige
Brucella sp.
Giardia lamblia
Masernvirus
Shigella sp.
Campylobacter sp.
Haemophilus influenzae
Mumpsvirus
Trichinella spiralis
Chlamydophila psittaci
Hantaviren
Mycobacterium leprae
Varizella-Zoster-Virus
Clostridium botulinum
Hepatitis-B-Virus
Mycobacterium tuberculosis
Vibrio cholerae O 1 und O 139
Corynebacterium diphtheriae
Hepatitis-C-Virus
Neisseria meningitidis
Yersinia enterocolitica
Coxiella burnetii
Hepatitis-D-Virus
Norovirus; Stuhl
Yersinia pestis
Cryptosporidium sp.
Hepatitis-E-Virus
Poliovirus
E. coli
Influenzaviren
Rabiesvirus
Example: German reporting system
Slide77
Notifiable pathogens
(Infection Protection Act)Surveillance of > 80 pathogens and > 400 countiesrecording ~ 500.000 cases/yeardetection ~ 20.000 outbreaks/year
local health agenciesdoctorslaboratoriesstate health agencies
Robert Koch Institute
ECDC / WHO
patient
German Reporting System
IfSG §7.1
Adenoviren
Ebolavirus
Lassavirus
Rickettsia prowazekii
Bacillus anthracis
EHEC
Legionella spp.
Rubellavirus
Bordetella parapertussis
Francisella tularensis
Leptospira interrogans
Salmonella Paratyphi
Bordetella pertussis
FSME-Virus
Listeria monocytogenes
Salmonella Typhi
Borrelia recurrentis
Gelbfiebervirus
Marburgvirus
Salmonella, sonstige
Brucella sp.
Giardia lamblia
Masernvirus
Shigella sp.
Campylobacter sp.
Haemophilus influenzae
Mumpsvirus
Trichinella spiralis
Chlamydophila psittaci
Hantaviren
Mycobacterium leprae
Varizella-Zoster-Virus
Clostridium botulinum
Hepatitis-B-Virus
Mycobacterium tuberculosis
Vibrio cholerae O 1 und O 139
Corynebacterium diphtheriae
Hepatitis-C-Virus
Neisseria meningitidis
Yersinia enterocolitica
Coxiella burnetii
Hepatitis-D-Virus
Norovirus; Stuhl
Yersinia pestis
Cryptosporidium sp.
Hepatitis-E-Virus
Poliovirus
E. coli
Influenzaviren
Rabiesvirus
Example: German reporting system
number of cases
Time (calender week)
Aggregate case data into timeseries:
Slide8Example: weekly case data of Salmonella
Slide9AI for Outbreak Detection - FG-AI4H
Data sources for outbreak detection algorithms
Different surveillance systems:national/ mandatory reporting systems(syndromic) surveillance systems:Near-real-time syndromic surveillance:e.g. routine data from emergency departments and hospitals (e.g. ESEG-project in Germany)Antibiotic Resistance Surveillance (ARS project in Germany)…potential other data sources: publicly available sources (e.g. meterological data,..)online data sources (wikipedia, google clicks, HealthTweets, Twitter, etc.)near real-time symptom data by self-assessment health apps…9
Slide10AI for Outbreak Detection - FG-AI4H
Benchmarking challenges: (1) Data
Definition label ‚Outbreak‘Exact start/end-time point size of outbreaks often unknownNumber of epidemiologically connected cases?Confirmation by simple lab test or molecular analysisLabel uncertaintiesHow to deal with unlabelled outbreak cases for benchmarking? Minor peaks (with no confirmation): outbreak or random variation?Data diversityHighly diverse outbreak data patterns of the different pathogens Detection necessary per pathogen and per feature combinations (regions, risk groups,..)Test data
Needs to reflect national/international outbreak realitiesEach country relies on individual national disease surveillance systems. How do we optimally define a test set (undisclosed) to serve as a gold standard for benchmarking?10
Slide11AI for Outbreak Detection - FG-AI4H
Benchmarking challenges: (2) Metrics
Definition of epidemiologically relevant metrics for AI-algorithm evaluation: Sensitivity & Specificity How to strike the balance?precise + early detection of outbreaksminimize number of false alarmsaccounting for case numbers: missed large outbreaks penalized more than missed small onesprecise outbreak size detection Timelinessby time (days/weeks) passed by number of occurred cases before outbreak detection
Pathogen-specific specific metrics according to pattern diversity?Metrics & test data need to be usable for evaluation comparison of AI and established statistical models for outbreak detection11
Slide12AI for Outbreak Detection - FG-AI4H
Call for participation
Contributions by:Collecting labelled test data data stream directly linked to outbreak labels (expert/lab confirmed) of high value. AI models and algorithms for outbreak detection contributing to the development of a viable benchmarking frameworkGeneral support on different aspects of this topic (data, methods, benchmarking, etc.)12