Travis R Goodwin 1 Dina DemnerFushman 1 Kin Wah Fung 1 and Phong Do 2 1 Lister Hill National Center for Biomedical Communications US National Library of Medicine ID: 1033395
Download Presentation The PPT/PDF document "Drug-Drug Interaction Extraction from St..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1. Drug-Drug Interaction Extraction from Structured Drug LabelsTravis R. Goodwin1, Dina Demner-Fushman1, Kin Wah Fung1, and Phong Do21 Lister Hill National Center for Biomedical Communications, U.S. National Library of Medicine2 Office of Health Informatics, U.S. Food and Drug Administration
2. MotivationPractical – needed to support FDA operationsInteresting – challenging NLP problemsImpactful – successful approaches will help avoid DDIs
3. Mechanisms of Drug-Drug InteractionA drug interaction results when the effects of a drug are altered in some way by the presence of another drug, by food, or by environmental exposure.The risk of developing an adverse drug reaction (ADR) secondary to a drug–drug interaction increases significantly with the number of medications a patient is receiving.Mechanisms of drug interactions:A pharmacodynamic interaction results when a drug interferes with a second drug at its target site, or changes in some way its anticipated pharmacologic response.Results in additivity, synergy, or antagonism of the intended effect. E.g., the synergism that results from combining two or more anti-infectives in the treatment of a resistant pathogen. A pharmacokinetic interactions occur when one drug alters the absorption, distribution, metabolism, or elimination of another drug, thereby changing its concentration in plasma and, consequently, at the targeted site of action. Clinically significant drug interactions are most often due to alterations in pharmacokinetics, secondary to modulation of drug metabolism. http://www.sciencedirect.com/topics/pharmacology-toxicology-and-pharmaceutical-science/drug-interaction
4. Goal: generate structured (index) document for distinct DDIs in SPLStructured Product Labels(SPLs)VHA National Drug File Medication Reference Terminology (MED-RT)Pharmacokinetic effectsPrecipitantsSpecific effects
5. Relevant SPL sectionsBoxed WarningContraindications Dosage and AdministrationDrug InteractionsLaboratory Test InteractionsDrug InteractionsPrecautionsWarnings and PrecautionsWarnings Clinical Pharmacology
6. Related work An attempt to combine all publically available PDDIs sources.The merged dataset consists of fourteen different sources of PDDIs information.Ayvaz S, Horn J, Hassanzadeh O, Zhu Q, Stan J, Tatonetti NP, Vilar S, Brochhausen M, Samwald M, Rastegar-Mojarad M, Dumontier M. Toward a complete dataset of drug–drug interaction information from publicly available sources. Journal of biomedical informatics. 2015 Jun 30;55:206-17.
7. TasksTask 1 [Mention]: Interacting Drugs/Substances & Specific Interactions Micro-average on surface forms (i.e., text) Task 2 [Relation]: Interaction Type (unspecified, pharmacokinetic, pharmacodynamics) & Outcome Micro-average for sentences Task 3 [Linking/Normalization]: Substance UNII, Drug class MED-RT, Outcomes SNOMED CT, PK types NCI ThesaurusMacro-average Task 4 [Document]: Distinct fully coded Interactions (Index file)Macro-average
8. Data406 drug labels325 training (180 + 22 + 57 + 66)81 testingDailyMed XML Basic XMLOnly maintain sectionsEntitiesPrecipitants: Drug, Drug class, Substance Interaction Triggers (annotated but not evaluated)Interaction EffectsInteractionsCaution (Unspecified)Increase / Decrease (Pharmacokinetic)Specific (Pharmacodynamic)“Index file”
9. NLM-180 Training set
10. FDA-22 training and test sets annotation tool
11. Annotations
12. SystemAffiliationT1T2T3T4IBMResearchIBM Research✓✓INK_BCShandong University of Finance and Economics (Chinese: 山东财经大学)✓✓UTDHLTRIThe Human Language Technology Research Institute (HLTRI) atthe University of Texas at Dallas (UTD)✓✓✓✓SRCBRicoh Software Research Center (Beijing) (Chinese: 理光软件研究所(北京)有限公司).✓✓✓Participants
13. Approachesbi-directional LSTM-CRFCRF & RNN CNN-based encoder + RNN-based decoder + dictionary- and rule-based methodsBiLSTM-CRF + Piecewise Attention-LSTM + learning to rank (RankSVM)SVMBiLSTMCNN
14. Results Task 1 (Mention)TeamRunPrecision (%)Recall (%)-score (%)IBMResearch173.4058.9465.38IBMResearch372.9858.8965.18IBMResearch273.0257.9564.62SRCB170.9356.5262.91SRCB372.4655.5262.87SRCB271.3355.8162.62UTDHLTRI324.7060.3035.04UTDHLTRI116.7967.8226.92UTDHLTRI216.7967.8226.92INK BC118.1528.7322.25TeamRunPrecision (%)Recall (%)IBMResearch173.4058.9465.38IBMResearch372.9858.8965.18IBMResearch273.0257.9564.62SRCB170.9356.5262.91SRCB372.4655.5262.87SRCB271.3355.8162.62UTDHLTRI324.7060.3035.04UTDHLTRI116.7967.8226.92UTDHLTRI216.7967.8226.92INK BC118.1528.7322.25
15. Results Task 2 (Relation)TeamRunPrecision (%)Recall (%)-score (%)IBMResearch158.2942.3149.03IBMResearch359.0840.9848.39IBMResearch257.8341.5148.33SRCB254.7040.8446.77SRCB153.8441.3246.76SRCB353.8441.3246.76UTDHLTRI219.7645.0927.48UTDHLTRI319.7645.0927.48UTDHLTRI119.9344.3427.50INK BC13.6184.511 4.016TeamRunPrecision (%)Recall (%)IBMResearch158.2942.3149.03IBMResearch359.0840.9848.39IBMResearch257.8341.5148.33SRCB254.7040.8446.77SRCB153.8441.3246.76SRCB353.8441.3246.76UTDHLTRI219.7645.0927.48UTDHLTRI319.7645.0927.48UTDHLTRI119.9344.3427.50INK BC13.6184.511 4.016
16. Results Task 3 (Normalization)TeamRunSentence-levelSPL-LevelPrecision (%)Recall (%)-score (%)Precision (%)Recall (%)-score (%)SRCB370.8858.4962.3976.1366.6269.35SRCB167.5559.3761.4373.3967.8669.03SRCB265.7856.4959.4371.6766.8767.65UTDHLTRI321.5754.4828.6637.9962.0743.81UTDHLTRI115.2062.8422.5327.7366.4336.06UTDHLTRI215.2062.8422.5327.7366.4336.06TeamRunSentence-levelSPL-LevelPrecision (%)Recall (%)Precision (%)Recall (%)SRCB370.8858.4962.3976.1366.6269.35SRCB167.5559.3761.4373.3967.8669.03SRCB265.7856.4959.4371.6766.8767.65UTDHLTRI321.5754.4828.6637.9962.0743.81UTDHLTRI115.2062.8422.5327.7366.4336.06UTDHLTRI215.2062.8422.5327.7366.4336.06
17. Results Task 4 (Output for FDA)TeamRunPrecision (%)Recall (%)-score (%)UTDHLTRI213.4131.8817.56UTDHLTRI313.4131.8817.56UTDHLTRI113.5529.6917.32TeamRunPrecision (%)Recall (%)UTDHLTRI213.4131.8817.56UTDHLTRI313.4131.8817.56UTDHLTRI113.5529.6917.32
18. ConclusionGoal: evaluate and draw attention to the important problem of identifying DDIs in drug labelsDDIs are described differently in different sectionsInternational participationThe tasks are challenging, but improved from previous year!Task 1: 12.36% (relative; 7.19% absolute) increase Task 2: 8.80% (relative; 3.97% absolute) increase Task 3: 236.47% (relative; 48.74% absolute) increase Task 4: 30% (relative; 4.09% absolute) increase* Relative increases are computed as the percent change from the top 2018 score to the top 2019 score using the top score on the first test set from 2018 as the reference.
19. AcknowledgmentsFunding:FDA/NLM Interagency AgreementsNLM Grant: "Addressing gaps in clinically useful evidence on drug-drug interactions" (R01LM011838)NLM Intramural Research ProgramAnnotators:Mark Sharp, Phong Do, Farinaz Beniesfahany, Wujin Kim, Mohammed Abuassi, Melissa Teng, Markos Gebru, Jessica Kim, Julia Xu, Kin Wah Fung, Britney Ann Stottlemyer and Amy Grizzle Development: Soumya GayenNIST