Extracting Symptom Burden from Clinical Narratives of Cancer Patients Using NLP - PowerPoint Presentation

Slide1

Extracting Symptom Burden from Clinical Narratives of Cancer Patients Using NLP

Meliha

Yetisgen

University of Washington

Ahmad Halwani

University of Utah

Ozlem

Uzuner

George Mason University

Treatment

DiagnosisRemission

SurvivorshipEnd of Life

pain

drowsiness

Shortness of breath

constipation

anxiety

anorexia

depression

tiredness

Most patients experience symptoms throughout their cancer journey leading to decreased quality of life

Only a minority of patients are referred to supportive oncology services.

Utilization of symptom burden information allows for prioritizing and facilitating patient referral to SOS, decreasing symptom burden in patients with cancer, improving QOL.

Symptom burden is documented in provider notes.

Anywhere between 25% to over 50% of patients on a single visit reported symptoms

Example provider noteSince last seen, Mr. Doe reports continued daily fatigue and lack of energy that is worse in the afternoon and limits his performance status; he can perform ADLs but tires easily and need to take a nap in the evening. Lower extremity

edema noted immediately after infusion (likely fluid overload) is now resolved. No chest pain or shortness of breath. His lymphadenopathy is improved but persists and his neck node continues to bother him. A week after his last cycle, he thought he felt a fever but did not take his temperature. This resolved by the next day. Appetite is decreased though oral intake is overall preserved. No chills. Daily

night sweats have resolved since starting treatment. No weight loss. The peripheral neuropathy he developed after cycle 1 (tingling in feet bilaterally, no weakness) is stable and does not bother him much.

Present symptomsAbsent symptoms

Present but Improved symptomsResolved symptoms

Present symptomsPresent symptomsPresent symptoms

Absent symptoms

Challenges Richness of language clinicians use in describing details of symptoms High cost of gold standard annotations needed to train and evaluate information extraction models Language use and reporting style differences across institutions

Aim 2: Detailed symptom extraction with NLPTwo major cancer centers:Seattle Cancer Care Alliance (SCCA)

Huntsman Cancer Institute (HCI)Two cancer patient populations:Prostate cancerDiffuse Large B Cell Lymphoma (DLBCL)Gold standard annotation with Active learning

Novel deep learning based text processing architectureEvaluation over two institutes for two cancer populations to ensure generalizability

Aim 1: Multi-institution gold standard annotation Goal: We will build Natural Language Processing technologies that can extract symptom information, along with their detailed attributes from clinical narratives.

Outcomes: Detailed symptom schema & annotation guidelines Active learning framework for sampling content rich notes to decrease the manual annotation effortFirst de-identified gold standard corpora annotated with symptoms

Outcomes:

Language modelsOpen source release of the NLP tools R21 Project – Start date May’2021

Aim 1 - Symptom Schema Trigger / Symptom Name (e.g., pain, headache) Assertion*Values: present, absent, possible, conditional, not-patient Change of state Values: no_change, worsened, improved, resolved Severity Values: mild, moderate, severe

Etiology Values: disease, treatment AnatomyCharacteristics DurationFrequency

Image source: https://www.cdc.gov/dengue/symptoms/index.html

Example Annotations

To capture this complex representation, many annotated training instances are required to train robust IE models

Gold standard annotation with Active LearningSample selection criteria:Informativeness: reduce classification uncertaintyDiversity: describe variation in selected samplesPrevious work on Social Determinants of Health gold standard corpora annotation

Lybarger K, Ostendorf M, Yetisgen M. Annotating social determinants of health using active learning, and characterizing determinants using neural event extraction. J Biomed Inform. 2021 Jan;113:103631.

Annotation planGold standard: Training set (75%) Random sampling + Active learningHeld-out Test set (25%)Random sampling: To capture true distribution of the dataset Ongoing work & Current progress: Datasets are created6 years of clinical notes of prostate cancer + DLBCL patients from SCCA and HCI

De-identified by an in-house de-identification toolAnnotation guidelines are finalized 6 medical students are working on annotation tasksQuery function design is in progress

Aim 2 - Information Extraction Framework Lybarger K, Ostendorf M, Thompson M, Yetisgen M. Extracting COVID-19 Diagnoses and Symptoms From Clinical Text: A New Annotated Corpus and Neural Event Extraction Framework. J Biomed Inform. 2021 Mar 26;:103761.

ArgumentPrecision

RecallF1Trigger0.810.850.83Assertion

0.770.800.79Change 0.450.050.09

Severity 0.450.310.37 Anatomy0.780.500.61

Characteristics0.660.250.36Duration0.540.560.55Frequency0.600.510.55

Baseline Framework

Thank you and Questions