Transcript Discovering Novel Adverse Drug Events Using Natural

Discovering Novel Adverse Drug
Events Using Natural Language
Processing and Mining of
Electronic Health Records
Carol Friedman, PhD
Department of Biomedical Informatics
Columbia University
July 21 - AIME 2009
Motivation: Severity of Problem
• Clinical trials do not test a broad population
• Adverse Drug Events (ADEs) world-wide
problem
• *Expense from ADEs is $5.6 billion annually
• *Estimated that over 2 million patients
hospitalized due to ADEs
• *ADEs are fourth leading cause of death
*In US alone
July 21 - AIME 2009
Motivation: Limitations of Approaches
• Manual review of case reports (Venulet J 1988)
• Spontaneous reporting to designated agency
(Evans JM 2001; Eland IA 1999; Wysowski DK 2005)
–
–
–
–
–
Serious ADEs reported less than 1-10% of time
Reporting is voluntary for physicians/patients
Recognition of ADEs is highly subjective
Difficult to determine cause of ADE
Biased by length of time on market and other
factors
– Cannot determine number of patients on drug or
percent at risk
• Drug prescribing/claims data (Hershman D 2007; Ray
WA 2009)
July 21 - AIME 2009
Severity of Under Reporting
Study showed 87% of time physicians
ignored patient reports of known ADEs
(Golumb et al. Physicians response to patient reports of
adverse drug effects. Drug Safety 2007)
July 21 - AIME 2009
Related Work
• Automated methods mainly based on
spontaneous reporting databases
– Most methods use (Evans SJ 2001; Szarfman A 2002)
• Surrogate observed-to-expected ratios
• Incidence of drug-event reporting compared to background
reporting across all drugs and events
• Some research aimed at improving
effectiveness of SPR databases
– Create ontology of higher order adverse events
• MedDRA
– Avoid fragmentation of signal
July 21 - AIME 2009
Related Work
• Pharmacoepidemiology databases used to
confirm suspicions
– General practice research database (GPRD) (Wood
& Martinez 2004)
– New Zealand Intensive Medicines Monitoring
(IMMP) (Coulter 1998)
– Medicine Monitoring Unit (MEMO) (Evans et al. 2001)
• EHR databases used to find signals (Brown JS et
al. 2007; Berlowitz DR et al. 2006; Wang X et al. 2009)
– Mainly coded data used
– Has potential for active real time surveillance
– Should reduce biased reporting
July 21 - AIME 2009
Related Work
• Consortiums involving multiple EHRs
– EU-ADR project (http://www.alert-project.org/)
– eHealth initiative
(http://www.ehealthinitiative.org/drugSafety/)
• Related work using EHR to detect known
ADEs – not aimed at discovering novel ADEs
(Bates DW 2003; Hongman B 2001)
July 21 - AIME 2009
Exploiting the Electronic Health Record
Text notes
primary
care
D
A
T
A
inpatient
progress
specialties
Applications
admit
history
Labs
bun
83
inr
1.3
hct
22
…
…
Centralized
Data
NLP +
Integration
Orders
lasix
…
pepcid
…
…
…
July 21 - AIME 2009
Executable
Data
•Decision support
•Patient Safety
•Acquire knowledge
•Discovery
•Guidelines
•Surveillance
•Patient management
•Clinical Trial
recruitment
•Improved
documentation
•Quality assurance
The Electronic Health Record (EHR)
• Rich source of patient information
• Mostly untapped
• Primary use for EHR
– Documenting care in multi-provider environment
– Manual review by providers
• More complete than coded ICD-9 codes
– Symptoms
– Clinical conditions not beneficial for billing
• Fragmented
• Heterogeneous
• Noisy
July 21 - AIME 2009
Research Opportunities: NLP Issues
• Occurrence of clinical events in natural language
– Drugs, diseases, symptoms
– Temporal information is critical
• Irregularity of reports
– Section headings important but abbreviated/missing
– Use of indentation, lists, run on sentences
– Tables & semi-structured data in reports
• Abbreviations
– 2/2 meaning secondary to
– co meaning cardiac output or complaining of
• Mapping terms in text to an ontology/controlled
vocabulary
– infiltrate in chest x-ray means chest infiltrate
Julylimited
21 - AIMEthan
2009 language
– ontology terms more
Research Opportunities: Statistical Issues
• Find associations between drug, symptoms,
and diseases
– Not explicit in EHR
• Large volumes of data
– Statistical significance vs. clinical significance
• Statistical associations – not relationships
– Drug treats condition / Drug causes condition
• Integrating time sequences is important
– For treats: condition must precede drug event
– For causes: drug event must precede condition
July 21 - AIME 2009
Research Opportunities: Statistical Issues
• Confounding (indirect associations)
– Metolazone treats heart failure (HF)
– HF is manifested by shortness of breath (SOB)
– Metolazone and SOB indirectly related
• Higher order associations
– Drug interactions: Drug1, drug2, condition
– Drug-contraindications: Drug, disease, condition
• Rare ADEs
July 21 - AIME 2009
Other Research Opportunities:
Knowledge Acquisition
• Structured Knowledge bases
– UMLS relations (may_be_treated_by)
– Proprietary ones – usually unavailable
• Text/Semi-Structured Knowledge (need NLP)
– Spontaneous reporting databases: indications,
drugs, adverse events
– Literature (Medline)
– Web sites (WebMD, Micromedix)
– Online medical textbooks
– Claims Data (Health IT payors)
July 21 - AIME 2009
Text Mining for Knowledge Acquisition
• Statistical methods: co-occurrences
– Discovered associations between diseases and
diets from literature (Weeber M 2002)
– Identified disease candidate genes ( Hristovski D 2005)
• NLP systems
– Trends in medications based on the literature and
narrative clinical reports (Chen ES 2007, 2008)
– Semantic relations in the literature (Hristovski D 2006)
July 21 - AIME 2009
Overview of Our NLP-EHR based
Pharmacovigilance System
Narrative
records
Coded
data
MedLEE NLP
Standardize &
integrate
EHR
Selecting &
filtering
Detect
associations
Medical
knowledge
Eliminate
confounding
July 21 - AIME 2009
ADE Signals
Natural Language Processing of EHR
Narrative
records
Coded
data
MedLEE NLP
Standardize &
integrate
EHR
Selecting &
filtering
Detect
associations
Medical
knowledge
Eliminate
confounding
July 21 - AIME 2009
ADE Signals
Meds:
Tegretol xr
Zocor
All:
Several sz meds
PMHx:
sz d/o - well controlled on tegretol
high chol - on zocor
CAD - 60% lesion in LADM by cath
MR - secondary to mitral prolapse
PSHx:
rib fx in 2001, shoulder fx secondary to trauma
Vitals: 130/80 12 80
A/P: 54 y/o m with mult med problems, all relatively well controlled. Pt sz
free, not anemic as of 2/2003. Concerned of MR and its possible long
term effects.
July 21 - AIME 2009
Coded Output from NLP
med:tegretol xr
sectname>> report medication item
code>> UMLS:C0592163_Tegretol XR
med:zocor
sectname>> report medication item
code>> UMLS:C0678181_Zocor
.........
problem:mitral valve regurgitation
sectname>> report past history item
code>> UMLS:C0026266_Mitral Valve Insufficiency
……..
problem:rib fracture
date>> 2001
sectname>> report past history item
July 21 - AIME 2009
Coding Issues
• Not all conditions have codes
– Non-communicative
• Some conditions are combinations of codes
– Difficulty sleeping
– Vascular injury
• Granularity of coding system
– Many different codes for a concept
Asthma: asthma exacerbation, asthma disturbing
sleep, moderate asthma, suspected asthma, …
July 21 - AIME 2009
Standardizing Coded Data
Narrative
records
HCT:20
Coded
data
MedLEE NLP
Standardize &
integrate
C0744727:
low hematocrit
EHR
Selecting &
filtering
Detect
associations
Medical
knowledge
Eliminate
confounding
July 21 - AIME 2009
ADE Signals
Standardizing Coded EHR Data:
Laboratory Tests and Medications
• Lab values denoting normal/abnormal vary
– Abnormal range may depend on age, sex, ethnicity, weight
– Change in lab values and duration must be considered
• Standardizing medications is complex & requires
additional knowledge
– Tradename to generic (Avandia  rosaglitazone)
– Handling of combination medications
• 1.5% Lidocaine with 1:200,000 Epinephrine
– Handling of dose & Route
• Diazepam 2 MG Oral Tablet
July 21 - AIME 2009
Selecting and Filtering
Narrative
records
Coded
data
MedLEE NLP
Standardize &
integrate
EHR
Selecting &
filtering
• Select using UMLS classes
(diseases, medications)
Filter out:
•negations, past info, …
• wrong time order
Detect
associations
Medical
knowledge
Eliminate
confounding
ADE Signals
July 21 - AIME 2009
Selecting and Filtering
• Dependence on accuracy of semantic
classification
– UMLS classification errors
- Finding: birth history, cardiac output, divorce
+ Finding: cardiomegaly, fever
• Temporal information difficult to obtain
– An adverse drug event should only follow drug event
– Processing of explicit time information is complex and vague
• Yesterday, last admission, 2/5
– Information typically occur in reports without dates
July 21 - AIME 2009
Detect Associations
Narrative
records
Coded
data
MedLEE NLP
Standardize &
integrate
EHR
Selecting &
filtering
Detect
associations
Medical
knowledge
Eliminate
confounding
July 21 - AIME 2009
• Obtain event frequencies
•Co-occurrence frequencies
•Form 2x2 tables
•Calculate associations
ADE Signals
Detect Associations
• Correct temporal sequence is critical
– Drug event should precede adverse event
– Dates are not usually stated along with events
– Section of reports helpful surrogate
• Statistical associations correspond to different
clinical relations
– For pharmacovigilance:
• Want drug causes adverse event
• Confounding caused by dependencies in data
July 21 - AIME 2009
Confounding Interdependencies
Disease
Manifested by
Treats
Drug
Cause_ADE
July 21 - AIME 2009
Adverse
Event
Confounding Interdependencies
HD
SOB
ML
ML: Metolazone; HD: Hypertensive Disease; SOB: Shortness of Breath
July 21 - AIME 2009
Drug Associations Network
Rx1-n
ADE
treatment
Rx
ADE
Sx
association
Sx1-n
process
treatment
process
Dx1-n
Dx
association
July 21 - AIME 2009
Reduce Confounding
Narrative
records
Coded
data
MedLEE NLP
Standardize &
integrate
EHR
Selecting &
filtering
Detect
associations
Medical
knowledge
Eliminate
confounding
July 21 - AIME 2009
ADE Signals
Reduce Confounding
• Collect knowledge from external sources and
associations
– Drug-treat-disease
– Disease-manifested by-symptom
– Drug-interacts with-drug
• Use Information theory
– Mutual Information (MI)
– Data processing inequality
MI3 < (MI1, MI3)
Disease
MI2
MI1
Drug
MI3
July 21 - AIME 2009
Adverse
Event
Initial Study: Methods
•
6 drugs chosen
–
–
–
•
•
•
•
•
Ibuprofen, Morphine, Warfarin: longtime on market with
known ADEs
Bupropion, Paroxetine, Rosiglitazone: ADEs discovered after
2004
1 drug class: ACE inhibitors
25,074 textual discharge summaries in 2004 from
NYPH processed using MedLEE NLP
Reference standard created using expert knowledge
sources
Drug-potential ADE pairs determined
Recall/precision calculated
Qualitative analysis performed to classify drugpotential ADE pairs detected
July 21 - AIME 2009
Initial Study: Results
• Quantitative
– recall (.75), precision (.30)
• Qualitative analysis: potential drug-ADE pairs
a. Known drug-ADEs: 30%
b. Drug-indication pairs: 30%
c. Remote drug-indication pair: 33%
d. Unknown clinical associations: 6%
July 21 - AIME 2009
Confounding Interdependencies
Disease
Disease2
Manifested by
Treats
Drug
Cause_ADE
July 21 - AIME 2009
Adverse
Event
Study 2: Reduction of Confounding
• Evaluation set
• 14 associations related to 2 drugs from Study 1
• Reference standard
• Drug-ADE associations determined and MI, DPI used
to automatically classify them
Drug-ADE Relation
Direct
Side effects of the drug (Rosiglitazone-headache)
Indirect
Conditions related to the disease/symptoms the drug
treats (Metolazone-shortness of breath)
Either
Conditions in both ‘direct’ and ‘indirect’ categories
(Rosiglitazone-chest Pain)
July 21 - AIME 2009
Results
• Precision
• 0.86 when handling confounding
• 0.31 when without handling confounding
July 21 - AIME 2009
Discussion: Limitations
& Future Directions
• Mutual information only strategy to handle
confounding
– More complex MI strategy will be explored
– Other statistical/knowledge based methods will be explored
• Inpatient data only/sicker patient population
– The same methods could be used for outpatient data as well possibly more noisy
• Drug dosage, drug-drug and more complex
interactions should be explored
July 21 - AIME 2009
Discussion: Limitations
& Future Directions
• Small evaluation data set
– More comprehensive evaluation
• Limitations inherent from NLP, coding,
association detection
• Limitations due to fragmented/incomplete
patient data
July 21 - AIME 2009
Summary
• Need for more pharmacovigilance research
– Based on the EHR
– Using available databases and text
• Studies demonstrated promising results
• Many interesting research opportunities
–
–
–
–
–
Natural language processing
Statistical methods
Integrating different sources of data
Gathering knowledge from different sources
Automated knowledge acquisition for evidence
based medicine
July 21 - AIME 2009
Acknowledgement
• NLP Data Mining group at DBMI at Columbia
–
–
–
–
–
–
–
–
George Hripcsak
Marianthi Markatou
Herb Chase
Xiaoyan Wang
David Albers
Jung-wei Fan
Lyudmila Shagina
Noemie Elhadad
• Grants
–
–
–
–
R01 LM007659 from NLM
R01 LM008635 from NLM
R01 LM06910 from NLM
5T15LM007079 from NLM training grant
July 21 - AIME 2009
QUESTIONS
THANK YOU!
July 21 - AIME 2009