Transcript Adverse Drug Event Extraction

IDENTIFYING ADVERSE DRUG EVENTS
FROM HEALTH SOCIAL MEDIA:
A Case Study on Heart Disease Discussion Forums
Xiao Liu1
Jing Liu2
Hsinchun Chen1
1 Artificial
Intelligence Lab, University of Arizona
2 Northwestern Polytechnical University
1
Outline
•
•
•
•
•
•
•
Introduction
Related work
Research gaps and questions
Research design
Experiments
Evaluation and results
Discussions and conclusions
2
INTRODUCTION
3
Introduction
• 25.6 million diagnosed heart disease patients in United States by
the year 2012 (CDC).
– 154.7 million overweight and obesity and 78 million hypertension
– A large number of population are under the risk of developing heart disease.
• Heart disease patients are vulnerable to adverse drug events due to their
advanced age, polypharmacy, and the influence of heart disease on drug
metabolism.
• The adverse drug events (ADE) potential for a particular heart
disease drug varies with the individual, the disease being treated,
and the extent of exposure to other drugs.
4
Introduction
• Listening to patients’ voice about their personal experience for
adverse drug event is critical to improve patient safety and reduce
mortality and morbidity.
• Available channels for adverse drug event reporting:
– FDA’s Adverse Event Reporting system (FAERS)
– Online patient discussion forums and social networks
• As a result, online discussion forums and social websites can
– Provide a fertile data source for researchers
– Help health professionals to identify and evaluate adverse drug events that
patients encounter (Mao et al. 2013).
5
Introduction
The table below shows sample posts from one heart disease forum. The types of
discussion related to heart disease treatments include:
Post ID
Sentence
219740
Since I started beta blockers[Drug] I have also had dizzy[Event] feelings, and I know it's
Adverse Drug Event
from the med.
20473
I was taking Hyzaar [Drug] and Norvasc [Drug] for my hypertension [Event].
84495
It is true the beta blocker [Drug] maintains a lower heart rate, but it also stabilizes the
heart rate to prevent arrhythmia [Event] (very fast heart rate that can harmful for some Prevention
individuals).
8309
236521
15628
Relation Type
Drug Indication
I have never [Negation] had dizziness [Event] with from Warfarin[Drug], beta Negated
blockers[Drug], yep they can cause dizziness.
Event
If they essentially appear with low heart rate[Event], beta blockers [Drug] usually don't Others
work.
Adverse
Drug
Metoprolol[Drug] is beta blocker, this drug blocks the B1 receptors in heart so if abrupt
withdrawal occurs then it can result in to arrhythmia[Event], so ideally it should be
withdrawn slowly.
6
Introduction
• To date, there are few studies that identify and analyze heart
disease related adverse drug events in health social media.
• The motivation of our study is to identify patient reports of
adverse drug events in heart disease discussion forums with
advanced information extraction techniques.
7
RELATED WORK
8
Related Work
• To form the basis of our research, we review the prior
studies that emphasized on mining adverse drug
events in health social media.
• We develop a taxonomy for previous studies based on
data sources, research focus, methods, and results.
Each of them will be explained in the following
section.
9
Prior Studies on Health Social Media
Methods
Previous
Studies
Leaman et
al. 2010
Data Source
Focus
Adverse
DailyStrength.com Drug Events
Nikfarjam et
Adverse
al. 2011
DailyStrength.com Drug Events
Classification
Not Applied
Medical Entity
Adverse drug event
Recognition
extraction
Lexicon based:
UMLS,
MedEffect, SIDER Co-occurrence based
Association rule
mining
Health Forums
Chee et al.
from Yahoo!
2011
Groups
Breastcancer.org,
Benton et al. komen.org,
2011
csn.cancer.org
Drugpatient
opinions
Not Applied
Ensemble
Classifier with
SVM and Naïve
Bayes
Adverse
Drug Events
Not Applied
Lexicon based:
CHV; FAERS
Yang et al.
2012
MedHelp
Adverse
Drug Events
Not Applied
Lexicon based:
CHV
Bian et al.
2012
Twitter
Adverse
Drug Events
Machine Learning: Lexicon based:
SVM
FAERS
Mao et al.
2013
Breast Cancer
Forums
Drug
switching
behaviors
Not Applied
Diabetes Forums
Adverse
Drug Events
Lexicon based:
Machine learning: UMLS, FAERS,
SVM
CHV
Liu et al.
2013
Co-occurrence based
Lexicon based:
UMLS,
MedEffect, SIDER Not Applied
Lexicon based:
CHV; FAERS
Results
Precision: 78.3%; Recall:
69.9%; F-measure:
73.9%
Precision: 70%
recall:66.3%
F-measure:67.9%
Co-occurrence based
The ensemble classifier
is able to identify drugs
for FDA's scrutiny
Precision 35.1%
Recall:77%
F-measure: 52.8%
Co-occurrence based
Promising to detect ADR
reported by FDA.
Not Applied
Co-occurrence based
Statistical learning+ rule
based classification
Accuracy: 74%; AUC
value: 0.82
Breast cancer drug sided
effects are related to
drug switching and
discontinuation
F-measure for MER: 87%
F-measure for ADE
extraction: 67%
10
Classification: 84.1%
Prior Studies on Health Social Media
Methods
Previous
Studies
Leaman et
al. 2010
Data Source
Focus
Adverse
DailyStrength.com Drug Events
Nikfarjam et
Adverse
al. 2011
DailyStrength.com Drug Events
Classification
Not Applied
Medical Entity
Adverse drug event
Recognition
extraction
Lexicon based:
UMLS,
MedEffect, SIDER Co-occurrence based
Association rule
mining
Health Forums
Chee et al.
from Yahoo!
2011
Groups
Breastcancer.org,
Benton et al. komen.org,
2011
csn.cancer.org
Drugpatient
opinions
Not Applied
Ensemble
Classifier with
SVM and Naïve
Bayes
Adverse
Drug Events
Not Applied
Lexicon based:
CHV; FAERS
Yang et al.
2012
MedHelp
Adverse
Drug Events
Not Applied
Lexicon based:
CHV
Bian et al.
2012
Twitter
Adverse
Drug Events
Machine Learning: Lexicon based:
SVM
FAERS
Mao et al.
2013
Breast Cancer
Forums
Drug
switching
behaviors
Not Applied
Diabetes Forums
Adverse
Drug Events
Lexicon based:
Machine learning: UMLS, FAERS,
SVM
CHV
Liu et al.
2013
Co-occurrence based
Lexicon based:
UMLS,
MedEffect, SIDER Not Applied
Lexicon based:
CHV; FAERS
Results
Precision: 78.3%; Recall:
69.9%; F-measure:
73.9%
Precision: 70%
recall:66.3%
F-measure:67.9%
Co-occurrence based
The ensemble classifier
is able to identify drugs
for FDA's scrutiny
Precision 35.1%
Recall:77%
F-measure: 52.8%
Co-occurrence based
Promising to detect ADR
reported by FDA.
Not Applied
Co-occurrence based
Statistical learning+ rule
based classification
Accuracy: 74%; AUC
value: 0.82
Breast cancer drug sided
effects are related to
drug switching and
discontinuation
F-measure for MER: 87%
F-measure for ADE
extraction: 67%
11
Classification: 84.1%
Prior Studies on Health Social Media
Methods
Previous
Studies
Leaman et
al. 2010
Data Source
Focus
Adverse
DailyStrength.com Drug Events
Nikfarjam et
Adverse
al. 2011
DailyStrength.com Drug Events
Classification
Not Applied
Medical Entity
Adverse drug event
Recognition
extraction
Lexicon based:
UMLS,
MedEffect, SIDER Co-occurrence based
Association rule
mining
Health Forums
Chee et al.
from Yahoo!
2011
Groups
Breastcancer.org,
Benton et al. komen.org,
2011
csn.cancer.org
Drugpatient
opinions
Not Applied
Ensemble
Classifier with
SVM and Naïve
Bayes
Adverse
Drug Events
Not Applied
Lexicon based:
CHV; FAERS
Yang et al.
2012
MedHelp
Adverse
Drug Events
Not Applied
Lexicon based:
CHV
Bian et al.
2012
Twitter
Adverse
Drug Events
Machine Learning: Lexicon based:
SVM
FAERS
Mao et al.
2013
Breast Cancer
Forums
Drug
switching
behaviors
Not Applied
Diabetes Forums
Adverse
Drug Events
Lexicon based:
Machine learning: UMLS, FAERS,
SVM
CHV
Liu et al.
2013
Co-occurrence based
Lexicon based:
UMLS,
MedEffect, SIDER Not Applied
Lexicon based:
CHV; FAERS
Results
Precision: 78.3%; Recall:
69.9%; F-measure:
73.9%
Precision: 70%
recall:66.3%
F-measure:67.9%
Co-occurrence based
The ensemble classifier
is able to identify drugs
for FDA's scrutiny
Precision 35.1%
Recall:77%
F-measure: 52.8%
Co-occurrence based
Promising to detect ADR
reported by FDA.
Not Applied
Co-occurrence based
Statistical learning+ rule
based classification
Accuracy: 74%; AUC
value: 0.82
Breast cancer drug sided
effects are related to
drug switching and
discontinuation
F-measure for MER: 87%
F-measure for ADE
extraction: 67%
12
Classification: 84.1%
Prior Studies on Health Social Media
Methods
Previous
Studies
Leaman et
al. 2010
Data Source
Focus
Adverse
DailyStrength.com Drug Events
Nikfarjam et
Adverse
al. 2011
DailyStrength.com Drug Events
Classification
Not Applied
Medical Entity
Adverse drug event
Recognition
extraction
Lexicon based:
UMLS,
MedEffect, SIDER Co-occurrence based
Association rule
mining
Health Forums
Chee et al.
from Yahoo!
2011
Groups
Breastcancer.org,
Benton et al. komen.org,
2011
csn.cancer.org
Drugpatient
opinions
Not Applied
Ensemble
Classifier with
SVM and Naïve
Bayes
Adverse
Drug Events
Not Applied
Lexicon based:
CHV; FAERS
Yang et al.
2012
MedHelp
Adverse
Drug Events
Not Applied
Lexicon based:
CHV
Bian et al.
2012
Twitter
Adverse
Drug Events
Machine Learning: Lexicon based:
SVM
FAERS
Mao et al.
2013
Breast Cancer
Forums
Drug
switching
behaviors
Not Applied
Diabetes Forums
Adverse
Drug Events
Lexicon based:
Machine learning: UMLS, FAERS,
SVM
CHV
Liu et al.
2013
Co-occurrence based
Lexicon based:
UMLS,
MedEffect, SIDER Not Applied
Lexicon based:
CHV; FAERS
Results
Precision: 78.3%; Recall:
69.9%; F-measure:
73.9%
Precision: 70%
recall:66.3%
F-measure:67.9%
Co-occurrence based
The ensemble classifier
is able to identify drugs
for FDA's scrutiny
Precision 35.1%
Recall:77%
F-measure: 52.8%
Co-occurrence based
Promising to detect ADR
reported by FDA.
Not Applied
Co-occurrence based
Statistical learning+ rule
based classification
Accuracy: 74%; AUC
value: 0.82
Breast cancer drug sided
effects are related to
drug switching and
discontinuation
F-measure for MER: 87%
F-measure for ADE
extraction: 67%
13
Classification: 84.1%
Prior Studies on Health Social Media
Methods
Previous
Studies
Leaman et
al. 2010
Data Source
Focus
Adverse
DailyStrength.com Drug Events
Nikfarjam et
Adverse
al. 2011
DailyStrength.com Drug Events
Classification
Not Applied
Medical Entity
Adverse drug event
Recognition
extraction
Lexicon based:
UMLS,
MedEffect, SIDER Co-occurrence based
Association rule
mining
Health Forums
Chee et al.
from Yahoo!
2011
Groups
Breastcancer.org,
Benton et al. komen.org,
2011
csn.cancer.org
Drugpatient
opinions
Not Applied
Ensemble
Classifier with
SVM and Naïve
Bayes
Adverse
Drug Events
Not Applied
Lexicon based:
CHV; FAERS
Yang et al.
2012
MedHelp
Adverse
Drug Events
Not Applied
Lexicon based:
CHV
Bian et al.
2012
Twitter
Adverse
Drug Events
Machine Learning: Lexicon based:
SVM
FAERS
Mao et al.
2013
Breast Cancer
Forums
Drug
switching
behavior
Not Applied
Diabetes Forums
Adverse
Drug Events
Lexicon based:
Machine learning: UMLS, FAERS,
SVM
CHV
Liu et al.
2013
Co-occurrence based
Lexicon based:
UMLS,
MedEffect, SIDER Not Applied
Lexicon based:
CHV; FAERS
Results
Precision: 78.3%; Recall:
69.9%; F-measure:
73.9%
Precision: 70%
recall:66.3%
F-measure:67.9%
Co-occurrence based
The ensemble classifier
is able to identify drugs
for FDA's scrutiny
Precision 35.1%
Recall:77%
F-measure: 52.8%
Co-occurrence based
Promising to detect ADR
reported by FDA.
Not Applied
Co-occurrence based
Statistical learning+ rule
based classification
Accuracy: 74%; AUC
value: 0.82
Breast cancer drug sided
effects are related to
drug switching and
discontinuation
F-measure for MER: 87%
F-measure for ADE
extraction: 67%
14
Classification: 84.1%
Prior Studies on Health Social Media
Methods
Previous
Studies
Leaman et
al. 2010
Data Source
Focus
Adverse
DailyStrength.com Drug Events
Nikfarjam et
Adverse
al. 2011
DailyStrength.com Drug Events
Classification
Not Applied
Medical Entity
Adverse drug event
Recognition
extraction
Lexicon based:
UMLS,
MedEffect, SIDER Co-occurrence based
Association rule
mining
Health Forums
Chee et al.
from Yahoo!
2011
Groups
Breastcancer.org,
Benton et al. komen.org,
2011
csn.cancer.org
Drugpatient
opinions
Not Applied
Ensemble
Classifier with
SVM and Naïve
Bayes
Adverse
Drug Events
Not Applied
Lexicon based:
CHV; FAERS
Yang et al.
2012
MedHelp
Adverse
Drug Events
Not Applied
Lexicon based:
CHV
Bian et al.
2012
Twitter
Adverse
Drug Events
Machine Learning: Lexicon based:
SVM
FAERS
Mao et al.
2013
Breast Cancer
Forums
Drug
switching
behaviors
Not Applied
Diabetes Forums
Adverse
Drug Events
Lexicon based:
Machine learning: UMLS, FAERS,
SVM
CHV
Liu et al.
2013
Co-occurrence based
Lexicon based:
UMLS,
MedEffect, SIDER Not Applied
Lexicon based:
CHV; FAERS
Results
Precision: 78.3%; Recall:
69.9%; F-measure:
73.9%
Precision: 70%
recall:66.3%
F-measure:67.9%
Co-occurrence based
The ensemble classifier
is able to identify drugs
for FDA's scrutiny
Precision 35.1%
Recall:77%
F-measure: 52.8%
Co-occurrence based
Promising to detect ADR
reported by FDA.
Not Applied
Co-occurrence based
Statistical learning+ rule
based classification
Accuracy: 74%; AUC
value: 0.82
Breast cancer drug sided
effects are related to
drug switching and
discontinuation
F-measure for MER: 87%
F-measure for ADE
extraction: 67%
15
Classification: 84.1%
Research Gaps
• A hybrid approach for ADE extraction
– Statistical learning and rule based filtering for ADE extraction and text
classification to extract reports based on patients’ accounts
– Proved with satisfying performance in a diabetes forum dataset (Liu et al,
2013).
– The effectiveness of this approach on datasets that have more complex
relation types hasn’t been tested.
• There is no prior study that emphasizes on adverse drug events of heart
disease, which is a major chronic disease in United States.
16
Research Question
• Based on the review of prior studies, we proposed the
following research question:
– How can we develop a high performance information
extraction framework for mining patient-reported adverse drug
events from heart disease forums?
17
RESEARCH DESIGN
18
Extracting Adverse Drug Event from
Health Social Forums
•
•
•
•
•
Patient Forum Data Collection: collect patient forum html files through a web crawler and parse
the raw files to structure data using a text parser
Data Preprocessing: remove noisy text including URL, duplicated punctuation, telephone number
etc., and sentence boundary detection
Medical Entity Extraction: identify drug entities and adverse event entities discussed in forums
Adverse Drug Event Extraction: extract the adverse drug events related discussion
Report Source Classification: extract adverse drug event reports based on patients’ experiences
19
Extracting Adverse Drug Event from
Health Social Forums
•
•
•
•
•
Patient Forum Data Collection: collect patient forum html files through a web crawler and parse
the raw files to structure data using a text parser
Data Preprocessing: remove noisy text including URL, duplicated punctuation, telephone number
etc., and sentence boundary detection
Medical Entity Extraction: identify drug entities and adverse event entities discussed in forums
Adverse Drug Event Extraction: extract the adverse drug events related discussion
Report source classification: extract adverse drug event reports based on patients’ experiences
20
Extracting Adverse Drug Event from
Health Social Forums
•
•
•
•
•
Patient Forum Data Collection: collect patient forum html files through a web crawler and parse
the raw files to structure data using a text parser
Data Preprocessing: remove noisy text including URL, duplicated punctuation, telephone number
etc., and sentence boundary detection
Medical Entity Extraction: identify drug entities and adverse event entities discussed in forums
Adverse Drug Event Extraction: extract the adverse drug events related discussion
Report Source Classification: extract adverse drug event reports based on patients’ experiences
21
Extracting Adverse Drug Event from
Health Social Forums
•
•
•
•
•
Patient Forum Data Collection: collect patient forum html files through a web crawler and parse
the raw files to structure data using a text parser
Data Preprocessing: remove noisy text including URL, duplicated punctuation, telephone number
etc., and sentence boundary detection
Medical Entity Extraction: identify drug entities and adverse event entities discussed in forums
Adverse Drug Event Extraction: extract the adverse drug events related discussion
Report Source Classification: extract adverse drug event reports based on patients’ experiences
22
Adverse Drug Event Extraction
Relation detection:
• Stanford Parser to generate the shortest dependency path between the drug and
event
• Syntactic classes and semantic classes to expand the feature set
• Shortest dependency path kernel (Liu et al. 2013)
23
Adverse Drug Event Extraction
Relation classification:
• NegEx: filter out negated adverse drug events
• FAERS: filter out drug indications
• Semantic templates based on signaling words to rule out prevention (e.g. prevent,
avoid, reduce the risk, reduce the chance of and decrease the chance’)
24
Extracting Adverse Drug Event from
Health Social Forums
•
•
•
•
•
Patient Forum Data Collection: collect patient forum html files through a web crawler and parse
the raw files to structure data using a text parser.
Data Preprocessing: remove noisy text including URL, duplicated punctuation, telephone number
etc., and sentence boundary detection.
Medical Entity Extraction: identify drug entities and adverse event entities discussed in forum.
Adverse Drug Event Extraction: extract the adverse drug events related discussion.
Report Source Classification: extract adverse drug event reports based on patients’ experiences.
25
Research Hypotheses
• H1. Statistical learning based adverse drug events extraction in patient
forums can outperform co-occurrence analysis based approaches.
– H1a. Rule based adverse drug event extraction will outperform co-occurrence
analysis based approaches.
– H1b. Conducting relation detection before relation classification will
outperform direct relation classification model.
• H2. Report source classification can improve the results of patient adverse
drug event report extraction as compared to not accounting for report
source issues.
26
EXPERIMENT
27
Experimental Dataset
• To verify the effectiveness of our framework, we investigated the forum
discussions in a well-known online health community, MedHelp
(http://www.medhelp.org).
• We selected three of the most popular heart disease discussion boards,
illustrated in the table below. There are total 251,472 posts, 66012 topics and
2,118,101 sentences.
28
Evaluation Metrics
•
•
We use standard machine-learning evaluation metrics, accuracy, precision, recall
and f-measure, to evaluate the performance of medical entity extraction, adverse
drug event extraction and report source classification. These metrics have been
widely used in information extraction studies (Li et al.2008).
The definition of the metrics are as follows:
Actual condition
Test Result
–
–
–
–
Positive
Negative
Positive
Negative
True positive(TP)
False negative(FN)
False positive(FP)
True negative(TN)
Precision=TP/(TP+FP)
Recall=TP/(TP+FN)
Accuracy=(TP+TN)/(TP+FP+TN+FN)
F-Measure=2*Precision*Recall/(Precision + Recall)
29
Evaluation
•
In order to conduct the evaluation, we manually annotated 250 sentences with
both drug and event entities. There were 462 drug and event pairs in these
sentences. To evaluate the performance of medical entity extraction more
accurately, especially the recall, we also annotated 350 sentences that with one
entity or without any entity.
Entity Type
# of Mentions
Relation Type
# of Occurrences
Drug
321
Has Relation
324
Event
343
ADE
136
Report Source
# of Sentences
Drug Indication
156
Patient Experience
160
Negated ADE
5
Hearsay
90
Prevention
22
Others
5
No Relation
138
30
Evaluation on Medical Entity Extraction
Results on Medical Entity Extraction
Precision
94.29%
81.32%
Drug
Recall
F-measure
87.33%
79.30%
73.25%
76.16%
Event
We achieved 94% precision, 81% recall and 87% f-measure for recognizing drug entity.
Our approach obtained 79% precision, 73% recall and 76% f-measure for medical event
extraction.
Our drug entity extraction performs better than the adverse event entity extraction.
31
Evaluation on Adverse Drug Event Extraction
Result of Adverse Drug Event Extraction
Accuracy
Precision
Recall
F-measure
100%
80.74%
73.53%
63.20%
45.49%
29.44%
65.36%
73.53%
69.20%
54.06%
42.74%
29.44%
CO
RC
RD+RC
 Compared to co-occurrence, direct rule-based relation classification generated higher precision
but lower recall. However, overall performance of RC is better than CO.
 kernel-based relation detection combined with rule-based relation classification(RD+RC)
performed best with 65.36 % precision and 73.53% recall among the three methods.
32
Evaluation on Report Source Classification
Performance Comparison Between RSC and without RSC
Accuracy
Precision
Recall
F-measure
100%
87.65%
91.42%
89.65%
90.53%
69.05%
52.73%
52.73%
Without RSC
RSC
 The F-measure has been improved from 69.05% to 90.53%.
33
Hypothesis Testing
•
In order to ensure that the assessment does not happen by chance, we conducted
pair wise single-sided t tests on f-measure. The p-values for the hypotheses testing for
adverse drug event extraction and report source classification are presented in the
table below.
Hypothesis No.
Hypothesis
P value for f-measure
1a
RC>CO
0.02299*
1b
RD+RC>RC
0.02565*
2
RSC> without RSC
0.000000657*
Note: Significance level *α=0.05
• H1a: RC outperforms CO (supported).
• H1b: RD+RC outperforms RC (supported).
• H2: RSC outperforms without RSC (supported).
34
DISCUSSIONS
35
Contrast of Our Proposed Framework to
Co-occurrence based approach
100%
33.86%
18.98%
Based on our approach , only 33.86% of all the relation instances contain adverse drug
events.
Among them, about 56.04% come from patient reports.
Only about 18.98% of all relation instances contain adverse drug events that come
from patient reports
36
Analysis of ADE Discussions in Patient Forums
• The most mentioned drug entity in the forums is beta blocker, a drug class
often prescribed to heart disease patients. There are 1,822 patient mentioned
medical events for beta blocker.
₋
Among them, 71% are adverse drug events, 19% are drug indications, 9% are
negated adverse drug events and 1% are preventions.
37
Analysis of ADE Discussions in Patient Forums
•


Beta blocker adverse events often co-occur with other medications. Some of them
belong to beta blocker drug class, others are co-medications.
Among the top 10 co-occurred treatments, Atenolol, Metoprolol, Tenormin, Coreg and Inderal are beta blocker
drugs. Ace inhibitor and calcium channel blocker are drug classes often used along with beta blocker to treat
heart disease. Aspirin and Verapamil are not beta blocker treatments.
Based on the analysis, we find that 50% of the adverse events related to beta blocker have other comedications, presenting great potential for identifying drug interactions from these discussions.
38
Analysis of ADE Reports in FAERS and ADE Discussions in
Patient Forums
• Comparison of top 10 reported drugs from both forum and
FAERS
 Beta blocker, calcium channel blocker and statin represented heart disease drug classes instead
of an individual drug.
 This finding indicates that heart disease patients prefer to use drug class names in discussion
instead of specific drug names.
39
Analysis of ADE Reports in FAERS and ADE Discussions in
Patient Forums
•
Comparison of top 20 most discussed adverse events related to Beta Blocker from
the forums with those from FAERS.
Anxiety
Completed Suicide
Dizziness
Bradycardia
Palpitation
Cardiac Arrest
Asthma
Death
Arrhythmias
Fatigue
Nausea
Allergy
Hypotension
Hypertension
Swollen
Drug Interation
Loss of Consciousness Myocardial Infarction Chest pain
Faint
Tachycardia
Atrial Fibrillation
Spasm
General Physical Health Deterioration
Fast heart rate
Headache
Cardiac Failure
Panic
Pneumonia
Cardiomyopathy
Asthenia
Angina
Condition Aggravated
Depression
Dyspnoea
FAERS
FORUM
 FAERS focuses on severe ADEs, such as “loss of consciousness”, “death”, while forum reports
concentrated on mild ADEs such as “dizziness”;
 Common and minor side effects that affect quality of life like fatigue, decreased libido, erectile
dysfunction, depression can lead to drug non-compliance.
 Patients’ reporting of these ADEs may explain treatment failures due to non-compliance and would
be of interest to treating clinicians and pharmaceutical companies. .
40
Analysis of ADE Reports in FAERS and ADE Discussions in
Patient Forums
•
•
There are several common ADES between FAERS and our system. As shown in the
following figure, among the top 10 reported adverse events of Beta Blocker, nine
of them were captured by FAERS. “Arrythmias” didn’t appear in the FAERS reports.
This shows that health social media adverse drug event reports can not only
capture well-established adverse events but also identify events not captured by
FAERS.
41
Conclusions
• In light of the need for mining social media for heart disease-related
adverse drug events, we developed an information extraction system
to identify patient-reported adverse drug events from major health
discussion forums.
• A series of experiments were conducted on a test bed encompassing
about 250, 000 posts from heart disease forum. The results reveal that
our system can effectively extract patient-reported adverse drug events
with high performance.
• This information extraction system borrowed some design perspectives
from our previous work on a diabetes forum (Liu et al. 2013). We
demonstrated the generalizability of this approach on different disease
discussion forums.
42
Conclusions
• Based on the analysis of extracted adverse drug events from heart disease
forums, we learn that
– Health social media provides information about adverse drug events that are not
captured by FDA’s Adverse Event Reporting System.
– Health social media adverse drug event reports also show great potential to help
identify drug interactions for heart disease treatment.
– Health social media adverse drug event reports are not biased to severe adverse
events as FAERS is.
– Patients’ reporting of these ADEs may explain treatment failures due to noncompliance and would be of interest to treating clinicians and pharmaceutical
companies.
43
References
•
•
•
•
•
•
•
•
•
•
•
•
•
De Smedt, R. H., Denig, P., van der Meer, K., Haaijer-Ruskamp, F. M., & Jaarsma, T. (2011). Self-reported adverse drug events and the
role of illness perception and medication beliefs in ambulatory heart failure patients: A cross-sectional survey. International journal of
nursing studies, 48(12), 1540-1550.
Benton A., Ungar L., Hill S., Hennessy S., Mao J., Chung A., & Holmes J. H. (2011). Identifying potential adverse effects using the web: A
new approach to medical hypothesis generation. Journal of biomedical informatics, 44(6), pp. 989-996
Mao, J. J., Chung, A., Benton, A., Hill, S., Ungar, L., Leonard, C. E., & Holmes, J. H. (2013). Online discussion of drug side effects and
discontinuation among breast cancer survivors. Pharmacoepidemiology and drug safety, 22(3), 256-262.
Chee B. W., Berlin R., & Schatz B. (2011). Predicting adverse drug events from personal health messages. In: AMIA Annual Symposium
Proceedings Vol. 2011, pp. 217-226
Leaman R, Wojtulewicz L, Sullivan R, et al. Towards internet-age pharmacovigilance: extracting adverse drug reactions from user posts
to health-related social networks. Proceedings of the 2010 workshop on biomedical natural language processing. Association for
Computational Linguistics, 2010: 117-125.
Nikfarjam A. & Gonzalez G.H. (2011). Pattern mining for extraction of mentions of Adverse Drug Reaction from user comments. In:
Proceeding of 2011 AMIA annual Symposium, pp. 1019-1026.
Yang C, Yang H, Jiang L, et al. Social media mining for drug safety signal detection. Proceedings of the 2012 international workshop on
Smart health and wellbeing. ACM, 2012: 33-40.
Liu X & Chen H. AZDrugMiner: an information extraction system for mining patient-reported adverse drug events in online patient
forums. Smart Health. Springer Berlin Heidelberg, 2013: 134-150.
Bian, J., Topaloglu, U., & Yu, F. (2012, October). Towards large-scale twitter mining for drug-related adverse events. In Proceedings of
the 2012 international workshop on Smart health and wellbeing (pp. 25-32). ACM.
MetaMap: http://metamap.nlm.nih.gov
Bunescu R C, Mooney R J. A shortest path dependency kernel for relation extraction. Proceedings of the conference on Human
Language Technology and Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2005: 724731.
SVM-light: http://svmlight.joachims.org
NegEx: https://code.google.com/p/negex
44