Transcript Clauson2008-Wikipedi.. - Social Security Online

Presented by Adam Martin
on September 27, 201o
Background
Scope, Completeness, and Accuracy of Drug Information in Wikipedia.
Kevin A Clauson, Hyla H Polen, Maged N Kamel Boulos, and Joan H
Dzenowagis. Annals of Pharmacotherapy, Dec 2008; 42: 1814 - 1821.
• Clauson is Associate Professor in the Pharmacy Practice Department at Nova
Southeastern University, College of Pharmacy.
• Interested in informatics:
•
“Pharmacists' duty to warn in the age of social media”
• “Readability of patient and health care professional targeted dietary supplement
leaflets used for diabetes and chronic fatigue syndrome”
• “Performance of online drug information databases as clinical decision support tools
in infectious disease medication management”
• Annals of Pharmacotherapy impact factor: 2.45
Relationship to Other Literature
 Health Information (Quality and Seeking Behavior)
 Giles J. Internet encyclopaedias go head to head. Nature
2005;438:900-1.
 “The average science entry in Wikipedia contained around four
inaccuracies; Britannica, about three.”
 Spate of Wikipedia articles in JAMIA:
 Seeking Health Information Online: Does Wikipedia Matter?
 “Based on its search engine ranking and page view statistics, the English
Wikipedia is a prominent source of online health information compared
to the other online health information providers studied.”
 Wikipedia and osteosarcoma: a trustworthy patients' information?

“The quality of osteosarcoma-related information found in the English
Wikipedia is good but inferior to the patient information provided by
the NCI”
Motivating the Study
 “Consumers search the Internet using suboptimal
techniques”(1814)
 “Wikipedia is a frequently accessed resource by consumers
seeking health information on the Web”(1814)
 “When information about drugs is in written form, it can
impact patients’ attitudes and behaviors”(1815)
 “(1) many articles are written anonymously, (2) many
contributors are students still studying the very topics
about which they write, (3) some Wikipedia authors selfidentify as ‘pre-hospital care workers,’ and (4) anyone can
purposefully add misinformation”(1815)
Purpose of the Study
 “To evaluate the scope, completeness, and accuracy
of Wikipedia for content that one-third of all Internet
health-seekers search for: information on
medications” (1815)
On Method
 Comparator Database:
 “had to be a freely accessible general drug information database that had been
previously evaluated and found to be broad in both scope and depth of drug
information that was gathered from authoritative references”(1815) and nonwiki.
 Medscape Drug Reference was selected as comparator.
 Categories and Questions:
 Eight Categories: “administration, adverse drug events, contraindications,
dosage, drug interactions, indications, mechanism of action, and use in
pregnancy and lactation”(1815)
 Each with ten questions covering topics like “inpatient and outpatient care,
non–Food and Drug Administration (FDA)-approved indications, novel
medications, and recently approved dosage changes”(1816)
 “Answers were verified with gold standard and authoritative references
used in general practice in the US, not including the comparator
database”(1816)
More on Method
 Evaluation:
 Scope: “If an answer was present, a score of 1 was
assigned; conversely, if an answer was absent, a score of
zero was given and it was also classified as an error of
omission.”(1816)
 Accuracy: “Factually erroneous answers (ie, those that
conflicted with the answer key) were also counted and
characterized.”(1816)
 Completeness: “Completeness was assessed using a 3point scale, with 3 being the most complete and 1 being
the leas complete.”(1816)
Still More on Method
 Authors also evaluated previous previous Wikipedia
entries.
 Compared “current” to the entries of 90 days prior.
 Only evaluated scope and completeness. Why not
accuracy?
Results: Scope
 “Wikipedia was able to answer significantly fewer
(40.0%) of the total number of drug information
questions compared with MDR (82.5%; p <
0.001).”(1816)
 “Wikipedia was able to answer more questions in the
indication category (60.0%) than was MDR
(50.0%)”.(1816)
 It “tied MDR in the scope score for the mechanism of
action category (80.0%)”.(1816)
 “Wikipedia was unable to answer any questions on
dosage, versus the MDR score of 90.0%”.(1816)
Results: Completeness
 “Answers in Wikipedia were significantly less complete
compared with those of MDR (p < 0.001).”(1817)
 “The answers that Wikipedia was able to provide were
only 76.0% complete, while MDR was able to provide
answers that were 95.5% complete.”(1817)
 “Wikipedia did not outscore MDR in completeness for
any of the drug information categories.”(1817)
Results: Accuracy
 “No factually inaccurate answers were provided by
Wikipedia, whereas 4 answers in MDR conflicted with the
answer key.”(1817)
 In MDR, the “two errors were due to lack of timely
updates for newly approved FDA indications, and 2 errors
occurred because the database provided conflicting
information in different sections of the monograph.”(1817)
 “Fourteen errors of omission were recorded for MDR and
48 errors of omission were noted with Wikipedia.”(1817)
 Ommission breakdown: contraindications = 7, pregnancy and
lactation = 7, drug interactions = 6, adverse drug reactions = 5
Results: Current vs. Prior Wiki
Entries
 “Scope scores between current and prior Wikipedia
evaluations were 40.0% and 33.8%, respectively”(1817)
at p = .024.
 “Prior page version overall completeness scored
76.5%, while current versions scored 76.0%”(1817) but
p = .77.
Discussion
 “While Wikipedia provided factually accurate drug information,
it was incomplete, much more likely to contain errors of
omission, and thus, of more limited scope than the information
available in MDR.”(1817)
 Wikis “may be perceived to be edited by altruistic ‘peers’ who
improve the content with each edit. This trust may spill over to
health and drug information, where it might be misplaced,
given that commercial considerations can drive content and
increase the risk of abuse.”(1818)
 “An important drawback of user-edited sites is that free and
anonymous editing potentially allows amateur or erroneous
entries, fraudulent material, or conflict of interest (whether
by inclusion or omission) to go unnoticed or unchallenged. In
particular, the increasing use of nonvalidated resources is
thought to pose a risk for consumers”(1818)
Limitations and Reflections
 “They didn't really give a reason WHY the change in scope
MAY have increased during the 90 days time, or if it would
be expected to continue.” – LK
 “Why was the comparison database needed instead of just
comparing Wikipedia to the gold standard references?” –
MS
 “This article seems to say we should denigrate wikipedia as
a source. I would argue instead we should be educating
people on how to search and recognize authoritative and
appropriate sources”. – BH
 The study concerns information seeking behavior but
doesn’t take into account actual consumer questions or
behaviors (e.g., ability to negotiate various interfaces).
Limitations and Reflections, Cont’d
 “It is disturbing that due to search engine manipulation, wiki may be one of
the very first "hits" a consumer gets when searching, so publicizing sites like
Pub Drug, RX WIKI, etc. should have more emphasis for the general public
to navigate to these sites rather than a general non-health credentialled site
like wiki. The issue if that with the number of intranet users geometrically
increasing, it is like letting people drive without a license, and then trying
to teach them the rules of the road - how can you possibly catch up at this
point? There does need to be some input by health professionals and
organizations like AMA, or American Academy of Family physicians, or
American Nurses Association…” – LK
 “My biggest problem is that wikipedia is a reference source, often pointing
you to good sources of information. So, when they evaluated the answers did
they read all the linked materials provided in that entry? Do we know if
average consumers do this or not?” – BH
 “Previously, physicians and pharmacists have been considered the "keepers
of the information", now the explosion of information all over the web, has
changed that role. Who is responsible for providing the literacy? Proliferation
of many authoritative sites often makes it difficult too, which one to choose?” LK