Transcript comp15_unit7_lecture_slides

Comp 15 - Usability &
Human Factors
Unit 7 - Decision Support
Systems: a Human Factors
Approach
This material was developed by Columbia University, funded by the Department of Health and Human Services, Office of the National Coordinator
for Health Information Technology under Award Number 1U24OC000003.
Outline
Understanding Human Decision Making
Clinical Decision Support Systems (CDSS)
Computer Provider Order Entry Systems and CDSS
• Promise and Pitfalls
• Factors that facilitate and impede CDSS tools
Barriers
Improving Design
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
2
Patient Safety
Landmark Institute of Medicine Report:
• 98,000 preventable deaths due to human error
• Eighth leading cause of death
• Adverse drug events (ADEs) are estimated to injure or kill more than 770 000 people
in hospitals annually
ADEs: any harm resulting from medication whether due to adverse drug
reaction or medical error
Complexities of medication management pose a significant safety risk for
hospitalized patients.
Each phases of medication process, namely prescribing, dispensing,
administration, and monitoring, provide opportunities for confusion or error
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
3
Human Factors Approach
Focus
Goal
• Human beings and their
interactions with
products/equipment, tasks
and environments
Component 15/Unit 7
• Design systems and system
components to match the
capabilities and limitations of
humans who use them;
optimize working and living
conditions
Health IT Workforce Curriculum
Version 2.0/Spring 2011
4
Understanding Decisions
Decision involves 3 components:
• Choice options & courses of actions
• Beliefs about objective states, processes & events in the world, including
outcomes states & means to achieve them
• Desires, values or utilities that describe the consequences associated with
the outcomes of each action-event combination
Good decisions effectively choose means available to
achieve the goals
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
5
Medical Decision Making
Research
Two sets of objectives:
• 1. Understand how clinicians & patients
make decisions in experimental & "realworld" settings
• 2. Develop ways to facilitate the decision
process (paper-based guidelines, computerassisted decision-support technologies and
training in decision methods)
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
6
Heuristics and Biases
Heuristics are rules of thumb for making decisions
• Intuitive and rapid
• Error prone
Biases systematic deviations from normative
standards
• Representativeness
• Availability
Biases impact the process of decision making & have
been well-documented in health-related decisions
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
7
Hindsight Bias
Occurs when decision makers inflate the probability
of a prior judgment (e.g., diagnose a patient) on the
basis of subsequent available information
Studies: Estimate the probability of a diagnosis for
a given clinical case: 4 potential diagnoses
• With/without knowledge of correct diagnosis
• Probability judgment should be the same
• W knowledge, physicians inflate probability
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
8
Hindsight Bias: So What?
Learning
• If physicians assume they would have predicted a
clinical outcome, they may fail to learn from a case.
• Unusual or noteworthy cases presented at grand
rounds
Error attribution
• Given hindsight, errors can seem glaringly obvious
• Failure to appreciate the context of an error
episode or other mitigating factors
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
9
Confirmation Bias
Bayes’ Theorem: Normative
theory for hypothesis testing
• Probability of a diagnosis is
determined by combining
prior probability of event with
the information value (relative
likelihood) of each piece of
relevant information
Component 15/Unit 7
Confirmation bias:
Overconfidence in one’s
judgment causes decision
maker to favor one hypothesis
over another
• Selectively attend to data and
not give adequate weight to
alternatives
Health IT Workforce Curriculum
Version 2.0/Spring 2011
10
The Cost of Confirmation
Bias
Desire to confirm a favorite son
hypothesis may contribute to
inefficiency
• Ordering of additional laboratory tests of limited
diagnostic values
Results of tests serve to reinforce bias
(clinician’s confidence) without actually
altering likelihood of diagnosis
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
11
Classic DM Problem
(Eddy, 1982)
•
•
•
•
Estimate the probability that a woman has breast cancer given that she has a
positive mammogram on the basis of the following information:
– The probability that a patient has breast cancer is 1%. (This provides the
prior probability)
– If the patient has breast cancer, the probability that the radiologist will
correctly diagnose it is 80% (This provides the sensitivity or hit rate)
– If the patient has a benign lesion (no breast cancer), the probability that the
radiologist will misdiagnose it is 9.6% (This provides the false positive rate).
What is the probability that a patient with a positive mammogram actually has
breast cancer?
Probability of breast cancer is only 7.8%, while Eddy reports that 95 out of 100
95 out of 100 doctors estimated this probability to be between greater than 75%
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
12
Framing Effect
Alternative representations
of a problem can give rise to
different judgments and
preferences
Component 15/Unit 7
Preference for a particular
course of action is different
when a problem is posed in
terms of potential gain rather
potential loss even though
the underlying situation is
identical
Health IT Workforce Curriculum
Version 2.0/Spring 2011
13
Survival vs. Mortality
• McNeil et al (1982) presented a hypothetical lung cancer
decision scenario to physicians and patients
– The treatment options were radiation therapy, which had an
immediate higher survival (lower mortality) rate, but a lower 5
year survival rate.
– Frame 1: treatments were described in terms of survival rates
– Frame 2: treatments were described in terms of mortality rates
• Results:
– Survival frame, clear preference for surgery,
– Mortality frame, the two choices were preferred almost equally.
• One possible explanation is that the positive framing
leads to more risk averse choices, while the negative
framing increases risk-seeking decision making
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
14
DM in Naturalistic Settings
Embedded in a broader social context
Decision-action cycle affected by monitoring and feedback
High volume and multiple streams of information
Situation assessment and serial evaluation of options
Changing, ill-defined, or competing goals
Substantial stress, time pressure, high risk
Multiple players coordinate decisions and actions
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
15
Decision Support Systems
Interactive computer-based systems help individuals use
communications, data, documents, knowledge and models to solve
problems and make decisions
DSS are auxiliary systems
• Intended to assist human decision makers rather than replace them
• Not a fully automated system
Designed for specific types of organizations including banks, insurance
companies and hospitals
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
16
Clinical Decision Support
Systems
Provides clinicians, staff, patients, with knowledge and person-specific
information, intelligently filtered or presented at appropriate times, to
enhance health and health care.
• Uses patient data to generate case-specific advice
Primary purpose assist clinicians at point of care
Designed to aid decision making for prevention, screening, diagnosis,
treatment, drug dosing, test ordering, and/or chronic disease
management
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
17
Star Trek Tricorder: The Ultimate
Clinical Decision Support Tool
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
18
Forms of CDSS Advice
Alerts
Reminders
Structured
order forms
Pick lists
Patientspecific dose
checking
Guideline
support
Medication
reference
information
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
19
The Case for Clinical
Decision Support
Knowledge base regarding effective medical therapies
continues to improve
• Practice of medicine continues to lag behind
CDS achieve the following objectives:
•
•
•
•
•
•
Reduced medication errors and adverse medical events
Improved management of specific acute and chronic conditions
Improved personalization of care for patients
Best clinical practices consistent with medical evidence
Cost-effective and appropriate prescription medication use
Effective communication and collaboration across
clinical/prescribing/dispensing/administering settings
• Better reporting and follow-up of adverse events
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
20
Degrees of CDSS
Computerization
The computer:
• Offers no assistance
• Offers a complete set of action
alternatives
• Narrows the selection
• Suggests one action alternative
• Executes that selection if the human
approves
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
21
Degrees of Computerization
Continued (6-10)
The computer:
• Allows human a restricted time to veto
before automatic execution
• Executes automatically, then informs
human
• Informs after execution only if asked
• Informs him if computer decides to
• The computer decides everything and acts
autonomously, ignoring the human
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
22
Computerized Provider Order
Entry Systems (CPOE)
Supports electronic entry of clinical orders for the treatment of patients
• Medication
• Investigative tests
Automate the medication ordering process
Orders communicated over a network to the medical staff or to the departments
• Pharmacy
• Laboratory
• Radiology- responsible for fulfilling the order
Typically includes decision support tools
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
23
Promise of Order-Entry
Systems
Ordering of drugs with computer support is a
promising application for reducing medication errors
• Most potential adverse events in patients occur at the stage of
drug ordering
CPOE offers real-time decision support, alerts and
reminders
Improvements in response time, efficiency of
dispensing and delivery of medication
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
24
Some Advantages of CPOE
Systems
Faster to reach pharmacy
Less subject to error associated with similar drug names
Easily integrated into medical records and decision-support systems
Easily linked to drug-drug interaction warnings
Able to link to ADE reporting systems
Well suited for training and education
Claimed to generate significant economic savings
With online prompts, CPOE systems can
• Link to algorithms to emphasize cost-effective medications
• Reduce under prescribing and over prescribing
• Reduce incorrect drug choices
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
25
ADrug-Drug Interaction
Scenario
“When ordering a new medication, a prescriber may not be aware that two
drugs interact, or may not be keeping in mind the other medications that the
patient is taking. As an example, consider the case of a hospitalized patient
who is being treated with venlafaxine (Effexor) for chronic depression and
develops an infection with a drug resistant bacterium requiring treatment with
linezolid, a new antimicrobial agent. The interaction between linezolid and
venlafaxine (serotonin syndrome -- altered mental status, including agitation,
confusion and coma, neuromuscular hyperactivity, and autonomic dysfunction)
is very severe but may not be known to the practitioner. While writing the order
for linezolid, an alert screen can warn the practitioner that these two drugs
should not be used together. The alert screen may offer the prescriber the
opportunity to cancel the order, to discontinue the existing medication that
interacts with the newly ordered medication, or to order a test that could detect
the interaction or monitor therapy. The alert screen may prompt the physician to
have a conversation with the patient regarding potential side effects of the
medications. Any of these consequences of the decision support software could
be beneficial.”
Kuperman et al, 2007. Online JAMIA Data Supplement
doi: 10.1197/jamia.M2170 J Am Med Inform Assoc 1 January 2007 vol. 14 no. 1 29-40
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
26
Challenges with Order Entry
Steep learning curve
Disruption of workflow and organization roles and routines
Perceived increase in completion time
• Decreases with gains in expertise
Can introduce new sources of error
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
27
CPOE Paradox
CPOE identified as important intervention to reduce
prescribing errors and yet the evidence-base for
their effectiveness is limited
Some studies have shown electronic prescribing
with CPOE significantly increases prescribing
quality in hospital inpatients
Also introduces new types of errors which have
been introduced following CPOE system
implementation
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
28
Cognitive Evaluation of
Interaction with a CDSS
Cognitive evaluation of an interaction with a CPOE system
• Focus on decision support for heparin dosing
Objective:
• characterize the interaction in terms of effectiveness
• changes to ordering behavior
• opportunities for error attributable to the interaction process
Representational format of information in alerts affects
performance and errors
• Timing of trigger in relation to workflow
• (Horsky et al, 2005)
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
29
Weight-Based Heparin
Ordering
Clinical scenario for
the administration of
weight-based heparin
• Decide on giving
bolus followed by IV
drip
• Calculate both
doses
Component 15/Unit 7
CPOE triggers
decision-support alert
• Calculates dose
automatically
Health IT Workforce Curriculum
Version 2.0/Spring 2011
30
Methods
Entry of hospital
admission orders
Cognitive walkthrough
• Step-by-step task
analysis – goals,
actions
• Completed by 2
researchers
• Characterize
information to
complete task
Component 15/Unit 7
• Completed by 7
clinicians
• Subjects instructed to
think aloud
• Screen progression
captured on video
• Transcripts of
comments coded for
analysis
Health IT Workforce Curriculum
Version 2.0/Spring 2011
31
CPOE Screen
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
32
Weight-based IV Heparin
Protocols
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
33
Results - Presentation Salience
Recognition of DS
purpose
Visual salience of
calculated values
• 2 clinicians confused
patient-specific dose
calculator with a
general guideline
• 6 clinician engaged
in extended reading
of text to derive alert
meaning
• Values cleared from
screen when users
needed to enter them
– memory recall
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
34
Results - User Behavior
Knowledge of DS availability
• 1 subject computed both doses before DS was
triggered – no indication of DS function
Dose estimated before DS
• 6 subjects used system-calculated value only as
reference – extra time, cognitive effort
Transparency of computation
• 3 subjects guessed system is using “80/18” formula
to compute weight-based dose
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
35
Summary
Suboptimal presentation format
• Extra time, few realized benefits of DS
• Inconsistent with workflow
Different representational form would enable a quick perceptual
judgment could reduce this extra cognitive effort.
• Leaving only the calculated dose in the frame with a clear description of how
the result was computed
• To take advantage, user needs to be able to invoke this feature on demand.
• May identify inefficient and error-prone screen configurations
Propose specific interface improvement
• Higher rate of decision support use
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
36
36
Role of CPOE Systems in
Facilitating Medical Errors
Koppel et al (2005) conducted multi-faceted study
investigating medication errors associated with CPOE use
CPOE system facilitated 22 types of medication error.
• Many appeared to occur with great frequency
Errors classified into:
• Information errors generated by fragmentation of data and failure to
integrate the hospital’s information systems
• human-machine interface flaws reflecting machine rules that do not
correspond to work organization or usual behaviors
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
37
Information Errors: Fragmentation
and Systems Integration Failure 1
Assumed Dose Information
• House staff rely on CPOE displays to determine
minimal effective or usual doses.
• dosages listed in display based on the
pharmacy’s warehousing and not clinical
guidelines.
• For example, normal dosages are 20 or 30 mg,
the pharmacy might stock only 10-mg doses, so
10-mg units are displayed on the CPOE screen
• Clinicians select inappropriate doses
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
38
Information Errors: Fragmentation
and Systems Integration Failure 2
Medication Discontinuation Failures
• Ordering new or modifying existing medications is
usually a separate process from canceling
• Without discontinuing the current dose, physicians
can increase or decrease medication
• Add new but duplicative medication
• Medication-canceling ambiguities exacerbated by
interface and multiple-screen displays of medications
• Viewing 1 patient’s medications may require 20
screens
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
39
Human-Machine Interface
Flaws
Patient Selection
• Easy to select wrong patient file because names and drugs are close together,
the font is small, and, patients’ names do not appear on all screens.
Unclear Log On/Log Off
• Physicians can order medications at computer terminals not yet "logged out" by
the previous physician
• result in either 1) unintended patients receiving medication or 2) patients not
receiving the intended medication
Failure to Provide Medications After Surgery
• When patients undergo surgery, CPOE cancels their previous medications.
• Physicians must reenter CPOE and reactivate each previously ordered
medication
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
40
Automation Bias
Clinicians using automated decision follows the aid
directs regardless of knowledge to the contrary
Errors of Omission:
• Less vigilant in checking drug orders because they assume the
computer will have already done the work
Errors of Commission:
• Continue with a dangerous drug order because the computer
did not alert them that the order was unsafe
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
41
Anti-Automation Bias
• Errors of dismissal, where computer
advice is ignored
• Clinicians routinely disable or ignore the
alarms or alerts on clinical monitoring
devices
– Legitimate reasons such as high false alarm
rates [or repetition of the same alarms
– Less valid reasons such as not wanting to be
interrupted
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
42
Barriers to Prescriber DecisionMaking and Clinical Workflow
Russ and colleagues
(2009) observed
medication
prescribing during
routine patient care
Through inductive
qualitative analysis,
they identified 15
barriers associated
with medication alerts
Facilitate medication
ordering by improving
the style and content
of the medication alert
display
Next 2 pages
describes 11 such
barriers
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
43
15 Barriers to Prescriber
Decision-Making
BARRIER
DESCRIPTION
1. Poor Screen
Display
Alert display does not support alert resolution and/or
prescriber workflow
2. Inadequate Alert
Specification
Alert does not provide information on why it was triggered
and/or the potential problem
3. Actual or
Perceived Lack of
Evidence
Alert is not evidence-based, does not provide a reference
to evidence that does exist
4. Unclear Level of
Risk
Alert does not provide clear information on relative risk of
harm for a given patient
5. Redundancy
Repeated alerts within the same encounter or over
multiple encounters for a given patient
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
44
Barriers Continued
6. Low Alert Signal
to Noise Ratio
Numerousness of alerts leads to information overload,
prescriber desensitization, and potential for missing key
alerts
7. Inadequate
Allergy Logic
Alert system does not distinguish between true allergies
and bothersome, but non-serious, side effects
8. Duplicate
Workload
Alert duplicates other required work processes
9. Paper
Prescriptions &
Limited CPOE
Some medications are not or cannot be entered
electronically, and therefore, are not reviewed by the alert
system
10. Unclear System Alert system does not adequately reveal its
Capabilities
capabilities/limitations to the prescriber; full functionality
of the alert system is ambiguous
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
45
Barriers Elaborated
Poor Screen Display
• Much of the alert text was in all capital letters
• multiple alerts grouped in one pop-up window
• Scroll box to view multiple alerts
Inadequate Alert Specification
• alerts did not show all clinically-relevant information needed for
decision-making
Unclear Level of Risk
• Significance of alert may be unknown (e.g., drug allergy versus
sensitivity).
• Difficult decision whether to discontinue course of treatment
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
46
Human Factors and
Information Management
Situation Awareness (SA)
• What must be known in order to complete a cognitive task
Mental Workload (MWL)
• High MWL occurs when a person’s mental capacity is exceeded
• when the mental demands imposed on the clinician because of
information overload, for example, exceed the clinician’s ability to keep it
all straight.
Poor SA and high MWL impair memory, problem identification
and decision making
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
47
Situation Awareness
Construct used in decision making research to characterize:
• awareness of what is happening around you
• and understanding what the information means to you now and in the
future
Three levels
• Perception of elements in the environment (e.g., cues/stimuli from patient
[pulse, color, weight change], chart, EHR, nurse)
• Comprehension of the meaning of those elements (by integrating the
disparate pieces of information and determining what is salient)
• Projection of future status so that decisions can be made
If SA is poor, results in impaired decision making
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
48
Mental Workload
As information management problems increase, MWL
increases
Time pressure makes it more important that CDS
automation be easy to use and useful
• Under time pressure with less time and patience to navigate
through poorly designed technology
• Under time pressure, people can adapt and still perform well by
exerting more mental effort or by concentrating harder
• Under more significant mental workload, individuals can no longer
adapt or compensate in order to maintain cognitive performance
• Demands imposed by the system (e.g., clinician needing to
remember the important facts of the most recent patient visit while
starting the next patient’s visit) exceed the attentional resources or
mental capacity of the person
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
49
CPOE/CDSS Design
Recommendations
Interfaces explicitly map to workflow patterns of clinicians
• CDSS systems must support, rather than impede, clinical workflows through speedy,
available, and usable algorithms that provide parsimonious, clear, concise, and
actionable warnings and advice
Clues in interface to optimally support users in medication ordering
Reduce layers of screens (to a maximum of 3 layers) to facilitate users
navigation
Alerts should be timed properly-the moment a clinician would himself
search for this information
Alerts displayed in a more prominent position on the screen
Organize screen-elements into logical groups, visually separated by
space and alignment
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
50
50
Designing for Better Workflow
All information should be available in a clinical information system
Clinical systems should help clinicians to see the right amount of the
right type of data wherever and whenever needed
Clinical information should be accessible in the shortest possible
amount of time
Data from disparate sources should be aggregated for completeness
• Clinicians are not forced to go to multiple different systems to obtain important
information
Clinical systems should reduce to a reasonable minimum the number
of steps required to obtain any information
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
51
Concluding Thoughts
• Computer-based decision support systems
offer great promise for reduction of errors in
medicine and facilitation of quality patient
care
• Results on system efficacy have thus far
been equivocal
– CDSS can lead to new types of errors
• Adherence to usability/human factors
principles can lead to superior design and
enhanced performance
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
52
References
Coiera E, Westbrook J, Wyatt J. The safety and quality of decision support systems. Methods Inf Med. 2006;45 Suppl
1(suppl 1):20–5.me06010020
Eddy, D. M. (1982). Probabilistic reasoning in clinical medicine: Problems and opportunities. In D. Kahneman, P. Slovic
& A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases (pp. 249-267). Cambridge, England:
Cambridge University Press.
Horsky, J., Kaufman, D. R., & Patel, V. L. (2005). When you come to a fork in the road, take it: strategy selection in
order entry. AMIA Annu Symp Proc, 350-354.
Institute of Medicine Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National
Academy Press, 2001.
J.A. Osheroff, J.M. Teich, B. Middleton, E.B. Steen, A. Wright and D.E. Detmer, A roadmap for national action on clinical
decision support, J Am Med Inform Assoc 14 (2) (2007), pp. 141–145.
Karsh B-T. Clinical practice improvement and redesign: how change in workflow can be supported by clinical decision
support. AHRQ Publication No. 09-0054EF; Rockville (MD): Agency for Healthcare Research and Quality; June
2009.
Khajouei R, Jaspers MW: CPOE system design aspects and their qualitative effect on usability. Stud Health Technol
Inform 2008 , 136:309-14
Koppel, R., Metlay, J. P., Cohen, A., Abaluck, B., Localio, A. R., Kimmel, S. E., et al. (2005). Role of computerized
physician order entry systems in facilitating medication errors. JAMA, 293(10), 1197-1203.
Kuperman, G. J., Bobb, A., Payne, T. H., Avery, A. J., Gandhi, T. K., Burns, G., et al. (2007). Medication-related clinical
decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc, 14(1), 29-40.
McNeil, B. J., Pauker, S. G., Sox, H. C., Jr., & Tversky, A. (1982). On the elicitation of preferences for alternative
therapies. N Engl J Med, 306(21), 1259-1262.
Reckmann, M. H., Westbrook, J. I., Koh, Y., Lo, C., & Day, R. O. (2009). Does computerized provider order entry
reduce prescribing errors for hospital inpatients? A systematic review. J Am Med Inform Assoc, 16(5), 613-623.
Russ, A. L., Zillich, A. J., McManus, M. S., Doebbeling, B. N., & Saleem, J. J. (2009). A human factors investigation of
medication alerts: barriers to prescriber decision-making and clinical workflow. AMIA Annu Symp Proc, 2009, 548552.
Teich JM, Osheroff JA, Pifer EA, Sittig DF, Jenders RA, Panel CDSER. Clinical decision support in electronic
prescribing: Recommendations and an action plan. J Am Med Inform Assoc 2005;12(4):365-76.
Component 15/Unit 7
Health IT Workforce Curriculum
Version 2.0/Spring 2011
53