Transcript Handout

Usability & Human Factors
Designing for Safety
Lecture a
This material (Comp15_Unit10a) was developed by Columbia University, funded by the Department of Health and Human
Services, Office of the National Coordinator for Health Information Technology under Award Number 1U24OC000003.
Designing for Safety
Learning Objectives
• Apply principles underlying the design of
healthcare systems for safety (Lecture a)
• Identify common sources of error
documented in research studies in
medicine (Lecture a)
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
2
Designing for Safety
•“First do
no harm”
First principle of
medical practice
"The origin of primum non nocere." http://en.wikipedia.org/wiki/British_Medical_Journal electronic responses and commentary, 1 September
2002.
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
3
Patient Safety
IOM report (1999: To err is human): 44,000 to
98,000 die from preventable medical errors
More than vehicle accidents, breast cancer, AIDS;
8th largest cause of death
Nosocomial and Iatrogenic
Feb 2010 – half of all infection deaths attributable to
hospital
Complexity v. complications v. simple errors
Medical complexity is a main barrier to safety
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
4
Errors
Computers switched patient record spontaneously
• Huffington Post Aug 4, 2010
Pharmacy orders failed to be delivered
• >shutdown
No national mandatory monitoring procedure
6 deaths and more than 200 injuries tied to EHRs
• FDA official Jeffrey Shuren
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
5
Pediatrics:
Increased Mortality with Computerized
Physicians Order Entry (CPOE)
Mortality rate significantly increased from:
• 2.80% (39 of 1394) before CPOE implementation
• to 6.57% (36 of 548) after CPOE implementation
Multivariate analysis revealed that CPOE remained
independently associated with increased odds of
mortality (odds ratio: 3.28; 95% confidence interval:
1.94–5.55) after adjustment for other mortality
covariables
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
6
Increased Mortality: Reasons
(from Sittig, 2009)
Hospital-wide implementation of CPOE + clinical
apps in 6 days
• After CPOE implementation, order entry not allowed until patient
physically present & registered into system
• All medications centrally located in the pharmacy dept
• Because drug orders could now only be processed after nurses
activate them, nurses had to spend time at computers and away
from the bedside
Order entry and activation through computers
reduced face-to-face doctor-nurse communication
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
7
Increased Mortality: Reasons
(cont.)
Increased time burden changed organization of
bedside care
• Before CPOE, clinicians converged at bedside to stabilize patient
• After CPOE, clinician stayed at computer to enter orders in the first
15min-1hr if patient was in extremis
• Entering stabilization orders required 10 clicks/order (1-2min/single
order) compared to a few seconds for the previous written order
Majority of terminals were wireless
• Bandwidth exceeded at peak periods
• Additional delays between each click
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
8
Horsky: Dosing Error
(Detailed Analysis)
Potassium chloride (KCl)
Error was interaction among human & system
agents
• methods limited in scope to their distinct analytical domains would
not identify
Errors in several converging aspects of the drug
ordering process:
•
•
•
•
Confusing on-screen laboratory results review
System usability difficulties
User training problems
Suboptimal clinical system safeguards
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
9
Errors, Safety, Perfectibility:
Errors Viewed in 2 Ways
Person approach
System approach
• ‘Perfectibility model’
• If clinicians work hard
enough & trained, errors
will be avoided
• Blame-oriented
• Emphasis on dealing with
increased complexity with
more training (Gawande)
• Multiple contributions to
error, including work
environment
• Recognize that perfection
will not happen,
anticipate and avoid
errors, build a resilient
system
• Aviation view (after
crash)
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
10
Human Factors (Elrod, 2009)
National Patient Safety Goals from JCAHO
NPSG 3 – medication safety
• E.g. CPOE look-alike, sound-alike drugs
Impact of drug names
Anticipate and prevent errors around drug selection
Conflict notification
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
11
Design Considerations
(from Kaye, 2010)
“Error tolerance” good attributes for many devices
Features of the User Interface (UI) that prevent
activation of critical actions following minor,
incorrect actions by user
• Potential difficulties are anticipated/identified
• Design of the UI to control their likelihood
Examples:
• Request verification before proceeding
• Parameter limits (e.g., values greater than “250” not accepted)
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
12
“Use Safety” Evaluation
(Kaye 2010)
• Major issues are best addressed prior to final/validation testing
through early user involvement and “formative” evaluations
• Test protocol focused according to identified priority of tasks or
“use scenarios.”
• Environment of actual system use and design configuration are
addressed
• Performance measured meaningfully
• Performance (a.k.a. “usability”) goals, such as “80% of users were successful
indicates up to 20% failure rate.
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
13
Retrospective Incident Analysis
Originally for ID of factors leading to failure
Now also for analysis for error recovery, ID near
misses
Near misses can ID recovery factors as well
Planned recovery opportunities v. unplanned
Of 52 medication errors (->death): 127 recovery
opportunities (absent, missed, failed); 0 to 11
opportunities/error, avg. 2.4; of the 52, only 4
presented no opportunities
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
14
Order Sets
Diffusion of standardized
evidence based
protocols, particularly
when the protocol is
appropriate for a number
of areas, such as all
intensive care units
Health IT Workforce Curriculum
Version 3.0/Spring 2012
CPOE system is a
massive, institution-wide
undertaking; sharing of
sets rather than
reinvention optimal
Usability & Human Factors
Designing for Safety
Lecture a
“When order sets are
implemented without
organization standards
and clinical review or
inadequately
maintained, they
become templates for
efficiently practicing
outdated medicine on a
widespread basis.”
(Bobb 2007)
15
Controversies Surrounding Order
Sets
• A number of design features would increase the utility and safety of
the care prescribed through order sets. Individual orders within order
sets should be linked, if so desired by the client.
For example, drug A is to begin at time zero, and linked orders for
drug B and drug C begin 4 and 8 hours after drug A.
When drug A is delayed by 2 hours, drug B and C are automatically
moved back by 2 hours.
• This decreases the risk for error and amount of downstream re-work
and is particularly useful for fully integrated EHRs with online
electronic medication administration records.
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
16
Patient Controlled Analgesia
Linked orders should also prompt the
clinician to discontinue all orders
originating from an order set when
appropriate.
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Patient Controlled Analgesia (PCA)
orders typically include a number of
additional orders for patient
monitoring, rescue medications, and
medication used to treat side effects.
When the PCA order is discontinued,
the user should be asked and given
the ability to discontinue all
associated orders with a single click.
Usability & Human Factors
Designing for Safety
Lecture a
17
Examples of CPOE Design
Features
Linked orders in an order set should be mutually exclusive
(esp. high-risk drug orders)
Multiple choices but exclusion of alternates after choice
logic to present only those options appropriate for the
patient (e.g. ‘dithered’ for allergy, unavailable for wrong
sex)
Standards (based on the literature, local consensus, and
institution-specific drug formulary) for common care
elements such as surgical antimicrobial prophylaxis, deep
vein thrombosis (DVT) prophylaxis, glucose management,
post-operative nausea
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
18
No Default Selections
Bobb, et al. (2007).
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
19
Other Design Requirements
Overall template design and details:
• Consistent naming conventions that facilitate finding
• No default selections when set opened
• Pre-selected orders ok for nursing and lab orders where
the same treatment is prescribed for virtually all patients
• Set location: e.g. set name, service name, procedure
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
20
Order Set Safety
Multidisciplinary review important esp. prior to clinician
CPOE experience can discuss changes to procedures,
standardization, items not suitable to electronic formats
Author can explain intent
Misinterpretation: at best
-> re-work
Health IT Workforce Curriculum
Version 3.0/Spring 2012
At worst -> error and
patient harm
Usability & Human Factors
Designing for Safety
Lecture a
21
Review and Supervision
Review by committees; new after deployment, for
set removal when necessary
Input from established oversight committees such
as Pharmacy and Therapeutics, Critical Care,
Blood Transfusion and Quality Committees.
Rapid advance of clinical knowledge poses
problems
Overall strategy required
Basic building blocks of a decision support program
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
22
Unintended Consequences of
CPOE (Campbell et al. 2006)
1.
2.
3.
4.
more/new work for clinician;
unfavorable workflow issues
never ending system demands
problems related to paper
persistence
5. untoward changes in
communication patterns and
practices
6. negative emotions
7. generation of new kinds of
errors
8. unexpected changes in the
power structure
9. overdependence on the
technology. Clinical decision
support features introduced
many of these unintended
consequences
Identifying Unintended Adverse
Consequences (UAC) can allow
design to avoid negative
consequences
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
23
Checklists – Gawande
Atul Gawande: ‘Checklist Manifesto’ 2010
Modern medicine very complex, changing, extensive
expertise, unheard of accomplishments, but
Falls short due to lack of completeness, expert memory;
checklists are a solution
Case: girl drowned under ice brought back to life; extensive
procedures (100s) with dependencies
Building: another example of extensive complex
procedures with dependencies; checklists used to
coordinate work
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
24
Checklists – Gawande (cont.)
Resistance to checklists due to self-perception
as expert
80% of surgeons agreed after using, saw it
catch errors, 20% still resistant
‘would you want it if being operated on?’ –
93% said yes
OR checklists based on aviation
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
25
Pronovost
Authorization for nurses to prompt if 5-item checklist not followed (for line
infections)
Backup from administration critical
Infections dropped from 11% to zero in first year
2 line infections in subsequent 15 months
Prevented 43 infections, 8 deaths, saved $2mil.
Keystone initiative: involved executive visits to problem-solve in ICU; critical to
obtaining resources
Nevertheless not implemented nationwide
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
26
Pronovost (cont.)
Had to change the way teams worked together and improve communication.
“Until a junior nurse can correct a senior physician who forgot to use the
checklist, until that conversation goes well, we will continue to harm patients”
2006 NEJM paper: nearly eliminated infections in hospital in 3 months (to 0
from 2.7/1000); deployed statewide, saved 1500 lives first year, decreased
infections 66%
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
27
Pronovost – UIC
Unintended consequences of CPOE can be good (e.g. ID
unnecessary procedures) or fatal
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
28
Shabot Ten Commandments for CIS
1. Speed is everything.
2. Realize that doctors won't wait for the computer's pearls.
3. Deliver “just-in-time” information.
4. Fit into the user's workflow.
5. Respect physicians' sense of autonomy.
6. Monitor implementation in real time and respond “right now.”
7. Beware of unintended consequences.
8. Be wary of uncovering long-standing process flaws.
9. Don't disrupt “magic nursing glue.”
10. Speed is everything.
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
29
Additional Reference:
Top 10 Sentinel Events (reviewed by JCAHO
2008) by type
Event
# reviewed in 2008
Wrong-site surgery
116
Suicide
102
Delay in Treatment
82
Unintended retention of foreign body
71
Patient fall
60
Operative/Post-operative complication
63
Medication error
Assault/rape/homicide
Perinatal death/loss of function
Medical equipment-related
46
41
32
23
1.1 Table: JCAHO, (2008).
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
30
Designing for Safety
Summary – Lecture a
•
•
•
•
Patient Safety
Error
Design Requirements
Ten Commandments of Clinical
Information Systems (CIS)
• Patient Control Analgesia
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
31
Designing for Safety
References – Lecture a
References:
1. The origin of primum non nocere." British Medical Journal electronic responses and commentary. Retrieved on 1
September 2002 from http://en.wikipedia.org/wiki/British_Medical_Journal.
2. To Err is Human: building a safer health system. (1999). Institute of Medicine Report. Retrieved on September 8th,
2010 from http://iom.edu/~/media/Files/Report%20Files/1999/To-Err-isHuman/To%20Err%20is%20Human%201999%20%20report%20brief.pdf.
3. Han, YY, Carcillo, JA, Venkataraman, ST, Clark, RSB, Watson, RS, Nguyen, TC, Bayier, H., Orr, RA Unexpected
Increased Mortality After Implementation of a Commercially Sold Computerized Physician Order Entry
System. Pediatrics Vol. 116 No. 6 December 1, 2005 pp. 1506 -1512.
4. Sittig, D.F. (2009). Eight rights of safe electronic health record use. JAMA,vol.302(10), p.1111-1113.
5. Ash, J.A., Sittig, D.F., Dykstra, R., Campbell, E., Guappone, K. (2009). The unintended consequences of
computerized provider order entry: findings from a mixed methods exploration. International Journal of Medical
Informatics, vol.78 (S1), p.S69-S76.
6. Bobb, AM, Payne, TH, Gross, PA. (2007). Viewpoint: controversies surrounding use of order sets for clinical
decision support in computerized provider order entry. Journal of the American Medical Informatics Association,
Volume: 14, Issue: 1, Publisher: American Medical Informatics Association, Pages: 41-47.
7. Shabot, MM. Ten commandments for implementing clinical information systems. Proc (Bayl Univ Med Cent). 2004
July; 17(3): 265–269.
8. Bobb, AM, Payne, TH, Gross, PA. (2007). Viewpoint: controversies surrounding use of order sets for clinical
decision support in computerized provider order entry. Journal of the American Medical Informatics Association,
Volume: 14, Issue: 1, Publisher: American Medical Informatics Association, Pages: 41-47.
9. Peter Pronovost, Dale Needham, Sean Berenholtz, David Sinopoli, Haitao Chu, Sara Cosgrove, Bryan Sexton,
Robert Hyzy, Robert Welsh, Gary Roth, Joseph Bander, John Kepros, Christine Goeschel. N Engl J Med 2006;
355:2725-2732December 28, 2006.
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
32
Designing for Safety
References – Lecture a
References (cont.):
10. Gawande A. (2007). The checklist. Retrieved on September 10th, 2010 from The New Yorker, December 10
2007. Available at http://www.newyorker.com/reporting/2007/12/10/071210fa_fact_gawande?currentPage=2.
11. Elrod J, Androwich IM.(2009). Applying human factors analysis to the design of the electronic health record. Stud
Health Technol Inform; 146:132-6.
12. Kaye R. (2010). Enhancing User Performance and Avoiding Safety Problems through Analysis, Discovery,
Prioritization and Design Considering usability for Health IT systems from a safety & effectiveness
perspective. National Institute of Standards and Technology Health IT Workshop, Gaithersburg MD. July 13, 2010.
13. Campbell, ME., Sitting, D.F., Ash, J.S., Guappone, K.P. (2006). Types of unintended consequences related to
computerized provider order entry. Journal of the American Medical Informatics Association, vol.13(5), p.547-556.
14. Shabot, MM. Ten commandments for implementing clinical information systems. Proc (Bayl Univ Med Cent). 2004
July; 17(3): 265–269.
15. Joint Commission on the Accreditation of Hospital Organizations (JCAHO). A Guide to The Joint Commission's
Medication Management Standards, Second Edition (PDF book). http://www.jcrinc.com/ebooks/EBMMS02/2100/ available at http://www.jointcommission.org/NR/rdonlyres/67297896-4E16-4BB7-BF0F5DA4A87B02F2/0/se_stats_trends_year.pdf.
Images
Slide 19: Bobb, AM, Payne, TH, Gross, PA. (2007). Viewpoint: controversies surrounding use of order sets for clinical
decision support in computerized provider order entry. Journal of the American Medical Informatics Association,
Volume: 14, Issue: 1, Publisher: American Medical Informatics Association, Pages: 41-47
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
33
Designing for Safety
References – Lecture a
Table:
1.1 Table: Joint Commission on the Accreditation of Hospital Organizations (JCAHO). A Guide to The Joint
Commission's Medication Management Standards, Second Edition (PDF book). http://www.jcrinc.com/ebooks/EBMMS02/2100/ available at http://www.jointcommission.org/NR/rdonlyres/67297896-4E16-4BB7-BF0F5DA4A87B02F2/0/se_stats_trends_year.pdf.
Health IT Workforce Curriculum
Version 3.0/Spring 2012
Usability & Human Factors
Designing for Safety
Lecture a
34