Patient Safety Tip of the Week
May 3, 2016 Clinical Decision
Support Malfunction
We are big fans of
clinical decision support systems (CDSS) as patient safety tools, keeping in
mind that too much CDSS may lead to alert fatigue and unintended consequences.
But well-reasoned clinical decision support rules that are also adequately
tested for both validity and usability may be very effective tools.
Brigham and Women’s
Hospital (BWH) in Boston probably has the most robust CDSS of any healthcare
organization anywhere and they just reported some disturbing findings on
malfunctions of CDSS alerts (Wright
2016). Serendipitously, the lead author had noted one such alert
malfunction while he was demonstrating the CDSS. It happened to be an alert
that would remind physicians to check the TSH level in patients who had been on
amiodarone for at least a year. The research team subsequently identified three
other examples of CDSS alert malfunctions and conducted a sample survey of
CMIO’s at various hospitals and found most of them had also experienced CDSS
malfunctions.
Alarmingly, they found that the alert malfunctions were
often very difficult to detect and some had eluded detection for long periods
of time (weeks or even years!). Moreover, the causes for the malfunctions were
sometimes even more difficult to elucidate. They were, however, able to
identify several contributing factors:
- EHR software updates
- Changes to data codes or
clinical terminology
- Inadvertent changes to
logic for the rules
Changes to some of the data codes or data fields are often
made by IT staff or external vendors who are not part of the CDSS team and such
changes may not be apparent to the CDSS team members. For example, in the
TSH/amiodarone example, a change had been made to the drug code for amiodarone.
Alert malfunctions were most often first identified by
end-users. But those most often occurred when suddenly there was a spike in the
frequency of alerts for one or many alerts. Alerts that failed to fire after
malfunction were far more likely to elude detection.
The authors have several important recommendations:
- CDSS rules (for both new
and existing alerts) should be tested any time there is a major EHR
software upgrade
- Any changes in coding or
clinical terminology must be communicated to all CDSS staff
- Tools must be developed
and implemented to test downstream effects of any changes in coding or
terminology
- Proactive monitoring tools
must be developed to detect possible alert malfunctions
- There must be proactive monitoring
of any external services that may impact CDSS
- Such external systems,
such as those impacting coding or clinical terminology, need to be more
fault-tolerant and robust
- CDSS logic, rules and
alerts should be tested by a different analyst than the one who built the
content
- Better quality assurance
testing is needed to ensure CDSS’s function properly
The BWH researchers have identified a significant
vulnerability in our CDSS operations, one that has important patient safety
implications. This is a study that every healthcare organization needs to pay
careful attention to and evaluate their own actual or potential
vulnerabilities.
And, of course, most of you by now have seen the results of
The Leapfrog Group’s most recent report on how hospitals perform on their CPOE
evaluation tool (Leapfrog
2016). We’ve written about the Leapfrog tool in several prior columns (see
our previous columns for July 27, 2010 “EMR’s
Still Have a Long Way to Go” and June
2012 “Leapfrog
CPOE Simulation: Improvement But Still Shortfalls” and March 2015 “CPOE
Fails to Catch Prescribing Errors”).
To fully meet Leapfrog’s standard, hospitals must:
- Demonstrate that the
system alerts physicians to at least 50% of common, serious prescribing errors;
and
- Order at least 75% of
inpatient medication orders through a CPOE system.
In 2015 nearly two-thirds of hospitals (64%) fully met the
standard, showing a considerable improvement compared to 14% in 2010. The
hospitals also demonstrated improved performance in medication reconciliation.
However, on the 2015 Leapfrog Hospital Survey, hospitals’ CPOE systems failed
to flag 39% of all potentially harmful drug orders, or nearly two out of every
five orders. The systems also missed 13% of potentially fatal orders.
The Wright study and the Leapfrog study demonstrate that it
is never enough to simply implement a CPOE system or e-prescribing system with clinical
decision support systems and assume your patients will be safe from medication
errors. Clearly, ongoing evaluation and assessment using validated tools are
important to identify vulnerabilities that may be unexpected. We, of course,
should expect better design and function from our IT vendors. However, the
Wright study clearly shows that problems may arise even when the initial design
and implementation were good yet changes to systems or files result in gaps
that may go unidentified for long periods.
See some of our other
Patient Safety Tip of the Week columns dealing with unintended consequences of
technology and other healthcare IT issues:
- June 19, 2007 “Unintended
Consequences of Technological Solutions”
- May 20, 2008 “CPOE
Unintended Consequences – Are Wrong Patient Errors More Common?”
- June 17, 2008 “Technology
Workarounds Defeat Safety Intent”
- August 26, 2008 “Pattern
Recognition and CPOE”
- September 9, 2008 “Less
is More….and Do You Really Need that Decimal?”
- December 16, 2008 “Joint
Commission Sentinel Event Alert on Hazards of Healthcare IT”
- February 2009 “Healthcare
IT The Good and The Bad”
- March 3, 2009 “Overriding
Alerts…Like Surfin’ the Web”
- October 2009 “A
Cautious View on CPOE”
- November 24, 2009 “Another
Rough Month for Healthcare IT
- April 20, 2010 “HIT’s
Limited Impact on Quality To Date”
- March
22, 2011 “An
EMR Feature Detrimental to Teamwork and Patient Safety”
- June
26, 2012 “Using
Patient Photos to Reduce CPOE Errors”
- June 2012 “Leapfrog
CPOE Simulation: Improvement But Still Shortfalls”
- July 17, 2012 “More
on Wrong-Patient CPOE”
- January 2013 “More
IT Unintended Consequences”
- April
30, 2013 “Photographic
Identification to Prevent Errors”
- October 8, 2013 “EMR
Problems in the ED”
- March 11, 2014 “We
Miss the Graphic Flowchart!”
- October 2014 “Ebola
Exposes Fundamental Flaw”
- January 2015 “Beneficial
Effect of EMR on Patient Safety”
- March 2015 “CPOE
Fails to Catch Prescribing Errors”
- March 31, 2015 “Clinical
Decision Support for Pneumonia”
- August 2015 “Newborn
Name Confusion”
- December 2015 “Opioid
Alert Fatigue”
- January 12, 2016 “New
Resources on Improving Safety of Healthcare IT”
- January 19, 2016 “Patient
Identification in the Spotlight”
- February 9, 2016 “It
was just a matter of time…”
- April 5, 2016 “Workarounds
Overriding Safety”
References:
Wright A, Hickman T-T T, McEvoy D, et al. Analysis of
clinical decision support system malfunctions: a case series and survey. JAMIA
2016; First published online: 28 March 2016
http://jamia.oxfordjournals.org/content/early/2016/03/28/jamia.ocw005
The Leapfrog Group. Hospitals’ Computerized Systems Proven
to Prevent Medication Errors, but More is Needed to
Protect Patients from Harm or Death. The Leapfrog Group 2016; April 7, 2016
full report
Print “PDF
version”
http://www.patientsafetysolutions.com/