View as “PDF version”

Patient Safety Tip of the Week

April 26, 2022

Challenges with Early Warning Systems

 

 

The concept behind early warning systems (EWS) is great. Theoretically, information from multiple sources can be aggregated and used to identify patients at risk for clinical deterioration before those caring for the patient might otherwise recognize.

 

But the success of early warning systems (and the associated rapid response systems) has been limited. We discussed barriers to success of these systems in our July 15, 2014 Patient Safety Tip of the Week “Barriers to Success of Early Warning Systems”.

 

A new study (Braun 2022) looks at early warning systems from a nursing perspective and has some important implications for design and implementation of such systems. Braun and colleagues interviewed nurses after implementation of a commercially available EWS algorithm. The algorithm collated data from a patient’s electronic health record (EHR), incorporating 26 EHR data points from laboratory values, vital signs, and nursing assessments. It then calculated a composite score, automatically updated as data were entered into a patient’s EHR. The healthcare team could view a patient’s EWS algorithm score, score trend and risk level via the patient’s EHR. The EWS algorithm triggered an alert when a patient’s score trended downward or fell below a threshold score.

 

A virtual care team (VCT) of 20 critical care nurse monitored the EWS algorithm alerts in real time, around the clock. When an EWS algorithm alert was triggered, the patient’s information would display on the shared monitor and the VCT nurse’s personal computer monitor. The assigned VCT nurse then followed a standardized protocol to briefly review the patient’s EHR before calling the patient’s acute care nurse to notify them about the EWS algorithm warning. The initial implementation had the VCT nurses calling the bedside nurse for every alert, but this was subsequently changed to allow VCT nurses discretion to not call on false-positive alerts.

 

Six principal themes emerged during the interviews of bedside nurses:

 

Timeliness

Because the bedside nurses were entering much of the information into the EHR, they were often aware of their patient’s deterioration and had already begun or completed the appropriate intervention before the EWS algorithm triggered and the VCT called. So, often the alert simply told them what they already knew and simply served as a distraction or interruption.

 

Workflow interruption

Those alerts and the calls from the VCT nurses often interrupted the bedside nurses while they were busy implementing the appropriate medical interventions.

 

Accuracy

Bedside nurses were often frustrated by false positives or false negatives. They often had to respond to calls for patients that were stable and they often did not get an alert or call on patients who were deteriorating. That undermined confidence in the EWS algorithm. Of interest is that the beside nurses often attributed false positives to the inclusion of subjective nursing assessment documentation. We had previously reported that inclusion of nursing impressions improved an EWS. But, in this study, the nurses placed little value on alerts that triggered based on such data due to inter-nurse variability.

 

Actionability

Some alerts had clear actions steps – such as activation of a rapid response team or transfer to an ICU. But others lacked good suggestions about next steps. Interestingly, “Some nurses appreciated the existence of the EWS programme as a safety net even though their clinical decision-making was rarely, if ever, impacted by the EWS programme.”

 

Underappreciation of core nursing skills

Many nurses felt that the EWS devalued their clinical skills and ability to recognize patients at risk for clinical deterioration. They also were concerned with the reliance on technology by a hands-on profession.

 

Opportunity cost

Many nurses felt that the money spent on the EWS would have been better used by hiring more staff. Yet, many appreciated the VCT nurses and sometimes called upon them for advice.

 

This study is, of course, reminiscent of alarm fatigue in an ICU or alert fatigue during CPOE. The basic problem seems to be presenting alerts that are neither timely nor actionable and are interruptive to normal clinical workflow.

 

The authors identified delays at multiple levels that contributed to frustration with the system. Delayed data input (charting) contributed to many of the problems encountered. But there were also delays due to the VCT nurses processing the alerts and delays in the VCT nurses contacting the bedside nurses, particularly when the VCT nurse could not reach the acute care nurse by phone in a timely fashion.

 

The Braun study does not describe how the charting and bedside nurse data input was handled. It says nurses reported a typical charting lag time of 0.5–2.5 hours, which would suggest to us that they were not using tablets or other input devices at the bedside.

 

As we said earlier, the value of an EWS is its ability to identify likely clinical deterioration in a patient before the clinical staff are aware of it. It sounds like the EWS in this study seldom did that. Couple that with the workflow interruptions and distractions caused by the EWS and it is no wonder nursing impressions of the system were not good.

 

Contrast that with the EWS discussed in our December 1, 2020 Patient Safety Tip of the Week “An Early Warning System and Response System That Work”. That system at Kaiser Permanente Northern California (KNPC) was based on data obtained from the electronic medical record (EMR). Predictors of events (unplanned transfers to the intensive care unit from a general med/surg floor) included laboratory tests, individual vital signs, neurologic status, severity of illness and longitudinal indexes of coexisting conditions, care directives, and health services indicators (e.g., length of stay). A score is generated and a threshold level was ascertained for prediction of events. For all three scores, about half of alerts occurred within 12 hours of the event, and almost two thirds within 24 hours of the event. Notably, the alerts provide 12-hour warnings and do not require an immediate response from clinicians. At the threshold used, the model generates one new alert per day per 35 patients. This minimized alert fatigue and did not cause the frequent workflow interruptions seen in the Braun study.

 

The ideal EWS works in the background, using data already available in the EHR and provides only alerts that are timely, accurate, actionable, and non-interruptive and don’t simply tell the clinicians what they were already aware of. The ideal EWS has not yet been implemented.

 

 

Some of our other columns on MEWS or recognition of clinical deterioration:

 

·       February 26, 2008 “Nightmares: The Hospital at Night”

·       April 2009 “Early Emergency Team Calls Reduce Serious Adverse Events”

·       December 15, 2009 “The Weekend Effect”

·       December 29, 2009 “Recognizing Deteriorating Patients”

·       February 22, 2011 “Rethinking Alarms”

·       March 15, 2011 “Early Warnings for Sepsis”

·       October 18, 2011 “High Risk Surgical Patients”

·       March 2012 “Value of an Expanded Early Warning System Score”

·       September 11, 2012 “In Search of the Ideal Early Warning Score”

·       May 2013 “Ireland First to Adopt National Early Warning Score”

·       September 17, 2013 “First MEWS, Now PEWS”

·       January 2014 “It MEOWS But Doesn’t Purr”

·       March 11, 2014 “We Miss the Graphic Flowchart!”

·       July 15, 2014 “Barriers to Success of Early Warning Systems”

·       November 11, 2014 “Early Detection of Clinical Deterioration”

·       February 2015 “Detecting Clinical Deterioration: Don’t Neglect Clinical Impression”

·       April 28, 2015 “Failure to Escalate”

·       September 8, 2015 “TREWScore for Early Recognition of Sepsis”

·       October 2015 “Even Earlier Recognition of Severe Sepsis”

·       December 15, 2015 “Vital Sign Monitoring at Night”

·       June 2016 “An EMR-Based Early Warning Score”

·       May 2018 “Pediatric Early Warning System Fails”

·       May 26, 2020 “Early Warning Scores”

·       December 1, 2020 “An Early Warning System and Response System That Work”

·       July 2021 “EPIC Sepsis Prediction Tool Falls Short”

·       July 13, 2021 “The Skinny on Rapid Response Teams”

·       December 2021 “Can AI Triage Postoperative Patients More Appropriately?”

 

 

Our other columns on rapid response teams:

 

 

 

References:

 

 

Braun EJ, Singh S, Penlesky AC, et al. Nursing implications of an early warning system implemented to reduce adverse events: a qualitative study. BMJ Quality & Safety 2022; Published Online First: 15 April 2022

https://qualitysafety.bmj.com/content/early/2022/04/14/bmjqs-2021-014498

 

 

 

 

 

 

Print “PDF version”

 

 

 

 

 

 

 

 

 

 

 

 


 

 

http://www.patientsafetysolutions.com/

 

Home

 

Tip of the Week Archive

 

What’s New in the Patient Safety World Archive