View as “PDF version”

Patient Safety Tip of the Week

May 26, 2020   Early Warning Scores

 

 

It’s been 2 years since our last column on early warning scores for early recognition of clinical deterioration. But last week, just days apart, there were 2 published reviews of early warning scores.

 

The concept behind early warning scores is simple. Scanning readily collected physiologic data, usually utilizing computer algorithms, can spot trends suggesting clinical deterioration that might otherwise not be recognized at an early stage where intervention might be useful.

 

Our previous columns, listed below, have highlighted both successes and failures of early warning scores, as well as the successes and failures of the rapid response interventions designed to rescue patients having signs of clinical deterioration.

 

Liu and colleagues (Liu 2020) retrospectively applied 5 commonly used early warning tools to data collected on a large cohort of adult inpatients outside the ICU in two states. The 5 tools were the National Early Warning Score (NEWS), Modified Early Warning Score (MEWS), Between the Flags (BTF), Quick Sequential Sepsis-Related Organ Failure Assessment (qSOFA), and Systemic Inflammatory Response Syndrome (SIRS). They also stratified results based upon whether the patients had suspected infection or not. The authors acknowledge that SIRS and qSOFA were not developed as early warning scores for all patients. They were specifically developed for assessing patients for sepsis, but many hospitals have utilized them for early warnings.

 

Assessing the area under the receiver operating characteristic curves (AUCs), they found the NEWS exhibited the highest discrimination for mortality (AUC 0.87 and 0.86 in California and Illiniois, respectively), followed by the MEWS (AUC’s 0.83 and 0.84), qSOFA (AUC’s 0.78 and 0.78), SIRS (AUC’s 0.76 and 0.76), and BTF (AUC’s 0.73 and 0.74).

 

A similar pattern was seen in the suspected infection cohort, with the NEWS demonstrating the highest AUC for both outcomes across both states, followed by the MEWS. Even among patients with infection, the discrimination of the NEWS and MEWS were higher in all cases than the infection-specific risk scores.

 

The authors conclude that, for the goal of detecting clinical deterioration in hospitalized, non-ICU patients, aggregate weighted risk scores, such as those determined with NEWS and MEWS, outperform infection-specific scores, even among patients with suspected infection.

 

We’ve always been biased against use of SIRS. When Medicare and other third party payers used SIRS criteria to allow for coding for a diagnosis of sepsis, we saw an interesting phenomenon: mortality for both sepsis and pneumonia decreased. Why? We would see a patient with pneumonia happily pushing his IV pole as he walked up and down the hallway on a typical med-surg floor. But because he met 2 of the SIRS criteria, the clinical documentation specialists that many hospitals contracted with, would recommend coding that patient as “sepsis”, hardly what we clinicians would have called sepsis in the past. The result was that patients not likely to die (like the one mentioned above) were now included with all patients with “sepsis”, diluting out that population that might have previously had a high mortality rate. Similarly, patients with pneumonia who might have had a high mortality rate were now moved to the sepsis category, reducing the overall “pneumonia” mortality rate.

 

But the Liu study found the scores of most of the early warning systems were good predictors of mortality. On the other hand, for the combined outcome of ICU transfer or death discrimination was only poor to adequate. That of course, is important because the goal of early warning systems is to improve the identification of high-risk patients and enable clinical interventions that can mitigate or prevent deterioration, including proactive transfer to the ICU.

 

That brings us to the second publication. Gerry and colleagues (Gerry 2020) performed a systematic review and critical analysis of early warning score tools and found that, despite their widespread use, many early warning scores in clinical use were found to have methodological weaknesses.

 

Death was the most frequent prediction outcome for development studies and validation studies, with different time horizons (the most frequent was 24 hours). The most common predictors were respiratory rate, heart rate, oxygen saturation, temperature, and systolic blood pressure. Age and sex were less frequently included. They found that key details of the analysis populations were often not reported in development studies or validation studies and small sample sizes and insufficient numbers of event patients were common. Moreover, missing data were often discarded. Only nine of the early warning scores that were developed were presented in sufficient detail to allow individualized risk prediction. All included studies were rated at high risk of bias.

 

The authors note that many of the early warning scoring systems were originally developed in the paper-based chart era and scores were calculated manually, necessitating simple scoring systems. They point out that points were often assigned equally to each vital sign, assuming that each vital sign has the same predictive value, which may not be the case. That may result in a total score that has little meaning. They do note that the move to computerized algorithms and machine learning has the potential to improve early warning systems. Many of the barriers to success of early warning systems that we discussed in our July 15, 2014 Patient Safety Tip of the Week “Barriers to Success of Early Warning Systems” had to do with manual data collection and computation of scores. Our November 11, 2014 Patient Safety Tip of the Week “Early Detection of Clinical Deterioration” described how use of wireless handheld computing devices to replace a paper-based vital sign charting and use of computerized tools led to early recognition of and response to patient deterioration, resulting in improved mortality rates. Several of our other columns have discussed EMR-based early warning scores with good predictability, especially for early identification of sepsis and septic shock (see our Patient Safety Tip of the Week for September 8, 2015 “TREWScore for Early Recognition of Sepsis” and our What's New in the Patient Safety World columns for October 2015 “Even Earlier Recognition of Severe Sepsis” and June 2016 “An EMR-Based Early Warning Score”).

 

Gerry et al. end with recommendations for approaches for developing and evaluating early warning scores and investigating the impact and safety of using these scores in clinical practice.

 

And our February 2015 What's New in the Patient Safety World column “Detecting Clinical Deterioration: Don’t Neglect Clinical Impression” reminded us not to neglect the value of the clinical impression a nurse or physician has about the patient’s status. In that column we noted a study that added the question “How likely is this patient to suffer a cardiac arrest or require emergent transfer to the ICU in the next 24 hours?” improved the predictive value of at least one early warning system tool (Patel 2015).

 

And, of course, the success or failure of any early warning system depends upon what you do with it. Our multiple columns listed below demonstrate the mixed outcomes of rapid response teams and rapid response systems.

 

Bottom line: early warning scoring systems and our programs to respond to such warnings still have opportunity for improvement. But the concepts remain good and we should not abandon efforts to build upon work already done.

 

 

Some of our other columns on MEWS or recognition of clinical deterioration:

 

 

 

Our other columns on rapid response teams:

 

 

 

 

References:

 

 

Liu VX, Lu Y, Carey KA, et al. Comparison of Early Warning Scoring Systems for Hospitalized Patients With and Without Infection at Risk for In-Hospital Mortality and Transfer to the Intensive Care Unit. JAMA Netw Open 2020; 3(5): e205191

https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2766002?utm_source=silverchair&utm_medium=email&utm_campaign=article_alert-jamanetworkopen&utm_content=wklyforyou&utm_term=052020

 

 

Gerry Stephen, Bonnici Timothy, Birks Jacqueline, Kirtley Shona, Virdee Pradeep S, Watkinson Peter J et al. Early warning scores for detecting deterioration in adult hospital patients: systematic review and critical appraisal of methodology BMJ 2020; 369 :m1501

https://www.bmj.com/content/369/bmj.m1501

 

 

Patel AR, Zadravecz FJ, Young RS, et al. The Value of Clinical Judgment in the Detection of Clinical Deterioration. JAMA Intern Med 2015; 175(3): 456-458 Published online January 05, 2015

https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2087874

 

 

 

 

Print “PDF version”

 

 

 

 

 

 

 

 

 

 

 

 


 

 

 

http://www.patientsafetysolutions.com/

 

Home

 

Tip of the Week Archive

 

What’s New in the Patient Safety World Archive