Most healthcare organizations have considerable experience now implemented care “bundles”, groups of evidence-based interventions that have individually demonstrated efficacy in well-designed clinical studies. Most are familiar with IHI’s VAP Bundle (see our September 2, 2008 Patient Safety Tip of the Week “Urinary Catheter-Related UTI’s: Bladder Bundles”). Many also consider the SCIP (Surgical Care Improvement Project) interventions as a “bundle”.”) or various “Bladder Bundles” (see our June 24, 2008 Patient Safety Tip of the Week “
One of the problems when we implement a bundle and see improvement is that we are never quite sure which of the components of that bundle was most responsible for the improvement. But we have usually documented that implementation of the bundles improved the outcomes we were measuring.
But now a new study showed a care bundle paradoxically worsened the very outcomes it was intended to improve. Anthony et al. (Anthony 2010) did a randomized controlled trial of a bundle of interventions in colorectal surgery. Their bundle consisted of 5 interventions that had been shown to reduce surgical site infections in studies where they were evaluated individually. These interventions included: (1) omission of mechanical bowel preparation (2) pre- and intra-operative warming (3) supplemental oxygenation (4) intraoperative fluid restriction and (5) use of a surgical wound edge protector. Rather than reducing the rate of SSI’s (surgical site infections), their intervention group actually had a significant increase in SSI’s.
But there were some other differences between their control group and the intervention group. The average surgical duration was 20 minutes longer in the intervention group. We previously discussed the relationship between SSI’s and surgical procedure duration in our March 10, 2009 Patient Safety Tip of the Week “Prolonged Surgical Duration and Time Awareness” and our January 2010 What’s New in the Patient Safety World column “Operative Duration and Infection”. Each half-hour extra is associated with a 2.5% higher SSI rate. So a 20-minute average difference in surgical duration is unlikely to explain the almost 2-fold difference in infections rates seen in the Anthony study. It would, however, be of interest to see whether implementation of the bundle in any way contributed to the longer surgical durations.
The vast majority of the SSI’s seen were superficial wound infections and there were no mortality differences between the two groups. By the way, the relatively high rate of SSI’s in this study is instructive in that many were diagnosed after discharge from the hospital. SSI rates in most studies have just included cases identified prior to discharge and may underestimate the true incidence.
The authors also postulate that maybe the evidence base for some of the interventions is not so strong. The authors of the current study also just published a companion article reviewing the evidence base for nonpharmacologic interventions designed to prevent SSI’s in colorectal surgery (Murray 2010).
But we agree with the authors’ main lesson: don’t assume that interventions which are combined will necessarily produce more improvement. They went into their study expecting to see at least an additive and perhaps synergistic effect of combining interventions. It is counterintuitive to think that combining interventions that proven efficacy individually would actually lead to worse outcomes. But that is exactly what they saw in this study.
The study comes on the heels of the recent report of a mixed impact of SCIP (Surgical Care Improvement Project) on actual outcomes (Stulberg 2010) that some found to be disappointing. That study showed that adherence to the individual infection prevention measures did not result in a significant reduction in infection rates. However, performance on the global composite score (a measure of adherence to all the individual measures) did correlate with a reduction in infection rates. In fact, demonstrated adherence to SCIP measures through the all-or-none global composite SCIP scores was associated with a reduction from 14.2 to 6.8 postoperative infections per 1,000 discharges. One could actually probably argue that this study did show a synergistic effect of the “bundle”. One factor that is very hard to cull out from studies like this is the impact of “culture”. Organizations that implement full bundles as opposed to a few individual interventions are more likely to have a “culture of safety”, which in the end may be more important than the individual interventions.
More importantly, it is a reminder that use of process measures or other surrogate measures (eg. adherence to a practice) rather than actual outcome measures may give rise to misleading impressions.
Anthony T, Murray BW, Sum-Ping JT, et al. Evaluating an Evidence-Based Bundle for Preventing Surgical Site Infection. A Randomized Trial.
Arch Surg. Published online November 15, 2010. doi:10.1001/archsurg.2010.249
Murray BW, Huerra S, Dineen S, Anthony T. Surgical Site Infection in Colorectal Surgery: A Review of Nonpharmacologic Tools of Prevention. J Am Coll Surg 2010; 211(6): 812-822
Stulberg JJ, Delaney CP, Neuhauser DV, et al. Adherence to Surgical Care Improvement Project Measures and the Association With Postoperative Infections. JAMA. 2010; 303(24): 2479-2485