When we look at published results of a clinical trial, particularly one that shows a dramatic improvement in outcomes with use of the drug or device being studied, we do so with a very skeptical eye. Study after study have used flawed methodologies to suggest to us their product is great. Or they highlight in the “conclusion” a subgroup analysis that had a positive trend when the primary outcomes measures were not met. See our February 16, 2010 Patient Safety Tip of the Week “Spin/Hype…Knowing It When You See It” for examples. If it looks too good to be true, it’s usually not.
So when we saw in bold headlines in all media that there was a 17% reduction in hospital-acquired conditions (HAC’s) in the past 3 years, saving 50,000 lives and $12 billion (AHRQ 2014), and largely attributed to government-sponsored programs, our “hype radar” went into high gear.
And this occurred at a time when public reporting showed substantial increases in serious adverse events in Massachusetts (Kowalczyk 2014, Biondolillo 2014) and Connecticut (Connecticut DPH 2014) and a report commissioned for the Betsy Lehman Center showed that many Massachusetts patients perceived medical errors to be common and not reported (Harvard School of Public Health 2014). And we’re not the only ones whose “hype radars” were raised. Healthcare bloggers Paul Levy and Susan Carr also discussed these mixed messages on patient safety.
Given the slow to nonexistent progress we’ve made in patient safety over the past 2 decades, is it plausible that such dramatic improvements could be made in 3 years or less? When we read a paper about a clinical trial we always look to see who sponsored or funded the trial. If it is funded by someone who stands to profit from results put in a positive light we are even more skeptical. For years we have lamented the fact that the percentage of studies funded and overseen by government organizations like NIH has dwindled and now most research is sponsored by those with vested interests. Notice we said we like government-sponsored research. That is because we always assume such research won’t be biased. But is that true for all government-sponsored studies? Government can also have self-interests that might lead to biases. Might it not want to justify large investments made in various projects?
But we did dig deeper into the evidence. First, much of the “increase” in serious events reported in Massachusetts and Connecticut appears to be due to changes in reporting requirements and may not reflect an actual increase in adverse patient events.
In the AHRQ study preliminary estimates for 2013 show a further 9 percent decline in the rate of hospital-acquired conditions (HACs) from 2012 to 2013, and a 17 percent decline, from 145 to 121 HACs per 1,000 discharges, from 2010 to 2013. About 40 percent of this reduction is from ADEs, about 20 percent is from pressure ulcers, and about 14 percent from catheter-associated urinary tract infections (CAUTIs)
The AHRQ report focused on the nine “core” events that were part of the improvement projects in the Partnership for Patients initiatives that worked with the HEN’s (Hospital Engagement Networks) funded through the Affordable Care Act (ACA). The hospital-acquired conditions that were the focus of the Partnership for Patients program are:
But those are really significant patient safety events that we all focus our attention on. The other events were lumped under the “all other HAC’s” category. And the AHRQ methodology was a sampling of all Medicare patients, not just those at the hospitals participating in the Partnership of Patients focused programs. So the methodology is actually pretty good for looking at trends over time.
Do we find other evidence that might corroborate the striking improvement in HAC’s in the AHRQ report? CDC has just reported its progress report on hospital-acquired infections (HAI’s) and it does show substantial improvement (CDC 2015). On the national level, the report found:
The significant reduction is CLABSI’s is not surprising as hospitals have adopted the practices developed by Peter Pronovost and promulgated by the success of the Michigan Keystone Project (see our March 2011 What’s New in the Patient Safety World column “Michigan ICU Collaborative Wins Big”).
And in our January 2015 What’s New in the Patient Safety World column “Beneficial Effect of EMR on Patient Safety” we highlighted a study using data from the Pennsylvania Patient Safety Authority (PPSA) and the HIMSS Analytics database that demonstrated a substantial decrease in patient safety events at hospitals with advanced EMR’s (Hydari 2014). The researchers found that advanced EMRs led to a 27 percent decline in patient safety events overall, driven by a 30 percent decline in events due to medication errors and 25 percent decline in events due to complications of tests, treatments and procedures. So as more and more hospitals have progressed to the “advanced EMR” status we likely are seeing a substantial reduction in events, particularly medication errors, which was one of the HAC’s with substantial improvement in the AHRQ report.
Other evidence comes from studies looking at the impact of CMS’s initiatives that withhold payments for certain HAC’s. Previous studies had suggested that CMS’s HAC (Hospital Acquired Condition) nonpayment initiative had not had significant impact on CLABSI’s or CAUTI’s (Lee 2012). But those results came from a limited data set. Now a new analysis (Waters 2015) of a much larger data set concludes that the CMS HAC nonpayment initiative was associated with significant reductions in CLABSI’s (11% decrease) and CAUTI’s (10% decrease) but no reduction in rates of injurious falls or decubiti (though the timeframe differs from that of the AHRQ study). The authors conclude that the success in reducing CLABSI’s and CAUTI’s reflected more robust evidence-based preventive interventions for those conditions than for the other two HAC’s. They also suggest that, since CLABSI’s and CAUTI’s are more frequent in specialized units like ICU’s, it is easier to get a small group of focused professionals addressing the issue. On the other hand, falls and decubiti occur throughout the hospital system and their reduction would require a much larger team-based approach.
The editorial accompanying the Waters study (Umscheid 2015) notes that the Lee study had 33 months of data on CLABSI’s and CAUTI’s before the CMS initiative began in 2008 whereas the Waters study only had 9 months of data. They argue that the more limited pre-implementation data may have been inadequate to define the true trend in CLABSI’s and CAUTI’s prior to the CMS action. Also, the CLABSI and CAUTI data in the Waters study apparently were limited to ICU events whereas CMS looks at data for the entire hospital.
That does raise an interesting point. In the AHRQ study the majority of deaths averted occurred as a result of reductions in the rates of pressure ulcers and ADEs, although declines in other HACs also contributed significantly to deaths averted. Yet the Waters study found no reduction in the rates of decubiti. But the timeframes differ. The Waters study looked at data from 2008 to 2010, the AHRQ study from 2010 to 2013.
So back to our original question – is it progress or is it propaganda? Looks like this is one instance where our “hype radar” was wrong. We’ll go with progress. It’s refreshing to see that a lot of hard work and plugging along are finally bearing fruit.
AHRQ. Interim Update on 2013 Annual Hospital-Acquired Condition Rate and Estimates of Cost Savings and Deaths Averted From 2010 to 2013. AHRQ Partnership for Patients 2014
Kowalczyk L. Mass. Hospitals’ Mistakes List Widens. Boston Globe August 14, 2014
Biondolillo M. Commonwealth of Massachusetts Department of Public Health. Serious Reportable Events 2011-2013. August 2014
Connecticut Department of Public Health. Legislative Report To The General Assembly. Adverse Event Reporting. Octobrer 2014
Harvard School of Public Health. The Public’s View on Medical Error in Massachusetts. Commissioned by Betsy Lehman Center for Patient Safety and Medical Error Reduction and Health Policy Commission. December 2014
Levy P. Falling behind on safety and quality in the Hub of the Universe. Not Running a Hospital Blog December 2, 2014
Carr S. Editor’s Note: Mixed Messages on Safety. PSHQ Blog December 9, 2014
CMS. Partnership for Patients.
CDC.Healthcare-associated Infections (HAI) Progress Report. January 2015
Hydari MZ, Telang R, Marella WM. Saving Patient Ryan - Can Advanced Electronic Medical Records Make Patient Care Safer? (September 30, 2014). Available at SSRN:
Lee GM, Kleinman K, Soumerai SB, et al. Effect of Nonpayment for Preventable Infections in U.S. Hospitals. N Engl J Med 2012; 367: 1428-1437
Waters TM, Daniels MJ, Bazzoli GJ, et al. Effect of Medicare’s Nonpayment for Hospital-Acquired Conditions. Lessons for Future Policy. JAMA Intern Med 2015; Published online January 05, 2015
Umscheid CA, Brennan PJ. Incentivizing “Structures” Over “Outcomes” to Bridge the Knowing-Doing Gap. JAMA Intern Med 2015; Published online January 05, 2015
Print “PDF version”