We adopt many interventions after they are shown to be “effective” in studies using “before/after” designs. Even though we rank such studies on the “evidence scale” much lower than randomized controlled trials (RCT’s), there are some studies that are not amenable to RCT’s. But all too often we don’t want to wait for RCT’s to be done and we adopt the intervention as “evidence based”.
A great paper (Joffe 2011) just came out that basically took a non-intervention and applied a before/after design, demonstrating the non-intervention was successful! Many pediatric hospitals had reported significant improvements in a variety of outcome parameters (eg. out-of-ICU cardiopulmonary arrests, hospital mortality, etc.) after implementing medical emergency teams (MET’s). This hospital did not implement a MET. But it compared its hospital mortality over time to that reported in hospitals purporting to have demonstrated a positive impact of MET’s. Indeed, during the time periods of 2 published MET studies, their hospital showed a similar improvement in hospital mortality!
They go on to discuss the many potential reasons for their improvement and they do discuss the pros and cons of MET’s. However, the major thrust of the article is that before/after studies like those published on MET’s (i.e. studies using historical controls without consideration of temporal trends, case mix or severity adjustment) may come to erroneous conclusions about the efficacy of an intervention.
How many things have you implemented based upon published results of such before/after studies? We bet it’s more than you are aware of. This is a very enlightening article!
Joffe AR, Anton NR, Burkholder SC. Reduction in Hospital Mortality Over Time in a Hospital Without a Pediatric Medical Emergency Team: Limitations of Before-and-After Study Designs. Arch Pediatr Adolesc Med. 2011;165(5):419-423