Patient Safety Tip of the Week

September 1, 2015    Smarter Checklists

 

 

We ended our August 25, 2015 Patient Safety Tip of the Week “Checklist for Intrahospital Transport” noting that our checklist was really chock full of items and were wary that we might contribute to a trend toward checklist fatigue that we discussed in our May 2015 What’s New in the Patient Safety World column “The Great Checklist Debate”.

 

Well, what if we could do something to make use of checklists easier and more manageable? A very insightful perspective by Eliot Grigg on improving use of checklists and avoiding checklist fatigue appears in the August 2015 issue of Anesthesia & Analgesia (Grigg 2015). While written from the view of an anesthesiologist, the concepts apply to almost any checklists in healthcare. We’ll discuss Grigg’s ideas in detail below. But first we need to look at some of the barriers and facilitators for checklist success.

 

A timely article in Nature (Anthes 2015) nicely summarizes some of the successful checklist implementations and also the failures. That article begins with some striking examples we noted in our May 2015 What’s New in the Patient Safety World column “The Great Checklist Debate”. The original introduction of the WHO Surgical Safety Checklist (Haynes 2009) was associated with striking reductions in both mortality and complication rates. But some studies, such as one done in Ontario, Canada (Urbach 2014) showed that implementation of surgical safety checklists was not associated with significant reductions in operative mortality or complications. And the stunning success of Peter Pronovost’s central line insertion checklist for preventing CLABSI’s at both Johns Hopkins and the Michigan Keystone Project (see our March 2011 What’s New in the Patient Safety World column “Michigan ICU Collaborative Wins Big”) had difficulty being replicated in the “Matching Michigan” program in the UK. So why such variable success in checklist project implementations?

 

One criticism of most studies on the impact of checklists is their before-after study designs. The argument is that there are often other patient safety initiatives and so many other confounding variables that might explain improvement in the “after” period. Also, the improvements in teamwork, communication, and culture that accompany such initiatives may play as important a role as the checklists themselves. Both are probably true. But in our May 2015 What’s New in the Patient Safety World column “The Great Checklist Debate” we did discuss a recent study using a different design (stepped wedge cluster randomized controlled trial) that demonstrated a significant positive impact of the WHO Surgical Safety Checklist on patient morbidity and mortality (Haugen 2015). The Haugen study even showed a “dose effect” in that larger reductions in complications were seen when all portions of the checklist were followed. And at least one other small randomized controlled trial showed a positive impact of a checklist on post-anesthesia patient handovers (Salzwedel 2013). So there is at least some evidence from better designed studies demonstrating a positive impact from checklists.

 

Mayer and colleagues found significant variability in use of the WHO Surgical Safety Checklist in the UK (Mayer 2015). Overall, checklist implementation was associated with reduced case-mix-adjusted complications after surgery and was most significant when all 3 components of the checklist were completed. Full, as opposed to partial, checklist completion provides a health policy opportunity to improve checklist impact on surgical safety and quality of care. You’ll also recall that in our May 2015 What’s New in the Patient Safety World column “The Great Checklist Debate” we noted a study that showed a similar “dose-response” effect as part of an overall significant positive impact of the WHO Surgical Safety Checklist on patient morbidity and mortality (Haugen 2015).

 

In a qualitative study of the implementation of the WHO Surgical Safety Checklist in the UK, Russ and colleagues (Russ 2015a) found 11 themes that represented barriers and 9 themes that represented facilitators. There was wide variation among English hospitals in the adoption and impact of checklist. You’ve often heard us use the saying “culture trumps ________” (fill in the blank with words like policy, procedure, strategy, tactics, vision, etc). In fact, “Culture trumps…Everything!”. Russ and colleagues found a major barrier to be general resistance to change, particularly among senior members of the staff, to be a major barrier. Hierarchical issues also were a barrier in that the person leading the checks (often a nurse) had difficulty when senior surgeons or anesthesiologists were not cooperative. Another organizational barrier was the manner in which the checklist implementation was rolled out. Where there was no planned approach or there was an “imposed” approach, there was little sense of ownership and buy-in was lacking.

 

But there were also barriers related to the time it took to complete the checklist and issues specific to the checklist. For example, there were design issues (eg. wording, layout, timing of items) and some of the items were not relevant to the particular type of surgery or procedure being done.

 

Interestingly, many providers questioned the validity of the evidence base for the success of the checklist. The pilot study showing such tremendous improvements included many developing world hospitals and many in the UK felt their systems were already better. A feeling that the checklist might also detract from other patient safety processes was also noted by some.

 

But Russ and colleagues also identified facilitators for use of the checklist. Having a “champion” is a success factor noted for many patient safety initiatives and the checklist is not different. Particularly having a senior clinician as a champion was seen as key. Also, having a person with strong leadership skills and an assertive presence in the OR leading the checklist process in the OR was particularly important. Organizational facilitators included provision of education and training, providing evidence of efficacy of the checklist and relevance to their local OR, training sessions on how to best use the tool, and feedback on impact of the checklist. Enforced accountability for compliance with the checklist was noted as a facilitator by some, a barrier by others. Inclusive input from the entire multidisciplinary team was a key facilitator. Integrating use of the checklist into existing workflows was an important facilitator, as was avoiding redundancy with other safety initiatives. Also important was the ability to modify the tool for the specific surgical context and to make it more user friendly. Making the checklist less of a “checking the boxes” process and more of a tool to enhance safety discussion was mentioned. Some accomplished this by displaying the checklist on the OR wall rather than using a paper checklist.

 

Russ and colleagues offer the following lessons for implementing change:

 

The issue of auditing is a sensitive one. If you implement any kind of improvement initiative in health care you need to collect data to determine whether it is producing the results you intended. But when auditing is perceived as a punitive measure you may inadvertently cause the project to fail. Clay-Williams and Colligan (Clay-Williams 2015) in their perspective on checklist in health care and aviation note that some of the intended benefits of checklists, such as enhanced teamwork and nurses speaking up, may be negated if compliance audits lead to sanctions.

 

Another study by Russ and colleagues (Russ 2015b) used a standardized observational tool to assess use of the WHO Surgical Safety Checklist on a sample of 5 English hospitals and found large variation across hospitals. They found that, on average, only two thirds of the items were checked, team members were absent in more than 40% of cases, and there was failure to pause or focus on the checks in more than 70% of cases.

 

“Matching Michigan” was a patient safety program aimed at decreasing central line infections in over 200 intensive care units (ICUs) in England, based upon the Michigan Keystone Project in the US. It largely failed to replicate the overwhelming success seen in The Michigan Keystone Project. CLABSI rates were already lower than those in the original Michigan cohort and improvements were already occurring due to other patient safety initiatives. Though the CLABSI rates did improve some, the “Matching Michigan” project was considered a failure by many. Dixon-Woods and colleagues conducted interview with staff of multiple English ICU’s participating in the program (Dixon-Woods 2013). One unit transformed its practices and culture in response to the program; five boosted existing efforts, and 11 made little change. They found that the perception of the program as a “top-down” initiative imposed on them and punitive in nature was detrimental to the potential success of the program. But individual ICUs’ histories and local context were also highly consequential: their past experience of quality improvement, the extent to which they were able to develop high quality data collection and feedback systems, and the success of local leaders in developing consensus and coalition all influenced the program’s impact on local practices.

 

Those researchers identified several things that needed to happen to ensure success at hospitals. One was that there had to be at least one senior physician who took a strong leadership role and worked with senior nurses. Also data collection needed to be rigorous and command credibility.

 

We’ll add one other critical factor that contributes to almost every patient safety or performance improvement program: size matters! By that we mean that initiatives involving smaller groups of staff are far more likely to be successful than big ones. The UK “Matching Michigan” program involved all the hospitals in the UK. The actual Michigan Keystone Project, on the other hand, involved only ICU’s at hospitals volunteering to participate. We think one of the secrets to success is the CUSP (Comprehensive Unit-Based Safety Program) approach (see our March 2011 What’s New in the Patient Safety World column “Michigan ICU Collaborative Wins Big”). In our own experience the patient safety and quality improvement projects that are most successful are those done in relatively small settings where the key participants all know each other and work closely together as affinity groups. Contrast the striking successes of the MHA Keystone ICU project to the relative lack of success of a large scale organizational intervention on patient safety at several UK hospitals. In the UK project there seemed to be a disconnect between the frontline staff and the group overseeing the project.

 

 

So what about that perspective by Eliot Grigg on improving use of checklists and avoiding checklist fatigue (Grigg 2015)? Grigg, of course, does offer many of the above-mentioned solutions like changing culture, flattening hierarchies, reducing the number of checklists, or changing the designs of checklists. But Grigg’s best argument is that we have technological capabilities to make the checklists more useable.

 

Grigg notes that “the goal of ‘smart checklists’ is not to threaten provider autonomy but to mentally offload the many repetitive tasks in health care that must be completed in a largely predictable sequence”. He stresses that the efficient checklist presents the right content at the right time and in the right context. He suggests “filtering” the checklists to only include those things that have yet to be done, often with the help of technology. In that regard, some of the following contextual factors need to be considered (for an OR case):

 

He gives as an example that a checklist item for antibiotic administration should not be needed if your Anesthesia Information System (AIS) or equivalent electronic OR management system has already documented that the antibiotic has been administered.

 

Grigg further discusses the aviation analogy of two types of checklist: normal and non-normal. “Normal” checklists are those that deal with the more mundane tasks that are an integral part of routine care. “Non-normal” checklists are those that deal with emergency situations (see our August 16, 2011 Patient Safety Tip of the Week “Crisis Checklists for the OR”). The latter need to be readily available when the emergency arises. An example might be a checklist for dealing with malignant hyperthermia. (Clay-Williams and Colligan (Clay-Williams 2015) refer to these “non-normal” checklists as “boldface” checklists” meaning they are items requiring immediate action.) Note also we have in several previous columns noted a guidance from the UK Civil Aviation Authority that has some excellent recommendations about the design of emergency checklists used in aviation (UK Civil Aviation Authority 2006). Many of those design recommendations could equally apply to the “crisis” checklists needed in places like the OR.

 

Grigg notes that a crucial component in implementing smart checklists is having devices and software systems in the OR communicate with each other. He notes that all devices should have USB ports and wireless capabilities and use a common language for communicating their current state with each other. He suggests that RFID technology could be used to identify and readily locate both equipment and providers in the OR. And the checklists then ought to be easily accessible electronically and interfaced with the AIS as well.

 

Let’s apply Grigg’s concepts to the “Ticket to Ride” type checklist we described in last week’s Patient Safety Tip of the Week “Checklist for Intrahospital Transport”. One of the success factors for that checklist was its availability online in their EMR. But let’s take it even further. Important flags, such as whether a patient is at risk for falls or for wandering, might be included on the checklist only if the information from the nursing assessments in the EMR indicates the patient is at risk. Items pertaining to MRI-compatibility of equipment could appear on the checklist only if the patient is being transported for an MRI. The EMR could also indicate whether an informed consent is necessary and has been completed for the particular procedure for which the transport is occurring. Presence of various pieces of equipment on the transport gurney could be indicated via RFID technology. Battery/power status indicators from the equipment could automatically appear on the checklist. Conceivably, the amount of oxygen remaining in the oxygen cylinder might be populated in the checklist automatically via Bluetooth or other wireless technology. Ideally even the availability of elevators needed for the transport might be conveyed in real time. And reminders about restarting feedings or insulin or other medications could populate the post-transport checklist only if they had been withheld prior to the transport. Having such “filtering” could obviously reduce the “Ticket to Ride” checklist to a more manageable size and free up staff to pay more attention to things like maintenance of lines and tubing and monitoring the patient during the transport.

 

So is there a downside to the type of system envisioned by Grigg? The obvious one is overreliance on electronic systems that might go down periodically. Should the system fail, we could be back at square one and fail to remember some key steps in various processes. However, having such electronic systems working 99+% of the time obviously would likely improve both efficiency and safety.

 

We remain active advocates of checklists in healthcare. But we’re also wary of checklist fatigue. The suggestions by Grigg to better use technology and filter checklist items out to make checklist more manageable are very valuable.

 

 

Some of our prior columns on checklists:

 

 

References:

 

 

Grigg E. Smarter Clinical Checklists: How to Minimize Checklist Fatigue and Maximize Clinician Performance. Anesth Analg 215; 121(2): 57-573

http://journals.lww.com/anesthesia-analgesia/Citation/2015/08000/Smarter_Clinical_Checklists___How_to_Minimize.39.aspx

 

 

Anthes E. Hospital checklists are meant to save lives - so why do they often fail? An easy method that promised to cut complications in surgery may not be so simple after all. Nature 2015; 28 July 2015

http://www.nature.com/news/hospital-checklists-are-meant-to-save-lives-so-why-do-they-often-fail-1.18057

 

 

Haynes A, Weiser T, Berry W, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009; 360(5): 491-499

http://www.nejm.org/doi/full/10.1056/NEJMsa0810119#t=articleTop

 

 

Urbach DR, Govindarajan A, Saskin R, et al. Introduction of surgical safety checklists in Ontario, Canada. N Engl J Med 2014; 370(11): 1029-1038

http://www.nejm.org/doi/full/10.1056/NEJMsa1308261

 

 

Haugen AS, Søfteland E, Almeland SK, et al. Effect of the World Health Organization Checklist on Patient Outcomes: A Stepped Wedge Cluster Randomized Controlled Trial. Annals of Surgery 2015; 261(5): 821-828

http://journals.lww.com/annalsofsurgery/Fulltext/2015/05000/Effect_of_the_World_Health_Organization_Checklist.1.aspx

 

 

Salzwedel C, Bartz H-J, Kühnelt I, et al. The effect of a checklist on the quality of post-anaesthesia patient handover: a randomized controlled trial. Int J Qual Health Care 2013; 25(2): 176-181

http://intqhc.oxfordjournals.org/content/25/2/176

 

 

Mayer EK, Sevdalis N, Rout S, et al. Surgical Checklist Implementation Project: The Impact of Variable WHO Checklist Compliance on Risk-adjusted Clinical Outcomes After National Implementation. A Longitudinal Study. Annals of Surgery 2015; Post Author Corrections: March 13, 2015 doi: 10.1097/SLA.0000000000001185

http://journals.lww.com/annalsofsurgery/Abstract/publishahead/Surgical_Checklist_Implementation_Project___The.97453.aspx

 

 

Russ SJ, Sevdalis N, Moorthy K, et al. A Qualitative Evaluation of the Barriers and Facilitators Toward Implementation of the WHO Surgical Safety Checklist Across Hospitals in England: Lessons From the “Surgical Checklist Implementation Project”. Annals of Surgery 2015; 261(1): 81-91

http://journals.lww.com/annalsofsurgery/Abstract/2015/01000/A_Qualitative_Evaluation_of_the_Barriers_and.14.aspx

 

 

Clay-Williams R, Colligan L. Back to basics: checklists in aviation and healthcare. BMJ Qual Saf 2015; 24(7): 428-431 Published Online First: 12 May 2015

http://qualitysafety.bmj.com/content/24/7/428.full?sid=f59681b2-08b7-47e3-bdda-0a84e412027b

 

 

Russ S, Rout S, Caris J, et al. Measuring Variation in Use of the WHO Surgical Safety Checklist in the Operating Room: A Multicenter Prospective Cross-Sectional Study. J Am Coll Surg 2015; 220(1): 1-11.e4

http://www.journalacs.org/article/S1072-7515%2814%2901719-0/fulltext

 

 

Dixon-Woods M, Leslie M, Tarrant C, Bion J. Explaining Matching Michigan: an ethnographic study of a patient safety program. Implement Sci 2013; 8: 70

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3704826/

 

 

Civil Aviation Authority (UK). CAP 676: Guidance on the Design, Presentation and Use

of Emergency and Abnormal Checklists. 2006.

https://www.caa.co.uk/docs/33/CAP676.PDF

 

 

 

 

 

 

 

 

Print “PDF version”

 

 

 

 

 

 

 


 

http://www.patientsafetysolutions.com/

 

Home

 

Tip of the Week Archive

 

What’s New in the Patient Safety World Archive