Checklists are great tools for helping ensure that important steps in a variety of patient safety activities are not missed. But checklists are only as good as the degree to which they are complied with, as exemplified in our May 2, 2017 Patient Safety Tip of the Week .
Weve often noted that most facilities have no idea whether their Universal Protocol or Surgical Safety Checklist is being used well. That is because they dont audit compliance with its use. Even those that think they have good compliance usually rely on self-reporting rather than more objective measurements. We recommend direct observation or use of video recording as ways to assess compliance. The former is obviously subject to the Hawthorne effect. The latter is more effective but many physicians and OR personnel have objected to video recording and few ORs use it today.
A third way is to use simple audio recording. While audio recording may miss some of the body language aspects that may provide important details about communication in the OR, it may still provide insight into how well OR teams are complying with the checklists. A recent study reported use of such an audio audit of use of a Surgical Safety Checklist at a US hospital (Salgado 2016). Whereas a compliance rate was previously reported as 97.6%, the audio-recorded audits revealed a precisely executed checklist only 73.6% of the time.
The Salgado article is useful not only for details about the method of recording but also for the checklist scoring system they developed.
Note that their version of the WHO Surgical Safety Checklist was modified for their own use, as is strongly recommended for any facility using the checklist. So some items on their checklist audit tool may differ from those you might audit at your facility. They audited items from all 3 components of the Surgical Safety Checklist (pre-procedure, just before skin incision or start of procedure, and post-procedure before patient leaves the OR).
Audit of the middle section of the checklist (just before skin incision or start of the procedure) revealed suboptimal compliance with two key items we have stressed before. Confirmation of the procedure with the consent form was only completed in 69% of audited cases. And asking whether essential imaging was required was completed in only 81% of audited cases. We found it somewhat surprising that the question about whether antibiotic prophylaxis has been given within the last 60 minutes was only completed in 80% of audited cases. On the other hand, we were pleased to see that this facility included a question about letting the alcohol prep dry before draping and that compliance with this item was 89%. We were also glad to see the item addressing anticipated surgical specimens and tests was completed in 94% of cases. The item asking whether venous thromboembolism prophylaxis is needed is also a question we think is important (though we usually recommend it as an item to be discussed during the pre-op huddle) but this item was only completed in 55% of audited cases.
Regarding the pre-procedure portion of the checklist, compliance was pretty good regarding the anesthesia check, medication check, and pulse oximetry items. Risk of hypothermia was formally completed in only 52%. The site marked item was complete in only 56%. That sounds awfully low, but it is not clear whether this is adjusted for those cases in which site marking is not indicated (eg. bilateral procedures).
Compliance with the post-procedure portion of the checklist was, in general, poor. Several items are of particular concern. One is that labeling of specimens was completed as an item in only 34% (it is not clear from the article whether this might reflect that specimens were not taken in some cases). And only 43% of cases completed the item asking if instrument, sponge and needle counts were complete. And only 22% asked whether any equipment problems needed to be addressed.
They were also able to identify some barriers to communication, such as distractions. Those identified included personal conversations, staff changes, and room setup noise as major distractors. The researchers also looked at whether quiet was requested for various parts of the checklist and whether quiet was actually achieved. Quiet was actually achieved in less than 50% for each section of the checklist and for only 20% for the post-procedure period.
Note that the Surgical Safety Checklist used by Salgado et al. asked for confirmation of the procedure with the informed consent. It did not require confirmation with the H&P. We feel that requiring the latter is equally important since informed consents are not without problem (see our September 10, 2013 Patient Safety Tip of the Week Informed Consent and Wrong-Site Surgery). And, as discussed in our May 2, 2017 Patient Safety Tip of the Week , it is critical that all participants have actually looked at these primary source documents (the consent and the H&P).
You should refer to the actual article for a description of the audio recording protocol.
The Salgado study is an excellent contribution to patient safety. Every facility should be auditing compliance with their checklists and both the audio format and the scoring tool developed by these researchers could be adapted to almost any facility. We also suspect that the results found in their audit are probably pretty representative of what the majority of facilities will find when they do their own audit.
Some of our prior columns related to wrong-site surgery:
September 23, 2008
June 5, 2007
March 11, 2008 Lessons from Ophthalmology
July 1, 2008
January 20, 2009
September 14, 2010
November 25, 2008 Wrong-Site Neurosurgery
January 19, 2010 Timeouts and Safe Surgery
June 8, 2010 Surgical Safety Checklist for Cataract Surgery
December 6, 2010
June 6, 2011 Timeouts Outside the OR
August 2011 New Wrong-Site Surgery Resources
December 2011 Novel Technique to Prevent Wrong Level Spine Surgery
October 30, 2012 Surgical Scheduling Errors
January 2013 How Frequent are Surgical Never Events?
January 1, 2013 Dont Throw Away Those View Boxes Yet
August 27, 2013 Lessons on Wrong-Site Surgery
September 10, 2013 Informed Consent and Wrong-Site Surgery
July 2014 Wrong-Sided Thoracenteses
May 17, 2016
July 19, 2016
September 13, 2016
May 2, 2017
Some of our prior columns on checklists:
Salgado D, Barber KR, Danic M. Objective Assessment of Checklist Fidelity Using Digital Audio Recording and a Standardized Scoring System Audit. Journal of Patient Safety 2016; Published Ahead of Print Post Author Corrections: November 2, 2016