Patient Safety Tip of the Week
March 5, 2013
Underutilized Safety Tools: The Observational Audit
We’ve frequently recommended performing audits on a variety
of processes in healthcare. To name just a few, we’ve discussed audits in MRI
safety, PCA (patient-controlled analgesia) safety, hi-alert medication safety,
and many others. But most of those are
retrospective audits and involve reviewing information that has been recorded
in the chart or electronic medical record. Often there are processes in
healthcare that are very necessary, even crucial, yet never get recorded
anywhere.
One of the most underutilized tools in patient safety is the observational audit. Other industries (eg. aviation, banking, pharmaceuticals, steel making, etc.) have used audit techniques successfully for many years. We’ve previously talked about the line operations safety audits (LOSA) audits in aviation where an independent observer sits in the cockpit and monitors and assesses multiple operations and procedures, then critiques the crew. An audit such as the LOSA is resource-intense, requires a well-trained auditor, and behavior may be quite different when a known audit is being undertaken. Nevertheless, the findings are often eye-opening. The power of real-time observational methodologies has largely been untapped in healthcare today.
We have previously discussed one form of observational audit, the real-time random safety audit (see our May 18, 2010 Patient Safety Tip of the Week “Real-Time Random Safety Audits”).
Perioperative processes are an area ripe for auditing and two recent studies in the surgical literature demonstrate how auditing using observational tools can be used to identify areas for improvement. They illustrate 2 slightly different methodologies that are helpful in identifying processes of care not well done.
First is a Metric
for Evaluating Task Execution in the Operating Room, better known as the
acronym METEOR (Russ
2013). Their underlying concept is that there are certain key tasks by the
OR team that are critical for patient safety and OR efficiency, coordination
and teamwork. Another key principle they considered is that safety and
efficiency in the OR are not often compromised by single, large, catastrophic
failures but rather more often by small failures that go unnoticed and
accumulate to impact safety or efficiency.
They note that the
list of such tasks goes well beyond what would typically be included in a
working checklist. You’ll recall from our previous discussions about checklists
that the number of items in a checklist should be small and should include
things you might otherwise forget (see our July 6, 2010 Patient Safety
Tip of the Week “Book
Reviews: Pronovost and Gawande” and our September 23, 2008 Patient Safety
Tip of the Week “Checklists
and Wrong Site Surgery”). Your
typical checklist used for patient safety purposes may have 10 or fewer items.
An exception is the SURPASS
(SURgical PAtient Safety System) checklist (see our November 30, 2010
Patient Safety Tip of the Week “SURPASS:
The Mother of All Checklists”).
But Russ and colleagues note that there are far more processes and tasks involved in surgery than you would include in a checklist. So they developed an audit tool that contains items for the preoperative, intraoperative, and postoperative stages and breaks the tasks down roughly into categories of patient tasks, communication tasks and equipment tasks. They first developed a list of 106 discrete OR tasks felt to be important in “getting things done”. They then had 2 general surgeons trained in use of the list observe 50 general surgical procedures in real-time and compute task completion rates for each of the 106 tasks in the list. If tasks were completed less than 70% of the time they considered that as either a signal of poor OR team performance or that the task was not really relevant for safety or efficiency. Those tasks with <70% completion rates were then evaluated by a panel of OR experts to be rated for the tasks’ contribution to OR safety or efficiency. The ratings were then reviewed again by a smaller select group of OR experts and 2 practicing clinicians. This iterative process ultimately led to the final list, METEOR, consisting of 80 OR tasks (34 preoperative, 18 intraoperative, and 28 postoperative). METEOR theoretically can be used to identify patient safety vulnerabilities and to identify factors contributing to OR inefficiencies. It serves as a good data collection tool that obviously needs to be combined with constructive feedback to the OR team members. Because of its size and the fact that a real-time observer is needed it is not something that would be applied on a daily basis in your OR suite. However, it has considerable potential as a tool for periodic auditing of compliance with good OR practices and safety measures.
The second study
used an observational methodology to assess failures in postoperative care (Symons
2013). Many of our columns have focused on problematic care of
postoperative patients. A variety of system failures have contributed to
adverse outcomes in such patients. These have included poor handoffs, failure
to identify patients at risk for specific complications, failure to identify
early clinical deterioration and failure to rescue, a variety of problems
related to pain management, and medication-related problems.
Most of those have
stemmed from specific case examples. But the Symons study used observational
tools on a population of general surgical patients to identify and quantify
such process failures. The population included 50 adults undergoing major upper
or lower gastrointestinal surgery (most often for cancer) at a large urban
teaching hospital. Two surgical residents, trained in patient safety research,
observed the care of these patients from the first postoperative day through
discharge. They attended rounds, reviewed patient charts and records, and did
structured interviews with care providers and identified any event that was not
considered routine in the postoperative period. They further broke those events
down into those with or without process failures and assessed the likely
preventability of these events. They also determined whether harm resulted from
the events
A total of 352
nonroutine events were identified in the 50 patients observed for 659 days of
inpatient care. Of these, 256 (73%) were deemed process failures or a median of
4.5 process failures per patient. 85% of process failures were felt to be
preventable and 51% directly led to patient harm or prolonged length of stay.
Process failures occurred in all aspects of care, the most frequent being
medication prescribing and administration, management of lines, tubes, and
drains, and pain control interventions. Process failures accounted for 57% of
all preventable adverse events. Although no patient died or suffered permanent
disability, the frequency of potentially preventable process failures here
points out potential vulnerabilities for serious adverse outcomes.
They further
analyzed their data for factors contributing to the failures and found that
communication failures and delays were the main causes, together accounting for
over half of all process failures and almost three quarters of adverse events.
The authors suggest
that interventions found to be successful in other venues, such as daily goals
and team training programs, might be applied to the postoperative setting.
This was a good study.
The observational methodology allowed them to identify process failures and
events that might not have been found using other methodologies.
It’s important that the feedback from the observational audits be constructive in nature. It should be used to identify gaps in the care processes that can significantly impact patient outcomes. However, the observational methodology also from time to time may uncover risky behaviors that would not otherwise come to attention. Of interest, the NTSB report in our October 2, 2007 Patient Safety Tip of the Week “Taking Off from the Wrong Runway” mentioned that a LOSA Collaborative showed that flight crewmembers who intentionally deviated from standard operating procedures were three times more likely to commit other types of errors, mismanage more errors, and find themselves in more undesired aircraft situations compared with those flight crewmembers who did not intentionally deviate from procedures. We suspect the numbers in healthcare would be similar. So auditing as above might also identify risk for other situations.
References:
Russ S, Arora S,
Wharton R, et al. Measuring Safety and Efficiency in the Operating Room:
Development and Validation of a Metric for Evaluating Task Execution in the
Operating Room. Journal of the American College of Surgeons 2013; 216(3): 472-481
http://www.journalacs.org/article/S1072-7515%2812%2901403-2/abstract
Symons NRA, Almoudaris AM, Nagpal K, et al.
An Observational Study of the Frequency, Severity, and Etiology of Failures in
Postoperative Care After Major Elective General
Surgery. Annals of Surgery 2012; published ahead of print 20 September 2012
Annals of Surgery.
257(1):1-5, January 2013
Print PDF
version
http://www.patientsafetysolutions.com/