## Too Many Steps?

In medicine, we often use probabilities to help us decide about diagnoses or whether a procedure should be done. The chance of errors occurring is a function of the number of steps involved in a process. Probabilities are not simply derived from addition of risks but rather probabilities are the result of multiplying the risks of each step.

We often quote that a 50-step process, such as might occur in the continuum from ordering a medication to the patient actually receiving the medication, with a 1% error rate at each step would result in an overall potential error rate of 39%.

But when it comes to multistep clinical issues, we often fail to take understand how probabilities are calculated and we may underestimate the chance of unwanted outcomes or overestimate the chance of successful outcomes.

A recent study illustrated this phenomenon. Arkes et al. (Arkes 2022) note that the probability of a conjunction of 2 independent events is the product of the probabilities of the 2 components and therefore cannot exceed the probability of either component. Violation of this basic law is called the conjunction fallacy. They note that conjunction fallacy can lead to diagnostic or prognostic errors.

They used 3 scenarios and surveyed 215 experienced physicians and found that 78.1% estimated the probability of a medical outcome resulting from a 2-step sequence to be greater than the probability of at least 1 of the 2 component events, a result that was mathematically incoherent.

One scenario, given to obstetricians, was that of a brow presentation discovered during labor. They were asked the probabilities that the brow presentation would resolve and that the delivery would be vaginal. 50 of 67 obstetricians (74.6%) committed the conjunction fallacy, overestimating the combined probability by 12.8%. This could result in delaying a C-section, which could be detrimental to the child or mother.

The second scenario was an incidentally discovered pulmonary nodule. The probability that a biopsy reveals cancer in the patient is a function of both the probability that the nodule is cancerous and the probability of the biopsy successfully detecting cancer in the presence of a cancerous nodule. 73 of 84 pulmonologists (86.9%) committed the conjunction fallacy, overestimating the combined probability by 19.8%.

The third scenario was a variation of the first one, but they attempted to “debias” the physicians’ estimates. But even in that one, 45 of 64 obstetricians (70.3%) committed the conjunction fallacy, overestimating the combined probability by 18.0%.

The authors caution that “because many diagnostic and prognostic decisions require more than 1 step or the consideration of more than 1 probability, this misestimation may have substantial implications for diagnostic and prognostic decision-making.”

References:

Arkes HR, Aberegg SK, Arpin KA. Analysis of Physicians’ Probability Estimates of a Medical Outcome Based on a Sequence of Events. JAMA Netw Open 2022; 5(6): e2218804