Let’s Not Always Go By The Numbers

Medicine is obsessed with numbers. Or rather, journalists and medical administrators are. Here are two related examples of how large a grain of salt one must put on numbers.

Cardiac surgical procedures, like everything else in medicine, have quality indicators. One of these is what we doctors call “30-day mortality”. What this term means is that surgeons are evaluated in part on how many of the patients they operated on died within a month of having surgery. Presumably a surgeon whose patients rarely die within 30 days is a better surgeon than one whose patients die all the time. 

The American Academy of Hospice and Palliative Medicine, whose members deal frequently with the elderly, thinks this number, 30, harms old people.

The problem, according to the New York Times, is that surgeons refuse to operate on people who are more likely to die within 30 days, and that they keep patients alive in ICUs until day 31 to keep their numbers up. Bad doctors!

The problem with the number 30 is not that it’s to short or too long, it is that it is a terrible metric for quality. Patients die despite everyone’s best efforts, especially patients who are at higher risk for dying to begin with. We need to find a metric that actually reflects quality of care. Of course doctors are going to be leery of operating on really sick people, if their jobs are at stake. I know people would like for doctors to be saints who take care of everyone all the time with nary a pecuniary thought, but I’m sorry, doctors are not saints. Neither are patients.

Speaking of risk, here’s number two reason numbers are evil. A recent article in the Journal of the American Medical Association reviewed the current literature on how accurate patients assessments of risks and benefits are. The authors found that 65% of the time patients overestimate the benefits and 67% of the time they underestimate the risks. 

The problem, according to Austin Frakt and Aaron Carroll of the New York Times, is that doctors don’t give patients adequate information about risks and benefits. Bad Doctors!

The problem is not that doctors don’t give people the numbers. The problem is that the numbers don’t influence patient’s decisions. Reams of research as well as best-selling books by people like Nobel prize winner Daniel Kahneman tell us that risk assessment has little to do with statistics. 

Humans estimate risk based on things like what is most prominent in the news, how they feel about the risk in question, and how closely they compare to others who have undergone the event in question. 

For example, women who have had bilateral mastectomies after a breast cancer diagnosis were asked how much the surgery had decreased their risk of recurrence. The average response was women felt their risk had gone from 76% to 11%. The actual risk before surgery is actually only 17%, so the surgery reduces the risk of recurrent breast cancer six percentage points. (This is for women who don’t have the BRCA gene). 

I’m sure women are told what the risk of recurrence is and how much the surgery decreases the risk. I’m sure they are. But the numbers are being told to women who are scared out of their minds about breast cancer and just want it to go away. They don’t hear nor care what the statistics are. That’s called being human. Pick a subject. Vaccinations – gross overestimation of risk because the guy down the street has an autistic kid. Dying in a plane crash – driving in your car is way more dangerous but the newspaper just had a big story about a horrific plane crash. Ebola – one case in the US but everyone is afraid they will get infected because it’s a really bad disease.

Please. No more numbers.

Shirie Leng, M.D., is an anesthesiologist. A version of this article originally appeared on the Health Care Blog.