Validating healthcare AI

The Cost of a False Positive: Why Medical Errors Are Not Symmetric

A false positive is a test that says disease is present when it is not, and a false negative is a test that says disease is absent when it is there. These two errors deserve separate names because they almost never cost the same.

A false positive is a test that says disease is present when it is not, and a false negative is a test that says disease is absent when it is there. These two errors deserve separate names because they almost never cost the same. Telling a healthy person they may be sick triggers anxiety, more tests, and sometimes a needless procedure; telling a sick person they are fine can let a treatable problem run unchecked. Because the consequences differ, the threshold at which you call a test positive should differ too, and choosing that threshold is a value judgment dressed up as a technical one. This is an educational piece, not medical advice, so use it to reason about tests and then decide with your own clinician.

I have spent time on both sides of this line. My doctoral research at the Lund University Diabetes Centre is about who is at risk for type 2 diabetes, which is a long argument about where to draw cutoffs, and in building AI clinical decision support I have watched the same threshold question decide whether a system is trusted or quietly switched off. The math is the easy part. The asymmetry is where the judgment lives, and it is the part most often left unspoken.

Why are false positives and false negatives not symmetric?

The cleanest way to see the asymmetry is to name what each error sets in motion. A false positive rarely ends with the test. It starts a cascade. The patient carries worry home before anyone can confirm anything, then comes the confirmatory workup, some of it invasive and each step with its own small risk. There can be a biopsy of a lump that was never going to harm anyone, or treatment of a finding that would have sat quietly for a normal lifetime. The label itself can linger in a chart and color how every later symptom is read.

A false negative fails in the opposite direction, by doing nothing when something was needed. The reassurance is the harm. A person told they are clear may skip the follow-up and arrive later with a problem that was once easier to treat. That cost lands downstream, sometimes much later, which is exactly what makes it easy to underweight.

So the two errors are not mirror images. One spends money and peace of mind on people who were fine; the other withholds attention from people who were not. Which is worse is not a fact about the test. It is a fact about the disease, the treatment, and the person in front of you.

What is a diagnostic threshold, and why does moving it trade one error for the other?

Most tests do not return a clean yes or no. They return a number on a continuum, a level in the blood, a shade on an image, a score from a model, and someone has to decide how high that number must climb before the result counts as positive. That cutoff is the threshold.

Picture two bell-shaped piles of scores side by side, one for people without the condition and one for people with it, at different averages but with their tails crossing in the middle. The threshold is a vertical line: everything to its right is called positive. Slide it left and you catch more true cases while the slice of healthy people you wrongly flag grows, buying sensitivity at the price of false positives. Slide it right and the reverse happens, buying specificity at the price of false negatives. Because the piles overlap, and they almost always do somewhere, no line makes both errors vanish. The threshold only lets you choose which mistake to make more often.

How should the consequences set the threshold?

The threshold should follow one question: what happens after each kind of mistake, and which does more harm here? When the answer changes, the right line moves with it.

Consider a condition that is dangerous, fast moving, and treatable, where missing it is catastrophic and the confirmatory workup is low risk. Here a false negative is the error to fear, so you set a low threshold, flag many people who turn out fine, and lean on a more specific second test to sort them. The false positives are the toll you pay to almost never miss a true case.

Now reverse it. Consider a finding that is common, slow, and often harmless, where the confirmatory step is invasive and treatment carries real risk of its own. Here a false positive is the error to fear, because chasing it can hurt people who were never in danger. You set a higher threshold and accept that some mild cases slip past, trusting that the dangerous ones declare themselves clearly enough to cross the line. Same machinery, opposite settings, both correct. Anyone who says a test should simply be "more accurate" without naming which error they want to reduce has not finished the thought.

Why base rates make the cost of a false positive worse than it looks

There is a second force that surprises even careful people, and it has nothing to do with the quality of the test. It is how rare the condition is to begin with. When a disease is uncommon in the group being tested, most positive results are false, even with a genuinely good test, because a small percentage of a very large healthy group can outnumber the true cases pulled from a small sick group. The false positives, each rare on its own, pile up faster than the true positives. That is the case for testing the right people rather than everyone: a test that performs well in a high-risk clinic can throw mostly false alarms across a low-risk crowd.

What this means for screening and for AI in the clinic

Screening healthy people is where the asymmetry bites hardest, because the population is, by definition, mostly well. The benefit lands on the few with early disease while the cost of false positives spreads across the many who were fine: the worry, the follow-up tests, the occasional treatment of something that would never have caused trouble. A screening program earns its place only when the harm of the misses it prevents clearly outweighs the harm of the false alarms it creates, and that balance has to be argued, not assumed.

The same logic governs the AI systems I help build. A model that outputs a probability is just a test with an adjustable threshold, and where you set the alert line is an ethical decision, not a default to inherit from the software. Set it loose and you flood clinicians with low-value alerts until they stop reading them. Set it strict and you miss the cases the tool was meant to catch. The honest version states which error it is tuned to avoid and why, and lets the people who carry the consequences question that choice.

A test result, then, is not a verdict handed down by nature. It is one piece of evidence whose meaning depends on who was tested, how rare the condition is, and which mistake the threshold was set to avoid. Ask of any test, including any algorithm, two questions. Which error did you choose to make more often, and was that the right error for me? A test that can answer is doing its job. One that pretends it never had to choose is hiding the most important thing about itself.

References and sources

  1. StatPearls Diagnostic Testing Accuracy
  2. NCBI ROC Analysis Threshold Trade-off
  3. Cancer Overdiagnosis in Screening Review

How this was researched. This explainer is built from the primary sources listed above and reflects Dr. Tojjar's own critical appraisal of that evidence. It explains and evaluates research and does not provide medical care.

This article is for general education and is not medical or professional advice. For guidance about your own health, talk with a qualified clinician.

Cite this article

Tojjar, D. (2023). The Cost of a False Positive: Why Medical Errors Are Not Symmetric. Dr. Damon Tojjar. https://readingtheevidence.org/articles/the-cost-of-a-false-positive/

Back to all insights