Prediction vs Explanation: Two Different Questions a Model Can Answer

A model that predicts well tells you what is likely to happen next. A model that explains tells you why it happens, in terms you could act on to change the outcome. These are different questions, and a model excellent at one can be useless at the other. The mistake I see most often, in medicine and machine learning alike, is reading a strong prediction as if it had delivered an explanation, then making a decision that only an explanation could justify. The fix is to ask, before anything else, which question the model was built to answer. (This is general education, not medical advice; decisions about your own care belong with a qualified clinician.)

Much of my week goes to reading models, and I have helped build one. I keep the claim "this forecasts the outcome" apart from the claim "this reveals the cause," because the evidence for each differs.

What is the difference between predicting and explaining?

Prediction is about accuracy on an outcome; explanation is about the mechanism that produces it. A predictor proves its value by being right about what comes next, regardless of how it gets there. An explanation proves its value by being right about why, which is what lets you change the outcome.

Consider a clinic that flags patients likely to be readmitted, using prior admissions as the main signal. The feature predicts beautifully and explains nothing you can act on, because you cannot lower a patient's risk by erasing their history. The model answered "who," the clinician quietly heard "why," and that gap is where trouble starts.

Why does a model that predicts well not always explain anything?

Because prediction only requires a stable statistical relationship, and a relationship can be stable without being causal. A variable can track the outcome reliably while being a marker of something else entirely.

The cleanest illustration is a proxy. Imagine an imaging model that detects disease with high accuracy, and someone later finds it keyed partly on a scanner artifact, because sicker patients happened to be imaged on older equipment at one site. The prediction was real on that data; the explanation was a fiction. Move to a hospital with newer machines and the accuracy collapses, because the model never learned the disease. It learned the equipment.

When prediction is exactly what you need

None of this makes prediction the lesser goal. For many decisions, what matters is the forecast, not the story behind it. If you are deciding which scans a radiologist should read first, a model that ranks urgency correctly is valuable even when its reasoning is opaque. The ranking earns its place, as long as no one reads it as a cause.

Why does a model that explains not always predict best?

Because the true mechanism is often a weaker predictor than a convenient proxy. The honest variable can be noisy, hard to measure, or only one of several roads to the outcome, while the proxy is clean and abundant.

My own research has run into this directly. A meta-analysis I published in Diabetes Care examined how insulin sensitivity and insulin response relate across populations, and the lesson was that one end point, high blood sugar, can be reached by different physiological routes in different people. A model built on that one mechanism may predict worse, on a mixed population, than a shortcut riding a correlation that holds across the whole group. The explanatory model is not failing; it is refusing to pretend one road is the only road.

Why does conflating the two mislead in medicine?

Because clinical action is usually an attempt to change an outcome, and only a mechanism tells you whether your lever is connected to anything. Treat a predictor as a cause, and you may push hard on a variable that moves nothing.

The classic shape of this error is intervening on a marker. A feature predicts poor outcomes well, a team reasons that pushing it the other way should help, but if the feature was a symptom rather than a driver, the intervention changes nothing. My peer-reviewed work on the genetics of type 2 diabetes, including a study in Diabetologia, lives on this distinction: a variant can sit beside a disease signal and predict it without causing the damage, which is why a strong association opens an investigation rather than justifying a treatment.

Why does conflating the two mislead in AI?

Because a high benchmark score invites a causal story the model never earned, and the danger grows when the system also offers an explanation, since a fluent rationale persuades whether or not it is true.

A predictive system can generate a plausible reason for its output that has nothing to do with how it computed that output. The reason reads like a mechanism, a reader treats it as one, and a forecast is suddenly doing the work of an explanation it never contained. This is why I stay cautious about saliency overlays and feature-importance charts presented as the model's reasoning. They show what the model attended to, not that the highlighted feature caused anything, and a confident display can manufacture unearned trust.

How can you tell which question a model actually answered?

The test is what the model was validated against, not what its authors hope it means. Ask whether the evidence on the table supports a forecast or a cause, and refuse to let one stand in for the other.

For a predictive claim, the right evidence is performance on a genuinely separate population, with both ranking and calibration reported. For a causal claim, the right evidence is an intervention: change the suspected cause and watch whether the effect moves. Those are different studies, and most published models have attempted only the first. When I co-developed EASY Diabetes and we took it into a registered randomized controlled trial (NCT03258268), the discipline that mattered was keeping the two claims apart.

A reviewer's short version

When I read a model now, I ask one question first: is this built to tell me what will happen, or why it happens? If the answer is "what," I judge it on out-of-sample accuracy and let no one narrate a mechanism from it. If the answer is "why," I look for an intervention.

Both kinds of model are worth having. The harm comes when a good forecast gets read as a good explanation and a decision rests on a footing it never had. Keeping them apart is how you avoid acting on a why that was only ever a what.