Validating Healthcare AI: Test It Like a Medicine, Not a Benchmark

If you want to know whether a clinical AI tool actually helps patients, there is one honest answer: test it the way we test a drug. Put it in front of real patients and real clinicians, randomize who gets it, and measure what happens to the outcomes that matter. A high score on a benchmark dataset tells you the model can pattern-match on data that looks like its training set. It tells you almost nothing about whether a doctor using it on a Tuesday afternoon will make a better decision for the person in front of them.

This distinction gets lost a lot, because benchmarks are cheap and trials are not. A model can hit 95 percent accuracy on a held-out dataset and still fail in clinic, because the clinic is not the dataset. Patients arrive with missing labs, confusing histories, and conditions the model rarely saw. Clinicians ignore alerts they do not trust, or trust alerts they should question. The gap between "the model is accurate" and "the tool improves care" is where most healthcare AI quietly underperforms.

A benchmark measures the model. A trial measures the system.

Here is the part that benchmarks structurally cannot capture: in real care, the unit that matters is not the model, it is the clinician plus the model plus the workflow. A recommendation that is technically correct but arrives three clicks too late, or in language a busy physician skims past, changes nothing. A tool that is slightly less accurate but fits the way a clinic already runs can outperform a sharper model that nobody uses properly.

That is why I argue clinical AI should be evaluated as a medical intervention rather than as software. A medicine is not approved because it works in a test tube. It is approved because, in people, it produces a benefit that outweighs its harms, measured against a comparison group. AI deserves the same bar. The relevant question is not "how accurate is the model" but "compared to standard care, did the patients managed with this tool end up better off." Those are different questions, and only the second one is the one patients care about.

What a real evaluation looks like

A credible evaluation of clinical AI borrows the architecture of a randomized controlled trial. You define the outcome before you start, not after you see the data. You randomize, so that the group using the tool and the group getting usual care are comparable and you are not just measuring which clinics were already doing well. You run it across enough sites that you are testing the tool, not one unusually motivated department. And you measure clinical outcomes and workflow effects, not just whether clinicians liked the interface.

We did this with EASY Diabetes, an AI clinical decision-support system for type 2 diabetes that I co-developed with patients, clinicians, and researchers. The trial, EASY-1 (registered as NCT03258268), was a multi-clinic randomized controlled trial that compared the tool against standard of care. The structure is what matters: a registered trial, a real comparator, multiple sites, prespecified endpoints. That is the kind of design that can tell you whether a tool helped, rather than merely that it shipped.

Lessons from running EASY-1

A few things stand out from doing this in practice rather than describing it in a slide.

First, build the tool with the people who will use it, not for them. EASY Diabetes was developed alongside patients and clinicians from the start. That is not a goodwill gesture. It is what keeps the model's outputs aligned with how decisions are actually made in a consultation, which is what determines whether the recommendation gets used at all.

Second, multi-site is not a luxury, it is the test. A tool that works in one clinic may be quietly relying on that clinic's habits. Spreading across many clinics is uncomfortable, because variation shows up, but that variation is exactly the real world you are claiming to improve.

Third, measure the workflow, not only the diagnosis. EASY-1 looked at efficiency alongside clinical outcomes because a tool that improves decisions while doubling a clinician's workload will not survive contact with a real schedule. Adoption is part of efficacy.

The regulators are converging on this view. Medical device frameworks now treat software, including AI and machine learning systems, as something that requires clinical evidence, not just a demonstration that the code runs. That is the right direction. The leaderboard culture of general AI is a poor fit for medicine, where the cost of a confident wrong answer is measured in people.

This article is educational and reflects my own view as a researcher. It is not medical advice, and it is not regulatory guidance for any specific product.