The EU AI Act and Medical Software, in Plain Terms

The EU AI Act is a horizontal law that sorts artificial-intelligence systems by how much risk they pose, and most software that helps make a clinical decision lands in its high-risk tier. That is the short answer to a question founders already buried by medical device rules keep asking: is this a new burden on top of the old one, or the same idea wearing a different hat? It is closer to a second layer than a replacement. The device rules ask whether your software is safe and works as promised. The AI Act asks, in addition, whether the artificial-intelligence part is governed well across its life.

This piece explains those principles in plain terms and shows how they relate to the medical device framework many readers already know. It is educational and general, written from a regulatory-training perspective rather than as legal advice. Anyone building a real product still needs a qualified assessment of it.

What the AI Act is trying to do

The Act starts from a simple premise. The same algorithm can be trivial in one setting and dangerous in another. A model that recommends a film is not the model that flags a tumor. So the law refuses to regulate "AI" as one thing and assigns obligations by use, scaling what it demands to what could go wrong if a system fails.

That produces a tiered structure. A small set of uses is prohibited outright. A larger set is treated as high-risk and carries the heaviest duties. Most everyday systems fall into lighter categories with mainly transparency expectations, such as telling a person they are talking to a machine. The center of gravity for medicine sits in the high-risk tier, because software that informs diagnosis or treatment is exactly the kind of use the law worries about most.

The instinct will feel familiar to anyone trained in clinical settings, where care is triaged by how much is at stake. The Act applies that same logic to software, and once you see it that way, the structure stops feeling arbitrary.

Why medical software usually lands in the high-risk tier

The Act has a specific route into its high-risk category for products already regulated as medical devices. In rough terms, if your software is a device or a safety component of one under the existing European device framework, and that framework already requires an independent body to review it, the AI Act treats it as high-risk as well. The two regimes are deliberately wired together rather than left to contradict each other.

That linkage answers the "is this double work" fear directly. The design intends the AI requirements to be met through the same conformity assessment a device already undergoes: one coherent file, examined by a body competent in both. So if your tool is the kind of software that climbs the device risk ladder because it drives a clinical decision, assume the high-risk duties apply too, and plan for both at once.

What the high-risk obligations actually ask for

Strip away the vocabulary and the high-risk duties describe a system a careful clinical team would want to build anyway. There must be a risk-management process that runs across the system's life, not a one-time sign-off. There must be governance of the data used to train, validate, and test the model, with attention to whether it is relevant, representative, and as free of harmful gaps as the use demands.

There must be documentation good enough that someone who did not build the system can understand what it does and reconstruct how it behaved. There must be transparency toward the people using it, so a clinician knows the tool's purpose and limits rather than treating its output as settled fact. There must be meaningful human oversight, so a person can understand, question, and override the system. And there must be a defensible standard of accuracy, robustness, and security, claimed and then actually measured.

That list amounts to the discipline that separates a tool that demos well from one that earns clinical trust. The evidence that earns trust comes from studying a tool against ordinary care rather than against its own training data. EASY Diabetes, a decision-support system I co-developed, was studied this way in a registered randomized controlled trial (NCT03258268). The Act is, in effect, asking for that habit in writing.

How it relates to the rules you already know

It helps to keep the two jobs separate. The device framework is mainly about the product as a medical claim: this software is intended for this purpose, here is the evidence it is safe and effective, here is how we watch it in the field. The AI Act is mainly about the artificial-intelligence system as an engineered artifact: how its data was governed, how its behavior is logged, how a human stays in control, how we know it has not silently drifted.

Those jobs overlap, which is the point of wiring the regimes together. Post-market surveillance under the device rules and post-market monitoring under the AI Act share an instinct, that approval is the start of a system's evaluated life and not the end of it. A serious team builds one monitoring plan and lets it satisfy both. The device rules say the software must be safe and work as claimed. The AI Act says how the artificial-intelligence part must be governed to stay that way. They describe complementary halves of one obligation.

A careful way to think about timing and scope

The details of dates, thresholds, and procedures are exactly what a qualified assessment exists to pin down for a given product. The Act phases in over time, and the precise obligations depend on the system's category and on how it interacts with the device rules. Treat anything you read, including this article, as orientation rather than the operative answer.

What is stable enough to act on is the direction. An AI system used in care should have a stated purpose, evidence proportionate to its claim, governed data, a human who can meaningfully intervene, and monitoring that outlives the launch. A team that designs for those principles from the first sprint finds the specific rules far easier to meet than one that bolts them on when a reviewer asks. The most useful early question is also the oldest one. What are we claiming, and what would it take to prove it and keep proving it?

This article is general education from a physician-scientist's perspective, not legal or medical advice, and no substitute for qualified regulatory counsel or a clinician's guidance on your care.