Good Machine Learning Practice: 10 Principles for Trustworthy Medical AI

What Good Machine Learning Practice actually is

Good Machine Learning Practice (GMLP) is a set of ten guiding principles, published jointly in October 2021 by the U.S. Food and Drug Administration, Health Canada, and the United Kingdom's Medicines and Healthcare products Regulatory Agency, that describe how a machine learning medical device should be built, tested, and watched over time. They are not a checklist that certifies a product, and they are not law. Think of them as a shared vocabulary for what "done responsibly" looks like, spanning the whole product lifecycle from the first dataset to the thousandth patient after launch. Read together, the ten principles answer a single question that matters to anyone who might be diagnosed or treated with the help of an algorithm: can I trust what this system does, and does anyone keep checking?

I find this framework worth explaining because it is refreshingly practical. It does not chase the hardest questions in AI ethics. It asks the boring, load-bearing ones that decide whether a model helps or quietly harms.

The ten principles, in plain terms

The principles overlap on purpose, but each carries its own weight. Here they are grouped the way I think about them.

Who builds it, and how carefully

Multidisciplinary expertise across the lifecycle. A good medical AI team reaches well beyond data scientists. It includes people who understand the clinical problem, the workflow, the users, and the intended patients. This expertise has to be present from the start, not summoned at the end to bless a finished model.

Good software engineering and security practices. Under the sophistication of a model sits ordinary software, and ordinary software fails in ordinary ways. This principle asks for sound engineering, data quality management, risk management, and cybersecurity, applied with the same discipline expected of any device that can affect a patient.

What it learns from

Representative data. A model reflects the data it was trained on. If the training population does not resemble the people who will actually be scanned, tested, or monitored, the system can work well in the lab and fail in the clinic. Here the ask is that datasets reflect the intended patient population, including relevant demographic and clinical variation, so that performance and limitations can be understood honestly.

Independence of training and test sets. To know whether a model has genuinely learned something, you must test it on data it has never seen and that is truly separate from what shaped it. When the same patients, sites, or hidden signals leak across both sets, accuracy numbers look better than reality. Independence is what keeps a validation result meaningful.

Appropriate reference standards. A model is measured against something we call the truth, often called ground truth. If that reference standard is weak, inconsistent, or biased, every downstream metric inherits the flaw. The principle asks that the reference be well characterized and clinically defensible, because you cannot validate against a moving or muddy target.

How it fits the real world

Model design matched to available data and the intended use. The design should suit the clinical problem, the data on hand, and the way the result will be used. A tool built for one setting can mislead in another. Matching the model to its intended purpose, and being clear about the boundaries of that purpose, is part of the design, not an afterthought.

The human and the AI as one team. Most medical AI is used by a person, so what matters is how the human and the system perform together. This principle asks developers to study that pairing directly. Does the tool help a clinician catch more, or does it lull attention and add automation bias? The unit of performance is the team.

Testing in clinically relevant conditions. A model should be evaluated under conditions that reflect real use, not only a curated benchmark. That means the intended patients, the real equipment and sites, and the actual workflow. Statistically sound study design, independent of the training data, is how the framework asks builders to earn a performance claim.

What users are told, and what happens next

Clear, essential information to users. People relying on the output need to understand what the model does, what it was trained and tested on, how it performs, and where it should not be used. This transparency thread runs through a companion set of guiding principles the same agencies later described for machine learning device transparency. Clarity is a safety feature, because a tool used outside its intended conditions can quietly become unreliable.

Monitored deployment. A model is not finished at launch. Real-world performance can drift as populations, practices, and equipment change, so deployed systems should be monitored and maintained. Related regulatory thinking, such as the concept of a predetermined change control plan, addresses how a manufacturer might describe planned updates in advance rather than freezing a model forever or changing it invisibly.

Why this framework holds up

What I appreciate about GMLP is that it treats an AI system as a clinical intervention with a lifecycle, not a product that ships once. The order of the principles tells a story. Assemble the right people, engineer carefully, learn from data that looks like the real world, test honestly and separately, design for the actual task, measure the human and machine as a team, evaluate in realistic conditions, tell users the truth, and keep watching. Miss any one link and the chain can still snap.

None of the ten principles requires believing that AI is magic or menace. They ask the same questions a careful clinician asks of any new test: where did the evidence come from, whom does it apply to, how was it checked, and who is responsible when it changes. The principles sit at a deliberately high level, which is their strength; they set direction while standards bodies and regulators work out specifics, and international harmonization efforts have continued to build on them.

For clinicians and patients, the practical takeaway is a set of fair questions to ask about any medical AI tool. Was it tested on people like me? Has anyone checked it on data it never saw during training? Is someone still watching it now? This article is educational and not medical, legal, or regulatory advice; for a decision about your own care or your own product, talk with your clinician or a qualified professional.