Watching an AI Device in the Wild: Real-World Performance Monitoring

A real-world performance monitoring plan is the ongoing process that tracks an approved AI medical device once it is used on actual patients, so that a quiet loss of accuracy is caught before it reaches the bedside. The central worry is data drift: an AI model can degrade over time as patient populations, imaging equipment, laboratory methods, documentation habits, and clinical guidelines shift away from the data the model learned on. Unlike a stethoscope or an infusion pump, an AI device that performed well at clearance is not guaranteed to keep performing well, which is why the U.S. Food and Drug Administration has framed monitoring as a lifecycle obligation rather than a one-time checkpoint. A credible plan names what will be measured, where the data will come from, what change will trigger an alarm, and who will act on it.

Why AI devices need watching that other devices do not

Traditional device review establishes a fixed performance baseline and assumes the device behaves the same way tomorrow as it did during testing. AI-enabled software breaks that assumption. FDA has described the problem directly: performance can change or degrade because of shifts in clinical practice, patient demographics, data inputs, and healthcare infrastructure, along with changes in how clinicians use the tool and how it fits into a workflow. A sepsis-prediction model validated at three academic hospitals may falter at a rural site with different lab reference ranges. An imaging model trained on one manufacturer's scanners can lose accuracy when a hospital replaces its fleet. The model itself has not changed, but the world feeding it has, and its outputs drift.

In its draft guidance on AI-enabled device software functions, issued January 7, 2025 and published in the Federal Register the same week, FDA laid out a total-product-lifecycle approach and recommended that developers build postmarket performance monitoring into the plan precisely because these devices can degrade in real-world use. The agency treats detection, assessment, and mitigation of that degradation as expected practice across the device's life, not an optional extra. The draft guidance opened a public comment period that ran through April 7, 2025, and it remains a draft, meaning it describes FDA's current thinking rather than a binding final rule.

What FDA is still working out

The science of monitoring is genuinely unsettled, and FDA has said so by asking for help. On September 30, 2025, the agency opened a Request for Public Comment (docket FDA-2025-N-4203, with comments due December 1, 2025) on how to measure and evaluate the real-world performance of AI-enabled devices, including strategies for identifying and managing performance drift by detecting changes in inputs and outputs. The request was explicitly not new guidance or a new regulatory expectation; it was an attempt to advance the conversation on practical methods.

The request grouped its questions into six areas that map neatly onto the anatomy of a monitoring plan: performance metrics and indicators; real-world evaluation methods and infrastructure; postmarket data sources and quality management; monitoring triggers and response protocols; human-AI interaction and user experience; and additional best practices. Reading those six headings is a useful exercise, because they are the same questions any hospital or developer must answer before it can claim to be watching a device rather than merely hoping it still works.

What a monitoring plan actually contains

A serious plan is concrete on five fronts.

Metrics and a baseline

You cannot detect decline without a fixed reference. The plan records the accuracy, sensitivity, specificity, calibration, and subgroup performance the device showed at authorization, and then re-measures the same quantities in the field. Subgroup tracking matters because overall accuracy can hold steady while performance quietly collapses for one demographic.

Input and output surveillance

Because outcome labels often arrive late, mature plans watch the data itself. Input monitoring flags when incoming cases no longer resemble the training distribution, so-called out-of-distribution data. Output monitoring watches for shifts in the rate or spread of the model's predictions. Both can signal trouble before a single confirmed error is counted. FDA's device-science laboratories have described research on proactively monitoring data drift and model performance, including federated evaluation approaches that let multiple sites contribute without pooling raw patient data.

Triggers and thresholds

A metric with no threshold is a dashboard nobody reads. The plan states, in advance, the drop in performance or the drift signal that moves the device from normal to investigate to intervene, so the response is a pre-agreed rule rather than an argument held during a crisis.

A response pathway

Detection is worthless without action. The plan names who is notified, how quickly, and what options exist: heightened human review, restricting the device to populations where it still performs, retraining, or withdrawal. It also connects to adverse-event reporting when a performance problem may have affected care.

Governance and cadence

Someone owns the plan. The document assigns responsibility, sets a review schedule, and records every finding, because monitoring is a continuous loop rather than an annual audit. Planned, pre-authorized model updates fall under a separate change-control mechanism, and monitoring is what tells you an update is needed in the first place.

Drift itself, its statistical flavors, and why it is the failure mode monitoring exists to catch are covered in more depth in the companion piece on model drift and monitoring within this Validating Healthcare AI collection. The short version is that monitoring is the operational answer to a scientific certainty: a model trained on the past is deployed into a future that keeps changing, and only measurement tells you when the gap has grown dangerous.

This article is educational and is not medical advice.