Real-World Evidence in Regulation: What Data Outside a Trial Can and Cannot Decide

Regulators already use real-world evidence, and they have for years, but they use it for a narrower set of questions than the enthusiasm around it suggests. Data generated during ordinary care can settle questions about safety in the broad population, describe how a product performs once it leaves the trial's protected conditions, and support certain label changes for already-approved products. What it struggles to settle on its own is the hard causal claim at the center of a first approval: that this product, and not the kind of patient who receives it, produced the benefit. The frameworks that agencies have built around real-world evidence are mostly machinery for telling those two situations apart.

The vocabulary matters. Regulators distinguish real-world data, the raw material drawn from electronic health records, insurance claims, product and disease registries, pharmacy records, and increasingly digital health devices, from real-world evidence, which is the clinical conclusion produced when that data is analyzed to answer a defined question. Data is not evidence until a method turns it into an argument, and the quality of the method is where most of the disagreement lives.

Why regulators built formal frameworks at all

In the United States, the current framework did not arise spontaneously. Section 3022 of the 21st Century Cures Act, signed in 2016, directed the Food and Drug Administration to create a program for evaluating whether real-world evidence could help support approval of new indications for already-approved drugs and help satisfy post-approval study requirements. The agency published a framework for that program in December 2018 and has issued guidance since on how it thinks about the reliability and relevance of the underlying data. The point of writing it down was to make the standard predictable, so that a sponsor considering a real-world study could know in advance roughly what would be persuasive.

In Europe, the European Medicines Agency has moved along a parallel track. It established DARWIN EU, a network that can query real-world healthcare databases across member states to produce evidence on how medicines are used and how they perform once in wide use. Alongside it, the agency has developed a data quality framework that assesses sources on qualities like conformance, completeness, and plausibility before their data is trusted for a regulatory question. Both jurisdictions arrived at the same instinct: real-world evidence is worth using, and precisely because it is worth using, its inputs need a formal quality gate.

What real-world evidence decides well

Safety is the clearest case. Rare adverse events often do not appear in a trial of a few thousand people watched for a year, but they surface in the records of millions treated over a decade. Post-marketing safety surveillance leans heavily on real-world data because the alternative, a trial large enough to catch a one-in-fifty-thousand event, would be impractical and often unethical to run. Here the observational setting is not a compromise. It is the only instrument with the reach to see the signal.

Real-world evidence also describes generalizability, which trials handle poorly by design. A trial enrolls selected people and excludes the frail, the multiply-medicated, and the non-adherent, yet those are exactly the people who fill a clinic. Data from ordinary care answers whether a benefit shown under ideal conditions survives contact with the messy population, and regulators treat that as a genuine and useful question. For some label expansions, particularly where a comparison is stark or a trial would be infeasible, agencies have accepted real-world evidence as part of the supporting package, judged case by case against the strength of the design.

Where the frameworks apply their brakes

Confounding is the reason a first causal approval rarely rests on observational data alone. When the reason a patient received a treatment is also related to how that patient was going to fare, the treatment can take credit, or blame, for an effect it did not produce. The sickest patients receive the most aggressive therapy, so a strong drug can look harmful when the underlying illness did the damage. People who adhere to any preventive therapy tend to be more health-engaged in ways no dataset fully records, so the therapy looks protective partly because of who its users already were. Statistical adjustment can correct for the confounders that were measured. It cannot correct for the column that was never in the table, and the most important variables, frailty, motivation, the clinical gestalt a physician forms in a room, are often the unmeasured ones.

Data quality is the second brake, and it is what the European framework's conformance, completeness, and plausibility checks are chasing. A record built for billing was not built for research, and it may code a diagnosis for reimbursement rather than accuracy, miss outcomes that happened at another institution, or contain values that are simply implausible. A feasibility assessment can reveal that a proposed source lacks information on key confounders or has too few relevant patients, and some of those gaps cannot be repaired after the fact. Evidence can be no better than the data beneath it, however sophisticated the analysis layered on top.

The third caution is the one methods people care about most: a question shaped after seeing the answer is not a test. Health datasets are wide enough that trying enough comparisons will surface something significant by chance. This is why regulators reward pre-specification, a written commitment to the population, comparison, outcome, and analysis before the data is examined. A useful discipline, and one agencies increasingly ask for, is to design the observational study to emulate the randomized trial you wish you could run, then defend it against the biases that emulation cannot remove.

How I have used the distinction in practice

In building the EASY Diabetes decision-support system, real-world usage data was invaluable for understanding how clinicians actually behaved and where a workflow quietly failed. But the hypothesis that the system could improve outcomes against standard care, we tested in EASY-1, a registered randomized controlled trial (NCT03258268), because that was the only design that could carry a causal claim of that weight. My training in FDA clinical investigation and in medical device regulation reinforced the same lesson from the regulator's side: the craft is matching the design to the claim, not ranking one source above the other.

Real-world evidence and randomized trials are not competitors. They answer different questions, and the frameworks agencies have built are best understood as instructions for keeping the questions straight. This article is educational and reflects public policy, not legal or medical advice; for decisions about your own care, talk with a clinician who knows your history.