Real-World Evidence in Regulatory Submissions

Introduction

Real-world evidence (RWE) is evidence about the use, benefits, and risks of medical products that is generated from analysis of real-world data (RWD). RWD can come from electronic health records, administrative claims, disease or product registries, pharmacy data, laboratory systems, linked datasets, and other routinely collected healthcare information.

The regulatory question is not simply whether a dataset is large. The central question is whether the data and study design can generate credible evidence for the specific regulatory question being asked.

RWD, RWE, and Regulatory Evidence

Where RWE Can Contribute

FDA states that RWD and RWE are used across the medical-product lifecycle, including premarket and postmarket evaluation of effectiveness and safety. FDA's August 2023 guidance describes considerations for using RWD/RWE to support regulatory decisions for drugs and biological products, including new indications and postapproval study requirements.

EMA similarly describes RWE as a way to complement evidence from clinical trials and support regulatory decision-making throughout the medicine lifecycle. Its RWE program includes regulator-led studies and the DARWIN EU network.

Start With the Regulatory Decision, Not the Database

A common RWE mistake is to begin with a convenient dataset and ask what can be analyzed. A stronger approach begins with the decision that the analysis must inform.

Write the question in a structured form:

For example, instead of saying “Use claims to study survival,” specify the population, treatment strategies, time zero, outcome, follow-up, and intercurrent-event strategy.

The Estimand Comes Before the Estimator

An RWE study should define what effect is intended to be estimated before selecting the statistical method.

Regulatory Fitness for Use

FDA's 2024 final guidance on EHR and medical-claims data emphasizes assessment of whether RWD are sufficiently reliable and relevant for the proposed clinical study. FDA's 2023 final guidance also emphasizes the need to consider data source selection, study design, and the ability of the data to support the regulatory question.

Data Source Selection

Different RWD sources provide different windows into clinical reality.

Data Provenance: The Hidden Backbone of a Submission

A regulator should be able to understand where the evidence came from and how it was transformed. A robust RWE package therefore treats provenance as a first-class analytical object.

Example: A Treatment-Effect Question

Suppose a sponsor wants to compare 1-year mortality after initiation of Drug A versus an active comparator in routine practice.

Even this simple expression hides important design decisions: who qualifies as a new user, what defines treatment initiation, when follow-up begins, how treatment switching is handled, which deaths can be observed, and how baseline confounding is controlled.

Target Trial Emulation

For comparative effectiveness questions, target trial emulation is a useful framework for making the causal question explicit. The observational study is designed to mimic the protocol of the hypothetical randomized trial that would answer the question.

Immortal-Time Bias

Immortal-time bias occurs when a period during which the outcome cannot occur under the study definition is incorrectly assigned to an exposure group.

Consider a study defining “treated” patients by requiring a prescription sometime during the first 90 days. If treated patients are followed from Day 0 but cannot be classified as treated until a later prescription, the exposure definition and time origin are misaligned.

A target-trial framework forces the analyst to confront this problem explicitly.

Confounding

Unlike randomization, routine clinical care does not generally assign treatment independently of patient characteristics. Treatment choice can depend on prognosis, disease severity, prior therapy, clinician preference, access, comorbidity, and many other factors.

The causal target can be written as:

where \(Y^{(1)}\) and \(Y^{(0)}\) represent potential outcomes under the two treatment strategies.

The core challenge is that only one potential outcome is observed for each patient. RWE methods attempt to construct a credible counterfactual comparison using measured information and design assumptions.

Directed Acyclic Graphs

A causal diagram can help identify variables that should be measured and adjusted for.

Propensity Scores

A propensity score is the probability of receiving the observed treatment conditional on measured baseline covariates.

Common applications include matching, weighting, stratification, and covariate adjustment.

For inverse probability of treatment weighting, a simple binary-treatment weight can be written:

Stabilized weights are often preferable in practice because they can reduce variance and improve numerical behavior.

Overlap Matters

A sophisticated propensity-score model cannot solve a lack of treatment overlap.

If some patients have a near-zero probability of receiving one treatment, comparisons for those patients are driven by extrapolation rather than observed clinical practice.

Illustrative Propensity-Score Overlap

Outcome Definition Is Often Harder Than It Looks

Regulatory-grade outcomes require operational definitions. “Hospitalization,” “progression,” “response,” and even “death” can be represented differently across data sources.

Validation and Outcome Algorithms

A code appearing in a database is not automatically equivalent to a clinical event. When possible, outcome definitions should be supported by validation evidence, chart review, adjudication, or other evidence appropriate to the question.

Document:

• Code lists and coding versions
• Look-back and look-forward windows
• Required combinations of diagnosis, procedure, medication, or laboratory evidence
• Positive and negative predictive value when available
• Known limitations of the algorithm

Missing Data

RWD are frequently incomplete. Missingness can arise because care occurred elsewhere, clinicians did not measure a variable, a field was not captured, or the data-generating system changed over time.

Let \(M\) indicate whether a variable is observed. Missingness mechanisms are often described as:

These labels are assumptions about the missing-data mechanism, not properties that can always be established from observed data.

Missingness Should Be Characterized, Not Hidden

Time-Varying Exposure

Real-world treatment often changes after initiation. Patients may discontinue, switch, restart, or add therapies.

A baseline exposure analysis and a time-varying exposure analysis answer different questions.

Time-varying methods can be appropriate, but they require careful handling of time-dependent confounding and treatment history.

External Control Arms

An external control arm can be attractive when randomized control data are difficult to obtain, especially in rare diseases or settings with substantial unmet need. But comparability is a central issue.

Questions include:

• Are eligibility criteria aligned?
• Is the index date defined consistently?
• Are treatment eras comparable?
• Are outcome definitions equivalent?
• Are prognostic factors measured in both sources?
• Could differences reflect referral or healthcare-system selection?

Calendar-Time Bias

Clinical practice changes. Diagnostic methods, supportive care, treatment sequencing, standards of care, and coding systems may differ across years.

If the treated cohort is primarily from a recent period while the external control is historical, the treatment effect may be confounded by calendar time.

Transportability and Generalizability

A database may contain millions of patients yet still represent only a subset of the population relevant to a regulatory decision.

Compare:

• Age and sex distribution
• Disease severity
• Comorbidity burden
• Prior treatment history
• Race and ethnicity where appropriate and legally/ethically permitted
• Healthcare setting
• Geographic coverage
• Specialist versus community care

Large sample size does not compensate for systematic differences between the study population and the population to which the regulatory conclusion will be applied.

FDA Perspective

FDA's August 2023 final guidance addresses considerations for using RWD and RWE to support regulatory decisions for drugs and biological products. FDA also issued final guidance in July 2024 on assessing EHR and medical-claims data for clinical studies supporting regulatory decisions.

For non-interventional studies, FDA's March 2024 document is a draft guidance and therefore should not be treated as a final implementation standard. It nevertheless provides useful discussion of design considerations such as confounding, data sources, study conduct, and reporting.

For safety-focused non-interventional studies, ICH M14 reached final FDA guidance status in March 2026. It addresses planning, designing, analyzing, and reporting non-interventional studies using RWD for safety assessment and states that many principles also apply to effectiveness studies.

EMA Perspective

EMA's RWE framework supports use of RWD in regulatory decision-making and includes guidance and reflection material on non-interventional studies and registry-based studies.

EMA's DARWIN EU network provides access to healthcare databases and uses standardized analytical approaches, including the OMOP Common Data Model, to generate evidence for regulatory questions. EMA reports that DARWIN EU studies can inform regulatory review, labeling changes, feasibility assessments, and pharmacovigilance.

As of the 2025 reporting period, EMA reported 59 regulator-led RWD studies conducted between February 2024 and February 2025 and described DARWIN EU as the main evidence-generation pathway for EMA-led studies.

FDA and EMA: A Practical Comparison

Registry-Based Evidence

Registries can be valuable when they capture disease-specific variables not available in generic claims or EHR data. FDA has a final guidance on assessing registries to support regulatory decision-making for drugs and biological products.

For registry-based evidence, examine enrollment mechanisms, site coverage, data completeness, endpoint definitions, follow-up, and changes in registry operations over time.

Study Protocol

A regulator-ready RWE study should have a protocol that makes the design reproducible before outcome analysis begins.

Statistical Analysis Plan

The SAP should translate the protocol into executable analysis rules.

Balance Diagnostics

After weighting or matching, report whether measured baseline characteristics are balanced.

For binary variables, analogous standardized differences can be used. The exact diagnostic threshold should not be treated as a universal pass/fail rule; the entire balance profile and clinical importance of covariates should be considered.

Illustrative Balance Plot

Confounding Sensitivity Analysis

Observed-data adjustment can only address measured or adequately proxied confounders. Sensitivity analyses can quantify how strong an unmeasured confounder would need to be to materially change the conclusion.

Examples include:

• Negative-control outcomes or exposures
• Quantitative bias analysis
• Alternative propensity-score specifications
• Restriction to better-measured subpopulations
• High-dimensional confounding approaches
• Alternative outcome definitions
• Alternative exposure grace periods

Competing Risks

When the event of interest can be precluded by another event, the estimand must clarify whether the analysis concerns a cause-specific hazard, cumulative incidence, or another quantity.

For example, in an oncology study, death can preclude a subsequent hospitalization endpoint. The choice of competing-risk method should follow the clinical question rather than being selected solely because a software procedure is available.

Survival Analysis

For time-to-event outcomes, a Cox model may be used:

But the hazard ratio is not automatically the most decision-relevant summary. Depending on the regulatory question, absolute risks, risk differences, restricted mean survival time, or cumulative incidence may be more interpretable.

Recurrent Events

Routine-care outcomes can occur repeatedly, such as hospitalizations or infections. The analysis should specify whether the estimand concerns time to first event, total event burden, event rate, or another measure.

Negative Controls

A negative-control outcome is an outcome that should not plausibly be caused by the exposure but may share the same confounding structure. Unexpected associations can reveal residual bias or data-quality problems.

Data Quality Checks

Before inferential analysis, run systematic data-quality checks.

R Implementation: A Reproducible Skeleton

library(dplyr)
library(survival)
library(survey)

analysis <- rwd %>%
  filter(eligible == 1) %>%
  mutate(
    index_date = treatment_start,
    followup_days = as.numeric(outcome_date - index_date),
    event = if_else(!is.na(outcome_date), 1, 0)
  )

ps_model <- glm(
  treatment ~ age + sex + disease_severity +
    comorbidity_index + prior_therapy,
  data = analysis,
  family = binomial()
)

analysis <- analysis %>%
  mutate(
    ps = predict(ps_model, type = "response"),
    wt = if_else(
      treatment == 1,
      1 / ps,
      1 / (1 - ps)
    )
  )

design <- svydesign(
  ids = ~1,
  weights = ~wt,
  data = analysis
)

fit <- svycoxph(
  Surv(followup_days, event) ~ treatment,
  design = design
)

summary(fit)

Do Not Treat Code as the Analysis Plan

Executable code is essential for reproducibility, but code should implement a prespecified scientific analysis rather than silently define it.

A strong submission package can map:

Each arrow should be auditable.

Reproducibility and Version Control

RWE programs often combine multiple data transformations and software environments. Record:

• Source database version and extraction date
• Data refresh and update history
• Code version or commit identifier
• R and package versions
• Configuration files
• Lookup tables and coding dictionaries
• Analysis dataset creation date
• QC outputs and discrepancy resolution

Traceability Matrix

What Goes Into a Regulatory Submission?

The exact submission location depends on the product, regulatory pathway, and agency. Conceptually, the RWE package should allow a reviewer to understand the question, data source, study design, analysis, limitations, and evidentiary interpretation without reconstructing the study from fragments.

Patient-Flow Reporting

A transparent cohort flow is particularly important for RWE because the initial database population may be dramatically larger than the final analysis population.

Handling Protocol Deviations in RWE

Observational studies do not have protocol deviations in exactly the same sense as randomized trials, but they have design deviations and data limitations that can affect the estimand. Examples include exposure misclassification, unavailable covariates, delayed outcome capture, and departures from intended follow-up.

Sensitivity Analysis Should Attack the Assumptions

A weak sensitivity analysis merely changes a model option. A stronger sensitivity analysis asks: What assumption is most likely to threaten the conclusion, and what analysis would reveal the impact?

Common RWE Failure Modes

1. Starting with the database. A convenient source is not necessarily a fit-for-purpose source.
2. Vague treatment definition. “Received Drug A” must become an operational exposure algorithm.
3. Misaligned time zero. Eligibility, treatment assignment, and follow-up must be synchronized.
4. Overadjustment. Adjusting for post-treatment mediators can distort a causal effect.
5. Ignoring missing care. Patients may receive care outside the observed system.
6. Assuming large N means low bias. Precision cannot repair systematic bias.
7. Reporting only the primary model. A credible RWE package should interrogate major assumptions.
8. Changing the analysis after seeing the result without disclosure. Exploratory analyses are acceptable when clearly identified, but should not be presented as prespecified evidence.
9. Weak provenance. If the reviewer cannot determine where a variable came from, confidence in the analysis suffers.
10. Confusing association with causal evidence. A statistically precise association can still be badly confounded.

Example of a Regulatory-Grade Interpretation

Totality of Evidence

RWE rarely exists in isolation. Regulatory decisions may integrate randomized clinical trials, pharmacology, natural-history evidence, safety surveillance, mechanistic evidence, registries, observational studies, and other sources.

The most persuasive RWE package is therefore not necessarily the largest one. It is the package whose question, data, design, analysis, and limitations are coherent with the broader evidence base.

RWE Quality-Control Checklist

The Most Important Concept

The central principle of regulatory RWE is simple:

A very large real-world dataset does not automatically generate persuasive evidence. Conversely, a carefully designed study using a smaller but highly relevant dataset can be highly informative.

References

FDA. Considerations for the Use of Real-World Data and Real-World Evidence To Support Regulatory Decision-Making for Drug and Biological Products. Guidance for Industry, August 2023.
FDA. Real-World Data: Assessing Electronic Health Records and Medical Claims Data To Support Regulatory Decision-Making for Drug and Biological Products. Guidance for Industry, July 2024.
FDA. Real-World Evidence: Considerations Regarding Non-Interventional Studies for Drug and Biological Products. Draft Guidance, March 2024.
FDA. M14 General Principles on Planning, Designing, Analyzing, and Reporting of Non-interventional Studies That Utilize Real-World Data for Safety Assessment of Medicines. Final Guidance, March 2026.
EMA. Real-world evidence guidelines. European Medicines Agency.
EMA. Reflection paper on use of real-world data in non-interventional studies to generate real-world evidence for regulatory purposes. EMA/99865/2025, 2025.
EMA. Data Analysis and Real World Interrogation Network (DARWIN EU). European Medicines Agency.
EMA. Real-world evidence framework to support EU regulatory decision-making: 3rd report. 2025.