Reviewing a Model-Based Bridging Analysis

Why Review the Analysis, Not Just the Estimate?

A model-based bridging analysis may produce a precise estimate for a new region or population. Precision alone does not show that the estimate answers the intended question. A reviewer needs to establish what population and treatment comparison the result concerns, how the data were generated, whether the model can reasonably connect the source and target settings, and how uncertainty affects the decision.

The goal of a review is not to recreate every modeling step or to replace scientific judgment with a checklist score. It is to trace the chain from the decision question to the data, assumptions, model predictions, and conclusion. A gap in that chain should be stated explicitly; it should not be hidden by a favorable point estimate.

A Reviewer’s Checklist

Review the analysis in a fixed order. This makes it easier to distinguish a problem with the question from a problem with the data or model.

What Justifies Transporting a Model Result?

A useful way to make the assumptions visible is to write the target-population treatment effect as an average of conditional effects. Let \(X\) denote measured characteristics that may modify treatment effect, let \(\Delta(x)\) be the treatment effect conditional on \(X=x\), and let \(f_T(x)\) describe the distribution of those characteristics in the target population. For a continuous \(X\), the standardized target effect is

For discrete strata, the integral becomes a weighted sum. This representation follows by averaging the conditional effect over the target distribution: each stratum contributes its effect multiplied by the proportion of the target population in that stratum. It also shows why a reviewer must check both the conditional effects and the target distribution used to average them.

This calculation supports a target-population interpretation only under assumptions appropriate to the analysis. In particular, the relevant effect modifiers must be measured well enough to represent the differences between populations; the conditional treatment effects estimated from source data must be applicable to the target at given values of those modifiers; and the target must have adequate support in the source data for the values being used. Treatment, outcome, and follow-up definitions must also be sufficiently comparable. These assumptions are scientific claims, not consequences of the formula.

A reviewer should ask how each claim is supported and what happens if it is weakened. A model may adjust for measured factors while remaining vulnerable to unmeasured effect modifiers, poor measurement, or a target subgroup absent from the source data. Earlier material on sensitivity analysis for unmeasured effect modifiers and sensitivity to model form provides tools for examining specific vulnerabilities; the review still needs to say which vulnerabilities matter to this analysis.

Data Provenance, Validation, and Overlap

Data provenance is the account of where the analysis data came from and how they became the analysis dataset. Reviewers should be able to trace records from the original studies or databases through eligibility rules, exclusions, variable construction, and any pooling or harmonization. Look for unexplained differences in endpoint definitions, assessment schedules, missing-data handling, or population selection. A target distribution taken from an administrative database, for instance, may not represent the population eligible for the proposed treatment.

Validation should address the prediction the model is being asked to make. Fit to source data is not, by itself, evidence of predictive performance in a different population. Relevant checks may include predicted-versus-observed outcomes in held-out or external data, calibration across important subgroups, and residual patterns by region or effect modifier. A good overall fit can mask poor fit in a small but decision-critical subgroup. The reviewer should note whether validation data were used in model selection or tuning, because that makes them less independent as a check.

Overlap asks whether the source data contain information about the covariate profiles represented in the target. One compact diagnostic for normalized nonnegative transport weights \(w_i\) is the effective sample size,

When weights are equal, \(n_{\mathrm{eff}}\) equals the number of observations. When weights are concentrated, it is smaller. It is a diagnostic of weight concentration, not a guarantee of validity or a literal count of independent, informative patients in every model.

Uncertainty and the Decision Being Made

A review should separate uncertainty in the estimated effect from uncertainty about whether the analysis assumptions are adequate. A confidence interval can summarize sampling uncertainty under a specified model and its assumptions; it does not automatically include uncertainty from unmeasured effect modifiers, poor source-target overlap, or an incorrect model form. Check what sources of uncertainty were propagated, which were assessed only in sensitivity analyses, and whether the reported range changes under plausible alternatives.

Decision relevance requires an explicit decision criterion. The criterion may be a minimum clinically meaningful effect, a safety constraint, or a threshold for selecting a dose or proceeding with development. The reviewer should check that it was defined in the right outcome scale and population, and distinguish a point estimate that crosses a threshold from evidence that remains persuasive across uncertainty.

Comparing Review Approaches and Recognizing Failure Modes

A structured checklist is useful because it keeps the review connected to the intended use. It is not a substitute for domain expertise or a mechanical pass/fail score. A single summary metric, such as overall model fit or effective sample size, is easier to report but can miss a target-definition error, a poorly measured effect modifier, or a subgroup with inadequate overlap. Conversely, a detailed model review that never returns to the treatment decision can catalogue diagnostics without answering whether the result is usable.

Other evidence strategies may be appropriate when the model cannot support the target prediction. New local data can directly address gaps in relevance or support, though a new study also needs a suitable design and adequate information. A simpler or differently structured analysis may be easier to interpret, but it does not remove the need to justify its assumptions. The reviewer’s task is to identify what each approach resolves and what remains uncertain, rather than to assume that one approach is always preferable.

Common warning signs include a target population defined only after seeing the results; data sources with unclear eligibility or endpoint definitions; important effect modifiers absent from the model; validation limited to the fitting dataset; concentrated weights or unsupported target profiles; and a conclusion based on a favorable point estimate while uncertainty crosses the decision threshold. Each warning sign should lead to a specific question, diagnostic, sensitivity analysis, or data need.