Sensitivity to Model Form in Population Transport

Why Model Form Matters in Population Transport

A transport analysis uses information from a source population to predict an outcome or treatment effect in a target population. Even when the source data are well described by a fitted model, the transported prediction may depend strongly on how the model behaves at target covariate values that are rare or absent in the source data. A smooth curve, a straight line, and a model with an interaction can agree closely where observations are concentrated and diverge elsewhere.

Sensitivity to model form is therefore not simply a search for the best-fitting equation. It asks whether the substantive conclusion is stable across plausible representations of the response, treatment effects, and variability. The earlier material on sensitivity analysis for unmeasured effect modifiers addresses uncertainty about omitted modifiers; here the focus is uncertainty about the form of the model used to represent measured covariates and outcomes.

Definition: Let $S=1$ denote membership in the source population and $S=0$ membership in the target population. Let $A$ be treatment, $X$ the measured covariates used for transport, and $Y(a)$ the potential outcome under treatment $a$. The target average treatment effect is $$ \tau_T=E_T[Y(1)-Y(0)]. $$ A model-based transported prediction estimates conditional mean outcomes $m_a(x)=E[Y\mid A=a,X=x,S=1]$, forms the conditional contrast $\tau(x)=m_1(x)-m_0(x)$, and averages it over the target covariate distribution. This requires, among other conditions, that the conditional outcome contrast is transportable given $X$, the target covariate distribution is adequately known, and the model gives a credible representation of the conditional means over the target values.

Standardization and the Role of Assumptions

For discrete covariates, the target effect is estimated by weighting conditional effects by the proportions of target individuals in each covariate pattern. For continuous covariates, the corresponding operation is integration. In either case, model form determines the conditional predictions being averaged; the target distribution determines how much each prediction contributes.

\tau_T=\int \{m_1(x)-m_0(x)\}\,dF_T(x) \qquad\text{or, for discrete }X,\qquad \tau_T=\sum_x \tau(x)P_T(X=x).

The formula follows from iterated expectation: $E_T[Y(1)-Y(0)]$ equals the target-population average of the conditional mean contrast given $X$. Replacing the unknown conditional means with fitted source-population means gives a plug-in prediction. That replacement is justified only if the source conditional relationship is applicable to the target after conditioning on the included covariates. In causal applications, consistency, treatment exchangeability within the source data, and treatment positivity are also needed to identify the source conditional treatment contrast. Transport additionally requires conditional transportability: given $X$, the relevant conditional outcome relationship is shared by source and target. These are scientific assumptions, not consequences of a good fit to source data.

A separate support condition matters. If target covariate values lie outside the range or combinations represented in the source data, the model must extrapolate. No amount of source-data fit alone can establish that the extrapolated shape is correct. This is why alternative forms should be evaluated at the target values, rather than compared only by their fit within the source sample.

Worked Example: Same Source Fit, Different Extrapolation

Suppose hypothetical source data contain covariate values $x=0$ and $x=1$, with estimated treatment contrasts of 2 and 2.5 response units, respectively. The target population has equal proportions at $x=1$ and $x=2$. Thus $x=2$ is outside the observed source support. Compare two plausible conditional-effect forms:

\tau_L(x)=2+0.5x,\qquad \tau_Q(x)=2+0.5x+0.4x(x-1).

At $x=0$, both forms predict 2 response units. At $x=1$, both predict $2+0.5=2.5$ response units. They therefore reproduce the same two source contrasts, but their target predictions differ.

For the linear form, the target stratum predictions are $\tau_L(1)=2.5$ and $\tau_L(2)=2+1=3$. The standardized effect is $0.5(2.5)+0.5(3)=1.25+1.5=2.75$ response units.

For the curved form, $\tau_Q(1)=2.5$, while $\tau_Q(2)=2+1+0.4(2)(1)=3.8$ response units. Its standardized effect is $0.5(2.5)+0.5(3.8)=1.25+1.9=3.15$ response units.

The difference, $3.15-2.75=0.40$ response units, comes entirely from the assumed behavior at $x=2$, where there are no source observations. The source contrasts cannot distinguish these forms. The result should therefore be reported as model-form sensitivity, not as evidence that the curved prediction is better supported by the source data.

Interactions and the Target Covariate Mix

An interaction allows the treatment contrast to vary with a covariate. Omitting it imposes a restriction: in a simple additive outcome model, the treatment contrast is constant across that covariate. Whether that restriction is consequential depends partly on the target population’s covariate distribution. A modest interaction can have a noticeable effect on the target average if the target contains many individuals at covariate values where the treatment contrast differs.

Worked Example: A Treatment–Covariate Interaction

Consider a hypothetical binary covariate $x$, taking values 0 or 1. Suppose a fitted outcome model has conditional mean $m_a(x)=5+2x+a(1+0.6x)$, where $a=0$ denotes control and $a=1$ denotes treatment. The baseline mean changes with $x$, and the treatment contrast also changes with $x$.

Subtracting the control mean from the treatment mean gives $\tau(x)=m_1(x)-m_0(x)=1+0.6x$. Thus the contrast is 1 response unit at $x=0$ and 1.6 response units at $x=1$.

Suppose the target population is 25% $x=0$ and 75% $x=1$. Standardizing the interaction model gives $0.25(1)+0.75(1.6)=0.25+1.20=1.45$ response units.

If the interaction is omitted while retaining the treatment main effect, this model imposes a constant contrast of 1 response unit, so its standardized prediction is 1 response unit. The difference is $1.45-1=0.45$ response units. It is driven by the target’s covariate mix and by the assumption that treatment effect does not vary with $x$. These calculations do not establish that an interaction is present; they show what changes if the constant-effect restriction is relaxed.

The interaction question should be considered on the scale of the estimand. An interaction on an additive outcome scale need not imply an interaction on a ratio scale. Consequently, alternative models should preserve a clearly stated target quantity and outcome scale; otherwise, apparent model sensitivity can partly reflect a change in estimand rather than a change in functional form.

Variance Structure and Prediction Uncertainty

Variance assumptions concern uncertainty as well as the fitted mean. A homoskedastic model assumes a common residual variance across the observations to which that assumption applies. A heteroskedastic model allows residual variance to differ by covariate, treatment, or population strata. If the mean model is unchanged, changing only the variance model need not change the standardized point prediction. It can, however, change its standard error and interval. In some fitting procedures, variance assumptions also affect estimates of the mean-model parameters.

Worked Example: Unequal Stratum Variances and Transport Uncertainty

Suppose a hypothetical target prediction combines two estimated conditional treatment effects. The target weights are 0.8 and 0.2; the estimates are 1 and 3 response units, with standard errors 0.2 and 0.8 response units. Assume, for this calculation, that the two estimates are independent and approximately normally distributed.

The standardized point estimate is $0.8(1)+0.2(3)=0.8+0.6=1.4$ response units. With independent stratum estimates, the variance of this weighted sum is the sum of the squared weights multiplied by their respective variances:

\widehat{\operatorname{Var}}(\hat{\tau}_T) =0.8^2(0.2^2)+0.2^2(0.8^2) =0.0256+0.0256=0.0512.

The standard error is $\sqrt{0.0512}=0.2263$ response units. If a common standard error of 0.2 response units were incorrectly imposed on both strata, the variance would instead be $0.8^2(0.2^2)+0.2^2(0.2^2)=0.0272$, giving a standard error of $\sqrt{0.0272}=0.1649$ response units. The common-variance calculation understates uncertainty here because the low-weight stratum has much greater uncertainty than assumed.

Under the stated normal approximation, a 95% interval using the unequal variances is $1.4\pm1.96(0.2263)$, or approximately 0.957 to 1.843 response units. The interval describes uncertainty under the specified model and assumptions; it does not account automatically for uncertainty about which variance structure is appropriate. The point estimate remains 1.4 in both calculations, but the uncertainty differs.

How to Compare Models and Diagnose Fragile Predictions

A useful sensitivity analysis defines a small set of scientifically plausible alternatives before focusing on which one gives the preferred conclusion. Alternatives might include a linear term versus a curved term, a treatment–covariate interaction versus a constant treatment effect, or a common residual variance versus stratum-specific variances. Each model should be used to predict the same target estimand using the same target covariate distribution. The range of resulting predictions is informative about dependence on the assumptions varied; it is not, by itself, a probability distribution over the true effect.

Diagnostics should make the extrapolation visible. Plot source observations and fitted conditional predictions across the covariate range, mark the target range or target values, and show where source data are sparse or absent. Compare observed and fitted patterns within the source support, inspect whether predictions change sharply near boundaries, and check whether residual spread varies across relevant groups. A good in-support fit is necessary for a useful model, but cannot validate an unsupported target-region curve.

Specify the target quantity.
State the outcome scale, treatment contrast, target population, and covariates over which predictions will be averaged.

Map support.
Compare source and target covariate distributions, including combinations of covariates, and identify target values that require extrapolation.

Fit plausible alternatives.
Vary functional form, relevant interactions, and variance structure while keeping the target estimand fixed.

Standardize and compare.
Calculate each model’s target prediction and uncertainty, then locate which assumptions account for material differences.

Qualify the conclusion.
Report the observed-data evidence separately from assumptions needed outside source support, and describe whether conclusions are robust across the alternatives.

A single flexible model, such as a spline, can represent curvature but does not remove the need for judgment: its behavior at unsupported values still depends on its basis, boundary behavior, and regularization. A simpler prespecified model is easier to interpret but can impose restrictive shape assumptions. Model averaging can summarize predictions across candidate models if the model set and weighting rule are justified; it does not make unsupported extrapolation empirically identified. The appropriate comparison depends on the scientific question and data, not on choosing the most complex model or the one with the best in-sample fit alone.

Common Pitfall and Limits

Wrong reasoning: “Both models fit the source data, so their predictions in the target population are equally supported.”
Correction: Similar fit within observed support does not validate behavior outside that support. Report how predictions differ at target values and identify the shape assumptions causing the difference.
Wrong reasoning: “The model with the smaller standard error is more credible.”
Correction: Precision is conditional on the model and variance assumptions. A narrower interval does not resolve uncertainty about functional form, transportability, or support.

Sensitivity analysis has limits. If all candidate models share the same unsupported assumption, their agreement can be misleading. If the target population has little or no covariate overlap with the source, changing model form cannot create information that the source data do not contain. Nor does this analysis correct unmeasured effect modification or an incorrect target covariate distribution. Its value is to expose which model assumptions drive the prediction and to make the limits of the available evidence explicit.

Key takeaway: A transported prediction is an average of conditional predictions over the target population. When target covariates extend beyond source support, alternative model forms can agree on observed data yet imply materially different target effects. Compare models at the target values, examine variance assumptions, and distinguish data-supported conclusions from extrapolation assumptions.

Check Your Understanding

Answer each question, then check the solutions below.

A target population has 40% of individuals at $x=0$ and 60% at $x=1$. A model gives treatment contrasts of 2 and 4 units, respectively. What is the standardized target contrast?
Two models agree at every observed source covariate value but predict different effects at a target value beyond source support. What can be concluded from their source-data agreement?
With target weights 0.5 and 0.5, independent stratum estimates each have standard error 0.3 units. Calculate the standard error of their weighted mean.
Why might omitting a treatment–covariate interaction alter the transported effect even if the source and target populations have the same outcome scale?

Solutions. (1) The standardized contrast is $0.4(2)+0.6(4)=0.8+2.4=3.2$ units. (2) The models are indistinguishable by their fitted predictions at those observed values, but the agreement does not establish which extrapolation is correct; their target predictions are assumption-dependent. (3) The variance is $0.5^2(0.3^2)+0.5^2(0.3^2)=0.045$, so the standard error is $\sqrt{0.045}=0.2121$ units. (4) Omitting the interaction imposes a constant conditional treatment contrast. If the true or alternative modeled contrast varies with the covariate, averaging that restricted contrast over the target distribution can differ from averaging a covariate-dependent contrast.