Sensitivity Analysis Across Model Specifications

Why Compare Model Specifications?

The parametric g-formula turns models for longitudinal covariate transitions and outcomes into regime-specific population summaries. Those summaries depend not only on the intervention regime and target population, but also on modeling choices: a logistic rather than probit link, a linear rather than transformed covariate effect, an interaction term, or a different distribution for a continuous transition. Sensitivity analysis asks whether the substantive conclusion persists across plausible choices.

A comparison is interpretable only when the quantity being compared is held fixed. Keep the target population, treatment regimes, follow-up horizon, outcome definition, and standardization procedure the same. Vary a specified modeling choice and report the resulting change. A model that changes both the estimand and the specification does not isolate model dependence.

For a finite set of possible histories, the calculation can be written as a sum. Write \(L_t\) for time-varying covariates, \(Y\) for the outcome, and \(p_0(l_0)\) for the target baseline distribution. A candidate specification \(m\) supplies conditional transition distributions \(p_{m,t}(l_t\mid \bar l_{t-1},\bar a_{t-1})\) and an outcome regression \(\mu_m(\bar l_T,\bar a_T)=E_m[Y\mid \bar L_T=\bar l_T,\bar A_T=\bar a_T]\). Then

For continuous covariates, the corresponding sum is an integral. In practice, it may be evaluated by simulation. The uncertainty arising from finite data and the variation arising from simulation are distinct from the sensitivity comparison itself; simulation size and Monte Carlo error are treated in their own right.

Conditions for an Interpretable Comparison

The causal interpretation of each regime-specific quantity requires the usual identification conditions for the parametric g-formula: consistency, sequential exchangeability given the measured history, and positivity for the regimes and histories relevant to the target population. In addition, the fitted conditional models must represent the relevant outcome and transition distributions well enough for the g-formula calculation. Sensitivity analysis does not establish these conditions. In particular, agreement among several models cannot rule out unmeasured confounding or shared misspecification.

The comparison also needs a fixed target baseline distribution. One can average each model over the same empirical baseline covariates or over a specified population distribution, but switching target populations between specifications confounds model sensitivity with population composition. Fit each candidate to the same analytic sample when that is appropriate, use the same regime definitions, and state which features are changed. A staged comparison—first links, then functional forms, then interactions, then transition distributions—makes the source of a change easier to identify.

Functional Form and Interactions

A link function determines how the linear predictor maps to a conditional mean. Transformations determine how a covariate enters that predictor: for example, \(L\), \(\log(L+c)\), or a spline basis. Interactions allow the association of one variable with the outcome to vary with another. These are distinct choices, although more than one may be changed in a candidate model. For a clean sensitivity analysis, vary them separately where feasible and then examine a small number of defensible combined specifications.

Transition Distributions and Nonlinear Averaging

Sensitivity to a transition distribution can arise even when two candidate distributions agree on their mean and variance. The reason is that the g-formula averages the outcome regression over the full distribution of the covariate, not just its first two moments. Under a nonlinear outcome model, changing the distribution's shape can change the average predicted outcome.

Two Results for Interpreting Sensitivity

Proof. With finite support, the g-formula is a finite sum of products of baseline probabilities, transition probabilities, and conditional outcome means. Under independent and identically distributed sampling, each empirical baseline probability converges in probability to its population probability by the law of large numbers. By assumption, each estimated model parameter converges in probability to its population value; continuity then implies convergence in probability of every fitted conditional probability or mean at every support point. Products and finite sums preserve convergence in probability. Consequently, the plug-in sum converges to the g-formula evaluated at the true conditional distributions. Identification equates that quantity with the causal regime-specific mean. \(\square\)

This result explains what a correctly specified parametric g-formula estimates; it does not say that different incorrect models will agree, nor does it certify that a candidate is correctly specified. For continuous histories, analogous arguments require additional conditions, such as suitable domination and valid convergence of the relevant integrals. Those conditions should not be assumed merely because a simulation produces a number.

Proof. By standardization, \(\psi_m-\psi_{m'}=\sum_{k=1}^K w_k\{r_m(x_k)-r_{m'}(x_k)\}\). The triangle inequality gives \(\left|\psi_m-\psi_{m'}\right|\leq\sum_{k=1}^K w_k|r_m(x_k)-r_{m'}(x_k)|\). Each absolute difference is at most its maximum over \(k\), and the weights sum to one. Hence the last expression is at most \(\max_k|r_m(x_k)-r_{m'}(x_k)|\). \(\square\)

This bound is useful as a localization diagnostic: a large overall change must be supported by at least one target stratum with a substantial conditional change. A small overall change, however, may conceal offsetting changes across strata, so inspect the stratum-specific predictions as well as the marginal estimate.

Diagnostics, Comparisons, and Limitations

A disciplined comparison reports more than a collection of point estimates. Keep a record of which component changes in each specification and inspect model predictions in the histories that contribute to each regime. Useful warning signs include implausible predicted risks, sparse treatment-history combinations, extreme or unstable transition predictions, large changes concentrated in poorly supported strata, and sensitivity to a transformation or interaction that is not supported by the observed data. Diagnostics for parametric longitudinal models help assess individual model fit; sensitivity analysis complements them by showing how conclusions respond to alternative plausible specifications.

Compare the candidates on the same effect scale. A risk difference and a risk ratio can differ because of their scales even when based on the same risks; that is a change in contrast, not necessarily a change in model. Likewise, compare standardized regime risks before attributing a difference in a contrast to a particular model component. If every model predicts almost the same standardized risks but a reported contrast changes, check the contrast calculation.

Sensitivity analysis is not a formal model-selection rule. Choosing whichever specification gives the desired result is not defensible, and averaging estimates across models does not automatically account for model uncertainty. A useful report gives the primary specification and its rationale, a prespecified set of plausible alternatives, their standardized estimates, and a short explanation of where and why they differ. Sampling uncertainty is still relevant; a spread across specifications is not a confidence interval. Bootstrap inference for g-computation addresses sampling variability, not the choice of model specification.

Common Pitfall

Wrong reasoning: “The estimates from two models are close, so the causal conclusion is robust.” This overlooks the possibility that both models share the same omitted nonlinearity, unsupported extrapolation, or misspecified transition mechanism. Conversely, a difference between models does not by itself show which one is correct.

Correction: Interpret closeness as limited sensitivity to the particular choices examined, not as proof of validity. Check the relevant assumptions and diagnostics, add alternatives that probe scientifically plausible departures, and locate any differences in the histories that drive the g-formula. State explicitly which dimensions of specification were varied and which were not.