Sensitivity Analysis for Unmeasured Temporal Confounding

Why Sensitivity Analysis Is Needed

In a platform trial, an arm may be compared with controls enrolled in earlier calendar periods when concurrent controls are unavailable or limited. Adjustment for recorded covariates and calendar time can address measured differences, but it cannot establish that all relevant changes in prognosis or care were measured. For example, supportive care may improve over time, or the mix of patients’ unrecorded risk factors may change. A sensitivity analysis asks how much such an unmeasured temporal difference would have to matter to change the arm’s estimated effect.

The aim is not to estimate an unknown temporal bias from the same data without additional information. Rather, the analyst specifies or explores plausible values for that bias and reports the resulting effects. This makes the dependence of the conclusion on an untestable assumption explicit.

Define the Effect and the Bias Parameter

Use an additive outcome scale for the definitions and examples below. Suppose the outcome is measured in points, with higher values indicating a better outcome. Let the target population be patients eligible for the arm during its current enrollment period. The treatment effect of interest is the mean difference between their outcome under treatment and the outcome those same patients would have had without treatment.

This sign convention is important. If current untreated outcomes would be better, comparing treated current patients with historical controls can make the treatment look more beneficial than it is. The correction therefore subtracts a positive \(\gamma\). If current untreated outcomes would instead be worse, \(\gamma\) is negative and the corrected effect is larger than the observed contrast.

The relationship is a sensitivity model, not a result that holds automatically. It assumes the target population and outcome are well defined; the treatment and control outcomes are measured comparably; measured adjustment adequately aligns the observed covariates; and, apart from the specified residual difference in untreated outcomes, the adjusted historical comparison identifies the target contrast. It also assumes the additive bias parameter is meaningful for the chosen outcome and target population. If treatment effects or temporal changes vary substantially across patient groups, a single \(\gamma\) may not represent the bias adequately.

Deriving the Adjustment and Tipping Points

The key justification follows by adding and subtracting the mean untreated outcome in the current target population. On the stated scale, the current treated-versus-historical-control contrast can be decomposed into the current treatment effect plus the difference between current and historical untreated outcomes:

Thus, if the adjusted observed contrast is \(\hat{\tau}_{\mathrm{obs}}\), subtracting the assumed \(\gamma\) gives the sensitivity-adjusted estimate. This derivation relies on comparing quantities defined for the same target and outcome scale; it does not show that the data identify \(\gamma\).

A point-estimate tipping value is the value of \(\gamma\) at which the adjusted estimate reaches a specified boundary, often the null value of zero. If the observed estimate is positive, the value that brings the point estimate to zero is \(\gamma=\hat{\tau}_{\mathrm{obs}}\). If its standard error is \(SE\), and \(\gamma\) is treated as fixed, an approximate 95% confidence interval shifts by the same amount:

The confidence interval describes sampling uncertainty conditional on the specified bias value and the model. It does not include uncertainty about whether that value of \(\gamma\) is plausible. A confidence interval that includes zero indicates that the data are compatible with effects on both sides of zero under that sensitivity scenario; it does not assign a probability to the treatment effect being positive or negative.

Translate an Unmeasured Difference Into Outcome Units

A bias value can be assessed directly in outcome units, or linked to a hypothesized unmeasured factor. One useful simple calculation applies when an unmeasured binary factor \(U\) has an additive, common association with the outcome and no relevant interaction with the current-versus-historical contrast. If \(U=1\) changes the mean untreated outcome by \(\delta_U\) points, then the bias attributable to its prevalence imbalance is:

This calculation is a calibration device: it asks what bias would follow from stated assumptions about imbalance and outcome association. It does not demonstrate that the factor is present, that its prevalence difference is known, or that the resulting bias is the only unmeasured bias.

Evaluate a Range of Assumptions

A single assumed value can give a misleading impression of certainty. It is often more informative to show estimates over a grid of \(\gamma\) values chosen to represent substantively plausible changes in prognosis or care. The grid should include both directions when the direction is uncertain. Explain the clinical or operational reasoning behind its range, and state whether each value refers to an absolute outcome difference, a standardized difference, or another scale.

If the plausible range is \([L,U]\), the corresponding range of point estimates is \([\hat{\tau}_{\mathrm{obs}}-U,\hat{\tau}_{\mathrm{obs}}-L]\). This is a sensitivity range, not a confidence interval: it represents variation across assumed bias values. If one also reports sampling intervals, calculate them for each fixed scenario, or report a conservative envelope across the scenarios. Do not combine the most favorable estimate from one scenario with the most favorable interval bound from another.

Diagnostics, Alternatives, and Limitations

Sensitivity analysis is most useful when its assumptions are transparent and its scale is interpretable. Before presenting results, check that:

• The outcome scale and direction are stated, and the sign of \(\gamma\) matches the definition of the current-minus-historical untreated difference.
• The effect estimate and \(\gamma\) refer to the same target population, covariate adjustment, and outcome summary.
• The assumed values have a substantive rationale, rather than being chosen only because they preserve a desired conclusion.
• There is adequate measured-covariate overlap for the adjusted historical comparison to be meaningful. Sensitivity analysis does not repair a lack of overlap.
• Outcome definitions, follow-up, and treatment versions are sufficiently comparable for the chosen contrast. A single additive parameter may be inadequate if these vary materially across periods.

A common mistake is to say, “The analysis adjusted for calendar time, so temporal confounding has been removed.” Adjustment can account for measured time patterns under the fitted model, but it cannot guarantee control of unmeasured changes in prognosis or care. The correction is to report what residual difference would alter the result and assess whether that difference is plausible.

Another mistake is to subtract a bias value in the wrong direction. With the convention used here, a positive \(\gamma\) means current untreated outcomes are better than historical untreated outcomes; it inflates a positive observed treatment contrast, so it is subtracted. Write the definition in words before calculating, especially when the outcome has the opposite direction (for example, where lower values are better).

A concurrent randomized comparison, when available and suitable for the target question, is generally less dependent on assumptions about temporal exchangeability than a historical-control comparison. Regression adjustment and standardization address measured differences; overlap restriction can avoid extrapolation into unsupported covariate regions. These approaches can improve the analysis, but none alone quantifies the remaining bias from unmeasured temporal differences. Sensitivity analysis complements rather than replaces them.

The additive model is not universal. For a binary outcome, a ratio or odds scale may be more appropriate, but the bias parameter and correction must then be defined on that scale; the additive formula should not be transferred without justification. Likewise, if the unmeasured temporal change affects treated and untreated outcomes differently, or if effects vary across groups, one scalar correction can conceal important structure. Report these limitations, and avoid presenting a sensitivity-adjusted estimate as if it were newly identified from the observed data.