Sensitivity Analysis for Residual Confounding

Why Sensitivity Analysis Is Needed

An exposure-response estimate can be statistically precise and still be vulnerable to residual confounding. In clinical development, exposure is often not randomly assigned: disease severity, adherence, organ function, concomitant treatment, or dose modification can affect both a patient’s exposure and outcome. Adjustment for measured covariates helps only to the extent that those covariates are adequately measured and modeled. A sensitivity analysis asks a narrower, useful question: how strong would remaining bias have to be to change the conclusion, and are assumptions of that strength plausible?

This analysis does not establish that an exposure effect is causal. A causal interpretation additionally requires a well-defined intervention or exposure contrast, adequate control of confounding, positivity, consistency, and appropriate handling of selection and missingness. Here the focus is on quantifying how conclusions depend on assumptions that cannot be verified from the observed data alone.

Definition: Residual confounding is bias in an exposure-response estimate that remains after adjustment for the covariates included in the analysis. It can arise from an omitted confounder, a poorly measured covariate, an incorrect functional form, or an inadequate adjustment set. Sensitivity analysis varies explicit assumptions about these sources and evaluates their effect on the estimate or its interpretation.

Omitted-Variable Bias After Adjustment

Consider a continuous outcome \(Y\), exposure \(E\), measured covariates \(L\), and a possible unmeasured confounder \(U\). For a transparent derivation, suppose their conditional mean relationship is linear and additive:

Y_i=\alpha+\beta E_i+\boldsymbol{\theta}^{T}L_i+\gamma U_i+\varepsilon_i.

Here \(\beta\) is the exposure coefficient conditional on \(L\) and \(U\), while \(\gamma\) is the conditional association between \(U\) and the outcome. Assume the model is correctly specified, the error has conditional mean zero given \(E,L,U\), and the relevant variances are nonzero. If \(U\) is omitted, the exposure coefficient can absorb some of its association with the outcome.

To state that bias precisely, residualize each variable on an intercept and the same covariates \(L\). Write \(\widetilde{E}\), \(\widetilde{Y}\), and \(\widetilde{U}\) for the resulting residuals. Residualization removes the linear component explained by \(L\); it does not leave the original outcome unchanged. Applying this operation to the outcome model gives:

\widetilde{Y}=\beta\widetilde{E}+\gamma\widetilde{U}+\widetilde{\varepsilon}.

The residualized error is orthogonal to the fitted components of \(L\). Also, \(\widetilde{E}\) is orthogonal to those fitted components by construction, so \(\operatorname{Cov}(\widetilde{E},Y)=\operatorname{Cov}(\widetilde{E},\widetilde{Y})\). In a population regression omitting \(U\), the slope is therefore:

\beta_{\mathrm{omit}} =\frac{\operatorname{Cov}(\widetilde{E},\widetilde{Y})} {\operatorname{Var}(\widetilde{E})} =\beta+\gamma\frac{\operatorname{Cov}(\widetilde{E},\widetilde{U})} {\operatorname{Var}(\widetilde{E})}.

This is the omitted-variable bias formula. It requires both an outcome association for \(U\) (\(\gamma\ne0\)) and residual association between \(E\) and \(U\) after adjustment for \(L\). If either is zero, this particular source of bias is zero under the stated model. The formula is a linear-model result, not a universal correction for nonlinear, time-varying, or selection processes.

Useful sensitivity parameterization: Let \(\sigma_{\widetilde{E}}\) and \(\sigma_{\widetilde{U}}\) be the residual standard deviations after adjustment for \(L\), and let \(r\) be the correlation between the residuals. Then the omitted-variable bias is \(\gamma r\sigma_{\widetilde{U}}/\sigma_{\widetilde{E}}\). This expresses the bias in terms of an assumed confounder-outcome effect, its residual association with exposure, and its residual variability.

Worked Example: Quantifying Bias From a Hypothetical Confounder

Suppose a hypothetical adjusted analysis estimates an exposure coefficient of \(0.80\) endpoint units per ng/mL. Investigators consider an unmeasured severity score \(U\), measured in score units. After adjustment for recorded covariates, assume for sensitivity analysis that \(\gamma=1.5\) endpoint units per score unit, \(\sigma_{\widetilde{U}}=0.50\) score units, \(\sigma_{\widetilde{E}}=2.0\) ng/mL, and \(r=0.40\). These are assumptions, not estimates from an observed study.

The implied bias is:

\text{Bias} =\gamma r\frac{\sigma_{\widetilde{U}}}{\sigma_{\widetilde{E}}} =1.5(0.40)\frac{0.50}{2.0} =0.15\ \text{endpoint units per ng/mL}.

If the confounder bias is positive in the direction of the observed association, the bias-adjusted coefficient is \(0.80-0.15=0.65\) endpoint units per ng/mL. Under these assumptions, the point estimate remains positive, but its magnitude is 0.15 units per ng/mL smaller. This calculation alone does not show whether the estimate is clinically meaningful or statistically distinguishable from zero; those judgments also require the uncertainty in the estimate and the clinical context.

Tipping Points and Plausibility

A sensitivity analysis is more informative when it identifies a tipping point: the assumptions at which a decision-relevant conclusion changes. For example, if an estimated beneficial association is positive, one may ask what combination of confounder strength and exposure-confounder association would reduce the coefficient to zero. From the formula above, the bias required to move an observed coefficient \(\widehat{\beta}_{\mathrm{omit}}\) to zero is that same coefficient, with the required direction.

Worked Example: Finding a Tipping Point

Suppose a hypothetical estimated coefficient is \(0.60\) endpoint units per ng/mL, and the interpretation would change if the coefficient were zero or negative. Assume a plausible residual exposure SD of \(1.5\) ng/mL and residual confounder SD of \(0.50\) score units. Under the linear model, the confounder effect and residual correlation needed to explain the whole estimate satisfy:

0.60=\gamma r\frac{0.50}{1.5}, \qquad\text{so}\qquad \gamma r=1.80.

For example, if one posited \(r=0.50\), the confounder would need an outcome effect \(\gamma=1.80/0.50=3.60\) endpoint units per score unit. Alternatively, if \(\gamma=2.0\), the required correlation would be \(r=1.80/2.0=0.90\). The first scenario may or may not be plausible depending on the score scale and clinical knowledge; the second requires a very strong residual correlation. The calculation frames the discussion but does not determine plausibility by itself.

For uncertainty, compare the sensitivity-adjusted estimate with its interval, not only with zero. If the original interval is recalculated under each sensitivity scenario, it should reflect uncertainty in the sampling estimate and, where possible, uncertainty in the sensitivity parameters. Treating assumed \(\gamma\), \(r\), and residual SDs as known produces a conditional calculation, not a complete uncertainty analysis. A grid of plausible values is often clearer than a single favored assumption.

Alternative Covariate Adjustment Sets

A practical companion to unmeasured-confounder calculations is to repeat the analysis using scientifically defensible adjustment sets. Such comparisons can reveal dependence on measured covariate choices, but they are not a direct test for unmeasured confounding. An adjusted coefficient can change because a covariate controls confounding, because it is measured with error, or because it is a mediator or collider whose adjustment changes the target contrast or introduces bias.

Covariates should therefore be chosen using the timing and causal role of the variables, not by selecting whichever model gives a preferred result. A pre-exposure common cause of exposure and outcome is a confounder candidate. A variable affected by exposure may be a mediator; adjusting for it can remove part of the total effect. Conditioning on a common effect of exposure and outcome causes can induce an association that was not present before adjustment.

Worked Example: Interpreting Changes Across Adjustment Sets

In a hypothetical analysis, the exposure coefficient is \(0.72\) endpoint units per ng/mL with baseline disease severity and renal function included. Adding a pre-exposure adherence measure changes it to \(0.58\). Adding a post-exposure biomarker changes it further to \(0.31\). The first change is a reduction of \(0.72-0.58=0.14\), about \(19.4\%\) of the first estimate. The second is a further reduction of \(0.27\).

These changes show that the coefficient is sensitive to the adjustment set. If adherence is a common cause measured before exposure, its inclusion could plausibly reduce confounding. If the biomarker is affected by exposure, the coefficient from the final model may target a different, more conditional association rather than a better-adjusted total effect. It would be incorrect to conclude from the numerical changes alone that the final estimate is least biased. The covariate timing and causal role must support the adjustment.

Exposure Definitions and Missing Outcomes

Changing an exposure definition is another sensitivity analysis, but it does not always answer the same scientific question. Average exposure, trough concentration, peak concentration, and cumulative exposure represent different aspects of pharmacology. They can have different measurement error, time windows, and units. If their coefficients are reported on different scales, compare clinically meaningful contrasts or standardized predictions rather than raw coefficient magnitudes. A change in result may reflect a changed estimand, not just robustness or bias.

Missing-outcome assumptions can also alter an exposure-response conclusion. An analysis relying on a missing-at-random assumption uses observed information to account for missingness; that assumption cannot be confirmed from observed data alone. A pattern-mixture sensitivity analysis can instead posit how the unobserved outcomes differ from predictions under the primary missing-data model. The shift parameter must be stated in outcome units and can vary by exposure group or other relevant strata.

Worked Example: A Pattern-Mixture Shift for Missing Outcomes

Consider hypothetical low- and high-exposure groups. Among participants with observed outcomes, the mean is 10 units in the low group and 14 units in the high group, a difference of \(14-10=4\) units. Suppose 10% of low-group outcomes and 30% of high-group outcomes are missing. For illustration, assume observed means represent the predicted means for each group under the primary missing-data model.

First suppose missing outcomes in both groups average 1 unit below their respective primary-model predictions. The implied group means are:

\bar{Y}_{\mathrm{low}}=0.90(10)+0.10(9)=9.90, \qquad \bar{Y}_{\mathrm{high}}=0.70(14)+0.30(13)=13.70.

The implied difference is \(13.70-9.90=3.80\) units. Now suppose the low-group shift remains \(-1\) unit, but the high-group missing outcomes average 3 units below their predictions. Then the high-group mean is \(0.70(14)+0.30(11)=13.10\), and the difference is \(13.10-9.90=3.20\) units. The result depends on the assumed outcomes among participants with missing data; neither calculation identifies those outcomes from the observed data.

Diagnostics, Alternatives, and Limitations

Useful diagnostics include checking whether conclusions change across prespecified, causally defensible adjustment sets; examining exposure overlap within important covariate strata; assessing whether exposure and covariate measurements precede the outcome; and inspecting missingness rates and patterns by exposure and prognostic factors. Poor overlap means some adjusted contrasts rely heavily on extrapolation. Large changes after small modeling choices, implausible coefficient signs, or strong sensitivity to a few influential observations should prompt investigation of model form and data quality as well as confounding.

The main approaches answer related but distinct questions:

Approach	What it examines	Main limitation
Alternative adjustment sets	Dependence on measured covariate choices	Does not reveal the true causal role of a variable or rule out unmeasured confounding
Omitted-variable bias parameters	How assumed confounder strength and exposure association shift a coefficient	Results depend on unverified parameters and model form
Alternative exposure definitions	Dependence on the pharmacologic exposure measure and time window	May change the scientific contrast, not merely its measurement
Missing-data sensitivity scenarios	Dependence on assumptions about unobserved outcomes	Cannot recover missing values without assumptions

No single approach is universally preferable. A quantitative bias analysis makes assumptions visible; adjustment-set comparisons use observed covariate information; and missing-data scenarios address a different source of uncertainty. These analyses do not replace a prespecified primary model, justify post hoc selection of favorable results, or guarantee that an estimate is unbiased. Their value is in showing which assumptions matter and whether conclusions remain stable over a clinically defensible range.

Common pitfall: “The estimate changed after adding covariates, so the new estimate must be less confounded.” This does not follow. A newly added variable may be a mediator, collider, poorly measured proxy, or part of a misspecified model. Define the target contrast and assess the variable’s timing and causal role before interpreting the change; then report the sensitivity of the conclusion, not just the preferred number.

A Practical Sensitivity-Analysis Workflow

State the target contrast.
Name the exposure measure, outcome, time window, covariate adjustment, and coefficient or prediction that supports the conclusion.

Identify plausible failure points.
Consider omitted common causes, covariate measurement and functional form, exposure definition, overlap, and missing-outcome assumptions.

Specify scenarios before interpreting results.
Use scientifically meaningful ranges for sensitivity parameters or missing-outcome shifts, and justify their scales.

Recalculate and compare.
Report the resulting estimate, uncertainty where available, units, and whether the decision-relevant conclusion changes.

Explain what remains unidentified.
Distinguish robustness across chosen scenarios from evidence that unmeasured bias is absent.

Check Your Understanding

Answer each question, then compare with the solutions below.

Under the omitted-variable bias formula, which two conditions are required for nonzero bias from \(U\), after adjustment for \(L\)?
Suppose \(\gamma=2\) outcome units per score unit, \(r=0.25\), \(\sigma_{\widetilde{U}}=0.40\) score units, and \(\sigma_{\widetilde{E}}=2\) ng/mL. What is the implied bias, in outcome units per ng/mL?
Why does adding a post-exposure biomarker not necessarily improve control of confounding for a total exposure effect?
In the missing-outcome example, what is the high-minus-low difference if the high-group shift is \(-2\) units and the low-group shift is \(-1\) unit?
Does stability across several adjustment sets prove absence of residual confounding? Explain.

Solutions. (1) The confounder must affect the outcome conditional on the modeled variables (\(\gamma\ne0\)), and it must remain associated with exposure after adjustment for \(L\) (\(\operatorname{Cov}(\widetilde{E},\widetilde{U})\ne0\)). (2) The bias is \(2(0.25)(0.40)/2=0.10\) outcome units per ng/mL. (3) The biomarker may be affected by exposure and lie on the pathway to the outcome, so conditioning on it can remove part of the total effect or change the target contrast. (4) The low-group mean is \(0.90(10)+0.10(9)=9.90\). The high-group mean is \(0.70(14)+0.30(12)=13.40\). Their difference is \(13.40-9.90=3.50\) units. (5) No. Stability shows robustness to those particular measured-covariate choices, but an unmeasured confounder could still bias all of the analyses similarly.