Assess Sensitivity to Unmeasured Confounding

Why Assess Unmeasured Confounding?

Regression adjustment and propensity-score methods can account for measured covariates, but they cannot directly adjust for a variable that was not measured. An adjusted association may therefore still reflect differences between exposure groups in an unrecorded factor related to both exposure and outcome. A sensitivity analysis asks a focused question: how strongly would such a factor have to be related to exposure and outcome to change the result?

This tutorial uses the E-value, a benchmark for associations reported as risk ratios. It turns an estimated association into a scale that can be compared with subject-matter knowledge about plausible unmeasured confounders. It does not identify whether confounding is present, or prove that an estimate is causal.

Set Up the Sensitivity Question

Suppose \(A\) denotes a binary exposure, \(Y\) a binary outcome, and \(C\) the measured covariates included in an analysis. Let \(RR_{obs}\) be the adjusted risk ratio comparing the exposed and unexposed groups. For example, \(RR_{obs}=1.8\) means that the estimated outcome risk in the exposed group is 1.8 times the estimated risk in the unexposed group, conditional on the adjustment used.

To describe a possible unmeasured confounder \(U\), consider two conditional associations: \(RR_{UA}\), the strength of association between \(U\) and exposure \(A\) given \(C\); and \(RR_{UY}\), the strength of association between \(U\) and outcome \(Y\) given \(A\) and \(C\). These are considered on the risk-ratio scale and oriented so values above 1 describe associations in the direction that could account for the observed exposure-outcome association. For a multilevel \(U\), the relevant strength is the largest association across its levels.

Definition: For an estimated risk ratio \(R\) above 1, its E-value is \(R+\sqrt{R(R-1)}\). It is the minimum equal strength, on the risk-ratio scale, that an unmeasured confounder would need to have in its associations with exposure and outcome, conditional on the measured covariates, to explain the point estimate fully. More generally, no confounder whose two relevant conditional associations are both weaker than the E-value can explain it under the sensitivity bound. For \(R<1\), calculate the E-value using \(1/R\). An estimate exactly at the null, \(R=1\), has E-value 1.

The E-value refers to the minimum of the stronger of the two confounder associations needed to reach the sensitivity threshold. It does not mean that every confounder must have exactly equal associations with exposure and outcome. Equal strength is the benchmark that yields the minimum possible maximum; unequal pairs can also reach the threshold if sufficiently strong.

Why the E-Value Has This Form

A standard risk-ratio sensitivity bound summarizes the potential confounding strength as \(B=RR_{UA}RR_{UY}/(RR_{UA}+RR_{UY}-1)\). Under the bound, the observed risk ratio divided by \(B\) is a lower bound on the association after allowing for the specified unmeasured confounding in the direction that weakens the observed association. If both confounder associations are no greater than \(r\), the largest value of this bound occurs when both are \(r\), giving \(B=r^2/(2r-1)\). To find the equal strength that could just account for an observed \(R>1\), set \(B=R\) and solve:

\frac{r^2}{2r-1}=R \quad\Longrightarrow\quad r^2-2Rr+R=0 \quad\Longrightarrow\quad r=R\pm\sqrt{R(R-1)}.

The larger root is the relevant strength threshold, so \(r=R+\sqrt{R(R-1)}\), the E-value. The smaller root is below 1 and does not represent the confounding-strength benchmark. This derivation explains why the E-value is a threshold, not an estimate of the actual strength of an unmeasured confounder.

Worked Example: An Elevated Risk Ratio and Its Confidence Limit

Suppose a hypothetical observational analysis estimates an adjusted risk ratio of 1.80 for an exposure and a health outcome, with a 95% confidence interval from 1.20 to 2.70. Calculate the E-value for the estimate and for the confidence limit nearest the null.

Point estimate. Since \(R=1.80>1\), \(E=1.80+\sqrt{1.80(1.80-1)}\). The product inside the square root is \(1.80(0.80)=1.44\), so \(E=1.80+1.20=3.00\).

Confidence limit nearest the null. The lower confidence limit, 1.20, is nearer to 1 than the upper limit. Its E-value is \(1.20+\sqrt{1.20(1.20-1)}=1.20+\sqrt{0.24}\approx1.69\).

On this scale, a single unmeasured confounder would need exposure and outcome associations of at least 3.00 each in the equal-strength benchmark to explain away the point estimate. To move the confidence interval to include the null, the corresponding benchmark is smaller: 1.69. These are sensitivity thresholds conditional on the measured covariates and the risk-ratio bound, not claims about what confounder strengths actually exist.

Protective Associations and the Null

For an estimated risk ratio below 1, invert the estimate before applying the formula. This places the comparison in the direction of an elevated risk ratio and measures distance from the null on the same multiplicative scale. When considering a confidence interval, use the limit closest to 1: the lower limit if the whole interval is above 1, and the upper limit if the whole interval is below 1.

Worked Example: A Risk Ratio Below One

In a hypothetical study, the adjusted risk ratio is 0.65, with a 95% confidence interval from 0.48 to 0.88. Calculate the E-values for the estimate and the confidence limit nearest 1.

For the point estimate, invert first: \(R=1/0.65\approx1.538\). Then \(E=1.538+\sqrt{1.538(1.538-1)}\approx1.538+\sqrt{0.828}\approx2.45\).

The limit nearest the null is 0.88. Its reciprocal is \(1/0.88\approx1.136\), so \(E=1.136+\sqrt{1.136(1.136-1)}\approx1.136+\sqrt{0.155}\approx1.53\).

The point-estimate benchmark is about 2.45, while the confidence-limit benchmark is about 1.53. The interpretation is in terms of the strength of associations needed to move the estimate or the interval toward the null, not the probability that the null hypothesis is true.

Compare Unequal Confounder Strengths

A useful sensitivity analysis need not assume that the two confounder associations are equal. One can examine plausible pairs of conditional associations using the risk-ratio bound. For \(RR_{UA}=a\) and \(RR_{UY}=y\), the bound is \(B=ay/(a+y-1)\). If the observed association is above 1, dividing it by \(B\) shows how far the bound could move it toward 1. A value at or below 1 means that this bound is large enough to encompass the null; a value above 1 means that this particular pair does not suffice to explain away the estimate under the bound.

Worked Example: Checking a Plausible Pair of Associations

Suppose a hypothetical analysis estimates \(RR_{obs}=1.50\). Investigators consider whether an unmeasured factor could be associated with exposure by a risk ratio of 2.0 and with outcome by a risk ratio of 3.0, conditional on measured covariates. Use the bound to assess this pair.

Here \(a=2.0\) and \(y=3.0\), so \(B=(2.0)(3.0)/(2.0+3.0-1)=6/4=1.50\). The resulting bound on the risk ratio after allowing for this confounding is \(RR_{obs}/B=1.50/1.50=1.00\).

Thus, this pair is strong enough to reach the null under the sensitivity bound. It does not show that the true risk ratio is 1.00, nor that such a confounder exists. It shows only that associations of these strengths could be sufficient to make the observed estimate compatible with the null under the stated bound.

For comparison, if both conditional associations were 1.5, then \(B=(1.5)(1.5)/(1.5+1.5-1)=2.25/2=1.125\), and \(RR_{obs}/B=1.50/1.125\approx1.33\). That pair would not be strong enough, by this bound, to explain away the point estimate. The E-value for 1.50 is \(1.50+\sqrt{1.50(0.50)}\approx2.37\), consistent with the fact that the equal pair 1.5 is below the threshold.

Assumptions and Limits of Interpretation

An E-value is useful only when its scale and target match the analysis. The formulas here concern risk ratios. They do not directly apply to a regression coefficient for a continuous outcome. Odds ratios and hazard ratios require additional considerations and should not automatically be substituted as though they were risk ratios. The examples also assume the estimated adjusted risk ratio is the quantity being assessed and that the measured-covariate adjustment has been carried out as intended.

The sensitivity calculation does not repair problems such as lack of covariate overlap, incorrect measurement, selection bias, or a misspecified outcome model. Nor does it test the identification assumptions. It isolates one question: how much residual confounding, represented by conditional exposure and outcome associations, could change the risk-ratio result? Plausibility must be judged using substantive knowledge about the setting, not the E-value alone.

For an interval, the E-value of the limit nearest 1 is often informative because it asks how strong confounding would need to be to make the interval include the null. If that limit is already at or across 1, its E-value is 1: the interval does not exclude the null before considering unmeasured confounding. This does not establish that there is no association; it describes the interval’s relation to the null.

Common Mistakes

Treating a large E-value as proof of no confounding. An E-value is a benchmark, not evidence that a confounder of that strength is absent. Compare it with plausible conditional associations in the substantive context.
Using the point estimate when asking about the interval. To assess how much confounding could move the interval to include the null, calculate the E-value for the confidence limit nearest 1.
Applying the risk-ratio formula to any regression coefficient. The formula is defined here for risk ratios. A coefficient on a different scale needs an appropriate sensitivity method.
Reading the threshold as a probability. An E-value of 2.37 does not mean a 2.37 probability of confounding. It describes a required association strength on the risk-ratio scale.
Claiming causation from the sensitivity result. A sensitivity benchmark neither establishes causal identification nor rules out other biases. It only evaluates how a specified form of unmeasured confounding could affect the association.

Key takeaway: For an adjusted risk ratio, an E-value summarizes how strong an unmeasured confounder’s conditional associations with exposure and outcome would need to be to explain away the estimate. Report the estimate’s E-value and, when relevant, the E-value for the confidence limit nearest the null, then assess those thresholds against plausible confounder strengths.

Check Your Understanding

Answer each question using the risk-ratio sensitivity framework.

An adjusted risk ratio is 2.0. Write the E-value formula for this estimate and calculate its value.
An adjusted risk ratio is 0.50. Which value should be used in the E-value formula, and why?
A confidence interval for a risk ratio is 1.10 to 2.20. Which confidence limit is used to assess how much confounding could make the interval include the null?
Explain what it means if a proposed pair of conditional confounder associations gives \(RR_{obs}/B=1.00\).
Why does a large E-value not establish that an adjusted observational association is causal?