Calibrating E-Values Against Known Confounders

Why Calibrate an E-Value?

An E-value is easier to interpret when it is compared with something concrete. If an estimate has a large E-value, one practical question is whether measured covariates in the study have associations with treatment and outcome of similar strength. This comparison can help describe the size of bias that would be needed relative to familiar features of the data.

The comparison is a calibration, not a test of whether unmeasured confounding exists. A measured covariate is not automatically a valid stand-in for an unmeasured one. Its observed associations can provide context, but they do not establish an upper bound on the strength of hidden bias, nor do they guarantee that adjustment has removed all confounding.

Definition: For a risk-ratio estimate $RR$ above 1, its point-estimate E-value is the minimum common value that the associations of an unmeasured confounder with treatment and outcome would each need to reach, on the risk-ratio scale and conditional on measured covariates, to explain the estimate fully under the standard E-value sensitivity model. It is $$ E=RR+\sqrt{RR(RR-1)}. $$ For a protective estimate below 1, first use its reciprocal $1/RR$ in this formula.

The equal-strength value is a convenient summary. In practice, a confounder need not have equal associations with treatment and outcome. To assess unequal strengths, use the joint bias factor. Let $RR_{UA}$ denote a risk-ratio-scale measure of the association between a confounder $U$ and treatment $A$, and let $RR_{UY}$ denote its association with outcome $Y$, conditional on treatment and the measured covariates. In each case, orient the comparison so that the association strength is at least 1.

Key result: Under the standard risk-ratio sensitivity model, the maximum confounding bias factor associated with strengths $RR_{UA}$ and $RR_{UY}$ is $$ BF=\frac{RR_{UA}\,RR_{UY}}{RR_{UA}+RR_{UY}-1}. $$ A bias factor of $BF$ can move a risk ratio away from the null by at most that multiplicative amount under the model. For an estimate above 1, the corresponding confounding-adjusted value is bounded below by the observed estimate divided by $BF$; for an estimate below 1, it is bounded above by the observed estimate multiplied by $BF$.

Why the E-Value Is an Equal-Strength Benchmark

The E-value is obtained by asking how large the two confounder associations would have to be if they were equal. Set $RR_{UA}=RR_{UY}=E$ in the bias-factor formula. The resulting factor is $E^2/(2E-1)$. To explain an estimate $RR>1$ fully, this factor must be at least $RR$. At the threshold, set it equal to $RR$ and solve:

\frac{E^2}{2E-1}=RR \quad\Longrightarrow\quad E^2-2RR\,E+RR=0 \quad\Longrightarrow\quad E=RR+\sqrt{RR(RR-1)}.

The other quadratic root, $RR-\sqrt{RR(RR-1)}$, is below 1 for $RR>1$, so it is not a confounder-strength value in the required range. This derivation also clarifies a limitation of reading the E-value alone: it summarizes a particular equal-strength combination. If one association is much stronger than the other, compare their joint bias factor with the observed estimate instead.

Benchmarking Measured Covariates

For each measured covariate considered as a benchmark, define a meaningful contrast and estimate both relevant associations on comparable scales. For example, the treatment association could compare treatment probabilities across two covariate levels, conditional on other measured covariates. The outcome association should compare outcome risks across those levels, conditional on treatment and the same relevant measured covariates. Their directions and reference groups should be arranged so that the reported strength is at least 1.

This is not a request to compare a treatment odds ratio with an outcome risk ratio and call them equivalent. The E-value and the bias factor use risk-ratio-scale associations. If a model produces a different effect measure, it must be translated appropriately or the comparison should be described as only qualitative. For a continuous or multi-level covariate, the contrast also matters: a one-unit contrast, a standard-deviation contrast, and a high-versus-low contrast can yield very different strengths.

Worked Example: Comparing Unequal Strengths With an E-Value

Suppose a hypothetical adjusted study reports a risk ratio of $2.4$. The point-estimate E-value is $$ E=2.4+\sqrt{2.4(2.4-1)} =2.4+\sqrt{3.36} =2.4+1.833 =4.233. $$ Thus, under the standard model, equal-strength associations of about $4.233$ each would be needed to explain the estimate fully.

Now suppose a measured covariate has a conditional treatment-association strength $RR_{UA}=5$ and an outcome-association strength $RR_{UY}=3$. Its joint bias factor is $$ BF=\frac{5(3)}{5+3-1}=\frac{15}{7}=2.143. $$ Since $2.143<2.4$, this pair of strengths does not reach the factor needed to explain the point estimate fully under the sensitivity model. The corresponding lower bound after allowing for this bias is $2.4/2.143=1.120$, still above 1. This is a model-based benchmark calculation, not a claim that the measured covariate is the strongest possible hidden confounder.

A useful interpretation keeps three statements separate: the estimate’s E-value describes a minimum equal-strength threshold; a measured covariate’s associations describe what was observed for that covariate under a specified contrast and adjustment set; and the bias factor translates a chosen pair of strengths into a potential amount of bias under the sensitivity model. None of these statements proves that the unmeasured confounder has the same structure or strength as the measured covariate.

Worked Example: A Measured Covariate Close to the Benchmark

Consider a hypothetical study with an estimated risk ratio of $3.0$. Its E-value is $$ E=3+\sqrt{3(3-1)} =3+\sqrt{6} =5.449. $$ A measured covariate has conditional association strengths $RR_{UA}=8$ and $RR_{UY}=3$. Neither needs to equal the E-value for the pair to be relevant. Their joint bias factor is $$ BF=\frac{8(3)}{8+3-1}=\frac{24}{10}=2.4. $$ Because $2.4<3.0$, that pair falls short of fully explaining the point estimate. Dividing gives $3.0/2.4=1.250$, the lower bound under this specified bias factor.

For comparison, if the treatment association were $10$ while the outcome association remained $3$, then $$ BF=\frac{10(3)}{10+3-1}=\frac{30}{12}=2.5, $$ which still falls short of $3.0$. This illustrates why looking at just the larger association can mislead: the joint formula uses both strengths. It also shows why the E-value $5.449$ should not be interpreted as saying that both associations must be at least $5.449$ in every possible scenario. Unequal pairs are assessed with the bias factor.

Protective Estimates and Direction

For a protective estimate, reverse the risk ratio before calculating the E-value so that the effect is expressed as a ratio above 1. When using the bias factor to calibrate a protective estimate, apply the factor toward the null: multiply the observed risk ratio by $BF$. This direction convention prevents a common mistake of dividing a protective estimate by a bias factor and making it appear more protective.

Worked Example: Calibrating a Protective Estimate

Suppose a hypothetical adjusted analysis reports $RR=0.60$. Reverse it to calculate the E-value: $$ \frac{1}{0.60}=1.667, \qquad E=1.667+\sqrt{1.667(1.667-1)} =1.667+\sqrt{1.111} =2.721. $$ The estimate therefore has an E-value of approximately $2.721$.

Suppose a measured covariate has $RR_{UA}=2$ and $RR_{UY}=3$. Its bias factor is $$ BF=\frac{2(3)}{2+3-1}=\frac{6}{4}=1.5. $$ For a protective estimate, move toward 1 by multiplying: $0.60(1.5)=0.900$. Under the specified strengths and model, the adjusted value would be no greater than $0.900$ in this bound calculation; this covariate’s benchmark alone does not move the estimate all the way to the null.

Diagnostics, Comparisons, and Limitations

Calibration is most informative when the measured-covariate associations are estimated and reported carefully. Before comparing them with an E-value, check that the exposure and outcome associations use the intended covariate contrast, adjustment set, and risk-ratio scale. Examine uncertainty as well as point estimates: noisy estimates of a covariate’s strength can make a seemingly precise benchmark misleading. Consider several plausible contrasts when the covariate is continuous or has multiple categories, and explain why those contrasts are substantively relevant.

Measured covariates can be confounded too. Their observed associations are not necessarily causal effects of the covariate on treatment or outcome, and they may be altered by measurement error, sparse data, model misspecification, or adjustment choices. A particularly strong measured covariate may be a useful empirical comparison, but an unmeasured factor could be stronger, interact with other factors, or be measured on a different scale. Conversely, a weak observed covariate does not imply that hidden confounding must also be weak.

The main alternatives answer different questions. Direct covariate adjustment or matching uses observed covariates to control confounding, subject to the assumptions and limitations of those methods. E-values quantify the strength of residual confounding needed to move a risk-ratio estimate to a specified target, and calibration provides context for that threshold. Rosenbaum bounds use a different sensitivity parameter for hidden treatment assignment bias in matched designs. Negative controls can probe certain forms of residual bias when their own identifying conditions hold. These approaches are complementary, not interchangeable, and an E-value calibration does not replace them.

The E-value also does not address every threat to causal interpretation. It does not repair selection bias, outcome or treatment measurement error, an inappropriate model, or violations of the study’s causal assumptions. Its value depends on a risk-ratio-scale effect estimate and the sensitivity model used. For these reasons, report the benchmark as a transparent comparison, not as a pass/fail rule for whether a study is credible.

Do not compare unlike quantities. Put the covariate associations on a risk-ratio scale and state the contrasts before comparing magnitudes.
Do not treat the E-value as a universal threshold. Unequal treatment and outcome associations should be assessed jointly using the bias factor.
Do not infer hidden-confounder strength from measured covariates. They are reference points, not guarantees or upper bounds.
Do not reverse the direction for protective estimates incorrectly. In the bias-factor bound, multiply a protective risk ratio toward 1.

Key takeaway: Calibrating an E-value against measured covariates turns an abstract sensitivity threshold into a data-informed comparison. The comparison is only as interpretable as its contrasts, scales, adjustment choices, and assumptions—and it cannot certify the absence of unmeasured confounding.

Check Your Understanding

Answer each question, then compare your reasoning with the brief solutions.

A hypothetical risk-ratio estimate is $2$. Compute its E-value to three significant figures.
A benchmark covariate has strengths $RR_{UA}=4$ and $RR_{UY}=2$. Compute its bias factor. Could that pair fully explain a risk ratio of $2.5$ under the stated sensitivity model?
Why is a measured covariate with treatment association strength $6$ not, by itself, evidence that an unmeasured confounder has strength at most $6$?
For a protective estimate of $0.50$ and a bias factor of $1.4$, which calculation moves the estimate toward the null: multiplying or dividing by the factor?

Solutions. (1) $E=2+\sqrt{2(1)}=3.414$. (2) $BF=4(2)/(4+2-1)=8/5=1.600$, which is less than $2.5$, so this pair does not fully explain that estimate under the model. (3) The measured covariate is only a benchmark; unmeasured factors may be stronger or differ in structure, and the observed association itself depends on its contrast and adjustment. (4) Multiply: $0.50(1.4)=0.700$, which is closer to 1.