E-Values for Matched-Study Estimates

Why an E-Value Needs the Right Matched Estimate

Matching can make treated and comparison groups more alike on measured characteristics. It does not, by itself, ensure that the groups are comparable on characteristics that were not measured or were measured poorly. An E-value offers a way to describe how strong residual confounding would need to be, on a risk-ratio scale, to explain away a reported association.

For a matched study, the first task is not the arithmetic. It is identifying the association measure and target population represented by the estimate. A matched-cohort analysis may report a risk ratio based on risks estimated in the matched sample. A matched case-control analysis often reports an odds ratio. The risk-ratio E-value formula applies to a risk ratio; it should not be applied directly to an odds ratio merely because the study used matching.

Definition: For a risk ratio $R$ comparing an outcome risk under exposure with the corresponding risk without exposure, the E-value for an estimate farther from the null than 1 is $$ E = R + \sqrt{R(R-1)} \quad \text{if } R \geq 1. $$ If $0<R<1$, reverse the comparison by using $1/R$: $$ E = \frac{1}{R} + \sqrt{\frac{1}{R}\left(\frac{1}{R}-1\right)}. $$ The E-value is the minimum common strength, on the risk-ratio scale, that an unmeasured confounder would need to have in its associations with exposure and outcome, conditional on the measured covariates, to explain the estimated association with the null under the standard E-value bound.

In a matched analysis, “conditional on the measured covariates” includes the factors controlled by the matching and by the analysis. The estimate might be a risk ratio within matched strata, or a marginal risk ratio standardized over the matched sample. These are not automatically the same estimand. Use the risk ratio that the analysis actually reports and interpret the E-value for that estimate and its target population.

What Matching Does—and Does Not—Establish

A causal interpretation of a matched risk ratio ordinarily requires a well-defined exposure and outcome, consistency, adequate overlap in exposure within relevant covariate patterns, and conditional exchangeability given the measured covariates if no unmeasured confounding is assumed. The matching procedure and analysis also need to support the reported estimand; for example, the analysis must account for the matched design where appropriate. An E-value does not verify these conditions.

Instead, the E-value asks a sensitivity question after the risk-ratio estimate has been obtained: how strong would residual confounding have to be, under a specified bound, to move this estimate to the null? It does not show that such a confounder exists, that one does not exist, or that the estimate is causal. It is a benchmark expressed on a scale that can be compared with plausible confounder associations.

A short justification explains the formula. Suppose the confounder’s associations with exposure and outcome, conditional on the measured covariates, are each bounded by a common value $G\geq 1$ on the risk-ratio scale. A standard upper bound on the bias factor is

B(G) = \frac{G^2}{2G-1}.

To explain an observed risk ratio $R\geq 1$ fully, this bound must be at least $R$. Setting $B(G)=R$, multiplying through, and solving the resulting quadratic gives

\frac{G^2}{2G-1}=R \quad\Longrightarrow\quad G^2-2RG+R=0 \quad\Longrightarrow\quad G=R\pm\sqrt{R(R-1)}.

The relevant solution is the larger root, $R+\sqrt{R(R-1)}$, because $G\geq1$ and the other root is not the required confounding-strength threshold. For a protective estimate, reversing the comparison first gives the formula using $1/R$.

Calculating from Matched-Cohort Risks

When a matched-cohort analysis reports risks in the matched target population, calculate the risk ratio from those risks if it is not already supplied. A standardized risk ratio, for example, compares risks averaged over a common distribution of matched strata. Do not calculate a crude ratio from unmatched group totals if that is not the analysis’s reported estimand.

Worked Example: A Standardized Risk Ratio in Matched Strata

Suppose a hypothetical matched cohort contains two equally weighted strata. In each stratum there are 50 exposed and 50 unexposed people. The recorded outcomes are:

Matched stratum	Exposed with outcome / total	Unexposed with outcome / total
1	6 / 50	3 / 50
2	6 / 50	4 / 50

Using equal stratum weights, the standardized exposed risk is $(6/50+6/50)/2=0.120$. The standardized unexposed risk is $(3/50+4/50)/2=0.070$. Thus the matched-population risk ratio is

R = \frac{0.120}{0.070} = \frac{12}{7} =1.7142857\ldots.

Retain the unrounded ratio in the E-value calculation:

E = \frac{12}{7}+\sqrt{\frac{12}{7}\left(\frac{12}{7}-1\right)} =1.7142857\ldots+\sqrt{1.2244898\ldots} =2.820852\ldots \approx 2.821.

Under the E-value bound, an unmeasured confounder would need associations with exposure and outcome, each of about 2.821-fold or stronger on the risk-ratio scale, conditional on the measured covariates, to explain this point estimate fully to the null. This is a sensitivity benchmark for the standardized matched-population estimate, not evidence that the outcome risk was caused by exposure.

This calculation depends on using the common stratum weights. If the analysis instead reports a different adjusted risk ratio, use that reported estimate. Inference and sensitivity interpretation should refer to the same matched population and analysis; silently switching between a conditional matched-set estimate and a marginal standardized estimate changes what the number describes.

Protective Estimates and the Direction of Comparison

For a protective risk ratio below 1, the E-value formula is applied after taking the reciprocal. This puts the comparison on the side of the null above 1 and avoids treating a protective association as though the formula’s $R\geq1$ case applied unchanged. The reciprocal changes the direction of comparison for the calculation; it does not change the original estimate’s meaning.

Worked Example: A Protective Matched Risk Ratio

A hypothetical matched-cohort analysis reports a risk ratio of $R=0.640$ for an intervention compared with no intervention. The estimate corresponds to a 36.0% lower risk in the intervention group, since $1-0.640=0.360$. For the E-value calculation, reverse the ratio:

\frac{1}{R}=\frac{1}{0.640}=1.5625.

Then

E=1.5625+\sqrt{1.5625(1.5625-1)} =1.5625+\sqrt{0.87890625} =1.5625+0.9375 =2.500.

Thus, under the standard bound, confounder associations of at least 2.500-fold each with intervention and outcome, conditional on the measured covariates, could be strong enough to explain the point estimate to the null. The value does not mean that a confounder is 2.5 times as prevalent, nor does it measure the probability that confounding is present.

When the Matched Analysis Reports an Odds Ratio

Matched case-control analyses commonly report an odds ratio, often based on the exposure patterns in matched sets. That odds ratio is not a risk ratio. The E-value formula above is defined for risk ratios, so inserting the odds ratio directly can produce a misleading sensitivity benchmark, especially when the outcome is not rare.

If a suitable baseline outcome risk is available from the relevant source population and the reported odds ratio is compatible with that risk’s conditioning, an odds ratio can sometimes be converted to a risk ratio. For a single covariate stratum, if $p_0$ is the unexposed outcome risk and $OR$ is the corresponding odds ratio, the conversion is

RR = \frac{OR}{(1-p_0)+p_0OR}.

In a matched analysis, the conditioning must agree: a stratum-specific odds ratio requires the appropriate baseline risk for that stratum. A marginal baseline risk cannot automatically convert a conditional matched odds ratio into the desired marginal risk ratio. Case-control sampling fractions also do not supply the source-population outcome risk.

Worked Example: Converting a Matched Odds Ratio Before Sensitivity Analysis

Consider a hypothetical matched case-control analysis for one defined covariate stratum. It reports a conditional odds ratio of $OR=0.50$. An external source-population estimate gives the unexposed outcome risk in that same stratum as $p_0=0.100$. Under the stated compatibility of these quantities, convert the odds ratio to a risk ratio:

RR = \frac{0.50}{(1-0.100)+(0.100)(0.50)} =\frac{0.50}{0.900+0.050} =\frac{0.50}{0.950} =0.5263158\ldots.

This is protective, so reverse it for the E-value:

\frac{1}{RR}=1.900.

Therefore,

E=1.900+\sqrt{1.900(1.900-1)} =1.900+\sqrt{1.710} =3.20767\ldots \approx 3.208.

The resulting benchmark pertains to the converted risk ratio for this stratum. If the baseline risk were unavailable, or if it did not correspond to the same stratum and target population, this calculation would not be justified. In particular, the fraction of sampled controls with the outcome is not a substitute for a source-population risk in a case-control design.

Uncertainty, Diagnostics, and Limitations

An E-value for a point estimate describes sensitivity of that estimate, not the precision of the study. If a confidence interval is available, the interval’s limit closest to the null can have a different E-value from the point estimate. When the interval includes the null, the confidence-limit E-value is 1 under the usual convention: sampling uncertainty alone does not rule out the null. The point-estimate E-value can still be reported, but it should not be presented as though it establishes a robust non-null conclusion.

Before interpreting an E-value from a matched study, check the following:

Measure: Is the estimate a risk ratio, rather than an odds ratio or hazard ratio?
Estimand: Does it describe the matched population, a matched stratum, or a standardized target population?
Analysis: Were the matching structure and any weights handled in producing the reported estimate?
Direction: For a ratio below 1, was the reciprocal used in the formula?
Uncertainty: Is the point-estimate E-value being distinguished from the E-value for the confidence limit nearest the null?
Context: Are plausible confounder associations being compared on the same conditional risk-ratio scale?

Matching can improve measured covariate balance, but neither balance nor a large E-value proves that residual confounding is absent. The E-value also does not address selection bias, measurement error, model misspecification, or other sources of bias. Rosenbaum bounds use a different sensitivity framework for hidden bias in matched designs; the two approaches quantify sensitivity differently and are not interchangeable. The appropriate report identifies the estimate, its scale, the relevant matched population, and the sensitivity interpretation rather than presenting the E-value alone.

Common mistake: “The matched study reported an odds ratio of 0.50, so I can put 0.50 into the protective risk-ratio E-value formula.” This is incorrect because an odds ratio and a risk ratio are different measures, and matching does not make them equal. Use a compatible risk ratio, or convert the odds ratio only when the required baseline risk and conditioning are available. If they are not, state that the risk-ratio E-value cannot be computed from the reported information alone.

Check Your Understanding

For each question, identify the relevant matched-study quantity and show enough reasoning to support your answer. Solutions are provided below.

A matched-cohort risk ratio is 2.25. What is its point-estimate E-value?
A matched study reports a risk ratio of 0.50. What value should enter the risk-ratio E-value formula, and what is the E-value?
A matched case-control study reports an odds ratio of 0.70 but gives no suitable source-population outcome risk. Can its risk-ratio E-value be calculated from this information alone?
In one matched stratum, $OR=2.00$ and the compatible unexposed outcome risk is $p_0=0.20$. Find the converted risk ratio. Is this conversion alone sufficient to claim the result is causal?

Solutions. (1) $E=2.25+\sqrt{2.25(1.25)}=2.25+\sqrt{2.8125}=3.927$ (approximately). (2) Reverse the ratio: $1/0.50=2$. Then $E=2+\sqrt{2(1)}=3.414$ (approximately). (3) No. The odds ratio is not a risk ratio, and without a compatible baseline risk the required conversion is unavailable; the sampled case-control outcome proportion is not a substitute. (4) $RR=2.00/[(1-0.20)+(0.20)(2.00)]=2/1.20=1.667$ (approximately). No: conversion puts the association on the risk-ratio scale but does not establish exchangeability, correct analysis, or a causal interpretation.