Start With the Question, Not the Metric
A matched analysis can be sensitive to unmeasured confounding in more than one sense. A researcher may ask whether hidden differences in treatment assignment could change a significance conclusion, or how strongly a confounder would need to relate to treatment and outcome to account for an estimated effect. Rosenbaum bounds and E-values address these questions using different sensitivity scales. They are not two ways of reporting the same quantity.
The choice should follow the study’s estimand, outcome scale, available design information, and intended interpretation. A matched-set assignment model makes Rosenbaum bounds a natural option when the goal is to test how hidden treatment selection could affect inference. An E-value is a natural option when an effect can be expressed on a risk-ratio scale and the question concerns the strength of associations a confounder would need with treatment and outcome. Neither metric repairs a poor design, and neither identifies whether an unmeasured confounder actually exists.
What Each Metric Measures
For Rosenbaum bounds, suppose matched set \(i\) contains individuals with treatment probabilities \(\pi_{ij}\). The sensitivity model limits the ratio of the treatment-assignment odds for any two individuals in the set:
The bound is conditional on the matched design and its measured covariates. It asks how inference could change if treatment assignment within a set is not as-if random, but instead may differ by as much as the specified factor in its odds because of hidden bias. For a specified test statistic and null hypothesis, the analysis calculates a worst-case p-value over assignment patterns allowed by \(\Gamma\). It may also produce bounds for a confidence interval. Those results depend on the outcome data, matched-set structure, test, and direction of the alternative; \(\Gamma\) alone is not an effect estimate or a measure of confounder-outcome association.
An E-value is on a different scale. For a risk ratio \(RR\) at least 1, the point-estimate E-value is
For a protective estimate below 1, first invert the ratio, using \(1/RR\), and apply the formula. When interpreting a confidence limit, use the limit closest to the null; if the interval includes the null, its confidence-limit E-value is 1. The E-value’s assumptions and interpretation concern the strength of conditional associations with treatment and outcome, not treatment-assignment odds within matched sets.
A Short Derivation: Why the E-Value Is an Equal-Strength Threshold
In the standard risk-ratio sensitivity model, let \(a\geq 1\) denote the maximum confounder-treatment association and \(b\geq 1\) the maximum confounder-outcome association, each expressed as a risk ratio conditional on measured covariates. A commonly used upper bound on the bias factor is
To find the equal-strength threshold, set \(a=b=x\). If the observed risk ratio is \(RR\geq1\), the equal-strength associations must be large enough for the bias factor to reach \(RR\). Solving the equality gives
where the root \(x\geq1\) is selected. Thus the E-value is the equal-strength value for which the bias-factor bound reaches the observed ratio. Unequal strengths can also reach that bias-factor threshold: one association may be stronger than the E-value and the other weaker. The E-value does not say that each association must independently be at least \(E\).
Worked Example: Choosing an E-Value for a Risk-Ratio Estimate
Suppose a hypothetical matched cohort study estimates a harmful-outcome risk ratio of \(1.8\). The intended question is: how strong would unmeasured confounding need to be, on the risk-ratio scale, to explain away this point estimate? This is an E-value question, provided the risk ratio is the relevant estimand and the conditional-association sensitivity model is appropriate.
Calculate the equal-strength threshold.
Under the E-value framework, associations of \(3.00\) with both treatment and outcome, conditional on measured covariates and on the relevant risk-ratio scales, would suffice to explain away the point estimate. The value \(3.00\) is the equal-strength threshold; it is not a minimum that each association must meet.
For example, take \(a=10\) and \(b=1.979\). Then
This bias-factor bound is approximately the observed ratio \(1.8\), even though \(b\) is below \(3.00\). The very strong treatment association is paired with a weaker outcome association. This illustrates why the E-value must not be read as requiring both associations to exceed its numerical value. The result does not establish that such a confounder exists; it describes a sensitivity threshold under the framework’s assumptions.
Let the Estimand and Outcome Scale Guide the Choice
The E-value formula above is defined for a risk ratio or a quantity that has been validly expressed on that scale. A risk difference does not by itself determine a risk ratio: the relevant baseline risk is also needed. Odds ratios and hazard ratios likewise require care, because they are not automatically risk ratios. The earlier material on converting odds ratios and on E-values for hazard ratios covers those scale-specific issues; the practical rule here is to avoid inserting a different effect measure into the risk-ratio formula without a justified conversion. If the conversion is unavailable or not meaningful, an E-value may not answer the intended question.
Rosenbaum bounds are tied instead to the matched assignment design and an inference procedure. They can be useful with binary or quantitative outcomes when a suitable test statistic is defined for the matched sets. The outcome scale affects the statistic and the resulting bounds, but the sensitivity parameter remains an assignment-odds bound. A value such as \(\Gamma=2\) does not mean that a confounder has a twofold association with the outcome, nor that it doubles the estimated effect.
Use the estimand to be precise about what “explain away” means. An E-value for a point estimate concerns the amount of confounding needed to move that estimate to the null under its model. A confidence-limit E-value concerns how much confounding would move the confidence limit nearest the null to the null. Rosenbaum analyses commonly focus on whether a test rejects a specified null as \(\Gamma\) increases, or on how confidence bounds change. A p-value sensitivity result and a point-estimate sensitivity result are different questions even when calculated from the same matched study.
Worked Example: Converting a Risk Difference Before Using an E-Value
In a hypothetical matched cohort, suppose the estimated risk of an outcome is \(0.32\) among treated individuals and \(0.20\) among untreated individuals. The study reports a risk difference of \(0.12\), or 12 percentage points. If the intended sensitivity question is about a risk-ratio effect, calculate that ratio from the stated risks first.
The point-estimate E-value is then
The interpretation is that equal conditional associations of about \(2.58\) on the relevant risk-ratio scales would suffice under the E-value framework to explain away the ratio estimate of \(1.60\). Unequal associations may also suffice if their bias factor is large enough. The E-value does not say that an association of \(2.58\) is required for each variable in every possible confounding structure.
The calculation uses the two risks because a risk difference alone would not supply the denominator needed to form a risk ratio. If those risks were not available, the risk-ratio E-value could not be recovered from the risk difference alone. Nor does this conversion change the original estimand: the study’s reported absolute contrast remains 12 percentage points.
Use Matched-Set Information When the Question Is About Inference
A Rosenbaum analysis is especially relevant when the matched sets and the outcome data allow a meaningful assignment-based sensitivity analysis. Before using it, specify the null hypothesis, direction of testing, test statistic, and matched-set structure. The interpretation also depends on the assumptions behind the matched analysis: observed covariates were adequately handled by the design, outcomes and treatment are consistently defined, and interference between individuals is absent or appropriately addressed. The sensitivity model then varies the allowed hidden imbalance in treatment odds within sets; it does not make these design assumptions true.
Worked Example: Interpreting a Rosenbaum Bound for Matched Pairs
Consider a hypothetical one-sided analysis of 10 matched pairs with continuous outcomes and no ties. Define a favorable pair as one in which the treated individual has the outcome in the hypothesized beneficial direction. Suppose 9 pairs are favorable and 1 is unfavorable. Use a sign test of the sharp null that treatment has no effect for any individual. For this simplified paired sign-test sensitivity calculation, assume the worst-case probability of a favorable sign is \(q=\Gamma/(1+\Gamma)\).
At \(\Gamma=1\), \(q=1/2\), so the one-sided p-value is the probability of at least 9 favorable signs under a binomial distribution with 10 trials and success probability \(1/2\):
Now allow \(\Gamma=1.5\). Then \(q=1.5/2.5=0.6\), giving the worst-case upper-tail probability
At \(\Gamma=1.6\), \(q=1.6/2.6\approx0.6154\), and
At significance level \(\alpha=0.05\), the worst-case p-value is below \(0.05\) at \(\Gamma=1.5\) and above \(0.05\) at \(\Gamma=1.6\). In this example, the rejection conclusion is therefore sensitive somewhere between these values. This calculation concerns the specified one-sided sharp-null sign test and simplified paired model; it is not a universal conversion from \(\Gamma\) to a confounder’s associations, and it does not quantify how the estimated effect changes.
A Practical Selection Rule
The two metrics can be used as complementary analyses when each answers a clearly stated question and its assumptions fit the study. They should not be compared numerically as though \(\Gamma=2\) were equivalent to an E-value of 2. One bounds within-set treatment odds; the other summarizes conditional risk-ratio associations capable of producing a specified bias factor. No general conversion exists between them.
| Study feature or question | More natural starting point | What it can tell you |
|---|---|---|
| Matched sets and a question about robustness of a test or interval to hidden assignment bias | Rosenbaum bounds | How a worst-case inference result changes as the allowed within-set treatment-odds imbalance increases |
| A meaningful risk ratio and a question about confounder association strengths needed to explain an estimate or confidence limit | E-value | An equal-strength threshold on conditional risk-ratio association scales, with unequal strengths also possible |
| A risk difference without the risks needed to form a ratio, or an effect measure without a justified risk-ratio conversion | Neither E-value formula by default | First clarify the estimand and scale; do not substitute an incompatible measure |
| No usable matched-set assignment information, but a suitable ratio-scale effect is available | E-value may be more practical | A scale-based confounding threshold, not a design-specific assignment analysis |
Diagnostics should check whether the chosen sensitivity analysis is answering the question claimed. For Rosenbaum bounds, verify the matched-set structure, outcome direction, statistic, and null; report how the bounded inference changes over a range of \(\Gamma\), not just one selected value. For E-values, verify the effect scale, use the correct direction for protective estimates, and distinguish the point-estimate value from the value for a confidence limit. For either method, explain why the sensitivity range is plausible in context. A large threshold alone is not evidence that residual confounding is absent.
Common Pitfalls and Limitations
- Reading an E-value as a minimum for each association. The E-value is the equal-strength threshold. Unequal associations can reach the same bias-factor threshold; for instance, a very strong treatment association can be paired with a weaker outcome association.
- Calling \(\Gamma\) a confounder-outcome association. \(\Gamma\) limits treatment-assignment odds within matched sets. It does not directly quantify confounder-outcome strength.
- Using the risk-ratio formula on an incompatible scale. A risk difference, odds ratio, and hazard ratio are not interchangeable with a risk ratio. Convert only when the necessary information and assumptions support conversion.
- Equating statistical significance with effect robustness. Rosenbaum bounds on a test result do not by themselves show that the point estimate is unchanged. An E-value for a point estimate does not show whether a particular test remains significant.
- Treating a sensitivity threshold as a test for confounding. Neither metric detects or rules out a hidden confounder. Both describe consequences under specified models.
The metrics also have limits in common: they cannot correct measurement error in recorded covariates, bad matching, selection problems, or an ill-defined estimand. Rosenbaum bounds depend on a specified matched design and test, while E-values depend on ratio-scale interpretation and a confounding-bias model. Select the metric that corresponds to the scientific claim, and state its assumptions and limitations alongside the result.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A matched study has binary outcomes and the investigator wants to know whether a test conclusion survives hidden imbalance in treatment assignment within pairs. Which metric is the more direct starting point, and what does its sensitivity parameter describe?
- A harmful-outcome risk ratio is \(2.0\). Calculate its point-estimate E-value and interpret it carefully.
- A protective risk ratio is \(0.50\). Which ratio should enter the E-value formula, and what is the resulting value?
- An E-value is \(3\). Must both confounder associations be at least 3? Explain using the bias-factor idea.
- A study reports only a risk difference of 0.10, with no baseline risk. Can its risk-ratio E-value be calculated from that information alone? Why?
Solutions. (1) Rosenbaum bounds are the direct starting point because the question concerns hidden treatment-assignment imbalance in matched pairs. \(\Gamma\) bounds the ratio of treatment-assignment odds within a pair, conditional on measured covariates; it is not an outcome-association measure. (2) \(E=2+\sqrt{2(2-1)}=2+\sqrt{2}\approx3.414\). Equal conditional associations of about 3.414 on the relevant risk-ratio scales would suffice under the framework to explain away the point estimate; unequal strengths may also suffice if their bias factor is large enough. (3) Invert the protective ratio: \(1/0.50=2.0\). The E-value is again \(2+\sqrt{2}\approx3.414\). (4) No. Three is the equal-strength threshold. For association strengths \(a\) and \(b\), the bias-factor bound is \(ab/(a+b-1)\), so unequal strengths can also reach the required threshold. (5) No. A risk difference alone does not specify the risk ratio; the risks, or sufficient information to determine them, are needed.