Why Effect Heterogeneity Matters for E-Values
An E-value summarizes the strength of association that an unmeasured confounder would need to have with both treatment and outcome, on the risk-ratio scale, to explain away an observed association under the E-value framework. That summary is tied to the risk ratio being analyzed. When treatment effects differ across subgroups, there may be no single risk ratio that describes every subgroup or every target population. A pooled E-value can therefore be mathematically correct for one marginal estimate while concealing substantial differences in subgroup effects and sensitivity summaries.
The key distinction is between effect heterogeneity and unmeasured confounding. Effect heterogeneity means the treatment effect varies across levels of a characteristic, such as age or baseline disease severity. Unmeasured confounding concerns bias from a common cause of treatment and outcome that has not been controlled. Heterogeneity does not itself establish confounding, and an E-value does not measure how heterogeneous effects are.
For a binary outcome, let \(p_{1s}\) and \(p_{0s}\) be the risks under treatment and control, respectively, in subgroup \(s\). The subgroup-specific causal risk ratio is \(RR_s=p_{1s}/p_{0s}\). If a target population has subgroup proportions \(w_s\), with \(\sum_s w_s=1\), its marginal risks under treatment and control are \(\sum_s w_s p_{1s}\) and \(\sum_s w_s p_{0s}\). The corresponding marginal causal risk ratio is their ratio. This standardization presumes that the subgroup-specific risks are appropriate for the target population and that the same target-population weights are used to describe both treatment strategies.
The second equality shows why the marginal risk ratio is generally not the simple average of the subgroup risk ratios. It is a weighted average of them with weights proportional to \(w_s p_{0s}\), the subgroup’s contribution to the target population’s control risk. If subgroup risks differ, changing the population composition can change those weights and therefore change the marginal risk ratio, even when every subgroup-specific risk remains the same.
Worked Example: Different Effects Across Two Subgroups
Worked Example: Different Effects Across Two Subgroups
Suppose a hypothetical study concerns a treatment and a binary adverse outcome. The following subgroup risks are assumed to be valid for a target population. They are hypothetical values, not results from a real study.
| Subgroup | Control risk | Treatment risk | Subgroup risk ratio |
|---|---|---|---|
| Lower baseline risk | 0.050 | 0.045 | 0.045 / 0.050 = 0.90 |
| Higher baseline risk | 0.300 | 0.150 | 0.150 / 0.300 = 0.50 |
The effect is protective in both subgroups, but its relative magnitude differs: the subgroup risk ratios are 0.90 and 0.50. The difference is effect heterogeneity on the risk-ratio scale.
For a target population in which 80% are in the lower-baseline-risk subgroup and 20% are in the higher-baseline-risk subgroup, the marginal control risk is \(0.80(0.050)+0.20(0.300)=0.040+0.060=0.100\). The marginal treatment risk is \(0.80(0.045)+0.20(0.150)=0.036+0.030=0.066\). Thus the marginal risk ratio is \(0.066/0.100=0.660\).
For this protective ratio, invert it before applying the formula: \(1/0.660=1.5151515\ldots\). Retaining the unrounded ratio in the calculation gives \((0.100/0.066)+\sqrt{(0.100/0.066)((0.100/0.066)-1)}=2.399\), to three decimal places.
The pooled E-value of 2.399 is for this marginal risk ratio in this target population. It does not mean each subgroup has that E-value, nor does it describe the difference between the subgroup effects. In particular, the subgroup E-values would differ: for 0.90, the calculation uses \(1/0.90\), giving \((1/0.90)+\sqrt{(1/0.90)((1/0.90)-1)}=1.462\); for 0.50 it gives \(2+\sqrt{2}=3.414\). These are sensitivity summaries for different subgroup ratios, not components to average into the marginal E-value.
Target-Population Composition Changes the Summary
Keep the same hypothetical subgroup risks, but consider a second target population in which 20% are in the lower-baseline-risk subgroup and 80% are in the higher-baseline-risk subgroup. The subgroup-specific ratios have not changed. The marginal risks and marginal ratio do change because the target-population weights differ.
Worked Example: Reweighting to Another Population
For the second target population, the marginal control risk is \(0.20(0.050)+0.80(0.300)=0.010+0.240=0.250\). The marginal treatment risk is \(0.20(0.045)+0.80(0.150)=0.009+0.120=0.129\). Therefore the marginal risk ratio is \(0.129/0.250=0.516\).
This is a protective ratio, so its reciprocal is \(0.250/0.129=1.9379845\ldots\). Using the unrounded ratio, \((0.250/0.129)+\sqrt{(0.250/0.129)((0.250/0.129)-1)}=3.286\), to three decimal places.
The two target populations have identical subgroup-specific risks, yet their marginal ratios are 0.660 and 0.516, and their E-values are 2.399 and 3.286. The second population gives more weight to the subgroup with the stronger protective relative effect and higher control risk. The change in the E-value is therefore not evidence that the treatment’s subgroup effects changed or that confounding differs between populations. It reflects a different marginal contrast, to which the E-value formula was applied.
Why the E-Value Is Not an Average of Subgroup Values
Two operations are involved: combining subgroup risks to obtain a marginal risk ratio, and transforming that ratio to obtain an E-value. Neither operation generally permits averaging the subgroup E-values. The marginal ratio is formed from weighted risks, and the E-value applies a nonlinear formula to the resulting ratio. In addition, each subgroup E-value uses its own subgroup risk ratio, while the marginal E-value uses a target-population risk ratio.
A brief derivation makes the weighting explicit. Since \(p_{1s}=p_{0s}RR_s\), summing treatment risks gives \(\sum_s w_s p_{1s}=\sum_s w_s p_{0s}RR_s\). Dividing by the marginal control risk \(\sum_s w_s p_{0s}\) yields the displayed weighted-average expression. The normalized weights are \(w_s p_{0s}/\sum_j w_jp_{0j}\), not simply \(w_s\). Unless the subgroup control risks are equal (or special numerical coincidences occur), these weights differ from the population proportions. Applying the E-value transformation afterward cannot turn this result into an ordinary average of subgroup E-values.
Worked Example: Checking the Weighting Directly
In the first population, the subgroup proportions are 0.80 and 0.20, and the control risks are 0.050 and 0.300. The control-risk contributions are \(0.80(0.050)=0.040\) and \(0.20(0.300)=0.060\), summing to 0.100. Thus the normalized weights for averaging subgroup risk ratios are 0.040/0.100 = 0.40 and 0.060/0.100 = 0.60, not 0.80 and 0.20.
The risk-ratio calculation is \(0.40(0.90)+0.60(0.50)=0.36+0.30=0.66\), agreeing with the marginal risks. By contrast, averaging subgroup ratios by population proportion gives \(0.80(0.90)+0.20(0.50)=0.82\), which is not the marginal ratio. The weighted subgroup E-values also do not yield the marginal E-value: \(0.80(1.462)+0.20(3.414)=1.852\), whereas the marginal E-value is 2.399. That arithmetic is not a valid pooled sensitivity analysis.
Interpreting and Diagnosing a Pooled E-Value
A pooled E-value should be read with its estimate, outcome, scale, and target population in view. In its standard sensitivity interpretation, it describes how strong associations an unmeasured confounder would need to have with treatment and outcome, conditional on the measured covariates and on the assumptions of the E-value framework, to explain away the specified observed risk-ratio association. It is not a probability that confounding exists, a test for heterogeneity, or evidence that the estimate is causal.
When effects vary, useful diagnostics and reporting practice include:
- State the target population and the subgroup weights used to obtain a marginal estimate.
- Show subgroup-specific risks or risk ratios alongside the marginal risks and ratio when subgroup comparisons are substantively important and estimable.
- Check whether the pooled estimate is crude, adjusted, or standardized, and ensure the E-value is calculated for that same clearly defined estimate and scale.
- Examine uncertainty and data support within subgroups. Sparse events can make subgroup ratios unstable; an apparent difference may reflect sampling error rather than real heterogeneity.
- Consider whether a reported summary is driven by a large subgroup or by a subgroup with a high baseline risk. Population proportions alone do not determine the marginal risk-ratio weights.
A sensible comparison is between a marginal analysis for a defined target population and a stratified or model-based analysis that estimates subgroup-specific effects. The marginal result answers a population-level question; the subgroup results address how effects may differ across strata. These are complementary estimands, not competing calculations of a single universal effect. If a different target population is of interest, standardize the subgroup risks to its weights and calculate that population’s marginal ratio before computing its E-value.
Common Pitfall and Limitations
A single summary can also conceal opposing subgroup directions in other settings. Even if a pooled association appears close to the null, subgroup ratios could be meaningfully above and below 1; conversely, a notable marginal association does not guarantee similar effects in every subgroup. The E-value formula does not diagnose such patterns. Nor does it solve transportability problems, correct model misspecification, or account for uncertainty simply by reporting a point-estimate E-value. Those questions require examining the estimates, their uncertainty, and the target population directly.
The purpose of the E-value remains narrower: it supplies a scale-based sensitivity summary for a specified estimate. Under heterogeneity, careful specification of that estimate becomes especially important. A pooled number can be useful, but only alongside a clear description of what was pooled and which population it represents.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- Two subgroups have control risks 0.10 and 0.40, and subgroup risk ratios 0.80 and 0.50. A target population is 50% in each subgroup. Calculate its marginal control risk, treatment risk, and risk ratio.
- For a protective marginal risk ratio of 0.70, which value is entered into the E-value formula?
- Why is a marginal risk ratio generally not the average of subgroup risk ratios weighted only by subgroup proportions?
- A second population has a different mix of subgroups but the same subgroup-specific risks. If its marginal E-value differs, does that alone show that confounding strength changed? Explain.
- What does an E-value fail to tell you when subgroup effects are heterogeneous?
Solutions. (1) The treatment risks are \(0.10(0.80)=0.08\) and \(0.40(0.50)=0.20\). The marginal control risk is \(0.50(0.10)+0.50(0.40)=0.25\), and the marginal treatment risk is \(0.50(0.08)+0.50(0.20)=0.14\). The marginal ratio is \(0.14/0.25=0.56\). (2) Use the reciprocal, \(1/0.70=1.4286\), because the formula is stated for ratios above 1. (3) The ratio combines risks; equivalently, it averages subgroup ratios with weights proportional to subgroup proportion times subgroup control risk, not subgroup proportion alone. (4) No. The marginal estimate changes with population composition, and the E-value changes because it is a transformation of that estimate. That difference alone does not establish a change in confounding. (5) It does not measure effect heterogeneity, show whether subgroup effects differ statistically, or identify the subgroup-specific sensitivity to unmeasured confounding. Those require subgroup estimates and their uncertainty.