How Outcome Scales Change Confounding Sensitivity

Why the Outcome Scale Matters

Sensitivity analysis asks how conclusions might change under assumptions about unmeasured confounding. But “how much bias?” has no unique answer until the effect measure is specified. A difference of 5 percentage points, a risk ratio of 1.5, and an odds ratio of 1.5 are different quantities, and the same pair of outcome risks can produce very different numerical contrasts on these scales.

For a binary treatment \(A\) and binary outcome \(Y\), let \(R_1\) and \(R_0\) denote the risks under treatment and no treatment in a specified target population. The effect measures below are defined using those same two risks. In observational data, an unadjusted comparison of observed risks need not equal a causal contrast; sensitivity analysis examines how assumptions about unmeasured confounding could explain that difference.

Definition: The risk difference (RD) is \(R_1-R_0\), measured in probability points. The risk ratio (RR) is \(R_1/R_0\), a relative risk. The odds ratio (OR) is \(\{R_1/(1-R_1)\}/\{R_0/(1-R_0)\}\), a ratio of odds. These are distinct effect measures, not interchangeable versions of one numerical effect.

The risk difference is often useful for expressing absolute changes in outcome frequency. The risk ratio expresses relative changes in risk. The odds ratio compares odds, not risks; when outcomes are common, its value can be appreciably farther from 1 than the corresponding risk ratio. None of these scales is universally best. The relevant scale depends on the scientific question and the decision being informed.

Observed and Standardized Risks Under a Binary Confounder

To make scale-specific sensitivity concrete, suppose \(U\) is a binary, unmeasured confounder, and any measured covariates have already been handled within the target population under study. Write \(p_a=P(U=1\mid A=a)\), and let \(q_{au}=P(Y=1\mid A=a,U=u)\) be the outcome risk in treatment group \(a\) and confounder group \(u\). The observed risk in treatment group \(a\) is a mixture over that group’s confounder distribution.

m_a=(1-p_a)q_{a0}+p_aq_{a1}.

Because the treatment groups can have different values of \(p_a\), the observed risks \(m_1\) and \(m_0\) may reflect both treatment and differences in confounder composition. To define causal risks for comparison, choose a target prevalence \(g=P(U=1)\) and standardize both treatment levels to that same distribution:

R_a^g=(1-g)q_{a0}+gq_{a1}.

Interpreting \(R_a^g\) as the causal risk under treatment level \(a\) requires assumptions. In particular, assume consistency, positivity of treatment within each \(U\) group, and no remaining unmeasured confounding conditional on \(U\). In a sensitivity analysis, the risks \(q_{au}\) and prevalences \(p_a\) may be varied over plausible values rather than treated as known. The point is to see what each assumed scenario implies on the chosen effect scale.

Worked Example: One Risk Pair, Three Effect Measures

Consider a hypothetical population in which the risk is 10% without treatment and 20% with treatment. Calculate the RD, RR, and OR, then repeat the calculation for a second hypothetical population with risks of 40% and 50%.

For the first population, \(R_0=0.10\) and \(R_1=0.20\):

\begin{aligned} \mathrm{RD}&=0.20-0.10=0.10,\\ \mathrm{RR}&=\frac{0.20}{0.10}=2.00,\\ \mathrm{OR}&=\frac{0.20/0.80}{0.10/0.90} =\frac{0.25}{0.1111}=2.25. \end{aligned}

The treatment risk is 10 percentage points higher; it is twice the untreated risk, and the treatment odds are 2.25 times the untreated odds. In the second population, \(R_0=0.40\) and \(R_1=0.50\):

\begin{aligned} \mathrm{RD}&=0.50-0.40=0.10,\\ \mathrm{RR}&=\frac{0.50}{0.40}=1.25,\\ \mathrm{OR}&=\frac{0.50/0.50}{0.40/0.60} =\frac{1}{0.6667}=1.50. \end{aligned}

The absolute difference is 10 percentage points in both populations, but the relative risk and odds ratio differ because the baseline risks differ. These calculations do not indicate confounding: they show that even a fixed risk difference translates into different relative contrasts at different baselines.

How a Binary Confounder Creates Bias on the Risk-Difference Scale

A simple exact bias expression is available on the RD scale under a particular model. Suppose the treatment risk difference is constant across the two \(U\) groups, and the confounder risk difference is also constant across treatment groups. Write \(\tau=q_{1u}-q_{0u}\) for the constant treatment RD and \(\delta=q_{a1}-q_{a0}\) for the constant difference in outcome risk between \(U=1\) and \(U=0\). Then the difference between the observed RD and the causal RD standardized to any common \(U\) distribution is determined by how much the prevalence of \(U\) differs between treatment groups.

Result: Under the stated constant-risk-difference assumptions, the observed risk difference is \(m_1-m_0=\tau+(p_1-p_0)\delta\). Thus the additive confounding bias, defined as observed RD minus causal RD, is \((p_1-p_0)\delta\).

To verify this result, let \(q_{00}\) be the risk for untreated people with \(U=0\). The assumptions imply \(q_{01}=q_{00}+\delta\), \(q_{10}=q_{00}+\tau\), and \(q_{11}=q_{00}+\tau+\delta\). Substituting these values into the mixture risks gives:

\begin{aligned} m_1&=(1-p_1)(q_{00}+\tau)+p_1(q_{00}+\tau+\delta) =q_{00}+\tau+p_1\delta,\\ m_0&=(1-p_0)q_{00}+p_0(q_{00}+\delta) =q_{00}+p_0\delta,\\ m_1-m_0&=\tau+(p_1-p_0)\delta. \end{aligned}

The same derivation shows why a common target distribution removes this component of bias: standardizing both groups to prevalence \(g\) yields \(R_1^g-R_0^g=\tau\). The formula depends on additive, constant effects; it is not a general bias formula for all outcome models. There is no corresponding universal shortcut that converts this RD bias into a fixed RR or OR bias without additional assumptions.

Worked Example: Calculating Additive Confounding Bias

In a hypothetical study, suppose \(P(U=1\mid A=1)=0.60\), \(P(U=1\mid A=0)=0.20\), and the outcome risks are \(q_{00}=0.10\), \(q_{01}=0.30\), \(q_{10}=0.20\), and \(q_{11}=0.40\). Assume consistency, positivity, and that these conditional risks identify the causal risks within \(U\) groups. Check the constant-RD assumptions and calculate the observed and standardized contrasts.

The treatment RD is \(0.20-0.10=0.10\) for \(U=0\), and \(0.40-0.30=0.10\) for \(U=1\). The confounder RD is \(0.30-0.10=0.20\) without treatment and \(0.40-0.20=0.20\) with treatment. Thus \(\tau=0.10\) and \(\delta=0.20\), satisfying the assumptions.

The observed risks are mixtures using each treatment group’s own \(U\) prevalence:

\begin{aligned} m_1&=(1-0.60)(0.20)+0.60(0.40)=0.32,\\ m_0&=(1-0.20)(0.10)+0.20(0.30)=0.14. \end{aligned}

The observed RD is \(0.32-0.14=0.18\), or 18 percentage points. Now standardize both treatment groups to a hypothetical target prevalence \(g=0.40\):

\begin{aligned} R_1^g&=(1-0.40)(0.20)+0.40(0.40)=0.28,\\ R_0^g&=(1-0.40)(0.10)+0.40(0.30)=0.18,\\ R_1^g-R_0^g&=0.28-0.18=0.10. \end{aligned}

The formula gives the same result: \((p_1-p_0)\delta=(0.60-0.20)(0.20)=0.08\), and \(0.18-0.10=0.08\). Under these assumptions, the crude RD exceeds the standardized causal RD by 8 percentage points. This is a statement about this hypothetical model and target population, not an estimate that can be transported to another scale without calculation.

Odds Ratios and Noncollapsibility

Confounding is not the only reason a conditional and marginal effect measure can differ. The odds ratio is noncollapsible: averaging over a prognostic variable can change an odds ratio even when that variable is independent of treatment and therefore does not confound the treatment-outcome comparison. This is a mathematical property of the odds ratio, not evidence by itself of bias.

A conditional OR compares odds within levels of \(U\). A marginal OR compares odds after risks have been averaged over a specified distribution of \(U\). They answer different questions. In general, calculating a conditional OR and calling it the marginal causal OR is not justified. For a marginal causal OR, first obtain the two risks standardized to the target distribution, then calculate the odds ratio from those risks.

Worked Example: A Conditional OR of 2 but a Marginal OR Below 2

Imagine a hypothetical randomized setting where \(U\) is present in half the population in both treatment groups, so \(U\) is not associated with treatment. Within each \(U\) group, suppose the treatment OR is 2. Without treatment, the outcome risks are 0.10 for \(U=0\) and 0.40 for \(U=1\). Calculate the treatment risks and the marginal OR.

For \(U=0\), the untreated odds are \(0.10/0.90=0.1111\). Doubling the odds gives \(0.2222\), corresponding to treatment risk \(0.2222/(1+0.2222)=0.1818\), or \(2/11\). For \(U=1\), the untreated odds are \(0.40/0.60=0.6667\). Doubling gives \(1.3333\), corresponding to treatment risk \(1.3333/(1+1.3333)=0.5714\), or \(4/7\). Thus the conditional OR is 2 in both groups.

With each \(U\) group making up half of each treatment group, the marginal risks are:

\begin{aligned} R_0&=0.5(0.10)+0.5(0.40)=0.25,\\ R_1&=0.5(2/11)+0.5(4/7)=29/77\approx0.3766. \end{aligned}

The marginal OR is:

\mathrm{OR}_{marginal} =\frac{(29/77)/(48/77)}{0.25/0.75} =\frac{29/48}{1/3} =\frac{87}{48} =1.813.

There is no confounding by \(U\) in this example because its distribution is the same in both treatment groups. Nevertheless, the marginal OR of 1.813 differs from the conditional OR of 2. As a comparison, the marginal RR is \(0.3766/0.25=1.506\), and the marginal RD is \(0.3766-0.25=0.1266\), or about 12.66 percentage points. The difference between conditional and marginal ORs must not automatically be described as confounding bias.

Baseline Risk Changes the Bias Interpretation

A useful sensitivity analysis specifies the outcome risks associated with \(U\), the prevalence of \(U\) by treatment group, and the target distribution. The resulting mixture risks can then be compared on each scale. This makes assumptions visible and avoids treating a “bias factor” calculated on one scale as though it applied unchanged on another.

Worked Example: The Same Relative Scenario at Two Baseline Risks

Consider two hypothetical scenarios with the same confounder prevalences: \(p_0=0.20\) and \(p_1=0.60\). In both, treatment multiplies the risk by 1.5 within each \(U\) group, and \(U=1\) doubles risk relative to \(U=0\). Standardize to \(g=0.40\). Compare a low-baseline scenario with untreated risks 0.05 and 0.10, and a higher-baseline scenario with untreated risks 0.20 and 0.40, for \(U=0\) and \(U=1\), respectively.

In the low-baseline scenario, the treatment risks are 0.075 and 0.15. The observed risks, using each group’s own confounder prevalence, are:

\begin{aligned} m_0&=0.80(0.05)+0.20(0.10)=0.06,\\ m_1&=0.40(0.075)+0.60(0.15)=0.12. \end{aligned}

So the observed RD is \(0.12-0.06=0.06\) and the observed RR is \(0.12/0.06=2.00\). Standardizing to \(g=0.40\) gives \(R_0^g=0.60(0.05)+0.40(0.10)=0.07\) and \(R_1^g=0.60(0.075)+0.40(0.15)=0.105\). The standardized RD is \(0.035\), or 3.5 percentage points, and the standardized RR is \(0.105/0.07=1.50\). The observed OR is \((0.12/0.88)/(0.06/0.94)=2.136\); the standardized OR is \((0.105/0.895)/(0.07/0.93)=1.559\).

In the higher-baseline scenario, treatment risks are 0.30 and 0.60. The observed risks are \(m_0=0.80(0.20)+0.20(0.40)=0.24\) and \(m_1=0.40(0.30)+0.60(0.60)=0.48\). The observed RD is \(0.24\), and the observed RR is \(0.48/0.24=2.00\). Standardized risks are \(R_0^g=0.60(0.20)+0.40(0.40)=0.28\) and \(R_1^g=0.60(0.30)+0.40(0.60)=0.42\), giving an RD of \(0.14\) and an RR of \(1.50\). The observed OR is \((0.48/0.52)/(0.24/0.76)=2.923\); the standardized OR is \((0.42/0.58)/(0.28/0.72)=1.861\).

The relative risks are the same across the two scenarios, both before and after standardization, because the assumed risk multipliers and confounder distributions are the same. But the RD bias is \(0.06-0.035=0.025\) in the low-baseline scenario and \(0.24-0.14=0.10\) in the higher-baseline scenario. The odds ratios also differ across baselines. Thus even when a relative-risk description is unchanged, the absolute bias and odds-scale contrast can change substantially.

Using Scales Carefully in Sensitivity Analysis

A useful analysis begins by naming the target effect measure and the population in which it is defined. Specify the causal risks being compared, the distribution to which they are standardized, and the assumptions about the unmeasured confounder. Then calculate the risks under each sensitivity scenario and transform those risks to the chosen scale. This order keeps the assumptions and the estimand distinct.

Define the target.
State whether the target is an RD, RR, or OR, and name the population or confounder distribution to which it applies.

Specify the sensitivity scenario.
Set plausible values for \(p_1\), \(p_0\), and the outcome risks \(q_{au}\), with any effect-structure assumptions stated explicitly.

Calculate and compare risks.
Compute the observed mixtures and the standardized risks. Derive the chosen effect measure from each pair of risks and interpret the difference on that scale.

When comparing sensitivity results, report the underlying risks as well as the effect measure where feasible. For example, an RR change from 1.5 to 1.2 does not state the change in outcome frequency without a baseline risk. Conversely, an RD may be more directly relevant to a decision about how many outcomes could be prevented or caused, but it does not by itself describe relative change.

Common Mistake

Wrong reasoning: “The adjusted OR is 2.0 and the marginal OR is 1.8, so the difference of 0.2 must be confounding bias.” This conclusion does not follow. As the randomized example shows, an OR can change between conditional and marginal comparisons solely because of noncollapsibility.

Correction: Identify whether the two ORs target the same population-level contrast. Compare risks standardized to the same target distribution, and separately assess whether treatment groups differ in the distribution of the unmeasured confounder. A difference between effect measures, a change in conditioning, and confounding are distinct issues. Also avoid comparing numerical sensitivity thresholds across RD, RR, and OR as if the units and interpretations were identical.

Key takeaway: Confounding sensitivity is scale-specific. The RD expresses absolute risk change, the RR expresses relative risk, and the OR expresses relative odds and is noncollapsible. Calculate each from clearly defined risks and a common target population; baseline risk and the chosen scale determine how bias is interpreted.

Check Your Understanding

Use the definitions and calculations in this tutorial to answer the following questions.

If \(R_0=0.20\) and \(R_1=0.30\), calculate the RD, RR, and OR. Which measure is in percentage points?
Under the constant-RD model for a binary \(U\), what is the additive bias when \(p_1=0.50\), \(p_0=0.25\), and \(\delta=0.12\)? State the assumptions needed for the formula.
Why does equal prevalence of a prognostic \(U\) across treatment groups not guarantee that the conditional and marginal ORs are equal?
In the two-baseline worked example, why did the RR bias comparison stay the same while the RD bias changed?
What additional information is needed to interpret an RR of 1.5 as an absolute difference in outcome risk?