Robustness Values for Explaining Away an Effect

What Does It Mean to Explain Away an Estimate?

An adjusted treatment-effect estimate can be statistically precise and still depend on an assumption: that no important confounder remains unmeasured. Sensitivity analysis makes this concern quantitative. For the question here, the target is specific: how strong would an omitted confounder have to be to move the fitted linear treatment coefficient all the way to zero?

This is different from asking what strength would make a confidence interval include zero. An estimate may lose statistical significance while remaining nonzero; explaining away the estimate means eliminating its point estimate. Neither calculation establishes that a hidden confounder exists. Each describes the strength it would need, under a stated model, to produce a specified change.

Model and Partial R-Squared Strengths

Consider an outcome \(Y\), a treatment or exposure \(D\), measured covariates \(X\), and an omitted variable or block of variables \(U\). The fitted model adjusts for \(X\), but not \(U\). Suppose the outcome is modeled linearly, the treatment coefficient is the target, and the same observations and covariate specification are used in the sensitivity calculation. The conventional OLS standard error and residual degrees of freedom are used. This is a sensitivity analysis of a linear coefficient; interpreting that coefficient causally also requires the relevant causal assumptions, including adequate control of confounding.

Definition: The partial \(R^2\) of \(D\) with \(U\) given \(X\), written \(R^2_{D\sim U\mid X}\), is the proportion of the variation in \(D\) remaining after adjustment for \(X\) that is explained by \(U\). The partial \(R^2\) of \(Y\) with \(U\) given \(D,X\), written \(R^2_{Y\sim U\mid D,X}\), is the proportion of the variation in \(Y\) remaining after adjustment for \(D\) and \(X\) that is explained by \(U\).

Both quantities range from zero to one. They describe conditional explanatory strength, not a causal effect of \(U\). For a block of omitted variables, each partial \(R^2\) refers to the block jointly. To bias the estimated coefficient toward zero, the omitted variable must also have a direction of association that produces bias opposite to the estimate. The strength measures alone do not determine that direction.

Let \(\hat{\tau}\) be the adjusted treatment coefficient, \(SE(\hat{\tau})\) its standard error, \(t=\hat{\tau}/SE(\hat{\tau})\) its t statistic, and \(df\) the residual degrees of freedom. The partial \(R^2\) of treatment with the outcome, conditional on \(X\), is \(t^2/(t^2+df)\). The robustness value for reducing the coefficient to zero asks for the smallest common value \(r\) such that a confounder with \(R^2_{D\sim U\mid X}=r\) and \(R^2_{Y\sim U\mid D,X}=r\) could produce bias as large as \(|\hat{\tau}|\), in the direction needed to eliminate it.

Formula and Derivation

Under the linear omitted-variable sensitivity calculation, the magnitude of the bias, expressed in units of the standard error of the adjusted coefficient, is bounded by

\frac{|\text{bias}|}{SE(\hat{\tau})} \leq \sqrt{ df\, \frac{R^2_{Y\sim U\mid D,X}\,R^2_{D\sim U\mid X}} {1-R^2_{D\sim U\mid X}} }.

The bound is attained when the omitted association has the direction that maximizes bias for these partial \(R^2\) values. To reduce the estimate to zero, the required bias magnitude equals \(|\hat{\tau}|\), so the left side must equal \(|t|\). If both partial \(R^2\) values equal \(r\), this condition becomes

|t|=\sqrt{df\,\frac{r^2}{1-r}}.

Set \(f=|t|/\sqrt{df}\). Squaring both sides and rearranging gives \(r^2+f^2r-f^2=0\). Its positive root is the relevant solution; the other root is negative and cannot be a partial \(R^2\). Therefore:

Definition: The robustness value for reducing the linear treatment-effect estimate to zero is

RV_0 = \frac{\sqrt{f^4+4f^2}-f^2}{2}, \qquad f=\frac{|t|}{\sqrt{df}}.

Equivalently, substitute \(f=|t|/\sqrt{df}\) directly into the formula. The result is a proportion: for example, \(RV_0=0.20\) means that equal partial \(R^2\) values of 0.20 with treatment and outcome could, in the biasing direction, reduce the adjusted coefficient to zero. It does not mean that the confounder has a 20% probability of existing or that 20% of the effect is known to be confounded.

Worked Calculations

Worked Example: Equal Strengths Needed to Eliminate a Positive Estimate

In a hypothetical observational analysis of 120 patients, an adjusted linear model estimates that treatment increases a symptom score by 0.60 points. The standard error is 0.20 points and the residual degrees of freedom are 100. How strong would an omitted confounder need to be, equally with treatment and outcome, to reduce the estimate to zero?

The observed statistic is \(t=0.60/0.20=3.00\). Thus \(f=3.00/\sqrt{100}=0.300\), and

RV_0 = \frac{\sqrt{0.300^4+4(0.300^2)}-0.300^2}{2} = \frac{\sqrt{0.3681}-0.0900}{2} = 0.258.

So the equal-strength benchmark is about 25.8%: an omitted variable would need partial \(R^2\) of approximately 0.258 with treatment given the measured covariates and 0.258 with symptom score given treatment and those covariates. It must also be associated in the direction that biases the positive coefficient downward. This is a substantial conditional strength in this model, but it is not proof that such confounding is absent.

Worked Example: Required Outcome Strength Given Treatment Strength

In a second hypothetical analysis, the estimated treatment coefficient is 1.20 recovery points, its standard error is 0.50 points, and \(df=80\). Suppose a proposed hidden confounder has \(R^2_{D\sim U\mid X}=0.20\). What partial \(R^2\) with the outcome would be needed to eliminate the estimate?

First calculate \(t=1.20/0.50=2.40\). Set the bias bound equal to \(|t|\), but keep the two strengths distinct:

2.40^2 = 80\, \frac{R^2_{Y\sim U\mid D,X}(0.20)}{1-0.20}.

Solving gives

R^2_{Y\sim U\mid D,X} = \frac{2.40^2(1-0.20)}{80(0.20)} = \frac{5.76(0.80)}{16} = 0.288.

Thus, if the confounder explains 20.0% of the residual treatment variation after \(X\), it would need to explain 28.8% of the outcome variation left after \(D\) and \(X\), with a suitable biasing direction, to reduce the coefficient to zero. The requirement is not that the two strengths always be equal; equality is just the convention used for \(RV_0\).

Worked Example: A Less Precise Estimate

A hypothetical model of treatment and hospital stay estimates a reduction of 1.20 days, with standard error 0.80 days and \(df=45\). Calculate the equal-strength robustness value.

Here \(t=-1.20/0.80=-1.50\), so \(f=1.50/\sqrt{45}\), and \(f^2=2.25/45=0.0500\). Therefore,

RV_0 = \frac{\sqrt{0.0500^2+4(0.0500)}-0.0500}{2} = \frac{\sqrt{0.2025}-0.0500}{2} = 0.200.

Equal partial \(R^2\) values of 20.0% could reduce this estimate to zero if the hidden confounding biases the negative coefficient upward. The units of the estimate are days; the partial \(R^2\) values are dimensionless. The result concerns eliminating the point estimate, not whether a confidence interval includes zero.

How to Use and Interpret the Result

A robustness value is most informative when reported alongside the coefficient, its uncertainty, and the adjustment set. It translates an estimate’s sensitivity into a conditional-strength scale. It does not identify an omitted variable, say whether a particular unmeasured factor is plausible, or remove the need for subject-matter reasoning about confounding.

Record the fitted model quantities.
Use the adjusted coefficient, its conventional standard error, and the residual degrees of freedom for the same model and analysis sample.

Calculate the observed t statistic.
Compute \(t=\hat{\tau}/SE(\hat{\tau})\), retaining its sign to understand the estimated direction; use \(|t|\) in the robustness-value formula.

Calculate and describe \(RV_0\).
Report the equal partial \(R^2\) benchmark and state that the omitted confounding must act in the direction that moves the estimate toward zero.

Assess relevance in context.
Compare the required conditional strengths with plausible knowledge about the covariates and potential confounders. The numerical value alone is not a verdict about robustness.

The calculation assumes the linear coefficient and partial \(R^2\) measures are suitable for the analysis. Model misspecification, measurement error, selection bias, or a different outcome scale are not automatically addressed by this robustness value. Nor does a value above a chosen benchmark certify that an estimate is causal. The benchmark only describes the strength required under the specified omitted-variable calculation.

Common Mistake

A common mistake is to report the robustness value as the amount of confounding that must exist, or to infer that the estimated causal effect is zero once the value is calculated. The value is a threshold in a sensitivity analysis: it says how strong an omitted factor would need to be under the model, not whether that factor exists. Another mistake is to use a robustness value for losing statistical significance as though it answered this question. A coefficient can remain nonzero after its confidence interval includes zero; here the target is specifically a zero point estimate.

Key takeaway: \(RV_0\) is the smallest equal pair of conditional partial \(R^2\) strengths—one linking the omitted confounder to treatment, the other linking it to outcome—that could explain away the fitted linear treatment coefficient in the biasing direction.

Check Your Understanding

Use the definitions and calculations above to answer these questions.

In a linear model, what does \(R^2_{D\sim U\mid X}\) measure, and what does \(R^2_{Y\sim U\mid D,X}\) measure?
Why does the robustness-value formula use \(|t|\) rather than the signed t statistic?
An omitted confounder has partial \(R^2=0.10\) with treatment given \(X\) and 0.30 with outcome given treatment and \(X\). Why is it not appropriate to summarize this pair as an equal-strength robustness value of 0.20?
Explain the difference between reducing a coefficient to zero and making its confidence interval include zero.
What additional condition about the direction of confounding is needed for an omitted variable to explain away a positive treatment estimate?