Uncertainty for Differences in Predicted Risks

Why a Difference Needs Joint Uncertainty

A logistic model can predict risk for two covariate profiles, such as a treated and an untreated profile. Their difference is often easier to interpret as an absolute change in probability than a difference in odds. But the uncertainty in that contrast is not generally the sum of two separate prediction variances: both predictions use the same estimated coefficients, so they are dependent.

The appropriate calculation treats the risk difference as one function of the coefficient vector. Its gradient describes how the contrast changes when the coefficients change, and the full coefficient covariance matrix carries the dependence into the resulting variance.

Definition and Delta-Method Variance

For a profile with linear predictor \(x^T\beta\), the derivative of its risk with respect to \(\beta\) is \(p(1-p)x\), where \(p=\operatorname{expit}(x^T\beta)\). Therefore the gradient of the risk difference, evaluated at \(\hat\beta\), is

Let \(\widehat V\) be the estimated covariance matrix of \(\hat\beta\), using the convention that it already includes the sample-size scaling. The delta-method variance estimate is

The design vectors include all model terms used for prediction: an intercept, covariates, indicator variables, and any interaction terms. The two vectors need not differ in only one entry.

Why the Formula Works

The relevant delta-method result is a multivariate version of first-order Taylor approximation. Suppose the true coefficient vector is \(\beta_0\), and, under the model’s usual regularity conditions, \(\sqrt n(\hat\beta-\beta_0)\xrightarrow{d}N(0,\Sigma)\). For logistic maximum likelihood, this asymptotic normality applies when the model is correctly specified, the true parameter is an interior point, the information is nonsingular, and standard likelihood regularity conditions hold. Let \(h(\beta)\) be differentiable at \(\beta_0\), with gradient \(g_0\).

Proof. Differentiability at \(\beta_0\) means that for \(u\) near zero, \(h(\beta_0+u)-h(\beta_0)=g_0^Tu+r(u)\), where \(r(u)/\lVert u\rVert\to0\) as \(u\to0\). Asymptotic normality implies \(\hat\beta-\beta_0=O_p(n^{-1/2})\), hence \(\hat\beta\xrightarrow{p}\beta_0\). It follows from the remainder condition that \(\sqrt n\,r(\hat\beta-\beta_0)=o_p(1)\). Thus

The first term converges in distribution to a mean-zero normal random variable with variance \(g_0^T\Sigma g_0\), because it is a linear transformation of the limiting multivariate normal vector. Slutsky’s theorem then gives the stated limit. \(\square\)

For the risk difference, take \(h(\beta)=\operatorname{expit}(x_A^T\beta)-\operatorname{expit}(x_B^T\beta)\). This function is differentiable, and its gradient is the difference of the two profile gradients displayed above. Replacing the unknown gradient and covariance with their fitted estimates gives the stated variance estimate. When \(\widehat V\) consistently estimates the coefficient covariance, this plug-in estimate is the corresponding first-order asymptotic variance.

The Covariance Between Predictions

The same result can be expressed in terms of the two fitted risks separately. Let \(g_A=p_A(1-p_A)x_A\) and \(g_B=p_B(1-p_B)x_B\), evaluated at the fitted coefficients when estimating. Their approximate variances and covariance are

Subtracting predictions subtracts their covariance twice. This is an algebraic identity, not an optional adjustment:

In matrix form, \((g_A-g_B)^TV(g_A-g_B)\) expands to \(g_A^TVg_A+g_B^TVg_B-2g_A^TVg_B\), which is exactly the same calculation. In general, the covariance is not zero just because the profiles are different: both fitted risks depend on \(\hat\beta\).

Worked Calculations

Interpretation and a Common Pitfall

For a fixed pair of profiles, the result estimates the difference between their model-fitted risks. The point estimate is measured in probability units; multiplying by 100 expresses it in percentage points. The Wald interval uses the same contrast and its estimated standard error. Because it is based on a normal approximation, its endpoints can sometimes fall outside the possible risk-difference range \([-1,1]\). Such endpoints signal a limitation of the approximation; they do not change the fitted risks or make a probability difference unbounded.

Common mistake: treating the predictions as independent. A tempting calculation is to add the two prediction variances and use that as the contrast variance. It omits the covariance term. For example, in the first worked example the covariance between the fitted risks is positive, so subtracting twice that covariance makes the difference more precise than the independence calculation would suggest. In other settings the covariance could be negative, in which case accounting for it increases the contrast variance. The correction is to calculate \(\hat g^T\widehat V\hat g\) directly, or to include the covariance term explicitly.

A second practical check is to keep the sign of the contrast consistent. If the definition is \(\hat p_A-\hat p_B\), reverse the sign of both the estimate and its gradient when reversing the profile order; the variance and standard error remain unchanged. Reporting the profile order and the risk-difference scale makes the result interpretable.