Delta-Method Uncertainty for Fitted Risks

Why a Fitted Risk Has Uncertainty

A logistic model gives a fitted probability for each covariate profile, but that probability is computed using estimated coefficients. If the coefficients would change across repeated samples, the fitted probability would change as well. The delta method translates uncertainty in the coefficient estimates into an approximate standard error for the fitted risk.

The calculation is specific to a fixed covariate profile. It describes uncertainty from estimating the model coefficients, conditional on the chosen covariate values. It does not include the additional randomness of whether a particular future person experiences the binary outcome.

The Fitted Risk and Its Gradient

Let \(\hat{\boldsymbol{\beta}}\) be the estimated coefficient vector of a logistic regression, and let \(\mathbf{x}_0\) be a fixed covariate vector, including the intercept component if the model has an intercept. The fitted linear predictor and risk at this profile are

Write \(V_{\hat{\boldsymbol{\beta}}}\) for an estimated covariance matrix of the coefficient estimator. The key derivative is the gradient of the fitted risk as a function of the coefficients.

To verify the derivative, set \(\eta=\mathbf{x}_0^{\mathsf T}\boldsymbol{\beta}\) and \(p=(1+\exp(-\eta))^{-1}\). Differentiation with respect to \(\eta\) gives \(dp/d\eta=p(1-p)\); the chain rule gives \(\nabla h=p(1-p)\mathbf{x}_0\). Therefore the plug-in standard error is

The quadratic form \(\mathbf{x}_0^{\mathsf T}V_{\hat{\boldsymbol{\beta}}}\mathbf{x}_0\) is the estimated variance of the fitted linear predictor at the profile. The inverse-logit derivative then scales that uncertainty by \(\hat{p}_0(1-\hat{p}_0)\). The full covariance matrix matters: coefficient covariances contribute to the quadratic form, not just the diagonal variances.

Delta-Method Result and Its Justification

The plug-in formula is an approximation justified by the asymptotic distribution of the coefficient estimator. The following version states the conditions directly. It applies, for example, to a fixed-dimensional, regular logistic regression when the coefficient estimator has the stated asymptotic normality and the estimated covariance matrix consistently estimates its asymptotic covariance on the same scale.

Proof. The inverse-logit function is continuously differentiable at every finite linear predictor. Its gradient at \(\boldsymbol{\beta}_0\) is \(\mathbf{a}=p_0(1-p_0)\mathbf{x}_0\). By differentiability, for \(\boldsymbol{\delta}\) tending to zero, \(h(\boldsymbol{\beta}_0+\boldsymbol{\delta})=h(\boldsymbol{\beta}_0)+\mathbf{a}^{\mathsf T}\boldsymbol{\delta}+r(\boldsymbol{\delta})\), where \(r(\boldsymbol{\delta})/\|\boldsymbol{\delta}\|\to0\). The assumed asymptotic normality implies \(\hat{\boldsymbol{\beta}}-\boldsymbol{\beta}_0=O_p(n^{-1/2})\), so the remainder satisfies \(\sqrt{n}r(\hat{\boldsymbol{\beta}}-\boldsymbol{\beta}_0)=o_p(1)\). Hence

The assumed limit distribution and Slutsky’s theorem give the stated normal limit, with variance \(\mathbf{a}^{\mathsf T}\Sigma\mathbf{a}\). Also, \(\hat{\boldsymbol{\beta}}\xrightarrow{p}\boldsymbol{\beta}_0\), so continuity gives \(\nabla h(\hat{\boldsymbol{\beta}})\xrightarrow{p}\mathbf{a}\). Together with \(nV_{\hat{\boldsymbol{\beta}}}\xrightarrow{p}\Sigma\), this implies

When this limit is positive, dividing the asymptotically normal error by its consistent standard-error estimate yields the stated standard normal limit. \(\square\)

A corresponding approximate two-sided confidence interval on the probability scale is \(\hat{p}_0\pm z_{1-\alpha/2}\widehat{\operatorname{SE}}(\hat{p}_0)\), where \(z_{1-\alpha/2}\) is the \(1-\alpha/2\) standard normal quantile. This follows by using the studentized normal approximation. It is an approximation, not an exact finite-sample guarantee.

Confidence Intervals and Their Limits

A probability-scale Wald interval is easy to calculate and interpret, but its endpoints are not guaranteed to fall between zero and one. This can happen when the fitted risk is close to a boundary or the standard error is large. A bounded alternative uses the linear predictor: form an interval for \(\eta_0=\mathbf{x}_0^{\mathsf T}\boldsymbol{\beta}\), then apply the inverse-logit to each endpoint. This preserves the probability range, though it is a different interval construction; it does not change the delta-method standard error.

What the Uncertainty Does—and Does Not—Measure

The covariance matrix may come from the model-based inverse information matrix or from another appropriate covariance estimator. The delta-method propagation uses the covariance estimate supplied; it does not decide whether that covariance estimate is suitable for the sampling design or model assumptions. Whatever the source, it must refer to the same coefficient vector and parameterization used to calculate the fitted risk.

The standard error concerns the estimated mean probability at the specified profile. It is not the standard deviation of a future Bernoulli response. If \(Y_{\mathrm{new}}\) is a binary outcome with success probability \(p_0\), its conditional variance is \(p_0(1-p_0)\); that outcome variability is distinct from the sampling uncertainty in \(\hat{p}_0\). The fitted-risk standard error instead quantifies how the estimated model probability would vary across repeated samples.

Common mistake: Treating the coefficient standard errors separately, multiplying each one by the corresponding profile value, and combining them while ignoring coefficient covariances. The correction is to use the complete quadratic form \(\mathbf{x}_0^{\mathsf T}V_{\hat{\boldsymbol{\beta}}}\mathbf{x}_0\), then multiply its square root by \(\hat{p}_0(1-\hat{p}_0)\). Also avoid interpreting the resulting standard error as outcome-level randomness: it is uncertainty in the fitted risk at a fixed profile.