Delta-Method Interpretation for Nonidentity Gaussian Links

From Linear-Predictor Uncertainty to Mean Uncertainty

In a Gaussian regression with a nonidentity link, the fitted mean is obtained by transforming a linear predictor. Coefficient estimates make that predictor uncertain, and the inverse-link function determines how this uncertainty carries through to the mean. The delta method gives a local, first-order approximation to the resulting standard error.

This tutorial focuses on fitted means at a specified covariate vector. It does not describe the uncertainty of a new Gaussian response, which also includes the response’s residual variation. The uncertainty here is about the estimated mean itself.

Suppose an estimated covariance matrix for \(\hat\beta\) is \(\widehat V_\beta\). The estimated variance of the linear predictor at \(x_0\) is \(s_{\hat\eta_0}^2=x_0^\top\widehat V_\beta x_0\), so its standard error is \(s_{\hat\eta_0}\). For Gaussian regression with independent observations and precision weights \(w_i\), if \(\operatorname{Var}(Y_i)=\phi/w_i\), a common covariance estimate is \(\widehat V_\beta=\hat\phi(X^\top W X)^{-1}\), where \(W\) has diagonal entries \(w_i\). The dispersion estimate and coefficient covariance must be chosen consistently with the model and inferential procedure.

The derivative measures how sensitive the mean is to a small change in the predictor near its fitted value. A small predictor standard error does not by itself guarantee a small mean standard error: the inverse link may be steep at the fitted predictor. Conversely, a shallow part of the inverse link can reduce the propagated uncertainty.

Why the Delta-Method Formula Works

The approximation follows from a first-order Taylor expansion. The result below states the conditions under which this local calculation yields an asymptotic distribution, rather than merely a heuristic.

Proof. Differentiability at \(\eta\) means that, for \(\delta\) approaching zero, \(h(\eta+\delta)=h(\eta)+h'(\eta)\delta+r(\delta)\), where \(r(\delta)/\delta\to0\). Set \(\delta_n=\hat\eta_n-\eta\). Consistency gives \(\delta_n\xrightarrow{p}0\). The assumed asymptotic normality implies \(\sqrt n\delta_n=O_p(1)\), so $$ \sqrt n r(\delta_n) = \sqrt n\delta_n\,\frac{r(\delta_n)}{\delta_n} \xrightarrow{p}0. $$ (The expression is defined as zero when \(\delta_n=0\).) Therefore, $$ \sqrt n\{h(\hat\eta_n)-h(\eta)\} = h'(\eta)\sqrt n\delta_n+\sqrt n r(\delta_n). $$ The first term converges in distribution to a normal variable with variance \(\{h'(\eta)\}^2v\), and the second converges in probability to zero. Slutsky’s theorem gives the stated limit. \(\square\)

In regression, \(\hat\eta_0=x_0^\top\hat\beta\) is a scalar estimated predictor. Its estimated variance supplies \(s_{\hat\eta_0}^2\), and substituting the fitted predictor into the derivative gives the practical standard-error formula. The approximation is local: it is most informative when uncertainty in the predictor is small enough that the inverse link is nearly linear over the relevant range.

Log-Link Gaussian Means

For the log link, \(g(\mu)=\log(\mu)\) and \(h(\eta)=\exp(\eta)\). The mean must be positive. Since \(h'(\eta)=\exp(\eta)=h(\eta)\), the fitted mean’s estimated standard error is approximately the fitted mean multiplied by the predictor’s standard error.

Reciprocal-Link Gaussian Means

For the reciprocal link, \(g(\mu)=1/\mu\) and \(h(\eta)=1/\eta\). On the positive-mean branch, both \(\mu\) and \(\eta\) must be positive. Since \(h'(\eta)=-1/\eta^2\), the magnitude of the derivative is \(1/\eta^2=\mu^2\). The derivative’s negative sign indicates that the inverse link decreases: increasing the predictor decreases the fitted mean.

Constructing and Interpreting Intervals

A common first-order confidence interval for a fitted mean is the symmetric delta-method interval \(\hat\mu_0\pm c\,\widehat{\operatorname{SE}}(\hat\mu_0)\). Here \(c\) is a suitable critical value, such as the standard normal 95% value \(1.96\) for a large-sample approximation. Alternatively, form an interval for the linear predictor, \([\hat\eta_0-cs_{\hat\eta_0},\hat\eta_0+cs_{\hat\eta_0}]\), and transform its endpoints with \(h\), reversing them if \(h\) is decreasing. For small samples, the critical value and coefficient uncertainty should follow the applicable Gaussian regression inference; the delta method itself does not dictate a particular critical value.

Justification. Taylor’s theorem around \(\hat\eta_0\) gives, for either sign, $$ h(\hat\eta_0\pm cs_{\hat\eta_0}) = h(\hat\eta_0) \pm c h'(\hat\eta_0)s_{\hat\eta_0} +\frac{1}{2}h''(\xi_\pm)c^2s_{\hat\eta_0}^2, $$ where \(\xi_\pm\) lies between \(\hat\eta_0\) and \(\hat\eta_0\pm cs_{\hat\eta_0}\). Continuity of \(h''\) near \(\hat\eta_0\) makes the final term \(O(s_{\hat\eta_0}^2)\). The first-order endpoint displacement has magnitude \(c|h'(\hat\eta_0)|s_{\hat\eta_0}\), which is precisely the delta-method margin. Reordering endpoints when needed produces the stated comparison. \(\square\)

Common Mistake and Practical Checks

A frequent mistake is to compute a predictor interval and transform its endpoints without checking whether the whole interval lies in the inverse link’s domain. For a reciprocal link, an interval crossing zero cannot be transformed by taking reciprocals of its two endpoints: zero has no reciprocal, and the inverse link is not continuous across that boundary. Check the domain first. For a log link, a symmetric delta interval can also extend below zero even though every fitted mean under the link is positive.

The two interval constructions answer related but distinct practical needs. The symmetric delta interval is a local approximation on the response scale. The transformed interval reflects the shape of the inverse link and can preserve its domain when the predictor interval lies entirely within it. Neither procedure adds residual variation for a future observation; that would concern prediction rather than uncertainty in the fitted mean.