Nonlinear Confidence Intervals With Quasi-Likelihood

Why Nonlinear Confidence Intervals Need a Gradient

A confidence interval for a regression coefficient does not directly answer every scientific question. Researchers may want a fitted mean, a ratio of two fitted means, or a probability at a specified covariate pattern. These quantities are usually nonlinear functions of the coefficient vector. Their uncertainty depends both on the uncertainty in the coefficients and on how rapidly the function changes near the estimated coefficients.

In quasi-likelihood regression, coefficient uncertainty is represented by a sandwich covariance estimate. The delta method carries that uncertainty through a differentiable function. The result is a large-sample confidence interval, not an exact finite-sample interval. In particular, the interval's standard error must use the full covariance matrix, including its off-diagonal entries when they affect the function.

The gradient is a column vector of partial derivatives. It describes the local sensitivity of \(g\) to changes in each coefficient. The sandwich covariance summarizes the joint uncertainty of those coefficients. Together, they determine the uncertainty in the target quantity.

The Delta-Method Wald Interval

Suppose the coefficient estimator satisfies the asymptotic normality established for quasi-estimators: for a fixed-dimensional parameter, \(\sqrt{n}(\hat{\beta}-\beta_0)\xrightarrow{d}N(0,V)\), and suppose the sandwich covariance estimate satisfies \(n\hat{\Sigma}\xrightarrow{p}V\), where \(V\) is positive definite. Let \(g\) be continuously differentiable in a neighborhood of \(\beta_0\), with \(\nabla g(\beta_0)\ne 0\). The nonzero-gradient condition ensures that the first-order approximation has positive variance.

Proof. Differentiability at \(\beta_0\) gives the first-order expansion $$ g(\hat{\beta})-g(\beta_0) = \nabla g(\beta_0)^\mathsf{T}(\hat{\beta}-\beta_0) + o_p(\|\hat{\beta}-\beta_0\|). $$ Asymptotic normality implies \(\hat{\beta}-\beta_0=O_p(n^{-1/2})\), so the remainder is \(o_p(n^{-1/2})\). Therefore, by the multivariate central limit theorem and the continuous mapping theorem, $$ \sqrt{n}\{g(\hat{\beta})-g(\beta_0)\} \xrightarrow{d} N\!\left(0,\nabla g(\beta_0)^\mathsf{T}V\nabla g(\beta_0)\right). $$ Consistency of \(\hat{\beta}\), continuity of the gradient, and \(n\hat{\Sigma}\xrightarrow{p}V\) imply $$ n\nabla g(\hat{\beta})^\mathsf{T}\hat{\Sigma}\nabla g(\hat{\beta}) \xrightarrow{p} \nabla g(\beta_0)^\mathsf{T}V\nabla g(\beta_0). $$ The limit is positive because \(V\) is positive definite and the gradient is nonzero. Slutsky's theorem now gives the stated standard normal limit. The interval follows by inverting this asymptotic pivot. \(\square\)

The formula uses the estimated covariance of \(\hat{\beta}\), not the covariance of \(\sqrt{n}\hat{\beta}\). If software instead reports an estimate of \(V\), the corresponding covariance scale is \(V/n\). Confusing these scales changes the standard error by a factor of \(\sqrt{n}\).

Transformed Means and Link-Scale Intervals

For a generalized linear model, write the linear predictor at covariate vector \(x\) as \(\eta=x^\mathsf{T}\beta\), and let the mean be \(\mu=h^{-1}(\eta)\), where \(h\) is the link function. The fitted mean is a nonlinear function of \(\beta\). If \(h^{-1}\) is differentiable, the gradient is $$ \nabla_\beta h^{-1}(x^\mathsf{T}\beta) = \frac{d h^{-1}(\eta)}{d\eta}\,x. $$ Thus, for covariance estimate \(\hat{\Sigma}\), the delta-method variance estimate of the fitted mean is $$ \left\{\frac{d h^{-1}(\hat{\eta})}{d\eta}\right\}^{2} x^\mathsf{T}\hat{\Sigma}x, \qquad \hat{\eta}=x^\mathsf{T}\hat{\beta}. $$

When the inverse link is positive and monotone, another useful procedure is to first form a Wald interval for \(\eta\), then transform both endpoints using \(h^{-1}\). For a log link, \(h^{-1}(\eta)=\exp(\eta)\), so the resulting interval is positive and generally asymmetric. It is not the same finite-sample interval as the symmetric delta-method interval for the mean, although both have the same first-order asymptotic coverage under the stated regularity conditions.

Ratios of Means

Ratios are common targets in clinical and epidemiological analyses. With a log link, a ratio of fitted means simplifies: for covariate patterns \(x_1\) and \(x_0\), $$ \frac{\mu(x_1)}{\mu(x_0)} = \exp\{(x_1-x_0)^\mathsf{T}\beta\}. $$ Let \(d=x_1-x_0\). The log ratio is the linear contrast \(d^\mathsf{T}\beta\), whose estimated variance is \(d^\mathsf{T}\hat{\Sigma}d\). A confidence interval on the log-ratio scale can therefore be exponentiated to obtain a positive interval for the ratio. Its endpoints are usually not symmetric around the estimated ratio.

Other Nonlinear Targets

The same calculation applies to any smooth scalar target. For a logistic mean \(p(\beta)=\{1+\exp(-x^\mathsf{T}\beta)\}^{-1}\), write \(\hat{p}=p(\hat{\beta})\). Differentiating gives $$ \nabla_\beta p(\hat{\beta})=\hat{p}(1-\hat{p})x. $$ Thus its delta-method variance estimate is \(\{\hat{p}(1-\hat{p})\}^{2}x^\mathsf{T}\hat{\Sigma}x\). Alternatively, a linear-predictor interval can be transformed by the logistic function, whose values lie between zero and one.

Why the Gradient and Covariance Both Matter

The delta method is a local approximation: it uses the slope of the target function at the estimated coefficients. This makes it broadly useful, but also identifies circumstances in which extra caution is warranted. If the gradient is nearly zero, the first-order variance may be tiny even when second-order curvature matters. If the estimated coefficients are imprecise and the function changes sharply across their plausible range, a symmetric Wald interval can be a poor approximation. A transformation-based interval can help respect a parameter's range, but it does not remove the need for a reliable large-sample approximation.

A useful general derivation for a contrast transformed by a smooth scalar function makes the role of the gradient explicit. Let \(q=a^\mathsf{T}\beta\), where \(a\) is fixed, and let the target be \(r(\beta)=k(q)\). The chain rule gives $$ \nabla r(\beta)=k'(q)a, \qquad \widehat{\operatorname{Var}}\{r(\hat{\beta})\} = \{k'(a^\mathsf{T}\hat{\beta})\}^{2}a^\mathsf{T}\hat{\Sigma}a. $$ For the ratio \(k(q)=\exp(q)\), the variance estimate is \(\exp(2a^\mathsf{T}\hat{\beta})a^\mathsf{T}\hat{\Sigma}a\). For the log of a positive fitted mean, the target is instead the linear predictor itself, so its variance is \(x^\mathsf{T}\hat{\Sigma}x\). These are related calculations, but they answer different questions and use different scales.

Common Mistake

A frequent error is to report the standard error of the linear predictor as the standard error of a transformed mean. For example, if \(\hat{\eta}=0.8\) with standard error \(\sqrt{0.17}\), reporting \(0.4123\) as the standard error of \(\exp(\hat{\eta})\) omits the derivative of the exponential. The correction is to multiply by \(\exp(0.8)\), giving approximately \(0.9176\). More generally, the standard error of \(g(\hat{\beta})\) is not the standard error of \(\hat{\beta}\); it is determined by the gradient and the full covariance matrix.

A related mistake is to treat a transformed interval as symmetric around the transformed estimate. If a log-scale interval is \((L,U)\), the corresponding ratio interval is \((\exp(L),\exp(U))\), not the estimate plus or minus a single symmetric margin on the ratio scale. Finally, using only the diagonal variances of coefficients is incorrect whenever the gradient has multiple nonzero components and the covariance matrix has relevant off-diagonal terms.