Confidence Intervals for Response-Scale Effects

Why Response-Scale Intervals Need Care

A generalized linear model (GLM) commonly estimates a mean on a link scale but communicates predictions on the response scale. For a binary outcome, for example, the linear predictor is a log-odds value while the prediction of practical interest is a probability. For a count outcome with a log link, the linear predictor is a log mean while the prediction is an expected count or rate. An interval for the linear predictor is therefore not yet an interval in the units a reader needs.

There are two standard approaches for a confidence interval for a fitted mean. One first forms an interval for the linear predictor and transforms both endpoints through the inverse link. The other uses the delta method to approximate the standard error of the fitted mean directly on the response scale, then forms a symmetric Wald interval there. These approaches are related, but they need not give the same endpoints.

Definition: For a GLM, let \(\eta\) be the linear predictor at a specified covariate pattern, let \(h=g^{-1}\) be the inverse link, and let \(\mu=h(\eta)\) be the corresponding fitted mean on the response scale. A response-scale confidence interval is an interval for \(\mu\), not for \(\eta\). Unless stated otherwise, intervals in this tutorial are pointwise intervals for a fitted mean at one specified covariate pattern.

A confidence interval for a fitted mean describes uncertainty due to estimating model parameters. It is not, by itself, a prediction interval for a future individual outcome. A future binary outcome can be 0 or 1 even when its fitted probability is near the middle of the scale; a future count can also vary around its fitted mean. That outcome variation is a separate source of uncertainty.

Transforming a Link-Scale Interval

Suppose a fitted GLM gives the estimated linear predictor \(\hat{\eta}\), with estimated standard error \(SE(\hat{\eta})\). Under the usual large-sample Wald approximation, a two-sided \(100(1-\alpha)\%\) link-scale interval is

\left[ \hat{\eta}-z_{1-\alpha/2}SE(\hat{\eta}), \ \hat{\eta}+z_{1-\alpha/2}SE(\hat{\eta}) \right].

Here \(z_{1-\alpha/2}\) is the corresponding standard normal quantile. For example, \(z_{0.975}=1.96\) gives a 95% interval. To obtain an interval for the fitted mean, apply the inverse link \(h\) to each endpoint. For the common logit and log links, \(h\) is increasing, so the order of the endpoints is unchanged.

\left[ h\!\left(\hat{\eta}-z_{1-\alpha/2}SE(\hat{\eta})\right), \ h\!\left(\hat{\eta}+z_{1-\alpha/2}SE(\hat{\eta})\right) \right].

For logistic regression, \(h(\eta)=1/(1+\exp(-\eta))\), so the resulting interval endpoints are probabilities between 0 and 1. For a log-link model, \(h(\eta)=\exp(\eta)\), so the endpoints are positive means. This method retains the asymmetry produced by a nonlinear inverse link.

Worked Example: A Probability Interval From a Logit Model

In a hypothetical binary-outcome model with a logit link, the fitted probability at a specified covariate pattern is \(0.40\), and the standard error of the estimated linear predictor is \(0.20\). Construct a 95% response-scale confidence interval using \(z_{0.975}=1.96\). Round the endpoints to four decimal places.

Method. Find the fitted linear predictor, form its Wald interval, and transform each endpoint using the inverse logit. This method avoids treating a probability interval as symmetric when the logit transformation is nonlinear.

The fitted probability is \(0.40\), so the estimated linear predictor is

\hat{\eta}=\log\left(\frac{0.40}{1-0.40}\right) =\log\left(\frac{2}{3}\right) =-0.405465.

The 95% margin on the linear-predictor scale is \(1.96(0.20)=0.392\). Thus the endpoints are \(-0.405465-0.392=-0.797465\) and \(-0.405465+0.392=-0.013465\). Transform each with the inverse logit:

\begin{aligned} h(-0.797465)&=\frac{1}{1+\exp(0.797465)}\approx 0.310568,\\ h(-0.013465)&=\frac{1}{1+\exp(0.013465)}\approx 0.496634. \end{aligned}

The 95% response-scale interval is therefore \([0.3106,\ 0.4966]\), rounded to four decimal places. It contains the fitted probability \(0.40\), and both endpoints are valid probabilities.

Common mistake and correction. Rounding the lower endpoint to \(0.3105\) is incorrect: the inverse-logit calculation gives approximately \(0.310568\), which rounds to \(0.3106\) at four decimal places. Also, adding and subtracting \(1.96(0.20)\) directly from \(0.40\) would mix a standard error on the logit scale with a probability on the response scale.

Why Transforming the Endpoints Works

The coverage argument relies on a simple property of a strictly increasing function. If the unknown linear predictor \(\eta\) lies between \(L\) and \(U\), then applying an increasing inverse link preserves the ordering: \(h(L)\leq h(\eta)\leq h(U)\). In particular, the events “\(\eta\) is in the link-scale interval” and “\(h(\eta)\) is in the transformed interval” are equivalent.

L\leq \eta\leq U \quad\Longleftrightarrow\quad h(L)\leq h(\eta)\leq h(U), \qquad \text{when }h\text{ is strictly increasing}.

Consequently, transforming the endpoints preserves the coverage probability of the link-scale interval. In practice, the Wald interval for \(\eta\) usually has approximate, rather than exact, coverage because its standard error and normal approximation are estimated. The transformed interval inherits that approximate coverage; the transformation does not repair a poor normal approximation or an unreliable model fit.

This reasoning applies directly to the usual increasing inverse links such as the inverse logit and exponential. If a transformation is decreasing, endpoint order must be reversed. The GLM examples here use increasing inverse links.

The Delta Method for a Predicted Mean

The delta method gives a local approximation to the uncertainty in a smooth function of an estimate. If \(\hat{\eta}\) is approximately normal with standard error \(SE(\hat{\eta})\), and the fitted response mean is \(\hat{\mu}=h(\hat{\eta})\), a first-order Taylor expansion gives

h(\hat{\eta})\approx h(\eta)+h'(\eta)(\hat{\eta}-\eta).

The constant term has no variance. Multiplying a random error by \(h'(\eta)\) multiplies its variance by \(h'(\eta)^2\). Replacing the unknown \(\eta\) with its estimate gives the delta-method standard error and Wald interval:

SE_{\Delta}(\hat{\mu}) \approx |h'(\hat{\eta})|SE(\hat{\eta}), \qquad \hat{\mu}\ \pm\ z_{1-\alpha/2}SE_{\Delta}(\hat{\mu}).

The absolute value ensures a nonnegative standard error. For the increasing inverse links used here, \(h'(\hat{\eta})\) is positive, so the absolute value does not change the calculation. The delta-method interval is symmetric around \(\hat{\mu}\) on the response scale. Because it is an approximation, its endpoints can sometimes fall outside the natural range of the response mean.

For a fitted linear predictor \(\hat{\eta}=\mathbf{x}^{\mathsf T}\hat{\boldsymbol{\beta}}\), its estimated variance is calculated from the coefficient covariance matrix:

\widehat{\operatorname{Var}}(\hat{\eta}) =\mathbf{x}^{\mathsf T}\widehat{\operatorname{Var}}(\hat{\boldsymbol{\beta}})\mathbf{x}, \qquad SE(\hat{\eta})= \sqrt{\mathbf{x}^{\mathsf T}\widehat{\operatorname{Var}}(\hat{\boldsymbol{\beta}})\mathbf{x}}.

Here \(\mathbf{x}\) is the design vector for the covariate pattern at which the mean is being estimated. This calculation includes covariance between coefficient estimates; adding coefficient standard errors as if they were independent is generally not valid.

Worked Example: Comparing Two Intervals for a Poisson Mean

A hypothetical Poisson GLM with a log link gives a fitted mean of \(12\) events for a specified exposure and covariate pattern. The standard error of its estimated log mean is \(0.15\). Treat the exposure as fixed and use \(z_{0.975}=1.96\). Find the transformed 95% interval and the delta-method interval. Round the final endpoints to two decimal places.

Method. For the transformed interval, construct the interval for the log mean and exponentiate its endpoints. For the delta method, use \(h(\eta)=\exp(\eta)\) and \(h'(\eta)=\exp(\eta)\). The comparison shows the effect of retaining versus locally approximating the curvature of the exponential function.

The estimated log mean is \(\hat{\eta}=\log(12)\approx 2.484907\). The link-scale margin is \(1.96(0.15)=0.294\), so the log-scale endpoints are \(2.484907-0.294=2.190907\) and \(2.484907+0.294=2.778907\). Exponentiating gives

\begin{aligned} \exp(2.190907)&=12\exp(-0.294)\approx 8.943,\\ \exp(2.778907)&=12\exp(0.294)\approx 16.101. \end{aligned}

The transformed interval is \([8.94,\ 16.10]\). The fitted mean of \(12\) is between the endpoints, and both endpoints are positive, as required for a log-link mean.

For the delta method, the derivative of the exponential function at \(\hat{\eta}\) is \(\exp(\hat{\eta})=12\). Therefore \(SE_{\Delta}(\hat{\mu})\approx12(0.15)=1.8\). The response-scale margin is \(1.96(1.8)=3.528\), giving

[12-3.528,\ 12+3.528]=[8.472,\ 15.528] \approx [8.47,\ 15.53].

The delta-method interval is narrower on its upper side here and symmetric about \(12\), whereas the transformed interval is asymmetric. Both use the same estimated log-scale uncertainty, but the delta method replaces the curved exponential transformation with a local linear approximation. This interval comparison concerns the fitted mean, not the range of possible future counts.

Common mistake avoided. The response-scale standard error is not \(0.15\); that value is on the log scale. The derivative multiplies it by \(12\), yielding the approximate response-scale standard error \(1.8\).

Worked Example: When a Delta-Method Probability Interval Is Misleading

A hypothetical logistic model gives a fitted probability of \(0.08\) and a standard error of \(0.35\) for the estimated logit. Construct a 95% transformed interval and a 95% delta-method interval, using \(z_{0.975}=1.96\). Round the final endpoints to four decimal places.

Method. Calculate both intervals from the same logit estimate and standard error. This illustrates how a symmetric response-scale approximation can behave differently near the boundary of the probability scale.

The estimated logit and its interval margin are

\hat{\eta}=\log\left(\frac{0.08}{0.92}\right) =\log\left(\frac{2}{23}\right)\approx -2.442347, \qquad 1.96(0.35)=0.686.

The logit endpoints are \(-2.442347-0.686=-3.128347\) and \(-2.442347+0.686=-1.756347\). Applying the inverse logit gives approximately \(0.0419\) and \(0.1473\). Thus the transformed 95% interval is \([0.0419,\ 0.1473]\).

For logistic regression, \(h'(\eta)=h(\eta)\{1-h(\eta)\}\). At the fitted probability \(0.08\), the delta-method standard error is

SE_{\Delta}(\hat{\mu}) \approx 0.08(1-0.08)(0.35) =0.08(0.92)(0.35) =0.02576.

The response-scale margin is \(1.96(0.02576)=0.0504896\). The delta-method interval is therefore \([0.08-0.0504896,\ 0.08+0.0504896]=[0.0295104,\ 0.1304896]\), or \([0.0295,\ 0.1305]\) to four decimal places. Both endpoints happen to be valid probabilities, but the two methods give visibly different intervals.

Common mistake avoided. A delta-method interval for a probability is not guaranteed to stay between 0 and 1. Its being inside the probability range in this example does not make the symmetric approximation equivalent to endpoint transformation. Near a boundary, the transformed interval often better reflects the scale’s asymmetry.

Choosing and Reporting the Interval

For a fitted mean with a nonlinear link, transforming a link-scale Wald interval is often a clear default: it respects the response range for the logit and log links and preserves the asymmetry implied by the transformation. The delta method is useful when a local response-scale approximation is required or when the quantity of interest is a smooth function of model estimates. The choice and the approximation should be reported rather than hidden.

Feature	Transformed link-scale interval	Delta-method interval
Construction	Transform the lower and upper link-scale endpoints	Use the derivative to estimate response-scale standard error, then use a symmetric Wald interval
Shape	May be asymmetric around the fitted mean	Symmetric around the fitted mean
Natural response range	Respects the range when the inverse link maps into that range	May extend outside the response range
Approximation	Inherits the link-scale interval's approximation	Also uses a first-order Taylor approximation

A precise report identifies the response quantity, covariate pattern, confidence level, and interval method. For example, say “the estimated event mean at this exposure is 12, with a transformed 95% interval from 8.94 to 16.10,” rather than simply reporting “the 95% interval is 8.94 to 16.10.” This matters because a reported probability, expected count, or rate must be distinguished from the link-scale coefficient and from a future individual outcome.

Use the standard error for the estimated linear predictor when constructing the link-scale interval.
Transform both endpoints; do not transform only the point estimate or apply a link-scale standard error directly to the response mean.
For the delta method, use the derivative of the inverse link evaluated at the fitted linear predictor.
Check whether response-scale delta-method endpoints respect the natural parameter range.
Describe a fitted-mean interval as uncertainty in the estimated mean, not as a prediction interval for a future observation.

Key takeaway: Endpoint transformation carries a link-scale interval through an increasing inverse link; the delta method instead linearizes that inverse link near the fitted value. Both can be useful, but they answer through different approximations and need not produce the same response-scale interval.

Careful construction and interpretation of GLM uncertainty intervals is aligned with the SOA Exam PA (Predictive Analytics) syllabus.

Check Your Understanding

For each problem, identify the interval method, keep the scale of each standard error straight, and check the final endpoints.

A hypothetical logistic model has fitted probability \(0.25\) and logit standard error \(0.10\). Find the 90% transformed interval using \(z_{0.95}=1.645\). Round to four decimal places. (Answer: approximately \([0.2204,\ 0.2821]\))
A hypothetical log-link model has fitted mean \(5\) and log-mean standard error \(0.08\). Find the 95% transformed interval using \(z_{0.975}=1.96\). Round to two decimal places. (Answer: approximately \([4.27,\ 5.85]\))
A logistic model has fitted probability \(0.60\) and logit standard error \(0.25\). Use the delta method to find a 95% response-scale Wald interval with \(z_{0.975}=1.96\). Round to four decimal places and state whether the endpoints are valid probabilities. (Answer: \([0.4824,\ 0.7176]\); both are valid probabilities.)
A hypothetical Poisson log-link model has fitted mean \(3.2\) and log-mean standard error \(0.30\). Compute both a 90% transformed interval and a 90% delta-method interval using \(z_{0.95}=1.645\). Round to two decimal places. (Answer: transformed \([1.95,\ 5.24]\); delta method \([1.62,\ 4.78]\).)

What You'll Learn