Profile Intervals for Fitted Means at Fixed Covariates

Why Profile a Fitted Mean?

A generalized linear model (GLM) describes how a response distribution changes with covariates. At a specified covariate vector, its fitted mean is a particular function of the regression coefficients. An interval for that mean must account for uncertainty in the whole coefficient vector, including coefficients that are not themselves the target.

Profiling does this by fixing a candidate mean and maximizing the likelihood over all coefficient vectors that produce it. The resulting likelihood-ratio interval is not generally symmetric around the fitted mean: the likelihood can fall at different rates on either side. The target here is the mean response at fixed covariates, not a contrast between covariate values and not a prediction interval for a future response.

Definition and Likelihood-Ratio Construction

Let $Y_1,\ldots,Y_n$ have a GLM likelihood $L(\beta)$, where $\beta$ is a $p$-dimensional coefficient vector. Let $x_0$ be a fixed covariate vector, including the intercept component if the model has an intercept. Write the linear predictor as $\eta_0=x_0^\mathsf{T}\beta$, and let $g$ be the link function, so the mean at $x_0$ is $\mu_0(\beta)=g^{-1}(x_0^\mathsf{T}\beta)$. Assume the link is one-to-one on the parameter range under consideration.

Definition: For a candidate mean $m$, the profile likelihood for the fitted mean at $x_0$ is $$ L_p(m)=\sup_{\beta:\,g^{-1}(x_0^\mathsf{T}\beta)=m}L(\beta). $$ Equivalently, the constraint is $x_0^\mathsf{T}\beta=g(m)$. The profile log-likelihood is $\ell_p(m)=\log L_p(m)$, and the likelihood-ratio statistic is $$ W(m)=2\{\ell(\hat\beta)-\ell_p(m)\}, $$ where $\hat\beta$ maximizes the full likelihood.

The constraint fixes one scalar function of the coefficients; it does not fix every coefficient. The remaining freedom is precisely where nuisance parameters are profiled out. If $x_0\ne 0$, choose a vector $v$ satisfying $x_0^\mathsf{T}v=1$, and let the columns of a matrix $B$ span the null space of $x_0^\mathsf{T}$. Every coefficient vector satisfying the constraint can then be written as

\beta=v\,g(m)+B\gamma,

where $\gamma$ varies over the nuisance coordinates. Maximizing $\ell(vg(m)+B\gamma)$ over $\gamma$ gives the profile log-likelihood. This parameterization also makes clear why simply plugging in $\hat\beta$ and ignoring the other coefficient directions does not produce a profile likelihood.

Theorem: Suppose the model is identifiable and regular, the true coefficient vector is an interior point, the scalar target $g^{-1}(x_0^\mathsf{T}\beta)$ is differentiable with nonzero gradient, and the usual likelihood regularity conditions for Wilks’ theorem hold. Then, under the true model, $$ W(\mu_0(\beta))\xrightarrow{d}\chi^2_1. $$ Thus an approximate $100(1-\alpha)\%$ profile likelihood confidence set is the set of feasible $m$ satisfying $W(m)\leq \chi^2_{1,1-\alpha}$.

The one degree of freedom comes from fixing one scalar target, even when $\beta$ has many components. For a 95% interval, the usual large-sample cutoff is $\chi^2_{1,0.95}\approx 3.841$. This is an asymptotic calibration; it is not an exact finite-sample coverage guarantee.

Worked Examples

Worked Example: A Logistic Mean with a Nuisance Slope

Suppose independent grouped binomial observations are recorded at $x=0$ and $x=1$. At $x=0$, there are 8 successes in 20 trials; at $x=1$, there are 3 successes in 10 trials. Fit a logistic regression $\operatorname{logit}(p(x))=\beta_0+\beta_1x$, and form a 95% profile interval for $p(0)$.

Set $p=p(0)$. Under the constraint $\beta_0=\operatorname{logit}(p)$, the slope remains free. At $x=1$, its free value lets the fitted probability vary independently over $(0,1)$. The likelihood is maximized there at $3/10=0.3$, regardless of the candidate $p$. Consequently, the $x=1$ likelihood contribution is constant in the profile, and the profile likelihood-ratio statistic reduces to the binomial contribution at $x=0$.

The unrestricted estimate at $x=0$ is $\hat p=8/20=0.4$. Apart from binomial coefficients, the log-likelihood contribution at that covariate value is $8\log p+12\log(1-p)$. Therefore

W(p)=2\left\{8\log\left(\frac{0.4}{p}\right)+12\log\left(\frac{0.6}{1-p}\right)\right\}.

For each endpoint, solve $W(p)=3.841$, restricting to $0<p<1$. Numerical root-finding gives $p_L\approx0.2071$ and $p_U\approx0.6171$. Checking the upper value, for example, gives

2\left\{8\log\left(\frac{0.4}{0.6171}\right)+12\log\left(\frac{0.6}{0.3829}\right)\right\}\approx 3.843.

The profile interval is approximately $[0.2071,\,0.6171]$. The slope was not treated as known: it was maximized over at each candidate mean. In this particular design its contribution is constant in the profile, which is why the calculation simplifies to the single binomial term.

Worked Example: A Poisson Mean at the Midpoint of Two Covariate Values

Let independent counts at $x=-1$ and $x=1$ be $Y_-\sim\operatorname{Poisson}(\mu_-)$ and $Y_+\sim\operatorname{Poisson}(\mu_+)$, with observed values 4 and 16. Fit $\log\mu(x)=\beta_0+\beta_1x$, and profile the mean $m=\mu(0)$.

The two fitted means are $\mu_-=\exp(\beta_0-\beta_1)$ and $\mu_+=\exp(\beta_0+\beta_1)$, so their product is $m^2$. The unrestricted estimates are $\hat\mu_-=4$ and $\hat\mu_+=16$, giving $\hat m=\sqrt{4\cdot16}=8$. To profile a candidate $m>0$, maximize the Poisson log-likelihood over positive $a=\mu_-$ and $b=\mu_+$ subject to $ab=m^2$. Write $a=me^{-t}$, $b=me^t$. Ignoring constants independent of $t$, the constrained log-likelihood is

\ell_m(t)=4\log(me^{-t})-me^{-t}+16\log(me^t)-me^t.

Differentiating and setting the derivative to zero gives

\frac{d\ell_m(t)}{dt}=-4+16+me^{-t}-me^t=0, \qquad\text{so}\qquad b-a=12.

The signs matter: at the unrestricted fitted means, the score equation reads $-4+16+4-16=0$. Combining $b-a=12$ with $ab=m^2$ gives the positive solution

a=\sqrt{36+m^2}-6,\qquad b=\sqrt{36+m^2}+6.

These values are positive for $m>0$, satisfy $ab=m^2$, and give $b-a=12$. They maximize the constrained log-likelihood: as a function of $t$, the second derivative is $-me^{-t}-me^t<0$, so the stationary point is the unique maximum.

The full model fits the two observed counts exactly. Substituting the profiled means into the likelihood-ratio statistic yields

W(m)=2\left\{4\log\left(\frac{4}{a}\right)+16\log\left(\frac{16}{b}\right)+a+b-20\right\},

where $a$ and $b$ are the functions of $m$ just derived. The 95% profile confidence set consists of the positive $m$ values for which this expression is at most 3.841. In particular, at $m=8$, $a=4$ and $b=16$, so $W(8)=0$, as required at the unrestricted estimate. For other candidate means, the two constrained means must be re-optimized; holding one of them at its unrestricted estimate would not compute the profile.

Worked Example: Enforcing a Logistic Mean Constraint in a Model with Several Coefficients

Consider a logistic model with three coefficients, $\operatorname{logit}(p(x))=\beta_0+\beta_1x_1+\beta_2x_2$. At the fixed covariate vector $(x_1,x_2)=(2,-1)$, the target mean is $m$. Its constraint is

\beta_0+2\beta_1-\beta_2=\operatorname{logit}(m).

Let $u=\beta_1$ and $v=\beta_2$. Solving the constraint for the intercept gives $\beta_0=\operatorname{logit}(m)-2u+v$. Thus, for the observed data log-likelihood $\ell$, the profile is obtained by maximizing

\ell_p(m)=\sup_{u,v}\ell\big(\operatorname{logit}(m)-2u+v,\ u,\ v\big).

For a candidate $m=0.5$, $\operatorname{logit}(m)=0$, so the constrained coefficients have the form $(-2u+v,u,v)$. For example, $u=0.3,\ v=0.2$ gives $\beta=(-0.4,0.3,0.2)$, and its linear predictor at $(2,-1)$ is $-0.4+2(0.3)-0.2=0$, producing fitted mean $0.5$. This verifies that the vector satisfies the target constraint. To obtain the profile likelihood, however, one must maximize over all $u,v$ using the actual data likelihood; this one feasible vector alone is not generally the constrained maximum.

Why the Profile Construction Works

The confidence set follows directly from comparing the best achievable likelihood under each candidate mean with the unrestricted best likelihood. Since the candidate constraint defines a subset of the full parameter space, $\ell_p(m)\leq\ell(\hat\beta)$, so $W(m)\geq0$. A candidate is retained when its constrained fit loses no more than the likelihood-ratio cutoff.

Result: If $g$ is one-to-one and the constraint is feasible, maximizing over coefficient vectors that produce mean $m$ is equivalent to maximizing over coefficient vectors that satisfy $x_0^\mathsf{T}\beta=g(m)$.

Translate the target.
By definition, $\mu_0(\beta)=m$ means $g^{-1}(x_0^\mathsf{T}\beta)=m$.

Apply the link.
Because $g$ is one-to-one on the range in question, applying $g$ to both sides gives $x_0^\mathsf{T}\beta=g(m)$.

Use the same feasible set.
The two equations describe exactly the same coefficient vectors. Therefore the supremum of $L(\beta)$ over either set is the same, proving the result. □

Under the stated regularity conditions, Wilks’ theorem supplies the asymptotic reference distribution because the constraint fixes one regular scalar target. This explains the one-degree-of-freedom cutoff even when profiling requires optimizing over many nuisance coefficients. The result does not say the profile curve is quadratic in $m$, or that the resulting endpoints are equally far from the estimate.

Interpretation and a Common Pitfall

For a logistic model, the target mean lies between 0 and 1; for a log-link Poisson model it is positive. An interval for the fitted mean describes uncertainty about the model’s conditional mean at the chosen covariates. It does not add the random variation of a future observation. For instance, a future Poisson count at that covariate value can vary around its mean even if the mean were known exactly.

Wrong: Fix the nuisance coefficients at their unrestricted estimates and vary only the target coefficient. This compares candidate means along one particular path through the coefficient space.
Correction: At every candidate mean, maximize the likelihood over all remaining coefficient directions allowed by the constraint. That is what defines the profile likelihood.
Wrong: Treat the profile interval as necessarily symmetric on the mean scale.
Correction: Solve the likelihood-ratio cutoff equation on the mean scale. The profile can be asymmetric, and its endpoints need not be equally distant from the estimate.

Key takeaway: To profile a fitted GLM mean at fixed covariates, translate a candidate mean into a linear-predictor constraint, maximize over every remaining coefficient, and retain candidates whose likelihood-ratio statistic is below the appropriate cutoff.

Check Your Understanding

Use these questions to check both the constraint and the role of profiling.

In a GLM with link $g$, write the coefficient constraint corresponding to a candidate mean $m$ at covariate vector $x_0$.
Why does a single scalar mean target generally lead to a one-degree-of-freedom likelihood-ratio cutoff?
In the logistic example, why can the fitted probability at $x=1$ be maximized independently of the candidate $p(0)$?
For the Poisson example, verify that the constrained means satisfy both $ab=m^2$ and $b-a=12$.
Explain why an interval for a fitted mean is not a prediction interval for a future response.