Pointwise Mean-Response Intervals Across a Design Grid

From One Design Point to a Grid

A mean-response interval at one design point answers a local question: how uncertain is the estimated mean response at this particular point? In many applications, however, conclusions are reported across several specified values of the predictors. Constructing an interval at each value is straightforward, but the interpretation requires care: a 95% guarantee for each interval separately is not automatically a 95% guarantee that all intervals cover their respective means at once.

We work with the full-rank normal linear model with a fixed design matrix. The grid is fixed in advance, and inference concerns the conditional mean response at each listed design point—not a future individual response.

Definition: Let $Y=X\beta+\varepsilon$, where $X$ is a fixed $n\times p$ matrix of rank $p$, $n>p$, and $\varepsilon\sim N_n(0,\sigma^2 I_n)$ with $\sigma^2>0$. For a specified design vector $\mathbf{x}_j\in\mathbb{R}^p$, the mean response is $\mu_j=\mathbf{x}_j^\mathsf{T}\beta$, estimated by $\hat\mu_j=\mathbf{x}_j^\mathsf{T}\hat\beta$, where $\hat\beta=(X^\mathsf{T}X)^{-1}X^\mathsf{T}Y$. Write $q_j=\mathbf{x}_j^\mathsf{T}(X^\mathsf{T}X)^{-1}\mathbf{x}_j$, and let $s^2=\mathrm{SSE}/(n-p)$. For nonzero $\mathbf{x}_j$, the exact $100(1-\alpha)\%$ pointwise confidence interval for $\mu_j$ is $\hat\mu_j\pm t_{n-p,\,1-\alpha/2}s\sqrt{q_j}$.

A design grid is a finite collection $\mathbf{x}_1,\ldots,\mathbf{x}_m$. “Pointwise” means that the stated coverage probability applies to any one specified grid point considered on its own. The intervals need not have equal widths: their standard errors depend on the corresponding $q_j$. If $\mathbf{x}_j=0$, the target is identically zero and its estimator has zero variance; the usual $t$-interval formula is intended for nonzero design vectors.

Why the Pointwise Interval Is Exact

The interval follows from the exact $t$ distribution of the standardized estimation error. The necessary ingredients are the normal linear model, a fixed full-rank design, and an estimated variance based on the residual sum of squares.

Theorem: Under the model assumptions above, for every fixed nonzero $\mathbf{x}_j$, $$ \frac{\hat\mu_j-\mu_j}{s\sqrt{q_j}}\sim t_{n-p}. $$ Consequently, the interval in the definition has coverage probability $1-\alpha$ for $\mu_j$.

Proof. Since $\hat\beta-\beta=(X^\mathsf{T}X)^{-1}X^\mathsf{T}\varepsilon$, its distribution is multivariate normal with mean zero and covariance matrix $\sigma^2(X^\mathsf{T}X)^{-1}$. Therefore,

\hat\mu_j-\mu_j =\mathbf{x}_j^\mathsf{T}(\hat\beta-\beta) \sim N(0,\sigma^2q_j).

The matrix $(X^\mathsf{T}X)^{-1}$ is positive definite because $X$ has full column rank. Thus $q_j>0$ when $\mathbf{x}_j\ne0$; it is the estimator $\hat\mu_j$, not the fixed target $\mu_j$, that has positive sampling variance. Hence $Z_j=(\hat\mu_j-\mu_j)/(\sigma\sqrt{q_j})\sim N(0,1)$.

Under the same model, $\mathrm{SSE}/\sigma^2\sim\chi^2_{n-p}$, and this quantity is independent of $\hat\beta$, and therefore of $Z_j$. It follows that

\frac{\hat\mu_j-\mu_j}{s\sqrt{q_j}} = \frac{Z_j}{\sqrt{\{\mathrm{SSE}/\sigma^2\}/(n-p)}} \sim t_{n-p}.

The central $t$ distribution is symmetric, so its $1-\alpha$ central probability lies between $-t_{n-p,\,1-\alpha/2}$ and $t_{n-p,\,1-\alpha/2}$. Rearranging this event gives

P\!\left( \hat\mu_j-t_{n-p,\,1-\alpha/2}s\sqrt{q_j} \le \mu_j \le \hat\mu_j+t_{n-p,\,1-\alpha/2}s\sqrt{q_j} \right)=1-\alpha.

This is a statement for the one specified $j$. It does not say that the corresponding event holds for every grid point simultaneously. $\square$

Calculating Intervals at Several Points

For each point, the recipe is the same: evaluate the fitted mean, calculate $q_j$, multiply $s\sqrt{q_j}$ by the appropriate $t$ critical value, and add and subtract that margin. The degrees of freedom are $n-p$ at every point because all intervals use the same fitted model and residual variance estimate.

Worked Example: A Three-Point Grid in a Linear Model

Suppose a simple linear regression has an intercept and slope, with six design rows: two observations at each of $x=-1,0,1$. The observed sufficient statistics are $\hat\beta_0=2$, $\hat\beta_1=0.5$, and $\mathrm{SSE}=4$. Construct separate 95% intervals for the mean response at $x=-1,0,1$.

The design matrix has $\sum x_i=0$ and $\sum x_i^2=4$, so $X^\mathsf{T}X=\operatorname{diag}(6,4)$. There are $n-p=6-2=4$ residual degrees of freedom and $s^2=4/4=1$. For a design point $x$, use the vector $\mathbf{x}=(1,x)^\mathsf{T}$. Then

\hat\mu(x)=2+0.5x, \qquad q(x)=\frac{1}{6}+\frac{x^2}{4}.

The 95% critical value is $t_{4,0.975}=2.776445$.

At $x=-1$, $\hat\mu=1.5$, $q=1/6+1/4=5/12$, and $s\sqrt q=\sqrt{5/12}\approx0.645497$. The margin is $2.776445(0.645497)\approx1.7922$, giving $(1.5-1.7922,\ 1.5+1.7922)=(-0.2922,\ 3.2922)$.

At $x=0$, $\hat\mu=2$, $q=1/6$, and $s\sqrt q=\sqrt{1/6}\approx0.408248$. The margin is $2.776445(0.408248)\approx1.1335$, giving $(0.8665,\ 3.1335)$.

At $x=1$, $\hat\mu=2.5$, $q=1/6+1/4=5/12$, and the standard error and margin match those at $x=-1$. The interval is $(2.5-1.7922,\ 2.5+1.7922)=(0.7078,\ 4.2922)$.

Each interval has exact 95% coverage for its own mean response under the model. The calculations alone do not establish 95% coverage for all three means at once.

Pointwise Coverage Is Not Simultaneous Coverage

Let $A_j$ be the event that the interval at grid point $j$ contains $\mu_j$. Separate 95% pointwise intervals state that $P(A_j)=0.95$ for each $j$. Simultaneous coverage is a different probability, $P(A_1\cap\cdots\cap A_m)$. Its value depends on how the interval events are related, so the individual probabilities do not determine it.

For illustration, if four events were independent and each had probability $0.95$, then the probability that all four occurred would be $0.95^4=0.81450625$, not $0.95$. This simple calculation illustrates the distinction between marginal and joint probabilities; it does not assert that interval events in a particular regression model are independent.

Theorem: If each of $m$ intervals has pointwise coverage at least $1-\alpha_0$, then $$ P(A_1\cap\cdots\cap A_m)\ge 1-m\alpha_0. $$ In particular, constructing each interval with pointwise noncoverage probability at most $\alpha/m$ guarantees simultaneous coverage of at least $1-\alpha$.

Proof. The complement of simultaneous coverage is the event that at least one interval misses its target:

P\big((A_1\cap\cdots\cap A_m)^c\big) =P\left(\bigcup_{j=1}^m A_j^c\right) \le \sum_{j=1}^m P(A_j^c) \le m\alpha_0.

The first inequality is the union bound and does not require independence. Taking complements gives $P(A_1\cap\cdots\cap A_m)\ge1-m\alpha_0$. Setting $\alpha_0=\alpha/m$ yields a lower bound of $1-\alpha$. This adjustment is conservative in general; it is a way to obtain a simultaneous guarantee, not a claim that unadjusted pointwise intervals already have one. $\square$

Worked Example: Four Intervals and a Bonferroni Guarantee

Suppose four specified design points are each assigned a 95% pointwise interval. The union-bound result gives a simultaneous coverage lower bound of $1-4(0.05)=0.80$. The intervals are not thereby known to have exactly 80% joint coverage; the bound only guarantees at least 80%.

If the goal is at least 95% simultaneous coverage, assign each point a noncoverage probability of $\alpha_0=0.05/4=0.0125$. Each interval must then have pointwise confidence level $1-0.0125=0.9875$, or 98.75%. In the normal linear model, use the two-sided critical value $t_{n-p,\,1-0.0125/2}=t_{n-p,\,0.99375}$ at every point. The probability that any one of the four intervals misses is at most $4(0.0125)=0.05$, so the probability all four cover is at least 0.95.

The adjustment changes the critical value and therefore the widths of the intervals. It does not change the target: each interval still concerns the mean response at its own specified design point.

Worked Example: Two Points With Different Standard Errors

Consider a simple linear regression with design values $x=-2,-1,0,1,2$. Suppose $\hat\beta_0=10$, $\hat\beta_1=2$, and $\mathrm{SSE}=12$. Find the separate 95% intervals at $x=0$ and $x=2$.

Here $n=5$, $p=2$, and $\sum x_i=0$, $\sum x_i^2=10$, so $X^\mathsf{T}X=\operatorname{diag}(5,10)$. Thus $s^2=12/(5-2)=4$, and

\hat\mu(x)=10+2x, \qquad q(x)=\frac{1}{5}+\frac{x^2}{10}.

The critical value is $t_{3,0.975}=3.182446$. At $x=0$, the fitted mean is 10 and $q=1/5$, so the standard error is $2\sqrt{1/5}=\sqrt{0.8}\approx0.894427$. The margin is $3.182446(0.894427)\approx2.8465$, giving the interval $(7.1535,\ 12.8465)$.

At $x=2$, the fitted mean is 14 and $q=1/5+4/10=0.6$, so the standard error is $2\sqrt{0.6}=\sqrt{2.4}\approx1.549193$. The margin is $3.182446(1.549193)\approx4.9302$, giving $(9.0698,\ 18.9302)$.

Both intervals use the same confidence level and degrees of freedom, but their estimated standard errors differ. Each has 95% pointwise coverage under the model; together they do not automatically constitute a 95% simultaneous confidence statement.

Interpretation and Common Pitfall

A grid is a collection of targets, not a single target. A valid report should make clear whether it presents separate pointwise intervals or a procedure designed to cover the entire grid simultaneously. The distinction matters whenever readers may compare or interpret several intervals as a group.

Pitfall: treating individual coverage as joint coverage. The incorrect reasoning is: “Every interval is 95%, so the probability that all of them cover is 95%.” This confuses the probabilities $P(A_j)$ with $P(\cap_j A_j)$.
Correction: State the intervals’ pointwise coverage explicitly. If a simultaneous guarantee is required, use a procedure calibrated for that goal, such as the Bonferroni adjustment proved above.
Pitfall: using the wrong target. These intervals estimate the conditional mean response $\mathbf{x}_j^\mathsf{T}\beta$, not an individual future outcome. An interval for a future observation has a different purpose.
Correction: Identify the target at each grid point before calculating an interval, and use the standard error appropriate to that target.

Key takeaway: At every specified nonzero design point, the normal-theory $t$ interval $\hat\mu_j\pm t_{n-p,\,1-\alpha/2}s\sqrt{q_j}$ has exact pointwise coverage $1-\alpha$. A collection of such intervals does not, without further calibration, guarantee that all grid-point means are covered simultaneously.

Check Your Understanding

Answer each question using the model assumptions and definitions above.

What is the target of a mean-response interval at a design vector $\mathbf{x}_j$, and how does it differ from the estimated target?
In the pointwise interval formula, what determines the degrees of freedom and what determines the standard error at a particular grid point?
Why does $\mathbf{x}_j\ne0$ ensure that the estimator has positive variance under a full-rank design?
If five intervals each have 95% pointwise coverage, what lower bound does the union bound give for their simultaneous coverage?
What pointwise confidence level should be used for each of $m$ Bonferroni-adjusted intervals when the desired simultaneous coverage is $1-\alpha$?