Pointwise Versus Joint Coverage for Several Predictions

One Prediction or a Set of Predictions?

A prediction interval is often reported for one future outcome. Its coverage claim concerns that outcome: over repeated applications of the procedure under the model assumptions, intervals constructed this way contain the future outcome at the stated rate. A different question arises when several predictions are needed together. For example, a clinic may want intervals for the outcomes of several patients, or a laboratory may need intervals for several future measurements. Covering each outcome separately at 95% does not mean there is a 95% chance that every outcome in the set will be covered.

The distinction is between a claim about each case considered individually and a claim about the whole collection. To make the second claim, the interval procedure must account for the possibility that at least one of the outcomes falls outside its interval. This usually requires wider intervals.

Definition: A collection of prediction intervals has pointwise coverage $1-\alpha$ if each interval, considered separately, covers its corresponding future outcome with probability $1-\alpha$ under the stated model and repeated-sampling conditions. The collection has joint coverage $1-\alpha$ if the probability that all corresponding future outcomes fall inside their intervals is $1-\alpha$. Equivalently, joint coverage concerns the event that no interval in the collection misses.

For exact t-based prediction intervals, the usual regression model conditions include a correctly specified mean model and independent, normally distributed errors with a common variance. Under these conditions, each interval can be calibrated to have its stated individual coverage. Joint coverage is a separate probability question: it depends on how the events of missing the different outcomes relate to one another.

Why Individual Coverage Does Not Carry Over to a Group

Suppose there are $m$ future outcomes, and interval $i$ has pointwise coverage $1-\alpha$. Define $E_i$ as the event that interval $i$ misses its outcome. Then $P(E_i)=\alpha$. The event that at least one interval misses is $E_1\cup E_2\cup\cdots\cup E_m$, and the event of joint coverage is its complement.

If the miss events are independent and each has probability $\alpha$, the probability that none occurs is the product of their individual probabilities. This gives one useful calculation, but independence should not be assumed automatically. Prediction intervals for several outcomes may share the same fitted regression model and estimated error variance. Even when the future errors are independent, those shared estimates can make the interval events dependent.

Worked Example: Three Separate 95% Predictions

Suppose a researcher constructs 95% pointwise prediction intervals for the next three laboratory measurements. For this calculation only, suppose the three coverage events are independent and each interval has its advertised 95% coverage. What is the probability that all three intervals cover?

For each interval, the probability of coverage is $0.95$. Under the independence assumption, the probability that all three cover is

P(\text{all three cover})=0.95^3=0.857375.

Thus, under this simplifying assumption, the chance that at least one interval misses is $1-0.857375=0.142625$, or about 14.3%. The 95% pointwise label applies to each prediction separately; it does not describe the probability that the complete set covers. Without independence, $0.857375$ need not be the joint coverage, so it is not a generally valid answer for three regression predictions.

A general bound is available even when the miss events are dependent. The probability of at least one miss cannot exceed the sum of the individual miss probabilities. This fact is called the union bound.

Theorem: If $E_1,\ldots,E_m$ are miss events, then $$ P(E_1\cup\cdots\cup E_m)\leq \sum_{i=1}^{m}P(E_i). $$ If each interval has pointwise coverage at least $1-\alpha$, then each miss probability is at most $\alpha$, so the probability of at least one miss is at most $m\alpha$. Consequently, joint coverage is at least $1-m\alpha$, when this lower bound is informative.

To see why the bound holds, consider any outcome of the experiment. If at least one miss occurs, the indicator of the union is 1, and the sum of the indicators of the individual miss events is at least 1. If no miss occurs, the indicator of the union is 0 and the sum is also 0. Therefore, for every outcome,

I(E_1\cup\cdots\cup E_m)\leq \sum_{i=1}^{m} I(E_i).

Taking expectations on both sides gives the probability inequality, because the expectation of an event indicator is the probability of that event. This argument does not require independence.

Bonferroni Intervals for a Joint Coverage Goal

The union bound suggests how to construct a collection intended to achieve joint coverage of at least $1-\alpha$. For $m$ predictions, allocate a miss probability of at most $\alpha/m$ to each interval. Then the chance of any miss is at most $m(\alpha/m)=\alpha$. The resulting intervals are called Bonferroni-adjusted intervals. They may be conservative because the union bound can be larger than the actual probability of any miss.

For a two-sided t-based prediction interval, allocating a total miss probability of $\alpha/m$ to one case puts $\alpha/(2m)$ in each tail. If $\hat{y}_i$ is the fitted prediction for future case $i$, $SE_{\mathrm{pred},i}$ is its prediction standard error, and $df$ is the residual degrees of freedom, the adjusted interval is

\hat{y}_i \pm t_{1-\alpha/(2m),\,df}\,SE_{\mathrm{pred},i}.

Here $t_{q,df}$ is the t critical value with cumulative probability $q$ and $df$ degrees of freedom. Under the model conditions that validate each t-based prediction interval, each adjusted interval has marginal coverage $1-\alpha/m$. The union bound then ensures that all $m$ outcomes are covered with probability at least $1-\alpha$. The guarantee does not depend on independence among the intervals.

Worked Example: Four Predictions With at Least 95% Joint Coverage

A hypothetical model is used to predict the future concentration, in mg/L, for four cases. The residual degrees of freedom are $40$. The fitted predictions are 50, 62, 70, and 55 mg/L, and the corresponding prediction standard errors are 3, 4, 5, and 3.5 mg/L. Assume the model conditions for t-based prediction intervals are satisfied. Compare ordinary 95% pointwise intervals with Bonferroni intervals intended to provide at least 95% joint coverage.

For the pointwise intervals, the critical value is approximately $t_{0.975,40}=2.021$. For example, the first interval is

50\pm 2.021(3)=50\pm6.063=(43.94,\ 56.06)\text{ mg/L}.

Using the same pointwise critical value for all four cases gives these intervals:

Case	Pointwise 95% interval (mg/L)
1	(43.94, 56.06)
2	(53.92, 70.08)
3	(59.90, 80.11)
4	(47.93, 62.07)

Each interval has 95% pointwise coverage under the model. That statement alone does not give a 95% guarantee that all four future concentrations will lie in their intervals.

For at least 95% joint coverage, set $m=4$ and $\alpha=0.05$. Each interval must have marginal coverage $1-\alpha/m=0.9875$. The two-tail critical value is approximately $t_{1-0.05/(2\cdot4),40}=t_{0.99375,40}=2.615$. The first adjusted interval is therefore

50\pm 2.615(3)=50\pm7.845=(42.16,\ 57.85)\text{ mg/L}.

Applying the same adjusted critical value to the other predictions gives:

Case	Bonferroni interval (mg/L)
1	(42.16, 57.85)
2	(51.54, 72.46)
3	(56.93, 83.08)
4	(45.85, 64.15)

Each adjusted interval has 98.75% marginal coverage. The probability that any of the four misses is at most $4(0.0125)=0.05$. Thus the probability that all four future concentrations are covered is at least 0.95. The intervals are wider than the ordinary pointwise intervals because the goal is stronger: protect the coverage claim for the set as a whole.

Choosing and Interpreting the Coverage Claim

The right procedure depends on the question. If the analysis will report one case-specific prediction at a time and each claim is considered separately, pointwise intervals may answer the question. If the intended claim is that every outcome in a planned collection is covered, a simultaneous procedure such as Bonferroni adjustment is appropriate. The phrase “95% prediction interval” should therefore be accompanied by clarity about whether 95% refers to each interval or to the entire collection.

Worked Example: Five Outcomes and a 90% Group-Wide Goal

Suppose five future measurements are each to be reported with a two-sided prediction interval, and the desired probability that all five are covered is at least 90%. First consider ordinary 90% pointwise intervals. If their coverage events were independent, the probability all five cover would be

$$ 0.90^5=0.59049. $$

Under that independence assumption, there would be about a 40.95% chance of at least one miss. Independence may not hold, but the calculation shows why a pointwise 90% label is not a group-wide 90% guarantee.

For a Bonferroni guarantee, use $\alpha=0.10$ and $m=5$. Each interval is assigned miss probability at most $0.10/5=0.02$, giving 98% marginal coverage. If $df=30$, the two-sided critical value is approximately $t_{0.99,30}=2.457$. For a hypothetical case with fitted prediction 10 units and prediction standard error 2 units, the adjusted interval is

10\pm2.457(2)=10\pm4.914=(5.086,\ 14.91)\text{ units}.

Using this adjusted level for each of the five cases makes the probability of at least one miss no greater than $5(0.02)=0.10$. Therefore, the probability that all five outcomes are covered is at least 0.90, without assuming independent coverage events.

Common Mistake and Practical Trade-Off

A common mistake is to reason, “Every one of these four intervals is 95%, so there is a 95% chance all four cover.” This confuses marginal coverage with joint coverage. With independent 95% coverage events, the joint probability for four cases would instead be $0.95^4$, about 81.45%; without independence, that product is not justified. A Bonferroni procedure provides a dependence-robust lower bound by using more conservative, wider intervals.

Bonferroni is not the only possible simultaneous method, and its guarantee may be conservative. Its advantage here is simplicity: it requires valid marginal intervals and the union bound, not a model for the dependence among miss events. Its cost is reduced precision for each case. As the number of cases grows, the per-interval error allowance $\alpha/m$ shrinks, and the intervals generally widen.

Key takeaway: Pointwise coverage describes each prediction considered separately. Joint coverage describes whether every outcome in a collection is covered at once. For $m$ predictions, Bonferroni intervals with per-case error allowance $\alpha/m$ guarantee joint coverage of at least $1-\alpha$, provided the individual intervals have valid coverage.

Check Your Understanding

Check that you can distinguish the two kinds of coverage and apply the group-wide adjustment.

In your own words, what event does 95% pointwise coverage describe, and what event does 95% joint coverage describe for four outcomes?
If three independent prediction intervals each have 90% coverage, calculate the probability that all three cover. Why would this calculation not generally be valid without independence?
For eight future outcomes and a desired joint coverage of at least 95%, what maximum miss probability should be allocated to each interval using Bonferroni?
Explain why Bonferroni-adjusted intervals are generally wider than ordinary intervals at the same nominal pointwise level.
Does the union-bound guarantee require the prediction intervals’ coverage events to be independent? Briefly justify your answer.