Marginal Intervals Versus Joint Coverage

One Interval for Each Coordinate Is Not Automatically a Joint Guarantee

When a parameter has several coordinates, it is natural to report one confidence interval for each coordinate. Each interval may have a stated coverage level, but that statement concerns one coordinate at a time. A different question is whether all coordinates are covered simultaneously. The distinction matters because the intervals are computed from the same data, so their coverage events may be dependent.

For a vector parameter \(\theta=(\theta_1,\theta_2)\), let \(I_{1,n}\) and \(I_{2,n}\) be random intervals constructed from a sample of size \(n\). Their marginal coverage probabilities are \(P(\theta_1\in I_{1,n})\) and \(P(\theta_2\in I_{2,n})\). The joint coverage probability is \(P(\theta_1\in I_{1,n},\theta_2\in I_{2,n})\). The latter is the probability that the parameter vector lies in the rectangle formed by the two intervals.

In particular, if both intervals have 95% marginal coverage, this does not by itself establish 95% joint coverage. The probability of the intersection of two events depends on their relationship. The intervals may miss together often, or their failures may occur on different samples. The joint Wald ellipsoid from the earlier material on Joint Wald Ellipsoids for Vector Parameters is one way to construct a region directly; here the focus is on interpreting coordinatewise intervals and their simultaneous coverage.

A Bivariate Normal Model for the Coverage Events

Suppose an estimator \(\hat{\theta}=(\hat{\theta}_1,\hat{\theta}_2)\) has standard errors \(s_1,s_2>0\), and consider symmetric intervals \(\hat{\theta}_j\pm c s_j\), for \(j=1,2\). Define the standardized errors \(Z_j=(\hat{\theta}_j-\theta_j)/s_j\). For these intervals to cover coordinate \(j\), it is necessary and sufficient that \(|Z_j|\le c\).

For a normal illustration, suppose \((Z_1,Z_2)\) has a bivariate normal distribution with mean zero, unit marginal variances, and correlation \(\rho\). Set \(c=z_{0.975}\), the 0.975 quantile of the standard normal distribution, approximately 1.96. Then each interval has exactly 95% marginal coverage, since \(P(|Z_j|\le c)=0.95\). Their joint coverage, however, is \(P(|Z_1|\le c,\ |Z_2|\le c)\), which depends on \(\rho\).

For a general nonsingular bivariate normal distribution with correlation \(-1<\rho<1\), the joint coverage can be written as

where \(\phi\) and \(\Phi\) are the standard normal density and distribution function. To see why, the conditional distribution of \(Z_2\) given \(Z_1=z\) is normal with mean \(\rho z\) and variance \(1-\rho^2\). The bracketed term is therefore \(P(-c\le Z_2\le c\mid Z_1=z)\). Multiplying by the density of \(Z_1\) and integrating over \(-c\le z\le c\) gives the probability that both coordinates are covered.

What Can Be Guaranteed Without Knowing the Dependence?

When the dependence between coordinatewise coverage events is unknown, the union bound gives a general guarantee. If \(A_j\) is the event that coordinate \(j\) is missed, then missing at least one coordinate is the event \(\bigcup_{j=1}^p A_j\). The union bound states that this event has probability no greater than the sum of the individual miss probabilities. It does not require independence.

Proof. Let \(A_j=\{\theta_j\notin I_{j,n}\}\). The event that at least one coordinate is missed is \(\bigcup_{j=1}^p A_j\). For any events \(A_1,\ldots,A_p\), the indicator of their union is at most the sum of their indicators:

Taking expectations gives \(P(\bigcup_j A_j)\le\sum_j P(A_j)\le\sum_j\alpha_j\). The event that every coordinate is covered is the complement of this union, so its probability is at least \(1-\sum_j\alpha_j\). \(\square\)

For a target joint level \(1-\alpha\), one simple choice is \(\alpha_j=\alpha/p\) for each coordinate. Each interval must then have marginal coverage at least \(1-\alpha/p\). The resulting product region has joint coverage at least \(1-\alpha\); the bound need not be an equality. This adjustment is often conservative, but it remains valid under arbitrary dependence.

Asymptotic Coverage Requires the Same Distinction

In large-sample inference, marginal and joint statements also have different forms. Suppose for each coordinate \(j\), \(P_\theta(\theta_j\in I_{j,n})\to1-\alpha_j\). This gives a limiting statement for each marginal probability; it does not specify the limiting probability of their intersection. To make a simultaneous asymptotic claim, one needs a joint argument, such as a multivariate limit calculation, or a finite-sample bound that remains valid as \(n\) grows.

For example, if the marginal miss probabilities satisfy \(\limsup_{n\to\infty}P_\theta(\theta_j\notin I_{j,n})\le\alpha_j\), then the union bound implies

This follows because the finite-sample joint probability is at least one minus the sum of marginal miss probabilities; taking the lower limit preserves the stated bound for a finite number of coordinates. It is a lower guarantee, not necessarily an exact limiting coverage.

Common Mistake and Practical Interpretation

Incorrect reasoning: “Both intervals have 95% coverage, so the probability that both cover is 95%.” This confuses two separate marginal probabilities with the probability of their intersection. As the independent normal example shows, the intersection probability is 90.25%.

Correction: State whether a reported level is marginal or joint, and identify how the joint claim is obtained. If coordinatewise intervals are being used as a rectangle, calculate or bound the probability that every coordinate is covered. Dependence matters for exact joint coverage; when it is unknown, Bonferroni calibration supplies a valid lower guarantee.

This distinction is important whenever a conclusion concerns several parameters at once. A collection of intervals may be suitable for describing each coordinate separately, while still offering weaker protection for a statement about the entire parameter vector. The intended interpretation should determine which coverage guarantee is reported.