Confidence Ellipsoids with Known Covariance

From an Exact Pivot to a Confidence Region

When observations are multivariate, uncertainty about a mean vector concerns several coordinates at once. Separate intervals for each coordinate do not, in general, provide the stated joint confidence level. With a known covariance matrix and a Gaussian sampling model, there is a direct solution: a quadratic form in the sample mean has an exact chi-square distribution. Inverting that pivot gives a confidence ellipsoid with finite-sample coverage, not merely an approximation for large samples.

Suppose \(X_1,\ldots,X_n\) are independent and identically distributed \(p\)-dimensional random vectors with distribution \(N_p(\mu,\Sigma)\). Here \(\mu\) is the unknown mean vector, and \(\Sigma\) is a known, positive-definite covariance matrix for each observation. Write \(\bar X=n^{-1}\sum_{i=1}^n X_i\). Positive definiteness ensures that \(\Sigma^{-1}\) exists and that the quadratic form used below is well defined.

The symbol \(\mu_0\) inside the set ranges over possible values of the unknown mean. The center of the set is the observed sample mean \(\bar X\); the factor \(n\) reflects the fact that the covariance matrix of \(\bar X\) is \(\Sigma/n\). The cutoff depends on the dimension \(p\), the desired confidence level, and not on an asymptotic approximation.

Why the Pivot Has a Chi-Square Distribution

The exact result follows from the sampling distribution of the mean. Under the stated model, \(\bar X\) has distribution \(N_p(\mu,\Sigma/n)\). Let \(\Sigma^{-1/2}\) denote the symmetric inverse square root of the positive-definite matrix \(\Sigma\). Standardizing the mean gives a vector with independent standard normal coordinates.

Proof. Independence and Gaussianity imply that the sample mean is Gaussian. Its expectation is \(E[\bar X]=\mu\), and independence gives $$ \operatorname{Cov}(\bar X) =\frac{1}{n^2}\sum_{i=1}^n\operatorname{Cov}(X_i) =\frac{\Sigma}{n}. $$ Therefore $$ Z=\sqrt n\,\Sigma^{-1/2}(\bar X-\mu) $$ is multivariate normal with mean zero and covariance $$ \operatorname{Cov}(Z) =n\Sigma^{-1/2}\frac{\Sigma}{n}\Sigma^{-1/2}=I_p. $$ Thus the \(p\) coordinates of \(Z\) are independent \(N(0,1)\) random variables. By definition, the sum of their squares has a chi-square distribution with \(p\) degrees of freedom. Finally, $$ Z^\mathsf{T}Z =n(\bar X-\mu)^\mathsf{T}\Sigma^{-1/2}\Sigma^{-1/2}(\bar X-\mu) =n(\bar X-\mu)^\mathsf{T}\Sigma^{-1}(\bar X-\mu), $$ which proves the result. \(\square\)

To construct the region, use the theorem with the true mean replaced by a candidate value \(\mu_0\). For any fixed candidate, the event that its quadratic form is no greater than the cutoff is the event that the pivot falls below its \(1-\alpha\) quantile. This gives the coverage statement directly.

Proof. By the definition of \(\mathcal C_{1-\alpha}\), the event that the true parameter belongs to the random set is $$ \{\mu\in\mathcal C_{1-\alpha}\} = \left\{ n(\bar X-\mu)^\mathsf{T}\Sigma^{-1}(\bar X-\mu) \leq q_{p,1-\alpha} \right\}. $$ The pivot theorem shows that the random quantity on the left side of this inequality has distribution \(\chi^2_p\). Hence $$ P_\mu(\mu\in\mathcal C_{1-\alpha}) =P(\chi^2_p\leq q_{p,1-\alpha})=1-\alpha, $$ by the definition of the quantile. The equality holds for every value of \(\mu\), so the coverage is exact and finite-sample. \(\square\)

Worked Calculations

Interpreting and Using the Result

The confidence level describes the long-run performance of the procedure: if the same sampling process and construction were repeated, the resulting random set would contain the fixed true mean in \(100(1-\alpha)\%\) of repetitions. In this known-covariance Gaussian model, that statement is exact for each sample size \(n\). It is not a claim that the fixed parameter has probability \(1-\alpha\) of lying in this particular observed set.

The covariance matrix enters through its inverse, so the calculation accounts for both the scale of each coordinate and dependence among coordinates. The number of degrees of freedom in the cutoff is the number of mean coordinates, \(p\). The sample size multiplies the quadratic form because averaging \(n\) independent observations reduces the covariance from \(\Sigma\) to \(\Sigma/n\). These pieces must be used together: changing the cutoff, the scaling, or the covariance matrix changes the coverage calibration.

This construction requires a known positive-definite covariance matrix and independent Gaussian observations with a common mean and covariance. If the covariance is estimated from the data, the pivot is no longer the chi-square pivot established here; a different exact calibration is needed. Likewise, under a non-Gaussian sampling model, the same quadratic form need not have an exact chi-square distribution.

Common Mistake

A frequent error is to use a one-dimensional normal cutoff for each coordinate and then call the resulting collection a joint \(100(1-\alpha)\%\) region. That procedure does not generally have the claimed joint coverage: the coordinates can be dependent, and even independent coordinate intervals each having a specified coverage do not automatically have that same coverage simultaneously. The correction is to use the single \(p\)-degree-of-freedom chi-square cutoff for the full quadratic form. Another common error is to omit the factor \(n\); doing so ignores the covariance reduction in the sample mean and gives the wrong region.