Confidence Ellipsoids with Estimated Covariance

From a Test Statistic to a Region for the Mean

When covariance is unknown, uncertainty in a mean vector has two sources: variation in the observations and the need to estimate their covariance structure. The exact Hotelling $T^2$ pivot accounts for both under multivariate normal sampling. Inverting its acceptance condition gives a finite-sample confidence ellipsoid: a set of plausible mean vectors whose size, orientation, and location are determined by the data and an F quantile.

The key practical point is that the cutoff is not just an F quantile. It includes a dimension- and sample-size-dependent factor, and the quadratic form includes a factor of $n$. Keeping these pieces in the right places is essential for the region to have its stated coverage.

Assumptions, Statistic, and Exact Calibration

Definition: Let $X_1,\ldots,X_n$ be independent and identically distributed $p$-dimensional random vectors with $X_i\sim N_p(\mu,\Sigma)$, where $\Sigma$ is positive definite and $n>p$. Define the sample mean and unbiased sample covariance by $$ \bar X=\frac{1}{n}\sum_{i=1}^n X_i,\qquad S=\frac{1}{n-1}\sum_{i=1}^n(X_i-\bar X)(X_i-\bar X)'. $$ For a candidate mean vector $\mu_0$, Hotelling’s statistic is $$ T^2(\mu_0)=n(\bar X-\mu_0)'S^{-1}(\bar X-\mu_0). $$

Under the stated assumptions, $S$ is nonsingular with probability one. The exact Hotelling calibration is

\frac{n-p}{p(n-1)}T^2(\mu)\sim F_{p,n-p},

where $F_{p,n-p}$ denotes an F distribution with numerator and denominator degrees of freedom $p$ and $n-p$, respectively. This is a finite-sample result; it does not use a large-sample approximation.

For $0<\alpha<1$, let $F_{p,n-p;1-\alpha}$ be the $1-\alpha$ quantile of that F distribution, so that $P(F_{p,n-p}\leq F_{p,n-p;1-\alpha})=1-\alpha$. Define the corresponding cutoff

c_\alpha=\frac{p(n-1)}{n-p}F_{p,n-p;1-\alpha}.

The Confidence Ellipsoid and Its Coverage

Theorem: Under the Gaussian sampling assumptions above, an exact $100(1-\alpha)\%$ confidence region for $\mu$ is $$ \mathcal C_{1-\alpha} = \left\{\mu_0\in\mathbb R^p: n(\bar X-\mu_0)'S^{-1}(\bar X-\mu_0)\leq c_\alpha \right\}. $$ That is, $P_\mu(\mu\in\mathcal C_{1-\alpha})=1-\alpha$.

The region is obtained by applying the same inequality to every candidate $\mu_0$: retain the candidate if its observed $T^2(\mu_0)$ does not exceed $c_\alpha$. Its coverage follows directly from the exact pivot.

Use the pivot at the true mean.
At $\mu_0=\mu$, the exact calibration gives $\frac{n-p}{p(n-1)}T^2(\mu)\sim F_{p,n-p}$.

Apply the upper-tail quantile.
By the definition of $F_{p,n-p;1-\alpha}$, the probability that the calibrated statistic is at most this quantile is $1-\alpha$.

Rearrange the event.
Multiplying the inequality by the positive factor $p(n-1)/(n-p)$ gives $T^2(\mu)\leq c_\alpha$, exactly the event that the true $\mu$ belongs to $\mathcal C_{1-\alpha}$.

Because the F distribution is continuous, the quantile has no point mass at its boundary. Thus the resulting coverage is exactly $1-\alpha$, not merely at least $1-\alpha$. This argument depends on the Gaussian model and the stated sample-size condition; it does not claim exact coverage for arbitrary distributions.

Geometry: Center, Orientation, and Axis Lengths

For observed data, the region is centered at $\bar X$, because its defining quadratic form is a squared distance from $\bar X$. Its orientation and relative widths are determined by $S$. To make that geometry explicit, write the spectral decomposition $S=Q\Lambda Q'$, where $Q$ is orthogonal and $\Lambda$ has positive diagonal entries $\lambda_1,\ldots,\lambda_p$. In coordinates $z=Q'(\mu_0-\bar X)$, the region becomes

\sum_{j=1}^p\frac{z_j^2}{\lambda_j}\leq \frac{c_\alpha}{n}.

Result: The ellipsoid’s principal axes point along the eigenvectors of $S$. Along the eigenvector corresponding to $\lambda_j$, its semiaxis length is $\sqrt{c_\alpha\lambda_j/n}$.

To verify the axis lengths, set all rotated coordinates except $z_j$ to zero. The boundary equation then reduces to $z_j^2/\lambda_j=c_\alpha/n$, so the two boundary points on that axis have coordinates $z_j=\pm\sqrt{c_\alpha\lambda_j/n}$. Thus greater sample variance in a direction produces a wider confidence region in that direction. The factor $1/n$ reflects the scale of the sample mean, while $c_\alpha$ incorporates the finite-sample cost of estimating covariance.

Worked Example: The One-Dimensional Case

Suppose $n=6$, the observed sample mean is $\bar x=12$, and the sample variance is $s^2=4$. Find the 95% confidence region for the scalar mean $m$.

For $p=1$, the cutoff is $c_{.05}=F_{1,5;.95}$. The identity $F_{1,5;.95}=t_{5;.975}^2$ gives the familiar two-sided $t$ critical value in squared form. Using $t_{5;.975}\approx2.570582$, the region is

6(12-m)^2/4\leq 2.570582^2.

Equivalently, $|12-m|\leq2.570582\sqrt{4/6}$. Since $\sqrt{4/6}\approx0.816497$, the margin is approximately $2.0989$, which rounds to $2.099$ to three decimal places. The resulting interval is approximately $[9.901,14.099]$. Using the unrounded critical value gives the same endpoints to three decimal places. This check illustrates why intermediate rounding should be delayed until the final reported limits.

Worked Example: A Two-Dimensional Ellipsoid with Known Axes

Let $n=10$, $p=2$, $\bar x=(5,3)'$, and $S=\begin{pmatrix}4&0\\0&1\end{pmatrix}$. Construct the 95% confidence region and find its semiaxis lengths.

For $F_{2,8}$, the 95th percentile is approximately $4.45897$. Therefore

c_{.05}=\frac{2(10-1)}{10-2}(4.45897) =2.25(4.45897) \approx10.03268.

Since $S^{-1}=\begin{pmatrix}1/4&0\\0&1\end{pmatrix}$, the region is

10\left\{\frac{(5-\mu_1)^2}{4}+(3-\mu_2)^2\right\}\leq10.03268.

The eigenvalues of $S$ are $4$ and $1$, so the semiaxis lengths are $\sqrt{10.03268(4)/10}\approx2.0033$ and $\sqrt{10.03268(1)/10}\approx1.0016$. The longer axis is in the first coordinate direction, where the observed variance is larger. The center is $(5,3)'$.

Worked Example: Checking a Candidate Mean with Correlated Coordinates

Suppose $n=12$, $p=2$, $\bar x=(2,1)'$, and $S=\begin{pmatrix}4&1.2\\1.2&1\end{pmatrix}$. Determine whether $\mu_0=(2.3,0.8)'$ belongs to the 90% confidence region.

First, $\det(S)=4-1.2^2=2.56>0$, and

S^{-1}=\frac{1}{2.56} \begin{pmatrix}1&-1.2\\-1.2&4\end{pmatrix}.

The difference from the sample mean is $d=\bar x-\mu_0=(-0.3,0.2)'$. The quadratic form is unchanged if both signs are reversed, so it can be calculated using $(0.3,-0.2)'$:

d'S^{-1}d =\frac{(0.3)^2-2(1.2)(0.3)(-0.2)+4(-0.2)^2}{2.56} =\frac{0.394}{2.56} =0.15390625.

Thus $T^2(\mu_0)=12(0.15390625)=1.846875$. For $F_{2,10}$, the 90th percentile is $5(10^{0.2}-1)\approx2.92447$; hence $c_{.10}=\frac{2(11)}{10}(2.92447)\approx6.43383$. Since $1.846875<6.43383$, the candidate lies inside the confidence ellipsoid. The nonzero off-diagonal entries of $S$ mean its principal axes are rotated relative to the coordinate axes.

Common Mistakes and Why the Region Matters

A confidence ellipsoid is a joint statement about the entire mean vector. Its exact coverage concerns whether the whole vector $\mu$ lies in the region; it does not say that each coordinate separately has 95% coverage. The shape also matters: using only the diagonal entries of $S$ would ignore correlations and generally change both orientation and membership decisions.

Using the F quantile as the $T^2$ cutoff. The cutoff is $\frac{p(n-1)}{n-p}F_{p,n-p;1-\alpha}$, not $F_{p,n-p;1-\alpha}$ alone.
Dropping the factor $n$. The statistic is $n(\bar X-\mu_0)'S^{-1}(\bar X-\mu_0)$. Omitting $n$ changes the region’s scale.
Replacing the joint region by separate coordinate intervals. Coordinatewise intervals do not generally have the ellipsoid’s simultaneous coverage or account for the covariance orientation.
Using the exact calibration outside its assumptions. The stated finite-sample coverage relies on independent Gaussian observations and an invertible sample covariance.

In applications, the ellipsoid gives a compact way to report uncertainty about a vector of related means while preserving their covariance structure. It can also show which combinations of mean changes are relatively well or poorly determined. The exact cutoff is especially useful when the sample is not large enough to justify relying solely on an asymptotic approximation.

Check Your Understanding

Use the assumptions and formulas in this tutorial to answer the following.

Write the exact $100(1-\alpha)\%$ confidence region for $\mu$ and identify the distribution used for its cutoff.
Why must the factor $p(n-1)/(n-p)$ multiply the F quantile?
If an eigenvalue of $S$ doubles while the cutoff and sample size stay fixed, how does the associated semiaxis length change?
For $p=1$, explain how the Hotelling confidence region reduces to the usual two-sided $t$-based interval.
Why can separate coordinatewise intervals not automatically be described as having the ellipsoid’s joint coverage?