Joint Wald Ellipsoids for Vector Parameters

From a Vector Limit to a Joint Confidence Set

An estimate of a vector parameter reports several quantities at once. For example, a model may estimate two regression coefficients, or a study may estimate two population means. A confidence set for the vector should account for their joint sampling variation: it should describe a region that covers the whole parameter vector, not separate uncertainty statements considered one coordinate at a time.

When an estimator has a multivariate normal large-sample limit, its standardized distance from the true vector has an asymptotic chi-square distribution. Estimating the covariance matrix then gives a practical confidence set: an ellipsoid centered at the estimated vector. Its size is set by a chi-square critical value, and its shape and orientation reflect the estimated covariance.

The matrix \(\hat V\) estimates the covariance of \(\sqrt{n}(\hat{\theta}-\theta)\), not the covariance of \(\hat{\theta}\) itself. Thus the estimated covariance matrix on the original estimator scale is \(\hat V/n\). Equivalently, the defining inequality can be written \((\hat{\theta}-\vartheta)^\mathsf{T}(\hat V/n)^{-1}(\hat{\theta}-\vartheta)\leq\chi^2_{p,1-\alpha}\).

Why the Chi-Square Cutoff Gives Joint Coverage

The key result turns the multivariate normal approximation into a coverage statement for the entire parameter vector. It requires both asymptotic normality and consistent estimation of the limiting covariance matrix. Positive definiteness matters: it makes the quadratic form well-defined and prevents any direction from having zero limiting variance.

Proof. Write \(Z_n=\sqrt{n}(\hat{\theta}-\theta)\). The event that the true vector belongs to the ellipsoid is exactly \[ \left\{Z_n^\mathsf{T}\hat V^{-1}Z_n\leq\chi^2_{p,1-\alpha}\right\}. \] Because \(V\) is positive definite, matrix inversion is continuous at \(V\). Therefore \(\hat V^{-1}\xrightarrow{p}V^{-1}\). By the assumed limit and Slutsky’s theorem, the quadratic form \(Z_n^\mathsf{T}\hat V^{-1}Z_n\) converges in distribution to \(Z^\mathsf{T}V^{-1}Z\), where \(Z\sim N_p(0,V)\).

To identify this limit, write \(Z=V^{1/2}U\), where \(U\sim N_p(0,I_p)\) and \(V^{1/2}\) is a symmetric square root of \(V\). Then \[ Z^\mathsf{T}V^{-1}Z = U^\mathsf{T}V^{1/2}V^{-1}V^{1/2}U = U^\mathsf{T}U = \sum_{j=1}^p U_j^2. \] The final sum has a chi-square distribution with \(p\) degrees of freedom. Its distribution is continuous at \(\chi^2_{p,1-\alpha}\), so convergence in distribution implies \[ P\!\left(Z_n^\mathsf{T}\hat V^{-1}Z_n\leq\chi^2_{p,1-\alpha}\right) \longrightarrow P\!\left(\chi^2_p\leq\chi^2_{p,1-\alpha}\right) = 1-\alpha. \] This is the claimed simultaneous coverage of the vector. \(\square\)

The coverage statement concerns one event: the true vector lies inside the random ellipsoid. The chi-square degrees of freedom are \(p\), the dimension of the vector. This is different from taking a one-dimensional normal cutoff and applying it separately to each coordinate.

Geometry of the Ellipsoid

The defining quadratic form measures distance from the center after accounting for covariance. A displacement along a direction with large estimated variance contributes less to the quadratic form than an equally long displacement along a direction with small estimated variance. Correlation also rotates the ellipsoid’s principal axes away from the coordinate axes.

Derivation. The spectral decomposition gives \(\hat V=Q\Lambda Q^\mathsf{T}\), where the columns of \(Q\) are the eigenvectors and \(\Lambda\) is diagonal with entries \(\lambda_j\). Let \(d=\hat{\theta}-\vartheta\), and write \(d=Qy\). Since \(Q^\mathsf{T}Q=I_p\), \[ n d^\mathsf{T}\hat V^{-1}d = n y^\mathsf{T}\Lambda^{-1}y = n\sum_{j=1}^p\frac{y_j^2}{\lambda_j}. \] The ellipsoid condition is therefore \(\sum_{j=1}^p y_j^2/(\chi^2_{p,1-\alpha}\lambda_j/n)\leq1\). In the \(j\)th eigenvector direction, the boundary occurs at \(|y_j|=\sqrt{\chi^2_{p,1-\alpha}\lambda_j/n}\), as claimed. \(\square\)

Common Mistake and Practical Interpretation

A common mistake is to use a separate one-dimensional normal cutoff for each coordinate and then call the resulting rectangular region a joint Wald ellipsoid. That changes both the shape of the region and the calibration of the joint event. The ellipsoid uses one quadratic form that includes the off-diagonal covariance terms, and its cutoff is a chi-square quantile with \(p\) degrees of freedom.

A second frequent error is to use \(\hat V\) as though it were already the covariance matrix of \(\hat{\theta}\). Under the stated convention, \(\hat V\) estimates the covariance on the \(\sqrt n\) scale; the covariance on the estimator scale is \(\hat V/n\). Omitting the factor \(n\) in the quadratic form therefore gives the wrong size of the confidence set.

The confidence level describes repeated-sampling coverage of the full vector: over repeated samples, the constructed ellipsoid contains the true vector with probability approaching \(1-\alpha\). It is not a probability assigned to the fixed parameter after the data have been observed. The ellipsoid also makes the dependence among estimated coordinates visible through its orientation and shape, while retaining a single joint coverage statement.