Testing A Diagonal Covariance Matrix

What Does a Diagonal Covariance Matrix Assert?

For a multivariate measurement, a diagonal covariance matrix says that every pair of distinct components has covariance zero. This is a useful restriction when the scientific question is whether measurements vary independently, or when a simpler covariance model is needed. The likelihood-ratio test compares the fit of this restricted model with the fit of an unrestricted covariance matrix.

The test is specifically a Gaussian covariance test. For a multivariate normal distribution, zero covariances between components imply their independence. Outside the Gaussian model, zero covariance alone does not generally imply independence, so the interpretation must respect the distributional assumption.

Suppose \(\mathbf X_1,\ldots,\mathbf X_n\) are independent observations from \(N_p(\boldsymbol\mu,\boldsymbol\Sigma)\), where both the mean vector and covariance matrix are unknown. Write \(\bar{\mathbf X}=n^{-1}\sum_{i=1}^n\mathbf X_i\) and \(S=n^{-1}\sum_{i=1}^n(\mathbf X_i-\bar{\mathbf X})(\mathbf X_i-\bar{\mathbf X})'\). This definition uses divisor \(n\), as appropriate for the maximum-likelihood covariance estimate.

Deriving the Likelihood-Ratio Statistic

Assume \(n>p\) and that the observed residual vectors span \(p\) dimensions, so \(S\) is positive definite. These conditions ensure that the unrestricted covariance maximum-likelihood estimate exists as a positive-definite matrix. The Gaussian likelihood is

For fixed positive-definite \(\boldsymbol\Sigma\), the likelihood is maximized at \(\hat{\boldsymbol\mu}=\bar{\mathbf X}\). Substituting this estimate gives the covariance likelihood. The unrestricted estimate is \(S\); under the diagonal restriction, the estimate of each variance is the corresponding diagonal entry \(s_{jj}\). The derivation below also shows why these estimates maximize their respective likelihoods.

To verify the estimates and statistic, let \(A=\sum_i(\mathbf X_i-\bar{\mathbf X})(\mathbf X_i-\bar{\mathbf X})'=nS\). After maximizing over the mean, the covariance-dependent part of the log-likelihood is \(-\tfrac n2\log|\boldsymbol\Sigma|-\tfrac12\operatorname{tr}(\boldsymbol\Sigma^{-1}A)\). For unrestricted \(\boldsymbol\Sigma\), its maximum occurs at \(A/n=S\). Under the null, write \(\boldsymbol\Sigma=\operatorname{diag}(v_1,\ldots,v_p)\). The covariance-dependent terms separate into \(-\tfrac n2\sum_j\log v_j-\tfrac12\sum_j A_{jj}/v_j\). Differentiating with respect to \(v_j>0\) gives \(-n/(2v_j)+A_{jj}/(2v_j^2)=0\), hence \(v_j=A_{jj}/n=s_{jj}\). The second derivative at this point is \(-n/(2s_{jj}^2)<0\), and the log-likelihood tends downwards at either boundary \(v_j\to0\) or \(v_j\to\infty\), confirming a maximum.

At either fitted covariance, the trace term in the maximized log-likelihood is \(np\): unrestricted, \(\operatorname{tr}(S^{-1}nS)=np\); under the null, \(\sum_j A_{jj}/s_{jj}=np\). These terms cancel in the likelihood ratio. The determinant under the restricted fit is \(\prod_j s_{jj}\), while the unrestricted determinant is \(|S|\). Taking the ratio of maximized likelihoods and then \(-2\) times its logarithm gives the stated formulas.

Calibration and Degrees of Freedom

The unrestricted covariance matrix has \(p(p+1)/2\) distinct parameters: \(p\) variances and \(p(p-1)/2\) distinct off-diagonal covariances. A diagonal covariance matrix has \(p\) free variance parameters. The mean vector contributes \(p\) nuisance parameters to both models, so it does not change the difference in dimensions.

For fixed \(p\), under \(H_0\), and as \(n\to\infty\), Wilks' theorem therefore gives \(T\xrightarrow{d}\chi^2_{p(p-1)/2}\). The regularity conditions are satisfied when the true covariance has strictly positive diagonal entries: the unrestricted covariance parameter lies in the positive-definite cone, and the null is a smooth restriction within that parameter space. This is an asymptotic calibration, not an exact finite-sample chi-square distribution. Large values of \(T\) count against diagonality; an approximate upper-tail \(p\)-value is computed from the stated chi-square distribution.

Worked Calculations

Correlation Form and Nonnegativity

The statistic can also be expressed using the sample correlation matrix. Let \(D=\operatorname{diag}(s_{11},\ldots,s_{pp})\) and \(R=D^{-1/2}SD^{-1/2}\). Since \(S=D^{1/2}RD^{1/2}\), determinants give \(|S|=|D||R|=(\prod_j s_{jj})|R|\). Therefore \(T=-n\log|R|\). This form shows that the test is unchanged by rescaling any variable by a positive constant.

For positive-definite \(S\), the matrix \(R\) is positive definite and has diagonal entries all equal to one. Hadamard's determinant inequality states that the determinant of a positive-definite matrix is at most the product of its diagonal entries, with equality only when it is diagonal. Applied to \(R\), it gives \(0<|R|\le1\), so \(T\ge0\). Equality occurs exactly when \(R\), and therefore \(S\), is diagonal.

What the Test Can and Cannot Establish

Under multivariate normality, a diagonal covariance matrix has a stronger interpretation than merely uncorrelated components: the components are mutually independent. In fact, the Gaussian density with diagonal covariance factors into the product of its univariate normal densities. Conversely, independence of jointly Gaussian components implies zero pairwise covariances. Thus, within the multivariate normal model, testing diagonality is also a test of mutual independence.

That equivalence does not hold for arbitrary distributions. For example, let \(Z\sim N(0,1)\), and define \(X=Z\) and \(Y=Z^2\). Then \(\operatorname{Cov}(X,Y)=E[Z^3]-E[Z]E[Z^2]=0\), but \(Y\) is determined by \(X\), so \(X\) and \(Y\) are not independent. A rejection of the diagonal-covariance null indicates covariance structure inconsistent with the null; a failure to reject does not establish independence without the Gaussian assumption.

A second practical caution is that the chi-square reference is asymptotic. It should not be presented as an exact finite-sample distribution, especially when the sample is small relative to the number of variables. Also, if \(S\) is singular, the determinant-based statistic and the usual unrestricted maximum-likelihood covariance estimate are not available in the stated form; the conditions for the standard test must be checked before calculating it.