Likelihood-Ratio Tests Of Block-Diagonal Covariance

Why Test a Block Structure?

In multivariate data, a natural question is whether two sets of measurements retain any linear association after accounting for their own within-set covariance. For example, a study might divide measurements into clinical markers and functional outcomes. A block-diagonal covariance model says that covariances between the two sets are zero, while allowing arbitrary covariance within each set. This is a structured hypothesis: it does not require all variables to be uncorrelated.

The Gaussian likelihood-ratio test compares the best-fitting unrestricted covariance matrix with the best-fitting covariance matrix obeying the specified block structure. Its statistic is determined by a ratio of determinants. The number of degrees of freedom is the number of distinct cross-block covariance entries set to zero.

Model and Hypothesis

Definition: Let $X_1,\ldots,X_n$ be independent observations from a $p$-variate normal distribution $N_p(\mu,\Sigma)$, where $\mu\in\mathbb{R}^p$ is unrestricted and $\Sigma$ is positive definite. Partition the $p$ variables into $k$ fixed, nonempty blocks of sizes $p_1,\ldots,p_k$, with $\sum_{g=1}^k p_g=p$. The block-diagonal covariance hypothesis is $$ H_0:\quad \Sigma=\operatorname{diag}(\Sigma_{11},\ldots,\Sigma_{kk}), $$ where each $\Sigma_{gg}$ is an unrestricted positive-definite $p_g\times p_g$ covariance matrix. Under the alternative, $\Sigma$ is any positive-definite $p\times p$ matrix.

The null sets every covariance between variables in different blocks to zero, but imposes no such restriction within a block. There are $p_gp_h$ distinct entries in the covariance submatrix joining blocks $g$ and $h$. Consequently, the number of cross-block restrictions is $$ q=\sum_{g<h}p_gp_h. $$ This count depends on the chosen partition. Making one block larger while holding the other blocks fixed adds cross-block restrictions against those other blocks. Merging blocks, however, removes the restrictions between the merged groups, so it can make the null less restrictive.

Deriving the Likelihood-Ratio Statistic

Write $\bar X=n^{-1}\sum_{i=1}^n X_i$, and define the maximum-likelihood covariance estimate under the unrestricted model by $$ S=\frac{1}{n}\sum_{i=1}^n(X_i-\bar X)(X_i-\bar X)'. $$ The divisor is $n$, not $n-1$, because this is the maximum-likelihood estimate. Assume $n>p$, so that $S$ is positive definite with probability one under a nonsingular multivariate normal model.

For any candidate $\mu,\Sigma$, the log-likelihood, up to its displayed constants, is $$ \ell(\mu,\Sigma)=-\frac{np}{2}\log(2\pi)-\frac{n}{2}\log|\Sigma|-\frac{1}{2}\sum_{i=1}^n(X_i-\mu)'\Sigma^{-1}(X_i-\mu). $$ For fixed positive-definite $\Sigma$, maximizing over the unrestricted mean gives $\hat\mu=\bar X$. To see how the covariance likelihood behaves, let $A=\sum_i(X_i-\bar X)(X_i-\bar X)'=nS$. Under the null, $\Sigma^{-1}$ is block diagonal, so $$ \operatorname{tr}(\Sigma^{-1}A)=\sum_{g=1}^k\operatorname{tr}(\Sigma_{gg}^{-1}A_{gg}). $$ Thus each block covariance can be maximized separately. Its maximum-likelihood estimate is $A_{gg}/n=S_{gg}$, the corresponding diagonal block of $S$. Under the unrestricted model the maximum-likelihood estimate is $S$.

Result: Let $L_0$ and $L_1$ be the maximized likelihoods under the block-diagonal null and unrestricted alternative, respectively. Then $$ \Lambda=\frac{L_0}{L_1} =\left(\frac{|S|}{\prod_{g=1}^k|S_{gg}|}\right)^{n/2}, \qquad -2\log\Lambda =n\log\left(\frac{\prod_{g=1}^k|S_{gg}|}{|S|}\right). $$ Large values of $-2\log\Lambda$ count as evidence against $H_0$.

To verify the likelihood ratio, the maximized log-likelihood under either model has the same mean and trace contributions, which equal $-np/2$ for the covariance part at its maximum. The determinant contribution under the null is $-\frac n2\sum_g\log|S_{gg}|$, while under the alternative it is $-\frac n2\log|S|$. Their difference is $\log\Lambda$, giving the stated expression. Since the unrestricted model includes every null model, its maximized likelihood cannot be smaller; correspondingly $0<\Lambda\leq1$ and the test statistic is nonnegative.

Worked Example: A Two-Block Determinant Test

Suppose $p=4$, with the first two variables in one block and the last two in another. Take $n=40$ and suppose the observed maximum-likelihood covariance matrix is $$ S=\begin{pmatrix} 1&0&0.5&0\\ 0&1&0&0.25\\ 0.5&0&1&0\\ 0&0.25&0&1 \end{pmatrix}. $$ Calculate the likelihood-ratio statistic and its asymptotic degrees of freedom.

The two within-block covariance matrices are both identity matrices, so $|S_{11}|=|S_{22}|=1$. Reorder the variables as $1,3,2,4$ to evaluate the full determinant. The matrix then separates into two $2\times2$ blocks, giving $$ |S|=(1-0.5^2)(1-0.25^2)=0.75(0.9375)=0.703125=\frac{45}{64}. $$ Therefore $$ -2\log\Lambda=40\log\left(\frac{1}{0.703125}\right) =40\log\left(\frac{64}{45}\right)\approx 14.09. $$ There are $2\times2=4$ cross-block covariance entries constrained to zero, so the asymptotic reference distribution has 4 degrees of freedom. At a chosen significance level, compare 14.09 with the corresponding upper-tail chi-square critical value. The determinant calculation alone does not specify a finite-sample exact reference distribution.

Degrees of Freedom and Wilks Calibration

The unrestricted symmetric covariance matrix has $p(p+1)/2$ free parameters. Under the null, block $g$ has $p_g(p_g+1)/2$ free covariance parameters. The difference is $$ \frac{p(p+1)}{2}-\sum_{g=1}^k\frac{p_g(p_g+1)}{2} =\sum_{g<h}p_gp_h. $$ For the equality, expand $p=\sum_g p_g$: the terms remaining after subtracting the within-block contributions are precisely the products $p_gp_h$ for each distinct pair of blocks. This also confirms that the parameter difference equals the number of distinct cross-block covariance entries constrained by the null.

Theorem: Suppose $p$ and the partition are fixed as $n\to\infty$, the observations are independent and identically distributed as $N_p(\mu,\Sigma)$, and the true $\Sigma$ is positive definite and satisfies $H_0$. Then Wilks’ theorem gives $$ -2\log\Lambda\xrightarrow{d}\chi^2_q, \qquad q=\sum_{g<h}p_gp_h. $$ An approximate level-$\alpha$ test rejects $H_0$ when the observed statistic exceeds the $1-\alpha$ quantile of $\chi^2_q$.

The hypotheses ensure a regular, fixed-dimensional covariance model with the true parameter in the interior of the positive-definite parameter space. This is an asymptotic calibration: it does not assert that the statistic has exactly a chi-square distribution at a finite sample size.

Worked Example: Counting Restrictions for Three Blocks

Let $p=4$ and use blocks of sizes $1,1,2$. The unrestricted covariance has $4(5)/2=10$ free entries. The null has $$ \frac{1(2)}{2}+\frac{1(2)}{2}+\frac{2(3)}{2}=1+1+3=5 $$ free entries. The difference is $10-5=5$, so the test has 5 asymptotic degrees of freedom.

The same result follows by counting cross-block entries: the three block pairs contribute $1\cdot1=1$, $1\cdot2=2$, and $1\cdot2=2$, for a total of 5. Notice why it would be incorrect to say that a partition with larger blocks is always more restrictive. If the two singleton blocks are merged, the partition becomes $2,2$; only $2\cdot2=4$ cross-block entries are constrained. If instead all four variables are singleton blocks, all $4\cdot3/2=6$ off-diagonal entries are constrained. Merging the singleton blocks removes their mutual zero-covariance restriction, reducing the number of constraints from 6 to 4 in this comparison.

Worked Example: A Three-Variable Test

Suppose $n=30$, with variables 1 and 2 in one block and variable 3 in the other. Let the observed covariance estimate be $$ S=\begin{pmatrix} 1&0.4&0.3\\ 0.4&1&0.2\\ 0.3&0.2&1 \end{pmatrix}. $$ The within-block determinants are $|S_{11}|=1-0.4^2=0.84$ and $|S_{22}|=1$. The full determinant is $$ |S|=1+2(0.4)(0.3)(0.2)-0.4^2-0.3^2-0.2^2 =1+0.048-0.29=0.758. $$ The leading principal minors are $1$, $0.84$, and $0.758$, all positive, so this symmetric matrix is positive definite. The statistic is $$ -2\log\Lambda=30\log\left(\frac{0.84}{0.758}\right)\approx 3.08. $$ There are $2\cdot1=2$ cross-block entries, so the asymptotic reference distribution is $\chi^2_2$. The test concerns the two covariances linking variable 3 to variables 1 and 2; it does not constrain the covariance $0.4$ within the first block.

What the Null Says About Independence

For a jointly multivariate normal vector, two subvectors are independent if and only if their cross-covariance matrix is zero. Therefore, under the stated Gaussian model, a block-diagonal covariance matrix implies mutual independence of the $k$ variable blocks. Conversely, mutual independence of those Gaussian blocks implies the block-diagonal covariance structure. This interpretation is specific to the joint normal model: outside that model, zero cross-covariances alone do not generally imply independence.

The determinant ratio also gives an intuitive description of the comparison. The product $\prod_g|S_{gg}|$ reflects the separate within-block covariance volumes, while $|S|$ reflects the joint covariance volume. A ratio substantially above one means that the unrestricted fit has a smaller determinant than the separate-block fit, producing a larger likelihood-ratio statistic. The test is about the joint collection of cross-block covariances, not about whether any one selected entry looks large in isolation.

Common Mistakes

Reversing the likelihood ratio. The convention here is $\Lambda=L_0/L_1\leq1$. The statistic is $n\log\{\prod_g|S_{gg}|/|S|\}$, which is nonnegative; reversing the determinant ratio changes its sign.
Counting all off-diagonal entries as restrictions. Only entries joining different blocks are set to zero. Covariances within a block remain free.
Assuming merging blocks adds restrictions. Merging blocks removes restrictions between the merged groups. The restriction count is $\sum_{g<h}p_gp_h$, and it must be recalculated for the actual partition.
Claiming general independence from zero covariance. The independence conclusion follows here because the joint distribution is multivariate normal. For nonnormal data, zero cross-block covariance does not by itself establish independence.
Treating the chi-square calibration as exact. Wilks’ result is asymptotic under the stated regularity conditions. It is not an exact finite-sample distribution claim.

Key takeaway: For a fixed partition of a Gaussian vector, test block-diagonal covariance with $-2\log\Lambda=n\log\{\prod_g|S_{gg}|/|S|\}$. Under the null its asymptotic degrees of freedom equal the number of cross-block covariance entries, $\sum_{g<h}p_gp_h$.

Check Your Understanding

Answer each question using the specified block partition and model assumptions.

For block sizes $2,3,1$, how many covariance parameters are constrained to zero, and what are the asymptotic degrees of freedom?
Why does the unrestricted covariance maximum-likelihood estimate use divisor $n$ rather than $n-1$?
If the partition is changed from $1,1,2$ to $2,2$, what happens to the number of cross-block restrictions? Explain in terms of the partition.
What feature of the statistic makes large values evidence against the block-diagonal null?
Under what distributional condition does block-diagonal covariance imply independence between the variable blocks?