Classical Tests of Canonical Correlations

Why Test Canonical Correlations?

Canonical correlation analysis (CCA) produces a sequence of sample canonical correlations, ordered from largest to smallest. A large first correlation suggests that some linear combination of the variables in one block is associated with some linear combination in the other. But a sample correlation is not, by itself, evidence that the corresponding population association is nonzero. Classical tests summarize one or more canonical correlations and compare the result with a reference distribution.

The main summary is Wilks’ lambda. For an overall test, it combines every canonical correlation; for a sequential test, it combines the correlations that have not yet been tested. Bartlett’s approximation converts the logarithm of this summary into an approximate chi-square statistic. The approximation is useful when its assumptions are reasonable, but it is not an exact finite-sample test.

Wilks’ Lambda and the Hypotheses

Let $X$ and $Y$ be two blocks of measurements on the same $n$ independent observational units. Let the block dimensions be $p$ and $q$, and let $s=\min(p,q)$. Write the ordered sample canonical correlations as $\hat{\rho}_1\geq\hat{\rho}_2\geq\cdots\geq\hat{\rho}_s\geq0$. The corresponding population canonical correlations are $\rho_1,\ldots,\rho_s$.

Definition: Wilks’ lambda for the roots from $k$ through $s$ is $$ \Lambda_k=\prod_{i=k}^{s}(1-\hat{\rho}_i^2),\qquad k=1,\ldots,s. $$ The overall statistic is $\Lambda_1$. A sequential test at position $k$ tests $H_0:\rho_k=\cdots=\rho_s=0$ against $H_a:$ at least one of $\rho_k,\ldots,\rho_s$ is nonzero. Its sample summary is $\Lambda_k$.

Each factor $1-\hat{\rho}_i^2$ lies between zero and one, so Wilks’ lambda does too. A small value indicates that the tested set of roots collectively departs from zero. The product is not a mean correlation: it is a joint measure, and a single strong root can make the product small even when later roots are weak.

The determinant connection explains the product form. In the usual full-rank CCA formulation, the squared canonical correlations are the nonzero eigenvalues associated with the cross-block association after within-block covariance has been accounted for. The determinant of identity minus that association matrix equals the product of one minus its eigenvalues. Thus, for the roots being tested, the determinant reduces to the product of $1-\hat{\rho}_i^2$. Taking logarithms converts the product into a sum:

-\log\Lambda_k=-\sum_{i=k}^{s}\log(1-\hat{\rho}_i^2).

This form is convenient for testing, and it shows why each tested root contributes to the statistic. It also makes clear that the test is joint: the final statistic depends on all roots included in $\Lambda_k$, not just the first root in that set.

Bartlett’s Approximate Chi-Square Test

For the classical test, the paired observations are assumed to be independent and identically distributed draws from a joint multivariate normal population. Both within-block covariance matrices must be nonsingular so that ordinary CCA is defined. Bartlett’s correction is used to improve the chi-square approximation in finite samples. Under the null hypothesis for roots $k,\ldots,s$, the approximation is

\chi^2_k=-\left[n-1-\frac{p+q+1}{2}\right]\log\Lambda_k, \qquad df_k=(p-k+1)(q-k+1).

The reference distribution is approximately chi-square with $df_k$ degrees of freedom. The degrees of freedom count the entries in the remaining cross-block association dimension: after the first $k-1$ roots, the dimensions are $p-k+1$ and $q-k+1$. The correction factor in this standard sequential formula uses the original block dimensions $p$ and $q$, including when $k$ is greater than one.

The p-value is the upper-tail probability $P(\chi^2_{df_k}\geq\chi^2_k)$ under the approximate null distribution. A small p-value is evidence against the stated null, not the probability that the null is true. When $k=1$, the test is overall; when $k>1$, it asks whether the roots remaining after the first $k-1$ are collectively zero. Sequential tests are often reported in order, but a testing plan should specify how the family of tests will be interpreted and whether multiplicity control is needed.

Worked Example: Testing the Last Root

Suppose a hypothetical CCA uses $n=80$ independent paired units and two blocks with $p=q=2$. The first sample canonical correlation is $0.60$, and the second is $0.50$. Test whether the last population canonical correlation is zero at $\alpha=0.05$. Assume the joint observations are reasonably modeled as multivariate normal and both within-block sample covariance matrices are full rank.

For the last-root test, $k=2$, so only the second root enters: $\Lambda_2=1-\hat{\rho}_2^2=1-0.50^2=0.75$. The degrees of freedom are $df_2=(2-2+1)(2-2+1)=1$. The Bartlett correction is $80-1-(2+2+1)/2=76.5$.

Substitution gives

\chi^2_2=-76.5\log(0.75) =-76.5(-0.2876821) \approx22.01,\qquad df=1.

The upper-tail p-value is approximately $2.7\times10^{-6}$, which is below $0.05$. Reject the null that the second population canonical correlation is zero. This conclusion concerns the last root after the first root; it does not say that the second sample correlation is the only source of association in the blocks.

Worked Example: A Sequential Test That Does Not Reject

Consider a second hypothetical analysis with $n=60$, $p=2$, and $q=3$. Suppose the estimated last canonical correlation is $\hat{\rho}_2=\sqrt{0.09}=0.30$. For $k=2$, the null is that the remaining population roots are zero. Assume independent multivariate-normal observations and full-rank within-block covariance matrices.

There is one remaining root, so $\Lambda_2=1-0.30^2=0.91$. Here $df_2=(2-2+1)(3-2+1)=2$, and the correction factor is $60-1-(2+3+1)/2=56$.

\chi^2_2=-56\log(0.91) =-56(-0.0943107) \approx5.281,\qquad df=2.

The upper-tail p-value is approximately $0.0713$, which exceeds $0.05$. Do not reject this sequential null at the stated significance level. This is not proof that the remaining population correlation is exactly zero; the sample does not provide sufficient evidence to reject that null using this approximate test.

Worked Example: Overall Test Across Two Blocks

For a third hypothetical analysis, suppose $n=50$ independent paired units have $p=2$ measurements in block $X$ and $q=3$ in block $Y$. The two sample canonical correlations are $\hat{\rho}_1=\sqrt{0.40}\approx0.6325$ and $\hat{\rho}_2=\sqrt{1/6}\approx0.4082$. Assume joint multivariate normality and nonsingular within-block covariance matrices.

For the overall test, $k=1$. The factors in Wilks’ lambda are $1-0.40=0.60$ and $1-1/6=5/6$, so $\Lambda_1=0.60(5/6)=0.50$. The correction factor is $50-1-(2+3+1)/2=46$, and the degrees of freedom are $df_1=(2)(3)=6$.

\chi^2_1=-46\log(0.50) =46(0.6931472) \approx31.88,\qquad df=6.

The upper-tail p-value is approximately $1.7\times10^{-5}$, below $0.05$. Reject the overall null that both population canonical correlations are zero. The result supports evidence of cross-block association under the model and approximation; it does not establish that one block causes changes in the other.

Assumptions, Diagnostics, and Limits

The classical reference distribution relies on the sampling model, not merely on the arithmetic of the formula. The observations must be independent paired units, the variables should have a joint multivariate normal distribution for the classical likelihood-based calibration, and the within-block covariance matrices must be nonsingular. In practice, the chi-square result is approximate, so its accuracy can be poor in small samples or when the data depart strongly from the assumed model.

Before interpreting a result, check that the number of independent units and the block dimensions make the covariance estimates credible, and verify that the CCA roots and Wilks’ lambda were computed from the same observations and preprocessing. Near-duplicate variables, severe outliers, or very small samples relative to the number of variables can undermine the classical calculation. If a covariance matrix is singular, the ordinary classical test is not directly available; a regularized or otherwise modified analysis needs an inferential approach appropriate to that method.

A small Wilks’ lambda is evidence against the joint null, but it does not identify which measured variables drive the association or guarantee useful out-of-sample prediction. Conversely, failure to reject can reflect weak association, limited sample size, or the approximation’s limitations. These tests answer a specific question about population canonical correlations; interpretation of canonical scores and loadings is a separate task.

Common Mistakes and Alternatives

Using only the largest root in the overall test. The overall Wilks’ lambda multiplies the factors for every root, $\Lambda_1=\prod_{i=1}^{s}(1-\hat{\rho}_i^2)$.
Using the wrong degrees of freedom. For the sequential test at $k$, use $(p-k+1)(q-k+1)$, not always $pq$.
Changing the Bartlett correction for a sequential test. In the stated formula, the correction uses the original $p$ and $q$; the sequential adjustment is reflected in $\Lambda_k$ and $df_k$.
Calling a nonsignificant result proof of no association. A p-value above $\alpha$ means the null was not rejected at that level; it does not establish that the population roots are zero.
Treating the chi-square approximation as exact. It is a classical approximation whose quality depends on the sample size and assumptions.

The main alternative is a resampling test, such as a permutation procedure, when its exchangeability conditions are justified and the classical distributional approximation is not trusted. Resampling is not automatically valid: the permutation scheme must preserve the design and the null being tested. Bartlett’s test is computationally simple and gives a familiar reference distribution under its assumptions; resampling can be more flexible but requires careful design and computation. Neither alternative removes the need to define the hypothesis and the unit of analysis clearly.

Key takeaway: Form the product of $1-\hat{\rho}_i^2$ over exactly the roots in the hypothesis, then use Bartlett’s corrected negative log, the matching sequential degrees of freedom, and an upper-tail chi-square probability. Report the null tested, assumptions, statistic, degrees of freedom, p-value, and contextual conclusion.

Check Your Understanding

Work each problem, then compare your reasoning with the solutions below.

For $p=3$, $q=2$, and squared sample canonical correlations $0.25$ and $0.10$, calculate the overall Wilks’ lambda.
For a sequential test with $p=3$, $q=2$, and $k=2$, find the degrees of freedom.
With $n=40$, $p=2$, $q=2$, and $\Lambda_1=0.80$, calculate the overall Bartlett statistic and degrees of freedom.
Explain why a p-value of $0.12$ at $\alpha=0.05$ does not establish that all population canonical correlations are zero.

Solutions. (1) $\Lambda_1=(1-0.25)(1-0.10)=0.75(0.90)=0.675$. (2) $df_2=(3-2+1)(2-2+1)=2(1)=2$. (3) The correction is $40-1-(2+2+1)/2=36.5$, so $\chi^2_1=-36.5\log(0.80)=36.5(0.2231436)\approx8.145$, with $df=(2)(2)=4$. (4) Since $0.12>0.05$, the null is not rejected at the chosen level. This is insufficient evidence against the null under the test; it is not proof that the population correlations are all zero.