A Worked Sequential Testing Example

From Sample Roots to a Sequence of Tests

Canonical correlation analysis can produce several sample canonical correlations, but a list of estimates alone does not say how many population associations the data support. Sequential testing addresses this question in stages: first, test whether any canonical association is present; if that null is rejected, test whether any association remains after the first canonical pair; continue in the same way until a null is not rejected or no roots remain.

The calculations use the squared canonical correlations, often called the canonical roots. The hypotheses are nested: the global null says all population canonical correlations are zero, while each later null concerns only the roots left after the earlier ones. A rejection at a stage is evidence against that stage’s remaining-root null; it does not show that every remaining root is nonzero.

Definition: Let \(p\) and \(q\) be the numbers of variables in the two sets, and let \(m=\min(p,q)\). Write the ordered sample canonical correlations as \(\hat\rho_1\ge \hat\rho_2\ge\cdots\ge\hat\rho_m\ge 0\), and the corresponding sample canonical roots as \(\hat\lambda_i=\hat\rho_i^2\). For \(s=0,\ldots,m-1\), the remaining-root null is \(H_s:\rho_{s+1}=\cdots=\rho_m=0\). Its sample Wilks statistic is \(\Lambda_s=\prod_{i=s+1}^{m}(1-\hat\lambda_i)\).

Thus \(H_0\) is the global null, and \(H_1\) tests whether any population canonical correlation remains after the first. In general, \(H_s\) concerns the roots after the first \(s\). Under the classical multivariate-normal model, Bartlett’s chi-square approximation uses

T_s=-c_n\log\Lambda_s,\qquad c_n=n-1-\frac{p+q+1}{2},\qquad T_s\ \dot\sim\ \chi^2_{(p-s)(q-s)}.

Here \(n\) is the sample size and \(\dot\sim\) denotes an approximate null distribution. This approximation is for independent observations from a nonsingular joint multivariate normal distribution, with fixed \(p\) and \(q\) as \(n\) grows; the correction factor must be positive. It is not an exact finite-sample distribution. At each stage, compare \(T_s\) with an upper-tail chi-square reference distribution using \((p-s)(q-s)\) degrees of freedom.

Why the Tail Product and Degrees of Freedom Change

The statistic at stage \(s\) uses only roots \(s+1,\ldots,m\). The next stage removes one more root from that product, and its null concerns a smaller remaining block of associations. The following identities make those changes explicit.

Derived result: For \(s=0,\ldots,m-2\), the tail products satisfy \(\Lambda_s=(1-\hat\lambda_{s+1})\Lambda_{s+1}\). Consequently, \(T_s-T_{s+1}=-c_n\log(1-\hat\lambda_{s+1})\), when both statistics use the same Bartlett correction \(c_n\).

To verify the identity, separate the first factor in the product defining \(\Lambda_s\):

\Lambda_s =\prod_{i=s+1}^{m}(1-\hat\lambda_i) =(1-\hat\lambda_{s+1})\prod_{i=s+2}^{m}(1-\hat\lambda_i) =(1-\hat\lambda_{s+1})\Lambda_{s+1}.

Taking logarithms gives \(\log\Lambda_s=\log(1-\hat\lambda_{s+1})+\log\Lambda_{s+1}\). Multiplication by \(-c_n\) yields the stated difference. This is an algebraic relation between the sample statistics, not a claim that each stage tests a separate root in isolation: \(T_s\) tests the joint remaining-root null.

The degrees of freedom decrease because the remaining hypothesis has dimensions \(p-s\) and \(q-s\). Its Bartlett reference has \((p-s)(q-s)\) degrees of freedom; after one more root is removed, this becomes \((p-s-1)(q-s-1)\). The degrees of freedom are not the number of roots being tested.

Start with the global null.
Use \(\Lambda_0=\prod_{i=1}^{m}(1-\hat\rho_i^2)\), with \(pq\) degrees of freedom.

Proceed only after rejection.
If \(H_s\) is rejected, calculate the next tail product \(\Lambda_{s+1}\) and test the remaining-root null with \((p-s-1)(q-s-1)\) degrees of freedom.

Stop at the first non-rejection.
Report that the data did not provide sufficient evidence against that remaining-root null. Do not describe it as proof that the remaining population roots are zero.

Worked Sequential Tests

Worked Example: A Three-by-Two Analysis

Suppose \(n=120\), \(p=3\), \(q=2\), and the sample canonical correlations are \(\hat\rho_1=0.50\) and \(\hat\rho_2=0.30\). Test the global null and then the null for the remaining root after the first pair. Use significance level \(0.05\).

First square the correlations: \(\hat\lambda_1=0.50^2=0.25\) and \(\hat\lambda_2=0.30^2=0.09\). The Bartlett correction is

c_n=120-1-\frac{3+2+1}{2}=119-3=116.

For the global null \(H_0:\rho_1=\rho_2=0\), the tail product and statistic are

\Lambda_0=(1-0.25)(1-0.09)=0.75(0.91)=0.6825, \qquad T_0=-116\log(0.6825)\approx44.31.

The degrees of freedom are \(pq=3(2)=6\). The 5% upper-tail critical value for \(\chi^2_6\) is about 12.59. Since \(44.31>12.59\), reject the global null.

Next test \(H_1:\rho_2=0\), the null that no association remains after the first pair. Here \(\Lambda_1=1-0.09=0.91\), so

T_1=-116\log(0.91)\approx10.94,\qquad \text{degrees of freedom}=(3-1)(2-1)=2.

The 5% critical value for \(\chi^2_2\) is about 5.99. Since \(10.94>5.99\), reject \(H_1\) as well. In this sequence, the results provide evidence against both the global null and the null that the second population canonical correlation is zero. The second conclusion follows from its own remaining-root test, not merely from the fact that the global null was rejected.

Worked Example: A Global Rejection Followed by a Non-Rejection

Suppose \(n=80\), \(p=q=2\), and the sample canonical correlations are \(0.40\) and \(0.15\). Calculate both stages at the 5% level.

The squared correlations are \(0.16\) and \(0.0225\), and the correction is \(c_n=80-1-(2+2+1)/2=76.5\). At the global stage,

\Lambda_0=(1-0.16)(1-0.0225)=0.84(0.9775)=0.8211, \qquad T_0=-76.5\log(0.8211)\approx15.08.

The global test has \(pq=4\) degrees of freedom. Its 5% critical value is about 9.49, so reject the global null.

For the remaining-root test after the first pair, \(\Lambda_1=1-0.0225=0.9775\). Therefore,

T_1=-76.5\log(0.9775)\approx1.74,\qquad \text{degrees of freedom}=(2-1)(2-1)=1.

The 5% critical value for \(\chi^2_1\) is about 3.84, and \(1.74<3.84\). Do not reject \(H_1\). The sequence supports an association somewhere in the two-dimensional sets, but this test does not provide sufficient evidence that a second population canonical correlation remains after the first.

Worked Example: Stop When a Remaining-Root Null Is Not Rejected

Suppose \(n=150\), \(p=4\), \(q=3\), and the sample canonical correlations are \(0.35\), \(0.25\), and \(0.10\). Use the 5% level and follow the sequential procedure.

The roots are \(0.1225\), \(0.0625\), and \(0.01\), and \(c_n=150-1-(4+3+1)/2=145\). For the global test,

\Lambda_0=(0.8775)(0.9375)(0.99)=0.8144297, \qquad T_0=-145\log(0.8144297)\approx29.76.

There are \(pq=12\) degrees of freedom; the 5% critical value is about 21.03. Reject the global null and proceed.

The next test concerns roots two and three. Its tail product and statistic are

\Lambda_1=(0.9375)(0.99)=0.928125, \qquad T_1=-145\log(0.928125)\approx10.82.

The degrees of freedom are \((4-1)(3-1)=6\), whose 5% critical value is about 12.59. Since \(10.82<12.59\), do not reject \(H_1\). Stop here. Although the global test rejected, the sequential procedure has not established evidence for an additional population canonical correlation beyond the first. Do not continue to treat the third-root test as an established result of this sequence.

Interpretation and a Common Pitfall

A sequential result should be reported stage by stage: give the tested null, the value of \(\Lambda_s\), the Bartlett statistic \(T_s\), the degrees of freedom, the chosen significance level, and the decision. State conclusions in terms of evidence against the tested null. A non-rejection is not proof that the null is true, and a rejection of \(H_s\) means that the full set of remaining roots cannot all be zero under that null; it does not identify which of those roots is responsible.

A common mistake is to test each sample canonical correlation separately using the global degrees of freedom \(pq\). That combines two errors: the remaining-root statistic uses only the tail product, and its reference degrees of freedom change with \(s\). For example, in the three-by-two analysis, the second-stage test uses only \(1-\hat\rho_2^2\) and has two degrees of freedom, not six. Another mistake is to interpret a large first sample correlation as enough to conclude that later roots are significant. The later conclusion requires the corresponding remaining-root test.

Key takeaway: At stage \(s\), use the product \(\prod_{i=s+1}^{m}(1-\hat\rho_i^2)\), multiply its negative logarithm by the Bartlett correction, and compare with a chi-square distribution having \((p-s)(q-s)\) degrees of freedom. Test in order, and stop at the first non-rejection.

Check Your Understanding

For each question, distinguish the tail product, the degrees of freedom, and the conclusion the test can support.

For \(p=4\), \(q=2\), and \(s=1\), how many degrees of freedom does the remaining-root test use?
Write \(\Lambda_1\) in terms of the sample canonical correlations when \(m=3\).
If the global null is rejected but \(H_1\) is not rejected, what can you conclude, and what should you avoid claiming?
Why does a test of \(H_s\) not test only the single root \(\hat\rho_{s+1}\)?
What quantities should be reported for each stage of a sequential analysis?

A Worked Sequential Testing Example

What You'll Learn