Worked Example of Classical CCA

From Paired Measurements to Canonical Scores

Canonical correlation analysis (CCA) looks for linear combinations of two sets of measurements that are maximally correlated across the same observational units. In a multi-omics setting, the two blocks might be measurements from two assays on the same participants. The pairing matters: each row of one block must correspond to the same unit as the row in the other block.

The calculation has several linked outputs. Canonical weights define the linear combinations, canonical scores are the resulting values for each unit, and canonical correlations quantify the association between paired scores. Correlations between original variables and scores, often called canonical loadings, help describe how the variables relate to those combinations.

For the classical calculation below, assume the rows are correctly paired and centered, and that both within-block sample covariance matrices are positive definite. Positive definiteness makes their inverses available and rules out exact linear dependencies among variables within a block. In a sample, the resulting quantities describe this dataset; interpreting them beyond it requires appropriate sampling assumptions and validation.

A Small Dataset and Its Covariances

Consider five hypothetical units measured on two variables in each of two blocks. The data are constructed from four centered, mutually orthogonal vectors \(a_1,a_2,c_1,c_2\), each with sample variance 1. Define the block variables by \(x_1=a_1\), \(x_2=a_2\), \(y_1=0.8a_1+0.6c_1\), and \(y_2=0.6a_2+0.8c_2\). The following entries are rounded to three decimal places; the exact construction, not the rounded display, determines the calculations.

Because the four generating vectors are centered, orthogonal, and have sample variance 1, the \(x\) variables each have variance 1 and covariance 0. The same holds for the \(y\) variables: for example, \(\operatorname{var}(y_1)=0.8^2+0.6^2=1\), and \(y_1\) and \(y_2\) use orthogonal generating vectors. The cross-block covariance is \(\operatorname{cov}(x_1,y_1)=0.8\), \(\operatorname{cov}(x_2,y_2)=0.6\), and the two off-diagonal covariances are 0. Thus, in the stated variable orders,

This construction gives positive-definite within-block covariance matrices, as required. In practical work, the same matrices are calculated from centered observations; the rounded table can produce small departures from the exact values, so calculations should use unrounded data when available.

Why These Weights Are Optimal

With identity within-block covariances, the denominator of the score correlation is 1 whenever both weight vectors have Euclidean length 1. Thus the optimization reduces to maximizing \(a^\mathsf{T}S_{XY}b\) subject to \(a^\mathsf{T}a=b^\mathsf{T}b=1\). For fixed \(a\), the maximum over unit-length \(b\) is \(\lVert S_{XY}^\mathsf{T}a\rVert\), by the Cauchy–Schwarz inequality. Maximizing this quantity over unit \(a\) yields the largest singular value of \(S_{XY}\). Here the singular values are 0.8 and 0.6, so 0.8 is the maximum possible first correlation.

More generally, when the within-block covariances are positive definite but not identity matrices, the classical CCA problem can be expressed using the matrix \(S_{XX}^{-1/2}S_{XY}S_{YY}^{-1/2}\). Its singular values are the canonical correlations. The inverse square roots rescale each block by its covariance structure; in this example they are identity matrices, so the singular values are directly those of \(S_{XY}\).

Scores, Loadings, and Interpretation

A canonical weight is a coefficient used to form a score; a canonical loading is a correlation between an original variable and a score. These are different quantities, even though they can coincide in simple examples. Here the first scores are \(U_1=x_1\) and \(V_1=y_1\). Their within-block loadings, listed in the variable orders \(x_1,x_2\) and \(y_1,y_2\), are \((1,0)\) in each block: \(x_1\) correlates 1 with \(U_1\), \(x_2\) correlates 0, and correspondingly \(y_1\) correlates 1 with \(V_1\) while \(y_2\) correlates 0. The first pair’s cross-loadings are \(\operatorname{corr}(x_1,V_1)=0.8\), \(\operatorname{corr}(x_2,V_1)=0\), \(\operatorname{corr}(y_1,U_1)=0.8\), and \(\operatorname{corr}(y_2,U_1)=0\).

Canonical signs are arbitrary: replacing both weights in a pair by their negatives reverses both score axes but preserves the correlation and the substantive relationship. Variable order is not arbitrary when reporting loading vectors. In particular, reporting \((1,0)\) for the second pair would be wrong in the stated order; that vector would select the first variable, not the second.

Diagnostics, Alternatives, and Limits

Before interpreting a CCA calculation, check the pairing, centering, variable order, covariance calculations, and score variances. Recalculate score correlations from the actual scores and confirm that they match the reported canonical correlations, allowing for rounding. For subsequent pairs, check that each new score is uncorrelated with earlier scores within its own block. If a covariance matrix is singular or nearly singular, the classical inverse-based formulation may be undefined or unstable; a high apparent association can then reflect redundant variables or overfitting rather than reproducible shared structure.

A useful check is to compare the computed score correlation with the optimization value. In this example, the first computed correlation, 0.8, agrees with the largest singular value. A discrepancy would suggest an error in centering, covariance scaling, weight normalization, or variable order. Rounding a displayed data table and then recomputing from those rounded entries can also create small discrepancies, but not a substantial disagreement.

CCA and two-block PLS answer related but distinct optimization questions. CCA maximizes score correlation, normalizing by the within-block score variances; two-block PLS seeks directions of high cross-block covariance. In this example the within-block covariance matrices are identity matrices, so the first directions coincide and the leading cross-covariance is also 0.8. That coincidence is not a general equivalence: when within-block variances differ, the correlation normalization can produce different directions from covariance maximization.

This example is deliberately small and well-conditioned. It does not establish statistical significance, generalize to a population, or show that either block causes the other. With many variables and few paired units, classical CCA can fit sample-specific associations; stable interpretation then depends on suitable validation and other design choices. Association is also not evidence that a score represents a single biological mechanism.

• Wrong reasoning: “The second pair’s loadings are \((1,0)\), because it is the first loading when I discuss that pair.”
  Correction: Keep the stated variable order fixed. For \(U_2=x_2\), the loadings on \(x_1,x_2\) are \((0,1)\); for \(V_2=y_2\), those on \(y_1,y_2\) are also \((0,1)\).
• Wrong reasoning: “The weights look reasonable, so their correlation must be canonical.”
  Correction: Canonical weights solve a constrained optimization. Check their score variances and compare the resulting correlation with the relevant singular value.
• Wrong reasoning: “A loading of 1 means the original variable and the other block’s score correlate 1.”
  Correction: A within-block loading is the correlation with the score formed from that same block. Here the first within-block loadings are 1, while the cross-loadings are 0.8.