Assumptions and Diagnostics for Classical CCA

Why Diagnostics Matter in Classical CCA

Classical canonical correlation analysis (CCA) searches for linear combinations of two measured blocks whose correlation is large. A strong in-sample canonical correlation is therefore not, by itself, evidence that the relationship is well represented by a stable linear pattern or that a classical test is well calibrated. A small number of unusual observations can drive the correlation, strong dependence among variables within a block can make the estimated weights unstable, and nonlinearity can be hidden by a single summary.

Diagnostics do not prove that assumptions hold. They identify patterns that make the analysis more or less credible and help distinguish a stable, broadly supported association from one that depends on a modeling choice or a few observations. In multi-omics applications, each row must represent the same biological unit in both blocks; a sample-order mismatch is a data error, not a statistical assumption that a diagnostic plot can repair.

Definition: Let \(X_i\) and \(Y_i\) be the two measured vectors for paired unit \(i\), with \(p\) and \(q\) variables, respectively. The combined vector is \(Z_i=(X_i^\mathsf{T},Y_i^\mathsf{T})^\mathsf{T}\). The usual classical inferential framework assumes independent, identically distributed paired vectors and, for exact multivariate-normal theory, \(Z_i\sim N_{p+q}(\mu,\Sigma)\), where \(\Sigma\) is positive definite. Interpreting CCA as a linear association also requires checking whether relevant relationships are adequately represented by linear combinations.

The joint normal model is stronger than saying that each variable separately looks approximately normal. It concerns the joint distribution of all variables in both blocks. Classical tests can also use approximations, but their calibration may be unreliable when the sample is small relative to the number of variables, the joint distribution is strongly nonnormal, or observations are dependent. Classical CCA additionally requires nonsingular within-block sample covariance matrices to compute ordinary canonical weights. The consequences and remedies for singular covariance matrices are a separate issue; here, an apparent lack of full rank is treated as a warning that ordinary CCA may not be applicable.

Check Linearity in the Measured Data

Start with scatterplots of scientifically relevant variable pairs across the blocks, as well as plots within each block. Add smooth curves or a straight-line fit when useful. Curvature, thresholds, clusters, or a changing spread can indicate that a linear summary is inadequate. A scatterplot matrix can be large, so prioritize pairs with clear scientific relevance or notable marginal relationships, then inspect the observations that depart from the general pattern.

Plots of canonical scores are useful for describing the fitted linear summaries, but they are not an independent confirmation of linearity: CCA selects weights to maximize the in-sample correlation of those scores. Inspecting the original measurements, and checking whether patterns persist under reasonable sensitivity analyses, gives more informative evidence. A residual plot from a simple linear fit can make curvature especially visible.

Worked Example: A Curved Pairwise Relationship

Suppose a hypothetical paired dataset includes measurements \(x=(-2,-1,0,1,2)\) and \(y=(4,1,0,1,4)\). These five pairs are used only to illustrate a diagnostic calculation. Fit the ordinary least-squares line predicting \(y\) from \(x\).

The means are \(\bar{x}=0\) and \(\bar{y}=2\). The fitted slope is

b_1=\frac{\sum_i(x_i-\bar{x})(y_i-\bar{y})}{\sum_i(x_i-\bar{x})^2} =\frac{(-2)(2)+(-1)(-1)+(0)(-2)+(1)(-1)+(2)(2)}{4+1+0+1+4} =\frac{0}{10}=0.

The intercept is \(b_0=\bar{y}-b_1\bar{x}=2\), so the fitted line is \(\hat{y}=2\). The residuals in the listed order are \(2,-1,-2,-1,2\). Their systematic pattern—positive at both ends and negative near the center—is evidence of curvature, not random scatter around a straight-line relationship. Indeed, these data follow \(y=x^2\) at the observed points. In a real analysis, this pattern would prompt closer inspection of the original paired variables and consideration of whether a linear CCA summary is scientifically adequate; it would not justify claiming that all relationships between the blocks are quadratic.

Assess Multivariate Distribution and Unusual Observations

Univariate histograms and Q–Q plots can reveal skewness or extreme marginal values, but they cannot establish joint multivariate normality. A targeted multivariate check uses the squared Mahalanobis distance from an observation \(z_i\) to the sample center:

D_i^2=(z_i-\bar{z})^\mathsf{T}S_Z^{-1}(z_i-\bar{z}),

where \(\bar{z}\) and \(S_Z\) are the sample mean vector and covariance matrix for the combined measurements. Under a multivariate normal model with known parameters, this distance has a chi-square distribution with degrees of freedom equal to the vector dimension. When the mean and covariance are estimated from the same sample, that reference is only an approximate diagnostic guide; the finite-sample distribution differs, and the approximation can be poor when the dimension is large relative to the sample size. A Q–Q plot of ordered distances against the relevant chi-square reference can reveal whether the upper tail is unusually heavy.

A flagged distance is not proof that an observation is erroneous or that the data are nonnormal. Check sample identifiers, measurement quality, and the observation’s scientific context. Also inspect marginal plots: an observation can be unusual only in a combination of variables, even when no single coordinate looks extreme.

Worked Example: Flagging a Jointly Unusual Observation

In a hypothetical four-variable diagnostic, suppose a standardized observation has coordinates \(z=(3,2,1,1)\), with sample center zero and covariance matrix treated as the identity matrix for this hand calculation. Its squared Mahalanobis distance is

D^2=z^\mathsf{T}z=3^2+2^2+1^2+1^2=15.

For four dimensions, the 99th percentile of \(\chi^2_4\) is approximately 13.28. Since \(15>13.28\), this point would be flagged for follow-up by this approximate reference. The calculation does not show that the observation is invalid or that the joint distribution fails normality. In actual data, the center and covariance are estimated, and the identity-covariance simplification does not apply unless justified by the chosen coordinates. The appropriate response is to check the sample and assess how much the CCA results depend on the point.

Check Within-Block Collinearity and Influence

Collinearity among variables inside either block is distinct from cross-block association. When two or more variables in one block are highly dependent, many weight vectors can produce similar canonical scores. The canonical correlation may remain fairly stable while the individual weights change substantially across samples. Inspect within-block correlation matrices, pairwise scatterplots, and—where appropriate—eigenvalues or condition indices. A variance inflation factor (VIF) for a variable is \(1/(1-R^2)\), where \(R^2\) is obtained by regressing that variable on the other variables in the same block. There is no universal VIF cutoff that determines whether CCA is acceptable; values should be interpreted with the sample size, purpose, and weight stability in mind.

Influence asks whether a particular observation materially changes the fitted canonical relationships. Refit CCA after omitting a potentially influential observation, or use a systematic leave-one-out analysis. Compare canonical correlations and scores, and compare weight directions only after accounting for their arbitrary signs. When canonical correlations are close, their associated directions may also exchange order or rotate within a nearly tied subspace; comparing individual weight vectors mechanically can then be misleading. Record whether conclusions about association, interpretation, or inference change under these checks.

Worked Example: Quantifying Within-Block Collinearity

Suppose two standardized variables in a hypothetical \(X\) block have sample correlation \(r=0.96\). In the regression of one on the other, \(R^2=r^2=0.9216\). The VIF for either variable is therefore

\mathrm{VIF}=\frac{1}{1-R^2} =\frac{1}{1-0.9216} =\frac{1}{0.0784} \approx 12.76.

For these two variables, the within-block correlation matrix has determinant \(1-r^2=0.0784\), a small positive value. This illustrates strong, though not exact, linear dependence. It warns that canonical weights may be sensitive to modest sampling changes; it does not by itself show that the cross-block association is spurious. A practical follow-up is to inspect the pairwise plot and compare the fitted directions under reasonable perturbations or observation omissions.

Worked Example: An Influence Check and Its Inferential Consequence

Consider a hypothetical paired dataset of 40 units and two variables in each block. The leading canonical correlation is \(0.91\) in the full data and \(0.68\) after omitting a flagged observation. The absolute change is

$$ |0.91-0.68|=0.23. $$

A change of 0.23 is substantial enough to investigate rather than report the full-data result without qualification. Suppose, separately, that the full-data classical omnibus test has an approximate chi-square statistic of 11.25 with 4 degrees of freedom. For \(\chi^2_4\), the upper-tail probability is

P(\chi^2_4\ge 11.25) =e^{-11.25/2}\left(1+\frac{11.25}{2}\right) \approx 0.02389.

At \(\alpha=0.05\), this nominal p-value is below the significance level. It is evidence against the test’s null hypothesis under its reference assumptions, not the probability that the null is true. Because the leading correlation changes markedly when one point is omitted, the investigator should report that sensitivity and examine whether the classical test’s assumptions and calibration are credible. The calculation does not imply a p-value for the reduced dataset; that would require refitting and retesting it.

A Practical Diagnostic Sequence

A disciplined sequence makes checks easier to reproduce and prevents a single plot from being treated as a pass/fail certificate.

Verify the paired design.
Confirm that each row in the two blocks refers to the same unit and that observations can reasonably be treated as independent. Identify repeated measures, families, batches, or other dependence that may require a different inferential model.

Inspect marginal and pairwise patterns.
Use distributions and selected scatterplots to look for extreme values, clusters, curvature, and changing spread. Check original variables, not only the canonical scores.

Check joint unusualness and block dependence.
Use multivariate-distance plots as approximate flags, and examine within-block correlations and collinearity measures. Relate each flag to the available sample size and dimension.

Assess sensitivity.
Refit after investigating influential observations and compare canonical correlations and interpretable score directions. Document material changes rather than choosing the result that looks most favorable.

Alternatives, Limitations, and Common Mistakes

The right response depends on which assumption is doubtful and what the analysis is meant to establish. If plots show curvature, a linear CCA may be a poor description even if its computation is stable; a scientifically motivated transformation or a method designed for nonlinear relationships changes the question being answered. If normality seems doubtful, robust or resampling-based inference may be considered, but its validity depends on the design and on the resampling scheme. If observations are dependent, simply resampling individual rows or applying an independent-observation test is not a fix. If within-block collinearity is severe, ordinary canonical weights may be unstable; other approaches can change the estimation target and should be justified rather than presented as a diagnostic repair.

Diagnostics have limits. Small samples provide little power to reveal distributional departures, while large samples can make minor departures visible. A clean set of plots cannot verify independence, and the absence of a flagged distance does not prove multivariate normality. Classical tests should therefore be interpreted with their assumptions and sensitivity evidence in view, not as a mechanical consequence of a single threshold.

Mistake: treating canonical-score correlation as proof of a linear relationship. The weights were selected to maximize that in-sample correlation. Inspect relevant original-variable plots and assess sensitivity.
Mistake: treating a Mahalanobis-distance flag as proof of an error. A flag identifies an unusual multivariate pattern under an approximate reference. Check the record and report the effect of the observation.
Mistake: equating a large VIF with a large cross-block association. VIF concerns dependence within a block and potential instability of weights; cross-block association is a different quantity.
Mistake: assuming an apparently significant p-value settles the analysis. A classical p-value is calibrated under its model assumptions. If an influential point or major distributional departure undermines that calibration, report the limitation and investigate sensitivity.

Key takeaway: Classical CCA diagnostics ask whether the linear summaries are adequate, the paired observations support the inferential reference, and the fitted directions are stable. No single diagnostic proves these conditions; a defensible analysis combines targeted plots, multivariate checks, and transparent sensitivity analyses.

Check Your Understanding

Answer each question, then compare with the solutions below.

Why is an approximately normal histogram for every measured variable insufficient evidence for joint multivariate normality?
For two variables with within-block correlation \(r=0.90\), calculate their VIF when each is regressed on the other.
A multivariate-distance diagnostic flags one observation. What two follow-up actions are appropriate before deciding how to report the CCA?
Why should a change in individual canonical weights be interpreted cautiously when two canonical correlations are nearly tied?

Solutions. (1) Marginal histograms describe each variable separately but not their joint dependence or joint tail behavior; joint multivariate normality is a statement about the combined vector. (2) \(R^2=0.90^2=0.81\), so \(\mathrm{VIF}=1/(1-0.81)=1/0.19\approx5.263\). (3) Verify the sample and measurement record, then refit with and without the observation and compare canonical results; do not label it an error solely because it was flagged. (4) Nearly tied canonical correlations can permit directions to rotate or exchange order with little change in fit, so weight-by-weight comparisons may not represent a meaningful change in the underlying fitted subspace.