A Worked Redundancy Calculation

Why a Worked Redundancy Calculation Needs Two Checks

Canonical correlation analysis (CCA) finds paired linear combinations of two variable sets that are highly correlated. But a large canonical correlation alone does not tell us how much of the original variables’ variance is represented or predicted. A redundancy calculation combines the correlation of each variable with its own set’s canonical scores and the correlations between the paired scores.

There are two distinct directions to calculate: how well \(Y\) predicts \(X\), and how well \(X\) predicts \(Y\). These need not agree. Before doing the arithmetic, check that the supplied variable-score correlations could come from variable sets of the stated dimensions. In particular, the correlations of a standardized variable with a complete orthonormal basis for its own set must have squared values that sum to one.

Definitions and Calculation Rules

Suppose \(X=(X_1,\ldots,X_p)'\) and \(Y=(Y_1,\ldots,Y_q)'\) are centered random vectors with finite second moments and positive-definite covariance matrices. Assume each variable has positive variance. For ordinary CCA, let \(s=\min(p,q)\). Write the paired canonical variates as \(U_k=a_k'X\) and \(V_k=b_k'Y\), scaled so that each has variance one. Their correlations are \(\rho_k=\operatorname{Corr}(U_k,V_k)\), ordered from largest to smallest. Within each set the canonical variates are uncorrelated, and across sets \(\operatorname{Corr}(U_k,V_l)=0\) when \(k\ne l\).

Definition: The variance-extracted proportion for root \(k\) in the \(X\) set is \(VE_{X,k}=p^{-1}\sum_{j=1}^{p}\operatorname{Corr}(X_j,U_k)^2\). It is the average, across the \(p\) variables, of the squared correlations with that root’s canonical variate. Define \(VE_{Y,k}=q^{-1}\sum_{l=1}^{q}\operatorname{Corr}(Y_l,V_k)^2\) for the \(Y\) set. The directional redundancy indices using the first \(m\) roots, where \(1\le m\le s\), are \(Red_{X|Y}(m)=\sum_{k=1}^{m}VE_{X,k}\rho_k^2\) and \(Red_{Y|X}(m)=\sum_{k=1}^{m}VE_{Y,k}\rho_k^2\). Here \(X|Y\) means variance in \(X\) represented by linear prediction from \(Y\); \(Y|X\) means the reverse.

The factor \(\rho_k^2\) matters: \(VE_{X,k}\) measures how much of the \(X\) variables’ variance is associated with \(U_k\), while \(\rho_k^2\) measures how strongly that canonical variate is associated with its paired variate in \(Y\). Multiplying them gives the root’s contribution to prediction across the sets. The \(Y\)-direction uses \(VE_{Y,k}\), not \(VE_{X,k}\).

Cross-Loadings: A Useful Identity

A structure correlation is a correlation between a variable and a canonical variate from its own set, such as \(\operatorname{Corr}(X_j,U_k)\). A cross-loading is a correlation with a canonical variate from the other set, such as \(\operatorname{Corr}(X_j,V_k)\). The cross-loading identity explains how to calculate redundancy directly from structure correlations.

Theorem: Under the CCA normalization above, for every variable \(X_j\) and root \(k\), \(\operatorname{Corr}(X_j,V_k)=\rho_k\operatorname{Corr}(X_j,U_k)\). Likewise, \(\operatorname{Corr}(Y_l,U_k)=\rho_k\operatorname{Corr}(Y_l,V_k)\).

Proof. The canonical equations imply \(\operatorname{Cov}(X,V_k)=\rho_k\operatorname{Cov}(X,U_k)\) and \(\operatorname{Cov}(Y,U_k)=\rho_k\operatorname{Cov}(Y,V_k)\). For the first identity, take its \(j\)th component and divide by \(\sqrt{\operatorname{Var}(X_j)\operatorname{Var}(V_k)}\). Since \(\operatorname{Var}(V_k)=1\), this gives \(\operatorname{Corr}(X_j,V_k)=\rho_k\operatorname{Corr}(X_j,U_k)\). This division does not assume that \(X_j\) has unit variance; it only requires its variance to be positive. The same argument, using \(\operatorname{Var}(U_k)=1\) and positive \(\operatorname{Var}(Y_l)\), proves the second identity. \(\square\)

Thus the contribution of root \(k\) to \(Red_{X|Y}\) is the average squared cross-loading of the \(X\) variables on \(V_k\): \(VE_{X,k}\rho_k^2=p^{-1}\sum_j\operatorname{Corr}(X_j,V_k)^2\). The corresponding \(Y\)-contribution is the average squared cross-loading on \(U_k\).

Worked Calculations

Worked Example: A Two-Variable Calculation in Both Directions

Suppose \(p=q=2\), the canonical correlations are \(\rho_1=0.8\) and \(\rho_2=0.3\), and the structure correlations in each set are the rows of the matrix \(\begin{pmatrix}0.8&0.6\\-0.6&0.8\end{pmatrix}\). These values are compatible with a two-variable set: each row has squared length \(0.8^2+0.6^2=1\), and the rows are orthogonal.

For either set, the first root’s variance extracted is \(VE_1=(0.8^2+(-0.6)^2)/2=(0.64+0.36)/2=0.50\). For the second root it is \(VE_2=(0.6^2+0.8^2)/2=(0.36+0.64)/2=0.50\).

Therefore, in either direction, using both roots gives \(Red=0.50(0.8^2)+0.50(0.3^2)=0.50(0.64)+0.50(0.09)=0.32+0.045=0.365\). Using only the first root gives \(0.50(0.64)=0.32\). The second root adds \(0.045\).

The equality of the two directional indices here follows from the particular equal variance-extracted proportions, not from a general rule that the two directions must match.

Worked Example: A Compatible Three-By-Two CCA

Let the canonical correlations be \(\rho_1=0.7\) and \(\rho_2=0.4\). Suppose the \(X\)-variable structure correlations are \((0.8,0.6)\), \((0.6,-0.8)\), and \((0,0)\), while the two \(Y\)-variable structure correlations are \((0.8,0.6)\) and \((-0.6,0.8)\). The first two \(X\) rows and both \(Y\) rows have squared lengths one. The third \(X\) variable has no component in the canonical-score space, which is possible because \(p=3\) and only two canonical roots are present.

One construction verifies that these numbers can describe a CCA. Take independent, unit-variance variables \(U_1,U_2,W,E_1,E_2\), and set \(V_1=0.7U_1+\sqrt{0.51}E_1\) and \(V_2=0.4U_2+\sqrt{0.84}E_2\). Then each \(V_k\) has variance one and \(\operatorname{Corr}(U_k,V_k)\) is respectively \(0.7\) and \(0.4\). Define \(X_1=0.8U_1+0.6U_2\), \(X_2=0.6U_1-0.8U_2\), and \(X_3=W\); define \(Y_1=0.8V_1+0.6V_2\) and \(Y_2=-0.6V_1+0.8V_2\). The rotations are orthogonal, so the stated structure correlations follow, all five observed variables have variance one, and the canonical variates can be recovered from the variables within each set.

For \(X\), the variance-extracted proportions are \(VE_{X,1}=(0.8^2+0.6^2+0^2)/3=1/3\) and \(VE_{X,2}=(0.6^2+(-0.8)^2+0^2)/3=1/3\). For \(Y\), \(VE_{Y,1}=(0.8^2+(-0.6)^2)/2=1/2\) and \(VE_{Y,2}=(0.6^2+0.8^2)/2=1/2\).

Using both roots, the variance in \(X\) represented by \(Y\) is \(Red_{X|Y}= (1/3)(0.7^2)+(1/3)(0.4^2)=(0.49+0.16)/3=0.2167\), to four decimal places. The reverse index is \(Red_{Y|X}=(1/2)(0.7^2)+(1/2)(0.4^2)=0.325\). The directions differ because the averages are over different variable sets, and the variance-extracted proportions are different.

The cross-loading check gives the same \(X\)-direction result. For root 1, the \(X\) cross-loadings on \(V_1\) are \(0.7(0.8)=0.56\), \(0.7(0.6)=0.42\), and \(0\). Their squared average is \((0.56^2+0.42^2+0^2)/3=(0.3136+0.1764)/3=0.1633\). For root 2 it is \((0.4^2(0.6^2)+0.4^2((-0.8)^2)+0)/3=(0.0576+0.1024)/3=0.0533\). Their sum is \(0.2167\), as above.

Worked Example: Why the Two Directions Must Be Reported Separately

Use the compatible three-by-two setup above, but retain only the first canonical root. In the direction predicting \(X\) from \(Y\), the index is \(Red_{X|Y}(1)=VE_{X,1}\rho_1^2=(1/3)(0.49)=0.1633\). In the reverse direction it is \(Red_{Y|X}(1)=VE_{Y,1}\rho_1^2=(1/2)(0.49)=0.245\). Both calculations use the same canonical correlation, but they use different variance-extracted proportions and therefore give different answers.

Adding the second root changes the indices by \((1/3)(0.4^2)=0.0533\) for \(X|Y\), and \((1/2)(0.4^2)=0.0800\) for \(Y|X\). Thus the totals become \(0.1633+0.0533=0.2167\) and \(0.245+0.0800=0.325\), respectively. These are proportions of the standardized variables’ average variance, not percentages of individuals correctly classified or a guarantee of predictive performance in new data.

Why the Indices Have This Form

The redundancy formula can also be derived from linear prediction. Consider predicting a standardized \(X_j\) from the canonical variates \(V_1,\ldots,V_s\). They are mutually uncorrelated and have variance one. The least-squares linear predictor is therefore \(\widehat X_j=\sum_{k=1}^{s}\operatorname{Cov}(X_j,V_k)V_k\). The residual is uncorrelated with every \(V_k\), so the variance explained by this projection is \(\sum_{k=1}^{s}\operatorname{Cov}(X_j,V_k)^2\). Because \(X_j\) and \(V_k\) have unit variance in this derivation, each covariance is their correlation.

The canonical variates form coordinates for the corresponding variable set; any additional canonical coordinates when the dimensions differ have zero correlation with the opposite set. Consequently, the projection onto the \(V_k\) captures the part of \(X_j\)'s linear prediction from \(Y\). Applying the cross-loading identity and averaging over \(j\) yields \[ \frac{1}{p}\sum_{j=1}^{p}\sum_{k=1}^{s}\operatorname{Corr}(X_j,V_k)^2 =\sum_{k=1}^{s}\rho_k^2\left(\frac{1}{p}\sum_{j=1}^{p}\operatorname{Corr}(X_j,U_k)^2\right) =\sum_{k=1}^{s}\rho_k^2VE_{X,k}. \] The equality follows by exchanging finite sums and substituting \(\operatorname{Corr}(X_j,V_k)=\rho_k\operatorname{Corr}(X_j,U_k)\). This proves that the directional redundancy is the average variance explained in the target set by the linear projection on the predictor set’s canonical coordinates. Reversing \(X\) and \(Y\) gives the other direction.

Compatibility Checks and Common Pitfalls

A calculation can be arithmetically correct but statistically impossible if its structure correlations cannot arise from the stated dimensions. For example, when \(q=2\) and the two \(V\) scores span the \(Y\) space, every standardized \(Y_l\) is represented by its projections on those two orthonormal scores. Its squared structure correlations must sum to one. A proposed row \((0.9,0)\) fails this check because \(0.9^2+0^2=0.81\), not one. A row \((0.2,0.7)\) also fails because \(0.2^2+0.7^2=0.53\). Such inputs should not be used to report a redundancy index as though they came from that CCA.

Pitfall: treating the two directions as identical. The wrong reasoning is that both indices use the same \(\rho_k^2\), so they must be equal. The correction is that \(Red_{X|Y}\) weights each root by \(VE_{X,k}\), whereas \(Red_{Y|X}\) weights it by \(VE_{Y,k}\).
Pitfall: reporting variance extracted as redundancy. \(VE_{X,k}\) describes association within the \(X\) set. It is not variance predicted across sets until it is multiplied by \(\rho_k^2\).
Pitfall: using structure correlations from the wrong set. For \(Red_{X|Y}\), average the squared correlations of \(X\) variables with \(U_k\), then weight by \(\rho_k^2\). Do not substitute \(VE_{Y,k}\).
Pitfall: skipping the dimension check. When the canonical scores span a whole variable set, each standardized variable’s squared correlations with that complete basis sum to one. Check this before trusting supplied values.

Key takeaway: Calculate \(VE_{X,k}\) and \(VE_{Y,k}\) separately, multiply each by \(\rho_k^2\), and sum over the roots retained. State which set is being predicted, and check that the structure correlations are compatible with the variable-set dimensions.

Check Your Understanding

Work through these questions by identifying the direction and checking the inputs before calculating.

In words, what does \(VE_{X,k}\) average, and what additional factor converts it into the root-\(k\) contribution to \(Red_{X|Y}\)?
If \(VE_{X,1}=0.40\) and \(\rho_1=0.5\), calculate the first-root contribution to \(Red_{X|Y}\).
Why can \(Red_{X|Y}\) differ from \(Red_{Y|X}\), even though both use the same canonical correlations?
A standardized variable in a two-dimensional set has correlations \(0.6\) and \(0.8\) with a complete pair of orthonormal canonical scores. Check whether the squared correlations satisfy the required sum.
State the cross-loading identity for \(X_j\) and explain how it relates structure correlations to redundancy.