Deflation Choices in Two-Block PLS

Why Deflation Changes Later Components

In two-block partial least squares (PLS), the first pair of weights is chosen to find scores with strong cross-block association. To obtain further components, the data blocks are modified after each pair is extracted. This modification is called deflation. A later component is then computed from the residual blocks, so the deflation rule helps determine what that component can represent.

There is not one universal residualization rule for every two-block PLS algorithm. This tutorial compares two explicit choices: symmetric score deflation, which removes each block’s own score, and X-score deflation, which uses the X-block score to residualize both blocks. These choices should not be treated as interchangeable: in general, they produce different residual cross-products and different later components.

Definition: Let \(X\) and \(Y\) be centered data matrices with the same \(n\) matched observational units in their rows. A two-block PLS-SVD component is obtained from the leading left and right singular vectors \(w\) and \(c\) of the cross-product matrix \(X'Y\). Its scores are \(t=Xw\) and \(u=Yc\). A deflation rule subtracts fitted rank-one terms from \(X\), \(Y\), or both before the next component is computed.

The description assumes the rows of the two blocks are correctly matched, both blocks use the same observations, and the scores used for deflation are nonzero. Centering is assumed here so that the analysis concerns variation about block means. Unless variables are explicitly rescaled, the cross-product and components depend on their measurement units and scales. No hypothesis test or population-level inference follows from the component calculation alone.

Two Deflation Rules

For a nonzero score vector \(t\), the least-squares coefficient row for regressing every column of \(X\) on \(t\) is \(p'=t'X/(t't)\). Thus \(tp'\) is the fitted rank-one matrix, and \(X-tp'\) is the residual block. The corresponding coefficient row for regressing \(Y\) on a nonzero score \(u\) is \(q'=u'Y/(u'u)\).

p'=\frac{t'X}{t't},\qquad q'=\frac{u'Y}{u'u}.

Under symmetric score deflation, use the score from its own block to residualize that block:

X_{\mathrm{sym}}=X-tp',\qquad Y_{\mathrm{sym}}=Y-uq'.

Under X-score deflation, residualize both blocks using the X score. The X block is unchanged from the symmetric rule, while the Y block is regressed on \(t\):

X_{\mathrm{X}}=X-tp',\qquad Y_{\mathrm{X}}=Y-t\,\frac{t'Y}{t't}.

The next two-block PLS-SVD component is calculated from the relevant residual cross-product, \(X_{\mathrm{sym}}'Y_{\mathrm{sym}}\) or \(X_{\mathrm{X}}'Y_{\mathrm{X}}\). A leading singular value of zero means there is no nonzero cross-block component left under that rule. Singular vectors may be multiplied by \(-1\) without changing the component’s substance; signs alone are not meaningful.

Why the Residual Cross-Products Differ

Let \(P_t=tt'/(t't)\) be the projection matrix onto the one-dimensional span of \(t\), and let \(P_u=uu'/(u'u)\) be the projection onto the span of \(u\). These matrices are defined when their respective scores are nonzero. Symmetric deflation gives \(X_{\mathrm{sym}}=(I-P_t)X\) and \(Y_{\mathrm{sym}}=(I-P_u)Y\); X-score deflation gives \(X_{\mathrm{X}}=(I-P_t)X\) and \(Y_{\mathrm{X}}=(I-P_t)Y\). Because these projection matrices are symmetric and idempotent, their residual cross-products are

X_{\mathrm{sym}}'Y_{\mathrm{sym}} =X'(I-P_t)(I-P_u)Y,\qquad X_{\mathrm{X}}'Y_{\mathrm{X}} =X'(I-P_t)Y.

The first expression removes score-aligned variation from each block separately; the second removes variation aligned with \(t\) from both. Unless \(P_u=P_t\), the residual cross-products need not match. Consequently, their leading singular vectors and singular values—and therefore the next weights and scores—can differ.

The projection formula also explains a useful computational check. For any nonzero scalar \(a\), replacing \(t\) by \(at\) leaves \(P_t\) unchanged:

\frac{(at)(at)'}{(at)'(at)} =\frac{a^2tt'}{a^2t't} =\frac{tt'}{t't}.

Thus, rescaling a score does not change the fitted projection, provided the regression coefficient is recalculated consistently. A deflation implementation that changes its residuals merely because a score was multiplied by a nonzero constant is not applying this projection rule correctly.

Worked Comparisons

Worked Example: Different Second Components Under Two Rules

Consider hypothetical centered data represented by three mutually orthonormal vectors \(e_1,e_2,e_3\) in the observational-unit space. Define the two blocks by

X=[\,e_1,\ e_1+e_2\,],\qquad Y=[\,3e_1-3e_2,\ e_2\,].

The cross-product is

X'Y= \begin{pmatrix} e_1'(3e_1-3e_2)&e_1'e_2\\ (e_1+e_2)'(3e_1-3e_2)&(e_1+e_2)'e_2 \end{pmatrix} = \begin{pmatrix} 3&0\\ 0&1 \end{pmatrix}.

The first singular value is \(3\), with weights \(w=c=(1,0)'\). Therefore \(t=e_1\) and \(u=3e_1-3e_2\). The first score is nonzero, and the singular values are distinct, so the leading directions are well-defined up to sign.

Symmetric score deflation. Since \(t'X=(1,1)\) and \(t't=1\), \(p'=(1,1)\). Hence

X_{\mathrm{sym}}=X-tp'=[\,0,\ e_2\,].

For the Y block, \(u'u=18\), \(u'y_1=18\), and \(u'y_2=-3\), where \(y_1=3e_1-3e_2\) and \(y_2=e_2\). Thus \(q'=(1,-1/6)\), giving

Y_{\mathrm{sym}} =[\,y_1,y_2\,]-u(1,-1/6) =[\,0,\ e_2+u/6\,] =[\,0,\ (e_1+e_2)/2\,].

Therefore

X_{\mathrm{sym}}'Y_{\mathrm{sym}} = \begin{pmatrix} 0&0\\ 0&1/2 \end{pmatrix}.

The second singular value is \(0.5\), with second weights \(w_2=c_2=(0,1)'\). The second residual scores are \(t_2=e_2\) and \(u_2=(e_1+e_2)/2\).

X-score deflation. Here \(t'Y=(3,0)\), so residualizing both blocks on \(t=e_1\) gives

X_{\mathrm{X}}=[\,0,\ e_2\,],\qquad Y_{\mathrm{X}}=[\,-3e_2,\ e_2\,].

It follows that

X_{\mathrm{X}}'Y_{\mathrm{X}}= \begin{pmatrix} 0&0\\ -3&1 \end{pmatrix}.

This rank-one matrix has singular value \(\sqrt{(-3)^2+1^2}=\sqrt{10}\approx3.162\), left singular vector \((0,1)'\), and right singular vector \((-3/\sqrt{10},1/\sqrt{10})'\) when written as the two-vector \((-3,1)'/\sqrt{10}\). Equivalently, the second X weight is \((0,1)'\), and the second Y weight is \((-3,1)'/\sqrt{10}\). The second X score is \(e_2\); the second Y score formed from the residual block is \(\sqrt{10}e_2\). The second component differs substantially from the symmetric-rule result. Both calculations follow their stated deflation rule; neither result can be substituted for the other without changing the analysis.

Worked Example: When the Rules Agree

Suppose hypothetical centered blocks are \(X=[e_1,e_2]\) and \(Y=[2e_1,e_2]\), with \(e_1,e_2\) orthonormal. Then \(X'Y=\begin{pmatrix}2&0\\0&1\end{pmatrix}\). The first weights select the first columns, giving \(t=e_1\) and \(u=2e_1\). The scores are nonzero, and their spans are identical, so \(P_t=P_u\).

For symmetric deflation, \(p'=(1,0)\) and \(q'=(1,0)\), so \(X_{\mathrm{sym}}=[0,e_2]\) and \(Y_{\mathrm{sym}}=[0,e_2]\). For X-score deflation, projecting \(Y\) on \(t=e_1\) gives the same residual \(Y_{\mathrm{X}}=[0,e_2]\). Under either rule the residual cross-product is \(\begin{pmatrix}0&0\\0&1\end{pmatrix}\), and the second singular value is \(1\). Agreement here follows from the equality of the two score spans, not from a general equivalence of the methods.

Worked Example: Checking Score-Scale Invariance

Let a hypothetical score be \(t=2e_1\) and let a column of a block be \(x=5e_1+e_2\), with \(e_1'e_1=e_2'e_2=1\) and \(e_1'e_2=0\). The fitted coefficient is

\frac{t'x}{t't} =\frac{(2e_1)'(5e_1+e_2)}{(2e_1)'(2e_1)} =\frac{10}{4}=2.5.

The fitted value is \(t(2.5)=5e_1\), so the residual is \(e_2\). If the score is instead written as \(e_1=t/2\), its coefficient is \(e_1'x/(e_1'e_1)=5\); the fitted value remains \(5e_1\), and the residual remains \(e_2\). This verifies by substitution that changing the score’s scale does not change the residual when the coefficient is computed by least squares.

Diagnostics, Comparisons, and Limitations

A practical implementation should record which block scores are used, how the score regression coefficients are calculated, and which residual cross-product supplies the next singular vectors. Several checks help identify mistakes:

Compute the initial \(X'Y\) directly from the stated blocks before calculating singular vectors. Each entry must equal the inner product of the corresponding columns.
After deflation, check the score orthogonality implied by the rule. For example, \(t'X_{\mathrm{sym}}=0\) and \(u'Y_{\mathrm{sym}}=0\), up to numerical rounding.
Recompute residual cross-products from the residual matrices. Do not infer them from the initial cross-product unless the relevant projection algebra justifies it.
Check that the residual singular value is compatible with the residual matrix. A claimed leading singular value cannot be smaller than the norm of any one column.
If residual score vectors are zero, the corresponding projection is undefined; stop or use a method designed for that degeneracy rather than dividing by zero.

Choice	Y-block residualization	What it targets	Important consequence
Symmetric score deflation	Regress Y on its own score u	Remove each block’s own current score direction	Maintains a symmetric treatment of the two blocks
X-score deflation	Regress Y on the X score t	Remove Y variation linearly explained by the current X score	Can leave or create different residual cross-block structure

These are algorithmic choices, not a basis for declaring one rule universally preferable. Symmetric deflation matches a two-block analysis that treats the blocks in parallel. X-score deflation is natural when the intended construction treats the X score as the predictor for residualizing Y, as in regression-style PLS. Neither rule makes later components independent of all earlier components in every possible sense; the orthogonality properties depend on the particular score, deflation, and weight conventions.

A common mistake is to say, “Both methods remove the first component, so they must give the same second component.” That reasoning overlooks that the methods subtract different fitted matrices from \(Y\). In the first worked example, the initial first component is the same, but the residual cross-products are \(\begin{pmatrix}0&0\\0&0.5\end{pmatrix}\) and \(\begin{pmatrix}0&0\\-3&1\end{pmatrix}\). Their second singular values and Y weights therefore differ. The correction is to state the deflation rule explicitly and recompute the residual blocks before extracting each later component.

Key takeaway: In sequential two-block PLS, deflation is part of the method’s definition. To interpret or reproduce a later component, specify which score residualizes each block, calculate the resulting residual cross-product, and extract the next component from that matrix.

Check Your Understanding

Answer each question using the deflation definitions above. Short solutions follow the questions.

What is the least-squares projection matrix onto a nonzero score vector \(t\), and why is it unchanged if \(t\) is multiplied by a nonzero constant?
Under what condition do symmetric score deflation and X-score deflation produce the same Y residual?
In the first worked example, what are the residual cross-products under the two rules, and why do the second components differ?
Why must the cross-product \(X'Y\) be checked against the data before its singular vectors are interpreted?

Solutions. (1) The projection is \(P_t=tt'/(t't)\). For nonzero \(a\), \(P_{at}=a^2tt'/(a^2t't)=P_t\). (2) They produce the same Y residual when the own-score and X-score projections are equal, in particular when \(u\) is a nonzero scalar multiple of \(t\). (3) They are \(\begin{pmatrix}0&0\\0&0.5\end{pmatrix}\) and \(\begin{pmatrix}0&0\\-3&1\end{pmatrix}\); their different residual matrices have different leading singular directions and values. (4) Singular vectors describe the matrix actually calculated. An incorrect cross-product gives incorrect weights, scores, and subsequent deflation calculations.