Why SDR Directions Are Easy to Misread
Sufficient dimension reduction (SDR) represents a multivariate predictor \(X\) through a lower-dimensional set of linear combinations. The quantities commonly reported as “directions” are coefficient vectors for those combinations. They are not, by themselves, uniquely oriented arrows, nor are they always uniquely determined one at a time. Correct interpretation separates the coordinate vectors from the subspace they span.
Let \(X\) be a \(p\)-dimensional predictor and \(Y\) a response. A subspace \(\mathcal S\subseteq\mathbb R^p\) is a dimension-reduction subspace if \(Y\) is conditionally independent of \(X\) given the projection of \(X\) onto \(\mathcal S\). If a \(p\times d\) matrix \(B\) has columns spanning \(\mathcal S\), the reduced predictors can be written \(B^T X\). Replacing \(B\) by another basis for the same space changes the coordinates used to describe the reduction, not the space itself.
Many SDR methods first standardize predictors. Write \(\mu=E[X]\), let \(\Sigma=\operatorname{Var}(X)\) be positive definite, and define \(Z=\Sigma^{-1/2}(X-\mu)\), so that \(\operatorname{Var}(Z)=I_p\). For example, the population SIR matrix is \(M_{\mathrm{SIR}}=\operatorname{Cov}(E[Z\mid Y])\), and a standard SAVE matrix is \(M_{\mathrm{SAVE}}=E[(I_p-\operatorname{Cov}(Z\mid Y))^2]\). These are symmetric positive semidefinite matrices. Under suitable conditions, their informative eigenspaces provide directions for a reduction; an eigenvector is not automatically a uniquely defined scientific effect.
Sign Is a Coordinate Choice
An eigenvector and its negative are both eigenvectors for the same eigenvalue. More importantly for interpretation, the one-dimensional subspaces they span are identical: the line through \(b\) is the line through \(-b\). Thus a reported direction \(b\) and a reported direction \(-b\) do not represent two different reductions.
Worked Example: Reversing an SIR Direction
Suppose \(Z=(Z_1,Z_2)^T\) has independent standard normal components and \(Y=Z_1\). Then \(E[Z\mid Y]=(Y,0)^T\), so
The leading unit eigenvector is \(b=(1,0)^T\), and its negative \(-b=(-1,0)^T\) is also a leading unit eigenvector. The resulting scores are \(b^TZ=Z_1\) and \((-b)^TZ=-Z_1\). Each score determines the other exactly, so they contain the same information and span the same one-dimensional space.
If an analyst labels high values of the score as “increasing response,” the verbal orientation must change when the sign is reversed. That is a reporting convention, not evidence for a different SDR subspace.
To verify the information statement, the mappings \(z\mapsto b^Tz\) and \(z\mapsto -b^Tz\) are invertible transformations of each other: \(b^Tz=-((-b)^Tz)\). Therefore each score is a measurable function of the other. The sigma-fields they generate are equal, so conditioning on one gives the same information as conditioning on the other. \(\square\)
A Basis Is Not the Subspace
When a reduction has dimension \(d>1\), its directions are usually presented as the columns of a matrix \(B\). The columns are a basis, but there are generally many bases for the same subspace. If \(Q\) is any invertible \(d\times d\) matrix, then \(BQ\) has the same column space as \(B\). For orthonormal bases, \(Q\) is orthogonal. The new reduced coordinates \(Q^TB^TX\) are simply an invertible re-expression of the old coordinates.
Worked Example: Rotating a Two-Dimensional SDR Basis
Suppose an SDR matrix in three standardized predictor coordinates is
The leading two-dimensional eigenspace is the plane spanned by \(e_1=(1,0,0)^T\) and \(e_2=(0,1,0)^T\). One orthonormal basis is \(B=(e_1,e_2)\). Rotating that basis by an angle \(\phi\) gives
Both columns of \(B_\phi\) lie in the first two coordinate plane, are orthonormal, and together span that entire plane. For example, at \(\phi=\pi/4\), the columns are \((1,1,0)^T/\sqrt{2}\) and \((-1,1,0)^T/\sqrt{2}\). Neither is the original \(e_1\) or \(e_2\), but the two-dimensional reduction is unchanged.
The equality of the spaces follows directly: \(B_\phi=BQ\), where \(Q\) is the two-dimensional rotation matrix. Since \(Q\) is invertible, every linear combination of columns of \(B_\phi\) is a linear combination of columns of \(B\), and conversely. Therefore their column spaces are equal.
This invariance also has a projection-matrix expression. For a full-column-rank basis \(B\), the orthogonal projector onto its column space is \(P_B=B(B^TB)^{-1}B^T\). If \(Q\) is invertible, then
Thus changing basis cannot change the subspace projector. This gives a precise way to compare estimated reductions: compare their spans or projection matrices, not the individual columns without first addressing sign and basis choices. \(\square\)
Eigenvalue Ordering Is a Selection Convention
For a symmetric positive semidefinite SDR matrix, eigenvalues are nonnegative. Software may list them in ascending or descending order, while presentations commonly discuss the largest eigenvalues first. The relevant directions are selected by their eigenvalues and the chosen dimension \(d\), not by a universal column number that is guaranteed across software.
For example, if the eigenvalues are \(\lambda_1\geq\lambda_2\geq\cdots\geq\lambda_p\), a leading-\(d\) reduction uses eigenvectors associated with \(\lambda_1,\ldots,\lambda_d\), subject to the method’s assumptions and the question of how many directions are supported. If a program returns ascending order, the same eigenvectors will appear at the opposite end of its output. A zero eigenvalue does not become informative merely because it is printed first.
Worked Example: Reading an Eigenvalue List Correctly
Suppose an estimated SIR matrix has eigenvalues \(0.49,\ 0.09,\ 0.01\), with corresponding orthonormal eigenvectors \(v_1,v_2,v_3\). If the analysis retains \(d=2\) leading directions, the retained subspace is \(\operatorname{span}(v_1,v_2)\), because \(0.49\) and \(0.09\) are the two largest eigenvalues. If software instead prints the list as \(0.01,\ 0.09,\ 0.49\), the retained directions are still those attached to \(0.09\) and \(0.49\), not the first two rows of that output.
The values are eigenvalues of this particular SDR matrix. They are not automatically percentages of response variation explained, and an eigenvalue from SIR should not be compared numerically with an eigenvalue from SAVE as if both were on a common effect-size scale. The methods construct different operators.
Standardized and Original-Scale Directions
A direction estimated using \(Z=\Sigma^{-1/2}(X-\mu)\) has coefficients in standardized, whitened coordinates. Those coefficients should not be copied unchanged and described as coefficients of the original predictors. If \(\gamma\) is a direction in \(Z\)-coordinates, then
Because \(\Sigma^{-1/2}\) is symmetric, the corresponding coefficient vector for the centered original predictors is \(\beta=\Sigma^{-1/2}\gamma\). This is a change of coordinates, not a new direction in the same numerical coordinate system. If predictors are not centered, the constant offset must also be accounted for when interpreting the score.
Worked Example: Converting a Direction Back to Original Units
Let \(X_1\) and \(X_2\) be independent with variances \(100\) and \(1\), and zero means. Then \(\Sigma^{-1/2}=\operatorname{diag}(1/10,1)\), so the standardized predictors are \(Z_1=X_1/10\) and \(Z_2=X_2\). Suppose the reduction score in standardized coordinates has direction \(\gamma=(1,1)^T\). Its score is
The corresponding original-scale direction is
Using \((1,1)^T\) directly with the original \(X\) would instead produce \(X_1+X_2\), a different score. For instance, at \(X=(10,0)^T\), the standardized score is \(1\), whereas the unconverted raw score is \(10\). The two coefficient vectors cannot be interpreted interchangeably.
When Is an Estimated Direction Unique?
There are several distinct forms of non-uniqueness. Even with a simple eigenvalue, an eigenvector is unique only up to sign. If an eigenvalue has multiplicity greater than one, any orthonormal basis of its eigenspace is valid, so individual eigenvectors within that space are not unique. The eigenspace itself can nevertheless be uniquely determined if it is separated from the rest of the spectrum.
For a symmetric matrix with ordered eigenvalues, a leading \(d\)-dimensional eigenspace is separated when \(\lambda_d>\lambda_{d+1}\). If this inequality fails, there is a tie at the boundary: selecting exactly \(d\) directions within the tied eigenspace does not define a unique leading-\(d\) space. Even when the population space is well defined, an estimated basis can vary with the sample. Near-tied eigenvalues can make individual estimated directions especially sensitive to small perturbations, while the estimated span may remain much more stable.
Worked Example: A Tie Makes the First Direction Non-Unique
Consider \(M=\operatorname{diag}(0.5,0.5,0)\). Every unit vector \(v=(a,b,0)^T\) in the first two coordinates satisfies \(Mv=0.5v\), provided \(a^2+b^2=1\). Thus there is no uniquely preferred first eigenvector. If the target dimension is \(d=2\), however, the leading two-dimensional space is uniquely the first two coordinate plane because \(0.5>0\).
Now perturb the matrix to \(\widehat M_\varepsilon=\operatorname{diag}(0.5+\varepsilon,0.5-\varepsilon,0)\), with \(0<\varepsilon<0.5\). Its first eigenvector is \(e_1\). Reversing the perturbation gives \(\operatorname{diag}(0.5-\varepsilon,0.5+\varepsilon,0)\), whose first eigenvector is \(e_2\). As \(\varepsilon\) tends to zero, both matrices approach \(M\), yet their first directions differ. The tied population matrix identifies the plane for \(d=2\), not a single first direction.
Common Mistakes and a Practical Interpretation Check
A frequent mistake is to treat a printed direction as a uniquely oriented, uniquely identified scientific vector: “the positive coefficient on predictor 1 means the effect points upward.” That reasoning is incomplete. The eigenvector could be multiplied by \(-1\) without changing its subspace, and a multidimensional basis could be rotated without changing the reduction. The correction is to report the dimension and subspace, explain the coordinate scale, and state any sign convention used for display.
Another mistake is to take the first eigenvectors in a software table without checking its ordering, or to call the largest eigenvalue the “most important effect” across different SDR methods. Instead, check which matrix was diagonalized, how its eigenvalues are ordered, how the structural dimension was selected, and whether the chosen subspace is separated by an eigenvalue gap. The method’s assumptions also matter: an eigenvector output does not, by itself, establish that the method has recovered the full central subspace.
Determine whether the reported coefficients multiply original predictors, centered predictors, or standardized predictors.
Confirm the eigenvalue order, the number of retained directions, and whether a tie occurs at the retention boundary.
For a single simple direction, allow for sign reversal; for multiple directions, interpret the span rather than insisting on a particular basis.
These checks matter in reporting, reproducibility, and inference. Two analysts can obtain different-looking direction vectors yet estimate the same reduction subspace. Conversely, matching signs or column labels do not guarantee that two analyses used the same predictor scaling, method, or retained dimension. When individual directions are not identifiable, reporting them as uniquely meaningful quantities overstates what the analysis determines.
Check Your Understanding
Use the distinction between a direction’s coordinates and the subspace it represents.
- Why do \(b\) and \(-b\) yield the same one-dimensional reduction, even though their scores have opposite signs?
- If two leading eigenvalues are equal and the target dimension includes both, what is identifiable: each eigenvector separately or their joint eigenspace?
- A program returns eigenvalues in ascending order. What should you check before choosing the first two eigenvectors?
- If an SDR direction was computed for \(Z=\Sigma^{-1/2}(X-\mu)\), how do you express its score using centered \(X\)-coordinates?
- Why is it generally inappropriate to compare the numerical size of an SIR eigenvalue directly with a SAVE eigenvalue as a common measure of response importance?