A Worked Comparison of SIR and SAVE

Why Compare SIR and SAVE?

Sliced inverse regression (SIR) and sliced average variance estimation (SAVE) both use the distribution of predictors within response slices, but they summarize different features of that distribution. SIR uses how slice means vary. SAVE uses how slice covariance matrices differ from the marginal predictor covariance. Consequently, SIR can show a direction that SAVE also shows, or SAVE can reveal a direction that SIR’s slice means do not detect.

The calculations below use a small, balanced sample whose overall predictor covariance is exactly the identity matrix. That makes the comparison transparent: the operators can be computed directly from the slice means and slice covariances. Their ranks describe the information in these particular sliced operators; they do not, by themselves, prove the dimension of a population central subspace.

The Two Slice Operators

Let $Z\in\mathbb{R}^p$ be a standardized predictor vector, so $E[Z]=0$ and $\operatorname{Cov}(Z)=I_p$. Partition the response into $H$ slices $S_1,\ldots,S_H$, each with positive probability $p_h=P(Y\in S_h)$, where $\sum_{h=1}^H p_h=1$. Define the slice mean $m_h=E[Z\mid Y\in S_h]$ and slice covariance $C_h=\operatorname{Cov}(Z\mid Y\in S_h)$.

Definition: The population SIR operator and SAVE operator for this slicing are $$ M_{\mathrm{SIR}}=\sum_{h=1}^H p_h m_hm_h^\top, \qquad M_{\mathrm{SAVE}}=\sum_{h=1}^H p_h(I_p-C_h)^2. $$ In a sample, replace the slice probabilities, means, and covariances by their sample estimates to obtain the corresponding estimated operators. Estimated directions are eigenvectors associated with nonzero eigenvalues; in practice, a numerical threshold is used to decide which eigenvalues count as nonzero.

The squared matrix in SAVE means $(I_p-C_h)(I_p-C_h)$. Since each $C_h$ is symmetric, $I_p-C_h$ is symmetric as well. Both operators are therefore symmetric and positive semidefinite. The slice weights matter: when slice sizes differ, an unweighted average generally does not match the defined operator.

A Balanced Standardized Example

Consider two predictors and three response slices, with four observations in each slice. For each slice, take all four combinations of the signs $s_1,s_2\in\{-1,1\}$, and define the predictor vector by $$ Z=\left(\mu_h+\frac{s_1}{\sqrt{3}},\;s_2\sqrt{v_h}\right), \qquad (\mu_1,\mu_2,\mu_3)=(-1,0,1), \qquad (v_1,v_2,v_3)=\left(\frac12,1,\frac32\right). $$ Each observation has weight $1/12$. The response label is the slice index $h$. This specifies all 12 observations explicitly without requiring a long table of repeated sign combinations.

Worked Example: Checking Slice and Overall Standardization

Within slice $h$, the four sign combinations occur equally often. Each sign averages to zero, and the product $s_1s_2$ also averages to zero. Thus the slice mean is $m_h=(\mu_h,0)^\top$, and the centered slice covariance is $C_h=\operatorname{diag}(1/3,v_h)$. In particular, the first coordinate has within-slice variance $1/3$, while the second has within-slice variance $v_h$; their within-slice covariance is zero.

The three slice means average to zero. The marginal variance of the first predictor is its average within-slice variance plus the variance of its slice means: $$ \frac13\left(\frac13+\frac13+\frac13\right) +\frac13\left((-1)^2+0^2+1^2\right) =\frac13+\frac23=1. $$ The marginal variance of the second predictor is $$ \frac13\left(\frac12+1+\frac32\right)=1. $$ The overall covariance is zero: within each slice the centered cross-product averages to zero, and the slice means have second coordinate zero. Therefore $E[Z]=0$ and $\operatorname{Cov}(Z)=I_2$, as required for a standardized example.

The covariance calculation also illustrates the total covariance identity: marginal covariance is the average within-slice covariance plus the covariance of the slice means. Here the within-slice covariance contributes $1/3$ to the first coordinate’s variance, and changes in slice means contribute the remaining $2/3$.

Computing SIR From the Slice Means

In this balanced example, each slice has probability $1/3$. The three means are $(-1,0)^\top$, $(0,0)^\top$, and $(1,0)^\top$. Substitution into the SIR definition gives the following result.

Worked Example: The SIR Operator and Direction

The outer products of the slice means are $$ m_1m_1^\top= \begin{pmatrix}1&0\\0&0\end{pmatrix}, \qquad m_2m_2^\top= \begin{pmatrix}0&0\\0&0\end{pmatrix}, \qquad m_3m_3^\top= \begin{pmatrix}1&0\\0&0\end{pmatrix}. $$ Weighting each by $1/3$ and adding, $$ M_{\mathrm{SIR}} =\frac13 \begin{pmatrix}1&0\\0&0\end{pmatrix} +\frac13 \begin{pmatrix}0&0\\0&0\end{pmatrix} +\frac13 \begin{pmatrix}1&0\\0&0\end{pmatrix} = \begin{pmatrix}2/3&0\\0&0\end{pmatrix}. $$ Its eigenvalues are $2/3$ and $0$. The only nonzero eigendirection is $e_1=(1,0)^\top$. Thus the estimated SIR operator has rank one and identifies the first coordinate direction, but its slice means provide no direction along $e_2=(0,1)^\top$.

Result: For positive slice weights, the rank of $M_{\mathrm{SIR}}$ equals the dimension of the span of the slice means. To see this, for any vector $a$, $$ a^\top M_{\mathrm{SIR}}a=\sum_{h=1}^H p_h(a^\top m_h)^2. $$ This is zero exactly when $a$ is orthogonal to every $m_h$. Hence the null space of $M_{\mathrm{SIR}}$ is the orthogonal complement of the span of the slice means. Because $M_{\mathrm{SIR}}$ is symmetric, its range is that span, proving the rank claim. $\square$

Computing SAVE From the Slice Covariances

SIR’s result does not mean the second predictor is irrelevant to the response. Its conditional variance changes across the slices, even though its conditional mean remains zero. SAVE uses those covariance changes directly.

Worked Example: The SAVE Operator and Directions

The slice covariance matrices are $$ C_1=\begin{pmatrix}1/3&0\\0&1/2\end{pmatrix}, \qquad C_2=\begin{pmatrix}1/3&0\\0&1\end{pmatrix}, \qquad C_3=\begin{pmatrix}1/3&0\\0&3/2\end{pmatrix}. $$ Subtract each from $I_2$ and square: $$ (I_2-C_1)^2=\begin{pmatrix}4/9&0\\0&1/4\end{pmatrix}, \quad (I_2-C_2)^2=\begin{pmatrix}4/9&0\\0&0\end{pmatrix}, \quad (I_2-C_3)^2=\begin{pmatrix}4/9&0\\0&1/4\end{pmatrix}. $$ The weighted average is $$ M_{\mathrm{SAVE}} =\frac13\left[ \begin{pmatrix}4/9&0\\0&1/4\end{pmatrix} +\begin{pmatrix}4/9&0\\0&0\end{pmatrix} +\begin{pmatrix}4/9&0\\0&1/4\end{pmatrix} \right] =\begin{pmatrix}4/9&0\\0&1/6\end{pmatrix}. $$ The eigenvalues are $4/9$ and $1/6$, both positive. The corresponding directions are $e_1$ and $e_2$, respectively. SAVE therefore has rank two for these slices. It retains the first direction, which also appeared in SIR, and adds the second direction through the changing conditional variance.

Result: For positive slice weights, $M_{\mathrm{SAVE}}$ is positive semidefinite, and its null space consists exactly of vectors $a$ for which $(I_p-C_h)a=0$ in every slice. Indeed, writing $A_h=I_p-C_h$, symmetry gives $$ a^\top M_{\mathrm{SAVE}}a =\sum_{h=1}^H p_h a^\top A_h^2a =\sum_{h=1}^H p_h\|A_ha\|^2. $$ Every term is nonnegative, and the weighted sum is zero exactly when every $A_ha=0$. Thus SAVE is sensitive to directions with conditional variance or covariance changes, even if their conditional means do not change. $\square$

Comparing the Estimated Structural Dimensions

For this example, the estimated SIR operator has one nonzero eigenvalue, so its empirical operator rank is one. The estimated SAVE operator has two nonzero eigenvalues, so its empirical operator rank is two. Both identify $e_1$; only SAVE identifies $e_2$. In the second coordinate, the slice mean is always zero, whereas its conditional variances are $1/2$, $1$, and $3/2$. This is precisely the contrast that makes the two calculations differ.

These ranks are properties of the operators calculated from the chosen slices and observations. Treating them as estimates of a population structural dimension requires appropriate modeling and slicing conditions, as well as a method for handling sampling variability. With real data, small eigenvalues may reflect noise rather than meaningful directions; a numerical rank or dimension estimate therefore depends on a stated threshold or inferential procedure.

Common Mistake: Reading SIR’s Zero Eigenvalue as Irrelevance

A common mistake is to see the zero second eigenvalue of $M_{\mathrm{SIR}}$ and conclude that the second predictor cannot carry information about the response. That conclusion is too strong: the SIR operator uses slice means, and all three second-coordinate means are zero. It does not check whether the conditional variance changes. Here, that variance changes across slices, and SAVE has a positive eigenvalue in the second coordinate. A zero SIR eigenvalue means that this sliced-mean calculation does not identify that direction; it is not, by itself, proof of irrelevance.

Key takeaway: Compare the inverse moments, not just the method labels. SIR summarizes variation among slice means; SAVE summarizes departures of slice covariance matrices from the standardized marginal covariance. In this example those summaries produce ranks one and two, respectively.

Check Your Understanding

Use the worked calculations to answer each question.

Why does the average of the three slice means equal the overall predictor mean in this balanced example?
Which feature of the slice moments gives SIR its nonzero direction?
Why does the second coordinate contribute nothing to SIR but contribute a positive eigenvalue to SAVE?
What are the empirical operator ranks of SIR and SAVE here, and why should those ranks not automatically be identified with the population central-subspace dimension?
If all slice covariance matrices equaled $I_2$, what would the SAVE operator be, regardless of the slice means?