Why a Real Regression Effect Can Disappear in an Inverse Mean
Sliced inverse regression (SIR) looks for changes in the predictor’s conditional mean as the response varies. This is a useful signal when the response separates predictor values in a directionally consistent way. But if a response depends on a predictor through its square, positive and negative predictor values can have the same response. Their contributions to the conditional mean then cancel, even though the response depends on the predictor. This is a structural limitation of the first inverse moment, not necessarily a problem with the number or placement of slices.
The relevant distinction is between the absence of a conditional-mean signal and the absence of a predictor-response relationship. To see it precisely, first define the population SIR operator for a standardized predictor.
The eigenvectors of this operator can provide dimension-reduction directions under appropriate conditions, including the linearity condition for inverse regression and a coverage condition. Those conditions help explain when SIR can recover directions; they do not turn a zero operator into evidence of independence. The examples below isolate what happens when symmetry removes the mean signal.
A Symmetry Argument for SIR Failure
Suppose \(X\) is centered and has a distribution symmetric about zero, meaning \(X\) and \(-X\) have the same distribution. Suppose also that the conditional distribution of \(Y\) given \(X=x\) depends on \(x\) only through an even function, so the response mechanism is unchanged when \(x\) is replaced by \(-x\). In that situation, conditional predictor values that are opposites have equal response likelihood, and their contributions to the conditional mean cancel.
Proof. Let \(h\) be any bounded measurable real function. By the stated joint symmetry, $$ E[Xh(Y)]=E[(-X)h(Y)]=-E[Xh(Y)]. $$ Thus \(E[Xh(Y)]=0\) for every such \(h\). The defining property of conditional expectation says that \(E[X\mid Y]\) has the same integrals against every bounded measurable function of \(Y\). Since those integrals are all zero, \(E[X\mid Y]=0\) almost surely. Standardization is a fixed linear transformation of the centered \(X\), so \(E[Z\mid Y]=0\) as well; consequently \(\Gamma_{\mathrm{SIR}}=0\). \(\square\)
Worked Example: A Normal Predictor Observed Through Its Square
Let \(X\sim N(0,1)\) and define \(Y=X^2\). Find the inverse conditional mean, calculate the SIR operator, and check whether \(X\) and \(Y\) are independent.
For any \(y>0\), the conditional distribution of \(X\) given \(Y=y\) places equal probability on \(\sqrt{y}\) and \(-\sqrt{y}\). Therefore $$ E[X\mid Y=y]=\tfrac12\sqrt y+\tfrac12(-\sqrt y)=0. $$ Since \(X\) is already standardized, \(\Gamma_{\mathrm{SIR}}=E[(E[X\mid Y])^2]=0\).
Yet \(X\) and \(Y\) are dependent: \(Y\) is determined by \(X\). For example, \(P(Y\leq 1\mid |X|\leq 1)=1\), while \(P(Y\leq 1)<1\) because a standard normal variable has positive probability of having absolute value greater than 1. Thus the zero SIR operator has not established independence; it records only that the inverse conditional mean is uninformative.
What the Conditional Second Moment Can Reveal
When the conditional mean stays fixed, the conditional covariance may still change. For a standardized \(Z\), define the conditional covariance matrix $$ C(Y)=\operatorname{Cov}(Z\mid Y). $$ If \(E[Z\mid Y]=0\), then \(C(Y)=E[ZZ^T\mid Y]\). It describes how the spread and orientation of the predictors vary across response values, rather than where their conditional means are located. A second-moment method can use that variation to detect structure hidden from SIR.
A useful basic identity follows by averaging conditional covariances. It also clarifies why a varying conditional covariance is possible even when the unconditional covariance is fixed.
Proof. Write \(m(Y)=E[Z\mid Y]\). By the definition of conditional covariance, $$ E[ZZ^T\mid Y]=C(Y)+m(Y)m(Y)^T. $$ Taking expectations on both sides gives $$ E[ZZ^T]=E[C(Y)]+E[m(Y)m(Y)^T]. $$ Because \(E[Z]=0\) and \(\operatorname{Cov}(Z)=I_p\), the left side is \(I_p\). Also \(E[m(Y)]=E[Z]=0\), so \(E[m(Y)m(Y)^T]=\operatorname{Cov}(m(Y))\). Substitution proves the identity. If \(m(Y)=0\) almost surely, its covariance is zero and \(E[C(Y)]=I_p\). \(\square\)
Worked Example: A Squared Projection in Two Dimensions
Let \(Z=(Z_1,Z_2)^T\sim N_2(0,I_2)\), set \(b=(1,1)^T/\sqrt{2}\), and let \(c=(1,-1)^T/\sqrt{2}\). Define \(U=b^TZ\), \(V=c^TZ\), and \(Y=U^2\). Find the conditional mean and covariance of \(Z\) given \(Y\), and identify what their variation says.
The vectors \(b\) and \(c\) are orthonormal, so \(U\) and \(V\) are independent \(N(0,1)\) variables and \(Z=bU+cV\). Given \(Y=y>0\), \(U\) is equally likely to be \(\sqrt y\) or \(-\sqrt y\). Hence \(E[U\mid Y=y]=0\) and \(\operatorname{Var}(U\mid Y=y)=y\). Independence gives \(E[V\mid Y=y]=0\) and \(\operatorname{Var}(V\mid Y=y)=1\). Therefore $$ E[Z\mid Y=y]=0,\qquad C(y)=ybb^T+cc^T. $$ Because \(I_2=bb^T+cc^T\), $$ C(y)-I_2=(y-1)bb^T. $$ The SIR operator is zero, but the conditional covariance varies along \(b\). Since \(Y=U^2\) has a chi-squared distribution with one degree of freedom, \(E[Y]=1\) and \(\operatorname{Var}(Y)=2\). Thus $$ E[(C(Y)-I_2)^2] =E[(Y-1)^2](bb^T)^2 =2bb^T, $$ where \((bb^T)^2=bb^T\) because \(b^Tb=1\). This matrix has eigenvalue \(2\) along \(b\) and eigenvalue \(0\) along \(c\), so it identifies the squared-projection direction even though SIR does not.
A pointwise check makes the conditional contrast concrete. At \(y=4\), \(C(4)-I_2=3bb^T\), with eigenvalue \(3\) along \(b\). If the contrast is rescaled by 3, then \(3\{C(4)-I_2\}=9bb^T\), whose eigenvalue along \(b\) is \(3(4-1)=9\), not 3. The unscaled contrast is the one used in the expectation above.
Second-Moment Alternatives and Their Limits
The example suggests a general diagnostic: inspect whether conditional predictor covariances, or conditional second moments when inverse means vanish, vary with \(Y\). Such variation can reveal directions that SIR’s conditional means miss. Methods built from conditional covariance changes are therefore natural alternatives for symmetric effects. SAVE is a formal method in this family; here the point is the underlying source of information, not its full operator or estimation procedure.
Second moments are not a universal solution. If a response depends on a predictor through features that leave both the conditional mean and conditional covariance unchanged, neither inverse first nor second moments will expose that dependence. The choice of moment is tied to the structure of the regression effect. Nor does a conditional covariance change automatically imply that a method recovers the entire central subspace: recovery also requires suitable conditions on the predictor distribution and on whether the inverse moments cover the relevant directions.
Worked Example: A Discrete Symmetric Effect
Let \(X\) take values \(-2,-1,1,2\), each with probability \(1/4\), and set \(Y=X^2\). Standardize \(X\) to \(Z=X/\sqrt{5/2}\). Compare its inverse conditional mean and second moment across response values.
The mean of \(X\) is zero, and $$ \operatorname{Var}(X)=E[X^2] =\tfrac14(4+1+1+4)=\tfrac52. $$ For either response value \(y=1\) or \(y=4\), the conditional possibilities for \(X\) are the two equally likely signs of \(\sqrt y\). Thus \(E[X\mid Y=y]=0\), and \(E[Z\mid Y=y]=0\) for both values. The SIR operator is zero.
The conditional second moment is different: $$ E[Z^2\mid Y=1]=\frac{1}{5/2}=\frac25,\qquad E[Z^2\mid Y=4]=\frac{4}{5/2}=\frac85. $$ Because the conditional means are zero, these are also the conditional variances. They differ across \(Y\), while their average returns the unconditional variance: $$ E[\operatorname{Var}(Z\mid Y)] =\tfrac12\left(\tfrac25+\tfrac85\right)=1. $$ So the response is again invisible to inverse means but visible to inverse second moments.
Common Pitfall: Treating a Zero SIR Signal as Independence
A frequent error is to reason, “The conditional mean does not change with the response, so the response does not depend on the predictors.” That conclusion is too strong: it confuses one summary of the conditional distribution with the full conditional distribution. In a symmetric effect, the conditional distribution can change in spread while its mean stays fixed. The correction is to say exactly what the calculation establishes: a constant inverse conditional mean gives no population SIR signal, but it does not rule out dependence or guarantee that every relevant direction has been found.
When a scientific model suggests an even or otherwise symmetric predictor effect, examine conditional second moments as well as means. A changing conditional covariance is evidence that the response indexes a change in predictor spread or orientation. If neither first nor second moments change, a different kind of signal may be needed; failure of these moments is not proof of independence.
Check Your Understanding
Use the definitions and calculations above to answer each question.
- For \(X\sim N(0,1)\) and \(Y=X^2\), why is \(E[X\mid Y]\) zero, and why does that not imply independence?
- In the two-dimensional squared-projection example, which eigenvector of \(E[(C(Y)-I_2)^2]\) identifies the regression direction, and what is its eigenvalue?
- State the identity relating the expected conditional covariance and the covariance of the inverse conditional mean for a standardized predictor.
- In the discrete example, calculate the two conditional variances of \(Z\) and explain why their average equals one.
- Give one reason conditional second-moment methods, like SIR, may fail to reveal a predictor-response relationship.