The Population SIR Operator

From Inverse Means to a Population Matrix

Inverse regression studies the predictor distribution after conditioning on the response. For a multivariate predictor, the conditional mean \(E[X\mid Y]\) is a vector, and its variation across response values can be summarized by a matrix. That matrix is the population SIR operator: it records which standardized predictor directions have conditional means that vary with the response.

Standardization matters because the coordinates of a predictor can have different units and scales. It gives the operator a common reference geometry and makes its eigenvectors interpretable as directions in standardized predictor coordinates. The matrix describes variation in conditional means; by itself, its rank is the dimension of that inverse-mean signal.

Write \(m(Y)=E[Z\mid Y]\). Conditional expectation is understood componentwise, up to almost-sure equality. Since \(E[Z]=0\), the tower property gives \(E[m(Y)]=0\), and consequently the operator also has the useful expression \(M_{\mathrm{SIR}}=E[m(Y)m(Y)^T]\). The conditional mean has finite second moments: conditional Jensen's inequality gives \(E[\|m(Y)\|^2]\leq E[\|Z\|^2]=p\).

Thus the operator is a \(p\)-by-\(p\) matrix even when the response is continuous. It summarizes the full conditional-mean function \(m(y)\) through its second moment, rather than listing one mean vector for each possible value of \(Y\).

Basic Properties and the Nonzero Eigenspace

The first property explains why eigenanalysis is appropriate: the SIR operator is symmetric and positive semidefinite. Its positive-eigenvalue directions are exactly the linear span of the conditional mean vectors, with null directions corresponding to contrasts that have zero conditional mean at almost every response value.

Proof. For any \(a\in\mathbb{R}^p\),

The matrix is symmetric because it is an expectation of symmetric outer products, and the displayed quadratic form proves positive semidefiniteness. To obtain the upper bound, apply the law of total covariance:

Conditional covariance matrices are positive semidefinite, so their expectation is positive semidefinite. Therefore \(I_p-M_{\mathrm{SIR}}\) is positive semidefinite, proving the upper bound.

For the subspace characterization, the quadratic-form identity shows

The last equivalence holds because a nonnegative random variable has expectation zero if and only if it is zero almost surely. The vectors orthogonal to \(S\) are precisely those \(a\) for which \(a^Tm(Y)=0\) almost surely. Hence \(\operatorname{Null}(M_{\mathrm{SIR}})=S^\perp\). For a symmetric matrix, its range is the orthogonal complement of its null space, so \(\operatorname{Range}(M_{\mathrm{SIR}})=S\). By the spectral theorem, the range of a positive semidefinite matrix is the span of its eigenvectors with positive eigenvalues. Its rank is the dimension of its range, completing the result. \(\square\)

This theorem identifies the operator's nonzero eigenspace without requiring a separate guess about which directions matter. It is the smallest fixed subspace that contains the conditional mean vectors, ignoring response values that occur only on a probability-zero set. Eigenvectors within that subspace give an orthonormal basis, but if eigenvalues are repeated, individual eigenvectors need not be unique.

Finite Response Categories and Rank

When \(Y\) takes finitely many values, the population operator has a direct sum representation. Let its categories be \(1,\ldots,K\), with \(p_k=P(Y=k)>0\), and let \(m_k=E[Z\mid Y=k]\). Since \(E[Z]=0\), the category means satisfy \(\sum_{k=1}^K p_km_k=0\).

Derivation. On the event \(Y=k\), the random vector \(m(Y)\) equals \(m_k\), so taking its outer-product expectation gives the stated sum. Its range is the span of the \(K\) vectors \(m_1,\ldots,m_K\), by the theorem. That span has dimension at most \(p\). Also, the weighted-zero-mean relation expresses one mean as a linear combination of the others:

Therefore their span has dimension at most \(K-1\), proving the bound. The bound is about the number of response categories and the predictor dimension; it does not guarantee that the rank reaches either limit.

Continuous Responses and the Span of a Curve

For a continuous response, the conditional mean \(m(y)\) can trace a curve or surface in predictor space. The operator still depends on its second moment, and its rank depends on the linear span of that curve, not on how many distinct response values are possible. The following model shows how a nonlinear conditional mean can occupy a low-dimensional subspace.

Interpretation and a Common Pitfall

A positive eigenvalue means that conditional means vary along the associated direction in standardized predictor coordinates. A zero eigenvalue means that the projection of the conditional mean onto that direction is zero almost surely. The rank therefore counts dimensions of variation in the conditional mean, not dimensions of variation in the predictors themselves: a standardized predictor can vary in a direction even when its conditional mean does not vary there.

A frequent mistake is to treat the rank as automatically equal to the dimension of the full regression structure. The matrix is constructed from conditional means only. Its rank precisely describes the span of those means, as proved above; any further interpretation as a regression-relevant subspace requires additional assumptions and arguments. Another error is to count response categories and declare the rank equal to \(K-1\). The rank is only bounded above by \(K-1\); the category mean vectors may span a smaller space or even all be zero.

The population operator is useful because it turns a function of the response, \(E[Z\mid Y]\), into a finite-dimensional matrix whose range and rank can be characterized exactly. Those properties describe what the SIR matrix itself contains, without conflating its inverse-mean signal with other features of the conditional distribution.