Testing Response-Space Contrasts

Why Change the Response Coordinates?

A multivariate regression records several outcomes for each subject, but a scientific question may concern only a particular combination of them. For example, a study might ask whether a treatment changes the difference between two outcomes, or whether a weighted average of several outcomes has a specified predictor effect. A response-space contrast makes that target explicit: it changes which response combinations are tested without removing predictors or altering the rest of the regression model.

Let \(Y\) be an \(n\times p\) response matrix and \(X\) an \(n\times k\) full-column-rank design matrix. Write the multivariate linear model as \(Y=XB+U\), where \(B\) is \(k\times p\). Assume the rows of \(U\) are independent and distributed as \(N_p(0,\Sigma)\), with \(\Sigma\) positive definite. A response contrast matrix \(M\), of size \(p\times q\), forms the transformed responses \(Z=YM\). The transformed coefficient matrix is \(BM\), and the row covariance of the transformed errors is \(\Sigma_M=M'\Sigma M\).

The dimensions help clarify the roles of the two contrast matrices. Multiplication by \(C\) selects predictor-side coefficient contrasts; multiplication by \(M\) selects response combinations. This tutorial focuses on using \(M\) to define the outcomes of interest. The transformed model still uses the same design matrix \(X\), so the other predictor effects are fitted rather than discarded.

From Response Contrasts to a Test

Define \(\hat B=(X'X)^{-1}X'Y\), the ordinary least-squares estimator in the full model, and let \(\hat\Gamma=\hat B M\). Set \(A=C(X'X)^{-1}C'\), which is positive definite when \(C\) has full row rank. The discrepancy from the null restriction is the \(r\times q\) matrix \(D=C\hat B M-\Theta\). The hypothesis and error sum-of-squares matrices for the transformed responses are

Both matrices are \(q\times q\). \(H_M\) measures the fitted discrepancy from the specified restriction, scaled for the predictor contrasts, while \(E_M\) is the residual cross-product matrix from fitting the full predictor model to the selected responses. The subscripts emphasize that these matrices depend on the response combinations selected by \(M\).

Here \(W_q(\Sigma_M,d)\) denotes the \(q\)-dimensional Wishart distribution with scale matrix \(\Sigma_M\) and \(d\) degrees of freedom. The theorem requires a nonsingular transformed covariance matrix; because \(\Sigma\) is positive definite and \(M\) has full column rank, \(\Sigma_M=M'\Sigma M\) is positive definite.

Why the Matrices Give the Restricted-Model Test

The connection between \(H_M\) and the restricted model follows by minimizing the transformed residual sum of squares subject to \(C\Gamma=\Theta\), where \(\Gamma=BM\). Its unrestricted least-squares estimate is \(\hat\Gamma=\hat B M\). The constrained estimate is

Indeed, multiplying by \(C\) gives \(C\tilde\Gamma=C\hat\Gamma-AA^{-1}(C\hat\Gamma-\Theta)=\Theta\). The adjustment is the least-squares correction satisfying the restrictions. The increase in residual cross-products from imposing them is

More explicitly, the cross-product increase is \((\tilde\Gamma-\hat\Gamma)'X'X(\tilde\Gamma-\hat\Gamma)\). Substituting the adjustment gives \(D'A^{-1}C(X'X)^{-1}X'X(X'X)^{-1}C'A^{-1}D=D'A^{-1}D\), because \(C(X'X)^{-1}C'=A\). The residuals from the unrestricted fit are orthogonal to the columns of \(X\), so the cross-terms between unrestricted residuals and the fitted-value adjustment are zero. Thus the restricted residual cross-product matrix is exactly \(E_M+H_M\). This proves that the matrices compare the restricted and unrestricted fits for the selected responses.

For the distribution statement, \(C\hat B M-\Theta\) is matrix normal under the null, with row covariance \(A\) and column covariance \(\Sigma_M\). Whiten its rows by \(A^{-1/2}\): the resulting \(r\times q\) matrix has independent \(N_q(0,\Sigma_M)\) rows. Its cross-product is \(H_M\), so \(H_M\sim W_q(\Sigma_M,r)\). The residual projection \(I-P_X\) has rank \(\nu=n-k\); under normality, an orthogonal change of coordinates separates the fitted and residual components. Hence \(E_M\sim W_q(\Sigma_M,\nu)\) and is independent of \(\hat B M\), and therefore of \(H_M\). Since \(\nu\ge q\) and \(\Sigma_M\) is positive definite, \(E_M\) is positive definite almost surely. These facts establish the theorem. \(\square\)

Worked Examples

What the Response Contrast Preserves

Selecting \(M\) changes the outcome being analyzed, not the predictor model. The transformed data satisfy \(YM=X(BM)+UM\), with the same \(X\) as before. Consequently, nuisance predictor coefficients remain in the fit and are accounted for by \(P_X\) and the residual degrees of freedom \(n-k\). A response contrast is therefore not a license to drop covariates that were part of the specified model.

The choice of \(M\) also determines what the test can say. If its columns are linearly independent, it selects \(q\) distinct response directions. A scalar contrast such as \(M=(1,-1)'\) tests one direction; a matrix \(M\) can select several combinations jointly. The test does not automatically describe effects in response directions outside the column space of \(M\).

Common Pitfall and Conditions

• Testing the wrong hypothesis. Testing \(C B M=\Theta\) is generally not the same as testing \(CB=\Theta\), nor is it the same as testing each response coefficient separately. Write out the response combinations represented by the columns of \(M\).
• Dropping nuisance predictors. The response transformation does not remove columns of \(X\). Use the full-model residual matrix \(E_M=M'Y'(I-P_X)YM\), not residuals from an unintended smaller predictor model.
• Ignoring rank or degrees of freedom. \(C\) and \(M\) should have the stated full ranks, \(X\) must have full column rank, and the usual nonsingular-error multivariate test requires \(\nu\ge q\). If these conditions fail, the displayed inverse or standard calibration may not be valid.
• Interpreting a multivariate result one response at a time. A joint rejection concerns the selected set of response combinations. It does not, by itself, identify which coordinate or contrast drives the result.