Why a Partial Test Is Needed
A coefficient block may be scientifically important even when a model also contains other predictors that should remain in the analysis. For example, a study may ask whether a treatment effect contributes beyond age and baseline severity. The relevant null hypothesis concerns the treatment coefficients; it does not say that age and severity have no association with the response.
A partial test compares a full model containing both the nuisance predictors and the tested block with a restricted model that retains the nuisance predictors but removes the tested block. The difference between their fits isolates the block’s incremental contribution. This is the nested-design interpretation of a coefficient-block test.
For the basic formulas below, assume \(Z\) has full column rank and that the columns of \(X\), after removing their projections onto the columns of \(Z\), are linearly independent. Thus the full design \([Z\ X]\) has rank \(p_0+q\). These assumptions make the coefficient block estimable and give the test \(q\) numerator degrees of freedom in the univariate case.
Residualize the Tested Block Against the Nuisance Design
Write \(P_Z=Z(Z'Z)^{-1}Z'\) for projection onto the nuisance design and \(M_Z=I-P_Z\) for projection onto its orthogonal complement. The columns of \(M_ZX\) are the parts of the tested predictors that cannot be explained by the nuisance predictors. Consequently, the tested block is identified by \(M_ZX\), not generally by the raw columns of \(X\) considered in isolation.
The Frisch–Waugh–Lovell principle says that the full-model least-squares estimate of \(B_1\) can be obtained by regressing the nuisance-adjusted responses \(M_ZY\) on the nuisance-adjusted predictors \(M_ZX\). Under the rank assumptions above, this gives
This is an adjustment, not a preliminary deletion of nuisance variables. The nuisance effects are removed from both responses and tested predictors for the purpose of estimating the tested block; the full model still includes the nuisance design.
Proof. Since \(M_ZX\) is orthogonal to the columns of \(Z\), the full-model column space is the orthogonal sum of the nuisance space and the space spanned by \(M_ZX\). The projection onto the latter space is \(P_{M_ZX}=M_ZX(X'M_ZX)^{-1}X'M_Z\). Therefore \(P_F=P_Z+P_{M_ZX}\). Substitution gives
Also \(P_F-P_Z\) and \(I-P_F\) are orthogonal projections onto perpendicular spaces. Their sum is \(I-P_Z=M_Z\). Thus \(H+E=Y'[(P_F-P_Z)+(I-P_F)]Y=Y'M_ZY\), as claimed. \(\square\)
Worked Examples
Worked Example: One Tested Predictor With a Nuisance Predictor
Consider one response and four observations. The nuisance design contains an intercept and \(z=(0,1,0,1)'\); the tested predictor is \(x=(0,0,2,0)'\). Let \(y=(1,10,9,4)'\). Test the coefficient of \(x\) while retaining the intercept and the coefficient of \(z\).
The nuisance-only fit gives the mean response within each level of \(z\). For \(z=0\), the responses are 1 and 9, with mean 5; for \(z=1\), they are 10 and 4, with mean 7. Thus the nuisance fitted values are \((5,7,5,7)'\), and \(M_Zy=(-4,3,4,-3)'\). The nuisance-only projection of \(x\) is \((1,0,1,0)'\), so \(M_Zx=(-1,0,1,0)'\).
The adjusted predictor has squared length \(2\), and its inner product with the adjusted response is \((-1)(-4)+(1)(4)=8\). Hence the partial least-squares estimate is \(\widehat\beta_x=8/2=4\). The hypothesis sum of squares is \(H=\widehat\beta_x^2(M_Zx)'(M_Zx)=4^2(2)=32\).
The unrestricted fitted values are the nuisance fit plus \(\widehat\beta_x M_Zx\): \((5,7,5,7)'+4(-1,0,1,0)'=(1,7,9,7)'\). The residuals are \((0,3,0,-3)'\), so \(E=0^2+3^2+0^2+(-3)^2=18\). The restricted residual sum of squares is \((-4)^2+3^2+4^2+(-3)^2=50\), which agrees with \(H+E=32+18=50\).
For this univariate model, \(q=1\) and the full-model residual degrees of freedom are \(4-3=1\). The partial \(F\) statistic is \(H/(E/1)=32/18=16/9\). Its small value is a consequence of the large unrestricted residual variation relative to the incremental contribution; it is not a calculation error.
Worked Example: A Two-Coefficient Block for Two Responses
Let \(n=4\), let \(Z\) contain only an intercept, and let the two tested predictor columns be \(u=(1,-1,0,0)'\) and \(v=(1,1,-2,0)'\). Take the response matrix to have rows \((8,1),(6,3),(1,-1),(-7,-7)\). Both tested predictors are centered, and \(u'v=1-1=0\), so they are orthogonal to each other and to the intercept.
To see the fitted components directly, define \(w=(1,1,1,-3)'\). It is orthogonal to the intercept, \(u\), and \(v\), and \(w'w=12\). The first response column is \(2\mathbf 1+u+2v+3w\); the second is \(-\mathbf 1-u+v+2w\). Thus the estimated tested coefficient matrix is \(\widehat B_1=\begin{pmatrix}1&-1\\2&1\end{pmatrix}\), with rows corresponding to \(u\) and \(v\).
Here \(X'X=\begin{pmatrix}2&0\\0&6\end{pmatrix}\), because \(X=(u\ v)\) and the tested predictors are already orthogonal to the intercept. The theorem gives
For example, the first diagonal entry is \(2(1^2)+6(2^2)=26\); the off-diagonal entry is \(2(1)(-1)+6(2)(1)=10\). The unrestricted residual columns are \(3w\) and \(2w\), so \(E=w'w\begin{pmatrix}3\\2\end{pmatrix}(3\ \ 2) =\begin{pmatrix}108&72\\72&48\end{pmatrix}\). As a check, \(H+E=\begin{pmatrix}134&82\\82&56\end{pmatrix}\) is the response cross-product matrix after removing the intercept alone. The nuisance intercept has remained unrestricted throughout; \(H\) measures only the added contribution of the two tested predictors.
Worked Example: Partial Contribution and Residual Degrees of Freedom
Take \(n=4\), an intercept and nuisance predictor \(z=(1,-1,0,0)'\), and tested predictor \(x=(0,0,1,-1)'\). Let \(y=(6,2,6,-2)'\). The three design columns are mutually orthogonal: each non-intercept column sums to zero, and \(z'x=0\).
The response decomposes as \(y=3\mathbf 1+2z+4x+w\), where \(w=(1,1,-1,-1)'\). Indeed, the right side is \((3+2+0+1,\ 3-2+0+1,\ 3+0+4-1,\ 3+0-4-1)'=(6,2,6,-2)'\). The restricted model retains the intercept and \(z\), and its residual is \(4x+w\). The full model additionally fits \(4x\), leaving residual \(w\).
Because \(x'w=0\), \(x'x=2\), and \(w'w=4\), the hypothesis sum of squares is \(H=4^2(2)=32\), while the error sum of squares is \(E=4\). The restricted residual sum of squares is \(32+4=36\). The full design has rank 3, so the error degrees of freedom are \(4-3=1\), and the partial \(F\) statistic is \(32/(4/1)=8\). The fraction of the restricted residual variation removed by adding \(x\) is \(H/(H+E)=32/36=8/9\); this describes the incremental fit, not the proportion of total response variation explained by the entire model.
The Exact Univariate Partial F Test
The projection decomposition also yields the exact classical \(F\) test when there is one response. Assume the fixed-design model \(y=Z\beta_0+X\beta_1+\varepsilon\), with \(\varepsilon\sim N(0,\sigma^2 I_n)\), \(\sigma^2>0\), and the rank conditions stated above. Let \(\nu=n-p_0-q>0\). Under \(H_0:\beta_1=0\), the partial statistic \((H/q)/(E/\nu)\) has an \(F_{q,\nu}\) distribution.
Proof. Let \(Q_H=P_F-P_Z\) and \(Q_E=I-P_F\). These are symmetric idempotent matrices, with ranks \(q\) and \(\nu\), respectively, and \(Q_HQ_E=0\). Under the null, \(M_Zy=Q_H\varepsilon+Q_E\varepsilon\), so \(H=y'Q_Hy=\varepsilon'Q_H\varepsilon\) and \(E=y'Q_Ey=\varepsilon'Q_E\varepsilon\). For a normal vector with covariance \(\sigma^2I_n\), an orthogonal change of coordinates diagonalizes each projection into zeros and ones. Consequently, \(H/\sigma^2\) is the sum of squares of \(q\) independent standard normal coordinates, and \(E/\sigma^2\) is the sum of squares of \(\nu\) other standard normal coordinates. They are independent because the projection spaces are orthogonal and the transformed normal coordinates are independent. Therefore \(H/\sigma^2\sim\chi^2_q\), \(E/\sigma^2\sim\chi^2_\nu\), independently. Dividing each by its degrees of freedom and taking their ratio gives \(F_{q,\nu}\). \(\square\)
For multiple responses, \(H\) and \(E\) remain the relevant matrix-valued decomposition, but the scalar \(F\) result above is not a general multivariate test. A multivariate procedure must account for the response covariance and the matrix structure; the decomposition identifies the tested effect and residual variation without itself choosing a multivariate criterion.
Common Mistake: Testing the Block Without Adjusting for Nuisance Predictors
A common error is to regress \(Y\) on \(X\) alone and interpret that coefficient as the partial effect of \(X\). This is generally wrong when the tested predictors are associated with the nuisance design. The \(X\)-only fit uses variation that may also be attributable to \(Z\), so it does not compare the intended restricted and unrestricted models.
The correction is to retain \(Z\) in both models, or equivalently to residualize both \(Y\) and \(X\) against \(Z\) before calculating the tested-block estimate. In the first worked example, the raw \(x\) is not the tested direction after adjustment: its nuisance projection is \((1,0,1,0)'\), and the relevant predictor is \(M_Zx=(-1,0,1,0)'\). The nested-model difference \(H\), rather than an unadjusted comparison, isolates the contribution being tested.
Check Your Understanding
Answer these questions by referring to the nested-model and projection interpretation.
- In \(Y=ZB_0+XB_1+U\), what is restricted under the partial null, and what remains unrestricted?
- Why must the tested predictors be adjusted for \(Z\) when estimating their partial contribution?
- State the projection formulas for \(H\) and \(E\), and explain why their sum equals the restricted residual cross-product matrix.
- For a univariate Gaussian linear model, what are the numerator and denominator degrees of freedom in the exact partial \(F\) test?
- Why does the univariate partial \(F\) result not by itself specify a test for a multivariate response?