Why Test a Categorical Predictor as a Block?
A categorical predictor with several levels is represented in a regression model by a set of indicator variables. For example, a four-level treatment group is represented by three indicators when one group is used as the reference. The coefficients describe category comparisons with that reference, conditional on the other predictors in the model.
It is often useful to ask a broader question: does this categorical predictor contribute to the model at all, after accounting for the other predictors? That question concerns the entire set of indicator coefficients, not any one coefficient separately. The partial F test answers it by comparing a full model containing the indicator block with a reduced model that omits the block but retains the other predictors.
Hypotheses and the Two Models
Suppose a categorical predictor has \(k\) levels and is represented by indicators \(D_1,\ldots,D_{k-1}\). Let the model also contain other predictors, such as quantitative variables. The block hypotheses are
The reduced model contains the other predictors but none of the indicators in this block. The full model contains those same predictors plus all \(k-1\) indicators. Thus, the models are nested: the reduced model is obtained by setting the block coefficients in the full model to zero. Keeping the other predictors in both models makes the comparison an assessment of the categorical predictor’s contribution adjusted for those predictors.
For the test, let \(SSE_R\) be the residual sum of squares from the reduced model, \(SSE_F\) the residual sum of squares from the full model, \(q\) the number of restrictions being tested, and \(df_F\) the full model’s residual degrees of freedom. For an ordinary \(k\)-level indicator block, \(q=k-1\).
The difference \(SSE_R-SSE_F\) cannot be negative when both models are fitted by least squares to the same data: adding predictors cannot make the minimum residual sum of squares larger. A large \(F\) statistic means the block reduces residual variation substantially relative to the full model’s residual variation. The p-value is the upper-tail probability \(P(F_{q,df_F}\geq F_{\text{observed}})\) under the joint null hypothesis.
Model Assumptions and Why the Statistic Works
The usual exact partial F inference assumes a correctly specified linear conditional mean, independent errors with mean zero and a common variance, and normally distributed errors. The predictor columns must also be estimable: the full design matrix must have full rank, so the coefficients being tested are not aliased with other columns. The two models must use the same response observations. These conditions justify the reference \(F\) distribution; a small p-value does not itself verify them.
The calculation follows from how the two nested models divide residual variation. Under the joint null, the \(q\) added coefficients have no true contribution. The reduction \(SSE_R-SSE_F\), scaled by the error variance, then behaves like a chi-square quantity with \(q\) degrees of freedom. The full-model residual sum of squares, scaled by the same variance, behaves like an independent chi-square quantity with \(df_F\) degrees of freedom. Dividing each by its degrees of freedom and taking their ratio cancels the unknown error variance:
Here \(\sigma^2\) is the common error variance in the assumed model. This argument also explains why the numerator degrees of freedom equal the number of restrictions: the block adds \(q\) parameters.
Worked Example: Testing a Four-Level Predictor
Suppose a hypothetical analysis uses 40 clinic visits to model an outcome in points. The predictors are age, a severity score, and clinic type, which has four levels. One clinic type is the reference, so clinic type contributes three indicators. The reduced model uses age and severity; its \(SSE_R=240\) points squared. The full model also includes the three indicators; its \(SSE_F=180\) points squared.
The full model estimates five slopes: age, severity, and three indicators. It also estimates an intercept, so \(df_F=40-5-1=34\). The block has \(q=3\) coefficients. The assumptions for using the test are that these nested models are appropriate for the outcome, the errors meet the stated independence, constant-variance and normality conditions, and the coefficients are estimable. The scenario supplies no evidence that the models use different visits or redundant columns.
Substitute into the partial F formula:
The reference distribution is \(F(3,34)\), and the upper-tail p-value is approximately \(0.019\). At \(\alpha=0.05\), reject the joint null hypothesis. These hypothetical data provide evidence that clinic type is associated with the outcome after accounting for age and severity. The result does not identify which clinic types differ, nor does it establish that clinic type causes the outcome to change.
Block Tests Are Not Individual Coefficient Tests
An individual coefficient test asks whether one particular coefficient is zero while the other model terms remain in the model. With reference-category coding, testing one indicator coefficient compares that category with the reference category, conditional on the other predictors. The block test instead asks whether the entire set of category comparisons can jointly be zero.
The distinction matters because evidence about one comparison does not by itself settle the joint question. A block test can detect a pattern across several category coefficients, while a particular category contrast may be weakly estimated. Conversely, a particular contrast can be of interest even when the block test is not statistically significant. The block test is not a collection of separate tests, and its p-value should not be described as the p-value for every category comparison.
Changing the reference category changes how the category comparisons are expressed as coefficients. It does not change the overall question of whether all category effects are zero, provided the same model space is represented. In contrast, the meaning of an individual indicator coefficient depends on the chosen reference category. Always state which question is being tested.
Worked Example: A Three-Level Predictor and Its Joint Test
In a hypothetical regression based on 36 observations, the outcome is measured in points. A continuous predictor \(x\) is included along with a three-level categorical predictor, represented by two indicators. The reduced model contains \(x\) only and has \(SSE_R=126\) points squared. The full model contains \(x\) and both indicators and has \(SSE_F=96\) points squared.
The full model estimates three slopes, one for \(x\) and two for the indicators, plus an intercept. Thus \(df_F=36-3-1=32\), and \(q=2\). Assume the same 36 observations are used in both models, the model terms are estimable, and the linear-model error assumptions for the exact test are reasonable.
The statistic is compared with \(F(2,32)\), giving an upper-tail p-value of approximately \(0.0129\). At the 0.05 level, there is evidence that the two indicator coefficients are not both zero after accounting for \(x\). This conclusion concerns the categorical predictor as a block; it does not say that each category differs significantly from the reference.
Worked Example: Testing One Category Contrast
Consider a separate hypothetical model with a three-level predictor. The reference category is A, and the coefficient for category C is estimated as \(3.0\) outcome points, with standard error \(1.5\) points. Suppose the full model has 27 residual degrees of freedom and meets the assumptions for the coefficient’s t test.
The individual null hypothesis \(H_0:\beta_C=0\) asks whether category C differs from category A, adjusted for the other predictors. Its test statistic is
For a two-sided test with 27 degrees of freedom, the p-value is approximately \(0.05565\). At \(\alpha=0.05\), this individual comparison does not meet the threshold for rejecting its null hypothesis. It is not a test of whether all coefficients for the three-level predictor are zero; that question requires a joint block test. Nor does this result establish that categories C and A have identical adjusted means.
Common Mistake and Practical Interpretation
A common mistake is to inspect one indicator’s t-test p-value and treat it as the overall test of the categorical predictor. That reasoning answers only whether that one category differs from the reference, not whether the complete indicator block contributes. The correction is to compare the reduced and full models with the partial F statistic, using \(q=k-1\) restrictions and the full model’s residual degrees of freedom.
A second important point is that “not statistically significant” is not the same as “no effect.” If a block test does not reject \(H_0\), the data do not provide sufficient evidence at the chosen significance level that the block improves the model. That conclusion may reflect limited precision as well as a small contribution. Report the model comparison and the question it addresses, rather than claiming the categories are equivalent.
Check Your Understanding
For each question, identify the relevant test or interpretation before calculating.
- A five-level predictor is coded with four indicators. How many restrictions are in its block test, and what is the joint null hypothesis?
- A reduced model has \(SSE_R=310\), a full model has \(SSE_F=250\), the full model adds three coefficients, and \(df_F=42\). Write the partial F statistic before evaluating it.
- Why must the reduced model retain the other predictors from the full model when testing a categorical block’s adjusted contribution?
- An indicator coefficient compares category B with the chosen reference category. What question does its individual t test answer, and what question does it not answer?
- If a block test is not significant at the selected level, is it correct to conclude that all category effects are exactly zero? Explain.