Testing Added Polynomial Terms

Why Test Added Polynomial Terms?

A straight-line regression may miss a curved pattern in the mean outcome. Adding a squared predictor, or still higher powers, lets a polynomial regression represent more shapes. But a more flexible model will fit the observed data at least as well as a nested, simpler model. The relevant question is therefore not whether the more complex model has a smaller residual sum of squares; it is whether the reduction in residual variation is large enough, relative to the number of added terms and the remaining error variation, to provide evidence of improved fit.

A partial F-test answers that question by testing the added powers jointly. It can compare, for example, a linear model with a quadratic model, or a linear model with a cubic model. It evaluates evidence for incremental fit within the specified models; by itself, it does not establish that the polynomial form is correct, that the relationship is causal, or that the improvement matters in practice.

Models and Hypotheses

Suppose the lower-degree model includes powers of a quantitative predictor \(x\) through degree \(r\), while the larger model includes powers through degree \(m\), where \(m>r\). The models may also include the same other predictors, such as control variables. Write their mean functions as:

\begin{aligned} \text{Reduced model:}\quad Y_i &= \beta_0+\beta_1x_i+\cdots+\beta_rx_i^r+\text{other terms}+\varepsilon_i,\\ \text{Full model:}\quad Y_i &= \beta_0+\beta_1x_i+\cdots+\beta_rx_i^r +\beta_{r+1}x_i^{r+1}+\cdots+\beta_mx_i^m +\text{other terms}+\varepsilon_i. \end{aligned}

The full model contains all terms in the reduced model plus \(q=m-r\) added powers, so the models are nested. The test asks whether all the added-power coefficients are zero:

H_0:\ \beta_{r+1}=\cdots=\beta_m=0 \qquad\text{versus}\qquad H_a:\ \text{at least one of }\beta_{r+1},\ldots,\beta_m\text{ is nonzero}.

The test is joint: a rejection supports the conclusion that the added powers, considered as a block, contribute evidence of improved fit. It does not say that every added coefficient is individually different from zero.

Definition: A partial F-test for added polynomial terms compares nested regression models that use the same outcome and observations. Its numerator measures the reduced model’s residual sum of squares (RSS) reduction per added coefficient; its denominator estimates the residual variance using the full model.

The Partial F Statistic

Let \(RSS_R\) and \(RSS_F\) be the residual sums of squares from the reduced and full models, respectively. Let \(q\) be the number of added terms, and let \(df_F=n-p_F\) be the full model’s residual degrees of freedom, where \(p_F\) is the number of estimated coefficients in that model, including the intercept. The test statistic is:

F=\frac{(RSS_R-RSS_F)/q}{RSS_F/df_F}.

The numerator is the average reduction in RSS per added coefficient. The denominator, \(RSS_F/df_F\), is the full model’s residual mean square and estimates the error variance. Under the null hypothesis and the assumptions below, \(F\) has an \(F\) distribution with \(q\) and \(df_F\) degrees of freedom. Large values indicate that the reduction in RSS is large relative to the full model’s residual variation.

For the usual exact test, the regression has a correctly specified conditional mean under the null, the errors are independent with mean zero and constant variance, and the errors are normally distributed. The predictor values may be treated as fixed, or inference may be conditional on the observed predictors. The model matrices must have full column rank so the coefficients are estimable. In practice, plots and knowledge of how the data were collected help assess whether the assumptions are reasonable. If they are not, the nominal F-test p-value may not have its stated interpretation.

Why the Statistic Has This Form

Under the null hypothesis, the reduced model’s mean function is sufficient: the extra terms in the full model should account only for random error. The difference \(RSS_R-RSS_F\) is the amount of variation removed by fitting the \(q\) added terms. In the fixed-design normal linear model, dividing this reduction by the error variance gives a chi-square quantity with \(q\) degrees of freedom under the null. Likewise, \(RSS_F\) divided by the error variance has a chi-square distribution with \(df_F\) degrees of freedom. These two quantities are independent because they correspond to orthogonal components of the fitted and residual variation. Dividing each by its degrees of freedom and taking their ratio cancels the unknown error variance, giving the stated F statistic and reference distribution.

Theorem: Under the null hypothesis and the regression assumptions, the partial F statistic has an \(F_{q,df_F}\) distribution. The p-value is the probability, assuming \(H_0\) and those assumptions, of obtaining a statistic at least as large as the observed \(F\).

Worked Comparisons

Worked Example: Testing a Quadratic Term With a Control Predictor

Suppose a hypothetical data set contains 24 independently observed patients. The outcome is a recovery score in points, and the predictors are a treatment-duration measure \(x\) and a baseline score \(z\). The reduced model includes an intercept, \(x\), and \(z\); the full model also includes \(x^2\). Assume the same 24 observations are used in both models, their design matrices have full column rank, and the residual checks and study design support independent, approximately normal errors with constant variance.

Suppose the reduced and full RSS values are 180 and 90 points squared. There is one added term, so \(q=1\). The full model estimates four coefficients: an intercept and coefficients for \(x\), \(x^2\), and \(z\). Thus \(df_F=24-4=20\), not 21. The calculation is:

F=\frac{(180-90)/1}{90/20} =\frac{90}{4.5} =20.0.

The upper-tail probability for \(F_{1,20}=20.0\) is approximately \(0.00023\). At a significance level of \(0.05\), reject \(H_0:\beta_{x^2}=0\). Under the model assumptions, the data provide strong evidence that adding the squared duration term improves fit beyond the linear duration and baseline-score terms. This is evidence of a contribution to the model’s fit, not evidence that changing treatment duration causes a change in recovery.

Worked Example: Testing Quadratic and Cubic Terms Together

In a second hypothetical analysis, 30 independently observed devices have a measured outcome in hours and a quantitative predictor \(x\). Compare a linear model with a cubic model that adds both \(x^2\) and \(x^3\). Suppose the observations are the same in both fits, the full design matrix has full column rank, and the residual and data-collection checks make the independent, normal, constant-variance error assumptions reasonable.

The cubic model estimates four coefficients: an intercept and the coefficients of \(x\), \(x^2\), and \(x^3\). Therefore \(df_F=30-4=26\); there are \(q=2\) added terms. If \(RSS_R=150\) hours squared and \(RSS_F=108\) hours squared, then:

F=\frac{(150-108)/2}{108/26} =\frac{21}{4.1538} \approx 5.056.

The p-value from \(F_{2,26}\) is approximately \(0.0140\). At the \(0.05\) level, reject the joint null that both added-power coefficients are zero. The results provide evidence that the quadratic and cubic terms together improve fit beyond the linear model. They do not establish that both terms individually contribute; the test concerns the two-term block.

Worked Example: A Small Reduction From Higher-Order Terms

Suppose a hypothetical set of 32 independently observed production batches has an outcome measured in grams. A quadratic model is compared with a quartic model, which adds the cubic and fourth-power terms. Assume the models use the same batches, the full design matrix has full column rank, and the study and residual checks support the stated regression assumptions.

The quartic model estimates five coefficients: an intercept and four power coefficients. Thus \(df_F=32-5=27\), and \(q=2\). If \(RSS_R=240\) grams squared and \(RSS_F=216\) grams squared, then:

F=\frac{(240-216)/2}{216/27} =\frac{12}{8} =1.50.

The p-value from \(F_{2,27}\) is approximately \(0.241\). At the \(0.05\) level, do not reject the joint null for the two added terms. The observed RSS reduction does not provide strong evidence, relative to residual variation, that adding the cubic and fourth-power terms improves fit beyond the quadratic model. This is not proof that the quadratic model is exactly correct or that the higher-order coefficients are zero.

Using the Result Responsibly

The partial F-test is useful when a scientific or practical question concerns whether a polynomial curve needs additional powers beyond a simpler model. For example, it can assess whether a linear description is adequate relative to a quadratic one, or whether a cubic model provides evidence of incremental fit over a linear model. A significant result indicates evidence against the joint null under the model assumptions. It does not quantify the practical importance of the change, guarantee good predictions outside the observed range, or show that the highest-order term alone is responsible.

A nonsignificant result is also limited: it says the data do not provide sufficient evidence against the joint null at the chosen significance level. It is not a demonstration that the added terms have no effect. The estimated RSS reduction and the context of the outcome remain relevant when describing the comparison, while practical importance requires a separate assessment.

Count coefficients, not just powers. The full model’s parameter count includes the intercept and any control-predictor coefficients. Its residual degrees of freedom are \(n-p_F\).
Test the block that answers the question. A linear-versus-cubic comparison jointly tests the quadratic and cubic terms, not just the cubic coefficient.
Do not compare unrelated fits with this formula. The models must be nested and use the same outcome and observations.
Do not equate significance with causation or importance. The test evaluates evidence about incremental fit under the model assumptions.

Key takeaway: To test added polynomial powers, compare the nested models’ RSS values, divide their reduction by the number of added terms, and compare that quantity with the full model’s residual mean square. Interpret the resulting F-test as evidence about incremental fit, subject to its assumptions.

Check Your Understanding

Answer each question by identifying the tested terms and the relevant model quantities.

A quadratic model is compared with a linear model. State the null hypothesis about the added polynomial coefficient.
A full model has \(n=40\) observations and estimates an intercept plus five predictor coefficients. What are its residual degrees of freedom?
A cubic model is compared with a linear model. How many terms are tested jointly, and which powers are they?
Why does a lower RSS for the full model not, by itself, establish that the added terms improve fit beyond random variation?
In a partial F-test, what does a small p-value mean, and what does it not establish?

What You'll Learn