Testing an Omitted Quadratic Term With a Partial F Test

Why Test a Squared Predictor?

A straight-line relationship can be a useful first description of how a response changes with a predictor. But the response may rise or fall at a changing rate: for example, it may rise quickly at first and then level off. A squared predictor lets a regression model represent this kind of curvature while retaining the original linear term.

A residual plot can suggest that a straight-line mean structure misses curvature. A partial F test gives a formal comparison between that straight-line model and a specified model that adds the squared predictor. The test asks whether the added term accounts for enough reduction in residual variation to be evidence against the simpler model, given the assumptions and the other predictors already included.

The Reduced and Full Models

Suppose \(Y\) is a quantitative response, \(x\) is the predictor whose curvature is in question, and \(z_1,\ldots,z_k\) are other predictors that should remain in both models. The reduced model has a linear term for \(x\); the full model adds \(x^2\). Write the conditional mean models as follows:

\begin{aligned} \text{Reduced:}\quad E(Y\mid x,z_1,\ldots,z_k) &=\beta_0+\beta_1x+\sum_{j=1}^{k}\delta_jz_j,\\ \text{Full:}\quad E(Y\mid x,z_1,\ldots,z_k) &=\beta_0+\beta_1x+\gamma x^2+\sum_{j=1}^{k}\delta_jz_j. \end{aligned}

Definition: A partial F test for an omitted quadratic term compares these nested models, using the same response observations and the same other predictors. Its null hypothesis is \(H_0:\gamma=0\); its two-sided alternative is \(H_a:\gamma\ne 0\). The test evaluates whether adding \(x^2\), while keeping \(x\) and the other terms, improves the model enough to count as evidence of curvature in the conditional mean.

The coefficient \(\gamma\) describes the quadratic component, conditional on the other included predictors. If \(\gamma>0\), the fitted mean bends upward as \(x\) increases; if \(\gamma<0\), it bends downward. The sign alone does not establish that the quadratic term is useful, and the F statistic itself has no sign. Examine the estimated coefficient and the test result together.

For a standard exact partial F test, assume the model errors are independent, have conditional mean zero and a common variance, and are normally distributed. Also require that the full model is estimable: its columns are not exactly linearly dependent, and the full model has positive residual degrees of freedom. The reduced model is nested in the full model, and both are fit to the same observations by least squares. These assumptions make the reference F distribution exact; in practice, residual diagnostics help assess whether the model is a reasonable description.

The Partial F Calculation

Let \(SSE_R\) and \(SSE_F\) be the residual sums of squares from the reduced and full models. Adding a term cannot increase the least-squares \(SSE\) on the same data, so \(SSE_R-SSE_F\) measures the reduction attributed to the added term. Let \(q\) be the number of added coefficients, \(p_F\) the number of coefficients estimated in the full model including the intercept, and \(n\) the number of observations. The full-model residual degrees of freedom are \(n-p_F\).

F=\frac{(SSE_R-SSE_F)/q}{SSE_F/(n-p_F)} \quad\sim\quad F_{q,n-p_F}\ \text{under }H_0.

For one added squared term, \(q=1\). Thus the numerator is the reduction in \(SSE\) per added coefficient, which here is simply \(SSE_R-SSE_F\). The denominator is the full model's residual mean square. A large value of \(F\) means the reduction from adding \(x^2\) is large relative to the remaining residual variation. The p-value is the probability, assuming \(H_0\) and the model assumptions, of obtaining an F statistic at least as large as the observed value.

Why the comparison has this form: Under \(H_0\), both models describe the same conditional mean structure. The added-term reduction in \(SSE\), divided by the error variance, has a chi-square distribution with \(q\) degrees of freedom; the full-model residual sum of squares, divided by the error variance, has a chi-square distribution with \(n-p_F\) degrees of freedom. Under the normal-error model these quantities are independent. Dividing each by its degrees of freedom therefore gives the stated F ratio. For a single added coefficient, this partial F statistic also equals the square of the usual \(t\) statistic for that coefficient.

For this comparison, keep the linear \(x\) term in the full model. Testing \(x^2\) while dropping \(x\) asks a different question and can distort the intended comparison. Also ensure that the reduced and full models use exactly the same observations; otherwise their \(SSE\) values do not give the required nested-model comparison.

Worked Example: Testing Curvature in a Clinical Measurement

In a hypothetical data set, a researcher models a clinical measurement \(Y\), in millimeters of mercury, using daily activity \(x\), in hours, age, and a medication indicator. The reduced model contains an intercept, \(x\), age, and medication; the full model adds \(x^2\). Both are fit to the same \(n=40\) people. Suppose \(SSE_R=210\) and \(SSE_F=186\), in squared response units. Assume observations are independent, the residual diagnostics show no substantial unequal spread or severe nonnormality, and the predictors do not create exact collinearity.

The full model has five coefficients: an intercept, \(x\), \(x^2\), and the two other predictors' coefficients. Thus \(p_F=5\), the full-model residual degrees of freedom are \(40-5=35\), and one coefficient is added, so \(q=1\).

Calculate the statistic.

\begin{aligned} F &=\frac{(210-186)/1}{186/35}\\ &=\frac{24}{5.3143}\\ &=\frac{840}{186}\\ &=4.516. \end{aligned}

With \(df_1=1\) and \(df_2=35\), the p-value is approximately \(0.041\). At \(\alpha=0.05\), reject \(H_0\): these data provide evidence that the quadratic term improves the conditional-mean model, after accounting for activity, age, and medication. This conclusion does not say that activity causes the clinical measurement to change. If the fitted \(\gamma\) is negative, the estimated curve bends downward; if positive, it bends upward.

A residual-versus-activity plot that shows a systematic curved pattern would be compatible with this test result. Such a plot is supporting diagnostic evidence, not an additional conclusion guaranteed by the p-value.

Read the Test Alongside Residual Evidence

A useful diagnostic question is whether residuals from the reduced model show a pattern consistent with the omitted structure. For a missing quadratic term, residuals plotted against \(x\) may tend to be negative at one range and positive at another, or may trace a curved pattern around zero. A predictor-specific residual smooth can make such structure easier to see, especially when other predictors are also in the model.

The test and the plot answer related but different questions. The partial F test assesses whether the specified added term gives a statistically detectable reduction in residual variation under its assumptions. The plot can reveal patterns the quadratic term may not capture, or warn that the model's assumptions are questionable. A small p-value does not prove that the quadratic model is adequate; a large p-value does not prove that the true relationship is exactly linear.

Worked Example: A Curved-Looking Plot With Weak Test Evidence

Suppose a hypothetical experiment relates plant yield, in grams, to fertilizer dose \(x\), while adjusting for soil pH. The reduced model has an intercept, \(x\), and pH; the full model adds \(x^2\). Both fit the same \(n=30\) plants. Assume independent errors, approximately constant residual spread, no major departure from normality, and no exact collinearity. The residual plot from the reduced model has a slight bowl-shaped appearance but no strong pattern.

Suppose the reduced-model \(SSE\) is 96 and the full-model \(SSE\) is 88. The full model estimates four coefficients, so its residual degrees of freedom are \(30-4=26\). One term is added.

\begin{aligned} F &=\frac{(96-88)/1}{88/26}\\ &=\frac{8}{3.3846}\\ &=\frac{208}{88}\\ &=2.364. \end{aligned}

For \(F_{1,26}=2.364\), the p-value is approximately \(0.136\). At the 0.05 level, do not reject \(H_0\). The data do not provide sufficiently strong evidence that adding the quadratic term improves this model. The slight bowl shape is a reason to inspect the specification, but it does not override the test or establish a quadratic relationship. The conclusion is not that curvature is impossible.

Worked Example: Curvature After Adjusting for Other Predictors

In a hypothetical education analysis, quiz score \(Y\), in points, is modeled using weekly practice hours \(x\), baseline score, and course format. The reduced model contains an intercept and these three predictors; the full model also contains \(x^2\). The same \(n=52\) students are used in both fits. Assume independent errors with approximately constant variance, residuals that are reasonably consistent with normality, and a full-rank design matrix. Suppose diagnostic plots for the reduced model show a curved trend in residuals against practice hours.

The reduced model has four coefficients, and the full model has five. Given \(SSE_R=154\) and \(SSE_F=130\), the full-model residual degrees of freedom are \(52-5=47\), and \(q=1\).

\begin{aligned} F &=\frac{(154-130)/1}{130/47}\\ &=\frac{24}{2.7660}\\ &=\frac{1128}{130}\\ &=8.677. \end{aligned}

For \(F_{1,47}=8.677\), the p-value is approximately \(0.005\). At \(\alpha=0.05\), there is evidence that the quadratic term improves the conditional-mean model after adjusting for baseline score and course format. If the fitted estimate of \(\gamma\) is \(-0.18\) points per squared practice hour, its negative sign describes downward bending in the fitted mean. It does not mean that each additional hour reduces the score by 0.18 points; the linear and quadratic terms jointly determine the fitted change at a given practice level.

Centering, Scope, and Common Mistakes

When \(x\) is large or far from zero, replacing it by a centered predictor \(x_c=x-c\) can make the coefficients easier to interpret or improve numerical stability. If the model includes an intercept and the linear term, adding \(x_c^2\) tests the same quadratic component as adding \(x^2\), because \(x_c^2=x^2-2cx+c^2\); the difference is absorbed by the existing linear and intercept terms. The linear coefficient changes with the choice of center, so interpret it with the actual coding in mind.

Wrong: “The full model has a larger \(R^2\), so curvature is established.” Adding terms cannot reduce ordinary \(R^2\), even when the added term has little value. Use the partial F test to evaluate the reduction relative to the full-model residual variation.
Wrong: “A significant quadratic test tells us the curve bends upward.” An F statistic is nonnegative and tests either direction. Check the estimated quadratic coefficient to determine the fitted direction.
Wrong: “The quadratic term is significant, so the model is correct.” The test only compares these two specified models. Inspect residual evidence and consider whether other misspecification remains.
Wrong: “Compare \(SSE\) values from fits with different observations.” The partial F calculation requires nested models fit to the same response observations and with the same retained predictors.
Wrong: “Drop the linear term when testing \(x^2\).” Keep \(x\) in the full model so the test asks whether curvature adds information beyond a linear relationship.

Key takeaway: Compare a reduced model containing \(x\) with a nested full model containing both \(x\) and \(x^2\). Calculate the partial F statistic from the two \(SSE\) values and the full-model residual degrees of freedom, then interpret the result together with the estimated coefficient and residual evidence.

Check Your Understanding

Answer each question using the model comparison and interpretation principles above.

Write the null and alternative hypotheses for adding \(x^2\) while retaining \(x\) and two other predictors.
A reduced model has \(SSE_R=125\); a full model has \(SSE_F=113\), with one added coefficient and 28 full-model residual degrees of freedom. Calculate the partial F statistic.
Why must the reduced and full models use the same observations for the partial F comparison?
A partial F test for \(x^2\) has a small p-value. What additional information is needed to describe the direction of the fitted curvature?
A residual smooth suggests curvature, but the quadratic-term test has a large p-value. State a careful interpretation that does not overclaim.