Balancing Fit and Simplicity When Comparing Curves

Why Fit Is Not the Only Goal

A more flexible regression curve can usually follow the observed data more closely than a simpler one. That improvement is not automatically useful: extra flexibility can follow random variation, make the fitted relationship harder to explain, and produce a curve whose shape is not well supported. A useful comparison therefore asks not only which curve fits better, but also whether the improvement is large and meaningful enough to justify the extra complexity.

This tutorial focuses on comparisons made using the same observations and outcome. Adjusted \(R^2\) and, for nested models, a partial \(F\) test provide formal summaries of fit improvement and complexity. Residual patterns and scientific interpretation remain important: a statistic alone does not establish that a curve is plausible or appropriate.

Complexity, Fit, and Comparable Models

A quadratic curve, for example, has an intercept, a linear coefficient, and a quadratic coefficient, so \(p=3\). A cubic curve adds a coefficient and has \(p=4\). For a smoothing spline, the number of basis coefficients may not directly describe its flexibility; effective degrees of freedom summarize that flexibility. In every comparison, the complexity measure should match the type of model being fitted.

Let \(\mathrm{SSE}\) be the sum of squared residuals and \(\mathrm{SST}\) the total sum of squares about the sample mean. Ordinary \(R^2=1-\mathrm{SSE}/\mathrm{SST}\) records the fraction of observed variation accounted for by the fitted model. When a model adds parameters, its \(\mathrm{SSE}\) cannot increase for nested least-squares models, so ordinary \(R^2\) cannot decrease. It does not, by itself, charge the more complex model for its added flexibility.

Adjusted \(R^2\) is not a probability that a model is correct, and it does not guarantee that the selected curve will predict well on new observations. It is most directly comparable when candidate fits use the same response, observations, and scale. A model with an extra parameter can have a higher ordinary \(R^2\) but a lower adjusted \(R^2\).

Comparing Nested Models with a Partial \(F\) Test

One formal comparison is available when the simpler model is a special case of the more complex model. Such models are nested. For instance, a quadratic curve is nested within a cubic curve: setting the cubic coefficient to zero gives the quadratic form. If the larger model adds \(q\) parameters, the partial \(F\) statistic measures the reduction in residual variation per added parameter relative to the larger model's residual mean square.

Under the null hypothesis and these assumptions, this statistic has an \(F\) distribution with \(q\) and \(n-p_{\mathrm{large}}\) degrees of freedom. A small p-value is evidence that the added terms improve the fit beyond what would be expected from random error under the null. It is not proof that the larger model is the best scientific explanation, nor does a non-small p-value prove that the simpler model is true.

Why the Adjusted Measure Penalizes Complexity

The adjustment uses \(\mathrm{SSE}/(n-p)\), rather than \(\mathrm{SSE}\) alone, to account for the degrees of freedom consumed by fitting parameters. The total variation is scaled by \(n-1\), the degrees of freedom used to estimate the sample mean. Consequently, adding a term must reduce residual variation enough to compensate for the smaller residual degrees of freedom if adjusted \(R^2\) is to rise.

For one added parameter, let the smaller model have \(p\) parameters and the larger model have \(p+1\). Because both use the same \(\mathrm{SST}\), adjusted \(R^2\) increases exactly when

Rearranging this inequality gives

The left side is the partial \(F\) statistic for that one added parameter. Thus adjusted \(R^2\) rises when this statistic exceeds 1. That is a much less demanding threshold than many conventional significance levels; an increase in adjusted \(R^2\) is not the same claim as statistically significant evidence. This distinction is one reason to use several kinds of evidence rather than treating adjusted \(R^2\) as a final decision rule.

Use Residual Evidence and Interpretation Too

Fit summaries answer limited questions. Adjusted \(R^2\) summarizes residual variation after a parameter-count adjustment. A partial \(F\) test evaluates a specified set of added terms under its assumptions. Neither tells you by itself whether the fitted shape corresponds to a credible relationship or whether remaining residual structure is important. Compare residual plots across candidates, paying attention to whether a more complex curve removes a broad, systematic pattern or merely follows a few observations. Also ask whether the added shape can be described in a way that makes sense for the problem.

For models that are not nested, the partial \(F\) comparison above does not apply. Adjusted \(R^2\) can still be a descriptive comparison when models use the same response, data, and outcome scale, but it should not be presented as a nested-model test. Interpretability and residual evidence then remain essential parts of the choice. Prediction on held-out observations is another kind of evidence, but it answers a different question.

Common Mistakes

• Choosing the largest ordinary \(R^2\). For nested least-squares fits, ordinary \(R^2\) does not decrease when terms are added. Compare complexity as well, and inspect residual evidence.
• Treating adjusted \(R^2\) as a significance test. It is a complexity-adjusted descriptive measure. Use a partial \(F\) test for the specified nested-model hypothesis when its assumptions are reasonable.
• Calling a non-small p-value proof of the simpler curve. It means the data do not provide strong evidence against the stated null under the assumptions; it does not prove that every added effect is absent.
• Using a partial \(F\) test for nonnested curves. The formula requires the smaller model to be a special case of the larger model. For nonnested candidates, do not interpret this statistic as the relevant test.
• Ignoring meaning and residuals. A numerical improvement does not by itself make a curve scientifically credible. Ask what shape the model asserts and whether the residual pattern supports it.