Cubic Regression Splines and Smoothness Constraints

Why Join Cubic Pieces?

A single polynomial can describe a curved relationship between a predictor and a response, but its shape is constrained across the whole range of the predictor. Piecewise regression offers another option: use different polynomial pieces over different ranges, then join them at selected predictor values called knots. The challenge is to avoid a fitted curve that makes an abrupt jump or changes direction unnaturally at a join.

For cubic pieces, the joins can be controlled by matching the pieces’ values and derivatives at each knot. This tutorial defines that construction and shows what the matching conditions mean. It focuses on the shape of the fitted function, rather than on how to count its parameters or choose a particular polynomial basis.

The word “cubic” describes the degree allowed for each piece; it does not mean the entire spline is one cubic polynomial. Different intervals can have different polynomial coefficients. The first derivative describes the local slope of the curve, and the second derivative describes how that slope changes. Thus, matching values joins the pieces without a jump, matching first derivatives avoids a sharp corner, and matching second derivatives avoids a jump in curvature.

The Regression Model and the Joining Conditions

Let \(x_i\) be a predictor value and \(Y_i\) the corresponding response for observation \(i\). A regression spline specifies a function \(f\) for the conditional mean of the response:

For the usual least-squares regression model, the errors are taken to be independent with constant variance, so \(\operatorname{Var}(\varepsilon_i\mid x_i)=\sigma^2\). These assumptions concern the observations around the curve; they do not themselves guarantee that the curve is a spline. The spline requirement is a separate condition on the form of \(f\). Gaussian errors are not needed to define a spline or to describe its smoothness.

Suppose a knot is located at \(x=\kappa\), with a polynomial \(p_L\) to its left and a polynomial \(p_R\) to its right. A C² join requires:

The first equality makes the curve continuous: both pieces give the same fitted response at the knot. The second makes the slope agree as the curve passes through the knot. The third makes the curvature agree there. A continuous curve with a matching slope but mismatched second derivatives is smoother than a curve with a corner, but it does not meet the C² condition.

Why the Conditions Match the Local Polynomial Shape

The joining rule can be justified by writing each cubic piece in terms of its value and derivatives at the knot. Any polynomial of degree at most 3 can be expressed around \(x=\kappa\) as a constant, a linear term, a quadratic term, and a cubic term. The coefficients of the first three terms are determined by the value, first derivative, and second derivative at \(\kappa\).

Here \(c\) is the coefficient of the cubic term in this centered expression. To see why the formula applies, write a general cubic as \(a+b(x-\kappa)+d(x-\kappa)^2+c(x-\kappa)^3\). Substituting \(x=\kappa\) gives \(p(\kappa)=a\); differentiating and substituting gives \(p'(\kappa)=b\); differentiating twice gives \(p''(\kappa)=2d\). Replacing \(a\), \(b\), and \(d\) yields the displayed expression.

Apply this expression to the left and right pieces at the same knot. If their values, first derivatives, and second derivatives match, all terms through degree 2 are identical on both sides. Their difference is then just a multiple of \((x-\kappa)^3\):

At \(x=\kappa\), this difference and its first two derivatives are zero. Thus, even if the cubic coefficients differ, the pieces meet with equal value, slope, and curvature. This also explains why the third derivative can differ: differentiating the difference three times gives \(6(c_R-c_L)\), which need not be zero.

Distinguishing Smoothness From Continuity Alone

A curve can be continuous without having a continuous slope. It can also have a continuous slope without having a continuous second derivative. These are different levels of smoothness, so checking only that the two pieces share the same value at the knot is not enough to establish a C² join.

Interpreting a Fitted Cubic Spline

A fitted cubic spline represents a flexible mean relationship using polynomial pieces, while its joining conditions keep the curve from jumping or acquiring a sharp corner at an interior knot. The curve’s local slope and curvature can still vary across the predictor range, because each interval has its own polynomial piece. The result is not a claim that the underlying relationship is exactly cubic everywhere; it is a chosen regression form for describing the mean response.

The derivatives also carry units. If the response is measured in milligrams and the predictor in hours, the first derivative is measured in milligrams per hour and the second derivative in milligrams per hour squared. Matching a derivative means that its left- and right-hand values agree in those units at the knot. It does not mean the derivative is zero.

The C² requirement concerns interior knots where two modeled pieces meet. It does not by itself specify behavior beyond the ends of the observed predictor range. Nor does a smooth join guarantee that the overall fitted curve is scientifically sensible: the choice of knots and the fit to the observed data still matter.

Common Mistake

• Wrong reasoning: “The pieces have the same value at the knot, so the spline is smooth.” Matching values establishes continuity only; the curve can still have a corner or a curvature jump.
• Correction: For a C² cubic spline, check the value, first derivative, and second derivative from both sides at every interior knot. The third derivatives may differ.
• Another mistake: “A cubic spline must be one cubic polynomial over the entire predictor range.” Each interval has its own polynomial piece; “cubic” limits the degree of those pieces, not the number of pieces.