Local-Support Spline Series Estimation

Why Use Local-Support Splines?

A regression series estimator represents an unknown regression function as a weighted sum of basis functions. Fourier bases are global: changing one coefficient can affect the fitted curve across the whole domain. B-splines offer a different structure. Each basis function is nonzero only over a short interval, so changing one coefficient alters the fit only locally. That compact support is useful both computationally and statistically: it produces sparse design matrices, while the knot spacing determines how finely the fitted curve can vary.

The central choices are the spline degree and the spacing between knots. Higher degree permits more accurate approximation of sufficiently smooth functions, while smaller spacing supplies more basis functions and reduces approximation error. But more coefficients must then be estimated from the same sample. The calculations below make this tradeoff explicit for fixed-design regression.

B-Spline Bases and Local Support

Definition: A spline of degree \(p\) is a function that is a polynomial of degree at most \(p\) on each interval between consecutive knots, with smoothness conditions imposed at the interior knots. With simple interior knots, the usual spline space is \(p-1\) times continuously differentiable there. The order is \(q=p+1\). A B-spline basis is a particular basis for this space whose functions are nonnegative and have compact support.

For concreteness, take the domain \([0,1]\), a uniform knot spacing \(h=1/J\), and a fixed degree \(p\). Use a standard boundary construction, such as a clamped knot sequence, so that the B-spline basis \(B_1,\ldots,B_K\) spans the spline space on \([0,1]\). Here \(K=J+p\) for this construction. The precise number can differ with boundary conventions, but for fixed \(p\), \(K\) is proportional to \(J=1/h\).

Each basis function is nonzero on at most \(q=p+1\) consecutive knot intervals. At any point \(x\), at most \(q\) basis functions are nonzero. The basis functions are nonnegative and, on the domain, sum to one:

B_k(x)\geq 0,\qquad \sum_{k=1}^{K}B_k(x)=1.

These properties mean that the fitted value \(\sum_k \beta_k B_k(x)\) depends on at most \(q\) coefficients at a given \(x\). They also distinguish local-support splines from global bases, while leaving the overall series-estimation framework unchanged.

Worked Example: Counting Active Basis Functions

Take degree \(p=2\), so the order is \(q=3\), with uniform spacing \(h=0.2\). A quadratic B-spline has support over at most three consecutive knot intervals, whose total length is at most \(3h=0.6\). At any particular \(x\), at most three basis functions are nonzero. Thus, although the full basis has about \(J=5\) functions in this small illustration, evaluating the fitted curve at one point requires at most three terms.

For a finer partition with \(J=100\), the full coefficient vector has roughly 100 entries, but a pointwise fitted value still uses at most three active basis functions. The number of coefficients grows as the knots get denser; the number of basis functions contributing at one point stays bounded when the degree is fixed.

Regression Series Estimator

Suppose the fixed-design observations follow

Y_i=m(x_i)+\varepsilon_i,\qquad i=1,\ldots,n,

where \(m\) is the regression function, the errors are independent with \(E[\varepsilon_i]=0\) and \(\operatorname{Var}(\varepsilon_i)=\sigma^2\), and the design points \(x_i\) are nonrandom. Let \(B(x)=(B_1(x),\ldots,B_K(x))^\mathsf{T}\), and let \(\mathbf{B}\) be the \(n\times K\) matrix whose \(i\)th row is \(B(x_i)^\mathsf{T}\). The least-squares spline series estimator is

\hat m(x)=B(x)^\mathsf{T}\hat\beta,\qquad \hat\beta=(\mathbf{B}^\mathsf{T}\mathbf{B})^{-1}\mathbf{B}^\mathsf{T}Y,

when \(\mathbf{B}^\mathsf{T}\mathbf{B}\) is invertible. This is the fixed-design series estimator expressed in a local-support basis. The support of the basis makes \(\mathbf{B}\) sparse: each observation has at most \(q\) nonzero entries in its row.

Worked Example: A Piecewise-Constant Spline Fit

Let \(p=0\), so \(q=1\), and divide \([0,1]\) into \(J=5\) equal intervals. The B-spline basis functions are the interval indicators. Suppose the fixed design has 100 points, with exactly 20 in each interval, and the errors have variance \(\sigma^2=4\). In each interval, the least-squares coefficient is the average of its 20 observations.

For any \(x\) in one interval, the fitted value is that interval's average. Its variance is

\operatorname{Var}\{\hat m(x)\} =\operatorname{Var}\left(\frac{1}{20}\sum_{i=1}^{20}Y_i\right) =\frac{1}{20^2}\sum_{i=1}^{20}4 =\frac{80}{400} =0.2.

Only the coefficient for that interval affects the fitted value at \(x\). With more intervals, the fit can change over shorter distances, but fewer observations are available to estimate each interval average unless \(n\) also increases.

Approximation Error and Spline Degree

The approximation property of splines describes how closely the spline space can represent a smooth function before any data are used. For fixed \(p\), a standard local B-spline quasi-interpolant \(Q_h\) has two properties: it reproduces every polynomial of degree at most \(p\), and its operator norm on bounded functions is bounded by a constant independent of \(h\). Its coefficients are constructed from function values in a bounded neighborhood of each basis function. These are the properties needed for the following error argument.

Approximation bound: If \(m\in C^{p+1}[0,1]\), then for a uniform knot spacing \(h\) and fixed degree \(p\), there is a spline \(s_h\) such that \(\|m-s_h\|_\infty\leq C h^{p+1}\|m^{(p+1)}\|_\infty\). The constant \(C\) may depend on \(p\) and the fixed boundary convention, but not on \(h\) or \(m\).

To see why, take \(s_h=Q_hm\). At a point \(x\), let \(T_x\) be the degree-\(p\) Taylor polynomial of \(m\) about \(x\). Taylor's theorem gives, for \(u\) in a fixed neighborhood of \(x\),

|m(u)-T_x(u)| \leq \frac{\|m^{(p+1)}\|_\infty}{(p+1)!}|u-x|^{p+1}.

The local construction of \(Q_h\) uses values within distance at most \(C_0h\) of \(x\), where \(C_0\) depends only on \(p\) and the boundary convention. Polynomial reproduction gives \(Q_hT_x=T_x\). Therefore,

|m(x)-Q_hm(x)| =|(m-T_x)(x)-Q_h(m-T_x)(x)| \leq (1+\|Q_h\|)\,\|m-T_x\|_{\text{local}} \leq C h^{p+1}\|m^{(p+1)}\|_\infty.

The uniform bound on \(\|Q_h\|\) and Taylor's remainder yield the stated rate. The argument shows the role of both ingredients: polynomial reproduction uses the degree, and the neighborhood of width proportional to \(h\) supplies the power of the knot spacing.

Worked Example: Linear Splines Approximating a Quadratic

Let \(m(x)=x^2\) on \([0,1]\), and use degree \(p=1\) with \(J=4\), so \(h=1/4\). On one interval \([a,a+h]\), the linear interpolant joins the endpoint values. Its error is

$$ m(x)-s_h(x)=(x-a)(x-a-h). $$

The error is zero at both endpoints. Its absolute value is largest at the midpoint \(x=a+h/2\), where it equals \(h^2/4\). Thus the maximum error over all four intervals is

\|m-s_h\|_\infty=\frac{h^2}{4} =\frac{(1/4)^2}{4} =\frac{1}{64} =0.015625.

This is an exact calculation for this function and this interpolant, and it agrees with the general \(h^{p+1}=h^2\) rate. A degree-2 spline space contains \(x^2\) exactly, so its approximation error for this particular function is zero. That exactness does not mean degree 2 always gives a better fitted estimator: its coefficients still have to be estimated from noisy observations.

Coefficient Variance from Local Support

To quantify estimation variance, impose an explicit fixed-design stability condition. Assume there are constants \(0<c<C<\infty\), independent of \(n\) and \(h\), such that

cnh\,\|a\|_2^2 \leq a^\mathsf{T}\mathbf{B}^\mathsf{T}\mathbf{B}a \leq Cnh\,\|a\|_2^2 \qquad\text{for every }a\in\mathbb{R}^K.

This condition says that the design supplies information across the spline basis without near-linear dependence in the sampled columns. It is appropriate only when the design covers the knot intervals adequately; compact support alone does not guarantee it.

Variance bound: Under independent errors with common variance \(\sigma^2\) and the stated Gram-matrix condition, \(\operatorname{Var}\{\hat m(x)\}\leq \sigma^2/(cnh)\) for every \(x\). Also, each coefficient has variance at most \(\sigma^2/(cnh)\).

Indeed, the estimator is linear in \(Y\), so

\operatorname{Cov}(\hat\beta) =\sigma^2(\mathbf{B}^\mathsf{T}\mathbf{B})^{-1}.

The lower Gram bound implies that the largest eigenvalue of this covariance matrix is at most \(\sigma^2/(cnh)\). Hence each coefficient variance, a diagonal entry of the covariance matrix, is at most that amount. For a fitted value, use the same eigenvalue bound and the B-spline properties:

\operatorname{Var}\{\hat m(x)\} =\sigma^2 B(x)^\mathsf{T}(\mathbf{B}^\mathsf{T}\mathbf{B})^{-1}B(x) \leq \frac{\sigma^2}{cnh}\|B(x)\|_2^2 \leq \frac{\sigma^2}{cnh}.

The last inequality follows because the entries of \(B(x)\) are nonnegative and sum to one, so \(\|B(x)\|_2^2\leq(\sum_k B_k(x))^2=1\). Thus the variance scale is \(1/(nh)\). Since \(K\) is proportional to \(1/h\), this is also the familiar order \(K/n\) for series estimation. Local support makes the pointwise calculation especially direct, but the Gram condition is essential to its conclusion.

Worked Example: A Variance Upper Bound for Linear Splines

Take degree \(p=1\), \(J=10\), \(h=0.1\), sample size \(n=200\), and error variance \(\sigma^2=1\). Suppose the fixed design satisfies the lower Gram bound with \(c=0.2\). Then \(cnh=0.2(200)(0.1)=4\). At every \(x\), the fitted-value variance therefore satisfies

\operatorname{Var}\{\hat m(x)\} \leq \frac{1}{4}=0.25.

This is an upper bound, not an exact variance. It also relies on the stated design condition. If many knots had no observations nearby, the lower Gram bound could fail, and the calculation would no longer justify a variance bound of this form.

Balancing Approximation and Estimation

For a smooth \(m\), the approximation error has order \(h^{p+1}\), so its squared contribution has order \(h^{2(p+1)}\). Under the fixed-design stability condition, the integrated variance has order \(1/(nh)\). When the least-squares spline fit inherits the approximation bound in the design norm and that norm is comparable to the integrated squared error, these terms give the usual risk balance:

\text{risk}\ \lesssim\ h^{2(p+1)}+\frac{1}{nh}.

The first term decreases as knots get closer; the second increases because smaller \(h\) means more coefficients. Balancing the terms gives

h^{2(p+1)}\asymp\frac{1}{nh} \quad\Longrightarrow\quad h\asymp n^{-1/(2p+3)}, \qquad \text{risk}\asymp n^{-2(p+1)/(2p+3)}.

This calculation is a rate comparison under the stated smoothness and design conditions, not a universal prescription. The degree should reflect the smoothness one is prepared to assume, and the knot spacing should leave enough observations to estimate the resulting coefficients.

Worked Example: Choosing Knot Spacing for Linear Splines

For degree \(p=1\), the approximation term is of order \(h^4\) and the variance term is of order \(1/(nh)\). With \(n=1000\), balancing gives \(h\asymp n^{-1/5}\). Since \(1000^{1/5}\) is approximately \(3.98\), this suggests about \(J=4\) intervals and \(h\) near \(0.25\).

At that spacing, the two rate terms, ignoring their unknown constants, are

h^4=(0.25)^4=0.00390625,\qquad \frac{1}{nh}=\frac{1}{1000(0.25)}=0.004.

They are close, as the balancing calculation predicts. This does not establish that four intervals are optimal for a particular data set: the constants, the function's smoothness, and the fixed-design distribution all matter.

Common Mistake: Compact Support Does Not Guarantee Stability

A common mistake is to reason that because each B-spline uses observations only nearby, its coefficient must be estimated precisely. Compact support only identifies which observations can contribute. It does not ensure that there are enough such observations, or that the columns of \(\mathbf{B}\) are well conditioned. If a knot interval contains too few design points, or relevant basis functions are not distinguishable on the observed design, the Gram matrix can be nearly singular. The variance can then be much larger than \(1/(nh)\), or the least-squares coefficients may not be uniquely defined.

The correction is to keep the two roles separate. Knot spacing and degree control the spline space's approximation capacity; the design and the Gram matrix control how reliably its coefficients can be estimated. A small approximation error is useful only when the data support stable estimation at that resolution.

Key takeaway: For fixed degree \(p\), B-splines have at most \(p+1\) active basis functions at any point. Smooth functions admit approximation error of order \(h^{p+1}\), while stable coefficient estimation gives variance of order \(1/(nh)\). Choosing \(h\) balances these opposing effects; compact support does not replace the need for adequate design coverage.

Check Your Understanding

Use the definitions and conditions above to answer the following.

For a degree-3 spline, what is its order, and how many basis functions can be nonzero at a given point?
If a smooth function has approximation error of order \(h^{p+1}\), what is the order of the squared approximation contribution to risk?
Why does the variance bound depend on a lower bound for the Gram matrix, rather than on compact support alone?
For degree \(p=2\), balance \(h^{2(p+1)}\) against \(1/(nh)\) and give the resulting order of \(h\).
Explain why increasing the number of knots can reduce approximation error while increasing coefficient-estimation variance.