Why a Nonorthogonal Basis Needs a System
When a basis is orthogonal, each projection coefficient can be found with a separate inner product. A nonorthogonal basis behaves differently: its basis polynomials interact, so changing one coefficient affects the inner product with several basis elements. The coefficients must be found together by solving a system of equations. The matrix of that system is the Gram matrix, whose entries record the inner products between basis polynomials.
We work over \(\mathbb{R}\). Let \(P_m\) denote the vector space of real polynomials of degree at most \(m\), equipped with a specified inner product \(\langle\cdot,\cdot\rangle\). Let \(S\) be a subspace with ordered basis \((\phi_1,\dots,\phi_k)\), and let \(f\in P_m\). An inner product is positive definite, symmetric, and bilinear; in particular, \(\langle g,g\rangle\geq 0\), with equality only when \(g\) is the zero polynomial.
To understand the equations, write the residual as \(r=f-p\). We want it orthogonal to every polynomial in \(S\), and it is enough to require orthogonality to each basis element. By bilinearity and symmetry, the \(i\)-th requirement is
These are precisely the rows of \(\mathbf{G}\mathbf{c}=\mathbf{b}\). The basis need not be orthogonal; the off-diagonal entries of \(\mathbf{G}\) account for its interactions.
Why the Gram Equations Have a Unique Solution
Proof. First, the Gram matrix is invertible. For any coefficient column \(\mathbf{d}=(d_1,\dots,d_k)^\top\), let \(g=\sum_{i=1}^k d_i\phi_i\). Symmetry and bilinearity give \(\mathbf{d}^{\top}\mathbf{G}\mathbf{d}=\langle g,g\rangle\). If \(\mathbf{d}\ne\mathbf{0}\), at least one coefficient is nonzero, and basis independence implies \(g\ne\mathbf{0}\). Positive definiteness then gives \(\mathbf{d}^{\top}\mathbf{G}\mathbf{d}>0\). In particular, \(\mathbf{G}\mathbf{d}=\mathbf{0}\) is possible only for \(\mathbf{d}=\mathbf{0}\). A square matrix with trivial null space is invertible, so \(\mathbf{G}\mathbf{c}=\mathbf{b}\) has a unique solution.
For this solution, the \(i\)-th Gram equation says \(\langle f,\phi_i\rangle=\sum_j c_j\langle\phi_j,\phi_i\rangle\). Therefore \(\langle f-p,\phi_i\rangle=0\) for each \(i\). Every \(s\in S\) is a linear combination of the \(\phi_i\), so bilinearity gives \(\langle f-p,s\rangle=0\); that is, \(f-p\perp S\).
Finally, take any \(q\in S\). Since \(p\in S\), the difference \(s=q-p\) belongs to \(S\), and \(f-q=(f-p)-s\). The residual \(f-p\) is orthogonal to \(s\), so expanding the squared norm gives \(\|f-q\|^2=\|f-p\|^2+\|s\|^2\). The second term is nonnegative and equals zero only when \(s=\mathbf{0}\), or \(q=p\). Thus \(p\) is the unique closest element of \(S\) to \(f\). \(\square\)
Worked Polynomial Projections
Worked Example: Projection Using an Integral Inner Product
Use the inner product \(\langle g,h\rangle=\int_{-1}^{1}g(x)h(x)\,dx\). Find the closest polynomial to \(f(x)=x^2\) in \(S=\operatorname{span}\{1,1+x\}\). The displayed polynomials are a basis: they are independent, since \(a+b(1+x)=0\) as a polynomial forces \(b=0\) from the coefficient of \(x\), then \(a=0\). They are not orthogonal, because \(\langle 1,1+x\rangle=2\ne0\).
Compute the entries of the Gram matrix and the right-hand side. The odd powers integrate to zero on \([-1,1]\):
Writing \(p(x)=a+b(1+x)\), solve the Gram equations. Subtract the first equation from the second to find \(b\), then use the first equation to find \(a\):
Thus \(p(x)=\frac{1}{3}\). Check the equations by substitution: \(2(\frac13)+2(0)=\frac23\), and \(2(\frac13)+\frac83(0)=\frac23\). The residual \(r(x)=x^2-\frac13\) is orthogonal to both basis polynomials:
The theorem shows that this is the unique closest polynomial in \(S\). As an additional check, its squared error is \(\int_{-1}^1(x^2-\frac13)^2\,dx=\frac25-\frac49+\frac29=\frac{8}{45}\).
Worked Example: A Coefficient Inner Product
On \(P_2\), define the inner product by taking the dot product of coefficient lists: if \(g(x)=a_0+a_1x+a_2x^2\) and \(h(x)=b_0+b_1x+b_2x^2\), set \(\langle g,h\rangle=a_0b_0+a_1b_1+a_2b_2\). Project \(f(x)=2+x+x^2\) onto \(S=\operatorname{span}\{1,1+x\}\). This is an inner product because it is the usual dot product on the three coefficients. The basis is nonorthogonal: \(\langle 1,1+x\rangle=1\).
The Gram entries are \(\langle1,1\rangle=1\), \(\langle1,1+x\rangle=1\), and \(\langle1+x,1+x\rangle=2\). The right-hand entries are \(\langle f,1\rangle=2\) and \(\langle f,1+x\rangle=3\). Hence, for \(p=a+b(1+x)\),
The equations are \(a+b=2\) and \(a+2b=3\). Subtracting the first from the second gives \(b=1\), and then \(a=1\). Therefore \(p(x)=1+(1+x)=2+x\). Substituting the coefficients back gives \(1+1=2\) and \(1+2=3\), as required.
The residual is \(f-p=x^2\). Its coefficient list is \((0,0,1)\), while those of the basis polynomials are \((1,0,0)\) and \((1,1,0)\). Their dot products with the residual are both zero. This confirms residual orthogonality, and the theorem guarantees that \(2+x\) is the unique closest element of \(S\) for this inner product.
Worked Example: Projection from Values at Three Points
On \(P_2\), use \(\langle g,h\rangle=g(0)h(0)+g(1)h(1)+g(2)h(2)\). This is an inner product: if \(\langle g,g\rangle=0\), then \(g\) vanishes at \(0,1,2\); a polynomial of degree at most \(2\) with three distinct roots must be zero. Project \(f(x)=x^2\) onto \(S=\operatorname{span}\{1,1+x\}\). These basis polynomials are independent as in the first example and are nonorthogonal, since \(\langle1,1+x\rangle=1+2+3=6\).
At \(0,1,2\), the values of \(1\) are \(1,1,1\), those of \(1+x\) are \(1,2,3\), and those of \(f\) are \(0,1,4\). Thus
For \(p=a+b(1+x)\), reduce the augmented matrix to solve the equations:
The second row gives \(2b=4\), so \(b=2\). The first row then gives \(3a+12=5\), hence \(a=-\frac73\). Therefore \(p(x)=-\frac73+2(1+x)=-\frac13+2x\). Substitution checks both equations: \(3(-\frac73)+6(2)=5\), and \(6(-\frac73)+14(2)=14\).
The residual values at \(0,1,2\) are \(\frac13,-\frac23,\frac13\). Their inner products with the basis are \(\frac13-\frac23+\frac13=0\) and \(\frac13-\frac43+1=0\). The squared error is \((\frac13)^2+(-\frac23)^2+(\frac13)^2=\frac23\). Residual orthogonality and the theorem confirm that this is the unique closest polynomial in \(S\).
A Common Mistake: Treating the Basis as Orthogonal
For an orthogonal basis, dividing \(\langle f,\phi_i\rangle\) by \(\langle\phi_i,\phi_i\rangle\) gives the coefficient in the \(\phi_i\) direction. That separate calculation does not generally work for a nonorthogonal basis: the interactions appear in the off-diagonal Gram entries.
For instance, in the first example, dividing each right-hand entry by the corresponding diagonal Gram entry would give \(a=(\frac23)/2=\frac13\) and \(b=(\frac23)/(\frac83)=\frac14\). But this fails even the first Gram equation: \(2(\frac13)+2(\frac14)=\frac76\ne\frac23\). It also fails residual orthogonality, since the resulting polynomial is \(\frac{7}{12}+\frac{x}{4}\), and the inner product of its residual with \(1\) is \(\frac23-\frac76=-\frac12\), not zero. The correction is to solve the full system, including its off-diagonal entries.
The method is useful whenever a polynomial subspace is specified by a convenient basis that is not orthogonal. The inner product may come from integration, coefficient lists, or evaluations at selected points, provided it is positive definite on the space being used. In each case, the same checks apply: solve the Gram equations, verify that the residual is orthogonal to every basis element, and use the squared-norm decomposition to justify that the answer is closest.
Check Your Understanding
For each question, use the stated inner product and check any proposed projection by testing its residual against the basis.
- For \(\langle g,h\rangle=\int_0^1 g(x)h(x)\,dx\), find the projection of \(x^2\) onto \(\operatorname{span}\{1,x\}\) by forming the Gram equations.
- In the coefficient inner product on \(P_2\), compute the Gram matrix for the ordered basis \((1,1+x)\). Is it diagonal?
- Explain why a Gram matrix built from a basis in a real inner product space has trivial null space.
- For the discrete inner product using values at \(0,1,2\), verify that the residual in the third worked example has squared norm \(\frac23\).
- Why can separate ratios of right-hand entries to diagonal Gram entries fail to give the projection coefficients?
