A Gram-Schmidt Workshop with Exact and Polynomial Examples

Gram–Schmidt as a Complete Calculation

The Gram–Schmidt process takes an ordered list of vectors and produces an orthogonal list, or an orthonormal list if each nonzero output is normalized. The calculations have two things to keep track of: the projection coefficients at each step, and whether a residual is zero. These examples work through both details for vectors in \(\mathbb{R}^4\) and for polynomials. In each case, direct checks will confirm orthonormality and show that the process preserves the span.

All spaces and inner products in this tutorial are real. In \(\mathbb{R}^n\), use the dot product \(\langle\mathbf{u},\mathbf{v}\rangle=\mathbf{u}^{\top}\mathbf{v}\) and norm \(\|\mathbf{u}\|=\sqrt{\langle\mathbf{u},\mathbf{u}\rangle}\). For a nonzero vector \(\mathbf{q}\) of norm \(1\), the projection of \(\mathbf{v}\) onto \(\mathbf{q}\) is \(\langle\mathbf{v},\mathbf{q}\rangle\mathbf{q}\). Subtracting this projection removes the component in the \(\mathbf{q}\) direction.

The index set in this definition is important: projections are taken only onto outputs that were actually produced. A zero residual has no normalized vector, so it cannot supply a direction for a later projection. When every residual is nonzero, the formula reduces to the familiar recursion that projects away from all earlier outputs.

Why the Process Produces Orthogonal Outputs and Preserves Span

Proof. We proceed in order through the inputs. Before the first step there are no outputs, so the previous outputs are an orthonormal list and their span equals the span of the empty list, namely \(\{\mathbf{0}\}\). Suppose the outputs through step \(k-1\) are orthonormal and span the first \(k-1\) inputs. For each earlier output \(\mathbf{q}_i\), the residual at step \(k\) satisfies \[ \langle\mathbf{u}_k,\mathbf{q}_i\rangle =\langle\mathbf{v}_k,\mathbf{q}_i\rangle- \sum_{j\in J_{k-1}}\langle\mathbf{v}_k,\mathbf{q}_j\rangle\langle\mathbf{q}_j,\mathbf{q}_i\rangle =0, \] because orthonormality gives \(\langle\mathbf{q}_j,\mathbf{q}_i\rangle=1\) when \(j=i\) and \(0\) otherwise. If \(\mathbf{u}_k\ne\mathbf{0}\), then \(\mathbf{q}_k=\mathbf{u}_k/\|\mathbf{u}_k\|\) has norm \(1\) and is orthogonal to every earlier output. If \(\mathbf{u}_k=\mathbf{0}\), no output is added, so the earlier outputs remain orthonormal.

For the span statement, each earlier \(\mathbf{q}_j\) belongs to the span of the first \(k-1\) inputs by the induction hypothesis. Thus \(\mathbf{u}_k\), which is \(\mathbf{v}_k\) minus a combination of earlier outputs, belongs to the span of the first \(k\) inputs. Any new \(\mathbf{q}_k\) is a scalar multiple of \(\mathbf{u}_k\), so all outputs through step \(k\) belong to that input span. Conversely, rearranging the residual formula gives \(\mathbf{v}_k=\mathbf{u}_k+\sum_{j\in J_{k-1}}\langle\mathbf{v}_k,\mathbf{q}_j\rangle\mathbf{q}_j\). If the residual is zero, this expression still holds, with \(\mathbf{u}_k=\mathbf{0}\); if it is nonzero, it is a scalar multiple of the new output. In either case \(\mathbf{v}_k\) lies in the span of the outputs through step \(k\), and the earlier inputs already lie there by induction. The two spans are equal. \(\square\)

One useful consequence is that an independent input list has no zero residuals. Indeed, if \(\mathbf{u}_k=\mathbf{0}\), the residual formula expresses \(\mathbf{v}_k\) as a combination of earlier outputs. Those outputs lie in the span of earlier inputs, so \(\mathbf{v}_k\) is a combination of earlier inputs. That contradicts independence. This consequence does not say that every list produces a new output; a dependent list can produce a zero residual and then continue.

Worked Example in \(\mathbb{R}^4\)

A Zero Residual Does Not Stop the Process

A Polynomial Example with an Integral Inner Product

For polynomials on \([-1,1]\), use the real inner product \(\langle f,g\rangle=\int_{-1}^{1}f(x)g(x)\,dx\) and norm \(\|f\|=\sqrt{\langle f,f\rangle}\). Consider the ordered list \(1,x,x^2\) in the space of real polynomials of degree at most \(2\). The projection formula is unchanged; only the inner products are now integrals.

Common Mistake: Use the Unit Vector in the Projection

The projection onto a unit vector \(\mathbf{q}\) is \(\langle\mathbf{v},\mathbf{q}\rangle\mathbf{q}\), not \(\langle\mathbf{v},\mathbf{q}\rangle\mathbf{v}_1\) when \(\mathbf{v}_1\) is the unnormalized earlier vector. In the first coordinate example, \(\langle\mathbf{v}_2,\mathbf{q}_1\rangle=1/\sqrt{2}\). An incorrect calculation would subtract \((1/\sqrt{2})\mathbf{v}_1\) and claim the residual is orthogonal to \(\mathbf{v}_1\). Its dot product with \(\mathbf{v}_1\) is \((1-1/\sqrt{2})+(-1/\sqrt{2})=1-\sqrt{2}\ne0\), so the claim fails. The correct projection is \((1/\sqrt{2})\mathbf{q}_1=\frac12\mathbf{v}_1\), giving \(\mathbf{u}_2=\begin{bmatrix}1/2\\-1/2\\1\\0\end{bmatrix}\), whose dot product with \(\mathbf{v}_1\) is \(1/2-1/2=0\).

Also distinguish an orthogonal list from an orthonormal one. Orthogonality requires distinct vectors to have inner product zero; orthonormality additionally requires every vector to have norm \(1\). Gram–Schmidt residuals are orthogonal, but they must be divided by their norms to obtain unit outputs. If a residual is zero, normalization is impossible, and that residual is omitted rather than treated as a new direction.