The Extension Problem
An orthonormal list contains vectors that are mutually perpendicular and each have length \(1\). Such a list is already linearly independent, but it need not span the whole space. The aim here is to enlarge a given orthonormal list until it becomes a basis, while preserving orthonormality at every step.
Throughout, let \(V\) be a finite-dimensional inner product space over \(\mathbb{R}\). A list \((\mathbf{q}_1,\dots,\mathbf{q}_k)\) is orthonormal if \(\langle \mathbf{q}_i,\mathbf{q}_j\rangle=0\) for \(i\ne j\) and \(\langle \mathbf{q}_i,\mathbf{q}_i\rangle=1\) for every \(i\). A basis is a linearly independent list that spans \(V\). Here, the span of a list is the set of all its linear combinations.
Choose any basis \((\mathbf{b}_1,\dots,\mathbf{b}_n)\) of \(V\). Starting with the given orthonormal list, examine the basis vectors in order. If a basis vector is already in the span of the current list, skip it. Otherwise, remove its components in the directions of the current orthonormal vectors, normalize what remains, and append that new unit vector. The skip test matters: a basis vector already in the current span has zero residual, and a zero vector cannot be normalized.
The sum is the component of \(\mathbf{b}\) in the span of the current orthonormal list. Subtracting it leaves a residual perpendicular to each current vector. Indeed, for any \(j\) between \(1\) and \(m\), linearity in the first slot and orthonormality give \(\langle\mathbf{r},\mathbf{q}_j\rangle=\langle\mathbf{b},\mathbf{q}_j\rangle-\sum_{i=1}^{m}\langle\mathbf{b},\mathbf{q}_i\rangle\langle\mathbf{q}_i,\mathbf{q}_j\rangle=\langle\mathbf{b},\mathbf{q}_j\rangle-\langle\mathbf{b},\mathbf{q}_j\rangle=0\).
Why the Procedure Produces a Basis
Proof. Suppose \(\sum_{i=1}^{m}c_i\mathbf{q}_i=\mathbf{0}\) for an orthonormal list and real scalars \(c_i\). Taking the inner product of both sides with \(\mathbf{q}_j\) gives \(\sum_{i=1}^{m}c_i\langle\mathbf{q}_i,\mathbf{q}_j\rangle=0\). Every term is zero except the \(i=j\) term, which is \(c_j\), so \(c_j=0\). This holds for every \(j\), and therefore the only dependence relation is the trivial one. \(\square
Proof. We show that each step preserves orthonormality and that the final list spans \(V\). Initially the list is orthonormal by assumption. Suppose the current list \((\mathbf{q}_1,\dots,\mathbf{q}_m)\) is orthonormal. For the next basis vector \(\mathbf{b}_j\), form its residual \(\mathbf{r}=\mathbf{b}_j-\sum_{i=1}^{m}\langle\mathbf{b}_j,\mathbf{q}_i\rangle\mathbf{q}_i\). The inner-product calculation above shows that \(\mathbf{r}\) is perpendicular to every current vector. If \(\mathbf{r}=\mathbf{0}\), the list is unchanged. If \(\mathbf{r}\ne\mathbf{0}\), then \(\mathbf{q}_{m+1}=\mathbf{r}/\|\mathbf{r}\|\) has norm \(1\), because \(\|\mathbf{q}_{m+1}\|=\|\mathbf{r}\|/\|\mathbf{r}\|=1\), and remains perpendicular to every earlier vector. Thus the list stays orthonormal at every step.
At each step, the span of the list after processing \(\mathbf{b}_j\) equals the span of the list before processing it together with \(\mathbf{b}_j\). If \(\mathbf{b}_j\) is already in the current span, this is true because adding it changes nothing. Otherwise, \(\mathbf{r}=\mathbf{b}_j-\sum_{i=1}^{m}\langle\mathbf{b}_j,\mathbf{q}_i\rangle\mathbf{q}_i\) belongs to the span of the current list together with \(\mathbf{b}_j\), and \(\mathbf{b}_j=\mathbf{r}+\sum_{i=1}^{m}\langle\mathbf{b}_j,\mathbf{q}_i\rangle\mathbf{q}_i\) belongs to the span after \(\mathbf{q}_{m+1}\) is appended. Hence the two spans are equal in this case as well.
After all basis vectors have been processed, the resulting list has the same span as the original list together with \(\mathbf{b}_1,\dots,\mathbf{b}_n\). The basis vectors span \(V\), so the resulting list spans \(V\). It is orthonormal, hence linearly independent by the lemma. A spanning, linearly independent list is a basis, as required. \(\square
Working Through the Extension
Worked Example: Extending a List in \(\mathbb{R}^3\)
Start with \(\mathbf{q}_1=(1,1,0)/\sqrt{2}\) and \(\mathbf{q}_2=(0,0,1)\). Extend this orthonormal list using the standard basis \((\mathbf{e}_1,\mathbf{e}_2,\mathbf{e}_3)\), in that order.
The starting list is orthonormal: both vectors have norm \(1\), and their dot product is \(0\). For \(\mathbf{e}_1=(1,0,0)\), the inner products are \(\langle\mathbf{e}_1,\mathbf{q}_1\rangle=1/\sqrt{2}\) and \(\langle\mathbf{e}_1,\mathbf{q}_2\rangle=0\). Thus
Its squared norm is \(1/4+1/4=1/2\), so \(\|\mathbf{r}_1\|=1/\sqrt{2}\). The appended vector is \(\mathbf{q}_3=\mathbf{r}_1/\|\mathbf{r}_1\|=(1/\sqrt{2},-1/\sqrt{2},0)\).
For \(\mathbf{e}_2=(0,1,0)\), the inner products with \(\mathbf{q}_1,\mathbf{q}_2,\mathbf{q}_3\) are \(1/\sqrt{2},0,-1/\sqrt{2}\), respectively. Therefore
So \(\mathbf{e}_2\) is skipped. The vector \(\mathbf{e}_3\) is already \(\mathbf{q}_2\), so its residual is zero as well. The final list is \((\mathbf{q}_1,\mathbf{q}_2,\mathbf{q}_3)\).
Check: Each vector has norm \(1\), and all pairwise dot products are zero. Also, \(\mathbf{e}_1=(\mathbf{q}_1+\mathbf{q}_3)/\sqrt{2}\), \(\mathbf{e}_2=(\mathbf{q}_1-\mathbf{q}_3)/\sqrt{2}\), and \(\mathbf{e}_3=\mathbf{q}_2\). Thus the list spans the standard basis and hence \(\mathbb{R}^3\).
Worked Example: A Nonstandard Basis in \(\mathbb{R}^4\)
Begin with \((\mathbf{q}_1,\mathbf{q}_2)=(\mathbf{e}_1,\mathbf{e}_2)\), and use the basis \(\mathbf{b}_1=\mathbf{e}_1+\mathbf{e}_3\), \(\mathbf{b}_2=\mathbf{e}_2+\mathbf{e}_4\), \(\mathbf{b}_3=\mathbf{e}_3\), \(\mathbf{b}_4=\mathbf{e}_4\). First, this is a basis: a combination with coefficients \(c_1,c_2,c_3,c_4\) is \((c_1,c_2,c_1+c_3,c_2+c_4)\). Given any \((y_1,y_2,y_3,y_4)\), the unique coefficients producing it are \(c_1=y_1\), \(c_2=y_2\), \(c_3=y_3-y_1\), \(c_4=y_4-y_2\).
For \(\mathbf{b}_1\), its projections onto \(\mathbf{e}_1,\mathbf{e}_2\) are \(\mathbf{e}_1,\mathbf{0}\), so \(\mathbf{r}_1=\mathbf{b}_1-\mathbf{e}_1=\mathbf{e}_3\). This residual has norm \(1\), and the new vector is \(\mathbf{q}_3=\mathbf{e}_3\). Next, the projections of \(\mathbf{b}_2\) onto \(\mathbf{e}_1,\mathbf{e}_2,\mathbf{e}_3\) are \(\mathbf{0},\mathbf{e}_2,\mathbf{0}\). Hence \(\mathbf{r}_2=\mathbf{b}_2-\mathbf{e}_2=\mathbf{e}_4\), which has norm \(1\), so \(\mathbf{q}_4=\mathbf{e}_4\).
Now \(\mathbf{b}_3=\mathbf{e}_3\) and \(\mathbf{b}_4=\mathbf{e}_4\) are already in the current span; both are skipped. The resulting list is \((\mathbf{e}_1,\mathbf{e}_2,\mathbf{e}_3,\mathbf{e}_4)\).
Check: The output is orthonormal because the standard coordinate vectors have dot product \(0\) when distinct and norm \(1\). It spans \(\mathbb{R}^4\), since every \((y_1,y_2,y_3,y_4)\) equals \(y_1\mathbf{e}_1+y_2\mathbf{e}_2+y_3\mathbf{e}_3+y_4\mathbf{e}_4\).
An Extension in a Polynomial Space
Worked Example: Extending a List in \(P_2\)
Let \(P_2\) be the real polynomials of degree at most \(2\), with inner product \(\langle f,g\rangle=\int_{-1}^{1}f(x)g(x)\,dx\). Begin with the one-vector orthonormal list \(q_1(x)=1/\sqrt{2}\), and process the basis \((1,x,x^2)\). This is an inner product: for a nonzero polynomial \(f\), there is an interval on which \(f(x)^2\) is positive, so \(\int_{-1}^{1}f(x)^2\,dx>0\). Also, \(\langle q_1,q_1\rangle=\int_{-1}^{1}1/2\,dx=1\).
The first basis vector, \(1\), is in the span of \(q_1\), so it is skipped. For \(x\), the projection coefficient is \(\langle x,q_1\rangle=(1/\sqrt{2})\int_{-1}^{1}x\,dx=0\). Thus the residual is \(r_2(x)=x\), with squared norm \(\int_{-1}^{1}x^2\,dx=2/3\), giving \(q_2(x)=\sqrt{3/2}\,x\).
For \(x^2\), the projection coefficients are \(\langle x^2,q_1\rangle=(1/\sqrt{2})(2/3)=\sqrt{2}/3\) and \(\langle x^2,q_2\rangle=\sqrt{3/2}\int_{-1}^{1}x^3\,dx=0\). The residual is \(r_3(x)=x^2-1/3\). Its squared norm is
Therefore \(q_3(x)=\sqrt{45/8}(x^2-1/3)\), and the extended list is \((q_1,q_2,q_3)\).
Check: The three squared norms are \(1\), \(1\), and \(1\), the last because \((45/8)(8/45)=1\). Also, \(\langle q_1,q_2\rangle=0\) and \(\langle q_2,q_3\rangle=0\) by oddness of the integrands; \(\langle q_1,q_3\rangle=0\) because \(\int_{-1}^{1}(x^2-1/3)\,dx=2/3-2/3=0\). The original basis is in the output span: \(1=\sqrt{2}q_1\), \(x=\sqrt{2/3}q_2\), and \(x^2=\sqrt{8/45}q_3+(\sqrt{2}/3)q_1\). Thus the output is an orthonormal basis of \(P_2\).
Common Pitfall: Normalizing a Zero Residual
Suppose the starting list in \(\mathbb{R}^2\) is \((\mathbf{e}_1)\), and the chosen basis is \((\mathbf{e}_1,\mathbf{e}_2)\). A wrong approach is to feed the concatenated list \((\mathbf{e}_1,\mathbf{e}_1,\mathbf{e}_2)\) directly into Gram–Schmidt and try to normalize every residual. For the second input, the calculation is \(\mathbf{e}_1-\langle\mathbf{e}_1,\mathbf{e}_1\rangle\mathbf{e}_1=\mathbf{e}_1-\mathbf{e}_1=\mathbf{0}\). Dividing this residual by its norm would require division by \(\|\mathbf{0}\|=0\), which is undefined.
The correction is to test whether each basis vector is already in the current span before appending it. Here \(\mathbf{b}_1=\mathbf{e}_1\) is skipped. For \(\mathbf{b}_2=\mathbf{e}_2\), the residual is \(\mathbf{e}_2-\langle\mathbf{e}_2,\mathbf{e}_1\rangle\mathbf{e}_1=\mathbf{e}_2\), which has norm \(1\). Appending it gives the orthonormal basis \((\mathbf{e}_1,\mathbf{e}_2)\). In general, a zero residual is a signal to skip the vector, not a vector to normalize.
The choice and order of the basis used in the procedure can affect which new vectors are produced. The required properties do not depend on those choices: the final list is orthonormal and spans the whole space. The useful strategy is therefore to keep the given orthonormal vectors unchanged, process a basis one vector at a time, and add a normalized residual only when it is nonzero.
Check Your Understanding
For each question, use the residual formula and verify any proposed output by checking orthogonality, norms, and spanning.
- Starting with \((\mathbf{e}_1)\) in \(\mathbb{R}^3\), process \((\mathbf{e}_1,\mathbf{e}_2,\mathbf{e}_3)\). Which vectors are skipped, and what is the final list?
- For an orthonormal list \((\mathbf{q}_1,\mathbf{q}_2)\), write the residual for a vector \(\mathbf{b}\). What does a zero residual tell you?
- In the polynomial example, why is the projection coefficient of \(x^2\) onto \(q_2(x)=\sqrt{3/2}\,x\) zero?
- Explain why a nonzero Gram–Schmidt residual is perpendicular to every vector already in the current orthonormal list.
- Why does spanning \(V\), together with orthonormality, suffice to conclude that the final list is a basis?
