From Orthogonal Residuals to Unit Vectors
In Gram–Schmidt, each new input has its components along the earlier orthogonal directions removed. The resulting residual is orthogonal to those earlier directions, but it need not have length \(1\). Dividing the residual by its norm changes its length to \(1\) while keeping it in the same direction. The result is an orthonormal list: its vectors have unit norm and are pairwise orthogonal.
Let \(V\) be a finite-dimensional inner product space over \(\mathbb{R}\) or \(\mathbb{C}\). We use the convention that a complex inner product is linear in its first slot and conjugate-linear in its second. The norm of \(\mathbf{v}\in V\) is \(\|\mathbf{v}\|=\sqrt{\langle\mathbf{v},\mathbf{v}\rangle}\). Two vectors are orthogonal when their inner product is zero. A list is orthonormal if each vector has norm \(1\) and distinct vectors in the list are orthogonal.
The displayed formula uses unit earlier outputs \(\mathbf{q}_j\). More generally, to remove the component along a nonzero orthogonal vector \(\mathbf{u}_j\), the coefficient is \(\langle\mathbf{v}_k,\mathbf{u}_j\rangle/\langle\mathbf{u}_j,\mathbf{u}_j\rangle\). When \(\mathbf{u}_j\) is replaced by \(\mathbf{q}_j\), its squared norm is \(1\), giving the shorter coefficient in the definition.
Normalization is defined only for a nonzero residual, since the zero vector has norm \(0\) and division by \(0\) is not defined. In particular, linearly independent inputs have nonzero residuals at every stage: if \(\mathbf{u}_k=\mathbf{0}\), then the formula expresses \(\mathbf{v}_k\) as a combination of earlier outputs, which span the earlier inputs. That would make the inputs dependent. This is the nonzero-residual property established in the earlier material on independent inputs and Gram–Schmidt.
Why Normalization Works
Proof. We argue successively through the list. At step \(k\), suppose the earlier outputs are orthonormal and their first \(k-1\) vectors span the first \(k-1\) inputs; for \(k=1\), there are no earlier vectors and both earlier spans are \(\{\mathbf{0}\}\). If \(\mathbf{u}_k=\mathbf{0}\), its defining equation gives \(\mathbf{v}_k=\sum_{j=1}^{k-1}\langle\mathbf{v}_k,\mathbf{q}_j\rangle\mathbf{q}_j\). Thus \(\mathbf{v}_k\) lies in the span of the earlier outputs, which by the induction hypothesis is the span of the earlier inputs. This contradicts independence, so \(\mathbf{u}_k\ne\mathbf{0}\) and \(\mathbf{q}_k\) is defined. Since \(\|\mathbf{u}_k\|\) is a positive real number, $$ \langle\mathbf{q}_k,\mathbf{q}_k\rangle =\frac{\langle\mathbf{u}_k,\mathbf{u}_k\rangle}{\|\mathbf{u}_k\|^2} =1. $$ For each \(i<k\), orthonormality of the earlier outputs gives $$ \langle\mathbf{u}_k,\mathbf{q}_i\rangle =\langle\mathbf{v}_k,\mathbf{q}_i\rangle -\sum_{j=1}^{k-1}\langle\mathbf{v}_k,\mathbf{q}_j\rangle \langle\mathbf{q}_j,\mathbf{q}_i\rangle =\langle\mathbf{v}_k,\mathbf{q}_i\rangle-\langle\mathbf{v}_k,\mathbf{q}_i\rangle =0. $$ Therefore \(\langle\mathbf{q}_k,\mathbf{q}_i\rangle=0\), so the new output is orthogonal to every earlier one. Finally, the residual formula shows that \(\mathbf{u}_k\), and hence \(\mathbf{q}_k\), lies in the span of \(\mathbf{v}_1,\dots,\mathbf{v}_k\), using the induction hypothesis for earlier outputs. Conversely, $$ \mathbf{v}_k=\sum_{j=1}^{k-1}\langle\mathbf{v}_k,\mathbf{q}_j\rangle\mathbf{q}_j+\|\mathbf{u}_k\|\mathbf{q}_k $$ lies in the span of \(\mathbf{q}_1,\dots,\mathbf{q}_k\). The earlier spans already agree, so adjoining the \(k\)-th vector to each gives equal spans at step \(k\). This proves the claims successively for all \(k\). \(\square\)
The theorem separates two effects of the procedure. Subtracting the earlier components makes the new residual orthogonal to earlier outputs. Dividing by its positive norm changes only its length: it does not change the direction or the span contributed at that stage. The following calculations show both features explicitly.
Worked Normalizations
Worked Example: Three Inputs in \(\mathbb{R}^3\)
Apply Gram–Schmidt with the usual dot product to \(\mathbf{v}_1=(1,1,0)\), \(\mathbf{v}_2=(1,0,1)\), and \(\mathbf{v}_3=(0,1,1)\), and normalize every residual.
First, \(\mathbf{u}_1=\mathbf{v}_1=(1,1,0)\), with \(\|\mathbf{u}_1\|=\sqrt{1+1}=\sqrt{2}\). Thus \(\mathbf{q}_1=(1,1,0)/\sqrt{2}\). For the second input, the coefficient along \(\mathbf{q}_1\) is \(\langle\mathbf{v}_2,\mathbf{q}_1\rangle=1/\sqrt{2}\), so
Its squared norm is \(1/4+1/4+1=3/2\), hence \(\|\mathbf{u}_2\|=\sqrt{6}/2\) and \(\mathbf{q}_2=(1,-1,2)/\sqrt{6}\). For the third input, the two coefficients are \(\langle\mathbf{v}_3,\mathbf{q}_1\rangle=1/\sqrt{2}\) and \(\langle\mathbf{v}_3,\mathbf{q}_2\rangle=1/\sqrt{6}\). Subtracting both components gives
Therefore \(\|\mathbf{u}_3\|=\sqrt{12}/3=2/\sqrt{3}\), and \(\mathbf{q}_3=(-1,1,1)/\sqrt{3}\). Check the normalization: the squared lengths of \(\mathbf{q}_1,\mathbf{q}_2,\mathbf{q}_3\) are respectively \(2/2=1\), \(6/6=1\), and \(3/3=1\). Their pairwise dot products have numerators \(1-1+0=0\), \(-1+1+0=0\), and \(-1-1+2=0\), so the outputs are orthonormal.
The successive spans are unchanged as well. In fact, \(\mathbf{v}_1=\sqrt{2}\mathbf{q}_1\), \(\mathbf{v}_2=(1/\sqrt{2})\mathbf{q}_1+(\sqrt{6}/2)\mathbf{q}_2\), and \(\mathbf{v}_3=(1/\sqrt{2})\mathbf{q}_1+(1/\sqrt{6})\mathbf{q}_2+(2/\sqrt{3})\mathbf{q}_3\). These expressions verify that each input lies in the span of the outputs up to its stage.
Worked Example: A Weighted Inner Product
On \(\mathbb{R}^2\), use \(\langle\mathbf{x},\mathbf{y}\rangle=x_1y_1+4x_2y_2\). Apply Gram–Schmidt to \(\mathbf{v}_1=(1,1)\) and \(\mathbf{v}_2=(2,0)\). This inner product weights the second coordinate, so the norm must be calculated using that weight.
The first residual is \(\mathbf{u}_1=(1,1)\), and \(\|\mathbf{u}_1\|^2=1+4=5\). Thus \(\mathbf{q}_1=(1,1)/\sqrt{5}\). The coefficient for the second input is \(\langle\mathbf{v}_2,\mathbf{q}_1\rangle=2/\sqrt{5}\). Therefore
Its squared weighted norm is $$ \|\mathbf{u}_2\|^2 =\left(\frac85\right)^2+4\left(-\frac25\right)^2 =\frac{64}{25}+\frac{16}{25} =\frac{16}{5}. $$ So \(\|\mathbf{u}_2\|=4/\sqrt{5}\), and division by this norm gives \(\mathbf{q}_2=(2/\sqrt{5},-1/(2\sqrt{5}))\).
Check its weighted squared norm: \(4/5+4(1/20)=4/5+1/5=1\). Also, \(\langle\mathbf{q}_1,\mathbf{q}_2\rangle=2/5+4(-1/10)=0\). Finally, \(\mathbf{v}_1=\sqrt{5}\mathbf{q}_1\), and \(\mathbf{v}_2=(2/\sqrt{5})\mathbf{q}_1+(4/\sqrt{5})\mathbf{q}_2\), confirming both successive span equalities.
Worked Example: Complex Inputs
In \(\mathbb{C}^2\), use \(\langle\mathbf{x},\mathbf{y}\rangle=x_1\overline{y_1}+x_2\overline{y_2}\), linear in the first slot. Apply the process to \(\mathbf{v}_1=(1,i)\) and \(\mathbf{v}_2=(1,0)\).
The first norm is \(\sqrt{1+|i|^2}=\sqrt{2}\), so \(\mathbf{q}_1=(1,i)/\sqrt{2}\). The coefficient for the second input is \(\langle\mathbf{v}_2,\mathbf{q}_1\rangle=1/\sqrt{2}\). Consequently, $$ \mathbf{u}_2=(1,0)-\frac{1}{\sqrt{2}}\frac{(1,i)}{\sqrt{2}} =\left(\frac12,-\frac{i}{2}\right). $$ Its squared norm is \(1/4+|-i/2|^2=1/2\), so \(\|\mathbf{u}_2\|=1/\sqrt{2}\) and \(\mathbf{q}_2=(1,-i)/\sqrt{2}\).
Both vectors have norm \(1\). To check orthogonality with the stated convention, compute \(\langle\mathbf{q}_2,\mathbf{q}_1\rangle=1/2+(-i)(-i)/2=0\). Also, \(\mathbf{v}_2=(1/\sqrt{2})\mathbf{q}_1+(1/\sqrt{2})\mathbf{q}_2\), so normalization has not changed the span of the two inputs.
Common Mistake: Use the Norm, Not Its Square
A frequent error is to compute a residual's squared norm correctly and then report that number as its norm. For the weighted residual above, one incorrect calculation would be to say \(\|\mathbf{u}_2\|=16/5\) after finding \(\|\mathbf{u}_2\|^2=16/5\). But the norm is the nonnegative square root: \(\|\mathbf{u}_2\|=\sqrt{16/5}=4/\sqrt{5}\). Dividing by \(16/5\) instead would not produce a unit vector. Indeed, the resulting vector would have norm \((4/\sqrt{5})/(16/5)=\sqrt{5}/4\), not \(1\). Always take the square root first, then divide the residual by that norm.
In a weighted inner product, another tempting error is to use the ordinary Euclidean formula. Here that would give the wrong squared norm \((8/5)^2+(-2/5)^2=68/25\), because the second coordinate's weight \(4\) has been omitted. Use the inner product specified for the space, and check the final vector's norm with that same inner product.
What the Normalized List Gives You
When the inputs are independent, normalization completes Gram–Schmidt by turning the orthogonal directions into unit vectors while retaining each successive span. In particular, every input up to stage \(k\) can be expressed using the first \(k\) outputs, and every output up to stage \(k\) can be expressed using the first \(k\) inputs. This is why normalization does not discard information: it changes the scale of a nonzero residual, not the set of combinations available from the list.
Once the outputs are orthonormal, their coefficients are especially simple. If \(\mathbf{x}\) is in their span, then its coefficient along \(\mathbf{q}_j\) is \(\langle\mathbf{x},\mathbf{q}_j\rangle\); this follows by taking the inner product of its expansion with \(\mathbf{q}_j\) and using orthonormality. Thus the normalized Gram–Schmidt list is useful both as a compact description of the same span and as a convenient system for expressing vectors in it.
Check Your Understanding
For each question, state the inner product being used and check any proposed normalized vector by its norm.
- If a nonzero residual \(\mathbf{u}\) has norm \(5\), what vector does Gram–Schmidt output at that step, and what is its norm?
- Explain why dividing a nonzero residual by its norm preserves its span.
- For the weighted inner product \(\langle\mathbf{x},\mathbf{y}\rangle=x_1y_1+4x_2y_2\), compute the squared norm of \((1,-1/2)\).
- Using the complex inner product linear in the first slot, calculate \(\langle(1,-i),(1,i)\rangle\) in \(\mathbb{C}^2\).
- State the two properties that distinguish an orthonormal list from an orthogonal list whose vectors may not have unit norm.
