Gram-Schmidt for Two Vectors

The Two-Vector Construction

Gram–Schmidt begins with vectors that may point in overlapping directions and replaces them with orthogonal vectors that retain the same span. For two independent vectors, the calculation has a direct geometric interpretation: keep the direction of the first vector, then remove from the second vector exactly the part pointing along that first direction. The result is a nonzero residual perpendicular to the first vector. Normalizing the two perpendicular vectors produces an orthonormal pair.

We work over \(\mathbb{R}\) in a real inner product space. For vectors \(\mathbf{x},\mathbf{y}\), the inner product is written \(\langle\mathbf{x},\mathbf{y}\rangle\), and the norm is \(\|\mathbf{x}\|=\sqrt{\langle\mathbf{x},\mathbf{x}\rangle}\). In \(\mathbb{R}^n\) with the standard inner product, \(\langle\mathbf{x},\mathbf{y}\rangle=\mathbf{x}^{\top}\mathbf{y}\). A pair is orthonormal if each vector has norm \(1\) and their inner product is \(0\).

The construction requires \(\mathbf{v}_1\ne\mathbf{0}\), so that \(\mathbf{u}_1\) is defined. Linear independence guarantees this. It also guarantees that \(\mathbf{w}_2\ne\mathbf{0}\), so the final normalization is defined. The order matters: first subtract the component along \(\mathbf{u}_1\), then normalize what remains. If the second vector is normalized before its component is removed, that does not replace the subtraction step.

The scalar \(\langle\mathbf{v}_2,\mathbf{u}_1\rangle\) is the coefficient of the component along the unit vector \(\mathbf{u}_1\). In particular, the vector being subtracted is \(\langle\mathbf{v}_2,\mathbf{u}_1\rangle\mathbf{u}_1\), not just the scalar coefficient. Since \(\mathbf{u}_1=\mathbf{v}_1/\|\mathbf{v}_1\|\), the same residual can be calculated directly from the original first vector:

This version can be convenient when the original vectors are given in coordinates. Its denominator is the squared norm of \(\mathbf{v}_1\), not the norm itself.

Why the Construction Works for Two Independent Vectors

Proof. Independence implies \(\mathbf{v}_1\ne\mathbf{0}\), so \(\|\mathbf{v}_1\|>0\) and \(\mathbf{u}_1\) is defined. Its norm is \(\|\mathbf{u}_1\|=\|\mathbf{v}_1\|/\|\mathbf{v}_1\|=1\). By the definition of the residual and the inner-product rules, \[ \langle\mathbf{w}_2,\mathbf{u}_1\rangle =\langle\mathbf{v}_2,\mathbf{u}_1\rangle -\langle\mathbf{v}_2,\mathbf{u}_1\rangle\langle\mathbf{u}_1,\mathbf{u}_1\rangle =0, \] because \(\langle\mathbf{u}_1,\mathbf{u}_1\rangle=1\). Thus the residual is orthogonal to \(\mathbf{u}_1\). If \(\mathbf{w}_2=\mathbf{0}\), then \(\mathbf{v}_2=\langle\mathbf{v}_2,\mathbf{u}_1\rangle\mathbf{u}_1\), so \(\mathbf{v}_2\) is a scalar multiple of \(\mathbf{u}_1\), and hence of \(\mathbf{v}_1\). That contradicts independence. Therefore \(\mathbf{w}_2\ne\mathbf{0}\), its norm is positive, and \(\mathbf{u}_2\) is defined. Scaling a nonzero vector to norm \(1\) gives \(\|\mathbf{u}_2\|=1\), and \(\langle\mathbf{u}_2,\mathbf{u}_1\rangle=\langle\mathbf{w}_2,\mathbf{u}_1\rangle/\|\mathbf{w}_2\|=0\). The output is orthonormal. For the span, \(\mathbf{u}_1=\mathbf{v}_1/\|\mathbf{v}_1\|\), so \(\mathbf{u}_1\) is in the span of the inputs and \(\mathbf{v}_1=\|\mathbf{v}_1\|\mathbf{u}_1\). Also, \(\mathbf{w}_2=\mathbf{v}_2-\langle\mathbf{v}_2,\mathbf{u}_1\rangle\mathbf{u}_1\) is in the span of the inputs, and \(\mathbf{u}_2=\mathbf{w}_2/\|\mathbf{w}_2\|\) is too. Thus the output span is contained in the input span. Conversely, \[ \mathbf{v}_2=\langle\mathbf{v}_2,\mathbf{u}_1\rangle\mathbf{u}_1 +\|\mathbf{w}_2\|\mathbf{u}_2. \] Both inputs are therefore in the output span, proving the reverse containment. \(\square\)

The proof gives practical checks for a calculation: the first output must have norm \(1\); the residual must have zero inner product with the first output; and the residual must not be zero when the inputs are independent. After normalizing the residual, the second output must also have norm \(1\). These checks catch arithmetic errors before the result is used.

Worked Calculations

Common Mistake: Omitting the Denominator

A frequent error is to use the original vector \(\mathbf{v}_1\) in the projection formula as if it were already a unit vector. For example, let \(\mathbf{v}_1=(1,1)^{\top}\) and \(\mathbf{v}_2=(1,0)^{\top}\). A wrong calculation subtracts \(\langle\mathbf{v}_2,\mathbf{v}_1\rangle\mathbf{v}_1=(1,1)^{\top}\), producing the residual \((0,-1)^{\top}\). But this residual is not orthogonal to \(\mathbf{v}_1\), since \(\langle(0,-1),(1,1)\rangle=-1\ne0\). The mistake is that \(\mathbf{v}_1\) has squared norm \(2\), not \(1\).

The corrected calculation normalizes first, or equivalently divides by the squared norm in the original-vector formula. Here the coefficient is \(\langle\mathbf{v}_2,\mathbf{v}_1\rangle/\langle\mathbf{v}_1,\mathbf{v}_1\rangle=1/2\), so \[ \mathbf{w}_2=(1,0)^{\top}-\frac12(1,1)^{\top} =(1/2,-1/2)^{\top}. \] Now \(\langle\mathbf{w}_2,\mathbf{v}_1\rangle=1/2-1/2=0\), as required. Its norm is \(1/\sqrt{2}\), and its normalized version is \((1/\sqrt{2},-1/\sqrt{2})^{\top}\).

Reading the Output

The calculation separates the two original vectors into directions that do not overlap. The first output \(\mathbf{u}_1\) points along \(\mathbf{v}_1\). The second original vector splits as \[ \mathbf{v}_2=\langle\mathbf{v}_2,\mathbf{u}_1\rangle\mathbf{u}_1+\mathbf{w}_2. \] The first term lies along \(\mathbf{u}_1\); the second is perpendicular to it. Normalization changes lengths but not directions, so the final orthonormal pair spans exactly the same subspace as the inputs.

For two independent vectors, the calculation should be done in this order:

1. Check that the first vector is nonzero and divide it by its norm.
2. Take the inner product of the second vector with the first unit vector.
3. Multiply that scalar by the first unit vector and subtract the resulting vector from the second input.
4. Check that the residual is nonzero and perpendicular to the first unit vector.
5. Divide the residual by its norm, then verify both output norms and their inner product.

The residual is not an arbitrary correction: it is the part of the second vector left after the first direction has been removed. Keeping track of the scalar coefficient and the vector being subtracted prevents a common mix-up. A complete answer should display both, since the coefficient alone is not the projected component.