Common Gram-Schmidt Calculation Errors

Why Gram–Schmidt Errors Can Be Hard to Spot

In a Gram–Schmidt calculation, each new vector is adjusted to remove its components in all the directions already constructed. A result can look plausible and still fail: one omitted subtraction may leave a component behind, an incorrect coefficient may not cancel the intended direction, or a normalization step may change a vector’s scale without changing the coefficient used with it. The reliable check is to test each residual against every earlier direction before moving on.

We work mainly in real inner product spaces. On $\mathbb{R}^n$, the inner product is the dot product $\mathbf{x}\cdot\mathbf{y}=\mathbf{x}^{\top}\mathbf{y}$. In a complex inner product space we use the convention that the inner product is linear in its first slot and conjugate-linear in its second. A vector $\mathbf{x}$ is orthogonal to $\mathbf{y}$ when $\langle\mathbf{x},\mathbf{y}\rangle=0$, and $\|\mathbf{x}\|^2=\langle\mathbf{x},\mathbf{x}\rangle$.

Suppose $\mathbf{v}_1,\dots,\mathbf{v}_p$ is an ordered list of input vectors. Gram–Schmidt constructs residuals $\mathbf{u}_1,\dots,\mathbf{u}_p$, using each earlier residual to remove the corresponding component from the next input. The formula below uses nonzero, mutually orthogonal earlier residuals. If a residual is zero, the process has detected dependence and cannot use that residual as a direction.

Gram–Schmidt recursion: Set $\mathbf{u}_1=\mathbf{v}_1$. For $k\ge 2$, provided $\mathbf{u}_1,\dots,\mathbf{u}_{k-1}$ are nonzero and mutually orthogonal, set $$ \mathbf{u}_k=\mathbf{v}_k-\sum_{j=1}^{k-1}\frac{\langle\mathbf{v}_k,\mathbf{u}_j\rangle}{\langle\mathbf{u}_j,\mathbf{u}_j\rangle}\mathbf{u}_j. $$ The coefficient uses the original input $\mathbf{v}_k$ and the squared norm $\langle\mathbf{u}_j,\mathbf{u}_j\rangle$ in the denominator. Over $\mathbb{R}^n$, replace inner products by dot products.

The Coefficient Is Fixed by Orthogonality

The coefficient is not a matter of choice. It is exactly the scalar that makes the adjusted vector orthogonal to the direction being removed.

Proposition: Let $\mathbf{u}\ne\mathbf{0}$ in an inner product space over $\mathbb{R}$ or $\mathbb{C}$, with the complex inner product linear in the first slot. For any $\mathbf{x}$, the vector $\mathbf{x}-c\mathbf{u}$ is orthogonal to $\mathbf{u}$ if and only if $$ c=\frac{\langle\mathbf{x},\mathbf{u}\rangle}{\langle\mathbf{u},\mathbf{u}\rangle}. $$

Proof. By linearity in the first slot, $\langle\mathbf{x}-c\mathbf{u},\mathbf{u}\rangle=\langle\mathbf{x},\mathbf{u}\rangle-c\langle\mathbf{u},\mathbf{u}\rangle$. Thus $\mathbf{x}-c\mathbf{u}$ is orthogonal to $\mathbf{u}$ exactly when this expression is zero. Since $\mathbf{u}\ne\mathbf{0}$, positive definiteness gives $\langle\mathbf{u},\mathbf{u}\rangle=\|\mathbf{u}\|^2>0$, so solving the equation gives the stated value of $c$. Substituting that value makes the expression zero, proving the converse as well. $\square$

This proposition supplies a direct diagnostic: if a proposed subtraction does not leave a vector orthogonal to the direction used, its coefficient or arithmetic is wrong. When several earlier residuals are mutually orthogonal, this test applies separately to each of them.

Worked Example: An Incorrect Coefficient

Take $\mathbf{u}_1=(1,1)$ and the next input $\mathbf{v}_2=(2,0)$ in $\mathbb{R}^2$. Find the correct residual, then test the tempting error of using the inner product as the coefficient without dividing by the squared norm.

The required coefficient is

\frac{\mathbf{v}_2\cdot\mathbf{u}_1}{\mathbf{u}_1\cdot\mathbf{u}_1} =\frac{2\cdot1+0\cdot1}{1^2+1^2} =\frac{2}{2}=1.

Therefore the correct residual is

\mathbf{u}_2=\mathbf{v}_2-\mathbf{u}_1 =(2,0)-(1,1)=(1,-1).

Check the defining condition: $\mathbf{u}_2\cdot\mathbf{u}_1=(1)(1)+(-1)(1)=0$.

The wrong calculation uses $2$, the numerator alone, as the coefficient. It gives $\mathbf{v}_2-2\mathbf{u}_1=(2,0)-(2,2)=(0,-2)$. But $(0,-2)\cdot(1,1)=-2\ne0$, so this is not an orthogonal residual. The missing denominator matters whenever the direction does not have squared norm $1$.

Subtract Every Earlier Component

At stage $k$, each earlier residual has its own component to remove. Subtracting only the component along the most recent residual is not enough in general: removing one direction does not automatically remove the others. The following calculation makes that error visible.

Worked Example: Omitting an Earlier Residual

Apply Gram–Schmidt to $\mathbf{v}_1=(1,1,0)$, $\mathbf{v}_2=(1,0,1)$, and $\mathbf{v}_3=(1,1,1)$ in $\mathbb{R}^3$. Then compare the correct third residual with the result of omitting the subtraction along $\mathbf{u}_1$.

Start with $\mathbf{u}_1=\mathbf{v}_1=(1,1,0)$. Its squared norm is $1+1+0=2$. The coefficient for the second input is $(\mathbf{v}_2\cdot\mathbf{u}_1)/(\mathbf{u}_1\cdot\mathbf{u}_1)=(1+0+0)/2=1/2$, so

\mathbf{u}_2=(1,0,1)-\frac12(1,1,0) =\left(\frac12,-\frac12,1\right).

Check that $\mathbf{u}_2\cdot\mathbf{u}_1=\frac12-\frac12+0=0$, and calculate $\mathbf{u}_2\cdot\mathbf{u}_2=\frac14+\frac14+1=\frac32$.

For the third input, the coefficient along $\mathbf{u}_1$ is $(\mathbf{v}_3\cdot\mathbf{u}_1)/2=(1+1+0)/2=1$. The coefficient along $\mathbf{u}_2$ is $(\frac12-\frac12+1)/(\frac32)=\frac23$. Both terms must be subtracted:

\begin{aligned} \mathbf{u}_3 &=(1,1,1)-(1,1,0)-\frac23\left(\frac12,-\frac12,1\right)\\ &=(0,0,1)-\left(\frac13,-\frac13,\frac23\right) =\left(-\frac13,\frac13,\frac13\right). \end{aligned}

Check against both earlier residuals:

\mathbf{u}_3\cdot\mathbf{u}_1=-\frac13+\frac13=0,\qquad \mathbf{u}_3\cdot\mathbf{u}_2=-\frac16-\frac16+\frac13=0.

If the subtraction along $\mathbf{u}_1$ is omitted, the candidate is instead $\mathbf{v}_3-\frac23\mathbf{u}_2=(\frac23,\frac43,\frac13)$. Its dot product with $\mathbf{u}_1$ is $\frac23+\frac43=2$, not zero. It may still be orthogonal to $\mathbf{u}_2$, but that single check cannot establish orthogonality to both earlier directions.

Residual audit: For each new residual $\mathbf{u}_k$, calculate $\langle\mathbf{u}_k,\mathbf{u}_j\rangle$ for every $j<k$. Each result must be zero. A zero dot product with only the most recent residual is not a complete check.

Normalization Is Safe Only When the Formula Matches

Normalization replaces a nonzero vector $\mathbf{u}$ by the unit vector $\mathbf{q}=\mathbf{u}/\|\mathbf{u}\|$. This changes its length, not its direction. It is valid to use normalized vectors during Gram–Schmidt, but the coefficient must then be computed using the normalized vector. The two consistent ways of writing the subtraction give the same vector.

Proposition: If $\mathbf{u}\ne\mathbf{0}$ and $\mathbf{q}=\mathbf{u}/\|\mathbf{u}\|$, then for any $\mathbf{x}$, $$ \frac{\langle\mathbf{x},\mathbf{u}\rangle}{\langle\mathbf{u},\mathbf{u}\rangle}\mathbf{u} =\frac{\langle\mathbf{x},\mathbf{q}\rangle}{\langle\mathbf{q},\mathbf{q}\rangle}\mathbf{q}. $$

Proof. The norm $\|\mathbf{u}\|$ is a positive real scalar and $\langle\mathbf{q},\mathbf{q}\rangle=1$. By linearity in the first slot, $\langle\mathbf{x},\mathbf{q}\rangle=\langle\mathbf{x},\mathbf{u}\rangle/\|\mathbf{u}\|$. Therefore the right-hand side is $(\langle\mathbf{x},\mathbf{u}\rangle/\|\mathbf{u}\|)(\mathbf{u}/\|\mathbf{u}\|)=\langle\mathbf{x},\mathbf{u}\rangle\mathbf{u}/\|\mathbf{u}\|^2$. Since $\|\mathbf{u}\|^2=\langle\mathbf{u},\mathbf{u}\rangle$, this equals the left-hand side. $\square$

Worked Example: Normalizing Too Soon

Let $\mathbf{v}_1=(2,0)$ and $\mathbf{v}_2=(1,1)$. Compute the second residual first with the unnormalized direction and then with its normalized version. Finally, check the inconsistent calculation that uses the old coefficient with the new vector.

Using $\mathbf{u}_1=(2,0)$, its squared norm is $4$, and the coefficient is $(2\cdot1+0\cdot1)/4=1/2$. Thus

\mathbf{u}_2=(1,1)-\frac12(2,0)=(0,1).

The check is $\mathbf{u}_2\cdot\mathbf{u}_1=0$. Now normalize the first direction: $\mathbf{q}_1=\mathbf{u}_1/\|\mathbf{u}_1\|=(1,0)$. Using this unit vector, the coefficient is $\mathbf{v}_2\cdot\mathbf{q}_1=1$, so the residual is again $(1,1)-1(1,0)=(0,1)$.

A mixed calculation uses the old coefficient $1/2$ but subtracts the normalized vector. It produces $(1,1)-\frac12(1,0)=(\frac12,1)$. The dot product with $\mathbf{q}_1$ is $\frac12\ne0$, exposing the error. Normalizing early is not the problem; mixing a coefficient for one scale with a vector of another scale is.

Check Residuals Before Normalizing

Normalization does not repair a residual that is not orthogonal to the earlier directions. It only changes the residual’s length. A useful order of operations is therefore: compute the residual, check all its inner products with earlier residuals, calculate its norm, and only then divide by that norm if a unit vector is required.

For example, in the three-vector calculation above, suppose a sign error gives the proposed residual $\widetilde{\mathbf{u}}_3=(-\frac13,\frac13,0)$. Its dot product with $\mathbf{u}_1$ is $-\frac13+\frac13=0$, so checking only $\mathbf{u}_1$ would miss the error. But

\widetilde{\mathbf{u}}_3\cdot\mathbf{u}_2 =-\frac16-\frac16+0=-\frac13\ne0.

The residual fails the full audit. The correct third coordinate is $\frac13$; with it, the dot product becomes $-\frac16-\frac16+\frac13=0$. This is why every earlier direction must be checked, not just one.

When a residual passes the orthogonality checks, verify normalization separately. In the corrected calculation, $\mathbf{u}_3=(-\frac13,\frac13,\frac13)$, so $\|\mathbf{u}_3\|^2=\frac19+\frac19+\frac19=\frac13$. Its unit version is $\mathbf{q}_3=\sqrt{3}\mathbf{u}_3$. Its squared norm is $3\cdot\frac13=1$, and scaling preserves its zero dot products with $\mathbf{u}_1$ and $\mathbf{u}_2$. Orthogonality and unit length are distinct conditions, so both need to be checked when an orthonormal list is requested.

A Reliable Calculation Routine

Record the current directions.
Keep track of whether each earlier vector is an unnormalized residual $\mathbf{u}_j$ or a unit vector $\mathbf{q}_j$.

Compute each coefficient from its formula.
For an unnormalized residual, divide by $\langle\mathbf{u}_j,\mathbf{u}_j\rangle$. For a unit vector, the denominator is $1$.

Subtract every earlier component.
The sum in the recursion contains one term for each earlier residual, not just the last one.

Audit before proceeding.
Take the inner product of the candidate residual with every earlier direction. Correct values are all zero.

Normalize only with its computed norm.
If a unit vector is required, check that the residual is nonzero, calculate its norm, divide by it, and verify unit length.

These checks are useful even when the final answer is only an orthogonal list rather than an orthonormal one. They localize errors: a nonzero inner product points to a missing or incorrect subtraction, while a non-unit norm after normalization points to a norm or division error. Checking the intermediate residuals is generally more informative than checking only the final list.

Check Your Understanding

For each question, support your conclusion with the relevant coefficient calculation or inner-product check.

For $\mathbf{u}=(1,2)$ and $\mathbf{x}=(3,0)$, find the coefficient that makes $\mathbf{x}-c\mathbf{u}$ orthogonal to $\mathbf{u}$, and verify the residual.
Why is it not sufficient to check that a new residual is orthogonal only to the immediately preceding residual?
If $\mathbf{q}=\mathbf{u}/\|\mathbf{u}\|$, what coefficient should be used when subtracting a component along $\mathbf{q}$?
A proposed residual has zero dot product with one earlier vector but a nonzero dot product with another. Is it a valid Gram–Schmidt residual? Explain.
After normalizing a nonzero residual, what two properties should be checked when the goal is an orthonormal list?

Common Gram-Schmidt Calculation Errors

What You'll Learn

Why Gram–Schmidt Errors Can Be Hard to Spot