QR Factorization and Orthonormal Bases

Why Factor a Matrix into Orthonormal Directions?

A matrix with linearly independent columns describes a collection of independent directions, but those directions need not be perpendicular or have unit length. The QR factorization separates these two features: one factor gives orthonormal directions spanning the same column space, while the other records how the original columns are assembled from those directions. This separation makes coordinates easier to interpret and calculations involving the column space more transparent.

We work over the real numbers. Let $A$ be an $m\times n$ matrix with $m\geq n$ and rank $n$. Its columns $a_1,\ldots,a_n$ are linearly independent. The reduced, or thin, QR factorization keeps only the $n$ orthonormal directions needed to span those columns.

Definition: A reduced QR factorization of a full-column-rank matrix $A\in\mathbb{R}^{m\times n}$ is a representation $A=QR$, where $Q\in\mathbb{R}^{m\times n}$ satisfies $Q^{T}Q=I_n$, and $R\in\mathbb{R}^{n\times n}$ is upper triangular with nonzero diagonal. We use the standard sign convention that every diagonal entry of $R$ is positive. The columns of $Q$ are an orthonormal basis for $\operatorname{Range}(A)$.

The condition $Q^{T}Q=I_n$ says that each column of $Q$ has length one and different columns are orthogonal. Since $A$ has rank $n$, the triangular factor $R$ must be invertible; the positive-diagonal convention makes the factorization unique. A full QR factorization can instead complete the columns of $Q$ to an orthonormal basis of $\mathbb{R}^m$, but that extra completion is not needed to represent the column space of $A$.

Constructing the Factors by Orthogonalization

The classical Gram–Schmidt procedure constructs $Q$ and $R$ one column at a time. Start with the first column of $A$, normalize it, then subtract from each later column its components in the directions already constructed. At each stage the remaining vector is perpendicular to all earlier basis vectors.

Construction: Set $u_1=a_1$, $r_{11}=\lVert u_1\rVert$, and $q_1=u_1/r_{11}$. For $j=2,\ldots,n$, set $r_{ij}=q_i^{T}a_j$ for $i<j$, then define $u_j=a_j-\sum_{i=1}^{j-1}r_{ij}q_i$, $r_{jj}=\lVert u_j\rVert$, and $q_j=u_j/r_{jj}$. The vectors $q_j$ form the columns of $Q$, and the entries $r_{ij}$ form $R$, with $r_{ij}=0$ when $i>j$.

Full column rank is essential in this construction. If a residual $u_j$ were zero, then $a_j$ would be a linear combination of $a_1,\ldots,a_{j-1}$, contradicting the independence of the columns of $A$. Thus each $r_{jj}$ is positive, and every normalization is well-defined.

Worked Example: Orthogonalizing Two Columns

Take $$ A=\begin{pmatrix}1&1\\1&0\\0&1\end{pmatrix}, $$ whose columns are $a_1=(1,1,0)^T$ and $a_2=(1,0,1)^T$. First, $$ r_{11}=\lVert a_1\rVert=\sqrt{2},\qquad q_1=\frac{1}{\sqrt{2}}\begin{pmatrix}1\\1\\0\end{pmatrix}. $$

The component of $a_2$ along $q_1$ is $$ r_{12}=q_1^Ta_2=\frac{1}{\sqrt{2}}. $$ Subtract that component: $$ u_2=a_2-r_{12}q_1 =\begin{pmatrix}1\\0\\1\end{pmatrix} -\begin{pmatrix}1/2\\1/2\\0\end{pmatrix} =\begin{pmatrix}1/2\\-1/2\\1\end{pmatrix}. $$ Its squared length is $1/4+1/4+1=3/2$, so $r_{22}=\sqrt{6}/2$. Normalizing gives $$ q_2=\frac{u_2}{r_{22}} =\frac{1}{\sqrt{6}}\begin{pmatrix}1\\-1\\2\end{pmatrix}. $$

Therefore, $$ Q=\frac{1}{\sqrt{6}}\begin{pmatrix}\sqrt{3}&1\\\sqrt{3}&-1\\0&2\end{pmatrix}, \qquad R=\begin{pmatrix}\sqrt{2}&1/\sqrt{2}\\0&\sqrt{6}/2\end{pmatrix}. $$ The columns of $Q$ have unit length and dot product zero. Multiplying gives $QR=A$: the first column is $(1,1,0)^T$, and the second is $(1,0,1)^T$.

What the Triangular Factor Records

Because $A=QR$, the $j$th column of $A$ is a linear combination of the columns of $Q$. Its coefficients are the entries in column $j$ of $R$: $$ a_j=\sum_{i=1}^{j} r_{ij}q_i. $$ There are no terms for $i>j$, because $R$ is upper triangular. Thus $R$ records the coordinates of each original column in the orthonormal basis supplied by $Q$. The diagonal coefficient $r_{jj}$ is the length of the part of $a_j$ that remains after removing its components in the earlier directions.

More generally, for any $x\in\mathbb{R}^n$, the vector $Ax$ lies in $\operatorname{Range}(A)$, and $$ Ax=Q(Rx). $$ Since the columns of $Q$ are orthonormal, $Rx$ is the coordinate vector of $Ax$ in that basis. This also identifies the coordinates of any $y\in\operatorname{Range}(A)$: if $y=Ax$, then its coordinates are $Rx=Q^Ty$.

Theorem: If $A=QR$ is a reduced QR factorization as defined above, then $Q$ has orthonormal columns spanning $\operatorname{Range}(A)$. For every $y\in\operatorname{Range}(A)$, its coordinate vector in the columns of $Q$ is $Q^Ty$. In particular, if $y=Ax$, those coordinates equal $Rx$.

Proof. From $Q^TQ=I_n$, the columns of $Q$ are orthonormal and linearly independent. The factorization $A=QR$ implies that every column of $A$ is in $\operatorname{Range}(Q)$. Conversely, $R$ is invertible because it is triangular with positive diagonal, so $Q=AR^{-1}$; every column of $Q$ is therefore in $\operatorname{Range}(A)$. The two ranges are equal. If $y=Qc$, multiplication by $Q^T$ gives $Q^Ty=Q^TQc=c$, proving the coordinate formula. Taking $y=Ax=Q(Rx)$ gives coordinates $Rx$. $\square$

Worked Example: QR for Already Orthogonal Columns

Consider $$ A=\begin{pmatrix}1&1\\1&-1\end{pmatrix}. $$ Its columns have squared lengths $2$, and their dot product is $1-1=0$. They are already orthogonal, so normalizing them gives $$ Q=\frac{1}{\sqrt{2}}\begin{pmatrix}1&1\\1&-1\end{pmatrix}, \qquad R=\begin{pmatrix}\sqrt{2}&0\\0&\sqrt{2}\end{pmatrix}. $$

Indeed, $Q^TQ=I_2$, $R$ is upper triangular with positive diagonal, and $QR=A$. The determinant of $Q$ is $-1$, but this does not violate orthogonality: an orthogonal matrix may have determinant $1$ or $-1$. The QR sign convention concerns the diagonal of $R$, not the determinant of $Q$.

Existence and Uniqueness

The construction above gives existence and also explains the role of the rank assumption. The following statement summarizes the result and establishes that the positive-diagonal convention removes ambiguity.

Theorem: Every real $m\times n$ matrix $A$ with $m\geq n$ and rank $n$ has a reduced QR factorization $A=QR$ with $Q^TQ=I_n$, $R$ upper triangular, and positive diagonal. This factorization is unique under the positive-diagonal convention.

Proof. Apply the Gram–Schmidt construction to the columns of $A$. At step $j$, the residual $u_j$ cannot be zero: otherwise $a_j$ would lie in the span of its predecessors, contradicting full column rank. Therefore $r_{jj}=\lVert u_j\rVert>0$, and $q_j$ is defined. By construction, each $q_j$ is a unit vector perpendicular to $q_1,\ldots,q_{j-1}$, so $Q^TQ=I_n$. The expansion of each $a_j$ uses only $q_1,\ldots,q_j$, which makes $R$ upper triangular, and these expansions give $A=QR$.

For uniqueness, suppose $A=Q_1R_1=Q_2R_2$ are two such factorizations. Since $R_1$ is invertible, multiplying the equality on the left by $Q_2^T$ and on the right by $R_1^{-1}$ gives $$ Q_2^TQ_1=R_2R_1^{-1}. $$ The left side is orthogonal, and the right side is upper triangular with positive diagonal. An upper-triangular orthogonal matrix with positive diagonal must be the identity: its first column has only its first entry possibly nonzero, and unit length makes that entry $1$; orthogonality then forces the other entries in the first row to be zero. Repeating this argument on the remaining rows and columns shows that every diagonal entry is $1$ and every off-diagonal entry is zero. Thus $Q_2^TQ_1=I_n$, which implies $Q_1=Q_2$. Substituting into $Q_1R_1=Q_2R_2$ gives $R_1=R_2$. $\square$

Worked Example: Recovering Coordinates from QR

Let $$ A=\begin{pmatrix}1&1\\0&2\\1&0\end{pmatrix}. $$ Its first column has length $\sqrt{2}$, so $q_1=(1,0,1)^T/\sqrt{2}$ and $r_{11}=\sqrt{2}$. The projection coefficient for the second column is $$ r_{12}=q_1^T(1,2,0)^T=\frac{1}{\sqrt{2}}. $$ Subtracting this component gives $$ u_2=\begin{pmatrix}1\\2\\0\end{pmatrix} -\frac{1}{\sqrt{2}}q_1 =\begin{pmatrix}1/2\\2\\-1/2\end{pmatrix}, \qquad r_{22}=\lVert u_2\rVert=\frac{3}{\sqrt{2}}. $$ Hence $q_2=(1,4,-1)^T/(3\sqrt{2})$, and $$ Q=\begin{pmatrix}1/\sqrt{2}&1/(3\sqrt{2})\\0&4/(3\sqrt{2})\\1/\sqrt{2}&-1/(3\sqrt{2})\end{pmatrix}, \qquad R=\begin{pmatrix}\sqrt{2}&1/\sqrt{2}\\0&3/\sqrt{2}\end{pmatrix}. $$

Take $x=(2,-1)^T$. Then $$ Ax=2\begin{pmatrix}1\\0\\1\end{pmatrix} -\begin{pmatrix}1\\2\\0\end{pmatrix} =\begin{pmatrix}1\\-2\\2\end{pmatrix}, \qquad Rx=\begin{pmatrix}2\sqrt{2}-1/\sqrt{2}\\-3/\sqrt{2}\end{pmatrix} =\begin{pmatrix}3/\sqrt{2}\\-3/\sqrt{2}\end{pmatrix}. $$ Thus $Rx$ gives the coordinates of $Ax$ in the columns of $Q$. Checking directly, $$ Q(Rx)=\frac{3}{\sqrt{2}}q_1-\frac{3}{\sqrt{2}}q_2 =\begin{pmatrix}3/2\\0\\3/2\end{pmatrix} -\begin{pmatrix}1/2\\2\\-1/2\end{pmatrix} =\begin{pmatrix}1\\-2\\2\end{pmatrix} =Ax. $$

Common Mistakes and Why QR Matters

QR is useful whenever a calculation benefits from separating a column space into orthonormal directions and coefficients. In statistical work, orthonormal directions help clarify which part of a vector lies in a model’s column space, while the triangular factor preserves the coordinates linking that space to the original columns. The factorization is therefore not merely a rewriting of $A$: it exposes the geometry of its range.

A frequent mistake is to treat any pair of factors with $A=QR$ as a QR factorization. The defining checks matter: $Q$ must have orthonormal columns, $R$ must be upper triangular, and under our convention its diagonal must be positive. Another common error is to assume the factorization is unique without fixing signs. For example, replacing one column of $Q$ and the matching row of $R$ by their negatives leaves $QR$ unchanged. Requiring positive diagonal entries in $R$ removes this sign ambiguity.

Confusing dimensions: In the reduced factorization, $Q$ is $m\times n$, not necessarily square, and $R$ is $n\times n$.
Ignoring rank: If the columns of $A$ are dependent, a residual in Gram–Schmidt can be zero, so the full-rank construction above does not apply as stated.
Checking only the product: $QR=A$ alone is not enough; also verify $Q^TQ=I_n$, triangularity, and the chosen diagonal convention.
Mixing up coefficients: The coordinates of $Ax$ in the columns of $Q$ are $Rx$, not generally $x$.

Key takeaway: For a full-column-rank matrix, QR expresses the same column space using an orthonormal basis $Q$, while the upper-triangular matrix $R$ records the coordinates of the original columns in that basis.

Check Your Understanding

Use the definitions and calculations above to answer the following questions.

For a reduced QR factorization of an $m\times n$ full-column-rank matrix, what are the dimensions of $Q$ and $R$, and what equations characterize them?
Why does full column rank guarantee that no residual in the Gram–Schmidt construction is zero?
If $y=Ax$, what is the coordinate vector of $y$ in the columns of $Q$, and why?
Why does the condition that $R$ have positive diagonal make the QR factorization unique?
What is wrong with checking only that $QR=A$ when someone proposes a QR factorization?