Gram Matrices as Orthonormality Tests

What a Gram Matrix Records

When a list of vectors is under consideration, its pairwise inner products collect the essential information about orthogonality and lengths. Arranging those numbers in a matrix makes the tests systematic: off-diagonal entries record interactions between different vectors, while diagonal entries record squared norms. This matrix is called the Gram matrix of the list.

We work over \(\mathbb{R}\) or \(\mathbb{C}\), with the inner product linear in its first slot, as in the course convention. Thus, over \(\mathbb{C}\), \(\langle a\mathbf{u},b\mathbf{v}\rangle=a\overline{b}\langle\mathbf{u},\mathbf{v}\rangle\). In \(\mathbb{R}^n\), the inner product is the dot product. The norm is \(\|\mathbf{v}\|=\sqrt{\langle\mathbf{v},\mathbf{v}\rangle}\). For a list \(\mathbf{v}_1,\dots,\mathbf{v}_p\), we use the following entry order. It is chosen so the Gram matrix agrees with \(\mathbf{X}^{*}\mathbf{X}\) when the vectors are columns of \(\mathbf{X}\) in the standard complex inner product.

Since \(\langle\mathbf{v}_j,\mathbf{v}_i\rangle=\overline{\langle\mathbf{v}_i,\mathbf{v}_j\rangle}\), the order of the two vectors matters in complex spaces. In real inner product spaces the entries are symmetric, so this order makes no numerical difference. The diagonal entries are always real and nonnegative, and are positive exactly when the corresponding vector is nonzero.

The Orthogonality Tests

Proof. By definition, a list of nonzero vectors is pairwise orthogonal exactly when \(\langle\mathbf{v}_j,\mathbf{v}_i\rangle=0\) whenever \(i\ne j\). These are precisely the off-diagonal entries of \(\mathbf{G}\), so the pairwise inner products vanish exactly when \(\mathbf{G}\) is diagonal. Also, \(G_{ii}=\|\mathbf{v}_i\|^2>0\) exactly when \(\mathbf{v}_i\ne\mathbf{0}\). This proves both directions of the first equivalence. For the second, if the list is orthonormal, its vectors are pairwise orthogonal and each has norm \(1\). Thus the off-diagonal entries are zero and \(G_{ii}=\|\mathbf{v}_i\|^2=1\), giving \(\mathbf{G}=\mathbf{I}_p\). Conversely, if \(\mathbf{G}=\mathbf{I}_p\), its off-diagonal entries give pairwise orthogonality, and its diagonal entries give \(\|\mathbf{v}_i\|^2=1\) for every \(i\). The list is therefore orthonormal. \(\square\)

The nonzero condition in the first test matters. A zero vector has inner product zero with every vector, so all off-diagonal entries in its row and column are zero. But its diagonal entry is also zero, and it is not a member of an orthogonal list of nonzero vectors. The matrix reveals this: check for positive diagonal entries as well as zero off-diagonal entries.

Calculating Gram Matrices in Other Spaces

The definition uses the inner product of the space, not a special formula tied to coordinate vectors in \(\mathbb{R}^n\). For example, matrices can themselves be vectors in an inner product space. With the Frobenius inner product \(\langle\mathbf{A},\mathbf{B}\rangle=\operatorname{tr}(\mathbf{A}\mathbf{B}^{\top})\) on real matrices of the same size, this inner product is the sum of the products of corresponding entries. The same diagonal and off-diagonal tests apply.

Complex Entries and Conjugation

In a complex inner product space, the Gram matrix may have complex off-diagonal entries. The two entries on opposite sides of the diagonal are complex conjugates, not necessarily equal. The diagonal remains real and nonnegative. This is why the complex test is still that the off-diagonal entries vanish, rather than that they are merely real or equal.

A Useful Identity and a Common Error

Proof. Write \(\mathbf{w}=\sum_{j=1}^{p}c_j\mathbf{v}_j\). Linearity in the first slot and conjugate-linearity in the second give \(\langle\mathbf{w},\mathbf{w}\rangle=\sum_{j=1}^{p}\sum_{i=1}^{p}c_j\overline{c_i}\langle\mathbf{v}_j,\mathbf{v}_i\rangle\). By the definition of the Gram matrix, \(\langle\mathbf{v}_j,\mathbf{v}_i\rangle=G_{ij}\). Therefore this sum is \(\sum_{i=1}^{p}\sum_{j=1}^{p}\overline{c_i}G_{ij}c_j=\mathbf{c}^{*}\mathbf{G}\mathbf{c}\). Since a squared norm is real and nonnegative, this quantity is nonnegative for every \(\mathbf{c}\), which proves positive semidefiniteness. Also, \(G_{ji}=\langle\mathbf{v}_i,\mathbf{v}_j\rangle=\overline{\langle\mathbf{v}_j,\mathbf{v}_i\rangle}=\overline{G_{ij}}\), so \(\mathbf{G}^{*}=\mathbf{G}\). \(\square\)

A common mistake is to multiply complex components without conjugating the second vector. For the vectors \(\mathbf{u}=(1,i)^{\top}\) and \(\mathbf{v}=(i,1)^{\top}\) above, this incorrect calculation gives \(1(i)+i(1)=2i\) for their inner product and wrongly declares them nonorthogonal. The correction is to conjugate the components of the second vector: \(\langle\mathbf{u},\mathbf{v}\rangle=1\overline{i}+i\overline{1}=-i+i=0\). Follow the stated inner-product convention before filling any Gram matrix entry.

In practice, a Gram matrix compresses several pairwise tests into one calculation. The diagonal tells you whether each vector has unit length, while the off-diagonal entries tell you whether different vectors are orthogonal. It does not list the vectors themselves; it records the inner products relevant to these tests. In particular, \(\mathbf{G}=\mathbf{I}_p\) is the complete test for an orthonormal list, while a diagonal matrix with positive entries certifies an orthogonal list whose vectors may have different lengths.