Using Cauchy-Schwarz To Bound Inner Products

From Inner Products to Bounds

An inner product can be difficult to calculate term by term, especially when it is a sum with many terms, a product of matrix entries, or an integral of a product of functions. Often the useful question is not the exact value, but how large the expression can be. The real Cauchy–Schwarz inequality answers this using only the norms of the two objects being paired.

For a real inner product space, the induced norm is defined by \(\|\mathbf{u}\|=\sqrt{\langle\mathbf{u},\mathbf{u}\rangle}\). In \(\mathbb{R}^n\) with the standard inner product, this is the Euclidean norm, and \(\langle\mathbf{u},\mathbf{v}\rangle=\mathbf{u}^{\top}\mathbf{v}\). We will use the real Cauchy–Schwarz inequality, established in the earlier material on the real Cauchy–Schwarz inequality.

The absolute value matters: the inner product can be positive or negative. Once the norms are known, Cauchy–Schwarz gives a two-sided bound without requiring the inner product itself to be evaluated. For an upper bound alone, it gives \(\langle\mathbf{u},\mathbf{v}\rangle\leq\|\mathbf{u}\|\|\mathbf{v}\|\); for a bound on magnitude, retain the absolute value.

Dot Products and Weighted Sums

For columns \(\mathbf{x},\mathbf{y}\in\mathbb{R}^n\), the dot product is \(\mathbf{x}^{\top}\mathbf{y}\), and its bound is \(|\mathbf{x}^{\top}\mathbf{y}|\leq\|\mathbf{x}\|\|\mathbf{y}\|\). This is useful when the coordinates make the norms straightforward to calculate, even if expanding the dot product would involve many terms.

Positive weights can be handled by absorbing their square roots into the coordinates. Suppose \(w_1,\dots,w_n\) are positive real numbers and \(a_i,b_i\) are real. The resulting bound is useful for weighted sums because it accounts for each weight in both norm factors.

Proof. Form the real columns \(\mathbf{u}=(\sqrt{w_1}a_1,\dots,\sqrt{w_n}a_n)^{\top}\) and \(\mathbf{v}=(\sqrt{w_1}b_1,\dots,\sqrt{w_n}b_n)^{\top}\). Their dot product is \(\mathbf{u}^{\top}\mathbf{v}=\sum_{i=1}^{n}w_i a_i b_i\), while \(\|\mathbf{u}\|^2=\sum_{i=1}^{n}w_i a_i^2\) and \(\|\mathbf{v}\|^2=\sum_{i=1}^{n}w_i b_i^2\). Applying real Cauchy–Schwarz to \(\mathbf{u}\) and \(\mathbf{v}\) proves the stated inequality. \(\square\)

Matrix Entry Products

For real matrices \(\mathbf{A},\mathbf{B}\) of the same size, their Frobenius inner product is the sum of products of corresponding entries. The Frobenius norm is the square root of the sum of squared entries. These definitions make a matrix a vector of its entries for the purpose of applying Cauchy–Schwarz. The matrix sizes must agree so that corresponding entries can be paired.

To justify this consequence, list the \(mn\) entries of each matrix in the same order as coordinates of two real columns. Their dot product is exactly the sum on the left, and their Euclidean norms are the two factors on the right. Real Cauchy–Schwarz therefore applies, including when one of the matrices is zero.

Integrals of Products

Let \(f\) and \(g\) be real continuous functions on a closed interval \([a,b]\). With the integral inner product \(\langle f,g\rangle=\int_a^b f(t)g(t)\,dt\), the induced norm is \(\|f\|=\left(\int_a^b f(t)^2\,dt\right)^{1/2}\). The real Cauchy–Schwarz inequality in this inner product gives an integral bound. Continuity ensures these integrals exist; the same inequality applies in any real inner product function space where the functions have finite norms.

This follows by using \(f\) and \(g\) as the two vectors in the real inner product space of continuous functions on \([a,b]\), with the integral inner product just defined. The right-hand side requires the two squared-norm integrals, but not the integral of the product.

Common Pitfalls and a Practical Method

Omitting the absolute value can invalidate a magnitude bound. For example, with \(\mathbf{u}=\begin{bmatrix}1\\0\end{bmatrix}\) and \(\mathbf{v}=\begin{bmatrix}-2\\0\end{bmatrix}\), a tempting but wrong claim is \(\mathbf{u}^{\top}\mathbf{v}\leq-\|\mathbf{u}\|\|\mathbf{v}\|\). The left side is \(-2\), while the right side is \(-2\), so this particular example is equality; it does not establish a general lower bound. A wrong calculation that does fail is \(\mathbf{u}=\begin{bmatrix}1\\0\end{bmatrix}\), \(\mathbf{v}=\begin{bmatrix}-1\\1\end{bmatrix}\): claiming \(\mathbf{u}^{\top}\mathbf{v}\leq-\|\mathbf{u}\|\|\mathbf{v}\|\) says \(-1\leq-\sqrt{2}\), which is false. The correction is to use the valid two-sided statement \(-\|\mathbf{u}\|\|\mathbf{v}\|\leq\mathbf{u}^{\top}\mathbf{v}\leq\|\mathbf{u}\|\|\mathbf{v}\|\), or equivalently its absolute-value form.

Do not drop positive weights. In a weighted sum the bound involves \(\sum w_i a_i^2\) and \(\sum w_i b_i^2\), not unweighted sums. For the weighted example above, using unweighted norms would give \(\sqrt{1+4+1}\sqrt{4+1+1}=6\), while the correct weighted bound is \(6\sqrt{3}\). The smaller number is not justified by the weighted Cauchy–Schwarz inequality. Incorporate each weight, or explicitly transform the coordinates by \(\sqrt{w_i}\).

For a new expression, first identify the two objects being paired and the inner product that produces the expression. Then calculate their norms, keeping any weights, matrix entries, or integral domain intact. Apply Cauchy–Schwarz, and finally check that the inequality concerns the intended quantity: an absolute value for a magnitude bound, or a one-sided inequality if only an upper or lower bound is needed.