Orthonormal Systems And Bessel's Inequality

From Orthogonality to a Coefficient Bound

In a Hilbert space, mutually orthogonal unit vectors allow a function to be decomposed into a finite sum of components along those vectors and a remainder perpendicular to them. The squared norm of the function is then the sum of the squared norms of those components and the remainder. Bessel's inequality is the resulting bound on the coefficients: no matter how many orthonormal vectors we select, the sum of their squared coefficients cannot exceed the squared norm of the function.

Let $(X,\mathcal{M},\mu)$ be any measure space; no assumption of finiteness, sigma-finiteness, or completeness is needed. We use the complex space $L^2(\mu)$, whose elements are equivalence classes, up to equality $\mu$-a.e., of measurable complex-valued functions $f$ with $\int |f|^2\,d\mu<\infty$. The inner product convention here is linear in the first variable:

\langle f,g\rangle=\int_X f\overline{g}\,d\mu, \qquad \|f\|_2^2=\langle f,f\rangle=\int_X |f|^2\,d\mu.

These definitions do not depend on the chosen representatives of the equivalence classes: changing a representative on a $\mu$-null set does not change the integrals. For real-valued functions, the conjugation may simply be omitted.

Definition: A family $(e_j)_{j\in J}$ in $L^2(\mu)$ is an orthonormal system if $\langle e_j,e_k\rangle=0$ whenever $j\ne k$, and $\|e_j\|_2=1$ for every $j\in J$. Equivalently, $\langle e_j,e_k\rangle=\delta_{jk}$, where $\delta_{jk}=1$ if $j=k$ and $0$ otherwise. These are equalities of inner products of $L^2$ classes, so representatives need not satisfy pointwise identities everywhere.

For a finite set of indices $F\subseteq J$, the coefficient of $f\in L^2(\mu)$ in the direction $e_j$ is $\langle f,e_j\rangle$. The corresponding finite sum is its orthogonal projection onto the span of the selected vectors. We first verify the norm identity that makes this projection useful.

The Finite Projection Identity

Proposition: Let $(X,\mathcal{M},\mu)$ be any measure space, let $(e_j)_{j\in J}$ be an orthonormal system in complex $L^2(\mu)$, and let $f\in L^2(\mu)$. For every finite $F\subseteq J$, put $$ p_F=\sum_{j\in F}\langle f,e_j\rangle e_j. $$ Then $f-p_F$ is orthogonal to every $e_k$ with $k\in F$, and $$ \|f\|_2^2=\sum_{j\in F}|\langle f,e_j\rangle|^2+\|f-p_F\|_2^2. $$

Proof. If $F$ is empty, then $p_F=0$; the orthogonality assertion has no indices to check, and the identity is $\|f\|_2^2=0+\|f\|_2^2$. Suppose $F$ is nonempty. Write $c_j=\langle f,e_j\rangle$. For each $k\in F$, linearity in the first variable and orthonormality give $$ \langle f-p_F,e_k\rangle =\langle f,e_k\rangle-\sum_{j\in F}c_j\langle e_j,e_k\rangle =c_k-c_k=0. $$ Thus $f-p_F$ is orthogonal to every vector in the selected system and hence to their finite linear combination $p_F$. Since $f=p_F+(f-p_F)$, expansion of the squared norm gives $$ \|f\|_2^2 =\|p_F\|_2^2+\|f-p_F\|_2^2, $$ because the two cross inner products vanish. Expanding $p_F$ and using $\langle e_j,e_k\rangle=\delta_{jk}$ yields $\|p_F\|_2^2=\sum_{j\in F}|c_j|^2$. Substitution proves the identity. $\square$

Bessel's Inequality

Theorem (Bessel's Inequality): Under the assumptions of the proposition, for every finite $F\subseteq J$, $$ \sum_{j\in F}|\langle f,e_j\rangle|^2\le \|f\|_2^2. $$ If the system is countable, indexed by $\mathbb{N}$, then the nonnegative series satisfies $$ \sum_{j=1}^{\infty}|\langle f,e_j\rangle|^2\le \|f\|_2^2. $$

Proof. The finite projection identity expresses $\|f\|_2^2$ as the sum of $\sum_{j\in F}|\langle f,e_j\rangle|^2$ and the nonnegative quantity $\|f-p_F\|_2^2$. Dropping that nonnegative term proves the finite inequality. For a countable system, apply the finite inequality to $F=\{1,\ldots,N\}$ for each positive integer $N$. The partial sums are nondecreasing and bounded above by $\|f\|_2^2$, so they converge to the series, with value at most $\|f\|_2^2$. This also covers a finite system by taking all its indices. $\square$

Taking $F=\{j\}$ gives the individual coefficient estimate $|\langle f,e_j\rangle|\le\|f\|_2$. The finite inequality is the essential statement: for any chosen finite collection, the total squared coefficient size is bounded. The countable version follows by increasing finite partial sums; it does not assert that the system spans all of $L^2(\mu)$.

Worked Coefficients on Lebesgue Measure

Worked Example: The Function x and Two Orthonormal Directions

Take $[0,1]$ with Lebesgue measure $m$, a finite complete measure space, and regard $f(x)=x$ as a real-valued element of complex $L^2([0,1],m)$. Define $e_1(x)=1$ and $e_2(x)=\sqrt{12}(x-\tfrac12)$. We compute the coefficients and check Bessel's inequality.

First, $\|e_1\|_2^2=\int_0^1 1\,dx=1$. Also, $$ \langle e_1,e_2\rangle =\sqrt{12}\int_0^1(x-\tfrac12)\,dx =\sqrt{12}\left(\tfrac12-\tfrac12\right)=0, $$ and $$ \|e_2\|_2^2 =12\int_0^1(x-\tfrac12)^2\,dx =12\left(\tfrac13-\tfrac12+\tfrac14\right) =12\cdot\tfrac1{12}=1. $$ Thus $e_1,e_2$ are orthonormal.

The first coefficient is $$ \langle f,e_1\rangle=\int_0^1x\,dx=\tfrac12. $$ For the second, $$ \langle f,e_2\rangle =\sqrt{12}\int_0^1(x^2-\tfrac12x)\,dx =\sqrt{12}\left(\tfrac13-\tfrac14\right) =\frac{\sqrt{12}}{12} =\frac1{\sqrt{12}}. $$ Consequently the sum of squared coefficients is $$ \left|\tfrac12\right|^2+\left|\tfrac1{\sqrt{12}}\right|^2 =\tfrac14+\tfrac1{12}=\tfrac13. $$ The norm of $f$ also satisfies $\|f\|_2^2=\int_0^1x^2\,dx=\tfrac13$. The bound holds with equality. Indeed, the projection is $p_F=\tfrac12e_1+\tfrac1{\sqrt{12}}e_2=\tfrac12+(x-\tfrac12)=x$, so the residual is zero. This also checks the coefficient calculation by substitution.

Worked Example: Projection onto Two Interval Indicators

Again use the finite complete Lebesgue measure space $([0,1],\mathcal{L},m)$. Let $I_1=[0,\tfrac12)$, $I_2=[\tfrac12,1]$, and define $e_1=\sqrt2\,\chi_{I_1}$, $e_2=\sqrt2\,\chi_{I_2}$. The intervals are disjoint, each has measure $\tfrac12$, and their union is $[0,1]$. Hence $\|e_1\|_2^2=2\cdot\tfrac12=1$, $\|e_2\|_2^2=2\cdot\tfrac12=1$, and $\langle e_1,e_2\rangle=0$.

For $f(x)=x$, the coefficients are $$ \langle f,e_1\rangle=\sqrt2\int_0^{1/2}x\,dx =\sqrt2\cdot\tfrac18=\frac{\sqrt2}{8}, \qquad \langle f,e_2\rangle=\sqrt2\int_{1/2}^{1}x\,dx =\sqrt2\cdot\tfrac38=\frac{3\sqrt2}{8}. $$ Their squared sum is $$ \frac{2}{64}+\frac{18}{64} =\frac1{32}+\frac9{32} =\frac5{16}. $$ Meanwhile, $\|f\|_2^2=\int_0^1x^2\,dx=\tfrac13$, and $\tfrac13-\tfrac5{16}=\tfrac1{48}>0$, as Bessel's inequality requires.

To check the difference independently, the projection is $\tfrac14$ on $I_1$ and $\tfrac34$ on $I_2$. The residual norm squared is therefore $$ \int_0^{1/2}(x-\tfrac14)^2\,dx +\int_{1/2}^{1}(x-\tfrac34)^2\,dx =\tfrac1{96}+\tfrac1{96}=\tfrac1{48}. $$ Adding this to the coefficient sum gives $\tfrac5{16}+\tfrac1{48}=\tfrac{15}{48}+\tfrac1{48}=\tfrac13$, in agreement with the finite projection identity.

Worked Example: A Coefficient Bound for a Counting-Measure Function

Let $X=\mathbb{N}$, let $\mathcal{M}$ be all subsets of $\mathbb{N}$, and let $\mu$ be counting measure. This measure space is complete and sigma-finite, but not finite. Its $L^2$ norm is the square root of the sum of squared absolute values. Take $f(1)=3$, $f(2)=4$, and $f(n)=0$ for $n\ge3$, and choose $e(1)=e(2)=1/\sqrt2$, $e(n)=0$ for $n\ge3$. Then $\|e\|_2^2=\tfrac12+\tfrac12=1$, so the one-element system $\{e\}$ is orthonormal.

The coefficient and its square are $$ \langle f,e\rangle=\frac3{\sqrt2}+\frac4{\sqrt2}=\frac7{\sqrt2}, \qquad |\langle f,e\rangle|^2=\frac{49}{2}. $$ The function norm is $\|f\|_2^2=3^2+4^2=25=\tfrac{50}{2}$, so the coefficient bound reads $\tfrac{49}{2}\le\tfrac{50}{2}$. The projection has first two values $\tfrac72,\tfrac72$, and the residual has values $-\tfrac12,\tfrac12$. Its squared norm is $\tfrac14+\tfrac14=\tfrac12$. Thus $\tfrac{49}{2}+\tfrac12=25$, verifying the identity directly.

Normalization and Equality: Common Pitfalls

Pitfall: applying the formula to a vector that is not normalized. On $([0,1],\mathcal{L},m)$, take $f(x)=1$ and $u(x)=2$. A mistaken calculation treats $u$ as a unit vector and asserts $|\langle f,u\rangle|^2\le\|f\|_2^2$. But $\langle f,u\rangle=2$, so this would give $4\le1$, which is false. The error is that $\|u\|_2^2=4$, not $1$. Normalizing gives $e=u/\|u\|_2=1$; now $|\langle f,e\rangle|^2=1=\|f\|_2^2$, as Bessel's inequality states. The unit-norm condition is essential to the stated coefficient formula.

Pitfall: assuming equality for every orthonormal system. On the same Lebesgue space, take only $e_1(x)=1$ and $f(x)=x$. The coefficient square is $|\langle f,e_1\rangle|^2=|\int_0^1x\,dx|^2=\tfrac14$, whereas $\|f\|_2^2=\int_0^1x^2\,dx=\tfrac13$. The residual $x-\tfrac12$ has squared norm $\int_0^1(x-\tfrac12)^2\,dx=\tfrac1{12}$, and $\tfrac14+\tfrac1{12}=\tfrac13$. The strict inequality records a nonzero component perpendicular to the chosen direction. Bessel's inequality supplies a bound; equality requires the residual for the selected projection to vanish.

The result is useful whenever a function is tested against many orthogonal directions: it gives a uniform constraint on the total squared coefficient size without requiring that the system span the entire space. For a countable system, it also guarantees that the coefficient sequence is square-summable. It does not, by itself, say that the coefficients recover the original function.

Check Your Understanding

For each question, state the relevant measure space and verify that the vectors used are orthonormal before applying the inequality.

On $[0,1]$ with Lebesgue measure, compute the coefficient of $f(x)=x$ along $e(x)=\sqrt{12}(x-\tfrac12)$, and check the one-vector Bessel bound.
For a finite orthonormal family, explain why subtracting its projection leaves a vector orthogonal to each selected member.
On $\mathbb{N}$ with counting measure, let $f(1)=1$, $f(2)=2$, and $f(n)=0$ for $n\ge3$. Compute its coefficient along $e(1)=e(2)=1/\sqrt2$, with $e(n)=0$ for $n\ge3$, and verify the bound.
Give an example where Bessel's inequality is strict, and identify the nonzero residual.
Why does the calculation using $u(x)=2$ on $[0,1]$ not contradict Bessel's inequality?