Operator Norm

Measuring the Size of a Linear Map

For a vector, its norm measures size. For a bounded linear map, the operator norm measures how much the map can enlarge vectors, relative to the norms chosen on its domain and codomain. It packages the global estimate from the previous tutorial into one number: the smallest constant that controls every output by the size of its input.

Throughout this tutorial, $V$ and $W$ are normed vector spaces over $\mathbb{R}$, with norms $\|\cdot\|_V$ and $\|\cdot\|_W$, and $T:V\to W$ is a bounded linear map. The Bounded Linear Maps Form a Normed Space theorem from earlier in the course defines the operator norm and establishes that it is a norm on the space of bounded linear maps. Here we examine how to interpret and calculate that norm.

Definition: The operator norm of $T$ is $$ \|T\|=\sup\left\{\|T(x)\|_W:x\in V,\ \|x\|_V\leq1\right\}. $$ The set over which the supremum is taken is the closed unit ball of $V$. Since $T$ is bounded, the supremum is finite.

The unit ball formulation asks for the largest output size among inputs whose size is at most one. It includes the zero vector, whose output is zero. If $V$ consists only of the zero vector, the set of output sizes is simply $\{0\}$, so the definition gives $\|T\|=0$.

The Unit-Ball and Ratio Formulas

When $V$ contains a nonzero vector, the same quantity can be computed by taking a ratio over all nonzero inputs. This form is useful when a convenient input is not already normalized.

Theorem (Unit-Ball and Ratio Formulas): If $V\neq\{0_V\}$, then $$ \|T\| =\sup_{\substack{x\in V\\ \|x\|_V\leq1}}\|T(x)\|_W =\sup_{\substack{x\in V\\x\neq0_V}}\frac{\|T(x)\|_W}{\|x\|_V} =\sup_{\substack{x\in V\\ \|x\|_V=1}}\|T(x)\|_W. $$

Proof. Write $A$ for the supremum over the closed unit ball. If $x\neq0_V$, then $u=x/\|x\|_V$ has norm one. Linearity and absolute homogeneity of the norms give $$ \|T(u)\|_W =\left\|T\left(\frac{x}{\|x\|_V}\right)\right\|_W =\frac{\|T(x)\|_W}{\|x\|_V}. $$ Since $u$ belongs to the unit ball, the ratio is at most $A$. Thus the supremum of the ratios is at most $A$. Conversely, if $u$ has norm one, its ratio is $\|T(u)\|_W$, so the supremum of the ratios is at least the supremum over the unit sphere.

It remains to compare the unit ball with the unit sphere. If $x$ belongs to the unit ball and $x\neq0_V$, then $$ \|T(x)\|_W =\|x\|_V\left\|T\left(\frac{x}{\|x\|_V}\right)\right\|_W \leq \left\|T\left(\frac{x}{\|x\|_V}\right)\right\|_W. $$ The normalized vector is on the unit sphere. For $x=0_V$, the output norm is zero. Therefore the supremum over the ball is no greater than the supremum over the sphere. The reverse inequality holds because the sphere is contained in the ball. These comparisons prove all the equalities. $\square$

The earlier Theorem (The Operator Norm Is the Least Uniform Bound) states that $\|T\|$ is the least constant $C$ such that $\|T(x)\|_W\leq C\|x\|_V$ for every $x\in V$. The ratio formula explains this characterization directly: each nonzero input tests the required constant by producing one ratio, and the supremum collects all those tests.

Computing an Operator Norm

A standard calculation has two parts. First, prove an upper bound that applies to every input in the unit ball. Then find an input that reaches the bound, or a sequence of inputs whose output sizes approach it. If an input of norm one reaches the upper bound, the operator norm is determined exactly.

Find an upper bound.
Estimate $\|T(x)\|_W$ in terms of $\|x\|_V$ for an arbitrary input.

Test the bound.
Look for a nonzero input where equality holds, or inputs whose ratios approach the proposed bound.

Conclude the value.
An upper bound and a matching value or limiting sequence identify the supremum.

Worked Example: A Matrix Between Supremum-Norm Spaces

Give $\mathbb{R}^2$ the supremum norm $\|(x,y)\|_\infty=\max\{|x|,|y|\}$ in both domain and codomain, and define $$ T(x,y)=(x+y,x-y). $$ For every $(x,y)$, $$ \|T(x,y)\|_\infty =\max\{|x+y|,|x-y|\} \leq |x|+|y| \leq2\max\{|x|,|y|\} =2\|(x,y)\|_\infty. $$ Thus the ratio formula gives $\|T\|\leq2$. For the input $(1,1)$, the input norm is $1$, while $$ T(1,1)=(2,0),\qquad \|T(1,1)\|_\infty=2. $$ The ratio at this input is $2/1=2$, so $\|T\|\geq2$. Combining the two bounds gives $\|T\|=2$.

Worked Example: An Averaging Functional

On $C[0,2]$ with the supremum norm, define $T:C[0,2]\to\mathbb{R}$ by $$ T(f)=\frac12\int_0^2 f(x)\,dx, $$ where $\mathbb{R}$ has its usual absolute-value norm. Linearity follows from linearity of the Riemann integral. For every $f\in C[0,2]$, the bound $|f(x)|\leq\|f\|_\infty$ gives $$ |T(f)| \leq\frac12\int_0^2|f(x)|\,dx \leq\frac12\int_0^2\|f\|_\infty\,dx =\|f\|_\infty. $$ Hence $\|T\|\leq1$. For the constant function $f(x)=1$, its supremum norm is $1$, and $$ T(f)=\frac12\int_0^2 1\,dx=1. $$ The ratio for this input is $1$, so $\|T\|\geq1$. Therefore $\|T\|=1$.

Worked Example: Differentiation on a Polynomial Space

Let $V=\mathcal{P}_2$, the space of polynomials of degree at most two, with norm $$ \|a_0+a_1x+a_2x^2\|_V=|a_0|+|a_1|+|a_2|. $$ Let $W=\mathcal{P}_1$ with the same coefficient-sum norm, and define $D:V\to W$ by $D(p)=p'$. This map is linear. If $p(x)=a_0+a_1x+a_2x^2$, then $$ \|D(p)\|_W=|a_1|+2|a_2| \leq2(|a_0|+|a_1|+|a_2|) =2\|p\|_V. $$ Thus $\|D\|\leq2$. For $p(x)=x^2$, the domain norm is $1$, and $D(p)=2x$ has $W$-norm $2$. Consequently, the ratio is $2$, which proves $\|D\|=2$. This calculation depends on the specified coefficient norms; the algebraic operation of differentiation alone does not determine an operator norm.

When Is the Supremum Attained?

A supremum need not be the value at any single input. In finite-dimensional spaces, compactness rules out this difficulty for the unit sphere. More generally, the same conclusion holds whenever the unit sphere of the domain is compact.

Theorem (Attainment on a Compact Unit Sphere): Suppose $V\neq\{0_V\}$ and the unit sphere $\{x\in V:\|x\|_V=1\}$ is compact. Then there is a vector $x_0\in V$ with $\|x_0\|_V=1$ such that $$ \|T\|=\|T(x_0)\|_W. $$

Proof. Define $\phi(x)=\|T(x)\|_W$ on the unit sphere. The least-uniform-bound property gives $$ \|T(x)-T(y)\|_W=\|T(x-y)\|_W\leq\|T\|\|x-y\|_V $$ for all $x,y\in V$. By the reverse triangle inequality for norms, $$ |\phi(x)-\phi(y)| \leq\|T(x)-T(y)\|_W \leq\|T\|\|x-y\|_V. $$ Thus $\phi$ is continuous. Because the unit sphere is compact and $\phi$ is continuous, its image under $\phi$ is a compact subset of $\mathbb{R}$. The Extreme-Value Theorem for compact subsets of $\mathbb{R}$ gives a point $x_0$ on the unit sphere where $\phi$ has a maximum. By the unit-ball and ratio formulas, the supremum of $\phi$ on the unit sphere equals $\|T\|$; hence $\|T(x_0)\|_W=\|T\|$. The finite-dimensional case is treated after finite-dimensional spaces are studied. $\square$

Compactness is a sufficient condition, not a feature of the definition itself. In spaces with noncompact unit spheres, a sequence of inputs may approach the operator norm without any individual input achieving it.

Worked Example: A Supremum That Is Not Attained

Let $V=c_{00}$ be the vector space of real sequences with only finitely many nonzero entries, equipped with the supremum norm $\|x\|_\infty=\sup_{n\geq1}|x_n|$. Define $T:V\to\mathbb{R}$ by $$ T(x)=\sum_{n=1}^{\infty}2^{-n}x_n. $$ The sum is finite for each $x\in c_{00}$, and the formula defines a linear map. If $\|x\|_\infty\leq1$, then $$ |T(x)| \leq\sum_{\{n:x_n\neq0\}}2^{-n}|x_n| \leq\sum_{\{n:x_n\neq0\}}2^{-n} <\sum_{n=1}^{\infty}2^{-n} =1. $$ The strict inequality holds because the support of $x$ is finite, so at least one positive term from the infinite geometric series is omitted. This proves $\|T\|\leq1$, and also shows that no input in the unit ball has output norm $1$.

For each positive integer $N$, let $x^{(N)}$ have its first $N$ entries equal to $1$ and all later entries equal to $0$. Then $\|x^{(N)}\|_\infty=1$, and $$ T(x^{(N)})=\sum_{n=1}^N2^{-n}=1-2^{-N}. $$ As $N$ tends to infinity, these values tend to $1$. Therefore the supremum is at least $1$, and together with the upper bound this gives $\|T\|=1$. The norm is not attained by any unit-ball input.

What the Operator Norm Depends On

An operator norm is not determined by a formula for $T$ in isolation. It depends on both the domain norm and the codomain norm, because these norms determine which inputs count as having size at most one and how output size is measured. Even for a fixed linear map, changing either norm can change the supremum.

It is also important to distinguish a supremum from a maximum. The definition always gives a finite supremum for a bounded map, but it does not promise an input where that supremum is reached. The compact-unit-sphere theorem supplies one useful condition guaranteeing attainment; the $c_{00}$ example shows why that condition cannot simply be omitted from an attainment claim.

Takeaway: The operator norm is the supremum of output sizes on the closed unit ball, equivalently the supremum of output-to-input ratios over nonzero inputs. Exact calculations require both a universal upper bound and a matching input or approximating sequence.

Check Your Understanding

Use the definitions and results in this tutorial to answer the following questions.

Why does normalizing a nonzero input lead to the ratio formula for the operator norm?
For the map $T(x,y)=(x+y,x-y)$ with supremum norms, which input proves that the upper bound $2$ is sharp?
What compactness assumption ensures that a bounded linear map attains its operator norm on the unit sphere?
In the $c_{00}$ example, why does each individual unit-ball input have output strictly less than $1$, even though the operator norm is $1$?
Which two choices must be specified, in addition to the linear map, to determine its operator norm?

What You'll Learn