Norms

From a Function Norm to a General Norm

The supremum norm measures the size of a bounded function by its largest absolute value, or by the least uniform bound when that value is not attained. The same idea applies to vectors and to elements of many other vector spaces: a norm assigns a nonnegative size to each element, in a way that interacts predictably with addition and scalar multiplication. This tutorial introduces the general definition and compares several norms on \(\mathbb{R}^n\).

The three conditions encode distinct requirements. The first ensures that only the zero vector has size zero. The second says that scaling a vector by \(a\) scales its size by \(|a|\). The third, the triangle inequality, says that combining two vectors cannot produce a size larger than the sum of their sizes. The next tutorial examines these norm axioms in detail. For now, they provide a common definition with which to describe familiar examples.

Three Norms on Coordinate Space

For \(x=(x_1,\ldots,x_n)\in\mathbb{R}^n\), three standard choices are the one-norm, the Euclidean norm, and the maximum norm. They summarize the same coordinates in different ways: by adding absolute values, by taking the square root of the sum of squares, or by taking the largest absolute coordinate.

Each formula satisfies the norm conditions. For the one-norm, positivity and homogeneity follow coordinate by coordinate, and \(|x_i+y_i|\leq |x_i|+|y_i|\) gives the triangle inequality after summing. For the maximum norm, the coordinate inequalities \(|x_i+y_i|\leq |x_i|+|y_i|\leq\|x\|_\infty+\|y\|_\infty\) give its triangle inequality; positivity and homogeneity follow directly from the maximum formula.

For the Euclidean norm, positivity and homogeneity are also immediate from its formula. Its triangle inequality follows from the Cauchy–Schwarz inequality. Indeed, expanding the square gives $$ \|x+y\|_2^2 =\|x\|_2^2+2\sum_{i=1}^n x_i y_i+\|y\|_2^2 \leq \|x\|_2^2+2\|x\|_2\|y\|_2+\|y\|_2^2 =(\|x\|_2+\|y\|_2)^2. $$ Both sides before taking square roots are nonnegative, so \(\|x+y\|_2\leq\|x\|_2+\|y\|_2\). The Cauchy–Schwarz inequality used here is \(\left|\sum_i x_i y_i\right|\leq\|x\|_2\|y\|_2\).

The Supremum Norm as One Instance

The general definition also explains why the supremum norm fits naturally into the study of function spaces. The bounded functions \(B(E)\) form a real vector space under pointwise addition and scalar multiplication. For a nonempty domain \(E\), the supremum norm \(\|f\|_\infty=\sup_{x\in E}|f(x)|\) is a norm on that space, as established in The Supremum Norm Is a Norm Theorem. The word “norm” therefore applies both to finite-dimensional vectors and to bounded functions; the underlying vector space and the chosen size function determine the setting.

Different Norms, Comparable Measurements

Different norms need not assign the same number to a vector, but on \(\mathbb{R}^n\) the standard norms can be bounded in terms of one another. These estimates are useful because they let a bound measured in one norm be translated into a bound in another.

Proof. Each squared coordinate \(x_i^2\) is at most \(\sum_j x_j^2\), so \(|x_i|\leq\|x\|_2\) for every \(i\). Taking the maximum gives \(\|x\|_\infty\leq\|x\|_2\). Also, $$ \|x\|_2^2=\sum_i |x_i|^2\leq\left(\sum_i|x_i|\right)^2=\|x\|_1^2, $$ because the square on the right expands to the sum of the squares plus nonnegative cross terms. Taking square roots gives \(\|x\|_2\leq\|x\|_1\). By the Cauchy–Schwarz inequality applied to \((|x_1|,\ldots,|x_n|)\) and \((1,\ldots,1)\), $$ \|x\|_1\leq\sqrt{\sum_i|x_i|^2}\sqrt{\sum_i1^2} =\sqrt{n}\,\|x\|_2. $$ Finally, since every \(|x_i|\leq\|x\|_\infty\), $$ \|x\|_2^2=\sum_i|x_i|^2\leq n\|x\|_\infty^2, $$ so taking square roots gives \(\|x\|_2\leq\sqrt{n}\,\|x\|_\infty\). All the inequalities hold also when \(x=0\). \(\square\)

In particular, the estimates show that each of these norms is bounded above and below by positive constant multiples of either of the others. This motivates the general term equivalent norms.

Equivalent norms need not give the same numerical size or the same ball at a chosen radius. Their comparison constants instead ensure that sufficiently small distances in either norm correspond to sufficiently small distances in the other. In finite dimensions, this relationship is not special to the three coordinate formulas: every norm on \(\mathbb{R}^n\) is equivalent to the Euclidean norm.

Proof. Let \(e_1,\ldots,e_n\) be the standard coordinate vectors. Writing \(x=\sum_{i=1}^n x_i e_i\), the triangle inequality and homogeneity give $$ N(x)\leq\sum_{i=1}^n |x_i|N(e_i) \leq \left(\sum_{i=1}^n N(e_i)^2\right)^{1/2}\|x\|_2. $$ The last step is Cauchy–Schwarz. Set \(C=\left(\sum_iN(e_i)^2\right)^{1/2}\), which is finite and positive. Thus \(N(x)\leq C\|x\|_2\).

The triangle inequality also implies the reverse triangle estimate \(|N(x)-N(y)|\leq N(x-y)\): apply the triangle inequality to \(x=(x-y)+y\), and then interchange \(x\) and \(y\). Combining this estimate with the upper bound just proved yields $$ |N(x)-N(y)|\leq C\|x-y\|_2. $$ Therefore \(N\) is continuous with respect to the Euclidean metric. The Euclidean unit sphere \(S=\{x\in\mathbb{R}^n:\|x\|_2=1\}\) is nonempty and compact by the Heine–Borel Theorem. The continuous function \(N\) attains a minimum \(m\) on \(S\). Every point \(u\in S\) is nonzero, so \(N(u)>0\); in particular, at a point where the minimum is attained, \(m>0\).

For any nonzero \(x\), the vector \(u=x/\|x\|_2\) belongs to \(S\). Homogeneity and the definition of \(m\) give $$ N(x)=\|x\|_2N(u)\geq m\|x\|_2. $$ The same inequality holds for \(x=0\). Together with the upper bound, this proves $$ m\|x\|_2\leq N(x)\leq C\|x\|_2 $$ for every \(x\), so \(N\) is equivalent to the Euclidean norm. If \(N_1,N_2\) are any two norms, write \(m_1\|x\|_2\leq N_1(x)\leq C_1\|x\|_2\) and \(m_2\|x\|_2\leq N_2(x)\leq C_2\|x\|_2\). Then $$ \frac{m_2}{C_1}N_1(x)\leq N_2(x)\leq\frac{C_2}{m_1}N_1(x), $$ which proves that \(N_1\) and \(N_2\) are equivalent. \(\square\)

What Equivalence Does—and Does Not—Say

Equivalence of norms guarantees the same convergent sequences. For example, if \(cN_1(x)\leq N_2(x)\leq CN_1(x)\), then \(N_1(x_k-x)\to0\) implies \(N_2(x_k-x)\leq CN_1(x_k-x)\to0\). In the other direction, \(N_1(x_k-x)\leq N_2(x_k-x)/c\), so convergence in \(N_2\) implies convergence in \(N_1\). The same estimates applied to \(x_j-x_k\) show that a sequence is Cauchy in one norm exactly when it is Cauchy in the other.

Equivalence does not mean the norms, their distances, or their balls are identical. For the three standard norms, the differences can be seen at the fixed radius \(1\) in every dimension \(n\geq2\), by using vectors with zero coordinates after the first two. For \(x=(4/5,4/5,0,\ldots,0)\), $$ \|x\|_\infty=\frac45<1,\qquad \|x\|_2=\frac{4\sqrt2}{5}>1,\qquad \|x\|_1=\frac85>1. $$ For \(y=(7/10,7/10,0,\ldots,0)\), $$ \|y\|_2=\frac{7\sqrt2}{10}<1,\qquad \|y\|_1=\frac75>1. $$ Thus each pair of these norms gives different unit-ball membership for some vector when \(n\geq2\). In dimension \(n=1\), however, the one-norm, Euclidean norm, and maximum norm all equal \(|x|\), so their fixed-radius requirements coincide.

The distinction between equivalence and equality is important. Equivalent norms provide the same convergence and Cauchy behavior, but their numerical values and fixed-radius balls can differ. The constants in the comparison tell us how to convert one measurement into another; they do not claim the measurements are interchangeable without adjustment.