Distances From Inner Product Norms

From Norm to Distance

A norm measures the size of one vector, while a distance compares two vectors. The connection is direct: subtract one vector from the other, then measure the resulting vector with the norm. This definition works in coordinate spaces, matrix spaces, and function spaces, provided the norm comes from an inner product. The calculation may look different in each setting, but the same principle applies.

Let \(V\) be a vector space over \(\mathbb{F}\), where \(\mathbb{F}=\mathbb{R}\) or \(\mathbb{C}\), equipped with an inner product and its induced norm \(\|\mathbf{x}\|=\sqrt{\langle\mathbf{x},\mathbf{x}\rangle}\). The norm is nonnegative, equals zero only for the zero vector, is absolutely homogeneous, and satisfies the triangle inequality. These properties were established for inner-product-induced norms earlier in the material on norms.

Subtraction matters: \(\mathbf{x}-\mathbf{y}\) is the vector that points from \(\mathbf{y}\) to \(\mathbf{x}\), and its norm is nonnegative regardless of which vector comes first. In coordinate examples, the vectors are columns. In other spaces, the same definition uses the subtraction operation and norm appropriate to that space.

Basic Properties of Inner-Product Distance

A distance function on \(V\) is a metric if it is nonnegative, is zero exactly when its inputs are equal, is symmetric, and satisfies the triangle inequality. The norm properties give all four facts for the distance just defined.

Proof. Since norms are nonnegative, \(d(\mathbf{x},\mathbf{y})=\|\mathbf{x}-\mathbf{y}\|\geq 0\). By the definiteness of the norm, \(d(\mathbf{x},\mathbf{y})=0\) exactly when \(\mathbf{x}-\mathbf{y}=\mathbf{0}\), which is equivalent to \(\mathbf{x}=\mathbf{y}\). Also, \(\mathbf{y}-\mathbf{x}=-(\mathbf{x}-\mathbf{y})\), so absolute homogeneity of the norm gives \(d(\mathbf{y},\mathbf{x})=\|-(\mathbf{x}-\mathbf{y})\|=d(\mathbf{x},\mathbf{y})\). For the triangle inequality, write \(\mathbf{x}-\mathbf{z}=(\mathbf{x}-\mathbf{y})+(\mathbf{y}-\mathbf{z})\). Then the norm triangle inequality gives \(d(\mathbf{x},\mathbf{z})\leq d(\mathbf{x},\mathbf{y})+d(\mathbf{y},\mathbf{z})\). For translation, \((\mathbf{x}+\mathbf{z})-(\mathbf{y}+\mathbf{z})=\mathbf{x}-\mathbf{y}\), so the distances are equal. For scaling, \(c\mathbf{x}-c\mathbf{y}=c(\mathbf{x}-\mathbf{y})\), and absolute homogeneity gives \(d(c\mathbf{x},c\mathbf{y})=|c|\|\mathbf{x}-\mathbf{y}\|=|c|d(\mathbf{x},\mathbf{y})\). These arguments also apply when \(c=0\), since both sides are then zero. \(\square\)

The translation property says that moving both vectors by the same vector leaves their separation unchanged. Scaling both vectors by \(c\) scales the distance by \(|c|\). If \(c=0\), the two scaled vectors coincide, so their distance is zero, as the formula states.

Squared Distance and the Inner Product

Sometimes it is convenient to compute the square of a distance first. With the course convention that a complex inner product is linear in its first slot and conjugate-linear in its second, expanding the inner product gives the following identity.

Proof. By the definition of the induced norm and additivity in each slot, \(\|\mathbf{x}-\mathbf{y}\|^2=\langle\mathbf{x}-\mathbf{y},\mathbf{x}-\mathbf{y}\rangle=\langle\mathbf{x},\mathbf{x}\rangle-\langle\mathbf{x},\mathbf{y}\rangle-\langle\mathbf{y},\mathbf{x}\rangle+\langle\mathbf{y},\mathbf{y}\rangle\). The diagonal terms are \(\|\mathbf{x}\|^2\) and \(\|\mathbf{y}\|^2\). Conjugate symmetry gives \(\langle\mathbf{y},\mathbf{x}\rangle=\overline{\langle\mathbf{x},\mathbf{y}\rangle}\), so the two cross terms sum to \(2\operatorname{Re}\langle\mathbf{x},\mathbf{y}\rangle\). Substituting yields the identity. \(\square\)

In real coordinate spaces, it is often simplest to subtract the vectors first and sum the squares of the resulting entries. The identity can instead be useful when the inner products and norms of the original vectors are already known. In either approach, the squared distance must be nonnegative, and the distance itself is the nonnegative square root.

Euclidean and Weighted Coordinate Distances

A weighted inner product assigns different positive weights to different coordinates. For example, with positive real numbers \(w_1,\dots,w_n\), the formula \(\langle\mathbf{x},\mathbf{y}\rangle_w=\sum_{i=1}^n w_i x_i y_i\) defines an inner product on \(\mathbb{R}^n\). Its induced squared distance is \(\sum_{i=1}^n w_i(x_i-y_i)^2\). Positivity of every weight matters: if a weight were zero, a nonzero vector supported only in that coordinate would have norm zero.

Distances Between Matrices

For real \(m\times n\) matrices, the Frobenius inner product is the sum of the products of corresponding entries: \(\langle\mathbf{X},\mathbf{Y}\rangle_F=\sum_{i=1}^m\sum_{j=1}^n x_{ij}y_{ij}\). Its induced norm is the Frobenius norm, \(\|\mathbf{X}\|_F=\sqrt{\sum_{i,j}x_{ij}^2}\). Consequently, the Frobenius distance is the square root of the sum of the squared entrywise differences. The matrices must have the same dimensions so that their difference is defined.

Distances Between Continuous Functions

For real-valued continuous functions on an interval \([a,b]\) with \(a<b\), the integral inner product is \(\langle f,g\rangle=\int_a^b f(t)g(t)\,dt\). The restriction \(a<b\) is essential here. It ensures that a continuous function with zero integral of its square must be identically zero on the interval, making this a positive-definite inner product. The induced distance is

To see why definiteness holds, suppose a continuous function \(h\) is nonzero at some point of \([a,b]\). Continuity then ensures that \(|h|\) is bounded below by some positive number on a small interval around that point, using a one-sided interval if the point is an endpoint. Since \(a<b\), this interval can be chosen to have positive length. Thus \(\int_a^b h(t)^2\,dt>0\). So an integral of zero forces \(h=0\) everywhere on \([a,b]\).

Common Pitfall: Subtracting the Norms

A common error is to compute \(\big|\|\mathbf{x}\|-\|\mathbf{y}\|\big|\) and call it the distance. That quantity can be a lower bound for distance, but it is generally not the distance itself. For a concrete example in \(\mathbb{R}^2\), take \(\mathbf{x}=(3,0)^{\top}\) and \(\mathbf{y}=(0,4)^{\top}\). The incorrect calculation gives \(|3-4|=1\). The correct calculation is

The correction is to subtract the vectors first and then take the norm. The two answers differ because the norm of the difference depends on the vectors themselves, not only on their separate lengths.

The same definition covers each setting in this tutorial: subtract the two objects in their space, use that space’s inner-product norm, and take the nonnegative result. In weighted coordinates, the weights affect the calculation; for matrices, each entry contributes; for functions, the squared difference is integrated over an interval with \(a<b\). In each case, equality of the two objects is exactly the condition for distance zero.