Deriving Projection by Minimizing Distance

Introduction

A projection onto a line answers a nearest-point question: among all vectors pointing along a chosen direction, which one is closest to a given vector? The answer can be derived by measuring the distance to a general candidate on the line and choosing the candidate that makes that distance as small as possible.

This approach connects the projection formula to a basic optimization problem. The unknown is a single real number: the scalar that multiplies the direction vector. Once the distance is written in terms of that scalar, the problem becomes minimizing a quadratic function.

Set Up the Distance Problem

Let \(\mathbf{a}\) be the vector we want to approximate, and let \(\mathbf{v}\) be a nonzero direction vector. The line through the origin with direction \(\mathbf{v}\) consists of all vectors \(t\mathbf{v}\), where \(t\) is a real number. The distance from \(\mathbf{a}\) to a particular candidate \(t\mathbf{v}\) is \(\|\mathbf{a}-t\mathbf{v}\|\).

It is more convenient to minimize the squared distance. Since distance is nonnegative and squaring preserves the order of nonnegative numbers, minimizing the distance and minimizing its square give the same candidate.

The condition \(\mathbf{v}\ne\mathbf{0}\) matters. It ensures that \(\mathbf{v}\cdot\mathbf{v}>0\), so the line has a genuine direction and the quadratic distance function has a positive coefficient on \(t^2\).

Deriving the Minimizing Scalar

Use the dot product identity \(\|\mathbf{x}\|^2=\mathbf{x}\cdot\mathbf{x}\), together with linearity and symmetry of the dot product, to expand \(D(t)\):

Because \(\mathbf{v}\ne\mathbf{0}\), the number \(\mathbf{v}\cdot\mathbf{v}\) is positive. We can therefore complete the square, keeping track of the part that does not depend on \(t\):

The squared term is always nonnegative, and its coefficient is positive. It is zero at exactly one value of \(t\). Thus this value gives the unique global minimum, not merely a candidate for a minimum.

The scalar \(t_*\) specifies how much of \(\mathbf{v}\) to take. The vector projection is the scalar multiplied by \(\mathbf{v}\). These are different kinds of quantities: \(t_*\) is a number, while \(\operatorname{proj}_{\mathbf{v}}\mathbf{a}\) is a vector.

Worked Examples

Why Rescaling Does Not Change the Projection

The last example illustrates a general fact. Replacing a nonzero direction vector \(\mathbf{v}\) by \(c\mathbf{v}\), with \(c\ne0\), changes its scale but not the line of its multiples. The formula confirms that the projected vector is unchanged:

The scalar that describes the multiple does change when the direction vector is rescaled; the closest vector itself does not. This distinction is useful when a direction is given in a convenient but non-unit form.

A Common Denominator Error

A frequent mistake is to divide by \(\|\mathbf{v}\|\) rather than by \(\mathbf{v}\cdot\mathbf{v}=\|\mathbf{v}\|^2\). For the first example, \(\mathbf{a}\cdot\mathbf{v}=11\) and \(\|\mathbf{v}\|=3\). Using \(11/3\) as the coefficient would give the incorrect candidate \((11/3,22/3,22/3)\).

The error can be detected by checking the distance function from that example. Its derivative is \(D'(t)=18t-22\); at the incorrect value \(t=11/3\), this is \(18(11/3)-22=66-22=44\), not zero. More directly, the completed-square form \(D(t)=9(t-11/9)^2+104/9\) shows that \(11/3\) is not the minimizing scalar.

The correction is to use the squared length in the denominator: \(\mathbf{v}\cdot\mathbf{v}=9\), so the coefficient is \(11/9\). The denominator is not an arbitrary adjustment. It is the positive quadratic coefficient that appears when the squared distance is expanded and completed to a square.

When the Derivation Applies

This minimization considers every real multiple \(t\mathbf{v}\), so it applies to a line through the origin with nonzero direction vector \(\mathbf{v}\). The formula requires \(\mathbf{v}\ne\mathbf{0}\); if \(\mathbf{v}=\mathbf{0}\), its denominator is zero and there is no direction line of this form.

The key method is to express the squared distance as a function of the scalar, expand using dot products, and complete the square. The resulting minimum is unique because the coefficient \(\mathbf{v}\cdot\mathbf{v}\) is strictly positive. This gives both the projection formula and a direct way to verify a proposed answer: its scalar must make the square term vanish.