Proving the Steepest-Ascent Direction

Why a Direction of Steepest Change Needs a Proof

At a point in a scalar field, the gradient summarizes the field’s first-order change in every direction. The directional derivative tells us how quickly the field changes when we move along a specified unit direction. These facts suggest that the gradient points toward the steepest increase, but the suggestion needs justification: among all possible unit directions, why should that particular one give the greatest increase?

The Cauchy–Schwarz inequality provides the answer. It bounds the dot product of the gradient and any unit direction, and its equality condition identifies exactly when that bound is attained. The result is local: it describes instantaneous change at one point, not what happens along an entire path or where a function reaches a global maximum.

Definition: Let $f$ be differentiable at a point $\mathbf{p}$ in its domain, with gradient $\nabla f(\mathbf{p})$. For a unit vector $\mathbf{u}$, the directional derivative of $f$ at $\mathbf{p}$ in direction $\mathbf{u}$ is $$ D_{\mathbf{u}}f(\mathbf{p})=\nabla f(\mathbf{p})\cdot\mathbf{u}. $$ Here, unit means $\|\mathbf{u}\|=1$.

The formula for the directional derivative follows from the chain rule applied to the line $\mathbf{p}+t\mathbf{u}$. We use it here as the starting point. The condition that $\mathbf{u}$ is a unit vector matters: it means the rate compares directions for the same unit distance of travel.

The Cauchy–Schwarz Bound

For vectors $\mathbf{a}$ and $\mathbf{b}$, the Cauchy–Schwarz inequality states that

|\mathbf{a}\cdot\mathbf{b}| \leq \|\mathbf{a}\|\,\|\mathbf{b}\|.

Apply it with $\mathbf{a}=\nabla f(\mathbf{p})$ and $\mathbf{b}=\mathbf{u}$. Since $\mathbf{u}$ is a unit vector,

\left|D_{\mathbf{u}}f(\mathbf{p})\right| = \left|\nabla f(\mathbf{p})\cdot\mathbf{u}\right| \leq \|\nabla f(\mathbf{p})\|.

Thus no unit direction can have a directional derivative greater than the length of the gradient, or less than the negative of that length. To identify the directions that attain these bounds, we need the equality condition, not just the inequality.

Theorem: Suppose $f$ is differentiable at $\mathbf{p}$ and $\nabla f(\mathbf{p})\neq\mathbf{0}$. Among unit directions, the directional derivative is greatest in the direction $$ \mathbf{u}_{\text{up}} = \frac{\nabla f(\mathbf{p})}{\|\nabla f(\mathbf{p})\|}, $$ and least in the direction $$ \mathbf{u}_{\text{down}} = -\frac{\nabla f(\mathbf{p})}{\|\nabla f(\mathbf{p})\|}. $$ The corresponding directional derivatives are $\|\nabla f(\mathbf{p})\|$ and $-\|\nabla f(\mathbf{p})\|$, respectively.

Why the Gradient Direction Attains the Bound

The upper bound follows from Cauchy–Schwarz, but we must also show that the proposed direction is a unit vector and actually attains the bound. Nonzero gradient is required so that dividing by its length is defined.

Normalize the gradient.
Because $\nabla f(\mathbf{p})\neq\mathbf{0}$, its length is positive. Therefore $\mathbf{u}_{\text{up}}=\nabla f(\mathbf{p})/\|\nabla f(\mathbf{p})\|$ is defined, and its length is $1$.

Evaluate the directional derivative.
Substitute this unit direction into the dot-product formula: $D_{\mathbf{u}_{\text{up}}}f(\mathbf{p})=\nabla f(\mathbf{p})\cdot\nabla f(\mathbf{p})/\|\nabla f(\mathbf{p})\|$.

Compare with every other unit direction.
Cauchy–Schwarz bounds every such derivative above by $\|\nabla f(\mathbf{p})\|$. The proposed direction attains that bound, so no other unit direction gives a greater first-order increase.

For completeness, the value in Step 2 simplifies because $\nabla f(\mathbf{p})\cdot\nabla f(\mathbf{p})=\|\nabla f(\mathbf{p})\|^2$. Hence

D_{\mathbf{u}_{\text{up}}}f(\mathbf{p}) = \frac{\|\nabla f(\mathbf{p})\|^2}{\|\nabla f(\mathbf{p})\|} = \|\nabla f(\mathbf{p})\|.

For the proposed downward direction, the same calculation gives

D_{\mathbf{u}_{\text{down}}}f(\mathbf{p}) = \nabla f(\mathbf{p})\cdot \left(-\frac{\nabla f(\mathbf{p})}{\|\nabla f(\mathbf{p})\|}\right) = -\|\nabla f(\mathbf{p})\|.

Cauchy–Schwarz also gives $D_{\mathbf{u}}f(\mathbf{p})\geq-\|\nabla f(\mathbf{p})\|$ for every unit $\mathbf{u}$. The negative normalized gradient attains this lower bound, proving the steepest-decrease claim as well.

Equality and the Meaning of “Direction”

When the gradient is nonzero, equality in Cauchy–Schwarz occurs when the two vectors are parallel. For the largest positive dot product, $\mathbf{u}$ must point in the same direction as the gradient; for the most negative dot product, it must point in the opposite direction. Requiring $\|\mathbf{u}\|=1$ fixes the scaling and gives the normalized vectors in the theorem.

This distinction between a vector and a direction is important. The gradient itself is generally not a unit vector. It contains both directional information and magnitude information: its orientation identifies the steepest-ascent direction, while its length gives the size of the greatest first-order rate.

Worked Example: Steepest Ascent in Two Variables

Let $f(x,y)=x^2+2xy+3y^2$, and consider the point $\mathbf{p}=(1,0)$. First compute the gradient:

\nabla f(x,y) = \left(2x+2y,\;2x+6y\right), \qquad \nabla f(1,0)=(2,2).

Its length is $\sqrt{2^2+2^2}=\sqrt{8}=2\sqrt{2}$. Thus the unit direction of steepest ascent is

\mathbf{u}_{\text{up}} = \frac{(2,2)}{2\sqrt{2}} = \left(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}}\right).

Substitution into the directional derivative formula gives

D_{\mathbf{u}_{\text{up}}}f(1,0) = (2,2)\cdot \left(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}}\right) = \frac{2}{\sqrt{2}}+\frac{2}{\sqrt{2}} = 2\sqrt{2}.

Check the result directly along the line in this direction. At parameter $t$, the point is $(1+t/\sqrt{2},t/\sqrt{2})$. Writing $a=t/\sqrt{2}$, the function along the line is $f(1+a,a)=(1+2a+a^2)+2(a+a^2)+3a^2=1+4a+6a^2=1+2\sqrt{2}t+3t^2$. Its derivative at $t=0$ is $2\sqrt{2}$, as computed. As a comparison, the unit direction $(1,0)$ gives rate $(2,2)\cdot(1,0)=2$, which is smaller.

Worked Example: Steepest Ascent and Descent in Two Variables

Let $f(x,y)=3x-4y+x^2$ at $\mathbf{p}=(0,0)$. The gradient and its length are

\nabla f(x,y)=(3+2x,-4), \qquad \nabla f(0,0)=(3,-4), \qquad \|\nabla f(0,0)\|=\sqrt{3^2+(-4)^2}=5.

Therefore the unit steepest-ascent direction is $(3/5,-4/5)$. Its directional derivative is

(3,-4)\cdot\left(\frac35,-\frac45\right) = \frac95+\frac{16}{5} = 5.

The unit steepest-descent direction is $(-3/5,4/5)$, and its derivative is $(3,-4)\cdot(-3/5,4/5)=-9/5-16/5=-5$.

Check the ascent rate along the line $(x,y)=(3t/5,-4t/5)$. Substitution gives $f(3t/5,-4t/5)=9t/5+16t/5+9t^2/25=5t+9t^2/25$. Differentiating with respect to $t$ and setting $t=0$ gives $5$, agreeing with the dot product. The descent direction gives the negative of this first-order rate.

Worked Example: A Steepest-Ascent Direction in Three Variables

Consider $f(x,y,z)=x^2-y+z^2$ at $\mathbf{p}=(1,0,1)$. Its gradient is $\nabla f(x,y,z)=(2x,-1,2z)$, so $\nabla f(1,0,1)=(2,-1,2)$. The length is $\sqrt{2^2+(-1)^2+2^2}=\sqrt{9}=3$. Normalizing gives

\mathbf{u}_{\text{up}} = \left(\frac23,-\frac13,\frac23\right).

The directional derivative is $(2,-1,2)\cdot(2/3,-1/3,2/3)=4/3+1/3+4/3=3$. To check along the corresponding line, substitute $(x,y,z)=(1+2t/3,-t/3,1+2t/3)$:

f\left(1+\frac{2t}{3},-\frac{t}{3},1+\frac{2t}{3}\right) = \left(1+\frac{4t}{3}+\frac{4t^2}{9}\right) +\frac{t}{3} +\left(1+\frac{4t}{3}+\frac{4t^2}{9}\right) = 2+3t+\frac{8t^2}{9}.

The derivative at $t=0$ is $3$, which verifies the result.

When the Gradient Is Zero

The theorem’s nonzero-gradient assumption cannot be omitted. If $\nabla f(\mathbf{p})=\mathbf{0}$, then for every unit vector $\mathbf{u}$,

D_{\mathbf{u}}f(\mathbf{p}) = \nabla f(\mathbf{p})\cdot\mathbf{u} = \mathbf{0}\cdot\mathbf{u} = 0.

Consequently, the first-order directional derivatives do not select one unit direction as the steepest ascent direction. This does not say that the function is constant near the point; it says only that its directional derivatives there are all zero. Also, $\nabla f(\mathbf{p})/\|\nabla f(\mathbf{p})\|$ is undefined when the gradient is zero, so the normalization argument cannot be used.

Common Pitfall: Treating the Gradient as a Unit Vector

At the origin in the second worked example, the gradient is $(3,-4)$, whose length is $5$, not $1$. A mistaken calculation might call $(3,-4)$ a unit direction and write

(3,-4)\cdot(3,-4)=25 \quad\text{and conclude that the directional rate is }25.

The dot product is correct, but the conclusion is not: $(3,-4)$ is not a unit direction. It represents a displacement of length $5$. Normalize it first: $(3,-4)/5=(3/5,-4/5)$. The directional derivative for this unit direction is then $(3,-4)\cdot(3/5,-4/5)=9/5+16/5=5$.

A related sign error is to use the negative normalized gradient for ascent. In this same example, the negative direction gives rate $-5$, so it is the direction of steepest decrease, not steepest increase.

Why This Result Matters

The proof turns a geometric intuition into a precise statement: the gradient’s orientation identifies a unit direction, and Cauchy–Schwarz proves that this direction attains the largest possible first-order increase. The opposite unit direction attains the largest first-order decrease. This is useful whenever a scalar field represents a quantity that changes with position, such as temperature or elevation, and the question concerns its immediate change at a particular point.

The conclusion is about local rates at that point. It does not guarantee that following the ascent direction for a substantial distance will continue to increase the function, because the gradient can change from point to point. Keeping this local interpretation, and the unit-length condition, clear is essential when applying the result.

Check Your Understanding

Use the dot-product formula and the equality condition in Cauchy–Schwarz to answer these questions.

Why must the direction vector be a unit vector when comparing directional rates?
If $\nabla f(\mathbf{p})=(6,8)$, what is the unit direction of steepest ascent?
For that gradient, what is the unit direction of steepest decrease, and what directional derivative does it produce?
What feature of Cauchy–Schwarz shows that the normalized gradient attains the greatest directional derivative?
What does a zero gradient imply about directional derivatives at that point, and what does it not imply about nearby function values?