Maximum and Minimum Directional Rates

Introduction

At a point in a scalar field, the directional derivative describes the rate of change in a chosen direction. If the direction is allowed to vary over all unit vectors, two natural questions arise: what is the greatest possible rate, and what is the least?

For a differentiable function, both answers are encoded in its gradient. The largest rate is the gradient’s magnitude, attained in the gradient’s direction. The smallest rate is the negative of that magnitude, attained in the opposite direction. The requirement that directions be unit vectors is essential: the rates are measured per unit distance.

Directional Rates and Unit Directions

Let $f:\mathbb{R}^n\to\mathbb{R}$ be differentiable at a point $\mathbf{p}$. A vector $\mathbf{u}$ is a unit direction if its magnitude is $1$. The directional derivative in that direction is the derivative of $f$ as we move from $\mathbf{p}$ along the line with direction $\mathbf{u}$:

D_{\mathbf{u}}f(\mathbf{p}) = \lim_{h\to 0} \frac{f(\mathbf{p}+h\mathbf{u})-f(\mathbf{p})}{h}.

Since $f$ is differentiable at $\mathbf{p}$, the gradient formula applies:

D_{\mathbf{u}}f(\mathbf{p}) = \nabla f(\mathbf{p})\cdot\mathbf{u}, \qquad \|\mathbf{u}\|=1.

Thus, finding the extreme directional rates is a problem about the dot product of the fixed vector $\nabla f(\mathbf{p})$ with unit vectors. Differentiability matters: the conclusions below follow from the gradient formula, so they are not asserted merely because some directional derivatives happen to exist.

Definition: The maximum directional rate at $\mathbf{p}$ is the largest value of $D_{\mathbf{u}}f(\mathbf{p})$ as $\mathbf{u}$ ranges over unit vectors. The minimum directional rate is the smallest such value.

The Extreme-Rate Theorem

Theorem: Suppose $f:\mathbb{R}^n\to\mathbb{R}$ is differentiable at $\mathbf{p}$, and write $\mathbf{g}=\nabla f(\mathbf{p})$. If $\mathbf{g}\ne\mathbf{0}$, then $$ \max_{\|\mathbf{u}\|=1}D_{\mathbf{u}}f(\mathbf{p})=\|\mathbf{g}\|, \qquad \min_{\|\mathbf{u}\|=1}D_{\mathbf{u}}f(\mathbf{p})=-\|\mathbf{g}\|. $$ The maximum is attained at $\mathbf{u}=\mathbf{g}/\|\mathbf{g}\|$, and the minimum at $\mathbf{u}=-\mathbf{g}/\|\mathbf{g}\|$. If $\mathbf{g}=\mathbf{0}$, every unit direction has directional derivative $0$, so both extreme rates are $0$.

The theorem identifies not just the extreme values, but also their attaining directions. Its proof follows from a dot-product bound and its equality conditions.

Why These Are the Extreme Rates

Assume first that $\mathbf{g}\ne\mathbf{0}$, and let $\mathbf{u}$ be any unit vector. Decompose $\mathbf{g}$ into its component along $\mathbf{u}$ and its component perpendicular to $\mathbf{u}$. The squared magnitude of the perpendicular component cannot be negative:

\left\|\mathbf{g}-(\mathbf{g}\cdot\mathbf{u})\mathbf{u}\right\|^2 = \|\mathbf{g}\|^2-(\mathbf{g}\cdot\mathbf{u})^2 \ge 0.

Here the equality uses $\|\mathbf{u}\|=1$. Rearranging gives $(\mathbf{g}\cdot\mathbf{u})^2\le\|\mathbf{g}\|^2$, and therefore

-\|\mathbf{g}\| \le \mathbf{g}\cdot\mathbf{u} \le \|\mathbf{g}\|.

Because $D_{\mathbf{u}}f(\mathbf{p})=\mathbf{g}\cdot\mathbf{u}$, no unit direction can give a rate larger than $\|\mathbf{g}\|$ or smaller than $-\|\mathbf{g}\|$.

Now take $\mathbf{u}=\mathbf{g}/\|\mathbf{g}\|$. This is a unit vector, since its magnitude is $\|\mathbf{g}\|/\|\mathbf{g}\|=1$, and its directional derivative is

\mathbf{g}\cdot\frac{\mathbf{g}}{\|\mathbf{g}\|} = \frac{\|\mathbf{g}\|^2}{\|\mathbf{g}\|} = \|\mathbf{g}\|.

So the upper bound is attained. Taking $\mathbf{u}=-\mathbf{g}/\|\mathbf{g}\|$ gives the rate $-\|\mathbf{g}\|$, so the lower bound is attained as well.

The equality condition also identifies the directions. Equality in the squared-magnitude inequality requires the perpendicular component $\mathbf{g}-(\mathbf{g}\cdot\mathbf{u})\mathbf{u}$ to be zero. Thus $\mathbf{g}$ must be parallel to $\mathbf{u}$. The positive dot product gives $\mathbf{u}=\mathbf{g}/\|\mathbf{g}\|$ for the maximum, and the negative dot product gives $\mathbf{u}=-\mathbf{g}/\|\mathbf{g}\|$ for the minimum.

If instead $\mathbf{g}=\mathbf{0}$, then for every unit vector $\mathbf{u}$, the gradient formula gives $D_{\mathbf{u}}f(\mathbf{p})=\mathbf{0}\cdot\mathbf{u}=0$. Hence every unit direction attains both extreme rates. There is no unique steepest direction in this case.

Worked Examples

Worked Example: A Three-Variable Field

Find the largest and smallest directional derivatives of $f(x,y,z)=x^2-y+\frac{z^2}{2}$ at $(1,0,2)$, and identify the unit directions attaining them.

First compute the gradient:

\nabla f(x,y,z) = \left(2x,-1,z\right), \qquad \nabla f(1,0,2) = (2,-1,2).

Its magnitude is

\|\nabla f(1,0,2)\| = \sqrt{2^2+(-1)^2+2^2} = \sqrt{4+1+4} = 3.

Therefore, the maximum rate is $3$, attained in the unit gradient direction, and the minimum is $-3$, attained in the opposite direction:

\mathbf{u}_{\max} = \frac{(2,-1,2)}{3} = \left(\frac23,-\frac13,\frac23\right), \qquad \mathbf{u}_{\min} = \left(-\frac23,\frac13,-\frac23\right).

Check that the first vector is a unit vector: its squared magnitude is $4/9+1/9+4/9=1$; the negative vector has the same magnitude. Substituting into the gradient formula gives

(2,-1,2)\cdot\left(\frac23,-\frac13,\frac23\right) = \frac43+\frac13+\frac43 = 3, $$ $$ (2,-1,2)\cdot\left(-\frac23,\frac13,-\frac23\right) = -\frac43-\frac13-\frac43 = -3.

As an additional check, move along an arbitrary unit vector $\mathbf{u}=(u_1,u_2,u_3)$ from the point. The value of the function along that line is

f(1+tu_1,tu_2,2+tu_3) = (1+tu_1)^2-tu_2+\frac{(2+tu_3)^2}{2} = 3+t(2u_1-u_2+2u_3)+t^2\left(u_1^2+\frac{u_3^2}{2}\right).

The coefficient of $t$ is the directional derivative, $2u_1-u_2+2u_3=(2,-1,2)\cdot\mathbf{u}$, consistent with the gradient formula. In particular, substituting the two directions above into this coefficient gives $3$ and $-3$.

Worked Example: A Two-Variable Quadratic

For $f(x,y)=x^2+3y^2$, find the extreme directional rates at $(1,-1)$, with the unit directions where they occur.

The gradient and its value at the given point are

\nabla f(x,y)=(2x,6y), \qquad \nabla f(1,-1)=(2,-6).

The magnitude is

\|\nabla f(1,-1)\| = \sqrt{2^2+(-6)^2} = \sqrt{4+36} = \sqrt{40} = 2\sqrt{10}.

Normalize the gradient to get the maximizing direction:

\mathbf{u}_{\max} = \frac{(2,-6)}{2\sqrt{10}} = \left(\frac1{\sqrt{10}},-\frac3{\sqrt{10}}\right).

Its squared magnitude is $1/10+9/10=1$, so it is a unit vector. The minimizing direction is its negative, $\mathbf{u}_{\min}=(-1/\sqrt{10},3/\sqrt{10})$, also of unit magnitude. The theorem gives the extreme rates $2\sqrt{10}$ and $-2\sqrt{10}$. Direct substitution checks them:

(2,-6)\cdot\left(\frac1{\sqrt{10}},-\frac3{\sqrt{10}}\right) = \frac2{\sqrt{10}}+\frac{18}{\sqrt{10}} = \frac{20}{\sqrt{10}} = 2\sqrt{10}, $$ $$ (2,-6)\cdot\left(-\frac1{\sqrt{10}},\frac3{\sqrt{10}}\right) = -\frac2{\sqrt{10}}-\frac{18}{\sqrt{10}} = -2\sqrt{10}.

Worked Example: A Zero Gradient

Find the largest and smallest directional derivatives of $f(x,y)=x^2+y^2$ at the origin, and determine which unit directions attain them.

The gradient is $\nabla f(x,y)=(2x,2y)$, so $\nabla f(0,0)=(0,0)$ and its magnitude is $0$. For every unit vector $\mathbf{u}=(u_1,u_2)$,

D_{\mathbf{u}}f(0,0) = (0,0)\cdot(u_1,u_2) = 0.

Consequently, the maximum and minimum are both $0$, and every unit vector attains both. To check directly, the function along the unit direction is $f(tu_1,tu_2)=t^2(u_1^2+u_2^2)=t^2$, since $u_1^2+u_2^2=1$. Its derivative at $t=0$ is $2t|_{t=0}=0$, as required.

A Common Pitfall: Forgetting to Normalize

The theorem concerns unit directions, not arbitrary vectors. For the gradient $(2,-1,2)$ from the first example, it would be incorrect to use the gradient itself as a direction and report a directional rate of $9$: the calculation $(2,-1,2)\cdot(2,-1,2)=9$ uses a vector of magnitude $3$, not a unit vector.

After normalization, the direction is $(2/3,-1/3,2/3)$, and the rate is $3$, not $9$. The correction is to divide a nonzero direction vector by its magnitude before interpreting its dot product with the gradient as a rate per unit distance. Equivalently, a nonunit vector scales the dot product and therefore does not represent the same unit-distance rate.

Interpreting the Result

The magnitude of the gradient is the size of the steepest first-order change at the point. The gradient’s orientation specifies which way that greatest increase occurs; reversing the orientation produces the greatest decrease. If the gradient is zero, the first-order directional rate vanishes in every direction, even though the function need not be constant near the point, as the quadratic example shows.

To apply the result reliably, first verify that the function is differentiable at the point, then evaluate the gradient there. Its magnitude gives the extreme rates. If the gradient is nonzero, divide it by its magnitude for the maximizing direction and negate that unit vector for the minimizing direction. Finally, check that the proposed directions have magnitude $1$ and that their dot products with the gradient equal the claimed rates.

Check Your Understanding

Use the gradient formula and the unit-direction condition to answer these questions.

If $\nabla f(\mathbf{p})=(3,4)$, what are the maximum and minimum directional rates at $\mathbf{p}$?
For the gradient in question 1, what unit directions attain those two rates?
Why must a direction be normalized before using it to interpret a dot product as a rate per unit distance?
If $\nabla f(\mathbf{p})=\mathbf{0}$ and $f$ is differentiable at $\mathbf{p}$, what directional rate occurs in every unit direction?
What differentiability condition is needed to use $D_{\mathbf{u}}f(\mathbf{p})=\nabla f(\mathbf{p})\cdot\mathbf{u}$ in this theorem?