Tutorials › Calculus III › Deriving the One-Constraint Lagrange Condition

Home  /  Tutorials  /  Lagrange Multipliers

Lagrange Multipliers · Tutorial 452 of 999

Deriving the One-Constraint Lagrange Condition

You will derive why constrained extrema on a regular level set require the objective gradient to be parallel to the constraint gradient.

Intermediate 9 min read

What You'll Learn

  • Define regular points and tangent directions for a one-constraint level set.
  • Explain why each tangent direction is the velocity of a curve on the constraint.
  • Derive the vanishing directional derivative condition at a constrained local extremum.
  • Prove that orthogonality to all tangent directions implies a scalar-multiple gradient relation.
  • Recognize why the condition may fail when the constraint gradient is zero.
  • Distinguish a necessary condition for an extremum from a sufficient one.

Why Gradients Align at a Constrained Extremum

A function can change in many directions in space, but a constraint restricts which directions are available. If points must remain on a curve or surface, a small movement is allowed only along that curve or surface. At a local extremum subject to the constraint, the objective cannot have a first-order increase or decrease along any such allowed direction.

This observation leads to the one-constraint Lagrange condition. The objective gradient is perpendicular to every tangent direction at the constrained extremum. For a regular constraint, the constraint gradient is perpendicular to those same tangent directions. The two gradients must therefore be scalar multiples of one another.

Key idea: At a regular constrained local extremum, the objective has zero directional derivative in every tangent direction. This implies that its gradient is a scalar multiple of the constraint gradient.

Level Sets, Regular Points, and Tangent Directions

Let \(f:\mathbb{R}^n\to\mathbb{R}\) be differentiable, and let \(g:\mathbb{R}^n\to\mathbb{R}\) be continuously differentiable. A single equality constraint has the form \(g(\mathbf{x})=c\), where \(c\) is a fixed real number. Its solutions form a level set of \(g\).

Definition: A point \(\mathbf{p}\) on the level set \(g(\mathbf{x})=c\) is a regular point if \(\nabla g(\mathbf{p})\ne\mathbf{0}\). The tangent space to the level set at a regular point is $$ T_{\mathbf{p}}=\{\mathbf{v}\in\mathbb{R}^n:\nabla g(\mathbf{p})\cdot\mathbf{v}=0\}. $$ A vector in \(T_{\mathbf{p}}\) is called a tangent direction.

The tangent-space equation says that tangent directions are perpendicular to \(\nabla g(\mathbf{p})\). Geometrically, the gradient points normal to the level set. In the plane, a regular level set is locally a curve; in three-dimensional space, it is locally a surface.

At a regular point, each vector in \(T_{\mathbf{p}}\) is the velocity of a differentiable curve lying on the level set. To see why, choose a coordinate \(x_j\) for which \(g_{x_j}(\mathbf{p})\ne 0\). The implicit function theorem lets us solve the level-set equation locally for \(x_j\) as a differentiable function of the other coordinates. Given a tangent vector \(\mathbf{v}\), vary the other coordinates at rates \(v_k\). The rate of change of \(x_j\) along the resulting curve must satisfy

$$ g_{x_j}(\mathbf{p})\,x_j'(0) + \sum_{k\ne j}g_{x_k}(\mathbf{p})v_k=0. $$

Because \(\mathbf{v}\) is tangent, it satisfies the same equation with \(x_j'(0)\) replaced by \(v_j\). Since \(g_{x_j}(\mathbf{p})\ne0\), these equations give \(x_j'(0)=v_j\). Thus the curve has velocity \(\mathbf{v}\) at \(\mathbf{p}\).

From Tangent Derivatives to the Lagrange Condition

A constrained local extremum is a point that is a local maximum or local minimum when attention is restricted to the level set. Suppose \(\mathbf{p}\) is such an extremum and is a regular point of the constraint. Choose any tangent direction \(\mathbf{v}\). As just described, there is a differentiable curve \(\boldsymbol{\gamma}(t)\) on the level set with \(\boldsymbol{\gamma}(0)=\mathbf{p}\) and \(\boldsymbol{\gamma}'(0)=\mathbf{v}\).

The single-variable function \(h(t)=f(\boldsymbol{\gamma}(t))\) has a local extremum at \(t=0\). The derivative of a differentiable function at an interior local maximum or minimum is zero, so the chain rule gives

$$ 0=h'(0) =\nabla f(\mathbf{p})\cdot\boldsymbol{\gamma}'(0) =\nabla f(\mathbf{p})\cdot\mathbf{v}. $$

The tangent direction was arbitrary. Therefore \(\nabla f(\mathbf{p})\) is perpendicular to every vector in \(T_{\mathbf{p}}\). The following linear-algebra argument turns this statement into the gradient equation.

1
Name the two gradients.
Write \(\mathbf{a}=\nabla f(\mathbf{p})\) and \(\mathbf{b}=\nabla g(\mathbf{p})\). Regularity means \(\mathbf{b}\ne\mathbf{0}\), and the tangent space consists of vectors \(\mathbf{v}\) for which \(\mathbf{b}\cdot\mathbf{v}=0\).
2
Make an arbitrary vector tangent.
For any \(\mathbf{w}\in\mathbb{R}^n\), set \(\mathbf{v}=\mathbf{w}-\frac{\mathbf{w}\cdot\mathbf{b}}{\mathbf{b}\cdot\mathbf{b}}\mathbf{b}\). Then \(\mathbf{b}\cdot\mathbf{v}=0\), so \(\mathbf{a}\cdot\mathbf{v}=0\).
3
Rewrite the orthogonality equation.
Expanding \(\mathbf{a}\cdot\mathbf{v}=0\) gives \(\mathbf{a}\cdot\mathbf{w}=\frac{\mathbf{a}\cdot\mathbf{b}}{\mathbf{b}\cdot\mathbf{b}}(\mathbf{b}\cdot\mathbf{w})\) for every \(\mathbf{w}\).
4
Identify the vectors.
Since the equation holds for every \(\mathbf{w}\), it says that \(\mathbf{a}\) and \(\frac{\mathbf{a}\cdot\mathbf{b}}{\mathbf{b}\cdot\mathbf{b}}\mathbf{b}\) have the same dot product with every vector. They are equal.
Derived condition: If \(\mathbf{p}\) is a constrained local extremum of \(f\) on \(g(\mathbf{x})=c\), and \(\nabla g(\mathbf{p})\ne\mathbf{0}\), then there is a scalar \(\lambda\) such that $$ \nabla f(\mathbf{p})=\lambda\nabla g(\mathbf{p}). $$ The scalar is \(\lambda=\frac{\nabla f(\mathbf{p})\cdot\nabla g(\mathbf{p})}{\nabla g(\mathbf{p})\cdot\nabla g(\mathbf{p})}\). It may be positive, negative, or zero.

This is a necessary first-order condition, not a guarantee of a maximum or minimum. A point satisfying the equation might instead be a constrained saddle point, and the equation does not compare objective values at different candidates.

Worked Example: A Linear Objective on a Circle

Worked Example: A Linear Objective on a Circle

Consider \(f(x,y)=x+y\) on the circle \(g(x,y)=x^2+y^2=1\). At \(\mathbf{p}=(1/\sqrt{2},1/\sqrt{2})\), verify the tangent-derivative condition and the gradient relation.

First check that the point is on the constraint:

$$ g(\mathbf{p})=\left(\frac{1}{\sqrt{2}}\right)^2+\left(\frac{1}{\sqrt{2}}\right)^2 =\frac12+\frac12=1. $$

It is a constrained local maximum. A local parametrization of the circle is \((x,y)=(\cos t,\sin t)\), and the objective along this curve is \(h(t)=\cos t+\sin t=\sqrt{2}\cos(t-\pi/4)\), which has a local maximum at \(t=\pi/4\).

The gradients at the point are

$$ \nabla f(\mathbf{p})=(1,1), \qquad \nabla g(\mathbf{p})=(2x,2y)\big|_{\mathbf{p}}=(\sqrt{2},\sqrt{2}). $$

A tangent vector \(\mathbf{v}=(v_1,v_2)\) satisfies \(\nabla g(\mathbf{p})\cdot\mathbf{v}=0\), or \(\sqrt{2}v_1+\sqrt{2}v_2=0\), so \(v_1+v_2=0\). Its objective directional derivative is \(\nabla f(\mathbf{p})\cdot\mathbf{v}=v_1+v_2=0\), as required.

Finally, choose \(\lambda=1/\sqrt{2}\). Direct substitution verifies the gradient equation:

$$ \lambda\nabla g(\mathbf{p}) =\frac{1}{\sqrt{2}}(\sqrt{2},\sqrt{2}) =(1,1) =\nabla f(\mathbf{p}). $$

Worked Example: A Linear Objective on a Sphere

Worked Example: A Linear Objective on a Sphere

Let \(f(x,y,z)=x+2y+2z\) and constrain the point to \(x^2+y^2+z^2=9\). Check the condition at \(\mathbf{p}=(1,2,2)\).

The constraint holds because \(1^2+2^2+2^2=1+4+4=9\). Also, \(f(\mathbf{p})=1+4+4=9\). This point is a global maximum: for any point on the sphere, the Cauchy–Schwarz inequality gives

$$ x+2y+2z \le \sqrt{1^2+2^2+2^2}\sqrt{x^2+y^2+z^2} =3\cdot3=9. $$

Here \(\nabla f=(1,2,2)\), while the constraint function \(g=x^2+y^2+z^2\) has \(\nabla g(\mathbf{p})=(2,4,4)\). A tangent vector \(\mathbf{v}=(v_1,v_2,v_3)\) obeys \(2v_1+4v_2+4v_3=0\), equivalently \(v_1+2v_2+2v_3=0\). Thus

$$ \nabla f(\mathbf{p})\cdot\mathbf{v} =v_1+2v_2+2v_3=0. $$

The gradient equation holds with \(\lambda=1/2\), since

$$ \lambda\nabla g(\mathbf{p}) =\frac12(2,4,4) =(1,2,2) =\nabla f(\mathbf{p}). $$

This example also shows why the multiplier need not equal one.

Worked Example: A Parabola Constraint

Worked Example: A Parabola Constraint

Let \(f(x,y)=x^2+y\) and constrain \(g(x,y)=y-x^2=0\). Check the condition at \(\mathbf{p}=(0,0)\).

The constraint curve can be parametrized as \(\boldsymbol{\gamma}(t)=(t,t^2)\). Along it, \(f(\boldsymbol{\gamma}(t))=t^2+t^2=2t^2\), which has a local minimum at \(t=0\). The point is regular because

$$ \nabla g(x,y)=(-2x,1), \qquad \nabla g(0,0)=(0,1)\ne(0,0). $$

A tangent vector \(\mathbf{v}=(v_1,v_2)\) must satisfy \((0,1)\cdot(v_1,v_2)=v_2=0\), so it has the form \((a,0)\). At the point, \(\nabla f(0,0)=(2x,1)\big|_{(0,0)}=(0,1)\). Therefore \(\nabla f(0,0)\cdot\mathbf{v}=0\cdot a+1\cdot0=0\) for every tangent vector. The gradient equation is verified with \(\lambda=1\): \(\lambda\nabla g(0,0)=(0,1)=\nabla f(0,0)\).

Why the Regularity Condition Matters

The requirement \(\nabla g(\mathbf{p})\ne\mathbf{0}\) is essential to this derivation. It ensures that the level set is locally a smooth curve or surface and that its tangent space is the hyperplane perpendicular to a nonzero constraint gradient. If the constraint gradient is zero, this reasoning may not apply.

For example, take \(g(x,y)=x^2+y^2\), constrain \(g(x,y)=0\), and use \(f(x,y)=x\). The constraint set contains only \((0,0)\), so that point is both a constrained local maximum and a constrained local minimum: there are no other feasible points to compare. But \(\nabla g(0,0)=(0,0)\), while \(\nabla f(0,0)=(1,0)\). No scalar \(\lambda\) can satisfy \((1,0)=\lambda(0,0)\). This does not contradict the derived condition, because its regularity hypothesis fails.

Common Mistake: Setting the Gradients Equal

The conclusion is that the gradients are parallel, not necessarily equal. For a concrete check, take \(f(x,y)=2x+2y\), \(g(x,y)=x^2+y^2=1\), and \(\mathbf{p}=(1/\sqrt{2},1/\sqrt{2})\). This point maximizes \(f\) on the circle, but

$$ \nabla f(\mathbf{p})=(2,2), \qquad \nabla g(\mathbf{p})=(\sqrt{2},\sqrt{2}). $$

The incorrect calculation \(\nabla f(\mathbf{p})=\nabla g(\mathbf{p})\) would claim \((2,2)=(\sqrt{2},\sqrt{2})\), which is false. The correct relation uses \(\lambda=\sqrt{2}\):

$$ \lambda\nabla g(\mathbf{p}) =\sqrt{2}(\sqrt{2},\sqrt{2}) =(2,2) =\nabla f(\mathbf{p}). $$

The multiplier accounts for the fact that the two perpendicular vectors can point in the same or opposite directions and can have different lengths.

When to Use the Condition

The derivation gives a way to generate necessary candidates for constrained local extrema: at a regular point, test whether the objective gradient is a scalar multiple of the constraint gradient. It does not establish that a candidate is an extremum. Classification and comparison require further analysis of the objective on the feasible set.

Keep the logical sequence clear: a constrained extremum makes every tangent directional derivative vanish; tangent directions are perpendicular to the constraint gradient; therefore the objective gradient lies in the direction of the constraint gradient. The regularity assumption is what makes this geometric argument valid.

Check Your Understanding

Use the tangent-direction argument and the examples above to answer these questions.

  1. What equation must a tangent vector satisfy at a regular point of \(g(\mathbf{x})=c\)?
  2. Why does a constrained local extremum make the directional derivative of \(f\) vanish along every tangent direction?
  3. In the linear-algebra derivation, why must \(\nabla g(\mathbf{p})\) be nonzero?
  4. For \(f(x,y)=x-y\) on \(x^2+y^2=1\), compute \(\nabla f\), \(\nabla g\), and a scalar relation between them at \((1/\sqrt{2},-1/\sqrt{2})\).
  5. Does satisfying \(\nabla f=\lambda\nabla g\) by itself prove that a point is a constrained maximum or minimum? Explain.

Previous

Next

Back to Tutorials