Proof of Lagrange Multipliers

The Proof as a Chain of Geometric and Linear-Algebraic Steps

The Lagrange Multipliers tutorial stated a necessary condition for a constrained local extremum. This tutorial proves that condition by making each step explicit. At a regular point of a level set, feasible curves have velocities in the kernel of the constraint derivative. At a constrained extremum, the objective has zero first-order change along every such velocity. The objective gradient must therefore lie in the normal space to the level set, which is the image of the transpose of the constraint derivative.

The proof uses the Tangent Space to a Regular Level Set Theorem, the Normal Space to a Regular Level Set Theorem, the Constrained Fermat Condition, and the Gradient Representation of the Derivative. These results supply the geometric and differential steps; the central task is to connect them carefully and interpret the resulting linear-algebra statement.

The Main Proof

Let \(F:U\to\mathbb{R}^k\) define the constraints \(F(x)=c\), and write \(M=F^{-1}(\{c\})\). At a feasible point \(a\), the condition that \(DF(a)\) has rank \(k\) says that the constraints are regular there. In particular, the level-set theorems apply at \(a\). The tangent space is \(\ker DF(a)\), and the normal space is its orthogonal complement, \(\operatorname{im}DF(a)^T\).

Proof. Since \(a\) is a constrained local extremum and \(F\) is continuously differentiable with \(DF(a)\) of rank \(k\), the Constrained Fermat Condition and the Tangent Space to a Regular Level Set Theorem give $$ Df(a)v=0 \qquad\text{for every }v\in\ker DF(a). $$ The Gradient Representation of the Derivative says \(Df(a)v=\nabla f(a)\cdot v\). Thus \(\nabla f(a)\) is orthogonal to every vector in \(\ker DF(a)\), so $$ \nabla f(a)\in(\ker DF(a))^\perp. $$ By the Normal Space to a Regular Level Set Theorem, $$ (\ker DF(a))^\perp=\operatorname{im}DF(a)^T. $$ Membership in this image means that there is a vector \(\lambda\in\mathbb{R}^k\) with \(\nabla f(a)=DF(a)^T\lambda\), as required. \(\square\)

Each hypothesis has a role. The constrained extremum and differentiability of \(f\) give vanishing first-order change along feasible directions. The regularity condition lets us identify those directions with the full kernel of \(DF(a)\) and use the stated normal-space description. Without regularity, the multiplier conclusion need not follow.

Reading the Matrix Equation

Write the component functions of \(F\) as \(F_1,\ldots,F_k\). The rows of \(DF(a)\) are the transposes of their gradients, so the matrix equation is equivalent to $$ \nabla f(a)=\lambda_1\nabla F_1(a)+\cdots+\lambda_k\nabla F_k(a). $$ Thus the objective gradient is a linear combination of the constraint gradients. The multipliers are the coefficients in this combination, and their order corresponds to the order in which the constraints are listed.

For one constraint \(g(x)=c\), the equation reduces to \(\nabla f(a)=\lambda\nabla g(a)\). For several constraints, the transpose in \(DF(a)^T\) matters: \(DF(a)\) maps directions in \(\mathbb{R}^n\) to changes in the \(k\) constraint values, while \(DF(a)^T\) maps coefficient vectors in \(\mathbb{R}^k\) back to combinations of the constraint gradients in \(\mathbb{R}^n\).

When Are the Multipliers Unique?

The theorem guarantees at least one multiplier. Full rank gives more: when the number of constraints is \(k\) and \(DF(a)\) has rank \(k\), the multipliers are unique. This conclusion concerns the coefficients in the gradient equation; it does not claim that the constrained extremum itself is unique.

Proof. Suppose \(\lambda\) and \(\mu\) both satisfy the equation. Subtracting gives $$ DF(a)^T(\lambda-\mu)=0. $$ The rows of \(DF(a)\) are linearly independent because its rank is \(k\). The equation above says that the linear combination of those rows with coefficients \(\lambda_1-\mu_1,\ldots,\lambda_k-\mu_k\) is zero. Linear independence forces every coefficient to be zero. Hence \(\lambda=\mu\). \(\square\)

This proof also identifies the exact algebraic reason for uniqueness. It is not a general property of multiplier equations at singular points: if the constraint gradients are dependent, different coefficient vectors can represent the same gradient. Regularity prevents that dependence.

Worked Examples

What the Proof Does—and Does Not—Establish

The proof is a necessary-condition argument. It shows that a constrained local extremum at a regular point must satisfy the multiplier equations. It does not show that every solution of those equations is an extremum, or that an extremum must be global. To classify candidates, one needs additional information, such as a direct comparison on the constraint set or a suitable second-order test.

The rank condition must also be checked at the point being studied. For example, if the single constraint is \(g(x,y)=x^2+y^2=0\), the feasible set consists only of the origin, and \(f(x,y)=x\) has both a constrained maximum and a constrained minimum there. But \(\nabla f(0,0)=(1,0)\) while \(\nabla g(0,0)=(0,0)\), so the multiplier equation cannot hold. The constraint derivative has rank zero at the origin, and the Lagrange Multiplier Theorem does not apply. This example shows why regularity is a genuine hypothesis, not a formality.

A reliable proof and application follow the same order: identify the feasible level set, verify the rank condition, use tangent directions to obtain vanishing first-order change, and then use the normal-space description to obtain the multiplier equation. Only after solving those equations should one decide whether the candidates are actually extrema.