Why Two Constraints Produce a Gradient-Span Condition
With one equality constraint, a regular constrained extremum has an objective gradient parallel to the constraint gradient. With two constraints, the feasible set is locally restricted in two independent normal directions. The objective gradient must therefore lie in the span of those two normals—not, in general, be parallel to either one alone.
The reasoning begins with directions in which a point can move while remaining feasible. At a regular point, those directions are orthogonal to both constraint gradients. At an extremum, the objective cannot change to first order in any such direction. Linear algebra then gives the gradient-span rule.
Constraints and Regular Points
Let \(f,g,h:\mathbb{R}^n\to\mathbb{R}\) be continuously differentiable near a feasible point \(\mathbf{p}\). The two equality constraints specify the feasible set
The gradients \(\nabla g(\mathbf{p})\) and \(\nabla h(\mathbf{p})\) are the normal directions associated with the constraints at \(\mathbf{p}\). We call \(\mathbf{p}\) a regular point for the two constraints if these gradients are linearly independent. In particular, neither gradient can be obtained by multiplying the other by a scalar.
Regularity matters because it ensures that, near \(\mathbf{p}\), the two constraints define a smooth set of dimension \(n-2\). Its tangent directions are precisely the directions perpendicular to both constraint gradients.
From Feasible Motion to Tangent Directions
A tangent direction describes an instantaneous direction of motion along the feasible set. At a regular point, a vector \(\mathbf{v}\in\mathbb{R}^n\) is tangent exactly when it satisfies both linear equations
To see why, take a differentiable curve \(\boldsymbol{\gamma}(t)\) in the feasible set with \(\boldsymbol{\gamma}(0)=\mathbf{p}\) and \(\boldsymbol{\gamma}'(0)=\mathbf{v}\). Since the constraints stay constant along the curve, the chain rule gives
Regularity gives the converse as well: every vector satisfying these two equations is the velocity of a local feasible curve. One way to understand this is to choose two coordinates for which the corresponding \(2\times2\) minor of the constraint derivative matrix is invertible. The implicit function theorem then expresses those two coordinates as differentiable functions of the other \(n-2\) coordinates. Differentiating that local parametrization shows that its tangent vectors are exactly the solutions of the two equations above.
Consequently the tangent space is
Derivation of the Gradient-Span Rule
We can now prove the theorem. At a constrained local extremum, \(f\) has a local maximum or minimum along every feasible curve through \(\mathbf{p}\). Therefore, the derivative of \(f\) along every such curve must be zero. For a feasible curve with velocity \(\mathbf{v}\), the chain rule gives that derivative as \(\nabla f(\mathbf{p})\cdot\mathbf{v}\).
For every \(\mathbf{v}\in T_{\mathbf{p}}S\), regularity ensures a feasible curve through \(\mathbf{p}\) with velocity \(\mathbf{v}\).
Because the point is a constrained local extremum, the derivative is zero, so \(\nabla f(\mathbf{p})\cdot\mathbf{v}=0\) for every \(\mathbf{v}\in T_{\mathbf{p}}S\).
The tangent space is the set of vectors orthogonal to both constraint gradients. Its orthogonal complement is therefore \(\operatorname{span}\{\nabla g(\mathbf{p}),\nabla h(\mathbf{p})\}\).
Since \(\nabla f(\mathbf{p})\) is orthogonal to every tangent vector, it belongs to that orthogonal complement. Thus it is a linear combination of the two constraint gradients.
The key linear-algebra fact in Step 3 can be checked directly. Both constraint gradients are orthogonal to every vector in \(T_{\mathbf{p}}S\), so their span lies in the orthogonal complement of \(T_{\mathbf{p}}S\). The two gradients are independent, so their span has dimension \(2\). The two independent tangent equations give a tangent space of dimension \(n-2\), whose orthogonal complement has dimension \(2\). The inclusion is therefore equality. This completes the derivation.
Worked Example: A Maximum on a Circle
Worked Example: A Maximum on a Circle
Let \(f(x,y,z)=x-y\), subject to \(g(x,y,z)=x^2+y^2+z^2=2\) and \(h(x,y,z)=x+y+z=0\). Verify the gradient-span condition at \(\mathbf{p}=(1,-1,0)\), and check that this point is a constrained maximum.
First check feasibility:
The gradients are \(\nabla f=(1,-1,0)\), \(\nabla g=(2x,2y,2z)\), and \(\nabla h=(1,1,1)\). At the point,
The constraint gradients are independent: a multiple of \((2,-2,0)\) has third coordinate \(0\), whereas \((1,1,1)\) has third coordinate \(1\). Moreover,
To check the tangent-direction reasoning, a tangent vector \(\mathbf{v}=(v_x,v_y,v_z)\) must satisfy \(2v_x-2v_y=0\) and \(v_x+v_y+v_z=0\). Thus \(v_x=v_y\), \(v_z=-2v_x\), and \(\mathbf{v}=t(1,1,-2)\) for some real \(t\). The objective's directional derivative is
Finally, on the feasible set, \(x+y=-z\), so \(x-y\) is the dot product of \((x,y,z)\) with \((1,-1,0)\). The Cauchy–Schwarz inequality gives \(x-y\leq\sqrt{2}\sqrt{2}=2\). At \(\mathbf{p}\), \(f(\mathbf{p})=1-(-1)=2\). Hence \(\mathbf{p}\) is indeed a constrained maximum, and its gradients satisfy the derived condition.
Worked Example: Both Multipliers Can Be Nonzero
Worked Example: Both Multipliers Can Be Nonzero
Consider \(f(x,y,z)=3x+y-z\), with constraints \(x^2+y^2+z^2=2\) and \(x+y+z=0\). Check the condition at \(\mathbf{p}=(1,0,-1)\), and verify that this point is a maximum.
The point is feasible because \(1^2+0^2+(-1)^2=2\) and \(1+0-1=0\). The gradients at the point are
The constraint gradients are independent: if \((1,1,1)\) were a multiple of \((2,0,-2)\), its second coordinate would have to be \(0\), but it is \(1\). Direct calculation gives
For a tangent vector, the two equations are \(2v_x-2v_z=0\) and \(v_x+v_y+v_z=0\). They imply \(v_z=v_x\) and \(v_y=-2v_x\), so \(\mathbf{v}=t(1,-2,1)\). Then
To check the maximum, use \(x+y+z=0\) to write \(f=2x-2z\). Thus, on the feasible set,
At \(\mathbf{p}\), \(f(\mathbf{p})=3(1)+0-(-1)=4\), so the upper bound is attained. Both multipliers in this example are nonzero: the objective gradient combines contributions from both constraint gradients.
Worked Example: Two Tangent Directions in Four Variables
Worked Example: Two Tangent Directions in Four Variables
Let \(f(x,y,z,w)=2x+y\), subject to \(x^2+y^2+z^2+w^2=1\) and \(x+y=1\). Check the rule at \(\mathbf{p}=(1,0,0,0)\), and verify that it is a constrained maximum.
At \(\mathbf{p}\), the constraints hold: \(1^2=1\) and \(1+0=1\). Their gradients and the objective gradient are
The first two vectors are independent, and the required combination is
A tangent vector \(\mathbf{v}=(v_x,v_y,v_z,v_w)\) satisfies \(2v_x=0\) and \(v_x+v_y=0\). Therefore \(v_x=v_y=0\), while \(v_z\) and \(v_w\) are unrestricted. The tangent directions have the form \((0,0,a,b)\), and \(\nabla f(\mathbf{p})\cdot(0,0,a,b)=0\).
For the maximum check, \(y=1-x\). Substitution into the sphere constraint gives
Since \(z^2+w^2\geq0\), we have \(2x^2-2x\leq0\), or \(x(x-1)\leq0\). Hence \(0\leq x\leq1\). On the feasible set \(f=2x+(1-x)=x+1\leq2\), with equality at \(\mathbf{p}\). Thus this point is a constrained maximum, and its tangent space has the expected two independent directions.
A Common Mistake: Requiring Parallelism to Each Gradient
A tempting but incorrect argument is to say that \(\nabla f\) must be parallel to \(\nabla g\), and also parallel to \(\nabla h\). For two independent constraint gradients, that would usually force an impossible conclusion. The condition is that \(\nabla f\) be a linear combination of the two gradients.
Worked Example: Correcting the Parallelism Error
Return to the second example. At \((1,0,-1)\), \(\nabla f=(3,1,-1)\), \(\nabla g=(2,0,-2)\), and \(\nabla h=(1,1,1)\). The incorrect calculation would demand \((3,1,-1)=\lambda(2,0,-2)\). Its second coordinate would give \(1=0\lambda=0\), which is impossible. Requiring \((3,1,-1)=\mu(1,1,1)\) is also impossible: the first coordinate requires \(\mu=3\), while the second requires \(\mu=1\).
The correction is to allow contributions from both normals:
This is exactly the gradient-span condition. It agrees with the tangent-direction test, which showed that \(\nabla f\) is orthogonal to every feasible first-order direction.
When to Use the Condition
The derivation gives a necessary equation for a regular constrained local extremum. Its assumptions should be checked before using it: the objective and constraints must be differentiable near the point, the point must satisfy both constraints, and the two constraint gradients must be linearly independent there. The equation alone does not classify a candidate as a maximum or minimum. A candidate may instead be neither, and a separate analysis is needed to determine its behavior.
The geometric idea is useful because it turns a constrained optimization question into a statement about directions. The constraints remove two independent normal directions from feasible motion. At an extremum, the objective has zero first-order change along what remains. This is why its gradient must be built from the constraint gradients.
Check Your Understanding
Use the tangent-direction argument and the regularity condition to answer these questions.
- What does it mean for the two constraint gradients to be linearly independent at a feasible point?
- Write the two equations that a tangent vector must satisfy for constraints \(g=c\) and \(h=d\).
- Why must the directional derivative of \(f\) vanish along every feasible tangent direction at a constrained local extremum?
- In the gradient-span condition, why is a linear combination of the two constraint gradients required rather than parallelism to each one separately?
- Does satisfying the gradient-span equation by itself prove that a feasible point is a local maximum or minimum? Explain.