When There Are No Feasible Directions
Lagrange multipliers are often used to find points where an objective can no longer improve by moving along a constraint set. That description suggests a situation with feasible directions: a curve, surface, or other set along which one can move while preserving the constraints.
But the number of independent equality constraints can match the number of variables. If the constraints are locally independent, there may be no nonzero direction that preserves all of them, even to first order. In that case the feasible point is isolated, and the usual multiplier equation is automatically solvable for any objective gradient.
Setup: A Square System of Constraints
Let \(f:\mathbb{R}^n\to\mathbb{R}\) be the objective, and let \(g_i:\mathbb{R}^n\to\mathbb{R}\), for \(i=1,\ldots,n\), be continuously differentiable constraint functions. Fix constraint values \(c_1,\ldots,c_n\). The feasible set consists of points satisfying all \(n\) equations.
Collect the constraint functions into \(\mathbf{G}=(g_1,\ldots,g_n)\), and write \(\mathbf{c}=(c_1,\ldots,c_n)\). The Jacobian matrix \(D\mathbf{G}(\mathbf{x})\) has the constraint gradients as its rows. The constraints have full rank at a feasible point \(\mathbf{x}_0\) when this \(n\times n\) matrix is invertible.
This full-rank condition means the \(n\) constraint gradients are linearly independent. Because there are \(n\) such vectors in \(\mathbb{R}^n\), they form a basis of \(\mathbb{R}^n\).
Why the Feasible Point Is Isolated
The inverse function theorem says that if a continuously differentiable map from \(\mathbb{R}^n\) to \(\mathbb{R}^n\) has an invertible Jacobian at \(\mathbf{x}_0\), then it is one-to-one on some neighborhood of \(\mathbf{x}_0\). Apply this theorem to \(\mathbf{G}\). Since \(\mathbf{G}(\mathbf{x}_0)=\mathbf{c}\), no other point sufficiently close to \(\mathbf{x}_0\) can also map to \(\mathbf{c}\).
The same conclusion appears in the tangent-space description. At a regular feasible point, tangent directions \(\mathbf{v}\) must satisfy the linearized constraints \(D\mathbf{G}(\mathbf{x}_0)\mathbf{v}=\mathbf{0}\). An invertible matrix has only the zero vector in its nullspace, so the only tangent direction is \(\mathbf{v}=\mathbf{0}\).
Why the Multiplier Equation Is Automatic
For equality constraints \(g_i(\mathbf{x})=c_i\), the Lagrange condition at a regular constrained extremum is \(\nabla f(\mathbf{x}_0)=\sum_{i=1}^n\lambda_i\nabla g_i(\mathbf{x}_0)\) for some multipliers \(\lambda_i\). In the square full-rank case, the constraint gradients form a basis. Every vector in \(\mathbb{R}^n\), including \(\nabla f(\mathbf{x}_0)\), is therefore a linear combination of them.
The inverse in this formula exists because \(D\mathbf{G}(\mathbf{x}_0)\) is invertible, and its transpose is also invertible. So the equation has a unique multiplier vector for every objective gradient. The equation is true, but it does not help distinguish candidate points: it imposes no restriction on \(\nabla f(\mathbf{x}_0)\).
Worked Example: Two Linear Constraints in Two Variables
Worked Example: Two Linear Constraints in Two Variables
Find the feasible point for \(x+y=2\) and \(x-y=0\). For the objective \(f(x,y)=x^2+3y\), compute multipliers satisfying the Lagrange equation and check the result.
Set \(g_1(x,y)=x+y\), \(g_2(x,y)=x-y\), and require \(g_1=2\), \(g_2=0\). Adding the equations gives \(2x=2\), so \(x=1\). Substituting into \(x-y=0\) gives \(y=1\). The feasible set contains just \((1,1)\).
The constraint gradients are \(\nabla g_1=(1,1)\) and \(\nabla g_2=(1,-1)\). Their Jacobian has determinant \(-2\), so the constraints have full rank. The objective gradient is \(\nabla f=(2x,3)\), which at \((1,1)\) is \((2,3)\). The multiplier equations are
Adding these equations gives \(2\lambda_1=5\), hence \(\lambda_1=5/2\). Subtracting the second equation from the first gives \(2\lambda_2=-1\), hence \(\lambda_2=-1/2\). Substitution checks both equations: \(5/2-1/2=2\) and \(5/2+1/2=3\). The objective value is \(f(1,1)=1+3=4\). Since the feasible set is a singleton, this point is both a global minimum and a global maximum on that set.
Worked Example: A Circle and a Line
Worked Example: A Circle and a Line
Consider \(x^2+y^2=1\) and \(x-y=0\), with objective \(f(x,y)=x+3y\). Find the feasible points, their objective values, and multipliers at each point.
The line constraint gives \(y=x\). Substitution into the circle gives \(2x^2=1\), so \(x=\pm 1/\sqrt{2}\), with \(y=x\). The two feasible points are
Write \(g_1(x,y)=x^2+y^2\) and \(g_2(x,y)=x-y\). Their gradients are \((2x,2y)\) and \((1,-1)\). At either feasible point, the determinant of the Jacobian is \(-2x-2y\), which is nonzero because \(x=y=\pm1/\sqrt{2}\). Thus each point is regular and isolated. Also \(\nabla f=(1,3)\).
At \(\mathbf{x}_{+}\), the multiplier equation becomes
Equating components gives \(\sqrt{2}\lambda_1+\lambda_2=1\) and \(\sqrt{2}\lambda_1-\lambda_2=3\). Adding yields \(2\sqrt{2}\lambda_1=4\), so \(\lambda_1=\sqrt{2}\). The first equation then gives \(\lambda_2=-1\). Checking, the left side is \((2-1,2+1)=(1,3)\). The objective value is \(4/\sqrt{2}=2\sqrt{2}\).
At \(\mathbf{x}_{-}\), the multiplier equation is
Its component equations are \(-\sqrt{2}\lambda_1+\lambda_2=1\) and \(-\sqrt{2}\lambda_1-\lambda_2=3\). Adding gives \(-2\sqrt{2}\lambda_1=4\), so \(\lambda_1=-\sqrt{2}\). The first equation then gives \(\lambda_2=-1\). Substitution gives \((2-1,2+1)=(1,3)\), as required. Here \(f=-2\sqrt{2}\). Comparing the two values, \(\mathbf{x}_{+}\) is the global maximum and \(\mathbf{x}_{-}\) the global minimum.
Worked Example: Three Constraints in Three Variables
Worked Example: Three Constraints in Three Variables
Consider \(x^2+y^2+z^2=1\), \(x=0\), and \(y=0\), with objective \(f(x,y,z)=2x-y+4z\). Find the feasible points and the multipliers at each.
The last two constraints force \(x=y=0\). The first then gives \(z^2=1\), so the feasible points are \((0,0,1)\) and \((0,0,-1)\). Let \(g_1=x^2+y^2+z^2\), \(g_2=x\), and \(g_3=y\). At \((0,0,\pm1)\), the rows of the constraint Jacobian are \((0,0,\pm2)\), \((1,0,0)\), and \((0,1,0)\). Its determinant is nonzero, so both points are isolated and full rank.
The objective gradient is \((2,-1,4)\). The multiplier equation is
Matching the first two coordinates gives \(\lambda_2=2\) and \(\lambda_3=-1\). At \(z=1\), the third coordinate gives \(2\lambda_1=4\), so \(\lambda_1=2\). At \(z=-1\), it gives \(-2\lambda_1=4\), so \(\lambda_1=-2\). In each case, substitution reproduces \((2,-1,4)\). The objective values are \(4\) and \(-4\), respectively, so the first point is the global maximum and the second is the global minimum.
Local and Global Conclusions
An isolated feasible point is automatically both a constrained local minimum and a constrained local maximum, regardless of the objective value there. Indeed, there is a neighborhood containing no other feasible points, so every feasible point in that neighborhood has the same objective value: the value at the isolated point.
This local conclusion does not mean that every isolated feasible point is a global minimum or a global maximum. If the feasible set contains several isolated points, their objective values may differ. The circle-and-line example shows this directly: each point is locally both kinds of extremum, while comparison of the values identifies the global minimum and maximum.
A Common Mistake
A tempting but incorrect conclusion is that a zero-dimensional tangent space forces the objective gradient to be zero. In the circle-and-line example, at \(\mathbf{x}_{+}\) the tangent space is \(\{\mathbf{0}\}\), but \(\nabla f(\mathbf{x}_{+})=(1,3)\), not \((0,0)\). The mistaken reasoning confuses the condition “the gradient has zero dot product with every tangent direction” with “the gradient is zero.” With no nonzero tangent directions, the dot-product condition is satisfied by every gradient.
The correct multiplier equation expresses the gradient as a combination of the constraint gradients. At \(\mathbf{x}_{+}\), the explicit check is
The gradient is not forced to vanish; the constraint gradients span the entire ambient space.
Why This Case Matters
In a problem with fewer independent constraints than variables, the tangent space can contain nonzero directions, and the multiplier condition can help identify candidates for constrained extrema. When the number of independent constraints equals the number of variables, a full-rank feasible point has no such directions. The multiplier equation remains valid, but by itself it cannot distinguish one full-rank feasible point from another.
The useful checks are therefore: verify feasibility, test whether the square constraint Jacobian is full rank, and then compare objective values across the feasible points when a global conclusion is needed. The multiplier values can be computed, but their existence is guaranteed by invertibility rather than by a special property of the objective.
Check Your Understanding
Use the full-rank square-constraint case to answer these questions.
- What does it mean for the Jacobian of \(n\) equality constraints in \(n\) variables to have full rank at a feasible point?
- Why does the inverse function theorem imply that such a feasible point is isolated?
- What is the tangent space when the square constraint Jacobian is invertible?
- Why can every objective gradient be written as a combination of the constraint gradients in this case?
- Why does being an isolated feasible point make a point both a local minimum and a local maximum, but not necessarily a global extremum?