Marginal Values with Multiple Constraints

Multipliers as Marginal Values

With two equality constraints, Lagrange multipliers do more than help identify candidate extrema. Under suitable regularity assumptions, each multiplier measures how the optimized objective value changes when the level of its corresponding constraint changes, while the other level is held fixed.

For instance, suppose a design must satisfy two resource requirements. If the best achievable objective value changes rapidly when the first requirement is increased, its multiplier records that sensitivity. The interpretation is local: it describes a rate of change near the constraint levels under consideration, not necessarily a reliable prediction for a large change.

Constraint Levels and the Value Function

Let $f:\mathbb{R}^n\to\mathbb{R}$ be an objective, and let $g_1,g_2:\mathbb{R}^n\to\mathbb{R}$ be two constraint functions. For levels $a$ and $b$, consider optimizing $f(\mathbf{x})$ subject to

g_1(\mathbf{x})=a, \qquad g_2(\mathbf{x})=b.

When the problem has an optimizer, denote a chosen optimizer by $\mathbf{x}^*(a,b)$. The corresponding optimized value is

V(a,b)=f\bigl(\mathbf{x}^*(a,b)\bigr).

The function $V$ is called the value function. Its partial derivative $V_a$, if it exists, measures the first-order change in the optimized value as $a$ varies while $b$ stays fixed. Similarly, $V_b$ describes the change as $b$ varies while $a$ stays fixed.

Convention: Use the stationarity equation $$ \nabla f(\mathbf{x}^*)=\lambda_1\nabla g_1(\mathbf{x}^*)+\lambda_2\nabla g_2(\mathbf{x}^*). $$ With this convention, the multipliers correspond to the constraint levels $g_1=a$ and $g_2=b$ with positive signs: $\lambda_1=V_a$ and $\lambda_2=V_b$, when the regularity conditions below hold.

When the Interpretation Applies

The multiplier equations alone do not guarantee that a multiplier is a marginal value. To make that interpretation, work near fixed levels $(a,b)$ and assume there is a differentiable choice of optimizer $\mathbf{x}^*(a,b)$ for nearby levels. Assume also that $f,g_1,g_2$ are continuously differentiable and that the two constraint gradients at the optimizer are linearly independent. This independence is the regularity condition: it ensures the constraint equations have rank two there.

Finally, assume the optimizer satisfies the Lagrange stationarity equation with multipliers $\lambda_1,\lambda_2$. These assumptions hold, for example, along a smooth branch of regular constrained local optima. To interpret the result as sensitivity of a global optimum, the chosen branch must also remain globally optimal for the nearby levels being considered.

Theorem: Under these assumptions, for $V(a,b)=f(\mathbf{x}^*(a,b))$, $$ \frac{\partial V}{\partial a}=\lambda_1, \qquad \frac{\partial V}{\partial b}=\lambda_2. $$

Why the Formula Works

The proof uses the chain rule and the fact that the constraint values move at known rates when the levels change. It does not require explicitly solving for the optimizer as a function of $a$ and $b$.

Differentiate the optimized objective.
Since $V(a,b)=f(\mathbf{x}^*(a,b))$, the chain rule gives $V_a=\nabla f(\mathbf{x}^*)\cdot\mathbf{x}^*_a$, where $\mathbf{x}^*_a$ is the partial derivative of the optimizer with respect to $a$.

Use stationarity.
Substitute $\nabla f=\lambda_1\nabla g_1+\lambda_2\nabla g_2$ into that chain-rule expression.

Differentiate the constraints.
From $g_1(\mathbf{x}^*)=a$ and $g_2(\mathbf{x}^*)=b$, the chain rule gives $\nabla g_1\cdot\mathbf{x}^*_a=1$ and $\nabla g_2\cdot\mathbf{x}^*_a=0$.

Combining these equations proves the first formula:

\begin{aligned} V_a &=\nabla f(\mathbf{x}^*)\cdot\mathbf{x}^*_a\\ &=\lambda_1\nabla g_1(\mathbf{x}^*)\cdot\mathbf{x}^*_a +\lambda_2\nabla g_2(\mathbf{x}^*)\cdot\mathbf{x}^*_a\\ &=\lambda_1(1)+\lambda_2(0)=\lambda_1. \end{aligned}

For differentiation with respect to $b$, the differentiated constraints give $\nabla g_1\cdot\mathbf{x}^*_b=0$ and $\nabla g_2\cdot\mathbf{x}^*_b=1$. Thus

\begin{aligned} V_b &=\nabla f(\mathbf{x}^*)\cdot\mathbf{x}^*_b\\ &=\lambda_1(0)+\lambda_2(1)=\lambda_2. \end{aligned}

The proof shows why the two multipliers have separate interpretations: changing one constraint level changes its own constraint value at rate one, while the other constraint value remains fixed.

Worked Example: A Maximum with Two Constraint Levels

Maximize $f(x,y,z)=-x^2+2y+3z$, subject to $g_1(x,y,z)=x+y=a$ and $g_2(x,y,z)=z=b$. First use the constraints to write $y=a-x$ and $z=b$. The objective on the feasible set becomes

$$ f=-x^2+2(a-x)+3b =-(x+1)^2+1+2a+3b. $$

Since $-(x+1)^2\leq 0$, the unique maximum occurs at $x=-1$. The constraints then give $y=a+1$ and $z=b$. Substitution verifies feasibility: $x+y=-1+(a+1)=a$ and $z=b$. The optimized value is

V(a,b)=1+2a+3b, \qquad V_a=2,\quad V_b=3.

Now verify the multipliers. At the optimizer, $\nabla f=(-2x,2,3)=(2,2,3)$, $\nabla g_1=(1,1,0)$, and $\nabla g_2=(0,0,1)$. Therefore

$$ (2,2,3)=2(1,1,0)+3(0,0,1), $$

so $\lambda_1=2$ and $\lambda_2=3$. They agree with $V_a$ and $V_b$, respectively. In this example increasing $a$ by a small amount changes the maximum by approximately twice that amount, while increasing $b$ changes it by approximately three times that amount. In fact, because $V$ is linear, those changes are exact for any change in the levels.

Worked Example: A Nonlinear Value Function

Maximize $f(x,y,z)=xy-z^2$ subject to $g_1=x+y=a$ and $g_2=x-y=b$. The constraints determine

x=\frac{a+b}{2}, \qquad y=\frac{a-b}{2}.

For these fixed $x$ and $y$, the objective $xy-z^2$ is largest at $z=0$, since $z^2\geq 0$, with equality exactly when $z=0$. Thus the optimizer is $\bigl((a+b)/2,(a-b)/2,0\bigr)$, and substituting gives

V(a,b)=xy =\frac{(a+b)(a-b)}{4} =\frac{a^2-b^2}{4}.

Consequently $V_a=a/2$ and $V_b=-b/2$. To check the multipliers, the constraint gradients $(1,1,0)$ and $(1,-1,0)$ are linearly independent. At the optimizer, $\nabla f=(y,x,0)$. The stationarity equation requires

(y,x,0)=\lambda_1(1,1,0)+\lambda_2(1,-1,0),

so $\lambda_1+\lambda_2=y$ and $\lambda_1-\lambda_2=x$. Adding and subtracting these equations gives

\lambda_1=\frac{x+y}{2}=\frac{a}{2}, \qquad \lambda_2=\frac{y-x}{2}=-\frac{b}{2}.

These values equal the two partial derivatives of $V$. In particular, the second multiplier can be negative: locally, increasing $b$ decreases the maximum when $b$ is positive.

A Minimum Has the Same Sensitivity Formula

The chain-rule argument applies to a differentiable branch of constrained minima as well as maxima. The stationarity equation is still written as $\nabla f=\lambda_1\nabla g_1+\lambda_2\nabla g_2$; the nature of the extremum is determined by additional information, not by changing the marginal-value identity.

Worked Example: Minimum Squared Distance

Minimize $f(x,y,z)=x^2+y^2+z^2$ subject to $x+y=a$ and $y+z=b$. Set $x=a-y$ and $z=b-y$. The objective as a function of $y$ is

$$ q(y)=(a-y)^2+y^2+(b-y)^2. $$

Differentiating and setting the derivative equal to zero gives $q'(y)=-2a+6y-2b=0$, hence $y=(a+b)/3$. Also $q''(y)=6>0$, so this is the unique minimum. Therefore $x=(2a-b)/3$ and $z=(2b-a)/3$. Substitution checks the constraints:

x+y=\frac{2a-b+a+b}{3}=a, \qquad y+z=\frac{a+b+2b-a}{3}=b.

Substituting the coordinates into the objective yields

\begin{aligned} V(a,b) &=\frac{(2a-b)^2+(a+b)^2+(2b-a)^2}{9}\\ &=\frac{6a^2-6ab+6b^2}{9} =\frac{2}{3}(a^2-ab+b^2). \end{aligned}

Thus $V_a=(4a-2b)/3$ and $V_b=(4b-2a)/3$. For the multiplier check, $\nabla f=(2x,2y,2z)$, while $\nabla g_1=(1,1,0)$ and $\nabla g_2=(0,1,1)$. The stationarity equation gives $\lambda_1=2x$ and $\lambda_2=2z$; its middle component requires $2y=\lambda_1+\lambda_2$. This holds because $x+z=(2a-b+2b-a)/3=(a+b)/3=y$. Hence

\lambda_1=\frac{2(2a-b)}{3}=V_a, \qquad \lambda_2=\frac{2(2b-a)}{3}=V_b.

Sign Conventions and Other Cautions

A common mistake is to use a multiplier obtained from one Lagrangian convention with the marginal-value formula for another convention. For example, consider maximizing $f(x,y,z)=x+y+2z$ subject to $x+y=a$ and $z=b$. The value is $V(a,b)=a+2b$, so the marginal values are $V_a=1$ and $V_b=2$.

With the convention $\nabla f=\lambda_1\nabla g_1+\lambda_2\nabla g_2$, where $g_1=x+y$ and $g_2=z$, stationarity gives $(1,1,2)=\lambda_1(1,1,0)+\lambda_2(0,0,1)$. Thus $\lambda_1=1$ and $\lambda_2=2$, as expected. But if instead one defines $L=f+\mu_1(g_1-a)+\mu_2(g_2-b)$, stationarity gives $\nabla f=-\mu_1\nabla g_1-\mu_2\nabla g_2$. Here $\mu_1=-1$ and $\mu_2=-2$; the marginal values are $-\mu_1$ and $-\mu_2$, not $\mu_1$ and $\mu_2$. The correction is to derive the sign from the chosen stationarity equation rather than memorize it without its convention.

There are also situations where a marginal-value interpretation is unavailable or needs qualification. If the constraint gradients are dependent, the multipliers may not be unique. If the optimizer does not vary differentiably with the levels, the chain-rule proof does not apply. Even with regular constraints, two competing optimal branches can exchange which one is globally best; the value function may then have a corner where its derivative does not exist. A computed multiplier describes the sensitivity of its regular optimizer branch, and represents the global value's derivative only when that branch continues to supply the global optimum nearby.

Takeaway: With two independent equality constraints and a differentiable optimizer branch, each multiplier equals the partial derivative of the optimized value with respect to its own constraint level, provided the stationarity convention is $\nabla f=\lambda_1\nabla g_1+\lambda_2\nabla g_2$.

Check Your Understanding

Use the stated multiplier convention and distinguish local sensitivity from claims about large changes in constraint levels.

For $g_1(\mathbf{x})=a$ and $g_2(\mathbf{x})=b$, what does $V_a$ measure, and what does $V_b$ measure?
In the chain-rule proof, what are $\nabla g_1\cdot\mathbf{x}^*_a$ and $\nabla g_2\cdot\mathbf{x}^*_a$, and why?
Why must the two constraint gradients be linearly independent for the regularity assumption used here?
If stationarity is written as $\nabla f=-\mu_1\nabla g_1-\mu_2\nabla g_2$, how do the marginal values relate to $\mu_1$ and $\mu_2$?
Give one reason a multiplier might fail to equal the derivative of the global optimized value.