Deriving the Linear System for Implicit Derivatives

Why a Linear System Appears

When several variables are determined by several equations, differentiating each constraint produces equations that involve several unknown derivatives. These equations are linear in the derivatives, even when the original constraints are nonlinear. Writing them as a matrix equation reveals both the structure of the calculation and the condition that lets us solve it uniquely.

Suppose $x=(x_1,\dots,x_n)$ denotes independent variables and $y=(y_1,\dots,y_m)$ denotes dependent variables. A system of $m$ constraints can be written as $G(x,y)=\mathbf{0}$, where $G$ is a vector-valued function with $m$ components. If the constraints determine $y$ as a function of $x$, then substituting $y=y(x)$ gives $G(x,y(x))=\mathbf{0}$.

The goal is to derive an equation for the matrix of partial derivatives of $y$ with respect to $x$. This is a direct application of the multivariable chain rule, arranged so that the unknown derivatives appear together.

Set Up the Derivative Matrices

Write the constraints componentwise as $G_i(x_1,\dots,x_n,y_1,\dots,y_m)=0$, for $i=1,\dots,m$. Define the Jacobian matrices with respect to the independent and dependent variables by

J_xG= \begin{bmatrix} \frac{\partial G_1}{\partial x_1}&\cdots&\frac{\partial G_1}{\partial x_n}\\ \vdots&&\vdots\\ \frac{\partial G_m}{\partial x_1}&\cdots&\frac{\partial G_m}{\partial x_n} \end{bmatrix}, \qquad J_yG= \begin{bmatrix} \frac{\partial G_1}{\partial y_1}&\cdots&\frac{\partial G_1}{\partial y_m}\\ \vdots&&\vdots\\ \frac{\partial G_m}{\partial y_1}&\cdots&\frac{\partial G_m}{\partial y_m} \end{bmatrix}.

The derivative matrix of the dependent variables is defined by

D y= \begin{bmatrix} \frac{\partial y_1}{\partial x_1}&\cdots&\frac{\partial y_1}{\partial x_n}\\ \vdots&&\vdots\\ \frac{\partial y_m}{\partial x_1}&\cdots&\frac{\partial y_m}{\partial x_n} \end{bmatrix}.

Thus $J_xG$ is $m$ by $n$, $J_yG$ is $m$ by $m$, and $D y$ is $m$ by $n$. All the partial derivatives of $G$ are evaluated at $(x,y(x))$.

Definition: The linear system for implicit derivatives is $$ J_yG\,D y=-J_xG. $$ It relates the derivatives of the dependent variables to the partial derivatives of the constraints.

Deriving the Matrix Equation

For each constraint $G_i(x,y(x))=0$, differentiate with respect to each independent variable $x_k$. The chain rule gives

\frac{\partial G_i}{\partial x_k} + \sum_{j=1}^{m} \frac{\partial G_i}{\partial y_j} \frac{\partial y_j}{\partial x_k} =0.

For a fixed $k$, the first terms across all $m$ constraints form column $k$ of $J_xG$. The sums form column $k$ of $J_yG\,D y$: the $i$-th entry of that product column is $\sum_{j=1}^{m}(\partial G_i/\partial y_j)(\partial y_j/\partial x_k)$. Consequently, the chain-rule equations for every $i$ and every $k$ combine into one matrix equation.

Differentiate each constraint.
Apply the chain rule to $G_i(x,y(x))=0$ with respect to each $x_k$.

Group the independent-variable terms.
These partial derivatives form the matrix $J_xG$.

Group the dependent-variable terms.
These terms form the matrix product $J_yG\,D y$.

Set the sum equal to zero.
Rearrange $J_xG+J_yG\,D y=\mathbf{0}$ to obtain the linear system.

This derivation also explains why the system is linear in the unknown derivatives. The entries of $J_yG$ and $J_xG$ are partial derivatives of the constraints at the point under consideration; the unknown entries are those of $D y$, and they occur only to the first power in the matrix product.

When the Determinant Allows a Unique Solution

The matrix $J_yG$ is square because the number of constraints equals the number of dependent variables. If its determinant is nonzero at a point satisfying the constraints, then it is invertible there. Multiplying the linear system on the left by its inverse gives

D y=-(J_yG)^{-1}J_xG.

This determines the derivative matrix uniquely at that point. If the constraint functions are continuously differentiable near the point and $\det(J_yG)\ne 0$ there, the implicit function theorem also guarantees that the constraints define the dependent variables as continuously differentiable functions of the independent variables in some neighbourhood of the point. The determinant condition is therefore more than a computational convenience: together with the differentiability and constraint assumptions, it guarantees a local implicit description.

These claims should not be conflated. An invertible matrix lets us solve the differentiated equations uniquely. The implicit function theorem supplies the local functions whose derivatives those equations describe. A zero determinant means this particular inverse-matrix method does not apply; it does not, by itself, prove that no implicit functions exist.

Worked Examples

Worked Example: One Independent Variable and Two Dependent Variables

Consider the constraints $G_1(u,v,x)=u+v-x=0$ and $G_2(u,v,x)=u-v-x^2=0$. At $(x,u,v)=(0,0,0)$, both constraints hold. Find $du/dx$ and $dv/dx$ at that point using the linear system.

The Jacobian with respect to the dependent variables is

J_yG= \begin{bmatrix} \frac{\partial G_1}{\partial u}&\frac{\partial G_1}{\partial v}\\ \frac{\partial G_2}{\partial u}&\frac{\partial G_2}{\partial v} \end{bmatrix} = \begin{bmatrix} 1&1\\ 1&-1 \end{bmatrix}.

The independent-variable derivatives at the point are $ \partial G_1/\partial x=-1$ and $\partial G_2/\partial x=-2x=0$. Thus $J_xG=\begin{bmatrix}-1\\0\end{bmatrix}$, and the equation for the derivative column is

\begin{bmatrix}1&1\\1&-1\end{bmatrix} \begin{bmatrix}du/dx\\dv/dx\end{bmatrix} = \begin{bmatrix}1\\0\end{bmatrix}.

In scalar form, $du/dx+dv/dx=1$ and $du/dx-dv/dx=0$. Adding the equations gives $2\,du/dx=1$, so $du/dx=1/2$. Subtracting the second equation from the first gives $2\,dv/dx=1$, so $dv/dx=1/2$. The determinant is $1(-1)-1(1)=-2$, which is nonzero.

Check by differentiating the original constraints: $du/dx+dv/dx-1=1/2+1/2-1=0$, and $du/dx-dv/dx-2x=1/2-1/2-0=0$ at $x=0$. Both differentiated constraints are satisfied.

Worked Example: Two Independent Variables

Let $G_1(u,v,x,z)=u+2v-x^2-z=0$ and $G_2(u,v,x,z)=3u-v-x+2z=0$. The point $(x,z,u,v)=(0,0,0,0)$ satisfies both constraints. Find the matrix $D y$, where $y=(u,v)$ and the independent variables are $x,z$.

The dependent-variable Jacobian and independent-variable Jacobian at the point are

J_yG= \begin{bmatrix}1&2\\3&-1\end{bmatrix}, \qquad J_xG= \begin{bmatrix}-2x&-1\\-1&2\end{bmatrix}_{(0,0)} = \begin{bmatrix}0&-1\\-1&2\end{bmatrix}.

The determinant of $J_yG$ is $-1-6=-7$, so the matrix is invertible. The derivative equation is

\begin{bmatrix}1&2\\3&-1\end{bmatrix} \begin{bmatrix}u_x&u_z\\v_x&v_z\end{bmatrix} = \begin{bmatrix}0&1\\1&-2\end{bmatrix}.

For the $x$-column, solve $u_x+2v_x=0$ and $3u_x-v_x=1$. The first equation gives $u_x=-2v_x$; substitution into the second gives $-7v_x=1$. Therefore $v_x=-1/7$ and $u_x=2/7$.

For the $z$-column, solve $u_z+2v_z=1$ and $3u_z-v_z=-2$. The first equation gives $u_z=1-2v_z$. Substitution into the second gives $3-7v_z=-2$, so $v_z=5/7$ and $u_z=-3/7$. Hence

D y= \begin{bmatrix}2/7&-3/7\\-1/7&5/7\end{bmatrix}.

Check the multiplication. Its first row after multiplication by $J_yG$ is $\begin{bmatrix}2/7-2/7&-3/7+10/7\end{bmatrix}=\begin{bmatrix}0&1\end{bmatrix}$. Its second row is $\begin{bmatrix}6/7+1/7&-9/7-5/7\end{bmatrix}=\begin{bmatrix}1&-2\end{bmatrix}$. This equals $-J_xG$, as required.

Worked Example: A Nonlinear Constraint System

Consider $G_1(u,v,x)=u^2+v-x=0$ and $G_2(u,v,x)=u+v^2-x-2=0$. At $(x,u,v)=(3,1,2)$, the first constraint gives $1^2+2-3=0$, and the second gives $1+2^2-3-2=0$. Find the derivatives of $u$ and $v$ with respect to $x$ there.

At this point,

J_yG= \begin{bmatrix}2u&1\\1&2v\end{bmatrix} = \begin{bmatrix}2&1\\1&4\end{bmatrix}, \qquad J_xG= \begin{bmatrix}-1\\-1\end{bmatrix}.

The determinant is $2(4)-1(1)=7$, so the derivative system has a unique solution. It is

\begin{bmatrix}2&1\\1&4\end{bmatrix} \begin{bmatrix}du/dx\\dv/dx\end{bmatrix} = \begin{bmatrix}1\\1\end{bmatrix}.

The equations are $2\,du/dx+dv/dx=1$ and $du/dx+4\,dv/dx=1$. Subtract the second equation from the first to obtain $du/dx-3\,dv/dx=0$, so $du/dx=3\,dv/dx$. Substitution into the second equation gives $3\,dv/dx+4\,dv/dx=1$, hence $dv/dx=1/7$ and $du/dx=3/7$.

Check in the differentiated constraints. The first gives $2u(du/dx)+dv/dx-1=2(1)(3/7)+1/7-1=0$. The second gives $du/dx+2v(dv/dx)-1=3/7+2(2)(1/7)-1=0$. Both equations hold.

A Sign Error to Avoid

A common mistake is to write $J_yG\,D y=J_xG$, omitting the minus sign. The first example makes the error visible. At $x=0$, this incorrect equation would give

\begin{bmatrix}1&1\\1&-1\end{bmatrix} \begin{bmatrix}du/dx\\dv/dx\end{bmatrix} = \begin{bmatrix}-1\\0\end{bmatrix},

which leads to $du/dx=-1/2$ and $dv/dx=-1/2$. But substituting these values into the differentiated first constraint gives $-1/2-1/2-1=-2$, not zero. The correct chain-rule equation is $J_xG+J_yG\,D y=\mathbf{0}$, so the right-hand side after rearranging must be $-J_xG$.

Why the Matrix View Matters

The matrix equation organizes many related derivative calculations without changing the underlying chain rule. Each column corresponds to one independent variable, and each row of the constraint system contributes one equation. This makes the dimensions and dependencies explicit, particularly when there are several independent and dependent variables.

Before solving, check that the number of constraints matches the number of dependent variables, confirm that $J_yG$ is evaluated at the specified point, and compute its determinant. If that determinant is nonzero, the derivative matrix is uniquely determined by the linear system. If it is zero, do not use an inverse that does not exist: further analysis is needed to decide what the constraints imply.

Check Your Understanding

Use the chain rule and the matrix formulation to answer these questions.

For $m$ constraints and $n$ independent variables, what are the dimensions of $J_xG$, $J_yG$, and $D y$?
Why does differentiating $G(x,y(x))=\mathbf{0}$ produce $J_xG+J_yG\,D y=\mathbf{0}$?
What does $\det(J_yG)\ne 0$ allow you to conclude about the differentiated system?
Why is a nonzero determinant, together with continuously differentiable constraints and a point satisfying them, relevant to the implicit function theorem?
In the first worked example, what incorrect derivatives result from omitting the minus sign, and how does substitution reveal the error?