Deriving the Multivariable Chain Rule

Why the Chain Rule Follows from Linearization

A differentiable function behaves, near a point, like a linear function plus an error that becomes negligible compared with the size of the input change. The multivariable chain rule follows by applying this idea twice: first to the inner map, then to the outer map.

The key is not merely to multiply two derivative matrices. We must also show that the errors from both approximations remain negligible after the maps are composed. That step explains why differentiability at the relevant points is the essential hypothesis.

Differentiability as a Local Linear Approximation

Let $F:U\to\mathbb{R}^m$, where $U$ is an open subset of $\mathbb{R}^n$, and let $\mathbf{a}\in U$. The function $F$ is differentiable at $\mathbf{a}$ if there is a linear map $A:\mathbb{R}^n\to\mathbb{R}^m$ such that

F(\mathbf{a}+\mathbf{h}) = F(\mathbf{a})+A\mathbf{h}+\mathbf{r}(\mathbf{h}), \qquad \frac{\|\mathbf{r}(\mathbf{h})\|}{\|\mathbf{h}\|} \longrightarrow 0 \quad\text{as }\mathbf{h}\to\mathbf{0}.

The vector $\mathbf{r}(\mathbf{h})$ is the remainder: its size is small relative to $\|\mathbf{h}\|$ when $\mathbf{h}$ is small. The linear map $A$ is the derivative of $F$ at $\mathbf{a}$, written $DF(\mathbf{a})$. In standard coordinates it is represented by the Jacobian matrix.

For a second map $G:V\to\mathbb{R}^p$, with $V$ open in $\mathbb{R}^m$, we can apply the same definition at a point $\mathbf{b}\in V$. The composition $G\circ F$ is defined near $\mathbf{a}$ when $F$ maps those nearby inputs into $V$.

Definition: The derivative of a differentiable map is the linear part of its local approximation. Differentiability requires the remaining error, divided by the size of the input change, to tend to zero.

Deriving the Rule

Suppose $F:U\to\mathbb{R}^m$ is differentiable at $\mathbf{a}$, and $G:V\to\mathbb{R}^p$ is differentiable at $\mathbf{b}=F(\mathbf{a})$. Assume the maps are defined on open sets so that the composition is defined for inputs sufficiently close to $\mathbf{a}$. Write $A=DF(\mathbf{a})$ and $B=DG(\mathbf{b})$.

Approximate the inner map.
For a small input change $\mathbf{h}$, write $F(\mathbf{a}+\mathbf{h})=\mathbf{b}+A\mathbf{h}+\mathbf{r}(\mathbf{h})$, where $\|\mathbf{r}(\mathbf{h})\|/\|\mathbf{h}\|\to0$.

Name the change passed to the outer map.
Set $\mathbf{k}=F(\mathbf{a}+\mathbf{h})-\mathbf{b}=A\mathbf{h}+\mathbf{r}(\mathbf{h})$. This change tends to zero as $\mathbf{h}\to\mathbf{0}$.

Approximate the outer map.
Differentiability of $G$ at $\mathbf{b}$ gives $G(\mathbf{b}+\mathbf{k})=G(\mathbf{b})+B\mathbf{k}+\mathbf{s}(\mathbf{k})$, where $\|\mathbf{s}(\mathbf{k})\|/\|\mathbf{k}\|\to0$.

Substitute and control the errors.
Substituting $\mathbf{k}=A\mathbf{h}+\mathbf{r}(\mathbf{h})$ gives the linear part $BA\mathbf{h}$. The remaining terms are $B\mathbf{r}(\mathbf{h})+\mathbf{s}(\mathbf{k})$, which are negligible compared with $\|\mathbf{h}\|$.

Here is the error estimate that justifies the last step. Because a linear map between finite-dimensional spaces is bounded, there is a constant $C$ with $\|A\mathbf{h}\|\le C\|\mathbf{h}\|$. Also, the definition of $\mathbf{r}$ implies $\|\mathbf{r}(\mathbf{h})\|\le\|\mathbf{h}\|$ for all sufficiently small $\mathbf{h}$. Thus $\|\mathbf{k}\|\le(C+1)\|\mathbf{h}\|$. In particular, $\mathbf{k}\to\mathbf{0}$, and the outer remainder satisfies $\|\mathbf{s}(\mathbf{k})\|/\|\mathbf{h}\|\to0$. If $\mathbf{k}=\mathbf{0}$, its remainder is zero, so this estimate still applies.

Likewise, boundedness of $B$ gives $\|B\mathbf{r}(\mathbf{h})\|\le C_B\|\mathbf{r}(\mathbf{h})\|$, which divided by $\|\mathbf{h}\|$ tends to zero. Therefore the total remainder after subtracting $G(\mathbf{b})+BA\mathbf{h}$ is negligible compared with $\|\mathbf{h}\|$. This proves the composition is differentiable at $\mathbf{a}$, with derivative $BA$.

Theorem: If $F$ is differentiable at $\mathbf{a}$, $G$ is differentiable at $F(\mathbf{a})$, and the composition is defined near $\mathbf{a}$, then $G\circ F$ is differentiable at $\mathbf{a}$, and $$ D(G\circ F)(\mathbf{a})=DG(F(\mathbf{a}))\,DF(\mathbf{a}). $$ The order is important: the derivative applied first, $DF(\mathbf{a})$, appears on the right.

Worked Examples

Worked Example: Composing Two Maps of Two Variables

Let $F(u,v)=(u^2+v,uv)$ and $G(x,y)=(x^2+y,xy)$. Find the derivative of $G\circ F$ at $(1,2)$ using the chain rule, then check it by expanding the composition locally.

First, $F(1,2)=(1^2+2,1\cdot2)=(3,2)$. The Jacobian matrices are $$ DF(u,v)= \begin{bmatrix} 2u&1\\ v&u \end{bmatrix}, \qquad DG(x,y)= \begin{bmatrix} 2x&1\\ y&x \end{bmatrix}. $$ Hence $$ DF(1,2)= \begin{bmatrix} 2&1\\ 2&1 \end{bmatrix}, \qquad DG(3,2)= \begin{bmatrix} 6&1\\ 2&3 \end{bmatrix}. $$ Multiplying in the order required by the theorem, $$ D(G\circ F)(1,2)= \begin{bmatrix} 6&1\\ 2&3 \end{bmatrix} \begin{bmatrix} 2&1\\ 2&1 \end{bmatrix} = \begin{bmatrix} 14&7\\ 10&5 \end{bmatrix}. $$

To check, perturb the input to $(1+h,2+k)$. The inner map becomes $$ F(1+h,2+k)= (3+2h+k+h^2,\;2+2h+k+hk). $$ Set $\delta x=2h+k+h^2$ and $\delta y=2h+k+hk$. Since $G(3+\delta x,2+\delta y)=((3+\delta x)^2+2+\delta y,(3+\delta x)(2+\delta y))$, expansion gives $$ G(F(1+h,2+k))= (11+14h+7k+6h^2+hk+\delta x^2,\; 6+10h+5k+2h^2+3hk+\delta x\delta y). $$ The terms after the linear parts have degree at least two in $h,k$, so divided by $\sqrt{h^2+k^2}$ they tend to zero. The constant value is $G(F(1,2))=(11,6)$, and the linear terms have matrix $\begin{bmatrix}14&7\\10&5\end{bmatrix}$, as predicted.

Worked Example: A Scalar Function Along a Parametrized Input

Let $F(t)=(t^2+1,2t)$ and $G(x,y)=x^2+xy$. Find $(G\circ F)'(1)$ by composing the derivatives and verify by differentiating the explicit composition.

At $t=1$, $F(1)=(2,2)$, and $DF(1)$ is the column vector $\begin{bmatrix}2\\2\end{bmatrix}$. The derivative of $G$ at $(x,y)$ is the row vector $\begin{bmatrix}2x+y&x\end{bmatrix}$, so $DG(2,2)=\begin{bmatrix}6&2\end{bmatrix}$. Therefore $$ (G\circ F)'(1)= \begin{bmatrix}6&2\end{bmatrix} \begin{bmatrix}2\\2\end{bmatrix} =12+4=16. $$

Directly, the composition is $$ G(F(t))=(t^2+1)^2+(t^2+1)(2t). $$ Its derivative is $$ 4t(t^2+1)+4t^2+2(t^2+1). $$ At $t=1$, this is $4(1)(2)+4+2(2)=8+4+4=16$. The direct calculation agrees with the derivative obtained from the local linear maps.

Worked Example: A Composition with Three Intermediate Coordinates

Let $F(u,v)=(u+v,u-v,uv)$ and $G(x,y,z)=xz+y^2$. Find the derivative of $G\circ F$ at $(1,2)$, and confirm the linear term by expanding.

We have $F(1,2)=(3,-1,2)$. The derivative matrices are $$ DF(1,2)= \begin{bmatrix} 1&1\\ 1&-1\\ 2&1 \end{bmatrix}, \qquad DG(3,-1,2)= \begin{bmatrix} z&2y&x \end{bmatrix}_{(3,-1,2)} = \begin{bmatrix}2&-2&3\end{bmatrix}. $$ Thus $$ D(G\circ F)(1,2)= \begin{bmatrix}2&-2&3\end{bmatrix} \begin{bmatrix} 1&1\\ 1&-1\\ 2&1 \end{bmatrix} = \begin{bmatrix}6&7\end{bmatrix}. $$

For a direct check, $F(1+h,2+k)=(3+h+k,-1+h-k,2+2h+k+hk)$. Put $\delta x=h+k$, $\delta y=h-k$, and $\delta z=2h+k+hk$. Then $$ G(F(1+h,2+k))=(3+\delta x)(2+\delta z)+(-1+\delta y)^2. $$ Expanding yields $$ 7+6h+7k+3hk+(h+k)(2h+k+hk)+(h-k)^2. $$ The last three terms are all of order at least two in $h,k$. The constant is $G(F(1,2))=7$, and the linear part is $6h+7k$, confirming the derivative row $\begin{bmatrix}6&7\end{bmatrix}$.

Why Differentiability Is the Right Condition

The proof depends on more than being able to write down partial derivatives. Differentiability supplies a single linear approximation that works for input changes in every direction, with a remainder that is negligible relative to the size of the full change. That control is what allows the errors to survive composition without affecting the derivative.

A common mistake is to use the chain rule based only on existing partial derivatives. For a concrete warning, define $g:\mathbb{R}^2\to\mathbb{R}$ by $$ g(x,y)= \begin{cases} \dfrac{xy}{x^2+y^2},&(x,y)\ne(0,0),\\ 0,&(x,y)=(0,0). \end{cases} $$ Both partial derivatives at the origin exist and equal zero: along either coordinate axis, $g$ is identically zero. Now let $F(t)=(t,t)$, so $F(0)=(0,0)$ and $F'(0)=(1,1)$. The tempting but incorrect calculation is $g_x(0,0)\cdot1+g_y(0,0)\cdot1=0$, as if the chain rule were justified.

But for $t\ne0$, $(g\circ F)(t)=t^2/(t^2+t^2)=1/2$, while $(g\circ F)(0)=0$. The composition is not even continuous at zero, so it has no derivative there. The error is that $g$ is not differentiable at the origin: its values along the line $y=x$ do not approach $g(0,0)$. The multivariable chain rule cannot be invoked merely from the existence of partial derivatives.

Key takeaway: To use the multivariable chain rule at a point, verify differentiability of the inner map at the input point and of the outer map at the inner map's output. The theorem then gives the derivative of the composition as the outer derivative multiplied by the inner derivative.

Check Your Understanding

Use the local-linearization argument, not only a memorized matrix formula, to answer these questions.

In the composition theorem, at which points must the inner and outer maps be differentiable?
Why does the change $\mathbf{k}=F(\mathbf{a}+\mathbf{h})-F(\mathbf{a})$ have size at most a constant times $\|\mathbf{h}\|$ near $\mathbf{h}=\mathbf{0}$?
If $DF(\mathbf{a})$ is an $m$-by-$n$ matrix and $DG(F(\mathbf{a}))$ is a $p$-by-$m$ matrix, what are the dimensions of the derivative of the composition?
Why do existing partial derivatives at a point not, by themselves, establish the multivariable chain rule there?
For the maps $F(u,v)=(u+v,u-v,uv)$ and $G(x,y,z)=xz+y^2$, what is the linear part of the composition's change at $(1,2)$?