Why the Remainders Need Attention
The Multivariable Chain Rule says that the derivative of a composition is the composition of the derivatives. Its formula is compact, but proving it requires one point of care: the outer function’s differentiability controls its remainder relative to a change in its own input, while the composition is differentiated relative to a change in the original input. The proof must connect those two sizes.
We use the Euclidean norm and the derivative-as-linear-map notation established earlier in this course. If \(f\) is differentiable at \(a\), with derivative \(A=Df(a)\), then its increment can be written as a linear part and a remainder: \(f(a+h)=f(a)+A(h)+r(h)\), where \(r(h)\) is negligible compared with \(\|h\|_2\) as \(h\to0\). Differentiability of an outer function gives a similar expansion, but with a remainder measured relative to the increment arriving at that outer function.
A small intermediate increment is not automatically negligible at the scale we need. The essential estimate is that the intermediate increment is at most a constant times \(\|h\|_2\) for small \(h\). This lets us transfer the outer remainder estimate back to the original input scale.
A Remainder Estimate for Intermediate Increments
Proof. If \(C=0\), the bound implies \(k(h)=0\) for all sufficiently small \(h\). Thus \(s(k(h))=s(0)=0\), and the conclusion follows. Now suppose \(C>0\). The bound gives \(k(h)\to0\) as \(h\to0\), so \(s(k(h))\) is defined for all sufficiently small \(h\). If \(k(h)=0\), then \(s(k(h))=0\). If \(k(h)\neq0\), then
As \(h\to0\), \(k(h)\to0\). Along the values with \(k(h)\neq0\), the final ratio tends to \(0\) by the hypothesis on \(s\). At values with \(k(h)=0\), the original ratio is already \(0\). Together these cases show that \(\|s(k(h))\|_2/\|h\|_2\to0\). \(\square\)
The lemma packages a useful proof technique: a remainder that is small compared with an intermediate increment is also small compared with the original increment, provided that intermediate increment is controlled by a constant multiple of the original one. The zero case matters because a ratio with denominator \(\|k(h)\|_2\) cannot be used when \(k(h)=0\).
Proof of the Multivariable Chain Rule
Proof. Put \(b=f(a)\), \(A=Df(a)\), and \(B=Dg(b)\). Differentiability of \(f\) at \(a\) gives a remainder \(r(h)\), defined for small \(h\), such that \(r(0)=0\),
Define the intermediate increment \(k(h)=f(a+h)-b\). Then \(k(0)=0\) and \(k(h)=A(h)+r(h)\). By the operator-norm bound for linear maps, \(\|A(h)\|_2\leq\|A\|_{\mathrm{op}}\|h\|_2\). Since \(r(h)/\|h\|_2\to0\), there is a neighborhood of \(0\) on which \(\|r(h)\|_2\leq\|h\|_2\). Consequently, for sufficiently small \(h\),
Thus \(k(h)\) is controlled by a constant times \(\|h\|_2\), and in particular \(k(h)\to0\). Differentiability of \(g\) at \(b\) gives a remainder \(s(u)\), with \(s(0)=0\), such that
Applying this expansion with \(u=k(h)\) gives \((g\circ f)(a+h)=g(b)+B(k(h))+s(k(h))\). Since \(k(h)=A(h)+r(h)\) and \(B\) is linear, we can rearrange this as
The expression \(B\circ A\) is linear. It remains to show that the term in parentheses is negligible compared with \(\|h\|_2\). The operator-norm bound for \(B\) gives \(\|B(r(h))\|_2\leq\|B\|_{\mathrm{op}}\|r(h)\|_2\), so \(\|B(r(h))\|_2/\|h\|_2\to0\). By the Remainder Along a Controlled Increment Lemma and the bound on \(k(h)\), we also have \(\|s(k(h))\|_2/\|h\|_2\to0\). The triangle inequality therefore yields
This is precisely the differentiability condition for \(g\circ f\) at \(a\), with derivative \(B\circ A\). Substituting \(A=Df(a)\) and \(B=Dg(f(a))\) proves the formula. \(\square\)
Worked Examples: Following the Remainders
Worked Example: A Composition from the Plane to the Plane
Let \(f:\mathbb{R}^2\to\mathbb{R}^2\) and \(g:\mathbb{R}^2\to\mathbb{R}^2\) be \(f(s,t)=(s+t^2,st)\) and \(g(x,y)=(x^2+y,xy)\). At \(a=(1,-1)\), substitution gives \(f(a)=(2,-1)\). The derivative matrices are
The chain rule gives
For a direct check, write \(q=s+t^2\) and \(r=st\), so \((g\circ f)(s,t)=(q^2+r,qr)\). At \((1,-1)\), \(q=2\), \(r=-1\), and \(Dq=(1,-2)\), \(Dr=(-1,1)\). Therefore \(D(q^2+r)=4(1,-2)+(-1,1)=(3,-7)\), while \(D(qr)=(-1)(1,-2)+2(-1,1)=(-3,4)\). These rows agree with the matrix product.
Worked Example: A Scalar Function Along a Curve
Let \(\gamma:\mathbb{R}\to\mathbb{R}^2\) be \(\gamma(t)=(t^2+1,2t-1)\), and let \(G:\mathbb{R}^2\to\mathbb{R}\) be \(G(x,y)=x^2+3xy+y^2\). At \(t=1\), \(\gamma(1)=(2,1)\) and \(D\gamma(1)=\begin{pmatrix}2\\2\end{pmatrix}\). Also, \(DG(x,y)=\begin{pmatrix}2x+3y&3x+2y\end{pmatrix}\), so \(DG(2,1)=\begin{pmatrix}7&8\end{pmatrix}\). Thus
To check this result directly, differentiating the composition gives \[ \frac{d}{dt}G(\gamma(t)) = 2(t^2+1)(2t) +3\bigl((2t-1)(2t)+(t^2+1)2\bigr) +2(2t-1)2. \] At \(t=1\), the three terms are \(8\), \(18\), and \(4\), respectively, so the derivative is \(30\).
Worked Example: Seeing the Small Outer Remainder
Let \(f:\mathbb{R}\to\mathbb{R}^2\) be \(f(t)=(t,t^2)\), and let \(g:\mathbb{R}^2\to\mathbb{R}\) be \(g(x,y)=x+y^2\). At \(0\), \(f(0)=(0,0)\), \(Df(0)(h)=(h,0)\), and \(Dg(0,0)(u,v)=u\). Hence the chain rule predicts \(D(g\circ f)(0)(h)=h\).
Here the inner increment at \(0\) is \(k(h)=(h,h^2)\), and \(\|k(h)\|_2\leq |h|+h^2\leq2|h|\) for \(|h|\leq1\). The outer function has the exact expansion \(g(u,v)=g(0,0)+Dg(0,0)(u,v)+v^2\), so its remainder at \(k(h)\) is \(h^4\). For \(h\neq0\), \(h^4/|h|=|h|^3\to0\). Indeed, direct substitution gives \((g\circ f)(t)=t+t^4\), whose derivative at \(0\) is \(1\). This example displays why controlling the size of \(k(h)\) is useful: the outer remainder is measured using that intermediate increment.
What the Proof Does—and Does Not—Assume
The proof uses differentiability at the two relevant points, not continuity of either derivative on a neighborhood. Differentiability of \(f\) provides a controlled increment \(k(h)\); differentiability of \(g\) provides a remainder small relative to that increment. The argument then shows that the resulting error is small relative to the original \(h\).
A common gap is to write that the outer remainder is negligible and stop there. Its defining estimate is relative to \(\|k(h)\|_2\), not directly to \(\|h\|_2\). The intermediate-increment bound bridges this gap. Another detail is the case \(k(h)=0\): in that case the outer remainder equals \(s(0)=0\), so no division by \(\|k(h)\|_2\) is needed.
The resulting derivative is \(Dg(f(a))\circ Df(a)\), in that order: \(Df(a)\) first transforms the original increment, and \(Dg(f(a))\) then transforms the intermediate one. With Jacobian matrices, this is \(J_g(f(a))J_f(a)\), consistent with the matrix representation of the derivative.
Check Your Understanding
Use the proof and its remainder estimates to answer the following questions.
- In the chain rule proof, what is the intermediate increment \(k(h)\), and why does it tend to \(0\)?
- How does differentiability of \(f\) imply a bound of the form \(\|k(h)\|_2\leq C\|h\|_2\) for small \(h\)?
- Why must the case \(k(h)=0\) be treated separately when estimating the outer remainder?
- Which point is used to evaluate the derivative of the outer function, and why?
- Does the proof require the derivatives of \(f\) and \(g\) to be continuous near the points in question?