Tutorials › Calculus III › The Multivariable Chain Rule in Matrix Form

Home  /  Tutorials  /  Chain Rule and Implicit Differentiation

Chain Rule and Implicit Differentiation · Tutorial 355 of 999

The Multivariable Chain Rule in Matrix Form

Use Jacobian matrices to calculate derivatives of composed multivariable functions, keeping the evaluation points and matrix order correct.

Intermediate 9 min read

What You'll Learn

  • Identify the dimensions of the Jacobian for a map from one Euclidean space to another.
  • State the matrix form of the multivariable chain rule and its differentiability conditions.
  • Evaluate each Jacobian at the correct input point in a composition.
  • Multiply Jacobians in the order dictated by the maps.
  • Verify a composed derivative by differentiating the explicit composition.
  • Recognize and correct common dimension and matrix-order errors.

Why Put the Chain Rule in Matrix Form?

When one multivariable function is used as the input to another, changes in the original variables pass through each map in sequence. The chain rule describes how those changes combine. Writing the rule with Jacobian matrices makes the calculation systematic: the dimensions show which matrices can be multiplied, and the order records the direction in which the changes propagate.

This notation is especially useful when the intermediate output has a different number of components from either the input or final output. A composition can begin in two variables, pass through three intermediate coordinates, and end in two output coordinates. The matrices make that changing dimension explicit.

The Jacobian Matrix

Let \(F:\mathbb{R}^n\to\mathbb{R}^m\) be a differentiable map, written in components as \(F=(F_1,\ldots,F_m)\). Its Jacobian matrix at \(\mathbf{x}\) records the first partial derivatives of each output component with respect to each input coordinate. It has one row for each output component and one column for each input coordinate.

Definition: The Jacobian matrix of \(F:\mathbb{R}^n\to\mathbb{R}^m\) at \(\mathbf{x}\) is the \(m\times n\) matrix
$$ J_F(\mathbf{x}) = \begin{bmatrix} \frac{\partial F_1}{\partial x_1}(\mathbf{x}) & \cdots & \frac{\partial F_1}{\partial x_n}(\mathbf{x})\\ \vdots & \ddots & \vdots\\ \frac{\partial F_m}{\partial x_1}(\mathbf{x}) & \cdots & \frac{\partial F_m}{\partial x_n}(\mathbf{x}) \end{bmatrix}. $$

The size \(m\times n\) follows from the direction of the map: \(n\) input coordinates are sent to \(m\) output coordinates. When \(F\) is differentiable, this matrix represents the linear map that approximates the change in \(F\) near \(\mathbf{x}\).

The Chain Rule in Matrix Form

Suppose \(F:\mathbb{R}^n\to\mathbb{R}^m\) is differentiable at \(\mathbf{a}\), and \(G:\mathbb{R}^m\to\mathbb{R}^p\) is differentiable at \(\mathbf{b}=F(\mathbf{a})\). Then \(G\circ F\) maps \(\mathbb{R}^n\) to \(\mathbb{R}^p\). The derivative of the composition is obtained by multiplying the Jacobian of the outer map by the Jacobian of the inner map.

Theorem: Under the differentiability conditions above,
$$ J_{G\circ F}(\mathbf{a}) = J_G\bigl(F(\mathbf{a})\bigr)\,J_F(\mathbf{a}). $$
The dimensions are \(p\times m\) for \(J_G(F(\mathbf{a}))\) and \(m\times n\) for \(J_F(\mathbf{a})\). Their product is \(p\times n\), as required for a map from \(\mathbb{R}^n\) to \(\mathbb{R}^p\).

The order is important. The input first passes through \(F\), so \(J_F\) acts first on a change in the original input. The resulting change is then passed through \(G\), whose Jacobian acts afterward. In matrix multiplication, the matrix for the later action appears on the left.

Why the Matrix Product Has This Order

The coordinate chain rule explains every entry of the matrix product. Write the components of \(F\) as \(F_1,\ldots,F_m\) and those of \(G\) as \(G_1,\ldots,G_p\). For output component \(G_r\circ F\) and input coordinate \(x_j\), the chain rule gives

$$ \frac{\partial (G_r\circ F)}{\partial x_j}(\mathbf{a}) = \sum_{k=1}^{m} \frac{\partial G_r}{\partial y_k}\bigl(F(\mathbf{a})\bigr) \frac{\partial F_k}{\partial x_j}(\mathbf{a}). $$

By the definition of matrix multiplication, the right-hand side is the entry in row \(r\), column \(j\) of \(J_G(F(\mathbf{a}))J_F(\mathbf{a})\): it is the sum of the row-\(r\), column-\(k\) entry of the first matrix times the row-\(k\), column-\(j\) entry of the second, over all intermediate coordinates \(k\). Thus each entry of the product is exactly the corresponding partial derivative of the composition. This also shows why the inner Jacobian must be on the right and the outer Jacobian on the left.

Worked Example: A Composition from \(\mathbb{R}^2\) Through \(\mathbb{R}^3\) to \(\mathbb{R}^2\)

Worked Example: A Composition from \(\mathbb{R}^2\) Through \(\mathbb{R}^3\) to \(\mathbb{R}^2\)

Define \(F:\mathbb{R}^2\to\mathbb{R}^3\) and \(G:\mathbb{R}^3\to\mathbb{R}^2\) by

$$ F(x,y)=(x+y,\ x^2,\ xy), \qquad G(u,v,w)=(u+v,\ w+2v). $$

Find the Jacobian of \(H=G\circ F\) at \(\mathbf{a}=(1,2)\). First evaluate the intermediate point:

$$ F(1,2)=(1+2,\ 1^2,\ 1\cdot2)=(3,1,2). $$

The Jacobian of \(F\), whose size is \(3\times2\), is

$$ J_F(x,y) = \begin{bmatrix} 1&1\\ 2x&0\\ y&x \end{bmatrix}, \qquad J_F(1,2) = \begin{bmatrix} 1&1\\ 2&0\\ 2&1 \end{bmatrix}. $$

The Jacobian of \(G\), of size \(2\times3\), is

$$ J_G(u,v,w) = \begin{bmatrix} 1&1&0\\ 0&2&1 \end{bmatrix}, \qquad J_G(3,1,2) = \begin{bmatrix} 1&1&0\\ 0&2&1 \end{bmatrix}. $$

Use the outer Jacobian at \(F(1,2)\), followed on the right by the inner Jacobian at \((1,2)\):

$$ J_H(1,2) = \begin{bmatrix} 1&1&0\\ 0&2&1 \end{bmatrix} \begin{bmatrix} 1&1\\ 2&0\\ 2&1 \end{bmatrix} = \begin{bmatrix} 1+2+0&1+0+0\\ 0+4+2&0+0+1 \end{bmatrix} = \begin{bmatrix} 3&1\\ 6&1 \end{bmatrix}. $$

Check by composing explicitly. The first component is \((x+y)+x^2\), and the second is \(xy+2x^2\). At \((1,2)\), the function value is

$$ H(1,2) = \begin{bmatrix} (1+2)+1^2\\ 1\cdot2+2(1^2) \end{bmatrix} = \begin{bmatrix} 4\\ 4 \end{bmatrix}. $$

Differentiating the explicit components gives

$$ J_H(x,y) = \begin{bmatrix} 1+2x&1\\ y+4x&x \end{bmatrix}, \qquad J_H(1,2) = \begin{bmatrix} 3&1\\ 6&1 \end{bmatrix}. $$

This agrees with the Jacobian product. The value \(H(1,2)=(4,4)\) is a separate check of the composition itself; the Jacobian calculation gives the local change in the output.

Worked Example: A Scalar-Valued Composition

Worked Example: A Scalar-Valued Composition

Let \(F:\mathbb{R}^2\to\mathbb{R}^2\) be \(F(x,y)=(x^2+y,xy)\), and let \(G:\mathbb{R}^2\to\mathbb{R}\) be \(G(u,v)=u^2+3v\). Find the derivative of \(H=G\circ F\) at \((1,2)\).

The intermediate point is

$$ F(1,2)=(1^2+2,\ 1\cdot2)=(3,2). $$

The two Jacobians, evaluated at the appropriate points, are

$$ J_F(x,y) = \begin{bmatrix} 2x&1\\ y&x \end{bmatrix}, \qquad J_F(1,2) = \begin{bmatrix} 2&1\\ 2&1 \end{bmatrix}, \qquad J_G(u,v)= \begin{bmatrix} 2u&3 \end{bmatrix}, \qquad J_G(3,2)= \begin{bmatrix} 6&3 \end{bmatrix}. $$

Here \(J_G\) is \(1\times2\), and \(J_F\) is \(2\times2\). Their product is a \(1\times2\) row matrix:

$$ J_H(1,2) = \begin{bmatrix}6&3\end{bmatrix} \begin{bmatrix}2&1\\2&1\end{bmatrix} = \begin{bmatrix} 6\cdot2+3\cdot2& 6\cdot1+3\cdot1 \end{bmatrix} = \begin{bmatrix}18&9\end{bmatrix}. $$

For a direct check, \(H(x,y)=(x^2+y)^2+3xy\). Its partial derivatives are \(H_x=2(x^2+y)(2x)+3y\) and \(H_y=2(x^2+y)+3x\). At \((1,2)\), these are \(2(3)(2)+3(2)=18\) and \(2(3)+3(1)=9\), respectively. The result matches the matrix product.

Worked Example: Checking the Order and Dimensions

Worked Example: Checking the Order and Dimensions

Define \(F:\mathbb{R}^2\to\mathbb{R}^3\) and \(G:\mathbb{R}^3\to\mathbb{R}^2\) by

$$ F(x,y)=(x+y,\ 2x-y,\ x^2), \qquad G(u,v,w)=(u-w,\ v+w). $$

At \((1,2)\), the intermediate point is \(F(1,2)=(3,0,1)\), and \(G(F(1,2))=(2,1)\). The Jacobians at the relevant points are

$$ J_F(1,2) = \begin{bmatrix} 1&1\\ 2&-1\\ 2&0 \end{bmatrix}, \qquad J_G(3,0,1) = \begin{bmatrix} 1&0&-1\\ 0&1&1 \end{bmatrix}. $$

The dimensions require the \(2\times3\) matrix \(J_G\) to multiply the \(3\times2\) matrix \(J_F\):

$$ J_{G\circ F}(1,2) = \begin{bmatrix} 1&0&-1\\ 0&1&1 \end{bmatrix} \begin{bmatrix} 1&1\\ 2&-1\\ 2&0 \end{bmatrix} = \begin{bmatrix} 1+0-2&1+0+0\\ 0+2+2&0-1+0 \end{bmatrix} = \begin{bmatrix} -1&1\\ 4&-1 \end{bmatrix}. $$

Direct composition gives \(H(x,y)=(x+y-x^2,\ 2x-y+x^2)\). Thus

$$ J_H(x,y) = \begin{bmatrix} 1-2x&1\\ 2+2x&-1 \end{bmatrix}, \qquad J_H(1,2) = \begin{bmatrix} -1&1\\ 4&-1 \end{bmatrix}. $$

This verifies both the product and its order. The outputs have two components, so the final Jacobian has two rows; the original inputs have two coordinates, so it has two columns.

Common Mistake: Reversing the Factors

A tempting but incorrect expression is \(J_F(\mathbf{a})J_G(F(\mathbf{a}))\). In the first worked example, this would try to multiply a \(3\times2\) matrix by a \(2\times3\) matrix. That product is defined, but it has size \(3\times3\), which cannot be the Jacobian of a map from \(\mathbb{R}^2\) to \(\mathbb{R}^2\). Worse, even in cases where reversed factors happen to be dimensionally compatible, their product does not represent the chain rule.

The correction is to follow the direction of the maps: the original input is acted on by \(J_F\), and its resulting change is acted on by \(J_G\). Therefore write \(J_G(F(\mathbf{a}))J_F(\mathbf{a})\), with the outer Jacobian on the left. Also evaluate each matrix at its own input: the inner Jacobian at \(\mathbf{a}\), the outer Jacobian at \(F(\mathbf{a})\).

Why the Matrix Form Is Useful

The matrix form is more than a compact way to write several scalar chain-rule equations. It helps prevent errors in complicated dependency structures, makes intermediate dimensions visible, and gives a reusable procedure for computing derivatives of vector-valued compositions. Once the Jacobians are known, ordinary matrix multiplication combines their local linear effects.

Before multiplying, check three things:

  • The inner map is evaluated at the original input point.
  • The outer map is evaluated at the output of the inner map.
  • The matrix sizes multiply to the dimension of the final derivative.

These checks do not replace differentiation, but they catch many mistakes before they spread through a calculation. For a composition \(\mathbb{R}^n\to\mathbb{R}^m\to\mathbb{R}^p\), the final derivative must be \(p\times n\), regardless of the formulas used to obtain it.

Check Your Understanding

Use the Jacobian dimensions and chain-rule order to answer each question.

  1. If \(F:\mathbb{R}^4\to\mathbb{R}^2\) and \(G:\mathbb{R}^2\to\mathbb{R}^3\), what are the dimensions and order of the matrices in \(J_{G\circ F}\)?
  2. At which points should \(J_F\) and \(J_G\) be evaluated for the composition \(G\circ F\) at \(\mathbf{a}\)?
  3. For \(F:\mathbb{R}^2\to\mathbb{R}^3\) followed by \(G:\mathbb{R}^3\to\mathbb{R}^2\), what size must \(J_{G\circ F}\) have?
  4. Why does the outer Jacobian appear on the left in the matrix chain rule?
  5. What dimension check can reveal that a proposed matrix product has its factors in the wrong order?

Previous

Next

Back to Tutorials