The Multivariate Delta Method and Jacobians

Why Transforming an Estimator Changes Its Uncertainty

Econometric questions often concern a function of several estimated parameters rather than the parameters individually. Examples include a ratio of coefficients, a sum of effects, or a contrast between two estimates. Even when the original estimator has a simple asymptotic normal distribution, the transformed estimator generally has a different variance. The multivariate delta method provides a systematic way to find that variance.

The central calculation is a first-order Taylor expansion. Its derivative is a matrix, the Jacobian, and it maps small changes in the original parameter vector into approximate changes in the transformed vector. Covariance therefore propagates through the transformation by multiplying on the left by the Jacobian and on the right by its transpose.

Setup and Main Result

Definition: Let $\theta_0$ be a $k$-dimensional parameter vector, and let $\hat{\theta}_n$ be an estimator such that $$ \sqrt{n}(\hat{\theta}_n-\theta_0)\xrightarrow{d}N(0,V), $$ where $V$ is a $k\times k$ covariance matrix. Let $g$ map $k$-dimensional vectors to $r$-dimensional vectors and be differentiable at $\theta_0$. Its Jacobian is the $r\times k$ matrix $$ Dg(\theta_0)= \left[ \frac{\partial g_i(\theta_0)}{\partial \theta_j} \right]_{i=1,\ldots,r;\,j=1,\ldots,k}. $$ Then $$ \sqrt{n}\{g(\hat{\theta}_n)-g(\theta_0)\} \xrightarrow{d} N\!\left(0,\ Dg(\theta_0)V Dg(\theta_0)'\right). $$

The dimensions help keep the calculation straight: $Dg(\theta_0)$ is $r\times k$, $V$ is $k\times k$, and the resulting covariance matrix is $r\times r$. The prime denotes transpose. The result also applies when $V$ is singular; the limiting normal distribution may then be degenerate.

The standard conditions are that the estimator has the stated joint asymptotic normal limit and that $g$ is differentiable at the parameter value in question. In the usual interior-parameter setting, $\hat{\theta}_n$ is also consistent for $\theta_0$. Differentiability in a neighborhood of that point is a common sufficient condition. The conclusion is an asymptotic distribution, not an exact finite-sample distribution.

Why the Jacobian Propagates Covariance

Differentiability gives a local linear approximation to the transformed estimator. The approximation is valid because the estimation error is of order $1/\sqrt n$, while the Taylor remainder is smaller than the size of that error.

Expand the transformation.
For $\hat{\theta}_n$ near $\theta_0$, differentiability gives $g(\hat{\theta}_n)-g(\theta_0)=Dg(\theta_0)(\hat{\theta}_n-\theta_0)+r_n$, where $\|r_n\|/\|\hat{\theta}_n-\theta_0\|\xrightarrow{p}0$.

Scale by $\sqrt n$.
The assumed limit implies $\sqrt n(\hat{\theta}_n-\theta_0)=O_p(1)$. Therefore $\sqrt n r_n=o_p(1)$: the remainder, after scaling, vanishes in probability.

Take the limiting distribution.
The leading term is $Dg(\theta_0)\sqrt n(\hat{\theta}_n-\theta_0)$. A fixed linear transformation of a mean-zero normal vector has covariance $Dg(\theta_0)V Dg(\theta_0)'$. Adding the $o_p(1)$ remainder does not change this limit.

This argument establishes the result: the transformed estimator is asymptotically the original estimation error passed through the Jacobian. When $r=1$, the Jacobian is the row vector $\nabla g(\theta_0)'$, and the asymptotic variance reduces to $\nabla g(\theta_0)'V\nabla g(\theta_0)$.

Worked Example: A Two-Component Transformation

Suppose a hypothetical estimator of $\theta_0=(2,3)'$ satisfies $\sqrt n(\hat{\theta}_n-\theta_0)\xrightarrow{d}N(0,V)$, with $V=\begin{pmatrix}4&1\\1&9\end{pmatrix}$. Consider $g(\theta_1,\theta_2)=(\theta_1+\theta_2,\ \theta_1\theta_2)'$. Find the asymptotic covariance matrix of the transformed estimator.

The Jacobian has one row for each component of $g$ and one column for each component of $\theta$: $$ Dg(\theta_1,\theta_2)= \begin{pmatrix} 1&1\\ \theta_2&\theta_1 \end{pmatrix}, \qquad Dg(2,3)= \begin{pmatrix} 1&1\\ 3&2 \end{pmatrix}. $$

First multiply the Jacobian by $V$: $$ Dg(2,3)V= \begin{pmatrix}1&1\\3&2\end{pmatrix} \begin{pmatrix}4&1\\1&9\end{pmatrix} = \begin{pmatrix}5&10\\14&21\end{pmatrix}. $$ Then multiply by the transposed Jacobian: $$ Dg(2,3)V Dg(2,3)'= \begin{pmatrix}5&10\\14&21\end{pmatrix} \begin{pmatrix}1&3\\1&2\end{pmatrix} = \begin{pmatrix}15&35\\35&84\end{pmatrix}. $$

Thus $\sqrt n\{g(\hat{\theta}_n)-g(\theta_0)\}$ converges to a mean-zero normal vector with covariance matrix $\begin{pmatrix}15&35\\35&84\end{pmatrix}$. The off-diagonal value, 35, is the asymptotic covariance between the two transformed components. The result describes their joint uncertainty, not two independent uncertainties.

Nonlinear Scalar Functions

For a scalar transformation, the Jacobian is a gradient row, so the formula gives one variance. A ratio is a useful example because both numerator and denominator are estimated and their covariance matters.

Worked Example: The Ratio of Two Estimated Parameters

Suppose $\theta_0=(2,4)'$ and $\sqrt n(\hat{\theta}_n-\theta_0)\xrightarrow{d}N(0,V)$, where $V=\begin{pmatrix}0.16&0.04\\0.04&0.09\end{pmatrix}$. The parameters are expressed in compatible units, so their ratio is meaningful. Find the asymptotic variance of $g(\theta)=\theta_1/\theta_2$, and the corresponding standard error when $n=400$.

The target ratio is $g(\theta_0)=2/4=0.5$. Its gradient is $$ Dg(\theta_1,\theta_2)= \begin{pmatrix} \frac{1}{\theta_2}&-\frac{\theta_1}{\theta_2^2} \end{pmatrix}, \qquad Dg(2,4)= \begin{pmatrix}\frac14&-\frac18\end{pmatrix}. $$

The asymptotic variance on the $\sqrt n$ scale is $$ \begin{aligned} Dg(2,4)V Dg(2,4)' &=\left(\frac14\right)^2(0.16) +\left(-\frac18\right)^2(0.09) +2\left(\frac14\right)\left(-\frac18\right)(0.04)\\ &=0.010000+0.00140625-0.002500\\ &=0.00890625. \end{aligned} $$

Consequently, $\sqrt n\{\hat{\theta}_{n,1}/\hat{\theta}_{n,2}-0.5\}$ is asymptotically normal with variance $0.00890625$. The approximate standard error of the ratio at $n=400$ is $\sqrt{0.00890625/400}=0.004719$, in the units of the ratio. This calculation uses the stated covariance matrix and evaluates the derivative at the true parameter for the theoretical limit.

The covariance contribution is negative here because the two partial derivatives have opposite signs and the covariance between the original estimates is positive. Dropping that term would give a different—and incorrect—variance.

Contrasts and Their Standard Errors

A contrast is a linear combination of parameters, often used to compare two effects. If $g(\theta)=c'\theta$ for a fixed $k$-vector $c$, then $Dg(\theta)=c'$ at every $\theta$. The delta-method variance is $c'Vc$. This includes contrasts as a special case of the general result.

Worked Example: Comparing Two Estimated Effects

Suppose two hypothetical effect estimates, $\hat{\theta}_{n,1}$ and $\hat{\theta}_{n,2}$, are measured in the same outcome units. Their joint asymptotic distribution has covariance matrix $V=\begin{pmatrix}0.25&0.10\\0.10&0.36\end{pmatrix}$ on the $\sqrt n$ scale. In a sample of $n=100$, the estimates are $1.25$ and $0.72$. Calculate the estimated contrast and its asymptotic standard error.

The contrast is $g(\theta)=\theta_1-\theta_2$, so $c=(1,-1)'$ and $Dg=c'$. The estimated contrast is $1.25-0.72=0.530$ outcome units. Its asymptotic variance on the $\sqrt n$ scale is $$ c'Vc= \begin{pmatrix}1&-1\end{pmatrix} \begin{pmatrix}0.25&0.10\\0.10&0.36\end{pmatrix} \begin{pmatrix}1\\-1\end{pmatrix} =0.25+0.36-2(0.10)=0.410. $$

The approximate variance of the estimated contrast at $n=100$ is $0.410/100=0.00410$ outcome-units squared. Its standard error is $\sqrt{0.00410}=0.06403$ outcome units. The covariance is counted twice with a minus sign because the contrast subtracts one estimate from the other. A calculation that assumes the estimates are uncorrelated would not match the stated joint covariance.

If the covariance matrix is replaced by a consistent estimate, the resulting standard error is an asymptotic plug-in estimate. Under the stated normal approximation, an approximate 95% interval would be $0.530\pm1.96(0.06403)$, or approximately $[0.4045,0.6555]$ outcome units. This interval is an inferential approximation, not a statement that the fixed contrast has a 95% probability of lying in this particular interval.

Using the Result in Practice

In applications, $V$ is usually unknown. If $\hat V$ consistently estimates $V$, and the Jacobian can be evaluated continuously at the estimated parameter, a common plug-in covariance estimate for the transformed estimator is $$ \widehat{\operatorname{Cov}}\{g(\hat{\theta}_n)\} \approx \frac{1}{n}Dg(\hat{\theta}_n)\hat V Dg(\hat{\theta}_n)'. $$ The factor $1/n$ appears because $V$ is the covariance in the limiting distribution of $\sqrt n(\hat{\theta}_n-\theta_0)$; it is not itself the approximate covariance of $\hat{\theta}_n$. For a scalar function, take the square root of this estimated variance to obtain its standard error.

A practical calculation therefore requires four choices to be made carefully: identify the parameter vector and its joint covariance, write the target function, differentiate it with respect to every component, and evaluate the resulting covariance product. Units provide a useful check: a transformed variance must have the square of the transformed quantity’s units.

Common Mistakes

Ignoring covariance. The wrong reasoning is to propagate uncertainty from each parameter separately and add only the diagonal contributions. The correction is to use the full matrix $DgVDg'$, including off-diagonal entries of $V$.
Using the wrong Jacobian orientation. For a map from $k$ inputs to $r$ outputs, the Jacobian has $r$ rows and $k$ columns. Check that the final covariance is $r\times r$.
Forgetting the sample-size scaling. If $V$ is the asymptotic covariance of $\sqrt n(\hat{\theta}_n-\theta_0)$, the approximate covariance of the unscaled transformed estimator includes $1/n$.
Treating the approximation as exact. The method gives a limiting distribution under its assumptions. It does not establish that a transformed estimator is exactly normal in a finite sample.
Evaluating derivatives at the wrong point. The theoretical limit uses $Dg(\theta_0)$. For a feasible standard error, estimate this matrix by evaluating the derivative at $\hat{\theta}_n$, when the required continuity conditions hold.

Key takeaway: If a vector estimator has asymptotic covariance $V$ on the $\sqrt n$ scale, a differentiable transformation $g$ has asymptotic covariance $Dg(\theta_0)VDg(\theta_0)'$. The Jacobian captures local sensitivity, and the full covariance matrix preserves dependence among estimated parameters.

Check Your Understanding

Use the dimensions of the Jacobian and the covariance propagation formula in each answer.

If $\hat{\theta}_n$ has three components and $g$ has two components, what are the dimensions of $Dg(\theta_0)$, $V$, and the transformed asymptotic covariance matrix?
For $g(\theta_1,\theta_2)=\theta_1+\theta_2$, write the Jacobian and the delta-method variance in terms of the entries of $V$.
Why does the variance of the contrast $\hat{\theta}_{n,1}-\hat{\theta}_{n,2}$ include minus twice the covariance?
If $\hat V$ estimates the covariance of $\sqrt n(\hat{\theta}_n-\theta_0)$, what factor is needed to estimate the covariance of $g(\hat{\theta}_n)$?
What does the multivariate delta method establish, and why should its normal approximation not be described as an exact finite-sample result?