Worked Problems on Multivariate Transformations

From a Joint Limit to Several Transformed Limits

In applications, several statistics are often calculated from the same random quantities. A sum and a product, for example, share the same inputs, so their limiting errors can be correlated. The multivariate delta method handles such quantities together: it uses the joint asymptotic distribution of the inputs and the derivative matrix of the transformation.

The calculations are compact, but they are easy to get wrong if a covariance term is omitted or a matrix product is miscomputed. The examples below show how to find the Jacobian at the limiting value, transform the full covariance matrix, and interpret the resulting joint limit.

The Multivariate Delta Method

The centering is \(g(\mu)\), and the derivative matrix is evaluated at \(\mu\), not at the observed value \(Z_n\). The theorem covers a vector of transformed statistics: each row of the Jacobian describes the first-order change in one output.

To justify the result, write \(J=Dg(\mu)\). Differentiability at \(\mu\) gives \(g(Z_n)=g(\mu)+J(Z_n-\mu)+r_n\), where \(\|r_n\|/\|Z_n-\mu\|\to 0\) as \(Z_n\to\mu\), with the remainder taken as zero when \(Z_n=\mu\). The assumed convergence implies \(\sqrt n(Z_n-\mu)=O_p(1)\), and therefore \(Z_n-\mu\xrightarrow{p}0\). It follows that \(\sqrt n\|r_n\|\xrightarrow{p}0\): it is the product of a term converging in probability to zero and a term bounded in probability. Thus \(\sqrt n\{g(Z_n)-g(\mu)\}=J\sqrt n(Z_n-\mu)+o_p(1)\). The continuous mapping theorem gives \(J\sqrt n(Z_n-\mu)\xrightarrow{d}JW\), where \(W\sim N_p(0,\Sigma)\); Slutsky’s theorem allows the \(o_p(1)\) remainder to be removed.

For completeness, the covariance transformation follows directly from the characteristic function of a multivariate normal vector. For any \(t\in\mathbb{R}^q\), \(E[\exp(i t^{\mathsf T}JW)]=\exp\{-\tfrac12(J^{\mathsf T}t)^{\mathsf T}\Sigma(J^{\mathsf T}t)\}=\exp\{-\tfrac12t^{\mathsf T}J\Sigma J^{\mathsf T}t\}\). This is the characteristic function of \(N_q(0,J\Sigma J^{\mathsf T})\), proving the claimed covariance formula. \(\square\)

Ratios and Cross-Covariances

A ratio is smooth near a point where its denominator is nonzero. When a ratio is combined with another statistic, its covariance with that statistic must be calculated from the same joint input covariance matrix. In particular, the off-diagonal entries of that matrix cannot be discarded.

Another Smooth Vector Transformation

Common Mistake: Treating the Inputs as Independent

A frequent error is to use only the diagonal entries of \(\Sigma\), as if the components of \(Z_n\) were independent. For a row vector \(a\), the correct transformed variance is \(a\Sigma a^{\mathsf T}\). In two dimensions this is \(a_1^2\Sigma_{11}+2a_1a_2\Sigma_{12}+a_2^2\Sigma_{22}\); the middle term disappears only when the input covariance \(\Sigma_{12}\) is zero.

In the logarithm-and-ratio example, the crucial multiplication is \(\Sigma(1/2,-1)^{\mathsf T}=(5/2,-3)^{\mathsf T}\), not \((0,-3)^{\mathsf T}\). The first component includes the contribution \(9/2-2=5/2\). Omitting it changes both the ratio variance and its covariance with the logarithm. A reliable practice is to write out each matrix-vector product before taking the final dot product.

The smoothness condition is also substantive. The Jacobian formulas for a logarithm require a positive argument near the limiting point; a ratio requires a denominator that is nonzero there. When the limiting point satisfies these conditions, continuity ensures the transformed sequence is in the relevant domain with probability tending to one. Transformations at a kink or at a boundary require different considerations and are not handled by this differentiable version of the method.