Deriving the Log-Determinant Differential

Why the Log-Determinant Differential Matters

The determinant summarizes how a linear transformation changes volume, while its logarithm turns products of scale factors into sums. In multivariate statistics, log-determinants arise in objectives involving covariance and precision matrices. To optimize such objectives, we need a differential that describes how a small matrix perturbation changes the log-determinant.

The central result is compact: the first-order change is the trace of the inverse matrix multiplied by the perturbation. The placement of the inverse and the meaning of the resulting gradient depend on the matrix space and inner product, so we will keep the differential and gradient distinct.

Definition: For a scalar-valued function \(f\) of a matrix \(X\), its differential at \(X\) is the linear map \(df_X\) such that \(f(X+H)=f(X)+df_X[H]+o(\|H\|_F)\) as \(H\) tends to zero. Under the Frobenius inner product \(\langle A,H\rangle_F=\operatorname{tr}(A^T H)\), the matrix gradient \(\nabla_X f\) is defined by \(df_X[H]=\langle\nabla_X f,H\rangle_F\).

Domain and the Differential Formula

For a real square matrix \(X\), \(\log\det X\) is real-valued when \(\det X>0\). Every positive-definite matrix has positive determinant, so the log-determinant is well-defined throughout the positive-definite cone. More generally, for any real invertible \(X\), \(\log|\det X|\) is defined in a neighborhood of \(X\). The absolute value is useful because an invertible matrix can have negative determinant; its determinant's sign cannot change under a sufficiently small perturbation.

Theorem: If \(X\) is a real invertible \(n\times n\) matrix, then \(d\log|\det X|[H]=\operatorname{tr}(X^{-1}H)\) for every \(n\times n\) perturbation \(H\). In particular, if \(\det X>0\), then \(d\log\det X[H]=\operatorname{tr}(X^{-1}H)\). This holds on the positive-definite cone as a special case.

To derive the result, use multiplicativity of the determinant:

\det(X+H)=\det X\,\det(I+X^{-1}H).

For any fixed matrix \(M\), the determinant expansion around the identity is

\det(I+M)=1+\operatorname{tr}(M)+O(\|M\|_F^2) \qquad\text{as }M\to 0.

The first-order term can be seen directly from the permutation formula for the determinant. The identity permutation contributes the product of the diagonal entries \(1+M_{ii}\), whose linear terms sum to \(\sum_i M_{ii}=\operatorname{tr}(M)\). Every nonidentity permutation moves at least two indices, so its term contains at least two entries from \(M\); it contributes no linear term. All remaining terms are bounded by a constant times \(\|M\|_F^2\) near zero, since the determinant is a polynomial with finitely many terms.

Set \(M=X^{-1}H\). Since multiplication by the fixed matrix \(X^{-1}\) is bounded, \(\|M\|_F=O(\|H\|_F)\), and therefore

\det(X+H)=\det X\left(1+\operatorname{tr}(X^{-1}H)+O(\|H\|_F^2)\right).

For sufficiently small \(H\), the factor in parentheses is positive and close to 1. Taking the logarithm of the absolute value therefore gives

\log|\det(X+H)| =\log|\det X|+\log\left(1+\operatorname{tr}(X^{-1}H)+O(\|H\|_F^2)\right).

The scalar expansion \(\log(1+u)=u+O(u^2)\) as \(u\to0\) now leaves the linear term \(\operatorname{tr}(X^{-1}H)\). The remainder is \(O(\|H\|_F^2)\), hence also \(o(\|H\|_F)\). This proves the stated differential. \(\square\)

Worked Example: A Directional Derivative Along a Matrix Path

Let \(X(t)=\begin{pmatrix}2+t&1\\1&3\end{pmatrix}\). Find the derivative of \(\log\det X(t)\) at \(t=0\), first using the differential and then by direct calculation.

At \(t=0\), \(X_0=\begin{pmatrix}2&1\\1&3\end{pmatrix}\) has determinant \(5\) and is positive definite: its leading principal minors are \(2>0\) and \(5>0\). The perturbation per unit change in \(t\) is \(H=X'(t)=\begin{pmatrix}1&0\\0&0\end{pmatrix}\).

The inverse at \(t=0\) is \(X_0^{-1}=\frac15\begin{pmatrix}3&-1\\-1&2\end{pmatrix}\). Thus the derivative along this path is

\left.\frac{d}{dt}\log\det X(t)\right|_{t=0} =\operatorname{tr}(X_0^{-1}H) =\operatorname{tr}\begin{pmatrix}3/5&0\\-1/5&0\end{pmatrix} =\frac35.

As a check, \(\det X(t)=3(2+t)-1=5+3t\). Hence \(\frac{d}{dt}\log\det X(t)=\frac{3}{5+3t}\), which equals \(3/5\) at zero. The matrix remains positive definite for \(t\) in a sufficiently small interval around zero, as its first leading principal minor \(2+t\) and determinant \(5+3t\) remain positive there.

From the Differential to the Gradient

The trace formula is a linear map of the perturbation \(H\). To read off a gradient, rewrite it in the Frobenius inner product's form \(\operatorname{tr}(G^T H)\). For an unrestricted real matrix \(X\), the gradient is generally not \(X^{-1}\), because the inverse need not be symmetric.

Theorem: On the open set of real invertible matrices with positive determinant, the Frobenius gradient of \(f(X)=\log\det X\) is \(\nabla_X f=X^{-T}\). On the space of symmetric positive-definite matrices, equipped with the Frobenius inner product restricted to symmetric perturbations, the gradient is \(X^{-1}\).

Indeed, for any \(H\),

df_X[H]=\operatorname{tr}(X^{-1}H) =\operatorname{tr}\big((X^{-T})^T H\big).

By the definition of the Frobenius gradient, this identifies \(\nabla_X f=X^{-T}\) on the unrestricted matrix space. If \(X\) is symmetric positive definite, then \(X^{-1}\) is also symmetric, so \(X^{-T}=X^{-1}\). Perturbations within the symmetric matrix space are themselves symmetric, and the same gradient represents the differential there. \(\square\)

Worked Example: The Gradient of a Shifted Log-Determinant

Let \(C=\begin{pmatrix}2&1\\1&2\end{pmatrix}\) and \(f(X)=\log\det(C+X)\) for symmetric positive-definite \(X\). Find the gradient at \(X_0=\begin{pmatrix}1&0\\0&2\end{pmatrix}\).

The matrix \(C\) is positive definite, and \(X_0\) is positive definite, so \(Y=C+X_0=\begin{pmatrix}3&1\\1&4\end{pmatrix}\) is positive definite. Its determinant is \(3\cdot4-1=11\), giving \(Y^{-1}=\frac1{11}\begin{pmatrix}4&-1\\-1&3\end{pmatrix}\). For a symmetric perturbation \(H\), the shift has differential \(dY[H]=H\), so the log-determinant formula yields

df_{X_0}[H]=\operatorname{tr}(Y^{-1}H) =\left\langle Y^{-1},H\right\rangle_F.

Therefore \(\nabla_X f(X_0)=\frac1{11}\begin{pmatrix}4&-1\\-1&3\end{pmatrix}\). This gradient also gives the directional derivative for any symmetric \(H\) by taking its Frobenius inner product with \(H\).

Gradients Through a Matrix Transformation

The differential is especially useful when the matrix inside a log-determinant is itself a function of \(X\). For example, take \(f(X)=\log\det(I+AXA^T)\), where \(X\) is symmetric positive definite and \(A\) is a fixed real matrix. The matrix \(I+AXA^T\) is positive definite: for nonzero \(v\), \(v^T(I+AXA^T)v=v^Tv+(A^Tv)^TX(A^Tv)>0\).

For a symmetric perturbation \(H\), the change in the matrix inside the determinant is \(AHA^T\). Applying the differential formula and then cyclically rearranging the trace gives

df_X[H] =\operatorname{tr}\big((I+AXA^T)^{-1}AHA^T\big) =\operatorname{tr}\big(A^T(I+AXA^T)^{-1}A\,H\big).

The matrix \(A^T(I+AXA^T)^{-1}A\) is symmetric, so it is the Frobenius gradient on the symmetric matrix space.

Worked Example: A Congruence-Based Log-Determinant Gradient

Use the preceding function with \(A=\begin{pmatrix}1&1\\0&1\end{pmatrix}\) and evaluate its gradient at \(X=I\). Here \(AA^T=\begin{pmatrix}2&1\\1&1\end{pmatrix}\), so \(Y=I+AA^T=\begin{pmatrix}3&1\\1&2\end{pmatrix}\). Its determinant is \(5\), and \(Y^{-1}=\frac15\begin{pmatrix}2&-1\\-1&3\end{pmatrix}\). The gradient is

A^TY^{-1}A = \begin{pmatrix}1&0\\1&1\end{pmatrix} \frac15\begin{pmatrix}2&-1\\-1&3\end{pmatrix} \begin{pmatrix}1&1\\0&1\end{pmatrix} = \frac15\begin{pmatrix}2&1\\1&3\end{pmatrix}.

To check what this means, let \(H=\begin{pmatrix}h_{11}&h_{12}\\h_{12}&h_{22}\end{pmatrix}\). The directional derivative is \(\operatorname{tr}(\nabla_X f\,H)=\frac25h_{11}+\frac25h_{12}+\frac35h_{22}\). The off-diagonal contribution appears twice in the trace, once from each symmetric position.

Common Mistakes and Why the Formula Is Useful

Using the inverse instead of the inverse transpose for an unrestricted matrix. The differential contains \(\operatorname{tr}(X^{-1}H)\), but the Frobenius gradient must satisfy \(\operatorname{tr}((\nabla_X f)^T H)=df_X[H]\). Thus the unrestricted gradient is \(X^{-T}\). It equals \(X^{-1}\) for symmetric positive-definite \(X\), which is why the distinction can be easy to miss.
Confusing a matrix gradient entry with a scalar coordinate derivative. If a symmetric matrix has one free off-diagonal coordinate \(x_{12}\), changing it changes both entries \(X_{12}\) and \(X_{21}\). For a symmetric gradient \(G\), the corresponding directional contribution is \(G_{12}H_{12}+G_{21}H_{21}=2G_{12}H_{12}\). The matrix gradient entry is not, by itself, the derivative with respect to that single coordinate.
Applying \(\log\det X\) outside its real-valued domain. For real positive-definite \(X\), the determinant is positive. For a general invertible real matrix, use \(\log|\det X|\) if a real-valued log-determinant is intended.

In applications, the formula turns a change in a matrix into a scalar first-order change that can be used in optimization or sensitivity analysis. The practical sequence is to verify the matrix inside the log-determinant is positive definite, identify its perturbation, apply the trace differential, and only then express the result as a gradient using the relevant inner product.

Key takeaway: For an invertible \(X\), the log-absolute-determinant differential is \(d\log|\det X|[H]=\operatorname{tr}(X^{-1}H)\). Under the Frobenius inner product the unrestricted gradient is \(X^{-T}\); on symmetric positive-definite matrices it is \(X^{-1}\).

Check Your Understanding

Use the differential and gradient conventions above to answer these questions.

For an invertible \(X\), what is the differential of \(\log|\det X|\) in the direction \(H\)?
Why is the unrestricted Frobenius gradient \(X^{-T}\) rather than \(X^{-1}\)?
If \(X\) is symmetric positive definite, what is the gradient of \(\log\det X\) within the symmetric matrix space?
For \(f(X)=\log\det(C+X)\), with \(C+X\) positive definite, express the gradient in terms of \(C\) and \(X\).
Why does a symmetric off-diagonal coordinate perturbation contribute twice the corresponding off-diagonal gradient entry to the directional derivative?