Higher Cumulants From Log-Partition Derivatives

Why Higher Derivatives Matter

The mean and variance describe a distribution’s location and spread, but they do not capture all of its shape. Two distributions can have the same mean and variance while differing in asymmetry or in the frequency of unusually large deviations. In a one-parameter exponential family, these additional features can be studied through the third and fourth derivatives of the log-partition function.

The third cumulant measures asymmetry on the original response scale; after standardization, it gives skewness. The fourth cumulant is related to, but is not itself, the fourth central moment. Its standardized version is excess kurtosis, which compares tail weight with that of a Gaussian distribution.

Cumulants and the Log-Partition Function

Consider a canonical one-parameter exponential family with a parameter-independent support and density or mass function, relative to a fixed measure \(\nu\),

The natural parameter space consists of the values of \(\theta\) for which the integral defining \(b(\theta)\) is finite. We work at an interior point of this space, so that \(\theta+t\) remains in the natural parameter space for all sufficiently small \(t\). This condition ensures that the moment-generating function exists in a neighborhood of zero.

Proof. For \(t\) sufficiently close to zero, the defining integral gives

Taking logarithms yields \(K_Y(t)=b(\theta+t)-b(\theta)\). Differentiating \(r\) times with respect to \(t\) and evaluating at zero gives \(K_Y^{(r)}(0)=b^{(r)}(\theta)\), as claimed.

The differentiations are justified locally: choose \(\epsilon>0\) small enough that \(\theta-\epsilon\) and \(\theta+\epsilon\) are in the natural parameter space. On any smaller interval around zero, derivatives of the integrand introduce factors \(y^r\). For each fixed \(r\), \(|y|^r e^{ty}\) is bounded by a constant times \(e^{\epsilon y}+e^{-\epsilon y}\) when \(|t|\) is sufficiently smaller than \(\epsilon\). The integrals of these exponential bounds under \(\nu\) are finite by the choice of \(\epsilon\). Differentiation under the integral is therefore justified by dominated convergence. \(\square\)

The theorem says that the first four derivatives \(b'(\theta), b''(\theta), b'''(\theta), b''''(\theta)\) are, respectively, the mean, variance, third cumulant, and fourth cumulant. To interpret the last two, we need their relationships to central moments.

Proof. Center the cumulant-generating function by defining \(C(t)=K_Y(t)-\mu t\). This is the cumulant-generating function of \(Y-\mu\), and its first derivative at zero is zero. Write its Taylor expansion through order four:

The moment-generating function of \(Y-\mu\) is \(e^{C(t)}\). Expanding the exponential through order four, the cubic coefficient is \(\kappa_3/6\), while the quartic coefficient is \(\kappa_4/24+\kappa_2^2/8\). Directly expanding \(E[e^{t(Y-\mu)}]\), the cubic and quartic coefficients are \(E[(Y-\mu)^3]/6\) and \(E[(Y-\mu)^4]/24\). Equating coefficients gives

Dividing the third central moment by \(\sigma^3\) defines skewness; dividing the fourth central moment by \(\sigma^4\) defines Pearson kurtosis. Since Pearson kurtosis equals \(3+\kappa_4/\sigma^4\), excess kurtosis is \(\kappa_4/\sigma^4\). \(\square\)

Worked Examples

Including a Fixed Dispersion Scale

Some exponential-family representations include a fixed positive scale \(a\), with parameter-dependent part \(\exp\{(y\theta-b(\theta))/a\}\). For fixed \(a\), the moment-generating function satisfies

This follows by replacing \(t\) in the exponential factor and applying the same partition-function ratio as before. Thus the scale affects cumulants of order \(r\) by the factor \(a^{r-1}\); it is important not to apply the unit-scale formula unchanged when the representation has this factor.

Common Mistake and Why the Result Matters

A frequent error is to call \(b''''(\theta)\) the fourth central moment. It is the fourth cumulant, which equals the fourth central moment minus three times the variance squared. For a Gaussian response, \(b''''(\theta)=0\), but its fourth central moment is not zero: it is \(3\sigma^4\). Confusing the two would incorrectly suggest that a Gaussian has no fourth-order variation.

The distinction is useful whenever a model’s distributional shape matters beyond its mean and variance. The sign of the third derivative indicates the direction of asymmetry, while the fourth derivative, standardized by the variance squared, describes excess kurtosis. These quantities are also a compact way to compare response families: the Bernoulli and Poisson examples have nonzero higher cumulants, whereas the Gaussian log-partition function is quadratic and all cumulants above order two vanish.