Moment-Generating Functions of Gaussian Quadratic Forms

From Gaussian Quadratic Forms to Transform Formulas

A Gaussian quadratic form can combine several squared coordinates, possibly with different weights and with a shift. Its distribution may not be a familiar named law, but its moment-generating function has a compact expression involving a determinant. That expression is useful only when its domain is stated precisely: for some real values of the transform argument, the defining expectation is infinite.

We consider a standard Gaussian vector and a fixed shift. This setting makes the role of the eigenvalues transparent, while also showing how the shift changes the exponential factor. The characteristic function follows from the same Gaussian integral, evaluated at imaginary arguments.

Definition: Let $Z\sim N_p(0,I_p)$, let $m\in\mathbb{R}^p$ be fixed, and let $A$ be a real symmetric $p\times p$ matrix. The Gaussian quadratic form considered here is $Q=(Z+m)'A(Z+m)$. Its moment-generating function at a real number $t$ is $M_Q(t)=E[\exp(tQ)]$, allowing the value $+\infty$.

Symmetry of $A$ entails no loss for a general quadratic expression $x'Ax$: its value equals $x'[(A+A')/2]x$. Write the eigenvalues of $A$ as $\lambda_1,\ldots,\lambda_p$. By the spectral theorem, an orthogonal change of coordinates diagonalizes $A$. Applying the same change to $Z$ preserves its standard Gaussian distribution, so in those coordinates the transformed entries of $Z$ are still independent standard normal random variables. Write the coordinates of the transformed shift as $c_1,\ldots,c_p$. Then $Q=\sum_{j=1}^p\lambda_j(Z_j+c_j)^2$, where the $Z_j$'s in this representation are independent $N(0,1)$ variables.

The One-Dimensional Gaussian Integral

The multivariate formula follows by multiplying one-dimensional calculations. For $\lambda,c,t\in\mathbb{R}$, consider one term $\lambda(Z+c)^2$, where $Z\sim N(0,1)$. The density gives the integral below when $d=1-2t\lambda>0$.

\begin{aligned} E\!\left[e^{t\lambda(Z+c)^2}\right] &=\frac{1}{\sqrt{2\pi}}\int_{\mathbb{R}} \exp\!\left\{-\frac{z^2}{2}+t\lambda(z+c)^2\right\}\,dz\\ &=\frac{1}{\sqrt{2\pi}}\int_{\mathbb{R}} \exp\!\left\{-\frac{d z^2}{2}+2t\lambda c z+t\lambda c^2\right\}\,dz\\ &=d^{-1/2} \exp\!\left\{t\lambda c^2+\frac{(2t\lambda c)^2}{2d}\right\}\\ &=(1-2t\lambda)^{-1/2} \exp\!\left\{\frac{t\lambda c^2}{1-2t\lambda}\right\}. \end{aligned}

For the third line, completing the square in $z$ contributes the factor $d^{-1/2}$ from the Gaussian integral and the constant $(2t\lambda c)^2/(2d)$. For the last line, $d+2t\lambda=1$, so $t\lambda c^2+(2t\lambda c)^2/(2d)=t\lambda c^2/d$. If $\lambda=0$, the random term is identically zero and the expectation is $1$, consistent with the formula.

Theorem: For $Q=(Z+m)'A(Z+m)$, with $Z\sim N_p(0,I_p)$, fixed $m$, and real symmetric $A$, the moment-generating function is finite exactly when $I_p-2tA$ is positive definite. On that domain, $$ M_Q(t)=\det(I_p-2tA)^{-1/2} \exp\!\left\{t\,m'A(I_p-2tA)^{-1}m\right\}. $$ Equivalently, if $\lambda_j$ and $c_j$ are the eigenvalues and transformed shift coordinates described above, then $$ M_Q(t)=\prod_{j=1}^p(1-2t\lambda_j)^{-1/2} \exp\!\left\{\sum_{j=1}^p\frac{t\lambda_jc_j^2}{1-2t\lambda_j}\right\}. $$

Proof and Exact Domain

In eigen-coordinates, independence makes the expectation a product of one-dimensional expectations whenever those expectations are finite. For a coordinate with eigenvalue $\lambda_j$, the exponent in its Gaussian integral has quadratic coefficient $-\tfrac12(1-2t\lambda_j)z_j^2$. If $1-2t\lambda_j>0$, completing the square gives the finite factor derived above.

If $1-2t\lambda_j<0$, the integrand grows quadratically in at least one tail, so its integral diverges. If $1-2t\lambda_j=0$, the quadratic term cancels. The remaining integrand is the exponential of a linear function of $z_j$ times a positive constant; it is either constant or fails to decay in one tail, and is not integrable over $\mathbb{R}$. Thus the full expectation is finite precisely when every factor $1-2t\lambda_j$ is strictly positive. This is equivalent to positive definiteness of $I_p-2tA$.

On that domain, multiplying the one-dimensional factors gives the product formula. Orthogonal diagonalization also gives $\det(I_p-2tA)=\prod_j(1-2t\lambda_j)$ and $m'A(I_p-2tA)^{-1}m=\sum_j\lambda_jc_j^2/(1-2t\lambda_j)$. Substitution yields the determinant expression. This proves both the formula and its exact real domain. $\square$

The domain can be read directly from the eigenvalues. Every positive eigenvalue requires $t<1/(2\lambda_j)$, while every negative eigenvalue requires $t>1/(2\lambda_j)$. Zero eigenvalues impose no restriction. The intersection of these inequalities is an open interval containing zero; if $A=0$, the interval is all of $\mathbb{R}$ and $Q=0$.

Worked Examples

Worked Example: A Central Form With Unequal Weights

Let $Z=(Z_1,Z_2)'\sim N_2(0,I_2)$, $m=0$, and $A=\operatorname{diag}(2,1)$. Then $Q=2Z_1^2+Z_2^2$. The eigenvalues are $2$ and $1$, so the domain conditions are $1-4t>0$ and $1-2t>0$. Together they give $t<1/4$.

The formula gives

M_Q(t)=(1-4t)^{-1/2}(1-2t)^{-1/2}, \qquad t<\frac14.

As a check, the derivative of the log of this expression at zero is $\left.\frac{d}{dt}\{-\tfrac12\log(1-4t)-\tfrac12\log(1-2t)\}\right|_{t=0}=2+1=3$. Directly, $E[Q]=2E[Z_1^2]+E[Z_2^2]=2+1=3$, in agreement.

Worked Example: A Shifted Form With a Zero Eigenvalue

Let $Z=(Z_1,Z_2,Z_3)'\sim N_3(0,I_3)$, $m=(0,5,-1)'$, and $A=\operatorname{diag}(0,-2,\tfrac12)$. Keeping the shift in each coordinate gives

Q=(Z+m)'A(Z+m) =-2(Z_2+5)^2+\frac12(Z_3-1)^2.

The first coordinate contributes nothing because its eigenvalue is zero. The negative eigenvalue $-2$ requires $1+4t>0$, or $t>-\tfrac14$. The positive eigenvalue $\tfrac12$ requires $1-t>0$, or $t<1$. Hence the exact domain is $-\tfrac14<t<1$.

The two nonzero coordinates contribute

M_Q(t)=(1+4t)^{-1/2}(1-t)^{-1/2} \exp\!\left\{-\frac{50t}{1+4t}+\frac{t/2}{1-t}\right\}, \qquad -\frac14<t<1.

The shift terms are essential: the coordinate with eigenvalue $-2$ contributes $-2t(5^2)/(1+4t)=-50t/(1+4t)$, and the coordinate with eigenvalue $\tfrac12$ contributes $(t/2)(1^2)/(1-t)$. A derivative check at zero gives $(\log M_Q)'(0)=(-2+\tfrac12)+(-50+\tfrac12)=-51$. Directly, $E[Q]=-2E[(Z_2+5)^2]+\tfrac12E[(Z_3-1)^2]=-2(1+25)+\tfrac12(1+1)=-51$.

Worked Example: An Indefinite Form and Its Domain

Let $A=\operatorname{diag}(1,-1)$ and $m=(1,2)'$. Then $Q=(Z_1+1)^2-(Z_2+2)^2$, for independent standard normal coordinates. The positive eigenvalue requires $t<\tfrac12$, and the negative eigenvalue requires $t>-\tfrac12$. The intersection is $-\tfrac12<t<\tfrac12$.

The determinant and shift terms give

M_Q(t)=(1-2t)^{-1/2}(1+2t)^{-1/2} \exp\!\left\{\frac{t}{1-2t}-\frac{4t}{1+2t}\right\}, \qquad -\frac12<t<\frac12.

For example, at $t=\tfrac14$, the eigenvalue conditions are $1-2t=\tfrac12>0$ and $1+2t=\tfrac32>0$, so the formula is finite. At $t=\tfrac12$, the first coordinate has a zero quadratic coefficient in the transformed Gaussian integral, and its integral diverges. An indefinite form therefore has a finite interval of real arguments, not a moment-generating function finite for every real $t$.

The Characteristic Function

The characteristic function is $\varphi_Q(u)=E[e^{iuQ}]$ for real $u$. Its expectation exists for every real $u$, since $|e^{iuQ}|=1$. Substituting $t=iu$ in the transform formula gives

Theorem: For every real $u$, $$ \varphi_Q(u)= \prod_{j=1}^p(1-2iu\lambda_j)^{-1/2} \exp\!\left\{\sum_{j=1}^p\frac{iu\lambda_jc_j^2}{1-2iu\lambda_j}\right\}. $$ The square roots are chosen continuously from the value $1$ at $u=0$. Equivalently, the product factor is $\det(I_p-2iuA)^{-1/2}$ with that same continuous choice of branch.

For justification, in each eigen-coordinate the Gaussian integral has complex quadratic coefficient $1-2iu\lambda_j$, whose real part is $1$. The Gaussian integral identity $\int_{\mathbb R}\exp(-az^2/2+bz)\,dz=(2\pi)^{1/2}a^{-1/2}\exp(b^2/(2a))$ holds for complex $a,b$ with $\operatorname{Re}(a)>0$, using the square-root branch continuous from positive real $a$. It follows by the real Gaussian integral and analytic continuation on the right half-plane; the integrals there are locally dominated by integrable Gaussian bounds. Taking $a=1-2iu\lambda_j$ and $b=2iu\lambda_jc_j$, and including the constant $iu\lambda_jc_j^2$, gives the stated factor. Multiplication over the independent coordinates proves the formula. $\square$

Worked Example: Characteristic Function of the Indefinite Form

For the form in the previous example, replace $t$ by $iu$ in its eigenvalue expression. For every real $u$,

\varphi_Q(u)=(1-2iu)^{-1/2}(1+2iu)^{-1/2} \exp\!\left\{\frac{iu}{1-2iu}-\frac{4iu}{1+2iu}\right\}.

The two denominators cannot vanish for real $u$, because $|1\pm2iu|^2=1+4u^2>0$. The square-root factors equal $1$ at $u=0$, so this branch choice gives $\varphi_Q(0)=1$, as required for any characteristic function. The characteristic function is defined for all real $u$, even though the real moment-generating function above is finite only for $-\tfrac12<t<\tfrac12$.

Common Pitfalls and Why the Formula Matters

Dropping the shift inside the quadratic form. For $Q=(Z+m)'A(Z+m)$, a coordinate with shift $c_j$ contributes $\lambda_j(Z_j+c_j)^2$. Its exponential factor depends on $c_j^2$; replacing it by $\lambda_jZ_j^2$ changes the distribution and gives the wrong transform.
Reporting a determinant without its domain. The algebraic expression may be written when a factor $1-2t\lambda_j$ is nonpositive, but the original expectation then diverges. The real MGF formula applies only when every such factor is strictly positive.
Confusing real and imaginary transform arguments. The real MGF can have a limited interval of finiteness. The characteristic function nevertheless exists for every real $u$, and its formula uses the complex Gaussian integral with $t=iu$.
Ignoring zero eigenvalues. A zero eigenvalue contributes neither a determinant restriction nor a shift term. Its coordinate does not affect $Q$, regardless of the corresponding entry of $m$.

The determinant expression packages a collection of independent one-dimensional Gaussian integrals into a coordinate-free formula. It gives the transform while preserving the information needed to interpret it: eigenvalues determine where the real expectation is finite, and the shift determines the additional exponential factor. The eigen-coordinate and product forms are often the simplest way to check the result in a particular model.

Check Your Understanding

Use the eigenvalue conditions and the one-dimensional formula to answer each question.

For $Q=3Z_1^2-Z_2^2$, where $Z_1,Z_2$ are independent standard normal variables, find the exact real domain of $M_Q(t)$.
If one eigenvalue of $A$ is zero, explain why its transformed shift coordinate does not appear in the moment-generating function.
For $Q=-(Z+2)^2$, with $Z\sim N(0,1)$, write $M_Q(t)$ and state its exact domain.
Why is the characteristic function defined for every real $u$, even when the real moment-generating function is not finite for every $t$?
For a symmetric matrix with both positive and negative eigenvalues, express the MGF domain as inequalities involving each positive and each negative eigenvalue.