Representing Gaussian Quadratic Forms

From Coordinates to a Matrix

A quadratic expression can be written as a sum of squared terms and cross-products, or compactly as a matrix product. The matrix notation is especially useful when the random vector has several components: it keeps the expression organized and makes changes of coordinates easier to state. The key detail is that a quadratic form depends only on the symmetric part of its coefficient matrix, even when the matrix initially supplied is not symmetric.

Throughout, vectors are column vectors, and a prime denotes transpose. Let \(X=(X_1,\ldots,X_p)'\) be a real-valued random vector and let \(A=(a_{ij})\) be a fixed real \(p\times p\) matrix. No Gaussian assumption is needed to define the form. Gaussian examples below use \(X\sim N_p(\mu,\Sigma)\), where \(\mu\) is the mean vector and \(\Sigma\) is the covariance matrix.

Definition: For a vector \(x\in\mathbb{R}^p\) and a fixed real \(p\times p\) matrix \(A\), the quadratic form with coefficient matrix \(A\) is the scalar \(q(x)=x'Ax\). For a random vector \(X\), the corresponding random variable is \(Q=X'AX\). The matrix \(A\) is called a coefficient matrix for the form; it need not be symmetric.

The word “quadratic” refers to the degree of the expression in the vector coordinates. Indeed, multiplying out the matrix product gives

x'Ax=\sum_{i=1}^p\sum_{j=1}^p a_{ij}x_ix_j =\sum_{i=1}^p a_{ii}x_i^2+ \sum_{1\leq i<j\leq p}(a_{ij}+a_{ji})x_ix_j.

The second equality groups together the two terms involving each distinct pair of coordinates: \(a_{ij}x_ix_j\) and \(a_{ji}x_jx_i\). Since \(x_ix_j=x_jx_i\), their combined coefficient is \(a_{ij}+a_{ji}\). This is why a coordinate expression specifies the diagonal entries directly but specifies each off-diagonal coefficient through a sum.

The Symmetric Part Determines the Form

Every real square matrix splits into a symmetric part and a skew-symmetric part. Define

A_{\mathrm{s}}=\frac{A+A'}{2}, \qquad A_{\mathrm{k}}=\frac{A-A'}{2}.

Then \(A=A_{\mathrm{s}}+A_{\mathrm{k}}\), \(A_{\mathrm{s}}'=A_{\mathrm{s}}\), and \(A_{\mathrm{k}}'=-A_{\mathrm{k}}\). The skew-symmetric part contributes nothing to \(x'Ax\).

Theorem: For every real vector \(x\) and every real square matrix \(A\), \(x'Ax=x'A_{\mathrm{s}}x\), where \(A_{\mathrm{s}}=(A+A')/2\). Thus \(A\) and \(A_{\mathrm{s}}\) represent the same quadratic form.

Proof. The scalar \(x'A_{\mathrm{k}}x\) equals its own transpose. Taking the transpose of this one-by-one matrix gives

x'A_{\mathrm{k}}x=(x'A_{\mathrm{k}}x)'=x'A_{\mathrm{k}}'x =-x'A_{\mathrm{k}}x.

Therefore \(2x'A_{\mathrm{k}}x=0\), so \(x'A_{\mathrm{k}}x=0\). Substituting \(A=A_{\mathrm{s}}+A_{\mathrm{k}}\) into \(x'Ax\) proves the stated identity. \(\square\)

The theorem is pointwise: it holds for every possible vector value, so it also holds for a random vector without any assumptions about its distribution or moments. In particular, \(X'AX=X'A_{\mathrm{s}}X\) for every realization of \(X\). If the coordinate expression has cross-product coefficient \(c_{ij}\) for \(x_ix_j\), a symmetric coefficient matrix representing it has entries \(a_{ij}=a_{ji}=c_{ij}/2\).

Worked Example: Replacing a Nonsymmetric Matrix

Let \(X\sim N_2((0,0)',I_2)\), and consider the matrix \(A=\begin{pmatrix}2&3\\-1&4\end{pmatrix}\). Write \(X'AX\) in coordinates and find a symmetric matrix representing the same form.

For a fixed vector \(x=(x_1,x_2)'\), direct multiplication gives \(Ax=(2x_1+3x_2,\,-x_1+4x_2)'\). Taking the inner product with \(x\),

$$ x'Ax=2x_1^2+3x_1x_2-x_1x_2+4x_2^2 =2x_1^2+2x_1x_2+4x_2^2. $$

The symmetric part is \(A_{\mathrm{s}}=\begin{pmatrix}2&1\\1&4\end{pmatrix}\), so the same expression is \(2x_1^2+2x_1x_2+4x_2^2\) when written as \(x'A_{\mathrm{s}}x\). To check numerically, at \(x=(1,2)'\) the original matrix gives \(Ax=(8,7)'\), hence \(x'Ax=8+14=22\). The symmetric matrix gives \(A_{\mathrm{s}}x=(4,9)'\), hence \(x'A_{\mathrm{s}}x=4+18=22\). These calculations check both representations.

Reading and Checking a Matrix Representation

When \(A\) is symmetric, the coordinate expansion takes a particularly simple form:

x'Ax=\sum_{i=1}^p a_{ii}x_i^2+ 2\sum_{1\leq i<j\leq p}a_{ij}x_ix_j.

The factor of \(2\) is essential: the matrix product includes both \(a_{ij}x_ix_j\) and \(a_{ji}x_jx_i\), and symmetry makes these equal. Conversely, to encode a coordinate expression in a symmetric matrix, put the coefficient of \(x_i^2\) on the diagonal and put half the coefficient of \(x_ix_j\) in each of the two corresponding off-diagonal entries. A useful check is to expand the proposed matrix product and compare every squared and cross-product term.

Worked Example: Encoding a Three-Coordinate Gaussian Expression

Let \(X\sim N_3((0,0,0)',I_3)\), and consider the expression \(X_1^2-2X_1X_2+X_2^2+2X_2X_3+3X_3^2\). Find its symmetric matrix representation and check it at \(x=(2,1,1)'\).

The squared-coordinate coefficients go on the diagonal: \(1,1,3\). The coefficient of \(x_1x_2\) is \(-2\), so the two symmetric entries are \(-1\). The coefficient of \(x_2x_3\) is \(2\), so those entries are \(1\). There is no \(x_1x_3\) term, so both entries for that pair are zero. Thus

A= \begin{pmatrix} 1&-1&0\\ -1&1&1\\ 0&1&3 \end{pmatrix}, \qquad x'Ax=x_1^2-2x_1x_2+x_2^2+2x_2x_3+3x_3^2.

At \(x=(2,1,1)'\), the coordinate calculation is \(4-4+1+2+3=6\). For the matrix calculation, \(Ax=(1,0,4)'\), so \(x'Ax=2(1)+1(0)+1(4)=6\). The two representations agree.

Centering a Gaussian Vector

A Gaussian vector is often written as its mean plus a centered random vector. This gives another useful translation between representations. Write \(X=\mu+Z\), where \(Z=X-\mu\). For any coefficient matrix \(A\), replace it first by its symmetric part \(A_{\mathrm{s}}\), which does not change the form. Expanding the matrix product then yields

X'AX=Z'A_{\mathrm{s}}Z+2\mu'A_{\mathrm{s}}Z+\mu'A_{\mathrm{s}}\mu.

The first term is quadratic in the centered coordinates, the second is linear in them, and the last is a constant. This is an algebraic identity, not a claim about the distribution of any of these terms.

Worked Example: Centering a Bivariate Gaussian Form

Let \(X\sim N_2(\mu,\Sigma)\), with \(\mu=(1,-2)'\) and \(\Sigma=\begin{pmatrix}1&1/2\\1/2&2\end{pmatrix}\). The covariance matrix is valid: its leading diagonal entry is positive and its determinant is \(2-1/4=7/4>0\). For \(A=\begin{pmatrix}2&1\\1&3\end{pmatrix}\), express \(X'AX\) in coordinates and in terms of \(Z=X-\mu\).

Since \(A\) is symmetric, its diagonal entries give the squared terms and its off-diagonal entries give twice the cross-product coefficient:

$$ X'AX=2X_1^2+2X_1X_2+3X_2^2. $$

Now \(X=Z+\mu\), with \(Z=(Z_1,Z_2)'\). The linear term is \(2\mu'AZ\). First calculate \(A\mu=(0,-5)'\), so \(2\mu'AZ=2(A\mu)'Z=-10Z_2\). The constant is \(\mu'A\mu=10\). Therefore

$$ X'AX=2Z_1^2+2Z_1Z_2+3Z_2^2-10Z_2+10. $$

For a numerical check, take \(x=(2,1)'\), so \(z=x-\mu=(1,3)'\). The original expression gives \(2(4)+2(2)(1)+3(1)=15\). The centered expression gives \(2(1)+2(1)(3)+3(9)-10(3)+10=2+6+27-30+10=15\).

Common Mistakes and Why the Representation Matters

Matrix notation is more than a compact way to write a long polynomial. It helps keep track of which coordinates interact, and it provides a consistent object for later analysis of Gaussian quadratic forms. At this stage, the essential task is to represent the same scalar accurately, not to infer a distributional property from its matrix.

Reading an off-diagonal entry as the full cross-product coefficient. For a symmetric matrix, \(a_{ij}\) contributes twice, so the coefficient of \(x_ix_j\) is \(2a_{ij}\). To encode a cross-product coefficient of \(6\), use \(3\) in each symmetric off-diagonal position.
Assuming the input matrix must be symmetric. It need not be. Replace it by \((A+A')/2\) to obtain a symmetric matrix for exactly the same form.
Keeping only one of two asymmetric cross-product contributions. For general \(A\), the coefficient of \(x_ix_j\) is \(a_{ij}+a_{ji}\), not just \(a_{ij}\).
Changing the matrix but not checking the expression. Expand the diagonal and cross-product terms, or evaluate both representations at a specific vector, to catch missing factors and sign errors.

Key takeaway: A quadratic form \(X'AX\) is determined by its squared-coordinate coefficients and the sums of paired off-diagonal coefficients. The symmetric matrix \((A+A')/2\) always represents the same form.

Check Your Understanding

Use coordinate expansion to check your answers.

For \(A=\begin{pmatrix}1&4\\2&3\end{pmatrix}\), what is the coefficient of \(x_1x_2\) in \(x'Ax\)?
Write \(5x_1^2-8x_1x_2+2x_2^2\) using a symmetric \(2\times2\) coefficient matrix.
Why does the skew-symmetric part of a real matrix contribute zero to \(x'Ax\)?
For \(A=\begin{pmatrix}2&0\\0&1\end{pmatrix}\) and \(\mu=(1,2)'\), identify the linear and constant terms when \(X=\mu+Z\).
Explain why replacing a nonsymmetric coefficient matrix by its symmetric part preserves the quadratic form for every realization of a random vector.