Cholesky Construction of Gaussian Vectors

From Independent Normal Inputs to a Target Covariance

A practical way to construct a multivariate Gaussian vector is to begin with independent standard normal variables and transform them linearly. The central task is choosing the transformation so that the resulting covariance is the one required. For a positive definite covariance matrix, the Cholesky factor gives a direct and uniquely specified choice.

This construction is useful both as a mathematical representation and as a method for simulation. It separates the random input, whose coordinates are independent, from the matrix calculation that creates the desired dependence. The key is to use the factor on the correct side of the covariance calculation.

Here \(L^T\) denotes the transpose of \(L\). Positive definiteness means that \(v^T\Sigma v>0\) for every nonzero vector \(v\in\mathbb{R}^p\). The condition ensures that the standard Cholesky factor exists and has nonzero diagonal entries. The vector \(Z\) is standard multivariate normal, with mean zero and covariance \(I_p\), the \(p\times p\) identity matrix.

Why the Cholesky Factor Exists and Is Unique

The factorization is not an arbitrary square-root convention: positive definiteness guarantees a lower-triangular factor with positive diagonal entries, and that convention makes the factor unique. The following argument also explains the recursive calculation used in practice.

Proof. We use induction on the dimension \(p\). For \(p=1\), a positive definite matrix is the positive number \(a\), and its factor is \(L=(\sqrt{a})\), which is unique.

For the induction step, write a symmetric positive definite matrix in block form as \(\Sigma=\begin{pmatrix}a&b^T\\ b&C\end{pmatrix}\), where \(a\) is a scalar. Positive definiteness gives \(a>0\), by testing the first coordinate vector. Define \(S=C-bb^T/a\). For any nonzero \(y\in\mathbb{R}^{p-1}\), the vector \(v=(-b^Ty/a,y^T)^T\) is nonzero, and direct multiplication gives \(v^T\Sigma v=y^TSy\). Thus \(S\) is positive definite. By the induction hypothesis, \(S=RR^T\) for a unique lower-triangular \(R\) with positive diagonal.

Now set \(L=\begin{pmatrix}\sqrt a&0\\ b/\sqrt a&R\end{pmatrix}\). Multiplying gives \(LL^T=\begin{pmatrix}a&b^T\\ b&bb^T/a+RR^T\end{pmatrix}=\Sigma\). This proves existence. For uniqueness, any lower-triangular factor with positive diagonal must have first diagonal entry \(\sqrt a\), its remaining entries in the first column must be \(b/\sqrt a\), and its trailing block must factor \(S\). The induction hypothesis makes that trailing block uniquely \(R\). Hence \(L\) is unique. \(\square\)

Verifying the Distribution and Its Parameters

The construction is Gaussian because every linear combination of its coordinates is a linear combination of independent normal variables. Its mean and covariance can also be checked directly; no inference from the shape of a simulated scatterplot is needed.

Proof. For any \(t\in\mathbb{R}^p\), \(t^TX=t^T\mu+(L^Tt)^TZ\). The characteristic function of \(Z\) is \(\exp(-u^Tu/2)\) at \(u\in\mathbb{R}^p\), since its coordinates are independent standard normals. Therefore \[ E[\exp(i t^TX)] =\exp(i t^T\mu)\exp\!\left(-\frac{1}{2}(L^Tt)^T(L^Tt)\right) =\exp\!\left(i t^T\mu-\frac{1}{2}t^TLL^Tt\right). \] This is the characteristic function of a Gaussian vector with mean \(\mu\) and covariance \(LL^T\), so it identifies the distribution. Directly, linearity of expectation gives \(E[X]=\mu+LE[Z]=\mu\). Since \(X-\mu=LZ\), its covariance is \[ \operatorname{Cov}(X)=L\operatorname{Cov}(Z)L^T=L I_p L^T=LL^T. \] Thus when \(LL^T=\Sigma\), the constructed vector has the specified Gaussian distribution and parameters. \(\square\)

Computing a Three-Dimensional Factor

For larger matrices, the same entry-matching principle proceeds down the columns. If \(\Sigma=(\sigma_{ij})\) and \(L=(l_{ij})\) is lower triangular, then \(\sigma_{ij}=\sum_{k=1}^{j}l_{ik}l_{jk}\) for \(i\ge j\). The diagonal entries are chosen positive; each new entry is determined by the already computed entries.

Covariance, Correlation, and the Side of the Factor

A covariance matrix specifies both the marginal variances and the dependence between coordinates. The diagonal entries of \(L\) alone are not generally the marginal standard deviations: those variances are the diagonal entries of \(LL^T\), which include contributions from all entries in the corresponding row of \(L\).

The product order matters. The construction is \(X=\mu+LZ\), so its covariance is \(L\operatorname{Cov}(Z)L^T=LL^T\). In general \(L^TL\) is a different matrix and does not provide the target covariance for this construction. If a different square root is used, it must be checked against the actual transformation and covariance calculation rather than assumed to be interchangeable.

Common Pitfalls and Practical Use

• Using the wrong product. Matching \(L^TL\) to \(\Sigma\) does not usually make \(\mu+LZ\) have covariance \(\Sigma\). Compute \(\operatorname{Cov}(LZ)=LL^T\).
• Forgetting to add the mean. \(LZ\) has mean zero. The prescribed nonzero mean requires the shift \(\mu\).
• Confusing factor entries with marginal standard deviations. The marginal variance of coordinate \(i\) is the sum of the squared entries in row \(i\) of \(L\), not necessarily \(l_{ii}^2\).
• Applying the standard factorization to a non-positive-definite matrix. A singular covariance does not have a Cholesky factor with strictly positive diagonal entries. The construction here assumes positive definiteness; singular Gaussian laws require a different factorization or representation.

In computation, this method turns a supply of independent normal draws into draws from a specified nonsingular multivariate Gaussian distribution. It also makes the role of the covariance model transparent: the factor mixes independent inputs, and the matrix product \(LL^T\) records exactly the variances and covariances that result.