Why Condition Through the Precision Matrix?
A Gaussian conditional distribution can be calculated from either the covariance matrix or its inverse, the precision matrix. The covariance formulation expresses the conditional covariance as a Schur complement. The precision formulation has a different, often simpler appearance: the conditional covariance is the inverse of a principal precision block, and the off-diagonal precision block determines the conditional mean shift. Keeping these two descriptions straight is especially useful when a model is specified through its precision matrix.
Let \(X\in\mathbb{R}^p\) and \(Y\in\mathbb{R}^q\) be jointly Gaussian, with a positive definite covariance matrix. Write their joint mean as \((\mu_X,\mu_Y)\), and let \(\Omega\) denote the inverse of the joint covariance matrix. Partition the covariance and precision matrices according to \(X\) and \(Y\):
Here \(A\) is \(p\times p\), \(B\) is \(p\times q\), and \(D\) is \(q\times q\). Positive definiteness of \(\Omega\) implies that its principal block \(A\) is positive definite, so \(A^{-1}\) exists.
Deriving the Conditional Distribution
The joint Gaussian density is proportional, as a function of \(x\) and \(y\), to the exponential of minus one half of the centered quadratic form. Put \(u=x-\mu_X\) and \(v=y-\mu_Y\). In terms of the precision blocks, that quadratic form is
When \(y\) is fixed, the term \(v^TDv\) does not depend on \(x\). To isolate the terms that do, complete the square:
The last term on the right also does not depend on \(x\). Since \(A\) is positive definite, the resulting kernel in \(x\) is a Gaussian density with precision \(A\), covariance \(A^{-1}\), and mean satisfying \(x-\mu_X=-A^{-1}Bv\). This proves the theorem. The joint density is positive everywhere, so this density calculation gives the conditional distribution for every \(y\in\mathbb{R}^q\). \(\square\)
The word “precision” can make the result easy to misread. The conditional precision of \(X\) given \(Y\) is \(A\), the \(XX\) principal block of the joint precision matrix. The conditional covariance is its inverse \(A^{-1}\), not \(A\) itself. The block \(B\) controls how the observed deviation \(y-\mu_Y\) changes the conditional mean.
Worked Example: A Two-Variable Gaussian
Suppose \((X,Y)\) is jointly Gaussian with mean \((2,-1)\) and precision matrix
The matrix is positive definite: its leading entry is \(2>0\), and its determinant is \(2(3)-1^2=5>0\). For the partition with \(X\) first, \(A=2\) and \(B=1\). Therefore
For example, at \(y=1\), the centered observation is \(y-\mu_Y=1-(-1)=2\). The conditional mean is \(2-\frac12(2)=1\), and the conditional variance is \(\frac12\).
Check against the covariance formulation by inverting the precision matrix:
The covariance formula gives conditional variance \(\frac35-\frac{(-1/5)^2}{2/5}=\frac35-\frac1{10}=\frac12\). Its conditional mean is \(2+\frac{\Sigma_{XY}}{\Sigma_{YY}}(y+1)=2-\frac12(y+1)\), exactly as obtained from the precision blocks.
Why the Covariance and Precision Formulas Agree
For comparison, the covariance-matrix expression for Gaussian conditioning is
The covariance term is the Schur complement of \(\Sigma_{YY}\) in \(\Sigma\). The next result shows directly how these expressions reduce to the precision-block formulas.
Proof. Define the Schur complement \(S=D-B^TA^{-1}B\). It is positive definite: for any nonzero \(z\in\mathbb{R}^q\), take \(w=(-A^{-1}Bz,z)\). Then \(w\ne0\), and \(z^TSz=w^T\Omega w>0\). In particular, \(S^{-1}\) exists. Block elimination, or direct multiplication by \(\Omega\), gives
Consequently, \(\Sigma_{YY}=S^{-1}\) and \(\Sigma_{XY}=-A^{-1}BS^{-1}\), so \(\Sigma_{XY}\Sigma_{YY}^{-1}=-A^{-1}B\). Also, \[ \Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YX} =(-A^{-1}B) (-S^{-1}B^TA^{-1}) =A^{-1}BS^{-1}B^TA^{-1}. \] Subtracting this from the displayed expression for \(\Sigma_{XX}\) leaves \(A^{-1}\). Both claimed identities follow. \(\square\)
This equivalence is useful in either direction. A covariance matrix gives a conditional covariance through a Schur complement; a precision matrix gives it by inverting a principal block. These are not competing formulas but two ways of describing the same conditional distribution.
Worked Example: A Two-Dimensional Conditioned Block
Let \(X=(X_1,X_2)^T\) and let \(Y\) be scalar, with mean \(\mu_X=(2,-1)^T\), \(\mu_Y=4\), and precision matrix
Here \(A^{-1}=\begin{bmatrix}1/2&0\\0&1\end{bmatrix}\). The precision matrix is positive definite because \(A\) is positive definite and its Schur complement is \(3-B^TA^{-1}B=3-(1/2+1)=3/2>0\). Thus, for \(d=y-4\),
At \(y=6\), \(d=2\), so the conditional mean is \((1,1)^T\) and the conditional covariance remains \(\operatorname{diag}(1/2,1)\). The off-diagonal precision block couples the observation to both components, with opposite signs; it does not imply that the conditional covariance has an off-diagonal entry.
Worked Example: Conditioning a Scalar on Two Variables
Now let \(X\) be scalar and \(Y=(Y_1,Y_2)^T\), with mean \(\mu_X=1\), \(\mu_Y=(0,2)^T\), and precision
In this partition, \(A=4\), \(B=(1,-1)\), and \(D=\operatorname{diag}(3,2)\). Positive definiteness follows because \(D\) is positive definite and \(A-BD^{-1}B^T=4-\frac13-\frac12=\frac{19}{6}>0\). The precision-block result therefore gives conditional variance \(1/4\) and conditional mean
For \(y=(3,1)^T\), the centered components are \(3\) and \(-1\), so their difference is \(4\). The conditional mean is \(1-\frac14(4)=0\), and the conditional variance is \(1/4\).
For an explicit covariance check, the block inverse formula gives \[ \Sigma_{XX}=\frac6{19},\quad \Sigma_{XY}=\begin{bmatrix}-2/19&3/19\end{bmatrix},\quad \Sigma_{YY}=\frac1{19}\begin{bmatrix}7&-1\\-1&11\end{bmatrix}. \] Indeed, \(\Sigma_{YY}(-1/4,1/4)^T=(-2/19,3/19)^T=\Sigma_{YX}\). Hence \(\Sigma_{XY}\Sigma_{YY}^{-1}=(-1/4,1/4)\), matching the conditional-mean coefficient. The covariance correction is \(\Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YX}=(-2/19)(-1/4)+(3/19)(1/4)=5/76\). Thus the conditional variance is \(6/19-5/76=24/76-5/76=19/76=1/4\), as required.
Common Mistake and Practical Interpretation
A frequent error is to read the \(XX\) precision block \(A\) as the conditional covariance. That reverses the role of a matrix and its inverse: \(A\) is the conditional precision, while \(A^{-1}\) is the conditional covariance. In the first example, using \(2\) as the conditional variance would be wrong; the variance is \(1/2\).
Another easy-to-miss detail is the sign in the conditional mean. Completing the square gives the shifted center \(u+A^{-1}Bv\), so the conditional mean is \(\mu_X-A^{-1}Bv\), with a minus sign. The covariance formula provides a useful check: the identity \(\Sigma_{XY}\Sigma_{YY}^{-1}=-A^{-1}B\) must give the same coefficient.
Conditioning through precision blocks is particularly convenient when the precision matrix is already available, as in some Gaussian models. It lets you compute the conditional distribution without first forming every block of the covariance matrix. The calculation still depends on the chosen partition: \(A\) must correspond to the variables whose conditional distribution is sought, and \(B\) must pair those variables with the ones observed.
Check Your Understanding
Use the block formulas and the examples above to check your reasoning.
- For a positive definite Gaussian precision matrix partitioned as \(\begin{bmatrix}A&B\\B^T&D\end{bmatrix}\), which block determines the conditional covariance of \(X\) given \(Y\), and how?
- Why does the conditional mean involve \(y-\mu_Y\) rather than \(y\) alone?
- For scalar \(X,Y\) with precision matrix \(\begin{bmatrix}5&-2\\-2&4\end{bmatrix}\), state the conditional variance of \(X\) given \(Y\) and the coefficient of \(y-\mu_Y\) in its conditional mean.
- Explain why the covariance formula for conditional variance and the inverse of the \(XX\) precision block give the same matrix.
- What is the common mistake in interpreting the \(XX\) precision block as a conditional covariance?