Deriving Intrinsic CAR Precision and Its Null Space

From Neighbourhood Structure to Precision

An intrinsic conditional autoregressive (ICAR) effect describes spatial variation through differences between neighbouring areas. Its precision matrix does not directly penalize the overall level of the effects: it penalizes changes across edges in the neighbourhood graph. That distinction is useful for representing local smoothing, but it also means the precision matrix is singular. Deriving the matrix and its null space makes clear exactly what is and is not identified by the spatial prior.

Suppose there are \(n\) areas. Represent their neighbourhoods by a symmetric matrix \(W=(w_{ij})\), where \(w_{ij}=w_{ji}\geq 0\), \(w_{ii}=0\), and a positive \(w_{ij}\) indicates a weighted connection between areas \(i\) and \(j\). For an unweighted adjacency graph, \(w_{ij}\) is 1 for neighbours and 0 otherwise. Let \(d_i=\sum_{j=1}^n w_{ij}\) be the total weight attached to area \(i\), and let \(D\) be the diagonal matrix with entries \(d_i\).

Definition: The intrinsic CAR precision matrix is \(Q=D-W\). For a positive smoothing precision parameter \(\tau\), its quadratic penalty for a latent-effect vector \(u=(u_1,\ldots,u_n)^\mathsf{T}\) is \(\tau u^\mathsf{T}Qu\). The corresponding intrinsic Gaussian kernel is proportional to \(\exp(-\tau u^\mathsf{T}Qu/2)\); it is not a proper density on all of \(\mathbb{R}^n\) when the graph has at least one connected component.

The graph is assumed to be undirected, with finite nonnegative weights. These conditions ensure that \(Q\) is symmetric. The usual conditional specification also requires \(d_i>0\) for every area to which it is applied. An isolated area, with \(d_i=0\), has no neighbours and is not governed by the usual neighbour-average conditional distribution.

Deriving the Quadratic Form and Conditional Distributions

For any vector \(u\), expand the quadratic form using \(Q=D-W\):

u^\mathsf{T}Qu =\sum_{i=1}^n d_i u_i^2-\sum_{i=1}^n\sum_{j=1}^n w_{ij}u_i u_j =\frac{1}{2}\sum_{i=1}^n\sum_{j=1}^n w_{ij}(u_i-u_j)^2.

The last equality follows because symmetry gives \(\sum_{i,j}w_{ij}u_j^2=\sum_{i,j}w_{ij}u_i^2=\sum_i d_i u_i^2\). Thus the expression on the right is nonnegative and charges a penalty only when connected areas have different effects. Since each undirected edge appears twice in the double sum, the same quadratic form can be written as \(\sum_{\{i,j\}}w_{ij}(u_i-u_j)^2\), where the sum is over unordered edges counted once.

To obtain the conditional distribution at area \(i\), collect the terms of \(u^\mathsf{T}Qu\) involving \(u_i\), holding all other effects fixed. Those terms are \(d_i u_i^2-2u_i\sum_jw_{ij}u_j\). Completing the square yields the standard intrinsic CAR conditional kernel:

u_i\mid u_{-i} \ \propto\ \exp\left[-\frac{\tau d_i}{2} \left(u_i-\frac{\sum_j w_{ij}u_j}{d_i}\right)^2\right], \qquad u_i\mid u_{-i}\sim N\left(\frac{\sum_jw_{ij}u_j}{d_i},\frac{1}{\tau d_i}\right), \quad d_i>0.

The conditional mean is the weighted average of neighbouring effects; the conditional variance decreases as either \(\tau\) or the total neighbour weight \(d_i\) increases. These conditionals describe local dependence, but they do not make the joint distribution proper: a joint kernel can be integrable in some directions and flat in others.

Worked Example: Deriving the Precision for a Three-Area Path

Consider three hypothetical areas arranged in a path: area 1 is adjacent to 2, and area 2 is adjacent to 3. Each edge has weight 1. The adjacency matrix, degree matrix, and resulting precision matrix are

W= \begin{pmatrix} 0&1&0\\ 1&0&1\\ 0&1&0 \end{pmatrix}, \qquad D= \begin{pmatrix} 1&0&0\\ 0&2&0\\ 0&0&1 \end{pmatrix}, \qquad Q=D-W= \begin{pmatrix} 1&-1&0\\ -1&2&-1\\ 0&-1&1 \end{pmatrix}.

For hypothetical effects \(u=(1,0,-1)^\mathsf{T}\), the edge-difference calculation gives \(u^\mathsf{T}Qu=(1-0)^2+(0-(-1))^2=1+1=2\). Direct multiplication agrees: \(Qu=(1,0,-1)^\mathsf{T}\), so \(u^\mathsf{T}Qu=1(1)+0(0)+(-1)(-1)=2\). The exponent in the intrinsic Gaussian kernel is therefore \(-\tau(2)/2=-\tau\).

The conditionals are \(u_1\mid u_2\sim N(u_2,1/\tau)\), \(u_2\mid u_1,u_3\sim N((u_1+u_3)/2,1/(2\tau))\), and \(u_3\mid u_2\sim N(u_2,1/\tau)\). Each variance follows from \(1/(\tau d_i)\), using degrees 1, 2, and 1 respectively.

Null Space: Constants Within Components

The edge-difference representation proves both positive semidefiniteness and the exact null space. A vector has zero quadratic form if and only if every nonnegative term for an edge with positive weight is zero. In other words, \(u_i=u_j\) along every edge. Following paths in the graph then shows that the effects are constant within each connected component. Conversely, a vector constant within each component has equal values at the ends of every edge, so its quadratic form is zero and \(Qu=0\).

Result: If the adjacency graph has \(c\) connected components, the null space of \(Q\) has dimension \(c\). It is spanned by the \(c\) component-indicator vectors, each equal to 1 on one component and 0 elsewhere. For a connected graph, the null space is exactly the span of the all-ones vector \(\mathbf{1}\), and \(Q\mathbf{1}=0\).

This result explains the word intrinsic: the precision does not supply information about a component’s constant level. For a connected graph, replacing \(u\) by \(u+a\mathbf{1}\) leaves all edge differences and the penalty unchanged. A singular precision therefore cannot define an ordinary, normalized Gaussian density over all \(n\) effect coordinates.

Worked Example: Finding the Null Space for a Disconnected Graph

Suppose four hypothetical areas form two separate edges: areas 1 and 2 are connected with weight 2, and areas 3 and 4 are connected with weight 1. There are no connections between the pairs. Then

Q= \begin{pmatrix} 2&-2&0&0\\ -2&2&0&0\\ 0&0&1&-1\\ 0&0&-1&1 \end{pmatrix}.

The quadratic form is \(u^\mathsf{T}Qu=2(u_1-u_2)^2+(u_3-u_4)^2\). It is zero exactly when \(u_1=u_2\) and \(u_3=u_4\). Thus the null space is spanned by \((1,1,0,0)^\mathsf{T}\) and \((0,0,1,1)^\mathsf{T}\); its dimension is 2, the number of connected components.

For example, \(u=(3,3,-2,-2)^\mathsf{T}\) is in this null space, even though its values differ between components. One overall constraint \(\sum_i u_i=0\) would not remove both null directions: the nonzero vector \((1,1,-1,-1)^\mathsf{T}\) satisfies that constraint and still has \(u^\mathsf{T}Qu=0\). A constraint is needed within each component, such as \(u_1+u_2=0\) and \(u_3+u_4=0\).

Constraints and a Proper Latent-Effect Distribution

A constraint removes the unpenalized constant direction while retaining the relative spatial contrasts. For a connected graph, the standard sum-to-zero constraint is \(\sum_i u_i=0\). For \(c\) components, impose one such constraint separately within each component. On this restricted subspace, every nonzero allowable vector has \(u^\mathsf{T}Qu>0\), so the Gaussian kernel can be normalized as a distribution on that subspace. This is a proper distribution relative to the subspace, not a full-dimensional density on \(\mathbb{R}^n\).

\sum_{i\in C_k}u_i=0,\quad k=1,\ldots,c, \qquad u^\mathsf{T}Qu>0\ \text{for every nonzero }u \text{ satisfying these constraints}.

For a connected graph, the constrained subspace has dimension \(n-1\). More generally, its dimension is \(n-c\), provided the components contain no isolated singleton that is forced to have its only effect equal to zero. A single overall sum-to-zero constraint is sufficient only when the graph is connected. With multiple components, it leaves \(c-1\) unpenalized component-level directions.

The need for constraints can also be seen in a GLMM linear predictor. If an intercept \(\beta_0\) and spatial effects \(u_i\) both enter as \(\beta_0+u_i\), then adding a constant \(a\) to every \(u_i\) and subtracting \(a\) from \(\beta_0\) leaves every predictor unchanged. The intrinsic penalty also remains unchanged. A sum-to-zero constraint selects a decomposition in which the intercept represents the overall level and the spatial effects represent deviations around it; it does not change the edge-based smoothing penalty.

Worked Example: Centering Effects Without Changing Spatial Contrasts

Consider four hypothetical areas on a path and raw latent effects \(u=(2,1,0,-1)^\mathsf{T}\). Their mean is \((2+1+0-1)/4=0.5\). Subtracting this mean gives constrained effects \(u^*=u-0.5\mathbf{1}=(1.5,0.5,-0.5,-1.5)^\mathsf{T}\), whose sum is \(1.5+0.5-0.5-1.5=0\).

For this path, \(u^\mathsf{T}Qu=(2-1)^2+(1-0)^2+(0-(-1))^2=3\). Centering does not alter any difference: \(u^*_{i}-u^*_{i+1}=u_i-u_{i+1}\). Therefore \((u^*)^\mathsf{T}Qu^*=1^2+1^2+1^2=3\). The constraint removes the overall level but leaves the spatial contrasts and their penalty intact.

If the GLMM intercept is hypothetically \(\beta_0=0.2\), the predictors before centering are \(0.2+u_i\). Using \(u^*\) and \(\beta_0^*=0.7\) gives the same predictors, because \(0.7+u_i-0.5=0.2+u_i\) for every area. Thus the constraint chooses a reference for separating the intercept from the spatial effects; it does not alter fitted linear predictors under this shift.

Assumptions, Diagnostics, and Alternatives

The derivation relies on a symmetric, nonnegative adjacency structure and on the stated graph defining the intended neighbourhoods. Check that \(W\) is symmetric, that its diagonal is zero, and that weights are finite and nonnegative. Inspect connected components and degrees before fitting. A zero degree identifies an isolated area; the standard conditional mean and variance divide by \(d_i\), so they are undefined there. That node requires a deliberate modeling decision, such as revising the graph or assigning a separate effect specification, rather than silently applying the usual ICAR conditional.

A useful numerical check is that \(Q\) is symmetric, \(Q\mathbf{1}=0\) within each connected component, and its quadratic form is nonnegative. A negative eigenvalue beyond numerical rounding can indicate a construction error, such as asymmetric weights or an incorrect diagonal. A rank deficiency equal to the number of connected components is expected, not evidence of a failed computation. Conversely, if an allegedly connected graph has additional null directions, inspect its edges and matrix construction.

The ICAR approach is a graph-based alternative to spatial models specified through a covariance function of distance: it uses adjacency and local differences rather than a distance-based covariance matrix. It is also distinct from a proper CAR model, which is an alternative spatial specification with a different precision construction. The intrinsic model’s defining singularity is not a minor numerical detail; it governs constraints, interpretation, and which latent directions the prior can control.

Common mistake: “The precision matrix is singular, so the model must be invalid.” Correction: Singularity is intrinsic to this construction. The kernel is improper on the full space, but becomes proper on the subspace after one independent constraint per connected component.
Another pitfall: imposing only one global sum-to-zero constraint when the graph is disconnected. That leaves unpenalized component shifts; impose a separate constraint for every component.
Limitation: the edge penalty controls relative differences, not a component’s overall level. Constraints resolve that null direction but do not add information about disconnected components’ relative levels.

Key takeaway: For a symmetric adjacency graph, \(Q=D-W\) penalizes squared differences across edges. Its null space contains exactly the vectors constant on each connected component, so a proper constrained ICAR distribution requires one sum-to-zero constraint per component.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

For a connected graph with \(n\) areas, what is the dimension and a basis vector for the null space of \(Q\)?
For an unweighted path of three areas, calculate \(u^\mathsf{T}Qu\) for \(u=(0,2,2)^\mathsf{T}\).
A graph has three connected components. How many independent sum-to-zero constraints are needed to remove all null directions?
Why does an isolated area make the usual ICAR conditional formula undefined?
In a GLMM with predictor \(\beta_0+u_i\), explain why centering the spatial effects can be accompanied by an intercept adjustment without changing the predictors.

Solutions. (1) The null space has dimension 1 and is spanned by \(\mathbf{1}=(1,\ldots,1)^\mathsf{T}\), since a connected graph requires equality of effects along every edge for the quadratic form to be zero. (2) The edge differences are \(0-2=-2\) and \(2-2=0\), so \(u^\mathsf{T}Qu=(-2)^2+0^2=4\). (3) Three constraints are needed, one for each component, because the null space has dimension 3. (4) Its degree is \(d_i=0\), so the conditional mean’s denominator \(d_i\) and the conditional precision \(\tau d_i\) are both zero; it has no neighbours to define the usual conditional. (5) If \(u_i^*=u_i-a\), set \(\beta_0^*=\beta_0+a\); then \(\beta_0^*+u_i^*=\beta_0+a+u_i-a=\beta_0+u_i\) for every area.