Kronecker Structure in Separable Space-Time Models

From separable dependence to a covariance matrix

In a space–time GLMM, a latent random effect may be indexed by both a spatial site and a time. A separable covariance says that the dependence between two such effects factors into a spatial part and a temporal part. The useful consequence is not just a compact formula for one covariance: on a complete grid, the entire covariance matrix has a Kronecker-product form. That structure makes the indexing explicit and can reduce storage and computation.

The earlier material on Separable Space-Time Covariance Models gives the motivating covariance assumption. Here the focus is its matrix representation and how to calculate particular entries without losing track of their indices.

Definition: Let $n_S$ spatial sites be observed at each of $n_T$ times, with one latent random effect $w_{ij}$ for site $i$ and time $j$. The grid is complete when every site–time combination occurs. Let $R_S$ be an $n_S\times n_S$ spatial correlation matrix and $R_T$ an $n_T\times n_T$ temporal correlation matrix. Under a separable covariance model with marginal variance $\tau^2$, $$ \operatorname{Cov}(w_{ij},w_{k\ell})=\tau^2(R_S)_{ik}(R_T)_{j\ell}. $$ Both correlation matrices are symmetric positive semidefinite, and their diagonal entries are one. The parameter $\tau^2$ is the variance of each $w_{ij}$.

For a GLMM, this statement concerns the latent random effects, not the conditional distribution of the observed responses. For example, one might specify a conditional response model given the fixed effects and the field $w$, with the effects jointly Gaussian as above. The covariance formula alone does not assert that the responses are Gaussian.

Deriving the Kronecker form

Use time-major stacking: list all spatial effects at time 1, then all effects at time 2, and continue. In vector notation this is $\mathbf{w}=(w_{11},\ldots,w_{n_S1},w_{12},\ldots,w_{n_S n_T})^\mathsf{T}$. The Kronecker product $R_T\otimes R_S$ is a block matrix whose block in temporal position $(j,\ell)$ is $(R_T)_{j\ell}R_S$. Multiplying by $\tau^2$ therefore gives exactly the covariance between the corresponding groups of effects.

\operatorname{Cov}(\mathbf{w})=\tau^2(R_T\otimes R_S).

This formula can be verified entry by entry. In time-major order, the entry for vector positions corresponding to $(i,j)$ and $(k,\ell)$ is the spatial entry $(R_S)_{ik}$ multiplied by the temporal entry $(R_T)_{j\ell}$, and by $\tau^2$. That is the defining covariance assumption. The dimension is $(n_Tn_S)\times(n_Tn_S)$: there are $n_T$ block rows and columns, and each block is $n_S\times n_S$.

Result: For a complete grid with time-major stacking, the block in row $j$, column $\ell$ of $\operatorname{Cov}(\mathbf{w})$ is $\tau^2(R_T)_{j\ell}R_S$. Thus every matrix entry can be checked by identifying its site pair and its time pair separately.

The ordering is part of the formula. If instead the vector is stacked site-major, with all times for site 1 followed by all times for site 2, the covariance matrix is $\tau^2(R_S\otimes R_T)$. The underlying covariances have not changed; only the positions assigned to them have changed. Applying the time-major formula to a differently ordered vector silently permutes the covariance entries.

Worked examples

Worked Example: Constructing the Matrix for a Small Grid

Suppose there are two sites and three times. Let the spatial correlation between the sites be $0.4$, and let the temporal correlations at lags 1 and 2 be $0.5$ and $0.25$, respectively. Assume $\tau^2=4$, and use time-major stacking.

The correlation matrices are $$ R_S=\begin{pmatrix}1&0.4\\0.4&1\end{pmatrix}, \qquad R_T=\begin{pmatrix}1&0.5&0.25\\0.5&1&0.5\\0.25&0.5&1\end{pmatrix}. $$ For example, the block connecting times 1 and 2 is $4(0.5)R_S=2R_S$. The diagonal block for time 1 is $4(1)R_S$. Therefore, $$ \operatorname{Cov}(\mathbf{w})= \begin{pmatrix} 4&1.6&2&0.8&1&0.4\\ 1.6&4&0.8&2&0.4&1\\ 2&0.8&4&1.6&2&0.8\\ 0.8&2&1.6&4&0.8&2\\ 1&0.4&2&0.8&4&1.6\\ 0.4&1&0.8&2&1.6&4 \end{pmatrix}. $$ The diagonal entries are all 4, as required by the marginal variance. The entry in row 1, column 4 is $0.8$: these vector positions are $w_{11}$ and $w_{22}$, whose covariance is $4(0.4)(0.5)=0.8$. This entry-by-entry check helps detect a block-ordering error.

Worked Example: Calculating a Covariance From Distance and Lag

Consider a hypothetical separable model with spatial correlation $\exp(-d/3)$, where $d$ is distance in kilometres, and temporal correlation $0.6^{|j-\ell|}$, where the indices denote equally spaced occasions. Let $\tau^2=2.25$. Calculate the covariance between effects 2 km apart and two occasions apart.

The spatial correlation is $\exp(-2/3)\approx0.5134$. The temporal correlation is $0.6^2=0.36$. Substituting these values into the separable covariance formula gives $$ \operatorname{Cov}(w_{ij},w_{k\ell}) =2.25(0.5134)(0.36) \approx0.4159. $$ The covariance is approximately $0.416$ in squared units of the latent effect. The spatial and temporal correlations are dimensionless; the variance parameter supplies the covariance scale. If either the distance or the time separation were zero, the corresponding correlation factor would be one.

Worked Example: Reading an Entry With Unequal Factor Dimensions

Suppose there are three sites and two times, with time-major stacking and $\tau^2=1.5$. Assume the spatial correlation between sites 1 and 3 is $0.1$, while the temporal correlation between times 2 and 1 is $0.7$. Find the covariance between $w_{1,2}$ and $w_{3,1}$, and identify where it appears in the matrix.

The time-major vector is $(w_{11},w_{21},w_{31},w_{12},w_{22},w_{32})^\mathsf{T}$. Thus $w_{1,2}$ is position 4 and $w_{3,1}$ is position 3. Their covariance is $$ 1.5(0.1)(0.7)=0.105. $$ So the entry in row 4, column 3 is $0.105$; symmetry requires the row 3, column 4 entry to be the same. The block connecting time 2 to time 1 is $1.5(0.7)R_S$, and its row 1, column 3 entry is also $1.5(0.7)(0.1)=0.105$. This second route checks the indexed calculation against the block structure.

Why the construction is valid, and when it is useful

The Kronecker matrix must itself be a valid covariance matrix. If $R_S$ and $R_T$ are positive semidefinite, their eigenvalues are nonnegative. The eigenvalues of $R_T\otimes R_S$ are all products of an eigenvalue of $R_T$ and an eigenvalue of $R_S$, so they are nonnegative as well. Multiplying by $\tau^2\geq0$ preserves this property. This verifies positive semidefiniteness, the essential requirement for a covariance matrix. If both factors are positive definite and $\tau^2>0$, the resulting covariance is positive definite.

The same structure gives computational identities when the factors are nonsingular. If their dimensions are $n_T$ and $n_S$, respectively, then $$ (R_T\otimes R_S)^{-1}=R_T^{-1}\otimes R_S^{-1}, \qquad \det(R_T\otimes R_S)=\det(R_T)^{n_S}\det(R_S)^{n_T}. $$ These identities can replace operations on a matrix of dimension $n_Tn_S$ with operations on the smaller factors. They do not remove the need to check that the assumed covariance is appropriate or that the factors are invertible for the particular calculation.

A complete grid is especially convenient because each time block contains the same set of sites, and the separable factors can be represented directly. If some site–time cells are absent, the observed effects are not a complete rectangular vector. One can still define the full-grid covariance and extract the rows and columns for observed cells, but the observed covariance is then a selected submatrix; it is not generally a smaller Kronecker product. Repeated measurements at a cell or different sets of sites at different times also require careful indexing rather than an unexamined complete-grid formula.

Diagnostics, alternatives, and limitations

Before using a Kronecker covariance in a model, check the assumptions and the implementation separately. The mathematical construction requires a common spatial factor and temporal factor whose product describes every site–time pair. In practice, also check that each factor is symmetric, has unit diagonal when it represents correlations, and is positive semidefinite. A failed positive-semidefiniteness check can indicate an invalid factor, a parameter value outside its allowed range, or a coding error.

Check dimensions. With $n_S$ sites and $n_T$ times, the full matrix must have $n_Sn_T$ rows and columns.
Check the diagonal. Under the stated model, every diagonal entry equals $\tau^2$.
Check one off-diagonal entry by hand. Compare its matrix position with $\tau^2(R_S)_{ik}(R_T)_{j\ell}$.
Check the vector order. Confirm that the data and the covariance matrix use the same site/time stacking convention.
Check the design. If observations are missing, use the covariance submatrix for the observed index pairs rather than treating absent cells as observed values.

A common mistake is to put $R_S\otimes R_T$ beside a time-major vector because the factors were written in that order. That reasoning ignores the block dimensions: time-major stacking requires each block to contain the spatial covariance, so the matrix is $R_T\otimes R_S$. The correction is to write the vector order first, identify the meaning of one block, and then verify a specific entry.

The principal alternative is a covariance model that cannot be expressed as one spatial factor times one temporal factor. Such a model may be needed if spatial dependence changes with time or temporal dependence varies across space. A single Kronecker product cannot represent those patterns without additional structure. Conversely, the separable model is parsimonious and computationally attractive, but its factorization is a substantive restriction, not a consequence of having data on a grid. The Kronecker form also does not by itself resolve missingness, identify covariance parameters, or establish that the assumed dependence is adequate for a particular application.

Check Your Understanding

Use time-major stacking, and assume the stated separable covariance model unless a question says otherwise.

If there are 4 sites and 5 times, what is the dimension of the full covariance matrix?
Suppose $\tau^2=3$, $(R_S)_{ik}=0.2$, and $(R_T)_{j\ell}=0.5$. What is $\operatorname{Cov}(w_{ij},w_{k\ell})$?
For 2 sites and 2 times, explain why the block connecting times 1 and 2 is $\tau^2(R_T)_{12}R_S$.
What changes if the effects are stacked site-major rather than time-major?
Why is a covariance matrix for an incomplete grid not generally a Kronecker product of smaller spatial and temporal matrices?

Solutions. (1) There are $4(5)=20$ effects, so the matrix is $20\times20$. (2) The covariance is $3(0.2)(0.5)=0.30$, in squared units of the latent effect. (3) Every entry in that block pairs an effect at time 1 with one at time 2; its temporal multiplier is $(R_T)_{12}$, while the entries within the block are the corresponding spatial correlations. Multiplication by the variance scale gives the stated block. (4) The same covariances are ordered differently, and the matrix becomes $\tau^2(R_S\otimes R_T)$. (5) Removing cells selects particular rows and columns from the full-grid covariance, generally disrupting the repeated block layout required for a direct Kronecker product.