Design First, Covariance Second
Choosing correlated random effects begins with how observations were collected. A study may repeatedly measure the same individuals, sample different locations once, or observe multiple locations at multiple times. Those designs do not support the same indexing or make the same assumptions plausible. A covariance formula should reflect the distances and repetitions that the design actually contains, rather than being selected only because software offers it.
Let \(Y_i\) denote an outcome and let \(b\) collect latent random effects. A GLMM specifies the conditional distribution of the outcomes and a link for their conditional means. For the correlated effects considered here, a useful general representation is
where \(x_i\) is the fixed-effect design vector, \(\beta\) is the fixed-effect coefficient vector, and \(u(q_i)\) is a latent effect indexed by the relevant location, time, subject, or combination. A common assumption is that responses are conditionally independent given the latent effects and covariates; dependence between responses can then arise because their latent effects are correlated. The outcome distribution and link are chosen for the outcome type, while the covariance structure describes dependence among the latent effects.
Before fitting a model, write down what one index represents. For repeated measurements on subject \(j\), a temporal effect might be \(u_j(t)\). For a location \(s\), a spatial effect might be \(u(s)\). For observations indexed by both location and time, use \(u(s,t)\). These choices answer different scientific questions: a subject-specific process describes within-subject persistence, whereas a spatial process describes similarity across locations. A design with several measurements per subject at several sites may require more than one component, provided the data can distinguish them.
Matching the Structure to the Sampling Design
A temporal covariance indexed by elapsed time is appropriate when observations close in time are expected to share more latent variation than observations farther apart. For irregular observation times, the covariance should use the actual time gap, not merely the difference between row numbers. An Ornstein–Uhlenbeck process is one stationary option: its covariance decays exponentially with elapsed time. A random walk instead expresses accumulated changes and has different assumptions, including the need to handle its intrinsic, nonstationary structure appropriately.
A spatial covariance is appropriate when locations have meaningful coordinates or a scientifically defined distance. An exponential covariance, for example, assumes that covariance decreases as distance increases. An isotropic structure depends on the distance between locations, not their direction. If direction matters, or if barriers and networks make Euclidean distance misleading, an isotropic distance-based structure may not represent the process. For areal units, adjacency-based effects may be more natural than coordinate-based distance effects.
For a space-time design, first ask whether spatial dependence is expected to have the same form at every time and temporal dependence the same form at every location. A separable covariance makes both assumptions: the covariance is a product of spatial and temporal terms. If the spatial pattern changes over time, or temporal persistence differs across regions, a nonseparable or spatially varying temporal structure may be more suitable. More flexible structures demand more information from the design.
Worked Example: Irregular Repeated Observation Times
Suppose a hypothetical study records a binary outcome for each participant at days \(0\), \(2\), and \(5\). A participant-specific latent temporal effect \(u_j(t)\) enters the logit of the outcome probability. Assume the effects are mean zero and follow a stationary Ornstein–Uhlenbeck covariance with variance \(0.8\) and decay rate \(0.3\) per day:
This choice assumes that the same within-participant process operates across the study period, and that covariance depends on elapsed days. For the gaps \(2\), \(3\), and \(5\), the covariances are \(0.8e^{-0.6}=0.4390\), \(0.8e^{-0.9}=0.3253\), and \(0.8e^{-1.5}=0.1785\), respectively. In the order \(0,2,5\), the latent covariance matrix is therefore
The irregular gaps are handled directly. Using a lag of one and two visits instead would treat the two-day and three-day gaps as equivalent, contrary to the stated model. This structure is plausible if within-participant dependence fades with time and the process is approximately stationary; it would be questionable if there is a major intervention or trend that changes the process over follow-up.
Worked Example: A Spatial Effect on Three Locations
Imagine a hypothetical study with measurements at three sites on a straight transect, located at coordinates \(0\), \(2\), and \(5\) distance units. For a spatial latent effect \(u(s)\), suppose the covariance is exponential with variance \(1.44\) and decay rate \(0.4\) per distance unit:
The pairwise distances are \(2\), \(5\), and \(3\). Thus the covariances for those pairs are \(1.44e^{-0.8}=0.6470\), \(1.44e^{-2}=0.1949\), and \(1.44e^{-1.2}=0.4337\), respectively. The covariance matrix, ordered by coordinates \(0,2,5\), is
The proposed structure uses distance alone, so it assumes isotropy and no abrupt spatial barriers along the transect. The ordering of the off-diagonal entries follows the actual distances: sites \(0\) and \(2\) are closer than sites \(0\) and \(5\), and accordingly have the larger covariance. If the study instead sampled administrative areas connected by travel routes, this coordinate-based distance might not be the relevant measure of proximity.
Choosing a Space-Time Structure
When each of several sites is observed at several times, the indexing should preserve both dimensions. A separable model is a reasonable candidate when the study supports a common spatial pattern over time and a common temporal pattern over locations. It is not automatically justified by having a rectangular data table: it is a substantive restriction on dependence.
Worked Example: A Separable Space-Time Covariance
Consider a hypothetical monitoring design with repeated measurements at multiple sites. Let \(u(s,t)\) be a mean-zero Gaussian latent effect with variance \(1.2\). Suppose spatial correlation decays as \(e^{-0.4d}\), where \(d\) is distance, and temporal correlation decays as \(e^{-0.2h}\), where \(h\) is elapsed time. The separable covariance is
For two observations separated by \(2\) distance units and \(3\) time units, the latent covariance is \(1.2e^{-0.8}e^{-0.6}=1.2e^{-1.4}=0.2959\). This is below the marginal variance \(1.2\), as expected for distinct indices under these positive decay rates. The design makes this candidate interpretable if each site is visited over time and the time pattern is represented across sites. If some sites are observed only early and others only late, spatial and temporal patterns may be difficult to separate, even if the formula can be written down.
The design can also motivate a sum of components rather than one product. For instance, a participant-level random intercept may represent persistent differences among participants, while a separate temporal process represents within-participant departures over time. Such components should correspond to distinct sources of variation and be estimable from the observed replication. Adding several correlated effects merely because they are available can create weakly identified variance and decay parameters, a concern addressed in the earlier material on identifiability of spatial and temporal covariance parameters.
A Practical Selection and Checking Workflow
Record the subject, site, time, and any other repeated-unit identifiers for each outcome. Decide whether the dependence to represent is within subject, across sites, across time, or across both dimensions.
Use elapsed time for temporal dependence and a defensible distance or adjacency relation for spatial dependence. Do not substitute visit order for time when visits are irregular unless that is the intended scientific scale.
Specify whether the process is stationary, isotropic, separable, or shared across subjects or locations. State whether conditional independence of responses is assumed after accounting for covariates and latent effects.
Look for repeated time points per subject, spatial coverage, overlap of sites across times, and enough distinct distances or lags to inform covariance parameters. Consider a simpler or differently indexed structure if the design cannot support the proposed complexity.
Initial checks should include a diagram or table of the observation pattern, the distribution of pairwise distances and time gaps, missing visits, and whether sites or subjects are nested or crossed. A covariance structure is suspect when it requires extrapolation across combinations scarcely represented in the design, or when its parameters can trade off against one another. For example, a strong short-range spatial effect and a large independent site effect may be hard to distinguish when each site has few observations. The later diagnostic workflow for sparse GLMMs addresses model checking more broadly; here the key issue is whether the proposed dependence structure is defensible and estimable from the design.
| Design feature | Candidate structure | Key assumption to examine |
|---|---|---|
| Repeated observations at irregular times within subjects | Continuous-time temporal process | Dependence varies with elapsed time in a common way |
| Observations at georeferenced locations | Distance-based spatial effect | Chosen distance captures proximity; isotropy if assumed |
| Repeated spatial surveys over a common time grid | Separable space-time effect | Spatial and temporal dependence combine multiplicatively and are stable across the other dimension |
| Time evolution differs by region or spatial pattern changes over time | Nonseparable or spatially varying temporal structure | Additional flexibility is supported by adequate coverage and replication |
Limitations and Common Mistakes
A covariance model is not a substitute for understanding the sampling process. A convenient distance metric may ignore barriers, and a stationary temporal process may fail across a major policy change. Separable models can be efficient and interpretable, but may understate dependence when spatial correlation changes over time. Flexible nonseparable models can better represent such change, but often require richer designs and more parameters.
A frequent mistake is to interpret a latent covariance as a response correlation. In a binary GLMM, for example, a covariance between two Gaussian random effects enters the linear predictors, while the observed binary outcomes have a different marginal dependence after integrating over those effects. Another mistake is to select a structure only by whether it converges. Convergence does not show that indexing, scientific assumptions, or parameter identifiability are adequate.
- Wrong reasoning: “The records are listed in time order, so an equally spaced lag structure is appropriate.” Correction: Use actual elapsed times when intervals differ, unless the scientific process is explicitly indexed by visit number.
- Wrong reasoning: “Every site was measured, so a separable space-time model is justified.” Correction: Separable covariance additionally assumes a common spatial pattern across times and common temporal dependence across locations.
- Wrong reasoning: “A larger fitted latent covariance means the observed binary responses have that same correlation.” Correction: Latent and response-scale dependence are not generally equal; the response distribution and link affect the marginal relationship.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A subject is measured at days 1, 4, and 10. Why should a continuous-time covariance use the observed day gaps rather than only visit-number gaps?
- For an exponential spatial covariance with marginal variance \(2\), decay rate \(0.5\) per distance unit, and two sites \(3\) units apart, calculate the covariance.
- What additional assumptions does a separable space-time covariance make beyond merely having observations at multiple places and times?
- A study samples one set of sites in the first year and a disjoint set in the second year. What design concern arises when trying to distinguish spatial and temporal dependence?
Solutions. (1) The gaps are \(3\), \(6\), and \(9\) days, while visit-number gaps do not preserve those elapsed intervals. A time-based process uses actual elapsed time so its covariance corresponds to the stated temporal scale. (2) Substitution gives \(2e^{-0.5(3)}=2e^{-1.5}=2(0.223130\ldots)=0.446260\ldots\approx0.4463\), in squared latent-effect units. (3) Separability assumes that spatial dependence has the same form across times and temporal dependence has the same form across locations, with joint covariance equal to the product of the two correlation structures times the variance. (4) Site and year are entangled: changes between years could reflect temporal change or differences between the sampled sites. With no locations shared across years, the design offers limited direct information for separating spatial from temporal dependence.