Temporal Covariance at Irregular Observation Times

Why Irregular Timing Changes the Covariance Matrix

Repeated observations are often not collected at equal intervals. A participant might be measured on days 0, 2, and 5, while another is measured on days 0, 1, and 5. A discrete-time correlation indexed only by visit number cannot represent these elapsed-time differences without additional assumptions. A continuous-time model uses the actual separation between observations: the farther apart two times are, the weaker their modeled temporal correlation.

This tutorial constructs that covariance matrix for a stationary continuous-time Ornstein–Uhlenbeck process. The earlier material on Continuous-Time Ornstein–Uhlenbeck Effects introduces the process; here the emphasis is on translating irregular observation times into pairwise covariances and checking the resulting matrix. In a GLMM, this matrix may be the covariance of a Gaussian latent temporal effect. It is not, by itself, the covariance matrix of the observed responses.

Definition: Let $t_1,\ldots,t_m$ be observation times, measured in a common time unit, and let $u(t)$ be a mean-zero stationary Gaussian temporal process with marginal variance $\tau^2>0$ and decay rate $\phi>0$. Its continuous-time exponential covariance is $$ \operatorname{Cov}\{u(t_i),u(t_j)\} =K_{ij} =\tau^2\exp\{-\phi|t_i-t_j|\}. $$ The correlation at a time gap $d\geq 0$ is $\exp(-\phi d)$. The parameter $\phi$ has units reciprocal to the time unit, so that $\phi d$ is dimensionless.

The absolute time difference matters, not the calendar origin or the order in which records happen to appear in a file. The diagonal entries are $K_{ii}=\tau^2$, because the gap from an observation to itself is zero. Each off-diagonal entry is the common marginal variance multiplied by the correlation for that pair's elapsed time.

Constructing the Matrix

A reliable construction begins by writing down the observation times in their chosen order and calculating every pairwise absolute difference. Apply the exponential decay to each difference, multiply by $\tau^2$, and place each result symmetrically in the matrix. For example, if $t_1<t_2<t_3$, then the covariance between the first and third observations depends on $t_3-t_1$, not merely on their being two positions apart.

Sorting the times is useful for checking the pattern, though it is not mathematically required. If observations are kept in a different order, the rows and corresponding columns of $K$ must be permuted together. The covariance represented by the matrix is unchanged; only its labeling changes.

Worked Example: A Four-Time Covariance Matrix

Suppose hypothetical latent effects are observed at days $0,1,3,4$, with marginal variance $\tau^2=2.25$ and decay rate $\phi=0.4$ per day. The model assumptions are that the latent effects follow the stationary Gaussian exponential process above and that the stated times use days as their unit.

The pairwise gaps in the upper triangle are $1,3,4$ days from day 0; $2,3$ days from day 1; and $1$ day from day 3. The corresponding covariance values include

\begin{aligned} K(1)&=2.25e^{-0.4}=2.25(0.670320\ldots)\approx1.508,\\ K(2)&=2.25e^{-0.8}=2.25(0.449329\ldots)\approx1.011,\\ K(3)&=2.25e^{-1.2}=2.25(0.301194\ldots)\approx0.6777,\\ K(4)&=2.25e^{-1.6}=2.25(0.201897\ldots)\approx0.4543. \end{aligned}

Using the gaps in the specified order gives

K=2.25 \begin{pmatrix} 1 & e^{-0.4} & e^{-1.2} & e^{-1.6}\\ e^{-0.4} & 1 & e^{-0.8} & e^{-1.2}\\ e^{-1.2} & e^{-0.8} & 1 & e^{-0.4}\\ e^{-1.6} & e^{-1.2} & e^{-0.4} & 1 \end{pmatrix} \approx \begin{pmatrix} 2.250 & 1.508 & 0.6777 & 0.4543\\ 1.508 & 2.250 & 1.011 & 0.6777\\ 0.6777 & 1.011 & 2.250 & 1.508\\ 0.4543 & 0.6777 & 1.508 & 2.250 \end{pmatrix}.

The diagonal is the marginal variance, and the symmetry reflects $|t_i-t_j|=|t_j-t_i|$. The two three-day gaps have the same covariance, even though they occur at different calendar positions. Rounding is applied to the final covariance values; calculations use the unrounded exponential values.

Equal and Unequal Time Gaps

For fixed $\phi$, the correlation depends on elapsed time alone. Thus two pairs separated by the same duration have the same correlation, regardless of when either observation occurred. For unequal positive gaps, the longer gap has the smaller correlation because the exponential function decreases as its nonnegative argument increases.

d_2>d_1\geq0,\ \phi>0 \quad\Longrightarrow\quad e^{-\phi d_2}<e^{-\phi d_1}.

The covariance has the same ordering when $\tau^2$ is fixed and positive. Correlation is often the clearest quantity for comparing gaps because it is dimensionless; covariance retains the squared units of the latent effect.

Worked Example: Comparing One-Day and Three-Day Gaps

Consider a hypothetical process with $\tau^2=1.6$ and $\phi=0.3$ per day. A pair of observations one day apart has correlation $e^{-0.3}$, while a pair three days apart has correlation $e^{-0.9}$. The time unit and positive decay rate meet the model's scaling and monotone-decay requirements.

\begin{aligned} \operatorname{Corr}\{u(t),u(t+1)\} &=e^{-0.3}=0.740818\ldots\approx0.7408,\\ \operatorname{Corr}\{u(t),u(t+3)\} &=e^{-0.9}=0.406570\ldots\approx0.4066. \end{aligned}

Using the unrounded correlations to calculate covariances,

\begin{aligned} \operatorname{Cov}\{u(t),u(t+1)\} &=1.6e^{-0.3}\approx1.185,\\ \operatorname{Cov}\{u(t),u(t+3)\} &=1.6e^{-0.9}\approx0.6505. \end{aligned}

The three-day pair is less correlated than the one-day pair. Any pair with a one-day gap has correlation about 0.7408 under this model, whether the pair occurs early or late in the observation period. These are statements about the latent process, not causal effects or direct claims about response-scale correlations in a GLMM.

Why the Constructed Matrix Is Positive Definite

A covariance matrix must be positive semidefinite. For distinct, ordered times and $\phi>0$, this exponential covariance is in fact positive definite. One justification comes from representing the process at successive observation times as a sequence of transitions. This argument also shows explicitly how an irregular gap enters the model.

Order the times.
Write $t_1<\cdots<t_m$ and let $\Delta_i=t_i-t_{i-1}>0$ for $i=2,\ldots,m$.

Set the transition for each gap.
Define $a_i=e^{-\phi\Delta_i}$. Then $0<a_i<1$, and use $u(t_i)=a_i u(t_{i-1})+\epsilon_i$, where the innovations are mutually independent and independent of $u(t_1)$.

Choose the stationary variances.
Let $u(t_1)$ have variance $\tau^2$ and let $\epsilon_i$ have variance $\tau^2(1-a_i^2)>0$. Each transition then preserves variance: $a_i^2\tau^2+\tau^2(1-a_i^2)=\tau^2$.

Recover the covariance and definiteness.
For $j>i$, repeated substitution gives $\operatorname{Cov}\{u(t_i),u(t_j)\}=\tau^2\prod_{k=i+1}^{j}a_k=\tau^2e^{-\phi(t_j-t_i)}$. The independent initial value and positive-variance innovations give a nonsingular triangular transformation, so the resulting covariance matrix is positive definite.

This derivation verifies both the exponential formula and its validity for unequal intervals. The transition coefficient is not one fixed value at all intervals: it is $e^{-\phi\Delta_i}$, which changes with each actual gap. If times coincide, a transition has zero gap and coefficient one; the corresponding latent values are identical under this single-process model, so the covariance matrix may be singular.

Worked Example: Three Unequally Spaced Observations

Suppose hypothetical latent effects are recorded at days $0,2,5$, with $\tau^2=1.44$ and $\phi=0.2$ per day. The successive gaps are 2 and 3 days, and the total gap is 5 days. All times use the same unit, and the positive decay rate is consistent with declining correlation as gaps increase.

The three distinct covariances are

\begin{aligned} K_{12}&=1.44e^{-0.2(2)} =1.44e^{-0.4}\approx0.9653,\\ K_{23}&=1.44e^{-0.2(3)} =1.44e^{-0.6}\approx0.7903,\\ K_{13}&=1.44e^{-0.2(5)} =1.44e^{-1}\approx0.5297. \end{aligned}

Therefore, in the order $0,2,5$,

K\approx \begin{pmatrix} 1.440 & 0.9653 & 0.5297\\ 0.9653 & 1.440 & 0.7903\\ 0.5297 & 0.7903 & 1.440 \end{pmatrix}.

The corresponding correlations are $e^{-0.4}\approx0.6703$, $e^{-0.6}\approx0.5488$, and $e^{-1}\approx0.3679$. The longest gap has the weakest correlation. The transition argument also checks positive definiteness: the two successive transition coefficients are $e^{-0.4}$ and $e^{-0.6}$, both strictly between zero and one, so their innovation variances are positive.

Assumptions, Diagnostics, and Alternatives

The covariance construction is appropriate when a single stationary process is intended: it has a constant marginal variance, and dependence is a function of the absolute time gap through one positive decay parameter. The times must be meaningful on a common continuous scale. In a GLMM, the covariance specification concerns the latent Gaussian vector; the response distribution and link function are specified separately.

Before using a constructed matrix, check the following:

Symmetry and diagonal: verify $K_{ij}=K_{ji}$ and $K_{ii}=\tau^2$.
Distance pattern: larger gaps must produce smaller correlations for $\phi>0$, while equal gaps must produce equal correlations.
Units: if time is converted, convert $\phi$ as well. For example, a rate per day cannot be applied directly to gaps measured in hours.
Numerical definiteness: fitted or assembled matrices should be symmetric and positive semidefinite, allowing for small numerical rounding error. A materially negative eigenvalue suggests a coding, ordering, or parameterization problem.
Repeated times: repeated measurements at an identical time share the same latent process value in this model. If separate observation-level variation is needed, it must be represented as an additional model component, not silently inserted into this temporal covariance.

A common mistake is to use $e^{-\phi|i-j|}$, where $i$ and $j$ are visit numbers, even when the visits are irregularly spaced. That calculation treats one visit step as the same duration everywhere. The correction is to use $e^{-\phi|t_i-t_j|}$ with actual times and a decay rate expressed in reciprocal time units.

At equally spaced times with spacing $\Delta$, this model has the same covariance pattern as a stationary discrete-time AR(1) model with adjacent-time correlation $\rho=e^{-\phi\Delta}$. For irregular spacing, the continuous-time formulation naturally assigns each interval its own transition coefficient $e^{-\phi\Delta_i}$; a single fixed AR(1) coefficient per visit does not encode unequal durations. An independence structure is another alternative, but it sets all off-diagonal covariances to zero and therefore does not model temporal dependence.

The exponential form is parsimonious, but it imposes a particular stationary decay pattern. A poor fit may arise if dependence does not decline monotonically with elapsed time, if marginal variability changes over time, or if one decay rate is inadequate. The matrix construction alone cannot establish that the assumptions fit a particular data set; model assessment and the scientific meaning of the latent effect remain necessary.

Key takeaway: For irregular observation times, compute every covariance from the actual time difference: $K_{ij}=\tau^2e^{-\phi|t_i-t_j|}$. Equal gaps imply equal correlations; longer gaps imply smaller correlations when $\phi>0$.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

For $\tau^2=4$, $\phi=0.5$ per day, and a two-day gap, calculate the correlation and covariance.
Under one stationary exponential process, should two pairs with the same three-day gap have equal correlations if they occur at different dates? Explain.
Why is using visit-index separation in place of elapsed time unsuitable when observation intervals differ?
For distinct ordered times and $\phi>0$, what feature of the transition representation establishes positive definiteness?

Solutions. (1) The correlation is $e^{-0.5(2)}=e^{-1}\approx0.3679$. The covariance is $4e^{-1}\approx1.472$, in squared units of the latent effect. (2) Yes. Both correlations equal $e^{-3\phi}$, since the model depends on the gap and not the calendar position. (3) Visit-index separation counts steps rather than measuring their duration, so it assigns the same dependence to unequal gaps. Use $|t_i-t_j|$ instead. (4) Each successive gap has a transition coefficient strictly between zero and one, giving a strictly positive innovation variance; together with the initial variance this yields a nonsingular triangular construction and a positive-definite covariance matrix.