Discrete-Time AR(1) Random Effects

Why Use a Discrete-Time AR(1) Latent Effect?

Repeated observations can remain related after measured covariates and other random effects are accounted for. A latent effect that evolves through time can represent this remaining temporal dependence in a GLMM. The discrete-time AR(1) structure is a compact choice when observations occur at equally spaced times and dependence is expected to weaken geometrically as the lag grows.

The process describes dependence among latent effects, not directly among the observed responses. For a binary or count GLMM, the response distribution is specified conditional on the latent effect; integrating over the latent process is what connects that structure to marginal response behavior. The earlier material on conditional GLMMs with spatial and temporal latent effects provides the general conditional-model setting.

Stationary Discrete-Time AR(1) Specification

Let $u_{it}$ denote a latent effect for series $i$ at equally spaced time $t=1,\ldots,T$. A stationary Gaussian AR(1) process has mean zero, marginal variance $\tau^2$, and persistence parameter $\rho$. Its transition representation is

u_{it}=\rho u_{i,t-1}+\varepsilon_{it}, \qquad |\rho|<1,

where the innovations $\varepsilon_{it}$ are mutually independent with distribution $N(0,\sigma_\varepsilon^2)$, independent of the preceding latent effects, and

\sigma_\varepsilon^2=\tau^2(1-\rho^2).

To initialize a finite series in its stationary distribution, take $u_{i1}\sim N(0,\tau^2)$, independently of its future innovations. For distinct series, assume independent processes unless the model explicitly specifies dependence between them. The condition $|\rho|<1$ gives a finite stationary variance for a nondegenerate process.

Definition: A stationary discrete-time AR(1) latent effect is a Gaussian process with constant mean zero and marginal variance $\tau^2$, whose dependence between two times $t$ and $s$ is $$ \operatorname{Cov}(u_{it},u_{is})=\tau^2\rho^{|t-s|}. $$ Its lag-$h$ correlation is $\rho^h$ for an integer $h\geq 0$.

In a GLMM, one possible conditional linear predictor is $\eta_{it}=x_{it}^{\mathsf T}\beta+u_{it}$, where $x_{it}$ is the covariate vector and $\beta$ is the fixed-effect vector. The response distribution is then specified conditional on the latent effects, with the appropriate link relating its conditional mean to $\eta_{it}$. The AR(1) covariance is a statement about $u$, not a claim that the observed responses have covariance $\tau^2\rho^{|t-s|}$.

What Persistence Means

The parameter $\rho$ is the correlation between latent effects one time step apart. When $\rho$ is positive, adjacent latent effects tend to move in the same direction relative to their mean, and the correlation decreases geometrically across longer lags. When $\rho$ is negative, adjacent effects tend to have opposite signs; the correlation alternates in sign across lags. When $\rho=0$, effects at distinct times are uncorrelated and, under the Gaussian process assumption, independent.

For any lag $h$, the covariance is the marginal variance multiplied by the lag-$h$ correlation. Thus the process separates two features: $\tau^2$ sets the overall latent-effect scale, while $\rho$ controls how that dependence changes with temporal separation. In particular, lag zero has covariance $\tau^2$, as it must, because $\operatorname{Cov}(u_{it},u_{it})=\operatorname{Var}(u_{it})$.

Worked Example: Positive Persistence at Several Lags

Suppose a hypothetical GLMM uses a stationary latent process with $\tau=0.5$ and $\rho=0.6$, and the time steps are equally spaced. Calculate its variance, innovation variance, and covariances at lags 1 and 3.

The marginal variance is $\tau^2=0.5^2=0.25$. The innovation variance required for stationarity is

\sigma_\varepsilon^2=\tau^2(1-\rho^2) =0.25(1-0.6^2) =0.25(0.64) =0.16.

At lag 1, the latent covariance is

\operatorname{Cov}(u_{it},u_{i,t-1}) =\tau^2\rho =0.25(0.6) =0.15.

At lag 3, it is

\operatorname{Cov}(u_{it},u_{i,t-3}) =\tau^2\rho^3 =0.25(0.6^3) =0.25(0.216) =0.054.

Both covariances are positive, and the lag-3 covariance is smaller than the lag-1 covariance. The corresponding correlations are $0.6$ and $0.6^3=0.216$. These quantify persistence on the latent scale, not a change of 0.15 or 0.054 in the response mean.

Why the Covariance Decays Geometrically

The covariance formula follows the same one-step dependence rule at every time. For lag $h\geq1$, substitute the AR(1) transition for the later latent effect. The innovation at time $t$ is independent of the earlier latent effect, so its covariance with that effect is zero:

\begin{aligned} \operatorname{Cov}(u_{it},u_{i,t-h}) &=\operatorname{Cov}(\rho u_{i,t-1}+\varepsilon_{it},u_{i,t-h})\\ &=\rho\operatorname{Cov}(u_{i,t-1},u_{i,t-h}) +\operatorname{Cov}(\varepsilon_{it},u_{i,t-h})\\ &=\rho\operatorname{Cov}(u_{i,t-1},u_{i,t-h}). \end{aligned}

Applying this one-step relation repeatedly multiplies the lag-1 covariance by $\rho$ once for each additional time step. Since the lag-1 covariance is $\tau^2\rho$, the lag-$h$ covariance is $\tau^2\rho^h$, equivalently $\tau^2\rho^{|h|}$ when the order of the two times is unrestricted. This gives the geometric decline, or sign alternation, that characterizes the AR(1) structure.

Worked Example: Negative Persistence

Consider a hypothetical stationary latent process with $\tau^2=0.36$ and $\rho=-0.5$. The equally spaced time index and $|\rho|<1$ make the stated stationary AR(1) specification valid. Find the innovation variance and covariances at lags 1, 2, and 3.

The innovation variance is

\sigma_\varepsilon^2 =0.36\{1-(-0.5)^2\} =0.36(0.75) =0.27.

At lag 1, the covariance is $0.36(-0.5)=-0.18$. At lag 2, it is $0.36(-0.5)^2=0.36(0.25)=0.09$. At lag 3, it is $0.36(-0.5)^3=0.36(-0.125)=-0.045$.

The alternating signs are important: a negative persistence parameter does not mean every pair of latent effects has negative covariance. Even lags have positive covariance, while odd lags have negative covariance. The correlation at lag 2 is $0.25$, and the correlation at lag 3 is $-0.125$.

Placing the Process in a GLMM

For a concrete conditional model, suppose a hypothetical binary response $Y_{it}$ has a Bernoulli distribution with conditional success probability $p_{it}$, and use a logit link:

Y_{it}\mid u_{it}\sim\operatorname{Bernoulli}(p_{it}),\qquad \operatorname{logit}(p_{it})=x_{it}^{\mathsf T}\beta+u_{it}.

This specification is completed by the stationary AR(1) assumptions for $u_{it}$. Conditional on the latent effects and covariates, responses are assumed independent under this model. Dependence among responses can nevertheless arise after the shared latent effects are integrated out. The latent covariance gives the structure being integrated over; it is not itself the response-scale covariance.

Worked Example: Covariance in a Logit-Scale Latent Process

Suppose a hypothetical repeated-measures binary GLMM has a latent logit-scale effect with marginal standard deviation $\tau=0.3$ and persistence $\rho=0.8$. Compute its marginal variance, innovation variance, and lag-2 latent covariance.

The marginal variance is $\tau^2=0.3^2=0.09$. The innovation variance is

\sigma_\varepsilon^2 =0.09(1-0.8^2) =0.09(0.36) =0.0324.

The lag-2 covariance is

\operatorname{Cov}(u_{it},u_{i,t-2}) =0.09(0.8^2) =0.09(0.64) =0.0576.

This means the latent logit contributions at times two steps apart have covariance $0.0576$ logit-squared units and correlation $0.64$. It does not mean that the two binary responses have covariance $0.0576$, nor that their success probabilities differ by $0.0576$. Those are response-scale quantities and depend on the link, covariates, and integration over the latent effects.

Assumptions, Diagnostics, and Alternatives

The AR(1) specification is most directly interpretable when time is discrete and equally spaced, the latent process is adequately represented as Gaussian, and its mean and covariance structure are stable over the modeled period. The series should also be initialized consistently with stationarity, or the treatment of the initial distribution should be stated. The independence assumptions for innovations and for distinct series matter: a common time effect or correlated series would require additional structure.

Diagnostics should assess whether the assumed temporal pattern is plausible, rather than treating a fitted $\rho$ as proof that it is. Useful checks include examining residual or simulation-based dependence over time, comparing observed patterns with model-based simulations, and checking whether unexplained trends, abrupt changes, or periodic patterns remain. Persistent residual dependence at lags not explained by the geometric pattern suggests that the AR(1) structure may be inadequate. Estimates of $\rho$ near $1$ or $-1$ can also signal weak information about stationarity or a process close to the boundary of the allowed parameter space.

The main alternatives answer different dependence questions. Independent time-specific latent effects correspond to $\rho=0$: they allow extra variation at each time but no temporal persistence. A random intercept shared across time induces the same latent contribution at every time, unlike an AR(1) process whose correlation changes with lag. The AR(1) structure is useful between these extremes when dependence weakens with separation, but it is not a universal temporal model. In particular, a single stationary geometric pattern may be unsuitable when the process has a changing mean, irregular observation spacing, or a more complex temporal pattern.

A common mistake is to substitute the response variance for $\tau^2$ in the covariance formula. That is incorrect: $\tau^2\rho^{|h|}$ is the covariance of the latent effects, while response covariance depends on the conditional response distribution and link. Another mistake is to treat $\rho$ as a covariance. It is a dimensionless lag-1 correlation; the covariance at lag $h$ is $\tau^2\rho^{|h|}$, in squared units of the latent effect.

Key takeaway: Specify the stationary AR(1) process with $|\rho|<1$, a marginal latent variance $\tau^2$, and innovation variance $\tau^2(1-\rho^2)$. At integer lag $h$, its latent covariance is $\tau^2\rho^{|h|}$, and its latent correlation is $\rho^{|h|}$.

Check Your Understanding

Answer each question, then compare your work with the solutions.

A stationary AR(1) latent process has $\tau^2=0.49$ and $\rho=0.4$. Find its innovation variance and lag-2 covariance.
For $\tau^2=0.16$ and $\rho=-0.7$, find the lag-1 and lag-2 correlations. Explain their signs.
What does $\rho=0$ imply about distinct latent effects in a Gaussian AR(1) process?
Why should a latent lag-1 covariance not be reported as the covariance of two binary responses in a logit GLMM?

Solutions. (1) The innovation variance is $0.49(1-0.4^2)=0.49(0.84)=0.4116$. The lag-2 covariance is $0.49(0.4^2)=0.0784$. (2) The lag-1 correlation is $-0.7$; the lag-2 correlation is $(-0.7)^2=0.49$. Odd lags have negative correlation and even lags positive correlation. (3) Distinct latent effects are uncorrelated, and because the process is Gaussian, they are independent. (4) The latent covariance is on the latent scale. The binary-response covariance depends on the conditional probabilities and the link after accounting for the latent distribution, so it is not equal in general to the latent covariance.