Why Within-Subject Dependence Needs Its Own Model
Repeated binary, count, and other non-Gaussian outcomes from one person are rarely independent in the population. A subject who tends to have high outcomes may remain relatively high across visits, and outcomes close together in time may share additional short-term influences. A longitudinal generalized linear mixed model (GLMM) can represent these two sources of dependence separately: persistent subject-level heterogeneity and a subject-specific temporal process.
The distinction matters because a random intercept alone produces the same latent covariance between every pair of visits for a subject. Temporal dependence instead allows that covariance to change with the separation between visits. The process is also subject-specific: absent a shared population-level shock or other cross-subject structure, one subject’s latent temporal deviations are independent of another’s.
A common specification takes \(b_i\sim N(0,D)\) and \(u_i(\cdot)\) to be a Gaussian process with covariance function \(K\). The standard assumptions are that different subjects’ pairs \((b_i,u_i)\) are independent and identically distributed, that \(b_i\) is independent of \(u_i(\cdot)\), and that the latent effects have mean zero. Conditional on covariates and latent effects, the outcome at a visit has the stated response distribution and no remaining serial dependence. In practice, the observation schedule and missingness also need to be non-informative given the modeled information, or their dependence must be modeled.
The fixed-effects part can describe population-average covariate patterns on the link scale, including a long-term time trend. Random slopes in \(b_i\) allow subjects to vary in their long-term trajectories. The process \(u_i(t)\), by contrast, describes residual, temporally correlated departures from those trajectories. These components should not be treated as interchangeable.
Covariance Within and Across Subjects
For two visits \(j\) and \(k\) from the same subject, write the latent contribution beyond the fixed effects as \(a_{ij}=z_{ij}^{\mathsf T}b_i+u_i(t_{ij})\). Under independence of \(b_i\) and \(u_i(\cdot)\),
The first term is covariance due to the subject-level random effects; the second is covariance due to temporal proximity or another feature encoded by \(K\). If \(b_i\) includes only a random intercept with variance \(\tau^2\), its contribution is \(\tau^2\) for every pair of visits. A temporal process can add a covariance that decreases, oscillates, or otherwise varies with the observation times.
For different subjects \(i\ne r\), the model’s independent-subject assumption gives
This is a statement about the latent effects under this model, not a claim that outcomes in different subjects can never be associated in reality. A shared calendar-time event, household, clinic, or other common source of variation would violate the assumption unless represented in the model.
For equally spaced visits, a stationary discrete-time AR(1) process is one possible choice:
Here \(\sigma_u^2\) is the process variance and \(\rho\) is the lag-one correlation. When observation times are irregular, an index-based AR(1) can misrepresent dependence because adjacent visit numbers need not be equally separated in time. A continuous-time Ornstein–Uhlenbeck (OU) process instead uses
The OU covariance depends on elapsed time rather than visit number; \(\lambda\) has inverse-time units. Both forms are examples, not universal choices. The assumed covariance must match the scientific time scale and the pattern of dependence the model is intended to capture.
Worked Example: Binary Outcomes With a Subject-Specific OU Process
Suppose a hypothetical study records a binary event at two visits per subject, at times 0 and 2 years. Consider the logit model \( \operatorname{logit}(\mu_{ij})=x_{ij}^{\mathsf T}\beta+b_i+u_i(t_{ij}) \), with \(b_i\sim N(0,0.36)\) and an independent OU process with \(\sigma_u^2=0.25\) and \(\lambda=0.40\) per year. Assume subjects are independent, and outcomes are conditionally Bernoulli and independent across visits given the latent effects.
The random-intercept contribution to the latent covariance between the two visits is \(0.36\). Their OU-process covariance is
Thus the total latent covariance is \(0.36+0.1123=0.4723\). At either visit, the latent random contribution has variance \(0.36+0.25=0.61\), so its latent correlation across the two visits is \(0.4723/0.61\approx0.7743\). This is a correlation on the latent predictor scale, not the correlation of the observed binary outcomes.
For two different subjects, the latent covariance is zero under the stated independent-subject assumption. The model therefore allows strong dependence between a subject’s visits while not assigning dependence between subjects. This example checks the structural assumptions by explicitly specifying the response distribution, process, random intercept, and cross-subject independence.
Likelihood and a Discrete-Time Example
Conditional independence of outcomes does not mean that the repeated outcomes are independent after the latent effects are integrated out. For subject \(i\), the marginal likelihood contribution is obtained by integrating the product of conditional response probabilities over that subject’s random effects and temporal process:
With independent subjects, the full likelihood is \(\prod_i L_i\). The integral is generally not available in closed form for non-Gaussian responses, so estimation uses an appropriate numerical or approximation method. The covariance specification changes this integration and the implied within-subject dependence; it is not merely a choice of standard-error adjustment.
Worked Example: Count Outcomes at Equally Spaced Visits
Consider hypothetical counts \(2,3,1,4\) for one subject at four equally spaced visits. Suppose a Poisson log-link GLMM uses a fixed-effects predictor \(x_{ij}^{\mathsf T}\beta\), a random intercept \(b_i\) with variance \(0.25\), and an independent stationary AR(1) process with \(\sigma_u^2=0.49\) and \(\rho=0.60\). The counts are conditionally independent Poisson outcomes given these latent effects.
The latent variance at a visit is \(0.25+0.49=0.74\). For visits two steps apart, the process covariance is
Adding the random-intercept covariance gives total latent covariance \(0.25+0.1764=0.4264\). The corresponding latent correlation is \(0.4264/0.74\approx0.5762\). For visits one step apart, the total latent covariance is \(0.25+0.49(0.60)=0.5440\); for visits three steps apart, it is \(0.25+0.49(0.60)^3=0.3559\). The constant \(0.25\) component persists at all lags, whereas the AR(1) contribution declines with lag.
These calculations describe the model’s latent covariance structure; the observed counts themselves are not normally distributed, and their covariance is not obtained by simply labeling these latent correlations as response correlations. The observed values alone do not determine the covariance parameters: those parameters are estimated using data from many subjects under the assumed model.
When the times are not equally spaced, the OU process offers a time-based alternative. For example, with hypothetical observations at 0, 1, and 4 years, \(\sigma_u^2=0.64\), and \(\lambda=0.20\) per year, the covariance between years 1 and 4 is
Worked Example: Calculating Dependence Across an Irregular Time Gap
Assume the same OU process is independently replicated across subjects, with no random intercept in this illustration. For the two visits separated by three years,
The process variance at either time is \(0.64\), so the latent correlation is \(0.3512/0.64\approx0.5488\). For observations from two different subjects, the covariance is zero under the independent-subject assumption. This specification responds to the elapsed three-year gap directly; treating these as three visit-index steps would have no general justification unless the index-based model matched the actual schedule.
Choosing a Structure and Checking Its Adequacy
The main alternatives encode different dependence assumptions. A random intercept captures stable subject heterogeneity but gives a constant latent covariance across visits. Random intercepts and slopes allow person-specific trajectory levels and rates of change, but do not automatically model additional short-term serial dependence. A temporal process can be added to those random effects to represent such dependence. A model with conditionally independent observations and no temporal process is simpler, but may leave within-subject dependence unexplained. Marginal approaches such as generalized estimating equations target a different modeling strategy; they should not be interpreted as interchangeable with a subject-specific latent-process GLMM.
| Structure | What it represents | Important limitation |
|---|---|---|
| Random intercept | Persistent subject-level shift | Same latent covariance contribution for all visit pairs |
| Random intercept and slope | Subject differences in trajectory level and rate | May not capture residual short-term temporal dependence |
| Subject-specific temporal process | Latent deviations whose covariance depends on time or lag | Requires enough repeated observations to inform its parameters |
| Conditionally independent GLMM | Mean structure and subject effects without an extra process | Can underrepresent residual within-subject dependence |
Useful checks include plotting fitted subject trajectories and residual summaries against time, examining whether unexplained within-subject patterns remain, and comparing plausible covariance structures using appropriate fit criteria or predictive checks. A process that is too weak may leave serial pattern in residuals; one that is too flexible may yield unstable estimates, boundary estimates, or large uncertainty. Repeated observations at only a few similar time gaps may provide little information for separating a random intercept from a slowly decaying process. Likewise, the process variance can be weakly identified if there are few visits per subject.
Interpret diagnostics cautiously: response residuals for binary or sparse count outcomes can be noisy, and apparent residual dependence may arise from a misspecified mean trend, omitted covariates, or informative observation timing rather than the covariance function alone. Conversely, a good in-sample fit does not establish that the subjects are truly independent. If subjects share a clinic, household, or calendar-time exposure, unexplained cross-subject patterns indicate that the independent-subject assumption may be inadequate.
The model has limits. It assumes the chosen link, response distribution, and latent covariance are adequate; Gaussian latent effects may be a poor description in some applications. Estimation can be computationally demanding, and covariance parameters can be difficult to identify in short or sparsely observed panels. A subject-specific process also does not, by itself, account for dependence between subjects. It should not be used to disguise a missing trend or common external shock.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A GLMM has a random intercept variance of \(0.20\) and an independent temporal process with covariance \(K(t,s)\). Write the same-subject latent covariance for visits \(j\) and \(k\).
- An OU process has \(\sigma_u^2=0.50\), \(\lambda=0.30\) per month, and visits three months apart. Calculate its process covariance.
- What latent covariance does the independent-subject assumption assign to two observations from different subjects?
- Why does a random intercept alone not generally represent a temporal process?
Solutions. (1) The covariance is \(0.20+K(t_{ij},t_{ik})\), since the random intercept contributes \(0.20\) and the independent process contributes \(K\). (2) \(K=0.50e^{-0.30(3)}=0.50e^{-0.9}\approx0.2033\). (3) It is zero for the latent effects under independent-subject sampling and no shared effect. (4) The random-intercept contribution is constant for every pair of visits, whereas a temporal process can vary with elapsed time or lag, so the intercept alone cannot express a changing covariance pattern.