Dependence in Repeated Categorical Outcomes

Why Repeated Outcomes Can Be Dependent

When a categorical outcome is recorded repeatedly for the same person, animal, or experimental unit, the observations need not be independent. A subject may have a persistent tendency toward one response category, or the category at one occasion may directly help predict the category at the next. These are different explanations of dependence. Treating them as interchangeable can lead to a model whose coefficient answers a different scientific question from the one intended.

A random-effects model represents persistent, unobserved differences among subjects through a subject-level random effect. A transition model represents dependence on observed response history by including one or more earlier outcomes as predictors. Either can be useful, and a model may include both structures when justified, but their coefficients have different interpretations.

Random-Effects Dependence

For a binary response, code \(Y_{it}=1\) for the event of interest and \(Y_{it}=0\) otherwise. One common random-intercept model is

Here \(x_{it}\) is a covariate, \(\beta_0\) and \(\beta_1\) are fixed effects, and \(b_i\) is shared across occasions for subject \(i\). A standard version assumes that, conditional on \(b_i\) and the modeled covariates, outcomes from different occasions are independent. The random effects are typically assumed independent across subjects and independent of the covariates. The chosen distribution for \(b_i\), commonly a mean-zero normal distribution, is part of the model and should be considered an assumption rather than a fact guaranteed by the data.

The coefficient \(\beta_1\) describes a change in the conditional log odds of the event for a one-unit increase in \(x_{it}\), holding the subject effect fixed. It is therefore a subject-specific, conditional effect. Because the logistic function is nonlinear, this conditional effect does not generally equal the effect calculated from probabilities averaged over the distribution of \(b_i\).

Even though observations are conditionally independent given \(b_i\), they are generally dependent after \(b_i\) is integrated out. For two binary outcomes at occasions \(s\) and \(t\), the law of total covariance gives

Under conditional independence, the first term is zero. If the occasions share the same covariate values and conditional event probability \(p(b_i)\), the second term is \(\operatorname{Var}\{p(b_i)\}\), which is nonnegative. Thus, variation in subjects' underlying response propensities induces positive marginal association between their repeated outcomes, even without a direct effect of one outcome on another.

Transition Dependence

A first-order transition model specifies the current outcome distribution conditional on the immediately preceding outcome and covariates. For a binary response, one form is

The coefficient \(\gamma_1\) describes how the log odds at occasion \(t\) differ when the previous recorded outcome is 1 rather than 0, at fixed current covariates. It is a conditional transition effect. It does not, by itself, represent a stable subject propensity, and it is not automatically a causal effect of changing the previous outcome. The model also needs a distribution for the first outcome, such as a separate regression for \(Y_{i1}\), because there is no observed \(Y_{i0}\) in the sequence.

The first-order Markov assumption is that, conditional on the current covariates and the immediately previous response, earlier outcomes add no further information about the current response: \(P(Y_{it}\mid Y_{i1},\ldots,Y_{i,t-1},x_{it})=P(Y_{it}\mid Y_{i,t-1},x_{it})\). This is a substantive restriction. If earlier response history still predicts the current category after accounting for \(Y_{i,t-1}\), a first-order transition model may be inadequate.

Nominal Outcomes and Transition Probabilities

For a nominal response with \(K\) categories, a transition model can specify a probability for each current category conditional on the previous category. Each row of the transition matrix corresponds to a previous category, and each column to a current category. Every entry is nonnegative and each row sums to 1. Covariates can be added by modeling these conditional probabilities, for example with a multinomial logit specification. The key interpretive feature remains: probabilities are conditioned on response history.

Choosing and Assessing the Dependence Structure

The distinction is primarily scientific, not just computational. A random-effects model asks whether subjects differ persistently in their response propensities after accounting for covariates. A transition model asks whether the current category depends on the observed recent category. They can imply similar observed correlations in some settings, but similar correlations do not make their parameters interchangeable.

Several checks help identify a poorly chosen dependence structure. For a random-effects model, inspect whether estimated subject heterogeneity is supported by the repeated data, whether the assumed random-effect distribution is plausible, and whether residual association remains across occasions after conditioning on the fitted effects and covariates. For a transition model, examine whether response history beyond the chosen lag still carries information, whether transition probabilities are adequately represented across covariate patterns, and whether sparse transitions make parameters unstable. In either approach, consider the actual observation schedule, missing outcomes, and whether the recorded prior response is the scientifically relevant history.

A transition model that omits important stable subject differences may attribute their persistence to the lagged response. Conversely, a random-intercept model may fail to capture short-term state dependence if the previous response has an additional role. Including both random effects and lagged outcomes is possible, but requires care: a prior outcome may be associated with the latent subject effect, so a simple conditional interpretation or standard likelihood factorization may not resolve all modeling concerns. The combined model must state its assumptions and estimation target clearly.

A population-averaged model is another possible approach when the target is a marginal mean or population-level association rather than a subject-specific or history-conditional effect. It does not automatically explain the source of dependence in the same way as either model above. The appropriate choice depends on the scientific question, the design, and whether the aim is subject-specific inference, transition prediction, or population-average description.