Tutorials › Generalized Linear Mixed Models (GLMM) › Counts Recorded Over Unequal Observation Windows

Responses: Binary, Count, Ordinal, Overdispersed · Tutorial 86 of 698

Counts Recorded Over Unequal Observation Windows

Learn to distinguish event counts from observation time and account for unequal exposure when specifying and interpreting a count GLMM.

Intermediate 12 min read

What You'll Learn

  • Distinguish a count response from the time or opportunity over which it is observed.
  • Derive the log-exposure offset from a constant-rate Poisson model.
  • Interpret fixed effects as conditional rate comparisons in a count GLMM.
  • See why modelling raw counts or dividing counts by exposure can be misleading.
  • Check the assumptions behind proportional exposure scaling and identify common failures.

Counts Depend on Both Events and Observation Time

A count records how many events occurred. But the same count can mean different things if one observation covers a week and another covers a year. Conversely, a longer observation window can produce a larger count even when the underlying event rate is unchanged. In a generalized linear mixed model (GLMM), the response remains the count; the length of time at risk is additional information used to specify its expected value.

For example, counting hospital admissions over follow-up requires distinguishing the number of admissions from the time each person was followed and eligible to be admitted. A count of four admissions over two years and a count of four over eight years are not, by themselves, evidence of the same event rate.

Definition: The count response \(Y_{ij}\) is the number of events observed for unit \(j\) in cluster \(i\). The exposure \(t_{ij}\) is the known, nonnegative amount of observation time or opportunity during which those events could be recorded. When the event rate is defined per unit of exposure, the count mean is modeled as exposure multiplied by that rate.

Exposure must have a meaningful, consistent unit, such as person-years, machine-hours, or days at risk. It is not necessarily the calendar duration of a study: if a person enters late, leaves early, or is not at risk during part of follow-up, their relevant exposure can be less than the study's full duration. Exposure is not the count, and it is not a binomial denominator unless the response process is actually a set of fixed opportunities with success or failure outcomes.

The Exposure Offset in a Count GLMM

Suppose the conditional event rate for unit \(j\) in cluster \(i\) is \(\lambda_{ij}\), measured as expected events per unit of exposure. Under a constant-rate Poisson process, the expected count over exposure \(t_{ij}\) is \(t_{ij}\lambda_{ij}\). A common Poisson GLMM specifies the rate through fixed and random effects:

$$ Y_{ij}\mid b_i \sim \operatorname{Poisson}(\mu_{ij}),\qquad \mu_{ij}=t_{ij}\lambda_{ij},\qquad \log(\lambda_{ij})=x_{ij}^{\mathsf T}\beta+z_{ij}^{\mathsf T}b_i. $$

Here \(x_{ij}\) and \(z_{ij}\) are the fixed-effect and random-effect design vectors, \(\beta\) is the fixed-effect coefficient vector, and \(b_i\) is the cluster-specific random-effect vector. Taking the logarithm of the mean gives

$$ \log(\mu_{ij})=\log(t_{ij})+x_{ij}^{\mathsf T}\beta+z_{ij}^{\mathsf T}b_i. $$

The term \(\log(t_{ij})\) is an offset: a known model term whose coefficient is fixed at 1. It makes the expected count proportional to exposure while allowing the covariates and random effects to describe the rate. Thus, holding the covariates and random effects fixed, doubling exposure doubles the expected count. A fixed effect \(\beta_k\) instead compares rates: increasing its predictor by one unit multiplies the conditional rate, and hence the conditional expected count at the same exposure, by \(\exp(\beta_k)\).

Assumptions for the basic Poisson offset model: Exposure is measured correctly and is positive for each modeled observation. Conditional on the random effects and included predictors, counts follow the chosen count distribution and are independent across modeled observations. For the simple Poisson rate interpretation, the conditional event rate is constant within each observation window, or the predictors and rate structure adequately represent its variation. The log-exposure coefficient is fixed at 1, so expected counts scale proportionally with exposure. If random effects are modeled, their distribution and their role in the conditional mean must also be specified.

These assumptions matter. Unequal exposure does not automatically make a Poisson model appropriate, nor does adding an offset correct omitted predictors, dependence, or an unsuitable variance distribution. A negative binomial GLMM can also include \(\log(t_{ij})\) as an offset when its conditional mean is proportional to exposure. Its additional variance structure addresses a different feature of the data; it does not remove the need to define exposure or justify the mean scaling.

Why the Offset Has Its Particular Form

The log offset follows from the rate definition, rather than being a discretionary adjustment. If \(\lambda_{ij}\) is the expected number of events per unit exposure, then the expected number in an observation window of length \(t_{ij}\) is \(t_{ij}\lambda_{ij}\). If the log rate is the linear predictor \(\eta_{ij}=x_{ij}^{\mathsf T}\beta+z_{ij}^{\mathsf T}b_i\), then \(\lambda_{ij}=\exp(\eta_{ij})\), so

$$ \mu_{ij}=t_{ij}\exp(\eta_{ij}) \quad\Longrightarrow\quad \log(\mu_{ij})=\log(t_{ij})+\eta_{ij}. $$

This also explains why the offset coefficient is 1. Exposure multiplies the rate before the logarithm is taken; its log therefore enters additively without an estimated multiplier. If the coefficient on \(\log(t_{ij})\) were estimated freely, the model would no longer impose the proportional scaling implied by a rate per unit exposure.

Worked Example: Same Rate, Different Windows

Suppose a hypothetical process has a conditional rate of 2 events per person-year for two otherwise identical people. One is observed for 0.5 person-years and the other for 2 person-years. Take their random effects to be zero for this calculation.

The log rate is \(\log(2)\). For the first person, the fitted mean count is

$$ \mu_1=0.5\exp(\log(2))=0.5(2)=1\ \text{event}. $$

For the second person, it is

$$ \mu_2=2\exp(\log(2))=2(2)=4\ \text{events}. $$

The expected count is four times as large because the exposure is four times as long; the modeled rate is 2 events per person-year in both cases. These are expected counts, not guarantees about the observed counts. Under the Poisson assumption, the two counts are random around their respective means.

Worked Rate Comparisons

A model without exposure adjustment can confuse a longer window with a higher event rate. The following hypothetical example shows the difference using a simple Poisson model with a group-specific rate. It illustrates the offset calculation; it does not include a cluster random effect.

Worked Example: Comparing Groups With Unequal Total Exposure

Suppose hypothetical group A has 12 events over 30 person-years and group B has 18 events over 20 person-years. Assume independent Poisson counts with a constant rate within each group. Under this model, the maximum-likelihood rate estimate is the total count divided by the total exposure in that group.

The estimated rates are

$$ \hat{\lambda}_A=\frac{12}{30}=0.400\ \text{events per person-year}, \qquad \hat{\lambda}_B=\frac{18}{20}=0.900\ \text{events per person-year}. $$

The estimated rate ratio for B versus A is

$$ \widehat{\mathrm{RR}}=\frac{0.900}{0.400}=2.25. $$

The equivalent log rate-ratio estimate is \(\log(2.25)=0.8109\). In a log-link model, group B's coefficient is therefore estimated as 0.8109 relative to group A, with the log of each group's exposure entered as an offset. The fitted result describes an estimated rate in group B 2.25 times the rate in group A in this hypothetical comparison. It does not establish that group membership caused a rate difference.

For comparison, dividing the counts alone gives \(18/12=1.50\), not 2.25. That raw-count ratio fails to account for group A's greater total exposure. A formal GLMM with clustered observations would additionally model the specified random effects; the group-level arithmetic here is not a substitute for that analysis.

In a GLMM, the rate comparison is usually conditional on the random effects: the fixed-effect rate ratio compares observations with the same random-effect values and other modeled predictors, at equal exposure. The offset accounts for exposure, while the random effects account for the modeled cluster-level variation. They have distinct roles.

Worked Example: Reading a GLMM Prediction With an Offset

Consider a hypothetical Poisson random-intercept model with log rate \(-1+0.4A+b_i\), where \(A=0\) for the reference condition and \(A=1\) for the comparison condition. The exposure is measured in years. For a prediction at \(b_i=0\), calculate the expected count for a reference observation with 0.5 years of exposure and a comparison observation with 2 years of exposure.

For the reference observation, the log rate is \(-1\), so its rate is \(\exp(-1)=0.3679\) events per year. Its mean count is

$$ \mu_{\mathrm{ref}}=0.5\exp(-1)=0.1840\ \text{events}. $$

For the comparison observation, the log rate is \(-1+0.4=-0.6\), so its rate is \(\exp(-0.6)=0.5488\) events per year. Its mean count is

$$ \mu_{\mathrm{comp}}=2\exp(-0.6)=1.098\ \text{events}. $$

The fixed-effect rate ratio is \(\exp(0.4)=1.492\), conditional on \(b_i\) and the other predictors. The ratio of these two predicted counts is larger because the comparison observation also has four times the exposure: \(1.098/0.1840\approx 5.97\), which equals \(4\exp(0.4)\). A count ratio across unequal windows is therefore not the fixed-effect rate ratio.

Choosing and Checking an Exposure Model

Including the offset is appropriate when exposure is known and the scientific model says that the expected count scales in proportion to it. A simple alternative is to use the rate \(Y_{ij}/t_{ij}\) as a continuous response. That changes the response distribution: the observed rate is not generally Poisson, can take noninteger values, and has a variability that depends on exposure. A count likelihood with an offset uses the count's support and lets the exposure inform its mean directly.

Another alternative is to enter \(\log(t_{ij})\) as an ordinary predictor and estimate its coefficient. This is useful only when the aim and assumptions justify estimating an exposure elasticity rather than fixing proportional scaling. For example, a coefficient of 0.8 would imply that multiplying exposure by two multiplies the expected count by \(2^{0.8}\approx1.741\), not by 2. It is not the standard offset model for a rate per unit exposure.

Check how the model behaves against the way observation windows were formed. Useful diagnostics and questions include:

  • Plot or summarize observed counts and fitted counts against exposure. Large systematic discrepancies can indicate a poor mean specification or inadequate covariates.
  • Check whether follow-up time is actually time at risk. Calendar time that includes periods when an event could not occur is not the appropriate denominator for that rate.
  • Investigate whether the rate changes over the window. If it does, a single constant-rate term may be inadequate; time-varying predictors or a model that represents changing rates may be needed.
  • Check whether exposure is related to unmodeled risk, event occurrence, or dropout. An offset corrects the arithmetic of unequal exposure; it does not by itself correct informative observation or selection.
  • Check the conditional variance and dependence assumptions separately. Extra-Poisson variation or within-cluster dependence may remain even with the correct offset, motivating a different count distribution or dependence structure.
Common pitfall: “Group B has more events, so its rate is higher.” This compares counts without their observation windows. The correction is to model the count with the log of valid exposure as an offset, or to compare counts per exposure only as a descriptive calculation when appropriate. A larger count can simply reflect a longer time at risk.

Exposure values must be positive in the ordinary log-offset formulation, because \(\log(0)\) is undefined and a zero-time window contains no opportunity for the modeled events. Such a row usually contributes no count information for that rate and should not be treated as an ordinary positive-exposure observation. Also, a zero count with positive exposure is entirely valid: it means no events were observed during that window, not that the exposure was zero.

The main limitation is that the offset imposes proportional scaling rather than discovering it from the data. If longer observation windows have systematically different event rates, or if exposure is measured inaccurately, the rate interpretation may be misleading. Choosing a count distribution and random-effects structure also requires attention to the response process; the offset alone cannot repair a misspecified GLMM.

Check Your Understanding

Try each question, then compare your reasoning with the solutions below.

  1. A hypothetical unit has 5 events in 3 months at risk. What is its observed rate per year? At that rate, what is the expected count over 1.5 months?
  2. A Poisson GLMM has log rate \(-2+0.5A+b_i\), with exposure \(t=4\) years. For \(A=1\) and \(b_i=0\), calculate the expected count and the rate ratio for \(A=1\) versus \(A=0\).
  3. Why can the raw count ratio between two groups differ from their rate ratio when exposures differ?
  4. What is wrong with applying the standard log-exposure offset to an observation with zero exposure? Is a zero count with positive exposure also invalid?

Solutions. (1) Three months is \(3/12=0.25\) years, so the observed rate is \(5/0.25=20\) events per year. One and a half months is \(1.5/12=0.125\) years, giving expected count \(20(0.125)=2.5\) events at that rate. (2) The log rate is \(-2+0.5(1)+0=-1.5\), so the rate is \(\exp(-1.5)=0.2231\) events per year. The expected count is \(4\exp(-1.5)=0.8925\) events. The rate ratio is \(\exp(0.5)=1.649\). (3) A count reflects both the rate and the length of the observation window. A group with more events could simply have more exposure, so rate comparisons must account for exposure. (4) The log offset \(\log(t)\) is undefined at \(t=0\), and a zero-time window has no opportunity to contribute events to this rate model. A zero count with positive exposure is valid: it is a possible outcome under a count distribution.

Clinical Biostats