Tutorials › Pattern-Mixture and Selection Models › Count Outcomes Across Dropout Patterns

Pattern-Mixture Models for Dropout · Tutorial 213 of 499

Count Outcomes Across Dropout Patterns

Specify count distributions within dropout patterns, check whether Poisson variation is adequate, and calculate fitted summaries without confusing observed counts with unobserved outcomes.

Advanced 12 min read

What You'll Learn

  • Write pattern-specific Poisson and negative-binomial models for repeated counts.
  • Interpret the mean and variance assumptions of each count distribution.
  • Calculate fitted Poisson means from counts and exposure.
  • Use a moment-based overdispersion check to illustrate negative-binomial dispersion.
  • Combine pattern-specific fitted means using pattern weights when the outcomes are observed.
  • Distinguish within-pattern count models from assumptions about missing outcomes and serial dependence.

Count outcomes within dropout patterns

A repeated count outcome might record the number of symptoms, hospital admissions, or episodes of a behavior at each visit. Counts are nonnegative integers, and their variability often increases with their mean. In a pattern-mixture analysis, the count distribution is specified separately for each dropout pattern. This allows, for example, the expected count trajectory among participants who complete follow-up to differ from the trajectory among those who drop out earlier.

Let \(Y_{it}\) be the count for participant \(i\) at visit \(t\), and let \(G_i=g\) denote that participant’s dropout pattern. Write \(\mu_{git}=E(Y_{it}\mid G_i=g,\mathbf{x}_{it})\), where \(\mathbf{x}_{it}\) contains the covariates used to model the count. A pattern-specific model has its own parameters for each \(g\), unless the analyst deliberately imposes constraints tying patterns together.

Definition: A pattern-specific count model specifies the distribution of \(Y_{it}\), conditional on dropout pattern \(G_i=g\) and included covariates. A Poisson model sets the conditional mean and variance equal. A negative-binomial model allows the conditional variance to exceed the mean through an additional dispersion parameter.

The count distribution is only one part of the repeated-outcome model. Repeated counts from the same participant may be associated. Specifying each visit’s conditional distribution does not, by itself, specify that within-participant dependence. A likelihood that treats observations as independent needs an appropriate independence assumption; otherwise, the joint model or the method used for uncertainty estimation must account for dependence.

Pattern-specific Poisson and negative-binomial models

For a Poisson model, a common specification is a log-linear mean:

$$ Y_{it}\mid G_i=g,\mathbf{x}_{it}\sim\operatorname{Poisson}(\mu_{git}), \qquad \log(\mu_{git})=\mathbf{x}_{it}^{\mathsf T}\boldsymbol{\beta}_g. $$

The vector \(\boldsymbol{\beta}_g\) can include an intercept, visit indicators, treatment, baseline covariates, and interactions that vary by dropout pattern. The log link guarantees a positive fitted mean. If counts are measured over different lengths of observation time, an exposure \(e_{it}>0\) can be included as an offset:

$$ \log(\mu_{git})=\log(e_{it})+\mathbf{x}_{it}^{\mathsf T}\boldsymbol{\beta}_g. $$

Here the model is for the expected count over exposure \(e_{it}\); the exponential of the linear predictor excluding the offset is the expected count per unit exposure. The offset coefficient is fixed at one, so this specification assumes expected counts are proportional to exposure.

Under the Poisson model, \(E(Y_{it}\mid G_i=g,\mathbf{x}_{it})=\mu_{git}\) and \(\operatorname{Var}(Y_{it}\mid G_i=g,\mathbf{x}_{it})=\mu_{git}\). This equality is a substantive distributional restriction, not just a computational convenience. If the data vary more than this restriction allows, estimated uncertainty and fitted probabilities can be unreliable.

A common alternative is the NB2 negative-binomial model. It retains a log-linear mean but adds a dispersion parameter \(\alpha_g\):

$$ E(Y_{it}\mid G_i=g,\mathbf{x}_{it})=\mu_{git}, \qquad \operatorname{Var}(Y_{it}\mid G_i=g,\mathbf{x}_{it}) =\mu_{git}+\alpha_g\mu_{git}^{2}, \qquad \alpha_g>0. $$

The parameter \(\alpha_g\) can be shared across patterns or estimated separately, if the data support that choice. As \(\alpha_g\) approaches zero, the variance approaches the Poisson variance. One equivalent parameterization uses size \(r_g=1/\alpha_g\) and success probability \(q_{git}=r_g/(r_g+\mu_{git})\); then \(Y_{it}\) has a negative-binomial distribution with mean \(\mu_{git}\) and variance \(\mu_{git}+\mu_{git}^2/r_g\).

Estimating and checking the count model

In a pattern-specific analysis, estimate the model using the observations available in that pattern and at the visits they actually contribute. The usual Poisson likelihood assumes the specified conditional Poisson distributions and, if it is formed as a product of visit-level probabilities, conditional independence of those observations. The analogous product likelihood for negative-binomial observations makes the corresponding independence assumption. If repeated outcomes are associated, that assumption must be justified or the dependence addressed in the analysis.

A basic check for extra-Poisson variation compares observed variation with the Poisson mean-variance restriction. For fitted means \(\hat{\mu}_{it}\), one summary is the Pearson dispersion ratio, the Pearson statistic divided by its residual degrees of freedom. A value notably above one suggests more variation than the Poisson model predicts, though it is not a stand-alone test and can be affected by an incorrect mean model or dependence. Also compare observed and fitted count frequencies, including zeros and large counts. A negative-binomial model can accommodate extra variation, but it does not automatically fix a misspecified mean, unmodeled serial association, or an unsuitable pattern definition.

Pattern-specific estimation can be unstable when a pattern has few participants, few observed counts, or very little variation. Warning signs include large standard errors, estimates that change substantially under small modeling changes, or difficulty obtaining a finite estimate of dispersion. A more flexible model is not necessarily better when the pattern-specific data are sparse.

Worked Example: Fitting Pattern-Specific Poisson Means

Suppose a hypothetical study records counts during a common one-week window at a visit. In the completer pattern, 40 participants have a total of 48 events. In the early-dropout pattern, 30 participants have a total of 18 events at this same visit, which occurs before those participants drop out. Fit an intercept-only Poisson model separately in each pattern, assuming equal one-week exposure and independent counts for this calculation.

For an intercept-only Poisson model, each participant in pattern \(g\) has mean \(\mu_g\). Ignoring constants, the log likelihood is \(\ell(\mu_g)=\sum_i [y_i\log(\mu_g)-\mu_g]\). Differentiating and setting the derivative to zero gives \(\sum_i y_i/\mu_g-n_g=0\), so \(\hat{\mu}_g=\sum_i y_i/n_g\).

For completers, \(\hat{\mu}_{C}=48/40=1.20\) events per participant per week. The fitted expected total for these 40 participants is \(40(1.20)=48\) events.

For early dropouts, \(\hat{\mu}_{E}=18/30=0.600\) events per participant per week. The fitted expected total for these 30 participants is \(30(0.600)=18.0\) events.

These are pattern-specific fitted means for an observed visit. They do not estimate the early-dropout participants’ count at a later visit after they have dropped out. Estimating that unobserved count would require additional assumptions beyond fitting these observed-data Poisson means.

Worked Example: Assessing Extra-Poisson Variation

Consider the following hypothetical counts from eight participants in one dropout pattern at a common visit: \(0,0,1,1,2,3,5,8\). Treat these counts as independent observations with a common mean for this illustrative moment calculation. Compare their sample variance with the Poisson variance and calculate a moment estimate of the NB2 dispersion.

The total is \(20\), so the sample mean is \(\bar{x}=20/8=2.50\) counts. The sum of squared observations is \(104\). Thus the sample variance is \(s^2=[104-20^2/8]/(8-1)=(104-50)/7=7.714\) counts squared.

A Poisson distribution with mean \(2.50\) predicts variance \(2.50\), whereas the observed sample variance is \(7.714\). The sample variance-to-mean ratio is \(7.714/2.50=3.086\), which indicates substantial extra variation in this small example.

For the NB2 variance formula, equating the sample variance to \(\bar{x}+\alpha\bar{x}^2\) gives the moment estimate \(\hat{\alpha}=(s^2-\bar{x})/\bar{x}^2=(7.714-2.50)/(2.50)^2=5.214/6.25=0.8342\). The resulting fitted variance is \(2.50+0.8342(6.25)=7.714\) counts squared, up to rounding.

This calculation illustrates the dispersion parameter; it is not a formal test or a substitute for fitting and checking the full model. With only eight observations, the estimate is especially uncertain. A fitted negative-binomial model would also need its mean structure and any repeated-measure dependence handled appropriately.

Calculating fitted count summaries

Once a model has been fitted, its mean gives an expected count, not a guarantee that any individual will have that count. Under a Poisson model, the probability of a particular count \(k\) is \(e^{-\mu}\mu^k/k!\). Under the NB2 parameterization above, \(r=1/\alpha\) and \(q=r/(r+\mu)\), so the probability of zero is \(q^r\). These probabilities describe the fitted distribution conditional on the specified pattern and covariates.

Pattern weights can be used to combine fitted means for a visit where the relevant pattern-specific outcomes are observed. If \(w_g=P(G_i=g)\), then the population mean at visit \(t\) is \(\sum_g w_g E(Y_{it}\mid G_i=g)\), provided each conditional mean refers to that same visit and target population. This is the standard mixture identity: partition the expectation by pattern, then sum the conditional expectations weighted by pattern probabilities. The earlier material on pattern weights and the population mixture develops this identity; here it is applied to counts.

Worked Example: A Pattern-Weighted Mean and Fitted Probability

Suppose, hypothetically, that at a visit observed before either pattern drops out, the pattern weights are \(w_C=0.60\) for completers and \(w_E=0.40\) for early dropouts. Fitted Poisson means are \(1.20\) and \(0.60\) events per participant per week, respectively. Calculate the population mean and the fitted probability of zero events.

The pattern-weighted mean is \(0.60(1.20)+0.40(0.60)=0.72+0.24=0.960\) events per participant per week.

For the probability of zero, first calculate it in each pattern. For completers, \(P(Y=0\mid C)=e^{-1.20}=0.3010\). For early dropouts, \(P(Y=0\mid E)=e^{-0.60}=0.5488\). The mixture probability is therefore \(0.60(0.3010)+0.40(0.5488)=0.1806+0.2195=0.4001\), or approximately \(0.400\).

The mixture mean and the probability of zero answer different questions: the first is an expected number of events per participant per week, and the second is the proportion with no events predicted by the mixture distribution. These calculations apply at the stated observed visit. They do not establish a mean or zero-count probability at visits where a pattern has no observed outcomes.

Assumptions, alternatives, and limitations

The pattern-specific Poisson model is most defensible when its conditional mean structure is appropriate and the Poisson variance restriction is reasonable. The negative-binomial model is an alternative when there is extra-Poisson variation that is adequately represented by its variance form. Neither distribution automatically captures every source of heterogeneity. For example, a negative-binomial variance does not itself specify how counts at different visits from the same participant are associated.

Compared with Poisson regression, negative-binomial regression adds a dispersion parameter and can produce wider predictive distributions when counts vary substantially around their fitted means. In return, it estimates an additional parameter and can be difficult to fit reliably in small dropout patterns. A Poisson fit may still be useful for its mean structure when the variance assumption is imperfect, but uncertainty calculations must then account for the discrepancy and any within-participant dependence; the ordinary independent Poisson likelihood alone does not do so.

A crucial limitation is that pattern membership does not reveal outcomes after dropout. The conditional distribution of counts observed in a pattern can be estimated from available data, but a pattern-specific fitted value at an unobserved post-dropout visit is not learned merely by extending a fitted curve. Such a prediction needs an explicit extrapolation assumption. Pattern-specific count modeling describes the observed pattern structure; it does not, by itself, solve the missing-data identification problem.

Common pitfall: Seeing a sample variance larger than the sample mean and concluding that negative-binomial regression has solved both overdispersion and dropout. The variance comparison is evidence against the simple Poisson variance restriction, not proof of the negative-binomial model. And neither count distribution identifies an outcome that was never observed; state the pattern, visit, and assumptions behind every fitted summary.

Check Your Understanding

Answer each question, then compare with the solutions below.

  1. A pattern has 25 participants, each observed for the same exposure window, with 40 total events. What is the intercept-only Poisson fitted mean per participant-window?
  2. For an NB2 model with fitted mean \(\mu=2\) and \(\alpha=0.5\), calculate the fitted variance.
  3. Two patterns have weights \(0.75\) and \(0.25\), and observed-visit fitted means of \(1.0\) and \(3.0\) events per week. What is the pattern-weighted mean?
  4. Why does a negative-binomial model not automatically account for correlation between two counts from the same participant?

Solutions. (1) The fitted mean is \(40/25=1.60\) events per participant-window. (2) The NB2 variance is \(2+0.5(2^2)=2+2=4.00\) counts squared. (3) The weighted mean is \(0.75(1.0)+0.25(3.0)=0.75+0.75=1.50\) events per week. (4) The negative-binomial specification gives a marginal conditional distribution and its mean-variance relationship; it does not specify a joint distribution across visits. Within-participant dependence must be modeled or otherwise accommodated separately.

Clinical Biostats