Count Outcomes With Overdispersed Dropout Models

Why Count Outcomes Need a Count-Specific Selection Model

Repeated outcomes such as symptom counts, hospital visits, or adverse-event counts are nonnegative integers. Their variability can exceed what a Poisson model allows, so an overdispersed count distribution may be appropriate. When dropout depends on the count at the current visit, the outcome model and dropout model must be combined: the dropout probability for a participant whose current count is unobserved is an average over possible counts, not the hazard evaluated at one substituted value.

The model below is conditional on information available before the current visit, such as baseline covariates and earlier outcomes. At visit $t$, let $R_{it}=1$ indicate that participant $i$ is still under follow-up and at risk of dropout. Let $Y_{it}$ be the current count, and let $D_{it}=1$ indicate dropout after that visit, with $D_{it}=0$ indicating continued follow-up. The ordering matters: the current count is the value on which the dropout hazard is allowed to depend.

Definition: A negative-binomial distribution with mean $\mu>0$ and size $r>0$ has probability mass function $$ f(y\mid \mu,r)=\frac{\Gamma(y+r)}{\Gamma(r)y!} \left(\frac{r}{r+\mu}\right)^r \left(\frac{\mu}{r+\mu}\right)^y,\qquad y=0,1,2,\ldots. $$ Its variance is $\mu+\mu^2/r$, which exceeds its mean for finite $r$. In a repeated-count model, parameters such as $\mu_{it}$ may depend on visit, covariates, and past outcomes. The size $r$ controls overdispersion; smaller $r$ gives greater variance for a fixed mean.

One common mean model is $\log(\mu_{it})=x_{it}^{T}\beta$, where $x_{it}$ contains predictors known for the current visit. The selection equation can use a logistic hazard:

h_{it}(y)=P(D_{it}=1\mid R_{it}=1,Y_{it}=y,H_{it}) =\operatorname{expit}\{\alpha_t+z_{it}^{T}\gamma+\delta y\}, \qquad \operatorname{expit}(u)=\frac{\exp(u)}{1+\exp(u)}.

Here $H_{it}$ denotes the relevant observed history, $z_{it}$ contains predictors in the dropout equation, and $\delta$ is the conditional change in log dropout odds for a one-count increase, holding the other terms fixed. It is an association parameter in this selection model, not by itself a causal effect. The model assumes that, conditional on the specified history and current count, the hazard is given by this equation. The outcome distribution and hazard must both be specified before calculating a marginal risk.

Why the Dropout Probability Requires a Sum

If $Y_{it}$ is unobserved, its value is unknown, but the model assigns probabilities to all its possible values. The law of total probability therefore averages the hazard over the negative-binomial distribution. Conditional on the history, write the outcome mass as $f_{it}(y)=P(Y_{it}=y\mid H_{it})$. Then

P(D_{it}=1\mid R_{it}=1,H_{it}) =\sum_{y=0}^{\infty} f_{it}(y)h_{it}(y).

This is a discrete integral. It is generally not equal to $h_{it}(\mu_{it})$, and it is not generally equal to the hazard evaluated at the mean count, because the expit function is nonlinear. If a current count is observed, its joint contribution with the dropout indicator is instead evaluated at that count:

P(Y_{it}=y,D_{it}=d\mid R_{it}=1,H_{it}) =f_{it}(y)\{h_{it}(y)\}^{d}\{1-h_{it}(y)\}^{1-d}, \qquad d\in\{0,1\}.

When the count is unobserved, sum the appropriate joint probabilities over its possible values. In a repeated study, the conditional outcome distribution and dropout hazard are defined at each at-risk visit; the observed history changes as visits accrue. The sum over counts is still required wherever the current count is unobserved.

Worked Example: Parameterizing an Overdispersed Count Outcome

Suppose a hypothetical visit-specific count has mean $\mu=1.5$ and negative-binomial size $r=2$. Find its variance and the probabilities of counts zero, one, and two.

The variance is $1.5+1.5^2/2=1.5+1.125=2.625$ counts squared, greater than the mean of 1.5 counts. Under the stated mean-size parameterization, $r/(r+\mu)=2/3.5=4/7$, and $\mu/(r+\mu)=1.5/3.5=3/7$. Because $r=2$, the mass function simplifies to

f(y)= (y+1)\left(\frac{4}{7}\right)^2\left(\frac{3}{7}\right)^y =(y+1)\frac{16}{49}\left(\frac{3}{7}\right)^y.

Substitution gives $f(0)=1(16/49)=0.326531$, $f(1)=2(16/49)(3/7)=0.279883$, and $f(2)=3(16/49)(3/7)^2=0.179925$. These are probabilities for counts under the specified distribution. In particular, using $2/3$ as the probability factor for the count term would correspond to a different mean, not to this example.

Worked Discrete Integration

The next calculation shows how overdispersion and current-count selection combine. All values are hypothetical. Let the current count have the distribution in the preceding example, and let the dropout hazard be $h(y)=\operatorname{expit}(-2+0.5y)$. The positive slope means that, conditional on the model's other terms, larger counts have higher dropout odds. It does not mean that the count causes dropout.

Worked Example: Marginal Dropout Risk and a Count Tail

Calculate the marginal dropout probability and the probability that the count is at least four among those who drop out at this visit. The second quantity is $P(Y\ge4\mid D=1,H)$ for the specified hypothetical history.

For each count $y$, calculate the negative-binomial mass $f(y)$, the hazard $h(y)$, and their product. The product is the contribution of that count to the marginal dropout probability.

Count $y$	$f(y)$	$h(y)=\operatorname{expit}(-2+0.5y)$	$f(y)h(y)$
0	0.326531	0.119203	0.038923
1	0.279883	0.182426	0.051058
2	0.179925	0.268941	0.048389
3	0.102814	0.377541	0.038817
4	0.055079	0.500000	0.027540
5	0.028326	0.622459	0.017632
6	0.014163	0.731059	0.010354

For example, at $y=2$, the contribution is $3(16/49)(3/7)^2\{\exp(-1)/(1+\exp(-1))\} =0.179925(0.268941)=0.048389$. Summing the products for $y=0,\ldots,6$ gives approximately 0.232713. Continuing the same calculation for $y=7,\ldots,12$ adds approximately 0.011305; the contributions from $y\ge13$ add approximately 0.000138. Thus

P(D=1\mid H) =\sum_{y=0}^{\infty}f(y)h(y) \approx 0.232713+0.011305+0.000138 =0.244156.

For the tail probability among dropouts, the numerator is the sum of $f(y)h(y)$ over $y\ge4$. The contributions at counts 4 through 6 sum to $0.027540+0.017632+0.010354=0.055526$. Counts 7 through 12 add 0.011305, and counts 13 and above add 0.000138. Therefore

P(Y\ge4\mid D=1,H) =\frac{\sum_{y=4}^{\infty}f(y)h(y)} {\sum_{y=0}^{\infty}f(y)h(y)} \approx\frac{0.055526+0.011305+0.000138}{0.244156} =0.2743.

Under this model and history, the marginal dropout probability is about 24.42%. Among those who drop out, about 27.43% have a count of at least four according to the model. These are model-based probabilities, not claims about an observed population unless the model is appropriate for that population.

Likelihood Contributions and Repeated Visits

The same ingredients give a direct check on the joint model. If the current count is observed as $y=2$ and the participant continues follow-up, the probability contribution for the count and continuation is $f(2)\{1-h(2)\}$. For the hypothetical parameters above, $1-h(2)=1-0.268941=0.731059$, so

P(Y=2,D=0\mid R=1,H) =0.179925(0.731059)=0.131536.

If instead dropout occurs after an observed count of two, the joint contribution is $f(2)h(2)=0.048389$. These two contributions add to $f(2)$, as they should: $0.131536+0.048389\approx0.179925$. If the current count is not observed, the corresponding dropout contribution is not $f(2)h(2)$; it is the sum over all possible counts. For continued follow-up with an unobserved count, the sum is $\sum_y f(y)\{1-h(y)\}$, which equals one minus the marginal dropout probability.

For multiple visits, define the negative-binomial parameters conditional on the covariates and history specified for each visit, and define a hazard for each visit at risk. The participant's contribution across visits follows the model's conditional sequence: at a visit with an observed count, use its count probability and the appropriate hazard or continuation probability; when a count is unobserved, sum over its possible values. This construction does not justify treating counts as marginally independent. Any dependence across visits must be represented by the conditional outcome model and its history terms.

Worked Example: Why a Conditional Hazard Is Not a Marginal Risk

For the same hypothetical model, compare the hazard at count two with the overall dropout probability. At $y=2$, the linear predictor is $-2+0.5(2)=-1$, so the conditional hazard is

h(2)=\frac{\exp(-1)}{1+\exp(-1)}=0.268941.

The marginal probability, however, averages across all count values: $P(D=1\mid H)=\sum_y f(y)h(y)\approx0.244156$, as calculated above. The two values differ because the conditional hazard at count two applies only to that count, whereas the marginal probability weights every count by its probability. Evaluating the hazard at the mean count, $h(1.5)=\operatorname{expit}(-1.25)\approx0.2227$, also does not reproduce the integrated value. This is a concrete consequence of averaging a nonlinear function under a nondegenerate count distribution.

Assumptions, Diagnostics, and Alternatives

The calculations rely on several substantive and mathematical conditions. The negative-binomial distribution must reasonably describe the conditional count distribution, including its mean, dispersion, and probability of zero. The dropout equation must describe the conditional hazard over the count range that matters, and the timing must support using the current count in that hazard. The histories and covariates used in both equations must be defined consistently. For a finite set of model parameters, each count mass is nonnegative and the masses sum to one; each logistic hazard lies between zero and one. Consequently the integrated dropout probability is also between zero and one.

Useful checks include comparing observed and fitted count frequencies, especially the zero frequency and upper tail, and checking whether the fitted variance is compatible with the observed overdispersion. Inspect the implied dropout hazards across plausible counts: a modest coefficient can still produce hazards close to zero or one at extreme counts. Numerically, sum enough terms that the remaining negative-binomial tail is negligible for the desired precision. Because $0\le h(y)\le1$, the omitted contribution after truncating at $K$ is no larger than $P(Y>K)$. The same bound applies to the omitted part of a numerator that sums $f(y)h(y)$ over a subset of counts.

A Poisson outcome model is more restrictive because its conditional variance equals its mean; it may be inadequate when counts are overdispersed. A negative-binomial outcome model accommodates extra variation, but it does not by itself address dropout dependence. A hazard depending only on baseline covariates or past outcomes is a different selection assumption: it omits the direct dependence on the current count specified here. Gaussian, binary, and ordinal outcome distributions require different outcome probabilities and corresponding integration. None is interchangeable with the negative-binomial count model simply because the dropout equation has the same logistic form.

A common pitfall is to replace an unobserved current count by its mean, then calculate $h(\mu)$, or to insert a guessed count into the hazard. That produces a conditional hazard for a chosen count, not the marginal dropout probability. The correction is to use the model's full conditional count distribution and calculate $\sum_y f(y)h(y)$, checking the tail error if the infinite sum is evaluated numerically. Another pitfall is to treat $\delta$ as a causal effect: it describes conditional selection association under the specified joint model, and its interpretation depends on that model and its assumptions.

Key takeaway: For overdispersed repeated counts, specify the conditional negative-binomial distribution and the current-count dropout hazard separately. When the current count is unobserved, obtain the dropout probability by summing the hazard weighted by the probability of every possible count.

Check Your Understanding

Answer each question, then compare your reasoning with the brief solutions.

For a negative-binomial count with mean 2 and size 4, what are its variance and the two probability factors $r/(r+\mu)$ and $\mu/(r+\mu)$?
Why is $h(\mu)$ generally not the marginal dropout probability when the hazard depends on the current count?
If the count is observed as $y$ and $D=0$, write the joint probability contribution.
When a numerical sum is truncated after count $K$, give an upper bound for its omitted dropout-probability contribution.

Solutions. (1) The variance is $2+2^2/4=3$; the factors are $4/(4+2)=2/3$ and $2/(4+2)=1/3$. (2) The marginal probability is $\sum_y f(y)h(y)$; a nonlinear hazard cannot generally be moved outside this average. (3) The contribution is $f(y)\{1-h(y)\}$. (4) Since $h(y)\le1$, the omitted contribution is at most $\sum_{y>K}f(y)=P(Y>K)$.