Predicting Dropout from Observed Outcome Histories

Why predict from a history?

In a shared-parameter model, an individual’s repeated outcomes and dropout process depend on a common latent effect. A person whose observed outcomes have been unusually high or low may therefore have a different predicted dropout risk from another person with the same measured covariates. The relevant information is not just the fitted population distribution of latent effects: it is the individual’s observed outcome and follow-up history.

That history does not reveal the latent effect exactly. Prediction must account for the posterior distribution of plausible effects given the observed data. This tutorial develops that calculation and shows why substituting a single estimated effect can give a different answer.

Definition: Let \(b_i\) be subject \(i\)’s shared latent effect, with population density or probability distribution \(f(b_i\mid\theta)\), where \(\theta\) denotes model parameters. Let \(H_{it}\) denote that subject’s observed history through time \(t\): the outcomes observed while under follow-up and the dropout or continued-follow-up information available through \(t\). The posterior distribution \(p(b_i\mid H_{it},\theta)\) describes uncertainty about the subject’s latent effect after conditioning on that history and the fitted model.

Constructing the posterior from observed history

Suppose \(Y_{ij}\) is an outcome observed at occasion \(j\), and \(h_{ij}(b_i)\) is the conditional probability of dropout in interval \(j\), given that the subject has not dropped out earlier. The outcome model has conditional density or probability mass \(g(Y_{ij}\mid b_i,\theta)\). A shared-parameter model assumes that, conditional on the latent effect and the included covariates, the specified outcome and dropout components give the appropriate joint likelihood. Their association is represented through their common dependence on \(b_i\).

For a subject observed through interval \(t\), with outcomes \(Y_{i1},\ldots,Y_{it}\) and no dropout through that interval, the history likelihood conditional on \(b_i\) is

L_i(H_{it}\mid b_i,\theta) = \left\{\prod_{j=1}^{t}g(Y_{ij}\mid b_i,\theta)\right\} \left\{\prod_{j=1}^{t}\left[1-h_{ij}(b_i)\right]\right\}.

The first product uses observed outcomes; the second uses the observed survival of follow-up. If dropout occurs in interval \(r\), the dropout contribution instead includes survival through earlier intervals and the event probability in interval \(r\):

\prod_{j=1}^{r-1}\left[1-h_{ij}(b_i)\right]h_{ir}(b_i).

Only outcomes actually observed before dropout contribute outcome likelihood factors. Unobserved outcomes after dropout are not treated as if they had been measured. Bayes’ rule combines the history likelihood with the population distribution of \(b_i\):

p(b_i\mid H_{it},\theta) = \frac{L_i(H_{it}\mid b_i,\theta)f(b_i\mid\theta)} {\int L_i(H_{it}\mid u,\theta)f(u\mid\theta)\,du}.

With a discrete approximation or quadrature nodes \(b_1,\ldots,b_K\), the integral becomes a weighted sum. Define \(q_k=L_i(H_{it}\mid b_k,\theta)P(b_i=b_k\mid\theta)\). Then

P(b_i=b_k\mid H_{it},\theta) = \frac{q_k}{\sum_{\ell=1}^{K}q_\ell}.

This is the same likelihood integration principle used in shared-parameter estimation, now applied to one subject’s history. In practice, prediction also has uncertainty in \(\theta\). The calculations below condition on a fitted \(\theta\); a fully Bayesian prediction averages over the joint posterior of \(b_i\) and \(\theta\).

Worked Example: Updating Latent-Effect Probabilities

Consider a hypothetical subject whose model uses three-point quadrature with latent-effect values \(b=-1,0,1\). After multiplying the prior mass at each point by the likelihood of the subject’s observed outcomes and continued follow-up, suppose the resulting unnormalized weights are \(0.032240,\ 0.267168,\ 0.182792\), in that order. These are likelihood-weight products, not probabilities yet.

Normalize. Their sum is \(0.032240+0.267168+0.182792=0.482200\). Dividing each weight by this total gives posterior probabilities approximately \(0.0669,\ 0.5541,\ 0.3791\). The rounded values sum to \(1.0001\) because each was rounded separately.

The outcome history and survival history have shifted probability toward \(b=0\) and \(b=1\), relative to what would be inferred from the prior alone. The posterior still assigns appreciable probability to all three values: the history has not identified the individual effect with certainty.

Predicting the next dropout interval

If the subject remains under follow-up at time \(t\), the one-interval-ahead dropout risk given the history is the conditional hazard averaged over the subject’s posterior distribution of \(b_i\). The key step is to average probabilities, not to insert the posterior mean effect into the hazard function.

P(D_{i,t+1}=1\mid H_{it},\theta) = \int h_{i,t+1}(b_i)\,p(b_i\mid H_{it},\theta)\,db_i.

Here \(D_{i,t+1}=1\) denotes dropout in the next interval, conditional on remaining at risk at its start. For discrete posterior support, this is a sum of posterior weights times the corresponding hazards.

Worked Example: One-Interval Posterior Predictive Risk

Use the posterior probabilities from the preceding example. Suppose the fitted hazard function is \(h(b)=\operatorname{expit}(-2+b)\), where \(\operatorname{expit}(x)=1/(1+\exp(-x))\). Thus the conditional hazards at \(b=-1,0,1\) are approximately \(0.0474,\ 0.1192,\ 0.2689\). The prediction conditions on the same history and the subject still being at risk.

The posterior predictive risk is

\begin{aligned} P(D_{i,t+1}=1\mid H_{it},\theta) &\approx 0.0669(0.0474)+0.5541(0.1192)+0.3791(0.2689)\\ &\approx 0.1712. \end{aligned}

The predicted probability of dropout in the next interval is therefore about \(17.1\%\), conditional on this observed history and the fitted model. It is not a statement that the subject has a \(17.1\%\) chance of belonging to a particular latent-effect category.

Prediction over more than one interval

For a fixed latent effect \(b_i\), the probability of surviving through the next \(m\) intervals is the product of the interval-specific survival probabilities, if the model specifies the hazards conditional on the same latent effect and the subject remains at risk. Consequently, the posterior predictive probability of dropout at least once over those intervals is

P(\text{dropout by }t+m\mid H_{it},\theta) = \int \left[ 1-\prod_{j=t+1}^{t+m}\{1-h_{ij}(b_i)\} \right]p(b_i\mid H_{it},\theta)\,db_i.

The latent effect is held fixed within each subject-specific product and uncertainty about it is averaged over afterward. Averaging the interval hazards first and then multiplying survival probabilities would generally describe a different calculation.

Worked Example: Two-Interval Dropout Risk

Continue with the previous hypothetical subject, and suppose the hazard at each of the next two intervals is constant over those intervals for a given \(b\). The two-interval dropout probabilities conditional on \(b=-1,0,1\) are, respectively,

\begin{aligned} 1-(1-0.0474)^2&\approx 0.0926,\\ 1-(1-0.1192)^2&\approx 0.2242,\\ 1-(1-0.2689)^2&\approx 0.4655. \end{aligned}

Average those risks over the posterior probabilities:

\begin{aligned} P(\text{dropout within two intervals}\mid H_{it},\theta) &\approx 0.0669(0.0926)+0.5541(0.2242)+0.3791(0.4655)\\ &\approx 0.3069. \end{aligned}

The predicted probability of dropout within the next two intervals is about \(30.7\%\). It exceeds the one-interval risk, as expected for the same subject and a longer horizon.

Why a point prediction can mislead

The posterior mean of \(b_i\) is a summary of the posterior, but generally it is not sufficient for calculating risk. In the first example, the posterior mean is approximately

(-1)(0.0669)+(0)(0.5541)+(1)(0.3791)\approx 0.3122.

Inserting this mean into the logistic hazard gives \(\operatorname{expit}(-2+0.3122)\), approximately \(0.156\), rather than the posterior-averaged risk \(0.1712\). The difference occurs because the hazard is nonlinear: in general, \(E\{h(b_i)\mid H_{it}\}\) is not equal to \(h\{E(b_i\mid H_{it})\}\).

A posterior average preserves the uncertainty that remains after observing the history. It is also a subject-specific prediction conditional on the fitted model and the stated follow-up history; it is not a causal effect or an unconditional population dropout rate.

Worked Example: Posterior Mean and Remaining Uncertainty

For a second hypothetical subject, suppose the three unnormalized weights for \(b=-1,0,1\), after incorporating the observed outcome and follow-up history, are \(0.021191,\ 0.214358,\ 0.179151\). Their sum is \(0.414700\). Normalization gives posterior probabilities approximately \(0.0511,\ 0.5169,\ 0.4320\).

The posterior mean effect is

E(b_i\mid H_{it},\theta) \approx (-1)(0.0511)+(0)(0.5169)+(1)(0.4320) =0.3809.

This mean is positive, but the posterior still assigns about \(5.11\%\) probability to a negative effect and \(51.7\%\) to zero. Reporting only \(0.3809\) would hide that uncertainty. For risk prediction, use all three posterior probabilities with their associated hazards; do not infer a hazard from the mean alone.

Diagnostics, alternatives, and limits

Posterior prediction is conditional on the joint model being adequate. Useful checks include whether the observed outcome history is plausible under the fitted outcome model, whether observed dropout patterns are reproduced, and whether predictions are sensitive to the latent-effect distribution and link function. Those model choices can matter substantially for posterior weights and hazards, especially for histories that are rare under the fitted model.

Inspect the posterior weights as well as the final risk. A prediction dominated by one quadrature point may be sensitive to that point, the assumed latent-effect distribution, or numerical integration accuracy. Compare quadrature resolutions or integration methods when feasible. Also check that the subject is at risk at the prediction origin and that the prediction horizon and hazard conditioning are stated correctly.

Approach	Calculation	Main limitation
Posterior averaging	Average each conditional hazard or cumulative risk over \(p(b_i\mid H_{it},\theta)\).	Depends on the fitted joint model and its posterior or quadrature approximation.
Plug-in latent-effect prediction	Evaluate the hazard at a point estimate, often the posterior mean of \(b_i\).	Can differ from posterior averaging when the hazard is nonlinear and hides latent-effect uncertainty.
Population-average prediction	Average over the population distribution of \(b_i\), without conditioning on this subject’s history.	Does not use the subject-specific information in the observed history.

A further limitation is parameter uncertainty. The displayed formulas condition on \(\theta\); treating estimated parameters as fixed can understate predictive uncertainty, particularly in small samples or near weakly identified model components. A Bayesian calculation integrates over \(p(b_i,\theta\mid\text{data})\). In a frequentist analysis, uncertainty can be assessed by refitting across bootstrap samples or another suitable procedure, while keeping the prediction target clear.

Common mistake

Common mistake: “The posterior mean latent effect is \(0.3122\), so the next-interval risk is just the hazard evaluated at \(0.3122\).” This replaces a distribution of plausible effects with one value before applying a nonlinear function. The correction is to calculate \(E\{h(b_i)\mid H_{it},\theta\}\), or for a longer horizon to average the complete conditional cumulative risk over that posterior distribution.

Another frequent error is to use outcome likelihood factors but omit observed continued follow-up from the history likelihood. Continued follow-up is evidence about dropout under the model and contributes survival factors. Conversely, after dropout, there are no later observed outcomes to include as if they had been recorded.

Check Your Understanding

Answer each question, then compare with the solutions.

A subject has posterior probabilities \(0.25\) and \(0.75\) for two latent-effect values, with corresponding next-interval hazards \(0.10\) and \(0.30\). What is the posterior predictive risk?
Why does an observed history of continued follow-up contribute to the posterior of the shared effect?
For two intervals with constant conditional hazard \(0.20\), what is the probability of dropout within the two intervals, conditional on that latent effect?
What is the main distinction between a subject-specific posterior prediction and a population-average prediction?

Solutions. (1) Average the hazards: \(0.25(0.10)+0.75(0.30)=0.25\), so the risk is \(25\%\). (2) Remaining under follow-up contributes survival likelihood factors \(1-h_{ij}(b_i)\), which vary with the latent effect and therefore update its posterior. (3) The two-interval risk is \(1-(1-0.20)^2=1-0.64=0.36\), or \(36\%\). (4) A subject-specific prediction averages over the posterior effect distribution conditional on that subject’s history; a population-average prediction averages over the population effect distribution without that history.