Worked Example with Three Decisions and Dropout

Why dropout changes the calculation

With three treatment decisions, a participant can depart from the regime of interest at any of three points. They can also become unobserved before the outcome is recorded. Longitudinal targeted maximum likelihood estimation (TMLE) accounts for both mechanisms: treatment probabilities describe how likely the observed treatment history is under the regime, and retention probabilities describe how likely the outcome remains observable. Their cumulative products enter the clever covariates used in targeting.

The calculation below concerns a fixed regime $a=(a_1,a_2,a_3)$. Use $H_t$ for the measured history available before treatment $A_t$, including baseline covariates and any time-varying covariates measured up to that point. Let $R_0=1$, and let $R_t=1$ indicate that the participant remains observed after decision $t$. Monotone dropout means $R_t=0$ implies $R_s=0$ for every $s>t$. The terminal outcome $Y$ is observed only when $R_3=1$.

Definition: For $t=1,2,3$, define the treatment probability under the observed-data law as $g_t(a_t\mid H_t)=P(A_t=a_t\mid H_t,R_{t-1}=1)$. Define the retention probability as $q_t(H_t,A_t)=P(R_t=1\mid H_t,A_t,R_{t-1}=1)$. The regime-specific mean is $\psi=E[Y^{a}]$, the mean outcome if everyone followed regime $a$, under the identification assumptions stated below.

Identification requires consistency, sequential exchangeability of treatment given the measured histories, and conditional independent censoring: among those still observed, retention is independent of the relevant future counterfactual outcomes given the measured history and treatment. Positivity is also needed: treatment and retention probabilities along the regime must be positive for histories with positive probability under that regime. These conditions make the regime mean identifiable; they do not ensure that fitted nuisance models will be accurate.

How the dropout-adjusted clever covariate is formed

A record contributes information about the terminal outcome under $a$ only if it follows the specified treatment path and remains observed. For someone with that path through time $t$, the conditional probability of the required treatment and retention events is

\prod_{s=1}^{t} g_s(a_s\mid H_s)\,q_s(H_s,a_s).

This follows by multiplying successive conditional probabilities: at each decision, given the history and retention to date, the next required treatment has probability $g_s$, and subsequent retention has probability $q_s$. The inverse of this product corrects for the fact that such complete, regime-consistent histories are selectively observed. Including the indicators for the required events gives the cumulative clever covariate

H_t^* = \begin{cases} \displaystyle \frac{ \prod_{s=1}^{t} I(A_s=a_s)R_s }{ \prod_{s=1}^{t} g_s(a_s\mid H_s)\,q_s(H_s,A_s) }, & R_t=1,\\[1em] 0, & R_t=0, \end{cases} \qquad t=1,2,3.

For records observed through time $t$, the numerator is zero as soon as treatment departs from the regime; for records with $R_t=0$, $H_t^*$ is defined to be zero without evaluating the denominator. In a targeting regression evaluated at the intervention history, the treatment and retention indicators are set to one; the corresponding prediction therefore uses the inverse-probability factor without observed-data indicators. The indicator form is used for observed records in the score equation.

Worked Example: Calculating the Three Cumulative Weights

Consider regime $a=(1,0,1)$. For one participant, suppose the relevant fitted probabilities are $g_1=.40,\ q_1=.90,\ g_2=.75,\ q_2=.80,\ g_3=.50,\ q_3=.50$. The participant follows the regime through decision 2 and remains observed through $R_2=1$, but drops out before the terminal outcome, so $R_3=0$.

Through decision 1, the participant matches the regime and is retained. Thus $H_1^*=1/(.40\times .90)=1/.36\approx2.778$.

Through decision 2, the cumulative denominator is $.40\times.90\times.75\times.80=.216$. Therefore $H_2^*=1/.216\approx4.630$.

At decision 3, the participant is not observed, so by definition $H_3^*=0$, without evaluating the denominator. The reciprocal $1/(.40\times.90\times.75\times.80\times.50\times.50)$, approximately $18.519$, is the inverse probability factor for a complete regime-consistent record, but it cannot supply an unobserved outcome.

If instead the participant had been retained through decision 3 and had followed the full regime, then the denominator would be $.054$, giving $H_3^*=1/.054\approx18.519$. This calculation shows why even moderate retention probabilities can produce large cumulative weights.

Targeting each regression and computing the plug-in mean

Write $Q_4(H_3,A_3)=E[Y\mid H_3,A_3,R_3=1]$ for the terminal outcome regression. For earlier steps, define $Q_t$ by backward recursion: after setting the treatment at decision $t$ to $a_t$, average the next regression over the covariate distribution under the regime. In particular, $Q_3$ averages $Q_4$ over the next history, $Q_2$ averages $Q_3$, and $Q_1$ averages $Q_2$. The final plug-in estimate is the sample mean of $Q_1$ evaluated at each participant’s baseline history and regime treatment.

At each step, a fluctuation updates the current regression using the corresponding cumulative clever covariate. For a continuous outcome and an identity-link squared-error fluctuation, the update has the form $Q_t^\epsilon=Q_t+\epsilon H_t^*$. Its empirical score equation sets the sum of clever covariate times residual to zero. Other outcome types or links require a fluctuation compatible with the outcome scale.

Derivation: At a given targeting step, let $Z_i$ be the current regression target and $Q_i$ its initial prediction for an eligible observed record. The empirical squared-error loss for the identity fluctuation is proportional to $\sum_i (Z_i-Q_i-\epsilon H_i^*)^2$. Differentiating with respect to $\epsilon$ gives $-2\sum_i H_i^*(Z_i-Q_i-\epsilon H_i^*)$. Setting this derivative to zero yields $$ \hat\epsilon = \frac{\sum_i H_i^*(Z_i-Q_i)} {\sum_i (H_i^*)^2}, $$ provided the denominator is positive. Substituting this value makes the empirical score $\sum_i H_i^*(Z_i-Q_i-\hat\epsilon H_i^*)$ exactly zero. This derives the update for this particular loss and fluctuation; it is not a universal formula for every link or outcome type.

Worked Example: Solving a Terminal Identity Fluctuation

Suppose three complete, regime-consistent records have terminal outcomes $Z=(0,1,1)$, initial predictions $Q=(.5,.5,.5)$, and terminal clever covariates $H^*=(2,4,1)$. For this numerical calculation, records outside the regime path contribute zero to the score.

The score numerator is $2(0-.5)+4(1-.5)+1(1-.5)=-1+2+.5=1.5$. The denominator is $2^2+4^2+1^2=21$, so $\hat\epsilon=1.5/21=1/14$.

The updated predictions are $.5+2/14=9/14\approx.643$, $.5+4/14=11/14\approx.786$, and $.5+1/14=4/7\approx.571$. Checking the score after updating gives $2(0-9/14)+4(1-11/14)+(1-4/7)=-9/7+6/7+3/7=0$. All three predictions remain between zero and one in this example. That check matters: an identity fluctuation does not generally guarantee bounded predictions.

Worked Example: Backward Recursion and the Regime Mean

Suppose the targeted terminal regression, evaluated at $A_3=a_3$, is $Q_4^*(L_3)=.20$ when $L_3=0$ and $.80$ when $L_3=1$. Under the regime, suppose $P(L_3=1\mid L_2=0)=.25$ and $P(L_3=1\mid L_2=1)=.75$. The next regression is therefore $Q_3^*(L_2=0)=.75(.20)+.25(.80)=.35$, and $Q_3^*(L_2=1)=.25(.20)+.75(.80)=.65$.

Now suppose $P(L_2=1\mid L_1=0)=.20$ and $P(L_2=1\mid L_1=1)=.80$. Averaging $Q_3^*$ gives $Q_2^*(L_1=0)=.80(.35)+.20(.65)=.41$, and $Q_2^*(L_1=1)=.20(.35)+.80(.65)=.59$.

Finally, let baseline $W$ have two equally common values, $0$ and $1$. Suppose under the regime $P(L_1=1\mid W=0)=.25$, while $P(L_1=1\mid W=1)=.75$. Then $Q_1^*(W=0)=.75(.41)+.25(.59)=.455$, and $Q_1^*(W=1)=.25(.41)+.75(.59)=.545$. The targeted regime-specific mean is $\hat\psi=(.455+.545)/2=.500$. Each average uses the covariate distribution under the regime, not the observed distribution among participants who happened to receive the regime.

Why this construction matters—and how it can fail

The recursion and the targeting updates have distinct jobs. Backward regression builds the outcome prediction under the specified regime; the clever covariates incorporate how rarely the observed data reveal that regime and an outcome. The final plug-in mean is formed from the recursively updated regression, rather than by simply averaging observed outcomes among people who happened to follow the regime.

Compared with pure inverse-probability weighting, longitudinal TMLE also uses outcome regressions and returns a plug-in estimate on the outcome scale when the fluctuation respects that scale. Compared with the parametric g-formula, it uses targeted updates informed by the treatment and censoring mechanisms. These methods differ in construction and can behave differently in finite samples; none removes the need for identification assumptions, adequate nuisance estimation, and positivity. The formal robustness conditions for combinations of nuisance-model correctness are a separate question.

Useful diagnostics include inspecting the distribution of cumulative clever covariates, the smallest fitted treatment and retention probabilities along the regime, and whether predictions remain in the outcome range. A few very large weights signal practical positivity problems even when every estimated probability is technically positive. Check that dropout is coded monotonically, that a missing terminal outcome is not treated as an observed zero, and that the time ordering of history, treatment, and retention matches the data-generating process. Truncating extreme weights can reduce variance but changes the estimating procedure and can introduce bias; it is not a repair for absent support.

The calculations also depend on the nuisance fits and their evaluation at the intervention histories. Incorrect temporal ordering, omitted predictors of dropout, or misspecified regression structure can distort the result. The worked identity update is intentionally transparent, but bounded outcomes may need a link and fluctuation that preserve their range. In every case, check both the empirical targeting score and the final recursion rather than assuming that a successful software fit guarantees a sensible estimate.

Set the regime and histories.
Write down the treatment value required at each decision and what is known before that decision.

Fit treatment and retention probabilities.
Use histories available at each decision, and check for near-zero probabilities along the regime.

Build cumulative clever covariates.
Multiply the required treatment and retention indicators in the numerator and their conditional probabilities in the denominator.

Target and recurse backward.
Update the regressions with the appropriate clever covariates, integrate over next-history distributions under the regime, and average the baseline predictions.

Check Your Understanding

Work through the questions, then check the short solutions below.

Under regime $(1,0,1)$, what is the terminal clever covariate if treatment matches through time 3 but dropout occurs after time 2?
Why does the complete-path inverse probability include both $g_s$ and $q_s$ at each time?
For $H^*=(1,3)$, $Z=(0,1)$, and initial $Q=(.5,.5)$, calculate the identity-fluctuation update.
Why is the final plug-in mean not generally the average outcome among observed participants who followed the regime?
Name two diagnostics that can reveal a problem with dropout-adjusted targeting.

Solutions. (1) It is zero because the terminal retention indicator $R_3$ is zero. (2) Each required treatment and subsequent retention event must occur, so their conditional probabilities multiply along the path. (3) The numerator is $1(-.5)+3(.5)=1$; the denominator is $1+9=10$, so $\hat\epsilon=.1$. The updated predictions are $.6$ and $.8$, and the score is $1(0-.6)+3(1-.8)=0$. (4) That average conditions on observed treatment and retention, whereas the target is a mean under the intervention and uses the recursively evaluated outcome regression. (5) Examples include extreme cumulative clever covariates, very small fitted retention probabilities, predictions outside the outcome range, and violations of monotone dropout coding.

Worked Example with Three Decisions and Dropout

What You'll Learn