From the Regime to the Numerical Updates
A longitudinal TMLE calculation is easiest to audit when each object has one role: the intervention specifies the treatment actions, the treatment mechanisms specify their probabilities, and the outcome regressions are updated by separate fluctuations. This tutorial carries out those steps for two binary treatment decisions and a binary outcome. The example is deliberately small enough to check by hand, but it exhibits the same recursion used with richer histories.
Let \(W\) be baseline covariates, \(A_1\) the first treatment, \(L_1\) a covariate measured after \(A_1\), \(A_2\) the second treatment, and \(Y\in\{0,1\}\) the outcome. A fixed regime is \(d=(d_1,d_2)\), where each \(d_t\in\{0,1\}\). Use \(g_1(a_1\mid w)\) and \(g_2(a_2\mid w,a_1,l_1)\) for the treatment probabilities; \(g_t\) does not denote the regime action.
The target is identified under consistency, sequential exchangeability at both treatment decisions given the recorded past, and positivity: each regime action must have positive conditional probability on histories reached under that regime. For the formulas below, assume the observed data are independent and identically distributed, the outcome is binary, and there is no censoring. These are identification and calculation conditions, not guarantees that fitted models are correct or that finite-sample estimates are stable.
Write \(\hat Q_2,\hat Q_1\) for initial estimates and \(\hat g_1,\hat g_2\) for fitted treatment probabilities. The two cumulative clever covariates for regime \(d\) are
Each is zero off the corresponding observed regime path. On that path, it is the reciprocal of the estimated probability of following the required treatment history. For a binary outcome, the terminal targeting step fits a logistic fluctuation with offset \(\operatorname{logit}\hat Q_2\) and covariate \(H_2^d\). The earlier step uses an offset \(\operatorname{logit}\hat Q_1\), covariate \(H_1^d\), and pseudo-outcome \(\bar Q_2^*(A_1,L_1,W)=Q_2^*(A_1,L_1,d_2,W)\).
What the Fluctuation Equations Require
For the terminal step, the fluctuation model and its score equation are
Here \(\operatorname{expit}(x)=1/(1+\exp(-x))\), and the displayed score is the likelihood score for \(\epsilon_2^d\). Its form follows by differentiating the Bernoulli log-likelihood: the derivative of a logistic linear predictor with respect to \(\epsilon_2^d\) is \(H_{2i}^d\), so the score contribution is \(H_{2i}^d(Y_i-Q_{2i}^*)\). The earlier fluctuation solves the corresponding equation
In the second equation, the pseudo-outcome is \(\bar Q_{2i}^*\), not the observed \(Y_i\). The targeting update is therefore recursive: first update the terminal regression using \(Y\), then use its regime-specific predictions as the response for the earlier regression.
To verify the result, the updated prediction on those observations is \(q^*=\operatorname{expit}\{\operatorname{logit}(q)+\epsilon h\}\). Since \(H=h\) there and is zero off the path, the score equation reduces to \(h\sum_i(Y_i-q^*)=0\), where the sum is over the contributing observations. Because \(h>0\), \(q^*=p\). Applying the logit to this equality gives the stated value of \(\epsilon h\). When both response values occur, \(p\in(0,1)\), and this intercept-type score has a finite solution. \(\square\)
Worked Example: Two Regimes in a Small Data Set
Worked Example: Two Regimes in a Small Data Set
Consider 12 observations with no varying \(W\) or \(L_1\), so both are constant and the regressions reduce to treatment-cell predictions. There are three observations in each treatment cell. The treatment probabilities are \(g_1(0)=g_1(1)=0.5\) and \(g_2(0\mid A_1)=g_2(1\mid A_1)=0.5\). Initialize both outcome regressions at 0.5 on every history.
| \(A_1\) | \(A_2\) | Number of observations | Number with \(Y=1\) | Observed cell mean |
|---|---|---|---|---|
| 0 | 0 | 3 | 1 | 1/3 |
| 0 | 1 | 3 | 2 | 2/3 |
| 1 | 0 | 3 | 1 | 1/3 |
| 1 | 1 | 3 | 2 | 2/3 |
First target regime \(d=(1,1)\). On its three aligned observations, \(H_2^{(1,1)}=1/(0.5\times0.5)=4\); it is zero on the other nine. The initial logit is \(\operatorname{logit}(0.5)=0\). The terminal score requires the updated prediction on the aligned cell to equal its observed mean, \(2/3\), so
Substitution checks the update: the regime-path logit is \(0+4(0.173287)\approx0.693147=\log(2)\), and \(\operatorname{expit}\{\log(2)\}=2/3\). At the earlier step, the pseudo-outcome for observations with \(A_1=1\), evaluated at \(A_2=1\), is therefore \(2/3\). There are six such observations, and \(H_1^{(1,1)}=1/0.5=2\) for them. Hence
Evaluating the final earlier regression on the regime path gives \(\operatorname{expit}\{0+2(0.346574)\}\approx2/3\). With constant baseline history, averaging these predictions gives \(\hat\psi_{(1,1)}=2/3\).
Now target \(d=(0,0)\). The aligned cell has mean \(1/3\), and its terminal clever covariate is again 4. Thus
The earlier pseudo-outcome on \(A_1=0\), evaluated at \(A_2=0\), is \(1/3\). Since \(H_1^{(0,0)}=2\) on that first-treatment path, \(\epsilon_1^{(0,0)}=-\log(2)/2\approx-0.346574\), and the final prediction is \(\operatorname{expit}\{-\log(2)\}=1/3\). Therefore \(\hat\psi_{(0,0)}=1/3\), and the plug-in contrast is
This contrast is a difference between the two estimated regime-specific means. It is not obtained by subtracting the two fluctuation parameters: those parameters operate on logit scales and use different clever covariates.
Worked Example: A Non-Unit Clever Covariate
Worked Example: A Non-Unit Clever Covariate
Suppose three observations follow a particular regime path, the terminal initial predictions are all \(q=0.2\), and their outcomes are \(1,0,0\). For the histories in question, let the probability of the first required action be \(0.4\) and the second required action be \(0.25\). The terminal clever covariate on each aligned observation is \(h=1/(0.4\times0.25)=10\). The updated prediction is constant across these three observations in this example.
Their outcome mean is \(p=1/3\). The score equation is \(10\{(1-q^*)-q^*-q^*\}=0\), or \(10(1-3q^*)=0\). Thus \(q^*=1/3\). The initial logit is \(\operatorname{logit}(0.2)=\log(1/4)=-1.386294\), while the updated logit is \(\operatorname{logit}(1/3)=-\log(2)=-0.693147\). Consequently,
Checking on the regime path, \(-1.386294+10(0.069315)\approx-0.693144\), which gives a fitted probability of approximately \(1/3\); the small discrepancy is rounding. The factor of 10 is essential: using the logit difference itself as \(\epsilon_2\) would apply a ten-times-too-large shift on this path.
Worked Example: Spotting a Positivity Warning
Worked Example: Spotting a Positivity Warning
At one history, suppose the probabilities of the required first and second actions are \(0.02\) and \(0.10\). The cumulative terminal clever covariate on an aligned observation is \(1/(0.02\times0.10)=500\). If its outcome is 1 and its current prediction is 0.2, its terminal score contribution at \(\epsilon_2=0\) is \(500(1-0.2)=400\). With probabilities \(0.5\) and \(0.5\), the corresponding covariate would be 4 and the same residual would contribute \(4(0.8)=3.2\).
This comparison does not by itself determine the fitted update, which depends on all observations and their covariates. It does show why a rare treatment path can make targeting unstable: a small number of observations may have very large score contributions, and modest changes in the data or fitted probabilities can substantially change the update. Inspect the distributions of \(\hat g_1,\hat g_2,H_1^d,H_2^d\), the fitted probabilities, and the fluctuation estimates. Near-zero probabilities, extreme clever covariates, or fitted risks pushed close to 0 or 1 merit investigation; clipping probabilities changes the procedure and should be reported and justified.
How to Read the Result
The final plug-in estimate is the empirical average of the targeted earlier regression evaluated under the regime:
This averaging step is the reason the final answer is not simply a weighted average of observed outcomes. The terminal outcome regression is updated first; its predictions feed the earlier regression; and that targeted regression is evaluated for every observed baseline covariate value under \(d_1\). In the worked data, \(W\) is constant, so the average equals the single regime prediction.
The calculation also clarifies the division of labor among common approaches. The parametric g-computation approach obtains a regime mean by predicting outcomes along the regime and averaging. Inverse-probability weighting instead relies directly on treatment-path weights applied to outcomes. Longitudinal TMLE combines outcome-regression predictions with targeted fluctuations indexed by clever covariates. The worked example shows those updates, not a general guarantee of robustness or a substitute for checking assumptions. In particular, finite-sample positivity problems remain consequential, and a successful score equation does not establish that the regime effect is identified or that the nuisance models are adequate.
Check Your Understanding
Work through each item using the definitions and calculations above.
- If the required treatment probabilities are \(0.5\) and \(0.25\), what are the first-step and cumulative two-step clever covariates on an aligned path?
- With an initial prediction of 0.5, a constant terminal clever covariate of 4, and aligned outcome mean \(3/4\), calculate the terminal fluctuation parameter.
- In the small data example, why does the earlier fluctuation use a pseudo-outcome rather than the observed binary outcome?
- What diagnostic concern is raised by treatment probabilities of \(0.02\) and \(0.10\), and what is the resulting cumulative clever covariate?
Solutions. (1) The first-step covariate is \(1/0.5=2\); the cumulative covariate is \(1/(0.5\times0.25)=8\). (2) The updated path logit must be \(\operatorname{logit}(3/4)=\log(3)\); since the offset is zero, \(4\epsilon_2=\log(3)\), so \(\epsilon_2=\log(3)/4\approx0.274653\). (3) The terminal regression has already been targeted and must be integrated forward under the second regime action; its prediction is the response for the earlier regression. (4) The cumulative covariate is \(1/(0.02\times0.10)=500\), indicating a potentially unstable, weakly supported treatment path.