Recursive Targeting at Earlier Time Points

Why targeting proceeds backward

In longitudinal TMLE, an update at an earlier time must use the outcome that the later stages now predict under the intervention. The later-stage update is therefore not an optional refinement: it defines the pseudo-outcome for the earlier conditional mean. Moving backward from the terminal outcome ensures that each conditional regression is targeted against the next targeted regression, rather than against a stale prediction.

Use decision times $t=0,\ldots,K$. Let $H_t$ denote the observed covariate history available just before treatment $A_t$, let $d_t(H_t)$ be the prescribed treatment at that time, and let $g_t(a\mid h)=P(A_t=a\mid H_t=h)$. Write $Q_t^0(H_t,A_t)$ for the initial estimate of the time-$t$ conditional mean. The terminal pseudo-outcome is $Z_K=Y$. For $t<K$, the pseudo-outcome is formed from the next targeted regression, evaluated at the prescribed next treatment.

Definition: At stage $t$, define the pseudo-outcome $Z_t$ by $Z_K=Y$ and, for $t<K$, $$ Z_t=Q_{t+1}^*\bigl(H_{t+1},d_{t+1}(H_{t+1})\bigr). $$ Here $Q_{t+1}^*$ is the regression after its targeting update. The stage-$t$ regression is an estimate of $E[Z_t\mid H_t,A_t]$. Its targeted version is denoted $Q_t^*$.

The update at time $t$ uses the cumulative clever covariate $$ C_t=\prod_{s=0}^{t}\frac{\mathbb{1}\{A_s=d_s(H_s)\}}{g_s\{d_s(H_s)\mid H_s\}}. $$ This covariate is zero when the observed treatment path fails to follow the regime by time $t$; on a matching path, it is the inverse probability of that path. If censoring is included, the corresponding observed-at-risk indicators and inverse conditional observation probabilities enter the product as specified by the censoring mechanism. Positivity is needed: the regime must have positive treatment probability, and positive observation probability where censoring is modeled, on histories relevant to the target.

The recursive targeting step

Suppose $Q_t^0$ is an initial conditional-mean estimate and $Z_t$ has been formed. For a binary outcome or a pseudo-outcome bounded in $[0,1]$, a logistic fluctuation is defined when the initial predictions lie strictly between zero and one:

\operatorname{logit} Q_t^{\epsilon_t}(H_t,A_t) = \operatorname{logit} Q_t^0(H_t,A_t)+\epsilon_t C_t.

The estimate $\hat\epsilon_t$ is obtained by maximizing the Bernoulli fluctuation likelihood, or equivalently solving its score equation. In a sample of size $n$, that equation is

\mathbb{P}_n\left[C_t\{Z_t-Q_t^{\epsilon_t}(H_t,A_t)\}\right]=0, \qquad \mathbb{P}_n f=\frac{1}{n}\sum_{i=1}^{n} f_i.

If an initial prediction is exactly zero or one, its logit is not finite. A prespecified truncation to $[\delta,1-\delta]$, for fixed $0<\delta<1/2$, makes this fluctuation defined; the truncation is part of the procedure and should be reported. It is not valid to apply the displayed logit formula unchanged to boundary predictions.

For a continuous outcome with an additive fluctuation, take $Q_t^{\epsilon_t}=Q_t^0+\epsilon_t C_t$. Under a normal working likelihood with constant variance, its score equation is the same residual equation. The bounded-outcome logistic fluctuation is usually preferable when it is important for updated predictions to remain in $[0,1]$.

Result: If the logistic score has a finite maximizer, the update satisfies $$ \mathbb{P}_n\left[C_t\{Z_t-Q_t^*(H_t,A_t)\}\right]=0, \qquad Q_t^*=Q_t^{\hat\epsilon_t}. $$ The score weights residuals by the inverse probability of following the regime, and includes indicators that the observed treatment path follows it.

To verify the result, the Bernoulli log-likelihood contribution at observation $i$, treating $Z_{t,i}\in[0,1]$ as a fractional Bernoulli response when needed, is $$ \ell_i(\epsilon_t)=Z_{t,i}\log Q_{t,i}^{\epsilon_t} +(1-Z_{t,i})\log(1-Q_{t,i}^{\epsilon_t}). $$ Differentiating the logit model gives $\partial Q_{t,i}^{\epsilon_t}/\partial\epsilon_t=C_{t,i}Q_{t,i}^{\epsilon_t}(1-Q_{t,i}^{\epsilon_t})$. Substitution into the derivative of $\ell_i$ yields $C_{t,i}(Z_{t,i}-Q_{t,i}^{\epsilon_t})$. Summing and setting the derivative to zero gives the stated equation. A finite solution requires that the likelihood not be completely separated in the fluctuation direction; if no finite maximizer exists, the update needs a prespecified remedy rather than an asserted finite score solution. $\square$

Worked Example: A Logistic Update at an Earlier Time

Consider an earlier decision $t=0$ in a two-decision study. The prescribed treatment is followed with probability $g_0=0.5$, so each matching observation has $C_0=1/0.5=2$. Suppose four such observations have initial prediction $Q_0^0=0.4$, with pseudo-outcomes $1,1,1,0$. Other observations, if any, have $C_0=0$ and contribute zero to this score.

Because the four matching observations have the same offset and covariate, the score equation is $$ 2\{3-4Q_0^{\epsilon_0}\}=0. $$ Thus $Q_0^{\epsilon_0}=3/4=0.75$. The initial logit is $\log(0.4/0.6)=\log(2/3)$; the updated logit must be $\log(0.75/0.25)=\log 3$. Since the covariate is 2, $$ \hat\epsilon_0=\frac{\log 3-\log(2/3)}{2} =\frac{\log(4.5)}{2}\approx 0.752. $$ Checking the score gives $2\{3-4(0.75)\}=0$. The pseudo-outcomes here are understood to have been formed from the already targeted next-stage regression; they are not necessarily the raw terminal outcomes.

Worked Example: Checking the Cumulative Clever Covariate

There are decisions at times 0 and 1. Suppose the regime prescribes $A_0=1$ and $A_1=0$. For one observation, $g_0(1\mid H_0)=0.5$ and $g_1(0\mid H_1)=0.25$. If both observed treatments match, then $$ C_0=\frac{1}{0.5}=2,\qquad C_1=\frac{1}{0.5}\frac{1}{0.25}=8. $$ The second-stage covariate is larger because it includes the inverse probability of matching at both decisions.

If instead $A_0=1$ but $A_1=1$, the path matches through time 0 but not time 1. Then $C_0=2$, while $C_1=0$, because the time-1 indicator is zero. These calculations check both features of the definition: the inverse probabilities appear in the denominator, and a nonmatching path is excluded by its indicator.

Why the order of updates matters

The backward sequence starts with the terminal regression, whose pseudo-outcome is $Y$. After targeting it, evaluate that targeted regression at the prescribed terminal treatment to construct the pseudo-outcome for the preceding decision. Target that earlier regression, evaluate it at its prescribed treatment, and continue toward time 0. Each time the next regression is updated, its predecessor’s pseudo-outcome must be constructed from the updated version.

Start at $K$.
Set $Z_K=Y$, update $Q_K^0$ using $C_K$, and obtain $Q_K^*$.

Move to $t=K-1$.
Set $Z_{K-1}=Q_K^*(H_K,d_K(H_K))$, then update $Q_{K-1}^0$ using $C_{K-1}$.

Repeat backward.
At each earlier time, use the next targeted regression evaluated under the regime, then solve that time’s fluctuation score.

This ordering follows directly from the pseudo-outcome definition. At time $t$, the score is computed using $Z_t$, which depends on $Q_{t+1}^*$, not on $Q_{t+1}^0$. Consequently, a change to $Q_{t+1}^*$ changes the target of the $t$-stage regression. Updating from early to late and failing to revisit the earlier stages would leave those stages targeted to obsolete pseudo-outcomes.

The score equation also explains why the clever covariate is inverse-probability weighted. Differentiating the fluctuation likelihood produces $C_t(Z_t-Q_t)$, not the residual multiplied by the probability of following the regime. This is the stage-specific score contribution; the cumulative product makes the contribution correspond to following the prescribed path through time $t$.

Worked Example: An Additive Update and Its Score Check

Suppose two observations have residuals $Z_i-Q_i^0$ equal to $0.2$ and $-0.1$, and clever covariates $C_i=2$ and $C_i=1$. For the additive fluctuation $Q_i^{\epsilon}=Q_i^0+\epsilon C_i$, the score equation is $$ \sum_{i=1}^{2} C_i\{Z_i-Q_i^0-\epsilon C_i\}=0. $$ Therefore $$ \hat\epsilon= \frac{\sum_i C_i(Z_i-Q_i^0)}{\sum_i C_i^2} =\frac{2(0.2)+1(-0.1)}{2^2+1^2} =\frac{0.3}{5}=0.06. $$ The fitted changes are $0.06(2)=0.12$ and $0.06(1)=0.06$. The updated residuals are $0.2-0.12=0.08$ and $-0.1-0.06=-0.16$; their score is $2(0.08)+1(-0.16)=0$, as required.

Assumptions, diagnostics, and alternatives

The recursion is an estimation procedure; interpreting its target as a regime-specific counterfactual mean still requires the identification assumptions established for longitudinal regimes, including consistency, sequential exchangeability, and positivity. For the targeting calculations, also require well-defined histories and regime rules, estimable conditional means, and treatment and observation probabilities that are positive on the histories used by the target. At each stage, the fluctuation likelihood or estimating equation must be well-defined. The logit version additionally requires initial predictions strictly inside the unit interval, or prespecified truncation.

Several diagnostics can reveal when the procedure is unstable or incorrectly implemented:

Extreme covariates: very large $C_t$ values indicate small treatment or observation probabilities and can make a few observations dominate the score.
Unsolved scores: report whether each fluctuation converged and check $\mathbb{P}_n[C_t(Z_t-Q_t^*)]$ numerically. A nonzero residual can indicate poor convergence or an incorrectly coded covariate.
Boundary predictions: inspect initial and updated predictions. A logit update cannot be applied to initial predictions of exactly zero or one without a specified boundary treatment.
Stale pseudo-outcomes: confirm that each $Z_t$ was recomputed from $Q_{t+1}^*$, not from the initial or an outdated next-stage regression.
Path indicators: check that nonmatching paths receive zero covariate at the first decision where they depart from the regime.

Inverse-probability weighting estimates a regime mean by weighting observed outcomes along matching paths. Recursive TMLE instead updates sequential outcome regressions, using inverse-probability clever covariates to target their fluctuations. The approaches are related through the role of the treatment probabilities, but they are not the same computation: recursive targeting retains the sequential regressions and substitutes targeted predictions into the backward recursion. This can be useful when outcome regressions provide stable information, but it does not remove positivity problems, misspecified nuisance models, or the need for appropriate identification assumptions.

A common mistake is to say that the earlier-stage residual is weighted by the probability of following the regime. That reverses the weighting: the clever covariate contains the inverse probability, multiplied by indicators that the observed treatment and, when modeled, observation paths follow the regime. Another mistake is to update all stages once while keeping earlier pseudo-outcomes fixed. The correction is to proceed backward and form each earlier pseudo-outcome only after the next regression has been targeted.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

If the probabilities of matching the regime at two decisions are 0.4 and 0.5, what is $C_1$ for a path matching at both decisions?
Why must $Z_t$ use $Q_{t+1}^*$ rather than $Q_{t+1}^0$?
For an additive fluctuation, what value of $\hat\epsilon$ solves the score if one observation has $C=3$ and residual $Z-Q^0=0.2$?
What must be done before using a logit fluctuation if an initial prediction equals zero?

Solutions. (1) $C_1=(1/0.4)(1/0.5)=2.5\times2=5$. (2) The earlier regression’s target is the next regression evaluated under the regime after that next regression has been updated; using $Q_{t+1}^0$ would target an obsolete pseudo-outcome. (3) The score is $3(0.2-\epsilon 3)=0$, so $0.6-9\epsilon=0$ and $\hat\epsilon=1/15$. (4) Use a prespecified boundary-handling procedure such as truncation to $[\delta,1-\delta]$; the logit of zero is not finite.

Recursive Targeting at Earlier Time Points

What You'll Learn