Deriving the Terminal-Step Targeting Update

Why the terminal update is a score equation

At the terminal step of longitudinal targeted maximum likelihood estimation, the outcome regression is modified so its residuals satisfy a weighted score equation. The weights are not arbitrary: they are the terminal clever covariate for the intervention regime. This tutorial derives that equation for a binary outcome and a logistic fluctuation, then works through the calculation and its main failure modes.

Use times $t=0,\ldots,K$, observed treatment $A_t$, covariate history $\mathcal H_t$, and a fixed deterministic regime $d_t(\mathcal H_t)$. Let $Y\in\{0,1\}$ be fully observed. Write $g_t(a\mid\mathcal H_t)$ for the treatment probability given the history available at time $t$. The initial terminal outcome regression, denoted $\hat Q_K$, predicts $Y$ from the terminal history and treatment. The derivation below assumes consistency, sequential exchangeability, positivity for the regime, and correctly defined histories; these are identification and data-structure conditions, not consequences of targeting.

Definition: For a deterministic regime and no censoring, the terminal clever covariate is $$ H_K(O)=\prod_{t=0}^{K}\frac{\mathbb{1}\{A_t=d_t(\mathcal H_t)\}}{g_t\{d_t(\mathcal H_t)\mid\mathcal H_t\}}. $$ It is zero for an observed treatment path that departs from the regime. For a path that follows the regime, it is the product of inverse treatment probabilities along that path. If censoring is present, the corresponding inverse censoring factors and at-risk indicators are included in the covariate and observed-data contribution.

Assume the initial predictions obey $0<\hat Q_K<1$, so their logits are finite. The one-dimensional logistic fluctuation adds $\epsilon H_K$ to the initial logit. Here $\epsilon$ is estimated from the observed data; it is not a separate coefficient for each history.

Definition: The terminal logistic fluctuation is $$ \operatorname{logit}\{\hat Q_K^\epsilon(O)\} =\operatorname{logit}\{\hat Q_K(O)\}+\epsilon H_K(O), \qquad \hat Q_K^\epsilon(O) =\operatorname{expit}\!\left[\operatorname{logit}\{\hat Q_K(O)\}+\epsilon H_K(O)\right], $$ where $\operatorname{expit}(x)=1/(1+\exp(-x))$. The terminal targeting estimate $\hat\epsilon$ solves the empirical Bernoulli score equation $$ \sum_{i=1}^{n}H_K(O_i)\{Y_i-\hat Q_K^{\hat\epsilon}(O_i)\}=0. $$

Deriving the update

Treat the initial logits as fixed offsets and fit the Bernoulli likelihood with the single covariate $H_K$. For observation $i$, the fluctuated mean is $\hat Q_{K,i}^\epsilon$. The log-likelihood, up to constants not depending on $\epsilon$, is

\ell(\epsilon)=\sum_{i=1}^{n}\left[ Y_i\log\{\hat Q_{K,i}^{\epsilon}\} +(1-Y_i)\log\{1-\hat Q_{K,i}^{\epsilon}\} \right].

Differentiating the logistic log-likelihood gives the score. Since the derivative of the linear predictor with respect to $\epsilon$ is $H_K(O_i)$, and the Bernoulli-logistic score with respect to its linear predictor is $Y_i-\hat Q_{K,i}^\epsilon$, the chain rule yields

\frac{\partial\ell(\epsilon)}{\partial\epsilon} =\sum_{i=1}^{n}H_K(O_i)\{Y_i-\hat Q_{K,i}^\epsilon(O_i)\}.

Thus maximizing this one-parameter likelihood gives exactly the displayed targeting equation, provided a finite maximizer exists. In computation, standard logistic-regression software can fit this model with the initial logit as an offset and $H_K$ as the sole covariate. No intercept is added: an intercept would create a different fluctuation and score equation.

Theorem: Suppose $H_K(O_i)\geq 0$ is finite, $0<\hat Q_{K,i}<1$, and at least one $H_K(O_i)>0$. The score $$ S(\epsilon)=\sum_{i=1}^{n}H_K(O_i)\{Y_i-\hat Q_{K,i}^\epsilon(O_i)\} $$ is continuous and nonincreasing. It is strictly decreasing for finite $\epsilon$ if at least one observation has $H_K(O_i)>0$. A unique finite root exists if and only if there is at least one weighted success and at least one weighted failure: $\sum_i H_K(O_i)Y_i>0$ and $\sum_i H_K(O_i)(1-Y_i)>0$.

Proof. For each observation, $$ \frac{d\hat Q_{K,i}^\epsilon}{d\epsilon} =H_K(O_i)\hat Q_{K,i}^\epsilon(1-\hat Q_{K,i}^\epsilon). $$ Therefore $$ S'(\epsilon) =-\sum_{i=1}^{n}H_K(O_i)^2 \hat Q_{K,i}^\epsilon(1-\hat Q_{K,i}^\epsilon)\leq 0. $$ The initial predictions are strictly between zero and one, so each fitted probability is strictly between zero and one at finite $\epsilon$. If any $H_K(O_i)>0$, its term makes the derivative strictly negative. The limits of the score are $$ \lim_{\epsilon\to-\infty}S(\epsilon)=\sum_i H_K(O_i)Y_i, \qquad \lim_{\epsilon\to+\infty}S(\epsilon)=-\sum_i H_K(O_i)(1-Y_i). $$ If both weighted totals are positive, continuity and these opposite signs give a root, and strict decrease gives uniqueness. If either total is zero, the score has one sign at every finite $\epsilon$, so no finite root exists. $\square$

Connection to the terminal residual equation

For a regime-specific mean, the terminal residual term in the empirical efficient influence function is proportional to $H_K(Y-\hat Q_K)$. The logistic fluctuation replaces $\hat Q_K$ with $\hat Q_K^{\hat\epsilon}$ and makes the sample mean of this terminal term zero. This is a direct consequence of the score derivation, not a claim that every term in the full empirical influence function is thereby zero: contributions from earlier times are handled by targeting at those times.

Result: If the terminal logistic update has a finite solution, then $$ \frac{1}{n}\sum_{i=1}^{n}H_K(O_i)\{Y_i-\hat Q_{K,i}^{\hat\epsilon}\}=0. $$ This holds because the left side is the fitted score divided by $n$. The result is exact for the empirical score equation, apart from numerical tolerance in the optimizer.

Worked Example: Solving the Score Equation Numerically

Suppose four terminal observations have initial predictions $\hat Q_{K,i}=0.20$, clever covariates $H_K=(1,1,2,2)$, and outcomes $Y=(1,0,1,1)$. Find the update and check its score. The initial logit is $\operatorname{logit}(0.20)=\log(0.2/0.8)=-1.3863$.

The score as a function of $\epsilon$ is $$ S(\epsilon)=1\{1-\operatorname{expit}(-1.3863+\epsilon)\} +1\{0-\operatorname{expit}(-1.3863+\epsilon)\} +2\{1-\operatorname{expit}(-1.3863+2\epsilon)\} +2\{1-\operatorname{expit}(-1.3863+2\epsilon)\}. $$ Combining terms gives $S(\epsilon)=5-2p_1(\epsilon)-4p_2(\epsilon)$, where $p_1(\epsilon)=\operatorname{expit}(-1.3863+\epsilon)$ and $p_2(\epsilon)=\operatorname{expit}(-1.3863+2\epsilon)$.

At $\epsilon=0$, both fitted probabilities are $0.20$, so $S(0)=5-2(0.20)-4(0.20)=3.80$. The score is positive, so the update must increase $\epsilon$. Solving $S(\epsilon)=0$ gives approximately $\hat\epsilon=1.975$. The updated predictions are $$ p_1(1.975)=\operatorname{expit}(0.5887)\approx0.643, \qquad p_2(1.975)=\operatorname{expit}(2.5637)\approx0.928. $$

Substitution checks the score: $$ S(1.975)\approx 5-2(0.643)-4(0.928) =5-1.286-3.712=0.002. $$ The small nonzero value is rounding error. Notice that the observations with $H_K=2$ receive twice the residual weight of those with $H_K=1$; the update is chosen to balance the weighted residuals, not the unweighted number of successes.

Worked Example: Updating a Single Prediction

An observation has initial prediction $\hat Q_K=0.30$, clever covariate $H_K=2$, and the fitted fluctuation parameter is $\hat\epsilon=0.40$. Its initial odds are $0.30/0.70=0.4286$. The updated logit adds $0.40(2)=0.80$, so the updated odds are $$ 0.4286\exp(0.80)\approx0.4286(2.2255)=0.9538. $$ Converting odds back to a probability gives $$ \hat Q_K^{\hat\epsilon} =\frac{0.9538}{1+0.9538}\approx0.488. $$

The prediction increases, as expected from the positive fluctuation and positive clever covariate. If $H_K=0$, the same formula leaves the initial prediction unchanged, regardless of $\hat\epsilon$. This is why observations whose treatment histories do not follow the regime do not contribute directly to this terminal score.

When the update fails or becomes unstable

Worked Example: No Finite Update Under Separation

Suppose every observation with $H_K>0$ has $Y=1$. Then for finite $\epsilon$, each corresponding fitted probability is less than one. Consequently, $$ S(\epsilon)=\sum_i H_K(O_i)\{1-\hat Q_{K,i}^{\epsilon}\}>0 $$ whenever at least one such observation has positive weight. The score approaches zero only as $\epsilon$ tends to $+\infty$; it never equals zero at a finite value. There is no finite logistic maximum-likelihood estimate for the fluctuation parameter.

For example, with one observation having $H_K=1$, $Y=1$, and $\hat Q_K=0.25$, the score is $1-\operatorname{expit}(\operatorname{logit}(0.25)+\epsilon)$, which is positive for every finite $\epsilon$. A software package may report a very large coefficient or fail to converge. That is evidence of separation, not evidence that an arbitrarily large update is scientifically appropriate.

Extreme but finite updates can also indicate practical positivity problems. If a treatment probability in the regime is tiny, its inverse contributes a large value to $H_K$. A few observations can then dominate the score, and the fitted predictions can move sharply. Inspect the distribution of the clever covariate, the range of fitted probabilities, the optimizer's convergence status, and the score evaluated at the reported $\hat\epsilon$. With censoring, also verify that the at-risk set and censoring factors used in the score match the observed-data definition.

The terminal logistic update is not the same as unweighted recalibration: it solves a clever-covariate-weighted residual equation. Nor is it simply inverse-probability weighting of the outcome; the targeting step modifies the initial outcome regression through a specified likelihood fluctuation. For a continuous outcome, the fluctuation and score differ; a logistic link is appropriate here because the outcome is binary and the updated predictions remain in $(0,1)$.

The update has limits. It cannot repair a structural positivity violation, recover information for histories never observed under the regime, or guarantee that the initial regression and treatment models are well estimated. A finite score root also does not by itself establish consistency or valid uncertainty quantification. In practice, report convergence and check the empirical score rather than relying only on the optimizer's success message.

Construct the terminal covariate.
Use the treatment and, if needed, censoring probabilities along the regime-consistent path.

Set the offset model.
Use $\operatorname{logit}(\hat Q_K)$ as the offset and $H_K$ as the single fluctuation covariate.

Fit and diagnose.
Estimate $\hat\epsilon$, inspect separation and extreme weights, and verify that the weighted residual score is numerically near zero.

Update predictions.
Apply the fitted fluctuation on the regime-consistent history for use in the subsequent recursion.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

Why does an observation with $H_K=0$ make no contribution to the terminal score?
For $\hat Q_K=0.50$, $H_K=1.5$, and $\hat\epsilon=-0.20$, calculate the updated prediction.
What conditions on the weighted successes and failures ensure a unique finite score root?
Why does a large fitted $\hat\epsilon$ warrant checking the clever-covariate distribution?

Solutions. (1) Its score contribution is $0\cdot(Y-\hat Q_K^\epsilon)=0$, and its logit offset receives no increment. (2) The initial logit is zero; the updated logit is $-0.20(1.5)=-0.30$, so the prediction is $\operatorname{expit}(-0.30)=1/(1+\exp(0.30))\approx0.426$. (3) Both $\sum_i H_K(O_i)Y_i$ and $\sum_i H_K(O_i)(1-Y_i)$ must be positive; then the continuous, strictly decreasing score changes sign. (4) Very large weights can allow a small number of observations to dominate the score and produce an extreme update, often reflecting weak practical positivity.

Deriving the Terminal-Step Targeting Update

What You'll Learn