Targeted Minimum Loss Updates

Why Target an Initial Estimate?

When some outcomes are missing, a mean can be estimated using covariates to predict outcomes for everyone and observation probabilities to account for who was observed. In practice, these nuisance functions are estimated, often flexibly. A targeted minimum loss update (TMLE) takes an initial outcome-regression estimate and changes it in a deliberately restricted way: the change is chosen so that the resulting estimate solves an empirical influence-function estimating equation.

This is more than an arbitrary adjustment to predictions. The update is defined by a fluctuation model and a loss function, and its score equation matches the residual part of the influence function. The final estimate is a plug-in mean of the updated predictions. This tutorial develops that construction for a mean outcome with missingness, using both binary and continuous outcomes.

Identification and the Influence Function

Let $O=(W,R,RY)$ denote an observation, where $W$ is a vector of covariates, $Y$ is the outcome of interest, and $R=1$ indicates that $Y$ is observed. When $R=0$, the recorded outcome component $RY$ is zero by convention, but the value of $Y$ itself is unavailable. The target is the population mean $\psi_0=E[Y]$.

The usual identification conditions are consistency of the recorded outcome when $R=1$, missing at random given $W$, and positivity. Missing at random means $R$ and $Y$ are conditionally independent given $W$. Positivity means that the observation probability is positive for covariate values in the target population. Define $Q_0(W)=E[Y\mid R=1,W]$ and $g_0(W)=P(R=1\mid W)$. Under these conditions, $\psi_0=E[Q_0(W)]$.

Definition: For this missing-outcome mean, the efficient influence function at the true nuisance functions $Q_0$ and $g_0$, and target $\psi_0$, is $$ D(O;Q_0,g_0,\psi_0) = \frac{R}{g_0(W)}\{Y-Q_0(W)\}+Q_0(W)-\psi_0, \qquad \psi_0=E[Q_0(W)]. $$ For candidate nuisance functions $Q$ and $g$, the same formula with $\psi=E[Q(W)]$ is a plug-in influence-function expression, not generally the true efficient influence function. The empirical plug-in estimating equation sets its sample average to zero.

In an application, begin with estimates $\widehat Q(W)$ and $\widehat g(W)$, and define the clever covariate $H_{\widehat g}(W)=1/\widehat g(W)$. The empirical equation at updated outcome predictions $Q^*$ is

\mathbb{P}_n\left[ \frac{R}{\widehat g(W)}\{Y-Q^*(W)\} +Q^*(W)-\widehat\psi \right]=0, \qquad \widehat\psi=\mathbb{P}_n Q^*(W).

Here $\mathbb{P}_n$ denotes the sample average over all $n$ individuals. Because $\widehat\psi$ is defined as that average, the $Q^*(W)-\widehat\psi$ part sums to zero. Thus it is enough for the update to solve

\mathbb{P}_n\left[ \frac{R}{\widehat g(W)}\{Y-Q^*(W)\} \right]=0.

The update therefore targets the weighted outcome residual equation. This equation involves observed outcomes only, because the factor $R$ is zero when the outcome is missing.

Logistic Fluctuation for a Binary Outcome

For binary $Y$, a common fluctuation uses the Bernoulli log-likelihood and a logit link. The initial prediction must satisfy $0<\widehat Q(W)<1$ for every relevant covariate value; if a fitted prediction is on the boundary, clip it to the open interval before taking its logit. The estimated observation probability must satisfy $0<\widehat g(W)\leq 1$. Define

\operatorname{logit}\{\widehat Q_\epsilon(W)\} = \operatorname{logit}\{\widehat Q(W)\} +\epsilon H_{\widehat g}(W), \qquad H_{\widehat g}(W)=\frac{1}{\widehat g(W)}.

The fluctuation parameter $\epsilon$ is estimated by maximizing the Bernoulli likelihood among observed outcomes, treating the initial logit as an offset. Equivalently, minimize the negative Bernoulli log-likelihood. The resulting score equation is

\mathbb{P}_n\left[ R H_{\widehat g}(W)\{Y-\widehat Q_\epsilon(W)\} \right]=0.

This is precisely the residual equation needed above. The use of the logit scale also ensures that every updated prediction remains strictly between zero and one for a finite $\epsilon$.

Worked Example: Targeting a Binary-Outcome Mean With Missingness

Consider six hypothetical individuals. Outcomes are observed for the first four, with $Y=(0,1,1,0)$; the last two outcomes are missing. Suppose the estimated clever covariates for the four observed individuals are $H=(1,2,1,2)$, corresponding to estimated observation probabilities $(1,0.5,1,0.5)$. For the two individuals with missing outcomes, suppose $H=1.5$, or $\widehat g=2/3$. These probabilities are all positive. Assume missingness is conditionally independent of the outcome given the recorded covariates, as required for the identification argument.

Let the initial outcome predictions for the four observed individuals be $1/9,\ 3/11,\ 3/7,\ 1/5$, all strictly between zero and one. Use a logistic fluctuation and consider $\epsilon=\log(2)$. Since adding $\epsilon H$ multiplies the initial odds by $\exp\{\epsilon H\}=2^H$, the first updated prediction has odds $(1/8)\times 2=1/4$, giving $Q^*_1=1/5=0.2$. The second has initial odds $3/8$; multiplying by $4$ gives odds $3/2$, so $Q^*_2=3/5=0.6$. The third has initial odds $3/4$; multiplying by $2$ gives odds $3/2$, so $Q^*_3=0.6$. The fourth has initial odds $1/4$; multiplying by $4$ gives odds $1$, so $Q^*_4=0.5$.

Check the weighted residual score using the observed outcomes:

\sum_{i:R_i=1} H_i(Y_i-Q^*_i) = 1(0-0.2)+2(1-0.6)+1(1-0.6)+2(0-0.5) =-0.2+0.8+0.4-1=0.

Thus $\epsilon=\log(2)$ solves the empirical score equation for these values. For the two missing-outcome individuals, suppose the initial predictions are $0.3$ and $0.5$. Their updated predictions are

\operatorname{expit}\{\operatorname{logit}(0.3)+1.5\log(2)\} \approx 0.5480, \qquad \operatorname{expit}\{0+1.5\log(2)\} \approx 0.7388.

The targeted plug-in estimate averages all six updated predictions:

\widehat\psi = \frac{0.2+0.6+0.6+0.5+0.5480+0.7388}{6} \approx 0.5311.

The small rounding in the displayed mean comes from rounding the last two predictions. The weighted residual term in the empirical influence-function equation is zero, and the centered plug-in predictions sum to zero by the definition of $\widehat\psi$. Hence the empirical influence-function equation is solved, up to numerical and display rounding.

Why the Score Equation Matches the Influence Function

The link between the loss and the influence function follows by differentiating the fluctuation likelihood. For a binary outcome, the contribution to the negative log-likelihood for an observed individual is $-[Y\log\{\widehat Q_\epsilon(W)\}+(1-Y)\log\{1-\widehat Q_\epsilon(W)\}]$. Its derivative with respect to $\epsilon$ is $-H_{\widehat g}(W)\{Y-\widehat Q_\epsilon(W)\}$. Missing outcomes contribute no likelihood term, so multiplication by $R$ gives the sample score. At a likelihood minimum, the score is zero, yielding the weighted residual equation.

This justification also clarifies what the update does and does not assert. It makes a particular empirical equation hold for the fitted nuisance functions; it does not prove that $\widehat Q$ or $\widehat g$ equals its true counterpart. Nor does it make a positivity violation harmless: very small estimated observation probabilities create large values of $H_{\widehat g}$, which can make the score unstable.

Linear Fluctuation for a Continuous Outcome

For a continuous outcome, a simple fluctuation uses squared-error loss and an identity link: $ \widehat Q_\epsilon(W)=\widehat Q(W)+\epsilon H_{\widehat g}(W)$. Its score equation is again the weighted residual equation. When the denominator is nonzero, solving it gives

\widehat\epsilon = \frac{\sum_{i=1}^n R_i H_{\widehat g}(W_i)\{Y_i-\widehat Q(W_i)\}} {\sum_{i=1}^n R_i H_{\widehat g}(W_i)^2}.

This follows by substituting the fluctuation into the score, expanding, and collecting the terms multiplying $\epsilon$. A linear fluctuation is appropriate when it respects the outcome scale and any required bounds; it is not suitable for every outcome type.

Worked Example: A Linear Targeted Update

Suppose four hypothetical individuals have $R=(1,1,1,0)$, estimated observation probability $0.5$ for each, and initial outcome predictions $(4,6,8,5)$. Thus $H=2$ for everyone. For the three observed outcomes, suppose $Y=(5,7,10)$. These calculations use a continuous outcome, squared-error loss, and a linear fluctuation. Assume the missingness and positivity conditions stated earlier hold.

The observed residuals are $1,1,2$. Substituting into the formula for the fluctuation parameter gives

\widehat\epsilon = \frac{2(1)+2(1)+2(2)}{2^2+2^2+2^2} = \frac{8}{12} = \frac{2}{3}.

The updated predictions are $4+2(2/3)=16/3$, $6+2(2/3)=22/3$, and $8+2(2/3)=28/3$ for the observed individuals. The missing-outcome individual’s prediction becomes $5+2(2/3)=19/3$. The weighted residual sum is

2(5-16/3)+2(7-22/3)+2(10-28/3) = -\frac{2}{3}-\frac{2}{3}+\frac{4}{3}=0.

The plug-in estimate is therefore $\widehat\psi=(16/3+22/3+28/3+19/3)/4=85/12\approx 7.083$, in outcome units. The update has solved the empirical residual equation; the assumptions, not the calculation alone, are what connect the identified mean to the population target.

Targeted Updates and Their Limits

A targeted update and a one-step estimate both use influence-function information, but they construct the estimate differently. A one-step estimate adds an empirical influence-function correction to an initial estimate. A TMLE instead modifies a nuisance prediction through a specified fluctuation and then substitutes the updated predictions into the parameter definition. In the missing-outcome mean example, that definition is the average of $Q^*(W_i)$.

The loss-based construction can preserve useful features of the prediction scale: the logistic fluctuation keeps binary predictions in $(0,1)$, and the resulting estimate is an average of those predictions. But targeting is not a guarantee of good finite-sample behavior. Large inverse observation probabilities can make a few observed outcomes dominate the score, and a logistic score equation may fail to have a finite solution in separated or extreme data. In such cases, report the difficulty and reconsider the fitting choices and positivity assumptions rather than claiming the equation has been solved.

Common Mistakes

Taking the logit of a boundary prediction. The logit is undefined at zero and one. Require initial predictions to be strictly inside $(0,1)$, or clip them to that interval before forming the offset.
Using the wrong score weight. The missingness-mean residual equation uses $R/\widehat g(W)$, not an unweighted residual among observed cases. Omitting the inverse observation probability generally changes the equation.
Confusing a solved sample equation with correct nuisance models. A zero empirical score establishes a property of the fitted update in this sample; it does not show that either nuisance function is correctly specified.
Ignoring positivity. A zero or very small estimated observation probability makes the clever covariate undefined or extremely large. Positivity must be considered, not repaired by algebra alone.

Key takeaway: Choose a fluctuation whose loss-score equation is the weighted residual component of the influence function, solve that equation, and average the updated predictions. The score match is the defining targeting step; identification and regularity assumptions remain essential.

Check Your Understanding

Answer each question by identifying the relevant equation or assumption.

For a missing-outcome mean, what does the missing-at-random condition assert about $R$, $Y$, and $W$?
Why does setting $\widehat\psi=\mathbb{P}_n Q^*(W)$ reduce the empirical influence-function equation to the weighted residual equation?
Why must a logistic fluctuation’s initial predictions lie strictly between zero and one before taking logits?
In a linear fluctuation, what condition makes the denominator in the formula for $\widehat\epsilon$ positive?
Does solving the empirical targeting equation establish that the estimated observation probabilities are correct? Explain.

What You'll Learn