Deriving the TMLE Update from the Efficient Influence Function

Why the Update Has a Score Equation

For a binary-outcome average treatment effect, TMLE does not merely replace initial outcome predictions with a convenient alternative. It changes those predictions along a specified one-dimensional path, chosen so that the empirical efficient influence function has a zero residual component at the updated fit. This tutorial derives the equation governing that change. The equation is also the score equation for a Bernoulli likelihood fluctuation, which explains why a logistic update is natural.

Let $O=(W,A,Y)$ denote one observation, where $W$ is a vector of pretreatment covariates, $A\in\{0,1\}$ is treatment, and $Y\in\{0,1\}$ is the outcome. Write $g(W)=P(A=1\mid W)$ for the propensity score and $Q(a,W)=E(Y\mid A=a,W)$ for the conditional outcome mean. The target parameter is the population average treatment effect (ATE), $\psi=E\{Q(1,W)-Q(0,W)\}$.

Definition: Under consistency, conditional exchangeability $Y(a)\perp A\mid W$, and positivity $0<P(A=1\mid W)<1$ almost surely, the ATE is identified by $\psi=E\{Q(1,W)-Q(0,W)\}$. In the nonparametric model, its efficient influence function is $$ D^*(O)=H(A,W)\{Y-Q(A,W)\}+Q(1,W)-Q(0,W)-\psi, $$ where $H(A,W)=A/g(W)-(1-A)/\{1-g(W)\}$. The same form can be evaluated with estimated nuisance functions; denote the estimated clever covariate by $\widehat H$.

The causal assumptions identify the target but do not ensure that a finite-sample fluctuation equation has a solution. For the derivation, assume independent observations, estimated propensity scores strictly between zero and one, and initial outcome predictions strictly between zero and one. These interior predictions have finite logits, so a logistic fluctuation is well defined. We also need at least one nonzero clever covariate for the fluctuation to change an observed prediction.

From the Influence Function to the Fluctuation

Let $\widehat Q(a,W_i)$ and $\widehat g(W_i)$ be initial estimates for observation $i$. Define $\widehat H_i=A_i/\widehat g(W_i)-(1-A_i)/\{1-\widehat g(W_i)\}$. The one-parameter logistic fluctuation keeps the initial predictions as offsets and changes their logits in the direction $\widehat H$:

\operatorname{logit}\{\widehat Q_\epsilon(a,W_i)\} = \operatorname{logit}\{\widehat Q(a,W_i)\} +\epsilon\left\{\frac{a}{\widehat g(W_i)}-\frac{1-a}{1-\widehat g(W_i)}\right\}.

Here $\operatorname{logit}(q)=\log\{q/(1-q)\}$, and $\operatorname{expit}(x)=1/\{1+\exp(-x)\}$. For the observed treatment arm, the covariate in the fluctuation is exactly $\widehat H_i$, so $\widehat Q_\epsilon(A_i,W_i)=\operatorname{expit}[\operatorname{logit}\{\widehat Q(A_i,W_i)\}+\epsilon\widehat H_i]$. The same coefficient updates both counterfactual predictions: its direction is $1/\widehat g(W_i)$ for treatment and $-1/\{1-\widehat g(W_i)\}$ for control.

To derive the equation from the influence function, define the plug-in estimate at a candidate fluctuation value by $\widehat\psi_\epsilon=n^{-1}\sum_i\{\widehat Q_\epsilon(1,W_i)-\widehat Q_\epsilon(0,W_i)\}$. The empirical mean of the estimated efficient influence function at this candidate is

\mathbb P_n\widehat D^*_\epsilon = \frac{1}{n}\sum_{i=1}^n\widehat H_i\{Y_i-\widehat Q_\epsilon(A_i,W_i)\} +\frac{1}{n}\sum_{i=1}^n\{\widehat Q_\epsilon(1,W_i)-\widehat Q_\epsilon(0,W_i)\} -\widehat\psi_\epsilon.

The last two terms cancel by the definition of the plug-in estimate. Consequently, setting the empirical influence function to zero while using that plug-in estimate requires the fluctuation score equation

S_n(\epsilon) = \sum_{i=1}^n\widehat H_i\{Y_i-\widehat Q_\epsilon(A_i,W_i)\} =0.

This cancellation is important: if a candidate value for $\psi$ is not defined as the plug-in mean, the full empirical influence-function equation also contains the difference between that candidate and the plug-in mean. The equation above is the residual score condition used for this fluctuation; it is not, on its own, a proof that the causal assumptions hold.

There is a second derivation that identifies $S_n$ as a likelihood score. For the observed outcomes, the Bernoulli log-likelihood along the fluctuation is $\ell(\epsilon)=\sum_i[Y_i\log\{\widehat Q_\epsilon(A_i,W_i)\}+(1-Y_i)\log\{1-\widehat Q_\epsilon(A_i,W_i)\}]$. Differentiating a Bernoulli log-likelihood with respect to its linear predictor gives $Y_i-\widehat Q_\epsilon(A_i,W_i)$; the derivative of that predictor with respect to $\epsilon$ is $\widehat H_i$. Thus

\frac{d\ell(\epsilon)}{d\epsilon} = \sum_{i=1}^n\widehat H_i\{Y_i-\widehat Q_\epsilon(A_i,W_i)\} =S_n(\epsilon).

The likelihood interpretation makes the usual logistic-regression calculation available, but the fluctuation has a specific purpose: its covariate is fixed by the ATE influence function. It is not an arbitrary refit of the outcome model.

Worked Score Calculations

Worked Example: A Finite Root With Four Observations

Consider hypothetical data with all initial observed outcome predictions equal to $0.5$. The fitted propensities and outcomes are as follows; the clever covariate is computed from the listed treatment and propensity.

Observation	$A_i$	$\widehat g(W_i)$	$Y_i$	$\widehat H_i$
1	1	0.5	1	2
2	0	0.5	0	-2
3	1	0.8	0	1.25
4	0	0.2	1	-1.25

For example, observation 3 has $\widehat H_3=1/0.8=1.25$, while observation 4 has $\widehat H_4=-1/(1-0.2)=-1.25$. Since each initial prediction is $0.5$, its logit is zero. Substituting the four observations into the score gives

\begin{aligned} S_4(\epsilon) &=2\{1-\operatorname{expit}(2\epsilon)\} +2\{1-\operatorname{expit}(2\epsilon)\}\\ &\quad-1.25\operatorname{expit}(1.25\epsilon) -1.25\operatorname{expit}(1.25\epsilon)\\ &=4\{1-\operatorname{expit}(2\epsilon)\} -2.5\operatorname{expit}(1.25\epsilon). \end{aligned}

The second observation contributes $2\{1-\operatorname{expit}(2\epsilon)\}$, because its clever covariate is $-2$ and $\operatorname{expit}(-x)=1-\operatorname{expit}(x)$. The fourth contributes $-1.25\operatorname{expit}(1.25\epsilon)$, not a constant: its outcome is one and its updated observed prediction is $\operatorname{expit}(-1.25\epsilon)$.

Solving $S_4(\epsilon)=0$ numerically gives $\widehat\epsilon\approx0.2753$. At this value, $\operatorname{expit}(0.5506)\approx0.6343$ and $\operatorname{expit}(0.3441)\approx0.5852$. The two groups of score contributions are therefore $4(1-0.6343)\approx1.463$ and $-2.5(0.5852)\approx-1.463$, respectively. Their sum is approximately zero, as required. The reported root is rounded; using more digits in the numerical solution gives a more nearly zero score.

Worked Example: A Root at the Initial Fit

Suppose two hypothetical observations both have $\widehat g(W_i)=0.5$, $\widehat Q(A_i,W_i)=0.5$, and $Y_i=1$, with one treated and one untreated. Their clever covariates are $2$ and $-2$. Let $p=\operatorname{expit}(2\epsilon)$. The treated observation contributes $2(1-p)$. The control observation has updated observed prediction $\operatorname{expit}(-2\epsilon)=1-p$, and contributes $-2\{1-(1-p)\}=-2p$. Hence

S_2(\epsilon)=2(1-p)-2p=2-4p.

The score is zero when $p=0.5$, which implies $2\epsilon=0$ and $\widehat\epsilon=0$. The fluctuation leaves the initial predictions unchanged in this example. The derivative at the root is $S_2'(0)=-8(0.5)(0.5)=-2$, so the score crosses zero rather than merely approaching it.

Worked Example: Separation and No Finite Root

Now consider two hypothetical observations with propensity $0.5$, initial observed predictions $0.5$, and outcomes $Y_1=1$ for the treated observation and $Y_2=0$ for the untreated observation. Their clever covariates are $2$ and $-2$. Writing $p=\operatorname{expit}(2\epsilon)$, the first score contribution is $2(1-p)$, and the second is also $2(1-p)$. Therefore

S_2(\epsilon)=4\{1-\operatorname{expit}(2\epsilon)\}>0 \quad\text{for every finite }\epsilon.

The score tends to zero as $\epsilon$ tends to positive infinity, but it never equals zero at a finite coefficient. The fitted fluctuation has separation: increasing the coefficient keeps improving the Bernoulli likelihood by moving the observed predictions toward their outcomes. A numerical optimizer may return a very large coefficient or stop at a tolerance threshold; neither is an exact finite root.

Existence, Diagnostics, and Interpretation

The score is non-increasing in $\epsilon$. Differentiating each term, using $d\widehat Q_\epsilon(A_i,W_i)/d\epsilon=\widehat H_i\widehat Q_\epsilon(A_i,W_i)\{1-\widehat Q_\epsilon(A_i,W_i)\}$, gives

S_n'(\epsilon) = -\sum_{i=1}^n\widehat H_i^2 \widehat Q_\epsilon(A_i,W_i)\{1-\widehat Q_\epsilon(A_i,W_i)\} \leq 0.

When at least one $\widehat H_i\ne0$ and all fitted probabilities are strictly interior at finite $\epsilon$, the derivative is strictly negative. Thus there can be at most one finite root. Strict decrease alone does not guarantee a root: a finite solution requires the score to cross zero rather than only approach it at an infinite coefficient. The separation example exhibits this distinction.

In practice, inspect the fitted propensity scores, the range of the clever covariate, the starting and ending score values, and the optimizer's convergence status. Propensities near zero or one produce large clever covariates and can make a fluctuation unstable. A large estimated coefficient, updated predictions near zero or one, failure of the score to approach zero, or sensitivity to numerical tolerances are warning signs. A score tolerance should be assessed on the scale of the summed or averaged score actually used by the software.

Common mistake: Treating every contribution from a pair with opposite clever covariates as a constant, or claiming that a likelihood optimizer's reported coefficient must solve the score exactly. The residual $Y-\widehat Q_\epsilon$ changes with $\epsilon$, so each contribution must be evaluated at the updated prediction. Check the actual score after fitting; if it does not cross zero, report and investigate the convergence problem rather than asserting an exact targeting equation.

An ordinary plug-in estimator averages the initial predictions without this score-driven update. The fluctuation instead changes the outcome regression in a direction specified by the influence function, while preserving its basic prediction structure. This does not repair violations of consistency, exchangeability, or positivity, and the score being zero does not certify correct nuisance estimation. The earlier material on AIPW covers a different use of the same influence-function structure; here the focus is the derivation and numerical solution of the logistic fluctuation score.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

Why does setting the empirical efficient influence function to zero reduce to a residual score equation when the target estimate is the plug-in mean?
For an observation with $A=0$ and estimated propensity $0.25$, calculate its clever covariate.
Why can the fluctuation score have at most one finite root when the predictions are interior and at least one clever covariate is nonzero?
What does it mean if the score approaches zero only as $\epsilon$ tends to infinity?

Solutions. (1) The empirical influence function is the average residual term plus the plug-in mean contrast minus the target estimate. The latter two terms cancel when the target estimate is defined as that plug-in mean, leaving $n^{-1}\sum_i\widehat H_i\{Y_i-\widehat Q_\epsilon(A_i,W_i)\}=0$. (2) $\widehat H=-1/(1-0.25)=-1.333$ to four significant figures. (3) Its derivative is $-\sum_i\widehat H_i^2\widehat Q_\epsilon(1-\widehat Q_\epsilon)$, which is strictly negative under the stated conditions, so the score is strictly decreasing and crosses zero at most once. (4) There is no finite coefficient solving the equation; the likelihood may be separated, and a large numerical estimate should not be mistaken for an exact root.

Deriving the TMLE Update from the Efficient Influence Function

What You'll Learn