TMLE Variance Estimation and Confidence Intervals

Why TMLE Needs an Influence-Function Standard Error

A targeted maximum likelihood estimate (TMLE) gives a point estimate, but a point estimate alone does not show how much it may vary across repeated samples. For inference, we need an estimate of its sampling variability. Under suitable conditions, TMLE is asymptotically linear: its estimation error is approximately the sample average of estimated influence-function values. Their empirical variance therefore supplies a standard error and, with an additional normal approximation, a confidence interval.

This tutorial focuses on the average treatment effect (ATE) with a binary treatment. Let $O_i=(W_i,A_i,Y_i)$ denote the observation for individual $i$, where $W_i$ is a vector of baseline covariates, $A_i\in\{0,1\}$ is treatment, and $Y_i$ is an outcome. Write $Q(a,w)=E(Y\mid A=a,W=w)$, and let $g(a\mid w)=P(A=a\mid W=w)$. For binary treatment, $g(1\mid w)$ is the propensity score and $g(0\mid w)=1-g(1\mid w)$.

Definition: The ATE is $\psi=E\{Q(1,W)-Q(0,W)\}$. An efficient influence function for this parameter in the nonparametric model is $$ D(O)=\frac{A}{g(1\mid W)}\{Y-Q(1,W)\} -\frac{1-A}{g(0\mid W)}\{Y-Q(0,W)\} +Q(1,W)-Q(0,W)-\psi. $$ It has mean zero under the true data distribution. Its variance is the asymptotic variance of the influence-function contribution for one observation.

For an observed sample, let $\hat\psi$ be the TMLE and let $\hat Q^*$ and $\hat g$ be the targeted outcome regression and estimated treatment mechanism used by the estimator. The estimated influence-function value for observation $i$ is

\hat D_i= \frac{A_i}{\hat g(1\mid W_i)}\{Y_i-\hat Q^*(1,W_i)\} -\frac{1-A_i}{\hat g(0\mid W_i)}\{Y_i-\hat Q^*(0,W_i)\} +\hat Q^*(1,W_i)-\hat Q^*(0,W_i)-\hat\psi.

The sample variance of these values estimates the variance of the influence function. Using the usual sample-variance convention,

\widehat{\operatorname{Var}}(\hat D)= \frac{1}{n-1}\sum_{i=1}^{n}(\hat D_i-\bar{\hat D})^2, \qquad \widehat{SE}(\hat\psi)= \sqrt{\frac{\widehat{\operatorname{Var}}(\hat D)}{n}}, \qquad \bar{\hat D}=\frac{1}{n}\sum_{i=1}^{n}\hat D_i.

The division by $n$ in the standard error is essential: the sample variance estimates variability for one influence-function contribution, while the estimator averages $n$ contributions. The values $\hat D_i$ are in the same units as the estimand. For a binary outcome and an ATE reported as a risk difference, these units are probability differences.

Justification and Conditions for the Confidence Interval

The variance formula follows from the asymptotic linear representation. For an estimator satisfying the conditions below,

\hat\psi-\psi= \frac{1}{n}\sum_{i=1}^{n}D(O_i)+o_p(n^{-1/2}).

The independent, identically distributed influence-function values have mean zero and finite variance. The central limit theorem then gives an approximately normal sampling distribution for $\sqrt{n}(\hat\psi-\psi)$, with variance $\operatorname{Var}\{D(O)\}$. Replacing the unknown influence-function variance with the empirical variance of estimated values yields the standard error above. Thus an approximate two-sided 95% Wald confidence interval is

\hat\psi\ \pm\ 1.96\,\widehat{SE}(\hat\psi).

This is an asymptotic statement, not a guarantee that the interval has exactly 95% coverage in a finite sample. In repeated samples, under the assumptions and regularity conditions, the procedure is designed to cover the target parameter about 95% of the time.

For a causal interpretation of the ATE, the identification conditions include consistency of the observed outcome with the relevant potential outcome, conditional exchangeability of treatment given measured baseline covariates, and positivity: both treatment probabilities must be positive for covariate values in the target population. In practice, useful inference also requires that the estimated propensity scores not make the relevant inverse-probability weights unstable.

Identification is not enough for the normal approximation. A sufficient regularity framework includes independent observations, a finite influence-function variance, consistent nuisance estimates, and a negligible higher-order remainder. For example, with cross-fitting and suitable positivity and moment conditions, a common sufficient rate condition is that the product of the estimation errors for the outcome regression and treatment mechanism is $o_p(n^{-1/2})$. The fitted nuisance functions must also be stable enough that the estimated influence-function values approximate the true ones. Cross-fitting can help control empirical-process problems with flexible nuisance estimation, but it does not repair nonpositivity, confounding, or inadequate nuisance estimation. These conditions concern the estimator’s actual construction; use the influence-function values corresponding to that construction.

Worked Example: Correcting the Variance Calculation

Suppose a hypothetical sample has $n=100$, a TMLE estimate $\hat\psi=0.120$, and estimated influence-function values consisting of 20 values equal to $0.3$, 30 equal to $0.1$, 30 equal to $-0.1$, and 20 equal to $-0.3$. These are illustrative values, not data from a real study. Their mean is zero because the positive and negative contributions cancel.

First compute the sample variance. The sum of squared values is

$$ 20(0.3^2)+30(0.1^2)+30((-0.1)^2)+20((-0.3)^2) =1.8+0.3+0.3+1.8=4.2. $$

Since the mean is zero, the estimated influence-function variance is

\widehat{\operatorname{Var}}(\hat D)=\frac{4.2}{99}=0.04242.

The estimated standard error of the TMLE is therefore

\widehat{SE}(\hat\psi)=\sqrt{\frac{0.04242}{100}}=0.02060.

The approximate 95% confidence interval is

0.120\pm1.96(0.02060) =0.120\pm0.04037 =(0.0796,\ 0.1604).

If $\psi$ is a risk difference, the point estimate is a 0.120 difference in outcome risk, or 12.0 percentage points. The interval runs from approximately 0.0796 to 0.1604, or 7.96 to 16.04 percentage points. Its interpretation is about repeated-sample coverage under the stated conditions; it is not a 95% probability statement about this fixed parameter after observing the data.

Computing and Checking the Interval

The calculation is simple once the correct influence-function values are available. The main practical challenge is ensuring that those values match the estimator and that the assumptions supporting their variance are plausible.

Specify the target and estimator.
Record the estimand, such as the ATE, and obtain the TMLE and the targeted nuisance predictions used to construct it.

Calculate one influence-function value per observation.
Substitute each observation’s treatment, outcome, covariates, fitted propensity, targeted outcome predictions, and the same $\hat\psi$ into the stated formula.

Estimate variance and standard error.
Center the $\hat D_i$ values by their sample mean, calculate their sample variance, and divide that variance by $n$ before taking the square root.

Construct and assess the interval.
Use the normal quantile for the chosen confidence level, then inspect overlap, influence values, and the estimation procedure for warning signs that could invalidate the approximation.

In many TMLE implementations, the targeting step makes the empirical mean of the estimated influence function close to zero, sometimes exactly zero for the targeted estimating equation. A non-negligible $\bar{\hat D}$ can indicate a coding or implementation mismatch, such as using predictions from a different fit or centering with an inconsistent estimate. It is a useful check, not proof that the model, identification assumptions, or standard error are valid.

Worked Example: A Small Illustrative Influence-Function Sample

Suppose a hypothetical analysis with $n=8$ has $\hat\psi=0.250$, and its estimated influence-function values are $-0.2,-0.1,0,0,0.1,0.2,-0.1,0.1$. Their sum is zero, so $\bar{\hat D}=0$. The sum of squared values is

$$ 0.04+0.01+0+0+0.01+0.04+0.01+0.01=0.12. $$

Thus the sample variance and standard error are

\widehat{\operatorname{Var}}(\hat D)=\frac{0.12}{8-1}=0.01714, \qquad \widehat{SE}(\hat\psi)=\sqrt{\frac{0.01714}{8}}=0.04629.

The approximate 95% interval is

0.250\pm1.96(0.04629) =0.250\pm0.09073 =(0.1593,\ 0.3407).

The interval is centered at the estimate and describes uncertainty in the ATE on its original scale. With only eight observations, however, the normal approximation and nuisance-estimation conditions deserve particular scrutiny; the arithmetic alone cannot establish adequate coverage.

Diagnostics, Alternatives, and Limitations

The influence-function variance accounts for first-order sampling variation in the estimator, including variation represented through the outcome residuals and treatment mechanism. It is not the sample variance of $Y$, nor is it the standard error from an outcome-only regression that ignores how treatment was assigned. Substituting either of those quantities generally does not estimate the TMLE’s sampling variance.

Inspect the estimated propensity scores and the magnitudes of the inverse-probability factors $1/\hat g(1\mid W_i)$ and $1/\hat g(0\mid W_i)$. Scores near zero or one can produce large influence-function values and a large or unstable estimated standard error. Also inspect the distribution of $\hat D_i$: a small number of extreme values may dominate the variance. These patterns can signal weak overlap, model instability, or influential observations. Truncating propensity scores may reduce numerical instability, but it changes the fitted procedure and can introduce bias; it does not establish positivity or automatically validate the original interval.

The Wald interval is convenient and uses the estimated influence-function variance directly. Its limitations include reliance on a large-sample normal approximation and the possibility that an interval for a bounded parameter extends beyond its possible range. A bootstrap is an alternative way to assess sampling variability, but it must repeat the relevant estimation and targeting steps and does not automatically solve poor overlap or misspecification. Model-based standard errors may be easier to obtain, but they are appropriate only if their variance assumptions match the estimator. With clustered observations, an independent-observation variance calculation is not appropriate; the sampling structure must be reflected in the variance estimator.

Worked Example: Interpreting an Unusually Large Influence Value

In a hypothetical ATE analysis, suppose most estimated influence-function values lie between $-0.1$ and $0.1$, but one observation has $\hat D_i=1.2$. The value may arise because the individual has a large outcome residual, a large inverse-probability factor, or both. Its squared contribution to the variance numerator is $1.2^2=1.44$, much larger than the $0.01$ contributed by a value of magnitude $0.1$.

The correct response is not to delete the observation solely because it widens the interval. Check the treatment probability, outcome, covariates, and implementation; assess whether the observation is valid and whether overlap is adequate. If the value is valid, it is evidence that the estimate may be imprecise or sensitive to the observed sample. The ordinary influence-function interval is justified only if the regularity conditions, including suitable control of influence-function tails and nuisance-estimation error, remain plausible.

Common Mistakes

Forgetting the final division by $n$. The variance of the influence-function values is not the variance of the estimator; divide it by sample size before taking the square root.
Using $n$ rather than $n-1$ without stating the convention. The sample variance here is defined with denominator $n-1$. Other consistent asymptotic conventions may differ slightly in finite samples.
Using outcome variability as the TMLE variance. The relevant values are the estimator’s influence-function contributions, not the raw outcomes alone.
Treating a near-zero mean of $\hat D_i$ as proof of validity. It is an implementation diagnostic, not a test of exchangeability, positivity, or nuisance-estimation quality.
Reading the confidence level as a probability about the fixed parameter. The 95% refers to the long-run coverage of the procedure under its assumptions, not a 95% posterior probability that the parameter is in the observed interval.

Key takeaway: For regular, asymptotically linear TMLE, estimate the standard error as the square root of the sample variance of its estimated influence-function values divided by $n$. The resulting Wald interval is credible only when identification, overlap, nuisance estimation, and the large-sample approximation are defensible.

Check Your Understanding

Try each question, then compare your reasoning with the solutions.

If the sample variance of estimated influence-function values is $0.08$ in a sample of $200$, what is the estimated standard error?
Why is a large estimated influence-function value a diagnostic concern, and why should it not be removed automatically?
What does a 95% Wald confidence interval mean in repeated sampling?
Can a near-zero sample mean of the estimated influence-function values establish that there is no unmeasured confounding? Explain.

Solutions. (1) $\widehat{SE}=\sqrt{0.08/200}=\sqrt{0.0004}=0.0200$. (2) Its square can contribute substantially to the estimated variance, potentially reflecting weak overlap or an influential residual; first investigate its cause, because a valid observation should not be discarded merely to narrow the interval. (3) Under the assumptions and regularity conditions, intervals formed by this procedure cover the fixed target parameter in about 95% of repeated samples. (4) No. A near-zero mean can help check implementation of the estimating equation, but cannot establish exchangeability or rule out unmeasured confounding.