Machine-Learned Nuisance Fits for Longitudinal TMLE

Why nuisance fits must respect time

Longitudinal TMLE uses several nuisance functions, each answering a different conditional prediction question. An outcome regression predicts a future outcome summary from a specified history; a treatment model predicts an action probability given the history before that action; a censoring model predicts continued observation given information available before censoring. Flexible machine learning can represent nonlinearities and interactions, but it cannot repair a function fitted to the wrong time point or the wrong risk set.

The aim here is not to select one universally best algorithm. It is to specify the prediction target precisely, fit it using information available at that point in the data process, and verify that its predictions have the right interpretation. The introductory material on longitudinal TMLE supplies the role of the sequential regressions and targeting step; this tutorial concentrates on estimating their nuisance inputs.

For clarity, consider a fixed regime \(d=(d_0,\ldots,d_K)\), where \(d_t(H_t)\) specifies the action at time \(t\). Write \(Q_{K+1}=Y\). In a simple ordering where the next covariate history is measured before the next decision, define the backward regressions by

Here \(H_{t+1}\) contains the history available at the next decision, including the intervening covariate information. A practical fit for \(Q_t\) regresses the fitted next-stage quantity on the current history among individuals with \(R_t=1\) who received the regime action and remained observed for the required next-stage information. The treatment and censoring functions are separately estimated on their corresponding at-risk rows. The exact timing of a censoring model must match when censoring occurs: if censoring follows measurement of \(L_{t+1}\), that measurement belongs in its conditioning history; if censoring precedes it, it does not.

What a machine-learned nuisance fit estimates

A learner is a procedure for estimating a conditional function, not a substitute for defining that function. For example, a random forest, gradient-boosted model, penalized regression, or ensemble may estimate \(g_t(1\mid H_t)\). Its output must be a conditional probability for treatment value 1 among the time-\(t\) risk set. A classifier score that is not a probability, or a prediction based on variables recorded after \(A_t\), does not meet that definition.

For a binary outcome regression, a learner may estimate \(E[Z_t\mid H_t]\), where \(Z_t\) is the outcome or the fitted next-stage pseudo-outcome. Squared-error regression targets the conditional mean directly. A binary likelihood model targets the conditional event probability when its response is binary. If the pseudo-outcome is a bounded number between 0 and 1, regression can target its conditional mean as well; the resulting prediction should be checked against the allowed range. A model family with a link function may help enforce that range, but the chosen loss and response must still match the prediction target.

Long format is often convenient: an individual contributes one row per time at which they are eligible for a particular fit, with time, the appropriate history, the response for that fit, and the risk-set indicator recorded explicitly. Separate time-specific learners are easy to audit. A pooled learner can share information across times, but should include time and allow time-specific relationships where substantively needed. Pooling rows does not make their histories or responses interchangeable.

Fit each conditional function on the correct rows

A treatment learner at time \(t\) uses rows with \(R_t=1\), response \(A_t\), and predictors contained in \(H_t\). Its prediction for a specified action is then evaluated at the corresponding history. A censoring learner uses rows at risk just before the censoring event, with a response indicating whether observation continues and predictors available at that event. These choices are part of the nuisance-function definition, not merely data-cleaning details.

Outcome regressions are usually fitted backwards. Start with the terminal outcome, then use its fitted conditional mean to form the preceding-stage response. At each stage, define the eligible rows and the regime action before fitting. If the regime is dynamic, its action is evaluated from each person's history. For a categorical action, the outcome regression is evaluated at the regime-specified action; for a continuous action, the prediction must be evaluated at the specified value using a model that supports such prediction.

Two facts behind flexible regression

The following results explain why regression learners are useful here and how their errors affect the sequential outcome fit. They concern conditional mean estimation and recursion; they do not guarantee that a particular algorithm will estimate the nuisance functions accurately from a finite sample.

Proof. Write \(Z-f(X)=(Z-m(X))+(m(X)-f(X))\), square, and take expectations. The cross term has expectation zero because \(E[(Z-m(X))(m(X)-f(X))]=E[E[Z-m(X)\mid X](m(X)-f(X))]=0\). The remaining terms give the stated identity. All terms are integrable by square-integrability, and \(E[Z-m(X)\mid X]=0\) by the definition of conditional expectation. Since the second term on the right is nonnegative, \(f=m\) minimizes the risk. \(\square\)

This identity justifies squared-error learning of conditional means, including conditional means of bounded pseudo-outcomes. It does not say that the most complex learner is best: estimation error can increase when a learner is too adaptive for the available data.

Proof. By definition, \(Q_t(x)=E[Q_{t+1}(H_{t+1})\mid X_t=x]\). Add and subtract \(m_t(x)\), and apply the triangle inequality:

The last inequality follows from the assumed uniform bound. Taking the supremum over \(x\) proves the claim. Repeated application gives a bound by the sum of the stagewise regression errors, \(\sum_{s=t}^{K}\delta_s\). The assumptions are deliberately strong and transparent: this is a deterministic error-propagation bound, not a finite-sample guarantee for any named machine-learning algorithm. \(\square\)

Diagnostics, alternatives, and limitations

Start with structural checks before interpreting fit statistics. At each time, tabulate the number of eligible rows, action counts, censoring outcomes, and missingness. Confirm that the history variables correspond to the correct time and that the response was constructed from the correct current or next-stage quantity. Inspect predictions by time and important history strata, including whether probabilities are finite and between 0 and 1 and whether outcome predictions respect the outcome range. Calibration summaries can reveal systematic prediction errors that a single overall accuracy score conceals.

A common mistake is to fit one model to all person-time rows and treat its output as a single generic prediction. The mistaken reasoning is that more rows always improve estimation, so time and risk-set distinctions can be ignored. The correction is that rows at different times may have different conditional targets and available histories. Pooling is reasonable only when the fitted specification represents those differences and the predictions are evaluated at the appropriate time-specific histories.

Another pitfall is to use a treatment score as though it were an outcome regression, or to predict treatment using post-treatment covariates. A treatment nuisance function is a probability conditional on pre-action history; an outcome nuisance function is a conditional mean for a specified future outcome target. They have different responses and different conditioning sets. Flexible prediction cannot make them interchangeable.

Conventional parametric regression offers transparent functional forms and can be stable in small samples, but may miss important nonlinearities or interactions. Machine-learning methods can capture richer patterns, but may be unstable with sparse time-specific risk sets and can produce poorly calibrated probabilities. A pooled model may borrow information across time, while separate models are easier to audit but can be noisy. These are trade-offs in nuisance estimation, not guarantees of causal validity.

No learner eliminates the identifying assumptions required for longitudinal causal interpretation, such as consistency, sequential exchangeability, and positivity. Nor does good average predictive performance guarantee accurate conditional predictions in every history stratum used by the regime. Severe lack of data for some histories can make fits unreliable; the diagnostics here can flag sparse support but do not solve it. Finally, this tutorial addresses how to construct the sequential nuisance predictions, not how to correct their empirical dependence through sample splitting or how nuisance estimation errors enter asymptotic product-rate conditions.