Cross-Fitting Sequential Nuisance Functions

Why Cross-Fit Longitudinal Nuisance Functions?

Longitudinal TMLE builds an intervention estimate from a sequence of conditional regressions and treatment probabilities. When those nuisance functions are machine learned, fitting and evaluating them on the same observations can make their predictions adapt to idiosyncrasies in those observations. Cross-fitting separates the data used to train a nuisance fit from the data on which its predictions enter the estimate. With longitudinal data, that separation must be implemented without breaking the order of histories, treatments, and outcomes.

The central bookkeeping rule is simple: each subject belongs to exactly one validation fold, and each initial nuisance fit and its preprocessing used to predict that subject exclude that subject’s fold. These cross-fitted predictions are distinct from any subsequent targeting update estimated using validation-fold data. The time order is just as important: a regression at time $t$ is trained using the appropriately constructed outcome from later times, and predictions must be evaluated at the right treatment history.

Definition: Let $O_1,\ldots,O_n$ be independent observations, each containing a baseline history $H_0$, treatments $A_t$, time-varying histories $H_t$, and an outcome $Y$. A $V$-fold partition is a set of disjoint index sets $I_1,\ldots,I_V$ whose union is $\{1,\ldots,n\}$. For fold $v$, $I_v$ is the validation set and $I_{-v}=\{1,\ldots,n\}\setminus I_v$ is the training set. A fold-specific nuisance fit is trained on $I_{-v}$ and used to predict observations in $I_v$.

For definiteness, suppose treatment is assigned at times $t=0,\ldots,K$, and $H_{it}$ is subject $i$’s history immediately before $A_{it}$. Let $d_t(H_{it})$ be the treatment specified by a deterministic regime at time $t$. Write $g_t(a\mid h)=P(A_t=a\mid H_t=h)$ for the treatment mechanism. In each training fold, estimate the $g_t$'s and the sequential outcome regressions. Their fold-specific versions are denoted $\widehat g_t^{\,v}$ and $\widehat Q_t^{\,v}$.

The longitudinal order is backward for outcome regression. Set the terminal regression equal to the outcome, $\widehat Q_{K+1}^{\,v}=Y$. At time $t$, the regression response is the later-time prediction evaluated under the regime at the next decision; for $t=K$, it is $Y$. The fitted conditional mean is then evaluated at the regime treatment $d_t(H_t)$. In short, later predictions are constructed first, and each earlier regression uses those predictions as its response. The precise history and response depend on the study’s observation scheme, but they must represent the same time ordering in every fold.

Fold-Specific Fitting and Prediction

A practical implementation begins by assigning each subject to one validation fold. For each $v$, train all required nuisance learners using only $I_{-v}$. This includes fitting treatment probabilities at every time and fitting the outcome regressions in backward temporal order. Any preprocessing that learns from the data—such as selecting variables, tuning hyperparameters, or estimating transformations—must also be performed using $I_{-v}$, then applied unchanged to $I_v$. Otherwise information from validation observations can enter the fitted prediction indirectly.

Next, generate predictions for every subject in $I_v$, using that fold’s fitted functions and that subject’s correctly ordered history. Store predictions with both the subject index and time index. Do not combine one subject’s prediction at time $t$ with another subject’s history, and do not use a fit from fold $u$ for a subject in fold $v$. These rules apply to all intermediate predictions used in the backward recursion, not just the final baseline prediction.

Assign folds.
Partition subjects into disjoint validation sets. Keep all records from a subject together; repeated person-time rows must not be split across folds.

Train within each complement.
For fold $v$, fit each treatment mechanism and each sequential outcome learner on $I_{-v}$, including training-only preprocessing and tuning.

Predict in temporal order.
For subjects in $I_v$, form the later-time predictions first, then the earlier-time predictions, maintaining their own time-indexed histories.

Target and store.
Apply the chosen fold-specific targeting procedure, evaluate each subject’s intervention prediction, and save one final prediction for each validation subject.

Assemble.
Combine the saved subject-level predictions over all validation folds, and average them to estimate the marginal intervention mean.

Worked Example: Two-Fold Prediction Without Leakage

Suppose eight subjects are divided into $I_1=\{1,2,3,4\}$ and $I_2=\{5,6,7,8\}$. For fold 1, the treatment and outcome learners are trained using subjects 5 through 8, and their predictions are made for subjects 1 through 4. For fold 2, training uses subjects 1 through 4 and prediction uses subjects 5 through 8.

Suppose the backward recursion gives subject 2 a regime prediction of $0.61$ from fold 1, and subject 6 a prediction of $0.48$ from fold 2. Both values are stored under their original subject indices. If the remaining predictions are $0.55,0.70,0.52,0.63,0.59,$ and $0.46$ for subjects 1, 3, 4, 5, 7, and 8, respectively, then the assembled estimate is

\widehat\psi =\frac{1}{8}(0.55+0.61+0.70+0.52+0.48+0.63+0.59+0.46) =\frac{4.54}{8} =0.5675.

Each subject contributes exactly one prediction, made by a learner that did not train on that subject. Averaging fold means without weighting would also give $0.5675$ here because both folds have four subjects. If fold sizes differed, the subject-level average—or equivalently the validation means weighted by their fold sizes—would be required.

Targeting Requires the Right Treatment-History Input

In a binary-outcome logistic fluctuation, a fold-specific update at time $t$ can be written using a cumulative clever covariate. To make its inputs explicit, let $\bar a_t=(a_0,\ldots,a_t)$ and let $\bar h_t$ denote the associated histories. Define

C_t^{\,v}(\bar a_t,\bar h_t) = \prod_{s=0}^{t} \frac{\mathbb{1}\{a_s=d_s(h_s)\}} {\widehat g_s^{\,v}(d_s(h_s)\mid h_s)}.

When fitting the fluctuation on observed validation data, the treatment arguments are the observed $A_{is}$. Thus the covariate in the fitting step is $C_t^{\,v}(\bar A_{it},\bar H_{it})$. For the usual sequential targeting recursion, evaluate the updated regression at the current regime treatment $d_t(H_{it})$, retaining the subject’s observed $H_{it}$ and observed past treatment path. The indicators for $s<t$ in the cumulative covariate therefore reflect the observed past treatments, while the current-time indicator is one. Do not replace past treatments with regime choices: a past departure from the regime makes the cumulative covariate zero.

For example, a logistic fluctuation can be expressed as

\operatorname{logit}\widetilde Q_t^{\,v} = \operatorname{logit}\widehat Q_t^{\,v} + \epsilon_t^v C_t^{\,v}(\bar a_t,\bar h_t).

The observed treatment arguments are used to estimate $\epsilon_t^v$ from validation observations. When the updated regression is evaluated for an intervention prediction, its covariate is reevaluated at the intervention treatment history. This distinction matters even at time 0: if $A_{i0}=0$, $d_0(H_{i0})=1$, and $\widehat g_0^{\,v}(1\mid H_{i0})=0.5$, then the observed-data covariate is $0$, while the intervention-input covariate is $1/0.5=2$. Using the observed-data value for the intervention prediction would incorrectly eliminate the update for this subject.

Worked Example: Evaluating the Fluctuation Under the Regime

At time 0, let the initial fitted probability at the intervention treatment be $\widehat Q_0^{\,v}(H_0,1)=0.40$, let the estimated probability of the regime treatment be $0.5$, and suppose the fitted fluctuation coefficient is $\epsilon_0^v=0.2$. For a subject observed with $A_0=0$ when the regime sets $d_0(H_0)=1$, the observed-data clever covariate is

\frac{\mathbb{1}\{A_0=d_0(H_0)\}} {\widehat g_0^{\,v}(d_0(H_0)\mid H_0)} = \frac{0}{0.5}=0.

This is the covariate value used for that subject’s observed-data fluctuation contribution. To predict under the regime, however, evaluate the update at $A_0=d_0(H_0)=1$. The covariate is then $1/0.5=2$, and the updated probability is

\operatorname{expit}\{\operatorname{logit}(0.40)+0.2(2)\} = \operatorname{expit}(-0.4055+0.4) \approx \operatorname{expit}(-0.0055) \approx 0.499.

The prediction is approximately $0.499$, not $0.40$. The latter would result from incorrectly carrying the observed treatment indicator into the intervention prediction.

At later times the same input distinction applies through $t$: evaluate the current treatment at $d_t(H_{it})$, while retaining the subject’s observed $H_{it}$ and observed past treatment path. The past indicators in the cumulative clever covariate reflect that observed path; they are not all set to the regime choices. When the regime depends on history, evaluate $d_t(H_{it})$ using the subject’s observed history at time $t$.

Assembling Subject-Level Targeted Estimates

After the fold-specific regressions have been targeted and evaluated through the backward recursion, let $\widetilde q_{0i}^{\,v}$ be subject $i$’s final baseline prediction when $i\in I_v$. For a marginal mean under the regime, the cross-fitted plug-in estimate is

\widehat\psi_{\mathrm{CF}} = \frac{1}{n}\sum_{v=1}^{V}\sum_{i\in I_v}\widetilde q_{0i}^{\,v}.

The label $v$ records which fold-specific nuisance fits produced the prediction; it does not mean that the subject contributes multiple predictions. For a contrast of two regimes, compute both predictions for each subject using the same fold assignment and form the subject-level contrast before averaging. Pairing the predictions this way preserves their shared subject and fold structure.

Result: If the validation sets partition all $n$ subjects, then the subject-level average equals the validation-fold means weighted by fold size. Writing $n_v=|I_v|$ and $\overline q_v=n_v^{-1}\sum_{i\in I_v}\widetilde q_{0i}^{\,v}$, $$ \widehat\psi_{\mathrm{CF}} = \sum_{v=1}^{V}\frac{n_v}{n}\overline q_v. $$

To verify the identity, substitute the definition of each fold mean:

\sum_{v=1}^{V}\frac{n_v}{n}\overline q_v = \sum_{v=1}^{V}\frac{n_v}{n} \left(\frac{1}{n_v}\sum_{i\in I_v}\widetilde q_{0i}^{\,v}\right) = \frac{1}{n}\sum_{v=1}^{V}\sum_{i\in I_v}\widetilde q_{0i}^{\,v}.

This also shows why an unweighted average of fold means is generally wrong when the fold sizes differ. It gives a smaller fold the same influence as a larger fold rather than weighting each subject equally. $\square$

Result: For independent observations and a fixed fold assignment, the fitted nuisance function, as a random function determined by training data, is independent of a validation observation conditional on the training data. Its evaluation at that observation’s history is generally not independent of the observation.

For a fixed fold $v$, the fitted nuisance function is a function of $\{O_j:j\in I_{-v}\}$. Under independent sampling, $O_i$ for $i\in I_v$ is independent of that collection. Thus, the function’s evaluation at an input fixed independently of $O_i$ is independent of $O_i$, conditional on the training data; evaluated at $O_i$’s own history, the prediction generally depends on $O_i$ through that input. This is the precise separation cross-fitting supplies. It does not say that a validation-fold targeting coefficient is independent of validation outcomes: if the fluctuation coefficient is estimated on that fold, it uses those outcomes. Nor does cross-fitting by itself establish consistency or asymptotic linearity. $\square$

Worked Example: Unequal Fold Sizes

Suppose a split has three subjects in fold 1 with targeted predictions $0.2,0.5,0.8$, and five subjects in fold 2 with predictions $0.4,0.6,0.5,0.7,0.3$. The fold means are

\overline q_1=\frac{0.2+0.5+0.8}{3}=0.5, \qquad \overline q_2=\frac{0.4+0.6+0.5+0.7+0.3}{5}=0.5.

The correctly assembled estimate is

\widehat\psi_{\mathrm{CF}} = \frac{3}{8}(0.5)+\frac{5}{8}(0.5)=0.5.

Here the unweighted fold average happens to agree because both fold means are equal. If instead the second fold mean were $0.7$, the subject-level estimate would be $3(0.5)/8+5(0.7)/8=0.625$, whereas the unweighted fold average would be $0.6$. The latter is not the average of the eight subject predictions.

Diagnostics, Alternatives, and Limitations

Cross-fitting is an implementation strategy, not a remedy for every modeling problem. Check that each subject appears in one and only one validation fold; that all records for a subject stay together; and that each prediction was generated from the complement of the subject’s fold. Audit the time indices, the treatment and history supplied to each learner, and the backward recursion’s response at every time. Inspect fold-specific treatment-probability predictions and regression predictions for missing values, implausible ranges, or abrupt fold-to-fold differences. Extreme inverse probabilities can signal instability even when the code is correctly cross-fitted.

A single training/validation split is simpler but uses only part of the sample to train and can make results sensitive to the split. Cross-fitting rotates the validation role so each observation is used for nuisance prediction, though each training fit still uses fewer than $n$ observations. Full-sample fitting followed by prediction on the same observations uses all data for training, but does not provide the same sample separation. These approaches are not interchangeable in their finite-sample behavior.

Cross-fitting also has limits. The fold assignment must respect the independent sampling unit; splitting person-time rows from one person across folds defeats the intended separation. Poor overlap, misspecified treatment mechanisms, or weak outcome regression can still produce unstable or biased estimates. Fold-specific targeting can be noisy in small validation sets, and fold-specific fluctuations may fail to converge in some samples. Cross-fitting should therefore be paired with diagnostics and a clearly specified targeting procedure, not treated as a guarantee of good performance.

Common Mistakes

Using an observed treatment indicator in a regime prediction. The observed indicator belongs in the validation-data fluctuation fit. Evaluate the clever covariate again with intervention-consistent treatment arguments when predicting under the regime.
Splitting repeated records rather than subjects. All observations from one subject must stay in the same fold when subjects are the independent units.
Predicting time points out of order. Construct the later-time outcome predictions first, then use them as responses for earlier regressions.
Averaging fold means equally when fold sizes differ. Average subject-level predictions or weight fold means by $n_v/n$.
Assuming cross-fitting fixes positivity or model failures. It separates nuisance fitting from validation prediction; it does not create treatment support or ensure accurate nuisance estimates.

Check Your Understanding

Answer each question, then check the solutions below.

For a subject in fold 3, which observations may be used to train the nuisance functions that predict that subject?
At time 0, the observed treatment disagrees with the regime. Which treatment input is used to fit the observed-data fluctuation, and which is used to evaluate the regime prediction?
Two validation folds have sizes 4 and 6, with fold means 0.3 and 0.7. What is the assembled subject-level estimate?
Why must all repeated records from one subject remain in one fold?

Solutions. (1) Use only observations outside fold 3, that is, $I_{-3}$, for training and any data-adaptive preprocessing. (2) Use the observed treatment for the observed-data fluctuation contribution; evaluate the covariate function at the regime treatment when predicting under the intervention. (3) Weight the fold means by their sizes: $(4/10)(0.3)+(6/10)(0.7)=0.12+0.42=0.54$. (4) If a subject’s records appeared in both training and validation sets, information from that subject could enter the fitted nuisance function used to predict the subject, undermining the intended sample separation.