Tutorials › Marginal Structural Models › Multiple Regime TMLE with Shared Nuisance Fits

Home  /  Tutorials  /  Longitudinal Targeted Maximum Likelihood

Longitudinal Targeted Maximum Likelihood · Tutorial 400 of 501

Multiple Regime TMLE with Shared Nuisance Fits

Use shared outcome and treatment fits to target multiple longitudinal regimes separately, then organize their estimates and contrasts without losing their joint sampling dependence.

Advanced 12 min read

What You'll Learn

  • Define the regime-specific mean and the assumptions that identify it from observed longitudinal data.
  • Construct the cumulative clever covariate separately for each treatment regime.
  • Describe a backward targeting workflow that reuses nuisance fits but keeps regime-specific updates distinct.
  • Verify a one-time-point targeting update and compute regime estimates and a contrast.
  • Explain why a contrast’s uncertainty must account for the dependence between estimates.
  • Diagnose practical problems such as poor overlap, separation, and incorrect reuse of targeting covariates.

Why Estimate Several Regimes Together?

A study may ask for the mean outcome under each of several treatment strategies: for example, sustained treatment versus no treatment, or two different rules that adapt treatment to a patient’s evolving history. Estimating these means together is useful because the scientific question is often a contrast between them. The nuisance models that describe outcomes and treatment assignment can be shared, while the targeting step must still reflect which strategy is being evaluated.

The central distinction is therefore between shared nuisance fits and regime-specific targeting. Reusing fitted nuisance functions can save computation and keep the regimes on a common modeling basis. It does not make the regime-specific clever covariates, fluctuations, or influence functions interchangeable.

Targets, Identification, and Regime-Specific Covariates

Suppose treatment is assigned at times \(t=0,\ldots,K-1\). Let \(H_t=(L_0,A_0,\ldots,A_{t-1},L_t)\) denote the observed history just before treatment \(A_t\), and let \(Y\) be the final outcome. A deterministic regime \(d=(d_0,\ldots,d_{K-1})\) assigns treatment \(d_t(H_t)\) at time \(t\). Write \(Y^d\) for the outcome that would be observed if that regime were followed, and define its mean by \(\psi_d=E[Y^d]\).

Identification from observed data requires consistency, no interference between individuals, sequential exchangeability of treatment given the measured history, and positivity: for each regime action, the conditional probability of receiving that action given the relevant history must be positive. Under these assumptions, the mean is identified by the longitudinal g-formula. In practice, the treatment probabilities and outcome regressions are estimated; this tutorial focuses on using their shared fits to target several means.

Definition: Let \(g_t(a\mid H_t)=P(A_t=a\mid H_t)\) denote the treatment mechanism. For regime \(d\), its cumulative clever covariate through time \(t\) is $$ C_t^d=\prod_{s=0}^{t} \frac{\mathbb{1}\{A_s=d_s(H_s)\}}{g_s(d_s(H_s)\mid H_s)}. $$ The indicator requires the observed treatment history to agree with the regime through time \(t\); the inverse probabilities correct for the chance of following that history. Changing \(d\) generally changes \(C_t^d\), even when every \(g_t\) fit is shared.

For clarity, let \(Q_t^d(H_t)\) be the conditional mean of the final outcome under regime \(d\), given history \(H_t\), when the regime is followed from time \(t\) onward. At the end of follow-up, the terminal regression is the conditional mean of \(Y\) given the observed terminal history. Earlier regressions are obtained recursively by conditioning the next-step regression on \(H_t\) and the action \(d_t(H_t)\). The target is the mean of \(Q_0^d(L_0)\) over the population distribution of baseline history.

A Shared-Fit, Separate-Targeting Workflow

Fit the treatment mechanisms \(g_t\) and the initial outcome-regression sequence using the full data or a cross-fitting scheme. These fits may be reused for all regimes. Then, for each regime, work backward through the outcome regressions: use the next-step targeted prediction as the pseudo-outcome, fluctuate the current regression using that regime’s \(C_t^d\), and evaluate the updated regression at the action prescribed by the regime. The final estimate is the sample average of the targeted baseline predictions under that regime.

For a binary outcome, a common fluctuation has the form

$$ \operatorname{logit} Q_{t,\epsilon}^{d} = \operatorname{logit} Q_{t}^{0} + \epsilon_t^d C_t^d. $$

Here \(Q_t^0\) is the initial regression at the relevant backward step. The coefficient \(\epsilon_t^d\) is estimated for that regime and step, using the next-step targeted pseudo-outcome in the fluctuation likelihood. For continuous outcomes, the fluctuation must instead respect the outcome’s support and use an appropriate link or fluctuation submodel.

1
Fit nuisance functions once.
Estimate the treatment mechanisms and initial outcome regressions, with a common fitting and cross-fitting plan across regimes.
2
Construct a covariate for each regime.
For every time point, calculate \(C_t^d\) using that regime’s prescribed actions and the shared treatment-probability fits.
3
Target each regime’s recursion.
Run the backward fluctuation steps with regime-specific covariates and coefficients, carrying each regime’s updated pseudo-outcome to the next step.
4
Store estimates and contrasts.
Average each targeted baseline prediction, then form requested differences or other contrasts from the resulting vector of estimates.

The regime-specific fluctuation coefficients should not be silently replaced by one coefficient shared across regimes. In general, a single update cannot solve the distinct empirical score equations induced by different clever covariates. A joint implementation is possible, but it must preserve the separate covariate columns and estimate the corresponding coefficients as a vector.

Worked Example: Two Regimes at One Treatment Time

Worked Example: Two Regimes at One Treatment Time

Consider six individuals, no baseline covariates, one binary treatment \(A\), and a binary outcome \(Y\). The known treatment probability is \(g(1)=g(0)=0.5\). Three individuals receive each treatment. Among those with \(A=1\), the outcomes are \(1,1,0\); among those with \(A=0\), they are \(1,0,0\). Suppose the shared initial outcome prediction is \(Q^0=0.4\) for either treatment.

For the regime \(d=1\), the clever covariate is \(C^1=2\mathbb{1}\{A=1\}\). Its logistic fluctuation is \(\operatorname{logit}Q^{1,*}=\operatorname{logit}(0.4)+2\epsilon^1\) among treated individuals. The fluctuation score is proportional to \(\sum_{A_i=1}(Y_i-Q_i^{1,*})\); setting it to zero makes the updated prediction equal the treated outcome mean, \(2/3\). Since \(\operatorname{logit}(0.4)=\log(2/3)\) and \(\operatorname{logit}(2/3)=\log 2\), $$ \epsilon^1 = \frac{\operatorname{logit}(2/3)-\operatorname{logit}(0.4)}{2} = \frac{\log 2-\log(2/3)}{2} = \frac{\log 3}{2} \approx 0.5493. $$ Thus the estimated regime mean is \(\hat\psi_1=2/3\).

For \(d=0\), \(C^0=2\mathbb{1}\{A=0\}\). The untreated outcome mean is \(1/3\), so the corresponding update satisfies $$ \epsilon^0 = \frac{\operatorname{logit}(1/3)-\operatorname{logit}(0.4)}{2} = \frac{-\log 2-\log(2/3)}{2} = \frac{1}{2}\log(3/4) \approx -0.1438. $$ The resulting estimate is \(\hat\psi_0=1/3\), and the estimated contrast is \(\hat\psi_1-\hat\psi_0=1/3\). The initial regression and treatment mechanism were shared; the two covariates and fluctuation coefficients were not.

Why Contrasts Must Retain Joint Dependence

Estimates for different regimes are computed from the same individuals and often use the same nuisance fits. They are therefore generally dependent. The following elementary result explains how to organize uncertainty for a difference.

Result: Suppose a vector of regime estimates \(\hat{\boldsymbol{\psi}}\) is asymptotically linear, so that $$ \sqrt{n}(\hat{\boldsymbol{\psi}}-\boldsymbol{\psi}) = \frac{1}{\sqrt{n}}\sum_{i=1}^n \boldsymbol{D}(O_i)+o_p(1), $$ where the observations are independent and identically distributed, \(E[\boldsymbol{D}(O)]=0\), and \(\operatorname{Var}(\boldsymbol{D}(O))=\Sigma\) is finite. For a fixed contrast \(c^\top\boldsymbol{\psi}\), its estimator \(c^\top\hat{\boldsymbol{\psi}}\) has influence function \(c^\top\boldsymbol{D}(O)\) and asymptotic variance \(c^\top\Sigma c/n\).

Proof. Multiply the assumed expansion by \(c^\top\): $$ \sqrt{n}\{c^\top\hat{\boldsymbol{\psi}}-c^\top\boldsymbol{\psi}\} = \frac{1}{\sqrt{n}}\sum_{i=1}^n c^\top\boldsymbol{D}(O_i)+o_p(1). $$ The summands have mean zero and variance \(E[(c^\top\boldsymbol{D})^2]=c^\top\Sigma c\). The multivariate central limit theorem, followed by the continuous mapping theorem, gives the asserted asymptotic variance. \(\square\)

For two regimes, the variance of a difference is therefore the variance of the first estimate plus that of the second, minus twice their covariance. Treating the two estimates as independent discards this covariance and can give an incorrect standard error. In practice, estimate the joint influence-function covariance using paired contributions from each individual, or use a resampling procedure that refits and retargets all regimes together.

Result: In the one-time-point binary example with known treatment probabilities and a logistic fluctuation using \(C^d=\mathbb{1}\{A=d\}/g(d)\), the fitted fluctuation score for regime \(d\) is zero exactly when the updated prediction among individuals with \(A=d\) equals their observed outcome mean, provided both outcome values occur in that group.

Proof. The score equation is $$ \sum_{i=1}^n C_i^d(Y_i-Q_i^{d,*})=0. $$ For individuals with \(A_i\ne d\), \(C_i^d=0\); for those with \(A_i=d\), the covariate is the same positive constant \(1/g(d)\). If the fitted prediction is constant within this no-covariate example, the score reduces to $$ \frac{1}{g(d)} \left\{ \sum_{i:A_i=d}Y_i - \sum_{i:A_i=d}Q_i^{d,*} \right\}=0. $$ Cancel the positive factor and divide by the number of individuals with \(A_i=d\). The fitted prediction equals the observed mean in that group. If both binary outcomes occur, this mean is strictly between zero and one, so a finite logistic intercept can attain it. \(\square\)

Diagnostics, Alternatives, and Limitations

Shared fits improve computational organization; they do not repair weak support. Before interpreting targeted estimates, inspect regime-specific treatment probabilities and the distribution of \(C_t^d\). Very large cumulative covariates indicate that few observed histories support a regime, and a targeting update may become unstable. Also check convergence, fitted probabilities near zero or one, and whether each fluctuation score is close to zero. A coefficient diverging under logistic targeting can signal separation or an empirical support problem, not successful targeting.

Common alternatives answer related but distinct questions. Untargeted sequential regression uses the outcome recursion without the targeting update; it may be simpler, but does not enforce the same empirical score equations. Longitudinal inverse-probability weighting emphasizes treatment weights rather than targeted outcome regressions and can be especially sensitive to extreme weights. Fitting a separate complete nuisance-model pipeline for every regime is possible, but can add computation and make comparisons harder to interpret. Whatever approach is used, contrasts still require uncertainty calculations that preserve the shared-sample dependence.

A practical pitfall is to compute \(C_t^d\) once and reuse it for every regime. That is wrong whenever the regimes prescribe different actions: the indicator factors in the product change, and so do the relevant treatment probabilities. The correction is to retain shared fitted \(g_t\) functions but recompute the entire covariate product for every regime and time point. Another limitation is that shared nuisance fits do not guarantee valid inference: identification, positivity, suitable nuisance estimation, and the regularity conditions for the chosen asymptotic procedure remain necessary.

Check Your Understanding

Work through the questions, then check the solutions below.

  1. For two regimes that prescribe different actions at time zero, which parts of the analysis can be shared, and which must be regime-specific?
  2. With one treatment time and \(g(d)=0.25\), what is \(C^d\) for an individual who received the regime action? What is it for an individual who did not?
  3. If two regime estimates have variances \(0.04\) and \(0.09\), and covariance \(0.03\), what is the variance of their difference?
  4. Why can a near-zero fitted treatment probability cause a problem even if the initial outcome predictions appear reasonable?

Solutions. (1) Treatment mechanisms and initial outcome-regression fits may be shared; clever covariates, fluctuation coefficients, targeted predictions, and influence-function contributions are regime-specific. (2) For a matching individual, \(C^d=1/0.25=4\); for a nonmatching individual, \(C^d=0\). (3) The variance is \(0.04+0.09-2(0.03)=0.07\). (4) The inverse probability can be large, making the cumulative clever covariate and targeting update unstable; reasonable initial predictions do not supply the missing treatment support.

Previous

Next

Back to Tutorials
Clinical Biostats