Deriving Cumulative Clever Covariates

Why the Covariate Accumulates Over Time

In longitudinal targeted maximum likelihood estimation, a regression at a later time is fitted only for people whose observed treatment history is relevant to the intervention being studied. The clever covariate expresses how the intervention’s distribution of that history differs from its distribution in the observed data. With several treatment decisions, this comparison must accumulate across decisions. If some outcomes or intermediate measurements can be censored, it must also account for the probability of remaining observed.

The key derivation is a likelihood-ratio calculation: intervention and observed-data laws share the same conditional laws for evolving covariates, while their treatment laws differ. Their ratio therefore consists of treatment likelihood ratios, multiplied by inverse censoring probabilities when the needed response is observed only among those uncensored. This tutorial derives that calculation and connects it to the covariate used to fluctuate a sequential outcome regression.

Notation, Assumptions, and the Definition

Let $t=0,\ldots,K$ index treatment decisions. Write $W$ for baseline covariates, $L_t$ for covariates measured before decision $t$, and $H_t=(W,\bar L_t,\bar A_{t-1})$ for the observed history just before that decision. Here $\bar L_t=(L_0,\ldots,L_t)$ and $\bar A_{t-1}=(A_0,\ldots,A_{t-1})$. Treatment $A_t$ is drawn from the observed treatment mechanism $g_t(a\mid H_t)$ among those still observed.

Let $R_0=1$, and let $R_{t+1}=1$ mean that the participant remains observed through the next required measurement after decision $t$. Among participants with $R_t=1$, define the conditional probability of remaining observed as $c_t(H_t,A_t)=P(R_{t+1}=1\mid R_t=1,H_t,A_t)$. Thus $\prod_{s=0}^t c_s$ is the conditional probability of being observed through the response following decision $t$, evaluated along the person’s history.

An intervention is described by treatment distributions $q_t(a\mid H_t)$. A static or dynamic deterministic regime $d=(d_0,\ldots,d_K)$ is the special case $q_t(a\mid H_t)=I\{a=d_t(H_t)\}$. Write $Z$ for a response or pseudo-outcome at the post-decision-$t$ regression node, available only when $R_{t+1}=1$.

Definition: The time-specific clever covariate for the regression at the post-decision-$t$ node is $$ H_t^{q} = \left\{\prod_{s=0}^{t} \frac{q_s(A_s\mid H_s)}{g_s(A_s\mid H_s)} \right\} \left\{\prod_{s=0}^{t}\frac{1}{c_s(H_s,A_s)}\right\}. $$ For a deterministic regime $d$, this becomes $$ H_t^{d} = \frac{\prod_{s=0}^{t} I\{A_s=d_s(H_s)\}} {\left\{\prod_{s=0}^{t} g_s(d_s(H_s)\mid H_s)\right\} \left\{\prod_{s=0}^{t} c_s(H_s,A_s)\right\}}. $$ The covariate is used on records with $R_{t+1}=1$; the response contribution is therefore weighted by $R_{t+1}H_t^q$.

The displayed censoring factors apply when the response at this node requires observation through $R_{t+1}$. If a particular node’s response is observed after fewer censoring transitions, its product stops at the last transition required to observe that response. This time-indexing convention prevents both omission of a needed censoring factor and inclusion of a later, irrelevant one.

The change-of-measure identity below is an algebraic consequence of the longitudinal factorization and positivity. To interpret the intervention law causally as the law under a regime, also require consistency, sequential exchangeability of treatment given the measured histories, and conditionally independent censoring given the recorded histories. Positivity requires $g_s(d_s(H_s)\mid H_s)>0$ and $c_s(H_s,A_s)>0$ on histories with positive probability under the intervention. For a stochastic intervention, require $g_s(a\mid H_s)>0$ wherever $q_s(a\mid H_s)>0$.

Deriving the Cumulative Likelihood Ratio

Consider the history through decision $t$ and its next covariate measurement. Under the intervention, the treatment factors in its density are $q_s(A_s\mid H_s)$; under the observed treatment mechanism, they are $g_s(A_s\mid H_s)$. The conditional densities of the evolving covariates are the same in both laws, so they cancel in the ratio. Observing the response also selects participants with $R_{t+1}=1$, whose conditional probability contributes the factor $\prod_{s=0}^t c_s$. Inverting that selection probability gives the censoring part of the covariate.

Theorem: Suppose the longitudinal law factors into baseline, sequential covariate, treatment, and censoring mechanisms as described above, and the required treatment and censoring probabilities are positive. For any integrable response $Z$ measurable at the post-decision-$t$ node, $$ E\left[R_{t+1}H_t^q Z\right]=E^q[Z], $$ where $E^q$ averages the history under the intervention through decision $t$, with the natural covariate laws retained. In particular, for a deterministic regime, the product of indicators in $H_t^d$ selects histories that follow the regime, while the denominators rescale their observed frequency to the regime’s frequency.

Proof. Conditional on baseline, the observed law for a history through the response node has treatment factors $\prod_{s=0}^{t}g_s(A_s\mid H_s)$, the relevant covariate transition factors, and probability $\prod_{s=0}^{t}c_s(H_s,A_s)$ of being observed through that node. The intervention law replaces the treatment factors by $\prod_{s=0}^{t}q_s(A_s\mid H_s)$ and retains the same covariate transition factors. On histories with positive intervention probability, multiplying the observed density by $R_{t+1}H_t^q$ cancels each observed treatment factor and each censoring probability, leaving precisely the intervention treatment factors and the shared covariate factors. Integrating $Z$ against the resulting density proves the identity. Positivity ensures the ratios are defined, and integrability ensures the expectations exist. $\square$

The products also obey a simple recursion. Define $H_{-1}^q=1$, and let $H_t^q$ denote the covariate through decision $t$. Then

H_t^q = H_{t-1}^q \frac{q_t(A_t\mid H_t)}{g_t(A_t\mid H_t)} \frac{1}{c_t(H_t,A_t)}.

This recursion is useful in implementation: each decision contributes one new treatment ratio and one new censoring factor. For a deterministic regime, the new treatment ratio is zero when the observed treatment disagrees with the rule; otherwise it is $1/g_t(d_t(H_t)\mid H_t)$.

The same ratio explains the covariate’s role in a fluctuation. For a logistic fluctuation with offset $\operatorname{logit}\{\hat Q\}$, linear predictor $\operatorname{logit}\{\hat Q\}+\epsilon H_t^q$, and response $Z$, the log-likelihood score at $\epsilon=0$ is

\left.\frac{\partial \ell(\epsilon)}{\partial\epsilon}\right|_{\epsilon=0} = \sum_{i=1}^n R_{i,t+1}H_{i,t}^q \left\{Z_i-\hat Q_i\right\}.

Thus the cumulative likelihood ratio weights the residual contribution from the observed records whose history can inform the intervention-specific regression. The particular fluctuation link and its use in targeting are developed separately; here the derivation identifies the covariate that multiplies the regression score.

Worked Calculations

Worked Example: Two Decisions Under a Deterministic Regime

There are two decisions, $t=0,1$, and no censoring. Consider the regime $d_0=1,d_1=0$. For a participant with $A_0=1,A_1=0$, suppose $g_0(1\mid H_0)=0.40$ and $g_1(0\mid H_1)=0.75$. Compute the time-specific covariates.

At time 0, the participant matches the regime, so $H_0^d=1/0.40=2.5$. At time 1, the participant also matches, and the second observed treatment probability is $0.75$. Therefore $H_1^d=2.5/0.75=3.333\ldots$, equivalently $1/(0.40\times0.75)=1/0.30=3.333\ldots$.

For a participant with $A_0=0$, the first indicator is zero, so both cumulative covariates are zero. This is not a small positive weight: that participant’s observed path does not follow the deterministic regime.

Worked Example: Three Decisions With Dropout

Let the regime be $d=(1,1,0)$, and consider a participant who follows it and remains observed through the terminal response. Suppose the treatment probabilities along this history are $0.50,0.40,0.20$, and the corresponding censoring probabilities are $0.90,0.75,0.80$. Calculate the terminal covariate.

The product of the treatment probabilities is $0.50\times0.40\times0.20=0.04$. The probability of remaining observed through all three transitions, conditional along this history, is $0.90\times0.75\times0.80=0.54$. Since all three regime indicators equal one,

H_2^d = \frac{1}{0.04\times0.54} = \frac{1}{0.0216} \approx 46.296.

The recursion gives the same result: the successive factors are $1/(0.50\times0.90)$, $1/(0.40\times0.75)$, and $1/(0.20\times0.80)$. Their product is $2.222\ldots\times3.333\ldots\times6.25\approx46.296$. Omitting censoring would instead give $1/0.04=25$, which does not correct for the fact that only a fraction $0.54$ of such histories remain observed.

Worked Example: A Stochastic Intervention

Suppose there are two decisions and the observed treatment values are $A_0=1,A_1=0$. At the first history, let $g_0(1\mid H_0)=0.40$ and $q_0(1\mid H_0)=0.80$. At the second, let $g_1(0\mid H_1)=0.50$ and $q_1(0\mid H_1)=0.30$. Let the censoring probabilities be $0.90$ and $0.80$. Find the covariate for this observed history.

The first treatment ratio is $0.80/0.40=2$, and the second is $0.30/0.50=0.60$. Their product is $1.20$. The censoring probability product is $0.90\times0.80=0.72$, so

H_1^q = \frac{2\times0.60}{0.90\times0.80} = \frac{1.20}{0.72} = 1.666\ldots.

Unlike the deterministic example, this observed treatment path is not either fully included or excluded. It receives a positive ratio determined by how much more (or less) likely the stochastic intervention makes that path than the observed treatment mechanism does.

Diagnostics, Alternatives, and Limitations

Large covariates can arise from small treatment or censoring probabilities. They are a warning about weak empirical support for the intervention, not merely an inconvenience in computation. Inspect the treatment and censoring probabilities by time and history, the distribution of the cumulative covariates, the largest values, and how much of the total weighted contribution comes from a few records. Also check that deterministic-regime indicators are zero for mismatching histories and that the censoring product ends at the response being targeted.

Approach	What it uses	Important distinction
Sequential outcome regression	Conditional outcome regressions evaluated along the regime	Does not itself reweight observations by the cumulative intervention-to-observed ratio.
Inverse-probability weighting	Products of treatment and, when needed, censoring likelihood ratios	Uses the ratios to form a weighted estimating equation or mean; the clever covariate uses them in a regression fluctuation.
Longitudinal TMLE	Sequential regressions with time-specific clever covariates	Combines regression updates with the intervention-to-observed ratios rather than treating the ratio as an outcome regression.

A frequent mistake is to use only the treatment probability at the current time. The correction is to accumulate every treatment ratio from time 0 through the current regression node: earlier decisions determine whether the observed history represents the intervention at all. A second mistake is to multiply by censoring probabilities instead of their inverses; because observed responses are selected with probability $\prod c_s$, correction requires $1/\prod c_s$. A third is to attach a censoring factor for a time after the response was measured, which changes the target weighting rather than correcting its observation.

These covariates cannot repair structural positivity violations: if a regime assigns positive probability to a treatment history that is impossible in the observed data, the required likelihood ratio is undefined. Even when probabilities are technically positive, very small values can produce unstable estimates and high variance. Truncating large weights may improve numerical stability but changes the estimating procedure and can introduce bias; it should not be presented as restoring the unsupported histories. The earlier material on treatment and censoring mechanism nuisance models supplies the conditional probabilities used here, but the cumulative product must still be checked as a whole.

Check Your Understanding

Work through each item, then compare with the solutions below.

For a deterministic regime, what makes the treatment part of a time-$t$ covariate equal zero?
With one decision, $g_0(d_0\mid H_0)=0.25$ and $c_0(H_0,d_0)=0.80$, what is the covariate for a matching, observed participant?
In a two-decision stochastic intervention, the treatment ratios are $1.5$ and $0.5$, and the censoring probabilities are $0.9$ and $0.8$. Calculate the cumulative covariate.
Why does the treatment likelihood ratio accumulate over earlier decisions, rather than using only $g_t$?
What does an extremely large covariate suggest, and what can it not establish by itself?

Solutions. (1) Any observed treatment differs from the deterministic regime at one or more decisions up to $t$, making the product of regime indicators zero. (2) $1/(0.25\times0.80)=1/0.20=5$. (3) The treatment-ratio product is $1.5\times0.5=0.75$; the censoring product is $0.9\times0.8=0.72$; hence the covariate is $0.75/0.72=1.0417$ (approximately). (4) Earlier treatment decisions are part of the history whose frequency under the intervention must be related to its observed frequency, so each earlier treatment factor remains in the likelihood ratio. (5) It suggests that the observed history had low treatment or censoring probability and may indicate instability or weak support. It does not, by itself, prove a positivity violation; inspect the conditional probabilities and the histories where the large values occur.