Common Errors in Clever-Covariate Construction

Why Clever-Covariate Errors Matter

In longitudinal targeted maximum likelihood estimation (LTMLE), a clever covariate places the outcome-regression update on the scale of the target intervention. A missing time factor can leave some stages improperly weighted; a reversed likelihood ratio can make rare treatment histories receive less, rather than more, influence. Including the wrong regime indicator or mishandling censoring changes the targeting equation itself.

The most reliable check is to derive the covariate from the data-generating sequence, one time point at a time. This tutorial fixes a convention for that sequence and uses it throughout. Other implementations may index censoring differently; translate their definitions to the actual ordering of treatment, censoring, and covariate observation before comparing formulas.

Set Up the Time Ordering First

Consider times \(t=0,\ldots,K\). At time \(t\), the observed history is \(H_t\), treatment \(A_t\) is assigned, and then the next covariate \(L_{t+1}\) is measured. The terminal outcome is \(Y=L_{K+1}\). Let \(R_0=1\), and let \(R_{t+1}=1\) indicate that the person remains observed to measure \(L_{t+1}\). Thus \(R_t=1\) is required to reach the treatment decision at time \(t\). Write \(g_t(a\mid h)=P(A_t=a\mid H_t=h,R_t=1)\), and write \(c_t(h,a)=P(R_{t+1}=1\mid H_t=h,A_t=a,R_t=1)\).

Let \(d_t(H_t)\) be the treatment prescribed by a deterministic regime. The regime-adherence indicator through time \(t\) is \(J_t^d=\prod_{s=0}^{t} I\{A_s=d_s(H_s)\}\). The product runs through \(t\), inclusive: it records whether every treatment decision from baseline through the current decision agrees with the regime.

Definition: Under this time-ordering convention, the observed-data clever covariate for the regression at stage \(t\) is \(H_t^d=J_t^d\big/\prod_{s=0}^{t}\{g_s(A_s\mid H_s)c_s(H_s,A_s)\}\), evaluated for individuals with \(R_{t+1}=1\). Equivalently, a score written over the full sample includes the multiplier \(R_{t+1}\), and contributes zero for individuals with \(R_{t+1}=0\).

The symbol \(H_t\) here denotes the observed history; \(H_t^d\) denotes the clever covariate. The superscript \(d\) identifies the regime, not a power. If censoring is absent, set each \(c_s=1\), and the censoring factors disappear. If censoring occurs at a different point in the time ordering, the factors must be adjusted to the observation mechanism rather than copied mechanically.

Derivation: Why the Factors Have This Form

Assume consistency, sequential exchangeability for treatment, and sequentially conditionally independent censoring: given the recorded history and current treatment, retention does not select on the next covariate or outcome information needed for the regime-specific distribution. Assume also positivity: along histories that can arise under the regime, the relevant treatment probabilities and retention probabilities are strictly positive. These conditions identify the regime distribution from the observed data.

At one stage, conditional on reaching \(t\) with history \(H_t=h\), the observed probability of taking the prescribed treatment and remaining observed is \(g_t(d_t(h)\mid h)c_t(h,d_t(h))\). Dividing by this probability cancels the observed probability of that event. The next covariate then follows its conditional transition distribution under the prescribed treatment, by the censoring assumption. Repeating this cancellation at each stage gives the following change-of-measure identity.

Result: For any integrable function \(f\) of the history through \(L_{t+1}\), under consistency, the stated sequential exchangeability and censoring assumptions, and positivity, \(E[R_{t+1}H_t^d f(H_{t+1})]\) equals the expectation of \(f(H_{t+1})\) under the regime \(d\). Here the right-hand expectation uses the baseline distribution and, at each stage \(s\leq t\), the conditional covariate transition under \(A_s=d_s(H_s)\).

Condition on the current history.
At stage \(s\), the observed-data contribution for the prescribed treatment and retention has probability \(g_s(d_s(H_s)\mid H_s)c_s(H_s,d_s(H_s))\).

Multiply by the inverse factor.
The factor \(I\{A_s=d_s(H_s)\}/\{g_s(A_s\mid H_s)c_s(H_s,A_s)\}\) removes that probability: the indicator restricts attention to the prescribed action, and the denominator cancels its observed treatment and retention probabilities.

Continue through time.
Applying the same conditional calculation for \(s=0,\ldots,t\) leaves the baseline distribution and the successive covariate transitions under the prescribed actions. Integrating \(f\) against that distribution proves the identity.

This argument also shows why the product must contain every factor through the stage being targeted. Omitting a stage leaves its observed treatment or retention probability uncancelled. Reversing the ratio multiplies by that probability instead of cancelling it.

Worked Audits

Worked Example: Checking Every Time Factor

Suppose \(K=1\), the regime prescribes \(A_0=A_1=1\), and an individual has \(A_0=A_1=1\). Their treatment probabilities are \(g_0(1\mid H_0)=0.50\) and \(g_1(1\mid H_1)=0.40\); their retention probabilities are \(c_0(H_0,1)=0.80\) and \(c_1(H_1,1)=0.75\). Calculate the stage-1 clever covariate.

The individual follows the regime at both times, so \(J_1^d=1\times1=1\). The denominator includes both stages and both types of probability: \(0.50\times0.80\times0.40\times0.75=0.12\). Therefore \(H_1^d=1/0.12=8.333\ldots\).

If the stage-0 treatment factor \(0.50\) is omitted, the result is \(1/(0.80\times0.40\times0.75)=4.167\ldots\), exactly half the correct value. If the stage-1 retention factor \(0.75\) is omitted, the result is \(1/(0.50\times0.80\times0.40)=6.25\), also too small. These discrepancies are not harmless notational choices: the omitted probabilities no longer cancel in the change of measure.

Worked Example: Catching a Reversed Ratio

At one stage, suppose a regime prescribes treatment \(1\), the observed treatment probability is \(g(1\mid h)=0.20\), and the retention probability is \(c(h,1)=0.90\). For an adherent, observed individual, the correct factor is \(1/(0.20\times0.90)=1/0.18=5.556\ldots\).

A reversed construction gives \(0.20\times0.90=0.18\). The error is substantial: the erroneous value divided by the correct value is \(0.18/5.556\ldots=0.0324\), or, equivalently, the correct value is about \(30.86\) times larger. The correct covariate gives greater weight to a prescribed treatment-and-retention path that was relatively unlikely under the observed process. The reversed version does the opposite.

This check is especially useful when a formula is described verbally as a “likelihood ratio.” For a deterministic regime, the numerator is the regime-adherence indicator. The observed treatment likelihood belongs in the denominator. A stochastic intervention can instead have an intervention density in the numerator; that is a different numerator, not a reason to reverse the deterministic-regime formula.

Worked Example: Is a Censoring Factor Needed?

At two stages, suppose treatment probabilities along an adherent path are \(0.50\) and \(0.40\), and retention probabilities are \(0.80\) and \(0.50\). An adherent person observed through the final outcome has \(H_1^d=1/(0.50\times0.80\times0.40\times0.50)=1/0.08=12.5\).

If there is no loss to follow-up, censoring is absent and both retention probabilities are \(1\); the corresponding value is \(1/(0.50\times0.40)=5\). In the stated example, omitting the second retention factor alone gives \(1/(0.50\times0.80\times0.40)=6.25\), half the correct value. Including a censoring factor is appropriate only if that factor represents a retention event required to observe the regression outcome or pseudo-outcome at that stage.

Regime Indicators and the Targeting Equation

The indicator \(J_t^d\) is a product of adherence indicators, not a product of treatment probabilities. If a person deviates at any time \(s\leq t\), one factor in the product is zero and \(H_t^d=0\). That is correct: the person's observed history does not represent the regime path at this stage. Do not replace the indicator by \(g_s(d_s(H_s)\mid H_s)\); doing so would weight nonadherent observations and change the covariate.

The denominator must also be evaluated at the treatment actually observed, \(g_s(A_s\mid H_s)\). For a contributing adherent person, \(A_s=d_s(H_s)\), so it equals the probability of the prescribed treatment. For a nonadherent person, the indicator makes the covariate zero; the denominator should still be defined and finite on the observed data used in the computation.

Result: For a logistic fluctuation with offset \(\operatorname{logit}\{\hat Q_t(H_t,A_t)\}\) and covariate \(H_t^d\), the score at fluctuation parameter \(\epsilon=0\), over the rows with \(R_{t+1}=1\), is \(\sum_{i=1}^n H_{t,i}^d\{\tilde Y_{t+1,i}-\hat Q_t(H_{t,i},A_{t,i})\}\), where \(\tilde Y_{t+1}\) is the regression target at that stage. This follows by differentiating the Bernoulli log-likelihood: for a logistic mean \(\operatorname{expit}(\operatorname{logit}\hat Q_t+\epsilon H_t^d)\), its derivative is \(H_t^d(\tilde Y_{t+1}-Q_{t,\epsilon})\). Thus the factor in the covariate directly determines which residuals enter the targeting equation and with what weight.

This score calculation is a useful construction check, not a substitute for checking the time ordering. If the regression target is only observed when \(R_{t+1}=1\), use that risk set or include \(R_{t+1}\) in a full-sample score. Do not include the retention indicator twice. Likewise, a treatment probability for a later stage must not appear in a covariate whose target is observed before that treatment decision.

Diagnostics, Alternatives, and Limitations

A practical audit can be done before fitting the fluctuation. For each stage \(t\), write a factor list with one row per \(s=0,\ldots,t\), then verify that the indicator, treatment probability, and (when applicable) retention probability refer to the same stage and history. Check several records by hand: one adherent through \(t\), one who deviates at an earlier stage, and one with missing outcome information. The expected covariate is positive for an observed adherent and zero for a nonadherent person.

Omitted time factor: compare the product indices with the stages required to reach the current regression target. The denominator must cover all relevant preceding treatment and retention events, through \(t\) under the convention used here.
Reversed ratio: for probabilities below one, inverse factors should be at least one on adherent records. A product of probabilities in the numerator is a warning sign for a deterministic regime.
Incorrect regime indicator: confirm that all adherence indicators through \(t\) are multiplied. A single current-treatment indicator does not enforce adherence at earlier times.
Censoring mismatch: state exactly when censoring occurs and what observation it prevents. Include the corresponding conditional retention probabilities only for events needed to observe the target.
Extreme values: very large covariates often signal small treatment or retention probabilities. They can make finite-sample targeting unstable; inspect the component probabilities rather than silently changing the formula.

The main alternative computational strategy is to express the same intervention using likelihood ratios or weights. For a deterministic intervention, those weights still contain the adherence indicator divided by observed treatment probabilities; with censoring adjustment, they also contain the relevant inverse retention probabilities. A stochastic intervention uses its intervention density in the numerator. These forms may look different, but they must encode the same change of measure. LTMLE uses the covariate in a targeted regression update, whereas longitudinal inverse-probability weighting uses weights directly in an estimating equation; neither approach repairs a wrongly indexed probability product.

Positivity is a substantive limitation, not merely a coding issue. If a required treatment or retention probability is zero, the corresponding regime history cannot be recovered from the observed data. If probabilities are merely very small, the covariate can be highly variable and the update unstable. Truncation or restriction may be considered as an explicit analysis choice, but it changes the procedure and does not make the original positivity assumption true.

Key takeaway: For a deterministic regime, build the covariate as “adherence through the target stage, divided by the observed probabilities of the events needed to follow and observe that path.” Write the event sequence before writing the product.

Check Your Understanding

Answer each question, then compare with the solutions below.

At \(t=1\), an adherent person has \(g_0=0.5\), \(c_0=0.8\), \(g_1=0.25\), and \(c_1=0.5\). What is the clever covariate?
Under the same probabilities, what is the covariate if the person deviated from the regime at time \(0\)?
Why is \(g_0g_1c_0c_1\), rather than its reciprocal, the wrong factor for an adherent deterministic-regime covariate?
If there is no censoring, what happens to the censoring factors? If censoring occurs but does not prevent observation of the target at the stage being analyzed, should that retention factor automatically be included?

Solutions. (1) Adherence gives \(J_1^d=1\), and the denominator is \(0.5\times0.8\times0.25\times0.5=0.05\), so \(H_1^d=20\). (2) The time-0 adherence indicator is zero, so \(J_1^d=0\) and \(H_1^d=0\). (3) The observed probability product must be cancelled to recover the regime distribution; multiplying by it instead downweights unlikely adherent paths. (4) With no censoring, set each \(c_s=1\). Otherwise include a retention factor only when that retention event is required to observe the regression target, according to the stated time ordering.