Visit-Specific Covariate Adjustment And Changing Comparisons

Why Visit-Specific Coefficients Can Be Different Comparisons

In longitudinal data, a treatment coefficient is not meaningful without specifying both which visit it concerns and which covariates are held fixed. At one visit the adjustment set might contain baseline covariates; at a later visit it might also contain a time-varying covariate measured before that treatment decision. Those models condition on different histories, and may compare treated and untreated people in different parts of the covariate distribution.

A difference between visit-specific coefficients can therefore arise even when the conditional treatment contrasts have not changed. Treatment prevalence, covariate distributions, and the amount of treatment overlap within covariate strata can all change over time. Conversely, matching coefficients do not establish that the visits represent the same comparison.

Let \(A_t\) denote treatment at visit \(t\), \(L_t\) a covariate measured before that visit’s treatment decision, and \(Y_t\) an outcome measured afterward. Write \(C_t\) for the covariates included in a particular regression at visit \(t\). The comparison of interest in this tutorial is a regression coefficient for \(A_t\), conditional on \(C_t\); it is not automatically the effect of a treatment regime.

The earlier material on treatment-by-confounder interactions established why a treatment effect may vary across covariate levels. Here the focus is on what a coefficient from a model without that interaction represents, and why that representation may change across visits.

What a Common Adjusted Coefficient Weights

First consider a single visit and a categorical covariate \(C\) with finitely many strata \(g\). Let \(p_g=P(C=g)>0\), let \(e_g=P(A=1\mid C=g)\), and suppose \(0<e_g<1\) in every stratum. Assume finite second moments and the conditional mean model \(E[Y\mid A,C=g]=m_g+\delta_g A\). Fit the population least-squares regression of \(Y\) on an intercept, indicators for \(C\), and \(A\), with no \(A\)-by-\(C\) interaction.

Proof. In the population regression, removing the stratum indicators from \(A\) leaves the residual \(A-e(C)\), where \(e(C)=P(A=1\mid C)\). This follows because the least-squares prediction of a binary \(A\) using stratum indicators is its mean within each stratum. The Frisch–Waugh–Lovell theorem therefore gives \[ \beta=\frac{E[(A-e(C))Y]}{E[(A-e(C))A]}. \] For a stratum \(g\), the conditional mean of \(A-e_g\) is zero, so the term involving \(m_g\) contributes nothing to the numerator. Since \(A\) is binary, \[ E[(A-e_g)Y\mid C=g] =e_g(1-e_g)E[Y\mid A=1,C=g] -e_g(1-e_g)E[Y\mid A=0,C=g] =e_g(1-e_g)\delta_g. \] Also, \[ E[(A-e_g)A\mid C=g] =e_g-e_g^2=e_g(1-e_g). \] Averaging these expressions over the strata gives the stated numerator and denominator. The denominator is positive because \(p_g>0\) and \(0<e_g<1\). Dividing proves the formula. \(\square\)

This result clarifies two features of the coefficient. First, a stratum with little treatment variation contributes little: \(e_g(1-e_g)\) is close to zero when almost everyone in the stratum receives the same treatment. Second, if the conditional contrasts vary, the common-slope coefficient need not equal an ordinary average over people or over strata. It is determined by the regression’s treatment variation within strata.

Changing Adjustment Across Visits

At visit \(t\), a regression may adjust for a covariate history \(C_t\) available before \(A_t\). At another visit the analyst may use a different history, such as adding a newly measured \(L_t\). Even if both regressions use the same outcome scale, their treatment coefficients need not represent the same conditional comparison. The coefficient at each visit combines contrasts among the strata in its own adjustment set, with weights determined by that visit’s covariate distribution and treatment probabilities.

The following example isolates this weighting change. It assumes correctly specified conditional mean differences within the displayed strata; it does not assume that the strata at the two visits are the same people or that the coefficients are causal regime effects.

A model that includes treatment-by-covariate interactions answers a different descriptive question: it allows the fitted treatment contrast to vary by stratum. For example, with a binary \(C\), the model \(E[Y\mid A,C]=\alpha+\gamma C+\beta A+\eta AC\) has fitted treatment contrast \(\beta\) in stratum \(C=0\) and \(\beta+\eta\) in stratum \(C=1\). This follows by subtracting the model’s fitted mean at \(A=0\) from its fitted mean at \(A=1\), separately in each stratum. The common-slope model instead constrains those two contrasts to be equal.

Adjustment Can Change the Conditional Comparison

The next example compares a regression that omits a covariate with one that adjusts for it. The calculations show why an adjusted coefficient need not be a simple refinement of an unadjusted coefficient: it can change both the composition of the comparison and the coefficient’s value. In longitudinal analyses, a newly included covariate may be a measured pre-treatment history at that visit. If it was itself affected by earlier treatment, conditioning on it does not, by itself, recover the total effect of a treatment history.

For a causal interpretation of a conditional contrast, additional assumptions are required. In particular, treatment must be well-defined and consistently recorded; there must be no interference relevant to the estimand; treatment must be conditionally exchangeable with the potential outcomes given the measured pre-treatment history; and both treatment levels must have positive probability at histories in the target population. The conditional mean model must also represent the relevant contrasts adequately. These conditions do not make a coefficient from a visit-specific regression a total effect of an entire treatment history. They support an interpretation tied to the treatment decision and history being conditioned on.

Comparing Visit-Specific Analyses

Separate visit-specific regressions are easy to interpret when each has a clearly stated adjustment set and target population. However, their coefficients are directly comparable only with care. Check that the outcome scale, analytic population, covariate definitions, and model form are comparable. If the adjustment sets differ, state explicitly that the coefficients condition on different histories.

A pooled model can include visit indicators and treatment-by-visit interactions to estimate distinct visit-specific coefficients while holding a chosen adjustment strategy fixed. This helps separate differences associated with visit from differences caused by changing model specifications. It does not automatically solve misspecification or make the resulting coefficients causal. Another descriptive option is to estimate conditional contrasts and average them over an explicitly selected reference covariate distribution; the resulting standardized contrast targets that distribution rather than the regression’s implicit overlap weights.

These approaches answer different questions. A common-slope regression summarizes conditional contrasts with its own weights; an interaction model reports contrasts by stratum; and standardization averages contrasts using a specified reference distribution. None should be called “the treatment effect” without naming the scale, the conditioning history, and the population to which the statement refers.

Diagnostics and Common Pitfalls

• Calling a common slope a population-average effect. The theorem shows that its weights depend on both stratum prevalence and treatment variation. Use the population-proportion average only when that is the quantity actually calculated or targeted.
• Reading a coefficient change as effect change. First check whether the adjustment sets, covariate distributions, overlap, or analytic samples changed. Any of these can change the coefficient even if the conditional contrasts are constant.
• Ignoring weak overlap. If a stratum has \(e_g\) near 0 or 1, its weight \(p_g e_g(1-e_g)\) is small, and its contrast may be poorly estimated. Report the treatment counts or probabilities by stratum rather than relying only on the pooled coefficient.
• Assuming adjustment guarantees causality. A regression adjustment does not establish exchangeability, correct measurement, positivity, or a valid functional form. Conditioning on a covariate affected by prior treatment also changes what history is being compared.
• Comparing unlike models. Different outcome scales, samples, or interaction terms can produce coefficients with different interpretations. Align these choices or explain why they differ before interpreting a trend across visits.