Likelihood for Sequential Binary Treatment Models

Why a Sequential Likelihood Is Needed

In a longitudinal study, treatment is assigned more than once. At each decision, the probability of receiving treatment can depend on information observed so far, including earlier treatment and time-varying covariates. The relevant likelihood therefore has one conditional contribution per decision, rather than a single probability for a treatment sequence that ignores its history.

This tutorial constructs that likelihood. Its factorization follows from the chain rule for sequential conditional distributions. It does not require treatment assignments to be marginally independent across time: dependence induced by past treatment and covariate history is represented by conditioning on that history.

Histories and Conditional Treatment Probabilities

Consider $K$ treatment decisions for subject $i$, indexed by $t=1,\ldots,K$. Let $A_{it}\in\{0,1\}$ be the treatment assigned at decision $t$. Write $H_{it}$ for the observed history available immediately before that decision. It can contain baseline covariates, time-varying covariates observed before the decision, and prior treatments. It must not contain information observed only after the decision.

Definition: The sequential treatment probability at decision $t$ is $g_t(a\mid h;\theta)=P(A_{it}=a\mid H_{it}=h)$, for $a\in\{0,1\}$. In a Bernoulli model, write $p_{it}(\theta)=P(A_{it}=1\mid H_{it})$. Thus the conditional probability of the observed assignment is $p_{it}(\theta)^{A_{it}}\{1-p_{it}(\theta)\}^{1-A_{it}}$.

The model may have a different parameter vector at each decision, or a common parameter vector, or parameters shared in some other specified way. We use $\theta$ for all treatment-model parameters. The factorization below does not itself choose a parameterization; it applies once the conditional probabilities $p_{it}(\theta)$ have been specified.

For likelihood calculations, assume the treatment assignments are observed at the decisions being modeled, and that their conditional distributions are Bernoulli with the stated probabilities. To multiply contributions across subjects, assume subjects are independent, conditional on any fixed design information. No assumption of independence across decision times is required. For a finite logistic model, fitted probabilities are between zero and one; in a general model, each observed assignment must have positive conditional probability for its likelihood to be positive.

Worked Example: The Probability of One Treatment Path

A subject has two treatment decisions. Given the observed pre-decision history, the modeled probability of treatment at decision 1 is $0.8$; at decision 2, conditional on the updated history, it is $0.3$. The observed path is $(A_{i1},A_{i2})=(1,0)$. Find its sequential likelihood contribution.

At decision 1 the observed assignment is 1, so its factor is $0.8$. At decision 2 the observed assignment is 0, so its factor is the probability of no treatment, $1-0.3=0.7$. Therefore

$$ L_i=0.8(1-0.3)=0.8(0.7)=0.56. $$

The corresponding log-likelihood contribution is $\log(0.56)=\log(0.8)+\log(0.7)\approx-0.5798$. The second factor is not $0.3$: that is the probability of the assignment that was not observed.

Factorization Across Decision Times and Subjects

For one subject, let $\bar A_i=(A_{i1},\ldots,A_{iK})$ denote the observed treatment sequence and let $\bar H_i$ denote the sequence of histories available at the decisions. The treatment-mechanism likelihood contribution is the product of the sequential conditional probabilities evaluated at the observed assignments and histories.

Theorem: Under the sequential Bernoulli model, the contribution of subject $i$ to the treatment-mechanism likelihood is $$ L_i(\theta)=\prod_{t=1}^{K} p_{it}(\theta)^{A_{it}}\{1-p_{it}(\theta)\}^{1-A_{it}}. $$ If subjects are independent, the sample likelihood is $$ L(\theta)=\prod_{i=1}^{n}\prod_{t=1}^{K} p_{it}(\theta)^{A_{it}}\{1-p_{it}(\theta)\}^{1-A_{it}}. $$

To see why the time factorization is valid, apply the chain rule in the order decisions occur. The probability factor for decision $t$ is conditional on the information available before that decision, including the realized earlier assignments. By definition, that conditional factor is $g_t(A_{it}\mid H_{it};\theta)$. Multiplying the factors gives the probability assigned by the sequential model to the observed path:

\prod_{t=1}^{K}g_t(A_{it}\mid H_{it};\theta) = \prod_{t=1}^{K}p_{it}(\theta)^{A_{it}} \{1-p_{it}(\theta)\}^{1-A_{it}}.

For independent subjects, the joint probability of their observed data under the treatment model is the product of their subject-specific contributions. Substituting the Bernoulli expression for each factor gives the stated sample likelihood. This proves the result. $\square$

This is a likelihood for the treatment mechanism, not necessarily the full likelihood for all observed data. For example, a full-data model could also include factors for covariate distributions or outcome distributions. If those factors involve distinct nuisance parameters, they can be separated from the treatment factors when estimating $\theta$. If parameters are shared across the factors, the treatment likelihood alone need not contain all information about them.

There is also an important conditioning distinction. The sequential factors condition on each pre-decision history, not on the subject’s entire covariate record including future covariates. A future covariate may be affected by current treatment; conditioning on it changes the conditional distribution being modeled. A joint density for the longitudinal data can be factorized into treatment and covariate transition factors, but that does not make the treatment likelihood equal to a treatment probability conditional on all future observations.

The Log-Likelihood and Its Contributions

Taking logarithms turns the product over decisions and subjects into a sum. Since $A_{it}$ is either zero or one, the log-likelihood can be written in a compact form that includes both treated and untreated decisions.

Result: The sequential Bernoulli log-likelihood is $$ \ell(\theta)= \sum_{i=1}^{n}\sum_{t=1}^{K} \left[ A_{it}\log p_{it}(\theta) +(1-A_{it})\log\{1-p_{it}(\theta)\} \right]. $$

This follows by applying $\log\prod_j q_j=\sum_j\log q_j$ to the likelihood and using $\log(x^a)=a\log x$. For each observed treatment, exactly one of the two terms in brackets contributes: the first if $A_{it}=1$, the second if $A_{it}=0$. This proves the displayed expression. $\square$

When the conditional probabilities are modeled by logistic regression, let $p_{it}(\beta)=\operatorname{expit}(X_{it}^{\mathsf T}\beta)$, where $\operatorname{expit}(u)=1/(1+\exp(-u))$ and $X_{it}$ is a vector of predictors from $H_{it}$. Substitution into the log-likelihood gives the familiar Bernoulli logistic log-likelihood, with one row for each subject-decision pair.

Derived score: For the logistic model above, the score for $\beta$ is $$ U(\beta)=\frac{\partial\ell(\beta)}{\partial\beta} =\sum_{i=1}^{n}\sum_{t=1}^{K}X_{it}\{A_{it}-p_{it}(\beta)\}. $$

For completeness, write $p=p_{it}(\beta)$ and $x=X_{it}$ for a single contribution. The derivative of the logit probability is $\partial p/\partial\beta=p(1-p)x$. Differentiating the contribution gives

\begin{aligned} \frac{\partial}{\partial\beta} \left[A\log p+(1-A)\log(1-p)\right] &=\left(\frac{A}{p}-\frac{1-A}{1-p}\right)p(1-p)x\\ &=(A-p)x. \end{aligned}

Summing this derivative over all decisions and subjects yields the score. The derivation shows why the score sums residual-like terms $A_{it}-p_{it}$, while the likelihood itself retains the correct conditional probability for each observed assignment.

Worked Example: Likelihood for Three Subjects

Suppose there are two decisions per subject, with the following conditional probabilities evaluated at a specified parameter value. The probabilities at decision 2 are understood to be conditional on each subject’s updated pre-decision history.

Subject	Observed path	$p_{i1}$	$p_{i2}$	Contribution
1	(1, 0)	0.7	0.2	$0.7(0.8)=0.56$
2	(0, 1)	0.7	0.4	$0.3(0.4)=0.12$
3	(1, 1)	0.5	0.6	$0.5(0.6)=0.30$

For subject 1, the first assignment contributes $0.7$, and the untreated second decision contributes $1-0.2=0.8$. For subject 2, the untreated first decision contributes $1-0.7=0.3$, followed by $0.4$ for treatment. Subject 3 contributes $0.5$ and $0.6$. Under independent subjects, the full treatment likelihood at this parameter value is

$$ L=0.56(0.12)(0.30)=0.02016. $$

The log-likelihood is the sum of the three subject contributions: $\log(0.56)+\log(0.12)+\log(0.30)\approx-0.579818-2.120264-1.203973=-3.904055$. Taking the logarithm of the product gives the same result: $\log(0.02016)\approx-3.904055$. The agreement checks both the multiplication and the log transformation.

Worked Example: A Logistic Score Contribution

At a particular parameter value, suppose $\beta=0$, so that the fitted treatment probability is $0.5$ for each observation. Consider three subject-decision observations with scalar predictor values $X=1,2,-1$ and assignments $A=1,0,1$. Find their combined score contribution.

The logistic score contribution for an observation is $X(A-p)$. The three contributions are $1(1-0.5)=0.5$, $2(0-0.5)=-1$, and $(-1)(1-0.5)=-0.5$. Thus the combined contribution is

$$ 0.5-1-0.5=-1. $$

This is a score evaluated at $\beta=0$, not an estimate of $\beta$. It indicates the direction of the local likelihood change: at this value, the derivative with respect to $\beta$ is negative for these observations.

Assumptions, Diagnostics, and Common Errors

The factorization is useful because it matches the sequence in which treatment decisions are made. Its validity as a statistical likelihood depends on the conditional model and the data structure, not on a causal interpretation of the fitted probabilities. Causal conclusions require additional assumptions beyond this likelihood construction.

Check the timing of histories. Each predictor in $H_{it}$ must be known before $A_{it}$ is assigned. Using a post-assignment covariate as though it were pre-decision information changes the conditioning statement.
Check the Bernoulli coding. For $A_{it}=0$, the factor is $1-p_{it}$; for $A_{it}=1$, it is $p_{it}$. A single reversed factor changes the path likelihood.
Inspect fitted probabilities and data support. Probabilities extremely close to zero or one, sparse histories, or treatment patterns that are rarely observed can make estimates unstable or make likelihood maximization problematic. These are substantive and numerical warnings, not evidence that the algebraic factorization is wrong.
Check the subject-level independence assumption. If subjects are clustered or otherwise dependent, multiplying their likelihood contributions as if they were independent may not describe the joint sampling model. The within-subject time factorization does not fix between-subject dependence.

A common mistake is to call the treatment decisions independent because the likelihood is a product over $t$. That conclusion is incorrect. The factor at time $t$ is conditional on $H_{it}$, which can include previous treatment and covariates. The product is the chain-rule factorization of sequential conditional probabilities; it does not assert that $P(A_{it}\mid A_{i1},\ldots,A_{i,t-1})=P(A_{it})$.

Another mistake is to multiply only the probabilities of treatment and omit the probabilities of no treatment. The correction is to use $p_{it}^{A_{it}}(1-p_{it})^{1-A_{it}}$ at every observed decision. A third mistake is to treat this as the full likelihood for the entire longitudinal record. It is specifically the likelihood contribution for the modeled treatment mechanism; a full-data likelihood may contain additional covariate and outcome factors.

The main alternative at this stage is not a different factorization of the same model, but a different modeling target: one could model a joint distribution for the full longitudinal data, or specify conditional treatment probabilities with a different parameter structure. Each approach answers a different modeling question and requires its own assumptions. The sequential treatment likelihood is particularly transparent about the information conditioning each assignment, but it cannot compensate for omitted history variables, misspecified conditional probabilities, unobserved assignments, or lack of data support. Choosing how parameters are shared across decision times is a separate modeling decision.

Key takeaway: A sequential binary treatment likelihood has one Bernoulli factor for every observed subject-decision pair. The factors are conditional on pre-decision histories, multiply across time by the chain rule, and multiply across subjects only under the assumed between-subject independence.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A subject receives no treatment at a decision where the modeled treatment probability is $0.25$. What is the likelihood factor for that decision?
For a two-decision path $(1,1)$, the conditional treatment probabilities are $0.6$ and $0.4$. Find the path contribution and its log.
Why does a product of conditional factors across time not imply marginal independence of treatment assignments?
What is the logistic score contribution for one observation with $X=2$, $A=0$, and fitted probability $p=0.3$?

Solutions. (1) The observed assignment is zero, so the factor is $1-0.25=0.75$. (2) Both observed assignments are one, so the contribution is $0.6(0.4)=0.24$, and its log is $\log(0.24)\approx-1.4271$. (3) Each later factor is conditional on the history, including prior assignments; the product is the chain-rule factorization, not a claim that the unconditional assignments are independent. (4) The score contribution is $X(A-p)=2(0-0.3)=-0.6$.

Subject	Observed path	\(p_{i1}\)	\(p_{i2}\)	Contribution
1	(1, 0)	0.7	0.2	\(0.7(0.8)=0.56\)
2	(0, 1)	0.7	0.4	\(0.3(0.4)=0.12\)
3	(1, 1)	0.5	0.6	\(0.5(0.6)=0.30\)