Sequential Integration and Forward Simulation

Two Ways to Average Over Regime-Specific Histories

Under a fixed treatment regime, the covariates that develop over time can still vary from person to person. A regime-specific expectation therefore averages the outcome over the distribution of covariate histories that would arise under that regime. There are two direct ways to do this: integrate the conditional distributions one time point at a time, or generate histories sequentially and average the resulting outcomes.

The key fact is that these are not different estimands. When they use the same conditional distributions, sequential integration and forward simulation describe the same probability law. Integration evaluates its expectation directly; simulation approximates that expectation with generated histories.

Setup: Let $L_0$ be baseline covariates, $A_t$ treatment at time $t$, $L_t$ covariates measured after treatment at time $t-1$, for $t=1,\ldots,K$, and $Y$ the final outcome after treatment $A_K$. A fixed regime is $a=(a_0,\ldots,a_K)$. Write $p_0(dl_0)$ for the baseline distribution, $q_t(dl_t\mid l_0,\ldots,l_{t-1},a_0,\ldots,a_{t-1})$ for the conditional distribution of $L_t$ under the regime, and $m(l_0,\ldots,l_K,a)=E[Y\mid L_0=l_0,\ldots,L_K=l_K,A_0,\ldots,A_K=a]$ for the conditional outcome mean. The regime-specific expectation is $$ \mu(a)=\int m(l_0,\ldots,l_K,a)\,p_0(dl_0)\prod_{t=1}^{K}q_t(dl_t\mid l_0,\ldots,l_{t-1},a_0,\ldots,a_{t-1}). $$

This expression uses conditional covariate distributions under the specified treatment values, not the observed treatment-assignment distributions. To interpret $\mu(a)$ causally using observed data requires the identification conditions established in the earlier material on the longitudinal g-formula, including consistency, sequential exchangeability, and positivity for the histories relevant to the regime. The calculations below concern what follows once the regime-specific conditional distributions and outcome mean have been specified. They do not establish that a fitted model is correct or that the causal assumptions hold.

Sequential Integration as a Backward Recursion

Sequential integration can be organized as a recursion from the final outcome back toward baseline. Start with the conditional mean of $Y$ at a complete history. Then average that value over the last covariate distribution, conditional on the preceding history. Continue integrating out covariates in reverse temporal order, and finally average over baseline covariates.

Definition: Set $V_K(l_0,\ldots,l_K)=m(l_0,\ldots,l_K,a)$. For $t=K,K-1,\ldots,1$, define $$ V_{t-1}(l_0,\ldots,l_{t-1}) =\int V_t(l_0,\ldots,l_t)\, q_t(dl_t\mid l_0,\ldots,l_{t-1},a_0,\ldots,a_{t-1}). $$ Then $\mu(a)=\int V_0(l_0)\,p_0(dl_0)$.

The recursion does not discard history: each integration is conditional on the covariates observed earlier in the generated history. That conditioning is what allows treatment-confounder feedback and other dependencies across time to be represented by the sequence of kernels.

Worked Example: Integrating a Two-Time-Point Binary History

Suppose baseline is fixed, there are two binary covariates $L_1,L_2$, and the regime-specific transition probabilities are $P(L_1=1)=0.4$, $P(L_2=1\mid L_1=0)=0.2$, and $P(L_2=1\mid L_1=1)=0.75$. Let the conditional outcome mean be $0.1,0.4,0.6,0.95$ for histories $(L_1,L_2)=(0,0),(0,1),(1,0),(1,1)$, respectively.

First integrate over $L_2$ for each value of $L_1$. For $L_1=0$, the probabilities of $L_2=0$ and $L_2=1$ are $0.8$ and $0.2$, so the conditional average is $0.8(0.1)+0.2(0.4)=0.16$. For $L_1=1$, those probabilities are $0.25$ and $0.75$, giving $0.25(0.6)+0.75(0.95)=0.15+0.7125=0.8625$.

Now average over $L_1$: the probability of $L_1=0$ is $0.6$, and that of $L_1=1$ is $0.4$. Therefore $$ \mu(a)=0.6(0.16)+0.4(0.8625)=0.096+0.345=0.441. $$ The conditional probabilities at each step sum to one, and the final average lies between the smallest and largest conditional outcome means, as expected.

Forward Simulation Uses the Same Kernels

Forward simulation follows time order rather than reverse order. Draw $L_0$ from $p_0$. Given that draw, generate $L_1$ from its conditional distribution under $a_0$; then generate $L_2$ given the history and $a_0,a_1$; continue until the final covariate. Finally draw $Y$ from its conditional outcome distribution, or record its conditional mean $m$ if the target is an expectation.

Initialize.
Draw baseline covariates from $p_0$.

Extend the history.
At each time $t$, draw $L_t$ from $q_t$, conditioning on the history already generated and the regime values through $a_{t-1}$.

Complete the outcome.
Evaluate $m$ at the simulated history, or draw $Y$ from its conditional distribution if simulating the outcome itself.

Repeat and average.
With independent repetitions, the sample average estimates the expectation under the simulated history distribution.

At every step, the treatment value is fixed by the regime. Treatment is not randomly drawn from its observed assignment model. The randomness in this procedure comes from baseline covariates, covariate transitions, and—if drawn—the outcome.

Worked Example: Generating Four Histories by Hand

Use the binary-history probabilities from the preceding example. Generate $L_1=1$ when $U_1<0.4$. Given $L_1=0$, generate $L_2=1$ when $U_2<0.2$; given $L_1=1$, use the threshold $U_2<0.75$. Generate a binary outcome $Y=1$ when $U_Y<m(L_1,L_2,a)$. Apply these rules to four triples of independent uniform draws:

$(U_1,U_2,U_Y)=(0.2,0.5,0.05)$: $L_1=1$, since $0.2<0.4$; $L_2=1$, since $0.5<0.75$; and $Y=1$, since $0.05<0.95$.
$(0.7,0.1,0.3)$: $L_1=0$, since $0.7\geq0.4$; $L_2=1$, since $0.1<0.2$; and $Y=1$, since $0.3<0.4$.
$(0.35,0.9,0.7)$: $L_1=1$, since $0.35<0.4$; $L_2=0$, since $0.9\geq0.75$; and $Y=0$, since $0.7\not<0.6$.
$(0.9,0.3,0.95)$: $L_1=0$, since $0.9\geq0.4$; $L_2=0$, since $0.3\geq0.2$; and $Y=0$, since $0.95\not<0.1$.

The simulated outcomes are $1,1,0,0$, with average $2/4=0.5$. The exact expectation is $0.441$, as calculated above. A four-draw average need not equal the exact expectation; it varies with the particular random draws. The example shows the mechanics of generating histories, not a guarantee that a small simulation is precise.

Why the Two Procedures Agree

Theorem: Suppose the baseline and transition distributions are probability kernels on standard Borel spaces, the outcome is measurable, and $E[|Y|]$ under the regime is finite. If forward simulation draws each successive covariate from the same kernels used in the iterated integral and draws the outcome from its conditional distribution, then the expectation of the simulated outcome equals $\mu(a)$. The same equality holds if the simulator records $m$ instead of drawing $Y$.

Proof. The sequential generation rule gives the joint distribution of the simulated history as the product of its successive conditional distributions: $$ p_0(dl_0)\prod_{t=1}^{K}q_t(dl_t\mid l_0,\ldots,l_{t-1},a_0,\ldots,a_{t-1}). $$ This follows directly from the chain rule for conditional distributions: the first draw has law $p_0$, and each subsequent draw has the specified conditional law given the preceding draws. By the conditional distribution of the final outcome, its conditional expectation given the generated history is $m(l_0,\ldots,l_K,a)$. The tower property of expectation therefore gives $$ E[Y]=E\{E[Y\mid L_0,\ldots,L_K,A_0,\ldots,A_K=a]\} =\int m(l_0,\ldots,l_K,a)\,p_0(dl_0)\prod_{t=1}^{K}q_t(dl_t\mid l_0,\ldots,l_{t-1},a_0,\ldots,a_{t-1}) =\mu(a). $$ Integrability permits the expectation and iterated integrals to be evaluated in this order. If the simulator records $m$, its expectation is directly the same final integral. Standard Borel spaces ensure the conditional distributions and sequential random draws used in this argument are well-defined. $\square$

The backward recursion is also just a regrouping of that same integral. For nonnegative $m$, Tonelli’s theorem permits the iterated integrations; for integrable $m$, Fubini’s theorem does so. Integrating first over $L_K$ produces $V_{K-1}$, integrating the result over $L_{K-1}$ produces $V_{K-2}$, and continuing yields $V_0$. Thus the recursion computes exactly the integral in the theorem, not a different quantity.

Dependence, Diagnostics, and Alternatives

Worked Example: Why Marginal Distributions Are Not Enough

In the binary example, the joint history probabilities are $P(0,0)=0.6(0.8)=0.48$, $P(0,1)=0.6(0.2)=0.12$, $P(1,0)=0.4(0.25)=0.10$, and $P(1,1)=0.4(0.75)=0.30$. These sum to $1.00$. The marginal probabilities are $P(L_1=1)=0.4$ and $P(L_2=1)=0.12+0.30=0.42$.

If one incorrectly treats $L_1$ and $L_2$ as independent, the implied history probabilities become $0.6(0.58)=0.348$, $0.6(0.42)=0.252$, $0.4(0.58)=0.232$, and $0.4(0.42)=0.168$, in the order $(0,0),(0,1),(1,0),(1,1)$. The resulting average is $0.348(0.1)+0.252(0.4)+0.232(0.6)+0.168(0.95)=0.0348+0.1008+0.1392+0.1596=0.4344$, not $0.441$. The discrepancy occurs because the marginal product changed the joint distribution of histories.

Exact summation or integration and forward simulation are alternatives for evaluating the same specified expectation. Exact calculation avoids random simulation error and is convenient when the history space is small, as in the first example. Simulation is often more practical when histories are high-dimensional or continuous, but its average is subject to Monte Carlo variation. Neither approach repairs an incorrect conditional model: if the kernels or outcome mean are misspecified, both methods can agree perfectly while estimating the wrong regime-specific expectation.

Useful checks follow directly from the calculation. Confirm that each transition distribution is normalized; verify that each generated covariate uses the conditional distribution for its actual preceding history; and check that simulated histories have plausible frequencies for simple history events. For a small discrete problem, compare the simulation results with exact enumeration. A difference between the exact integral and a simulation average is not, by itself, evidence of an error; the simulation is random. A systematic discrepancy in a large simulation, however, can indicate a coding error, such as using a marginal transition probability in place of a conditional one or applying the wrong regime value at a time point.

A further limitation is that the equality proved here is computational, conditional on using the same distributions. It does not show that observed data identify those distributions, that the models are adequate, or that the target regime has sufficient support. Those are causal identification and modeling questions, not consequences of the integration-simulation equivalence.

Common Mistakes

Breaking the order of conditioning. Drawing $L_t$ from a marginal distribution instead of its distribution given the generated history changes the joint law of trajectories.
Changing treatment assignment into a random draw. Under a fixed regime, set each treatment to its prescribed value; do not simulate treatment from the observed treatment mechanism.
Expecting a small simulated average to equal the exact integral. A finite average fluctuates. The theorem equates the expectation of the simulation result with the integral, not every realization with that integral.
Taking agreement as evidence of correct models. Two calculations using the same misspecified kernels can agree because they compute the same wrong input-based quantity.

Key takeaway: Sequential integration and forward simulation are two evaluations of one regime-specific probability law. Preserve the conditional history at every step, and distinguish computational agreement from causal validity or model adequacy.

Check Your Understanding

Answer each question, then compare with the solutions below.

In the binary example, what is $P(L_1=1,L_2=0)$, and which conditional probability is used to calculate it?
Why does a four-draw simulated average of $0.5$ not contradict the exact expectation $0.441$?
What changes if $L_2$ is drawn from its marginal distribution rather than conditional on $L_1$?
Does equality between exact integration and forward simulation establish that the regime-specific expectation is causally identified? Explain.

Solutions. (1) $P(L_1=1,L_2=0)=P(L_1=1)P(L_2=0\mid L_1=1)=0.4(0.25)=0.10$. (2) The theorem says the simulation result has expectation $0.441$; an individual finite average varies with its draws and can equal $0.5$. (3) It generally changes the joint distribution of histories and can change the outcome average; in the worked calculation it changes the answer to $0.4344$. (4) No. The equality holds given the supplied kernels and outcome model. Causal interpretation additionally requires the relevant identification assumptions and adequate support.