Changing The Order Of Conditional Factors

Why Change the Order of Conditional Factors?

A joint outcome distribution can be written as a product of conditional distributions in more than one order. This is useful when describing outcomes collected at several visits or when working within a response pattern: one ordering may match a chosen modeling strategy, while another may be easier to interpret or calculate. The key is to distinguish the order of a factorization from the joint law being factorized. Reordering exact conditional distributions changes the conditioning sets, but it does not change that joint law.

For example, the joint distribution of two visit outcomes can be factored by first describing the distribution of the first visit and then the second visit conditional on the first. It can also be factored in the reverse order. Neither representation is inherently the unique one. However, the two conditional distributions are generally different, so their parameters and interpretations should not be carried from one order to the other without checking.

Definition: Let $Y_1,\ldots,Y_m$ be outcomes with joint distribution $f(y_1,\ldots,y_m)$. For a permutation $\pi$ of the indices $1,\ldots,m$, the chain-rule factorization in order $\pi$ is $$ f(y_1,\ldots,y_m) = \prod_{k=1}^{m} f\!\left(y_{\pi(k)}\mid y_{\pi(1)},\ldots,y_{\pi(k-1)}\right). $$ For $k=1$, the first factor is the marginal distribution $f(y_{\pi(1)})$, since there are no preceding outcomes to condition on. Each factor is taken from the same joint law.

The notation can also condition throughout on a response pattern $R=r$ and covariates $X=x$. In that case, the outcomes in the factorization are those being modeled within that pattern, and all the conditional distributions are evaluated at the same $r$ and $x$. The earlier material on pattern-mixture factorization supplies this pattern-specific setting; the chain rule applies to its joint outcome distribution just as it applies to any joint distribution.

The Two-Outcome Chain Rule

For two outcomes, $Y_1$ and $Y_2$, the two possible orders are

f(y_1,y_2) = f(y_1)f(y_2\mid y_1) = f(y_2)f(y_1\mid y_2).

The first order uses $Y_1$ as the initial marginal and conditions the distribution of $Y_2$ on $Y_1$. The reverse order uses the marginal distribution of $Y_2$ and conditions $Y_1$ on $Y_2$. The conditional distributions are related through the common joint distribution. They need not be numerically or parametrically identical.

Worked Example: Reordering a Binary Joint Distribution

Suppose a hypothetical population has two binary outcomes, $Y_1$ and $Y_2$, with the following joint probabilities. The four probabilities are nonnegative and sum to one, so they define a valid joint distribution.

$(Y_1,Y_2)$	Joint probability
(0, 0)	0.42
(0, 1)	0.18
(1, 0)	0.12
(1, 1)	0.28

First factor in the order $Y_1,Y_2$. The marginal probability of $Y_1=1$ is $0.12+0.28=0.40$, and the probability of $Y_1=0$ is $0.42+0.18=0.60$. The conditional probabilities for $Y_2=1$ are

P(Y_2=1\mid Y_1=0)=\frac{0.18}{0.60}=0.30, \qquad P(Y_2=1\mid Y_1=1)=\frac{0.28}{0.40}=0.70.

For instance, the factorization reconstructs the joint probability of $(Y_1,Y_2)=(1,1)$ as $P(Y_1=1)P(Y_2=1\mid Y_1=1)=0.40(0.70)=0.28$. For $(0,1)$, it gives $0.60(0.30)=0.18$.

Now reverse the order. The marginal probability of $Y_2=1$ is $0.18+0.28=0.46$, and the probability of $Y_2=0$ is $0.42+0.12=0.54$. Thus

P(Y_1=1\mid Y_2=0)=\frac{0.12}{0.54}\mathrel{\approx}0.2222, \qquad P(Y_1=1\mid Y_2=1)=\frac{0.28}{0.46}\mathrel{\approx}0.6087.

The reverse-order factorization gives $P(Y_1=1,Y_2=1)=0.46(0.6087)\mathrel{\approx}0.28$, with the small discrepancy due only to rounding. The joint probability is unchanged; both the initial marginal and the conditional probabilities have changed.

Why Both Orders Give the Same Joint Law

For two outcomes, the identity follows directly from the definition of conditional probability. When $P(Y_1=y_1)>0$,

P(Y_1=y_1)P(Y_2=y_2\mid Y_1=y_1) = P(Y_1=y_1) \frac{P(Y_1=y_1,Y_2=y_2)}{P(Y_1=y_1)} = P(Y_1=y_1,Y_2=y_2).

The same argument, conditioning on $Y_2=y_2$, gives $P(Y_2=y_2)P(Y_1=y_1\mid Y_2=y_2)=P(Y_1=y_1,Y_2=y_2)$. Therefore both products equal the same joint probability. For continuous outcomes, the corresponding density identity holds wherever the relevant marginal density is positive.

For more outcomes, apply the two-variable identity successively. After the first $k-1$ outcomes in a chosen order have been specified, factor the joint distribution of those outcomes and the next one into the marginal distribution of the first $k-1$ and the conditional distribution of the $k$th given them. Repeating this step yields one factor per outcome and the chain-rule formula. It does not require an independence assumption. The needed regularity is that the relevant conditional distributions are defined under the joint law; at conditioning values with zero probability (or zero marginal density), the conditional distribution may be undefined or defined only by convention, but those values do not affect the joint distribution almost surely.

Worked Example: Changing Order Within a Response Pattern

At fixed covariates $X=x$, consider a hypothetical response pattern $R=r$ in which outcomes $Y_1$ and $Y_3$ are observed. Suppose their conditional joint probabilities within this pattern are $0.30$ for $(0,0)$, $0.20$ for $(0,1)$, $0.10$ for $(1,0)$, and $0.40$ for $(1,1)$. These nonnegative values sum to one.

In the order $Y_1,Y_3$, the pattern-specific marginal probability of $Y_1=1$ is $0.10+0.40=0.50$. The conditional probabilities are

P(Y_3=1\mid Y_1=0,R=r,X=x)=\frac{0.20}{0.50}=0.40, \qquad P(Y_3=1\mid Y_1=1,R=r,X=x)=\frac{0.40}{0.50}=0.80.

Consequently, the joint probability for $(Y_1,Y_3)=(1,1)$, conditional on $R=r,X=x$, is $0.50(0.80)=0.40$. In the reverse order, $P(Y_3=1\mid R=r,X=x)=0.20+0.40=0.60$, and

P(Y_1=1\mid Y_3=1,R=r,X=x)=\frac{0.40}{0.60}\mathrel{\approx}0.6667.

The reverse product is $0.60(0.6667)\mathrel{\approx}0.40$, again allowing for rounding. The calculations use the same response pattern and covariate values in both orders. Changing the order does not change which pattern-specific joint distribution is being described.

Conditional Distributions Depend on the Order

A useful way to see how the factors change is to consider a joint distribution whose conditional means can be calculated explicitly. Even when two outcomes are jointly normal, reversing the order changes both the regression used for the conditional mean and the conditional variance.

Worked Example: Reversing the Order for Jointly Normal Outcomes

Suppose hypothetical outcomes $Y_1$ and $Y_2$ have a bivariate normal distribution with means 10 and 20, variances 4 and 9, and covariance 3. These values define the covariance matrix with positive determinant $4(9)-3^2=27$, so it is positive definite and gives a valid nonsingular bivariate normal distribution.

For a bivariate normal pair, the conditional distribution of one outcome given the other is normal. Its conditional mean is the marginal mean plus the covariance divided by the conditioning outcome’s variance times that outcome’s deviation from its mean. Its conditional variance is the outcome’s variance minus the squared covariance divided by the conditioning outcome’s variance.

In the order $Y_1,Y_2$, this gives

Y_2\mid Y_1=y_1 \sim N\!\left(20+\frac{3}{4}(y_1-10),\;9-\frac{3^2}{4}\right) = N\!\left(20+0.75(y_1-10),\;6.75\right).

At $Y_1=12$, the conditional mean is $20+0.75(12-10)=21.5$, and the conditional variance is $6.75$. In the reverse order,

Y_1\mid Y_2=y_2 \sim N\!\left(10+\frac{3}{9}(y_2-20),\;4-\frac{3^2}{9}\right) = N\!\left(10+\frac{1}{3}(y_2-20),\;3\right).

At $Y_2=23$, the reverse conditional mean is $10+\frac{1}{3}(23-20)=11$, and the conditional variance is 3. The two conditional models have different slopes and residual variances because they condition in opposite directions. Both are conditional distributions derived from the stated same joint normal law.

Assumptions, Diagnostics, and Limitations

The chain rule is a probability identity, not a missingness assumption. It does not require outcomes to be independent, nor does choosing a particular order establish a temporal, causal, or scientific direction. A visit order may be natural for a sequential model, but a factorization written in the reverse order is still a valid representation of the joint law.

A distinction matters when the factors are modeled rather than obtained from a fully specified joint distribution. If every factor is the exact conditional distribution implied by one joint law, each ordering reconstructs that law. But independently imposing a restricted model on each factor in different orders need not produce the same joint distributions. For example, setting a conditional regression slope to zero in one order imposes a particular restriction; setting the reverse-order slope to zero is a different restriction in general. Thus, changing the written order while keeping the same numerical parameters or the same restrictions is not automatically a valid transformation.

Check the following points when working with a factorization:

Conditioning sets: Each factor after the first must condition on exactly the outcomes that precede it in the selected order. Omitting one or adding a later outcome changes the factor.
Support and positivity: A conditional probability $P(A\mid B)$ requires $P(B)>0$. For densities, the relevant conditioning values must lie in the support where the conditional density is defined. In sparse data, some combinations may have zero counts even when their population probabilities are positive.
Normalization: Each conditional distribution must sum or integrate to one over its outcome, for each conditioning value where it is defined. A product of arbitrary terms that fail this condition is not necessarily a valid chain-rule factorization.
Pattern and covariates: In a pattern-specific expression, keep $R=r$ and $X=x$ fixed throughout unless a factor explicitly changes what is conditioned on. Otherwise the product may refer to different distributions.

A frequent mistake is to treat the first visit as the only valid first factor because it happened earlier in time, or to assume $P(Y_2\mid Y_1)=P(Y_1\mid Y_2)$. The correction is to specify the factor order, write the conditioning set for each factor, and verify that the product equals the intended joint distribution. The two conditional probabilities in the binary example were $0.70$ and approximately $0.6087$; they differ because their conditioning events and marginal probabilities differ.

Changing the order is therefore a representation choice, not a method for adding information. It can make a conditional model more convenient to state, but the resulting factors may be harder to estimate in a particular data set, especially when some conditioning histories are rare. It also does not, by itself, determine an observed-data likelihood or show that two restricted statistical models are equivalent. Those questions require additional analysis beyond the chain-rule identity.

Key takeaway: Any permutation of outcomes gives a valid chain-rule factorization of the same joint law when its factors are the corresponding conditional distributions from that law. The order changes which outcomes appear in each conditioning set; it does not make those conditional distributions interchangeable.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

For outcomes $Y_1,Y_2,Y_3$, write the chain-rule factorization in the order $Y_2,Y_1,Y_3$.
Suppose $P(Y_1=1,Y_2=1)=0.24$ and $P(Y_2=1)=0.60$. Find $P(Y_1=1\mid Y_2=1)$.
Does reversing a factorization imply that the visit occurring first causes the later outcome? Explain.
Why can two restricted conditional models, each fitted in a different order, imply different joint distributions?

Solutions. (1) $f(y_1,y_2,y_3)=f(y_2)f(y_1\mid y_2)f(y_3\mid y_2,y_1)$; the order is $Y_2,Y_1,Y_3$. (2) By the definition of conditional probability, $P(Y_1=1\mid Y_2=1)=0.24/0.60=0.40$. (3) No. A factorization order is a representation of the joint law and, by itself, makes no causal claim. (4) Restrictions such as zero conditional regression slopes depend on which outcome is conditioned on. Applying the same-looking restriction in a different order generally imposes a different constraint on the joint distribution.