Factorizing Nonmonotone Missingness Patterns

Why Nonmonotone Patterns Need Their Own Representation

In a longitudinal study, a participant may miss one assessment and return for a later one. Such a history is not monotone dropout: the observed visits do not simply stop at a particular time. To describe the joint distribution of outcomes and response histories, we therefore need to represent each distinct history as its own pattern, rather than treating every history as a dropout time.

The central factorization is simple, but its interpretation matters. It separates the probability of a response pattern from the distribution of the full outcome vector among individuals with that pattern. That conditional distribution includes outcomes that were not observed for some individuals. The factorization is a representation of a joint law; it does not, by itself, make those missing outcomes observable or specify how they should be estimated.

The set of possible patterns is \(\mathcal R=\{0,1\}^T\), so there are \(2^T\) patterns before any are ruled out as impossible by the study design. For a pattern \(r\), define its probability as \(\pi_r=P(R=r)\). Define \(f_r(y)\) as the conditional distribution of the complete outcome vector \(Y\) given \(R=r\). When \(Y\) is continuous, \(f_r\) denotes a density; when \(Y\) is discrete, it denotes a probability mass function.

Encoding and Factorizing the Patterns

The ordering of visits must be fixed and used consistently. A one in position \(t\) means that visit \(t\) was observed; a zero means it was not. Patterns \(r=(1,0,1)\) and \(r=(1,1,0)\) are distinct, even though both have two observed visits, because they identify different sets of observed outcomes. In particular, a count of observed visits is not enough to identify a pattern.

For a nonmonotone pattern, the observed components can be separated from the missing components using the positions indicated by \(r\). For example, with three visits and \(r=(1,0,1)\), the observed components are \(Y_1\) and \(Y_3\), while \(Y_2\) is missing. The conditional law \(f_{101}(y_1,y_2,y_3)\) is still a law for all three outcomes; it is not merely a distribution of the two observed values. Its marginal distribution for \((Y_1,Y_3)\) describes the observed outcomes in that pattern, while its distribution for \(Y_2\) involves an unobserved outcome.

Why the Factorization Holds

The factorization follows directly from the definition of conditional probability, not from a missingness assumption. For any event \(A\) involving the complete outcome vector and any pattern \(r\) with positive probability,

If \(Y\) has a density or mass function, the same identity applies to infinitesimal or pointwise outcome events, giving \(p(r,y)=\pi_r f_r(y)\). Summing the joint law over all patterns recovers the marginal outcome distribution:

This is a mixture representation: the full outcome distribution is a weighted average of the pattern-specific distributions, with weights equal to the pattern probabilities. The identity requires only that the joint law exists. If \(\pi_r=0\), the conditional distribution given \(R=r\) is not determined by the observed joint law; the corresponding joint contribution is zero, so no value for \(f_r\) is needed to represent that contribution.

Pattern-Specific Distributions and the Overall Distribution

The pattern-mixture representation makes clear that outcome distributions may differ across response histories. Such differences are part of the joint law and are not ruled out by the factorization. Conversely, the factorization does not claim that response history causes outcome differences. It is a way to organize the joint distribution, not a causal interpretation.

For an outcome component with a finite conditional mean in each pattern, the overall mean is a weighted average of the pattern-specific means. For example, writing \(\mu_{t,r}=E(Y_t\mid R=r)\), the law of total expectation gives

This identity includes patterns in which \(Y_t\) is missing. In those patterns, \(\mu_{t,r}\) is a feature of the full conditional distribution, but it cannot generally be calculated from that pattern’s observed values alone. Any estimate of such a quantity requires information or assumptions beyond the algebraic factorization.

Assumptions, Checks, and Failure Modes

No missing-at-random, missing-completely-at-random, or missing-not-at-random condition is needed for the identity \(p(r,y)=\pi_r f_r(y)\). It is a factorization of a joint probability distribution. The practical requirements are instead that the response pattern be coded unambiguously, the pattern probabilities be valid, and each conditional distribution be defined on the full outcome space. If one imposes a parametric model for \(\pi_r\) or \(f_r\), that model adds assumptions that must be assessed separately.

Several checks help catch errors before the factorization is used:

• Check visit order and coding. Verify that every position corresponds to the same scheduled visit for every participant and that 1 always means observed.
• Check pattern distinctions. Preserve the location of missing visits. Patterns with the same number of observed outcomes are not interchangeable.
• Check normalization. The \(\pi_r\) values must be nonnegative and sum to 1. For each positive-probability pattern, \(f_r\) must integrate or sum to 1 over the full outcome space.
• Check the outcome space. A conditional distribution for \(Y\mid R=r\) must account for every component of \(Y\), not only the components observed under \(r\).
• Check empty patterns. A pattern absent from a finite sample can still have positive probability in the population. Do not conclude its population probability is exactly zero just because no sampled person has that pattern.

A particularly common mistake is to write \(f(y_{\mathrm{obs}}\mid r)\) and treat it as the full pattern-specific outcome distribution. That notation describes only the observed components. The pattern-mixture factorization uses a conditional law for the complete \(Y\); the observed-component distribution is a marginal of that law. Another mistake is to treat a nonmonotone history such as \(101\) as if it were equivalent to dropout after visit 1. That loses the information that visit 3 was observed and assigns outcomes to the wrong observed set.

How This Factorization Relates to Alternatives

The pattern-mixture factorization groups the joint law by response pattern: first specify how probable each pattern is, then specify the outcome distribution within that pattern. The selection factorization instead writes the joint law in terms of an outcome distribution and a response distribution conditional on outcomes. These are alternative factorizations of the same joint law, not automatically different laws. Converting between their components requires care and is a separate calculation.

Sequential response-indicator factorizations can describe response histories through a sequence of conditional probabilities. They may be convenient for modeling visit-by-visit response, but the pattern-mixture representation keeps the full response history explicit and associates each history with its own outcome distribution. For nonmonotone data, this makes it especially important not to collapse histories into a single dropout time.

The representation has limitations. With many visits there are \(2^T\) possible patterns, and some may be rare or absent in the sample. Pattern-specific distributions can then be difficult to estimate. Moreover, the conditional distributions of outcomes missing under a pattern are not directly observed in that pattern. The factorization alone supplies neither a unique estimation procedure nor assumptions that resolve this information gap.