Why Factor the Joint Distribution?
A dataset with missing values contains information about two things: the values that could have been recorded, and which values were actually recorded. A useful starting point is to describe these together. The factorization separates the distribution of the complete data from the process that determines the response pattern. This makes the roles of the two parts explicit without, by itself, deciding why values are missing.
Use \(Y\) for the complete data that would be available if every relevant value were recorded, and \(R\) for the response indicators. For a single variable, \(R=1\) means the value is observed and \(R=0\) means it is missing. With several variables or units, \(R\) is a vector or pattern of such indicators. The observed values are denoted by \(Y_{\mathrm{obs}}\); values not observed are \(Y_{\mathrm{mis}}\).
The first factor describes how the complete-data values vary. The second describes the probability of each response pattern for a specified complete-data value. In particular, the mechanism is conditional on \(Y\), not just on the values that happened to be observed. Whether it can be simplified to depend on observed values is a separate question, taken up when missingness mechanisms are classified.
What the Factorization Does—and Does Not—Assume
The factorization follows from the probability chain rule. For any joint distribution of \(Y\) and \(R\), and for values where the conditional distribution is defined, the joint probability is the probability of \(Y=y\) multiplied by the conditional probability of \(R=r\) given \(Y=y\). Thus it is a way to express the joint distribution, not an independence assumption.
For continuous data, the same identity is written using a joint density and a conditional probability mass or density for \(R\). With model parameters, one can write \(f(y\mid\theta)g(r\mid y,\psi)\) when the model is parameterized in those two parts. This notation does not imply that \(\theta\) and \(\psi\) are independent, or that the two parts can be estimated separately without further assumptions. Those are additional modeling or inferential claims.
If the units are independent and each unit’s response depends only on its own complete data, then a sample-level distribution can be written as a product:
The product requires those independence conditions. Repeated observations, clustering, or a response process linking units may require a joint data model or a joint conditional mechanism instead. The basic factorization still applies to the full vectors: \(p(y,r)=p(y)p(r\mid y)\).
Worked Example: A Binary Value and Its Response Indicator
Suppose a hypothetical complete-data variable \(Y\) is binary, with \(P(Y=1)=0.40\) and \(P(Y=0)=0.60\). Assume the response mechanism has \(P(R=1\mid Y=1)=0.80\) and \(P(R=1\mid Y=0)=0.50\). Find the four joint probabilities.
The data model supplies \(P(Y=y)\); the mechanism supplies \(P(R=r\mid Y=y)\). For \(Y=1,R=1\), $$ P(Y=1,R=1)=P(Y=1)P(R=1\mid Y=1)=0.40(0.80)=0.32. $$
For \(Y=1,R=0\), the conditional probability of nonresponse is \(1-0.80=0.20\), so the joint probability is \(0.40(0.20)=0.08\). For \(Y=0,R=1\), it is \(0.60(0.50)=0.30\). Finally, \(P(R=0\mid Y=0)=1-0.50=0.50\), giving \(0.60(0.50)=0.30\).
| Complete value | Response pattern | Calculation | Joint probability |
|---|---|---|---|
| \(Y=1\) | \(R=1\) | \(0.40(0.80)\) | 0.32 |
| \(Y=1\) | \(R=0\) | \(0.40(0.20)\) | 0.08 |
| \(Y=0\) | \(R=1\) | \(0.60(0.50)\) | 0.30 |
| \(Y=0\) | \(R=0\) | \(0.60(0.50)\) | 0.30 |
The four probabilities sum to \(0.32+0.08+0.30+0.30=1.00\), as required. The overall response probability is \(0.32+0.30=0.62\). This is a joint distribution with a valid data model and mechanism; the calculation does not require calling the mechanism by a missingness category.
From the Joint Distribution to Observed-Data Probabilities
When \(R=0\), the corresponding value of \(Y\) is not available in the observed record. A probability for the observed record must therefore combine the joint probabilities over all compatible missing values. For discrete missing values this means summing; for continuous missing values it means integrating. This operation is sometimes called marginalizing over the missing data.
where the sum is replaced by an integral for continuous \(Y_{\mathrm{mis}}\). The expression keeps both components: the data model weights possible complete values, and the mechanism weights the probability of the observed response pattern at each such value.
Worked Example: Summing Over an Unobserved Value
Use the hypothetical binary model above. Consider a record with \(R=0\), so \(Y\) is not observed. Find the probability of that observed record and the conditional probability that \(Y=1\) among records with \(R=0\).
There are two complete values compatible with \(R=0\). Their joint probabilities are \(P(Y=1,R=0)=0.08\) and \(P(Y=0,R=0)=0.30\). Summing over the unobserved value gives $$ P(R=0)=0.08+0.30=0.38. $$
Among records with \(R=0\), the conditional probability of \(Y=1\) is the joint probability for \(Y=1,R=0\) divided by the total probability of \(R=0\): $$ P(Y=1\mid R=0)=\frac{P(Y=1,R=0)}{P(R=0)} =\frac{0.08}{0.38}=0.2105. $$
Thus the observed record \(R=0\) has probability 0.38, and in this hypothetical model about 21.1% of the nonresponders have \(Y=1\). The latter is a model-based conditional probability; it is not obtained by reading \(Y\) from those records, because their \(Y\) values are missing.
Worked Example: Two Data Variables and One Response Indicator
Consider hypothetical binary variables \(X\) and \(Y\), where \(X\) is observed for everyone and \(R\) indicates whether \(Y\) is observed. Let \(P(X=0)=P(X=1)=0.50\), \(P(Y=1\mid X=0)=0.20\), and \(P(Y=1\mid X=1)=0.70\). Suppose the mechanism has \(P(R=1\mid X=0,Y=y)=0.90\) for either value of \(y\), while \(P(R=1\mid X=1,Y=0)=0.60\) and \(P(R=1\mid X=1,Y=1)=0.40\). Find the joint probability of \(X=1,Y=1,R=1\), and the probability of \(X=1,R=1\).
The data model can be factored as \(P(X)P(Y\mid X)\); this is a conditional factorization and does not assert that \(X\) and \(Y\) are independent. For the first requested probability, the three factors are $$ P(X=1,Y=1,R=1) =P(X=1)P(Y=1\mid X=1)P(R=1\mid X=1,Y=1) =0.50(0.70)(0.40)=0.14. $$
For \(X=1,R=1\), \(Y\) is observed when \(R=1\), but this event does not specify its value. Add the two possible joint probabilities. For \(Y=0\), \(P(Y=0\mid X=1)=1-0.70=0.30\), and its contribution is \(0.50(0.30)(0.60)=0.09\). The \(Y=1\) contribution is 0.14, so $$ P(X=1,R=1)=0.09+0.14=0.23. $$
The conditional response probability differs across the specified complete-data combinations. This example illustrates why the mechanism must be stated with its conditioning variables; it is not enough to report a single overall response rate.
Checks, Alternatives, and Common Pitfalls
A factorized model is useful only if each component is a valid distribution and their product describes the intended joint process. For every \(y\), the response probabilities must satisfy $$ \sum_r g(r\mid y)=1, $$ where the sum includes all possible response patterns. The data model must likewise sum or integrate to one over its support. In the examples, checking that the joint probabilities sum to one provides an additional arithmetic check.
The factorization is not the only way to specify a joint distribution. One could list or model \(p(y,r)\) directly. The direct form can be convenient for a small finite table, but it may obscure which probabilities describe the complete data and which describe response behavior. The factorized form makes that distinction explicit and is readily extended to many response patterns. Neither representation is automatically correct: the modeling choices must fit the setting and respect its support.
A frequent mistake is to treat \(f(y)\) as though it were the distribution of the observed values alone. It is the distribution of the complete data, including values that are not recorded. Another mistake is to omit \(g(r\mid y)\) and conclude that missingness has no role in the joint law. A third is to infer independence from the product notation. In \(f(y)g(r\mid y)\), the second factor is conditional on \(y\); dependence between \(Y\) and \(R\) is allowed.
A practical diagnostic is to write down, for each response pattern, exactly which values are observed and which are being summed or integrated out. Then check that the mechanism covers every possible pattern, that its conditional probabilities add to one for each complete-data value, and that the resulting joint probabilities are nonnegative and normalized. If any of these checks fail, the issue may be an arithmetic error, a response-indicator coding error, or an incomplete model specification.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- If \(P(Y=1)=0.30\) and \(P(R=1\mid Y=1)=0.70\), what is \(P(Y=1,R=1)\)?
Solution: Multiply the data-model probability by the conditional mechanism probability: \(0.30(0.70)=0.21\). - In \(p(y,r)=f(y)g(r\mid y)\), which factor describes the response process, and what is it conditioned on?
Solution: \(g(r\mid y)\) describes the response process; it is conditioned on the complete-data value \(y\). - For a discrete variable with \(P(Y=1,R=0)=0.12\) and \(P(Y=0,R=0)=0.28\), find \(P(R=0)\).
Solution: Sum over the unobserved value: \(P(R=0)=0.12+0.28=0.40\). - Does writing \(p(y,r)=f(y)g(r\mid y)\) imply that \(Y\) and \(R\) are independent? Explain briefly.
Solution: No. The mechanism is conditional on \(Y\), so it may vary with \(y\). Independence would require the additional condition \(g(r\mid y)=g(r)\) for all relevant \(y\).