Why Compare Complete-Data Laws Through What Is Recorded?
A complete-data law describes the joint distribution of all outcomes, including outcomes that may not be recorded, and the missingness pattern. In practice, only the recorded outcomes and the pattern are available for analysis. Consequently, two different complete-data laws may make exactly the same probability statements about everything that could be observed. This is observational equivalence.
The key comparison is not whether two models assign the same probabilities to every complete record. It is whether, after missing outcomes are integrated or summed out, they assign the same probabilities to every possible observed record. The earlier material on Observed-Data Distributions and Identification Targets introduced the observed record as the object whose distribution is available from data; here we formalize when distinct complete-data laws induce the same such distribution.
This definition concerns equality of distributions, not equality of the outcomes in a particular sample. For equivalence, every possible observed pattern and every possible recorded outcome value must have the same probability under the two laws. A close fit to one realized data set, or equal likelihood values for that data set, is not by itself enough.
The Observed-Data Comparison Rule
Suppose the complete-data law has density or mass function \(p(y,r)\). For a fixed pattern \(r\), partition \(y\) into the recorded components \(y_{\mathrm{obs}(r)}\) and the missing components \(y_{\mathrm{mis}(r)}\). The observed-data law for that pattern is obtained by summing or integrating over the missing components:
For discrete outcomes, the integral is a sum over all possible missing values. The pattern \(r\) remains part of the observed record, so this expression includes both the probability of the pattern and the distribution of the recorded outcomes within it.
The criterion follows directly from the definition of the observed record. To find the probability or density of a recorded value under a given pattern, all complete records that could have produced it must be counted. Those records differ only in values hidden by that pattern, so their probabilities are summed or integrated. Equality of the resulting quantities for every pattern and recorded value is equality of the entire observed-data distribution.
This marginalization is a many-to-one operation: different distributions over complete records can give the same totals after hidden values are combined. That is the basic mechanism behind observational equivalence. If no outcomes are ever missing, the observed record contains the full outcome vector, and this particular loss of information does not occur.
Worked Examples
Worked Example: Two Distinct Laws With the Same Observed Distribution
Consider two binary outcomes, \(Y_1,Y_2\in\{0,1\}\), with monotone missingness patterns \(R=11\), \(R=10\), and \(R=00\). Pattern \(11\) records both outcomes; pattern \(10\) records \(Y_1\) but not \(Y_2\); pattern \(00\) records neither. The following hypothetical laws specify unconditional probabilities for complete records.
| Pattern and complete outcomes | Law A | Law B |
|---|---|---|
| \(R=11,\ (Y_1,Y_2)=(0,0)\) | 0.125 | 0.125 |
| \(R=11,\ (Y_1,Y_2)=(0,1)\) | 0.125 | 0.125 |
| \(R=11,\ (Y_1,Y_2)=(1,0)\) | 0.125 | 0.125 |
| \(R=11,\ (Y_1,Y_2)=(1,1)\) | 0.125 | 0.125 |
| \(R=10,\ (Y_1,Y_2)=(0,0)\) | 0.150 | 0 |
| \(R=10,\ (Y_1,Y_2)=(0,1)\) | 0 | 0.150 |
| \(R=10,\ (Y_1,Y_2)=(1,0)\) | 0 | 0.150 |
| \(R=10,\ (Y_1,Y_2)=(1,1)\) | 0.150 | 0 |
| \(R=00,\ (Y_1,Y_2)=(0,0)\) | 0.200 | 0 |
| \(R=00,\ (Y_1,Y_2)=(1,1)\) | 0 | 0.200 |
Each column sums to \(0.500+0.300+0.200=1.000\), so each specifies a probability law. Under either law, pattern \(11\) has probability \(0.500\), and each of its four recorded outcome pairs has probability \(0.125\). Under pattern \(10\), the observed record \(Y_1=0\) has probability \(0.150\), and \(Y_1=1\) has probability \(0.150\); the different values of \(Y_2\) are hidden. Pattern \(00\) has probability \(0.200\) under both laws and records no outcomes. Thus every observed record has the same probability under A and B.
The complete-data laws are nevertheless distinct. For instance, under pattern \(10\), Law A places mass \(0.150\) on \((Y_1,Y_2)=(0,0)\), while Law B places that mass on \((0,1)\). They also assign the pattern \(00\) mass to different outcome pairs. As an additional check, the complete-data mean of \(Y_2\) differs:
In the first term, the four equally weighted \(R=11\) cells contribute \(0.250\) to the mean under either law. The \(R=10\) cells contribute \(0.150\) under both, while the hidden \(R=00\) outcomes contribute \(0\) under A and \(0.200\) under B. The laws disagree about complete outcomes even though no comparison based solely on the observed record can distinguish them.
Worked Example: A Difference in Pattern Probability Rules Out Equivalence
Use Law A from the preceding example as one candidate. Consider a second hypothetical law, Law C, with probability \(0.1125\) for each of the four \(R=11\) outcome pairs, probability \(0.175\) for each of \((R=10,Y_1=0,Y_2=0)\) and \((R=10,Y_1=1,Y_2=1)\), and probability \(0.200\) for \((R=00,Y_1=0,Y_2=0)\). All other complete records have probability zero.
The probabilities sum to \(4(0.1125)+2(0.175)+0.200=0.450+0.350+0.200=1.000\). But the probability of observing pattern \(11\) is \(0.450\) under C and \(0.500\) under A. Since \(R\) is always observed, this difference is directly visible in the observed-data distribution. Therefore A and C are not observationally equivalent; comparing outcome distributions within patterns is unnecessary once this observable discrepancy is found.
Worked Example: Comparing Two Completions of a Three-Category Outcome
Suppose one outcome \(Y\) has categories \(a,b,c\), and \(R=1\) means it is observed. A hypothetical observed-data distribution is \(P(R=1,Y=a)=0.24\), \(P(R=1,Y=b)=0.18\), \(P(R=1,Y=c)=0.18\), and \(P(R=0)=0.40\). Two complete-data laws agree on all the \(R=1\) probabilities. Under Law D, allocate the missing-pattern mass as \(0.20,0.10,0.10\) to \(Y=a,b,c\); under Law E, allocate it as \(0.05,0.15,0.20\).
Both allocations sum to \(0.40\), and neither changes any observed \(R=1\) probability or the probability of \(R=0\). Hence both laws induce exactly the stated observed-data distribution. Yet their complete-data category probabilities differ. Under D they are \(P(Y=a)=0.24+0.20=0.44\), \(P(Y=b)=0.18+0.10=0.28\), and \(P(Y=c)=0.18+0.10=0.28\). Under E they are \(0.24+0.05=0.29\), \(0.18+0.15=0.33\), and \(0.18+0.20=0.38\), respectively. These are different complete-data laws with identical distributions for every observed record.
Assumptions, Diagnostics, and Common Errors
The comparison rule assumes that the observed record is defined consistently under both candidate laws: the same outcomes are measured, the same pattern labels are used, and a recorded outcome equals its corresponding component of the complete outcome. It also assumes the laws refer to the same population and sampling unit. The rule itself does not require a particular missingness mechanism, a parametric model, or independence between outcomes and missingness.
In an applied model, a useful diagnostic is to compare the implied observed-data distributions pattern by pattern. Check the pattern probabilities first, then the distribution of the recorded outcomes within each pattern. For discrete outcomes this means checking every cell; for continuous outcomes it means comparing the corresponding densities or probability measures, not just a few summaries. A difference in any observable probability establishes non-equivalence. Agreement on selected means, proportions, or fitted values does not establish equivalence unless it implies agreement of the full distributions.
- Mistake: treating equal likelihood for one sample as equivalence. Two laws can assign the same likelihood to the observed sample but different probabilities to other possible observed records. Equivalence requires equality of the full observed-data distributions.
- Mistake: comparing only complete records. Equal distributions among units with \(R=11\) do not suffice if a pattern probability or another pattern-specific recorded distribution differs.
- Mistake: assuming distinct complete-data laws must be distinguishable. Hidden outcomes are summed or integrated out. As the first and third examples show, that operation can conceal differences in the complete-data law.
Observational equivalence describes what the observed-data distribution can distinguish; it does not say that two complete-data laws are scientifically interchangeable or equally plausible. A model restriction may rule out one law, but the observed distribution alone cannot confirm an untestable restriction merely because the restricted model fits the recorded data. Conversely, an observable discrepancy can refute equivalence without resolving which complete-data law is appropriate.
Check Your Understanding
Answer each question, then check the solutions below.
- State the criterion for two complete-data laws to be observationally equivalent.
- One law assigns probability \(0.31\) to \(R=1\), and another assigns probability \(0.34\). Can the laws be observationally equivalent if \(R\) is observed?
- If no outcome is ever missing and the observed-data distributions agree, can the complete-data laws still differ on the outcome distribution?
- Under \(R=0\), suppose two laws allocate the same probability \(0.25\) differently across hidden outcome categories, while agreeing on all observed-pattern probabilities. What can be concluded about their observational equivalence?
Solutions. (1) For every pattern and recorded outcome value, the pattern-specific observed-data distributions must agree, with almost-everywhere equality for continuous outcomes. (2) No. The pattern probability itself is observable and differs by \(0.03\). (3) No, provided the full outcome vector is recorded: the observed record then contains the complete outcome, so equality of its distribution is equality of the complete-data outcome distribution. (4) The laws can be observationally equivalent; if those categories are always hidden under \(R=0\), their different allocation does not change any observed-record probability.