What Can Observed Data Establish?
A pattern-mixture restriction assigns a distribution to outcomes that are unobserved in a particular missingness pattern by relating them to outcomes in another pattern. Such a restriction can identify quantities that the observed data alone do not determine. That identifying role does not make the restriction a fact that the observed data can verify. A fitted model may reproduce every observed value or distribution perfectly while its assumptions about missing outcomes remain wrong.
The important distinction is between checking an observable implication and validating a restriction about an unobserved outcome. An observable implication is a condition on quantities that are actually observed. If the restriction entails such a condition, observed data can provide evidence against it. But passing that check does not generally establish that the restriction is true: different full-data distributions can agree on all observed data and disagree about the missing outcomes.
For notation, let \(Y_1,\ldots,Y_T\) be outcomes at successive visits, and let \(G\) denote the dropout pattern. For example, \(G=2\) might mean that \(Y_1\) and \(Y_2\) are observed and later outcomes are missing. The observed history before a missing outcome can be denoted by \(H\). A restriction about \(Y_t\) in pattern \(G=g\) might equate its conditional distribution given \(H\) to that in a donor pattern \(G=d\). The left-hand distribution concerns outcomes missing for pattern \(g\); it is not directly estimated from that pattern’s observed records.
Why a Restriction on Missing Outcomes Is Not Directly Testable
Suppose \(Y_2\) is observed in pattern \(G=2\) but missing in pattern \(G=1\), and \(Y_1\) is observed in both. A complete-case restriction might assert
for histories \(h\) represented in both patterns. The right-hand side can be estimated from observed records in \(G=2\). The left-hand side cannot be estimated from \(G=1\), because \(Y_2\) is absent there. No comparison of observed \(Y_2\) values in \(G=1\) and \(G=2\) is available: there are no such \(Y_2\) values in \(G=1\).
The restriction can still be used to identify a distribution, but that distribution is supplied by the assumption, not confirmed by a comparison with missing values. To see why, hold the observed-data distribution fixed and assign different conditional distributions to \(Y_2\) in pattern \(G=1\). All those assignments leave the observed data unchanged. One assignment may satisfy the equality; another may violate it. The observed data cannot distinguish the two.
This argument also applies to available-case and neighboring-case restrictions whenever the recipient pattern lacks the outcome being assigned. The names specify which observed pattern supplies the donor distribution; they do not turn the recipient’s missing distribution into an observed quantity.
Worked Example: A Complete-Case Restriction Is Not Verified by Its Fit
Consider two hypothetical patterns. In \(G=1\), \(Y_1\) is observed and \(Y_2\) is missing. In \(G=2\), both outcomes are observed. Suppose the observed records in \(G=2\) give the following conditional proportions for the binary outcome \(Y_2\).
| Observed history \(Y_1\) | \(G=2\) records | Records with \(Y_2=1\) | Observed proportion |
|---|---|---|---|
| 0 | 10 | 4 | 4/10 = 0.400 |
| 1 | 10 | 6 | 6/10 = 0.600 |
Suppose \(G=1\) contains 8 people with \(Y_1=0\) and 12 with \(Y_1=1\). Both histories occur in the donor pattern, so the basic common-support requirement for this calculation is met. The complete-case restriction assigns the donor conditional probabilities, 0.400 and 0.600, to the missing \(Y_2\) values in \(G=1\). It therefore gives the following expected proportion with \(Y_2=1\) in that pattern:
The value 0.520 is a restriction-based implication for the missing outcomes, not an observed estimate from \(G=1\). For example, the observed records in \(G=1\) are equally compatible with a full-data assignment that gives \(Y_2=1\) to 5 of its 20 people, or to 13 of its 20 people: neither assignment changes the observed \(Y_1\) values or the observed \(G=2\) records. The restriction selects a conditional distribution for the missing outcomes, but these observed data cannot tell us whether it is the right selection.
Observable Implications and Compatibility Checks
Some restrictions, or combinations of restrictions, do entail observable conditions. Those conditions can be examined. For example, if a model asserts equality of the joint distributions of \(Y_1\) and \(Y_2\) between two patterns where both outcomes are observed, then it entails equality of their observed marginal distributions. This implication follows by taking a limit of the joint cumulative distribution function:
Thus, equality of the joint distributions implies equality of the \(Y_1\) marginal distributions. This is a condition on observed data when both outcomes are observed in both patterns. A discrepancy can count against that joint-distribution restriction, subject to sampling uncertainty. It does not follow that a restriction only about a missing \(Y_2\) has the same observable implication. The implication depends on what the restriction actually asserts.
A useful compatibility check can also arise when a set of restrictions assigns the same missing distribution using multiple donors. If one restriction equates \(F_{Y_2\mid H,G=1}\) to \(F_{Y_2\mid H,G=2}\), and another equates it to \(F_{Y_2\mid H,G=3}\), then together they imply
When \(Y_2\) is observed in both donor patterns, this equality concerns observable distributions. Rejecting it challenges the joint package of restrictions. It does not identify which restriction fails, and it does not show that either restriction considered alone is testable.
Worked Example: A Testable Implication of Two Donor Restrictions
Suppose \(Y_2\) is missing in \(G=1\) and observed in donor patterns \(G=2\) and \(G=3\). In the stratum \(Y_1=0\), hypothetical donor data are:
| Pattern | Observed \(Y_2\) values | Observed proportion \(Y_2=1\) |
|---|---|---|
| \(G=2\) | 8 successes among 20 | 8/20 = 0.400 |
| \(G=3\) | 3 successes among 10 | 3/10 = 0.300 |
Assume both restrictions are stated conditional on \(Y_1=0\), and that this history occurs in all three patterns. The support condition is therefore satisfied for this stratum. If the restriction for \(G=1\) equates its missing \(Y_2\) distribution to \(G=2\), and a second restriction equates it to \(G=3\), the two restrictions require the donor probabilities to be equal. Their empirical difference is
The sample proportions differ by 0.100, so the data suggest a potential compatibility problem. The difference is not, by itself, a formal test or proof that the population probabilities differ: sampling variability matters, and a suitable statistical test would account for the sample sizes and the comparison being made. If sufficiently strong evidence rejects donor equality, the joint package of restrictions is incompatible with the observed donor distributions. Even if the proportions were exactly equal, the result would not verify either equality involving \(G=1\), because \(Y_2\) remains unobserved there.
What the Different Restrictions Do—and Do Not—Offer
Complete-case, available-case, and neighboring-case restrictions differ in which patterns supply the donor distribution. Their testability depends on the full set of statements being made, not just on the label.
| Restriction choice | Donor distribution | What can be checked directly? |
|---|---|---|
| Complete-case | Patterns with complete outcome records | Donor fit and covariate or history support; not the missing recipient distribution itself |
| Available-case | Patterns in which the particular outcome is observed | Observed donor distributions and their compatibility; not equality to a recipient distribution that is unobserved |
| Neighboring-case | An adjacent dropout pattern, as specified by the restriction | Whether the neighboring donor has relevant histories and whether observable donor constraints cohere; not the missing outcome equality alone |
There are useful diagnostics, but each answers a narrower question than “is the restriction true?” First, inspect support: if the recipient has histories not represented in the donor pattern, a conditional donor distribution cannot be estimated there without further modeling or extrapolation. Second, inspect the donor data for sparse strata, unusual distributions, and sensitivity to how histories are grouped. Third, check whether multiple restrictions imply observable equalities that the donor data contradict. These checks can reveal a restriction package that is infeasible, poorly supported, or fragile.
They cannot establish that the unobserved recipient outcomes follow the donor distribution. In particular, similar observed histories across patterns do not prove similar missing outcomes. A restriction may also concern a conditional distribution, while a proposed diagnostic compares only marginal distributions. A marginal comparison is not a substitute for a conditional one when the distributions of histories differ.
A Short Justification: Why Observed Fit Cannot Settle the Missing Distribution
Let \(O\) denote everything observed: the pattern \(G\) and the outcomes recorded in that pattern. The observed-data law is the probability distribution \(P(O)\). Let \(Y_{\mathrm{mis}}\) denote the outcomes missing for a given pattern. The observed-data law does not include their realized values. Consequently, two full-data laws \(P_1\) and \(P_2\) can have the same observed-data law while assigning different distributions to \(Y_{\mathrm{mis}}\):
Any diagnostic computed solely from \(O\) has the same distribution under these two full-data laws. It therefore cannot distinguish which missing-outcome distribution is correct. If a restriction implies an additional equality involving only observed quantities, that equality can still be checked; but the check addresses that implication, not every aspect of the full-data law.
Common Pitfalls and Practical Conclusions
- “The model fits the observed data, so the restriction is confirmed.” Many distinct assignments to missing outcomes yield the same observed-data fit. A good fit is not evidence that one such assignment is uniquely correct.
- “The donor and recipient histories look similar, so their missing outcomes must agree.” Similar observed histories may support the plausibility of a restriction, but equality of missing outcomes remains an assumption unless it has an observable implication.
- “A donor comparison rejects one restriction.” A comparison rejects a restriction only if that restriction, alone or together with explicitly stated assumptions, entails the observable equality being tested. Otherwise it is a descriptive diagnostic, not a direct test of the missing-outcome equality.
- “Equal marginal distributions establish conditional equality.” They do not in general. Conditional distributions can differ across histories even when marginal distributions agree; the comparison must match the conditioning in the restriction.
- “An observed difference proves incompatibility.” Sample differences are subject to sampling error. Use an appropriate inferential procedure and distinguish evidence against a population equality from a finite-sample discrepancy.
The main alternative is not to treat restrictions as facts, but to report them as identifying assumptions and examine how conclusions change under plausible alternatives. Observed-data diagnostics can rule out some restriction packages, identify unsupported extrapolation, and reveal conflicts among observable implications. They cannot certify a complete-case, available-case, or neighboring-case equality for outcomes that were never observed in the recipient pattern.
Check Your Understanding
Answer each question, then compare with the solutions.
- In what sense is a restriction about \(Y_2\) in a pattern where \(Y_2\) is missing untestable from observed data?
- Two restrictions equate the same missing-outcome distribution to donor distributions in \(G=2\) and \(G=3\). What observable implication follows if the outcome is observed in both donor patterns?
- Why does equality of a joint distribution imply equality of an observed marginal distribution?
- A donor pattern has no observations for a history found in the recipient pattern. What diagnostic problem does this reveal?
- If observed donor proportions match, has the restriction about the recipient’s missing outcomes been validated? Explain.
Solutions. (1) Different full-data distributions can assign different values or distributions to the missing \(Y_2\) while leaving every observed record unchanged, so the observed data cannot distinguish them. (2) The donor distributions must equal each other on the relevant histories; this is observable and can be assessed with allowance for sampling uncertainty. (3) A marginal is obtained from the joint distribution by taking a limit, for example \(F_{Y_1\mid G=g}(y)=\lim_{z\to\infty}F_{Y_1,Y_2\mid G=g}(y,z)\). (4) The donor data do not support estimating the required conditional distribution at that history; a model or extrapolation is needed. (5) No. Matching donor proportions is compatible with the restriction package but does not reveal the missing recipient outcomes or verify their distribution.