Observable Restrictions From Multiple Missingness Patterns

Why Multiple Observation Patterns Matter

In a longitudinal study, participants may have different subsets of their scheduled outcomes recorded. One participant might contribute all three visits, another only the first two, and another the first and third. These distinct observation patterns determine which comparisons the data can support. A comparison can be checked against the data only when the outcomes involved are actually observed in the relevant patterns.

This distinction matters because a model may make two very different kinds of claims. It may restrict the distribution of an outcome that is observed in multiple patterns, producing an implication that can be examined in the data. Or it may specify how participants in one pattern would have responded at a visit they missed. That second claim concerns unobserved measurements and cannot be checked directly from the observed data alone.

Definition: Let \(Y=(Y_1,Y_2,Y_3)\) denote three scheduled outcomes and \(R=(R_1,R_2,R_3)\) their observation indicators, where \(R_j=1\) means \(Y_j\) is observed and \(R_j=0\) means it is missing. An observation pattern is a value \(r\) of \(R\), such as \(111\) or \(101\). The pattern-specific observed distribution is the distribution of the outcomes that are observed among participants with \(R=r\). An observable restriction is a constraint on the distribution of observed data implied by a specified model or assumption.

For example, participants with pattern \(111\) have all three outcomes observed. Participants with pattern \(101\) have \(Y_1\) and \(Y_3\) observed, but not \(Y_2\). The distribution of \(Y_1\) can be compared across these two patterns, as can the distribution of \(Y_3\). The distribution of \(Y_2\) in pattern \(101\) is not observed, so its equality or difference from the \(Y_2\) distribution in pattern \(111\) cannot be established from those data.

What the Observed Data Contain

For a fixed pattern \(r\), write \(Y_{\mathrm{obs}}(r)\) for the components of \(Y\) observed under that pattern, and \(Y_{\mathrm{mis}}(r)\) for the components missing under it. The observed-data distribution for that pattern concerns \(Y_{\mathrm{obs}}(r)\), not the entire outcome vector.

The distinction follows directly from marginalization. For a discrete outcome vector, the probability of an observed outcome value among pattern-\(r\) participants is obtained by summing the full-data probabilities over all possible values of the missing outcomes:

P\{Y_{\mathrm{obs}}(r)=y_{\mathrm{obs}}\mid R=r\} = \sum_{y_{\mathrm{mis}}} P\{Y_{\mathrm{obs}}(r)=y_{\mathrm{obs}},Y_{\mathrm{mis}}(r)=y_{\mathrm{mis}}\mid R=r\}.

The left side can be estimated from participants in pattern \(r\). The individual terms on the right involving the missing outcomes cannot be separated using that pattern’s observed data alone. Consequently, two full-data models can agree on all observed-data probabilities while assigning different distributions to the missing outcomes.

Some assumptions do imply restrictions on observed comparisons. For instance, missing completely at random (MCAR) asserts that \(R\) is independent of the full outcome vector \(Y\). It therefore implies that the distribution of any outcome component observed in two patterns is the same across those patterns. A discrepancy can count against MCAR, subject to sampling uncertainty. But equality of distributions for shared observed outcomes is only a necessary implication: passing such a check does not prove independence of \(R\) and the full, partly unobserved vector \(Y\).

Worked Example: A Checkable Association Between Pattern and Outcome

Suppose a hypothetical three-visit study has 180 independent participants in two patterns: 100 with pattern \(111\) and 80 with pattern \(110\). A four-category outcome \(Y_1\) is recorded for everyone. The observed counts are as follows.

Pattern	Category A	Category B	Category C	Category D	Total
111	30	20	30	20	100
110	32	8	24	16	80
Total	62	28	54	36	180

A Pearson chi-square test can assess the observed-data null hypothesis that \(Y_1\)'s category distribution is the same in the two patterns. Under this null, each expected count is its row total multiplied by its column total and divided by 180. For pattern \(111\), the expected counts are \(100(62,28,54,36)/180=(34.444,15.556,30,20)\). For pattern \(110\), they are \(80(62,28,54,36)/180=(27.556,12.444,24,16)\). Each expected count exceeds 5, and the participants are independent, so the usual large-sample approximation is reasonable for this hypothetical comparison.

The four nonzero cell contributions are:

\begin{aligned} \frac{(30-34.444)^2}{34.444}&=0.57348, & \frac{(20-15.556)^2}{15.556}&=1.26984,\\ \frac{(32-27.556)^2}{27.556}&=0.71685, & \frac{(8-12.444)^2}{12.444}&=1.58730. \end{aligned}

The two remaining categories contribute zero because their observed and expected counts match. Thus \(X^2=0.57348+1.26984+0.71685+1.58730=4.14747\). The degrees of freedom are \((2-1)(4-1)=3\), giving a \(p\)-value of approximately \(0.246\). At a conventional significance level such as \(\alpha=0.05\), these data do not provide strong evidence against equality of the observed \(Y_1\) distributions across the two patterns.

This conclusion concerns only the distribution of \(Y_1\), which is observed in both patterns. It neither establishes MCAR nor determines the distribution of \(Y_3\) among participants with pattern \(110\), for whom \(Y_3\) is missing.

Comparing a Shared Outcome With a Two-Proportion Test

When the shared observed outcome is binary, the same question can be expressed as a comparison of two proportions. The test is meaningful only for an outcome observed in both patterns. For the pooled two-proportion \(z\) test, the null hypothesis is equality of the population proportions in the two groups; the pooled estimate is used to calculate the standard error under that null. Independent participants and sufficiently large expected success and failure counts are standard conditions for the normal approximation.

Worked Example: Comparing a Binary Outcome Across Two Patterns

In a separate hypothetical study, \(Y_1\) is binary and observed for 100 participants with pattern \(111\) and 80 with pattern \(110\). There are 60 positive outcomes in the first pattern and 32 in the second. The estimated proportions are \(60/100=0.600\) and \(32/80=0.400\), a difference of \(0.200\).

Under \(H_0:p_{111}=p_{110}\), the pooled estimate is \(\hat p=(60+32)/(100+80)=92/180=0.5111\). The standard error under the null is:

SE=\sqrt{\hat p(1-\hat p)\left(\frac{1}{100}+\frac{1}{80}\right)} =\sqrt{0.5111(0.4889)(0.02250)} \approx 0.07498.

Therefore \(z=(0.600-0.400)/0.07498\approx2.667\). The two-sided \(p\)-value is approximately \(0.007646\). The expected success and failure counts under the pooled estimate are \(51.11\) and \(48.89\) for the first group, and \(40.89\) and \(39.11\) for the second; all are large enough for the normal approximation. With independent participants, the data provide evidence that the observed \(Y_1\) proportions differ between these two patterns.

This difference is an observable association between pattern membership and \(Y_1\). It is not, by itself, evidence about the unobserved \(Y_2\) or \(Y_3\) values in pattern \(110\). Nor does it establish why participants have different patterns.

Observed Comparisons Do Not Reveal Missing Outcomes

A pattern-specific comparison is available only for outcomes observed in the patterns being compared. The next example separates a genuine observed comparison from a different, unobserved one. The counts are hypothetical and are not intended to describe a real study.

Worked Example: Shared and Unshared Visits

Suppose 40 participants have pattern \(111\) and 60 have pattern \(101\). The binary outcome \(Y_3\) is positive for 18 participants in each group. The observed proportions are \(18/40=0.450\) in pattern \(111\) and \(18/60=0.300\) in pattern \(101\); their observed difference is \(0.150\), or 15 percentage points. Since both patterns record \(Y_3\), this comparison is available in the observed data.

The same patterns do not both record \(Y_2\): it is observed in pattern \(111\) and missing in pattern \(101\). Thus the data provide no count of positive \(Y_2\) outcomes in pattern \(101\). For example, asserting that its \(Y_2\) positive proportion is \(0.450\), the observed proportion in pattern \(111\), would be an assumption about missing measurements, not a result of the displayed data. A different assumed proportion could also be compatible with those same observed counts.

The distinction can be summarized as follows. A model may impose equality of observed distributions in two patterns, or independence between pattern membership and a shared observed outcome. Those claims yield restrictions that can be checked, with sampling uncertainty, using observed data. A model may instead impose equality between an observed distribution in one pattern and a distribution of outcomes missing in another. The latter comparison has no direct observed-data test.

The testable implication is often a necessary but not sufficient consequence of a broader assumption. For example, if the MCAR implication of equal shared-outcome distributions is rejected, MCAR is inconsistent with the data, subject to the test’s assumptions. If it is not rejected, the result does not confirm MCAR: unobserved components may still be associated with the pattern in ways that the observed comparisons cannot detect. An assumption such as missing at random (MAR) is also not established simply because observed pattern comparisons look similar; such comparisons are not a general test of how missing outcomes relate to their missingness.

Diagnostics, Alternatives, and Limitations

Before interpreting an observed restriction, check that it is genuinely observable and that the comparison is supported by the study design. In particular:

Check shared observation. List which outcomes each pattern records. Do not compare pattern-specific distributions of an outcome that is missing in one of the groups.
Check overlap and sample size. A very small or empty pattern may leave a comparison unstable or impossible. Sparse expected counts can make a chi-square approximation unreliable; combine categories only when substantively defensible, or use an appropriate alternative such as an exact test.
Check measurement comparability. The outcome must have the same definition and scale across patterns. Different instruments, timing, or coding can create apparent differences unrelated to pattern membership.
Check dependence and multiplicity. Standard tests assume independent participants. Repeated measurements within a participant do not create additional independent participants. Testing many outcomes and patterns also increases the chance of chance findings, so report the comparisons conducted and interpret isolated small \(p\)-values cautiously.
Separate statistical evidence from model implications. A small \(p\)-value can contradict a particular observable restriction. A large \(p\)-value is not proof that the restriction holds, and neither result directly validates a distribution assigned to an unobserved outcome.

These checks complement, but do not replace, identification analysis. Pattern-mixture models describe outcome distributions by pattern and must specify or restrict distributions for missing components to make full-data claims. Selection models instead describe the relationship between outcomes and the observation process. These are alternative representations of the missing-data problem, but neither representation makes a claim about unobserved outcomes empirically checkable merely by fitting observed data well. The earlier material on point identification versus partial identification provides the relevant distinction: observed restrictions may rule out some models, while multiple full-data distributions can remain compatible with the same observed distribution.

A common mistake is to reason, “The observed outcome distributions are similar across patterns, so participants with missing outcomes must have had the same distribution.” The correction is that similarity concerns only components recorded in the groups compared; it does not reveal a component that is never recorded in one of those groups. The reverse mistake is also possible: a difference in a shared observed outcome may be treated as a direct estimate of the missing outcomes’ distribution. It is not. It is evidence about an observed association, whose implications for missing values depend on additional assumptions.

Key takeaway: Multiple patterns create useful observable checks wherever they share measured outcomes. Such checks can challenge specific restrictions, but they cannot by themselves determine distributions of measurements that are absent from a pattern.

Check Your Understanding

Answer each question, then compare your reasoning with the brief solutions.

In patterns \(111\) and \(101\), which outcomes can be compared directly across patterns?
A test of a shared outcome gives \(p=0.31\). Does this establish that the outcome distributions are equal? Why or why not?
Suppose MCAR implies equal distributions of a shared observed outcome across two patterns, and a well-specified test rejects that equality. What conclusion is justified, and what conclusion is not?
In pattern \(110\), \(Y_3\) is missing. Can observed data establish that its \(Y_3\) distribution equals the distribution in pattern \(111\)? What additional ingredient would be needed to make that claim?
Why can a model fit the observed data well while still assigning an incorrect distribution to missing outcomes?

Solutions. (1) \(Y_1\) and \(Y_3\) are observed in both patterns; \(Y_2\) is not observed in pattern \(101\). (2) No. The test fails to find strong evidence against equality; it does not prove equality or verify assumptions about unobserved outcomes. (3) The data are evidence against that observable implication and hence against MCAR, provided the test conditions are appropriate. Rejection does not identify the missing-outcome distribution. (4) No; \(Y_3\) is absent in pattern \(110\). Equality would require an additional assumption or information, not just the observed data shown. (5) Different full-data distributions can yield the same observed-data distribution after missing components are marginalized out, as in the summation identity above.