Monotone And Nonmonotone Patterns Versus Mechanisms

Patterns Describe What Is Missing; Mechanisms Describe How

A data set can have gaps arranged in a recognizable structure, but the arrangement alone does not explain why values are absent. In multiple imputation, this distinction matters: pattern structure describes which variables are observed together, while a missingness mechanism describes how the probability of each response pattern relates to the complete data. These are separate questions, and answering one does not automatically answer the other.

Let \(Y=(Y_1,\ldots,Y_p)\) denote the complete-data vector for one unit. For each component, define \(R_j=1\) if \(Y_j\) is observed and \(R_j=0\) if it is missing. The vector \(R=(R_1,\ldots,R_p)\) is the response pattern. The observed data include the values with \(R_j=1\) and the pattern \(R\); values with \(R_j=0\) are not observed.

Definition: A missingness pattern is monotone relative to a specified variable order if every missing component is followed only by missing components in that order. A collection of realized patterns is monotone if there is one common variable order under which every pattern in the collection is monotone. A collection is nonmonotone if no single order works for all its patterns.

For example, in the order \(Y_1,Y_2,Y_3\), the pattern \((1,0,0)\) is monotone: once a missing value appears, all later components are also missing. The pattern \((1,0,1)\) is nonmonotone in that order because \(Y_3\) is observed after \(Y_2\) is missing. Monotonicity is always assessed relative to an order; for a collection of patterns, that order must be shared.

Mechanisms Are Conditional Probability Statements

A missingness mechanism is described by the conditional probabilities \(P(R=r\mid Y=y)\) for possible response patterns \(r\) and complete-data values \(y\). For each fixed \(y\), these probabilities must be nonnegative and sum to 1 over all \(2^p\) possible patterns. The standard mechanism labels describe which components of \(y\) these probabilities may depend on.

Definition: A mechanism is MCAR (missing completely at random) if \(P(R=r\mid Y=y)\) does not depend on \(y\) for any pattern \(r\). It is MAR (missing at random) if, for each pattern \(r\), \(P(R=r\mid Y=y)\) depends only on the values observed under \(r\). It is MNAR (missing not at random) when the mechanism is not MAR for the variables and conditioning set under consideration.

The observed values relevant to MAR can differ by pattern: values that are observed under one pattern may be missing under another. MCAR is a special case of MAR because a probability that depends on no data values also depends only on the values observed under a pattern. Whether a mechanism is MAR or MNAR is a statement about the complete-data conditional probabilities, not a label that can be read directly from a pattern’s shape.

The MCAR condition also imposes a basic probability check. If the pattern probabilities are constant in \(y\), they still must make a probability distribution:

\sum_{r\in\{0,1\}^p} P(R=r\mid Y=y)=1

For instance, assigning probability \(2^{-p}\) to each of the \(2^p\) patterns gives a valid MCAR mechanism, since \(2^p(2^{-p})=1\). Assigning the same arbitrary probability to every pattern is not valid unless the probabilities sum to 1.

Worked Examples

Worked Example: Checking a Common Monotone Order

Suppose three variables are recorded in the specified order \(Y_1,Y_2,Y_3\), and the realized patterns and counts are \((1,1,1)\) for 8 units, \((1,1,0)\) for 3 units, \((1,0,0)\) for 2 units, and \((0,0,0)\) for 1 unit. Is this collection monotone in the specified order?

For \((1,1,0)\), the only missing component is last, so it is followed by no observed components. For \((1,0,0)\), the missing \(Y_2\) is followed by missing \(Y_3\). For \((0,0,0)\), both later components after each missing component are also missing. The complete pattern \((1,1,1)\) satisfies the condition because it has no missing component.

Therefore all four realized patterns are monotone in the common order \(Y_1,Y_2,Y_3\). The total number of units is \(8+3+2+1=14\); the pattern counts describe the arrangement and frequency of missing values, not the mechanism that generated them.

Worked Example: Why One Order Must Work for the Whole Collection

For two variables \(Y_1,Y_2\), suppose the realized patterns are \((0,1)\) and \((1,0)\). Can the collection be made monotone by choosing a variable order?

In the order \(Y_1,Y_2\), the pattern \((0,1)\) is not monotone: \(Y_1\) is missing but the later \(Y_2\) is observed. Reversing the order to \(Y_2,Y_1\) makes \((0,1)\) monotone, but then \((1,0)\) has an observed \(Y_2\) followed by missing \(Y_1\), so it is not monotone. Thus no single order makes both patterns monotone, and the collection is nonmonotone.

Each pattern separately can be made monotone by a suitable order. That does not make the collection monotone: one common order is required. This distinction prevents a vacuous classification in which the order is changed from pattern to pattern.

Worked Example: A Valid MCAR Mechanism

For \(p=2\) variables, suppose the conditional probabilities of the four patterns are each \(1/4\), for every possible \(y\). Verify validity and classify the mechanism.

There are \(2^2=4\) patterns, so for any fixed complete-data value \(y\),

\sum_r P(R=r\mid Y=y) =\frac14+\frac14+\frac14+\frac14=1.

All probabilities are nonnegative and sum to 1. Since each is constant in \(y\), the mechanism is MCAR and hence also MAR. This conclusion concerns the specified conditional probability model. It does not follow merely from seeing all four patterns, or from seeing similar counts for them in a sample.

Worked Example: A MAR Mechanism with a Monotone Pattern Structure

Suppose \(Y_1\) is always observed and \(Y_2\) may be missing. Let \(Y_1\) be binary and define \(q(0)=0.9\) and \(q(1)=0.6\). Set

P(R=(1,1)\mid Y=y)=q(y_1),\qquad P(R=(1,0)\mid Y=y)=1-q(y_1).

The two probabilities are nonnegative and sum to 1 for either value of \(y_1\): when \(y_1=0\), they are \(0.9\) and \(0.1\); when \(y_1=1\), they are \(0.6\) and \(0.4\). The other two patterns, \((0,1)\) and \((0,0)\), have probability zero. Every possible realized pattern is monotone in the order \(Y_1,Y_2\), since \(Y_1\) is observed and any missing \(Y_2\) is last.

The mechanism is MAR: \(Y_1\) is observed in both possible patterns, so dependence on \(y_1\) is dependence only on an observed value. It is not MCAR because, for example, \(P(R=(1,1)\mid Y_1=0)=0.9\), whereas \(P(R=(1,1)\mid Y_1=1)=0.6\). This example shows that a monotone pattern structure does not imply MCAR.

Worked Example: An MNAR Mechanism Can Have the Same Pattern Shape

Now consider a single binary variable \(Y_1\), with response probabilities \(P(R_1=1\mid Y_1=0)=0.8\) and \(P(R_1=1\mid Y_1=1)=0.4\). What is the mechanism, and what can be said about pattern monotonicity?

For \(Y_1=0\), the missing probability is \(P(R_1=0\mid Y_1=0)=1-0.8=0.2\). For \(Y_1=1\), it is \(P(R_1=0\mid Y_1=1)=1-0.4=0.6\). Each pair of response and missing probabilities sums to 1. Because the probability of response changes with \(Y_1\), and \(Y_1\) is unavailable when \(R_1=0\), the mechanism is MNAR. With only one variable, either pattern \((1)\) or \((0)\) is monotone under the sole variable order.

Thus, monotonicity does not imply MAR: this mechanism has only monotone patterns but is MNAR. Nor does a monotone arrangement identify the mechanism.

Why the Distinction Matters

Pattern structure is directly summarized from the observed response indicators. A pattern table can show whether values tend to be missing in a sequential way, which variables are often observed together, and how many units have each combination. Mechanism labels, by contrast, make claims about conditional probabilities given complete-data values, including values that are missing. The observed pattern table alone does not reveal those probabilities.

The distinction is useful in practice for two reasons. First, monotone structure can affect which computational strategies are convenient, but it is not itself evidence that MCAR or MAR holds. Second, the appropriateness of an imputation model depends on the assumed mechanism and included variables, not just on whether the pattern table looks simple. An analyst should describe the patterns and state the mechanism assumptions separately.

A basic justification for keeping the two questions separate is that pattern shape concerns which values of \(R\) occur, while mechanism classification concerns how their conditional probabilities vary with \(y\). For example, a pattern with a missing \(Y_2\) may occur under MCAR with a constant probability, under MAR with a probability depending on an observed \(Y_1\), or under MNAR with a probability depending on the missing \(Y_2\). The pattern itself does not encode which conditional dependence generated it.

Diagnostics, Comparisons, and Limitations

Begin by tabulating realized response patterns and recording the variable order used to assess monotonicity. For a proposed common order, check every pattern: whenever \(R_j=0\), verify that \(R_k=0\) for every later position \(k>j\). If any pattern violates this condition, that order does not establish monotonicity. If the question is whether some order works, search for one order that satisfies the condition for the entire collection, rather than selecting a different order for each row.

Mechanism assessment requires a different line of reasoning. State the variables in the conditioning set, write or describe \(P(R=r\mid Y=y)\), and check whether each pattern probability depends only on the values observed under that pattern. Also check that the probabilities are valid: each is between 0 and 1 and they sum to 1 over all patterns for every \(y\). Associations between observed variables and response indicators can challenge an MCAR assumption, but cannot by themselves verify MAR; the values missing under a pattern are not available to directly check that pattern’s conditional dependence on them.

The main alternatives are not competing labels for the same feature. A pattern description is a structural summary of the response indicators. An MCAR, MAR, or MNAR description is an assumption about the data-generating mechanism. Report both when relevant. For analyses that depend on untestable mechanism assumptions, consider how conclusions might change under alternatives rather than treating a pattern classification as proof of an assumption.

Order-dependent judgment: A pattern may be monotone in one order and not another. State the order used.
Inconsistent orders: A set can contain patterns that are individually monotone under different orders but is not monotone as a collection.
Invalid probabilities: Equal conditional probabilities across patterns imply MCAR only when they form a valid distribution and sum to 1.
Overinterpreting a pattern table: Pattern counts describe observed arrangements; they do not identify the mechanism or establish MAR.
Changing the conditioning set: MAR and MNAR classifications refer to the variables included in \(Y\) and the stated conditioning information.

The practical limitation is fundamental: response patterns are observable, but mechanism claims involve complete-data values that may not be observed. Pattern summaries can expose structural features and help clarify assumptions, but they cannot settle the mechanism question on their own.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In the order \(Y_1,Y_2,Y_3\), is \((1,0,0)\) monotone? Is \((1,0,1)\)? Explain.
For two variables, a collection contains \((0,1)\) and \((1,0)\). Why is it not enough that each pattern can be made monotone under some order?
For \(p=3\), a proposed mechanism gives each of the eight patterns probability \(1/8\), independent of \(y\). Is it a valid mechanism, and how is it classified?
A variable is always observed, and the probability that a second variable is observed depends on the first. What can be concluded about the mechanism if the dependence is otherwise specified using only the first variable?

Solutions. 1. \((1,0,0)\) is monotone: missing \(Y_2\) is followed by missing \(Y_3\). \((1,0,1)\) is not, because missing \(Y_2\) is followed by observed \(Y_3\). 2. The definition for a collection requires one common order. The order \(Y_1,Y_2\) fails for \((0,1)\), while the order \(Y_2,Y_1\) fails for \((1,0)\). 3. It is valid because \(8(1/8)=1\), and it is MCAR because the probabilities do not depend on \(y\); therefore it is also MAR. 4. If the first variable is observed in every possible pattern and the response probabilities depend only on its value, the mechanism is MAR. It is not MCAR if that dependence changes the probabilities across values of the first variable.