Tutorials › Marginal Structural Models › Intermittent Missingness Versus Monotone Dropout

Censoring and Informative Dropout · Tutorial 234 of 501

Intermittent Missingness Versus Monotone Dropout

Learn how observation patterns determine whether censoring weights apply, and what assumptions are needed to weight marginal means when measurements can be missed and later resumed.

Advanced 11 min read

What You'll Learn

  • Distinguish monotone dropout from intermittent observation gaps using longitudinal observation indicators
  • Define conditional observation probabilities for both types of observation process
  • Derive the inverse-probability identity for monotone censoring
  • Identify conditions under which a single-time mean can be weighted with intermittent observation
  • Explain why dropout weights do not recover arbitrary missing histories
  • Recognize observation-pattern and positivity failures that undermine weighting

Two Different Observation Patterns

A participant who leaves follow-up is not the same as a participant whose measurement is missed and later recorded again. The distinction matters because the usual inverse-probability-of-censoring construction represents loss to follow-up as an absorbing event: after censoring, later outcomes are unavailable. Intermittent missingness does not have that structure. An analysis that treats every missing visit as permanent dropout can discard valid later measurements, while a dropout weight applied to arbitrary gaps may assign the wrong observation probabilities.

Let \(Y_t\) be the outcome at visit \(t\), for \(t=1,\ldots,K\), and let \(R_t=1\) if \(Y_t\) is observed and \(R_t=0\) otherwise. The complete outcome vector is \((Y_1,\ldots,Y_K)\), while the observation pattern is \((R_1,\ldots,R_K)\). Assume for now that \(R_0=1\), so everyone is observed at baseline. The observation pattern \(1,1,0,0\) is monotone dropout; \(1,0,1,0\) is intermittent missingness.

Definition: Observation is monotone if \(R_t=0\) implies \(R_s=0\) for every later visit \(s>t\). Equivalently, once observation stops, it never resumes. The first unobserved visit can then be represented by a censoring time. Observation is intermittent when a participant can have \(R_t=0\) and \(R_s=1\) for some later \(s\); a single absorbing censoring time does not describe that pattern.

These are properties of the recorded observation process, not of the outcome distribution. A missed visit may be unrelated to prognosis or highly informative; a permanent dropout may also be informative. In either case, weighting requires assumptions about why measurements are observed, as well as sufficient probability of observation for the histories relevant to the target.

What Monotone Censoring Weights Represent

Under monotone observation, define the conditional probability of remaining observed at visit \(j\), among those observed through visit \(j-1\), as \(q_j(H_{j-1})=P(R_j=1\mid R_{j-1}=1,H_{j-1})\). Here \(H_{j-1}\) denotes the history available before the observation decision at \(j\), such as baseline variables and previously observed outcomes and covariates. The probability of being observed through visit \(t\), conditional on the observed history along the way, is represented by the product of these conditional probabilities.

Definition: The monotone observation probability through visit \(t\) is \(G_t=\prod_{j=1}^{t}q_j(H_{j-1})\). The corresponding unstabilized inverse-probability-of-censoring weight for a record observed through \(t\) is \(1/G_t\). Its interpretation depends on the absorbing dropout structure: it is the inverse probability of surviving every observation decision from visit 1 through visit \(t\), given the histories used in those decisions.

For the standard weighting identity, suppose \(Z\) is a full-data quantity whose mean is the target. Assume: (1) \(Z\) is integrable; (2) each \(q_j(H_{j-1})\) is the correctly specified conditional probability of observation among those at risk and is positive wherever the target population has support; and (3) at each decision, conditional on the observed history among those still observed, observation is independent of \(Z\). In applications, this is a sequential independent-censoring assumption: the history must contain the predictors needed to account for informative censoring. It is not established merely by fitting a model to the observed data.

Result: Under the assumptions above and monotone observation, \(E[R_t Z/G_t]=E[Z]\). In a sample, this identity motivates estimating the target mean with the average of \(R_t Z/\hat G_t\), subject to the usual sampling and model-estimation considerations.

Proof. Write \(G_j=\prod_{m=1}^{j}q_m(H_{m-1})\) and \(G_0=1\). Conditional on the history through \(j-1\) for a participant still observed, the observation decision has probability \(q_j(H_{j-1})\). By the sequential independence assumption, weighting the observed cases by \(1/q_j(H_{j-1})\) reverses that selection for \(Z\). In particular, the conditional expectation of the contribution at step \(j\), given the at-risk history and \(Z\), is \(E[R_j Z/q_j(H_{j-1})\mid R_{j-1}=1,H_{j-1},Z]=Z\). Because observation is monotone, \(R_t=1\) requires every earlier observation decision to be successful. Applying the conditional expectation identity successively at visits \(1,\ldots,t\) therefore gives \(E[R_t Z/G_t]=E[Z]\). Positivity ensures that each division is defined on the target support; integrability ensures the expectations exist. \(\square\)

Worked Example: Recovering a Mean Under Monotone Dropout

Suppose a target population has 100 people and a binary history \(H\) measured before the outcome visit. Fifty have \(H=0\), with mean outcome \(E[Y\mid H=0]=4\), and 50 have \(H=1\), with mean \(E[Y\mid H=1]=10\). The probabilities of remaining observed through the outcome visit are 0.50 for \(H=0\) and 0.80 for \(H=1\). Assume the outcome is independent of observation conditional on \(H\).

The target mean is \((50\cdot4+50\cdot10)/100=7\). In expectation, 25 people with \(H=0\) are observed and 40 with \(H=1\) are observed. Their weighted total is \(25(4/0.50)+40(10/0.80)=200+500=700\). Dividing by the target population size, 100, gives \(7\), as required.

The unweighted observed mean is \((25\cdot4+40\cdot10)/65=500/65\), approximately \(7.69\). It is too high because the higher-mean group is more likely to remain observed. This example uses expected group counts and means; an actual finite sample would vary around those values.

Intermittent Observation Requires a Different Probability

When observation can resume, the probability \(G_t\) above is not the probability of observing visit \(t\). For a participant missing visit \(t-1\), the chance of being observed at \(t\) may be positive. The relevant probability depends on the observation process and the information available before that visit. To estimate one marginal mean \(E[Y_t]\), a direct observation-probability argument can sometimes suffice, but it is not automatically a longitudinal censoring-weight analysis.

Result: Let \(H_t\) be observed information available for modeling whether \(Y_t\) is recorded, and let \(\pi_t(H_t)=P(R_t=1\mid H_t)\). If \(Y_t\) is integrable, \(\pi_t(H_t)>0\) on the target support, and \(R_t\) is independent of \(Y_t\) conditional on \(H_t\), then \(E[R_tY_t/\pi_t(H_t)]=E[Y_t]\).

Derivation. Since \(\pi_t(H_t)\) is determined by \(H_t\), conditional independence gives \(E[R_tY_t/\pi_t(H_t)\mid Y_t,H_t]=Y_t E[R_t\mid Y_t,H_t]/\pi_t(H_t)=Y_t\). Taking expectations on both sides yields the identity. Positivity makes the ratio defined, and integrability is needed for the expectations. This result concerns a single-time marginal mean. If missingness depends on unobserved information even after conditioning on \(H_t\), or if positivity fails, the identity does not follow.

Worked Example: A Single-Time Mean with Intermittent Missingness

At visit 3, suppose 40 of 100 participants have \(H_3=0\), with mean \(Y_3=5\), and 60 have \(H_3=1\), with mean \(Y_3=10\). The probabilities of recording \(Y_3\) are 0.50 and 0.75, respectively. Assume recording is conditionally independent of \(Y_3\) given \(H_3\). Some participants may have missed visit 2 and returned at visit 3.

The target mean is \(0.40(5)+0.60(10)=8\). Expected observed counts are \(40(0.50)=20\) and \(60(0.75)=45\). Weighting the observed outcomes by the inverse visit-3 probabilities gives total \(20(5/0.50)+45(10/0.75)=200+600=800\). Dividing by 100 gives 8. The calculation uses the probability of observing visit 3 given its relevant history; it does not require that participants had no gap at visit 2. That probability and the conditional independence assumption still need to be defensible.

Why a Dropout Weight Does Not Fill an Observation Gap

A monotone dropout model treats \(R_{j-1}=0\) as being out of the risk set for later observation. Under that model, \(P(R_j=1\mid R_{j-1}=0)=0\). In an intermittent process this probability can be positive. Therefore a product of “remaining uncensored” probabilities either assigns zero probability to a resumed record or misrepresents the chance that its measurement was observed. It cannot recover an unrecorded outcome from a gap, and it does not by itself make a partially observed history equivalent to a complete one.

Worked Example: A Reobserved Visit Is Not Permanent Dropout

Consider three visits and an intermittent pattern \(R=(1,0,1)\). Suppose, among people observed at visit 1, \(P(R_2=1)=0.60\), and among those unobserved at visit 2, \(P(R_3=1\mid R_2=0)=0.50\). The probability of the path \(1,0,1\) under these transitions is \(1(1-0.60)(0.50)=0.20\).

A monotone model would instead interpret \(R_2=0\) as permanent dropout and set \(P(R_3=1\mid R_2=0)=0\). It would assign probability zero to the observed path, even though the intermittent process assigns it probability 0.20. The path-specific inverse probability would be \(1/0.20=5\) under the stated transition model, but that number is not a generic dropout weight: its validity depends on a correctly specified observation-pattern model and suitable assumptions about missingness. Moreover, it can weight the observed \(Y_3\), not supply the missing \(Y_2\).

The distinction is also important for analyses involving multiple visits together. A weight that identifies \(E[Y_t]\) at each visit separately does not automatically identify a joint quantity involving \((Y_1,\ldots,Y_K)\), such as a function requiring all visits to be observed. The joint observation pattern and the assumptions linking it to the full outcome history must match the target. A participant with pattern \(1,0,1\) contributes observed information at visits 1 and 3, but not the missing value at visit 2. Multiplying marginal visit-specific weights does not turn that record into a fully observed trajectory unless a suitable joint derivation justifies the construction.

Diagnostics, Alternatives, and Limits

Begin by tabulating observation patterns across visits. A pattern with a 0 followed by a 1 is direct evidence that a single absorbing censoring time is inadequate for the recorded process. Also inspect estimated observation probabilities by visit and history. Probabilities near zero imply large inverse weights; an estimated zero makes the corresponding ratio undefined and signals a positivity problem, a sparse history, or model failure. These checks can reveal structural mismatch and unstable estimation, but they cannot establish conditional independence between observation and missing outcomes.

The main alternative depends on the target. For an individual visit mean, an observation-probability model for \(R_t\) conditional on observed predictors may support the single-time identity above. For a longitudinal target involving several outcomes, one may need a model for the joint or sequential observation process that explicitly allows missed visits and return, or an analysis that models the outcome distribution under stated missing-data assumptions. These approaches require their own assumptions and do not make arbitrary missingness ignorable. In particular, neither inverse weighting nor a pattern model repairs missing-not-at-random dependence on unobserved outcomes without additional identifying assumptions or sensitivity analysis.

Common pitfall: “Every \(R_t=0\) is censoring, so multiply the usual remaining-observed probabilities through \(t\).” This treats a temporary gap as an absorbing event and gives the wrong observation probability when later measurements can be recorded. First identify whether the observation pattern is monotone. If it is intermittent, specify the probability relevant to the actual target and observation pattern, then state the conditional-independence and positivity assumptions that justify it.

Check Your Understanding

Answer each question, then check the solutions below.

  1. Is the pattern \(1,0,0,0\) monotone or intermittent? What about \(1,0,1,1\)?
  2. For a single-time mean, state the two key assumptions, in addition to integrability, behind weighting by \(R_t/\pi_t(H_t)\).
  3. Suppose a dropout model assigns probability zero to observing visit 4 after a participant missed visit 3. What observation pattern assumption has it made?
  4. Why does identifying each visit-specific marginal mean not automatically identify a mean that depends jointly on outcomes at all visits?

Solutions. (1) \(1,0,0,0\) is monotone: after the first missing visit, all later visits are also missing. \(1,0,1,1\) is intermittent because observation resumes. (2) Require conditional independence \(R_t\perp Y_t\mid H_t\) and positivity \(\pi_t(H_t)>0\) on the target support; also \(\pi_t(H_t)\) must be the relevant conditional observation probability. (3) It has assumed absorbing, monotone dropout: once unobserved, a participant cannot be observed again. (4) Separate marginal identities concern one outcome at a time. A joint target depends on the joint distribution and observation pattern, and separate weighting does not reconstruct unobserved components of a participant’s trajectory.

Clinical Biostats