Why Run a Grid of Selection-Model Fits?
A baseline missing-at-random analysis makes a particular assumption about how dropout relates to outcomes that were not observed. A selection model provides a way to examine how estimates change when dropout is allowed to depend directly on an outcome that may be missing. In a Diggle–Kenward-style analysis, this dependence is controlled by a sensitivity parameter that is fixed at a sequence of prespecified values. The model is refitted at each value, and the resulting estimates and diagnostics are compared.
The grid is a sensitivity analysis, not a procedure for discovering the “true” dependence from the observed data alone. The unobserved outcomes do not generally identify the sensitivity parameter without additional assumptions or information. The purpose of the workflow is to make the consequences of plausible assumptions explicit, while keeping the outcome model, dropout model, analysis population, and other settings consistent across runs.
Model and Assumptions
Suppose subject \(i\) has scheduled outcomes \(Y_{i1},\ldots,Y_{iT}\), baseline covariates \(X_i\), and monotone dropout: after dropout, no later outcomes are observed. Let \(C_i\) be the last visit with an observed outcome. For a subject who completes follow-up, \(C_i=T\); for a subject who drops out before visit \(t\), \(C_i=t-1\). Define \(D_{it}=1\) if subject \(i\) drops out before the outcome at visit \(t\) is recorded, among subjects still under follow-up immediately before that visit.
Let \(f_\theta(y_i\mid X_i)\) denote the joint outcome model, with parameter vector \(\theta\). It may be expressed as a sequence of conditional distributions for \(Y_{it}\) given covariates and previous outcomes. A Diggle–Kenward-style hazard specifies the conditional probability of dropout at visit \(t\) given the outcome history through \(t\), including the current outcome:
Here \(a_t\) is a visit-specific intercept, \(h_{it}\) contains specified covariates or observed history terms, and \(s(Y_{it})\) is a stated scale or transformation of the current outcome. The sensitivity parameter \(\gamma\) is fixed for a fit; \(\theta\), the dropout nuisance parameters \(a_t,\lambda\), and any other nuisance parameters are estimated. If larger values of a symptom score indicate worse symptoms and \(s(Y)=Y\), then positive \(\gamma\) means higher current symptom scores are associated with higher dropout odds, conditional on the other terms.
The usual assumptions for this workflow include independent subjects, a correctly specified outcome distribution and dropout hazard, correctly encoded visit and dropout histories, and a valid likelihood calculation for the observed data. Monotone dropout is part of the formulation above; intermittent missing outcomes require a likelihood and risk-set definition that explicitly accommodates them. The scale of \(s(Y)\) must be documented because it determines the meaning of a unit change in \(\gamma\). Centering or scaling the outcome can improve interpretation and numerical behavior, but the same transformation must be used in every fit.
When \(\gamma=0\), dropout does not depend on the current outcome after conditioning on the included covariates and observed history. Provided the hazard includes the relevant observed-history terms, this is the MAR reference fit. It should use the same outcome model, data, and other specifications as the saved baseline MAR analysis if the results are to be compared directly.
Observed-Data Likelihood and Grid Workflow
The likelihood must account for the fact that the current outcome at dropout can be unobserved. For a participant observed through visit \(C_i<T\), the likelihood contribution integrates over the missing outcomes. In particular, the dropout event at visit \(C_i+1\) contributes a hazard that depends on \(Y_{i,C_i+1}\), which is missing. The contribution also includes the probabilities of remaining under follow-up at visits where dropout did not occur.
The integral is over missing outcomes, and each \(q_{it}\) is evaluated using the complete outcome vector inside the integral. For a completer, the contribution contains the outcome density and the probabilities of remaining observed through the final scheduled visit; there is no dropout-event factor within follow-up. In computation, the integral may require numerical integration or another likelihood method appropriate to the chosen outcome model. It is not valid to substitute a missing outcome with its mean and treat that value as observed.
A reliable run separates the fixed sensitivity settings from the quantities estimated at each setting. For every grid value, start from a controlled initialization, optimize the same observed-data likelihood, calculate the covariance matrix under the same convention, and save the fit result even if convergence fails. A useful workflow is:
Record the analysis data version, outcome model, dropout hazard, outcome scaling, covariates, optimization settings, and fixed grid before fitting.
Set \(\gamma=g\), maximize the observed-data likelihood over the nuisance and outcome parameters, and obtain standard errors only from an accepted fit.
Store the fixed value, parameter estimates, standard errors, objective value, convergence information, and a clear fit status, including for failed attempts.
Check that the \(\gamma=0\) run reproduces the intended MAR reference under matching specifications, then compare estimates and diagnostics across the grid.
The grid should be specified on a meaningful scale before inspecting the resulting treatment estimates. Choose values that represent defensible departures from MAR, and include zero when the MAR comparison is relevant. A numerical grid is not automatically a scientifically plausible range: for a logistic hazard, a one-unit increase in \(s(Y)\) changes the conditional dropout odds by a factor of \(\exp(\gamma)\), holding other terms fixed.
Worked Example: Interpreting a Fixed Sensitivity Grid
Consider a hypothetical two-arm trial with a continuous symptom score measured in points; higher scores indicate worse symptoms. The analysis estimates a treatment-minus-control difference \(\beta\), in points. The same Gaussian outcome model and dropout hazard are fitted at each fixed \(\gamma\). The following illustrative output is hypothetical:
| Fixed \(\gamma\) | \(\hat{\beta}\) (points) | \(SE(\hat{\beta})\) (points) | Fit status |
|---|---|---|---|
| -0.4 | 1.8 | 0.82 | Converged |
| 0 | 2.4 | 0.80 | Converged |
| 0.4 | 3.1 | 0.91 | Converged |
At \(\gamma=0.4\), a one-point higher current score multiplies the modeled dropout odds by \(\exp(0.4)=1.492\), conditional on the other hazard terms. The treatment estimate is \(3.1\) points with standard error \(0.91\) points. Using a normal approximation, its conditional 95% Wald interval is
Thus, under this fixed selection-model value and the fitted model, the interval for the treatment-minus-control difference is approximately 1.32 to 4.88 points. The estimate changes from 2.4 points at the MAR reference to 3.1 points at \(\gamma=0.4\), so the point estimate is sensitive to this departure. The interval is conditional on \(\gamma=0.4\); it is not an interval that accounts for uncertainty across the sensitivity grid.
Likelihood Calculation in a Small Case
A small calculation helps show why the dropout component cannot simply be evaluated at a guessed missing outcome. Suppose a hypothetical participant has an observed first-visit score \(Y_{i1}=8\), then drops out before visit 2. Conditional on the first score, suppose the outcome model assigns probabilities \(0.25, 0.50, 0.25\) to possible second-visit scores \(6,8,10\), respectively. For illustration, let the dropout hazard be \(\operatorname{logit}(q_{i2})=-2+0.5(Y_{i2}-8)\), with the sensitivity value fixed at \(0.5\).
The dropout probabilities at the three possible scores are
Marginalizing over the missing second-visit score gives the conditional probability of dropout at visit 2:
This is the dropout factor after integrating over the outcome model for visit 2; the full participant likelihood also includes the modeled density or probability for the observed first score and any earlier at-risk factors. Plugging in \(Y_{i2}=8\) instead would give \(0.1192\), which differs from the integrated contribution. The finite three-value calculation is only an illustration; a continuous outcome model requires integration over its distribution, often numerically.
Saving Results and Checking the Fits
The saved output should make each attempted fit auditable and interpretable. At minimum, retain the fixed \(\gamma\), the estimates and standard errors for parameters of interest, the nuisance estimates needed to reproduce the fit, the maximized log likelihood or objective, iteration and convergence information, and an explicit status. Include the outcome scale and model version in run-level metadata. Keep failed rows rather than silently dropping them: a missing result at one grid value can otherwise make a sensitivity curve appear smoother or more complete than the computation supports.
For an accepted fit, standard errors are usually derived from the estimated covariance matrix based on the observed information, conditional on the fixed sensitivity value. Check that the matrix is finite and appropriate for the reported parameters. A convergence flag alone is not sufficient evidence of a trustworthy fit. Compare gradients or optimizer diagnostics, inspect boundary estimates and implausible nuisance values, and check whether the objective and estimates are stable under reasonable alternative starting values. These checks are especially important when large \(|\gamma|\) values make the likelihood sharply curved or weakly identified.
- Nonconvergence or a singular covariance matrix: report the fit as failed or requiring review; do not present an ordinary standard error as if it were reliable.
- Unexpected changes at adjacent grid values: check coding, outcome scaling, initial values, and optimization diagnostics before interpreting the change as substantive.
- A mismatch at \(\gamma=0\): verify the hazard’s observed-history terms and confirm that the outcome model, data, and analysis population match the MAR reference.
- Different effective data or specifications across runs: rebuild the grid from a single frozen specification; otherwise the estimates do not isolate sensitivity to \(\gamma\).
Comparison and Limitations
A selection-model grid differs from a single MAR fit because it deliberately varies the assumed dependence of dropout on a potentially unobserved current outcome. It also differs from a pattern-mixture sensitivity analysis, which parameterizes assumptions through outcome distributions conditional on missingness patterns. The two approaches express sensitivity in different model components; their parameter values are not directly interchangeable. The main practical advantage of the fixed-\(\gamma\) grid is that it gives a structured set of refitted likelihood analyses under clearly stated selection assumptions. Its limitation is equally important: observed data alone generally cannot settle which sensitivity value is correct, and results depend on both the outcome model and the dropout hazard specification.
A grid also does not automatically provide an overall uncertainty interval that combines sampling uncertainty with uncertainty about \(\gamma\). Nor does a statistically stable fit establish that its assumptions are clinically credible. Report the grid, the scale and interpretation of \(\gamma\), fit failures, and the conditional nature of standard errors alongside the estimates. Avoid selecting a sensitivity value solely because it yields a preferred conclusion.
Worked Example: Preserving a Failed Grid Attempt
Suppose a hypothetical run uses the fixed values \(-0.4, 0, 0.4, 0.8\). The first three fits converge, but the optimizer at \(0.8\) stops with a non-finite covariance matrix. A correct output retains four rows. The first three contain estimates and standard errors with status “Converged; diagnostics reviewed.” The last contains \(\gamma=0.8\), the available optimizer message and objective value, missing estimate or standard-error fields where they cannot be trusted, and status “Review required: covariance matrix non-finite.”
The fourth attempt must not be removed merely because it cannot be plotted. If a graph is produced, show a gap or another explicit failure indication at \(0.8\), rather than drawing an interpolated estimate. The conclusion is that the model did not yield a reportable covariance estimate at this grid value under the recorded run settings—not that the treatment effect at \(0.8\) is zero or equal to a neighboring estimate. Any retry should be logged with its changed settings so it remains distinguishable from the original attempt.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- In the hazard above, what does \(\exp(\gamma)\) represent when \(s(Y)=Y\)?
- Why does a dropout at visit \(C_i+1\) require integration over \(Y_{i,C_i+1}\) when that outcome is missing?
- A fit at fixed \(\gamma\) has a small standard error. Does that quantify uncertainty about the choice of \(\gamma\)?
- At \(\gamma=0.3\), a higher symptom score is associated with dropout. What is the modeled odds multiplier for a one-point increase?
- One grid fit fails while adjacent fits converge. What should be saved and reported?
Solutions. (1) It is the multiplier in conditional dropout odds for a one-unit increase in the outcome, holding the other hazard terms fixed. (2) The dropout probability depends on that current outcome, so the observed-data likelihood averages the hazard over the outcome distribution for the unobserved value. (3) No. It is conditional on the fixed \(\gamma\) and does not quantify uncertainty about that sensitivity assumption. (4) \(\exp(0.3)=1.350\), approximately, so the modeled odds are multiplied by 1.350 per one-point increase, conditionally. (5) Retain the attempted grid row and its failure diagnostics, mark estimates or standard errors unavailable when unreliable, and do not silently interpolate or omit the failed value.