Diagnosing SMC-FCS Draws for Derived-Term Models

Why Diagnose SMC-FCS Draws?

Substantive-model-compatible fully conditional specification (SMC-FCS) is designed to make imputation draws reflect the substantive model, including nonlinear terms such as a quadratic effect. A run can nevertheless be inefficient or implemented incorrectly. Rejection sampling may accept few proposals, imputed covariate values may be implausible in context, or a derived term may not correspond to the covariate value from which it should be calculated. Each problem calls for a different check.

Diagnostics are evidence about the computation, not a guarantee of correct inference. A plausible-looking draw does not establish that the substantive model is correctly specified, that the missingness assumptions are appropriate, or that the imputation model captures relevant predictors. Conversely, a draw outside the observed sample range is not automatically an error. Assessment requires the model, units, support, and data context.

Acceptance Rates: Efficiency and Warning Signs

In rejection sampling, a proposal is accepted with a probability chosen so that accepted values follow the desired target distribution. For instance, if a proposal density is \(q(x)\) and an unnormalized target density is \(g(x)\), a valid rejection rule requires a finite constant \(M\) such that \(g(x)\le Mq(x)\) wherever the target has support. The acceptance probability is \(g(x)/(Mq(x))\). This rule assumes that the proposal covers the target support and that the stated bound is valid. If either condition fails, the rejection sampler is not valid, regardless of its observed acceptance rate.

For \(N\) proposals with acceptance indicators \(A_j\), where \(A_j=1\) means proposal \(j\) was accepted and \(A_j=0\) otherwise, the empirical rate is

Under a fixed proposal and target, the expected acceptance probability is the proposal average of the acceptance rule. In the rule above, it is \(\int q(x)g(x)/(Mq(x))\,dx=Z/M\), where \(Z=\int g(x)\,dx\) is the normalizing constant. Thus a low rate can result from a proposal that poorly covers important target regions or from a conservative (loose) bound \(M\). It principally signals computational inefficiency. It does not, by itself, show that accepted draws have the wrong target distribution.

There is no universal acceptance-rate threshold that separates acceptable from unacceptable SMC-FCS runs. Record the number of proposals and accepted draws, and examine rates by incomplete record, iteration, and relevant covariate or outcome patterns. A pooled rate may hide records for which almost all proposals are rejected. Also verify that the implementation uses the intended target and a valid acceptance rule; a seemingly high rate cannot certify either.

Covariate Ranges and Conditional Plausibility

For each incomplete continuous covariate, examine the values actually sampled, not just the final minimum and maximum. Summaries such as quantiles, histograms, and plots by iteration or record can reveal concentration at bounds, long tails, gaps, or abrupt changes. Compare draws with observed values, known measurement limits, units, and scientifically possible values. Where relevant, compare them within important outcome or covariate patterns rather than relying only on a marginal summary.

The observed minimum and maximum are sample extremes, not necessarily the support of the population. A draw just beyond an observed extreme may be reasonable under the imputation model. A draw outside a genuine physical or design limit, such as a negative duration that cannot be negative, warrants investigation. Likewise, a range consistent with the observed data does not prove the conditional distribution is appropriate. Check whether the location, spread, and tails of the draws are credible given the modeled predictors.

For nonlinear substantive models, range checks have an additional purpose: the substantive effect can be especially sensitive to values in the tails. For a quadratic term, a few large absolute values of \(X\) can have substantial influence because \(X^2\) grows rapidly. Inspect the covariate scale and the derived-term scale together, and determine whether those values are plausible under the model and in the application.

Derived-Term Behavior and Chain Monitoring

A derived term must be calculated from the completed covariates for the same record and the same iteration. If an imputed \(X\) is paired with a separately imputed \(X^2\), the identity can fail, and the substantive model is no longer using the claimed quadratic term. The basic check is direct: for each completed record, calculate \(X^2\) from \(X\) and compare it with the stored term. For an interaction, compare the stored term with the product of the corresponding covariate values. Differences should be zero apart from any explicitly justified numerical rounding.

Identity checks do not establish that the distribution of \(X\) is appropriate. Inspect the distribution of \(X^2\) as well: it is nonnegative, and its shape is determined by the distribution of \(X\). For an interaction \(X_1X_2\), check the product against both inputs, since unusual combinations can arise even when each marginal range looks ordinary. Plot derived terms against their source covariates when useful, and review summaries across iterations to see whether the chain settles into a stable range rather than drifting or showing persistent changes.

A trace plot displays a sampled value or summary against iteration. Persistent drift, long stretches with little variation, or large abrupt shifts can indicate poor exploration or an unstable computation. A trace that looks stable is reassuring but not conclusive; a run may appear steady while exploring only part of the relevant distribution. Where draws are highly dependent, a large raw number of iterations can also overstate how much independent information the chain contains. Use such plots alongside acceptance, range, and deterministic-identity checks rather than treating any single display as a verdict.

A Practical Diagnostic Sequence

Use several checks together. First confirm that each derived term is computed from the matching covariate draws. Then summarize the sampling process and the covariate draws, investigate patterns that appear concentrated in particular records or iterations, and assess whether the algorithm’s assumptions and support are credible.

Common Pitfalls and Limits

A useful comparison is with passive imputation of a derived term. Passive calculation makes the deterministic identity explicit, but does not by itself ensure that the imputed covariate distribution respects the substantive model. SMC-FCS targets compatibility with that model, but its draws still require implementation and plausibility checks. Neither a low acceptance rate nor an identity check alone answers whether one approach is adequate for a particular analysis.

A common mistake is to treat a low acceptance rate as proof that the accepted draws are invalid, or to treat an observed-range exceedance as an automatic error. The correction is to separate efficiency from validity: investigate the sampling rule and proposal for the first issue, and assess support and conditional plausibility for the second. Similarly, a perfect derived-term identity proves only that the transformation was applied consistently; it does not validate the imputation model.

Diagnostics cannot repair an incorrect substantive model, omitted predictors, incompatible data assumptions, or a missingness process that is inadequately represented. Acceptance rates can also vary naturally across records, and finite runs make empirical rates noisy. Document what was checked, the scale and units used, where any warnings arose, and what changes followed. A rerun should be evaluated afresh rather than declared successful merely because a single rate improved.