Imputing Intermittently Missing Longitudinal Values

Why an Internal Gap Needs Both Sides of the Series

A value missing between observed repeated measurements is an intermittent missing value. Unlike a value missing at the end of a person’s record, it may have observed measurements both before and after it. Those later measurements can contain information about the missing value, just as earlier measurements do. An imputation model that uses only the past discards that information.

The challenge is to use both sides without treating a person’s repeated records as independent. Measurements from the same person usually share stable characteristics and may also be serially dependent: values closer in time tend to be more alike, even after accounting for the person’s general level and time trend. Irregular visit spacing makes the actual elapsed time important as well.

Definition: Let \(Y_{ij}\) denote the outcome for person \(i\) at their \(j\)th recorded time \(t_{ij}\). A missing \(Y_{ij}\) is intermittent when there is at least one observed outcome for that person at an earlier time and at least one at a later time. The ordering is by \(t_{ij}\), not necessarily by equally spaced visit number.

An internal gap is a pattern in the observed data, not a separate missingness mechanism. The appropriateness of an imputation still depends on assumptions about the missingness process, the outcome model, and the predictors included. The earlier material on irregularly timed repeated measures motivates using actual times rather than assuming equal spacing.

Conditional Imputation With a Time-Structured Model

A useful reference case is a Gaussian longitudinal model. For one person, suppose the outcome has a time-dependent mean \(\mu_i(t)\), and the remaining residuals have covariance matrix \(\Sigma_i\). The mean can include measured covariates and person-level terms; the covariance can represent dependence between records from the same person. Let \(M\) index missing records and \(O\) observed records for that person. Under this model, the conditional distribution of the missing outcomes given the observed ones is:

Y_{iM}\mid Y_{iO} \sim N\left( \mu_{iM}+\Sigma_{i,MO}\Sigma_{i,OO}^{-1}(Y_{iO}-\mu_{iO}), \ \Sigma_{i,MM}-\Sigma_{i,MO}\Sigma_{i,OO}^{-1}\Sigma_{i,OM} \right).

The first term in the conditional mean is the fitted mean at the missing times. The adjustment uses observed residuals—the differences between observed values and their fitted means—weighted according to the covariance structure. The conditional variance describes remaining uncertainty after using the observed records. In a multiple-imputation analysis, the imputed value is drawn from a predictive distribution rather than set equal to its conditional mean.

This formula presumes that the Gaussian mean and covariance model is an adequate description for the outcome, conditional on included covariates and person-level terms. It also presumes that the relevant covariance matrices are well-defined and that the observed data provide information for the parameters being estimated. For a chained-equation implementation, the corresponding conditional models must be plausible and should retain person-level and time structure. A collection of conditional models need not correspond exactly to one joint model, so compatibility cannot simply be assumed.

Worked Example: Conditioning on Records Before and After a Gap

Suppose a hypothetical outcome is measured in score points, with fitted mean \(\mu_i(t)=20+1.5t\). For the person considered here, the missing measurement is at \(t=0\); observed measurements are at \(t=-0.2\) and \(t=0.8\). Their observed residuals are 1 and 2 score points, respectively. Assume a Gaussian residual process with variance 1 and exponential correlation \(\operatorname{Cor}\{e_i(t),e_i(s)\}=\exp(-|t-s|)\). The model’s Gaussian and covariance assumptions are taken as the working assumptions for this calculation.

The correlations between the target residual and the two observed residuals are \(r_1=\exp(-0.2)=0.8187\) and \(r_2=\exp(-0.8)=0.4493\). The observed residuals are one time unit apart, so their correlation is \(r_{12}=\exp(-1)=0.3679\). For two observed values, the conditional-mean weights are:

\beta_1=\frac{r_1-r_2r_{12}}{1-r_{12}^2} =\frac{0.8187-0.4493(0.3679)}{1-0.3679^2} \approx 0.7557, \qquad \beta_2=\frac{r_2-r_1r_{12}}{1-r_{12}^2} =\frac{0.4493-0.8187(0.3679)}{1-0.3679^2} \approx 0.1713.

Thus the conditional mean residual is \(0.7557(1)+0.1713(2)=1.0983\) score points. Since the fitted mean at \(t=0\) is 20, the conditional mean of the missing outcome is \(20+1.0983=21.10\) score points. The conditional residual variance is \(1-\beta_1r_1-\beta_2r_2\approx 0.3043\), giving a conditional standard deviation of \(\sqrt{0.3043}\approx 0.5516\) score points. For example, a draw using a standard-normal value of \(0.50\) is \(21.0983+0.50(0.5516)=21.37\) score points. That draw, not the conditional mean alone, represents an imputed value in this illustrative calculation.

Using the Actual Times

With irregular visits, replacing elapsed time by visit order can distort dependence. Two consecutive records may be close together for one person and far apart for another. A time trend should use recorded times, and a serial-correlation model should relate correlation to elapsed time or otherwise represent the actual timing structure. The next example uses the same exponential-correlation model, but with uneven distances.

Worked Example: An Unevenly Spaced Internal Gap

For a hypothetical person, suppose the outcome mean is \(10+2t\) score points and the residual variance is 1 score-point squared. The residual correlation is \(\exp(-|t-s|/2)\), where time is measured in the study’s time units. The outcome is missing at \(t=4\); observed records are at \(t=1\) and \(t=5\), with residuals 1.2 and \(-0.4\) score points. Assume this Gaussian mean and covariance model is suitable and, for this calculation, condition on these two observed records.

The target-to-observation correlations are \(r_1=\exp(-3/2)=0.2231\) and \(r_2=\exp(-1/2)=0.6065\). The observed records are four time units apart, so \(r_{12}=\exp(-4/2)=0.1353\). The conditional-mean weights are:

\beta_1=\frac{0.2231-0.6065(0.1353)}{1-0.1353^2}\approx 0.1437, \qquad \beta_2=\frac{0.6065-0.2231(0.1353)}{1-0.1353^2}\approx 0.5871.

The conditional mean residual is \(0.1437(1.2)+0.5871(-0.4)\approx-0.0624\) score points. The fitted mean at \(t=4\) is \(10+2(4)=18\), so the conditional mean outcome is approximately \(17.94\) score points. The conditional residual variance is \(1-0.1437(0.2231)-0.5871(0.6065)\approx0.6119\), and its standard deviation is about \(0.7822\) score points. A draw with standard-normal value \(0.50\) would be \(17.9376+0.50(0.7822)\approx18.33\) score points. This calculation shows why the values on either side need not receive equal weight: the observation at \(t=5\) is closer to the missing time than the one at \(t=1\).

In an actual model, conditioning may use more than two records. The matrix formula then combines all included observed outcomes, covariates, and any current imputations as specified by the model. The two-record calculation is a transparent special case, not a rule to discard other informative observations.

Putting the Conditional Model Into MICE

In chained equations, each incomplete variable is updated from its specified conditional model in turn. For a longitudinal outcome with an internal gap, a conditional model can include the person’s records before and after the target time, actual time terms, relevant stable and time-varying covariates, and person-level structure. If a predictor value is itself missing, its current imputation is used in that update. The sequence is repeated over iterations; each completed data set retains a draw rather than a fixed fitted mean.

A Gaussian conditional model is not appropriate for every outcome. The conditional distribution should match the outcome type, while its predictors and dependence structure should reflect the longitudinal design. For example, a binary outcome requires a model for a binary conditional distribution, not a Gaussian draw that can produce impossible values. Whatever model is chosen, uncertainty in its parameters and person-level effects should be represented in the imputation procedure; otherwise, the imputations can be too concentrated.

Worked Example: A Conditional Equation for a Single Update

Consider a hypothetical person with an internal gap at visit 2. For illustration, suppose a fitted chained-equation model, with coefficients and person effect taken from one model draw, is:

Y_{i2}\mid Y_{i1},Y_{i3},x_i,b_i \sim N(4+0.30Y_{i1}+0.40Y_{i3}+0.50x_i+b_i,\ 4).

Assume this conditional Gaussian model is the chosen working specification for this update. The observed neighboring outcomes are \(Y_{i1}=12\) and \(Y_{i3}=16\), the stable covariate is \(x_i=2\), and the drawn person effect is \(b_i=1\). The conditional mean is \(4+0.30(12)+0.40(16)+0.50(2)+1=16\). The residual standard deviation is \(\sqrt{4}=2\). If the predictive draw uses a standard-normal value of \(-0.40\), then the imputed value is \(16+(-0.40)(2)=15.20\) outcome units.

This is one update in one chain, not a complete longitudinal model for all applications. The coefficients, person effect, and variance would be drawn or estimated with uncertainty as required by the imputation procedure. In particular, fixing them at single point estimates across all imputations would omit parameter uncertainty.

Diagnostics, Alternatives, and Limitations

Review the completed data as longitudinal records, not merely as separate columns. For each time and person, check that imputations are in the outcome’s allowable range and that their distributions are plausible conditional on observed earlier and later records. Compare observed and imputed values by time, covariate patterns, and relevant person-level characteristics. Also examine trace plots or other convergence summaries for imputed quantities and model parameters, and inspect whether the model has difficulty fitting sparse time regions or large gaps.

Warning signs include implausibly smooth imputed trajectories, values that ignore a strong neighboring record, systematic shifts between observed and imputed values, and imputations outside the possible range. Such patterns can reflect an inappropriate time scale, omitted person-level dependence, a misspecified mean or covariance model, inadequate overlap in the observed data, or an unsuitable conditional distribution. A gap spanning a long period may be much less predictable than a short gap; the fact that it is bracketed by observations does not guarantee precise imputation.

Deterministic linear interpolation is a simple alternative for constructing a value between two observed records, but it does not by itself represent person-level dependence, model uncertainty, or imputation uncertainty. Last observation carried forward ignores later measurements and can misrepresent change. Complete-case analysis avoids imputing the gap but discards records and can yield biased results depending on the missingness process. A joint longitudinal model is another alternative: it specifies a shared distribution for the repeated outcomes, whereas chained equations specify a sequence of conditional models. The choice depends on the outcome, predictors, and analysis; no method makes unsupported assumptions harmless.

Common pitfall: “The missing value lies between two observed values, so I can average them and treat that average as known.” Being between observed times does not make an outcome linear in time, and a fixed average contains no imputation uncertainty. Use an appropriate conditional model that can use both sides of the gap and draw from its predictive distribution.

The usual missing-at-random assumption for likelihood or multiple-imputation reasoning is that missingness is conditionally unrelated to the unobserved value after conditioning on observed data included in the analysis. It is not established merely because values are missing intermittently. In addition, estimates rely on adequate outcome-model specification and information in the observed data. If missingness may depend on the unseen value even after conditioning, or if a long gap has little supporting information, the imputed results may be sensitive to assumptions and should be interpreted accordingly.

Check Your Understanding

Answer each question, then compare with the solutions below.

A person has observed outcomes at times 1 and 4 and a missing outcome at time 2. Is the missing value intermittent? What timing feature should a dependence model use?
In a Gaussian conditional imputation, why is the conditional mean not generally the completed value for every imputation?
For an exponential correlation model, if the target is one time unit from an observed record and the correlation scale is 2, what is their correlation?
Name one limitation of deterministic interpolation for multiple imputation.

Solutions. (1) Yes: there is an observed outcome earlier at time 1 and later at time 4. Dependence should reflect actual elapsed time, not assume that visit labels are equally spaced. (2) A conditional mean omits the residual predictive variation; a stochastic draw represents that variation, along with parameter uncertainty when the procedure draws parameters. (3) The correlation is \(\exp(-1/2)\approx0.6065\). (4) Interpolation gives a fixed value and does not represent predictive uncertainty; it may also impose an unsupported linear trajectory.