Deriving The Random-Slope Conditional Model

Why the Random Slope Changes the Conditional Model

In a clustered data set, the association between a predictor and a response may differ across clusters. A random-intercept model allows clusters to have different baseline response levels, but assumes a common predictor slope. A random-slope model also allows the slope to vary. For imputing a missing response, this means that observed outcomes in a cluster inform both its baseline and its response to the predictor. The amount of information they provide depends on the predictor values at which those outcomes were observed.

The derivation below focuses on a continuous response and predictor under a Gaussian model. It first conditions on the cluster effects, then updates those effects using observed responses in the same cluster. The resulting predictive distribution accounts for uncertainty about the cluster-specific intercept and slope, not only residual variation.

Here \(\beta_0\) and \(\beta_1\) are the population intercept and slope; \(b_{0j}\) and \(b_{1j}\) are cluster \(j\)'s departures from them. Write \(\mathbf b_j=(b_{0j},b_{1j})^\mathsf{T}\), and assume \(\mathbf b_j\sim N(\mathbf 0,\mathbf G)\), where

The residuals are assumed independent across individuals and independent of the random effects, with \(\epsilon_{ij}\sim N(0,\sigma^2)\). The covariance matrix \(\mathbf G\) must be positive semidefinite. In the derivations, model parameters \(\boldsymbol\beta=(\beta_0,\beta_1)^\mathsf{T}\), \(\mathbf G\), and \(\sigma^2\) are treated as known. This makes the conditional distribution explicit; a multiple-imputation procedure must also account for uncertainty in estimated model parameters.

Define the design vector \(\mathbf z_{ij}=(1,x_{ij})^\mathsf{T}\). Then the model mean given the cluster effects is \(\mathbf z_{ij}^\mathsf{T}(\boldsymbol\beta+\mathbf b_j)\). The key feature is that the random slope is multiplied by \(x_{ij}\): its contribution to a response changes with the predictor value.

From the Model to a Predictive Distribution

First consider the response conditional on the cluster effects. Since the residual is normal with variance \(\sigma^2\),

This is the conditional model if the cluster effects were known. In practice they are not known: the observed responses in cluster \(j\) provide information about them. Let \(O_j\) denote the individuals in that cluster whose responses are observed. Stack their responses into \(\mathbf y_{Oj}\), their fixed-effect design rows into \(\mathbf X_{Oj}\), and their random-effect design rows into \(\mathbf Z_{Oj}\). Each row of \(\mathbf Z_{Oj}\) is \((1,x_{ij})\). Under the stated assumptions,

Subtract the fixed part to obtain the observed residual vector \(\mathbf r_j=\mathbf y_{Oj}-\mathbf X_{Oj}\boldsymbol\beta\). Its distribution, after accounting for the random effects, is \(\mathbf r_j\sim N(\mathbf 0,\mathbf Z_{Oj}\mathbf G\mathbf Z_{Oj}^\mathsf{T}+\sigma^2\mathbf I)\). A convenient way to obtain the conditional distribution of \(\mathbf b_j\) is to combine its normal prior with the normal likelihood of \(\mathbf r_j\).

These expressions assume \(\mathbf G\) is invertible. If it is singular, the same conditional normal distribution can be obtained using the general Gaussian conditioning formula without requiring \(\mathbf G^{-1}\).

Derivation and the Missing-Response Formula

For completeness, the posterior expressions follow by multiplying the density of \(\mathbf b_j\) by the likelihood of \(\mathbf r_j\). Ignoring factors that do not involve \(\mathbf b_j\), the exponent in their product is

Expanding the second quadratic gives a quadratic term in \(\mathbf b_j\) with precision matrix \(\mathbf G^{-1}+\mathbf Z_{Oj}^\mathsf{T}\mathbf Z_{Oj}/\sigma^2\), and a linear term \(\mathbf b_j^\mathsf{T}\mathbf Z_{Oj}^\mathsf{T}\mathbf r_j/\sigma^2\). Completing the square therefore gives covariance \(\mathbf V_j\) and mean \(\mathbf m_j\) as defined above. This update shows why predictor spread matters: the observed design rows contain both a column of ones, informing the intercept, and a column of predictor values, informing the slope.

Now let \(Y_{mj}\) be a response missing for an individual with observed predictor \(x_{mj}\), and write \(\mathbf z_{mj}=(1,x_{mj})^\mathsf{T}\). Conditional on \(\mathbf b_j\), the response has mean \(\mathbf z_{mj}^\mathsf{T}(\boldsymbol\beta+\mathbf b_j)\) and residual variance \(\sigma^2\). Averaging over the conditional distribution of \(\mathbf b_j\) gives the missing response's predictive distribution:

The predictive mean uses the updated cluster-specific intercept and slope, \(\boldsymbol\beta+\mathbf m_j\). The predictive variance has two components: residual variance \(\sigma^2\) and uncertainty in the cluster effects, \(\mathbf z_{mj}^\mathsf{T}\mathbf V_j\mathbf z_{mj}\). In particular, uncertainty about the random slope contributes more at predictor values farther from zero, because the slope's contribution is multiplied by \(x_{mj}\).

For several missing responses in one cluster, the same reasoning gives a joint normal predictive distribution. If their design matrix is \(\mathbf Z_{Mj}\), their conditional mean is \(\mathbf X_{Mj}\boldsymbol\beta+\mathbf Z_{Mj}\mathbf m_j\), and their covariance is \(\sigma^2\mathbf I+\mathbf Z_{Mj}\mathbf V_j\mathbf Z_{Mj}^\mathsf{T}\). Thus predictions for missing responses in the same cluster are generally correlated through uncertainty about the shared effects.

Assumptions, Diagnostics, and Alternatives

The formula is useful only to the extent that its model assumptions are reasonable. It assumes a linear response in \(x\) within the specified random-effects structure, normally distributed random effects and residuals, a common residual variance, and independence of residuals conditional on the cluster effects. For an imputation to be appropriate, the response model must also include relevant information associated with missingness; under a missing-at-random mechanism conditional on the included observed data, and the usual distinctness conditions, the missingness process can be ignored for this conditional-model derivation.

A particularly important practical issue is whether observed predictor values within a cluster provide information about its slope. If the observed responses all have nearly the same \(x\), the data may not separate a cluster-specific intercept from a cluster-specific slope. Then \(\mathbf V_j\) can retain substantial slope uncertainty, and missing responses at predictor values far from the observed range may have large predictive variances. This is a warning against treating the estimated cluster slope as known.

• Check whether observed responses and fitted residuals show systematic patterns across \(x\) or across clusters; such patterns can signal a misspecified mean structure.
• Check the observed range and number of distinct predictor values within clusters. Sparse within-cluster information weakens estimation of cluster-specific slopes.
• Inspect whether estimated variance components are near a boundary or whether the fitted model is unstable. Such behavior can make conditional predictions sensitive to modeling choices.
• Compare predictive distributions, not just fitted means. A model that yields a plausible mean but implausibly narrow uncertainty can produce overconfident imputations.

A random-intercept-only model is a simpler alternative, but it constrains the cluster-specific slopes to be equal. It may be inadequate when slopes truly vary. Conversely, adding random slopes without enough within-cluster predictor information can make estimation unstable. A single-level regression also omits the shared cluster effects and does not represent this conditional dependence. Which structure is suitable depends on the data and scientific model, rather than on a desire to make the imputation model maximally complex.

Finally, the displayed distribution conditions on fixed parameter values. Plugging in estimated parameters and using only this distribution does not, by itself, propagate uncertainty in those estimates. In multiple imputation, imputations should reflect relevant parameter uncertainty as well as response-level uncertainty. The derivation here isolates the cluster-effect contribution so that its role in the predictive mean and variance is clear.

Common Mistake