Prediction with Group-Specific Covariate Information

Why a Group’s Covariate Values Matter

A group can have many observations and still provide little direct information about how its outcome changes with a covariate. If almost every observation is at the same covariate value, the data can help locate the group’s linear predictor there, but have difficulty separating the group’s random intercept from its random slope. This distinction matters whenever a prediction is requested at a different covariate value, particularly one outside the group’s observed range.

The focus here is the effect of where a group’s covariate observations lie and how widely they are spread on random-slope prediction in a logistic GLMM. The conditional distribution and Laplace approximation used below are treated as tools for examining that information; the central question is how the group’s design affects the resulting prediction and its uncertainty.

Model and Local Information

Definition: For group $j$, let $Y_{ij}$ be a binary outcome and $x_{ij}$ its covariate value. A logistic random-intercept and random-slope model is $$ Y_{ij}\mid b_j \ \text{independent},\qquad Y_{ij}\mid b_j\sim \operatorname{Bernoulli}(p_{ij}),\qquad \operatorname{logit}(p_{ij})=\beta_0+\beta_1x_{ij}+b_{0j}+b_{1j}x_{ij}, $$ where $b_j=(b_{0j},b_{1j})^\mathsf{T}\sim N(0,G)$. The group-specific random contribution at covariate value $x$ is $b_{0j}+xb_{1j}$. Here $G$ is a positive-definite random-effects covariance matrix, and observations are conditionally independent given the random effects and model parameters.

For a group’s observed covariate value $x_{ij}$, write $z_{ij}=(1,x_{ij})^\mathsf{T}$. Conditional on fitted fixed-effect and covariance parameters, the group’s posterior density for $b_j$ is proportional to its Bernoulli likelihood multiplied by the $N(0,G)$ density. The conditional mode is the value of $b_j$ that maximizes this density. A local Gaussian (Laplace) approximation uses the inverse negative Hessian of the log posterior at that mode as an approximate posterior covariance, denoted $V_j$.

At a mode with fitted probabilities $\hat p_{ij}$, the likelihood contribution to the negative Hessian is $$ I_j=\sum_i w_{ij}z_{ij}z_{ij}^\mathsf{T},\qquad w_{ij}=\hat p_{ij}(1-\hat p_{ij}). $$ The prior contributes $G^{-1}$, so the local posterior precision is $G^{-1}+I_j$, and $V_j\approx(G^{-1}+I_j)^{-1}$. Thus the covariate values affect both the likelihood information and the uncertainty after accounting for the random-effects distribution. If probabilities are near 0 or 1, the weights are small: those observations contribute less local information than observations with probabilities near 0.5.

Worked Example: Same Group Size, Different Covariate Spread

Consider two hypothetical groups, each with ten binary outcomes. For a transparent design comparison, suppose the fitted probabilities are all near 0.5, so use the approximate weight $w_{ij}=0.25$ for every observation. Group A’s covariates are all 0.9. Group B has five covariate values of 0.5 and five of 1.3. Compare their likelihood information matrices before adding the random-effects prior.

For Group A, $z=(1,0.9)^\mathsf{T}$ for all ten observations, so $$ I_A=10(0.25) \begin{pmatrix}1&0.9\\0.9&0.81\end{pmatrix} = \begin{pmatrix}2.50&2.25\\2.25&2.025\end{pmatrix}. $$ Its determinant is $2.50(2.025)-(2.25)^2=5.0625-5.0625=0$. The likelihood information has rank one: it informs the combination $b_{0j}+0.9b_{1j}$, but not the intercept and slope separately.

For Group B, summing the five observations at each covariate value gives $$ I_B=0.25 \begin{pmatrix} 10&5(0.5)+5(1.3)\\ 5(0.5)+5(1.3)&5(0.5^2)+5(1.3^2) \end{pmatrix} = \begin{pmatrix}2.50&2.25\\2.25&2.425\end{pmatrix}. $$ Now $\det(I_B)=2.50(2.425)-(2.25)^2=6.0625-5.0625=1.00$. The positive determinant shows that both random-effect directions receive likelihood information. The prior still matters, but Group B’s spread gives its data a better basis for distinguishing the intercept from the slope.

Why Spread Separates Intercept and Slope

The determinant calculation has a general form. Write $w_i=\hat p_{ij}(1-\hat p_{ij})$, suppressing the group subscript. Then $$ I= \begin{pmatrix} \sum_i w_i&\sum_i w_ix_i\\ \sum_i w_ix_i&\sum_i w_ix_i^2 \end{pmatrix}. $$ Expanding its determinant gives a useful identity: $$ \begin{aligned} \det(I) &=\left(\sum_iw_i\right)\left(\sum_iw_ix_i^2\right)-\left(\sum_iw_ix_i\right)^2\\ &=\sum_{i<k}w_iw_k(x_i-x_k)^2. \end{aligned} $$

To see the second equality, expand the first product as $\sum_{i,k}w_iw_kx_k^2$ and the squared sum as $\sum_{i,k}w_iw_kx_ix_k$. Pairing the terms for $(i,k)$ and $(k,i)$ leaves $w_iw_k(x_i^2+x_k^2-2x_ix_k)=w_iw_k(x_i-x_k)^2$; the diagonal terms are zero. The determinant is therefore zero if all positively weighted covariate values are identical, and grows as weighted pairwise covariate differences grow. It describes separation of the two coefficient directions, not the amount of information in every direction by itself.

Centering the covariate can make this geometry easier to interpret. It changes the coordinate description of the random intercept and slope, but does not create covariate spread or new data information. As established in the earlier material on centering, predictions at the same original covariate values remain the relevant quantities to compare.

Worked Example: Uncertainty at the Observed Region and Beyond

Suppose a hypothetical group’s observations are concentrated around $x=0.9$. At its conditional mode, the group’s fitted probability at that region is $\hat p=\operatorname{logit}^{-1}(0.168)\approx0.5419$. For ten observations at that probability, the approximate total likelihood weight is $$ w=10\hat p(1-\hat p)\approx10(0.5419)(1-0.5419)=2.4824. $$ Assume a fitted Laplace calculation, using the unrounded quantities internally, gives the local covariance $$ V_j\approx \begin{pmatrix} 0.176949&-0.065849\\ -0.065849&0.190909 \end{pmatrix}. $$ The displayed covariance entries are rounded; the following quadratic forms use the unrounded covariance. These are hypothetical model-based quantities, not a reconstruction of individual outcomes from the group size and fitted probability alone.

At a target covariate value $x$, the random contribution is $b_{0j}+xb_{1j}$. Its approximate posterior variance is $q(x)=(1,x)V_j(1,x)^\mathsf{T}$. At $x=0.9$, $$ q(0.9)=0.176949+2(0.9)(-0.065849)+(0.9)^2(0.190909)\approx0.2131. $$ Thus its approximate posterior standard deviation is $\sqrt{0.2131}\approx0.4616$ log-odds units. This is uncertainty about the group’s random contribution near the region where its observations are concentrated.

At $x=1.8$, the corresponding calculation is $$ q(1.8)=0.176949+2(1.8)(-0.065849)+(1.8)^2(0.190909)\approx0.5584, $$ with approximate standard deviation $\sqrt{0.5584}\approx0.7473$ log-odds units. This larger uncertainty illustrates why a group’s slope prediction can be weakly supported when the target is far from its observed covariates. It is not a standard error for the observed binary outcome, which also has Bernoulli variation.

From Random Contribution to a Group-Specific Prediction

At a target $x$, the conditional-mode estimate of the group’s linear predictor is $\beta_0+\beta_1x+\hat b_{0j}+x\hat b_{1j}$. Applying the inverse logit gives a plug-in probability prediction. The local variance $q(x)$ describes approximate posterior uncertainty in the random contribution on the log-odds scale, conditional on fitted model parameters. Because the inverse-logit transformation is nonlinear, simply transforming the endpoints of a symmetric log-odds interval does not generally produce a symmetric probability interval; the resulting limits and interpretation depend on the specified interval procedure.

The pattern of covariates matters for both the estimated random effects and the precision of predictions. With observations tightly clustered near $c$, the likelihood mainly constrains $b_{0j}+cb_{1j}$. A prediction at $x$ can be expressed as $$ b_{0j}+xb_{1j}=(b_{0j}+cb_{1j})+(x-c)b_{1j}. $$ When $x$ is near $c$, uncertainty in the second term is attenuated by the small factor $x-c$. As the target moves away, uncertainty about the slope has more influence. The random-effects prior supplies information about poorly identified directions, so the final prediction is not necessarily infinitely uncertain even if the likelihood information matrix is rank deficient.

Worked Example: Choosing a Supported Prediction Target

A hypothetical clinic group has visits recorded only for covariate values between 0.7 and 1.1. Its fitted fixed-effect and random-effect estimates imply a plug-in probability of 0.62 at $x=0.9$. The local standard deviation of the group contribution at $x=0.9$ is 0.4616 log-odds units; at $x=1.8$ it is 0.7473 log-odds units, as in the preceding calculation. Which prediction is better supported by this group’s own covariate data?

The prediction at $x=0.9$ is inside the group’s observed region. The linear combination $b_{0j}+0.9b_{1j}$ is close to the combination directly informed by those observations, so it has the smaller local uncertainty. The prediction at $x=1.8$ is outside the observed range and has the larger random-contribution standard deviation. Its plug-in probability can still be calculated, but the group-specific data provide weaker direct support for it; the prior and model structure have greater influence on that extrapolation.

The probability 0.62 is a point prediction, not a measure of its precision. The uncertainty comparison comes from the two variances on the log-odds scale, and it does not by itself include uncertainty in the fitted fixed effects, $G$, or any other estimated model parameters.

Diagnostics, Alternatives, and Limitations

A group-specific covariate design should be checked before interpreting a random-slope prediction. Useful checks include the group’s covariate range and distribution, the weighted information matrix, the fitted probabilities that determine the weights, and the local covariance $V_j$. A nearly singular information matrix, a large posterior correlation between intercept and slope, or a sharply increasing $q(x)$ as $x$ moves away from the group’s data indicates that slope-based predictions may depend substantially on the prior and model assumptions.

Covariate spread is not the only determinant of information. A group with a broad range can still have limited effective information if most fitted probabilities are near 0 or 1, because $p(1-p)$ is then small. Conversely, a group with modest spread may have useful information if its outcomes yield substantial weights and the prior is informative. The local Hessian is a diagnostic approximation: in a binary model, a skewed or otherwise non-Gaussian conditional distribution can make the Laplace approximation imperfect, especially with sparse data or near separation.

A separate logistic fit for each group is an alternative to a GLMM prediction, but it can be unstable or even fail when a group has little covariate spread or separated outcomes. A GLMM uses the random-effects distribution to regularize group predictions, trading some group-specific flexibility for stability. A prediction based only on fixed effects is another comparison: it describes the population-level linear predictor without that group’s estimated random contribution. Neither alternative answers exactly the same question as a group-specific prediction. The earlier material on standard GLM analysis provides the fixed-effects setting; the present issue is how group-specific design affects random-effect prediction.

Finally, the displayed Laplace uncertainty is conditional on fitted model parameters. It is not total uncertainty for a future binary outcome, nor does it automatically include uncertainty from estimating the fixed effects and $G$. Predictions far outside the observed covariate range are especially sensitive to the assumed linear slope and random-effects distribution. Report the target covariate value and observed group range alongside such predictions, and avoid treating a numerically available extrapolation as equally well supported as an interpolation.

Common Pitfall

Common mistake: “This group has ten observations, so its random slope must be estimated precisely.” Group size alone does not determine slope information. If all ten covariate values are 0.9, then every design vector is proportional to $(1,0.9)^\mathsf{T}$, and the group likelihood informs the same intercept-slope combination repeatedly. The correction is to examine the covariate spread, the weights $p_{ij}(1-p_{ij})$, and uncertainty at the particular target value. A group may have a well-supported prediction near its observed covariate cluster but a weakly supported slope or extrapolated prediction.

Check Your Understanding

Answer each question, then compare with the solutions below.

If all observations in a group have $x=2$, which random-effect combination is directly informed by the group’s likelihood?
For two covariate values with positive weights, what happens to their contribution to $\det(I)$ as their separation increases?
Why might ten observations at fitted probabilities near 0.99 provide less local information than ten observations near 0.5?
A group’s covariates range from 0.4 to 1.2. Which target, $x=0.8$ or $x=2.0$, is more directly supported by its covariate data, and why?

Solutions. (1) The likelihood informs $b_{0j}+2b_{1j}$; without covariate spread it cannot separate the two effects from its own information alone. (2) The pair contributes $w_iw_k(x_i-x_k)^2$, so greater separation increases that contribution when the weights are fixed and positive. (3) The local Bernoulli weight is $p(1-p)$, which is about 0.0099 at $p=0.99$ but 0.25 at $p=0.5$; hence the near-0.99 observations contribute less local information. (4) $x=0.8$ lies within the observed range, whereas $x=2.0$ lies outside it, so the former is more directly supported by this group’s data.