What Does “Prediction for a New Group” Mean?
A fitted multilevel model can describe a response in a group represented in the data or predict a response in a group not yet observed. These are different prediction targets. For an observed group, its data provide information about its group-specific random effect. For a new group, there is no group-specific data from which to estimate that effect. A prediction must instead account for the distribution of possible effects across groups.
The distinction matters especially in a generalized linear mixed model (GLMM). Because the inverse link is generally nonlinear, setting an unavailable random effect to its mean of zero does not usually give the average response across new groups. The linear predictor at a zero random effect and the average response after integrating over random effects are different quantities.
Model and Assumptions
For a simple two-level nested GLMM, let observation \(i\) belong to lower-level group \(j\), which belongs to higher-level group \(k\). A representative model is
Here \(g\) is the link function, \(\boldsymbol{\beta}\) is the vector of fixed effects, \(u_k\) is the random effect for higher-level group \(k\), and \(v_{jk}\) is the random effect for lower-level group \(j\) nested within \(k\). Random effects are commonly modelled as mean-zero multivariate normal variables with specified covariance matrices; effects from distinct groups are independent, and the levels are independent of one another unless the model specifies otherwise. Conditional on the random effects and covariates, responses are assumed to follow the stated response distribution and to be independent.
Prediction also depends on the prediction target. For a conditional mean on the response scale, apply the inverse link to the linear predictor if the random effects are treated as fixed at specified values. For a new-group mean, average that inverse-link value over the unavailable random effects. If the target is an actual future response rather than its conditional mean, response variability must also be included. Parameter estimates and estimated variance components add further uncertainty to prediction intervals.
For an observed group, the random effect is not literally known: it is estimated using the fitted model and that group’s data. An empirical Bayes estimate, such as a conditional mode or mean, is often used for a point prediction. In a nonlinear model, applying the inverse link at this single estimate is generally an approximation to integrating over the conditional distribution of the random effect given the observed data.
Why Integration Changes the Prediction
Suppose a single random intercept \(b\) has distribution \(N(0,\sigma_b^2)\), and the linear predictor excluding that intercept is \(\eta\). For a new group, its mean response is
not generally \(g^{-1}(\eta+\operatorname{E}(b))=g^{-1}(\eta)\). The second expression evaluates the response mean at the average random effect. It is not the average of the response means across groups.
The reason is that averaging and applying a nonlinear function do not generally commute. In the special case of a Gaussian response with an identity link, the inverse link is linear, so integrating a mean-zero random intercept leaves the mean unchanged. For logistic and log links, the inverse links are nonlinear, so the integrated prediction can differ substantially.
Worked Predictions
Worked Example: A New Group in a Logistic Random-Intercept Model
Suppose a hypothetical binary-outcome model has logit link, fixed part \(\eta=-1\) for a specified covariate pattern, and group random intercept \(b\sim N(0,0.64)\). The response is whether a hypothetical patient experiences an event. Compare the point prediction for an observed group with the new-group mean.
Observed group. Suppose the fitted conditional estimate of this group’s intercept is \(\hat b=0.7\). Its plug-in conditional prediction is
This is the estimated event probability conditional on this group’s estimated effect, not the average event probability across groups.
New group. Here \(\sigma_b=0.8\), so the new-group mean is \(\operatorname{E}\{\operatorname{logit}^{-1}(-1+b)\}\). To show the calculation, use three-point Gauss–Hermite quadrature for a normal random variable: evaluate at \(b=0\) with weight \(2/3\), and at \(b=\pm\sqrt{3}(0.8)\) with weight \(1/6\) each. Thus,
The quadrature gives an approximate new-group event probability of 0.2925, compared with the observed group’s plug-in estimate of 0.4256. Both are probabilities for the stated covariate pattern, but they answer different questions. The new-group calculation averages over the modelled variability between groups.
Worked Example: A New Lower-Level Group Within an Observed Group
Consider a hypothetical logistic model with a school random intercept \(u_k\) and a classroom random intercept \(v_{jk}\) nested within school. For a specified student covariate pattern, let the fixed part be \(-1\), the fitted school effect be \(0.5\), and the classroom effects have distribution \(N(0,0.36)\). Find the estimated event probability for a new classroom in this observed school.
Because the school is observed, condition on its fitted effect: the predictor before the classroom effect is \(-1+0.5=-0.5\). The classroom is new, so integrate over \(v_{jk}\), whose standard deviation is \(0.6\). Using the same three-point quadrature, the two nonzero nodes are \(\pm\sqrt{3}(0.6)=\pm1.039\):
The prediction is approximately 0.3864 for the new classroom, conditional on the fitted effect of this school. If instead the classroom were observed and its fitted intercept were \(-0.2\), the plug-in probability would be \(\operatorname{logit}^{-1}(-1+0.5-0.2)=\operatorname{logit}^{-1}(-0.7)\approx0.3318\). The latter uses information specific to that classroom; the former averages over possible classroom effects.
If both the school and classroom were new, the prediction target would average over both effects. If their random intercepts are independent, their sum has variance equal to the sum of their variances. The point prediction would integrate over both distributions rather than condition on the observed school’s fitted effect.
Worked Example: A New Group in a Poisson Random-Intercept Model
Suppose a hypothetical Poisson GLMM models event counts over a fixed exposure window, with log link and random intercept \(b\sim N(0,0.25)\). For the specified covariates and exposure, let the fixed linear predictor be \(\eta=\log(2)\). Find the expected count for a new group.
The conditional expected count at random intercept \(b\) is \(\exp\{\log(2)+b\}\). A normal random variable satisfies \(\operatorname{E}\{\exp(b)\}=\exp(\sigma_b^2/2)\). Therefore,
The predicted mean is approximately 2.266 events per stated exposure window for a new group. Simply setting \(b=0\) would give \(\exp\{\log(2)\}=2\), the conditional mean at a zero random intercept, not the average mean across new groups. For comparison, if an observed group had fitted intercept \(0.4\), its plug-in conditional mean would be \(\exp\{\log(2)+0.4\}\approx2.984\) events per exposure window.
A Practical Prediction Workflow
State the covariate pattern, response scale, and which levels are observed or new. Distinguish a mean response from a prediction for an individual future response.
For each random effect, determine whether its group was represented in the fitted data and whether the target conditions on that group.
Include the fixed part and any group-specific effects appropriate to the target. For a new group, do not substitute an effect estimated from a different group.
Average the inverse-link mean over the relevant random-effect distribution, using analytic integration where available or numerical integration otherwise.
Name the population of groups represented by the average and account for uncertainty in fitted parameters, variance components, random effects where relevant, and future response variation.
Diagnostics, Alternatives, and Limitations
A new-group prediction is only as credible as the model’s description of between-group variation. Check that the grouping structure and random-effect distribution are plausible, and look for influential groups or systematic residual patterns. For a model intended to predict new groups, assess performance by holding out entire groups, not merely random observations within groups. Otherwise, information from the same group may appear in both training and validation data, making the predictions look more successful than they are for genuinely new groups.
A useful comparison is between predictions conditional on estimated effects for observed groups and predictions integrated over effects for new groups. The first can be valuable for forecasting further observations in an existing group. The second addresses average performance across groups not used to estimate group-specific effects. Setting new effects to zero is a quick plug-in alternative, but with a nonlinear inverse link it generally targets the response mean at the average random effect, not the population-average response mean. It may be a reasonable approximation when random-effect variation is negligible, but that approximation should be checked rather than assumed.
Integration over random effects does not by itself produce a complete prediction interval. A confidence interval for the average response mean and a prediction interval for a future observation describe different uncertainty. A future-observation interval must include response variation; a new-group prediction must also reflect variation among possible group effects. Uncertainty in estimated fixed effects and variance components may matter, particularly when the number of groups is limited.
Finally, predictions for new groups assume that the new groups are drawn from the same modelled population as the groups used to fit the GLMM. If new groups differ systematically, or their covariates lie outside the observed range, integration under the fitted random-effect distribution does not repair that mismatch. The model’s prediction target and the intended population of groups must therefore be stated clearly.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A logistic random-intercept model is used to predict for an already observed group. What group-specific information distinguishes this prediction from one for a new group?
- For a new group, why is \(g^{-1}(\eta)\) generally not the mean response when the random intercept has mean zero?
- A new classroom belongs to a school already represented in the data. Which effect can the prediction condition on, and which effect must it average over?
- In a Poisson log-link model, a new-group random intercept has variance \(0.16\), and the fixed linear predictor is \(\log(3)\). What is the integrated mean count for the stated exposure?
- Why should validation for a new-group prediction hold out entire groups?
Solutions. (1) The observed group’s data inform an estimated conditional distribution or empirical Bayes estimate of its random effect; a new group has no such group-specific data. (2) The inverse link is nonlinear, so averaging \(g^{-1}(\eta+b)\) over \(b\) does not generally equal applying \(g^{-1}\) at \(\operatorname{E}(b)=0\). (3) The prediction may condition on the observed school effect, while integrating over the new classroom effect. (4) Using the normal moment-generating function, the mean is \(\exp\{\log(3)+0.16/2\}=3\exp(0.08)\approx3.250\) counts per stated exposure. (5) Holding out observations while retaining other observations from the same group leaks group-specific information into validation; holding out groups assesses prediction when group effects are genuinely unavailable.