Normal and Heavy-Tailed Random-Effect Distributions

Why the Random-Effect Distribution Matters

A subject-level random effect describes how an individual parameter departs from its typical value after accounting for modeled covariates. The distribution assigned to that effect determines more than its variance: it also determines how often the model predicts unusually large departures. This matters when an analysis must describe individuals at the extremes, not only the center of the population.

For a positive parameter such as clearance, a common specification is \(\theta_i=\theta_{\mathrm{typ}}\exp(\eta_i)\), where \(\theta_i\) is the parameter for subject \(i\), \(\theta_{\mathrm{typ}}\) is its typical value for the specified covariates, and \(\eta_i\) is the subject-level random effect. The exponential transformation keeps \(\theta_i\) positive. The distribution of \(\eta_i\) therefore controls the distribution of the multiplicative departure from \(\theta_{\mathrm{typ}}\).

The normal distribution is a conventional starting point. A Student-t distribution is one possible heavy-tailed alternative: it assigns more probability to sufficiently distant values than a normal distribution with the same variance, although it need not do so at every cutoff. Comparing them meaningfully requires matching their variances rather than merely using the same scale parameter.

The Student-t scaling is essential. A standard \(t_\nu\) variable has variance \(\nu/(\nu-2)\) when \(\nu>2\). Multiplying it by \(\sqrt{(\nu-2)/\nu}\) gives variance one; multiplying that result by \(\omega\) gives variance \(\omega^2\). If \(\nu\leq2\), the Student-t variance is not finite, so matching it to \(\omega^2\) is not possible. In that case, a scale parameter must not be described as a standard deviation.

In either model, symmetry about zero makes zero the median of \(\eta_i\). Under the exponential parameter equation, \(\theta_{\mathrm{typ}}\) is consequently the median individual parameter, conditional on the covariates. It is not generally the arithmetic mean. The distributional comparison below concerns one random effect at a time; it does not specify relationships among multiple random effects.

Comparing Tail Probabilities at Equal Variance

A fair comparison holds \(\omega\) fixed in both models. For a cutoff of \(q\omega\), the normal model has \(P(|\eta_i|>q\omega)=P(|Z|>q)\) for \(Z\sim N(0,1)\). Under the variance-matched \(t_\nu\) model, the corresponding cutoff for \(T_i\) is \(q\sqrt{\nu/(\nu-2)}\). Thus the Student-t tail probability is \(P(|T_i|>q\sqrt{\nu/(\nu-2)})\).

For \(\nu=4\), the scaled effect is \(\eta_i=(\omega/\sqrt{2})T_i\). Its tail probabilities can be calculated from the \(t_4\) density. For positive \(x\), set \(u=x/\sqrt{x^2+4}\). Substitution into the density gives the density of \(u\) as \(3(1-u^2)/4\). Integrating from the value corresponding to \(x\) to 1 yields the one-sided tail:

This result is useful for checking numerical comparisons and shows why “heavy-tailed” should not be read as “larger probability beyond every possible cutoff.” After the variance is matched, the \(t_4\) distribution is more concentrated near its center than the normal distribution, but its far tails decline more slowly.

From Random-Effect Tails to Parameter Extremes

With \(\theta_i=\theta_{\mathrm{typ}}\exp(\eta_i)\), an individual parameter exceeds \(c\theta_{\mathrm{typ}}\), for \(c>1\), exactly when \(\eta_i>\ln(c)\). The lower-tail analogue is \(\theta_i<\theta_{\mathrm{typ}}/c\) exactly when \(\eta_i<-\ln(c)\). These transformations let us translate a random-effect distribution into predictions about unusually high or low individual parameters.

Assumptions, Diagnostics, and Alternatives

The distributional comparison assumes that the parameter equation and covariate effects have been specified appropriately, that the random effect is centered as stated, and that the chosen distribution describes variation among subjects. It also assumes the same \(\omega\) when comparing the normal and variance-matched Student-t alternatives. Changing the distribution does not repair a misspecified typical-value model, omitted covariates, or an unsuitable residual error model.

A heavy-tailed specification can be useful when a normal model systematically underpredicts extreme individual departures. However, unusually large fitted individual effects alone do not establish that the population distribution is heavy-tailed. Such estimates can be affected by sparse or noisy observations, model misspecification, and the degree to which individual effects are informed by each subject’s data. Diagnostics should therefore examine whether the model reproduces the observed data patterns and the frequency of extreme outcomes, preferably through simulation from the fitted model. Check that conclusions about extreme predictions are not driven by a small number of subjects or by implausible extrapolation on the parameter scale.

A normal distribution remains attractive for its simplicity, finite exponential moments, and familiar interpretation through \(\omega\). A Student-t distribution adds a degrees-of-freedom parameter that controls tail behavior; very low degrees of freedom can make extreme predictions and arithmetic means problematic. A finite mixture is another alternative when the scientific hypothesis is that subjects arise from distinct subpopulations rather than from one continuous heavy-tailed population. These alternatives encode different assumptions and are not interchangeable merely because each can accommodate a few unusual observations.

Model comparison should consider more than whether a fitted heavy-tailed model assigns greater likelihood to observed data. Examine the implied distribution of individual parameters, its tail probabilities at scientifically relevant cutoffs, and whether its simulated observations resemble the data. If a Student-t model is used for a log-scale effect, report the degrees of freedom and clarify whether its scale has been converted to a standard deviation. Also explain that an untruncated Student-t effect can imply an infinite arithmetic mean after exponentiation.