Partial Pooling for Multiple Subgroup Effects

Why Pool Subgroup Effects?

A subgroup analysis may produce several treatment-effect estimates, one for each age band, disease stage, or other prespecified group. Estimates based on small subgroups can be highly variable: a few outcomes may move an estimate substantially, and in some models sparse data can cause separation or unstable estimates. Treating every estimate as entirely independent leaves that instability untouched. Treating all subgroups as having exactly the same effect, however, can erase genuine effect modification.

Partial pooling offers an intermediate approach. It models subgroup effects as related, but not identical. An estimate with little information is pulled more toward the distributional center of the subgroup effects than an estimate with high precision. The result can be more stable than separate estimates while still allowing treatment effects to differ across groups.

A Normal Hierarchical Model

Let \(\theta_j\) be the causal treatment effect for subgroup \(j\), defined on a particular effect scale and for a stated target population. Let \(y_j\) be an estimate of that effect, and let \(s_j\) be its standard error. A simple normal hierarchical model is:

The first distribution describes sampling uncertainty: if the subgroup effect were \(\theta_j\), its estimated value \(y_j\) would vary with standard error \(s_j\). The second describes variation in the underlying effects across subgroups. Here, \(\mu_\theta\) is the center of the subgroup-effect distribution, and \(\tau\) is the between-subgroup standard deviation. A small \(\tau\) says that effects tend to be close to the center; it does not assert that they are exactly equal.

This model is often used as a normal approximation to the subgroup estimates and their sampling distributions. For a binary outcome, for example, an analyst might estimate effects on the risk-difference scale and use this approximation when it is reasonable. A hierarchical model can instead be specified directly for outcome data, perhaps on a log-odds or log-rate scale. In that case, the modeled effects and their interpretation are on that scale unless effects are subsequently transformed or standardized.

The hierarchy assumes that the subgroup effects are exchangeable conditional on the model: before seeing the subgroup-specific data, the model treats their labels symmetrically and describes their effects using a common distribution. This is a modeling assumption, not a claim that the groups are identical or interchangeable in every respect. Subgroups with known, systematic differences may call for a model that includes those differences rather than one common center alone.

How Much Shrinkage?

If \(\mu_\theta\) and \(\tau\) are treated as known, the normal model gives a direct expression for the subgroup effect’s posterior distribution. Its mean is a weighted average of the subgroup estimate and the common center:

The factor \(\lambda_j\) determines how much weight the subgroup estimate receives. If \(s_j\) is small relative to \(\tau\), the estimate is precise compared with the expected variation across groups, and \(\lambda_j\) is near 1. The fitted effect then stays close to \(y_j\). If \(s_j\) is large relative to \(\tau\), the subgroup estimate is noisy relative to the modeled between-subgroup variation, and \(\lambda_j\) is smaller. The fitted effect is pulled more toward \(\mu_\theta\).

The corresponding conditional posterior variance is:

The variance reflects uncertainty about the subgroup effect under the model, conditional on the stated values of \(\mu_\theta\) and \(\tau\). In an actual analysis those quantities are usually unknown and must be estimated or assigned probability distributions. Their uncertainty should then be accounted for in the resulting inference; simply inserting estimated values into these formulas can understate uncertainty, especially when there are few subgroups.

Why the Formula Gives a Weighted Average

The shrinkage expression follows by combining two normal distributions: the sampling model for \(y_j\) and the distribution for \(\theta_j\). As a short derivation, the conditional density for \(\theta_j\) is proportional to the likelihood times the subgroup-effect distribution. Keeping only the terms involving \(\theta_j\), its exponent is:

Collecting the quadratic and linear terms in \(\theta_j\) gives a normal distribution with precision \(1/s_j^2+1/\tau^2\). Its variance is the reciprocal of that precision, and its mean is:

The data contribute more weight when their precision \(1/s_j^2\) is high; the hierarchical distribution contributes more weight when its precision \(1/\tau^2\) is high. This explains why low-information subgroups are shrunk more strongly.

Partial Pooling Is Not Complete Pooling

The value of \(\tau\) controls how strongly the model links subgroup effects. If \(\tau=0\), the model forces every \(\theta_j\) to equal \(\mu_\theta\): this is complete pooling. If \(\tau\) is very large compared with the subgroup standard errors, the model gives little shrinkage and approaches separate estimation. Partial pooling lies between these extremes. Its strength is learned from the amount of variation among subgroup estimates relative to their sampling uncertainty.

In practice, a model may estimate \(\mu_\theta\) and \(\tau\) from all subgroup data, using maximum likelihood, restricted maximum likelihood, or a Bayesian model with priors for those parameters. The details depend on the outcome model and analysis goal. A small estimated \(\tau\) can lead to strong shrinkage, while a larger \(\tau\) preserves more subgroup-specific variation. With few subgroups, the data may provide little information about \(\tau\), so the estimate and the resulting shrinkage can depend appreciably on modeling choices.

Causal Interpretation and Model Checks

Pooling improves the stability of estimates under the hierarchical model; it does not, by itself, make an estimate causal. The subgroup effects must first be defined for a clear population and effect scale. In a randomized trial, random assignment can support causal comparisons within subgroups, subject to issues such as loss to follow-up and adherence definitions. In observational data, causal interpretation requires suitable identification assumptions, including conditional exchangeability and positivity within the relevant subgroups, as well as a valid analysis of the outcome data. Shrinkage cannot remove confounding, selection bias, or measurement error.

The normal model also deserves scrutiny. Effects may not be well described by a normal distribution, subgroup estimates may have skewed sampling distributions, and a single shared distribution may be inappropriate when groups have distinct known characteristics. In addition, a model fitted on one scale does not establish homogeneity on another scale. For example, shrinking log-odds effects does not mean risk differences are being shrunk by the same amount. State the scale and the model clearly, and interpret results on that scale.

Common Mistake: “Pooling Means the Effects Are the Same”

• Wrong reasoning: “The model uses information from all groups, so it assumes every group has one common treatment effect.”
• Correction: Complete pooling imposes one common effect. Partial pooling instead allows subgroup effects to vary around a shared distribution. The model shrinks uncertain estimates toward the center, with the amount of shrinkage determined by subgroup precision and between-subgroup variation.
• Another mistake: Treating a shrunken estimate as evidence that the original subgroup estimate was false. Shrinkage is a model-based response to uncertainty, not proof that an extreme estimate is an error. If the model is unsuitable, its stabilized estimates can still mislead.