What a Subgroup Estimate Should Communicate
A subgroup result is not just a number with an interval attached. In a partially pooled analysis, the estimate for a subgroup reflects both its own data and information shared through the hierarchical model. A useful report therefore tells readers what the subgroup parameter means, how its posterior estimate was summarized, how uncertain that estimate is, and what the model says about variation across subgroups.
The central reporting challenge is transparency. A posterior mean may be pulled away from the subgroup’s raw estimate toward the distribution shared across subgroups. That movement is part of the model’s inference, not a correction to the observed data. Showing the subgroup data alongside the posterior result helps readers understand the role of borrowing without implying that every subgroup has the same effect.
The parameter \(\theta_j\) must be defined in context before it is reported. For example, it might be the subgroup’s response probability, mean outcome difference in points, or log odds ratio. The estimate and interval should use the same scale unless the report explicitly labels a transformation. A mean on the log-odds scale is not a response probability, and exponentiating a log-odds estimate does not turn it into a probability.
State the Model and the Summary Being Reported
For a continuous subgroup effect, a common hierarchical structure is a sampling model for each subgroup estimate \(y_j\), together with a distribution for subgroup effects:
Here \(s_j\) is the known or estimated sampling standard error for \(y_j\), \(\mu\) is the overall location of the subgroup effects, and \(\tau\) is the between-subgroup standard deviation on the scale of \(\theta_j\). The normal sampling model assumes that the subgroup estimate is approximately normal around its true subgroup effect, with sampling variance \(s_j^2\). The second model assumes that subgroup effects are exchangeable draws from a common distribution, conditional on \(\mu\) and \(\tau\). Exchangeability does not assert that the effects are equal.
In practice, \(\mu\) and \(\tau\) are assigned priors and estimated with the subgroup effects. A reported interval should usually come from the marginal posterior \(p(\theta_j\mid\text{all data})\), which accounts for uncertainty in those hyperparameters. A calculation that fixes \(\mu\) and \(\tau\) is useful for understanding the direction and amount of shrinkage, but does not include that additional uncertainty.
For every subgroup, report the estimate, the interval type and level, and the unit or scale. State whether the point estimate is a posterior mean or median: these can differ for asymmetric posteriors. If the endpoint has a clinically meaningful threshold, name it and define its direction. Do not describe an interval merely as “95% certain”; under the fitted model, a 95% credible interval contains 95% posterior probability for the parameter.
How Partial Pooling Changes an Estimate
The normal model gives a compact illustration. If \(\mu\) and \(\tau\) are fixed and the sampling variance is \(s_j^2\), the posterior distribution of \(\theta_j\) is normal. Its mean is a weighted average of the subgroup estimate and the shared mean. The weight on the subgroup estimate increases when its sampling error is small relative to between-subgroup variation.
The shrinkage weight on the subgroup’s departure from \(\mu\) is \(\tau^2/(\tau^2+s_j^2)\). It is between zero and one when both variances are positive. Noisy subgroup estimates receive more shrinkage toward \(\mu\); precise estimates retain more of their observed departure. When \(\tau\) is small relative to \(s_j\), the estimate is pulled more strongly toward the shared mean. When \(\tau\) is large relative to \(s_j\), it is pulled less.
Worked Example: Showing the Shrinkage Calculation
Suppose a hypothetical continuous outcome is a treatment benefit measured in points. For one subgroup, the observed estimate is \(y_j=0.60\) points with standard error \(s_j=0.20\) points. For illustration only, condition on \(\mu=0.20\) points and \(\tau=0.10\) points.
The weight on the observed subgroup estimate is:
The conditional posterior mean is therefore:
The conditional posterior standard deviation is \(\sqrt{0.01(0.04)/0.05}=\sqrt{0.008}=0.08944\) points. A normal 95% interval is \(0.28\pm1.96(0.08944)\), or approximately \([0.1047,0.4553]\) points. This calculation assumes the hyperparameters are fixed; a full hierarchical analysis would account for their posterior uncertainty. It illustrates why the subgroup estimate moves toward the shared mean, not why the subgroup effect should be treated as equal to it.
Report Heterogeneity Separately
A subgroup interval describes uncertainty about one subgroup’s effect. A heterogeneity summary describes uncertainty about how much effects vary across subgroups. These are different quantities and should not be substituted for one another. In the normal hierarchy, \(\tau\) is the between-subgroup standard deviation on the \(\theta_j\) scale. Reporting its posterior median and credible interval gives a direct summary of uncertainty about that variation.
The value of \(\tau\) should be interpreted in units of the modeled effect. If effects are on a log odds ratio scale, \(\tau\) is also on the log odds ratio scale; it is not a standard deviation of response probabilities. If \(\tau\) has substantial posterior uncertainty, reporting only a point estimate can conceal how little the data determine the degree of heterogeneity. A posterior probability such as \(P(\tau>c\mid\text{data})\) may supplement the interval when \(c\) is a prespecified, scientifically interpretable threshold on that same scale.
A prediction interval for a new subgroup, when scientifically relevant, answers another question: how wide a range of effects is plausible for a subgroup treated as exchangeable with those modeled. It is not the credible interval for the overall mean \(\mu\), nor an interval for a particular observed subgroup’s \(\theta_j\). Keep that distinction explicit if reporting both.
Worked Example: Reporting Posterior Summaries Across Subgroups
Consider a hypothetical basket analysis of mean treatment benefit in points. A hierarchical model is fitted to four subgroups. The following values are illustrative output from posterior draws; the intervals are equal-tail 95% credible intervals. The subgroup sample sizes and observed estimates are included to make the data contribution visible.
| Subgroup | n | Observed estimate (SE), points | Posterior mean, points | 95% credible interval, points |
|---|---|---|---|---|
| A | 18 | 0.60 (0.20) | 0.35 | 0.02 to 0.68 |
| B | 35 | 0.30 (0.14) | 0.28 | 0.04 to 0.52 |
| C | 12 | −0.10 (0.25) | 0.12 | −0.21 to 0.44 |
| D | 42 | 0.20 (0.12) | 0.22 | 0.01 to 0.43 |
A report could state: “Posterior mean treatment benefits ranged from 0.12 to 0.35 points. For subgroup A, the posterior mean was 0.35 points (95% credible interval 0.02 to 0.68); for subgroup C, it was 0.12 points (95% credible interval −0.21 to 0.44). Intervals were obtained from the marginal posterior distributions under the hierarchical model.” This names the summary, interval, units, and model basis.
Suppose the same fit estimates a posterior median \(\tau=0.16\) points, with a 95% credible interval from 0.03 to 0.39 points. Report this separately as uncertainty about between-subgroup standard deviation. Do not call the interval for subgroup C a heterogeneity interval. Nor does the overlap between subgroup intervals establish that the subgroup effects are equal; the posterior distributions describe uncertainty, not a test of equality.
Make the Degree of Borrowing Visible
A posterior estimate alone does not show how much it moved from the subgroup’s own data. A clear report can pair the subgroup sample size and observed estimate with the posterior estimate and interval, using the same effect scale. The contrast helps readers see where estimates changed, while avoiding a claim that the raw estimate is itself the correct final answer. Where useful, a table can also give a shrinkage summary defined consistently with the model and effect scale.
For a continuous outcome, one simple descriptive quantity is the difference between the posterior mean and the observed subgroup estimate. In the preceding example, subgroup A’s difference is \(0.35-0.60=-0.25\) points, while subgroup C’s is \(0.12-(-0.10)=0.22\) points. These differences describe movement in the estimates; they are not measures of treatment-effect benefit or evidence that one subgroup responds better.
For binary outcomes, give counts and denominators alongside the posterior response probability, and specify whether any displayed effect is a probability difference, risk ratio, or odds ratio. The posterior summary must be calculated on the model’s stated scale, with any transformation clearly described. When back-transforming interval endpoints, transform the endpoints as well as the point estimate; do not assume that an interval remains symmetric after transformation.
Worked Example: Reporting a Binary Subgroup Estimate
In a hypothetical subgroup, 8 of 20 participants respond. Suppose a fitted hierarchical beta-binomial model yields a posterior mean response probability of 0.33 and an equal-tail 95% credible interval from 0.17 to 0.51. The raw proportion is \(8/20=0.40\), since \(8\div20=0.40\).
A transparent statement is: “In subgroup A, 8 of 20 participants responded (observed proportion 0.40). Under the hierarchical model, the posterior mean response probability was 0.33 (95% credible interval 0.17 to 0.51).” The interval is on the probability scale, and its interpretation is conditional on the model and prior. The posterior mean being below 0.40 indicates shrinkage relative to this raw proportion; it does not mean the observed count has been changed. The example’s posterior summaries are illustrative model output, not values calculated from the count alone.
Alternatives, Diagnostics, and Reporting Limits
Partial pooling is best understood alongside two alternatives. No pooling estimates each subgroup independently, so subgroup estimates are not drawn toward a shared distribution; this avoids exchangeability assumptions but can leave noisy subgroups with unstable estimates. Complete pooling assumes a common effect for all subgroups, eliminating modeled between-subgroup variation. Partial pooling allows variation while using a shared distribution to stabilize estimates. These alternatives answer different modeling assumptions, not merely different formatting preferences.
| Approach | Subgroup effects | Reporting implication |
|---|---|---|
| No pooling | Estimated separately | Show that subgroup estimates do not borrow across groups; sparse estimates may be imprecise. |
| Complete pooling | Constrained to a common effect | Do not present the common estimate as evidence that subgroup effects were separately estimated. |
| Partial pooling | Related through a hierarchical distribution | Report subgroup posteriors and heterogeneity, and explain that the amount of shrinkage depends on data and model. |
Before reporting posterior summaries, check that computation supports them: inspect convergence diagnostics, effective sample sizes, and trace behavior for subgroup effects and heterogeneity parameters. Look for subgroup estimates that barely move from noisy raw values despite strong apparent sharing, or estimates pulled together despite credible evidence of differences. These patterns can signal weak hierarchical structure, a mismatch between model and data, or an influential prior. Assess sensitivity to reasonable prior choices, especially when few subgroups make \(\tau\) weakly identified.
Partial pooling has limits. Exchangeability may be implausible if subgroups differ systematically in ways omitted from the model. A single heterogeneity parameter may not capture complex variation, and results can depend on the effect scale. Do not claim that borrowing is transparent merely because a hierarchical model was fitted: state the model structure, show the subgroup data and posterior results, and summarize uncertainty in heterogeneity. Avoid ranking subgroups as if their posterior means alone establish differences.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- In a normal hierarchical model, what does \(\tau\) measure, and what units should it have?
- A subgroup’s raw estimate is 0.50 and its posterior mean is 0.32. What can be reported about the movement, and what can’t be concluded from it alone?
- Why should a report distinguish a subgroup credible interval from an interval for heterogeneity?
- What additional information makes borrowing more transparent when reporting a posterior subgroup estimate?
Solutions. (1) \(\tau\) is the between-subgroup standard deviation of effects under the hierarchical distribution; it has the same units as the modeled subgroup effect. (2) The posterior mean is 0.18 units below the raw estimate, a descriptive change consistent with shrinkage. It alone does not show that the subgroup differs from another or that the treatment effect is absent. (3) A subgroup interval concerns uncertainty about one \(\theta_j\); a heterogeneity interval concerns uncertainty about variation across subgroup effects, such as \(\tau\). (4) Give subgroup sample size and observed data or estimate on the same scale, alongside the posterior summary, interval, and a description of the hierarchical model. This shows the data context without treating the unpooled estimate as the final inference.