Posterior Dependence Induced By Shared Hyperparameters

Why Shared Hyperparameters Link Subgroups

In a hierarchical basket-trial model, each subgroup has its own treatment effect or response rate, but the subgroup-specific parameters are related through shared hyperparameters. Even when the subgroup data are independent conditional on those hyperparameters, uncertainty about the shared quantities can make the subgroup effects dependent in the posterior. As a result, evidence from one basket can change the posterior distribution for another basket’s effect.

This is a form of information borrowing, but it is not the same as simply combining all patients into one analysis. The amount and direction of the influence depend on what the shared hyperparameters represent, how uncertain they are, and how the data update them. The key distinction is between conditioning on a fixed hyperparameter value and averaging over uncertainty about that value.

Definition: Subgroup effects $\theta_1,\ldots,\theta_J$ are conditionally independent given shared hyperparameters $\eta$ if their joint posterior distribution, conditional on $\eta$ and all subgroup data $D$, factors as $$ p(\theta_1,\ldots,\theta_J\mid \eta,D) = \prod_{j=1}^{J}p(\theta_j\mid \eta,D_j). $$ They are posterior dependent after averaging over $\eta$ when their joint posterior generally does not factor into the product of their separate marginal posteriors. The dependence arises because the same uncertain $\eta$ is used for multiple subgroups.

How Evidence Travels Between Subgroups

Write $D_j$ for the data from subgroup $j$, and $D=(D_1,\ldots,D_J)$ for all subgroup data. In a conditionally independent hierarchical model, the posterior distribution of the shared hyperparameters has the form

p(\eta\mid D)\propto p(\eta)\prod_{k=1}^{J}m_k(D_k\mid\eta), \qquad m_k(D_k\mid\eta)=\int p(D_k\mid\theta_k)p(\theta_k\mid\eta)\,d\theta_k.

Here $m_k(D_k\mid\eta)$ is the marginal likelihood contribution from subgroup $k$, obtained by integrating out its subgroup-specific parameter. Thus data from subgroup $k$ can change $p(\eta\mid D)$. The posterior for another subgroup $j$ averages its conditional posterior over this updated distribution:

p(\theta_j\mid D) = \int p(\theta_j\mid \eta,D_j)\,p(\eta\mid D)\,d\eta.

The data $D_k$ do not usually enter the conditional distribution $p(\theta_j\mid\eta,D_j)$ directly when $\eta$ is fixed. They affect it indirectly by changing the posterior weights assigned to possible values of $\eta$. If $\eta$ were known exactly, this route for cross-subgroup influence would disappear.

Worked Example: A Shared Concentration Hyperparameter

Consider two hypothetical baskets with two binary outcomes each. Let $X_A$ and $X_B$ be the numbers of responses, and let $\theta_A$ and $\theta_B$ be their response probabilities. A shared, unknown indicator $H$ selects one of two prior concentrations:

P(H=S)=P(H=W)=0.5,\qquad \theta_A,\theta_B\mid H \text{ are independent},\qquad \theta_j\mid H=S\sim\operatorname{Beta}(10,10),\quad \theta_j\mid H=W\sim\operatorname{Beta}(1,1).

Both priors have mean $0.5$. The $S$ prior is more concentrated around that mean; the $W$ prior allows more variation in subgroup response probabilities. Suppose the hypothetical results are $X_A=2$ of $2$ responses and $X_B=0$ of $2$.

For $X_j=x$ out of $n=2$, the beta-binomial marginal probability under a $\operatorname{Beta}(a,b)$ prior is $\binom{2}{x}B(a+x,b+2-x)/B(a,b)$. For either all-response or no-response data, the binomial coefficient is 1. Under $H=S$,

P(X_A=2\mid S)=P(X_B=0\mid S) =\frac{10\cdot11}{20\cdot21} =\frac{11}{42}\approx0.2619.

Under $H=W$, each of those probabilities is \[ \frac{1\cdot2}{2\cdot3}=\frac{1}{3}\approx0.3333. \] Using only basket A, Bayes’ rule gives

P(S\mid X_A=2) = \frac{(1/2)(11/42)} {(1/2)(11/42)+(1/2)(1/3)} =\frac{11}{25}=0.44.

Conditional on $S$, basket A’s posterior is $\operatorname{Beta}(12,10)$, with mean $12/22=6/11$. Conditional on $W$, it is $\operatorname{Beta}(3,1)$, with mean $3/4$. Therefore its posterior mean using basket A alone is

E(\theta_A\mid X_A=2) = \frac{11}{25}\frac{6}{11} +\frac{14}{25}\frac{3}{4} =0.6600.

Now include basket B. Its zero responses update the same $H$. Because the subgroup data are conditionally independent given $H$, the likelihood contributions multiply:

P(S\mid X_A=2,X_B=0) = \frac{(11/42)^2}{(11/42)^2+(1/3)^2} =\frac{121}{317}\approx0.3817.

The posterior mean for $\theta_A$ becomes

E(\theta_A\mid X_A=2,X_B=0) = \frac{121}{317}\frac{6}{11} +\frac{196}{317}\frac{3}{4} =\frac{213}{317}\approx0.6719.

Basket B has changed basket A’s posterior mean from $0.6600$ to $0.6719$, even though basket A’s own data did not change. The zero responses in B make the less-concentrated prior regime more plausible, which reduces the shrinkage of A’s estimate toward $0.5$. This change is a consequence of shared uncertainty about $H$, not of treating the two baskets as having the same response probability.

A Continuous-Outcome Illustration

The same mechanism appears in a normal-normal hierarchy. Suppose the observed subgroup estimates $Y_j$ satisfy $Y_j\mid\theta_j\sim N(\theta_j,s_j^2)$, independently across subgroups given their effects, and $\theta_j\mid\mu\sim N(\mu,\tau^2)$, independently given a shared mean $\mu$. Assume $\tau^2$ is known and $\mu\sim N(m_0,A^2)$. Here $s_j$ is the known standard error for subgroup $j$, and $A^2$ describes prior uncertainty about the shared mean.

Worked Example: A Shared Mean Changes a Basket Estimate

Suppose two hypothetical subgroups have continuous treatment-effect estimates $Y_1=6$ and $Y_2=2$, both with known standard error $1$ outcome point. Set $m_0=4$ outcome points, $A^2=4$ outcome-points squared, and $\tau^2=1$ outcome-points squared.

Given $\mu$ and $Y_1$, normal updating gives a posterior mean for $\theta_1$ equal to \[ \frac{\tau^2}{\tau^2+s_1^2}Y_1+ \frac{s_1^2}{\tau^2+s_1^2}\mu =0.5Y_1+0.5\mu. \] To average over $\mu$, first note that $Y_j\mid\mu\sim N(\mu,\tau^2+s_j^2)=N(\mu,2)$. Using only subgroup 1, the posterior mean of $\mu$ is

E(\mu\mid Y_1=6) = \frac{(1/4)4+(1/2)6}{1/4+1/2} =\frac{4}{3/4} =\frac{16}{3}\approx5.333.

The resulting posterior mean for subgroup 1 is $0.5(6)+0.5(16/3)=17/3\approx5.667$ outcome points. With both subgroups, the shared mean has posterior mean

E(\mu\mid Y_1=6,Y_2=2) = \frac{(1/4)4+(1/2)6+(1/2)2}{1/4+1/2+1/2} =\frac{5}{5/4}=4.

Thus $E(\theta_1\mid Y_1=6,Y_2=2)=0.5(6)+0.5(4)=5.000$ outcome points. The estimate in subgroup 2 moves the posterior for the shared mean, which in turn moves the posterior for subgroup 1. These are model-based estimates under the stated normal assumptions; the calculations do not establish that the treatment effect is causal.

What the Dependence Does—and Does Not—Mean

The direction of cross-subgroup influence depends on the shared parameter. An uncertain shared mean tends to move subgroup estimates toward the updated common center: evidence for a lower mean in one basket can pull estimates in other baskets downward. An uncertain heterogeneity parameter has a different role. Evidence about how variable subgroup effects are can alter the degree of shrinkage applied across baskets. In the beta-binomial example, evidence changed the posterior weight on a shared concentration regime, thereby changing the amount of shrinkage.

There is no universal rule that posterior dependence must be positive, or that evidence from one subgroup must move another subgroup’s estimate toward that subgroup’s observed value. The effect depends on the likelihood, the hyperparameterization, the other data, and the prior. In particular, evidence may change a subgroup’s posterior uncertainty substantially even when its posterior mean changes little.

Conditional independence is also not the same as marginal independence. If the hyperparameters were known, the subgroup-specific posterior distributions in the examples would factor given their own data. Once uncertainty about the shared quantities is included, the subgroup posteriors generally no longer factor. This distinction is important when interpreting separate subgroup summaries: those summaries are linked through their common posterior distribution for the hyperparameters.

Assumptions, Diagnostics, and Alternatives

Cross-subgroup borrowing is justified only to the extent that the hierarchical model is appropriate. The model must specify a meaningful shared structure, and its likelihood and prior distributions must adequately represent the data and plausible subgroup variation. Conditional independence of subgroup data given effects is a modeling assumption, not a guarantee; shared controls, common measurement processes, or other dependencies may require additional terms in the likelihood.

Useful checks focus on whether the shared structure is driving conclusions in a credible way. Examine the posterior distribution of the hyperparameters, not just subgroup-specific summaries. Check whether a small number of baskets dominate that distribution, whether it is broad or multimodal, and whether the subgroup conclusions change under reasonable alternative hyperpriors. For computational fits, inspect convergence and effective sample sizes for both hyperparameters and subgroup effects; poor exploration of a hyperparameter can make apparent borrowing unreliable. Prior-predictive checks and posterior-predictive checks can reveal whether the model generates plausible subgroup response patterns and heterogeneity.

A highly concentrated hyperprior can make the hyperparameter behave almost as if it were known, limiting the amount of learning from the baskets. Conversely, sparse subgroup data may leave the shared hyperparameters weakly identified, so the prior can have substantial influence. A poorly chosen exchangeability assumption can also produce misleading borrowing when baskets differ in clinically important ways. Sensitivity analysis is therefore part of interpretation, not an optional repair after a result appears surprising.

The main alternatives make different assumptions. Complete pooling imposes a common subgroup effect and does not represent genuine between-subgroup variation. No pooling estimates each subgroup separately and avoids this form of borrowing, but can be imprecise when subgroup data are sparse. Partial pooling uses shared hyperparameters to allow both subgroup variation and information sharing. These approaches are not interchangeable: the appropriate choice depends on the scientific rationale for relating the subgroups and the consequences of misspecification.

Common mistake: “The subgroup likelihoods are independent given the hyperparameters, so the posterior for one subgroup cannot depend on another subgroup’s data.” The correction is that conditional independence holds only after fixing the shared hyperparameters. In the marginal posterior, another subgroup’s data can update those hyperparameters, and the updated hyperparameter distribution changes the first subgroup’s posterior.

Key takeaway: Shared hyperparameters create a pathway for information transfer: another subgroup’s data update the hyperparameter posterior, and that update changes the distribution used to infer each subgroup effect. The strength and direction of the change are model-dependent.

Worked Example: Checking the Role of a Fixed Hyperparameter

Use the normal model from above, but suppose the shared mean is known to be $\mu=4$ outcome points, with $\tau^2=s_1^2=1$. For subgroup 1, whose estimate is $Y_1=6$, the posterior mean is

E(\theta_1\mid Y_1=6,\mu=4) = \frac{\tau^2}{\tau^2+s_1^2}Y_1+ \frac{s_1^2}{\tau^2+s_1^2}\mu =0.5(6)+0.5(4)=5.

If subgroup 2’s estimate changes from $2$ to $4$, this conditional posterior mean remains $5$, because $\mu$ is fixed and subgroup 2’s data do not enter subgroup 1’s conditional likelihood. If instead $\mu$ is uncertain, subgroup 2 can update its posterior, as in the preceding example. This comparison isolates the source of the cross-subgroup influence: uncertainty about a shared hyperparameter.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

Why can subgroup effects be conditionally independent given a shared hyperparameter yet dependent in the posterior after that hyperparameter is averaged over?
In the beta-binomial example, what feature of basket B’s data changes basket A’s posterior mean?
In a normal hierarchy with known $\mu$, can subgroup 2’s data alter subgroup 1’s posterior if the subgroup data are conditionally independent? Explain.
Does a shared heterogeneity parameter guarantee that evidence from one basket moves every other basket’s estimate in the same direction? Why or why not?

Solutions. (1) Given the hyperparameter, each subgroup posterior uses its own data and the fixed shared value. After averaging, all subgroup posteriors depend on the same uncertain hyperparameter; data from one subgroup can update its distribution and thereby affect the others. (2) Basket B’s zero responses change the posterior probability of the shared concentration regime $H$, changing the mixture of conditional posteriors for $\theta_A$. (3) No. With known $\mu$ and the stated conditional-independence assumptions, subgroup 2’s data do not enter subgroup 1’s conditional posterior. (4) No. The direction depends on which hyperparameter is shared, the likelihood and prior, and the observed data; heterogeneity learning can change shrinkage in ways that do not imply a universal direction of movement.