Sensitivity Analysis for Variance-Component Priors

Why Compare Variance-Component Priors?

A variance-component prior is not merely a technical detail. It expresses how much group-level variability is plausible before observing the outcome data. When the data contain limited information about that variability, reasonable prior choices can lead to noticeably different posterior estimates or uncertainty. Sensitivity analysis makes this dependence visible rather than allowing one chosen prior to appear inevitable.

A fair comparison changes the prior under examination while holding the likelihood, data, and other prior choices fixed. In this tutorial, \(\tau\) denotes a nonnegative random-effect standard deviation, so the corresponding variance component is \(\tau^2\). For a scalar random effect \(b_j\), the usual scale interpretation is \(b_j\mid\tau\sim N(0,\tau^2)\). The focus is comparing plausible prior distributions for \(\tau\), not choosing a specific prior for a particular random-intercept or random-slope structure.

For a GLMM, let \(y\) denote the observed responses and let \(b\) collect the random effects. Conditional on \(b\) and the fixed-effect parameters, the response distribution and link function specify the likelihood. For a candidate prior \(\pi_k(\tau)\), the posterior for \(\tau\) is based on the marginal likelihood

The displayed integral allows nuisance parameters to be integrated under the same specification in every candidate analysis. The posterior exists as a probability distribution when its normalizing integral is finite and positive. The likelihood, the data, and the treatment of nuisance parameters must be comparable across analyses; otherwise, differences cannot be attributed to the variance-component prior alone.

Matching Prior-Predictive Scales

If \(b_j\mid\tau\sim N(0,\tau^2)\), then integrating over a prior on \(\tau\) gives \(E(b_j^2)=E(\tau^2)\), provided this second moment is finite. Thus \(\sqrt{E(\tau^2)}\) is the prior-predictive root-mean-square (RMS) scale of a random effect. Matching this quantity is one useful way to put alternative priors on a common scale before examining the outcome data.

This calibration does not make the priors equivalent. Priors with the same \(E(\tau^2)\) can place different probability near zero, have different medians, and allocate different probability to large values. Those differences are often precisely what the sensitivity analysis should reveal. Matching the RMS is a reference point, not a proof that either prior is appropriate.

Posterior Comparisons: A Fully Worked Illustration

The next hypothetical calculation uses a simple one-dimensional Gaussian summary likelihood to make the prior-to-posterior arithmetic transparent. It is an illustrative likelihood, not a claim that a binary or count GLMM has this likelihood. Assume an observed summary \(Y=1\) has likelihood, as a function of \(\tau\),

This likelihood is proportional to the density of a normal observation with mean zero and variance \(\tau^2+0.25\), evaluated at one; the common normalizing constant is omitted because it cancels in posterior probabilities. Apply it to the two RMS-matched priors from the previous example. The assumptions for this calculation are that the likelihood is the same for both analyses, the prior probabilities are as stated, and the priors are proper. For a real GLMM, the corresponding marginal likelihood must be computed from that model’s actual response likelihood and random-effect distribution.

What a Sensitivity Analysis Should Compare

A useful analysis goes beyond choosing a prior that yields a preferred answer. First specify a small set of scientifically defensible candidates and explain what differs between them. Then fit each version of the same GLMM, using the same data, fixed-effect specification, likelihood, and nuisance-parameter priors. Compare quantities that answer the scientific question, not only a single summary of \(\tau\).

For each candidate, useful reports may include the posterior median or mean of \(\tau\), a credible interval, and posterior probabilities for scientifically meaningful ranges, such as \(P(\tau>c\mid y)\) for a stated threshold \(c\). State the units and describe the event in context. A 95% posterior credible interval is an interval containing 95% posterior probability under the specified model and prior; it is not a prior-free guarantee. When the posterior summaries are similar across well-calibrated priors, that is evidence of limited sensitivity for those summaries and those candidates. It does not prove that every plausible prior would give the same result.

The same scale calibration can still allow substantial differences in tail probabilities. For example, compare a half-normal prior with scale \(s=0.8\) to an exponential prior with rate \(\lambda=\sqrt{2}/0.8\). Their second moments are \(s^2=0.64\) and \(2/\lambda^2=0.64\), respectively, so both have RMS \(0.8\). Yet their probabilities of exceeding \(1\) differ:

Assumptions, Diagnostics, and Failure Modes

Interpret a comparison as prior sensitivity only when its conditions are controlled. The random-effect distribution and definition of \(\tau\) must be the same across candidate fits; the likelihood and data must be unchanged; and the priors on other parameters must also be unchanged unless their joint prior specification is the object of the analysis. Candidate priors should be proper for the posterior comparisons described here, and the matched second moment must exist. The chosen scale is meaningful only relative to the random-effect parameterization, so any rescaling of predictors or random effects must be applied consistently to every prior.

For fitted GLMMs, also verify that posterior computation is adequate under each prior. Check sampler convergence and effective sample sizes when using simulation-based inference, and compare Monte Carlo error with the differences being interpreted. Inspect prior-predictive implications for group effects and, where appropriate, the resulting response probabilities or counts. A prior can have a seemingly reasonable RMS yet imply implausible extremes or too much concentration near zero. If the posterior remains close to the prior, the data may be weakly informative about that component; if computational diagnostics differ markedly between candidates, apparent sensitivity may instead reflect unreliable computation.

A prior with a continuous density on positive \(\tau\) assigns probability zero to the exact event \(\tau=0\). Consequently, posterior probability for a meaningful small-scale range is not the same as posterior probability of an exactly zero variance component. Point-mass priors address that distinct model-comparison question; do not interpret a continuous-prior sensitivity analysis as a test that establishes a positive component.

The main alternatives to RMS matching include matching a prior median, matching a chosen upper quantile, or calibrating a tail probability with subject-matter input. Each emphasizes a different feature of the prior. RMS matching is particularly interpretable when the target is the random-effect magnitude, but it can be sensitive to tail mass because it uses a second moment. Quantile or threshold matching can be more directly tied to plausible ranges, though those choices do not guarantee a common average squared effect. No calibration rule removes the need to justify the candidate priors and examine their prior-predictive implications.

Sensitivity analysis has limits. It cannot compensate for a misspecified conditional response distribution, omitted structure, poor covariate scaling, or a random-effect distribution that is unsuitable for the problem. Nor does agreement across a few candidates establish robustness to all possible priors. Its value is narrower and practical: it identifies which posterior conclusions are stable across a transparent set of plausible prior assumptions and which warrant cautious reporting.