Choosing Priors for Nuisance Parameters

Why Nuisance Priors Deserve Attention

A clinical trial analysis often targets a treatment contrast, such as a risk difference, log odds ratio, or mean difference. The model also contains parameters that are needed to describe the data but are not themselves the main clinical question. Examples include the control-group event probability, a regression intercept, and the residual standard deviation. These are nuisance parameters relative to the chosen estimand.

“Nuisance” does not mean “irrelevant.” The likelihood for a treatment effect can depend strongly on the baseline rate or outcome variability. With sparse events, for example, the estimated treatment contrast may be sensitive to assumptions about the baseline probability. In a continuous-outcome model, the standard deviation directly determines how much information the observations provide about a mean difference. A prior on a nuisance parameter can therefore change the posterior uncertainty, and sometimes the posterior location, of the treatment effect.

The choice depends on the endpoint and likelihood. For a binary endpoint, a baseline event probability belongs to \((0,1)\). For a logistic regression, the intercept belongs to the log-odds scale. For a normally distributed continuous endpoint, a standard deviation is positive and measured in the outcome’s units. A prior must respect those constraints and represent plausible values for the population, endpoint definition, and follow-up period being modeled.

Baseline Event Probabilities

For a simple control arm with a fixed follow-up period, suppose each of \(n\) participants has an independent Bernoulli outcome with common event probability \(p_0\). If \(X\) is the number of events, then \(X\mid p_0\sim\operatorname{Binomial}(n,p_0)\). A beta prior is a natural model for the unknown probability:

The beta distribution has support on \((0,1)\), and its shape can express different degrees of concentration or skewness. Its parameters are not automatically literal counts of prior patients; that interpretation is a useful algebraic analogy in the beta-binomial model, not a claim that the prior data actually existed. The baseline probability must also match the trial’s definition of an event and its time horizon. A 30-day event probability is not interchangeable with a one-year probability.

A baseline prior should not be carried mechanically from a different clinical setting. Changes in eligibility criteria, background care, outcome ascertainment, or follow-up can alter the relevant event probability. A prior that is well calibrated for one target population may be too concentrated or centered incorrectly for another. The earlier material on priors for baseline event probabilities provides the basic probability-scale formulations; here the key concern is their role as inputs to a treatment-effect model.

Intercept Priors in Regression Models

In a two-arm logistic model, let \(T_i=0\) indicate control and \(T_i=1\) indicate treatment. A common specification is \(\operatorname{logit}\{P(Y_i=1\mid T_i)\}=\alpha+\beta T_i\). Here \(\alpha\) is the control-arm log odds and \(\beta\) is the treatment log odds ratio. The intercept is a nuisance parameter when the primary estimand is \(\beta\). A normal prior on \(\alpha\) is a prior on log odds, not a normal prior on the event probability.

A normal prior on \(\alpha\) induces a distribution on the baseline probability through the inverse-logit transformation. Its center on the log-odds scale transforms to the median probability, because the inverse-logit function is increasing; it need not transform to the mean probability. Therefore, report the scale and parameterization, and check what probabilities the prior allows. For example, \(\alpha\sim N(-0.980,0.707^2)\) has an approximate central 95% interval \(-0.980\pm1.96(0.707)\), or \([-2.366,0.406]\). Transforming its endpoints gives \(\operatorname{logit}^{-1}(-2.366)\approx0.0858\) and \(\operatorname{logit}^{-1}(0.406)\approx0.6001\).

Independent priors for \(\alpha\) and \(\beta\) are convenient, but independence is an assumption, not a universal truth. If credible clinical information links baseline risk and treatment effect, or if a model’s parameterization makes such dependence important, an independent specification may be inadequate. Conversely, imposing a correlation without substantive or modeling justification adds assumptions. The practical check is to examine whether plausible baseline values combined with plausible treatment effects produce coherent arm-specific risks.

Variance Priors for Continuous Outcomes

For a continuous endpoint, consider a simple model in which observations in each arm are conditionally independent and normally distributed with a common residual variance: \(Y_{ij}\mid\mu,\tau,\sigma\sim N(\mu+\tau T_i,\sigma^2)\). Here \(T_i\) is the treatment indicator, \(\tau\) is the treatment mean difference, and \(\sigma\) is the residual standard deviation. The variance is \(\sigma^2\), while \(\sigma\) is positive and has the same units as the outcome. A prior specified for one must not be interpreted as a prior for the other.

Positive-scale choices include a half-normal distribution or a half-\(t\) distribution for \(\sigma\). These are distributions for a nonnegative standard deviation, with scale chosen in outcome units. A prior on \(\log(\sigma)\) is another option and induces a positive, often right-skewed distribution for \(\sigma\). Inverse-gamma priors are sometimes assigned directly to \(\sigma^2\), but their behavior depends strongly on both shape and scale; a seemingly vague choice can still concentrate substantial mass in scientifically consequential regions. The earlier material on priors for continuous-outcome variability covers the basic formulations.

Justification, Diagnostics, and Alternatives

The preceding examples illustrate why nuisance priors can affect treatment inference. In the normal two-arm model, with independent observations and equal arm sizes \(n\), the variance of the difference in sample means conditional on \(\sigma\) is \(\sigma^2/n+\sigma^2/n=2\sigma^2/n\). Therefore its standard error is \(\sigma\sqrt{2/n}\). Increasing plausible values of \(\sigma\) increases uncertainty in the treatment contrast. In the binary model, the probability of each possible count depends on the baseline rate as well as the treatment contrast; when event counts are sparse, the data may not sharply distinguish those parameters. Integrating over uncertainty in a nuisance parameter can consequently alter the posterior for the target parameter.

Several checks help reveal a poor specification:

• Check the scale and support. A probability prior must remain between zero and one; a standard deviation must be positive and measured in the endpoint’s units. Check that transformations are applied with the correct sign.
• Check concentration. A tightly concentrated prior can dominate weakly informative trial data. Ask whether the implied range is credible given population, endpoint, and follow-up.
• Check the likelihood assumptions. A common binomial probability may be unsuitable with differing follow-up or substantial participant heterogeneity. A common normal residual variance may be unsuitable if arm variances differ materially.
• Check treatment-effect consequences. Assess whether plausible nuisance values materially change the posterior location or uncertainty of the primary estimand. Large changes indicate that conclusions rely on nuisance-prior assumptions and should be reported as such.

No single parameterization is best in every setting. A beta prior directly models a probability and is convenient for a binomial likelihood; a normal prior on the logit of a probability can be convenient in a regression model with covariates. A half-normal or half-\(t\) prior expresses beliefs about a standard deviation on its original scale; a prior on log standard deviation gives a different shape and interpretation. In each comparison, the question is not which distribution sounds least informative, but which prior yields defensible values on the scale relevant to the model.

A limitation is that a prior that seems broad marginally may still create consequential joint behavior when combined with a treatment-effect prior. Likewise, a marginally reasonable baseline prior can be poorly matched to a model that has important predictors or varying follow-up. Nuisance-prior checks cannot repair a misspecified likelihood, and no prior choice eliminates the need to state model assumptions. The introductory material on baseline probabilities and continuous-outcome variability provides the foundational distributions; the additional task here is to check how those assumptions feed into treatment-effect estimation.