Posterior Inference for Continuous Outcome Variance

Why Infer the Variance?

In a clinical trial, an endpoint’s mean describes its average level, while its variance describes how much individual outcomes vary around that level. Variance matters when assessing the precision of estimated means and when planning future studies. It is also a parameter with its own uncertainty: a data set may suggest a plausible range of variances rather than pinning down one value.

This tutorial develops posterior inference for the variance of a continuous endpoint when both the population mean and variance are unknown. It uses a conjugate prior, chosen so that the posterior distribution can be written in a familiar form. The mean and variance are updated jointly, but the focus here is the marginal posterior distribution of the variance.

For example, if the endpoint is a change in blood pressure measured in mmHg, then \(\mu\) is measured in mmHg and \(\sigma^2\) in \(\text{mmHg}^2\). A variance is not itself a standard deviation: the standard deviation is \(\sigma=\sqrt{\sigma^2}\), measured in the endpoint’s original units.

The Normal-Inverse-Gamma Prior and Posterior

Use a normal-inverse-gamma prior. Conditional on the variance, the mean has a normal prior; the variance itself has an inverse-gamma prior. We use the shape-scale convention for the inverse-gamma distribution below, and state its density explicitly to prevent confusion with other parameterizations.

Let \(\bar{x}\) be the sample mean and let \(S=\sum_{i=1}^n(x_i-\bar{x})^2=(n-1)s^2\), where \(s\) is the sample standard deviation. The posterior is again normal-inverse-gamma. Its parameters are

In particular, the marginal posterior distribution of the variance is \( \sigma^2\mid x_1,\ldots,x_n\sim\operatorname{InvGamma}(\alpha_n,\beta_n) \). The update to \(\beta_n\) contains two data contributions: variability within the observations, through \(S\), and disagreement between the sample mean and prior mean, through \((\bar{x}-\mu_0)^2\). The latter term accounts for uncertainty about the unknown mean; simply treating the observed deviations from \(\bar{x}\) as deviations from a known population mean would omit this contribution.

Why the Update Has This Form

The likelihood is proportional to \( (\sigma^2)^{-n/2}\exp\{-\sum_i(x_i-\mu)^2/(2\sigma^2)\} \). The identity \( \sum_i(x_i-\mu)^2=S+n(\bar{x}-\mu)^2 \) separates the within-sample variation from the part involving \(\mu\). Multiplying the likelihood by the prior densities gives an exponent whose terms involving the mean can be completed to a square.

The first term on the right supplies the conditional normal posterior for \(\mu\), with precision proportional to \(\kappa_0+n\). The remaining, mean-independent term contributes \( \kappa_0 n(\bar{x}-\mu_0)^2/[2(\kappa_0+n)] \) to the inverse-gamma scale. The likelihood contributes \(n/2\) to the inverse-gamma shape and \(S/2\) to its scale; the prior contributes \(\alpha_0\) and \(\beta_0\). Collecting these terms yields the stated posterior parameters. This derivation also shows why an unknown mean cannot generally be ignored when updating a variance.

Posterior Summaries and Credible Intervals

For \(V\sim\operatorname{InvGamma}(\alpha,\beta)\) under the density convention above, the posterior mode is \(\beta/(\alpha+1)\). The posterior mean exists only when \(\alpha>1\), in which case it is \(\beta/(\alpha-1)\). Its variance exists only when \(\alpha>2\), and is \(\beta^2/[(\alpha-1)^2(\alpha-2)]\). These conditions matter: for a weakly informed prior and a small sample, a posterior distribution can be proper even when its mean or variance is not finite.

A convenient way to obtain interval limits is to use the reciprocal transformation. If \(V\sim\operatorname{InvGamma}(\alpha,\beta)\), then \(1/V\) follows a gamma distribution with shape \(\alpha\) and rate \(\beta\). Equivalently, \(2\beta/V\) has a chi-square distribution with \(2\alpha\) degrees of freedom. If \(\chi^2_{q,\nu}\) denotes the \(q\)-quantile of a chi-square distribution with \(\nu\) degrees of freedom, an equal-tailed \(100(1-\gamma)\%\) credible interval for the variance is

The order of the quantiles reverses because a larger value of \(2\beta_n/\sigma^2\) corresponds to a smaller value of \(\sigma^2\). A 95% posterior credible interval means that, conditional on the model, prior, and observed data, the posterior probability that \(\sigma^2\) lies in the interval is 0.95. It does not mean that the fixed variance has a 95% probability of changing its value or being inside an interval generated repeatedly from data.

Variance Uncertainty Is Not Mean Uncertainty

The variance posterior is positive and typically right-skewed. It is expressed in squared units, and its uncertainty concerns the amount of spread among individual outcomes. By contrast, uncertainty about \(\mu\) concerns the location of the outcome distribution and is expressed in the endpoint’s original units. Under the normal-inverse-gamma model, integrating out \(\sigma^2\) gives a Student \(t\) marginal posterior for \(\mu\), not the inverse-gamma distribution used for \(\sigma^2\).

For the first worked example, the marginal posterior for \(\mu\) has 9 degrees of freedom, location 10 points, and scale \( \sqrt{\beta_n/(\alpha_n\kappa_n)}=\sqrt{9/(4.5\cdot6)}=0.577 \) points. A 95% equal-tailed interval is approximately \(10\pm2.262(0.577)\), or \([8.694,11.306]\) points. This is an interval for the population mean, not for the variance. The variance interval from that example, \([0.946,6.667]\) points squared, answers a different question. Taking the square root of variance limits would produce an interval for the standard deviation, not for the mean.

Model Checks, Alternatives, and Limitations

Conjugacy makes the update transparent, but it does not make its assumptions automatic. Review the endpoint distribution and study design before interpreting the posterior:

• Dependence: repeated measurements or clustering can make observations dependent. Treating them as independent can overstate the information in the sample.
• Non-normal outcomes or outliers: strong skewness, heavy tails, or extreme observations can make a normal likelihood a poor description. Because squared deviations enter the update, extreme values can have substantial influence.
• Unequal variances: if different trial groups or patient subgroups have different spreads, a single common \(\sigma^2\) does not describe all of them.
• Prior sensitivity: \(\alpha_0,\beta_0,\kappa_0,\mu_0\) affect the posterior, especially when \(n\) is small. Examine whether plausible alternative prior choices materially change the variance summaries.

A direct inverse-gamma prior is also conjugate when the population mean is known: the posterior parameters are \(\alpha_0+n/2\) and \(\beta_0+\sum_i(x_i-\mu)^2/2\). That is a different assumption from the unknown-mean model developed here. With an unknown mean, the normal-inverse-gamma prior accounts for uncertainty in both parameters; it should not be replaced by the known-mean update merely because the sample mean is available.

Other prior choices or likelihood models may be preferable when prior beliefs are not well represented by this conjugate family or when the normal model is inadequate. Conjugacy is a computational convenience, not evidence that the assumptions fit. A posterior summary is conditional on the chosen likelihood and prior; it cannot repair an incorrect model or compensate for unrecorded dependence.

Common Mistakes

• Using \(s^2\) as if it were the posterior variance. The sample variance is a data summary, not a posterior distribution. Combine the data with the specified prior using the posterior update.
• Forgetting the sample-mean discrepancy term. When \(\mu\) is unknown, the update for \(\beta_n\) includes the prior-data mean difference. Omitting it silently changes the model calculation.
• Reversing credible-interval limits incorrectly. Inverting chi-square quantiles reverses their order; the larger chi-square quantile gives the lower variance limit.
• Confusing variance, standard deviation, and mean. Variance uses squared units, standard deviation uses the endpoint units, and the mean describes location rather than spread.
• Calling the posterior mean variance a typical standard deviation. The square root of \(E(\sigma^2\mid x)\) is not generally \(E(\sigma\mid x)\). State which posterior summary is being reported.