Normal Mean Inference With Unknown Variance

Why Model the Mean and Variance Together?

A continuous clinical endpoint, such as change in a laboratory measurement, varies across participants. When the endpoint is modeled as normal, its mean describes the location of the outcome distribution and its variance describes its spread. In many small trials, neither quantity is known. Treating the variance as if it were known can understate uncertainty about the mean, especially when the data provide little information about spread.

The normal-inverse-gamma model gives a conjugate Bayesian update for the mean and variance together. Conjugacy means that, after observing normally distributed data, the joint posterior retains the same distributional form as the prior. The update depends on the sample size, sample mean, and within-sample sum of squares.

Sampling Model and Prior

Let \(y_1,\ldots,y_n\) denote continuous endpoint measurements from \(n\) participants. Assume that, conditional on the unknown mean \(\mu\) and variance \(\sigma^2\), the observations are independent and identically distributed normal random variables:

The common variance assumption means that each participant has the same conditional outcome variance. Independence rules out dependence between participants that would need to be modeled, such as clustering or repeated measurements. Normality is an assumption about the outcome distribution conditional on the model parameters; random assignment alone does not establish it.

Use the following prior specification, where all hyperparameters are fixed before observing these endpoint data:

Here \(\kappa_0>0\), \(\alpha_0>0\), and \(\beta_0>0\). We use the inverse-gamma density \(p(\sigma^2)\propto(\sigma^2)^{-(\alpha_0+1)}\exp(-\beta_0/\sigma^2)\) for \(\sigma^2>0\). Thus \(\beta_0\) is the scale in the inverse-gamma density as parameterized here. The conditional prior mean of \(\mu\) given \(\sigma^2\) is \(\mu_0\), and its conditional variance is \(\sigma^2/\kappa_0\). The marginal prior mean of \(\mu\) is \(\mu_0\) only if \(\alpha_0>1/2\); for smaller \(\alpha_0\), that marginal mean does not exist.

The units help check a prior specification. If \(y_i\) and \(\mu_0\) are measured in milligrams per deciliter, then \(\sigma^2\) and \(\beta_0\) have squared units, while \(\kappa_0\) and \(\alpha_0\) are dimensionless.

Deriving the Posterior Update

Write \(\bar{x}=n^{-1}\sum_{i=1}^n y_i\) for the sample mean and \(S=\sum_{i=1}^n(y_i-\bar{x})^2\) for the within-sample sum of squares. The likelihood can be expressed using these two summaries and the sample size:

Multiplying the likelihood by the prior gives terms involving \(\mu\) and \(\sigma^2\). To collect the terms involving \(\mu\), complete the square:

where \(\kappa_n=\kappa_0+n\) and \(\mu_n=(\kappa_0\mu_0+n\bar{x})/\kappa_n\). The first term on the right gives the updated conditional normal distribution for \(\mu\). The remaining term contributes to the updated inverse-gamma parameter, along with the observed within-sample sum of squares.

To see why the last expression is the correct update, the likelihood contributes the factor \((\sigma^2)^{-n/2}\), increasing the inverse-gamma shape by \(n/2\). The data contribute \(S/2\) to the inverse-gamma scale. Completing the square contributes the additional nonnegative term \(\kappa_0n(\bar{x}-\mu_0)^2/[2(\kappa_0+n)]\), which measures disagreement between the prior location and observed mean, adjusted by their relative information.

Posterior Summaries

The posterior center \(\mu_n\) is a weighted average of the prior location and sample mean. The weights are \(\kappa_0\) and \(n\), respectively. This describes the center of the conditional posterior for \(\mu\) at every \(\sigma^2\); it is not by itself a complete description of uncertainty about \(\mu\).

Integrating out \(\sigma^2\) gives a Student \(t\) marginal posterior for \(\mu\), with \(2\alpha_n\) degrees of freedom, location \(\mu_n\), and scale \(\sqrt{\beta_n/(\alpha_n\kappa_n)}\). Its mean exists when \(\alpha_n>1/2\), in which case it equals \(\mu_n\). Its variance exists when \(\alpha_n>1\), in which case it is \(\beta_n/[\kappa_n(\alpha_n-1)]\). The posterior mean of \(\sigma^2\) exists when \(\alpha_n>1\), and then equals \(\beta_n/(\alpha_n-1)\).

Assumptions, Diagnostics, and Alternatives

The update is exact when the sampling model and prior specification apply. In a trial, check that the endpoint is measured on a scale for which a normal model is plausible, that participants’ observations are conditionally independent, and that a common variance is defensible. Inspect the observed distribution and residuals for strong skewness, extreme outliers, or multiple clusters. With very small \(n\), such checks have limited power, so clinical understanding of the endpoint and sensitivity to plausible prior choices matter.

Also check the computation itself. The posterior shape must be \(\alpha_0+n/2\), not \(\alpha_0+(n-1)/2\). The scale uses \(S/2\), where \(S\) is the sum of squared deviations, not the sample standard deviation alone. Every term added to \(\beta_0\) must have squared endpoint units and be nonnegative. A negative update term or a scale expressed in the endpoint’s unsquared units signals an error.

Compared with the known-variance model, this approach propagates uncertainty about \(\sigma^2\) into the marginal posterior for \(\mu\), producing a Student \(t\) distribution rather than a normal marginal posterior. Plugging in a sample variance and then proceeding as if it were fixed is another possible approximation, but it does not account for uncertainty in that estimate. The joint conjugate model is convenient and transparent, but its common-variance normal assumptions may be unsuitable for skewed endpoints, heavy tails, mixtures, or clustered observations; those settings require an appropriate alternative model.

Common mistake: A reader may say, “The sample variance is zero for one participant, so the posterior variance must be zero.” That reasoning confuses observed within-sample spread with uncertainty about the population variance. With one observation, \(S=0\), but the posterior scale still includes \(\beta_0\) and the prior-data location adjustment. The posterior for \(\sigma^2\) remains a distribution, not a point mass at zero.