Normal-Inverse-Gamma Updates For Unknown Mean And Variance

Why update mean and variance together?

A continuous trial endpoint is often modeled as normally distributed within a treatment arm. If the outcome variance is unknown, uncertainty about the mean cannot be separated from uncertainty about that variance: the precision of the estimated mean depends on the variance, while the observed spread informs the variance. The Normal-Inverse-Gamma prior provides a conjugate update for both quantities and summarizes the data through the sample size, sample mean, and centered sum of squares.

This tutorial uses a single arm with a continuous endpoint. It does not by itself estimate a treatment contrast or account for covariates, repeated measurements, or unequal variances across arms. Those require additional model structure.

Model and prior parameterization

Let \(Y_1,\ldots,Y_n\) be the endpoint values for \(n\) trial participants in one arm. Assume, conditional on the unknown mean \(\mu\) and variance \(\sigma^2\), that the observations are independent and identically distributed:

Thus, this model assumes a common mean and variance for the observations in the arm and conditional independence across participants. The outcome is treated as continuous and normally distributed; a markedly skewed or heavy-tailed outcome may not be adequately described by this likelihood.

Use the following Normal-Inverse-Gamma prior parameterization. Conditional on \(\sigma^2\), the prior for \(\mu\) is normal, and the prior for \(\sigma^2\) is inverse-gamma:

The factor \(\kappa_0\) controls the conditional prior variance of \(\mu\): given \(\sigma^2\), that variance is \(\sigma^2/\kappa_0\). The inverse-gamma convention matters. In particular, \(\beta_0\) is the scale appearing in the exponential as \(-\beta_0/\sigma^2\), not a rate multiplying \(\sigma^2\).

For the observed values, define the sample mean and centered sum of squares by

If the sample standard deviation \(s\) is reported instead, then \(S=(n-1)s^2\). The update uses this centered sum of squares, not the sum of squared deviations from zero.

Posterior update and its derivation

In this posterior, \(\mu\mid\sigma^2,x\) is normal with mean \(\mu_n\) and variance \(\sigma^2/\kappa_n\), while \(\sigma^2\mid x\) is \(\operatorname{Inv\text{-}Gamma}(\alpha_n,\beta_n)\).

To see why these are the updates, the likelihood kernel as a function of \(\mu\) and \(\sigma^2\) is proportional to \((\sigma^2)^{-n/2}\exp\{-\sum_i(x_i-\mu)^2/(2\sigma^2)\}\). The prior contributes a conditional-normal kernel and an inverse-gamma kernel. Two identities combine the terms involving \(\mu\):

The first identity separates the within-sample spread from the difference between the sample mean and \(\mu\). The second completes the square in \(\mu\). Together they show that the posterior normal kernel has precision multiplier \(\kappa_0+n\) and center \(\mu_n\). The remaining constant terms in the exponent add to \(\beta_0+S/2+\{\kappa_0n/(\kappa_0+n)\}(\bar{x}-\mu_0)^2/2\), giving \(\beta_n\). Finally, the likelihood contributes the factor \((\sigma^2)^{-n/2}\), so the inverse-gamma shape increases by \(n/2\), giving \(\alpha_n\). These are precisely the kernels of the stated posterior distributions.

Worked updates from summaries and in stages

Interpretation, diagnostics, and alternatives

The update has several features that help with interpretation. The posterior location \(\mu_n\) is a weighted average of the prior location and sample mean, with weights \(\kappa_0\) and \(n\). The posterior precision multiplier \(\kappa_n\) adds the prior multiplier and the sample size. The shape parameter increases by \(n/2\). The inverse-gamma scale increases by half the observed centered sum of squares and by a nonnegative term for the distance between \(\bar{x}\) and \(\mu_0\).

As a diagnostic, check the model and the inputs before trusting the formulas. Confirm that observations use the same endpoint definition and units, that each participant contributes one independent value under the assumed model, and that a common normal distribution is plausible for the arm. Inspect the distribution for strong skewness, extreme outliers, or distinct subgroups that could make a single normal mean and variance misleading. Also verify \(\kappa_0>0\), \(\alpha_0>0\), and \(\beta_0>0\), and check that \(S\geq 0\) and \(\beta_n\geq\beta_0\). A negative centered sum of squares or a posterior scale smaller than \(\beta_0\) signals a calculation or data-summary error.

The known-variance normal update uses a fixed \(\sigma^2\); it is not a substitute for this update when the variance is unknown. A plug-in analysis that replaces \(\sigma^2\) by a single estimate also omits uncertainty about the variance. The Normal-Inverse-Gamma model propagates that uncertainty through the joint posterior. Conversely, conjugacy is not a reason to keep a poor model: a robust likelihood, a hierarchical model, or another nonconjugate specification may be more appropriate for the trial question and data, but may not have these closed-form updates.

A common mistake is to use \(s^2\) itself as \(S\). The sample variance has denominator \(n-1\), whereas the update needs \(S=(n-1)s^2\). Another frequent error is to treat \(\beta_n\) as a variance estimate. It is an inverse-gamma scale parameter, with units of outcome squared; it is not itself the posterior variance of \(\mu\) or the posterior mean of \(\sigma^2\). Interpret posterior quantities using the distributions and parameterization stated above.