Scaled Inverse-Chi-Squared Priors for Variances

Why Specify a Prior for a Gene-Wise Variance?

In an expression microarray analysis, each gene is fitted with a linear model, and its residual variance describes variability left after accounting for the design. With few samples, an individual gene’s residual variance estimate can be unstable. The earlier material on empirical Bayes shrinkage of gene-wise variances motivates sharing information across genes; here the focus is the probability distribution used to represent that shared prior information.

A variance must be positive, and its sampling uncertainty is often asymmetric, especially in small samples. The scaled inverse-chi-squared distribution is a conjugate prior for the variance in a Gaussian model. Its two parameters have distinct roles: the scale sets the variance level, while the prior degrees of freedom govern how concentrated the prior is. Interpreting these parameters correctly matters because the scale is not generally the prior mean.

Deriving the Density and Its Connection to the Model

The density follows by changing variables from \(U\). Write \(c=\nu_0s_0^2\), so that \(U=c/v\) and \(\left|dU/dv\right|=c/v^2\). The chi-squared density is \(f_U(u)=u^{\nu_0/2-1}e^{-u/2}/[2^{\nu_0/2}\Gamma(\nu_0/2)]\) for \(u>0\). Applying the change-of-variables formula gives

This is a proper density on positive variances when \(\nu_0>0\) and \(s_0^2>0\). It is also an inverse-gamma density: under the convention that an inverse-gamma distribution with shape \(a\) and scale \(b\) has density proportional to \(v^{-a-1}\exp(-b/v)\), the corresponding parameters are \(a=\nu_0/2\) and \(b=\nu_0s_0^2/2\). Different inverse-gamma conventions use different meanings for “scale,” so the density or the chi-squared representation should be checked whenever parameters are translated.

For gene \(g\), suppose the expression vector follows the Gaussian linear model \(y_g=X\beta_g+\epsilon_g\), where the design matrix \(X\) has rank \(r\), the sample size is \(n\), and \(\epsilon_g\sim N(0,\sigma_g^2 I)\). Then the residual degrees of freedom are \(\nu_g=n-r\), and the residual sum of squares \(RSS_g\) satisfies \(RSS_g/\sigma_g^2\sim\chi^2_{\nu_g}\). The usual residual variance estimate is \(s_g^2=RSS_g/\nu_g\). Its likelihood as a function of the variance is proportional to

This likelihood result assumes the specified linear mean model and independent Gaussian errors with a common variance for the samples of that gene. The prior describes a separate modeling assumption: before using this gene’s residual information, its variance is treated as drawn from a shared positive distribution with parameters \(\nu_0\) and \(s_0^2\). In empirical Bayes analysis these hyperparameters are typically estimated from many genes and then treated as fixed for the gene-wise calculation.

What the Prior Scale and Degrees of Freedom Mean

The prior scale \(s_0^2\) sets the scale of the distribution, but it is not usually its arithmetic mean. From \(\sigma_g^2=\nu_0s_0^2/U\), and the chi-squared moments, the prior mean exists only if \(\nu_0>2\). When it exists,

The prior mode is \(\nu_0s_0^2/(\nu_0+2)\), which is also not generally \(s_0^2\). A useful alternative interpretation is \(E(1/\sigma_g^2)=1/s_0^2\) for every \(\nu_0>0\). Thus the scale sets the reciprocal-variance expectation, while its relation to the variance mean depends on the degrees of freedom. For \(\nu_0\leq2\), the prior is proper but its variance mean is infinite; that does not make it an invalid probability distribution, but it does make the mean a poor description of its center.

The prior degrees of freedom control concentration. In the representation, \(U/\nu_0\) is centered at 1 and its relative variability decreases as \(\nu_0\) grows. Consequently, for fixed \(s_0^2\), the prior concentrates near \(s_0^2\) as \(\nu_0\) becomes large. Small \(\nu_0\) permits a wider range of variances and a heavier upper tail. The parameter \(\nu_0\) acts like prior or pseudo-degrees of freedom in the likelihood kernel, but it is not a literal number of additional observed arrays.

Multiplying the gene-wise likelihood kernel by this prior kernel shows why these parameters are conjugate. The prior contributes a power of \(\sigma_g^2\) and an exponential term involving \(\nu_0s_0^2\); the likelihood has the same functional form, with \(\nu_g\) and \(RSS_g=\nu_gs_g^2\). The combined expression therefore remains in the scaled inverse-chi-squared family. This conjugacy is the algebraic reason the prior is convenient; the resulting posterior variance and its use in moderated tests are separate calculations.

Calibration, Diagnostics, and Alternatives

In an empirical Bayes model, \(s_0^2\) and \(\nu_0\) summarize the distribution of gene-wise residual variances across the analyzed genes. They should be interpreted on the variance scale, which has squared expression units. A scale calibrated on a different expression transformation or measurement scale is not directly interchangeable. The prior is most credible when the genes used to estimate its hyperparameters provide a reasonably comparable variance population under the chosen design and preprocessing.

Diagnostics begin with the gene-wise model. Residual plots and checks for influential observations can reveal departures from a linear mean structure or unusually large residuals. Strong outliers, dependence among samples, or a variance that changes across the design groups undermine the simple Gaussian, common-within-gene-variance likelihood. Across genes, inspect whether the fitted prior is broadly compatible with the observed spread of residual variance estimates. A severe mismatch can indicate that one shared prior is inadequate for the genes being analyzed; it should not be repaired simply by describing the prior scale as the mean.

The scaled inverse-chi-squared and inverse-gamma forms are equivalent when their parameters are translated as above. The choice between those names is mainly a parameterization choice, and using a named distribution without stating its density can invite scale-parameter errors. A lognormal prior on variance is a different model: it also restricts variance to positive values, but it does not have the same conjugate density or pseudo-degrees-of-freedom interpretation. A prior directly on the standard deviation is different again; transforming it to a variance prior requires the appropriate change-of-variables factor.

No single prior form removes the need for model checks. A common prior can be too restrictive if the genes form substantially different variance groups, and plug-in empirical Bayes hyperparameters do not fully represent uncertainty about their estimation. The prior also does not correct a misspecified gene-wise mean model, poor experimental design, or measurement artifacts. Its role is narrower: provide a tractable distribution for positive gene-wise residual variances and a clear way to express their shared level and prior concentration.

Common Mistakes

• Calling \(s_0^2\) the prior mean. The mean is \(\nu_0s_0^2/(\nu_0-2)\) only when \(\nu_0>2\); the scale and mean differ at finite \(\nu_0\).
• Treating \(\nu_0\) as literal extra samples. It is a prior concentration parameter, or pseudo-degrees of freedom, not a count of measured arrays.
• Forgetting the variance units. If expression is measured on a log scale, both \(s_0^2\) and \(\sigma_g^2\) have squared log-expression units.
• Mixing inverse-gamma conventions. Check the density before matching “scale” parameters; conventions vary.
• Assuming conjugacy validates the model. Conjugacy is an algebraic property, not evidence that the Gaussian errors or the shared prior describe the data adequately.