Ordinary and Moderated Inference Compared

Why Compare the Two t-Statistics?

In a microarray analysis, a linear model estimates a contrast for each gene. The estimated contrast describes the effect on the model’s expression scale; its standard error describes uncertainty in that estimate. For genes with few residual degrees of freedom, the gene-specific residual variance can be imprecise. Ordinary inference uses that variance on its own, while moderated inference combines it with information from the variance pattern across genes.

This comparison matters because the two procedures can give different standard errors, degrees of freedom, test statistics and evidence against a null hypothesis. Moderation does not change the fitted contrast estimate. It changes how uncertain that estimate is treated as being. The introductory material on moderated t-statistics gives the underlying procedure; here the emphasis is on understanding what the comparison looks like for genes with low, typical and high residual variances.

Model and Notation for the Comparison

For gene \(g\), suppose the expression measurements satisfy a linear model \(y_g=X\beta_g+\varepsilon_g\), where \(X\) is the design matrix and \(\beta_g\) is the vector of gene-specific coefficients. For a contrast \(c\), the estimated effect is \(\hat{\theta}_g=c^\mathsf{T}\hat{\beta}_g\). Let \(s_g^2\) be the gene’s residual variance estimate and \(df_g\) its residual degrees of freedom. Let \(v_g=c^\mathsf{T}(X^\mathsf{T}X)^{-1}c\), or the corresponding contrast variance factor for the fitted design. Thus, the estimated variance of the contrast is \(s_g^2v_g\).

Ordinary inference treats each gene’s residual variance estimate separately. Moderated inference uses an empirical-Bayes variance prior estimated across genes. Write \(d_0\) for the prior degrees of freedom and \(s_0^2\) for the prior variance. The posterior residual variance and moderated degrees of freedom are

s_{g,\mathrm{post}}^2 = \frac{d_0s_0^2+df_gs_g^2}{d_0+df_g}, \qquad df_{g,\mathrm{mod}}=d_0+df_g.

The ordinary and moderated standard errors and t-statistics are therefore

SE_{g,\mathrm{ord}}=s_g\sqrt{v_g}, \qquad t_{g,\mathrm{ord}}=\frac{\hat{\theta}_g}{s_g\sqrt{v_g}}, \qquad t_{g,\mathrm{mod}}=\frac{\hat{\theta}_g}{s_{g,\mathrm{post}}\sqrt{v_g}}.

Definition: An ordinary t-statistic uses the gene’s residual variance estimate alone and, under the linear-model assumptions, is compared with a t distribution with \(df_g\) degrees of freedom. A moderated t-statistic uses the posterior residual variance combining that estimate with the across-gene prior, and is compared with a t distribution with \(df_g+d_0\) degrees of freedom.

The standard assumptions include a correctly specified linear mean model, independent errors across samples within a gene, and normally distributed errors with a common residual variance for that gene. The empirical-Bayes model additionally assumes that the gene-specific variances are reasonably described by a shared prior distribution, possibly after accounting for a mean-variance trend or robust treatment of outlying genes. In calculations below, the prior quantities are treated as fixed estimates. If the design, contrast or variance model is inappropriate, the arithmetic can be correct while the inference is not.

Why Moderation Shrinks Variances

A common variance model expresses the gene-specific estimate conditionally as \(df_gs_g^2/\sigma_g^2\sim\chi^2_{df_g}\), where \(\sigma_g^2\) is the gene’s unknown residual variance. Under a scaled inverse-chi-squared prior with degrees of freedom \(d_0\) and scale \(s_0^2\), the posterior variance estimate has the weighted-average form above. Its weights are determined by the prior and data degrees of freedom:

s_{g,\mathrm{post}}^2 = \left(\frac{d_0}{d_0+df_g}\right)s_0^2 + \left(\frac{df_g}{d_0+df_g}\right)s_g^2.

This makes the direction of shrinkage explicit. If \(s_g^2<s_0^2\), the posterior estimate is larger than the observed estimate; if \(s_g^2>s_0^2\), it is smaller. If the two variances match, moderation leaves the variance unchanged. The estimate is pulled toward the prior, not necessarily toward the arithmetic mean of all observed variances: \(s_0^2\) is the fitted prior scale on the model’s variance scale.

The moderated degrees of freedom also increase. This reflects the added information about variance from the prior; it does not mean the experiment acquired additional samples. Both changes matter for inference: the standard error may rise or fall, while the larger reference degrees of freedom generally make the t distribution less heavy-tailed. It is not correct to conclude that moderation must make every gene more significant.

Worked Example: Low, Typical and High Residual Variances

Suppose three hypothetical genes are fitted with the same contrast, \(v_g=1\), and each has \(df_g=4\). Assume the fitted prior has \(d_0=4\) and \(s_0^2=1\), in squared log-expression units. The estimates and residual variances are: gene A, \(\hat{\theta}_A=1.6\), \(s_A^2=0.25\); gene B, \(\hat{\theta}_B=2.2\), \(s_B^2=1\); and gene C, \(\hat{\theta}_C=3.0\), \(s_C^2=4\). These are hypothetical values chosen to illustrate the calculation.

For gene A, whose residual variance is below the prior scale, the ordinary standard error is \(\sqrt{0.25}=0.5\), giving \(t_{A,\mathrm{ord}}=1.6/0.5=3.2\). Its posterior variance is \(s_{A,\mathrm{post}}^2=(4(1)+4(0.25))/8=5/8=0.625\). Thus the moderated standard error is \(\sqrt{0.625}=0.7906\), and \(t_{A,\mathrm{mod}}=1.6/0.7906=2.024\).

For gene B, the observed variance equals the prior scale. Its ordinary standard error is \(\sqrt{1}=1\), so \(t_{B,\mathrm{ord}}=2.2\). The posterior variance is \((4(1)+4(1))/8=1\), giving the same standard error and moderated statistic, \(t_{B,\mathrm{mod}}=2.2\). Its moderated two-sided p-value, using 8 degrees of freedom, is approximately \(0.05899\).

For gene C, the observed variance is above the prior scale. Its ordinary standard error is \(\sqrt{4}=2\), so \(t_{C,\mathrm{ord}}=3.0/2=1.5\). The posterior variance is \((4(1)+4(4))/8=20/8=2.5\), and the moderated standard error is \(\sqrt{2.5}=1.581\). Hence \(t_{C,\mathrm{mod}}=3.0/1.581=1.897\).

The ordinary ranking by absolute t-statistic is A, B, C: \(3.2>2.2>1.5\). The moderated ranking is B, A, C: \(2.2>2.024>1.897\). The ranking changed because the low variance for A was pulled upward and the high variance for C was pulled downward. The estimated effects, \(1.6\), \(2.2\), and \(3.0\), did not change. For a two-sided test at \(\alpha=0.05\), the moderated 5% critical value with 8 degrees of freedom is about 2.306; none of these moderated statistics exceeds it. In particular, gene B’s p-value is above 0.05, despite its moderated statistic being 2.2.

How to Read Changes in Evidence

A t-statistic measures an estimated effect in standard-error units. For the same estimated effect and contrast factor, a larger standard error gives a smaller absolute t-statistic. In the example, moderation increased gene A’s standard error and reduced its absolute statistic; for gene C it reduced the standard error and increased the statistic. Gene B’s standard error did not change, but its reference degrees of freedom did.

A two-sided p-value is the probability, under the specified null model, of a t-statistic at least as extreme in absolute value as the one observed. Its reference distribution must match the procedure: ordinary inference uses \(df_g\), whereas moderated inference uses \(df_g+d_0\). Comparing just the numerical t-statistics while ignoring their degrees of freedom is incomplete. A p-value is not the probability that the null hypothesis is true, nor is a smaller p-value a measure of the size or practical importance of the estimated expression difference.

Worked Example: Comparing Evidence for One Gene

Consider a hypothetical gene with an estimated contrast of \(2.2\) log-expression units, \(s_g^2=1\), \(v_g=1\), and \(df_g=4\). Use the prior values \(d_0=4\) and \(s_0^2=1\).

The ordinary standard error is \(1\), and the ordinary t-statistic is \(2.2/1=2.2\), referred to a t distribution with 4 degrees of freedom. For moderation, the posterior variance is \((4(1)+4(1))/8=1\), so the moderated standard error is also \(1\) and the moderated statistic is also \(2.2\). However, the moderated reference distribution has 8 degrees of freedom. Its two-sided p-value is \(2P(T_8\geq 2.2)\approx0.05899\).

At a 5% significance level, the moderated result does not cross the two-sided threshold: \(0.05899>0.05\). The unchanged statistic does not imply unchanged evidence, because the degrees of freedom differ. The conclusion concerns evidence against a zero contrast for this gene under the model; it does not establish that the true effect is zero.

Comparing Confidence Intervals

The standard error and reference degrees of freedom also determine the interval for a contrast. An ordinary \(100(1-\alpha)\%\) interval is \(\hat{\theta}_g\pm t_{1-\alpha/2,df_g}SE_{g,\mathrm{ord}}\); its moderated counterpart uses \(df_g+d_0\) and \(SE_{g,\mathrm{mod}}\). Since both the critical value and standard error can change, the moderated interval is not guaranteed to be narrower.

Worked Example: Interval Width Can Change in Both Ways

Suppose a hypothetical gene has \(\hat{\theta}_g=2.4\) log-expression units, \(s_g^2=2.25\), \(v_g=1\), and \(df_g=4\). Let \(d_0=4\) and \(s_0^2=1\). Its ordinary standard error is \(\sqrt{2.25}=1.5\), and its posterior variance is \((4(1)+4(2.25))/8=1.625\). The moderated standard error is \(\sqrt{1.625}=1.2748\).

Using 95% critical values of approximately 2.776 for 4 degrees of freedom and 2.306 for 8 degrees of freedom, the ordinary interval is \(2.4\pm2.776(1.5)=2.4\pm4.164\), or \((-1.764,6.564)\) log-expression units. The moderated interval is \(2.4\pm2.306(1.2748)=2.4\pm2.940\), or approximately \((-0.540,5.340)\) log-expression units. In this case the moderated interval is narrower because the posterior variance is smaller than the observed variance and the critical value is also smaller. Both intervals include zero, so neither procedure provides a 5%-level two-sided rejection. The 95% interpretation is about the long-run coverage of the interval procedure under its assumptions, not a 95% probability that this fixed parameter lies in this particular interval.

Diagnostics, Alternatives and Limitations

The comparison is useful only if the variance model and design support it. Before interpreting a difference, check that the contrast represents the intended comparison, the design is full rank for the coefficients of interest, and residual patterns do not show gross departures from the fitted model. Across genes, inspect whether the variance prior is a reasonable summary, whether a mean-variance trend is needed, and whether a small number of extreme variances are exerting undue influence. These checks do not prove the assumptions, but can reveal important mismatches.

Possible failure: A strong mean-variance relationship is ignored, so genes with different average expression are compared to a single unsuitable prior scale. A trend-aware variance model may be more appropriate.
Possible failure: A few hypervariable genes distort the prior fit. Robust empirical-Bayes fitting can reduce their influence, but it does not repair misspecified sample-level models.
Possible failure: Samples are dependent or a key design factor is omitted. Shrinking variances cannot correct dependence or confounding; the design and error model must address them.
Interpretive limitation: Moderation borrows strength for variance estimation, not for the gene’s estimated contrast. It does not make the effect estimates identical across genes or guarantee correct rankings.

Ordinary inference remains a direct gene-wise option when residual variance estimates are reliable and the model assumptions hold. Moderated inference is particularly useful when each gene has limited residual degrees of freedom and many genes provide a credible shared variance distribution. A third possibility is to model variance trends or use robust prior fitting when a single common prior is inadequate. These are not interchangeable fixes: the choice depends on why gene-wise variances differ and whether that structure is represented in the model.

Common pitfall: “Moderation always lowers standard errors and makes genes more significant.” This is wrong. A gene with \(s_g^2<s_0^2\) is generally assigned a larger posterior variance, while a gene with \(s_g^2>s_0^2\) is generally assigned a smaller one. The correct comparison calculates the posterior variance, standard error and reference degrees of freedom for each gene; the direction and size of the evidence change depend on all of them.

Check Your Understanding

Work through each question, then compare your reasoning with the solutions.

With \(df_g=4\), \(d_0=4\), \(s_0^2=1\), and \(s_g^2=0.25\), calculate the posterior variance. Is it above or below the observed variance, and why?
A gene has \(\hat{\theta}_g=2\), \(s_g^2=1\), and \(v_g=1\). Find its ordinary and moderated t-statistics under \(df_g=4\), \(d_0=4\), and \(s_0^2=1\). What differs between the procedures?
For \(df_g=4\), \(d_0=4\), \(s_0^2=1\), and \(s_g^2=4\), calculate the ordinary and moderated standard errors when \(v_g=1\). Which is larger?
Why can two genes exchange their ranking by absolute moderated t-statistic even though their estimated contrasts do not change?

Solutions. (1) The posterior variance is \((4(1)+4(0.25))/8=0.625\), which is above the observed \(0.25\) because the observation is below the prior scale and is pulled upward. (2) Both variances are 1, so both standard errors are 1 and both t-statistics are \(2/1=2\). The ordinary reference distribution has 4 degrees of freedom; the moderated one has \(4+4=8\). (3) The ordinary standard error is \(\sqrt{4}=2\). The posterior variance is \((4(1)+4(4))/8=2.5\), so the moderated standard error is \(\sqrt{2.5}=1.581\); the ordinary standard error is larger. (4) Moderation shrinks each gene’s variance toward the prior scale by an amount depending on its observed variance and degrees of freedom. Since the standard error changes by different amounts across genes, their absolute t-statistics can change order even though each numerator, the estimated contrast, stays fixed.