Reading Evidence and Effect Size Separately
A moderated test statistic can rank genes by how clearly their estimated effects differ from a null value, but it does not measure biological importance by itself. In expression microarray analyses, the same test statistic may accompany a small, precisely estimated log-fold change or a larger, uncertain one. Reading the result well means considering the estimated contrast, its uncertainty, and the test together.
For a gene-wise linear model, let \(\hat{\beta}_g\) be the estimated coefficient vector for gene \(g\), and let \(c\) specify the contrast of interest. The estimated contrast is \(\hat{\theta}_g=c^\top\hat{\beta}_g\). When expression is modeled on a log base 2 scale and the contrast compares two conditions, \(\hat{\theta}_g\) is a log2 fold change. Its exponentiated value, \(2^{\hat{\theta}_g}\), is the estimated expression ratio on the original scale.
For the standard empirical Bayes variance model, the gene-specific residual variance estimate \(s_g^2\) is combined with a prior variance \(s_0^2\) estimated across genes. If gene \(g\) has residual degrees of freedom \(d_g\), and the prior contributes \(d_0\) degrees of freedom, the moderated variance estimate is \(\tilde{s}_g^2=(d_0s_0^2+d_gs_g^2)/(d_0+d_g)\). For contrast variance factor \(q_g=c^\top(X^\top X)^{-1}c\), the moderated standard error is \(SE_g=\sqrt{\tilde{s}_g^2q_g}\), and the moderated \(t\)-statistic for \(H_0:\theta_g=0\) is \(t_g=\hat{\theta}_g/SE_g\), with \(d_0+d_g\) degrees of freedom in this model.
These expressions assume that the gene-wise linear model is appropriate for the design, residuals have mean zero and the modeled variance structure is adequate, and the cross-gene variance prior is a reasonable representation of the variance distribution. The moderation formula shown is the common shared-prior form; trend or robust variance models alter how the prior is estimated or applied, not the interpretive distinction between estimate and evidence. The earlier material on variance trends and hyperparameter fits is relevant when those assumptions need scrutiny.
How the Confidence Interval Fits In
A two-sided \(100(1-\alpha)\%\) moderated confidence interval for the contrast is \(\hat{\theta}_g\pm t_{1-\alpha/2,\,d_0+d_g}SE_g\), where \(t_{1-\alpha/2,\,d_0+d_g}\) is the relevant \(t\)-distribution quantile. Its endpoints are in the units of the contrast, such as log2 fold-change units. Exponentiating both endpoints gives an interval for the expression ratio, while retaining their order.
The interval gives a range of effect sizes compatible with the model and data at the stated confidence level. If it excludes zero, the corresponding two-sided test rejects a zero contrast at that level, subject to the same model and variance assumptions. But excluding zero does not show that the effect exceeds a scientifically meaningful threshold. Conversely, an interval that includes zero does not establish that the effect is zero; it can reflect imprecision.
Worked Example: Strong Evidence for a Modest Change
Suppose a hypothetical gene has estimated log2 fold change \(\hat{\theta}=0.12\), moderated standard error \(SE=0.03\), and moderated degrees of freedom \(24\). Assume the linear model and variance moderation are suitable for this contrast. Test \(H_0:\theta=0\) against a two-sided alternative.
The statistic is \(t=0.12/0.03=4.00\). With \(24\) degrees of freedom, the two-sided p-value is approximately \(0.0005269\). This is strong evidence against a zero contrast under the model; it is not the probability that the null hypothesis is true.
Using \(t_{0.975,24}\approx2.064\), the 95% interval is \(0.12\pm2.064(0.03)=0.12\pm0.06192\), or approximately \((0.0581,0.1819)\) log2 fold-change units. The estimated expression ratio is \(2^{0.12}\approx1.087\), about an 8.7% increase. The interval for the ratio is approximately \((2^{0.0581},2^{0.1819})=(1.041,1.134)\).
Thus the data provide strong statistical evidence of an increase, while the estimated increase is modest in magnitude. The small p-value is not evidence that the biological effect is large.
Large Estimates Can Still Be Uncertain
An effect estimate is not made precise merely by being large. Its uncertainty depends on the standard error, which reflects residual variability, design information, and the contrast. In small samples or noisy measurements, even a substantial estimated change can have an interval spanning zero and a wide range of plausible effects. Moderation can stabilize variance estimates across genes; it does not remove uncertainty in the estimate or guarantee that every variance prior is appropriate.
Worked Example: A Larger Estimate With Inconclusive Evidence
For a second hypothetical gene, suppose \(\hat{\theta}=1.00\) log2 fold-change units, \(SE=0.45\), and moderated degrees of freedom \(8\). Assume the specified contrast and variance model are appropriate. The estimated ratio is \(2^{1.00}=2.00\), a doubling on the fitted expression scale.
The moderated statistic is \(t=1.00/0.45=2.222\). A two-sided \(t\)-test with \(8\) degrees of freedom gives \(p\approx0.05698\). At \(\alpha=0.05\), this does not reject the zero-contrast null. That result should not be described as proof of no effect.
Since \(t_{0.975,8}\approx2.306\), the 95% interval is \(1.00\pm2.306(0.45)=1.00\pm1.0377\), or approximately \((-0.0377,2.038)\) log2 units. The corresponding ratio interval is about \((0.974,4.10)\). It includes a ratio near 1 as well as much larger increases. The point estimate is large, but the uncertainty is also substantial, so these data do not pin down its magnitude precisely.
Why Two Genes Can Have Similar Estimates but Different Evidence
The test statistic is a signal-to-uncertainty ratio. For a fixed estimated effect, a smaller standard error produces a larger absolute statistic and typically a smaller p-value. This is why results should not be interpreted by sorting on p-value alone: that ranking emphasizes evidence relative to uncertainty, not effect size alone. In high-dimensional analyses, a gene-level p-value also does not automatically account for testing many genes; if adjusted values are supplied, interpret them as a separate multiplicity-related measure rather than as an effect-size summary.
Worked Example: Same Effect, Different Precision
Consider two hypothetical genes with the same estimated log2 fold change, \(0.60\), and the same moderated degrees of freedom, \(12\). Suppose gene A has moderated \(SE=0.20\), while gene B has moderated \(SE=0.50\). Assume the models and contrasts are appropriate for both genes.
For gene A, \(t_A=0.60/0.20=3.00\), with two-sided \(p\approx0.01107\). For gene B, \(t_B=0.60/0.50=1.20\), with two-sided \(p\approx0.253\). The estimated ratio for each is \(2^{0.60}\approx1.516\). The estimates are identical, but gene A has more precise evidence relative to the zero-contrast null.
With \(t_{0.975,12}\approx2.179\), gene A's 95% interval is \(0.60\pm2.179(0.20)=(0.164,1.036)\). Gene B's is \(0.60\pm2.179(0.50)=(-0.490,1.690)\). The first excludes zero and the second does not. Neither comparison shows that gene A's biological effect is larger: both point estimates are \(0.60\), and their ratio-scale point estimates are the same.
Interpretation Checks and Common Pitfalls
A useful reading sequence is to identify the contrast and its units, examine the estimated effect, assess the standard error and interval, then read the test statistic and p-value in that context. Check that the interval, statistic, and p-value agree: for a two-sided zero-contrast test, an interval at confidence level \(1-\alpha\) should exclude zero exactly when the corresponding test rejects at level \(\alpha\), apart from rounding.
- Confusing significance with magnitude. “The p-value is tiny, so the change is large” is wrong. A small p-value indicates incompatibility with the null relative to the model and uncertainty; report the estimated log-fold change and interval to discuss size.
- Calling a nonsignificant result no effect. A p-value above the chosen threshold does not prove a zero effect. Inspect the interval: it may include both negligible and important changes.
- Reading log2 units as ordinary percentages. A log2 fold change of 1 means a ratio of 2, not a 1% increase. Convert with \(2^{\hat{\theta}}\) when the model uses log base 2 expression.
- Treating a zero-null test as a minimum-effect test. A significant result against zero does not establish that the effect exceeds a practical threshold. When a minimum effect is the scientific claim, use the relevant threshold-based procedure rather than overstating a zero-null result.
- Ignoring the fitted model. A moderated statistic is conditional on the design, contrast, and variance assumptions. Unexpected results or implausible intervals warrant checking those inputs and the variance-fit diagnostics rather than interpreting the number in isolation.
Ordinary gene-wise tests use the individual gene's residual variance estimate, whereas moderated tests stabilize variance estimation by borrowing information across genes; when the prior is appropriate, this can improve inference with limited replication. A confidence interval makes uncertainty and effect units especially visible, while a moderated test summarizes evidence against a specified null. Neither replaces the other. A threshold-based test addresses a different question: whether the data support an effect beyond a specified minimum. These methods are alternatives for distinct inferential targets, not interchangeable labels for “significance.”
Check Your Understanding
Answer each question using the distinction between estimated effect, uncertainty, and evidence.
- A gene has log2 fold change \(0.25\), moderated \(SE=0.05\), and \(df=20\). What is its moderated statistic, and what is the approximate expression ratio?
- A second gene has a larger estimated log2 fold change than the first, but a much larger standard error and a confidence interval that includes zero. Can you conclude its effect is larger with strong evidence? Explain.
- What does a 95% interval for a log2 fold change that excludes zero imply about the corresponding two-sided test at \(\alpha=0.05\)? What does it not establish?
- Why does a small p-value not by itself show that a gene has an important biological effect?
Solutions. (1) The statistic is \(0.25/0.05=5.00\). The ratio is \(2^{0.25}\approx1.189\), or an estimated increase of about 18.9%. (2) No. Its point estimate is larger, but the wide interval and large standard error indicate substantial uncertainty; the data do not provide strong evidence of a nonzero contrast if the interval includes zero. (3) It implies rejection of the zero-contrast null by the corresponding two-sided test at the 0.05 level, subject to the model. It does not establish that the effect exceeds a practical or biological threshold. (4) The p-value measures evidence relative to a specified null and the modeled uncertainty, not the size or importance of the effect; a small, precisely estimated change can have a small p-value.