Why the Alternative Determines the Test
A linear contrast describes a particular comparison among model parameters. Once that contrast and its null value are specified, the alternative hypothesis determines which departures from the null count as evidence: only larger values, only smaller values, or values in either direction. The test statistic can be the same in all three cases, but the rejection region and p-value must match the direction in the alternative.
For example, an estimated contrast above its null value supports an upper-tailed alternative, not a lower-tailed one. A two-sided alternative treats departures in both directions as evidence. Choosing the direction after seeing the estimate changes the testing procedure and does not give the advertised one-sided significance level.
Contrast and Exact t Statistic
Consider the fixed-design normal linear model \(Y=X\beta+\varepsilon\), where \(Y\) is an \(n\)-vector, \(X\) is a known \(n\times p\) matrix of full column rank \(p\), \(\beta\) is a parameter vector, and \(\varepsilon\sim N(0,\sigma^2 I_n)\) with \(\sigma^2>0\). Assume \(n>p\). Let \(c\) be a specified nonzero \(p\)-vector and let \(L=c^\mathsf{T}\beta\) be the contrast of interest. Write \(\hat L=c^\mathsf{T}\hat\beta\) for its least-squares estimator, \(s^2=\mathrm{SSE}/(n-p)\) for the residual mean square, and \(\nu=n-p\) for the residual degrees of freedom.
The exact null distribution follows because \(\hat L\) is normally distributed with mean \(L\) and variance \(\sigma^2 c^\mathsf{T}(X^\mathsf{T}X)^{-1}c\), while \(\nu s^2/\sigma^2\sim\chi^2_\nu\) independently of \(\hat L\). Thus, under \(H_0\), \(T\) is a standard normal divided by the square root of an independent chi-square variable divided by its degrees of freedom. The direction of the alternative does not change this statistic or its null distribution.
Let \(F_\nu\) denote the cumulative distribution function of \(t_\nu\), and define \(t_{\nu,q}\) by \(F_\nu(t_{\nu,q})=q\), for \(0<q<1\). The t distribution is continuous and symmetric about zero.
Three Alternatives, Three Rejection Regions
For a chosen significance level \(0<\alpha<1\), the rejection region is placed in the tail or tails specified by the alternative. The p-value is calculated from the observed statistic \(t_{\mathrm{obs}}\), with the same directional choice.
| Alternative | Reject \(H_0\) at level \(\alpha\) when | p-value |
|---|---|---|
| \(H_1:L>L_0\) (upper-tailed) | \(T\ge t_{\nu,1-\alpha}\) | \(1-F_\nu(t_{\mathrm{obs}})\) |
| \(H_1:L<L_0\) (lower-tailed) | \(T\le t_{\nu,\alpha}\) | \(F_\nu(t_{\mathrm{obs}})\) |
| \(H_1:L\ne L_0\) (two-sided) | \(|T|\ge t_{\nu,1-\alpha/2}\) | \(2\min\{F_\nu(t_{\mathrm{obs}}),1-F_\nu(t_{\mathrm{obs}})\}\) |
The non-strict inequalities make the decision rule consistent with the convention “reject when the p-value is less than or equal to \(\alpha\).” Since the null distribution is continuous, the probability of landing exactly on a critical boundary is zero. Using strict instead of non-strict inequalities would therefore leave the size unchanged, but the decision convention at the boundary would need to be stated consistently.
Worked Example: An Upper-Tailed Contrast Test
Suppose an analysis estimates a contrast as \(\hat L=4.8\), with null value \(L_0=0\), standard error \(s\sqrt{c^\mathsf{T}(X^\mathsf{T}X)^{-1}c}=2.0\), and \(\nu=12\). The prespecified alternative is \(H_1:L>0\).
Calculate the statistic. The observed value is $$ t_{\mathrm{obs}}=\frac{4.8-0}{2.0}=2.4. $$
Calculate the p-value. For the upper-tailed alternative, $$ p=1-F_{12}(2.4)\approx 0.01676. $$
Make the level-\(0.05\) decision. The upper critical value is \(t_{12,0.95}\approx1.782\). Since \(2.4\ge1.782\), reject \(H_0\). Equivalently, \(0.01676\le0.05\). The evidence is in the direction \(L>L_0\); this test does not establish that \(L\) differs from \(L_0\) in either direction.
Worked Example: A Lower-Tailed Contrast Test
Suppose \(\hat L=1.8\), \(L_0=3.0\), the standard error is \(0.5\), and \(\nu=12\). The scientific alternative is \(H_1:L<3.0\).
Calculate the statistic. $$ t_{\mathrm{obs}}=\frac{1.8-3.0}{0.5}=-2.4. $$
Calculate the p-value. This alternative puts evidence in the lower tail, so $$ p=F_{12}(-2.4)\approx0.01676. $$ By symmetry, \(F_{12}(-2.4)=1-F_{12}(2.4)\).
Make the level-\(0.05\) decision. The lower critical value is \(t_{12,0.05}\approx-1.782\). Since \(-2.4\le-1.782\), reject \(H_0\). A positive statistic would not support this specified lower-tailed alternative, even if its absolute value were large.
Worked Example: A Two-Sided Contrast Test
Now suppose \(\hat L=3.2\), \(L_0=2.0\), the standard error is \(0.5\), and \(\nu=12\), with alternative \(H_1:L\ne2.0\).
Calculate the statistic. $$ t_{\mathrm{obs}}=\frac{3.2-2.0}{0.5}=2.4. $$
Calculate the p-value. A two-sided test counts equally extreme departures in both directions: $$ p=2\min\{F_{12}(2.4),1-F_{12}(2.4)\} =2(0.0167593)\approx0.03352. $$
Make the level-\(0.05\) decision. The two-sided critical value is \(t_{12,0.975}\approx2.179\). Since \(|2.4|\ge2.179\), reject \(H_0\). At level \(0.01\), the critical value is \(t_{12,0.995}\approx3.055\), so the same observation would not lead to rejection. This illustrates why a decision must state its significance level as well as its alternative.
Why the Rules Have the Claimed Size and p-Value Decisions
Proof. Under \(H_0\), \(T\sim t_\nu\), whose continuous distribution function is \(F_\nu\). For the upper-tailed test, $$ P(T\ge t_{\nu,1-\alpha})=1-F_\nu(t_{\nu,1-\alpha})=\alpha. $$ Also, \(1-F_\nu(T)\le\alpha\) holds exactly when \(T\ge t_{\nu,1-\alpha}\), because \(F_\nu\) is increasing. For the lower-tailed test, $$ P(T\le t_{\nu,\alpha})=F_\nu(t_{\nu,\alpha})=\alpha, $$ and \(F_\nu(T)\le\alpha\) holds exactly when \(T\le t_{\nu,\alpha}\).
For the two-sided test, symmetry gives \(t_{\nu,\alpha/2}=-t_{\nu,1-\alpha/2}\). Therefore, $$ P(|T|\ge t_{\nu,1-\alpha/2}) =P(T\ge t_{\nu,1-\alpha/2})+P(T\le-t_{\nu,1-\alpha/2}) =\frac{\alpha}{2}+\frac{\alpha}{2} =\alpha. $$ Finally, \(2\min\{F_\nu(T),1-F_\nu(T)\}\le\alpha\) holds exactly when \(F_\nu(T)\le\alpha/2\) or \(F_\nu(T)\ge1-\alpha/2\), which is exactly the stated two-tail rejection region. Continuity ensures that equality at a boundary has probability zero. \(\square\)
In particular, with the non-strict rejection rules shown here, the p-value is the smallest significance level at which the observed statistic is rejected: at \(\alpha=p\), the p-value decision rule already rejects. This boundary detail is one reason to state whether a test uses “\(p\le\alpha\)” or “\(p<\alpha\)” rather than switching conventions mid-argument.
Proof. Since \(T\) has a continuous distribution function under \(H_0\), the probability integral transform gives \(U=F_\nu(T)\sim\operatorname{Uniform}(0,1)\). For an upper-tailed test, \(1-U\) is uniform; for a lower-tailed test, \(U\) is uniform. For the two-sided p-value \(P_*=2\min(U,1-U)\), take any \(x\in[0,1]\). The events \(U\le x/2\) and \(U\ge1-x/2\) are disjoint, and each has probability \(x/2\). Hence $$ P(P_*\le x)=P(U\le x/2)+P(U\ge1-x/2)=\frac{x}{2}+\frac{x}{2}=x. $$ Thus \(P_*\) is uniform as well. \(\square\)
Uniformity describes calibration under the null; it does not mean that the p-value is the probability that the null hypothesis is true. It also does not make a one-sided and a two-sided test interchangeable: their p-values answer different alternative-specific questions.
Reversing a Contrast and Choosing Direction
Changing a contrast’s sign also changes the null value and the alternative direction if the goal is to express the same scientific comparison. If \(L\) is replaced by \(-L\), then the null \(L=L_0\) becomes \(-L=-L_0\), and the upper alternative \(L>L_0\) becomes the lower alternative \(-L<-L_0\). The centered estimate reverses sign: $$ (-\hat L)-(-L_0)=-(\hat L-L_0). $$ The standard error is unchanged, so the t statistic changes sign. The corresponding lower-tailed p-value for the reversed contrast equals the original upper-tailed p-value.
Worked Example: Reverse the Contrast Without Changing the Question
Suppose the original contrast has estimate \(\hat L=3\), null value \(L_0=1\), standard error \(1\), and \(\nu=12\), and the question is whether \(L>1\). Its statistic is $$ T=\frac{3-1}{1}=2. $$ The upper-tail p-value is \(1-F_{12}(2)\).
For the reversed contrast, the estimate is \(-3\), the null value is \(-1\), and the equivalent alternative is \(-L<-1\). Its statistic is $$ T_{\mathrm{rev}}=\frac{-3-(-1)}{1}=-2. $$ The lower-tail p-value is \(F_{12}(-2)=1-F_{12}(2)\), exactly the same p-value. By contrast, keeping the null value at \(1\) after reversing the contrast would test the different null \(-L=1\), not the original comparison expressed with reversed signs.
The alternative should be chosen from the scientific question before examining the observed sign. If the question is genuinely about any departure from a specified value, use a two-sided test. If it asks whether the contrast exceeds or falls below that value, use the matching one-sided test. A one-sided p-value should not be selected after observing which tail gives the smaller number.
Common Mistakes
- Using the wrong tail. For \(H_1:L<L_0\), a large positive \(T\) is not evidence in the stated direction. Use \(F_\nu(t_{\mathrm{obs}})\), not \(1-F_\nu(t_{\mathrm{obs}})\).
- Doubling the wrong tail. A two-sided p-value is twice the smaller tail probability, or equivalently \(2[1-F_\nu(|t_{\mathrm{obs}}|)]\) by symmetry. Doubling the upper-tail probability for a negative statistic can produce an invalid value.
- Reversing the contrast but retaining its old null value. Reversal changes \(L_0\) to \(-L_0\) when expressing the same null. Holding the old value fixed defines a different hypothesis.
- Choosing a one-sided alternative after seeing the data. That choice makes the reported one-sided calibration inappropriate. Specify the direction based on the question, not on the observed statistic.
- Mixing boundary conventions. With the rules here, reject when \(p\le\alpha\) and use non-strict statistic inequalities. Strict rules can be used too, but their boundary convention must match the stated p-value decision.
The central practical lesson is that an exact t statistic does not determine a test by itself. The null value, direction of the alternative, rejection tail, and p-value formula must all describe the same inferential question.
Check Your Understanding
Use the alternative hypothesis to decide which tail or tails contain evidence against the null.
- For \(H_1:L>L_0\), write the exact p-value formula in terms of \(F_\nu\) and \(t_{\mathrm{obs}}\).
- For a lower-tailed test at level \(\alpha\), express the rejection region using \(t_{\nu,\alpha}\).
- Why is the two-sided p-value twice the smaller of the two tail probabilities?
- If \(L\) is replaced by \(-L\), what changes about the null value and the direction of an equivalent alternative?
- Explain why changing a strict rejection inequality to a non-strict one does not change the size of an exact t test, and how the p-value decision rule should be stated here.