What the Sign of a Result Says That the P-Value Does Not

Direction and Extremeness Answer Different Questions

A test result is often summarized with an estimated effect and a p-value. These quantities answer different questions. The sign of the estimate tells which way the observed difference points, once the comparison has been defined. The p-value describes how extreme the observed test statistic is relative to outcomes expected under the null model and the chosen alternative. In a two-sided test, the p-value treats equally large positive and negative statistics alike, so it does not retain the direction of the observed result.

This distinction matters when reading statements such as “the result was statistically unusual.” That description alone does not say whether the sample estimate was above or below the null value. To learn the observed direction, read the estimate and its definition. To learn how extreme the result was under the test, read the p-value together with the test specification.

Definition: The sign of an estimated effect indicates its direction relative to the stated comparison or reference value. A positive estimate is above that reference; a negative estimate is below it. A p-value is calculated from the observed test statistic under the null model, according to the outcomes counted as at least as extreme as the observation. A two-sided p-value counts extremeness in both directions.

The Sign Depends on the Defined Contrast

For a difference between two estimated means, the order of subtraction must be stated. If the effect is defined as mean in group A minus mean in group B, then a positive estimate means the observed mean in A is larger, and a negative estimate means it is smaller. Reversing the order changes the sign: B minus A is the negative of A minus B. The underlying observations have not changed; only the direction used to express the contrast has changed.

A test statistic often preserves the sign of the estimate. For example, a standardized difference is the estimate minus its null value, divided by a positive standard error. Because the standard error is positive, dividing by it changes the scale but not the sign. The p-value is then obtained from the statistic using the selected tail or tails.

Result: For a two-sided standard-normal test with observed statistic $z_{\mathrm{obs}}$, the p-value is $$ p=2P(Z\geq |z_{\mathrm{obs}}|), \qquad Z\sim N(0,1). $$ Since this formula uses the absolute value, changing $z_{\mathrm{obs}}$ to $-z_{\mathrm{obs}}$ leaves the p-value unchanged. The p-value reports extremeness under the two-sided rule, not whether the observed statistic was positive or negative.

Worked Examples

Worked Example: A Positive Difference and Its Two-Sided P-Value

In a hypothetical comparison, researchers define the effect as mean recovery time for group A minus mean recovery time for group B, measured in days. Suppose the estimated difference is $+3.0$ days and its standard error is $1.5$ days. A two-sided test uses $H_0:\mu_A-\mu_B=0$, $H_a:\mu_A-\mu_B\neq 0$, and a standard-normal null reference distribution. Assume the test’s sampling conditions justify that reference model and that the stated standard error is appropriate under the null.

The observed standardized statistic is the estimate minus its null value, divided by the standard error:

z_{\mathrm{obs}}=\frac{3.0-0}{1.5}=2.00.

The positive estimate means that, in this hypothetical sample, group A’s estimated mean recovery time is 3.0 days longer than group B’s. For the two-sided test, the p-value counts outcomes at least as far from zero as $2.00$, in either direction:

p=P(|Z|\geq 2.00) =2P(Z\geq 2.00) \approx 2(0.02275) \approx 0.0455.

The estimate gives the observed direction: A minus B is positive. The p-value gives the two-sided extremeness under the null model. It does not itself say “positive”; that information comes from the estimate and the stated subtraction order.

Worked Example: Reversing the Contrast

Use the same hypothetical comparison, but now define the effect as mean recovery time in group B minus mean recovery time in group A. The estimated contrast is therefore $-3.0$ days, with standard error $1.5$ days. Consider the corresponding two-sided test against a null difference of zero, using the same standard-normal reference model and assumptions as in the preceding example.

The test statistic is

z_{\mathrm{obs}}=\frac{-3.0-0}{1.5}=-2.00.

The estimate is negative because the B-minus-A contrast is below zero. This says the observed mean for B is 3.0 days less than the observed mean for A. The two-sided p-value is

p=P(|Z|\geq |-2.00|) =2P(Z\geq 2.00) \approx 2(0.02275) \approx 0.0455.

The p-value matches the previous example even though the statistic’s sign changed. Reversing the contrast changed the direction in which the difference was written, but not its distance from the null value. This is why the p-value cannot tell which group had the larger observed mean.

Worked Example: The Same Negative Result Under Two One-Sided Alternatives

Suppose a hypothetical estimate of a treatment-minus-control difference in a measured outcome is $-4.0$ units, with standard error $2.0$ units. The test statistic is $z_{\mathrm{obs}}=(-4.0-0)/2.0=-2.00$. Assume a standard-normal null reference distribution is appropriate. The estimate is negative: the sample difference is 4.0 units below the null value.

For an upper-tail alternative $H_a:\delta>0$, where $\delta$ is the treatment-minus-control population difference, outcomes at least as large as the observation are counted in the upper tail. The p-value is

p=P(Z\geq -2.00)\approx 0.9772.

For the lower-tail alternative $H_a:\delta<0$, outcomes at least as far into the lower tail as the observed statistic are counted. The p-value is

p=P(Z\leq -2.00)\approx 0.0228.

The estimate is negative in both calculations; changing the alternative does not change the data or the estimate. It changes which outcomes count as extreme for the test. Thus a one-sided p-value must be interpreted with its alternative hypothesis in view. A p-value by itself, without knowing the tail rule, does not reliably identify the observed direction.

Why a Two-Sided P-Value Loses the Sign

The symmetry of the standard-normal null distribution explains the loss of direction. Under the two-sided rule, the observed statistic $z_{\mathrm{obs}}$ is compared with values whose absolute magnitude is at least as large. A statistic of $+2$ and one of $-2$ are equally far from zero, so their two-sided p-values are equal.

The equality can be shown directly. If $Z\sim N(0,1)$, its distribution is symmetric about zero, so $P(Z\leq -a)=P(Z\geq a)$ for any positive $a$. Therefore, for $a=|z_{\mathrm{obs}}|$,

P(|Z|\geq a) =P(Z\leq -a)+P(Z\geq a) =2P(Z\geq a).

Only the magnitude $a$ appears in the final expression. The original sign has been removed by taking the absolute value. This is not a defect in the p-value: a two-sided test is designed to count evidence of departure in either direction. It simply means that direction must be reported separately.

The argument also clarifies why the estimate and p-value should not be treated as interchangeable summaries. The estimate describes the observed effect on its original scale and in its defined direction. The p-value uses a standardized statistic and a reference distribution to describe extremeness under the null model. Neither quantity substitutes for the other.

Reading and Reporting Both Pieces

A clear report identifies the estimate, its units, the contrast order, and the p-value from the stated test. For example: “The estimated difference in mean recovery time, defined as group A minus group B, was $+3.0$ days; the two-sided test gave $p=0.0455$.” The sign and contrast order communicate that A’s sample mean was higher. The p-value communicates the two-sided tail probability under the null reference model and test procedure.

A complete interpretation also avoids claiming that the p-value gives the probability that the null hypothesis is true or the probability that chance caused the data. It is calculated assuming the null model, and its meaning depends on the test procedure and the definition of “at least as extreme.” The p-value does not establish the population effect’s direction; the sample estimate provides the observed direction, while the test concerns compatibility with the null model.

Read the contrast definition.
Check which group or condition is subtracted from which, and identify the null value.

Read the estimate’s sign.
Interpret positive or negative in context, using the stated contrast order and units.

Read the test specification.
Identify $H_a$, the reference distribution, and whether the p-value is one-sided or two-sided.

Interpret the p-value separately.
Describe extremeness under the null model and selected tail rule; do not use it to infer the observed sign.

Common Mistake

Wrong reasoning: “The two-sided p-value is small, so the treatment must have increased the outcome.” A two-sided p-value counts departures in both directions. The same value can accompany a positive estimate or a negative estimate, as long as the standardized statistics have the same absolute magnitude.

Correction: Use the estimate and the definition of the contrast to state the direction. Then use the p-value, together with the null model and alternative, to describe how extreme the test statistic is. For example, $p=0.0455$ alone does not say whether the estimated difference is positive or negative; a report must also give the estimate and specify what was subtracted from what.

Key takeaway: The sign of an estimate describes the direction of the observed result relative to a defined reference. A two-sided p-value describes extremeness in either direction and is unchanged when the sign is reversed. Report both, with the contrast order and units, to communicate direction and test evidence accurately.

Check Your Understanding

Answer each question using the distinction between direction and extremeness.

A difference is defined as mean in group X minus mean in group Y. What does a negative estimate say about the two sample means?
For a two-sided standard-normal test, why do observed statistics $+1.8$ and $-1.8$ have the same p-value?
A two-sided p-value is reported as 0.12, but the estimate and contrast definition are omitted. Can you determine the observed direction? Explain.
For an observed statistic of $-1.5$, would the upper-tail and lower-tail p-values be equal? Explain which tail is more extreme in each test.
Write one sentence reporting an estimated difference of $-2.4$ hours defined as group A minus group B, with a two-sided p-value of 0.18.