Test Statistics as Data Summaries

Why Reduce Data to a Test Statistic?

A sample may contain many observations, but a hypothesis test usually focuses on a particular feature of those data. For a claim about a population proportion, the number of observed successes may be the relevant feature. For a claim about a population mean, the sample mean may be central. A test statistic turns the sample into a numerical summary chosen to reflect how the data relate to a null hypothesis.

The choice depends on both the model and the alternative hypothesis. If the alternative says a parameter is larger than its null value, larger values of a suitable statistic should represent movement in that direction. If the alternative says it is smaller, smaller values should represent movement in that direction. For a two-sided alternative, departures in either direction matter, so the magnitude of a signed statistic is often the useful summary.

Definition: A test statistic is a function of the observed sample and the specified hypothesis model, calculated according to a rule chosen before interpreting the data. It reduces the sample to a quantity relevant to the comparison between the null hypothesis and the alternative. A statistic by itself does not decide whether to reject a hypothesis; interpreting it also requires a rule that accounts for its behavior under the null model.

Write the observed sample as $X_1,\ldots,X_n$, and let $T=g(X_1,\ldots,X_n)$ be a test statistic. The function $g$ is selected to capture the aspect of the sample that is informative about the hypotheses. The same data can produce different statistics for different questions. For example, a sample mean is relevant to a hypothesis about a population mean, but it is not automatically a good summary for a hypothesis about a population variance.

Direction and Standardization

A statistic may be a raw count, a sample mean, or a standardized difference. Standardization is useful when the estimate’s ordinary units do not make its distance from the null value easy to interpret. Suppose an estimator $\widehat{\theta}$ has null-model mean $\theta_0$ and null-model standard deviation $\tau_0>0$. A signed standardized statistic is

Z=\frac{\widehat{\theta}-\theta_0}{\tau_0}.

The numerator records direction: it is positive when the estimate exceeds the null value and negative when it falls below. Dividing by $\tau_0$ expresses the difference in units of the estimator’s standard deviation under the null model. Thus $Z=2$ means the estimate is two such standard deviations above the null value, while $Z=-2$ means it is two below. This describes the statistic’s scale and direction; it does not, by itself, say how unusual that value is.

Why standardization works: Under the stated null model, assume $E[\widehat{\theta}]=\theta_0$ and $\operatorname{Var}(\widehat{\theta})=\tau_0^2$. Subtracting $\theta_0$ centers the estimator at zero because $E[\widehat{\theta}-\theta_0]=0$. Dividing by $\tau_0$ gives $$ E[Z]=0 \qquad\text{and}\qquad \operatorname{Var}(Z)=\frac{\operatorname{Var}(\widehat{\theta})}{\tau_0^2}=1. $$ The statistic is therefore measured on a common, dimensionless scale. These mean and variance statements do not establish a particular probability distribution for $Z$; that requires additional model results.

For an upper-tailed alternative, a large positive signed statistic points in the direction specified by the alternative. For a lower-tailed alternative, a large negative value does so. For a two-sided alternative, a large absolute value represents a substantial departure in either direction. The signed value should still be retained when direction is meaningful.

Alternative’s direction	Useful feature of a signed statistic	What the direction means
Parameter greater than its null value	Large positive values	The estimate is above the null value
Parameter less than its null value	Large negative values	The estimate is below the null value
Parameter different from its null value	Large absolute values	The estimate is far from the null value in either direction

Worked Examples

Worked Example: A Larger Claim Frequency

A hypothetical administrator reviews 160 independently selected files. Each file either contains a specified type of claim or does not. Assume the files follow a common Bernoulli model with success probability $p$. The null value is $p_0=0.25$, and the alternative is $p>0.25$. There are 52 files with the specified claim. Calculate a score statistic whose larger values point toward the alternative.

Method and assumptions. Use the one-sample proportion score statistic. Under the null model, the 160 indicators are independent Bernoulli variables with probability $p_0=0.25$. If $\widehat p=X/n$, its null-model standard error is $\sqrt{p_0(1-p_0)/n}$. This is the score standard error, which uses the null value.

First calculate the observed proportion:

\widehat p=\frac{52}{160}=0.325.

Next calculate the null-model standard error:

\sqrt{\frac{0.25(1-0.25)}{160}} =\sqrt{\frac{0.1875}{160}} =\sqrt{0.001171875} \approx 0.03423.

The observed proportion is $0.325-0.25=0.075$ above the null value. Therefore,

Z=\frac{0.325-0.25}{0.0342327}\approx 2.191.

The statistic is positive, as expected for data above the null value, and its direction is consistent with the upper-tailed alternative. Check: $2.191(0.0342327)\approx0.075$, recovering the observed difference. The value summarizes the sample; assessing how unusual it is requires the null reference behavior and a decision rule, neither of which is supplied by the calculation alone.

Common mistake avoided: Substituting $\widehat p$ into the standard error while calling the result this score statistic mixes two formulas. A statistic using an estimated standard error can be defined for another method, but it must be identified and interpreted consistently; the score statistic here uses $p_0$.

Worked Example: A Smaller Average Processing Time

A hypothetical unit’s processing times are modeled as independent normal observations with unknown mean $\mu$ and known standard deviation $\sigma=4$ minutes. For $n=25$ observations, the total processing time is 420 minutes. The hypotheses concern whether the mean is below $\mu_0=18$ minutes. Calculate the signed standardized statistic.

Method and assumptions. Under the stated model, $\overline X$ has mean $\mu$ and standard deviation $\sigma/\sqrt n$. Because $\sigma$ is known, standardize the sample mean’s difference from the null value using $\sigma/\sqrt n$. The normal model and independent observations justify this standard error for the sample mean.

The sample mean is

\overline x=\frac{420}{25}=16.8\text{ minutes}.

The null standard error is

\frac{\sigma}{\sqrt n}=\frac{4}{\sqrt{25}}=\frac{4}{5}=0.8\text{ minutes}.

Thus the signed statistic is

Z=\frac{16.8-18}{0.8}=\frac{-1.2}{0.8}=-1.5.

The negative sign correctly indicates that the observed mean is below the null value, in the direction of the alternative. Check: multiplying the statistic by its standard error gives $(-1.5)(0.8)=-1.2$ minutes, the original difference.

Common mistake avoided: Replacing $-1.5$ by its absolute value $1.5$ discards the direction. Absolute magnitude is useful for a two-sided question, but this alternative is specifically about a smaller mean. Also, the statistic is not a decision: its value alone does not establish whether the null should be rejected.

Worked Example: A Two-Sided Paired Comparison

A hypothetical laboratory compares two settings on each of 16 matched specimens. Define each difference as setting A’s reading minus setting B’s reading. Assume these paired differences are independent observations from a normal population with unknown mean difference and variance. The null mean difference is zero; the alternative is that it is not zero. The observed differences have sum 64 and sum of squared deviations from their sample mean equal to 960. Calculate the signed t-type statistic and the two-sided summary.

Method and assumptions. The observations to analyze are the 16 paired differences, not the two sets of readings as independent samples. Use the one-sample t statistic for their mean. Its sample standard deviation uses divisor $n-1$, and its standard error is $s/\sqrt n$. The stated independent normal model for the differences is the model for this t procedure.

The sample mean difference is

\overline d=\frac{64}{16}=4.

The supplied sum of squared deviations gives

s^2=\frac{960}{16-1}=\frac{960}{15}=64, \qquad s=\sqrt{64}=8.

The estimated standard error is $8/\sqrt{16}=8/4=2$. Therefore, the signed statistic is

t=\frac{\overline d-0}{s/\sqrt n} =\frac{4}{2}=2.

For the two-sided alternative, the relevant departure summary is $|t|=|2|=2$; the positive sign also shows that setting A’s observed readings average 4 units higher than setting B’s. Check: the statistic multiplied by its estimated standard error is $2(2)=4$, equal to the observed mean difference. This computation does not provide a probability or a rejection decision.

Common mistake avoided: Treating the paired readings as two independent samples ignores the matched structure. The data summary supplied is the set of within-specimen differences, so the standard deviation and standard error must be computed from those differences.

What a Statistic Does—and Does Not—Tell You

A well-chosen statistic preserves the part of the sample that matters for the hypothesis while leaving out detail that is not needed for that particular comparison. In the first example, the count of successes determines the observed proportion. In the second, the sample mean captures the observed shift in average processing time. In the paired example, the mean and variability of the within-pair differences determine the t-type statistic.

This reduction is specific to the question. The same sample can be summarized differently when testing a different feature of the population. A test statistic is also not automatically a sufficient statistic, which is a distinct technical concept about how a statistic retains information about a model parameter. Here, “summary” means that the statistic is the selected numerical input to a testing procedure.

Most importantly, a calculated value is not yet a conclusion. To judge whether a value is compatible with a null hypothesis, one needs its behavior under that null model and a stated decision procedure. The statistic provides the data summary on which those later steps operate. It should not be interpreted as a probability, nor should its numerical size alone be called significant.

Key takeaway: Define the hypotheses and model, choose a statistic that reflects the relevant sample feature, and check that its sign or magnitude matches the alternative. A test statistic summarizes the evidence-bearing direction and scale; it does not, by itself, supply a p-value or a decision.

Check Your Understanding

For each setting, identify or calculate a statistic suited to the stated alternative. The answers check the statistic and its direction, not a hypothesis-test decision.

A hypothetical audit has 100 independent Bernoulli records, with 38 successes. Under the null, $p_0=0.30$; the alternative is $p>0.30$. Calculate the score statistic using the null standard error. (Answer: $\widehat p=0.38$, null standard error $=\sqrt{0.30(0.70)/100}\approx0.04583$, and $Z\approx1.746$.)
Independent normal measurements have known $\sigma=6$, sample size $n=36$, and total 630. The null mean is 18.5 and the alternative is that the mean is smaller. Calculate the signed standardized statistic. (Answer: $\overline x=630/36=17.5$, standard error $=6/6=1$, and $Z=(17.5-18.5)/1=-1.000$.)
For nine independent normal paired differences, the sample mean difference is $-3$ and the sample standard deviation is $4$. The null mean difference is zero and the alternative is that it is smaller. Calculate the t-type statistic. (Answer: standard error $=4/\sqrt9=4/3$, so $t=-3/(4/3)=-2.25$.)
A hypothetical Bernoulli sample has $n=200$ and 112 successes. The null proportion is $0.50$ and the alternative is two-sided. Calculate the signed score statistic and its absolute value. (Answer: $\widehat p=0.56$, null standard error $=\sqrt{0.25/200}\approx0.03536$, $Z\approx1.697$, and $|Z|\approx1.697$.)
For 25 independent normal observations with unknown population standard deviation, the sample mean is 104 and the sample standard deviation is 10. The null mean is 100 and the alternative is that it is larger. Give the appropriate t-type statistic and explain its direction. (Answer: $t=(104-100)/(10/\sqrt{25})=4/2=2$, positive and therefore in the direction of the upper-tailed alternative.)

What You'll Learn