Exact Trend Tests in Ordered Tables

Why Test for Trend?

When groups have a natural order—such as increasing dose, disease stage, or age band—an association test that treats the groups as unordered can miss the question of interest. A trend test asks whether the response tends to increase or decrease in that order. An exact trend test makes this assessment using the finite conditional distribution of a score statistic, rather than relying on a large-sample approximation.

Consider $K$ ordered groups and a binary outcome, called a success for convenience. Let $n_j$ be the fixed size of group $j$, and let $Y_j$ be its number of successes. Assign group $j$ a prespecified numerical score $a_j$, with $a_1\leq\cdots\leq a_K$. The statistic $T=\sum_j a_jY_j$ is large when successes tend to occur in groups with larger scores. The scores encode the ordering and spacing to be tested; they should be chosen before examining the outcomes.

Definition: For an ordered $K$-group table with binary outcomes, the linear trend score statistic for fixed group scores $a_1,\ldots,a_K$ is $$ T=\sum_{j=1}^K a_jY_j. $$ Write $N=\sum_{j=1}^K n_j$ for the total number of observations and $m=\sum_{j=1}^K Y_j$ for the total number of successes. A conditional trend test holds the group sizes $n_j$ and the total $m$ fixed.

The usual null hypothesis is that success probability is the same in every group. Under this null, a large value of $T$ supports a positive trend when the scores increase with group order. A small value supports a negative trend. The conditional distribution removes the common success probability, so the resulting test does not require its value to be specified.

The Conditional Distribution

Under the null, suppose observations are independent Bernoulli trials with a common success probability $p$, where $0<p<1$. Given the group sizes, the group counts are independent with $Y_j\sim\operatorname{Bin}(n_j,p)$. Conditioning further on $m$, the total number of successes, gives the exact distribution over all feasible vectors $(y_1,\ldots,y_K)$ satisfying $0\leq y_j\leq n_j$ and $\sum_j y_j=m$.

Theorem: Under the common-probability null, conditional on the group sizes and the total number of successes $m$, $$ P(Y_1=y_1,\ldots,Y_K=y_K\mid n_1,\ldots,n_K,m) = \frac{\prod_{j=1}^K {n_j\choose y_j}}{{N\choose m}} $$ for every feasible vector $y$. This conditional law does not depend on $p$.

Proof. Before conditioning, the probability of a feasible vector is

\prod_{j=1}^K {n_j\choose y_j}p^{y_j}(1-p)^{n_j-y_j} = \left(\prod_{j=1}^K {n_j\choose y_j}\right)p^m(1-p)^{N-m}.

For every vector with total $m$, the factors $p^m(1-p)^{N-m}$ are the same. The sum of the remaining weights over feasible vectors is ${N\choose m}$: choosing $m$ successes from all $N$ individuals can be partitioned according to how many are chosen from each group. Dividing each weight by that sum gives the stated conditional probability, with no dependence on $p$. $\square$

This is the multivariate hypergeometric distribution. It can also be understood by imagining that, under the null and conditional on $m$, the $m$ successes occupy a uniformly selected $m$-subset of the $N$ individuals. A table’s weight is the number of such subsets that produce its group counts.

Worked Example: Enumerating the Conditional Distribution

Three ordered groups have sizes $n=(2,1,2)$, and the total number of successes is $m=2$. Use scores $a=(0,1,2)$. Enumerate all feasible tables and calculate their conditional probabilities.

The feasible success-count vectors must sum to 2 and obey the group capacities. They are $(0,0,2)$, $(0,1,1)$, $(1,0,1)$, $(1,1,0)$, and $(2,0,0)$. Their respective weights $\prod_j{n_j\choose y_j}$ are

1,\quad 2,\quad 4,\quad 2,\quad 1.

For example, vector $(1,0,1)$ has weight ${2\choose1}{1\choose0}{2\choose1}=2\cdot1\cdot2=4$. The normalizing denominator is ${5\choose2}=10$, which also equals the sum $1+2+4+2+1$. The scores $T=\sum_j a_jy_j$ for these vectors, in the same order, are $4,3,2,1,0$. Thus, for instance, $P(T=2\mid\text{margins})=4/10$. This enumeration illustrates that tables with the same fixed margins need not be equally probable.

Exact Trend Probabilities

For an observed table $y^{\mathrm{obs}}$, calculate $T_{\mathrm{obs}}=\sum_j a_jy_j^{\mathrm{obs}}$. For a prespecified increasing trend, the one-sided exact p-value is the conditional probability of a score at least as large as the observed score:

p_+=P(T\geq T_{\mathrm{obs}}\mid n_1,\ldots,n_K,m).

To calculate it, sum the conditional table probabilities over all feasible vectors whose score meets the inequality. For a decreasing-trend alternative, use $P(T\leq T_{\mathrm{obs}}\mid\text{margins})$. A two-sided exact p-value is not unique unless its tail convention is specified. One possible convention, used in the next example, counts scores at least as far from the conditional null mean as the observed score:

p_{\mathrm{2s}} = P\left(\left|T-E[T\mid\text{margins}]\right| \geq \left|T_{\mathrm{obs}}-E[T\mid\text{margins}]\right| \;\middle|\;\text{margins}\right).

Worked Example: A Two-Sided Exact Trend Test

Use ordered group sizes $n=(1,2,2)$, scores $a=(1,2,3)$, and $m=2$ total successes. Suppose the observed success counts are $(0,1,1)$. Calculate the one-sided increasing-trend p-value and the two-sided p-value defined by distance from the conditional mean.

The five feasible vectors and their weights are $(0,0,2)$ with weight 1, $(0,1,1)$ with weight 4, $(0,2,0)$ with weight 1, $(1,0,1)$ with weight 2, and $(1,1,0)$ with weight 2. The weights sum to ${5\choose2}=10$. Their score-weight pairs are, respectively, $(6,1),(5,4),(4,1),(4,2),(3,2)$. The observed vector has $T_{\mathrm{obs}}=5$. Therefore, the upper-tail probability is

p_+=P(T\geq5\mid\text{margins})=\frac{1+4}{10}=0.5.

The conditional mean is the weighted average

E[T\mid\text{margins}] = \frac{6(1)+5(4)+4(1)+4(2)+3(2)}{10} = \frac{44}{10}=4.4.

The observed score is $0.6$ above this mean. Scores at least $0.6$ away from $4.4$ are $3,5,$ and $6$: their distances are $1.4,0.6,$ and $1.6$. Score $4$ is only $0.4$ away, so neither table with that score belongs in this two-sided tail. The required weights are $2+4+1=7$, giving $p_{\mathrm{2s}}=7/10=0.7$. The exact answer depends on the stated two-sided convention; including score 4 merely because it is numerically different from the observed score would be incorrect.

Conditional Mean and Variance of the Score

The conditional mean provides a natural center for the score, and the variance describes its spread under the null. To derive both, list the group score $a_j$ once for each individual in group $j$, producing $N$ individual scores $a_1^*,\ldots,a_N^*$. Define their mean and finite-population variance by

\bar a=\frac{1}{N}\sum_{i=1}^N a_i^*, \qquad V_a=\frac{1}{N}\sum_{i=1}^N(a_i^*-\bar a)^2.

Result: Conditional on the margins, $$ E[T\mid\text{margins}]=m\bar a. $$ If $N>1$, then $$ \operatorname{Var}(T\mid\text{margins}) = \frac{m(N-m)}{N-1}V_a. $$ If $N=1$, the conditional variance is $0$.

Derivation. Given $m$, let $I_i$ indicate whether individual $i$ is one of the $m$ successes. The success positions are a uniformly selected $m$-subset, so $E[I_i\mid m]=m/N$. Since $T=\sum_i a_i^*I_i$, linearity of expectation gives

E[T\mid m]=\sum_{i=1}^N a_i^*\frac{m}{N}=m\bar a.

For $N>1$, the indicator variances and distinct-indicator covariances are

\operatorname{Var}(I_i\mid m)=\frac{m(N-m)}{N^2}, \qquad \operatorname{Cov}(I_i,I_k\mid m) = \frac{m(m-1)}{N(N-1)}-\frac{m^2}{N^2} = -\frac{m(N-m)}{N^2(N-1)} \quad(i\ne k).

Set $b_i=a_i^*-\bar a$, so $\sum_i b_i=0$ and $T-m\bar a=\sum_i b_iI_i$. Expanding the variance using the indicator variances and covariances, and using $\sum_{i\ne k}b_ib_k=-\sum_i b_i^2$, yields

\begin{aligned} \operatorname{Var}(T\mid m) &=\frac{m(N-m)}{N^2}\sum_i b_i^2 -\frac{m(N-m)}{N^2(N-1)}\sum_{i\ne k}b_ib_k\\ &=\frac{m(N-m)}{N^2}\sum_i b_i^2 +\frac{m(N-m)}{N^2(N-1)}\sum_i b_i^2\\ &=\frac{m(N-m)}{N(N-1)}\sum_i b_i^2 =\frac{m(N-m)}{N-1}V_a. \end{aligned}

When $N=1$, there is only one possible individual and the fixed margins determine whether that individual is a success; hence $T$ is fixed and its conditional variance is $0$. This handles the case separately rather than using a formula with denominator $N-1$. $\square$

Worked Example: Checking the Conditional Center

For the margins and scores in the preceding two-sided example, verify the conditional mean using the individual-score formula. The five individual scores are $1,2,2,3,3$, so $N=5$ and $\bar a=(1+2+2+3+3)/5=11/5=2.2$. With $m=2$, the result gives $E[T\mid\text{margins}]=2(2.2)=4.4$, matching the weighted enumeration.

The same result follows directly from group totals: $m\bar a=2(11/5)=22/5$. A check from the enumeration is $[6(1)+5(4)+4(1)+4(2)+3(2)]/10=44/10=22/5$. The agreement checks both the score weights and the conditional probabilities.

Why the Exact p-Value Controls Error

For any fixed margins, the score has a finite conditional distribution. The upper-tail p-value $P(T\geq T_{\mathrm{obs}}\mid\text{margins})$ is therefore a valid, possibly conservative, conditional p-value under the null. To see why, list the distinct possible score values in increasing order. The upper-tail probability is nonincreasing as the observed score increases. At a chosen level $\alpha$, the scores that cause rejection form an upper segment of this list. If $t_*$ is its smallest score, the conditional probability of rejection is $P(T\geq t_*\mid\text{margins})$, which is itself at most $\alpha$ by the definition of the rejecting segment. If there is no such score, the rejection probability is zero. Thus the conditional type I error is at most $\alpha$, for every feasible set of margins.

The same reasoning applies to a prespecified two-sided statistic such as $D=|T-E[T\mid\text{margins}]|$, using its upper tail. Discreteness means the rejection probability will often be strictly less than the nominal level. This conservatism is not a computational defect: it reflects the fact that the available conditional probabilities may not permit a rejection region of exactly size $\alpha$.

Worked Example: Unequally Spaced Group Scores

Take three ordered groups with sizes $n=(2,2,1)$, total successes $m=2$, and scores $a=(0,1,4)$. Suppose the observed success vector is $(1,0,1)$, giving $T_{\mathrm{obs}}=4$. Find its exact increasing-trend p-value.

The feasible vectors, their weights, and their scores are $(2,0,0)$ with weight 1 and score 0; $(1,1,0)$ with weight 4 and score 1; $(0,2,0)$ with weight 1 and score 2; $(1,0,1)$ with weight 2 and score 4; and $(0,1,1)$ with weight 2 and score 5. The weights add to $10={5\choose2}$. The observed score is 4, so the upper tail includes the last two vectors and has probability

p_+=\frac{2+2}{10}=0.4.

The jump from score 1 to score 4 is intentional: the scores define the trend statistic, so the test is not treating the ordered groups as equally spaced. Different prespecified scores can produce a different ranking of tables and a different exact p-value.

Practical Choices and Common Pitfalls

An exact calculation requires only the fixed margins, the score definition, and the observed table. For small tables, enumerate every feasible vector and sum its hypergeometric weight. For larger tables, enumeration can be computationally demanding; the test definition remains the same, but efficient computation becomes important.

Ignoring the order. A generic association test and a trend test answer different questions. The trend score uses the category order and a chosen set of scores.
Using equal probabilities for feasible tables. The conditional probability is proportional to $\prod_j{n_j\choose y_j}$, not constant across tables. Normalize by ${N\choose m}$.
Choosing scores after seeing the results. Score selection changes the statistic and its tail ordering. Prespecify scores to keep the test interpretable.
Leaving the two-sided convention unstated. There is no single automatic two-sided rule for every discrete score distribution. State the rule and include exactly the tables it specifies.

The central idea is straightforward: order the groups, define a score that expresses the trend of interest, and compare the observed score with its conditional distribution under the null. Conditioning on both margins makes the calculation exact while removing the unknown common success probability.

Check Your Understanding

Use the definitions and conditional calculations above to answer the following questions.

For fixed group sizes and total successes, why does the null conditional distribution not depend on the common success probability?
With scores increasing across groups, which conditional tail tests for a positive trend? Which tail tests for a negative trend?
For $n=(2,1,2)$ and $m=2$, what is the conditional probability of the table $(1,0,1)$, and why is it not $1/5$?
Why must the $N=1$ case be handled separately in the stated variance formula?
What information must be specified before reporting a two-sided exact trend p-value?