Why Exact Coefficient Tests Use Conditional Allocations
In logistic regression, a coefficient describes how the log odds change with a predictor while other model terms are held fixed. When samples are small or outcomes are sparse, a large-sample reference distribution for a coefficient test can be unreliable. A conditional exact test instead uses the finite set of response allocations compatible with specified sufficient statistics, and calculates the null distribution directly on that set.
The key design choice is which statistics to fix. For a test of one coefficient, conditioning on sufficient statistics for the other slopes can remove their nuisance parameters. For a joint test of several coefficients, those tested coefficients must not be conditioned away. The resulting conditional sample spaces, and therefore the tests, are different.
The Conditional Allocation Law
Consider independent binary responses \(Y_i\), with fixed predictor vectors \(x_i\), and logistic probabilities \(\operatorname{logit}P(Y_i=1)=\alpha_{s(i)}+x_i^\top\beta\). Here \(s(i)\) identifies the stratum of observation \(i\), \(\alpha_s\) is its stratum intercept, and \(\beta\) is the vector of slopes. Write \(y=(y_1,\ldots,y_n)\) for a possible response allocation. Define the stratum totals \(R_s(y)=\sum_{i:s(i)=s}y_i\) and slope statistics \(T(y)=\sum_i x_i y_i\).
The conditional probability of an allocation is proportional to \(\exp\{\sum_s\alpha_sR_s(y)+\beta^\top T(y)\}\). On \(\Omega_r\), the intercept contribution is constant. Normalizing over the allocations therefore gives the following result.
To verify the cancellation, the unconditioned allocation probability is \[ P_{\alpha,\beta}(Y=y)= \frac{\exp\{\sum_s\alpha_sR_s(y)+\beta^\top T(y)\}} {\prod_i[1+\exp\{\alpha_{s(i)}+x_i^\top\beta\}]}. \] For all \(y\in\Omega_r\), the denominator is the same and \(\sum_s\alpha_sR_s(y)=\sum_s\alpha_sr_s\) is constant. Dividing each allocation's probability by the sum of these probabilities over \(\Omega_r\) cancels both constants and yields the displayed conditional law. \(\square\)
Testing One Coefficient
Write \(\beta=(\beta_j,\beta_{-j})\), where \(\beta_j\) is the coefficient of interest. In addition to the stratum totals, condition on \(T_{-j}=t_{-j}\), the sufficient statistics for the other slopes. Among allocations satisfying both sets of constraints, the terms involving \(\beta_{-j}\) are constant. Thus, if \(T_j\) is the statistic for the coefficient being tested, its conditional distribution has the form \[ P_{\beta_j}(T_j=t\mid R=r,T_{-j}=t_{-j}) =\frac{h(t)\exp(\beta_jt)} {\displaystyle\sum_v h(v)\exp(\beta_jv)}. \] Here \(h(t)\) counts the allocations in the conditioned sample space with \(T_j=t\). Only values with \(h(t)>0\) are possible. This formula makes a test of \(H_0:\beta_j=\beta_0\) a one-parameter exact calculation, with null probabilities found by substituting \(\beta_0\).
Worked Example: A One-Sided Test for a Single Slope
There is one stratum with four observations, predictor values \(x=(0,1,2,3)\), and total number of responses equal to two. The observed responses are \(y=(0,0,1,1)\), so \(T=\sum_i x_i y_i=2+3=5\). Test \(H_0:\beta=0\) against \(H_A:\beta>0\).
The six possible allocations select two of the four observations. Their predictor sums are \(0+1=1\), \(0+2=2\), \(0+3=3\), \(1+2=3\), \(1+3=4\), and \(2+3=5\). Under the null, every allocation has weight \(\exp(0T)=1\), so each has probability \(1/6\).
For the specified positive-slope alternative, larger \(T\) is evidence against the null. The inclusive upper-tail p-value is therefore \(P_0(T\geq 5)=1/6\). For the explicitly chosen two-sided statistic \(|T-E_0[T]|\), the null mean is \(E_0[T]=(1+2+3+3+4+5)/6=3\). The observed distance is \(|5-3|=2\); only sums 1 and 5 have distance at least 2. This two-sided p-value is \(2/6=1/3\). The two values answer different alternative hypotheses.
A two-sided exact p-value is not unique until its ordering statistic is specified. One may, for example, use a distance from the null mean, a likelihood-based ordering, or twice a smaller one-sided tail, with an appropriate cap at one. State the choice and include outcomes tied with the observed statistic. With discrete distributions, attainable significance levels are limited; an exact nonrandomized test can consequently be conservative.
Removing Nuisance Slopes by Conditioning
Worked Example: Testing One Slope With a Nuisance Predictor
Consider one stratum with five observations and the model \(\operatorname{logit}P(Y_i=1)=\alpha+\beta_xx_i+\beta_zz_i\). The fixed predictor values are shown below. The observed responses equal one for observations 2 and 4, so the total is two, \(T_z=1\), and \(T_x=1+2=3\). Test \(H_0:\beta_x=0\) against \(H_A:\beta_x>0\), allowing \(\beta_z\) to be unknown.
| Observation | \(x_i\) | \(z_i\) |
|---|---|---|
| 1 | 0 | 0 |
| 2 | 1 | 0 |
| 3 | 0 | 1 |
| 4 | 2 | 1 |
| 5 | 3 | 2 |
Condition on the total of two and on \(T_z=1\), the sufficient statistic for the nuisance slope. The allocations must select one observation with \(z=0\) and one with \(z=1\). There are exactly four: \(\{1,3\},\{1,4\},\{2,3\},\{2,4\}\). Their \(T_x\) values are respectively \(0,2,1,3\).
Under \(H_0\), the conditional weight of each allocation is \(\exp(\beta_zT_z)=\exp(\beta_z)\), the same for all four. After normalization each has probability \(1/4\), regardless of the unknown \(\beta_z\). The observed value \(T_x=3\) is the largest, so the one-sided exact p-value is \(P_0(T_x\geq3\mid R=2,T_z=1)=1/4\).
This test can be uninformative if the chosen conditioning constraints leave \(T_x\) with only one possible value. Before interpreting a conditional test, check that its conditioned sample space contains variation in the statistic for the coefficient of interest.
Joint Hypotheses Require a Different Conditioning Set
For a joint null \(H_0:\beta_S=0\), where \(S\) is a set of coefficients, condition on stratum totals and on sufficient statistics for slopes outside \(S\), if there are such nuisance slopes. Do not condition on the statistics for the coefficients in \(S\): doing so would remove the variation needed to test them. Under the joint null, the conditional distribution over the tested statistics is determined by the allocation counts, and a chosen statistic can summarize evidence in several directions.
Worked Example: A Joint Test of Two Slopes
In one stratum, take five observations with predictor pairs \((x_i,z_i)=(0,0),(0,1),(1,0),(1,1),(1,1)\). Suppose the total response count is two and the observed successes are observations 4 and 5. Test \(H_0:\beta_x=\beta_z=0\). Use the statistic \(S=\max(|T_x-E_0T_x|,|T_z-E_0T_z|)\), a specified two-direction measure of departure.
Under the joint null and conditional on the total, all \(\binom{5}{2}=10\) allocations are equally likely. Each predictor is one for three observations, so the null conditional means are \(E_0T_x=2(3/5)=1.2\) and \(E_0T_z=2(3/5)=1.2\). The observed allocation \(\{4,5\}\) has \((T_x,T_z)=(2,2)\), giving \(S_{\mathrm{obs}}=\max(|2-1.2|,|2-1.2|)=0.8\).
Enumerating all allocations gives the following statistics: \(\{1,2\}:1.2\), \(\{1,3\}:1.2\), \(\{1,4\}:0.2\), \(\{1,5\}:0.2\), \(\{2,3\}:0.2\), \(\{2,4\}:0.8\), \(\{2,5\}:0.8\), \(\{3,4\}:0.8\), \(\{3,5\}:0.8\), and \(\{4,5\}:0.8\). Seven of the ten values are at least the observed \(0.8\), so the exact p-value for this statistic is \(7/10\).
This calculation tests the joint null, not two separate one-coefficient nulls. A different joint statistic can give a different exact p-value; its definition should reflect what departures from the joint null matter scientifically.
Why the Inclusive Exact p-Value Controls Type I Error
The finite-sample validity of the calculation does not depend on a normal approximation. For a fixed null distribution and any prespecified statistic \(S\), define the inclusive upper-tail p-value as \(p(s)=P_0(S\geq s)\).
To prove the result, list the distinct possible statistic values in increasing order as \(s_1<\cdots<s_m\), and let \(q_k=P_0(S\geq s_k)\). The sequence \(q_k\) is nonincreasing. The event \(p(S)\leq\alpha\) consists of statistic values \(s_k\) for which \(q_k\leq\alpha\). If this set is empty, its probability is zero. Otherwise let \(k_0\) be its smallest index. By monotonicity, the event is exactly \(\{S\geq s_{k_0}\}\), whose probability is \(q_{k_0}\leq\alpha\). This proves the inequality. Discreteness explains why the inequality may be strict: a nonrandomized test may not be able to attain every nominal level. \(\square\)
For a one-coefficient test with unknown nuisance slopes, this guarantee applies to the conditional null distribution after fixing their sufficient statistics. It is a conditional guarantee for the specified conditioned sample space; it does not make an unconditioned large-sample approximation exact, nor does it imply that every possible two-sided ordering has identical power.
Common Mistakes
- Conditioning on a tested coefficient's statistic. This fixes the very variation that would provide evidence about that coefficient. For a joint test, retain the sufficient statistics corresponding to all tested coefficients.
- Using the one-coefficient conditioning scheme for a joint null. Conditioning on the other slope statistics is appropriate when removing nuisance slopes for a single-coefficient test; it is not the same as testing all slopes jointly.
- Reporting a two-sided p-value without its ordering rule. Exact two-sided p-values depend on how outcomes in both directions are ranked. State the statistic and include ties.
- Dropping outcomes tied with the observation. The inclusive tail \(P_0(S\geq S_{\mathrm{obs}})\) includes ties and yields the stated finite-sample level control.
- Assuming conditioning always leaves useful information. If the conditioned space has no variation in the target statistic, the conditional test cannot distinguish its possible coefficient values.
Check Your Understanding
Use the conditioning set and the null hypothesis to guide each answer.
- In a stratified logistic model, which response totals remove the stratum intercepts from the conditional allocation law?
- Why can conditioning on the sufficient statistic for a nuisance slope help test one target slope?
- For a joint null on two slopes, why should their two sufficient statistics not both be fixed?
- What additional information must be stated to make a two-sided exact p-value reproducible?
- Explain why an inclusive exact p-value can yield a test whose size is below, rather than equal to, its nominal level.