Why the Choice Depends on the Data Structure
Conditional exact and asymptotic procedures answer related questions, but they rely on different approximations. A conditional exact procedure uses the distribution of the data given specified statistics, often fixing margins to remove nuisance parameters. An asymptotic procedure instead uses a limiting reference distribution, such as a normal or chi-square distribution. Exact inference can be attractive when the conditional sample space is small or the observed data are sparse. It is not automatically preferable: conditioning can discard information, exact distributions can be costly to calculate, and discrete tests can be conservative.
There is no universal cell-count threshold that settles the choice. A count that looks small may or may not matter, depending on the model, the statistic, and how much information the data contain. The practical question is whether the approximation used by an asymptotic test is credible for the particular design and parameter value, and whether the conditional analysis targets the intended scientific question.
What the Conditional Distribution Buys
A simple \(2\times2\) table shows why conditioning is useful. Let \(A\) be the number of successes in a group of size \(n_1\), and let the other group have size \(n_0\). Suppose the total number of successes is \(m\), and the odds ratio comparing the groups is \(\theta\). Under independent binomial sampling within groups, conditioning on both group sizes and the total number of successes removes the baseline success odds. The remaining conditional distribution depends on \(\theta\) alone.
To derive the result, write the success odds in the second group as \(\lambda\); then the first group's odds are \(\theta\lambda\). For a possible value \(A=a\), the joint binomial probability, apart from factors not depending on \(a\), is \( \binom{n_1}{a}\binom{n_0}{m-a}(\theta\lambda)^a\lambda^{m-a} \). Because \(a+(m-a)=m\), the odds contribution is \(\theta^a\lambda^m\). Conditional on the margins, \(\lambda^m\) and the binomial denominators are constant across feasible tables and cancel in normalization. Thus the conditional probabilities have the stated weights. When \(\theta=1\), those weights are the hypergeometric probabilities. \(\square\)
This derivation illustrates both the strength and the cost of conditioning. It gives a finite-sample distribution that no longer depends on the baseline odds, but inference is now based on variation remaining after the margins have been fixed. If the margins leave few possible tables, the resulting distribution may be highly discrete.
Worked Example: A Sparse Table and an Exact Tail
Suppose both groups have size 5, there are 5 successes in total, and 4 successes occur in the first group. The table is \((4,1)\) in the first group and \((1,4)\) in the second. Test the null odds ratio \(\theta=1\) against the one-sided alternative \(\theta>1\), using the conditional upper tail.
Under the null, \(A\) has a hypergeometric distribution with support \(0,1,2,3,4,5\). Its unnormalized probabilities are \(\binom{5}{a}\binom{5}{5-a}=\binom{5}{a}^2\). The weights for \(a=0,\ldots,5\) are \(1,25,100,100,25,1\), which sum to 252. The exact one-sided p-value is therefore \(P(A\geq4\mid\text{margins},\theta=1)=(25+1)/252=26/252\approx0.1032\).
For comparison, the null conditional mean is 2.5 and the hypergeometric variance is \(5(5/10)(1-5/10)(10-5)/(10-1)=25/36\). The observed standardized value is \((4-2.5)/(5/6)=1.8\), giving an uncorrected normal upper-tail approximation of about 0.0359. That approximation is noticeably different because the conditional support has only six points. This is not a universal verdict against the approximation; it is a warning that a smooth limiting distribution may represent a small discrete sample space poorly.
The table's expected cell counts under the null are all 2.5. Such counts are a useful diagnostic, not a theorem specifying when an asymptotic test fails. The exact calculation is transparent here because the support is small; in larger problems, obtaining the exact distribution may itself be difficult.
When Asymptotic Inference Is Credible
Wald, score, and likelihood-ratio procedures are often justified by large-sample theory. In a regular finite-dimensional model, typical requirements include independent sampling units (or a correctly modelled dependence structure), an identifiable parameter in the interior of its parameter space, a smooth likelihood, nonsingular information, and increasing information as the sample grows. Under suitable moment and convergence conditions, the estimator is approximately normal and common test statistics have their usual normal or chi-square limits under the null.
These are asymptotic statements, not guarantees for every finite sample. Sparse outcomes can leave little effective information even when the total sample size appears substantial. In logistic regression, for example, separation can make a maximum-likelihood coefficient diverge rather than yield a finite estimate. In that setting, a Wald calculation based on a finite estimate and standard error is not merely imprecise; its usual construction is unavailable. Conditional exact methods can offer a valid finite-sample alternative for a specified conditional question, but they do not repair a misspecified model or create information absent from the data.
Worked Example: A Small Matched-Set Analysis
Suppose there are three independent matched pairs that are discordant on exposure and outcome in the same direction, with no discordant pair in the opposite direction. Let \(A\) count discordant pairs in that direction, and let \(\theta\) be the conditional odds ratio. Given that all three pairs are discordant, the model gives \(A\sim\operatorname{Binomial}(3,\theta/(1+\theta))\).
Under \(\theta=1\), each pair is equally likely to favor either direction, so \(P(A=3)= (1/2)^3=1/8=0.125\). The exact one-sided p-value for the observed \(A=3\) is 0.125. The conditional likelihood is proportional to \(\theta^3/(1+\theta)^3\), which increases toward 1 as \(\theta\) increases. Its maximum occurs only in the limit \(\theta\to\infty\); there is no finite maximum-likelihood estimate.
A usual Wald interval built from the log of a finite estimate cannot be formed. An exact one-sided 95% lower confidence limit can still be obtained by inverting the upper-tail test: set \(p^3=0.05\), where \(p=\theta/(1+\theta)\). Then \(p=0.05^{1/3}\approx0.3684\), so \(\theta=p/(1-p)\approx0.5833\). The resulting one-sided interval is approximately \([0.583,\infty)\). It is wide because three discordant pairs provide limited information, not because exact calculation has failed.
Calibration, Discreteness, and Computational Cost
A nonrandomized exact tail test controls its conditional type I error because its p-value is super-uniform. The finite-support condition matters: it is appropriate for a finite conditional sample space, but should not be silently extended to arbitrary discrete distributions.
For a proof, list the distinct statistic values in decreasing order, \(t_1>\cdots>t_J\), and let \(q_j=P_0\{T(X)\geq t_j\}\). The sequence \(q_j\) is nondecreasing. At a point with statistic value \(t_j\), the p-value is \(q_j\). If there is no \(j\) with \(q_j\leq\alpha\), the event \(\{p(X)\leq\alpha\}\) is empty. Otherwise let \(r\) be the largest such index. Since \(q_j\) is nondecreasing, the event consists exactly of statistic levels \(t_1,\ldots,t_r\), whose total null probability is \(q_r\leq\alpha\). This proves the claim. Ties are included in the tail probability, which is why the result can be conservative rather than exactly equal to \(\alpha\). \(\square\)
Inverting conservative exact tests gives confidence sets with at least the nominal coverage under the stated model and conditioning scheme. Discreteness often makes such sets wider than a smooth approximation would suggest, but there is no rule that every exact interval must be wider in every sample. The choice of tail ordering, nuisance-parameter treatment, and available information all matter.
Computational cost also depends on the conditional sample space, not simply on the number of observations. As a simple illustration, a stratum with row and column margins both equal to 2 has three feasible values for its upper-left cell: 0, 1, or 2. For \(S\) such strata, the joint space has \(3^S\) combinations if enumerated directly. With 20 strata this is \(3^{20}=3,486,784,401\) combinations. Algorithms can sometimes exploit structure or combine distributions more efficiently, but exhaustive enumeration may still be infeasible. Monte Carlo conditional methods can reduce computation, at the cost of simulation uncertainty and the need to use a valid simulation-based procedure.
A Practical Decision
State what is being tested or estimated, which statistics are conditioned on, and whether the resulting conditional question addresses the scientific aim.
Inspect the pattern of counts, effective sample size, parameter estimates, and model regularity. Sparse cells, boundary estimates, or separation make routine asymptotic calculations questionable.
Determine whether the relevant support can be enumerated or computed efficiently, and whether the resulting exact test or interval will be informative enough for the decision.
Explain the assumptions, conditioning scheme, and computation used. If feasible, compare a carefully matched asymptotic analysis with the conditional result rather than treating either label as a guarantee of quality.
Conditioning has limitations beyond computation. Fixing margins can remove nuisance parameters, but it can also discard variation that would otherwise inform the parameter of interest. Conditional inference may therefore be less efficient than an appropriate unconditional analysis. Moreover, exactness refers to calibration under a specified probability model; it does not protect against selection bias, measurement error, dependence that the model ignores, or a wrongly chosen estimand. A small exact p-value cannot compensate for an unsuitable design or model.
Check Your Understanding
Use the model assumptions and inferential goal to justify each answer.
- Why can a conditional exact test be conservative even when its stated level is valid?
- In the sparse \(2\times2\) example, what feature of the conditional distribution helps explain the difference between the exact and normal-approximation p-values?
- Name two regularity or information conditions that support routine asymptotic inference.
- Why does conditioning on margins sometimes reduce information about the parameter of interest?
- For 12 independent strata, each with three feasible conditional tables, how many joint configurations are there under direct enumeration?
- Does an exact conditional analysis remain trustworthy if the assumed sampling model is wrong? Explain.