Why Test an Odds Ratio?
A confidence interval for an odds ratio describes a range of values compatible with the data. A hypothesis test answers a focused question: do the data provide evidence that the population odds ratio differs from one? An odds ratio of one means that the two groups have equal odds of the outcome. Testing this value is useful when the comparison is naturally expressed in odds and the goal is to evaluate evidence against equal odds.
The test uses the logarithm of the sample odds ratio. This transforms the null value from one to zero and, under suitable large-sample conditions, gives a statistic with an approximately standard normal distribution. The test concerns an association between group and outcome; by itself, it does not establish that group membership caused the outcome.
Definition and Hypotheses
Suppose two independent groups are classified according to whether a binary outcome occurred. Label the outcome of interest as the event. Let \(a\) and \(b\) be the event and non-event counts in group 1, and let \(c\) and \(d\) be the event and non-event counts in group 2.
| Event | No event | |
|---|---|---|
| Group 1 | \(a\) | \(b\) |
| Group 2 | \(c\) | \(d\) |
The usual two-sided test is \(H_0:\mathrm{OR}=1\) versus \(H_a:\mathrm{OR}\ne1\). A one-sided alternative may be appropriate if the research question was specified in advance: \(H_a:\mathrm{OR}>1\) or \(H_a:\mathrm{OR}<1\). The direction refers to group 1 relative to group 2, so the group order and event definition must be stated consistently.
The Log Odds Ratio Test
The test is based on \(\log(\widehat{\mathrm{OR}})\). If the null hypothesis is true, the population log odds ratio is \(\log(1)=0\). For positive cell counts, the estimated standard error is
The Wald test statistic is the observed log odds ratio minus its null value, divided by its estimated standard error. Under the null, it is approximately standard normal for sufficiently large samples:
For a two-sided alternative, the p-value is \(2P(Z\geq |z_{\mathrm{obs}}|)\), where \(Z\sim N(0,1)\). For \(H_a:\mathrm{OR}>1\), it is \(P(Z\geq z_{\mathrm{obs}})\); for \(H_a:\mathrm{OR}<1\), it is \(P(Z\leq z_{\mathrm{obs}})\). Compare the p-value with the chosen significance level \(\alpha\). A p-value no greater than \(\alpha\) is evidence against \(H_0\) in favor of the stated alternative. It is not the probability that \(H_0\) is true.
Why This Standard Error Has This Form
A short approximation explains the four reciprocal counts. Within one group, suppose the event probability is \(p\), based on \(n\) independent binary observations. The sample proportion has approximate variance \(p(1-p)/n\). The log odds corresponding to a probability \(p\) is \(\log\{p/(1-p)\}\); its derivative with respect to \(p\) is \(1/[p(1-p)]\). The delta method therefore gives the approximate variance of the estimated log odds:
where \(a=np\) and \(b=n(1-p)\) are the event and non-event counts. For independent groups, the variance of the difference between their estimated log odds is the sum of their variances. That gives \(1/a+1/b+1/c+1/d\), and taking the square root gives the standard error above. This is a large-sample approximation, not an exact formula for every possible table.
Assumptions and Conditions
Before using the test, make sure the data and design support the comparison. The groups must be independent, and observations within each group must be independent. Each observation contributes to exactly one group and has a binary outcome recorded in one of the two categories. The sampling or assignment design must support inference to the populations named in the conclusion. To use the normal approximation reliably, the cell counts should not be too small; all four counts must be positive for the displayed estimate and standard error to be defined.
The examples below use independent groups and counts large enough for the normal approximation to be reasonable. When cell counts are very small or a cell is zero, this Wald calculation can be unstable or unavailable; it should not be used mechanically.
Worked Example: Evidence That Event Odds Differ
In a hypothetical comparison, two independently sampled groups each contain 200 people. The event occurred for 80 people in group 1 and 50 people in group 2. The other 120 and 150 people, respectively, did not experience the event. Test \(H_0:\mathrm{OR}=1\) against \(H_a:\mathrm{OR}\ne1\) at \(\alpha=0.05\).
Check the setup. The groups are independent, each person's outcome is recorded once as event or no event, and all four counts are at least 50. Assuming the independent sampling described, the normal approximation is reasonable for this illustration.
Estimate the odds ratio. Group 1 has event odds \(80/120\), and group 2 has event odds \(50/150\). Thus
Calculate the standard error and statistic.
For the two-sided alternative, \(p\text{-value}=2P(Z\geq3.180)\approx0.00147\). Since \(0.00147<0.05\), reject \(H_0\). These data provide evidence that the population event odds differ between the groups; the estimated odds are higher in group 1, with a sample odds ratio of 2. This conclusion is about odds, not a claim that the event is twice as probable.
Worked Example: A One-Sided Test for Lower Odds
In a second hypothetical study, an independent group 1 has 18 event outcomes and 82 non-events; an independent group 2 has 30 events and 70 non-events. Test \(H_0:\mathrm{OR}=1\) against the pre-specified alternative \(H_a:\mathrm{OR}<1\) at \(\alpha=0.05\).
Check the setup. The groups are independent, the outcome is binary, and the four positive counts are 18, 82, 30, and 70. With these counts, the large-sample approximation is usable for this illustration.
The sample odds ratio and its logarithm, calculated from the counts, are
The standard error and test statistic are
Because the alternative is lower odds, the p-value is the lower-tail probability: \(P(Z\leq-1.970)\approx0.0244\). Since \(0.0244<0.05\), reject \(H_0\). The data provide evidence that the population event odds in group 1 are lower than those in group 2. The estimated odds ratio, about 0.512, describes the observed comparison; the test does not establish a causal effect.
Worked Example: Equal Sample Odds Do Not Prove Equal Population Odds
In a third hypothetical comparison, each independent group has 40 event outcomes and 60 non-events. Test \(H_0:\mathrm{OR}=1\) against \(H_a:\mathrm{OR}\ne1\). Assume independent observations, a binary outcome, and a sampling design that supports inference to the populations of interest. All four counts are positive and sufficiently large for this example's normal approximation.
The sample odds ratio is exactly one, so its logarithm is zero. The standard error is
The two-sided p-value is \(2P(Z\geq0)=1\). These data provide no evidence against equal population odds. They do not prove that the population odds are equal: a sample estimate can equal one even when uncertainty about the population comparison remains.
Common Mistakes and Why the Test Matters
- Reversing the comparison. The odds ratio is group 1 odds divided by group 2 odds. Reversing the group order changes the estimate to its reciprocal and reverses the sign of the log odds ratio. State the group order and event before interpreting the result.
- Using the wrong standard error. This test uses the standard error of the log odds ratio, \(\sqrt{1/a+1/b+1/c+1/d}\). Do not substitute a pooled standard error for a difference in proportions; that is a different test statistic and targets a different parameter.
- Treating a nonsignificant result as proof of equality. A large p-value means the data do not provide strong evidence against the null at the chosen threshold. It does not show that the population odds are exactly equal.
- Calling the p-value the probability that the null is true. The p-value is calculated under the assumption that \(H_0\) is true. It measures how unusual a test statistic at least this extreme would be under that null model and the test assumptions.
- Interpreting an odds ratio as a probability ratio. An odds ratio compares odds, not event probabilities. Keep the conclusion in terms of odds unless a separate analysis supports a probability-based interpretation.
Testing an odds ratio against one supplies a direct, scale-appropriate test of equal odds. The logarithm centers the null at zero, and the reciprocal-count standard error quantifies the estimated sampling variability. A sound conclusion depends on using the correct group and event definitions, checking the independence and approximation conditions, and interpreting the p-value in the context of the stated alternative.
Check Your Understanding
Use the definitions and procedure above to answer these questions.
- For a table with counts \(a,b,c,d\), write the sample odds ratio and state what a value of one means.
- Why is the test performed on \(\log(\widehat{\mathrm{OR}})\) rather than directly comparing the estimate with one using the same standard error?
- If \(z_{\mathrm{obs}}=-2.1\) and the alternative is \(H_a:\mathrm{OR}<1\), which tail of the standard normal distribution gives the p-value?
- What happens to the odds ratio and the sign of its logarithm if the labels for group 1 and group 2 are switched?
- Explain why a p-value greater than \(\alpha\) does not establish that the population odds ratio equals one.