Why the Outcome Counts Matter
In a two-group study with a binary outcome, the sample sizes alone do not determine the precision of a relative effect estimate. For a log odds ratio, both events and non-events in each group contribute to precision. For a log relative risk, the event counts and the estimated risks determine its variance. Consequently, two studies with the same enrollment can provide quite different information about these effects.
Here, group 1 is the intervention or exposed group and group 0 is the comparison group. Let \(n_1\) and \(n_0\) be their sample sizes, and let \(x_1\) and \(x_0\) be their numbers of participants with the outcome. Write \(\hat p_1=x_1/n_1\) and \(\hat p_0=x_0/n_0\). The corresponding numbers without the outcome are \(n_1-x_1\) and \(n_0-x_0\).
The formulas below assume independent participants, independent groups, and a binary outcome recorded for each participant. Within each group, outcomes are modeled as independent Bernoulli observations with a common probability \(p_g\), so \(X_g\sim\operatorname{Binomial}(n_g,p_g)\). The group sizes are treated as fixed. These assumptions exclude, for example, unaccounted clustering or repeated outcomes on the same person.
Information for the Log Odds Ratio
The odds in group \(g\) are \(p_g/(1-p_g)\). The log odds ratio comparing group 1 with group 0 is \(\theta_{\mathrm{OR}}=\log\{[p_1/(1-p_1)]/[p_0/(1-p_0)]\}\). Its sample estimate is the logarithm of the sample odds ratio. When all four outcome cells are positive, its usual large-sample variance estimate is a sum of four reciprocal cell counts.
Thus the plug-in information is the reciprocal of the displayed variance. The formula shows why the number of events alone is insufficient: the non-event cells also enter. A small cell contributes a large reciprocal and therefore increases the estimated variance. In the corresponding model-based expression, the contribution from group \(g\) is \(1/[n_g p_g(1-p_g)]\); substituting \(\hat p_g\) gives \(1/x_g+1/(n_g-x_g)\).
Worked Example: Log Odds Ratio From Four Cells
Suppose, hypothetically, that 40 of 100 participants in group 1 and 25 of 100 in group 0 experience the outcome. The other cell counts are 60 and 75. The groups are independent, and all four cells are nonzero, so the usual plug-in Wald calculation is defined.
The sample odds ratio is \((40/60)/(25/75)=2\), giving \(\widehat\theta_{\mathrm{OR}}=\log(2)=0.6931\). Its estimated variance is \(1/40+1/60+1/25+1/75=0.02500+0.01667+0.04000+0.01333=0.09500\).
Therefore the standard error is \(\sqrt{0.09500}=0.3082\), and the approximate information is \(1/0.09500=10.53\). This is information for the log odds ratio, not a count of participants or events. A large-sample Wald interval would be \(0.6931\pm1.96(0.3082)\), approximately \((0.0890,1.2972)\) on the log scale. The interval is an approximation; it does not repair any failure of the large-sample assumptions.
Information for the Log Relative Risk
The relative risk is \(p_1/p_0\), and its log is \(\theta_{\mathrm{RR}}=\log(p_1)-\log(p_0)\). The sample estimate is \(\widehat\theta_{\mathrm{RR}}=\log(\hat p_1/\hat p_0)\), defined when both observed event counts are positive. To obtain its approximate variance, use the binomial variance of each sample proportion and the delta method.
The delta method therefore gives \(\operatorname{Var}\{\log(\hat p_g)\}\approx(1-p_g)/(n_gp_g)\). Since the groups are independent, the variance of the difference of their log risks is the sum of the two group variances. Substituting observed proportions yields a convenient count-based formula:
Unlike the log odds ratio formula, this expression contains no reciprocal of the non-event count as a separate term. Non-events still matter through the risk estimate and the subtraction of \(1/n_g\). The formula is a large-sample approximation for the log of a risk ratio, not an exact variance identity.
Worked Example: Log Relative Risk and Its Information
In a hypothetical study, 36 of 120 participants in group 1 and 24 of 120 in group 0 experience the outcome. The independent-binomial assumptions are plausible for this calculation, and both event counts are sufficiently positive for a basic large-sample approximation to be considered.
The estimated risks are \(\hat p_1=36/120=0.3000\) and \(\hat p_0=24/120=0.2000\). Thus the estimated relative risk is \(0.3000/0.2000=1.500\), and the estimated log relative risk is \(\log(1.500)=0.4055\).
Using the count formula, the variance estimate is \((1/36-1/120)+(1/24-1/120)\) \(=(0.02778-0.008333)+(0.04167-0.008333)=0.05278\). The standard error is \(\sqrt{0.05278}=0.2297\), and the approximate information is \(1/0.05278=18.95\). The log-scale Wald interval is approximately \(0.4055\pm1.96(0.2297)=(-0.0448,0.8557)\); exponentiating gives a relative-risk interval of approximately \((0.9562,2.353)\). These are approximate intervals, and their interpretation concerns the population relative risk, not the probability that a fixed parameter lies in the computed interval.
What Counts Say About Precision
The reciprocal terms make the influence of the counts transparent. For the log odds ratio, a group contributes \(1/x_g+1/(n_g-x_g)\). Precision can be limited by few events or by few non-events. For the log relative risk, the group contribution is \(1/x_g-1/n_g\); when events are rare, it is close to \(1/x_g\), so the event count is especially influential. For either effect, doubling enrollment does not necessarily double information if the outcome probabilities or allocation change at the same time.
Information is useful in sequential designs because it quantifies precision on the scale of the effect being monitored. If the target parameter is a log odds ratio, use its variance formula; if it is a log relative risk, use the log-relative-risk formula. These information measures are not interchangeable. The information fraction is based on information for the particular statistic and parameter, not simply on the fraction of participants enrolled.
Worked Example: Sparse Counts and a Warning About Wald Information
Consider a hypothetical study with 2 events among 80 participants in group 1 and 1 event among 80 in group 0. Participants are assumed independent, but the event counts are very small. The formulas can be evaluated because no event cell is zero; the small counts should nevertheless trigger caution about normal approximations.
For the log relative risk, the estimate is \(\log\{(2/80)/(1/80)\}=\log(2)=0.6931\). Its plug-in variance is \((1/2-1/80)+(1/1-1/80)=0.4875+0.9875=1.475\). Thus the estimated standard error is \(\sqrt{1.475}=1.2145\), and the plug-in information is \(1/1.475=0.6780\).
For the log odds ratio, the estimated variance is \(1/2+1/78+1/1+1/79=1.5255\), giving information \(1/1.5255=0.6555\). These large variances correctly signal weak precision, but the Wald approximations may still be poorly calibrated: one event cell contains only a single observation, so the estimated sampling distribution can be far from normal. A numerical information value should not be mistaken for evidence that the approximation is reliable.
Derivation and Conditions for Use
The two formulas follow from independent group estimates. For the relative risk, apply the derivative \(1/p_g\) to the variance of \(\hat p_g\), then add the group contributions because the groups are independent. For the odds ratio, apply the derivative of \(\log\{p/(1-p)\}\), which is \(1/[p(1-p)]\), to each sample proportion. This gives \(1/[n_gp_g(1-p_g)]\) for group \(g\); replacing \(p_g\) by \(x_g/n_g\) gives the two reciprocal-cell terms for that group. Adding across groups yields the stated log odds ratio variance.
These derivations rely on large-sample approximations. In practice, inspect all relevant cells, not only total enrollment. A zero event count makes a log relative risk estimate undefined; a zero event or non-event cell makes the ordinary log odds ratio estimate or its reciprocal-cell variance undefined. Very small positive cells can also produce unstable plug-in estimates and unreliable Wald inference. Do not silently add a continuity correction and then describe the result as the unmodified formula: any correction changes the calculation and should be specified and justified.
The independent-binomial model can also fail if participants are clustered, observations are paired, or outcome probabilities vary in a way not represented by the model. Such settings require a variance calculation suited to their dependence or modeling structure. Likewise, if the scientific target is adjusted for covariates, these two unadjusted formulas do not automatically describe the information in the adjusted contrast.
Comparison, Diagnostics, and Limitations
| Approach | What determines the variance | Main caution |
|---|---|---|
| Log odds ratio plug-in formula | Events and non-events in both groups | Every one of the four cells must be positive for the ordinary formula. |
| Log relative risk delta-method formula | Event counts, group sizes, and estimated risks | Both observed event counts must be positive; small counts undermine the approximation. |
| Risk difference approach | Variances of the two estimated risks | It targets a different effect scale and cannot be substituted for log-scale information. |
| Likelihood-based or exact methods | The specified binomial model and observed data | They can avoid relying on a simple Wald approximation, but require their own assumptions and inferential procedure. |
A useful diagnostic is to recompute each group contribution separately. If one reciprocal term dominates, precision is being driven by a small cell. Check that the observed risks are not at the boundary, that counts and group sizes agree, and that the direction of the contrast matches the chosen group ordering. For planning, use plausible prespecified probabilities and sample sizes to calculate expected variance; observed plug-in information from a small interim sample can be highly variable.
A common mistake is to say that a study with 200 participants has the same information as any other study with 200 participants. That reasoning ignores outcome frequency and, for the log odds ratio, the non-event cells. The correction is to calculate the variance for the actual effect scale from the group-specific counts or from planning probabilities. Another mistake is to treat a finite reciprocal variance as proof that a Wald approximation is adequate. Sparse cells may yield a number while still making the underlying normal approximation poor.
Check Your Understanding
Answer each question, then compare with the solutions below.
- In the log odds ratio variance formula, why can event counts alone not determine the estimated variance?
- A hypothetical group has \(n_1=100\) and \(x_1=20\). What is its contribution to the log relative risk variance?
- What happens to the ordinary log relative risk estimate if one group has no observed events?
- Why can a calculated Wald information value based on a very small cell still be misleading?
Solutions. (1) The variance includes reciprocal counts for both events and non-events in each group; changing non-event counts changes precision even if event counts are held fixed. (2) The contribution is \(1/20-1/100=0.05000-0.01000=0.04000\). (3) The estimated risk in that group is zero, so the log of the estimated risk ratio is undefined; the ordinary plug-in formula cannot be used as stated. (4) Its formula may be numerically evaluable, but sparse data can make the sampling distribution far from normal, undermining the Wald approximation and its calibration.