Information for Binary Risk Differences

Why Information Depends on Both Risks

For a binary outcome, the treatment effect may be expressed as an absolute risk difference: the event probability in one group minus the event probability in the other. Its precision is not determined by the total sample size alone. It also depends on the event probabilities in both groups and on how participants are allocated between them. These details matter when planning a study and when interpreting the information available at a group sequential analysis.

Let group 1 be the treatment group and group 0 the control group. Suppose \(n_1\) and \(n_0\) independent participants have observed binary outcomes in the two groups. Write \(p_1\) and \(p_0\) for their respective event probabilities, and define the risk difference as \(\delta=p_1-p_0\). The estimated risk difference is \(\hat{p}_1-\hat{p}_0\).

The variance formula is exact under the stated independent Bernoulli model; the large-sample qualification concerns using a normal distribution to approximate the estimator’s sampling distribution. In practice, because the probabilities are unknown, the variance is estimated by replacing \(p_1\) and \(p_0\) with \(\hat{p}_1\) and \(\hat{p}_0\). For planning, specified or anticipated event probabilities are used instead.

Deriving the Variance and Information

Assume each participant has one binary outcome, outcomes are independent within and between groups, and each group has a common event probability throughout the period being analyzed. The sample size in each group is treated as fixed. These assumptions exclude, for example, unaccounted clustering, repeated outcomes from the same participant, and systematic changes in event risk over time.

Let \(X_{ij}\) equal 1 if participant \(j\) in group \(i\) has the event and 0 otherwise. Then \(X_{ij}\) is Bernoulli with variance \(p_i(1-p_i)\), and \(\hat{p}_i\) is the average of the \(n_i\) outcomes. Independence gives the variance of each sample proportion. Since the groups are independent, the covariance between their proportions is zero, so the variance of their difference is the sum of the two variances:

The corresponding standard error is the square root of this variance. Define \(v_i=p_i(1-p_i)\). Then the effective information \(I_{\mathrm{RD}}\) for the risk difference is

This reciprocal-variance definition is useful because, under the normal approximation, a more precise estimate has smaller variance and therefore greater information. Information here has units of inverse squared risk difference; it is not a count. A risk difference is a probability difference, often reported in percentage points.

The same result can be justified using Fisher information. For a single Bernoulli observation with probability \(p_i\), the Fisher information about \(p_i\) is \(1/[p_i(1-p_i)]\). Thus the information matrix for \((p_1,p_0)\), from independent groups, is diagonal with entries \(n_1/v_1\) and \(n_0/v_0\). The gradient of \(\delta=p_1-p_0\) with respect to \((p_1,p_0)\) is \((1,-1)\). Applying the large-sample variance formula for a smooth function of estimated parameters gives

Taking the reciprocal gives the effective information for this contrast. This derivation also clarifies why the two groups’ information cannot simply be added as if they estimated the same single probability: the target is a contrast, and uncertainty from both probabilities contributes.

How Event Probabilities Affect Precision

A Bernoulli outcome has variance \(p(1-p)\), which is largest at \(p=0.5\) and decreases as \(p\) approaches either 0 or 1. Consequently, a group with an event probability near 0.5 contributes more sampling variance to a risk-difference estimate, for a given sample size, than a group with a probability near an endpoint. The variance depends on both arm-specific rates, not just on the pooled event rate.

For equal group sizes \(n\), the risk-difference variance is \(\{p_1(1-p_1)+p_0(1-p_0)\}/n\). This makes it possible to compare scenarios with the same allocation and sample size. The comparison is about absolute precision on the risk-difference scale; it does not say that a rare outcome is always easier to study for every scientific or design objective.

The relationship is not simply “fewer events means more information” or “more events means more information.” The relevant quantities are \(p_i(1-p_i)\) in each group. Rates near 0.5 maximize those quantities, while rates near 0 or 1 reduce them. Also, for a fixed risk difference, changing one probability can constrain the possible value of the other; comparisons should therefore state both probabilities rather than describe only a generic event rate.

Allocation and Information per Participant

With a fixed total sample size \(N=n_1+n_0\), allocating more participants to one group reduces that group’s variance contribution but leaves fewer participants in the other group. To see the trade-off, let \(r=n_1/n_0\). Then \(n_1=Nr/(1+r)\) and \(n_0=N/(1+r)\), so

For a fixed \(N\) and known \(v_1,v_0\), minimizing this variance over positive allocation ratios gives \(r=\sqrt{v_1/v_0}\). This follows by differentiating the variance expression with respect to \(r\): its derivative is proportional to \(v_0-v_1/r^2\), which is zero at that ratio, and its second derivative is positive. Equal allocation is optimal when the two Bernoulli variances are equal, but need not be optimal when they differ. Practical considerations such as treatment availability, cost, and ethical constraints can justify a different allocation.

Using the Result in Sequential Studies

At a particular analysis, use the numbers observed by that look: \(n_1\), \(n_0\), and the arm-specific event probabilities, or their estimates. Under the assumptions above, the variance of the cumulative risk-difference estimate follows directly from those quantities. If the underlying event probabilities and allocation remain stable as participants accrue, additional independent observations contribute additional information. If event risks change over time, the simple calculation based on one probability per group may not represent the sampling variance well.

In planning, the calculation is a way to translate assumed event risks and allocation into expected precision. At analysis, replacing probabilities with sample proportions yields an estimated variance and information. These are not automatically the variance and information used by every hypothesis test. A test based on a null-constrained score statistic, for example, can use a different variance estimate. Keep the estimand, estimator, and variance convention aligned: this tutorial’s formula describes precision for the difference of two independent sample proportions using the arm-specific Bernoulli variances.

The risk difference also has a different interpretation from a relative effect: it reports an absolute change in event probability. A standard error for the risk difference is in probability units, so it can be translated directly to percentage points. Ratios of risks and odds use different effect scales and have their own information calculations; the risk-difference formula should not be transferred to those scales.

Diagnostics, Limitations, and a Common Pitfall

Before relying on this information calculation, check whether the data and design support its assumptions. Confirm that the outcome is binary and observed for the participants included, that participants contribute independent outcomes, and that the probability within each group is reasonably stable for the analysis being summarized. In sequential settings, inspect event rates by accrual period and group; substantial drift can signal that a single \(p_i\) is inadequate. If observations are clustered, paired, repeated, or incompletely observed, the independent Bernoulli variance does not generally apply without adjustment.

The normal approximation also needs care when event counts are very small or probabilities are close to 0 or 1. In such settings, the estimated variance can be unstable and the sampling distribution of the risk-difference estimate can be far from normal. The formula still states the variance under the independent Bernoulli model, but the adequacy of a normal approximation is a separate question. Diagnostics should include the observed success and failure counts in each group, not just the calculated standard error.

• Pitfall: using total sample size as information. Two studies with the same total enrollment can have different information because allocation and event probabilities differ. Calculate both variance contributions rather than treating each participant as contributing the same amount.
• Pitfall: pooling event rates for this variance. Substituting one pooled probability into both arms discards the arm-specific variance structure. For the unpooled difference in proportions, use \(p_1(1-p_1)/n_1+p_0(1-p_0)/n_0\), or its arm-specific plug-in estimate.
• Pitfall: interpreting lower variance as universally better. Lower variance means greater precision for the absolute risk difference under the model. It does not by itself determine the scientific importance of the effect or the suitability of a design.