Testing Equality of Risks With a Relative Risk

Why Test a Relative Risk?

A risk is the probability that an event occurs in a specified population over a defined period. Comparing the risk in two groups can be expressed as a difference or as a ratio. The ratio, called the relative risk, describes how many times as large the risk is in one group as in the other. Equality of the two risks is therefore represented by a relative risk of one.

For example, a relative risk of 1.5 means the risk in group 1 is estimated to be 1.5 times the risk in group 0. A relative risk below one means the estimated risk is lower in group 1. To test whether the population risks differ, we can test whether the population relative risk departs from one. The test developed here uses the logarithm of the estimated relative risk, which has a more nearly normal sampling distribution when sample sizes and event counts are sufficiently large.

Definition: Let \(p_1\) and \(p_0\) be the event probabilities in populations 1 and 0. The population relative risk is \(RR=p_1/p_0\), provided \(p_0>0\). For observed event counts \(a\) out of \(n_1\) in group 1 and \(c\) out of \(n_0\) in group 0, the sample risks are \(\hat{p}_1=a/n_1\) and \(\hat{p}_0=c/n_0\). When \(c>0\), the estimated relative risk is \(\widehat{RR}=\hat{p}_1/\hat{p}_0\). The log-relative-risk test also requires \(a>0\), so that \(\log(\widehat{RR})\) is defined.

Hypotheses and the Test Statistic

The equality question is expressed as \(H_0:RR=1\), which is equivalent to \(H_0:p_1=p_0\), assuming the risks are defined. A two-sided alternative is \(H_a:RR\ne1\). A directional alternative can be \(H_a:RR>1\) or \(H_a:RR<1\); choose it before examining the results if the scientific question is directional.

Under the independent Bernoulli model for two groups, observations within each group are independent Bernoulli outcomes with a common group-specific event probability, and the two groups are independent. Under these conditions, the sample risks estimate their respective population risks. For sufficiently large samples and nonzero event counts, the logarithm of the estimated relative risk is approximately normal. Its estimated standard error is

SE\{\log(\widehat{RR})\} = \sqrt{\frac{1}{a}-\frac{1}{n_1}+\frac{1}{c}-\frac{1}{n_0}}.

The large-sample test statistic is

z= \frac{\log(\widehat{RR})-\log(1)} {SE\{\log(\widehat{RR})\}} = \frac{\log(\widehat{RR})} {\sqrt{1/a-1/n_1+1/c-1/n_0}}.

When \(H_0\) is true and the large-sample conditions are adequate, this statistic has an approximate standard normal distribution. For a two-sided test, the p-value is \(2P(Z\ge |z|)\), where \(Z\) has a standard normal distribution. For \(H_a:RR>1\), it is \(P(Z\ge z)\); for \(H_a:RR<1\), it is \(P(Z\le z)\).

Why the Standard Error Has This Form

A short derivation explains both why the test uses logarithms and why its standard error involves the event counts. For a sample proportion \(\hat{p}\) from \(n\) independent Bernoulli observations, the variance is \(p(1-p)/n\). For a proportion near a positive value \(p\), a first-order Taylor approximation gives

\log(\hat{p})\approx \log(p)+\frac{\hat{p}-p}{p}.

The approximate variance of the right-hand side is the variance of \(\hat{p}\) divided by \(p^2\):

\operatorname{Var}\{\log(\hat{p})\} \approx \frac{p(1-p)/n}{p^2} = \frac{1-p}{np}.

For independent groups, \(\log(\widehat{RR})=\log(\hat{p}_1)-\log(\hat{p}_0)\). The variance of a difference of independent quantities is the sum of their variances. Substituting the sample risks for the population risks therefore estimates the variance as

\frac{1-\hat{p}_1}{n_1\hat{p}_1} + \frac{1-\hat{p}_0}{n_0\hat{p}_0} = \left(\frac{1}{a}-\frac{1}{n_1}\right) + \left(\frac{1}{c}-\frac{1}{n_0}\right).

Taking the square root gives the stated standard error. The approximation relies on the sample risks being sufficiently well behaved for the logarithm to be close to normal; it is not reliable when the counts are sparse or a group has no observed events.

Conditions Before Calculating

The formula is a large-sample approximation, not a guarantee that every set of counts can be analyzed this way. Check that the two groups are independent, that outcomes within each group reasonably follow the independent Bernoulli model, and that the samples represent the populations to which the conclusion will refer. In a randomized experiment, random assignment supports comparisons between assigned groups; in an observational study, differences between groups may also reflect other factors.

The log method requires \(a>0\) and \(c>0\). It is also important that event and non-event counts are not too small. A common practical rule of thumb is to look for at least about 10 events and 10 non-events in each group; this is a guide to the normal approximation, not a universal cutoff. If counts are sparse, the calculated approximation may be unreliable, and this procedure should not be used without an appropriate alternative.

Worked Example: Evidence That Risk Is Higher in Group 1

In a hypothetical comparison, 40 of 200 people in group 1 experience an event, while 20 of 200 in group 0 experience it. Test \(H_0:RR=1\) against \(H_a:RR\ne1\) at \(\alpha=0.05\), and report an uncertainty interval.

Check the conditions. Assume the observations are independent within and between the two groups, and that the sampling design supports inference to the populations of interest. The event counts are 40 and 20; the non-event counts are 160 and 180. All are at least 10, so the large-sample approximation is reasonable.

Estimate the relative risk. The sample risks are \(\hat{p}_1=40/200=0.200\) and \(\hat{p}_0=20/200=0.100\). Thus \(\widehat{RR}=0.200/0.100=2.00\): the observed risk in group 1 is twice the observed risk in group 0.

Calculate the test statistic.

SE= \sqrt{\frac{1}{40}-\frac{1}{200}+\frac{1}{20}-\frac{1}{200}} = \sqrt{0.06500} = 0.2550, \qquad z=\frac{\log(2.00)}{0.2550} = \frac{0.6931}{0.2550} = 2.719.

The two-sided p-value is approximately \(2P(Z\ge2.719)=0.0066\). Because \(0.0066<0.05\), the data provide evidence against equal population risks in favor of a difference. This does not prove that the risks differ, nor does it establish a cause for any difference.

Quantify uncertainty. A 95% interval on the log scale is \(\log(\widehat{RR})\pm1.96(SE)\):

0.6931\pm1.96(0.2550) = [0.1934,\ 1.1929].

Exponentiating the endpoints gives a 95% confidence interval for the relative risk of \([1.213,\ 3.296]\). In context, the data are compatible with a population risk in group 1 roughly 1.21 to 3.30 times the risk in group 0, under the model and sampling assumptions. The interval excludes one, consistent with the two-sided test at the 5% level.

Worked Tests and What Their P-Values Say

The next example illustrates that a point estimate above one is not necessarily strong evidence that the population relative risk exceeds one. The standard error and the resulting sampling uncertainty matter as much as the estimate itself.

Worked Example: A Relative Risk Near One

In a hypothetical study, 12 of 100 individuals in group 1 and 10 of 100 in group 0 experience the event. Test equality against a two-sided alternative at \(\alpha=0.05\), and calculate a 95% interval.

Assume the groups are independent, the Bernoulli model is appropriate, and sampling supports the intended population inference. The event counts are 12 and 10, and the non-event counts are 88 and 90, so the usual count rule of thumb is met.

The estimated risks are \(0.120\) and \(0.100\), giving \(\widehat{RR}=1.20\). Its log-scale standard error and test statistic are

SE= \sqrt{\frac{1}{12}-\frac{1}{100}+\frac{1}{10}-\frac{1}{100}} = \sqrt{0.1633} = 0.4041, \qquad z=\frac{\log(1.20)}{0.4041} = \frac{0.1823}{0.4041} = 0.451.

The two-sided p-value is approximately \(2P(Z\ge0.451)=0.652\). The data do not provide strong evidence against equality at the 5% level. This is not evidence that the population risks are exactly equal: the estimate is quite imprecise.

For a 95% interval, the log-scale endpoints are \(0.1823\pm1.96(0.4041)=[-0.6098,\ 0.9744]\). Exponentiating gives a relative-risk interval of approximately \([0.543,\ 2.650]\). It includes one and also allows effects in either direction, illustrating the substantial uncertainty in this estimate.

For a one-sided test, the direction of the alternative changes which tail of the normal distribution supplies the p-value. It does not change the observed statistic or the estimated relative risk. The direction must reflect the question of interest, not be selected after seeing which result looks more favorable.

Worked Example: Testing Whether Risk Is Lower in Group 1

In a hypothetical comparison, 18 of 300 people in group 1 and 30 of 300 in group 0 experience an event. Test \(H_0:RR=1\) against \(H_a:RR<1\) at \(\alpha=0.05\). Also calculate a 95% two-sided interval.

Assume independent Bernoulli observations and an appropriate sampling design. There are 18 and 30 events and 282 and 270 non-events. Since all four counts exceed 10, the large-sample calculation is reasonable.

The estimated risks are \(18/300=0.0600\) and \(30/300=0.100\), so \(\widehat{RR}=0.600\). The test calculation is

SE= \sqrt{\frac{1}{18}-\frac{1}{300}+\frac{1}{30}-\frac{1}{300}} = \sqrt{0.08222} = 0.2867, \qquad z=\frac{\log(0.600)}{0.2867} = \frac{-0.5108}{0.2867} = -1.782.

For the lower-tail alternative, the p-value is \(P(Z\le-1.782)\approx0.0374\). At the 5% level, this is evidence that the risk in group 1 is lower than the risk in group 0, under the stated model and design assumptions.

The 95% two-sided interval on the log scale is \(-0.5108\pm1.96(0.2867)=[-1.073,\ 0.051]\). Exponentiating gives approximately \([0.342,\ 1.053]\). This interval includes one. That does not contradict the one-sided test: a one-sided test at 5% corresponds to a 90%, not a 95%, two-sided interval. A 90% interval here is about \([0.374,\ 0.962]\), which lies below one.

Interpreting the Result Carefully

A p-value is the probability, assuming \(H_0\) and the model assumptions are true, of obtaining a test statistic at least as extreme in the direction or directions specified by \(H_a\) as the statistic observed. It is not the probability that the null hypothesis is true, and it does not measure the size or practical importance of an effect. Interpret it alongside the estimated relative risk, its interval, and the context of the study.

A confidence interval on the relative-risk scale expresses uncertainty about the population relative risk. Its long-run interpretation is that intervals built by this procedure would contain the true relative risk at the stated rate across repeated sampling under the assumptions. It is not correct to say there is a 95% probability that a fixed population parameter lies in a particular 95% interval.

The ratio itself also needs context. A relative risk describes a multiplicative comparison, not the absolute difference in event probabilities. For example, a relative risk of two could represent an increase from 1% to 2% or from 30% to 60%. The same ratio can therefore accompany very different absolute changes. Moreover, a relative risk comparison alone does not establish that group membership caused the difference.

Common Mistakes

Reversing the comparison. \(\widehat{RR}=\hat{p}_1/\hat{p}_0\) compares group 1 with group 0. Reversing the numerator and denominator changes the ratio and its interpretation.
Taking a logarithm when an event count is zero. If \(a=0\) or \(c=0\), the log-relative-risk calculation shown here is undefined. Do not silently apply this test or add an arbitrary correction and report the result as though it followed from this method.
Using the ratio as proof of a difference. An estimate such as 1.20 can have a wide interval and a large p-value. Report the uncertainty rather than treating the point estimate as definitive.
Misreading a p-value. A p-value is calculated under the null model; it is not the probability that the null is true.
Turning association into causation. A difference in observed risks does not by itself show that group membership caused the difference. The study design and other possible explanations matter.

Key takeaway: Test equality of two positive population risks with \(H_0:RR=1\). When independent-group and large-sample conditions are reasonable, calculate the test on the log-relative-risk scale, interpret the p-value under the null model, and report an interval to show the uncertainty in the estimated ratio.

Check Your Understanding

Try these questions without looking back.

Why is equality of the population risks equivalent to a relative risk of one?
For \(a=24,\ n_1=120,\ c=12,\ n_0=120\), calculate the estimated relative risk. Is the log-relative-risk test defined from these counts?
What assumptions and count conditions should be checked before using the large-sample test?
In words, what does a two-sided p-value of 0.03 mean when testing \(H_0:RR=1\)?
Why does a 95% confidence interval that includes one indicate that the two-sided test at the 5% level would not reject the null, when both use the same large-sample method?
Explain why a relative risk of two does not, by itself, describe the absolute change in risk or establish a causal effect.