Common Errors In Case-Control Analysis

Why Case-Control Analysis Invites Particular Errors

Case-control sampling deliberately selects people according to their outcome: cases are sampled from those with the outcome, and controls from those without it. This can make comparisons between cases and controls informative, but it also means that the sample is not generally a miniature version of the population. The central discipline is to ask, for each quantity, which population and which sampling mechanism does it describe?

Let \(Y=1\) denote the outcome and \(Y=0\) its absence, let \(X\) denote covariates, and let \(S=1\) indicate inclusion in the analyzed sample. Write \(p(x)=P(Y=1\mid X=x)\) for population risk and \(q(x)=P(Y=1\mid X=x,S=1)\) for the outcome probability among sampled people. These are different probabilities unless the sampling mechanism happens to make them equal.

Error 1: Treating the Sample Case Fraction as Prevalence

In many case-control studies, investigators choose how many cases and controls to recruit. If they recruit the same number of each, the case fraction is one half by design—even if the outcome occurs in only one person in a hundred in the population. The sample case fraction can be useful for describing the recruited data, but it does not reveal how common the outcome is in the target population.

The difference can be expressed using outcome-specific sampling probabilities. Suppose a person with \(Y=y\) is sampled with probability \(s_y=P(S=1\mid Y=y)\), and selection within each outcome group does not depend on other variables. The sample case fraction depends on both the population prevalence and these two sampling probabilities.

To verify the formula, the joint probability of being a sampled case is \(P(Y=1,S=1)=\pi s_1\). The probability of being sampled is the sum of the sampled-case and sampled-control probabilities: \(P(S=1)=\pi s_1+(1-\pi)s_0\). Dividing the first probability by the second gives the stated conditional probability.

Error 2: Reading Sample Probabilities as Population Risks

A logistic model fitted to an ordinary case-control sample can produce probabilities for membership in that sample, or fitted conditional odds under the sampling scheme. Those probabilities are not automatically population risks. Under outcome-only sampling, the odds identity below shows why: sampling multiplies the population odds by the ratio of the case and control sampling probabilities.

Proof. By the definition of conditional probability, \(q(x)=P(Y=1,S=1\mid X=x)/P(S=1\mid X=x)\). For the odds, it is convenient to divide the sampled-case probability by the sampled-control probability:

The second equality uses the definition of \(s_y(x)\); the last uses \(P(Y=0\mid X=x)=1-p(x)\). Positivity makes the odds ratios well-defined. \(\square\)

If selection depends on outcome alone, then \(s_1(x)=s_1\) and \(s_0(x)=s_0\) are constant across covariate values. Taking logarithms shows that selection adds the constant \(\log(s_1/s_0)\) to the population log odds. In a correctly specified logistic model this shifts the intercept, while leaving the covariate slopes unchanged. That fact does not make the sample intercept or its uncorrected fitted probabilities population risks.

Error 3: Ignoring Covariate-Dependent Selection

The slope-preservation result depends on the ratio \(s_1(x)/s_0(x)\) being constant in \(x\). If control selection, case selection, or both vary with covariates, the added term in the selected log odds can itself depend on \(x\). A fitted association may then reflect the sampling mechanism as well as the population relationship.

This follows by taking logarithms of the proved odds identity. In particular, if the population risk does not vary with \(x\), but \(s_1(x)/s_0(x)\) does, then the sampled outcome probability can vary with \(x\). Conversely, if \(s_1(x)=s_0(x)\) for every \(x\), the ratio is one and selection does not change the conditional outcome odds given \(X\), even though the distribution of \(X\) in the sample may differ from its population distribution.

Error 4: Applying the Rare-Disease Approximation Automatically

For a binary exposure, the population odds ratio and risk ratio are related, but they are not generally equal. If \(p_0\) and \(p_1\) are the risks in the unexposed and exposed groups, respectively, then \(\mathrm{OR}=\{p_1/(1-p_1)\}/\{p_0/(1-p_0)\}\) and \(\mathrm{RR}=p_1/p_0\). Algebra gives the exact relationship \(\mathrm{OR}=\mathrm{RR}(1-p_0)/(1-p_1)\). When both risks are small, the factor \((1-p_0)/(1-p_1)\) is close to one, so the odds ratio can approximate the risk ratio. When risks are not small, the approximation may be poor. A case-control design does not make this approximation true; it concerns the sizes of the population risks.

A Practical Diagnostic Checklist

Before interpreting a case-control analysis, state the target population and outcome period, then check what the sampling design permits you to estimate. Earlier material on intercept correction, covariate-dependent selection, and population risk prediction develops the corresponding methods; the key here is not to confuse a design-dependent quantity with its population counterpart.

Common Mistakes

• “Half the sample are cases, so prevalence is 50%.” That conclusion ignores outcome-dependent recruitment. The sample fraction equals population prevalence only under particular sampling conditions.
• “The fitted probability is the patient's risk.” A sample probability can incorporate the case-to-control sampling ratio. Recovering population risk requires appropriate calibration information and assumptions.
• “Case-control sampling preserves every logistic slope.” It preserves the population slopes under outcome-only sampling with a constant sampling ratio, not under arbitrary covariate-dependent selection.
• “The rare-disease approximation applies because this is a case-control study.” The approximation depends on population risks being small in both groups; the design does not guarantee that.