Population Risk Prediction From Case-Control Fits

From a Case-Control Fit to a Population Risk

A logistic model fitted to a case-control sample can describe how the odds of disease vary with covariates while still giving incorrect absolute risks. The reason is that case-control sampling deliberately changes the proportion of cases in the data. A fitted probability therefore refers to the sampled data unless the sampling design is accounted for. Population risk prediction requires restoring the source-population intercept, or otherwise calibrating predicted risks to the source population.

This distinction matters in clinical use. A model may rank patients well, yet systematically overstate or understate their probabilities of disease. Correcting the sampling-induced shift can repair absolute-risk predictions without changing their ordering, provided sampling depends on the outcome alone and the same correction is applied to every person.

The earlier results on case-control logistic regression establish that outcome-dependent sampling shifts the intercept by \(\log r\) while preserving the slopes. If the sampled-data linear predictor is \(\eta_s(x)\), then the source-population linear predictor is \(\eta_s(x)-\log r\). The following calculation shows why this yields a risk correction.

Proof. By the definition of conditional probability, \(P(Y=y\mid x,S=1)=P(S=1\mid Y=y,x)P(Y=y\mid x)/P(S=1\mid x)\). Taking the ratio for \(y=1\) and \(y=0\) cancels the common denominator \(P(S=1\mid x)\), giving \[ \frac{P(Y=1\mid x,S=1)}{P(Y=0\mid x,S=1)} = \frac{s_1P(Y=1\mid x)}{s_0P(Y=0\mid x)}. \] Thus sampled odds equal source odds multiplied by \(r\). Taking logarithms and rearranging gives source log-odds \(\operatorname{logit}(p_s(x))-\log r\). Applying the expit function gives the stated risk. \(\square\)

Using an External Population Prevalence

Sometimes the outcome-specific sampling fractions are not available, but a reliable source-population prevalence is known. In that situation, the model’s slope estimates can be retained while its intercept is adjusted so that average predicted risk matches the external prevalence. This needs covariate values representative of the source population, or appropriate information to account for how the case-control sample differs from that population. The prevalence alone does not determine individual risks if the covariate distribution needed for the average is unavailable.

Proof. Define \(g(a)=E[\operatorname{expit}(a+Z)]\). The expit function is bounded between zero and one, so the expectation exists. If \(a_m\to a\), then \(\operatorname{expit}(a_m+Z)\to\operatorname{expit}(a+Z)\) almost surely and the terms are bounded by 1. The dominated convergence theorem therefore gives \(g(a_m)\to g(a)\), so \(g\) is continuous. If \(a_2>a_1\), then \(\operatorname{expit}(a_2+Z)>\operatorname{expit}(a_1+Z)\) almost surely because \(Z\) is finite and expit is strictly increasing. Hence \(g(a_2)>g(a_1)\).

As \(a\to-\infty\), \(\operatorname{expit}(a+Z)\to0\) almost surely; as \(a\to+\infty\), it tends to 1 almost surely. Dominated convergence, again using the bound 1, gives \(g(a)\to0\) and \(g(a)\to1\), respectively. Continuity and the intermediate value theorem imply that \(g(a)=\pi\) for some finite \(a\), since \(0<\pi<1\). Strict increase implies there is only one such value. \(\square\)

For a finite collection of source-representative covariate values \(x_i\) with weights \(w_i\) summing to one, the expectation in the calibration equation becomes \(\sum_i w_i\operatorname{expit}(a+x_i^\mathsf{T}\beta)\). This equation is generally solved numerically. The weights must represent the source population: treating the case-control sample’s covariate mix as the population mix can produce a wrong intercept even when the external prevalence is correct.

Calibration and Discrimination Are Different

Calibration concerns the agreement between predicted probabilities and event frequencies in the target population. In ideal calibration, among individuals assigned risk \(p\), the event probability is \(p\). A practical assessment uses source-population validation data, for example by comparing observed and predicted frequencies across risk groups or by estimating a calibration curve. If the validation sample is itself outcome-dependent, its observed event frequencies cannot be read as population frequencies without accounting for that sampling design.

Discrimination concerns whether people who experience the event tend to receive higher scores than people who do not. One measure is the area under the receiver operating characteristic curve (AUC), defined for a score \(T\) as \(P(T_1>T_0)+\tfrac12P(T_1=T_0)\), where \(T_1\) and \(T_0\) are independent scores for a randomly selected case and noncase from the target population. AUC evaluates pairwise ordering, not whether a score is a correct probability.

Proof. For any case score \(T_1\) and noncase score \(T_0\), strict increase gives \(T_1>T_0\) if and only if \(h(T_1)>h(T_0)\), and \(T_1=T_0\) if and only if \(h(T_1)=h(T_0)\). Thus the probabilities of a win and a tie in the AUC definition are each unchanged by the transformation. Their stated combination is therefore unchanged. \(\square\)

Common Pitfall and Practical Checks

A frequent error is to treat good discrimination as evidence that predicted risks are accurate. AUC can remain exactly the same after a large intercept correction, as the example shows. A high AUC does not establish that a predicted risk of 0.30 corresponds to a 30% event rate in the source population. Conversely, an intercept correction that improves average risk calibration need not improve how well the model ranks people.

Another error is to subtract \(\log r\) when \(r\) was defined in the opposite direction. Under the convention here, \(r=s_1/s_0\): oversampling cases means \(r>1\), which raises sampled odds, so \(\log r\) must be subtracted to recover source odds. Always check the direction with a simple odds calculation. In the first worked example, sampled odds of 4 become source odds of 1, not 0.25.

Finally, a single intercept shift is justified only when inclusion probabilities depend on outcome but not on covariates. If selection varies with \(X\) within outcome groups, the sampled-to-source odds ratio need not be a constant multiplier across \(x\). In that setting, use a method that represents the selection process rather than applying one universal intercept correction.