Risk Calibration When Sampling Fractions Are Unknown

Why Absolute Risk Is Not Identified

A case-control study deliberately samples people according to their outcome: cases are included at one rate and controls at another. This design can preserve information about how covariates relate to the outcome, but the observed fraction of cases is chosen by the sampling plan. It is therefore not, by itself, the population prevalence. When the sampling fractions are unknown, the case-control data do not determine the population logistic intercept or absolute risks.

Suppose \(Y\) is a binary outcome, \(X\) is a covariate vector, and the source-population risk follows a logistic model. Write \(\operatorname{expit}(u)=1/(1+\exp(-u))\). If selection depends on outcome but not on \(X\) within outcome groups, the sampled logistic model has the same slopes but a shifted intercept.

The intercept shift follows directly from Bayes' rule. For any covariate value \(x\) with positive probability of selection,

Taking logarithms gives the stated shift. If \(s_1/s_0\) is unknown, a fitted sampled intercept cannot be translated into the source-population intercept. The sampled case proportion does not resolve the problem: it reflects the design as well as the source population.

Calibrating With Prevalence and a Covariate Distribution

An external estimate of population prevalence supplies the marginal constraint needed to locate the intercept. Let \(\pi=P(Y=1)\) be the source-population prevalence and let \(F_X\) denote the source-population distribution of \(X\). For a given slope vector \(\beta\), an intercept \(\alpha\) must satisfy

The left side is the average model-predicted risk over the target population's covariate distribution. If \(F_X\) is available from a representative population sample, it can be used directly. In case-control data, it can instead be reconstructed from the covariate distributions among cases and controls, provided sampling within each outcome group is representative.

Proof. Define \(g(\alpha)=\int\operatorname{expit}(\alpha+\beta^\mathsf{T}x)\,dF_X(x)\). For every finite \(x\), the integrand is continuous and strictly increasing in \(\alpha\), and it lies between zero and one. Bounded convergence therefore implies that \(g\) is continuous. If \(\alpha_1<\alpha_2\), the integrand at \(\alpha_1\) is strictly smaller than at \(\alpha_2\) for every \(x\), so integrating gives \(g(\alpha_1)<g(\alpha_2)\). By bounded convergence again, \(g(\alpha)\to0\) as \(\alpha\to-\infty\) and \(g(\alpha)\to1\) as \(\alpha\to+\infty\). Continuity and the intermediate value theorem give a solution for each \(\pi\in(0,1)\), and strict increase makes it unique. \(\square\)

When only outcome-specific covariate distributions are available, write \(F_1\) for the distribution of \(X\) among source-population cases and \(F_0\) for its distribution among source-population controls. The law of total probability gives

This identity is useful because case-control sampling can estimate \(F_1\) and \(F_0\) separately even though it distorts the overall case proportion. The mixture uses the external prevalence, not the proportion of cases enrolled in the study.

The equation has a unique finite solution under the same conditions as the theorem. With empirical distributions from \(n_1\) cases and \(n_0\) controls, it becomes

The left side is continuous and strictly increasing in \(\alpha\), with limits zero and one. A one-dimensional root-finding method, such as bisection, can therefore find the calibrated intercept. Predictions \(\operatorname{expit}(\hat\alpha+\hat\beta^\mathsf{T}x)\) are then calibrated to the supplied prevalence over the specified target covariate distribution.

Worked Calibration Examples

Assumptions, Interpretation, and Common Pitfalls

This calibration is meaningful only when its inputs refer to the same target population and period. The external prevalence must describe the population whose risks are wanted. The case and control samples must represent the covariate distributions within their respective outcome groups. The fitted slopes must be appropriate for the source population, as under a correctly specified logistic model with sampling independent of covariates conditional on outcome. If selection also depends on \(X\) within outcome groups, the simple intercept-shift argument and the mixture reconstruction need not hold.

The calibration equation enforces one marginal constraint; it does not establish that the logistic model is correctly specified or that subgroup risks are accurate. It also does not make prevalence alone sufficient when the target distribution of \(X\) is unknown. In the case-control version above, the outcome-specific covariate distributions provide the information needed to reconstruct that target distribution under the stated sampling assumptions. Uncertainty in the external prevalence, slopes, and covariate distributions should be recognized when drawing inferential conclusions.

• Mistake: treating the sampled case fraction as prevalence. The wrong reasoning is to use \(n_1/(n_1+n_0)\) as \(\pi\). That fraction is determined partly by the sampling plan. Use a prevalence estimate for the target population.
• Mistake: changing only the risks without stating the target covariate distribution. Calibration averages predictions over \(F_X\). If that distribution is unknown, prevalence by itself does not specify which average-risk constraint to apply.
• Mistake: assuming the external prevalence changes the slopes. Under the stated outcome-dependent sampling model, calibration reconstructs the intercept while retaining the fitted source-population slopes. It does not repair slope bias caused by model misspecification or covariate-dependent selection.