Prior-Probability Shift and Decision Adjustment

Why Prevalence Changes Affect Classification

A discriminant model can describe how features differ across classes, yet its predicted probabilities also depend on how common each class is. If a model is developed in one population and deployed in another, the classes may have different prevalences even when the distribution of features within each class is stable. In that situation, the original posterior probabilities need adjustment: the evidence from the features is retained, while the prior odds are changed.

This setting is called prior-probability shift (also called label shift). It differs from a change in the class-conditional feature distributions: the assumption is specifically that the distribution of features given the class remains the same. The adjustment is useful for interpreting Gaussian discriminant probabilities and for making decisions using target-population prevalences.

Definition: Let $Y$ take values in $\{1,\ldots,K\}$, and let $X$ be the feature vector. Write $\pi_k^{(s)}=P_s(Y=k)$ for the source prevalence and $\pi_k^{(t)}=P_t(Y=k)$ for the target prevalence. Prior-probability shift assumes $P_t(X\mid Y=k)=P_s(X\mid Y=k)$ for every class $k$, while the prevalences may differ. The source prevalence must be positive for each class whose source posterior is to be adjusted; its target prevalence may be zero, in which case the adjusted posterior for that class is zero wherever the target posterior is defined. Positive target prevalences are required for log adjustments and odds identities.

In Gaussian discriminant analysis, the shared class-conditional distribution can be $X\mid Y=k\sim N_p(\mu_k,\Sigma_k)$, with positive-definite $\Sigma_k$. The covariance matrices may be common across classes, as in linear discriminant analysis, or class-specific, as in quadratic discriminant analysis. The correction below does not require either covariance structure in particular; it requires the class-conditional distributions to remain stable between source and target.

Posterior Correction

Let $q_k(x)=P_s(Y=k\mid X=x)$ denote the source posterior, and let $p_k^{(t)}(x)=P_t(Y=k\mid X=x)$ denote the target posterior. Bayes’ rule expresses each posterior as a class-conditional density multiplied by its prevalence, then normalized across classes. Because the class-conditional densities are unchanged, their common factors cancel when source and target posterior odds are compared.

Theorem: Under prior-probability shift, for every feature value $x$ where the relevant densities are defined, $$ p_k^{(t)}(x) = \frac{q_k(x)\,\pi_k^{(t)}/\pi_k^{(s)}} {\displaystyle\sum_{j=1}^{K}q_j(x)\,\pi_j^{(t)}/\pi_j^{(s)}}. $$ For any two classes $k$ and $l$ with positive source and target prevalences and $q_l(x)>0$, $$ \frac{p_k^{(t)}(x)}{p_l^{(t)}(x)} = \frac{q_k(x)}{q_l(x)} \frac{\pi_k^{(t)}/\pi_l^{(t)}}{\pi_k^{(s)}/\pi_l^{(s)}}. $$

Proof. Let $f_k(x)$ be the common class-conditional density for class $k$. Bayes’ rule gives $q_k(x)=\pi_k^{(s)}f_k(x)/\sum_j\pi_j^{(s)}f_j(x)$. Therefore, $f_k(x)=q_k(x)\sum_j\pi_j^{(s)}f_j(x)/\pi_k^{(s)}$. The target posterior is $p_k^{(t)}(x)=\pi_k^{(t)}f_k(x)/\sum_j\pi_j^{(t)}f_j(x)$. Substitute the expression for $f_k(x)$ in numerator and denominator; the common source normalizing factor cancels, leaving the displayed formula. Taking the ratio for classes $k$ and $l$ cancels the target posterior’s common denominator and gives the odds identity. $\square$

The formula has a direct interpretation: multiply each source posterior by the relative prevalence change for its class, then renormalize so the adjusted probabilities sum to one. For Gaussian discriminant analysis, this is also an intercept adjustment to the class scores. If the source score for class $k$ is $\delta_k^{(s)}(x)$, with posterior proportional to $\exp\{\delta_k^{(s)}(x)\}$, a target score is $\delta_k^{(t)}(x)=\delta_k^{(s)}(x)+\log\{\pi_k^{(t)}/\pi_k^{(s)}\}$, up to a common additive constant. The feature-dependent Gaussian terms do not change.

Worked Example: A Binary Gaussian Discriminant

Suppose the source model has $X\mid Y=0\sim N(0,1)$ and $X\mid Y=1\sim N(2,1)$, with source prevalences $\pi_0^{(s)}=\pi_1^{(s)}=0.5$. In the target population, let $\pi_0^{(t)}=0.1$ and $\pi_1^{(t)}=0.9$. Find the target posterior probability of class 1 at $x=1.5$, and compare the source and target classification boundaries under equal error costs.

For normal distributions with variance 1, the log density ratio is $\log\{f_1(x)/f_0(x)\}=-(x-2)^2/2+x^2/2=2x-2$. At $x=1.5$, this is $1$, so the likelihood ratio is $e$. Since the source prevalences are equal, the source posterior odds are $e$, and $q_1(1.5)=e/(1+e)$.

The target-to-source prevalence ratio for class 1 is $0.9/0.5=1.8$, and for class 0 it is $0.1/0.5=0.2$. Thus the target odds are the source odds multiplied by $1.8/0.2=9$: they equal $9e$. Hence $p_1^{(t)}(1.5)=9e/(1+9e)\approx 0.9607$.

Under equal error costs, classify as class 1 when the posterior odds exceed 1. In the source population this means $2x-2>0$, or $x>1$. In the target population it means $2x-2+\log 9>0$, or $x>1-(\log 9)/2\approx -0.0986$. The feature evidence is unchanged, but the much higher target prevalence of class 1 lowers the decision boundary.

Multiclass Adjustment in Practice

For $K$ classes, the adjustment applies to every class separately. It is often convenient to compute on the log scale: add $\log\{\pi_k^{(t)}/\pi_k^{(s)}\}$ to each source log score, then normalize the scores to obtain probabilities. This avoids multiplying very small probabilities and makes the effect of a prevalence change visible as a class-specific score shift.

Worked Example: Correcting Three Gaussian Posterior Probabilities

At a feature value $x=0$, suppose a three-class Gaussian discriminant gives source posteriors $q(x)=(0.2,0.3,0.5)$. Consider source prevalences proportional to $(e^{0.8},e^{-0.1},1)$, and target prevalences $(1/3,1/3,1/3)$. These source prevalences are valid after dividing each component by their sum. The target-to-source prevalence ratios are a common positive constant times $(e^{-0.8},e^{0.1},1)$; the common constant cancels when probabilities are normalized.

The unnormalized adjusted probabilities are therefore $0.2e^{-0.8}$, $0.3e^{0.1}$, and $0.5$. Numerically, these are approximately $0.0898658$, $0.3315511$, and $0.5000000$. Their sum is $0.9214168$. Dividing each by this sum gives $p^{(t)}(x)\approx(0.0975,0.3598,0.5426)$. The probabilities sum to 1 up to rounding.

The Gaussian specification can be made explicit. Take all three class-conditional distributions to be $N(\mu_k,1)$, and define $\mu_k=\sqrt{2\log\{\pi_k^{(s)}/(0.1q_k(x))\}}$, where $\pi_k^{(s)}$ is the normalized source prevalence just specified. Each logarithm is positive. At $x=0$, the density for class $k$ is proportional to $e^{-\mu_k^2/2}$, so $\pi_k^{(s)}e^{-\mu_k^2/2}=0.1q_k(x)$. Normalization therefore yields exactly the stated source posterior vector. This example shows how an already computed Gaussian posterior is adjusted without changing the class-conditional model.

Decision Rules and Target Risk

A posterior probability is not itself a decision: the action depends on the losses attached to decisions. For two classes, suppose action 1 incurs loss $C_{10}$ when the true class is 0, and action 0 incurs loss $C_{01}$ when the true class is 1; correct decisions have zero loss. Given target posterior $p=P_t(Y=1\mid X=x)$, the conditional expected losses are $C_{10}(1-p)$ for action 1 and $C_{01}p$ for action 0. Therefore, for positive costs, action 1 is optimal when $p\ge C_{10}/(C_{10}+C_{01})$; it is uniquely optimal when $p>C_{10}/(C_{10}+C_{01})$, and at equality both actions are optimal. Replacing $p$ by the uncorrected source posterior can select a different action.

Theorem: For a fixed classifier $g$ with binary actions under prior-probability shift, its target expected loss is $$ R_t(g) = \pi_0^{(t)}C_{10}P_s\{g(X)=1\mid Y=0\} + \pi_1^{(t)}C_{01}P_s\{g(X)=0\mid Y=1\}. $$ This requires the stated losses, finite conditional error probabilities, and stable class-conditional feature distributions.

Proof. By conditioning on the target class, expected loss is the sum of each class’s target prevalence times its conditional expected loss. For class 0, the loss is $C_{10}$ precisely when $g(X)=1$, so its conditional expected loss is $C_{10}P_t\{g(X)=1\mid Y=0\}$. For class 1, it is $C_{01}P_t\{g(X)=0\mid Y=1\}$. Prior-probability shift equates each target conditional probability with its source counterpart. Substitution gives the formula. $\square$

Worked Example: Target Cost and Risk for a Gaussian Rule

Use the Gaussian class-conditionals from the binary example: $X\mid Y=0\sim N(0,1)$ and $X\mid Y=1\sim N(2,1)$. Let target prevalences be $(0.1,0.9)$, let $C_{10}=4$ and $C_{01}=1$, and assign zero loss to correct decisions. Find the target cost-sensitive boundary and its target risk.

The posterior threshold is $C_{10}/(C_{10}+C_{01})=4/5$, so action 1 is chosen when target posterior odds exceed $4$. Target prior odds are $0.9/0.1=9$, and the Gaussian likelihood ratio is $\exp(2x-2)$. Thus the decision condition is $9\exp(2x-2)>4$, or $x>(2+\log(4/9))/2\approx0.5945$.

For this boundary $c$, the class-0 false-positive probability is $P(X>c\mid Y=0)=1-\Phi(c)\approx1-\Phi(0.5945)\approx0.2761$. The class-1 false-negative probability is $P(X\leq c\mid Y=1)=\Phi(c-2)=\Phi(-1.4055)\approx0.0799$. The target expected loss is therefore $0.1(4)(0.2761)+0.9(1)(0.0799)\approx0.1824$.

For comparison, retaining the source-prevalence odds but using the same costs would require the likelihood ratio to exceed 4, giving the boundary $x>(2+\log 4)/2\approx1.6931$. Its conditional error probabilities are approximately $0.0452$ and $0.3795$, so its target expected loss is $0.1(4)(0.0452)+0.9(1)(0.3795)\approx0.3596$. The comparison demonstrates why a decision rule must be assessed using the target prevalences and the intended costs.

Assumptions, Interpretation, and a Common Pitfall

The correction is valid only when class-conditional feature distributions are stable and the source prevalence for each adjusted class is positive. If a class has zero source prevalence, source data provide no posterior evidence for that class; division by its source prevalence is undefined, and the formula cannot recover its target posterior. In practice, prevalences and Gaussian parameters may be estimated, so the corrected probabilities inherit uncertainty from those estimates. The formulas describe the population adjustment when their inputs are known.

A common mistake is to treat source posterior probabilities as if they were just feature evidence and to use them unchanged after deployment. For example, if class 1 becomes nine times as prevalent relative to class 0 as it was in the source population, the posterior odds must also be multiplied by nine under prior shift. The correction does not claim that the features themselves became more indicative of class 1; it accounts for the changed prior odds. Conversely, changing prevalences alone does not justify this correction if the distribution of features within a class has also changed.

Key takeaway: Under prior-probability shift, retain the Gaussian class-conditional model, add the log prevalence-ratio adjustment to each class score, and normalize to get target posteriors. Choose decisions and evaluate expected loss using target prevalences and the specified costs.

Check Your Understanding

Try these questions using the definitions and formulas above.

State the assumption that distinguishes prior-probability shift from a change in class-conditional feature distributions.
For two classes, explain how the posterior odds change when the target-to-source prevalence ratio for class 1 is four times the corresponding ratio for class 0.
Why can a class with zero source prevalence not be handled by the posterior correction formula?
In the binary Gaussian example with equal error costs, how does increasing the target prevalence of class 1 affect the decision boundary?
What two conditional error probabilities and target prevalence values are needed to calculate the binary target risk with asymmetric costs?