Balanced Accuracy and Class-Weighted Summaries

Why Balance the Classes?

Ordinary accuracy is the proportion of all predictions that are correct. It is useful when the observed mix of classes represents the population of interest and the consequences of errors are appropriately summarized by a single proportion. But if one class is much more common than the other, that class contributes more observations—and therefore more influence—to ordinary accuracy. A model can achieve high accuracy while performing poorly on the less common class.

Balanced accuracy addresses this issue by calculating the correct classification rate separately within each actual class, then taking the mean of those rates. In a binary classification problem, these class-specific rates are sensitivity and specificity. The calculation therefore gives each class equal influence, regardless of its frequency in the evaluation data.

Definition: For a binary classifier, let $TP$ and $FN$ be the numbers of true positives and false negatives, and let $TN$ and $FP$ be the numbers of true negatives and false positives. Provided both actual classes are present, sensitivity is $TP/(TP+FN)$, specificity is $TN/(TN+FP)$, and balanced accuracy is their arithmetic mean: $$ \text{Balanced accuracy} =\frac{1}{2}\left(\frac{TP}{TP+FN}+\frac{TN}{TN+FP}\right). $$

Sensitivity is the proportion of actual positive cases classified as positive. Specificity is the proportion of actual negative cases classified as negative. Balanced accuracy is not the proportion of all predictions that are correct; it averages two rates whose denominators are different.

How Class Weighting Produces Balanced Accuracy

Suppose an evaluation set contains $n_+=TP+FN$ actual positive cases and $n_-=TN+FP$ actual negative cases. Assign every positive case weight $1/(2n_+)$, and every negative case weight $1/(2n_-)$. The total weight assigned to the positive class is $n_+/(2n_+)=1/2$; the total weight assigned to the negative class is also $1/2$. Thus, each class contributes half of the total weight.

The weighted proportion of correct predictions is the sum of the weights of all correctly classified observations. The positive observations classified correctly contribute $TP/(2n_+)$, and the negative observations classified correctly contribute $TN/(2n_-)$. Adding these contributions gives balanced accuracy.

\frac{TP}{2n_+}+\frac{TN}{2n_-} =\frac{1}{2}\left(\frac{TP}{TP+FN}+\frac{TN}{TN+FP}\right).

This identity explains the weighting: balanced accuracy is ordinary accuracy after giving each actual class an equal total weight, rather than letting the more numerous class dominate. The weights are a way to define a summary, not a change to the model's predictions.

More generally, a class-standardized accuracy can use a chosen target proportion $q$ for the positive class and $1-q$ for the negative class:

\text{Weighted accuracy}(q) =q\,\text{sensitivity}+(1-q)\,\text{specificity}.

Here $q$ describes the class mix for the summary, not necessarily the observed prevalence. Setting $q=1/2$ gives balanced accuracy. Setting $q$ to the observed positive proportion gives ordinary accuracy. A different $q$ answers a different question: how well would the classifier perform, as a weighted correct-classification rate, under that specified class mix?

Worked Example: A Rare Positive Class

A hypothetical screening classifier is evaluated on 200 cases. Of these, 20 are actually positive and 180 are actually negative. It correctly identifies 14 positive cases and correctly classifies 162 negative cases. Calculate ordinary accuracy, sensitivity, specificity, and balanced accuracy. Then verify balanced accuracy using equal total class weights.

Method. First complete the error counts: $FN=20-14=6$, and $FP=180-162=18$. The confusion-matrix counts are therefore $TP=14$, $FN=6$, $TN=162$, and $FP=18$.

The number correctly classified is $TP+TN=14+162=176$, out of 200 cases. Ordinary accuracy is $176/200=0.88$, or 88%.

Sensitivity is $TP/(TP+FN)=14/20=0.70$. Specificity is $TN/(TN+FP)=162/180=0.90$. Balanced accuracy is $(0.70+0.90)/2=0.80$, or 80%.

For the class-weight check, each positive case receives weight $1/(2\times20)=0.025$, and each negative case receives weight $1/(2\times180)=1/360$. The total weight of correct positive predictions is $14(0.025)=0.35$. The total weight of correct negative predictions is $162/360=0.45$. Their sum is $0.35+0.45=0.80$, as required.

Check. Each class has total weight $0.50$: the positive class has $20(0.025)=0.50$, and the negative class has $180(1/360)=0.50$. The weighted correct total agrees with the direct balanced-accuracy calculation. The common mistake to avoid is reporting 88% as balanced accuracy; 88% is ordinary accuracy and is influenced by the much larger negative class.

Comparing Models With Different Class-Specific Performance

When two classifiers are evaluated on the same cases, ordinary accuracy and balanced accuracy can rank them differently. That difference is not a calculation error. The summaries assign different importance to the actual classes: ordinary accuracy weights each observation equally, while balanced accuracy gives the two classes equal total influence.

Worked Example: Accuracy and Balanced Accuracy Rank Models Differently

Two hypothetical classifiers are evaluated on 1,000 cases: 100 are actually positive and 900 are actually negative. Model A has $TP=80$, $FN=20$, $TN=810$, and $FP=90$. Model B has $TP=60$, $FN=40$, $TN=855$, and $FP=45$. Calculate both summaries for each model and state which model each summary ranks higher.

Method. Both models use the same class denominators: 100 positives and 900 negatives.

For Model A, ordinary accuracy is $(80+810)/1{,}000=890/1{,}000=0.890$. Its sensitivity is $80/100=0.800$, and its specificity is $810/900=0.900$. Its balanced accuracy is $(0.800+0.900)/2=0.850$.

For Model B, ordinary accuracy is $(60+855)/1{,}000=915/1{,}000=0.915$. Its sensitivity is $60/100=0.600$, and its specificity is $855/900=0.950$. Its balanced accuracy is $(0.600+0.950)/2=0.775$.

Ordinary accuracy ranks Model B higher, since $0.915>0.890$. Balanced accuracy ranks Model A higher, since $0.850>0.775$. Model B classifies more of the numerous negative cases correctly, but Model A has the stronger average of its positive-class and negative-class correct rates.

Check. The counts in each row sum to 1,000: for A, $80+20+810+90=1{,}000$; for B, $60+40+855+45=1{,}000$. The common mistake to avoid is saying that one ranking is universally correct. Each metric answers a different question, and a model choice should match the evaluation objective.

Choosing a Target Class Mix

Equal class weighting is a particular standardization, not a rule that every analysis must use. If an analyst has a justified target class proportion, the weighted-accuracy formula can use that value. The choice should be stated explicitly: otherwise, readers may mistake a class-standardized summary for observed accuracy.

Worked Example: Standardizing to a Specified Class Mix

A hypothetical classifier is evaluated on 220 cases. There are 60 actual positives and 160 actual negatives. The classifier has $TP=45$, $FN=15$, $TN=136$, and $FP=24$. Calculate its ordinary accuracy, balanced accuracy, and weighted accuracy for a target population with 30% positive cases.

Method. Sensitivity is $45/(45+15)=45/60=0.75$. Specificity is $136/(136+24)=136/160=0.85$.

There are $45+136=181$ correct classifications among 220 cases. Ordinary accuracy is $181/220=0.8227$ to four decimal places.

Balanced accuracy is $(0.75+0.85)/2=1.60/2=0.80$. For a target positive proportion $q=0.30$, weighted accuracy is $0.30(0.75)+0.70(0.85)=0.225+0.595=0.820$.

Check. The observed positive proportion is $60/220=0.2727$, which differs from the target proportion of 0.30. Accordingly, ordinary accuracy (0.8227) need not equal the standardized weighted accuracy (0.8200). The common mistake to avoid is using $q=0.50$ automatically when the question specifies a different target mix.

Interpretation and Limitations

Balanced accuracy is useful when performance on both classes matters and class frequencies are unequal. For example, if missing positive cases is a concern, it prevents a large negative class from overwhelming the summary. It does not, by itself, establish that the classifier is suitable: a single number can conceal the individual sensitivity and specificity values, so report those rates alongside balanced accuracy when the distinction matters.

Balanced accuracy is not an expected financial cost, and equal class weighting does not assert that errors in the two classes have equal consequences. It also does not select a classification threshold. Those are separate questions involving the decision context. For evaluation, the threshold and the class definitions must be clear, and the metric should be computed on data appropriate to the evaluation design.

The formula requires at least one actual case from each class. If $TP+FN=0$, sensitivity is undefined; if $TN+FP=0$, specificity is undefined. In either situation, the standard binary balanced accuracy cannot be calculated from that evaluation set. Do not silently set the missing class-specific rate to zero or one; report the limitation and use an evaluation design that can assess both classes.

For more than two classes, a common analogous summary is macro-average recall: calculate the correct-classification rate separately within each actual class, then take the unweighted mean across classes. The binary definition in this tutorial is the mean of sensitivity and specificity, which are the two class-specific correct rates in the binary case.

Common Exam Traps

Averaging counts instead of rates. Balanced accuracy averages sensitivity and specificity, not $TP$ and $TN$.
Using precision in place of specificity. Specificity uses actual negatives as its denominator: $TN/(TN+FP)$. Precision uses predicted positives, so it is a different quantity.
Assuming balanced accuracy equals ordinary accuracy. They coincide in some cases, but generally use different class weights.
Ignoring the stated target mix. A weighted accuracy with $q\ne0.50$ is not balanced accuracy; use the specified class proportion.
Calculating with a missing class. If either actual class has no observations, its class-specific rate—and hence standard binary balanced accuracy—is undefined.

Key takeaway: Balanced accuracy is the mean of sensitivity and specificity. Equivalently, it is a weighted correct-classification rate that assigns each actual class one-half of the total weight. Always state the metric used and retain the two class-specific rates so the summary can be interpreted.

Check Your Understanding

For each problem, calculate the requested summary and check which class denominators are being used.

A classifier has $TP=36$, $FN=4$, $TN=171$, and $FP=9$. Calculate sensitivity, specificity, balanced accuracy, and ordinary accuracy. (Answer: sensitivity $=0.9000$, specificity $=0.9500$, balanced accuracy $=0.9250$, ordinary accuracy $=207/220=0.9409$.)
An evaluation set has 50 actual positives and 450 actual negatives. A model has sensitivity $0.72$ and specificity $0.96$. Calculate ordinary accuracy and balanced accuracy. (Answer: ordinary accuracy $=0.50(0.72)+0.50(0.96)$ is not appropriate for the observed mix; using the class proportions gives $0.10(0.72)+0.90(0.96)=0.9360$. Balanced accuracy $=(0.72+0.96)/2=0.8400$.)
A classifier has sensitivity $0.64$ and specificity $0.88$. Calculate its weighted accuracy for a target population that is 25% positive. (Answer: $0.25(0.64)+0.75(0.88)=0.8200$.)
Two classifiers are evaluated on 80 actual positives and 320 actual negatives. Model A has $TP=64$, $FN=16$, $TN=256$, $FP=64$. Model B has $TP=56$, $FN=24$, $TN=288$, $FP=32$. Calculate ordinary and balanced accuracy for each, then identify the ranking under each metric. (Answer: A: ordinary accuracy $=320/400=0.8000$, balanced accuracy $=(0.80+0.80)/2=0.8000$. B: ordinary accuracy $=344/400=0.8600$, balanced accuracy $=(0.70+0.90)/2=0.8000$. Ordinary accuracy ranks B higher; balanced accuracy ties them.)
An evaluation set contains no actual positive cases. Can standard binary balanced accuracy be calculated? Explain which quantity is undefined. (Answer: No. Sensitivity has denominator $TP+FN=0$, so sensitivity is undefined, and the standard balanced accuracy is therefore undefined.)

What You'll Learn