F Scores and the Precision-Recall Trade-Off

Why Combine Precision and Recall?

A binary classifier’s performance depends on which class is designated positive and on the threshold used to turn scores into decisions. At a chosen threshold, precision measures the fraction of predicted positives that are truly positive, while recall measures the fraction of actual positives that are detected. Improving one can come at the expense of the other: a stricter threshold may reduce false positives but also miss more actual positives.

An F score combines precision and recall into a single summary. It can be useful when both matter, but it does not eliminate the need to say which class is positive, which threshold produced the counts, or what practical objective the metric is intended to represent.

Definition: For a binary classifier, let $TP$, $FP$, and $FN$ denote the numbers of true positives, false positives, and false negatives. When $TP+FP>0$, precision is $P=TP/(TP+FP)$. When $TP+FN>0$, recall is $R=TP/(TP+FN)$. For $\beta>0$, when $\beta^2P+R>0$, the $F_\beta$ score is $$ F_\beta=(1+\beta^2)\frac{PR}{\beta^2P+R}. $$ The special case $\beta=1$ is the $F_1$ score. If the rate-based formula is undefined because $P=R=0$, use the count-based convention described below when its denominator is positive; if that denominator is zero, the score is undefined.

The parameter $\beta$ sets the relative emphasis: $\beta>1$ gives more weight to recall, while $0<\beta<1$ gives more weight to precision. The weighting is through $\beta^2$, so changing $\beta$ does not change the classifier’s predictions or its confusion matrix. It changes only how precision and recall are summarized.

Calculating F Scores From Counts

The rate formula is convenient when precision and recall are already available. A count-based form is often safer when the confusion matrix is given, especially when there are no true positives:

F_\beta= \frac{(1+\beta^2)TP} {(1+\beta^2)TP+\beta^2FN+FP}, \qquad (1+\beta^2)TP+\beta^2FN+FP>0.

When $TP>0$, both precision and recall are defined and positive. Substituting $P=TP/(TP+FP)$ and $R=TP/(TP+FN)$ into the rate formula gives the count-based form. The derivation is useful because it also clarifies exactly when the rate-based algebra is valid.

Substitute the rates.
For $TP>0$, write $P=TP/(TP+FP)$ and $R=TP/(TP+FN)$ in $F_\beta=(1+\beta^2)PR/(\beta^2P+R)$.

Combine the numerator.
The product of the rates is $PR=TP^2/[(TP+FP)(TP+FN)]$.

Combine the denominator.
$\beta^2P+R=TP[\beta^2(TP+FN)+(TP+FP)]/[(TP+FP)(TP+FN)]$.

Cancel common factors.
Since $TP>0$, cancellation yields $F_\beta=(1+\beta^2)TP/[\beta^2(TP+FN)+TP+FP]$, which is the stated count-based form.

This derivation uses $TP>0$, equivalently $P>0$ and $R>0$ when both rates are defined. It must not be used by dividing through by $PR$ when $TP=0$. In that case, if the count-formula denominator is positive, its numerator is zero and the count-based score is $0$. This is an explicit zero-score convention; software packages may use different conventions when a metric is undefined, so the convention should be reported.

Worked Example: Calculating F1 and F2 From a Confusion Matrix

A hypothetical claims-screening model flags some submitted claims for additional review. At a fixed threshold, it produces $TP=25$, $FP=15$, $FN=10$, and $TN=150$. Calculate precision, recall, $F_1$, and $F_2$. Round final rates and scores to four decimal places.

Method. Calculate precision and recall from their definitions, then use the count formula for each F score. This avoids treating the four confusion-matrix counts as if they all enter the F score.

Precision. There are $TP+FP=25+15=40$ flagged claims, of which 25 are true positives: $P=25/40=0.625$, or $0.6250$.

Recall. There are $TP+FN=25+10=35$ actual positives, of which 25 are detected: $R=25/35=5/7=0.7142857\ldots$, or $0.7143$.

F1. Set $\beta=1$, so $\beta^2=1$: $F_1=2(25)/[2(25)+10+15]=50/75=0.6666667\ldots$, or $0.6667$.

F2. Set $\beta=2$, so $\beta^2=4$: $F_2=5(25)/[5(25)+4(10)+15]=125/(125+40+15)=125/180=0.6944444\ldots$, or $0.6944$.

The results are consistent with the weighting: recall ($0.7143$) exceeds precision ($0.6250$), and the recall-emphasizing $F_2$ is higher than $F_1$. The common mistake is to include $TN$ in the F-score denominator; neither $F_1$ nor $F_2$ uses true negatives.

How Beta Changes the Comparison

The formula is a weighted harmonic mean, not an ordinary arithmetic average. In reciprocal form,

\frac{1}{F_\beta} = \frac{1}{1+\beta^2} \left(\frac{\beta^2}{R}+\frac{1}{P}\right), \qquad P>0,\ R>0.

The reciprocal of recall has weight $\beta^2$, while the reciprocal of precision has weight 1. A low value for either rate therefore pulls the combined score downward, and increasing $\beta$ increases recall’s relative influence. The score is not a direct statement about financial cost: choosing $\beta$ expresses a metric preference, but does not by itself specify the dollar consequences of false positives and false negatives.

Worked Example: Choosing a Summary for Different Priorities

Two hypothetical screening thresholds have these measured rates on the same evaluation population: Threshold A has precision $0.80$ and recall $0.50$; Threshold B has precision $0.60$ and recall $0.75$. Calculate $F_1$, $F_2$, and $F_{0.5}$ for each, and identify which threshold each score prefers. Round final scores to four decimal places.

Threshold A, $F_1$. $F_1=2(0.80)(0.50)/(0.80+0.50)=0.80/1.30=0.6153846\ldots$, or $0.6154$.

Threshold A, $F_2$. $F_2=5(0.80)(0.50)/[4(0.80)+0.50]=2.00/3.70=0.5405405\ldots$, or $0.5405$.

Threshold A, $F_{0.5}$. Here $\beta^2=0.25$, so $F_{0.5}=1.25(0.80)(0.50)/[0.25(0.80)+0.50]=0.50/0.70=0.7142857\ldots$, or $0.7143$.

Threshold B, $F_1$. $F_1=2(0.60)(0.75)/(0.60+0.75)=0.90/1.35=0.6666667\ldots$, or $0.6667$.

Threshold B, $F_2$. $F_2=5(0.60)(0.75)/[4(0.60)+0.75]=2.25/3.15=0.7142857\ldots$, or $0.7143$.

Threshold B, $F_{0.5}$. $F_{0.5}=1.25(0.60)(0.75)/[0.25(0.60)+0.75]=0.5625/0.90=0.6250$.

Thus $F_1$ prefers B ($0.6667>0.6154$), and $F_2$ also prefers B ($0.7143>0.5405$). But $F_{0.5}$ prefers A ($0.7143>0.6250$), consistent with its greater emphasis on precision. The trap is to assume one threshold must be “best” regardless of the chosen objective.

What an F Score Does Not Tell You

An F score is a compact summary, not a complete evaluation of a classification model. Its count-based formula contains $TP$, $FP$, and $FN$, but not $TN$. Consequently, two evaluation results can have the same F score while having different numbers of true negatives and different specificity. F scores also do not describe the quality of predicted probabilities, the ranking of cases at other thresholds, or the consequences and costs of each error.

Because precision depends on the mix of positive and negative cases in the evaluated population, an F score can change when that population changes, even if a model’s underlying behavior is otherwise similar. Always report the positive class, the evaluation population, and the threshold alongside the score. When comparing models, use the validation plan established for the analysis; do not choose a threshold on a protected final test set and then present its performance as an unbiased final evaluation.

Summary	Uses	Does not directly summarize
F score	True positives, false positives, false negatives, and the selected beta	True negatives, calibration, or performance at other thresholds
Precision	False discoveries among predicted positives	How many actual positives were missed
Recall	Actual positives detected	How many predicted positives are false alarms

Zero Counts and Exam Traps

Check denominators before calculating. If $TP+FP=0$, no cases were predicted positive, so precision is undefined. If $TP+FN=0$, there are no actual positives, so recall is undefined. The rate-based F formula cannot be used when either needed rate is undefined. Under the count-based convention used here, calculate $F_\beta$ when its denominator is positive; if $TP=0$, the resulting score is $0$. If $TP=FP=FN=0$, the count-formula denominator is zero and the score is undefined.

For example, let $TP=0$, $FP=1$, and $FN=1$. Both rates are defined, and $P=0/(0+1)=0$, $R=0/(0+1)=0$. Substituting into the rate formula gives $0/0$, not a valid calculation. The count denominator is $(1+\beta^2)0+\beta^2(1)+1=\beta^2+1>0$, while the numerator is zero, so the count-based convention gives $F_\beta=0$. This distinction prevents a common error: “both rates are defined” is not enough to ensure the rate-formula denominator is positive.

Key takeaway: State the positive class and threshold, calculate precision and recall carefully, and select $\beta$ to match the intended relative emphasis. Use the count formula when $TP=0$, and do not mistake an F score for a summary of true negatives, probability calibration, or error costs.

Check Your Understanding

For each problem, use the count or rate formula as appropriate. Unless noted otherwise, round final scores to four decimal places.

A classifier has $TP=18$, $FP=6$, and $FN=12$. Calculate $F_1$. (Answer: $36/(36+6+12)=0.6667$)
A model has precision $0.70$ and recall $0.40$. Calculate $F_2$, and state which rate receives greater emphasis. (Answer: $1.40/(2.80+0.40)=0.4375$; recall receives greater emphasis.)
Model A has precision $0.75$ and recall $0.60$; Model B has precision $0.60$ and recall $0.80$. Which has the higher $F_1$? (Answer: A has $F_1=0.9000/1.3500=0.6667$; B has $F_1=0.9600/1.4000=0.6857$, so B is higher.)
A classifier has $TP=0$, $FP=3$, $FN=7$, and $TN=40$. Under the stated count-based convention, calculate $F_2$. Explain why substituting $P$ and $R$ directly into the rate formula is invalid. (Answer: $0/[0+4(7)+3]=0.0000$; $P=R=0$, so the rate formula has denominator $0$.)
Two evaluations have counts $(TP,FP,FN,TN)=(8,2,2,88)$ and $(8,2,2,8)$. Calculate $F_1$ for both and compare their specificities. (Answer: Both have $F_1=16/(16+2+2)=0.8000$. Specificities are $88/(88+2)=0.9778$ and $8/(8+2)=0.8000$, respectively; the equal F scores do not imply equal specificity.)

What You'll Learn