Matthews Correlation Coefficient

Why Use the Matthews Correlation Coefficient?

A classification model can appear successful by correctly labeling many observations from a common class while doing little to identify the less common class. Accuracy alone may hide this pattern. The Matthews correlation coefficient, usually abbreviated MCC, summarizes the relationship between actual and predicted binary labels using all four cells of a confusion matrix. It is often useful when the class counts are uneven, but it is not a substitute for understanding the decision context.

The calculation starts with the confusion matrix in a fixed orientation. Here, a positive prediction means the model assigns the event to the positive class, and a positive actual label means the event truly belongs to that class. Confusion-matrix counts must use the same orientation throughout.

Definition: Let $TP$ be true positives, $TN$ true negatives, $FP$ false positives, and $FN$ false negatives. When all four row and column margins are nonzero, the Matthews correlation coefficient is $$ \operatorname{MCC} = \frac{TP\cdot TN-FP\cdot FN} {\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}}. $$ Its value lies between $-1$ and $1$. A value of $1$ indicates perfect agreement between actual and predicted labels; $-1$ indicates perfect reversal; and $0$ indicates zero correlation between the two binary label variables.

The four factors in the denominator are margins of the confusion matrix: predicted positives, actual positives, actual negatives, and predicted negatives. Thus, the formula uses every count, not just the number of correct classifications. If any of these margins is zero, the denominator is zero and the MCC is undefined; it should not be reported as zero.

Worked Example: Calculating MCC From a Small Validation Set

A hypothetical screening classifier is evaluated on 100 records. It identifies 30 actual positives correctly, misses 10 actual positives, incorrectly flags 10 actual negatives, and correctly clears 50 actual negatives. Calculate the MCC and interpret it.

Step 1: Record the counts and check the total. The counts are $TP=30$, $FN=10$, $FP=10$, and $TN=50$. Their sum is $30+10+10+50=100$, as required.

Step 2: Calculate the numerator. $TP\cdot TN-FP\cdot FN=30\cdot50-10\cdot10=1{,}500-100=1{,}400$.

Step 3: Calculate the denominator margins. The four factors are $TP+FP=40$, $TP+FN=40$, $TN+FP=60$, and $TN+FN=60$. Their product is $40\cdot40\cdot60\cdot60=5{,}760{,}000$, whose square root is $2{,}400$.

Step 4: Divide and check. $\operatorname{MCC}=1{,}400/2{,}400=0.5833$, rounded to four decimal places. The positive value indicates agreement between the predictions and actual labels, but it is not a percentage correct and does not by itself establish whether the classifier is suitable for a particular decision.

Why the Formula Is a Correlation

The MCC has a useful interpretation: it is the ordinary correlation between the actual binary label and the predicted binary label when each is coded as 1 for positive and 0 for negative. This connection explains both the product of margins in the denominator and the range from $-1$ to $1$.

Let $Y$ be the actual label and $Z$ the predicted label, with each coded 1 for positive and 0 for negative. Let $N=TP+TN+FP+FN$. Across the $N$ evaluated observations, the empirical means are $\bar{Y}=(TP+FN)/N$ and $\bar{Z}=(TP+FP)/N$. The mean of the product $YZ$ is $TP/N$, because $YZ=1$ only for a true positive. Therefore, their empirical covariance is

\operatorname{Cov}(Y,Z) = \frac{TP}{N} - \frac{(TP+FN)(TP+FP)}{N^2} = \frac{TP\cdot TN-FP\cdot FN}{N^2}.

The final equality follows by expanding the numerator after putting the terms over $N^2$: $TPN-(TP+FN)(TP+FP)=TP\cdot TN-FP\cdot FN$. The empirical variances are $\operatorname{Var}(Y)=(TP+FN)(TN+FP)/N^2$ and $\operatorname{Var}(Z)=(TP+FP)(TN+FN)/N^2$. Substituting these expressions into the correlation formula $\operatorname{Cov}(Y,Z)/\sqrt{\operatorname{Var}(Y)\operatorname{Var}(Z)}$ cancels the $N^2$ terms and gives exactly the MCC formula. This derivation requires nonzero variance for both label variables, which is equivalent to having both actual classes and both predicted classes represented.

Worked Example: A Confusion Matrix With Uneven Margins

A hypothetical model flags a rare event in 1,000 cases. Its confusion matrix has $TP=18$, $FP=30$, $FN=12$, and $TN=940$. Calculate MCC from the four counts, show how each margin enters, and interpret the result.

Step 1: Check the counts. There are $18+30=48$ predicted positives and $12+940=952$ predicted negatives. There are $18+12=30$ actual positives and $30+940=970$ actual negatives. The total is $48+952=1{,}000$.

Step 2: Calculate the numerator. $TP\cdot TN-FP\cdot FN=18\cdot940-30\cdot12=16{,}920-360=16{,}560$.

Step 3: Multiply the four margins. $(TP+FP)(TP+FN)(TN+FP)(TN+FN)=48\cdot30\cdot970\cdot952$. First, $970\cdot952=923{,}440$, and then $923{,}440\cdot1{,}440=1{,}329{,}753{,}600$. The square root is approximately $36{,}465.8$.

Step 4: Calculate and check. The MCC is $16{,}560/36{,}465.8\approx0.4541$, rounded to four decimal places. The positive value indicates that the predicted labels agree with the actual labels more than they disagree in the correlation sense. It does not mean that 45.41% of cases are correct; the fraction correct is a separate measure.

Interpreting MCC With Imbalanced Classes

When one actual class is much more common than the other, accuracy can be high even for a classifier that misses every positive case. MCC can reveal that the predictions have little or no useful association with the actual labels, because its numerator and denominator incorporate false positives and false negatives as well as correct classifications. This is why it can be a more informative single summary than accuracy in some imbalanced settings.

However, MCC is not unaffected by class prevalence, nor does it guarantee good performance for the less common class. It summarizes the particular confusion matrix produced on a particular evaluation sample at a particular classification threshold. A change in the threshold can change all four counts and therefore the MCC. It also assigns no different importance to errors based on their consequences: a false negative and a false positive affect the formula through the counts, but the formula does not know their relative costs.

Worked Example: High Accuracy Can Coexist With Undefined MCC

A hypothetical event occurs in 20 of 1,000 evaluated cases. A classifier predicts every case to be negative. Calculate accuracy and MCC. Then compare them with a second classifier having $TP=12$, $FN=8$, $FP=38$, and $TN=942$.

Step 1: Evaluate the all-negative classifier. Its counts are $TP=0$, $FN=20$, $FP=0$, and $TN=980$. Accuracy is $(TP+TN)/N=(0+980)/1{,}000=0.98$, or 98.00%.

Step 2: Check whether its MCC is defined. The predicted-positive margin is $TP+FP=0$. Consequently, the denominator contains a zero factor and the MCC is undefined. It is not correct to assign an MCC of zero simply because the classifier predicts only one class.

Step 3: Calculate the second classifier's MCC. Its numerator is $12\cdot942-38\cdot8=11{,}304-304=11{,}000$. Its four margins are $TP+FP=50$, $TP+FN=20$, $TN+FP=980$, and $TN+FN=950$. Their product is $50\cdot20\cdot980\cdot950=931{,}000{,}000$, whose square root is approximately $30{,}512.3$. Thus its MCC is $11{,}000/30{,}512.3\approx0.3605$, rounded to four decimal places.

Step 4: Compare carefully. The second classifier has accuracy $(12+942)/1{,}000=95.40\%$, lower than the all-negative classifier's 98.00%, but it identifies some actual positives and has a defined, positive MCC. The comparison shows why accuracy alone can be misleading here. It does not establish that the second classifier is preferable for every use: the consequences of the errors and the operational requirements still matter.

Common Calculation Traps and Limits

The most frequent arithmetic error is using a cell count where a margin is required. In the first worked example, the factor $TN+FP$ is 60, not $TN=50$. Incorrectly substituting 50 gives a denominator of $\sqrt{40\cdot40\cdot50\cdot60}=\sqrt{4{,}800{,}000}\approx2{,}190.9$, and an incorrect MCC of approximately $1{,}400/2{,}190.9=0.6390$. The correct factor is $TN+FP=50+10=60$; the correct denominator is 2,400 and the correct MCC is 0.5833.

Other interpretation errors are also worth avoiding:

Calling MCC a percent correct. It is a correlation, not an accuracy rate. An MCC of 0.45 does not mean 45% of predictions are correct.
Reporting zero when the denominator is zero. If all actual labels or all predictions belong to one class, at least one variance is zero, so MCC is undefined.
Assuming a positive MCC proves the model is useful. The coefficient does not include decision costs, capacity limits, or the consequences of errors.
Ignoring the evaluation design. MCC describes the confusion matrix on the evaluated observations. Its interpretation depends on the validity and representativeness of that evaluation.
Forgetting threshold dependence. MCC uses binary predictions, so changing the rule that converts scores to labels can change the coefficient.

For a defensible assessment, report the MCC together with the confusion-matrix counts and relevant context. This makes class representation, types of errors, and any zero margins visible instead of compressing them into one number. MCC is most useful as a compact summary of agreement, not as a complete model-selection or decision rule.

Key takeaway: Calculate MCC as $(TP\cdot TN-FP\cdot FN)$ divided by the square root of the product of all four confusion-matrix margins. It uses all four counts and can expose misleading accuracy under class imbalance, but it is undefined for zero margins and does not encode error costs or remove threshold dependence.

Check Your Understanding

For each problem, identify the four margins before calculating. Round numerical MCC values to four decimal places.

A hypothetical quality-control classifier has $TP=27$, $FP=13$, $FN=9$, and $TN=151$. Check the total, calculate MCC, and state whether agreement is positive or negative. (Answer: $N=200$; margins $40,36,164,160$; MCC $=0.6442$, positive.)
A model for a hypothetical service interruption has $TP=8$, $FP=12$, $FN=2$, and $TN=78$. Calculate MCC and compare it with the accuracy. (Answer: MCC $=0.5000$; accuracy $=(8+78)/100=0.8600$, or 86.00%.)
For a hypothetical alert system, $TP=14$, $FP=6$, $FN=6$, and $TN=74$. Calculate the numerator, denominator, and MCC. (Answer: numerator $=14\cdot74-6\cdot6=1{,}000$; denominator $=\sqrt{20\cdot20\cdot80\cdot80}=1{,}600$; MCC $=0.6250$.)
A classifier predicts all 500 cases to be negative. Among them, 15 are actually positive and 485 are actually negative. Calculate its accuracy and determine whether MCC is defined. (Answer: accuracy $=485/500=0.9700$, or 97.00%; MCC is undefined because $TP+FP=0$.)
A hypothetical model has $TP=36$, $FN=24$, $FP=40$, and $TN=900$. Check the total and calculate MCC, showing the four margins. (Answer: $N=1{,}000$; margins $76,60,940,924$; numerator $=31{,}440$; denominator $=\sqrt{3{,}960{,}633{,}600}\approx62{,}933.6$; MCC $=0.4996$.)

What You'll Learn