Confusion Matrix and Accuracy: Counting the Four Outcomes
A single correct percentage hides the kind of mistake
Accuracy is the fraction of predictions that are correct. It is easy to understand, but it does not say whether the errors were missed positives or false alarms. A confusion matrix keeps those outcomes visible.
Key Points
- Positive means the condition or class of interest.
- Negative means the condition is absent.
- True means the prediction agrees with reality.
- False means the prediction disagrees with reality.
The four cells have names
Key Points
- True Positive (TP): predicted positive and actually positive.
- True Negative (TN): predicted negative and actually negative.
- False Positive (FP): predicted positive but actually negative.
- False Negative (FN): predicted negative but actually positive.
The word “false” describes the prediction, not the person or event. A false negative is a missed positive. A false positive is a false alarm.
Build a matrix from labels
1actual = [1, 1, 1, 0, 0, 0]
2predicted = [1, 0, 1, 1, 0, 0]
3
4tp = sum(real == 1 and guess == 1 for real, guess in zip(actual, predicted))
5tn = sum(real == 0 and guess == 0 for real, guess in zip(actual, predicted))
6fp = sum(real == 0 and guess == 1 for real, guess in zip(actual, predicted))
7fn = sum(real == 1 and guess == 0 for real, guess in zip(actual, predicted))
8
9print("TP TN FP FN:", tp, tn, fp, fn)TP TN FP FN: 2 2 1 1
The four counts add to the number of examples. Here 2 + 2 + 1 + 1 = 6.
Accuracy is correct predictions divided by all predictions
Accuracy = (TP + TN) ÷ (TP + TN + FP + FN). In the example, accuracy is (2 + 2) ÷ 6 = 0.667, or about 66.7%.
1tp, tn, fp, fn = 2, 2, 1, 1
2accuracy = (tp + tn) / (tp + tn + fp + fn)
3print(f"Accuracy: {accuracy:.3f}")Accuracy: 0.667
Key Points
- Accuracy counts TP and TN as correct.
- Accuracy treats FP and FN as one error each.
- That equal treatment may not match the cost of mistakes.
- Accuracy is meaningful only with the class distribution and decision context.
What the headline says
99.9% accurate
99.9%
accuracy
Out of 10,000 transactions it got 9,990 right. Reported on its own, this number ends the conversation.
What the matrix says
It caught nothing
caught
0
false alarm
0
missed
10
correctly cleared
9,990
Ten frauds, none found. A model that answers no to everything scores 99.9% here, and this is that model.
Read the matrix from the decision point of view
For a disease-screening model, a false negative may delay care while a false positive may lead to another test. For a marketing model, a false positive may waste contact time while a false negative may miss a customer. The cells need a domain meaning before a metric can guide action.
Key Points
- Ask which class is positive.
- Name the consequence of FP and FN separately.
- Check whether the threshold can be changed.
- Report counts as well as rates when the audience needs operational planning.
A baseline can make accuracy look impressive
If 95% of examples are negative, a classifier that always predicts negative has 95% accuracy while never finding a positive. Compare with a baseline and inspect the full confusion matrix before celebrating.
Key Points
- Majority-class accuracy is a useful baseline.
- High accuracy can coexist with zero recall for the positive class.
- Class distribution belongs in every accuracy report.
- Precision, recall, and F1 provide other views in the next lesson.
Counts can change when the threshold changes
A probability model can produce the same scores but different confusion matrices at different thresholds. Lowering the threshold usually moves some examples from predicted negative to predicted positive, changing FP, FN, TP, and TN.
Key Points
- The matrix belongs to a model and threshold.
- Compare thresholds on validation data.
- Do not choose a threshold from the final test set repeatedly.
- Accuracy may rise while the important positive class becomes harder to catch.
Where people get confused
Key Points
- “Positive means good.” Positive means the class selected for attention.
- “False positive means the data is false.” It means the model predicted positive when the actual class was negative.
- “Accuracy is 70%, so 70% of positive cases were found.” That is not necessarily true; recall measures positive cases found.
- “The confusion matrix is only for binary problems.” Multiclass confusion matrices extend the idea with one row and column per class.
- “More correct predictions always means a better model.” The consequences of FP and FN may differ.
Quick check
What is a true positive?
The model predicts the positive class and the actual label is also positive.
What is a false negative?
The actual label is positive but the model predicts negative. It is a missed positive case.
How is accuracy calculated?
Add true positives and true negatives, then divide by all four confusion-matrix counts.
Why can a majority-class model have high accuracy but be useless?
If one class dominates, always predicting that class can be correct often while completely missing the minority class.
Recap
Key Points
- A confusion matrix preserves TP, TN, FP, and FN.
- Accuracy is the overall fraction of correct labels.
- Class balance and error consequences determine whether accuracy is informative.
- The next lesson turns the same four counts into precision, recall, and F1.
Theia
Can you explain this concept?
Practise Confusion Matrix and Accuracy: Counting the Four Outcomes with realistic interview questions. Choose spoken or typed answers, get feedback on your reasoning, then return to the material that needs work.
Sign in to practise this conceptThe session stays focused on this lesson. Your selected interview level sets the bar.