Classification vs Regression: Numbers or Categories?
Start with the shape of the answer
Machine learning problems become easier to frame when you ask what the target means. Regression predicts a numerical quantity such as rent or delivery time. Classification predicts a category such as spam or not spam.
Key Points
- Regression output is a number on a scale.
- Classification output is a class or label.
- The features can look similar in both problems.
- The target definition, not the algorithm name, determines the first framing choice.
Regression
The answer is a number
The gap between two answers means something. Being out by four minutes is worse than being out by one.
Classification
The answer is a label
Same model, plus one decision: late means over 30 minutes. Change that number and every label changes with it.
Tip:“Will this customer leave?” is classification. “How many days until they leave?” is regression, even if both use the same customer features.
Examples from everyday decisions
Key Points
- House rent: regression.
- Transaction is fraudulent: binary classification.
- Support ticket category: multiclass classification.
- Number of support minutes: regression.
- Whether a medical scan needs review: classification.
A category can have two classes or many classes. A binary target might be 0 or 1. A multiclass target might be red, amber, or green. The labels are names for outcomes, not quantities whose spacing automatically has meaning.
The target type changes the loss
For regression, squared or absolute distance between a prediction and the target can make sense. For classification, treating “cat” as 2 and “dog” as 3 would invent a distance that may not exist. Classification models need a loss and evaluation method designed for probabilities and class decisions.
Key Points
- Regression asks how far a number is from the target.
- Classification asks whether the class decision is correct and how confident the model was.
- Class labels are often encoded as integers for software, but those integers are not automatically measurements.
- The business cost of false positives and false negatives matters.
Binary classification has two related outputs
A binary classifier can produce a probability for the positive class and a final class label. For example, it may estimate a 0.82 chance that a payment is fraudulent, then label it fraud when the chosen threshold is 0.5.
1probabilities = [0.12, 0.82, 0.47, 0.91]
2threshold = 0.50
3labels = [int(probability >= threshold) for probability in probabilities]
4
5print("Probabilities:", probabilities)
6print("Predicted labels:", labels)Probabilities: [0.12, 0.82, 0.47, 0.91] Predicted labels: [0, 1, 0, 1]
The threshold is a decision setting. Changing it changes false positives and false negatives without retraining the probability model.
Multiclass and multilabel are different
In multiclass classification, one example belongs to one class from several choices, such as billing, login, or delivery. In multilabel classification, one example can receive several labels, such as “urgent” and “billing” together.
Key Points
- Multiclass: one of K classes.
- Multilabel: any combination of labels can apply.
- The output shape and evaluation metrics change with this distinction.
- Do not call a multilabel target multiclass just because both have several names.
Features do not guarantee the target is predictable
A classification framing can be correct while the model remains weak. If the features do not contain useful signal, if labels are inconsistent, or if the future differs from the training data, changing the algorithm will not create reliable information.
Key Points
- Define when the prediction is made.
- Define what each class means and how ambiguous cases are labelled.
- Check class frequencies and label quality.
- Choose an evaluation design that resembles use.
A small target table
1records = [
2 {"message_length": 18, "has_link": 0, "label": "safe"},
3 {"message_length": 42, "has_link": 1, "label": "spam"},
4 {"message_length": 25, "has_link": 0, "label": "safe"},
5]
6
7features = [[row["message_length"], row["has_link"]] for row in records]
8labels = [row["label"] for row in records]
9
10print("Feature rows:", features)
11print("Class labels:", labels)Feature rows: [[18, 0], [42, 1], [25, 0]] Class labels: ['safe', 'spam', 'safe']
The feature table still looks familiar from regression. What changes is the target meaning and the evaluation question.
Choose the model after framing the problem
Key Points
- First define the target and prediction time.
- Then decide whether it is numerical, binary, multiclass, or multilabel.
- Choose losses and metrics that reflect the decision.
- Only then compare algorithms and tune their settings.
Logistic Regression, trees, K-Nearest Neighbours, Naive Bayes, SVMs, and ensembles can all be classification tools. They make different assumptions and produce different probability behaviour.
Where people get confused
Key Points
- “If labels are 0 and 1, this is regression.” Not necessarily. The numbers may only encode two categories.
- “Classification only predicts yes or no.” It can also be multiclass or multilabel.
- “A probability of 0.8 is the same as a label.” The probability carries confidence; the label depends on a threshold.
- “Accuracy is always the right metric.” It can hide failures in rare or costly classes.
- “A classification model proves the event will happen.” It estimates a class under a data-generating process.
Quick check
What is the basic difference between regression and classification?
Regression predicts a numerical quantity. Classification predicts a class or category, often with a probability and a final thresholded label.
Why should class labels encoded as 0, 1, and 2 not automatically be treated as measurements?
The integers may be names for categories. Their spacing may not represent a meaningful amount or order.
What is the difference between a probability and a binary label?
A probability expresses estimated confidence for a class. A binary label is the decision produced after applying a threshold.
What should you define before choosing a classification algorithm?
Define the target classes, prediction time, available features, label quality, and the error costs that matter to the decision.
Recap
Key Points
- Regression predicts numbers; classification predicts categories.
- Binary, multiclass, and multilabel tasks have different output structures.
- Probabilities and class decisions are related but not identical.
- The next lesson turns a weighted score into a probability with Logistic Regression.
Theia
Can you explain this concept?
Practise Classification vs Regression: Numbers or Categories? with realistic interview questions. Choose spoken or typed answers, get feedback on your reasoning, then return to the material that needs work.
Sign in to practise this conceptThe session stays focused on this lesson. Your selected interview level sets the bar.