Multiclass Classification: One-vs-all Method
Now we will approach the classification of data when we have more than two categories. Instead of with two classes, we will expand our definition so that with classes.
Examples of multi-class classification:
- Email tagging: Work, Friends, Family, Hobby
- Medical condition: Not ill, Cold, Flu
- Weather: Sunny, Cloudy, Rain, Snow
One-vs-all (One-vs-rest) method
We divide our problem into binary classification problems. Then we train a Logistic regression classifier for each class to predict the probability that .
We are basically choosing one class and then lumping all the others into a single second class. We do this repeatedly, applying binary logistic regression to each case.
On a new input , to make a prediction, pick the class that maximizes , i.e., use the hypothesis that returned the highest value as our prediction.