Loss Functions

Understand Cross-Entropy Loss for Classification

A classifier needs more than a way to count correct answers. During training, it needs a signal that says not only whether a prediction was wrong, but also how the model’s scores should change. Suppose the correct class is cat. A model that assigns cat probability 0.49 and another class 0.51 is wrong, but it is close to the decision boundary. A model that assigns cat probability 0.001 is also wrong, and much more confident in that mistake. Treating those predictions as equally bad throws away useful information.