Probability Calibration for Classification Models
A classifier can rank examples correctly while producing probabilities that are poor estimates of real-world likelihood.
If a model assigns 0.8 probability to many comparable cases, calibration asks whether roughly 80% of those cases are actually positive. This matters whenever probabilities drive decisions such as pricing, triage, alert thresholds, expected value, or human review.
Discrimination and calibration are different Metrics such as ROC AUC evaluate how well a model ranks positive examples above negative ones. They do not require predicted probabilities to match observed frequencies.
