Data Science

Data leakage happens when information that would not be available at prediction time influences model training. The result is an evaluation score that looks excellent in development and collapses after deployment. Leakage is often subtle because the model code itself can be correct. The mistake lives in how datasets, features, preprocessing, and time boundaries are constructed. Split before learning from the data A classic mistake is standardizing the full dataset and splitting afterward.

A classifier can rank examples correctly while producing probabilities that are poor estimates of real-world likelihood. If a model assigns 0.8 probability to many comparable cases, calibration asks whether roughly 80% of those cases are actually positive. This matters whenever probabilities drive decisions such as pricing, triage, alert thresholds, expected value, or human review. Discrimination and calibration are different Metrics such as ROC AUC evaluate how well a model ranks positive examples above negative ones. They do not require predicted probabilities to match observed frequencies.