https://nalar.dev/tags/machine-learning/ Recent content in Machine Learning on Nalar Hugo en-us Tue, 01 Sep 2026 00:00:00 +0700 https://nalar.dev/avoiding-data-leakage-machine-learning/ Tue, 01 Sep 2026 00:00:00 +0700 https://nalar.dev/avoiding-data-leakage-machine-learning/ <p>Data leakage happens when information that would not be available at prediction time influences model training. The result is an evaluation score that looks excellent in development and collapses after deployment.</p> <p>Leakage is often subtle because the model code itself can be correct. The mistake lives in how datasets, features, preprocessing, and time boundaries are constructed.</p> <h2 id="split-before-learning-from-the-data">Split before learning from the data</h2> <p>A classic mistake is standardizing the full dataset and splitting afterward.</p> https://nalar.dev/probability-calibration-for-classification-models/ Tue, 01 Sep 2026 00:00:00 +0700 https://nalar.dev/probability-calibration-for-classification-models/ <p>A classifier can rank examples correctly while producing probabilities that are poor estimates of real-world likelihood.</p> <p>If a model assigns 0.8 probability to many comparable cases, calibration asks whether roughly 80% of those cases are actually positive. This matters whenever probabilities drive decisions such as pricing, triage, alert thresholds, expected value, or human review.</p> <h2 id="discrimination-and-calibration-are-different">Discrimination and calibration are different</h2> <p>Metrics such as ROC AUC evaluate how well a model ranks positive examples above negative ones. They do not require predicted probabilities to match observed frequencies.</p> https://nalar.dev/time-series-cross-validation-with-walk-forward-splits/ Tue, 01 Sep 2026 00:00:00 +0700 https://nalar.dev/time-series-cross-validation-with-walk-forward-splits/ <p>Random train/test splits assume examples are exchangeable. Time-series data violates that assumption because the future occurs after the past, and production models normally predict observations that were not available during training.</p> <p>Walk-forward validation preserves that chronology.</p> <h2 id="why-random-splitting-is-misleading">Why random splitting is misleading</h2> <p>Suppose you want to predict next week&rsquo;s demand from historical sales. A random split can place March observations in the test set while April observations appear in training.</p> <p>Even if features do not explicitly contain future values, the evaluation now uses a model fitted on a future regime. Seasonality, pricing, inventory, customer behavior, and economic conditions can all make the score more optimistic than deployment reality.</p>