https://nalar.dev/tags/representation-learning/ Recent content in Representation Learning on Nalar Hugo en-us Fri, 04 Sep 2026 00:00:00 +0700 https://nalar.dev/understand-negative-sampling-for-representation-learning/ Fri, 04 Sep 2026 00:00:00 +0700 https://nalar.dev/understand-negative-sampling-for-representation-learning/ <p>Some representation-learning problems have an awkward shape: each training example has one observed target, but the model could choose from thousands or millions of alternatives. Computing a score and normalization term for every alternative on every update can become a major training cost.</p> <p><strong>Negative sampling</strong> changes the training problem. Instead of comparing the observed target with every possible alternative, the model learns from the observed <strong>positive pair</strong> and a small set of deliberately sampled <strong>negative pairs</strong>. The update becomes much cheaper, but it also optimizes a sampled discrimination objective rather than the exact full-class objective.</p>