https://nalar.dev/tags/sharpness/ Recent content in Sharpness on Nalar Hugo en-us Mon, 01 Jan 0001 00:00:00 +0000 https://nalar.dev/measure-neural-network-sharpness-with-weight-perturbations/ Fri, 11 Sep 2026 00:00:00 +0700 https://nalar.dev/measure-neural-network-sharpness-with-weight-perturbations/ <h1 id="measure-neural-network-sharpness-with-weight-perturbations">Measure Neural Network Sharpness with Weight Perturbations</h1> <p>Two neural networks can reach similar validation loss yet behave very differently when their weights move slightly. One may tolerate small parameter changes with little effect on loss. Another may sit in a region where a tiny change raises the loss sharply.</p> <p><strong>Sharpness</strong> is a family of measurements for this local sensitivity. The basic idea is to perturb model weights around a checkpoint and observe how much the loss can increase. This sounds simple, but the result depends on the perturbation size, parameter scaling, data used for measurement, and method used to search for a damaging direction.</p>