https://nalar.dev/tags/vector-quantization/ Recent content in Vector Quantization on Nalar Hugo en-us Thu, 24 Sep 2026 00:00:00 +0000 https://nalar.dev/codebook-collapse-leaves-vector-quantizers-with-unused-entries/ Thu, 24 Sep 2026 00:00:00 +0000 https://nalar.dev/codebook-collapse-leaves-vector-quantizers-with-unused-entries/ <p>A vector quantizer can expose thousands of codebook entries while repeatedly selecting only a small subset. The configured vocabulary then overstates the discrete capacity that is actually used. This state is often called codebook collapse: some entries receive frequent assignments, while others become inactive or nearly inactive.</p> <p>The failure is not a shortage of parameters by itself. It comes from the interaction between encoder outputs, the assignment rule, and the procedure used to update code vectors. Once assignments concentrate around a subset of entries, unused entries may receive little or no signal that would move them toward regions occupied by encoder outputs.</p>