https://nalar.dev/tags/variational-autoencoders/ Recent content in Variational Autoencoders on Nalar Hugo en-us Fri, 11 Sep 2026 00:00:00 +0700 https://nalar.dev/prevent-vae-posterior-collapse-with-free-bits/ Fri, 11 Sep 2026 00:00:00 +0700 https://nalar.dev/prevent-vae-posterior-collapse-with-free-bits/ <h1 id="prevent-vae-posterior-collapse-with-free-bits">Prevent VAE Posterior Collapse with Free Bits</h1> <p>A variational autoencoder can appear to train normally while its latent representation becomes nearly useless. The decoder learns to explain the data without depending on the latent variable, the encoder moves toward the prior, and the KL divergence shrinks toward zero. This failure mode is called <strong>posterior collapse</strong>.</p> <p><strong>Free bits</strong> is a small change to the VAE objective that can reduce one source of that collapse. It stops the KL term from rewarding the optimizer for squeezing an already-small amount of latent information even closer to zero. The technique is simple, but its name and common shorthand can lead to a misleading mental model. Free bits does not force a latent variable to contain a chosen amount of information. It changes the optimization pressure below a threshold.</p>