Artificial Intelligence on Nalar

Control Diffusion Generation with Classifier-Free Guidance

Fri, 11 Sep 2026 00:00:00 +0700

Control Diffusion Generation with Classifier-Free Guidance

A text-conditioned diffusion model can understand a prompt yet still produce a sample that only loosely follows it. Increasing prompt influence can make the requested content more prominent, but pushing too hard can reduce variation or introduce visual artifacts.

Classifier-free guidance (CFG) gives diffusion systems a direct control for this trade-off. At each denoising step, the model produces one prediction with the condition and another without it. The sampler combines those predictions so the conditional signal can be strengthened.

Inspect Transformer Residual Streams with the Logit Lens

Fri, 11 Sep 2026 00:00:00 +0700

Inspect Transformer Residual Streams with the Logit Lens

A language model returns token probabilities only after its final transformer block, but developers often need to inspect what happens before that point. A prompt may produce an unexpected completion, a fine-tuned checkpoint may behave differently from its base model, or a model modification may alter predictions several blocks before the final output.

The logit lens is a simple diagnostic for these cases. It takes an intermediate residual-stream vector, maps it through the model’s output machinery, and ranks vocabulary tokens as if that intermediate state were already ready for prediction.

Use an Exponential Moving Average of Model Weights

Fri, 11 Sep 2026 00:00:00 +0700

Use an Exponential Moving Average of Model Weights

Neural network parameters move after every optimizer step. Near the end of training, those updates can leave a model bouncing around a useful region instead of sitting at one especially representative point.

An exponential moving average, usually shortened to EMA, keeps a second set of weights that changes more smoothly. The optimizer still updates the normal model. EMA simply tracks a weighted history of those parameters and can provide a separate set of weights for evaluation or deployment.

Use Gated Feed-Forward Blocks in Transformers

Fri, 11 Sep 2026 00:00:00 +0700

Use Gated Feed-Forward Blocks in Transformers

A transformer block does more than attention. After attention mixes information across token positions, a feed-forward block transforms each token representation independently. That second stage often accounts for a large share of the model’s parameters and computation.

A gated feed-forward block changes this stage by adding a second projection that controls how much of another projected signal passes through. GLU, GEGLU, and SwiGLU are common names for members of this family.