LoRA

Understand LoRA for Parameter-Efficient Fine-Tuning

Fine-tuning a large model does not always require updating every model parameter. Low-Rank Adaptation (LoRA) takes advantage of this idea by keeping the original model weights frozen and learning much smaller matrices that modify selected layers. For developers, the important benefit is not simply that LoRA is “smaller fine-tuning.” It changes what must be trained, stored, and moved between experiments. Understanding that distinction makes it easier to decide when LoRA is useful, what it does not save, and how adapter choices affect model behavior.