LoRA
Low-Rank Adaptation, a parameter-efficient fine-tuning method that learns small low-rank update matrices while keeping most pretrained weights frozen.
Generalisation and Measurement Boundary
LoRA is one parameter-efficient adaptation technique and should not be treated as synonymous with all PEFT methods.
Related Machine-Learning Concepts
- PEFT
- Fine-Tuning
- Quantization
- Adapter
Source
- https://arxiv.org/abs/2106.09685