learn.ifs.name

Post-training

Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 8

7 cards in “8.2 Parameter-efficient fine-tuning” · whole deck

8.2 Parameter-efficient fine-tuning

  1. Besides instruction tuning, when might we fine-tune a pretrained model?
    To adapt it to a new task, domain (legal, medical) or language/variety whose data wasn’t sufficiently represented in pretraining.
  2. Why is it hard to fine-tune all the parameters of very large LMs?
    Every batch must backpropagate through enormous numbers of parameters in many huge layers, which is very expensive in compute, memory and time; many labs can’t do it.
  3. What is parameter-efficient fine-tuning (PEFT)?
    Fine-tuning that freezes most parameters and updates only a small chosen subset.
  4. What is the intuition of LoRA?
    Low-Rank Adaptation: freeze the dense weight matrices (e.g. \(W^Q, W^K, W^V, W^O\)) and instead learn a low-rank update with far fewer parameters.
  5. How does LoRA update a frozen matrix \(W\) of shape \([k \times d]\)?
    It learns \(A\) \([k \times r]\) and \(B\) \([r \times d]\) with \(r \ll \min(d, k)\), replacing \(W + \Delta W\) by \(W + AB\). The forward pass becomes \(h = xW + xAB\).
    W (frozen)k × d+Ak × r×Br × donly A, B trained
    With small r, A and B have far fewer parameters than W.
  6. What are the advantages of LoRA?
    Much lower hardware requirements (no gradients for most parameters); no extra inference time, since \(AB\) has the shape of \(W\) and can be added into it; and modules for different domains can be swapped in and out by adding or subtracting them.
  7. To which matrices was LoRA originally applied?
    Only the attention matrices: \(W^Q, W^K, W^V, W^O\). Many variants exist.

Source