Parameter-Efficient Fine-Tuning (Prefix-Tuning)

What is Prefix-Tuning?

Prefix-tuning is a parameter-efficient fine-tuning method that learns continuous task-specific prefix vectors prepended to the model’s activations, while keeping the original transformer weights largely frozen.

Why it matters

It delivers task adaptation with far fewer trainable parameters than full fine-tuning, enabling faster iteration and smaller adapter artifacts—similar in spirit to other PEFT methods like LoRA.

Further reading

Authority reference: Prefix-Tuning: Optimizing Continuous Prompts for Generation (Li & Liang, 2021).