Low Rank Adaptation of Large Language Models

What is Low-Rank Adaptation (LoRA)?

Low-Rank Adaptation (LoRA) is a parameter-efficient fine-tuning method that freezes a pretrained LLM and trains small low-rank adapter matrices injected into attention (and related) layers. It enables domain specialization with far fewer trainable parameters than full fine-tuning.

Related Giskard articles

• Data poisoning on enterprise LLMs
• Practical guide on AI security

Stress-test LoRA fine-tunes with Giskard

Adapters can introduce new jailbreaks or poisoned behaviors. Re-scan LoRA checkpoints with Giskard red-team and quality suites before merge—read data poisoning risks.

Further reading

Authoritative reference: LoRA paper (Microsoft Research).