.svg)
What is OWASP LLM04 (2025) Data and Model Poisoning?
Data and model poisoning means an attacker (or a careless pipeline) puts misleading or malicious content where your system learns or retrieves from. That includes fine-tuning sets, knowledge bases, and embedding stores - not just pre-training.
What this looks like in production
One poisoned wiki page can rank highly in RAG and steer thousands of answers. A backdoor fine-tune might sit quiet until a trigger phrase appears in production. Your QA suite passed; the retrieval layer did not.
What teams usually do about it
- Track provenance for every dataset and index you ingest.
- Monitor retrieval rankings and answer drift for odd clusters.
- Run poison scenarios in Test & Evaluation, not only at training time.
