What is OWASP ASI09 Human-Agent Trust Exploitation?
Human-agent trust exploitation uses automation's perceived authority to pressure people into approvals - fraudulent payments, policy exceptions, or risky overrides.
What this looks like in production
Sycophancy and misguidance in regulated advice show the softer version: the agent tells you what you want to hear until someone gets hurt.
What teams usually do about it
- Design approval UX that resists urgency framing.
- Show provenance and risk context on every approval request.