.svg)
BPCE was about to put a GenAI assistant in front of its banking customers. The team knew attackers would probe it in ways they hadn't imagined. And with hundreds of test scenarios to validate, doing it by hand was impossible. Here's how they got to production.
First step: mapping risks for BPCE's customer chatbot
The Customer Assistant is an AI chatbot embedded across BPCE's digital channels. Any customer can ask a question in natural language about anything related to banking and get an answer.
For a banking group, the stakes are high on both sides of that promise. Every answer engages BPCE's responsibility toward its customers, accuracy is key and a reputational obligation. But opening a chatbot exposed on the internet is an attack surface.
"The chatbot was going to be exposed on the internet," says Mikael Le Bars of BPCE's Data & AI division. "We were potentially going to face attacks and security breaches we wouldn't have thought of, this was all new to us."
The platform the assistant ran on was still under development when the project started. There was no established playbook for testing it. That's when BPCE brought in Giskard.
"Giskard really helped us get a handle on our risks and identify them. That was a key point both for product development and for the legal side." mentions Léa Poirier, Product Owner, Chat AI at BPCE.
The challenge: quality, security, and organization, all at once
Shipping the assistant meant clearing three bars simultaneously:
- Product quality. Customers needed answers that were reliable, relevant, and genuinely useful day to day.
- Security. An internet-facing chatbot at a major bank invites prompt injections, denial-of-service attempts, and attack patterns the team had never dealt with before.
- Organization. The assistant's knowledge base changes constantly. The team had to reorganize around testing: initial test runs, non-regression testing, bug fixes, and the documentation work each release required.
Expert red teaming, so vulnerabilities never reached production
Giskard assigned two security researchers to BPCE's assistant. For several weeks, they attacked it: prompt injections, denial-of-service scenarios, and more.
"We were able to detect vulnerabilities at build time. That allowed us to properly secure the chatbot and put it into production safely." Mikael Le Bars, Data & AI Division, BPCE
Security wasn't the only gate. Before launch, BPCE validated hundreds of functional tests, pulling in business experts to define what a correct answer looks like in their domain.
"We're talking about hundreds of tests. Given the volume, it's impossible to do everything by hand." Léa Poirier, Product Owner, Chat AI — BPCE
The result: continuous confidence in production
BPCE's team can now demonstrate that their assistant performs as expected.
- Security: the team is more confident about the absence of vulnerabilities, backed by testing rather than hope.
- Performance: the chatbot's quality is measured regularly against a concrete performance baseline.
- Continuous red teaming: newly discovered attack techniques are pushed by Giskard directly into the Hub, so BPCE stays current without running its own threat research.
"Giskard really helped secure our product launch by validating all the upstream tests, both for the product and for legal." Léa Poirier, Product Owner, Chat AI — BPCE
From one chatbot to an AI testing practice
The Customer Assistant is no longer the only project. BPCE now runs three AI assistants through Giskard — Customer Assistant, HR Assistant, and Network Assistant — with testing built in as a continuous practice.
BPCE's knowledge base evolves constantly, and every documentation update can break answers that worked yesterday. So testing runs continuously while continuous red teaming keeps the security side current, pushing newly discovered attack techniques into the Hub as they emerge.
Ready to ship your AI agent with a green light?
If you want to secure your agents like BPCE, start with an AI assessment: our red teaming platform probe your agent for security and quality failures, rank them by severity, and deliver a clear go/no-go recommendation.
