End-to-End Evaluation

What is End-to-End Evaluation?

End-to-End Evaluation assesses an AI product as users experience it: prompt in, tools and retrieval in the middle, final answer or action out--scoring the whole path rather than a single model call in isolation.

What this looks like in production

A perfect LLM answer that called the wrong tool, or a correct SQL query with a misleading summary, only fails when the full trajectory is scored.

What teams usually do about it

• Define scenarios with expected tools, data, and outcomes.
• Combine deterministic tool checks with answer-quality judges.
• Run continuously as models, prompts, and tools change.

Related Giskard articles

• Continuous LLM red teaming
• Function calling and tool-use testing
• Giskard OSS v3

Run end-to-end agent evals with Giskard

Build multi-turn scenarios in Giskard that assert tool calls, SQL/retrieval traces, and final answers together. See continuous red teaming and function-calling tests.

Further reading

Authority reference: NIST AI Risk Management Framework.