.svg)
What is Code Execution Metric?
A code execution metric scores generated code by running it against tests or expected outputs—pass@k style—rather than only comparing text to a reference. It measures whether the program works, not whether it looks similar.
Related Giskard articles
Test executable agent code with Giskard — pair execution metrics with tool-misuse probes when agents can run code. See function-calling security.
Authority: OpenAI HumanEval / pass@k (arXiv).
