GPT-4o
⚙️ Technical
Advanced
Agent Evaluation Framework
Build a systematic framework for evaluating AI agent performance — with metric definitions, test set construction, and a scored eval runcard.
The Prompt
# Agent Evaluation Framework You are an AI quality assurance engineer designing a framework for systematically evaluating agent performance. ## My Agent - Agent type: [AGENT TYPE — e.g., research agent, support agent, code review agent] - Key tasks it performs: [LIST 3–5 CORE TASKS] - Current success metric: [HOW YOU MEASURE GOOD OUTPUT TODAY, OR "no formal metric yet"] - Failure modes observed: [KNOWN FAILURE PATTERNS OR "not sure"] - Evaluation cadence: [HOW OFTEN TO RUN EVALS — e.g., per PR, daily, weekly] ## Evaluation Framework Design Build a complete agent evaluation framework: 1. Metric definitions: define 5–8 quantitative and qualitative metrics with scoring rubrics (e.g., accuracy 1–5, latency ms, hallucination rate %) 2. Test set construction: how to build a benchmark dataset of inputs and expected outputs 3. Automated scoring: which metrics can be scored programmatically and how 4. Human eval protocol: a structured rubric for the metrics that require human judgment 5. Regression detection: how to catch when a model update degrades performance 6. Dashboard spec: the key visualizations to track evaluation results over time Produce a sample eval scorecard for one run of [AGENT TYPE] using placeholder results.
📝 Fill in the blanks
Replace these placeholders with your own content:
[AGENT TYPE — e.g., research agent, support agent, code review agent]
[LIST 3–5 CORE TASKS]
[HOW YOU MEASURE GOOD OUTPUT TODAY, OR "no formal metric yet"]
[KNOWN FAILURE PATTERNS OR "not sure"]
[HOW OFTEN TO RUN EVALS — e.g., per PR, daily, weekly]
[AGENT TYPE]
How to use this prompt
1
Hit "Run it in your AI"
We open ChatGPT, Claude, Gemini, Perplexity, Grok or Copilot with this prompt in the box, ready for you to press Enter — and copied to your clipboard as a backup. Pick your AI once and we remember it.
2
Replace the placeholders
Swap out anything in [BRACKETS] with your specific details.
3
Send it
This prompt is tuned for GPT-4o, but it works well in any of them. Prefer to do it by hand? Copy to clipboard is always there too.
Get prompts like this every Monday
Free weekly digest — hand-picked prompts, new tools, AI news.