A test set in a spreadsheet.
An “eval” is just a fixed set of questions with known good answers, run whenever something changes. You don't need to code to build one.
Build it
One Afternoon
- Twenty to fifty real questions or tasks
- A good answer written for each
- Include tricky and edge cases
- Keep it in one spreadsheet
Run it
Whenever Things Change
- New tool or model version
- Changed instructions or prompts
- Updated source documents
- Score each answer: pass or fail
Same test, every change.