Best reads for PMs & Designers
One email every Tuesday: the best reads and one trend, picked for your work.
Free · No ads · See a sample email
Topics
Sources
Updated 6 Nov
Product teams now write evals that spell out what good AI output looks like
Metrics & Experimentation
- Practitioners describe writing down what good output looks like as the job a requirements document used to do.
- Supabase open sourced a benchmark that runs Claude Code, Codex and OpenCode against real tasks in its own product.
- Intercom launched evals for its Fin agent, and Anthropic added eval commands to a Claude skill.
- Most of this is advice and examples, with no numbers on how many teams actually do it.
17
sources
+14
+10
last 30d
+7
First seen 21 Nov 2025
If you are a PM
BeforeNow
Requirements are written as sentencesExamples of good output carry the spec
Eval tool makers define goodCheck whose idea of good you adopt
BeforeNow
The requirements doc gives you the detailAsk to read the eval set
One great screenshot proves the feature
Who said it
Sid, AI PMx.com · 21 October 2026 · 3 posts
…Look at failures, categorize them, write better rubrics, validate your LLM-as-judge against human scores, then automate. …Eidos Design | Building with AIeidosdesign.substack.com · 20 October 2026
AI in UX Studio #19: Evals for Designers: Know If Your AI Feature Got Better or Worse…Learn the S.C.O.R.E. Loop: write the scenarios and the pass/fail criteria that define "good" for your AI feature, then use them to decide if a new model is ready to ship. …
