Best reads for PMs & Designers
Get 5 personalized best reads each week, with TL;DR and clear next steps.
One free email every Tuesday · No sponsored posts · See a sample email
Topics
Sources
Teams write down what good output looks like instead of a requirements document
Metrics & Experimentation
- Practitioners describe writing down what good output looks like as the job a requirements document used to do.
- Supabase open sourced a benchmark that runs Claude Code, Codex and OpenCode against real tasks in its own product.
- LangChain published a guide to lining up a model judge with what humans actually prefer.
- About a third of this is job ads from unrelated companies, and much of the rest is one vendor's blog, LangChain.
41
sources
+38
+15
last 30d
+12
First seen 21 November 2025
If you are a PM
- Worth asking what your requirements document is still for, once the examples of good output carry the spec.
- Worth checking which of your acceptance criteria could be written as an example instead of a sentence.
- One vendor's blog is much of this evidence, so worth checking whose definition of good you would adopt.
If you are a designer
- Worth asking to read the eval set, since it holds the detail a requirements document used to give you.
- Ask whether one good screenshot is standing in for a judgement, when a single output is only an example.
- Worth checking whether anything in that eval describes how an answer is presented, not only whether it is correct.
Who said it
YouTube: AI Engineeryoutube.com · 19 September 2026
Two Bugs That Hid in Plain Sight: A vLLM Debugging Detective Story — Asaf Gardin & Yuval Belfer