Lightspeedlsvp.com · 18 September 2026

In December 2025, we led Raindrop’s seed round on a simple thesis: as agents become the predominant form of software, observability has to fundamentally change. The company predicted that traditional evals would give way to production monitoring that surfaces the failures humans — or even traditional monitoring tools — would miss. Today, we’re proud to double down on that thesis and this team as part of their Series A bringing total funding to $50 million. Over the past year, demand for production monitoring has only grown, with agents being deployed into industries like defense, healthcare, and financial services. Research from METR shows that the length of tasks frontier AI agents can complete autonomously with 50% reliability has roughly doubled every seven months since 2019. This means that bad behavior is no longer just an inconvenience, but a liability. Agents don’t fail like traditional software. There’s no stack trace or 500 error. An agent fails by doing the wrong thing convincingly, at scale, until someone happens to notice. We believe Raindrop was built for this moment: its product re

Tony Dinhx.com · 18 September 2026

Need a new set of evals for models like Jev

Sidebar.ioearendil.com · 15 September 2026 · not counted as a party

LLMs have become almost perfect at generating code, but that isn’t the end of the story. Just because the code is formally correct doesn’t mean that it is not introducing unnecessary abstractions, creating duplicates, or just making bad decisions overall. This is not a groundbreaking observation, most people who have vibe-coded a project, have realized that each additional feature can sometimes lead to an explosion of lines of code (LOC). This results in a loss of human agency, because in projects that are adding millions of LOC per month, it is hard for humans to keep up.1 Some people might say that that is not an issue at all, because they trust their agents to deal with it. I have bad news for you, agents can't really deal with the slop either. Coming from a physics background, I always had an experimental/quantitative approach to solving problems. When I started at Earendil, with the task of figuring out how to measure code sloppiness, my natural instinct was to first take a deep dive into the literature and then check what other companies were doing. To be frank, with the exception of a few insightful research papers, I was disappointed at how “v

Hugging Face bloghuggingface.co · 15 September 2026

Your agent works in rehearsal, but during the live demo, it takes a different path and fails the same task. That is embarrassing onstage. In production, it is a reliability problem: a workflow that succeeded once may fail the next time a user makes the same request. For mission-critical work, such as reconciling a financial transaction or checking a contract for an obligation, that can be a showstopper. Most benchmarks hide this variability behind an average. On AppWorld, a ReAct agent using GPT-4.1 succeeded on 77.4% of runs across five repetitions. But it succeeded in all five runs for only 53.0% of tasks — a 24.4-point consistency gap. Most benchmarks report the first number. We built a way to measure the second — and improve it. In an earlier post, we introduced ALTK-Evolve — a system that turns an agent's own past trajectories into reusable guidelines, distilled automatically and injected back at inference time. It measurably improves task success, but those results only asked the average-case question too. This post introduces consistency guidelines, a new guideline type in altk-evolve built on top of a diagnostic tool we call th

Greg Kamradt, ARC Prize presidentx.com · 7 September 2026

To be clear - Astra building tools is fair game