Evaluation
-
The Agent Checked Its Own Work and Called It Done
Six preprints from October and November measured agent reliability instead of asserting it. Chained coding tasks drop from 95% completion to 51%, 11.5% of tool calls return success without doing the work, and swapping durable-execution hosts changed nothing.
-
The Observability Layer Just Became an Agent
LangSmith Engine v2 does not just watch your agent. It red-teams it, flags inefficient trajectories, and validates its own fixes before handing you a pull request. Observability is no longer a dashboard; it is an operator.
-
Beyond the Prompt: A Self-Hosted Agent Stack That Survives Production
The prompt stopped being the hard part. What makes an agent survive tool failures, context drift, and non-deterministic outputs is the open-source harness around it, and a solo operator can now self-host the whole thing.
-
The Verification Gap: Why Agentic Does Not Mean Autonomous
Reasoning models got good because verification works: verifiable rewards, consensus, and learned scoring. Agentic workflows skip it. Dan Luu's experiment shows agents cannot check their own work even when told how.
-
Observability Was Table Stakes. The Eval Layer Is the Next Open-Source Battle.
ClickHouse bought Langfuse and Gartner minted a new category. The agent stack's trust layer is consolidating around observability and evaluation, and eval is still the under-built half.