Inference

MAServeBench held the generated work fixed and changed only the order in which agents were allowed to run. Median deadline attainment went from 100% to 4%. October's measurements put the cost of a swarm at the deadline, and the token meter is the smaller bill.

AIBrix v0.8.0 adds Redis-backed session pinning to its inference gateway, so a multi-turn agent keeps hitting the replica that already holds its KV-cache state. What that changes about serving agents, and what it does not.