Scaling Agents, Paying the Bill
Plus MCP migration, latency design, and agent benchmarks
Like Justin’s career after cutting 80% of *NSYNC. Just don’t mention the tour.
HOT TAKE
Frequent f-Liar
Low-value hallucinations may be cheaper to tolerate than prevent.
What should guardrails minimize: Frequency or Consequence?
LAST WEEK’S TAKE
Tough Call
Models aren’t the only ones who struggle to make a call - a dead 50/50 on whether the server or client should own tool overload.
HIDDEN GEMS
Curated finds to help you stay ahead
Using a shopping-cart MCP server, the demo compares sticky-session and stateless deployments, with Cloud Run logs, pod recycling, and concurrent requests showing how client-carried state enables round-robin scaling.
Coding Agent Harness Efficiency Benchmark
Holding DeepSeek V4 Flash constant, the benchmark compares Pi, OpenCode, Claude Code, and Nanocoder across bug fixes, finding similar graded quality but substantial differences in tokens, tool calls, and completion time.
Pairwise SRE Agent Evaluation Method
Comparing similar incident-investigation traces with an LLM judge and converting pairwise wins into Elo scores provides an unsupervised signal for tracking SRE agent quality without immediate ground truth.
SkyPilot Multi-Cloud GPU Platform Launch
Combining fragmented GPU capacity across clouds and clusters, the platform provides fleet management, governance, and support for pretraining, post-training, multi-cluster serving, and sandboxes without replacing existing SkyPilot workflows.
MCP 7-28
Five Guides to the New Specification
The MCP community marked the 7-28 release with launch parties around the world this week. To help teams understand what changed and plan their transition, we’ve collected five AAIF blogs covering migration, stateless architecture, platform governance, deprecation, authorization, and long-running tasks.
MCP 2026-07-28: What’s Changing and How to Migrate
A practical migration guide to the release’s stateless core, Tasks and MCP Apps extensions, authorization changes, and deprecations, with concrete checks for MCP client and server maintainers.
Migrate Sessions to Stateless Requests with MCP 2026-07-28
A code-led TypeScript migration guide showing how to replace session IDs and transport registries with per-request server instances, explicit state handling, standard load balancing, and controlled legacy-client support.
7-28 Hands MCP to Platform Teams
A platform engineering guide to the release’s stateless architecture, authorization boundaries, schema validation, and observability changes, with practical steps for governing and scaling MCP services.
MCP Just Handed You a Deprecation Policy. Steal It.
A practical framework for applying MCP’s new feature lifecycle to internal tools and connectors, covering ownership, versioning, migration paths, sunset dates, and communication with dependent teams.
MCP Graduates to Enterprise Infrastructure
A broad technical overview of MCP’s stateless architecture, authorization hardening, formal feature lifecycle, and official extensions for interactive interfaces and durable long-running operations.
More MCP on August 6
Couldn’t make one of the launch parties, or still working through what 7-28 means for your systems?
MCP Jam is a free virtual event on August 6, featuring technical talks, live demos, and lessons from developers, maintainers, and platform teams building with MCP.
Register here
JOB OF THE WEEK
Principal ML Platform Engineer // Synthesia // Europe (Remote)
Synthesia develops AI video software for business use. This principal-level IC role shapes the ML platform supporting generative model training, evaluation, deployment, and serving, with responsibility for infrastructure reliability, automation, GPU workloads, internal tooling, and architectural direction.
Responsibilities
Design platform systems for model training, evaluation, and production serving.
Build reliable, scalable infrastructure and tooling for ML workloads.
Improve GPU workload scheduling, monitoring, debugging, and resource efficiency.
Translate researcher and product needs into robust platform capabilities.
Requirements
Build and operate reliable, scalable, maintainable production systems at scale.
Work hands-on with cloud infrastructure, Linux, and infrastructure automation.
Operate Kubernetes-based distributed workloads in complex production environments.
Write strong Python code for backend systems and internal tooling.
MLOPS COMMUNITY
What an Anthropic Engineer Thinks About MCP
MCP’s biggest protocol change removes persistent server sessions, clearing a major scaling obstacle for deployments handling millions of requests.
Stateless transport lets load balancers move requests between servers without synchronizing session data.
Multi-round-trip requests carry required context across tool calls while preserving agent state.
SDK updates should keep migrations small, while tool search and programmatic calling reduce context bloat.
The release shifts MCP closer to standard web infrastructure without stripping away the state agents still need.
AI Hype vs. Real Value
AI projects can look productive until token and compute bills erase the gains. The discussion argues that real value comes from redesigning complete workflows, rather than scattering pilots across the business.
Compare AI-assisted teams with control groups and measure changes in speed, cost, revenue, or output.
Match models to tasks, limit token use, and apply FinOps at workflow level.
Keep humans involved in sensitive decisions, then reduce oversight as accuracy improves.
AI pays off when workflow gains survive cost and risk checks.
How iFood Designed Rosie Around Latency
When dinner is late, a fast wrong answer can cost more than a slow correct one. iFood’s support agent is built around that production reality, treating latency as something to design for.
Async callbacks keep slow tools from blocking the conversation.
Rollback merges rapid-fire messages before the agent replies.
A small reviser model checks outgoing messages, while end-state evals measure whether the user’s problem was resolved.
The system stays responsive without assuming every dependency will be fast.
IN-PERSON EVENTS
San Francisco - July 30
Pune, India - August 1
New York - August 14
Toronto - September 10
San Francisco, Voice Agents Forum - September 16
VIRTUAL EVENTS
MCP Jam - August 6
MEME OF THE WEEK
ML CONFESSIONS
Buying Time
The support tickets all said the same thing. Requests hanging for twenty seconds, sometimes longer. I would open the dashboard, see p95 sitting under 500ms, and close it. The graph and the tickets could not both be right, and I decided it was the tickets.
I raised the client timeout. A week later I raised it again. I told the team it was almost certainly upstream, and I believed it, because the number I looked at every morning kept coming back clean.
The number came back clean because we wrote it only after inference returned. Every request that timed out or lost its worker died before it reached that line. I had a dashboard that could report on the requests that worked and nothing else.
We added a start event and a timeout counter. The next incident showed up on the graph while it was still happening.
Share your confession here.