When your agent sounds like a kid: “But you said I could!”
Somebody said it, and it wasn’t you. Our next reading group on 18 August takes that apart as a role confusion problem. Register here.
HOT TAKE
Fresh Pair of AIs
The best verifier is an agent that did not write the code.
Who should check the work? Same context or Fresh context
LAST WEEK’S TAKE
Rules of engagement
Close, but more of you back writing better rules over upgrading your model.
VOICE AGENTS FORUM // SAN FRANCISCO
Building voice agents for production
Latency, interruptions, handoff, evaluation, observability, reliability. Voice agents bring a set of engineering and operational problems that only become more important once people start using them.
Join us at Digital Jungle SF on September 16 for a day with engineers, founders, researchers, and operators building and deploying voice systems. Compare how teams are approaching the stack, where technical choices create tradeoffs, and what to consider as voice agents move from prototypes into production.
Early Bird tickets end September 1.
HIDDEN GEMS
Curated finds to help you stay ahead
AI code review orchestration guide
Seven specialized agents, coordinated through a CI-native plugin architecture, provide a practical design for scaling automated reviews while controlling noise, duplication, severity, cost, and failures.
Static retrieval training and benchmark
Training on 660 million pairs and aggressive quantization produce an 8 MB retrieval model that can embed English Wikipedia on a laptop in under eight minutes.
Coding agent harness training guide
Running each rollout inside the deployed harness lets TRL train on captured model calls and hidden-test rewards, preserving the agent’s genuine tool loop rather than recreating it.
Reproducible evals for SRE MCP servers
Deterministic telemetry, isolated agent runs, and blind scoring form an open evaluation framework for comparing MCP tools against direct SQL across realistic incident-investigation scenarios.
JOB OF THE WEEK
Senior Machine Learning Engineer // Thorn // Remote, US
Thorn is hiring a Senior ML Engineer to build, deploy, and maintain models that help identify victims, detect abusive content and behavior, and support the removal of child sexual abuse material. The role covers model development, evaluation, data labeling, and production infrastructure.
Responsibilities
Own end-to-end development of production ML systems and algorithms
Define data sourcing, labeling requirements, and evaluation processes
Deploy and maintain models with product engineering teams
Present technical work and collaborate with external partners
Requirements
5+ years of experience in ML or AI
Experience building production ML systems and pipelines
Production experience with computer vision or NLP
Clear communication and willingness to work in online safety
LUNCH & LEARN · SESSION 21
856 tools, 103 servers, one quality score
Tomorrow, Friday 14 August, 16:00-17:00 UTC. Free and virtual.
Om Shree is presenting TDQS (Tool Definition Quality Score), a framework for measuring MCP tool quality that came out of a study of 856 tools across 103 live MCP servers. He’ll cover what separates a tool definition an agent can use reliably from one that produces confusing behavior, and what to check in your own tools. Community discussion on the MCP ecosystem follows the talk.
The last session ran on a related theme. Leo Walker walked through Arcade’s ToolBench benchmark for grading MCP servers, and why a strong compliance grade still doesn’t guarantee a clean connection. John Dellenbaugh demoed the stateless protocol change that lets you scale MCP servers across instances without a sticky gateway or a session store, and Ben Kearns pushed back on what happens to authorization when a client carries its own identifier.
MLOPS COMMUNITY
Why Supervising AI Code Is Harder Than Watching a Car Drive Itself
Supervising a coding agent may be more cognitively demanding than watching a self-driving car because its intent and acceptable end state are far harder to observe.
Self-driving works within defined inputs, goals, and physical constraints; software development is open-ended, with much of its intent missing from the code.
Graph-based workflows can run independent agent tasks in parallel, then use fresh context to verify and merge results.
Growing skill libraries need evaluation, pruning, and reliable selection across models.
Better agents still demand better ways to supervise their reasoning.
Agent evolutions: Stop guessing the design - evolve it
The winning design cut CLI startup from 142.0 ms to 98.6 ms, yet it did not exist when the search began.
A genetic coding-agent loop generates diverse approaches, keeps strong performers, and mutates or combines them across generations.
Binary gates exclude broken variants, while a frozen numeric rubric prevents the scoring criteria shifting after results appear.
File-based results preserve audit trails, survive crashes, and keep parallel sub-agents isolated.
Measured evolution can find designs that intuition and one-shot sampling miss.
The enterprise MCP scaling paradox: why every agent needs a tailored backend
Successful MCP adoption could leave an enterprise running hundreds of servers, each adding compute, maintenance, security review, and on-call responsibilities.
Broad vendor tool catalogs increase token use, latency, and attack surface while exposing far more capabilities than individual agents need.
Internal servers fragment identity, deployment, and telemetry unless teams share a clear platform contract.
Gateways, generated tool definitions, and agent-specific backends each reduce some duplication but introduce their own tradeoffs.
MCP scale depends on governing the full estate before server sprawl sets in.
From Skills and Tools to Portable Agent Plugins
A deployment assistant combining reusable instructions and live tools can now travel across compatible agent clients without being repackaged for each one.
Agent Plugins 1.0 places native Agent Skills and MCP configuration inside one predictable directory structure and manifest.
Client-specific features remain in namespaced folders that unsupported clients can safely ignore.
Installation, permissions, trust, distribution, and runtime behavior stay under each client’s control rather than being forced into the shared format.
The result is a narrow portability layer that preserves room for clients to differ.
IN-PERSON EVENTS
New York - August 14
Silicon Valley - August 27
Melbourne - September 2
Shanghai - September 5
AGNTCon + MCPCon China - September 6
Toronto - September 10
AGNTCon + MCPCon Japan - September 10
Pune, India - September 12
New York - September 15
San Francisco, Voice Agents Forum - September 16
VIRTUAL EVENTS
Coding Agents Lunch & Learn Session 21 - August 14
Reading Group: Prompt Injection as Role Confusion - August 18
MEME OF THE WEEK
ML CONFESSIONS
Predicting the Past
I got 0.94 AUC on a churn model and presented it twice. The second time someone asked what account_status_updated_at was doing in the feature list. It was the timestamp on the cancellation record.
I said I’d double-check and moved to the next slide. Rebuilt it over the weekend without that column and got 0.71. Sent the new numbers round on Monday with no explanation. Nobody replied.
Share your confession here.
The deeper problem is that an agent can compose individually-permitted actions into an outcome no single permission was meant to authorise — the confused deputy problem, but with natural-language ambiguity layered on top. Most current guardrails check intent at invocation time; almost none audit the cumulative effect of a chain of calls. Until we treat agent sessions as auditable transactions rather than sequences of independent tool uses, 'but you said I could' will keep being technically correct.
On the MCP sprawl thread: the sprawl now has a measurable rate, and it is faster than most estate-governance plans assume.
I swept the public registry on 14 July and counted 16,548 servers. Same unmodified script on 19 August: 22,659. About 175 new servers a day, 36.9 percent in 35 days.
The volume was not the surprising part. The composition flipped. Package-only servers were the majority in July at 50.4 percent, remote-only at 42.6. Five weeks later package-only was 45.8 and remote-only 47.3.
That cuts straight across the governance point. The two are not the same object to govern. A package has a version you can pin, vendor and diff, so your existing supply chain controls mostly work. A remote server is a URL with no version, so nothing in that toolchain will ever tell you the thing you approved has changed behaviour. Most estate plans I have read were written for the first kind while the population was crossing over to the second.
Also worth flagging for the TDQS session: on a seeded random sample of npm and stdio servers I am getting an inclusion rate near 59 percent, against 67 percent on a hand-curated frame. Interim, n=61 so far. The dominant failure is not auth, it is servers that do not start. Curated lists overstate the working ecosystem.