When Logs Talk Back
Plus... agent guardrails, AI memory, and real-time feature consistency.
As if AI scams weren’t ugly enough.
HOT TAKE
Dodge-board
Logs are underrated because dashboards made us lazy.
If agents can reconstruct metrics and traces from logs, what would you rather debug with: Dashboards or Chat?
DASHBOARDS or CHAT
LAST WEEK’S TAKE
Go with the flow
Agents still working on that workflow-level trust.
HIDDEN GEMS
How LLM serving works in production, covering the systems choices behind latency, throughput, memory use, and cost.
Making the case for MCP as better than Skills for connecting LLMs to external services, while Skills work best for reusable instructions and context.
Better results from reasoning agents by giving them simple, predictable keyword search instead of complex retrieval APIs.
Turning recommendation retrieval into a served model, replacing separate indexing, filtering, and ranking systems with GPU-served layers.
PRODUCTION AI INFRASTRUCTURE
Real-time ML Is a Consistency Problem
Real-time ML usually sounds cleaner in diagrams than it feels in production.
The easy version is: events arrive, features update, models get fresh values, done. The harder version is keeping all of that consistent with the training data, cheap enough to run, and fast enough to serve when the feature is not just “latest value” but something like average transaction amount per user over the last 7 days.
That is where the feature layer starts to get messy.
If you compute those features in batch, you get correctness and repeatability, but freshness suffers. If you run a streaming job, you get fresher online values, but now you have to maintain state, handle window eviction, seed the stream with historical data, recover from data issues, and still generate point-in-time-correct training data. If the online path and the offline backfill path are built separately, they also have to keep agreeing as feature definitions change. That agreement is usually where production systems start to drift.
This is the unglamorous middle layer of production AI: not model architecture, not agent UX, but the machinery that decides whether the values going into the model are fresh, affordable, and historically correct.
One of Zipline’s posts surveys how different systems handle streaming features, comparing bring-your-own-compute approaches like Feast, batch-based systems like Snowflake, naive streaming approaches like Databricks declarative features, and Zipline’s own architecture.
For streaming features, Zipline drives both online serving and offline backfills from the same declarative feature definition. A 7-day average transaction amount per user can be served online while still being backfilled for arbitrary observation timestamps, so the training data matches what the system would have served at the time. Batch jobs seed and correct the streaming state, and the window state is compressed to a constant size relative to window length. Increasing the window size only increases the batch workload, and only on the first run. It does not grow the streaming state or the read-time payload, which is how Zipline says serving can stay under 10ms regardless of window size.
The batch computation post tackles the other side of the same problem. A naive rolling-window SQL job ends up scanning most of the same data every day. For a 5-day window, four of those five days are duplicated work on each run. Zipline’s engine breaks aggregations into reusable intermediate representations, such as count and sum for an average, then combines or subtracts those cached pieces to produce the right window. For training backfills it reuses partial aggregates for the middle of a window and only reads raw events at the head and tail, where exact timestamp accuracy matters.
Real-time ML is rarely just a streaming problem. It is a consistency problem. The online feature value, the historical backfill, the batch correction path, and the training set all need to tell the same story. When they do not, you get models trained on one version of the world and deployed against another.
For the mechanics, read Zipline’s posts on batch computation and approaches to streaming features.
💡Job of the week
Senior Software Engineer, AI Platform & Agents // Coalition (Austin, TX)
Coalition is hiring a senior engineer to build internal AI platform infrastructure for agents and RAG pipelines. The role focuses on SDKs, orchestration components, APIs, testing, observability, and production services used by engineering, data, and applied science teams.
Responsibilities
Build core services and SDKs for internal AI platform use.
Design orchestration components for agent and RAG workflows.
Work with data teams on retrieval and context pipelines.
Support engineers onboarding AI use cases onto the platform.
Requirements
Strong Python API and service development experience.
Experience building RAG pipelines and vector database systems.
Familiarity with multi-agent orchestration frameworks and patterns.
Production knowledge of scaling, observability, Docker, and Git.
MLOPS COMMUNITY
Logs Are All You Need: Rethinking Observability with AI Agents
What happens when production breaks and nobody wants to open another dashboard? This discussion argues that AI-native observability could replace manual telemetry hunting with agents that read logs, inspect code, and suggest what failed.
Logs over three pillars: Reconstructing metrics and traces from logs could simplify instrumentation.
Agents as investigators: Sandboxes, Git-backed memory, and repo access help agents connect errors to code changes.
Evals get messier: CLI tools, side effects, and external systems make agent testing harder than simple tool-call checks.
The sharp idea is that observability may become less about watching systems and more about interrogating them.
The Control-vs-Magic Spectrum Building Agents
A restaurant owner sends a WhatsApp message to pay staff, check cash flow, or handle an invoice. Behind that simple chat is an agent deciding when to act, when to ask for confirmation, and how much control the user needs.
Fintech agents need guardrails: Payments and credit require explicit approvals, authentication, and careful task boundaries.
Personalization comes from context: Order history, opening hours, cuisine, and account data shape each interaction.
Agent architecture grows from evals: Start simple, then split workflows when real usage shows failure points.
The useful lesson is that practical agents are less about magic and more about knowing exactly where not to use it.
Human Memory As The Perfect Template For AI Memory
An agent that remembers everything is not useful if it remembers the wrong things at the wrong time. This blog argues that AI memory needs to work more like human memory, using context, relevance, and constraints rather than treating retrieval as storage with better search.
Memory needs structure: Agents must handle multimodal inputs, semantic links, and partial recall.
Context changes the answer: Work, personal, team, and customer memories need different boundaries.
Constraints matter: Time, behavior, privacy, and changing preferences should shape decisions.
The core idea is that useful AI memory is less about storing more and more about knowing what matters now.
From Agent Demos to Production Reality
Why do agents that work in a demo break in production? Patrick Kelly walked through Code Mode and the Agent Voyager Project at last week’s Coding Agents Lunch & Learn - one for cutting agent token costs, the other for measuring what an agent did across a run. Sahil Kathpal followed with a lightning talk on running agents reliably for 45 to 90 minutes without supervision.
Read the short PDF write up here.
The next Coding Agents Lunch & Learn is tomorrow, June 12, at 5:00 PM BST / 4:00 PM UTC. Session 15 covers memory, context, and state management for long-running agents, with a live walkthrough of an open-source framework from the Agentic AI Foundation ecosystem.
Join the next session here.
IN PERSON EVENTS
VIRTUAL EVENTS
Reading Group: Securing the Agent: Vendor-Neutral, Multitenant Enterprise Retrieval and Tool Use - June 11
Coding Agents Lunch & Learn Session - June 12
MEME OF THE WEEK
ML CONFESSIONS
Creature Feature
I run a setup at home that classifies photos off the trail cameras on my parents’ farm. Foxes, badgers, the neighbour’s cat, the occasional deer. Fine-tuned a small model on a few thousand labelled crops.
It got very good very fast. High 90s on the held-out set within about two epochs, which for the amount of data I had and the state of my labelling felt generous.
I assumed I’d got lucky with an easy split and left it running for a couple of months, occasionally showing people the confusion matrix like it was a holiday photo.
The cameras stamp the date and time in the corner of every frame. Burned into the pixels. The fox shots clustered around dusk, the cat around mid-morning when it came over for a wander, the deer early. The model wasn’t learning animals. It was reading the clock and guessing who tends to show up when.
I only worked it out because a fox went through at two in the afternoon and the model called it a badger.
Share your confession here.