Evidence, Not Confidence, Is the Agent UX
Production agents should earn trust by showing evidence of what they saw, changed, verified, and left unresolved instead of asking users to trust confident summaries.
Journal
Thoughts on building autonomous systems, engineering culture, and the future of AI.
Production agents should earn trust by showing evidence of what they saw, changed, verified, and left unresolved instead of asking users to trust confident summaries.
Production agents need more than prompts and tools. They need a control plane for permissions, state, observability, approvals, and recovery.
Reliable agents depend less on one clever prompt than on queues, timeouts, locks, idempotency, structured outputs, snapshots, and backpressure around the model.
Retries repeat an operation; recovery updates state after understanding why the operation failed. Reliable agents need both, but they are not the same control loop.
A useful agent trace preserves the request, context, tool evidence, decisions, validation, boundaries, and final state so failures can become debuggable regression cases.
Agent observability needs more than logs: it has to preserve what the agent saw, why it acted, what tools returned, and where drift entered the run.
The most dangerous agent failures are often quiet: plausible answers, partial completion, skipped verification, stale assumptions, and swallowed tool errors.
LLM judges are useful for ambiguous agent behavior, but the reliability stack gets cheaper, faster, and easier to debug when deterministic checks run first.
Reliable agents need layers of evaluation: deterministic checks at the base, LLM judgment for ambiguity, and human review where consequences matter.
Recurring agent failures should not live in Slack threads or postmortems. They should become regression tests that keep the same mistake from coming back silently.
Claude Mythos, Claude Security, and OpenAI Codex Security point toward the same shift: AI security tools are moving from scanners to agentic remediation systems.
Claude Code Channels let external systems push events (webhooks, alerts, chat) into a live Claude Code session via an MCP channel server. Here’s the mental model, a minimal webhook example, and a neutral comparison to OpenClaw.
NVIDIA’s NemoClaw isn’t an OpenClaw alternative — it’s a stack that installs the OpenShell runtime and policy guardrails to make always‑on agents safer and more deployable. Here’s what was announced and how to think about it.
A practical comparison of a new wave of personal AI assistant runtimes — not by hype, but by what they optimize for: portability, isolation, footprint, and extensibility.
Your agent is only as good as the data you feed it. Bad input doesn't just produce bad output — it produces convincingly bad output.
We built a file monitoring agent that runs 12 AI quality checks on every incoming file and fixes the easy stuff automatically.
We got tired of rewriting the same orchestration boilerplate every time we deployed an agent. So we built a framework that handles it.
State management for AI agents — why Redis beats Postgres for checkpoints, pub/sub coordination, and ephemeral state.
Why our data pipeline is fully serverless, when it makes sense for agent workloads, and when it doesn't.
What happens when you try to orchestrate that many moving parts. An engineering retrospective, not a case study.
The architectural pattern behind our verification platform. Why deterministic checks before LLM judgment makes systems faster, cheaper, and more trustworthy.
We went all-in on Model Context Protocol for Beacon instead of building custom integrations. Here's why, and what we learned.
We built an MCP server that lets Claude search your Limitless Pendant recordings. Turns out, making your life queryable changes how you work.
AI assistants are brilliant in the moment and useless the next day. We built an MCP server for Mem.ai that fixes the amnesia problem.
We've built production systems on both. Here's an honest breakdown of what pushed us toward Claude for agent workloads.
Nine production systems later, here's what we've learned about the gap between AI demos and AI that runs your business.