The Agent Boom
A growing share of software today is written by AI. Recent industry estimates suggest over 40% of new code now includes AI-generated or AI-assisted contributions, with some Y Combinator-backed startups using AI to generate up to 95% of their codebases. Tools such as GitHub Copilot have surpassed 20 million users, while AI-native IDEs and coding agents continue to proliferate across engineering teams.
At the same time, building the agents that generate this code has become dramatically easier, with frameworks like LangGraph, Vercel AI SDK, Claude Code, and OpenAI Agents SDK. At Arga Labs, we have shared our internal agent builder, Attractor Visual Builder, as an open-source feature. It implements StrongDM's Attractor framework and makes it easy to build agent harnesses without writing orchestration code.
The foundations of these systems trace back to research. Work like ReAct introduced the idea of language models that combine reasoning with actions, while Toolformer demonstrated that models could learn to call external APIs as part of their reasoning process. Modern frameworks have turned these ideas into practical engineering tools.
But while the infrastructure for building agents has matured quickly, the infrastructure for running them safely in production has not.
As agents increasingly write code, trigger workflows, and interact with real systems, a new question emerges:
How do we ensure their actions are actually correct, and safe?
Production Is Not a Demo
Agents often look impressive in demos. They operate on small problems, clean inputs, and limited environments. But real production systems behave very differently.
Modern software systems are large, distributed networks of services constantly generating noisy data, metrics, logs, and alerts. Engineers rely on observability platforms like Sentry, Prometheus, and Grafana just to understand what's happening inside them. In these environments, failures rarely have one singular primitive cause. Instead, problems propagate through chains of dependencies. A small issue in one service can trigger alerts—and sometimes cascading degradations—across many others.
An AI agent investigating an incident might follow a process in the following order:
- Detect anomaly
- Investigate logs
- Attempt to identify the root cause
- Apply fix
But if the agent misidentifies the root cause even slightly, which remains a common challenge for non-trivial failures, especially without strong observability and error-handling, fixes can make the system even worse or introduce new issues.
This complexity is well documented in distributed systems research. Work like Google's Dapper tracing system and the broader Site Reliability Engineering literature shows that modern infrastructure behaves as large dependency graphs where failures cascade across services.
The Core Failure Modes of AI Agents
Even when agents operate with strong models and well-designed workflows, they still fail in production for a few recurring reasons.
1. Hallucinated reasoning: Language models can generate confident but incorrect conclusions. Research on LLM reliability consistently shows that models often produce plausible answers that are factually wrong, particularly when solving complex or ambiguous problems.
2. Context drift across long workflows: Many agents perform multi-step reasoning. For example, investigating signals, gathering information, forming hypotheses, and then taking actions. As these reasoning chains grow longer, the probability of error increases, and the agent can lose track of earlier assumptions.
3. Unsafe execution: Modern agents are increasingly capable of editing code, triggering deployment pipelines, modifying infrastructure, or updating system configurations. Production systems already include strong guardrails like CI tests and permission controls to prevent unsafe changes from reaching users. The problem is that AI systems can now generate changes far faster than engineers can meaningfully validate. As a result, most AI-generated code never makes it to production because it still requires extensive human review to determine whether it is actually correct.
As agents generate more code and operate deeper inside production systems, the gap between the volume of AI-generated code and safely shipped features dramatically increases.
The Missing Layer: An Intelligent Validation Framework For AI Agents
Today's development stack assumes a human engineer is reviewing every change before it reaches production. But when agents are generating large volumes of code and infrastructure changes, manual review becomes the limiting factor.
What's missing is infrastructure that can systematically validate agent-generated actions before they affect production systems.
Instead of only being able to ship code, agents should produce a PR that runs through a validation pipeline. The system gathers relevant context like the code files, internal documentation, and operational signals from tools like Slack, Jira, Sentry, Grafana, or CloudWatch and runs checks against staging environments to verify the change behaves correctly.
The output will be a validated PR with a report showing what changed and why the system believes it is safe. Over time, as validation becomes reliable, smaller non-structural changes can be merged automatically while larger changes still surface for human review.
The shift is simple but fundamental. Today, AI generates changes and humans validate them. In the future, validation itself becomes an automated infrastructure.