DigitalOcean tool exposes silent AI agent failures

DigitalOcean's own framing of the problem is blunt: a tool call can return a clean success code and still be wrong. The company's illustrative case is a breaking change to an internal refund CLI tool that silently switched units from dollars to cents, causing a support agent to issue refunds at 1% of the intended value while every log line reported success ^[1]. Signals is built to catch exactly that class of failure by watching live traffic on Managed Agents and the Inference Engine and sorting problem sessions into three buckets - interaction signals (a frustrated user), execution signals (an unreliable tool call), and environmental signals (an infrastructure fault) - without requiring teams to add any extra instrumentation ^[1]. On the tau-bench benchmark, DigitalOcean reports Signals reaches an 82% informativeness rate and a 1.52x efficiency gain over randomly sampling sessions for review ^[1], a concrete claim that the flagging is doing more than just surfacing noise.

The obvious fix for silent failures - run an LLM judge over every single agent session - breaks down at scale. DigitalOcean notes that raw agent trace volume can exceed 100 GB/s ^[1], which makes exhaustive AI grading of every interaction impractical and expensive. Signals instead relies on cheap, rule-based triage to narrow that flood down to a shortlist of sessions worth closer scrutiny, and only then converts flagged sessions into evaluation test sets with one click ^[2]. The more expensive step - an LLM judge actually scoring whether a proposed fix works - is reserved for Simulations, which replays a flagged scenario through the same harness and tools the agent uses in production before any change reaches a real user ^[1]. In DigitalOcean's internal testing, ten production failure scenarios were replayed end to end in under a minute, and the tooling works across agent frameworks including Claude Code, Codex, and OpenCode ^[2].

The launch lands against a backdrop of eroding enterprise confidence in autonomous agents. A VentureBeat Intelligence survey found that the share of enterprises willing to let agents make unreviewed production changes fell from 75% to 56% in a single month, and dropped even further among purchasing decision-makers, from 88% to 61% ^[5]. More tellingly, 61% of organizations that run pre-deployment evaluations still reported an agent that passed internal testing yet caused a customer-facing failure within the prior year, with 22% reporting multiple such incidents ^[5]- evidence that evaluation alone isn't catching what production traffic reveals. That same survey found only 29% of organizations use real-time automated quality checks as their primary monitoring method, while 36% still lean on simple trace logging that confirms a system is running without validating that it's right ^[5]. DigitalOcean's decision to keep a human in the loop for every proposed fix, rather than letting Simulations auto-approve changes, reflects that same note of caution ^[2].

Signals and Simulations don't stand alone - they sit on top of Managed Agents, which DigitalOcean launched in public preview on 2026-09-21 with Harness Runtime (per-agent microVMs) and Action Gateway, giving agents governed access to more than 16,000 tools across 500-plus providers ^[4]. Barely three weeks later, DigitalOcean used the same announcement window to unveil Agent Learning and Data Services, aimed at making deployed agents 'recursively self-improving' ^[6], which positions reliability detection as just one layer in a much larger product line covering an agent's entire post-launch lifecycle. That full-stack bundling strategy is also what independent commentary has actually responded to so far: AI industry voices praised the underlying Managed Agents platform's cost focus and its pause-when-idle, governed-tool operational pattern ^[3]- reactions to the broader infrastructure play rather than endorsements of Signals and Simulations specifically, since the latter launched too recently for independent reviewers to weigh in directly.