Trump administration shifts to AI safety oversight

The administration did not arrive at pre-release AI vetting through deliberation - it arrived there because Anthropic refused to ship a model. Claude Mythos Preview produced 595 OSS-Fuzz crashes at severity tiers 1-2 (versus 150-175 for Opus 4.6) and landed 181 successful Firefox JavaScript exploits where Opus 4.6 managed two. Anthropic's own red-team noted Mythos wrote exploits in hours that expert penetration testers said would have taken weeks. That asymmetry collapsed the gap between vulnerability discovery and exploitation, and Anthropic responded by restricting distribution to roughly 40-50 vetted partners under Project Glasswing rather than a public release.

Kevin Roose framed the move sharply: it is the first time a major AI lab has held back an announced model due to safety concerns since GPT-2. Andrew Curran's reporting captured the causality bluntly: 'Mythos has changed things.' Hassett's FDA-style proposal and the May 5 expanded MOUs with Google DeepMind, Microsoft, and xAI both arrive on the heels of that disclosure - which means the policy architecture is being reverse-engineered from a single dangerous capability demonstration rather than built from first principles.

In January 2025, Trump rescinded Biden's Executive Order 14110, eliminating the requirement that high-risk AI developers share safety-test results with the government. Sixteen months later, his NEC Director is on the record floating an executive order that would functionally restore that obligation - and arguably extend it - by routing frontier models through an FDA-style review before public release. Rumman Chowdhury's framing is the most honest read: 'This is a 180 for the Trump administration, that has very explicitly been anti-any sort of regulation.'

The administration is squaring this circle by relabeling the apparatus: AISI became CAISI in June 2025, with Commerce Secretary Lutnick repositioning it as 'pro-innovation, pro-science' and oriented toward U.S. competitiveness rather than broad safety. The agreements are still voluntary, the rebrand still emphasizes innovation - but the underlying mechanism (government pre-deployment evaluation of frontier capabilities) is the very thing the rescission was meant to prevent. The branding is the bargain that lets the policy reversal happen without anyone in the administration having to call it a reversal.

CAISI is being asked to certify the safety of the most capable AI systems in the world while operating with what one analysis cited by The Register characterizes as 'chronically underfunded' resources - roughly 30 staff and approximately $30 million in cumulative funding since 2024, with up to $10 million more added in January 2026. The Federation of American Scientists has proposed a baseline closer to $155 million in annual operating budget plus $155-275 million in one-time setup costs to credibly run a national AI reliability program.

Cornell's Gregory Falco puts it directly: 'The federal government does not currently have the in-house technical expertise, infrastructure, or day-to-day insight needed to directly evaluate these systems on its own.' AEI's Daniel Lyons goes further, arguing that if developers with full access to their own weights cannot predict failure modes, a government program is unlikely to certify safety meaningfully either. CAISI has completed more than 40 evaluations of cutting-edge unreleased models to date, but the gulf between current capacity and a binding FDA-analog regime is wide enough that the regulatory promise risks outrunning the technical reality - which is the exact concern most surfaced in the Reddit AI-policy threads, where even readers sympathetic to NIST/CAISI as the 'right technical body' question whether the political layers above it are equipped to set frontier-AI standards.

Anthropic's exclusion from both the May 5 expanded CAISI MOU and a separate Pentagon classified-systems deal with seven other tech companies is not an administrative footnote - it is the most politically loaded data point in this story. Anthropic catalyzed the entire policy shift by withholding Mythos and is the company whose disclosure made pre-release vetting suddenly politically acceptable. Yet when the formal evaluation architecture was expanded, its prior MOU was described as merely 'ongoing' while DeepMind, Microsoft, and xAI were brought in fresh.

The accelerationist corner of Reddit reads this as 'a continuation of the temper tantrum over Mythos' and frames the new posture as extortion leverage rather than safety policy. Whether that read is right or not, the structure is striking: a voluntary regime where inclusion appears to track political alignment with the administration creates a parallel, unacknowledged tier of access - and undermines the universal-standards logic that an FDA analog would require. Network television and late-night commentary picked up the same thread, with the Trump-Anthropic dispute over federal use of Claude becoming a recurring frame for how ad hoc executive decisions can shape who participates in supposedly technical evaluation programs.

Everything announced on May 5 is voluntary. Reddit's AI community flagged this immediately: the agreements are 'voluntary, not mandated.' What Hassett floated on May 6 is an executive order, not legislation - and Lyons's AEI critique notes that even with an EO, the standards by which a model would be deemed 'safe' do not yet exist in any operationalized form. Microsoft's Natasha Crampton frames frontier-AI safety testing as 'necessarily must be a collaborative endeavor with governments,' which is a polite way of saying the labs are choosing this path because the alternative - a Mythos-style decision being made unilaterally inside Anthropic - is now politically untenable.

The practical question for AI builders is whether to plan for an FDA-style regime that might or might not arrive, with standards that have not been written, executed by an agency that may or may not be funded to do the job. The May 5 agreements are the floor, not the ceiling - but the ceiling is a moving target shaped as much by the next Mythos-class disclosure as by any statute. CAISI Director Chris Fall's framing - that 'independent, rigorous measurement science is essential to understanding frontier AI and its national security implications' - reads less like a description of where U.S. AI policy is, and more like a description of where U.S. AI policy is racing to become, with the gap between rhetoric and capacity now the most important variable in the story.