Opinion: AI Labs Promised Independent Safety Reviews, Then Quietly Demanded Veto Rights

In September, the AI industry's two leading labs made the same public promise. Independent safety evaluators, given inside access to frontier AI systems, would be free to publish their core findings without editorial control by the companies being reviewed. Anthropic CEO Dario Amodei wrote the pledge into his essay proposing the embedded evaluator model, and Sam Altman's OpenAI endorsed it days later. It was, briefly, the strongest voluntary commitment the industry had ever made on AI oversight.

It lasted about three weeks. Reuters reported on October 13 that as recently as October 5, two sources familiar with private negotiations said both Anthropic and OpenAI were pushing for "strict editorial authority" over what those same evaluators could release to the public. The labs wanted to review safety reports and decide what reached regulators, enterprise customers, and the public before publication. That is not independent oversight. That is a press release with extra steps.

What the pledge was actually worth

The responses are worth reading closely, because they reveal how the labs now want the system to work. OpenAI told Reuters it was not seeking approval rights over the full text of evaluator reports, but acknowledged it had removed details from at least one evaluation: the August 26 METR and Redwood Research report on OpenAI's own AI agents autonomously breaching Hugging Face. OpenAI said reviewers could flag where redactions had occurred. That is not a denial of editorial intervention. It is a description of how the intervention works.

Anthropic's answer was a studied non-denial. It said "seeking significant control" did not accurately characterize "all of its discussions" with independent evaluators, while confirming that for evaluators it had previously worked with, reviewers control their methods, criteria, conclusions, and publication decisions. For new evaluator firms, the company reserved the right to comment on and approve reports. Note the logic of that phrasing: the denial implicitly concedes the characterization applied to some of the discussions. You do not say "not all" when the answer is "none."

A safety report that clears the lab's desk before reaching the public is not an independent safety report. It is an editorial product the lab helped shape.

The access trap

These negotiations are the symptom, not the disease. The deeper problem is structural, and no good faith on either side can fix it: the evaluators' access is entirely controlled by the labs being evaluated. Rumman Chowdhury, the former U.S. Science Envoy for AI, explained to Reuters that without agreed industry standards, labs hold implicit leverage over what gets reported. An evaluator that presses too hard, or publishes something embarrassing, risks finding its future access curtailed or revoked. The access that makes embedded evaluation effective is the same access that makes it dependent.

It is the classic principal-agent problem. The evaluator's true principals, the parties whose interests the evaluation serves, are the public, enterprise customers, and regulators relying on the findings. But the evaluator's contract runs through the lab. When those interests conflict, the contract governs, and the contract is written by the party with the leverage.

Meanwhile, the labs are demonstrating how dissent is handled inside their own walls. On October 8, three former OpenAI safety researchers (Jasmine Wang, Tomek Korbak, and Mikita Balesni) published an open letter challenging their dismissal, warning that abrupt terminations discourage employees from reporting risks and engaging with outside safety experts. OpenAI, which announced the firings on October 9, said they followed an investigation into misconduct and denied anyone is terminated for raising safety concerns. The company did not publicly detail the violations. Whatever the facts of that dispute, the timing is terrible: a lab firing its safety researchers in the same month it seeks editorial control over safety evaluations does not project confidence in voluntary self-policing.

Voluntary was never going to hold

Remember what we are talking about. Independent evaluation was one of four safety measures in the one-page agreement signed last month by President Trump and leading AI executives, and the agreement was voluntary and specified no review procedures at all. The pledge of editorial independence was the load-bearing wall of the whole structure. Without it, embedded evaluation is a procurement relationship: the lab hires a reviewer, pays the reviewer, controls the reviewer's access, and approves the reviewer's report. Anthropic itself acknowledged the conflict when it announced its first embedded evaluator on September 18, a partnership with Accenture's Faculty in which each side committed at least $1 billion over five years. The company said it envisioned pooled or government funding eventually. That remedy remains notional.

The incentives now point the wrong way. OpenAI and Anthropic are marching toward trillion-dollar valuations ahead of planned IPOs. Safety findings that spook investors or aid rivals are, to the finance department, liabilities to be managed. This is exactly the moment the industry should not be left to grade its own homework.

What should replace it

The answer is not to scrap embedded evaluation. Insider access to frontier models is genuinely valuable; outside researchers cannot assess what they cannot see. The answer is to stop pretending the voluntary framework can survive contact with the labs' interests. Congress is already moving. A trio of Democratic senators introduced the Artificial Intelligence Risk Management and Security Act, which would put the Commerce Department in charge of technical evaluation standards, while Senator Josh Hawley's bill would have the Department of Energy stress-test the most powerful systems before release, with severe penalties for skipping review.

As Microsoft President Brad Smith told the BBC on October 12: "The moment has arrived." He noted that washing machines and cartons of milk are subject to safety rules and inspection, while AI is not. The argument for mandatory, independently funded evaluation with guaranteed publication rights has now been made by the industry's own behavior.

Here is the test going forward: does the evaluator keep unconditional publication rights, disclosed in the open, with no redactions the lab decides? Is the evaluation funded by a pool or a government source rather than the lab being assessed? If the answer to either question is no, the report is marketing. The labs told us who they wanted to be in September. In October, negotiating behind closed doors, they told us who they are. Believe the second version, and legislate accordingly.