OpenAI CEO Sam Altman announced that OpenAI has signed an agreement with the National AI Safety Lab, a federal government agency that aims to assess and address risks in AI platforms, to provide early access to the next major generative AI model for safety testing.
Altman’s announcement, posted late Thursday evening on X, was short on details, but it appeared to counter narratives that OpenAI had deprioritized its work on AI safety in pursuit of more capable and robust generative AI technologies, along with a similar deal it struck with the U.K.’s AI Safety Authority in June.
In May, OpenAI effectively disbanded the division that was working on developing controls to prevent “superintelligent” AI systems from going rogue. According to reports, including ours, OpenAI set aside the team’s safety research to focus on new products, which ultimately led to the resignations of the team’s two co-leaders, Jan Leike (who now leads safety research at AI startup Anthropic) and OpenAI co-founder Ilya Sutskever (who started his own safety-focused AI company, Safe Superintelligence Inc.).
As criticism grew, OpenAI said it would drop the restrictive disparagement clause that implicitly discouraged whistleblowing, create a safety committee, and dedicate 20 percent of its compute to safety research. (The disbanded safety team had promised OpenAI 20 percent of its compute to its work, but ultimately never got it.) Altman reiterated the 20 percent pledge and reaffirmed that OpenAI eliminated the disparagement clause for new and existing employees in May.
But the move did little to appease some observers, especially after OpenAI formed a safety committee comprised entirely of company insiders, including Altman, and recently reassigned its chief AI safety officer to another organization.
Five senators, including Hawaii Democrat Brian Schatz, recently sent a letter to Altman questioning OpenAI’s policies. Jason Kwon, OpenAI’s chief strategy officer, responded to the letter today, writing that OpenAI is “committed to implementing rigorous safety protocols at every step of the way.”
The timing of OpenAI’s agreement with the U.S. AI Safety Institute seems a bit suspicious given that the company earlier this week supported the Future Innovation Act, a Senate bill authorizing the Safety Institute as an administrative agency to set standards and guidelines for AI models. The move could be perceived as an attempt to dominate regulation, or at least as an attempt by OpenAI to influence AI policy decisions at the federal level.
Altman sits on the Department of Homeland Security’s Artificial Intelligence Safety and Security Commission, which provides recommendations for “safe and secure AI development and deployment” across America’s critical infrastructure. And OpenAI has dramatically increased its spending on federal lobbying this year, spending $800,000 in the first half of 2024, compared to $260,000 in all of 2023.
The U.S. AI Safety Lab, part of the Commerce Department’s National Institute of Standards and Technology, consults with Anthropic and a consortium of companies including Google, Microsoft, Meta, Apple, Amazon, and Nvidia. The industry group is working on measures outlined in President Joe Biden’s October AI executive order, including developing guidelines for AI red teams, capability assessments, risk management, safety and security, and synthetic content watermarking.
