Anthropic's Safety Promise in Question: A Battle with Pentagon Over AI Red Lines
Anthropic, a company founded by former OpenAI employees concerned about AI's dangers, is reconsidering its core safety principles in the face of intense competition. The company is moving away from self-imposed guardrails that have guided its AI model development, opting for a non-binding safety framework that it claims can and will evolve.
In a blog post, Anthropic outlined its new policy, acknowledging that its two-year-old Responsible Scaling Policy had limitations that could hinder its ability to compete in a rapidly expanding AI market. This announcement is surprising, given that Anthropic has positioned itself as the AI company with a "soul," emphasizing its commitment to safety.
The timing is particularly intriguing, as it coincides with a significant dispute with the Pentagon over AI red lines. Defense Secretary Pete Hegseth issued an ultimatum to Anthropic CEO Dario Amodei, demanding the company rollback its AI safeguards or face the loss of a $200 million Pentagon contract and potential placement on a government blacklist.
Anthropic's new approach involves separating its safety plans from industry recommendations, marking a departure from its previous policy. The company's original safety principles aimed to encourage a 'race to the top' among AI developers, but it now suggests that this hasn't materialized. Anthropic's stance on AI-controlled weapons and mass domestic surveillance of American citizens remains unchanged, as it believes AI is not yet reliable enough for these applications.
AI researchers have praised Anthropic's stance on social media, expressing concerns about government surveillance. Despite its long-standing commitment to safety, including research on potential AI blackmail, Anthropic has faced increasing pressure and competition from both the government and rival companies. Jared Kaplan, the company's chief science officer, suggests that the shift was primarily driven by a focus on safety rather than competitive pressures.
The company's new 'Frontier Safety Roadmap' offers more flexibility, with public goals that will be openly evaluated. This change comes amidst a tense relationship with the Pentagon, as Hegseth plans to invoke the Defense Production Act, designating Anthropic as a supply chain risk if it doesn't comply with Pentagon demands.
