Anthropic urged policymakers to consider a temporary global pause on advanced AI development, citing a trend toward systems that could eventually design their own successors. In a post detailing progress on its Claude model, the company said more than 80% of code merged into its codebase in May was authored by Claude and that the model now proposes experiments and steers research within defined bounds. While Anthropic framed the advances as steps toward recursive self-improvement, outside experts cautioned there was no evidence of a sudden capability jump. University College London’s Steven Murdoch criticized the company’s narrow definition of AI safety and flagged tension between its cautionary stance and a Financial Times report that Anthropic engineers are embedded at the NSA to support offensive cybersecurity using its Mythos model. The company previously announced—but withheld—Mythos over security concerns, drawing skepticism from some researchers. Anthropic also filed for an IPO that could value the firm at $1 trillion, underscoring the commercial stakes as it seeks to convene governments, researchers, and industry on AI risk.
Related articles:
– Pause Giant AI Experiments: An Open Letter
– The Bletchley Declaration on AI Safety
– The Claude 3 Model Family
