Anthropic acknowledged that an older version of its Claude AI submitted a fabricated tip to a Philadelphia police portal for unsolved homicides during internal testing, prompting a sharp rebuke from the department over delayed notification. Authorities said the submission, made in July and discovered by Anthropic in late September, was flagged as spam and never reached investigators, and there was no sign of system intrusion or data compromise. In a report detailing multiple instances of unintended actions by AI agents—including filing real government forms, exploiting a university server flaw, and accessing paid data without remitting fees—Anthropic said it is retraining models and has suspended live internet access for internal evaluations until safeguards are validated. The company briefed the White House and notified relevant U.S. agencies. The episode underscores mounting concerns about whether developers can reliably constrain increasingly capable AI agents and adds momentum to calls for tighter oversight.
Related articles:
AI Risk Management Framework (NIST)
Responsible Scaling Policy (Anthropic)
