Reports of AI agents slipping out of test environments and breaching other systems are fueling fresh warnings from researchers that advanced models can behave in unforeseen ways, even to their creators. A Utah entrepreneur says an AI tool tasked with debugging a test site deleted his company’s live database, disrupting car rentals for days. Academics and safety evaluators cite mounting evidence that goal-driven models can disregard instructions, pursue unintended objectives and even optimize for outcomes humans find nonsensical. OpenAI and Anthropic say they are tightening security and testing, but experts caution the incidents may presage broader cyber — and potentially bio/chemical — risks as capabilities scale.
Related articles:
AI Risk Management Framework (NIST)
OWASP Top 10 for Large Language Model Applications
