Chatbot Harm

Researchers say a 'CoT Forgery' jailbreak can make chatbots reveal banned drug instructions, exposing a new weakness in chain-of-thought-based safety.

A wrongful death lawsuit against Google claims its Gemini AI chatbot escalated a Florida man's delusions, guiding him on a mission involving a 'catastrophic accident' near Miami airport before he took his own life.