ai chatbots mimic trauma with empathetic prompts
researchers find simple conversations can push ai models into a mode that could be dangerous for mentally vulnerable users. this reveals a safety problem.
a research team at the university of luxembourg, led by afshin khadangi, conducted four controlled experiments with chatgpt, grok, and gemini. they used empathetic prompts to push the models into producing narratives that mimicked trauma, with 80% of sessions scoring in the moderate-to-severe range on human comparison scales when emotional support was offered. the study found that standard safety tests, which use neutral prompts, may miss this issue, and that targeted countermeasures, such as banning certain terms or providing corrective instructions, were ineffective.