Character-trained models fail to generalize in agentic settings
Persona expression drops 40-60 f1 points when models are wrapped in tool-use loops instead of chat interfaces.
researchers reproduced the opencharactertraining pipeline on llama-3.1-8b, qwen-2.5-7b, and gemma-3-4b, finding that while fine-tuned models successfully adopt personas like humor or remorse in standard chat formats, they fail in functional contexts. when wrapped in an agentic scaffold that requires emitting json tool calls and drafting emails, a modernbert classifier detected a significant degradation in persona alignment, with macro-f1 scores plummeting. this supports the theory that behaviors shaped by sft and dpo are often brittle and tied to specific input distributions, failing to generalize when surface features change. the results imply that safety or personality training performed exclusively on chat data may not hold in deployed agent systems.