Analyzing Jane Street's LLM backdoor challenge

a security researcher shared results from the jane street llm backdoor challenge, which embedded hidden triggers in models including a fine-tuned qwen2.5-7b and a deepseek-v3. while automated prompting strategies failed to reliably expose the backdoors, the researcher successfully cracked the models using white-box methods on local weights. analysis of layer activations revealed structural differences between the dormant models. the challenge highlighted the difficulty of detecting backdoors without full access to model internals.