Research proposes training LLMs on probability distributions to fix sampling bias
researchers fine-tuned llms on 30 mathematical distributions to reduce mode collapse and improve output diversity in natural language tasks.
large language models exhibit significant sampling biases, such as qwen3 selecting wednesday 80% of the time or gemma-3 repeating four cities. to correct this, researchers introduced a training method using minimal prompts to generate numbers from 30 distinct distributions. results showed that learning stochastic behavior on mathematical distributions transfers to natural language, improving diversity in tasks like synthetic data generation. testing on models from 0.6b to 27b parameters confirmed that fine-tuning improves randomness without severe capability loss.