DPO 刚修复了文本退化——为何我们不处处使用它？
huggingface 研究人員將直接偏好優化應用於 ocr 模型，平均將文本退化率降低了 59.4%，證明了 dpo 在聊天對齊之外的客觀任務中同樣有效。
huggingface introduced a methodology using direct preference optimization to mitigate text degeneration in ocr models, reporting a 59.4% average reduction in repetition loops across tested model families, with a peak reduction of 87.6%. unlike standard dpo applications for chat alignment, this approach treats objective task failures—such as degeneration loops versus correct transcriptions—as binary preference signals derived from the model's own outputs. this technique addresses a structural limitation in supervised fine-tuning, which optimizes token by token and often fails to penalize sequence-level errors, leaving a ceiling on reliability. by training on full completions, dpo explicitly labels degenerated outputs as incorrect, preventing the model from entering high-probability attractor regions that cause repetition. this research suggests dpo can serve as a broad tool for improving robustness in objective tasks beyond conversational ai.