DPO가 텍스트 퇴화 문제를 해결했습니다 — 왜 어디에나 활용하지 않을까요?

huggingface introduced a methodology using direct preference optimization to mitigate text degeneration in ocr models, reporting a 59.4% average reduction in repetition loops across tested model families, with a peak reduction of 87.6%. unlike standard dpo applications for chat alignment, this approach treats objective task failures—such as degeneration loops versus correct transcriptions—as binary preference signals derived from the model's own outputs. this technique addresses a structural limitation in supervised fine-tuning, which optimizes token by token and often fails to penalize sequence-level errors, leaving a ceiling on reliability. by training on full completions, dpo explicitly labels degenerated outputs as incorrect, preventing the model from entering high-probability attractor regions that cause repetition. this research suggests dpo can serve as a broad tool for improving robustness in objective tasks beyond conversational ai.