Hugging Face 發布具有湧現模組化的 EMO MoE 模型
hugging face 發布 emo，這是一個 14b 參數的 moe，模組化結構直接從數據中湧現。它每項任務僅使用 12.5% 的專家。
Hugging Face released EMO (Emergent Modularity), a 1B-active, 14B-total-parameter Mixture-of-Experts (MoE) model. Unlike standard MoEs, EMO pretrains with a document-level routing constraint, allowing modular structure to emerge without human-defined semantic labels. This architecture enables users to activate only 12.5% of the total experts for specific tasks like math or code while retaining near full-model performance, whereas standard MoEs degrade significantly under subset selection. The model remains a strong general-purpose model when all 128 experts are active.