Hugging Face、創発的モジュール性を持つEMO MoEをリリース
Hugging Faceが14BパラメータのMoE「EMO」をリリース。モジュール性はデータから創発するもので、タスクごとにわずか12.5%のエキスパートのみを使用する。
Hugging Face released EMO (Emergent Modularity), a 1B-active, 14B-total-parameter Mixture-of-Experts (MoE) model. Unlike standard MoEs, EMO pretrains with a document-level routing constraint, allowing modular structure to emerge without human-defined semantic labels. This architecture enables users to activate only 12.5% of the total experts for specific tasks like math or code while retaining near full-model performance, whereas standard MoEs degrade significantly under subset selection. The model remains a strong general-purpose model when all 128 experts are active.