Hugging Face 发布具有涌现模块化的 EMO MoE 模型
hugging face 发布 emo，这是一个 14b 参数的 moe，模块化结构直接从数据中涌现。它每项任务仅使用 12.5% 的专家。
Hugging Face released EMO (Emergent Modularity), a 1B-active, 14B-total-parameter Mixture-of-Experts (MoE) model. Unlike standard MoEs, EMO pretrains with a document-level routing constraint, allowing modular structure to emerge without human-defined semantic labels. This architecture enables users to activate only 12.5% of the total experts for specific tasks like math or code while retaining near full-model performance, whereas standard MoEs degrade significantly under subset selection. The model remains a strong general-purpose model when all 128 experts are active.