Hugging Face introduces decentralized community evaluations
hugging face is decentralizing ai evals. dataset repos now host leaderboards via eval.yaml. users submit results via pull requests with .eval_results/*.yaml. verified badges ensure reproducibility. this creates a single source of truth for benchmarks like mmlu-pro and gpqa.
Hugging Face has launched a beta feature that decentralizes AI benchmark reporting. Dataset repositories can now register as benchmarks and host their own leaderboards using an `eval.yaml` specification. This standardization allows models to store evaluation results in `.eval_results/*.yaml` files, which then aggregate automatically to the dataset cards. Community members can submit results via pull requests, fostering transparency and reproducibility across platforms like MMLU-Pro, GPQA, and HLE. This initiative addresses the fragmentation of reported scores and the saturation of current leaderboards.