ERNIE 5.0 Technical Report
ERNIE Team, Baidu · 2026
Read ERNIE 5.0 Technical Report by ERNIE Team, Baidu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-04705.txt.
Description
Haifeng Wang and colleagues introduce ERNIE 5.0 in a 2026 technical report, a foundation model built to understand and generate text, image, video, and audio within one system. All modalities train from scratch under a unified prediction objective.
The architecture uses an ultra-sparse mixture-of-experts design, where routing between experts ignores modality boundaries. A novel elastic training scheme is central. Within one pretraining run, the model learns a family of smaller sub-models with different depths and capacities. Users can then trade performance against size and speed for constrained settings. The report also addresses scaling reinforcement learning for such models. Experiments showed strong, balanced performance across modalities at production scale.
Catalog record
| CREATOR | ERNIE Team, Baidu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 103 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-04705