Bielik 11B v2 Technical Report
Krzysztof Ociepa, Łukasz Flis, Remigiusz Kinas, Krzysztof Wróbel, Adrian Gwoździej · 2025
Read Bielik 11B v2 Technical Report by Krzysztof Ociepa, Łukasz Flis, Remigiusz Kinas, Krzysztof Wróbel, Adrian Gwoździej (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-02410.txt.
Description
Krzysztof Ociepa and four co-authors present a language model built for Polish, in this 2025 technical report. The model, Bielik 11B v2, starts from the Mistral 7B design and grows to eleven billion parameters by depth up-scaling, adding layers rather than widening them. It aims for top Polish performance while staying multilingual.
Two training ideas stand out. Weighted instruction cross-entropy loss grades training examples by quality, so good instructions count more. An adaptive learning rate shifts with context length as training proceeds. Across many benchmarks the model beat rivals two to six times its size and clearly passed other Polish specialists, from language understanding to hard reasoning. Quantization options let it run on varied hardware. The authors present it as a step forward for capable AI in less-represented languages.
Catalog record
| CREATOR | Krzysztof Ociepa, Łukasz Flis, Remigiusz Kinas, Krzysztof Wróbel, Adrian Gwoździej |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 118 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-02410