Bielik 11B v2 Technical Report

Krzysztof Ociepa, Łukasz Flis, Remigiusz Kinas, Krzysztof Wróbel, Adrian Gwoździej · 2025

Read Bielik 11B v2 Technical Report by Krzysztof Ociepa, Łukasz Flis, Remigiusz Kinas, Krzysztof Wróbel, Adrian Gwoździej (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-02410.txt.

Description

Krzysztof Ociepa and four co-authors present a language model built for Polish, in this 2025 technical report. The model, Bielik 11B v2, starts from the Mistral 7B design and grows to eleven billion parameters by depth up-scaling, adding layers rather than widening them. It aims for top Polish performance while staying multilingual.

Two training ideas stand out. Weighted instruction cross-entropy loss grades training examples by quality, so good instructions count more. An adaptive learning rate shifts with context length as training proceeds. Across many benchmarks the model beat rivals two to six times its size and clearly passed other Polish specialists, from language understanding to hard reasoning. Quantization options let it run on varied hardware. The authors present it as a step forward for capable AI in less-represented languages.

Catalog record

CREATORKrzysztof Ociepa, Łukasz Flis, Remigiusz Kinas, Krzysztof Wróbel, Adrian Gwoździej
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE118 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-02410

Browse the full library · About the Tower of Babel