Mecellem Models: Turkish Models Trained from Scratch and Continually Pre-trained for the Legal Domain

Özgür Uğur, Mahmut Göksu, Mahmut Çimen, Musa Yılmaz, Esra Şavirdi, Alp Talha Demir, Rumeysa Güllüce, İclal Çetin, Ömer Can Sağbaş · 2026

Read Mecellem Models: Turkish Models Trained from Scratch and Continually Pre-trained for the Legal Domain by Özgür Uğur, Mahmut Göksu, Mahmut Çimen, Musa Yılmaz, Esra Şavirdi, Alp Talha Demir, Rumeysa Güllüce, İclal Çetin, Ömer Can Sağbaş (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-16018.txt.

Introduction

Özgür Uğur and a large team build language models for Turkish law in a 2026 arXiv paper. The models are the Mecellem family. Legal language is a specialized domain in any tongue. Turkish legal text adds its own terms and forms. The team trains models both from scratch and by continued pretraining. One contribution is an encoder model built from zero. It follows the ModernBERT design and trains on a Turkish-dominant corpus.

That corpus holds 112.7 billion tokens, per the abstract. The authors stress joint evaluation: pretraining choices and downstream task scores together. Domain adaptation often looks good on one axis and poor on the other. Legal tools for Turkish speakers need models that truly read their law. This work supplies candidates and a frank way to judge them.

Catalog record

CREATORÖzgür Uğur, Mahmut Göksu, Mahmut Çimen, Musa Yılmaz, Esra Şavirdi, Alp Talha Demir, Rumeysa Güllüce, İclal Çetin, Ömer Can Sağbaş
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE132 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-16018

Browse the full library · About the Tower of Babel