Sabiá-4 Technical Report

Thiago Laitz, Thales Sales Almeida, Hugo Abonizio, Roseval Malaquias Junior, Giovana Kerche Bonás, Marcos Piau, Celio Larcher, Ramon Pires, Rodrigo Nogueira · 2026

Read Sabiá-4 Technical Report by Thiago Laitz, Thales Sales Almeida, Hugo Abonizio, Roseval Malaquias Junior, Giovana Kerche Bonás, Marcos Piau, Celio Larcher, Ramon Pires, Rodrigo Nogueira (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-10213.txt.

Introduction

Laitz and coauthors present Sabiá-4 and Sabiazinho-4, a new set of Portuguese language models centred on Brazilian Portuguese. Training runs in four stages: more pre training on Portuguese corpora and Brazilian legal texts, long context growth, fine tuning on taught data, and preference alignment. Tests span chat, Brazilian law, long context reading, taught task following, exams and agentic tool use. The pair offer a good cost to score balance against rival models.

Catalog record

CREATORThiago Laitz, Thales Sales Almeida, Hugo Abonizio, Roseval Malaquias Junior, Giovana Kerche Bonás, Marcos Piau, Celio Larcher, Ramon Pires, Rodrigo Nogueira
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE23352
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.10213

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Sabiá-4 Technical Report Thiago Laitz, Thales Sales Almeida, Hugo Abonizio, Roseval Malaquias Junior, Giovana Kerche Bonás, Marcos Piau, Celio Larcher, Ramon Pires, Rodrigo Nogueira Abstract This technical report presents Sabiá-4 and Sabiazinho-4, a new generation of Portuguese language models with a focus on Brazilian Portuguese language. The models were developed through a four-stage training pipeline: continued pre-training on Portuguese and Brazilian legal corpora, long-context extension to 128K tokens, supervised fine-tuning on instruction data spanning chat, code, legal tasks, and function calling, and preference alignment. We evaluate the models on six benchmark categories: conversational capabilities in Brazilian Portuguese, knowledge of Brazilian legislation, long-context understanding, instruction following, standardized exams, and agentic capabilities including tool use and web navigation. Results show that Sabiá-4 and Sabiazinho-4 achieve a favorable cost-performance trade-off compared to other models, positioning them in the upper-left region of the pricing-accuracy chart. The models show improvements over previous generations in legal document drafting, multi-turn dialogue quality, and agentic task completion. 1 Introduction This technical report introduces the new generation of language models: Sabiá-4 and Sabiazinho-4. Designed with a focus on cost-effectiveness and high performance in complex tasks, these models represent a significant advancement over previous versions [ 14 , 3 , 2 ] . We report improvements in the legal domain, including greater accuracy in drafting legal documents and judicial decisions. The models also demonstrate enhanced capabilities in handling long documents, following instructions, and agent-like functionalities. These advancements expand their potential for use in structured workflows such as retrieval-augmented generation (RAG), making them more versatile and efficient for real-world applications. Similar to our previous generations of models, we applied continued learning in a generalist model to expand its capabilities. For this, we leveraged four training phases: (i) continued pre-training, in which we train the model using our Portuguese corpus; (ii) context expansion, where we extend the model’s capabilities for long contexts; (iii) supervised fine-tuning, including a variety of domains and chat styles; and finally, (iv) preference alignment, where we adjust the model’s outputs to align with human preferences and help the model understand small nuances of the language while being more strict with required formats. Figure 1 illustrates the cost-performance trade-off across several state-of-the-art models. Sabiá-4 and Sabiazinho-4 consistently occupy favorable positions in the upper-left region of the chart, achieving competitive benchmark accuracy at a fraction of the cost of comparable alternatives. This makes them particularly attractive for production deployments where both quality and cost efficiency are critical. Figure 1: Total inference cost versus average benchmark accuracy. Optimal models appear in the upper-left quadrant (cheaper and better). Sabiá models highlighted in green. Currency conversion: BRL/USD = 5.4. In the following sections, we present in more detail the four stages we used to train the models, as well as all benchmarks and metrics used to assess their capabilities across different domains. 2 Methodology For training, we used Google Cloud TPUs v5p and v6e with JAX as the framework for distributed training. The training consisted of four stages, which are described in Figure 2 . During pre-training, we first adapted a general-purpose base model to Portuguese through continued learning on both general and legal domain corpora, followed by long-context training to extend the context window to 128k tokens. For post-training, we applied supervised fine-tuning on a diverse instruction dataset spanning chat, code, legal, instruction following, and function calling/agentic tasks, followed by a preference alignment stage. This approach is supported by recent research demonstrating that domain specialization through continued pretraining can effectively enhance model performance in targeted areas without requiring the massive computational resources typically associated with training large-scale models from scratch [ 9 ] . Such findings suggest that smaller, domain-specialized models can serve as a cost-effective alternative for achieving competitive performance in specific domains. Figure 2: Training pipeline overview. Pre-training consists of Portuguese continual learning on general and legal corpora, followed by long context training. Post-training includes supervised fine-tuning on diverse instruction data and preference alignment. 2.1 Pre-training We performed continued pre-training using a large-scale Portuguese corpus combined with a Brazilian legal corpus to improve the model’s understanding of the Brazilian legal domain. The inclusion of legal data during pre-training was essential for the model to achieve strong performance on legal tasks in the post-training stage. To maximize the quality of training data, we created a data processing pipeline that includes quality filtering, relevance scoring, and document rewriting to ensure the model can effectively extract useful information from the source documents, some examples in the literature are [ 11 , 5 , 4 , 13 , 1 , 17 ] . For long context training, we specifically curated data sources containing naturally long documents to enable extended context capabilities, allowing the model to achieve 128K tokens of context. 2.2 Post-training The post-training stage consists of supervised fine-tuning (SFT) followed by preference alignment. During SFT, we trained the model to follow the chat template, handle function calling, and improve instruction-following capabilities. Previous model generations [ 14 , 3 , 2 ] exhibited limitations in following instructions and handling zero-shot scenarios, which motivated several data collection efforts. To achieve strong performance on agentic tasks and function calling, we developed a synthetic data pipeline for generating diverse function call examples [ 12 , 16 ] . We also expanded our multi-turn conversation data, as previous models exhibited degraded quality in extended dialogues. For preference alignment, we focused on refining the model’s writing style, improving its ability to interpret subtle nuances in user writing, and enhancing its attention to fine-grained details in prompts. 3 Benchmarks In this section, we present the benchmarks used to evaluate the models. We grouped our evaluation datasets into six categories: conversational capabilities, knowledge of the Brazilian legal system, long-context understanding, instruction following, performance on standardized exams, and agentic capabilities. Table 1 provides an overview of all benchmarks, their descriptions, and the metrics used to compare models. Tables 2 and 3 compile results for both a range of models by size and cost ranges. For benchmarks with heterogeneous metrics (e.g., scores ranging from 0–10), we normalized the results to a 0–100 scale in the pricing versus performance analysis to ensure comparability across all evaluations. Benchmark Description Metric OAB Bench Legal Drafting (Attorney Style) Average score [0, 10] Magis Bench Legal Drafting (Judge Style) Average score [0, 10] Brazilian laws Knowledge of Brazilian law Accuracy (5 alternatives) Agentic capabilities Tool usage in four environments in Portuguese Pass^3 and success@1 Brazilian exams 13 exams (ENEM, CFC, Revalida, CPNU, OAB etc) Accuracy (4 and 5 alternatives) Portuguese Multi-IF Instruction-following capability Strict Average over 3 turns BRACEval Portuguese conversational abilities Win rate against GPT-4o Table 1: List of benchmarks used to evaluate model capabilities across different domains and tasks. Benchmark sabiazinho-4 gpt-oss-120b gpt-4.1-mini gpt-5-mini gemini-2.5-flash-lite OAB-Bench (Lawyer Evaluation) 7.02 6.01 5.50 6.37 6.25 Magis-Bench (Judge Evaluation) 4.50 3.62 3.67 4.47 4.25 Laws (Legal Knowledge) 85.0 52.3 57.0 68.2 72.1 Agent Capabilities (4 environments) 55.2 60.9 59.4 85.1 18.0 Multiple Choice Exams (13 exams) 81.0 77.0 81.0 84.6 76.2 Multi-IF PT (Instruction Following) 81.0 82.0 79.6 85.8 80.8 Braceval (Conversations) 66.0 55.8 32.7 56.3 50.9 Table 2: Comparison of models for the same price range: cost-effective models. Benchmark sabia-3.1 sabia-4 Qwen3 235b gpt-4.1 gpt-5.2 (instant) gpt-5.2 (high)

Read the complete text