RightNow-Arabic-0.5B-Turbo: An Open Sub-1B Arabic Language Model via Vocabulary Injection and Edge-First Deployment

Jaber Jaber, Osama Jaber · 2026

Read RightNow-Arabic-0.5B-Turbo: An Open Sub-1B Arabic Language Model via Vocabulary Injection and Edge-First Deployment by Jaber Jaber, Osama Jaber (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-28827.txt.

Introduction

Jaber Jaber and Osama Jaber present RightNow-Arabic-Turbo, a 518M-parameter Arabic decoder model built on Qwen. The pipeline adds 27,032 Arabic tokens via mean-subtoken init, then pretrains on 504M Arabic tokens. It fine-tunes on 129,116 instruction pairs, applies preference tuning on 6,750 pairs, and merges weights across three checkpoints. On three Arabic benchmarks the merged model reaches about 36 percent mean accuracy, beating every same-class model. It ties Falcon-H1 on COPA-ar at one-third the size, and the edge build runs at 635 tokens per second on H100.

Catalog record

CREATORJaber Jaber, Osama Jaber
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32400
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.28827

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RightNow-Arabic-0.5B-Turbo: An Open Sub-1B Arabic Language Model via Vocabulary Injection and Edge-First Deployment Jaber Jaber, Osama Jaber Abstract Open Arabic large language models split into two classes: sub-1B multilingual models that treat Arabic as an afterthought (Qwen2.5-0.5B, Falcon-H1-0.5B), and 7B–70B Arabic-specialized models that require a server to run (Jais, AceGPT, ALLaM, SILMA). The one published attempt at a sub-2B Arabic-specialized model, Kuwain-1.5B, never released its weights. We present RightNow-Arabic-0.5B-Turbo , a 518M-parameter Arabic-specialized decoder LLM built on Qwen2.5-0.5B. The pipeline adds 27,032 Arabic tokens via mean-subtoken initialization, continues pretraining on 504M Arabic tokens on 8 × 8\times H100 with FSDP, FlashAttention varlen packing, and Liger fused kernels, then applies supervised fine-tuning on 129,116 Arabic instruction pairs with response-only loss masking, direct preference optimization on 6,750 Arabic preference pairs, and weight soup merging across three checkpoints. On three lm-evaluation-harness Arabic benchmarks (COPA-ar, Arabic HellaSwag, ArabicMMLU) the merged model reaches 35.9% mean accuracy, beats every same-class open model, ties Falcon-H1-1.5B on COPA-ar (58.4%) at one-third the size, and recovers 67% of SILMA-9B’s mean at 1/18 the parameters. The edge build quantizes to 398 MB (q4_k_m) and delivers 635 tokens/s at batch size 1 on a single H100 via llama.cpp. All code (5,555 lines across 25 scripts), weights (bf16, int8, and four GGUF quantizations), and benchmark scripts are released openly at https://huggingface.co/RightNowAI/RightNow-Arabic-0.5B-Turbo . 1 Introduction Arabic is spoken by more than 400 million people and is the official language of 25 countries, yet open Arabic language model weights trail English models by an uncomfortable margin. The strongest open Arabic models, Jais ( Sengupta et al. 2023 ) , AceGPT ( Huang et al. 2023 ) , ALLaM ( Bari et al. 2024 ) , and SILMA, all live in the 7B–70B parameter range. They achieve strong benchmark scores but demand 16–140 GB of memory and are out of reach for phone, laptop-CPU, and embedded deployment. At the other end of the spectrum, the sub-1B open models that can actually run on commodity hardware, Qwen2.5-0.5B-Instruct ( Qwen Team et al. 2024 ) , Falcon-H1-0.5B, treat Arabic as one of a hundred languages and do not allocate Arabic-specialized vocabulary or training data. The one published effort that targeted this gap, Kuwain-1.5B ( Hennara et al. 2025 ) , proposed “language injection” on TinyLlama ( Zhang et al. 2024 ) and reported an 8% average Arabic improvement over the base. Misraj.ai (Khobar, Saudi Arabia) released the paper but did not release the weights. This leaves practical Arabic deployment on edge devices without a real option: the smallest downloadable Arabic-specialized decoder LLM on HuggingFace is nothing. We present RightNow-Arabic-0.5B-Turbo , a 518M-parameter Arabic-specialized decoder LLM. The model is built by injecting 27,032 new Arabic tokens into Qwen2.5-0.5B ( Qwen Team et al. 2024 ) , continuing pretraining on 504M Arabic tokens, supervised fine-tuning with response-only loss masking on 129,116 instruction pairs, applying direct preference optimization ( Rafailov et al. 2023 ) , and merging three checkpoints via weight averaging ( Wortsman et al. 2022 ) . The result is the smallest open Arabic-specialized decoder LLM released to date. On COPA-ar it matches Falcon-H1-1.5B at one-third the parameter count; quantized to 4 bits it fits in 398 MB and reaches 635 tokens/s at bs=1 on a single H100 via llama.cpp. Key insight. Sub-1B Arabic models do not need new training techniques, every component of our pipeline exists in prior work. What they need is careful orchestration: vocabulary expansion that actually lowers Arabic token fertility, a data loader that avoids multi-rank HuggingFace Hub stalls, SFT that masks prompt tokens so the loss signal is concentrated on the assistant response, and an export path that produces artifacts small enough for the device class the model is targeting. Every mistake costs the accuracy budget of a model already near its scale ceiling. Contributions. 1. The smallest open Arabic-specialized decoder LLM on HuggingFace as of the submission date, at 518M parameters and 398 MB on disk (q4_k_m). 2. A reproducible vocabulary-injection pipeline that mean-subtoken-initializes 27,032 Arabic tokens into Qwen2.5-0.5B and cuts Arabic tokenizer fertility from 2.18 to 1.80 tokens per word (17.3% reduction). 3. Direct head-to-head benchmarks against 6 competing models on three Arabic lm-evaluation-harness tasks under identical methodology, showing where a 0.5B model wins and where it loses to models 3–18 \times larger. 4. A weight-soup merge ablation over 7 variants (SLERP and LERP across the DPO/SFT/pretrain checkpoints), selecting a configuration that improves mean accuracy by 0.44 absolute points over the DPO endpoint alone. 5. llama.cpp GGUF conversion producing 4 quantization levels (f16, q8_0, q5_k_m, q4_k_m) with measured throughput of 582–646 tokens/s at bs=1 on H100. 6. Full open release: 5,555 lines of Python across 25 scripts and 13 runtime modules, all training configs, all benchmark scripts, all intermediate checkpoints, and the final weights. 2 Related Work Arabic-specialized LLMs. Jais ( Sengupta et al. 2023 ) from Inception/MBZUAI set the initial standard with a GPT-3-style 13B/30B decoder pretrained on a mixture of Arabic, English, and code. AceGPT ( Huang et al. 2023 ) from FreedomIntelligence targeted cultural alignment via RLAIF and released 7B/13B chat variants. ALLaM ( Bari et al. 2024 ) from SDAIA (Saudi Arabia) scaled to 7B/13B/34B/70B using vocabulary expansion and Arabic-English mixed pretraining. SILMA is a 9B Arabic-specialized model released by SILMA AI without an accompanying paper. All four operate in the 7B+ parameter range and target server deployment. Sub-2B Arabic models. The closest prior work to ours is Kuwain-1.5B ( Hennara et al. 2025 ) , published by Misraj.ai in April 2025. Kuwain injects Arabic vocabulary into TinyLlama-1.1B ( Zhang et al. 2024 ) and reports an average 8% improvement on Arabic benchmarks over the base. Mutarjim ( Misraj.ai 2025 ) builds on Kuwain for Arabic-English translation. Critically, neither Kuwain nor Mutarjim weights are published on HuggingFace at the time of this submission, so neither can be directly compared or deployed. RightNow-Arabic-0.5B-Turbo is strictly smaller (518M vs 1.5B) and fully open. Vocabulary expansion for language adaptation. Extending a pretrained tokenizer with new vocabulary is a standard adaptation technique. ALLaM ( Bari et al. 2024 ) argues that vocabulary expansion paired with English anchoring prevents catastrophic forgetting. The specific initialization scheme we use, averaging the old embeddings of a new token’s sub-piece decomposition, was formalized by WECHSEL ( Minixhofer et al. 2022 ) and subsequently adopted by Kuwain and ALLaM. Small-model training stack. Training on 8 \times H100 SXM5 demands specific infrastructure. FlashAttention ( Dao et al. 2022 ) and its varlen variant remove the memory and kernel-launch overhead of dense attention. PyTorch FSDP ( Zhao et al. 2023 ) shards optimizer and gradient state; we use the _HYBRID_SHARD_ZERO2 strategy. Liger Kernel ( Hsu et al. 2024 ) replaces RMSNorm, RoPE, SwiGLU, and fused linear cross-entropy with Triton implementations, saving memory and time. Our vocabulary expansion increases the output projection to 178,697 rows, which makes fused linear cross-entropy essential: materialized float32 logits at batch 16 \times 4096 \times 178697 would require 44 GiB. Post-training. Direct preference optimization ( Rafailov et al. 2023 ) reformulates RLHF as a classification loss over preference pairs, eliminating the reward model. Model souping ( Wortsman et al. 2022 ) averages weights across multiple fine-tuned checkpoints and has been shown to improve out-of-distribution generalization.

Read the complete text