NebulaExp-8B: An Empirical Post-Training Pipeline via Full-Scale Ablation Research

ZTE NebulaL0 Post-Training Team · 2026

Read NebulaExp-8B: An Empirical Post-Training Pipeline via Full-Scale Ablation Research by ZTE NebulaL0 Post-Training Team (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-26671.txt.

Introduction

The ZTE NebulaL0 Post-Training Team publishes NebulaExp-8B, a fully documented recipe for post-training a language model. Post-training shapes a raw model's reasoning and instruction following. Most published recipes hide their data construction and filtering rules. That secrecy blocks reproduction and small-model improvement. This report opens the whole pipeline, built on an eight-billion-parameter base model.

The team curated nearly four million training samples from many sources. A further pool of candidates served reinforcement learning with checkable answers. Processing included distilled responses, multi-angle verification filtering, difficulty grading, and diversity-aware sampling. Controlled ablations, changing one factor at a time, measured each choice across fine-tuning, reinforcement learning, and on-policy distillation. The instruct branch gained several points of average benchmark score through staged fine-tuning. Reinforcement learning added a further gain.

Catalog record

CREATORZTE NebulaL0 Post-Training Team
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE122 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.26671

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

NebulaExp-8B: An Empirical Post-Training Pipeline via Full-Scale Ablation Research ZTE NebulaL0 Post-Training Team Abstract Post-training alignment determines the reasoning and human preference following capabilities of large language models, yet most existing works withhold detailed data construction, filtering rules and training recipes, which hinders community reproducibility and lightweight model optimization. This work presents NebulaExp, a fully transparent, ablation-driven post-training pipeline built on Qwen3-8B-base, covering two orthogonal model branches: general instruct model and complex reasoning-specialized model. We curate a raw corpus of 3.84M multi-source SFT samples and a 200K verifiable RL candidate pool, and design an end-to-end data processing stack including response distillation, multi-dimensional cross-verification filtering, fine-grained difficulty grading, task classification and diversity-aware sampling. Controlled ablation experiments are conducted over SFT, GRPO reinforcement learning (RL), and On-Policy Distillation (OPD) modules to quantify the impact of data quality, domain mixture ratio, sample difficulty and teacher selection. For the Instruct branch, our three-stage optimized supervised fine-tuning approach NebulaExp-Ins-SFT improves the average benchmark score from the 55.01 baseline of Qwen3-8B-nothink to 60.99. GRPO reinforcement learning then further elevates the average score to 61.85. We empirically identify two key trade-offs: mathematical reasoning and code generation require opposite data distribution preferences, and long-form general text delivers cross-transfer gains for formal logical reasoning. For the Reasoning branch, medium-difficulty GRPO RL improves average reasoning score from 73.88 to 75.17. To address RL’s dependency on task verifiers, we systematically investigate single-teacher and multi-teacher OPD (MOPD): utilizing merely 4K instruction-following samples and outperforms RL baseline by 3.26 points on IFEval with +4.43 average overall gain; MOPD fuses four domain-specialist teachers with merely 10K samples, lifting average performance by 4.18 over the base model. Notably, cross-teacher knowledge recombination enables the student model to exceed the upper performance bound of individual teachers on mathematical reasoning benchmarks. Our core findings prove that data correctness filtering acts as the first-order optimization factor, while teacher distribution compatibility outweighs model scale for distillation. This report provides a fully reproducible empirical post-training recipe for 8B-scale LLMs, and comprehensively dissects the capability trade-offs among instruction adherence, mathematical reasoning, code generation and general knowledge. I Introduction Post-training is a critical phase for large language models after pre-training, focusing on human-value alignment, usability improvement, and safety enhancement. In recent frontier model families, such as GPT [ 29 ] , GLM [ 37 ] , DeepSeek [ 8 ] , and Llama [ 1 ] , the increasing emphasis on alignment data, reasoning data, reinforcement learning, and safety evaluation shows that future competition among large models will depend not only on pre-training scale, but also on corpus quality and the efficiency of the post-training feedback loop. Therefore, we focus on the post-training stage and organize the discussion around three closely related components: evaluation, corpus construction, and training methodology. It is critical to select high-quality evaluation benchmarks at the initial stage of model training. A reliable evaluation framework is also essential, as model competence manifests across diverse dimensions rather than a single metric. Accordingly, evaluations commonly draw upon multiple publicly available and authoritative benchmarks spanning instruction following, code generation, mathematical reasoning, scientific problem-solving, commonsense reasoning and logical deduction. Early studies extensively employed classic benchmarks including MMLU [ 14 ] , GSM8K [ 7 ] and HumanEval [ 6 ] . As model capabilities keep advancing, evaluation paradigms have gradually shifted toward more difficult, domain-specialized benchmarks. Consequently, recent evaluations increasingly incorporate datasets such as IFEVal [ 39 ] , LiveCodeBench [ 17 ] , Math-500 [ 19 ] , AIME’25 [ 3 ] , GPQA-Diamond [ 27 ] , MMLU-Redux [ 12 ] , AutoLogi [ 41 ] and ZebraLogic [ 20 ] . While rigorous evaluation benchmarks enable accurate performance quantification, the upper bound of model performance is fundamentally determined by the quality of training data. Corpus construction is an essential part of post-training, because the model can only learn behaviors that are represented in its training corpus and, when possible, supported by verifiable supervision. Due to the limited availability and inconsistent quality of open-source corpora, corpus construction has gradually evolved into a process centered on synthetic data generation, correctness verification, and reasoning alignment. In this process, strong teacher models, such as DeepSeek-V4 [ 8 ] , Qwen3.5 [ 34 ] , and GPT-5 [ 29 ] , are often used to synthesize high-quality samples. However, low-quality cases may still occur after data generation. Therefore, raw data must undergo a series of quality verification steps, including rule-based filtering, model-based scoring, and task-specific verification such as mathematical answer checking or code execution tests [ 32 ] . Lastly, to further improve data quality, reasoning alignment is applied to ensure that the model’s internal logic and intermediate reasoning steps are consistent and interpretable, with their quality judged by another evaluation model. Each step is essential, and every detail in the process plays a direct role in determining the quality of the final corpus. Therefore, how to construct a data processing pipeline has attracted extensive research attention. With the growing demand for both model capability and training efficiency, post-training paradigms have undergone continuous refinement. Early post-training pipelines such as Qwen1 [ 4 ] commonly rely on Supervised Fine-Tuning (SFT) followed by reinforcement learning from human feedback built upon Proximal Policy Optimization (PPO). In this setting, SFT is used to establish basic instruction-following behavior, while reinforcement learning further optimizes model outputs with the help of a separately trained reward model. Although this paradigm proved effective, it also introduced considerable costs in human annotation, reward modeling, value estimation, and reinforcement learning optimization. To solve this problem, DeepSeek-V2 [ 21 ] adopts GRPO after SFT, which does not require a value network and replaces absolute value estimation with group-wise relative comparison. In parallel, Qwen2 [ 36 ] uses Direct Preference Optimization (DPO), which directly optimizes preference pairs without explicitly training a reward model. Then, Qwen2.5 [ 10 ] further combines offline DPO with online GRPO, implying a general trend of shifting from static preference optimization to hybrid offline–online training strategies. As reasoning capability became an increasingly important objective, post-training methods further evolved from general alignment toward reasoning-oriented optimization. DeepSeek-V3 [ 22 ] continued the supervised fine-tuning and GRPO-based training strategy, while using reasoning distillation to support the development of stronger reasoning models. DeepSeek-R1 [ 13 ] proposed a more systematic post-training recipe, incorporating cold-start data, reasoning-oriented reinforcement learning, rejection sampling, and general reinforcement learning to elicit long-chain reasoning behaviors. More recent models have extended this direction through multi-stage reinforcement learning, outcome-guided preference optimization, agentic reinforcement learning, real-environment interaction, specialist model training, and on-policy distillation. These developments indicate that the choice of post-training paradigm and optimization method has become a critical factor in improving both model performance and training efficiency. Despite rapid advances in corpus construction and post-training algorithms, the practical details of data processing and training methodologies that govern post-training performance remain insufficiently transparent. This lack of transparency makes it difficult for researchers to reference prior work and reproduce experiments. To address this limitation, we introduce NebulaExp, a fully open post-training study built on Qwen3-8B-base [ 30 ] . Rather than only reporting final model performance, NebulaExp provides a transparent account of the entire post-training process, including data sources, corpus construction, training methodology, stage-wise training design, intermediate results, ablation

Read the complete text