Open-Source Multimodal Moxin Models with Moxin-VLM and Moxin-VLA

Pu Zhao, Xuan Shen, Zhenglun Kong, Yixin Shen, Sung-En Chang, Arash Akbari, Timothy Rupprecht, Lei Lu, Enfu Nan, Changdi Yang, Yumei He, Weiyan Shi, Xingchen Xu, Yu Huang, Wei Jiang, Wei Wang, Yue Chen, Yong He, Yanzhi Wang · 2025

Read Open-Source Multimodal Moxin Models with Moxin-VLM and Moxin-VLA by Pu Zhao, Xuan Shen, Zhenglun Kong, Yixin Shen, Sung-En Chang, Arash Akbari, Timothy Rupprecht, Lei Lu, Enfu Nan, Changdi Yang, Yumei He, Weiyan Shi, Xingchen Xu, Yu Huang, Wei Jiang, Wei Wang, Yue Chen, Yong He, Yanzhi Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-22208.txt.

Introduction

Pu Zhao and twenty co-authors introduce the Moxin family of open-source language models, built around the Moxin 7B base model. Proprietary systems like GPT-4 dominate attention, while open models such as LLaMA and Mistral spread through easy customization. Moxin follows the Model Openness Framework, sharing not just weights but training data, code, and implementation details for full transparency.

Three variants extend the base model: Moxin-VLM for vision and language, Moxin-VLA for vision, language, and action, and Moxin-Chinese for Chinese capability. The team trained everything with open-source systems and open data, and reports strong results across their evaluations. Models, data, and code are all released to support a healthier open ecosystem.

Catalog record

CREATORPu Zhao, Xuan Shen, Zhenglun Kong, Yixin Shen, Sung-En Chang, Arash Akbari, Timothy Rupprecht, Lei Lu, Enfu Nan, Changdi Yang, Yumei He, Weiyan Shi, Xingchen Xu, Yu Huang, Wei Jiang, Wei Wang, Yue Chen, Yong He, Yanzhi Wang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-22208

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Open-Source Multimodal Moxin Models with Moxin-VLM and Moxin-VLA Pu Zhao, Xuan Shen, Zhenglun Kong, Yixin Shen, Sung-En Chang, Arash Akbari, Timothy Rupprecht, Lei Lu, Enfu Nan, Changdi Yang, Yumei He, Weiyan Shi, Xingchen Xu, Yu Huang, Wei Jiang, Wei Wang, Yue Chen, Yong He, Yanzhi Wang Abstract Recently, Large Language Models (LLMs) have undergone a significant transformation, marked by a rapid rise in both their popularity and capabilities. Leading this evolution are proprietary LLMs like GPT-4 and GPT-o1, which have captured widespread attention in the AI community due to their remarkable performance and versatility. Simultaneously, open-source LLMs, such as LLaMA and Mistral, have made great contributions to the ever-increasing popularity of LLMs due to the ease to customize and deploy the models across diverse applications. Moxin 7B is introduced as a fully open-source LLM developed in accordance with the Model Openness Framework, which moves beyond the simple sharing of model weights to embrace complete transparency in training, datasets, and implementation detail, thus fostering a more inclusive and collaborative research environment that can sustain a healthy open-source ecosystem. To further equip Moxin with various capabilities in different tasks, we develop three variants based on Moxin, including Moxin-VLM, Moxin-VLA, and Moxin-Chinese, which target the vision-language, vision-language-action, and Chinese capabilities, respectively. Experiments show that our models achieve superior performance in various evaluations. We adopt open-source framework and open data for the training. We release our models, along with the available data and code to derive these models. Homepage with all codes: https://github.com/moxin-org/Moxin-LLM Base model: https://huggingface.co/moxin-org/Moxin-7B-LLM Instruct model: https://huggingface.co/moxin-org/Moxin-7B-Instruct Reasoning model: https://huggingface.co/moxin-org/Moxin-7B-Reasoning VLM model: https://huggingface.co/moxin-org/Moxin-7B-VLM VLA model: https://huggingface.co/moxin-org/Moxin-7B-VLA Chinese model: https://huggingface.co/moxin-org/Moxin-7B-Chinese 1 Introduction The field of natural language processing has witnessed the most exciting discoveries of the last ten years with the emergence of large language models (LLMs). At the forefront of this evolution are LLMs such as GPT-4 [ 3 ] , Claude [ 4 ] , and Gemini [ 85 ] , which have captured the attention of the AI community due to their performance and versatility. Meanwhile, the recent emergence of openly accessible yet highly capable LLMs such as LLaMA [ 20 ] , Falcon [ 65 ] , and Mistral [ 27 ] allow researchers and practitioners to easily obtain, customize, and deploy LLMs in more various environments and for more diverse use cases. The trends have people eagerly asking about what’s next and some suggest “a general intelligence” is right around the corner. Moxin-7B is introduced as a fully open-source LLM developed by complying with the Model Openness Framework (MOF) introduced by [ 93 ] . The MOF provides a systematic ranking classification system to rate AI models based on their completeness and openness, incorporating the principles of open science, open source, open data, and open access. By promoting transparency and reproducibility, the MOF serves as a crucial tool to combat “openwashing” and to establish completeness and openness as primary criteria alongside the core tenets of responsible AI. It moves beyond the simple sharing of model weights to embrace complete transparency in training, datasets, and implementation detail, which is crucial for fostering a more inclusive and collaborative research environment that can sustain a healthy open-source ecosystem [ 30 ] . The wide adoption of MOF and open-source state-of-the-art models will cultivate a more open AI ecosystem, benefiting research and innovation. To equip Moxin with various capabilities in different tasks, we develop multimodal Moxin models, including Moxin-VLM, Moxin-VLA and Moxin-Chinese. Moxin-VLM is our vision language model (VLM), with our Moxin model as the LLM backbone. It enables Moxin to process text and image inputs with multi-modal capabilities. Furthermore, we develop and release our vision language action model (VLA) with Moxin as the LLM backbone. The VLA model can achieve outstanding performance for robotic control. Moreover, we release Moxin-Chinese which targets to enhance the Moxin performance in Chinese, especially for English and Chinese translation, which is important for global conference. Experiments show that our models achieve superior performance in various evaluations such as zero-shot evaluation and few-shot evaluation. For example, Moxin-VLM can achieve higher average accuracy (across multiple evaluation benchmarks) compared to popular LLM backbones such Llama and Mistral with non-marginal improvements (about 2 percentage points or higher). Our homepage is https://github.com/moxin-org/Moxin-LLM . We summarize our contributions below: • To equip Moxin with various capabilities in different tasks, we develop multimodal Moxin models, including Moxin-VLM, Moxin-VLA and Moxin-Chinese. For Moxin-Chinese, we extend the Moxin vocabulary with additional Chinese tokens and post-train the model with more Chinese data, thus improving Chinese encoding capability and efficiency. • We develop Moxin-VLM with Moxin as the LLM backbone. Based on the open-source VLM framework Prismatic VLMs, we train our VLM on fully open-source datasets with Moxin as the LLM backbone and DINOv2 & SigLIP as the visual backbone. Our experiments demonstrate that our VLM outperforms other VLM models or LLM backbones. • We further finetune Moxin-VLM with the recipe from OpenVLA-OFT [ 35 ] to develop Moxin-VLA for robotic control. Our approach streamlines this training paradigm by employing the Moxin-VLM backbone in conjunction with the highly efficient OpenVLA-OFT fine-tuning recipe. Our Moxin-VLA achieves superior performance under comprehensive evaluations. 2 Related Work 2.1 Models, Tokenizers, and Training Models. State-of-the-art large language models (LLMs) typically comprise a substantial number of parameters, often approaching or exceeding 100 billion [ 20 , 3 , 85 ] . To facilitate broader accessibility, smaller models with fewer than 20 billion parameters, and even those around 7 billion parameters, have been developed [ 6 , 96 , 20 , 27 ] . In addition, efficiency-enhancing techniques, such as pruning [ 43 , 109 , 56 , 17 , 37 , 111 , 81 , 78 , 77 , 51 , 80 , 97 , 44 , 81 ] , quantization [ 75 , 103 , 97 , 94 , 101 , 41 , 42 ] , token reduction [ 102 , 104 , 36 , 79 , 38 , 76 , 110 ] or implementing MAMBA-based architectures in Jamba, have been employed to optimize performance [ 45 , 88 ] . Tokenizers. Tokenizers are essential to convert raw data into a suitable format for model processing. Many contemporary models employ Byte-Pair Encoding (BPE) [ 72 ] , with OpenAI’s tiktoken tokenizer [ 89 ] being a notable implementation. However, for languages that handle tokens differently from Romance languages, alternatives such as SentencePiece [ 40 ] are utilized, as seen in XLNet [ 98 ] . Hugging Face offers an excellent summary of state-of-the-art tokenizers with practical examples [ 87 ] . Moreover, tokenization extends beyond text modalities; many foundational models now include multimodal capabilities, processing documents, audio, images, and even videos [ 68 , 58 , 106 , 105 ] . Training. To enhance the performance of smaller models beyond their inherent limitations, various training strategies can be employed. A notable example is the application of Mixture of Experts (MoE) training, which has achieved significant success in models such as Mixtral [ 28 ] . 2.2 Open-source LLMs Large language models (LLMs) have rapidly evolved into a diverse ecosystem that spans closed-source, open-weight, and fully open-sourced paradigms. On one end of the spectrum, closed-source models such as GPT-4 [ 3 ] ,

Read the complete text