Xiaomi MiMo-VL-Miloco Technical Report
MiLM-Plus Xiaomi · 2025
Read Xiaomi MiMo-VL-Miloco Technical Report by MiLM-Plus Xiaomi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-17436.txt.
Introduction
Jiaze Li and a Xiaomi team release MiMo-VL-Miloco-7B, an open vision-language model tuned for the smart home. Built on Xiaomi's MiMo-VL-7B base, it watches rooms and clips, names what people are doing, and reads hand gestures. The team also ships a smaller quantized copy and a public test kit for home scenes.
Training ran in two stages: guided fine-tuning, then reinforcement learning that rewards better answers. Extra supervision taught the model to reason step by step while watching its token budget. On home-scene tests it posted leading scores for gestures and daily activity reading. It also gained on broad video and language benchmarks, trading a little skill only on document-heavy tasks.
Catalog record
| CREATOR | MiLM-Plus Xiaomi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-17436
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Xiaomi MiMo-VL-Miloco Technical Report MiLM-Plus Xiaomi Abstract We open-source MiMo-VL-Miloco-7B and its quantized variant MiMo-VL-Miloco-7B-GGUF , a pair of home-centric vision-language models that achieve strong performance on both home-scenario understanding and general multimodal reasoning. Built on the MiMo-VL-7B backbone, MiMo-VL-Miloco-7B is specialized for smart-home environments, attaining leading F1 scores on gesture recognition and common home-scenario understanding, while also delivering consistent gains across video benchmarks such as Video-MME, Video-MMMU, and Charades-STA, as well as language understanding benchmarks including MMMU-Pro and MMLU-Pro. In our experiments, MiMo-VL-Miloco-7B outperforms strong closed-source and open-source baselines on home-scenario understanding and several multimodal reasoning benchmarks. To balance specialization and generality, we design a two-stage training pipeline that combines supervised fine-tuning with reinforcement learning based on Group Relative Policy Optimization, leveraging efficient multi-domain data. We further incorporate chain-of-thought supervision and token-budget-aware reasoning, enabling the model to learn knowledge in a data-efficient manner while also performing reasoning efficiently. Our analysis shows that targeted home-scenario training not only enhances activity and gesture understanding, but also improves text-only reasoning with only modest trade-offs on document-centric tasks. Model checkpoints, quantized GGUF weights, and our home-scenario evaluation toolkit are publicly available at https://github.com/XiaoMi/xiaomi-mimo-vl-miloco to support research and deployment in real-world smart-home applications. Figure 1: Per-category F1 score comparison on home-scenario gesture recognition (left) and daily-activity recognition (right) benchmarks for MiMo-VL-Miloco-7B and baseline models (MiMo-VL-7B-SFT-2508, Gemini-2.5-Pro, and Qwen2.5-VL-7B). MiMo-VL-Miloco-7B attains the highest F1 scores on most gestures (OK, Thumbs Up, V Sign, Shaka Sign, Open Palm) and activities (Watch TV, Reading, Play on the Phone, Play Esports, Workout), with the annotated values indicating its per-category F1 scores. 1 Introduction Recently, vision-language models and, more broadly, multimodal large language models have rapidly become the core infrastructure for modern AI systems, enabling agents to perceive visual scenes, reason over images and videos, and interact with the physical and digital worlds in a unified way [ 2 , 23 , 27 , 39 , 21 , 20 , 19 ] . In the smart-home domain, these capabilities are especially critical: to move beyond simple “if-this-then-that” rules, home systems must be able to understand what people are doing, interpret fine-grained visual cues, and make safe, context-aware decisions in real time [ 4 , 10 , 40 ] . In particular, privacy and latency constraints call for compact models that can run locally on edge devices instead of relying solely on cloud inference [ 9 , 32 ] . In this work, we present MiMo-VL-Miloco-7B , a 7B-parameter vision-language model tailored for home scenarios and deployed as the perception core of the Xiao Mi lo cal co pilot framework. Built on top of MiMo-VL-7B [ 38 ] , MiMo-VL-Miloco-7B inherits a strong general-purpose multimodal backbone and further specializes it for home environments, covering recognizing common daily activities (Play Esports, Workout, Watch TV, Reading, Play on the Phone) and understanding hand gestures (Thumbs Up, V Sign, Open Palm, OK, Shaka Sign). By connecting to Xiaomi MiJIA cameras as visual sensors and integrating with the Xiaomi IoT ecosystem, the model enables rich automation, ranging from instructions such as “turn on the reading lamp when someone is reading” to more complex instructions such as “if the baby cries after 10 p.m., dim the nursery lights to 10 %, play white-noise audio on the ceiling speaker, and push a quiet notification to the parents’ phones.” [ 34 , 26 ] . The development of MiMo-VL-Miloco-7B follows a two-stage training pipeline designed to inject valuable home-scenario knowledge while preserving the base model’s generality [ 23 , 8 ] . In the first stage, we perform supervised fine-tuning (SFT) on curated data from home environments. This stage emphasizes chain-of-thought supervision so that the model not only predicts labels but also learns explicit reasoning patterns about activities in home scenarios [ 37 ] . In addition, we adopt token-budget-aware reasoning data that encourage concise, task-oriented responses. Together, chain-of-thought supervision and token-budget-aware reasoning enable the model to learn knowledge in a data-efficient manner while also performing reasoning efficiently, which is important for latency-sensitive edge deployment [ 44 , 47 ] . In the second stage, we apply GRPO-based reinforcement learning on top of the SFT model [ 30 , 14 ] . Leveraging the Time-R1 [ 35 ] data construction strategy, we assemble efficient training sets across multiple domains and optimize the model with mixed reinforcement signals [ 35 , 31 ] to further enhance its perception and reasoning capabilities while avoiding over-specialization. This stage focuses primarily on improving performance on generic multimodal benchmarks [ 41 , 11 ] , while maintaining strong home-scenario understanding, including gesture recognition and activity classification. Empirically, MiMo-VL-Miloco-7B achieves strong F1 scores on internal home-scenario evaluations, while also showing gains on several public multimodal understanding and reasoning benchmarks [ 35 , 41 , 11 ] , with only modest degradation on tasks that are less relevant to its target use cases, such as OCR or mathematical reasoning. Our contributions can be summarized as follows: 1. Home-centric multimodal modeling on the edge. We introduce MiMo-VL-Miloco-7B, an edge-deployable vision-language model specifically optimized for home environments, capable of robust activity recognition, gesture understanding, and natural-language interaction with home scenarios, achieving competitive performance across multimodal benchmarks. 2. A two-stage tuning framework for home scenarios. We design a effective two-stage training pipeline, in which the first stage performs chain-of-thought enhanced supervised fine-tuning and the second stage applies GRPO-based reinforcement learning with difficulty-aware data filtering. This training strategy substantially improves home-scenario understanding, while preserving the generalization ability of the pretrained backbone. 3. Open ecosystem for smart-home research and deployment. We release both MiMo-VL-Miloco-7B and its quantized variant MiMo-VL-Miloco-7B-GGUF, along with a Gradio demo and integration into the open-source Xiaomi Miloco framework. These resources provide a practical foundation for researchers and developers to explore privacy-preserving, on-device multimodal intelligence in real-world smart-home settings. Taken together, these efforts advance smart-home systems beyond manual rule configuration toward genuinely context-aware, privacy-preserving multimodal agents that can continuously “see and think” alongside users in their daily lives. By serving as a unified, edge-deployable perception and reasoning backbone for home environments, MiMo-VL-Miloco-7B provides a practical foundation for research and large-scale deployment of vision-language copilots in real-world smart-home ecosystems. 2 Approach 2.1 Supervised Fine-Tuning (SFT) In this section, we first present the architectural design of MiMo-VL-Miloco-7B, followed by a detailed description of the data composition and the training implementation for the supervised fine-tuning stage. 2.1.1 Architecture of MiMo-VL-Miloco The architecture of MiMo-VL-Miloco-7B consists of three primary components: (1) a Vision Transformer (ViT) supporting native resolution, which encodes visual inputs; (2) a projector composed of a Multi-Layer Perceptron (MLP), serving to align visual encodings with the latent space of the Large Language Model (LLM); and (3) the LLM backbone, responsible for text comprehension and reasoning. All components, including the vision encoder, the projector, and the LLM backbone, are initialized using their respective pre-trained weights from MiMo-VL [ 38 ] , thereby inheriting its exceptional capabilities in multimodal understanding and reasoning. We show the architecture of MiMo-VL-Miloco-7B in Fig. 2 . Figure 2: The overview of MiMo-VL-Miloco-7B. The model takes as input a video depicting either a home scene or user gestures. Video frames are encoded by a Vision Transformer (ViT) and projected into the LLM embedding space via