CrowdVLM-R1: Expanding R1 Ability to Vision Language Model for Crowd Counting using Fuzzy Group Relative Policy Reward
1 st Zhiqiang Wang, 2 nd Pengbin Feng, 3 rd Yanbin Lin, 4 th Shuzhang Cai, 5 th Zongao Bian, 6 th Jinghua Yan, 7 th Xingquan Zhu · 2025
Read CrowdVLM-R1: Expanding R1 Ability to Vision Language Model for Crowd Counting using Fuzzy Group Relative Policy Reward by 1 st Zhiqiang Wang, 2 nd Pengbin Feng, 3 rd Yanbin Lin, 4 th Shuzhang Cai, 5 th Zongao Bian, 6 th Jinghua Yan, 7 th Xingquan Zhu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-03724.txt.
Introduction
Zhiqiang Wang and six co-authors teach in 2025 a vision-language model to count crowds more honestly, by rewarding almost-right answers. Counting people in dense photos is a classic hard vision task, and the reinforcement method they borrow usually pays only for exactly right counts, a reward too sparse to learn from.
Their fuzzy reward gives partial credit that fades smoothly with the size of the error, guiding the model like a patient teacher. Plain group-relative training with all-or-nothing rewards actually did worse than ordinary fine-tuning. With fuzzy rewards, their models beat every baseline they tested, at two sizes.
Catalog record
| CREATOR | 1 st Zhiqiang Wang, 2 nd Pengbin Feng, 3 rd Yanbin Lin, 4 th Shuzhang Cai, 5 th Zongao Bian, 6 th Jinghua Yan, 7 th Xingquan Zhu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-03724
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CrowdVLM-R1: Expanding R1 Ability to Vision Language Model for Crowd Counting using Fuzzy Group Relative Policy Reward 1 st Zhiqiang Wang, 2 nd Pengbin Feng, 3 rd Yanbin Lin, 4 th Shuzhang Cai, 5 th Zongao Bian, 6 th Jinghua Yan, 7 th Xingquan Zhu Abstract We propose CrowdVLM-R1, which expands the R1 base model for accurate crowd counting, using a novel framework that integrates the fuzzy group relative policy optimization reward function (FGRPR) to enhance learning efficiency. Unlike the conventional binary (0/1) accuracy reward, our fuzzy reward model, FGRPR, which contains both format and precision rewards, provides nuanced incentives to encourage the R1 model to learn to adjust policies towards precise outputs. Supervised fine-tuning (SFT) is also integrated for the CrowdVLM-R1 model to learn from a handful of inputs to enable both in-domain and out-of-domain counting. Experimental results demonstrate that GRPO with a standard binary accuracy reward underperforms compared to SFT. In contrast, FGRPR, applied to Qwen2.5-VL-(3B/7B), surpasses all baseline models, including GPT-4o, LLaMA2-70B and SFT, in five domain datasets. For out-of-domain datasets, FGRPR achieves performance comparable to SFT but excels when target values are larger, as its fuzzy reward function assigns higher rewards to closer approximations. This approach is broadly applicable to tasks where the precision of the answer is critical. The code and data are available at: https://github.com/yeyimilk/CrowdVLM-R1 Fig. 1: An overview of our framework, which integrates group relative policy optimization (GRPO) and the proposed fuzzy group relative policy reward (FGRPR) to train a visual language model (VLM) for crowd counting tasks. From left to right, an image and prompt are input to a base LLM to obtain initial counting outcomes (The ground truth value on the top left corner of the input image is just an indicator number for visualization purposes. It is only used for rewards calculation, but will not send to LLM as part of the input. The instruction model uses a policy model, parameterized by \theta , to obtain policy and generate outputs. The proposed fuzzy group relative policy reward is then used to adjust the reward for the individual image. The adjusted reward is used to calculate the objective function value, which in turn updates the \theta parameters for supervised fine-tuning. I Introduction Recently, DeepSeek R1 [ 1 ] has drawn much attention among advances in large language models (LLMs), as it demonstrates how reinforcement learning (RL) can be the primary driver of reasoning. The model is a Mixture-of-Experts (MoE) transformer trained using the GRPO algorithm [ 2 ] ; specifically, it applies simple rule-based rewards to guide the model toward complex reasoning skills. While DeepSeek R1 is an LLM, its training paradigm can be extended to vision-language models (VLMs). There are various types of visual tasks, such as crowd counting [ 3 , 4 , 5 ] , image classification [ 6 , 7 , 8 ] , and visual question answering [ 9 , 10 ] ; in the context of VLM, they require the model to go beyond simple pattern recognition and perform precise reasoning. Latest works explore applying RL for visual tasks, one of which is VLM-R1 [ 11 ] , which assigns rewards based on the intersection over union (IoU) score - a binary reward of 1 if IoU > 0.5, and 0 otherwise. However, this approach does not distinguish between near-correct and marginally correct detections (e.g., an IoU of 0.55 vs. 0.95), then better results do not receive higher rewards. This suggests the need for more fine-grained reward schemes when applying RL to vision-based reasoning tasks. In this paper, we develop a novel approach based on GRPO to handle crowd counting tasks. It has been a fundamental yet highly challenging problem and the primary goal is to accurately estimate the number of instances of a particular object. In such scenarios, objects can overlap or vary in size, and they usually fade into complex backgrounds. The VLM must avoid either double-counting or missing partially covered objects. This demands strong spatial reasoning and fine-grained attention. Therefore, we explore key research questions regarding the application of RL to VLMs for the counting task: • RQ1: How effective is GRPO for VLMs for crowd counting tasks? • RQ2: How can a better reward function be designed to enhance the numerical reasoning ability of VLMs? • RQ3: How to improve VLMs for both within- and cross-domain counting tasks? To address RQ1, we first conduct a systematic investigation into the impact of RL on VLMs. Specifically, we adapt GRPO, originally used in DeepSeek R1, for fine-tuning mainstream VLMs. Our experiments confirm that GRPO is highly feasible for the counting task in VLMs. However, it requires a customized reward function, as conventional reward function has certain limitations. Issues like this highlight the need to address our RQ2. A well-structured reward function should provide feedback with finer granularity and ensure that slight improvements in counting accuracy are properly incentivized rather than treated as binary outcomes. We solve this research question by introducing a fuzzy group relative policy reward (FGRPR). Our reward function considers both the model’s output format and the numerical precision of its predictions. The format reward is designed as a binary signal to encourage the model to provide an actual count for the scene. Meanwhile, the accuracy reward is based on the difference between the predicted count and the ground truth, guiding the model toward more precise predictions by penalizing deviations from the correct count. To address RQ3, we create a new, comprehensive counting dataset by selectively combining multiple existing image datasets. Given the complexity of the counting task, which involves a wide variety of scenes such as sparse and dense object distributions, as well as large and small objects, we ensure that our dataset captures this diversity. Our goal is to enable the model to adapt to various scene types. Additionally, to evaluate the model’s out-of-domain capabilities, we incorporate the Manatee dataset, which will be used exclusively for testing to assess the model’s zero-shot counting accuracy. The contributions of this work are as follows: • We adapt reinforcement learning (GRPO) to train VLMs for crowd counting and propose CrowdVLM-R1, which achieves state-of-the-art performance on the counting task; • We design a novel fuzzy reward function that takes into account both the format and counting precision, enhancing the model’s ability to provide accurate predictions; • We introduce a new comprehensive counting dataset, offering diverse scenes to assess the model’s counting capabilities for in-domain and out-of-domain scenes. II Related Work Traditional methods for Crowd Counting. Early research on Crowd Counting can be grouped into three categories: detection-based, regression-based, and density estimation-based methods. Detection-based approaches typically utilize detectors for a specific object, e.g., human head or animal body, and perform counting based on the detection results [ 12 , 13 ] . These methods have achieved good accuracy for sparse scenes, but struggle under dense or occluded scenarios. Regression-based methods, on the other hand, map the image patches directly to the count of objects via regression techniques [ 14 ] . Density estimation methods get rid of learning to localize individual objects by estimating the count with density maps [ 15 , 16 ] . With the success of Convolution Neural Networks (CNNs) in