Querying Autonomous Vehicle Point Clouds: Enhanced by 3D Object Counting with CounterNet DOI: 10.1145/3746027.3755043 Conference: Proceedings of the 33rd ACM International Conference on Multimedia; October 27–31, 2025; Dublin, Ireland. Proceedings of the 33rd ACM International Conference on Multimed
Xiaoyu Zhang, Zhifeng Bao, Hai Dong, Ziwei Wang, Jiajun Liu · 2025
Read Querying Autonomous Vehicle Point Clouds: Enhanced by 3D Object Counting with CounterNet DOI: 10.1145/3746027.3755043 Conference: Proceedings of the 33rd ACM International Conference on Multimedia; October 27–31, 2025; Dublin, Ireland. Proceedings of the 33rd ACM International Conference on Multimed by Xiaoyu Zhang, Zhifeng Bao, Hai Dong, Ziwei Wang, Jiajun Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-19209.txt.
Introduction
Xiaoyu Zhang and co-authors tackle searching the huge three-dimensional scans that self-driving cars collect, in this 2025 paper. Each drive yields clouds of points mapping the road scene, yet only a slice matters for any one task. Tasks range from spotting crash risks to counting traffic. The authors set out clear rules for what such searches should do.
They define three query types: fetching matched items, counting objects, and summing up results. Each type leans on accurate object counts, so they built a counting tool called CounterNet. Older work indexed flat video and trusted detectors to count well, an approach that breaks down in three dimensions. Their method aims to keep counts sharp so query answers stay trustworthy.
Catalog record
| CREATOR | Xiaoyu Zhang, Zhifeng Bao, Hai Dong, Ziwei Wang, Jiajun Liu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-19209
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Querying Autonomous Vehicle Point Clouds: Enhanced by 3D Object Counting with CounterNet DOI: 10.1145/3746027.3755043 Conference: Proceedings of the 33rd ACM International Conference on Multimedia; October 27–31, 2025; Dublin, Ireland. Proceedings of the 33rd ACM International Conference on Multimedia (MM ’25), October 27–31, 2025, Dublin, Ireland ISBN: 979-8-4007-2035-2/2025/10 CCS: Computing methodologies Computer vision problems Xiaoyu Zhang, Zhifeng Bao, Hai Dong, Ziwei Wang, Jiajun Liu Abstract . Autonomous vehicles generate massive volumes of point cloud data, yet only a subset is relevant for specific tasks such as collision detection, traffic analysis, or congestion monitoring. Effectively querying this data is essential to enable targeted analytics. In this work, we formalize point cloud querying by defining three core query types: RETRIEVAL, COUNT, and AGGREGATION, each aligned with distinct analytical scenarios. All these queries rely heavily on accurate object counts to produce meaningful results, making precise object counting a critical component of query execution. Prior work has focused on indexing techniques for 2D video data, assuming detection models provide accurate counting information. However, when applied to 3D point cloud data, state-of-the-art detection models often fail to generate reliable object counts, leading to substantial errors in query results. To address this limitation, we propose CounterNet, a heatmap-based network designed for accurate object counting in large-scale point cloud data. Rather than focusing on accurate object localization, CounterNet detects object presence by finding object centers to improve counting accuracy. We further enhance its performance with a feature map partitioning strategy using overlapping regions, enabling better handling of both small and large objects in complex traffic scenes. To adapt to varying frame characteristics, we introduce a per-frame dynamic model selection strategy that selects the most effective configuration for each input. Evaluations on three real-world autonomous vehicle datasets show that CounterNet improves counting accuracy by 5% to 20% across object categories, resulting in more reliable query outcomes across all supported query types. 1. Introduction The rise of autonomous driving has significantly increased the importance of point cloud data analysis, as autonomous vehicles heavily rely on point clouds to perceive their surroundings accurately. While vast amounts of data are collected, 3D detection alone can generate approximately 5TB of point cloud data per hour ( Kazhamiaka et al. 2021 ) , not all data is relevant to every training task. For instance, detecting vehicle collisions and analyzing traffic congestion differ in their focus. Collision detection often involves identifying sudden, localized changes in vehicle count within short time frames, while congestion analysis requires aggregating vehicle counts over longer durations to identify sustained traffic buildup. These tasks, therefore, depend on different object types, time scales, and aggregation strategies, reinforcing the need for flexible point cloud analytics designed for specific use cases. Figure 1 . Example of different query types To support this, we define a query model that captures common patterns of analytical interest. Drawing on prior work in traffic video and point cloud querying, we identify three key query types: RETRIEVAL ( Kang et al. 2019 ; Kang et al. 2022 ; Li et al. 2025 ) , COUNT ( Cao et al. 2022 ) , and AGGREGATION ( Bang et al. 2023 ) . These cover a broad range of analysis needs for point cloud data, as illustrated in Figure 1 . Specifically, RETRIEVAL queries extract frames that satisfy specific conditions, useful for tasks such as object tracking and scene filtering. COUNT queries quickly quantify how many frames meet a given condition, enabling statistical analysis, monitoring, and quality control. AGGREGATION queries summarize patterns across multiple frames, supporting trend analysis. These query types form the foundation of our approach. However, their effectiveness relies heavily on accurate object counts within individual frames. Unfortunately, current 3D detection models struggle to provide reliable counts across object categories, as shown in Figure 2 . This undermines the assumption made by many existing query systems that detection outputs can serve as ground truth. Prior work mostly focused on 2D traffic video querying ( Kang et al. 2022 ; Kang et al. 2019 ; Bang et al. 2023 ; Cao et al. 2022 ) . These studies generally assume that 2D object detection provides ground truth semantic information, and focus primarily on building efficient and effective indexing mechanisms to reduce the cost of query execution. For example, BlazeIt ( Kang et al. 2019 ) and TASTI ( Kang et al. 2022 ) used proxy models to filter simple scenes, Seiden ( Bang et al. 2023 ) used sampling to minimize model invocations, and Figo ( Cao et al. 2022 ) dynamically selected detectors based on video complexity. Although fewer works address 3D point cloud querying, approaches like MAST ( Li et al. 2025 ) extended Seiden ( Bang et al. 2023 ) to the 3D domain by incorporating the spatial-temporal filters into the indexing process. These methods made a key assumption from their 2D counterparts: that object detection models yield ground truth results to serve as a foundation for query execution. However, this assumption does not hold in the 3D point cloud setting. As shown in Figure 2 , SOTA 3D detection models such as VoxelNext ( Chen et al. 2023 ) and TransFusion ( Bai et al. 2022 ) exhibit significant variance in object counting accuracy across different classes of objects. As a result, existing 3D query systems, which rely on indexing over potentially inaccurate outputs, struggle to deliver trustworthy results. Unlike these indexing-focused approaches, our work takes a different path: we focus on improving object counting accuracy in point cloud data, a foundational requirement for reliable query execution. Notably, rather than improving indexing itself, we target a more fundamental problem that current methods tend to overlook. Figure 2 . Counting ACC of baselines v.s. CounterNet (ours) We propose CounterNet, a heatmap-based baseline designed for accurate object counting in complex traffic scenes. Our approach focuses on enhancing semantic information extraction from point cloud frames, with a particular emphasis on achieving precise, category-aware object counts. To improve its adaptability across object types and sizes, we refine CounterNet to support multi-class counting through scale-aware mechanisms. In addition, we introduce a per-frame dynamic model selection strategy, which assigns the most suitable counting model configuration to each frame based on its complexity, further boosting overall query accuracy without sacrificing efficiency. It is worth noting that our study’s primary focus is query accuracy, rather than efficiency. Our contributions are summarised as below: • To support query accuracy, we introduce CounterNet, a heatmap-based network that serves as our baseline solution. CounterNet focuses on detecting object centers, enabling object counting within large-scale traffic point cloud frames (Section 4 ). • We enhance CounterNet with a feature map partitioning strategy to improve performance in dense object counting (Section 5.1 ). To better handle large objects, we introduce overlapping partitions, providing a lightweight and effective solution (Section 5.2 ). Finally, we propose per-frame dynamic model selection, which adaptively selects the most suitable model trained with different partition numbers and overlap ratios for each frame (Section 5.3 ). • We evaluated our approach on three autonomous vehicle datasets across three query types. The results show that CounterNet significantly outperforms 3D object detection models in query accuracy. Adding partitioning to CounterNet notably improved the detection of small objects, particularly