Token Entropy Regularization for Multi-modal Antenna Affiliation Identification
Dong Chen, Ruoyu Li, Xinyan Zhang, Jialei Xu, Ruosen Zhao, Zhikang Zhang, Lingyun Li, Zizhuang Wei · 2026
Read Token Entropy Regularization for Multi-modal Antenna Affiliation Identification by Dong Chen, Ruoyu Li, Xinyan Zhang, Jialei Xu, Ruosen Zhao, Zhikang Zhang, Lingyun Li, Zizhuang Wei (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21280.txt.
Introduction
Dong Chen and seven co-authors work on matching antennas to the towers they belong to. Network crews now do this matching by climbing towers and checking by hand. That process is slow and easy to get wrong, and errors hurt network quality.
The team fuses three kinds of evidence: video of the site, the shape and position of each antenna, and cell identity signals. The job becomes a matching task across these sources. A measure based on token spread helps the model sort out hard cases. Automating the check could cut field work and fix records that guide repairs and upgrades.
Catalog record
| CREATOR | Dong Chen, Ruoyu Li, Xinyan Zhang, Jialei Xu, Ruosen Zhao, Zhikang Zhang, Lingyun Li, Zizhuang Wei |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-21280
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Token Entropy Regularization for Multi-modal Antenna Affiliation Identification Dong Chen, Ruoyu Li, Xinyan Zhang, Jialei Xu, Ruosen Zhao, Zhikang Zhang, Lingyun Li, Zizhuang Wei Abstract Accurate antenna affiliation identification is crucial for optimizing and maintaining communication networks. Current practice, however, relies on the cumbersome and error-prone process of manual tower inspections. We propose a novel paradigm shift that fuses video footage of base stations, antenna geometric features, and Physical Cell Identity (PCI) signals, transforming antenna affiliation identification into multi-modal classification and matching tasks. Publicly available pretrained transformers struggle with this unique task due to a lack of analogous data in the communications domain, which hampers cross-modal alignment. To address this, we introduce a dedicated training framework that aligns antenna images with corresponding PCI signals. To tackle the representation alignment challenge, we propose a novel Token Entropy Regularization module in the pretraining stage. Our experiments demonstrate that TER accelerates convergence and yields significant performance gains. Further analysis reveals that the entropy of the first token is modality-dependent. Code will be made available upon publication. 1 Introduction Accurate calibration of antenna parameters is essential for network optimization but currently relies on inefficient manual tower surveys 2 ; 27 . These surveys are challenging due to the large number of visually similar antennas installed at height. The Physical Cell Identity (PCI) signal offers a promising solution to this problem 21 . As a unique identifier in 4G and 5G networks, the PCI signals allow user devices to synchronize with a cell, thereby providing a direct digital fingerprint that links radio signals to their source antenna 25 ; 1 . Despite this potential, current automation efforts remain inadequate 37 . Existing visual analysis methods fail to solve the underlying parameter-matching problem, while multi-modal AI lacks a dedicated paradigm to leverage communication signals like the PCI for this specific calibration task 24 ; 28 . Figure 1: Schematic diagram of the multi-modal antenna affiliation identification framework. (A) The practical needs of identifying antenna affiliations. (B) A multi-modal approach is proposed with base station videos, antenna geometric features, and PCI signals. (C) The task is achieved through a two-stage training process: classifying antenna types, and matching affiliations. End-to-end models often struggle to align multi-modal features, and existing pretrained large models lack specialization for communication network tasks 15 ; 7 . To address this gap, we propose a multi-modal fusion matching scheme to associate devices with their signals. Our training paradigm consists of two sequential phases. First, we employ CLIP-based contrastive learning to establish an initial alignment between visual antenna features and their corresponding PCI-level signals 6 ; 18 ; 33 . Subsequently, a fine-tuning phase adapts these pretrained representations to the practical challenge of matching PCI signals to an unknown and variable set of antenna devices. To also address representation learning with limited data, we draw inspiration from sparse rate reduction, which maximizes information gain through a coding rate function to facilitate representation learning 42 ; 14 . This principle has been explored in the Coding Rate Reduction Transformer (CRATE), a transformer-like model that learns compressed and sparse representations 42 . Building on this foundation, we introduce a Token Entropy Regularization (TER) module. TER consists of Enhanced Token Entropy (ETE) layer, which computes the entropy of token representations along the channel axis to assess encoding complexity, and a Token Entropy Loss (TEL) function, which acts as an external regularizer. In summary, this paper has the following contributions: (1) A Novel Problem Formulation and Foundational Paradigm: We are the first to formally define antenna affiliation identification as an open-world multi-modal matching task, bridging antenna classification and matching tasks. Departing from prior vision-only or generic multimodal methods, we introduce the first paradigm that integrates visual antenna data with domain-specific Physical Cell Identity (PCI) signals, establishing a new direction for data-driven network optimization. (2) Introducing an Innovative Module for Feature Alignment: To address the core challenge of aligning heterogeneous data, we propose a Token Entropy Regularization (TER) module. This framework provides a novel, theoretically-grounded method for disciplined and effective fusion of visual and signal representations, advancing multimodal learning methodology. (3) A Validated, Efficient Pathway for Practical Deployment: We demonstrate an efficient training strategy combining multimodal pretraining with targeted fine-tuning. This approach enables robust, data-efficient antenna-signal matching with substantial performance gains for 4G/5G networks, underscoring its direct viability for real-world deployment. Figure 2: An overview of the proposed multimodal framework for antenna affiliation identification. (A) Antenna type classification via a supervised DINOv3-Transformer. (B, C) The antenna affiliation task comprising a pretraining stage followed by supervised fine-tuning under different data-matching conditions. (D) The Token Entropy Regularization (TER) module, which encourages model convergence by regularizing the distribution of token representations. 2 Related Work 2.1 Multi-modal Encoders The dominance of Convolutional Neural Networks (CNNs) for visual recognition was challenged by the Vision Transformer (ViT) 9 , which established the foundation by applying a pure transformer architecture directly to image patches. Subsequent work focused on improving ViT’s efficiency and adaptability for vision-specific tasks. The Data-efficient Image Transformer (DeiT) 31 introduced distillation strategies to train ViTs effectively on ImageNet alone. More importantly, hierarchical architectures like the Swin Transformer 16 and the Pyramid Vision Transformer (PVT) 35 ; 36 reintroduced inductive biases for multi-scale processing, enabling their use as general-purpose backbones for dense prediction. This period also saw fruitful co-evolution with modernized CNNs like ConvNeXt 19 and alternative architectures like MLP-Mixer 30 . The transformer paradigm was successfully extended to video by modelling spatiotemporal relationships. Models like TimeSformer 4 , ViViT 3 , and Video Swin Transformer 17 introduced novel attention mechanisms across frames, while hierarchical models like MViT 10 offered integrated multiscale video understanding. Beyond pixels, this unified architecture proved remarkably versatile. For other modalities, such as 3D point clouds, specialized encoders like the Point Transformer 45 were developed, while the vanilla Transformer encoder 32 itself served as a powerful, permutation-invariant backbone for diverse structured data, from sensor signals (e.g., PCI points) to geometric features. Two major learning paradigms on these backbones have driven progress. In self-supervised learning, masked autoencoding methods like MAE 13 and knowledge distillation approaches like DINOv3 23 have learned rich, general-purpose visual features from pixels alone. In multimodal learning, contrastive models like CLIP 26 and its more efficient successor SigLIP 44 aligned visual and textual spaces, enabling zero-shot transfer. This trend towards joint representation learning has culminated in models like ImageBind 12 , which embeds multiple modalities (image, text, audio, depth, etc.) into a single, shared space, pushing the unified transformer paradigm to its logical extreme. 2.2 Token Entropy Utilization Token entropy, which quantifies the information complexity or uncertainty of a token, has become a pivotal tool for analyzing and optimizing transformer-based models. It is widely studied for applications such as token selection 20 ; 38 , compression and pruning 22 ; 29 , to improve efficiency without sacrificing performance. Beyond efficiency, token entropy provides critical insights for model refinement. In reasoning tasks, research shows that a minority of high-entropy ”forking tokens” drive most performance gains, enabling selective optimization that matches full-parameter training efficiency 34 . For long-context modeling, frameworks like SirLLM leverage token entropy to identify and retain high-information phrases, significantly improving long-term memory in streaming dialogues without fine-tuning 40