Zeros can be Informative: Masked Binary U-Net for Image Segmentation on Tensor Cores

Chunshu Wu, Ruibing Song, Sushant Kondguli, Tong Geng, Ang Li · 2026

Read Zeros can be Informative: Masked Binary U-Net for Image Segmentation on Tensor Cores by Chunshu Wu, Ruibing Song, Sushant Kondguli, Tong Geng, Ang Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-11660.txt.

Introduction

Chunshu Wu and four coauthors argue that zeros can be informative. Their 2026 preprint gives a masked binary U-Net for segmentation. Real-time segmentation enables AR, VR, robots, and drones. Tight budgets bind accuracy, latency, and energy on edge devices.

U-Net balances accuracy and efficiency better than big transformers. High-resolution input still strains compute, memory, and power. Binary networks cut weights to one bit for speed. Masking turns many values to zero in the network. Those zeros, the team shows, still carry useful signal. Running on Tensor Cores targets the hardware in edge chips. The aim is real-time masks within strict budgets.

Catalog record

CREATORChunshu Wu, Ruibing Song, Sushant Kondguli, Tong Geng, Ang Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-11660

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Zeros can be Informative: Masked Binary U-Net for Image Segmentation on Tensor Cores Chunshu Wu, Ruibing Song, Sushant Kondguli, Tong Geng, Ang Li Abstract Real-time image segmentation is a key enabler for AR/VR, robotics, drones, and autonomous systems, where tight accuracy, latency, and energy budgets must be met on resource - constrained edge devices. While U - Net offers a favorable balance of accuracy and efficiency compared to large transformer - based models, achieving real - time performance on high - resolution input remains challenging due to compute, memory, and power limits. Extreme quantization, particularly binary networks, is appealing for its hardware - friendly operations. However, two obstacles limit practicality: (1) severe accuracy degradation, and (2) a lack of end - to - end implementations that deliver efficiency on general - purpose GPUs. We make two empirical observations that guide our design. (1) An explicit zero state is essential: training with zero masking to binary U - Net weights yields noticeable sparsity. (2) Quantization sensitivity is uniform across layers. Motivated by these findings, we introduce Masked Binary U - Net (MBU - Net), obtained through a cost - aware masking strategy that prioritizes masking where it yields the highest accuracy - per - cost, reconciling accuracy with near - binary efficiency. To realize these gains in practice, we develop a GPU execution framework that maps MBU - Net to Tensor Cores via a subtractive bit - encoding scheme, efficiently implementing masked binary weights with binary activations. This design leverages native binary Tensor Core BMMA instructions, enabling high throughput and energy savings on widely available GPUs. Across 3 segmentation benchmarks, MBU - Net attains near full - precision accuracy (3% average drop) while delivering 2.04 × \times speedup and 3.54 × \times energy reductions over a 16-bit floating point U - Net. 1 Introduction Image segmentation is essential for applications such as AR/VR, autonomous drones, autopilot systems, and robotics. In these domains, real-time perception and sustained operation are typically required. For instance, segmentation at roughly 60 Hz frame rate is fundamental for Meta’s AR glasses to deliver a smooth, context-aware user experience. Likewise, inspection or delivery drones often must operate continuously for 30 minutes, sometimes even hours on a single battery. Consequently, in addition to accuracy, processing speed and power efficiency are also decisive for practical deployment on edge devices. With rising demand for high-resolution data and high frame rates, these constraints have become increasingly stringent. Compared to large, over-parameterized architectures such as Vision Transformer [ 13 ] , U-Net [ 43 ] is substantially cheaper in compute, memory, and energy for dense prediction, making it a more promising choice for edge scenarios. Additionally, U-Net’s characteristic encoder-decoder structure with skip connections has demonstrated high effectiveness for pixel-level tasks [ 7 , 54 ] , preserving spatial details and facilitating accurate pixel-level predictions. Nevertheless, running a U-Net on high-resolution video in real time can still exceed the strict power, latency, and memory limits of many edge devices – further optimizations are necessary to reconcile accuracy with deployability. Quantization [ 16 , 51 , 19 ] is a natural path to efficiency. At the extreme, binary networks [ 25 , 35 ] use 1-bit weights and activations, replacing Multiply-Accumulations (MACs) with simple bitwise operations (XNOR/XOR) followed by a population count (popcount) [ 40 ] . Since most processors handle data in chunks rather than individual bits, practical implementations group many binary values together – allowing a single instruction to execute many binary operations in parallel. This approach is highly efficient on CPUs and GPU CUDA cores [ 31 , 6 ] , and can be further extended to exploit Tensor Cores [ 32 ] . Collectively, these techniques leverage inexpensive, hardware-friendly operations to sharply reduce compute, memory traffic, and energy, making extreme low-bit inference attractive for resource-constrained edge deployments. However, two practical challenges hinder the effectiveness of extreme quantization. (1) Accuracy degradation . Binary networks usually suffer from a significant drop in accuracy due to the aggressive 1-bit quantization, and the impact on U-Net in segmentation tasks can be especially severe [ 3 , 18 ] . (2) Scarce practical solutions on general-purpose processors . Many binary/ternary methods [ 50 , 20 , 56 , 53 ] are algorithmic proofs-of-concept or tailored to custom FPGA [ 34 , 15 , 36 ] and ASIC [ 49 ] datapaths, leaving end-to-end U-Net designs and optimizations on general-purpose processors (e.g., GPUs) underexplored. To address these challenges, this paper aims to answer: (1) How to achieve near-full-precision accuracy with near-binary efficiency for U-Net on segmentation tasks? (2) How to deliver that efficiency on widely available GPUs without relying on specialized accelerators? To address the first challenge, we design our solution based on two key observations: (1) An explicit zero state is essential. Pure binary representations ({-1, +1}) force every connection to contribute, offering no neutral state to suppress uncertain or noisy signals. Our experiments demonstrate that, when training with zero masking on U-Net weights, in addition to substantially enhanced accuracy, there is noticeable sparsity – over 90% zeros in many layers, some consistently exceeding 95% – far outnumbering the +1 and -1 states. This indicates an abundance of signals that should be suppressed by zeros. (2) Uniform sensitivity across layers. An exhaustive sweep over 4,000 per - layer quantization configurations shows that masking any single layer has a comparable effect on accuracy. Since masking adds overhead, and masking all layers is often unnecessary if a subset already provides sufficient suppression of information flow, low-cost layers can be prioritized for masking to balance computation and accuracy. For example, masking transposed convolution layers has a strong impact on accuracy while incurring negligible computational cost. Since extra costs are incurred due to masking, we can strategically pick which layer to enable masking for computing and accuracy tradeoff, as it may not be necessary to apply masking for all layers, some of them can already ensure sufficient suppression for information pass. For instance, transposed convolution layers have strong impact on accuracy, despite their negligible computing cost. Guided by these findings, we propose a cost-aware masking strategy that identifies critical layers where masking is most beneficial, and introduce Masked Binary U-Net (MBU-Net), which achieves near full-precision accuracy with near-binary efficiency as highlighted in Figure 1 . Figure 1: Overview. Obtained through a cost-aware masking strategy and optimized for Tensor Core on GPUs, MBU-Net achievs full-precision level accuracy and binary-level efficiency. To address the second challenge, we target Tensor Cores, mapping MBU-Net via bit-packing and dedicated APIs. On GPUs with Tensor Cores, performance is typically memory-bound rather than compute-bound, thus we retain binary activations to reduce data movement and power. This approach is timely: Tensor Cores are now common even on edge GPUs (e.g., NVIDIA Jetson Xavier with Volta Tensor Cores, Jetson Orin with Ampere Tensor Cores), positioning them as practical platforms for efficient edge application deployment. Although prior frameworks have accelerated pure Binary Neural Networks (BNNs) on GPUs [ 31 , 32 ] , to the best of our knowledge, no high-performance framework exists for selectively masked binary models on Tensor Cores, especially for U-Net. To bridge this gap, we introduce a subtractive bit-encoding method that naturally realizes masked binary weights

Read the complete text