LiVeAction: a Lightweight, Versatile, and Asymmetric Neural Codec Design for Real-time Operation

Dan Jacobellis, Neeraja J. Yadwadkar · 2026

Read LiVeAction: a Lightweight, Versatile, and Asymmetric Neural Codec Design for Real-time Operation by Dan Jacobellis, Neeraja J. Yadwadkar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-06628.txt.

Introduction

Dan Jacobellis and Neeraja J. Yadwadkar design LiVeAction, a lightweight neural codec for sensors that must send data over tight bandwidth and power budgets. Standard codecs like JPEG target human perception, limiting use for machine tasks or odd modalities such as hyperspectral and 3D medical images. Generative codecs model signals well but are heavy, data-hungry, and modality-specific.

Their asymmetric design shrinks the encoder with an FFT-like structure and fewer layers, while a variance-based rate penalty replaces adversarial and perceptual losses to simplify training across modalities. The result beats leading generative tokenizers on rate-distortion while staying deployable on low-power devices, with code and a Python library released.

Catalog record

CREATORDan Jacobellis, Neeraja J. Yadwadkar
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
LICENSEhttp://creativecommons.org/publicdomain/zero/1.0/
RIGHTSCC0
SOURCEarXiv

Source: https://arxiv.org/abs/2605.06628

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LiVeAction: a Lightweight, Versatile, and Asymmetric Neural Codec Design for Real-time Operation Dan Jacobellis, Neeraja J. Yadwadkar Abstract Modern sensors generate rich, high-fidelity data, yet applications operating on wearable or remote sensing devices remain constrained by bandwidth and power budgets. Standardized codecs such as JPEG and MPEG achieve efficient trade-offs between bitrate and perceptual quality but are designed for human perception, limiting their applicability to machine-perception tasks and non-traditional modalities such as spatial audio arrays, hyperspectral images, and 3D medical images. General-purpose compression schemes based on scalar quantization or resolution reduction are broadly applicable but fail to exploit inherent signal redundancies, resulting in suboptimal rate-distortion performance. Recent generative neural codecs, or tokenizers, model complex signal dependencies but are often over-parameterized, data-hungry, and modality-specific, making them impractical for resource-constrained environments. We introduce a Li ghtweight, Ve rsatile, and A symmetric neural codec architecture (LiVeAction), that addresses these limitations through two key ideas. (1) To reduce the complexity of the encoder to meet the resource constraints of the execution environments, we impose an FFT-like structure and reduce the overall size and depth of the neural-network-based analysis transform. (2) To allow arbitrary signal modalities and simplify training, we replace adversarial and perceptual losses with a variance-based rate penalty. Our design produces codecs that deliver superior rate-distortion performance compared to state-of-the-art generative tokenizers, while remaining practical for deployment on low-power sensors. We release our code, experiments, and python library at https://github.com/UT-SysML/liveaction . I Introduction Modern sensors—from wearables and medical devices to satellites—generate rich streams of high-resolution data [ 1 , 2 ] . Efficient compression is critical for applications in health monitoring, remote sensing, and autonomous systems, as these deployments operate under strict power and bandwidth constraints. Standardized codecs (JPEG and MPEG) provide strong bitrate–quality trade-offs at low computational cost, but their human-centric design makes them unsuitable for machine-perception tasks and non-standard modalities where perceptual quality is not the target [ 3 ] . General-purpose methods, such as scalar quantization [ 4 ] and resolution reduction [ 5 ] remain widely used for their simplicity and universality. They apply to arbitrary signals, provide analytical guarantees on information loss, and combine easily with domain-specific approaches [ 6 , 7 ] . But, being agnostic to real-world data, they fail to exploit inherent redundancies, leading to poor rate–distortion performance [ 8 ] . Recent advances in deep neural network (DNN)–based autoencoders [ 9 , 10 ] and generative codecs [ 11 , 12 ] show that data-driven models can capture complex signal dependencies, greatly improving compression efficiency and realism. These tokenizer-style codecs use learned transforms and perceptual losses to reconstruct high-quality outputs at low bitrates but remain impractical for resource-constrained settings. Their deep, wide encoders dominate computational cost, and their architectures are often modality-specific. Additionally, generative codecs often depend on perceptual or adversarial losses tuned to human perception, making them ill-suited for scientific or machine-perception tasks. Such objectives are undefined for many signal types and can destabilize training, preventing these models from serving as general-purpose codecs, especially in low-power or embedded settings. To address these limitations, we propose LiVeAction, a Li ghtweight, Ve rsatile, and A symmetric neural codec designed to achieve efficient, high-fidelity compression across diverse signal modalities. LiVeAction is built to meet three primary goals: (1) extreme computational encoding efficiency, (2) competitive rate–distortion performance, and (3) versatility across signal modalities. Fig. 1: Rate-distortion-complexity trade-off for RGB images measured on the kodak dataset. BD-rate is averaged between SSIM and DISTS [ 13 ] ). Throughput is measured on a low power mobile CPU (Intel Arrow Lake 255U). Extreme computational encoding efficiency. Real-time sensing on mobile or remote platforms demands encoders that are computationally efficient and power-conscious. Most neural autoencoders use symmetric architectures, where analysis and synthesis transforms share nearly identical DNN layers [ 14 , 12 ] . However, increasing encoder depth or width yields diminishing returns [ 15 ] . LiVeAction adopts an asymmetric design with a lightweight encoder that minimizes computation while preserving representational quality. LiVeAction improves efficiency using structured, FFT-inspired operations instead of dense projections. These impose a block-diagonal structure reminiscent of ShuffleNet [ 16 ] and Monarch matrices [ 17 , 18 ] , allowing multiple layers with alternating nonlinear activations at roughly the cost of one dense layer. Competitive rate-distortion performance. To enable applications with severe bandwidth limitations, the rate-distortion performance must match or exceed conventional standards like JPEG or MPEG. Existing autoencoder designs (e.g. Stable diffusion [ 19 ] , Stable Audio [ 14 ] , and Cosmos [ 12 ] ) rely heavily on perceptual and adversarial losses, enabling the decoder to synthesize realistic, but hallucinated details. Prior work shows that removing these losses can improve compressed-domain learning by maximizing the dimension–distortion trade-off [ 3 ] . In LiVeAction, the training objective is purely to optimize the rate-distortion trade-off, similar to learned image compression systems [ 9 ] . To simplify the training process and increase accessibility for new modalities, we replace the continuously-relaxed probability model and auxiliary optimizer with a simplified rate penalty based on the sample variance. Compared to codecs with generative or adversarial losses, this formulation requires fewer hyperparameters and provides stable training for a wide range of signal types using thousands, rather than millions, of training examples. Versatility for use with any modality. LiVeAction is designed for architectural and loss-function generality to support diverse sensing applications. Prior autoencoders are often tied to specific modalities through custom objectives such as LPIPS [ 20 ] , optical flow loss [ 12 ] , or adversarial losses [ 21 , 22 ] . In contrast, LiVeAction shows that a simple mean-squared-error (MSE) based rate–distortion objective suffices across modalities, eliminating the need for perceptual losses. Existing DNN architecture designs also limit versatility. The convolutional and transformer-based architectures underlying previous autoencoders are meticulously engineered for specific modalities. LiVeAction’s analysis and synthesis transforms are modality-agnostic and apply to any uniformly grid-sampled signal. Additionally, simple heuristics are sufficient to choose hyperparameters, avoiding costly searches when adapting to new sensors. Together, these design choices reduce development cost while maintaining strong performance across various modalities. Contributions. Using LiVeAction, we create codecs for a wide range of signal types—spatial audio arrays, hyperspectral images, and 3D medical CT—as well as standard audio, image and video signals. Even compared to state-of the art neural tokenizers using modality-specific designs and trained with orders of magnitude more data and compute, we show improvements in the rate-distortion-complexity trade-off. For example, compared to Cosmos [ 12 ] , LiVeAction provides a 34% BD-rate improvement while encoding more than 10 \times faster (see Fig. 1 ). II Background and related work We build on prior work in (1) high-throughput, training-free lossy compression, (2) autoencoder design for compressed learning and generative modeling, and (3) efficiency optimizations in convolution- and attention-based neural network layers. Computationally efficient lossy compression. Transform-based standards such as JPEG and MPEG remain dominant for their strong trade-offs among rate, distortion, and computational cost. They combine energy-compacting transforms with tuned quantization matrices to minimize perceptual distortion for human observers. However, many signals fall outside standard audio, image, or video modalities, where imperceptible details may still matter. In such cases, training-free codecs based on scalar quantization offer high throughput and bounded

Read the complete text