DiVeQ: Differentiable Vector Quantization Using the Reparameterization Trick
Mohammad Hassan Vali, Tom Bäckström & Arno Solin · 2025
Read DiVeQ: Differentiable Vector Quantization Using the Reparameterization Trick by Mohammad Hassan Vali, Tom Bäckström & Arno Solin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-26469.txt.
Description
Mohammad Hassan Vali, Tom Bäckström, and Arno Solin make a core compression step trainable end to end, in a 2025 paper. Vector quantization squeezes data by snapping each piece to the nearest entry in a learned codebook. The snapping is a hard choice, which blocks the gradient signals that training relies on.
Their method, DiVeQ, treats the snap as adding an error vector that mimics the distortion. The forward pass still snaps hard, but gradients flow past as if the step were smooth. A variant spreads inputs along a curve through the codebook, cutting error and using every codeword. Tests on image and speech coding beat rival approaches. Cleaner training for quantization helps every model that compresses through a codebook.
Catalog record
| CREATOR | Mohammad Hassan Vali, Tom Bäckström & Arno Solin |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 93 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-26469