DB-KSVD: Scalable Alternating Optimization for Disentangling High-Dimensional Embedding Spaces

Romeo Valentin, Sydney M. Katz, Vincent Vanhoucke, Mykel J. Kochenderfer · 2025

Read DB-KSVD: Scalable Alternating Optimization for Disentangling High-Dimensional Embedding Spaces by Romeo Valentin, Sydney M. Katz, Vincent Vanhoucke, Mykel J. Kochenderfer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18441.txt.

Description

Romeo Valentin and three co-authors present, in 2025 work, a faster way to pull apart the inner codes of large language models. Teams use sets of learned features, like words in a word book, to read what a model has stored. The usual tools use a simple first step, though the true task is known to be very hard.

Their method, DB-KSVD, adapts a classic word-book method and splits the work into small batches. That lets it run on millions of items across thousands of code slots. Tested on two text models and two image models, it matched the usual tools on six checks. The team reads that match two ways: the usual tools work well, and classic maths can scale to modern jobs.

Catalog record

CREATORRomeo Valentin, Sydney M. Katz, Vincent Vanhoucke, Mykel J. Kochenderfer
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-18441

Browse the full library · About the Tower of Babel