ADAPT: Hybrid Prompt Optimization for LLM Feature Visualization

João Cardoso, Arlindo L. Oliveira, Bruno Martins · 2026

Read ADAPT: Hybrid Prompt Optimization for LLM Feature Visualization by João Cardoso, Arlindo L. Oliveira, Bruno Martins (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-17867.txt.

Description

João Cardoso and team introduce ADAPT, a hybrid prompt tuning method for language model feature viewing. Grasping what features are stored by learned paths in activation space requires inputs that fire them hard. Feature viewing tunes inputs to maximally fire a target path, offering an alternative to costly dataset search, but it remains underexplored for language models given the discrete nature of text.

Existing prompt tuning tricks suit this domain poorly, which is highly prone to local minima. ADAPT combines beam search start with tuning gradient-guided mutation, designed around those failure modes. Tests on sparse autoencoder latents from Gemma 2 2B use metrics grounded in dataset firing stats for strict comparison, showing ADAPT always beating prior methods across layers and latent types. Results show that feature viewing for language models is doable, but requires design guesses tailored to the domain.

Catalog record

CREATORJoão Cardoso, Arlindo L. Oliveira, Bruno Martins
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-17867

Browse the full library · About the Tower of Babel