Interpret and Control Dense Retrieval with Sparse Latent Features

Hao Kang, Tevin Wang, Chenyan Xiong · 2024

Read Interpret and Control Dense Retrieval with Sparse Latent Features by Hao Kang, Tevin Wang, Chenyan Xiong (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-00786.txt.

Description

A 2024 arXiv research paper by Hao Kang, Tevin Wang, Chenyan Xiong. Dense embeddings deliver strong retrieval performance but often lack interpretability and controllability. This paper introduces a novel approach using sparse autoencoders (SAE) to interpret and control dense embeddings via the learned latent sparse features. Our key contribution is the development of a retrieval-oriented contrastive loss, which ensures the sparse latent features remain effective for retrieval tasks and thus meaningful to interpret.

Catalog record

CREATORHao Kang, Tevin Wang, Chenyan Xiong
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2411-00786

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Interpret and Control Dense Retrieval with Sparse Latent Features Hao Kang, Tevin Wang, Chenyan Xiong Abstract Dense embeddings deliver strong retrieval performance but often lack interpretability and controllability. This paper introduces a novel approach using sparse autoencoders (SAE) to interpret and control dense embeddings via the learned latent sparse features. Our key contribution is the development of a retrieval-oriented contrastive loss, which ensures the sparse latent features remain effective for retrieval tasks and thus meaningful to interpret. Experimental results demonstrate that both the learned latent sparse features and their reconstructed embeddings retain nearly the same retrieval accuracy as the original dense vectors, affirming their faithfulness. Our further examination of the sparse latent space reveals interesting features underlying the dense embeddings and we can control the retrieval behaviors via manipulating the latent sparse features, for example, prioritizing documents from specific perspectives in the retrieval results. Figure 1: An overview of our framework. We first train the k -sparse autoencoder with our retrieval-oriented contrastive loss, which produces sparse latent features that are effective for retrieval. Next, we interpret these latents using N2G approach and demonstrate controllability via retrieval on the manipulated embeddings. 1 Introduction In the realm of information retrieval, dense embeddings derived from large language models (LLMs) have achieved state-of-the-art performances Khattab and Zaharia 2020 ; Reimers 2019 . While these representations offer remarkable accuracy in matching queries to documents, their “black-box” nature poses challenges in applications that demand transparency and control, such as retrieval in bias-sensitive tasks, where users may need to understand the rationale behind the retrieved results and adjust the process to ensure fairness. In contrast, in bag-of-word base sparse retrieval, each dimension is a meaningful word, allowing users to see why certain documents are retrieved, and making it intuitive for users to revise their query keywords to control the retrieval results. Interpretability and controllability are important for building trust with users and facilitate the wide adoption of search technologies Croft et al. 2010 . In this paper, we present a novel approach that leverages sparse autoencoders (SAE) to interpret and control dense retrieval systems. Sparse autoencoders have recently been used to improve the interpretability of LLMs by transforming neuron activation patterns into sparse dictionaries ( Bricken et al. 2023 ; Templeton et al. 2024 ) . We upgrade this approach to dense embeddings, incorporating a retrieval-oriented recovery loss which ensures the extracted sparse features remain faithful for retrieval, forming the basis of our interpretability analysis. Our experiments demonstrate the success of this approach. Retrieval using the learned latent sparse features and their reconstructed embeddings both recover the majority of the original dense retrieval accuracy on the MsMarco and Beir benchmarks, ensuring that these features offer genuine interpretability rather than an illusion. Then we explore the interpretability of these sparse features with Neuron to Graph (N2G) approach ( Foote et al. 2023 ) , and discover that various fine-grained concepts have been captured in the latent sparse space. To understand controllability through latent features, we conduct quantitative studies by amplifying query-relevant features, which successfully improved retrieval accuracy on the manipulated embeddings, both on the query side and the document side. Then, we perform case studies on multi-perspective queries and confirm that selectively manipulating sparse features from a specific perspective causes the reconstructed embeddings to prioritize documents from that perspective during retrieval. Our source code and extracted features are available at GitHub 1 1 1 https://github.com/cxcscmu/embedding-scope . 2 Methodology In this section, we describe the methodology used to train the sparse autoencoder with our retrieval-oriented recovery loss. As illustrated in Figure 1 , for an embedding vector x\in\mathbb{R}^{d} , we employ the k -sparse autoencoder as proposed in Makhzani and Frey 2013 , which controls the number of active latent features using the TopK activation function. The encoder and decoder are described in Equation 1 , where n denotes the latent dimension for W_{\text{enc}}\in\mathbb{R}^{n\times d} . The reconstructed embedding is represented by \hat{x}\in\mathbb{R}^{d} . \displaystyle h \displaystyle=\text{TopK}(W_{\text{enc}}(x-b_{\text{dec}})+b_{\text{enc}}) (1) \displaystyle\hat{x} \displaystyle=W_{\text{dec}}h+b_{\text{dec}} Building on previous efforts to extract interpretable features from LLMs ( Gao et al. 2024 ; Bricken et al. 2023 ; Lieberum et al. 2024 ) , we incorporate mean-squared error (MSE) as part of the training objective for reconstruction. By minimizing the squared differences, MSE forces each dimension of the reconstructed embedding to closely approximates the original value. However, the focus of MSE is to minimize the error for individual points in the embedding space. It does not explicitly account for the relative positioning. For information retrieval, embeddings are typically divided into queries and documents, with the need to effectively capture the relevance between a query and its associated documents. Therefore, we employ contrastive learning via Kullback–Leibler divergence (KLD) to ensure that the distribution of reconstructed query and document embedding aligns with the original Xiong et al. 2021 ; Liu et al. 2022 . The formulation of the loss function is presented in Equation 2 , where q represents the query embedding, D^{+} denotes the relevant documents, and f(q,d) computes the retrieval score, such as dot product. \displaystyle\mathcal{L}_{\text{KLD}}=\sum_{q} \displaystyle\sum_{d\in D^{+}}P(q,d)\times\log{\frac{P(q,d)}{P(\hat{q},\hat{d})}} (2) \displaystyle\text{where} \displaystyle P(q,d)=\frac{e^{f(q,d)}}{\sum_{D^{+}}e^{f(q,d)}} In short, the k -sparse autoencoder is trained with MSE for accurate reconstruction and KLD to preserve the query-document relationship. Table 1: Reconstruction evaluation of sparse latent features and the reconstructed embeddings learned by our k -sparse autoencoder. MSE measures the embedding differences between original and reconstructed embeddings. MsMarco Beir MSE MRR P@10 R@10 MSE MRR P@10 R@10 Original – 0.3605 0.0649 0.6211 – 0.3699 0.0891 0.5415 Sparse Latent (K=32) – 0.2721 0.0507 0.4869 – 0.2420 0.0581 0.3590 Sparse Latent (K=64) – 0.3062 0.0564 0.5406 – 0.2923 0.0708 0.4212 Sparse Latent (K=128) – 0.3306 0.0601 0.5760 – 0.2981 0.0735 0.4461 Reconstructed (K=32) 0.00022 0.2984 0.0552 0.5291 0.00043 0.2549 0.0619 0.3768 Reconstructed (K=64) 0.00017 0.3194 0.0583 0.5589 0.00033 0.2913 0.0721 0.4361 Reconstructed (K=128) 0.00011 0.3455 0.0626 0.5991 0.00019 0.3407 0.0818 0.4954 (a) MsMarco (Rec.) (b) MsMarco (Spr.) (c) BEIR (Rec.) (d) BEIR (Spr.) Figure 2: Retrieval performance of reconstructed (Rec.) embeddings and the sparse latent features (Spr.) before and after the contrastive loss KLD is applied, on MsMarco and Beir . 3 Experiments This section outlines the training procedures for the k -sparse autoencoder and our experiments on interpretability and controllability. Training Procedures. We train the autoencoder on top of the base-sized Bge model 2 2 2 https://huggingface.co/BAAI/bge-base-en-v1.5 , which was trained on diverse tasks such as retrieval, classification, and semantic similarity Xiao et al. 2023 . Embeddings are generated from the MsMarco dataset, containing 8.8M passages for retrieval tasks ( Bajaj et al. 2016 ) . Details of the training hyperparameters are available in Appendix A . For evaluation, we first calculate MSE on the validation queries and their relevant documents. We then perform dense retrieval on the reconstructed embeddings and sparse dot product retrieval on the latent features. Reported metrics include mean reciprocal rank (MRR), precision at rank 10 (P@10), and recall at rank 10 (R@10). For generalizability on diverse retrieval tasks, we additionally evalute the sparse autoencoder on datasets from the Beir benchmark, such as TrecCovid , NaturalQuestions , and DBPediaEntity ( Kwiatkowski et al. 2019 ; Hasibi et al. 2017 ; Thakur et al. 2021 ) . Figure 3: Frequency distribution

Read the complete text