Focus Directions Make Your Language Models Pay More Attention to Relevant Contexts

Youxiang Zhu, Ruochen Li, Danqing Wang, Daniel Haehn, Xiaohui Liang · 2025

Read Focus Directions Make Your Language Models Pay More Attention to Relevant Contexts by Youxiang Zhu, Ruochen Li, Danqing Wang, Daniel Haehn, Xiaohui Liang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-23306.txt.

Introduction

Youxiang Zhu, Ruochen Li, Danqing Wang, Daniel Haehn, and Xiaohui Liang wrote this paper in two thousand twenty-five. It traces why long-context language models get distracted, to a small crew of attention heads, and finds a way to steer them.

Long inputs bury the key facts in noise. The authors spot the contextual heads, the few that set where the model looks as a whole. Distraction strikes when those heads short the key spans. Inside their key and query states sit focus directions, pushes that lift attention to the right spans without naming them. Across long-context tasks the pushes mend much of the drift. The find could guide fresh work on how long-context models are aligned.

Catalog record

CREATORYouxiang Zhu, Ruochen Li, Danqing Wang, Daniel Haehn, Xiaohui Liang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-23306

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Focus Directions Make Your Language Models Pay More Attention to Relevant Contexts Youxiang Zhu, Ruochen Li, Danqing Wang, Daniel Haehn, Xiaohui Liang Abstract Long-context large language models (LLMs) are prone to be distracted by irrelevant contexts. The reason for distraction remains poorly understood. In this paper, we first identify the contextual heads, a special group of attention heads that control the overall attention of the LLM. Then, we demonstrate that distraction arises when contextual heads fail to allocate sufficient attention to relevant contexts and can be mitigated by increasing attention to these contexts. We further identify focus directions, located at the key and query activations of these heads, which enable them to allocate more attention to relevant contexts without explicitly specifying which context is relevant. We comprehensively evaluate the effect of focus direction on various long-context tasks and find out focus directions could help to mitigate the poor task alignment of the long-context LLMs. We believe our findings could promote further research on long-context LLM alignment. 1 Introduction Long-context large language models enable multiple applications, such as many-shot in-context learning Li et al. (2024c) ; Agarwal et al. (2025) ; Bertsch et al. (2024) , summarization Chang et al. (2023) ; Kim et al. (2024) , and retrieval-augmented generation Lee et al. (2024) . Given a long context window such as 128k tokens, only a small amount of the contexts are relevant to the task, and a large amount of contexts are irrelevant. Long context LLM may be distracted by irrelevant contexts Liu et al. (2024) ; Shi et al. (2023) . Such distraction may result in generating false information, error reasoning, and negative social impacts. The reason for LLMs being distracted by irrelevant context is poorly understood. In this paper, we aim to reveal the cause of the distraction (§ 2 ). As shown in Figure 1 , starting with a dataset with labels of relevant and irrelevant context, we first introduce a contextual scoring method, which measures the strength of the attention to the relevant context during text generation. Based on such a scoring method, we identify contextual heads , a special group of attention heads with the highest score. We then adjust the strength of attention of these heads to the relevant contexts based on the label of the relevant context. We found that increasing attention on these heads to the relevant context increases the downstream task performance, while decreasing attention decreases the performance. While other non-contextual heads have minimal such effects. We conclude that contextual heads could control the overall attention of LLMs to the contexts. Building upon the finding of the contextual head, we further wonder if a mechanism exists that could make LLMs figure out which is the relevant context by itself instead of relying on the external labels. Recent works on activation addition show that LLMs’ behavior, such as refusal Arditi et al. (2024) , sentiment Han et al. (2023) , and truthfulness Li et al. (2024b) can be changed by adding or subtracting a single directional vector in some intermediate activation space, following the linear representation hypothesis Park et al. (2023) . Inspired by such works, we hypothesize and verify the existence of focus directions (§ 3 ), which could enable LLMs to pay more attention to the relevant contexts. Focus directions are located at the key and query activations of the transformer attention heads. We found that adding focus directions vector to the key and query activation could increase the attention to the relevant context, and subtracting a direction could decrease the attention to the relevant context. Such focus directions enable the LLMs’ attention behavior control at inference time. To understand how focus directions affect the capability of long-context LLMs (§ 4 ), we apply focus directions to three families of LLMs and evaluate them on HELMET Yen et al. (2024) , a comprehensive long-context task benchmark. We found that focus directions could help mitigate poor task alignment of the LLMs. At last, we discuss the potential application of the focus directions. Figure 1: Overview of this work. We first introduce contextual scoring , measuring the attention distribution over inputs during response generation. Based on contextual scoring, we identify the contextual heads , which control the overall attention of LLMs. We further find out focus directions , which make LLMs pay more attention to the relevant contexts. 2 Cause of distraction To reveal the cause of LLMs being distracted by irrelevant contexts, we first identify the attention heads that are mostly responsible for extracting information from relevant contexts, which we named contextual heads § 2.1 . Then, we study the basic properties of the contextual heads, including their location and behavior in different cases § 2.2 . At last, we demonstrate that increasing attention to relevant contexts on these heads could mitigate distractions § 2.3 . 2.1 Identifying contextual heads To identify contextual heads, we introduce a contextual scoring method to identify the attention distribution of different parts of input for each attention head in the transformer architecture. Our method is based on the Multi-Document Question Answering (QA) data introduced by the “lost in the middle” paper Liu et al. (2024) . Multi-Document Question Answering data. The data is initiated with the NaturalQuestions-Open data Lee et al. (2019) ; Kwiatkowski et al. (2019) . Each samples have a question and a list of answers. The questions are user queries from Google search, and the answers are human-annotated based on Wikipedia. The authors of Liu et al. (2024) further matched each question and answer pair with a set of documents using a retrieval system. In these documents, only one contains the answer (i.e., relevant context), and others do not contain the answer (i.e., irrelevant contexts). Experiment settings. The above dataset has 2654 samples in total, we randomly split them into half training and half testing. The input is defined as [I_{p},\hat{C}_{before},C,\hat{C}_{after},I_{q}] , where I_{p} and I_{q} are instructions, specifying the QA task (e.g., a system prompt and a question). The C stands for relevant context, \hat{C}_{before} , and \hat{C}_{after} stands for the irrelevant contexts before and after the relevant context, which can be zero, one or more documents. We consider 20 document cases where one of the documents in the input is relevant and the rest of the 19 documents are irrelevant. We put the relevant documents in positions 1, 5, 10, 15, and 20. The input is fed into an LLM, in our case we use Llama-3.2-3B instruction model 1 1 1 https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct , to obtain an LLM response R using greedy decoding. The evaluation metric is the exact match (EM) accuracy. If the model output matches one of the answers in the output list, then it is considered to be correct; otherwise, it is wrong. Contextual scoring. Based on the above data and experiment settings, we introduce the following contextual scoring method, which aims to find a set of attention heads in the LLM that pay the most attention to the relevant contexts during generation. Let W\in\mathbb{R}^{T\times T} be the attention weight matrix of an attention head, where T is the sequence length. For each token r_{i} in the generated response R=[r_{start},\dots,r_{end}] , we extract the attention weights corresponding to

Read the complete text