Just Pass Twice: Efficient Token Classification with LLMs for Zero-Shot NER

Ahmed Ewais, Ahmed HashishAmr Ali [0.5em] WitnessAI[0.3em] {ahmed, ahmed.hashish, amr}@witness.ai · 2026

Read Just Pass Twice: Efficient Token Classification with LLMs for Zero-Shot NER by Ahmed Ewais, Ahmed HashishAmr Ali [0.5em] WitnessAI[0.3em] {ahmed, ahmed.hashish, amr}@witness.ai (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-05158.txt.

Introduction

Ewais, Hashish, and Ali propose Just Pass Twice, a method that lets causal models do classification with bidirectional context for named entity recognition. Causal attention lets tokens see only preceding context, blocking disambiguation that needs context. Generative approaches that list entities suffer slow decoding and errors. The key insight: concatenating the input to itself lets each token in the second pass attend to the sentence. With definition-guided entity embeddings, it beats the best by 7.9 F1 on average across CrossNER and MIT, running over 20 times faster than methods.

Catalog record

CREATORAhmed Ewais, Ahmed HashishAmr Ali [0.5em] WitnessAI[0.3em] {ahmed, ahmed.hashish, amr}@witness.ai
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55937
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.05158

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Just Pass Twice: Efficient Token Classification with LLMs for Zero-Shot NER Ahmed Ewais, Ahmed HashishAmr Ali [0.5em] WitnessAI[0.3em] {ahmed, ahmed.hashish, amr}@witness.ai Abstract Large language models encode extensive world knowledge valuable for zero-shot named entity recognition. However, their causal attention mechanism, where tokens attend only to preceding context, prevents effective token classification when disambiguation requires future context. Existing approaches use LLMs generatively, prompting them to list entities or produce structured outputs, but suffer from slow autoregressive decoding, hallucinated entities, and formatting errors. We propose Just Pass Twice (JPT) , a simple yet effective method that enables causal LLMs to perform discriminative token classification with full bidirectional context. Our key insight is that concatenating the input to itself lets each token in the second pass attend to the complete sentence, requiring no architectural modifications. We combine these representations with definition-guided entity embeddings for flexible zero-shot generalization. Our approach achieves state-of-the-art results on zero-shot NER benchmarks, surpassing the previous best method by +7.9 F1 on average across CrossNER and MIT benchmarks, being over 20 × \times faster than comparable generative methods. Code and pretrained model weights will be available through our project page . † † footnotetext: † Equal contribution. † † footnotetext: Project page: https://witness-ai-jpt-ner.hf.space/ Figure 1: Just Pass Twice (JPT) enables bidirectional token classification in causal LLMs. (Top) Standard causal masking restricts the target token “Paris” from attending to future context like “album” (red barrier), leading to ambiguity. (Bottom) JPT duplicates the input sequence. In the second pass, the target token (green box) attends backwards to the entire original sequence (green arrows), resolving the entity type without architectural modifications. 1 Introduction Named Entity Recognition (NER) is a foundational natural language processing task, underpinning numerous downstream applications such as information extraction, privacy-preserving text processing, knowledge graph construction, entity linking, question answering, and document understanding pipelines ( 15 ) . Early approaches framed NER as a token-level sequence labeling problem, most commonly using the BIO tagging scheme ( 6 ; 1 ; 22 ; 8 ) , where every token is assigned a label indicating whether it begins, is inside, or is outside an entity mention. The dominant approach to NER has been discriminative token classification using bidirectional encoders such as BERT ( 8 ) , RoBERTa ( 19 ) , and DeBERTa ( 12 ) . This paradigm underlies a wide range of NER systems, including BioNER based on BERT ( 7 ) , RoBERTa-based approaches such as NuNER ( 4 ) , and DeBERTa-based systems such as GLiNER ( 33 ) . These models naturally support token-level labeling: their bidirectional attention allows each token to attend to both preceding and subsequent context, enabling effective disambiguation. However, encoder-based methods face inherent limitations: they are typically small ( < 1B parameters), have limited context windows, and encode less world knowledge than larger models, which can limit generalization to novel entity types and domains. Large language models (LLMs) offer a compelling alternative. With billions of parameters and training on vast corpora, LLMs encode rich world knowledge and demonstrate remarkable reasoning capabilities across diverse NLP tasks. These properties make them seemingly ideal for zero-shot NER. However, existing approaches to LLM-based NER diverge from the sequence labeling paradigm, instead employing generative formulations. Recent work has fine-tuned open-source LLMs on diverse NER datasets to enhance domain adaptability: InstructUIE ( 30 ) trains on a wide range of information extraction datasets using instruction tuning; UniversalNER ( 35 ) distills from ChatGPT and queries entity types one at a time for improved recall; and GoLLIE ( 24 ) uses code-style annotation guidelines to improve zero-shot generalization. Empirical evaluations show that even “vanilla” prompting of large models like ChatGPT yields suboptimal results compared to smaller supervised baselines ( 31 ; 16 ) . Despite these advances, all generative NER approaches suffer from fundamental drawbacks: (1) Speed : autoregressive token generation is inherently slow compared to single-pass classification; (2) Cost : output tokens cost 3–4 \times more than input tokens in commercial APIs because decoding is sequential and memory-bound, requiring a forward pass per generated token; (3) Hallucinations and Format Errors : Generative models are prone to hallucinating entities not present in the input and can produce outputs that fail to parse into valid structured data, requiring error handling or re-prompting ( 16 ; 29 ) . A natural question arises: why not apply the successful token classification paradigm from encoders directly to LLMs? The fundamental obstacle is architectural. Modern LLMs employ causal (unidirectional) attention, where each token can only attend to itself and preceding tokens. While essential for efficient autoregressive generation, this creates a critical limitation for token classification: when classifying a token, the model lacks access to subsequent context that may be essential for disambiguation. Consider the sentence “Paris released a new album” (Figure 1 ). To correctly classify Paris as a person rather than a location, a model must see the full sentence; the noun phrase “a new album” reveals this refers to the musician, not the city. Under causal attention, when the model processes Paris , it has not yet observed released , new , or album . The causal mask prevents the classifier from accessing this disambiguating future context, which is precisely why standard token classification fails with decoder-only LLMs. We propose Just Pass Twice (JPT), a method that bridges this gap through two key innovations. First, we enable bidirectional token classification in causal LLMs without architectural changes. Our insight is simple: concatenating the input sequence to itself allows each token in the second pass to attend to the complete sentence from the first pass. While recent works duplicate inputs and pool the sequence into a single global vector to improve text embeddings for retrieval tasks ( 25 ) , JPT leverages duplication specifically to bypass the causal attention bottleneck, extracting unpooled, bidirectional hidden states for individual tokens to perform fine-grained NER. Despite doubling the input length, the operation occurs entirely in the highly parallel prefill phase. Unlike autoregressive decoding, which is sequential and memory-bound, this makes JPT dramatically faster than generative alternatives. Second, we introduce definition-guided entity typing for flexible zero-shot generalization. Rather than encoding entity types by name alone (e.g., “PERSON”), we encode rich natural language definitions that precisely specify what each type encompasses. We inject definitions through two complementary channels: (1) as embeddings that the classifier matches against token representations, and (2) directly in the LLM’s input prompt, where they guide token encoding via attention. Inspired by recent work showing that definitions outperform simple label names for zero-shot NER ( 7 ) , this approach decouples the model from any fixed label vocabulary while providing fine-grained control: users can specify boundary cases directly in natural language, such as whether PRICE captures only explicit amounts (“$50”) or also qualitative descriptors (“budget-friendly”). Since definitions remain fixed at inference time, entity embeddings are computed offline and cached, adding no runtime overhead. We implement JPT by adding lightweight LoRA adapters ( 14 ) , projection layers, and a bilinear classifier to frozen Qwen3 backbones (4B and 8B parameters), trained on an in-house Wikipedia-derived NER dataset with no overlap with evaluation benchmarks. On zero-shot evaluation across CrossNER and MIT benchmarks, JPT surpasses state-of-the-art methods by +7.9 F1 on

Read the complete text