Will It Still Be True Tomorrow? Multilingual Evergreen Question Classification to Improve Trustworthy QA
Sergey Pletenev, Maria Marina, Nikolay Ivanov, Daria Galimzianova, Nikita Krayko, Mikhail Salnikov, Vasily Konovalov, Alexander Panchenko, Viktor Moskvoretskii · 2025
Read Will It Still Be True Tomorrow? Multilingual Evergreen Question Classification to Improve Trustworthy QA by Sergey Pletenev, Maria Marina, Nikolay Ivanov, Daria Galimzianova, Nikita Krayko, Mikhail Salnikov, Vasily Konovalov, Alexander Panchenko, Viktor Moskvoretskii (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-21115.txt.
Introduction
Sergey Pletenev and co-authors ask a simple question with wide reach: will this question's answer still be true tomorrow? Their 2025 paper notes that language models often fabricate answers. One hidden cause is time: some questions are evergreen, with stable answers, while others change as the world moves, and models rarely tell the two apart.
The team built EverGreenQA, the first multilingual data set labeled for this trait. They tested twelve modern models on whether they can state or sense a question's stability. A small trained classifier beat them all at the task. Uses include checking model self-knowledge, cleaning question sets, and explaining search behavior.
Catalog record
| CREATOR | Sergey Pletenev, Maria Marina, Nikolay Ivanov, Daria Galimzianova, Nikita Krayko, Mikhail Salnikov, Vasily Konovalov, Alexander Panchenko, Viktor Moskvoretskii |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-21115
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Will It Still Be True Tomorrow? Multilingual Evergreen Question Classification to Improve Trustworthy QA Sergey Pletenev, Maria Marina, Nikolay Ivanov, Daria Galimzianova, Nikita Krayko, Mikhail Salnikov, Vasily Konovalov, Alexander Panchenko, Viktor Moskvoretskii Abstract 1 1 footnotetext: * Equal contribution. Large Language Models (LLMs) often hallucinate in question answering (QA) tasks. A key yet underexplored factor contributing to this is the temporality of questions – whether they are evergreen (answers remain stable over time) or mutable (answers change). In this work, we introduce EverGreenQA , the first multilingual QA dataset with evergreen labels, supporting both evaluation and training. Using EverGreenQA , we benchmark 12 modern LLMs to assess whether they encode question temporality explicitly (via verbalized judgments) or implicitly (via uncertainty signals). We also train EG-E5 , a lightweight multilingual classifier that achieves SoTA performance on this task. Finally, we demonstrate the practical utility of evergreen classification across three applications: improving self-knowledge estimation, filtering QA datasets, and explaining GPT-4o’s retrieval behavior. 1 Introduction Large language models (LLMs) often struggle with question answering (QA) due to hallucinated answers Huang et al. 2025 . To improve trustworthiness, recent research has focused on estimating LLMs’ self-knowledge – their ability to recognize what they do and do not know Yin et al. 2023 ; Moskvoretskii et al. 2025 – and on integrating up-to-date external information through Retrieval-Augmented Generation (RAG) Su et al. 2024 ; Jeong et al. 2024 ; Trivedi et al. 2023 . EverGreenQA TimeQA MuLan FreshQA TAQA (our work) Chen et al. 2021 Fierro et al. 2024 Vu et al. 2024 Zhao et al. 2024 Both EG and mutable questions Train-Test split Human-Evaluated Multilinguality Overall size 4757 ~40k ~246k 600 ~20k Table 1: Comparison of EverGreenQA to other time-sensitive datasets. A particularly important but underexplored factor affecting question difficulty is whether a question is evergreen or mutable Wei et al. 2024a – that is, whether its correct answer remains stable over time, depicted with illustrative Figure 1 . Mutable questions are especially challenging because they often require access to up-to-date information, which may be missing from a model’s fixed, parametric knowledge. Despite its practical importance, evergreen-ness remains an underexplored factor in evaluating and improving LLM behavior. Most existing studies are limited to small-scale, English-only datasets and focus primarily on QA accuracy, rarely examining its broader implications Vu et al. 2024 ; Cheng et al. 2024 . As a result, the role of question evergreen-ness in shaping LLM reliability and interpretability remains largely unexamined. Figure 1: Some questions have answers that stay the same ( evergreen ), like facts of nature. Others have answers that change over time and will change in the future ( mutable ), like global trends or statistics. To address this gap, we conduct a comprehensive study of question evergreen-ness and its practical applications. We introduce EverGreenQA - the first multilingual human-curated evergreen-aware QA dataset, which includes a train–test split suitable for model training. Using EverGreenQA , we evaluate 12 modern LLMs to determine whether they encode temporal knowledge explicitly (through direct prompting) or implicitly (via uncertainty-based signals). Further, we develop EG-E5 – a lightweight SoTA classifier trained to identify evergreen questions. We demonstrate the usefulness of EG-E5 in several downstream tasks: (1) improving self-knowledge estimation, (2) curating QA datasets to support fairer evaluation, and (3) effectively explaining GPT-4o’s black-box retrieval behavior. Our contributions and findings are as follows: 1. We construct EverGreenQA – the first multilingual dataset for question evergreen-ness classification, covering 7 languages with 4,757 examples in total. 2. We conduct the first comprehensive evaluation of question evergreen knowledge in LLMs, assessing 12 models using both explicit signals (via prompting) and implicit signals (via uncertainty estimation). 3. We develop EG-E5 – a multilingual lightweight classifier for identifying evergreen questions, which serves as SoTA approach for question evergreen-ness classification while remaining suitable for low-compute settings. 4. We demonstrate the utility of EG-E5 across three applications: (1) improving self-knowledge estimation, (2) curating QA datasets for fairer evaluation, and (3) effectively explaining GPT-4o’s retrieval behavior. We release the model and data for further usage. 1 1 1 https://huggingface.co/collections/s-nlp/evergreen-683465909575cb89d6b904fe 2 Related Work Reasoning about time remains a fundamental challenge in question answering (QA) tasks, as temporal dynamics often complicate both the interpretation of questions and the retrieval of accurate answers. Working with time in QA tasks has been improved thanks to datasets like TimeQA Chen et al. 2021 , which has 20,000 question-answer pairs that need thinking about time. While helpful, it only addressed simple reasoning. SituatedQA Zhang and Choi 2021 showed the importance of context by situating questions in time and place. StreamingQA highlighted the need for temporal adaptation, revealing LLMs’ difficulty tracking changing facts Liska et al. 2022 . TemporalAlignmentQA (TAQA) Zhao et al. 2024 further enhances possibility for temporal alignment by providing 20K time-sensitive questions and their answers for each year from 2000 to 2023. MuLan Fierro et al. 2024 differentiated questions by change rate and fact type, respectively. Most recently, FreshQA Vu et al. 2024 introduced a benchmark focused on freshness-sensitive information, further illustrating LLMs’ limitations in handling temporally dynamic knowledge. These studies indicate a need for specialized temporal reasoning Fierro et al. 2024 . The comparison of datasets is presented in Table 1 . Retrieval-Augmented Generation (RAG), such as DRAGIN Su et al. 2024 , IRCoT Trivedi et al. 2023 or Rowen Ding et al. 2024 was used to solve the problem of time sensitive QA, addressed this through dynamic retrieval decisions, but showed limited results. Dynamic retrieval decisions required self-knowledge estimation. In other words, before QA systems can be trusted, they need to know what they don’t know. Often, LLMs struggle to identify questions they can’t answer Yin et al. 2023 , but using self-knowledge can reduce mistakes in tasks that need a lot of knowledge Wang et al. 2023 ; Moskvoretskii et al. 2025 . While retrieval-based methods address temporal knowledge gaps externally, another direction is to update the internal knowledge of LLMs. Updating internal knowledge in LLMs is computationally expensive, as retraining or editing models often requires substantial resources and cannot be performed daily or hourly in practice. Techniques like LLM Surgery Veldanda et al. 2024 and parameter-efficient fine-tuning Ge et al. 2024 ; Pletenev et al. 2025 have attempted to make such updates more practical, but still face issues with large-scale changes or factual hallucinations. Model Russian English French German Hebrew Arabic Chinese AVG LLaMA 3.1-8B-it 0.677 0.699 0.686 0.677 0.667 0.659 0.652 0.674 LLaMA 3.1-70B-it 0.889 0.879 0.895 0.87 0.874 0.829 0.873 0.875 Qwen 2.5 7B-it 0.782 0.789 0.786 0.794 0.692 0.711 0.774 0.761 Qwen 2.5 32B-it 0.882 0.885 0.875 0.883 0.862 0.862 0.872 0.874 Qwen 2.5 72B-it 0.806 0.815 0.802 0.805 0.781 0.758 0.768 0.791 Phi-3 medium 4k-it 0.556 0.577 0.498 0.473 0.499 0.498 0.420 0.503 Phi-3 medium 128k-it 0.415 0.489 0.342 0.335 0.385 0.304 0.289 0.366 Gemma 2-9B-it 0.755 0.728 0.694 0.723 0.740 0.711 0.746 0.728 Gemma 2-27B-it 0.830 0.878 0.836 0.827 0.838 0.831 0.826 0.838 Mistral 7B-it-v0.3 0.736 0.722 0.726 0.729 0.670 0.666 0.731 0.711 Mistral Small-24B-it-2501 0.827 0.739 0.768 0.789 0.847 0.834 0.839 0.806 GPT-4.1 0.806 0.794 0.816 0.813 0.803 0.811 0.809 0.807 UAR Cheng et al. 2024 0.550 0.500 0.510 0.600 0.670 0.710 0.710 0.490 MULAN Fierro