InFact: Informativeness Alignment for Improved LLM Factuality
Roi Cohen, Russa Biswas, Gerard de Melo · 2025
Read InFact: Informativeness Alignment for Improved LLM Factuality by Roi Cohen, Russa Biswas, Gerard de Melo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-20487.txt.
Introduction
A true sentence can still be a lazy one: 'born in the United States' informs less than 'born in Honolulu'. Roi Cohen, Russa Biswas, and Gerard de Melo argue in 2025 that language models, beyond hallucinating, often pick the correct-but-thin option. Their mechanism, InFact, aligns models toward informativeness using scores from recent factuality benchmarks.
Training or preference-tuning on that objective lifted informativeness as intended. The surprise: factual accuracy rose with it, as if aiming for fuller truth discourages fabrication as a side effect. The authors present the alignment as a cheap upgrade for any system graded on facts.
Catalog record
| CREATOR | Roi Cohen, Russa Biswas, Gerard de Melo |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-20487
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
InFact: Informativeness Alignment for Improved LLM Factuality Roi Cohen, Russa Biswas, Gerard de Melo Abstract Factual completeness is a general term that captures how detailed and informative a factually correct text is. For instance, the factual sentence “Barack Obama was born in the United States” is factually correct, though less informative than the factual sentence “Barack Obama was born in Honolulu, Hawaii, United States” . Despite the known fact that LLMs tend to hallucinate and generate factually incorrect text, they might also tend to choose to generate factual text that is indeed factually correct and yet less informative than other, more informative choices. In this work, we tackle this problem by proposing an informativeness alignment mechanism. This mechanism takes advantage of recent factual benchmarks to propose an informativeness alignment objective. This objective prioritizes answers that are both correct and informative. A key finding of our work is that when training a model to maximize this objective or optimize its preference, we can improve not just informativeness but also factuality. 1 1 1 We release our code and data at https://github.com/roi-hpi/informativeness-alignment . Figure 1: Illustration that an LLM knows a more informative answer than its initial answer. 1 Introduction Large language models (LLMs) are known to capture and store extensive amounts of factual knowledge Petroni et al. (2019) ; Brown et al. (2020) ; Roberts et al. (2020) ; Cohen et al. (2023a) ; Pan et al. (2023) , as they are trained on vast quantities of text, which includes a significant body of factual knowledge. However, they often hallucinate or generate factually incorrect text Maynez et al. (2020) ; Devaraj et al. (2022) ; Tam et al. (2023) ; Kaddour et al. (2023) ; Huang et al. (2024) . Although the way LLMs represent their knowledge remains unclear Rai et al. (2024) , it can be effectively accessed via prompting Veseli et al. (2023) . For example, modern LLMs are likely to correctly complete the input prompt: “Barack Obama was born in” , with multiple different answers – United States , Hawaii and Kapiolani Medical Center for Women & Children . While these answers are all factually correct, they differ greatly in specificity and significantly in the level of their informativeness. This highlights an important gap: even when LLMs do not hallucinate, they often fail to provide the most informative answer. Therefore, in this work, we focus beyond the factual correctness of the LLMs to examine the informativeness by evaluating how well the correct entities are represented in the LLM outputs. An answer is considered highly informative if it includes either the most specific entity or all the correct entities. 2 2 2 In this study, we assume that, unless stated otherwise, the most informative answer that is also fully correct is preferred. While other preferences may be chosen for different users or applications, we argue that to produce increasingly reliable, trustworthy, and knowledgeable models it is advantageous to possess both maximal informativeness and factuality. Our main assumption and motivation for this work is that an LLM might indeed have correct and informative parametric knowledge about a given query, yet generate a less informative answer – as illustrated in Figure 1 . This might occur due to statistical or even spurious correlations between the input text (prompts or questions) and the correct yet less informative answer. For example, “Barack Obama” and “The United States” is more likely substantially co-occur higher than “Barack Obama” and “Kapiolani Medical Center for Women & Children” in the training data, leading to the LLM to prefer less specific but more frequent answer. Therefore, we aim to align the model to prefer the most informative or specific answer the LLM knows. Additionally, we also address the well-known existing factual precision problem of the LLMs Augenstein et al. (2023) by incorporating this into our alignment mechanism. To this end, we begin by formulating the informativeness-evaluation task and propose a novel framework to create a general informativeness dataset. In this dataset, each question or input text is paired with a set of answers associated with a hierarchical set of labels, representing different levels of informativeness. This dataset structure, which includes the informativeness metadata for each label is incorporated directly into the training. Building on this dataset, we propose the novel training framework InFACT , to align a pre-trained or instruction-tuned LLM model to generate more informative and complete facts. This procedure consists of two stages – Structure Tuning and Informativeness-Alignment . Structure Tuning phase aims to teach the model to consider the informativeness problem setup as well as to abstain from generating misinformation, whereas the Informativeness-Alignment seeks to train the model to answer with the most informative and complete factual answers to the questions. Our contributions can be summarized as follows: 1. We formulate a factual informativeness evaluation task, which is integrated into both the training and evaluation pipeline. Empirical results show that our method leads to more informative answers, validating the effectiveness of this evaluation formulation. 2. We propose a novel two-stage training framework called InFACT , enhancing the model to generate more informative answers. 3. We evaluate our proposed framework with different models and demonstrate that it is effective in both improving information and factual accuracy. Our findings show significant improvement in factual precision, suggesting that the model learned to refrain from answering questions it would have made mistakes on, while overall high factual recall is preserved despite minor drops. 4. Finally, we conduct in-depth analyses of both methods individually and potential spurious correlations. Our results confirm that the proposed framework outperforms several strong baselines focused on factual accuracy alone, enhancing both informativeness and factual reliability simultaneously. 2 Informativeness Problem Setup 2.1 Background We first rigorously define a general setup of the informativeness evaluation task which is the foundation of our alignment model. In factual question answering, each question q_{i} , e.g., 1 What is the location of the capital of Australia?” has a correct answer t_{i} (e.g., the city of Canberra” ), which may appear in different string representations but refers to the same real-world entity. However, for a certain factual question q_{i} , there might exist several different correct answers. We distinguish the following three scenarios (see Fig. 2 for details): 1. Multiple-Answer Questions. A question with multiple correct answers – such as the question “What are the awards received by Barack Obama?” , which has several correct answers referring to different entities. 2. Descriptive QA . Responses may differ in their verbosity, yet point to the same real-world entity. For example – the text “The city is located inland, about 150 kms from the coast, and is surrounded by picturesque mountains …“ is the correct answer to 1 . 3. Granularity-based QA . Answers to certain questions may have a varying level of completeness yet be factually correct; for instance, the valid answers to the question “Where was Barack Obama born?” , would be “Kapiolani Medical Center for Women & Children” ; or “Honolulu” where the hospital is located; or “United States” where the city is located; or all of them, depending on the level of granularity. Considering these types of QA, we formulate the dataset and the evaluation. 2.2 Dataset Formulation An