ConExion: Concept Extraction with Large Language Models

Ebrahim Norouzi, Sven Hertling, Harald Sack · 2025

Read ConExion: Concept Extraction with Large Language Models by Ebrahim Norouzi, Sven Hertling, Harald Sack (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-12915.txt.

Introduction

Ebrahim Norouzi, Sven Hertling, and Harald Sack offer ConExion in a 2025 paper on pulling concepts from text. Old tools pull out key phrases that sum up a file. ConExion aims higher and tries to list all concepts in a field that a file holds, not just the top ones.

The method uses large language models that were trained in advance, guided by prompts. Tests on two well used data sets show a clear gain in F1 score over the best past methods. The authors also test prompts for work with no labeled data at all. The pulled concepts can help check how well a field map covers its field and can help build such maps.

Catalog record

CREATOREbrahim Norouzi, Sven Hertling, Harald Sack
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-12915

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ConExion: Concept Extraction with Large Language Models Ebrahim Norouzi, Sven Hertling, Harald Sack Abstract In this paper, an approach for concept extraction from documents using pre-trained large language models (LLMs) is presented. Compared with conventional methods that extract keyphrases summarizing the important information discussed in a document, our approach tackles a more challenging task of extracting all present concepts related to the specific domain, not just the important ones. Through comprehensive evaluations of two widely used benchmark datasets, we demonstrate that our method improves the F 1 F_{1} score compared to state-of-the-art techniques. Additionally, we explore the potential of using prompts within these models for unsupervised concept extraction. The extracted concepts are intended to support domain coverage evaluation of ontologies and facilitate ontology learning, highlighting the effectiveness of LLMs in concept extraction tasks. Our source code and datasets are publicly available at https://github.com/ISE-FIZKarlsruhe/concept_extraction . *1 keywords Concept Extraction ,Present Keyphrase Extraction ,Large Language Models † † copyrightyear: 2025 † † copyright: Copyright © 2025 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). † † venue: 2nd International Workshop on Natural Scientific Language Processing and Research Knowledge Graphs (NSLP 2025), co-located with ESWC 2025, June 01–02, 2025, Portorož, Slovenia † † email: [email protected] † † email: [email protected] † † email: [email protected] † † address: FIZ Karlsruhe – Leibniz Institute for Information Infrastructure, Hermann-von-Helmholtz-Platz 1, 76344 Eggenstein-Leopoldshafen, Germany † † address: Karlsruhe Institute of Technology (AIFB), Kaiserstr. 89, 76133 Karlsruhe, Germany † † corresp: Corresponding author. 1 Introduction Concept/Keyword extraction is recognized as a fundamental task in Natural Language Processing (NLP), crucial for identifying and extracting noun phrases that summarize and represent the main topics discussed in a document Hasan and Ng (2014) . It has been extensively utilized in various applications, including information retrieval, text summarization, and text categorization Meng et al. (2017) . However, it is not only helpful in those fields of research but also in ontology evaluation methods. Usually, competency questions (CQs) are created to specify the ontology requirements and are used later for the evaluation of the ontology Bezerra et al. (2013) . In cases where the ontology is already developed and no CQs and domain experts are available, other approaches need to be applied. Raad and Cruz (2015) presents four approaches: (1) Gold Standard-based, (2) Corpus-based, (3) Task-based, and (4) Criteria based. The idea behind corpus-based methods is to compare the ontology with concepts extracted from a text corpus that significantly covers the given domain. Thus, a good approach is needed to extract those concepts from a given text. The task of concept extraction can be formally defined as follows: Given a document D represented as a sequence of words D=[w_{1},w_{2},w_{3},...,w_{n}] , the goal is to extract a set C={(c_{1},s_{1}),(c_{2},s_{2}),...,(c_{m},s_{m})} where c_{i} consists of one or multiple words that best represent the topics of the document D . The additional score s_{i}\in[0,1] provides a confidence value of the approach to be able to specify the importance of each extracted concept further. The score can also be used to rank the phrases and only extract the top p concepts. Concept extraction is typically categorized into two types: (i) extracting present keyphrases (extractive), which are directly found within the input document such that k_{i}\in D , and (ii) extracting absent keyphrases (abstractive), which are generated even though they do not appear explicitly in the document Xie et al. (2023) . In this paper, we concentrate on extracting present concepts from documents using large language models (LLMs) because this approach best fits the task of ontology evaluation. Ontology evaluation Gómez-Pérez (2004) methods are used to judge an ontology’s content with respect to a reference framework throughout its development lifecycle. Since ontology evaluation includes verification and validation, it’s important to trace each extracted concept back to where it appears in the original reference framework. Extractive methods naturally fulfill this requirement by ensuring that all identified concepts are grounded in the original document, enabling a more accurate assessment of an ontology’s coverage and relevance. The approaches are evaluated on two common datasets for concept extraction, which are about scientific publications. We show that with a simple yet powerful and reproducible setup, we can surpass the state-of-the-art approaches. Our contributions include: • Utilizing large language models for effective concept extraction from scientific documents. • Providing comprehensive evaluations on benchmark datasets. • Demonstrating improved performance over state-of-the-art techniques in extracting present concepts. The paper is structured as follows: Section 1 introduces the problem and outlines the objectives of this study. Section 2 reviews the existing literature on concept extraction and large language models, highlighting the gap this study aims to fill. Section 3 details the methodology, the experimental setup, and the various prompts tested. Section 4 presents the evaluation metrics, the datasets used, and the results of the experiments, with a detailed analysis of the performance of different models and prompts. Section 5 concludes the paper, summarizing the key findings and outlining directions for future work. 2 Related Work Several comprehensive surveys provide a thorough overview of concept extraction techniques Hasan and Ng (2014) ; Xie et al. (2023) ; Song et al. (2023) . This paper focuses specifically on unsupervised methods, which offer several advantages such as domain independence and the lack of a requirement for training data. These methods are currently recognized as state-of-the-art in terms of performance. Recent advancements in unsupervised concept extraction have led to the development of several innovative approaches that do not rely on annotated data. Statistical methods, such as TF-IDF Jones (2021) and YAKE Campos et al. (2018) , identify significant words by calculating statistical features like word frequencies and co-occurrences, thereby selecting candidates for keyphrases. Graph-based methods have also been widely explored. TextRank Mihalcea and Tarau (2004) represents text as a graph, where words serve as nodes and their co-occurrences act as edges. These methods employ node ranking algorithms, such as PageRank Page et al. (1999) , to rank words and extract the top-k words as keyphrase candidates. Bougouin et al. Bougouin et al. (2013) introduced TopicRank, which clusters candidate phrases into topics in the initial phase and subsequently ranks these topics based on their significance within the document. Positionrank Florescu and Caragea (2017) further enhances this approach by incorporating the positional information of words in the text to improve ranking accuracy. Embedding-based models have emerged as another effective approach for keyphrase extraction. EmbedRank Bennani-Smires et al. (2018) , for instance, utilizes part-of-speech tagging to identify potential keyphrases from a document. These keyphrases are then represented as low-dimensional vectors using a pretrained embedding model and ranked according to their Cosine similarity to the document’s overall embedding vector. This technique leverages the semantic representations provided by embedding models to improve keyphrase extraction. MultPAX Zahera et al. (2022) employs semantic similarity between candidate keyphrases and an input document using the pretrained embedding of BERT model. AutoKeyGen Shen et al. (2022) constructs a phrase bank by combining keyphrases from all documents into a corpus, considering both lexical and semantic similarities for selecting top candidate keyphrases for each input document. PromptRank Kong et al. (2023) is an unsupervised approach based on a pre-trained language model (PLM) with an encoder-decoder

Read the complete text