ORKG ASK: a Neuro-symbolic Scholarly Search and Exploration System

Allard Oelen, Mohamad Yaser Jaradeh, Sören Auer · 2024

Read ORKG ASK: a Neuro-symbolic Scholarly Search and Exploration System by Allard Oelen, Mohamad Yaser Jaradeh, Sören Auer (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-04977.txt.

Introduction

Allard Oelen, Mohamad Yaser Jaradeh, and Sören Auer present ORKG ASK in a 2024 paper, a neuro-symbolic system for scholarly search and exploration. To find the right papers is slow and hard work, yet it is key to doing science, and the flood of new papers makes the search worse each year.

The system joins three parts: semantic search, large language models, and knowledge graphs. Search first draws a set of apt papers. Facts are then pulled from that set and shown to the user in a clear form. The team call the system ready for real use, and first tests back the approach.

Catalog record

CREATORAllard Oelen, Mohamad Yaser Jaradeh, Sören Auer
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE11 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-04977

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ORKG ASK: a Neuro-symbolic Scholarly Search and Exploration System Allard Oelen, Mohamad Yaser Jaradeh, Sören Auer Abstract Purpose: Finding scholarly articles is a time-consuming and cumbersome activity, yet crucial for conducting science. Due to the growing number of scholarly articles, new scholarly search systems are needed to effectively assist researchers in finding relevant literature. Methodology: We take a neuro-symbolic approach to scholarly search and exploration by leveraging state-of-the-art components, including semantic search, Large Language Models (LLMs), and Knowledge Graphs (KGs). The semantic search component composes a set of relevant articles. From this set of articles, information is extracted and presented to the user. Findings: The presented system, called ORKG ASK (Assistant for Scientific Knowledge), provides a production-ready search and exploration system. Our preliminary evaluation indicates that our proposed approach is indeed suitable for the task of scholarly information retrieval. Value: With ORKG ASK, we present a next-generation scholarly search and exploration system and make it available online. Additionally, the system components are open source with a permissive license. [orcid=0000-0001-9924-9153, [email protected], ] [orcid=0000-0001-8777-2780, [email protected], ] [orcid=0000-0002-0698-2864, [email protected], ] keywords Neuro-symbolic AI ,Large Language Models ,Scholarly Knowledge Graphs ,Scholarly Search System † † copyrightyear: 2024 † † copyright: Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). † † venue: SEMANTiCS 2024 EU: 20th International Conference on Semantic Systems, September 17-19, 2024, Amsterdam, The Netherlands † † address: TIB – Leibniz Information Centre for Science and Technology, Hannover, Germany † † address: L3S Research Center, Leibniz University of Hannover, Hannover, Germany 1 Introduction Finding scholarly articles and exploring the body of scholarly literature consumes a significant share of a researcher’s time. Due to the growing number of scholarly articles, this issue only becomes more apparent Landhuis 2016 . Current scholarly search systems passively assist users with their information needs by providing a list of relevant articles. If instead active assistance were provided, the users’ information needs, such as a research question, would be answered for them. We present ORKG ASK (Assistant for Scientific Knowledge), a new generation scholarly search and exploration system 1 1 1 Available online via https://ask.orkg.org . ORKG ASK helps researchers find relevant literature and automatically extract knowledge from the retrieved literature, actively supporting researchers with their information needs. The approach consists of three main components: 1) Semantic Search, 2) a Large Language Model (LLM), and 3) Knowledge Graphs (KGs). First, the semantic search addresses the previously discussed challenge of retrieving articles based on their relevance to a specific information need. In ORKG ASK users can formulate their information need as a research question, which is entered as a search query. Second, an LLM is leveraged to answer the research question by prompting with the context of the set of relevant articles. In addition to answers to the research question, a set of properties is extracted, among others, a summary, materials, methods, and results of the contributions described in the articles. Third, KGs are used to provide more fine-grained information extraction as well as for curating extraction results. This includes results filtering based on mentioned concepts in scholarly articles. Figure 1 : Design of the search result page of the ORKG ASK application. 2 Background There are various established, large, and multidisciplinary scholarly search systems, among others, Google Scholar, Semantic Scholar, and Scopus Gusenbauer and Haddaway 2020 . Other systems, such as PubMed and ACM Digital Library, are domain-specific. These search systems take a similar approach where articles are ranked based on relevance, but where users have to manually extract relevant information from articles. A new approach provides active support via automatic information extraction by systems such as Elicit, Consensus, and Scispace Bolanos et al. 2024 . These systems are not open-source, leaving details about their approach, such as the model and dataset, to be unknown, in turn making results harder to reproduce. This makes such systems less suitable for systematic literature reviews where reproducibility is a key aspect of the approach MacFarlane et al. 2022 . To extract knowledge from a large set of scholarly documents, a Retrieval-Augmented Generation (RAG) approach can be used to provide the LLM with relevant context Lewis et al. 2020 . The Retrieval aspect retrieves a set of documents, commonly done using vector databases. The Augmented aspect, augments the user query with the found context. Finally, the Generation aspect creates the response. To our knowledge, the previously mentioned AI-supported scholarly search systems use this approach and are thus similar to the approach we propose with ORKG ASK. 3 System Overview Figure 2 : ORKG ASK system workflow integrating neuro-symbolic components. We now discuss the ORKG ASK system in more detail. The service is designed and developed in such a way that it provides a solid foundation for a sustainable service. Additionally, we focus on accessibility by providing a dark mode (for low-light conditions), a responsive interface (for mobile usage, or high zoom levels for visually impaired users), and implementing ARIA accessibility attributes where necessary. The ORKG ASK code base is published as open source under an MIT license and available online. 2 2 2 https://gitlab.com/TIBHannover/orkg/orkg-ask Figure 1 depicts the design of the search result page for a specific research question. 3.1 User-Oriented Features The Question Answering feature as depicted in Node 1 in Figure 1 illustrates the input field for the research question. Nodes 2 and 4 present the answers to the question. Node 2 shows a synthesized answer of the first five displayed results. The Information Extraction feature extracts additional information from an article (node 4). There are several default columns displayed, but users can customize the extracted information to their needs (node 3). The Filtering feature enables users to filter articles based on user-provided criteria (node 5). This includes the ability to filter based on year, language, words that appear in the title or abstract, the number of citations, author names, etc. The Bibliography Managing featured called “My Library” provides a bibliography manager where users can store and curate a list of articles. Articles are added by clicking on the bookmark icon in the interface (node 8) or added manually via the My Library page (via DOI, title, or BibTeX). Articles from My Library can be manually added to a search query, which prepends the manually selected articles to the search results. The Data Reuse feature supports citing articles in APA, Vancouver, Harvard, citation styles, and exports to BibTeX, RIS, and CLS-JSON (node 7). The export button is displayed in node 8. Furthermore, there is an option to export the entire search result table to CSV and ORKG Auer et al. 2020 CSV (node 6). Finally, the Entity Linking feature links entities in article abstracts to their respective DBpedia entries. This provides the ability to filter articles based on semantically identical concepts, providing an additional means to more targeted information retrieval. 3.2 System Workflow Figure 2 depicts the system workflow. It starts with a user asking a research question. A set of relevant documents for this question is retrieved. The neural search component uses vectorized representations of the query and articles via the Nomic embeddings model to retrieve a set of relevant documents. Optionally, the search space can be narrowed down

Read the complete text