Comparative Analysis of OpenAI GPT-4o and DeepSeek R1 for Scientific Text Categorization Using Prompt Engineering

Aniruddha Maiti 1 , Samuel Adewumi 1 , Temesgen Alemayehu Tikure 1 , Zichun Wang 1 , Niladri Sengupta 2 , Anastasiia Sukhanova 3 , Ananya Jana 3 1 Department of Mathematics, Engineering, and Computer Science, West Virginia State University, Institute, WV, USA 2 Fractal Analytics Inc, USA 3 Departmen · 2025

Read Comparative Analysis of OpenAI GPT-4o and DeepSeek R1 for Scientific Text Categorization Using Prompt Engineering by Aniruddha Maiti 1 , Samuel Adewumi 1 , Temesgen Alemayehu Tikure 1 , Zichun Wang 1 , Niladri Sengupta 2 , Anastasiia Sukhanova 3 , Ananya Jana 3 1 Department of Mathematics, Engineering, and Computer Science, West Virginia State University, Institute, WV, USA 2 Fractal Analytics Inc, USA 3 Departmen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-02032.txt.

Introduction

Aniruddha Maiti and co-authors compare two leading chatbots at sorting sentences from scientific papers, in a 2025 study. Sorting text by the relationships it states helps organize research knowledge, but the newer model in the pair was untested on this job.

The team built a cleaned data set of papers from many fields and a scoring method made for this task. Prompt design guided both models to assign each sentence to a fixed relationship category. Their consistency and accuracy were then compared side by side. The study offers a benchmark for a job where general chatbot fame may not predict performance, and a data set others can reuse.

Catalog record

CREATORAniruddha Maiti 1 , Samuel Adewumi 1 , Temesgen Alemayehu Tikure 1 , Zichun Wang 1 , Niladri Sengupta 2 , Anastasiia Sukhanova 3 , Ananya Jana 3 1 Department of Mathematics, Engineering, and Computer Science, West Virginia State University, Institute, WV, USA 2 Fractal Analytics Inc, USA 3 Departmen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-02032

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Comparative Analysis of OpenAI GPT-4o and DeepSeek R1 for Scientific Text Categorization Using Prompt Engineering Aniruddha Maiti 1 , Samuel Adewumi 1 , Temesgen Alemayehu Tikure 1 , Zichun Wang 1 , Niladri Sengupta 2 , Anastasiia Sukhanova 3 , Ananya Jana 3 1 Department of Mathematics, Engineering, and Computer Science, West Virginia State University, Institute, WV, USA 2 Fractal Analytics Inc, USA 3 Department of Computer Sciences and Electrical Engineering, Marshall University, Huntington, WV, USA Corresponding author: [email protected] Abstract This study examines how large language models categorize sentences from scientific papers using prompt engineering. We use two advanced web-based models, OpenAI’s GPT-4o and DeepSeek R1, to classify sentences into predefined relationship categories. DeepSeek R1 has been tested on benchmark datasets in its technical report. However, its performance in scientific text categorization remains unexplored. To address this gap, we introduce a new evaluation method designed specifically for this task. We also compile a dataset of cleaned scientific papers from diverse domains. This dataset provides a platform for comparing the two models. Using this dataset, we analyze their effectiveness and consistency in categorization. 1. Introduction The field of artificial intelligence (AI) has advanced rapidly in recent years. These advancements have led to the development of powerful language models that are now used in many domains. Among them, OpenAI’s GPT-4o and DeepSeek R1 have received significant attention due to their capabilities and design. OpenAI introduced GPT-4o in May 2024 [ 1 ] . This model builds on earlier versions and improves performance in text, speech, and vision tasks. It provides GPT-4-level intelligence. At the same time, it operates faster and it has lower computational cost. These improvements make GPT-4o applicable across a wide range of tasks. DeepSeek, an AI startup, released DeepSeek R1 in January 2025 [ 2 ] . This model offers performance comparable to leading proprietary systems but at a lower cost. It is open-source, which allows AI researchers to study and modify it. This transparency has made DeepSeek R1 an important contribution to the AI research community. However, there exist concerns regarding the handling of questions related to politically sensitive topics by DeepSeek’s web-based platform [ 3 ] . The introduction of DeepSeek R1 has influenced AI development in multiple ways. Its cost efficiency presents a challenge to companies that invest heavily in proprietary models. The open source approach has also advantageous for more research and development [ 4 ] . Although DeepSeek R1 has been evaluated on general benchmarks [ 5 ] , its performance in the specific task of categorization of sentences in scientific text has not been analyzed in a systematic way. In particular, there has been little research on its ability to categorize scientific text. This study introduces an evaluation method designed for this task. In addition, we construct a dataset using texts from 10 scientific papers. This dataset allows a direct comparison of GPT-4o and DeepSeek R1 in sentence classification. By categorizing sentences into predefined relationship types by these two models, we examine the strengths and weaknesses. 2. Overview of GPT-4o and DeepSeek R1 2.1. GPT-4o: OpenAI’s Multimodal Model OpenAI introduced GPT-4o in May 2024 as an improvement over its previous models [ 1 ] . Unlike earlier versions, GPT-4o processes text, vision, and audio together. It provides GPT-4-level intelligence but runs faster and more efficiently. Some key features of GPT-4o are as follows: • Multimodal Processing : GPT-4o handles text, images, and voice inputs in a single model. • Faster Performance : OpenAI reports that GPT-4o runs at least twice as fast as GPT-4 Turbo while maintaining similar accuracy. • Lower Cost : The API pricing is lower than earlier versions, making it more affordable for researchers and developers. • Better Reasoning and Coding : GPT-4o improves performance in math, programming, and long-context tasks. OpenAI designed GPT-4o to increase efficiency and support real-time AI applications. 2.2. DeepSeek R1: An Open-Source Alternative DeepSeek released DeepSeek R1 in January 2025 as an open-source large language model (LLM) [ 2 , 5 ] . Unlike many proprietary models, DeepSeek R1 is available for public use and modification. It also applies reinforcement learning to improve logical reasoning. Some key features of DeepSeek R1 are as follows: • Reinforcement Learning-Based Training : The model improves its reasoning ability through reinforcement learning [ 5 ] . • Open-Source Access : Multiple versions, including DeepSeek R1-Zero and smaller distilled models, are available. • Lower Computational Cost : It performs at a level similar to OpenAI’s o1-1217 but requires fewer resources. • Community Involvement : The open-source nature allows AI researchers to modify and build on the model. • Benchmark Performance : The DeepSeek R1 technical report shows its competitive performance on standard reasoning and text benchmarks [ 5 ] . Recent analyses have raised concerns regarding DeepSeek’s web based platform’s handling of questions related to politically sensitive topics [ 3 ] . However, DeepSeek R1 has contributed to making AI more accessible by providing a cost-effective, open-source model [ 4 ] . 2.3. Comparative Considerations GPT-4o and DeepSeek R1 are designed for different users. GPT-4o focuses on real-time, multimodal applications while DeepSeek R1 provides an open-source alternative with reinforcement learning improvements. Their ability to classify scientific text has not been studied in depth. This study introduces a method to evaluate both models for this task. We compile a dataset of 10 cleaned scientific articles and compare their sentence classification results. The next sections describe the relevant work on taxonomy of semantic relations for knowledge representation based on which the categorization is performed. Next, we briefly discuss prior work and then describe experimental design, methodology, and results. 3. Literature Review The categorization of scientific text relies on the structured understanding of semantic relationships. Previous research has explored various methods for modeling these relationships to enhance AI-driven classification systems. Maia and Lima [ 6 ] proposed a taxonomy of semantic relations for knowledge representation, providing a foundation for categorizing scientific text. Their work highlights the importance of structuring relationships between entities to improve information retrieval and classification. Wang et al. [ 7 ] extended this idea by introducing ICAD-MI, a method for interdisciplinary concept association discovery through metaphor interpretation. Their findings underscore the significance of nuanced relationship modeling, which can be applied to AI-driven text categorization. Corpus-based studies have also played a crucial role in understanding semantic transformations. Kunch and Kharchuk [ 8 ] examined how deterministic vocabulary changes in artistic discourse, while Albota et al. [ 9 ] focused on the semantic transformation of the lexeme ”virus” using corpus technologies. These works highlight the dynamic nature of language and the challenges AI models face in categorizing scientific text accurately. Additionally, Kunch et al. [ 10 ] studied orthographic norm dynamics in media discourse, offering insights into how variations in scientific writing affect text classification. The research by Vysotska et al. [ 11 ] on data integration in business analytics systems further demonstrates the role of information structuring in AI-driven applications. The main idea of this study is to integrate insights from semantic taxonomies, corpus linguistics, and interdisciplinary concept modeling and use those insights in prompt engineering to examine comparative ability of two large language models, GPT-4o and DeepSeek R1 in the

Read the complete text