Rethinking Search: A Study of University Students’ Perspectives on Using LLMs and Traditional Search Engines in Academic Problem Solving

Md. Faiyaz Abdullah Sayeedi, Md. Sadman Haque, Zobaer Ibn Razzaque, Robiul Awoul Robin, Sabila Nawshin · 2025

Read Rethinking Search: A Study of University Students’ Perspectives on Using LLMs and Traditional Search Engines in Academic Problem Solving by Md. Faiyaz Abdullah Sayeedi, Md. Sadman Haque, Zobaer Ibn Razzaque, Robiul Awoul Robin, Sabila Nawshin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-17726.txt.

Introduction

Faiyaz Abdullah Sayeedi and four co-authors study how students search for answers. University students now switch between Google and large language models. The team asked which tool students trust for school problems. They also asked how each tool fits daily study habits.

The researchers surveyed one hundred nine students and interviewed twelve more. Students used Google for sources they could check and cite. They used chatbots for summaries, plain explanations, and first drafts. Neither tool alone felt like enough for the students. So the team built a chatbot inside a search page as a test.

Catalog record

CREATORMd. Faiyaz Abdullah Sayeedi, Md. Sadman Haque, Zobaer Ibn Razzaque, Robiul Awoul Robin, Sabila Nawshin
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-17726

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Rethinking Search: A Study of University Students’ Perspectives on Using LLMs and Traditional Search Engines in Academic Problem Solving Md. Faiyaz Abdullah Sayeedi, Md. Sadman Haque, Zobaer Ibn Razzaque, Robiul Awoul Robin, Sabila Nawshin Abstract With the increasing integration of Artificial Intelligence (AI) in academic problem solving, university students frequently alternate between traditional search engines like Google and large language models (LLMs) for information retrieval. This study explores students’ perceptions of both tools, emphasizing usability, efficiency, and their integration into academic workflows. Employing a mixed-methods approach, we surveyed 109 students from diverse disciplines and conducted in-depth interviews with 12 participants. Quantitative analyses, including ANOVA and chi-square tests, were used to assess differences in efficiency, satisfaction, and tool preference. Qualitative insights revealed that students commonly switch between GPT and Google: using Google for credible, multi-source information and GPT for summarization, explanation, and drafting. While neither tool proved sufficient on its own, there was a strong demand for a hybrid solution. In response, we developed a prototype, a chatbot embedded within the search interface, that combines GPT’s conversational capabilities with Google’s reliability to enhance academic research and reduce cognitive load. 1 Introduction The rapid advancement of artificial intelligence has significantly reshaped the ways in which university students seek academic information and engage in research activities ( Pirzado et al. 2024 ) . Traditionally, search engines like Google have served as the dominant tool for retrieving scholarly content due to their accessibility, breadth of indexed materials, and access to verified sources. However, the emergence of LLMs, such as OpenAI’s ChatGPT, has introduced a new paradigm—offering students direct, conversational responses and contextualized summaries that can streamline information consumption ( Alberth 2023 ) . This evolution in digital research tools raises important questions about how students perceive and utilize these systems, particularly in academic settings where accuracy, credibility, and efficiency are critical. Prior research suggests that while LLMs facilitate rapid content summarization and task-specific assistance, their reliability varies depending on context and task complexity ( Divekar et al. 2024 ; Xu et al. 2023 ) . Conversely, search engines provide access to a wide range of authoritative sources but often require users to sift through multiple links and evaluate conflicting information independently. Several studies have documented this complementary behavior—students tend to use LLMs for explanation and drafting while relying on search engines for fact-checking and source validation ( Caramancion 2024 ; Spatharioti et al. 2023 ) . Despite their respective strengths, both tools also have well-documented limitations. LLMs can generate confident yet incorrect outputs, potentially misleading users ( Xu et al. 2023 ) , while traditional search engines can lead to information overload and inefficiency in time-sensitive academic contexts. As a result, students are increasingly adopting a hybrid approach—strategically switching between LLMs and search engines to balance speed with credibility ( Sakirin and Said 2023 ; Kapoor et al. 2024 ) . However, this constant toggling between tools introduces cognitive overhead and fragmented workflows, especially when performing complex academic tasks. To investigate these dynamics systematically, this study addresses the following research questions: • RQ1: How do university students perceive the usability, efficiency, and satisfaction of LLMs compared to traditional search engines in academic problem-solving? • RQ2: What patterns of tool usage emerge when students perform academic tasks with either or both tools? • RQ3: What are students’ preferences and expectations for an integrated solution that combines the strengths of both systems? 2 Literature Review LLMs have significantly reshaped how individuals learn, make decisions, and retrieve information. While traditional search engines like Google have long been the primary tool for academic information seeking, recent research increasingly explores how LLMs compare in terms of usability, task performance, and user trust. Divekar et al. 2024 examined how university students use LLMs like ChatGPT alongside traditional search engines for learning new topics. Their findings indicate that while LLMs support rapid summarization and ease of understanding, their effectiveness varies depending on the complexity and nature of the task. In a similar vein, Kumar et al. 2024 analyzed how students use LLMs to generate SQL queries. They observed that LLM assistance improved query formulation and contributed positively to the learning experience. Several studies have also investigated task completion performance. Spatharioti et al. 2023 conducted a randomized experiment and found that LLM users completed decision-making tasks more quickly and with fewer queries. However, the authors warned of a major drawback: users often overtrust LLM outputs, especially when incorrect answers are presented confidently. They suggested the inclusion of confidence indicators to mitigate this issue. Xu et al. 2023 echoed this concern, emphasizing the need for rigorous fact-checking when relying on LLM responses. In terms of task preference, Caramancion 2024 evaluated 20 types of information-seeking scenarios and concluded that users favored traditional search engines for fact-based queries, while preferring LLMs for creative or complex tasks. Supporting this, Sakirin and Said 2023 found that nearly 70% of participants preferred ChatGPT-style conversational interfaces due to their personalization, perceived efficiency, and convenience. Extending these findings, Wazzan et al. 2024 studied image geolocation tasks and observed that tool selection often influenced user strategy: LLMs were used more intuitively, while traditional tools required structured navigation. The issue of credibility remains central. Kapoor et al. 2024 argued that despite the convenience and rapidity of AI tools, traditional search methods remain more reliable for academic research. In contrast, LLMs often lack source transparency, which can be problematic in scholarly settings. To address this trade-off, researchers have proposed hybrid models. Bal and Nath 2009 explored metasearch engines that aggregate content from various sources to improve accuracy, and Caramancion 2024 advocated for systems that combine the contextual depth of LLMs with the source validation strengths of search engines. However, existing studies have primarily evaluated LLMs and traditional search engines in isolation or through task-specific comparisons, without fully exploring how students naturally combine both tools in academic workflows ( Xu et al. 2023 ) . There is a lack of empirical research that integrates both performance metrics and user perspectives to understand this hybrid usage behavior ( Bansal 2023 ) . While tools like Perplexity AI ( Perplexity AI 2024 ) attempt to bridge this gap by combining AI-generated responses with source links, and Google has introduced AI summaries through its Search Generative Experience (SGE), these systems remain largely static, lacking personalization, real-time adaptation, and task-specific reasoning. This study addresses these limitations through a mixed-methods approach and the design of a user-informed, context-aware prototype. 3 Methodology To explore university students’ preferences and usage behaviors regarding LLMs and traditional search engines for academic tasks, we employed a mixed-methods approach that combined quantitative and qualitative data collection and analysis, shown in Figure 1 . This design allowed us to examine both broad patterns and deeper user experiences in a complementary manner. Figure 1: Overview of the study methodology. A mixed-methods approach was employed in this study. (1) The survey phase (n=109) captured quantitative data and analyzed using different statistical tests. (2) The qualitative phase included in-person interviews (n=12) , where participants completed six academic tasks and were grouped based on tool usage. Thematic analysis of open-ended responses and interview transcripts led to four themes. We first conducted an online survey that collected responses from 109 university

Read the complete text