Information retrieval in big data using cognitive approaches

Santanu Acharjee, Ripunjoy Choudhury, e-mails: [email protected], [email protected] author: Santanu Acharjee · 2025

Read Information retrieval in big data using cognitive approaches by Santanu Acharjee, Ripunjoy Choudhury, e-mails: [email protected], [email protected] author: Santanu Acharjee (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-20837.txt.

Introduction

Santanu Acharjee and Ripunjoy Choudhury propose a topology based method for finding information in big data stores guided by how human cognition matches queries to documents. Data growth has made plain keyword search too weak, so retrieval must align the meaning of a query with the meaning held in large collections.

Their cognitive retrieval plan uses similarity distances, metric spaces, and retrieval topologies to formalize that semantic alignment. Logical connectives between query terms are folded into the same framework, extending basic matching to compound requests. The note is short and programmatic, setting out concepts and aims rather than large scale tests, and it frames future work on smarter search over very large data sets.

Catalog record

CREATORSantanu Acharjee, Ripunjoy Choudhury, e-mails: [email protected], [email protected] author: Santanu Acharjee
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-20837

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Information retrieval in big data using cognitive approaches Santanu Acharjee, Ripunjoy Choudhury, e-mails: [email protected], [email protected] author: Santanu Acharjee Abstract Abstract: Due to the exponential growth of big data in this digital era, an advanced method for effective information retrieval becomes essential. The basic objective of this paper is to propose a topology-based method for cognitive information retrieval (CIR) in big data environments. By using concepts such as cognitive similarity distances, metric spaces, retrieval topologies, etc., this paper aims to propose the semantic alignment between user queries and document repositories. The paper also extends this approach to incorporate logical connectives in cognitive information retrieval. Keywords: Cognitive information retrieval; big data; pseudo metric; logic; connectives. 2020 AMS Classifications: 94A16; 68T09; 68P01; 68T27; 62R40;30L15 . 1 Introduction: Indexing and searching are the two primary components of information retrieval (IR), and more specifically document retrieval. Indexing is a process used to store a document in a database, and searching involves using queries. Searching explains how users communicate their need for information or a subject they’d want to learn more about. The internet is not the start of the lengthy history of information retrieval. Only in the past ten years web search engines have evolved into search, and pervasiveness has been incorporated into desktop and mobile operating systems. Information retrieval (IR) technologies were used in intelligence and commercial applications in the 1960s, before search engines became widely used in daily life [ 1 ] . Inspired by remarkable inventions in the first half of the 20th century, the first computer-based search systems were constructed in the late 1940s [ 1 ] . The capabilities of the retrieval system increased as the processor speed and storage capacity increased, as is the case with many computer technologies [ 1 ] . Information that is relevant to a user’s inquiry is found by an IR system. Typically, an IR system looks through collections of semi-structured or unstructured data. When a collection grows too large for conventional cataloging methods to handle, an IR system becomes necessary [ 1 ] . The advancement of the World Wide Web has produced tremendous volumes of data quickly and many strategies are being used to handle this massive growth of data. According to a recent survey [ 2 ] , global data production, capture, copying and consumption were expected to increase rapidly, with a projected total of 64.2 zettabytes in 2020. It is expected that after five more years of growth, the amount of data created worldwide will exceed 180 zettabytes in 2025 [ 2 ] . The quantity of data reached a record level in 2020 [ 2 ] . The increased demand for the COVID-19 pandemic, which resulted in more people working and learning at home and using home entertainment alternatives more frequently, led the expansion to be greater than expected [ 2 ] . The exponential growth of data in the digital era has led to the advent of big data, characterized by its volume, variety, velocity, veracity and value. The term ‘big data’ itself signifies a large volume of data which is growing at an unprecedented rate [ 3 ] . In the upcoming years, the data size is anticipated to grow from petabytes to exabytes [ 3 ] . As technology advances, the amount of data is increasing rapidly, and appropriate tools should be available to manage this enormous data explosion [ 3 ] . This surge necessitates advanced methods for efficient information retrieval (IR), ensuring relevant and accurate data extraction from massive datasets. Traditional information retrieval methods can be classified into four parts: Boolean, vector space, probabilistic and inference network model. One may refer to [ 4 , 5 ] to know more about each of these methods. Along with these methods, theoretical methods of studying IR based on topology were also developed by Cater [ 6 ] in 1986. After that, many researchers were involved in developing these new topology-based methods. Egghe [ 7 ] , Egghe and Rousseau [ 8 ] , considered (DS, QS, Sim,\tau) to be a retrieval system consisting of a document space DS, a query space QS, and a function Sim , expressing the similarity between a document and a query. The topology \tau is generated by the set of such documents whose similarity with queries is less than some threshold value. In this regard, one may refer to [ 7 , 8 ] . In recent times, some studies related to the topology of the IR system and its applications to many fields can be seen like LLM [ 9 ] , some simulated information ecosystem [ 10 ] and 3DIR [ 11 ] . It ensures that how much the topology-based method of IR is relevant in today’s world too. Thus, the uses of topology-based methods for IR can be seen as an efficient tool in this big data era. Though the topology-based methods invented by Carter [ 6 ] and further largely used by Egghe and Rousseau [ 8 ] that time, yet to cope with it in this era, need some further modification in the methods. Their methods were useful to retrieve documents on the basis of similarity functions, like cosine similarity, Jaccard similarity, etc., but they fail to understand the cognitive perspective of users’ queries. Thus, a new and an upgraded version of the topology-based IR method using cognitive theory is developed in this paper. 2 Preliminaries: In this section, we procure some established definitions and results. They will be used in our next section. Definition 2.1 . [ 7 ] Let DS be the set (space) of all documents and QS a set of queries and Sim(.,Q) be a function of DS in W, measuring the similarity Sim(D,Q) between a document D and a query Q . Most commonly, its values are in \mathbb{R}^{+} or even [0,1], but this restriction is not necessary. Then, the retrieval topology \tau on DS is defined to be the topology generated by the sets (as a subbasis) R(Q,r)=\{\,D\in DS \mid Sim (D,Q)>r\,\}, for Q\in QS and r\in\mathbb{R}. Definition 2.2 . [ 7 ] In the system (DS, QS, sim,\tau) , the set ret {}_{\tau}(Q)=\{\,R(Q,r)\mid r\in\mathbb{R}\,\} is called the set of retrievals of Q\in QS. Definition 2.3 . [ 12 ] A non-empty set X with a map d:X\times X\longrightarrow[0,\infty) is called a metric space if the map d has the following properties: 1. d(x,y)\geq 0 \forall x,y\in X, 2. d(x,y)=0 if and only if x=y , 3. d(x,y)=d(y,x) , \forall x,y\in X , 4. d(x,y)\leq d(x,z)+d(z,y) , \forall x,y,z,\in X. In [ 13 ] , Acharjee and Gogoi introduced some important notions related to cognition, which are stated below: Definition 2.4 . [ 13 ] Let \mathcal{C} be a cognitive consequence space. The cognitive similarity distance on \mathcal{C} can be defined as a map Cog:\mathcal{C}\times\mathcal{C}\rightarrow[0,1], which satisfies the following properties: 1. Cog(x,y)\geq 0, 2. Cog(x,y)=0\iff x\approx y, 3. Cog(x,y)=Cog(y,x), 4. If x\approx z, then Cog(x,y)=Cog(z,y), 5. Cog(x,y)\leq Cog(x,z)+Cog(z,y) , where x,y,z\in C. In [ 13 ] , Acharjee and Gogoi further mentioned that x\approx y indicates that the two thoughts x and y are semantically similar or cognitively similar or identical. In short, we say that x and y

Read the complete text