RoboKA: KAN Informed Multimodal Learning for RoboCall Surveillance System

Nitin Choudhury 1 , Nikhil Kumar 1 , Aditya Kumar Sinha 1 , Abhijeet Anand 1 , Hossein Salemi 2 , Orchid Chetia Phukan 1 , Hemant Purohit 2 , Arun Balaji Buduru 1 · 2026

Read RoboKA: KAN Informed Multimodal Learning for RoboCall Surveillance System by Nitin Choudhury 1 , Nikhil Kumar 1 , Aditya Kumar Sinha 1 , Abhijeet Anand 1 , Hossein Salemi 2 , Orchid Chetia Phukan 1 , Hemant Purohit 2 , Arun Balaji Buduru 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-00156.txt.

Introduction

Nitin Choudhury and 7 co-authors tackle robocall detection, a field held back by a lack of public data due to privacy concerns. They first build Robo-SAr, a synthetic dataset with about two hundred unwanted calls and about twelve hundred legitimate ones. The fake calls attack along three lines: manipulative wording, emotion-driven speech, and cloned voices.

Their system, RoboKA, combines sound and language cues in one model. It aligns the two kinds of signal, then classifies calls with a Kolmogorov-Arnold Network head, a design meant to capture tangled links between cues. In tests both inside and outside its home data, RoboKA beat strong single-mode and mixed baselines on recall and F1 score, the measures that matter most for catching scam calls.

Catalog record

CREATORNitin Choudhury 1 , Nikhil Kumar 1 , Aditya Kumar Sinha 1 , Abhijeet Anand 1 , Hossein Salemi 2 , Orchid Chetia Phukan 1 , Hemant Purohit 2 , Arun Balaji Buduru 1
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.00156

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RoboKA: KAN Informed Multimodal Learning for RoboCall Surveillance System Nitin Choudhury 1 , Nikhil Kumar 1 , Aditya Kumar Sinha 1 , Abhijeet Anand 1 , Hossein Salemi 2 , Orchid Chetia Phukan 1 , Hemant Purohit 2 , Arun Balaji Buduru 1 Abstract Wide exploration on robocall surveillance research is hindered due to limited access to public datasets, due to privacy concerns. In this work, we first curate Robo-SAr , a synthetic robocall dataset designed for robocall surveillance research. Robo-SAr comprises of ∼ \sim 1200 unwanted and ∼ \sim 1200 legitimate synthetic robocall samples across three realistic adversarial axes: psycholinguistics-manipulated transcripts, emotion-eliciting speech, and cloned voices. We further propose RoboKA , a Kolmogorov–Arnold Network (KAN)-based multimodal fusion framework designed to model structured nonlinear interactions between acoustic and linguistic cues that characterize diverse adversarial robocall strategies. RoboKA first leverages cross-modal contrastive learning to align latent modality representations and feeds the resulting embeddings to a KAN-projection head for final classification. We benchmark RoboKA against strong unimodal and multimodal baselines in both in-domain and out-of-domain setups, finding RoboKA to surpass all baselines in terms of recall and F1-score. I Introduction and Related Work Robocalls are defined as the automated calls generated using text-to-speech (TTS) systems or pre-recorded audio, primarily designed to share information with a large audience in an efficient manner [ 3 , 5 ] . These calls are being utilized across various sectors, including healthcare, customer service, and local government, due to their reliability, scalability, and cost efficiency. However, the widespread availability of such systems leads malicious users to craft deceptive calls, misleading users into revealing sensitive information or making fraudulent payments. Recent advancements in neural TTS models have further intensified this threat, increasing the quality, expressiveness, and naturalness of such deceptive robocalls. In parallel, large language models (LLMs) have demonstrated remarkable capabilities in generating linguistically sophisticated and psychologically persuasive content. In particular, with access to these advanced TTS and LLM models, adversaries can now leverage such systems, combined with LLMs, to generate highly deceptive robocalls that are difficult to distinguish from legitimate ones [ 22 ] . In recent years, i.e., 2024 and 2025, an approximate volume of 96.8 billion robocalls was reported in the USA, resulting in a massive loss of approximately 28.6 billion USD [ 32 , 33 ] , demonstrating essential need for effective robocall surveillance systems. Addressing the growing threat of robocalls, the regulatory framework STIR/SHAKEN has been introduced, and it has shown effectiveness against certain attack vectors [ 2 , 38 ] . However, they authenticate caller identities to reduce call spoofing; they fall short in addressing robocalls originating from international sources or exploiting infrastructure loopholes [ 2 , 31 ] . Similarly, network-based and behavior-based surveillance systems frequently fail against evasive and dynamic tactics used by advanced robocall campaigns. In response to these challenges, researchers have explored audio and text modalities, leveraging conventional acoustic features as well as pretrained language and speech model representations for the task of identifying such deceptive robocalls. Experimental results indicate that unimodal detection systems—based solely on audio or transcript—can achieve strong performance in controlled settings [ 6 , 31 , 30 ] . Despite exploration and claims, most of the research suffers from critical limitations: their datasets are either proprietary, small-scale, or not publicly released, making reproducibility difficult and raising concerns about the reliability of reported results. In contrast, the robustness of such defensive systems under distributional shifts remains unexplored. This lack of reproducible benchmarks hinders cumulative progress in robocall defense research. Unimodal robocall detectors are inherently fragile, as attackers can independently manipulate either speech acoustics or linguistic content, whereas jointly reasoning over audio and text constrains such evasions and is therefore necessary for robust detection. Addressing the issue of reproducibility, this work curates and releases Robo-SAr ( Robo Call– S imulated A dversa r ial dataset) , a novel adversarially enriched synthetic corpus. Robo-SAr targets three adversarial factors in transcript and speech synthesis: (i) LLM-driven psycholinguistic manipulation, (ii) TTS-driven emotion elicitation, and (iii) TTS-driven voice cloning. We use a large language model to curate and manipulate the psycholinguistics of the transcripts, and four widely used open-access neural text-to-speech (TTS) models with fourteen distinct voices to synthesize speech. Among these, we elicit eight emotions with two voices. In addition, we incorporate a real-world robocall dataset from the FTC Do Not Call repository [ 34 ] , which contains real-world, genuine unwanted calls. Together, these components capture both plausible adversarial robocalls and real-world distribution shifts, enabling a robust and reproducible benchmark. Building on this, we propose RoboKA ( Unwanted Robo Call Detection guided by KA N ), a multimodal framework that (i) uses Cross-Modal Contrastive Learning (CMCL) on modality-specific pretrained embeddings to align audio and text representations and make them more label-consistent and robust under shifts, and (ii) replaces linear/MLP heads with modality-specific KAN projection heads plus stacked KAN fusion layers to capture nonlinear interactions and sharpen the boundary between unwanted and legitimate robocalls. We hypothesize that robocall adversaries implies modality-specific noise, requiring fusion models to perform conditional calibration and expressive nonlinear interaction modeling, which MLP fusion often lacks to preserve. Addressing this, we propose a KAN-based fusion approach to enable more stable evidence aggregation via expressive non-linear interactions. To summarize, the contributions of this work are as follows: • We curate and release Robo-SAr , a novel, adversarially enriched synthetic robocall corpus that explicitly targets three attack axes in both transcript and speech generation: psycholinguistic manipulation, emotion elicitation, and voice cloning, using LLM-based transcript design and four neural TTS models with fourteen voices and eight emotions, and is validated by expert annotators 1 1 1 We commit to releasing code and data upon reviews in compliance with ethical standards. . • We propose RoboKA , a multimodal framework that applies cross-modal contrastive learning to modality-specific pre-trained downstream embeddings and refines them via two KAN-based projection heads to enhance nonlinear expressiveness. To our knowledge, we are the first to leverage KAN nonlinear projection and fusion for multimodal robocall detection under adversarial distribution shifts. • We demonstrate that RoboKA yields more discriminative decision boundaries between unwanted and legitimate robocalls and consistently outperforms unimodal and multimodal baselines under both in-domain (InD) and out-of-domain (OoD) evaluations. II Background II-A Text-to-Speech (TTS) TTS systems are a core component of speech synthesis, converting written text into human-audible speech [ 26 ] . Based on previous literature [ 20 , 15 , 11 , 12 ] , four state-of-the-art TTS systems: Bark [ 1 ] , OpenAI TTS [ 29 ] , SpeechT5 [ 8 ] , and xTTS [ 4 ] are chosen for speech synthesis purpose. A total of fourteen voice samples from both male and female are considered for the speech synthesis process. II-B Large Language Model (LLM) Recent advances in large language models have demonstrated higher linguistic manipulation capabilities. In particular, adversaries can now leverage these LLMs, combined with advanced TTS, to generate highly deceptive robocalls. In this work, we consider ChatGPT by OpenAI [ 28 ] for psycholinguistic manipulation as it demonstrates superior performance in linguistic manipulation in previous literature [ 37 , 40 , 21 ] . II-C Pre-trained Models (PTMs) Audio PTMs. We utilize three audio PTMs: Wav2Vec2

Read the complete text