FOCA: Multimodal Malware Classification via Hyperbolic Cross-Attention
Nitin Choudhury, Bikrant Bikram Pratap Maurya, Orchid Chetia Phukan, Arun Balaji Buduru · 2026
Read FOCA: Multimodal Malware Classification via Hyperbolic Cross-Attention by Nitin Choudhury, Bikrant Bikram Pratap Maurya, Orchid Chetia Phukan, Arun Balaji Buduru (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-17638.txt.
Introduction
Nitin Choudhury and three co-authors present FOCA for malware classification. FOCA fuses audio and visual views of malware data. Most fusion methods work in flat Euclidean space. FOCA works in hyperbolic space instead. That curved space fits hierarchical links between the two views well. Cross-attention joins the streams inside that space.
The authors claim FOCA is first to use that structure for this task. Hierarchy is natural in malware families, which branch like trees. A space that matches that shape can sort samples with more care. Security teams face new malware daily. Better classification speeds response and cuts harm. The work adds a new geometry to the defender's toolkit.
Catalog record
| CREATOR | Nitin Choudhury, Bikrant Bikram Pratap Maurya, Orchid Chetia Phukan, Arun Balaji Buduru |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-17638
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FOCA: Multimodal Malware Classification via Hyperbolic Cross-Attention Nitin Choudhury, Bikrant Bikram Pratap Maurya, Orchid Chetia Phukan, Arun Balaji Buduru Abstract In this work, we introduce FOCA , a novel multimodal framework for malware classification that jointly leverages audio and visual modalities. Unlike conventional Euclidean-based fusion methods, FOCA is the first to exploit the intrinsic hierarchical relationships between audio and visual representations within hyperbolic space. To achieve this, raw binaries are transformed into both audio and visual representations, which are then processed through three key components: (i) a hyperbolic projection module that maps Euclidean embeddings into the Poincaré ball, (ii) a hyperbolic cross-attention mechanism that aligns multimodal dependencies under curvature-aware constraints, and (iii) a Möbius addition–based fusion layer. Comprehensive experiments on two benchmark datasets—Mal-Net and CICMalDroid2020–show that FOCA consistently outperforms unimodal models, surpasses most Euclidean multimodal baselines, and achieves state-of-the-art performance over existing works. † † address: IIIT-Delhi, India † † address: IIIT-Delhi, India Index Terms : Audio, Audio-Visual Learning, Binary, Classification, Hyperbolic Space, Malware, Multimodal, Visual 1 Introduction Malware is a continuously evolving threat that compromises both system integrity and user privacy. Its classification has become a high-stakes “cat-and-mouse” race, with attackers deploying obfuscation and polymorphism to evade defenses. Malware family classification enhances threat understanding and strengthens incident response by linking variants to their origin and enabling targeted defenses. Researchers have explored rule-based, machine learning (ML), and deep learning (DL) approaches for malware classification, yet current methods remain vulnerable as malware continues to evolve at scale [ 1 ] . These challenges highlight the urgent need for robust and reliable classification frameworks. The traditional approach of malware classification primarily relies on a signature-based or rule-based approach, including sample hash matching or string matching and so on, which is effective for previously seen malware, but fails when exposed to a new or obfuscated variant of it [ 5 ] . Addressing this limitation, researchers explored static and dynamic features such as opcode n-grams, API call sequences, entropy statistics, and control flow graphs combined with ML and DL algorithms to enhance the categorization of malware samples [ 16 , 4 ] . In contrast, several studies have explored alternative representations of binaries by converting them into other modalities, such as binary-to-audio or binary-to-visual (image) , thereby enabling the application of deep learning techniques beyond raw binary analysis [ 18 , 19 ] . While image-based representations have shown competitive results in malware classification when used alone or combined with conventional features [ 6 , 17 , 22 , 11 ] , audio representations remain relatively underexplored and are often restricted to spectrograms derived from traditional signal processing techniques, limiting their expressive potential. More recently, multimodal frameworks that integrate audio and visual modalities have been investigated for improved malware classification, employing fusion strategies such as cross-modal attention, feed-forward integration, or simple concatenation [ 13 , 25 , 27 ] . However, none of the prior research has attempted to exploit the implicit hierarchical relationships between audio and visual modalities [ 10 ] during fusion—a missed opportunity with significant potential to advance malware classification. To address this, we propose FOCA ( F usi O n with Hyperbolic C ross- A ttention), a novel multimodal framework that fuses audio and visual modalities while explicitly modeling their hierarchical relationship for malware classification. For representation extraction for both modalities, we leverage state-of-the-art (SOTA) pre-trained models. Our idea is that audio modality encodes fine-grained byte-level characteristics of binary data, whereas visual modality captures higher-level spatial organization and structural patterns—together forming a hierarchy across the two modalities. To effectively model this hierarchy, FOCA makes use of hyperbolic space. Specifically, it consists of three key components: (i) projecting both modalities into a shared hyperbolic space, (ii) aligning them with a novel hyperbolic cross-attention mechanism, and (iii) merging them using Möbius addition. Extensive experiments on two benchmark datasets—Mal-Net and CICMalDroid2020—demonstrate that FOCA consistently surpasses unimodal approaches and Euclidean-based multimodal baselines, achieving SOTA performance over prior works. To the best of our knowledge, this is the first study to leverage hyperbolic space for fusion of multimodal representations for malware classification. To summarize, the key contributions of the proposed work are as follows : (i) We are the first to explore the hierarchical relationship between audio and visual modalities for malware classification, (ii) We propose FOCA , a novel multimodal framework that explicitly models this hierarchy by leveraging hyperbolic space. We propose a novel hyperbolic cross-attention mechanism for aligning the audio and visual modalities. (iii) Through extensive experiments on two benchmark datasets—Mal-Net and CICMalDroid2020— FOCA consistently outperforms unimodal models and Euclidean-based multimodal baselines, achieving SOTA performance over previous works and establishing hyperbolic multimodal fusion as a promising new direction for malware classification. The code and trained models are available here: https://github.com/nitinc24009/FOCA.git . 2 Methodology In this section, we first describe the conversion of binaries into audio and visual modalities. We then present the pre-trained models (PTMs) used for extracting representations from these modalities, followed by a detailed discussion of our proposed framework, FOCA . 2.1 Binary-to-X Transformation Binary-to-Audio Modality : Raw byte sequences from the APK-dex files are mapped into waveform samples and stored as .wav files. This representation treats binaries as continuous time-series signals, and this will enable audio PTMs to process them directly as raw waveforms. Binary-to-Image Modality : The raw bytes from APK-dex files are read as unsigned integers (0–255), concatenated into a 1D array, and then padded and reshaped into a 2D matrix with a width determined by file size. The header, data, and remaining sections are mapped to the red, green, and blue channels, respectively, producing an image representation suitable for vision PTMs. 2.2 Pre-Trained Models (PTMs) Audio PTMs : We consider SOTA audio PTMs in our study, namely, Wav2vec2 [ 2 ] , WavLM [ 23 ] , and HuBERT [ 26 ] . Wav2vec2 was pre-trained in a self-supervised format and learns contextual representations via contrastive learning. WavLM is also trained in a self-supervised format. It solves both the masked speech prediction task and speech denoising during its pre-training. HuBERT is pre-trained in a BERT-like masked prediction objective and builds on the Wav2vec2 architecture. For Wav2vec2, WavLM, and HuBERT, we use the base versions available in Huggingface, extracting 768-dimensional representations from the frozen PTMs’ last hidden layer via mean pooling. Vision PTMs : We consider three SOTA vision PTMs: ResNet50 [ 12 ] , VGG19 [ 14 ] , and ViT [ 3 ] . ResNet50 and VGG-19 are CNN-based PTMs pretrained on ImageNet for large-scale image classification. In contrast, ViT is a purely attention-based, convolution-free architecture. We extract 2048, 4096, and 768-dimensional representations from the last layer of frozen ResNet50, VGG19, and ViT, respectively, using average pooling. 2.3 Unimodal Modeling As the downstream network for unimodal representations, we employ a CNN-based architecture. Extracted PTM representations are passed through two 1D-CNN layers (64 and 128 filters, kernel size 3), each followed by max pooling. The resulting features are flattened and fed into a fully connected layer of 128 neurons, before an output layer with softmax activation produces the class probabilities. Figure 1 : FOCA ; HCA ( \oplus ) represents Hyperbolic Cross-attention, mobius addition respectively 2.4 FOCA We propose FOCA , a novel framework for the