CoSense-LLM: Semantics-at-the-Edge with Cost- and Uncertainty-Aware Cloud–Edge Cooperation
Hasan Akgul, Mari Eplik, Javier Rojas, Aina Binti Abdullah, Pieter van der Merwe · 2025
Read CoSense-LLM: Semantics-at-the-Edge with Cost- and Uncertainty-Aware Cloud–Edge Cooperation by Hasan Akgul, Mari Eplik, Javier Rojas, Aina Binti Abdullah, Pieter van der Merwe (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-19670.txt.
Introduction
Hasan Akgul and four co-authors present CoSense-LLM. The framework reads live sensor streams at the edge of the network, not in a far-off cloud. Sensors such as Wi-Fi channel data, motion units, audio, and RFID make raw streams. Those streams are heavy to send and risk privacy. The system turns streams into compact semantic tokens on the device itself.
Four parts work together. An encoder squeezes sensor data into short code sequences. A local retrieval layer grounds the output in site policies and notes. A router picks one of three paths: answer on the device, answer with retrieval help, or send a compact request to the cloud. It weighs cost against how unsure the system is. A secure path redacts data, so raw signals never leave the device. In home, office, and clinic tests, replies stayed under a second, bandwidth costs fell, and privacy held.
Catalog record
| CREATOR | Hasan Akgul, Mari Eplik, Javier Rojas, Aina Binti Abdullah, Pieter van der Merwe |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 91 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-19670
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CoSense-LLM: Semantics-at-the-Edge with Cost- and Uncertainty-Aware Cloud–Edge Cooperation Hasan Akgul, Mari Eplik, Javier Rojas, Aina Binti Abdullah, Pieter van der Merwe Abstract We present CoSense-LLM , an edge-first framework that converts continuous multimodal sensor streams (e.g., Wi-Fi CSI, IMU, audio, RFID, lightweight vision) into compact, verifiable semantic tokens and coordinates with large language models (LLMs) under explicit latency, energy, bandwidth, and privacy constraints. CoSense-LLM comprises four components: (i) SenseFusion , a lightweight encoder that aligns sensor embeddings with language and compresses them into short, discrete code sequences; (ii) Edge-RAG , a local hybrid retrieval layer that grounds generation in site-specific policies and notes; (iii) PromptRouter , a cost- and uncertainty-aware policy that selects among edge-only generation, edge+retrieval, or compact cloud escalation; and (iv) Secure Execution , an auditable redaction path that enforces data minimization so raw waveforms never leave the device. The system is compatible with modern serving optimizations—including paged/streaming KV caches, FlashAttention-style kernels, speculative decoding, and quantized LoRA adapters—and supports on-device personalization and federated updates under non-IID drift. Across home, office, and clinic deployments with day-long streams, CoSense-LLM delivers grounded explanations while meeting tight service-level objectives: it sustains sub-second (p95) end-to-end latency on edge-dominant paths, reduces inter-tier token and bandwidth costs by preferring local, retrieval-grounded responses, and preserves privacy by transmitting only discrete codes and redacted metadata. Ablations show that Edge-RAG improves factual consistency and reduces contradictions, calibrated uncertainty enables selective abstention and controlled escalations, and KV/decoding accelerators materially lower energy per decision. The results support an edge-first design that treats semantics, privacy, and predictable latency as co-equal goals for large-model deployments in interference-prone environments. I Introduction Edge intelligence is moving from a vision of offloading a few convolutional layers to a concrete capability of running, splitting, and personalizing large language and multimodal models close to where data is produced. Recent surveys and system papers converge on two forces behind this shift: (i) the need to reduce end-to-end latency and bandwidth while respecting privacy; and (ii) the emergence of algorithm–system co-design that compresses models, manages memory, and orchestrates collaborative execution across device, edge, and cloud [ 30 , 46 , 58 , 19 ] . Cloud–edge collaboration for LLMs is now studied not only at the scheduler level but also at the model-graph level, enabling partitioned or cooperative inference and training [ 22 , 70 , 43 , 26 ] . Meanwhile, pervasive sensing applications—e.g., Wi-Fi CSI and RFID based activity or gesture understanding—motivate low-leakage, low-latency interpretation pipelines at the network edge rather than sending raw signals to distant servers [ 29 , 9 ] . A central challenge in edge LLM serving is memory and state management under tight resources. Modern serving stacks rely on paged key–value (KV) caches, dynamic cache admission/eviction, and streaming to decouple prefill and decode without stalling the device [ 47 , 24 , 7 , 37 , 15 , 71 , 12 ] . Kernel-level optimizations such as FlashAttention-2 and GPU-side decoding pipelines further increase throughput [ 11 , 6 ] . To keep conversational quality under constrained context windows, streaming designs maintain “attention sinks” and consistent states across long sessions [ 44 ] . System-level disaggregation and scheduling control prefill–decode tradeoffs and cluster utilization [ 16 , 23 , 54 ] , while shadow and hierarchical caches improve long-context efficiency [ 41 ] . These techniques are complementary to robustness-oriented sensing pipelines that must survive interference or distribution drifts before any language reasoning happens [ 13 , 5 ] . Offloading and storage-aware execution provide another axis of feasibility for edge scenarios. Single-GPU and heterogeneous CPU/NVMe pipelines amortize model state movement [ 14 , 35 ] , and performance-model-guided offloaders decide tensor placement and parallelism under device limits [ 28 ] . In-storage attention and serverless runtimes reduce hot-standby costs while keeping tail latency controlled for bursty workloads at the network edge [ 50 , 8 ] . Beyond placement, inference-time acceleration leverages program-style LM execution [ 1 ] , speculative decoding with verification or multi-head drafting [ 31 , 40 , 53 , 72 ] , token-utility–aware strategies [ 36 , 73 ] , and kernel-level attention improvements already noted [ 11 ] . These advances complement asynchronous and communication-efficient learning frameworks that adapt to edge heterogeneity during training [ 51 , 55 ] . Personalization and compression are equally decisive at the edge. Low-rank adaptation and its quantized fine-tuning variant enable per-tenant specialization within tight memory budgets [ 74 , 20 ] . Post-training quantization and activation-aware schemes, combined with serving-time co-design, bring 4–8-bit regimes into production without prohibitive quality loss [ 2 , 4 , 10 ] . Multi-tenant adapter serving avoids cold-start and context bloat by dynamically selecting adapters for tenants and tasks [ 3 , 52 ] . In parallel, federated and semi-supervised edge learning methods tailor encoders and adapters for user or site-specific distributions [ 60 , 61 , 75 , 76 , 77 , 17 ] . Such ingredients are crucial when the edge must understand noisy signals or site-specific semantics before handing off compact evidence to an LLM. Retrieval-augmented generation (RAG) at the edge addresses knowledge staleness and privacy by grounding responses in local corpora and telemetry. Online indexing and cacheable knowledge chunks improve hit rates under intermittent connectivity [ 33 , 18 ] . Hardware–algorithm co-design shows how computing-in-memory can robustly realize RAG primitives close to sensors [ 78 ] , while collaborative or decentralized serving amortizes expensive backbone layers across peers [ 26 ] . Bridging continuous signals and language further benefits from instruction-tuned multimodal models that map non-textual inputs to textual interfaces [ 57 , 39 ] . This is especially relevant for Wi-Fi/IMU/RFID/vision scenarios such as attention-based gesture recognition, unobtrusive emotion inference, pulmonary function sensing, open-set gestures, and affective modeling under label noise [ 25 , 34 , 32 , 45 , 21 ] , as well as RFID-based writing and identity sensing in the wild [ 48 ] . Security and privacy considerations sharpen the case for edge-first pipelines. Physical-layer authentication can be subverted via fingerprint-level attacks, motivating local verification, data minimization, and verifiable reasoning where possible [ 42 ] . Side-channel risks such as acoustic keystroke eavesdropping also argue for on-device processing and careful prompt/trace handling to limit raw data exposure [ 38 ] . In response, modern edge intelligence stacks pair robust signal encoders with privacy-preserving learning and adapter isolation, and exploit streaming or shard-based execution to avoid shipping sensitive context to the cloud [ 22 , 70 , 43 , 14 ] . Against this backdrop, our work targets low-latency, privacy-preserving event understanding that converts raw, noisy multimodal signals into compact semantic evidence at the edge, and coordinates with large models only when necessary. We build on scheduling and cache-management advances [ 47 , 24 , 7 , 37 , 15 , 71 , 12 ] , offloading and kernel pipelines [ 28 , 14 , 35 , 16 , 23 , 54 , 11 , 6 ] , and decoding accelerators [ 31 , 40 , 53 , 72 , 36 , 73 ] , while weaving in federated and decentralized optimization to cope with dynamic, non-IID edge populations [ 64 , 63 , 66