LLM-assisted Agentic Edge Intelligence Framework

Chinmaya Kumar Dehury, Siddharth Singh Kushwaha, Qiyang Zhang, Alaa Saleh, Praveen Kumar Donta · 2026

Read LLM-assisted Agentic Edge Intelligence Framework by Chinmaya Kumar Dehury, Siddharth Singh Kushwaha, Qiyang Zhang, Alaa Saleh, Praveen Kumar Donta (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-09607.txt.

Introduction

Dehury and four colleagues address edge analytics that stay hard-coded until engineers rewrite and redeploy scripts. Their LEI framework lets a cloud-hosted language model coordinate device-side logic as conditions change, removing manual business rules. Candidate programs are generated and checked against data and constraints. The winner is deployed to each device, tailored by sample data, metadata, and resource limits. Tests on four datasets show low processor and memory use while adapting to change: air quality, temperature and humidity, wind, and soil.

Catalog record

CREATORChinmaya Kumar Dehury, Siddharth Singh Kushwaha, Qiyang Zhang, Alaa Saleh, Praveen Kumar Donta
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63072
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.09607

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LLM-assisted Agentic Edge Intelligence Framework Chinmaya Kumar Dehury, Siddharth Singh Kushwaha, Qiyang Zhang, Alaa Saleh, Praveen Kumar Donta Abstract Edge intelligence delivers low-latency inference, yet most edge analytics remain hard-coded and must be redeployed as conditions change. When data patterns shift or new questions arise, engineers often need to write new scripts and push updates to devices, which slows iteration and raises operating costs. This limited adaptability reduces scalability and autonomy in large, heterogeneous, and resource-constrained edge deployments, and it increases reliance on human oversight. Meanwhile, large language models (LLMs) can interpret instructions and generate code, but their compute and memory requirements typically prevent direct deployment on edge devices. We address this gap with the LLM-assisted Edge Intelligence (LEI) framework, which removes the need for manually specified business logic. In LEI, a cloud-hosted LLM coordinates the creation and update of device-side logic as requirements evolve. The system generates candidate lightweight programs, checks them against available data and constraints, and then deploys the selected version to each device. This lets each device receive a tailored program based on sample data, metadata, context, and current resource limits. We evaluate LEI on four heterogeneous datasets, including air quality, temperature & humidity, wind, and soil datasets using multiple LLM backends. The experimental results show that the framework maintains low average CPU and memory utilization during the execution. These results indicate that the framework adapts efficiently to changing conditions while maintaining resource efficiency. keywords Edge Intelligence, Large Language Models, Agentic AI, Resource Management, Context-aware Orchestration 1 Introduction Edge computing refers to the ability to process and analyze locally generated data without requiring constant communication with centralized cloud services 14 . Beyond data processing, the integration of AI with edge computing, referred to as Edge Intelligence, enables on-device decision-making, reduces latency and cloud dependency 26 ; 22 . For example, in smart traffic management systems, edge devices—such as smart traffic lights, cameras, and sensors—work independently to adjust traffic signals in real time based on local data about the crowd and other emergency situations. These systems can detect traffic congestion, accidents, or pedestrian movement and make decisions instantly, improving traffic flow, reducing delays, and enhancing safety 6 . Embedded analytics on edge devices support sensing and basic decision-making, but the underlying logic is usually fixed at deployment. Devices therefore rely on predefined rules and models, with limited ability to adjust when conditions shift. Changes in data distributions, sensing modalities, operational goals, or available compute and energy can quickly invalidate existing pipelines. When data formats or semantics evolve, or when analytical goals change, updates are often required not only to the processing logic but also to task scheduling and execution on-device 13 . Hardware diversity adds another layer of difficulty, since the same workload can behave very differently across devices and may require device-specific tuning to avoid overload or failure. Similarly, introducing new sensors disrupts the entire development-to-deployment pipeline. As a result, developers must manually analyze metadata, interpret context, redesign algorithms, ensure hardware compatibility, optimize for constrained resources, and redeploy updated code to each device. This process is time-consuming, error-prone, and may require temporary service interruption. Continuous human supervision throughout the life-cycle limits scalability, delays adaptation, and increases maintenance overhead across distributed edge infrastructures 5 . Figure 1: Conventional Edge devices with AI model On the other hand, Large Language Models (LLMs), such as OpenAI’s GPT-5.2 1 1 1 https://openai.com/index/chatgpt/ , Anthropic’s Claude 2 2 2 https://claude.ai/ , or DeepSeek-V3.1 3 3 3 https://www.deepseek.com/en , are growing rapidly and capable of understanding and generating human-like text based on large-scale datasets. Such models are used in a wide variety of applications ranging from natural language understanding and translation to content generation and code generation 11 . These capabilities have been recently adapted in several fields, including healthcare, education, customer service, and finance, by automating tasks that require sophisticated understanding and reasoning 19 . However, deploying LLMs directly (refer to Figure 1 ) on edge devices is not feasible due to the computational demands of these models. Edge devices are typically resource-constrained, with limited processing power, memory, and storage, and hence, are not a suitable environment to run such large-scale models 17 . Edge device hardware and software stacks generally cannot accommodate models of that size, even when real-time inference is not a requirement 1 . Given this, LLMs can be leveraged in edge computing systems through two primary strategies: (i) compressing and optimizing models for direct on-device deployment, and (ii) enhancing edge hardware to satisfy LLM resource requirements. In the first approach, techniques such as quantization, pruning, and distillation reduce memory footprint and computational demand, making inference feasible on resource-constrained devices; however, aggressive compression often degrades accuracy, robustness, and reasoning performance by reducing representational capacity and limiting effectiveness on complex or out-of-distribution tasks. In the second approach, equipping edge nodes with more capable processors (e.g., GPUs/NPUs), larger memory, and greater storage can sustain higher-quality LLM inference, but this increases capital and operating costs, energy consumption, and thermal constraints, and it does not scale easily across large, heterogeneous edge infrastructures—making widespread deployment economically impractical in many settings 1 . While both approaches have their merits, they also present significant challenges that must be carefully addressed. In aggregate, these limitations motivate a shift from purely model- or hardware-centric solutions toward agentic, LLM-assisted orchestration frameworks that enable edge systems to adapt, optimize, and evolve autonomously under tight resource constraints. There is increasing demand for a framework that enables edge devices to autonomously upgrade their intelligence using LLMs, with minimal to no human involvement. Such a framework should close the gap between static, hard-coded edge logic and adaptive, continuously improving edge behavior. In response, we propose LLM-assisted Edge Intelligence (LEI) — an agentic architecture in which the LLM serves as a cognitive orchestrator. LEI ingests sample data, metadata, and contextual signals; profiles device-side constraints (e.g., compute, memory, energy, and latency); synthesizes resource-aware, task-specific programs; and deploys them back to the edge for local execution. This design supports continuous capability evolution while preserving low latency and efficient resource use. Agentic decision-making at the orchestration layer further allows edge nodes to respond to environmental changes and workload shifts without slowing time-critical inference. Our contributions are summarized as follows: 1. We have designed and developed a framework that dynamically generates business logic based on real-time context using LLMs, minimizing the need for manual intervention. 2. To enhance system robustness and security, a dedicated component is introduced to validate faulty business logic, developed by LLMs, before further processing. 3. Our proposed framework is adaptive in nature, which generates lightweight code and consumes minimal CPU and memory usage based on the available resources of edge devices, allowing users to perform other tasks without worrying about the resources of edge devices. The remainder of this paper is organized as follows: Section 2 reviews the existing literature on large language models (LLMs) and their deployment on resource-constrained devices. Section 3 presents a high-level overview of the proposed architecture. Section 4 describe the implementation across different use cases, models and performance metrics. The experimental results are presented in Section 4.5 . Finally, Section 6 concludes the paper. Table- 1 provides the list of acronyms used in this paper. Table 1: List

Read the complete text