A Serverless Edge-Native Data Processing Architecture for Autonomous Driving Training
1 st Fabian Bally, 2 nd Michael Schötz, 3 rd Thomas Limbrunner · 2026
Read A Serverless Edge-Native Data Processing Architecture for Autonomous Driving Training by 1 st Fabian Bally, 2 nd Michael Schötz, 3 rd Thomas Limbrunner (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22919.txt.
Introduction
Fabian Bally, Michael Schötz, and Thomas Limbrunner design a data pipeline for training self-driving cars. Data both enables and bottlenecks that training, since cars record far more than teams can process. Rare but safety-critical events matter most, yet finding them takes long hours of driving.
Their serverless, edge-native design processes data near where it is gathered, scaling up only when needed. Smart selection can surface the rare events that teach cars to handle danger. The goal is training sets that are balanced, not just big. Balanced coverage is the stated goal, since a fleet can drive safely for months yet still miss the one dangerous case.
Catalog record
| CREATOR | 1 st Fabian Bally, 2 nd Michael Schötz, 3 rd Thomas Limbrunner |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-22919
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Serverless Edge-Native Data Processing Architecture for Autonomous Driving Training 1 st Fabian Bally, 2 nd Michael Schötz, 3 rd Thomas Limbrunner Abstract Data is both the key enabler and a major bottleneck for machine learning in autonomous driving. Effective model training requires not only large quantities of sensor data but also balanced coverage that includes rare yet safety-critical scenarios. Capturing such events demands extensive driving time and efficient selection. This paper introduces the Lambda framework, an edge-native platform that enables on-vehicle data filtering and processing through user-defined functions. The framework provides a serverless-inspired abstraction layer that separates application logic from low-level execution concerns such as scheduling, deployment, and isolation. By adapting Function-as-a-Service (FaaS) principles to resource-constrained automotive environments, it allows developers to implement modular, event-driven filtering algorithms while maintaining compatibility with ROS 2 and existing data recording pipelines. We evaluate the framework on an NVIDIA Jetson Orin Nano and compare it against native ROS 2 deployments. Results show competitive performance, reduced latency and jitter, and confirm that lambda-based abstractions can support real-time data processing in embedded autonomous driving systems. The source code is available at https://github.com/LASFAS/jblambda. I Introduction Autonomous driving systems depend on large volumes of sensor data to develop reliable perception, prediction, and planning capabilities. While extensive datasets are essential for these subsystems, increasing data volume alone does not guarantee improved performance. As shown by Makansi et al. [ 13 ] , the distribution of recorded scenarios, particularly the inclusion of rare yet safety critical events, plays a decisive role in determining model generalization and reliability. Rare but safety critical events occur infrequently compared to routine driving, making their collection a major challenge for dataset design. As shown by Peng et al. [ 14 ] , this imbalance between common and rare scenarios significantly limits perception performance in long tail traffic situations. Capturing such events requires filtering through extensive driving data; however, performing this selection offline after recording leads to excessive storage and labeling overhead. Executing the filtering process directly on the vehicle enables early removal of irrelevant data, thereby reducing bandwidth and storage demands while preserving the diversity and relevance of the resulting dataset. A promising approach to mitigate this imbalance is selective data recording at the edge. Instead of indiscriminate logging, the vehicle performs onboard relevance evaluation and retains only those segments deemed valuable based on criteria such as novelty, uncertainty, or scenario criticality. This strategy reduces storage and labeling costs while prioritizing events of greater significance for downstream model development. However, deploying such methods in vehicular environments introduces nontrivial challenges: algorithms must operate under strict latency, memory, and energy constraints typical of embedded systems, while accommodating the inherent heterogeneity of vehicle hardware, sensor configurations, and software stacks [ 11 ] . To address these challenges, this work proposes a serverless edge computing framework for selective data capture. The framework abstracts platform specific concerns such as memory management and task scheduling, providing an execution environment analogous to cloud based function invocation. Developers can focus exclusively on the task specific algorithm, while the runtime manages orchestration, isolation, and deployment across vehicles. By integrating principles of Function as a Service (FaaS) computing into resource constrained automotive systems, the framework enables rapid algorithm development, simplifies integration, and enhances portability across heterogeneous embedded platforms, while maintaining compliance with the stringent latency, memory, and energy requirements inherent to on vehicle data recording. II Related work II-A Datasets in Automotive and their Distributions To understand distributions in datasets, it is best to consider open datasets such as NuScenes [ 3 ] or the Zenseact Open Dataset [ 2 ] . Mingyu Liu et al. [ 10 ] provide a comprehensive study of the distributions of various open datasets across different modalities. By comparing dataset size, scenario types, citation counts, and other aspects, they derive a so-called impact score for all datasets. Notably, the size of a dataset does not directly correlate with its impact score . Junyao Guo et al. [ 5 ] examine how scene complexity in autonomous driving datasets impacts machine learning performance. Their Driveability Assessment considers static and dynamic factors such as weather, lighting, and time of day but lacks a single quantitative metric. Although they identify diverse datasets, the study concludes that open datasets remain insufficient for robust autonomous driving algorithms. The authors propose targeted data acquisition, involving synthetic scenario creation, driver participation, and richer annotations to enhance data diversity. II-B Data Filtering Algorithms for Autonomous Driving Training and Test Data Data engines are used to iteratively collect data that aligns with a target distribution, often leveraging machine learning models. In [ 9 ] , the authors present a data engine implementation called AIDE, which employs vision-language models (VLMs) to identify gaps in the dataset, retrain a detection model within the loop, and verify whether the retraining was successful. It is important to note that while this approach shows promising results, it is not intended to be applied to the realtime data stream of a recording vehicle. Similarly, the work in [ 17 ] introduces a joint out-of-distribution filtering and active learning strategy that prioritizes informative samples during continuous data collection. The method operates within an iterative cloud-based pipeline and focuses on improving dataset diversity rather than optimizing for edge-level execution. A complementary approach is presented by [ 12 ] , where the authors propose an active learning framework for autonomous driving that selects samples based on their contribution to downstream driving performance. While these methods effectively reduce labeling effort, they assume centralized computation and abundant resources, which limits their applicability to embedded or in-vehicle environments. II-C Function-as-a-Service Architectures for Edge and Automotive Applications Function-as-a-Service (FaaS) abstracts application logic into independently deployable and event-driven functions that execute within managed runtime environments. Li et al. [ 8 ] provide a comprehensive survey of serverless computing, discussing architectural components, scheduling mechanisms, and performance implications. Their work highlights that while cloud-based FaaS achieves elasticity and ease of management, it often introduces substantial overhead when applied to latency-sensitive or resource-constrained systems. Hall et al. [ 6 ] present an execution model for serverless functions at the network edge, emphasizing reduced cold-start latency and efficient resource utilization through lightweight runtime management. Similarly, Shillaker and Pietzuch [ 18 ] propose Faasm, a stateful FaaS runtime employing lightweight isolation to improve execution efficiency and memory sharing, which is particularly relevant for embedded workloads. Raith et al. [ 16 ] conduct a recent survey on serverless edge computing, identifying key research challenges such as function placement, energy efficiency, and quality-of-service guarantees across the cloud–edge continuum. As these works address serverless execution at the edge in a broad sense, they focus on general-purpose deployment models and system-level challenges beyond the scope of this paper. In contrast, our work concentrates on a narrowly defined execution model and evaluates its impact under a controlled and consistent baseline, which motivates comparison against native implementations rather than against full-fledged serverless platforms. In the automotive context, Alam et al. [ 1 ] introduce Serverless Vehicular Edge Computing (SVEC), extending FaaS principles to the Internet of Vehicles. Their work focuses on offloading computation to external edge infrastructure and evaluating feasibility using emulation. In contrast, this paper targets serverless execution directly on in-vehicle hardware