DriveSafe: A Hierarchical Risk Taxonomy for Safety-Critical LLM-Based Driving Assistants

Abhishek Kumar, Riya Tapwal, Carsten Maple · 2026

Read DriveSafe: A Hierarchical Risk Taxonomy for Safety-Critical LLM-Based Driving Assistants by Abhishek Kumar, Riya Tapwal, Carsten Maple (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-12138.txt.

Introduction

Abhishek Kumar, Riya Tapwal, and Carsten Maple build a risk taxonomy for language model driving aides. Such aides now sit in cars, where bad or vague advice can cause real harm. It can also breach law or ethics on the road.

Past safety lists are too broad to catch driving risks. Their DriveSafe taxonomy sorts risks by the driving case. A clear list helps teams test aides on road cases before launch. It also helps map which faults are most grave in a car. A shared list helps car firms test aides the same way.

Catalog record

CREATORAbhishek Kumar, Riya Tapwal, Carsten Maple
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE23 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-12138

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DriveSafe: A Hierarchical Risk Taxonomy for Safety-Critical LLM-Based Driving Assistants Abhishek Kumar, Riya Tapwal, Carsten Maple Abstract Large Language Models (LLMs) are increasingly integrated into vehicle-based digital assistants, where unsafe, ambiguous, or legally incorrect responses can lead to serious safety, ethical, and regulatory consequences. Despite growing interest in LLM safety, existing taxonomies and evaluation frameworks remain largely general-purpose and fail to capture the domain-specific risks inherent to real-world driving scenarios. In this paper, we introduce DriveSafe, a hierarchical, four-level risk taxonomy designed to systematically characterize safety-critical failure modes of LLM-based driving assistants. The taxonomy comprises 129 fine-grained atomic risk categories spanning technical, legal, societal, and ethical dimensions, grounded in real-world driving regulations and safety principles and reviewed by domain experts. To validate the safety relevance and realism of the constructed prompts, we evaluate their refusal behavior across six widely deployed LLMs. Our analysis shows that the evaluated models often fail to appropriately refuse unsafe or non-compliant driving-related queries, underscoring the limitations of general-purpose safety alignment in driving contexts. 1 Introduction Large Language Models (LLMs) are rapidly being integrated into modern vehicles, transforming them from passive information systems into conversational copilots that support natural interaction, real-time guidance, and contextual decision support. Major automotive manufacturers, including Mercedes-Benz, have begun integrating advanced conversational AI into in-vehicle assistants such as MBUX, powered by large-scale generative models Shakir 2025 , and BMW, which is enhancing its Intelligent Voice Assistant with large-model based capabilities to provide personalized suggestions and context-aware responses BMW UK 2025 , are investing heavily in AI-driven voice interfaces. In parallel, other industry players such as Tesla have also introduced generative AI-based assistants like Grok and voice interaction features designed to support natural dialogue and in-vehicle assistance, signaling a broader shift toward conversational AI in automotive systems TeslaAccessories.com 2025 . Renault Group has similarly announced the integration of large language models, including ChatGPT-4o, through its partnership with OpenAI to support in-vehicle assistants and automotive software platforms Renault UK 2026 , while General Motors and Volkswagen have reported ongoing efforts to deploy generative AI for infotainment, diagnostics, and driver interaction systems Volkswagen Group 2025 ; General Motors 2025 . Beyond industry adoption, research has proposed frameworks for leveraging LLMs to enable human-like interaction and reasoning in autonomous vehicles, demonstrating the potential of these models to interpret, contextualize, and respond to rich verbal input in driving scenarios Cui et al. 2024 . These developments highlight the rapid movement toward LLM-enabled automotive systems that promise richer dialogue, improved user experience, and enhanced decision support in real-world driving environments. While these advancements promise more natural human-machine interaction, they simultaneously introduce a new class of safety-critical risks. Unlike traditional voice assistants that operate on fixed templates, LLMs generate free-form responses that may (i) hallucinate incorrect information Ji et al. 2023 , potentially leading to illegal driving advice, (ii) provide inconsistent interpretations of traffic laws across jurisdictions Cao et al. 2024 , (iii) display ethical or moral misjudgments in ambiguous driving situations, or (iv) be susceptible to adversarial prompts crafted by malicious passengers Shu et al. 2023 ; Arditi et al. 2024b . Errors that would be benign in everyday chat become high-stakes when produced inside a moving vehicle. Existing safety frameworks and benchmarks, such as SafetyBench Zhang et al. 2024a , XSTest Röttger et al. 2024b , SafeWorld Li et al. 2024 , AIR-Bench Zeng et al. 2025 , and regulatory standards like NIST AI Risk Management Framework of Standards and Technology 2023 or ISO/IEC 42001 for Standardization 2023 , offer valuable general-purpose safety evaluations. However, they lack the domain-specific operational detail required for driving environments, where model behavior must align with traffic law, situational awareness, human factors, environmental variability, and ethical principles unique to transportation. Driving assistants may encounter potentially harmful queries such as: “Can I overtake on the right if the left lane is blocked?” General-purpose safety benchmarks cannot meaningfully evaluate these scenarios, nor do they capture the nuanced failure modes that arise when LLMs operate in vehicular settings. Driving contexts are governed by strict legal constraints, temporal decision pressure, and complex safety trade-offs that are not adequately represented in existing evaluations. As a result, LLMs are increasingly deployed in vehicles without any dedicated benchmark that reflects the operational, legal, and ethical realities of real-world driving. To address this gap, we introduce DriveSafe, a domain-specific safety framework grounded in a hierarchical, four-level taxonomy of driving-related risks. The proposed taxonomy is informed by real-world regulations and safety principles, including the UK’s Automated Vehicles Act 2024 UK Parliament 2024 and the Statement of Safety Principles UK Department for Transport 2024 ; European Union 2024 , as well as prior work on legal-risk modeling such as SafeLawBench Cao et al. 2025a . It comprises 129 safety-relevant risk categories spanning technical, legal, ethical, real-time, and fairness-related dimensions, capturing challenges such as semantic drift in legal interpretation, ambiguous jurisdiction-dependent rules, cognitive overload, and adversarial jailbreak scenarios. The taxonomy was reviewed by domain experts to ensure practical relevance and alignment with real-world policy and regulatory considerations. 2 Related Work 2.1 Refusal Behavior in LLMs Recent research has focused on making LLM refusals more precise and controllable. Arditi et al. Arditi et al. 2024a found that refusal behavior lies in a one-dimensional subspace, enabling direct manipulation but exposing the fragility of safety tuning. Wen et al. Wen et al. 2025 expanded this by framing refusal as a meta-skill tied to the query, model, and human values. To enable context-aware abstention, Jain et al. Jain et al. 2025 introduced refusal tokens, though scalability remains a challenge. Dabas et al. Dabas et al. 2025 addressed over-refusal via ACTOR, which fine-tunes latent components to reduce false positives. Building further, Cao et al. Cao et al. 2025b proposed SCANS to steer activations, effectively balancing safety and helpfulness. 2.2 Safety Evaluation Benchmarks and Taxonomies Recent efforts have focused on building robust benchmarks to evaluate LLM safety across diverse contexts. Zhang et al. Zhang et al. 2024b introduced SafetyBench, covering over 11,000 questions across seven safety domains, revealing that even top models like GPT-4 leave significant gaps. Expanding to regulatory compliance, Zeng et al. Zeng et al. 2025 proposed AIR-Bench, exposing jurisdictional inconsistencies in legal alignment across 22 models. Addressing cultural and regional nuances, SAFEWORLD Yin et al. 2025 tested geo-sensitive scenarios and showed that their fine-tuned SafeWorldLM outperformed GPT-4 in contextually appropriate responses. For multimodal systems, Liu et al. Liu et al. 2024 introduced MM-SafetyBench, revealing visual safety bypasses and proposing robust defenses. Finally, Röttger et al. Röttger et al. 2024a developed XSTest to detect exaggerated safety behaviors, showing that many models over-refuse safe prompts, underscoring the need for finer safety calibration. Figure 1: Hierarchical Risk Taxonomy for Driving AI Systems. The taxonomy is divided into four high-level domains: Techni- cal, Business, Societal and Ethical Risks, which further decompose into categories, failure modes, and specific risk types. 2.3 Safety and Alignment in High-Stakes Domains Recent work emphasizes the importance of aligning LLMs with real-world values and domain-specific safety needs. Wang et al. Wang et al. 2025 advocated for stakeholder-centric design in healthcare, involving diverse voices to ensure cultural sensitivity and trust. Extending to dynamic environments, Hua et al. Hua et al. 2024 proposed TrustAgent, which uses an “Agent

Read the complete text