Automated Factual Benchmarking for In-Car Conversational Systems using Large Language Models

Rafael Giebisch, Ken E. Friedl, Lev Sorokin, Andrea Stocco · 2025

Read Automated Factual Benchmarking for In-Car Conversational Systems using Large Language Models by Rafael Giebisch, Ken E. Friedl, Lev Sorokin, Andrea Stocco (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-01248.txt.

Introduction

Voice assistants in cars can state falsehoods with confidence, and checking every answer by hand is slow. Rafael Giebisch and co-authors build an automatic checker in this 2025 paper that uses large language models to grade a car assistant's answers against the vehicle manual.

They try five grading setups, mixing model ensembles and role prompts to steady the verdicts. The best setup agreed with expert judges on factual correctness more than nine times in ten, and it answered in a few seconds. Testing ran on a new data set built for the car domain. The results suggest model-based grading is a workable way to vet in-car assistants.

Catalog record

CREATORRafael Giebisch, Ken E. Friedl, Lev Sorokin, Andrea Stocco
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-01248

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Automated Factual Benchmarking for In-Car Conversational Systems using Large Language Models Rafael Giebisch, Ken E. Friedl, Lev Sorokin, Andrea Stocco Abstract In-car conversational systems bring the promise to improve the in-vehicle user experience. Modern conversational systems are based on Large Language Models (LLMs), which makes them prone to errors such as hallucinations, i.e., inaccurate, fictitious, and therefore factually incorrect information. In this paper, we present an LLM-based methodology for the automatic factual benchmarking of in-car conversational systems. We instantiate our methodology with five LLM-based methods, leveraging ensembling techniques and diverse personae to enhance agreement and minimize hallucinations. We use our methodology to evaluate CarExpert, an in-car retrieval-augmented conversational question answering system, with respect to the factual correctness to a vehicle’s manual. We produced a novel dataset specifically created for the in-car domain, and tested our methodology against an expert evaluation. Our results show that the combination of GPT-4 with the Input Output Prompting achieves over 90% factual correctness agreement rate with expert evaluations, other than being the most efficient approach yielding an average response time of 4.5s. Our findings suggest that LLM-based testing constitutes a viable approach for the validation of conversational systems regarding their factual correctness. I Introduction Researchers and companies are exploring the potential of Large Language Models (LLMs) to tackle complex tasks across diverse domains, such as content quality classification of Q&A websites [ 1 ] , code translation [ 2 ] , security vulnerability detection [ 3 ] and autonomous driving [ 4 , 5 , 6 ] . In the automotive domain, LLMs can be utilized to develop modern in-car conversational systems. These systems enhance driver and passenger experiences by enabling natural, context-aware interactions for navigation, entertainment, and vehicle control [ 7 ] . These systems are expected to provide real-time information, adapt to user preferences, thereby reducing the need for manual inputs, and enhancing overall driving experience and safety [ 8 ] . However, despite their potential advantages, LLM-based in-car conversational systems also present critical development challenges in terms of factual correctness, latency, and privacy. In this paper, we target the automated testing of factual correctness of answers provided by a LLM-based conversational system, as it represents the vital requirement to be satisfied and thoroughly tested [ 9 ] (i.e., the system should respond with factually accurate information), without which such systems would be hardly accepted in production [ 10 ] . Particularly, in this paper we focus on the evaluation of CarExpert [ 11 ] , an in-car conversational system developed at BMW for quality assessment. Among the many features, CarExpert is designed to engage drivers in natural, multi-turn conversations about the vehicle and its features ( Figure 1 ). The system’s responses are informed by data derived from the owner’s manual, which has been parsed, annotated by domain experts, embedded, and stored in a vector database to serve as ground truth for CarExpert. In literature, there exists several manual approaches to help minimize hallucinations and factual inaccuracies in conversational system responses [ 12 , 13 , 14 ] . However, manually debugging these defects is generally impractical for engineers due to the extensive domain knowledge and time required for a comprehensive inspection. This paper investigates the problem of building a black-box testing framework for the automated benchmarking of the factual correctness of in-car conversational-systems, such as CarExpert [ 11 ] , to reduce the manual workload for engineers. The black-box approach is necessary as these systems are often developed, fully or partially, by third parties. Our approach leverages LLMs, due to their impressive performance in natural language processing, while minimizing hallucinations through techniques such as ensembling, diverse personae, and majority voting. Fig. 1 : Example dialogues showing both positive (top) and negative (bottom) interactions between a user and CarExpert. In the second response, CarExpert fails to combine documents from the manual to provide a correct response. In our study, we evaluate the property of factual correctness using two dimensions, namely factual consistency and factual relevance. In the context of CarExpert, a system response is considered as factually consistent if it fully aligns with the information in the owner’s manual. The owners’ manual serves as a carefully curated dataset and source of ground truth. No additional information should be introduced, assumed, misinterpreted, taken out of context, or fabricated in the reponse to the user. A system response is factually relevant , if it is adequately addressing the user’s question. A response can be factually consistent, but not relevant, and vice versa (see Figure 1 ). In this paper, we first curated a dataset of question and answer pairs for the automotive domain, where questions were generated by humans, while answers were provided by CarExpert. Domain experts performed then a factual correctness evaluation on the answers. We then selected five different types of LLMs and reasoning methods, each with custom prompt templates. Each prompt received the generated answer by CarExpert and the relevant paragraphs retrieved from the manual by CarExpert as input. The prompt is then passed to our framework LLMs to evaluate the degree of factual correctness. We assessed the effectiveness of our framework by comparing our systems factually correctness evaluation responses with those collected using human experts. Additionally, we analyzed its efficiency to determine its suitability for real-time use. Our results show that combining multiple LLMs and methods yields over 90% accuracy in both factual relevance and consistency. GPT-4, utilizing the Input-Output Prompting method, achieved the best tradeoff between effectiveness and efficiency, with accuracy scores of 90% (relevance) and 92% (concistency), and an average execution time of 4.5 seconds per request. Fig. 2 : Overall process of our testing framework for the evaluation of factual correctness of in-car conversational systems. II Testing Conversational Systems II-A Conversational Question Answering Systems At BMW, CarExpert is an LLM-based question answering system. It can engage with the user in natural, human-like, multi-turn conversations about the car and its functionalities. Its data and ground truth are based on the contents of the owner’s manual along with related documents such as press articles. These documents have been parsed and annotated by domain experts, embedded and stored in a vector database. In this work, we aim to evaluate the answer generation capabilities of CarExpert, which we describe next. Given a user utterance, CarExpert retrieves in-car-domain-specific relevant documents which may include the potential answer. The system uses Retrieval Augmented Generation (RAG) for information retrieval and answer generation. Specifically, the modular architecture of CarExpert includes four primary sub-components: 1) orchestration, 2) semantic search, 3) answer generation, and 4) answer moderation. To improve factually correct system utterance generation, CarExpert includes control mechanisms on three modules: 1) it uses an input filter as part of the orchestrator, 2) it controls the answer generation by prompt design and 3) it filters it through an output filter. A heuristic is also applied to select the optimal answer, prioritizing both a close match to the user’s question and maintaining as much of the original wording as possible to ensure factual accuracy in information extraction and generation. Further, CarExpert integrates text-to-speech and speech-to-text translation, enabling seamless voice-based interaction. For a comprehensive description, we refer the reader to the relevant literature [ 11 ] . The use of

Read the complete text