Assessing LLM code generation quality through path planning tasks
Wanyi Chen, Meng-wen Su, Mary Cummings · 2025
Read Assessing LLM code generation quality through path planning tasks by Wanyi Chen, Meng-wen Su, Mary Cummings (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-21276.txt.
Introduction
Code written by chat models may look right and still steer a car wrong. Wanyi Chen, Meng-wen Su, and Mary Cummings graded six such models in 2025 on a task with teeth. The models had to write code for three route-planning methods, then run it on maps of three hardness grades. Route math is a fair stand-in for safety work, since a wrong turn there is a crash in the real world.
The verdict was grim. The made code held real hazards, from stalls to bad paths, that a quick read would miss. The team say known code test sets are too soft to catch such faults. Their rule for safety work is plain: no model code ships without hard tests of the kind they ran.
Catalog record
| CREATOR | Wanyi Chen, Meng-wen Su, Mary Cummings |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-21276
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Assessing LLM code generation quality through path planning tasks Wanyi Chen, Meng-wen Su, Mary Cummings Abstract As LLM-generated code grows in popularity, more evaluation is needed to assess the risks of using such tools, especially for safety-critical applications such as path planning. Existing coding benchmarks are insufficient as they do not reflect the context and complexity of safety-critical applications. To this end, we assessed six LLMs’ abilities to generate the code for three different path-planning algorithms and tested them on three maps of various difficulties. Our results suggest that LLM-generated code presents serious hazards for path planning applications and should not be applied in safety-critical contexts without rigorous testing. 1 Introduction As code generated by Large Language Models (LLMs) is increasingly adopted in many domains, evaluation is needed to assess the possible risks and problems of using LLM-generated code, especially in safety-critical contexts like path planning. Path-planning algorithms play a crucial role in aircraft and vehicle navigation, especially in autonomous systems such as drones and self-driving cars. For these systems, errors in planning could lead to safety hazards and even deaths. To this end, we assessed six LLMs’s ability to generate code for the task of generating three path-planning algorithms: Dijkstra’s algorithm, the Voronoi algorithm, and the Rapidly-exploring Random Tree (RRT) algorithm. We evaluated the generated code as well as how effective the LLMs were in optimizing for the stated objectives. We then discuss the implications of the evaluation results, and hazards that result from LLM-generated code used in safety-critical contexts. 2 Background and related work Most code generation evaluation benchmarks have problem sets similar to programming competitions. The problem setup typically includes a function signature and a docstring, which is natural language text that describes the purpose of the function. The LLMs are expected to automatically complete the function, and the correctness of the generated code is assessed through pre-written unit tests. HumanEval [ 6 ] is a widely used example of this type of benchmark. It includes 164 handwritten programming problems and an average of 7.7 unit tests per problem [ 6 ] . A survey by Jiang et al. [ 12 ] provides a more comprehensive list of programming competition-style benchmarks. However, real-world software engineering tasks are often more complex than the problems addressed in the programming competition-style benchmarks. To evaluate LLMs’ ability to complete real-world software engineering tasks, some researchers proposed task-based benchmarks, such as SWE-bench [ 13 ] . SWE-bench utilizes 2,294 software engineering problems drawn from real GitHub issues and corresponding pull requests across 12 popular Python repositories [ 13 ] . Given a codebase and a description of an issue to be resolved, a language model is tasked with editing the codebase to address the issue [ 13 ] . However, other researchers found that some solutions were provided directly in the issue report or comments and therefore doubted whether SWE-bench could meaningfully assess LLMs’ software engineering abilities [ 3 ] . This experiment differed from programming competition-style benchmarks in that we did not provide function signatures. The LLMs were expected to determine the function’s input and output and generate appropriate function signatures based on natural language descriptions. Furthermore, whereas most programming competition tasks focus on implementing a single function, in our experiment, the LLMs needed to generate multiple functions. The LLMs determined the number of helper functions as well as how the main function should call the helper functions. Our experiment better captured the context and complexity of real-world software engineering tasks, and further avoided the problem present in SWE-bench: no solutions were offered in our prompts. Previous evaluations of LLM-generated code found several patterns. First, LLM code generation is non-deterministic [ 19 ] . LLMs could return very different code for the same prompt [ 19 ] . In our experiment, we accounted for the non-determinism into consideration by allowing the LLMs to regenerate responses for up to five times given the same prompt. Second, LLM-generated code is error prone. Tambon et al. identified ten distinctive bug patterns in LLM-generated code [ 21 ] , and Liu et al. identified five categories of hallucinations [ 17 ] . Their evaluations were primarily based on programming competition-style questions such as HumanEval [ 17 , 21 ] . In our experiments, we continue to explore and evaluate patterns in LLM-generated code in safety-critical contexts. 3 Method To assess LLM code generation quality, we tasked six LLMs to generate three path planning algorithms. Path planning is essentially an optimization task in that such algorithms can find the best path for the shortest distance, lowest runtime, maximum clearance around obstacles, etc. Our experiments aim to answer two research questions (RQs): 1) RQ 1: If no optimization objectives are specified, how effective is the LLM code, and what are the default LLM objectives, and 2) RQ 2: If an optimization objective is specified, how effective is the LLM code in optimizing for the stated objective? 3.1 LLMs and Algorithms We tested six LLMs of various sizes: Mistral 7B (developed by Mistral AI, 7.3 billion parameters), Gemini (developed by Google DeepMind, 70 billion parameters), DeepSeek (developed by DeepSeek, 671 billion parameters), GPT-4o (developed by OpenAI, 1.8 trillion parameters), Copilot (developed by OpenAI, 1.8 trillion parameters, fine-tuned for code generation), and Grok 3 (developed by xAI, 2.7 trillion parameters). We prompted each LLM to generate the following path-planning algorithms, which are listed in order of increasing complexity: • Dijkstra’s algorithm : Dijkstra’s algorithm is a fundamental method for finding the shortest paths from a source node to all other nodes in a graph. It uses a priority queue to track the minimum cost to reach each node, initially set to infinity, except for the source node which starts at zero. The algorithm selects the node with the lowest cost, updates its neighbors’ costs, and adjusts the priority queue accordingly. This continues until the shortest paths to all nodes are determined, utilizing a systematic and greedy approach [ 7 ] . • Voronoi algorithm : The Voronoi diagram can be used to optimize path planning by maximizing the distance from obstacles. It partitions space into regions, forming a graph with edges that are equidistant from multiple obstacles. The Voronoi algorithm constructs a Voronoi diagram from the obstacles, removes edges that intersect with obstacles, and connects the start and goal to the nearest vertices. A shortest path algorithm, such as A or Dijkstra’s algorithm, is then used to find a path that travels along the edges of the Voronoi diagram. The Voronoi algorithm is especially desirable in cases where maintaining maximum clearance from obstacles is crucial [ 5 ] . • Rapidly-exploring random tree (RRT) : RRT is a key method in path finding and motion planning for robotics and autonomous vehicles. It can efficiently explore high-dimensional spaces by building a space-filling tree randomly. Starting from an initial configuration, the tree expands toward randomly chosen points while avoiding collisions and meeting movement constraints. This process continues until a path to the goal is found or a maximum iteration limit is reached. RRT is favored for its speed and effectiveness in complex environments, unlike traditional grid-based algorithms that struggle in these situations [ 15 ] . 3.2 Experimental setup Figure 1: