TransitGPT: A Generative AI-based framework for interacting with GTFS data using Large Language Models Journal: Public Transport
Saipraneeth Devunuri, Lewis Lehe · 2024
Read TransitGPT: A Generative AI-based framework for interacting with GTFS data using Large Language Models Journal: Public Transport by Saipraneeth Devunuri, Lewis Lehe (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-06831.txt.
Introduction
Saipraneeth Devunuri and Lewis Lehe build TransitGPT, a chatbot that answers plain questions about public transit data. Transit data ships in GTFS, a standard feed of routes, stops, and times that few riders or planners can query without code. Their 2024 framework has a language model write Python code for each question. The code runs on a server holding the feed, and the results come back as numbers, tables, or interactive charts.
Users need no knowledge of the feed format or of programming. The model is steered by prompts alone, with no custom training. Open code lets any agency plug in its own feed. The authors aim to open transit data to staff and citizens who think in words, not queries.
Catalog record
| CREATOR | Saipraneeth Devunuri, Lewis Lehe |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-06831
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TransitGPT: A Generative AI-based framework for interacting with GTFS data using Large Language Models Journal: Public Transport Saipraneeth Devunuri, Lewis Lehe Abstract This paper introduces a framework that leverages Large Language Models (LLMs) to answer natural language queries about General Transit Feed Specification (GTFS) data. The framework is implemented in a chatbot called TransitGPT with open-source code. TransitGPT works by guiding LLMs to generate Python code that extracts and manipulates GTFS data relevant to a query, which is then executed on a server where the GTFS feed is stored. It can accomplish a wide range of tasks, including data retrieval, calculations and interactive visualizations, without requiring users to have extensive knowledge of GTFS or programming. The LLMs that produce the code are guided entirely by prompts, without fine-tuning or access to the actual GTFS feeds. We evaluate TransitGPT using GPT-4o and Claude-3.5-Sonnet LLMs on a benchmark dataset of 100 tasks, to demonstrate its effectiveness and versatility. The results show that TransitGPT can significantly enhance the accessibility and usability of transit data. 1 Introduction 1.1 Background The General Transit Feed Specification (GTFS) is an open data standard for transit data. Started in 2005 as a collaboration between Google and TriMet (Portland, OR’s transit agency) ( Roth 2010 ; McHugh 2013 ) , today over 10,000 agencies from over 100 countries use GTFS to share and store their transit feeds publicly ( MobilityData 2024b ) . The scope of GTFS has expanded beyond schedules to include information about real-time updates, flexible services, and fares. Its widespread adoption has led to the development of software, programming libraries, and plugins for popular GIS software that help agencies create or visualize GTFS feeds. The principal use of GTFS is for navigation applications such as Google Maps, Apple Maps, Transit, etc. But in addition to helping people use transit, GTFS data is increasingly used to analyze , measure and understand transit systems. A spatio-temporal decomposition of GTFS has been utilized to construct O-D travel time matrices, generate isochrones, and estimate travel time uncertainty ( Pereira et al. 2021 ; Liu et al. 2024 ) . This decomposition has been used to conduct transit accessibility analysis by examining socioeconomic and spatial-temporal inequalities ( Fayyaz et al. 2017 ; Prajapati et al. 2020 ; Yan et al. 2022 ; Pereira et al. 2023 ) . Devunuri and Lehe 2024a ; Devunuri et al. 2024a decomposes GTFS feeds into segments for all US and Canadian transit agencies to measure statistics about bus stop spacings. Fortin et al. 2016 modeled the transit timetables as graphs and developed graph-oriented indicators, including connectivity between stops and the proportion of active stop pairs over time. GTFS has also been used to create informative visualizations. Kunama et al. 2017 developed ‘GTFS-Viz’, a tool to animate the movement of buses with time and visualize the number of buses in use. Para et al. 2024 created a user-friendly interface to visualize speeds, service rates, and headways. In conjunction with GTFS-Realtime, Park et al. 2020 examined delay and its propagation in the context of transit system performance. One challenge faced in analyzing GTFS data is that the specification itself is complex. A feed may include over 30 .txt files containing over 200 fields connected in complex ways. Some fields are required, some are optional, and others are conditionally required or forbidden (i.e., only included or excluded if another optional field is present). Feeds also include a variety of qualitatively different types of data—including times, stop coordinates, route shapes, identifier strings, integer counts, fare rules, fare amounts, and color hexadecimal. Thus, to analyze a system, the user must appreciate the relationships among these fields. Devunuri et al. 2024b shows that two popular Large Language Models (LLMs), GPT-3.5-Turbo and GPT-4, can ameliorate this challenge somewhat by answering questions about the GTFS specification with reasonable accuracy. A second challenge is that to obtain specific information about an agency (or agencies) from its GTFS feed, an analyst (even one who understands the GTFS specification well) must also figure out how to read and manipulate the data with a spreadsheet application or some computer code in order to obtain the answers sought. To this end, Devunuri et al. 2024b also briefly explores using LLMs to write code that carries out simple retrieval tasks on GTFS data—an application of what is known as program synthesis ( Haluptzok et al. 2023 ) . 1.2 Plan of paper This paper introduces an architecture that uses LLMs to answer questions about transit systems from GTFS data. This architecture is implemented in a chatbot interface called TransitGPT . The link to the chatbot as well as the code behind the TransitGPT is available on our GitHub repository located at https://github.com/UTEL-UIUC/TransitGPT . The reader is encouraged to experiment with TransitGPT by selecting one of the ten agencies we have prepared and asking questions about it. Figure 1 shows some examples of questions and answers generated by TransitGPT. TransitGPT can answer questions that a rider might have, such as: “When does the last Orange bus arrive at University and Victor on a Tuesday?” It can also answer system-level questions such as: “Which routes have headway shorter than 15 minutes on the weekend?” A goal of this paper is to explain the architecture behind TransitGPT and how it was built. In a nutshell, what we do is (i) ask the LLM to write Python code that can answer a question given that the GTFS feed is available; then (ii) run that code on a server where the feed is stored. While this two-step combination conveys the gist, TransitGPT involves many embellishments which are explained more thoroughly below. In particular, we have created a large system prompt : a text file containing instructions that guide the LLM as it writes code to answer the questions. This system prompt can be passed to any public-facing LLM, without ‘pre-training’ or ‘fine-tuning’. This emphasis on prompt engineering (rather than fine-tuning) aligns with the concept of “Prompting Is All You Need” ( Feng and Chen 2024 ) , which avoids the need for extensive training data and compute resources. To evaluate different prompt designs and to compare the performance of LLMs, we employ a 100-question benchmark. At the outset, it is important to distinguish our approach from another one which is perhaps more intuitive: passing an entire GTFS feed to an LLM and asking questions about the data. One reason we do not take this direction is that most GTFS feeds are too large to pass to the popular, public-facing LLMs currently due to context length limitations ( Hsieh et al. 2024 ) . Ostensibly, one could use Retrieval Augmented Generation (RAG) ( Lewis et al. 2021 ; Gao et al. 2024 ) to retrieve the feed in chunks, but an obstacle to this approach is the inter-related nature of GTFS files (i.e., there is no single file which contains all information about, say, a route). In either case, a more fundamental problem is that many queries require not only looking up data but also manipulating data, doing calculations with data, and connecting different pieces of data according to the complex rules of GTFS. While LLMs excel at tasks such as translation, summarization, and classification,