NSW-EPNews: A News-Augmented Benchmark for Electricity Price Forecasting with LLMs
Zhaoge Bi, Linghan Huang, Haolin Jin, Qingwen Zeng, Huaming Chen · 2025
Read NSW-EPNews: A News-Augmented Benchmark for Electricity Price Forecasting with LLMs by Zhaoge Bi, Linghan Huang, Haolin Jin, Qingwen Zeng, Huaming Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-11050.txt.
Introduction
Zhaoge Bi, Linghan Huang, Haolin Jin, Qingwen Zeng and Huaming Chen introduce NSW-EPNews in 2025. The benchmark tests how well models forecast electricity prices in New South Wales when news is added to past prices. Older methods used numbers alone and left out text signals from the same time.
The set holds more than 175,000 half-hourly spot prices from 2015 to 2024, plus weather data and market news summaries. News helped older models only a little. Large language models such as GPT-4o gained some ground but often made up false price series. The work shows a gap between these models and the needs of energy forecasting.
Catalog record
| CREATOR | Zhaoge Bi, Linghan Huang, Haolin Jin, Qingwen Zeng, Huaming Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 79 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-11050
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
NSW-EPNews: A News-Augmented Benchmark for Electricity Price Forecasting with LLMs Zhaoge Bi, Linghan Huang, Haolin Jin, Qingwen Zeng, Huaming Chen Abstract Electricity price forecasting is a critical component of modern energy-management systems, yet existing approaches heavily rely on numerical histories and ignore contemporaneous textual signals. We introduce NSW-EPNews, the first benchmark that jointly evaluates time-series models and large language models (LLMs) on real-world electricity-price prediction. The dataset includes over 175,000 half-hourly spot prices from New South Wales, Australia (2015–2024), daily temperature readings, and curated market-news summaries from WattClarity. We frame the task as 48-step-ahead forecasting, using multimodal input, including lagged prices, vectorized news and weather features for classical models, and prompt-engineered structured contexts for LLMs. Our datasets yields 3.6k multimodal prompt-output pairs for LLM evaluation using specific templates. Through compresive benchmark design, we identify that for traditional statistical and machine learning models, the benefits gain is marginal from news feature. For state-of-the-art LLMs, such as GPT-4o and Gemini 1.5 Pro, we observe modest performance increase while it also produce frequent hallucinations such as fabricated and malformed price sequences. NSW-EPNews provides a rigorous testbed for evaluating grounded numerical reasoning in multimodal settings, and highlights a critical gap between current LLM capabilities and the demands of high-stakes energy forecasting. 1 Introduction Time-series forecasting is central to modern infrastructure, and electricity-price prediction is its canonical, high-stakes application. Accurate and transparent price outlooks steer load balancing, dispatch planning, and wholesale trading across deregulated markets [ 1 ] . Classical econometric tools—ARIMA and its seasonal extension SARIMA—remain popular thanks to their interpretability, yet their linear assumptions limit fidelity to long-range, nonlinear structure in prices [ 2 , 3 ] . Conventional machine-learning regressors (LR, RF, SVR) capture richer covariate effects, but still falter when the signal is buried in complex temporal dependencies or interleaved with unstructured context such as news headlines [ 4 , 5 , 6 ] . In contrast, transformer-based large language models (LLMs) have recently demonstrated strong zero-/few-shot competence on time-series tasks by treating numeric sequences as another modality of “language” and conditioning on heterogeneous side information [ 7 ] . Their capacity to ingest both structured features and free-form text—e.g., market commentary and temperature forecasts—raises the prospect of materially improved electricity-price prediction. Motivation. Electricity prices are driven by a confluence of factors that extend far beyond historical values. Market commentary and policy news shape expectations and trading behaviour [ 8 , 9 , 10 ] . Weather also exerts a first-order effect: extreme temperatures trigger spikes in cooling or heating demand, propagating into price volatility [ 11 , 12 ] . Harnessing such exogenous signals therefore promises improved forecast accuracy; When information from multiple sources—such as weather data and news content—is integrated, the natural language processing capabilities of LLMs become particularly advantageous. Gap. In recent years, time-series forecasting evaluation has coalesced around public benchmarks—spanning traffic, energy load, macro-economic, and weather datasets—with error metrics like MSE and MAE (and variants) plus significance testing to compare models [ 13 , 14 ] . In electricity-price forecasting, open datasets covering multiple regional markets and year-long test sets against state-of-the-art baselines have become standard [ 14 ] . Yet these benchmarks overlook unstructured news data, despite evidence that external text can enhance numerical prediction [ 15 ] . With large language models (LLMs), zero-shot forecasting and multimodal fusion show great promise—for example, GPT4MTS’s integration of news events and numerical series yields significant accuracy gains [ 16 ] . To advance the field, a benchmark that fuses electricity-price time series with news text is both novel and necessary—but must be paired with rigorous prompt engineering and fine-tuning to mitigate LLM hallucination risks [ 17 ] . Despite the intuitive appeal of text-enriched forecasting, the community lacks a systematic benchmark for assessing how well LLMs integrate unstructured information into price predictions. Whether LLMs can reliably translate news sentiment and weather cues into numerical forecasts— and whether their outputs can be trusted in high-stakes markets —remains an open question [ 7 , 18 ] . Our contribution can be summarized as follows: • Dataset Construction: We introduce NSW-EPNews, a large-scale multimodal dataset and benchmark for electricity price forecasting that integrates historical electricity price time series, meteorological data (temperature records), and relevant news articles from New South Wales (NSW, Australia) covering 2015–2024. This dataset provides a comprehensive testbed for analyzing how rich multimodal inputs (weather and news) influence electricity price dynamics over nearly a decade of real-world data. Data Availability: We have fully released our benchmark dataset, experimental process, experimental results, which is now available at the following link: https://figshare.com/s/e25f3a98679d347f2a2e . • Prompt Template Design: We design a prompting methodology that enables both classical time-series forecasting models and large language models (LLMs) to ingest and make predictions from the combined multimodal data. In particular, we craft input templates that present the numerical (price and weather) data and textual news information in a unified format, allowing models of different types to effectively utilize the heterogeneous inputs and enabling direct comparison of traditional forecasting approaches against LLM-based methods on the same tasks. • Evaluation Framework: We develop an evaluation framework to rigorously assess model performance, quantifying both standard forecasting accuracy and hallucination-related failure modes. This framework computes conventional prediction error metrics for price forecasts while also detecting and measuring any hallucinated content in model outputs, thereby evaluating each model’s predictive accuracy alongside the factual consistency and reliability of its generated explanations or reports. • Key Findings: Extensive experiments show that (i) traditional statistical and machine-learning baselines gain only marginal benefit from naïvely vectorised news, and (ii) state-of-the-art LLMs incur large errors (MAE > 100 AUD/MWh on some splits) and frequent hallucinations. Current prompting and model architectures are therefore insufficient for robust, news-aware electricity-price forecasting, underscoring the need for improved prompt engineering, retrieval grounding and fine-tuning strategies. 2 Background Traditional Time Series Forecasting vs LLMs. Traditional time-series forecasting has long been dominated by statistical models and machine learning methods. Classical models like ARIMA rely on assumptions of linearity, stationarity, and carefully tuned seasonal heuristics [ 17 ] . These models perform well for small-scale data with stable patterns but struggle as data complexity grows. In recent years, deep learning approaches (e.g. LSTMs and Transformers) have become state-of-the-art on many forecasting benchmarks, demonstrating the ability to learn non-linear and long-term dependencies directly from data [ 19 ] . Yet even advanced deep models often require structured inputs and feature engineering, making it difficult to incorporate unstructured information (such as free-form text or images). They can still miss hidden complex patterns in large-scale, diverse sequences [ 19 ] , highlighting an opportunity for more flexible frameworks. Large Language Models (LLMs) have emerged as a promising new avenue for forecasting, inspired by their success in capturing complex patterns in natural language. LLM-based models leverage extensive pre-training on vast corpora to integrate broader contextual knowledge and act as general pattern learners beyond just language [ 15 ] . Initial studies have only begun exploring LLMs for time-series tasks; for example, TimeGPT and other LLM forecasters have demonstrated competitive accuracy in certain conditions, though their advantages vary with market volatility. Notably, the energy domain presents an open opportunity – until recently no LLM had been applied to