TimeHF: Billion-Scale Time Series Models Guided by H uman F eedback

Yongzhi Qi, Hao Hu, Dazhou Lei, Jianshen Zhang, Zhengxin Shi, Yulin Huang, Zhengyu Chen, Xiaoming Lin, Zuo-Jun Max Shen · 2025

Read TimeHF: Billion-Scale Time Series Models Guided by H uman F eedback by Yongzhi Qi, Hao Hu, Dazhou Lei, Jianshen Zhang, Zhengxin Shi, Yulin Huang, Zhengyu Chen, Xiaoming Lin, Zuo-Jun Max Shen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-15942.txt.

Introduction

Yongzhi Qi and co-authors build TimeHF, a billion-scale forecasting model tuned with human feedback, in a research paper from 2025. Time series networks forecast sales, demand, and other streams well, but they scale poorly and can falter on fresh data. The team took cues from large language models.

TimeHF holds six billion parameters and reads long series through a patch-based embedding. A new training scheme, time-series policy optimization, folds human feedback into learning. The model now runs inside the supply chain of the retailer JD dot com, planning restocks for over 20,000 products. Its forecasts beat past methods by 33.21 percent in accuracy. The authors present the system as a step toward large, general time series models.

Catalog record

CREATORYongzhi Qi, Hao Hu, Dazhou Lei, Jianshen Zhang, Zhengxin Shi, Yulin Huang, Zhengyu Chen, Xiaoming Lin, Zuo-Jun Max Shen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-15942

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TimeHF: Billion-Scale Time Series Models Guided by H uman F eedback Yongzhi Qi, Hao Hu, Dazhou Lei, Jianshen Zhang, Zhengxin Shi, Yulin Huang, Zhengyu Chen, Xiaoming Lin, Zuo-Jun Max Shen Abstract Time series neural networks perform exceptionally well in real-world applications but encounter challenges such as limited scalability, poor generalization, and suboptimal zero-shot performance. Inspired by large language models, there’s interest in developing large time series models (LTM) to address these issues. However, current methods struggle with training complexity, adapting human feedback, and achieving high predictive accuracy. We introduce TimeHF, a novel pipeline for creating LTMs with 6 billion parameters, incorporating human feedback. We use patch convolutional embedding to capture long time series information and design a human feedback mechanism called time-series policy optimization. Deployed in JD.com’s supply chain, TimeHF handles automated replenishment for over 20,000 products, improving prediction accuracy by 33.21% over existing methods. This work advances LTM technology and shows significant industrial benefits. 1 Introduction Benefiting from the success of large models in fields such as language processing ( Dubey et al. 2024 ) and computer vision ( Kirillov et al. 2023 ) , large-scale time series models (LTM) have rapidly developed in recent years. Figure 1: Left: Traditional time series foundation models, while performing well in common scenarios, may be overly sensitive to noises in training datasets, resulting in hallucinations in complex scenarios. Right: With RLHF, feedback contrast pairs—constructed by interpretable small models created by human experts—guide the model to gradually shift toward more accurate predictions. Roughly, two typical strategies can be adopted to construct LTM: one involves designing appropriate prompts and tokenizers to discretize and align the time-series modality with the textual modality, followed by leveraging pre-trained large language models (LLMs) for inference ( Zhou et al. 2023 ) ; the other is based on the transformer architecture, formulating a pure LTM from scratch ( Woo et al. 2024 ) . Scaling laws state that increasing the size of models and training datasets typically leads to performance improvements, which have been widely validated in language and vision domains ( Alabdulmohsin et al. 2022 ) . However, existing publicly available time series datasets generally suffer from limited data volume and strong regularity, small models can already effectively capture these regular patterns, making it difficult for scaling laws to take effect. Currently, the largest pure time series model contains only 710 million parameters ( Ansari et al. 2024 ) , significantly smaller than the large language models which reach up to 405 billion parameters ( Dubey et al. 2024 ) . Although LTM may perform well in some common scenarios, time series can exhibit various patterns or distributions across different fields. In a specific domain such as forecasting sales of a long-tail product, prediction models must be adjusted or tuned to mitigate hallucinations ( Xu et al. 2024 ) , preventing inaccurate predictions or the emergence of outliers. Popular tuning methods in the LTM domain include Supervised Fine-Tuning (SFT) ( Liu et al. 2024 ) and Retrieval-Augmented Generation (RAG) ( Yang et al. 2024 ) . To align with human preferences, Reinforcement Learning from Human Feedback (RLHF) ( Knox and Stone 2011 ) directly optimizes model behavior using human feedback. RLHF has demonstrated superior performance in complex tasks and dynamic environments and has been widely applied in fields such as text generation ( Ouyang et al. 2022 ) . In this paper, we propose a novel framework for building LTMs to address the aforementioned challenges (see Figure 1 for an overview). Leveraging JD.com’s extensive sales data, we construct a high-quality time series dataset that spans a variety of scenarios, including standard products, new products, seasonal items, bestsellers, long-tail items, and intermittent items. This diverse dataset significantly enhances the richness and complexity of training data, thereby enabling the development of the LTMs of greater scale. We train PCLTMs with 300M, 1.6B, and 6B parameters, zero-shot experiments demonstrate their superior performance. To further effectively incorporate human expertise into LTMs, we propose the first RLHF approach for time series prediction models, termed Timeseries policy optimization (TPO). It utilizes specialized predictive models (small models) built by human experts as proxies for expert knowledge and cognition to generate prediction pairs reflecting diverse human preferences in complex time series scenarios. Compared to state-of-the-art models (GPT4TS), our framework achieves an average reduction in Mean Squared Error (MSE) of 7.09%. Our contributions are summarized as follows: • We propose a standard for constructing large-scale, high-quality time series datasets, including data augmentation such as high-dimensional aggregated and interpretable component prediction data, data balancing, diversity ranking, etc. Based on this standard, we construct a large-scale time series dataset of 210B data points with intricate and diverse temporal patterns. • To address the intricate cross-dependencies in complex, long time series, we introduce a novel patch-based convolutional large time series model (TimeHF). Based on this approach, We propose the first billion-scale pure time series model and successfully achieve industrial deployment in JD.com. • We propose Timeseries policy optimization (TPO), the first RLHF framework applicable to time series forecasting, which enables LTMs to learn the tacit knowledge of experienced time-series experts. Numerical studies show that our framework can improve the zero-shot capabilities of LTMs significantly. Table 1: Comparison of Our Method with Existing LTMs Category Method Training Token Trainable Parameter Zero-Shot Forecast SFT RLHF LLM-based TimeLLM - 6M \surd \surd GPT4TS - 4M \surd \surd Pure LTM Chronos 84B 710M \surd \surd Moirai 28B 311M \surd - Timer 28B 67M \surd \surd MOMENT 1B 385M \surd \surd TimesFM 100B 200M \surd \surd TimeHF 210B 6B \surd \surd \surd 2 Related Work 2.1 Pretraining Dataset in LTMs Table 1 provides a summary of the key differences between recent LTMs, they can be broadly categorized into two types. The first type relies on foundational models from the language or image domains, assuming inherent similarities in time series data ( Gruver et al. 2024 ; Zhou et al. 2023 ; Jin et al. 2023 ; Rasul et al. 2023 ; Ansari et al. 2024 ; Chen et al. 2024 ) , requiring additional data transformation modules. The second type, including TimesFM ( Das et al. 2023 ) , MoiRAI ( Woo et al. 2024 ) , and Timer ( Liu et al. 2024 ) . These models can extract deeper insights from time series data, but they all treat patches as independent segments, limiting the model’s ability to capture inter-patch information. Pre-training data are crucial for LTMs, but currently available public time series datasets are insufficient in richness and scale, and existing construction methods are simplistic, lacking emphasis on data quality. These issues may lead to insufficient diversity and balance in the data. Common datasets for LTMs include TSLib ( Wu et al. 2022 ) , the Monash Database ( Godahewa et al. 2021 ) , the GluonTs library ( Alexandrov et al. 2020 ) , and the LOTSA dataset ( Woo et al. 2024 ) . However, these publicly available datasets are far smaller than those used for LLMs. For example, LOTSA integrates various accessible datasets but contains only 27B observations, while TimesFM uses a mixed dataset of 100B observations. While TimesFM’s synthetic data reflects common patterns, it sacrifices some authenticity. Our pretraining

Read the complete text