Advances in Temporal Point Processes: Bayesian, Neural, and LLM Approaches

Feng Zhou [email protected], Quyu Kong [email protected], Jie Qiao [email protected], Cheng Wan [email protected], Yixuan Zhang [email protected], Ruichu Cai [email protected] · 2025

Read Advances in Temporal Point Processes: Bayesian, Neural, and LLM Approaches by Feng Zhou [email protected], Quyu Kong [email protected], Jie Qiao [email protected], Cheng Wan [email protected], Yixuan Zhang [email protected], Ruichu Cai [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-14291.txt.

Introduction

Feng Zhou and co-authors wrote a 2025 survey of models for events that occur over time. Such models track things like clicks, trades, or quakes as points on a timeline. Classic models have served many fields for years, but they can be rigid when patterns get complex.

The survey sorts recent work into three lines: models based on prior belief updates, models built on neural networks, and models that use large language models. It covers basic terms, how each line is built, and how its values are fit. Classic uses are revisited, and the authors close with open problems and paths for new work.

Catalog record

CREATORFeng Zhou [email protected], Quyu Kong [email protected], Jie Qiao [email protected], Cheng Wan [email protected], Yixuan Zhang [email protected], Ruichu Cai [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE106 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-14291

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Advances in Temporal Point Processes: Bayesian, Neural, and LLM Approaches Feng Zhou [email protected], Quyu Kong [email protected], Jie Qiao [email protected], Cheng Wan [email protected], Yixuan Zhang [email protected], Ruichu Cai [email protected] Abstract Temporal point processes (TPPs) are stochastic process models used to characterize event sequences occurring in continuous time. Traditional statistical TPPs have a long-standing history, with numerous models proposed and successfully applied across diverse domains. In recent years, advances in deep learning have spurred the development of neural TPPs, enabling greater flexibility and expressiveness in capturing complex temporal dynamics. The emergence of large language models (LLMs) has further sparked excitement, offering new possibilities for modeling and analyzing event sequences by leveraging their rich contextual understanding. This survey presents a comprehensive review of recent research on TPPs from three perspectives: Bayesian, deep learning, and LLM approaches. We begin with a review of the fundamental concepts of TPPs, followed by an in-depth discussion of model design and parameter estimation techniques in these three frameworks. We also revisit classic application areas of TPPs to highlight their practical relevance. Finally, we outline challenges and promising directions for future research. Reviewed on OpenReview: https://openreview.net/forum?id=SXgGKkShhT 1 Introduction Many application scenarios generate time-stamped event sequences, which can be effectively modeled using temporal point processes (TPPs) ( Daley and Vere-Jones, 2007 ) . Examples include neural spike train data in neuroscience ( Linderman and Adams, 2015 ) , ask and bid orders in high-frequency financial trading ( Bacry and Muzy, 2014 ) , as well as tweets and retweets on social media ( Kong et al., 2023 ) . These event sequences, composed of asynchronous events, often influence one another and exhibit complex dynamics, making them more challenging to analyze compared to traditional synchronous time series problems. Investigating the underlying dynamic processes of such event sequences not only facilitates the prediction of future events but also helps uncover causal relationships. In the statistics community, TPPs have a long-standing history of research, with numerous statistical TPP models proposed over the years. Examples include the classic Poisson process ( Kingman, 1992 ) , Hawkes process ( Hawkes, 1971 ) , and self-correcting process ( Isham and Westcott, 1979 ) , among others. Each of these models is particularly well-suited to specific applications. For instance, the Poisson process was used to model telephone call arrivals, while the Hawkes process, due to its ability to capture self-exciting characteristics, has been widely applied to model earthquakes and aftershocks. Early TPP models are primarily parametric, requiring explicit specification of the parametric form of the model. However, this imposes limitations on their expressive power. To address these limitations, various nonparametric TPPs have been proposed within the statistics community, including approaches from both the frequentist and Bayesian frameworks, enabling more flexible modeling without the constraints of fixed parametric forms. In recent years, driven by rapid advancements in deep learning, the machine learning community has introduced approaches that combine neural network architectures with TPPs, referred to as neural TPPs. These models further enhance expressive power and are often more intuitive, simpler, and easier to train compared to statistical nonparametric TPPs. Over the past several years, the emergence of large language models (LLMs) has brought transformative changes to the field of artificial intelligence. With their rich contextual understanding and ability to process multimodal data, LLMs offer new possibilities for modeling event sequences. This paper reviews recent advances in TPPs based on Bayesian methods, deep learning, and LLMs, with a focus on model design and parameter estimation. Due to space limitations, we do not aim to cover every method in detail but instead emphasize fundamental principles and core ideas. We also revisit classic applications of TPPs and discuss key challenges and future research directions in the field. This taxonomy of recent advances in TPPs is illustrated in Figure 1 . This survey is expected to give comprehensive background knowledge, research trends and technical insights for TPPs. Comparison with Existing TPP Surveys Several surveys on TPPs in machine learning already exist, such as Yan (2019) and Shchur et al. (2021) . The former summarizes advances in statistical and neural TPPs, while the latter provides a more detailed overview of neural TPPs. Additionally, surveys in other domains, such as finance, include Hawkes (2018) . Compared with these works, this paper provides a comprehensive update. In the domain of statistical TPPs, we emphasize recent progress in Bayesian nonparametric TPPs, which has been largely overlooked, as most prior reviews (e.g., Yan (2019) ) primarily focus on frequentist approaches. For neural TPPs, Shchur et al. (2021) covers works up to 2020, whereas this survey reviews advances from 2020 to 2025. Furthermore, we include a systematic review of the emerging area of LLM-based TPPs, which has gained significant attention in recent years but has not yet been comprehensively surveyed. A detailed comparison with existing surveys is provided in Table 1 . Survey Methodology. To construct this survey, we conducted a structured literature review covering major venues in machine learning, statistics, and related application domains. Specifically, we collected papers from top conferences and journals such as NeurIPS, ICML, ICLR, AISTATS, KDD, AAAI, JMLR, and IEEE/ACM Transactions, as well as relevant statistical journals. Our primary focus is on works published between 2020 and 2025, while also including earlier foundational studies for completeness. We include papers based on the following criteria: (i) the work proposes a novel TPP model or inference method, (ii) it introduces new training or estimation techniques applicable to TPPs, or (iii) it demonstrates significant applications or extensions (e.g., multimodal or LLM-based TPPs). The collected literature is then organized into three main categories—Bayesian, neural, and LLM-based approaches—based on modeling paradigms. Due to space constraints, this survey does not aim to be exhaustive, but instead focuses on representative and influential works that highlight key methodological developments and research trends. Table 1: Survey comparison. Survey Years Covered Frequentist TPP Bayesian TPP Neural TPP LLM-based TPP Training Methods Applications Yan (2019) \leq 2019 ✔ ✗ ✔ ✗ Limited ✔ Shchur et al. (2021) \leq 2021 ✔ ✗ ✔ ✗ Limited ✔ Hawkes (2018) \leq 2018 ✔ ✗ ✗ ✗ Limited Finance This survey \leq 2025 ✔ ✔ ✔ ✔ ✔ ✔ {forest} Figure 1: The taxonomy of Bayesian, neural, and LLM-based TPPs. 2 Background of TPPs We first briefly review some of the core concepts of TPPs. For readers unfamiliar with TPPs, we recommend Rasmussen (2018) for a comprehensive introduction. 2.1 Unmarked TPP A TPP is a stochastic process that models the occurrence of events over a time window [0,T] . A trajectory from a TPP can be represented as an ordered sequence \mathcal{T}=(t_{1},\ldots,t_{N}) , N(t)=\max\{n:t_{n}\leq t,t\in[0,T]\} represents the associated counting process. Here, N denotes the random number of events within the interval [0,T] . TPPs can be defined using different parameterizations. One approach is to specify the distribution of the time intervals between consecutive events. We denote the f(t_{n+1}\mid\mathcal{H}_{t_{n}}) to be the conditional density function of t_{n+1} given the history of previous events t_{1},\ldots,t_{n} . In this work, \mathcal{H}_{t^{-}} denotes the history of events up to but excluding time t , while \mathcal{H}_{t} includes whether an event occurs at time t . The conditional density function sequentially specifies the distribution of all timestamps. Consequently, the joint distribution of all

Read the complete text