Transformer-Based Temporal Information Extraction and Application: A Review

Xin Su, Phillip Howard, Steven Bethard · 2025

Read Transformer-Based Temporal Information Extraction and Application: A Review by Xin Su, Phillip Howard, Steven Bethard (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-07470.txt.

Introduction

Xin Su, Phillip Howard, and Steven Bethard survey how transformer models pull timelines out of plain text. Their 2025 review covers temporal information extraction, the task of finding dates, durations, and event order buried in unstructured writing. Such structure feeds reasoning in fields from healthcare to news and intelligence work.

Transformer language models have reshaped the field, yet no full review had gathered their use for this task. The authors sort the existing work into a clear map: what is extracted, how models are built, and where they are applied. They weigh strengths across approaches and mark open problems. The review closes with directions the authors see as most promising for the next round of research.

Catalog record

CREATORXin Su, Phillip Howard, Steven Bethard
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-07470

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Transformer-Based Temporal Information Extraction and Application: A Review Xin Su, Phillip Howard, Steven Bethard Abstract Temporal information extraction (IE) aims to extract structured temporal information from unstructured text, thereby uncovering the implicit timelines within. This technique is applied across domains such as healthcare, newswire, and intelligence analysis, aiding models in these areas to perform temporal reasoning and enabling human users to grasp the temporal structure of text. Transformer-based pre-trained language models have produced revolutionary advancements in natural language processing, demonstrating exceptional performance across a multitude of tasks. Despite the achievements garnered by Transformer-based approaches in temporal IE, there is a lack of comprehensive reviews on these endeavors. In this paper, we aim to bridge this gap by systematically summarizing and analyzing the body of work on temporal IE using Transformers while highlighting potential future research directions. 1 Introduction Temporal information extraction (IE) is a critical task in natural language processing (NLP). Its objective is to extract structured temporal information from unstructured text, thereby revealing the implicit timelines within the text. This not only helps improve temporal reasoning in other NLP tasks, such as timeline summarization and temporal question answering, but also helps human users in gaining a deeper understanding of the evolution of text content over time. For example, Figure 1 displays a snippet of George Washington’s Wikipedia page and the timeline of his position changes; relying solely on text-heavy documents to trace his position changes over different years is time-consuming and may lack accuracy as facts and temporal expressions are scattered throughout the text. In contrast, a timeline enables both NLP models and humans to understand the changes in these positions over time more succinctly and clearly. The application of this structured temporal information is not limited to Wikipedia but is also widely used in other domains such as healthcare Styler IV et al. (2014) . Figure 1: A snippet from George Washington’s Wikipedia page and a timeline regarding his positions. The advent of the Transformer architecture Vaswani et al. (2017) has sparked a revolutionary change in the field of NLP, particularly with the recent Transformer-based generative large language models (LLM), such as LLAMA3 Dubey et al. (2024) and GPT-4 Achiam et al. (2023) , demonstrating exceptional performance across many tasks. Nevertheless, there has yet to be an in-depth study that provides a comprehensive review or analysis of the Transformer architecture’s application in the field of temporal IE. Existing surveys Lim et al. (2019) ; Leeuwenberg and Moens (2019) ; Alfattni et al. (2020) ; Olex and McInnes (2021) focus on rule-based systems or traditional machine learning models (e.g., support vector machines) which are reliant on hand-crafted features. Only Olex and McInnes (2021) touches on the application of Transformer models, but they offer only a brief description of BERT-style models and focus largely on the clinical domain. To address this gap, we systematically review the applications of Transformer-based models in the field of temporal IE. Broadly, temporal IE refers to any tasks involving the extraction of temporal information from text. We focus on three important tasks which are defined in the most widely adopted temporal IE annotation framework, TimeML James (2003) : time expression identification, time expression normalization, and temporal relation extraction. Our contributions are summarized as follows: (1) We systematically review, summarize, and categorize the existing temporal IE datasets, Transformer-based methods, and applications. (2) We identify and highlight the research gaps in the field of temporal IE and suggest potential directions for future research. 2 Overview The goal of temporal IE is to extract structured temporal information from unstructured text, facilitating its interpretation and processing by computers, thereby achieving a transformation from text to structure. The final result of a temporal IE system is the construction of a directed acyclic graph, or a temporal graph, which represents the structured temporal information in the text. In the temporal graph, nodes represent time expressions and events (temporal entities), while edges depict the temporal relations between these nodes, such as “before,” “after,” etc. For instance, Figure 2 illustrates a text snippet from George Washington’s Wikipedia page and its corresponding temporal graph. Figure 2: A snippet from George Washington’s Wikipedia page and the corresponding temporal graph. Constructing a temporal graph involves several sub-tasks: time expression identification, time expression normalization, event extraction, and temporal relation extraction. The following is a brief introduction to these sub-tasks; see Appendix A for a discussion of common evaluation methods. Time Expression Identification and Normalization Time expression identification refers to identifying specific time points, durations, or periods within the text, such as the explicitly dateable expression “February 25, 2024,” or more ambiguous expressions like “three days ago” James (2003) . Time normalization involves converting identified expressions into a standardized format to improve their interpretability. For example, under the ISO-TimeML framework Pustejovsky et al. (2010) , “February 25, 2024” might be converted into the TIMEX3 format as “2024-02-25”. Event Trigger Extraction In temporal IE, event extraction differs from other NLP event extraction tasks; it simply marks the event trigger words that represent actions, such as “accident” in “about two weeks after the accident occurred”. We will not review event extraction works because, to our knowledge, there is currently no temporal IE research focused solely on event extraction. Furthermore, most existing work on temporal IE assumes that event triggers have already been identified. For a comprehensive survey of event extraction, we refer readers to Li et al. (2022) . Temporal Relation Extraction The task of temporal relation extraction aims to identify the temporal relations among given events and time expressions. Common temporal relations include before, after, and simultaneous. For example, in Figure 2 , the temporal relation between “June 14, 1775” and the event “become” is marked as “after”. 3 Datasets A clearly defined annotation framework is essential when constructing a dataset for temporal IE. It needs to precisely define time expressions, events, and their relations. We summarize all the datasets in Table 1 of Appendix B . 3.1 TimeML Annotation Framework Datasets An end-to-end temporal IE dataset encompasses various tasks, including the identification and normalization of time expressions and the extraction of temporal relations. Most end-to-end temporal information datasets have been based on the TimeML framework James (2003) or its derivatives, such as ISO-TimeML Pustejovsky et al. (2010) . We present datasets based on the TimeML framework in the first section of Table 1 . TimeBank James (2003) was the first dataset to adopt the TimeML framework, focusing on the English news domain. Follow-up works included the TempEval shared task series Verhagen et al. (2007) ; Verhagen et al. (2010) ; UzZaman et al. (2013) , covering multiple languages, including Chinese, English, Italian, French, Korean, and Spanish. There are also language-specific datasets like French TimeBank Bittar et al. (2011) , Spanish TimeBank Nieto et al. (2011) , Portuguese TimeBank Costa and Branco (2012) , Japanese TimeBank Asahara et al. (2013) , Italian TimeBank Bracchi et al. (2016) , and Korean TimeBank Lim et al. (2018) . Similarly, the MeanTime dataset Minard et al. (2016) offers data in English, Italian, Spanish, and Dutch. Datasets based on TimeML and its variants showcase language diversity and also cover several different domains: the Spanish TimeBank focuses on history text, the Korean TimeBank is

Read the complete text