TempViz : On the Evaluation of Temporal Knowledge in Text-to-Image Models

Carolin Holtermann, Nina Krebs, Anne Lauscher · 2026

Read TempViz : On the Evaluation of Temporal Knowledge in Text-to-Image Models by Carolin Holtermann, Nina Krebs, Anne Lauscher (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-14951.txt.

Introduction

Carolin Holtermann, Nina Krebs, and Anne Lauscher test whether image generators grasp time. Their 2026 arXiv paper deals with text-to-image models. Time changes how things look, from objects and places to animals.

A model that ignores time can draw the wrong scene. A landscape in spring should not match the same landscape in winter. The authors present TempViz, a way to gauge temporal knowledge in these models. The study asks how well these models use time cues, such as seasons or ages, when they build an image from a written prompt.

Catalog record

CREATORCarolin Holtermann, Nina Krebs, Anne Lauscher
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-14951

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TempViz : On the Evaluation of Temporal Knowledge in Text-to-Image Models Carolin Holtermann, Nina Krebs, Anne Lauscher Abstract Time alters the visual appearance of entities in our world, like objects, places, and animals. Thus, for accurately generating contextually-relevant images, knowledge and reasoning about time can be crucial (e.g., for generating a landscape in spring vs. in winter). Yet, although substantial work exists on understanding and improving temporal knowledge in natural language processing, research on how temporal phenomena appear and are handled in text-to-image (T2I) models remains scarce. We address this gap with TempViz , the first data set to holistically evaluate temporal knowledge in image generation, consisting of 7.9k prompts and more than 600 reference images. Using TempViz , we study the capabilities of five T2I models across five temporal knowledge categories. Human evaluation shows that temporal competence is generally weak, with no model exceeding 75% accuracy across categories. Towards larger-scale studies, we also examine automated evaluation methods, comparing several established approaches against human judgments. However, none of these approaches provides a reliable assessment of temporal cues – further indicating the pressing need for future research on temporal knowledge in T2I. We publish all data and code at https://github.com/TAI-HAMBURG/TempViz . 1 Introduction “Time is the longest distance between two places.” – Tennessee Williams, 1944 An artwork from 1900 looks different than an artwork from 1960 , and a dog at age 3 months looks different than a dog at age 10 years . Similarly, landscapes change their appearance across seasons, such as winter or summer – Time influences the visual appearance of the world around us. Figure 1: Distribution of the temporal categories within TempViz with example images of different T2I models. The effects of time and their perception are intrinsically intertwined with our senses ( 18 , e.g.,) , and play a major role in most human societies. For accurately representing the world, an understanding of the temporal effects (e.g., aging) and corresponding visual effects is thus crucial. Users might specify in an input prompt to image generation (e.g., to a Stable Diffusion model 28 ) the particular epoch or year they are looking for, expecting the generated output to be aligned with their instruction. In natural language processing (NLP), many studies have analyzed temporal knowledge capabilities of NLP systems, for instance, relating to the temporal reasoning capabilities of large language models (e.g., 8 ; 6 , inter alia ) . However, despite the large body of existing works on temporal knowledge in text-only models, and the central role of temporal knowledge and reasoning for accurate visual representations, the landscape on resources and evaluation of temporal knowledge in text-to-image (T2I) models remains largely unexplored. In this work, we address this research gap by posing the following research question: How well do existing T2I models perform at following temporal instructions across different subject categories? To answer this question, we introduce a novel resource for analyzing temporal knowledge in T2I models and conduct both manual and automatic evaluations. We find that even the strongest T2I model among those evaluated correctly applies the correct temporal visual cues in at most 75% of cases for only one category, and that for some categories the temporal instruction is barely followed (for example, as low as 15% for historical maps). Strikingly, none of the automated evaluators we tested was able to reliably judge the correct application of temporal knowledge above a macro F1 of 72%. Together, these results motivate more targeted research on temporal knowledge in image generation and evaluation. Contributions Our contributions are threefold: ( 1 ) We present TempViz , the first dataset specifically designed for analyzing temporal knowledge in T2I models. It comprises 7,940 prompts across five diverse categories (e.g., animals , artworks ), each paired with category-appropriate temporal cues (e.g., lifespans , art periods , seasons ), and includes reference images that exemplify correct generations for a subset of categories. ( 2 ) Using TempViz , we conduct a human evaluation of five recent text-to-image models, measuring image quality, subject fidelity, and adherence to the specified temporal cue. ( 3 ) We complement the manual study with an extensive assessment of automatic evaluation methods, analyzing their suitability for judging whether temporal knowledge has been applied correctly in generated images. 2 Related Work We present an overview of datasets for evaluating temporal knowledge in NLP, as well as datasets designed to assess T2I models. Temporal Knowledge Datasets In large language models, temporal knowledge has been examined from multiple perspectives. Early work on temporal relation extraction introduced annotated corpora such as TimeBank 24 . Subsequent benchmarks probe time-varying factual knowledge and temporal reasoning, including the TempLAMA benchmark 5 and its derived datasets such as LAMA-TK 37 and TempReason 30 . Other work in the direction of LLM probing focuses rather on time-sensitive question answering 3 ; 10 . Another line of work in the direction of temporal NLI focuses on inferring knowledge about temporal relations and more complex temporal reasoning. This includes the ordering of temporal events 32 ; 33 and temporal grounding 17 ; 6 . In addition, several studies investigate methods for improving temporal reasoning through specialized training objectives 35 ; 31 . Text-to-Image Evaluation Datasets Compared to the amount of benchmarks available to evaluate the capabilities of LLMs, the landscape of resources to evaluate T2I models still remains scarce. Most benchmarks focus on the evaluation of the general text-image alignment ( 12 ; 36 , e.g.,) . 22 specifically focus on the question of how well these models generalize to novel word compositions. 15 introduce HEIM, a benchmark for the holistic evaluation of T2I models, which includes 12 evaluation aspects such as text-image alignment, image quality, aesthetics, originality, reasoning, knowledge, bias, toxicity, fairness, robustness, multilinguality, and efficiency; still neglecting temporally dependent depictions. Concurrent to our work 19 present WISE, a benchmark for assessing world knowledge reasoning in T2I models with 1,000 prompts spanning cultural commonsense, spatio-temporal reasoning, and natural science. While they also cover aspects of temporal knowledge, their focus is rather broad and the scale of their dataset is modest: Of the dataset, only 167 prompts concern temporal knowledge, targeting mostly narrow aspects like period-specific instruments instead of the wider visual manifestations of temporal change. As a result, WISE provides limited insights for analyzing and improving these fine-grained temporal effects in image generation. 3 TempViz We analyze how established T2I models interpret and represent temporal cues during image generation. To this end, we introduce TempViz , a carefully curated dataset of 7,940 instructions spanning five subject categories (see Figure 1 ), chosen to reflect common use cases where temporal effects are visually observable. Moreover, the prompts cover multiple dimensions of temporal variation at varying levels of difficulty, enabling a focused assessment of models’ temporal reasoning. We intentionally exclude prompts that generate human individuals due to ethical concerns: varying both temporal cues and subjects within this category would necessarily involve prompting different demographic backgrounds, introducing complex issues of stereotype reinforcement and bias that would shift the scope from temporal reasoning to fairness considerations. We therefore chose to omit humans and focus on categories where temporal variation can be analyzed with fewer confounds. To support reliable human annotation, we also collect

Read the complete text