Advancing Semantic Future Prediction through Multimodal Visual Sequence Transformers

Efstathios Karypidis, Ioannis Kakogeorgiou, Spyros Gidaris, Nikos Komodakis · 2025

Read Advancing Semantic Future Prediction through Multimodal Visual Sequence Transformers by Efstathios Karypidis, Ioannis Kakogeorgiou, Spyros Gidaris, Nikos Komodakis (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-08303.txt.

Description

Efstathios Karypidis and co-authors published a 2025 paper on predicting how a street scene will look a few seconds ahead. Self-driving systems need such forecasts to move safely through changing traffic. Their method, FUTURIST, labels each part of the coming scene by what it is: road, car, person, and so on. It works from several kinds of input at once, such as camera images and depth.

The system uses a transformer, the design behind most modern AI, arranged to handle video-like sequences. A masking trick during training teaches it to fill in hidden parts from all input types together. A new way of chopping scenes into tokens keeps the computing cost down. It also allows training from start to finish on sharp, high-resolution input. On the standard Cityscapes street dataset, the method set new best scores for both near and slightly longer forecasts.

Catalog record

CREATOREfstathios Karypidis, Ioannis Kakogeorgiou, Spyros Gidaris, Nikos Komodakis
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-08303

Browse the full library · About the Tower of Babel