ST-VLM: Kinematic Instruction Tuning for Spatio-Temporal Reasoning in Vision-Language Models
Dohwan Ko Sihyeon Kim Yumin Suh Vijay Kumar B.G, Minseo Yoon Manmohan Chandraker Hyunwoo J. Kim · 2025
Read ST-VLM: Kinematic Instruction Tuning for Spatio-Temporal Reasoning in Vision-Language Models by Dohwan Ko Sihyeon Kim Yumin Suh Vijay Kumar B.G, Minseo Yoon Manmohan Chandraker Hyunwoo J. Kim (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-19355.txt.
Description
Programs that answer questions about video often stumble when asked how fast something moved or how far it went. Dohwan Ko and six co-authors tackle that gap in a 2025 paper. They built a training set from real videos with three-dimensional notes on motion, covering distance traveled, speed, direction, and how objects move relative to each other. A second pipeline labels ordinary videos automatically using rebuilt 3D scenes.
Trained on this data, their model beat rivals on their new test set and on older video benchmarks too. The authors show the skill also supports harder, multi-step questions about moving scenes.
Catalog record
| CREATOR | Dohwan Ko Sihyeon Kim Yumin Suh Vijay Kumar B.G, Minseo Yoon Manmohan Chandraker Hyunwoo J. Kim |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-19355