VideoRoPE: What Makes for Good Video Rotary Position Embedding?
Xilin Wei, Xiaoran liu, Yuhang Zang, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Jian Tong, Haodong Duan, Qipeng Guo, Jiaqi Wang, Xipeng Qiu, Dahua Lin · 2025
Read VideoRoPE: What Makes for Good Video Rotary Position Embedding? by Xilin Wei, Xiaoran liu, Yuhang Zang, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Jian Tong, Haodong Duan, Qipeng Guo, Jiaqi Wang, Xipeng Qiu, Dahua Lin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-05173.txt.
Description
A team of co-authors wrote a machine learning paper in two thousand twenty-five on how video models should encode position. Rotary position embedding is a popular way to tell a model where each token sits, prized for handling long inputs. Video adds time and space together, and stretching the one-dimensional trick to video is not obvious.
The authors first name four traits a good video version needs. They build a hard test in which moving distractors are planted in a visual search task, and older variants fail it. Their design, VideoRoPE, uses a three-dimensional layout with slow time frequencies and a diagonal arrangement to keep space and time relations intact.
Catalog record
| CREATOR | Xilin Wei, Xiaoran liu, Yuhang Zang, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Jian Tong, Haodong Duan, Qipeng Guo, Jiaqi Wang, Xipeng Qiu, Dahua Lin |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-05173