PhysGame: Uncovering Physical Commonsense Violations in Gameplay Videos
Meng Cao, Haoran Tang, Haoze Zhao, Hangyu Guo, Jiaheng Liu, Ge Zhang, Ruyang Liu, Qiang Sun, Ian Reid, Xiaodan Liang · 2024
Read PhysGame: Uncovering Physical Commonsense Violations in Gameplay Videos by Meng Cao, Haoran Tang, Haoze Zhao, Hangyu Guo, Jiaheng Liu, Ge Zhang, Ruyang Liu, Qiang Sun, Ian Reid, Xiaodan Liang (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-01800.txt.
Description
Meng Cao and colleagues introduce PhysGame, a benchmark for testing whether video language models understand physical commonsense. Gameplay videos often contain glitches that defy everyday physics. Those glitches make the videos a natural test of whether a model notices when the visual world breaks its own rules.
PhysGame contains eight hundred eighty videos of glitches spanning mechanics, kinematics, optics, and material properties, covering twelve kinds of physical commonsense. Evaluating current video models, the team found that open-source systems lag well behind proprietary ones. To close the gap they built PhysInstruct, an instruction-tuning dataset with more than one hundred forty thousand question-answer pairs. They also built PhysDPO, a preference dataset designed to resist shortcuts such as relying on misleading titles or too few frames. The benchmark gives researchers a concrete way to measure and improve physical reasoning in video models.
Catalog record
| CREATOR | Meng Cao, Haoran Tang, Haoze Zhao, Hangyu Guo, Jiaheng Liu, Ge Zhang, Ruyang Liu, Qiang Sun, Ian Reid, Xiaodan Liang |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-01800