Cross-View Completion Models are Zero-shot Correspondence Estimators
Honggyu An, Jinhyeon Kim, Seonghoon Park, Jaewoo Jung, Jisang Han, Sunghwan Hong, Seungryong Kim · 2024
Read Cross-View Completion Models are Zero-shot Correspondence Estimators by Honggyu An, Jinhyeon Kim, Seonghoon Park, Jaewoo Jung, Jisang Han, Sunghwan Hong, Seungryong Kim (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-09072.txt.
Description
Honggyu An and co-authors show that models trained to complete one view of a scene from another can double as match finders between views, with no extra training. Their 2024 paper draws a line from cross view completion to self supervised correspondence learning. The key object is the cross attention map inside the completion model.
That map, they find, captures which points match across views better than links drawn from encoder or decoder features. They test the claim three ways: zero shot matching, learned geometric matching, and depth estimation over many frames. Each test backs the map as a strong correspondence signal. A project page hosts the details.
Catalog record
| CREATOR | Honggyu An, Jinhyeon Kim, Seonghoon Park, Jaewoo Jung, Jisang Han, Sunghwan Hong, Seungryong Kim |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 88 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-09072