Cross-View Completion Models are Zero-shot Correspondence Estimators

Honggyu An, Jinhyeon Kim, Seonghoon Park, Jaewoo Jung, Jisang Han, Sunghwan Hong, Seungryong Kim · 2024

Read Cross-View Completion Models are Zero-shot Correspondence Estimators by Honggyu An, Jinhyeon Kim, Seonghoon Park, Jaewoo Jung, Jisang Han, Sunghwan Hong, Seungryong Kim (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-09072.txt.

Description

Honggyu An and co-authors show that models trained to complete one view of a scene from another can double as match finders between views, with no extra training. Their 2024 paper draws a line from cross view completion to self supervised correspondence learning. The key object is the cross attention map inside the completion model.

That map, they find, captures which points match across views better than links drawn from encoder or decoder features. They test the claim three ways: zero shot matching, learned geometric matching, and depth estimation over many frames. Each test backs the map as a strong correspondence signal. A project page hosts the details.

Catalog record

CREATORHonggyu An, Jinhyeon Kim, Seonghoon Park, Jaewoo Jung, Jisang Han, Sunghwan Hong, Seungryong Kim
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE88 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-09072

Browse the full library · About the Tower of Babel