Quantifying the Limits of Segmentation Foundation Models: Modeling Challenges in Segmenting Tree-Like and Low-Contrast Objects

Yixin Zhang, Nicholas Konz, Kevin Kramer, Maciej A. Mazurowski · 2024

Read Quantifying the Limits of Segmentation Foundation Models: Modeling Challenges in Segmenting Tree-Like and Low-Contrast Objects by Yixin Zhang, Nicholas Konz, Kevin Kramer, Maciej A. Mazurowski (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-04243.txt.

Description

Yixin Zhang and three co-authors measure where image-cutting AI fails in a 2024 paper. Foundation models that outline any object in a picture are impressive, but they stumble on certain shapes: dense, branching forms like blood vessels, and objects that barely differ from their background.

The team defines two plain scores, how tree-like an object is and how separable its texture is from the scene. On rigged synthetic tests and real data sets, the scores predict failure across several leading models. The authors trace the flaw to a mix-up: the models read fine local structure as background texture. Extra training on such cases did not fix it, which points to a built-in limit. Their framework gives researchers the first numbers for this weakness, useful for deciding where these models can be trusted and where humans must check.

Catalog record

CREATORYixin Zhang, Nicholas Konz, Kevin Kramer, Maciej A. Mazurowski
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE71 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-04243

Browse the full library · About the Tower of Babel