Quantifying the Limits of Segmentation Foundation Models: Modeling Challenges in Segmenting Tree-Like and Low-Contrast Objects
Yixin Zhang, Nicholas Konz, Kevin Kramer, Maciej A. Mazurowski · 2024
Read Quantifying the Limits of Segmentation Foundation Models: Modeling Challenges in Segmenting Tree-Like and Low-Contrast Objects by Yixin Zhang, Nicholas Konz, Kevin Kramer, Maciej A. Mazurowski (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-04243.txt.
Description
Yixin Zhang and three co-authors measure where image-cutting AI fails in a 2024 paper. Foundation models that outline any object in a picture are impressive, but they stumble on certain shapes: dense, branching forms like blood vessels, and objects that barely differ from their background.
The team defines two plain scores, how tree-like an object is and how separable its texture is from the scene. On rigged synthetic tests and real data sets, the scores predict failure across several leading models. The authors trace the flaw to a mix-up: the models read fine local structure as background texture. Extra training on such cases did not fix it, which points to a built-in limit. Their framework gives researchers the first numbers for this weakness, useful for deciding where these models can be trusted and where humans must check.
Catalog record
| CREATOR | Yixin Zhang, Nicholas Konz, Kevin Kramer, Maciej A. Mazurowski |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-04243