Extremal Contours: Gradient-driven contours for compact visual attribution
Reza Karimzadeh, Albert Alonso, Frans Zdyb, Julius B. Kirkegaard, Bulat Ibragimov · 2025
Read Extremal Contours: Gradient-driven contours for compact visual attribution by Reza Karimzadeh, Albert Alonso, Frans Zdyb, Julius B. Kirkegaard, Bulat Ibragimov (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-01411.txt.
Introduction
Reza Karimzadeh and colleagues explain vision models with contours instead of masks. The usual explanation tools shade dense pixel masks showing what a classifier used, but those masks come out fragmented and overfitted, needing cleanup before a person can read them.
Their method, Extremal Contours, needs no training. A star convex region, described compactly by a truncated Fourier series, is optimized against an extremal preserve and delete objective using the classifier's own gradients. The result is guaranteed to be one simply connected mask with orders of magnitude fewer free parameters and stable updates. On ImageNet classifiers it matched the fidelity of dense masks with better run to run consistency, and explicit area control yields importance contour maps. A multi contour extension localizes several objects at once.
Catalog record
| CREATOR | Reza Karimzadeh, Albert Alonso, Frans Zdyb, Julius B. Kirkegaard, Bulat Ibragimov |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-01411
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Extremal Contours: Gradient-driven contours for compact visual attribution Reza Karimzadeh, Albert Alonso, Frans Zdyb, Julius B. Kirkegaard, Bulat Ibragimov Abstract Faithful yet compact explanations for vision models remain a challenge, as commonly used dense perturbation masks are often fragmented and overfitted, needing careful post-processing. Here, we present a training-free explanation method that replaces dense masks with smooth tunable contours. A star-convex region is parameterized by a truncated Fourier series and optimized under an extremal preserve/delete objective using the classifier gradients. The approach guarantees a single, simply connected mask, cuts the number of free parameters by orders of magnitude, and yields stable boundary updates without cleanup. Restricting solutions to low-dimensional, smooth contours makes the method robust to adversarial masking artifacts. On ImageNet classifiers, it matches the extremal fidelity of dense masks while producing compact, interpretable regions with improved run-to-run consistency. Explicit area control also enables importance contour maps, yielding a transparent fidelity-area profiles. Finally, we extend the approach to multi-contour and show how it can localize multiple objects within the same framework. Across benchmarks, the method achieves higher relevance mass and lower complexity than gradient and perturbation based baselines, with especially strong gains on self-supervised DINO models where it improves relevance mass by over 15% and maintains positive faithfulness correlations. 1 Introduction In the last decade, deep neural networks have consistently represented the state-of-the-art in the computer vision field, achieving strong performance in classification, detection, and segmentation tasks [ 1 , 2 , 3 , 4 ] . As these models are increasingly deployed in sensitive domains such as medical imaging [ 5 , 6 ] and autonomous driving [ 7 ] , among others, interpretability is needed to establish trust, diagnose errors, and ensure reliability [ 8 , 9 ] . A family of explanation techniques are saliency maps, which attribute importance scores to individual input pixels or regions [ 10 , 11 , 12 ] . Gradient-based methods, such as saliency backpropagation and integrated gradients [ 13 ] , visualize local sensitivities of the prediction with respect to the input. While computationally efficient, these approaches often highlight many, diffuse regions, are sensitive to noise, and may fail sanity checks that test for faithfulness [ 14 ] . An alternative line of work uses perturbation-based explanations, which measure how predictions change when parts of the input are masked or altered. By optimizing perturbations, methods such as Meaningful Perturbations [ 15 ] and Extremal Perturbations [ 16 ] identify regions that are most responsible for the output of a model. Perturbation approaches are more closely tied to causal influence [ 17 ] , but typically rely on dense masks obtained through the gradients [ 16 ] or by training an auxiliary network [ 9 ] . A natural requirement for explanations is the ability to highlight compact, coherent regions that are sufficient to preserve or suppress a model prediction [ 18 ] . Such regions are easier to interpret, compare across inputs, and analyze quantitatively, but because of their lack of topological guarantees, masking methods are susceptible to noisy, fragmented, or multi-component outputs and require strong regularization [ 16 , 19 , 20 ] . Recent work has attempted to impose continuity and structural constraints on explanation masks, for example by learning implicit neural representations that generate smooth, area-conditioned masks [ 21 ] . While such formulations improve continuity, they still lack explicit geometric control and topological guarantees. This work addresses these limitations by proposing a structured representation for perturbation masks based on gradient-driven contours. Instead of optimizing every pixel of the mask, we parameterize a closed star-convex region using a truncated Fourier series that defines the radial extent of the mask relative to a learnable center. Such geometry-aware parameterizations echo ideas from computational geometry, where analytic formulations like surface-patch Voronoi diagrams ensure smooth and topologically consistent boundaries [ 22 ] . This compact representation guarantees smooth, simply connected masks by construction and can be optimized end-to-end through any differentiable criterion, similar to how differentiable rendering is used to refine detections [ 23 ] . Compared to existing extremal methods [ 16 ] , our approach reduces the dimensionality of learnable parameters by one to two orders of magnitude and converges reliably without dataset-level optimization. The result is a concise, topology-preserving explanation that retains the faithfulness of perturbation-based approaches while avoiding the instability and complexity of learnable pixel masks. Figure 1: Comparison of explanation methods on ImageNet validation images for the DINO [ 24 ] model. While gradient-based maps (e.g., Gradient SHAP [ 25 ] , Integrated Gradients [ 13 ] , Grad-CAM++ [ 12 , 26 ] ) and dense perturbation masks (Smooth Mask [ 16 ] ) typically produce diffuse or sometimes fragmented attributions, our parameterization yields a single smooth, simply connected contour (Extremal Contour) that encloses the object of interest, highlighting a different representational paradigm for explainability. 2 Method We represent explanations as smooth star-convex masks optimized under a perturbation objective. Each region is parameterized relative to a learnable center location c\in\mathbb{R}^{2} by a truncated Fourier expansion, \hat{r}(\theta)=r_{0}+\Re_{+}\!\!\left(\sum_{k=1}^{K}w_{k}e^{ik\theta}\right), (1) with complex coefficients w_{k}\in\mathbb{C} , yielding closed, smooth contours from only 2K{+}3 free parameters. The operator \Re_{+} takes the real part normalized to the positive range. For a pixel p=(x,y) in polar coordinates relative to c , with angle \theta_{p} and radius \rho_{p} , we define its mask value as m(p)=\frac{1}{1+\exp\!\big(\tau\cdot[\hat{r}(\theta_{p})-\rho_{p}]\big)}, (2) where \tau controls boundary sharpness. Perturbations follow the extremal principle [ 15 , 16 ] . A blurred background \tilde{x} is produced by Gaussian smoothing of the input x , and the mask defines preserved and deleted variants, \displaystyle x_{\mathrm{p}} \displaystyle=m\odot x+(1-m)\odot\tilde{x}, (3) \displaystyle x_{\mathrm{d}} \displaystyle=(1-m)\odot x+m\odot\tilde{x}. (4) The loss we minimize is the sum of three term: (1) the extremal loss, (2) a term that prefers smaller areas, and (3) a shape regularizer, \mathcal{L}=\mathcal{L}_{\mathrm{extremal}}+\lambda_{a}\alpha_{r}+\lambda_{r}\mathcal{L}_{\mathrm{spec}}. (5) Similar to Møller et al. 2024 , we construct the loss to encourage preserved regions to retain the model feature embedding, while deletion suppresses it: \mathcal{L}_{\mathrm{extremal}}=-\cos(e_{p},e_{o})+\cos(e_{d},e_{o}). (6) Here, e_{o},e_{p},e_{d} are the embeddings of the classifier backbone of the original image, the preserved and the deleted variants. Since Eq. ( 6 ) benefits from large masked areas, we regularize the solution contour by its area fraction given by \alpha_{r}=\frac{1}{2\cdot S}\int_{0}^{2\pi}\hat{r}(\theta)^{2}\,d\theta, (7) normalized to the [-1,1]^{2} image domain ( S{=}4 ). The hyperparameter \lambda_{a} controls how much a decrease is loss is valued compared to an increase in area as \partial_{a}\mathcal{L}_{\mathrm{extremal}}\approx-\lambda_{a} at the minimum. This is a user preference. In our experiments, we found that tuning it dynamically by \lambda_{a}=\min(5,\frac{1}{1-\cos(e_{o},e_{p})}) (8) gives good results across image types. This schedule increases when the preserved embedding diverges from the original, ensuring that compactness is only enforced once fidelity is maintained. At \cos(e_{o},e_{p})=0 , the method is forced to identify a loss for which \partial_{a}\mathcal{L}_{\mathrm{extremal}}\approx-1 , which is guaranteed possible by the mean value theorem. We do not propagate gradients through Eq. ( 8 ), which only serves to balance the contribution of the two loss terms. Finally, since we typically prefer rounded, smoother shapes, high-frequency oscillations are discouraged by penalizing Fourier energy, \mathcal{L}_{\mathrm{spec}}=\sum_{k=1}^{K}k^{2}|w_{k}|^{2}, (9) which suppresses unstable boundaries. Optimization uses adaptive gradient steps [ 27 ] and \tau annealing for improved convergence (see Appendix A