Object-IR: Leveraging Object Consistency and Mesh Deformation for Self-Supervised Image Retargeting Journal: Pattern Recognition

Tianli Liao, Ran Wang, Siqing Zhang, Lei Li, Guangen Liu, Chenyang Zhao, Heling Cao, Peng Li · 2025

Read Object-IR: Leveraging Object Consistency and Mesh Deformation for Self-Supervised Image Retargeting Journal: Pattern Recognition by Tianli Liao, Ran Wang, Siqing Zhang, Lei Li, Guangen Liu, Chenyang Zhao, Heling Cao, Peng Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-27236.txt.

Introduction

Tianli Liao leads seven co-authors in Object-IR, a self-supervised system for resizing images without warping important objects. Retargeting changes aspect ratio for new screens, but stretching faces and key objects ruins results. Object-IR reformulates the task as learning-based mesh warping, where a network predicts how a grid over the image should deform.

Three losses guide learning: one keeps important objects looking consistent, one limits important meshes to simple scaling, and one enforces a clean rectangular border. Supervision comes from the input image own geometry and semantics, so no hand-labeled retargeting data are needed. On the RetargetMe benchmark it reaches state-of-the-art scores and visual quality while processing typical photos in milliseconds on consumer hardware.

Catalog record

CREATORTianli Liao, Ran Wang, Siqing Zhang, Lei Li, Guangen Liu, Chenyang Zhao, Heling Cao, Peng Li
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-27236

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Object-IR: Leveraging Object Consistency and Mesh Deformation for Self-Supervised Image Retargeting Journal: Pattern Recognition Tianli Liao, Ran Wang, Siqing Zhang, Lei Li, Guangen Liu, Chenyang Zhao, Heling Cao, Peng Li Abstract Eliminating geometric distortion in semantically important regions remains an intractable challenge in image retargeting. This paper presents Object-IR, a self-supervised architecture that reformulates image retargeting as a learning-based mesh warping optimization problem, where the mesh deformation is guided by object appearance consistency and geometric-preserving constraints. Given an input image and a target aspect ratio, we initialize a uniform rigid mesh at the output resolution and use a convolutional neural network to predict the motion of each mesh grid and obtain the deformed mesh. The retargeted result is generated by warping the input image according to the rigid mesh in the input image and the deformed mesh in the output resolution. To mitigate geometric distortion, we design a comprehensive objective function incorporating a) object-consistent loss to ensure that the important semantic objects retain their appearance, b) geometric-preserving loss to constrain simple scale transform of the important meshes, and c) boundary loss to enforce a clean rectangular output. Notably, our self-supervised paradigm eliminates the need for manually annotated retargeting datasets by deriving supervision directly from the input’s geometric and semantic properties. Extensive evaluations on the RetargetMe benchmark demonstrate that our Object-IR achieves state-of-the-art performance, outperforming existing methods in quantitative metrics and subjective visual quality assessments. The framework efficiently processes arbitrary input resolutions (average inference time: 0.009s for 1024×683 resolution) while maintaining real-time performance on consumer-grade GPUs. The source code will soon be available at https://github.com/tlliao/Object-IR . 1 Introduction Figure 1: Image retargeting comparisons for 0.5 \times width resizing. Image retargeting is an advanced technique in computer vision and graphics with wide-ranging practical applications across web design, digital photography, multimedia communication, and augmented reality. Its primary goal is to adapt an image’s size or aspect ratio to suit different display platforms and devices, from large desktop screens to mobile interfaces and wearable displays. Traditional resizing methods, such as scaling and cropping, often lead to geometric distortions or the loss of critical content. Image retargeting preserves the semantic importance and spatial structure of the scene. This makes it particularly valuable in scenarios where maintaining both visual quality and contextual meaning is essential, such as delivering consistent user experiences across devices, optimizing visual content for social media and e-commerce, or ensuring clarity and interpretability in medical and aerial imaging applications. Various adaptive retargeting methods were proposed to ensure critical contents within images, such as objects, faces, and text, remain recognizable and visually appealing across devices. Among them, seam searching-based methods [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 ] and image warping-based techniques [ 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 ] achieved good performances. The former utilizes edges, saliency, or high-level semantics to calculate the importance map and iteratively search and remove the most unnoticeable seam to change the aspect ratio. However, information loss and artifacts in salient geometrical structures may be introduced. The latter formulates retargeting as an image warping problem, in which the input image is partitioned into meshes and deformation constraints (energy terms) are imposed on the mesh grids. The optimal deformation is obtained by minimizing the corresponding energy functions. Benefiting from the high deformation flexibility of the warping model, these methods can reduce information loss and better preserve geometric structures. However, the handcrafted nature of their deformation constraints often necessitates computationally expensive iterative solvers, and they typically require trade-offs between achieving extreme retargeting ratios and maintaining geometric fidelity. They often fail in certain scenarios or retargeting size, as shown in Fig. 1 . Recent years have witnessed growing interest in leveraging deep neural networks for image retargeting tasks [ 17 , 18 , 19 , 20 ] . A fundamental challenge in this paradigm lies in the dataset and corresponding label construction. However, the explosion of possible aspect ratios or retargeting sizes renders conventional supervised learning approaches impractical. Current approaches circumvent this limitation through weakly- or self-supervised frameworks, where a deformation from input to output is learned by auxiliary objectives like classification consistency [ 17 ] or perceptual coherence [ 18 ] . Although these approaches achieve improved geometric preservation through carefully designed objective functions, their deformation models offer limited flexibility, being restricted either to one-dimensional parameterization or to simple scaling factors. As illustrated in Fig. 1 , this architectural constraint still leads to suboptimal geometric preserving in certain retargeting scenarios. Parallel progress in geometric vision tasks using a learning-based warping framework, such as homography estimation [ 21 , 22 ] , image rectangling [ 23 ] , image rectification [ 24 , 25 ] , image stitching [ 26 , 27 ] and versatile warping model [ 28 ] , indicates a promising research direction. These methods constructed datasets and labels first. Then, they integrated learned warping mechanisms with geometric and pixel-level constraints, eliminating the necessity for handcrafted energy functions while maintaining geometrical structures. This framework simultaneously maintains the high-deformation freedom of warping models and the outstanding representational (or fitting) capacity of deep learning. Their proven effectiveness and robustness suggest significant potential for image retargeting applications. In this paper, we demonstrate the viability of such adaptation to image retargeting tasks through a novel self-supervised neural warping framework without label construction, called Object-IR . Our model is trained solely by enforcing appearance consistency for important semantic objects after retargeting and constraining the meshes within these regions to undergo only scale transformations. Compared with the classification or feature consistency, the object consistency provides spatial and structural supervision, which is stronger and more generalizable. Specifically, we predefine a uniform rigid mesh for the output resolution and use a simple but effective convolutional neural network to estimate the grids’ motion to obtain the deformed mesh. To train the network in a self-supervised way, we design a comprehensive objective function consisting of an object loss, a geometric loss, and a boundary loss. The object loss aims to enforce object consistency between the input and output images, the geometric loss is proposed to prevent the meshes within the object from distortion, and the boundary loss is included to enforce a rectangular output. To fully assess the retargeting quality, we propose a metric that uses object consistency to evaluate the distortion error in the retargeted images. Compared with the existing methods, our Object-IR can effectively mitigate the geometric distortion in the retargeted images for arbitrary retargeting sizes. Our contributions are summarized as follows: 1. We propose a neural network that casts image retargeting into mesh-based warping and can directly produce deformed meshes given any aspect ratio. 2. We design a comprehensive objective function that enables the retargeting method to be trained in a self-supervised way. 3. We propose a retargeting quality metric to evaluate the distortion error in the retargeted images. 4. Our Object-IR can be trained once and achieves the best retargeting quality given any aspect ratio. The remainder of this paper is structured as follows. Sec. 2 briefly reviews prior methods and associated technical challenges. Sec. 3 details our proposed method. Sec. 4 presents the implementation details

Read the complete text