A Deep Single Image Rectification Approach for Pan-Tilt-Zoom Cameras

Teng Xiao, Qi Hu, Qingsong Yan, Wei Liu, Zhiwei Ye, Fei Deng · 2025

Read A Deep Single Image Rectification Approach for Pan-Tilt-Zoom Cameras by Teng Xiao, Qi Hu, Qingsong Yan, Wei Liu, Zhiwei Ye, Fei Deng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-06965.txt.

Introduction

Teng Xiao and colleagues present FDBW-Net, a deep learning system that straightens single warped images from pan-tilt-zoom cameras. Such cameras use wide-angle lenses for surveillance, which bend straight lines. Prior deep methods often lose fine geometric detail, so the fix stays rough.

The network first builds barrel-distorted training images with a forward distortion model, which cuts pixel waste and blur. A pyramid encoder with attention then predicts backward warping flows that keep detail, and a multi-scale decoder restores the image. Tests on public sets, simulated camera views and real camera scenes put the method at the top of current results, widening where these cameras can be used.

Catalog record

CREATORTeng Xiao, Qi Hu, Qingsong Yan, Wei Liu, Zhiwei Ye, Fei Deng
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-06965

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Deep Single Image Rectification Approach for Pan-Tilt-Zoom Cameras Teng Xiao, Qi Hu, Qingsong Yan, Wei Liu, Zhiwei Ye, Fei Deng Abstract Pan-Tilt-Zoom (PTZ) cameras with wide-angle lenses are widely used in surveillance but often require image rectification due to their inherent nonlinear distortions. Current deep learning approaches typically struggle to maintain fine-grained geometric details, resulting in inaccurate rectification. This paper presents a Forward Distortion and Backward Warping Network (FDBW-Net), a novel framework for wide-angle image rectification. It begins by using a forward distortion model to synthesize barrel-distorted images, reducing pixel redundancy and preventing blur. The network employs a pyramid context encoder with attention mechanisms to generate backward warping flows containing geometric details. Then, a multi-scale decoder is used to restore distorted features and output rectified images. FDBW-Net’s performance is validated on diverse datasets: public benchmarks, AirSim-rendered PTZ camera imagery, and real-scene PTZ camera datasets. It demonstrates that FDBW-Net achieves SOTA performance in distortion rectification, boosting the adaptability of PTZ cameras for practical visual applications. † † footnotetext: * Corresponding author. This work was funded by the National Natural Science Foundation of China (42301491, 62376089) I Introduction Pan-Tilt-Zoom (PTZ) cameras equipped with wide-angle lenses are widely used in surveillance due to their flexible remote control, which allows for effective monitoring of large areas. However, the inherent nonlinear optical distortion of wide-angle lenses presents significant challenges for image-based visual computing such as object localization [ 1 ] and scene understanding [ 2 ] , making image rectification essential prior to application. Traditional image rectification approaches have relied on multi-view geometry theories, estimating internal camera parameters from a set of multi-view images [ 3 , 4 ] . However, these methods typically require large datasets and adherence to strict geometric constraints. In fixed-mounted PTZ camera setups, where the camera adjusts its view primarily through rotation and zooming rather than positional movement, the applicability and flexibility of traditional methods are limited. To achieve flexible image rectification for PTZ cameras, innovative works explored the implementation using sparse-view images [ 5 ] . However, because of insufficient and unstable geometric information, these approaches exhibited limitations in rectification performance. Consequently, single-image rectification methods [ 6 ] based on deep learning have gained significant attention. We also follow this idea. Fig. 1: This represents the two stages of the training process of image rectification. The top is the traditional pipeline, and the bottom is our method. The training process of these methods can be divided into two stages [ 7 ] : distorted data synthesis and rectification network (see Fig. 1 ). However, both are limited by the distortion strategy. In the stage of distorted data synthesis, distortion models are preferred for their ability to accurately fit complex lens distortion characteristics. Traditional methods mostly use the inverse distortion model [ 8 , 9 ] , which maps the distorted image back to the expected undistorted value. This is very natural, as it is similar to the process of image rectification. However, experiments indicate that employing this model during the data synthesis stage results in image blur and a loss of details. Instead, we use a forward distortion model [ 10 ] , which applies distortion to camera rays made by projecting 3D points onto an image and calculates the offset directly. The resulting synthetic image has a very accurate pixel mapping, which helps to preserve image details during network training. In rectification network, these methods typically employ two approaches: parameter regression and image generation. The former employs deep neural networks to estimate distortion parameters [ 11 , 12 ] , while the latter directly generates rectified images in an end-to-end manner [ 13 , 9 ] . However, these methods still face great challenges in image detail restoration, such as non-integer pixel redundancy and artifacts. To address existing challenges, we propose a Forward Distortion and Backward Warping Network (FDBW-Net). It begins by using a forward distortion model instead to synthesize barrel-distorted images to mitigate pixel redundancy and avoid image blur. The network leverages a pyramid context encoder to hierarchically extract and learn regional latent features. It incorporates a Backward Warping Estimation Module (BWEM), which applies channel-wise and spatial attention mechanisms to predict the precise backward warping flows for distortion rectification, enriched with geometric details. Additionally, the multi-scale decoder employs a Layer-by-Layer Rectification Module (LLRM) to restore image details and output rectified images. For each distortion layer, the decoder progressively adjusts the distorted pixels using a backward warping strategy to ensure content consistency. For evaluation, we validate the proposed FDBW-Net using public image datasets, synthetic PTZ camera imagery (we rendered them in AirSim [ 14 ] ), and real-scene PTZ camera datasets. This paper makes three contributions: • We proposed FDBW-Net, a novel framework for wide-angle image rectification that enhances detail restoration by considering distortion strategies at both distorted data synthesis and rectification network. • We used AirSim [ 14 ] to render a set of image datasets from PTZ cameras at different perspectives and zooms to train the FDBW-Net, which built a bridge for application to PTZ cameras. • Experiments demonstrate that FDBW-Net achieves SOTA performance in distortion rectification and has very good practicality in real-scene PTZ camera images. II Related Work Distorted Data Synthesis. Various models have been developed to describe radial distortion. Blind [ 8 ] explored six models of geometric distortion (e.g., barrel, pincushion, and wave) to increase data diversity, hoping to fully represent the real distortion of the camera, but this greatly increases the complexity of the algorithm and is not flexible enough in practice. Zhao et al. [ 7 ] introduced a cascade model inspired by fisheye lenses, which combines multiple reversible distortion models into a unified framework. However, it did not specifically consider the perspective of PTZ cameras and was not verified in real-scene PTZ camera imagery. The inverse distortion model used in PCN [ 9 ] is a popular barrel distortion model: \theta_{u}=\sum_{i=1}^{n}k_{i}\theta_{d}^{2i-1},\quad(n=1,2,3,\ldots). (1) where \theta_{u} and \theta_{d} are the angles in undistorted and distorted lenses, with k_{i} as coefficients. Although this model was designed for image rectification, it often introduces blur and leads to a loss of detail in synthetic images. To address this, we instead employ a forward distortion model, ensuring accurate pixel mapping from the undistorted image to the distorted space. This reduces artifacts and preserves details, making it highly effective for distortion rectification. Rectification Network The networks for image rectification can be broadly classified into two approaches: parameter regression and image generation. In the former, Blind [ 8 ] predicted forward warping flows to rectify perspective distortions in single images, but this often resulted in missing regions in the rectified image. DeepCalib [ 15 ] and Ordinal [ 11 ] estimated distortion parameters and focal lengths for effective rectification. However, they rely on limited parameters and might lead to inaccuracies and rectification errors. The recent RDTR [ 16 ] employed a unified radial distortion model with distortion-aware pre-training to achieve robust geometric corrections, yet it struggled to maintain control over the rectified image quality. Generative approaches were also investigated. DR-GAN [ 13 ] introduced the first adversarial network for radial distortion rectification, achieving pixel-level accuracy. However, this method sometimes produced blurred image content. PCN [ 9

Read the complete text