Rotational Odometry using Ultra Low Resolution Thermal Cameras
\IEEEauthorblockN Ali Safa \IEEEauthorrefmark 1 \IEEEauthorieeemembermark 1 \IEEEauthorblockA \IEEEauthorrefmark 1College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar \IEEEauthorieeemembermark 1Member, IEEE · 2024
Read Rotational Odometry using Ultra Low Resolution Thermal Cameras by \IEEEauthorblockN Ali Safa \IEEEauthorrefmark 1 \IEEEauthorieeemembermark 1 \IEEEauthorblockA \IEEEauthorrefmark 1College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar \IEEEauthorieeemembermark 1Member, IEEE (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-01227.txt.
Description
A 2024 arXiv research paper by \IEEEauthorblockN Ali Safa \IEEEauthorrefmark 1 \IEEEauthorieeemembermark 1 \IEEEauthorblockA \IEEEauthorrefmark 1College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar \IEEEauthorieeemembermark 1Member, IEEE. This letter provides what is, to the best of our knowledge, a first study on the applicability of ultra-low-resolution thermal cameras for providing rotational odometry measurements to navigational devices such as rovers and drones.
Catalog record
| CREATOR | \IEEEauthorblockN Ali Safa \IEEEauthorrefmark 1 \IEEEauthorieeemembermark 1 \IEEEauthorblockA \IEEEauthorrefmark 1College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar \IEEEauthorieeemembermark 1Member, IEEE |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 15 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-01227
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Rotational Odometry using Ultra Low Resolution Thermal Cameras \IEEEauthorblockN Ali Safa \IEEEauthorrefmark 1 \IEEEauthorieeemembermark 1 \IEEEauthorblockA \IEEEauthorrefmark 1College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar \IEEEauthorieeemembermark 1Member, IEEE Abstract This letter provides what is, to the best of our knowledge, a first study on the applicability of ultra-low-resolution thermal cameras for providing rotational odometry measurements to navigational devices such as rovers and drones. Our use of an ultra-low-resolution thermal camera instead of other modalities such as an RGB camera is motivated by its robustness to lighting conditions, while being one order of magnitude less cost-expensive compared to higher-resolution thermal cameras. After setting up a custom data acquisition system and acquiring thermal camera data together with its associated rotational speed label, we train a small 4-layer Convolutional Neural Network (CNN) for regressing the rotational speed from the thermal data. Experiments and ablation studies are conducted for determining the impact of thermal camera resolution and the number of successive frames on the CNN estimation precision. Finally, our novel dataset for the study of low-resolution thermal odometry is openly released with the hope of benefiting future research. \IEEELSENSarticlesubject Sensor Applications \IEEEtitleabstractindextext {IEEEkeywords} Thermal camera, Odometry, Convolutional Neural Network. Supplementary Material The dataset used in this work is openly available at: https://tinyurl.com/y385prj4 1 Introduction \IEEEPARstart Odometry estimation is a fundamental aspect of any navigational device such as rovers, drones and cars [ 1 , 2 ] . Using odometry, navigational devices can estimate key inertial measures such as their acceleration, their rotational and translational speed, and their position in the environment [ 3 ] . Traditionally, odometry is provided by using Inertial Measurement Units (IMUs) embedding an accelerometer, a gyroscope and a magnetic compass in one integrated sensor [ 4 ] . Thanks to Micro-Electro-Mechanical System (MEMS) technology, small-size IMU chips have become ubiquitous in many robotics and navigational applications [ 5 ] . On the other hand, using IMUs alone is known to suffer from growing estimation errors as the inertial measurement provided by the IMU are integrated through time [ 2 ] . This is due to the slowly-varying biases and non-idealities affecting the IMU readout [ 6 ] . For this reason, IMUs are often fused with visual data (from e.g., an RGB camera) forming a visual-inertial odometry system (VIO) [ 7 ] . The VIO fusion approach has been successfully used in many navigational settings to provide more precise positioning [ 8 , 9 ] , as well as in Simultaneous Localization and Mapping (SLAM) setups in order to concurrently map new environments while localizing the navigational device into the map [ 10 , 11 ] . But using RGB cameras to form VIO systems also comes with the fundamental issue that cameras are greatly affected by lighting conditions [ 16 ] . This can lead to a significant degradation in the odometry estimation when using VIO systems in low-light and nigh-time conditions [ 9 ] . Hence, in order to increase the robustness of VIO systems to lighting conditions, the use of other sensing modalities such as radar, LIDAR, high-dynamic-range (HDR) cameras, event-based cameras and high-resolution thermal cameras have been explored in literature [ 15 , 16 , 9 , 12 ] . Among these modalities, the use of thermal cameras has recently attracted a large attention due to its advantages in terms of sensing robustness, payload size and power consumption compared to the other aforementioned modalities [ 12 , 17 ] . Indeed, radars are known to be power-hungry due to their use of multiple antennas with multiple power amplifiers used to attain the emitted power required at high frequency (e.g., 79-GHz is a typical frequency) [ 15 ] . LIDARs are still bulky [ 16 ] and both HDR and event-based cameras can be expensive while still not being immune to total night-time conditions [ 13 , 19 ] . On the other hand, even though high-resolution thermal cameras can be both power- and size-efficient, they can still be expensive, costing in the \sim 500 \$ range [ 14 , 18 ] . Figure 1: Data acquisition setup. The 24\times 32 thermal camera is connected to a readout board which translates its I2C interface to a serial interface via USB. A 100\mu F decoupling capacitor is used for providing a stable power supply to the thermal camera. The thermal camera is mounted on top of a servo motor controlled by a micro-controller via serial interface over USB. This setup enables the acquisition of thermal camera data while rotating the camera at precisely-controlled speeds. Figure 2: CNN architecture for the estimation of rotational speed from thermal camera data. The CNN is composed of two convolutional layers (with max pooling in between), followed by two fully-connected layers and an linear output layer. This small-size architecture has been designed with the aim of reducing the CNN compute complexity for potential implementation in CNN accelerator hardware [ 20 ] . In order to reduce the system cost of thermal-based odometry systems, this paper present what is, to the best of our knowledge, the first demonstration of thermal-based rotational odometry using an ultra-low-resolution ( 24\times 32 ) thermal camera (reducing the sensor costs to the \sim 50 \$ range) [ 14 , 18 ] . The contributions of this paper are the following: 1. We build a custom data acquisition setup for acquiring low-resolution ( 24\times 32 ) thermal camera data with a precise control of the camera’s azimuth rotation speed in order to obtain a labelled dataset of thermal camera frames and rotational speed. 2. We study the use of small-scale Convolutional Neural Networks (CNNs) for regressing the rotational speed from the thermal camera frames. 3. We provide a study on the impact of thermal camera resolution and the number of consecutive input frames on the CNN odometry accuracy. 4. We release our dataset as open-source to help future research. This letter is organized as follows. Section 2 provides a description of our dataset and data acquisition hardware. Section 3 describes our CNN design choices and training approach. Section 4 describes our experimental results. Finally, Section 5 provides conclusions. 2 Data Acquisition In order to acquire labelled datasets of thermal camera data together with their azimuth rotational speed, the data acquisition setup depicted in Fig. 1 has been assembled. The data acquisition setup of Fig. 1 is controlled via a Python script running in an external laptop which sweeps the thermal camera at varying rotation speeds and jointly record the thermal camera data in order to build a labelled dataset \{\tilde{X},y\} where \tilde{X} is a sequence of thermal data associated to a rotation speed y . We acquire the dataset described in Table 1 by recording data in different environments, both indoor and outdoor. During each acquisition, the camera rotation speed is swept from 20 deg/s to 200 deg/s (for both the positive and negative directions). The camera frame rate is set to 8 fps. Doing so, we obtain a rich dataset containing 51561 thermal camera frames corresponding to y different rotation speeds and acquired in a total of 18 different environmental settings. Table 1: Dataset description. Data is acquired in four different environments: i) a laboratory where