Active Illumination Control in Low-Light Environments using NightHawk

Yash Turkar, Youngjin Kim, Karthik Dantu · 2025

Read Active Illumination Control in Low-Light Environments using NightHawk by Yash Turkar, Youngjin Kim, Karthik Dantu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-06394.txt.

Introduction

Yash Turkar, Youngjin Kim, and Karthik Dantu give robots a way to manage their own lights in the dark. Underground passages like culverts defeat robot cameras: light is dim and surfaces offer few landmarks. A lamp helps, but it can glare, wash out the image, and drain the battery. Their framework, NightHawk, adjusts lamp brightness and camera exposure together, searching online for the setting that makes each scene most useful.

A new score based on how many image features can be found and matched guides the search, which reruns whenever conditions change. The team mounted the system on a legged robot walking a culvert under the Erie Canal. Feature detection and matching improved by roughly half to nearly threefold over fixed settings. The authors say the gains make position estimates steadier in hard lighting.

Catalog record

CREATORYash Turkar, Youngjin Kim, Karthik Dantu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-06394

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Active Illumination Control in Low-Light Environments using NightHawk Yash Turkar, Youngjin Kim, Karthik Dantu Abstract Subterranean environments such as culverts present significant challenges to robot vision due to dim lighting and lack of distinctive features. Although onboard illumination can help, it introduces issues such as specular reflections, overexposure, and increased power consumption. We propose NightHawk 1 1 1 This project was partially funded by NSF #1846320 and a gift from MOOG Inc. , a framework that combines active illumination with exposure control to optimize image quality in these settings. NightHawk formulates an online Bayesian optimization problem to determine the best light intensity and exposure-time for a given scene. We propose a novel feature detector-based metric to quantify image utility and use it as the cost function for the optimizer. We built NightHawk as an event-triggered recursive optimization pipeline and deployed it on a legged robot navigating a culvert beneath the Erie Canal. Results from field experiments demonstrate improvements in feature detection and matching by 47-197% enabling more reliable visual estimation in challenging lighting conditions. Figure 1: Active illumination and exposure control on Boston Dynamics Spot, inspecting culverts under the Erie Canal in Medina, NY. (Left) Spot approaches entrance of culvert 110, a 66m long, subterranean environment with extremely low ambient illumination. (Right) Spot entering and exiting the culvert equipped with NightHawk 1 Introduction Environment illuminance plays a critical role in the performance of robot perception algorithms, many of which rely heavily on feature detection and matching. Illuminance directly influences image brightness and camera exposure, ultimately affecting image utility. This impact is especially pronounced in scenarios where lighting conditions are variable, such as indoor-outdoor transitions, shadowed areas, or environments with dynamic or low lighting. Standard vision sensors, including monocular, RGB-D, and stereo cameras, rely on autoexposure (AE) to adjust settings like exposure time (shutter speed), gain (ISO), and aperture. While AE performs well in typical (well-lit) scenarios, it struggles in extreme environments. The primary limitation of AE stems from its objective: maintaining a mean pixel intensity around 50%, aiming for neither over nor under exposure. However, this approach may not be optimal for robot vision tasks. The goal in these applications is to reliably detect and match stable features, which a neutrally exposed image doesn’t necessarily guarantee. Several exposure control methods have been proposed [ 24 ] , [ 6 ] , [ 25 ] , [ 8 ] , [ 13 ] , [ 10 ] , [ 5 ] . [ 11 ] , controls camera exposure-time while [ 23 ] controls both exposure-time and gain. Some of these methods propose image utility metrics that aim to quantify the quality of images from a feature detection and matching perspective like M_{shim} [ 21 ] , M_{softperc} [ 26 ] and NEWG [ 9 ] . These metrics typically revolve around using image gradients, as most feature detectors exploit gradients for keypoint detection. These exposure control methods adjust camera parameters such as shutter speed and gain, but often fall short in challenging low-light or varying-light conditions. Insufficient scene radiance can necessitate excessively long exposures or high gains leading to reduced frame-rates and increased noise. Integrating an onboard light source offers a promising solution by augmenting scene illumination. However, naive control of onboard lighting can introduce undesirable artifacts such as specular reflections [ 3 ] and overexposure, while also consuming significant robot power. Careful tuning and adjustment of light intensity is essential to mitigate these drawbacks. Our motivation stems from our efforts to inspect culverts beneath the Erie Canal in western New York. These culverts are long pipes with a 1-meter diameter that extend across the canal. As shown in Figure 1 , they are dimly lit with extreme light variations at the entrances. They are characterized by repeating textures and features, posing a significant challenge for visual estimation. We address these challenges with NightHawk , a novel framework that combines active illumination with exposure control. We also present a new image utility metric ( M_{feat} ) which leverages learning-based feature detectors to assess image quality and demonstrates strong correlation with feature matching performance in low-light settings. This metric serves as the cost function for our online Bayesian optimization process enabling us to determine optimal external light intensity and camera attribute values. Our key contributions are: 1. Novel image utility metric ( M_{feat} ) based on a modern learning-based feature detector that effectively quantifies feature performance 2. An active illumination and exposure control framework ( NightHawk ) which uses online Bayesian optimization to find optimal external-light intensity ( P ) and exposure-time ( \Delta t ) 3. Experimental validation in a challenging subterranean environment—a 66-meter-long, 1-meter-diameter culvert beneath the Erie Canal—to demonstrate enhanced feature matching performance. 2 NightHawk Design NightHawk is an external light and camera exposure-time control algorithm that uses event-triggered Bayesian Optimization to provide optimal lighting and camera configuration. This section describes the image utility metric and illumination control strategy. 2.1 Image Utility Effective exposure control requires a reliable mechanism to quantify image quality. Conventional auto-exposure (AE) mechanisms rely on irradiance aimed to maintain a mean intensity of 50% or 128 (8-bit). Prior approaches proposed by [ 21 ] , [ 26 ] , [ 9 ] control exposure by using image gradient-based utility metrics. Recently, there have been numerous learning-based feature detectors (e.g., SiLK [ 4 ] , R2D2 [ 17 ] ) that are trained in a self-supervised manner to estimate probabilities of being "interesting" per pixel. Further, detectors such as R2D2 explicitly output repeatability and reliability tensors aligned with image dimensions, enabling a more nuanced assessment of feature quality. Inspired by this literature, our intuition is to leverage such probabilities as direct feedback to assess image utility. To quantify image utility, we utilize R2D2 as the base feature detection network. We compute the product of the mean repeatability and the square of the mean reliability that yields a single scalar performance index M_{feat} per image: M_{\text{feat}}=\left(\frac{1}{N}\sum_{i=1}^{N}R_{i}\right)\cdot\left(\frac{1}{N}\sum_{i=1}^{N}Q_{i}\right)^{2} (1) This metric describes the image’s utility for successful feature detection. The square of the mean reliability amplifies its influence on the final score, reflecting the critical role of the descriptor in accurate matching performance. Here, R_{i} is repeatability per pixel, Q_{i} is reliability per pixel, and N is the total number of pixels. We evaluated M_{feat} ’s performance and compared it against other metrics in section 3 where M_{feat} shows a strong correlation with feature matching performance. 2.2 Illumination and Exposure Control The illumination and exposure control problem is formulated as a multi-variable Bayesian optimization (BO) where the optimal value of the light-intensity ( P ) and exposure-time ( \Delta t ) are determined by maximizing our image quality metric M_{feat} . The Gaussian process (GP) provides the surrogate model to parameterize the influence of P and \Delta t with zero mean Gaussian noise n=\mathcal{N}(0,\sigma_{n}^{2}) . The mean \mu and covariance \sigma^{2} of the predictive distribution is given by: \displaystyle\mu(\textbf{x}_{*})=k_{*}^{T}(K+\sigma_{n}^{2}I)^{-1}y (2) \displaystyle\sigma^{2}(\textbf{x}_{*})=k(\textbf{x}_{*},\textbf{x}_{*})-k^{T}_{*}(K+\sigma_{n}^{2}I)^{-1}k_{*} (3) where x = [ P , \Delta T ] gives inputs \textbf{x}_{*}=\{\textbf{x}_{0},\dots,\textbf{x}_{m}\} and outputs y=\{y_{0},\dots,y_{m}\} . In this work, the Matérn kernel [ 16 ] function is selected to construct the correlation matrix K and vector k . In addition, Expected Improvement [ 22 ] (EI)

Read the complete text