Power Spectral Density Estimation via Universal Truncated Order Statistics Filtering

David Campos Anchieta, John R. Buck · 2025

Read Power Spectral Density Estimation via Universal Truncated Order Statistics Filtering by David Campos Anchieta, John R. Buck (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-16359.txt.

Introduction

Underwater listening posts must measure background noise, but loud passing sounds corrupt those measures. David Campos Anchieta and John R. Buck blend several ranked-data filters into one noise estimator that adapts on its own. Older filters each pick a cutoff rank to dodge loud bursts, a choice that fails when burst rates shift without warning.

The blended version weights its parts by recent performance, favoring whichever has the least spread lately. Theory says it nears the best fixed choice, and tests on simulated and real data agree. It cleans spectrograms without anyone setting a threshold rank by hand.

Catalog record

CREATORDavid Campos Anchieta, John R. Buck
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-16359

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Power Spectral Density Estimation via Universal Truncated Order Statistics Filtering David Campos Anchieta, John R. Buck Abstract Loud transient signals in underwater acoustic data increase the bias and variance of background noise power spectral density (PSD) estimates based on sample mean. Recently, two PSD estimators mitigated the loud transient impact on PSD estimates by applying order statistics filtering (OSF). The first, the Schwock and Abadi Welch Percentile, scales a single rank order statistic (OS) of consecutive periodograms. The second, the truncated linear order statistics filter, is a weighted sum of OS up to a chosen rank. In order to minimize variance, both OSFs must carefully choose the highest rank that still eliminates the loud transients. However, in real-time applications in dynamic environments, loud transients occur at unpredictable rates, requiring dynamic adjustment of the OSF ranks to keep low bias and variance. To circumvent the challenges of real-time rank selection, this paper proposes a convex sum of OSFs across ranks with blending weights that are sequentially adjusted to favor the lowest variance OSFs over a recent time window. The performance of the blended sum provably approaches the performance of the best fixed rank OSF. Simulations and real data confirm the blended OSFs effectively filter loud transients out of spectrograms without explicitly choosing a threshold rank. I Introduction Many underwater acoustic signal processing applications depend on accurate background noise power spectral density (PSD) estimates. Estimating the ambient noise PSD is a critical step in data pre-whitening for constant false alarm rate (CFAR) detectors [ 11 , 1 ] . Background PSDs also contain information about rain or surface wind for remote sensing of meteorological events, for example [ 15 , 16 , 22 , 23 ] . However, underwater acoustic data often include loud transients – brief, high-amplitude signals –, which introduce bias and increase the variance of PSD estimators such as Welch’s Overlapped Segment Averaging (WOSA) [ 26 ] . Estimating the background noise PSD of acoustic environments with loud transients requires an alternative approach that is more robust to outliers than the sample mean. To mitigate the increased bias and variance introduced by loud outliers, order statistics filters (OSF) exclude data samples that exceed a chosen threshold rank in the order statistics (OS) when estimating parameters [ 17 , 6 , 8 ] . This paper focuses on two OSFs recently applied to PSD estimation. The first, the Schwock and Abadi Welch Percentile (SAWP) method, replaces the averaging step in the WOSA with an OS of the periodograms, rescaling each OS so it is an unbiased estimate of the underlying power for exponential random variables [ 18 ] . The second, the truncated linear order statistics filter (TLOSF), computes a weighted average over the lowest-ranked OS [ 4 , 20 ] . These weights are chosen to minimize variance while maintaining an unbiased estimate. By selecting a threshold rank OS below the maximum, OSFs reduce the impact of loud transients, while scaling and weighted averaging help maintain unbiased estimates. However, the OSFs must carefully choose a fixed threshold rank for each noise environment — a challenging task in real-time applications due to the unpredictable nature of loud transients. Both the frequency of occurrence and the power of the loud transients in underwater acoustic data are time-varying, requiring real-time selection of the threshold rank. Consider two asymptotic extreme cases. Setting the threshold rank to the minimum ensures maximum robustness to transients but also maximizes the variance of the OSF. Conversely, setting the threshold rank to the maximum (for TLOSF) or to the 80th percentile (for SAWP) minimizes variance but increases susceptibility to bias and mean squared error (MSE) from loud transients. Ideally, the OSF seeks a rank between these extremes, selecting the highest threshold rank that still excludes the transients present in the current noise environment while reducing the estimator variance. Choosing the threshold rank of an OSF to balance robustness to loud transients while reducing estimator variance remains a challenging model-order selection problem. To circumvent the challenges of rank selection in real-time applications of OSFs, this paper proposes a performance-weighted sum of OSFs across threshold ranks. The blending weights are sequentially adjusted over time to favor the OSFs performing best over a recent time window. Several signal processing applications have recently applied this same universal method for model-order weighting [ 25 , 24 , 19 , 7 , 9 , 10 ] . This performance-based blending yields an OSF that is universal over the selected threshold rank – an estimator that performs nearly as well as the best fixed-rank OSF in hindsight – under the right conditions. By dynamically adjusting its weights according their recent performance, the universal OSFs (UOSFs) not only circumvent the threshold rank selection, but adapt to changes in the occurrence of loud transients over time. Both UOSFs were effective in smoothing spectrograms of underwater acoustic data while filtering out loud broadband clicks. The remainder of this paper is organized as follows. Section II reviews some background on OSFs for PSD estimation. Section III describes the universal method by model-order weighting and its application on a UOSFs. Section IV presents simulation results comparing the universal SAWP and universal TLOSF with their fixed-rank counterparts, and real data comparing the UOSFs against WOSA in spectrograms of hydrophone recordings. Finally, Section V summarizes the results and draws conclusions. II Background This section is divided into four parts that present background material while establishing the mathematical notation of this paper. The first part gives a brief description of the WOSA spectral estimator. The second part describes the SAWP estimator as an alternative to the WOSA. The third part presents the TLOSF as an alternative OSF. The last part introduces a generalized mathematical representation for the OSFs, which will be convenient when developing a universal version of both. II-A Welch’s Overlapped Segment Averaging The WOSA is a classic spectral estimator that averages periodograms of overlapping segments of data into a background noise PSD estimate [ 26 ] . Each periodogram is the squared magnitude of a tapered segment of the time domain signal [ 12 ] : \hat{P}_{\mathrm{per}}\left[n\right]=\frac{1}{L}\left|\sum_{l=0}^{L-1}w[l]x[nD+l]e^{-j2\pi f_{0}l}\right|^{2}, (1) where x[l] is the signal in the time domain, w[l] is the taper window function, L is the number of time samples producing each periodogram, D is the shift between segments in number of samples, 0\leq n\leq N-1 with N being the total number of periodograms, and 0\leq f_{0}\leq 0.5 is the frequency bin normalized by the Nyquist rate. Each short-time WOSA PSD estimate is an average of a number R of consecutive periodograms: \begin{gathered}\hat{P}_{\mathrm{wosa}}\left[t\right]=\frac{1}{R}\sum_{n=0}^{R-1}\hat{P}_{\mathrm{per}}\left[tQ+n\right],\end{gathered} (2) where R\leq N and 0<t<T-1 , with T being the total number of short-time PSD estimates, and ( R-Q ) is the overlap between consecutive PSD estimates in number of periodograms. Each PSD estimate incorporates [(R-1)D+L]/f_{s} seconds of the time domain signal, where f_{s} is the sampling rate. The overlaps between periodograms D and between PSD estimates Q can be bigger or smaller according to the desired time resolution for the PSD estimates. In order for the OSF to filter loud transients, each PSD estimate must incorporate a segment of the input signal that is longer than the longest loud transient. II-B The SAWP Loud transients

Read the complete text