High-pass filter periodogram: an improved power spectral density estimator for unevenly sampled data.
Ezequiel Albentosa-Ruiz, Nicola Marchili · 2024
Read High-pass filter periodogram: an improved power spectral density estimator for unevenly sampled data. by Ezequiel Albentosa-Ruiz, Nicola Marchili (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-02656.txt.
Description
Abstract Accurate time series analysis is essential for studying variable astronomical sources, where detecting periodicities and characterizing power spectral density (PSD) are crucial. The Lomb-Scargle periodogram, commonly used in astronomy for analyzing unevenly sampled time series data, often suffers from noise introduced by irregular sampling.
Catalog record
| CREATOR | Ezequiel Albentosa-Ruiz, Nicola Marchili |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-02656
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
High-pass filter periodogram: an improved power spectral density estimator for unevenly sampled data. Ezequiel Albentosa-Ruiz, Nicola Marchili Abstract Accurate time series analysis is essential for studying variable astronomical sources, where detecting periodicities and characterizing power spectral density (PSD) are crucial. The Lomb-Scargle periodogram, commonly used in astronomy for analyzing unevenly sampled time series data, often suffers from noise introduced by irregular sampling. This paper presents a new high-pass filter (HPF) periodogram, a novel implementation designed to mitigate this sampling-induced noise. By applying a frequency-dependent high-pass filter before computing the periodogram, the HPF method enhances the precision of PSD estimates and periodicity detection across a wide range of signal characteristics. Simulations and comparisons with the Lomb-Scargle periodogram demonstrate that the HPF periodogram improves accuracy and reliability under challenging sampling conditions, making it a valuable complementary tool for more robust time series analysis in astronomy and other fields dealing with unevenly sampled data. I Introduction Time series analysis plays an essential role in the study of variable astronomical sources. The detection of possible periodicities and the correct characterisation of the power spectral density (PSD) of the signal emitted by a source are among the most valuable pieces of information that could be gained through time series analysis; they can provide us with essential indications concerning the physical processes occurring in the emitting regions, and help us to discriminate between different emission models. This explains the crucial position occupied in variability studies by the periodogram (see Schuster 1898 for the classical expression; for uneven data, it is mostly used in the formulation known as Lomb-Scargle periodogram, see Lomb 1976 and Scargle 1982 ), which is able to extract from the light curves information about both periodicities and PSD slope. Similarly to any other tool of time series analysis, the periodogram is affected by the sampling and the length of the analysed datasets. The finite duration of the observations gives rise to sidelobes, i.e. a spectral leakage to nearby frequencies. The most serious issue for a reliable characterisation of the variability, however, is caused by the sampling. VanderPlas 2018 provides a clear and thorough discussion of how the sampling affects the spectral information returned by Discrete Fourier Transform, classical periodogram, and Lomb-Scargle periodogram. When the sampling of the analysed datasets is even, the spectral power is transferred to distant frequencies through aliasing; the frequencies to which the power is transferred are strictly related to the sampling interval \Delta t, as they are multiples of 1/ \Delta t. In the case of uneven sampling, the power transfer caused by the sampling is randomised by the different time intervals between consecutive observations, leading to non-structured frequency peaks whose superposition results in a large and unpredictable increase of noise at high frequencies; this can completely hide the real characteristics of the underlying signal. The development of the Lomb-Scargle periodogram proceeds from the need to find a way to retrieve reliable spectral information from astronomical datasets, whose sampling is generally uneven. The main purpose of the modification proposed by Scargle 1982 , and based on the previous work of Lomb 1976 , is to make sure that the simple statistical behaviour of the evenly spaced case remains valid also with uneven samplings. The Lomb-Scargle (LS) periodogram (from now on, we will refer to it simply as ”the periodogram”; the classical algorithm by Schuster 1898 will not be further considered) at angular frequency \omega is defined as follows: \displaystyle P_{X}(\omega)=\frac{1}{2}\Bigg\{ \displaystyle\frac{\left[\sum_{j}X_{j}\cos\omega(t_{j}-\tau)\right]^{2}}{\sum_{j}\cos^{2}\omega(t_{j}-\tau)} \displaystyle+\frac{\left[\sum_{j}X_{j}\sin\omega(t_{j}-\tau)\right]^{2}}{\sum_{j}\sin^{2}\omega(t_{j}-\tau)}\Bigg\}, (1) where X is the physical variable measured at a set of times t_{j} , and \tau is defined as \tan(2\omega\tau)=\frac{\sum_{j}\sin 2\omega t_{j}}{\sum_{j}\cos 2\omega t_{j}}. (2) The success in achieving a simple statistical behaviour is not matched by a similar favourable outcome in reducing the sampling-induced noise in the results. In this sense, the Lomb-Scargle periodogram formulation often does not show large improvements compared to the classical algorithm. In this paper, a new implementation of the periodogram, the high-pass filter (HPF) periodogram, is proposed. Aim of this new analysis method is not to replace the periodogram, as it was developed starting from complementary purposes; its main aim is to reduce as much as possible the noise introduced by the sampling, in order to be able to establish with much better precision the PSD of the analysed signal. As will be shown in the following, a better estimation of the PSD can also favour a more reliable detection of periodicities. The paper is organized as follows: In Section II , we present our new implementation of the periodogram. Section III details the simulations and the testing pipeline developed to evaluate the PSD estimation provided by the LS and HPF periodograms. It also explains the theoretical basis of periodicity detection using the periodogram, defines the detection threshold, and outlines the pipeline for injecting periodic signals into our simulations and using the periodograms for periodicity identification. In Section IV , we present the results from the testing pipelines. Finally, Section V summarizes the conclusions of this paper. II The high-pass filter periodogram The algorithm proposed in this paper aims to reduce the sampling-induced noise in the periodogram and is the result of considerations concerning the origin of the noise. The power of the analysed signal returned by the periodogram at a given frequency f is transferred to other frequencies, in a way that is determined by the irregular sampling. The higher the power in the signal, the more significant the peaks induced at other frequencies by the sampling. In the common astronomical case where the signal can be approximated, within a large range of frequencies, by red noise, i.e. PSD( f ) \propto f^{-2} , or at least with flicker noise, i.e. PSD( f ) \propto f^{-1} , the low frequency components of the signal carry much more power than the high frequency ones. Therefore, the power aliased from low frequencies is more substantial compared to that aliased from high frequencies. This explains why the low frequency values of the periodogram are marginally affected by the sampling, while the high ones can be heavily dominated by noise. Based on these considerations, we developed an approach that applies a frequency-dependent high-pass filter to the signal before calculating the periodogram value at a given frequency 1 1 1 The high-pass filter periodogram tool (DOI: 10.5281/zenodo.13917829 ) is available for download on GitHub: https://github.com/ealruiz/HPF_Periodogram . . For each frequency f , we compute the periodogram of the signal after suppressing, in the time domain, the variability components of frequencies lower than f through the data de-trending algorithm developed by ( Villata et al. 2002 ) . The detailed steps of the process are outlined as follows: • For a given frequency f , the original light curve is averaged over time bins of size \Delta t=1/ f . • The averaged points are fitted with a spline curve, which is resampled to match the original data. This provides an estimate of the long-term trend on timescales longer than \Delta t. • The resampled spline is subtracted from the original data, yielding a de-trended light curve with the same sampling of the original data. • The periodogram at frequency f is calculated on the de-trended light curve. This procedure is repeated for all the