Detecting False Positives With Derived Planetary Parameters: Experimenting with the KEPLER Dataset

Ayan Bin Rafaih, Zachary Murray · 2025

Read Detecting False Positives With Derived Planetary Parameters: Experimenting with the KEPLER Dataset by Ayan Bin Rafaih, Zachary Murray (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-13801.txt.

Introduction

Ayan Bin Rafaih and Zachary Murray sort real planets from false signals in Kepler data in a 2025 paper. Telescopes flag many candidate transits that later prove false. Most classifiers read full light curves, which are long and heavy to process.

The pair instead fit only derived planet parameters, a compact summary of each signal. Four model types were tested on precision, recall, and accuracy trade-offs. The compact approach caught up to about ninety percent of false positives, suggesting the parameters hold most of the telling information. Random forests and neural networks scored best overall. Performance also varied with a stellar eclipse flag in the data.

Catalog record

CREATORAyan Bin Rafaih, Zachary Murray
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-13801

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Detecting False Positives With Derived Planetary Parameters: Experimenting with the KEPLER Dataset Ayan Bin Rafaih, Zachary Murray Abstract Recent developments in computational power and machine learning techniques motivate their use in many different astrophysical research areas. Consequently, many machine learning models have been trained to classify exoplanet transit signals - typically done by using time series light curve. In this work we attempt a different approach and try to improve the efficiency of these algorithms by fitting only derived planetary parameters, instead of full time-series light curves. We investigate and evaluate 4 models (Logistic Regression, Random Forest, Support Vector Machines and Convolutional Neural Networks) on the KEPLER dataset, using precision-recall trade-off and accuracy metrics. We show that this approach can identify up to ≈ 90 % \approx 90\% of false positives, implying the planetary parameters encompass most of the relevant information contained in a light curve. Random Forest and Convolutional Neural Networks produce the highest accuracy and the best precision-recall trade-off. We also note that the accuracies as a function of the stellar eclipse flag ( SS ) have the best performance. Kepler False Positives keywords Exoplanets — Machine Learning — Convolutional Neural Networks — False Positives — Logistic Regression — Random Forest — Support Vector Machines 1 Introduction The first exoplanet was confirmed in 1992 by pulsar timing ( Wolszczan & Frail 1992 ) , this detection was followed shortly by the first detection of an exoplanet around a main-sequence star in 1995 via radial velocities ( Mayor & Queloz 1995 ) . The first detected transit of an exoplanet was not until 2000, when HD 209458b was found to transit its star ( Charbonneau et al. 2000 ) . Despite this rather late start, the transit method has since proved to be the most productive method of detecting new exoplanets, with more than five thousand confirmed planets found to date. It is also the main method employed by large exoplanet searches, including the Kepler and TESS missions ( Koch et al. 2010 ; Ricker et al. 2015 ) and in addition to numerous ground based searches, including the WASP, TRAPPIST, KELT, NGST and many others ( Pepper et al. 2007 ; Butters et al. 2010 ; Jehin et al. 2011 ; Wheatley et al. 2018 , e.g) . Large transit surveys typically search for transit-like features in their lightcurves using a box least squares algorithm ( Kovács et al. 2002 ) , or transit least squares algorithm ( Hippke & Heller 2019 ) . However, not every candidate detected with these algorithms corresponds to an exoplanet. Sources of false positives include eclipsing binaries ( Brown 2003 ; O’Donovan et al. 2006 ) , stellar contamination from blended sources, and instrumental artifacts ( Coughlin et al. 2016 ) . False positives are often identified with additional spectroscopic or photometric observations. However, when such observations are not available, statistical methods are often used to estimate a false positive probability. The early algorithms included the VESPA code ( Morton 2012 ; Morton 2015 ) , while more modern approaches include TRICERATOPS ( Giacalone et al. 2021 ) or other machine learning algorithms ( Armstrong et al. 2021 ; Malik et al. 2022 ; Tardugno Poleo et al. 2024 ) . In general, these false positive detection scripts operate on raw light curves to maximize the amount of available information for classification. However, this approach has the downside of requiring rather sophisticated algorithms to work with the raw data. In this work, we take a different approach. We focus instead on the standard transit parameters (for example, transit depth and the impact parameter) computed for these false positives. Since the planetary parameters are essentially low-dimensional, yet physically motivated, summaries of the light curves, using them instead of the full light curves results in a significant simplification of the machine learning models. In this work, we investigate this trade off in accuracy. We find algorithms that detect these false positives, as closely as possible, using only the Kepler derived planetary parameters. We show that even simple regressions can separate the false positives from the true planets with accuracies exceeding 70\% , and more sophisticated models can do so with accuracies of \approx 90\% . Finally we show that the models accuracies are maintained for certain types of contamination, especially those with features that are not-transit like or stellar eclipses, but do struggle for other types of systematics. 2 Methods For this study, we utilize data collected by the Kepler Space Telescope, launched by NASA in 2009 to survey a 115\mathrm{deg^{2}} field of view of the northern sky for exoplanets ( Borucki et al. 2010 ; Koch et al. 2010 ) . Our training data is sourced from the Kepler Objects of Interest catalogue cumulative list, which is currently hosted at the NASA Exoplanet Archive. 1 1 1 https://exoplanetarchive.ipac.caltech.edu/ On date of access, the data set contains a total of 9,564 entries and the individual characteristics of each body, including observed properties (e.g planet-star radius ratio) and derived ones (e.g. the effective temperature). The Kepler catalogue also provides a ternary classification feature, derived form a numerical disposition score. This score is a numerical value that expresses the confidence in the category classification and is calculated using a Monte Carlo algorithm that equates the score’s value to the proportion of iterations where the Kepler Robovetter algorithm ( Coughlin 2017 ; Thompson et al. 2018 ; Coughlin 2020 ) classifies each threshold-crossing event as a potential candidate. The catalogue consists of three types of entries: (i) confirmed exoplanets using the NASA exoplanet archive disposition, (ii) candidate exoplanets using the Kepler Catalogue Disposition, and (iii) False Positive exoplanets. For candidates, a higher score represents a greater amount of confidence in its disposition, while for false positives, a higher value indicates less confidence in its classification. For our study, we choose a binary Kepler disposition feature list of exoplanets as either ”Candidates” or ”False Positives”, from the ternary set in the archive. There are in total 9,564 entries in our extracted binary catalogue of which 4,717 are candidates and 4,847 are false positives. As shown in Figure 1 , the distribution of scores for both the categories follows a uniform distribution: the false positives are concentrated towards score of \sigma=0 , while the candidates are concentrated towards \sigma=1 . A large proportion of the candidates planets are converted to confirmed exoplanets, while some retain their Kepler vetting status. Since the number of candidates with \sigma<0.2 is very low, we adopt the binary categories provided by Kepler disposition. Additionally, we can also see the relatively concentrated spread of the candidates that don’t become confirmed exoplanets, near the \sigma>0.80 threshold, showing the high confidence in the remaining planets. Figure 1: We show the distribution of \sigma (koi_score) across the categories provided in the disposition. We note that the false positives remain the same in number, with high confidence in their dispositions. On the other hand, most of the candidates either preserve their vetting status, or become confirmed exoplanets through dispositional vetting methods employed outside of the Kepler data. There’s a very small, thin spread of candidates across the entire range of koi_score, which is however insignificant. For example, we note

Read the complete text