StarWhisper Telescope: An AI framework for automating end-to-end astronomical observations

Cunshi Wang 1 2 , Yu Zhang 2 , Yuyang Li 1 2 , Xinjie Hu 2 3 , Yiming Mao 1 2 4 , Xunhao Chen 1 2 , Pengliang Du 2 , Rui Wang 2 , Ying Wu 2 5 , Hang Yang 1 2 , Yansong Li 6 , Beichuan Wang 1 2 , Haiyang Mu 2 , Xiaohan Chen 1 2 , Shunxuan He 1 2 , Hao Mo 1 2 , Liyue Zhang 1 2 , Lin Du 1 2 , Yunning Z · 2024

Read StarWhisper Telescope: An AI framework for automating end-to-end astronomical observations by Cunshi Wang 1 2 , Yu Zhang 2 , Yuyang Li 1 2 , Xinjie Hu 2 3 , Yiming Mao 1 2 4 , Xunhao Chen 1 2 , Pengliang Du 2 , Rui Wang 2 , Ying Wu 2 5 , Hang Yang 1 2 , Yansong Li 6 , Beichuan Wang 1 2 , Haiyang Mu 2 , Xiaohan Chen 1 2 , Shunxuan He 1 2 , Hao Mo 1 2 , Liyue Zhang 1 2 , Lin Du 1 2 , Yunning Z (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-06412.txt.

Introduction

Wang and co-authors present StarWhisper, an artificial intelligence system that runs a telescope survey from planning to follow-up on its own. Large telescope arrays now gather sky data faster than staff can plan sessions, process images, and decide what to do next. Those steps have become a bottleneck for the study of fleeting sky events.

StarWhisper links a large language model to specialist tools in a modular workflow. It builds observing lists tuned to each site, carries out observations in real time, and handles the data that comes back. The system was built for a survey hunting supernovae in nearby galaxies. Automating the routine chain frees astronomers to focus on the rare events that matter most.

Catalog record

CREATORCunshi Wang 1 2 , Yu Zhang 2 , Yuyang Li 1 2 , Xinjie Hu 2 3 , Yiming Mao 1 2 4 , Xunhao Chen 1 2 , Pengliang Du 2 , Rui Wang 2 , Ying Wu 2 5 , Hang Yang 1 2 , Yansong Li 6 , Beichuan Wang 1 2 , Haiyang Mu 2 , Xiaohan Chen 1 2 , Shunxuan He 1 2 , Hao Mo 1 2 , Liyue Zhang 1 2 , Lin Du 1 2 , Yunning Z
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE76 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-06412

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

StarWhisper Telescope: An AI framework for automating end-to-end astronomical observations Cunshi Wang 1 2 , Yu Zhang 2 , Yuyang Li 1 2 , Xinjie Hu 2 3 , Yiming Mao 1 2 4 , Xunhao Chen 1 2 , Pengliang Du 2 , Rui Wang 2 , Ying Wu 2 5 , Hang Yang 1 2 , Yansong Li 6 , Beichuan Wang 1 2 , Haiyang Mu 2 , Xiaohan Chen 1 2 , Shunxuan He 1 2 , Hao Mo 1 2 , Liyue Zhang 1 2 , Lin Du 1 2 , Yunning Zhao 1 2 , Jianfeng Tian 2 , Liang Ge 2 5 , Yongna Mao 1 , Shengming Li 2 , Zheng Wang 2 , Xiaomeng Lu 2 , Jinhang Zou 2 , Yang Huang 1 2 , Ningchen Sun 1 2 , Jie Zheng 2 , Min He 2 , Yu Bai 2 5 , Junjie Jin 2 , Hong Wu 1 2 , Jifeng Liu 1 2 5 1 University of Chinese Academy of Sciences, Beijing, China; 2 Key Laboratory of Optical Astronomy, National Astronomical Observatories, Chinese Academy of Sciences, Beijing, China; 3 School of Computing Science, Simon Fraser University, Burnaby, British Columbia V5A1S6, Canada; 4 College of Science, Tibet University, Tibet, China; 5 Institute for Frontiers in Astronomy and Astrophysics, Beijing Normal University, Beijing, China; 6 Liii Network Co., Zhejiang, China Abstract The exponential growth of large-scale telescope arrays has boosted time-domain astronomy development but introduced operational bottlenecks, including labor-intensive observation planning, data processing, and real-time decision-making. Here we present the StarWhisper Telescope system, an AI agent framework automating end-to-end astronomical observations for surveys like the Nearby Galaxy Supernovae Survey. By integrating large language models with specialized function calls and modular workflows, StarWhisper Telescope autonomously generates site-specific observation lists, executes real-time image analysis via pipelines, and dynamically triggers follow-up proposals upon transient detection. The system reduces human intervention through automated observation planning, telescope controlling and data processing, while enabling seamless collaboration between amateur and professional astronomers. Deployed across Nearby Galaxy Supernovae Survey’s network of 10 amateur telescopes, StarWhisper Telescope has detected transients with promising response times relative to existing surveys. Furthermore, StarWhisper Telescope’s scalable agent architecture provides a blueprint for future facilities like the Global Open Transient Telescope Array, where AI-driven autonomy will be critical for managing 60 telescopes. 1 Introduction As time domain astronomy develops, large scientific facilities are in the planning. The Global Open Transient Telescope Array (GOTTA [ 1 , 2 ] , which also referred to the SiTian project.), aims to construct 60 1-meter telescopes in its first stage to build an array to monitor the sky. The labor cost for the observation of the GOTTA is huge, with an estimated observation personnel sequence exceeding 200 people. Most of the telescopes will be deployed at the Lenghu Observatory, with a latitude of about 4,200 meters. Due to the large number of telescopes, automated observations are necessary for future time-domain surveys. With the help of a Large Language Model (LLM) based agent, new automated observations will aid astronomers by taking the observations and processing the data, realizing the concept of an ”AI Astronomer”. Astronomical observation can be broadly divided into three phases: (1) observation planning, (2) observation execution, and (3) data processing. These phases are executed sequentially, starting with planning, followed by execution, and concluding with data processing. Notably, the results of data processing feed back into the planning phase, influencing future observation strategies and helping to optimize the overall observational process. The observation planning phase aims to create an observation list for each telescope participating in the night’s observations. This list includes the observation time (in UTC time), the astronomical objects to be observed, their coordinates (typically in Right Ascension, R.A., and Declination, Dec.), and other relevant information. Observation control is typically managed by the Observation Control System (OCS, [ 3 ] ), which operates based on the provided observation list and additional parameters, such as the exposure time, number of exposures, required filters, telescope position and focus control. The images observed by the telescope are subsequently transferred to the data pipeline [ 4 ] for processing, extracting the photometric information required by astronomers, and identifying transients as high-value objects. The observation, planning, and data processing phases are labor-intensive and require considerable effort. Generating the observation list involves selecting the most suitable objects from an input catalog containing more than 100,000 entries while taking into account the object’s sky visibility at each telescope’s position. Typically, each observation is first reviewed by the astronomer prior to being processed. The review process involves examining nearly 20 GB of raw image data per telescope to detect and confirm the presence of transient astronomical events that are of interest to astronomers. To free astronomers from these phases with tasks, we developed the StarWhisper Telescope (SWT) system, a LLM [ 5 , 6 ] agent-based astronomical observation system designed to assist astronomers by efficiently managing observations across a telescope array and saving astronomers’ time. While LLMs have demonstrated extraordinary capabilities in many fields, traditional LLMs, such as Qwen [ 7 , 8 ] , the GPT series [ 9 ] , and DeepSeek [ 10 ] , have not been equipped with tools or trained with specific data for astronomical observations, and more likely to present well-known celestial objects, no matter it can be observed or not (See Supplementary Section 1, Observation planning by Qwen). The SWT system addresses these challenges by integrating function calls to expand the capacities of LLMs and LLM-enhanced workflows to infuse domain expertise. Function calls enable LLMs to interact with external tools or systems by recognizing predefined operations, extending their capabilities beyond text generation to perform tasks or retrieve real-time information. This functionality is developed through training on datasets pairing input instructions with expected outputs, enhancing the models’ instruction-following abilities [ 8 , 9 ] . When equipped with function call capabilities, LLMs are often referred to as LLM-based agents, or simply agents in these paper [ 11 , 12 , 13 , 14 , 15 , 16 ] . LLM-enhanced workflow connects steps in a series or parallel configuration, incorporating tools, Application Programming Interfaces (APIs), and LLMs to handle complex, multi-stage tasks and do function calls. Compared to traditional workflows [ 17 ] , it can process unstructured data, make flexible judgments, and adaptively link each step of the process. This approach increases the credibility and usability of the system through the operation and interaction logic of natural language, while maintaining robust input/output structures [ 18 , 15 , 11 ] . The workflow in this manuscript infers the LLM-enhanced workflow. Figure 1: Workflow comparison between traditional observation methods and the SWT enhanced system. The TNS stands for Transient Name Server. The SWT system is applied to the Nearby Galaxy Supernovae Survey (NGSS, Section 2 ) to help the observers save time (Figure 1 ). The NGSS project is a time-domain survey with 10 amateur-level telescopes, with a simplified telescope organizational structure similar to GOTTA. In summary, the SWT observation module (Section 3 ) consists of observation planning(Section 3.2 ), observation control (Section 3.3 ) and a data pipeline (Section 3.4 ). Before and after

Read the complete text