StylePitcher: Generating Style-Following and Expressive Pitch Curves for Versatile Singing Tasks

Jingyue Huang, Qihui Yang, Fei Yueh Chen, Julian McAuley, Randal Leistikow, Perry R. Cook, Yongyi Zang · 2025

Read StylePitcher: Generating Style-Following and Expressive Pitch Curves for Versatile Singing Tasks by Jingyue Huang, Qihui Yang, Fei Yueh Chen, Julian McAuley, Randal Leistikow, Perry R. Cook, Yongyi Zang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21685.txt.

Introduction

Jingyue Huang and six co-authors generate pitch curves for singing that carry a particular singer's personal flair. A pitch curve is the melody's fine trajectory, complete with slides and vibrato. Existing generators flatten individual style. Most were also built as side modules for one job, like fixing pitch or converting voices. They transfer poorly between jobs.

StylePitcher learns style from a reference recording while staying loyal to the intended score. It is built on rectified flow matching. It accepts the symbolic score and surrounding pitch as flexible conditions. It moves between singing tasks without retraining. Judged by machines and by listeners, it raises style similarity and sound quality across tasks. It holds pitch accuracy level with specialist baselines. One general tool can thus serve correction, synthesis, and conversion alike.

Catalog record

CREATORJingyue Huang, Qihui Yang, Fei Yueh Chen, Julian McAuley, Randal Leistikow, Perry R. Cook, Yongyi Zang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-21685

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

StylePitcher: Generating Style-Following and Expressive Pitch Curves for Versatile Singing Tasks Jingyue Huang, Qihui Yang, Fei Yueh Chen, Julian McAuley, Randal Leistikow, Perry R. Cook, Yongyi Zang Abstract Existing pitch curve generators face two main challenges: they often neglect singer-specific expressiveness, reducing their ability to capture individual singing styles. And they are typically developed as auxiliary modules for specific tasks such as pitch correction, singing voice synthesis, or voice conversion, which restricts their generalization capability. We propose StylePitcher, a general-purpose pitch curve generator that learns singer style from reference audio while preserving alignment with the intended melody. Built upon a rectified flow matching architecture, StylePitcher flexibly incorporates symbolic music scores and pitch context as conditions for generation, and can seamlessly adapt to diverse singing tasks without retraining. Objective and subjective evaluations across various singing tasks demonstrate that StylePitcher improves style similarity and audio quality while maintaining pitch accuracy comparable to task-specific baselines. 1]Smule Labs 2]University of California, San Diego 3]University of Rochester \contribution [†]Work done during internship at Smule 1 Introduction Pitch curves, or fundamental frequency (F0) curves, are the backbone of expressive singing. They encode not only the melody but also the subtle variations that define unique styles of different singers, such as their vibrato, ornaments, pitch bending, and others ( Dai et al. 2023 ) . Therefore, pitch curves serve as critical intermediate representations across diverse singing generation and conversion tasks, such as automatic pitch correction (APC) ( Perrotin and D’alessandro 2016 ; Zhuang et al. 2022 ; Hai and Elhilali 2023 ) , singing voice synthesis (SVS) ( Liu et al. 2022 ; Zhang et al. 2022 ; He et al. 2023 ; Dai et al. 2024 ; Zhang et al. 2024a ; Zhang et al. 2024b ) , and singing voice conversion (SVC) ( Chen et al. 2019 ; Deng et al. 2020 ; Guo et al. 2022 ; Li et al. 2022 ; Polyak et al. 2020 ; Bai et al. 2024 ; Zhou et al. 2025 ) . Despite their importance, existing approaches face two main limitations. First, most of them overlook singer-specific styles encoded in pitch curves, treating pitch as a singer-agnostic feature and reusing the same curve across different singers ( Chen et al. 2019 ; Deng et al. 2020 ) . This limitation is critical: the same melody sung by different singers can produce distinct pitch patterns, which reflect individual singing techniques and styles. Losing these patterns will neglect their singer-specific expressiveness and the essence of their performance ( Dai et al. 2024 ) . Second, while some recent approaches ( Hai and Elhilali 2023 ; Zhang et al. 2024a ) support style-informed pitch curve generation, they often develop the pitch curve generator as an auxiliary module for specific tasks, such as pitch correction ( Hai and Elhilali 2023 ) and singing voice synthesis ( Zhang et al. 2024a ) . This task-specific design constrains their generalization capability across different singing applications, as researchers have to retrain these modules with different hyperparameters, inputs and outputs for each adaptation. Therefore, a general-purpose model capable of generating style-following pitch curves is essential for diverse singing applications. We propose StylePitcher , the first style-following pitch curve generation model for versatile singing tasks. We formulate pitch curve generation as a masked infilling problem: given surrounding pitch context and symbolic music scores, StylePitcher learns to generate missing pitch segments that continue the style patterns from context. This approach enables implicit style modeling without requiring explicit singer labels or embeddings, allowing generalization to unseen voices. We employ a rectified flow model ( Liu et al. 2023 ) for stable, efficient, and high-quality generation process. In addition, we introduce a smoothing algorithm to construct reliable conditioning signals (i.e., symbolic music scores), removing the need for manual annotations. By separately modeling F0 and performing inpainting, StylePitcher generates pitch curves that follow the style of provided audio without any task-specific retraining. Once trained, it serves as a plug-and-play module for diverse applications, including pitch correction, zero-shot singing voice synthesis with style transfer and style-informed singing voice conversion. Figure 1 : Illustration of the methods. The unvoiced condition is omitted for clarity. Subscripts off and in denote features from off-key and in-key singing in the APC task; ref and tgt refer to reference and target content for SVS and SVC tasks. Our contributions are threefold: • StylePitcher 1 1 1 Demos available at https://stylepitcher.github.io/ is the first general-purpose, style-following pitch curve generator supporting diverse singing tasks. • We introduce a flow matching architecture to pitch curve generation, a smoothing algorithm for data annotation, and an inpainting mechanism for flexible task adaptation. • Objective and subjective evaluations across multiple singing tasks show that StylePitcher achieves superior or compatible performance on style similarity, audio quality and pitch accuracy relative to previous baselines. 2 Related Work Many singing generation frameworks explicitly predict pitch curves as an intermediate step. In automatic pitch correction, models generate pitch contours conditioned on target note sequences to correct out-of-tune vocals ( Perrotin and D’alessandro 2016 ; Zhuang et al. 2022 ; Hai and Elhilali 2023 ) . Singing voice synthesis aims to generate singing voice from lyrics and scores, with many approaches predicting F0 to model pitch variation ( Liu et al. 2022 ; Zhang et al. 2022 ; He et al. 2023 ; Dai et al. 2024 ; Zhang et al. 2024b ) . Specifically, StyleSinger ( Zhang et al. 2024a ) adopts style-specific and style-agnostic pitch predictors to capture singing styles from references. Singing voice conversion models transform the voice in a singing signal into that of a target singer while preserving content and melody. Most existing works ( Chen et al. 2019 ; Deng et al. 2020 ; Polyak et al. 2020 ; Li et al. 2022 ; Guo et al. 2022 ; Bai et al. 2024 ; Zhou et al. 2025 ) treat pitch as singer-agnostic, reusing the unchanged or key-shifted F0 sequences from the original singing, which converts timbre but overlooks singer-specific expressiveness. Unlike prior works that embed task-specific F0 predictors within complex frameworks, the proposed StylePitcher is a plug-and-play module for diverse singing tasks, supporting style preservation or transfer as needed. 3 Methods 3.1 Rectified Flow Rectified flow ( Liu et al. 2023 ) defines generative modeling as learning a deterministic transport map from a prior distribution \pi_{0} (e.g. Gaussian) to the data distribution \pi_{1} . It parameterizes a time-dependent velocity field v_{\theta}(x_{t},t,c) to push x_{0}\sim\pi_{0} towards x_{1}\sim\pi_{1} under the ordinary differential equation (ODE): dx_{t}=v_{\theta}(x_{t},t,c)dt\vskip-2.84544pt (1) where c denotes conditions, t\in[0,1] is the scalar time parameter, and x_{t} follows a rectified linear interpolation: x_{t}=(1-t)x_{0}+tx_{1}.\vskip-2.84544pt (2) The model is trained with the flow matching objective: \mathcal{L}(\theta)=\mathbb{E}_{x_{0}\sim\pi_{0},x_{1}\sim\pi_{1},t,c}\|v_{\theta}(x_{t},t,c)-(x_{1}-x_{0})\|^{2}_{2}.\vskip-2.84544pt (3) Once trained, starting from \pi_{0} , samples from \pi_{1} are obtained by integrating the learned ODE from t=0 to t=1 , producing high-quality results with few sampling steps. 3.2 Model Architecture As illustrated in Figure 1 , we formulate pitch curve generation as a conditional infilling task, following Voicebox ( Le et al. 2023 ) . Given a fundamental frequency curve x=(x^{1},\cdots,x^{N}) from a singing voice, the corresponding note sequence y=(y^{1},\cdots,y^{N}) , and a binary mask m\in\{0,1\}^{N} , our model p(x_{\text{mask}}|y,x_{\text{ctx}}) predicts the masked segments x_{\text{mask}}=m\odot x

Read the complete text