E2E-AEC: IMPLEMENTING AN END-TO-END NEURAL NETWORK LEARNING APPROACH FOR ACOUSTIC ECHO CANCELLATION
Yiheng Jiang, Biao Tian, Haoxu Wang, Shengkui Zhao, Bin Ma, Daren Chen, Xiangang Li · 2026
Read E2E-AEC: IMPLEMENTING AN END-TO-END NEURAL NETWORK LEARNING APPROACH FOR ACOUSTIC ECHO CANCELLATION by Yiheng Jiang, Biao Tian, Haoxu Wang, Shengkui Zhao, Bin Ma, Daren Chen, Xiangang Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-16774.txt.
Introduction
Yiheng Jiang, Biao Tian, Haoxu Wang, Shengkui Zhao, Bin Ma, Daren Chen, and Xiangang Li tackle echo in calls. When a speaker's voice loops back through a mic, talks get hard. Old fixes first strip the straight part of the echo. They also gauge the lag of the echo path. Their 2026 preprint drops both old steps for one neural net.
Their net learns the whole clean-up task end to end. It runs as a stream, fit for live calls. A step-wise training plan lifts its skill bit by bit. The goal is clear speech with no harsh cuts. Such nets could serve phones, meets, and smart speakers.
Catalog record
| CREATOR | Yiheng Jiang, Biao Tian, Haoxu Wang, Shengkui Zhao, Bin Ma, Daren Chen, Xiangang Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-16774
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
E2E-AEC: IMPLEMENTING AN END-TO-END NEURAL NETWORK LEARNING APPROACH FOR ACOUSTIC ECHO CANCELLATION Yiheng Jiang, Biao Tian, Haoxu Wang, Shengkui Zhao, Bin Ma, Daren Chen, Xiangang Li Abstract We propose a novel neural network-based end-to-end acoustic echo cancellation (E2E-AEC) method capable of streaming inference, which operates effectively without reliance on traditional linear AEC (LAEC) techniques and time delay estimation. Our approach includes several key strategies: First, we introduce and refine progressive learning to gradually enhance echo suppression. Second, our model employs knowledge transfer by initializing with a pre-trained LAECbased model, harnessing the insights gained from LAEC training. Third, we optimize the attention mechanism with a loss function applied on attention weights to achieve precise time alignment between the reference and microphone signals. Lastly, we incorporate voice activity detection to enhance speech quality and improve echo removal by masking the network output when near-end speech is absent. The effectiveness of our approach is validated through experiments conducted on public datasets. † † address: Tongyi Lab, Alibaba Group, China 1 Introduction Acoustic echo cancellation (AEC) is a crucial technology in voice interaction systems, addressing the significant challenge of eliminating acoustic feedback to ensure high-quality audio communication [1]. Traditionally, AEC systems relied on digital signal processing methods, typically starting with time delay estimation (TDE) to align microphone and far-end reference signals in the presence of latency. Following this alignment, adaptive filters, such as those based on the normalized least mean square (NLMS) [2] algorithm or its variants, were employed to estimate and cancel echo [3]. These methods, referred to as linear AEC (LAEC), proved effective under stable linear conditions. However, they struggled to perform adequately when echo paths were time-varying or non-linear, leading to either leakage of echo or suppression of the near-end talker [4]. In recent years, hybrid systems that combine traditional methods with neural networks (NN) have been widely explored [5-7]. In such systems, LAEC is typically employed for preliminary echo cancellation, with NN subsequently addressing residual echo suppression. In [5], a multi-filter LAEC followed by a gated complex convolutional recurrent neural network is explored for AEC task. Alternatively, MTFAA-Net [7] implements a hybrid AEC system by integrating TDE, LAEC, multi-scale NN modeling, and an attention mechanism, achieving superior performance. Recently, some efforts have been made to develop end-toend (E2E) NN methods that bypass TDE and LAEC [8-10]. A major challenge for these E2E approaches is the absence of TDE algorithms, which may lead to severe performance degradation in scenarios with large time delays [11], requiring the model to learn alignment internally. In [8], researchers propose cross-attention as an alternative to TDE and employ a convolutional recurrent network to suppress echo, eliminating the need for LAEC. DeepVQE [9] is another E2E method that employs a distinct attention mechanism to explicitly align microphone and reference signals, while also optimizing other network modules to achieve outstanding performance. Despite these advancements, hybrid systems remain predominant in industrial applications due to their robustness and reliability. To further advance E2E methods toward realworld deployment, we propose a novel neural network-based E2E-AEC framework. This framework integrates an optimized progressive learning (PL) [12] to jointly tackle noise reduction and echo cancellation. Our model is initialized using a pre-trained LAEC-based NN, enabling effective knowledge transfer from a hybrid AEC system. In addition, we employ an improved attention mechanism guided by loss functions for precise time alignment. Finally, we utilize a loss function for near-end speech voice activity detection (VAD), which could enhance speech quality while enabling strategic masking of far-end echo during near-end silence. Overall, this research highlights the potential of E2E approaches and contributes to advancing this challenging field. 2 METHODS 2.1 Problem Formulation In the field of AEC, the microphone signal in time domain is typically modeled as: y(n)=r(n)*h_{r}(n)+x(n)*h_{x}(n)+v(n) (1) where * denotes the convolution operation, n indexes time samples, h_{r}(n) and h_{x}(n) are acoustic transfer functions [13], r(n),x(n) and v(n) are the far-end reference signal, near-end target speech and additive background noise, respectively. Given observation y(n) and reference r(n) , the AEC task focuses on suppressing the echo component r(n)*h_{r}(n) . In this work, we employ an E2E-AEC model to not only suppress echo, but also mitigate reverberation and background noise, thereby recovering the anechoic target speech x(n) . 2.2 Overall Network Architecture Figure 1 shows the architecture of the proposed E2E-AEC model. The network takes short-time Fourier transform (STFT) features as input and output, with a frame length of 20 ms and a frame shift of 10 ms . Each fundamental recurrent neural network (RNN) [14] block consists of two GRU layers [15], following the TF-GridNet design [16] to capture full-band and sub-band dependencies. The GRU is uni-directional to support streaming inference. The hidden dimension is set to 64 , the kernel size and stride size in the unfold operation are 4 and 1 , respectively. The input features are encoded by RNN blocks, followed by an attention mechanism that generates aligned ref, with a supervised loss imposed on the attention weights. The aligned ref and mic features are then concatenated and passed through 8 RNN blocks. The network outputs complex convolving masks [9] in the middle and final layers, applied to the mic spectrum to recover the stage-specific targets for PL training. Additionally, a near-end speech VAD prediction is derived from the middle layer. The model has 1.2 M parameters, and as the innovation of this work does not lie in the model structure, our approach can generalize to networks with more or fewer parameters. 2.3 Progressive Enhancement Progressive learning (PL) [12] divides the model into multiple stages, where the training target for each stage has a higher signal-to-noise ratio (SNR) than the previous one. This design ensures that the later stage focuses on reconstructing signals with better quality, guiding the model toward incremental enhancements in speech quality. Figure 1: Figure 1. E2E-AEC system overview. The outputs include: delay (time delay estimation), vad (near-end speech VAD), and spec (spectrum estimations of different PL stages). In AEC scenarios, we modify the PL framework such that stage targets differ by signal components instead of SNR (see Figure 1). The target of the first stage is near-end speech mixed with background noise and reverberation but without echo, which enables the network to focus exclusively on removing dominant echo. The target of the second stage is nearend speech only, refining the output by eliminating residual echo, background noise, and reverberation. By substantially reducing echo in the first stage, the second stage lightens the burden of echo removal, enabling more effective suppression of noise and reverberation. After PL training, the final predicted signal is produced from the last stage. 2.4 Knowledge Transfer Hybrid AEC systems utilize TDE and LAEC for time alignment and initial echo removal, reducing the burden on subsequent modules and improving overall performance [17]. To enable our E2E-AEC model to leverage this, transfer learning can be implemented through two approaches: (1) a teacherstudent framework [18, 19], where a pre-trained hybrid system guides the E2E model, and (2) parameter initialization, which uses the parameters of a pre-trained hybrid model to initialize the E2E-AEC model. Both strategies provide a mechanism for transferring relevant knowledge learned by the hybrid system to the E2E model. In this work, we adopt the