Enabling Long FFT Convolutions on Memory-Constrained FPGAs via Chunking
Peter Wang, Neelesh Gupta, Viktor K. Prasanna · 2026
Read Enabling Long FFT Convolutions on Memory-Constrained FPGAs via Chunking by Peter Wang, Neelesh Gupta, Viktor K. Prasanna (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-06065.txt.
Introduction
Long convolutions let some neural networks handle very long inputs, but the memory they need exceeds what small chips hold. Peter Wang and 2 co-authors present a fix in a 2026 paper on field-programmable gate arrays, chips whose circuits can be reconfigured. Their setting is the Hyena architecture, which uses fast Fourier transform convolutions.
Their method splits the work into chunks and rebuilds the result with an overlap-add step. On an Alveo U200 chip with very limited block memory, they ran convolutions where both sequence and filter reached about 450,000 in length. Throughput grew with chunk size, and the longest runs lost only a small share of speed. Careful memory management made long-context work fit on edge hardware.
Catalog record
| CREATOR | Peter Wang, Neelesh Gupta, Viktor K. Prasanna |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 8 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-06065
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Enabling Long FFT Convolutions on Memory-Constrained FPGAs via Chunking
Peter Wang, Neelesh Gupta, Viktor K. Prasanna
Abstract
The need for long-context reasoning has led to alternative neural network architectures besides Transformers and self-attention, a popular model being Hyena, which employs causal 1D-convolutions implemented with FFTs. Long convolutions enable efficient global context mixing, but requirements for intermediate results exceed the 2-3 MB Block RAM capacity of FPGAs. We present a chunked FFT convolution approach enabling 450K length sequence by 450K length filter convolutions on an Alveo U200 FPGA with 2.8 MB BRAM through chunking and overlap-add reconstruction. We find that throughput scales proportionally with chunk size while degrading minimally by 7% for our longest sequences, demonstrating that careful memory management enables deployment of long-context primitives on edge FPGAs without sacrificing performance.
I
Introduction
Advances in sequence modeling have led to convolution-based alternatives to attention mechanisms. Models like Hyena
[
5
]
show that long convolutions can match Transformer performance while scaling sub-quadratically with sequence length.
HyenaDNA
[
4
]
applies this architecture to genomic foundation models, where capturing long-range interactions requires processing sequences up to 1M context length at single nucleotide resolution, necessitating efficient long convolution.
Fig. 1:
HyenaDNA architecture
While state-of-the-art implementations like FlashFFTConv
[
2
]
have achieved impressive speedups on high-end GPUs, these are impractical for edge deployment cases with constrained power and memory. FPGAs are an attractive alternative through their reconfigurable architecture and superior power efficiency. However, FPGA implementations face the
challenge of limited on-chip memory, typically a few MB of Block RAM (BRAM) compared to gigabytes of GPU VRAM.
This work addresses the FPGA memory constraint through
chunked FFT convolution
: decomposing large convolutions into chunks that fit within BRAM constraints, processing each chunk independently, and reconstructing the final result via overlap-add. Our contributions are as follows:
•
We implement a chunked FFT convolution algorithm enabling up to 450K sequence length by 450K filter length 1D-convolutions on FPGAs with 2.8 MB BRAM.
•
We analyze end-to-end performance, finding that FFT convolution increasingly dominates execution time, from 27% for our shortest sequence to 35% for our longest.
•
We evaluate performance across chunk sizes and sequence lengths, finding that throughput depends on chunk size and that it degrades by about 7% as sequence length increases from 32K to 450K.
II
Background
II-A
FFT-Based Convolution Latency in Hyena
We tested FFT-Convolution and the Hyena Block on GPU. Latency for convolution compared to the whole block increases from about 27% for the smallest sequence length to 35% for the longest, demonstrating that FFT-Convolution dominates in longer contexts, making it the main challenge.
While existing libraries like cuFFT have been previously shown to complete an 8M point FFT in about 0.2 seconds on an A100 (300W TDP, 1.4 GHz clock rate), we aim to eventually enable up to 1M context FFT-Convolution on an FPGA U200 (200W TDP, 300MHz clock rate).
II-B
FPGA Memory
Block RAM (BRAM) provides uniform low-latency random access, making it ideal for portions (such as FFT butterfly) that exhibit data-dependent, non-sequential access patterns with poor spatial locality. The constraint is the limited capacity; the Xilinx Alveo U200 which we use offers 2.8 MB BRAM.
II-C
Problem Definition
For input
x[n]
of length
N_{x}
and filter
h[n]
of length
N_{h}
, direct convolution requires
O(N_{x}\cdot N_{h})
operations, which is prohibitive for long sequences where the filter length matches input length, as in Hyena.
Our objective is computing the convolution
y[n]
of length
N_{y}=N_{x}+N_{h}-1
on FPGA using the FFT-based method instead, which runs in
O(N\log N)
.
y[n]=\mathcal{F}^{-1}\{\mathcal{F}\{x[n]\}\cdot\mathcal{F}\{h[n]\}\}
(1)
III
Proposed Methodology
We decompose the problem into (1) a radix-2 FFT kernel optimized for on-chip memory constraints and (2) a host-side layer managing data-transfer and overlap-add reconstruction.
III-A
FFT Kernel Design
We implement the Cooley-Tukey radix-2 decimation-in-time FFT algorithm on an Alveo U200 FPGA with Vitis HLS. The kernel processes chunks up to 8,192 points, determined experimentally. We adopt this strategy to enable longer-context sequence modeling primitives at lower energy cost.
Fig. 2:
Our CPU-host and FPGA-kernel implementation
III-B
Host Side Chunked Convolution Strategy
We partition the input sequence
x[N_{x}]
and filter
h[N_{h}]
into:
\displaystyle x[n]
\displaystyle\rightarrow\{x_{0}[n],x_{1}[n],\ldots,x_{M_{x}-1}[n]\}
(2)
\displaystyle h[n]
\displaystyle\rightarrow\{h_{0}[n],h_{1}[n],\ldots,h_{M_{h}-1}[n]\}
(3)
with
M_{x}=\lceil N_{x}/C\rceil
and
M_{h}=\lceil N_{h}/C\rceil
for chunk size
C
.
Each chunk pair is multiplied in the frequency domain, and then
the final output is reconstructed via overlap-add:
y[n]=\sum_{i=0}^{M_{x}-1}\sum_{j=0}^{M_{h}-1}y_{i,j}[n-iC-jC]
(4)
IV
Preliminary Results
We test on sequences from the hg38 human
reference genome with filters of equal length generated in each case. Sequence length of our tests follows those in HyenaDNA.
IV-A
Performance Results
Table I shows execution time across chunk sizes and sequence lengths.
TABLE I:
Execution Time (seconds) for Different Chunk Sizes
Dataset Size
8K Chunk
4K Chunk
2k Chunk
450K by 450K
47
188
750
160K by 160K
6
27
96
32K by 32K
1
1
3
Table II shows throughput increasing with chunk size.
In a given chunk size, throughput is highest among smaller inputs, with an average degradation of about 7% from smallest to largest inputs. Bandwidth for all tests was about 0.8 GB/s.
TABLE II:
Throughput (MFLOPS) by chunk size and dataset size
Dataset Size
8K Chunk
4K Chunk
2k Chunk
450K by 450K
104
46
22
160K by 160K
109
48
24
32K by 32K
110
50
25
IV-B
Compute vs. Transfer Analysis
Table
III
decomposes execution time into kernel lauch and execution, data transfer, and CPU processing.
TABLE III:
Time Breakdown: Compute / H2D+D2H Transfer / CPU (%)
Dataset Size
8K Chunk
4K Chunk
2K Chunk
450K
97.9 / 1.4 / 0.6
98.2 / 1.4 / 0.4
98.4 / 1.3 / 0.2
160K
97.9 / 1.4 / 0.6
98.2 / 1.4 / 0.3
98.2 / 1.5 / 0.2
32K
98.0 / 0.8 / 0.6
98.3 / 1.3 / 0.3
98.3 / 1.4 / 0.2
IV-C
Performance Comparison
Recent FPGA-based implementations have achieved high throughput for standalone FFT operations. Li et al.
[
3
]
achieved 94 MSamples/s on the Virtex4 with their configurable architecture for arbitrary-length FFTs, while Bai et al.
[
1
]
achieved up to 400 MSamples/s for 1024-point FFT operations on the Alveo U200 at 280 MHz operating frequency compared to our maximum of about 10 MSamples/s. While we are working with long convolutions with more overhead, these numbers and our results showing compute dominating our workflow indicate room for optimization in our approach.
V
Conclusion
We presented an approach that enables long FFT convolutions on FPGAs, showing that they serve as viable accelerators for memory-intensive deep learning operations. Employing chunked convolution, we processed up to 450K length sequence by 450K length filter convolutions on FPGA hardware with limited BRAM. Results show compute dominating performance and throughput scaling proportionally with chunk size, while maintaining performance across workloads.