FAME: F PGA A cceleration of Secure M atrix Multiplication with Homomorphic E ncryption
Zhihan Xu ∗ , Rajgopal Kannan † and Viktor K. Prasanna ∗ · 2025
Read FAME: F PGA A cceleration of Secure M atrix Multiplication with Homomorphic E ncryption by Zhihan Xu ∗ , Rajgopal Kannan † and Viktor K. Prasanna ∗ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-15515.txt.
Introduction
Zhihan Xu, Rajgopal Kannan, and Viktor Prasanna present FAME, an FPGA accelerator for matrix multiplication on encrypted data, in a 2025 paper. Homomorphic encryption lets cloud servers compute on data they cannot read, but its matrix work is very slow. Speeding that step up is key for private machine learning.
The team first models how much on chip memory a given setup needs, then designs a new datapath for the main bottleneck step that reuses data finely and cuts memory traffic. Built on an Alveo U280 chip, FAME handles many matrix shapes and parameter sets. It runs on average 221 times faster than leading CPU based versions in their tests.
Catalog record
| CREATOR | Zhihan Xu ∗ , Rajgopal Kannan † and Viktor K. Prasanna ∗ |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-15515
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FAME: F PGA A cceleration of Secure M atrix Multiplication with Homomorphic E ncryption Zhihan Xu ∗ , Rajgopal Kannan † and Viktor K. Prasanna ∗ Abstract Homomorphic Encryption (HE) enables secure computation on encrypted data, addressing privacy concerns in cloud computing. However, the high computational cost of HE operations, particularly matrix multiplication (MM), remains a major barrier to its practical deployment. Accelerating Homomorphic Encrypted MM (HE MM) is crucial for applications such as privacy-preserving machine learning. In this paper, we present a bandwidth-efficient FPGA implementation of HE MM. We first develop a cost model to evaluate the on-chip memory requirement for a given set of HE parameters and input matrix sizes. Our analysis shows that optimizing on-chip memory usage is critical for scalable and efficient HE MM. To this end, we design a novel datapath for Homomorphic Linear Transformation (HLT), the major bottleneck in HE MM. Our datapath significantly reduces off-chip memory traffic and on-chip memory demand by enabling fine-grained data reuse. Leveraging the proposed datapath, we introduce FAME, the first FPGA-based accelerator specifically tailored for HE MM. FAME supports arbitrary matrix shapes and is configurable for a wide range of HE parameter sets. We implement FAME on Alveo U280 and evaluate its performance over diverse matrix sizes and shapes. Experimental results show that FAME achieves an average of 221 × \times speedup over state-of-the-art CPU-based implementations, demonstrating its scalability and practicality for large-scale consecutive HE MM and real-world workloads. I Introduction Cloud computing has revolutionized many applications across various sectors such as healthcare, finance, and government [ 42 , 43 ] . Despite its computational advantages, security and data privacy remain concerns in adopting cloud computing services [ 4 ] . Homomorphic Encryption (HE) offers a promising solution by enabling computations directly on encrypted data without decrypting the data at the server side. However, HE introduces substantial computational overhead, often several orders of magnitude higher than equivalent plaintext operations [ 13 ] . As a result, accelerating HE applications has become a focal point of recent research efforts [ 37 , 14 , 50 , 40 , 49 , 47 , 38 , 46 ] . Existing HE acceleration efforts [ 37 , 14 , 50 , 40 , 49 , 47 , 38 , 46 ] primarily focus on scenarios where the machine learning (ML) model remains in plaintext and only the input data is encrypted. For example, works such as [ 47 , 38 , 46 ] accelerate matrix-vector multiplication where the matrix (i.e., the model) is unencrypted. While this setup reduces computational cost, it is not sufficient for applications requiring stronger privacy guarantees, where both the model and data must remain encrypted. Two representative scenarios illustrate this need: (1) a cloud service provider owns the model, trains the model with the encrypted data from the data owner, and uses the encrypted trained model to make inferences on new encrypted inputs; (2) a third-party model provider (e.g., researchers or companies) owns the trained encrypted model and uploads it to the cloud service provider to perform inference on encrypted inputs from various data owners. In both scenarios, both the model and the input data are encrypted, necessitating secure and efficient HE matrix multiplication (MM) to support fully encrypted inference workflows. HE MM is defined as matrix multiplication in which both input matrices are encrypted with homomorphic encryption. The result of HE MM is encrypted output matrix. HE MM is a fundamental kernel in many HE applications. Several algorithmic advancements [ 25 , 23 , 22 , 16 ] to reduce its computational complexity have been made. However, CPU-based implementations remain prohibitively slow, particularly for large matrix sizes. For example, multiplying two modestly sized matrices (e.g., 64\times 64 ) under a small HE parameter set can still take several tens of seconds [ 22 ] , which is approximately 1000 \times slower than the plaintext operation. This substantial performance gap highlights the need for hardware acceleration to make HE MM based applications practical. FPGAs offer a unique advantage for accelerating HE MM due to their configurable on-chip memory hierarchy and customizable compute resources, enabling fine-grained datapath control to enhance on-chip data reuse. In addition, leading FPGA vendors also incorporate High Bandwidth Memory (HBM) to overcome traditional DRAM bandwidth limitation, making them well-suited for handling the high-throughput demands of encrypted data transfers. The primary bottleneck in HE MM is homomorphic linear transformation (HLT), accounting for over 95% of the total runtime [ 25 ] . This inefficiency arises from two key challenges: (1) HLT requires multiple ciphertext rotations - one of the most computationally expensive operations in HE, and (2) it generates numerous intermediate ciphertexts during computation, leading to substantial off-chip memory traffic that limits the scalability of HE MM to larger matrices and HE parameter sets. To mitigate the first challenge, a pioneering work [ 17 ] introduced the hoisting technique to amortize certain sub-operations across multiple rotations, which has been widely adopted in subsequent works [ 10 , 18 , 13 , 44 ] . However, the second challenge remains unaddressed due to the lack of hardware-aware solutions. In this work, we address both challenges by designing a novel HLT datapath that integrates algorithmic optimizations to minimize off-chip memory traffic. Our key contributions are summarized as follows: • We develop a cost model to analyze the on-chip memory requirements of HE MM for a given matrix size, shape, and HE parameter set. The analysis shows that HE MM is not scalable without explicit datapath optimizations due to substantial off-chip ciphertext traffic. • We propose an on-chip memory-optimized datapath for HLT, the bottleneck in HE MM, through hardware-software co-design. The datapath significantly reduces off-chip memory traffic by fusing sub-operations and enabling fine-grained on-chip ciphertext reuse. • Leveraging this datapath, we design FAME, the first accelerator tailored for HE MM. FAME achieves resource efficiency through reduced on-chip memory usage. The architecture can be configured to support diverse matrix sizes, shapes, and HE parameter sets. • We implement FAME on Alveo U280. Experimental results show that FAME achieves 221\times speedup on average over state-of-the-art CPU implementations across diverse matrix sizes, shapes, and HE parameters, demonstrating its scalability and practicality for large-scale HE MM. II Background II-A Threat Model We assume a semi-honest (honest but curious) cloud service provider as previous works [ 36 , 27 , 34 , 46 ] . The client sends the encrypted input matrix to the cloud server to perform HE MM. Importantly, the other input matrix is also encrypted, whether it comes from the client, a third-party model provider, or the server itself. This threat model differs from prior HE acceleration works [ 46 , 38 , 47 ] , which typically assume only two parties (client and server) and a plaintext model. In contrast, our approach addresses scenarios with stronger privacy guarantees by ensuring both inputs remain encrypted throughout the computation. II-B Homomorphic Encryption (HE) Among various HE schemes [ 8 , 7 , 15 , 12 ] , we focus on the CKKS [ 12 ] , which supports real-number computations, making it suitable for HE machine learning (ML) applications. In CKKS, a message vector \bm{m} of N/2 numbers is first encoded into