RecFlash: Fast Recommendation System on In-Storage Computing with Frequency-Based Data Mapping This work is an extended version of a paper presented at the IEEE APCCAS 2025 [ 2 ] . Compared to the conference version, this manuscript includes online training support with adaptive remapping, extended hardware implementation details, and a broader evaluation on real datasets. Details are summarized in Appendix A . * Equal contribution. 4 Corresponding Author
Jangho Baik 2 * , Sunghyun Kim 2 * , Gisan Ji 2 , Wonbo Shim 3 , and Sungju Ryu 4 · 2026
Read RecFlash: Fast Recommendation System on In-Storage Computing with Frequency-Based Data Mapping This work is an extended version of a paper presented at the IEEE APCCAS 2025 [ 2 ] . Compared to the conference version, this manuscript includes online training support with adaptive remapping, extended hardware implementation details, and a broader evaluation on real datasets. Details are summarized in Appendix A . * Equal contribution. 4 Corresponding Author by Jangho Baik 2 * , Sunghyun Kim 2 * , Gisan Ji 2 , Wonbo Shim 3 , and Sungju Ryu 4 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-25338.txt.
Introduction
Baik and four colleagues put forth RecFlash, a fast recommender speed-up that remaps data for NAND flash in-storage computing. Recommender systems are widely used, but fast-growing user data makes live handling hard. Flash in-storage computing holds huge user data well. Unlike other neural uses, recommenders show spotty, random memory reads. Most data pulled to the page buffer goes unused, wasting bandwidth and slowing inference. RecFlash cuts latency and power use by up to 81 percent and 91.9 percent over past in-storage computing.
Catalog record
| CREATOR | Jangho Baik 2 * , Sunghyun Kim 2 * , Gisan Ji 2 , Wonbo Shim 3 , and Sungju Ryu 4 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51231 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.25338
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
RecFlash: Fast Recommendation System on In-Storage Computing with Frequency-Based Data Mapping This work is an extended version of a paper presented at the IEEE APCCAS 2025 [ 2 ] . Compared to the conference version, this manuscript includes online training support with adaptive remapping, extended hardware implementation details, and a broader evaluation on real datasets. Details are summarized in Appendix A . * Equal contribution. 4 Corresponding Author Jangho Baik 2 * , Sunghyun Kim 2 * , Gisan Ji 2 , Wonbo Shim 3 , and Sungju Ryu 4 Abstract Recommendation system has gained a large popularity for a variety of personalized suggestion tasks, but the ever-increasing number of user data makes real-time processing of recommendation systems difficult. NAND flash memory-based in-storage computing scheme can be one of favorable candidates among the various acceleration approaches because the flash memory typically has a larger memory capacity than the other memory types, so it can efficiently handle a large amount of user data for the recommendation inference services. However, different from other neural network applications where data is sequentially fetched from memory, the recommendation system shows the irregular random memory access pattern. Hence, most of the data loaded from the NAND flash array to the page buffer are not used, so a large portion of the internal bandwidth is underutilized, which degrades the performance on the inference acceleration of the recommendation tasks. In this paper, we propose RecFlash, a fast recommendation inference accelerator utilizing a data remapping algorithm with NAND flash-based in-storage computing (ISC). The experimental results show that our proposed method improves the latency and energy consumption by up to 81% and 91.9%, respectively, over the existing NAND flash-based ISC architecture. I Introduction Recommendation systems provide personalized suggestions by analyzing data such as streaming history, clicks, and social network interactions. Companies like Amazon, eBay, and Alibaba boost sales with relevant product recommendations, while platforms like YouTube, Netflix, and Meta enhance competitiveness with tailored videos and news feeds. Collaborative filtering [ 7 ] assumes that past experiences determine future preferences, while content-based filtering [ 22 ] analyzes correlations between items and user profiles. DNN-based recommendation systems can handle complex non-linear relationships between users and items, providing feature combinations beyond traditional methods. The DLRM [ 23 ] uses sparse-length sum (SLS) operations to convert categorical features into continuous ones. Embedding layers retrieve vectors from lookup tables (LUTs), perform pooling to encode user history, and process data through fully connected layers. However, DNN-based systems require significant computational resources. In typical data centers, over 80% of hardware is dedicated to personalized recommendation services. In Meta’s data centers, DLRM inferences are performed trillions of times daily [ 6 , 14 ] . While the embedding layer itself has low computational intensity, it demands a substantial memory footprint and shows random access patterns to the memory. To relieve the Von Neumann bottleneck between processor and memory, many DRAM-based near-memory computing approaches have been explored. TensorDIMM [ 15 ] introduced a near-memory processing method which increased the effective bandwidth for the embedding layers by placing the SLS units on the buffer chip in the DIMM. RecNMP [ 10 ] performed in-depth workload characterization which improves the cache hit ratio by exploiting temporal reuse patterns. By doing so, the RecNMP reduced the number of memory accesses for embedding lookup operations, and it obtained significant SLS speedup in the DLRM workload. SPACE [ 9 ] proposed the heterogeneous near-memory processing architecture. It utilizes the two types of hierarchies to balance the bandwidth between the HBM and DRAM, and it also exploited the locality of the partial sum for the embedding layers. RECROSS [ 21 ] improves the effective bandwidth by parallelizing the computation on multiple memory hierarchies including rank/bank-group/subarray-level parallelism. Using the proposed partitioning technique for the embedding table, the tiles are assigned to corresponding NMP-levels for the efficient parallel computing. Fig. 1: Memory access patterns on (a) general matrix multiplication and (b) embedding layer for recommendation system. Meanwhile, the data size for the personalized recommendation system has been exponentially growing. As embedding tables grow in size, the recommendation inference services often consume more than hundreds of gigabytes of the storage. For example, the recent Criteo Click Logs dataset is more than tera-byte level [ 4 ] , and it cannot be handled in the DRAM-level [ 28 , 25 ] . As a result, the processor must frequently access the lower level disk to obtain a large number of parameters, which leads to a large latency for the access to the disk. The NAND flash memory offers faster read/write speed and higher bandwidth compared to the traditional hard-disk-based storage solutions. To exploit the advantage of storage density with the NAND flash memory, several approaches to NAND flash-based in-storage computing (ISC) have been introduced. RecSSD [ 28 ] enhances the performance of SSD-based recommendation systems by processing embedding lookups in the SSD, while RM-SSD [ 25 ] uses a low-cost FPGA to offload the recommendation system to the SSD and optimize MLP layer processing. Meanwhile, during the read operation of the NAND flash memory, data stored in the array is first fetched to the page buffer. Afterward, the buffered data is sent to the outside memory chip. In recommendation inference tasks, embedding vectors range from 64 to 512B, while the page buffer size in commercial single-level cell (SLC) NAND flash is 4KB [ 25 ] , much larger than the embedding vector size. In the recent multi-level cell arrays (eg., MLC, TLC, QLC) [ 11 , 24 , 12 , 8 ] , the page buffer size increases (8KB-32KB). Note that different from other neural network workloads where data is sequentially loaded from memory, SLS operation in the embedding layer requires the random memory access pattern. Only a part of the data loaded to the page buffer from memory are used, and most of the data cannot be utilized, because embedding vectors are much smaller than the page buffer. As a result, most of the internal memory bandwidth in the NAND flash is underutilized, which degrades the system performance. To relieve such a limitation of the recommendation system inference in the NAND flash-based ISC, we maximize the effective internal memory bandwidth for the SLS computations by introducing a data remapping scheme and the page-wise cache for the embedding tables. The key observations of our work [ 2 ] are summarized as follows. 1. We address and analyze the bandwidth under-utilization issue in the NAND flash-based ISC on the personalized recommendation system inference tasks. 2. We present the access frequency-based data remapping scheme of the embedding vectors to improve the reusability of the data stored in the page buffer of the NAND flash memory. 3. We introduce a page-wise cache which maximizes the reusability of the embedding vectors with a page-wise least recently used (LRU) replacement policy. 4. Based on the observation above, we introduce the NAND flash-based ISC architecture called RecFlash which achieves fast and energy-efficient personalized recommendation inference acceleration. II Preliminaries II-A Random Data Access on Recommendation System Most of the conventional neural network layers such as matrix-matrix multiplication, matrix-vector multiplication, convolution, and attention typically show the sequential memory access pattern. For example, in the matrix-vector multiplication, the