Optimization Performance of Factorization Machine with Annealing under Limited Training Data

MAYUMI NAKANO 1, YUYA SEKI 1, 2, SHUTA KIKUCHI 1, 2, AND SHU TANAKA 1, 2, 3, 4 · 2025

Read Optimization Performance of Factorization Machine with Annealing under Limited Training Data by MAYUMI NAKANO 1, YUYA SEKI 1, 2, SHUTA KIKUCHI 1, 2, AND SHU TANAKA 1, 2, 3, 4 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-21024.txt.

Introduction

Some search problems are black boxes: you can test an input and see the output, but the rule inside is unknown. Mayumi Nakano, Yuya Seki, Shuta Kikuchi, and Shu Tanaka worked on one method for such problems in 2025. The method, FMQA, learns a stand-in model from past tests and uses a special annealing machine to propose the next input to try.

The team found that as test data piles up, new points lose their influence on the stand-in model, and progress stalls. Their fix, SWIFT-FMQA, keeps only a sliding window of the most recent points for training. In numerical tests, the new method found lower-cost solutions with fewer black-box evaluations than the original.

Catalog record

CREATORMAYUMI NAKANO 1, YUYA SEKI 1, 2, SHUTA KIKUCHI 1, 2, AND SHU TANAKA 1, 2, 3, 4
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-21024

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Optimization Performance of Factorization Machine with Annealing under Limited Training Data MAYUMI NAKANO 1, YUYA SEKI 1, 2, SHUTA KIKUCHI 1, 2, AND SHU TANAKA 1, 2, 3, 4 Abstract Black-box (BB) optimization problems aim to identify an input that minimizes the output of a function (the BB function) whose input-output relationship is unknown. Factorization machine with annealing (FMA) is a promising approach to this task, employing a factorization machine (FM) as a surrogate model to iteratively guide the solution search via an Ising machine. Although FMA has demonstrated strong optimization performance across various applications, its performance often stagnates as the number of optimization iterations increases. One contributing factor to this stagnation is the growing number of data points in the dataset used to train FM. It is hypothesized that as more data points are accumulated, the contribution of newly added data points becomes diluted within the entire dataset, thereby reducing their impact on improving the prediction accuracy of FM. To address this issue, we propose a novel method for sequential dataset construction that retains at most a specified number of the most recently added data points. This strategy is designed to enhance the influence of newly added data points on the surrogate model. Numerical experiments demonstrate that the proposed FMA achieves lower-cost solutions with fewer BB function evaluations compared to the conventional FMA. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. † † history: Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000. † † doi: 10.1109/ACCESS.2024.0429000 † † address: Graduate School of Science and Technology, Keio University, Kanagawa 223-8522, Japan † † address: Keio University Sustainable Quantum Artificial Intelligence Center (KSQAIC), Keio University, Tokyo 108-8345, Japan † † address: Department of Applied Physics and Physico-Informatics, Keio University, 3-14-1 Hiyoshi, Kohoku-ku, Yokohama-shi, Kanagawa 223-8522, Japan † † address: Human Biology-Microbiome-Quantum Research Center (WPI-Bio2Q), Keio University, Tokyo 108-8345, Japan † † titlenote: This work was partially supported by the Japan Society for the Promotion of Science (JSPS) KAKENHI (Grant Numbers JP23H05447 and 25K07172), the Council for Science, Technology, and Innovation (CSTI) through the Cross-ministerial Strategic Innovation Promotion Program (SIP), “Promoting the application of advanced quantum technology platforms to social issues” (Funding agency: QST), Japan Science and Technology Agency (JST) (Grant Number JPMJPF2221). In addition, this paper is partially based on results obtained from a project, JPNP23003, commissioned by the New Energy and Industrial Technology Development Organization (NEDO). The computations in this work were partially performed using the facilities of the Supercomputer Center, the Institute for Solid State Physics, The University of Tokyo. M. N. would like to express his sincere gratitude to The SATOMI Scholarship Foundation for their financial support. † † corresponding: Corresponding author: Mayumi Nakano (e-mail: [email protected]). I Introduction Combinatorial optimization problems seek to search for a set of decision variables that satisfy all given constraints and minimize an objective function. Representative examples of combinatorial optimization problems include the traveling salesman problem, quadratic assignment problem, quadratic knapsack problem, and low-autocorrelation binary sequences problem (LABS). The problem size of combinatorial optimization problems is characterized by the number of decision variables. As the number of decision variables increases, the number of candidate solutions grows exponentially, leading to a combinatorial explosion, which makes exhaustive search for optimal solutions difficult. In this context, research on metaheuristics that can find high-quality solutions in a short time has progressed. Recently, Ising machines, which are hardware implementations of algorithms such as simulated annealing (SA) [ 1 , 2 , 3 ] and quantum annealing (QA) [ 4 , 5 , 6 , 7 ] , both being types of metaheuristics, have attracted significant attention [ 8 , 9 , 10 ] . Ising machines are capable of sampling for good solutions in a wide range of applications, including financial portfolio [ 11 , 12 ] , online advertisement [ 13 ] , traffic [ 14 , 15 , 16 , 17 , 18 , 19 ] , material simulation [ 20 , 21 , 22 , 23 ] , and computer-aided engineering [ 24 , 25 , 26 ] . In case of solving combinatorial optimization problems using an Ising machine, it is necessary to formulate the problem as an energy function of an Ising model or in the form of quadratic unconstrained binary optimization (QUBO). Various combinatorial optimization problems can be represented by these formulations [ 27 ] . Among combinatorial optimization problems, some are difficult to formulate as an Ising model or QUBO. A typical example is the black-box (BB) optimization problem. A characteristic of BB optimization problems is that the objective function is not explicitly given. This situation arises when obtaining the output value of the objective function requires conducting simulations or experiments. For example, applications include drug design with affinity for specific protein targets [ 28 ] , optimization of interfacial thermal conductivity in nanostructured materials [ 29 ] , and the design of convolutional neural network cell architectures [ 30 ] . The objective function in BB optimization problems is called a black-box (BB) function. In real-world problems such as those described above, calling the BB function incurs time and financial costs. Therefore, it is required to conduct BB optimization while minimizing the number of BB function calls. Under the drawback that objective function is not explicitly defined, Ising machines cannot solve these problems. Factorization machine with annealing (FMA) has been proposed to overcome this drawback [ 31 ] . FMA is a BB optimization method that repeats the following four steps: prediction of the BB function using a machine learning model called factorization machine (FM) [ 32 ] as a surrogate model, sampling solutions using an Ising machine, calling the BB function for evaluation, constructing the dataset for FM training. FMA solves the problem of implicitly given objective functions by approximately formulating the BB function in QUBO form using FM. Additionally, by utilizing FM as a surrogate model instead of directly using the BB function, FMA can reduce the number of BB function calls. Due to these characteristics, FMA is expected to be a useful method for BB optimization problems [ 33 ] . Actually, FMA has been reported to be capable of searching for lower-cost solutions than the conventional BB optimization methods such as Bayesian optimization of combinatorial structures [ 34 ] , genetic algorithms [ 35 ] , and particle swarm optimization [ 36 ] in applications such as printed circuit board layout design [ 37 ] , barrier material design [ 38 ] , and photonic crystal laser structural design [ 39 ] . FMA iteratively generates data points by using an Ising machine for FM training and they are added to the initial dataset. This increase in training data is expected to improve FM’s prediction accuracy, thereby leading to better solutions. Fig. 1: Illustration of flow of FMA including the proposed method. Unlike the conventional methods, the proposed method incorporates a process that reduces the number of data points in the dataset. Meanwhile, it has been reported that FMA tends to experience stagnation in solution improvement as the optimization process repeats [ 40

Read the complete text