Initialization and training of matrix product state probabilistic models

Xun Tang, Yuehaw Khoo, Lexing Ying · 2025

Read Initialization and training of matrix product state probabilistic models by Xun Tang, Yuehaw Khoo, Lexing Ying (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-06419.txt.

Introduction

Xun Tang, Yuehaw Khoo, and Lexing Ying fix a training failure in tensor-network models that mimic quantum states to represent probabilities, in a 2025 paper. Randomly started matrix product states, trained by plain gradient descent, miss the strong links between the ends of a ring-shaped chain. In one algorithm the trained model even behaves as if causes flow one way, ignoring cross-ring ties.

Two remedies are offered. A natural gradient method that respects the curved geometry of these tensors escapes the bad minima within a few steps. A warm start borrowed from a sketching algorithm also avoids the trap and converges fast. Experiments back both fixes.

Catalog record

CREATORXun Tang, Yuehaw Khoo, Lexing Ying
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-06419

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Initialization and training of matrix product state probabilistic models Xun Tang, Yuehaw Khoo, Lexing Ying Abstract Modeling probability distributions via the wave function of a quantum state is central to quantum-inspired generative modeling and quantum state tomography (QST). We investigate a common failure mode in training randomly initialized matrix product states (MPS) using gradient descent. The results show that the trained MPS models do not accurately predict the strong interactions between boundary sites in periodic spin chain models. In the case of the Born machine algorithm, we further identify a causality trap , where the trained MPS models resemble causal models that ignore the non-local correlations in the true distribution. We propose two complementary strategies to overcome the training failure—one through optimization and one through initialization. First, we develop a natural gradient descent (NGD) method, which approximately simulates the gradient flow on tensor manifolds and significantly enhances training efficiency. Numerical experiments show that NGD avoids local minima in both Born machines and in general MPS tomography. Remarkably, we show that NGD with line search can converge to the global minimum in only a few iterations. Second, for the BM algorithm, we introduce a warm-start initialization based on the TTNS-Sketch algorithm. We show that gradient descent under a warm initialization does not encounter the causality trap and admits rapid convergence to the ground truth. mathx”17 I Introduction Recent advances in generative modeling enable one to learn complex high-dimensional distributions [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 ] . For discrete distributions, tensor network (TN) architectures have emerged as a prevalent method for probabilistic modeling [ 9 , 10 , 11 , 12 ] . One prominent class of tensor network models takes the perspective of Born machine (BM) [ 9 ] . Under this setting, one takes a tensor network to represent a quantum state. The probabilistic function follows from the Born rule and is thus represented by the squared modulus of the quantum state. This work focuses on Born machines using matrix product states (MPS)—a one-dimensional tensor network with strong expressivity [ 13 ] . Qubits entangled with a specific arrangement of local quantum circuits can be exactly modeled by an MPS representation [ 14 ] . The BM algorithm minimizes the negative log-likelihood (NLL) over observed samples, fitting the squared MPS amplitude to empirical data. The BM formulation is a special case of MPS quantum state tomography [ 15 , 16 ] , with the key distinction that the BM algorithm fits MPS models against measurements made on the computational basis. Training of the BM algorithm can be done classically, involving only conventional numerical linear algebra. In contrast, explicitly training a variational quantum circuit to fit the given samples would involve automatic differentiation [ 17 , 18 , 19 ] on quantum hardware, which is more difficult due to the presence of noise in obtaining the gradient through measurement. The use of MPS is proven successful in minimizing the energy of a quantum system, a well-celebrated example being the density-matrix renormalization group (DMRG) algorithm [ 13 ] . However, unsupervised generative modeling with MPS is a nonconvex optimization setting with unique challenges. While there is extensive literature on training neural networks in such settings (e.g., [ 20 , 21 , 22 ] ), the performance of the MPS ansatz in general optimization tasks is less understood. In this work, we use numerical evidence to show that gradient descent (GD) can lead to training failures for nonconvex optimization tasks under the MPS ansatz, both for the BM algorithm and for MPS tomography in general. We demonstrate that a randomly initialized MPS model fails to converge to the global minimum using standard gradient descent. We show that even substantial over-parameterization does not enable the model to escape these minima. The trained model exhibits rank degeneracy and overlooks important non-local correlations, making inferences from such models questionable. In the BM setting, we characterize the failure mode as a “causality trap,” wherein training converges to a simplified causal model, a graphical model class with significantly less approximation power. In addition, the causality trap phenomenon also occurs in the DMRG-type training method considered in [ 9 ] . The observed local minima issues share some similarities with the barren plateau phenomenon in quantum machine learning [ 23 , 24 , 25 , 26 , 27 , 28 , 29 ] . However, we show the challenge facing an MPS model in this case is a mild local minimum problem typical for nonconvex optimization. This work proposes two training strategies to prevent such issues. First, we propose a natural gradient descent (NGD) method that performs optimization in the function space of high-dimensional tensors, rather than directly on parameters. Mathematically, the proposed training process is the discretization of a projected gradient flow in the space of quantum states. The proposal allows efficient convergence to the global minimum in both BM and MPS tomography. Second, for the BM algorithm setting, we propose a warm-start initialization protocol. We propose a warm-start procedure using the tensor tree network states via sketching (TTNS-Sketch) algorithm, which gives a consistent density estimator with sample complexity guarantees [ 11 ] . Our finding shows that a non-random initialization allows the gradient descent method to converge to the ground truth. The effect of adopting the proposed methods is shown in Figure 1 . For the BM algorithm, the choices of initialization are (i) random initialization and (ii) the proposed warm initialization based on TTNS-sketch. The choices of training methods include: (a) the GD method, (b) the 2-site DMRG method in [ 9 ] , and (c) the proposed NGD method. In terms of training method, Figure 1 shows that the NGD method can converge to the global minimum under random initialization, while both GD and 2-site DMRG encounter local minima issues. Moreover, in Figure 1 , the warm initialization is sufficiently close to the global minimum that no further training is needed. Therefore, one can see that the local minima issue in the Born machine can be addressed through either improved initialization or improved training methods. Figure 1: Illustration of NGD and warm initialization for the Born machine algorithm. One sees that the gradient descent method and the 2-site DMRG method do not converge to the optimal log-likelihood level. The ground truth model is a periodic ferromagnetic Ising model, and the experiment details are in Section II . The remainder of the paper is organized as follows: Section II presents the causality trap in Born machine settings; Section III reports local minima issues in MPS tomography; Section IV presents the NGD method for MPS optimization; Section V introduces the TTNS-Sketch warm initialization for the BM algorithm; and Section VI offers concluding remarks. (a) Cycle graph with n=16 (b) Line graph with n=16 Figure 2: Graphical representations of the underlying model p^{\star} (Fig. 2a ) and of the mis-specified model p^{\mathrm{causal}} (Fig. 2b ). II Causality trap in the Born machine algorithm II.1 Background in Born machine We begin with a brief introduction to maximum likelihood estimation and the Born machine (BM) algorithm. Suppose one is given an underlying distribution p^{\star} and a parameterized family of

Read the complete text