Learning on hexagonal structures and Monge–Ampère operators
Noemie C. Combe · 2024
Read Learning on hexagonal structures and Monge–Ampère operators by Noemie C. Combe (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-04407.txt.
Introduction
Noemie Combe links the geometry of learning to a classic equation from shape and heat studies. Her 2024 paper works with data spaces that carry two flat views at once, a rich frame for studying how learning proceeds. Boltzmann spaces, tied to a famed physics name, serve as a lead case.
Such spaces turn out to carry the target operator in their structure, she proves. For Boltzmann machines, that operator steers the learning rule itself. Web-like charts on these spaces can be laid out flat, by a broad form of an old triangle result. Under set rules from field theory, learning rests locally on hexagons. The view offers a fresh shape-based way to define best learning.
Catalog record
| CREATOR | Noemie C. Combe |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-04407
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learning on hexagonal structures and Monge–Ampère operators Noemie C. Combe Abstract . Dually flat statistical manifolds provide a rich toolbox for investigations around the learning process. We prove that such manifolds are Monge-Ampère manifolds. Examples of such manifolds include the space of exponential probability distributions on finite sets and the Boltzmann manifolds. Our investigations of Boltzmann manifolds lead us to prove that Monge-Ampère operators control learning methods for Boltzmann machines. Using local trivial fibrations (webs) we demonstrate that on such manifolds the webs are parallelizable and can be constructed using a generalisation of Ceva’s theorem. Assuming that our domain satisfies certain axioms of 2D topological quantum field theory we show that locally the learning can be defined on hexagonal structures. This brings a new geometric perspective for defining the optimal learning process. Keywords: Classical differential geometry, affine differential geometry, Monge-Ampère equation, Web theory, Frobenius manifold, differential geometric aspects of statistical manifolds and information geometry MSC2020 : 53Axx, 53A15, 35J96, 14J33, 53A60, 53B12 1. Introduction A rich differential geometric approach can be used in information geometry to study topics important to statistics [ 4 , 7 , 6 , 16 , 13 , 14 ] , such as learning problems [ 10 , 17 , 1 , 22 , 25 ] . This rich subject includes neural networks, decision theory [ 11 ] , manifolds of probability distributions (on discrete or continuous sample spaces) and learning methods such as the Ackley–Hinton–Sejnowski method [ 1 ] . Consider a Boltzmann machine, for instance. It is a network of stochastic N neurons. In our setting, we define a neural network as a graph G=(V,E) with a set of vertices V being the neurons and the set of edges E , allowing the synaptic connections between the neurons. We assume that the number of vertices/edges is finite and take |V|=N . To each neuron one associates a finite set of values. In the simplest cases, neurons can take for instance values in the set \mathbb{F}_{2}=\{1,0\} , which corresponds to whether a given neuron is excited or not, respectively. The vector of values attached to the set of vertices {\bf x}=(x_{1},x_{2},\cdots,x_{N}) is the state of the Boltzmann machine. Given two vertices i and j connected by an edge e_{ij} there corresponds a real (positive) value w_{ij}\in\mathbb{R}^{+} which is the weight of the edge. We can encode the collection of weights in an adjacency matrix (w_{ij})_{1\leq i,j\leq N} . This is a symmetric matrix, where entries are the weights w_{ij} . Given that self-loops are not allowed in the graph (i.e. there do not exist edges starting and ending on the same vertex) we have automatically that w_{ii}=0 , implying that the symmetric weighted matrices are of null trace. The set of all such Boltzmann machines with fixed topology forms a geometric manifold, where (modifiable) synaptic weights of connections specify networks. Define the space of weight parameters (this is a subspace of an Euclidean space \mathbb{R}^{M} of symmetric matrices with trace equal to 0). let \mathscr{B} be the set of all probability distributions (given by Eq. 10 ) provided by the Boltzmann machines. The connection matrix W specifies each distribution uniquely. A Boltzmann manifold is therefore defined by \mathscr{B} and its coordinates are given by the w_{ij} . By [ 26 ] , any weight matrix W of trace 0 can be expressed as a commutator i.e. W=XY-YX=[X,Y] , where X,Y are matrices in \mathbb{R}^{N(N+1)/2} . Therefore, we can say that the space which parametrizes the manifold of probability distributions is a space of commutators. Note that the abelian case is, in some sense, pointless to consider since it corresponds to the case where there are no interactions occurring between the neurons. This commutator space parametrises a statistical manifold M of probability distributions of exponential type, defined over the state space and consisting of 2^{N} states {\bf x} . Each point on this manifold corresponds to a probability distribution. The procedure of learning corresponds —on both manifolds—to a curve. We shall consider the learning process, locally. In particular, we show that the learning curve can be done in an optimal way, by using optimal transport provided by a Monge–Ampere operator. This is possible due to the fact that on the Boltzmann neural network manifold there exists a natural invariant Riemannian metric and a dual pair of affine connections. By [ 16 , 15 ] this structure is related to a Monge–Ampere manifold. Therefore, in Theorem 3 we show that: Theorem 3 and Corollary 1 : Everywhere locally on a dually flat manifold M there exists an optimal learning procedure from one point to another, controlled by a pair of Monge–Ampere operators. Note that this statement is general (not only for discrete sample spaces). Having introduced the notion of “very optimal learning” in Definition 4 , we provide the following statement: Theorem 5 : Let \mathcal{D}\subset N be a domain lying in a totally geodesic submanifold N of a dually flat manifold \mathscr{S} . Suppose that \mathcal{D} satisfies additionally Eq. 1 . Then, \mathcal{D} is equipped with a hexagonal web. The learning on \mathcal{D} can thus be described using a local hexagonal lattice and it is very optimal. A flat totally geodesic satisfying the axioms of a Frobenius manifold has interesting geometric features. Frobenius manifolds are affine/Hessian manifolds such that the tangent sheaf carries the structure of a Frobenius algebra. A Frobenius algebra is a commutative, associative, unital algebra where the multiplication operation is invariant under the symmetric bilinear product. Rewriting the associativity of the multiplication operation \circ in the usual way as (\partial_{a}\circ\partial_{b})\circ\partial_{c}=\partial_{a}\circ(\partial_{b}\circ\partial_{c}) we obtain a non–linear system of Associativity Equations , partial differential equations for \Phi : (1) \forall a,b,c,d:\quad\sum_{ef}\Phi_{abe}g^{ef}\Phi_{fcd}=\sum_{ef}\Phi_{bce}g^{ef}\Phi_{fad}, where \Phi is a potential function and \partial_{a}=\frac{\partial}{\partial_{a}x_{a}} are flat vector fields. In the community of physicists, they are known as WDVV (Witten–Dijkgraaf–Verlinde–Verlinde) equations. We prove the existence of differential geometric webs of hexagonal type on these manifolds (see Thm. 5 ). Choosing the webs so that they form regular hexagons, this can enable us to provide our domain \mathcal{D} (everywhere locally) with a regular hexagonal/ honeycomb lattice. Hexagonal lattices simplify considerations around our statistical data greatly. In particular, if the hexagons are regular, one can use the (regular) lattice to consider the learning process locally and benefit from dihedral symmetries arising from the hexagons and group web properties. Acknowledgements This research is part of the project No. 2022/47/P/ST1/01177 co-founded by the National Science Centre and the European Union’s Horizon 2020 research and innovation program, under the Marie Sklodowska Curie grant agreement No. 945339 . 2. Networks and manifolds of probability distributions 2.1. Neural networks As stated in the introduction, we take as a starting point the example of some neural network coming from the Ising–Bolzman framework. Those networks are connected graphs without self-loops i.e. we omit the cases where edges are connected to only one vertex. The initial motivation for the study of manifolds of probability distributions is that one obtains a richer amount of information on the behavior of such neural networks if we study them not as individual objects but rather as a family of networks with fixed topology i.e. satisfying a given set of properties. This set of properties includes the