Activation Functions, Statistics and Learning of Higher-Order Interactions in Restricted Boltzmann Machines

G. di Sarra, Y. Roudi · 2026

Read Activation Functions, Statistics and Learning of Higher-Order Interactions in Restricted Boltzmann Machines by G. di Sarra, Y. Roudi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-19178.txt.

Description

Giovanni di Sarra and Yasser Roudi study how hidden unit shape and weight stats set what a restricted Boltzmann machine can learn. They test four common activation functions: linear, step, ReLU, and exponential, on binary visible units. In a group of machines with Gaussian weights, distributions with strong higher order links among visible nodes turn out rare and hard to learn. The exponential activation is the exception: it alone lets such distributions be learned. The work links single neuron nonlinearity to the reach of the whole model.

Catalog record

CREATORG. di Sarra, Y. Roudi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE83443
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.19178

Browse the full library · About the Tower of Babel