Fully Distributed, Flexible Compositional Visual Representations via Soft Tensor Products

Bethia Sun, Maurice Pagnucco, Yang Song · 2024

Read Fully Distributed, Flexible Compositional Visual Representations via Soft Tensor Products by Bethia Sun, Maurice Pagnucco, Yang Song (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-04671.txt.

Introduction

Bethia Sun, Maurice Pagnucco, and Yang Song present a study from twenty twenty-four on how neural nets can hold ideas built from parts. A long debate asks if nets can match the neat build of symbols. Work on split traits gave nets a way to store parts, but in a stiff, list like form. That form clashes with the smooth spread of deep learning.

Their fix grows out of a tensor product code due to Smolensky. The soft form spreads the part build across the whole code. A matched autoencoder learns such codes by design. Tests on vision tasks beat split trait rivals in split quality, speed of learning, and skill with few samples.

Catalog record

CREATORBethia Sun, Maurice Pagnucco, Yang Song
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE193 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-04671

Browse the full library · About the Tower of Babel