NEST: Nested Event Stream Transformer for Sequences of Multisets

Minghui Sun, Haoyu Gong, Xingyu You, Jillian Hurst, Benjamin Goldstein, Matthew Engelhard · 2026

Read NEST: Nested Event Stream Transformer for Sequences of Multisets by Minghui Sun, Haoyu Gong, Xingyu You, Jillian Hurst, Benjamin Goldstein, Matthew Engelhard (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-00520.txt.

Description

Minghui Sun and colleagues keep the layers of event streams intact instead of flattening multisets into one sequence. Flattening wastes compute on dense attention, learns false links within sets, and yields weak set-level vectors from pooling. Their Nested Event Stream Transformer, or NEST, keeps that hierarchy as a built-in bias, lifting both speed and representation quality. The team adds Masked Set Modeling, a way to strengthen set-level learning. Tests on real multiset sequence data show NEST tracking dynamics while lifting pretraining speed and downstream results.

Catalog record

CREATORMinghui Sun, Haoyu Gong, Xingyu You, Jillian Hurst, Benjamin Goldstein, Matthew Engelhard
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56405
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602.00520

Browse the full library · About the Tower of Babel