NEST: Nested Event Stream Transformer for Sequences of Multisets
Minghui Sun, Haoyu Gong, Xingyu You, Jillian Hurst, Benjamin Goldstein, Matthew Engelhard · 2026
Read NEST: Nested Event Stream Transformer for Sequences of Multisets by Minghui Sun, Haoyu Gong, Xingyu You, Jillian Hurst, Benjamin Goldstein, Matthew Engelhard (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-00520.txt.
Description
Minghui Sun and colleagues keep the layers of event streams intact instead of flattening multisets into one sequence. Flattening wastes compute on dense attention, learns false links within sets, and yields weak set-level vectors from pooling. Their Nested Event Stream Transformer, or NEST, keeps that hierarchy as a built-in bias, lifting both speed and representation quality. The team adds Masked Set Modeling, a way to strengthen set-level learning. Tests on real multiset sequence data show NEST tracking dynamics while lifting pretraining speed and downstream results.
Catalog record
| CREATOR | Minghui Sun, Haoyu Gong, Xingyu You, Jillian Hurst, Benjamin Goldstein, Matthew Engelhard |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56405 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602.00520