Numerical Analysis of HiPPO-LegS ODE for Deep State Space Models

Jaesung R. Park Jaewook J. Suh Youngjoon Hong Ernest K. Ryu Mathematical Sciences CMOR Mathematical Sciences Mathematics Seoul National University Rice University Seoul National University UCLA [email protected] [email protected] [email protected] [email protected] · 2024

Read Numerical Analysis of HiPPO-LegS ODE for Deep State Space Models by Jaesung R. Park Jaewook J. Suh Youngjoon Hong Ernest K. Ryu Mathematical Sciences CMOR Mathematical Sciences Mathematics Seoul National University Rice University Seoul National University UCLA [email protected] [email protected] [email protected] [email protected] (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-08595.txt.

Description

Jaesung Park, Jaewook Suh, Youngjoon Hong, and Ernest Ryu supply the missing math base for a memory unit used in deep state space models. Those models use HiPPO memory units to catch long-range links in long sequences. A HiPPO unit tracks the past of a signal through an ordinary differential equation. The singular Legendre-scaled form works well in tests. But its well-posedness and its step-by-step theory were unsettled.

The team shows that the HiPPO-LegS equation is well-posed in spite of its singularity. It lacks the freedom of any start values, though. They also prove that the numeric schemes tied to it converge for Riemann integrable input functions. The work matters since state space models are only as sound as the equations under them. With the base settled, users can reason about accuracy and stability.

Catalog record

CREATORJaesung R. Park Jaewook J. Suh Youngjoon Hong Ernest K. Ryu Mathematical Sciences CMOR Mathematical Sciences Mathematics Seoul National University Rice University Seoul National University UCLA [email protected] [email protected] [email protected] [email protected]
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE75 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-08595

Browse the full library · About the Tower of Babel