Numerical Analysis of HiPPO-LegS ODE for Deep State Space Models
Jaesung R. Park Jaewook J. Suh Youngjoon Hong Ernest K. Ryu Mathematical Sciences CMOR Mathematical Sciences Mathematics Seoul National University Rice University Seoul National University UCLA [email protected] [email protected] [email protected] [email protected] · 2024
Read Numerical Analysis of HiPPO-LegS ODE for Deep State Space Models by Jaesung R. Park Jaewook J. Suh Youngjoon Hong Ernest K. Ryu Mathematical Sciences CMOR Mathematical Sciences Mathematics Seoul National University Rice University Seoul National University UCLA [email protected] [email protected] [email protected] [email protected] (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-08595.txt.
Description
Jaesung Park, Jaewook Suh, Youngjoon Hong, and Ernest Ryu supply the missing math base for a memory unit used in deep state space models. Those models use HiPPO memory units to catch long-range links in long sequences. A HiPPO unit tracks the past of a signal through an ordinary differential equation. The singular Legendre-scaled form works well in tests. But its well-posedness and its step-by-step theory were unsettled.
The team shows that the HiPPO-LegS equation is well-posed in spite of its singularity. It lacks the freedom of any start values, though. They also prove that the numeric schemes tied to it converge for Riemann integrable input functions. The work matters since state space models are only as sound as the equations under them. With the base settled, users can reason about accuracy and stability.
Catalog record
| CREATOR | Jaesung R. Park Jaewook J. Suh Youngjoon Hong Ernest K. Ryu Mathematical Sciences CMOR Mathematical Sciences Mathematics Seoul National University Rice University Seoul National University UCLA [email protected] [email protected] [email protected] [email protected] |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 75 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-08595