Layer-Adaptive State Pruning for Deep State Space Models

Minseon Gwak, Seongrok Moon, Joohwan Ko, PooGyeon Park · 2024

Read Layer-Adaptive State Pruning for Deep State Space Models by Minseon Gwak, Seongrok Moon, Joohwan Ko, PooGyeon Park (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-02824.txt.

Description

A 2024 arXiv research paper by Minseon Gwak, Seongrok Moon, Joohwan Ko, PooGyeon Park. Due to the lack of state dimension optimization methods, deep state space models (SSMs) have sacrificed model capacity, training search space, or stability to alleviate computational costs caused by high state dimensions. In this work, we provide a structured pruning method for SSMs, L ayer- A daptive ST ate pruning (LAST), which reduces the state dimension of each layer in minimizing model-level output energy loss by extending modal truncation for a single system.

Catalog record

CREATORMinseon Gwak, Seongrok Moon, Joohwan Ko, PooGyeon Park
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE70 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2411-02824

Browse the full library · About the Tower of Babel