EgoAdapt : Adaptive Multisensory Distillation and Policy Learning for Efficient Egocentric Perception

Sanjoy Chowdhury, Subrata Biswas, Sayan Nag, Tushar Nagarajan, Calvin Murdock, Ishwarya Ananthabhotla, Yijun Qian, Vamsi Krishna Ithapu, Dinesh Manocha, Ruohan Gao · 2025

Read EgoAdapt : Adaptive Multisensory Distillation and Policy Learning for Efficient Egocentric Perception by Sanjoy Chowdhury, Subrata Biswas, Sayan Nag, Tushar Nagarajan, Calvin Murdock, Ishwarya Ananthabhotla, Yijun Qian, Vamsi Krishna Ithapu, Dinesh Manocha, Ruohan Gao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-21080.txt.

Description

Chowdhury and co-authors build EgoAdapt, a system that makes first person perception models cheaper to run without losing much skill. Models for video from a wearer's view, mixing sight and sound, are heavy, and heavy models strain small devices. Their 2025 framework learns when to lean on a big teacher model and when a light path will do, guided by a policy module.

The policy adapts to each task's choices, so one framework serves action recognition, active speaker spotting, and behavior prediction. Across three datasets, the method cut compute by up to about eighty nine percent and parameters by up to about eighty two percent. Energy use fell as much as 9.6 fold. Accuracy stayed level with, and often beat, the best prior models.

Catalog record

CREATORSanjoy Chowdhury, Subrata Biswas, Sayan Nag, Tushar Nagarajan, Calvin Murdock, Ishwarya Ananthabhotla, Yijun Qian, Vamsi Krishna Ithapu, Dinesh Manocha, Ruohan Gao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-21080

Browse the full library · About the Tower of Babel