IM-BERT: Enhancing Robustness of BERT through the Implicit Euler Method

Mihyeon Kim 1 Juhyoung Park 2 Youngbin Kim 1, 3 1 Department of Artificial Intelligence, Chung-Ang University 2 School of Computer Science and Engineering, Chung-Ang University 3 Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University {mh10967, wngud51, ybkim85}.cau.a · 2025

Read IM-BERT: Enhancing Robustness of BERT through the Implicit Euler Method by Mihyeon Kim 1 Juhyoung Park 2 Youngbin Kim 1, 3 1 Department of Artificial Intelligence, Chung-Ang University 2 School of Computer Science and Engineering, Chung-Ang University 3 Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University {mh10967, wngud51, ybkim85}.cau.a (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-06889.txt.

Description

Mihyeon Kim, Juhyoung Park, and Youngbin Kim present a way to make a large language model called BERT harder to fool. Such models do well on many word tasks, but small tricks in the input can break them, and tuning them on small data sets can make them brittle. The team views each layer of the model as a step in a math process over time.

They compare two ways to take those steps and show that one way stays more stable when the input is slightly disturbed. Their new link between layers uses that stable way. Tests show stronger defense against trick inputs, even when training data is scarce. The method adds no extra parts that need training.

Catalog record

CREATORMihyeon Kim 1 Juhyoung Park 2 Youngbin Kim 1, 3 1 Department of Artificial Intelligence, Chung-Ang University 2 School of Computer Science and Engineering, Chung-Ang University 3 Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University {mh10967, wngud51, ybkim85}.cau.a
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-06889

Browse the full library · About the Tower of Babel