Toward F aithfulness-guided E nsemble I nterpretation of Neural Network

Siyu Zhang, Kenneth Mcmillan · 2025

Read Toward F aithfulness-guided E nsemble I nterpretation of Neural Network by Siyu Zhang, Kenneth Mcmillan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-04588.txt.

Description

Siyu Zhang and one co-author, Kenneth Mcmillan, argue that explanations of neural networks must preserve the network's actual computation, not just its output, in a 2025 research paper. Faithfulness scores usually insert or delete features and watch the answer change. The authors show rival pathways can game those scores: perturbations reroute computation through different feature detectors while the output looks unchanged. Activation preservation, keeping internal activity close to the original, serves as a workable stand-in for pathway preservation.

Their method, FEI, optimizes both external scores and internal closeness together, using ensemble quantile tuning and selective gradient clipping. On image benchmarks with VGG and ResNet models, it topped insertion and deletion scores while deviating far less inside.

Catalog record

CREATORSiyu Zhang, Kenneth Mcmillan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-04588

Browse the full library · About the Tower of Babel