Structure-Aware Masking for Protein Representation Learning

Thomas A. Walton, Ayan Goel, Amirali Aghazadeh · 2026

Read Structure-Aware Masking for Protein Representation Learning by Thomas A. Walton, Ayan Goel, Amirali Aghazadeh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-16581.txt.

Description

Walton, Goel, and Aghazadeh introduce Bucket Masking in a 2026 paper, a structure-aware masking strategy for protein language models. Standard training masks single residues at random. Bucket Masking instead masks groups of residues close together in three-dimensional space. That steers training toward long-range contacts that govern protein function. Across four protein fitness prediction tasks it improves on standard random masking by up to 14%. It excels at higher-order mutational interactions, and ablations credit mask placement rather than span size.

Catalog record

CREATORThomas A. Walton, Ayan Goel, Amirali Aghazadeh
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45532
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.16581

Browse the full library · About the Tower of Babel