Structure-Aware Masking for Protein Representation Learning
Thomas A. Walton, Ayan Goel, Amirali Aghazadeh · 2026
Read Structure-Aware Masking for Protein Representation Learning by Thomas A. Walton, Ayan Goel, Amirali Aghazadeh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-16581.txt.
Description
Walton, Goel, and Aghazadeh introduce Bucket Masking in a 2026 paper, a structure-aware masking strategy for protein language models. Standard training masks single residues at random. Bucket Masking instead masks groups of residues close together in three-dimensional space. That steers training toward long-range contacts that govern protein function. Across four protein fitness prediction tasks it improves on standard random masking by up to 14%. It excels at higher-order mutational interactions, and ablations credit mask placement rather than span size.
Catalog record
| CREATOR | Thomas A. Walton, Ayan Goel, Amirali Aghazadeh |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45532 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.16581