ProtSent: Protein Sentence Transformers

Dan Ofer, Oriel Perets, Michal Linial, Nadav Rappoport · 2026

Read ProtSent: Protein Sentence Transformers by Dan Ofer, Oriel Perets, Michal Linial, Nadav Rappoport (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-06830.txt.

Description

Dan Ofer, Oriel Perets, Michal Linial, and Nadav Rappoport present ProtSent, a post-training framework that teaches protein language models to place related proteins near each other in embedding space. Standard protein embeddings come from averaging residue vectors never trained for retrieval geometry, so related proteins may sit far apart. ProtSent adds contrastive supervision from family membership, structural clusters, and interaction data, with mutation fitness ordering as an optional signal.

Applied to several ESM models, it sharply improves family recovery and retrieval from frozen embeddings, beating its bases and, at ranking depth, a sensitive alignment search. Under a trained linear probe, however, the tuned embeddings fall below the untouched backbone. Better neighborhoods thus trade against probe accuracy, a tension the paper documents honestly.

Catalog record

CREATORDan Ofer, Oriel Perets, Michal Linial, Nadav Rappoport
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.06830

Browse the full library · About the Tower of Babel