When Softmax Fails at the Top: Extreme - Value Corrections for InfoNCE

Melihcan Erol, Suat Evren, Oktay Ozel, Alexander Morgan, Jongha Jon Ryu, Lizhong Zheng · 2026

Read When Softmax Fails at the Top: Extreme - Value Corrections for InfoNCE by Melihcan Erol, Suat Evren, Oktay Ozel, Alexander Morgan, Jongha Jon Ryu, Lizhong Zheng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-00262.txt.

Description

Erol and co-authors fix a blind spot in the top pick of a key learn law. InfoNCE is the go-to law for contrast-style learning. Its soft-max form is not just a handy trick. It bakes in a guess on how the top match in a batch is picked.

That guess clashes with the norms of set codes in fresh work, they show. Their fix, WEINCE, blends soft-max scores with a short-fall fix at the edge. Batch stats drawn live feed the blend. No new train weights are added. Tests span five vision sets. Scores rise in a steady way on frozen trait checks. The work shows hard-case stats, read right, can lift contrast laws.

Catalog record

CREATORMelihcan Erol, Suat Evren, Oktay Ozel, Alexander Morgan, Jongha Jon Ryu, Lizhong Zheng
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE101 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.00262

Browse the full library · About the Tower of Babel