When Softmax Fails at the Top: Extreme - Value Corrections for InfoNCE
Melihcan Erol, Suat Evren, Oktay Ozel, Alexander Morgan, Jongha Jon Ryu, Lizhong Zheng · 2026
Read When Softmax Fails at the Top: Extreme - Value Corrections for InfoNCE by Melihcan Erol, Suat Evren, Oktay Ozel, Alexander Morgan, Jongha Jon Ryu, Lizhong Zheng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-00262.txt.
Description
Erol and co-authors fix a blind spot in the top pick of a key learn law. InfoNCE is the go-to law for contrast-style learning. Its soft-max form is not just a handy trick. It bakes in a guess on how the top match in a batch is picked.
That guess clashes with the norms of set codes in fresh work, they show. Their fix, WEINCE, blends soft-max scores with a short-fall fix at the edge. Batch stats drawn live feed the blend. No new train weights are added. Tests span five vision sets. Scores rise in a steady way on frozen trait checks. The work shows hard-case stats, read right, can lift contrast laws.
Catalog record
| CREATOR | Melihcan Erol, Suat Evren, Oktay Ozel, Alexander Morgan, Jongha Jon Ryu, Lizhong Zheng |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 101 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.00262