A scalable and unbiased discordance metric with H+
Nathan Dyjack, Daniel N Baker, Vladimir Braverman, Ben Langmead, Stephanie C Hicks · 2022
Read A scalable and unbiased discordance metric with H+ by Nathan Dyjack, Daniel N Baker, Vladimir Braverman, Ben Langmead, Stephanie C Hicks (2022) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-10724244.txt.
Description
Nathan Dyjack and co-authors fix a yardstick problem in clustering: how to score clusters when no true labels exist. Internal scores like tightness depend on the distance measure used, so results across measures cannot be compared fairly. An earlier scale-free discordance metric solved the comparison issue but ran slowly and skewed with group sizes. Their paper proposes a modified metric that removes both flaws.
The new measure stays stable whether clusters come out balanced or lopsided, and it computes fast enough for large data. Simulations and examples show the bias of the old metric and the steadiness of the replacement. Data scientists who must cluster without ground truth get a referee that does not favor one distance or one cluster shape.
Catalog record
| CREATOR | Nathan Dyjack, Daniel N Baker, Vladimir Braverman, Ben Langmead, Stephanie C Hicks |
|---|---|
| YEAR | 2022 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |