Assessing the Sensitivity and Alignment of FOL Closeness Metrics

Ramya Keerthy Thatikonda Wray Buntine Ehsan Shareghi · 2025

Read Assessing the Sensitivity and Alignment of FOL Closeness Metrics by Ramya Keerthy Thatikonda Wray Buntine Ehsan Shareghi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-08613.txt.

Description

Ramya Keerthy Thatikonda and co-authors grade the graders for machine written logic, in a 2025 paper. Tools now turn English claims into formal logic for solver programs, yet few checks tell a right formula from a wrong one. The authors test how well current scoring methods spot planted flaws.

They twisted true formulas in word and symbol ways and watched each score react. A word overlap score overreacted to surface changes. A graph score caught shape changes but missed some symbol swaps. A meaning based score best matched a strong model acting as judge. Mixing scores beat any single one. The study guides builders of logic checking tools.

Catalog record

CREATORRamya Keerthy Thatikonda Wray Buntine Ehsan Shareghi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-08613

Browse the full library · About the Tower of Babel