Manner implicatures in large language models

Description

Yan Cong asks whether large language models grasp a subtle part of human talk, in a 2024 study of manner implicature. People often mean more than they say, and hearers infer extra meaning when a speaker says a thing in an odd, roundabout way. Cong tests whether trained models can tell apart shades such as plain sameness, one claim following from another, and a hinted extra sense.

Three checks were used: word surprise scores, maps of word meaning, and plain prompts. Some models did better than chance at broad splits, such as telling a neutral link from a real link. None showed firm, steady skill at the finer splits, and the map method did little better than an older word method. Cong concludes that standard training does not yet give models a strong grasp of this kind of meaning.

Catalog record

Source: https://europepmc.org/articles/PMC11586417