A rigorous Turing test: a foundation for evaluating artificial general intelligence

Sharon Temtsin, Diane Proudfoot, David Kaber, Christoph Bartneck · 2025

Read A rigorous Turing test: a foundation for evaluating artificial general intelligence by Sharon Temtsin, Diane Proudfoot, David Kaber, Christoph Bartneck (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-17629.txt.

Description

Sharon Temtsin, Diane Proudfoot, David Kaber, and Christoph Bartneck ran a strict Turing test on a large language model, reported in a 2025 paper. The test asks whether a machine can pass as human in typed chat. Many recent claims say models have passed, but the authors say none followed Turing's own setup with care. They ran his three-player game by the book.

As a benchmark, they also ran a man pretending to be a woman under the same rules. In the machine game, only one judge wrongly took the model for human. Both games needed more than five minutes of chat to judge well. The authors argue that short time limits explain earlier claims of success, and the test still matters.

Catalog record

CREATORSharon Temtsin, Diane Proudfoot, David Kaber, Christoph Bartneck
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-17629

Browse the full library · About the Tower of Babel