Comparative evaluation of OpenAI O1 and human performance in higher order cognition

Ehsan Latif, Yifan Zhou, Shuchen Guo, Yizhu Gao, Lehong Shi, Matthew Nyaaba, Arne Bewerdorff, Xiantong Yang, Xiaoming Zhai · 2025

Read Comparative evaluation of OpenAI O1 and human performance in higher order cognition by Ehsan Latif, Yifan Zhou, Shuchen Guo, Yizhu Gao, Lehong Shi, Matthew Nyaaba, Arne Bewerdorff, Xiantong Yang, Xiaoming Zhai (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-12847924.txt.

Description

Can an AI model outthink people on hard mental tasks? Ehsan Latif led a team that pitted OpenAI's o1-preview model against human test-takers across seven kinds of higher-order thinking, from critical and creative thinking to logic and data skills. They used standard tests with published human scores at several levels of schooling.

The model beat undergraduates and postgraduates on a critical thinking essay test and scored far above humans on a systems thinking task. It also led on data skills, originality in creative tasks, logic questions, and a near-perfect run on a science reasoning test. Its weak spots were open-ended problem solving and adapting on the fly. The team sees promise for teaching, with care over ethics and oversight.

Catalog record

CREATOREhsan Latif, Yifan Zhou, Shuchen Guo, Yizhu Gao, Lehong Shi, Matthew Nyaaba, Arne Bewerdorff, Xiantong Yang, Xiaoming Zhai
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE85 KB
VERSION1.0
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC12847924

Browse the full library · About the Tower of Babel