Perish or Flourish? A Holistic Evaluation of Large Language Models for Code Generation in Functional Programming

Nguyet-Anh H. Lang, Eric Lang, Thanh Le-Cong, Bach Le, Quyet-Thang Huynh · 2026

Read Perish or Flourish? A Holistic Evaluation of Large Language Models for Code Generation in Functional Programming by Nguyet-Anh H. Lang, Eric Lang, Thanh Le-Cong, Bach Le, Quyet-Thang Huynh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-02060.txt.

Description

Functional programming promises reliable software but its steep learning curve keeps adoption low, and code-generating language models have mostly been judged on imperative languages. Nguyet-Anh Lang, Eric Lang, Thanh Le-Cong, Bach Le, and Quyet-Thang Huynh introduce FPEval, built on FPBench, a benchmark of seven hundred twenty-one tasks at three difficulty levels in Haskell, OCaml, and Scala.

The framework checks generated code with full test suites and static analysis for style and maintainability, and evaluates models including GPT-3.5, GPT-4o, and GPT-5 against a Java baseline. Newer models improved substantially, yet error rates stayed far higher in Haskell and OCaml than in Scala or Java, and models often wrote imperative-style code in functional dress. Given static analysis feedback, models could partly repair both correctness and quality problems.

Catalog record

CREATORNguyet-Anh H. Lang, Eric Lang, Thanh Le-Cong, Bach Le, Quyet-Thang Huynh
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-02060

Browse the full library · About the Tower of Babel