Perish or Flourish? A Holistic Evaluation of Large Language Models for Code Generation in Functional Programming
Nguyet-Anh H. Lang, Eric Lang, Thanh Le-Cong, Bach Le, Quyet-Thang Huynh · 2026
Read Perish or Flourish? A Holistic Evaluation of Large Language Models for Code Generation in Functional Programming by Nguyet-Anh H. Lang, Eric Lang, Thanh Le-Cong, Bach Le, Quyet-Thang Huynh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-02060.txt.
Description
Functional programming promises reliable software but its steep learning curve keeps adoption low, and code-generating language models have mostly been judged on imperative languages. Nguyet-Anh Lang, Eric Lang, Thanh Le-Cong, Bach Le, and Quyet-Thang Huynh introduce FPEval, built on FPBench, a benchmark of seven hundred twenty-one tasks at three difficulty levels in Haskell, OCaml, and Scala.
The framework checks generated code with full test suites and static analysis for style and maintainability, and evaluates models including GPT-3.5, GPT-4o, and GPT-5 against a Java baseline. Newer models improved substantially, yet error rates stayed far higher in Haskell and OCaml than in Scala or Java, and models often wrote imperative-style code in functional dress. Given static analysis feedback, models could partly repair both correctness and quality problems.
Catalog record
| CREATOR | Nguyet-Anh H. Lang, Eric Lang, Thanh Le-Cong, Bach Le, Quyet-Thang Huynh |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-02060