Benchmarking large language models for clinical data extraction from Portuguese medical notes in a university hospital

Breno Gabriel Araújo Sampaio de Jesus, Tomaz Castrillon Figueiredo, Clariele de Almeida Pereira, Bruno Henrique Lira Anjos, Ney Cristian Amaral Boa-Sorte, André Rodrigues Durães, Carlos Brites, Eduardo Martins, Leandro Maciel Almeida, Cleber Zanchettin, Vinícius de Oliveira Menezes, Marcos Antônio Dórea Machado · 2026

Read Benchmarking large language models for clinical data extraction from Portuguese medical notes in a university hospital by Breno Gabriel Araújo Sampaio de Jesus, Tomaz Castrillon Figueiredo, Clariele de Almeida Pereira, Bruno Henrique Lira Anjos, Ney Cristian Amaral Boa-Sorte, André Rodrigues Durães, Carlos Brites, Eduardo Martins, Leandro Maciel Almeida, Cleber Zanchettin, Vinícius de Oliveira Menezes, Marcos Antônio Dórea Machado (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/PMC13431991.txt.

Description

Jesus and colleagues present a 2026 benchmark of large language models extracting structured data from Portuguese medical notes at a university hospital. Free-text records in non-English settings are doubly hard: language and scarce computing. Five models, from GPT-4o mini to open models run in compressed form, were tested against hand-curated cardiology and infectious disease records.

Qwen-32B performed best in both domains, reaching balanced accuracy around nine tenths, with DeepSeek and others trailing by variable type. Some clinical variables extracted cleanly, while others, likely those needing context across the note, lagged for every model. The study's practical point is that a strong open model running locally can approach commercial performance, which matters for hospitals that cannot send patient text to foreign clouds. Benchmarks in the local language, not English leaderboards, should drive those choices.

Catalog record

CREATORBreno Gabriel Araújo Sampaio de Jesus, Tomaz Castrillon Figueiredo, Clariele de Almeida Pereira, Bruno Henrique Lira Anjos, Ney Cristian Amaral Boa-Sorte, André Rodrigues Durães, Carlos Brites, Eduardo Martins, Leandro Maciel Almeida, Cleber Zanchettin, Vinícius de Oliveira Menezes, Marcos Antônio Dórea Machado
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
LICENSEhttps://creativecommons.org/licenses/by/4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/article/MED/PMC13431991

Browse the full library · About the Tower of Babel