TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Law

Zheng Hui, Yijiang River Dong, Ehsan Shareghi, Nigel Collier · 2025

Read TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Law by Zheng Hui, Yijiang River Dong, Ehsan Shareghi, Nigel Collier (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-21134.txt.

Description

Language models are moving into law, finance, and medicine, where a wrong or unethical answer can do real harm. Hui, Dong, Shareghi, and Collier build a benchmark, Trident-Bench, to test model safety in these three fields. They first write down safety principles drawn from the ethics codes of doctors, lawyers, and financial advisers. The benchmark then probes whether models honor those principles in realistic professional situations.

Nineteen models were tested, both general-purpose and specialized. Strong general models met basic expectations, but specialized models often missed subtle ethical points in their own fields. The authors say skill in a domain does not guarantee safe behavior there. They release the benchmark as a shared tool for exposing such gaps. The work argues that domain-specific safety needs its own yardsticks, not just general ones.

Catalog record

CREATORZheng Hui, Yijiang River Dong, Ehsan Shareghi, Nigel Collier
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE75 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-21134

Browse the full library · About the Tower of Babel