NLP Security and Ethics, in the Wild
Heather Lent, Erick Galinkin, Yiyi Chen, Jens Myrup Pedersen Leon Derczynski Johannes Bjerva · 2025
Read NLP Security and Ethics, in the Wild by Heather Lent, Erick Galinkin, Yiyi Chen, Jens Myrup Pedersen Leon Derczynski Johannes Bjerva (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-06669.txt.
Introduction
Heather Lent and five co-authors audited how research on language model security handles the ethics of its own work, in a paper published in 2025. The field tests how models can be tricked or attacked and builds defenses. Such work can protect users, but careless practice can expose privacy or help harmful models. The authors read recent papers in the field and scored them against the ethical norms of the security profession.
They found wide gaps. Many papers skipped harm checks and quiet notification to the owners of the flaws found, steps security work treats as basic. The team calls the gap a culture clash, because the field grew out of language study rather than security practice. They offer practical fixes, from harm plans to timed disclosure, and call the approach white hat NLP. Their aim is a culture where safety work does not itself cause harm.
Catalog record
| CREATOR | Heather Lent, Erick Galinkin, Yiyi Chen, Jens Myrup Pedersen Leon Derczynski Johannes Bjerva |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-06669
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
NLP Security and Ethics, in the Wild Heather Lent, Erick Galinkin, Yiyi Chen, Jens Myrup Pedersen Leon Derczynski Johannes Bjerva Abstract As NLP models are used by a growing number of end-users, an area of increasing importance is NLP Security (NLPSec): assessing the vulnerability of models to malicious attacks and developing comprehensive countermeasures against them. While work at the intersection of NLP and cybersecurity has the potential to create safer NLP for all, accidental oversights can result in tangible harm ( e.g. , breaches of privacy or proliferation of malicious models). In this emerging field, however, the research ethics of NLP have not yet faced many of the long-standing conundrums pertinent to cybersecurity, until now. We thus examine contemporary works across NLPSec, and explore their engagement with cybersecurity’s ethical norms. We identify trends across the literature, ultimately finding alarming gaps on topics like harm minimization and responsible disclosure. To alleviate these concerns, we provide concrete recommendations to help NLP researchers navigate this space more ethically, bridging the gap between traditional cybersecurity and NLP ethics, which we frame as “white hat NLP”. The goal of this work is to help cultivate an intentional culture of ethical research for those working in NLP Security. 1 Introduction Securing large language models (LLMs) and NLP technology in general has not been a priority until recently. Yet mass adoption of this technology has led to deployment in contexts where security failures present a risk to individuals, organizations, and society at large (demonstrated by inter alia LLM-assisted identity fraud Ackerman (2022) , phishing campaigns Hazell (2023) , and automated influence operations Goldstein et al. (2023) ). If the latest NLP technologies are to withstand an increasing barrage of threats, NLP practitioners must now educate themselves on cybersecurity and cybersecurity practitioners must educate themselves on NLP. Yet in this process, as one culture of research adapts to another, there is potential for long-standing intra-community norms to be lost in translation, including ethical norms. In interdisciplinary research, an accidental oversight of ethical norms from one field can risk reintroducing previously resolved ethical dilemmas. Discussions around ethical research conduct have long been a concern for both cybersecurity Molander and Siang (1998) ; Himma and Tavani (2008) ; Matwyshyn et al. (2010) ; Bailey et al. (2012) ; Christen et al. (2020) ; Kohno et al. (2023) and for NLP Wiener (1960) ; Samuel (1960) ; Dennett (1997) ; Moor (2006) ; Anderson and Anderson (2007) ; Hovy and Spruit (2016) ; Leidner and Plachouras (2017) , but given that these disciplines have historically been disjoint fields, the specific ethical and sociocultural norms of both fields have developed in separate silos. To better understand how interdisciplinary NLPSec has adapted to the ethical norms and values across both disciplines, we examine a set of peer-reviewed NLPSec publications from NLP venues to gauge the compliance with norms in cybersecurity. We find that several principles regarded by the cybersecurity community as best practices have not been widely adopted in NLPSec research, despite measures in the NLP peer-review process to improve research practices. Simultaneously, we find that NLP ethical norms regarding lower-resourced languages are at risk of being overlooked in NLPSec. To help save NLPSec the potential growing pains of reinventing the wheel, in this work, we seek to address this issue through an interdisciplinary conversation about ethics, with the goal of cultivating a culture of white hat NLP . Ethical NLP Security In cybersecurity, a ‘‘white hat’’ hacker is typically a professional hired by a company with the specific purpose of maintaining or increasing the existing security of a computer system. White hat hackers 1 1 1 This verbiage has long been common parlance in cybersecurity, though it has recently been criticized for its connoted colorism. As there is no widely adopted alternative, we trust readers to accept our good faith usage of the term. , referred to more generally as “ethical hackers”, are keen to engage with vendors – either their employer or a third party – whose products they have found flaws in. They aim to get security issues fixed quickly and release details of their findings to the public so defenders can evaluate their own environments and prioritize concomitant patching and mitigation efforts. In contrast, a “black hat” hacker represents a cybercriminal, and somewhere in between is the “gray hat” hacker, who infiltrates others’ computer systems without permission, with the intention of enhancing security Falk (2004) . This is generally mapped to similar activity in the context of LLMs, where LLM red teaming is defined as a limit-seeking, manual, and non-malicious activity Inie et al. (2025) . Outside the strict boundaries of ethical hackers, gray hat hackers rely on their own moral compass, which can lead to potentially precarious situations, like introducing the risk of authorized system intrusions Christen et al. (2020) . Alarmingly, much of the contemporary work in NLPSec is arguably similar to the above gray hat scenario (Figure 1 ). Due to the distributed and decentralized nature of the research landscape, most NLPSec researchers will necessarily lack a mandate from organizations to investigate security vulnerabilities inherent to models. Researchers also lack direct accountability to those most affected by the security breaches they study. While the ACM Code of Ethics 2 2 2 https://www.acm.org/code-of-ethics provides overarching guidance, it naturally cannot provide exact guidance for every ethical conundrum, leaving individual researchers to rely on their own moral compass, like a gray hat hacker. It is in this light that we aim to extend the framework of “white hat” to NLP. We thus define the scope of white hat NLPSec to consist of works which are intentionally and carefully grounded in the established ethical best practices of both cybersecurity and NLP . Figure 1: We argue current works across NLPSec occupy a gray area in research ethics, comparable to the cybersecurity concept of a “gray hat” hacker. In this work, we provide concrete recommendations to help move the field of NLPSec towards more ethically-grounded research practices. Contributions We examine the current culture of research ethics in NLPSec through a survey of 80 peer-reviewed works across NLP venues, measuring their compliance with typical ethical practices from cybersecurity. To start, we introduce these ethical practices, and touch upon their relevance to works in NLPSec (Section 2 ). We then describe our selection process for papers included in the survey, and our annotation process identifying compliance with these cybersecurity ethical best practices (Section 3 ). Concretely, we find that ethical norms like harm minimization and responsible disclosure are not widely adopted in NLPSec (Section 4 ). We identify several challenges of adopting ethical norms from both cybersecurity and NLP to the field of NLPSec, and we provide concrete advice to help researchers engage in research more ethically than previously (Section 5 ). While the recommendations presented in this work cannot provide solutions to all potential ethical conundrums latent to NLPSec, the discussions in this work aim to highlight the urgency for discussion on research ethics in NLPSec and serve as a conversation starter. 2 The Culture of Ethics in Cybersecurity Within any field, some research may have negative consequences, and foreseeable risks generally serve as a guiding force for calibrating ethical norms. This