On Large Language Models in Mission-Critical IT Governance: Are We Ready Yet?

Matteo Esposito, Francesco Palagiano, Valentina Lenarduzzi, Davide Taibi · 2024

Read On Large Language Models in Mission-Critical IT Governance: Are We Ready Yet? by Matteo Esposito, Francesco Palagiano, Valentina Lenarduzzi, Davide Taibi (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-11698.txt.

Introduction

Matteo Esposito and co-authors ask whether language models are ready for mission-critical IT governance in this 2024 paper. Systems vital to national security need fast, robust oversight. Cyber warfare keeps raising the stakes for those who guard them.

The team surveyed practitioners who build security for such systems. They gathered hands-on hopes, fears, and expectations about generative AI help. Prior work showed models can aid risk analysis, which motivated the poll. Safe use, the findings stress, demands teamwork across fields. Researchers should build regulation-aware models with clear accountability. Practitioners flagged data protection and openness, while policy makers must set shared global rules.

Catalog record

CREATORMatteo Esposito, Francesco Palagiano, Valentina Lenarduzzi, Davide Taibi
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-11698

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

On Large Language Models in Mission-Critical IT Governance: Are We Ready Yet? Matteo Esposito, Francesco Palagiano, Valentina Lenarduzzi, Davide Taibi Abstract Context . The security of critical infrastructure has been a pressing concern since the advent of computers and has become even more critical in today’s era of cyber warfare. Protecting mission-critical systems (MCSs), essential for national security, requires swift and robust governance, yet recent events reveal the increasing difficulty of meeting these challenges. Aim . Building on prior research showcasing the potential of Generative AI (GAI), such as Large Language Models, in enhancing risk analysis, we aim to explore practitioners’ views on integrating GAI into the governance of IT MCSs. Our goal is to provide actionable insights and recommendations for stakeholders, including researchers, practitioners, and policymakers. Method. We designed a survey to collect practical experiences, concerns, and expectations of practitioners who develop and implement security solutions in the context of MCSs. Conclusions and Future Works . Our findings highlight that the safe use of LLMs in MCS governance requires interdisciplinary collaboration. Researchers should focus on designing regulation-oriented models and focus on accountability; practitioners emphasize data protection and transparency, while policymakers must establish a unified AI framework with global benchmarks to ensure ethical and secure LLMs-based MCS governance. I Introduction Since the advent of computers [ 1 , 2 ] , security has been a persistent concern [ 3 ] . In today’s era of cyber warfare, where conflicts unfold over fiber optic networks [ 4 ] , protecting critical infrastructure has become increasingly vital [ 5 ] . While daily activities heavily rely on digital platforms [ 4 ] , telecommunication systems used for casual tasks differ significantly in criticality and fault tolerance from those required for national crises, such as military and population coordination [ 6 , 7 , 8 ] . These vital systems, known as “mission-critical systems” (MCSs) [ 6 ] , operate within a “mission-critical context” (MCC). MCSs require immediate and robust security measures [ 3 ] , yet their governance standards remain fragmented and incompatible across nations [ 9 ] . Governance of MCS (GMCS) involves directing, managing, and controlling IT resources to ensure full compliance with organizational objectives, value maximization, and mitigation of risks associated with core operations in a structured, reproducible method [ 9 , 3 ] . According to the Italian National Security Authority (ANS), GMCS is ( i ) time-sensitive, ( ii ) cyber-physically fault-tolerant, and ( iii ) reproducible. In other words, practitioners design guidelines and enact workflows that are quick and easy to follow (time-sensitive), thoroughly documented (reproducible), and risks are extensively analyzed and managed to minimize them and account for backup plans (cyber-physically fault-tolerant). Recent events have highlighted that experts often struggle to meet the three core requirements of GMCS [ 10 , 11 ] . Motivated by these challenges and documented failures [ 10 , 11 ] , we previously explored the potential of Generative AI (GAI) to assist human experts in performing comprehensive risk analysis, one of the key tasks outlined in ANS and ISO standards [ 3 , 10 , 12 , 13 ] . Our findings demonstrated that GAI, particularly Large Language Models (LLMs), can support and outperform human experts in speed and accuracy. Therefore, stemming from insights and strong interest from industrial partner regarding the integration of GAI into GMCS [ 3 , 10 ] , we leveraged such insights, preliminary designs, ethics, and regulations on the topic, to drive our contributions to the community in future research. Based on those experiences, we tried to study practitioners’ views about the usage of GAIs, especially LLMs, inside GMCS. More specifically: ( i ) we provide the first survey on practitioners feelings and perceptions on integrating GAI in GMCS; ( ii ) we elaborate implications tailored to researchers to guide future works; ( iii ) we suggest actionable insights for practitioners; ( iv ) we provide suggestions for policymakers to craft informed regulations and policies. Our survey highlights the essential role of interdisciplinary collaboration among researchers, practitioners, and policymakers in advancing the safe, ethical, and effective use of LLMs in GMCS. Our findings urge Researchers to focus on regulation-oriented models, while practitioners highlight the need for data protection, transparency, regulatory compliance and accountability to complement human expertise. Finally, policymakers should work towards a unified AI framework , fostering global benchmarks and advisory boards to ensure that LLMs in GMCS can match the needed security, ethical, and accountability standards. Paper Structure: Section II describes the study design. Section III presents the obtained results, and Section IV discusses them. Section V highlights the threats to the validity of our study, and Section VI discusses related works. Finally, Section VII draws the conclusion. Data Availability Statement We publicly provide all raw answers, gathered insights, and questionnaires as part of our replication package, which is hosted on Zenodo 1 1 1 https://doi.org/10.5281/zenodo.13918422 . II Study Design This section details the goal and research questions, data collection, and data analysis. Our empirical study follows established guidelines [ 14 , 15 , 16 ] and previous similar survey works [ 17 ] . II-A Goal and Research Questions The goal of our empirical study is to investigate the practitioners perceived benefits, effectiveness, and potential ethical and privacy issues on using LLMs in the governance of mission-critical information technology systems. Our perspective is of practitioners seeking to understand the balance between innovation in the decision process and thorough application of regulations in the given context. Based on our goal, we defined the following four Research Questions (RQs): RQ_{1} . What is the current level of familiarity and experience among MCS practitioners with LLMs? Recent years have witnessed the proliferation of AI-enabled technologies’ general availability [ 18 ] . What was a nice cutting-edge lab experiment is now driving innovation in every possible field [ 18 ] . Our context is peculiar and dominated by a conservative approach to new technologies. For instance, in our previous investigation in this context [ 3 , 10 ] , we were the first to introduce LLMs in Risk Analysis (RA), a fundamental part of GMCS itself. We highlighted LLMs’ role in RA as a valuable assistant to speed up and enhance human experts’, i.e., practitioners, capabilities. However, to extend the reach of LLMs capabilities in GMCS and to investigate practitioners’ perceptions of the technology and its introduction, they should have a minimal background in using LLMs in MCS. With this first research question, we aim to investigate the current level of familiarity of practitioners with LLMs. Nonetheless, familiarity is only one face of a multi-faced dice. For instance, innovation is usually blocked by negative experiences or feelings towards a specific technology [ 19 ] . Practitioners exposed to LLMs in MCS for the first time can have different perceived benefits and potential limitations that may allow or prevent the adoption. Hence, we ask: RQ_{2} . What are the future roles, perceived benefits and potential limitations of LLMs in GMCS? Research is usually faster than industries when the latter are not involved in research and innovation (RI) initiatives [ 20 ] . Therefore, researcher-driven innovation can be far from real-world immediate actionability. Hence, research can lead

Read the complete text