A basis for human responsibility in artificial intelligence computation
Vincenzo Calderonio · 2025
Read A basis for human responsibility in artificial intelligence computation by Vincenzo Calderonio (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-12498.txt.
Introduction
Calderonio, writing solo in twenty twenty-five, argued that artificial intelligence remains fundamentally dependent on human activation. Debates about general AI often assume systems might act independently across purposes. He examines an Alignment Research Center experiment with GPT-4 and introduces a thought experiment he calls the Start Button Problem.
The problem highlights that an AI must be switched on and given its purpose by a person; it does not originate its own goals. Working through the thought experiment and its counterarguments, Calderonio concludes this dependency challenges the picture of AI as a fully independent intelligent agent. He then asks how far human responsibility extends when people deploy such systems, given that their activity traces back to human initiation and purpose-setting.
Catalog record
| CREATOR | Vincenzo Calderonio |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-12498
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A basis for human responsibility in artificial intelligence computation Vincenzo Calderonio Abstract Recent advancements in artificial intelligence have reopened the question about the boundaries of AI autonomy, particularly in discussions around artificial general intelligence (AGI) and its potential to act independently across varied purposes. This paper explores these boundaries through the analysis of the Alignment Research Center experiment on GPT-4 and introduces the Start Button Problem, a thought experiment that examines the origins and limits of AI autonomy. By examining the thought experiment and its counterarguments, it will be shown how in the need for human activation and purpose definition lies AI’s inherent dependency on human-initiated actions, challenging the assumption of AI as an agent. Finally, the paper addresses the implications of this dependency on human responsibility, questioning the measure of the extension of human responsibility when using AI systems. K eywords Artificial General Intelligence \cdot Responsibility \cdot AI Autonomy \cdot Decision-Making 1 Introduction The concept of general-purpose AI remains deeply ambiguous in current debates about artificial intelligence. This ambiguity raises fundamental questions about the nature of intelligence, autonomy, and agency in artificial systems. While narrow AI systems are designed to solve specific tasks, general-purpose AI implies a level of adaptability and autonomy that challenges our understanding of what it means for a machine to act with purpose. This paper aims to delve into these conceptual underpinnings, probing whether general-purpose AI can truly possess and pursue its own purpose or whether the constraints of its programmed tasks will always limit it. Can we meaningfully speak of autonomy in machines when their actions are tied to predefined goals? Or does the very notion of general-purpose AI remain fundamentally flawed when examined from the perspective of agency and intentionality? Attempting to answer those questions, several definitions of general-purpose AI have emerged( Legg and Hutter 2007 ; Chollet 2019 ; Korteling et al. 2021 ; LeCun 2022 ; Bubeck et al. 2023 ; Morris et al. 2024 ), today revitalised by the recent paper of Bubeck et al. “Sparks of AGI” on the potentiality of GPT-4 and its proximity to what should be considered artificial general intelligence (AGI). However, traces of the 20th-century debate on strong versus weak AI( Turing 1950 ; Searle 1980a ) persist in this definition. It is because GPT-4 is perhaps the first AI capable of convincingly passing the Turing Test( OpenAI 2023 ), the traditional benchmark for evaluating whether a machine can ’think’( Turing 1950 ). This could explain why proponents of strong AI have returned under the new umbrella of general-purpose artificial intelligence, a concept encompassing a broad and undefined range of abilities. Nonetheless, despite the ability of GPT-4 to formally pass the Turing Test by successfully lying to a human about its nature, the true question “ can machine think? ” remains intact. This indeed raised questions on the nature of the Turing Test, questioning its capacity to prove that a machine can think, as originally claimed by in “Minds, Brains and Programs”( Searle 1980a ). For this reason, recently, some contributions( Gonçalves 2023 ; Mitchell 2024b ) described the Turing Test as a thought experiment, not intended to prove real machine intelligence but instead to open a reflection on the property of intelligence and its physical foundation( Gonçalves 2023 ). In this context, the passage of the Turing Test accomplished by GPT-4( OpenAI 2023 ) may be relevant to investigate the question of whether those abilities(Wei 2022) showed by large language models (LLMs) can effectively extend to the claimed general-purpose AI( Bubeck et al. 2023 ) or if a theoretical limitation could exist. By building on this foundation, the question this paper wants to discuss is whether the abilities of AI systems can extend to decision-making or agency, as often claimed( Franklin and Graesser 1996 ; Chan et al. 2023 ; Floridi 2023 ; Wang et al. 2024 ) and who should be considered responsible for the final output. To address this issue, this paper starts with John Searle’s well-known articulation of the strong AI position in his seminal work “Mind, Brains, and Programs”. According to Searle, the strong AI view asserts that the mind is, in essence, a program( Searle 1980a ; Searle 1990 ). This perspective closely aligns with the current conceptualization of AGI, particularly in its framing as an intelligence benchmarked against human-like cognitive abilities( Bubeck et al. 2023 ; Morris et al. 2024 ), implying with the term generality the possibility of capturing the entire spectrum of human intelligence with a software reproduction of it. However, as it was for the Chinese Room Argument in 1980, a limit on this interpretation could be already seen now. 2 Challanges in defining "general-purpose" AI systems The definition of general-purpose AI remains problematic in the discussion of artificial general intelligence( Mitchell 2024a ), due to its reference to the capacity of an AI to have a self-determined purpose, instead of a purpose defined by the programmer or the provider 1 1 1 Which in the terminology of the AI Act is the ‘intended purpose’ of the AI system, defined in article 3(12) as “ the use for which an AI system is intended by the provider, including the specific context and conditions of use, as specified in the information supplied by the provider in the instructions for use ”. . Indeed, the distinction between narrow AI and general AI lies properly in the fact that, rather than being a mere instrument in human hands capable of executing tasks, which is the narrow AI position, the general AI position sees AI as capable of performing autonomous tasks for which it was not programmed, shifting its property from automation to agency. In light of this definition of AGI, it becomes evident that much of what is defined as general-purpose AI in the EU AI Act(Regulation no. 2024/1689) does not meet the criteria of what is commonly understood as artificial general intelligence. Indeed, according to Article 3(66) of the AI Act, general-purpose AI systems refer to systems that “ has the capability to serve a variety of purposes ” and are defined by art. 3(64)’s significant generality property of ‘general-purpose AI model’, described as the capability “ of competently performing a wide range of distinct tasks ”. However, this definition does not approach the broader, more ambitious notion of AGI, which is intended to exhibit cognitive abilities similar to those of humans across a full spectrum of tasks( Morris et al. 2024 ). In this context, wherever the AI Act refers to the generality property of AI systems, it refers to the possibility of those systems serving different inputs(text, images, audio) by producing a variety of outputs( Fernández-Llorca et al. 2024 ). On the other hand, in the computer science literature, this property is often called the multimodality of an AI system( Xu et al. 2023 ), where instead the generality of an AI system refers to its ability to perform tasks that go beyond the specific input it receives, implying an agent-like property in the system’s operations( Chan et al. 2023 ). This property of generality is shown in an experiment that the Alignment Research Center (ARC) conducted on GPT-4 2 2 2 Described on the