Prompt Engineering vs. Fine-Tuning for LLM-Based Vulnerability Detection in Solana and Algorand Smart Contracts

Biagio Boi, Christian Esposito · 2025

Read Prompt Engineering vs. Fine-Tuning for LLM-Based Vulnerability Detection in Solana and Algorand Smart Contracts by Biagio Boi, Christian Esposito (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-11250.txt.

Introduction

Biagio Boi and Christian Esposito compare prompt engineering against fine-tuning for using large language models to find flaws in smart contracts on Solana and Algorand. Smart contracts automate deals on blockchains, but sloppy code can lock in serious vulnerabilities. Most detection research targets Ethereum-style contracts, leaving these two platforms understudied and short of labeled data.

The authors built a synthetic data set of annotated code snippets in Rust and PyTeal, sorted by a flaw taxonomy drawn from OWASP guidance. Prompting alone proved broadly robust, while fine-tuning sharpened precision and recall on the leaner TEAL language. Platform design shaped how flaws appear and how well each method detects them.

Catalog record

CREATORBiagio Boi, Christian Esposito
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-11250

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Prompt Engineering vs. Fine-Tuning for LLM-Based Vulnerability Detection in Solana and Algorand Smart Contracts Biagio Boi, Christian Esposito Abstract Smart contracts have emerged as key components within decentralized environments, enabling the automation of transactions through self-executing programs. While these innovations offer significant advantages, they also present potential drawbacks if the smart contract code is not carefully designed and implemented. This paper investigates the capability of large language models (LLMs) to detect OWASP-inspired vulnerabilities in smart contracts beyond the Ethereum Virtual Machine (EVM) ecosystem, focusing specifically on Solana and Algorand. Given the lack of labeled datasets for non-EVM platforms, we design a synthetic dataset of annotated smart contract snippets in Rust (for Solana) and PyTeal (for Algorand), structured around a vulnerability taxonomy derived from OWASP. We evaluate LLMs under three configurations: prompt engineering, fine-tuning, and a hybrid of both, comparing their performance on different vulnerability categories. Experimental results show that prompt engineering achieves general robustness, while fine-tuning improves precision and recall on less semantically rich languages such as TEAL. Additionally, we analyze how the architectural differences of Solana and Algorand influence the manifestation and detectability of vulnerabilities, offering platform-specific mappings that highlight limitations in existing security tooling. Our findings suggest that LLM-based approaches are viable for static vulnerability detection in smart contracts, provided domain-specific data and categorization are integrated into training pipelines. I Introduction Smart contracts represent the backbone of decentralized applications built on top of blockchain platforms, enabling the automation of transactions through self-executing programs. While these innovations offer significant advantages—such as increased transparency in transactions, which has made a substantial impact on areas like supply chain management [ 7 ] and digital certification systems [ 15 ] —they also present potential drawbacks if the smart contract code is not carefully designed and implemented. These self-executing programs that reside on the blockchain are particularly exposed due to their public visibility, permanent deployment, and financially critical logic. Once deployed, these contracts cannot be altered, making any flaws in their design or implementation particularly dangerous. Large Language Models (LLMs) are increasingly used to generate code, including smart contracts for blockchain platforms like Ethereum, with an increasing delegation of code generation to such tools [ 21 ] . LLMs are also being used for the verification of the code written by programmers, as they are able to find bugs and vulnerabilities there. LLMs have proved themselves to be useful for code generation, while vulnerability assessment is still at its beginning with a not satisfying prediction [ 1 ] . In addition, when users prompt LLMs to write smart contracts, the model generates code based on its training data. However, the model might rely on incomplete knowledge about security standards, and, more critically, they have not been properly fine-tuned for verification compared to the mere syntactic check. Given the recent expansion of alternative blockchain platforms, each characterized by distinct programming languages and architectural frameworks [ 4 ] , the role of vulnerability assessment is progressively more important than in the past. On the one hand, the current literature lacks a proper analysis of the vulnerability types outside the Solidity and Ethereum framework. On the other hand, despite having tools available for blockchains such as Solana and Algorand, which differ significantly from Ethereum, there is no precise evaluation of their effectiveness in detecting the most common vulnerabilities. In light of current needs and to overcome these limitations, in this study, we: • Investigate security challenges across emerging blockchain platforms—specifically Algorand and Solana—by analyzing critical vulnerabilities according to the recent OWASP Top 10 [ 23 ] . • Examine the role of LLMs in vulnerability detection for domain-specific programming languages, and compare the effectiveness of prompt engineering and fine-tuning techniques. II Background II-A Blockchain Security The deterministic execution of smart contracts requires all nodes in the network to arrive at the same state, meaning that even a single exploitable bug can be globally catastrophic. While traditional security models such as CIA (Confidentiality, Integrity, Availability) remain partially applicable, the decentralized and immutable nature of blockchains introduces new attack surfaces. Most popular vulnerabilities in smart contracts can be categorized into various types. (1) Control flow issues, such as reentrancy attacks, where recursive function calls allow an attacker to drain funds before state updates are completed. (2) Arithmetic errors, including integer underflow and overflow, which can alter logic or bypass checks. (3) Access control failures, where insecure permission structures grant unintended users the ability to modify critical variables. (4) Unchecked external calls, which can lead to unexpected interactions with untrusted code. Recent work (e.g., OWASP Top 10 for Blockchain, 2025 [ 23 ] ) categorizes these threats into protocol-level attacks, implementation-level bugs, and vulnerabilities within smart contract logic. Several widely adopted taxonomies have emerged to describe and organize these risks. The most relevant one, apart OWASP Top 10, is the SWC Registry (Smart Contract Weakness Classification and Test Cases), a structured vulnerability repository maintained by the smart contract auditing community. Anyway, to effectively assess vulnerabilities across different blockchain platforms requires dedicated evaluation tailored to their execution environments and programming models. For instance, Ethereum’s EVM supports tools like SmartCheck [ 30 ] and Slither [ 12 ] , but those cannot be applied to Algorand and Solana blockchains due to different characteristics both in programming language and blockchain structure. As an example, while Algorand leverages TEAL programming language, Solana leverages Rust. Another example is that aforementioned Ethereum-specific tools often rely on symbolic execution, data flow analysis, or fuzzing techniques to surface vulnerabilities prior to deployment, which requires knowing the environment in which the execution will take place. To better explain these differences, in this section, we briefly introduce some concepts about the characteristics of Algorand and Solana blockchains. In the subsequent section, we delve deeper into the specific vulnerabilities affecting these platforms and analyze how the LLM can manage these vulnerabilities. II-A 1 Algorand The Algorand blockchain was born to overcome a critical aspect of decentralized systems, known as Blockchain Trilemma , characterized by a tradeoff of decentralization, security, and scalability. It is hard to maintain all the three properties in a blockchain. According to a recent evaluation presented in [ 13 ] , Algorand it is able to easily surpass the Ethereum 2.0 performance by offering increased indicators for all three characteristics. Unlike traditional blockchains that rely on energy-intensive proof-of-work mechanisms, Algorand uses a Pure Proof-of-Stake (PPoS) consensus algorithm. This allows it to process thousands of transactions per second with finality in seconds. Unlike permissioned blockchains or those with limited validator sets, Algorand’s PPoS protocol randomly selects participants from the entire set of token holders. This randomness and inclusiveness make it resistant to centralization and censorship. Considering the strong consensus protocol, most of the vulnerabilities come from the smart contract logic, transaction configuration, and application-layer design [ 29 ] . During the validation of the tool discussed in [ 29 ] , it was observed that approximately 4.04% of on-chain applications exhibited vulnerabilities, with the majority of these issues arising from arbitrary deletion operations. As will be elaborated in subsequent sections of this manuscript, vulnerabilities can be easily introduced by smart contract programmer, despite having clear in mind which are the main drawbacks they can face. II-A 2 Solana

Read the complete text