On the Validity of Traditional Vulnerability Scoring Systems for Adversarial Attacks against LLMs

Atmane Ayoub MANSOUR BAHAR, Ahmad Samer WAZAN · 2024

Read On the Validity of Traditional Vulnerability Scoring Systems for Adversarial Attacks against LLMs by Atmane Ayoub MANSOUR BAHAR, Ahmad Samer WAZAN (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-20087.txt.

Introduction

Atmane Mansour Bahar and Ahmad Samer Wazan test whether standard security scoring works for attacks on language models, in work from 2024. The Common Vulnerability Scoring System grades software flaws, and practitioners have begun applying it to adversarial attacks on AI.

The authors scored 56 published attacks using several metrics, with three language models supplying the ratings. Scores barely varied from attack to attack. That flatness suggests many scoring factors, especially fixed-choice and context-specific ones, cannot tell a mild attack from a severe one. The authors call for flexible, purpose-built metrics for AI attacks. Their analysis gives security teams a reason to distrust off-the-shelf scores here.

Catalog record

CREATORAtmane Ayoub MANSOUR BAHAR, Ahmad Samer WAZAN
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE194 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-20087

Browse the full library · About the Tower of Babel