Generating Poisoning Attacks against Ridge Regression Models with Categorical Features
Monse Guedes-Ayala, Lars Schewe, Zeynep Şuvak, Miguel Anjos · 2025
Read Generating Poisoning Attacks against Ridge Regression Models with Categorical Features by Monse Guedes-Ayala, Lars Schewe, Zeynep Şuvak, Miguel Anjos (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-07275.txt.
Introduction
Monse Guedes-Ayala and co-authors craft poisoning attacks against ridge regression models that use categorical features. Attackers corrupt training data to bend a model's behavior. Defenders need strong sample attacks to test their shields.
Categories are often ignored by earlier attack designs. Here they are modeled explicitly as special set variables. The attack becomes a two-level optimization problem. The upper level is mixed-integer and nonconvex. The lower level stays a simple convex problem. A single-level reformulation makes it solvable. The method poisons both numbers and categories jointly. It gives security teams a tougher benchmark attack for red teams to use in drills and tests.
Catalog record
| CREATOR | Monse Guedes-Ayala, Lars Schewe, Zeynep Şuvak, Miguel Anjos |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-07275
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Generating Poisoning Attacks against Ridge Regression Models with Categorical Features Monse Guedes-Ayala, Lars Schewe, Zeynep Şuvak, Miguel Anjos Abstract . Machine Learning (ML) models have become a very powerful tool to extract information from large datasets and use it to make accurate predictions and automated decisions. However, ML models can be vulnerable to external attacks, causing them to underperform or deviate from their expected tasks. One way to attack ML models is by injecting malicious data to mislead the algorithm during the training phase, which is referred to as a poisoning attack. We can prepare for such situations by designing anticipated attacks, which are later used for creating and testing defence strategies. In this paper, we propose an algorithm to generate strong poisoning attacks for a ridge regression model containing both numerical and categorical features that explicitly models and poisons categorical features. We model categorical features as SOS-1 sets and formulate the problem of designing poisoning attacks as a bilevel optimization problem that is nonconvex mixed-integer in the upper-level and unconstrained convex quadratic in the lower-level. We present the mathematical formulation of the problem, introduce a single-level reformulation based on the Karush-Kuhn-Tucker (KKT) conditions of the lower level, find bounds for the lower-level variables to accelerate solver performance, and propose a new algorithm to poison categorical features. Numerical experiments show that our method improves the mean squared error of all datasets compared to the previous benchmark in the literature. 1. Introduction Machine learning (ML) is a branch of artificial intelligence (AI) that is concerned with developing algorithms that use statistical techniques to identify patterns in data and make predictions. However, these algorithms are susceptible to adversarial attacks, which can significantly damage their performance. An example of an attack ML models can be subject to is poisoning attacks . In these attacks, a malicious actor introduces contaminated data points into the training dataset, causing the model to learn incorrect patterns and degrade its performance on unseen data. An example of a poisoning attack is the generation of emails to mislead spam filters. We can introduce a small number of carefully crafted emails into the training data to cause the model to learn a biased decision boundary and overlook some spam emails. One common defence against adversarial attacks is a technique called adversarial training, which involves generating adversarial examples during the training process to make the model more robust to such attacks. By systematically designing and executing attack strategies, we can not only develop effective defence mechanisms but also gain valuable insights into the vulnerabilities of machine learning models to external manipulation. This paper focusses on developing poisoning attacks specifically targeting regression algorithms with categorical features using tools from mathematical optimization. The work presented here can be used to test the model’s robustness and identify vulnerabilities the attackers might exploit. The main challenge for attackers lies in identifying the most effective data points to manipulate without being detected. This process can be framed as an optimization problem, as the attacker seeks to maximise the impact on the machine learning model. Since the attacker optimises the attacks while taking into account the optimal response of the machine learning model to the new data, there is a hierarchical nature in this process. Because of this hierarchical nature, it is possible to frame it as a bilevel optimization problem. Bilevel optimization covers a special class of mathematical optimization problems that model sequential and hierarchical decision-making between two or more players. The first player optimises their decision by taking into account the optimal reaction of a second player. This reaction will, in turn, influence the outcome of the first player. The first player is usually referred to as the upper level or leader, while the second player is called the lower level or follower. Mathematically, the lower-level problem appears in the constraints of the upper level and only those decisions that are optimal for the follower are feasible for the leader. In the case of poisoning attacks, the leader is the attacker, who decides first on which data samples to inject. Then, the follower, the ML model, learns the optimal parameters using, among other samples, the samples injected by the attacker. The main focus of the research on poisoning attacks has been on classification algorithms, where the goal of the model is to predict the class to which data samples belong. The most employed strategy to solve this problem is to use gradient-based approaches to find optimal poisoning attacks. More recent papers have also considered poisoning attacks for regression models, where predictions consist of assigning continuous numerical values. To the best of our knowledge, Jagielski et al. [ 6 ] were the first to study poisoning attacks for linear regression models. They formulate the attack as a bilevel optimization problem and propose two attack strategies: an adaptation of a previously proposed gradient-based approach for classification models, and a statistical-based poisoning attack. Then, the papers [ 11 , 17 ] aimed to improve the performance of [ 6 ] in terms of the impact of the attack on the regression model and computational time. Li et al. [ 8 ] focus on poisoning attacks to modify regression parameters in a specific direction and solve the optimization problem by formulating its closed form. Şuvak et al. [ 15 ] study in more depth the bilevel formulation of this problem and find the optimal poisoning samples by solving to local optimality the equivalent single-level reformulation of the problem. Even though the authors of [ 15 ] discuss the possibility of considering categorical variables for this type of attack, they only solve the optimization problem for variables associated with numerical features. None of these approaches explicitly optimise categorical variables. Instead, these variables are treated as numerical variables and then heuristically rounded to meet the binary structure of categorical features. However, many datasets used for regression tasks involve a combination of numerical and categorical features [ 12 , 13 , 5 ] and categorical variables can be the key to making predictions. One can think, for example, how much the predictions of house prices in London would change if the neighbourhood is not taken into account. Therefore, poisoning attacks that do not fully optimise categorical variables can be too weak for some applications. This paper introduces a novel mixed-integer bilevel optimization model for poisoning attacks in adversarial machine learning, specifically targeting cases with categorical features modelled as binary variables. To the best of our knowledge, this is the first application of mixed-integer bilevel optimization to poisoning attacks with categorical data. Following this introduction, in Section 2 we present the mixed-integer bilevel optimization model, followed by its single-level reformulation in Section 3 . In Section 4 , we propose a method for bounding previously unbounded lower-level variables. This is followed by Section 5 , which details a heuristic algorithm for generating poisoning attacks. Computational experiments are presented in Section 6 , testing on two publicly available datasets. We conclude with some remarks and future research directions in Section 7 . Main contributions include: the development of a mixed-integer bilevel optimization model for poisoning attacks involving categorical features, a method for bounding previously unbounded lower-level variables using a sensitivity analysis of the lower