A Deeper Dive into the Irreversibility of PolyProtect: Making Protected Face Templates Harder to Invert
Vedrana Krivokuća Hahn, Jérémy Maceiras, Sébastien Marcel · 2026
Read A Deeper Dive into the Irreversibility of PolyProtect: Making Protected Face Templates Harder to Invert by Vedrana Krivokuća Hahn, Jérémy Maceiras, Sébastien Marcel (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-03857.txt.
Introduction
Vedrana Krivokuća Hahn, Jérémy Maceiras, and Sébastien Marcel take a deeper look at whether PolyProtect truly makes face templates hard to invert. PolyProtect guards biometric embeddings by mapping them through polynomials with many variables. The coefficients and exponents differ for each enrolled subject. The polynomial is applied to overlapping chunks of the embedding. If the map can be inverted, the raw face data are exposed.
The authors show protected templates are easier to invert with a numerical solver based on cosine distance than with the Euclidean approach used earlier. They then propose a key selection algorithm. It picks the polynomial keys to harden templates against inversion, rather than leaving them purely random. Experiments show the chosen keys make inversion much harder. They also roughly equalize protection across generated templates. That matters because face systems store such templates for many people.
Catalog record
| CREATOR | Vedrana Krivokuća Hahn, Jérémy Maceiras, Sébastien Marcel |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.03857
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Deeper Dive into the Irreversibility of PolyProtect: Making Protected Face Templates Harder to Invert Vedrana Krivokuća Hahn, Jérémy Maceiras, Sébastien Marcel Abstract This work presents a deeper analysis of the ‘‘irreversibility’’ property of PolyProtect, a biometric template protection method initially proposed for securing face embeddings. PolyProtect transforms embeddings into protected templates via multivariate polynomials, whose coefficients and exponents are distinct for each subject enrolled in the face recognition system. A polynomial is applied to consecutive sets of elements from a given embedding, where the amount of overlap between the sets is a tunable parameter. We begin our irreversibility analysis by demonstrating that PolyProtected templates are easier to invert using a numerical solver based on cosine distance, as opposed to Euclidean distance (used in the earlier PolyProtect work). To make this inversion more difficult, we then propose a ‘‘key selection algorithm’’, which tries to choose ‘‘keys’’ (coefficients and exponents of the PolyProtect polynomial) that enhance the irreversibility of PolyProtected templates, compared to when the keys are purely random. Our experiments show that this algorithm is effective at generating PolyProtected templates that are significantly more difficult to invert, and that it approximately equalises the irreversibility of PolyProtected templates generated using different ‘‘overlap’’ parameters. This allows for better control of the irreversibility versus accuracy trade-off, known to exist across different overlaps. We also show that accuracy in the PolyProtected domain can be affected by the range in which the embedding elements lie, but that this can be improved by normalizing the embeddings prior to applying PolyProtect. This work is reproducible using our open-source code 1 1 1 Link will be provided upon paper acceptance. . I Introduction Our faces are becoming indispensable tools for proving our identities in various applications (e.g., unlocking smartphones, accessing bank accounts, verifying passports at electronic gates). The convenience of this form of authentication is evident, as is the heightened security and identity assurance offered by the uniqueness of our faces. There is, however, a hidden cost: our privacy. As more and more organisations collect our face data for such purposes, concerns arise around how this data is being stored and whether (and with whom) it is being shared. The problem is that, the more our face data gets distributed, the more likely it is to be misused; e.g., by creating presentation attacks or deepfakes to impersonate us and potentially gain unlawful access to protected resources, or by tracking us (for profiling or stalking purposes) across different applications in which the same data is enrolled. So, it is crucial that face data never be stored in the clear: it should always be protected, such that, if the databases of the underlying face recognition systems are jeopardised, the original, irreplaceable face information remains inaccessible. This is the realm of Biometric Template Protection (BTP). The aim of BTP is to convert a biometric feature vector, or “template”, into a protected template, from which it is impossible to recover the original one. In this paper, we are interested in the application of BTP to face templates. Modern face recognition systems are based on deep learning architectures that are trained to map face images to fixed-length numerical representations called “embeddings”. It has been shown that face embeddings are invertible [ 3 ] , in that they can be used to recover an approximation of the underlying face image [ 33 , 4 , 22 , 23 ] , and that certain soft biometric attributes (e.g., sex, race, age, hair colour) can be extracted from these representations [ 9 , 32 ] . These findings underline the need to protect embeddings in order to prevent recovery of the original face information, thereby protecting the privacy of the face recognition system users. This is where BTP comes into play. A recent survey of face BTP methods [ 17 ] , as well as Part II of the new Handbook of Biometric Template Protection [ 15 ] , revealed two broad approaches: applying handcrafted (human-designed) BTP methods to learned face embeddings, and using neural networks to learn the BTP algorithms. Handcrafted BTP methods include: feature transformations [ 6 ] , which involve transforming a biometric template from its original feature space to a new, protected space (e.g., [ 7 , 18 ] ); biometric cryptosystems [ 27 ] , which most commonly involve binding an external key with the biometric template (e.g., [ 11 , 28 ] ); and homomorphic encryption [ 2 ] , which allows us to compare reference and probe templates in encrypted form (e.g., [ 20 , 1 , 8 ] ). Learned BTP methods [ 19 ] include: training a neural network to learn the mapping from a biometric template to a pre-defined random code (e.g., [ 24 , 14 ] ); or training a neural network to learn its own representation of a protected template (e.g., [ 31 , 25 , 21 ] ). While learned BTP methods may allow for higher complexity and thus potentially the generation of more secure protected templates, handcrafted methods have two chief advantages that make them easier to adopt in practice: (i) they can more readily be integrated into existing biometric systems (e.g., as a module after the feature extractor), and (ii) they tend to be easier to evaluate, since the algorithms are designed by humans (i.e., not learned by neural networks) so their properties are more explicit. For these reasons, in our work we adopt a handcrafted BTP method. The context for this work is defined by a project that aims to use BTP to develop a privacy-preserving face identification system for humanitarian aid distribution. This is similar to the context in [ 30 ] , except that the end goal of our project is different. Nevertheless, as the global target in both cases is a humanitarian use-case, the criteria for selecting a suitable BTP method were similar. In particular, for our work, the chosen BTP method should possess the following characteristics: easy and lightweight to implement, modular (can be integrated into an existing face recognition system), satisfies the main BTP criteria (recognition accuracy, irreversibility, unlinkability), and ideally comes with open-source code. So, similarly to the reasoning outlined in [ 30 ] , these requirements narrowed our selection to handcrafted , as opposed to learned, BTP methods. Then, within this category, biometric cryptosystems were eliminated due to known accuracy issues and envisaged implementation difficulties in our use-case, and homomorphic encryption was deemed unsuitable due to its computational complexity and the need to safely store the decryption key (which cannot be guaranteed in a volatile humanitarian setting). So, we settled on the feature transformations category, in which the PolyProtect method [ 18 ] was found to most closely align with our project requirements. PolyProtect transforms face embeddings into protected templates via multivariate polynomials, whose parameters (coefficients, C , and exponents, E ) are unique for each subject enrolled in the face recognition system. A polynomial is applied to consecutive sets of elements from a given embedding, where the amount of overlap between the sets is a tunable parameter. PolyProtect has already been shown [ 18 , 30 ] to satisfy the three crucial BTP criteria: recognition accuracy, irreversibility, and unlinkability. Our goal is