ArtBrain: An Explainable end-to-end Toolkit for Classification and Attribution of AI-Generated Art and Style

Ravidu Suien Rammuni Silva, Ahmad Lotfi, Isibor Kennedy Ihianle, Golnaz Shahtahmassebi, Jordan J. Bird, Department of Computer Science, Department of Physics, MathematicsNottingham Trent University, United Kingdom · 2024

Read ArtBrain: An Explainable end-to-end Toolkit for Classification and Attribution of AI-Generated Art and Style by Ravidu Suien Rammuni Silva, Ahmad Lotfi, Isibor Kennedy Ihianle, Golnaz Shahtahmassebi, Jordan J. Bird, Department of Computer Science, Department of Physics, MathematicsNottingham Trent University, United Kingdom (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-01512.txt.

Introduction

AI image makers now produce art good enough to fool casual viewers, and telling machine work from human work is a growing need. Ravidu Suien Rammuni Silva and four co-authors built a toolkit for that job in a 2024 paper. Their system, ArtBrain, both sorts images and explains its calls.

The team first built AI-ArtBench, a set of 185,015 images across ten art styles, mixing 125,015 AI-made works with 60,000 human ones. Their network, a tuned version of a known design with an attention step, named the style and origin of each piece. It traced images to their source model with near-perfect accuracy. A web app opens the tool to non-experts. In a side test with 50 people, humans spotted AI images only about 58 percent of the time, far below the model.

Catalog record

CREATORRavidu Suien Rammuni Silva, Ahmad Lotfi, Isibor Kennedy Ihianle, Golnaz Shahtahmassebi, Jordan J. Bird, Department of Computer Science, Department of Physics, MathematicsNottingham Trent University, United Kingdom
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-01512

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ArtBrain: An Explainable end-to-end Toolkit for Classification and Attribution of AI-Generated Art and Style Ravidu Suien Rammuni Silva, Ahmad Lotfi, Isibor Kennedy Ihianle, Golnaz Shahtahmassebi, Jordan J. Bird, Department of Computer Science, Department of Physics, MathematicsNottingham Trent University, United Kingdom Abstract Recently, the quality of artworks generated using Artificial Intelligence (AI) has increased significantly, resulting in growing difficulties in detecting synthetic artworks. However, limited studies have been conducted on identifying the authenticity of synthetic artworks and their source. This paper introduces AI-ArtBench, a dataset featuring 185,015 artistic images across 10 art styles. It includes 125,015 AI-generated images and 60,000 pieces of human-created artwork. This paper also outlines a method to accurately detect AI-generated images and trace them to their source model. This work proposes a novel Convolutional Neural Network model based on the ConvNeXt model called AttentionConvNeXt. AttentionConvNeXt was implemented and trained to differentiate between the source of the artwork and its style with an F1-Score of 0.869. The accuracy of attribution to the generative model reaches 0.999. To combine the scientific contributions arising from this study, a web-based application named ArtBrain was developed to enable both technical and non-technical users to interact with the model. Finally, this study presents the results of an Artistic Turing Test conducted with 50 participants. The findings reveal that humans could identify AI-generated images with an accuracy of approximately 58%, while the model itself achieved a significantly higher accuracy of around 99%. K eywords AI Art \cdot Explainable AI \cdot Diffusion Models \cdot Convolutional Neural Networks 1 Introduction A key advancement of recent generative AI models is their ability to produce visually appealing and realistic art [ 1 , 2 , 3 ] . Generative AI models have become very advanced; an art generated using the model won a fine art competition meant for human artists [ 4 ] . A recent example is the ‘Sony World Photography Award’ [ 5 ] , which was won by an AI-generated image. Taking into account these recent developments, the future of the internet can be envisioned to be filled with synthetic artwork generated using Image Generative Models (IGMs) and other synthetic media content. The ownership and responsibility aspects of this synthetic content are still an ongoing debate and a heavily understudied topic. Due to this, the detection of synthetic data is crucial, especially for art, where due credit should always be given to the artist and the artist’s skill. IGMs were inspired by the ideas published in [ 6 ] . One of the first generative models published was Generative Adversarial Models (GAN), a type of IGM that generates images using two competing models [ 7 ] . Variational Auto Encoder (VAE) has been explored for image generation using an Encoder-Decoder-based model setup [ 8 ] . The encoder encodes the input image in a vector-based form, while the decoder uses the vector to recreate the image. GANs rapidly improved with new variants including StyleGAN [ 9 ] , generating increasingly more realistic images. Diffusion models [ 10 ] are a type of IGM that generates images by learning to destroy visual information and trying to recreate them. Later, it was improved to exceed the potentiality and capabilities of GANs [ 11 , 12 ] . The diffusion models were based on the mathematical stochastic model called Markov Chains, which explains the probabilistic background of a linked chain of events. These types of models have significantly improved the generation quality of synthetic images, including photorealistic images [ 13 ] and artistic images [ 14 ] . Standard Diffusion [ 2 ] and DALL-E [ 3 ] are two of the most popular Diffusion-based Models, especially for fine art image generation. These models are also called text-to-image models because text prompts can be provided as input explaining details of the expected output image. The capabilities of diffusion models have recently become so advanced that it has become increasingly hard to differentiate between AI-generated art and human-drawn art by only using a digital image, creating a startling debate between what is real and what is not. These abilities of generative models also pose a serious issue regarding the originality of digitalised art, as authors in [ 15 ] ask, “Who Gets Credit for AI-Generated Art?”. This study does not seek to provide a definitive answer to the mentioned question but aims to offer an effective toolkit for the broader research community. This toolkit will assist in making informed decisions about artworks and help evaluate their authenticity. This toolkit includes a web-based application integrated with a model developed as a part of this study and an AI-generated artistic dataset of 10 major art styles and 180,000+ samples. Furthermore, the study compares human abilities to distinguish between human and AI-generated art with the AI’s performance in this task. The main scientific contributions of this work are as follows; 1. A concise review of current research on AI-driven image generation and existing studies focused on detecting such images. 2. Introducing AI-ArtBench - a novel AI artistic dataset, which includes samples generated using two diffusion-based models. This is the only dataset of its kind available as of this document submission. 3. Novel CNN-based architecture, AttentionConvNeXt, based on ConvNeXt [ 16 ] and Attention mechanism [ 17 ] . 4. Web-based applications that can be used to verify the authenticity of artistic images via the developed models. 5. Evaluate the human ability to identify AI-generated art images through an ‘Artistic Turing Test’. This is the first Artistic Turing Test conducted using complete AI art images generated by diffusion models. The remainder of this paper is initiated with a review of the existing work in Section 2 concisely discussing the existing status of generation, detection and attribution of AI art. Then in Sections 3 and 4 the theories and techniques used in the implementation of the model and developed web-based application are explained followed by the evaluation results of the models in Section 5 . The paper concludes with a discussion of the limitations of the current study and an exploration of future work. 2 Related Work This section reviews the existing work on AI art generation and detection while investigating its performance. The section also presents major datasets that can potentially be used to train models for tasks of differentiating AI art from Human art. 2.1 Using AI to Analyse Art Many artistic styles have been identified throughout history that have been created by various artists. These are often called art movements. Art movements represent the artistic philosophies and ideas of contemporary artists who similarly saw art. Cubism [ 18 ] , Surrealism [ 19 ] , Impressionism [ 20 ] , Romanticism [ 21 ] , Minimalism [ 22 ] and Expressionism [ 23 ] are some examples of these styles. Most of these artistic styles were pioneered in the 19th and 20th centuries, predominantly based in the European region, except for movements like Ukiyo-e [ 24 ] which was formed in Japan. Datasets like [ 25 , 26 , 27 , 28 , 29 , 30 , 31 ] which includes artworks with various styles mentioned above, can be used to train AI models for a wide range of tasks, from

Read the complete text