Artificial Intelligence for Geometry-Based Feature Extraction, Analysis and Synthesis in Artistic Images : A Survey
Mridula Vijendran, Jingjing Deng, Shuang Chen, Edmond S. L. Ho, Hubert P. H. Shum · 2024
Read Artificial Intelligence for Geometry-Based Feature Extraction, Analysis and Synthesis in Artistic Images : A Survey by Mridula Vijendran, Jingjing Deng, Shuang Chen, Edmond S. L. Ho, Hubert P. H. Shum (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-01450.txt.
Introduction
Mridula Vijendran and co-authors survey how shape and geometry help artificial intelligence work with artistic images. Programs that sort or create digital art struggle because works in one class can look very different, styles drift across domains, and style is hard to pull apart from content. Adding geometric facts, such as edges, curves, and spatial layout, gives models firmer ground to stand on. The review covers ways to pull such facts out of paintings and drawings, and how people themselves read shape when they view art.
The authors also look at how geometry helps sorting tasks, judging quality, and making new images that stay true to a design. Better labeling methods can raise the quality of training data, and shape cues help models adjust to new material. They close by mapping where the field might go next, arguing that geometry makes art-focused AI both sharper and more flexible.
Catalog record
| CREATOR | Mridula Vijendran, Jingjing Deng, Shuang Chen, Edmond S. L. Ho, Hubert P. H. Shum |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 123 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-01450
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Artificial Intelligence for Geometry-Based Feature Extraction, Analysis and Synthesis in Artistic Images : A Survey Mridula Vijendran, Jingjing Deng, Shuang Chen, Edmond S. L. Ho, Hubert P. H. Shum Abstract Artificial Intelligence significantly enhances the visual art industry by analyzing, identifying and generating digitized artistic images . This review highlights the substantial benefits of integrating geometric data into AI models, addressing challenges such as high inter-class variations, domain gaps, and the separation of style from content by incorporating geometric information. Models not only improve AI-generated graphics synthesis quality, but also effectively distinguish between style and content, utilizing inherent model biases and shared data traits. We explore methods like geometric data extraction from artistic images , the impact on human perception, and its use in discriminative tasks. The review also discusses the potential for improving data quality through innovative annotation techniques and the use of geometric data to enhance model adaptability and output refinement. Overall, incorporating geometric guidance boosts model performance in classification and synthesis tasks, providing crucial insights for future AI applications in the visual arts domain . keywords Artificial intelligence, machine learning, feature extraction, geometrical analysis, content synthesis. 1 Introduction Figure 1 : T-SNE visualization of the art domain as compared to the real-world images using the PACS dataset. The art modalities with paintings, cartoons and sketches showcase the clustering of art modalities close to photos, exaggerated geometries and no color or texture respectively. The dimensionality reduction uses a pre-trained VGG-19 model as a feature extractor and removes the fully connected head. Artificial intelligence (AI) techniques find use in the art industry for tasks such as 3D scan analysis, art recommendation systems, identification of art design principles, deconstructivism art generation with fragment models Bellaiche et al. (2023) ; Hirsch et al. (2021) . These techniques mainly involve three key processes: extraction, analysis, and synthesis. Extraction methods classify paintings based on style, identify and authenticate artwork, and provide exhibit and tour information to establishments like museums Rani et al. (2023) and historic cathedrals Sklodowski et al. (2014) to enhance their visitors’ experience. Analysis techniques interpret and facilitate searching and comparing art elements such as geometric patterns of compositional elements across multiple scales in art collections Pintus et al. (2016) . Synthesis methods are used for scanning and enhancing the details of artifacts in the Cultural Heritage field Borg et al. (2020) ; Remondino et al. (2011) and deal with the preservation, documentation and collection of historical and cultural objects. They classify paintings based on style, identify and authenticate artwork, and provide virtual access to historic cathedrals to enhance their consumers’ experience. It is used for scanning and enhancing the details of artifacts in the Cultural Heritage field Borg et al. (2020) ; Pintus et al. (2016) ; Remondino et al. (2011) which deals with the preservation, documentation and collection of historical and cultural objects. This paper explores a broad spectrum of AI techniques applied to artworks, encompassing both deep learning and non-deep learning methods. While deep learning models, such as CNNs and GANs, have shown significant success in tasks like object detection and image synthesis, we also discuss traditional methods, including Deformable Part Models (DPM), Histogram of Oriented Gradients (HoG), and Thin Plate Spline (TPS) interpolation. These non-deep learning techniques are crucial for understanding specific geometric features and enhancing overall model performance when integrated with deep learning approaches. Learning from Art datasets using artificial intelligence models is challenging due to the generally smaller dataset size and larger inter-class variations Redmon et al. (2016) ; Mathieu et al. (2014) , as well as incomplete or inaccurate data annotations Milani et al. (2022) ; Crowley and Zisserman (2013) . This review paper broadly covers 3D art such as sculptures, archaeological sites and surface art such as walls, cloth or tattoos. Additionally, it also considers 2D forms such as paintings, sketches, digital art, cartoons, logos, anime and manga. Synthetic art through the stylization of real-world data without content separation commonly suffers from the bleeding of colors from the foreground to the background and the blurring of boundaries. Despite depicting the same content, the stylistic differences between various art media highlight the importance of separating style and content in model design for different art-related tasks. To illustrate these problems, we use t-distributed Stochastic Neighbor Embedding (T-SNE), which is a commonly used nonlinear dimensionality reduction algorithm to visualize embeddings and understand similar images from the dataset according to the model, VGG-19, embedding the data. From the T-SNE visualization in Figure 1 , we see that art domains of one type cluster closely, with some overlap for those sharing similar shape representations with cartoons and sketches. The large inter-class variations in the art modalities lead to their distributions looking roughly like anisotropic Gaussians, with their spread creating overlaps with images of similar subjects between the other classes. The stylization of real-world data without content separation leads to the bleeding of colors from the foreground to the background and the blurring of boundaries. Our paper discusses paintings, sketches, digital art, cartoons, logos, anime and manga as visual art media. It covers a variety of styles and genres such as abstract art, realism, impressionism, expressionism, surrealism, cubism, pop art. These styles range from those that resemble real world objects, leaning towards figurative art, to the other end of the spectrum using basic shapes and geometric deformations that diverge from reality. 1.1 Background Computational art is a field that finds applications in the visual art industry for modeling collections of art Özgün and Alaçam (2023) , simulating visual art for artwork exploration Stork (2006) , and replicating artworks for preservation Li et al. (2018) . As the field grows, it is essential to recognize that the distinction between human-created and AI-generated artwork remains subjective Bellaiche et al. (2023) ; Hertzmann (2018) ; Augello et al. (2016) . The papers discussed in the literature use datasets from the art domain for training, thus modeling the domain, even if the art dataset is a subtask or a subset of the training dataset at least. For clarity, we use the term ’AI-generated graphics’ to refer to AI-generated works, acknowledging the ongoing debate over whether AI art qualifies as visual artwork. Additionally, when using the terms ‘art’, ‘visual art’ and ‘artwork’ we refer to the input data collection. The survey paper discusses research where AI image synthesis and generation models result in different interpretations of artworks from existing styles, diversifying visual arts datasets Ernst (2023) or producing artistic counterparts to real-world images in the introduction and synthesis sections. AI tools assist in the extraction and analysis of growing collections of art to enhance interactive experiences while expanding interpretation across selected geometry priors in collections Fan and Liang (2023) ; Rani et al. (2023) . It evolved from a form of static procedural art, building from preset rules with random perturbations through models such as the AARON model in the 1960s. Its later iterations were more data-centric, allowing for generation of more dynamic art using computer graphics through non-photorealistic rendering techniques that distilled artistic styles to simple parameters such as brush strokes and other learned statistics forming popular techniques such as style transfer Hertzmann (2018) . These artworks were used to form interactive art