Investigating the Gestalt Principle of Closure in Deep Convolutional Neural Networks

Yuyan Zhang 1, Derya Soydaner 2,3, Fatemeh Behrad 2,3, Lisa Koßmann 2, and Johan Wagemans 2,3 1- KU Leuven, Department of Computer Science, Leuven, Belgium 2- KU Leuven, Department of Brain and Cognition, Leuven, Belgium 3- Leuven.AI, KU Leuven Institute for AI, Leuven, Belgium · 2024

Read Investigating the Gestalt Principle of Closure in Deep Convolutional Neural Networks by Yuyan Zhang 1, Derya Soydaner 2,3, Fatemeh Behrad 2,3, Lisa Koßmann 2, and Johan Wagemans 2,3 1- KU Leuven, Department of Computer Science, Leuven, Belgium 2- KU Leuven, Department of Brain and Cognition, Leuven, Belgium 3- Leuven.AI, KU Leuven Institute for AI, Leuven, Belgium (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-00627.txt.

Description

Abstract Deep neural networks perform well in object recognition, but do they perceive objects like humans? This study investigates the Gestalt principle of closure in convolutional neural networks. We propose a protocol to identify closure and conduct experiments using simple visual stimuli with progressively removed edge sections. We evaluate well-known networks on their ability to classify incomplete polygons.

Catalog record

CREATORYuyan Zhang 1, Derya Soydaner 2,3, Fatemeh Behrad 2,3, Lisa Koßmann 2, and Johan Wagemans 2,3 1- KU Leuven, Department of Computer Science, Leuven, Belgium 2- KU Leuven, Department of Brain and Cognition, Leuven, Belgium 3- Leuven.AI, KU Leuven Institute for AI, Leuven, Belgium
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE12 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2411-00627

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Investigating the Gestalt Principle of Closure in Deep Convolutional Neural Networks Yuyan Zhang 1, Derya Soydaner 2,3, Fatemeh Behrad 2,3, Lisa Koßmann 2, and Johan Wagemans 2,3 1- KU Leuven, Department of Computer Science, Leuven, Belgium 2- KU Leuven, Department of Brain and Cognition, Leuven, Belgium 3- Leuven.AI, KU Leuven Institute for AI, Leuven, Belgium Abstract Deep neural networks perform well in object recognition, but do they perceive objects like humans? This study investigates the Gestalt principle of closure in convolutional neural networks. We propose a protocol to identify closure and conduct experiments using simple visual stimuli with progressively removed edge sections. We evaluate well-known networks on their ability to classify incomplete polygons. Our findings reveal a performance degradation as the edge removal percentage increases, indicating that current models heavily rely on complete edge information for accurate classification. The data used in our study is available on Github . 1 Introduction Neural networks were designed inspired by the working mechanism of the human brain and have since achieved remarkable success across various fields. While psychology still aims to better understand the human brain, computer science strives to enhance understanding of neural networks. The primary goal in neural network research is to develop models capable of performing similar tasks as the human brain, rather than to recreate it. Interestingly, neural networks tend to exhibit more human-like behavior than expected, despite not being explicitly designed for this purpose. In particular, recent findings suggest that CNNs may exhibit certain aspects of Gestalt laws of perceptual organization [ 1 ] , which explain how the human brain interprets complex visual stimuli, albeit possibly subject to certain thresholds and limitations. The narrow scope of previously explored neural network architectures, coupled with limited datasets and insufficient experiments, necessitates a more comprehensive investigation. We focus on the Gestalt principle of closure, which states that the human brain naturally fills in gaps to perceive figures as complete wholes, when parts are occluded or fragmented. We present a dataset specifically designed to examine closure from various psychology-based perspectives and conduct experiments across a broad range of CNNs to investigate their alignment with this principle. Our work provides a comprehensive analysis of CNNs regarding closure, identifying limits and thresholds that define their applicability in performing closure over gradually manipulated stimulus classes. 2 Preliminaries 2.1 Gestalt laws of perceptual organization Human visual perception has multiple, hierarchical processing stages. At the low-level processing stages, the visual system encodes features like luminance and color, at the mid-level stages elements are grouped and organized, and at the high-level stages tasks of identifying and understanding are performed. One significant contribution to research on visual perception was the formulation of the principles of perceptual grouping, often also referred to as “Gestalt Laws”. Grouping describes the phenomenon that observers perceive certain elements as belonging to each other compared to others, which appear distinct. The classic laws of grouping are: proximity, similarity, common fate, good continuation, symmetry, parallelism and closure [ 2 ] . 2.2 Law of closure The law of closure refers to a shared human preference for closed shapes over disjoined individual elements. This leads the visual system to complete contours and to create closed shapes. Depending on the elements and mechanisms at play, this can be referred to as contour integration or contour completion. Contour integration occurs when distinct elements such as dots or lines are integrated into the contour of a shape [ 2 ] . Contour completion refers to the integration of smooth contours [ 2 ] . If a shape or scene is completed behind an occluder, this is referred to as amodal completion. If the perception of an illusory contour or surface is triggered by stimulus characteristics, this can be referred to as modal completion [ 3 ] . The distinction between the two is made by the presence or absence of certain visual qualities in the completion [ 3 ] . 3 Related Work Amanatiadis et al. [ 4 ] trained AlexNet [ 5 ] and Inception V1 (GoogLeNet) [ 6 ] on the MNIST [ 7 ] and ImageNet [ 8 ] datasets to explore six core Gestalt laws. They measured closure as a function of occlusion percentage and found that the closure principle is effective in the CNNs up to approximately 30% occlusion, beyond which the models’ performance decreases. Another study examined a wider range of models including ResNet-152 [ 9 ] and DenseNet-201 [ 10 ] , and found mixed evidence of perceptual grouping [ 11 ] . Ehrensperger et al. [ 12 ] analyzed how AlexNet and Inception V1 classify Kanizsa triangles (a famous example of modal completion) and modified triangles with sections of the edges removed, reporting closure. Kim et al. [ 13 ] showed that both Inception and a simple CNN, trained to classify natural images, exhibit closure on synthetic displays of edge fragments. They tested these CNNs on incomplete triangles and demonstrated that they were more likely to be recognized as similar to complete triangles rather than disordered fragments. Baker et al. [ 14 ] fine-tuned AlexNet for a shape discrimination task, training it to classify wireframes and Kanizsa squares as ‘fat’ and ‘thin’. In contrast to earlier work, they concluded that neural networks do not perceive illusory contours. With limited stimuli and metrics employed on only a few CNNs, making a definitive statement about the presence of closure is not possible. Our method trains on complete polygons and tests on incomplete ones. Our approach, along with our examination of a broader range of CNN models, provides a more comprehensive assessment of closure performance in CNNs. 4 Convolutional neural networks CNNs have greatly advanced computer vision by effectively learning spatial features from images through a series of convolutional layers. Notable CNN architectures have emerged, each characterized by unique design choices and strengths. Early CNN models like AlexNet [ 5 ] and VGG16 [ 15 ] achieved impressive accuracy but suffered from high computational complexity due to their reliance on numerous small convolutional filters stacked together. Recent advancements, exemplified by architectures such as ResNet, SqueezeNet [ 16 ] , and DenseNet, have addressed this challenge by introducing innovative techniques such as residual connections, and efficient filter design. Moreover, Inception network [ 6 ] and ShuffleNet [ 17 ] improve accuracy by incorporating filters of varying sizes within a single layer and leveraging channel shuffle operations, respectively. EfficientNet [ 18 ] employs a compound scaling method to optimize network depth, width, and resolution according to specific resource constraints. MobileNet [ 19 ] , with its lightweight design and depth-wise separable convolutions, is particularly suitable for deployment on mobile devices. In our experiment, we freeze all layers of the mentioned CNNs. The last layer is replaced with a fully-connected layer with 10 nodes corresponding to 10 different polygons in the dataset. The networks are fine-tuned for 25 epochs with a batch size of 32. We use the Adam optimizer [ 20 ] with a learning rate of .001 and cross-entropy loss function. 5 Experimental results 5.1 Dataset For the experiment, we generated a novel dataset with a training set (Fig. 1 (a)) consisting of 320 complete polygons, which vary in the number of sides (3 to

Read the complete text