Modeling visual memorability assessment with autoencoders reveals characteristics of memorable images Authors: Elham Bagheri, Yalda Mohsenzadeh Published: 2026 ABSTRACT Image memorability refers to the phenomenon where certain images are more likely to be remembered than others. It is a quantifiable and intrinsic image attribute, defined as the likelihood of an image being remembered upon a single exposure. Despite advances in understanding human visual perception and memory, it is unclear what features contribute to an image’s memorability. To address this question, we propose a deep learning-based computational modeling approach. We employ an autoencoder-based approach built on convolutional neural networks (CNNs). We examine the relationship between autoencoder reconstruction error and memorability, analyze the distinctiveness of latent space representations, and train a multi-layer perceptron (MLP) on the encoder’s latent representations for memorability prediction. Additionally, we perform interpretability analysis using Integrated Gradients (IG) to visualize the key visual characteristics that are most influential in our memorability prediction model. Our results demonstrate a significant correlation between the images’ memorability score and the autoencoder’s reconstruction error, as well as the robust predictive performance of its latent representations. Latent space distinctiveness correlated significantly with memorability. Additionally, certain visual characteristics were identified as features contributing to image memorability prediction in our model. These findings suggest that autoencoder-based representations are associated with image memorability, providing new insights into the computational modeling of memorability. Introduction Understanding why certain images are more memorable than others is a key question in both cognitive science and computer vision. In a foundational study by Isola et al., over 2,000 scene images were presented to participants in a repeat-detection task to identify recurring images, defining memorability as the likelihood of an image being remembered after a single exposure. This study revealed significant variation in memorability scores among images, with some being more memorable than others. This variation was consistent across participants, as shown by Spearman’s rank correlation analysis (split-half rank correlation: r = 0.75), indicating intrinsic properties of images that make them memorable to a broad audience1. This consistency has been confirmed in several subsequent studies. Goetschalckx et al. confirmed that memorability is stable across different memory tasks and intervals2. Bainbridge et al. demonstrated the consistency of memorability across individuals and viewing conditions, particularly with faces3. Bylinskii et al. further validated that memorability is resilient to contextual changes4, while Khosla et al. confirmed its robustness across large datasets and participants5. More recently, Almog et al. showed that memorability remains consistent across individuals of varying ages6. Related work has also examined memorability prediction under category-aware training settings and cross-age transfer scenarios, further highlighting generalization across semantic groupings and observer populations7. These findings collectively establish memorability as an intrinsic, measurable characteristic of images, defined by the behavioral outcomes of memory. Despite its robustness as a psychological phenomenon, pinpointing precise attributes that contribute to an image’s memorability is complex4,8,9. Advancements in artificial intelligence (AI), particularly in deep learning, have provided strong tools not only to predict memorability1,5,10–13 but also to modify it14,15. Notably, Khosla et al. introduced MemNet, a CNN-based model that predicts image memorability by analyzing regions of images and generating memorability heatmaps5. Further advancements in the field include the work of Praveen et al., who proposed ResMem-Net, a novel deep learning architecture combining Long Short-Term Memory (LSTM) networks with CNNs. This architecture leverages information from the CNN’s hidden layers to compute an image’s memorability score, aiming to provide a more accurate estimation of memorability10. Similarly, Needell and Bainbridge evaluated several alternative deep learning models, including residual neural networks, to enhance memorability estimation11. Another notable advancement is ViTMem, a model presented by Hagen and Espeseth based on Vision Transformer (ViT) architectures. ViTMem was found to be particularly sensitive to the semantic content that drives memorability in images, marking a significant step forward in the precision of memorability prediction models12. In the domain of face memorability, Younesi and Mohsenzadeh proposed several state-of-the-art deep neural network architectures specialized in predicting the memorability of face photographs13. The authors further developed a method to modify and control the memorability of face images using the latent space of StyleGAN14. Their work demonstrated the effectiveness of using generative models to control and manipulate memorability in a targeted manner. Lin et al. developed a sparse coding model to analyze how compressed image representations relate to memory encoding, focusing on the reconstruction error of these representations. The model, based on ResNet-50 embeddings, explored general memory mechanisms, including how well sparse representations align with memorability scores. Although their approach provides insights into how perceptual processing interfaces with memory, it was not exclusively focused on visual memorability but rather on understanding broader memory processes in relation to perception, guided by the level-of-processing theory16. In this work, we develop a reconstruction-based computational framework that draws operational inspiration from visual memory experiments used to quantify image memorability. Specifically, we study whether reconstruction error and latent-space distinctiveness derived from autoencoder models are statistically associated with memorability scores, and whether autoencoder latent representations can support memorability prediction. To investigate the primary hypothesis, we evaluate a convolutional autoencoder implemented as a VGG16-based autoencoder17. We additionally test two ImageNet-pretrained reconstruction architectures (AutoencoderKL VAE18,19 and VQGAN20) as supplementary robustness comparisons under the same protocol, with summary results reported in the Supplementary Information. All models are fine-tuned end to end using an unsupervised reconstruction objective (i.e., a reconstruction loss), and we do not use any image categories or memorability labels during autoencoder fine-tuning. We use ImageNet initialization to start from a broadly useful visual representation. To examine the relationship between image structure and memorability, we analyze image reconstruction error and latent space distinctiveness, hypothesizing that images that are more atypical with respect to the model’s learned representation may show higher memorability scores. We further assess the predictive capacity of latent space representations by training a multi-layer perceptron to estimate memorability scores. We hypothesize that these learned representations capture visual information useful for distinguishing between memorable and less memorable images. To identify the specific image features most influential for the model’s memorability predictions, we employ Integrated Gradients (IG), an explainable AI technique that quantitatively and visually highlights the most influential image attributes. While many models have been developed to predict memorability scores from images, the goal of this study is not limited to proposing a new predictive framework from scratch. Instead, we leverage autoencoder-based reconstruction models to examine reconstruction error, latent-space distinctiveness, and attribution-derived feature importance as model-derived signals associated with memorability. Together, these analyses allow us to characterize visual properties that contribute to memorability prediction in our model, without treating them as cognitive mechanisms of human memory. Our findings may inform practical applications in areas such as content creation, advertising, and education, where understanding visual memorability is important. Methods Reconstruction-based modeling approach Autoencoders are neural networks primarily used for representation learning. By compressing input data into a lower-dimensional latent space and subsequently reconstructing it, autoencoders learn compact representations of the input. In this study, we use autoencoders in a single-pass reconstruction setting inspired by single-exposure memorability experiments, fine-tuning a reconstruction model for one pass over the dataset and then analyzing its behavior on the same images. In these experiments, image memorability is operationalized as the likelihood that an image will be remembered after a single exposure, typically measured using a repeat-detection task1,21. This single-pass fine-tuning setting is intended as an operational analogue to the single-exposure evaluation protocol used in memorability studies; we do not treat network optimization dynamics as a mechanistic model of human exposure or encoding. Autoencoders learn compressed representations optimized for reconstruction, and their reconstruction error can reflect how typical or atypical an image is with respect to the model’s learned image statistics. We therefore test whether higher reconstruction error and latent-space distinctiveness are statistically associated with higher memorability scores, treating these quantities as model-derived signals (predictive correlates) rather than cognitive mechanisms or direct proxies for human memory. Dataset We used the MemCat dataset, introduced by Goetschalckx et al21,22, comprising 10,000 images across five categories: animals, sports, food, landscapes, and vehicles. Each category contains 2,000 images. The dataset’s memorability scores were obtained through a repeat-detection memory game conducted on Amazon Mechanical Turk21,22. We used MemCat’s false alarm-corrected scores computed by correcting each image’s hit rate by the observer’s false-alarm rate (i.e., recognition performance corrected for spurious “repeat” responses), following the definition used in the original MemCat paper21. Figure 1 demonstrates the distribution of memorability scores per category for the MemCat dataset, which we observe is not homogeneous across categories.Fig. 1Distribution of false alarm-corrected memorability scores across the categories of the MemCat dataset. Distribution of false alarm-corrected memorability scores across the categories of the MemCat dataset. Model development For the reconstruction-based memorability analysis, we used a CNN-based VGG16 autoencoder initialized from weights pretrained on ImageNet using a reconstruction objective23,24. This pretraining is a form of self-supervised learning, in which the input image itself serves as the training target, and does not require ImageNet class labels. VGG16 is a well-established model in computer vision17, commonly used in tasks such as image classification and style transfer25. It utilizes deep convolutional neural networks (CNNs), which are loosely inspired by how the human visual cortex processes images. CNNs like VGG16 perform hierarchical feature extraction, starting with simple features such as edges and gradually capturing more complex patterns. The deep convolutional layers of VGG16 allow it to extract rich and efficient features from images, which aligns with our objective of understanding image memorability. These reconstruction-pretrained ImageNet weights provide a broad visual representation learned from large-scale natural image statistics, which we use as an initialization. The convolutional VGG16-based autoencoder used in this study consists of a symmetrical encoder and decoder designed to compress and reconstruct input images. The encoder mirrors the structure of the VGG16 model, leveraging its hierarchical deep convolutional network design to progressively reduce the spatial dimensions of the input image while extracting increasingly complex visual features. The encoder is composed of five stages of convolutional layers with ReLU activations, with each stage increasing the number of filters while reducing the spatial resolution through max-pooling operations. Specifically, the first stage applies two 3x3 convolutional layers with 64 filters, followed by max-pooling. The second stage utilizes two 3x3 convolutional layers with 128 filters and max-pooling. The third stage incorporates three 3x3 convolutional layers with 256 filters, while the fourth and fifth stages both employ three 3x3 convolutional layers with 512 filters each, with max-pooling after every stage. As the input image progresses through these stages, its spatial dimensions are reduced from the initial 224x224 to 7x7 in the final layer, with a depth of 512 channels. The output from this final convolutional block forms a latent space representation of size 7x7x512, which is subsequently flattened into a 25,088-dimensional vector. The encoder further processes this representation with two fully connected layers, each containing 4096 neurons and utilizing ReLU activations, with dropout applied for regularization to reduce overfitting. This encoding process compresses the input image into a compact yet information-rich latent vector that encapsulates both low-level features like edges and textures, and high-level abstractions like shapes and objects. The decoder, which mirrors the encoder, uses transposed convolutions (also referred to as deconvolutions) to progressively upsample the compressed latent representation back to the original image dimensions. The decoder starts with three transposed convolutional layers with 512 filters, followed by another three transposed convolutional layers with 512 filters, corresponding to the deepest layers of the encoder. The subsequent stages reconstruct the feature maps using two transposed convolutional layers with 256 filters, followed by two layers with 128 filters, and finally two transposed convolutional layers with 64 filters in the output layer. A sigmoid activation function is applied at the output of the decoder to constrain the pixel values between 0 and 1, which matches the input image range. In addition to the VGG16-based autoencoder, we evaluated two additional ImageNet-pretrained reconstruction architectures (AutoencoderKL VAE18,19 and VQGAN20) under the same one-epoch fine-tuning protocol. Their architectural details, latent extraction definitions, and summary correlation/distinctiveness results are reported in the Supplementary Information. Autoencoder fine-tuning We fine-tuned the autoencoder on the whole MemCat dataset for one epoch under the single-pass reconstruction protocol described above. Fine-tuning was performed using various loss functions, which we categorize into pixel-wise discrepancy, structural-similarity, perceptual and style-based objectives. The Mean Squared Error (MSE) was used to quantify pixel-wise discrepancies as the per-pixel squared \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\ell _2$$\end{document} difference between the reconstruction and the input. Additionally, Multi-Scale Structural Similarity (MS-SSIM)26, an extension of the Structural Similarity Index (SSIM)27, was employed to assess visual quality by analyzing local luminance, contrast, and structural agreement across multiple image scales. To evaluate perceptual similarity, the Learned Perceptual Image Patch Similarity (LPIPS) metric28 was used, which computes distances in deep feature spaces of fixed pretrained networks such as VGG17, SqueezeNet29, and AlexNet30. Furthermore, Style Loss (StyLoss)31 was employed to capture the stylistic elements of images by comparing Gram-matrix statistics of deep feature activations. These diverse metrics helped to identify the best-performing models and determine which loss functions yielded stronger associations with memorability in our modeling setup. The autoencoder was fine-tuned with a batch size of 1, and the image order was kept fixed across all experiments to ensure consistency. Learning rates ranging from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$10^{-7}$$\end{document} to 0.1 were tested to determine the optimal settings. After fine-tuning, the autoencoder was tested on the same set of images, and the reconstruction error for each image was calculated to measure the autoencoder’s ability to replicate the input after a single exposure. All experiments were repeated five times with different random seeds to ensure the robustness of the results. Memorability prediction from autoencoder latent representations We investigate the potential usefulness of autoencoder latent representations for predicting image memorability. We extract feature embeddings from the MemCat dataset by passing each image through the encoder component of our fine-tuned autoencoder. These embeddings provide a compressed representation of the input images learned for reconstruction, and we test whether they are predictive of memorability scores. We employed an MLP to develop predictive models for memorability classification on these embeddings. In practice, we attached an MLP head to the fine-tuned encoder, froze the encoder weights, and trained only the MLP head. The MLP is used for capturing nonlinear relationships in the latent space. A hyperparameter search was conducted to optimize model performance, evaluating learning rates between 0.01 and 0.00001 and batch sizes of 8, 16, 32, and 64. The models were trained using the AdamW optimizer, which incorporates decoupled weight decay for better generalization. To prevent overfitting, weight decay (L2 regularization) values of 1e-5, 1e-4, and 0.01 were explored. Additionally, dropout (ranging from 0.1 to 0.5) and layer normalization were incorporated to stabilize training and mitigate covariate shift. One to three hidden layers of 256 to 2048 neurons were tested. To ensure training efficiency and prevent overfitting, early stopping was applied with a patience of five epochs, halting training when validation loss stopped improving. These architectural and optimization choices ensured that the model effectively leveraged the encoder’s latent representations while maintaining robustness and avoiding overfitting. The MemCat dataset was partitioned into training, validation, and test subsets, consisting of 5892, 931, and 3177 images, respectively, ensuring a balanced distribution with a mean memorability score of approximately 0.693. The primary objective of this approach was to assess whether the autoencoder’s latent space contains features that are predictive of memorability scores. Success in this endeavor would indicate that these latent codes carry information that aligns with memorability. Model interpretation using integrated gradients To better understand what image features contribute to memorability prediction using the latent space representations, we employed Integrated Gradients (IG), an Explainable AI (XAI) technique that attributes the prediction of a neural network to its input features by computing the gradients of the output with respect to the input at points interpolated between a given baseline and the actual input32. This method is particularly suited for our complex architecture, which includes an autoencoder followed by a feed-forward model. The baseline input was set as a blurred mean image, computed by first averaging all training images and then applying a Gaussian blur (kernel size = 15). This ensures a smooth baseline representation that retains overall structure but removes fine details. IG attributes were computed by averaging the gradients over 50 steps between the baseline and the original image. These gradients provided a detailed map of feature contributions, helping us identify which aspects of the image were most influential for the model’s predicted memorability score. The analysis was conducted using the Captum library33 in PyTorch. IG attributions indicate which pixels or regions most influence the memorability predictor’s output; they are used here to interpret the model’s decision-making and to compare with known correlates of memorability, and do not provide experimental evidence about human memory mechanisms or causal features for memorability. Feature analysis We further explored which high-level image features relate to memorability scores and to the model’s attributions. The features we considered included semantic, structural, color-based, and texture-related properties. These features were analyzed by correlating their values with both memorability scores and Integrated Gradients (IG) attributions, allowing us to examine which measurable image properties were associated with the model’s predictions and with the original memorability scores. A pre-trained ResNet-50 model was used to extract deep features, summarizing the high-level visual information present in the images34. Specific metrics calculated included the number of distinct classes detected in each image, the total number of objects, and the average object size, computed as the mean area covered by detected objects. These measures provided descriptors of scene complexity and object diversity that could be compared with memorability scores and model attributions. For segmentation and object detection, we employed pre-trained DeepLabV3 and Faster R-CNN models35,36 to quantify object characteristics and scene complexity. The Separation Score, derived from DeepLabV3, measured how well objects stood out from the background, capturing foreground-background contrast, a property relevant to visual saliency and memorability-related image structure. Meanwhile, the Faster R-CNN model detected objects in the image, providing metrics such as the number of objects and their average size, which further informed analysis of object richness. Color features were extracted using the CIE-LAB and HSV color spaces, which better align with human perception. We computed mean values of the L (Lightness), A (green-red), and B (blue-yellow) channels to assess dominant color tones37. Saturation, representing color intensity, was measured in the HSV color space as a color-saliency feature relevant to memorability-related image structure. Color diversity, computed using Shannon entropy, quantified the variation in color composition, reflecting the presence of multiple hues and measuring visual richness. To analyze texture and structural complexity, we computed image entropy, which quantifies the randomness in pixel intensities, reflecting the information content in an image38. Texture energy, derived from the Gray Level Co-occurrence Matrix (GLCM), captured local variations in surface details, providing insight into the richness of textures39. The clutter score, calculated using edge detection via the Canny algorithm40, quantified the density of structural details and was used as a measure of structural complexity. Each feature was analyzed by correlating its values with average IG attributions from our trained model and with memorability scores. This analysis was used to examine which visual properties were associated with the model’s predictions and how these associations compared with the behavioral memorability scores. Results Higher memorability is associated with higher reconstruction error and latent distinctiveness To evaluate whether there is a difference in the model’s performance in reconstructing images with varying memorability, we calculated Spearman’s rank correlation between the images’ memorability scores and their reconstruction errors. Reconstruction error served as a quantitative measure to assess the autoencoder’s accuracy in reproducing the input data. Our results show a significant positive correlation between the autoencoder’s reconstruction error and memorability scores. Among the three tested autoencoders, the convolutional VGG-based model yielded the strongest results. The model fine-tuned on all MemCat images using the LPIPS (SqueezeNet) loss function with a learning rate of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$5 \times 10^{-6}$$\end{document} achieved the highest correlation. The Spearman’s rank correlation for this selected model was 0.452 (95% CI: [0.442, 0.467], \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p = 5 \times 10^{-310}$$\end{document}). This observation indicates that images with higher memorability scores tended to produce higher reconstruction errors in this model. Figure 2 illustrates the distribution of reconstruction errors across varying memorability scores with memorability scores binned at intervals of 0.2 from very low to very high memorability. The figure shows higher reconstruction error distributions for more memorable images; this pattern is generally observed across categories, but is less pronounced for the food category. Furthermore, we investigated the distinctiveness of images within the autoencoder’s latent space, hypothesizing that more distinctive images are more likely to be memorable. Distinctiveness in memorability research is typically defined as the Euclidean distance between a target image and its nearest neighbor16. However, this measure can be sensitive to the dataset’s size, as well as the scale and distribution of distances within a given dataset. To address this, we introduced a normalized metric, standardizing the distances to the nearest neighbor using Z-score standardization. This provides a more robust, dataset-independent metric of distinctiveness by reducing sensitivity to the absolute scale and spread of distances within a dataset, supporting stable and consistent comparisons across datasets of different sizes and preserving the interpretation of how unique each image is within its context. Our results show that latent-space distinctiveness, as measured using the normalized metric, exhibited a positive correlation of 0.31 (95% CI: [0.292, 0.328], \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p = 3 \times 10^{-223}$$\end{document}) with memorability scores. This indicates that images that are more distinctive in the model’s latent space (i.e., farther from their nearest neighbors) tend to have higher memorability scores. Figure 3 shows the distribution of distinctiveness for binned memorability scores. This figure shows the general trend of higher distinctiveness for more memorable images for all categories except the ’food’ category. The VGG-based model was selected for the remainder of the analysis in this paper. Summary correlation and distinctiveness results for the AutoencoderKL VAE and VQGAN models are broadly consistent with these findings and are reported in the Supplementary Information. In the reconstruction error analysis (Figure 2), the variability of reconstruction errors across memorability bins for food images is lower compared to other categories. While other categories show a clear increase in reconstruction error with memorability, the food category exhibits a more uniform distribution, indicating that the autoencoder’s reconstruction capability is less sensitive to variations in food image memorability. This could be attributed to the visual homogeneity of food images in terms of color, texture, and composition, which may limit the model’s ability to distinguish between memorable and less memorable images. Similarly, in the distinctiveness analysis (Figure 3), the food category shows a narrower range of distinctiveness scores across memorability bins. Although distinctiveness generally increases with memorability, this trend is subdued for food images. The limited variability in distinctiveness scores may stem from the memorability scores of food images being concentrated within a narrow range, as observed in Figure 1. This concentration may reduce the relative distinctiveness of food images within the dataset. Overall, the food category highlights the importance of category-specific features in the relationship between model-derived signals and memorability. The uniformity in visual characteristics of food images appears to weaken the correlation between memorability and both reconstruction error and distinctiveness. These findings suggest that the memorability of food images may rely on subtler cues that are not effectively captured by the current modeling approach. Future studies could explore alternative feature representations or loss functions tailored to homogeneous categories like food to enhance the model’s sensitivity to variations in memorability.Fig. 2Distribution of reconstruction errors across varying memorability scores for every category.Fig. 3Distribution of the normalized distinctiveness measure across varying memorability scores for every category. Distribution of reconstruction errors across varying memorability scores for every category. Distribution of the normalized distinctiveness measure across varying memorability scores for every category. The reconstruction–memorability association weakens with additional training epochs To examine how continued fine-tuning affects the observed correlation between model reconstruction error and memorability, the autoencoder was trained for up to 20 epochs. The correlation between reconstruction error and memorability decreased across additional training epochs. After 20 epochs, the correlation dropped to 0.22 (95% CI: [0.21, 0.24], \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p = 2.8 \times 10^{-114}$$\end{document}). This pattern indicates that continued optimization reduces differences in reconstruction difficulty across images under this training setup, thereby weakening the association with memorability. Although this is qualitatively consistent with reduced novelty under repeated optimization, training epochs are not equivalent to repeated human exposures and should not be interpreted as direct evidence about human memory. Perceptual losses yield stronger reconstruction–memorability associations By comparing all fine-tuned models, we evaluated the impact of different loss functions on the model’s ability to predict memorability, as detailed in Table 1. The table presents the correlations between reconstruction errors and memorability scores for each loss function, along with confidence intervals and learning rates of the best-performing model in terms of correlations. For each row, the reconstruction error used in the correlation is computed using the same criterion as the fine-tuning objective. Results show that using this model architecture, models trained with perceptual loss functions, especially LPIPS (SqueezeNet), followed by style-loss, showed stronger reconstruction–memorability associations than models trained with traditional pixel-wise metrics like MSE or structural metrics like MS-SSIM.Table 1Spearman’s rank correlation between reconstruction errors and memorability scores for models trained using different loss functions.Loss functionCorrelation95% CILearning rateLPIPS (SqueezeNet)0.452(0.442, 0.467)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$5 \times 10^{-6}$$\end{document}LPIPS (VGG)0.434(0.416, 0.444)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$5 \times 10^{-4}$$\end{document}Style Loss0.371(0.361, 0.388)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$5 \times 10^{-6}$$\end{document}LPIPS (AlexNet)0.331(0.321, 0.336)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1 \times 10^{-6}$$\end{document}Mean Squared Error (MSE)0.235(0.229, 0.249)0.1Multi-Scale SSIM0.152(0.135, 0.159)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1 \times 10^{-7}$$\end{document} Table 1Spearman’s rank correlation between reconstruction errors and memorability scores for models trained using different loss functions.Loss functionCorrelation95% CILearning rateLPIPS (SqueezeNet)0.452(0.442, 0.467)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$5 \times 10^{-6}$$\end{document}LPIPS (VGG)0.434(0.416, 0.444)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$5 \times 10^{-4}$$\end{document}Style Loss0.371(0.361, 0.388)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$5 \times 10^{-6}$$\end{document}LPIPS (AlexNet)0.331(0.321, 0.336)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1 \times 10^{-6}$$\end{document}Mean Squared Error (MSE)0.235(0.229, 0.249)0.1Multi-Scale SSIM0.152(0.135, 0.159)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1 \times 10^{-7}$$\end{document} Spearman’s rank correlation between reconstruction errors and memorability scores for models trained using different loss functions. Memorability can be predicted from autoencoder latent representations The best-performing model was selected based on performance on the validation set. The selected prediction model consists of a multilayer perceptron (MLP) head attached to the frozen encoder. The MLP has two hidden layers of 1024 and 512 neurons, trained with a learning rate of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$5 \times 10^{-5}$$\end{document}, a batch size of 8, a dropout regularization rate of 0.3, and layer normalization. The model was optimized using AdamW with a weight decay of 0.01 and trained with early stopping using a patience of 5 epochs. The model was trained for 23 epochs. According to the results, our memorability prediction model demonstrated robust performance in predicting memorability from the latent representations generated by the encoder. For binary classification of memorability, on the held-out MLP test set, the model achieved an area under the receiver operating characteristic (ROC) curve (AUC) of 0.76 (95% CI: [0.74, 0.78]) and a precision-recall (PR) AUC of 0.74 (95% CI: [0.71, 0.76]). Additionally, the model’s predicted scores correlated significantly with the original memorability scores with correlation = 0.56 (95% CI: [0.53, 0.58], \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p = 2.21 \times 10^{-258}$$\end{document}), supporting the effectiveness of latent representations in capturing memorability-related features. Using the median memorability score as the threshold for binary classification into high and low memorability, the model achieved an accuracy of 0.69 (95% CI: [0.68, 0.71]), a precision of 0.69 (95% CI: [0.67, 0.71]), a sensitivity of 0.70 (95% CI: [0.67, 0.72]), a specificity of 0.69 (95% CI: [0.67, 0.71]), and an F1 score of 0.69 (95% CI: [0.67, 0.71]). Model interpretation highlights image characteristics associated with memorability prediction We employed the Integrated Gradients (IG) method to interpret the predictions of our memorability model, visualizing which regions in an image contributed most to its predicted memorability score. Figure 4 illustrates the IG attributions for selected images from the MemCat dataset, where red regions indicate areas with higher attribution values. These regions are identified as the most influential for the model’s prediction of memorability. In the animal category, such as images of the rabbit, lion, and rooster, the IG heatmaps emphasize facial regions, particularly the eyes and prominent features like fur patterns. These patterns indicate that the model places strong weight on distinct facial features and textures when assigning higher memorability predictions. For example, the lion’s mane and the rooster’s sharp contours further underscore the importance of high-contrast and well-defined features in the model’s predictions. For food-related images, including yogurt, wine, coffee, and hot dogs, the IG attributions concentrate on vivid textures, strong contrasts, and central elements. The smooth surface of the wine glass and the detailed textures of the food toppings are highly attributed. This suggests that the model associates vibrant colors, intricate patterns, and centrally positioned salient elements with higher predicted memorability in food images. In landscape scenes, such as mountains, geysers, and vineyards, the IG heatmaps show strong attributions for horizon lines, separation between foreground and background, and specific details like trees or water surfaces. The prominence of natural structures and high-contrast boundaries suggests that these elements are important for higher memorability predictions in landscape scenes. For sports and activity images, such as basketball players, rowers, and runners, the IG attributions highlight human figures and interactions. The focus on human faces, gestures, and motion-related elements, such as the oars and leg movements, underscores the model’s sensitivity to dynamic, socially relevant features. This emphasis is consistent with prior findings that human presence and action are associated with memorability. In the vehicle category, including airplanes, trucks, and machinery, the IG maps emphasize distinct shapes and structural details, such as wheels, propellers, and edges of the vehicle. These findings suggest that the model associates clear geometric patterns and mechanical features with higher predicted memorability in vehicle-related imagery. Overall, the IG analysis reveals that the model consistently focuses on distinct, high-contrast regions, central objects, and unique features across all categories. Elements like bright colors, textures, and recognizable structures (e.g., faces, objects) appear to be important for the model’s memorability predictions. Furthermore, the attribution maps highlight the role of image composition, with the arrangement of features and their saliency appearing important for the predictor’s output. Notably, the distribution of attributions differs slightly between categories. While human-related and animal features often dominate their respective categories, natural landscapes and vehicles rely more on structural complexity and contrast. This variability suggests that the model’s predictions depend not only on individual features but also on the contextual arrangement and saliency of elements within an image.Fig. 4Integrated Gradients attributions for sample images from the MemCat dataset, showing regions influencing the memorability predictor outputs. The numbers on top of the images show the memorability score. Red areas indicate strong contributions to the model’s predictions. Each row corresponds to one of the five categories: animals, food, landscapes, sports, and vehicles, with images arranged in ascending order of memorability scores. Integrated Gradients attributions for sample images from the MemCat dataset, showing regions influencing the memorability predictor outputs. The numbers on top of the images show the memorability score. Red areas indicate strong contributions to the model’s predictions. Each row corresponds to one of the five categories: animals, food, landscapes, sports, and vehicles, with images arranged in ascending order of memorability scores. Feature importance in model attributions and associations with memorability scores The feature analysis combines two complementary quantities: IG attributions, which provide model-level feature importance for the predictor output, and correlations with behavioral memorability scores, which identify image properties associated with memorability in the dataset. Using these analyses, we identified several MemCat features that were both important for the model’s predictions and associated with memorability scores. The features with significant Spearman correlations with original memorability scores and IG attributions are listed in Table 2. The p-values indicate the statistical significance of these correlations. These findings provide insights into visual attributes that contribute to memorability prediction in our model, while remaining correlational with respect to behavioral memorability scores. The results indicate that object presence and arrangement are strongly associated with memorability. Images containing a greater number of distinct classes, a higher number of objects, and larger objects tend to be more memorable. This suggests that increased scene complexity and diversity may provide more distinctive visual cues. Larger objects may draw more visual attention, while distinct elements within an image may support richer and more distinguishable visual representations. Visual complexity also appears to be associated with memorability. The clutter score, derived from edge detection, quantifies the presence of structural details in an image. The positive correlation with memorability suggests that denser, detail-rich images are associated with higher memorability scores. Image entropy, which quantifies randomness in pixel intensities, further supports this observation. Higher entropy values indicate greater visual information content, reinforcing the idea that diverse textures and non-uniform compositions are associated with higher memorability. Color properties were also associated with memorability. The strong correlation between memorability and the LAB color channels suggests that color balance across green-red (A) and blue-yellow (B) axes is related to memorability scores. Higher values along these axes indicate distinct and contrasting color compositions, which may capture visual attention. Higher saturation was also positively correlated with memorability, indicating that vivid and intense colors are associated with higher memorability. Conversely, the negative correlation with mean color LAB L (lightness) suggests that excessively bright images may reduce contrast, making objects within them less distinguishable, which may relate to lower memorability scores. The separation score, measuring foreground-background distinction, was positively associated with memorability. This suggests that images where the main subject stands out clearly from the background tend to have higher memorability scores, potentially due to clearer object-background separation and reduced perceptual competition. Texture-related features exhibited mixed effects. Texture energy, which reflects uniformity in patterns, was negatively correlated with memorability, indicating that highly uniform textures may provide fewer distinctive texture cues. Higher levels of structural complexity, captured through clutter and edge density, were positively correlated, suggesting that diverse and complex textures provide additional visual cues relevant to memorability prediction and memorability scores. The comparison between IG attributions and original memorability scores revealed overlap in several feature patterns. Features that strongly correlated with memorability scores, such as the number of distinct objects, object size, and separation score, also exhibited high correlations with IG attributions. This suggests that the predictor assigns importance to several interpretable visual properties that are also associated with behavioral memorability scores. While most features maintained similar correlations across both measures, some variations were observed. Saturation and image entropy exhibited stronger correlations with IG attributions than with original memorability scores, indicating that the model assigns more importance to these properties than is reflected in their direct behavioral-score correlations alone. Conversely, some structural complexity measures, such as texture energy, had weaker correlations with IG attributions, suggesting that although these features contribute to memorability, they are not directly emphasized by the model in prediction tasks. Overall, these findings highlight that a combination of object properties, color composition, structural complexity, and foreground-background separation is important for interpreting the model’s memorability predictions and is also associated with behavioral memorability scores. The agreement between IG attributions and feature-score correlations suggests that the model emphasizes interpretable visual properties that are relevant to memorability prediction, while the observed relationships remain correlational.Table 2Spearman’s rank correlation of image features with Integrated Gradients attributions and original memorability scores, including 95% confidence intervals and p-values.FeatureIntegrated Gradients AttributionsMemorability ScoresCorr.95% CIp-valueCorr.95% CIp-valueMean color LAB A0.585[0.557, 0.608]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$6.47 \times 10^{-291}$$\end{document}0.347[0.314, 0.377]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.27 \times 10^{-90}$$\end{document}Distinct object categories0.531[0.506, 0.556]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.15 \times 10^{-230}$$\end{document}0.440[0.411, 0.471]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.32 \times 10^{-150}$$\end{document}Num. objects0.463[0.437, 0.489]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$9.79 \times 10^{-169}$$\end{document}0.350[0.316, 0.380]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$5.88 \times 10^{-92}$$\end{document}Mean color LAB B0.418[0.390, 0.450]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2.10 \times 10^{-134}$$\end{document}0.228[0.194, 0.260]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$9.32 \times 10^{-39}$$\end{document}Saturation0.413[0.384, 0.445]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2.05 \times 10^{-131}$$\end{document}0.157[0.122, 0.191]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$6.59 \times 10^{-19}$$\end{document}Average object size0.404[0.374, 0.432]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$9.29 \times 10^{-125}$$\end{document}0.480[0.449, 0.510]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$3.63 \times 10^{-183}$$\end{document}Separation score0.321[0.287, 0.351]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$6.69 \times 10^{-77}$$\end{document}0.273[0.241, 0.305]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.91 \times 10^{-55}$$\end{document}Color diversity0.276[0.244, 0.308]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.04 \times 10^{-56}$$\end{document}0.115[0.084, 0.154]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$9.42 \times 10^{-11}$$\end{document}Image entropy0.160[0.128, 0.193]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.21 \times 10^{-19}$$\end{document}0.102[0.066, 0.134]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$7.36 \times 10^{-9}$$\end{document}Clutter score0.136[0.102, 0.169]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.35 \times 10^{-14}$$\end{document}0.124[0.086, 0.158]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2.79 \times 10^{-12}$$\end{document}Mean color LAB L−0.123[−0.156, −0.088]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$3.61 \times 10^{-12}$$\end{document}−0.058[−0.092, −0.022]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$9.82 \times 10^{-4}$$\end{document}Texture energy−0.108[−0.140, −0.075]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.10 \times 10^{-9}$$\end{document}−0.078[−0.113, −0.046]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$9.57 \times 10^{-6}$$\end{document} Table 2Spearman’s rank correlation of image features with Integrated Gradients attributions and original memorability scores, including 95% confidence intervals and p-values.FeatureIntegrated Gradients AttributionsMemorability ScoresCorr.95% CIp-valueCorr.95% CIp-valueMean color LAB A0.585[0.557, 0.608]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$6.47 \times 10^{-291}$$\end{document}0.347[0.314, 0.377]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.27 \times 10^{-90}$$\end{document}Distinct object categories0.531[0.506, 0.556]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.15 \times 10^{-230}$$\end{document}0.440[0.411, 0.471]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.32 \times 10^{-150}$$\end{document}Num. objects0.463[0.437, 0.489]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$9.79 \times 10^{-169}$$\end{document}0.350[0.316, 0.380]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$5.88 \times 10^{-92}$$\end{document}Mean color LAB B0.418[0.390, 0.450]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2.10 \times 10^{-134}$$\end{document}0.228[0.194, 0.260]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$9.32 \times 10^{-39}$$\end{document}Saturation0.413[0.384, 0.445]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2.05 \times 10^{-131}$$\end{document}0.157[0.122, 0.191]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$6.59 \times 10^{-19}$$\end{document}Average object size0.404[0.374, 0.432]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$9.29 \times 10^{-125}$$\end{document}0.480[0.449, 0.510]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$3.63 \times 10^{-183}$$\end{document}Separation score0.321[0.287, 0.351]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$6.69 \times 10^{-77}$$\end{document}0.273[0.241, 0.305]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.91 \times 10^{-55}$$\end{document}Color diversity0.276[0.244, 0.308]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.04 \times 10^{-56}$$\end{document}0.115[0.084, 0.154]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$9.42 \times 10^{-11}$$\end{document}Image entropy0.160[0.128, 0.193]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.21 \times 10^{-19}$$\end{document}0.102[0.066, 0.134]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$7.36 \times 10^{-9}$$\end{document}Clutter score0.136[0.102, 0.169]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.35 \times 10^{-14}$$\end{document}0.124[0.086, 0.158]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2.79 \times 10^{-12}$$\end{document}Mean color LAB L−0.123[−0.156, −0.088]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$3.61 \times 10^{-12}$$\end{document}−0.058[−0.092, −0.022]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$9.82 \times 10^{-4}$$\end{document}Texture energy−0.108[−0.140, −0.075]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.10 \times 10^{-9}$$\end{document}−0.078[−0.113, −0.046]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$9.57 \times 10^{-6}$$\end{document} Spearman’s rank correlation of image features with Integrated Gradients attributions and original memorability scores, including 95% confidence intervals and p-values. Discussion The results indicate that, under our training setup, images that the model reconstructs with higher error tend to have higher memorability scores. Because reconstruction error can be influenced by both high-level content and low-level image properties, we interpret reconstruction error as a model-derived statistical signal rather than a direct measure of semantic meaning or a cognitive mechanism. In addition, we found that latent-space distinctiveness correlates with memorability, suggesting that images that are more atypical in the representation space learned by the model tend to be more memorable. Memorability is influenced by factors beyond visual statistics, including cultural context, emotional valence, personal relevance, and prior experience, as well as task and observer effects. Our reconstruction-based model does not represent these factors, and reconstruction error may be driven by low-level image properties (e.g., texture, illumination, high-frequency detail, or noise) that are not necessarily mnemonic in humans. Therefore, our findings should be interpreted as statistical alignment between model-derived signals and dataset memorability scores under our experimental setup, not as a mechanistic account of human memory. We further observed that the correlation between reconstruction error and memorability diminished as the model trained on the images for additional epochs. This result suggests that, within the model, the signals associated with memorability become less pronounced with continued optimization. Although this pattern is qualitatively consistent with ideas from cognitive psychology in which repeated exposure can reduce novelty and contribute to habituation, the comparison should be made cautiously, since training epochs are not equivalent to human repeated exposure. This finding generates a new testable hypothesis: changes in image novelty or distinctiveness may contribute to changes in memorability, potentially through reduced distinctiveness in the latent space. Future experiments can explore how varying levels of exposure affect the underlying features that contribute to memorability, providing deeper insights into the relationship between exposure, distinctiveness, and memory retention. Our approach supports previous studies that have suggested items that stand out from their context are better remembered4,41–50. However, unlike these earlier studies, our model enables an unsupervised, reconstruction-based, computational analysis of signals that align with memorability scores, complementing behavioral assessment methods with model-derived correlates of memorability, interpretability, and feature analyses. Our findings on the correlation of reconstruction errors and image memorability are closely related to the work by Lin et al.16. They introduced a sparse coding model that used reconstruction error and distinctiveness as metrics for memory formation and retention, focusing on general perceptual memory processes rather than image memorability specifically. While both studies identify reconstruction error as an important predictor of memorability, our approach differs in several ways. First, we apply autoencoders rather than sparse coding, allowing us to reconstruct images directly while analyzing their latent space representations. Second, our work focuses explicitly on image memorability rather than broader perceptual memory encoding, demonstrating that autoencoder reconstruction error is a predictive correlate of memorability scores in our setting. Finally, whereas Lin et al. focused on compressing feature embeddings extracted from an object recognition model, our model analyzes memorability scores as an image-level behavioral attribute and examines how they align with reconstruction-based representations. The positive correlation between the autoencoder reconstruction error and memorability score was consistently observed across the different loss functions we tested on the convolutional autoencoder. However, the highest correlation was achieved when perceptual losses (LPIPS family) were used for fine-tuning, followed by style loss, then pixel-wise loss (MSE), and finally structural-similarity losses (MS-SSIM/SSIM-based errors), which showed the weakest positive associations in our setting. This pattern indicates that the choice of fine-tuning objective and the reconstruction-error metric materially affects how strongly reconstruction-based signals align with memorability scores in this study. The reasonable accuracy of the prediction model on the autoencoder’s latent representations further indicates that reconstruction-trained latent representations contain information predictive of memorability scores. This supports the view that reconstruction-trained embeddings can align with memorability-related structure in the data, while not implying a mechanistic account of human memory. In addition, the use of Integrated Gradients (IG) for model interpretation provided new insights into the visual features most influential for the model’s memorability predictions, complementing prior predictive work with an explicit attribution-based analysis. By attributing memorability predictions to specific visual features, our results provide an interpretable view of which image regions most influence the memorability predictor, which we relate to established correlates of memorability. Our findings are consistent with several established studies in the field. For example, Isola et al. found that semantic content, such as the presence of objects and scenes, plays a significant role in predicting memorability, with certain types of content (e.g., human figures, interiors) being more memorable than others (e.g., natural scenes, backgrounds)1. Khosla et al. reported that memorable images tend to include distinct objects, people, and salient actions or events, while forgettable images are often characterized by natural landscapes or scenes that lack distinguishable features5. Bylinskii et al. also noted that overly simple or uniform images tend to be less memorable4. An exception to the observed trends was found in the food category, where the correlation between memorability and both reconstruction error and distinctiveness was lower than in other categories. Unlike objects, landscapes, or human figures, food images exhibit greater visual homogeneity in color, texture, and composition, which likely reduces their distinguishability in the model’s latent space. This suggests that food memorability may depend more on semantic, cultural, or personal associations rather than purely visual distinctiveness. Future work should explore incorporating multimodal embeddings (e.g., text-based descriptions, prior exposure modeling) to better capture non-visual memorability cues in such categories. Limitations and future directions While our findings are promising, there are certain limitations to our approach. First, the MemCat dataset, though diverse, may not fully capture the full range of image types and contexts that are relevant to memorability. Future studies should explore broader and more diverse datasets to validate and extend these findings. Second, the reconstruction architectures considered here, although informative for testing associations between reconstruction-based representations and memorability, do not exhaust the broader space of possible representation-learning models. Future work should examine additional architectures, including brain-inspired models, vision transformers, and other generative or self-supervised models, to evaluate how different representation-learning frameworks capture memorability-related visual structure. Third, while Integrated Gradients provided valuable insights into model interpretation, the choice of baseline images and the complexity of the model could influence the reliability of these interpretations. Future research should investigate additional explainable AI techniques to provide a more comprehensive understanding of how visual features influence memorability predictions. In addition, our conclusions are based on statistical associations and do not establish causality between any visual feature and human memory. The model also cannot capture important non-visual contributors to memorability, such as cultural context, emotional valence, personal relevance, or prior experience, and reconstruction error may reflect low-level image properties that are not necessarily mnemonic in humans. These factors limit the extent to which reconstruction difficulty should be interpreted beyond a model-derived correlate of memorability. Finally, future work should explore the broader application of these findings to real-world contexts, such as advertising, education, and media. While our work suggests potential uses for AI-assisted analysis of visual memorability, ethical considerations must be addressed, particularly in fields where memorability could be exploited to manipulate viewer behavior. Researchers and practitioners should follow ethical guidelines to ensure that AI systems designed to predict or modify memorability, or to guide memorability-oriented design, are used responsibly. Potential applications include AI-assisted content generation, advertising, and educational media,where understanding visual memorability may help improve engagement and information retention. In digital marketing, such systems could support the evaluation of ad creatives, while in education, they could help guide the design of learning materials that use visually memorable content. In medical imaging or scientific visualization, related approaches could potentially help identify visually salient regions for further expert review. By integrating cognitive science motivation with AI-based feature extraction, this study contributes to both theoretical understanding and practical applications in AI-driven memorability modeling. These findings may pave the way for novel computational tools to evaluate and enhance visual content based on memorability, ultimately improving user engagement, information retention, and human-computer interaction strategies. Supplementary Information Supplementary Information. Supplementary Information.