ViT-Transformer: Self-attention mechanism based constitutive modeling for nonlinear heterogeneous materials Journal: arXiv

Yijing Zhou, Shabnam J. Semnani · 2025

Read ViT-Transformer: Self-attention mechanism based constitutive modeling for nonlinear heterogeneous materials Journal: arXiv by Yijing Zhou, Shabnam J. Semnani (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-16575.txt.

Introduction

Yijing Zhou and Shabnam J. Semnani build a stand-in model for odd materials in a 2025 paper. Mixed materials bend and stretch in ways that defy plain rules. Full sims of each tiny cell cost far too much time. Their model, ViT-Transformer, learns the link from cell pattern to force reply. It reads cell images with a vision tool that weighs which parts count.

A masked decoder then joins shape codes with the strain path fed in. Strain is how much the sample is stretched or squeezed. A random-chunk training trick makes the model cope with paths of any length. Tests on varied cell images and load paths show sharp, true replies. Fast stand-ins let engineers test many designs in a day.

Catalog record

CREATORYijing Zhou, Shabnam J. Semnani
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-16575

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ViT-Transformer: Self-attention mechanism based constitutive modeling for nonlinear heterogeneous materials Journal: arXiv Yijing Zhou, Shabnam J. Semnani Abstract Multi-scale simulations of nonlinear heterogeneous materials and composites are challenging due to the prohibitive computational costs of high-fidelity simulations. Recently, machine learning (ML) based approaches have emerged as promising alternatives to traditional multiscale methods. However, existing ML surrogate constitutive models struggle in capturing long-range dependencies and generalization across microstructures. The recent advancements in attention-based Transformer architectures open the door to a more powerful class of surrogate models. Attention mechanism has demonstrated remarkable capabilities in natural language processing and computer vision. In this work, we introduce a surrogate (meta) model, namely ViT-Transformer, using a Vision Transformer (ViT) encoder and a Transformer-based decoder which are both driven by the self-attention mechanism. The ViT encoder extracts microstructural features from material images, while the decoder is a masked Transformer encoder that combines the latent geometrical features with the macroscopic strain input sequence to predict the corresponding stress response. To enhance training, we propose a random extract training algorithm that improves robustness to sequences of variable length. We design and construct a compact yet diverse dataset via data augmentation, and validate the surrogate model using various composite material images and loading scenarios. Several numerical examples are provided to show the effectiveness and accuracy of the ViT-Transformer model and the training algorithm. 1 Introduction Composites materials are ubiquitous in many engineering applications such as mechanical and aerospace engineering. These materials have a complex microstructure which determines their mechanical behavior, making accurate simulation of composites a challenging task. Moreover, prediction of composite behavior from the microstructural information is necessary for design of composites with enhanced properties. Conventional finite element method (FEM) requires exceedingly fine mesh to capture the material microstructure, which renders high-fidelity computational simulations infeasible. Therefore, various multiscale methods kanoute2009multiscale , fish2021mesoscopic , semnani2020inelastic have been developed to allow for capturing the microstructural effects on the macroscopic material response. However, computational multiscale methods, e.g. FE^{2} method feyel2000fe2 , are prohibitively expensive for path-dependent behavior of composite materials. Therefore, there is a need for novel approaches which can link the microstructures of nonlinear composite and heterogeneous materials to their macroscopic responses. The emergence of artificial intelligence has introduced new opportunities in computational mechanics. Machine learning (ML) based techniques have demonstrated promising results in acceleration of multiscale simulations logarzo2021smart , xu2020data , kim2024data , zhou2025machine . One of the common approaches for this purpose is to develop trained ML surrogate models or metamodels that eliminate the need for expensive high-fidelity simulations by learning inherent nonlinear relationships. Artificial neural networks (ANNs) represent the earliest and most widely adopted machine learning architecture for surrogate modeling of representative volume elements (RVEs). They have been applied to constitutive responses such as plasticity jang2021machine , viscoplasticity furukawa1998implicit , benabou2021implementation , li2019machine , and to capture the behavior of polycrystalline metals ali2019application and hyperelastic crystal structures im2021neural . In recent years, attention has turned to more advanced ML architectures, such as graph neural networks (GNNs), which have shown promise both in accelerating finite-element analyses gulakala2023graph , jiang2023graph , maurizi2022predicting and in predicting material properties dai2021graph . Recurrent Neural Networks (RNNs) have proven highly effective in capturing path-dependent and rate-dependent material responses gorji2020potential , he2023machine , li2025molecular . The hidden state embedded in RNNs serves as an internal memory that preserves information from preceding steps. This feature enables RNNs to excel at handling time-series data and step-wise predictions. To address vanishing-gradient issues, RNN variants such as Gated Recurrent Units (GRUs) and Long Short-Term Memory (LSTM) networks were developed, which have been applied to simulating elasto-plastic he2023machine , im2021surrogate , guan2023neural , heidenreich2024recurrent , tancogne2021recurrent , dettmer2024framework , heidenreich2024transfer and viscoelastic chen2021recurrent behaviors. Various researchers have applied RNNs to learn path-dependent constitutive laws within homogenization-based multiscale schemes WU2020113234 , ghavamian2019accelerating , logarzo2021smart , haghighi2022single , wu2024self , kim2023surrogate , vijayaraghavan2023data , friemann2023micromechanics , zhou2025machine . More recently, advanced techniques have been introduced to reduce training data requirements and enhance performance, including approaches based on physics-informed ML tandale2022physics , maia2023physically , borkowski2022recurrent , deng2024data , qu2021towards , masi2022multiscale , koric2024deep , neural differential equations he2024incremental , and deep operator networks koric2024deep . Despite extensive progress, ML-aided simulation techniques face various challenges. RNN-based surrogate models exhibit performance degradation when processing very long sequences because they rely on hidden-state propagation to capture historical information and long-range dependencies. The success of large language models naveed2025comprehensive in natural language processing has highlighted the Transformer architecture’s strong capacity for modeling long-range dependencies NIPS2017_3f5ee243 . Transformer architectures have been employed in various applications such as large-scale physical-systems simulation zhdanov2025erwin , crystalline-material property prediction NEURIPS2022_6145c70a , protein structure prediction jumper2021highly , gene-expression prediction avsec2021effective , and global weather forecasting lam2023learning . Considering their promise in modeling long sequences and parallelizability, researchers have begun to explore the applications of Transformer-based models in computational mechanics for constitutive modeling zhongbo2024pre , pitz2024neural , field prediction buehler2022fieldperceiver , buehler2022end , and de novo design of architected materials yang2021words . In stress sequence prediction tasks, Zhongbo and Hien zhongbo2024pre employed an encoder-only architecture to directly process homogenized strain sequences, while Pitz and Pochiraju pitz2024neural integrated Transformers with convolutional neural networks (CNNs) to handle homogenized data derived from RVEs of different geometries. Transformer-based models have been utilized to predict multiscale physical fields and nonlinear material properties buehler2022fieldperceiver without reliance on convolutional layers. Additionally, attention mechanisms have been introduced to predict stress fields and fracture patterns buehler2022end . With the rapid development of computer vision techniques, convolutional neural networks (CNNs) have been applied to the analysis of composites and heterogeneous media to learn the relationship between microstructural features and macroscopic stress responses. For example, Bhaduri et al. BHADURI2022109879 and Gupta et al. gupta2023accelerated mapped fiber layouts directly to stress fields using CNNs to expedite multiscale composite simulations. Subsequent studies extended this framework to elasto-plastic matrices saha2024prediction and to viscoplastic polycrystalline materials khorrami2023artificial . Additionally, CNNs have been employed to predict effective properties of composite materials su2023three , chang2022predicting , peng2022ph . Existing homogenization-based surrogate models often have limited capability to generalize to unseen microstructures and require re-generating training datasets and re-training models when material microstructure or properties change. Some researchers have proposed using CNNs as feature encoders to extract the microstructural information of composites materials pitz2024neural . Recently, Vision Transformers (ViTs) dosovitskiy2020image have gained increasing attention in computer vision, and evidence suggests that their performance can surpass CNN-based models with less computational requirements for training dosovitskiy2020image , liu2021swin , touvron2021training . The adoption of ViT architectures within computational mechanics, however, remains largely unexplored. In this work, we introduce ViT-Transformer, a surrogate model that combines a Vision Transformer encoder responsible for capturing microstructural information from RVE images with a Transformer decoder that predicts macroscopic stresses using the encoder’s latent features and an input sequence of macroscopic strains. The resulting model shows high accuracy and strong generalization across diverse composite microstructures. To further improve robustness, we devise a novel training algorithm that enhances the model’s ability to handle sequences of varying lengths without modifying the underlying dataset. We also provide a detailed data generation protocol and present a set of experiments to validate the performance of the proposed model in unseen scenarios. The proposed model

Read the complete text