DDNet: disaster damage detection for buildings based on dual-temporal joint attention network Authors: Xiaosan Ge, Lin Zhou, Di Meng Published: 2025 ABSTRACT Rapid and accurate assessment of building damage is crucial for effective post-disaster emergency response. The use of pre-disaster and post-disaster satellite imagery is a common approach for detecting building damage. This task involves two essential subtasks: building localization and damage classification. In building localization, the imbalance between buildings and background, along with low recall rates, often leads to boundary deviations, which negatively impact the accuracy of subsequent damage classification. In damage classification, features from both pre-disaster and post-disaster images, combined with localization results, are used; however, variations in imaging modalities and insufficient feature extraction from temporal images can introduce interference and reduce classification performance. To address these challenges, we propose a novel two-stage network, referred to as DDNet. In the first stage, the building localization network utilizes differential upsampling connections to enhance detailed feature acquisition and employs a unified focal loss to mitigate class imbalance between buildings and background, thereby balancing precision and recall. In the second stage, a joint attention module is introduced to effectively mine features from pre-disaster and post-disaster images, leading to improved classification accuracy. Finally, a connected component analysis algorithm is applied to convert pixel-level detection results into building-level damage outputs. On the xBD dataset, the proposed framework achieves a total F1 score of 79.56%, an F1 localization score of 86.38%, and an F1 damage classification score of 76.64%. Introduction Natural disasters with high destructive potential, including earthquakes, hurricanes, floods, and tsunamis, can result in significant loss of life and extensive infrastructure damage1,2. In such contexts, the effectiveness of post-disaster emergency response becomes critically important. Rapid and accurate assessment of building damage is essential for emergency responders to capture the spatiotemporal characteristics of disaster impacts3. Traditional on-site building damage surveys provide highly reliable information but are time-consuming and labor-intensive when applied over large areas. In contrast, remote sensing technology offers rapid, accurate, and large-scale observation capabilities4–6. With the continuous advancement of remote sensing techniques, numerous methods for building damage detection have been developed7–9 Methods based on single post-disaster images face inherent limitations: without complete building outlines for comparison, these approaches often fail to accurately localize building areas, resulting in inevitable detection errors10–12. Approaches that leverage both pre-disaster and post-disaster remote sensing images utilize pre-disaster imagery to identify changes in building areas and types across the image pairs. However, manually designed feature extraction methods for pre-disaster and post-disaster image changes often rely on empirical heuristics and struggle to capture the most discriminative and effective features. Deep learning eliminates the reliance on hand-crafted image features, enabling automatic learning of representative features from low to high levels. Xu et al.13 compared the performance of four different CNN architectures for detecting building damage caused by the Haitian earthquake. Presa-Reyes et al.14 employed a dual-branch CNN to establish correspondences between pre-disaster and post-disaster image features, categorizing building damage into four levels. Weber et al.15 treated building damage detection as a semantic segmentation task, in which a dual-branch CNN connected pre-disaster and post-disaster features in its final layer for building damage segmentation. UNet, a widely used CNN-based semantic segmentation network, introduces skip connections to the encoder-decoder structure, effectively integrating low- and high-level features and mitigating information loss during sampling. Due to these advantages, UNet has been extensively applied in remote sensing image processing tasks, including building extraction16, road extraction17, change detection, and damage assessment18–20. Despite these strengths, CNNs generally lack a global understanding of images and have limited ability to model non-local relationships between pixels. While UNet enhances feature representations through skip connections, it often introduces redundant low-level features. Hao et al.21 incorporated a self-attention module into each branch of a dual-branch network to improve damage-level classification by considering information from the entire input image. Wu et al.19 evaluated various backbone networks with different attention mechanisms and introduced attention gates to high-resolution feature maps connected via skip connections. These self-attention-based enhancement methods are effective at capturing global information; however, merely using attention to re-weight fused features across channels remains insufficient for modeling long-range dependencies. YOLOv822, designed to properly detect buildings and evaluate building damage levels in satellite pictures. YOLO, renowned for its real-time object detection capability, was employed in experiments using YOLOv8 Medium and YOLOv8 Nano on a customized annotated building dataset. The evaluation results are as follows: Faster R-CNN (with RESNET-50) achieved a validation accuracy of 82%, while YOLOv8 Medium and YOLOv8 Nano achieved validation accuracies of 54% and 52%, respectively23. Sofia Tilon et al.24 utilized the xBD dataset, which contains pre-disaster and post-disaster satellite imagery across multiple disaster types, and proposed the use of a state-of-the-art Anomaly Detecting Generative Adversarial Network (ADGAN). The results indicate that, in the European region, the best-performing model achieved a recall of 0.59, a precision of 0.97, and an F1-score of 0.74. The transformer architecture, introduced in 2017, has demonstrated outstanding performance in natural language processing. Equipped with a global self-attention mechanism, transformers possess strong capabilities for modeling long-range dependencies and global context. In numerous computer vision tasks, including image classification25, object detection, semantic segmentation26–29 and super-resolution30, transformers achieve results comparable to or surpassing those of state-of-the-art CNNs. The remarkable performance of transformers, especially when combined with CNNs, has facilitated their successful application in change detection31–33 and building damage assessment34. The objective of change detection is to assign binary labels—change or no change—to each pixel within a region by comparing co-registered images of the same area captured at different times35. In contrast, building damage detection typically involves comparing pre-disaster and post-disaster images to identify building locations and classify the severity of damage sustained. Damage detection must address two key tasks: building localization and building damage classification. Building localization aims to delineate building areas, whereas building damage classification focuses on categorizing the extent of damage based on the building locations. Change detection methods generally utilize both pre-disaster and post-disaster images for learning. However, post-disaster imagery can introduce interference, leading to suboptimal building localization. While change detection primarily identifies changed and unchanged regions, applying it directly to damage detection presents limitations. Pseudo-damage results derived from changed regions may include non-building objects such as fallen trees or damaged roads, while unchanged regions typically contain intact buildings. Therefore, directly employing a change detection network is insufficient to fully meet the requirements of building damage assessment. Building damage detection can be divided into two subtasks: building localization and building damage classification. Building localization is performed to obtain accurate building boundary information. Cao et al.36 proposed a method that combines remote sensing imagery with previously observed geographic features, transforming the damage detection task into a multiclass classification problem for known building locations. However, methods relying on known building positions face significant limitations in detecting building damage. Alternatively, unknown building localization methods utilize the complete architectural characteristics present in pre-disaster images to differentiate buildings from the background. This approach often results in an imbalance between positive and negative samples, which can introduce classification biases during model inference. Such biases manifest as both false positives (FPs) and false negatives (FNs). An FP occurs when background pixels are incorrectly classified as building pixels, leading to an overestimation of the building boundary. Conversely, an FN occurs when building pixels are misclassified as background, resulting in an underestimation of the building boundary. To address the imbalance between positive and negative samples, Chen et al.34 introduced the Dice loss, which emphasizes the consistency between labels and predictions. The Dice loss generally outperforms the cross-entropy loss in scenarios with imbalanced positive and negative samples, as the latter treats all pixels equally. However, the Dice loss can be unstable, often resulting in incomplete training boundaries. Xie et al.37 proposed incorporating a contrastive loss along with a boundary loss to improve building localization. Nevertheless, the boundary loss suffers from similar instability as the Dice loss, and the benefits of the contrastive loss are limited. More recently, the unified focal loss38, was proposed, combining the strengths of the focal loss and the focal Tversky loss39. This approach balances recall and precision through the focal Tversky component while addressing class imbalance via both the focal loss and focal Tversky loss. Additionally, an offset parameter is employed to enhance positive sample learning and suppress background classification, thereby improving the recall rate of positive examples. For building damage classification, the effective utilization of pre-disaster and post-disaster image information is a critical concern. Siam-U-Net-Attn-diff40 performs subtractive fusion on dual-branch features, which can alleviate interference caused by irrelevant changes. However, simple subtraction reduces the distinctions between visually similar damage categories, such as no damage and slight damage, thereby increasing the difficulty of classification. Wu et al.19 did not fully account for the correlations between dual-temporal images in their dual-branch feature design, which limits the network’s ability to focus on the true regions of interest. Deng and Wang20 introduced an attention mechanism to perform weighted fusion of pre-disaster and post-disaster images, but their approach did not consider interference from uncorrelated changes, such as meteorological variations, changes in illumination, or differences in imaging angles. Therefore, effective building damage classification requires not only exploiting the correlations between pre-disaster and post-disaster images but also suppressing interference from unrelated changes. Building damage classification is derived from pixel-level damage detection. The damage experienced by each building can vary, with a single building potentially containing multiple damaged areas, each exhibiting different levels of severity. The primary goal of building damage detection is to obtain information regarding the extent of damage sustained by individual buildings, making it essential to optimize the classification results. The connected component analysis method functions as a voting algorithm, assigning unique labels to all pixels within each connected component (i.e., each object). This enables labeling of individual buildings based on pre-disaster images, analysis of the damaged areas within each building, and the assignment of a single damage level to each building. In summary, existing methods for disaster damage detection still face several critical challenges in practical applications. First, building localization often suffers from insufficient accuracy, particularly for densely distributed or small-scale structures, where low recall rates and boundary inaccuracies are common. Second, variations between pre-disaster and post-disaster images—such as differences in illumination, season, weather, and imaging angles—introduce substantial non-disaster-related interference, thereby reducing the robustness of damage classification. Third, the distribution of damage categories exhibits a pronounced long-tail effect, leading to suppressed recognition performance for both minor and severe damage classes. To address the aforementioned challenges, this paper proposes a two-stage building damage detection method that combines a CNN and a transformer, referred to as DDNet. First, for building localization, the TransUNet architecture is employed. A D-Cat module is incorporated into the skip connections to differentially enhance shallow and deep feature maps, reducing redundant information between layers while increasing the amount of detailed information acquired. The focal Dice loss and focal loss are combined to form a unified focal loss, and an offset parameter is applied to both building and background classes to obtain more accurate building boundaries and effective coverage areas. Second, for damage classification, a dual-branch skip network structure is utilized. A joint attention module is integrated into the feature interaction process to mine correlation features between pre-disaster and post-disaster images, thereby suppressing interference caused by irrelevant changes during imaging. Finally, connected component analysis is applied to improve the interpretability and readability of the building damage detection results. Datasets and preprocessing xBD dataset The training model in this study utilizes the xBD dataset3, a large-scale benchmark dataset for building damage assessment. The dataset comprises 22,068 pairs of pre-disaster and post-disaster images collected from nineteen natural disaster events worldwide via the Maxar/Digital Globe Open program. To enhance the robustness and generalization capacity of the model, data augmentation techniques were employed. Specifically, random geometric transformations, including translation, rotation, and cropping, were applied to both pre-disaster and post-disaster images and their corresponding labels during training, thereby altering the position, orientation, and size of image pairs and label pairs. Additionally, random adjustments to brightness, contrast, saturation, and hue were performed on the attributes of pre-disaster and post-disaster images, accompanied by the random introduction of Gaussian noise. All images have undergone geometric correction and precise registration, ensuring high positional accuracy. Each image has a resolution of 1024 × 1024 pixels with a ground sampling distance of 0.8 m. The dataset spans a total area of 45,361.79 square kilometers and contains 850,736 annotated buildings. Damage annotations are categorized into four levels, defined as follows:No damage: The building remains undisturbed, with no signs of water, structural or roof damage, or burn marks.Minor Damage: Partial roof or structural damage, nearby water or volcanic flow, missing roof elements, or visible cracks.Major Damage: Partial collapse of walls or roof, encroaching volcanic flow, or surrounded by water/mud.Destroyed: Scorched, completely collapsed, partially or fully submerged in water/mud, or otherwise no longer present. No damage: The building remains undisturbed, with no signs of water, structural or roof damage, or burn marks. No damage: The building remains undisturbed, with no signs of water, structural or roof damage, or burn marks. Minor Damage: Partial roof or structural damage, nearby water or volcanic flow, missing roof elements, or visible cracks. Minor Damage: Partial roof or structural damage, nearby water or volcanic flow, missing roof elements, or visible cracks. Major Damage: Partial collapse of walls or roof, encroaching volcanic flow, or surrounded by water/mud. Major Damage: Partial collapse of walls or roof, encroaching volcanic flow, or surrounded by water/mud. Destroyed: Scorched, completely collapsed, partially or fully submerged in water/mud, or otherwise no longer present. Destroyed: Scorched, completely collapsed, partially or fully submerged in water/mud, or otherwise no longer present. All annotations were reviewed and corrected by the California Air National Guard, NASA, and the Federal Emergency Management Agency. Masks were generated based on the building annotations extracted from the xBD dataset JSON files. Figure 1 presents a set of representative examples from the xBD dataset.Fig. 1Examples of images from the xBD dataset. Examples of images from the xBD dataset. Methods DDNet framework Figure 2 shows that DDNet is a two-stage network model including building localization and damage classification subnetworks. The basic architecture contains an encoder-decoder structure, the skip connection between the encoder and the decoder retains details, and the encoder is equipped with a combined CNN-transformer structure.Fig. 2DDNet framework. DDNet framework. Stage 1: Building localization As illustrated in Fig. 2a, pre-disaster buildings possess complete and well-defined outlines. Therefore, pre-disaster images are used as inputs to train the building localization network. At this stage, the encoder’s CNN layers extract features, which are further processed by 12 transformer encoder layers to obtain a deep feature map. The decoder progressively upsamples the features to the original image resolution, while the upsampled deep features and shallow features are differentially enhanced using the D-Cat module to retain detailed information. The final output of this network is a binary building segmentation map, representing the spatial distribution of buildings in the pre-disaster image. The formulation is given as follows:1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{P}}_{\text{B}}={\text{argmax}}({\text{P}}_{\text{b}})$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_{B} \varepsilon \{ 0,1\}^{1 \times H \times W}$$\end{document}, with 1 representing buildings and 0 representing the background, Pb ∈ R2×H×W, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${P}_{b}$$\end{document} denotes the probability distribution indicating whether a pixel belongs to the *building* or *background* category. Loss function Building localization provides spatial guidance for damage classification tasks, thereby improving the accuracy of damage detection. Consequently, building localization must achieve precise delineation of building boundaries and accurately determine effective area coverage. This task is formulated as a binary classification problem, where pixels are classified as either building or background. Since building pixels are far fewer than background pixels, an imbalance between positive and negative samples arises. Addressing this imbalance helps reduce the occurrence of false positives and false negatives, enabling more accurate boundary delineation. The effective area coverage, defined as the proportion of buildings correctly recognized by the model, can be improved by increasing the recall rate. The unified focal loss38 integrates the characteristics of the focal loss and the focal Tversky loss41, balancing recall and precision through the focal Tversky component while mitigating class imbalance via both the focal loss and the focal Tversky loss. The focal loss incorporates two hyperparameters, α and γ, into the binary cross-entropy loss. These parameters, referred to as focal parameters, control the class weighting and the degree of down-weighting for easily classified pixels, respectively. Specifically, α adjusts the weight of the negative (background) class, while γ modulates the reduction of loss contribution from pixels that are easy to classify, as defined by the following formulation:2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${L}_{FL}\left(\alpha ,\gamma \right)=\alpha [{\left(1-p\right)}^{\gamma }log\left(p\right)+{\left(p\right)}^{\gamma }log\left(1-p\right)]$$\end{document}where p denotes the predicted probability of a positive sample. The focal Tversky loss modifies the standard Tversky loss by introducing the parameter γ , as follows:3\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${L}_{FTL}\left(\beta \right)={{\sum }_{C=1}^{C}(1-DSC)}^{\frac{1}{\gamma }}$$\end{document}where DSC represents the F1 score. A simple weighted combination of the focal loss and focal Tversky loss introduces six hyperparameters, which is excessive. The enhancement or suppression effects of the focal parameters in both the focal loss and focal Tversky loss are applied uniformly across all classes, potentially affecting model convergence during the final stages of training. To address this, we suppress and enhance the effects of the focal loss and focal Tversky loss by combining functionally equivalent hyperparameters and introducing asymmetry to adjust the focal parameters. The focal loss is formally defined as follows:4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${L}_{\text{CFL}}=\left[-\frac{\beta }{N}ylog\left(p\right)-\frac{1-\beta }{N}{\left(1-y\right)}^{\gamma }log\left(1-p\right)\right]$$\end{document} Here, the weights of the building and background classes are controlled by β, while γ suppresses the background class and enhances the building class in the focal loss. p represents the predicted class probability, and y denotes the ground truth label.5\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$L_{{{\text{CFTL}}}} = \sum _{{C = 1}}^{C} \left( {1 - \frac{{\sum\limits_{{i = 1}}^{N} {p_{{0i}} } g_{{0i}} }}{{\sum\limits_{{i = 1}}^{N} {p_{{0i}} } g_{{0i}} + \beta \sum\limits_{{i = 1}}^{N} {p_{{0i}} } g_{{1i}} + (1 - \beta )\sum\limits_{{i = 1}}^{N} {p_{{1i}} } g_{{0i}} }}} \right)^{{\frac{1}{\gamma }}}$$\end{document} Here, α balances recall and precision while enhancing the representation of the building class in the focal Tversky loss. We set α = 0.9 and β = 0.3. In this formulation, p denotes the predicted class probability, and g represents the ground truth. The modified focal loss and the focal Tversky loss are integrated into a unified focal loss, which is defined as follows, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mu$$\end{document} is set to 0.5.6\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{L}}_{\text{loc}}\left(\uplambda ,\upbeta ,\upgamma \right)=\uplambda {\text{L}}_{\text{CFL}}+\left(1-\uplambda \right){\text{L}}_{\text{CFTL}}$$\end{document} In this way, we introduce the unified focal loss. The offset parameter is designed to enhance the building class while suppressing the background class, thereby improving the recall rate of building extraction through the adjustment of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta$$\end{document}. Transformer encoder The transformer architecture serves as a global self-attention modeling framework for sequential data and provides an effective network structure for capturing long-range contextual dependencies. Originally developed for natural language processing (NLP), it has since been widely adopted in computer vision (CV). The architecture typically consists of a patch embedding layer followed by multiple encoder layers. Each encoder layer incorporates a multi-head self-attention (MHSA) mechanism, a multilayer perceptron (MLP) module, a normalization layer, and a residual connection. Prior to entering the encoder layers, the input image undergoes patch embedding, in which the image is partitioned into patches of a specified size, and the information within each patch is flattened into a vector sequence. To preserve spatial information, positional encodings are subsequently added to the sequence42. The resulting sequence of embedding vectors, as illustrated in Fig. 3, is sequentially processed within the encoder layer, where it is first updated by the MHSA weights and subsequently transformed through dimensional linearization by the MLP. The final output is a feature representation map. MHSA constitutes the core of the transformer architecture and is composed of multiple independent attention heads. In single-head attention, a sequence of vectors is multiplied by initialized weight matrices to obtain the query (Q), key (K), and value (V) vectors. These vectors are computed as follows:Fig. 3Transformer encoder.7\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\text{Attention}(\text{Q},\text{K},\text{V})=\text{softmax}(\frac{{\text{QK}}^{\text{T}}}{\sqrt{{\text{d}}_{\text{k}}}})\text{V}$$\end{document} Transformer encoder. Let dk represent the dimensionality of the sequence k. The product of the query Q and key K is scaled by dividing it by the square root of dk, after which the resulting values are normalized using the softmax function. The normalized weights are then multiplied by the value V to produce the final output. The Multi-Head Self-Attention (MHSA) mechanism is constructed by concatenating the outputs of multiple self-attention heads and applying a linear transformation, enabling the model to capture information from diverse representation subspaces:8\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$MultiHead(Q,K,V)=Concat(hea{d}_{1}\cdots hea{d}_{n}){W}^{O}$$\end{document} Differential concatenation structure (D-Cat) In the original skip connection structure, the deep feature map is upsampled and concatenated with the shallow feature map for fusion. This approach helps mitigate the loss of feature information caused by upsampling but often introduces considerable redundancy between the upsampled deep features and the shallow features. To address this issue, this paper proposes an upsampling-based differential splicing structure, illustrated in Fig. 4. In this structure, the upsampled deep feature map is subtracted from the corresponding shallow feature map to generate a differential feature map. This operation effectively removes redundant information between the two feature layers while supplementing missing information in the deep feature map, thereby enabling the network to more efficiently leverage the boundary information present in the shallow feature map. Subsequently, the upsampled deep feature map is concatenated with the differential feature map for further processing. Taking a single feature layer as an example, the feature map of this layer is upsampled to match the original image resolution corresponding to that layer.Fig. 4The structure of D-Cat. The structure of D-Cat. This operation can be formally expressed by the following equation:9\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${FM{\prime}}_{i-1}=Upsample(Conv\left({FM}_{i-1}\right))$$\end{document}where Upsample is a bilinear interpolation upsampling operation. Then, the feature maps of layers \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${FM}_{i}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${FM{\prime}}_{i-1}$$\end{document} are used to obtain the missing \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${FM{\prime}}_{i-1}$$\end{document} information by subtraction. Finally, a concatenation operation is used to obtain the output feature \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${FM}_{i-1}$$\end{document}:10\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${FM}_{iout}=Cat({FM{\prime}}_{i-1},\left({FM{\prime}}_{i-1}-{FM}_{i}\right))$$\end{document} Stage 2: Damage classification Building damage classification aims to assess the extent of damage by comparing the changes in a building’s state before and after a disaster. As illustrated in Fig. 2b, pre-disaster and post-disaster images are fed into a dual-branch network, from which multilevel feature maps are extracted via a CNN. To enhance the network’s focus on damage-relevant features and suppress interference caused by varying imaging conditions, dual-temporal joint attention is applied to the last three layers of the feature maps, modulating the interactions between pre-disaster and post-disaster features. Prior to producing the final outputs of the dual-branch network, the building localization segmentation results are incorporated as follows:11\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{P}}_{\text{D}}=\text{argmax}({\text{P}}_{\text{B}}\cdot {\text{P}}_{\text{d}})$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${P}_{d}\in {R}^{C\times H\times W}$$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C$$\end{document} denotes the number of damage levels, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${P}_{B}\in {R}^{H\times W}$$\end{document} is the result of building localization. Loss function Building damage classification is inherently a long-tailed data detection problem. The distribution of annotations across the categories of no damage, minor damage, major damage, and destroyed buildings is presented in Table 1. Notably, the number of annotations for undamaged buildings is 3.18 times greater than that for the combined minor, major, and destroyed categories. For these tail categories, the quality of the classification fit significantly impacts the overall network accuracy. To address this challenge, this study employs a combination of the Combo loss43 and the Seesaw loss44, incorporating single-label weighting to effectively handle the underrepresented classes:12\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{L}}_{{{\text{dam}}}} \left( {\lambda_{{\text{i}}} ,\alpha } \right) = \mathop \sum \limits_{{{\text{i}} = 0}}^{4} \lambda_{{\text{i}}} {\text{L}}_{{{\text{CL}}}} + 2{\text{L}}_{{{\text{SL}}}} = \mathop \sum \limits_{{{\text{i}} = 0}}^{4} \lambda_{{\text{i}}} \left[ {\left( {1 - {\text{DSC}}} \right) + \left( {1 - {\text{P}}_{{\text{t}}} } \right)^{\alpha } {\text{L}}_{{{\text{BCE}}}} } \right] + 2 - \left[ {\mathop \sum \limits_{{{\text{i}} = 0}}^{4} {\text{w}}_{{{\text{y}}_{{\text{i}}} }} {\text{L}}_{{{\text{BCE}}}} \left( {{\text{p}}_{{\text{i}}} ,{\text{y}}_{{\text{i}}} } \right)} \right]$$\end{document}where the Combo loss and Seesaw loss are combined with a weighting ratio of 1:2. The single-label weighted Combo loss comprises Dice loss and focal loss, and is calculated separately for each label to enable the network to better address tail categories and mitigate competitive relationships among classes. Specifically, the weights for minor, major, and destroyed damage are set to 0.4, the major damage weight is set to 0.3, and the destroyed weight is set to 0.1, so \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\upmu }_{\text{i}}=[\text{0.1,0.1,0.4,0.3},.0.1]$$\end{document}. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\text{DSC}$$\end{document} is the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{F}}_{1}$$\end{document} score, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{p}}_{\text{t}}$$\end{document} is the intersection percentage between the model output and the label computation, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} is set to 2. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{L}}_{\text{BCE}}$$\end{document} is the cross-entropy loss function. The Seesaw loss dynamically adjusts the category weights by the number of examples. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{W}}_{({\text{y}}_{\text{i}})}$$\end{document} denotes the weight adaptively updated based on both the sample frequency and the global sample mean.Table 1The distribution of the damaged buildings in the xBD dataset.Damage levelNo damageMinor damageMajor damageDestroyedPolygons313,00336,86029,90431,560Ratio8.471.000.931.03 Table 1The distribution of the damaged buildings in the xBD dataset.Damage levelNo damageMinor damageMajor damageDestroyedPolygons313,00336,86029,90431,560Ratio8.471.000.931.03 The distribution of the damaged buildings in the xBD dataset. Dual-Temporal Joint Attention Module (DJA) The acquisition of dual-temporal images is often subject to substantial interference arising from varying seasonal and meteorological conditions, as well as diverse imaging angles. Effective suppression of such interference is necessary prior to exploring correlations between features extracted from pre-disaster and post-disaster images. Given that bitemporal images contain a large amount of task-irrelevant information, it is crucial for the attention mechanism to accurately identify features indicative of actual building damage. In this study, the dual-temporal joint attention (DJA) module is implemented as a combination of self-attention and cross-temporal cross-attention. The self-attention operation is defined in Eq. 12, while the cross-temporal cross-attention incorporates partial feature representations from the alternate branch and is computed as follows:13\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$CrossAtt\left( {Q_{a} ,K_{b} ,V_{b} } \right) = Soft\max \left( {Q_{a} ,K_{b}^{T} c} \right) \cdot V_{b}$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Q}_{a}$$\end{document}、\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${K}_{a}$$\end{document}、\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V}_{a}$$\end{document}、\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Q}_{b}$$\end{document}、\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${K}_{b}$$\end{document}、\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V}_{b}$$\end{document} are obtained by applying convolution operations to the pre-disaster and post-disaster images, respectively. The distinction between cross-temporal cross-attention and the dual-temporal joint attention (DJA) module is as follows. Taking images A and B as examples, cross-temporal cross-attention uses the query vector Q of each image to learn the key vector K and value vector V from the same image. In contrast, the DJA module concatenates the Q vectors of both images while learning the K and V of the corresponding image. By integrating both the global self-attention of each image and the cross-attention between dual-temporal images, the proposed approach effectively mitigates interference arising from seasonal and meteorological variations as well as differing imaging angles. As a result, the network is able to focus on the features representing changes between pre-disaster and post-disaster conditions, thereby enabling more precise analysis. As illustrated in Fig. 5, the matrix generation process of the pre-disaster branch is used as an example.The feature maps of the pre-disaster and post-disaster images are concatenated with Q through K and V, which are obtained via 1 × 1 convolution. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${JA}_{1}$$\end{document} is obtained by multiplying \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Q}_{cat}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${K}_{a}$$\end{document} and then executing the softmax function. The DJA is calculated as follows:14\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C_{a} = DJA\left( {\left\{ {Q_{a} ,Q_{b} } \right\},\left\{ {K_{a} ,V_{a} } \right\}} \right) + input_{a} { = }Soft\max \left( {Concat\left( {Q_{a} ,Q_{b} } \right) \bullet K_{a}^{T} } \right) \bullet V_{a} + input_{a}$$\end{document}where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Q}_{a}$$\end{document} ,\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${K}_{a}$$\end{document} ,\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V}_{a}$$\end{document} ,\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Q}_{b}$$\end{document} ,\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${K}_{b}$$\end{document} ,\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V}_{b}$$\end{document} are the queries, keys, and values of images a and b obtained after the convolution. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${input}_{a}$$\end{document} is the raw input.Fig. 5The structure of the DJA module. The structure of the DJA module. The proposed Dual-Temporal Joint Attention (DJA) mechanism, while conceptually related to dual-branch attention fusion structures such as those in Siamese networks, differs substantially in implementation and objectives. Conventional cross-attention typically enables unidirectional branch-wise interactions, where the Query of one branch attends to the Key-Value of the other. In contrast, DJA integrates self-attention and cross-temporal cross-attention, concatenating Queries from pre- and post-disaster images for joint modeling. This design preserves temporal contextual consistency and more effectively captures cross-temporal variations. Unlike general Siamese or cross-attention methods aimed at detecting the presence of change, DJA is tailored for post-disaster building damage detection. It not only distinguishes changed from unchanged regions but also discriminates among damage levels, suppressing pseudo-changes induced by illumination, weather, or viewpoint variations while highlighting structural features of real damage. Furthermore, DJA is embedded only in the final three layers of the damage classification encoder, where high-level semantic features are both more sensitive to imaging discrepancies and more relevant to structural damage patterns. This strategy enhances robustness and task specificity while ensuring computational efficiency. The connected component analysis algorithm The semantic information within the spatial domain of each building must be unified, as inconsistencies in the representation of damaged buildings can hinder the effective use of the network’s output, despite the capability of deep learning architectures to detect building damage. Connected domain analysis is a widely used method for regional delineation of image pixels, in which regions composed of pixels with the same value and their neighbors are labeled to form a connected domain, enabling the analysis of the geometric semantics of each region. As illustrated in Fig. 6, the process begins by calculating the number of pixels within the building damage area based on a connected domain analysis of the building location map. Subsequently, the same connected domain analysis is applied to the damage detection image to obtain the number of pixels corresponding to damage changes. Finally, by comparing the damage change area with the building area, the algorithm outputs the damage assessment result for each individual building.Fig. 6Illustration of the connected component analysis algorithm. Illustration of the connected component analysis algorithm. Experiment and results Training details The proposed two-stage model is implemented in Python 3.7 with PyTorch 1.10 and trained and tested on an NVIDIA Tesla V100 GPU. Due to GPU memory constraints, both the training and testing images were seamlessly cropped into 73,464 and 7464 image patches, respectively, each with a resolution of 512 × 512 pixels. To enhance the model’s robustness and generalization, data augmentation techniques were applied, including random translations and rotations of pre-disaster and post-disaster images during training, as well as stochastic adjustments to brightness, contrast, saturation, hue, and Gaussian noise.The dataset was randomly split into training (80%), validation (10%), and test (10%) subsets to ensure model training, parameter tuning, and performance evaluation. Both the training and testing processes were initialized with pretrained weights. In the building localization phase, the model was trained with a learning rate of 0.01, a batch size of 8, and 100 epochs, whereas in the building classification phase, a learning rate of 0.01, a batch size of 12, and 100 epochs were used. Metrics Since the damage classification network relies on the outputs of the localization network to detect building damage in this experiment, the performance of the localization network is comprehensively evaluated using Precision, Recall, F1 score, Overall Accuracy (OA), and Intersection over Union (IoU). The calculation methods for these metrics are defined as follows:15\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Precision=\frac{TP}{TP+FP}$$\end{document}16\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Recall=\frac{TP}{TP+FN}$$\end{document}17\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$OA=\frac{TP+TN}{TP+TN+FP+FN}$$\end{document}18\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$IoU=\frac{TP}{TP+FP+FN}$$\end{document}19\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${F}_{1loc}=2\frac{Precision*Recall}{Precision+Recall}$$\end{document} The damage classification network categorizes image pixels into four damage levels: no damage, minor damage, major damage, and destroyed. Accordingly, a corresponding metric value is computed for each category. The \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{F}}_{1\text{cls}}$$\end{document} metric across the four damage levels is computed using the harmonic mean3, which is defined as follows:20\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_{1cls} = \frac{4}{{\frac{1}{{F_{1no dam} }} + \frac{1}{{F_{1minor dam} }} + \frac{1}{{F_{1major dam} }} + \frac{1}{{F_{1destroyed} }}}}$$\end{document} The overall evaluation metric used in this study is the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{F}}_{1\text{overall}}$$\end{document} , which is calculated as the weighted average of the localization \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{F}}_{1\text{loc}}$$\end{document} and the damage \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{F}}_{1\text{cls}}$$\end{document}. This metric is widely adopted for evaluating performance on the xBD dataset.21\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${F}_{1overall}{=0.3F}_{1loc}{+0.7F}_{1cls}$$\end{document} Results In the building localization stage, since most previous studies do not report specific evaluation metrics for building localization, we compare the performance of the trained classical semantic segmentation models with that of DDNet. Table 2 presents the comparative results. Our model achieves an F1-score of 0.8638. Compared with SCSE-ResNeXtUNet (SERes-UNet), DeepLabV3 + 45, and U-Net, DDNet exhibits improvements of 0.0384, 0.0483, and 0.0644, respectively. Its overall accuracy is 0.0398 higher than that of the lowest-performing model, DeepLabV3 + . Additionally, DDNet achieves recall improvements of 0.0384, 0.0560, and 0.0945, respectively, demonstrating that it reliably maintains high recall in building localization compared with other models.Table 2Evaluation metrics of the building localization results.ModelF1locAccuracyRecallIOUOAU-Net0.79940.86170.74550.66580.9803DeepLabV3 + 0.81550.83710.79520.68860.9810SCSE-ResNext-UNet0.82540.83840.81280.70270.9819TransUNet0.85760.87410.84170.75070.9853DDNet0.86380.87690.85120.76030.9858 Table 2Evaluation metrics of the building localization results.ModelF1locAccuracyRecallIOUOAU-Net0.79940.86170.74550.66580.9803DeepLabV3 + 0.81550.83710.79520.68860.9810SCSE-ResNext-UNet0.82540.83840.81280.70270.9819TransUNet0.85760.87410.84170.75070.9853DDNet0.86380.87690.85120.76030.9858 Evaluation metrics of the building localization results. Figure 7 presents a comparison of the extraction results for dense buildings, elongated buildings, and small buildings across the evaluated methods. As shown, DDNet is capable of capturing detailed representations of building rooftops and accurately delineating the boundaries between adjacent buildings. The proposed method demonstrates superior performance in accurately extracting small buildings and achieves the highest levels of continuity and smoothness when delineating elongated buildings.Fig. 7Visualization of building localization results. Visualization of building localization results. The subsequent damage detection subtask involves classifying the severity of damage sustained by individual buildings. The corresponding data comparison is presented in Table 3, with evaluation primarily based on the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{F}}_{1\text{cls}}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{F}}_{\text{1overall}}$$\end{document} metrics. Weber15 performs damage detection by integrating features through a change detection–based approach. Hao21 integrated the U-Net model with a dual-branch CNN architecture, employing the U-Net to perform semantic segmentation of buildings, while the dual-branch network was utilized for damaged building classification. A self-attention module was introduced to enhance performance by incorporating long-range contextual information from the entire image. RescueNet46 adopts a dual-branch learning strategy based on pre-disaster and post-disaster images, where deep features are concatenated across multiple scales to enhance the information exchange between pre-disaster and post-disaster representations. ChangeOS47 employs the pre-disaster image for building localization and the post-disaster image for damage classification, incorporating post-processing to ensure consistency of damage detection results within building areas. DamFormer34 enhances building localization and damage classification by introducing cross-layer fusion within a combined CNN–transformer architecture. BSSNet37 incorporates contrastive loss and boundary loss into the building localization process, thereby improving the accuracy of building segmentation results.Table 3Overall building damage detection results.MethodF1overallF1locF1cls\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{F}}_{1\text{cls}}^{\text{No damage}}$$\end{document}\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{F}}_{1\text{cls}}^{\text{Minor}}$$\end{document}\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{F}}_{1\text{cls}}^{\text{Major}}$$\end{document}\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{F}}_{1\text{cls}}^{\text{Destroyed}}$$\end{document}Ethan Weber150.74100.83500.70000.90600.49300.72200.8370Hao210.71900.73000.71400.92300.39800.67500.8120RescueNet460.77000.84000.73500.88500.56300.77100.8080ChangeOS470.78570.85410.75640.92660.60140.74180.8345DAMFormer340.77020.86860.72810.89860.56780.72560.8079BSSNet370.77200.86300.73300.90400.53300.77200.8460DDNet0.79560.86380.76640.93140.60520.75420.8567 Table 3Overall building damage detection results.MethodF1overallF1locF1cls\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{F}}_{1\text{cls}}^{\text{No damage}}$$\end{document}\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{F}}_{1\text{cls}}^{\text{Minor}}$$\end{document}\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{F}}_{1\text{cls}}^{\text{Major}}$$\end{document}\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{F}}_{1\text{cls}}^{\text{Destroyed}}$$\end{document}Ethan Weber150.74100.83500.70000.90600.49300.72200.8370Hao210.71900.73000.71400.92300.39800.67500.8120RescueNet460.77000.84000.73500.88500.56300.77100.8080ChangeOS470.78570.85410.75640.92660.60140.74180.8345DAMFormer340.77020.86860.72810.89860.56780.72560.8079BSSNet370.77200.86300.73300.90400.53300.77200.8460DDNet0.79560.86380.76640.93140.60520.75420.8567 Overall building damage detection results. Weber exhibits lower performance in building damage classification, which can be attributed to its reliance on a change detection–based approach. Our method achieves the highest building damage classification performance, attaining the best \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{F}}_{1\text{cls}}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{F}}_{\text{1overall}}$$\end{document} among all compared methods. Specifically, the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{F}}_{1\text{cls}}$$\end{document} of our approach is 0.0100 higher than that of the second-best method, and its \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{F}}_{\text{1overall}}$$\end{document} is 0.0099 higher than that of the second-best method. Nevertheless, our method has certain limitations. As shown in Table 1, major damage is frequently misclassified as either minor damage or destroyed, indicating that the model still struggles to fully address the imbalanced data distribution. Figure 8 presents the visualization of damage detection results generated by several models. DDNet produces clearer segmentation boundaries and reduces the number of misclassified damaged buildings. In the first image of Fig. 8, the upper portion of the white building in the lower-left corner is significantly displaced, and the lower portion is partially damaged; ChangeOS47 classifies it as minor damage, whereas DDNet correctly identifies the damage level. In the third image, the building at the top is slightly damaged; ChangeOS classifies it as major damage, while DDNet correctly classifies it. Overall, DDNet demonstrates improved accuracy and effectiveness in building damage classification.Fig. 8Classifcation of building damage in xBD dataset. Classifcation of building damage in xBD dataset. Ablation study Differential concatenate (D-Cat) Table 4 presents the performance improvements achieved by incorporating D-Cat into the localization network. The inclusion of D-Cat enhances both the accuracy and recall metrics, resulting in a 0.0032 increase in the F1loc score. When both D-Cat and O-UFL are integrated into the localization network, the F1loc score improves by 0.0062 compared with the baseline network without these enhancements.Table 4Ablation study on D-Cat and O-UFL.ModelF1locAccuracyRecallBaseline0.85760.87410.8417Cat0.86080.87810.8442O-UFL0.86160.87320.8504D-Cat + O-UFL0.86380.87690.8512 Table 4Ablation study on D-Cat and O-UFL.ModelF1locAccuracyRecallBaseline0.85760.87410.8417Cat0.86080.87810.8442O-UFL0.86160.87320.8504D-Cat + O-UFL0.86380.87690.8512 Ablation study on D-Cat and O-UFL. The offsetting focal parameters of the unified focal loss (O-UFL) O-UFL incorporates three hyperparameters in total. Specifically, γ is empirically set to 0.3 to balance positive and negative examples, while μ is fixed at 0.5 following the formulation of Combo loss. Increasing β leads to an improvement in recall; however, this gain is accompanied by a corresponding reduction in accuracy. In this study, three distinct values of β—0.5, 0.7, and 0.9—are evaluated to investigate their influence on the performance of disaster-damaged building detection. The comparative results, summarized in Table 5, demonstrate that setting β to 0.9 effectively extends the acquisition range for buildings and enhances the overall performance of the network.Table 5The impact of the hyperparameter \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\upbeta$$\end{document}.\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\upbeta$$\end{document}F1locAccuracyRecallF1cls0.50.86710.88800.84720.72270.70.86470.88590.84450.74340.90.86380.87690.85120.7664 Table 5The impact of the hyperparameter \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\upbeta$$\end{document}.\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\upbeta$$\end{document}F1locAccuracyRecallF1cls0.50.86710.88800.84720.72270.70.86470.88590.84450.74340.90.86380.87690.85120.7664 The impact of the hyperparameter \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\upbeta$$\end{document}. DJA and the connected-component analysis algorithm To evaluate the effectiveness of incorporating the dual-temporal joint attention (DJA) module for capturing correlations and extracting change features from pre-disaster and post-disaster images, this study compares the base two-branch model, the self-attention (SA) module, and the DJA module. The base structure refers to the unenhanced two-branch model. Table 6 presents a comparative analysis of the results. The inclusion of the DJA module improves accuracy across all damage detection categories, indicating that the joint attention mechanism effectively guides the network to learn informative features from both pre-disaster and post-disaster images. When both the DJA module and the connected component analysis algorithm are integrated into the classification network, the F1cls index increases by 1.17% relative to the base model.Table 6Analysis of the DJA module and the algorithm for connected-component analysis.Damage classification networkF1clsNo damageMinorMajorDestroyedbase0.75470.92380.59880.74160.8338base + SA0.75630.92700.59830.74430.8369base + DJA0.76200.93060.60720.74700.8408base + DJA + connected-component0.76640.93140.60520.75420.8567 Table 6Analysis of the DJA module and the algorithm for connected-component analysis.Damage classification networkF1clsNo damageMinorMajorDestroyedbase0.75470.92380.59880.74160.8338base + SA0.75630.92700.59830.74430.8369base + DJA0.76200.93060.60720.74700.8408base + DJA + connected-component0.76640.93140.60520.75420.8567 Analysis of the DJA module and the algorithm for connected-component analysis. Discussion Summary In post-disaster scenarios, buildings are of critical concern, and the rapid and accurate extraction of damage-related building information from remote sensing images is essential for effective emergency response. This study employs deep learning techniques and utilizes high-resolution imagery from both pre-disaster and post-disaster phases to perform multi-task change detection, with a focus on identifying disaster-affected structures. To address challenges such as inaccurate boundary delineation, interference suppression, and the effective integration of features from pre-disaster and post-disaster data, this research emphasizes the tasks of building localization and damage classification. A two-stage DDNet model is proposed to tackle these challenges. The main findings and conclusions of this study are summarized as follows: A novel two-stage DDNet model was proposed to address the inherent challenges in building localization and damaged building classification. By integrating CNN and Transformer architectures, the building localization phase overcomes the limitations of TransUNet through the use of a Unified Focal Loss function with offset focal parameters, thereby expanding the effective feature extraction region. In addition, a differential connection module was incorporated to enhance boundary delineation. In the damage classification phase, dual-temporal joint attention modules were employed to encode relevant features from both pre-disaster and post-disaster images while mitigating interference caused by variations in imaging conditions. Subsequently, buildings were segmented to identify damage zones, and a connected component analysis algorithm was applied to generate distinct damage assessment results. Compared with alternative methods, DDNet demonstrated superior performance on the xBD dataset, achieving an overall F1 score of 79.56%, with F1 scores of 86.38% for building localization and 76.64% for damaged building classification. Challenges and future directions The proposed models effectively address the challenge of extracting disaster-damaged buildings. Experimental results demonstrate that the DDNet model significantly improves extraction accuracy. Nevertheless, certain limitations persist, which warrant further investigation and discussion. Model improvement Model performance often deteriorates in real-world applications. Although incremental and transfer learning approaches can improve accuracy through additional training, they incur substantial time costs. Moreover, the limited sample sizes of minor and severe damage classes, combined with the constraints of loss function–based methods, pose significant challenges. Future research should focus on enhancing model generalization and addressing the difficulties associated with these challenging damage categories. Integration of multi-modal data This study utilizes high-resolution RGB optical imagery, which captures rooftop details but lacks information regarding building facades and sides. As a result, the damage assessment is limited and potentially imprecise, restricting its ability to support a comprehensive evaluation. Future research could integrate multi-modal data sources, including SAR imagery, multi-view point clouds, and geographic information, to enable more detailed and holistic damage assessments. Furthermore, disasters often involve secondary events influenced by both natural and social factors, rendering static damage assessments insufficient. Subsequent investigations should focus on developing deep learning models for multi-temporal imagery to facilitate dynamic monitoring of disaster progression. Limitations of two-dimensional imagery and the necessity of multimodal integration It is important to note that buildings are inherently three-dimensional structures, and their damage states often involve multi-level structural features. The aerial and satellite optical imagery employed in this study primarily provides a two-dimensional perspective of rooftops, lacking critical façade and depth information. Such limitations in data naturally constrain the accuracy of multi-level damage detection and the granularity of classification, particularly for fine-grained or subtle damage patterns. Consequently, analyses relying solely on 2D imagery may not fully capture the complete damage state of buildings. To partially mitigate these limitations, the proposed approach leverages cross-temporal feature modeling and regional consistency optimization. Specifically, the Dual-temporal Joint Attention (DJA) mechanism emphasizes semantic differences in texture, shape, and contour between pre- and post-disaster images, thereby capturing structural variations at the rooftop level and partially compensating for the absence of façade information. Additionally, connected-component analysis is applied during the classification stage to enforce regional consistency across entire building areas, which helps reduce prediction instability arising from missing local features or incomplete structural information. These strategies allow our 2D-based method to achieve improved localization and classification performance despite the intrinsic limitations of the input data. Nevertheless, the fundamental constraints of two-dimensional optical imagery remain. Without integrating three-dimensional data sources—such as LiDAR point clouds, oblique aerial images, stereo image pairs, or photogrammetric reconstructions—the method cannot fully characterize the volumetric structure and nuanced damage patterns of buildings. Future research will focus on the fusion of multimodal and 3D spatial data with 2D cross-temporal feature modeling. By combining the complementary strengths of 2D and 3D information, we expect to enhance both the accuracy and granularity of post-disaster building damage detection, and to increase the overall reliability and interpretability of the results, particularly for complex or multi-level structural damages. Conclusions In this study, we propose DDNet for detecting buildings damaged by disasters. The localization network employs a hybrid architecture that integrates CNN and Transformer components, utilizing a Unified Focal Loss with offset focal parameters to expand the effective acquisition range. Additionally, a differential concatenation module is incorporated to enhance the extraction of building boundary information. To capture correlation features between pre-disaster and post-disaster images while mitigating interference caused by varying imaging conditions, the damage classification network is equipped with a dual-temporal joint attention module. Subsequently, buildings of interest are segmented to identify damaged regions, and a connected component analysis algorithm is applied to generate distinct damage assessment results. Compared with existing methods, DDNet demonstrates superior performance on the xBD dataset. Future research will focus on further improving the model’s generalization capabilities and enhancing its detection accuracy based on the current findings. In addition, we plan to collect post-disaster visible-light aerial imagery and conduct building damage detection experiments using end-to-end models. This will further validate the applicability and performance differences of detection-based methods across different data modalities, and help establish a multi-modal disaster analysis framework spanning from visible-light detection to remote sensing segmentation.