A lightweight multimodal image fusion and enhancement method for smoke scenes Authors: Jie Ma, Ming Yang, Xianli Jin, Yangyang Zhao Published: 2026 ABSTRACT AbstractMultimodal fusion under smoke conditions faces the challenge of being unable to obtain clear fused images. Traditional fusion methods assume clear imaging conditions and fail to address nonlinear degradation. Existing stepwise smoke removal fusion processes are complex, prone to error accumulation, and lack lightweight solutions that balance performance and efficiency. This paper proposes a direct smoke removal multimodal image fusion framework based on two stage training and a lightweight CNN-Transformer architecture. The method employs an encoder-decoder backbone. The encoder combines CNN for local feature extraction and Transformer for global dependency modeling. A lightweight latent feature mapping network has achieved smoke suppression. Systematic ablation experiments validate the network structure and fusion strategy. Experiments on a multimodal test dataset with varying smoke concentrations show that the proposed method outperforms current mainstream fusion methods in PSNR, SSIM, MSE, and ASC. This research provides a new technical pathway for lightweight and efficient of multimodal perception in complex harsh environments.Keywords: Multimodal image fusion, Smoke scene, Smoke removal, CNN-transformer, Two-stage trainingSubject terms: Engineering, Mathematics and computing Introduction For perception tasks in complex scenarios1, multimodal fusion technology enhances the system’s comprehensive environmental understanding and robustness by integrating information from different modalities, thereby overcoming the limitations of single-modal data. In many fields, such as security surveillance, multimodal fusion methods have demonstrated significant value. For example, by leveraging the complementary nature of visible light and infrared information, more reliable object detection and scene analysis can be achieved under varying lighting and weather conditions. However, as application scenarios extend to complex environments and special working conditions, traditional multimodal fusion technologies face new challenges. Under extreme visual conditions2–4, such as haze, rain, snow, and smoke interference, the information quality and feature consistency among different modalities are severely affected. This imposes higher demands on the generalization capability and adaptability of fusion models. Smoke-filled environments represent a typical form of visual interference and are commonly encountered in real-world scenarios such as fire scenes, industrial emissions, and adverse weather conditions5. In such environments, visible images often suffer severe detail loss and color distortion. Infrared images, while possessing a certain penetration capability, are susceptible to uneven temperature distribution and noise interference, resulting in blurred object edges and reduced signal-to-noise ratio. Consequently, smoke environments pose specific challenges for infrared and visible light image fusion tasks. On the one hand, smoke induces non-uniform and non-linear degradation across different modal images, making traditional fusion methods based on clear-image assumptions difficult to apply directly. On the other hand, existing fusion models typically lack the capability to effectively model degradation characteristics under smoke interference, leading to fusion results that struggle to balance detail preservation with smoke suppression. Therefore, research on multimodal fusion and data enhancement methods tailored for smoke environments holds theoretical value and practical importance. Currently, mainstream multimodal fusion methods, particularly infrared and visible fusion techniques, predominantly assume clear environmental conditions. The design of their models and fusion strategies largely fails to adequately account for feature degradation caused by extreme conditions such as smoke. In smoke-affected environments, conventional fusion methods exhibit suboptimal performance when processing degraded images, revealing several unresolved critical challenges. First, existing approaches often rely on stepwise processing strategies, such as performing smoke removal followed by fusion. Such methods not only entail complex workflows but are also prone to error accumulation. Consequently, there is an urgent need to develop a unified model capable of simultaneously achieving smoke removal and information fusion to enhance fusion efficiency and overall performance. Second, prevailing fusion models are often characterized by large parameter sizes and high computational complexity, making them difficult to deploy in resource-constrained real-time systems. Therefore, designing lightweight network architectures is essential to improve inference speed. Furthermore, although existing research has explored the performance of different fusion strategies on clear images, the mechanisms through which fusion strategies affect model performance within a unified smoke-removal and fusion framework remain unclear, and systematic analysis and comparison are notably lacking. In response to the aforementioned challenges, this paper conducts an in-depth investigation into lightweight multimodal fusion and enhancement methods under smoke-affected environments, proposing a lightweight direct smoke-removal and fusion framework based on two-stage training. This approach achieves end-to-end degradation-aware processing and information fusion. The main contributions of this paper encompass the following four aspects. Addressing the issue of image degradation in smoke-affected environments, a unified smoke removal and fusion method is proposed. This method simultaneously accomplishes smoke suppression and multimodal information fusion, effectively enhancing the quality and usability of the fused images. Addressing the issue of image degradation in smoke-affected environments, a unified smoke removal and fusion method is proposed. This method simultaneously accomplishes smoke suppression and multimodal information fusion, effectively enhancing the quality and usability of the fused images. The lightweight modeling mechanism of latent feature mapping networks during training is explored. By designing an efficient network architecture, the model parameter and computational burden are reduced while maintaining fusion performance. The lightweight modeling mechanism of latent feature mapping networks during training is explored. By designing an efficient network architecture, the model parameter and computational burden are reduced while maintaining fusion performance. A systematic analysis of the performance impact of different fusion strategies in fusion models based on latent feature mapping is conducted, providing a theoretical foundation for the selection and optimization of fusion modules. A systematic analysis of the performance impact of different fusion strategies in fusion models based on latent feature mapping is conducted, providing a theoretical foundation for the selection and optimization of fusion modules. The proposed model is comprehensively validated on a large-scale smoke scene dataset. Experimental results demonstrate that the proposed method achieves excellent direct smoke removal and fusion performance while retaining its lightweight advantages, offering effective technical support for multimodal perception tasks in complex environments. The proposed model is comprehensively validated on a large-scale smoke scene dataset. Experimental results demonstrate that the proposed method achieves excellent direct smoke removal and fusion performance while retaining its lightweight advantages, offering effective technical support for multimodal perception tasks in complex environments. The structure of this paper is organized as follows. Section "Related work" introduces related research. Section "Lightweight multimodal image fusion method based on CNN-transformer" elaborates in detail on the proposed lightweight multimodal image fusion method based on CNN-Transformer, including the overall framework, training strategy, evaluation metrics, and the smoke dataset. Section "Experiment" presents the experimental study to validate the performance of the proposed method, encompassing the experimental setup, analysis of the encoder training process, quantitative and qualitative comparisons with advanced algorithms under varying smoke concentrations, analysis of smoke removal capability, and ablation experiments. Finally, Sect. "Conclusion" summarizes the entire work. Related work Conventional image fusion method For common scenarios, multimodal image fusion mainly falls into non-deep learning methods6,7 and deep learning methods8,9. In recent years, thanks to the development of machine learning theory, especially deep learning, a large number of representative methods in the field of deep learning-based multimodal image fusion have emerged. In the field of classical image fusion, Liu et al. proposed the DSFusion method in 202410, aiming to collaboratively preserve details and scene information. This method enhances feature extraction capability by designing a local attention module and an improved channel attention mechanism, and constructs a novel loss function to balance information components in the fused image. In the same year, Zheng et al. developed the FISCNet model11, innovatively performing fusion processing simultaneously in the frequency domain and spatial domain. This approach leverages Fourier transform to analyze frequency-domain characteristics and integrates phase information with a spatial compensation mechanism to account for both salient targets and texture details. Chen et al. designed the LENFusion network for low-light nighttime environments12. This framework achieves interactive optimization of low-light enhancement and image fusion through a brightness adjustment network and a feedback loop structure, and introduces a no-reference color loss function. In the domain of multimodal medical image fusion, Liu et al. proposed the MM-Net multiscale network based on MixFormer13. This model effectively captures local and global contextual features using an improved Transformer backbone network, and accomplishes high-quality fusion of anatomical and functional images through multi-source spatial attention and multiscale feature interaction modules. In recent years, with the continuous expansion of application scenarios, research on multimodal image fusion for complex conditions has gradually increased, aiming to enhance the robustness and practicality of the algorithms in harsh environments. Under this trend, in 2024, Luo et al. proposed a fusion algorithm that emphasizes visual fidelity and significant information correction14. This method optimizes the fusion effect in complex environments through adaptive Taylor approximation and multi-scale decomposition. In the same year, Liu et al. developed the PromptFusion method15, introducing contour wave autoencoder and prompt learning mechanism, achieving harmonious fusion of cross-modal semantics under the guidance of visual language models. In 2024, Qian et al. reported LiMFusion, which decomposes the source images through fourth-order partial differential equations and combines the attention mechanism, enhancing the feature retention ability of the interested regions16. Li et al. proposed the UMCFuse framework in 202517. This method framework combines traditional decomposition strategies with the attention mechanism, aiming to solve the information conflict problem between source images in complex scenarios. In extremely dark environments, in 2025, Li et al. designed the VCIF model18, incorporating the low-light enhancement task into a unified framework to generate better visual fusion results. In the same year, Xu et al. proposed the feature-focused network FFSR based on scale refinement19. This method utilizes cross-scale parallel convolution flow and information exchange module, effectively improving the performance of multi-scale feature extraction and fusion, especially in night scenes. Image fusion method for smoke scenes In the field of multimodal image fusion under adverse weather conditions such as smoke, researchers both domestically and internationally are increasingly focusing on addressing the performance degradation caused by image quality deterioration in complex environments. In response to this challenge, Bi et al. proposed the FIVFusion method in 202520. This approach synergistically integrates defogging and fusion processes through aggregation and decoupling strategies and constructs a foggy weather dataset to validate the superiority of the generated images in terms of texture details and salient targets. In the same year, Gao et al. developed the HaIVFusion model3. This model designs a scene information restoration network to recover textures obscured by smoke in visible light images and employs a color consistency loss to reduce distortion. In 2024, Huang et al. proposed DeDNet, which utilizes a decomposition-driven network to jointly optimize fusion and noise removal21. By incorporating constraints into the loss function, this method effectively enhances detail restoration capabilities. To address complex illumination interference in nighttime environments, Wang et al. proposed the FRFusion network in 202522. This method suppresses halo artifacts by constructing flare masks and flare-aware fusion losses, while combining a lightweight multiscale fusion module with adaptive Fourier filters to achieve high-quality fusion in nighttime conditions. Collectively, these studies advance multimodal fusion technologies toward greater robustness and practicality in smoke and adverse weather scenarios. Overall, research on smoke scene image fusion is still in its early stages. Within this specialized field, numerous challenges remain unresolved. The limited existing literature only offers preliminary solutions, with lightweight models for simultaneous smoke removal and image fusion being relatively scarce. This study aims to provide a novel solution in this area and contribute to further advancements in the field. Lightweight multimodal image fusion method based on CNN-transformer Overall fusion method A lightweight multimodal image fusion method based on CNN-Transformer has been proposed. This method adopts a two-stage training strategy and combines the spatial feature extraction ability of the convolutional neural network and the global dependency modeling advantage of Transformer. The overall network architecture consists of an encoder, a lightweight latent feature mapping module, a fusion module, and a decoder. This model can effectively handle the feature extraction and fusion tasks of multimodal images in a smoke environment. The detailed system architecture diagram is shown in Fig. 1. Overall structure diagram In the encoder part, a hybrid structure was designed, which includes a CNN branch and a Transformer branch. The CNN branch adopts a combination of initial feature extraction and residual connections, and extracts local spatial features of the image through multiple convolution operations. The Transformer branch utilizes the self-attention mechanism to capture the global context information of the image. The features of the two branches are effectively fused at the intermediate layer to form complementary feature representations. The potential feature mapping part is the core of direct smoke fusion in the smoke scene. For the design of the potential feature mapping in this paper, the lightweight design principle was followed. To reduce the model size, the model structure of the potential feature mapping adopts a three-connected double convolution network. A deeper network structure may increase the fusion accuracy, but it will also lead to a decrease in the fusion speed. To balance the fusion performance and fusion speed, we finally chose the network structure as shown in Fig. 2. Potential feature mapping network The fusion module has multiple fusion strategies. These strategies include average fusion based on addition, fusion based on channel attention, fusion based on spatial attention, and fusion based on axial attention. These strategies can be flexibly selected according to the requirements of different fusion tasks to achieve effective integration at the feature level. To reduce the error loss in the fusion process, the basic average fusion strategy was ultimately selected in the model shown in Fig.1. The decoder part adopts a mixed structure of double convolution and single convolution. It gradually reduces the number of channels of the fused features until they return to the original image size. Finally, fused images are generated. The entire network design fully considers the balance between computational efficiency and feature expression ability, ensuring a high-quality fusion effect while maintaining a lightweight structure. One stage training The main purpose of the first stage of training is to pre-train the encoder-decoder network so that it can effectively reconstruct the input image. This stage adopts the self-supervised learning method, optimizing the network parameters by minimizing the difference between the reconstructed image and the original image. The model in this stage can also use the pre-trained model parameters that have been trained by other models. During the training process, the mean square error loss, the structural similarity loss, and the gradient loss are jointly used to form the objective function. The training loss function of the first stage is shown in equation 1. The mean square error loss ensures the pixel-level reconstruction accuracy, the structural similarity loss preserves the structural information of the image, and the gradient loss retains the edge and texture details of the image. The weighted combination of these three losses can effectively supervise the learning process of the network. For one stage, the overall training function is as shown in equation 1. Where, represents the pixel loss in the first-stage training, represents the structural similarity loss in the first-stage training, and represents the gradient loss in the first-stage training. The calculation formula for is shown in equation 2. Where, represents the pixel value of the input image at the i-th position, Oi represents the pixel value of the output image at the i-th position, and N represents the total number of pixels. The calculation formula of is shown in equation 3. Where, I represents the input image, O represents the output image, and ssim represents the structural similarity function and its calculation is as shown in equation (4). Where, both ssim(I,F) and ssim(I,S) used equation (6) for calculation. and are the pixel mean values respectively, and and are the variances respectively, is the covariance respectively, and and are both small constants. The calculation equation of LG is shown as (5). Where, the content in the first bracket represents the gradient of image I; the content in the second bracket represents the gradient of image O, and m is the quantity. For the second stage, the overall training function is as shown in equation (6). Where, represents the loss between the input image and the output image, represents the perceptual loss, and λ is the adjustment coefficient. The calculation of is as shown in equation (7). The calculation of is as shown in equation (8). Where, represents the feature map of the j-th layer of the pre-trained network, denotes the number of channels of the j-th layer, indicates the height of the feature map of the j-th layer, and represents the width of the feature map of the j-th layer. A more detailed training process is as follows. Firstly, the input images of the training set are sent to the encoder to extract features, and then the images are reconstructed through the decoder. The multi-scale loss between the reconstructed image and the original image is calculated, and the network parameters are updated through backpropagation. The training uses the Adam optimizer, and the learning rate is dynamically adjusted according to the training progress. In terms of data preprocessing, the input images are randomly cropped and normalized to increase the diversity of the data. The batch size is set to an appropriate value to ensure the stability of the training. The number of training cycles is determined based on the performance of the validation set to avoid the occurrence of overfitting. After the first stage of training is completed, the encoder has acquired a powerful ability to extract features, which can provide high-quality feature representations for the subsequent fusion tasks. This stage of pre-training lays a solid foundation for the second stage of training. Two stage training The second stage training is based on the first stage pre-training and specifically focuses on training and optimizing the potential feature mapping network. This stage employs supervised learning and uses paired multimodal image data with and without smoke to train the potential feature mapping network. The loss function is designed to comprehensively consider pixel-level consistency, structural similarity, and perceptual quality. Besides the mean square error and structural similarity loss, perceptual loss is also introduced. During the training process, a cosine annealing learning rate scheduling strategy is adopted, gradually reducing the learning rate as the number of training steps increases. This helps the model converge to a better local minimum. At the same time, an early stopping mechanism is used to prevent overfitting. Training is terminated when the performance change in the validation set is small or no longer improves. The key innovation of the two-stage training lies in using the encoder features pre-trained in the first stage as the input, and further optimizing the feature representation through a lightweight latent feature mapping network. This two stage training strategy not only ensures the quality of feature extraction but also improves the fusion performance while maintaining the lightweight nature of the model. Lightweight design In terms of the lightweight design of the encoder and decoder, this method adopts a strategy that combines shallow CNNs with shallow Transformers. Specifically, the CNN branch in the encoder uses only a few convolutional layers and skip connections, effectively reducing the number of parameters. Meanwhile, the Transformer branch is also limited to a relatively shallow network depth, using self-attention mechanisms only at key layers to capture global context information without constructing deep cascaded structures. The encoder layers are also composed of a few convolutional layers in series and avoid introducing additional attention mechanism modules. This design compresses the model size and reduces computational costs while retaining the ability to extract local and global features. In the lightweight design of the latent feature mapping part, this method adopts a three-layer cascaded dual convolutional network structure. By controlling the network depth to be three basic modules and avoiding the use of complex attention layers such as self-attention within each module, only simple residual connections are retained when necessary. Compared with the configuration using a deeper network, the three-layer dual convolutional structure reduces the number of parameters and inference time while ensuring the fusion accuracy. Although a deeper structure can slightly improve the fusion accuracy, it will lead to a significant speed drop. Therefore, the final choice of a three-layer structure is made as an intermediate solution to balance the fusion performance and speed, achieving the lightweight design. Evaluation index To comprehensively evaluate the effect of image fusion, this study employed a variety of quantitative evaluation metrics to measure the quality of the fused images from different perspectives. The main indicators used include PSNR, MSE, SSIM and ACS. Regarding the basic index, psnr represents the peak signal-to-noise ratio of two images, and its calculation equation is as shown in (9). PSNR is the peak signal-to-noise ratio of multimodal images, and its calculation equation is as shown in (11). Where, psnr(, ) is the peak signal-to-noise ratio between and , mse(, ) is the mean square error between the and , m and n represent the image height and image width, and (i, j) and (i, j) are the corresponding pixels. I and S are the input images of multimodal images, F is the fused image of multimodal images. For MSE, its calculation is as shown in equation (12). Where, mse(I,F) and mse(S,F) can be calculated using equation (10). For SSIM, the calculation is as shown in equation (13). Where, I and S represent the input multimodal images, while F represents the fused image. Since the range of ssim is from 0 to 1, the range of SSIM is from 0 to 2. The SC represents the smoke concentration of the tested image. The SC formula23 is as shown in equation (14). Where, x and y represent the pixel points of the image. SC is a smoke concentration indicator obtained by performing weighted averaging and normalization on the minimum values of the RGB three channels of each image pixel within a 15×15 neighborhood after minimum value filtering. For the calculation of ASC, the SC values of the tested fused image are calculated separately, and then their average values are calculated. Furthermore, during the analysis of the ablation experiments, to comprehensively evaluate the influence of each component of the model, three additional evaluation indexes (SCD, EN, and Qabf) were introduced. These supplementary indicators provide auxiliary analysis bases from the perspectives of visual saliency preservation, information richness, and edge detail transfer. Smoke scene dataset For the self-supervised training of the encoder and decoder, this paper uses the public COCO dataset for training. For the training of the latent feature mapping, this paper uses the training set of the public dataset HazeScene23. To verify the adaptability of the proposed method in complex smoke scenarios, this research constructs a multimodal image fusion performance verification dataset that includes images with different smoke concentrations. The dataset covers three smoke conditions: medium concentration, high concentration, and ultra-high concentration, simulating the changes in visibility in real scenarios. The smoke concentrations reach 0.5, 0.75, and 0.9 respectively. The larger the value, the higher the smoke concentration, with 1 representing the full smoke concentration. The adopted dataset is derived from the M3FD dataset24. The establishment of this smoke dataset provides a standard testing platform for multimodal image fusion research, and is particularly suitable for evaluating the performance of algorithms in harsh environments. The proposal of this dataset helps to promote the development of image fusion technology in extreme smoke applications. The HazeScene includes the training dataset and the test dataset. The training dataset contains multi-modal smoke images of original size, segmented multi-modal smoke images, single-modal smoke images of original size, and segmented single-modal smoke images. The quantities of these are 2400 pairs, 12468 pairs, 2400 pairs, and 12468 pairs respectively. For the two-stage training, it is recommended to use the dataset with segmentation and quantity enhancement to increase the size of the data. The test dataset contains five categories: ground truth, light, medium, heavy, and heavy-various. Each category contains 82 pairs of test images. The five categories respectively represent pairs of smoke-free images, low-concentration images, medium-concentration images, heavy-concentration images, and mixed-concentration images. Experiment Experimental setup During training, the CPU frequency of the computer was set at 2GHz, with a four-core architecture. The system memory was 32GB. The video memory size was 16GB. For the training of the potential feature mapping network, the total number of training steps was 20,000, the evaluation steps were set at 20, the evaluation interval was 1,000, the initial learning rate was 0.0001, and the crop size was 256. The dataset used for training the model was the HazeScene dataset, and the dataset for performance verification was the extreme haze scene dataset proposed in this paper. Analysis of the training process of encoder and decoder In the analysis of the training process of the encoder and decoder, we recorded the trend of the total loss, which was composed of pixel loss, structural similarity loss, and gradient loss. The specific trend is shown in Fig. 3. From the experimental data, the total loss decreased rapidly in the early stage of training and then gradually converged to a relatively stable low value range. To more clearly display the subtle changes in the loss function in the later stage of convergence, we performed local magnification on the tail data with smaller values. This visualization method effectively revealed the stable characteristics of the loss curve as it approached convergence, and also highlighted the possible minor fluctuations in the later training stage. From the perspective of the data content of the training, the gradient loss was the main loss for the training. However, overall, the losses of the encoder and decoder of the model were at a relatively low level. Training losses of the encoder and decoder Comparison of fusion performance under different smoke concentrations We conducted a quantitative comparison of the integration performance of our method with existing advanced algorithms under different smoke concentrations. The experiment selected four key indexes: PSNR, MSE, SSIM, and ASC. The performance of DSFusion, FISCNet, LENFusion, MMNet, and our proposed method was evaluated respectively in medium, high, and ultra-high smoke concentration scenarios. Under the condition of medium smoke concentration, as shown in Table 1, the method proposed in this paper demonstrates certain advantages in all indexes. Its PSNR value reaches 63.57, which is the highest among all methods. Meanwhile, the MSE value is the lowest, at 0.035. In terms of image structure preservation, the SSIM of this method is 1.405, superior to other comparison algorithms. Moreover, the ASC value obtained by this method is only 0.319, much lower than other methods, indicating that its smoke removal effect is the most thorough. In high smoke concentration scenarios, as shown in Table 2, this method still maintains overall superiority. Its PSNR value is 63.21, and the MSE value is 0.036, both being the best. The SSIM value of 1.380 and the ASC value of 0.364 are also the best among all methods, confirming the robustness of this method under moderate interference. Facing the challenge of extremely high smoke concentration, the data in Table 3 further consolidates the superiority of this method. Its PSNR value is 62.72 and the SSIM value is 1.354, both being the highest, while the MSE value is 0.038 and the ASC value is 0.387, both being the lowest. In summary, in different smoke concentration environments, this method consistently outperforms the existing advanced algorithms such as DSFusion, FISCNet, LENFusion, and MMNet in terms of PSNR, SSIM, MSE, and ASC. This fully demonstrates the effectiveness and superiority of its fusion strategy in improving image quality and enhancing smoke removal capabilities. Table 1Performance comparison of medium smoke concentrationIndexDSFusionFISCNetLENFusionMMNetOursPSNR58.0257.5259.4162.4763.57MSE0.1060.1190.0830.0390.035SSIM1.2471.2451.1511.3671.405ASC0.6200.5640.4210.4640.319 Performance comparison of medium smoke concentration Table 2Performance comparison of high smoke concentrationIndexDSFusionFISCNetLENFusionMMNetOursPSNR56.1955.8759.4961.1863.21MSE0.1590.1700.0820.0510.036SSIM1.1611.1361.1601.3231.380ASC0.7110.6380.4170.5160.364 Performance comparison of high smoke concentration Table 3Performance comparison of ultra-high smoke concentrationIndexDSFusionFISCNetLENFusionMMNetOursPSNR55.2354.9559.4660.6262.72MSE0.1990.2110.0840.0590.038SSIM1.1091.0561.2001.2951.354ASC0.7630.6740.4320.5340.387 Performance comparison of ultra-high smoke concentration Figure 4 presents a comparison of the smoke removal fusion results obtained by different fusion methods under ultra-high smoke concentration. It can be observed that DSFusion and FISCNet do not have the ability to perform smoke removal fusion. LENFusion has the ability to enhance contrast. While MMNet and our method have better smoke removal fusion capabilities. Comparison of smoke removal and fusion results using different fusion methods under ultra-high smoke concentration Compared with MMNet, the method proposed in this paper has a stronger advantage in the ability to remove smoke. In general, the method proposed in this paper can both remove smoke and integrate the information of the two modalities. Meanwhile, our method is closer to the color of the original image. Analysis of smoke removal capability comparison This section conducts a comparative analysis of the smoke removal capabilities. The first 50 data points were selected from three smoke concentration scenarios of medium, high, and ultra-high. Detailed performance evaluations were carried out on DSFusion, FISCNet, LENFusion, MMNet, and the method proposed in this paper. The SC curves plotted are shown in Fig. 5. Comparison of smoke removal capabilities of multimodal fusion methods in different smoke scenarios The experimental data indicate that at medium smoke concentration, the method proposed in this paper performs the best. Its value remains at the lowest level throughout the observation range and is lower than that of the comparison methods. The curves of DSFusion and FISCNet are relatively higher and have more obvious fluctuations. The performance of LENFusion and MMNet is in the middle range, but neither has achieved the smoke removal effect as good as that of the method. This pattern also applies to high smoke concentration and ultra-high smoke concentration. The LENFusion method has excellent smoke removal capabilities. On a small portion of the data, this method even outperformed the method proposed by us in terms of smoke removal ability. However, by combining the three subgraphs, it can be observed that as the smoke concentration increases, the stability of this method shows a downward trend. By combining the analysis results of the three concentrations, it can be seen that the method we proposed can achieve lower de-smoke concentration indicators under different smoke concentration conditions, and its performance curve is located at the bottom of the range to the greatest extent. This further verifies the effectiveness and stability of this method in the de-smoke task. Lightweight analysis Regarding the model parameter quantity, this method achieves a compact model design through lightweighting technology. The encoder adopts a shallow parallel structure of CNN and Transformer. The CNN branch only uses a few convolutional layers and does not introduce additional attention mechanisms. The Transformer branch is also limited to a relatively shallow depth. The latent feature mapping network adopts a simple structure of three-layer dual convolutions, avoiding the parameter expansion caused by deep stacking or complex attention modules. Quantitative comparison results are shown in Table 4. The model weight of this method is 19.64 MB, which is lower than 56.5 MB of MMNet. Table 4Weights and average fusion speed of different fusion methodsMethodWeights (M)Medium smoke (s/pair)High smoke (s/pair)Ultra-high smoke (s/pair)DSFusion1.811.841.822.03FISCNet1.416.996.676.83LENFusion2.5413.9113.2614.15MMNet56.51.941.861.91Ours19.640.740.720.73 Weights and average fusion speed of different fusion methods In terms of reasoning speed, this method demonstrates certain advantages. Since both the encoder and the latent feature mapping part adopt lightweight designs, the speed of feature extraction and fusion is relatively fast. We tested the average fusion speed under different smoke concentration conditions, and the results are shown in Table 4. Under medium smoke concentration, the average fusion speed of this method is 0.94 seconds per pair, which is much faster than 1.84 seconds per pair for DSFusion, 6.99 seconds per pair for FISCNet, 13.91 seconds per pair for LENFusion, and 1.94 seconds per pair for MMNet. Under high smoke and ultra-high smoke conditions, this method still maintains the fastest fusion speed, which is 0.72 seconds per pair and 0.73 seconds per pair respectively. This is mainly due to two aspects. First, the use of the average fusion strategy instead of complex attention transformation fusion reduces the computational cost of the fusion layer. Second, the entire fusion network is relatively lightweight. The above quantitative evidence fully validates the practicality of this method in lightweight design. Ablation experiment Different lightweight feature mapping structures The potential feature mapping network is one of the very core structures of the direct smoke fusion model. Its structure and model size will affect the degree of lightweight of the entire model. To further analyze the influence of different lightweight structures on the potential feature mapping network, a total of three different lightweight structures were selected, named topology1, topology2, and topology3. Among these structures, topology1 has a structure of a single convolution, ReLU, and another single convolution in series. While topology2 consists of two topology1s in series, and topology3 consists of three topology1s in series. The reason for not selecting more repetitions is to reduce the model complexity. Figures 6 and 7 present the performance trends of training ssim and test loss under different intervals for different topological structures. It can be observed that under the same training conditions, the final training ssim value of topology 3 reaches the maximum, which is slightly higher than that of topology 2. Regarding the test loss, topology 3 is also at a relatively lower level, which is slightly lower than that of topology 2. The trend of training ssim under different intervals for different topological structures The trend of training test loss under different intervals for different topological structures Although as the number of basic topological structures increases, the training performance of the model shows an upward trend. To balance the rapid integration of the model and its complexity, the model in this paper selects topology 3 as the final structure of the potential feature mapping model. Different fusion strategies We conducted a systematic study on various fusion strategies. The experimental design included four representative fusion strategies, and the strategies are the simple average fusion, the channel attention fusion based on weight distribution in the channel dimension, the spatial fusion focusing on the relationship between pixels, and the axis attention fusion combining feature axis information. These strategies were aimed at exploring the impact of different feature integration mechanisms on the final smoke removal performance, and were comprehensively evaluated through multiple objective indexes. Table 5 clearly reveals the performance differences among various fusion strategies, and the bolded text represents the optimal values. The channel attention fusion achieved the best results in terms of PSNR and SSIM, reaching 62.82 and 1.365 respectively, indicating its advantages in preserving image structural information and improving signal-to-noise ratio. However, the average fusion strategy achieved the optimal values in MSE and SCD, with MSE as low as 0.038 and SCD as high as 0.949, which shows that this strategy is more effective in suppressing errors and enhancing spectral consistency. The spatial fusion and axis attention fusion achieved generally lower indexes compared to the former two strategies. Among them, the axis attention fusion had an MSE as high as 0.101 and a SSIM of only 1.120, indicating relatively poor performance. In summary analysis, channel attention fusion is better at fine feature selection and structure preservation, while average fusion performs better in global error control and color fidelity. Table 5Performance comparison of different fusion strategiesIndexAverage fusionChannel attention fusionSpatial attention fusionAxis attention fusionPSNR62.7262.8261.1959.43MSE0.0380.0400.0520.101SCD0.9490.8740.7440.891EN6.2615.9464.7505.760Qabf0.1010.0750.0710.094SSIM1.3541.3651.2901.120 Performance comparison of different fusion strategies After comprehensive consideration of all the indexes, especially to effectively control the mean square error and enhance the stability of the overall fusion, this paper finally chose the average fusion as the core fusion strategy. This strategy not only achieved the lowest MSE value, but also its SCD and EN were at the leading level, achieving a good balance in the key quality evaluation dimensions. This laid a reliable feature foundation for the subsequent smoke removal processing. Conclusion This paper addresses the challenges of image degradation, model complexity, and process separation in multimodal image fusion under smoke environments, and proposes a lightweight CNN-Transformer direct smoke removal fusion method based on two-stage training. The core contribution of this method lies in constructing a lightweight end-to-end unified framework that can simultaneously achieve effective smoke suppression and high-quality infrared-visible information fusion, avoiding the complexity of step-by-step processing. Through ablation experiments, the performance of different fusion strategies under the unified framework was compared and analyzed. Finally, the average fusion strategy with better performance in error control and stability was selected. The method was fully validated on datasets of different smoke concentration scenarios. The experimental results show that this method outperforms current mainstream methods in multiple key indexes such as PSNR, SSIM, MSE, and ASC, demonstrating excellent smoke removal ability and information integration capability. However, this study mainly focuses on the fusion of static images. Future work can further explore the scalability and real-time performance of this method in video sequence fusion tasks. Additionally, how to adapt the framework more flexibly to other extreme visual conditions such as rain, snow, and dust for multimodal fusion is also a direction worthy of in-depth research. Author contributions J.M. and Y.Z. wrote the main manuscript text and M.Y. and X.J. provided guidance on the whole content. All authors reviewed the manuscript. Funding This work was supported in part by the National Natural Science Foundation of China under Grant 62541123, and in part by Fujian Provincial Natural Science Foundation of China under Grant 2025J08243, 2026J0011094, and in part by the Introduction of Talents to Start Scientific Research Projects in Putian University under Grant 2023015, 2025035, 2025036, and in part by the Putian Science and Technology Plan Project under Grant 2023GZ2001PTXY19. Data availability The data supporting this study are available at https://github.com/windrunners/LMIF. Declarations Competing interests The authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Contributor Information Jie Ma, Email: majie2021@seu.edu.cn. Yangyang Zhao, Email: zhaoyangyangcn@163.com. References Associated Data Data Availability Statement The data supporting this study are available at https://github.com/windrunners/LMIF.