Cross-Modal Interaction Fusion-Based Uncertainty-Aware Prediction Method for Industrial Froth Flotation Concentrate Grade by Using a Hybrid SKNet-ViT Framework Authors: Fanlei Lu, Weihua Gui, Yulong Wang, Jiayi Zhou, Xiaoli Wang Published: 2025 ABSTRACT In froth flotation, the features of froth images are important information to predict the concentrate grade. However, the froth structure is influenced by multiple factors, such as air flowrate, slurry level, ore properties, reagents, etc., which leads to highly complex and dynamic changes in the image features. Additionally, issues such as the immeasurability of ore properties and measurement errors pose significant uncertainties including aleatoric uncertainty (intrinsic variability from ore fluctuations and sensor noise) and epistemic uncertainty (incomplete feature representation and local data heterogeneity) and generalization challenges for prediction models. This paper proposes an uncertainty quantification regression framework based on cross-modal interaction fusion, which integrates the complementary advantages of Selective Kernel Networks (SKNet) and Vision Transformers (ViT). By designing a cross-modal interaction module, the method achieves deep fusion of local and global features, reducing epistemic uncertainty caused by incomplete feature expression in single-models. Meanwhile, by combining adaptive calibrated quantile regression—using exponential moving average (EMA) to track real-time coverage and adjust parameters dynamically—the prediction interval coverage is optimized, addressing the inability of static quantile regression to adapt to aleatoric uncertainty. And through the localized conformal prediction module, sensitivity to local data distributions is enhanced, avoiding the limitation of global conformal methods in ignoring local heterogeneity. Experimental results demonstrate that this method significantly improves the robustness of uncertainty estimation while maintaining high prediction accuracy, providing strong support for intelligent optimization and decision-making in industrial flotation processes. 1. Introduction Froth flotation is a critical process in mineral processing, which separates valuable minerals from gangue based on their physicochemical properties. Accurate and timely measurement of concentrate grade is paramount for optimization and control of the flotation process to enhance resource utilization [1]. However, complex on-site environment, high cost, and maintenance demand of the online analysis instruments mean that the industrial operations often rely on manual observation or offline laboratory analysis (e.g., X-ray fluorescence). These methods are subjective and introduce significant time lags (typically 1–2 h), which is inadequate for real-time closed-loop control [2]. Since froth flotation images contain a wealth of process state information, image-based grade soft sensing methods have become an active research focus. Meanwhile, with the rapid development of deep learning and cumulative of process data, automatic feature extraction and representation methods, and prediction methods based on deep neural networks for flotation grade have been researched broadly [3]. Methodologically, most works rely on Convolutional Neural Networks (CNNs) because froth images carry rich cues. Wang et al. [4] proposed a two-step operating condition recognition strategy based on image sequences, where deep learning was used to classify the images to obtain the conditions of antimony flotation, and satisfactory classification results were achieved. Fu et al. [5,6] investigated several popular CNNs for classification of arsenic flotation conditions, and experiment results showed that the ResNet achieved the best classification results. Zarie et al. [7] developed a froth image classification method based on CNN for process monitoring. The study demonstrated that the CNN classifier effectively extracts froth features and its classification accuracy outperforms traditional artificial neural networks (ANNs). Jia et al. [8] proposed a CNN-based performance evaluation method with feature learning and expert feedback fusion, which improves robustness and accuracy under image distortions. Beyond purely spatial cues, sequence models capture temporal dynamics. Zhou et al. [9] proposed a domain-knowledge-embedded deep Long Short-Term Memory (LSTM) soft measurement framework (DKST-DLSTM), which significantly improved the prediction accuracy of key indicators such as concentrate grade by extracting spatio-temporal features in the flotation process. Pu et al. [10] used LSTM to predict iron and silicon waste concentrations by using industrial data, which outperforms random forests, and Zhang et al. [11] combined feed grade refined embedding (FGRE) and nonoverlapping patch encoding (NOPE) with RNNs to enhance dynamic monitoring accuracy. In addition, graph-based and attention models have been explored too. Wang et al. [12] proposed a sparse-attention spectral graph neural network (SASGNN) for state-variable prediction, while Lu et al. [13] developed a multimode performance assessment method using a deep embedded graph clustering network that integrates froth images and process variables via Bayesian fusion. Complementary to these architectures, Xie et al. [14] employed an mRMR plus multiswarm PSO-based sensitive feature selection with collaborative search, coupled with a self-adjusting RBF neural network, to further improve flotation prediction accuracy. Despite progress in deep learning for flotation grade prediction, most methods provide only point estimates without calibrated uncertainty. Uncertainty arises from two sources: aleatoric (intrinsic process variability such as changing ore conditions, operating fluctuations, and sensor noise) and epistemic (limited/delayed labels, unobserved variables, and covariate shift). Hence, soft sensors should output accurate predictions with reliable intervals under distributional change. While CNNs emphasize local details [15] and ViTs capture global context [16], a single model struggles to account for both local variability and global trends. For modeling uncertainty in industrial processes, primary approaches include probabilistic distribution modeling [17], quantile regression [18], and conformal prediction [19]. While these methods provide interval estimates, they largely depend on global statistical characteristics, making it difficult to adapt in real-time to local distributional variations caused by raw material or process fluctuations in flotation operations. Moreover, static calibration may result in prediction intervals being either too wide or too narrow [20], thereby affecting reliability. Although research on uncertainty prediction for flotation processes is scarce, advanced techniques have emerged in other complex systems. For instance, in probabilistic distribution prediction, Dong et al. [21] proposed an uncertainty-guided graph neural network for air quality forecasting, Sun et al. [22] developed a multi-distribution ensemble framework, and Trapero et al. [23] combined kernel density estimation with forecast errors. In terms of hybrid modeling and interval estimation, Wang et al. [24] employed an integrated Empirical Mode Decomposition (EMD) and Relevance Vector Machine (RVM) model for photovoltaic power output, Khosravi et al. [25] proposed a neural network-based method for constructing prediction intervals, and Hou et al. [26] developed an uncertainty-aware deep learning framework for risk identification. Other methods, such as the non-parametric quantile approach by Bremnes et al. [27] or the fuzzy theory-based intervals by Pierre et al. [28], are often limited by linear structure assumptions or reliance on expert knowledge, respectively. Despite these advances, their direct application is challenged by the unique multi-modal data and dynamic, nonlinear behavior of flotation—even as effective multi-modal fusion strategies have been developed for other scenarios [29,30]. Therefore, how to integrate multi-modal deep features and achieve dynamic, locally sensitive uncertainty quantification remains an urgent and unresolved challenge. Although intelligent methods have been extensively explored for uncertainty modeling in industrial processes [31], they often suffer from insufficient robustness and limited generalization capability when they face high-noise and highly dynamic of the input data, making it difficult to meet the practical demand for high-confidence interval prediction in production. Therefore, integrating multi-modal structures with stronger expressive power and theoretically guaranteed out-of-distribution uncertainty calibration has become a key breakthrough in current research. In addition, traditional quantile regression and global conformal prediction methods also exhibit significant limitations in constructing prediction intervals. On one hand, these methods typically rely on the statistical characteristics of global data, making it difficult to adapt in real-time to local distributional differences in flotation processes caused by variations in ore properties or fluctuations of operating conditions. On the other hand, static calibration methods often produce prediction intervals that are either too wide or too narrow in the face of dynamically changing data distributions, thereby reducing the reliability of decision-making in practical applications. To address the aforementioned challenges, this paper proposes a cross-modal interactive fusion-based uncertainty quantification regression framework for high-precision prediction and uncertainty quantification of concentrate grade in flotation. First, a hybrid architecture integrating SKNet and ViT is constructed, in which bidirectional information interaction is achieved through a cross-attention mechanism, so that local details and global features are effectively combined. Subsequently, we design an adaptive calibrated quantile regression method that introduces an exponential moving average (EMA) update strategy for coverage rate and an adaptive loss function, thereby enabling dynamic adjustment of calibration parameters and enhancing the coverage capability of prediction intervals under dynamic operating conditions. Finally, a conformal prediction module based on local data distribution is developed, which improves the model’s responsiveness to local data fluctuations and further enhances the reliability of prediction intervals by incorporating local neighborhood analysis and nonlinear interval width adjustment. The main contributions of this paper can be summarized as follows:(1)A novel cross-modal feature fusion architecture is proposed, which is based on the integration of SKNet (convolutional features) and ViT (global structure) with cross-attention. By designing a cross-modal interaction module, complementary learning between different feature extractors is achieved. This approach enables simultaneous capture of local details and global information in flotation process data, significantly enhancing the model’s capacity to represent complex data patterns.(2)An adaptive calibrated quantile regression framework is developed, coupling an EMA-driven coverage-rate updater with a loss comprising quantile error and a calibration-penalty term, enabling real-time reweighting and improved interval coverage under nonstationary conditions.(3)A localized conformal prediction method based on local neighborhood analysis is proposed. By mining the local features of the calibration set and adjusting the interval width nonlinearly, the prediction intervals become highly sensitive to local data distributions. Automatic iterative optimization is used to effectively enhance the reliability of intervals for samples from different regions.(4)A complete end-to-end framework covering data preprocessing, feature fusion, model training, and uncertainty prediction interval evaluation is constructed. The method’s superior performance on real flotation datasets is systematically validated through multi-dimensional experiments and comprehensive metrics (MAE, MSE, coverage, interval width, etc.), providing technical support for industrial applications. (1)A novel cross-modal feature fusion architecture is proposed, which is based on the integration of SKNet (convolutional features) and ViT (global structure) with cross-attention. By designing a cross-modal interaction module, complementary learning between different feature extractors is achieved. This approach enables simultaneous capture of local details and global information in flotation process data, significantly enhancing the model’s capacity to represent complex data patterns. A novel cross-modal feature fusion architecture is proposed, which is based on the integration of SKNet (convolutional features) and ViT (global structure) with cross-attention. By designing a cross-modal interaction module, complementary learning between different feature extractors is achieved. This approach enables simultaneous capture of local details and global information in flotation process data, significantly enhancing the model’s capacity to represent complex data patterns. (2)An adaptive calibrated quantile regression framework is developed, coupling an EMA-driven coverage-rate updater with a loss comprising quantile error and a calibration-penalty term, enabling real-time reweighting and improved interval coverage under nonstationary conditions. An adaptive calibrated quantile regression framework is developed, coupling an EMA-driven coverage-rate updater with a loss comprising quantile error and a calibration-penalty term, enabling real-time reweighting and improved interval coverage under nonstationary conditions. (3)A localized conformal prediction method based on local neighborhood analysis is proposed. By mining the local features of the calibration set and adjusting the interval width nonlinearly, the prediction intervals become highly sensitive to local data distributions. Automatic iterative optimization is used to effectively enhance the reliability of intervals for samples from different regions. A localized conformal prediction method based on local neighborhood analysis is proposed. By mining the local features of the calibration set and adjusting the interval width nonlinearly, the prediction intervals become highly sensitive to local data distributions. Automatic iterative optimization is used to effectively enhance the reliability of intervals for samples from different regions. (4)A complete end-to-end framework covering data preprocessing, feature fusion, model training, and uncertainty prediction interval evaluation is constructed. The method’s superior performance on real flotation datasets is systematically validated through multi-dimensional experiments and comprehensive metrics (MAE, MSE, coverage, interval width, etc.), providing technical support for industrial applications. A complete end-to-end framework covering data preprocessing, feature fusion, model training, and uncertainty prediction interval evaluation is constructed. The method’s superior performance on real flotation datasets is systematically validated through multi-dimensional experiments and comprehensive metrics (MAE, MSE, coverage, interval width, etc.), providing technical support for industrial applications. The structure of this paper is arranged as follows: Section 1 reviews the relevant theoretical foundations and research progress; Section 2 provides a detailed description of the proposed method and its implementation; Section 3 presents experimental design, result analysis, and discussion; and finally, Section 4 concludes the paper and discusses future research directions. 2. Methodology: Cross-Modal Modeling Method with Adaptive and Localized Uncertainty The system begins by acquiring real-time flotation froth images, which undergo preprocessing to ensure high-quality inputs for subsequent feature extraction. The multi-scale feature extraction module employs both a multi-scale variable receptive field convolutional layer (SKNet) and Vision Transformer (ViT) to extract rich features at different scales. Positional embeddings and a transformer encoder further enhance the representation of spatial structures. In the cross-modal feature fusion stage, the multi-scale features extracted from the images are fused to leverage the correlations between different modalities, thereby enhancing the expressive power of the features. This results in a high-dimensional fused feature vector used for prediction. To address the uncertainty in grade prediction, the system incorporates an uncertainty modeling module. Specifically, we use quantile regression to generate interval predictions. This is combined with localized conformal prediction to quantify the uncertainty of the results while maintaining accuracy. An adaptive calibration mechanism further refines the confidence intervals to align with observed data, improving the model’s reliability in industrial settings. Finally, the system outputs not only the predicted grade value but also the corresponding prediction interval and an analysis of uncertainty, providing a scientific basis for intelligent process control and risk-informed decision-making. The overall architecture of the proposed grade prediction and uncertainty quantification framework is illustrated in Figure 1. 2.1. Feature Extraction Backbone: SKNet and ViT Flotation froth images are characterized by complex local textures and global structural patterns. To simultaneously capture these multi-granular features, we construct a dual-branch backbone comprising a Selective Kernel Network (SKNet) and a Vision Transformer (ViT). SKNet is employed to extract multi-scale local features. By dynamically adjusting the receptive fields of its convolutional kernels based on the input information, SKNet effectively captures fine-grained details of froth bubbles at varying scales [32]. Complementarily, ViT serves as the global feature extractor. Utilizing the multi-head self-attention mechanism, ViT models long-range dependencies and global semantic contexts across the entire image, compensating for the limited receptive field of traditional CNNs [33]. The detailed mathematical formulations and architectural specifications of both SKNet and ViT are provided in Appendix A. While these two branches provide rich feature representations independently, simply concatenating them fails to exploit their semantic correlations. To address this, we introduce a cross-modal interaction fusion module in the following section to achieve deep bidirectional information integration. 2.2. Fusion of Cross-Modal Features To fully leverage SKNet’s local detail perception and ViT’s global semantic understanding capabilities, we design a cross-modal interaction fusion module. This module achieves mutual enhancement of features through bidirectional attention mechanisms and generates unified feature representations through deep fusion. Unlike simple concatenation or summation used in standard CNN-transformer hybrids, our proposed cross-modal interaction module (Equations (1)–(4)) employs a bidirectional attention mechanism. This allows the local features from SKNet to modulate the global representations of ViT and vice versa, explicitly modeling the semantic alignment between texture details and global structural patterns. 2.2.1. Bidirectional Attention Modulation Let the features extracted by SKNet and ViT be FSK∈Rd and FViT∈Rd, respectively. First, we calculate attention weights of local features on global features:(1)αsv=σ(Wsv⋅FSK+bsv) where Wsv∈Rd×d is a learnable projection matrix, and σ(⋅) is the sigmoid activation function. This attention weight is then used to modulate global features:(2)F^ViT=FViT⊙αsv Similarly, we calculate attention modulation of global features on local features:(3)αvs=σ(Wvs⋅FViT+bvs)(4)F^SK=FSK⊙αvs 2.2.2. Fusion of Multi-Level Features Original features and interaction-modulated features are concatenated as,(5)Fconcat=[FSK;FViT;F^SK;F^ViT]∈R4d Deep fusion is performed through multi-layer perceptron (MLP):(6)Fhidden=ReLU(W1⋅Fconcat+b1)(7)Ffused=W2⋅Fhidden+b2 where W1∈R2d×4d,W2∈Rd×2d are weight matrices. This fusion strategy preserves unique information from each modality while enhancing feature representation through interaction. 2.3. Adaptive Quantile Regression Model To quantify the uncertainty, we propose an adaptive calibrated quantile regression method which not only predicts multiple quantiles to construct prediction intervals but also ensures interval reliability through dynamic calibration mechanisms. 2.3.1. Quantile Regression Network For fused features Ffused, K quantiles are predicted through independent fully connected networks:(8)y^τi=fτi(Ffused;θτi),i=1,2,…,K where τi∈(0,1) is the target quantile, and fτi(⋅) is the corresponding prediction network. 2.3.2. Pinball Loss Function Quantile regression typically employs the Pinball loss function:(9)Lτ(y,y^τ)={τ(y−y^τ)if y≥y^τ(1−τ)(y^τ−y)if y