Predicting removal of arsenic from groundwater by iron based filters using deep neural network models Authors: Muhammad Ali Martuza, Md Shafiquzzaman, Husnain Haider, Amimul Ahsan, Abdelkader T Ahmed Published: 2024 ABSTRACT AbstractArsenic (As) contamination in drinking water has been highlighted for its environmental significance and potential health implications. Iron-based filters are cost-effective and sustainable solutions for As removal from contaminated water. Applying Machine Learning (ML) models to investigate and optimize As removal using iron-based filters is limited. The present study developed Deep Learning Neural Network (DLNN) models for predicting the removal of As and other contaminants by iron-based filters from groundwater. A small Original Dataset (ODS) consisting of 20 data points and 13 groundwater parameters was obtained from the field performances of 20 individual iron-amended ceramic filters. Cubic-spline interpolation (CSI) expanded the ODS, generating 1600 interpolated data points (IDPs) without duplication. The Bayesian optimization algorithm tuned the model hyper-parameters and IDPs in a Stratified fivefold Cross-Validation (CV) setup trained all the models. The models demonstrated reliable performances with the coefficient of determination (R2) 0.990–0.999 for As, 0.774–0.976 for Iron (Fe), 0.934–0.954 for Phosphorus (P), and 0.878–0.998 for predicting manganese (Mn) in the effluent. Sobol sensitivity analysis revealed that As (total order index (ST) = 0.563), P (ST = 0.441), Eh (ST = 0.712), and Temp (ST = 0.371) are the most sensitive parameters for the removal of As, Fe, P, and Mn. The comprehensive approach, from data expansion through DLNN model development, provides a valuable tool for estimating optimal As removal conditions from groundwater.Keywords: Artificial intelligence, Deep learning neural networks, Groundwater, Arsenic, Pollutant removal, Iron-based filterSubject terms: Environmental sciences, Natural hazards, Engineering, Mathematics and computing Introduction Arsenic (As) contamination in drinking water sources is a significant environmental concern that causes severe health impacts and hazards to the ecosystem1. Arsenic contamination and its prevalence have become an essential concern in developing countries where groundwater is the only source of drinking water supplies—long-term exposure to As can lead to As poisoning, which may cause various health issues. Chronic exposure is linked to skin lesions, multiple cancers (e.g., skin, lung, bladder, and kidney cancer), cardiovascular diseases, and developmental effects2. Arsenic contamination can result in a widespread public health crisis in areas where groundwater is the primary source of drinking water. Arsenite [As(III)] and arsenate [As(V)] are the most hazardous forms of inorganic arsenic, often found in drinking water sources and highly toxic metals3, contributing to human health risks4. Typical concentrations of As in contaminated surface and ground waters range from 0.1 to 230 mg/L across various countries5,6. High levels have been observed in regions like the Bengal Delta Plain and the Ganges River alluvial deposits, where concentrations approached as high as 2000 mg/L7. Approximately 140–200 million people globally are reported to be at risk due to the consumption of arsenic-contaminated groundwater8. Hence, eco-friendly and cost-effective As removal methods are inevitable for ensuring access to safe drinking water in developing countries, especially where As contamination is prevalent9. A passive groundwater treatment method for As removal utilizing native ferrous iron (Fe(II)) in groundwater is a potentially effective means for removing As from groundwater. The process involves the oxidation of Fe(II) and the subsequent co-precipitation of arsenic with in-situ oxidized Fe(III) hydroxides10–12. Several Fe-based arsenic removal technologies have been successfully employed in arsenic-affected regions13–15. These technologies include the SONO filter, Kanchan arsenic filter (KAF), and arsenic removal ceramic filter13–15. A common feature of these technologies is using Fe-amended sand or ceramic filters for arsenic removal. The removal mechanism of these technologies involves forming Fe(III) oxides in water. The corrosion of metal iron components, such as iron nets, iron nails, or iron fillings, facilitates the formation of Fe(III) oxides13–15. Recently developed iron-amended ceramic filters (IACF) for As removal from groundwater have shown high potential to apply in developing countries16,17. IACF is a sustainable method that utilizes locally made ceramic filters and iron nets to supplement Fe(II)13,16. The removal efficiency of these iron-based technologies depends on water biogeochemistry and several other factors. For instance, co-existing cations/anions, such as Fe, P, Si, and Mn, in the natural groundwater significantly impact the As removal18. Recently, laboratory experiments investigated the effect of Fe, Si, and P on As removal by IACF18. However, this approach might oversimplify the complex interactions and influences that multiple factors can have on As removal efficiency. There is a lack of comprehensive multi-factor analysis in the current research. Understanding the interactions of various factors to As removal for accurately representing real-field conditions and the relative contributions of multiple factors to As removal efficiency is challenging, time-consuming, and complex. Developing a comprehensive framework that elucidates the relative importance of different factors in arsenic removal using Fe-based filters is crucial. The interdisciplinary nature of machine learning (ML) technology and its wide-ranging applicability across various research domains allow using large volumes of complex and multidimensional data to develop predictive models19–22. ML can explore the potential of various adsorbents as a tool for the remediation of heavy metals in the environment23–25. The sorption modeling of organic compounds using ML has also been studied extensively22,26. ML showed their potential in other applications such as the oxidation of micro-pollutants27, understanding As(III) and As(V) adsorption onto iron-modified carbon28, immobilization of heavy metals in soil29, distribution of arsenic in groundwater30, and the presence of As in surface water and drinking water sources31. ML theory centers on designing and analyzing algorithms, enabling computers to "learn autonomously." ML algorithms analyze existing data structures to mine rules, allowing the computer to make judgments and predictions about unknown samples. The capability of ML methods facilitates the identification of causal relationships among variables and uncovers the hidden details within data, which is particularly valuable when dealing with complex datasets where conventional analysis may fall short. Therefore, ML can be a useful tool for addressing challenges associated with As contamination. ML can predict the removal efficiency of As in diverse scenarios, aiding in the optimization of treatment systems and enhancing overall remediation effectiveness. Artificial neural networks (ANN) were frequently employed as ML algorithms for modeling various adsorption processes32,33. Other machine learning algorithms (MLAs) have also been used for modeling adsorption processes, such as decision trees, support vector regression, random forest, genetic models, particle swarm optimization, and the adaptive network-based fuzzy inference system (ANFIS)22,24,34. As per recent literature, deep-learning neural network (DLNN) models have better prediction ability than other MLAs35,36. The utilization of ML techniques in As removal by Fe-based treatment methods has not yet been explored. Primarily, the use of the real filed experimental data set to predict the As removal has not been studied to date; this is the first study that addresses this current gap in applying ML to model As removal from groundwater. The study aims to develop DLNN models for predicting the post-filtration concentrations of As, Fe, P, and Mn. The specific objectives are to i) compile a comprehensive filed experimental dataset from the previous studies, focusing on Arsenic (As), Iron (Fe), Phosphorus (P), and Manganese (Mn) removal from groundwater12,13,37, ii) interpolate data points (IDPs) from the field-measured data using the DLNN models, iii) assess the model’s performance using fresh data and, iv) Conduct Sobol sensitivity analysis to identify the critical factors affecting the removal process. Four DLNN models were developed to predict the As, Fe, P, and Mn post-filtration concentrations in groundwater. Each of these models has thirteen inputs measured before filtering groundwater, namely, pH, Eh (redox potential), electrical conductivity (EC), temperature (Temp), dissolved oxygen (DO), and concentration of As, Fe, P, Calcium (Ca), Magnesium (Mg), Silicate (Si), Mn, and total organic Carbon (TOC). The cubic-spline interpolation (CSI) technique was used for data interpolation. Subsequently, each DLNN model was trained in 5-iterations where the MSE was the loss function. This research will provide a framework for identifying key factors influencing inorganic arsenic (As), Fe, P, and Mn removal by Fe-based filters. This framework also facilitates experimental adjustments, possibly reducing the necessity for extensive experimental work. The implementing models have great potential in designing and optimizing treatment processes for As removal from water. Materials and methods Methodological framework The DLNN model was developed based on the available filed experimental data. Figure 1 presents the details of the research methodological formwork. The framework begins by defining the study’s monitoring, collection, and selection of filed data. The physic-chemical water quality data were obtained from the field studies. After getting the field data, the research statistical analysis of the data gained insights, and correlation analysis was performed to understand relationships between different variables within the dataset. To overcome the small dataset issue, the DLNN models interpolated data points (IDPs) generated from the field-measured data without duplication. These models are likely used to predict or analyze aspects related to water quality based on the input features. The next step assessed the model’s performance on data it hadn’t seen during the training phase. Finally, Sobol sensitivity analysis, using the developed DLNN models, identified the direct and indirect vital aspects of removing As, Fe, P, and Mn by the IACFs from groundwater. Methodological framework adopted to conduct the study. Field experiments and data collection The research used the data from the field experiments conducted in Bangladesh under natural conditions. In total, 20 Iron-Amended Ceramic Filters (IACF) were continuously operated for 30 days. The study focuses on three areas in Bangladesh affected by arsenic contamination in groundwater, based on the criterion that 66% of the groundwater in these regions exceeds the national standard limit of 50 μg/L for arsenic in drinking water. It is also worth mentioning that the groundwater in these selected areas is considered representative of Bangladesh’s groundwater geochemistry38. Twenty filters were installed on randomly selected household-owned tubewells in the three Ar-contaminated areas. Figure 2 outlines the operational procedures of IACF in field conditions. The filters were operated by a simple three steps process involving pumping groundwater using a hand tubewell, conducting aeration, and feeding water to the filtration bucket (Fig. 1a). Details about the operational procedures can be found in previous studies by the authors12,13. Figure 2b presents the detailed schematic of the filter, offering a visual representation of its components and the filtration process. (a) Scahmetic of of ICAF and, (b) Filter operational procedure. Twenty filters were operated and monitored continuously for 30 days, and regular samples were taken from both the influent (raw groundwater) and effluent (after filtration) to analyze the water quality parameters for evaluating the As, Fe, and Mn removal performances of the filter. A comprehensive analysis of water samples was conducted to assess various water quality parameters in both the influent and effluent from the filtration process. The water quality parameters that were measured during the study included pH, Eh (redox potential), EC (electrical conductivity), Temp (temperature), DO (dissolved oxygen), As (arsenic), Fe (iron), P (phosphorous), Mg( magnesium), Ca (calcium), Mn (Manganese), Silica (Si) and TOC ( total organic carbon). Tables 1 and 2 present the statistical data on the influent and effluent. The field data showed wide variations in groundwater quality, as reflected by high standard deviations in the data set (Table 1). The wide range of groundwater quality indicates the data set’s suitability and reliability for the proposed prediction models. The dataset would provide a robust basis for DLNN models to handle diverse conditions, making the predictions more reliable across different scenarios. As and Fe exceeded the standard limit for WHO drinking water among the measured water quality parameters. As concentrations ranged from 259.4 to 587.4 µg/L, with a high standard deviation of 149.2 µg/L, and Fe concentrations varied between 2.2 and 16.3 mg/L, with a standard deviation of 3.6 mg/L. The contaminant removal (effluent) data in Table 2 also showed a similar variability as the influent data in Table 1. The similarity in variability between the influent data in Table 1 and the effluent data in Table 2 effectively demonstrates the consistency of the IACF groundwater treatment process. Table 1Statistical presentation of the influent water.ParameterUnitAverage (n = 20)Maximum (n = 20)Minimum (n = 20)Standard deviationpH-7.27.56.90.22Eh(mV)75.614140.932.5ECmS/cm1.434.90.660.93Temp℃27.128.225.50.67DOmg/L6.47.135.80.34Asµg/L259.4587.423.7149.2Femg/L9.816.32.23.6Pmg/L1.72.40.0790.65Camg/L140.6192.095.028.3Mgmg/L47.6130.823.527.1Simg/L19.224.211.03.9Mnmg/L0.281.20.060.30TOCmg/L8.917.61.35.2 Statistical presentation of the influent water. Table 2Statistical presentation of the effluent water.ParameterUnitAverage (n = 20)Maximum (n = 20)Minimum (n = 20)Standard deviationpH7.928.177.730.15Eh(mV)319.38361.87292.1721.65ECmS/cm1.244.920.631.06Temp℃38.43174.4727.7737.64DOmg/L5.346.883.901.09Asug/L64.83194.0410.4056.08Femg/L0.070.340.000.08Pmg/L0.070.180.010.05Camg/L74.69162.5543.1728.19Mgmg/L45.11138.1823.5029.95Simg/L16.8521.499.173.64Mnmg/L0.060.140.010.04TOCmg/L6.3811.011.503.93 Statistical presentation of the effluent water. To visualize the correlation between all the variables of the obtained data, we analyzed their Pearson correlation coefficient (PCC), which is well known for identifying linear relationships. Mathematically, the PCC between x and y variables can be determined by: where number of samples, mean of x variable in the dataset, and mean of y variable in the dataset. Equation (1) interprets that means there is no correlation exists; means there is a strong positive correlation exists; and means three is a strong negative correlation exists. Figure 3 demonstrates the PCC map of the obtained dataset that includes 13 influent and four effluent water parameters. PCC-map of the filed data’s influent and effluent water parameters. Developing deep learning neural network (DLNN) models The present work developed four DLNN models to predict the removal performances of IACF for As, Fe, Mn, and P from groundwater. Each of these models has thirteen inputs measured before filtering groundwater, namely, pH, Eh, electrical conductivity (EC), temperature (Temp), dissolved oxygen (DO), and concentration of As, Fe, P, Calcium (Ca), Magnesium (Mg), Silicate (Si), Mn, and total organic Carbon (TOC). We followed an identical approach to develop these 4 DLNN models for individually predicting the As, Fe, P, and Mn concentrations. Figure 4 illustrates the complete DLNN model creation, training, testing, and analysis workflow. Workflow of DLNN models creation, training, testing, and analyzing. Our original data set (ODS) contained 20 experimental data points; however, to train and build effective ML models, a reasonably large dataset was required to cover the input data range uniformly39,40. The model development process was initiated with the data interpolation step that generated 400 unique data points for training and testing every model subsequently. Hence, this work’s total number of interpolated data points (IDP) was 1600 (i.e., 4 × 400). The cubic-spline interpolation (CSI) technique was used for data interpolation, as its effectiveness for interpolating small experimental datasets has been demonstrated in various research domains24,39,41. The CSI generated IDPs by mimicking the data features of the ODS without any duplication between them. The CSI technique is discussed further in the following subsection. After that, the IDP was normalized (n-IDPs) following (2): where x is the input vector, xnorm is the normalized input vector; xmean, and xstd are the mean and standard deviations of x. Then, the n-IDPs were split into 5-folds following the stratified k-fold cross-validation (CV) technique that is effective for developing ML models with a small dataset as the entire data points are utilized in this approach to train the models in k-iterations39,42. To stratify the process, we divided the n-IDPs into 15 subgroups based on the output concentration distribution of the model. We ensured that each k-fold contains n-IDPs evenly from those 15 subgroups. After that, the model hyperparameters, such as neural network (NN) architectures, activation functions, optimizers, learning rates, loss functions, dropout rates, and training epochs, are tuned by the Bayesian optimization algorithm, which is a probability-based approach to find the global optimum in the shortest time43. Table 3 summarizes the tuned hyperparameters of this work’s DLNN models. Table 3Hyperprameters for the DLNN models after the Bayesian optimization.HyperparametersOptimized Optionhidden layers4DLNN architectureFor As modeling: 13-(128–104-40–40)-1For Fe modeling: 13-(24–40-24–16)-1For P modeling: 13-(104–110-128–128)-1For Mn modeling: 13-(80–104-104–24)-1kernel initializerGlorotUniformactivation functionTanh (for the hidden layers)Linear (for the output layer)model optimizerAdamlearning rate10− 2loss functionMeans Square Error (MSE)epochs500–800dropout rateFor As, Mn and P modeling: ≥ 5%For Fe modeling: 10%-15% Hyperprameters for the DLNN models after the Bayesian optimization. For As modeling: 13-(128–104-40–40)-1 For Fe modeling: 13-(24–40-24–16)-1 For P modeling: 13-(104–110-128–128)-1 For Mn modeling: 13-(80–104-104–24)-1 Tanh (for the hidden layers) Linear (for the output layer) For As, Mn and P modeling: ≥ 5% For Fe modeling: 10%-15% As shown in Table 3, the Bayesian optimization provided different DLNN architectures for each model. All the models have 4 hidden layers with 13 neurons in the input layer and 1 neuron in the output layer. However, the number of neurons in hidden layer#1 (n1), hidden layer#2 (n2), hidden layer#3 (n3), and hidden layer#4 (n4) for each model are different, as demonstrated in Table 3. Figure 5 visually shows the 4-hidden-layer DLNN architecture 13-(n1-n2-n3-n4)-1 to build the four models for predicting As, Fe, P, and Mn concentration in this work. Illustration of the 4-hidden-layer neuron architecture 13-(n1-n2-n3-n4)-1 used to build the DLNN models for predicting As, Fe, P, and Mn concentration. Subsequently, each DLNN model was trained in 5-iterations where the MSE was the loss function. In every iteration, unique 4-folds of n-IDPs trained the model, and the remaining onefold was kept unseen to the model to be later used for testing purposes. The early-stoping (ES) and dropout regularization techniques were applied to avoid the overfitting issue, which is typical for training models with smaller datasets40,44,45. The best-trained models were chosen for the testing phases by comparing the performance matrices of each of the trained models. In the testing phase, the performances of the best-trained models were evaluated using two types of unseen datasets: the ODS and the unused onefold n-IDPs. Cubic-spline interpolation (CSI) CSI is a data interpolation technique that builds a piecewise-cubic polynomial function g(x) with (n-1) pieces by imitating a real-value function f(x) whose n points are known46,47. For each interval of [xi, xi+1], i = 0,1,2,… n, it constructs a cubic polynomial as shown in (3): All the polynomial coefficients ci_j, j = 0,…,3; i = 0, …,n are adjusted so that every Pi(x) satifies this: Thus, in the end, we will get g(xi) = f(xi) on the [x0, xn] intervale. To achieve a smooth and natural curve, CSI imposed additional conditions during the construction of g(x): At the joining point of two polynomials, the adjacent arches will have the same slope and curvature, i.e., their 1st and 2nd derivatives will be the same. Mathematically, these can be expressed as:56 At the joining point of two polynomials, the adjacent arches will have the same slope and curvature, i.e., their 1st and 2nd derivatives will be the same. Mathematically, these can be expressed as:56 The curvature will be zero at the two boundaries (i.e., x0 and xn). Mathematically, it can be expressed as:7 The curvature will be zero at the two boundaries (i.e., x0 and xn). Mathematically, it can be expressed as:7 The smother curves, few interpolation errors, and distortion-free boundary regions gave CSI the upper hand compared to other interpolation techniques using higher-order polynomials and spatial interpolation39,46. Performance metrics of the models The prediction performances of the models during the training and testing phases were measured by the Mean Absolute Error (MAE), Mean Square Error (MSE), Coefficient of Determination (R2), and Residual Error (RE). Their mathematical definitions are described in (8)-(11): where, is the i’th actual outcome, is the i’th predicted outcome, is the mean of all actual outcomes, and N is the total number of data points. Sensitivity analysis of the models Sensitivity analysis is a statistical approach to understanding how the uncertainty of a model’s output can be correlated with the uncertainty of its input parameters48. Thus, it identifies the most impactful parameters on the model output49. This work used Sobol’s method, which is well exercised in different research domains to analyze global sensitivities49–51. In this method, the impact of an input parameter is determined by the ratio of output variance caused individually or mutually by it and the model’s total output variances caused by all the input parameters. We calculated three types of Sobol indices, namely 1st order index (S1st), 2nd order index (S2nd), and total order index (ST), to analyze the sensitivity of each model. For a model whose output is Y and has X1, X2, X3,…, XN input parameters, mathematically, S1st for the input Xi can be defined as48: where Var(Y) is the variance of the output when all input parameters are changing, and Var[E(Y|Xi)] is the conditioned variance of the expected output due to Xi only. From Eq. (12), it is understood that S1st_i will be ~ 1 if Xi has a very high impact on Y, and S1st_i will be ~ 0 if Xi has almost no impact on Y. As S1st does not provide precise sensitivity information by ignoring all the mutual interactions among Xi and the rest of the input parameters, calculating the higher-order Sobol index is essential. S2nd considers the mutual interaction between a pair in input parameters. Mathematically, S2nd for the input Xi and Xj can be defined as48: where Var[E(Y|Xi, Xj)] is the conditioned variance of the expected output due to the interaction between Xi and Xj, similarly, as demonstrated in (13), we can calculate up to the Sobol Nth order index (SNth_ijk…N) that will provide the sensitivity information for the mutual interactions between all the N input parameters of the model. The total order index ST is the summation of all the sensitivity indices from S1st to SNth where a specific input parameter is involved; thus, it provides the most comprehensive sensitivity information for that input parameter of the model. Mathematically, ST for the input Xi can be defined as48: Modeling platforms and libraries The present work developed, trained, and tested all the DLNN models using Python 3.12 programming language and TensorFlow 2.13-based Keras APIs on the cloud-based Google Collaboratory (Colab) platform backed by the Jupyter notebook service52,53. An open-source library named Regressio performed the CSI for this research54, and SALib conducted the Sobol sensitivity analysis55. Results and discussion Comparing the ODS and IDPs. Figure 6 shows the distribution of As, Fe, P, and Mn concentrations in the filtered groundwater (effluent) for the ODS and IDPs. All the distributions are uneven and discontinuous for the ODS, with only 20 data points. In contrast, the distributions for IDPs are uniform with more data points, which is preferable for building effective ML models39,40. Table 4 indicates that the minimum, maximum, and standard deviation of As, Fe, P, and Mn concentrations are identical for ODS and IDPs, with some minor differences in mean values due to the distinct data distributions of IDPs compared to ODS in Fig. 6. The primary goal of IDPs is to provide sufficient data points with similar data features as the ODPs without duplication so that the DLNN models extract those features during the training phases and can predict the As, Fe, P, and Mn concentrations from the unknown field data. The IDPs enable the DLNN models to learn and predict As, Fe, P, and Mn concentrations in unknown field data. Concentration distribution of As, Fe, P, and Mn (after groundwater filtering) in ODS and IDPs. Table 4Statistical properties of post-filtration As, Fe, P, and Mn concentration in ODS and IDPs.Statistical propertiesODSIDPsAs (μg/L)Fe (mg/L)P (mg/L)Mn (mg/L)As (μg/L)Fe (mg/L)P (mg/L)Mn (mg/L)Min10.400.000.010.0110.400.000.010.01Max194.040.340.180.14193.580.340.180.14Mean64.830.070.070.06101.990.170.090.07Std. Dev54.660.080.050.0353.010.100.050.04 Statistical properties of post-filtration As, Fe, P, and Mn concentration in ODS and IDPs. Analyzing the prediction performances After selecting the best models, their training performance was evaluated using known IDPs (i.e., the IDPs used to train the models), and testing performance was assessed using unknown IDPs, as well as ODS (i.e., the IDPs and ODS not used to train the models). Table 5 provides a summary of these predictions, presenting the performance metrics of the models. Table 5Training and testing performances of the DLNN models using IDPs and ODS.Model#Concentration predicted forTraining performances (using known IDPs)Testing performances(using unknown IDPs)(using unknown ODS)MAEMSER2MAEMSER2MAEMSER21As0.0200.0010.9990.0230.0010.9990.0850.0100.9902Fe0.1390.0250.9750.1360.0240.9760.2950.1490.7753P0.1680.0540.9470.1770.0660.9340.1570.0490.9544Mn0.0410.0020.9980.0420.0020.9980.2310.1050.878 Training and testing performances of the DLNN models using IDPs and ODS. The model performances focus on the Mean Absolute Error (MAE), Mean Squared Error (MSE), and R2 scores for different scenarios. Columns 3–8 show the model performances when the IDPs were used as input. It can be observed that, for all the models, the MAE, MSE, and R2 are identical regardless of whether IDPs are known or unknown. This consistency between the training–testing performances suggests that the models did not overfit during training, likely due to the application of Early Stopping (ES) and dropout regularizations. Reflecting the actual prediction capability of the models, columns 9–11 show the model performances for the ODS, which is crucial as the ODS are collected from the field measurements of groundwater and are entirely unknown to the model. For As and P predicting models, the MAE, MSE, and R2 scores are higher and very close for IDPs and ODS. However, using ODS for Fe and Mn prediction models results in slightly lower scores. This discrepancy is attributed to the assumption that the 13 groundwater parameters used as input for these models may not be the primary influencers of the physical–chemical reactions determining Fe and Mn concentrations in the effluent Water. Figure 6 illustrates the performances of these four modes for predicting the unknown ODS and IDPs 6. Figure 7 (a,c,e,g) respectively shows the measured vs. predicted effluent concentration of As, Fe, P, and Mn by the four best models developed in this work. The green dashed line in each subfigure of Fig. 7 represents the perfect prediction, where the predicted concentration matches the measured concentration. Generally, the IDPs align better with the green line than the ODS. This alignment results in higher R2 scores for the IDPs, indicating better predictive accuracy. As we trained our models using only the IDPs, during the testing phase, all the models showed better accuracy for the IDPs even though the k-fold CV techniques ensured no overlapping between the training and testing IDPs. Figure 7 a-e also illustrates that the ODS points for As and P predictions are closely allied with the green line compared to the ODS points in Fig. 7 (c,g) for Fe and Mn predictions. This observation aligns with the data in Table 5, where the Fe and Mn predicting models using ODS showed slightly lower scores than those predicting As and P. Performance analysis of the 4 DLNN models using unknown ODS and IDPs: (a, c, e, g) measured vs. predicted As, Fe, P, and Mn concentration in effluent water and (b, d, f, h) their RE analysis. Further analysis is conducted on each model to visualize the RE and its distribution for the unknown IDPs and ODS. Figure 7 (b, d, f, h) illustrates the RE and its distribution for the models. RE represents the difference between predicted and measured concentrations relative to the measured concentration. A notable finding is that many predictions fall within a narrow RE range. It is observed that 80–100% perditions for ODS and 93.8–100% perditions for IDPs have RE ≤ ± 0.5. The right-hand side axis of Fig. 7b,d,f,h also confirms that all the distribution of RE follows Gaussian distribution where the peak is near the RE = 0 line, indicating a tendency for predictions to be centered around the true values. In general, the distribution width of IDPs is narrower than that of ODS for all the models. This observation explains the higher R2 scores for IDPs, suggesting that the predictions are more tightly clustered around the true values than ODS. Analyzing the removal sensitivities. After developing the 4 DLNN models and confirming their reliability in prediction, Sobol S1st, S2nd, and ST sensitivity analysis was performed. The study aims to understand each input parameter’s direct, indirect, and total impact on As, Fe, P, and Mn concentrations in effluent. The AI-driven analysis was intended to provide insights into physic-chemical reactions and their influences on groundwater, potentially aiding in designing more effective filters for removing As, Fe, P, and Mn from the groundwater. Figure 8a, c, e, g demonstrates the S1st and ST scores of all 13 input parameters for the filtration process. Figure 8b, d, f, h illustrates the S2nd scores of the ten most interactive input pairs and 95% confidence intervals for As, Fe, P, and Mn filtration. Table 6 consolidates the analysis results presented in Fig. 8 and lists the three most sensitive parameters for each filtration process along with their S1st, S2nd, and ST scores. (a, c, e, g) S1st and ST analysis of the 13 input parameters and (b, d, f, h) S2nd scores of the 10 most interactive input parameter pairs in the As, Fe, P, and Mn filtration process. Table 6The three most sensitive parameters identified by Sobol S1st, ST, and S2nd analysis in the As, Fe, P, and Mn filtration process.For the filtration process ofS1st analysisST analysisS2nd analysis3 most sensitive parametersScore3 most sensitive parametersScore3 most sensitive parameter pairsScoreAsAs0.244As0.563As-Fe0.047Fe0.102Fe0.310As–Si0.045Eh0.046Temp0.276As-Ca0.036FeP0.218P0.441Eh-Si0.023As0.146Mn0.294Eh-EC0.019EC0.105As0.285Eh-Mg0.017PEh0.364Eh0.712Eh-Fe0.051Fe0.051Temp0.322Eh-Mn0.050As0.039As0.230Eh-Ca0.046MnTemp0.241Temp0.371EC-Temp0.027Mn0.183Mn0.258As-Ca0.026EC0.053EC0.180As-Fe0.024 The three most sensitive parameters identified by Sobol S1st, ST, and S2nd analysis in the As, Fe, P, and Mn filtration process. Generally, the S1st top scores align with the ST top scores (Fig. 8 a,c,e,g, and Table 6), indicating that the parameters with the highest direct impact contribute significantly to the total impact. The exceptions are the impact of temperature in the As and P filtration process and the role of Mn in the Fe filtration process. In the above cases, even when Temperature and Mn show no significant S1st scores, their high interaction with other parameters contributes to notable ST scores. It is also comprehended that As and Eh dominate the As and P filtration process, as indicated by their ST scores exceeding 0.5 (ST > 0.5). Figure 8 b,d,f,h, and Table 6 display that i) As interaction with Fe, Si, Ca, and Mn is critical for the As filtration, ii) Eh interaction with Fe, Mn, Ca, and Temp is crucial to the P filtration, iii) Eh interaction with Si, Ec, Mg, and P is critical for the Fe filtration; and iv) both Temperature interaction with Ec and Ca and As interaction with Ca and Fe are essential for the Mn filtration, as they achieved higher S2nd score compared to other pairs. In summary, the analysis provides a nuanced understanding of the interactions and impacts of input parameters on the filtration processes for As, Fe, P, and Mn. Identifying critical interactions can guide researchers in focusing on specific parameters and their combinations for more effective water filtration strategies. Discussion This study developed 4 DLNN models for predicting As, Fe, P, and Mn removal by IACF and analyzed their sensitivities. All the models are trained in fivefold CV to identify and pick the best-performing model that extracts the data features more accurately. To avoid overfitting, ES and Dropout regulations are applied. Table 5 shows that all four models performed well evenly for the known and unknown datasets, thus confirming no overfittings. The tuned hyperparameters (Table 3) obtained by the Bayesian optimization algorithm are believed to be crucial in attaining good precision for the models. The As prediction model achieved the highest accuracy (R2 ≥ 0.99) in the validation step, which may provide a deceptive overfitting indication. Generally, an ML model can be defined as overfitted when it demonstrates very high precision in predicting the known data (e.g., the data already fed to the model for training) and considerably lower precision in predicting the unknown data (e.g., the data never fed to the model before)40,42. However, Table 5 (row#1) shows this is not the case for the As predicting model, as it showed high prediction efficiency for the known and two different types of unknown datasets. Thus, it can be affirmed that the As predicting model is not overfitted; instead, it is a generalized model that predicts both the known and unknown data pretty well. Moreover, this high precision of the As model can be justified by Fig. 3, which shows that As (Efflu.) has a strong correlation (i.e., PCC = 0.47–0.91) with most of the input parameters (i.e., 8 out of 13) that resulted in very high R2 for the As model. The RE followed the Gaussian distribution for all the models, where most predictions reside in RE ≤ ± 0.5. The sensitivity analysis (S1, S2, and ST) revealed the most impactful factors for the As, Fe, P, and Mn removal processes. Influent As is the most sensitive parameter for predicting effluent As, which agrees with the previous studies for predicting As removal using different filters and ML models56,57. This agreement affirms the credibility of the proposed research results. Understanding and optimizing the As, Fe, P, and Mn removal process in IACFs through experiments is challenging due to the need for extensive time, costs, and human supervision. The proposed work contributes substantially to the relevant research domains to understand the complex multidimensional interactions involved in the As, Fe, P, and Mn removal process using the DLNN models and design the IAFs effectively. The developed models are easy to deploy in the user-friendly interface where the researchers can access the prediction results by entering the input parameters in real time without prior knowledge of running the DLNN models. Conclusion The research developed four DLNN models to predict As, Fe, P, and Mn concentrations in the IACF effluent water. All models were trained by the IDPs generated from field-measured ODSs without duplication by applying the CSI. The high R2 score and low RE achieved in the testing and validation of each model ensured that the predictions were very close to the actual measurements. The As and P predicting models demonstrated better accuracy than the Fe and Mn predicting models. Sobol S1 and ST analysis of the developed models found that As, P, Eh, and Temp play the most vital roles in determining As, Fe, P, and Mn in the effluent, while Sobol S2 analysis of the developed models disclosed that As-Fe and As–Si are the most interacted pairs to regulate As concertation indirectly. Eh-Si and Eh-EC were found to be the most interacted pairs to regulate Fe concertation indirectly, Eh-Fe and Eh-Mn as the most interacted pairs to regulate P concertation indirectly, and EC-Temp and As-Ca as the most interacted pairs to regulate Mn concertation indirectly. The findings of this work will benefit scientists by helping them understand the concealed interactions and factors in the filtration process of As, Fe, P, and Mn in groundwater without the need for extensive experiments. The study will assist in improving the filter’s design to remove As, Fe, P, and Mn from contaminated groundwater. Acknowledgements The researchers would like to thank the Deanship of Graduate Studies and Scientific Research at Qassim University for financial support (QU-APC-2024-9/1) Abbreviations Adaptive network-based fuzzy inference system Arsenic Arsenite Arsenate Calcium Google collaboratory Cubic-spline interpolation Cross-validation Deep learning neural network Dissolved oxygen Electrical conductivity Effluent groundwater Early stopping Iron Iron-amended ceramic filter Interpolated data points Influent ground water Kanchan arsenic filter Mean absolute error Magnesium Machine learning Machine learning algorithms Manganese Mean square error Neurons in hidden layer#1 Neurons in hidden layer#2 Neurons in hidden layer#3 Neurons in hidden layer#4 Neural network Original dataset Phosphorus Pearson correlation coefficient Coefficient of determination Residual error Sobol first order index Sobol 2nd order index Silicate Sobol total order index Temperature Total organic carbon Author contributions Muhammad Ali Martuza: Conceptualization, Methodology, Software, formal analysis, Writing—original draft. Md. Shafiquzzaman: Conceptualization, Methodology, Data curation, Formal analysis, Supervision, Writing—original draft. Husnain Haider: Visualization, Validation, Formal analysis, Writing—review & editing. Amimul Ahsan: Writing—review & editing, Abdelkader T. Ahmed: Writing—review & editing. Data availability The datasets generated during and (or) analyzed during the current study are available from the corresponding author upon reasonable request. Declarations Competing interests The authors declare no competing interests. Ethical approval All authors have read, understood, and have complied as applicable with the statement on "Ethical responsibilities of Authors" as found in the Instructions for Authors. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Data Availability Statement The datasets generated during and (or) analyzed during the current study are available from the corresponding author upon reasonable request.