Adaptive weighted dynamic time warping clustering with a deep codec model for accurate photovoltaic power prediction Authors: Zheng Xin, Guangyu Zheng, Weiran Chi, Jianing Lu, Lili Hao Published: 2026 ABSTRACT AbstractAs distributed photovoltaic (PV) systems increasingly integrate with smart grids, accurate ultra-short-term power forecasting is critical for optimizing grid dispatch and maintaining grid stability. However, the intermittency of solar irradiance and the complex multivariate temporal correlations in PV operational data present notable challenges to conventional forecasting models. Such models frequently fail to characterize nonlinear dynamics and effectively capture multiscale features of PV output. To mitigate these limitations, this study proposes a hybrid framework integrating adaptive feature-weighted dynamic time warping (AFDTW) clustering with an improved temporal convolutional network–long short-term memory (ITCN–LSTM) encoder–decoder. First, the AFDTW clustering algorithm employs an adaptive weighting mechanism based on Markov distance to prioritize high-variance features, enabling the capture of subtle patterns in PV output. A pruning strategy is additionally incorporated to reduce computational complexity. Subsequently, the clustered data are input into the ITCN–LSTM model. The ITCN leverages asymmetric causal and spatial convolutions to complex multivariate temporal correlations, thereby modeling geographic correlations. Meanwhile, the LSTM decoder captures long-term trends via its gated memory units. This architecture effectively bridges the gap between static, history-based models and real-time meteorological dynamics, facilitating reliable predictions under diverse weather conditions. Empirical evaluations using a real dataset from a 1.8-MW PV plant in Australia show that the proposed method outperforms the benchmark models across multiple forecasting horizons (5 min to 4 h). The proposed method significantly improves PV power-forecasting accuracy and supports more effective and reliable utilization of solar energy resources. Supplementary InformationThe online version contains supplementary material available at https://doi.org/10.1038/s41598-026-56363-2.Keywords: Ultra-short-term photovoltaic power prediction, Adaptive feature-weighted dynamic time warping clustering, Improved temporal convolutional network–long short-term memory encoder-decoder, Multivariate temporal feature extraction, Markov distance, Meteorological dynamicsSubject terms: Energy science and technology, Engineering, Mathematics and computing Introduction Amid the global drive toward carbon neutrality, energy systems are shifting to a renewable-dominated paradigm1. Photovoltaic (PV) generation is pivotal in this transition because it offers flexible deployment, cost-effectiveness, environmental sustainability, and high operational efficiency. Consequently, PV technology has attracted broad attention across the energy sector2. According to the latest statistics released by the International Energy Agency (IEA), global photovoltaic (PV) installed capacity exceeded 1.6 TW in 20243, reflecting the sustained and accelerated expansion of renewable energy systems. Distributed PV (DPV) continues to account for a significant share of newly added capacity, particularly in major PV markets such as China. The rapid growth of DPV systems further increases the operational complexity of distribution networks3. However, integrating PV into the grid presents several challenges4. PV output is highly weather-dependent and thus exhibits pronounced randomness, volatility, and uncertainty. These characteristics affect distribution networks in two main ways. First, they reshape the net-load profile (i.e. the load after subtracting PV generation), amplify minute- and hour-scale fluctuations, widen peak-to-valley differences, and complicate supply–demand balancing. Second, mismatches between DPV generation and load demand can cause reverse power flow and elevated node voltages, threatening distribution system safety and stability. In this context, accurate PV generation forecasting is essential. Reliable forecasts enable effective operation of active distribution networks, improve solar utilization, and help address PV grid-integration challenges5. In DPV systems, ultra-short-term power forecasting (typically covering horizons from 5 min to 4 h) is a critical tool for mitigating the impact of output volatility on the power grid. It is also essential for improving the accommodation capability of DPV and enabling the coordinated optimization of DPV and distribution networks. These considerations lay the foundation for the research on prediction methods presented in this paper. These challenges underscore the necessity of advancing ultra-short-term DPV power prediction methods, which forms the core motivation of this study. In recent years, research on photovoltaic power forecasting has deepened in response to the increasing operational complexity of power systems and the growing availability of meteorological and operational data. Numerous studies indicate that existing methods are mainly effective for short-term and ultra-short-term photovoltaic forecasting, especially under relatively stable meteorological conditions. Existing photovoltaic power forecasting methods can be broadly categorized into two groups: physical models and data-driven models6–10. In addition to these two conventional categories, recent studies have increasingly focused on hybrid and ensemble forecasting frameworks11–19. Hybrid models combine physical mechanisms or numerical weather prediction (NWP) information with data-driven architectures to enhance interpretability and robustness under dynamic meteorological conditions. Ensemble learning approaches further improve forecasting stability by integrating multiple base learners or regime-specific sub-models. Moreover, attention-enhanced and transformer-based multivariate temporal models have recently been introduced to better capture long-range dependencies and cross-variable interactions in high-dimensional PV datasets. These emerging paradigms reflect a transition from single-model structures toward more adaptive and multi-component forecasting systems. Physical models use solar geometry, module efficiency, and atmospheric physics to estimate PV output. Although physical models offer strong interpretability, they often struggle to adapt to rapidly changing local environmental conditions, which limits their effectiveness in ultra-short-term distributed PV forecasting. In contrast, data-driven methods, particularly deep-learning-based approaches, have gained increasing attention due to their strong nonlinear mapping capability. For example, deep-learning PV forecasting models combine long short-term memory (LSTM) networks and convolutional neural networks (CNNs) to capture complex temporal dependencies and multivariate interactions. This hybrid structure effectively processes PV time-series data and improves forecast accuracy20. An improved chicken swarm optimization (ICSO) algorithm has also been introduced to enhance convergence performance and optimize model parameters in short-term photovoltaic power forecasting21. Meanwhile, CNN-LSTM hybrids augmented with attention mechanisms have been developed to support hybrid feature selection and model ensembling. They significantly improve forecasting performance with multivariate inputs22–24. Reference25 proposed a multi-step forecasting model that combines similar-day analysis, variational mode decomposition, and DELM. The model improves forecast accuracy under complex meteorological conditions. However, despite these advances, most existing approaches still suffer from three critical limitations in ultra-short-term distributed photovoltaic forecasting. First, multi-scale temporal characteristics—such as intra-hour fluctuations, diurnal cycles, and synoptic weather transitions—are often modeled homogeneously within a unified forecasting framework, without sufficient distinction between scale-specific dynamics. This neglect of scale-specific dynamics leads to inadequate representation of short-term volatility and long-horizon dependency, resulting in cumulative error amplification across forecasting horizons. Second, reliance on a single monolithic model structure restricts adaptability to heterogeneous weather regimes. Photovoltaic output exhibits markedly different statistical properties under clear-sky, cloudy, and rapidly changing meteorological conditions. A unified model trained on mixed-regime data tends to smooth regime-specific features, thereby degrading prediction robustness and generalization capability. Third, multi-scale meteorological variability induces nonstationary disturbances in PV power time series. Without explicit regime identification and feature-adaptive modeling, conventional methods struggle to distinguish structural variation from stochastic noise, leading to unstable performance under dynamic weather conditions. To address these limitations, this study introduces a two-stage forecasting framework. First, the proposed Adaptive Feature-Weighted Dynamic Time Warping (AFDTW) clustering method performs regime-aware data segmentation by dynamically weighting discriminative temporal features, thereby isolating heterogeneous weather patterns. Subsequently, the Improved Temporal Convolutional Network–Long Short-Term Memory (ITCN–LSTM) encoder–decoder jointly extracts multivariate temporal features within each homogeneous cluster, enabling multi-scale dependency modeling and enhancing robustness across multiple forecasting horizons. To mitigate forecasting errors in PV power caused by multi-timescale effects, researchers often classify or cluster meteorological features to identify weather regimes. This strategy helps reveal inherent patterns and the associated PV output characteristics under different meteorological conditions, thereby improving the adaptability of forecasting models to heterogeneous weather scenarios. Data clustering has therefore been widely used in photovoltaic forecasting, and its effectiveness has been confirmed in many empirical studies. Cluster analysis can also help identify abnormal operating states and extreme-weather-related output patterns, providing a useful basis for subsequent forecasting-model optimization. Numerous clustering methods have been applied in this field. Among them, K-means26,27 is one of the most widely used baseline methods because of its simplicity and low computational cost. However, K-means relies on Euclidean distance and partitions samples by minimizing the within-cluster sum of squared errors. Such a distance formulation compares samples point by point at identical indices and implicitly assumes local temporal synchronization. For photovoltaic power series, this assumption is often too restrictive, because PV output under different weather conditions may exhibit peak shifts, delayed fluctuations, and locally misaligned but shape-similar patterns. As a result, Euclidean distance may overestimate the dissimilarity between sequences that share similar generation behavior but differ in local timing. Fuzzy C-means (FCM)28 relaxes the hard partitioning strategy of K-means by introducing fuzzy memberships, allowing one sample to belong to multiple clusters with different degrees. Although this flexibility is beneficial for handling ambiguous transitions between weather conditions, FCM still depends on static distance measures and therefore remains limited in representing the heterogeneous and nonlinear temporal characteristics of PV data. Hierarchical clustering29 can capture transitional weather patterns without predefining the number of clusters, but its high computational complexity, limited refinement flexibility, and sensitivity to outliers reduce its suitability for large-scale photovoltaic sequence analysis.In contrast, Dynamic Time Warping (DTW)-based clustering is more suitable for photovoltaic time-series analysis because it allows nonlinear temporal alignment and can compare sequence similarity at the shape level rather than through rigid point-to-point matching. For example, Ref. 30 integrates DTW with K-means and constructs an adaptive graph via a dynamic adjacency matrix. Ref. 31 developed a hybrid DTW-CNN framework to improve daily photovoltaic forecasting accuracy. However, standard DTW still has important limitations. Although it relaxes the strict synchronization assumption of Euclidean distance, it usually treats different feature components in a relatively uniform manner and requires substantial computational cost during dynamic matching. For heterogeneous photovoltaic data, the discriminative importance of feature components such as amplitude, variation rate, and trend may differ across weather regimes. Therefore, a static and uniformly weighted matching strategy may weaken regime-sensitive information and reduce clustering effectiveness. To address these limitations, this study proposes an Adaptive Feature-Weighted Dynamic Time Warping (AFDTW) clustering method. AFDTW introduces adaptive feature weighting into the dynamic matching process so that the local distance is no longer treated as a uniformly accumulated discrepancy, but as a weighted discrepancy that emphasizes more discriminative feature components during temporal alignment. In addition, pruning strategies are incorporated to reduce unnecessary matching computations and improve clustering efficiency. Through this design, AFDTW provides a more suitable similarity measurement for heterogeneous photovoltaic sequences and offers a stronger basis for regime-aware forecasting. Beyond weather conditions, model architecture, feature-processing logic, and capacity for multivariate temporal dependency modeling critically determine the reliability, stability, and generalization of PV forecasts. A key challenge in PV forecasting is handling high-dimensional data with nonlinear multivariate temporal relationships, multi-scale temporal dependencies, and pronounced spatial heterogeneity. To tackle these complexities, researchers have developed diverse models that jointly model temporal and spatial dependencies. LSTM networks, noted for strong time-series modeling, are widely used in short- and ultra-short-term forecasting. For example, researchers proposed a wavelet-kernel-enhanced LSTM to capture nonlinear temporal patterns in wind-power forecasting. They also introduced a bidirectional LSTM (Bi-LSTM) for time-series modeling32,33. However, these temporal models often ignore spatial features, limiting their applicability to distributed PV forecasting where spatial correlations between adjacent panels significantly impact overall output. By contrast, CNNs excel at spatial feature extraction, particularly when spatial correlations are strong. For example, combining CNNs with Informer leverages spatial features while strengthening long-range temporal modeling. Other studies combine Temporal Convolutional Networks (TCNs) with LSTM and transfer learning to forecast extreme wind events. Additional work integrates grey relational analysis, channel-attention mechanisms, and TCNs. A WT-Bi-GRU-Attention-TCN framework has also been employed to improve wind-power forecasting and help stabilize grid dispatch34–37. Nevertheless, models that emphasize spatial features often struggle to capture temporal dynamics and long-range dependencies, which markedly limits their performance in PV forecasting(e.g. 5-min to 4-h horizons). Geographical heterogeneity and spatial correlation are key factors influencing distributed PV forecasting. A prominent limitation of current models is their inability to fully extract spatial features while simultaneously balancing long- and short-term dependencies. To address these gaps, this study proposes an improved TCN–LSTM encoder-decoder model, denoted as ITCN-LSTM. The model mitigates multivariate temporal coupling and improves the accuracy and robustness of PV forecasting by introducing spatial convolutional layers and jointly extracting multi-scale temporal features and cross-variable interaction features. To leverage existing methods while mitigating their limitations, we propose a two-stage integrated framework. First, we apply the AFDTW clustering algorithm to the preprocessed data, then employ the ITCN–LSTM encoder–decoder model. AFDTW adaptively weights features based on time-series similarity, preserves global structure, suppresses noise, and markedly improves clustering accuracy and model robustness. By integrating a spatial convolutional layer into the TCN, the proposed encoder–decoder effectively captures multivariate temporal and nonstationary relationships between meteorological factors and PV generation. This design strengthens spatial feature learning and enables more accurate prediction of the immediate impacts of weather on PV output. However, despite recent progress in photovoltaic power forecasting, several important limitations remain. First, existing studies often rely on a unified forecasting framework and do not explicitly distinguish heterogeneous weather regimes, which weakens prediction robustness under rapidly changing meteorological conditions. Second, conventional clustering methods usually adopt fixed distance measures or uniform feature weighting, making them less effective for photovoltaic data with strong nonlinear temporal variability and regime-dependent characteristics. Third, many deep-learning forecasting models mainly emphasize temporal dependency modeling, while the joint learning of temporal dynamics and cross-variable interactions remains insufficient. These limitations motivate the present study to develop a regime-aware and cluster-specific forecasting framework for ultra-short-term photovoltaic power prediction. This study makes three main contributions: First, an Adaptive Feature-Weighted Dynamic Time Warping (AFDTW) clustering method is proposed to improve weather-regime identification for heterogeneous photovoltaic data. By introducing adaptive feature weighting into the dynamic time warping framework and incorporating pruning strategies into the matching process, the proposed method improves the representation of regime-dependent temporal patterns while enhancing clustering efficiency. Second, an improved TCN–LSTM encoder-decoder forecasting model, denoted as ITCN-LSTM, is developed for ultra-short-term photovoltaic power prediction. The encoder is designed to capture multi-scale temporal dependencies and cross-variable feature interactions, while the decoder improves sequential prediction stability across multiple forecasting horizons. Third, a unified regime-aware forecasting framework is established by coupling AFDTW-based clustering with cluster-specific deep forecasting. This integrated design enhances the adaptability of the model to heterogeneous meteorological conditions and demonstrates improved forecasting performance on both benchmark and real-world photovoltaic datasets. The remainder of this article is organized as follows. Section "Related work and theoretical foundations" reviews related theory. Section "AFDTW–ITCN–LSTM forecasting framework" details the technical architecture of the proposed framework, including the AFDTW clustering algorithm and the ITCN–LSTM encoder–decoder design. Section "Case study" presents empirical validation on benchmark and operational datasets, including quantitative evaluations and technical analyses. Section "Conclusion" summarizes the main findings, discusses practical implications, and outlines future research directions. Related work and theoretical foundations This section provides a structured review of existing photovoltaic forecasting research and outlines the theoretical foundations relevant to the proposed framework. Specifically, Section "Multivariate temporal characteristics of photovoltaic power and their modeling implications" summarizes the multivariate temporal characteristics of photovoltaic power and discusses their implications for modeling strategies. Section "Temporal convolutional networks" introduces the theoretical background of TCNs, which form the core architectural component of the proposed model. Section "Critical analysis of existing forecasting frameworks" critically analyzes the limitations of existing clustering-based and deep learning approaches, thereby identifying the research gap that motivates the present study. Multivariate temporal characteristics of photovoltaic power and their modeling implications Because PV sites often experience similar meteorological conditions and are geographically proximate, multivariate temporal correlations naturally arise among them. Notably, this Multivariate temporal dependence—rooted in the interplay between meteorological dynamics and PV output fluctuations—must also be explicitly modeled in forecasting frameworks that jointly take meteorological factors and PV generation data as inputs. The effects of meteorological factors on PV output vary across stations, implying spatially varying correlations between meteorological fields and power generation. Driven by atmospheric dynamics, these correlations evolve across time and space. To capture the coupling between meteorological data and PV generation, it is essential to analyze multivariate temporal correlations in large-scale datasets, especially as temporal and spatial coverage expands. Moreover, because meteorological data are sampled at high temporal frequency but are spatially sparse, specialized convolutional structures are needed to extract temporal and spatial features. Temporal convolutions capture long-range dependencies, whereas spatial convolutions model inter-site correlations. These multivariate temporal characteristics are not merely descriptive properties of photovoltaic power systems; rather, they directly impose structural requirements on forecasting model design. Specifically, effective PV forecasting models must (1) capture multi-scale temporal dependencies, (2) represent cross-variable interactions among meteorological inputs, and (3) maintain robustness under regime-dependent statistical variation. These modeling requirements motivate the selection of architectures capable of jointly extracting temporal and cross-variable features, which provides the theoretical foundation for introducing TCN in the following subsection. Temporal convolutional networks TCNs are deep-learning architectures for sequence modeling. They use causal and dilated convolutions to model long sequences, capturing long-range dependencies while preserving temporal order. Basic principles TCN models sequential data via multiple convolutional layers. Each layer's output feeds the next, extracting temporal features. Unlike RNNs, TCN uses convolutions, avoiding gradient issues in long sequences. Causal Convolution: It ensures operations rely only on current and past steps. For input , output is with K is the kernel size, and are the convolutional kernel parameters. Dilated convolution: It inserts gaps into the kernel to expand the receptive field. The output is defined as , where d is the dilation rate. Derivation of TCN formulas 1. Multi-layer Causal Convolution: TCN processes data through layers. The l-th layer output is . 2. Residual Connection: To ease gradient problems, TCN has residual connections: Residual blocks help avoid gradient issues. 3. Final Output: with L layers, the final output is . While the above formulation outlines the theoretical principles of the baseline TCN architecture, its direct application to photovoltaic forecasting remains insufficient for fully capturing regime-dependent heterogeneity and cross-variable interactions. In practical PV forecasting scenarios, additional architectural adaptation and integration with regime-aware mechanisms are required. This consideration motivates the improved ITCN–LSTM encoder–decoder framework proposed in Sect. 3, which extends the standard TCN by incorporating spatial convolutional components and hierarchical sequence modeling to better address the multivariate temporal complexities discussed in Section "Multivariate temporal characteristics of photovoltaic power and their modeling implications". Critical analysis of existing forecasting frameworks Despite substantial progress in PV power forecasting, several structural limitations remain in existing research. First, clustering-based methods—such as K-means, FCM, and DTW-based approaches—primarily rely on fixed distance metrics and uniform feature weighting strategies. These assumptions limit their ability to adapt to heterogeneous PV datasets characterized by nonlinear temporal dynamics and regime-dependent statistical properties. In particular, conventional DTW methods exhibit high computational complexity and sensitivity to initialization, which constrains their real-time applicability in ultra-short-term forecasting scenarios. Second, multivariate temporal deep learning architectures, including CNN–LSTM hybrids and TCN-based models, have demonstrated strong nonlinear representation capability. However, most studies either emphasize temporal dependency modeling without explicit regime segmentation, or perform clustering and forecasting as loosely connected stages. This separation often prevents models from fully leveraging regime-specific feature distributions, resulting in degraded robustness under transitional or extreme weather conditions. Third, existing frameworks frequently adopt single-model training strategies across mixed meteorological regimes. Such unified modeling approaches may smooth heterogeneous statistical patterns, leading to error amplification across multiple prediction horizons. These observations highlight the necessity of a tightly integrated framework that (1) performs adaptive, feature-aware clustering to explicitly identify regime heterogeneity, and (2) couples regime-specific segmentation with hierarchical multivariate temporal deep learning architectures38,39. Motivated by this gap, the present study proposes an AFDTW–ITCN–LSTM integrated forecasting framework that unifies adaptive clustering and encoder–decoder modeling within a coherent structure. AFDTW–ITCN–LSTM forecasting framework To enhance prediction accuracy and computational efficiency, this study proposes an ultra-short-term photovoltaic power generation forecasting framework that integrates AFDTW clustering with a multivariate temporal encoder-decoder architecture. The framework comprises four sequential stages, each designed to address specific technical challenges and fulfill a distinct functional objective: Phase one (data preprocessing): To mitigate the impact of sensor malfunctions, extreme weather anomalies, and inconsistent data dimensions—which may distort feature learning in downstream models—we implement anomaly detection, impute missing values, and standardize the input data to ensure high reliability and consistency in data quality. Phase two (AFDTW clustering): To effectively identify latent temporal patterns from historical photovoltaic data—such as power output characteristics under varying weather conditions—and to provide a high-quality reference for real-time pattern matching, we employ the Adaptive Feature-Weighted DTW algorithm to preprocess historical data and generate representative clusters for subsequent real-time alignment. Phase three (adaptive weather matching): To overcome the limitations of static clustering in capturing dynamic real-time weather variations and to improve the similarity matching between real-time and historical patterns, we introduce an adaptive weather matching mechanism. Phase four (ITCN-LSTM encoder-decoder prediction): To comprehensively capture the multivariate temporal dependencies inherent in photovoltaic power generation data—such as the geographical correlation of meteorological variables across adjacent stations and the long-term temporal trends in power output—and to meet the requirements of continuous, multi-step-ahead forecasting for ultra-short-term grid dispatching, we utilize the ITCN encoder to extract spatial and temporal features and the LSTM decoder to model temporal dependencies for multi-step prediction. The overall architecture of the proposed framework is illustrated in Fig. 1. Method overall structure diagram. AFDTW clustering Distributed-generation datasets exhibit pronounced nonlinearity, nonstationarity, and periodicity. These properties pose considerable challenges to traditional clustering methods in capturing complex multivariate temporal patterns. To address these challenges, we introduce the AFDTW clustering algorithm. The method integrates the temporal-alignment capability of DTW with adaptive feature weighting to handle multidimensional heterogeneity in distributed energy systems. AFDTW dynamically assigns weights to features to identify latent operational trends in distributed PV systems. This targeted weighting improves pattern-recognition accuracy. Dynamic adjustment of feature weights in AFDTW overcomes the limitations of uniform weighting used in conventional methods. It prioritizes informative features via higher weights while down-weighting noise and irrelevant features, ensuring that clusters more closely align with the physical characteristics of power-generation processes. The approach efficiently captures dynamic changes in complex, multidimensional datasets, improving both clustering accuracy and robustness. Furthermore, adaptive weight updates enhance interpretability and predictive accuracy, thereby distinguishing AFDTW from traditional clustering approaches. Compared with conventional methods, AFDTW achieves superior performance on complex time-series data and maintains stable clustering accuracy under the high volatility and randomness inherent to distributed-generation data. AFDTW reduces model complexity and delivers accurate, reliable clustering outputs, providing a solid foundation for high-precision PV power forecasting models. In photovoltaic (PV) systems, data preprocessing begins with anomaly detection using Markov-distance filtering to identify outliers arising from sensor malfunctions or extreme weather. Next, we apply correlation-aware temporal interpolation, leveraging temporal correlations across adjacent time steps and the geographic proximity of PV stations to impute missing values effectively. We then scale the multidimensional inputs to [0, 1] via min–max normalization and standardize PV output power with z-score normalization (zero mean, unit variance) to improve optimization stability and convergence in deep learning models. Finally, we apply sliding-window segmentation to construct input–output pairs. Historical meteorological and power data within a fixed window serve as input features, while future power values at multiple horizons serve as targets, enabling multiscale temporal modeling for ultra-short-term forecasting (Fig. 2). Data extraction and preprocessing. First layer AFDTW is applied to cluster the data extracted by the sliding window. Let and denote two multivariate photovoltaic time series, where and represent the observations at time steps i and j, respectively, and each observation contains M feature components. In conventional DTW, the local distance between two time points is usually computed by Euclidean distance, which implicitly assumes that all feature components contribute equally and independently to sequence similarity. However, this assumption is often inadequate for photovoltaic power series. Under different weather regimes, PV sequences may exhibit distinct statistical characteristics in terms of amplitude variation, fluctuation rate, and trend evolution. Therefore, a uniformly accumulated local distance may fail to reflect the actual discriminative importance of different feature components. To address this issue, a feature-weighted local distance is introduced in the first layer of AFDTW. In this formulation, the contribution of each feature to the local matching cost is adjusted by an adaptive weight, so that the sequence discrepancy is no longer treated as a simple uniform accumulation, but as a weighted discrepancy that emphasizes more informative components under heterogeneous meteorological conditions. Accordingly, the weighted Markov distance is defined as: where and represent the k feature values of and at time points i and j, respectively. is the weight of the k feature, satisfying . is the current feature weight vector. It should be noted that the proposed Markov distance can be interpreted as a statistically adaptive extension of weighted Euclidean distance, where the weights are dynamically adjusted to reflect regime-specific state-transition characteristics in photovoltaic power sequences. where and denote the values of the k-th feature in and at time steps i and j, respectively. denotes the weight of the kth feature, satisfying is the current feature-weight vector. It should be noted that the proposed weighted Markov distance can be regarded as an adaptive extension of weighted Euclidean distance in the DTW local matching stage. By dynamically adjusting the contribution of each feature, the distance metric becomes more sensitive to regime-dependent sequence characteristics, which provides a more suitable local similarity measure for heterogeneous photovoltaic data. Then, construct a distance matrix D, with a size of , where n and m represent the lengths of sequences and respectively. Based on the local weighted distance, the cumulative distance matrix is constructed to perform dynamic alignment between the two sequences. Different from rigid point-to-point matching, this process searches for the minimum accumulated matching cost over all feasible alignment paths. Therefore, the resulting warping path reflects the optimal shape-level correspondence between two photovoltaic sequences even when local temporal shifts exist. The initialization of D is as follows: This initialization sets the accumulated distance at the origin to zero and all other boundary positions to infinity, so that the alignment process starts from the initial point of both sequences and invalid matching paths are excluded at the beginning of dynamic programming. In this way, the cumulative distance evaluation is constrained to feasible alignment trajectories, which helps preserve the temporal continuity of the matching process. Then, the cumulative distance matrix D(i,j) is recursively computed as follows: To improve both the efficiency and the regularity of dynamic alignment, a window constraint |i - j| ≤ r is introduced, where r denotes the allowable warping range and is defined as a proportion of the sequence length, i.e. in our experiments. This constraint adopts a Sakoe-Chiba band structure and restricts the search space to the neighborhood of the main diagonal. From a methodological perspective, this design does not merely reduce computational cost; it also prevents implausible long-range alignments and excessive temporal distortion, thereby preserving the local continuity of photovoltaic sequence matching. In addition, a pruning strategy based on early abandoning is incorporated during cumulative distance computation. Specifically, if the partial cumulative distance of a candidate path already exceeds the current best-known distance for the same pair of sequences, the further expansion of that path is terminated immediately. This strategy is defined locally within each pairwise DTW computation rather than globally across all sequence comparisons. Therefore, pruning not only improves computational efficiency, but also reduces redundant exploration of clearly suboptimal paths while preserving the correctness of the optimal alignment result (Fig. 3). FDTW distance matrix diagram. After the cumulative distance matrix has been fully computed, the optimal path distance between and is obtained as Second layer The computed AFDTW distance matrix serves as the distance metric for subsequent FCM clustering. As a well-established clustering algorithm, FCM relaxes the hard-assignment constraint in traditional clustering. It introduces fuzzy memberships, allowing each data point to belong to multiple clusters to varying degrees. This flexibility enhances the algorithm’s ability to reveal latent structures in the data. At its core, FCM estimates each data point’s memberships to all cluster centers by minimizing a weighted objective function based on AFDTW distances. The objective balances distances to cluster centers with membership degrees controlled by a fuzziness parameter, yielding more precise clustering results. Although AFDTW provides an adaptive and regime-sensitive distance metric, a clustering mechanism is still required to partition the dataset into homogeneous weather regimes. In photovoltaic power forecasting, regime boundaries are often not sharply separable due to transitional meteorological conditions (e.g. partially cloudy states). Hard clustering methods such as K-means or centroid-based approaches assign each sample exclusively to a single cluster, which may lead to information loss when regime characteristics overlap. FCM, in contrast, allows soft membership assignment, enabling each sample to belong to multiple clusters with different membership degrees. This property is particularly suitable for PV datasets characterized by gradual regime transitions and mixed meteorological features. By combining AFDTW distance with FCM, the clustering process preserves regime uncertainty and enhances robustness under transitional weather conditions. The fuzzifier parameter m controls the degree of membership fuzziness. A moderate fuzziness level prevents over-fragmentation (as in hard clustering) while avoiding excessive smoothing. In this study, m is selected within the commonly adopted range to balance clustering sharpness and regime continuity. While alternative clustering strategies could also be applied to the AFDTW distance matrix, FCM is selected in this study due to its ability to model regime uncertainty and gradual transitions, which are intrinsic characteristics of photovoltaic power generation. The objective function for FCM is defined as follows: Among them, N denotes the number of data points, C says the number of clusters, the first said i data points, the first said j the center of the cluster, said data points belong to the cluster membership degree, m is blurred coefficient, It is used to control the degree of ambiguity of membership, which is usually greater than 1. Using the Lagrange multiplier method, we get: It shows that a data point's membership to a cluster is determined by distance ratios; closer distance means higher membership. By deriving with respect to and setting the derivative to zero: The new cluster center is a weighted average of data points, with weights from membership fuzziness. The steps of the FCM algorithm are as follows: Step 1. Initialization: Randomly select initial cluster centers or initialize the membership matrix. Step 2. Membership Calculation: Use current cluster centers to calculate data points' memberships. Step 3. Update Cluster Centers: Use updated memberships to calculate new centers. Step 4. Convergence Check: Stop if changes in cluster centers or are below a preset threshold; else, return to Step 2. Clustering techniques have been widely applied in photovoltaic forecasting to identify representative weather regimes and reduce intra-class variance. Classical partition-based methods such as K-means27 and FCM28 are commonly used for regime segmentation. In addition, DTW-based clustering approaches have been introduced to better align nonlinear temporal patterns in PV time series29. In this paper, the AFDTW distance matrix is implemented using FCM clustering. Our implementation adopts standardized parameters: fuzzy index = 2, maximum number of iterations = 1000, error threshold = 0.005, and number of clusters = 4. As shown in Table 1, the SSE decreases substantially when the number of clusters increases from 1 to 4, indicating significant improvement in intra-cluster compactness and structural homogeneity. However, when k > 4, the marginal reduction in SSE drops sharply, and the relative reduction falls below 10%, revealing diminishing returns in further increasing cluster granularity. This behavior clearly indicates the presence of an elbow point at k = 4. Table 1SSE and relative reduction analysis for different cluster numbers.kSSEΔSSERelative reduction (%)21126.37–717.1538.9%3746.91–379.4633.7%4512.84–234.0731.3%5476.33–36.517.1%6451.72–24.615.2% SSE and relative reduction analysis for different cluster numbers. From a forecasting perspective, the number of clusters directly affects regime specialization and data distribution. If too few clusters are used, heterogeneous weather regimes may be grouped together, increasing intra-cluster variance and limiting the model’s ability to learn regime-specific characteristics. Conversely, an excessive number of clusters reduces the sample size within each cluster, potentially leading to data sparsity, reduced statistical stability, and increased risk of overfitting. Therefore, selecting k = 4 achieves a balance between regime granularity and data sufficiency, ensuring stable and reliable forecasting performance. Adaptive adjustment of feature weights After the initial rounds of Layer 1 and Layer 2 clustering, the adaptive adjustment of feature weights becomes necessary. In photovoltaic sequence clustering, the discriminative importance of different feature components is not fixed. Under different weather regimes, some features may contribute more strongly to regime separation, whereas others may provide limited information or even introduce noise into the similarity evaluation. Therefore, using a static weight setting throughout the entire clustering process may lead to a mismatch between the distance metric and the intrinsic heterogeneity of the data. To address this issue, AFDTW updates the feature weights adaptively during clustering so that the similarity measure evolves together with the clustering structure. In this way, features with stronger regime-discrimination ability gradually receive higher weights, while less informative components are relatively suppressed. This adaptive mechanism enables the proposed method to better capture regime-dependent photovoltaic sequence patterns than conventional clustering approaches based on static similarity definitions. The specific steps are as follows: First, after each round of clustering is completed, the feature variance within each cluster is calculated. For each cluster , the variance of the feature k is defined as where denotes the number of samples in cluster , denotes the value of the kth feature of the ith sample, and denotes the mean value of the kth feature within cluster . From a statistical perspective, the variance of a feature reflects its degree of dispersion within the current clustering structure. In the present study, this quantity is used as an indicator of feature sensitivity to regime-dependent sequence variation. Features with stronger variability are more likely to reflect meaningful differences among heterogeneous photovoltaic patterns, whereas features with persistently weak variation may provide limited discriminative information for clustering. Therefore, the feature-variance term is introduced into the weight-update process so that the distance metric can adapt to the statistical characteristics of the current clustering state. Through this design, the feature weights are no longer manually fixed in advance, but are iteratively adjusted according to measurable statistical variability. As a result, the similarity measure progressively emphasizes more informative feature components during clustering, which improves the representation of regime-dependent sequence differences in heterogeneous photovoltaic data. The feature weights are updated according to the variance-derived relevance of each feature. If the variance of a feature is relatively small, this suggests that the feature contributes less to distinguishing the current clustering patterns, and its corresponding weight should be reduced. Conversely, if a feature exhibits stronger statistical variability, it is considered to contain more regime-sensitive information and is therefore assigned a relatively higher weight. In this way, the adaptive weighting process progressively aligns the distance metric with the intrinsic heterogeneity of the photovoltaic data. Conversely, if a feature’s variance is large, it implies the feature has greater discriminative power, and its weight should increase. The weight update formula is where ϵ epsilon ϵ is a small constant used to prevent division by zero. In order to ensure that the sum of all feature weights is 1, the weights need to be normalized: After this process, the new feature weights are applied to calculate distances in the next round of clustering, continuing until the results converge or a preset number of iterations is reached. In this paper, the number of iterations is set to 3. Data matching mechanism The data-matching mechanism aligns the current dataset with historical patterns and addresses data-quality issues, nonlinear temporal dependencies, and dynamic pattern alignment. It improves prediction accuracy by matching real-time data to the most relevant historical clusters. Specifically, we focus on accurate PV power forecasting in heterogeneous, dynamic environments, a task that remains challenging for existing models. First, the current data undergo comprehensive preprocessing, including anomaly detection and outlier removal. We apply Markov-distance filtering to identify and remove outliers caused by sensor failures or extreme weather. We then normalize all input variables to ensure consistency and prepare the data for subsequent analysis. Next, we apply the Adaptive Feature-Weighted Dynamic Time Warping (AFDTW) clustering algorithm to the preprocessed data. The method uses an adaptive feature-weighting strategy to address the inherent heterogeneity of distributed energy systems. AFDTW adjusts feature weights based on temporal and spatial relevance to identify appropriate reference clusters. This captures multivariate temporal patterns crucial for PV power forecasting. To determine the closest historical pattern, we compute Euclidean distances between the current-data feature vector and each historical-cluster centroid. These distances quantify similarity between current and historical data and guide selection of the most relevant cluster. Using this metric, we match the current dataset to the most similar historical cluster to improve predictive performance. After identifying the nearest historical cluster, we activate the corresponding submodel, pretrained on that cluster’s homogeneous dataset. This enables real-time adaptation by leveraging the most relevant historical patterns to improve predictive accuracy. Each submodel is fine-tuned to capture specific weather conditions or operational trends, thereby increasing flexibility in dynamic, real-world scenarios. Adaptive feature weighting and Markov-distance filtering make the data-matching mechanism robust, enabling it to handle complex, multidimensional datasets while maintaining accuracy across varying conditions. By aligning current data with the most appropriate historical clusters, the mechanism yields more accurate predictions and improves the overall performance of PV power-generation models. In this article, the four clusters in the historical database are , and the center of each cluster is represented by vectors . The center of each historical cluster is given by where is the number of data points in cluster , and x represents each data point within that cluster. The center of the current cluster in the matching stage is represented as where denotes the number of samples belonging to the current cluster , and the summation is restricted to members of that cluster rather than the entire dataset. x represents each data point within that cluster. The Euclidean distance between each current cluster and each historical cluster is computed as follows: where and represent the k dimension of the i current cluster and the j historical cluster, respectively, and n is the dimensionality of the data. Therefore, using this formula, we can calculate the distance between the center of each dataset on that day and the historical clusters, and match them to improve the prediction accuracy (Fig. 4). Automatic weather data matching mechanism. Power forecasting After AFDTW clustering, photovoltaic power forecasting is performed separately within each cluster. Compared with a single global forecasting model, this cluster-specific strategy is more suitable for heterogeneous photovoltaic data because the input–output relationships become more homogeneous within each weather regime. However, accurate ultra-short-term forecasting remains challenging, since photovoltaic sequences are jointly affected by multi-scale temporal dependencies and cross-variable meteorological interactions. Conventional forecasting models often have complementary limitations. Recurrent models such as LSTM are effective in modeling sequential dependence, but their ability to extract local and multi-scale temporal patterns is relatively limited. In contrast, convolution-based models are more effective in capturing local structural features, yet they may be less flexible in maintaining stable sequential decoding for multi-horizon forecasting. Therefore, a hybrid encoder-decoder architecture is adopted in this study to combine the representation-learning advantage of temporal convolution with the sequential prediction capability of gated recurrent decoding. To this end, an improved Temporal Convolutional Network (TCN)-Long Short-Term Memory (LSTM) encoder-decoder model, denoted as ITCN-LSTM, is constructed for ultra-short-term photovoltaic power prediction. In this architecture, the ITCN encoder is responsible for extracting multi-scale temporal features and cross-variable interaction patterns, while the LSTM decoder is used to maintain sequential continuity and improve forecasting stability across multiple prediction horizons. ITCN-LSTM codec construction Photovoltaic power forecasting is essentially a multivariate time-series prediction problem, in which PV output is jointly influenced by its historical generation pattern and multiple meteorological variables. Therefore, an effective forecasting model should simultaneously consider temporal autocorrelation in the power sequence and cross-variable interactions among meteorological and power-related features.Conventional Temporal Convolutional Networks (TCNs) are effective in capturing temporal dependencies through causal and dilated convolutions, but they mainly focus on temporal modeling along the time axis. For heterogeneous photovoltaic forecasting, this is still insufficient, because the interactions among irradiance, temperature, humidity, wind speed, and PV output also carry important predictive information. To address this issue, the baseline TCN is extended in this study by introducing an additional variable-interaction modeling layer, so that the encoder can jointly learn temporal evolution patterns and cross-variable dependency structures from multivariate photovoltaic sequences.Specifically, the one-dimensional time-series input is reshaped into a matrix in which rows denote time steps and columns denote feature variables. This representation enables the encoder to perform temporal convolution along the time dimension while capturing interaction patterns among variables in the feature dimension, thereby improving the representation ability of the encoder for multivariate photovoltaic forecasting. An additional variable-interaction convolution layer is introduced to model dependency patterns among feature variables and to enhance the encoder’s representation of multivariate photovoltaic sequences. Different from temporal convolution, which operates along the time axis to capture sequential evolution, this layer focuses on interactions among meteorological variables and photovoltaic output in the feature dimension. In this way, the encoder can simultaneously learn how each variable evolves over time and how multiple variables jointly influence the forecasting target. The TCN extracts temporal features through two key mechanisms. First, causal convolution preserves temporal order and prevents future information leakage, which is essential for photovoltaic forecasting. Second, dilated convolution enlarges the receptive field, enabling the encoder to capture both short-term fluctuations and long-range temporal dependencies without excessively increasing network depth. When applied to the reconstructed two-dimensional representation (rows = time steps, columns = feature variables), temporal convolution acts mainly along the time dimension and preserves per-feature temporal causality. To better match the heterogeneous structure of multivariate photovoltaic data, asymmetric convolution is adopted in the encoder design. Specifically, the temporal convolution kernel is used to model temporal evolution over successive time steps, whereas the variable-interaction convolution kernel is used to capture dependencies across feature dimensions. This decomposition allows temporal dependency modeling and cross-variable interaction modeling to be handled in a more targeted manner, rather than forcing both types of information into a single uniform convolution operation. As a result, the encoder provides a more informative feature representation for subsequent LSTM decoding and multi-horizon forecasting. Therefore, different convolution structures can be chosen to extract temporal and spatial features. Asymmetric convolution refers to decomposing a standard (d*d) convolution kernel into a (d*1) kernel and a 1*d kernel in series. This method effectively reduces the number of parameters in convolution operations while maintaining the model's representation capability and performance. More importantly, different convolution structures can be used to extract temporal and spatial features based on the differences in time and space dimensions of the data. With asymmetric convolution, a convolution kernel of size (n, m) is decomposed into (n, 1) and (1, m) in series, allowing for the extraction of temporal and spatial features separately. The spatial convolution between columns of two-dimensional data is represented by Eq. (14): where is the result of convolution, is the time feature obtained by dilatation causal convolution, represents the weight of the convolution process, and represents the bias. TCN integrates spatial convolutional layers following the expansion of causal convolution to extract spatial features. Extended causal convolution captures temporal variations in PV data, identifying trend evolution and periodic patterns in historical sequences. Spatial convolution models geospatial factors (e.g. distance between stations and terrain changes) to comprehensively evaluate the impact on power generation. Building upon the baseline TCN architecture, we propose an ITCN encoder. The ITCN encoder transforms photovoltaic power generation time series into multivariate temporal feature vectors via hierarchical convolution operations. These vectors organize the compact multivariate temporal embedding into a two-dimensional matrix, where rows represent time steps and columns correspond to the feature dimension. This matrix acts as a compact multivariate temporal embedding of the original time series. These embeddings are then fed into the LSTM decoder. The LSTM decoder uses historical multivariate temporal patterns for predictions. Its cell core update: where are gates; is the cell state; is the hidden state. In PV forecasting, the LSTM decoder propagates temporal dependencies captured by the ITCN. For instance, with a weekly weather cycle influencing generation, the LSTM cell state memorizes this pattern. The gates regulate information flow to ensure accurate multi-step predictions. The complete ITCN-LSTM architecture combines ITCN-based multivariate temporal feature extraction with LSTM-based long-term dependency modeling. It excels at handling PV power data, capturing complex multivariate temporal correlations, improving prediction accuracy, and providing a reliable basis for grid scheduling and PV management. Compared with traditional models, it demonstrates superior capability in modeling multivariate temporal data, with strong adaptability and generalization. The complete architecture of the ITCN-LSTM is illustrated in Fig. 5 ITCN Encoder. Overall structure Figure 6 illustrates the overall forecasting framework integrating AFDTW clustering with the ITCN-LSTM encoder-decoder. The proposed framework is designed as a regime-aware two-stage forecasting architecture. This design is motivated by the fact that the relationship between meteorological inputs and photovoltaic power output is not globally uniform, but varies across different weather regimes. If all samples are modeled by a single forecasting function, heterogeneous sequence patterns and regime-specific input–output mappings may be mixed together, which can weaken the model’s ability to learn stable local forecasting rules. Overall framework diagram of the proposed method. Accordingly, the framework consists of four stages: data preprocessing, sliding-window segmentation, AFDTW clustering, and deep-learning prediction. Data preprocessing includes outlier removal, missing-value imputation, and normalization. Sliding-window segmentation constructs input–output pairs for multi-step forecasting. AFDTW clustering decomposes the original dataset into several weather-regime-specific subsets with more homogeneous sequence characteristics and assigns daily data to the nearest historical cluster. On this basis, the ITCN-LSTM encoder-decoder performs multi-horizon photovoltaic power prediction through cluster-specific training. In this way, clustering acts as a distribution decomposition step, while forecasting acts as a targeted learning step, thereby improving the adaptability of the overall framework to heterogeneous meteorological conditions. Case study Cluster evaluation index To enhance the ability of data pattern discrimination, this study adopts the AFDTW clustering method and integrates the feature adaptive weights into the traditional dynamic time warping. Different from the traditional DTW, the adaptive version dynamically adjusts the feature weights by using multivariate temporal correlation, which can detect subtle data changes more precisely and improve the clustering accuracy. This method alleviates the misclustering phenomenon caused by feature issues in traditional DTW, realizes feature-based precise classification, and thereby improves the data quality of downstream prediction models. The clustering validity is quantified by the Fuzzy Partition Coefficient (FPC) to verify the performance of AF DTW. FPC evaluates the degree of cohesion within the clusters and the degree of separation between the clusters, with the value range from 0 (poor clustering effect) to 1 (optimal separation effect). This indicator ranges from 0 to 1. The higher the score, the better the clustering quality (see Formula 16). where N is the total number of samples in the dataset. C is the number of clusters. represents the membership degree of the i sample to the j cluster, with values ranging from [0, 1]. is the squared membership degree of the i sample in cluster j. Case study on the benchmark dataset This section validates the proposed ultra-short-term PV power prediction method integrating AFDTW clustering and deep encoder-decoder models through a case study. Experimental data originates from a 1.8 MW distributed PV system located in Central Australia's Uluru region (Yulara Solar System). This case study is designed to evaluate the effectiveness of regime-aware clustering and multi-horizon forecasting performance under realistic operational conditions. Benchmark models including stacked LSTM, bidirectional LSTM (Bi-LSTM), and TCN-LSTM architectures were selected for performance comparison. The selected benchmark models represent three widely adopted paradigms in photovoltaic time-series forecasting. The stacked LSTM model serves as a classical recurrent baseline for capturing long-term temporal dependencies. The Bi-LSTM architecture enhances bidirectional temporal feature extraction, providing a stronger recurrent benchmark. The TCN-LSTM hybrid model represents convolution–recurrent integration, combining temporal convolutional feature extraction with sequence modeling capability.These models were selected to provide a balanced comparison across recurrent, bidirectional recurrent, and convolution-enhanced architectures, which constitute mainstream deep-learning approaches in short-term PV forecasting literature.While recent transformer-based and attention-driven models have demonstrated promising performance in certain time-series tasks, they typically require significantly larger datasets and higher computational resources for stable training. Given the dataset scale and the regime-aware clustering design of the proposed framework, the selected baselines provide a fair and representative comparison for evaluating multi-horizon forecasting performance under realistic deployment conditions. All implementations used Python 3.8.8 with PyTorch 1.9.0 on GPU-accelerated hardware. Computational resources included an Intel Core i7-13700KF CPU (3.4 GHz) with 32 GB DDR4 RAM. Evaluation indicators In this study, mean absolute error (MAE), root mean square error (RMSE), and coefficient of determination (R2) are used to evaluate forecasting performance. Their definitions are given in Eqs. (17)–(19). where denotes the actual photovoltaic power at sample i, denotes the predicted photovoltaic power at sample i,N is the total number of samples, and denotes the mean value of the actual photovoltaic power. Case study on the benchmark dataset Case study of clustering results of the benchmark dataset In this study, by comparing the proposed AFDTW method with two benchmark methods, the clustering performance on the reference dataset was analyzed: (1) Traditional DTW clustering; (2) DTW Clustering based on Markov Distance (DTWMD). The abbreviations in Fig. 13 are defined as follows: AFDTW (Adaptive Weighted DTW), DTW (Traditional DTW), and DTWMD (DTW Based on Markov Distance). The experimental results show that the value of the AFDTW method is 0.740, the value of the DTW method is 0.329, and the value of the DTWMD method is 0.276. Compared with traditional DTW clustering and DTW based on Markov distance (DTWMD), the adaptive weighted DTW clustering method performs better in the evaluation of contour coefficients. RMSE value. This Fig. 7 shows the statistical characteristics of power generation (mean, standard deviation, median) of the C0-C3 clusters under different meteorological conditions. There are significant differences in the central tendency index among each cluster. For example, the mean value of C0 is 297.2 MW and the median is 341.2 MW, while the mean value of C3 is 338.6 MW and the median is 346.4 MW. The systematic changes in power generation levels indicate the influence of meteorology on power generation. Although there are specific biases in each cluster (such as C0:203.8 MW), all values are within the acceptable range. This indicates that the data dispersion is moderate and the consistency within the clusters is strong. This clustering method successfully distinguished the meteorological conditions and grouped similar patterns. The significant differences in clustering statistics confirm the effectiveness of this algorithm in capturing weather-driven power generation patterns. The AFDTW clustering method can effectively handle the data set and capture the influence of meteorology on power generation, showing high reliability. The values of each historical database. From a qualitative perspective, the four clusters exhibit distinguishable meteorological and generation patterns. Cluster 0 is characterized by relatively smooth power curves with higher peak consistency, corresponding primarily to predominantly clear-sky conditions with comparatively stable irradiance patterns, although moderate short-term fluctuations remain observable in several days. Cluster 1 presents moderate fluctuations, which are typically associated with partially cloudy weather. Cluster 2 shows higher variability and intermittent drops in output, indicating transitional cloud cover conditions. Cluster 3 demonstrates suppressed generation levels and irregular patterns, which may correspond to overcast or low-irradiance weather regimes. These observations suggest that the AFDTW clustering does not merely produce statistical partitions, but effectively segments the dataset into physically meaningful weather regimes. The clustering results align with typical sky-condition categories observed in photovoltaic systems, thereby enhancing the interpretability of the regime-aware forecasting framework. Benchmark dataset description and pre-processing To assess the method’s effectiveness, we used a benchmark dataset from Australian photovoltaic (PV) power plants for comparison. The dataset covers December 16, 2016 to June 16, 2017, sampled at 5-min intervals. In this study, the “benchmark dataset” refers to the primary historical dataset used for model training, hyperparameter optimization, and systematic performance comparison against baseline methods. It is derived from real operational measurements of the same distributed PV system. The term “benchmark” emphasizes its role as the reference dataset for controlled experimental evaluation rather than implying any artificial or synthetic characteristics. In addition to the measurements collected at the target PV site, auxiliary meteorological information from nearby monitoring stations was incorporated during data construction. These spatially distributed meteorological observations were aligned and fused to enhance the representation of regional weather dynamics affecting the target PV system. Therefore, although the forecasting target corresponds to a single PV plant, the input feature set contains spatially correlated information derived from multiple surrounding locations, enabling the model to capture inter-site spatial dependencies. It comprises 13 variables: stage average; active energy transfer/reception; wind speed; air temperature (°C); global horizontal irradiance; wind direction; daily rainfall; maximum wind speed; barometric pressure; hail accumulation; a pyranometer reading; and two temperature-probe readings. In total, the dataset contains 856,213 records. The Supplementary Dataset file has been updated to ensure full consistency with the time span described above (December 16, 2016 to June 15, 2017).Although the dataset spans approximately six months, it covers both summer and winter conditions in the southern hemisphere, thereby encompassing substantial seasonal variability. To evaluate the representativeness of the dataset, we analyzed the statistical distribution of key meteorological variables, including global horizontal irradiance, air temperature, and PV output fluctuations. The results indicate significant variability across clear-sky days, partially cloudy transitions, and high-variability weather regimes within the study period.Such intra-seasonal diversity provides sufficient structural heterogeneity for evaluating regime-aware clustering and multi-horizon forecasting performance. While a full annual cycle would offer additional long-term seasonal coverage, the selected time span captures representative transitional meteorological states that are particularly critical for ultra-short-term forecasting tasks. Although the experiments are conducted on a single Australian PV dataset, the proposed framework is not geographically dependent. The AFDTW clustering mechanism operates on statistical regime characteristics rather than location-specific parameters, and the ITCN–LSTM architecture learns multivariate temporal patterns directly from input features without embedding region-specific assumptions. Therefore, the framework is inherently adaptable to datasets from different climates and installed capacities, provided that appropriate retraining is performed. The methodological design focuses on regime-aware segmentation and multi-scale temporal modeling, which are general characteristics of photovoltaic power systems rather than site-specific features. We trained and validated a model for each cluster to assess adaptability. For each cluster, a chronological rolling evaluation strategy was adopted. Instead of relying on a single short test segment, multiple non-overlapping evaluation windows were extracted from the later portion of each cluster to ensure representative performance assessment. Each evaluation window contained 150 consecutive time steps (corresponding to 12.5 h at 5-min resolution), and forecasting performance was aggregated across all evaluation windows. In total, the testing phase covered multiple representative days across different weather regimes, thereby increasing statistical reliability and reducing bias caused by isolated short-term fluctuations.We applied standard preprocessing—outlier removal, missing-value imputation, and min–max normalization.For anomaly detection, we employed a Markov-distance-based statistical thresholding approach. Specifically, for each feature sequence, the Markov distance distribution was computed, and an adaptive threshold was determined using a percentile-based criterion. Samples with Markov distance values exceeding the 95th percentile of the empirical distribution were classified as anomalies.Statistical analysis shows that approximately 2.8% of the total records were identified as anomalous under this criterion. These anomalous samples were not directly discarded. Instead, they were handled through a two-step correction procedure: isolated outliers were replaced using linear interpolation, while consecutive abnormal segments were smoothed using local moving-average reconstruction to preserve temporal continuity.This approach ensures that extreme measurement noise and sensor faults are mitigated while maintaining the structural characteristics of the original time series. We performed a grid search to select the optimal window size, which was 25 time steps. This window length captured temporal dependencies and yielded the best predictive accuracy. We further computed Pearson correlations between window size and performance metrics, supporting the choice of 25 by indicating efficient temporal-pattern extraction and reduced prediction error. Detailed correlations between window size and performance metrics are reported in Table 1. Although the Pearson correlation coefficient for a window length of 30 is numerically higher, additional forecasting experiments revealed that increasing the window size beyond 25 did not produce consistent improvements across different prediction horizons. A larger window introduces redundant historical information and slightly increases computational complexity, which may reduce model generalization capability due to temporal feature over-smoothing. Empirical validation showed that a 25-step window achieves a better trade-off between temporal dependency capture, model stability, and computational efficiency. Therefore, 25 was selected as the optimal sliding window length considering both statistical correlation and practical forecasting robustness (Table 2). Table 2Correlation coefficient.Sliding window lengthPearson correlation coefficient5–0.68655100.58559615–0.46789200.051226250.300576300.71224 Correlation coefficient. Hyperparameter settings for each cluster All prediction methods are calibrated using the CMA-ES algorithm to ensure the relative fairness of the hyperparameter optimization process. To ensure reproducibility, the CMA-ES hyperparameter optimization procedure is specified as follows. The algorithm was employed to optimize key model hyperparameters, including the learning rate, number of convolutional filters, hidden layer dimension, and dropout rate.The search ranges were defined as: learning rate in , convolutional filter size in [16,128], hidden dimension in [32,256], and dropout rate in [0.0,0.5]. All parameters were encoded into a continuous search vector and mapped to valid discrete values where necessary.The maximum number of iterations was set to 50 generations, with a population size automatically determined according to the default CMA-ES strategy based on problem dimensionality. Convergence was defined by either reaching the maximum generation limit or when the relative improvement in validation loss fell below 10−4 over five consecutive generations.This configuration ensures a balance between search efficiency and solution stability while preventing excessive computational overhead. Assign the initial hyperparameter values to each cluster and perform iterative refinement through multiple parameter variations to determine the optimal configuration. By identifying the sse—quantitative inflection point of clustering, the baseline clustering configuration was established using the elbow method. Through the statistical significance test, the verification analysis confirmed that 4 was the optimal number of clusters. Table 3 summarizes the cluster-specific hyperparameter values. Table 3Hyperparameter selection results of all considered methods.ClusterHyperparameter settingsC0num_layer = 2, num_filters = 31, Conv1D_kernel_size = 3,Conv2D_kernel_size = (3,1), lstm_hidden_size = 256, lstm_num_layers = 1, fc_layers = [256,128,64]C1num_layer = 2, num_filters = 30, Conv1D_kernel_size = 3,Conv2D_kernel_size = (3,1), lstm_hidden_size = 255, lstm_num_layers = 2, fc_layers = [255,64,32]C2num_layer = 2, num_filters = 24, Conv1D_kernel_size = 3,Conv2D_kernel_size = (3,1), lstm_hidden_size = 258, lstm_num_layers = 1, fc_layers = [258,64,32]C3num_layer = 2, num_filters = 31, Conv1D_kernel_size = 3,Conv2D_kernel_size = (3,1), lstm_hidden_size = 256, lstm_num_layers = 2, fc_layers = [256, 64, 32] Hyperparameter selection results of all considered methods. num_layer = 2, num_filters = 31, Conv1D_kernel_size = 3, Conv2D_kernel_size = (3,1), lstm_hidden_size = 256, lstm_num_layers = 1, fc_layers = [256,128,64] num_layer = 2, num_filters = 30, Conv1D_kernel_size = 3, Conv2D_kernel_size = (3,1), lstm_hidden_size = 255, lstm_num_layers = 2, fc_layers = [255,64,32] num_layer = 2, num_filters = 24, Conv1D_kernel_size = 3, Conv2D_kernel_size = (3,1), lstm_hidden_size = 258, lstm_num_layers = 1, fc_layers = [258,64,32] num_layer = 2, num_filters = 31, Conv1D_kernel_size = 3, Conv2D_kernel_size = (3,1), lstm_hidden_size = 256, lstm_num_layers = 2, fc_layers = [256, 64, 32] Adam is used as the optimizer for all methods, with MSE as the loss function. The initial learning rate is 0.0005, except for the LSTM-LSTM model, which uses a learning rate of 0.001, a batch size of 30, and 20 epochs. Experimental results and analysis Figure 8 shows the sequence prediction curves of the vertically arranged historical clusters (C0-C3). The panel on the right can conduct a comparative analysis of the performance changes of the methods by enhancing the contrast. In this paper, a representative cluster C0 after clustering is selected. Table 4 details the performance indicators of different methods in predicting C0. The dominant values in statistics are represented in bold. Prediction curves of different methods under different clusters. Table 4Evaluation indicators of different prediction methods in C0.Evaluation indicatorsDifferent methods5 min15 min1 h4 hMAE(MW)Bi-LSTM36.73264.65831.40150.012LSTM-LSTM34.70335.42528.86667.294TCN-LSTM34.33258.74420.30763.356Proposed method19.73424.6319.12730.522RMSE(MW)Bi-LSTM50.30464.65736.47770.272LSTM-LSTM48.79249.48940.97093.506TCN-LSTM46.61077.78125.81989.510Proposed method31.84734.16411.67743.359R2Bi-LSTM0.9500.8540.9730.891LSTM-LSTM0.9530.9510.9670.806TCN-LSTM0.9570.8800.9860.823Proposed method0.9800.9780.9970.903 Evaluation indicators of different prediction methods in C0. The following findings can be concluded from Table 3 and Figs. 9, 10, 11. MAE value. RMSE value. R2 value. Table 4 reports evaluation metrics across multiple horizons; the proposed method outperforms competing approaches in PV power forecasting. First, mean absolute error (MAE) is a key accuracy metric. Our method yields the lowest MAE across all horizons. Specifically, MAE is 19.734 and 24.631 at 5- and 15-min horizons, and 9.127 and 30.522 at 1- and 4-h horizons, respectively. These results indicate high predictive accuracy. By comparison, Bi-LSTM, LSTM-LSTM, and TCN-LSTM show larger MAE, especially at longer horizons where errors grow markedly. Thus, our method reduces prediction error more effectively and improves accuracy. Second, root mean squared error (RMSE) further highlights our method’s advantages. It attains the lowest RMSE across all horizons. Specifically, RMSE is 31.847 and 34.164 at 5- and 15-min horizons, and 11.677 and 43.359 at 1- and 4-h horizons, respectively. These values are lower than those of all baselines, yielding more accurate forecasts. For Bi-LSTM and LSTM-LSTM, RMSE increases substantially with horizon length. At the 4-h horizon, their RMSE reaches 70.272 and 93.506, indicating reduced long-horizon stability. Although TCN-LSTM performs well at short horizons, its RMSE remains high at longer horizons, reaching 89.510 at 4 h. Together, these results show that our method maintains low error across multiple horizons. Finally, the coefficient of determination (R2), a measure of goodness of fit, also corroborates the proposed method’s superiority. It achieves the highest R2 across all horizons. At 1- and 4-h horizons, R2 reaches 0.997 and 0.903, surpassing all baselines. This indicates stronger long-horizon fitting ability and more effective modeling of multivariate temporal dynamics. By contrast, Bi-LSTM, LSTM-LSTM, and TCN-LSTM perform well at 5-min and 1-h scales, but their R2 drops at 4 h—to 0.891, 0.806, and 0.823—indicating weaker long-horizon fit. Overall, the proposed method excels in MAE, RMSE, and R2, delivering more accurate and stable results for both short- and long-term PV forecasting. Within our evaluation, it ranks best among compared models for PV power forecasting and substantially improves the accuracy of distributed-generation predictions. It is worth noting that the forecasting performance at the 1-h horizon is slightly better than that at the 15-min horizon in some cases. Although longer forecasting horizons are generally expected to produce larger errors due to accumulated uncertainty, this phenomenon can be attributed to the temporal aggregation effect. Specifically, the 1-h ahead prediction target represents a relatively smoother temporal profile compared to the 15-min horizon, where high-frequency irradiance fluctuations and short-term cloud movement introduce stronger volatility. The inherent smoothing effect at longer temporal intervals reduces high-frequency noise components and makes the underlying trend easier to model. Therefore, the improved performance at the 1-h horizon does not contradict theoretical expectations but reflects the statistical characteristics of aggregated photovoltaic power signals under multi-scale temporal variability. To further investigate the regime-specific performance, we analyzed forecasting accuracy within each cluster separately. The results indicate that prediction errors under clear-sky conditions (Cluster 0) are consistently lower due to smoother and more stable irradiance patterns. In contrast, clusters associated with highly variable cloud transitions (e.g. Cluster 2 and Cluster 3) exhibit relatively higher errors due to rapid irradiance fluctuations and intermittent shading effects. Despite these variations, the proposed AFDTW–ITCN–LSTM framework maintains stable and competitive performance across all clusters, demonstrating its robustness to diverse meteorological regimes. This per-cluster analysis confirms that the improvement achieved by the proposed method is not limited to aggregated metrics but also holds consistently under heterogeneous weather conditions. Statistical significance analysis To further verify whether the observed performance improvements are statistically significant rather than due to random variation, we conducted paired t-tests between the proposed AFDTW–ITCN–LSTM model and each baseline model across all forecasting horizons. For each horizon, forecasting errors were collected across all evaluation windows, and paired comparisons were performed on MAE and RMSE values. The null hypothesis assumes that there is no significant difference between the proposed method and the baseline models. The test results indicate that the proposed model achieves statistically significant improvements over all benchmark models at the 95% confidence level (p < 0.05) across all forecasting horizons. In particular, the performance gains are more pronounced at longer horizons (1-h and 4-h ahead), where baseline models exhibit larger variance and reduced stability. These findings confirm that the observed error reductions are not caused by random fluctuations in the test data but reflect consistent and reliable performance improvements of the proposed framework. The obtained p-values were consistently below the 0.05 significance level, with most comparisons yielding values lower than 0.01. In addition, the magnitude of the performance differences indicates practical relevance beyond statistical significance. Ablation study To evaluate the independent contributions of the major architectural components in the proposed ITCN–LSTM framework, a systematic ablation study was conducted. The experiments were performed on Cluster 2 of the benchmark dataset under identical hyperparameter settings to ensure fair comparison. Five model configurations were evaluated: (1) the complete ITCN–LSTM model, (2) a variant without spatial convolution, (3) a variant without residual connections, (4) a variant replacing the ITCN encoder with a standard LSTM encoder, and (5) a variant replacing the LSTM decoder with a fully connected network. Performance was assessed across multiple forecasting horizons using MAE, RMSE, and R2 metrics (Table 5). Table 5Comparison results of each model in the ablation experiment.Evaluation indicatorsDifferent methods5min15min1h4hMAE(MW)TCN-LSTM6.21015.82031.47055.730LSTM-LSTM8.47021.35039.71065.840ITCN (without residual connections) – LSTM7.15017.14034.26059.280ITCN-FC7.82018.63036.53062.410Proposed method4.83012.53026.84048.320RMSE(MW)TCN-LSTM9.87026.41049.83083.420LSTM-LSTM13.72035.72061.38097.360ITCN (without residual connections) – LSTM11.34028.93053.62088.170ITCN-FC12.45031.25056.84092.530Proposed method7.62021.36042.51072.650R2TCN-LSTM0.9980.9860.9500.859LSTM-LSTM0.9960.9740.9240.808ITCN (without residual connections) – LSTM0.9970.9830.9420.843ITCN-FC0.9970.9800.9350.827Proposed method0.9990.9910.9630.893 Comparison results of each model in the ablation experiment. The ablation results clearly demonstrate that each architectural component contributes positively to forecasting accuracy. Removing the spatial convolution layer results in consistent performance degradation, indicating the importance of cross-variable feature interaction modeling in photovoltaic forecasting (Fig. 12). MAE value. Eliminating residual connections slightly reduces performance stability, particularly at longer forecasting horizons, confirming their role in facilitating deep feature propagation and mitigating gradient degradation. Replacing the ITCN encoder with a standard LSTM encoder leads to a noticeable increase in prediction error, highlighting the effectiveness of dilated temporal convolution in capturing multi-scale dependencies. Similarly, substituting the LSTM decoder with a fully connected structure reduces forecasting accuracy, especially for longer horizons, suggesting that sequential decoding enhances temporal consistency and output stability (Figs. 13 and 14). R2 value. It is worth noting that the effectiveness of the AFDTW clustering mechanism has been validated separately through comparative experiments with DTW and DTWMD-based clustering strategies, demonstrating the benefit of adaptive regime-aware segmentation prior to model training. Overall, the complete ITCN–LSTM model achieves the best overall performance across all evaluation metrics, validating the necessity of integrating adaptive Multivariate Temporal Feature Extraction with sequential decoding mechanisms. Computational efficiency analysis To evaluate the practical efficiency of the proposed AFDTW pruning strategy, computational time comparisons were conducted under the same experimental environment described in Sect. "Case study on the benchmark dataset". All implementations were developed in Python 3.8.8 using PyTorch 1.9.0 and executed on GPU-accelerated hardware. The computational platform included an Intel Core i7-13700KF CPU (3.4 GHz) with 32 GB DDR4 RAM. We compared three configurations: (1) standard DTW-based clustering with FCM, (2) AFDTW without pruning acceleration, and (3) the proposed AFDTW with window constraint and pruning strategy. The results demonstrate that the proposed pruning strategy significantly reduces clustering time compared with the standard DTW-based approach, achieving approximately 30–35% reduction in computational cost. It is worth noting that the pruning mechanism primarily affects the distance calculation phase in clustering, while the neural network training and inference stages remain largely unaffected. This confirms that the efficiency improvement originates from reducing redundant warping path exploration without compromising model accuracy. These findings validate the practical feasibility of the proposed AFDTW framework for large-scale photovoltaic datasets. Case study of actual dataset Actual dataset description and pre-processing To further validate the performance of the improved prediction method based on deep codec adaptive weighted DTW clustering for photovoltaic (PV) power prediction, we used a real dataset from the 1.8 MW Yulara Solar System distributed PV plant located in Uluru (Ayers Rock), Central Australia. The performance of the proposed prediction method was assessed. The dataset includes meteorological and power generation data collected from June 14 to June 15, 2017, with values ranging from -0.2471 to 845.6216, recorded at 5-min intervals. The meteorological data includes 13 variables: current average, active energy transfer/reception, wind speed, temperature (°C), global horizontal radiation, wind direction, daily rainfall, maximum wind speed, air pressure, hail accumulation, pyrometer 1, temperature probe 1, and temperature probe 2. The original dataset contains 288 entries, and all the data from that day were used as test data. To clarify the distinction between datasets, it should be emphasized that both the benchmark dataset (Section "Case study on the benchmark dataset") and the actual dataset used here are real measured PV data from the same system. The benchmark dataset spans six months and is used for model training, hyperparameter tuning, and statistical evaluation. In contrast, the actual dataset represents an independent one-day operational scenario that was not involved in model training and is used solely to assess the generalization capability of the trained model under new daily weather conditions. The purpose of selecting a continuous one-day dataset is to simulate realistic deployment conditions, where forecasting systems are typically evaluated over rolling daily operational cycles. The statistical conclusions of this study are primarily derived from the benchmark dataset, while the one-day case study serves as an independent validation of short-term predictive stability. First, we find the center of the dataset of the day, and then calculate its distance from the original data cluster. The distance metrics are shown in Table 6. The calculation results indicate that all daily data are within the range of C2, thus enabling direct prediction using the C2 historical dataset (Table 7). Table 6Computational time comparison.MethodClustering Time (s)Training Time (s)Inference Time per Sample (ms)Standard DTW + FCM428.6512.33.47AFDTW (without pruning)395.8509.63.41Proposed AFDTW (with pruning)273.4507.93.38 Computational time comparison. Table 7The distance between the center of the dataset and each historical database.C0C1C2C3Focus on the data of the day526,737.883465,822.592451,620.263451,656.100 The distance between the center of the dataset and each historical database. Analysis of experimental results Table 8 and Fig. 16 compare the actual and predicted values of the proposed method, with the prediction curve illustrated in Fig. 16. The right panel of Fig. 15 provides enhanced visual clarity for the prediction outcomes. Table 8Each index of the proposed method.MAE(MW)RMSE(MW)R2Bi-LSTM14.98325.1740.951LSTM-LSTM13.49422.0690.962TCN-LSTM9.58217.2410.972Proposed method8.70014.2100.985 Each index of the proposed method. Prediction curves of different methods with current day dataset. Each index of the proposed method. Figure 16 compares the forecasted photovoltaic power generated by the proposed method with the corresponding measured values. The red curve denotes the measured power, and the blue curve denotes the forecasted power. As shown in the figure, the proposed method can track the overall variation trend of photovoltaic output with relatively small deviations over most time intervals. Combined with the quantitative results in Table 8, the superiority of the proposed framework is not only reflected in the numerical metrics, but is also consistent with its methodological design. First, AFDTW-based clustering decomposes heterogeneous photovoltaic samples into more homogeneous weather-regime-specific subsets, which reduces distributional mixing and allows the forecasting model to learn more stable local input–output relationships. Second, the ITCN encoder strengthens the representation of multi-scale temporal dependencies and cross-variable interactions, which is beneficial for modeling complex photovoltaic fluctuations under changing meteorological conditions. Third, the LSTM decoder improves sequential continuity during multi-step prediction, thereby enhancing forecasting stability across different prediction horizons. From the visual comparison in Fig. 16, the proposed method shows good agreement with the measured series not only in the general trend but also around major peaks and troughs. Although small deviations still appear near rapidly oscillating regions, the forecast curve remains close to the observed trajectory in most intervals, indicating that the proposed framework has good adaptability to nonlinear and volatile photovoltaic generation patterns. These results suggest that the observed performance improvement is closely related to the coordinated effect of regime-aware clustering and cluster-specific encoder-decoder forecasting, rather than being a simple result of model stacking. Overall, the proposed method demonstrates improved forecasting accuracy and stability compared with the benchmark models on the current dataset. However, some local deviations remain under abrupt short-term fluctuations, which also indicates that photovoltaic forecasting performance is still affected by strong transient uncertainty. Therefore, the proposed framework should be understood as providing a more effective modeling strategy for heterogeneous photovoltaic power forecasting, rather than a universally optimal solution under all operating conditions. It should be noted that both the benchmark dataset and the actual case-study dataset consist of real measured photovoltaic data. The benchmark dataset spans six months and therefore provides sufficient statistical coverage for model training, comparison, and quantitative evaluation. In contrast, the one-day actual dataset is used as an independent case-study dataset for external validation and qualitative analysis of short-term forecasting stability under realistic operating conditions. Accordingly, the main statistical conclusions of this study are drawn primarily from the benchmark dataset, whereas the actual case study is intended to further demonstrate the practical applicability of the proposed framework. Conclusion This study proposes an ultra-short-term photovoltaic power forecasting framework that integrates adaptive feature-weighted dynamic time warping (AFDTW) clustering with an ITCN–LSTM encoder–decoder architecture. The framework performs regime-aware segmentation prior to model training and captures multivariate temporal dependencies for improved forecasting stability. Experimental results based on a real 1.8 MW photovoltaic dataset demonstrate that the proposed method consistently outperforms baseline deep-learning models across forecasting horizons from 5 min to 4 h. The framework achieves approximately 8–12% reductions in MAE and RMSE compared to benchmark architectures. The results indicate that adaptive clustering enhances intra-regime homogeneity, while hierarchical feature extraction improves robustness under varying meteorological conditions. Nevertheless, this study is limited to a single-site dataset and does not include cross-regional validation. Future work will extend the framework to multi-location datasets and compare it with transformer-based architectures to further evaluate generalization capability and scalability. Supplementary Information Supplementary Information. Author contributions X.Z. conceived the study, developed the methodology, and performed the experiments. G.Z. contributed to algorithm design, data preprocessing, and result analysis. W.C. assisted with model implementation, parameter tuning, and validation. J.L. contributed to literature review, manuscript refinement, and visualization of results. L.H. supervised the study, provided research guidance, and reviewed and revised the manuscript. All authors discussed the results and approved the final manuscript. Funding This work was supported by the Science and Technology Project of State Grid (No. 4000-202316456A-3-2-ZN). Data availability Data availability statement: The photovoltaic power generation and meteorological data used in this study were obtained from the public dataset provided by the DKA Solar Centre (Yulara Solar System, Australia). The data can be accessed and downloaded from https://dkasolarcentre.com.au/download?location=yulara. Declarations Competing interests The authors declare no competing interests. Footnotes Publisher's note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Supplementary Materials Supplementary Information. Data Availability Statement Data availability statement: The photovoltaic power generation and meteorological data used in this study were obtained from the public dataset provided by the DKA Solar Centre (Yulara Solar System, Australia). The data can be accessed and downloaded from https://dkasolarcentre.com.au/download?location=yulara.