Version upgrades in treatment planning Systems affect clinical application of a multi-Centre knowledge-based planning model Authors: Karen McGoldrick, Colin Hornby, Peter Griffin, Katrina Woodford, Flavio Nelli, Cathy Markham, Tomas Kron, Nick Hardcastle, Vanessa Panettieri Published: 2026 ABSTRACT AbstractPurposeTo determine if version upgrades to treatment planning system software have an impact on the clinical application of a knowledge-based planning model built in a previous version.MethodsTwenty patients’ data sets treated for prostate cancer were retrospectively selected for planning on three different versions of Eclipse (Varian Medical Systems) treatment planning system, (Vn13.6, Vn15.6, Vn16.1) using a multi-centre RapidPlan model that was built in Vn13.6. Each plan was created using identical beam arrangement, and optimisation objectives and calculation settings. The RapidPlan model was also re-trained in Vn15.6 and Vn16.1 and the datasets were replanned using these models with the same method.ResultsSeven plan comparisons were performed, evaluating four target metrics and eight organ at risk (OAR) metrics. Statistically significant differences to median values for target dose metrics were observed in 10 out of 28 plan comparisons and for OARs in 23 out of 56 plan comparisons. Re‑extracted and re‑trained models in Vn15.6 and Vn16.1 improved target coverage and several OAR dose metrics. Small increases were observed in some OAR metrics for a subset of plans using the Vn15.6 model; however, the largest variations were minor (≤1.4%), including Bladder_65Gy (<1.3%), Bladder_70Gy (<0.7%), Rectum_65Gy (<0.9%), and Rectum_70Gy (<1.4%).ConclusionEclipse version upgrades will have varying effects on the resultant plans optimised using a RapidPlan models developed and trained in earlier versions. This study confirms the recommendation from the vendor that it is important to re-extract and re-train RapidPlan models when an Eclipse version upgrade is undertaken to ensure optimal dosimetry is achieved. Introduction Knowledge based planning (KBP) has become increasingly important in radiotherapy planning as a tool to standardise treatment plans within and across institutions and reduce treatment plan quality variation between individual planners [1], [2], [3], [4], [5], as this has been shown to improve plan quality and consistency [3], [5], [6], [7], [8]. KBP is also widely used within radiotherapy departments to ensure quality plans are produced in an efficient time. Among different solutions RapidPlan (RP) is the Varian Eclipse™ (Varian Medical Systems, Inc) treatment planning system (TPS) automated tool which allows clinicians to optimise new plans using model–based Dose Volume Histogram (DVH) estimations and optimisation objectives derived by model training using machine learning from previous site-specific treatment plans. As described elsewhere [4], [9], [10] the RP knowledge-based planning engine consists of three main subsystems: i) a model training environment, ii) a dose volume histogram prediction environment, and iii) the generation of personalized dose-volume constraints for the plan optimisation. In 2016 the Victorian Public Sector RapidPlan Group (VPSRG) was formed which consists of multidisciplinary representatives from nine radiotherapy departments in the state of Victoria, Australia. The purpose of the group was to collaboratively develop automated models by sharing and combining high quality plans to use for the training set that represented clinical practice at all institutions. One of the first models developed and published by the VPSRG was a Prostate RP model for use across the nine public radiotherapy departments in the state which has now been in use since 2016 [5]. This project has facilitated an on-going collaboration to implement an automated approach with the aim of reducing inter and intra-centre planning variability. Automated plans produced using the VPSRG prostate model were generally of equivalent or higher quality to the manually optimised ones after a single optimisation and no further planner intervention [5]. There are many publications showing that RP models can achieve high quality plans, often outperforming clinically accepted manually produced modulated plans [11]. However, the time required to develop and validate RP models can be a barrier to how many can built at a time, even when a sufficient number of quality plans exist - highlighting that model building may be more achievable in a consortium like the VPSRG where the workload can be distributed [6], [11]. As the VSPRG is a collaboration of nine departments, the possibility of different departments using different versions of Eclipse occurred over time, and there was a need to have a greater understanding of the impact of these different versions on RP models built collaboratively. As each department underwent Eclipse TPS version upgrades, they not only updated the main modules of the TPS software, but also more importantly there were updates to the calculation algorithms used for optimisation and dose calculation. This introduced changes that could potentially affect the dosimetry and quality of the patient treatment plans and effectively require re-commissioning of the dose calculations [12]. For RapidPlan, the vendor recommends re-extracting, re-training and then re-validating the model at each upgrade. This required every centre to undertake re-validation of models to ensure they were still providing the optimal clinical plan. Validation of RP models can be a resource intensive exercise particularly when a model needs to be applied to, and analysed across, a large cohort of patients across a number of sites [4], [5], [6], [9], [10], [11]. Current literature describes dose differences between versions of a TPS that use different calculation and optimisation algorithms and the resultant effect on treatment plans [13], [14], [15]. However, there is little evidence relating to whether these differences have an impact on KBP. Several factors may affect the performance of KBP following a TPS upgrade, including changes to dose calculation algorithms, dose optimisations and the application of the planning model. The primary aim of the study was to determine if there was any significant difference to plans generated on a more recent version of Eclipse using a model developed in an earlier version, and by implication if there was any benefit to re-training the model in the newer version. This would inform institutions on the impact of TPS upgrades on previously developed models and whether they are suitable to use clinically. Method Data planning Twenty datasets from previously treated patients with prostate cancer were selected from the training dataset and used for this study. In this cohort 14 were intact prostate and 6 were prostate bed post-surgery where the target volume included seminal vesicles. For these clinical cases the prescription doses ranged from 66 to 78 Gy, which represented a spread of protocols in use across all departments involved in the study. Datasets were selected from all sites from within the VPSRG to fully represent the multi-centre setting and all datasets used were a part of the original training dataset. Ethics approval was obtained to undertake the study (HREC Reference Number: HREC/Alfred/126, Local Reference Number: SSA/41107/PMCC-2018). Six plans were generated for each dataset as shown in Fig. 1. The first plan was optimised on Eclipse Vn13.6 using the original VPSRG v2.1 RP model and was labelled v136–1. This original VPSRG RP prostate model (VPSRG v2.1) was built using a total of 135 approved and treated training prostate plans that were submitted from eight of the VPSRG departments. These plans were used to train the model in Vn13.6 of the Eclipse TPS and both closed and open loop validation [6]was performed prior to clinical implementation. Flow diagram of naming for the 6 optimised plans analysed for each data set. Green describes the RapidPlan version the VPSRG v2.1 model was trained in, and blue describes the version of Eclipse used for the plan optimisation and calculation. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.) The second plan was optimised on Eclipse Vn15.6 using the original VPSRG v2.1 model and was labelled v156–1. The third plan was optimised on Eclipse Vn16.1 using the original VPSRG v2.1 model and was labelled v161–1. The next step was re-extracting and re-training the original VSPRG v2.1 model in Eclipse Vn15.6 and then the fourth plan was optimised in Eclipse Vn15.6 using this newly re-extracted and re-trained model. This plan was labelled v156–2. The fifth plan was optimised and calculated in Eclipse Vn16.1 using the this newly re-extracted and re-trained model, and the plan was labelled v161.3. Lastly, the VSPRG v2.1 model was re-extracted and re-trained in Eclipse Vn16.1 and a sixth plan was optimised in Eclipse Vn16.1 using this model. This plan was labelled v161–2. All data sets contained the same structures using the VPSRG agreed naming conventions and contouring guidelines. The volumes employed for this study were CTV_Dose, PTV_Dose, Bladder, Femur_Head_L, Femur_Head_R and Rectum. The target volume (TV) and organ at risk (OAR) naming convention used was based on TG263 [16] and the contouring guidelines were aligned to Global Harmonisation Group consensus guidelines [17]. To reduce any planner bias, all plans for the study were created using the exact same criteria. The plan geometry was standardised with each isocentre placed centrally to target with 2 full opposing VMAT arcs. Arc stop and start angles were 179–181° (counterclockwise) with a collimator angle 10° and 181–179°(clockwise), with collimator angles 350°. A radiation quality of nominally 10MV was chosen for all beams. Jaws were set using autofit to PTV with a 0.5 cm margin and jaw tracking was used for all beams. The dose optimisation settings were set with convergence mode turned on and the aperture shape controller was set to moderate. Plans were optimised with auto crop turned off to enable Eclipse Vn13.6 to be included in the evaluation, so an additional target volume corresponding to PTV minus CTV was created to be used in dose optimisation. GPU was turned on. In the optimiser Automatic intermediate dose was selected and so was Automatic optimisation. Plan normalisation was set to No plan normalisation. A single optimisation was completed without any adjustments or planner intervention to minimise bias. Data analysis Clinical goals extracted from the agreed departmental protocols were used to compare the difference between plans. To evaluate target coverage the CTV D98%, PTV D98%, D2% and D50% (defined as the percentage of prescription dose received by the specified percentage of the target) were used. For the OARs the bladder V65Gy, V70Gy and the rectum V30Gy, V40Gy, V50Gy, V60Gy, V70Gy (defined as percentage of the volume receiving a specified dose) were analysed. Doses to Femur_L & Femur_R were low, and no analysis performed. The dose calculation algorithm for plans calculated in Eclipse Vn13.6 (v136–1 plans) was Anisotropic Analytical Algorithm (AAA). All other plans were calculated using Acuros XB (dose-to-medium) with the same version upgraded (i.e. for 156–1 AXB 15.6). Statistical analyses were carried out in IBM SPSS Statistics (Version 28/29). A Wilcoxon Signed Rank Test was used to compare median values for all dose metrics as the data was not normally distributed. Comparisons were made using the following pairings v136–1 vs. v156–1, v136–1 vs. v156–2, v136–1 vs. v161–1, v136–1 vs. v161–2, v156–1 vs. 156–2, v161–1 vs. 161–2 and v156.2 vs 161.3. The measure of significance was defined by a p-value less than 0.007, using the Bonferroni correction to account for multiple comparisons [18]. Results A total of 120 plans were generated (6 per dataset). A summary of the patient datasets used are provided in TABLE 1. Boxplots of the target volume and OAR metrics per plan are displayed in Fig. 2. Planning goals, Median Values and p-values are provided in TABLE 2. Table 1– Summary of patient data used in analysis.Intact ProstateProstate BedNumber of Cases146Prescription Dose (Gy)74–7866–70Number of Fractions36–3933–35Dose per Fraction (Gy)2–2.12Target Vol Mean (cc)149.64 (32.27)269.1 (47.35) – Summary of patient data used in analysis. – Boxplots showing dosimetric parameters in plans calculated in Eclipse Vn13.6 with original model (v136–1), Eclipse Vn15.6 with original model (v156–1) and Eclipse Vn15.6 with retrained model in Vn15.6 (v156–2), Eclipse Vn16.1 with original model (v161–1), Eclipse Vn16.1 with retrained model in Vn16.1 (v161–2) and Eclipse Vn16.1 with retrained model in Vn15.6 (v161.3) for metrics described in TABLE 2. Goals are displayed on the vertical axis. Table 2– Median values for defined target and organs at risk metrics for plans optimised with a RP model built in Eclipse Vn13.6 and then recalculated in later Eclipse versions with and without retraining and p-value for difference between median values for defined target and organs at risk metrics between plans optimised and final calculation done on different Eclipse versions.MetricGoalMedian Dose (IQR)p-valueOptimised with original VPSRG v2.1 ModelOptimised with re-trained modelsv136.1 v v156.1v136.1 v v161.1v136.1 v v156.2v136.1 v v161.2v156.1 v v156.2v161.1 v v161.2v156.2 v v161.3v136–1v156–1v161–1v156–2v161–2v161–3CTV_D98%(%)>98%101.70(0.67)101.56(1.79)101.73(0.21)101.66(0.15)101.77(0.14)101.77(0.17)0.0400.6400.8960.1980.0040.004<0.001PTV_D98%(%)>95%96.90(1.68)96.97(2.07)97.67(0.91)97.99(0.68)98.09(0.65)98.22(0.79)0.370<0.001<0.001<0.001<0.001<0.0010.009PTV_D2%(%)<105%103.60(0.92)104.15(2.00)104.13(0.16)104.26(0.32)104.09(0.25)104.05(0.20)0.7370.0200.0060.0350.1350.6810.004PTV_D50%(%)<102%,>99%102.30(0.83)102.05(1.92)102.19(0.26)102.20(0.20)102.21(0.27)102.21(0.28)0.0790.9260.8400.9400.1390.4930.879Bladder_V65Gy(%)<25%11.03(9.63)10.25(6.9)10.16(10.64)11.51(11.43)11.10(10.79)11.02(10.62)0.0020.0020.0010.173<0.001<0.001<0.001Bladder_V70Gy(%)<20%7.65(8.85)6.96(7.66)7.40(9.08)7.65(10.39)7.53(10.17)7.60(9.88)0.1070.048<0.0010.014<0.001<0.001<0.001Rectum_V30Gy(%)<60%45.92(15.49)43.16(10.5)42.68(9.81)41.59(14.51)41.18(15.40)41.42(13.56)0.0060.0010.0100.0070.1910.3130.018Rectum_V40Gy(%)<45%34.08(13.88)32.03(12.83)31.61(13.66)31.93(14.38)30.80(14.61)30.83(14.36)0.006<0.0010.0150.0050.3320.2960.002Rectum_V50Gy(%)<35%25.74(14.41)24.57(15.83)24.42(14.34)24.64(14.23)23.70(13.74)23.95(13.71)0.0420.0050.0700.0120.4110.2470.002Rectum_V60Gy(%)<30%18.22(11.53)17.60(14.1)17.88(12.25)18.21(12.43)17.53(12.42)17.76(12.26)0.9550.5880.8230.2630.7790.2710.001Rectum_V65Gy(%)<25%13.90(8.76)13.79(9.95)14.28(10.28)14.67(10.79)14.12(10.75)14.41(10.39)0.2870.0440.0400.4220.4780.211<0.001Rectum_V70Gy(%)<20%6.70(5.40)7.07(7.26)8.52(7.99)8.47(7.76)8.00(7.45)8.24(7.49)0.4780.011<0.0010.156<0.0010.0640.002 – Median values for defined target and organs at risk metrics for plans optimised with a RP model built in Eclipse Vn13.6 and then recalculated in later Eclipse versions with and without retraining and p-value for difference between median values for defined target and organs at risk metrics between plans optimised and final calculation done on different Eclipse versions. As shown in Table 2, 33 out of 84 comparisons were statistically different. Overall, there was a general trend toward improved target coverage with equivalent or reduced OAR doses with the model re-trained in Eclipse Vn16.1 and optimised in Eclipse Vn16.1. Fig. 3, Fig. 4 display line plots for each target and OAR goal for each dataset and plan. The plans calculated in Eclipse Vn15.6 with the original model (v156–1) displayed the greatest range of variation in the target metrics with one plan not achieving the planning goal of CTV_D98% > 98% and 3 plans not achieving PTV_D98% > 95%. Out of the 120 plans analysed, planning goals were not achieved in 15 plans for Bladder_V65Gy and 14 plans for Bladder_V70Gy. These failures were observed across only three out of the 20 datasets and may therefore be attributable to patient-specific factors, such as target–OAR geometry rather than the TPS version. For rectal constraints, seven plans did not meet Rectum_V30Gy, 12 failed Rectum_V40Gy, and 17 failed Rectum_V50Gy. Fewer plans failed higher-dose constraints, with six plans not achieving Rectum_V60Gy and Rectum_V65Gy, while all plans met the Rectum_V70Gy planning goal. These failures were similarly confined to five datasets and are likely explained by the same patient-specific influences. A correlation between reduced target coverage and reduced bladder and rectum V70Gy was also observed in Vn15.6.1 plans, representing the most notable finding. Line plots for each clinical target goal by each dataset and plan. Line plots for each clinical OAR goal by each dataset and plan. Discussion In this study, we investigated the impact of TPS upgrades on the performance of RapidPlan models developed in earlier software versions, to determine whether their continued use in clinical practice for treatment plan generation is acceptable. After comparing plans that were calculated in different versions of Eclipse there was a significant difference observed to both target and critical organs at risk in 39% of the plans. Every comparison between versions featured at least one significant difference. Results of this work show a trend of Rapidplan optimisation generally improving over time supported by a notable tightening of the confidence intervals. The PTV_D98% and rectum V30Gy, V40Gy & V50Gy for example consistently improved with each new Eclipse version irrespective of the model used with the improvements being greatest when the model was retrained. However, there was an obvious degradation in plan quality observed in plans that were calculated in Vn 15.6 using the original model trained in Vn13.6, particularly for target coverage. Given the tissue homogeneity of the target volume, it is unlikely that the difference in dose calculation algorithm is the only contributor to this difference [19] and it is most likely due to the changes in Eclipse Vn 15.5 and above within the KBP algorithm. This is further supported by the results for plans calculated in Vn16.1 with the original model trained in Vn13.6. These plans demonstrated improved target coverage, even relative to the original Vn13.6 plans, suggesting that changes to the optimisation algorithm and the addition of new features in Vn16.1 had a greater impact on model performance than changes to the calculation algorithms. From the data it can be seen that Vn 15.6 optimisations were pushing OAR doses down when using a model trained in Vn 13.6 but this came at the expense of CTV and/or PTV coverage significantly for around 25% of cases. However, after the model was re-extracted and re-trained in Vn 15.6 target coverage improved significantly. This can be explained by the modifications in Eclipse Vn 15.5 and above of improved OAR and target overlap modelling and the ability to use line objective type of either target preferring or OAR preferring [20]. There was an improvement in the consistency of target coverage observed in plans calculated in Vn 15.6 with the re-trained model and Vn 16.1 with both original and re-trained model, with all plans achieving above the planning goal of D98 > 95%. For doses to organs at risk, sparing of normal tissue was similar in all plans however there is a statistical significance in the median values in 23 out of 56 plan comparisons. It was observed there was a minor reduction to intermediate rectal dose in re-trained models but there was also a minor increase in higher rectal doses in these plans. The results confirmed the recommendation from the vendor to re-extract and re-train RP models whenever there is an Eclipse version upgrade to ensure the new features of an upgrade are fully utilised and there is no degradation to plan quality with the application of an existing RP model. This requires further consideration when RP models are developed collaboratively across multiple radiotherapy departments, as models built on the current Eclipse version may be affected over time when participating departments upgrade to newer Eclipse releases. One advantage in a collaboration is that only one department needs to undertake the task of re-extracting, re-training and re-validating for newer version of Eclipse which significantly reduces each centre's workload for each new version. This is particularly important where many models exist [21]. However, model training processes might limit building of models by a group of centres, as a RP model can only be re-extracted and re-trained at the department that built the model and has all the patient data. Thus, if the developing site is not on the most recent version of Eclipse, the other departments in the collaboration that are on a more recent version must either wait for the building department to upgrade or would need to (re)build a new model. It may be appropriate to continue to use a RP model that was developed in an earlier version, when understanding the impact it has if used in newer versions and assessing the resources required to re-extract, re-train and re-validate a new model. These considerations potentially also extend to multicentre clinical trials where the KBP models can be used to guide planning or provide quality assurance [21]. Differences should also be evaluated with a clinical focus. Even though the results indicate a statistically significant difference in over a third of the values analysed, whether this translates to any clinically significant difference needs to be considered. Each plan only had a single optimisation, without any further interaction for the purpose of the study. In a clinical setting, dosimetrist interaction would most likely lead to all plans meeting clinical goals for these data sets. This may be more important to consider for plans that include doses to serial organs where the differences due to TPS version may have greater clinical significance. It should also be noted the small sample size is a limitation of this study and further work with a larger sample size could strengthen the results. The VRPSRG was formed with the aim of improving plan consistency across the state. The results also show a general improvement trend in patient-to-patient consistency of TV and OAR metrics with newer versions and the most gains being realised in the retrained models. This also supports the routine retraining of models in new versions of Eclipse software by the developing site as it distributes the advantage of any improvements to all sites within a consortium. While this study focused on one of the tools available for automated planning, its learnings may be applicable to other vendors' solutions or perhaps any future automation software. These findings reinforce the importance of human oversight in the rapidly evolving landscape of automated radiation therapy planning [22], [23], [24]. Even with highly advanced automation tools, clinical plans will always require validation and refinement from clinical experts. Conclusion Knowledge based planning provides an opportunity to increase treatment planning consistency, efficiency and potentially also improve plan quality. This study has shown that Eclipse RapidPlan software version upgrades can offer further improvements in plan quality using existing models, however the most optimal outcomes are achieved when those models are re-extracted and re-trained in the newest version of Eclipse. We therefore confirm that the recommendation from Varian should ideally be followed with each new version of Eclipse. Having to re-train models with each new version mitigates some of the efficiency gains of knowledge-based planning and must be considered when sharing models or using shared models. The impact of this trade-off can be minimised where models are collaboratively developed and shared with other departments within the group, where only one department needs to re-train the model for all departments to gain the benefits from any new features of software upgrades. It is also recommended that any department using a RP model trained in an earlier version of Eclipse should retrain the model as soon as practicable to obtain the optimal patient dosimetry. CRediT authorship contribution statement Karen McGoldrick: Writing – review & editing, Writing – original draft, Validation, Resources, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. Colin Hornby: Writing – review & editing, Supervision, Methodology, Conceptualization. Peter Griffin: Writing – review & editing, Data curation. Katrina Woodford: Writing – review & editing, Supervision. Flavio Nelli: Writing – review & editing, Conceptualization. Cathy Markham: Writing – review & editing, Conceptualization. Tomas Kron: Writing – review & editing, Supervision, Methodology, Conceptualization. Nick Hardcastle: Writing – review & editing, Methodology, Conceptualization. Vanessa Panetteri: Writing – review & editing, Writing – original draft, Supervision, Methodology, Formal analysis, Data curation, Conceptualization. Declaration of competing interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: The Department of Health provided Varian RapidPlan licenses to participating centers in the VPSRG but neither Varian nor the Department of Health influenced any content contained within this article. Acknowledgements The authors wish to acknowledge the Victorian Department of Health for establishing the VPSRG and providing the funding for the RapidPlan licenses. References