Fine-Tuning a Simulation-Driven Estimator Braghadeesh Lakshminarayanan, Margarita A. Guerrero, Cristian R. Rojas Abstract Many industries now deploy high-fidelity simulators (digital twins) to represent physical systems, yet their parameters must be calibrated to match the true system. This motivated the construction of simulation-driven parameter estimators, built by generating synthetic observations for sampled parameter values and learning a supervised mapping from observations to parameters. However, when the true parameters lie outside the sampled range, predictions suffer from an out-of-distribution (OOD) error. This paper introduces a fine-tuning approach for the Two-Stage estimator that mitigates OOD effects and improves accuracy. The effectiveness of the proposed method is verified through numerical simulations. I Introduction Recent advances in high-fidelity simulation have popularized digital twins (DTs)–software replicas of physical systems that let engineers explore operating conditions safely and cheaply before deployment [ 1 , 2 , 3 ] . In practice, once a DT is engineered, most modeling choices are fixed; what remains is calibrating a small set of parameters so the DT replicates a specific physical system. When a DT is available, one can generate synthetic input–output data [ 4 ] over a range of parameter settings and train a supervised machine learning model that maps observations to these settings, thereby yielding a simulation-driven estimator . At implementation, this simulation-driven estimator takes real measurements and returns parameter estimates [ 5 , 6 , 7 ] . The Two-Stage (TS) estimator [ 6 ] is an attractive simulation-driven estimator with a simple structure that involves two stages: (i) first, the observations are compressed into representative features, and then (ii) these compressed features, along with their corresponding parameter values, are used to train a supervised machine learning model. This structure avoids the need for an explicit likelihood function and it requires low computation time [ 8 , App. B.2] for producing estimates, due to extensive offline training, compared to standard point estimators like the prediction error method (PEM) and the dual Extended Kalman Filter (dual-EKF) [ 9 ] . The TS estimator can also be used to learn control design rules, effectively taking the human out of the loop [ 10 ] . A limitation of TS is prior-range misspecification: when the parameters of the true system lie outside the range of the synthetic parameter values used during training, the accuracy of the pretrained TS estimator can be low; this is called an out-of-distribution (OOD) case. In this paper, we tackle this issue by fine-tuning only the second stage of TS, implemented as a multi-head neural network for its end-to-end differentiability, using the DT and the new observations. Our novel fine-tuning procedure first compares the compressed features from real data to those produced by the DT at the TS initial prediction, to determine whether an OOD case has occurred by applying a statistical test based on a whitened feature discrepancy measure constructed from estimated feature covariances. If an OOD case is detected, we: (i) compute Gauss–Newton (GN) iterations on the initial prediction by TS, (ii) generate a confidence set around the GN update and re-simulate a targeted local training dataset with the DT, and (iii) fine-tune the final portion of the second stage of the TS estimator, which is inspired from transfer learning [ 11 ] approaches to system identification [ 12 , 13 ] . Our work differs from modern simulation-based inference methods [ 14 , 15 , 16 ] , which aim to infer a full Bayesian posterior. In addition, within these methods, some neural posterior estimation methods use differentiable simulator gradients [ 14 ] , and others use truncated proposals for sequential refinement [ 15 ] . In contrast, our approach is a non-Bayesian scheme for a pre-trained point estimator, requiring no differentiable simulator. We perform a lightweight, final-layer update, which is conceptually closer to other transfer learning methods in system identification [ 12 , 13 ] and broader OOD adaptation strategies [ 17 , Ch. 19-20] . In particular, our contributions are the following: • We derive a feature-space OOD test based on DT-derived feature statistics and whitening; • we propose a label-free fine-tuning approach that combines Gauss–Newton iterations with a confidence set to generate a fresh synthetic dataset for the retraining of TS; • we design a minimal update of the second-stage of TS, adjusting only its final layers; and • we numerically validate our proposed approach against standard baselines. The remainder of the paper is organized as follows. In Section II , we introduce the problem statement. Section III describes the TS estimator. In Section IV , our method for fine-tuning the pretrained TS estimator is detailed, and in Section V , we demonstrate the effectiveness of our approach through several numerical examples. Section VI outlines the future work. Finally, Section VII concludes the paper. II Problem Statement We consider a data-generating mechanism \mathcal{M}(\boldsymbol{\theta}) , a high-fidelity simulator or DT, parameterized by \boldsymbol{\theta}\in\Theta\subseteq\mathbb{R}^{d} . For notational simplicity, we assume a single-input single-output (SISO) setting: given an N -length scalar input sequence \boldsymbol{u}=(u_{1},\dots,u_{N})^{\top}\in\mathbb{R}^{N} , the DT produces a (possibly stochastic) scalar output sequence \boldsymbol{y}=(y_{1},\ldots,y_{N})^{\top}\in\mathbb{R}^{N} . Concretely, \boldsymbol{y}=\boldsymbol{y}(\boldsymbol{\theta};\omega) , where \omega denotes the simulator’s noise seed (or any internal randomness). Assume that a length- N trajectory \boldsymbol{z}_{0}=(u_{1},y_{0,1},\ldots,u_{N},y_{0,N})\in\mathbb{R}^{2N} has been measured from the DT at some unknown parameter \boldsymbol{\theta}_{0}\in\Theta . The goal is to estimate \boldsymbol{\theta}_{0} from \boldsymbol{z}_{0} by constructing a simulation-driven parameter estimator . Given a calibration set \Theta_{p}\subseteq\Theta , constructed from prior domain knowledge and possible operating regimes, we draw independent and identically distributed (i.i.d.) parameter settings \{\tilde{\boldsymbol{\theta}}_{i}\}_{i=1}^{m} from \Theta_{p} . For each draw \tilde{\boldsymbol{\theta}}_{i} , we run the simulator with (possibly) its own noise seed \omega_{i} and the chosen input sequence 1 1 1 Due to space limitations, in this paper the input sequence is considered fixed, even though the treatment can be extended to random inputs. \boldsymbol{u} , obtaining \boldsymbol{z}_{i}=(u_{1},y_{i,1},\ldots,u_{N},y_{i,N}) . This yields a synthetic, labeled dataset \mathcal{D}=\{(\boldsymbol{z}_{i},\tilde{\boldsymbol{\theta}}_{i})\}_{i=1}^{m} . An estimator \hat{f}_{\mathrm{pre}}\colon\mathbb{R}^{2N}\!\to\mathbb{R}^{d} is then fitted by solving the following minimization problem over a prescribed function class \mathcal{F} (e.g. a neural network): \hat{f}_{\mathrm{pre}}\in\arg\min_{F\in\mathcal{F}}\frac{1}{m}\sum_{i=1}^{m}L\!\left(F(\boldsymbol{z}_{i}),\,\tilde{\boldsymbol{\theta}}_{i}\right), (1) where L\colon\mathbb{R}^{d}\times\mathbb{R}^{d}\to\mathbb{R}_{\geq 0} is a suitable loss function. Following Eq. ( 1 ), an estimate of \boldsymbol{\theta}_{0} is obtained as \hat{\boldsymbol{\theta}}_{0}=\hat{f}_{\mathrm{pre}}(\boldsymbol{z}_{0}), (2) where \boldsymbol{z}_{0} are the observations (input-output data) collected from the system \mathcal{M}(\boldsymbol{\theta}_{0}) . When \boldsymbol{\theta}_{0}\in\Theta_{p} , suitable choices of \mathcal{F} and L can lead to an accurate estimate \hat{f}_{\mathrm{pre}}(\boldsymbol{z}_{0}) of \boldsymbol{\theta}_{0} . However, because \Theta_{p} is user-specified and may omit plausible regimes, an out-of-distribution (OOD) situation may occur if \boldsymbol{\theta}_{0}\notin\Theta_{p} , or if there are not enough samples \tilde{\boldsymbol{\theta}}_{i} in the neighborhood of \boldsymbol{\theta}_{0} . Hence, performance can degrade under an OOD, motivating mechanisms that refine \hat{f}_{\mathrm{pre}} at implementation phase. In this paper, we focus on a special class of simulation-driven estimators \hat{f}_{\mathrm{pre}} , known as the TS estimator. III The TS Estimator The TS estimator [ 6 ] decomposes ( 1 ) into two-stages: a fixed compression of \boldsymbol{z} into low-dimensional features, followed by a supervised machine learning model that map these features to the parameters. III-A Stage 1 (Compression) Choose a deterministic feature map h\colon\mathbb{R}^{2N}\to\mathbb{R}^{n},\qquad n\ll 2N, (3) e.g., the estimated coefficients of an AR/ARX/ARMAX model fitted to the data. Let \boldsymbol{x}_{i}=h(\boldsymbol{z}_{i}) and form the synthetic dataset \mathcal{D}_{\mathrm{tr}}=\{(\boldsymbol{x}_{i},\tilde{\boldsymbol{\theta}}_{i})\}_{i=1}^{m}=\{(h(\boldsymbol{z}_{i}),\tilde{\boldsymbol{\theta}}_{i})\}_{i=1}^{m}\subset\mathbb{R}^{n}\times\mathbb{R}^{d}. (4) III-B Stage 2 (Supervised mapping) Let \boldsymbol{x}=h(\mathbf{z})\in\mathbb{R}^{n} be the Stage-1 feature vector. The Stage-2 function g_{\mathrm{pre}}\colon\mathbb{R}^{n}\to\mathbb{R}^{d} is a fully connected deep neural network with trunk and head layers. III-B 1 Trunk Layers The trunk of the second stage neural network is mathematically described as follows: Let \boldsymbol{c}^{(0)}=\boldsymbol{x} be the input layer and for r=1,\dots,R-1 , the output of the r^{\mathrm{th}} hidden layer is given by \boldsymbol{c}^{(r)}\;=\;\sigma_{r}\!\big(\boldsymbol{W}_{r}\,\boldsymbol{c}^{(r-1)}\big),\,\,\boldsymbol{W}_{r}\in\mathbb{R}^{p_{r}\times p_{r-1}}, where \boldsymbol{W}_{r} denotes the weights of the hidden layer r , \sigma_{r} denotes an activation function (e.g. ReLU( \cdot )), p_{0}=n and p_{R-1}=p . These R layers constitute the trunk of g_{\mathrm{pre}} . The trunk output is \boldsymbol{t}(\boldsymbol{x})\coloneqq\boldsymbol{c}^{(R-1)}\in\mathbb{R}^{p} and its parameters are \boldsymbol{\phi}\coloneqq\{\boldsymbol{W}_{r}\}_{r=1}^{R-1}. III-B 2 Head Layers To finally predict the targets \tilde{\boldsymbol{\theta}}_{i}\in\Theta_{p}\subseteq\mathbb{R}^{d} , which can be multidimensional ( d>1 ), we append V-1 subsequent hidden layers for each parameter dimension, and these V-1 layers corresponding to each individual parameter constitute the head of g_{\mathrm{pre}} , denoted as \boldsymbol{h}_{j}^{(v)} , where v=1,\ldots,V-1 . We additionally impose that the weights of the first of these V-1 layers are least correlated, which is needed to ensure that each parameter is correctly identified. We precisely formalize the structure of the head as the following: For each parameter index j\in\{1,\dots,d\} , starting from \boldsymbol{h}_{j}^{(0)}=\boldsymbol{t}(\boldsymbol{x}) and for v=1,\dots,V-1 define \boldsymbol{h}_{j}^{(v)}\;=\;\sigma_{j,v}\!\big(\boldsymbol{U}_{j,v}\,\boldsymbol{h}_{j}^{(v-1)}\big),\,\boldsymbol{U}_{j,v}\in\mathbb{R}^{q_{j,v}\times q_{j,v-1}}, where \boldsymbol{U}_{j,v} corresponds to the weights of hidden layer v of head j , and \sigma_{j,v} corresponds to the activation function of v^{\mathrm{th}} hidden layer of head j , with q_{j,0}=p . The scalar output for head j is linear: \widehat{{\theta}}_{j}(\boldsymbol{x})\;=\;\boldsymbol{a}_{j}^{\top}\boldsymbol{h}_{j}^{(V-1)},\qquad\boldsymbol{a}_{j}\in\mathbb{R}^{q_{j,V-1}}. The head parameters are then defined as \boldsymbol{\psi}_{j}~\coloneqq\{(\boldsymbol{U}_{j,v})_{v=1}^{V-1},\boldsymbol{a}_{j}\}, and \boldsymbol{\Psi}\coloneqq(\psi_{1},\dots,\psi_{d}) . Stacking the d heads yields the second stage output as g_{\mathrm{pre}}(\boldsymbol{x};\boldsymbol{\phi},\boldsymbol{\Psi})\coloneqq\begin{bmatrix}\widehat{\theta}_{1}(\boldsymbol{x})\\[-2.84526pt] \vdots\\[-2.84526pt] \widehat{\theta}_{d}(\boldsymbol{x})\end{bmatrix}\in\mathbb{R}^{d},\,\big[g_{\mathrm{pre}}(\boldsymbol{x};\boldsymbol{\phi},\boldsymbol{\Psi})\big]_{j}\coloneqq\widehat{\theta}_{j}(\boldsymbol{x}). (5) Given the training dataset \mathcal{D}_{\mathrm{tr}}=\{(\boldsymbol{x}_{i},\tilde{\boldsymbol{\theta}}_{i})\}_{i=1}^{m} , we learn (\boldsymbol{\phi},\boldsymbol{\Psi}) by solving the following optimization problem via stochastic gradient descent: \begin{split}(\hat{\boldsymbol{\phi}},\hat{\boldsymbol{\Psi}})\in\arg\min_{\boldsymbol{\phi},\boldsymbol{\Psi}}\;&\frac{1}{m}\sum_{i=1}^{m}\sum_{j=1}^{d}\tilde{w}_{j}\,l\Big(\widehat{\theta}_{j}(\boldsymbol{x}_{i}),\,[\tilde{\boldsymbol{\theta}}_{i}]_{j}\Big)\\ &+\lambda_{\mathrm{orth}}\!\!\sum_{1\leq j0,\end{split} (8) where \boldsymbol{H}_{k:}=(\boldsymbol{x}^{(k)})^{\top} , and \boldsymbol{I}_{n} is an n\times n identity matrix. The covariance matrix is then used to define a whitening transform, \boldsymbol{S}^{-1/2} , which normalizes the feature space by accounting for correlations and different scales among the features. The test statistic for our observed data is the whitened squared residual norm: s_{\mathrm{obs}}=\big\|\boldsymbol{S}^{-1/2}\big(\boldsymbol{x}_{\mathrm{obs}}-\boldsymbol{\mu}\big)\big\|_{2}^{2}, (9) Finally, we establish a critical value for this statistic using the bootstrap principle. We compute the same whitened statistic for each of the K simulated feature vectors: s_{k}=\big\|\boldsymbol{S}^{-1/2}\big(\boldsymbol{H}_{k:}^{\top}-\boldsymbol{\mu}\big)\big\|_{2}^{2}. (10) The set \{s_{k}\}_{k=1}^{K} provides an empirical distribution of the test statistic under H_{0} . We define our decision threshold, q_{1-\alpha} , as the empirical (1-\alpha) -quantile of this distribution. Decision rule: If s_{\mathrm{obs}}>q_{1-\alpha} , we reject the null hypothesis H_{0} , conclude that an OOD scenario is likely, and proceed to Step 2. Otherwise, the observation is deemed statistically consistent with the variability at \hat{\boldsymbol{\theta}}_{\mathrm{init}} , and we retain the pretrained estimator without fine-tuning. IV-B Step 2: Synthetic data generation If s_{\mathrm{obs}}>q_{1-\alpha} , then the compressed features from the first-stage of TS evaluated at \hat{\boldsymbol{\theta}}_{\mathrm{init}} fail to explain \boldsymbol{x}_{\mathrm{obs}} in the whitened space. In that case, we refine \hat{\boldsymbol{\theta}}_{\mathrm{init}} by running GN iterations that minimize the whitened feature residual between the observed feature and the first-stage generated feature. Because this residual is random, the GN estimate, denoted \hat{\boldsymbol{\theta}}_{\mathrm{GN}} , still carries parameter uncertainty. To effectively fine-tune the second stage of the TS estimator, we need to generate a cloud of synthetic data that covers the region of plausible parameter values around this estimate. We shall now describe precisely the synthetic dataset generation for fine-tuning the second stage of TS as follows. GN update We compute the GN update starting from the TS initial estimate \hat{\boldsymbol{\theta}}_{\mathrm{init}} . We define a seed-averaged simulator feature map \bar{\boldsymbol{f}}(\boldsymbol{\theta}) and its corresponding whitened residual \boldsymbol{r}(\boldsymbol{\theta}) : \displaystyle\bar{\boldsymbol{f}}(\boldsymbol{\theta}) \displaystyle:=\frac{1}{K_{\mathrm{avg}}}\sum_{\omega\in\mathcal{S}_{\mathrm{avg}}}{h}\!\big(\boldsymbol{z}(\boldsymbol{\theta},\omega;\boldsymbol{u})\big), (11) \displaystyle\boldsymbol{r}(\boldsymbol{\theta}) \displaystyle:=\boldsymbol{S}^{-1/2}\big(\bar{\boldsymbol{f}}(\boldsymbol{\theta})-\boldsymbol{x}_{\mathrm{obs}}\big)\in\mathbb{R}^{n}, where \boldsymbol{z}(\boldsymbol{\theta},\omega;\boldsymbol{u}) is the simulator trajectory for parameter \boldsymbol{\theta} and seed \omega . We then minimize the following regularized objective function using iterative GN steps (with a backtracking line search): v(\boldsymbol{\theta})=\tfrac{1}{2}\|\boldsymbol{r}(\boldsymbol{\theta})\|_{2}^{2}+\tfrac{\gamma}{2}\|\boldsymbol{\theta}-\hat{\boldsymbol{\theta}}_{\mathrm{init}}\|_{2}^{2},\qquad\gamma>0, (12) A few iterations of minimizing this objective yield the refined estimate \hat{\boldsymbol{\theta}}_{\mathrm{GN}} . Let \partial\bar{\boldsymbol{f}}(\boldsymbol{\theta})/\partial\boldsymbol{\theta}\in\mathbb{R}^{n\times d} denote the Jacobian of the seed-averaged feature map, and let \boldsymbol{S}(\boldsymbol{\theta}) be the feature covariance estimated from simulator draws at parameter \boldsymbol{\theta} , analogous to Eq. ( 8 ). Define the whitened Jacobian \boldsymbol{J}\;\coloneqq\;\boldsymbol{S}^{-1/2}\,\partial\bar{\boldsymbol{f}}(\boldsymbol{\theta})/\partial\boldsymbol{\theta}. We re-compute \boldsymbol{\mu},\boldsymbol{S} and \boldsymbol{J} at \hat{\boldsymbol{\theta}}_{\mathrm{GN}} , and set \boldsymbol{G}\coloneqq\boldsymbol{J}^{\top}\boldsymbol{J}+\lambda\boldsymbol{I}_{d}. Sampling procedure \hat{\boldsymbol{\theta}}_{\mathrm{GN}} is a point estimate derived from a single, noisy observation and is therefore uncertain. To effectively fine-tune the TS estimator, we must generate a cloud of synthetic data that explores the region of plausible parameter values around this estimate. We employ a hybrid sampling strategy that combines a confidence-ellipsoid based method with an alternative sampling scheme. Our primary approach is to sample from the approximate 1-\beta confidence ellipsoid for the true parameter \boldsymbol{\theta}_{0} . This region is defined by the set of all \boldsymbol{\theta} satisfying: (\boldsymbol{\theta}-\hat{\boldsymbol{\theta}}_{\mathrm{GN}})^{\top}\boldsymbol{G}(\boldsymbol{\theta}-\hat{\boldsymbol{\theta}}_{\mathrm{GN}})\leq\chi^{2}_{d}(\beta), (13) where \chi^{2}_{d}(\beta) is the chi-squared quantile and \boldsymbol{G} is the approximate Fisher Information Matrix (FIM). This ellipsoid represents a high-confidence region, and sampling from it is statistically efficient for exploring well-identified parameters. However, this primary method can fail if the model is insensitive to one or more parameters. In such cases, the FIM, \boldsymbol{G} , becomes ill-conditioned, causing the confidence ellipsoid to be stretched along the insensitive directions, and thereby making the sampling procedure numerically unstable. To overcome this, we alternate between this sampling approach and sensitivity sampling. The latter sampling procedure handles the ill-conditioned directions. It works by generating samples along individual parameter axis with a step size that is inversely proportional to the simulator’s sensitivity to that parameter. The sensitivity of coordinate j is defined as s^{\prime}_{j}:=\|\boldsymbol{J}\boldsymbol{e}_{j}\|_{2} . To set the sampling scale, we define two quantities. First, \Delta_{\mathrm{misfit}}:=\|\boldsymbol{r}(\hat{\boldsymbol{\theta}}_{\mathrm{GN}})\|_{2} , measures the discrepancy between the observed summary statistic and the one generated by the simulator at the point estimate \hat{\boldsymbol{\theta}}_{\mathrm{GN}} . Second, we compute d_{\mathrm{med}}:=\mathrm{median}_{k}\,d_{k} , based on the spread of pre-computed summary features, where each d_{k}:=\|\boldsymbol{S}^{-1/2}(\boldsymbol{H}_{k:}-\boldsymbol{\mu})\|_{2} represents the whitened residual. We then set coordinate scales, r_{j} , that increase exploration for less sensitive parameters (small s^{\prime}_{j} ) while also adapting to the simulator’s noise level and the current feature mismatch as r_{j}\;=\;\frac{d_{\mathrm{med}}}{s^{\prime}_{j}}\,\Delta_{\mathrm{misfit}}. Our sampling procedure combines confidence ellipsoid and sensitivity sampling, yielding a hybrid sampling scheme, given by \bar{\boldsymbol{\theta}}_{k}=\hat{\boldsymbol{\theta}}_{\mathrm{GN}}+\boldsymbol{\Delta\theta}_{k} , where \boldsymbol{\Delta\theta}_{k} is drawn as \displaystyle\text{(confidence set sampling)}\quad\boldsymbol{\Delta\theta}_{k}=\boldsymbol{B}\,\boldsymbol{z},\;\;\boldsymbol{z}\sim\mathcal{N}(\boldsymbol{0},\boldsymbol{I}_{d}),\;\; \displaystyle\quad\quad\qquad\qquad\qquad\qquad\qquad\qquad\quad\boldsymbol{B}\boldsymbol{B}^{\top}\approx\boldsymbol{G}^{-1}, \displaystyle\text{(sensitivity sampling)}\,\text{pick }i\sim\mathrm{Unif}\{1,\ldots,d\},\;\zeta\sim\mathcal{N}(0,r_{i}^{2}), \displaystyle\quad\qquad\qquad\qquad\qquad\,\,\quad\quad\boldsymbol{\Delta\theta}_{k}=\zeta\,\boldsymbol{e}_{i}, (14) where \boldsymbol{e}_{i}\in\mathbb{R}^{d} is the i -th standard basis vector ( 1 at position i , zeros elsewhere), and \boldsymbol{B} is the Cholesky factor of \boldsymbol{G}^{-1} . To create the fine-tuning dataset, \mathcal{D}_{\mathrm{fine}} , we generate a synthetic observation \bar{\boldsymbol{z}}_{k} from the simulator for each parameter sample \bar{\boldsymbol{\theta}}_{k} . Each observation is then compressed using h , forming a dataset of pairs: \mathcal{D}_{\mathrm{fine}}=\{(h(\boldsymbol{\bar{z}}_{k}),\bar{\boldsymbol{\theta}}_{k})\}_{k=1}^{M_{\mathrm{FT}}} , where M_{\mathrm{FT}} is the number of new synthetic samples. Remark IV.1 The quantity \Delta_{\mathrm{misfit}} can be artificially small due to noise induced by the randomness of the simulator \mathcal{M} , so the naive radius \Delta_{\mathrm{misfit}}/s^{\prime}_{j} would under–explore. Multiplying by d_{\mathrm{med}} rescales the step to the typical feature fluctuation at \hat{\boldsymbol{\theta}}_{\mathrm{init}} , giving r_{j}=(d_{\mathrm{med}}/s^{\prime}_{j})\,\Delta_{\mathrm{misfit}} . This preserves the desired inverse–sensitivity ( 1/s^{\prime}_{j} ) while making the samples noise–aware. Remark IV.2 (Theoretical justification of Steps 1 and 2) For Step 1, under the null hypothesis (in-distribution), the observed features \boldsymbol{x}_{\mathrm{obs}} and the simulated features \boldsymbol{x}^{(k)} are asymptotically normal with the same mean and covariance. This implies that their whitened residuals ( s_{\mathrm{obs}} and s_{k} ) converge to the same \chi^{2} distribution, validating the bootstrap test. For Step 2, the GN objective v(\boldsymbol{\theta}) is an M-estimator whose expectation is minimized at the true \boldsymbol{\theta}_{0} . Therefore, the GN estimate \hat{\boldsymbol{\theta}}_{\mathrm{GN}} is consistent [ 23 , Ch. 5] , and the FIM \boldsymbol{G} , as a consistent estimator of the asymptotic covariance, defines an asymptotically valid confidence ellipsoid for sampling. IV-C Step 3: Final–Layer Transfer on a Targeted Set Finally, we freeze the early layers of g_{\mathrm{pre}}(\cdot;\boldsymbol{\phi},\boldsymbol{\Psi}) (i.e., keep \boldsymbol{\phi} fixed) and modify only the last layers, yielding g_{\mathrm{ft}}(\cdot;\boldsymbol{\phi},\boldsymbol{\Psi}_{\mathrm{ft}}) . We fit \boldsymbol{\Psi} on \mathcal{D}_{\mathrm{fine}} with a small number of gradient steps using a weighted objective (cf. ( 6 )): \displaystyle\hat{\boldsymbol{\Psi}}\in\arg\min_{\boldsymbol{\Psi}}\;\frac{1}{M_{\mathrm{FT}}}\sum_{i=1}^{M_{\mathrm{FT}}} \displaystyle\sum_{j=1}^{d}\tilde{w}_{j}\ell([g_{\mathrm{pre}}(h(\bar{\boldsymbol{z}}_{i});\boldsymbol{\phi},\boldsymbol{\Psi})]_{j},[\bar{\boldsymbol{\theta}}_{i}]_{j}) (15) \displaystyle+\lambda_{\mathrm{orth}}\!\!\sum_{1\leq j