APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation

Javier Marín [email protected] · 2025

Read APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation by Javier Marín [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-19912.txt.

Introduction

Javier Marín fine tunes language models by try and keep in a solo 2025 paper. Standard tuning follows one slope downhill, which can shake a model's wider skills. His method, Adjacent Possible Exploration, borrows from evolution. It tries many small updates on data slices and keeps only those that clear a set bar. The rest are dropped before they can harm the base.

On news summary tasks the kept updates lifted the BLEU score by about a third and cut perplexity by over a third as well. The gains came on a lean compute budget, since most trials are small and short. The author pitches the filter as a practical way to adapt models with control, trading a bit of search for much steadier ground.

Catalog record

CREATORJavier Marín [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE15 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-19912

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation Javier Marín [email protected] Abstract We present Adjacent Possible Exploration (APE), a selective fine-tuning method for adapting large language models that systematically explores parameter modifications while maintaining model stability. Inspired by evolutionary optimization principles, APE evaluates multiple candidate parameter updates through fine-tuning on small data subsets and accepts only those exceeding a performance threshold. Unlike standard fine-tuning that follows single gradient directions, APE implements a filtered selection process that prevents destabilizing parameter changes while enabling systematic improvement. Our method achieves 33.9% BLEU improvement and 36.2% perplexity reduction on news summarization tasks while using minimal computational resources. The approach provides a practical framework for controlled model adaptation that balances performance gains with representational stability. 1 Introduction Adapting large pre-trained language models to specific tasks requires balancing performance improvement with preservation of learned capabilities. Standard fine-tuning approaches optimize a single objective function through gradient descent, often leading to catastrophic forgetting [ 16 ] or instability in learned representations. Parameter-efficient methods like LoRA [ 11 ] constrain modifications to low-dimensional subspaces but limit adaptation scope. We propose Adjacent Possible Exploration (APE), a selective fine-tuning approach that explores multiple parameter modification directions while implementing acceptance criteria to maintain model stability. The method draws conceptual inspiration from evolutionary optimization principles, particularly the biological constraint that viable changes must preserve essential system properties while enabling incremental improvement. APE operates by generating multiple candidate parameter updates through fine-tuning on randomly sampled data subsets, then selecting only those updates that exceed a performance improvement threshold. This creates a filtered optimization process that systematically explores beneficial parameter modifications while rejecting changes that fall within noise levels or potentially destabilize learned representations. Our key contributions include: (1) A practical algorithm for selective fine-tuning that balances exploration and stability, (2) Empirical validation showing superior performance compared to standard adaptation methods, and (3) Analysis of why selective acceptance of parameter modifications leads to more robust model adaptation. The approach demonstrates that systematic exploration of parameter space through filtered selection can achieve better adaptation results than unconstrained optimization, providing a principled framework for controlled model modification that maintains stability while enabling significant performance improvements. 2 Method 2.1 Core Algorithm APE implements selective fine-tuning through systematic exploration of parameter modifications with acceptance criteria. At each iteration, the method generates a candidate parameter update by fine-tuning on a small randomly sampled data subset, evaluates the resulting performance change, and accepts the update only if the improvement exceeds a predefined threshold. The mathematical formulation follows: \theta_{t+1}=\begin{cases}\theta_{candidate}&\text{if }F(\theta_{candidate})>F(\theta_{t})+\tau\\ \theta_{t}&\text{otherwise}\end{cases} (1) where \theta_{t} represents the current parameters, \theta_{candidate} is the candidate update generated through fine-tuning, F(\cdot) measures model performance, and \tau defines the acceptance threshold. Each candidate update is generated through standard fine-tuning on a subset \mathcal{D}_{\text{subset}}\subset\mathcal{D} with fixed size |\mathcal{D}_{\text{subset}}|=200 : \theta_{candidate}=\theta_{t}-\eta\nabla_{\theta}\mathcal{L}(\theta_{t};\mathcal{D}_{\text{subset}}) (2) The subset size balances providing sufficient signal for meaningful parameter updates while maintaining the locality of modifications. The small subset ensures that individual fine-tuning steps create bounded parameter changes, preventing dramatic modifications that might destabilize learned representations. Algorithm 1 Adjacent Possible Exploration (APE) 1: Input : Pre-trained model \theta_{0} , training dataset \mathcal{D} , subset size k=200 , acceptance threshold \tau , maximum iterations T 2: Output : Adapted model \theta_{T} 3: Initialize \theta^{*}\leftarrow\theta_{0} and F^{*}\leftarrow F(\theta_{0}) 4: for t=1 to T do 5: Sample subset \mathcal{D}_{t}\subset\mathcal{D} with |\mathcal{D}_{t}|=k 6: Fine-tune: \theta_{candidate}\leftarrow\theta^{*}-\eta\nabla_{\theta}\mathcal{L}(\theta^{*};\mathcal{D}_{t}) 7: Evaluate: F_{candidate}\leftarrow F(\theta_{candidate}) 8: Compute improvement: \Delta F\leftarrow F_{candidate}-F^{*} 9: if \Delta F>\tau then 10: Accept update: \theta^{*}\leftarrow\theta_{candidate} and F^{*}\leftarrow F_{candidate} 11: end if 12: end for 13: Return \theta^{*} 2.2 Design Rationale The selective acceptance mechanism addresses two key limitations of standard fine-tuning: susceptibility to noise in gradient estimates and potential destabilization of learned representations through large parameter modifications. Noise Filtering : Individual gradient steps computed on small data subsets contain significant noise. The acceptance threshold filters out spurious "improvements" that fall within measurement uncertainty, ensuring that only statistically significant performance gains are retained. Stability Preservation : By constraining each candidate update to small subset fine-tuning and requiring performance improvements for acceptance, APE naturally avoids parameter modifications that significantly alter learned representations without providing clear benefits. Exploration vs. Exploitation : The random subset sampling explores diverse parameter modification directions, while the acceptance criteria ensure that exploration is guided toward beneficial changes rather than random wandering. 2.3 Comparison with Related Methods APE differs from standard optimization approaches in implementing discrete selection among candidate updates rather than continuous gradient flow. This creates fundamentally different dynamics: Standard Fine-tuning follows the gradient direction for the entire dataset: \theta_{t+1}=\theta_{t}-\eta\nabla_{\theta}\mathcal{L}(\theta_{t};\mathcal{D}) (3) This approach optimizes rapidly but may lead to configurations that destabilize learned representations or overfit to specific aspects of the training data. Parameter-Efficient Methods like LoRA constrain modifications to low-rank subspaces, limiting the scope of possible adaptations while reducing computational requirements. APE explores full parameter space through multiple candidate directions while implementing selection criteria that maintain stability. This provides broader adaptation capability than parameter-efficient methods while offering better stability than unconstrained fine-tuning. 3 Experimental Validation 3.1 Experimental Setup We evaluate APE on the CNN/DailyMail news summarization dataset [ 9 ] using T5-base [ 19 ] as the base model. The evaluation uses multiple metrics to assess different aspects of generation quality: BLEU [ 18 ] for n-gram precision, ROUGE-1 [ 15 ] for lexical overlap, BERTScore [ 20 ] for semantic similarity, and perplexity for fluency. Experimental parameters: subset size k=200 , learning rate \eta=3\times 10^{-6} , acceptance threshold \tau corresponding to 2% relative improvement, maximum iterations T=20 . The threshold value was selected based on preliminary experiments to balance acceptance rate with noise filtering. 3.2 Quantitative Results Table 1 presents comprehensive performance comparison across metrics. Table 1: Performance comparison on CNN/DailyMail summarization task. APE results after 17 iterations with 4,000 training samples. Metric Baseline APE Result Improvement BLEU 0.062\pm 0.083 0.083\pm 0.100 +33.9\% ROUGE-1 0.290\pm 0.121 0.329\pm 0.128 +13.4\% BERTScore 0.343\pm 0.142 0.398\pm 0.135 +16.0\% Perplexity 13.0\pm 12.0 8.3\pm 7.3 -36.2\% The results demonstrate substantial improvements across all metrics. The 33.9% BLEU improvement indicates enhanced n-gram precision, while the 36.2% perplexity reduction suggests improved fluency and coherence. Consistent improvements across diverse metrics indicate that APE enhances fundamental generation capabilities rather than optimizing specific targets. Figure 1 shows performance evolution across iterations, revealing characteristic patterns: rapid initial improvement as easily accessible beneficial modifications are discovered, followed by plateau behavior as the method approaches local optimization limits. Figure 1: Performance evolution across APE iterations showing normalized metrics. The pattern demonstrates rapid initial improvement followed by stabilization around optimal configurations. 3.3 Comparison with Alternative Methods Table 2 compares APE with established adaptation approaches. Table 2: Comparison across adaptation methods on CNN/DailyMail summarization. Method BLEU ROUGE-1 BERTScore Perplexity Parameters T5-base (baseline) 0.062 0.290 0.343 13.0 220M LoRA adaptation 0.080 0.320 0.385 9.0 \sim 0.5M Adapter insertion 0.078 0.315 0.380 9.5 \sim 1M APE (proposed) 0.083 0.329 0.398 8.3 220M APE achieves superior performance across all metrics while maintaining full parameter accessibility. Parameter-efficient methods constrain optimization to reduced subspaces, limiting adaptation scope. APE provides efficiency through intelligent exploration rather than dimensional reduction. 3.4 Human Evaluation Table 3 presents human evaluation results across quality dimensions. Table 3: Human evaluation results (100 articles, 7 evaluators, 700 total ratings on 1-5 scale). Quality

Read the complete text