Diversity-Aware Adaptive Collocation for Physics-Informed Neural Networks via Sparse QUBO Optimization and Hybrid Coresets
Hadi Salloum, Maximilian Mifsud Bonici, Sinan Ibrahim, Pavel Osinenko, Alexei Kornaev · 2026
Read Diversity-Aware Adaptive Collocation for Physics-Informed Neural Networks via Sparse QUBO Optimization and Hybrid Coresets by Hadi Salloum, Maximilian Mifsud Bonici, Sinan Ibrahim, Pavel Osinenko, Alexei Kornaev (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-06761.txt.
Introduction
Salloum and colleagues rethink collocation choice for physics-informed neural networks as coreset construction. Uniform sampling and residual-based refinement can oversample smooth regions and waste training time. From a large candidate pool they select a fixed-size subset that is both informative for PDE error and diverse under a space-time similarity notion. They frame this as a sparse graph-based QUBO with a repair step enforcing the budget. Hybrid coverage anchors guarantee enforcement, and tests on viscous Burgers flow match or beat variants with less overhead.
Catalog record
| CREATOR | Hadi Salloum, Maximilian Mifsud Bonici, Sinan Ibrahim, Pavel Osinenko, Alexei Kornaev |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26677 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.06761
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Diversity-Aware Adaptive Collocation for Physics-Informed Neural Networks via Sparse QUBO Optimization and Hybrid Coresets Hadi Salloum, Maximilian Mifsud Bonici, Sinan Ibrahim, Pavel Osinenko, Alexei Kornaev Abstract Physics-Informed Neural Networks (PINNs) enforce governing equations by penalizing PDE residuals at interior collocation points, but standard collocation strategies-uniform sampling and residual-based adaptive refinement-can oversample smooth regions, produce highly correlated point sets, and incur unnecessary training cost. We reinterpret collocation selection as a coreset construction problem: from a large candidate pool, select a fixed-size subset that is simultaneously informative (high expected impact on reducing PDE error) and diverse (low redundancy under a space–time similarity notion). We formulate this as a QUBO/BQM objective with linear terms encoding residual-based importance and quadratic terms discouraging redundant selections. To avoid the scalability issues of dense k-hot QUBOs, we propose a sparse graph-based BQM built on a kNN similarity graph and an efficient repair procedure that enforces an exact collocation budget. We further introduce hybrid coverage anchors to guarantee global PDE enforcement. We evaluate on the 1D time-dependent viscous Burgers’ equation with shock formation and report both accuracy and end-to-end time-to-accuracy, including a timing breakdown of selection overhead. Results demonstrate that sparse and hybrid formulations reduce selection overhead relative to dense QUBOs while matching or improving accuracy at fixed collocation budgets. 1 Introduction Physics-Informed Neural Networks (PINNs) embed differential equation constraints into training objectives, offering a flexible framework for solving forward and inverse problems governed by partial differential equations (PDEs) ( Raissi et al. 2019 ; Karniadakis et al. 2021 ) . Despite strong promise, practical PINN training often hinges on collocation strategy: which interior points should be used to evaluate and penalize the PDE residual ( Wang et al. 2021a ; Wang et al. 2022 ) . Uniform sampling is simple but frequently inefficient, especially in problems with localized structures (e.g., shocks, boundary layers, sharp fronts), where informative regions occupy a small fraction of the domain. Residual-based adaptive refinement ( Lu et al. 2021 ; Wu et al. 2023 ) improves focus but can still generate highly correlated point sets and can neglect global coverage, potentially harming stability and generalization. This work studies combinatorial, diversity-aware collocation selection. Instead of only repeatedly adding points at high-residual locations, we explicitly seek a compact subset of interior collocation points that maximizes informational value while minimizing redundancy. The core idea parallels coreset construction in machine learning ( Feldman 2020 ; Mirzasoleiman et al. 2020 ) : select fewer training constraints without sacrificing solution quality. Contributions. We present a QUBO/BQM-based framework for collocation selection and several practical variants designed for real PINN workloads: • QUBO formulation for collocation coresets: linear terms encode point importance (e.g., PDE residual magnitude) while quadratic terms penalize redundant pairs via similarity weights. • Sparse BQM + repair: we avoid dense cardinality penalties (which produce all-to-all couplers) by solving a sparse BQM on a kNN similarity graph and applying an efficient repair/refinement step to enforce an exact collocation budget. • Hybrid anchors for global coverage: we reserve a fraction of collocation points as stratified “coverage anchors” and use QUBO to select the remainder, preventing overconcentration on localized regions. 2 Related Work Physics-Informed Neural Networks. PINNs were popularized as a unified approach for forward and inverse problems by Raissi et al. ( Raissi et al. 2019 ) ; see Karniadakis et al. 2021 for a comprehensive review. Subsequent work revealed critical training pathologies: Wang et al. 2021a identified gradient flow imbalances across loss terms, Wang et al. 2022 analyzed convergence failures through the neural tangent kernel, and Krishnapriyan et al. 2021 characterized failure modes as PDE stiffness increases. Remedies include self-adaptive loss weighting ( McClenny & Braga-Neto 2023 ) , causal training schedules ( Wang et al. 2024 ) , and Fourier feature embeddings to address spectral bias ( Wang et al. 2021b ) . Adaptive collocation and residual refinement. Residual-based adaptive refinement (RAR) was introduced in DeepXDE ( Lu et al. 2021 ) and has since been extended in several directions. Wu et al. 2023 provided a systematic comparison of non-adaptive and residual-based adaptive sampling strategies across thousands of configurations, establishing RAR-D and RAD as strong baselines. Importance sampling approaches reweight or resample collocation points proportional to the PDE residual ( Nabian et al. 2021 ) , while deep generative models can learn residual-guided distributions ( Tang et al. 2023 ) . Failure-informed sampling ( Gao et al. 2023 ) casts adaptive collocation as a reliability problem, and the R3 method ( Daw et al. 2023 ) addresses propagation failures via evolutionary point retention. Most recently, PINNACLE ( Lau et al. 2024 ) jointly optimizes all training point types using NTK-based criteria with greedy selection. However, purely residual-driven methods may still produce redundant point clusters and may under-enforce the PDE globally. Coresets, diversity, and subset selection. Diversity-promoting selection appears across ML as facility location, k-center, determinantal point processes (DPPs), and submodular maximization. Coresets—compact weighted subsets that approximate full-data objectives—have been applied to efficient neural network training via submodular facility location ( Mirzasoleiman et al. 2020 ) and gradient matching ( Killamsetty et al. 2021 ) . The k-center formulation for active learning ( Sener & Savarese 2018 ) and submodular data selection ( Wei et al. 2015 ) provide the geometric diversity perspective that motivates our approach. DPPs offer a principled repulsive sampling framework ( Kulesza & Taskar 2012 ) , with fixed-size variants ( Kulesza & Taskar 2011 ) and applications to mini-batch diversification ( Zhang et al. 2017 ) . The classical greedy approximation guarantee for monotone submodular maximization ( Nemhauser et al. 1978 ) underpins many of these methods. Our approach focuses on QUBO/BQM formulations that naturally support both classical heuristics (simulated annealing) and quantum-ready solvers. Conceptual comparison to adaptive and diversity-aware selection. Many PINN adaptive collocation methods are sequential and importance-driven : they repeatedly add or resample interior points where the current PDE residual is large, which improves focus but can lead to clustered, highly correlated constraints Lu et al. 2021 ; Wu et al. 2023 ; Nabian et al. 2021 ; Gao et al. 2023 ; Tang et al. 2023 ; Daw et al. 2023 . In contrast, our method treats interior collocation as a fixed-budget subset selection problem: from a candidate pool we select exactly K points by minimizing a binary quadratic objective that rewards residual-based utility while penalizing space–time redundancy via pairwise similarity on a sparse k NN graph, followed by an exact- K repair step enforcing the cardinality constraint. Conceptually, this aligns with diversity-aware subset selection (coresets, submodular selection, and k -DPPs) Sener & Savarese 2018 ; Wei et al. 2015 ; Nemhauser et al. 1978 ; Krause et al. 2008 ; Kulesza & Taskar 2012 ; Kulesza & Taskar 2011 , but realizes the trade-off through an explicit QUBO/BQM energy minimized with annealing-style heuristics Kochenberger et al. 2014 ; Kirkpatrick et al. 1983 . QUBO optimization. QUBO is a standard form for combinatorial optimization ( Kochenberger et al. 2014 ) and is compatible with quantum annealing ( Kadowaki & Nishimori 1998 ; Johnson et al. 2011 ) and classical approximate solvers such as simulated annealing ( Kirkpatrick et al. 1983 ) . Practical QUBO modeling techniques are surveyed in Glover