Policy-Controlled Generalized Share: A General Framework with a Transformer Instantiation for Strictly Online Switching-Oracle Tracking
Hongkai Hu · 2026
Read Policy-Controlled Generalized Share: A General Framework with a Transformer Instantiation for Strictly Online Switching-Oracle Tracking by Hongkai Hu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-28198.txt.
Introduction
Hu studies online prediction when the best expert keeps changing over time. Chasing one fixed expert is then the wrong goal. His framework, Policy-Controlled Generalized Share, fixes the classic share recursion but lets the post-loss update controls adapt each round. A causal Transformer acts as that controller, seeing each loss only after the round ends. Hu proves regret bounds against switching expert paths. On synthetic tests and a household power benchmark, the method posted the lowest dynamic regret.
Catalog record
| CREATOR | Hongkai Hu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 133379 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.28198
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Policy-Controlled Generalized Share: A General Framework with a Transformer Instantiation for Strictly Online Switching-Oracle Tracking Hongkai Hu Abstract Static regret to a single expert is often the wrong target for strictly online prediction under non-stationarity, where the best expert may switch repeatedly over time. We study Policy-Controlled Generalized Share (PCGS), a general strictly online framework in which the generalized-share recursion is fixed while the post-loss update controls are allowed to vary adaptively. PCGS separates the analysis of the share backbone from the design of the controller. Its principal instantiation in this paper is PCGS-TF , which uses a causal Transformer as an update controller: after round t t finishes and the loss vector is observed, the Transformer outputs the controls that map w t w_{t} to w t + 1 w_{t+1} without altering the already committed decision w t w_{t} . Under admissible ℱ t \mathcal{F}_{t} -measurable update controls, we obtain a pathwise weighted regret guarantee for general time-varying learning rates, and a standard dynamic-regret guarantee against any expert path with at most S S switches under the constant-learning-rate specialization. In both cases, the switching term depends directly on ∑ t : π t + 1 ≠ π t − log q t ( π t + 1 ) \sum_{t:\,\pi_{t+1}\neq\pi_{t}}-\log q_{t}(\pi_{t+1}) , making restart allocation part of the certified difficulty of tracking a switching comparator. Empirically, all results labeled PCGS-TF correspond to this Transformer-controlled instantiation. On a controlled synthetic suite with exact DP switching-oracle evaluation, PCGS-TF attains the lowest mean dynamic regret in all seven non-stationary families and its advantage increases with larger expert pools. On a reproduced household-electricity benchmark, PCGS-TF also attains the lowest normalized dynamic regret for S ∈ { 5 , 10 , 20 } S\in\{5,10,20\} . K eywords strictly online learning; non-stationary time series; prediction with expert advice; switching oracle; dynamic regret; Fixed Share; Generalized Share; strictly causal update control; restart distribution; causal Transformer controller; in-context algorithm control; heavy tails and jumps; robust clipping; reproducibility 1 Introduction 1.1 Problem setting and metric We study prediction with expert advice under a strictly online protocol. At each round t\in\{1,\dots,T\} , the learner must commit to a probability distribution w_{t}\in\Delta_{K}\qquad\text{where}\qquad\Delta_{K}\triangleq\Big\{w\in\mathbb{R}_{+}^{K}:\sum_{k=1}^{K}w(k)=1\Big\}, before observing the current loss vector \ell_{t}=(\ell_{t,1},\dots,\ell_{t,K})\in[0,1]^{K}. The incurred loss is the mixed loss \langle w_{t},\ell_{t}\rangle=\sum_{k=1}^{K}w_{t}(k)\ell_{t,k}, and the learner may update its internal state only after the round has been completed. This temporal ordering is essential: it rules out any use of \ell_{t} in the construction of w_{t} and therefore enforces a genuine online, rather than one-step-delayed offline, evaluation regime. The classical benchmark in expert advice is static regret with respect to the single best expert in hindsight. That metric is appropriate when the environment is effectively stationary, or when one is willing to compare against a fixed action chosen after observing the entire sequence. In the present setting, however, the identity of the best expert may evolve over time: it may drift gradually, change abruptly at regime boundaries, or alternate repeatedly among different specialists. In such environments, comparison to a single static expert is often too coarse to capture the real difficulty of the task. A learner that tracks regime changes well may still exhibit large static regret simply because no single expert is uniformly best over the whole horizon. For this reason, the natural comparator class is not a fixed expert but a switching path of experts. Let \pi_{1:T}\in[K]^{T} denote a comparator path, where \pi_{t} is the expert chosen at time t , and let \#\mathrm{sw}(\pi)=\sum_{t=2}^{T}\mathbf{1}\{\pi_{t}\neq\pi_{t-1}\} be the number of switches in that path. For a switch budget S\in\{0,\dots,T-1\} , one considers the oracle value L_{T}^{\mathrm{sw}}(S)\triangleq\min_{\pi:\,\#\mathrm{sw}(\pi)\leq S}\sum_{t=1}^{T}\ell_{t,\pi_{t}}, and evaluates the learner through the corresponding dynamic regret \mathrm{DynRegret}_{S}\triangleq\sum_{t=1}^{T}\langle w_{t},\ell_{t}\rangle-L_{T}^{\mathrm{sw}}(S). This metric explicitly quantifies the price of adaptation relative to a comparator that is allowed a controlled amount of non-stationarity. It is therefore the right target when the main challenge is not prediction against a fixed latent regime, but rapid tracking of a sequence of changing best experts. 1.2 Limits of fixed restart schemes A canonical baseline for online expert aggregation is Hedge, or exponential weights, which is well known to enjoy optimal worst-case guarantees for static regret. Its appeal is undeniable: the update is simple, computationally efficient, and supported by a clean minimax theory. However, the geometry of multiplicative updates also induces a structural limitation in non-stationary environments. Once the weight assigned to an expert becomes very small, that expert can recover only gradually under subsequent multiplicative reweighting, even if it later becomes optimal. In other words, the posterior may become excessively committed to a regime that has already ended. Share-type algorithms were introduced precisely to address this recoverability problem. Rather than relying solely on multiplicative reweighting, they inject a portion of probability mass back into the expert pool after each round. In Fixed Share, the amount of restarted mass is constant and the restart distribution is typically uniform. Generalized Share extends this basic mechanism by allowing greater flexibility in how restart mass is redistributed. The conceptual lesson is that, in changing environments, good performance depends not only on identifying which experts performed well on the current round, but also on maintaining the capacity to reallocate mass efficiently when the underlying regime changes. This perspective immediately suggests that a fixed restart rule is unlikely to be uniformly adequate across heterogeneous non-stationary regimes. If the environment is locally stable, aggressive restart may introduce unnecessary diffusion and slow concentration. If the environment undergoes abrupt changes, insufficient restart can leave too much mass trapped on stale experts. Likewise, uniform restart destinations ignore potentially informative structure in the loss history: once evidence accumulates that certain experts are plausible “next winners,” it is suboptimal to spread all restart mass indiscriminately. Thus the main practical difficulty is not merely whether to restart, but how much to restart, when to restart, and where to send the restarted mass. These observations expose a gap between classical theory and practical algorithm design. Existing fixed or heuristic restart schemes capture the broad intuition that some form of forgetting is necessary, but they do not provide a principled way to adapt restart intensity and restart destination to the realized online history. Closing this gap is the main starting point of the present work. 1.3 Policy-Controlled Generalized Share and PCGS-TF We study Policy-Controlled Generalized Share (PCGS), a strictly online framework that retains the generalized-share backbone while allowing its update controls to be selected adaptively from observed history. The update takes the form \bar{w}_{t+1}(k)\propto w_{t}(k)\exp(-\eta_{t}\ell_{t,k}),\qquad w_{t+1}=(1-\rho_{t})\bar{w}_{t+1}+\rho_{t}q_{t}, where \eta_{t} is a learning-rate parameter, \rho_{t} is a restart intensity, and q_{t}\in\Delta_{K} is a restart distribution. The three quantities (\eta_{t},\rho_{t},q_{t}) constitute the post-loss update controls of the algorithm. The key structural distinction in this paper is between two levels of abstraction. At the general level, PCGS is the framework itself: it fixes the generalized-share recursion and allows the controls (\eta_{t},\rho_{t},q_{t}) to be chosen by any admissible strictly causal rule. At the algorithmic level, the principal instantiation we study is PCGS-TF , in which a causal Transformer serves as the controller that outputs these post-loss update controls. This distinction is not cosmetic. The framework level is the correct object for theory: it is the generalized-share recursion with history-dependent, strictly online-admissible controls that admits regret analysis.