Policy-DRIFT: Dynamic Reward-Informed Flow Trajectory Steering

Atharva MahajanDepartment of Aerospace EngineeringUniversity of Michigan, Ann Arbor [email protected], Abhijeet VishwasraoDepartment of Aerospace EngineeringUniversity of Michigan, Ann Arbor [email protected] Yuning WangDepartment of Aerospace EngineeringUniversity of Michigan, Ann Arbor [email protected], Ricardo VinuesaDepartment of Aerospace EngineeringUniversity of Michigan, Ann Arbor [email protected] · 2026

Read Policy-DRIFT: Dynamic Reward-Informed Flow Trajectory Steering by Atharva MahajanDepartment of Aerospace EngineeringUniversity of Michigan, Ann Arbor [email protected], Abhijeet VishwasraoDepartment of Aerospace EngineeringUniversity of Michigan, Ann Arbor [email protected] Yuning WangDepartment of Aerospace EngineeringUniversity of Michigan, Ann Arbor [email protected], Ricardo VinuesaDepartment of Aerospace EngineeringUniversity of Michigan, Ann Arbor [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-14022.txt.

Introduction

Mahajan and three co-authors tackle skin-friction drag from wall-bounded turbulent flows, a large share of energy use in planes, wind power and shipping. Deep reinforcement learning leads real-time flow control, yet its ceiling comes from the reward shape, since a plain scalar goal misses the physics. Policy-DRIFT moves reward cues from policy gradients into generative inference. A conditional flow matching model maps the space of feasible flow states, and terminal reward guidance steers samples toward good control regimes.

Catalog record

CREATORAtharva MahajanDepartment of Aerospace EngineeringUniversity of Michigan, Ann Arbor [email protected], Abhijeet VishwasraoDepartment of Aerospace EngineeringUniversity of Michigan, Ann Arbor [email protected] Yuning WangDepartment of Aerospace EngineeringUniversity of Michigan, Ann Arbor [email protected], Ricardo VinuesaDepartment of Aerospace EngineeringUniversity of Michigan, Ann Arbor [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47941
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.14022

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Policy-DRIFT: Dynamic Reward-Informed Flow Trajectory Steering Atharva MahajanDepartment of Aerospace EngineeringUniversity of Michigan, Ann Arbor [email protected], Abhijeet VishwasraoDepartment of Aerospace EngineeringUniversity of Michigan, Ann Arbor [email protected] Yuning WangDepartment of Aerospace EngineeringUniversity of Michigan, Ann Arbor [email protected], Ricardo VinuesaDepartment of Aerospace EngineeringUniversity of Michigan, Ann Arbor [email protected] Abstract Skin-friction drag induced by wall-bounded turbulent flows accounts for a substantial fraction of energy consumption across commercial aerospace, wind energy, and marine transport. Its active reduction is one of the highest-value targets in engineering fluid dynamics. Deep reinforcement learning (DRL) has emerged as the leading approach for real-time flow control, yet its performance ceiling is set not by algorithmic capability but by reward structure, the naive scalar objective does not optimally reflect the underlying physics. Policy-DRIFT bypasses this ceiling by relocating reward information from policy gradients to generative model inference: a conditional flow matching model (CFM) constructs a physically-grounded manifold of realisable flow states spanning multiple control regimes, Terminal Reward Guidance (TRG) steers samples toward reward-maximising targets at inference, and a lightweight DRL policy, structurally decoupled from reward quality, tracks these full-field targets via root-mean-squared error (RMSE) minimisation. The test case is turbulent channel flow simulated using direct numerical simulation (DNS) at friction Reynolds number of Re τ = 180 \mathrm{Re}_{\tau}=180 , which is the canonical benchmark for wall-bounded turbulence. Policy-DRIFT achieves 49 % 49\% drag reduction approaching the theoretical upper bound, which is ≈ 16 % \approx 16\% higher than the DRL benchmark, while consuming 37 × \times less actuation energy. Our approach combines generative methods with active flow control, marking a paradigm shift towards controlling complex physical systems efficiently. 1 Introduction Active control of complex physical systems is central to modern engineering: from stabilising plasma in tokamak reactors to regulating flow in pipeline networks and cardiovascular devices, the ability to manipulate dynamical processes in real time translates directly into performance gains and energy savings. Deep reinforcement learning (DRL) has emerged as the leading paradigm for this challenge, delivering state-of-the-art controllers across robotic manipulation ( 30 ) , molecular design ( 33 ) , plasma stabilisation ( 10 ) , and turbulent flow control ( 15 ; 3 ) . Wall-bounded turbulent flows represent a particularly high-value target: skin-friction drag accounts for roughly half of total aerodynamic drag on commercial aircraft ( 44 ) , and even a few percent reduction helps towards improving global fuel consumption efficiencies and CO_{2} emissions significantly. Recently, DRL controllers have demonstrated sustained drag reduction systematically outperforming classical heuristics such as opposition control ( 9 ) . Yet a fundamental limitation persists across DRL applications regardless of algorithmic advances, the reward signal remains a proxy of the true objective, and the policy ceiling is set entirely by how well this surrogate reflects what optimal actually looks like ( 39 ; 34 ) . In high-fidelity physical simulation, this is compounded by the prohibitive cost of sustained online DNS interaction during training, restricting policy improvement to what the proxy reward allows ( 12 ) . Prior remedies through inverse reward design ( 17 ) and reward-hacking mitigation ( 24 ) refine the proxy without eliminating the underlying problem: as long as reward gradients flow directly through the policy, learned behaviour remains fundamentally coupled to surrogate quality. Policy-DRIFT addresses reward misspecification by relocating where reward acts: not as a per-step gradient shaping the DRL policy, but as guidance over a generative model of the controlled system. A conditional flow matching (CFM) model trained across a spectrum of control strategies constructs a physically-grounded manifold of realisable flow states. Terminal Reward Guidance (TRG) then navigates this manifold at inference, steering ODE trajectories toward terminal states that maximise a designer-specified reward, fully decoupling the policy’s learning signal from reward quality. A lightweight DRL policy learns exclusively to track these generative targets via RMSE minimisation. Its learning signal is a full-field spatial objective, never a spatially averaged reward gradient, so that optimal behaviour emerges from target tracking rather than reward engineering. Crucially, the CFM model requires no retraining when the control objective changes. Instantiated with a cost-aware objective on turbulent channel flow DNS at a moderate friction Reynolds number of \mathrm{Re}_{\tau}=180 ( \mathrm{Re}_{\tau}=u_{\tau}h/\nu where u_{\tau} is the friction velocity, h is the channel height and \nu is the viscosity.), Policy-DRIFT outperforms all available baselines: it achieves 48.95\% drag reduction, being 16.2\% higher than vanilla DRL and 38.9\% higher than opposition control, while consuming approximately {\sim}37\times less actuation energy than DRL. Unlike standard DRL, which reacts to instantaneous scalar reward signals, Policy-DRIFT operates as a receding-horizon controller: at each control horizon, CFM with TRG computes the physically-realisable, reward-maximising target one horizon ahead; the DRL policy then executes 8 actuation steps to track it. Note that neither drag reduction nor energy savings are directly optimised by the policy, which emerge implicitly from target tracking. Our contributions are as follows: • Policy-DRIFT framework. We introduce Policy-DRIFT, a generative control framework that replaces naive DRL reward signals with physically-grounded target states via CFM trained jointly on uncontrolled, opposition-controlled, and DRL-controlled flow regimes, enabling flexible reward specification without reward gradients entering the policy. • Terminal Reward Guidance (TRG). We propose TRG, which steers CFM ODE trajectories toward reward-maximising terminal states via gradients of a learned reward predictor, extending inference-time guidance to full-field PDE-governed state spaces with a manifold-regularising pre-placement design that prevents reward hacking without constraining the reward objective. • Generative target generation for turbulence control. CFM combined with TRG provides physically-grounded, reward-maximising flow targets during training, decoupling target discovery from policy execution such that the deployed policy acts reactively from wall-parallel sensing alone, without generative model queries at inference time. • Empirical validation on DNS benchmark. On \mathrm{Re}_{\tau}=180 turbulent channel flow, Policy-DRIFT simultaneously achieves 16.2\% higher drag reduction than standard DRL and 38.9\% higher than opposition control, while using approximately {\sim}37\times less actuation energy than DRL, with both gains emerging without the policy directly optimising either quantity. 2 Related Work Turbulent flow control. 4 established that receding-horizon optimal control with full DNS access achieves over 50% drag reduction and complete flow relaminarisation in turbulent channel flow at \mathrm{Re}_{\tau}=180 , setting the physical upper bound for wall-transpiration control, but at the cost of iterative full-state DNS at every control step, identified by the authors as impossible to implement in practice. Deep reinforcement learning addressed this intractability by replacing per-step DNS with learned policies trained on custom reward signals, delivering state-of-the-art drag reduction under realistic partial-state sensing ( 15 ) . Yet sustained online interaction with high-fidelity DNS during training remains computationally expensive ( 12 ) , and the reliance on hand-crafted reward proxies introduces a compounding limitation: proxy objectives routinely diverge from the true control goal ( 39 ; 34 ) , with prior remedies through inverse reward design ( 17 ) and occupancy-measure regularisation ( 24 ) narrowing but not closing the gap. Generative models for physical systems. Score-based diffusion ( 40 ) established generative modelling over continuous high-dimensional state spaces. Subsequently, several applications confirmed that generative models can represent the complex statistics of turbulent PDE state spaces ( 26 ; 45 ) . Flow matching ( 28 ; 2 ; 29 ; 42 ) emerged as a simulation-free, ODE-based alternative, achieving comparable expressiveness at substantially lower

Read the complete text