Fractional Stochastic Neural Networks

Yuecai Han [email protected], Jianming Xu [email protected] · 2026

Read Fractional Stochastic Neural Networks by Yuecai Han [email protected], Jianming Xu [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-29438.txt.

Introduction

Han and Xu develop fractional stochastic neural networks, whose hidden dynamics follow residual links driven by fractional Brownian motion. That noise process has long memory, meaning distant past values still shape the present, unlike ordinary Brownian motion. Such memory suits some time series and noisy signals. The authors want training rules and guarantees that respect this fractional driver.

They present a discrete-time stochastic maximum principle for the network, which yields a matching adjoint recursion, a backward pass that computes gradients. For fixed parameters, they prove that projected samplewise stochastic gradient descent converges in mean square. Tests cover a closed-form convergence check, noisy regression with uncertainty estimates, and long memory time series generation. Image classification under structured noise is also tested. Results identify settings where the fractional driver improves long memory recovery or robustness over Brownian and plain baselines, rather than helping everywhere.

Catalog record

CREATORYuecai Han [email protected], Jianming Xu [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE68 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.29438

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Fractional Stochastic Neural Networks Yuecai Han [email protected], Jianming Xu [email protected] Abstract In this paper, we develop a fractional stochastic neural network with residual dynamics driven by fractional Brownian motion. By introducing a discrete stochastic maximum principle for the network, we construct the corresponding adjoint recursion. For deterministic network parameters, we prove mean square convergence of projected samplewise stochastic gradient descent. Numerical experiments include a closed form convergence test, noisy regression with uncertainty quantification, long memory time series generation and image classification under structured perturbations. The results identify settings in which fractional drivers improve long memory recovery or robustness relative to Brownian and deterministic baselines. Yuecai Han and Jianming Xu † † heading: 0 2026 1- 6/26 † † shortheadings: Fractional Stochastic Neural Networks / Han and Xu † † firstpage: 1 keywords stochastic neural networks, fractional Brownian motion, stochastic maximum principle, backward stochastic difference equations, stochastic gradient descent 1 Introduction Deep residual networks admit a dynamical reading in which the layer index plays the role of time and the layer map plays the role of a vector field. This perspective, popularised by the neural ordinary differential equations (ODEs) ( 5 ; 18 ; 27 ) , has been extended to stochastic dynamics through neural stochastic differential equations (SDEs) ( 25 ; 30 ; 24 ; 21 ) . Embedding a Brownian driver in the layer dynamics endows each prediction with an intrinsic distribution, and has been used as a principled source of epistemic uncertainty in deep classifiers ( 23 ; 2 ; 1 ; 8 ) . Training such a model amounts to a stochastic optimal control problem with a gradient that admits a backward equation representation, generalising reverse mode automatic differentiation ( 29 ; 32 ; 1 ) . A wide range of physical and financial signals nevertheless escape the Brownian template: log volatility in finance is rough ( 12 ; 6 ) , geophysical records exhibit long range dependence ( 3 ) , and network traffic and turbulent flows carry persistent increments ( 13 ; 19 ) . The standard Gaussian process with tunable memory is the fractional Brownian motion (fBm), parameterised by a Hurst exponent that ranges over the open unit interval: for H>1/2 the increments are positively correlated at every lag, for H<1/2 they are negatively correlated and the path is rough, and for H=1/2 one recovers standard Brownian motion ( 26 ; 4 ) . Hayashi and Nakagawa ( 17 ) showed that replacing the Brownian driver of a generative neural SDE by an fBm can improve the representation of long memory samples, but did not analyse discriminative learning in this setting. The training theory most directly related to our setting is based on Brownian dynamics. Samplewise adjoint sensitivity for Brownian systems is rooted in backward stochastic differential equations ( 10 ; 33 ) and underlies training algorithms for neural SDEs ( 24 ; 21 ; 1 ) . Archibald et al. ( 1 ) , in particular, establish a samplewise stochastic maximum principle at H=1/2 and an inverse-in-iteration rate for projected stochastic gradient descent (SGD). Related stochastic architectures include the diffusion limit formulation of Tzen and Raginsky ( 30 ) and neural controlled differential equations for path dependent data ( 22 ) . For non-semimartingale signals, rough path theory provides a deterministic pathwise calculus ( 11 ; 14 ) , used in neural rough differential equations for irregular time series ( 28 ) . In stochastic control, the stochastic maximum principle (SMP) for control systems driven by fBm of Han, Hu, and Song ( 15 ) and the backward stochastic differential equations (BSDEs) driven by fBm of Hu and Peng ( 20 ) provide the closest continuous time counterparts. Passing from H=1/2 to H\neq 1/2 does more than change the covariance of the injected noise. Fractional Brownian motion is neither a martingale nor a semimartingale, and its increments are not independent, so the conditional expectation of the next increment is generally nonzero. The usual Brownian adjoint recursion based on independent increments and the associated martingale representation therefore cannot be transferred unchanged. The continuous fractional maximum principle reflects the same difficulty through an adjoint driven jointly by the fBm and its underlying Brownian motion and through Malliavin derivative terms in the stationarity condition ( 15 ) . Ordinary reverse mode differentiation can still be run on a realised finite computation graph, but relating that reverse sweep to the adapted optimality system and proving its unbiasedness requires a separate argument. The contribution of this work has a numerical analysis aspect and an algorithm implementation aspect. On the numerical analysis aspect, we focus on deriving the adapted optimality system and an error estimate for projected samplewise backpropagation. In the Brownian setting, the independence of successive increments permits the adjoint equation to be represented through a standard one step martingale projection. This argument is no longer available for fractional Brownian increments, as their conditional expectation contains information from the entire observed history. Our strategy is to decompose each fractional increment into its predictable part and a fresh white noise innovation. This enables us to derive an adapted backward stochastic difference equation and a discrete stochastic maximum principle, and to show that the samplewise reverse sweep gives an unbiased gradient estimator for deterministic network parameters. On the algorithm implementation aspect, we formulate the fractional stochastic neural networks (FSNNs) directly as a controlled stochastic difference equation on the network depth grid for the full Hurst range. The implementable algorithm differentiates one realised fractional noise computation graph at each iteration and therefore requires only one forward sweep and one reverse sweep, without solving the adapted adjoint over the entire state space or evaluating Malliavin derivatives. The correlated increments are reconstructed from independent white noise innovations, and their coefficients can be precomputed once, while fast fractional noise samplers reduce the cost for long networks. The resulting framework allows the diffusion to depend on the layer, state and trainable control, and is examined on regression, long memory time series and image classification problems. The rest of this paper is organized as follows: In Section 2 , we introduce the discrete fractional Brownian increments, the underlying white noise innovations and the abstract well-posedness lemma for the discrete adjoint. In Section 3 , we define the FSNNs as a controlled stochastic difference equation and state the training problem. Section 4 derives the adjoint equation, the discrete stochastic maximum principle, the samplewise gradient and the training procedure. Section 5 proves the convergence theorem and its main corollaries. We examine the performance of FSNNs through several numerical experiments in Section 6 , and section 7 concludes. Generic positive constants are denoted by C . The Hurst exponent is H\in(0,1) , with H<1/2 , H=1/2 and H>1/2 corresponding respectively to rough anti-persistent, Brownian and persistent increments. The network depth is N , the step size is h , and t_{n}=nh . We write B^{H} for fractional Brownian motion, \xi_{n}^{H} for its normalised increment, \eta_{n} for the underlying white noise innovation, and \mathcal{F}_{n} for the discrete filtration. The state, control and objective are denoted by X_{n} , u_{n} and J . Partial derivatives are written as subscripts, and DJ(u)v denotes the directional derivative of J at u in the

Read the complete text