Dynamics, Noise, Delays and the Gibbs and Conditional Entropy Authors: Michael C. Mackey, Marta Tyran-Kamińska Published: 2026 ABSTRACT In this paper we review the concepts of the Gibbs and conditional entropies and examine their dynamic behaviour when the underlying dynamics are described by ordinary differential equations or stochastic differential equations. We then go on to introduce techniques for the analogous examination when the dynamics involve delays and noise. It is found that the effects of stochastic perturbations and/or delayed dynamics may be such that the approach of the entropies to equilibrium are not necessarily monotone and are dependent on system parameters. 1. Introduction The inexorable passage of time both fascinates and terrifies humans: witness the amount of money spent yearly trying to cosmetically reverse, or slow down, the effects of the passage of time, to obtain an idea of just how concerning people find it. This one-way evolution of time is, of course, simply a reflection of the operation of the second law of thermodynamics. Were it not for the consequences of the second law there would presumably be far less panic about the passage of time. We are quite sure, however, there would still be a lingering curiosity about why time evolves in only one direction. The development of thermodynamics was spurred on by developments in industrial processes during the early to mid-nineteenth century as people had to deal with the concepts of heat, work, and temperature in the pursuit of increased efficiency of steam engines. The initial work by Sadi Carnot led to the formulation of the laws of thermodynamics, due to the efforts of William Thompson (later Lord Kelvin) and Rudolf Clausius. These developments in thermodynamics, including the ‘laws of thermodynamics’, culminated in the development of statistical mechanics, which makes predictions of the collective macroscopic behaviour of particles from their microscopic behaviours. The second law of thermodynamics states that in a closed (no transfer of matter or energy in or out) system the entropy of the system can only increase (or stay constant) as time progresses. We may characterize this as the strong form of the second law, while the weak form would simply specify that the entropy will approach zero in the limit of long times. The strong form clearly implies the weak form. In the latter part of the nineteenth century, in the developing field of statistical mechanics, individuals started to seriously contemplate how to define and compute this quantity known as entropy. Three names of giants come to mind—Ludwig Boltzmann (Germany), Henri Poincaré (France), and J. Willard Gibbs (USA)—for their insights into the concept of entropy. For reasons that we will discuss later, Gibbs was the clear winner, and Section 2 will introduce the Gibbs entropy and a generalization of it. The remainder of the paper will deal with the issue of the temporal change in entropy, and this will be based on considerations from physics. Section 3 commences our consideration of the temporal evolution of entropy, as enshrined in the second law of thermodynamics, by reviewing entropy evolution when the underlying dynamics are defined by ordinary differential equations. We then continue in the same section to consider the added effects of noise on entropy evolution. This is a review of the material presented in [1,2]. Section 4 constitutes the bulk of this paper, adding the possibility of delays in dynamics to the mix of entropy evolution and the second law. In Section 4.1 we briefly review multidimensional stochastic functional differential equations, and in Section 4.2 we turn to a consideration of general linear equations with Gaussian noise, as it is only for such equations that we know what the density evolution is and what the entropies are. This examination of the effects of Gaussian noise in linear systems as a first approximation is only possible because of the well-developed theory of Gaussian processes with continuous sample paths. We derive various formulas for entropy dynamics, and show that the conditional entropy will approach zero but need not do so as a monotone function of time. Though linear dynamics driven by Gaussian noise is well motivated by analytical tractability and by its relevance to a broad class of stochastic processes, it is essential to emphasize to the reader that the closed-form entropy expressions we obtain here rely crucially on these assumptions. In particular, the Gaussian structure and linear evolution ensure that the state remains fully characterized by its first two moments, a property that generally fails in nonlinear and/or non-Gaussian systems. Consequently, the results derived here should not be interpreted as generic entropy laws applicable beyond the linear/Gaussian framework. In Section 4.3 we give examples of non-monotone convergence to zero of the conditional entropy. First we consider a one-dimensional example as initially presented in [3], and next we present completely new material on a second-order differential delay equation perturbed by noise. Section 4.4 contains a derivation of analytical expressions for the variances of the stationary position and velocity variables from Section 4.3. Finally in Section 4.5 we introduce one possible approach to obtain a Fokker–Planck-type equation for stochastic delay differential equations, extending the material presented in [3,4] to delay equations with both discrete and distributed delays. We conclude with some brief remarks in Section 5. 2. Entropies: Gibbs and Conditional In this paper we are concerned with the dynamic behaviour of entropies that are relevant for physical systems, and thus we confine our attention to the Gibbs entropy, first considered in Gibbs [5], and the conditional entropy which generalizes the Gibbs entropy. The latter has been the object of study by a number of different authors [6,7,8]. In his original work Gibbs assumed that he was dealing with a system operating in a phase space X which attains a steady-state density f*. Given this steady-state density he defined the ‘index of probability’ as lnf*(x) (ln is the natural logarithm) and identified the steady-state (equilibrium) entropy to be(1)HG(f*)=−∫Xf*(x)lnf*(x) dx.The equilibrium entropy definition, according to Gibbs, in (1) is the standard for equilibrium computations in thermodynamics and statistical mechanics and has led to the identification of HG(f*) with the equilibrium thermodynamic entropy. There is no particular reason to restrict the definition of the Gibbs entropy in (1) to equilibrium situations, and indeed we extend it to define the time-dependent Gibbs entropy of f(x,t) by(2)HG(f(·,t))=−∫Xf(x,t)lnf(x,t) dx. In addition to generalizing the Gibbs entropy to time-dependent situations we can also generalize the Gibbs entropy to situations in which we have a non-uniform steady-state density f*. This generalization has acquired the name of conditional entropy [9] and is given by(3)Hc(f(·,t)|f*)=−∫Xf(x,t)lnf(x,t)f*(x)dx.It should be noted that −Hc(f(·,t)|f*) is also known as the Kullback–Leibler divergence (I-divergence or relative entropy) [10]. It measures the distance between the two densities and it satisfies −Hc(f(·,t)|f*)≥0. But what about the Austrian giant Ludwig Boltzmann and his original entropy [11] (actually there were several different varieties of Boltzmann entropies and [12] has untangled and clarified their inter-relationships)? Problems with the Boltzmann entropy have been very elegantly and clearly analyzed in [13]. The Boltzmann entropy fails to give proper equilibrium entropies in several physical situations [14], while the Gibbs entropy does not suffer from this defect. Secondly, the Gibbs and Boltzmann entropies are only in agreement for extremely dilute gases in which the constituents do not interact. Finally, studies [2] have identified situations in which the Boltzmann entropy behaves non-physically but the Gibbs entropy does not. There is a very nice study of Boltzmann versus Gibbs entropy [15] for the interested reader. To conclude this section, we recall the amusing quotation from Eddington [16] re-framing the issue of temporal unidirectionality in terms of the behaviour of entropy, saying “The law that entropy always increases holds, I think, the supreme position among the laws of Nature. If someone points out to you that your pet theory of the universe is in disagreement with Maxwell’s equations—then so much the worse for Maxwell’s equations. If it is found to be contradicted by observation—well, these experimentalists do bungle things sometimes. But if your theory is found to be against the Second Law of Thermodynamics I can give you no hope; there is nothing for it to collapse in deepest humiliation.” 3. Behavior of Entropies for Dynamics Described by Differential Equations and Their Stochastic Perturbations 3.1. Density Evolution Equations Consider the dynamics defined by the system of ordinary differential equationsdxidt=Fi(x), i=1,…,d, with initial conditions xi(0)=xi,0 from a subset of Rd. We rewrite it in the vector form asx′(t)=F(x(t)), t>0, x(0)=x0.The probability density f(x,t), if it exists, of the variable x(t) satisfies the generalized Liouville equation(4)∂f∂t=−∑i∂(fFi)∂xi, and if there exists a steady-state density f*, then it is the stationary solution of Equation (4):∑i∂(f*Fi)∂xi=0. We next turn our attention to the Itô-type stochastic differential equation(5)dx(t)=F(x(t))dt+Σ(x(t)) dw(t), t>0, x(0)=x0, where Σ(x)=[σij(x)] is a d×n matrix and w is an n-dimensional standard Wiener process on an underlying probability space (Ω,F,P). Recall that a standard Wiener process is a Gaussian process with continuous paths and stationary independent increments for whichE(w(t)−w(s))=0, E((w(t)−w(s))(w(t)−w(s))T)=I|t−s|, t,s≥0, where the superscript T denotes the transpose and I is the identity matrix. Here and later E is the expectation operator corresponding to the probability measure P. We assume that both F and Σ satisfy a global Lipschitz condition. It is known [17,18,19] that if x(t) is the solution to the stochastic differential Equation (5) then the evolution of the density function f(x,t) of x(t) is described by the Fokker–Planck equation (Kolmogorov forward equation)(6)∂f∂t=−∑i=1d∂[Fi(x)f]∂xi+12∑i,j=1d∂2[aij(x)f]∂xi∂xj whereaij(x)=∑k=1nσik(x)σjk(x).Moreover, if there exists a stationary density f*(x) then it is the solution of the stationary Fokker–Planck Equation (6):(7)−∑i=1d∂[Fi(x)f]∂xi+12∑i,j=1d∂2[aij(x)f]∂xi∂xj=0. 3.1.1. Gibbs and Conditional Entropies Differentiating with respect to time the Gibbs entropy, as given by Equation (2), leads todHGdt=∫f∑i∂Fi(x)∂xi−12∑i,j∂2aij(x)∂xi∂xj dx+12∫1f∑i,j=1daij(x)∂f∂xi∂f∂xj dx.Note that if aij values do not depend on x then we obtain(8)dHGdt=∫f∑i∂Fi(x)∂xi dx+12∑i,j=1daij∫1f∂f∂xi∂f∂xj dx.In particular, the second term in the right-hand side of (8) is non-negative definite, but the first term is of indeterminant sign [20] (Equation (14)). Thus the temporal behavior of the Gibbs entropy in its approach to equilibrium is unclear. Next, if we differentiate Equation (3) for the conditional entropy with respect to time, then we obtaindHcdt=12∫f*2f∑i,j=1daij(x)∂∂xiff*∂∂xjff* dx, where we used (6), integration by parts and Equation (7) for the stationary density f*. Consequently,(9)dHcdt≥0, because the matrix (aij(x)) is non-negative definite. Hence, we conclude that the conditional entropy is a monotone increasing function of time and satisfies the strong form of the second law. In particular, if all aij values are equal to zero (no noise) then dHcdt=0, implying that in the noise-free case when f is the solution of the Liouville Equation (4), then(10)Hc(f(·,t)|f*)≡Hc(f0|f*). This last result deserves some consideration since it is so important. 1.All of the formulations of fundamental classical, relativistic, and quantum physics are in terms of ordinary or partial differential equations and without noise. All of the formulations of fundamental classical, relativistic, and quantum physics are in terms of ordinary or partial differential equations and without noise. 2.Thus any temporal evolution of entropy in these systems would appear to be impossible. Thus any temporal evolution of entropy in these systems would appear to be impossible. 3.It could only occur that the entropy might change (see (9)) in the presence of noise. It could only occur that the entropy might change (see (9)) in the presence of noise. This is astonishing and pinpoints exactly the conundrum that physics has always faced (Equation (10)) in terms of trying to find a dynamical foundation for the second law of thermodynamics (either weak or strong). 3.1.2. Convergence to Zero of the Conditional Entropy Various conditions implying convergence to zero of the conditional entropy are known; see, e.g., Mackey and Tyran-Kamińska [1] for a review. Here we only indicate how one can obtain a lower bound on the conditional entropy Hc(f(·,t)|f*) knowing that the probability density function p(x,t|x0) of x(t) conditional on x(0)=x0 exists. One identifies the transition probability p(x,t|x0) as a solution of the Fokker–Planck Equation (6) with the initial condition δ(x−x0). Then we obtainf(x,t)=∫p(x,t|x0)f0(x0) dx0, where f0 is the density of the initial condition x0, leading to(11)0≥Hc(f(·,t)|f*)≥∫Hc(p(·,t|x0)|f*)f0(x0)dx0 by Jensen’s inequality (see [3]). Consequently, if limt→∞Hc(p(·,t|x0)|f*)=0 for all x0 thenlimt→∞Hc(f(·,t)|f*)=0. 3.2. Effects of Noise Specific examples for which one can determine the densities f(·,t) and f* and the corresponding entropies are mainly known for linear differential equations with additive noise. Our first example is an Ornstein–Uhlenbeck process that was developed as a perturbation to the velocity v of a Brownian particle and is given by the Langevin equationdvdt=−γv+σξ(t), where γ>0,σ>0 and ξ(t) is white noise. In this case (5) reduces todv(t)=−γv(t)dt+σdw(t), where w(t) is a standard one-dimensional Wiener process. Another frequently used example is a noisy harmonic oscillator(12)d2dt2x(t)+γddtx(t)+βx(t)=σξ(t) with positive constant coefficients. If we introduce the variablesx(t)=x(t)v(t) with v(t)=x′(t), then we obtaindx(t)=Ax(t)dt+Σdw(t), whereA=01−β−γ and Σ=0σ. Thus, in this section we consider the general multidimensional Ornstein–Uhlenbeck process(13)dx(t)=Ax(t)dt+Σdw(t) where A is a d×d matrix and Σ is a d×n matrix. The solution to Equation (13) is(14)x(t)=etAx(0)+∫0te(t−r)AΣ dw(r) where etA=∑k=0∞tkk!Ak. Note that X(t)=etA is the solution to X′(t)=AX(t) satisfying X(0)=I and it is called the fundamental solution of the deterministic equation x′(t)=Ax(t). The properties of stochastic integrals imply that the processζ(t)=∫0tX(t−s)Σ dw(r), t≥0, is Gaussian. Its mean vector is equal to 0 and its covariance function is given byE(ζ(t)ζ(t+s)T)=∫0te(t−r)AΣΣTe(t+s−r)ATdr, by Itô’s isometry. In particular, the distribution of ζ(t) is Gaussian with mean 0 and the variance–covariance matrix(15)R(t):=E(ζ(t)ζ(t)T)=∫0terAΣΣTerATdr.Observe that the matrix R(t) is symmetric. Since erAΣΣTerAT is non-negative definite for all r≥0, the matrix R(t) is also nonnegative definite, but it might not be positive definite. In particular, if the Kalman rank condition holds [21,22], i.e., for the d×d matrix A and the d×n matrix Σ, the d×dn block matrix[A|Σ]:=[Σ AΣ … Ad−1Σ] is of full rank (rank[A|Σ]=d), then the matrix R(t) is of full rank. Consequently, R(t) is positive definite for all t>0, or equivalently detR(t)>0. Let fm,Q be a Gaussian density with mean vector m and positive definite covariance matrix Q, i.e.,(16)fm,Q(x)=1(2π)d/2(detQ)1/2e−12(x−m)TQ−1(x−m), x∈Rd, where Q−1 is the inverse of the matrix Q. Assume that the density f0 of x(0) is fm,Q. Then x(t) as given in (14) has a Gaussian distribution with mean vector E(x(t))=etAm and the covariance matrix R¯(t) is of the form(17)R¯(t)=etAQetAT+R(t), t>0.Consequently, the distribution of x(t) does not depend on t if and only if m=0 and there exists a covariance matrix R* such that(18)R*=etAR*etAT+R(t), t>0.Differentiating (18) with respect to t and then setting t=0 we obtain the following equation for R*:(19)AR*+R*AT+ΣΣT=0.Thus the covariance matrix R* satisfies (19). Conversely, if R* is the solution of (19) then (18) holds as well. Note that if R(t) is positive definite then R* is also positive definite by (18). 3.2.1. Existence of a Stationary Density In what follows we assume that rank[A|Σ]=d. Then x(t) with x(0)=x0 has a Gaussian distribution with mean etAx0 and a positive definite covariance R(t) as in (15). Letting R(t)−1 be the inverse matrix of R(t), the density of x(t) is given byp(x,t|x0)=1(2π)d/2(detR(t))1/2exp{−12(x−etAx0)TR(t)−1(x−etAx0)}.A stationary density f* exists if and only if the matrix A has only eigenvalues with negative real parts. In that case, the stationary density f* has the form(20)f*(x)=1(2π)d/2(detR*)1/2exp−12xTR*−1x, with the positive definite matrix R* being the unique symmetric matrix satisfying (19) and given byR*=∫0∞esAΣΣTesATds.Since R(t)→R* and etA→0 as t→∞, we havelimt→∞p(x,t|x0)=f*(x).Consequently, every stationary solution of the corresponding Fokker–Planck equation is a constant multiple of the stationary density f*. A stationary solution of (13) can be represented asZ(s)=∫−∞se(s−r)AΣdw(r), s∈R, where we extend the Wiener process w to R taking its independent copy {w¯(t):t≥0} and setting w(r)=−w¯(−r) for r<0. Observe thatZ(t)=etAZ(0)+ζ(t), t≥0, implying that Z(t) is a solution of (13). Since ζ(t) is a Gaussian distributed vector, we see that Z(t) has a Gaussian distribution with mean 0 and variance–covariance matrix R*. Consequently, Z(t), t≥0, is a stationary solution of (13) and f* is the density of Z(t). 3.2.2. Noise and the Temporal Behaviour of Gibbs and Conditional Entropies For the Gaussian density fm,Q in (16), the Gibbs entropy is(21)HG(fm,Q)=12(d+ln[(2π)ddetQ]), while for two Gaussian densities fm1,Q1 and fm2,Q2 the conditional entropy is(22)Hc(fm1,Q1|fm2,Q2)=12lndetQ1detQ2+12tr[(Q1−1−Q2−1)Q1]−12tr[Q2−1(m1−m2)(m1−m2)T]. Consequently, if f0=fm,Q with m=E(x(0)) and Q=E(x(0)−m)(x(0)−m)T, then for the density f(x,t) of x(t) we obtain the following(23)HG(f(·,t))=12(d+ln[(2π)ddetR¯(t)]), where R¯(t) is given by (17) and(24)Hc(f(·,t)|f*)=12lndetR¯(t)detR*+12tr(I−R*−1R¯(t))−12tr(R*−1etAmmTetAT).Note that if Q=R* then R¯(t)=R* and Equation (24) reduces toHc(f(·,t)|f*)=−12tr(R*−1etAmmTetAT) which is always an increasing function of time when m≠0. Observe that if m=x(0)=:x0 and Q=0, then we recover Hc(p(·,t|x0)|f*) from Equation (24). Thus, for an arbitrary initial density f0 with finite variance, we obtain from (11) the lower boundHc(f(·,t)|f*)≥12lndetR(t)detR*+12tr(I−R*−1R(t))−12∫tr(R*−1etAx0x0TetAT)f0(x0)dx0 for the conditional entropy. Sincelimt→∞etA=0 and limt→∞R(t)=R*, it is clear that limt→∞Hc(f(·,t)|f*)=0. Remember that we have H˙c(f(·,t)|f*)≥0 by (9) so we have the strong form of the second law obeyed in this case. Thus, the conditional entropy Hc for these stochastic systems increases monotonically to 0. To see what the temporal behavior of the Gibbs entropy is, consider first the one-dimensional Ornstein–Uhlenbeck process. If we start with a Gaussian density with mean m0 and variance σ02f0(v0)=12πσ02exp−(v0−m0)22σ02, then by Equations (17) and (15) we obtainR¯(t)=σ02e−2γt+∫0tσ2e−2γsds=σ02e−2γt+σ22γ(1−e−2γt) and the stationary density is given byf*(v)=12πσ*2exp−v22σ*2 with σ*2=σ22γ.Thus, the Gibbs entropy isHG(f(·,t))=12+12ln(2π)+12ln(σ*2+(σ02−σ*2)e−2γt), and it is either an increasing function of time when σ02>σ*2 or a decreasing function of time when σ02<σ*2, while the conditional entropy, given byHc(f(·,t)|f*)=12ln1+σ02σ*2−1e−2γt−12σ02σ*2−1e−2γt−m022σ*2e−2γt, is a strictly increasing function of time for σ02≠σ*2 or m0≠0. The temporal behavior of the Gibbs entropy can have more complicated patterns; see Figure 1 and Figure 2 and [1,2] in the case of the noisy harmonic oscillator. 4. Adding Delays in the Dynamics Though the subject of the dynamics of differential delay equations (DDEs) is well developed and relatively mature, things are not quite so clear when one wishes to examine the role of delays in the temporal evolution of entropy. This is because in studying entropy dynamics, the density in a system plays a central role, and there are great difficulties in the study of density evolution under the action of delayed dynamics. Thus we investigate the temporal behavior of entropies for linear systems with additive perturbations leading to Gaussian processes with densities being determined through a vector-valued mean function and a matrix-valued variance–covariance function extending the one-dimensional linear differential equations with a single delay in [3]. We show that the temporal behavior of the conditional entropy depends on the initial condition and is not necessarily a monotone function of time. The monograph [23] treats in detail the techniques that have been tried unsuccessfully to examine density evolution in delayed systems, while [24] examines certain aspects of this problem from the perspective of microphysical systems. Historically, important approaches have been made by a number of different groups ([24,25,26,27,28,29,30,31,32]), but it is unfortunately the situation that foundational issues are still unresolved and a satisfactory and unified theory allowing one to examine the evolution of densities in (infinite dimensional) delayed systems is yet to be developed. We [3,4] have employed various techniques to circumvent the problems outlined in [23] for the single delay case. At the end of this section, we show how this approach can be extended to functional differential equations with both discrete and distributed delays. The presentation is somewhat technical, but unavoidable, because of the difficulties encountered due to the infinite dimensional nature of systems with delayed dynamics. 4.1. Stochastic Functional Differential Equations DDEs or more generally functional differential equations (FDEs) are equations in which the rate of change of a system at the present time depends not only on its current state but also on its state at one or more times in the past. The evolution of the system cannot be determined from a single initial value, because the equation explicitly refers to past states. Instead, one must specify a history function, which prescribes the state of the system over a whole time interval in the past—typically from time −τ up to 0, where τ is the maximum delay. This history function acts as the ‘initial condition’ for the differential equation: once it is given, the equation can be used to compute the solution forward in time, which motivates working on the space of paths that are continuous functions on the interval [−τ,0]. In this path-space setting, the solution map, denoted below as St, provides a clean way to describe a dynamical system showing how the initial function evolves forward in time. Because delayed dynamics are inherently non-Markovian at the level of the observed state, we need to work on this space of functions to recover the Markov property when adding noise. In Section 4.5 we recall the associated extended generator that captures the infinitesimal time evolution of observables on this enlarged space. Once this framework is in place, we can project back from paths to the observed state variable, recovering evolution equations for its probability density. We refer the reader to [33,34,35] for a thorough introduction to functional differential equations and their stochastic perturbations. Suppose τ>0 is a given real number. For a continuous x:[−τ,∞)→Rd we introduce the notation xt∈C, wherext(s)=x(t+s) for s∈[−τ,0],t≥0, and C:=C([−τ,0],Rd) is the space of continuous functions ϕ:[−τ,0]→Rd. Given a mapping F:C→Rd we consider the functional differential equation(25)x′(t)=F(xt), t≥0,x0=ϕ.We assume that F satisfies a global Lipschitz condition so that a unique solution x:[−τ,∞)→Rd of (25) exists for each initial condition x0=ϕ∈C, and it depends continuously on (t,ϕ) and xt∈C for t>0. Then the solution map St:C→C given bySt(ϕ)(s)=xt(s) for s∈[−τ,0], is well defined for all t≥0 and gives rise to a semi-dynamical system {St}t∈R+ on the Banach space C equipped with the supremum norm [33]. In particular ifF(ϕ)=F(ϕ(0),ϕ(−τ)), ϕ∈C, for some transformation F:Rd×Rd→Rd, then we recover the case of a single delay equation whose density evolution was studied in [3,4]. We next look at a noise-perturbed version of (25):(26)dx(t)=F(xt)dt+Σ(xt)dw(t), t≥0,x0=ϕ, where {w(t)}t≥0 is again an n-dimensional standard Wiener process and the stochastic perturbation has an amplitude (diffusion coefficient) Σ, potentially continuously dependent on xt, with values in the space Rn×d of n×d matrices. We assume that both F and Σ satisfy a global Lipschitz condition, so that the stochastic functional differential Equation (26) (see [35]) has a pathwise solution x and that {xt}t≥0 is a Markov process with values in the space C, where again(27)xt(s)=x(t+s), s∈[−τ,0],t≥0. We end this section with a remark about how one can obtain a lower bound on the conditional entropy as in Section 3, but now for the densities defined by solutions of the stochastic functional differential equation considered in this section. If it happens that the solution x(t)=xt(0) of (26) with each deterministic initial condition ϕ∈C has a probability density function, say p(x,t|ϕ), then the functionf(x,t)=∫Cp(x,t|ϕ)μ0(dϕ), x∈Rd,t≥0, defines the density of x(t)=xt(0) when the initial condition x0 has the distribution μ0, which is now a distribution on the function space C. Using Jensen’s inequality, we obtain [3]Hc(f(·,t)|f*)≥∫CHc(p(·,t|ϕ)|f*)μ0(dϕ).Consequently, if Hc(p(·,t|ϕ)|f*) converges to zero as t→∞ for all ϕ∈C then we conclude thatlimt→∞Hc(f(·,t)|f*)=0. 4.2. A Linear Functional Differential Equation A particular example of a linear functional differential equation isx′(t)=Ax(t)+Bx(t−τ)+∫−τ0A(θ)x(t+θ)dθ, where A and B are d×d matrices and A(·):[−τ,0]→Rd×d is a continuous matrix-valued function. Once again, to proceed, we require some technical developments specific to linear delayed dynamics. Suppose that F:C→Rd is a continuous linear mapping. By the Riesz representation theorem, we can writeF(ϕ)=Aϕ(0)+∫−τ0dη(θ)ϕ(θ), where η=[ηij]i,j=1,…,d is a matrix-valued function of bounded variation and the integral is the Lebesgue–Stieltjes integral. Note that F(ϕ)−Aϕ(0) is a d-dimensional vector with coordinates∑j=1d∫−τ0ϕj(θ)dηij(θ), i=1,…,d.Let St:C→C be the solution map of the functional linear equation(28)x′(t)=Ax(t)+∫−τ0dη(θ)x(t+θ), t≥0,x0=ϕ.In particular, we havex(t)=Stϕ(0), t≥0.The stability of the trivial solution x(t)≡0 is determined by the roots of the characteristic (eigenvalue) equation(29)deth(λ)=0, h(λ)=λI−A−∫−τ0eθλdη(θ), where h:C→Cd×d. If all solutions of (29) have negative real parts, then all solutions of (28) tend to 0 as t→∞. The fundamental solution X(t) of Equation (28) is the d×d matrix function satisfying(30)X′(t)=AX(t)+∫−τ0dη(θ)X(t+θ) with(31)X(s)=0 for s∈[−τ,0) and X(0)=I, where I is the identity matrix. Let(32)α0=sup{Re(λ):deth(λ)=0}.As in the scalar case the fundamental matrix solution X has the following bound [33]: for each α>α0 there exists a constant C>0 such that(33)|X(t)|≤Ceαt for all t>0, where |·| is a norm of the d×d matrix. Let Σ be a d×n matrix and w be an n-dimensional standard Wiener process. Given an initial continuous function ϕ:[−τ,0]→Rd we can write the solution of(34)dx(t)=(Ax(t)+∫−τ0dη(θ)x(t+θ))dt+Σdw(t), t>0,x0=ϕ, asx(t)=Stϕ(0)+∫0tX(t−r)Σdw(r), where St is the solution map and X(t) is the fundamental matrix solution of the deterministic part (28). This corresponds to (14) in the case of τ=0. The processζ(t)=∫0tX(t−r)Σdw(r), t≥0, is a Gaussian process with values in Rd that have zero mean vector and the matrix-valued covariance function of the formE(ζ(t)ζ(t+s)T)=∫0tX(t−r)ΣΣTX(t+s−r)Tdr, t,s≥0, by Itô’s isometry. The variance of ζ(t) is thus given by(35)R(t):=Var(ζ(t))=E(ζ(t)ζ(t)T)=∫0tX(r)ΣΣTX(r)Tdr, t≥0.Note that R(t) is as in (15) when τ=0. Although the matrix R(t) in (35) is symmetric and non-negative definite, it does not need be positive definite. Observe that ζ defines for each t≥0 the segment ζt byζt(s)=ζ(t+s),if s+t≥0,s∈[−τ,0],0,if s+t<0,s∈[−τ,0].Consequently, the process xt in (27) being defined by the solution of (34) has the following representation:(36)xt=Stϕ+ζt, x0=ϕ. Let μt be the distribution of xt as in (36). Now suppose that x0=ϕ is a Gaussian process. Then xt is a Gaussian process and x(t) is an Rd-valued Gaussian random vector. To characterize its distribution μ(t) it is enough to find its mean vector and the variance–covariance matrix. We haveE(x(t))=Stm0(0), where Stm0 is the solution of (28) with the initial condition being the mean functionm0(s)=E(ϕ(s)), s∈[−τ,0], and the variance of x(t) is given by the equation(37)R¯(t):=Var(x(t))=Var(Stϕ(0))+R(t), t>0.In particular, if x0=ϕ is deterministic, so that the function ϕ is fixed, then xt is also a Gaussian process and Var(Stϕ(0))=0. If, on the other hand, Stϕ is an Rd-valued Gaussian process with mean vector Stm0 then we can assume that m0≡0. 4.2.1. Existence of a Stationary Density Suppose that α0<0, where α0 is defined in (32). Then Stϕ(0)→0 as t→∞. It follows from (33) that for some α<0 and C>0|∫t∞X(r)ΣΣTX(r)Tdr|≤C|ΣΣT|∫t∞e2αrdr→0 as t→∞, implying that the following limit exists and is finite:(38)limt→∞R(t)=∫0∞X(r)ΣΣTX(r)Tdr:=R*.If R(t) is non-singular (equivalently, R(t) is positive definite), then the density p of the distribution μ(t) of x(t) is given by the Gaussian density(39)p(x,t|ϕ)=1(2π)d/2(detR(t))1/2e−12(x−Stϕ(0))TR(t)−1(x−Stϕ(0)), x∈Rd, where R(t)−1 is the inverse matrix of the covariance matrix R(t). If R(t) is positive definite, we see that its limit R* is also positive definite and thus invertible. Then we have(40)limt→∞p(x,t|ϕ)=1(2π)d/2(detR*)1/2e−12xTR*−1x=f*(x), and f* is as in (20) with R* now given by (38). Equation (34) has a stationary solution of the form [35] (Corollary 4.2.1)(41)Z(s)=∫−∞sX(s−r)Σdw(r), s∈R and the distribution of Z(s) is Gaussian with density f* as in (40). 4.2.2. The Gibbs and Conditional Entropies In this section, we assume that the matrix R(t) is positive definite for all sufficiently large t. Let the initial condition ϕ be a Gaussian process and let f(·,t) be the Gaussian density of x(t) with variance R¯(t) as in (37) and the mean vector Stm0 with m0 being the mean vector of the initial condition ϕ. By (21) we have(42)HG(f(·,t))=12d+ln(2π)ddetR¯(t)), and thusdHG(f(·,t))dt=12detR¯(t)ddetR¯(t)dt.By Jacobi’s formula we havedlndetR¯(t)dt=trR¯(t)−1dR¯(t)dt, implying thatdHG(f(·,t))dt=12trR¯(t)−1dR¯(t)dt. If we take fm1,Q1 as in (39) and fm2,Q2 as in (40), then Q1=R¯(t), m1=Stm0(0), and Q2=R*, m2=0. Hence, by (22), we have(43)Hc(f(·,t)|f*)=12lndet(R*−1R¯(t))+12tr[I−R*−1R¯(t)]−12tr[R*−1Stm0(0)(Stm0(0))T] and so the temporal derivative of the conditional entropy satisfies(44)dHc(f(·,t)|f*)dt=12trR¯(t)−1dR¯(t)dt−R*−1dR¯(t)dt−R*−1dStm0(0)(Stm0(0))Tdt. If we take as the initial condition x0=ϕ with ϕ(s)=Z(s)+m0(s) for s∈[−θ,0], where Z is the Gaussian process, being the stationary solution of (34), and m0 is a non-zero initial mean function, then R¯(t)=R* and we obtain from (43) that(45)Hc(f(·,t)|f*)=−12tr[R*−1Stm0(0)(Stm0(0))T].However, we cannot conclude that the conditional entropy Hc, given by the function −12tr[R*−1Stm0(0)(Stm0(0))T], is strictly increasing although it converges to zero. See Section 4.3 for examples. If, on the other hand, we take m0=ϕ with ϕ fixed, so that the initial condition is deterministic, then R¯(t)=R(t). Hence, we recover HG(p(·,t|ϕ)) from Equation (42) and Hc(p(·,t|ϕ)|f*) from (43). Since the matrix dR(t)dt is non-negative definite, we see that the matrix R(t)−1dR(t)dt is also nonnegative definite. Consequently, its trace is nonnegative, implying that(46)dHG(p(·,t|ϕ))dt≥0 so the Gibbs entropy of the density p is monotone increasing and thus the strong form of the second law holds. SinceR(t)−1dR(t)dt−R*−1dR(t)dt=R*−1(R*−R(t))R(t)−1dR(t)dt and the matrix R*−R(t), given byR*−R(t)=∫t∞X(r)ΣΣTX(r)Tdr, is non-negative definite, we conclude thattrR(t)−1dR(t)dt−R*−1dR(t)dt≥0.However, as in the one-dimensional case we cannot claim that−trR*−1dStϕ(0)(Stϕ(0))Tdt≥0 and once again the presence of the delay has opened the possibility of a non-monotone approach of the conditional entropy to zero and, thus, a shift to the weak form of the second law. Finally, for the density f(x,t) of xt(0) with initial condition x0 chosen according to a distribution μ0 we obtainHc(f(·,t)|f*)≥12ln detR(t)detR*+12tr[I−R*−1R(t)]−12∫Ctr[R*−1Stϕ(0)(Stϕ(0))T]μ0(dϕ).Since R(t)→R* and Stϕ(0)→0 as t→∞, we conclude that(47)limt→∞Hc(f(·,t)|f*)=0. 4.2.3. The Covariance of the Stationary Solution To find R* let us look at the matrix-valued covariance of the process {Z(s):s∈R}:K(s)=E(Z(t)Z(s+t)T), s∈R.The covariance function is given byK(s)=∫−∞min{t,s+t}X(t−u)ΣΣTX(s+t−u)Tdu by the Itô isometry. Changing the variables in the integral leads toK(s)=∫0∞X(r)ΣΣTX(r+s)Tdr, s≥0, andK(s)=∫0∞X(r−s)ΣΣTX(r)Tdr, s<0.Hence K(s)=K(−s)T for s<0. Note thatR*=K(0)=K(0)T. Let us now derive a differential equation for the matrix K(t) for t>0. We haveK′(t)=∫0∞X(r)ΣΣTX′(r+t)Tdr.It follows from (30) thatX′(r+t)T=X(r+t)TAT+∫−τ0X(r+t+θ)TdηT(θ), r,t>0, implying that K satisfies the equation(48)K′(t)=K(t)AT+∫−τ0K(t+θ)dηT(θ), t>0.Moreover, we haveddr(X(r)ΣΣTX(r)T)=X′(r)ΣΣTX(r)T+X(r)ΣΣTX′(r)T and∫0∞X(r)ΣΣTX′(r)Tdr=K′(0), where K′(0)=lims↓0K′(s).Hence, we obtain(49)K′(0)T+K′(0)=−ΣΣT with K′(0)=K(0)AT+∫−τ0K(θ)dηT(θ).Consequently, if we want to find R*=K(0), then we need to solve Equation (48) subject to condition (49). In general, it is impossible to find a solution of (48), though in the one-dimensional case with a single discrete delay one was derived in [36]; see also [28]. Here we obtain a second-order differential equation for the covariance K of the stationary solution of equation(50)dx(t)=[Ax(t)+Bx(−τ)]dt+Σdw(t), where B is a d×d matrix. It follows from (48) that the covariance K(t) of the stationary solution satisfiesK′(t)=K(t)AT+K(t−τ)BT, t>0, and K(s)=K(−s)T for s≤0. For t∈(0,τ) we have K(t−τ)=K(τ−t)T. Thus, we obtain(51)K′(t)=K(t)AT+K(τ−t)TBT, t∈(0,τ).Next, differentiating the function U(t)=K(τ−t)T we getU′(t)=ddt[K(τ−t)T]=−AK(τ−t)T−BK(t) implying thatK′′(t)=K′(t)AT+U′(t)BT=K′(t)AT−AK(τ−t)TBT−BK(t)BT.From Equation (51) we obtainK(τ−t)TBT=K′(t)−K(t)AT.Consequently, the covariance satisfies for t∈(0,τ) the second-order matrix differential equation(52)K′′(t)=K′(t)AT−AK′(t)+AK(t)AT−BK(t)BT with(53)K′(0)+K′(0)T=−ΣΣT, K′(0)=K(0)AT+K(τ)TBT. Let us summarize symbols used in Section 4.2:F—a general linear continuous mapping on C=C([−τ,0],Rd), A, B – d×d matrices;St—the solution map of (28); Stϕ(0) is the solution of (28);X(t)—the fundamental matrix solution (30) and (31);x(t)—the solution of (34); its variance is R¯(t);ζ(t)—a Gaussian vector with zero mean and the variance equal to R(t) as in (35);R*—the limit of the variance R(t) as defined in (38);Z(s),s∈R—the Gaussian process as in (41) that is the stationary solution of (34);K(s)—the matrix-valued covariance function of the stationary process Z; K(0)=R*. F—a general linear continuous mapping on C=C([−τ,0],Rd), A, B – d×d matrices; F—a general linear continuous mapping on C=C([−τ,0],Rd), A, B – d×d matrices; St—the solution map of (28); Stϕ(0) is the solution of (28); St—the solution map of (28); Stϕ(0) is the solution of (28); X(t)—the fundamental matrix solution (30) and (31); X(t)—the fundamental matrix solution (30) and (31); x(t)—the solution of (34); its variance is R¯(t); x(t)—the solution of (34); its variance is R¯(t); ζ(t)—a Gaussian vector with zero mean and the variance equal to R(t) as in (35); ζ(t)—a Gaussian vector with zero mean and the variance equal to R(t) as in (35); R*—the limit of the variance R(t) as defined in (38); R*—the limit of the variance R(t) as defined in (38); Z(s),s∈R—the Gaussian process as in (41) that is the stationary solution of (34); Z(s),s∈R—the Gaussian process as in (41) that is the stationary solution of (34); K(s)—the matrix-valued covariance function of the stationary process Z; K(0)=R*. K(s)—the matrix-valued covariance function of the stationary process Z; K(0)=R*. 4.3. Examples 4.3.1. A Linear Equation with Single Delay in One Dimension Consider the single delay Equation (50) in the case of d=1, so that A=a and B=b are real constantsdx(t)=(ax(t)+bx(t−τ))dt+σdw(t). The entirety of the (aτ,bτ) plane inside the hatched wedge-shaped area in Figure 3 corresponds to parameter values (a,b,τ) such that this equation has a stationary solution and a stationary density. Equation (52) for the variance K(t) of the stationary solution Z reduces toK′′(t)=(a2−b2)K(t), with initial and boundary conditionsK′(0)=−σ22, K′(0)=aK(0)+bK(τ).It follows from [36] that if a2≠b2 then the variance K(0)=V* is given byV*=σ22bsinh(a2−b2τ)−a2−b2a2−b2(bcosh(a2−b2τ)+a).Note that if τ=0, b=0 and a=−γ we recover the variance σ*2 of the stationary Ornstein–Uhlenbeck process from Section 3.2. The conditional entropy as given in (45) reduces to(54)Hc(f(·,t)|f*)=−12m2(t)σ*2, where m(t)=Stm0(0) is the solution of the equation x′(t)=ax(t)+bx(t−τ) with the initial condition function m0(s), s∈[−τ,0]; see Figure 4. 4.3.2. A Second-Order Equation with One Delay We now consider a particle of mass m, located at a position x(t) and moving with velocity v(t) and subject to a delayed restoring force [−βx(t−τ)] as well as a delayed frictional force −γv(t−τ) and a fluctuating force σξ(t), where β>0 and γ>0, while σ≥0 and ξ(t) is the white noise. Thus, the dynamics of this noisy, delayed harmonic oscillator are given by(55)ddtx(t)=v(t),mddtv(t)=−γv(t−τ)−βx(t−τ)+σξ(t).Note that the system (55) with m=1 in the absence of noise (σ=0) is equivalent to the second-order linear delayed equation(56)d2dt2x(t)=−γddtx(t−τ)−βx(t−τ).This equation is a special case of the general second-order delay differential equation(57)d2dt2x(t)+a1ddtx(t)+a0x(t)=b0x(t−τ)+b1ddtx(t−τ), where a1,a0,b1,b0 are real constants. The stability of the trivial solution x(t)≡0 of the deterministic Equation (57) is determined by the roots of the characteristic (eigenvalue) equation(58)λ2+a1λ+a0−b1λe−λτ−b0e−λτ=0, obtained with the ansatz x(t)∼eλt. One can infer conditions for asymptotic stability from the literature [39,40,41,42]. In particular, if b0≥a0 then the trivial solution is unstable. In what follows we thus assume that a0>b0 and that all roots of (58) have negative real parts, i.e.,a0>b0 and α0:=sup{Re(λ):λ satisfies (58)}<0.Note that if we take τ=0 in (58), then we have α0<0 iff a0>b0 and a1>b1. See Section 4.4.2 for necessary and sufficient conditions for α0<0 when τ>0. The stability region of parameters for Equation (56) is presented in Figure 5. Now consider Equation (57) with noised2dt2x(t)+a1ddtx(t)+a0x(t)=b0x(t−τ)+b1ddtx(t−τ)+σξ(t), and see [29,31,32] for particular examples. Introducing the variablesx(t)=x(t)v(t) with v(t)=x′(t), then we obtain(59)dx(t)=[Ax(t)+Bx(t−τ)]dt+Σdw(t), where w(t) is a standard one-dimensional Wiener process andA=01−a0−a1, B=00b0b1, Σ=0σ.We can define a stationary solution {Z(t):t∈R} for (59) as in the previous section. Let K(t)=(kij(t))i,j=1,2 be its covariance matrix t≥0. First, we show that(60)k12(t)+k21(t)=0 for all t≥0.Recall that(61)K(t)=∫0∞X(r)ΣΣTX(r+t)Tdr.Since the fundamental solution X(t) is a 2×2 matrix, which we represent as X(t)=(xij(t))i,j=1,2, and furthermore, it satisfiesX′(t)=AX(t)+BX(t−τ), t>0,X(0)=I, X(s)=0 for s<0, we see thatx12′(t)=x22(t) for t>0, x12(s)=0 for s≤0.Thus, we obtainX(r)Σ=σx12(r)x12′(r) andX(r)ΣΣTX(r+t)T=σ2x12(r)x12(r+t)x12(r)x12′(r+t)x12′(r)x12(r+t)x12′(r)x12′(r+t).We havek12(t)=∫0∞σ2x12(r)x12′(r+t)dr.Since x12(r)→0 as r→∞ and x12(0)=0, integrating by parts leads tok12(t)=−∫0∞x12′(r)x12(r+t)dr=−k21(t).Next it follows from (60) that the matrix R*=K(0) is diagonal, since K(0)T=K(0). Thus the stationary density f* as given by (40) will be of the formf*(x,v)=12πσx2σv2e−x22σx2+v22σv2, x,v∈R, where the variances(62)σx2=k11(0) and σv2=k22(0) and k11(0), k22(0) are the diagonal elements of the matrix K(0) as given by (85). The Gibbs and conditional entropies behave as described in Section 4.2. Although the conditional entropy always converges to zero, it might not be an increasing function of time, as the following example shows. We consider the initial condition x0 to be x0(s)=Z(s)+m0(s), s∈[−τ,0], where Z is the stationary Gaussian process and m0 is a non-zero mean vector function with m0(s)=(m0x(s),m0x′(s))T, s∈[−τ,0]. Then the conditional entropy is by (45), equal to(63)Hc(f(·,t)|f*)=−12mx2(t)σx2+mv2(t)σv2, where mx(t) is the solution of (57) with initial condition m0x, while mv(t)=mx′(t). Particular examples are given in Figure 6 and Figure 7, where we have a0=0 and a1=0 together with b0=−β and b1=−γ with the values of (β,γ,τ) in the hatched area of Figure 5. 4.4. Determination of the Variances in (62) This section is devoted to a derivation of the variances k11(0) and k22(0) as functions of parameters a1,a0,b1,b0 and the delay τ for the stationary solution of (59). Let K(t)=(kij(t))i,j=1,2 be its covariance matrix function (61). To find the equation for kij(t) for t∈(0,τ) we infer from (51) and (60) thatk11′(t)−k21′(t)k21′(t)k22′(t)=k11(t)−k21(t)k21(t)k22(t)0−a01−a1+k11(τ−t)k21(τ−t)−k21(τ−t)k22(τ−t)0b00b1, implying that(64)k11′(t)=−k21(t), k21′(t)=k22(t) and(65)k22′(t)=−a0k21(t)−a1k22(t)−b0k21(τ−t)+b1k22(τ−t).Using (52) we arrive atk22′′(t)=−2a0k21′(t)+(a02−b02)k11(t)+(a12−b12)k22(t) and applying (64) we obtain the following system of second-order differential equations:(66)k11′′(t)=−k22(t),k22′′(t)=(a02−b02)k11(t)+(a12−b12−2a0)k22(t).The initial boundary conditions for this equation, based on (53), turn intok11′(0)=0, k22′(0)=−σ22, andk21′(0)=k22(0),−k21′(0)=−a0k11(0)+b0k11(τ)+b1k21(τ),k22′(0)=−a1k22(0)−b0k21(τ)+b1k22(τ).Combining them together with k21(τ)=−k11′(τ) by (64), we conclude that Equation (66) is supplemented with(67)a0k11(0)−k22(0)−b0k11(τ)+b1k11′(τ)=0,a1k22(0)−b0k11′(τ)−b1k22(τ)=σ22,k11′(0)=0, k22′(0)=−σ22.Consequently, we may rewrite (66) as a second-order system of equations in the vector formk11′′(t)k22′′(t)=Gk11(t)k22(t) with(68)G=0−1a02−b02a12−b12−2a0.The general solution of this equation is(69)k11(t)k22(t)=C(t)k11(0)k22(0)+S(t)k11′(0)k22′(0), where(70)C(t)=∑k=0∞t2k(2k)!Gk and S(t)=∑k=0∞t2k+1(2k+1)!Gk.Since C′(t)=GS(t) and S′(t)=C(t), we also have(71)k11′(t)k22′(t)=GS(t)k11(0)k22(0)+C(t)k11′(0)k22′(0). We can rewrite (67) as the following equation:(72)G1k11(0)k22(0)+G2k11(τ)k22(τ)+G3k11′(τ)k22′(τ)=0σ22, where(73)G1=a0−10a1, G2=−b000b1, G3=b10−b00.Using (69) and (71) for t=τ, we rewrite condition (72) as(G1+G2C(τ)+G3GS(τ))k11(0)k22(0)+(G2S(τ)+G3C(τ))0−σ22=0σ22.Consequently, we obtain(74)k11(0)k22(0)=(G1+G2C(τ)+G3GS(τ))−1(I+G2S(τ)+G3C(τ))0σ22 andk11(t)k22(t)=C(t)k11(0)k22(0)+S(t)0−σ22, t∈(0,τ). Our approach implies that the function k22 satisfies, by (66), the following fourth-order differential equation (75) k22(iv)(t)−(a12−b12−2a0)k22′′(t)+(a02−b02)k22=0. If we go back to (64) and (65), and if we take m=1, a1=a, a0=k, b1=−b, b0=0, k21(t)=g(t), and k22(t)=C(t), then we get the same equation as Equation (10) in [29], where it is postulated that a general solution of Equation (10) is of the form C(t)=Accos(ω(t−τ/2))+Assin(ω(t−τ/2)), with constants Ac, As, ω derived separately. However, observe that no matter what the constants are, we obtain C′′(t)=−ω2C(t), but C(t) should be a general solution of (75). A similar approach was used in [31] (Appendix B) with Equation (B2), where b1=0, b0=g/Q0, a1=1/Q0, a0=1, and k22(t)=ϕvv(t). 4.4.1. General Form of the Matrices C(t) and S(t) in (70) To calculate the matrices C(t) and S(t), we find the eigenvalues of the matrix G. The characteristic equation det(zI−G)=0 for the matrix G in (68) is given byz2−tr(G)z+det(G)=0, where(76)tr(G)=a12−b12−2a0 and det(G)=a02−b02.Thus, we arrive at(77)z2−(a12−b12−2a0)z+(a02−b02)=0, whose roots are(78)z±=12(a12−b12−2a0±Δ), whereΔ=(a12−b12−2a0)2−4(a02−b02). Suppose that Δ>0, so that there are two different real eigenvalues. If detG<0 then z+z−<0 while if detG>0 then z±<0 or z±>0. The case of detG=0 implies that one of the roots is equal to 0. If Δ<0 then z±=αe±iβ, where α≥0 and β∈R, and we write z±=αe±iβ/2. Finally, if Δ=0 and detG≠0 then z+=z−≠0, while if detG=0 then z+=z−=0. We have(79)C(t)=c0(t)I+c1(t)G, S(t)=s0(t)I+s1(t)G, for some functions ci, si, i=0,1 that we now specify. 1.If Δ≠0 and det(G)≠0 thenc0(t)=z+c−(t)−z−c+(t), c1(t)=c+(t)−c−(t)s0(t)=z+s−(t)−z−s+(t), s1(t)=s+(t)−s−(t) wherec±(t)=cosh(tz±)z+−z−, s±(t)=sinh(tz±)(z+−z−)z±. If Δ≠0 and det(G)≠0 thenc0(t)=z+c−(t)−z−c+(t), c1(t)=c+(t)−c−(t)s0(t)=z+s−(t)−z−s+(t), s1(t)=s+(t)−s−(t) wherec±(t)=cosh(tz±)z+−z−, s±(t)=sinh(tz±)(z+−z−)z±. 2.If Δ≠0 and det(G)=0 so that one of the roots is equal to zero, then c0 and c1 stay the same ands0(t)=0, s1(t)=sinh(tz+)z+z+. If Δ≠0 and det(G)=0 so that one of the roots is equal to zero, then c0 and c1 stay the same ands0(t)=0, s1(t)=sinh(tz+)z+z+. 3.Now suppose that Δ=0 and det(G)≠0, so that z+=z−≠0. Thenc0(t)=cosh(tz+)−12z+tsinh(tz+), c1(t)=t2z+sinh(tz+) ands0(t)=32z+sinh(tz+)−t2cosh(tz+),s1(t)=t2z+cosh(tz+)−12z+z+sinh(tz+). Now suppose that Δ=0 and det(G)≠0, so that z+=z−≠0. Thenc0(t)=cosh(tz+)−12z+tsinh(tz+), c1(t)=t2z+sinh(tz+) ands0(t)=32z+sinh(tz+)−t2cosh(tz+),s1(t)=t2z+cosh(tz+)−12z+z+sinh(tz+). 4.Finally, when Δ=0 and det(G)=0, then z+=z−=0 and we havec0(t)=1, c1(t)=12t2, s0(t)=t, s1(t)=16t3. Finally, when Δ=0 and det(G)=0, then z+=z−=0 and we havec0(t)=1, c1(t)=12t2, s0(t)=t, s1(t)=16t3. 4.4.2. Stability Analysis of the Deterministic Delay Equation To find the stability region for the second-order delay equation [39,40], substitute λ=iω with ω>0 into the characteristic Equation (58), separating it into real and imaginary parts:a0−ω2=b0cos(ωτ)+b1ωsin(ωτ), a1ω=b1ωcos(ωτ)−b0sin(ωτ), and leading to a quartic equation for ω:(80)ω4+(a12−b12−2a0)ω2+(a02−b02)=0.Now observe that if we substitute z=−ω2 into (77) then we obtain Equation (80). Consequently, solutions of (80) satisfy ω2=−z±, where z± are as in (78). Thus, we can find a positive solution of (80) if and only if z+<0 or z−<0. Here we classify the existence of these positive solutions in terms of det(G) and tr(G) as given in (76). Let m be the number of positive solutions of (80). Then m=0 iff det(G)>0 and tr(G)>−2det(G) or det(G)=0 and tr(G)≥0; m=0 iff det(G)>0 and tr(G)>−2det(G) or det(G)=0 and tr(G)≥0; m=1 iff det(G)<0 or det(G)=0 and tr(G)<0 or det(G)>0 and tr(G)=−2det(G); m=1 iff det(G)<0 or det(G)=0 and tr(G)<0 or det(G)>0 and tr(G)=−2det(G); m=2 iff det(G)>0 and tr(G)<−2det(G). m=2 iff det(G)>0 and tr(G)<−2det(G). Note that det(G)>0 iff |a0|>|b0|. Recall that we assume a0>b0. Then α0<0 if and only if one of the following holds (see [39,40,41,42]):1.a1>b1, m=0 and τ≥0;2.a1>b1, m=1 and τ∈[0,τ0+) for some τ0+>0;3.a1>b1, m=2 andτ∈[0,τ0+)∪⋃j=1k(τj−1−,τj+), for some k and a sequence of delays τj±;4.a1≤b1, m=2 andτ∈⋃j=0k(τj−,τj+). 1.a1>b1, m=0 and τ≥0; a1>b1, m=0 and τ≥0; 2.a1>b1, m=1 and τ∈[0,τ0+) for some τ0+>0; a1>b1, m=1 and τ∈[0,τ0+) for some τ0+>0; 3.a1>b1, m=2 andτ∈[0,τ0+)∪⋃j=1k(τj−1−,τj+), for some k and a sequence of delays τj±; a1>b1, m=2 andτ∈[0,τ0+)∪⋃j=1k(τj−1−,τj+), for some k and a sequence of delays τj±; 4.a1≤b1, m=2 andτ∈⋃j=0k(τj−,τj+). a1≤b1, m=2 andτ∈⋃j=0k(τj−,τj+). 4.4.3. Formulas for Variances We need to find the inverse of the matrix H:=G1+G2C(τ)+G3GS(τ) and the second column of the matrix I+G2S(τ)+G3C(τ). To this end, observe thatG1+G2C(τ)+G3GS(τ)=G1+c0(τ)G2+c1(τ)G2G+s0(τ)G3G+s1(τ)G3G2, where ci=ci(τ), si=si(τ), i=0,1 are from (79) and the matrices G1,G2,G3 are as in (73). We haveG2G=−b000b10−1det(G)tr(G)=0b0b1det(G)b1tr(G) andG3G=b10−b00G=0−b10b0, G3G2=−b1det(G)−b1tr(G)b0det(G)b0tr(G), implying that the elements of the matrix G1+G2C(τ)+G3GS(τ)=(hij(t))i,j=1,2 are given by(81)h11=a0−b0c0−b1det(G)s1, h12=−1+b0c1−b1q1,h21=det(G)[b1c1+b0s1], h22=a1+b1q0+b0q1, where(82)q0=c0+tr(G)c1, q1=s0+tr(G)s1.The determinant of H isdet(H)=(a0−b0c0−b1det(G)s1)(a1+b1q0+b0q1)−det(G)(b1c1+b0s1)(−1+b0c1−b1q1) which reduces to(83)det(H)=a0a1+a0(b1q0+b0q1)−a1(b0c0+b1det(G)s1)+det(G)(b0s1+b1c1)−(b0c0+b1det(G)s1)(b1q0+b0q1)−det(G)(b1c1+b0s1)(b0c1−b1q1).Next, we find the vectorh1h2=(I+G2S(τ)+G3C(τ))01.We haveI+G2S(τ)+G3C(τ)=I+s0G2+s1G2G+c0G3+c1G3G=1−b0s0+b1c0s1b0−b1c1b1det(G)s1−b0c01+b1s0+b1tr(G)s1+b0c0 implying that(84)h1=b0s1−b1c1, h2=1+b0c0+b1q1.Consequently, the diagonal matrix K(0) has on its diagonal by (74) the following values(85)k11(0)=σ22det(H)[h22h1−h12h2], k22(0)=σ22det(H)[h11h2−h21h1], where det(H) is as in (83), hij as in (81) with qi in (82), and hi as in (84). We haveh22h1−h12h2=(a1+b1q0+b0q1)(b0s1−b1c1)−(−1+b0c1−b1q1)(1+b0c0+b1q1)=1+a1(b0s1−b1c1)+b0(c0−c1)+b0b1(c0q1−q0s1)+b12q0c1+b02q1s1−b11q12 andh11h2−h21h1=(a0−b0c0−b1det(G)s1)(1+b0c0+b1q1)−det(G)(b1c1+b0s1)(b0s1−b1c1).Note that if τ=0, b0=0,b1=0, then we have det(H)=a0a1 andk11(0)=σ22a0a1, k22(0)=σ22a1, that are the variances of the stationary solution of the noisy harmonic oscillator from Section 3.2 with β=a0 and γ=a1. 4.5. Is There a Fokker–Planck-like Equation for Differential Delay Equations? We start with a few preliminaries with the ultimate goal of deriving a Fokker–Planck-like equation for the evolution of a ‘density’ under the action of noisy, delayed dynamics. The process {xt}t≥0 as in (27) defines a transition semigroup {Tt}t≥0 byTtψ(ϕ)=E(ψ(xt)|x0=ϕ), t≥0,ϕ∈C,ψ∈B(C), on the space B(C) of bounded Borel measurable functions ψ:C→R. For each t≥0 we define the Borel probability measure μt on the space C as the distribution of xt, so that〈ψ,μt〉=〈Ttψ,μ0〉, t≥0,ψ∈B(C), where we use the customary scalar product notation〈ψ,μ〉=∫Cψ(ϕ)μ(dϕ).Then {μt}t≥0 is said to solve the equation(86)∂∂tμt=L*μt, where L* is a formal adjoint of the so-called extended generatorL of the Markov process. The linear operator L:D→B(C) with domain D⊂B(C) is such that∫0t〈|Lψ|,μr〉dr<∞ and〈ψ,μt〉=〈ψ,μ0〉+∫0t〈Lψ,μr〉dr for all t>0 and ψ∈D. Now, instead of dealing with the abstract Equation (86) we proceed as in [3,4,43]. We introduce a differential operator L:Cc2(Rd)→B(C) defined byLg(ϕ)=∑i=1dFi(ϕ)∂g∂xi(ϕ(0))+12∑i,j=1d(ΣΣT)ij(ϕ)∂2g∂xi∂xj(ϕ(0)), ϕ∈C, for g∈Cc2(Rd), where Cc2(Rd) is the space of functions with compact support that possess continuous derivatives up to second order. Here, [(ΣΣT)ij]i,j=1,…,d is the matrix ΣΣT, where ΣT is the transpose of the matrix Σ. Consider the distribution μ(t) of x(t)=xt(0), t≥0, defined on B(Rd), the σ-algebra of Borel subsets of Rd, byμ(t)(B)=μt{ϕ∈C:ϕ(0)∈B}, B∈B(Rd).Then {μt}t≥0 is said to solve the equation(87)∂∂tμ(t)=L*μt if∫0t〈|Lg|,μr〉dr<∞ and(88)∫Rdg(x)μ(t)(dx)=∫Rdg(x)μ(0)(dx)+∫0t〈Lg,μr〉dr for all t>0 and g∈Cc2(Rd). Note that we can represent the Borel measure μ(t) as μ(t)=μt∘π0−1, where the projection map π0:C→Rd is given by π0(ϕ)=ϕ(0), ϕ∈C. Thus, Equation (87) can be compared with (86) as it can be rewritten as∂∂tμt∘π0−1=L*μt. Suppose that the drift coefficient F and the diffusion coefficient Σ in (26) are those with discrete delaysF(ϕ)=F(ϕ(τ0),…,ϕ(τk)), Σ(ϕ)=σ(ϕ(τ0),…,ϕ(τk)), where k∈N, F and σ are given transformations on Rd(k+1) and0=τ0<…<τk=τ.We introduce the projection mapping πτ0,…,τk:C→Rd(k+1)πτ0,…,τk(ϕ)=(ϕ(−τ0),…,ϕ(−τk)), ϕ∈C, and the measureν(t)(B)=μt∘πτ0,…,τk−1(B)=μt{ϕ∈C:πτ0,…,τk(ϕ)∈B}, t≥0,B∈B(Rd(k+1)).Observe that ν(t) is the distribution of (x(t−τ0),…,x(t−τk)). By using the change-of-variables formula we obtain〈Lg,μt〉=∫Rd∫Rdk∑i=1dFi(x,y)∂g∂xi(x)+12∑i,j=1daij(x,y)∂2g∂xi∂xj(x)ν(t)(dx,y), where aij=(σσT)ij. Next, if fν is the density of the measure ν(t), assuming that it exists, with respect to the Lebesgue measure on Rd(1+k), i.e., ν(t)(dx,dy)=fν(x,y,t)dxdy, then f, the density of x(t), exists and it is the marginal density obtained from fν byf(x,t)=∫Rdkfν(x,y,t)dy.Consequently, Equation (88) rewritten as ∫Rdg(x)f(x,t)dx=∫Rdg(x)f(x,0)dx +∫0t∫Rd∫Rdk∑i=1dFi(x,y)∂g∂xi(x)+12∑i,j=1daij(x,y)∂2g∂xi∂xj(x)fν(x,y,r)dxdydr, is the weak form of the Fokker–Planck-type evolution equation∂∂tf(x,t)=−∑i=1d∂∂xi∫RdkFi(x,y)fν(x,y,t)dy+12∑i,j=1d∂2∂xi∂xj∫Rdaij(x,y)fν(x,y,t)dy.This is the central and desired result of this rather long and technical section. 5. Conclusions It is clear from the above considerations that the dynamic origin of the (weak or strong) second law of thermodynamics is far from obvious. If the dynamics are given by typical (time-reversal-invariant) ordinary differential equations the behaviour of the Gibbs entropy is unclear (8) while in the same circumstance the conditional entropy is constant (10). The presence of noise does not alter the indeterminate behaviour of the Gibbs entropy but does induce a monotonic approach of the conditional entropy to its equilibrium value of zero (9). The inclusion of temporal delays in the dynamics does not clarify things, but does increase the computational complexity of the situation. The Gibbs entropy might be monotone increasing (46) (strong form of the second law), whereas the conditional entropy may display a non-monotone approach (44) to equilibrium (47) (weak form of the second law). We have emphasized the somewhat primitive state of the development of the underlying theory of the evolution of densities in infinite dimensional (delayed) dynamical systems, and why correction of this must necessarily precede a satisfactory understanding of the effects of delayed dynamics on entropy behaviours. There is much to be done! Why Bother? One might wonder, given all of the technical problems involved in the treatment of density evolution in delayed systems, why is one bothering? The reason is straightforward. Biological processes are rife with delays modifying the dynamics, and models in many fields of the biological sciences contain significant delays at a variety of levels, e.g., ranging from gene regulation [44,45], through immunology [46], up to the cellular–clinical treatment level [47,48,49,50]. If we are ever to build a successful non-equilibrium thermodynamics for biological processes it will be essential to solve the problems pointed out in this study.