Stochastic dynamical systems

From Scholarpedia

This article is undergoing 1 initial review (1 completed); It may contain inaccuracies and unapproved changes made by anonymous reviewers.

Peer review status

You are not logged in.

Reviewer A: Awaits author's response (last modifications by author - 100 days ago)
Reviewer B: Modified and approved on 6 September 2007.
This article is not accepted to Scholarpedia yet. It may contain inaccuracies and unapproved changes made by anonymous reviewers.
Dr. Andre Longtin prefers old-fashioned peer-review process: all comments should be put into the 'reviews' part of the article.

Author: Dr. Andre Longtin, Physics Department, University of Ottawa, Ottawa, Canada

A stochastic dynamical system is a dynamical system subjected to the effects of noise. Such effects of fluctuations have been of interest for over a century since the seminal work of Einstein (1905). Fluctuations are classically referred to as "noisy" or "stochastic" when their suspected origin implicates the action of a very large number of variables or "degrees of freedom". For example, the action of many water molecules on the motion of a large protein can be seen as noise. In principle the equations of motion for such high-dimensional dynamics can be written and studied analytically and numerically. However, it is possible to study a system subjected to the action of the large number of variables by coupling its deterministic equations of motion to a "noise" that simple mimicks the perpetual action of many variables.

The coupling of noise to nonlinear deterministic equations of motion can lead to non-trivial effects (Schimansky-Geier 1985; Horsthemke 1985; H\"anggi, Talkner and Borkovec, 1990; H\"anggi and Marchesoni 2005). For example, noise can stabilize unstable equilibria and shift bifurcations, i.e. the parameter value at which the dynamics change qualitatively (Arnold 2003). Noise can lead to transitions between coexisting deterministic stable states or attractors. More interestingly still, noise can induce new stable states that have no deterministic counterpart. At the very least, noise excites internal modes of oscillation in both linear and nonlinear systems. In the latter case, it can even enhance the response of a nonlinear system to external signals (Jung, 1993; Gammaitoni et al., 1998; Lindner et al. 2004).

It is often thought that the action of noise merely amounts to a blurring of trajectories of the deterministic system. That is indeed the case for "observational" or "measurement" noise. However, in nonlinear systems where noise acts as a driving force, noise can drastically modify the deterministic dynamics. We discuss these issues using a basic level of description which couples a stochastic process to a deterministic equation of motion: the stochastic differential equation (SDE).

Contents

[edit]

Noise probability density, correlation time and intensity

Noise as a random variable \eta(t) is a quantity that fluctuates aperiodically in time. This variable takes on a different set of values everytime we sample it. However, to be a useful quantity to describe the real world, this random variable should have well-defined statistical properties, often experimentally accessible, such as a probability density of values \rho(\eta,t) and a two-point autocorrelation function C(t,s)=\langle \eta(t)\eta(s) \rangle. This function is related to the power spectral density of the process by Fourier transform, via the Wiener-Khintchine relations. Its characteristic decay time gives a measure of the noise "correlation time", i.e. of the level of correlation between successive values of the stochastic process. Here the brackets denote averaging over an infinite ensemble of possible realizations of the noise, in the theoretical case, or a finite ensemble in an experimental setting. Such ensemble averages can be replaced by time averages if the system is ergodic. The integral of the autocorrelation function over all times is the noise intensity.

Note also that the density can be time-dependent, some of its moments may not be defined, and the correlation function may depend on both its arguments, or one their difference if the system is stationary. Figure 1 shows different realizations of a noise process known as the Wiener process. The density \rho(\eta,t) can be constructed using a large number of realizations.

Stochastic dynamical systems
Enlarge
Figure 1: Two realizations of the Wiener process W(t). The density of this process is Gaussian with a variance equal to the time t. The time derivative of this non-stationary process is Gaussian white noise.
[edit]

Coupling the nosie to the dynamics

One of the difficulties with modeling noise is that we may not have access to the noise variable itself, but rather, to a state variable perturbed by one or more sources of noise. Thus, one may have to make assumptions about the nature of the noise and its coupling to the dynamical state variables. The accuracy of these assumptions can later be assessed by looking at the agreement between the predictions of the resulting model and the experimental data.

[edit]

Observation noise

In this case, the dynamical system evolves deterministically, but the measurements on this system are contaminated by noise. For example, suppose a one-dimensional dynamical system described by one state variable x with the following time evolution:

(1)
\frac{dx}{dt} = a(x,\mu )

where \mu is a parameter. Then observational noise corresponds to the measurement of y(t) \equiv F(x(t)) +\eta (t) where F is a static function. Here the measurement y, but not the evolution of the system x, is affected by the presence of noise. While this is often an important source of noise, and the simplest to deal with mathematically, it is also the most boring form of noise: it merely blurs deterministic solutions of x(t).

[edit]

Parametric noise

Noise can also affect the parameter \mu. This "parametric" noise can mimic the fluctuations in the environment of the system, such as pressure or reagent concentration. Mathematically, we can model this type of noisy forcing by setting \mu=<\mu>+\eta(t), where <\mu> denotes the constant average value of the coupling parameter and \eta(t) represents the fluctuations. The deterministic evolution equation Eq.1 becomes a stochastic differential equation (SDE) in the presence of the noisy forcing, and can lead to either of the following two possibilities.

[edit]

Additive noise

A classification exists according to how the noise and dynamical variable interact. If the coefficient of \eta in the evolution equation is independent of the state x of the system:

(2)
\frac{dx}{dt} = a(x,\langle\mu \rangle ) + \eta (t),

the noise is said to be "additive". In other words, the noise is simply added to the deterministic part of the dynamics.

[edit]

Multiplicative noise

Alternatively, one can have multiplicative noise, for which the coefficient of the noise depends on the value of one or many state variables. In such a case, the evolution equation would take the form:

(3)
\frac{dx}{dt} = a(x,\langle\mu\rangle)+g(x)\eta (t).

Now the strength of the noise is dependent on x, so if e.g. g(x(t)) is large at time t, the effect of noise will be large at that time.

[edit]

Commonly used noise processes

[edit]

Wiener process and Gaussian white noise

A commonly used simple process is the Gaussian white noise, denoted as \xi(t), where Gaussian refers to the shape of the distribution, whereas "white" refers to the fact that the correlation time is zero for t \neq s. Alternatively, this means that the Fourier transform of the autocorrelation (the power spectrum) is flat. Specifically, the autocorrelation is <\xi(t )\xi(t')>=\delta(t-t'). The noise \xi (t) is an extremely "spiky" looking function, which in fact is nowhere differentiable. It is the time derivative of the Wiener process (also known as Brownian motion), i.e.

\frac{dW}{dt} = \xi(t)

with W(0)=0 (by definition). Its intensity is the integral of the autocorrelation function, here equal to 1. This can be scaled by a factor D to achieve other intensities. This latter markov process begins at zero, and has Gaussian transition probabilities. Gaussian white noise is a good approximation to a colored noise process (see below) in the case where the characteristic time scales of the deterministic system are much larger than the noise correlation time. This is called the Quasi-White approximation.

[edit]

Ornstein-Uhlenbeck process

The Ornstein-Uhlenbeck process (OU) was proposed to model the velocity of a particle executing Brownian motion (its position is then obtained by integration). It is the only stationary Markovian process that is Gaussian and a diffusion process. Its realizations are continuous, and successive values are correlated exponentially. This latter property makes the OU process a "colored" noise. It is characterized by two parameters: its variance \sigma and its correlation time \tau. This noise is particularly useful to investigate the effects of the noise correlation time on the evolution of a nonlinear system. Different scalings of the OU process are used in the literature. A common scaling for the time evolution equation of the OU noise y is:

\frac{dy}{dt} = -\frac{y}{\tau} + \sqrt{\frac{2\sigma^2}{\tau}}\xi(t)

where \xi is a zero-mean Gaussian white noise with autocorrelation function \langle \xi(t)\xi(s)\rangle = \delta (t-s). The mean of y is zero, its density is

p(y) = \frac{1}{\sqrt{2\pi\sigma^2}} \exp \left[ -\frac{y^2}{2\sigma^2}\right] \,,

which is independent of the correlation time. Its autocorrelation function is:

C(t,s)=C(|t-s |)=\langle y(t)y(s)\rangle = \sigma^2 \exp\left[ -|t-s |/\tau \right]
Stochastic dynamical systems
Enlarge
Figure 2: Two realizations of the Ornstein-Uhlenbeck process for parameters \tau=1.0 and \sigma=0.1 (black curve), and for \tau=0.1 and \sigma=0.31622 (red curve). In both cases the noise intensity is \sigma^2\tau=0.01. The red curve represents a noise that more closely mimicks Gaussian white noise. Both realizations begin here at x(0)=1.0, after which the mean decays exponentially to zero with time constant \tau.

The noise intensity is the integral of the autocorrelation over all times - and by the Wiener-Khintchine relations, the value of the power spectral density at the origin S(0). For the OU process in this scaling, this intensity is \sigma^2\tau. The variance of the OU process is equal to C(0), i.e. the integral of the power spectral density; here it is equal to \sigma^2. One can explore limits of the parameters by keeping either the intensity or the variance constant. For example, in Fig.2 are shown two realizations of the OU process for two values of the noise correlation time, at constant intensity; continuing this process for smaller correlation times yields the Gaussian white noise limit.

[edit]

Poisson dichotomous noise

Another important type of noise is the Poisson dichotomous process, characterized by a discrete two-state space \xi(t)=\pm \Delta with an exponentially decaying autocorrelation function. It is also referred to as the random telegraph signal. This type of noise is thus described by its amplitude \Delta and its correlation time \tau. One of its advantages is to lead to an exact stationary probability density for one-variable dynamical systems (see e.g. L'Heureux and Kapral 1988). In this case it is possible to obtain a linear equation which describes the time-evolution of the probability density. This equation is more complicated than the Fokker-Planck equation (see below), but the stationary density can still be obtained analytically, at least for the case where there is only one dynamical variable. In contrast, no exact evolution equation for the probability density of the state variable can be obtained for the case of the OU noise.

[edit]

Ito or Stratonovich calculus

This distinction concerns stochastic differential equations that involve Gaussian white noise. A rewriting of a stochastic differential equation in integral form yields

x(t) = x(0) + \int_0^t dt' a(x(t'),t')+\int_0^t dW(t') b(x(t'),t') \,.

There are two interpretations of the stochastic integral on the right hand side (see e.g. Gardiner 1985), known as the Ito and Stratonovich interpretations. It is important to establish what interpretation or "calculus" one assumes at the outset of an analysis. Further, it is necessary to use a numerical integration method (Kloeden and Platen 1992) that is compatible with this calculus in order to match up simulation to theory. The Stratonovich calculus obeys the usual laws of calculus (such as for changes of variables), but this is not the case for the Ito calculus. Nevertheless it is possible to convert from one form of calculus to the other, and to restate an SDE in one form into the other (as well as their corresponding Fokker-Planck equations - see below). The properties obtained with both calculi are identical when the Gaussian white noise is additive.

[edit]

Analysis of a stochastic dynamical system

One may be interested in the behavior of individual trajectories, features of which can be compared to those from experimental measurements, or in the evolution of probability densities. The SDE approach is concerned with the former, and involves either exact or approximate analytical solutions, or numerical solutions; the Fokker-Planck approach (or more generally, the Chapman-Kolmogorov approach - see below) focusses on time-dependent probability densities. The SDE approach can also be used to compute densities relevant to the latter appraoch.

[edit]

Numerical integration

The SDE of the type seen in Eq.3 defines a "Langevin equation", which is particularly interesting in the case where f(x,\langle\mu\rangle) is nonlinear in x. One can numerically integrate such a nonlinear Langevin equation with flow f(x,\langle\mu\rangle ) using a simple Euler-Maruyama method (Sancho et al. 1982; Kloeden and Platen 1992) with a fixed time step \Delta t:

x(t+\Delta t) = x(t) + a(x,\langle\mu\rangle,t) \Delta t + g(x,t)\sqrt{\Delta t}\, \Delta W_n.

where we have added an explicit time dependence in the functions a and g for generality. The variables {\Delta W_n} are known as increments of the Wiener process; they are Gaussian numbers generated in an uncorrelated fashion, for example by using a pseudo-random number generator in combination with the Box-M\"uller algorithm. Such generators are in fact high-dimensional chaotic systems. Such algorithms must be "seeded", i.e., provided with an initial condition. One of the results of the theory of Gaussian white noise is that the random term in Eq.4 is multiplied by the square root of the time step rather than by the time step itself. Appropriate averages of the relevant properties can be computed over many realizations, each with a different seed. More precise numerical algorithms for solving SDE's are also available, and are easily generalized to multiplicative noise processes of the type shown in Eq.3 (see e.g. Sancho et al., 1982; Kloeden and Platen, 1992; Honeycutt and Fox 1992).

[edit]

Fokker-Planck equation

One can study how an ensemble of initial conditions, characterized by an initial density, propagates under the action of the SDE Eq.3. The evolution of this density is governed by a deterministic partial differential equation in the density variable \rho (x,t), known as the Fokker-Planck equation (Gardiner 1985, Risken 1989). In one-dimension, this equation reads:

(4)
\frac{\partial \rho}{\partial t} = \frac{1}{2} \frac{\partial^2\left[ g^2(x)\rho\right]}{\partial x^2} - \frac {\partial\left[ a(x)\rho\right]}{\partial x}

Note that the probabilistic aspect of the problem has been shifted from individual realizations of a SDE to a probability density that evolves according to a deterministic linear evolution equation.

Setting the left hand side of Eq.4 to zero and solving the resulting ordinary differential equation yields the stationary density, \rho_s \equiv \rho (x,\infty ). For simple systems, it is possible to calculate \rho_s, and sometimes even \rho (x,t). However, for general nonlinear problems, usually one can at best approximate \rho_s. In either cases, a knowledge of the stationary probability density gives us a wealth of statistical information in the asymptotic regime. One can also solve the Fokker-Planck equation numerically, or adopt the Langevin approach, and estimate the density directly from realizations of the SDE.

[edit]

Theoretical basis

[edit]

Chapman-Kolmogorov equation and Markov process

The foundation for the study of stochastic dynamical systems is built on the Chapman-Kolmogorov equation which, for a continuous state space for the variable x and the time ordering t_1\ge t_2 \ge t_3, reads:

p(x_1,t_1|x_3,t_3)=\int \,dx_2 p(x_1,t_1|x_2,t_2) p(x_2,t_2|x_3,t_3) \,.

It embodies the Markov assumption according to which singly (as opposed to multiply) conditioned probabilities are sufficient to compute transition probabilities in state space. This is a statement of the fact that only the present state is necessary to compute the future state; the past is irrelevant for this computation. This Markov assumption is not strictly valid in any physical setting, where the immediate history will play a role in the future evolution (see Gardiner 1985; Risken 1989). Nevertheless, the mathematical idealization that is the Markov process is useful for describing reality.

The differential form of the Chapman-Kolmogorov equation can also studied. For a one-dimensional system it reads (multivariate forms can be found in Gardiner 1985; Risken 1989):

\frac{\partial p(z,t|y,s)}{\partial t} = - \frac{\partial}{\partial z} \left[A(z,t)p(z,t|y,s)\right] + \frac{1}{2} \frac{\partial^2}{\partial z^2} \left[ B(z,t) p(z,t|y,s)\right] + \int\, dx W(z|x,t) p(x,t|y,s) - \int\, dx W(x|z,t)p(z,t|y,s) \,\,,

where the conditional quantity W(x|z,t) must be distinguished from the Wiener process used above. This equation assumes the following definitions, with \epsilon=|x-z |:

\lim_{\Delta t\rightarrow 0} p(x,t+\Delta t|z,t)/\Delta t = W(x,t)
\lim_{\Delta t\rightarrow 0} \frac{1}{\Delta t} \int_{\epsilon} dx (x-z) p(x,t+\Delta t|z,t) = A(z,t) + O(\epsilon )
\lim_{\Delta t\rightarrow 0} \frac{1}{\Delta t} \int_{\epsilon} dx (x-z)^2 p(x,t+\Delta t|z,t) = B(z,t)+O(\epsilon ) \,.

It can be shown that the quantities A and B are associated with continuous motion, while W is associated with discontinuous motion.

[edit]

Master, Liouville and Fokker-Planck equation

The differential form of the Chapman-Kolmogorov equation in which A(z,t)=B(z,t)=0 is known as a Master equation. The system admits solutions which are constants separated by finite jumps with density W(x|z,t). The jumps occur at discrete time points, and accordingly this process is sometimes called a jump process.

For B=W=0, the differential form of the Chapman-Kolmogorov equation is known as the Liouville equation, which describes completely deterministic motion as in classical mechanics.

If the quantity W(x|z,t) is zero, the differential form of the Chapman-Kolmogorov equation reduces to the Fokker-Planck equation. The process then has continuous paths. The quantities A and B are known as the drift and diffusion of the process, identified respectively with the functions a and g^2 in Eq.3.

[edit]

Noise-induced states

Noise-induced states are a nontrivial effect of noise. Their study requires the prior definition of the notion of a "state" in a stochastic sense, distinct from that of a "state variable". A stochastic state is the analogue of an attractor in a deterministic dynamical system. Specifically, it is the value of the dynamical variable for which the stationary probability distribution is a maximum. There can be more than one such state. These states may be the same as in the noiseless case. However, the positions and even the number of stochastic states may differ from the deterministic case.

In the case where the number of stochastic states is larger than the number of stable deterministic fixed points, one speaks of the creation of stochastic states by noise. Examples across a variety of disciplines in the natural sciences can be found in the book by Horsthemke and Lefever (1984), and in Schimansky-Geier et al. (1985), Hanggi et al (2005). Generally noise reveals the presence of nearby bifurcations by producing behavior that is stereotyped for a given bifurcation (Wiesenfeld 1985). It can produce stochastic versions of various deterministic phenomena such as phase locking (Longtin and Chialvo, 1998) as in neurons and other excitable systems, in which also can create very long time scales, e.g., a "noise-induced" memory, (Chialvo et al., 2000).

[edit]

Noise as chaos

As mentioned in the introduction, noise can have high-dimensional deterministic origins. In fact, a pseudo-random number generator is one such system operating in discrete time, i.e. it is a high-dinemsional map operating in a chaotic regime. Chaotic systems share many properties with noisy systems (Lasota and Mackey, 1994), such as their ability to synchronize (Pikovsky et al., 2001). A recent review of the effect of chaotic dynamics as "deterministic brownian motion" on other dynamical systems can be found in Mackey and Tyran-Kaminska (2006).

[edit]

References

  • A Einstein, Ann. Physik 17, 549, 1905.
  • P Hänggi, P Talkner and M Borkovec, Rev. Mod. Phys. 62, 251, 1990.
  • I L'Heureux and R Kapral, Transition rates in a bistable system driven by external dichotomous noise, J. Chem. Phys. 88, 7468, 1988.
  • RL Honeycutt, Stochastic Runge-Kutta algorithms. I. White noise, Phys. Rev. A 45, 600, 1992; II. Colored noise, Phys. Rev. A 45, 604, 1992.
  • DR Chialvo, G Cecchi, M Magnazco, Noise-induced memory in extended excitable systems. Phys Rev. E. 61,5656, 2000.
  • P Jung, Periodically driven stochastic systems. Phys. Rep. 234, 175, 1993.
  • B Lindner, J Garcia-Ojalvo, A Neiman and L Schimansky-Geier, Effects of noise in excitable systems. Phys. Rep. 392, 321, 2004.
  • A Longtin and D Chialvo, Stochastic and deterministic resonances in excitable systems. Phys. Rev. Lett. 81, 4012, 1998.
  • JM Sancho, M San Miguel, SL Katz and JD Gunton, Analytical and numerical studies of multiplicative noise, Phys. Rev. A 26, 1589, 1982. An early seminal study of stochastic numerical integration.
  • L Schimansky-Geier, AV Tolstopjatenko and W Ebeling, Noise induced transitions due to external additive noise. Phys. Lett. A 108, 329, 1985.
  • K Wiesenfeld, Noisy precursors of nonlinear instabilities. J. Stat. Phys. 38, 1071, 1985.
  • Books and reviews
    • VS Anishchenko, VV Astakhov, AB Neiman, TE Vadivasova, and L Schimansky-Geier, Nonlinear Dynamics of Chaotic and Stochastic Systems. (Springer Verlag, Berlin, 2002).
    • L Arnold, Random Dynamical Systems. (Springer-Verlag, Berlin, 1998). Corrected 2nd printing 2003.
    • J Garcia-Ojalvo and JM Sancho, Noise in Spatially Extended Systems. (Springer Verlag, Berlin, 1999).
    • J Honerkamp, Stochastic Dynamical Systems (VCH, Weinheim, 1994).
    • PE Kloeden and E Platen, Numerical Solution of Stochastic Differential Equations. (Springer Verlag, Berlin, 1992).
    • A Lasota and MC Mackey, Chaos, Fractals and Noise. Stochastic Aspects of Dynamics. Second Ed. (Springer-Verlag, Berlin, 1994).
    • MC Mackey and M Tyran-Kaminska, Deterministic Brownian motion: The effects of perturbing a dynamical system by a chaotic semi-dynamical system. Physics Rep. 422, 167-222, 2006.
    • H Risken, The Fokker-Planck Equation. Methods of Solution and Applications. (Springer Verlag, Berlin, 1989).
    • NG van Kampen. Stochastic Processes in Physics and Chemistry. (Elsevier, New York, 1990).
[edit]

Recommended reading

  • A recent and very useful review of stochastic resonance phenomena:
    • L Gammaitoni, P Hänggi, P Jung and F Marchesoni, Stochastic resonance, Rev. Mod. Phys. 70, 223, 1998.
  • A classic book in which the mathematics of stochastic processes and the Fokker-Planck apparatus are clearly (for a physicist) developed:
    • CW Gardiner, Handbook of Stochastic Methods for Physics, Chemistry and the Natural Sciences. Third edition (Springer-Verlag, Berlin, 2004). This edition also has a chapter on simulations of stochastic differential equations.
  • A Special Issue devoted to stochastic dynamics:
    • P Hänggi and F Marchesoni, Introduction: 100 years of Brownian motion. Chaos 15, 026101, 2005.
  • A classic book in the field with an emphasis on the existence of noise-induced states in many nonlinear systems:
    • W Horsthemke and R Lefever, Noise-Induced Transitions. Theory and Applications in Physics, Chemistry and Biology. (Springer-Verlag, Berlin, 1984).
  • This book discusses many aspects of stochastic forcing of dynamical systems:
    • A Pikovsky, M Rosenblum and J Kurths, Synchronization. A Universal Concept in Nonlinear Science. (Cambridge University Press, Cambridge, 2001).
[edit]

External links

[edit]

See also

Action editor: Dr. Eugene M. Izhikevich, Editor-in-Chief of Scholarpedia, the peer-reviewed open-access encyclopedia
Retrieved from "http://www.scholarpedia.org/article/Stochastic_dynamical_systems"
For authors