text:
# Predictability: a way to characterize Complexity

## 1 Introduction

The ability to predict the future state of a system, given the present one, stands at the foundations of scientific knowledge with relevant implications from a conceptual and applicative point of view. The knowledge of the evolution law of the system may induce one to conclude that this aim has been attained. This is the classical deterministic point of view as clearly stated by Laplace : once the evolution laws of the system are known, the state at a certain time $`t_0`$ completely determines the subsequent states for every time $`t>t_0`$111In this review we shall always consider the usual setting where a system is studied by an external observer, so as to avoid the problem of the self-prediction .. However it is well established now that this cannot be accomplished in practice.

One limitation occurs in systems with many degrees of freedom, namely the impossibility to manage the huge amount of data required for a detailed description of a single state of a macroscopic body. This aspect, which is not discussed in this review, has led to the development of statistical mechanics.

Another source of difficulty, which arises even in low dimensional systems, is related to the unavoidable uncertainty in the initial condition. As clearly stated by Poincaré, this implies that one can make long-time predictions only if the evolution law does not amplify the initial uncertainty too rapidly. This aspect had a relevant role in the development of the theory of dynamical chaos.

Therefore, from the point of view of predictability, we need to know how an error in the initial state of the system grows in time. In deterministic chaotic systems, i.e., with sensitive dependence on initial condition, one has an exponential growth of errors and, consequently, a severe limitation on the ability to predict the future states. In addition, since the details of the evolution laws are not completely known (or, at least, cannot be specified with an arbitrary accuracy) or some degrees of freedom cannot be resolved, one has another unavoidable source of unpredictability. This is also true for systems with discrete states.

A branch of the theory of dynamical systems has been developed with the aim of formalizing and quantitatively characterizing the sensitivity to initial conditions. The Lyapunov exponent and the Kolmogorov-Sinai entropy are the two indicators for measuring the rate of error growth and information produced by the dynamical system. A complementary approach has been developed in the context of information theory, data compression and algorithmic complexity theory. Nowadays it is rather clear that the latter point of view is closely related to the dynamical systems one. If a system is chaotic then the predictability is limited up to a time which is related to the first Lyapunov exponent, and the time sequence generated from one of its chaotic trajectories cannot be compressed by an arbitrary factor, i.e. is algorithmically complex. On the contrary, a regular trajectory can be easily compressed (e.g., for a periodic trajectory it is sufficient to have the sequence for a period) so it is “simple”.

In this review we will discuss how these points of view are related and how they complete each other in giving a quantitative understanding of complexity arising in dynamical systems. In particular, we shall consider the extension of this approach, nowadays well established in the context of low dimensional systems and for asymptotic regimes, to high dimensional systems with attention to situations far from asymptotic (i.e. finite time and finite observational resolution).

It is worth remarking that the approach to complexity here discussed is not able to cover the many aspects of what in recent years has been indicated under this term . Indeed complexity has been considered in many different fields of science and its meaning has become (sometimes) vague. A challenging problem in this direction is the definition of indicators which are able to fulfill the intuitive idea of complexity, namely one looks for quantities which give a low complexity value both for pure random sequences and completely regular ones . Even if very interesting this last issue is not addressed here: from the point of view of predictability both a chaotic system and a purely random one are highly complex, i.e. unpredictable.

The review is organized as follows. Sect. 2 is devoted to the introduction of the basic concepts and ideas of dynamical systems, information theory and algorithmic complexity. In particular, we discuss the relations among Lyapunov exponents, Kolmogorov-Sinai entropy and algorithmic complexity and their relevance for predictability. All these quantities are properly defined only in specific asymptotic limits, that are: very long times and arbitrary accuracy. Since in realistic situations one has to deal with finite accuracy and finite time — as Keynes said, “in the long run we shall all be dead” — it is appealing to treat the predictability problem by taking into account these limitations. This is the subject of Sect. 3 where, relaxing the request of infinite time, we discuss the relevance of the finite time fluctuations of the “effective” Lyapunov exponent. In addition, relaxing the limit of infinitesimal perturbations, we introduce suitable tools, such as the Finite Size Lyapunov Exponent (FSLE) and the $`ϵ`$-entropy, for the treatment of non arbitrary high accuracy, i.e. non infinitesimal perturbations.

Sects. 4 and 5 focus on high dimensional dynamical systems which deserve particular attention. Indeed because of the many degrees of freedom, and its interest in applications (e.g. in weather forecasting), it is necessary to consider the detailed behavior of perturbations and not only the asymptotic features (i.e. long time and infinitesimal amplitudes). Sect. 5 is devoted to fully developed turbulence (here introduced as an important prototype of high dimensional system) and its finite resolution properties in the inertial range.

In Sect. 6 we consider the effects of uncertainties on the evolution laws and we discuss systems containing some randomness. In such a situation there are two ways to approach the predictability: by considering either two trajectories generated with the same realization of randomness, or two trajectories evolving with different realizations. Both approaches are physically relevant in different contexts, and the results can be very different in presence of strong intermittency.

For the sake of completeness in Sect. 7 we discuss dynamical systems with discrete states, e.g., Cellular Automata.

Sect. 8 is dedicated to a discussion on data analysis. In particular we discuss the use of $`ϵ`$-entropy and FSLE for a pragmatic classification of signals.

Sect. 9 reports some concluding remarks. In the Appendices we discuss some more technical details.

## 2  Two points of view

### 2.1  Dynamical systems approach

Two standard – tightly linked – indicators are largely used to quantify the behavior of a dynamical system with respect to the asymptotic evolution of an infinitesimal uncertainty: the largest Lyapunov exponent (LE) and the Kolmogorov-Sinai (or metric) entropy .

#### 2.1.1  Characteristic Lyapunov exponents

The characteristic Lyapunov exponents are somehow an extension of the linear stability analysis to the case of aperiodic motions. Roughly speaking, they measure the typical rate of exponential divergence of nearby trajectories. In this sense they give information on the rate of growth of a very small error on the initial state of a system.

Consider a dynamical system with an evolution law given, in the case of continuous time, by the differential equation

$$\frac{d𝐱}{dt}=𝐅(𝐱),$$
(2.1)
or, in the case of discrete time, by the map

$$𝐱(t+1)=𝐆(𝐱(t)).$$
(2.2)
In both cases, for simplicity, we suppose that a vector $`𝐱\mathrm{I}R^d`$ uniquely specifies one state of the system. We also assume that $`𝐅`$ and $`𝐆`$ are differentiable functions, that the evolution is well-defined for time intervals of arbitrary extension, and that the motion takes place in a bounded region of the phase space. We intend to study the separation between two trajectories, $`𝐱(t)`$ and $`𝐱^{}(t)`$, starting from two close initial conditions, $`𝐱(0)`$ and $`𝐱^{}(0)=𝐱(0)+\delta 𝐱(0)`$, respectively.

As long as the difference between the trajectories, $`\delta 𝐱(t)=𝐱^{}(t)𝐱(t)`$, remains small (infinitesimal, strictly speaking), it can be regarded as a vector, $`𝐳(t)`$, in the tangent space. The time evolution of $`𝐳(t)`$ is given by the linearized differential equations:

$$\frac{dz_i(t)}{dt}=\underset{j=1}{\overset{d}{}}\frac{F_i}{x_j}|_{𝐱(t)}z_j(t)$$
(2.3)
or, in the case of discrete time maps:

$$z_i(t+1)=\underset{j=1}{\overset{d}{}}\frac{G_i}{x_j}|_{𝐱(t)}z_j(t).$$
(2.4)
Under rather general hypothesis, Oseledec  proved that for almost all initial conditions $`𝐱(0)`$, there exists an orthonormal basis $`\{𝐞_i\}`$ in the tangent space such that, for large times,

$$𝐳(t)=\underset{i=1}{\overset{d}{}}c_i𝐞_ie^{\lambda _it},$$
(2.5)
where the coefficients $`\{c_i\}`$ depends on $`𝐳(0)`$. The exponents $`\lambda _1\lambda _2\mathrm{}\lambda _d`$ are called characteristic Lyapunov exponents. If the dynamical system has an ergodic invariant measure, the spectrum of LEs $`\{\lambda _i\}`$ does not depend on the initial condition, except for a set of measure zero with respect to the natural invariant measure.

Loosely speaking, (2.5) tells us that in the phase space, where the motion evolves, a $`d`$-dimensional sphere of small radius $`ϵ`$ centered in $`𝐱(0)`$ is deformed with time into an ellipsoid of semi-axes $`ϵ_i(t)=ϵ\mathrm{exp}(\lambda _it)`$, directed along the $`𝐞_i`$ vectors. Furthermore, for a generic small perturbation $`\delta 𝐱(0)`$, the distance between a trajectory and the perturbed one behaves as

$$|\delta 𝐱(t)||\delta 𝐱(0)|e^{\lambda _1t}\left[1+O\left(e^{(\lambda _1\lambda _2)t}\right)\right].$$
(2.6)
If $`\lambda _1>0`$ we have a rapid (exponential) amplification of an error on the initial condition. In such a case, the system is chaotic and, de facto, unpredictable on the long times. Indeed, if we put $`\delta _0=|\delta 𝐱(0)|`$ for the initial error, and we want to predict the states of the system with a certain tolerance $`\mathrm{\Delta }`$ (not too large), then the prediction is possible just up to a predictability time given by

$$T_p\frac{1}{\lambda _1}\mathrm{ln}\left(\frac{\mathrm{\Delta }}{\delta _0}\right).$$
(2.7)
This equation shows that $`T_p`$ is basically determined by the largest Lyapunov exponent, since its dependence on $`\delta _0`$ and $`\mathrm{\Delta }`$ is very weak. Because of its preeminent role, very often one simply refers to $`\lambda _1`$ as “the Lyapunov exponent”, and one indicates it with $`\lambda `$.

Eq. (2.6) suggests how to numerically compute $`\lambda _1`$. We introduce the response, after a time $`t`$, to a perturbation on $`𝐱(\tau )`$, defined as follows:

$$R_\tau (t)\frac{|𝐳(\tau +t)|}{|𝐳(\tau )|}=\frac{|\delta 𝐱(\tau +t)|}{|\delta 𝐱(\tau )|},$$
(2.8)
where, again, $`|\delta 𝐱(\tau )|`$ and $`|\delta 𝐱(\tau +t)|`$ are infinitesimal. The LE $`\lambda _1`$ is obtained by averaging the logarithm of the response over the initial conditions or along the trajectory:

$$\lambda _1=\underset{t\mathrm{}}{lim}\frac{1}{t}\mathrm{ln}R_\tau (t),$$
(2.9)
where $``$ denotes the time average $`lim_T\mathrm{}(1/T)_{\tau _0}^{\tau _0+T}()𝑑\tau `$. The Oseledec’s theorem implies that $`(1/t)\mathrm{ln}R_\tau (t)`$, for large $`t`$, is a non-random quantity, i.e. for almost all the initial conditions its value does not depend on the specific initial condition. Therefore, for large times, the average in (2.9) can be neglected.

As the typical growth rate of a generic small segment in phase space is driven by the largest LE, the sum of the first $`n`$ ($`d`$) Lyapunov exponents controls the variations of small $`n`$-dimensional volumes in phase space. This gives us a way to compute the sub-leading Lyapunov exponents. After the selection of $`nd`$ non parallel tangent vectors $`[𝐳^{(1)}(0),\mathrm{},𝐳^{(n)}(0)]`$, one introduces the $`n`$-order response $`R_\tau ^{(n)}(t)`$

$$R_\tau ^{(n)}(t)\frac{|𝐳_1(t+\tau )\times 𝐳_2(t+\tau )\times \mathrm{}\times 𝐳_n(t+\tau )|}{|𝐳_1(\tau )\times 𝐳_2(\tau )\times \mathrm{}\times 𝐳_n(\tau )|}.$$
(2.10)
Analogously to the LE, it can be shown that

$$\underset{i=1}{\overset{n}{}}\lambda _i=\underset{t\mathrm{}}{lim}\frac{1}{t}\mathrm{ln}R_\tau ^{(n)}(t).$$
(2.11)
Let us stress that the Lyapunov exponents give information on the typical behaviors along a generic trajectory, followed for infinite time and keeping the perturbation infinitesimally small. In this respect, they are global quantities characterizing fine-grained properties of a system.

#### 2.1.2  Kolmogorov-Sinai entropy

The LE, $`\lambda `$, gives a first quantitative information on how rapidly we loose the ability of predicting the evolution of a system. A state, initially determined with an error $`\delta 𝐱(0)`$, after a time enough larger than $`1/\lambda `$, may be found almost everywhere in the region of motion. In this respect, the Kolmogorov-Sinai (KS) entropy, $`h_{KS}`$, supplies a more refined information. The error on the initial state is due to the maximal resolution we use for observing the system. For simplicity, let us assume the same resolution $`ϵ`$ for each degree of freedom. We build a partition of the phase space with cells of volume $`ϵ^d`$, so that the state of the system at $`t=t_0`$ is found in a region of volume $`V_0=ϵ^d`$ around $`𝐱(t_0)`$. Now we consider the trajectories starting from $`V_0`$ at $`t_0`$ and sampled at discrete times $`t_j=j\tau `$ ($`j=1,2,3,\mathrm{},t`$); in the case of a map one can put $`\tau =1`$. Since we are considering motions that evolve in a bounded region, all the trajectories visit a finite number of different cells, each one identified by a symbol. In this way a unique sequence of symbols $`\{s(0),s(1),s(2),\mathrm{}\}`$ is associated with a given trajectory. In a chaotic system, although each evolution $`𝐱(t)`$ is univocally determined by $`𝐱(t_0)`$, a great number of different symbolic sequences originates by the same initial cell, because of the divergence of nearby trajectories. The total number of the admissible symbolic sequences, $`\stackrel{~}{N}(ϵ,t)`$, increases exponentially with a rate given by the topological entropy

$$h_T=\underset{ϵ0}{lim}\underset{t\mathrm{}}{lim}\frac{1}{t}\mathrm{ln}\stackrel{~}{N}(ϵ,t).$$
(2.12)
However, if we consider only the number of sequences $`N_{eff}(ϵ,t)\stackrel{~}{N}(ϵ,t)`$ which appear with very high probability in the long time limit – those that can be numerically or experimentally detected and that are associated with the natural measure – we arrive at a more physical quantity called the Kolmogorov-Sinai or metric entropy :

$$h_{KS}=\underset{ϵ0}{lim}\underset{t\mathrm{}}{lim}\frac{1}{t}\mathrm{ln}N_{eff}(ϵ,t)h_T.$$
(2.13)
$`h_{KS}`$ quantifies the long time exponential rate of growth of the number of the effective coarse-grained trajectories of a system. This suggests a link with information theory where the Shannon entropy measures the mean asymptotic growth of the number of the typical sequences – the ensemble of which has probability almost one – emitted by a source. In the following we will discuss in more detail the KS-entropy and its relation with the information theory. Here we obtain, by means of a heuristic reasoning, the relation among $`h_{KS}`$ and Lyapunov exponents.

We may wonder what is the number of cells where, at a time $`t>t_0`$, the points that evolved from $`V_0`$ can be found, i.e. we wish to know how big is the coarse-grained volume $`V(ϵ,t)`$, occupied by the states evolved from $`V_0`$, if the minimum volume we can observe is $`ϵ^d`$. As stated at the end of the preceding subsection, we have $`V(t)V_0\mathrm{exp}(t_{i=1}^d\lambda _i)`$. However, this is true only in the limit $`ϵ0`$. In this (unrealistic) limit, $`V(t)=V_0`$ for a conservative system (where $`_{i=1}^d\lambda _i=0`$) and $`V(t)<V_0`$ for a dissipative system (where $`_{i=1}^d\lambda _i<0`$). As a consequence of limited resolution power, in the evolution of the volume $`V_0=ϵ^d`$ the effect of the contracting directions (associated with the negative Lyapunov exponents) is completely lost. We can experience only the effect of the expanding directions, associated with the positive Lyapunov exponents. As a consequence, in the typical case, the coarse grained volume behaves as

$$V(ϵ,t)V_0e^{(_{\lambda _i>0}\lambda _i)t},$$
(2.14)
when $`V_0`$ is small enough. Since $`N_{eff}(ϵ,t)V(ϵ,t)/V_0`$, one has

$$h_{KS}=\underset{\lambda _i>0}{}\lambda _i.$$
(2.15)
This argument can be made more rigorous with a proper mathematical definition of the metric entropy. In this case one derives the Pesin relation

$$h_{KS}\underset{\lambda _i>0}{}\lambda _i.$$
(2.16)
Because of its relation with the Lyapunov exponents – or by the definition (2.13) – it is clear that also $`h_{KS}`$ is a fine-grained and global characterization of a dynamical system.

The metric entropy is an invariant characteristic quantity of a dynamical system , i.e. given two systems with invariant measures, their KS-entropies exist and they are equal if the systems are isomorphic . This intrinsic quantity may be properly defined by means of tools borrowed from the mathematical theory of communication.

### 2.2  Information theory approach

In experimental investigations of physical processes, we typically have access to the system only trough a measuring device which produces a time record of a certain observable, i.e. a sequence of data. In this regard a system, whether or not chaotic, generates messages and may be regarded as a source of information. This observation opens the possibility to study dynamical systems from a very interesting point of view.

Information has found a proper characterization in the framework of the theory of communication to cope with the practical problem of transmitting a message in the cheapest way without losing information. The characterization of the information contained in a sequence can be approached by two very different points of view. The first one, that of information theory , is a statistical approach, i.e., it does not consider the transmission of a specific message (sequence) but refers to the statistical properties of all the messages emitted by the source. Information theory approach characterizes the source of information, so that it gives us a powerful method to characterize chaotic systems.

The second point of view considers the problem of characterizing a single sequence. This latter has led to the theory of algorithmic complexity and algorithmic information theory .

#### 2.2.1 Shannon entropy

At the end of forties Shannon  introduced rather powerful concepts and techniques for a systematic study of sources emitting sequences of discrete symbols (e.g. binary digit sequences). Originally information theory was introduced in the practical context of electric communications, nevertheless in a few years it became an important branch of both pure and applied probability theory with strong relations with other fields as computer science, cryptography, biology and physics .

For the sake of self-consistency we briefly recall the basic concepts and ideas about the Shannon entropy. Consider a source that can output $`m`$ different symbols; denote with $`s(t)`$ the symbol emitted by the source at time $`t`$ and with $`P(C_N)`$ the probability that a given word $`C_N=(s(1),s(2),\mathrm{},s(N))`$, of length $`N`$, is emitted:

$$P(C_N)=P(s(1),s(2),\mathrm{},s(N)).$$
(2.17)
We assume that the source is stationary, so that for the sequences $`\{s(t)\}`$ the time translation invariance holds: $`P(s(1),\mathrm{},s(N))=P(s(t+1),\mathrm{},s(t+N))`$.

Now we introduce the $`N`$-block entropies

$$H_N=\underset{\{C_N\}}{}P(C_N)\mathrm{ln}P(C_N),$$
(2.18)
and the differential entropies

$$h_N=H_{N+1}H_N,$$
(2.19)
whose meaning is the average information supplied by the $`(N+1)`$-th symbol, provided the $`N`$ previous ones are known. One can also say that $`h_N`$ is the average uncertainty about the $`(N+1)`$-th symbol, provided the $`N`$ previous ones are given. For a stationary source the limits in the following equations exist, are equal and define the Shannon entropy $`h_{Sh}`$:

$$h_{Sh}=\underset{N\mathrm{}}{lim}h_N=\underset{N\mathrm{}}{lim}\frac{H_N}{N}.$$
(2.20)
The $`h_N`$ are non increasing quantities: $`h_{N+1}h_N`$; that is: the knowledge of a longer past history cannot increase the uncertainty on the next outcome. In the case of a $`k`$-th order Markov process $`h_N=h_{Sh}`$ for all $`Nk`$. This is because a $`k`$-th order Markov process has the property that the conditional probability to have a given symbol only depends on the results of the last $`k`$ times, i.e.

$`P(s(t)|s(t1),s(t2),\mathrm{})=P(s(t)|s(t1),s(t2),\mathrm{},s(tk)).`$
The Shannon entropy is a measure of the “surprise” the source emitting the sequences can reserve to us, since it quantifies the richness (or “complexity”) of the source. This can be precisely expressed by the first theorem of Shannon-McMillan  that applies to stationary ergodic sources:
If $`N`$ is large enough, the ensemble of $`N`$-long subsequences can be partitioned in two classes, $`\mathrm{\Omega }_1(N)`$ and $`\mathrm{\Omega }_0(N)`$ such that all the words $`C_N\mathrm{\Omega }_1(N)`$ have the same probability $`P(C_N)\mathrm{exp}(Nh_{Sh})`$ and

$$\underset{C_N\mathrm{\Omega }_1(N)}{}P(C_N)1\text{ for }N\mathrm{}$$
(2.22)
while

$$\underset{C_N\mathrm{\Omega }_0(N)}{}P(C_N)0\text{ for }N\mathrm{}.$$
(2.23)
The meaning of this theorem is the following. An $`m`$-states process admits in principle $`m^N`$ possible sequences of length $`N`$, but the number of typical sequences $`N_{eff}(N)`$ (those ones in $`\mathrm{\Omega }_1(N)`$) effectively observable is

$$N_{eff}(N)\mathrm{exp}(Nh_{Sh}).$$
(2.24)
Note that $`N_{eff}m^N`$ if $`h_{Sh}<\mathrm{ln}m`$. Moreover the entropy per symbol, $`h_{Sh}`$, is a property of the source. Because of the ergodicity it can be obtained by analyzing just one single sequence in the ensemble of the typical ones, and it can also be viewed as a property of each one of the typical sequences. Therefore, as in the following, one may speak about the Shannon entropy of a sequence.

The above theorem in the case of processes without memory is nothing but the law of large numbers. Let us observe that (2.24) is somehow the equivalent in information theory of the Boltzmann equation in statistical thermodynamics: $`S\mathrm{ln}W`$, being $`W`$ the number of possible microscopic configurations and $`S`$ the thermodynamic entropy. This justifies the name “entropy” for $`h_{Sh}`$. Under rather natural assumptions it is possible to prove that the Shannon entropy, apart from a multiplicative factor, is the unique quantity which characterizes the “surprise” .

Let us now mention another important result about the Shannon entropy. It is not difficult to recognize that the quantity $`h_{Sh}`$ sets the maximum compression rate of a sequence $`\{s(1),s(2),s(3),\mathrm{}\}`$. Indeed a theorem of Shannon states that, if the length $`T`$ of a sequence is large enough, one cannot construct another sequence (always using $`m`$ symbols), from which it is possible to reconstruct the original one, whose length is smaller than $`(h_{Sh}/\mathrm{ln}m)T`$ . In other words $`h_{Sh}/\mathrm{ln}m`$ is the maximum allowed compression rate.

The relation between Shannon entropy and the data compression problem is well highlighted by considering the Shannon-Fano code to map $`𝒩`$ objects (e.g. the $`N`$-words $`C_N`$) into sequences of binary digits $`(0,1)`$ . The main goal in building a code is to define the most efficient coding procedure, i.e. the one which generates the shortest possible (coded) sequence. The Shannon-Fano code is as follows. At first one orders the $`𝒩`$ objects according to their probabilities, in a decreasing way, $`p_1,p_2,\mathrm{},p_𝒩`$. Then the passage from the $`𝒩`$ objects to the symbols $`(0,1)`$ is obtained by defining the coding $`E(r)`$, of binary length $`\mathrm{}(E(r))`$, of the $`r`$-th object with the requirement that the expected total length, $`_rp_r\mathrm{}_r`$, be the minimal one. This can be realized with the following choice

$$\mathrm{ln}_2p_r\mathrm{}(E(r))<1\mathrm{ln}_2p_r.$$
(2.25)
In this way highly probable objects are mapped into short code words while the low probability ones are mapped to longer code words. So that the average code length is bounded by

$$\frac{H_N}{\mathrm{ln}2}\underset{r}{}p_r\mathrm{}(E(r))\frac{H_N+1}{\mathrm{ln}2},$$
(2.26)
and in the limit $`N\mathrm{}`$ one has

$$\underset{N\mathrm{}}{lim}\frac{\mathrm{}_N}{N}=\underset{N\mathrm{}}{lim}\frac{\underset{r}{}p_r\mathrm{}(E(r))}{N}=\frac{h_{Sh}}{\mathrm{ln}2},$$
(2.27)
i.e., in a good coding, the mean length of a $`N`$-word is equal to $`N`$ times the Shannon entropy (apart from a multiplicative factor, due to the fact that in the definition (2.20) of $`h_{Sh}`$ we used the natural logarithm and here we want to work with a two symbol code).

An alternative coding method, based on variable length words, is due to Ziv and Lempel . Remarkably it is very efficient for data compression and gives the same asymptotic result of the Shannon-Fano code.

#### 2.2.2 Again on the Kolmogorov-Sinai entropy

After the introduction of the Shannon entropy we can give a more precise definition of the KS-entropy. Consider a trajectory, $`𝐱(t)`$, generated by a deterministic system, sampled at the times $`t_j=j\tau `$, with $`j=1,2,3,\mathrm{}`$. Perform a finite partition $`𝒜`$ of the phase space. With the finite number of symbols $`\{s\}_𝒜`$ enumerating the cells of the partition, the time-discretized trajectory $`𝐱(t_j)`$ determines a sequence $`\{s(1),s(2),s(3),\mathrm{}\}`$, whose meaning is clear: at the time $`t_j`$ the trajectory is in the cell labeled by $`s(j)`$. To each subsequence of length $`N\tau `$ one can associate a word of length $`N`$: $`W_j^N(𝒜)=(s(j),s(j+1),\mathrm{},s(j+(N1)))`$. If the system is ergodic, as we suppose, from the frequencies of the words one obtains the probabilities by which one calculates the block entropies $`H_N(𝒜)`$:

$$H_N(𝒜)=\underset{\{W^N(𝒜)\}}{}P(W^N(𝒜))\mathrm{ln}P(W^N(𝒜)).$$
(2.28)
It is important to note that the probabilities $`P(W^N(𝒜))`$, computed by the frequencies of $`W^N(𝒜)`$ along a trajectory, are essentially dependent on the stationary measure selected by the trajectory. This implies a dependence on this measure of all the quantities defined below, $`h_{KS}`$ included. We shall always understand to consider the natural invariant measure and do not indicate this kind of dependence. The entropy per unit time of the trajectory with respect to the partition $`𝒜`$, $`h(𝒜)`$, is defined as follows:

$`h_N(𝒜)`$ $`=`$ $`{\displaystyle \frac{1}{\tau }}[H_{N+1}(𝒜)H_N(𝒜)],`$ (2.29)
$`h(𝒜)`$ $`=`$ $`\underset{N\mathrm{}}{lim}h_N(𝒜)={\displaystyle \frac{1}{\tau }}\underset{N\mathrm{}}{lim}{\displaystyle \frac{1}{N}}H_N(𝒜).`$ (2.30)
Notice that, for the deterministic systems we are considering, the entropy per unit time does not depend on the sampling time $`\tau `$ . The KS-entropy, by definition, is the supremum of $`h(𝒜)`$ over all possible partitions

$$h_{KS}=\underset{𝒜}{sup}h(𝒜).$$
(2.31)
It is not simple at all to determine $`h_{KS}`$ according to this definition. A useful tool in this respect would be the Kolmogorov-Sinai theorem, by means of which one is granted that $`h_{KS}=h(𝒢)`$ if $`𝒢`$ is a generating partition. A partition is said to be generating if every infinite sequence $`\{s(n)\}_{n=1,\mathrm{},\mathrm{}}`$ individuates a single initial point. However the difficulty now is that, with the exception of very simple cases, we do not know how to construct a generating partition. We only know that, according to the Krieger theorem , there exists a generating partition with $`k`$ elements such that $`e^{h_{KS}}<ke^{h_{KS}}+1`$. Then, a more tractable way to define $`h_{KS}`$ is based upon considering the partition $`𝒜_ϵ`$ made up by a grid of cubic cells of edge $`ϵ`$, from which one has

$$h_{KS}=\underset{ϵ0}{lim}h(𝒜_ϵ).$$
(2.32)
We expect that $`h(𝒜_ϵ)`$ becomes independent of $`ϵ`$ when $`𝒜_ϵ`$ is so fine to be “contained” in a generating partition.

For discrete time maps what has been exposed above is still valid, with $`\tau =1`$ (however, Krieger’s theorem only applies to invertible maps).

The important point to note is that, for a truly stochastic (i.e. non-deterministic) system, with continuous states, $`h(𝒜_ϵ)`$ is not bounded and $`h_{KS}=\mathrm{}`$.

#### 2.2.3 Algorithmic complexity

We saw that the Shannon entropy puts a limit on how efficiently the ensemble of the messages emitted by a source can be coded. We may wonder about the compressibility properties of a single sequence. This problem can be addressed by means of the notion of algorithmic complexity, that is concerned with the difficulty in reproducing a given string of symbols.

Everybody agrees that the binary digits sequence

$$0111010001011001011010\mathrm{}$$
(2.33)
is, in some sense, more random than

$$1010101010101010101010\mathrm{}$$
(2.34)
The notion of algorithmic complexity, independently introduced by Kolmogorov , Chaitin  and Solomonov , is a way to formalize the intuitive idea of randomness of a sequence.

Consider a binary digit sequence (this does not constitute a limitation) of length $`N`$, $`(i_1,i_2,\mathrm{},i_N)`$, generated by a certain computer code on some machine $``$. One defines the algorithmic complexity, or algorithmic information content, $`K_{}(N)`$ of a $`N`$-sequence as the bit length of the shortest computer program able to give the $`N`$-sequence and to stop afterward. Of course, such a length depends not only on the sequence but also on the machine. However, a result of Kolmogorov  proves the existence of a universal computer, $`𝒰`$, that is able to perform the same computation a program $`p`$ makes on $``$ with a modification of $`p`$ that depends only on $``$. This implies that for all finite strings:

$$K_𝒰(N)K_{}(N)+C_{},$$
(2.35)
where $`K_𝒰(N)`$ is the complexity with respect to the universal computer and $`C_{}`$ depends only on the machine $``$. At this point we can consider the algorithmic complexity with respect to a universal computer – and we can drop the machine dependence in the symbol for the algorithmic complexity, $`K(N)`$. The reason is that we are interested in the limit of very long sequences, $`N\mathrm{}`$, for which one defines the algorithmic complexity per unit symbol:

$$𝒞=\underset{N\mathrm{}}{lim}\frac{K(N)}{N},$$
(2.36)
that, because of (2.35), is an intrinsic quantity, i.e. independent of the machine.

Now coming back to the two $`N`$-sequences (2.33) and (2.34), it is obvious that the second one can be obtained with a small-length minimal program, i.e.

$$\mathrm{"}\mathrm{PRINT}\mathrm{\hspace{0.33em}10}\frac{N}{2}\mathrm{TIMES}\mathrm{"}.$$

The bit length of the above program is $`O(\mathrm{ln}N)`$ and therefore when taking the limit $`N\mathrm{}`$ in (2.36), one obtains $`𝒞=0`$. Of course $`K(N)`$ cannot exceed $`N`$, since the sequence can always be obtained with a trivial program (of bit length $`N`$)

$$\mathrm{"}\mathrm{PRINT}i_1,i_2,\mathrm{},i_N\mathrm{"}.$$

Therefore, in the case of a very irregular sequence, e.g., (2.33) one expects $`K(N)N`$ i.e. $`𝒞0`$. In such a case one calls the sequence complex (i.e. of non zero algorithmic complexity) or random.

Algorithmic complexity cannot be computed. Since the algorithm which computes $`K(N)`$ cannot have less than $`K(N)`$ binary digits and since in the case of random sequences $`K(N)`$ is not bounded in the limit $`N\mathrm{}`$ then it cannot be computed in the most interesting cases. The un-computability of $`K(N)`$ may be understood in terms of Gödel’s incompleteness theorem . Beyond the problem of whether or not $`K(N)`$ is computable in a specific case, the concept of algorithmic complexity brings an important improvement to clarify the vague and intuitive notion of randomness. For a systematic treatment of algorithmic complexity, information theory and data compression see .

There exists a relation between the Shannon entropy, $`h_{Sh}`$, and the algorithmic complexity $`𝒞`$. It is possible to show that

$$\underset{N\mathrm{}}{lim}\frac{K(N)}{H_N}=\frac{1}{\mathrm{ln}2},$$
(2.37)
where $`K(N)=_{C_N}P(C_N)K_{C_N}(N)`$, being $`K_{C_N}(N)`$ the algorithmic complexity of the $`N`$-words. Therefore the expected complexity $`K(N)/N`$ is asymptotically equal to the Shannon entropy (apart the $`\mathrm{ln}2`$ factor).

Eq. (2.37) stems from the results of the Shannon-McMillan theorem about the two classes of sequences (i.e. $`\mathrm{\Omega }_1(N)`$ and $`\mathrm{\Omega }_0(N)`$). Indeed in the limit of very large $`N`$, the probability to observe a sequence in $`\mathrm{\Omega }_1(N)`$ goes to $`1`$, and the entropy of such a sequence as well as its algorithmic complexity equals the Shannon entropy. Apart from the numerical coincidence of the values of $`𝒞`$ and $`h_{Sh}/\mathrm{ln}2`$ there is a conceptual difference between the information theory and the algorithmic complexity theory. The Shannon entropy essentially refers to the information content in a statistical sense, i.e. it refers to an ensemble of sequences generated by a certain source. On the other hand, the algorithmic complexity defines the information content of an individual sequence . Of course, as noted above, the fact that it is possible to use probabilistic arguments on an individual sequence is a consequence of the ergodicity of the system, which allows to assume good statistical properties of arbitrary long $`N`$-words.

For a dynamical system one can define the notion of algorithmic complexity of the trajectory starting from the point $`𝐱`$, $`𝒞(𝐱)`$. This requires the introduction of finite open coverings of the phase space, the consideration of symbolic sequences thus generated, for the given trajectory, sampled at constant time intervals, and the searching of the supremum of the algorithmic complexity per symbol at varying the coverings . Then it can be shown – Brudno’s and White’s theorems  – that for almost all $`𝐱`$ (we always mean with respect to the natural invariant measure) one has:

$$𝒞(𝐱)=\frac{h_{KS}}{\mathrm{ln}2},$$
(2.38)
where, as before, the factor $`\mathrm{ln}2`$ is a conversion factor between natural logarithms and bits.

This result says that the KS-entropy quantifies not only the richness, or surprise, of a dynamical system but also the difficulty of describing (almost) everyone of its typical sequences.

### 2.3 Algorithmic complexity and Lyapunov Exponent

Summing up, the theorem of Pesin together with those of Brudno and White show that a chaotic dynamical system may be seen as a source of messages that cannot be described in a concise way, i.e. they are complex. We expose here two examples that may help in understanding the previous conclusion and the relation between the Lyapunov exponent, the $`KS`$-entropy and the algorithmic complexity.

Following Ford , let us consider the shift map

$$x(t+1)=2x(t)\mathrm{mod}\mathrm{\hspace{0.17em}1},$$
(2.39)
which has $`\lambda =\mathrm{ln}2`$. If one writes an initial condition in binary representation, i.e., $`x(0)=_{j=1}^{\mathrm{}}a_j\mathrm{\hspace{0.17em}2}^j`$, such that $`a_j=0`$ or $`1`$, it is clearly seen that the action of the map (2.39) on $`x(0)`$ is just a shift of the binary coordinates:

$$x(1)=\underset{j=1}{\overset{\mathrm{}}{}}a_{j+1}\mathrm{\hspace{0.17em}2}^j\mathrm{}x(t)=\underset{j=1}{\overset{\mathrm{}}{}}a_{j+t}\mathrm{\hspace{0.17em}2}^j.$$
(2.40)
With this observation it is possible to verify that $`K(N)N`$ for almost all the solutions of (2.39). Let us consider $`x(t)`$ with accuracy $`2^k`$ and $`x(0)`$ with accuracy $`2^l`$, of course $`l=t+k`$. This means that, in order to obtain the $`k`$ binary digits of the output solution of (2.39), we must use a program of length no less than $`l=t+k`$. Basically one has to specify $`a_1,a_2,\mathrm{},a_l`$. Therefore we are faced with the problem of the algorithmic complexity of the binary sequence $`(a_1,a_2,\mathrm{},a_{\mathrm{}})`$ which determines the initial condition $`x(0)`$. Martin-Löf  proved a remarkable theorem stating that, with respect to the Lebesgue measure, almost all the binary sequences $`(a_1,a_2,\mathrm{},a_{\mathrm{}})`$, which represent a real number in $`[0,1]`$, have maximum complexity, i.e. $`K(N)N`$. In practice no human being will ever be able to distinguish the typical sequence $`(a_1,a_2,\mathrm{},a_{\mathrm{}})`$ from the output of a fair coin toss.

Finally let us consider a $`1d`$ chaotic map

$$x(t+1)=f(x(t)).$$
(2.41)
If one wants to transmit to a friend on Mars the sequence $`\{x(t),t=1,2,\mathrm{},T\}`$ accepting only errors smaller than a tolerance $`\mathrm{\Delta }`$, one can use the following strategy :

1. Transmit the rule (2.41): for this task one has to use a number of bits independent of the length of the sequence $`T`$.
2. Specify the initial condition $`x(0)`$ with a precision $`\delta _0`$ using a finite number of bits which is independent of $`T`$.
3. Let the system evolve till the first time $`\tau _1`$ such that the distance between two trajectories, that was initially $`\delta x(0)=\delta _0`$, equals $`\mathrm{\Delta }`$ and then specify again the new initial condition $`x(\tau _1)`$ with precision $`\delta _0`$.
4. Let the system evolve and repeat the procedure (2-3), i.e. each time the error acceptance tolerance is reached specify the initial conditions, $`x(\tau _1+\tau _2),x(\tau _1+\tau _2+\tau _3)\mathrm{}`$, with precision $`\delta _0`$. The times $`\tau _1,\tau _2,\mathrm{}`$ are defined as follows: putting $`x^{^{}}(\tau _1)=x(\tau _1)+\delta _0`$, $`\tau _2`$ is given by the minimum time such that $`|x^{^{}}(\tau _1+\tau _2)x(\tau _1+\tau _2)|\mathrm{\Delta }`$ and so on.

By following the steps $`(14)`$ the friend on Mars can reconstruct with a precision $`\mathrm{\Delta }`$ the sequence $`\{x(t)\}`$ simply iterating on a computer the system (2.41) between $`1`$ and $`\tau _11`$, $`\tau _1`$ and $`\tau _1+\tau _21`$, and so on.

Let us now compute the amount of bits necessary to implement the above transmission (1-4). For simplicity of notation we introduce the quantities

$$\gamma _i=\frac{1}{\tau _i}\mathrm{ln}\frac{\mathrm{\Delta }}{\delta _0}$$
(2.42)
which can be considered as a sort of effective Lyapunov exponents (see Sect. 3.1). The LE $`\lambda `$ can be written in terms of $`\{\gamma _i\}`$ as follows

$$\lambda =\gamma _i=\frac{\underset{i}{}\tau _i\gamma _i}{_i\tau _i}=\frac{1}{\overline{\tau }}\mathrm{ln}\frac{\mathrm{\Delta }}{\delta _0}$$
(2.43)
where

$$\overline{\tau }=\frac{1}{N}\tau _i,$$

is the average time after which we have to transmit the new initial condition (let us observe that to obtain $`\lambda `$ from the $`\gamma _i`$’s one has to perform the average (2.43) because the transmission time, $`\tau _i`$, is not constant). If $`T`$ is large enough the number of transmissions, $`N`$, is $`T/\overline{\tau }\lambda T/\mathrm{ln}(\mathrm{\Delta }/\delta _0)`$. Therefore, noting that in each transmission for the reduction of the error from $`\mathrm{\Delta }`$ to $`\delta _0`$ one needs to use $`\mathrm{ln}_2(\mathrm{\Delta }/\delta _0)`$ bits, the total amount of bits used in the whole transmission is

$$\frac{T}{\overline{\tau }}\mathrm{ln}_2\frac{\mathrm{\Delta }}{\delta _0}=\frac{\lambda }{\mathrm{ln}2}T.$$
(2.44)
In other words the number of bits for unit time is proportional to $`\lambda `$.

In more than one dimension, we have simply to replace $`\lambda `$ with $`h_{KS}`$ in (2.44). To intuitively understand this point one has to repeat the above transmission procedure in each of the expanding directions.

In this section, we briefly discussed how the limit of predictability, the fact that a sequence cannot be arbitrarily compressed, and the information contained in a signal are deeply related. In the following we will mainly discuss the dynamical system point of view, i.e., in terms of Lyapunov exponents, Kolmogorov Sinai entropy and their generalizations for less ideal cases.

## 3 Limits of the Lyapunov exponent for predictability

We saw how a first obvious way for quantifying the predictability of a physical system is in terms of the predictability time $`T_p`$, i.e. the time interval on which one can typically forecast the system. A simple argument previously suggested

$$T_p\frac{1}{\lambda }\mathrm{ln}\left(\frac{\mathrm{\Delta }}{\delta _0}\right).$$
(3.1)
However, in any realistic system, relation (3.1) is too naive to be of actual relevance. Indeed, it does not take into account some basic features of dynamical systems:

\- The Lyapunov exponent (2.9) is a global quantity: it measures the average rate of divergence of nearby trajectories. In general there exist finite-time fluctuations and the probability distribution functions (pdf) of these fluctuations is important for the characterization of predictability. The generalized Lyapunov exponents have been introduced with the purpose to take into account these fluctuations .

\- The Lyapunov exponent is defined for the linearized dynamics, i.e., by computing the rate of separation of two infinitesimally close trajectories. On the other hand, for measuring the predictability time (3.1) one is interested in a finite tolerance $`\mathrm{\Delta }`$, because the initial error $`\delta _0`$ is typically finite. A recent generalization of the Lyapunov exponent to finite size errors extends the study of the perturbation growth to the nonlinear regime, i.e. both $`\delta _0`$ and $`\mathrm{\Delta }`$ are not infinitesimal .

### 3.1 Characterization of finite-time fluctuations

Let us consider the linear response, at a delay $`t`$, to an infinitesimal perturbation $`\delta 𝐱(0)`$:

$$R(t)=\frac{|\delta 𝐱(t)|}{|\delta 𝐱(0)|},$$
(3.2)
from which the LE is computed according to (2.9). In order to take into account the finite-time fluctuations, we can compute the different moments $`R(t)^q`$ and introduce the so-called generalized Lyapunov exponents (of order $`q`$) :

$$L(q)=\underset{t\mathrm{}}{lim}\frac{1}{t}\mathrm{ln}R(t)^q$$
(3.3)
where $`\mathrm{}`$ indicates the time average along the trajectory (see Sect. 2). It is easy to show that

$$\lambda _1=\frac{dL(q)}{dq}|_{q=0}.$$
(3.4)
In the absence of fluctuations, $`\lambda _1`$ completely characterizes the error growth and we have $`L(q)=\lambda _1q`$, while in the general case $`L(q)`$ is concave in $`q`$ . Before discussing the properties of the generalized Lyapunov exponents, let us consider a simple example with a non trivial $`L(q)`$. The model is the one-dimensional map

$$x(t+1)=\{\begin{array}{cc}\frac{x(t)}{a}\hfill & \text{for }0xa\hfill \\ \frac{1x(t)}{1a}\hfill & \text{for }a<x1\hfill \end{array},$$
(3.5)
which for $`a=1/2`$ reduces to the tent map. For $`a1/2`$ the system is characterized by two different growth rates. The presence of different growth rates makes $`L(q)`$ non-linear in $`q`$. Since the map (3.5) is piecewise linear and with a uniform invariant density, the explicit computation of $`L(q)`$ is very easy. The moments of the response after a time $`t`$ are simply given by

$$R(t)^q=\left[a\left(\frac{1}{a}\right)^q+(1a)\left(\frac{1}{1a}\right)^q\right]^t.$$
(3.6)
From (3.3) and (3.6) we have:

$$L(q)=\mathrm{ln}\left[a^{1q}+(1a)^{1q}\right],$$
(3.7)
which recovers the non intermittent limit $`L(q)=q\mathrm{ln}2`$ in the symmetric case $`a=1/2`$. In the general case, assuming $`0a<1/2`$, we have that for $`q+\mathrm{}`$, $`L(q)`$ is dominated by the less probable, most unstable contributions and $`L(q)/q\mathrm{ln}(a)`$. In the opposite limit, $`q\mathrm{}`$, we obtain $`L(q)/q\mathrm{ln}(1a)`$.

We now show how $`L(q)`$ is related to the fluctuations of $`R(t)`$ at finite time $`t`$. Define an “effective” Lyapunov exponent $`\gamma (t)`$ at time $`t`$ by

$$R(t)e^{\gamma (t)t}.$$
(3.8)
In the limit $`t\mathrm{}`$, the Oseledec theorem  assures that, for typical trajectories, $`\gamma (t)=\lambda _1=a\mathrm{ln}a(1a)\mathrm{ln}(1a)`$. Therefore, for large $`t`$, the probability density of $`\gamma (t)`$ is peaked at the most probable value $`\lambda _1`$. Let us introduce the probability density $`P_t(\gamma )`$ of observing a given $`\gamma `$ on a trajectory of length $`t`$. Large deviation theory suggests

$$P_t(\gamma )e^{S(\gamma )t},$$
(3.9)
where $`S(\gamma )`$ is the Cramer function . The Oseledec theorem implies that $`lim_t\mathrm{}P_t(\gamma )=\delta (\gamma \lambda _1)`$, this gives a constraint on the Cramer function, i.e. $`S(\gamma =\lambda _1)=0`$ and $`S(\gamma )>0`$ for $`\gamma \lambda _1`$.

The Cramer function $`S(\gamma )`$ is related to the generalized Lyapunov exponent $`L(q)`$ trough a Legendre transform. Indeed, at large $`t`$, one has

$$R(t)^q=𝑑\gamma P_t(\gamma )e^{q\gamma t}e^{L(q)t},$$
(3.10)
by a steepest descent estimation one obtains

$$L(q)=\underset{\gamma }{\mathrm{max}}\left[q\gamma S(\gamma )\right].$$
(3.11)
In other words each value of $`q`$ selects a particular $`\gamma ^{}(q)`$ given by

$$\frac{dS(\gamma )}{d\gamma }|_\gamma ^{}=q.$$
(3.12)
We have already discussed that, for negligible fluctuations of the “effective” Lyapunov exponents, the LE completely characterizes the error growth and $`L(q)=\lambda _1q`$. In presence of fluctuations, the probability distribution for $`R(t)`$ can be approximated by a log-normal distribution. This can be understood assuming weak correlations in the response function so that (3.2) factorizes in several independent contributions and the central limit theorem applies. We can thus write

$$P_t(R)=\frac{1}{R\sqrt{2\pi \mu t}}\mathrm{exp}\left(\frac{(\mathrm{ln}R\lambda _1t)^2}{2\mu t}\right),$$
(3.13)
where $`\lambda _1`$ and $`\mu `$ are given by

$`\lambda _1`$ $`=`$ $`\underset{t\mathrm{}}{lim}{\displaystyle \frac{1}{t}}\mathrm{ln}R(t)`$
$`\mu `$ $`=`$ $`\underset{t\mathrm{}}{lim}{\displaystyle \frac{1}{t}}\left(\mathrm{ln}R(t)^2\mathrm{ln}R(t)^2\right).`$ (3.14)
The log-normal distribution for $`R`$ corresponds to a Gaussian distribution for $`\gamma `$

$$S(\gamma )=\frac{\left(\gamma \lambda _1\right)^2}{2\mu },$$
(3.15)
and to a quadratic in $`q`$ generalized Lyapunov exponent:

$$L(q)=\lambda _1q+\frac{1}{2}\mu q^2.$$
(3.16)
Let us remark that, in general, the log-normal distribution (3.13) is a good approximation for non extreme events, i.e. small fluctuation of $`\gamma `$ around $`\lambda _1`$, so that the expression (3.16) is correct only for small $`q`$ (see Fig. 1). This is because the moments of the log-normal distribution grow too fast with $`q`$ . Indeed from (3.12) we have that the selected $`\gamma ^{}(q)`$ is given by $`\gamma ^{}(q)=\lambda _1+\mu q`$ and thus $`\gamma ^{}(q)`$ is not finite for $`q\mathrm{}`$. This is unphysical because $`\gamma ^{}(\mathrm{})`$ is the fastest error growth rate in the system and, we may reasonably suppose that it is finite.

Let us consider again the map (3.5). In this case we have $`\lambda _1=L^{}(0)=a\mathrm{ln}(a)(1a)\mathrm{ln}(1a)`$ and $`\mu =L^{\prime \prime }(0)=a(1a)\left(\mathrm{ln}(a)\mathrm{ln}(1a)\right)^2`$, which are nothing but the coefficients of the Taylor expansion of (3.7) around $`q=0`$. For large $`q`$ the log-normal approximation gives $`L(q)/qq\mu /2`$ while the correct limit is the constant $`L(q)/q\mathrm{ln}(a)`$.

Nevertheless, $`\lambda _1`$ and $`\mu `$ are the two basic parameters for characterizing the intermittency of a system. To be more specific, let us remark that $`P_t(R)`$ (3.13) reaches its maximum for

$$R_{max}(t)=e^{(\lambda _1\mu )t},$$
(3.17)
so that for $`t\mathrm{}`$:

$$\begin{array}{cc}R_{max}\mathrm{}\hfill & \text{if }\mu /\lambda _1<1\hfill \\ R_{max}0\hfill & \text{if }\mu /\lambda _1>1.\hfill \end{array}$$

Thus in the weak intermittency limit, $`\mu /\lambda _1<1`$, the most probable response function $`R_{max}(t)`$ follows the correct behavior (with the corrected exponent $`\lambda _1\mu `$). In the strong intermittent limit, $`\mu /\lambda _1>1`$, the most probable estimation breaks down because it predicts an asymptotic stable phase $`R_{max}(t)0`$ instead of the chaotic exponential growth.

As in the case of the first LE, it is possible to introduce higher order generalized Lyapunov exponents. Starting from the $`n`$-order response function $`R^{(n)}(t)`$ (2.10), we define

$$L^{(n)}(q)=\underset{t\mathrm{}}{lim}\frac{1}{t}\mathrm{ln}R^{(n)}(t)^q,$$
(3.18)
where $`L^{(1)}(q)=L(q)`$. From (2.11) we have

$$\underset{i=1}{\overset{n}{}}\lambda _i=\frac{dL^{(n)}(q)}{dq}|_{q=0}.$$
(3.19)
The generalized $`L^{(n)}(q)`$ represents the fluctuations of the exponential divergence of a $`n`$-dimensional volume in phase space . The properties of $`L^{(n)}(q)`$ are analogous to the properties of $`L(q)`$, i.e. $`L^{(n)}(q)`$ is a concave function of $`q`$ for any $`n`$ and for a non-intermittent behavior they are linear in $`q`$.

### 3.2 Renyi entropies

In Section 2.1.2 we defined the Kolmogorov-Sinai entropy (2.13) and discussed its relation with the Lyapunov exponents by means of the Pesin relation (2.16). Analogously to the generalized LE, it is possible to introduce a generalization of the Kolmogorov-Sinai entropy in order to take into account the intermittency.

Let us recall the definition of Kolmogorov-Sinai entropy

$$h_{KS}=\underset{\tau 0}{lim}\underset{ϵ0}{lim}\underset{N\mathrm{}}{lim}\frac{1}{N\tau }\underset{\{W^N(𝒜_ϵ)\}}{}P(W^N(𝒜_ϵ))\mathrm{ln}P(W^N(𝒜_ϵ))$$
(3.20)
where $`𝒜_ϵ`$ is a partition of the phase space in cells of size $`ϵ`$ and $`W^N(𝒜_ϵ)`$ indicates a sequence of length $`N`$ in this partition. The generalized Renyi entropies , $`K_q`$, can be introduced by observing that (3.20) is nothing but the average of $`\mathrm{ln}P(W^N)`$ with the probability $`P(W^N)`$:

$$K_q=\underset{\tau 0}{lim}\underset{ϵ0}{lim}\underset{N\mathrm{}}{lim}\frac{1}{N\tau (q1)}\mathrm{ln}\left(\underset{\{W^N(𝒜_ϵ)\}}{}P(W^N(𝒜_ϵ))^q\right).$$
(3.21)
As in (3.4) one has $`h_{KS}=lim_{q1}K_q=K_1`$; in addition from general results of probability theory, one can show that $`K_q`$ is monotonically decreasing with $`q`$.

It will not be surprising that the generalized Renyi entropies are related to the generalized Lyapunov exponents $`L(q)`$. Introducing the number of non-negative Lyapunov exponents $`n^{}`$ (i.e. $`\lambda _n^{}0`$, $`\lambda _{n+1}<0`$), the Pesin relation (2.16) can be written as

$$h_{KS}=\underset{i=1}{\overset{n^{}}{}}\lambda _i=\frac{dL^{(n^{})}(q)}{dq}|_{q=0}.$$
(3.22)
Moreover, one has :

$$K_{q+1}=\frac{L^{(n^{})}(q)}{q}.$$
(3.23)
### 3.3 The effects of intermittency on predictability

We have seen that intermittency can be described, at least at a qualitative level, in terms of $`\lambda _1`$ and $`\mu `$, which are the two parameters characterizing the log-normal approximation. We discuss now the relevance of the log-normal approximation for the predictability time $`T_p`$.

The predictability time $`T_p`$ is defined as the time it takes for the error of initial size $`\delta _0`$ to grow up to the tolerance $`\mathrm{\Delta }`$

$$T_p=\mathrm{min}\left[t:R(t)\frac{\mathrm{\Delta }}{\delta _0}\right].$$
(3.24)
In the framework of the log-normal approximation, we can write

$$\mathrm{ln}R(t)=\lambda _1t+\sqrt{\mu }w(t),$$
(3.25)
where $`w(t)`$ is a Wiener process with $`w(0)=0`$, $`w(t)=0`$ and $`w(t)w(t^{})=\mathrm{min}(t,t^{})`$. In this case the computation of $`T_p`$ is reduced to a first exit problem, which is well known in stochastic process theory . The solution gives the pdf of $`T_p`$ :

$$p(T_p)=\frac{\mathrm{ln}(\mathrm{\Delta }/\delta _0)}{\sqrt{2\pi \mu T_p^3}}\mathrm{exp}\left[\frac{(\lambda _1T_p\mathrm{ln}(\mathrm{\Delta }/\delta _0))^2}{2\mu T_p}\right].$$
(3.26)
Notice that the (3.26) is not normalized, since there is a finite probability of “no exit”. Of course, this is an artifact of the approximation in terms of the stochastic process (3.25). In non pathological chaotic systems one expects that $`p(T_p)`$ is normalized.

In the limit of weak intermittency $`\mu /\lambda _11`$, $`p(T_p)`$ is almost Gaussian and the mean value $`T_p`$ is close to the most probable value of (3.26) corresponding to the naive estimation (3.1). On the contrary, in the strong intermittent limit, $`\mu /\lambda _11`$, the pdf of $`T_p`$ shows an asymmetric “triangular shape” and the most probable value is

$$T_p=\frac{1}{3\mu }\mathrm{ln}(\mathrm{\Delta }/\delta _0)^2.$$
(3.27)
In order to see the effects of intermittency on the predictability time, let us consider as an example the Lorenz system :

$$\{\begin{array}{ccc}\frac{dx}{dt}\hfill & =\hfill & \sigma (yx)\hfill \\ \frac{dy}{dt}\hfill & =\hfill & x(rz)y\hfill \\ \frac{dz}{dt}\hfill & =\hfill & xybz\hfill \end{array}$$
(3.28)
with the standard values $`\sigma =10`$ and $`b=8/3`$. For $`r=28`$, the Lorenz model is very weakly intermittent, $`\mu /\lambda 7\times 10^2`$, and the pdf of the predictability time is very close to a Gaussian (Fig. 2). On the contrary, for $`r=166.3`$ the Lorenz model becomes strongly intermittent , $`\mu /\lambda 3.3`$ and the pdf of the predictability time displays a long exponential tail responsible for the deviation from (3.1).

This qualitative behavior is typical of intermittent systems. In Section 5.3 we will see a more complex example in the context of turbulence.

### 3.4 Growth of non infinitesimal perturbations

In realistic situations, the initial condition of a system is known with a limited accuracy. In this case the Lyapunov exponent is of little relevance for the characterization of predictability and new indicators are needed. To clarify the problem, let us consider the following coupled map model:

$$\{\begin{array}{cc}𝐱(t+1)\hfill & =𝐑𝐱(t)+\epsilon 𝐡(y(t))\hfill \\ y(t+1)\hfill & =G(y(t)),\hfill \end{array}$$

where $`𝐱\mathrm{I}R^2`$, $`y\mathrm{I}R^1`$, $`𝐑`$ is a rotation matrix of arbitrary angle $`\theta `$, $`𝐡`$ is a vector function and $`G`$ is a chaotic map. For simplicity we consider a linear coupling $`𝐡(y)=(y,y)`$ and the logistic map $`G(y)=4y(1y)`$.

For $`\epsilon =0`$ we have two independent systems: a regular and a chaotic one. Thus the Lyapunov exponent of the $`𝐱`$ subsystem is $`\lambda _x(\epsilon =0)=0`$, i.e., it is completely predictable. On the contrary, the $`y`$ subsystem is chaotic with $`\lambda _y=\lambda _1=\mathrm{ln}2`$.

If we now switch on the (small) coupling ($`\epsilon >0`$) we are confronted with a single three-dimensional chaotic system with a positive global Lyapunov exponent

$$\lambda =\lambda _y+O(\epsilon ).$$
(3.29)
A direct application of (3.1) would give

$$T_p^{(x)}T_p\frac{1}{\lambda _y},$$
(3.30)
but this result is clearly unacceptable: the predictability time for $`𝐱`$ seems to be independent of the value of the coupling $`\epsilon `$. Let us underline that this is not due to an artifact of the chosen example. Indeed, one can use the same argument in many physical situations . A well known example is the gravitational three body problem with one body (asteroid) much smaller than the other two (planets). If one neglects the gravitational feedback of the asteroid on the two planets (restricted problem) one has a chaotic asteroid in the regular field of the planets. As soon as the feedback is taken into account (i.e. $`\epsilon >0`$ in the example) one has a non-separable three body system with a positive LE. Of course, intuition correctly suggests that it should be possible to forecast the motion of the planets for very long times if the asteroid has a very small mass ($`\epsilon 0`$).

The apparent paradox arises from the use of (3.1), which is valid only for the tangent vectors, also in the non infinitesimal regime. As soon as the errors become large one has to take into account the full nonlinear evolution. The effect is shown for the model (3.4) in Figure 3. The evolution of $`\delta 𝐱`$ is given by

$$\delta 𝐱(t+1)=𝐑\delta 𝐱(t)+\epsilon \delta 𝐡(y),$$
(3.31)
where, with our choice, $`\delta 𝐡=(\delta y,\delta y)`$. At the beginning, both $`|\delta 𝐱|`$ and $`\delta y`$ grow exponentially. However, the available phase space for $`y`$ is finite and the uncertainty reaches the saturation value $`\delta yO(1)`$ in a time $`t1/\lambda _1`$. At larger times the two realizations of the $`y`$ variable are completely uncorrelated and their difference $`\delta y`$ in (3.31) acts as a noisy term. As a consequence, the growth of the uncertainty on $`𝐱`$ becomes diffusive with a diffusion coefficient proportional to $`\epsilon ^2`$

$$|\delta 𝐱(t)|\epsilon t^{1/2}$$
(3.32)
so that:

$$T_p^{(x)}\epsilon ^2.$$
(3.33)
This example shows that, even in simple systems, the Lyapunov exponent can be of little relevance for the characterization of the predictability.

In more complex systems, in which different scales are present, one is typically interested in forecasting the large scale motion, while the LE is related to the small scale dynamics. A familiar example is weather forecast: the LE of the atmosphere is indeed rather large due to the small scale convective motion, but (large scale) weather prediction is possible for about $`10`$ days . It is thus natural to seek for a generalization of the LE to finite perturbations from which one can obtain a more realistic estimation for the predictability time. It is worth underlining the important fact that finite errors are not confined in the tangent space but are governed by the complete nonlinear dynamics. In this sense the extension of the LE to finite errors will give more information on the system.

Aiming to generalize the LE to non infinitesimal perturbations let us now define the Finite Size Lyapunov Exponent (FSLE)  (see Appendix A for the computational details). Consider a reference trajectory, $`𝐱(t)`$, and a perturbed one, $`𝐱^{^{}}(t)`$, such that $`|𝐱^{^{}}(0)𝐱(0)|=\delta `$ ($`|\mathrm{}|`$ is the Euclidean norm but one can also consider other norms). One integrates the two trajectories and computes the time $`\tau _1(\delta ,r)`$ necessary for the separation $`|𝐱^{^{}}(t)𝐱(t)|`$ to grow from $`\delta `$ to $`r\delta `$. At time $`t=\tau _1(\delta ,r)`$ the distance between the trajectories is rescaled to $`\delta `$ and the procedure is repeated in order to compute $`\tau _2(\delta ,r),\tau _3(\delta ,r)\mathrm{}`$.

The threshold ratio $`r`$ must be $`r>1`$, but not too large in order to avoid contributions from different scales in $`\tau (\delta ,r)`$. A typical choice is $`r=2`$ (for which $`\tau (\delta ,r)`$ is properly a “doubling” time) or $`r=\sqrt{2}`$. In the same spirit of the discussion leading to Eq.s (2.42) and (2.43), we may introduce an effective finite size growth rate:

$$\gamma _i(\delta ,r)=\frac{1}{\tau _i(\delta ,r)}\mathrm{ln}r.$$
(3.34)
After having performed $`𝒩`$ error-doubling experiments, we can define the FSLE as

$$\lambda (\delta )=\gamma (\delta ,r)_t=\frac{1}{\tau (\delta ,r)}_t\mathrm{ln}r=\frac{1}{\tau (\delta ,r)_e}\mathrm{ln}r,$$
(3.35)
where $`\tau (\delta ,r)_e`$ is

$$\tau (\delta ,r)_e=\frac{1}{𝒩}\underset{n=1}{\overset{𝒩}{}}\tau _n(\delta ,r).$$
(3.36)
(see Appendix A and  for details). In the infinitesimal limit, the FSLE reduces to the Lyapunov exponent

$$\underset{\delta 0}{lim}\lambda (\delta )=\lambda _1.$$
(3.37)
In practice this limit means that $`\lambda (\delta )`$ displays a constant plateau at $`\lambda _1`$ for sufficiently small $`\delta `$ (Fig. 3). For finite value of $`\delta `$ the behavior of $`\lambda (\delta )`$ depends on the details of the non linear dynamics. For example, in the model (3.4) the diffusive behavior (3.32), by simple dimensional arguments, corresponds to $`\lambda (\delta )\delta ^2`$.

Since the FSLE measures the rate of divergence of trajectories at finite errors, one might wonder whether it is just another way to look at the average response $`\mathrm{ln}(R(t))`$ (3.2) as a function of time. A moment of reflection shows that this is not the case. Indeed taking the average at fixed time is not the same as computing the average doubling time at fixed scale, as in (3.35). This is particularly clear in the case of strongly intermittent system, in which $`R(t)`$ can be very different in each realization. In presence of intermittency averaging over different realizations at fixed times can produce a spurious regime due to the superposition of exponential and diffusive contributions by different samples at the same time .

The FSLE method can be easily applied for data analysis .

For other approaches to address the problem of non-infinitesimal perturbations see .

### 3.5 The $`ϵ`$-entropy

The Kolmogorov-Sinai entropy, $`h_{KS}`$ (2.13), of a system measures the amount of information per unit time necessary to record without ambiguity a generic trajectory of a system. Since the computation of $`h_{KS}`$ involves the limit of arbitrary fine resolution and infinite times (see Sect. 2.1.2), it turns out that, practically, for most systems it is not possible to compute $`h_{KS}`$. Nevertheless, in the same philosophy of the FSLE, by relaxing the strict requirement of reproducing a trajectory with arbitrary accuracy, one can introduce the $`ϵ`$-entropy which measures the amount of information for reproducing a trajectory with accuracy $`ϵ`$ in phase-space. Roughly speaking the $`ϵ`$-entropy can be considered the counterpart, in information theory, of the FSLE (as the $`KS`$-entropy is for the Lyapunov exponent). Such a quantity was originally introduced by Shannon , and by Kolmogorov . Recently Gaspard and Wang  made use of this concept to characterize a large variety of processes.

We start with a continuous (in time) variable $`𝐱(t)\mathrm{I}R^d`$, which represents the state of a $`d`$-dimensional system, we discretized the time by introducing an interval $`\tau `$ and we consider the new variable

$$𝐗^{(m)}(t)=(𝐱(t),𝐱(t+\tau ),\mathrm{},𝐱(t+(m1)\tau )).$$
(3.38)
Of course $`𝐗^{(m)}(t)\mathrm{I}R^{md}`$ and it corresponds to the trajectory which lasts for a time $`T=m\tau `$.

In data analysis, the space where the state vectors of the system live is not known. Moreover, usually only a scalar variable $`u(t)`$ can be measured. In such a case, one considers vectors $`(u(t),u(t+\tau ),\mathrm{},u(t+m\tau \tau ))`$, that live in $`\mathrm{I}R^m`$ and allow a reconstruction of the original phase space, known as delay embedding in the literature  (see also ), and it is a special case of (3.38).

Introduce now a partition of the phase space $`\mathrm{I}R^d`$, using cells of edge $`ϵ`$ in each of the $`d`$ directions. Since the region where a bounded motion evolves contains a finite number of cells, each $`𝐗^{(m)}(t)`$ can be coded into a word of length $`m`$, out of a finite alphabet:

$$𝐗^{(m)}(t)W^m(ϵ,t)=(i(ϵ,t),i(ϵ,t+\tau ),\mathrm{},i(ϵ,t+m\tau \tau )),$$
(3.39)
where $`i(ϵ,t+j\tau )`$ labels the cell in $`\mathrm{I}R^d`$ containing $`𝐱(t+j\tau )`$. From the time evolution of $`𝐗^{(m)}(t)`$ one obtains, under the hypothesis of ergodicity, the probabilities $`P(W^m(ϵ))`$ of the admissible words $`\{W^m(ϵ)\}`$. We can now introduce the $`(ϵ,\tau )`$-entropy per unit time, $`h(ϵ,\tau )`$ :

$`h_m(ϵ,\tau )`$ $`=`$ $`{\displaystyle \frac{1}{\tau }}[H_{m+1}(ϵ,\tau )H_m(ϵ,\tau )]`$ (3.40)
$`h(ϵ,\tau )`$ $`=`$ $`\underset{m\mathrm{}}{lim}h_m(ϵ,\tau )={\displaystyle \frac{1}{\tau }}\underset{m\mathrm{}}{lim}{\displaystyle \frac{1}{m}}H_m(ϵ,\tau ),`$ (3.41)
where $`H_m`$ is the block entropy of block length $`m`$:

$$H_m(ϵ,\tau )=\underset{\{W^m(ϵ)\}}{}P(W^m(ϵ))\mathrm{ln}P(W^m(ϵ)).$$
(3.42)
For the sake of simplicity, we ignored the dependence on details of the partition. To make $`h(ϵ,\tau )`$ partition-independent one has to consider a generic partition of the phase space $`\{𝒜\}`$ and to evaluate the Shannon entropy on this partition: $`h_{Sh}(𝒜,\tau )`$. The $`\epsilon `$-entropy is thus defined as the infimum over all partitions for which the diameter of each cell is less than $`\epsilon `$ :

$$h(\epsilon ,\tau )=\underset{𝒜:\mathrm{diam}(𝒜)\epsilon }{inf}h_{Sh}(𝒜,\tau ).$$
(3.43)
Note that the time dependence in (3.43) is trivial for deterministic systems, and that in the limit $`ϵ0`$ one recovers the Kolmogorov-Sinai entropy

$$h_{KS}=\underset{ϵ0}{lim}h(ϵ,\tau ).$$

The above entropies $`H_m(ϵ)`$ have been introduced by using a partition and the usual Shannon entropy; however it is possible to arrive at the same notion, starting from other entropy-like quantities, that are numerically more convenient. For example Cohen and Procaccia  proposed to estimate $`H_m(ϵ)`$ as follows. Given a signal composed of $`N`$ successive records and the embedding variable $`𝐗^{(m)}`$, let us introduce the quantities:

$$n_j^{(m)}=\frac{1}{Nm}\underset{ij}{}\mathrm{\Theta }(ϵ|𝐗^{(m)}(i\tau )𝐗^{(m)}(j\tau )|),$$
(3.44)
then the block entropy $`H_m(ϵ)`$ is given by

$$H_m^{(1)}(ϵ)=\frac{1}{(Nm+1)}\underset{j}{}\mathrm{ln}n_j^{(m)}(ϵ).$$
(3.45)
In practice $`n_j^{(m)}(ϵ)`$ is an approximation of $`P(W^m(ϵ))`$. From a numerical point of view, correlation entropies  are sometimes more convenient, so that one studies

$$H_m^{(2)}(ϵ)=\mathrm{ln}\left(\frac{1}{Nm+1}\underset{j}{}n_j^{(m)}(ϵ)\right)H_m^{(1)}(ϵ).$$
(3.46)
This corresponds to approximate the Shannon by the Renyi entropy of order $`q=2`$ .

The $`(ϵ,\tau )`$-entropy $`h(ϵ,\tau )`$ is well defined also for stochastic processes. Actually the dependence of $`h(ϵ,\tau )`$ on $`ϵ`$ can give some insight into the underlying stochastic process , for instance, in the case of a stationary Gaussian process with spectrum $`S(\omega )\omega ^2`$ one has :

$$\underset{\tau 0}{lim}h(ϵ,\tau )\frac{1}{ϵ^2}.$$
(3.47)
However, we have to stress that the behavior predicted by Eq. (3.47) may be difficult to be experimentally observed mainly due to problems related to the choice of $`\tau `$  (see also Appendix C).

## 4 Predictability in extended systems

Here we consider extended dynamical systems, whose degrees of freedom depend on space and time, and which can display unpredictable behaviors both in the time and space evolution, i.e. spatio-temporal chaos. The inadequacy of the Lyapunov exponents in characterizing predictability becomes now well evident.

Following Hohenberg and Shraiman  (see also ) we give a more precise meaning to the terms spatio-temporal chaos and extended systems. For a generic system of size $`L`$, we can define three characteristic lengths: $`\mathrm{}_D`$, $`\mathrm{}_E`$, $`\xi `$ respectively associated to the scales of dissipation (i.e. the scale at which dissipation becomes effective, smaller scales can be considered as inactive), excitation (i.e. the scale at which energy is injected in the system) and correlation (that we assume can be suitably defined). Now one has two limiting situations.

When all the characteristic lengths are of the same order ($`\mathrm{}_D,\mathrm{}_E,\xi O(L)`$) distant regions of the system are strongly correlated. Because of the coherence, the spatial nature is not very important and one speaks of temporal chaos, i.e. the system is basically low dimensional.

When $`L\xi \mathrm{}_D`$ distant parts of the system are weakly correlated so that the number of (active) degrees of freedom and consequently the number of positive Lyapunov exponents, the Kolmogorov-Sinai entropy and the attractor dimension, $`D_F`$, are extensive quantities, i.e. they increase with the system size, $`L`$. Here, spatial aspects are crucial and one speaks of spatio-temporal chaos, e.g., Rayleigh-Bérnad convection for large aspect ratio .

The above picture is just an approximative scenario (see  for further details) but sufficiently broad to include systems ranging from fluid dynamics to biological and chemical reactions . In spite of the fact that turbulent flows fit in this broad definition we shall discuss the predictability problem in turbulence in the next Section.

For detailed discussions on different physical and chemical systems which can be included in the above definition see . Here we discuss the predictability problem in a restricted class of models, which are relatively simple from a computational and theoretical point of view but, nevertheless, possess the essential phenomenology of spatio-temporal chaos.

### 4.1 Simplified models for extended systems and the thermodynamic limit

A great simplification in the study of extended systems, usually described by partial differential equations (PDE), can be achieved by considering discrete time and space models, and introducing the Coupled Map Lattices (CML) , i.e. maps defined on a discrete lattice. A typical $`1`$-dimensional CML (the extension to $`d`$-dimensions is straightforward) can be written in the following way:

$$𝐱_i(t+1)=(1\epsilon _0)𝐟_a[𝐱_i(t)]+\frac{1}{2}\underset{j=L/2}{\overset{L/2}{}}\epsilon _j\left(𝐟_a[𝐱_{i+j}(t)]+𝐟_a[𝐱_{ij}(t)]\right),$$
(4.1)
with $`\epsilon _0=_{j=1}^{L/2}\epsilon _j`$. $`L`$ is the lattice size, $`i=L/2,\mathrm{},L/2`$, $`𝐱\mathrm{I}R^n`$ is the state variable which depends on the site and time, and $`𝐟_a\mathrm{I}R^n\mathrm{I}R^n`$ is a non linear map, which drives the local dynamics and depends on a control parameter $`a`$. Usually, periodic boundary conditions $`𝐱_{i+L}=𝐱_i`$ are assumed and, for scalar variables ($`n=1`$), one studies coupled logistic maps, $`f_a(x)=ax(1x)`$ or tent maps, $`f_a(x)=a|1/2|x1/2||`$.

The parameters $`\{\epsilon _i\}`$ rule the strength and the form of the coupling and they are chosen according to the physical system under investigation. For example, with $`\epsilon _j=0`$ for $`j2`$, i.e. nearest neighbor coupling, one can mimic PDE’s describing reaction diffusion processes (indeed formally the equation assumes the structure of a discrete Laplacian). However, it could be misleading to consider CML’s simply as discrete approximation of PDE’s. Indeed, since the local map $`𝐟_a`$ is usually chaotic, chaos in CML, differently from PDE, is the result of many interacting chaotic sub-systems. Hence, the correspondence between the instability mechanisms in the two type of models is not straightforward .

Other kinds of coupling can be considered to mimic different physical situations, e.g., asymmetric CML (see Sect. 4.5) for studying convective instabilities , or mean field (globally coupled maps) version of (4.1) (Sect. 4.3) for studying neural network or population dynamics . Further generalizations are quoted in Ref. .

Lyapunov exponents, attractor dimensions and entropies can be defined (and, at least the Lyapunov exponents, numerically computed) also for extended systems. In particular, for $`L<\mathrm{}`$ the CMLs have finite dimensional phase space and the above quantities are well defined. In PDE’s some difficulties can rise due to the problem of the non equivalence of the norms : Lyapunov exponents and consequently the characterization of the predictability may depend on the chosen norm. We shall see in Sect. 4.8 that this is not just a subtle mathematical problem.

In order to build a statistical description of spatio-temporal chaos, as Ruelle pointed out , one has to require the existence of a good thermodynamic limit for the Lyapunov spectrum $`\{\lambda _i(L)\}_{i=1,L}`$. This means the existence of the limit

$$\underset{L\mathrm{}}{lim}\lambda _i(L)=\mathrm{\Lambda }(x),$$
(4.2)
where $`x=i/L`$ is a continuous index in $`[0,1]`$, and $`\mathrm{\Lambda }(x)`$ is a non increasing function. The function $`\mathrm{\Lambda }(x)`$ can be viewed as a density of Lyapunov exponents. If such limit does not exist, the possibility to build a statistical description of spatio-temporal chaos would be hopeless, i.e., the phenomenology of these systems would depend on $`L`$.

Once the existence of a Lyapunov density is proved, one can generalize some results of low dimensional systems , namely the Kaplan-Yorke conjecture  and the Pesin relation (2.16). For instance, one can generalize (2.16) to

$$_{KS}=\underset{L\mathrm{}}{lim}\frac{h_{KS}}{L}=_0^1𝑑x\mathrm{\Lambda }(x)\theta (\mathrm{\Lambda }(x))$$
(4.3)
being $`\theta (x)`$ the step function. In the same way one can suppose the existence of a dimension density $`𝒟_F`$, that is to say a density of active degrees of freedom, i.e. $`𝒟_F=lim_L\mathrm{}D_F/L`$ which by the Kaplan-Yorke  conjecture is given by :

$$_0^{𝒟_F}𝑑x\mathrm{\Lambda }(x)=0.$$
(4.4)
The existence of a good thermodynamic limit is supported by numerical simulations  and some exact results . Recently Eckmann and Collet  have proved the existence of a density of degrees of freedom in the complex Ginzburg-Landau equation. See also Refs.  and references therein for a discussion on such a problem.

### 4.2 Overview on the predictability problem in extended systems

In low dimensional systems, no matter how the initial disturbance is chosen, after a – usually short – relaxation time, $`T_R`$, the eigendirection with the largest growth rate dominates for almost all the initial conditions (this, e.g., helps in the numerical estimates of the Lyapunov exponents ). On the contrary, in high-dimensional systems this may not be true . Indeed, in systems with many degrees of freedom there is room for several choices of the initial perturbation according to the specific problem under investigation (e.g., localized on certain degrees of freedom or homogeneous in all the degrees of freedom), and it is not obvious that for all of them the time $`T_R`$ needed to align along the maximally expanding direction is the same.

In general the situation can be very complicated. For instance, it is known that, also considering initially homogeneous disturbances, the Lyapunov vectors can localize (permanently or slowly wandering) on certain degrees of freedom . Of course, this will severely affect the prediction of the future evolution of the system. Indeed, regions of large predictability time could coexist with regions of relatively short predictability time. In Ref.  one finds an abundance of examples displaying this phenomenology. A detailed analysis of this problem is far from the aims of this review; we just mention that the behavior of the Lyapunov vectors can range from a strongly localized regime (the origin of which can be understood by the analogy with Anderson localization of the wave function in disordered potential ) to localized wandering structures. In particular, in the latter case there is strong numerical evidence  that for a large class of ($`1`$-dimensional) systems the dynamics of the Lyapunov vectors (actually the logarithm of them) falls into the universality class of the $`1`$-dimensional KPZ equation .

In these situations the main contribution to the predictability time comes from the time needed for the perturbation to propagate through the system or to align along the maximally expanding direction, which can be of the order of the system size . As a consequence the predictability time can be much longer than the rough estimation $`T_p1/\lambda `$.

Moreover, the LE can also be unsatisfactory if one is interested in perturbations with particular space-time shapes. Indeed, these features have lead to the introduction of a number of new indicators; for instance, the temporal (or specific) Lyapunov exponents , the spatial Lyapunov exponents  (which characterize respectively perturbations exponentially shaped in time and space) or the comoving Lyapunov exponents  for the characterization of the spatio-temporal evolution of localized perturbation  and of the convective instabilities .

Convectively unstable systems are rather interesting because, even if the LE (computed in the stationary frame) is negative, some features of the motion can be highly unpredictable (see Sect. 4.6). It is also worth mentioning the existence of systems with exponentially long (in the system size) transients during which the dynamics is essentially unpredictable despite the fact that the LE is negative . This phenomenon, known under the name of stable chaos , will be briefly discussed in Sect. 7.3.

In high-dimensional systems one is also interested in predicting the behavior of some average observable to achieve a macroscopic description of the system. The coarse grained (hydrodynamic like) dynamics may be non trivial, and the largest LE, which is related to the fine grained dynamics, is not relevant for characterizing the predictability at a coarse grained level (see Sect. 4.7).

We conclude this brief overview by mentioning another interesting feature: in spatially distributed systems coherent structures may appear. They move maintaining for rather long times their shape. In different contexts one can be interested in predicting the evolution of such structures; e.g., cyclonic/anti-cyclonic structures in the atmosphere. A reasonable question could be the prediction of the center and orientation of such structures: limiting to these aspects one can indeed hope to have a rather long predictability time compared to the rough estimate $`T_pO(1/\lambda )`$. However, since usually such phenomena arise in fields, whose evolution is ruled by PDE, the non equivalence of the norms makes a general approach to the problem unfeasible. Therefore, one has to resort to ad hoc treatments, based on physical intuition to identify the most suitable norm to be used for the particular needs (see Sect. 4.8).

### 4.3 Butterfly effect in coupled map lattices

In spatially extended systems it is important to understand the way an uncertainty initially localized in some region will spread. Here we study in particular the time needed for a perturbation, initially seeded in the central site of a lattice of coupled maps, to reach a preassigned value at the border of the lattice  (see also  and Sect. 4.6). In other terms we wonder about the “butterfly effect” starting from the center of the lattice and arriving up to the boundary.

We shall discuss the properties of such time by varying the coupling range from local to non local in the $`1`$-dimensional CML (4.1) with periodic boundary conditions. We consider two cases: local coupling, i.e. $`\epsilon _j=0`$ if $`j2`$, and non-local coupling, e.g.

$$\epsilon _1=C_1\mathrm{and}\epsilon _j=\frac{C_2}{j^\alpha }\mathrm{for}j2$$
(4.5)
where $`\alpha `$ measures the strength of non-locality. The initial perturbation is on the central site, i.e.

$$|\delta x_i(0)|=\delta _0\delta _{i,0}.$$
(4.6)
We look at the predictability time $`T_p`$ needed for the perturbation to reach a certain threshold $`\delta _{max}`$ on the boundary of the lattice, i.e. the maximum time, $`t`$, such that $`|\delta x_{L/2}(t)|\delta _{max}`$.

For nearest neighbor coupling, one has obviously that $`\delta x_{L/2}(t)=0`$ for $`t<L/2`$. Indeed, by a numerical integration of (4.1) for the short range coupling one observes that $`\delta x_{L/2}(t)=0`$ for times $`t<t^{}L`$; while for $`t>t^{}`$ the perturbation, due to the (local) chaotic dynamics, grows as $`\delta x_{L/2}(t)\delta _0\mathrm{exp}[\lambda (tt^{})]`$. Thus for local interactions, the predictability is mainly determined by the waiting time $`t^{}`$, necessary to have $`|\delta x_{L/2}|>\delta _0`$, which is roughly proportional to the system size $`L`$. This is confirmed by Fig. 5, where it is shown that the average predictability time $`<T_p>`$ as a function of $`L`$ goes as

$$<T_p>=t_1+GL,$$
(4.7)
where the time $`t_1\lambda ^1`$ is due to the exponential error growth after the waiting time and can be neglected in large enough lattices. This agrees with the existence of a finite speed for the perturbation spreading ; indeed $`G`$ is related to the propagation velocity (see Sect. 4.6).

The scenario changes for non-local interactions (4.5). Now, due to the long range coupling, the perturbation (4.6) may propagate without any delay. The numerical results show that even for weak non-locality (e.g. $`C_2<<C_1`$ and rather large $`\alpha `$-values), the waiting time $`t^{}`$ does not increase (or increases very slowly) with $`L`$, so that

$$<T_p>t_1\lambda ^1.$$
(4.8)
As shown in Fig. 5, weakly non-local couplings, and mean field interactions ($`\epsilon _j=C_2/N`$) have the same qualitative behavior. Very accurate numerical computations have confirmed that the dependence on $`L`$ is indeed very weak (only logarithmic), i.e. $`<T_p>t_1+\alpha \mathrm{ln}L/\lambda _1`$ .

This example demonstrates that the predictability time is given by two contributions: the waiting time $`t^{}`$ and the characteristic time $`t_1\lambda ^1`$ associated with chaos. For non-local interactions, the waiting time practically does not depend on the system size $`L`$, while for local interactions it is proportional to $`L`$. Let us underline that in these results the non-linear terms in the evolution of $`\delta 𝐱(t)`$ are rather important. One numerically observes that the waiting time $`t^{}`$ is not just the relaxation time $`T_R`$ of $`\delta 𝐱`$ on the tangent eigenvector. Actually, $`T_R`$ is much larger than $`t^{}`$.

### 4.4 Comoving and Specific Lyapunov Exponents

A general feature of systems evolving in space and time is that a generic perturbation not only grows in time but also propagates in space. Aiming at a quantitative description of such phenomena Deissler and Kaneko  introduced a generalization of the LE to a non stationary frame of reference: the comoving Lyapunov exponent. For the sake of simplicity, we consider again the case of a 1-dimensional CML.

Let us consider an infinitesimally small perturbation initially different from zero only in one site of the lattice (4.6). By looking at the perturbation evolution along the line defined by $`j(t)=0+[vt]`$ (where $`[\mathrm{}]`$ denotes the integer part), one expects:

$$|\delta x_j(t)||\delta x_0(0)|e^{\lambda \left(v\right)t},$$
(4.9)
where $`\lambda (v)`$ is the largest comoving Lyapunov exponent, i.e.

$$\lambda (v)=\underset{t\mathrm{}}{lim}\underset{L\mathrm{}}{lim}\underset{|\delta x_0(0)|0}{lim}\frac{1}{t}\mathrm{ln}\left(\frac{|\delta x_{[vt]}(t)|}{|\delta x_0(0)|}\right).$$
(4.10)
In Eq. (4.10) the order of the limits is important to avoid boundary effects. For $`v=0`$ one recovers the usual LE. Moreover, one has that $`\lambda (v)=\lambda (v)`$ (and the maximum value is obtained at $`v=0`$ ) when a privileged direction does not exist, otherwise $`\lambda (v)`$ can be asymmetric and the maximum can be attained at value $`v0`$ (see Sect. 4.6). By writing the response function (2.8) in the moving frame one can also introduce the generalized comoving Lyapunov exponents $`L_q(v)`$ for studying finite time effects .

There are other two indicators related to the comoving LE: the local Lyapunov exponent  and the specific (or temporal) Lyapunov exponents. Here we only briefly discuss the latter which is indeed nothing but the Legendre transform of $`\lambda (v)`$.

The specific Lyapunov exponent, $`\mathrm{\Lambda }(\mu )`$, has been introduced by Politi and Torcini  to characterize the growth of exponentially shaped perturbations, i.e.

$$\delta x_i(t)=\mathrm{\Phi }_i(t)e^{\mu |i|}i=\frac{L}{2},\mathrm{},\frac{L}{2},$$
(4.11)
where $`\mathrm{\Phi }_i(t)`$ gives the fluctuation with respect to the pure exponential shape. One can see that $`\mathrm{\Lambda }(\mu )`$ is connected through a Legendre transform to $`\lambda (v)`$ . Indeed, Eq. (4.9) defines a local exponential profile with $`\mu =d\lambda (v)/dv`$, which means that in term of the specific Lyapunov exponents one expects the perturbation to grow according to

$$\delta x_i(t)\mathrm{exp}\{\mathrm{\Lambda }(\mu )t\mu i\},i=[vt].$$
(4.12)
Note that for $`\mu =0`$, $`\mathrm{\Lambda }(\mu )`$ reduces to the standard LE. Therefore, the comoving Lyapunov exponent is given by

$$\lambda (v)=\mathrm{\Lambda }(\mu )\mu \frac{d\mathrm{\Lambda }(\mu )}{d\mu }.$$
(4.13)
The last equation defines a Legendre transform from $`(\lambda (v),v)`$ to $`(\mathrm{\Lambda }(\mu ),\mu )`$ . By inverting the transformation (4.13) one obtains $`v=d\mathrm{\Lambda }(\mu )/d\mu `$.

Working in tangent space by using standard algorithms , one computes the specific Lyapunov spectrum $`\mathrm{\Lambda }_i(\mu )`$ with $`i=1,\mathrm{},L`$ for each $`\mu `$. In the limit $`L\mathrm{}`$ a density of such exponents exists and an interpretation of it is discussed in .

### 4.5 Convective chaos and spatial complexity

So far we have considered CML’s with symmetric spatial coupling, however there are many physical systems in which a privileged direction exists, e.g., boundary layers, thermal convection and wind induced water waves. The term usually employed for denoting such a class of systems is flow systems. See  for a discussion of flow systems in different physical contexts.

In recent years it has received much attention a simplified model for flow systems which is able to capture the basic phenomenology . A minimal model is a chain of maps with unidirectional coupling :

$$x_n(t+1)=(1c)f_a(x_n(t))+cf_a(x_{n1}(t)),$$
(4.14)
where $`t`$ and $`n`$ $`(=1,\mathrm{},L)`$ are the discrete time and space respectively; the map $`f_a(x)`$ is usually chosen to be the logistic map. One can consider different boundary conditions, $`x_0(t)`$. For instance, $`x_0(t)=x^{}`$ with $`x^{}`$ being an unstable fixed point of the map $`f_a(x)`$, or more generic time dependent boundary conditions where $`x_0(t)`$ is equal to a known function of time $`y(t)`$, which can be periodic, quasi-periodic or chaotic. Here, following Pikovsky , we consider a quasi-periodic boundary condition $`x_0(t)=0.5+0.4\mathrm{sin}(\omega t)`$, with $`\omega =\pi (\sqrt{5}1)`$. However, the results we are going to discuss do not depend too much on the details of the boundary conditions, i. e. on using $`x_0(t)`$ quasi-periodic or chaotic.

A central concept in the study of flow systems is the one of convective instability, i.e. when a perturbation grows exponentially along the flow but vanishes locally.

We may give a description of the phenomenology of flow systems in terms of the largest LE and of the comoving LE. The absolute stability is identified by the condition $`\lambda (v)<0`$ for all $`v0`$; the convective instability corresponds to $`\lambda _1=\lambda (v=0)<0`$ and $`\lambda (v)>0`$ for some velocities $`v>0`$ and finally standard chaos (absolute instability) is present when $`\lambda _1=\lambda (v=0)>0`$. See Fig. 6 for a sketch of the possible behaviors.

The convective instability is conceptually very interesting, because even if the largest LE is negative the behavior of the system may be very hard to predict, as Figure 7 suggests.

For this kind of spatial “complexity” there is not a simple and systematic characterization. A first explanation for these features may be found in the sensitivity of convective unstable systems on small perturbations at the beginning of the chain (always present in physical system), which grow exponentially while they are transmitted along the flow. This simple intuition can be made more quantitative defining an indicator which measures the degree of sensitivity on the boundary conditions. We wonder how an uncertainty $`|\delta x_0(t)|=\delta _0`$ in the knowledge of the boundary conditions will affect the system. We consider only the case of infinitesimal perturbations, i.e. $`\delta x_n`$ evolves according to the tangent space dynamics, and, for the moment we do not consider intermittency (i.e. time fluctuations of the comoving Lyapunov exponents).

The uncertainty $`\delta x_n(t)`$, on the determination of the variable at time $`t`$ and site $`n`$, is given by the superposition of the evolved $`\delta x_0(t\tau )`$ with $`\tau =n/v`$:

$$\delta x_n(t)\delta x_0(t\tau )e^{\lambda (v)\tau }𝑑v=\delta _0e^{[\lambda (v)/v]n}𝑑v.$$
(4.15)
Since we are interested in the asymptotic spatial behavior, i.e. large $`n`$, we can write:

$$\delta x_n(t)\delta _0e^{\mathrm{\Gamma }n},$$
(4.16)
The quantity $`\mathrm{\Gamma }`$ can be considered as a sort of spatial-complexity-index, an operative definition of which is the following:

$$\mathrm{\Gamma }=\underset{n\mathrm{}}{lim}\frac{1}{n}\mathrm{ln}\frac{|\delta x_n|}{\delta _0},$$
(4.17)
where the brackets mean a time average.

In the particular case of a non intermittent system, a simple saddle-point estimate of Eq. (4.15) gives

$$\mathrm{\Gamma }=\underset{v}{\mathrm{max}}\left[\frac{\lambda (v)}{v}\right].$$
(4.18)
Equation (4.18) is a link between the comoving and the “spatial” Lyapunov exponent $`\mathrm{\Gamma }`$, i.e. a relation between the convective instability of a system and its sensitivity to the boundary conditions.

Eq. (4.18) holds exactly only in absence of intermittency; in general the relation is more complicated. One can introduce the effective comoving Lyapunov exponent, $`\stackrel{~}{\gamma }_t(v)`$, that gives the exponential changing rate of a perturbation, in the frame of reference moving with velocity $`v`$, on a finite time interval $`t`$. According to general arguments (see Sect. 3.1 and ) one has $`\stackrel{~}{\gamma }_t(v)=\lambda (v)`$. Then, instead of (4.15) one obtains

$$\delta x_n(t)\delta _0e^{[\stackrel{~}{\gamma }_t(v)/v]n}𝑑v,$$
(4.19)
and therefore:

$$\mathrm{\Gamma }=\underset{n\mathrm{}}{lim}\frac{1}{n}\mathrm{ln}\frac{|\delta x_n|}{\delta _0}=\underset{n\mathrm{}}{lim}\frac{1}{n}\mathrm{ln}\frac{|\delta x_n^{typical}|}{\delta _0}=\underset{v}{\mathrm{max}}\left[\frac{\stackrel{~}{\gamma }_t(v)}{v}\right].$$
(4.20)
Therefore, because of the fluctuations, it is not possible to write $`\mathrm{\Gamma }`$ in terms of $`\lambda (v)`$, although one can obtain a lower bound :

$$\mathrm{\Gamma }\underset{v}{\mathrm{max}}\left[\frac{\stackrel{~}{\gamma }_t(v)}{v}\right]=\underset{v}{\mathrm{max}}\left[\frac{\lambda (v)}{v}\right]\mathrm{\Gamma }^{}.$$
(4.21)
In Fig. 8 we show $`\mathrm{\Gamma }`$ and $`\mathrm{\Gamma }^{}`$ vs. $`a`$ for a fixed value of $`c`$. There is a large range of values of the parameter $`a`$ for which $`\mathrm{\Gamma }`$ is rather far from $`\mathrm{\Gamma }^{}`$. This difference is only due to intermittency, as investigations of the map $`f_a(x)=ax`$ mod $`1`$ or the computation of the generalized spatial Lyapunov exponents $`L_s(q)`$  confirm.

Concluding, we underline that the spatial complexity displayed by these systems indicates that the unpredictability of a system cannot be completely reduced to the existence of at least one positive LE.

### 4.6 Space-time evolution of localized perturbations

In Fig. 9 we show the evolution of a perturbation $`|\delta x_i(t)|`$ initialized as (4.6) as a function of space and time ($`i,t`$), for a 1-dimensional lattice of locally coupled tent maps. The perturbation grows in time and propagates linearly in space creating a sort of predictability “horizon”: this defines a propagation velocity $`V_F`$ .

The velocity $`V_F`$ is defined in terms of the left and right edges of the disturbance i.e. the first left (right) site for which at time $`t`$ the perturbation reaches a preassigned arbitrary threshold. Numerically it has been found that $`V_F`$ is independent either of the amplitude of the initial perturbation, $`\delta _0`$, and of the threshold value, so that it is a well defined quantity .

It is easy to realize that $`V_F`$ is nothing but the velocity of a moving frame of reference in which the perturbation is seen neither to grow nor to decrease (i.e. the frame comoving with the edges of the perturbation). Therefore, $`V_F`$ is given by

$$\lambda (V_F)=0.$$
(4.22)
The interesting point in Eq. (4.22) is that it gives not only a definite prescription to derive the propagation velocity but also a link between the velocity and the stability properties of the system. From this point of view it is instructive to look at the propagation velocity in another way .

The perturbation at different times resembles a propagating front, similar to those encountered in reaction-diffusion processes. But while in the latter context the front usually separates two stable phases or a stable from an unstable phase, here one phase is unstable and the other chaotic . Made this analogy one can ask if it is possible to obtain the propagation velocity as in reaction-diffusion phenomena, where we know that the dynamics spontaneously selects the minimum allowed velocity .

Torcini et al.  have shown that this is indeed the case. They studied the evolution of a perturbation with an exponential profile (4.11) which, according to the definition of the specific Lyapunov exponent, evolves as in Eq.(4.12), i.e. $`\delta x_i(t)\mathrm{exp}\{\mathrm{\Lambda }(\mu )t\mu i\}`$. This last relation tells us that the velocity of a front with shape given by $`\mu `$ is $`V(\mu )=\mathrm{\Lambda }(\mu )/\mu `$. According to the analogy with reaction-diffusion systems, one expects that a generic localized perturbation develops an exponential decaying shape (at the edges) with a definite exponent $`\mu _0`$ (selected by the dynamics) . This means that the propagation velocity $`V_F`$ is determined by the relation $`V_F=V(\mu _0)`$. Now the problem is to compute $`\mu _0`$.

From Eq. (4.13), which relates $`\mathrm{\Lambda }(\mu )`$ and $`\lambda (v)`$ through a Legendre transformation, one obtains

$$\frac{dV}{d\mu }=\frac{1}{\mu }\left(\frac{d\lambda }{d\mu }\frac{\lambda }{\mu }\right)=\frac{\lambda (v)}{\mu ^2}.$$
(4.23)
Moreover, since $`\lambda (V_F)=0`$ (4.22) one has that $`dV/d\mu =0`$ at $`\mu _0`$ such that $`V(\mu _0)=V_F`$, i.e. $`\mu _0`$ selects the minimal velocity. Indeed $`\mathrm{\Lambda }(\mu )`$ is convex (being a Legendre transform), so that the minimum is unique and

$$V_F=\frac{\mathrm{\Lambda }(\mu _0)}{\mu _0}=\frac{d\mathrm{\Lambda }(\mu )}{d\mu }|_{\mu =\mu _0}.$$
(4.24)
Thus for an infinitesimal perturbation, the selected velocity is the lowest possible one .

Summarizing, for short range coupling the speed of propagation is finite and fully determines the spatio-temporal evolution of the perturbation. The situation becomes different for long-range coupling as (4.5). In this case the velocity of propagation is unbounded . For the sake of completeness, we mention that the long-range coupling case has been also investigated in terms of a specific-like Lyapunov exponent which characterizes power law shaped perturbations . The result of this analysis shows that the perturbation propagates exponentially fast with a rate given by the ratio of the largest LE and the power of the coupling.

We conclude this Section by mentioning that there are cases in which the analysis in terms of $`\lambda (v)`$ or, equivalently, $`\mathrm{\Lambda }(\mu )`$ fails to give the measured propagation velocity. Indeed, it has been found that $`V_F`$ can be larger than the velocity for which $`\lambda (v)=0`$. A finite propagation velocity has been measured even in systems with $`\lambda <0`$ (the so-called stable chaos phenomenon, see Section 7.3) for which the above analysis predicts $`V_F=0`$ .

This failure is related to the presence of strong non linearities. Recently, it has been proposed to generalize (4.22) to the non linear regime of the perturbation growth by the definition of the Finite Size Comoving Lyapunov Exponent , $`\lambda (\delta ,v)`$. It measures the divergence rate of perturbations of size $`\delta `$ (not necessarily infinitesimal) in a moving frame. The algorithm is a generalization of the FSLE (Sect. 3.4), where now one follows an initially localized perturbation along the line $`[vt]`$. In Fig. 10 we compare $`\lambda (v)`$ with $`\lambda (\delta ,v)`$ for a CML of shift maps, i.e. $`f(x)=rx\text{mod}\mathrm{\hspace{0.17em}1}`$. The latter goes to zero exactly at the directly measured propagation velocity $`V_F`$. Similar results hold for other maps . These results suggest that a generalization of Eq.(4.22), which is able to take into account also possible non linear effects, is:

$$\underset{\delta }{\mathrm{max}}\left[\lambda (\delta ,V_F)\right]=0.$$

The numerical evidences also suggest that the condition which should be accomplished in order to have deviation from the linear prediction given by (4.22) and (4.24) is that $`\lambda (\delta ,v=0)>\lambda (0,0)=\lambda `$, confirming a conjecture done in . However, even if interesting such a behavior seems to be rather non generic.

### 4.7 Macroscopic chaos in Globally Coupled Maps

Recently the emergence of non trivial collective behaviors in high-dimensional dynamical systems has gathered much attention. A limit case of macroscopic coherence is the global synchronization of all the parts of the system. Beyond synchronization there exist other interesting phenomena, among which we just mention: clustering and collective motion in globally coupled maps (GCM). The latter behavior, in the case that we call macroscopic chaos  (see below), is the subject of this Section.

Let us consider a globally coupled map (GCM) defined as follows

$$x_n(t+1)=(1\epsilon )f_a(x_n(t))+\frac{\epsilon }{N}\underset{i=1}{\overset{N}{}}f_a(x_i(t)),$$
(4.25)
where $`N`$ is the total number of elements.

The evolution of a macroscopic variable, e.g., the center of mass

$$m(t)=\frac{1}{N}\underset{i=1}{\overset{N}{}}x_i(t),$$
(4.26)
upon varying $`\epsilon `$ and $`a`$ in Eq. (4.25), displays different behaviors :

(a) Standard Chaos: $`m(t)`$ obeys a Gaussian statistics with a standard deviation $`\sigma _N=\sqrt{m(t)^2m(t)^2}N^{1/2}`$;

(b) Macroscopic Periodicity: $`m(t)`$ is a superposition of a periodic function and small fluctuations $`O(N^{1/2})`$;

(c) Macroscopic Chaos: $`m(t)`$ displays an irregular motion as it can be seen by looking at the plot of $`m(t)`$ vs. $`m(t1)`$ that appears as a structured function (with thickness $`N^{1/2}`$), and suggests a chaotic motion for $`m(t)`$.

Phenomena (a) and (b) also appear in CML with local coupling in high enough dimensional lattices , for the interesting case (c), as far as we know, there is not a direct evidence in finite dimensional CMLs.

In the case of macroscopic chaos one can expect that the center of mass evolves with typical times longer than the characteristic time of the full dynamics (i.e. the microscopic dynamics); the order of magnitude of the latter time may be estimated as $`1/\lambda _1`$. Indeed, conceptually, macroscopic chaos for GCM can be thought of as the analogous of the hydro-dynamical chaos for molecular motion, where, in spite of a huge microscopic Lyapunov exponent ($`\lambda _11/\tau _c10^{11}s^1`$, $`\tau _c`$ being the collision time), one can have rather different behaviors at a hydro-dynamical (coarse grained) level, i.e.: regular motion ($`\lambda _{hydro}0`$) or chaotic motion ($`0<\lambda _{hydro}\lambda _1`$). In principle, if one knows the hydrodynamic equations, it is possible to characterize the macroscopic behavior by means of standard dynamical system techniques. However, in generic CML there are no general systematic methods to build up the macroscopic equations, apart from particular cases . Therefore, here we discuss the macroscopic behavior of the system relying upon the full microscopic level of description.

The microscopic Lyapunov exponent cannot give a characterization of the macroscopic motion. To this purpose, recently different approaches have been proposed based on the evaluation of the self-consistent Perron-Frobenius (PF) operator  and on the FSLE . Despite the conceptual interest of the former (in some sense the self-consistent PF-operator plays a role similar to that of the Boltzmann equation for gases ), here we shall only discuss the latter which seems to us more appropriate to address the predictability problem.

We recall that for chaotic systems, in the limit of infinitesimal perturbations $`\delta 0`$, one has $`\lambda (\delta )\lambda _1`$, i.e. $`\lambda (\delta )`$ displays a plateau at the value $`\lambda _1`$ for sufficiently small $`\delta `$. However, for non infinitesimal $`\delta `$, one can expect that the $`\delta `$-dependence of $`\lambda (\delta )`$ may give information on the characteristic time-scales governing the system, and, hence, it could be able to characterize the macroscopic motion. In particular, at large scales, i.e. $`\delta 1/\sqrt{N}`$, one expects the (fast) microscopic components to saturate and $`\lambda (\delta )\lambda _M`$, where $`\lambda _M`$ can be fairly called the “macroscopic” Lyapunov exponent.

The FSLE has been determined by looking at the evolution of $`|\delta m(t)|`$, which has been initialized at the value $`\delta m(t)=\delta _{min}`$ by shifting all the elements of the unperturbed system by the quantity $`\delta _{min}`$ (i.e. $`x_i^{^{}}(0)=x_i(0)+\delta _{min}`$), for each realization. The computation has been performed by choosing the tent map as local map, but similar results can be obtained for other maps .

Figure 11a shows $`\lambda (\delta )`$ versus $`\delta `$ in the case of macroscopic chaos. One has two plateaus: at small values of $`\delta `$ ($`\delta \delta _1`$), as expected from general considerations, $`\lambda (\delta )=\lambda _1`$; for $`\delta \delta _2`$ one has another plateau, the “macroscopic” Lyapunov exponent, $`\lambda (\delta )=\lambda _M`$. Moreover, $`\delta _1`$ and $`\delta _2`$ decrease at increasing $`N`$: indeed, by looking at Fig. 11b one can see that $`\delta _1,\delta _21/\sqrt{N}`$. It is important to observe that the macroscopic plateau, being almost non-existent for $`N=10^4`$, becomes more and more resolved and extended on large values of $`\delta \sqrt{N}`$ at increasing $`N`$ up to $`N=10^7`$. Therefore we can argue that the macroscopic motion is well defined in the limit $`N\mathrm{}`$ and one can conjecture that in this limit the microscopic signature in the evolution of $`\delta m(t)`$ completely disappears in favor of the macroscopic behavior. In the case of standard chaos ($`\lambda _M<0`$) one has only the microscopic plateau and then a fast decreasing of $`\lambda (\delta )`$.

### 4.8 Predictability in presence of coherent structures

Here we discuss some problems which arise in characterizing the predictability in continuous systems, described by PDE. In this case the norms are not equivalent  and the computation of the LE can give different results. Rather than discussing the problem in general terms, we consider here two-dimensional turbulence as a specific example. The choice of this example is due to several reasons. First of all, two-dimensional turbulence is a continuous system of relevance in atmospheric physics, and it has been extensively investigated in the last years . The statistical theory for two-dimensional turbulence has been developed by Kraichnan and Batchelor  on a similar basis to the Kolmogorov theory for three-dimensional turbulence. The main formal difference is the existence of a second inviscid invariant, the enstrophy (average square vorticity). As a consequence, in the limit of high Reynolds numbers, the energy cannot be dissipated by viscosity and one expects a direct cascade of enstrophy. With an input source at intermediate scales, the energy injected into the system is transferred to large scales by an inverse cascade. A large numbers of numerical simulations  and experiments  have demonstrated the universality of the inverse cascade with spectral index very close to the predicted Kolmogorov exponents.

The situation is much less clear for what concerns the direct cascade. The predicted spectral slope (Kraichnan–Batchelor spectrum) is seldom observed and even universality with respect to the forcing or to the form of dissipation is questioned . The freely decaying evolution is characterized by the emergence of coherent structures  which eventually dominate the dynamics. Coherent structures are weakly dissipative, rather regular regions of fluids in the turbulent background flow whose interactions can be approximately described by a conservative dynamics . The spontaneous emergence of coherent structures makes two-dimensional turbulence a prototype model for geophysical flows  and, most important for our purpose, gives a natural example for illustrating the effects of choosing different error norms.

The equation for describing two-dimensional turbulence is the Navier–Stokes equation written for the scalar vorticity $`\omega =\times 𝐯`$ as

$$\frac{\omega }{t}+\frac{(\psi ,\omega )}{(x,y)}=(1)^{p+1}\nu _p\mathrm{}^p\omega $$
(4.27)
where $`\psi `$ is the stream function such that $`𝐯=(_y\psi ,_x\psi )`$ and $`\mathrm{}\psi =\omega `$. As customary in direct numerical simulations, the dissipation is modified by employing high order viscosity $`p>1`$ in order to achieve larger Reynolds numbers. The numerical results discussed below are obtained by integrating (4.27) by means of a standard pseudo–spectral code on a periodic computational domain with resolution $`N\times N`$.

The classical theory of predictability in turbulence  studies the evolution of a difference (or error) field, defined as

$$\delta \omega (𝐱,t)=\frac{1}{\sqrt{2}}\left(\omega ^{}(𝐱,t)\omega (𝐱,t)\right)$$
(4.28)
where $`\omega `$ and $`\omega ^{}`$ are solutions of (4.27) started from slightly different initial conditions. The “error” is computed from $`\delta \omega `$ and measured in terms of a given norm which is the subject of our discussion. Indeed the method used for defining the distance between the reference and perturbed field is a delicate point for continuous systems such as Navier–Stokes equations. Classical norms are based on the invariants of (4.27) in the inviscid limit $`\nu _p=0`$, i.e. enstrophy and energy norms

$`Z_\delta (t)`$ $`=`$ $`{\displaystyle \frac{1}{2}}{\displaystyle d^2x|\delta \omega (𝐱,t)|^2}={\displaystyle _0^{\mathrm{}}}𝑑kZ_\delta (k,t)`$ (4.29)
$`E_\delta (t)`$ $`=`$ $`{\displaystyle _0^{\mathrm{}}}dkk^2Z_\delta (k,t).={\displaystyle _0^{\mathrm{}}}dkE_\delta (k,t)`$ (4.30)
where we have also introduced the enstrophy ($`Z_\delta `$) and energy ($`E_\delta `$) error spectra. It is also natural to introduce the relative errors, defined as

$$r(t)=\frac{E_\delta (t)}{E(t)},z(t)=\frac{Z_\delta (t)}{Z(t)}$$
(4.31)
where $`E(t)=1/2v^2(x)𝑑x`$ and $`Z=1/2\omega ^2(x)𝑑x`$, and the relative error spectrum

$$r(k,t)=\frac{E_\delta (k,t)}{E(k,t)}=\frac{Z_\delta (k,t)}{Z(k,t)},$$
(4.32)
This issue was already addressed in  where the infinitesimal (linear) error growth was computed using several “Eulerian norms” as (4.29)-(4.30).

We will consider an initial error given by complete uncertainty at small scales, i.e. $`r(k,0)=0`$ for $`k<k_0`$ and $`r(k,0)=1`$ for $`k>k_0`$. This assumption is physically justified by the finite resolution of any measurement device and/or the numerical simulation scheme. For an infinitesimal perturbation, the error is expected to grow exponentially with the largest LE $`\lambda _1`$. Because we are dealing with a dissipative system which ultimately collapses on the trivial fixed point $`\omega =0`$, $`\lambda _1`$ is formally negative. However, this is only a formal problem. Indeed in high Reynolds number turbulence the dissipation time scale is much longer than the dynamical time and we can make use of the effective LE $`\gamma (t)`$ (3.8). For $`t`$ much smaller than the dissipation time, we can consider, from any point of view, $`\gamma (t)`$ as the Lyapunov exponent of the decaying turbulence.

The exponential growth regime ends at times much smaller than the dissipative time, as soon as the separation of the two fields cannot be any more considered infinitesimal and the finite error regime sets in. The predictability time is defined by means of the accepted tolerance $`\mathrm{\Delta }`$ or, which is equivalent, by a threshold for the relative errors (4.31). We will follow the classical prescription for the predictability time $`r(T_p)=1/4`$ . In Figure 12 we plot relative errors (4.31) as functions of time for a $`512^2`$ resolution simulation . For small times ($`t<250`$) we can see an exponential growth for both $`r(t)`$ and $`z(t)`$ with effective LE $`\gamma 0.08`$. At larger times the error curves bend and a predictability time estimation with energy norm gives $`T_p395`$. From Figure 12 we learn at least two lessons. First (but not surprisingly) about half of the predictability time is governed by non-exponential error growth behavior. This is another demonstration of the little relevance of LE for characterizing predictability in realistic complex systems. The second observation is that the different norms $`r(t)`$ and $`z(t)`$ give qualitatively similar results. Because the error is initially confined to small scales $`k>k_0`$, the vorticity-based norm is always larger than the energy-based norm, but the predictability time is essentially independent of the norm used. It is not difficult to understand that any Eulerian norm would give comparable result. Because the error propagates from small to large scales, a norm which emphasizes small scale features (as the enstrophy norm) saturates earlier than a large scale based norm (energy, in our example), but the results remain essentially the same.

In Figure 13 we plot the vorticity field of the reference $`\omega (𝐱,T_p)`$ and perturbed field $`\omega ^{}(𝐱,T_p)`$ at the predictability time $`T_p`$. Although the two fields differ, by definition, by 25% in energy and about 65% in enstrophy, they look still remarkably similar for what concerns the distribution of vortices. Most of the large coherent structures are almost in the same positions.

In Figure 13 we also plot the difference field $`\delta \omega (𝐱,T_p)`$. The typical bipolar configuration, usually observed in simulations , indicates that the error is concentrated in correspondence of the vortices and that it is essentially due to the different position of the vortex structures in the two realizations.

This result suggests that a Lagrangian measure of the error, based on the vortex positions, would be more suitable for the present system. For example, to emphasize the limits of the Eulerian measure for the error (4.29,4.30), consider the limiting case of singular point vortices, where an infinitesimal error in the coordinates gives error saturation and hence zero predictability time. In general, we expect that, in presence of vortices, an Eulerian-based measure underestimates the predictability time.

This problem can be overcome by resorting to the natural distance among vortex centers. We use a vortex tracking algorithm which recognizes and follows vortices during the dynamics. First we need a definition of vortex, the one here adopted is: a connected region $`D_\alpha `$ in the computational domain with vorticity maximum $`z_\alpha `$ larger (in absolute value) than a given threshold and vorticity larger than a fraction (we used $`0.2`$) of the vorticity peak. Given the vortex domains $`D_\alpha `$, all the physical quantities are computed by integrating inside the domains. For example, vortex circulation is defined as $`\mathrm{\Gamma }_\alpha =_{D_\alpha }d^2x\omega (𝐱)`$ and vortex center $`𝐱_\alpha `$ is the center of mass computed from the vorticity field. Finally, vortex trajectories are reconstructed by matching center positions at different times.

A Lagrangian, vortex-based, measure of the error can, e.g., be defined as

$$d^2(t)=\frac{1}{_\alpha |\mathrm{\Gamma }_\alpha |}\underset{\alpha }{}|\mathrm{\Gamma }_\alpha ||𝐱_\alpha ^{}𝐱_\alpha |^2$$
(4.33)
where $`𝐱_\alpha `$ and $`𝐱_\alpha ^{}`$ are the vortex positions respectively in the reference and perturbed field. In Figure 14 we plot $`d^2`$ obtained from our simulation. We observe that at the classical predictability time, the mean vortex separation is $`d(T_\mathrm{p})0.5`$, well below the saturation value ($`d_{\mathrm{max}}L/2=\pi `$ in the periodic computational box). This result is a quantitative confirmation of the observations drawn from Figure 13, i.e. the existence of an intermediate regime in which the (finite) error is ruled by the displacement of the strong coherent structures. If one is interested in predicting, with some tolerance, positions and intensities of coherent structures, it is possible to have a much larger predictability time.

## 5 Predictability in fully developed turbulence

### 5.1 Basic facts of turbulence

Perhaps, fully developed turbulence  is the most important instance of high-dimensional chaotic system. To give an example, let us consider a classical experiment in fluid dynamics: in a wind tunnel, an air mass conveyed by a large fan impinges some obstacles, which perturb significantly the velocity of fluid particles. Big and small whirls appear, and the flow evolves irregularly in time. One could wonder whether the features of the flow depend crucially on the physical properties of the fluid, the size and shape of the obstacle, the mean wind velocity, and so on. It is easy to understand that, with a given geometry the only relevant parameter which characterizes the flow is the Reynolds number $`Re=UL/\nu `$, where $`U`$ is the mean wind velocity, $`L`$ is the typical size of the obstacle and $`\nu `$ is the kinematic viscosity of the fluid. When $`Re`$ is very large, i.e., of the order of a thousand or more  turbulence is called fully developed. The fundamental physical interest in this regime is motivated by the existence of universal properties with respect to the details of the experimental setup . If a velocity probe is placed at some distance past the obstacle, it is possible to record a temporal series that gives us statistical information. If one sits far enough from the obstacle, there the small-scale properties of the flow do not depend sensitively on the precise site and orientation of the probe, that is the turbulence is approximately homogeneous and isotropic. Since the flow is swept across the probe at a mean velocity $`U`$, that largely exceeds the magnitude of the fluctuations, one can expect that the time record essentially amounts to a one-dimensional spatial section of the velocity field. Thus time-scales and length-scales are interchangeable, this is the essence of the Taylor hypothesis . Assuming the above hypothesis, we can safely reinterpret temporal variations of the velocity, on an interval $`\tau `$, in a fixed-point of the space as spatial increments on scale $`\mathrm{}=U\tau `$, at a fixed-time.

The first important result about the expected universality is the behavior of the velocity power spectrum which closely follows a power law decay $`E(k)k^{5/3}`$ on a given range of wave-numbers . At larger wave-number the spectrum falls off with an exponential-like behavior, whereas the form at small $`k`$ (i.e. large scales) depends on the mechanism of forcing and/or boundary conditions. For $`k0`$ one often observes a self-similar energy spectrum $`E(k)k^s`$ with scaling exponent $`s>0`$. In incompressible decaying turbulence, there are some arguments indicating that asymptotically $`s4`$ where the limiting value $`s=4`$ is observed if initially the spectrum has $`s>4`$ . A typical turbulence spectrum is shown in Figure 15.

The two crossovers unveil the presence of two characteristic scales: a large excitation scale $`Lk_L^1`$, associated with the energy containing eddies, and a small dissipation scale $`\mathrm{}_Dk_D^1`$, related to the smallest active eddies. The appearance of a power law in between these two extremes unveils that no other characteristic scale is involved.

A simple and elegant explanation of these experimental findings is due to A.N. Kolmogorov : in a nutshell, it is assumed the existence of a range of scales where the energy – injected at the scale $`L`$ – flows down (with a cascade process, as remarked by Richardson ) to the dissipative scale $`\mathrm{}_D`$, where it is dissipated by molecular viscosity. Since, practically, neither injection nor dissipation takes place in this interval, it is called the inertial range. In this range the only relevant quantity is the average energy transfer rate $`\overline{\epsilon }`$: dimensional counting imposes then a power spectrum $`E(k)\overline{\epsilon }^{2/3}k^{5/3}`$ in agreement with the experimental observations discussed above. The scaling for the spectrum is equivalent to a power law dependence for the second order structure function (SF)

$$S_2(\mathrm{})=\delta v_{\mathrm{}}^2=\left(v(x+\mathrm{})v(x)\right)^2\mathrm{}^{2/3}.$$
(5.1)
The original Kolmogorov theory (K41) assumes self-similarity of the turbulent flow. As a consequence, the scaling behavior of higher order structure functions $`S_p(\mathrm{})=|v(x+\mathrm{})v(x)|^p\mathrm{}^{\zeta _p}`$ is described by a single scaling exponent. The value of the exponent is determined by the so-called “4/5 law”, an exact relation derived by Kolmogorov from the Navier–Stokes equations , which, under the assumption of stationarity, homogeneity and isotropy states

$$\delta v_{||}^3(\mathrm{})=\frac{4}{5}\overline{\epsilon }\mathrm{},$$
(5.2)
where $`\delta v_{||}(\mathrm{})`$ is the longitudinal velocity difference between two points at distance $`\mathrm{}`$, and $`\overline{\epsilon }`$ is the average rate of energy transfer. The structure function exponent $`\zeta _p`$ is thus predicted by Kolmogorov similarity theory to be $`\zeta _p=p/3`$.

Several experimental investigations  have shown that the Kolmogorov scaling is not exact and $`\zeta _p`$ is a nonlinear function (with $`\zeta _3=1`$ as a consequence of the “4/5 law”). This means a breakdown of the self–similarity in the turbulent cascade. Larger and larger excursions from mean values are observed as one samples smaller and smaller scales. This phenomenon goes under the name of intermittency . A complete theoretical understanding of intermittency in Navier-Stokes turbulence is still lacking. Nevertheless, there are approaches, as the multifractal model , which are able to characterize at a phenomenological level the intermittency.

In brief the basic idea of the multifractal model  consists in assuming a local scale-invariance for the velocity fluctuations, i.e. one has $`\delta v_{\mathrm{}}\mathrm{}^h`$, with a continuous spectrum of (Hölder) exponents $`h`$, each belonging to a given fractal set. In other words, in the inertial range one has

$$\delta v_{\mathrm{}}(x)\mathrm{}^h,$$
(5.3)
if $`xS_h`$, and $`S_h`$ is a fractal set with dimension $`D(h)`$ and $`h`$ ($`h_{min}`$, $`h_{max}`$). The probability to observe a given scaling exponent $`h`$ at the scale $`\mathrm{}`$ is thus $`P_{\mathrm{}}(h)\mathrm{}^{3D(h)}`$. In this language the Kolmogorov similarity theory  corresponds to the case of only one singularity exponent $`h=1/3`$ with $`D(h=1/3)=3`$, see also Appendix B.

### 5.2 Reduced model of turbulence

In numerical simulations of the Navier-Stokes equations in the regime of fully developed turbulence, one has to discretize the original PDE to obtain a set of approximate ODE which must be integrated numerically. This is the direct numerical simulation approach which, in its simplest form, is implemented on a regular $`3D`$ grid of $`N^3`$ points. Since the dissipative scale (Kolmogorov scale) is related to the Reynolds number as $`\mathrm{}_DLRe^{3/4}`$, an estimate of the number $`𝒩`$ of active spatial degrees of freedom leads to

$$𝒩(L/\mathrm{}_D)^3Re^{9/4}.$$
(5.4)
An obvious consequence of the fast growth of $`𝒩`$ with the Reynolds number is the unfeasibility of a complete turbulent simulations at high $`Re`$. The maximum limit of present computers is about $`N=10^3`$ which corresponds to $`Re10^4`$.

An alternative approach has been introduced with the so called shell models by the works of Obukhov, Gledzer and Desnyansky and Novikov (see  for a detailed discussion). The basic idea, originally motivated in the context of closure theory, is to implement a dynamical cascade with a set of variables $`u_n`$ ($`n=1,\mathrm{},N`$) each representing the typical magnitude of the velocity fluctuation in a shell of wave-numbers $`k_n<|𝐤|<k_{n+1}`$. The representative wave-numbers are spaced geometrically, $`k_n=k_02^n`$, in this way, assuming locality in the cascade, interactions are confined to neighboring shells only.

We will discuss a specific model, known as GOY model (see  for a review), which makes use of complex velocity variables $`u_n`$ and for which the equations of motion are

$`\left({\displaystyle \frac{d}{dt}}+\nu k_n^2\right)u_n=ik_n\left(u_{n+1}u_{n+2}{\displaystyle \frac{1}{4}}u_{n1}u_{n+1}{\displaystyle \frac{1}{8}}u_{n2}u_{n1}\right)^{}+f_n,`$
(5.5)
where $`\nu `$ is the viscosity and $`f_n`$ is a forcing term (typically restricted on the first shells). The coefficients in the nonlinear term (which has the same structure of Navier-Stokes equations) are chosen to conserve energy $`E1/2_n|u_n|^2`$ in the unforced, inviscid limit.

Without entering in the details, we recall that shell model (5.5) displays energy cascade à la Kolmogorov from the large scales of forcing to the dissipative scales ($`nN`$) with a statistical constant energy flux $`\overline{\epsilon }`$. On these inertial range scales, the moments of velocity show power law scaling $`|u_n|^pk_n^{\zeta _p}`$ with exponents close to those experimentally observed for fully developed turbulence.

The number of shells $`N`$ necessary to mimic the cascade mechanism of fully developed turbulence is rather small, due to the geometrical progression in $`k_n`$ one has $`N\mathrm{log}_2Re`$. We have thus a chaotic dynamical system with a reasonably small number of degrees of freedom where standard methods of deterministic chaos can be used in order to relate the “turbulent” statistical description in terms of structure functions and intermittency, and dynamical properties, such as the spectrum of Lyapunov exponents. The absence of any stochastic term in (5.5) makes the shell model a natural model for investigating the predictability problem in turbulence.

### 5.3 Effects of intermittency on predictability of infinitesimal perturbations

The sensitive dependence on initial conditions makes the long term forecasting in turbulent flow practically impossible. For instance, Ruelle  remarked that thermal fluctuations in the atmosphere produces observable changes on a scale of centimeters after only few minutes. As a consequence after one or two weeks, the large scale atmospheric circulation would be completely unpredictable, even if the exact evolution equations were known. This is the so-called butterfly effect, in the words of Lorenz: A butterfly moving its wings in Brazil might cause the formation of a tornado over Texas. To support this argument, one can observe that the largest LE of fully developed turbulence is roughly proportional to the inverse of the smallest characteristic time of the system, the turn-over time $`\tau _D`$ of eddies of the size of the Kolmogorov length $`\mathrm{}_D`$. From $`\mathrm{}_DLRe^{3/4}`$ one obtains

$$\tau _D\mathrm{}_D/\delta v_D\tau _LRe^{1/2},$$
(5.6)
where $`\tau _LL/U`$ is the eddy turn-over time of the energy containing scales. As a consequence, as first pointed out by Ruelle , the largest LE scales with $`Re`$ like $`\lambda 1/\tau _DRe^{1/2}/\tau _L`$. Thus fully developed turbulence is characterized by a Lyapunov exponent diverging with $`Re`$.

Nevertheless a large value of the LE does not prevent the possibility of long term prediction, at least if one is interested in predicting the large scales behavior (which is related to finite errors), see Sect. 5.4.

Remaining in the framework of infinitesimal perturbations, we discuss the effects of intermittency on the predictability time. The multifractal model  predicts a spectrum of viscous cut-offs: each singularity exponent $`h`$ selects a different damping scale, $`\mathrm{}_D(h)LRe^{1/(1+h)}`$, and hence a spectrum of (dissipative) turn-over times, $`\tau _D(h)`$, such that (5.6) becomes

$$\tau _D(h)\mathrm{}_D(h)/\delta v_D\tau _LRe^{\frac{1h}{1+h}},$$
(5.7)
(see Appendix B for details). To obtain the largest Lyapunov exponent now we have to integrate $`\tau _D(h)^1`$, at the scale $`\mathrm{}=\mathrm{}_D(h)`$, over the $`h`$-distribution $`P_{\mathrm{}}(h)\mathrm{}^{3D(h)}`$:

$$\lambda \tau (h)^1P_{\mathrm{}}(h)𝑑h\frac{1}{\tau _L}\left(\frac{\mathrm{}_D}{L}\right)^{hD(h)+2}𝑑h.$$
(5.8)
Since the viscous cut-off vanishes in the limit $`Re\mathrm{}`$, the integral can be estimated by the saddle-point method, i.e.

$$\lambda \frac{1}{\tau _L}Re^\alpha \text{with}\alpha =\underset{h}{\mathrm{max}}\left[\frac{D(h)2h}{1+h}\right].$$
(5.9)
The value of $`\alpha `$ depends on the shape of $`D(h)`$. By using the function $`D(h)`$ obtained by fitting the exponents $`\zeta _q`$ with the random $`\beta `$-model  one finds $`\alpha =0.459..`$, slightly smaller than the Ruelle prediction $`\alpha =0.5`$. This result is confirmed by numerical simulations on the shell model (5.5) (see Fig. 16).

We remind that the fluctuations of the effective Lyapunov exponent $`\gamma (t)`$ can be characterized by the ratio of $`\mu /\lambda `$ (Sect. 3.1). The variance $`\mu `$ is

$$\mu =\underset{t\mathrm{}}{lim}t\left[\gamma (t)^2\gamma (t)^2\right]t_c(\gamma \lambda )^2$$
(5.10)
where in the last expression we have introduced the integral correlation time $`t_c=C(t)𝑑t`$ of the effective Lyapunov exponent , where $`C(t)`$ is the normalized correlation function of the fluctuation of $`\gamma (t)`$ (i.e. $`\gamma (t)\lambda `$).

The quantity $`(\gamma \lambda )^2`$ can be computed by repeating the argument for $`\lambda `$:

$$\gamma ^2\tau ^2\frac{1}{\tau _L^2}Re^y.$$
(5.11)
An explicit calculation  gives $`y=1`$ independently of intermittency. Assuming that the correlation time $`t_c`$ vanishes as a power of $`Re`$

$$t_c\tau _LRe^z$$
(5.12)
one ends with the prediction

$$\mu \frac{1}{\tau _L}Re^w\text{with}w=1z.$$
(5.13)
Numerical simulations on the shell model (5.5) give $`w0.8`$ (see Figure 16). Because $`w>\alpha `$ we obtain that $`\mu /\lambda `$ diverges with $`Re`$. From Figure 16 we see that the strong intermittency regime begins, for the shell model, at $`Re10^6`$. Let us stress that in the absence of intermittency one would expect that $`t_c\lambda ^1`$ and thus $`z=1/2`$ and $`\mu /\lambda `$ constant. The fact that $`z0.2`$ indicates that the presence of quiescent periods in the turbulent activity is much more relevant for the decay rate of time correlations than for the Lyapunov exponent.

We have seen in Sect. 3.3 that the fluctuations of the effective LE affect the distribution of predictability time, and thus we expect a similar effect in fully developed turbulence. In the shell model one can estimate the predictability time by computing the time $`T_p`$ at which the difference $`\delta u_m(t)`$ (where $`m`$ corresponds to the integral scale) among two realizations of the model becomes larger that the tolerance $`\mathrm{\Delta }`$. The initial difference $`\delta _0`$ is restricted to the shell $`u_n^{}`$ on the Kolmogorov scale and $`mn^{}`$. The predictability time distribution function is computed at two different Reynolds number. At $`Re=10^6`$ we are at the border of the weak intermittent range: the observed PDF (Figure 17) is indeed close to a Gaussian with mean value

$$T_p\frac{1}{\lambda }\mathrm{ln}\left(\frac{\mathrm{\Delta }}{\delta _0}\right).$$
(5.14)
On the contrary, at $`Re=2\times 10^9`$, the PDF exhibits the asymmetric triangular shape and the mean value is ruled by $`\mu `$ according to (3.27).

### 5.4 Growth of non-infinitesimal perturbations

The classical theory of predictability in turbulence has been developed by Lorenz  (see also ) using physical arguments, and by Leith and Kraichnan  on the basis of closure approximations. The fundamental ingredients of the Lorenz approach stem from dimensional arguments on the time evolution of a perturbation in an energy cascade picture. In this framework, it is rather natural to assume that the time $`\tau _{\mathrm{}}`$ for a perturbation at scale $`\mathrm{}/2`$ to induce a complete uncertainty on the velocity field on the scale $`\mathrm{}`$, is proportional to the typical eddy turn-over time at scale $`\mathrm{}`$: $`\tau _{\mathrm{}}\mathrm{}/\delta v_{\mathrm{}}`$ where $`\delta v_{\mathrm{}}`$ is the typical velocity difference at scale $`\mathrm{}`$. Kolmogorov scaling (5.1) gives

$$\tau _{\mathrm{}}\mathrm{}^{2/3}.$$
(5.15)
Because of the geometric progression (5.15), the predictability time to propagate an uncertainty $`O(\delta v_D)`$ from the Kolmogorov scale $`\mathrm{}_D`$ up to the scale of the energy containing eddies $`L`$, is dominated by the longest time

$$T_p\tau _\mathrm{}_d+\tau _{2\mathrm{}_d}+\mathrm{}+\tau _L\tau _L\frac{L}{\delta v_L}.$$
(5.16)
Closure approximations, where one still uses dimensional arguments, confirm this result .

It is important to stress that, in the Lorenz approach, the predictability time is independent of the Reynolds number. This is only in apparent contradiction with the increase of the Lyapunov exponent with $`Re`$ (5.9). From the point of view of an observer interested in forecasting the large scales (i.e. not infinitesimal perturbations) the Lyapunov exponent is not physically relevant. Large scale predictability in turbulence is hence another example where a large LE coexists with a long predictability time. We will see that a coherent description that includes these two features of predictability is given by the finite size Lyapunov exponent (3.35).

It is easy to estimate the scaling behavior of $`\lambda (\delta )`$ when the perturbation is in the inertial range $`\delta v_D\delta \delta v_L`$. Following the phenomenological ideas of Lorenz, the doubling time of an error of magnitude $`\delta `$ can be identified with the turn-over time $`\tau _{\mathrm{}}`$ of an eddy with typical velocity difference $`\delta v_{\mathrm{}}\delta `$. Using the scaling (5.1) one has $`\tau _{\mathrm{}}\tau _L(\mathrm{}/L)^{2/3}\tau _L(\delta v_{\mathrm{}}/\delta v_L)^2`$. In conclusion we obtain

$$\lambda (\delta )\delta ^2.$$
(5.17)
In the dissipative range $`\delta <\delta v_D`$, the error can be considered infinitesimal, implying $`\lambda (\delta )=\lambda `$.

Accounting for intermittency, in the framework of the multifractal approach, one has

$$\lambda (\delta )\tau _L^1𝑑h(\delta /\delta v_L)^{[3D(h)]/h}(\delta /\delta v_L)^{11/h}.$$
(5.18)
From the basic inequality of the multifractal model $`D(h)3h+2`$ (see Appendix B), we have

$$\frac{2+hD(h)}{h}2\text{for all}h.$$
(5.19)
As a result of the constancy of the energy flux in the inertial range, $`\overline{\epsilon }=v^3(\mathrm{})/\mathrm{}`$, the equality holds for $`h=h^{}(3)`$, and gives $`3h^{}(3)+3D(h^{}(3))=1`$. Therefore a saddle point estimation of (5.18) gives again (5.17). The dimensional scaling of the FSLE in fully developed turbulence $`\lambda (\delta )\delta ^2`$ is thus not affected by intermittency corrections. This is a direct consequence of the exact result (5.2)

These findings have been numerically tested on the shell model (5.5) for the energy cascade. Figure 18 shows the scaling of $`1/\tau (\delta v,r)_t`$ as a function of $`\delta v`$ in the GOY model, where $`\tau (\delta v,r)`$ is the “doubling time”, i.e. the time necessary for a perturbation of size $`\delta v`$ to increase by a factor $`r`$ (see Sect. 3.4 and Appendix A).

For comparison we also plot the eddy turn-over times $`\tau _{\mathrm{}}^1=|\delta v_{\mathrm{}}|^2^{1/2}/\mathrm{}`$. We see that below the Kolmogorov scale, the doubling time displays a constant plateau corresponding to the Lyapunov exponent (3.37). At larger errors we observe a good agreement with the prediction (5.17). Let us observe that, even at this high Reynolds number, the scaling range for the doubling time is rather small.

It is interesting to look at the doubling time as a function of the Reynolds number. For small thresholds the inverse of the doubling time scales as the Lyapunov exponent, i.e. roughly as $`Re^{1/2}`$. We also observe that the bend away from the infinitesimal growth rate occurs at smaller scales for larger Reynolds numbers. This suggests the following scaling ansatz: times and errors are scaled with the turn-over time and the typical scale of fluctuations at the Kolmogorov scale, that is by $`\text{Re}^{1/2}`$ and $`\text{Re}^{1/4}`$, respectively. In Figure 19 we show the re-scaled data. The data collapse is reasonable, allowing to conclude that small-scale predictability, with small error amplitudes, behaves (apart from intermittency corrections) as predicted by Ruelle , whereas large-scale predictability, characterized by large error amplitudes, is well described by Lorenz arguments.

To improve the data collapse, taking into account the multifractal correction as described in Appendix B, one has to make a multiscaling collapse, i.e. to rescale $`\mathrm{ln}1/\tau (\delta v,r)`$ and $`\mathrm{ln}(\delta v/V_0)`$ with $`\mathrm{ln}(Re/Re_0)`$ where $`V_0`$ and $`Re_0`$ are two parameters to be fixed . The result is shown in Figure 20. The data collapse is clearly improved.

Finite size predictability has been investigated also in two-dimensional turbulence, which is relevant for atmospheric flows. As discussed in Sect. 4.8, two-dimensional turbulence in the inverse energy cascade regime is characterized by a scaling à la Kolmogorov  with no intermittency . As discussed above, the scaling exponent in (5.17) is not affected by intermittency; however intermittency does reduce the scaling range because of the intermediate dissipative range (see Appendix B). The absence of intermittency corrections in $`2D`$ turbulence suggests that the dimensional scaling (5.17) is observable even in direct numerical simulations at moderate Reynolds number.

Let us consider two realizations of the vorticity field in (4.27) starting from very close initial conditions. The error $`\delta `$ is defined, following (4.30), as $`\delta (t)=\sqrt{E_\delta (t)}`$. In Figure 21 it is shown the FSLE $`\lambda (\delta )`$. It is remarkable the rather wide scaling range for $`\lambda (\delta )\delta ^2`$ with respect to the shell model simulations (Fig. 18) obtained at much larger Re. As a consequence of the absence of intermittency, also the crossover from the infinitesimal regime $`\lambda (\delta )=\lambda `$ to the inertial range regime (5.17) is sharp.

From a general point of view, it is interesting to observe that even in the absence of intermittency, fixed scale analysis based on the FSLE overpasses fixed time analysis in the characterization of predictability. Dimensional considerations and closure approximations  predicts a linear growth of the error in the inverse energy cascade as

$$E_\delta (t)=G\overline{\epsilon }t,$$
(5.20)
where $`G`$ is an adimensional constant. It is easy to realize that (5.20) is equivalent to (5.17), $`\lambda (\delta )`$ having the dimension of an inverse time and $`\delta =\sqrt{E_\delta }`$. The result obtained in numerical simulations is shown in Figure 22, which has to be compared with Figure 21. The scaling law (5.20) in Figure 22 is barely visible, making the determination of $`G`$ difficult. On the contrary, inverting (5.17) to (5.20) one can measure $`G`$ directly from Figure 21. The result obtained is in close agreement with closure computations .

### 5.5 $`ϵ`$-entropy for turbulent flows

A complementary way to look at the predictability of turbulent flows is in terms of its entropy (see Sects. 2.1.2 and 3.5).

Unfortunately a direct measurement of the Kolmogorov-Sinai entropy is practically infeasible. Indeed for $`Re\mathrm{}`$ due to the huge number of active degrees of freedom, the KS-entropy diverges, so that one needs velocity measurements with an extremely high resolution and lasting for extremely long times, far beyond the actual experimental possibilities. Nevertheless, limiting the analysis to not very high resolution, one can hope to extract some interesting piece of information by investigating the behavior of the $`ϵ`$-entropy, $`h(ϵ)`$. As far as the $`ϵ`$-entropy of turbulence is concerned, two questions can be raised.

$`i)`$ Since a direct measurement of the full $`3`$-dimensional velocity field is infeasible, one has usually access just to a time signal measured in one spatial point: which kind of information can we extract from the $`ϵ`$-entropy per unit time of such a signal?

$`ii)`$ Taking into account $`i)`$, can we say something about the $`ϵ`$-entropy of the full $`3`$-dimensional velocity field?

In $`ii)`$ we are referring to the $`ϵ`$-entropy, $`h^{ST}(ϵ)`$, per unit time and volume (the symbol ST means space-time). In other words, we are assuming that the total entropy of a turbulent flow observed for a (very long) time $`T`$ on a (very large) volume $`V`$ of the $`3`$-dimensional space has the form $`H(V,T,ϵ)VTh^{ST}(ϵ)`$. See Ref.  for an introduction of this concept.

Both in $`i)`$ and $`ii)`$, as we will see, a crucial role is played by the sweeping of the large scales of the flow on the small ones, i.e. the Taylor hypothesis (see Sect. 5.1).

#### 5.5.1 $`ϵ`$-entropy for a time signal of turbulence

In order to estimate the $`ϵ`$-entropy of a given signal one has to compute the Shannon entropy of the symbolic sequence obtained by making an $`(ϵ,\tau )`$ grid in phase-space (Sect. 3.5). Unfortunately, this method is rather inefficient for signals in which many scales are excited , e.g., as in turbulence. Therefore, here we resort to a recently proposed method  based on the exit-time analysis.

In a few words, the idea consists in looking at a sequence of data not at fixed sampling times but at fixed fluctuation (see Appendix C), i.e. when the fluctuation of the signal exceeds a given threshold, $`ϵ`$. In practice we code the signal $`v(t)`$ of total duration $`T`$ in a symbolic sequence $`\mathrm{\Omega }^M(ϵ)=\{t_i(ϵ),k_i(ϵ)\}_{i=1}^M`$, where $`t_i(ϵ)`$ is the first times such that $`|v(t_0+_{k=1}^{i1}t_k(ϵ)+t_i(ϵ))v(t_0+_{k=1}^{i1}t_k(ϵ))|ϵ/2`$ (being $`t_0`$ a reference time) and $`k_i=\pm 1`$ tells us in which direction (up or down with respect to $`v(t_0+_{k=1}^{i1}t_k(ϵ))`$) the fluctuation has been realized. $`M`$ is the total number of exit events, i.e. $`_{i=1}^Mt_i(ϵ)=T`$. Note that $`\mathrm{\Omega }^M(ϵ)`$ is a faithful coding of the signal within the required accuracy $`ϵ`$. Now the evaluation of the entropy goes as usual through the evaluation of the Shannon entropy, $`h^\mathrm{\Omega }(ϵ)`$, of the sequence $`\mathrm{\Omega }^M(ϵ)`$. Finally the $`ϵ`$-entropy per unit time is given by :

$$h(ϵ)\frac{h^\mathrm{\Omega }(ϵ,\tau _r)}{t(ϵ)},$$
(5.21)
where a coarse-graining of the possible values assumed by $`t(ϵ)`$ with a resolution time $`\tau _r`$ has been considered, and $`t(ϵ)`$ is the average exit time, i.e. $`t(ϵ)=(1/M)_{i=1,M}t_i(ϵ)`$. The formula (5.22) is exact in the limit $`\tau _r0`$ (in Appendix C one finds the derivation of (5.21) and the details of the method).

This procedure allows a noticeable improvement of the computational possibility to measure the $`ϵ`$-entropy. In particular, if one is interested in the leading scaling behavior of $`h(ϵ)`$ with $`ϵ`$, one only needs to estimate the scaling of $`t(ϵ)`$. Indeed, the correction induced by $`h^\mathrm{\Omega }(ϵ,\tau _r)`$ can be shown to be sub-leading (in particular, logarithmic).

Now, we estimate the average exit time for the velocity signal $`v(t)`$. This can be done assuming the Taylor hypothesis and the multifractal model (see Appendix B). In this framework we can assume that, for $`t`$ corresponding to scales $`R=Ut`$ in the inertial range, the following relation holds $`|\delta _tv|=|v(t_0+t)v(t)|t^h`$ and each $`h`$ is picked with probability $`P(h)t^{3D(h)}`$. Since we are interested in the statistics of the first times necessary to observe a fluctuation $`|\delta _tv|ϵ`$, one can “invert” the above relation :

$$t(ϵ)ϵ^{1/h}\text{with}P(h)ϵ^{(3D(h))/h}.$$
(5.22)
The exit-time moments , also called inverse structure functions , can be estimated in the multifractal framework as follows

$$t^q(ϵ)dhϵ^{\frac{q+3D(h)}{h}}ϵ^{\chi (q)}$$
(5.23)
where $`\chi (q)`$ may be obtained with a saddle point estimate in the limit of small $`ϵ`$:

$$\chi (q)=\underset{h}{\mathrm{min}}\left[\frac{q+3D(h)}{h}\right].$$
(5.24)
The average $`[\mathrm{}]`$, obtained by counting the number of exit-time events $`M`$, and the average $`[\mathrm{}]`$, computed with the uniform time sampling are connected by the relation

$$t^q(ϵ)=\underset{M\mathrm{}}{lim}\underset{i=1}{\overset{M}{}}t_i^q\frac{t_i}{_{j=1}^Mt_j}=\frac{t^{q+1}(ϵ)}{t(ϵ)},$$
(5.25)
where the term $`t_i/_{j=1}^Mt_j`$ takes into account the non-uniformity of the exit-time statistics. Therefore the quantity we are looking for, i.e. the mean exit-time, is given by $`t(ϵ)=t^1(ϵ)^1(ϵ)^{\chi (1)}`$. By noting that

$$\frac{1+3D(h)}{h}=\frac{2D(h)}{h}3\text{for all}h,$$
(5.26)
which is nothing but Eq. (5.19), i.e. the $`4/5`$ law of turbulence, we finally obtain

$$h(ϵ)ϵ^3.$$
(5.27)
In Fig. (23) we report the evaluation of the upper and lower bound (see Appendix C) of $`h(ϵ)`$ for a synthetic signal, $`v(t)`$, constructed in such a way to reproduce the statistical properties of turbulence .

Let us now compare the above results with a previous study of the $`ϵ`$-entropy in turbulence , where it was argued that:

$$h(ϵ)ϵ^2,$$
(5.28)
a behavior that differs from the prediction (5.27). The behavior (5.28) has been obtained by assuming that $`h(ϵ)`$, at scale $`ϵ`$, is proportional to the inverse of the typical eddy turnover time at that scale: since the typical eddy turnover time for velocity fluctuations of order $`\delta vϵ`$ is $`\tau (ϵ)ϵ^2`$, Eq. (5.28) follows. Indeed this is the argument used to derive (5.17) for the FSLE. The difference between (5.28) and (5.27) can be understood by considering that even if $`\lambda (\delta )`$ and $`h(ϵ)`$ are two complementary concepts (the fact that for both the estimate of the scaling behavior reduces to the “$`4/5`$ law” is not a coincidence), in the latter case one has to consider the sweeping induced by the large scales. On the contrary, since the former is related to the distance of two realizations which differ in the small scales ($`<\delta `$) but not on the large scales ($`>\delta `$), the sweeping of the large scales is not effective.

#### 5.5.2 $`ϵ`$-entropy of turbulence and the Taylor Hypothesis

Now we study the $`ϵ`$-entropy per unit time and volume for the velocity field of turbulent flows in $`3+1`$ dimensions, $`h^{ST}(ϵ)`$. We will show that, by assuming the usually accepted Taylor hypothesis, one has a spatial correlation which can be quantitatively characterized by an “entropy” dimension $`𝒟=8/3`$. As already remarked, $`h^{ST}(ϵ)`$ cannot be directly measured so we will discuss its estimation in a theoretical framework by introducing a multi-affine field. For the sake of simplicity, we neglect intermittency by assuming a pure self-affine field with a unique Hölder exponent $`h=1/3`$.

Let us first introduce a multi-affine field with the proper spatial and temporal scaling . The idea consists in defining the signal as a dyadic three-dimensional superposition of wavelet-like functions $`\phi ((𝐱𝐱_{n,𝐤}(t))/\mathrm{}_n)`$ whose centers move due to the sweeping. The coefficients of the decomposition $`a_{n,𝐤}(t)`$ are stochastic functions chosen with suitable self-affine scaling properties both in time and in space. A field with spatial Hölder exponent $`h`$ in $`d`$-dimensions is (see Appendix D):

$$v(𝐱,t)=\underset{n=1}{\overset{M}{}}\underset{k=1}{\overset{2^{d(n1)}}{}}a_{n,k}(t)\phi \left(\frac{𝐱𝐱_{n,k}(t)}{\mathrm{}_n}\right),$$
(5.29)
where $`𝐱_{n,k}`$ is the center of the $`k^{th}`$ wavelets at the level $`n`$, i.e. for eddies with size $`\mathrm{}_n2^n`$. According to the Richardson-Kolmogorov cascade picture, one assumes that sweeping is present, i.e., $`𝐱_{n+1,k}=𝐱_{n,k^{}}+𝐫_{n+1,k}`$ where $`(n,k^{})`$ labels the “mother” of the $`(n+1,k)`$-eddy and $`𝐫_{n+1,k}`$ is a stochastic vector which depends on $`𝐫_{n,k^{}}`$ and evolves with characteristic time $`\tau _n(\mathrm{}_n)^{1h}`$.

If the coefficients $`\{a_{n,k}\}`$ and $`\{𝐫_{n,k}\}`$ have characteristic time $`\tau _n(\mathrm{}_n)^{1h}`$ and $`\{a_{n,k}\}(\mathrm{}_n)^h`$, it is possible to show (see Appendix D for details) that the field (5.29) has the correct spatio-temporal statistics, i.e.

$`|v(𝐱+𝐑,t_0)v(𝐱,t_0)|`$ $``$ $`|𝐑|^h`$ (5.30)
$`|v(𝐱,t_0+t)v(𝐱,t_0)|`$ $``$ $`t^h.`$ (5.31)
In addition the proper Lagrangian sweeping is satisfied. Now we are ready for the $`ϵ`$-entropy analysis of the field (5.29). If one wants to look at the field $`v`$ with a resolution $`ϵ`$, one has to take $`n`$ in (5.29) up to $`N`$ given by:

$$(\mathrm{}_N)^hϵ,$$
(5.32)
in this way one is sure to consider velocity fluctuations of order $`ϵ`$. Then the number of terms contributing to (5.29) is

$$\mathrm{\#}(ϵ)(2^d)^Nϵ^{d/h}.$$
(5.33)
By using a result of Shannon  one estimates the $`ϵ`$-entropy of the single process $`a_{n,k}(t)`$ (and also of $`𝐫_{n,j}`$) as:

$$h_n(ϵ)\frac{1}{\tau _n}\mathrm{ln}\left(\frac{1}{ϵ}\right),$$
(5.34)
where the above relation is rigorous if the processes $`a_{n,k}(t)`$ are Gaussian and with a power spectrum different from zero on a band of frequency $`1/\tau _n`$. The terms which give the main contribution are those with $`nN`$ with $`\tau _N(\mathrm{}_N)^{1h}ϵ^{(\frac{1h}{h})}`$. Their number is given by (5.33) so that, collecting the above results, one finds

$$h^{ST}(ϵ)\frac{\mathrm{\#}(ϵ)}{\tau _N}ϵ^{\frac{dh+1}{h}}.$$
(5.35)
For the physical case $`d=3`$, $`h=1/3`$, one obtains

$$h^{ST}(ϵ)ϵ^{11}.$$
(5.36)
By denoting with $`v_\eta `$ the typical velocity at the Kolmogorov scale $`\eta `$, one has that Eq. (5.36) holds in the inertial range, i.e., $`ϵv_\eta Re^{1/4}`$, while for $`ϵv_\eta `$, $`h^{ST}(ϵ)=`$ constant $`Re^{11/4}`$.

Let us now consider an alternative way to compute the $`ϵ`$-entropy of the field $`v(𝐱,t)`$: divide the $`d`$-volume in boxes of edge length $`\mathrm{}(ϵ)ϵ^{1/h}`$ and look at the signals $`v(𝐱_\alpha ,t)`$, where the $`𝐱_\alpha `$ are the centers of the boxes. Denoting with $`h^{(\alpha )}(ϵ)`$ the $`ϵ`$-entropy of the temporal sequence of the velocity field measured in $`𝐱_\alpha `$, we have

$$h^{(\alpha )}(ϵ)ϵ^{1/h}$$
(5.37)
because of the scaling (5.31). Therefore, $`h^{ST}(ϵ)`$ is obtained summing up all the “independent” contributions (5.37), i.e.

$$h^{ST}(ϵ)𝒩(ϵ)h^{(\alpha )}(ϵ)𝒩(ϵ)ϵ^{1/h},$$
(5.38)
where $`𝒩(ϵ)`$ is the number of independent cells. It is easy to understand that the simplest assumption $`𝒩(ϵ)l(ϵ)^dϵ^{d/h}`$ gives a wrong result, indeed one obtains

$$h^{ST}(ϵ)ϵ^{\frac{d+1}{h}},$$
(5.39)
which is not in agreement with (5.35). In order to obtain the correct result (5.36) it is necessary to assume

$$𝒩(ϵ)l(ϵ)^𝒟,$$
(5.40)
with $`𝒟=dh`$. In other words, one has that the sweeping implies a nontrivial spatial correlation, quantitatively measured by the exponent $`𝒟`$, which can be considered as a sort of “entropy” dimension. Incidentally, we note that $`𝒟`$ has the same numerical value of the fractal dimensions of the velocity iso-surfaces . From this observation, at first glance, one could conclude that the above result is somehow trivial since it is simply related to a geometrical fact. However, a closer inspection reveals that this is not true. Indeed, one can construct a self-affine field with spatial scaling $`h`$ and thus with the fractal dimension of the velocity iso-surfaces given by $`dh`$ for geometrical reasons, while $`𝒟=d`$. Such a process can be simply obtained by eliminating the sweeping, i.e.,

$$v(𝐱,t)=\underset{n=1}{\overset{M}{}}\underset{k=1}{\overset{2^{d(n1)}}{}}a_{n,k}(t)\phi \left(\frac{𝐱𝐱_{n,k}}{\mathrm{}_n}\right),$$
(5.41)
where now the $`𝐱_{n,k}`$ are fixed and no longer time-dependent, while $`a_{n,k}(\mathrm{}_n)^h`$ but $`\tau _n\mathrm{}_n`$.

We conclude by noting that it is possible to obtain (see ) the scaling (5.35) using equation (5.41), i.e. ignoring the sweeping, assuming $`\tau _n(\mathrm{}_n)^{1h}`$ and $`a_{n,k}(\mathrm{}_n)^h`$; this corresponds to take separately the proper temporal and spatial spectra. However, this is not satisfactory since one has not the proper scaling in one fixed point (see Eq. (5.37) the only way to obtain this is through the sweeping).

## 6 Uncertainty in the evolution equations

The study of a large class of problems in science (physics, chemistry, biology,…) is reduced to the investigation of evolution laws, which describe some aspects of the system. The assumption that natural processes can be described by mathematical models is at the foundation of this approach . The purpose of this Section is to discuss how the unavoidable uncertainty in the equation of motion puts limits on the long time forecasting.

To be more concrete, let us consider a system described by a differential equation:

$$\frac{d}{dt}𝐱(t)=𝐟(𝐱,t),𝐱,𝐟\mathrm{I}R^n.$$
(6.1)
As a matter of fact, we do not know exactly the equations, so we have to devise a model which is different from the true dynamics. In practice this means that we study

$$\frac{d}{dt}𝐱(t)=𝐟_ϵ(𝐱,t)\text{where}𝐟_ϵ(𝐱,t)=𝐟(𝐱,t)+ϵ\delta 𝐟(𝐱,t).$$
(6.2)
Therefore, it is natural to wonder about the relation between the true evolution (reference or true trajectory $`𝐱_T(t)`$) given by (6.1) and the one effectively computed (perturbed or model trajectory $`𝐱_M(t)`$) given by (6.2). A typical example is the relation between the true dynamics of the physical system and the one obtained by a computer simulation. This issue is of particular relevance for the study of weather forecast where it is referred to as predictability of the second kind .

In this context it is particularly relevant the shadowing lemma  which implies that, for Anosov systems, a computer may not calculate the true orbit but what it does find is nevertheless an approximation of the true one. As a consequence, the statistical properties are well reproduced by an accurate numerical integration .

A central point in the discussion of the second kind predictability problem is the issue of structural stability : since the evolution laws are known only with finite precision it is highly desirable that at least certain properties were not too sensitive to the details of the equations of motion. For example, in a system with a strange attractor, small generic changes in the evolution laws should not change drastically the statistical properties of the dynamics .

In order to see that a non generic perturbation, although very “small” in some sense, can produce dramatic changes in the statistical properties of the dynamics, following Refs. , we consider the one-dimensional chaotic map $`x(t+1)=f(x(t))`$ with $`f(x)=4x`$ mod $`1`$, and a perturbed version of it:

$$f_ϵ(x)=\{\begin{array}{cc}8x\frac{9}{2}\hfill & x[\frac{5}{8},\frac{247}{384}]\hfill \\ & \\ \frac{1}{2}x+\frac{1}{3}\hfill & x[\frac{247}{384},\frac{265}{384}]\hfill \\ & \\ 8x\frac{29}{6}\hfill & x[\frac{265}{384},\frac{17}{24}]\hfill \\ & \\ 4x\text{mod}\mathrm{\hspace{0.33em}1}\hfill & \text{otherwise}.\hfill \end{array}$$
(6.3)
The perturbed map is identical to the original outside the interval $`[5/8,17/24]`$, and the perturbation is small in $`L_2`$ norm. Nevertheless, the fixed point $`x=2/3`$, which is unstable in the original dynamics, becomes stable in the perturbed one, and it is a global attractor for $`f_ϵ(x)`$, i.e. almost every point in $`[0,1]`$ asymptotically approaches $`x=2/3`$.

If one compares the trajectories obtained iterating $`f(x)`$ or $`f_ϵ(x)`$ it is not difficult to understand that they may remain identical for a certain time but unavoidably differ utterly in the long time behavior. The transient chaotic behavior of the perturbed orbits can be rendered arbitrarily long by reducing the interval in which the two dynamics differ .

As for the problem of predictability with respect to perturbations on the initial conditions, the problem of second kind predictability in the limit of infinitesimal perturbations is essentially understood in terms of the Lyapunov exponents. Indeed, it is possible to show (see below) that a small uncertainty on the evolution laws of chaotic systems has the same effects of an error of the same order of magnitude on the initial conditions. However, also in the case of second kind predictability one has often to deal with errors which are far from being infinitesimal. Moreover, in real systems the size of an uncertainty on the evolution equations is determinable only a posteriori, based on the ability of the model to reproduce some of the features of the phenomenon. Typical examples are systems described by partial differential equations (e.g. turbulence, atmospheric flows). The numerical study of these systems is performed by using a model with unavoidable severe approximations, the most relevant due to the necessity to cut some degrees of freedom off (i.e. the small scale variables). A relevant problem in this case is to quantify the effect of the unresolved scales on the predictability of the resolved ones.

From a general point of view, in the second kind predictability problem we can distinguish three main cases depending on the original dynamics. In particular, Eq. (6.1) may display:

(i) trivial attractors: asymptotically stable fixed points or attracting periodic orbits;

(ii) marginally stable fixed points or periodic/quasi-periodic orbits as in integrable Hamiltonian systems;

(iii) chaotic behavior.

In case (i) small changes in the equations of motion do not modify the qualitative features of the dynamics. Case (ii) is not generic and the outcome strongly depends on the specific perturbation $`\delta 𝐟`$, i.e. it is not structurally stable (see  for a discussion on this point). In the chaotic case (iii) one expects that the perturbed dynamics is still chaotic. In the following we will consider only this latter case.

In chaotic systems, the effects of a small uncertainty on the evolution law is, for many aspects, similar to those due to imperfect knowledge of initial conditions. As an example let us consider the Lorenz system (3.28). In order to mimic an indetermination in the evolution law we assume a small error $`ϵ`$ on the parameter $`r`$: $`rr+ϵ`$. Let us consider the difference $`\delta 𝐱(t)=𝐱_M(t)𝐱_T(t)`$, for simplicity, $`\delta 𝐱(0)=0`$, i.e. we assume a perfect knowledge of the initial condition. For small $`ϵ`$ one has, with obvious notation:

$$\frac{d\delta 𝐱}{dt}=𝐟_ϵ(𝐱_M)𝐟(𝐱_T)\frac{𝐟}{𝐱}\delta 𝐱+\frac{𝐟_ϵ}{r}ϵ.$$
(6.4)
Since at time $`t=0`$ one has $`|\delta 𝐱(0)|=0`$, $`|\delta 𝐱(t)|`$ initially grows under the effect of the second term in (6.4). At later times, when $`|\delta 𝐱(t)|O(ϵ)`$ the first term becomes the leading one, and we recover the first kind predictability for an initial uncertainty $`\delta _0ϵ`$. Therefore, apart from an initial growth, which depends on the specific perturbation, for small enough $`ϵ`$ the evolution of $`\mathrm{ln}(|\delta 𝐱(t)|)`$ follows the usual linear growth with the slope given by the largest LE. Typically the value of the LE computed by using the model dynamics differs from the true one by a small amount of order $`ϵ`$, i.e. $`\lambda _M=\lambda _T+O(ϵ)`$ .

A picture of the error growth, valid also for finite errors, can be obtained by considering the Finite Size Lyapunov Exponent. In addition to the FSLE of the model, $`\lambda _{MM}(\delta )`$, we introduce the FSLE for the true dynamics (6.1) $`\lambda _{TT}(\delta )`$ (which cannot be measured in real situations) and $`\lambda _{TM}(\delta )`$, the FSLE computed following the distance between one true trajectory and one model trajectory starting at the same point. In the case of a perfect model $`\lambda _{MM}(\delta )=\lambda _{TT}(\delta )`$. The results of the computation for the Lorenz model (3.28) are shown in Figure 25. $`\lambda _{TT}(\delta )`$ displays the chaotic plateau with $`\lambda 1.2`$. As discussed above, for $`\delta >ϵ`$ the second term in (6.4) becomes negligible and we observe $`\lambda _{TM}(\delta )\lambda _{TT}(\delta )\lambda `$. In this range of errors the model system recovers the intrinsic predictability of the true system. For very small errors, $`\lambda _{TM}`$ is dominated by the second term in (6.4) and deviates from $`\lambda _{TT}`$.

### 6.1 Uncertainty introduced by numerical computations

In numerical computations, an unavoidable source of errors is due to the representation of numbers on the computer, as computers work with integers. This has two main consequences: the phase space of the simulated system is necessarily discrete (and finite); and the computation introduces a sort of noise due to the round-off.

A direct consequence of the discreteness in phase space is that any numerical trajectory is periodic. At first sight, this seems a very serious problem, especially when integrating chaotic systems which have non periodic behavior. However, as discussed in , apart from cases in which one uses very low precision, and very low dimensional systems, the period is usually extremely large and one works with an effective continuous phase space dynamical system (see Sect. 7.1).

The round-off produces on (6.1)-(6.2) a perturbation $`\delta 𝐟(𝐱,t)`$ of order $`ϵ10^\alpha `$ ($`\alpha =`$number of digits in floating point representation) which depends on $`𝐟`$ and on the software . In general, the round-off error is very small and may have a positive role in selecting the physical probability measure, the so-called natural measure, from the set of the admissible invariant ones .

In order to show the effect of the numerical precision on the predictability, let us consider again the Lorenz model (3.28). At variance with the previous Section, here we assume to have a perfect knowledge of the model (i.e. of the parameter $`r`$), and the error is introduced only by the numerical integration, e.g. by different time step $`\mathrm{\Delta }t`$. The most precise integration with smallest $`\mathrm{\Delta }t`$ is taken as the reference trajectory and the other is the perturbed one. The result is shown in Figure 26 for two different values of $`\mathrm{\Delta }t`$ for the perturbed integration. In both cases, for small values of the error, the exponential growth rate is given by the largest LE $`\lambda `$. The same behavior is observed by introducing the numerical error in other ways, e.g. by using different precision (single or double) or different integration algorithms .

### 6.2 Finite resolution effects and parameterization of unresolved scales

Let us now consider more complex situations, in which many interacting degrees of freedom and different characteristic times are involved . We will consider the particular examples of an extremely simplified model of global circulation  and the shell model (Sect. 5.2).

For systems with many different scales usually one is able to represent only the large scale variables. A typical situation is the discretization of partial differential equations. The small scale modes, below the computational grid, are unresolved and are typically parameterized according to some phenomenological prescription (e.g. the eddy viscosity parameterization of the small scales ). So we consider systems of the following form

$$\frac{d𝐱}{dt}=𝐟(𝐱,𝐲)\frac{d𝐲}{dt}=𝐠(𝐱,𝐲)$$
(6.5)
where $`𝐱\mathrm{I}R^n`$ represent the large (and typically slow) variables while $`𝐲\mathrm{I}R^m`$ represent the small (and fast) ones. As explained above, in many practical situations the small variables cannot be explicitly resolved. In this framework, a natural question is: how must we parameterize the unresolved modes in order to predict the resolved ones? In this respect, the optimal parameterization is that one for which the predictability on the resolved modes is not worse than the intrinsic predictability of the same variables in the complete system, i.e. in our notation $`\lambda _{TM}=\lambda _{TT}`$.

An example in which it is relatively simple to develop a model for the small scale modes is represented by skew systems, i.e., $`𝐠`$ depends only on the fast variables $`𝐲`$. In this case, simply neglecting the fast variables or parameterizing them with a suitable stochastic process does not drastically affect the prediction of the slow variables .

On the other hand, in typical cases $`𝐲`$ feels some feedback from $`𝐱`$, and, therefore, we cannot simply neglect the unresolved modes (see Ref.  for details). In practice one has to construct an effective equation for the resolved variables:

$$\frac{d𝐱}{dt}=𝐟_M(𝐱,𝐲(𝐱)),$$
(6.6)
where the functional form of $`𝐲(𝐱)`$ and $`𝐟_M`$ is built by phenomenological arguments and/or by numerical studies of the full dynamics (if available).

Let us now discuss a simplified model for atmosphere circulation  which includes large scales $`x_k`$ (synoptic scales) and small scales $`y_{j,k}`$ (convective scales):

$$\begin{array}{ccc}\frac{dx_k}{dt}\hfill & =\hfill & x_{k1}\left(x_{k2}x_{k+1}\right)\nu x_k+F_{j=1}^Jy_{j,k}\hfill \\ \frac{dy_{j,k}}{dt}\hfill & =\hfill & cby_{j+1,k}\left(y_{j+2,k}y_{j1,k}\right)c\nu y_{j,k}+x_k,\hfill \end{array}$$
(6.7)
where $`k=1,\mathrm{},K`$ and $`j=1,\mathrm{},J`$. As in  we assume periodic boundary conditions on $`k`$ ($`x_{K+k}=x_k`$, $`y_{j,K+k}=y_{j,k}`$) while for $`j`$ we impose $`y_{J+j,k}=y_{j,k+1}`$. The variables $`x_k`$ represent some large scale atmospheric quantities in $`K`$ sectors extending on a latitude circle, while the $`y_{j,k}`$ represent quantities on smaller scales in $`JK`$ sectors. The parameter $`c`$ is the ratio between fast and slow characteristic times and $`b`$ measures the relative amplitude (both larger than unity). Model (6.7), even if rather crude, contains some non trivial aspects of the general circulation problem, namely the coupling among variables with very different characteristic times.

Being interested in forecasting the large scale behavior of the atmosphere by using only the slow variables, a natural choice for the model equations is:

$$\frac{dx_k}{dt}=x_{k1}\left(x_{k2}x_{k+1}\right)\nu x_k+FG_k(𝐱),$$
(6.8)
where $`G_k(𝐱)`$ represents the parameterization of the fast components in (6.7). Following the approach discussed in Ref. , a physical reasonable parameterization is

$$G_k(𝐱)=\nu _ex_k,$$
(6.9)
where $`\nu _e`$ is a numerically determined parameter.

In Figure 27 we plot $`\lambda _{TM}(\delta )`$ obtained from different choices of $`G_k`$. The simplest possibility is to neglect the fast variable, i.e. $`G_k=0`$. Also for large errors we have $`\lambda _{TM}(\delta )>\lambda _{TT}(\delta )`$ because this crude approximation is not able to capture the intrinsic predictability of the original system. More refined parameterizations in terms of stochastic processes with the correct probability distribution function and correlation time do not improve the forecasting ability. On the contrary Eq. (6.9) gives the result shown in Figure 27. At small scales we still observe deviations from $`\lambda _{TT}`$ but, at variance with the previous case, we recover intrinsic predictability for error of the size of the resolved scale.

As a more complex example, let us consider a version of the shell model discussed in Sect. 5.2, more precisely we study :

$`{\displaystyle \frac{du_n}{dt}}=i(k_{n+1}u_{n+1}^{}u_{n+2}{\displaystyle \frac{1}{2}}k_nu_{n1}^{}u_{n+1}`$ $`+`$ $`{\displaystyle \frac{1}{2}}k_{n1}u_{n2}u_{n1})`$ (6.10)
$``$ $`\nu k_n^2u_n+f_n,`$
with $`n=1,\mathrm{},N`$.

At variance with the previous example, here we have a set of scales $`\mathrm{}_n1/k_n`$ with characteristic times $`\tau _nk_n^{2/3}`$ (see Sect. 5.4). In order to simulate a finite resolution in the model, we consider a model of (6.10) in terms of an eddy viscosity :

$`{\displaystyle \frac{du_n}{dt}}=i(k_{n+1}u_{n+1}^{}u_{n+2}{\displaystyle \frac{1}{2}}k_nu_{n1}^{}u_{n+1}`$ $`+`$ $`{\displaystyle \frac{1}{2}}k_{n1}u_{n2}u_{n1})`$ (6.11)
$``$ $`\nu _n^{(e)}k_n^2u_n+f_n,`$
where now $`n=1,\mathrm{},N_M<N`$ and the eddy viscosity, restricted to the last two shells, has the form

$$\nu _n^{(e)}=\kappa \frac{|u_n|}{k_n}\left(\delta _{n,N_M1}+\delta _{n,N_M}\right),$$
(6.12)
where $`\kappa `$ is a constant of order $`1`$ . In the model equation $`N_M<N`$ the molecular viscosity term is much smaller than the eddy viscosity term and can be simply neglected. Model equations (6.11-6.12) are essentially the large eddy simulation for the shell model. Thus, although shell models are not realistic models for large scale geophysical flows (being nevertheless a good model for small scale turbulent fluctuations), the study of the effect of truncation in term of eddy viscosity is of general interest.

In Figure 28 we show $`\lambda _{MM}(\delta )`$, i.e. the FSLE computed for the model equations (6.11) with $`N=24`$ at different resolutions $`N_M=9,15,20`$. A plateau is detected for small amplitudes of the error $`\delta `$, corresponding to the LE, which increases with the resolution according to $`\lambda k_{N_M}^{2/3}`$. At larger $`\delta `$, the curves collapse onto the $`\lambda _{TT}(\delta )`$, showing that large-scale statistics of the model is not affected by small-scales resolution.

The ability of the model to predict satisfactorily the features of the “true” dynamics is not anyway determined by $`\lambda _{MM}(\delta )`$ but by $`\lambda _{TM}(\delta )`$, which is shown in Figure 29. Increasing the resolution $`N_M=9,15,20`$ towards the fully resolved case $`N=24`$ the model improves, in agreement with the expectation that $`\lambda _{TM}`$ approaches $`\lambda _{TT}`$ for a perfect model. At large $`\delta `$ the curves practically coincide, showing that the predictability time for large error sizes (associated with large scales) is independent of the details of small-scale modeling.

### 6.3 Lyapunov exponents and complexity in dynamical systems with noise

We saw how in deterministic dynamical systems there exist well established ways to define the complexity of a temporal evolution, either in terms of the Lyapunov exponents and the Kolmogorov-Sinai entropy, or by means of their generalization to non infinitesimal perturbations, like $`FSLE`$ and $`ϵ`$-entropy. The situation is much more ambiguous with random perturbations, which are always present in physical systems as a consequence of thermal fluctuations or hidden changes of control parameters, and, in numerical experiments, because of the roundoff errors .

The combined effect of the noise and the deterministic part of the evolution law can produce highly non-trivial behaviors . Let us mention stochastic resonance, where there is a synchronization of the jumps between two stable points  (for a recent review see ), and the phenomena of the so called noise-induced order  and noise-induced instability .

When facing systems with noise, the simplest possibility is to treat the random term as a time-dependent term, that is to consider the separation of two close trajectories with the same realization of noise. In this way one computes the largest LE, $`\lambda _\sigma `$, associated with the separation rate of two nearby trajectories with the same realization of the stochastic term (where $`\sigma `$ indicates the noise strength). Although $`\lambda _\sigma `$ is a well defined quantity, i.e. the Oseledec theorem  holds, it is not the most useful characterization of complexity. In addition, a moment of reflection shows that it is practically impossible to extract $`\lambda _\sigma `$ from experimental data.

We will show how, for noisy and random systems, a more natural indicator of complexity can be obtained by computing the separation rate of nearby trajectories evolving with different noise realizations. This measure of complexity, defined in , and inspired by the contributions of Shannon  and Kolmogorov , is related to the mean number of bits per unit time necessary to specify the sequence generated by a random evolution law.

#### 6.3.1 The naive approach: noise treated as a standard function of time

The approach in which one treats the random term as an usual time-dependent external force can lead to misleading results, as illustrated in the following example.

Let us consider a one-dimensional Langevin equation

$$\frac{dx}{dt}=\frac{V(x)}{x}+\sqrt{2\sigma }\eta ,$$
(6.13)
where $`\eta (t)`$ is a white noise and $`V(x)`$ diverges for $`x\mathrm{}`$, like, e.g., the usual double well potential $`V=x^2/2+x^4/4`$.

The Lyapunov exponent $`\lambda _\sigma `$, associated with the separation rate of two nearby trajectories with the same realization of $`\eta (t)`$, is defined as

$$\lambda _\sigma =\underset{t\mathrm{}}{lim}\frac{1}{t}\mathrm{ln}|z(t)|$$
(6.14)
where the evolution of the tangent vector is given by:

$$\frac{dz}{dt}=\frac{^2V(x(t))}{x^2}z(t).$$
(6.15)
Since the system is ergodic with invariant probability distribution $`P(x)=Ce^{V(x)/\sigma }`$, one has:

$$\begin{array}{cc}\lambda _\sigma \hfill & =lim_t\mathrm{}\frac{1}{t}\mathrm{ln}|z(t)|=lim_t\mathrm{}\frac{1}{t}_0^t_{xx}^2V(x(t^{}))dt^{}=\hfill \\ & \\ & C_{xx}^2V(x)e^{V(x)/\sigma }dx=\frac{C}{\sigma }(_xV(x))^2e^{V(x)/\sigma }𝑑x<0.\hfill \end{array}$$
(6.16)
This result has a rather intuitive meaning: the trajectory $`x(t)`$ spends most of the time in one of the “valleys” where $`_{xx}^2V(x)<0`$ and only short intervals on the “hills” where $`_{xx}^2V(x)>0`$, so that the distance between two trajectories evolving with the same noise realization decreases on average. Notice that in Ref. , supported by a wrong argument, an opposite conclusion has been claimed.

A negative value of $`\lambda _\sigma `$ implies a fully predictable process only if the realization of the noise is known. In the case of two initially close trajectories evolving under two different noise realizations, after a certain time $`T_\sigma `$, the two trajectories can be very distant, because they can be in two different valleys. For $`\sigma 0`$, due to the Kramers formula , one has $`T_\sigma \mathrm{exp}\mathrm{\Delta }V/\sigma `$, where $`\mathrm{\Delta }V`$ is the difference between the values of $`V`$ on the top of the hill and at the bottom of the valley. The result obtained for the one dimensional Langevin equation can easily be generalized to any dimension for gradient systems if the noise is small enough .

Another example showing the limitations of this approach is provided by the case of stochastic resonance in chaotic systems. In this case, in fact, one can find the same qualitative behavior both for a positive and a negative LE. We refer to  for more details.

#### 6.3.2 An information theory approach

The main difficulties in defining the notion of “complexity” of an evolution law with a random perturbation already appears in 1D maps. The generalization to $`N`$-dimensional maps or to ordinary differential equations is straightforward.

Therefore, we consider the model

$$x(t+1)=f[x(t),t]+\sigma w(t),$$
(6.17)
where $`t`$ is an integer and $`w(t)`$ is an uncorrelated random process, e.g. $`w`$ are independent random variables uniformly distributed in $`[1/2,1/2]`$. For the largest LE $`\lambda _\sigma `$, as defined in (6.14), now one has to study the equation

$$z(t+1)=f^{}[x(t),t]z(t),$$
(6.18)
where $`f^{}=df/dx`$.

Following the approach of Sect. 2.3, let $`x(t)`$ be the trajectory starting at $`x(0)`$ and $`x^{}(t)`$ be the trajectory starting from $`x^{}(0)=x(0)+\delta x(0)`$. Let $`\delta _0|\delta x(0)|`$ and indicate by $`\tau _1`$ the minimum time such that $`|x^{}(\tau _1)x(\tau _1)|\mathrm{\Delta }`$. Then, we put $`x^{}(\tau _1)=x(\tau _1)+\delta x(0)`$ and define $`\tau _2`$ as the time such that $`|x^{}(\tau _1+\tau _2)x(\tau _1+\tau _2)|>\mathrm{\Delta }`$ for the first time, and so on. In this way the Lyapunov exponent can be defined as

$$\lambda =\frac{1}{\overline{\tau }}\mathrm{ln}\left(\frac{\mathrm{\Delta }}{\delta _0}\right)$$
(6.19)
being $`\overline{\tau }=\tau _i`$ (see also Appendix A). If the above procedure is applied by considering the same noise realization for both trajectories, $`\lambda `$ in (6.19) coincides with $`\lambda _\sigma `$ (if $`\lambda _\sigma >0`$). Differently, by considering two different realizations of the noise for the two trajectories, we have a new quantity

$$K_\sigma =\frac{1}{\overline{\tau }}\mathrm{ln}\left(\frac{\mathrm{\Delta }}{\delta _0}\right),$$
(6.20)
which naturally arises in the framework of information theory  and algorithmic complexity theory. The times $`\tau _1,\tau _2,\mathrm{}`$ are nothing but the intervals at which it is necessary to repeat the transmission of $`x(t)`$, with a precision $`\delta _0`$, and $`K_\sigma /\mathrm{ln}2`$ is the number of bits per unit time one has to specify in order to transmit the sequence. If the fluctuations of the effective Lyapunov exponent $`\gamma (t)`$ are very small (i.e. weak intermittency) one has:

$$K_\sigma =\lambda +O(\sigma /\mathrm{\Delta }).$$
(6.21)
The interesting situation happens for strong intermittency when there are alternations of positive and negative $`\gamma `$ during long time intervals: this induces a dramatic change for the value of $`K_\sigma `$. This becomes particularly clear when we consider the limiting case of positive $`\gamma ^{(1)}`$ in an interval $`T_1>>1/\gamma ^{(1)}`$ followed by a negative $`\gamma ^{(2)}`$ in an interval $`T_2>>1/|\gamma ^{(2)}|`$, and again a positive effective LE and so on. During the intervals with positive effective LE the transmission has to be repeated rather often with $`T_1/(\gamma ^{(1)}\mathrm{ln}2)`$ bits at each time, while during the ones with negative effective LE no information has to be sent. Nevertheless, at the end of the contracting intervals one has $`|\delta x|=O(\sigma )`$, so that, at variance with the noiseless case, it is impossible to use them to compensate the expanding ones. This implies that in the limit of very large $`T_i`$ only the expanding intervals contribute to the evolution of the error $`\delta x(t)`$ and $`K_\sigma `$ is given by an average of the positive effective Lyapunov exponents:

$$K_\sigma \gamma \theta (\gamma ).$$
(6.22)
Note that it may happen that $`K_\sigma >0`$ with $`\lambda _\sigma <0`$. We stress again that (6.22) holds only for strong intermittency, while for uniformly expanding systems or rapid alternations of contracting and expanding behaviors $`K_\sigma \lambda _\sigma `$.

Note that $`K_\sigma `$ is a sort of $`ϵ`$-entropy (see Sect. 3.5), indeed, the complexity we consider is defined for $`\delta _0`$ not too small ($`\delta _0\sigma `$). If $`\delta _0`$ and $`\mathrm{\Delta }`$ are small enough, but still much larger than $`\sigma `$, $`K_\sigma `$ is essentially independent of their values.

The relation $`K_\sigma \gamma \theta (\gamma )`$ is the time analogous of the Pesin relation (2.15) $`h_{KS}_i\lambda _i\theta (\lambda _i)`$. The latter relation expresses the fact that negative Lyapunov exponents do not decrease the value of $`h_{KS}`$, because the contraction along the corresponding directions cannot be observed for any finite space partition. In the same way the contracting time intervals, if long enough, do not decrease $`K_\sigma `$. Another important remark is that in the usual treatment of the experimental data, where noise is usually present, one practically computes $`K_\sigma `$ and the result can be completely different from $`\lambda _\sigma `$.

Let us now briefly discuss some numerical results obtained with two different systems (Fig. 30 and Fig. 31). The first example consists in a periodic alternation of two piecewise linear maps of the interval $`[0,1]`$ into itself:

$$f[x,t]=\{\begin{array}{cc}ax\mathrm{mod}\mathrm{\hspace{0.17em}1}\mathrm{if}(2n1)Tt<2nT;\hfill & \\ bx\mathrm{mod}\mathrm{\hspace{0.17em}1}\mathrm{if}2nTt<(2n+1)T\hfill & \end{array}$$
(6.23)
where $`a>1`$ and $`b<1`$. Note that in the limit of small $`T`$, $`K_\sigma \mathrm{max}[\lambda _\sigma ,0]`$, because it is a non-negative quantity as shown in Fig. 30.

The second example (Fig. 31), strongly intermittent without external forcing, is the Beluzov-Zhabotinsky map , introduced for describing the famous chemical reaction:

$$f(x)=\{\begin{array}{c}\left[(1/8x)^{1/3}+a\right]e^x+b\mathrm{if}0x<1/8\hfill \\ \\ \left[(x1/8)^{1/3}+a\right]e^x+b\mathrm{if}1/8x<3/10\hfill \\ \\ c(10xe^{10x/3})^{19}+b\mathrm{if}3/10x\hfill \end{array}$$
(6.24)
with $`a=0.50607357,b=0.0232885279,c=0.121205692`$. The map exhibits a chaotic alternation of expanding and contracting time intervals. In Figure 31, one sees that while $`\lambda _\sigma `$ passes from negative to positive values at decreasing $`\sigma `$, $`K_\sigma `$ is not sensitive to this transition . Considering the system with a given realization of noise as the “true” evolution law, one has that $`\lambda _\sigma `$ corresponds to $`\lambda _{TT}`$ while $`K_\sigma `$ corresponds to $`\lambda _{TM}`$.

The previous results show that the same system can be regarded either as regular (i.e. $`\lambda _\sigma <0`$), when the same noise realization is considered for two nearby trajectories, or as chaotic (i.e. $`K_\sigma >0`$), when two different noise realizations are considered.

### 6.4 Random dynamical systems

We discuss now dynamical systems where the randomness is not simply given by an additive noise. This kind of systems has been the subject of interest in the last few years in relation to the problems involving disorder , such as the characterization of the so-called on-off intermittency  and to model transport problems in turbulent flows . In these systems, in general, the random part represents an ensemble of hidden variables believed to be implicated in the dynamics. Random maps exhibit very interesting features ranging from stable or quasi-stable behaviors, to chaotic behaviors and intermittency. In particular on-off intermittency is an aperiodic switching between static, or laminar, behavior and chaotic bursts of oscillation. It can be generated by systems having an unstable invariant manifold, within which it is possible to find a suitable attractor (i.e. a fixed point). For further details we refer to .

A random map can be defined in the following way. Denoting with $`𝐱(t)`$ the state of the system at discrete time $`t`$, the evolution law is given by

$$𝐱(t+1)=𝐟(𝐱(t),J(t)),$$
(6.25)
where $`J(t)`$ is a random variable.

As for the case of additive noise examined in the previous Section, the simplest approach is the introduction of the LE $`\lambda _J`$ computed considering the separation of two nearby trajectories evolving with the same realization of the random process $`J(t)=i_1,i_2,\mathrm{},i_t`$. The Lyapunov exponent $`\lambda _J`$ generalizes $`\lambda _\sigma `$ of Sect. 6.3.1 and can be computed from the tangent vector evolution:

$$\lambda _J=\underset{N\mathrm{}}{lim}\frac{1}{N}\mathrm{ln}|𝐳(N)|$$
(6.26)
where

$$z_m(t+1)=\underset{n}{}\frac{f_m(𝐱(t),i_t)}{x_n}z_n(t).$$
(6.27)
On the other hand, also for these systems, as in the case of additive noise, it is possible to introduce a measure of complexity, $`K`$, which better accounts for their chaotic properties

$$Kh_{Sh}+\lambda _J\theta (\lambda _J),$$
(6.28)
where $`h_{Sh}`$ is the Shannon entropy of the random sequence $`J(t)`$. The meaning of $`K`$ is rather clear: $`K/\mathrm{ln}2`$ is the mean number of bits, for each iteration, necessary to specify the sequence $`x_1,\mathrm{},x_t`$ with a certain tolerance $`\mathrm{\Delta }`$. Note that there are two different contributions to the complexity: (a) one has to specify the sequence $`J(1),J(2),\mathrm{},J(t)`$ which implies $`h_{Sh}/\mathrm{ln}2`$ bits per iteration; (b) if $`\lambda _J`$ is positive, one has to specify the initial condition $`x(0)`$ with a precision $`\mathrm{\Delta }\mathrm{exp}^{\lambda _JT}`$, where $`T`$ is the time length of the evolution. This requires $`\lambda _J/\mathrm{ln}2`$ bits per iteration; if $`\lambda _J`$ is negative the initial condition can be specified using a number of bits independent of $`T`$.

#### 6.4.1 A toy model: one dimensional random maps

Let us discuss a random map which, in spite of its simplicity, captures some basic features of this kind of systems :

$$x(t+1)=a_tx(t)(1x(t)),$$
(6.29)
where $`a_t`$ is a random dichotomous variable given by

$$a_t=\{\begin{array}{c}4\text{with probability }p\hfill \\ 1/2\text{with probability }1p.\hfill \end{array}$$
(6.30)
For $`x(t)`$ close to zero, we can neglect the non linear term to obtain

$$x(t)=\underset{j=0}{\overset{t1}{}}a_jx(0);$$
(6.31)
from the law of large numbers one has that the typical behavior is

$$x(t)x(0)e^{<\mathrm{ln}a>t}.$$
(6.32)
Since $`\mathrm{ln}a=p\mathrm{ln}4+(1p)\mathrm{ln}1/2=(3p1)\mathrm{ln}2`$ one has that, for $`p<p_c=1/3`$, $`x(t)0`$ for $`t\mathrm{}`$. On the contrary for $`p>p_c`$ after a certain time $`x(t)`$ escapes from the fixed point zero and the non-linear term becomes relevant. Figure 32 shows a typical on-off intermittency behavior for $`p`$ slightly larger than $`p_c`$. Note that, in spite of this irregular behavior, numerical computations show that the LE $`\lambda _J`$ is negative for $`p<\stackrel{~}{p}_c0.5`$: this is essentially due to the non-linear terms.

By introducing a finite threshold $`ϵ`$, in order to discriminate laminar and intermittent phases, we can define a complexity $`K(ϵ)`$. We denote with $`l_L`$ and $`l_J`$ the average life times respectively of the laminar and of the intermittent phases for $`p`$ close to $`p_c`$ ($`l_J<<l_L`$). The mean number of bits, per iteration, one has to specify in order to transmit the sequence is

$$\frac{K(ϵ)}{\mathrm{ln}2}\frac{l_Jh_{Sh}}{(l_J+l_L)\mathrm{ln}2}\frac{l_J}{l_L}\frac{h_{Sh}}{\mathrm{ln}2}.$$
(6.33)
To obtain (6.33) first notice that on an interval $`T`$ one has approximatively $`T/(l_J+l_L)`$ intermittent bursts and the same number of laminar phases. Then notice that, during a laminar phase, there is not an exponential growth of the distance between two trajectories initially close and computed with the same sequence of $`a_t`$. Since during a laminar phase one has to send a number of bits which does not depend on its duration, one can send all the necessary information simply by giving the sequence of $`a_t`$ during the intermittent bursts. Eq.(6.33) has an intuitive interpretation: in systems with a sort of “catastrophic” events, the most important feature is the mean time between two subsequent events.

#### 6.4.2 Sandpile models as random maps

Another example of a system which can be treated in the framework of random maps is represented by the so-called sandpile models . These models are a paradigmatic example of the Self-Organized Criticality (SOC) . This term refers to the tendency of some large dynamical systems to evolve spontaneously toward a critical state characterized by spatial and temporal self-similarity. The original Sandpile Models are probabilistic cellular automata inspired to the dynamics of avalanches in a pile of sand. Dropping sand slowly, grain by grain on a limited base, one reaches a situation where the pile is critical, i.e. it has a critical slope. This means that a further addition of sand will produce sliding of sand (avalanches) that can be small or cover the entire size of the system. In this case the critical state is characterized by scale-invariant distributions for the size and the lifetime and it is reached without tuning of any critical parameter.

We will refer in particular to the Zhang model , a continuous version of the original sandpile model , defined on a $`d`$-dimensional lattice. The variable on each site $`x_i`$ (interpretable as energy, sand, heat, mechanical stress etc.) can vary continuously in the range $`[0,1]`$ with the threshold fixed to $`x_c=1`$. The dynamics is the following:

(a) one chooses at random a site and adds to it an energy $`\delta e`$,

(b) if at a certain time $`t`$ the energy in a site, say $`i`$, exceeds the threshold $`x_c`$ a relaxation process is triggered defined as:

$$\{\begin{array}{c}x_{i+nn}x_{i+nn}+x_i/2d\hfill \\ x_i0,\hfill \end{array}$$
(6.34)
where $`\mathrm{𝑛𝑛}`$ indicates the $`2d`$ nearest neighbors of the site $`i`$;

(c) one repeats point (b) until all the sites are relaxed;

(d) one goes back to point (a).

Let us now discuss the problem of predictability in sandpile models on the basis of the rigorous results , which clarify the role of the LE for this class of systems.

In Ref.  it has been proved that the LE $`\lambda _J`$ is negative. In fact the dynamics of a little difference between two configurations follows the same rules (a)-(d), i.e., the “error” is redistributed to the nearest neighbors site, so that one has

$$\lambda _J\frac{\mathrm{const}}{R^2}$$
(6.35)
where $`R`$ is the diameter of the system.

As for other examples already discussed, the existence of a negative LE does not mean a perfect predictability. This can be understood by looking at the growth of the distance, $`\delta (t)`$, between two initially close trajectories computed with two different realizations of randomness, i.e., by adding sand in different sites. Let us consider the case of the “minimal error”: in the reference realization one adds sand on a site $`i`$ chosen at random. In the perturbed realization, instead, one adds a sand grain at one of the nearest sites of $`i`$. In such a case $`\delta (t)`$ increases up to a maximal distance in few avalanches . Practically, one has the same kind of phenomenon, already discussed, of the Langevin equation with two noise realizations.

Let us now estimate the complexity $`K`$ of this system. An upper bound can be given by using (6.28) $`K=h_{Sh}+\lambda _J\theta (\lambda _J)`$, where $`h_{Sh}`$ is the entropy of the random sequence of addition of energy. In sandpile models, since each site has the same probability to be selected, one has $`h_{Sh}=\mathrm{ln}V`$, where $`V`$ is the number of sites of the system. Since the Lyapunov exponent is negative, the complexity is just determined by $`h_{Sh}`$.

## 7 Irregular behavior in discrete dynamical systems

For the sake of completeness we include in this review a discussion on the characterization of irregular behaviors in systems whose states are discrete. Such systems include Cellular Automata (CA), which have been intensively studied both for their intrinsic interest  and for applications as, e.g., to simulate hydrodynamic equations  or to study various forms of chemical turbulence . Other interesting systems with discrete states are the neural networks used for modeling some brain functions . It is also relevant to note that in every simulation with a computer, because of the finite number of digits it can use, one deals with a system with discrete states (see Sect. 6.1). In addition, the general problem of dynamics of systems with discrete states is important in the debated issue of quantum chaos. Indeed quantum mechanics can be regarded as a discretized version of the classical one, acting on a suitable lattice in phase space, where the number of the possible states is proportional to the inverse of the Planck constant .

If a system consists of $`N`$ elements and each element can assume an integer number $`k`$ of distinct values, $`𝒩=k^N`$ is the number of states. When these states evolve with a deterministic rule, the dynamics can be depicted in terms of oriented graphs: a set of points, representing the states, are connected by arrows, indicating the time evolution. Of course, each point has one, and only one, outgoing arrow; but different arrows can end at the same point. For any finite system each initial condition evolves to a definite attractor, which can be either a fixed point (as in Fig. 33a), or a periodic orbit (Fig. 33b).

In systems of this kind, obviously, it is not possible to use the previously introduced indicators of chaos, e.g. the Lyapunov exponents or the Kolmogorov-Sinai entropy, whose definitions rely on the continuous character of the system states. Moreover, the asymptotic periodic behavior seems to force the conclusion that discrete states systems are trivial, from an entropic or algorithmic complexity point of view.

The above conclusions, although mathematically correct, are rather unsatisfactory from the physical point of view, indeed from this side the following questions deserve some interest:

1. What is the “typical” period, $`𝒯`$, in a system with $`N`$ elements, each assuming $`k`$ distinct values ?
2. When $`𝒯`$ is very large, how can we characterize the (possible) irregular behavior of the trajectories, on times that are large enough but still much smaller than $`𝒯`$ ?
3. What does it happen in the transition from discrete to continuous states, i.e. in the limit $`k\mathrm{}`$ ?

In the next subsections we will deal to the above questions.

### 7.1 Dependence of the period on the number of the states

For deterministic discrete state systems the dependence of the period of the attractor on the number of the states, may be addressed with a statistical approach in terms of random maps . We recall that this problem is important for computer simulations of chaotic systems (see Sect. 6.1). If $`𝒩=k^N1`$ is the number of states of the system, the basic result for the average period, $`𝒯`$, is

$$𝒯(𝒩)\sqrt{𝒩}.$$
(7.1)
In the following we give a simple argument, by Coste and Hénon .

For simplicity of notation, we consider the case with $`k=2`$, so that the state of the system is a string of $`N`$ bits. A deterministic evolution of such a system is given by a map which is one among the possible functions connecting the $`2^N`$ states. Let us now assume that all the possible functions can be extracted with the same probability. Denoting with $`I(t)`$ the state of the system, for a certain map we have a periodic attractor of period $`m`$ if $`I(p+m)=I(p)`$ and $`I(p+j)I(p)`$, for $`j<m`$. The probability, $`\omega (m)`$, of this periodic orbit is obtained by specifying that the $`(p+m1)^{th}`$ first successive images of the map are distinct from all the previous ones and the $`(p+m)^{th}`$ iterates coincides with the $`p^{th}`$ one. Since one has $`I(p+1)I(p+m)`$, with probability $`\left(11/𝒩\right)`$; $`I(p+2)I(p+m)`$, with probability $`\left(12/𝒩\right)`$; $`\mathrm{}\mathrm{};I(p+m1)I(p+m)`$, with probability $`\left(1(m1)/𝒩\right)`$; and, finally, $`I(p+m)=I(p)`$ with probability $`\left(1/𝒩\right)`$, one obtains

$$\omega (m)=\left(1\frac{1}{𝒩}\right)\left(1\frac{2}{𝒩}\right)\mathrm{}\left(1\frac{(m1)}{𝒩}\right)\frac{1}{𝒩}.$$
(7.2)
The average number, $`M(m)`$, of cycles of period $`m`$ is

$$M(m)=\frac{𝒩}{m}\omega (m)\stackrel{(𝒩1)}{}\frac{e^{m^2/2𝒩}}{m},$$
(7.3)
from which one obtains $`𝒯\sqrt{𝒩}`$ for the average period.

It is here appropriate to comment on the relevance of Eq. (7.1) for computer-generated orbits of chaotic dynamical systems. Because of the finite number, $`n`$, of digits used in the floating point representation, when one iterates a dynamical system, one basically deals with a discrete system with a finite number $`𝒩`$ of states. If $`d_2`$ indicates the correlation dimension of the system , one can reasonably assume that $`𝒩10^{nd_2}`$, so that, from Eq. (7.1) one has:

$$𝒯10^{nd_2/2}.$$
(7.4)
This estimation gives an upper limit for the typical number of meaningful iterations of a map on a computer. Note that this number, apart from the cases of one or two dimensional maps with few digits, is very large for almost all practical purposes.

### 7.2 The transition from discrete to continuous states

Following the basic ideas of Ford , as discussed in Sect. 2.3, and the results of Sect. 6 – on the predictability in systems whose evolution law is not completely known – we describe now a way to introduce a practical definition of chaos for systems with discrete states. In addition, we deal with the problem of the transition from discrete to continuous states.

Given a system with $`𝒩`$ possible states, denoting with $`𝐈(t)`$ its state at time $`t`$ we can write its evolution law as:

$$𝐈(t+1)=𝐅[𝐈(t)].$$
(7.5)
A single state $`𝐈`$ is a sequence of (at most) $`\mathrm{ln}_2𝒩`$ bits, and its time evolution for $`M`$ steps can be surely translated in a binary sequence $`\mathrm{\Sigma }`$ of length $`\mathrm{}_\mathrm{\Sigma }(M,𝒩)M\mathrm{ln}_2𝒩`$.

Relying one the definition of algorithmic complexity (Sect. 2.2.3) we can make the following classification: we call regular (compressible) those sequences that can be encoded by a computer program whose length $`\mathrm{}_\mathrm{\Sigma }(M,𝒩)`$ increases less than linearly in $`M`$, when $`M`$ ranges over a physically significant interval, at fixed values of $`𝒩`$. Otherwise the system will be called chaotic or incompressible. Let us call $`\mathrm{}_F`$ the binary length of the algorithm for one step: $`\mathrm{}_F2𝒩\mathrm{ln}_2𝒩`$. The sequence $`\mathrm{\Sigma }`$ can be expressed by the record composed by the initial state $`𝐈(0)`$ (specified by $`\mathrm{ln}_2𝒩`$ bits), the number of steps $`M`$ (specified by $`\mathrm{ln}_2M`$ bits) and the rule $`𝐅`$ for one step (specified by $`\mathrm{}_F`$ bits). Therefore

$$\mathrm{}_\mathrm{\Sigma }(M,𝒩)(2𝒩+1)\mathrm{ln}_2𝒩+\mathrm{ln}_2M+O(1).$$
(7.6)
Let us note that from the above equation one has that – when $`M`$ grows indefinitely and $`𝒩`$ is constant – $`\mathrm{}_\mathrm{\Sigma }`$ is logarithmically bounded and hence the sequence appears to be compressible. This is somewhat trivial since, because of the discrete nature of the states, the motion at $`M>𝒩`$ (in practice $`M>𝒯\sqrt{𝒩}`$) is periodic. Therefore it is interesting to consider only $`1M<𝒯𝒩`$. Although the evolution law (7.5) can be carried out, in principle, in exact arithmetic, in practice in real computations one has unavoidable errors due to truncations and approximations. Let us now regard the evolution law (7.5) as a computer program with input $`𝐈(0)`$ and a set $`𝐂`$ of parameters, with $`C`$ components, needed to define the algorithm $`𝐅`$. If these parameters are all known within precision $`O(2^q)`$, the binary length of the coding of $`𝐂`$ is $`O(qC)`$.

Consider the following problem: given two identical initial conditions $`𝐈^{(1)}(0)=𝐈^{(2)}(0)=𝐈(0)`$, and two different realizations $`𝐂^{(1)}`$ and $`𝐂^{(2)}`$ of the set of coefficients $`𝐂`$ (with difference $`O(2^q)`$), what is the dependence on $`ϵ=2^q`$ of the first-error time $`\stackrel{~}{M}`$ (i.e. the first time for which $`𝐈^{(1)}(t)𝐈^{(2)}(t)`$)? Of course, the answer depends on the realizations of the components of $`𝐂`$ and on the initial conditions $`𝐈(0)`$. Let us consider $`𝐂^{(2)}`$ as an $`ϵ`$-perturbation of $`𝐂^{(1)}`$, i.e. we pose, for each component of the parameter vector:

$$C_i^{(2)}=C_i^{(1)}+ϵ_i,$$
(7.7)
where the random variables $`ϵ_i`$ are uniformly distributed in $`[2^{(q1)},2^{(q1)}]`$. Let us note that the coding length $`O(Cq)+\mathrm{ln}_2𝒩`$ is enough to define the sequence $`\mathrm{\Sigma }`$ up to the first error time $`\stackrel{~}{M}`$. Performing an average on the $`ϵ_i`$ and on the initial conditions $`𝐈(0)`$, one can compute an average first-error time $`\stackrel{~}{M}(ϵ)`$, and a typical first-error time $`\stackrel{~}{M}_{typ}(ϵ)=\mathrm{exp}\mathrm{ln}\stackrel{~}{M}(ϵ)`$. For $`\stackrel{~}{M}`$ and $`\stackrel{~}{M}_{typ}`$ the dependence on $`ϵ`$ can be of the following type:

1. $`\stackrel{~}{M}ϵ^\beta 2^{q\beta }`$ ,
2. $`\stackrel{~}{M}\mathrm{ln}_2(1/ϵ)q`$.

In the case $`(a)`$ we say that the system is compressible, while if $`(b)`$ holds one has a chaotic (incompressible) case. The above classification is rather obvious: in case $`(a)`$ a trajectory of length $`\stackrel{~}{M}`$ can be coded by a program of length $`O(\mathrm{ln}_2𝒩)+O(\mathrm{ln}\stackrel{~}{M})`$, while in case $`(b)`$ one has a length $`O(\mathrm{ln}_2𝒩)+O(\stackrel{~}{M})`$. For a detailed discussion see Ref. .

Let us now discuss in an explicit example the previous approach and the problems in the transition to the continuous state case. We consider a discretized standard map, as obtained by considering lattice points in the torus $`[0,2\pi ]^2`$ of the form $`(x,y)`$ with $`x=2\pi Q/L`$ and $`y=2\pi P/L`$, where $`P`$ and $`Q`$ are integers between $`1`$ and $`L`$. The action of the map is

$$\{\begin{array}{cc}Q(t+1)=\hfill & \left[Q(t)+\alpha \frac{L}{2\pi }\mathrm{sin}(P(t)\frac{2\pi }{L})\right]\mathrm{mod}\mathrm{L}\hfill \\ & \\ P(t+1)=\hfill & (P(t)+Q(t+1))\mathrm{mod}\mathrm{L}\hfill \end{array}$$
(7.8)
where $`\alpha `$ is the control parameter and $`[]`$ means integer part. From the results of Sect. 7.1 one has that the typical period for the map (7.8) is $`𝒯_LL`$; so if $`L`$ is large the periodic motion will be seen only for sufficiently large times. In the system (7.8) one has just one parameter, i.e. the “kick strength” $`\alpha `$. Numerical evidence supports the following picture: at fixed $`L`$, the first-error time is roughly constant for large values of the error $`ϵ`$, while it goes as $`ϵ^1`$ for small errors $`ϵ`$. The transition between the two regimes occurs at a critical value $`ϵ_\mathrm{c}(L)`$ which scales as $`ϵ_\mathrm{c}1/L`$. In formulae:

$$\stackrel{~}{M}(ϵ)\{\begin{array}{cc}O(1)\hfill & \text{for }ϵ>ϵ_\mathrm{c}(L)\hfill \\ 1/ϵ\hfill & \text{for }ϵ<ϵ_\mathrm{c}(L).\hfill \end{array}$$
(7.9)
It is rather easy to give analytical estimates supporting the numerical evidence

$$\stackrel{~}{M}(ϵ)=\{\begin{array}{cc}\left(\frac{\pi l}{ϵ}\right)\hfill & \text{for }ϵ<l\hfill \\ \left(\frac{ϵ}{\pi l}(1\mathrm{cos}\theta )+1\frac{2\theta }{\pi }+\frac{l}{\pi ϵ}\mathrm{ln}\mathrm{tan}(\frac{\theta }{2})\right)^1\hfill & \text{for }ϵ>l\text{.}\hfill \end{array}$$
(7.10)
where the angle $`\theta `$ is defined via $`\mathrm{sin}\theta =l/ϵ`$, and $`l=2\pi /L`$. Numerical simulations show that the behavior proposed in Eq. (7.9) holds. To have a comparison with the usual standard map, we have computed the average time $`M_\mathrm{\Delta }(ϵ)`$ required for two trajectories to reach lattice points farther than a fixed distance $`\mathrm{\Delta }`$ in the discrete phase space of Eq. (7.8). We found:

$$M_\mathrm{\Delta }(ϵ)\{\begin{array}{cc}(1/\lambda )\mathrm{ln}(\mathrm{\Delta }/ϵ)\hfill & \text{for }ϵ>ϵ_\mathrm{c}(L)\hfill \\ 1/ϵ\hfill & \text{for }ϵ<ϵ_\mathrm{c}(L).\hfill \end{array}$$
(7.11)
We remark that when $`ϵ<ϵ_\mathrm{c}(L)`$, $`M_\mathrm{\Delta }(ϵ)`$ is weakly dependent, i.e. logarithmically, on $`\mathrm{\Delta }`$. This is just another verification of the similarity of the effect of a small disturbance on the equations of motion and of a small error in the initial conditions for a dynamical evolution (see Sect. 6).

These results unveil the nature of the dynamics of this discrete system: its trajectories are incompressible and therefore chaotic only for large values of $`ϵ`$, the cutoff value decreasing as $`1/L`$. This helps us also to understand the extent to which the dynamics of the discrete standard map Eq. (7.8) is equivalent to its continuum counterpart. When $`\alpha `$ is large, and $`ϵ>ϵ_\mathrm{c}`$, the two systems possess chaotic trajectories. Simple calculations show that, to the leading order, $`\stackrel{~}{M}\mathrm{log}_2(L)`$. After this time, the discrete system appears “regular”, i.e. compressible and predictable. Therefore, continuous and discrete systems are similar (as far as chaos is concerned) only over logarithmically short times. It is important to stress that the system appears “regular” on time scales much smaller than the typical period $`𝒯_LL^{d_2/2}`$ (being $`d_2`$ the correlation dimension of the attractor ).

Recently Mantica  studied the algorithmic complexity in classical polygonal billiards with $`L`$ sides. The system, for any finite value of $`L`$, is regular; on the other hand, as $`L\mathrm{}`$, it tends to a curved billiard, which can be chaotic. This system is very similar to the discrete dynamical system (7.8) and may be used to study the transition from quantum to classical mechanics and the principle of correspondence. The average complexity of symbolic trajectories in the polygonal billiards has the same scaling behavior (as function of $`L`$ and of the precision $`ϵ`$) of that one of the system (7.8), i.e. a compressible (regular) regime for $`ϵ<ϵ_c1/L`$ and an incompressible (chaotic) one for $`ϵ<ϵ_c`$.

It is interesting to note that a similar feature is characteristic of quantum dynamics of systems whose classical behavior is chaotic. Roughly speaking, a quantum system behaves as a system with discrete states whose number is proportional to $`\mathrm{}^1`$. A semi-classical wave function follows a dynamics which is approximately classical up to a time $`t_c(1/\lambda )\mathrm{ln}(I/\mathrm{})`$, where $`\lambda `$ is the Lyapunov exponent of the classical motion, and $`I`$ is a typical action of the motion. Over this time, the quantum system has the same complexity of its classical counterpart, while for larger times its quantal (quasi-periodic) nature appears .

### 7.3 Entropies and Lyapunov exponents in Cellular Automata

Cellular automata (CA) consist of discrete valued state variables, $`\sigma _i(t)`$, defined on a discrete lattice, and updated synchronously at discrete time according to a local rule. They can be defined in any dimensions and for any finite number of possible values for $`\sigma _i(t)`$. For the sake of simplicity we consider Boolean CA, i.e. $`\sigma _i(t)=\{0,1\}`$, in a 1-dimensional lattice. An evolution rule can be written as:

$$\sigma _i(t+1)=F[\sigma _{ir}(t),\mathrm{},\sigma _i(t),\mathrm{},\sigma _{i+r}(t)],i=1,\mathrm{},N,$$
(7.12)
where $`r`$ defines the range of the coupling, i.e. the variable in a site depends on the variables in the $`2r`$ neighbor sites. If $`F`$ in (7.12) only depends on the sum of the state variables, one speaks of “totalistic” CA. Another usual requirement is to have symmetric rules. For further details we refer to Ref. , where the standard scheme for the classification of the possible rules and a number of examples of CA-behavior are discussed.

In the following we refer to $`1d`$ Boolean cellular automata with local symmetric rules – as those ones systematically studied by Wolfram .

#### 7.3.1 Classification of Cellular Automata according to the transient times

For finite lattices with $`N`$ sites the number of possible states of CA is finite and equal to $`𝒩=2^N`$. As already discussed, this means that, strictly speaking, from an entropic (or algorithmic) point of view CA are trivial. Therefore the problem of the characterization of irregular behaviors in CA has, in principle, some meaning only in the limit $`N\mathrm{}`$. In more physical terms, for finite $`N`$ one expects the characterization in terms of entropy to be possible for times shorter than the typical period $`𝒯(N)`$ or the typical transient time $`\stackrel{~}{𝒯}(N)`$, provided $`𝒯(N)`$ and $`\stackrel{~}{𝒯}(N)`$ are long enough.

Actually, cellular automata behaviors can be classified according to the dependence of $`𝒯(N)`$ and $`\stackrel{~}{𝒯}(N)`$ on $`N`$. One has three possible classes of behavior.

Regular cellular automata (class 1 and 2 in Wolfram’s classification ) evolve either on homogeneous states both in time and space (the analogous of fixed point in dynamical systems) either to a set of simple stable periodic structures (analogous to limit cycles) which, in general, depend on the initial configuration (Fig. 34a). In these CA, $`𝒯(N)`$ and $`\stackrel{~}{𝒯}(N)`$ can range from being almost independent of $`N`$ to be, at maximum, proportional to $`N`$.

Chaotic cellular automata (class 3 in ) yield disordered patterns (Fig. 34b). For any finite $`N`$ these CA reach periodic states, but there are rather clear numerical evidences that the transient time $`\stackrel{~}{𝒯}`$ increases exponentially with the system size:

$$\stackrel{~}{𝒯}(N)\mathrm{exp}(cN).$$
(7.13)
Moreover, also the cycle period shows in most of the cases a similar dependence on $`N`$, this is a reminiscence of what we discussed in Sect. 7.1.

Complex cellular automata (class 4 in , Fig. 34c) usually evolve toward complex localized structures (gliders) which interact in a complicate way. For these CA numerical simulations  have shown that both the transient time and the cycle period display a non trivial $`N`$-dependence (i.e. the average, the typical values or the median depend in a different way on $`N`$). The unpredictability of these system manifests itself in the distribution of these times. In particular, the large variability of these times in dependence of the initial conditions and the lattice size inhibits any forecasting of the duration of the transient.

In the following we limit the discussion to chaotic rules, i.e. class 3 in the Wolfram classification. A detailed characterization of complex CA would require the introduction of concepts and tools that are beyond the aim of this review, for further details see Refs. .

#### 7.3.2 Sensitive dependence on initial conditions

A first intuitively reasonable characterization of irregular behaviors is in terms of sensitive dependence on initial conditions, but in CA it is not possible to have arbitrary small distances between two states. Nevertheless, for large $`N`$, when considering two states with only one different element, one can say that, in some sense (i.e. in an appropriate norm), the difference is small. Denoting with $`R_t`$ the number of different elements at time $`t`$, we can define the damage propagation speed as

$$v=\underset{t\mathrm{}}{lim}\frac{R_t}{2t}.$$
(7.14)
It is not difficult to see that $`v`$ is, in a proper space, a Lyapunov exponent (i.e. it measures the rate of divergence of two configurations) . Consider two initial bi-infinite configurations $`𝝈(0)=(\mathrm{},\sigma _2(0),\sigma _1(0),\sigma _1(0),\sigma _2(0),\mathrm{})`$ and $`𝝈^{^{\mathbf{}}}(0)=(\mathrm{},\sigma _2^{^{}}(0),\sigma _1^{^{}}(0),\sigma {}_{}{}^{}{}_{1}{}^{}(0),\sigma _2^{^{}}(0),\mathrm{})`$, with $`\sigma _i(0)=\sigma _i^{^{}}(0)`$ for $`|i|<N_0`$, and their evolutions $`𝝈(t)`$ and $`𝝈^{^{}}(t)`$. One can define a distance, $`\delta 𝝈(t)`$, between $`𝝈(t)`$ and $`𝝈^{^{}}(t)`$, as follows:

$$\delta 𝝈(t)=\underset{n=1}{\overset{\mathrm{}}{}}\frac{|\delta \sigma _n(t)|+|\delta \sigma _n(t)|}{2^n}$$
(7.15)
where $`\delta \sigma _n=\sigma _n^{^{}}\sigma _n`$. With the above norm two systems can be arbitrarily close: one only needs $`N_01`$. At this point it is possible to define the Lyapunov exponent as

$$\lambda =\underset{t\mathrm{}}{lim}\underset{\delta 𝝈(0)0}{lim}\frac{1}{t}\mathrm{ln}\frac{\delta 𝝈(t)}{\delta 𝝈(0)}.$$
(7.16)
Note that in (7.16) it has been implicitly taken the limit $`N\mathrm{}`$.

Noting that $`\delta \sigma _n(t)=0`$ for $`|nN_0|>R_t/2vt`$, while $`|\delta \sigma _n(t)|=1`$ for $`|nN_0|<R_t/2vt`$, from the definition (7.15) one has:

$$\delta 𝝈(t)2^{N_0+vt},$$
(7.17)
and therefore

$$\lambda =v\mathrm{ln}2.$$
(7.18)
In other words, the linear damage spreading in the physical space corresponds to an exponential growth in the norm (7.15). Oono and Yeung  stressed a conceptual (and practical) difficulty with the above approach. In systems with continuous states it is clear that by performing an infinitesimal change on a typical configuration one does not destroy the “typicality”, i.e. the new initial condition will generate a trajectory belonging to the same attractor. On the contrary, it is not obvious that for a, however large, system with discrete states in a typical configuration a change of only one element gives another typical state. For instance, this seemingly innocent change can induce a jump among basins of attraction, so that the perturbed trajectory goes to a different attractor . However, taking into account the above criticism, numerically one finds, for most the initial conditions, $`v>0`$ for chaotic CA, and $`v=0`$ for regular CA.

We conclude this subsection mentioning a proposal, by Bagnoli et. al. , to introduce a Lyapunov exponent for cellular automata, defining it in analogy with continuous states dynamical systems.

In this approach, the equivalent of an infinitesimal perturbation (as for the damage spreading analysis) is the difference between the system and one of its replicas in which one site has been flipped at time $`t=0`$. Then one formally introduces the Boolean derivatives, a sort of Jacobian of the rule, $`F_{i,j}^{^{}}`$, the elements of which are $`0`$ or $`1`$. Here, for simplicity, we consider a generic nearest neighbor ($`r=1`$) rule so that $`F_{i,j}^{^{}}=0`$ for $`|ij|2`$ and

$$F_{i,i1}^{^{}}=\frac{\sigma _i(t+1)}{\sigma _{i1}(t)}F[\sigma _{i1},\sigma _i,\sigma _{i+1}]\mathrm{𝐗𝐎𝐑}F[\sigma _{i1}\mathrm{𝐗𝐎𝐑}\mathrm{\hspace{0.17em}1},\sigma _i,\sigma _{i+1}]$$

where the other nonzero terms are obtained by shifting the $`\mathrm{𝐗𝐎𝐑}`$ operation to $`i`$ and $`i+1`$ (respectively). We recall that $`\mathrm{𝐗𝐎𝐑}`$ is the Boolean exclusive operation (i.e. $`0\mathrm{𝐗𝐎𝐑}\mathrm{\hspace{0.17em}0}=0`$, $`1\mathrm{𝐗𝐎𝐑}\mathrm{\hspace{0.17em}1}=0`$, $`0\mathrm{𝐗𝐎𝐑}\mathrm{\hspace{0.17em}1}=1`$ and $`1\mathrm{𝐗𝐎𝐑}\mathrm{\hspace{0.17em}0}=1`$). Of course as time goes on the initial perturbation spreads, i.e. new defects appear. As in continuous systems, one needs to maintain the perturbation “infinitesimal”. One introduces a vector $`𝐍`$ (whose components, $`N_i`$, take integer values) which plays the role of the tangent vector. In order to mimic an infinitesimal perturbation at the initial time one assumes $`N_i(0)=\delta _{ij}`$, i.e. only one defect on the site $`j`$. The dynamics of $`N_i`$ is ruled by the Boolean derivative i.e.

$$N_i(t+1)=\underset{j}{}F_{ij}^{^{}}(t)N_j(t).$$
(7.19)
Finally, putting $`|𝐍(t)|=_jN_j(t)`$, one can define the “Lyapunov exponent”, $`\lambda _B`$, of the cellular automaton as:

$$\lambda _B=\underset{T\mathrm{}}{lim}\frac{1}{T}\mathrm{ln}(|𝐍(T)|).$$
(7.20)
Now, in analogy with continuous systems, $`\lambda _B<0`$ indicates an exponential decrease of the perturbation, while for $`\lambda _B>0`$ the damage spreads. Just to give an example, if one considers the rule $`150`$ of Wolfram classification, i.e. ($`F[0,0,1]=F[0,1,0]=F[1,0,0]=F[1,1,1]=1`$ and $`0`$ otherwise) it is easy to see that $`F_{ij}^{^{}}`$ is a tridiagonal matrix with all the elements equal to $`1`$ so that $`\lambda =\mathrm{ln}(3)`$. For a generic rule one has to compute a suitable average over a long trajectory or on many initial configurations.

The Lyapunov exponent, $`\lambda _B`$, has been demonstrated to be relevant in the synchronization problem  and allows for a qualitative characterization of the cellular automata in agreement with the classification proposed by Wolfram .

#### 7.3.3 Entropies

For cellular automata one can define a spatial/temporal entropy density by looking at the evolution of the elements in a subset $`_L`$, of size $`L`$, of the system. Denoting with $`C(L,T)`$ a “word” of spatial size $`L`$ and time length $`T`$ appearing in the time evolution of the elements in $`_L`$, one defines the entropy of the subset $`_L`$,

$$h(L)=\underset{T\mathrm{}}{lim}\frac{1}{T}\underset{C(L,T)}{}P(C(L,T))\mathrm{ln}P(C(L,T)),$$
(7.21)
and then the spatio-temporal entropy density as

$$h^{ST}=\underset{L\mathrm{}}{lim}\frac{1}{L}h(L).$$
(7.22)
This entropy cannot be practically computed. A more accessible quantity is the temporal entropy:

$$h^T=h(1)=\underset{T\mathrm{}}{lim}\frac{1}{T}\underset{C(1,T)}{}P(C(1,T))\mathrm{ln}P(C(1,T)),$$
(7.23)
i.e. the Shannon entropy of the time sequence of one element $`(\sigma _n(0),\sigma _n(1),\mathrm{})`$. In principle, $`h^T`$ can depend on the site $`n`$ and one can classify as nontrivial a system for which the majority of the elements have $`h^T>0`$ . An average measure of the “temporal disorder” is given by the spatial average $`h^T`$. A systematic study of $`h(1),h(2),h(3),\mathrm{}`$ – although very difficult in practice – could give, in principle, relevant information on the spatial/temporal behavior. A characterization of the spatial properties can be obtained studying, at a given time $`t`$, the spatial sequences. In practice, one studies $`C(L,1)`$ at increasing $`L`$:

$$h^S=\underset{L\mathrm{}}{lim}\frac{1}{L}\underset{C(L,1)}{}P(C(L,1))\mathrm{ln}P(C(L,1)).$$
(7.24)
One can associate to $`h^S`$ a sort of “effective” dimension $`d=h^S/\mathrm{ln}2`$ . In a completely disordered cellular automaton configuration one has $`d=1`$, as expected, while a homogeneous (or spatially periodic) configuration gives $`d=0`$.

From the definition of cellular automata (7.12) one easily sees that the value of $`\sigma _i(t)`$ depends on sites at maximum distance $`r`$ from $`i`$ at the previous time step. This means that after $`T`$ time steps, the value $`\sigma _i(t)`$ can depend (at maximum) on sites at distance $`rT`$ on both direction, so that the maximum speed for information propagation is $`r`$ (i.e. the range of interaction). However, for many CA the actual velocity of information propagation, $`v_p`$, is less than $`r`$, i.e. $`\sigma _i(T)`$ depends only on $`v_pT<rT`$ sites.

By considering a simple construction (see Fig. 36) one can understand that the spatial and temporal entropies are related to each other by the inequality :

$$h^T2v_ph^S,$$
(7.25)
where a good estimate of $`v_p`$ can be given in terms of the damage spreading velocity (7.18) .

The possible scenario arising from (7.25) can be summarized as follows. One can have “spatial chaos” ($`h^S>0`$) in absence of “temporal chaos” ($`h^T=0`$), while the existence of “temporal chaos” requires not only a non zero spatial entropy but also the existence of a finite propagation velocity. This confirms somehow that the classifications of a CA as chaotic in terms of damage spreading velocity and entropy are related to each others.

However, as stressed by Oono and Kohomoto , the seemingly natural assumption of calling “turbulent” a cellular automaton for which one has $`h^S>0`$ and $`h^T>0`$ is not correct in general. This is particularly clear by considering a single direction shift imposed on a “frozen” disordered background. Nevertheless, in spite if this specific counterexample, the attempts based on entropic concepts, for the characterization of the irregular spatial and/or temporal behavior of systems with discrete states, in our opinion, are the most promising ones. In this context Casartelli and coworkers  introduced the concept of rational partitions in order to define a complexity measure for systems which can be reduced to $`1d`$ CA.

Let us conclude this Section with a brief discussion and comparison between the unpredictability which characterizes cellular automata evolution with respect to the one encountered in the context of continuous states dynamics, e.g. in coupled map lattices (see Sect. 4). The latter indeed seems to be the natural candidate for such a comparison. We limit the discussion to 1-dimensional lattices with $`r=1`$, i.e. CML and CA with nearest neighbor coupling.

Let us now ask the amount of information we have to specify for knowing all the $`LT`$ sites of spatial size $`L(<N)`$ and temporal length $`T`$, as shown in Fig. 37. Since both CA and CML are ruled by a local deterministic dynamics one needs to specify the rule of evolution and the values of the $`L+2(T1)`$ states at the border, in black in Figure 37. Basically, one has to specify the initial conditions on the $`L`$ sites and the “boundaries” $`\sigma _1(t)`$ and $`\sigma _L(t)`$ for $`1<tT`$. But while for CA this specification unambiguously determines the $`LT`$ values, for a chaotic CML this is not enough. Indeed, one has to specify the precision, $`ϵ`$, with which he wants to know the $`LT`$ values. Once specified $`ϵ`$, one knows the necessary initial precision, $`ϵ_0`$, on the $`L+2(T1)`$ sites in black. A conservative estimate gives $`ϵ_0ϵ\mathrm{exp}(LT_{KS})`$, where $`_{KS}`$ is the entropy density defined in Eq. (4.3). This very simple argument suggests that the main difference between CA and continuous systems is the absence of “local” production of information, i.e. in CA the complexity only arises by the spatial propagation of information . Nevertheless there exist counterexamples in which starting from simple initial configuration complex pattern are generated .

From this point of view it is interesting to consider the behavior of certain CMLs which, in spite of their continuous nature, seem to be rather similar to “chaotic” cellular automata. Indeed, it has been found that a class of stable (i.e. $`\lambda <0`$) CMLs  displays an unpredictable dynamics on times exponentially large with the system size. So that in the limit of infinite lattices they are completely unpredictable. Moreover, these CMLs have a finite velocity of propagation for initially localized disturbances (provided that the value of the disturbance was $`O(1)`$) . Recalling the discussion of Sect. 4.6, we know that this cannot be predicted in terms of the comoving Lyapunov exponents, it is a fully non linear phenomenon. The strong analogies with “chaotic” CA have been furtherly explored in Ref. , where it has been proposed to classify these CML as large memory cellular automata according to the behavior of their spatial and temporal entropy.

## 8 The characterization of the Complexity and system modeling

In the previous Sections we discussed the characterization of dynamical behaviors when the evolution laws are known either exactly or with an uncertainty. On the other hand in experimental investigations only time records of some observable are typically available, and the equation of motions are not known. For the predictability problem, this latter case, at least from a conceptual point of view, can be treated in the same framework of when the evolution laws are known. Indeed, in principle, with the embedding technique one can reconstruct the phase space . Nevertheless there are rather severe limitations in high dimensional systems  and even in low dimensional ones non trivial features appear in presence of noise .

In this Section we show that an entropic analysis at different resolution scales allows us for a pragmatic classification of a signal and gives suggestions for modeling of systems. In particular we illustrate, using some examples, how quantities such as the $`ϵ`$-entropy or the FSLE can display a subtle transition from the large to the small scales. A negative consequence of this is the difficulty in distinguishing, only from data analysis, a genuine deterministic chaotic system from one with intrinsic randomness . On the other hand, the way the $`ϵ`$-entropy or Finite Size Lyapunov Exponent depends on the (resolution) scale, allows us for a pragmatic classification of the stochastic or chaotic character of the signal, and this gives some freedom in modeling the system.

### 8.1 How random is a random number generator?

It is rather natural to wonder about the “true character” of the number sequence $`(x_1,x_2,\mathrm{})`$ obtained with a (pseudo) random number generator (PRNG) on a computer. One would like to have a sequence with a random character; on the other hand, one is forced to use deterministic algorithms to generate $`(x_1,x_2,\mathrm{})`$. This subsection is mainly based on the paper . A simple and popular PRNG is the multiplicative congruent one :

$$\begin{array}{cc}z_{n+1}=\hfill & N_1z_n\text{mod}N_2\hfill \\ x_{n+1}=\hfill & z_{n+1}/N_2,\hfill \end{array}$$

with an integer multiplier $`N_1`$ and modulus $`N_2`$. The $`\{z_n\}`$ are integer numbers and one hopes to generate a sequence of random variables $`\{x_n\}`$, which are uncorrelated and uniformly distributed in the unit interval. A first problem one has to face is the periodic nature of (8.1), because of its discrete character (see Sect. 7). In practice one wants to fix $`N_1`$ and $`N_2`$ in such a way to maximize this period. Note that the rule (8.1) can be interpreted as a deterministic dynamical system, i.e.

$$x_{n+1}=N_1x_n\text{mod}\mathrm{\hspace{0.25em}\hspace{0.25em}1},$$
(8.1)
which has a uniform invariant measure and a KS entropy $`h_{KS}=\lambda =\mathrm{ln}N_1`$. When imposing the integer arithmetics of Eq. (8.1) onto this system, we are, in the language of dynamical systems, considering an unstable periodic orbit of Eq. (8.1), with the particular constraint that, in order to achieve the period $`N_21`$ (i.e. all integers $`<N_2`$ should belong to the orbit of Eq. (8.1)) it has to contain all values $`k/N_2`$, with $`k=1,2,\mathrm{},N_21`$. Since the natural invariant measure of Eq. (8.1) is uniform, such an orbit represents the measure of a chaotic solution in an optimal way. Every sequence of a PRNG is characterized by two quantities: its period $`𝒯`$ and its positive Lyapunov exponent $`\lambda `$, which is identical to the entropy of a chaotic orbit of the equivalent dynamical system. Of course a good random number generator has a very large period, and as large as possible entropy.

It is natural to ask how this apparent randomness can be reconciled with the facts that (a) the PRNG is a deterministic dynamical systems (b) it is a discrete state system.

If the period is long enough on shorter times one has to face only point (a). In the following we discuss this point in terms of the behavior of the $`ϵ`$-entropy, $`h(ϵ)`$ (see Sect. 3.5). It seems rather reasonable to think that at a high resolution, i.e. $`ϵ1/N_1`$, one should realize the true deterministic chaotic nature of the system and, therefore, $`h(ϵ)h_{KS}=\mathrm{ln}N_1`$. On the other hand for $`ϵ1/N_1`$ one expects to observe the “apparent random” behavior of the system, i.e. $`h(ϵ)\mathrm{ln}(1/ϵ)`$.

When the spatial resolution is high enough so that every point of this periodic orbit is characterized by its own symbol, then, for arbitrary block length $`m`$, one has a finite number of $`m`$-words whose probabilities are different from $`0`$. Therefore, the block entropy $`H_m`$ (3.42) is $`m`$-independent and $`h_m=0`$.

In Fig. 38 it is shown the behavior of $`h_m(ϵ)`$, computed on sequences of length 60000 of the PRNG with three different pairs $`(N_1,N_2)`$ chosen to be ($`7^5`$,$`2^{32}`$), (2,494539), and (107,31771). The first one is optimal and no deviation from the stochastic behavior is visible. The second one has a small pseudo-entropy, and this is seen by the saturation of all $`h_m(ϵ)`$ at $`\mathrm{ln}N_1=\mathrm{ln}2`$, and the last one has large entropy but a rather short period, so that all $`h_m(ϵ)`$ drop to zero for some $`ϵ_m`$, where $`ϵ_m`$ becomes dramatically larger for increasing $`m`$ (strong fluctuations arise from the fact that data are confined to a grid of spacing 1/31771).

### 8.2 High dimensional systems

Now we discuss high-dimensional systems that show non-trivial behavior at varying the resolution scales. Olbrich et al.  analyzed an open flow system described by unidirectionally coupled map lattice:

$$x_j(t+1)=(1\sigma )f(x_{j+1}(t))+\sigma x_j(t)$$
(8.2)
where $`j=1,\mathrm{},N`$ denotes the site of a lattice of size $`N`$, $`t`$ the discrete time and $`\sigma `$ the coupling strength. A detailed numerical study (also supported by analytical arguments) of the $`ϵ`$-entropy $`h_m(ϵ)`$ at different $`ϵ`$, in the limit of small coupling, gives the following scale-dependent scenario: for $`1ϵ\sigma `$ there is a plateau $`h(ϵ)\lambda _s`$ where $`\lambda _s`$ is the Lyapunov exponent of the single map $`x(t+1)=f(x(t))`$. For $`\sigma ϵ\sigma ^2`$ another plateau appears at $`h(ϵ)2\lambda _s`$, and so on: for $`\sigma ^{n1}ϵ\sigma ^n`$ one has $`h(ϵ)n\lambda _s`$ (see Fig. 39).

Similar results hold for the correlation dimension which increases step by step as the resolution increases, showing that the high-dimensionality of the system becomes evident only as $`ϵ0`$. Therefore one understands that the dynamics at different scales is basically ruled by a hierarchy of low-dimensional systems whose “effective” dimension $`n_{eff}(ϵ)`$ increases as $`ϵ`$ decreases :

$$n_{eff}(ϵ)\left[\frac{\mathrm{ln}(1/ϵ)}{\mathrm{ln}(1/\sigma )}\right],$$
(8.3)
where $`[\mathrm{}]`$ indicates the integer part. In addition, for a given resolution $`ϵ`$, it is possible to find a suitable low-dimensional noisy system (depending on $`ϵ`$) which is able to mimic $`x_1(t)`$ given by Eq. (8.2). It is interesting to note that, on an extended range of values of $`ϵ`$ ($`ϵ\sigma ^N`$), $`h(ϵ)`$ can be roughly approximated as log-periodic fluctuations around

$$h(ϵ)\mathrm{ln}\frac{1}{ϵ}$$
(8.4)
i.e. the typical behavior of a stochastic process. Of course for $`ϵ\sigma ^N`$ one has to realize that the system is deterministic and $`h(ϵ)=O(N\lambda _s)`$.

Let us now briefly reconsider the issue of the macroscopic chaos, discussed in Sect. 4.7. The main result can be summarized as follows:

* at small $`ϵ(1/\sqrt{N})`$, where $`N`$ is the number of elements, one recovers the “microscopic” Lyapunov exponent222From hereafter we use the same symbol $`ϵ`$ both for the FSLE and the $`ϵ`$-entropy in order to make a direct comparison between the two quantities, i.e. $`\lambda (ϵ)\lambda _{micro}`$
* at large $`ϵ(1/\sqrt{N})`$ one observes another plateau $`\lambda (ϵ)\lambda _{macro}`$ which can be much smaller than the microscopic one.

The emerging scenario is that at a coarse-grained level, i.e. $`ϵ1/\sqrt{N}`$, the system can be described by an “effective” hydro-dynamical equation (which in some cases can be low-dimensional), while the “true” high-dimensional character appears only at very high resolution, i.e.

$$ϵϵ_c=O\left(\frac{1}{\sqrt{N}}\right).$$

### 8.3 Diffusion in deterministic systems and Brownian motion

Consider the following map which generates a diffusive behavior on the large scales :

$$x_{t+1}=[x_t]+F\left(x_t[x_t]\right),$$
(8.5)
where $`[x_t]`$ indicates the integer part of $`x_t`$ and $`F(y)`$ is given by:

$$F(y)=\{\begin{array}{cc}(2+\alpha )y\hfill & \text{if}y[0,1/2[\hfill \\ (2+\alpha )y(1+\alpha )\hfill & \text{if}y[1/2,1].\hfill \end{array}$$

The largest Lyapunov exponent $`\lambda `$ can be obtained immediately: $`\lambda =\mathrm{ln}|F^{}|`$, with $`F^{}=dF/dy=2+\alpha `$. One expects the following scenario for $`h(ϵ)`$:

$$h(ϵ)\lambda \mathrm{for}ϵ<1,$$
(8.6)
$$h(ϵ)\frac{D}{ϵ^2}\mathrm{for}ϵ>1,$$
(8.7)
where $`D`$ is the diffusion coefficient, i.e.

$$\left(x_tx_0\right)^22Dt\mathrm{for}\mathrm{large}t.$$
(8.8)
Consider now a stochastic system, namely a noisy map

$$x_{t+1}=[x_t]+G\left(x_t[x_t]\right)+\sigma \eta _t,$$
(8.9)
where $`G(y)`$, as shown in Fig. 40, is a piece wise linear map which approximates the map $`F(y)`$, and $`\eta _t`$ is a stochastic process uniformly distributed in the interval $`[1,1]`$, and no correlation in time. When $`|dG/dy|<1`$, as is the case we consider, the map (8.9), in the absence of noise, gives a non-chaotic time evolution.

Now we compare the finite size Lyapunov exponent for the chaotic map (8.5) and for the noisy one (8.9). In the latter the FSLE has been computed using two different realizations of the noise. In Fig. 41 we show $`\lambda (ϵ)`$ versus $`ϵ`$ for the two cases. The two curves are practically indistinguishable in the region $`ϵ>\sigma `$. The differences appear only at very small scales $`ϵ<\sigma `$ where one has a $`\lambda (ϵ)`$ which grows with $`ϵ`$ for the noisy case, remaining at the same value for the chaotic deterministic case.

Both the FSLE and the $`(ϵ,\tau )`$-entropy analysis show that we can distinguish three different regimes observing the dynamics of (8.9) on different length scales. On the large length scales $`ϵ>1`$ we observe diffusive behavior in both models. On length scales $`\sigma <ϵ<1`$ both models show chaotic deterministic behavior, because the entropy and the FSLE are independent of $`ϵ`$ and larger than zero. Finally on the smallest length scales $`ϵ<\sigma `$ we see stochastic behavior for the system (8.9) while the system (8.5) still shows chaotic behavior.

### 8.4 On the distinction between chaos and noise

The above examples show that the distinction between chaos and noise can be a high non trivial task, which makes sense only in very peculiar cases, e.g., very low dimensional systems. Nevertheless, even in this case, the entropic analysis can be unable to recognize the “true” character of the system due to the lack of resolution. Again, the comparison between the diffusive map (8.5) and the noisy map (8.9) is an example of these difficulties. For $`\sigma ϵ1`$ both the system (8.5) and (8.9), in spite of their “true” character, will be classified as chaotic, while for $`ϵ1`$ both can be considered as stochastic.

In high-dimensional chaotic systems, with $`N`$ degrees of freedom, one has typically $`h(ϵ)=h_{KS}O(N)`$ for $`ϵϵ_c`$ (where $`ϵ_c0`$ as $`N\mathrm{}`$) while for $`ϵϵ_c`$, $`h(ϵ)`$ decreases, often with a power law . Since also in some stochastic processes the $`ϵ`$-entropy obeys a power law, this can be a source of confusion.

These kind of problems are not abstract ones, as a recent debate on “microscopic chaos” demonstrates . The detection of microscopic chaos by data analysis has been recently addressed in a work of Gaspard et al. . These authors, from an entropic analysis of an ingenious experiment on the position of a Brownian particle in a liquid, claim to give an empirical evidence for microscopic chaos. In other words, they state that the diffusive behavior observed for a Brownian particle is the consequence of chaos at a molecular level. Their work can be briefly summarized as follows: from a long ($`1.5\times 10^5`$ data) record of the position of a Brownian particle they compute the $`ϵ`$-entropy with the Cohen-Procaccia method  (Sect. 3) from which they obtain:

$$h(ϵ)\frac{D}{ϵ^2},$$
(8.10)
where $`D`$ is the diffusion coefficient. Then, assuming that the system is deterministic, and making use of the inequality $`h(ϵ>0)h_{KS}`$, they conclude that the system is chaotic. However, their result does not give a direct evidence that the system is deterministic and chaotic. Indeed, the power law (8.10) can be produced with different mechanisms:

1. a genuine chaotic system with diffusive behavior, as the map (8.3);
2. a non chaotic system with some noise, as the map (8.9), or a genuine Brownian system;
3. a deterministic linear non chaotic system with many degrees of freedom (see for instance );
4. a “complicated” non chaotic system as the Ehrenfest wind-tree model where a particle diffuses in a plane due to collisions with randomly placed, fixed oriented square scatters, as discussed by Cohen et al.  in their comment to Ref. .

It seems to us that the weak points of the analysis in Ref.  are:

a) the explicit assumption that the system is deterministic;

b) the limited number of data points and therefore limitations in both resolution and block length.

The point (a) is crucial, without this assumption (even with an enormous data set) it is not possible to distinguish between 1) and 2). One has to say that in the cases 3) and 4) at least in principle it is possible to understand that the systems are “trivial” (i.e. not chaotic) but for this one has to use a huge number of data. For example Cohen et al.  estimated that in order to distinguish between 1) and 4) using realistic parameters of a typical liquid, the number of data points required has to be at least $`10^{34}`$.

Concluding, we have the apparently paradoxical result that “complexity” helps in the construction of models. Basically, in the case in which one has a variety of behaviors at varying the scale resolution, there is a certain freedom on the choice of the model to adopt. In Sect. 8.3 one can see that, for some systems, the behavior at large scales can be realized both with chaotic deterministic models or suitable stochastic processes. From a pragmatic point of view, the fact that in certain stochastic processes $`h(ϵ)ϵ^\alpha `$ can be indeed extremely useful for modeling such high-dimensional systems. Perhaps, the most relevant case in which one can use this freedom in modeling is the fully developed turbulence whose non infinitesimal (the so-called inertial range) properties can be successfully mimicked in terms of multiaffine stochastic process (see Ref.  Sect. 5.5 and Appendix D).

## 9 Concluding Remarks

The guideline of this review has been how to interpret the different aspects of the predictability of a system as a way to characterize its complexity.

We have discussed the relation between the Kolmogorov-Sinai entropy and the algorithmic complexity (Sect. 2). As clearly exposed in the seminal works of Alekseev and Yakobson  and Ford , the time sequences generated by a system with sensitive dependence on initial conditions have non-zero algorithmic complexity. A relation exists between the maximal compression of a sequence and its KS-entropy. Therefore, one can give a definition of complexity, without referring to a specific description, as an intrinsic property of the system.

In presence of intrinsic randomness (Sect. 6.3), one can introduce two different Lyapunov exponents, $`\lambda _\sigma `$ in the case of trajectories with the same realization of noise and $`K_\sigma `$ for different realizations. In general $`\lambda _\sigma `$ and $`K_\sigma `$ do not coincide and characterize different aspects of the system. Both quantities have their own relevance, the comparison between $`\lambda _\sigma `$ and $`K_\sigma `$ has shown to be useful in the understanding of apparently intricate phenomena, such as noise-induced order and noise-induced instability.

As an example of system with many degrees of freedom and characteristic times scales, we investigated fully developed turbulence (Sect. 5). In this case the Lyapunov exponent and the KS-entropy are somehow of limited relevance because they only characterize small scales properties. On the other hand there exist suitable generalizations – the finite size Lyapunov exponent, $`\lambda (ϵ)`$, and $`ϵ`$-entropy, $`h(ϵ)`$ – which characterize the predictability properties at different scales. The scaling of the predictability time with the resolution $`ϵ`$, $`\lambda (ϵ)ϵ^2`$, has an algorithmic correspondence in the behavior of the $`ϵ`$-entropy of the signal measured in one point, $`h(ϵ)ϵ^3`$. In the words of Lorenz, one can say that the butterfly effect is not so terrible for $`ϵ`$-resolution in the inertial range.

Complexity in a system can also manifest in the spatial properties as, for example, in open flows with convective chaos but with negative Lyapunov exponents (Sect. 4). The presence of convective chaos implies a sensitivity on the boundary conditions. An uncertainty, $`\delta x_0`$, on the boundary condition is exponentially amplified with the distance, $`n`$, from the boundary as $`\delta x_n\delta x_0e^{\mathrm{\Gamma }n}`$. The “spatial” Lyapunov exponent $`\mathrm{\Gamma }`$ is related with the comoving Lyapunov exponent and gives a characterization of the spatial “complexity”.

The study of these different aspects of predictability constitutes a useful method for a quantitative characterization of “complexity”, suggesting the following equivalences:

The above point of view, based on dynamical systems and information theory, quantifies the complexity of a sequence considering each symbol relevant but it does not capture the structural level. Let us clarify this point with the following example. A binary sequence obtained with a coin tossing is, from the point of view adopted in this review, complex since it cannot be compressed (i.e. it is unpredictable). On the other hand such a sequence is somehow trivial, i.e. with low “organizational” complexity. It would be important to introduce a quantitative measure of this intuitive idea. The progresses of the research on this intriguing and difficult issue are still rather slow. We just mention some of the most promising proposals as the logical depth  and the sophistication , see Ref. .

As a final question one can wonder what can one learn by the presented material for practical prediction problems (e.g. weather forecast). The main lesson concerns the framework and limitations about the possible well posed questions in prediction and modeling. The first relevant fact, now well established, is that in presence of deterministic chaos one cannot hope to reach an arbitrary accuracy in prediction by merely refining the model. A less recognized aspect is that the Lyapunov exponent is usually not sufficient to characterize the limits of predictability in real situations. An appropriate generalization of the Lyapunov exponent is necessary to account for the large scale properties. Moreover in weather forecast the predictability time, which is typically of $`5`$ days, may be as little as $`2`$ days or as much as $`10`$ days . Thus, simply quoting an average value does not give a satisfactory answer. At a more conceptual level, one has severe limitations in distinguish between deterministic or stochastic nature of systems displaying complex behavior. This implies a certain freedom in the choice of the details of the model, in particular whether to adopt a deterministic or a stochastic model.

We are particularly grateful to A. Celani, for the long collaboration on the subject here discussed and for his contributions at a first stage of this review. We thank M. Abel, K.H. Andersen, A. Baldassarri, E. Aurell, F. Bagnoli, R. Baviera, A. Bigazzi, R. Benzi, L. Biferale, M. Casartelli, P. Castiglione, A. Crisanti, D. P. Eckmann, U. Frisch, P. Grassberger, K. Kaneko, H. Kantz, M.H. Jensen, G. Lacorata, R. Livi, V. Loreto, R. Mantegna, G. Mantica, U. Marini Bettolo Marconi, A. Mazzino, P. Muratore-Ginanneschi, E. Olbrich, G. Parisi, R. Pasmanter, M. Pasquini, A. Pikovsky, A. Politi, A. Provenzale, A. Puglisi, S. Ruffo, M. Serva, A. Torcini, M. Vergassola, D. Vergni and E. Zambianchi for collaborations, correspondence and discussions in the last years. We are grateful to F. Benatti, P. Castiglione, A. Politi and A. Torcini for useful remarks and detailed comments on the manuscript. We acknowledge F. di Carmine, R. T. Lampioni, B. Marani and I. Poeta for their friendly and warm encouragement.

## Appendix A On the computation of the Finite Size Lyapunov Exponent

This Appendix is devoted to the methods for computing the Finite Size Lyapunov Exponent. As we will see there are mainly three possible ways to compute the FSLE. Let us start with a modification of the standard technique for computing the largest Lyapunov exponent.

Suppose to integrate on a computer the equations of motion of a system. After a long integration time, in order for the motion to settle onto the attractor of the system, we introduce a very small perturbation, i.e. we consider the “reference” trajectory $`𝐱(0)`$, which is supposed to be on the attractor, and generate a “perturbed” trajectory starting from $`𝐱^{^{}}(0)=𝐱(0)+\delta 𝐱(0)`$. We need the perturbation to be initially very small in some chosen norm $`\delta (t=0)=\delta 𝐱(t=0)=\delta _{min}1`$. Then, in order to study the perturbation growth through different scales, one defines a set of thresholds $`\delta _n`$, e.g.: $`\delta _n=\delta _0r^{(n)}`$ with $`1\delta _0\delta _{min}`$ and $`n=0,\mathrm{},N`$. To avoid saturation on the maximum allowed separation (i.e. the attractor size) one has to choose $`\delta _N<𝐱𝐲_\mu `$ with $`𝐱,𝐲`$ generic points on the attractor. Note that $`r`$ should be larger than $`1`$ but not too large in order to avoid interferences of different length scales. Typically, one chooses $`r=2`$ or $`r=\sqrt{2}`$.

In order to measure the perturbation growth rate at scale $`\delta _n`$, one lets the system to evolve from $`\delta _{min}`$ up to the desired scale $`\delta _n`$ ensuring the perturbation to be on the attractor and aligned along the maximally expanding direction. After $`\delta _n`$ is reached, one computes the first time, $`\tau _1(\delta _n,r)`$, to reach the following threshold, $`\delta _{n+1}`$, and after that the perturbation is rescaled to $`\delta _n`$, keeping the direction $`𝐱^{}𝐱`$ constant. This procedure is repeated $`𝒩`$ times for each thresholds obtaining the set of the doubling times $`\{\tau _i(\delta _n,r)\}`$ for $`i=1,\mathrm{},𝒩`$ error-doubling experiments. Now if we introduce the effective doubling rates:

$$\gamma _i(\delta _n,r)=\frac{1}{\tau _i(\delta _n,r)}\mathrm{ln}r,$$
(1.1)
we can define their time averages as the the effective LEs on the scale $`\delta _n`$. Therefore, we have

$$\lambda (\delta _n)=\gamma (\delta _n,r)_t=\frac{1}{T}_0^T\gamma 𝑑t=\frac{\underset{i}{}\gamma _i\tau _i}{_i\tau _i}=\frac{1}{\tau (\delta _n,r)_e}\mathrm{ln}r.$$
(1.2)
where $`\tau (\delta _n,r)_e=\tau _i/𝒩`$ and $`T=_i\tau _i`$.

To obtain Eq. (1.2) we assumed the distance between the two trajectories to be continuous in time. This is not true for maps or for discrete sampling in time and the method has to be slightly modified. In this case the doubling time, $`\tau (\delta _n,r)`$, is defined as the minimum time such that $`\delta (\tau )r\delta _n`$. Because now $`\delta (\tau )`$ is a fluctuating quantity, from (1.2) we have

$$\lambda (\delta _n)=\frac{1}{\tau (\delta _n,r)_e}\mathrm{ln}\left(\frac{\delta (\tau (\delta _n,r))}{\delta _n}\right)_e.$$
(1.3)
Let us stress some points.

The computation of the FSLE is not more expensive than the one of the Lyapunov exponent by standard algorithm. One has simply to integrate two copies of the system (or two different systems for second kind predictability) and this can be done without particular problems.

At difference with $`\lambda `$, $`\lambda (\delta )`$ may also depend on the norm one chooses. This fact, apparently disturbing, is however physically reasonable: when one looks at the non linear regime, for instance, for the predictability problem the answer may depend on the involved observable. A similar problem appears in infinite dimensional system where the norms are not equivalent .

A possible problem with the above described method is that we have implicitly assumed that the statistically stationary state of the system is homogeneous with respect to finite perturbations. Actually one may plausibly expect the attractor to be fractal, i.e., not at all equally dense at all distances, this may cause an incorrect sampling of the doubling times at large $`\delta _n`$.

A possible way to overcome such a problem is to compute the FSLE avoiding to rescale the perturbation at finite $`\delta _n`$. This can be accomplished by the following modification of the previous method. One defines the thresholds $`\{\delta _n\}`$ and initializes the perturbation at $`\delta _{min}\delta _0`$ as before. Then one lets the system to reach the first threshold, $`\delta _0`$. Hence, one starts to measure the doubling time $`\tau (\delta _n,r)`$ following the perturbation growth from $`\delta _0`$ up to $`\delta _N`$. In practice, one register the time $`\tau (\delta _n,r)`$ for going from $`\delta _n`$ to $`\delta _{n+1}`$ for each value of $`n`$. The evolution of the error from the initial value $`\delta _{min}`$ to the largest threshold $`\delta _N`$ carries out a single error-doubling experiment. When the largest threshold, $`\delta _N`$ has been reached the “perturbed” trajectory is rescaled at the initial distance, $`\delta _{min}`$, with respect to the “reference” trajectory and one starts another experiment measuring a second set of doubling times, $`\{\tau _2(\delta _n,r)\}`$. The procedure is then repeated $`𝒩`$ times to have statistics. In this way one obtains the set of the doubling times $`\{\tau _i(\delta _n,r)\}`$ for $`i=1,\mathrm{},𝒩`$. The FSLE is finally obtained by using Eq. (1.2) or Eq. (1.3), which are accurate also in this case, according to the continuous time and discrete time nature of the system respectively. One understands that with this method, since finite perturbations are realized by the dynamics (i.e. the perturbed trajectory is on the attractor) and not imposed by hand, the problems related to the attractor inhomogeneity are not present.

In any case, in most numerical experiments, one does not find significant differences between the two numerical methods.

A further possibility to compute the FSLE is to remove the threshold condition used for defining $`\tau (\delta _n,r)`$ and simply compute the average error growth rate at every time step. In other words, at every time step $`\mathrm{\Delta }t`$ in the integration, the perturbed trajectory $`𝐱^{}(t)`$ is rescaled to the original distance $`\delta `$, keeping the direction $`𝐱𝐱^{}`$ constant. The FSLE is then obtained by the average of the one-step exponential divergence:

$$\lambda (\delta )=\frac{1}{\mathrm{\Delta }t}\mathrm{ln}\left(\frac{\delta 𝐱(t+\mathrm{\Delta }t)}{\delta 𝐱(t)}\right)_t,$$
(1.4)
which, if non negative, is equivalent to the definition (1.2). Let us note that the above procedure is nothing but the finite scale version of the usual algorithm of Benettin et al.  for the LE. The one-step method (1.4) can be, in principle, generalized to compute the sub-leading finite-size Lyapunov exponent following the standard ortho-normalization method . One introduces $`k`$ perturbed trajectories $`𝐱^{(1)},\mathrm{},𝐱^{(k)}`$ each at distance $`\delta `$ from $`𝐱`$ and such that $`𝐱^{(k)}𝐱`$ are orthogonal each to the others. At every time step, any difference $`𝐱^{(k)}𝐱`$ is rescaled at the original value and orthogonalized, while the corresponding finite size Lyapunov exponent is accumulated according to (1.4).

Here we have again the problem of the implicitly assumed homogeneity of the attractor, but also a problem of isotropy when we re-orthogonalize the perturbations. We note that this could be a more serious problem.

## Appendix B The multifractal model of turbulence

The multifractal model of turbulence  assumes that the velocity has a local scale-invariance, i.e. it does not have a unique scaling exponent $`h`$ such that $`\delta v_{\mathrm{}}\mathrm{}^h`$, but a continuous spectrum of exponents, each of which belonging to a given fractal set. In other words, in the inertial range one has

$$\delta v_{\mathrm{}}(x)\mathrm{}^h,$$
(2.1)
if $`xS_h`$, and $`S_h`$ is a fractal set with dimension $`D(h)`$ and $`h`$ ($`h_{min}`$, $`h_{max}`$). The probability to observe a given scaling exponent $`h`$ at the scale $`\mathrm{}`$ is thus $`P_{\mathrm{}}(h)\mathrm{}^{3D(h)}`$, so the scaling of the structure function assumes the form:

$$S_p(\mathrm{})=\delta v_{\mathrm{}}^p_{h_{min}}^{h_{max}}\mathrm{}^{hp}\mathrm{}^{3D(h)}dh\mathrm{}^{\zeta _p},$$
(2.2)
where in the last equality, being $`\mathrm{}1`$, a steepest descent estimation gives

$$\zeta _p=\underset{h}{\mathrm{min}}\left\{hp+3D(h)\right\}=h^{}p+3D(h^{})$$
(2.3)
where $`h^{}=h^{}(p)`$ is the solution of the equation $`D^{}(h^{}(p))=p`$. The Kolmogorov “4/5” law

$$S_3(\mathrm{})=\frac{4}{5}ϵ\mathrm{}$$
(2.4)
imposes $`\zeta _3=1`$ which implies that

$$3h+2D(h)0,$$
(2.5)
the equality is realized by $`h^{}(3)`$. The Kolmogorov similarity theory corresponds to the case of only one singularity exponent $`h=1/3`$ with $`D(h=1/3)=3`$.

A nontrivial consequence of the intermittency in the turbulent cascade is the fluctuations of the dissipative scale which implies the existence of an intermediate region between the inertial and dissipative range . The local dissipative scale $`\mathrm{}_D`$ is determined by imposing the effective Reynolds number to be of order unity:

$$Re(\mathrm{}_D)=\frac{\delta v_D\mathrm{}_D}{\nu }1,$$
(2.6)
therefore the dependence of $`\mathrm{}_D`$ on $`h`$ is thus

$$\mathrm{}_D(h)LRe^{\frac{1}{1+h}}$$
(2.7)
where $`Re=Re(L)`$ is the large scale Reynolds number. The fluctuations of $`\mathrm{}_D`$ modifies the evaluation of the structure functions (2.2): for a given $`\mathrm{}`$, the saddle point evaluation of (2.2) remains unchanged if, for the selected exponent $`h^{}(p)`$, one has $`\mathrm{}_D(h^{}(p))<\mathrm{}`$. If, on the contrary, the selected exponent is such that $`\mathrm{}_D(h^{}(p))>\mathrm{}`$ the saddle point evaluation is not consistent, because at scale $`\mathrm{}`$ the power–law scaling (2.1) is no longer valid. In this intermediate dissipation range  the integral in (2.2) is dominated by the smallest acceptable scaling exponent $`h(\mathrm{})`$ given by inverting (2.7), and the structure function of order $`p`$ a pseudo–algebraic behavior, i.e. a power law with exponent $`ph(\mathrm{})+3D(h(\mathrm{}))`$ which depends on the scale $`\mathrm{}`$. Taking into account the fluctuations of the dissipative range, one has for the structure functions

$$S_p(\mathrm{})\{\begin{array}{ccc}\mathrm{}^{\zeta _p}\hfill & \text{if}\hfill & \mathrm{}_D(h^{}(p))<\mathrm{}\hfill \\ \mathrm{}^{h(\mathrm{})p+3D(h(\mathrm{}))}\hfill & \text{if}\hfill & \mathrm{}_D(h_{min}<\mathrm{}<\mathrm{}_D(h^{}(p)).\hfill \end{array}$$
(2.8)
A simple calculation  shows that it is possible to find a universal description valid both in the inertial and in the intermediate dissipative ranges. Let us discuss this point for the energy spectrum $`E(k)`$. Introducing the rescaled variables

$$F(\theta )=\frac{\mathrm{ln}E(k)}{\mathrm{ln}Re}\text{and}\theta =\frac{\mathrm{ln}k}{\mathrm{ln}Re}$$
(2.9)
one obtains the following behavior

$$F(\theta )=\{\begin{array}{ccc}(1+\zeta _2)\theta \hfill & \text{for}\hfill & \theta <\frac{1}{1+h^{}(2)}\hfill \\ 22\theta +\theta D(\theta ^11)\hfill & \text{for}\hfill & \frac{1}{1+h^{}(2)}<\theta <\frac{1}{1+h_{min}}\hfill \end{array}$$
(2.10)
The prediction of the multifractal model is that $`\mathrm{ln}E(k)/\mathrm{ln}Re`$ is an universal function of $`\mathrm{ln}k/\mathrm{ln}Re`$. This is in contrast with the usual scaling hypothesis according which $`\mathrm{ln}E(k)`$ should be a universal function of $`\mathrm{ln}(k/k_D)`$). The multifractal universality has been tested by collapsing energy spectra obtained from turbulent flow in a wide range of $`Re`$ , see also .

## Appendix C How to compute the $`ϵ`$-entropy with exit times

The approach based on exit times differs from the usual one (see Sect. 3.5) in the procedure to construct the coding sequence of the signal at a given level of accuracy . Indeed an efficient coding procedure reduces the redundancy and improves the quality of the results. The method here discussed is particularly suited for computing the $`ϵ`$-entropy in processes in which many scales are excited as, e.g., in turbulence .

The coding of a signal, $`x(t)`$, by the exit-time approach is the following. Given a reference starting time $`t=t_0`$, one measures the first exit-time, $`t_1`$, from a cell of size $`ϵ`$, i.e. the first time necessary to have $`|x(t_0+t_1)x(t_0)|ϵ/2`$. Then one restarts from the time $`t=t_0+t_1`$ to look for the next exit-time $`t_2`$, i.e., the first time such that $`|x(t_0+t_1+t_2)x(t_0+t_1)|ϵ/2`$ and so on. Finally one obtains a sequence of exit-times, $`\{t_i(ϵ)\}`$, and one also records the labels $`k_i=\pm 1`$, which distinguish the direction of the exit (up or down out of a cell).

At the end of this construction, the trajectory is coded without ambiguity, with the required accuracy $`ϵ`$, by the sequence $`\{(t_i,k_i),i=1,\mathrm{},M\}`$, where $`M`$ is the total number of exit-time events observed during the total time $`T`$. Now, performing a coarse-graining of the possible values assumed by $`t(ϵ)`$ with a resolution time $`\tau _r`$, we accomplish the goal of obtaining a symbolic sequence. One now studies the “exit-time words”, $`\mathrm{\Omega }_i^N`$, of various lengths $`n`$: $`\mathrm{\Omega }_i^N(ϵ,\tau _r)=((\eta _i,k_i),(\eta _{i+1},k_{i+1}),\mathrm{},(\eta _{i+N1},k_{i+N1}))`$, where $`\eta _j`$ labels the cell (of width $`\tau _r`$) containing the exit-time $`t_j`$. From the probabilities of these words one evaluates the block entropies (2.18) at the given time resolution, $`H_N^\mathrm{\Omega }(ϵ,\tau _r)`$, and then the exit-time $`(ϵ,\tau _r)`$-entropies:

$$h^\mathrm{\Omega }(ϵ,\tau _r)=\underset{N\mathrm{}}{lim}H_{N+1}^\mathrm{\Omega }(ϵ,\tau _r)H_N^\mathrm{\Omega }(ϵ,\tau _r).$$
(3.1)
The limit of infinite time-resolution gives us the $`ϵ`$-entropy per exit, i.e.:

$$h^\mathrm{\Omega }(ϵ)=\underset{\tau _r0}{lim}h^\mathrm{\Omega }(ϵ,\tau _r).$$
(3.2)
The link between $`h^\mathrm{\Omega }(ϵ)`$ and the $`ϵ`$-entropy (3.43) can be obtained as follows. We note that there is a one-to-one correspondence between the (exit-time)-histories and the $`(ϵ,\tau )`$-histories (in the limit $`\tau 0`$) originating from a given $`ϵ`$-cell. The Shannon-McMillan theorem  assures that the number of the typical $`(ϵ,\tau )`$-histories of length $`N`$, $`𝒩(ϵ,N)`$, is such that: $`\mathrm{ln}𝒩(ϵ,N)h(ϵ)N\tau =h(ϵ)T`$. For the number of typical (exit-time)-histories of length $`M`$, $`(ϵ,M)`$, we have: $`\mathrm{ln}(ϵ,M)h^\mathrm{\Omega }(ϵ)M`$. If we consider $`T=Mt(ϵ)`$, where $`t(ϵ)=1/M_{i=1}^Mt_i`$, we must obtain the same number of (very long) histories. Therefore, from the relation $`M=T/t(ϵ)`$ we obtain finally for the $`ϵ`$-entropy per unit time:

$$h(ϵ)=\frac{Mh^\mathrm{\Omega }(ϵ)}{T}=\frac{h^\mathrm{\Omega }(ϵ)}{t(ϵ)}\frac{h^\mathrm{\Omega }(ϵ,\tau _r)}{t(ϵ)}.$$
(3.3)
Where the last equality is valid at least for small enough $`\tau _r`$ . In most of the cases, the leading $`ϵ`$-contribution to $`h(ϵ)`$ in (3.3) is given by the mean exit-time $`t(ϵ)`$ and not by $`h^\mathrm{\Omega }(ϵ,\tau _r)`$. Anyhow, the computation of $`h^\mathrm{\Omega }(ϵ,\tau _r)`$ is compulsory in order to recover, e.g., a zero entropy for regular (e.g. periodic) signals.

One can easily estimate an upper and a lower bound for $`h(ϵ)`$ which can be computed in the exit time scheme . We use the following notation: for given $`ϵ`$ and $`\tau _r`$, $`h^\mathrm{\Omega }(ϵ,\tau _r)h^\mathrm{\Omega }(\{\eta _i,k_i\})`$, and we indicate with $`h^\mathrm{\Omega }(\{k_i\})`$ and $`h^\mathrm{\Omega }(\{\eta _i\})`$ respectively the Shannon entropy of the sequence $`\{k_i\}`$ and $`\{\eta _i\}`$. By applying standard results of information theory  one obtains the inequalities (see  for more details):

$$h^\mathrm{\Omega }(\{k_i\})h^\mathrm{\Omega }(\{\eta _i,k_i\})h^\mathrm{\Omega }(\{\eta _i\})+h^\mathrm{\Omega }(\{k_i\}).$$
(3.4)
Moreover, $`h^\mathrm{\Omega }(\{\eta _i\})H_1^\mathrm{\Omega }(\{\eta _i\})`$, where $`H_1^\mathrm{\Omega }(\{\eta _i\})`$ is the one-symbol entropy of $`\{\eta _i\}`$, (i.e. the entropy of the probability distribution of the exit-times measured on the scale $`\tau _r`$) which can be written as

$$H_1^\mathrm{\Omega }(\{\eta _i\})=c(ϵ)+\mathrm{ln}\left(\frac{t(ϵ)}{\tau _r}\right),$$

where $`c(ϵ)=p(z)\mathrm{ln}p(z)dz`$, and $`p(z)`$ is the probability distribution function of the rescaled exit-time $`z(ϵ)=t(ϵ)/t(ϵ)`$. Finally, using the previous relations, one obtains the following bounds for the $`ϵ`$-entropy:

$$\frac{h^\mathrm{\Omega }(\{k_i\})}{t(ϵ)}h(ϵ)\frac{h^\mathrm{\Omega }(\{k_i\})+c(ϵ)+\mathrm{ln}(t(ϵ)/\tau _r)}{t(ϵ)}.$$
(3.5)
Note that such bounds are relatively easy to compute and give a good estimate of $`h(ϵ)`$. In particular, as far as the scaling behavior of $`h(ϵ)`$ is concerned, the exit-time method allows for very efficient and good estimates of the scaling exponent. The reason is that at fixed $`ϵ`$, $`t(ϵ)`$ automatically selects the typical time at that scale. Consequently, it is not necessary to reach very large block sizes – at least if $`ϵ`$ is not too small. So that the leading contribution is given by $`t(ϵ)`$, and $`h^\mathrm{\Omega }(ϵ,\tau _r)`$ introduces, at worst, a sub-leading logarithmic contribution $`h^\mathrm{\Omega }(ϵ,\tau _r)\mathrm{ln}(t(ϵ)/\tau _r)`$ (see Eq. (3.5)).

In Ref.  one can found the details of the derivation and some applications.

## Appendix D Synthetic signals for turbulence

In this Appendix we recall some recently introduced methods for the generation of multi-affine stochastic signals , whose scaling properties are fully under control. The first step consists in generating a $`1`$-dimensional signal, and the second in decorating it such as to build the most general $`(d+1)`$-dimensional process, $`v(𝐱,t)`$, with given scaling properties in time and in space.
For the $`1`$-dimensional case there are at least two different kind of algorithms. One is based on a dyadic decomposition of the signal in a wavelet basis with a suitable assigned series of stochastic coefficients . The second is based on a multiplication of sequential Langevin-processes with a hierarchy of different characteristic times .
The first procedure suits particularly appealing for modeling of spatial turbulent fluctuations, because of the natural identification between wavelets and eddies in the physical space. The second one looks more appropriate for mimicking the turbulent time evolution in a fixed point of the space.

### D.1 Reproducing the spatial properties or the temporal ones

A non-sequential algorithm for $`1`$-dimensional multi-affine signal in $`[0,1]`$, $`v(x)`$, can be introduced as :

$$v(x)=\underset{n=1}{\overset{N}{}}\underset{k=1}{\overset{2^{(n1)}}{}}a_{n,k}\phi \left(\frac{xx_{n,k}}{\mathrm{}_n}\right)$$
(4.1)
where we have a set of reference scales $`\mathrm{}_n=2^n`$ and $`\phi (x)`$ is a wavelet-like function , i.e. of zero mean and rapidly decaying in both real space and Fourier-space. The signal $`v(x)`$ is built in terms of a superposition of fluctuations, $`\phi ((xx_{n,k})/\mathrm{}_n)`$ of characteristic width $`\mathrm{}_n`$ and centered in different points of $`[0,1]`$, $`x_{n,k}=(2k+1)/2^{n+1}`$. In  it has been proved that provided the coefficients $`a_{n,k}`$ are chosen by a random multiplicative process, i.e. the daughter is given in terms of the mother by a random process, $`a_{n+1,k^{}}=Xa_{n,k}`$ with $`X`$ a random number identical, independent distributed for any $`\{n,k\}`$, then the result of the superposition is a multi-affine function with given scaling exponents, namely:

$$|v(x+R)v(x)|^pR^{\zeta (p)},$$

with $`\zeta (p)=p/2\mathrm{log}_2X^p`$ and $`\mathrm{}_NR1`$. In this Appendix $``$ indicates the average over the probability distribution of the multiplicative process.

Besides the rigorous proof, the rationale for the previous result is simply that due to the hierarchical organization of the fluctuations, one may easily see that the term dominating the expression of a velocity fluctuation at scale $`R`$, in (4.1) is given by the couple of indices $`\{n,k\}`$ such that $`nlog_2(R)`$ and $`xx_{n,k}`$, i.e. $`v(x+R)v(x)a_{n,k}`$. The generalization (4.1) to d-dimension is given by:

$$v(𝐱)=\underset{n=1}{\overset{N}{}}\underset{k=1}{\overset{2^{d(n1)}}{}}a_{n,k}\phi \left(\frac{𝐱𝐱_{n,k}}{\mathrm{}_n}\right),$$

where now the coefficients $`\{a_{n,k}\}`$ are given in terms of a d-dimensional dyadic multiplicative process.
On the other hand, as previously said, sequential algorithms look more suitable for mimicking temporal fluctuations. Let us now discuss how to construct these stochastic multi-affine fields. With the application to time-fluctuations in mind, we will denote now the stochastic 1-dimensional functions with $`u(t)`$. The signal $`u(t)`$ is obtained by a superposition of functions with different characteristic times, representing eddies of various sizes :

$$u(t)=\underset{n=1}{\overset{N}{}}u_n(t).$$
(4.2)
The functions $`u_n(t)`$ are defined by the multiplicative process

$$u_n(t)=g_n(t)x_1(t)x_2(t)\mathrm{}x_n(t),$$
(4.3)
where the $`g_n(t)`$ are independent stationary random processes, whose correlation times are supposed to be $`\tau _n=(\mathrm{}_n)^\alpha `$, where $`\alpha =1h`$ (i.e. $`\tau _n`$ are the eddy-turn-over time at scale $`\mathrm{}_n`$) in the quasi-Lagrangian frame of reference  and $`\alpha =1`$ if one considers $`u(t)`$ as the time signal in a given point, and $`g_n^2=(\mathrm{}_n)^{2h}`$, where $`h`$ is the Hölder exponent. For a signal mimicking a turbulent flow, ignoring intermittency, we would have $`h=1/3`$. Scaling will appear for all time delays larger than the UV cutoff $`\tau _N`$ and smaller than the IR cutoff $`\tau _1`$. The $`x_j(t)`$ are independent, positive defined, identical distributed random processes whose time correlation decays with the characteristic time $`\tau _j`$. The probability distribution of $`x_j`$ determines the intermittency of the process.

The origin of (4.3) is fairly clear in the context of fully developed turbulence. Indeed we can identify $`u_n`$ with the velocity difference at scale $`\mathrm{}_n`$ and $`x_j`$ with $`(\epsilon _j/\epsilon _{j1})^{1/3}`$, where $`\epsilon _j`$ is the energy dissipation at scale $`\mathrm{}_j`$ .

The following arguments show, that the process defined according to (4.2,4.3) is multi-affine. Because of the fast decrease of the correlation times $`\tau _j=(\mathrm{}_j)^\alpha `$, the characteristic time of $`u_n(t)`$ is of the order of the shortest one, i.e., $`\tau _n=(\mathrm{}_n)^\alpha `$. Therefore, the leading contribution to the structure function $`\stackrel{~}{S}_q(\tau )=|u(t+\tau )u(t)|^q`$ with $`\tau \tau _n`$ stems from the $`n`$-th term in (4.2). This can be understood nothing that in $`u(t+\tau )u(t)=_{k=1}^N[u_k(t+\tau )u_k(t)]`$ the terms with $`kn`$ are negligible because $`u_k(t+\tau )u_k(t)`$ and the terms with $`kn`$ are sub-leading. Thus one has:

$$\stackrel{~}{S}_q(\tau _n)|u_n|^q|g_n|^qx^q^n\tau _n^{\frac{hq}{\alpha }\frac{\mathrm{log}_2x^q}{\alpha }}$$
(4.4)
and therefore for the scaling exponents:

$$\zeta _q=\frac{hq}{\alpha }\frac{\mathrm{log}_2x^q}{\alpha }.$$
(4.5)
The limit of an affine function can be obtained when all the $`x_j`$ are equal to $`1`$. A proper proof of these result can be found in .
Let us notice at this stage that the previous “temporal” signal for $`\alpha =1h`$ is a good candidate for a velocity measurements in a Lagrangian, co-moving frame of reference . Indeed, in such a reference frame the temporal decorrelation properties at scale $`\mathrm{}_n`$ are given by the eddy-turn-over times $`\tau _n=(\mathrm{}_n)^{1h}`$. On the other hand, in the laboratory reference frame the sweeping dominates the time evolution in a fixed point of the space and we must use as characteristic times of the processes $`x_n(t)`$ the sweeping times $`\tau _n^{(s)}=\mathrm{}_n`$, i.e., $`\alpha =1`$.

### D.2 Reproducing both the spatial and the temporal properties

We have now all the ingredients to perform a merging of temporal and spatial properties of a turbulent signal in order to define stochastic processes able to reproduce in a realistic way both spatial and temporal fluctuations (5.30)-(5.31). We just have to merge in a proper way the two previous algorithms.
For example, for a d-dimensional multi-affine field such as, say, one of the three components of a turbulent field in a Lagrangian reference frame we can use the following model:

$$v_L(𝐱,t)=\underset{n=1}{\overset{N}{}}\underset{k=1}{\overset{2^{d(n1)}}{}}a_{n,k}(t)\phi \left(\frac{𝐱𝐱_{n,k}}{\mathrm{}_n}\right).$$
(4.6)
where the temporal dependency of $`a_{n,k}(t)`$ is chosen following the sequential algorithm while its spatial part are given by the dyadic structure of the non-sequential algorithm. In (4.6) we have used the notation $`v_L(𝐱,t)`$ in order to stress the typical Lagrangian character of such a field.
We are now able to guess a good candidate for the same field measured in the laboratory-reference frame, i.e. where the time properties are dominated by the sweeping of small scales by large scales. Indeed, in order to reproduce the sweeping effects one needs that the centers $`\{𝐱_{n,k}\}`$ of the wavelets-like functions move according a swept-dynamics.

To do so, let us define the Eulerian model:

$$v_E(𝐱,t)=\underset{n=1}{\overset{N}{}}\underset{k=1}{\overset{2^{d(n1)}}{}}a_{n,k}(t)\phi \left(\frac{𝐱𝐱_{n,k}(t)}{\mathrm{}_n}\right).$$
(4.7)
where the difference with the previous definition is in the temporal dependency of the centers of the wavelets, $`𝐱_{n,k}(t)`$. According to the Richardson-Kolmogorov cascade picture, one assumes that sweeping is present, i.e., $`𝐱_{n,k}=𝐱_{n1,k^{}}+𝐫_{n,k}`$ where $`(n,k^{})`$ labels the “mother” of the $`(n,k)`$-eddy and $`𝐫_{n,k}`$ is a stochastic vector which depends on $`𝐫_{n1,k^{}}`$ and evolves with characteristic time $`\tau _n(\mathrm{}_n)^{1h}`$. Furthermore, its norm is $`O(\mathrm{}_n)`$: $`c_1<|𝐫_{n,k}|/\mathrm{}_n<c_2`$ where $`c_1`$ and $`c_2`$ are constants of order one.
We now see that if we measure in one fixed spatial point a fluctuations over a time delay $`\mathrm{\Delta }t`$, is like to measure a simultaneous fluctuations at scale separation $`R=U\mathrm{\Delta }t`$, i.e. due to the sweeping the main contribution to the sum will be given by the terms with scale-index $`n=\mathrm{log}_2(R=U\mathrm{\Delta }t)`$ while the temporal dependency of the coefficients $`\{a_{n,k}(t)\}`$ will be practically frozen on that time scale. Therefore, one has the proper spatial and temporal statistics, see Ref.  for details. This happens because in presence of the sweeping the main contribution is given by the displacement of the center at large scale, i.e. $`\delta r_0=|𝐫_\mathrm{𝟎}(t+\mathrm{\Delta }t)𝐫_\mathrm{𝟎}(t)|U\mathrm{\Delta }t`$, and the eddy turnover time at scale $`\mathrm{}_n`$ is $`O((\mathrm{}_n)^{1h})`$ always large that the sweeping time $`O(\mathrm{}_n)`$ at the same scale.
In the previous discussion for the sake of simplicity we did not consider the incompressibility condition. However one can take into account this constraint by the projection on the solenoidal space.
In conclusion we have a way to build up a synthetic signal with the proper Eulerian (laboratory) properties, i.e. with sweeping, and also with the proper Lagrangian properties.