This article is from the Nonlinear Science FAQ, by James D. Meiss firstname.lastname@example.org with numerous contributions by others.
The transition to chaos for a Hamiltonian (conservative) system is somewhat
different than that for a dissipative system (recall [2.5]). In an integrable
(nonchaotic) Hamiltonian system, the motion is "quasiperiodic", that is motion
that is oscillatory, but involves more than one independent frequency (see
also [2.12]). Geometrically the orbits move on tori, i.e. the mathematical
generalization of a donut. Examples of integrable Hamiltonian systems include
harmonic oscillators (simple mass on a spring, or systems of coupled linear
springs), the pendulum, certain special tops (for example the Euler and
Lagrange tops), and the Kepler motion of one planet around the sun.
It was expected that a typical perturbation of an integrable Hamiltonian
system would lead to "ergodic" motion, a weak version of chaos in which all of
phase space is covered, but the Lyapunov exponents [2.11] are not necessarily
positive. That this was not true was rather surprisingly discovered by one of
the first computer experiments in dynamics, that of Fermi, Pasta and Ulam.
They showed that trajectories in nonintegrable system may also be surprisingly
stable. Mathematically this was shown to be the case by the celebrated theorem
of Kolmogorov Arnold and Moser (KAM), first proposed by Kolmogorov in 1954.
The KAM theorem is rather technical, but in essence says that many of the
quasiperiodic motions are preserved under perturbations. These orbits fill out
what are called KAM tori.
An amazing extension of this result was started with the work of John Greene
in 1968. He showed that if one continues to perturb a KAM torus, it reaches a
stage where the nearby phase space [2.4] becomes self-similar (has fractal
structure [3.2]). At this point the torus is "critical," and any increase in
the perturbation destroys it. In a remarkable sequence of papers, Aubry and
Mather showed that there are still quasiperiodic orbits that exist beyond this
point, but instead of tori they cover cantor sets [3.5]. Percival actually
discovered these for an example in 1979 and named them "cantori."
Mathematicians tend to call them "Aubry-Mather" sets. These play an important
role in limiting the rate of transport through chaotic regions.
Thus, the transition to chaos in Hamiltonian systems can be thought of as the
destruction of invariant tori, and the creation of cantori. Chirikov was the
first to realize that this transition to "global chaos" was an important
physical phenomena. Local chaos also occurs in Hamiltonian systems (in the
regions between the KAM tori), and is caused by the intersection of stable and
unstable manifolds in what Poincaré called the "homoclinic trellis."
To learn more: See the introductory article by Berry, the text by Percival and
Richards and the collection of articles on Hamiltonian systems by MacKay and
Meiss [4.1]. There are a number of excellent advanced texts on Hamiltonian
dynamics, some of which are listed in [4.1], but we also mention
Meyer, K. R. and G. R. Hall (1992), Introduction to Hamiltonian dynamical
systems and the N-body problem (New York, Springer-Verlag).