Topological size of scrambled sets

François Blanchard¹ ; Wen Huang² ; L'ubomír Snoha³

¹ LAMA – UMR 8050 (CNRS-Université Paris Est) 5 boulevard Descartes 77454 Marne-la-Vallée Cedex 2, France
² Department of Mathematics University of Science and Technology of China Hefei Anhui 230026, P.R. China
³ Department of Mathematics Faculty of Natural Sciences Matej Bel University Tajovského 40 974 01 Banská Bystrica, Slovakia

Colloquium Mathematicum, Tome 110 (2008) no. 2, pp. 293-361

Résumé

A subset $S$ of a topological dynamical system $(X,f)$ containing at least two points is called a scrambled set if for any $x,y\in S$ with $x\neq y$ one has $$\liminf_{n\to \infty} d(f^n(x), f^n(y)) = 0 \quad \hbox{and} \quad \limsup_{n\to \infty} d(f^n(x), f^n(y)) > 0,$$ $d$ being the metric on $X$. The system $(X,f)$ is called Li–Yorke chaotic if it has an uncountable scrambled set.These notions were developed in the context of interval maps, in which the existence of a two-point scrambled set implies Li–Yorke chaos and many other chaotic properties. In the present paper we address several questions about scrambled sets in the context of topological dynamics. There the assumption of Li–Yorke chaos, and also stronger ones like the existence of a residual scrambled set, or the fact that $X$ itself is a scrambled set (in these cases the system is called residually scrambled or completely scrambled respectively), are not so highly significant. But they still provide valuable information.First, the following question arises naturally: is it true in general that a Li–Yorke chaotic system has a Cantor scrambled set, at least when the phase space is compact? This question is not answered completely but the answer is known to be yes when the system is weakly mixing or Devaney chaotic or has positive entropy, all properties implying Li–Yorke chaos; we show that the same is true for symbolic systems and systems without asymptotic pairs, which may not be Li–Yorke chaotic. More generally, there are severe restrictions on Li–Yorke chaotic dynamical systems without a Cantor scrambled set, if they exist.A second set of questions concerns the size of scrambled sets inside the space $X$ itself. For which dynamical systems $(X,f)$ do there exist first category, or second category, or residual scrambled sets, or a scrambled set which is equal to the whole space $X$?While reviewing existing results, we give examples of systems on arcwise connected continua in the plane having maximal scrambled sets with any prescribed cardinalities, in particular systems having at most finite or countable scrambled sets. We also give examples of Li–Yorke chaotic systems with at most first category scrambled sets. It is proved that minimal compact systems, graph maps and a large class of symbolic systems containing subshifts of finite type are never residually scrambled; assuming the Continuum Hypothesis, weakly mixing systems are shown to have second category scrambled sets. Various examples of residually scrambled systems are constructed. It is shown that for any minimal distal system there exists a non-disjoint completely scrambled system. Finally, various other questions are solved. For instance, a completely scrambled system may have a factor without any scrambled set, and a triangular map may have a scrambled set with non-empty interior.

Détail

DOI : 10.4064/cm110-2-3

Keywords: subset topological dynamical system containing least points called scrambled set neq has liminf infty x quad hbox quad limsup infty x being metric system called yorke chaotic has uncountable scrambled set these notions developed context interval maps which existence two point scrambled set implies yorke chaos many other chaotic properties present paper address several questions about scrambled sets context topological dynamics there assumption yorke chaos stronger existence residual scrambled set itself scrambled set these cases system called residually scrambled completely scrambled respectively highly significant still provide valuable information first following question arises naturally general yorke chaotic system has cantor scrambled set least phase space compact question answered completely answer known yes system weakly mixing devaney chaotic has positive entropy properties implying yorke chaos symbolic systems systems without asymptotic pairs which may yorke chaotic generally there severe restrictions yorke chaotic dynamical systems without cantor scrambled set exist second set questions concerns size scrambled sets inside space itself which dynamical systems there exist first category second category residual scrambled sets scrambled set which equal whole space while reviewing existing results examples systems arcwise connected continua plane having maximal scrambled sets prescribed cardinalities particular systems having finite countable scrambled sets examples yorke chaotic systems first category scrambled sets proved minimal compact systems graph maps large class symbolic systems containing subshifts finite type never residually scrambled assuming continuum hypothesis weakly mixing systems shown have second category scrambled sets various examples residually scrambled systems constructed shown minimal distal system there exists non disjoint completely scrambled system finally various other questions solved instance completely scrambled system may have factor without scrambled set triangular map may have scrambled set non empty interior