Structure of the complementary series and special representations of the groups $O(n,1)$ and $U(n,1)$

A. M. Vershik   ; M. I. Graev

Trudy Matematicheskogo Instituta imeni V.A. Steklova, Tome 61 (2006) no. 5, pp. 799-884

Résumé

This is a survey of several models (including new models) of irreducible complementary series representations and their limits, special representations, for the groups $SU(n,1)$ and $SO(n,1)$. These groups, whose geometrical meaning is well known, exhaust the list of simple Lie groups for which the identity representation is not isolated in the space of irreducible unitary representations (that is, which do not have the Kazhdan property) and hence there exist irreducible unitary representations of these groups, so-called ‘special representations’, for which the first cohomology of the group with coefficients in these representations is non-trivial. For technical reasons it is more convenient to consider the groups $O(n,1)$ and $U(n,1)$, and most of this paper is devoted to the group $U(n,1)$. The main emphasis is on the so-called commutative models of special and complementary series representations: in these models, the maximal unipotent subgroup is represented by multipliers in the case of $O(n,1)$, and by the canonical model of the Heisenberg representations in the case of $U(n,1)$. Earlier, these models were studied only for the group $ SL(2,\mathbb R)$. They are especially important for the realization of non-local representations of current groups, which will be considered elsewhere. Substantial use is made of the ‘denseness’ of the irreducible representations under study for the group $SO(n,1)$: their restrictions to the maximal parabolic subgroup $P$ are equivalent irreducible representations. Conversely, in order to extend an irreducible representation of $P$ to a representation of $SO(n,1)$, it is necessary to determine only one involution. For the group $U(n,1)$, the situation is similar but slightly more complicated.

Bibliographie

[1] Yu. A. Neretin, G. I. Olshanskii, “Granichnye znacheniya golomorfnykh funktsii, osobye unitarnye predstavleniya grupp $O(p,q)$ i ikh predely pri $q\to\infty$”, Teoriya predstavlenii, dinamicheskie sistemy, kombinatornye i algoritmicheskie metody. I, Zapiski nauch. sem. POMI, 223, ed. A. M. Vershik, Nauka, SPb., 1995, 9–91 | MR | Zbl

[2] A. M. Vershik, S. I. Karpushev, “Kogomologii grupp v unitarnykh predstavleniyakh, okrestnost edinitsy i uslovno polozhitelno opredelennye funktsii”, Matem. sb., 119(161):4(12) (1982), 521–533 | MR | Zbl

[3] D. A. Kazhdan, “O svyazi dualnogo prostranstva gruppy so stroeniem ee zamknutykh podgrupp”, Funkts. analiz i ego prilozh., 1:1 (1967), 71–74 | MR | Zbl

[4] C. Delaroche, A. Kirillov, “Sur les relations entre l'espace dual d'un groupe et la structure de ses sous-groupes fermés”, Séminaire Bourbaki 1967/68, 20e année, exp. 343, Benjamin, 1969 | MR | MR | Zbl

[5] M. Bekka, P. de la Harpe, A. Valette, Kazhdan's property ($T$), New Math. Monogr., 11, Cambridge Univ. Press, Cambridge, 2008 | MR | Zbl

[6] Y. Shalom, “Rigidity, unitary representation of semisimple groups, and fundamental groups of manifolds with rank one transformation group”, Ann. of Math. (2), 152:1 (2000), 113–182 | DOI | MR | Zbl

[7] P. Delorme, “1-cohomologie des représentations unitaires des groupes de Lie semi-simples et résolubles. Produits tensoriels continues de représentations”, Bull. Soc. Math. France, 105:3 (1977), 281–336 | MR | Zbl

[8] R. S. Ismagilov, Representations of infinite-dimensional groups, Transl. Math. Monogr., 152, Amer. Math. Soc., Providence, RI, 1996 | MR | Zbl

[9] A. Gisharde, Kogomologii topologicheskikh grupp i algebr Li, Mir, M., 1984 | MR | MR | Zbl

[10] G. Kuhn, A. Vershik, “Canonical semigroups of states and cocycles for the group of automorphisms of a homogeneous tree”, Algebr. Represent. Theory, 6:3 (2003), 333–352 | DOI | MR | Zbl

[11] A. M. Vershik, I. M. Gelfand, M. I. Graev, “Neprivodimye predstavleniya gruppy $G^X$ i kogomologii”, Funkts. analiz i ego prilozh., 8:2 (1974), 67–69 | MR | Zbl

[12] J. Faraut, K. Harzallah, “Distances hilbertiennes invariantes sur un espace homogène”, Ann. Inst. Fourier (Grenoble), 24:3 (1974), 171–217 | MR | Zbl

[13] P. de la Harpe, A. Valette, La propriété $(T)$ de Kazhdan pour les groupes localement compacts, Astérisque, 175, Soc. Math. France, Paris, 1989 | MR | Zbl

[14] A. M. Vershik, I. M. Gelfand, M. I. Graev, “Predstavleniya gruppy $SL(2,\mathbf R)$, gde $\mathbf R$ – koltso funktsii”, UMN, 28:5 (1973), 83–128 | MR | Zbl

[15] A. Guichardet, “Représentations de $G^X$ selon Gelfand et Delorme”, Séminaire Bourbaki 1975/76, 28-e année, exp. 486, Lecture Notes in Math., 567, Springer, Berlin, 1977, 238–255 | MR | Zbl

[16] G. Segal, “Two papers on representation theory”, Representation theory, London Math. Soc. Lecture Note Series, 69, Cambridge Univ. Press, Cambridge, New York, 1982, 1–13 | MR | Zbl

[17] A. M. Vershik, I. M. Gelfand, M. I. Graev, “Kommutativnaya model osnovnogo predstavleniya gruppy $GL(2,\mathbb R)^X$ otnositelno unipotentnoi podgruppy”, Teoretiko-gruppovye metody v fizike, Trudy mezhdunarodn. sem., t. 2 (Zvenigorod, 1982), Nauka, M., 1983, 472–487 | MR | Zbl

[18] A. M. Vershik, I. M. Gelfand, M. I. Graev, “Kommutativnaya model predstavleniya gruppy tokov $SL(2,\mathbb R)^X$, svyazannaya s unipotentnoi podgruppoi”, Funkts. analiz i ego prilozh., 17:2 (1983), 70–72 | MR | Zbl

[19] A. M. Vershik, I. M. Gelfand, M. I. Graev, Kommutativnaya model osnovnogo predstavleniya gruppy $SL(2,\mathbb R)^X$ otnositelno unipotentnoi podgruppy, Preprint No 169, IPM AN SSSR, M., 1982 | MR

[20] I. M. Gel'fand, M. I. Graev, A. M. Vershik, “Models of representations of current groups”, Representations of Lie groups and Lie algebras (Budapest, 1971), Akad. Kiadó, Budapest, 1985, 121–179 | MR | Zbl

[21] A. M. Vershik, M. I. Graev, “Kommutativnaya model predstavleniya gruppy tokov $O(n,1)^X$ i obobschennaya lebegova mera v prostranstve raspredelenii”, Funkts. analiz i ego prilozh., 39:2 (2005), 1–12 | MR | Zbl

[22] M. I. Graev, A. M. Vershik, “The basic representation of the current group $O(n,1)^X$ in the $L^2$ space over the generalized Lebesgue measure”, Indag. Math. (N.S.), 16:3–4 (2005), 499–529 | DOI | Zbl

[23] I. M. Gelfand, M. I. Graev, “Osobye predstavleniya gruppy $SU(n,1)$ i proektivnye unitarnye predstavleniya gruppy tokov $SU(n,1)^X$”, Dokl. RAN, 332:3 (1993), 280–282 | MR | Zbl

[24] I. M. Gelfand, M. I. Graev, “Proektivnye neunitarnye predstavleniya grupp tokov”, Dokl. RAN, 338:3 (1994), 298–301 | MR | Zbl

[25] F. A. Berezin, “Predstavleniya nepreryvnogo pryamogo proizvedeniya universalnykh nakryvayuschikh gruppy dvizhenii kompleksnogo shara”, Tr. MMO, 36, 1978, 275–293 | MR | Zbl

[26] H. Araki, “Factorizable representations of the current algebra. Noncommutative extension of the Lévy–Khinchin formula and cohomology of a solvable group with values in a Hilbert space”, Publ. Res. Inst. Math. Sci. Ser. A, 5:3 (1969/70), 361–422 | DOI | MR | Zbl

[27] H. Araki, E. J. Woods, “Complete Boolean algebras of type I factors”, Publ. Res. Inst. Math. Sci. Ser. A, 2:2 (1966), 157–242 | DOI | MR | Zbl

[28] K. R. Parthasarathy, K. Schmidt, Infinitely divisible projective representations, cocycles, and Lévy–Khinchine–Araki formula on locally compact groups, Research Report No 17, Manchester–Sheffield School of Probability and Statistics, 1970

[29] K. R. Parthasarathy, K. Schmidt, “Factorisable representations of current groups and the Araki–Woods embedding theorem”, Acta Math., 128:1 (1972), 53–71 | DOI | MR | Zbl

[30] K. R. Parthasarathy, K. Schmidt, Positive definite kernels, continuous tensor products, and central limit theorems of probability theory, Lecture Notes in Math., 272, Springer-Verlag, Berlin, 1972 | MR | Zbl

[31] R. F. Streater, “Current communication relations, continuous tensor products, and infinitely divisible group representations”, Rend. Sci. Istit. Fis. E. Fermi, II (1969), 247–263

[32] R. F. Streater, “Infinitely divisible representations of Lie algebras”, Z. Wahrscheinlichkeitstheor. Verw. Geb., 19:1 (1971), 67–80 | DOI | MR | Zbl

[33] A. M. Vershik, I. M. Gelfand, M. I. Graev, Predstavleniya gruppy diffeomorfizmov, svyazannye s beskonechnymi konfiguratsiyami, Preprint No 46, IPM AN SSSR, M., 1975

[34] A. M. Vershik, I. M. Gelfand, M. I. Graev, “Predstavleniya gruppy diffeomorfizmov”, UMN, 30:6 (1975), 3–50 | MR | Zbl

[35] I. M. Gel'fand, M. I. Graev, “Principal representatons of the group $\operatorname{U}(\infty)$”, Representation of Lie groups and related topics, Adv. Stud. Contemp. Math., 7, Gordon and Breach, New York, 1990, 119–153 | MR | Zbl

[36] G. I. Olshanskii, “Sfericheskie funktsii i kharaktery na gruppe $U(\infty)$”, UMN, 37:2 (1982), 217–218 | MR | Zbl

[37] A. Guichardet, Symmetric Hilbert spaces and related topics. Infinitely divisible positive definite functions. Continuous products and tensor products. Gaussian and Poissonian stochastic processes, Lecture Notes in Math., 261, Springer-Verlag, Berlin, 1972 | MR | Zbl

[38] I. M. Gelfand, N. Ya. Vilenkin, Obobschennye funktsii. Vyp. 4. Nekotorye primeneniya garmonicheskogo analiza. Osnaschennye gilbertovy prostranstva, Fizmatgiz, M., 1961 | MR | Zbl

[39] F. A. Berezin, “Kvantovanie v kompleksnykh simmetricheskikh prostranstvakh”, Izv. AN SSSR. Ser. matem., 39:2 (1975), 363–402 | MR | Zbl

[40] N. Tsilevich., A. Vershik, M. Yor, “An infinite-dimensional analogue of the Lebesgue measure and distinguished properties of the gamma process”, J. Funct. Anal., 185:1 (2001), 274–296 | DOI | MR | Zbl

[41] I. M. Gelfand, M. I. Graev, N. Ya. Vilenkin, Obobschennye funktsii. Vyp. 5. Integralnaya geometriya i svyazannye s nei voprosy teorii predstavlenii, Fizmatgiz, M., 1962 | MR | Zbl

[42] I. M. Gelfand, M. I. Graev, I. I. Pyatetskii-Shapiro, Teoriya predstavlenii i avtomorfnye funktsii, Nauka, M., 1966 | MR | Zbl

[43] G. Beitmen, A. Erdeii, Vysshie transtsendentnye funktsii. [T. 1:] Giperbolicheskaya funktsiya. Funktsii Lezhandra, Nauka, M., 1973 ; [Рў. 2:] Функции Р‘РμСЃСЃРμля, функции параболичРμСЃРєРѕРіРѕ цилиндра, ортогональныРμ многочлРμРЅС‹, Наука, Рњ., 1974 | MR | MR | Zbl | MR | Zbl

[44] A. M. Perelomov, Obobschennye kogerentnye sostoyaniya i ikh primeneniya, Nauka, M., 1987 | MR | Zbl

[45] G. Beitmen, A. Erdeii, Tablitsy integralnykh preobrazovanii. T. 1: Preobrazovaniya Fure, Laplasa, Mellina, Nauka, M., 1969 | MR | MR | Zbl

[46] M. I. Graev, “Unitarnye predstavleniya veschestvennykh prostykh grupp Li”, Tr. MMO, 7, 1958, 335–389 | MR | Zbl

[47] M. I. Graev, “Neprivodimye unitarnye predstavleniya gruppy matrits tretego poryadka, sokhranyayuschikh indefinitnuyu ermitovu formu”, Dokl. AN SSSR, 113:5 (1957), 966–969 | MR | Zbl

[48] F. A. Berezin, I. M. Gelfand, M. I. Graev, M. A. Naimark, “Predstavleniya grupp”, UMN, 11:6 (1956), 13–40 | MR | Zbl

[49] Khua Lo-ken, Garmonicheskii analiz funktsii mnogikh kompleksnykh peremennykh v klassicheskikh oblastyakh, IL, M., 1959 | MR | Zbl

[50] G. van Dijk, S. C. Hille, “Canonical representations related to hyperbolic spaces”, J. Funct. Anal., 147:1 (1997), 109–139 | DOI | MR | Zbl

[51] V. F. Molchanov, L. I. Grosheva, “Canonical and boundary representations on the Lobachevsky plane”, The 2000 Twente conference on Lie groups (Enschede), Acta Appl. Math., 73:1–2 (2002), 59–77 | DOI | MR | Zbl

[52] V. F. Molchanov, “Canonical representations on the two-sheeted hyperboloid”, Indag. Math., 16:3–4 (2005), 609–630 | DOI | MR | Zbl

[53] F. A. Berezin, “Kvantovanie”, Izv. AN SSSR. Ser. matem., 38:5 (1974), 1116–1175 | MR | Zbl

[54] I. S. Gradshtein, I. M. Ryzhik, Tablitsy integralov, summ, ryadov i proizvedenii, Fizmatgiz, M., 1962 | MR | Zbl