Index hypergeometric transform and imitation of analysis of Berezin kernels on hyperbolic spaces

Yu. A. Neretin

Sbornik. Mathematics, Tome 192 (2001) no. 3, pp. 403-432

Résumé

The index hypergeometric transform (also called the Olevskii transform or the Jacobi transform) generalizes the spherical transform in $L^2$ on rank 1 symmetric spaces (that is, real, complex, and quaternionic Lobachevskii spaces). The aim of this paper is to obtain properties of the index hypergeometric transform imitating the analysis of Berezin kernels on rank 1 symmetric spaces. The problem of the explicit construction of a unitary operator identifying $L^2$ and a Berezin space is also discussed. This problem reduces to an integral expression (the $\Lambda$-function), which apparently cannot be expressed in a finite form in terms of standard special functions. (Only for certain special values of the parameter can this expression be reduced to the so-called Volterra type special functions.) Properties of this expression are investigated. For some series of symmetric spaces of large rank the above operator of unitary equivalence can be expressed in terms of the determinant of a matrix of $\Lambda$-functions.