Deformation Theory of Discontinuous Groups 9783110765298, 9783110765304, 9783110765397, 2022934617

Table of contents :
Preface
Contents
1 Structure theory
2 Proper actions on homogeneous spaces
3 Deformation and moduli spaces
4 The deformation space for nilpotent Lie groups
5 Local and strong local rigidity
6 Stability concepts and Calabi–Markus phenomenon
7 Discontinuous actions on reduced nilmanifolds
8 Deformation of topological modules
Bibliography
Index

Citation preview

Ali Baklouti Deformation Theory of Discontinuous Groups

De Gruyter Expositions in Mathematics

|

Edited by Lev Birbrair, Fortaleza, Brazil Victor P. Maslov, Moscow, Russia Walter D. Neumann, New York City, New York, USA Markus J. Pflaum, Boulder, Colorado, USA Dierk Schleicher, Bremen, Germany Katrin Wendland, Freiburg, Germany

Volume 72

Ali Baklouti

Deformation Theory of Discontinuous Groups |

Mathematics Subject Classification 2020 Primary: 22E25, 22E27, 22E40, 22G15, 32G05, 57S30; Secondary: 81S10, 57M25, 57M27, 57S30 Author Prof. Ali Baklouti Faculté des Sciences de Sfax Département de Mathématiques Route de Soukra 3038 Sfax Tunisia [email protected]

ISBN 978-3-11-076529-8 e-ISBN (PDF) 978-3-11-076530-4 e-ISBN (EPUB) 978-3-11-076539-7 ISSN 0938-6572 Library of Congress Control Number: 2022934617 Bibliographic information published by the Deutsche Nationalbibliothek The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic data are available on the Internet at http://dnb.dnb.de. © 2022 Walter de Gruyter GmbH, Berlin/Boston Typesetting: VTeX UAB, Lithuania Printing and binding: CPI books GmbH, Leck www.degruyter.com

Preface Discontinuous actions of groups play an important role in many fields of mathematics, especially in the study of Riemann surfaces. This research axis appears to be a significant and indispensable framework because of its close relationship with so many other fields in mathematics, such as geometry, topology, number theory, algebraic geometry, differential geometry and with different fields, such as physics and other various areas. The study of Kleinian groups (discrete groups of orientation preserving isometries of hyperbolic spaces), Fuchsian groups, and the theory of automorphic forms are all rich areas of mathematics with many deep results. The work of Thurston on 3-manifolds and as a generalization the deformations of Kleinian groups have given additional focus to this very rich field of discontinuous group actions. When it comes to the setting of solvable groups actions, the literature is somewhat scarce in this area. This book is devoted mainly to studying various geometric and topological concepts related to the deformation and moduli spaces of discontinuous group actions and building some interrelationships between these concepts. It contains the most recent developments of the theory, extending from basic concepts to a comprehensive exposition, and highlighting the newest approaches and methods in deformation theory. It presents full proofs of recent results, computes fundamental examples and serves as an introduction and reference for students and researchers in Lie theory, discontinuous groups and deformation (and moduli) spaces. It also includes the most recent solutions to many open questions over the last decades and brings related newest research results in this area. The first chapter aims to record some main backgrounds on nilpotent, solvable and exponential solvable Lie groups and some compact extensions. Fundamental and basic examples, such as Heisenberg groups, threadlike groups, Euclidean motion groups and Heisenberg motion groups are treated with extensive details for further developments and use. As a preparation to discontinuous actions, an explicit description of closed and discrete subgroups of these groups is also well developed. The important notion of syndetic hull of closed subgroups is also introduced and many existence and unicity results are proved, including the extensions of homomorphisms of discrete subgroups to their syndetic hulls, which appears to be of major role in the computation of the parameter, deformation and moduli spaces. The second chapter focuses on the characterization of proper action of closed subgroups on solvmanifolds and on some homogeneous spaces of compact extensions. In the case of m-step nilpotent Lie groups, the proper action of a closed connected subgroup is shown to be equivalent to its free actions for m ≤ 3. Such a fact fails in general to hold otherwise. We also generate geometric criteria of the proper action of a discontinuous group on an arbitrary homogeneous space, where the group in question stands for the semidirect product group K ⋉ ℝn , where K is a compact subgroup of GL(n, ℝ). In the case of Heisenberg motion groups, the same requires the classification into three categories of all discrete subgroups. As shown, this will be a capital https://doi.org/10.1515/9783110765304-201

VI | Preface role in the study of many geometrical concepts related to corresponding deformation and moduli spaces. We also define the notions of weak and finite proper actions and substantiate that these are equivalent to free actions of connected closed subgroups operating on special and maximal solvmanifolds. We pay attention in Chapter 3 to the determination of the parameter, deformation and moduli spaces of the action of a discontinuous group Γ ⊂ G on a homogeneous space G/H in numerous settings, G being a Lie group and H a closed subgroup of G. This issue is of major relevance to understand the local geometric structures of these spaces as many examples reveal. The strategy basically consists in building up accurate cross-sections of adjoint orbits of deformation parameters. Toward such goal, the first step consists in generating an algebraic characterization of the above spaces making use of the results on the existence of syndetic hulls developed in the first chapter. Introducing the Grassmannian topology, we then show that the parameter space is stratified into G-invariant layers, endowed with the structure of a total space of a principal fiber bundle. This allows to explicitly determine (to a certain extent) the parameter and deformation spaces in many fundamental cases. For instance, the setting of Heisenberg groups is extensively pursued in the fourth chapter, where a necessary and sufficient condition for which the deformation space is endowed with a smooth manifold structure is obtained. This further allows to extend the study to the setting of the direct product of Heisenberg groups. We also deal with the setting of general m-step nilpotent Lie groups in Chapter 4, where a description of the parameter and deformation spaces are derived (m ≤ 3). A necessary condition for the Hausdorfness of the deformation space is also obtained. The setting of threadlike groups is also studied and an explicit determination of the deformation space is provided. In the case of a non-Abelian discontinuous group of rank k, the deformation space is shown to be endowed with a smooth manifold structure if and only if k > 3. The fifth chapter is devoted to study the local rigidity property of deformations introduced by A. Weil in the Riemannian case and generalized further by T. Kobayashi. We state the local rigidity conjecture in the nilpotent setting, which asserts that the local rigidity fails to hold for any nontrivial discontinuous group acting on nilpotent homogeneous space. We further extend our study to many exponential and solvable settings. Namely, we show that the local rigidity fails when the Lie algebra l of the syndetic hull of Γ is not characteristically solvable and in the exponential setting where l is Abelian and dim(l) ≥ 2. Besides, we prove the existence of formal colored discontinuous groups in the general solvable setting. That is, the parameter space admits a mixture of locally rigid and formally nonrigid deformations. In the case where G is the diamond group and Γ a nontrivial finitely generated subgroup of G (not necessarily discrete), then there is no open G-orbits in Hom(Γ, G). In particular, if Γ is a discontinuous group for a homogeneous space G/H, then the strong local rigidity property fails to hold.

Preface | VII

We are also concerned with an analogue of the so-called Selberg–Weil–Kobayashi local rigidity theorem in the context of a real exponential group G and H a maximal subgroup of G, where the local rigidity property is shown to hold if and only if the group G is isomorphic to Aff(ℝ), the group of affine transformations of the real line. For more generality where G is a Lie group and Γ a finite group, we show that the space Hom(Γ, G)/G is discrete and at most countable. This space is finite if in addition G has finitely many connected components. This helps to show an analogue of the local rigidity conjecture holds in both cases where G stands for the compact extension K ⋉ ℝn and for the Heisenberg motion groups. Chapter 6 deals with the stability property, a different geometrical concept of deformations, which measures in general the fact that in a neighborhood of φ ∈ Hom(Γ, G), the properness property of the action on G/H is preserved. The determination of stable points is a very difficult problem in general, which mainly reduces to describe explicitly the interior of the subset of Hom0d (Γ, G) of injective homomorphisms with discrete image. We are then led to investigate about several kinds of questions of geometric nature related to the structure of the deformation space and as a result, many stability theorems will be established in the nilpotent and exponential cases and also in the context of some compact extensions. On the other hand, it may then happen that there does not exist an infinite discrete subgroup Γ of G, which acts properly discontinuously on G/H. This phenomenon is called the Calabi–Markus phenomenon. Based on several upshots proved in previous chapters, such a phenomenon together with the question of existence of compact Clifford–Klein forms are subject of a study in the context of some compact extensions of nilpotent Lie groups. The seventh chapter is devoted to resume some of the previous upshots once we remove the assumption on the groups in question to be simply connected. This means that the center may be compact and we show in this case that many previously open questions in the simply connected setting get answered. For instance, in the case of r reduced Heisenberg groups H2n+1 , the deformation space turns out to be a Hausdorff space and even endowed with a smooth manifold structure for any arbitrary connected subgroup H of G and any arbitrary discontinuous group Γ for G/H and that the r stability property holds, which is also the case of the product Lie group G = H2n+1 × r H2n+1 and H = ΔG , the diagonal subgroup of G. On the other hand, a (strong) local r r r rigidity theorem is obtained for both H2n+1 and H2n+1 × H2n+1 . That is, the parameter space admits a (strong) locally rigid point if and only if Γ is finite. The setting of reduced threadlike groups is also considered through similar questions. We show that a local rigidity conjecture holds for Abelian discontinuous groups and that non-Abelian discontinuous groups are stable. We also single out the notion of stability on layers and show that any Abelian discontinuous group is stable on layers. The purpose of the last chapter is to describe a dequantization procedure for topological modules over a deformed algebra. We define the characteristic variety of a topological module as the common zeroes of the annihilator of the representation obtained

VIII | Preface by setting the deformation parameter to zero. On the other hand, the Poisson characteristic variety is defined as the common zeroes of the ideal obtained by considering the annihilator of the deformed representation, and then setting the deformation parameter to zero. We next apply such a dequantization procedure to the case of representations of Lie groups. Let V = ℝd be a linear Poisson manifold. Then the dual V ∗ of linear forms on V form a Lie subalgebra g of the algebra S(gℂ ) of polynomials on V endowed with the Poisson bracket. We then regard the Poisson manifold V as the dual g∗ of the Lie algebra g. In the case where G = exp g is an exponential solvable Lie group, the orbit method appears to be a fundamental tool to smoothly link their unitary duals with the space of coadjoint orbits. We first bring explicit computations of the characteristic and the Poisson characteristic varieties in many fundamental Poisson-linear examples. In the nilpotent case, we show that any coadjoint orbit appears as the Poisson characteristic variety of a well-chosen topological module. We then substantiate the Zariski closure conjecture claiming that for an irreducible unitary representation of G, associated to a coadjoint orbit Ω via the Kirillov orbit method, the Poisson characteristic variety associated to a topological module with an adequate way coincides with the Zariski closure in g∗ of the orbit Ω. We also prove the conjecture in many restrictive cases, notably in the nilpotent setting (with a different approach) and in the case where the representation is induced from a normal polarizing subgoup. We finally investigate the bicontinuity of Kirillov and Dixmier maps in the light of this dequantization process. Ali Baklouti

Contents Preface | V 1 1.1 1.1.1 1.1.2 1.1.3 1.1.4 1.2 1.2.1 1.2.2 1.2.3 1.2.4 1.3 1.3.1 1.3.2 1.4 1.4.1 1.4.2 1.4.3

Structure theory | 1 Solvable Lie groups | 1 Solvable and exponential solvable Lie groups | 1 Heisenberg Lie groups | 5 Threadlike Lie groups | 6 Maximal subgroups of solvable Lie groups | 7 Euclidean motion groups | 10 On orthogonal matrices | 10 Some structure results | 13 Discrete subgroups of I(n) | 15 Closed subgroups of I(n) | 26 Heisenberg motion groups | 35 First preliminary results | 35 Discrete subgroups of Heisenberg motion groups | 36 Syndetic hulls | 43 Existence results for completely solvable Lie groups | 43 Case of exponential Lie groups | 44 Case of reduced exponential Lie groups | 48

2 2.1 2.1.1 2.1.2 2.1.3 2.1.4 2.1.5 2.2 2.3 2.4 2.4.1 2.4.2 2.4.3 2.4.4 2.4.5 2.5

Proper actions on homogeneous spaces | 51 Proper and fixed-point actions | 51 Discontinuous groups | 52 Clifford–Klein forms | 52 Weak and finite proper actions | 53 Campbell–Baker–Hausdorff series | 56 Proper actions and coexponential bases | 57 Proper actions for 3-step nilpotent Lie groups | 58 Special nilpotent Lie groups | 62 Proper actions on solvable homogeneous spaces | 65 Proper actions on special solvmanifolds | 65 Weak and finite proper actions on solvmanifolds | 67 Proper actions on maximal solvmanifolds | 72 Connected subgroups acting properly on maximal solvmanifolds | 72 From continuous to discrete actions | 74 Proper action for the compact extension K ⋉ ℝn | 78

X | Contents 2.5.1 2.6

Criterion for proper action | 80 Proper actions for Heisenberg motion groups | 84

3 3.1 3.1.1 3.1.2 3.1.3 3.2 3.2.1 3.2.2 3.3 3.3.1 3.3.2 3.3.3 3.3.4 3.4 3.4.1 3.4.2

Deformation and moduli spaces | 87 Deformation and moduli spaces of discontinuous actions | 87 Parameter, deformation and moduli spaces | 87 Case of effective actions | 89 Deformation of (G, X )-structures | 90 Algebraic characterization of the deformation space | 91 The deformation and moduli spaces in the exponential setting | 91 On pairs (G, H) having Lipsman’s property | 96 Case of Abelian discontinuous groups | 97 Analysis on Grassmannians | 98 The parameter space for normal subgroups | 102 The deformation space for normal subgroups | 103 Examples | 107 Non-Abelian discontinuous groups | 109 Structure of a principal fiber bundle | 109 The context where [Γ, Γ] is uniform in [G, G] | 113

4 4.1 4.1.1 4.1.2 4.1.3 4.1.4 4.1.5 4.1.6 4.1.7 4.1.8 4.1.9 4.2 4.2.1 4.2.2 4.2.3 4.3 4.3.1 4.3.2 4.3.3 4.3.4 4.3.5 4.4

The deformation space for nilpotent Lie groups | 117 Deformation and moduli spaces for Heisenberg groups | 117 A criterion of the proper action, continued | 117 The deformation space for non-Abelian actions | 118 Deformation and moduli spaces when H contains the center | 123 The case when H does not meet the center | 125 Case of compact Clifford–Klein forms | 133 Examples | 137 A smooth manifold structure on T (Γ, H2n+1 , H) | 140 Proof of Theorem 4.1.26 | 151 From H2n+1 to the product group H2n+1 × H2n+1 | 152 Case of 2-step nilpotent Lie groups | 156 Description of the deformation space T (l, g, h) | 159 Decomposition of Hom1 (l, g) | 160 Hausdorffness of the deformation space | 163 The 3-step case | 166 Some preliminary results | 166 On the quotient space Hom(l, g)/G | 170 Description of the parameter and the deformation spaces | 190 Hausdorffness of the deformation space | 195 Illustrating examples | 195 Deformation space of threadlike nilmanifolds | 200

Contents | XI

4.4.1 4.4.2 4.4.3 4.4.4 5 5.1 5.1.1 5.1.2 5.1.3 5.1.4 5.2 5.2.1 5.3 5.3.1 5.3.2 5.3.3 5.3.4 5.4 5.4.1

Description of Hom(l, g) | 200 Description of the parameter space R (Γ, G, H) | 203 Description of the deformation space T (Γ, G, H) | 207 Case of non-Abelian discontinuous groups | 213

5.4.2 5.4.3 5.4.4 5.4.5 5.5 5.5.1 5.5.2 5.5.3 5.5.4 5.6 5.6.1 5.6.2 5.6.3 5.7 5.7.1 5.7.2 5.7.3

Local and strong local rigidity | 225 The local rigidity conjecture | 226 The concept of (strong) local rigidity | 226 The nilpotent setting | 226 Case of 2-step nilpotent Lie groups | 227 The threadlike case | 228 Local rigidity for exponential Lie groups | 228 A local rigidity theorem where [L, L] = [G, G] | 230 Selberg–Weil–Kobayashi local rigidity theorem | 231 Maximal exponential homogeneous spaces | 232 Local rigidity for small-dimensional exponential Lie groups | 236 Passing through the quotients | 238 Proof of Theorem 5.3.9 | 240 Criteria for local rigidity | 242 Necessary condition for local rigidity using the automorphism group Aut(l) | 242 Case of graded Lie subalgebras | 244 Abelian discontinuous groups | 246 Removing the assumption on Γ to admit a syndetic hull | 248 Exponential Lie algebras of type T | 250 Local rigidity in the solvable case | 252 The notion of colored discrete subgroups | 253 The rank-one solvable case | 254 The setting where the action of G on G/H is effective | 258 The rank-two case | 260 The case of Diamond groups | 263 Dilation invariant subgroups | 266 Strong local rigidity results | 270 Proofs of Theorem 5.6.11 and Corollary 5.6.12 | 273 A local rigidity theorem for finite actions | 274 (Strong) local rigidity for K ⋉ ℝn | 277 (Strong) local rigidity for Heisenberg motion groups | 279 A variant of the local rigidity conjecture | 281

6 6.1 6.2

Stability concepts and Calabi–Markus phenomenon | 283 Stability concepts | 284 Stability of nilmanifold actions | 285

XII | Contents 6.2.1 6.2.2 6.2.3 6.2.4 6.2.5 6.3 6.3.1 6.4 6.4.1 6.4.2 6.4.3 6.4.4 6.5 6.5.1 6.5.2 6.5.3 6.5.4 6.5.5 6.5.6 7 7.1 7.1.1 7.1.2 7.1.3 7.1.4 7.1.5 7.1.6 7.2 7.2.1 7.2.2 7.2.3 7.2.4 7.2.5 7.2.6 7.3 7.3.1 7.3.2 7.3.3 7.3.4 7.3.5 7.4

Case of Heisenberg groups | 285 From H2n+1 to H2n+1 × H2n+1 /Δ | 287 Case of 2-step nilpotent Lie groups | 292 The threadlike case | 296 Stability of discrete subgroups | 307 Stability in the exponential setting | 309 Case of general compact Clifford–Klein forms | 309 Stability for Euclidean motion groups | 311 Geometric stability | 314 Near stability | 315 Case of crystallographic discontinuous groups | 316 Further remarks | 320 The Calabi–Markus phenomenon | 324 Case of Euclidean motion groups | 324 The case of SOn (ℝ) ⋉ ℝn | 327 Case of the semidirect product K ⋉ ℝn | 327 Case of Heisenberg motion groups | 327 Existence of compact Clifford–Klein forms | 328 Concluding remarks | 333 Discontinuous actions on reduced nilmanifolds | 335 Reduced Heisenberg groups | 335 Backgrounds | 335 r Discrete subgroups of H2n+1 | 336 A matrix-like writing of elements of Hom(Γ, G) | 338 Proper action on reduced homogeneous spaces | 346 The parameter space | 347 Stability of discrete subgroups | 348 r r r From H2n+1 to (H2n+1 × H2n+1 )/Δ | 349 Posed problems and main results | 350 r r Discontinuous groups for (H2n+1 × H2n+1 )/Δ | 351 The deformation parameters set Hom(Γ, G) | 353 The parameter space | 358 Proof of Theorem 7.2.1 | 359 Proof of Theorem 7.2.2 | 365 Reduced threadlike groups | 366 Proper action of closed subgroups of G | 367 Hom(Γ, G) for an Abelian discrete subgroup Γ | 369 Parameter and deformation spaces | 379 The local rigidity problem | 390 The stability problem | 391 A stability theorem for non-Abelian actions | 396

Contents | XIII

7.4.1 7.4.2 7.4.3 7.4.4

Description of Hom(Γ, G) | 397 Explicit determination of the parameter space | 404 Proof of Theorem 7.4.1 | 406 A concluding remark | 410

8 Deformation of topological modules | 411 8.1 Geometric objects associated with deformed algebras | 412 8.1.1 Topological modules on the ring of formal series | 413 8.1.2 Divisible ideals | 415 8.1.3 Cancelations | 415 8.1.4 Involutivity | 416 8.1.5 Characteristic manifolds | 418 8.1.6 Involution and ∗-representations | 419 8.1.7 Topologically convergent modules | 422 8.2 Case of linear Poisson manifolds | 422 8.2.1 The algebra (A , ∗) | 424 8.2.2 Converging topological free modules on (A , ∗) | 425 8.2.3 Unitarity | 426 8.3 The Zariski closure conjecture | 427 8.3.1 Kirillov–Bernat theory for exponential groups | 427 8.3.2 Construction of a representation πν of 𝒜 | 430 8.3.3 The characteristic variety | 432 8.3.4 Fundamental examples | 433 8.3.5 The Heisenberg groups | 434 8.3.6 The n-step threadlike Lie algebras | 436 8.3.7 The affine group of the real line | 437 8.3.8 A 3-dimensional exponential solvable Lie group | 438 8.3.9 The Zariski closure conjecture | 439 8.3.10 The nilpotent case | 440 8.3.11 First approach to Conjecture 8.3.10 | 440 8.3.12 Partial solution of Conjecture 8.3.10 | 443 8.3.13 The normal polarization case | 445 8.3.14 The nilpotent case revisited | 446 8.3.15 Dixmier and Kirillov maps | 448 8.4 Some nonexponential restrictive cases | 449 8.4.1 The two-dimensional motion group | 449 8.4.2 The diamond groups | 452 8.4.3 The semisimple case: Verma modules | 455 8.5 A deformation approach of the Kirillov map | 459 8.5.1 Poisson ideals | 459 8.6 Type-I-ness and consequences | 462 8.6.1 Linear Poisson manifolds and primitive Poisson ideals | 462

XIV | Contents 8.6.2 8.6.3 8.6.4 8.6.5

The unitary dual and the orbit space | 463 The Tangent groupoid and Rieffel’s strict quantization | 463 One-parameter families of representations | 464 Kirillov map revisited | 470

Bibliography | 473 Index | 479

1 Structure theory The material on solvable Lie groups and Lie algebras is quite standard; the readers may consult [42] and [59] and many references therein. The goal of the chapter is to record some main backgrounds on nilpotent, solvable and exponential solvable Lie groups and some of their compact extensions. We first recall the definitions of real solvable and nilpotent Lie algebras and define their associated Lie groups. We also go through some details with famous and important examples, such as Heisenberg groups, threadlike groups, Euclidean motion groups and Heisenberg motion groups. An explicit description of closed and discrete subgroups is also provided for further developments and use. We also define the notion of the syndetic hull of a closed subgroup of a Lie group, and prove existence and unicity results in the case of (reduced) completely solvable Lie groups. In the setup of exponential solvable Lie groups, this has major interest and use in the computation of the parameter, deformation and moduli spaces of the action of discontinuous groups on homogeneous spaces as will be dealt with throughout the book.

1.1 Solvable Lie groups 1.1.1 Solvable and exponential solvable Lie groups Let gℂ denote the complexified Lie algebra of a given Lie algebra g, where ℝ ⊂ ℂ denote the fields of complex and real numbers. Let adX , X ∈ g be the adjoint endomorphism defined by adX (Y) = [X, Y], where [, ] denotes the Lie bracket. When [X, Y] = 0 for any X, Y ∈ g, we say that g is commutative or Abelian. A linear subspace a of g is an ideal if [X, a] ⊂ a for any X ∈ g. Then the quotient space g/a endowed with the Lie bracket induced from g, becomes a Lie algebra. Let Dg = [g, g] = ℝ-span{[X, Y]; X, Y ∈ g}, then obviously Dg is an ideal of g and the quotient Lie algebra g/Dg is commutative. We define the following sequences of vector spaces: g = D0 g ⊃ D1 g ⊃ D2 g ⊃ ⋅ ⋅ ⋅

(1.1)

g = C0 g ⊃ C1 g ⊃ C2 g ⊃ ⋅ ⋅ ⋅ ,

(1.2)

and

https://doi.org/10.1515/9783110765304-001

2 | 1 Structure theory where D0 g = g,

Dk g = D(Dk−1 g)

(k = 1, 2, . . .),

C 0 g = g,

C k g = [g, ck−1 g]

(k = 1, 2, . . .).

and

In both cases, the decreasing sequences (1.1) and (1.2) turn out to be respectively flags of ideals of g. The Lie algebra g is said to be solvable, if there exists k such that Dk g = {0} and nilpotent if there exists k such that C k g = {0}. On the other hand, g is said to be m-step nilpotent (for some integer m), if C m g ≠ {0} and C m+1 g = {0}. Besides, a connected Lie group is said to be solvable (resp., nilpotent, m-step nilpotent) if its Lie algebra is solvable (resp., nilpotent, m-step nilpotent). Comparing (1.1) and (1.2), nilpotent Lie groups are solvable Lie groups, but the gap between these two classes is quite wide. Here, we define some classes of solvable non-necessarily nilpotent Lie groups. Let exp = expG : g → G

(1.3)

be the exponential mapping of G. Definition 1.1.1. Let G be a solvable Lie group and g its Lie algebra. Then G said to be exponential solvable if, the exponential mapping (1.3) is a C ∞ -diffeomorphism from g onto G. In this case, let log = logG designate the inverse map of expG . Any exponential solvable Lie group is therefore connected and simply connected. The following result provides a characterization of such a class. Theorem 1.1.2 ([106]). Let G be a connected, simply connected and solvable Lie group and let g be its Lie algebra. Then the following are equivalent: (1) G is exponential solvable. (2) expG is an injective mapping. (3) expG is a surjective mapping. (4) No endomorphism adX , X ∈ g has purely imaginary eigenvalues. Definition 1.1.3. Let g be a Lie algebra such that dim g = n. When there exists a sequence of ideals, {0} = g0 ⊂ g1 ⊂ ⋅ ⋅ ⋅ ⊂ gn−1 ⊂ gn = g,

dim gj = j

(0 ≤ j ≤ n),

we say that g is completely solvable. Any completely solvable Lie algebra is an exponential solvable Lie algebra for which any endomorphism adX , X ∈ g has only real

1.1 Solvable Lie groups |

3

eigenvalues. A connected Lie group is said to be completely solvable, if its Lie algebra is completely solvable. Example 1.1.4 (Affine group of the real line). Consider the 2-dimensional Lie group defined by a Aff(ℝ) = {( 0

b ) , a, b ∈ ℝ and a > 0} , 1

known to be the “ax + b” group. Let g = ℝX ⊕ ℝY denote its Lie algebra, where 1 0

X=(

0 ) 0

0 0

and Y = (

1 ) 0

and whose Lie bracket is given by [X, Y] = Y. Then g is a completely solvable Lie algebra said to be the Lie algebra of the affine group Aff(ℝ) of the real line, and will be briefly denoted by aff(ℝ). The following notion of coexponential bases to closed subgroups of connected Lie groups (cf. [106]) plays a capital role in the sequel. Definition 1.1.5. Let G be a connected and simply connected Lie group and let H be a closed connected subgroup of G. Let g, h be the Lie algebras of G and H, respectively. A basis {X1 , . . . , Xp }, p = dim(g/h), is said to be coexponential to h in g if the map: φg,h : ℝp × H

((t1 , . . . , tp ), h)

→ 󳨃→

G exp tp Xp ⋅ ⋅ ⋅ exp t1 X1 ⋅ h

is a diffeomorphism. We have the following. Theorem 1.1.6 ([106, Proposition 2]). Let G be a connected, simply connected and solvable Lie group. Then every connected closed subgroup of G admits a coexponential basis. Remark 1.1.7. A constructive proof of Theorem 1.1.6 is based on the three following assertions: (𝚤) If h is a one-codimensional ideal of g, then any vector in g ∖ h is a coexponential basis. (𝚤𝚤) If g ⊃ h′ ⊃ h and {b1 , . . . , bd }, respectively, {c1 , . . . , cr } is a coexponential basis for h′ in g, respectively, for h in h′ , then {b1 , . . . , bd , c1 , . . . , cr } is a coexponential basis for h in g. (𝚤𝚤𝚤) If h is a maximal subalgebra of g, which is not an ideal of g, then any coexponential basis for h ∩ [g, g] in [g, g] is also a coexponential basis for h in g.

4 | 1 Structure theory As a direct consequence from Remark 1.1.7, we get the following lemmas (cf. [106]). Lemma 1.1.8. Let h be a subspace of g containing [g, g] and V a linear subspace of g such that g = h ⊕ V. Then any basis of V is a coexponential basis to h in g. Proof. Take a basis {u1 , . . . , ur } of V. As h contains [g, g], then the sequence h0 = h ⊂ ⋅ ⋅ ⋅ ⊂ hr , where hi = ℝ-span{u1 , . . . , ui } ⊕ h is a sequence of ideals of g. Then the result comes directly from assertions (𝚤) and (𝚤𝚤). Lemma 1.1.9. Let h be a subalgebra of g such that h + [g, g] = g. Then there exists a coexponential basis {u1 , . . . , ur } to h in g such that [g, g] = W ⊕ h ∩ [g, g], where W is the linear span of the family {u1 , . . . , ur }. Definitions 1.1.10. (1) With the notation above, a strong Malcev (or Jordan–Hölder) basis {Z1 , . . . , Zm } of g is a basis of g such that gi = ℝ-span{Z1 , . . . , Zi } is an ideal of g for every i ∈ {1, . . . , m}. The obtained flags of ideals {0} = g0 ⊂ g1 ⊂ ⋅ ⋅ ⋅ ⊂ gm = g is called strong Malcev (or Jordan–Hölder) sequence of g. It is well known that such basis exists if g is nilpotent. (2) A family of vectors {X1 , . . . , Xp }, p = dim(g/h) is said to be a Malcev basis of g relative to h if g = span{X1 , . . . , Xp } ⊕ h and for all i ∈ {1, . . . , p}, the vector space ℝ-span{X1 , . . . , Xi } ⊕ h is a subalgebra of g. Remark 1.1.11. (1) It is well known that nilpotent Lie groups admit Jordan–Hölder bases. (2) Let g be a nilpotent Lie algebra and let B = {Z1 , . . . , Zm } be a strong Malcev basis of g and 0

1

Sg : {0} = g ⊂ g ⊂ ⋅ ⋅ ⋅ ⊂ g

m−1

⊂ gm = g

(1.4)

the associated Jordan–Hölder sequence of g associated to the basis B . Here, gi = ℝ-span{Z1 , . . . , Zi }, i = 1, . . . , m. Such a notion of basis is very important and will be crucially used to adapt our setup with convenient coordinates, which provide an adequate model for the adjoint action. Given a subalgebra h of g, we denote by Igh = {i1 < ⋅ ⋅ ⋅ < ip } (p = dim h) the set of indices i (1 ≤ i ≤ m) such that h + gi−1 = h + gi . We note for all is ∈ Igh , Z̃ s = Zis + ∑r 2 and that the result holds for any integer k < n. Let Oi1 , i1 ∈ J having an eigenvalue λi1 ≠ ±1. Consider Fλi = Eλi ⊕ Eλ 1

1

i1

and for any j ∈ J, Oj,i1 , the matrix corresponding to the restriction endomorphism associated to Oj on Fλi . The family {Oj,i1 }j∈J , is thus a family of commuting diagonalizable 1 matrices in M2p1 (ℂ), for which 2p1 = dim(Fλi ). 1 If 2p1 = n, then if the spectrum of some Oj , j ∈ J contains ±1 as an eigenvalue, then it only does with an even multiplicity. On the other hand, there exists a common unitary basis of eigenvectors (v1 , . . . , vp1 ) of Eλi and (v1 , . . . , vp1 ) of Eλ , which we arrange i1

1

as (v1 , v1 , . . . , vp1 , vp1 ) to obtain a basis of Fλi . This allows to get the result in this case. 1 More generally, fixing the complex eigenvalues λi1 , . . . , λik of some Oi1 , . . . , Oik , respectively, for i1 , . . . , ik ∈ J, one can write ℂn = Fλi ⊕ ⋅ ⋅ ⋅ ⊕ Fλi ⊕ Hk , 1

k

(1.16)

where Hk denotes the orthogonal supplementary subspace of Fλi ⊕ ⋅ ⋅ ⋅ ⊕ Fλi . Now, for 1 k any j ∈ J, the restriction of Oj to Hk admits no nonreal eigenvalues. So if dim(Hk ) = 0 we are done; otherwise, the spectrum of Oj |H is sitting inside {−1, 1} for any j ∈ J and, k therefore, Oj |H coincides with k

Aj (−1, 1) = (

±1

..

) ∈ Mq (ℝ),

. ±1

where q = dim Hk . Let E(±1) = ⋂ ker(Oj ∓ I), j∈J

1.2 Euclidean motion groups | 13

and H(−1, 1) the orthogonal supplementary of E(1) ⊕ E(−1) in Hk , according to the decomposition (1.16). This yields the following refined decomposition of ℝn as ̃k , ℝn = F1 ⊕ ⋅ ⋅ ⋅ ⊕ Fl ⊕ H

(1.17)

̃k is the orthogonal supplement where 2l = ∑kj=1 dim Fλi , dimFi = 2 for i ∈ {1, . . . , l} and H j

of F1 ⊕ ⋅ ⋅ ⋅ ⊕ Fl and the spectrum of Oj |H̃ is sitting in ℝ. If m+ = dim E(1), m− = dim E(−1) k and m± = dim H(−1, 1), there exists S ∈ On (ℝ) such that for any j ∈ J, Im+ −Im− ( ( S−1 Oj S = ( (

dj (−1, 1)

r(θ1,j )

..

.

(

) ) ), )

(1.18)

r(θl,j ))

where for any j ∈ J, ε1,j

dj (−1, 1) = (

..

.

εm± ,j

) ∈ Mm± (ℝ),

εk,j ∈ {−1, 1}

for k ∈ {1, . . . , m± }.

This achieves the proof with the convention that if one of the integers m+ , m− , m± , l is zero, the corresponding block does not show up. Note here that according to the matrix form in Lemma 1.2.1 above, ℝn decomposes into direct sums of subspaces as follows: ℝn = E(1) ⊕⊥ E(−1) ⊕⊥ H(−1, 1) ⊕⊥ F1 ⊕⊥ ⋅ ⋅ ⋅ ⊕⊥ Fl ,

(1.19)

where E(±1) = ⋂j∈J ker(Oj ∓ I), Fi are two-dimensional subspaces of ℝn and H(−1, 1) is an orthogonal supplement for which the restriction S−1 Oi S coincides with di (−1, 1). 1.2.2 Some structure results For any g = (A, x) ∈ G, denote by O(g) and O(A) ∈ ℕ∗ ∪ {∞}, the orders of g and A, respectively. Let also EA (1) = ker(A − I) and let P(A) be the orthogonal projection on EA (1). We have the following. Lemma 1.2.2. P(A) is a polynomial upon A. In particular, if A is of the finite order p, say, then P(A) =

1 (I + A + ⋅ ⋅ ⋅ + Ap−1 ). p

(1.20)

14 | 1 Structure theory Proof. The first statement is a well-known fact. Assume that A has a finite order p. If x ∈ EA (1), then clearly P(A)(x) = x. On the other hand, equation (1.20) says that P(A)(I − A) = 0. As such, I − A induces an isomorphism on EA (1)⊥ . Then if y ∈ EA (1)⊥ , there exists y′ ∈ EA (1)⊥ such that y = (I − A)y′ and, therefore, P(A)y = P(A)(I − A)y′ = 0. Let pr1 and pr2 designate the natural projections from I(n) onto On (ℝ) and ℝn , respectively. Then equation (1.14) says that pr1 (Γ) is a subgroup of On (ℝ) for any subgroup Γ of G. Note that pr2 (Γ) is not a subgroup of ℝn in general. Take indeed Γ generated by (A, x) such that x ≠ 0 and A is of order k ∈ ℕ∗ such that P(A)(x) = 0. Then p

pr2 (Γ) = { ∑ Ai x, p ∈ {0, . . . , k − 1}}, i=0

which is not a subgroup of ℝn . For discrete subgroups of G, the following appears to be immediate. Lemma 1.2.3. Let Γ be a discrete subgroup of G. Then Γ is infinite if and only if pr2 (Γ) is also. Proof. Indeed, let G be a topological group, K a compact subgroup of G and Γ a discrete subgroup of G. Then Γ is finite if and only if it has a finite orbit in G/K, which is enough to conclude. Remark 1.2.4. One can develop another elementary proof of Lemma 1.2.3. Indeed, given an infinite discrete subgroup Γ for which pr2 (Γ) = {x1 , . . . , xk } (k ∈ ℕ∗ ), one can write k

Γ = ⋃ Axi × {xi }, i=1

where Axi = {A ∈ On (ℝ) | (A, xi ) ∈ Γ}. There exists therefore i0 ∈ {1, . . . , k} such that the set Axi × {xi0 } is an infinite set sitting inside the compact set On (ℝ) × {xi0 }, so it cannot 0 be discrete. Lemma 1.2.5. Let Γ be a discrete subgroup of G and γ = (A, x) ∈ Γ. Then O(γ) = p if and only if O(A) = p and P(A)(x) = 0. That is, for γ ∈ Γ, O(γ) = +∞ if and only if P(A)(x) ≠ 0. Proof. Let p = O(γ). Remark first that p−1

γ p = (Ap , ( ∑ As )x) = (I, 0). s=0

1.2 Euclidean motion groups | 15

Then the order of A divides, that is, of γ. If q = O(A) < p, then q−1

γ q = (I, ( ∑ As )x) s=0

p−1 s s and, therefore, O(γ) = ∞ if ever (∑q−1 s=0 A )x ≠ 0. Hence, q = p and (∑s=0 A )x = pP(A)(x) = 0. The converse is trivial.

The following establishes a necessary and sufficient condition for two elements to be conjugate. Lemma 1.2.6. Two elements (A, x) and (A′ , x′ ) are conjugate in G if and only if there exists S ∈ On (ℝ) such that S−1 AS = A′ and Sx ′ − x ∈ EA (1)⊥ . In particular, if (A, x) and (A′ , x′ ) are of finite orders, then they are conjugate in G if and only if A and A′ are conjugate in On (ℝ). Proof. Let g = (S, y) ∈ G. A direct computation shows that g −1 (A, x)g = (S−1 AS, S−1 [x + (A − I)y]).

(1.21)

So A′ = S−1 AS and x′ = S−1 [x + (A − I)y]. Therefore, Sx′ − x = (A − I)y, which gives in turn that P(A)(Sx′ − x) = 0, and finally Sx ′ − x ∈ EA (1)⊥ . Conversely, if A′ = S−1 AS and Sx′ − x ∈ EA (1)⊥ , then the equation (A − I)z = Sx′ − x has at least a solution, say z0 . Take y = z0 , then (A − I)y = Sx ′ − x and, therefore, x′ = S−1 [x + (A − I)y]. This gives that (A′ , x′ ) = (S−1 , −S−1 y)(A, x)(S, y). If (A, x) and (A′ , x′ ) are of finite orders, then by Lemma 1.2.5, x ∈ EA (1)⊥ and x ′ ∈ EA′ (1)⊥ . So if there exists S ∈ On (ℝ) such that A′ = S−1 AS, then Sx′ ∈ EA (1)⊥ and also Sx′ − x ∈ EA (1)⊥ . Then the arguments above allow to conclude. 1.2.3 Discrete subgroups of I(n) We now prove our first upshot. We have the following.

16 | 1 Structure theory Proposition 1.2.7. Let {γi }i∈J be a commuting family of G. There exist some integers m+ , m− , m± and l satisfying m+ + m− + m± + 2l = n and g ∈ G such that for any i ∈ J, Im+ −Im− ( ( ( ( g −1 γi g = ( ( ( (

di (−1, 1)

r(θ1,i )

..

.

( (

ym+ ,i 0m− ) ( 0m ) ) ) ( ±) ) ),( 0 ) ), ) ( 2 ) ) .. . r(θl,i )) ( 02 ) )

(1.22)

where ym+ ,i ∈ ℝm+ , ε1,j

dj (−1, 1) = (

..

.

εm± ,j

) ∈ Mm± (ℝ)

(εk,j ∈ {−1, 1} for k ∈ {1, . . . , m± }),

θ1,i , . . . , θl,i ∈ ℝ and 02 , 0m− and 0m± are the zeros of ℝ2 , ℝm− and ℝm± , respectively. Proof. Let γi = (Ai , xi ) for i ∈ J. A direct application of Lemma 1.2.1 shows that there exists S ∈ On (ℝ) such that for i ∈ J, (S, 0)−1 γi (S, 0) = (S−1 Ai S, S−1 xi ) := δi is of the form Im+ −Im− ( ( ( ( δi = ( ( ( (

di (−1, 1)

r(θ1,i )

( (

..

.

ym+ ,i ym− ,i ) ) (zm±,i ) ) ) ) ( ), ),( x ) ) ) ( 1,i ) .. .

r(θl,i )) ( xl,i ) )

where the integers m+ , m− , m± and l are as in Lemma 1.2.1. Trivially, they do not depend upon i ∈ J and if one of them is zero. This reduces to the fact that the correspondent block of the ℝn side does not show up. For any i, j ∈ J, Im+ ( ( ( ( δi δj = ( ( ( ( ( (

Im−

di,j (−1, 1)

r(θ1,i,j )

..

.

ym+ ,i + ym+ ,j ym− ,i − ym− ,j ) ( ) ) z ) (m±,i + di (−1, 1)zm±,j ) ) ) , ( x + r(θ )x ) ), 1,i 1,j ) ) ( 1,i ) .. .

r(θl,i,j )) ( xl,i + r(θl,i )xl,j ) )

1.2 Euclidean motion groups | 17

where di,j (−1, 1) = di (−1, 1)dj (−1, 1) and θs,i,j = θs,i + θs,j for 1 ≤ s ≤ l. The fact that γi and γj commute for any i, j ∈ J says that ym− ,i = ym− ,j := ym− and that (di (−1, 1), zm± ,i ) and (dj (−1, 1), zm± ,j ) of I(m± ) commute. Let us opt for the notation: εi,1

..

(di (−1, 1), zm± ,i ) = ((

τi,1 . ) , ( .. )) εi,m± τi,m±

.

(i ∈ J),

where for k ∈ {1, . . . , m± }, εi,k ∈ {−1, 1} and τi,k ∈ ℝ. Hence, the commutativity condition says (1 − εj,k )τi,k = (1 − εi,k )τj,k . As εik ,k = −1, for some ik ∈ J, then (1 − εj,k ) 2

τik ,k = τj,k ,

and τj,k = 0 whenever εj,k = 1. Let for k ∈ {1, . . . , m± }, τk = τik ,k . Then εi,1

(di (−1, 1), zm± ,i ) = ((

..

1−εi,1 τ1 2

.

εi,m±

),(

.. .

1−εi,m± τm± 2

)) .

Let, on the other hand, for all k ∈ {1, . . . , l} the integer ik ∈ J such that r(θk,ik ) := r(θk ) ≠ I2 and xk′ := xk,ik . So for j ∈ J, xk′ + r(θk )xk,j = xk,j + r(θk,j )xk′ , and equivalently, xk,j = (I2 − r(θk,j ))(I2 − r(θk )) xk′ , −1

(1.23)

18 | 1 Structure theory for any j ∈ J. Let xk = (I2 − r(θk ))−1 xk′ . Then (S, 0)−1 γi (S, 0) takes the following form:

ym+ ,i ym− 1−εi,1 τ1 2 .. .

Im+ −Im− ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( (

εi,1

..

.

εi,m±

r(θ1,i )

..

.

( (

) ( ) ) ) ( ) ) ) ( ) ) ) ( ) ) ) ( ) ) ) , ( 1−εi,m± ) ). ) ( ) ) τm± ) ) ( ) 2 ) ((I − r(θ ))x ) ) ) ( 2 ) 1,i 1) .. .

r(θl,i )) ( (I2 − r(θl,i ))xl ) )

Now, for t = t (t 0m+ , 21 t ym− , 21 τ1 , . . . , 21 τm± , t x1 , . . . , t xl ) and g = (S, 0)(I, t) = (S, St), one gets

Im+ −Im− ( ( ( ( ( ( ( ( ( ( −1 g γi g = ( ( ( ( ( ( ( ( ( (

εi,1

..

.

εi,m±

r(θ1,i )

( (

..

.

ym+ ,i 0m− ) ) ( 0 ) ) ) ) ( ) ) ( . ) ) ) ( .. ) ) ) ) ( ). ) ),( ) ) ( 0 ) ) ) ) ( ) ) ( 0 ) ) ) ( 2 ) .. . r(θl,i )) ( 02 ) ) (1.24)

Generation families of discrete subgroups of I(n)

We first pose the following.

Definition 1.2.8. The rank of a group is the smallest cardinality of a family of its generators.

We next prove the following. Lemma 1.2.9. Let Γ be a subgroup of G. Then Γ is finite if and only if pr1 (Γ) is finite and

any element in Γ is of finite order. Furthermore, if Γ is compact, then Γ∩{I}×(ℝn \{0}) = 0. Proof. The necessary condition is trivial. For the converse, suppose that pr1 (Γ) =

{A1 , . . . , Ap } and suppose that Γ is infinite, which is equivalent to the fact that pr2 (Γ) is

1.2 Euclidean motion groups | 19

infinite as in Lemma 1.2.3. Then Γ = ⨆ {(Ai , x), (Ai , x) ∈ Γ}, 1≤i≤p

and necessarily there exists i0 ∈ {1, . . . , p} such the set {(Ai0 , x), (Ai0 , x) ∈ Γ} is infinite. So for x′ ≠ x, (Ai0 , x)(Ai0 , x′ )−1 = (I, x − x′ ) ≠ (I, 0) is an element of Γ of infinite order. This is absurd. If now Γ contains (I, y), y ≠ 0, then {(I, my), m ∈ ℤ} ⊂ Γ. So Γ cannot be compact. The compact subgroups of G are of paramount importance along this work, and we quote the following result (cf. [82]). Fact 1.2.10 ([82, Lemma 14.1.1]). Let G = K ⋉ V be the semidirect product Lie group, where V is a finite-dimensional vector space and K is compact. Then for any compact subgroup U ⊆ G, there exists v ∈ V with vUv−1 ⊆ K. Corollary 1.2.11. Let G = K ⋉ V be the semidirect product Lie group, where V is a finitedimensional vector space and K is compact. Then for any subgroup U ⊆ G such that π2 (U) is compact, there exists v ∈ V with vUv−1 ⊆ K. Here, π2 designates the projection from K ⋉ V into V. Proof. Consider a sequence {(Bp , zp )}p∈ℕ inside U, which converges to some (B, z) ∈ U, where U is the closure of U. Since π2 (U) is compact, then obviously the sequence {zp }p∈ℕ converges to z ∈ π2 (U). Hence, U is compact. By Fact 1.2.10, there exists v ∈ V such that vUv−1 ⊆ vUv−1 ⊆ K. Let us announce next the following result, which will be of use. Lemma 1.2.12 (cf. [115, Lemma 4 and Lemma 5]). Let γ1 = (A, x) and γ2 = (B, y) be in G such that γ1 and γ2 generate a discrete subgroup of G. If ‖A − I‖ < 21 and ‖B − I‖ < 21 then γ1 and γ2 commute. We now prove the following. Proposition 1.2.13. Let Γ be a discrete subgroup of G. Then Γ is finite if and only if any element of Γ is of finite order. Proof. When Γ is finite, the statement is clear. Thanks to Lemma 1.2.9, it is sufficient to prove that pr1 (Γ) is finite. Assume then that pr1 (Γ) is infinite (so not discrete), there exists an infinite sequence {γp = (Ap , xp )}{p∈ℕ} in Γ, for which the sequence {Ap }{p∈ℕ} converges to I. As above, for a given p0 ∈ ℕ we have ‖Ap −I‖ < 21 for p ≥ p0 . Then for any p > q ≥ p0 , γp and γq commute thanks to Lemma 1.2.12. Now, apply Proposition 1.2.7

20 | 1 Structure theory to the commuting family {γp }{p≥p0 } to get that γp is conjugate to Im+ −Im− ( ( ( ( ( ( ( ( ( ( ′ γp = ( ( ( ( ( ( ( ( ( (

ε1,p

..

.

εm± ,p

( (

r(θ1,p )

..

.

ym+ ,p 0m− ) ) ( 0 ) ) ) ) ( ) ) ( . ) ) ) ( .. ) ) ) ) ( ), ) ),( ) ) ( 0 ) ) ) ) ( ) ) ( 0 ) ) ) ( 2 ) .. . r(θl,p )) ( 02 ) )

where r(θi,p ) ∈ SO2 (ℝ) for i ∈ {1, . . . , l}, and εi,p ∈ {−1, 1} for i ∈ {1, . . . , m± }. Thanks to Lemma 1.2.5, we get ym+ ,p = 0 and then an infinite sequence inside the compact On (ℝ) × {0}. We end up with a convergent infinite extract sequence inside Γ. This is absurd and then pr1 (Γ) is finite. The following is thus immediate. Corollary 1.2.14. A discrete subgroup Γ of G is infinite if and only if it contains an element of infinite order. The following result, known as Bieberbach’s theorem (cf. [43] and [44]), for which a simpler proof can be found in [115, Theorem 1] will be next utilized. Theorem 1.2.15. Any discrete subgroup Γ of G contains an Abelian normal subgroup of finite index in Γ. We next record the following. Theorem 1.2.16 (cf. [84, main theorem]). A closed solvable subgroup of a locally compact, almost connected group G′ (the quotient G′ /G0′ is compact, where G0′ designates the of G′ ) is compactly generated. Now the following is immediate. Corollary 1.2.17. A discrete subgroup of an Euclidean motion group is finitely generated. Proof. Let Γ be a discrete subgroup of an Euclidean motion group. By Theorem 1.2.15, Γ admits an Abelian normal subgroup Γa of finite index q say in Γ. Let then Γ/Γa = {e, δ1 , . . . , δq−1 }, where e designates the identity of G. By Theorem 1.2.16, the Abelian discrete subgroup Γa is finitely generated since G is almost connected. Let then {γ1 , . . . , γk } be a family of generators of Γa for some positive integer k, then {γ1 , . . . , γk , δ1 , . . . , δq−1 } generates Γ.

1.2 Euclidean motion groups | 21

As above, our strategy is to get a full description of Abelian discrete subgroups of G up to a conjugation. Let Γ be a discrete subgroup of G and Γ∞ a subgroup of Γ fulfilling Theorem 1.2.15. Lemma 1.2.18. There exists t ∈ ℝn such that for any (A, x) ∈ Γ∞ , we have (I, t)(A, x)(I, −t) = (A, PA (x)). In addition, for any (A, x) and (B, y) in Γ∞ , we get (A, PA (x))(B, PB (y)) = (AB, PA (x) + PB (y)).

(1.25)

Proof. Since Γ∞ is Abelian, then it follows from Lemma 1.2.1 that there exist some integers m+ , m− , m± and l satisfying m+ + m− + m± + 2l = n and g = (S, v) ∈ I(n) such that, for any γ = (A, x) ∈ Γ∞ , Im+ −Im− ( ( ( ( gγg −1 = ( ( ( (

d(−1, 1)

r(θ1 )

( (

..

.

xm+ 0m− ) ) (0m ) ) ) ( ±) ), ),( 0 ) ) ) ( 2) .. .

r(θl )) ( 02 ) )

is as in equation (1.22). Here, xm+ ∈ ℝm+ , ε1

d(−1, 1) = (

..

.

εm±

) ∈ Mm± (ℝ),

εk ∈ {−1, 1},

for k ∈ {1, . . . , m± }, cos θi sin θi

r(θi ) = (

− sin θi ), cos θi

with θi ∈ ℝ, for i ∈ {1, . . . , l} and 02 , 0m− and 0m± are the zeros of ℝ2 , ℝm− and ℝm± , respectively. Set x′ = t (xm+ , 0m− , 0m± , 02 , . . . , 02 ), then clearly x ′ ∈ ker SAS−1 − I. Hence, S−1 x′ ∈ ker A − I. As x′ = S(x + (I − A)S−1 v) = S ⋅ PA (x), it is enough to take t = S−1 v. Let now (A, x) and (B, y) be two elements of Γ∞ and set (A′ , x ′ ) = g(A, x)g −1 and ′ ′ (B , y ) = g(B, y)g −1 . Then equation (1.22) gives (A′ , x ′ )(B′ , y′ ) = (A′ B′ , x ′ + y′ ). Hence, A′ y′ = y′ , which gives in turn APB (y) = PB (y). This completes the proof. One important class of discrete subgroups was studied by Bieberbach (cf. [43] and [44]) and is defined as follows.

22 | 1 Structure theory Definition 1.2.19. A discrete subgroup Γ of a semidirect product K ⋉ ℝn is said to be crystallographic if it contains an Abelian normal subgroup of finite index, which is generated by n free translations. We now quote the following due to Bieberbach (cf. [43] and [44]). Lemma 1.2.20 (Bieberbach). Let Γ1 and Γ2 be two isomorphic crystallographic subgroups of G. For any isomorphism φ from Γ1 into Γ2 , there exists g ∈ GLn (ℝ) ⋉ ℝn such that φ(γ) = gγg −1 , for any γ ∈ Γ1 . We now record the following result proved in [121]. Theorem 1.2.21. If W is a subgroup of a free group F, then W is a free group. Moreover, if W has finite index m in F, the rank of W is precisely nm + 1 − m where n is the rank of F. As a consequence, we get the following. Lemma 1.2.22. Any discrete free subgroup of an Euclidean motion group is cyclic (and, therefore, Abelian). Proof. Let Γ be a discrete free subgroup of an Euclidean motion group G. Due to Theorem 1.2.15, Γ admits an Abelian normal subgroup Γa of finite index q say in Γ. By Corollary 1.2.17, Γ and Γa are finitely generated, and let p denote the rank of Γ and l that of Γa . Theorem 1.2.21 gives that Γa is free and, therefore, Γa is trivial or isomorphic to ℤ. Since Γ is torsion-free, then Γa is isomorphic to ℤ, and hence, l = 1, which gives in turn that 1 = 1 + q(p − 1), and conclusively q(p − 1) = 0. This allows to conclude that p = 1 and then Γ is cyclic. One important issue to study the deformation of discontinuous groups of Euclidean motion groups is to have an idea about how the generators of a discrete subgroup look like. First, we prove the following. Proposition 1.2.23. Let {y1 , . . . , yk } be a family of vectors of rank k generating a subspace E of ℝn . Let also Σ be a subgroup of On (ℝ) leaving invariant E. There exists S ∈ On (ℝ) such that: (i) S−1 yi = (

(ii)

yk,i ), 0n−k

for some yk,i ∈ ℝk and where 0n−k is the zero vector of ℝn−k . For any A ∈ Σ, Ck,A 0

S−1 AS = (

′ for some Ck,A ∈ Ok (ℝ) and Cn−k,A ∈ On−k (ℝ).

0 ) ′ Cn−k,A

1.2 Euclidean motion groups | 23

Proof. Consider the transition matrix S from the standard basis of ℝn to BE ∪BE ⊥ where BE and BE ⊥ designate two orthonormal bases of E and E ⊥ , respectively. Immediately, S ∈ On (ℝ) and it satisfies (i) and (ii). The following upshot describes to some extent the form of generators of a discrete subgroup. Theorem 1.2.24. Let G = I(n) be the Euclidean motion group. For any infinite discrete subgroup Γ of G, there exists g ∈ G such that the subgroup Γg := g −1 Γg admits generators {γ1 , . . . , γk , γk+1 , . . . , γk0 , δ1 , . . . , δq−1 } meeting the following: For 1 ≤ i ≤ k0 , Ik

Im+ −k

( ( ( ( ( ( γi = ( ( ( ( ( (

−Im−

di (−1, 1)

r(θ1,i )

( (

..

.

yk,i 0m+ −k ) ) ( 0m− ) ) ) ) ( ) ) ( 0m ) ), ),( ± ) ) ) ( 0 ) ) ) ( 2 ) .. . r(θl,i )) ( 02 ) )

where yk,i ∈ ℝk satisfying {yk,1 , . . . , yk,k } is of rank k, for k + 1 ≤ i ≤ k0 , yk,i = 0k and where for 1 ≤ i ≤ k0 , di (−1, 1) is a diagonal matrix with coefficients ±1, of Om± (ℝ) and for any 1 ≤ s ≤ l, θs,i ∈ ℝ. Furthermore, the family {γi : 1 ≤ i ≤ k0 } generates an Abelian normal subgroup Γga of Γg such that Γg /Γga = {e, δ1 , . . . , δq−1 } where for 1 ≤ i ≤ q − 1, Ck,i ( δi = ( ( (

Cm+ −k,i

zk,i

Cm− ,i

Cm± ,i

0m+ −k ). ) , ( 0m ) − 0m± Pi ) ( 02l ) )

Here, Ck,i , Cm+ −k,i , Cm− ,i , Cm± ,i and Pi belong to Ok (ℝ), Om+ −k (ℝ), Om− (ℝ), Om± (ℝ) and O2l (ℝ), respectively. The integers m+ , m− , m± and l satisfy m+ + m− + m± + 2l = n. Proof. First, remark that Γ admits a normal Abelian subgroup Γa of the finite index q, say, in Γ due to Theorem 1.2.15. Let {a1 , . . . , ak0 } be some generators of Γa , then the

family {a1 , . . . , ak0 , b1 , . . . , bq−1 } generates Γ, where {e, b1 , . . . , bq−1 } is a complete representative of the finite set Γ/Γa . Thanks to Lemma 1.2.1, there exists g ∈ G such that the subgroup Γga := g −1 Γa g is now being generated by the family {γi = g −1 ai g}1≤i≤k0 as in formula (1.22). If ym+ ,i = 0 for any i ∈ {1, . . . , k0 } or m+ = 0, then thanks to Lemma 1.2.5 all the γi ’s are of finite orders and, therefore, due to Lemma 1.2.9, Γ is finite. Otherwise, let k ≤ k0 be the rank of the family {ym+ ,i }1≤i≤k0 and let L be the subgroup of ℝm+ generated by {ym+ ,i }1≤i≤k0 . Suppose that there exists an infinite sequence {ym+ (p)}p∈ℕ ⊂ L , which converges to some ym+ ∈ ℝm+ . Therefore, by writing for any

24 | 1 Structure theory integer p, ym+ (p) = ∑1≤i≤k0 ni (p)ym+ ,i , we define a sequence {γ(p)}p∈ℕ ⊂ Γ, which is of the general form I γ(p) = (( m+

A(p)

ym+ (p) )) . 0n−m+

),(

For p large enough, γ(p) ∈ On (ℝ) × B(ym+ , 1), which is absurd because Γ is discrete. Hence, L is a discrete subgroup of ℝm+ . Assume without loss of generality that the first ym+ ,i (1 ≤ i ≤ k) generate L . Take then Γ∞ = ⟨γ1 , . . . , γk ⟩, which is the subgroup of Γga generated by the γi ’s (1 ≤ i ≤ k). We now can and do assume that there exists k ′ > k such that γk′ ∈ ̸ Γ∞ . Otherwise, we can take k0 = k. There exist therefore some integers n1 (k ′ ), . . . , nk (k ′ ) such that k

ym+ ,k′ = ∑ni (k ′ )ym+ ,i , i=1

and then a routine computation gives: k

n (k ′ )

γk′ ′ := γk−1′ ∏γi i i=1

Im+ ( ( ( ( =( ( ( ( ( (

εk′ ′ Im−

dk′ ′ (−1, 1)

′ r(θ1,k ′)

..

.

′ r(θl,k ′)

0m+ 0m− ) (0m ) ) ) ( ±) ) ),( 0 ) ), ) ( 2) ) .. . ) ( 02 ) )

where εk′ ′ ∈ {−1, 1} and dk′ ′ (−1, 1) is a diagonal matrix of diagonal values ±1, and finally γk′ ′ is of finite order. This entails that Γga , being Abelian, is generated by {γ1 , . . . , γk } ∪ {γk′ ′ }k+1≤k′ ≤k0 . The family {γk′ ′ }k+1≤k′ ≤k0 generates a discrete subgroup Γ0 such that all of its elements are of finite order, and finite thanks to Lemma 1.2.9. For the sake of simplicity, let us write {γk′ ′ }k+1≤k′ ≤k0 = {γk+1 , . . . , γk0 }. That is, the subgroup Γg being generated by {γ1 , . . . , γk , γk+1 , . . . , γk0 , δ1 , . . . , δq−1 } where γi = g −1 ai g (1 ≤ i ≤ k0 ) and δj = g −1 bj g (1 ≤ j ≤ q − 1) is such that the γi ’s (1 ≤ i ≤ k0 ) are of the requested form. On the other hand, Γg /Γga = {e, δ1 , . . . , δq−1 }. We examine now the elements δi =: (Si , zi ), for any i ∈ {1, . . . , q − 1}. As we have δi−1 Γga δi = Γga , then for any γ = (Aγ , xγ ) ∈ Γga , we get δi−1 γδi = (Si−1 Aγ Si , Si−1 [xγ + (Aγ − I)zi ]) = (Aγ′ , xγ′ ) ∈ Γga

(1.26)

1.2 Euclidean motion groups | 25

and implies: Im+

Im+ ±Im−

Si−1 (

dγ (−1, 1)

±Im−

) Si = ( Rγ

dγ′ (−1, 1)

). Rγ′

Thanks to the arguments of Proposition 1.2.1, this equation gives that for the decomposition (1.19) of ℝn into direct sums is such that E(1), E(−1), H(−1, 1) and F1 ⊕⊥ ⋅ ⋅ ⋅ ⊕⊥ Fl are stable vector subspaces by all the Si ’s. This gives the following writing: Cm+ ,i (Si , zi ) = ((

Cm− ,i

Cm± ,i

zm+ ,i z ) , ( m− ,i )) , zm± ,i z2l,i Pi

where Cm+ ,i ∈ Om+ (ℝ), Cm− ,i ∈ Om− (ℝ), Cm± ,i ∈ Om± (ℝ) and Pi ∈ O2l (ℝ). Hence, the matrix form of (1.26) reads: Im+ ((

±Im−

−1 Cm d (−1, 1)Cm± ,i ± ,i γ

−1 Cm y + ,i m+ ,γ −1 (−2Cm )zm− ,i − ,i )) , (1.27) ) , ( −1 (Cm± ,i (dγ − Im± ))zm± ,i Pi−1 Rγ Pi (Pi−1 (Rγ − I2l ))z2l,i

which is an element of g −1 Γa g. For γ ∈ Γa , the following hold: zm− ,i = 0m− ,

(dγ − Im± )zm± ,i = 0m±

and (Rγ − I2l )z2l,i = 02l .

Let τ1,i . = ( .. ) τm± ,i

zm± ,i

and z2l,i

u1,i = ( ... ) , ul,i

where τs,i ∈ ℝ for s ∈ {1, . . . , m± } and us,i ∈ ℝ2 for s ∈ {1, . . . , l}. On the one hand, we get (εγ,s − 1)τs,i = 0, for s ∈ {1, . . . , m± } for a given γ such as εγ,s = −1. This implies that τs,i = 0 and so zm± ,i = 0m± . On the other hand, (r(θγ,s ) − I2 )us,i = 0 for any s ∈ {1, . . . , l} for a given γ such that r(θγ,s ) ≠ I2 . This implies that us,i = 0 and so z2l,i = 02l . Hence, Cm+ ,i δi = ((

Cm− ,i

Cm± ,i

zm+ ,i 0 ) , ( m− )) . 0m± Pi 02l

26 | 1 Structure theory Furthermore, equation (1.27) entails that the subspace of ℝm+ generated by {ym+ ,i }i∈{1,...,k} is stable by the subgroup Σm+ of Om+ (ℝ) generated by {Cm+ ,j }j∈{1,...,q−1} . Applying Proposition 1.2.23, there exists Sm+ ∈ Om+ (ℝ) such that for any j ∈ {1, . . . , q − 1}, Ck,j

−1 (Sm , 0m+ )(Cm+ ,j , zm+ ,j )(Sm+ , 0m+ ) = (( +

Cm+ −k,j

zk,j )) , zm+ −k,j

),(

and for any i ∈ {1, . . . , k}, Ik

−1 (Sm , 0m+ )(Im+ , ym+ ,i )(Sm+ , 0m+ ) = (( +

Im+ −k

),(

yk,i

0m+ −k

)) .

This gives in turn that the subgroup Λm+ −k of I(m+ − k) generated by the (Cm+ −k,j , zm+ −k,j )’s is finite. Then Fact 1.2.10 asserts that there exists tm+ −k ∈ ℝm+ −k such that (Im+ −k , −tm+ −k )(Cm+ −k,j , zm+ −k,j )(Im+ −k , tm+ −k ) = (Cm+ −k,j , 0m+ −k ), which completes the proof.

1.2.4 Closed subgroups of I(n) Connected semisimple Lie subgroups of I(n) Let a be a Lie algebra. A symmetric bilinear form b : a × a 󳨀→ ℝ is said to be invariant if b([x, y], z) = b(x, [y, z]), for any x, y, z ∈ a. Further, a is said to be compact if there exists a positive definite, invariant and symmetric bilinear form on a. Recall now the following fact, which will be of use later. Fact 1.2.25 ([82, Theorem 12.1.17]). Any connected semisimple Lie group with compact Lie algebra is compact and of finite center. Let N be a connected, simply connected nilpotent Lie group, C a compact group of automorphisms of N, and H ⊂ NC := C ⋉ N a connected Lie subgroup of NC . Let g := c ⊕ n be the Lie algebra of NC . Thanks to Levi decomposition (see [132]), any maximal semisimple subgroup S of H (called Levi factor), we have H = S ⋅ R, where R is the solvable radical of H. Besides, S is unique up to a conjugation and S ∩ R is discrete. The following characterizes connected semisimple Lie subgroups of G. Proposition 1.2.26. Any connected semisimple Lie subgroup S of NC is compact. Proof. Let S be a connected semisimple Lie subgroup of NC . We denote by s its corresponding Lie algebra. Set φ : s 󳨀→ c, (X, u) 󳨃󳨀→ X. It is clear that φ is a homomorphism of Lie algebras. Further, ker(φ) = {(0, u) | (0, u) ∈ s} is a nilpotent ideal of s. This means that φ is injective, s and φ(s) are isomorphic Lie algebras, and in turn, s is a

1.2 Euclidean motion groups | 27

semisimple compact Lie algebra as c is a compact Lie algebra. By applying Fact 1.2.25, the proof is complete. As a direct consequence, we get the following. Corollary 1.2.27. Any closed connected subgroup of NC contains a normal, closed, connected solvable subgroup cocompactly. Connected closed solvable subgroups of I(n) For any matrix A of On (ℝ), let PA denote the orthogonal projection onto ker(A − I). By Lemma 1.2.2, PA is a polynomial of A and that PA (I − A) = 0. For an arbitrary subgroup H of I(n), set VH := {t | (I, t) ∈ H} ⊂ ℝn , EH := ℝ-span{PA (x) | (A, x) ∈ H} ⊂ ℝn and VH′ , ̃H := ℝ-span(VH ) in EH . For any g ∈ I(n), the orthogonal complement subspace of V g −1 put H := g Hg. We have the following. Lemma 1.2.28. EH is stable by pr1 (H). Proof. Consider {(Ai , xi )}1≤i≤k a family of H such that {PAi (xi )}1≤i≤k spans the subspace EH . For any (B, t) ∈ H, we have (B, t)−1 (Ai , xi )(B, t) = (B−1 Ai B, B−1 [xi + (Ai − I)t]) and, therefore, PB−1 Ai B (B−1 [xi + (Ai − I)t]) = B−1 (PAi [xi + (Ai − I)t]) = B−1 PAi (xi ), for any i ∈ {1, . . . , k}, as PAi is a polynomial depending upon Ai . This shows that B−1 PAi (xi ) ∈ EH and in turn ℝ-span{B−1 PAi (xi )}1≤i≤k ⊆ EH , which gives that B−1 EH = EH . Throughout this subsection, R denotes a closed connected solvable subgroup of I(n). We record the following. Fact 1.2.29 ([82, Theorem 14.4.1]). Let R be a connected solvable Lie group and T ⊆ R be a maximal torus. Then T is maximal compact in R, and there exists a closed submanifold M ≅ ℝq ⊆ R (for some q ∈ ℕ) such that the map M × T → R, (m, t) 󳨃→ mt is a diffeomorphism. As a direct consequence, we get the following. Corollary 1.2.30. pr1 (R) is a connected Abelian subgroup of On (ℝ). The following will be used later. Lemma 1.2.31. VR is a closed subgroup of ER stable by pr1 (R).

28 | 1 Structure theory Proof. Let {tp }p∈ℕ be a converging sequence of VR , and denote t ∈ ℝn its limit. Obviously, {(I, tp )}p∈ℕ is a converging sequence of R. Since R is closed, then (I, t) ∈ R and, therefore, t ∈ VR . Let now (A, x) ∈ R and (I, t) ∈ R, then (A, x)(I, t)(A, x)−1 = (I, At). This shows that At ∈ VR . Lemma 1.2.32. Let H be a subgroup of I(n), g ∈ I(n) and S = pr1 (g). Then EH g = S−1 ⋅EH . Proof. For g = (S, v) ∈ I(n) and (A, x) ∈ H, we have g −1 (A, x)g = (S−1 AS, S−1 [x + (A − I)v]). From ker(S−1 AS − I) = S−1 ker(A − I), we get PS−1 AS (S−1 [x + (A − I)v]) = S−1 ⋅ PA (x). ̃R , there exist r ∈ R and y ∈ V ̃R ⊥ such that r = (A, x + y). Lemma 1.2.33. For any x ∈ V Proof. As VR is a closed subgroup of ℝn , it writes VR = FR ⊕ DR , where FR is a linear subspace of ℝn and DR is a discrete subgroup of ℝn . We first prove that FR is stable by pr1 (R). For any A ∈ pr1 (R) and x ∈ A(FR ), there exist xF ∈ FR and xD ∈ DR such that x = xF + xD . For α ∈ ℝ, αx = αxF + αxD ∈ A(FR ). This shows that αxD ∈ D, and hence, xD = 0. We deduce that A(F) ⊂ F. As F and A(F) have the same dimension, we get A(FR ) = FR . Let {x1 , . . . , xq } be a family of generators of DR , where q designates the rank of DR . ̃R = FR ⊕ F ′ , where F ′ stands for the orthogonal complement of FR in V ̃R . Consider V R R ′ ′ For any i ∈ {1, . . . , q}, xi = vi + ui where vi ∈ FR and ui ∈ FR . Let DR be the subgroup generated by {u1 , . . . , uq }. Note that the fact that VR = FR ⊕ DR = FR ⊕⊥ D′R entails q = dim(ℝ-span D) = dim(ℝ-span D′ ) = dim(F ′ ), ̃R ) = rk(D)+dim(F). Therefore, the rank of DR and D′ coincide with q and then as dim(V R D′R is a lattice of FR′ . As FR′ and VR are stable by pr1 (R), it is also the case for D′R . Write now ℝn = FR′ ⊕ FR ⊕ VR⊥ , as a direct sum, we easily check that there exists g := (S, 0) ∈ G such that any r ∈ Rg writes A′ (r) r = ((

AF (r)

x ′ (r) ) , (xF (r))) . B(r) y(r)

(1.28)

This already gives that the subgroup Λ generated by {A′ (r), r ∈ Rg } is a connected Abelian subgroup of Oq (ℝ) and leaves the lattice D′R invariant. This gives in addition that any A′ (r), r ∈ Rg is of finite order. Indeed, the set {A′ (r)p .ui }p∈ℕ is discrete and lie inside the sphere S(0, ‖ui ‖) and then finite. There exists therefore some pi (r) such that

1.2 Euclidean motion groups | 29

A′ (r)pi (r) .ui = ui , and A′ (r)(p(r)) = Iq for some integer p(r). We conclusively deduce that Λ is trivial, and then equation (1.28) reads Iq r = ((

AF (r)

x′ (r) ) , (xF (r))) . B(r) y(r)

(1.29)

This implies that FR′′ := {x′ (r), r ∈ Rg } is a connected subgroup of ℝq , and hence, a subspace of ℝq . As D′R ⊂ F ′′ , then FR′′ = ℝq = FR′ . Write now g −1 rg = (A, x + y) with x′ (r) x = S ( xF (r) ) 0n−q−η1

0q and y = S ( 0η1 ) , y(r)

where η1 designates the dimension of FR , and the proof is complete. The following result shows how an element in a closed connected subgroup of I(n) writes up to a conjugation with respect to the orthogonal sum FR′ ⊕ FR ⊕ VR′ ⊕ ER⊥ . Proposition 1.2.34. Set q = dim(FR′ ), η1 := dim(FR ), η2 := dim(VR′ ) and η := η1 + η2 + q. There exists g ∈ I(n), such that for any r ∈ Rg , Iq r = ((

A1 (r)

Iη2

x ′ (r) x (r) ) , ( 1 )) , u(r) 0n−η A3 (r)

where A1 (r) ∈ Mη1 (ℝ), A3 (r) ∈ Mn−η (ℝ), x′ (r) ∈ FR′ , x1 (r) ∈ FR and u(r) ∈ VR′ g . Proof. Using Lemma 1.2.1 and Lemma 1.2.33, consider the decomposition ℝn = FR′ ⊕ FR ⊕ VR′ ⊕ ER⊥ and let S be a transition matrix from the standard basis of ℝn to an orthonormal basis adapted to the above decomposition. For h := (S, 0), any r ∈ Rh reads Iq r = ((

A1 (r)

A2 (r)

x ′ (r) x (r) ) , ( 1 )) . x2 (r) A3 (r) x3 (r)

Here, A1 (r) ∈ Mη1 (ℝ), A2 (r) ∈ Mη2 (ℝ), A3 (r) ∈ Mn−η (ℝ) and x ′ (r) ∈ FR′ , x1 (r) ∈ FR , x2 (r) ∈ VR′ h , x3 (r) ∈ ER⊥h .

30 | 1 Structure theory Clearly, 0q x1 (r) ( ) ∈ FR ⊂ VR , 0η2 0n−η and then by Lemma 1.2.33, 0q −A (r)−1 x1 (r) ) ∈ VRh . ( 1 0η2 0n−η We then get 0q −A1 (r)−1 x1 (r) )) a(r) := r ⋅ (I, ( 0η2 0n−η Iq = ((

A1 (r)

A2 (r)

x ′ (r) 0 ) , ( η1 )) ∈ Rh . x2 (r) A3 (r) x3 (r)

By Corollary 1.2.30, for any r, r ′ in Rh , the commutator reads 0q 0 η1 [a(r), a(r ′ )] = (I, ( )) . (Iη2 − A2 (r ′ ))x2 (r) − (Iη2 − A2 (r))x2 (r ′ ) (In−η − A3 (r ′ ))x3 (r) − (In−η − A3 (r))x3 (r ′ ) This means that 0q 0η1 ( ) ∈ VRh (Iη2 − A2 (r ′ ))x2 (r) − (Iη2 − A2 (r))x2 (r ′ ) (In−η − A3 (r ′ ))x3 (r) − (In−η − A3 (r))x3 (r ′ ) and, therefore, (Iη2 − A2 (r ′ ))x2 (r) − (Iη2 − A2 (r))x2 (r ′ ) = 0η2 ,

(In−η − A3 (r ′ ))x3 (r) − (In−η − A3 (r))x3 (r ′ ) = 0n−η .

1.2 Euclidean motion groups | 31

It follows that (A2 (r), x2 (r)) and (A2 (r ′ ), x2 (r ′ )) commute and so do (A3 (r), x3 (r)) and (A3 (r ′ ), x3 (r ′ )). Applying Lemma 1.2.1, there exist ρ = (U2 , v2 ) ∈ I(η2 ) and k ∈ ℕ such that for any r ∈ Rh , ρ−1 (A2 (r), x2 (r))ρ is of the general form: I (( k

x′ (r) ) , ( k )) . ′ 0η2 −k A2 (r)

As x′ (r) } { } { { } { x1 (r) } )} ( { } { x (r) } { 2 } { { 0n−η }r∈Rh

and

x ′ (r) } { } { { } { x1 (r) } )} ( { } { 0 } { η2 } { { 0n−η }r∈Rh

̃ h , respectively, the following set spans V ′ h , span ERh and V R R 0q } { } { } { } { 0η1 . ) ( { } { x2 (r) } } { } { { 0n−η }r∈Rh

(1.30)

Let U=(

Iη1

U2

In−η

).

Then by (1.30), we get x′ (r) U −1 VR′ h = ℝ-span {( k )} , 0η2 −k r∈Rh and hence, we easily get k = η2 and A2 (r) = Iη2 . There exists therefore t ∈ ℝn such that Iq (I, t) a(r) (I, −t) = (I, t) (( Iq = ((

A1 (r)

A1 (r)

Iη2

Iη2

0q 0 ) , ( η1 )) (I, −t) x2 (r) A3 (r) x3 (r)

0q 0η1 ),( )) . x2 (r) PA3 (r) (x3 (r)) A3 (r)

32 | 1 Structure theory Let now Iq A := (

A1 (r)

Iη2

)

and

A3 (r)

0q 0η1 z := ( ). x2 (r) PA3 (r) (x3 (r))

Then clearly z ∈ ker(A − I), and then PA (z) = z, which implies that z ∈ ERh as in Lemma 1.2.32. Hence, PA3 (r) (x3 (r)) = 0. Corollary 1.2.35. Let R be a solvable connected subgroup of I(n). Then R is compact if and only if ER = {0}. Proof. If R is compact, then R is conjugate to a subgroup of On (ℝ). It follows from Lemma 1.2.32 that ER = {0}. Conversely, ER = {0} entails that VR = {0} and VR′ = {0}. Proposition 1.2.34 completes the proof. We now give a description of the structure of a given, closed, connected, solvable subgroup of I(n). Proposition 1.2.36. Any closed, connected, solvable subgroup R of I(n) writes R = AR ⋅ FR , where AR is a closed, connected, Abelian subgroup of R and FR is a subspace of ℝn . Proof. Let Iq r = ((

A1 (r)

Iη2

x ′ (r) x (r) ) , ( 1 )) ∈ Rg . u(r) 0n−η A3 (r)

Then 0q Iq x1 (r) r = (I, ( )) ⋅ (( 0η2 0n−η

A1 (r)

Iη2

x ′ (r) 0 ) , ( η1 )) , u(r) 0n−η A3 (r)

as 0q x1 (r) (I, ( )) ∈ Rg . 0η2 0n−η

1.2 Euclidean motion groups | 33

Let now

ARg

Iq { { { { := {(( { { { {

A1 (r)

Iη2

x ′ (r) 0 ) , ( η1 )) u(r) 0n−η A3 (r)

󵄨󵄨 󵄨󵄨 } } 󵄨󵄨 } } 󵄨󵄨 󵄨󵄨 r ∈ Rg , 󵄨󵄨 } } 󵄨󵄨 } } 󵄨󵄨 󵄨󵄨 }

which is Abelian by Corollary 1.2.30. As Rg is connected, then so is ARg . Corollary 1.2.37. There exists τ := (I, v) ∈ I(n) such that pr2 (Rτ ) = ER . Proof. Keep the same notation as in Proposition 1.2.36. Remark that one can write pr2 (Rg ) = FRg + W, where x′ (r) { { { { 0 W := {( η1 ) { u(r) { { { 0n−η

󵄨󵄨 󵄨󵄨 } } 󵄨󵄨 } } 󵄨󵄨 g 󵄨󵄨 r ∈ R = pr2 (ARg ), 󵄨󵄨 } } 󵄨󵄨 } } 󵄨󵄨 󵄨󵄨 }

for some g := (S, v) ∈ I(n). As ARg is a connected Abelian group, then so is W. Let {tp }p∈ℕ be a sequence of W, which converges to some t ∈ ℝn . Then for some sequence {(Ap )}p∈ℕ , we can and do assume that {(Ap , tp )}p∈ℕ converges to some (A, t) ∈ ARg , and then t ∈ F. This justifies that W is a closed, connected subgroup of ℝn , and hence, a subspace of ℝn . By (1.30), W = VR′ g ⊕ FR′ g , and then ERg = pr2 (Rg ). By Lemma 1.2.32, we have ERg = S−1 ER , and a direct computation gives pr2 (Rg ) = S−1 {x + (A − I)v | (A, x) ∈ R} = S−1 pr2 (Rτ ), where τ = (I, v). Finally, ER = pr2 (Rτ ). Let H be a closed connected subgroup of I(n). We now prove the following result, which will be of use later. Proposition 1.2.38. Let H be a closed connected subgroup of I(n) and R its solvable radical, then EH = ER . Proof. Clearly, ER ⊂ EH . Conversely, let H = R ⋅ S be a Levi decomposition of H. Since S is compact, then we can assume that S is a subgroup of On (ℝ). For any M ∈ S and (A, x) ∈ R, we have M(A, x)M −1 = (MAM −1 , Mx) ∈ R. This gives PMAM −1 (Mx) = MPA (x), which proves that ER is stable by pr1 (H). By considering a basis adapted to the decomposition ℝn = ER ⊕ ER⊥ , there exists Q ∈ On (ℝ) such

34 | 1 Structure theory that any element of Q−1 SQ writes M1 0

(

0 ) M2

and any element of R writes A (( 0

0 x ) , ( 1 )) , B 0

as in Proposition 1.2.34. Let h ∈ H, there exists M1 0

M := (

0 )∈S M2

A r := (( 1 0

and

x 0 ) , ( 1 )) ∈ R 0 A2

such that AM h = r ⋅ M = (( 1 1 0

0 x ) , ( 1 )) . A2 M2 0

To conclude, it is enough to observe that PA1 M1 (x1 ) ∈ ER , as PA1 M1 depends polynomially upon A1 M1 . We also get the following. Corollary 1.2.39. Let H be a closed, connected subgroup of I(n). Then H is compact if and only if EH = {0}. We close the chapter by the following, which will be of use. Proposition 1.2.40. Let Γ be a discrete subgroup of I(n) and Γa a normal Abelian subgroup of finite index in Γ. Then EΓ = EΓa . In particular, Γ is finite if and only if EΓ = {0}. Proof. Take back the same notation as in Theorem 1.2.24. For a given g := (S, v), any γ ∈ Γg reads C γ = (( k,j(γ)

wk (γ) )) , ̃ ),( 0 C n−k n−k

for some j(γ) ∈ {1, . . . , q − 1}. As δjq ∈ Γga , we get PCk,j(γ) (wk (γ)) ∈ ℝ-span{yk,1 , . . . , yk,k }. On the other hand, we easily check that ℝ-span {(

yk,i ) 1 ≤ i ≤ k} = EΓga . 0n−k

Hence, EΓga = EΓg and by Lemma 1.2.32, we get EΓa = EΓ . The following is an immediate consequence from Proposition 1.2.40.

1.3 Heisenberg motion groups | 35

Corollary 1.2.41. Let Γ be a discrete subgroup of I(n). There exists τ = (I, v) ∈ I(n) such that pr2 (Γτ ) ⊂ EΓ .

1.3 Heisenberg motion groups Consider the Heisenberg group ℍn = ℂn × ℝ as defined in Subsection 1.1.2. The group 𝕌n of n × n complex unitary matrices acts on ℍn by the automorphisms A(z, t) = (Az, t)

for each A ∈ 𝕌n , z ∈ ℂn and t ∈ ℝ.

The Heisenberg motion group is the Lie group defined by the semidirect product Gn = 𝕌n ⋉ ℍn with the multiplication law: (A, z, t)(B, w, s) = (AB, z + Aw, t + s −

1 Im⟨z, Aw⟩). 2

We deal with the topology structure on the product 𝕌n × ℂn × ℝ arising from the standard Hermitian inner product ⟨⋅, ⋅⟩ on ℂn with the related norm on 𝕌n . Furthermore, 𝕌n yields a maximal compact connected subgroup of Aut(ℍn ) and Gn . For the sake of simplicity of notation, Gn will be merely denoted by G, unless explicit mention. Denote i = √−1, and for any g ∈ G and a subgroup H of G, H g := gHg −1 . For any u ∈ ℂn , denote τu := (I, u, 0) where I designates the identity matrix of Mn (ℂ) and set e = τ0 . Define the following: – G1 = 𝕌n × {0} × ℝ ⊂ G. – pr1 , pr2 and pr3 , the natural projections of G on 𝕌n , ℂn and ℝ, respectively. – pr : G 󳨀→ 𝕌n ⋉ ℂn , (A, z, t) 󳨃→ (A, z) the continuous homomorphism. – For (A, z, t) ∈ G, EA (1) = ker(A − I) and PA the orthogonal projection on EA (1). (It is known that PA depends polynomially upon A.)

1.3.1 First preliminary results Lemma 1.3.1. For any g ∈ G, there exists u ∈ ℂn such that τu gτ−u = (A, z, t), where PA (z) = z (i. e., Az = z) and tz = 0. Proof. Let g = (A, y, ρ) ∈ G, write y = z + (I − A)z ′ where z ∈ EA (1) and z ′ ∈ EA (1)⊥ . Then g ′ = τ−z ′ gτz ′ = (A, z, t ′ ) for some t ′ ∈ ℝ. If z = 0 or t ′ = 0, we are done. Otherwise, we t′ ′ easily check that ταz (A, z, t ′ )τ−αz = (A, z, 0) for α = −i ‖z‖ 2 . Hence, for u = αz − z , τu gτ−u is of the requested form. Lemma 1.3.2. Let (A, z, t), (B, w, s) ∈ G. If (A, z, t) and (B, w, s) are conjugate to each other in G, then PA (z) = 0 if and only if PB (w) = 0.

36 | 1 Structure theory Proof. Let g = (S, w, l) ∈ G. By a direct computation, we have (B, w, s) = g −1 (A, z, t)g = (S−1 AS, S−1 [z + (A − I)w],

1 1 Im⟨w, z + Aw⟩ − Im⟨z, Aw⟩ + t). 2 2

Therefore, PS−1 AS (S−1 [z + (A − I)w]) = S−1 PA (z + (A − I)w)

= S−1 PA (z) + S−1 P ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ A (A − I) w = S PA (z),

=0

−1

as is to be shown. Now we prove the following propositions. Proposition 1.3.3. Let γ = (A, z, t) ∈ G be of finite order. Then PA (z) = 0 and γ is conjugate to an element of Un × {0} × {0} in G. Proof. Let γ = (A, z, t) ∈ G be of finite order k0 . By Lemma 1.3.1, there exists u ∈ ℂn such that τu γτ−u = (A, w, s), where PA (w) = w and sw = 0. Then τu γ k0 τ−u = (Ak0 , k0 w, k0 s) = (I, 0, 0). This implies that PA (w) = w = 0 and s = 0. Hence, by Lemma 1.3.2, we have PA (z) = 0 and for g = τu , gγg −1 = (A, 0, 0) ∈ Un × {0} × {0}. Proposition 1.3.4. Let γ = (A, z, t) ∈ G. If PA (z) = 0, then γ is conjugate to an element of G1 . Proof. According to Lemma 1.3.1, there exists u ∈ ℂn such that τu γτ−u = (A, w, s), where PA (w) = w and sw = 0. If PA (z) = 0, then Lemma 1.3.2 gives that PA (w) = w = 0. Hence, gγg −1 = (A, 0, s) ∈ G1 for g = τu . Proposition 1.3.5. Let γ = (A, z, t) ∈ G. If PA (z) ≠ 0, then γ is conjugate to an element (B, w, s) of Un × ℂn × {0} with Bw = w. Proof. By Lemma 1.3.1, there exists u ∈ ℂn such that τu γτ−u = (A, w, s), where PA (w) = w and sw = 0. If PA (z) ≠ 0, then by Lemma 1.3.2, PA (w) = w ≠ 0. Hence, s = 0 and for g = τu we have gγg −1 = (A, w, 0) ∈ Un × ℂn × {0}. 1.3.2 Discrete subgroups of Heisenberg motion groups Subgroups of the semidirect product group NC = C ⋉ N We first record the following result, which will be next utilized.

1.3 Heisenberg motion groups | 37

Fact 1.3.6 ([4, Theorem 4.1 and Remark 4.2] and also [9]). Let N be a connected, simply connected, nilpotent Lie group, C a compact group of automorphisms of N and Γ ⊂ NC = C ⋉ N a discrete subgroup. Then there exist b ∈ N, a connected Lie subgroup NΓ of N and a Γ∗ of Γ, which is isomorphic to a discrete subgroup of NΓ satisfying the following conditions: (1) bΓb−1 preserves the subset NΓ of N via the natural action of C ⋉ N on N. (2) The orbit space (bΓb−1 )\NΓ is compact. (3) The induced (bΓ∗ b−1 )-action on NΓ is free. It has been shown in [84] that any closed, solvable subgroup of an almost connected Lie group is compactly generated (hence, finitely generated if in addition discrete). Combined with Fact 1.3.6, one gets immediately the following. Lemma 1.3.7. Any discrete subgroup of NC is finitely generated. Proof. Let Γ be a discrete subgroup of G, by Fact 1.3.6, Γ contains a subgroup Γ∗ of finite index q. Take {e, δ1 , . . . , δq−1 } for a complete representative of the finite set Γ/Γ∗ , where e designates the identity of G. Furthermore, Γ∗ is isomorphic to a discrete subgroup of N, which is finitely generated, and then so is Γ∗ . Let {γ1 , . . . , γk } be a family of generators of Γ∗ for some positive integer k, then {γ1 , . . . , γk , δ1 , . . . , δq−1 } generates Γ. We go back now to a Heisenberg motion group G. We take a finite index subgroup Γ of Γ as in Fact 1.3.6. Then Γ∗ is isomorphic to a discrete subgroup of ℍn and then D(Γ∗ ) is isomorphic to a subgroup of Z(ℍn ). Hence, D(Γ∗ ) is a torsion-free subgroup of rank 1 or is trivial. Here, for a group G′ , Z(G′ ) designates the center of G′ and D(G′ ) = [G′ , G′ ]. The following is also straight to obtain the following. ∗

Proposition 1.3.8. Let G = 𝕌n ⋉ℍn be a Heisenberg motion group and let Γ be a discrete subgroup of G. Then the following are equivalent: (1) Γ is finite. (2) Any element of Γ is of finite order. (3) Γ∗ is trivial. Proof. As Γ∗ is isomorphic to a discrete subgroup of ℍn , then it is torsion-free. Hence, Γ∗ is trivial whenever any element of Γ is of finite order. The other implications are trivial. For any positive integer n, recall the semidirect product I(n) := On (ℝ) ⋉ ℝn of the orthogonal group On (ℝ) (with respect to the canonical Euclidean product ⟨⋅, ⋅⟩ on ℝn ) and ℝn .

38 | 1 Structure theory

that

For U = A + iB and z = x + iy, where A, B ∈ Mn (ℝ) and x, y ∈ ℝn , we easily check ϕn : 𝕌n ⋉ ℂn (A + iB, x + iy)

󳨅→

I(2n)

󳨃→

((

A B

−B x ) , ( )) A y

(1.31)

is a group homomorphism. Note here that for any closed subgroup L of 𝕌n ⋉ ℂn , ϕn (L) is a closed subgroup of I(2n). By a direct computation, we see that the following two conditions on L are equivalent: – PU (z) = 0 for any (U, z) ∈ L. – PM (x) = 0 for any (M, x) ∈ ϕn (L). Corollary 1.3.9. Let {γj }j∈J be a commuting family of 𝕌n ⋉ ℂn . There exist an integer 0 ≤ k ≤ n and g ∈ 𝕌n ⋉ ℂn such that for any j ∈ J, I g −1 γj g = (( k

An−k,j

),(

zk,j )) , 0n−k

(1.32)

where zk,j ∈ ℂk , 0n−k is the zero of ℂn−k and An−k,j ∈ 𝕌n−k . Proof. For any j ∈ J, let γj = (Aj , zj ) and δj := (Bj , xj ) = ϕn (γj ), where ϕn is in (1.31). By Lemma 1.2.1, there exist h = (C, y) ∈ E(2n) and 0 ≤ s ≤ 2n such that for any j ∈ J, Is

δj′ := (C, y)δj (C, y)−1 = ((

B2n−s,j

),(

ys,j )) , 02n−s

for some B2n−s,j ∈ O2n−s (ℝ) and ys,j ∈ ℝs (j ∈ J). Therefore, for j0 ∈ J, δj′′0 := (C −1 , 0)δj′0 (C, 0) = (Bj0 , xj′0 ), for some ys,j0 ) ∈ ⋂ EBj (1). O2n−s j∈J

xj′0 = C −1 (

On the other hand, δj′′0 = (I2n , C −1 y)δj0 (I2n , −C −1 y) by a simple computation. Set now

a −1 C −1 y = ( 1 ) for a1 , a2 ∈ ℝn . For (I, u) = (I, a1 + ia2 ) = ϕ−1 n ((I2n , C y)), one gets a2 ′′ −1 ′ ϕ−1 n (δj0 ) = (I, u)ϕn (δj0 )(I, −u) = (Aj0 , zj0 ),

for some zj′0 ∈ ℂn . We easily check that zj′0 ∈ ⋂j∈J EAj (1). Let Q be the transition matrix from the standard basis of ℂn into an orthonormal basis adapted to the orthogonal

1.3 Heisenberg motion groups | 39

direct sum [⋂j∈J EAj (1)] ⊕ [⋂j∈J EAj (1)]⊥ . Then for g = (Q, Qu), gγj g −1 is of the requested form (1.32), where k = dim ⋂j∈J EAj (1). We next have the following. Proposition 1.3.10. Let Γ be a discrete subgroup of G such that for any γ = (B, z, t) ∈ Γ, PB (z) = 0. Then there exists u ∈ ℂn such that Γτu ⊂ G1 . Proof. Making use of Fact 1.3.6, Γ contains a 2-step nilpotent torsion-free subgroup Γ∗ and of finite index in Γ. Assume that D(Γ∗ ) is of rank 1 and has a generator γ0 = (A0 , u0 , s0 ), say. By Proposition 1.3.4, there exists w ∈ ℂn such that τw γ0 τ−w = (A0 , 0, t0 ) ∈ G1 . For any (B, z) ∈ pr[(Γ∗ )τw ], we have τ

(B, z)−1 (A0 , 0)(B, z) = (B−1 A0 B, B−1 (A0 − I)z) ∈ pr[D(Γ∗ ) w ] and then z ∈ EA0 (1). Let Q be the transition matrix from the standard basis of ℂn to an orthonormal basis of EA0 (1) ⊕ EA0 (1)⊥ and consider {(Bj , zj )}j∈J to be a complete representative set of pr((Γ∗ )τw )/ pr(D[(Γ∗ )τw ]) where (Bj , zj ) designates the equivalent class of (Bj , zj ) modulo pr(D[(Γ∗ )τw ]). For ω1 = (Q, 0, 0) ∈ 𝕌n , any (C, v) ∈ pr((Γ∗ )τw ω1 ) reads 0 I v B ) , ( k )), ) , ( j )) (( k (C, v) = (( 1,j 0 0 C B n−k n−k 2,j ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟

(1.33)

∈pr(D((Γ∗ )τw ω1 ))

= ω1 (Bj ,zj )ω−1 1

where k = dim EA0 (1). It turns out that {(B1,j , vj )}j∈J generates an Abelian subgroup Λk of 𝕌k ⋉ ℂk , say. By applying Corollary 1.3.9, there exist an integer k ′ ≤ k and ω′k = (Sk , u′k ) ∈ 𝕌k ⋉ ℂk such that for any j, ω′k (B1,j , vj )ω′k

−1

I′ = (( k

ṽj ̃ j ) , (0k−k′ )) , B

̃ ’s to 𝕌 ′ . Since for all γ = (B, z, t) ∈ Γ, and where all the ṽj ’s belong to ℂk and the C j k−k PB (z) = 0, ṽj = 0k′ for any j ∈ J, then one can easily justify that for ′

Sk−1 u′k ), 0n−k

w′ = Q−1 (

and v = w′ + w, (Γ∗ )τv ⊂ G1 . Take a complete representative set {δ1 , . . . , δq } of Γτv /(Γ∗ )τv . For any j ∈ {1, . . . , q}, denote δj = (Nj , xj , tj ). Then q

τ

τ

Γτv = ⋃ δj ⋅ (Γ∗ ) v = {(Nj A, xj , tj + t) : (A, 0, t) ∈ (Γ∗ ) v , j ∈ {1, . . . , q}}. j=1

(1.34)

40 | 1 Structure theory On the other hand, remark that pr2 (Γτv ) = {x1 , . . . , xq } and, therefore, by Corollary 1.2.11 there exist some u′ ∈ ℂn such that (I, u′ )(pr(Γτv ))(I, −u′ ) ⊂ 𝕌n × {0}. Conclusively, for u = u′ + v, Γτu ⊂ G1 . To complete the proof, just remark that when D(Γ∗ ) is trivial, and it is sufficient to take Cn−k = In−k , the identity matrix, in equation (1.33). We now look at the analogue of Proposition 1.3.10 in the case where H is connected. We first record the following (cf. [82]). Fact 1.3.11 ([82, Theorem 12.2.2]). For a compact connected Lie group C with Lie algebra c, the following assertions hold: (i) A subalgebra t ⊆ c is maximal Abelian if and only if it is the Lie algebra of a maximal torus of C. (ii) For two maximal tori T and T ′ , there exists c ∈ C with cTc−1 = T ′ . (iii) Every element of C is contained in a maximal torus. We get conclusively the following. Corollary 1.3.12. For a closed, connected, solvable subgroup R of G, pr1 (R) is a connected Abelian subgroup of 𝕌n . Proof. Consider the compact connected Lie group pr1 (R), the closure of the solvable subgroup pr1 (R) of 𝕌n . By Fact 1.3.11, pr1 (R) contains a maximal torus T. Since pr1 (R) is solvable (cf. [48, Corollary 2, p. 342]), then by Fact 1.2.29, for some submanifold M ≅ ℝn ⊆ pr1 (R) the map M × T → pr1 (R), (m, t) 󳨃→ mt is a diffeomorphism. By the compacity of pr1 (R), M is trivial, and hence, pr1 (R) = T. This completes the proof. Lemma 1.3.13. Let H be a closed, connected subgroup of G such that for any h = (B, z, t) ∈ H, PB (z) = 0. Then there exists u ∈ ℂn such that H τu ⊂ G1 . Proof. Let H = S ⋅ R be a Levi decomposition, where S is a maximal semisimple subgroup of H and R is the solvable radical of H. By Proposition 1.2.26, S is compact. We easily see that pr(R) is Abelian. Indeed, for any (B, x, t), (B′ , x ′ , t ′ ) ∈ R, B and B′ commute as in Corollary 1.3.12, and then the commutator [(B, x, t)(B′ , x ′ , t ′ )] is of the form [(B, x, t)(B′ , x′ , t ′ )] = (I, a, s), for some a ∈ ℂn and s ∈ ℝ. Since (I, a, s) ∈ H, by the assumption of H, we have a = PI (a) = 0 as it is to be shown. Corollary 1.3.9 asserts now that there exist an integer 0 ≤ k ≤ n and g = (Q, w) ∈ 𝕌n ⋉ ℂn such that for any γ = (A, z) ∈ pr(R), I g −1 γg = (( k

An−k,γ

zk,γ )) , 0n−k

),(

1.3 Heisenberg motion groups | 41

where zk,γ ∈ ℂk , 0n−k is the zero of ℂn−k and An−k,γ ∈ 𝕌n−k . Hence, for any r = (A, z, t) ∈ R, (Q, w, 0)−1 r(Q, w, 0) = (Q−1 AQ, Q−1 (z + (A − I)w), tr′ ) I = (( k

An−k,pr(r)

zk,pr(r) ) , tr′ ) , 0n−k

),(

for some tr′ ∈ ℝ. Finally, zk,pr(r) = 0k as PQ−1 AQ (Q−1 (z + (A − I)w)) = 0. Therefore, for any r ∈ R, (Q, w, 0)−1 r(Q, w, 0) = (Q−1 AQ, 0, tr′ ), which entails that Rτ−w ⊂ G1 . Conclusively, pr2 (H τ−w ) = pr2 (Sτ−w ), which is compact and then by Corollary 1.2.11 there exists v ∈ ℂn such that (I, v) pr(H τ−w )(I, −v) ⊂ 𝕌n × {0}. Thus, H τu ⊂ G1 for u = −w + v. Proposition 1.3.14. Let H be a closed subgroup of G such that for any h = (B, z, t) ∈ H, PB (z) = 0. Then there exists u ∈ ℂn such that H τu ⊂ G1 . Proof. Consider H0 to be the connected component of H, thanks to Lemma 1.3.13, we can assume that H0 ⊂ G1 . Take {(Bj , zj , tj )}j∈J a complete representative of the group

H/H0 . H/H0 = {δj = (Bj , zj , tj )}j∈J is a discrete group. Without loss of generality, we can let j ∈ J and (S, 0, t) ∈ H0 , and we have −1 −1 −1 (B−1 j , −Bj zj , −tj )(S, 0, t)(Bj , zj , tj ) = (Bj ABj , Bj (S − I)zj , t +

1 Im⟨zj , Szj ⟩) ∈ H0 . 2

This entails that for any j ∈ J , zj ∈ V :=

⋂

S∈pr1 (H0 )

ES (1).

With respect to the decomposition ℂn = V ⊕ V ⊥ , there exists ω = (Q, 0, 0) ∈ G such that for any h ∈ H ω = ωHω−1 , B 0 z I 0 h = (( j,k ) , ( j,k ) , tj ) (( k ) , 0, t), 0 B 0 0 A j,n−k n−k ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ ωδj ω−1

(1.35)

∈ ωH0 ω−1

where k = dim(V), Bj,k ∈ 𝕌k , Bj,n−k and An−k belong to 𝕌n−k and zj,k ∈ ℂk . If pr3 (H0 ) ≠ {0}, then for any j ∈ J , there exists (Sj , 0, −tj ) ∈ ωH0 ω−1 such that (Sj , 0, −tj )(Bj , zj , tj ) = (Sj Bj , Sj zj , 0) = (B′j , zj′ , 0). Equation (1.35) then reads B′j,k 0

h = ((

0

′ zj,k I ) , 0) (( k 0 0

),( B′j,n−k

0 ) , 0, t ′ ) , A′n−k

42 | 1 Structure theory ′ and we easily verify that Λk := {(B′j,k , zj,k , 0)}j∈J is a discrete subgroup of 𝕌k ⋉ ℍk .

Making use of Proposition 1.3.10, there exists v ∈ ℂk such that τv Λk τv−1 ⊂ 𝕌k × {0} × ℝ. This accomplishes the proof in this case. If pr3 (H0 ) = {0}, then F := {(Bj,k , zj,k , tj )}j∈J is a discrete subgroup of 𝕌k ⋉ ℍk . Again by Proposition 1.3.10, F is conjugate to a subgroup of 𝕌k ×{0}×ℝ and so is H. The following observation will be of use later.

Lemma 1.3.15. Let H be a closed subgroup of G such that H ∩(𝕌n ×B(0, r)×ℝ) is compact for any r > 0. Then pr(H) is a closed subgroup of 𝕌n ⋉ ℂn . Proof. Let {(Bp , zp )}p∈ℕ be a sequence of pr(H) converging to (B, z) ∈ 𝕌n ⋉ ℂn . One can define a sequence {(Bp , zp , tp )}p∈ℕ ⊂ H. As for p large enough, (Bp , zp ) ∈ 𝕌n × B(0, r) for some r > 0, then {(Bp , zp , tp )}p∈ℕ ⊂ H ∩ (𝕌n × B(0, r) × ℝ). Thus one can find a subsequence {(Bα(p) , zα(p) , tα(p) )}p∈ℕ , which converges to (B, z, t) ∈ H for some t ∈ ℝ and this implies that (B, z) ∈ pr(H). Taking all the above mentioned into account, one can conclusively categorize discrete subgroups of G as follows. Definition 1.3.16. Let Γ be an infinite discrete subgroup of G. Γ is said to be: (1) of type (A), if Γ is conjugate to a subgroup of G1 = 𝕌n × {0} × ℝ, (2) of type (B), if any element of Γ is conjugate to an element of 𝕌n × ℂn × {0}, (3) of type (C), if Γ contains at least a mixture of two infinite subgroups of type (A) and type (B). We now prove the following. Proposition 1.3.17. Any infinite discrete subgroup of Gn is either of type (A) or (B) or (C). Furthermore, two conjugate infinite discrete subgroups are of the same type. Proof. Let Γ be an infinite discrete subgroup of G. Suppose that Γ is neither of type (A) nor (B). We shall prove that Γ is of type (C). By combining with the assumption for Γ and Propositions 1.3.10, 1.3.3 and 1.3.5, we can find an element γ1 = (A1 , z1 , t1 ), γ2 = (A2 , z2 , t2 ) ∈ Γ such that γ1 , γ2 are both of infinite order, PA1 (z1 ) ≠ 0 and PA2 (z2 ) = 0. In particular, by applying Propositions 1.3.4 and 1.3.5, we have that γ1 (resp., γ2 ) is conjugate to an element of Un × ℂn × {0} (resp., of G1 ). Therefore, Γ contains infinite discrete subgroups of type (B) as ⟨γ1 ⟩ and type (A) as ⟨γ2 ⟩. This means that Γ itself is of type (C).

1.4 Syndetic hulls | 43

1.4 Syndetic hulls Definition 1.4.1. Let G be a Lie group, Γ a closed subgroup of G and Z c (G) the maximal compact subgroup of Z(G), the center of G. A syndetic hull of Γ is any connected Lie subgroup L of G, which contains Γ ⋅ Z c (G) cocompactly. Then obviously, L contains Γ cocompactly. When G is simply connected and solvable, then Z c (G) is trivial. In this case, by a syndetic hull of Γ, we mean any connected Lie subgroup of G, which contains Γ cocompactly.

1.4.1 Existence results for completely solvable Lie groups In [122], Saito proved that any closed subgroup of a completely solvable Lie group admits a unique syndetic hull. Here, we give a simpler proof. Theorem 1.4.2. Let G be a simply connected completely solvable Lie group. Then any closed subgroup of G admits a unique syndetic hull. Proof. Let Γ be a closed subgroup of G, being completely solvable. We first show that the syndetic hull is unique provided its existence. Let L1 and L2 be two connected, closed subgroups such that L1 /Γ and L2 /Γ are compact. Note first that L1 ∩ L2 is connected as being the Lie group associated to the Lie algebra l1 ∩ l2 , where li denotes the Lie algebra of Li , i = 1, 2. We claim that Li /(L1 ∩ L2 ), i = 1, 2 are also compact. To see that, consider for i = 1, 2 the canonical surjection πi : Li 󳨀→ Li /(L1 ∩ L2 ), which factors through the canonical surjection ρi : Li → Li /Γ to a surjection π̃ i : Li /Γ → Li /(L1 ∩ L2 ) such that πi = π̃ i ∘ ρi . The map π̃ i is surjective and continuous, and thus its image Li /(L1 ∩ L2 ) is compact. Now, G is connected simply connected, solvable Lie group, the quotient Li /(L1 ∩ L2 ) is diffeomorphic to ℝd , where d = dim Li − dim(L1 ∩ L2 ). It follows therefore that d = 0 and that L1 ∩ L2 = L1 = L2 , as it is to be shown. We tackle now the proof of existence, we proceed by induction on the dimension of G. Let now G0 be a one-codimensional, closed, normal subgroup of G. Provided that Γ ⊂ G0 , we are done using the induction hypothesis. We assume henceforth that Γ ⊄ G0 and we consider Γ0 = Γ ∩ G0 , which is a closed subgroup of G0 . Let G̃ = G/G0 ≃ ℝ and π : G → G̃ the canonical projection. Provided that Γ0 is trivial, the homomorphism ̃ π̃ = π|Γ : Γ → Γ̃ := π(Γ) appears to be a groups isomorphism. This gives then that Γ is therefore Abelian and the result follows. We assume from now on that Γ0 is not trivial. There exists by the induction hypothesis, a closed, connected subgroup L0 of G0 , which contains Γ0 cocompactly. As such, there exists a compact set C in G contained

44 | 1 Structure theory in G0 and fulfills the following identity: L0 = CΓ0 .

(1.36)

Assume for a while that L0 is normal in G, then L0 Γ is a subgroup of G and by equation (1.36), we get that L0 Γ = CΓ0 Γ = CΓ,

(1.37)

which is closed in G as Γ is. Let G′ = G/L0 and π : G → G′ the associated canonical surjection. Then Γ′ = π(L0 Γ) is a closed subgroup of G′ . Using the induction hypothesis, there exists a connected closed subgroup S′ of G′ such that S′ /Γ′ is compact, in particular, there exists by Lemma 1 in [74] a compact set C ′ of G such that S′ = C ′ Γ′ , where C ′ is the image of C ′ by π. Let S be the preimage of S′ by π, then S is closed subgroup of G, which contains L0 . Moreover, we have S = C ′ L0 ΓL0 = C ′ L0 Γ = C ′ CΓ by (1.36) and (1.37), which merely entails that S/Γ is compact. On the other hand, we have S′ = S/L0 with S′ and L0 are connected. This shows that S is connected. We finally treat the case where L0 is not normal. Let NG (L0 ) be the normalizer of L0 in G. The subgroup G0 being normal in G, we get that for any γ ∈ Γ, γΓ0 γ −1 ⊂ G0 ∩ Γ = Γ0 , and then Γ0 is normal in Γ. Therefore, for any γ ∈ Γ the subgroup γL0 γ −1 is connected and closed in G0 such that γL0 γ −1 /Γ0 is compact. By the uniqueness of L0 , we get that γL0 γ −1 = L0 and that Γ ⊂ NG (L0 ). Recall that NG (L0 ) is a connected, closed subgroup of G and dim NG (L0 ) < dim G. The result follows using again the induction hypothesis.

1.4.2 Case of exponential Lie groups In the case where G is more generally an exponential solvable Lie group, the result of Theorem 1.4.2 fails to hold in general (a counterexample is given in [122]), except when Γ turns out to be Abelian. In this case, we have the following. Proposition 1.4.3. Assume that G is an exponential solvable Lie group and Γ is an Abelian closed subgroup of G. Then Γ admits a unique syndetic hull. Proof. We first prove the following. Lemma 1.4.4. Let G be an exponential Lie group and g its Lie algebra. For all X, Y ∈ g the following properties are equivalent: (𝚤) exp(X) exp(Y) = exp(Y) exp(X) (𝚤𝚤) [X, Y] = 0.

1.4 Syndetic hulls | 45

Proof. From the injectivity of the exponential map, the first property (𝚤) is equivalent to Ad(exp(X))(Y) = Y. This gives that for all t ∈ ℝ, we have Ad(exp(X))(tY) = tY. Exponentiating back, we obtain exp(tY) exp(X) exp(−tY) = exp(X), which merely entails that Ad(exp(tY))(X) = X for any t ∈ ℝ. Therefore, the analytic map tp p (ad(Y)) (X) p! p=1 ∞

a(t) = ∑

vanishes on ℝ and so do its coefficients. We get conclusively that [X, Y] = 0. The converse is trivial. We now go back to the proof of Proposition 1.4.3. Let g be the Lie algebra of G, Λ = log Γ and l = ℝ-span(Λ). We get from (1.4.4) that l is an Abelian subalgebra of g and then L = exp(l) = exp(ℝX1 ) ⋅ ⋅ ⋅ exp(ℝXk ) is isomorphic to ℝk , where X1 , . . . , Xk belong to Λ. It is then clear that exp(ℤX1 ) ⋅ ⋅ ⋅ exp(ℤXk ) ⊂ Γ, which means that L/Γ is compact. To see that L is unique, let L′ be another connected subgroup of G, which contains Γ cocompactly. Then obviously L ⊂ L′ and then L′ /L is compact. By choosing an coexponential basis of L′ in L, we see that L/L′ is trivial, which achieves the proof of the proposition. A straight application of Theorem 1.4.2 and Proposition 1.4.3 is to establish a homeomorphism between the spaces Hom(Γ, G) and Hom(l, g) where l stands for the Lie algebra of the syndetic hull L of Γ. Recall that in [122], it is proved that any homomorphism from Γ to G can be uniquely extended to a homomorphism from the syndetic hull of Γ to G. We now give an alternative simpler proof of this upshot and even develop this aspect later (cf. Proposition 1.4.7 below). We first have the following. Theorem 1.4.5. Let G be a completely solvable Lie group, Γ a closed subgroup of G and L its syndetic hull in G. Then any continuous homomorphism from Γ to G uniquely extends to a continuous homomorphism from L to G. This also applies to the case where G is exponentially solvable and Γ is Abelian.

46 | 1 Structure theory Proof. First, we provide a proof of existence. We start with a continuous homomorphism φ from Γ to G, and we denote by Γ̂ ⊂ L × G its graph. Clearly, Γ̂ is a closed subgroup of L × G. By Theorem 1.4.2, Γ̂ has a unique syndetic hull K. Let p : L × G → L,

q :L×G →G

be the natural homomorphisms projections. We now proceed to prove that q|K ∘p−1 |K is a well-defined continuous homomorphism from L to G and that its restriction to Γ coincides with φ. We first show that p|K realizes an isomorphism between K and L. Indeed, p is a continuous homomorphism and K is a connected exponential Lie subgroup of L × G. Let ψ be the differential of p|K at e and k = log K, the Lie algebra associated to K. Then p|K = expL ∘ψ ∘ logK

and

p(K) = expL (ψ(k)),

(1.38)

where ψ(k) is a Lie subalgebra of l = log L, in particular, p(K) is closed, connected subgroup of L. Let π be the canonical surjection L → L/Γ. The subgroup Γ is included in p(K) being a saturated (with respect to π) closed set in L. This means that π(p(K)) is closed in L/Γ and then p(K)/Γ is compact. From the uniqueness of the syndetic hull of Γ, we get that p(K) = L, and that p|K is a surjective homomorphism from K to L. To prove the injectivity, we need the following lemma. Lemma 1.4.6. The maps p|Γ̂ and p|K are proper. That is, the inverse image of any compact set of L by p|Γ̂ and p|K is compact. Proof. Let C be a compact set in L, then p−1 (C) ∩ Γ̂ = (C × G) ∩ Γ̂

= {(x, φ(x)), x ∈ C ∩ Γ},

which is compact as Γ is closed in G. This shows that p|Γ̂ is proper.

Let now C be a compact set of L and C1 a compact set of K such that K = C1 Γ.̂ Clearly, p−1 (C) ∩ K is closed in K and we have p−1 (C) ∩ K = p−1 (C) ∩ C1 Γ̂ ⊂ C1 (C1−1 p−1 (C) ∩ Γ)̂ ⊂ C1 (p−1 (p(C1−1 )C) ∩ Γ)̂ = C1 p−1 (p(C1−1 )C). |Γ̂ To conclude that p−1 (C)∩K is compact, it is thus sufficient to prove that C1 p−1 (p(C1−1 )C) |Γ̂

is a compact in K. By the continuity of p, we see that p(C1−1 )C is a compact set in L. Now p|Γ̂ is proper. Then p−1 (p(C1−1 )C) is compact in Γ̂ and the result follows. |Γ̂

Now the map p|K is a continuous surjective homomorphism and from its properness, we see that its kernel is a compact subgroup of K. Up to this step, p|K is shown

1.4 Syndetic hulls | 47

to be a continuous bijective homomorphism. Using the first equality in (1.38), we can see that −1 p−1 |K = expK ∘ψ ∘ logL ,

which is also continuous. This conclusively shows that p|K is an isomorphism. Furthermore, q|K is a continuous homomorphism, then q|K ∘ p−1 |K is a continuous homomorphism from L to G and for x ∈ Γ, we have q|K ∘ p−1 |K (x) = q|K (x, φ(x)) = φ(x), which entails that the restriction of q|K ∘ p−1 |K to Γ is φ. This achieves the proof of existence. As for the uniqueness, let L′ be the subgroup of G generated by the family of the one parameter subgroups (γ(t), t ∈ ℝ) for γ ∈ Γ defined by the derivative at t = 0, dγ(t) (0) = log(γ). dt Then L′ is a connected subgroup of G, contained in L and contains Γ, which entails that L = L′ . Let ψ be a continuous homomorphism from L to G and γ ∈ Γ. By the continuity of ψ, we get ψ(γ(t)) = exp(t log(ψ(γ))),

t ∈ ℝ.

(1.39)

Let ψ and ψ′ be a two continuous homomorphisms from L to G. By continuity, ψ = ψ′ if and only if ψ|L′ = ψ′|L′ . But L′ is generated by the subgroups γ(t), γ ∈ Γ. This means therefore that ψ = ψ′ if and only if ψ(γ(t)) = ψ′ (γ(t)) for all γ ∈ Γ, which is equivalent by means of (1.39) to ψ|Γ = ψ′|Γ . The same proof is adopted to the case where G is exponential solvable and Γ is Abelian. This achieves the proof of the theorem. We finally prove the following result, which will be of great interest in the sequel. Proposition 1.4.7. Let G be a completely solvable Lie group, Γ a closed subgroup of G and L its syndetic hull in G. Let Homc (L, G) and Homc (Γ, G) designate the sets of continuous homomorphisms from L to G and Γ to G, respectively, endowed with the pointwise convergence topology. Then the restriction natural map R : Homc (L, G) → Homc (Γ, G) : ψ 󳨃→ ψ|Γ is a homeomorphism. This also applies to the case where G is exponentially solvable and Γ is Abelian. Proof. We obviously get from Theorem 1.4.5 that R is bijective, and it is clear that R is continuous. To prove the continuity of its inverse, let (φn )n be a sequence in Hom(Γ, G), which converges to some element φ. We denote by (ψn )n (resp., ψ) the extensions of (φn )n (resp., φ). To prove that the sequence (ψn )n converges to ψ, it is sufficient to show

48 | 1 Structure theory that (ψn (γ(t)))n converges to ψ(γ(t)) for every γ ∈ Γ. Finally, by (1.39) we have for all γ ∈ Γ, lim ψn (γ(t)) = lim exp(t log(φn (γ)))

n→∞

n→∞

= exp(t log(φ(γ)))

= ψ(γ(t)).

1.4.3 Case of reduced exponential Lie groups Let us first recall the notion of the universal covering of a Lie group. Here, we record some results, which will be of interest in our study and prove the existence of a unique syndetic hull for any closed subgroup of a reduced completely solvable Lie group. We first record the following results. Theorem 1.4.8 ([56, Theorem XII.10]). Let G be a connected Lie group. Then there exists ̃ and a Lie group homomorphism π : G ̃ → G, a connected simply connected Lie group G ̃ In which is a covering. The kernel of π is a discrete, normal subgroup, so central in G. ̃ addition, up to isomorphism, G is unique. The set is called the universal covering of G. Theorem 1.4.9 ([80, Proposition C.8]). Let G and H be two Lie groups associated to Lie algebras g and h, respectively. Let F be a continuous group homomorphism from G to H. Then there exists an algebra homomorphism f from g to h such that F ∘ expG = expH ∘f where as in (1.3), expG and expH are the exponential mappings of G and H, respectively. Definition 1.4.10. A reduced solvable Lie group G (resp., exponential solvable, com̃ is solvable (resp., exponential solvable, completely solvable, nilpotent) is such that G pletely solvable, nilpotent). Throughout this section, g will denote a real exponential solvable Lie algebra and ̃ its simply connected associated Lie group. That is, G ̃ is connected and simply conG nected and it is the universal covering of a connected Lie group G, for which the expõ have the same Lie algebra. nential mapping may fail to be injective. Besides, G and G Furthermore, the following is true. Proposition 1.4.11. Let G be a reduced connected exponential solvable Lie group. Then the exponential mapping expG is surjective. ̃ be the universal covering of G and π : G ̃ → G the associated covering as Proof. Let G ̃ in Theorem 1.4.8. We have expG̃ : g → G is a diffeomorphism. Therefore, according to Theorem 1.4.9, there exists a Lie algebra endomorphism f of g such that π ∘ expG̃ = expG ∘f . As π ∘ expG̃ is surjective, then so is expG ∘f . Thus, the map expG is surjective.

1.4 Syndetic hulls | 49

The following result generalizes Theorem 1.4.2 and establishes the existence of the syndetic hull of any closed subgroup of a reduced connected completely solvable Lie group. More precisely, we have the following. Theorem 1.4.12. Any closed subgroup Γ of a reduced connected completely solvable Lie group admits a unique syndetic hull L, where L = expG l, l = ℝ-span(logG̃ π −1 (Γ)) and logG̃ is as in Definition 1.1.1. Proof. Let Γ be a closed subgroup of a reduced completely solvable Lie group G. First, we show that there exists a connected Lie subgroup of G, which contains Γ cocom̃ be the universal covering of G, and π be the associative covering π : pactly. Let G ̃ G 󳨀→ G. We denote by Λ = ker π. As π is continuous, Γ̃ = π −1 (Γ) is a closed subgroup ̃ Since G ̃ is a connected, simply connected and completely solvable, then accordof G. ̃ Let us prove that L = π(L) ̃ ing to Theorem 1.4.2, Γ̃ has unique syndetic hull, say, L. ̃ ̃ is a connected Lie subgroup of G, which contains Γ cocompactly. We have Γ ⊂ L, so ̃ Γ, ̃ of G, ̃ then ̃=C ̃ for some compact set C Γ ⊂ π(π −1 (Γ)) ⊂ L. In addition, we have L ̃ Γ) ̃ ̃ = π(C)π( ̃ = π(C)Γ. π(L) ̃ Let log ̃ Λ = ̃ is closed in G. We must show that L is closed in G, which means that LΛ G ̃ ̃A ̃ is ℤ-span(Z1 , . . . , Zd ) for some d ∈ ℕ, a = ℝ-span(Z1 , . . . , Zd ) and A = expG̃ a. Then L ̃ as L ̃ ̃A ̃ is a connected subgroup in a simply connected solvable Lie group G. closed in G ̃ ̃ ̃ Then by Theorem 1.1.6, there exits a coexponential basis of L in LA, which means that ̃A ̃ is diffeomorphic to L ̃ × ℝs and conclusively LΛ ̃ to L ̃ × ℤs for some s ⩽ d. Therefore, L ′ c ̃ Let Γ = ΓZ (G), which is a closed subgroup of G. Then Γ′ admits at ̃ is closed in G. LΛ

least one connected Lie subgroup L of G containing it cocompactly. We now show that L is unique. Indeed, if L1 = expG l1 and L2 = expG l2 are two such Lie groups. We claim that Li /(L1 ∩ L2 ), i = 1, 2 are compact. To see that, consider for i = 1, 2 the canonical surjection si : Li 󳨀→ Li /(L1 ∩ L2 ),

which factors through the canonical surjection ρi : Li → Li /Γ to a surjection s̃i : Li /Γ → Li /(L1 ∩ L2 ) such that si = s̃i ∘ ρi . The map s̃i is surjective and continuous, and thus its image Li /(L1 ∩ L2 ) is compact. Moreover, it is obvious that Li /(L1 ∩ L2 ) is homeomorphic to (Li /Z c (G))/((L1 ∩ L2 )/Z c (G)), which is homeomorphic to ℝp for some p ∈ ℕ. Indeed, G/Z c (G) turns out to be a connected simply connected completely solvable Lie group and the existence of the coexponential basis of (L1 ∩ L2 )/Z c (G) in Li /Z c (G) allows us to conclude. Finally, as this quotient is compact, we get conclusively that p = 0. Hence, L1 ∩ L2 = L1 = L2 , as was to be shown. ̃ be the universal covering of a connected Lie group G and π : G ̃→G Let as above G −1 the covering map. A pre-Abelian subgroup Γ of G is a subgroup such that Γ̃ = π (Γ) is

50 | 1 Structure theory Abelian. When more generally G is exponential solvable and connected, the following could also be seen. Proposition 1.4.13. Any pre-Abelian closed subgroup of a reduced connected exponential solvable Lie group admits a unique syndetic hull. Proof. Keep the same notation as in the proof of Theorem 1.4.12. In this situation, Γ̃ is ̃ exists by Proposition 1.4.3. Then l = log ̃ (L) ̃ is an Abelian Lie subalgebra Abelian and L G of g. Finally, expG (l) is a syndetic hull of Γ and the unicity is immediate.

2 Proper actions on homogeneous spaces The topological property of being proper of an action of a group G on a locally compact space M assures a good behavior for a topological group action. Such actions particularly admit nice properties (the stabilizers are compact, the G-orbits are closed, etc.) when G and M are submitted to some conditions, which has important consequences for the structure of the G-space and the orbit space M/G. On the other hand, the determination of a criterion of the proper action appears to be a key ingredient point in the computation of the parameter, deformation and moduli spaces of the action of a discontinuous group, as we shall see throughout the next coming chapters. We first give basic definitions of proper and free actions and also define weak and finite proper actions, discontinuous groups and Clifford–Klein forms. We shall then focus on the characterization of the proper (and weak and finite) action of connected, closed subgroups on special and maximal solvmanifolds, stating that it is equivalent to free actions (cf. [35] and [36]). Passing to discrete actions, many phenomena happen and some criteria of properness are also obtained. In the case where G is n-step nilpotent, the proper action of a closed, connected subgroup is shown (with detailed proofs) to be equivalent to its free actions whenever n ≤ 3. We also generate a geometric criteria of the proper action of a discontinuous group on an arbitrary homogeneous space, where the group in question is the semidirect product group K ⋉ℝn (K a compact subgroup of GL(n, ℝ)) and in the case of Heisenberg motion groups. As shown, this will be a capital role in the study of many geometrical concepts related to corresponding deformation and moduli spaces.

2.1 Proper and fixed-point actions Let M be a locally compact space and K a locally compact topological group. The continuous action of the group K on M is said to be: (1) Proper if, for each compact subset S ⊂ M the set KS = {k ∈ K : k ⋅ S ∩ S ≠ 0} is compact (cf. [116]). (2) Fixed point free (or free) if, for each m ∈ M , the isotropy group Km = {k ∈ K : k ⋅ m = m} is trivial. (3) The action has the compact intersection property, denoted (CI), if for each x ∈ X, the isotropy group Kx is compact (see [93]). In the case where G is a Lie group and H and K are closed subgroups of G, the action of K on the homogeneous space M = G/H is proper if SHS−1 ∩ K is compact for any compact set S in G. Likewise the action of K on M is (CI) (resp., free), if for every g ∈ G, K ∩ gHg −1 is compact (resp., K ∩ gHg −1 = {e}). For the sake of brevity, one says that the triple (G, H, K) is proper, (CI) or free, respectively. https://doi.org/10.1515/9783110765304-002

52 | 2 Proper actions on homogeneous spaces Here, for two sets A and B of the locally compact topological group G, the product AB is the subset {ab : a ∈ A, b ∈ B}.

2.1.1 Discontinuous groups Let M be a locally compact space and K a locally compact topological group. The action of the group K on M is said to be properly discontinuous if K is discrete, and for each compact subset S ⊂ M , the set KS is finite. The group K is said to be a discontinuous group for the homogeneous space M if K is discrete and acts properly and fixed point free on M .

2.1.2 Clifford–Klein forms For any given discontinuous group Γ for a homogeneous space G/H, the quotient space Γ\G/H is said to be a Clifford–Klein form for the homogeneous space G/H. It is then well known that any Clifford–Klein form is endowed through the action of Γ with a manifold structure for which the quotient canonical surjection π : G/H → Γ\G/H

(2.1)

turns out to be an open covering and particularly a local diffeomorphism. On the other hand, any Clifford–Klein form Γ\G/H inherits any G-invariant geometric structure (e. g., complex structure, pseudo-Riemanian structure, conformal structure, symplectic structure, etc.) on the homogeneous space G/H through the covering map π defined as in equation (2.1) below. Definition 2.1.1 (cf. [95]). Let H and K be subsets of a locally compact topological group G. We denote by H ⋔ K in G if the set SHS−1 ∩ K is relatively compact for any compact set S in G. Here, SHS−1 = {ahb−1 , a, b ∈ S, h ∈ H}. We denote by H ∼ K in G if there exists a compact set S of G such that K ⊂ SHS−1 and H ⊂ SKS−1 . Let also ⋔gp (Γ : G) be the set of closed subgroups belonging to ⋔ (Γ : G). With the notation above, we have the following results. Fact 2.1.2 (cf. [95]). Let G be a locally compact topological group, H, H ′ and K be three subsets of G. Then: (i) H ⋔ K in G if and only if H acts on G/K properly. (ii) If H ∼ H ′ and if H ⋔ K in G, then H ′ ⋔ K in G. (iii) H ⋔ K in G if and only if K ⋔ H in G. (iv) H ⋔ K in G if and only if H ⋔ K in G. Here, K denotes the closure of K in G.

2.1 Proper and fixed-point actions | 53

Fact 2.1.3 (cf. [95]). Suppose G′ is a closed subgroup of a locally compact topological group G. Let H and K be subsets of G′ . (i) If H ∼ K in G′ , then H ∼ K in G. (ii) If H ⋔ K in G, then H ⋔ K in G′ .

2.1.3 Weak and finite proper actions In this section, we single out the definitions of weak and finite proper actions. We shall show that these notions of actions are equivalent. Definition 2.1.4. Let K be a locally compact group and X be a K-locally compact space. We say that the action of K on X is: (1) weakly proper if for every compact set S in X; the set Kx,S = {k ∈ K : k ⋅ x ∈ S} is compact for every x ∈ X, (2) finitely proper if for every finite set S in X; the set KS = {k ∈ K : k ⋅ S ∩ S ≠ 0} is finite. We have the following. Lemma 2.1.5. Let G be a locally compact topological group, and let H and K be closed subgroups of G. Then: (i) K acts weakly properly on G/H if and only if for every compact set S in G and every g ∈ G, the set K ∩ SHg is compact. (ii) K acts weakly properly on G/H if and only if H acts weakly properly on G/K. (iii) K acts finitely properly on G/H if and only if for every finite set S in G the set K ∩ SHS−1 is finite. (iv) K acts finitely properly on G/H if and only if H acts finitely properly on G/K. (v) Both finite proper action and weak proper action imply (CI)-action. (vi) Proper action implies weak proper action. Proof. Let S be a compact of G then S̄ = SH is a compact of G/H. Let g ∈ G, x = gH and k ∈ Kx,̄ S̄ . Then kgH ⊂ SH and, therefore, k ∈ SHg −1 . Finally, Kx,̄ S̄ = K ∩ SHg −1 , which proves assertion (i). Remark now that for every compact set S in G and every g ∈ G, we have K ∩ SHg ⊂ S(S−1 Kg −1 ∩ H)g, which proves (ii).

54 | 2 Proper actions on homogeneous spaces To prove (iii), it is then sufficient to see that for every finite set S in G, KS̄ = K ∩ SHS−1 , where S̄ = SH is also a finite set of G/H. The assertions (iv), (v) and (vi) are clear. In [101], the authors define the notion of the right strongly similar of two nonempty subsets L and H in a locally compact group G denoted by L ∼s H. We point out here that one can similarly define the notion of the left strongly similar denoted by L ∼ℓs H, which means that there exist g0 ∈ G and a compact S0 in G such that L ⊂ S0 Hg0 and H ⊂ S0 Lg0 . In the setting where the sets L and H are subgroups of G, the notions ∼s and ∼ℓs are evidently equivalent. Moreover, the following result stems directly from the definition above. Proposition 2.1.6. Let G be a locally compact topological group and let H, L and K be closed subgroups of G. Assume that L ∼ℓs H. Then the triple (G, H, K) is weakly proper if and only if the triple (G, L, K) is. We will need throughout the book to use an induction on the dimension of the group G descending to lower-dimensional subgroups. The following elementary fact due to T. Kobayashi dealing with proper actions under an equivariant map will be used. Fact 2.1.7 ([94]). Let Gi (i = 1, 2) be locally compact groups and Hi , Ki ⊂ Gi be closed subgroups. Suppose that f : G1 → G2 is a continuous homomorphism such that f (H1 ) ⊂ H2 and f (K1 ) ⊂ K2 . Assume that f (K1 ) is closed in G2 and K1 ∩ ker f is compact. If K2 acts properly on G2 /H2 , then K1 acts properly on G1 /H1 . As remarked by T. Kobayashi, there is a strong relationship between the proper action of a cocompact discrete subgroup and the proper action of its syndetic hull on a locally compact Hausdorff space. More precisely, one has the following. Fact 2.1.8 (cf. [92, Lemma 2.3]). Suppose a locally compact group L acts on a Hausdorff, locally compact space X. Let Γ be a cocompact discrete subgroup of L. Then the L-action on X is proper if and only if the Γ-action on X is properly discontinuous. In [92], T. Kobayashi made a bridge between the action of a discrete group and that of a connected group by noticing that if Γ is a cocompact discrete subgroup of a connected subgroup K, then the action of K on X is proper if and only if the action of Γ on X is properly discontinuous. In this context, and as an analogy of fundamental questions on discontinuous groups, he poses the following problems in the continuous setting.

2.1 Proper and fixed-point actions | 55

Problem 1: Find a criterion on the triple (G, H, K) such that the action of K on G/H is proper. Problem 2: Find a criterion on the triple (G, H, K) such that the double coset K \ G/H is compact for the quotient topology. The following problem due to T. Kobayashi is strongly linked to Problem 1 and somehow worth being emphasized: Problem 3: When is it true that the property (CI) implies the properness of the action of K on X? In [95], an affirmative answer was given by T. Kobayashi himself in the case where each of the triple (G, H, K) is reductive. Moreover, the last question may have a negative answer even in the case where G is reductive or Abelian (see [92] and [114]). For n ≥ 1, let Gn = Nn (ℝ) ⋉ ℝn , where Nn (ℝ) designates the group of n-upper real triangular matrices. Several authors considered the action given by an affine transformation subgroup contained in Gn . For n = 2, more in fact is true. T. Kobayashi shows in [93] that for G = GL(2, ℝ) ⋉ ℝ2 and H = GL(2, ℝ), the triple (G, H, K) is (CI) if and only if K acts properly on G/H for any connected closed subgroup K in G. In [108], R. Lipsman established the last result for n = 3 taking N3 (ℝ) for one of the subgroups in question. He conjectured the following. Conjecture 2.1.9. Let G be a simply connected nilpotent Lie group, H and K be connected subgroups of G. Then the triple (G, H, K) has the (CI) property if and only if K acts properly on G/H. T. Yoshino proved in [136] that the result holds for n = 4 as well, but it fails for every n ≥ 5 (see [134]). This implies that Conjecture 2.1.9 may fail for n-step nilpotent Lie groups (n ≥ 4). We here aim to partially answer the above questions in some situations of exponential solvable homogeneous spaces. More precisely, we shall prove Conjecture 2.1.9 below for some classes of nilpotent Lie groups and go beyond the nilpotent for some restricted contexts as well. We first introduce the following. Definition 2.1.10. Let G be an exponential solvable Lie group and H a connected and closed subgroup of G. A pair (G, H) is said to have the Lipsman property, if for any closed, connected subgroup L of G, there is equivalence between proper and fixed point- free for the triple (G, H, L). Exponential solvable Lie groups admit no nontrivial connected compact subgroups, which merely entails that properties (2) and (3) in Definitions 2.1 are equivalent as the isotropy group Kx is K ∩ gHg −1 for x = gH. On the other hand, it is obvious that Property (1) implies both Properties (2) and (3) in this case; so, it appears natural to seek the converse. With the above in mind, the following lemma is immediate.

56 | 2 Proper actions on homogeneous spaces Lemma 2.1.11. Let G be an exponential solvable Lie group; H and K are closed connected subgroups of G. Then the following conditions are equivalent: (i) The triple (G, H, K) has the (CI) property. (ii) The action of K on G/H is free, that is, K ∩ gHg −1 = {e} for any g ∈ G. (iii) k ∩ Adg h = {0} for any g ∈ G. Here, h and k are the Lie algebras respectively of H and K.

2.1.4 Campbell–Baker–Hausdorff series Let G be an exponential solvable Lie group. In this context, the Campbell–Baker– Hausdorff formula allows to reconstruct the group G with its multiplication law knowing only the structure of g. Furthermore, for every A, B ∈ g, we have exp A exp B = exp(A ∗ B) = exp C(A, B), where C(A, B) = ∑n≥1 Cn (A, B) and Cn (A, B) is determined by the recursion formula (see [128]): C1 (A, B) = A + B and for n ≥ 1, 1 (n + 1)Cn+1 (A, B) = [A − B, Cn (A, B)] 2 + ∑ K2p ∑ p≥1,2p≤n

[Ck1 (A, B), [⋅ ⋅ ⋅ [Ck2p (A, B), A + B] ⋅ ⋅ ⋅]],

k1 ,...,k2p >0 k1 +⋅⋅⋅+k2p =n

where the rational numbers K2p , p ≥ 1 are given by +∞ z 1 z = 1 + − ∑ K2p z 2p . 1 − e−z 2 p=1

This series is absolutely convergent. In particular, 1 C2 (A, B) = [A, B] 2 and C3 (A, B) =

1 1 [A, [A, B]] + [[A, B], B]. 12 12

Furthermore, if G is nilpotent, then C(A, B) is a polynomial function in both the variables A and B.

2.1 Proper and fixed-point actions | 57

2.1.5 Proper actions and coexponential bases We have then the following. Proposition 2.1.12. Let G be a connected simply connected solvable Lie group, H a connected subgroup and let {X1 , . . . , Xp } be a coexponential basis of H in G as in Theorem 1.1.6. Let S be a compact set of G, then there exist two compact sets SH and S1 = ∏1i=p exp(Ii Xi ) such that (Ii )1≤i≤p are compact sets of ℝ, SH ⊂ H and S ⊂ S1 SH . Proof. Let S be a compact set in G, then (φg,h )−1 (S) is a compact set in ℝp × H. So, there exist some compact sets (Ii )1≤i≤p of ℝ and a compact set SH of H such that (φg,h )−1 (S) ⊂ I1 × ⋅ ⋅ ⋅ × Ip × SH . Then S ⊂ ∏1i=p exp(Ii Xi )SH . Corollary 2.1.13. Let G be a connected simply connected solvable Lie group, and H and K be connected subgroups of G. Then the action of K on G/H is proper if and only if for every compact set S of G of the form S = ∏1i=p exp(Ii Xi ), where (Ii )1≤i≤p are compact sets of ℝ, and the set SHS−1 ∩ K is relatively compact in G. Proof. Let S be a compact set in G, using Proposition 2.1.12, we have that S ⊂ S1 SH where S1 = ∏1i=p exp(Ii Xi ), and SH is a compact set in H. Then SHS−1 ∩ K ⊂ S1 SH HSH−1 S1−1 ∩ K = S1 HS1−1 ∩ K, which is relatively compact in G according to our assumption. The following result will be used later and deals with proper action when one of the subgroups is normal in G. Proposition 2.1.14. Let G be a connected, simply connected, solvable Lie group and H, K connected subgroups of G. Assume that one of the subgroups H or K is normal in G, then K acts properly on G/H if and only if triple (G, H, K) has the (CI) property. Proof. Without loss of generality, we may and do assume that H is normal. Then the result follows from Fact 2.1.7 with f being the natural quotient map f : G → G/H. We must show that f (K) is closed in G/H, which is equivalent to the fact that HK is closed in G. To see that, we engage an induction on the dimension of G. This is obvious for small dimensions. Let G0 be a normal subgroup of G of codimension one containing H. If K ⊂ G0 , we are done. Otherwise, let X ∈ k \ g and K0 = K ∩ G0 , which is a closed subgroup of G0 . Obviously, thanks to Remark 1.1.7, {X} is a coexponential basis of G0 in G, and we have HK = HK0 ⋅ exp(ℝX), which closes the proof as HK0 is a closed in G.

58 | 2 Proper actions on homogeneous spaces

2.2 Proper actions for 3-step nilpotent Lie groups We consider hereafter the case of 3-step nilpotent Lie groups. In this context, the ascending central sequence is defined by g(0) = {0}, g(j) = {X ∈ g : X is central mod g(j−1) }. Then each g(j) is an ideal of g, g(1) = z(g) is the center of g, g(2) = d = [g, g] and we have g(0) ⊊ g(1) = z(g) ⊊ g(2) ⊊ g(3) = g. Furthermore, for every A, B ∈ g, we have (cf. Section 2.1.4) 1 1 1 exp A exp B = exp(A ∗ B) = exp(A + B + [A, B] + [A, [A, B]] + [B, [B, A]]). 2 12 12 In the case of 2-step nilpotent Lie groups (i. e., g(2) = g), we prove the following result. An alternative proof is also provided in [114]. Theorem 2.2.1. Let G be a connected, simply connected at most 2-step nilpotent Lie group, H and K connected subgroups of G. Then K acts properly on G/H if and only if the triple (G, H, K) has the (CI) property. Proof. Suppose that the action of K on G/H is not proper, then there exists a compact set S in G such that K ∩ SHS−1 is not relatively compact. Hence, one can find sequences Vn ∈ h, Wn ∈ k, An , Bn ∈ g, such that: (2.2.1.1) exp An ∈ S, exp Bn ∈ S, limn→+∞ An = A, limn→+∞ Bn = B, (2.2.1.2) limn→+∞ ‖Vn ‖ = limn→+∞ ‖Wn ‖ = +∞, W V (2.2.1.3) limn→+∞ ‖Vn ‖ = V, limn→+∞ ‖Wn ‖ = W, where V ∈ h, W ∈ k and n n ‖V‖ = ‖W‖ = 1, (2.2.1.4) exp(Wn ) = exp(An ) exp(Vn ) exp(−Bn ). The last equation (2.2.1.4) gives 1 Wn = An ∗ (−Bn ) + Vn + [An + Bn , Vn ] = Adexp( An +Bn ) Vn + An ∗ (−Bn ). 2 2 Let αn =

‖Vn ‖ , ‖Wn ‖

then

Wn ‖Wn ‖

tends to +∞, we obtain

= αn Adexp( An +Bn ) 2

Vn ‖Vn ‖

+

1 (An ‖Wn ‖

W = lim αn Adexp( An +Bn ) n→+∞

2

∗ (−Bn )). Taking the limit as n

Vn . ‖Vn ‖

2.2 Proper actions for 3-step nilpotent Lie groups | 59

Since V ≠ 0, the limit limn→+∞ Adexp( An +Bn ) 2

Vn ‖Vn ‖

= Adexp( A+B ) V is not zero, which im2

plies that the sequence αn converges to some α ∈ ℝ∗+ , and finally W = Adexp( A+B ) (αV), which is impossible by the (CI) property of (G, H, K).

2

We generalize now the above result. We prove the following. Theorem 2.2.2. Let G be a connected simply connected at most 3-step nilpotent Lie group, and H and K be connected subgroups of G. Then K acts properly on G/H if and only if the triple (G, H, K) has the (CI) property. Proof. Let us drop for the moment the assumption that G is 3-step. Let S : {0} = g0 ⊂ g1 ⊂ ⋅ ⋅ ⋅ ⊂ gm = g

(2.2)

be a strong Malcev sequence and Gi = exp(gi ), i = 0, . . . , m. Choosing for every j ∈ {1, . . . , m} a vector Zj in gj \ gj−1 , we obtain a strong Malcev basis B = {Z1 , . . . , Zm } of g. We denote by I

h

= I = {i1 < ⋅ ⋅ ⋅ < ia }

(a = dim h)

the set of indices i (1 ≤ i ≤ m) such that h ∩ gi ≠ h ∩ gi−1 . Let us put J

g/h

= J = {j1 < ⋅ ⋅ ⋅ < jp } = {1, . . . , m} \ I ,

with p = dim(g/h) = m − a. So, it appears clear that we can pick the vectors Zk , k ∈ I of B in such that Zk ∈ h. This implies that the basis {X1 = Zj1 , . . . , Xp = Zjp } is a Malcev basis of g relative to h, which is a coexponential basis of h in g. So by Corollary 2.1.13, the action of K on G/H is proper if and only if for every compact set S of G of the form S = ∏1i=p exp(Ii Xi ), where (Ii )1≤i≤p are compact sets of ℝ; the set SHS−1 ∩ K is relatively compact in G. Even more, the following more general fact holds. Proposition 2.2.3. We keep the same notation and hypotheses. The subgroup K acts on G/H properly if and only if for every q ∈ {1, . . . , m} and for every compact set Sq of G included in Gq , the set SHS−1 ∩ K is relatively compact in G, where S=

∏

k∈J , k>q

exp(Ik Zk )Sq

and (Ik )k∈J , k>q are compact sets in ℝ. We come back now to 3-step nilpotent Lie groups. Recall the notation d = [g, g] and D = exp(d). We prove first the following. Lemma 2.2.4. Let G be a connected, simply connected at most 3-step nilpotent Lie group, and H and K be connected subgroups of G such that (G, H, K) has the (CI) property. Let S be a compact set of G. Then:

60 | 2 Proper actions on homogeneous spaces (1) If S is included in D, the set SHS−1 ∩ K is relatively compact in G. (2) If H or K is included in D, the action of K on G/H is proper. Proof. Let S be a compact set in G such that SHS−1 ∩ K is not relatively compact. Then there exist some sequences An , Bn ∈ g, Vn ∈ h and Wn ∈ k meeting conditions (2.2.1.1), . . . , (2.2.1.4) of Theorem 2.2.1. So, if S ⊂ D according to (2.2.4.1) or if one of the subgroups H or K (assume, e. g., H) is included in D according to (2.2.4.2), then equation (2.2.1.4) gives 1 Wn = An ∗ (−Bn ) + Vn + [An + Bn , Vn ] = An ∗ (−Bn ) + Adexp( An +Bn ) Vn . 2 2 Hence, the same procedure as in the proof of Theorem 2.2.1 gives us a contradiction. We suppose from now on that both h and k are not included in d. Take by the way {Y1 , . . . , Yq }, a Malcev basis of h relative to d ∩ h and {Y1′ , . . . , Yr′ }, a Malcev basis of k relative to d ∩ k. Let {i1 , . . . , is } be a maximal set of indices in {1, . . . , r} such that the vectors {Y1 , . . . , Yq , Yi′1 , . . . , Yi′s } are linearly independent modulo d. We note Tk = Yi′k , k = 1, . . . , s. Remark that q

d ⊕ (⨁ ℝYk ) = d + h k=1

and that q

s

d ⊕ (⨁ ℝYk ) ⊕ (⨁ ℝTk ) = d + h + k. k=1

k=1

In the case where d + h + k = g, the action of K on G/H is proper. In fact, let S be a flag of ideals as in (2.2) such that gd = d and constructed by means of the vectors {Y1 , . . . , Yq , T1 , . . . , Ts }. Let also S = ∏1k=s exp(Ik Tk )Sd be a compact set of G where (Ik )1≤k≤s are compact sets of ℝ and Sd is a compact set of D. We have to show according to Proposition 2.2.3 that K ∩ SHS−1 is relatively compact. In fact, 1

s

K ∩ SHS−1 = K ∩ ∏ exp(Ik Tk )(Sd H(Sd ) ) ∏ exp(−Ik Tk ) 1

−1

k=s

k=1 s

= ∏ exp(Ik Tk )(K ∩ Sd H(Sd ) ) ∏ exp(−Ik Tk ). k=s

−1

k=1

But K ∩ Sd H(Sd )−1 is relatively compact using the assertion (1) of Lemma 2.2.4, which completes the proof in this case. Suppose finally that d + h + k ⊊ g. Let {X1 , . . . , Xl } be a

2.2 Proper actions for 3-step nilpotent Lie groups | 61

Malcev basis of g relative to d + h + k, and {Z1 , . . . , Zd } a strong Malcev basis of d passing through d ∩ h. The basis B = {Z1 , . . . , Zd , X1 , . . . , Xl , T1 , . . . , Ts , Y1 , . . . , Yq },

is a strong Malcev basis of g. We take a corresponding flag of ideals of g such that gd = d. We prove now the following. Lemma 2.2.5. Let G be a connected, simply connected at most 3-step nilpotent Lie group, and H and K connected subgroups of G such that (G, H, K) has the (CI) property. With the same notation as above, the action of K on G/H is proper if and only if for every compact set of G of the form S = ∏1k=l exp(Ik Xk )Sd , where (Ik )1≤k≤l , Sd are respectively compact sets of ℝ and D; the set SHS−1 ∩ K is relatively compact in G. Proof. In order to prove that the action of K on G/H is proper, it is clear that according to Proposition 2.2.3, we merely have to show that S1 HS1−1 ∩ K is relatively compact in G for every compact S1 of G of the form S1 = ∏1k=s exp(Jk Tk ) ∏1k=l exp(Ik Xk )Sd , where (Ik )1≤k≤l and (Jk )1≤k≤s are compact sets in ℝ and Sd is a compact set of D. Write SK = ∏1k=s exp(Jk Tk ) and S = ∏1k=l exp(Ik Xk )Sd . We then have S1 HS1−1 ∩ K ⊂ SK SHS−1 SK−1 ∩ K = SK (SHS−1 ∩ K)SK−1 , which is relatively compact in G using our hypothesis. Let us go back to the proof of Theorem 2.2.2. Suppose that the action of K on G/H is not proper, then there exists by Lemma 2.2.5 a compact subset S = ∏1k=l exp(Ik Xk )Sd of G such that (Ik )1≤k≤l are compact sets of ℝ, Sd is a compact set of D and K ∩ SHS−1 is not relatively compact in G. Thus, we can find as above some sequences Vn ∈ h, Wn ∈ k, An , Bn ∈ g meeting conditions (2.2.1.1), . . . , (2.2.1.4) of Theorem 2.2.1. By equation (2.2.1.4), we have that exp(Wn ) exp(Cn ) = exp(An ) exp(Vn ) exp(−An ) = Adexp An Vn ,

(2.3)

where Cn = Bn ∗ (−An ). Then we have by (2.3) that Wn + Cn = Vn mod(d).

(2.4)

It follows according to our choice of the basis B and (2.4), that Wn = Vn mod(d) and that Cn ∈ d. Then equation (2.3) gives Cn + Adexp(− 1 C ) Wn = Adexp An Vn 2

n

62 | 2 Proper actions on homogeneous spaces as [Cn , g] ⊂ z(g). Writing as above αn =

‖Vn ‖ , ‖Wn ‖

we get

Wn V Cn + Adexp(− 1 C ) = αn Adexp An n . n 2 ‖Wn ‖ ‖Wn ‖ ‖Vn ‖ Taking the limit as n tends to +∞, we obtain Adexp(− 1 C) W = α Adexp A V, 2

where C = limn→+∞ Cn , α = limn→+∞ αn and A = limn→+∞ An . This contradicts the (CI) property of the triple (G, H, K), which achieves the proof of the theorem. An alternative proof of Theorem 2.2.2 is subject of the paper [133].

2.3 Special nilpotent Lie groups We consider in this section Conjecture 2.1.9 in another setting. A solvable Lie algebra g is said to be special, if it admits a codimensional one Abelian ideal. In terms of groups, the Lie group G associated to g turns out to be a semidirect product ℝ ⋉ ℝm being noncommutative. Let Gm = exp(gm ) be the threadlike nilpotent Lie group as defined in Subsection 1.1.3. We begin this section by proving Conjecture 2.1.9 for Gm , m ≥ 2. Proposition 2.3.1. Conjecture 2.1.9 holds for Gm , m ≥ 2. Proof. Recall the notation, K = exp(k), H = exp(h) and G0 = exp(g0 ). We proceed by induction on the step m of G. If m = 2, then G is 2-step and we are already done by Theorem 2.2.1. We suppose then that m > 2 and we shall discuss several cases. Suppose in a first time that both h and k are included in g0 and suppose that the K action on G/H is not proper. There exists a compact set S in G such that K ∩SHS−1 is not relatively compact. There exist by Proposition 2.2.3 a compact set S0 in g0 and a compact set I in ℝ such that S ⊂ exp(IX)S0 . Thus, we can find sequences An , Bn ∈ g0 , tn , sn ∈ I, Vn ∈ h and Wn ∈ k, such that: (2.3.1.1) (2.3.1.2) (2.3.1.3) (2.3.1.4)

exp(An ) ∈ S0 , exp(Bn ) ∈ S0 , limn→+∞ tn = t, t ∈ I and limn→+∞ sn = s, s ∈ I, limn→+∞ ‖Vn ‖ = limn→+∞ ‖Wn ‖ = +∞, V W limn→+∞ ‖Vn ‖ = V, limn→+∞ ‖Wn ‖ = W, where V ∈ h, W ∈ k and n n ‖V‖ = ‖W‖ = 1, (2.3.1.5) exp(Wn ) = exp(tn X) exp(An ) exp(Vn ) exp(−Bn ) exp(−sn X). The last equation gives exp(Wn ) = exp(tn X) exp(An + Vn − Bn ) exp(−tn X) exp((tn − sn )X).

2.3 Special nilpotent Lie groups | 63

Thus, sn = tn and Wn = Adexp tn X (An − Bn + Vn ), as g0 is Abelian. Denoting αn = n ∈ ℕ, we get

‖Vn ‖ , ‖Wn ‖

Wn A − Bn + Vn = αn Adexp tn X ( n ). ‖Wn ‖ ‖Vn ‖ V +A −B

Since V ≠ 0 and limn→+∞ Adexp(tn X) ( n ‖Vn ‖ n ) = Adexp(tX) V ≠ 0, αn converges to some n α ∈ ℝ∗+ , and finally W = α Adexp tX V, which is impossible by the (CI) property. Suppose now that both h and k are not contained in g0 . We can then assume that X ∈ h. So, there exists a one-codimensional ideal g1 of g, which contains h. As [X, Yi ] ∈ g1 , i = 1, . . . , m − 1, it follows that g1 = span{X, Y2 , . . . Ym }. When k ⊂ g1 , we write m−1 k = ℝY ⊕ k0 where k0 = k ∩ g0 and Y = X + ∑m i=2 yi Yi , for some yi ∈ ℝ. Let T = ∑i=1 yi+1 Yi . Then m−1

Adexp T Y = Y + [ ∑ yi+1 Yi , Y] m

i=1

m−1

= X + ∑ yi Yi + ∑ yi+1 [Yi , X] i=2

i=1

m

m−1

i=2

i=1

= X + ∑ yi Yi − ∑ yi+1 Yi+1 = X, which contradicts the (CI) property of (G, H, K). Suppose now that k ⊄ g1 . Write likewise k = ℝX1 ⊕ k1 where k1 = k ∩ g1 . It is clear that {X1 } is a coexponential basis of g1 in g. Let S a compact set of G, there exist a compact set J in ℝ and a compact set S1 in G1 = exp(g1 ) such that S ⊂ exp(JX1 )S1 . Then, noting K 1 = exp(k1 ), we have K ∩ SHS−1 ⊂ exp(JX1 )(K ∩ S1 HS1 −1 ) exp(−JX1 )

= exp(JX1 )(K 1 ∩ S1 HS1 −1 ) exp(−JX1 ).

(2.5)

Since g1 is (m − 1)-step threadlike Lie algebra and the triple (G1 , H, K 1 ) fulfills the (CI) property, the set K 1 ∩ S1 HS1 −1 turns out to be relatively compact in G1 by the induction hypothesis. We conclude then that K acts properly on G/H by (2.5). Finally, when only one of the subalgebras h or k is included in g0 , we argue similarly as above replacing G1 by G0 , K 1 by K 0 = exp(k0 ) and using the fact that G0 is Abelian. We extend now the above result to an arbitrary nilpotent special Lie group, and we have the following. Theorem 2.3.2. Conjecture 2.1.9 holds for special nilpotent Lie groups. Proof. Let G be a special nilpotent Lie group and g its Lie algebra. We write g = g0 ⊕ ℝX where g0 is the one-codimensional Abelian ideal of g and G0 = exp(g0 ) and X ∈ g \ g0 . Recall the notation z(g) of the center of g. We proceed by induction on dim G.

64 | 2 Proper actions on homogeneous spaces We suppose in a first time that h or k contains a nonzero central vector of g; take, for example, h ∩ z(g) ≠ {0}. Let then Z ∈ h ∩ z(g) and i = span{Z}. We note I = exp(i) and G = G/I. We consider the canonical surjection: f : G → G. Then f is a continuous homomorphism, and let f (H) = H and f (K) = K. As i ⊂ g0 , it is clear that G is a special Lie group and if the triple (G, H, K) has the (CI) property, then the triple (G, H, K) does. In fact, let B ∈ h, A ∈ k and T ∈ g such that A = Adexp T B, then there exists α ∈ ℝ such that B = Adexp T A + αZ. This implies that B − αZ = Adexp T A. Finally, B = αZ as (G, H, K) has the (CI) property and, therefore, B = 0. We apply the induction hypothesis to G to obtain that the action of K on G/H is proper. Moreover, f (K) is closed in G and K ∩ ker f = K ∩ I = {e} is compact. Using Lemma 2.1.7, we conclude that the action of K on G/H is proper. Hence, we can suppose from now on that h ∩ z(g) = k ∩ z(g) = {0}.

(2.6)

The case where h and k are included in g0 is settled exactly with the same way as in Proposition 2.3.1. We tackle now the remaining cases. Suppose for a while that H ⊂ G0 and K ⊄ G0 , then we can assume that X ∈ k and then that dim k = 1 by our assumption (2.6) above. In fact, suppose that dim K > 1, then there exists Y ∈ k \ z(g) such that n X and Y are linearly independent. Let n0 be the largest integer such that adX0 (Y) ≠ 0 n +1 n and adX0 (Y) = 0. Then adX0 (Y) ∈ k ∩ z(g), which is absurd. Let now S be a compact set of G. Then there exist by Proposition 2.1.12 a compact set I of ℝ and a compact set S0 of G0 , such that S ⊂ exp(IX)S0 . we have then that K ∩ SHS−1 ⊂ K ∩ exp(IX)S0 HS0−1 exp(−IX) = exp(IX)(K ∩ S0 HS0−1 ) exp(−IX) = exp(IX) exp(−IX), which is compact in G. Hence, K ∩ SHS−1 is relatively compact. Finally, if H ⊄ G0 and K ⊄ G0 , then as in the previous case, we have that dim H = dim K = 1. Write h = span{X} and then k = span{X + U0 } for some nonzero U0 ∈ g0 . If the action of K on G/H is not proper, then choosing a flag of ideals S as in (2.2) passing through g0 and h, according to Proposition 2.2.3, we can suppose that there exists a compact subset S0 of G0 such that K ∩ S0 HS0−1 is not relatively compact. Thus, we can find sequences xn , yn ∈ ℝ, An , Bn ∈ g0 , such that: (2.3.2.1) exp(An ) ∈ S0 , exp(Bn ) ∈ S0 , (2.3.2.2) limn→+∞ |xn | = +∞, limn→+∞ |yn | = +∞, (2.3.2.3) exp(yn (X + U0 )) = exp(An ) exp(xn X) exp(−Bn ) = exp(xn X + An − Bn + C(xn , An , Bn ))

2.4 Proper actions on solvable homogeneous spaces | 65

for some C(xn , An , Bn ) ∈ d = [X, g] ⊂ g0 . It follows in view of the last equation that xn = yn for every n ∈ ℕ and that xn U0 = An − Bn mod(d).

(2.7)

If U0 ∈ d, then there exists T0 ∈ g0 \{0} such that U0 = [T0 , X]. Whence, Adexp(T0 ) X = X + U0 ∈ k, which contradicts the (CI) property of the triple (G, H, K). Thus, U0 ∈ ̸ d, which means that xn is a bounded sequence by (2.7). We get then an absurdity taking into account our assumption (2.3.2.3). This completes the proof of the theorem.

2.4 Proper actions on solvable homogeneous spaces 2.4.1 Proper actions on special solvmanifolds The following result has been obtained earlier in Theorem 2.3.2 in the setup of nilpotent Lie groups. We prove here that it still remains true for general connected and simply connected solvable Lie groups. Theorem 2.4.1. Let H, K be closed, connected subgroups of a connected and simply connected, special, solvable Lie group G. Then K acts properly on G/H if and only if the triple (G, H, K) has the (CI) property. In other words, the statement of Conjecture 2.1.9 holds. Proof. We write g = g0 ⊕ ℝX where g0 is the one-codimensional Abelian ideal of g and G0 = exp g0 . We can adopt the proof of the nilpotent situation to assume that none of the subalgebras h and k contain a nonzero ideal of g. Likewise, we can argue as in the proof of Theorem 2.3.2 to settle the situation when both h and k are included in g0 and also if merely one of them is. So we have only to tackle the case where h ⊄ g0 and k ⊄ g0 , then we can assume that dim h = dim k = 1 by our assumption above. Write h = span{X} and then k = span{X + U0 } for some nonzero U0 ∈ g0 . If the action of K on G/H is not proper, we can suppose that there exists a compact subset S0 of G0 such that H ∩ S0 KS0−1 is not relatively compact. In fact, for every compact set S of G, there exist by Proposition 2.1.12 a compact set S0 in g0 and a compact set I in ℝ such that S ⊂ exp(IX)S0 , and we have H ∩ SKS−1 ⊂ H ∩ exp(IX)S0 KS0−1 exp(−IX) = exp(IX)(H ∩ S0 KS0−1 ) exp(−IX). Thus, we can find sequences xn , yn ∈ ℝ, An , Bn ∈ g0 , such that:

66 | 2 Proper actions on homogeneous spaces (2.4.2.1) exp(An ) ∈ S0 , exp(Bn ) ∈ S0 , (2.4.2.2) limn→+∞ |xn | = +∞, limn→+∞ |yn | = +∞, (2.4.2.3) exp(xn X) = exp(An ) exp(yn (X + U0 )) exp(−Bn ). Denote d = [g, g] and D the Lie group associated to d. We show now that for all t ∈ ℝ, exp(t(X + U0 )) = exp(tX) exp(tU0 ) mod D.

(2.8)

In fact, thanks to the Campbell–Baker–Hausdorff formula in a neighborhood V0 of 0, there exists an analytic map ϱ from V0 to D such that exp(−tX) exp(t(X + U0 )) = exp(tU0 ) ⋅ ϱ(t).

(2.9)

We consider the analytic map ψ(t) = exp(−tU0 ) exp(−tX) exp(t(X + U0 )) and let {X1 , . . . , Xp } be a coexponential basis of d in g. There exist then p + 1 analytic functions t ∈ ℝ 󳨃→ αi (t) ∈ ℝ, i ∈ {1, . . . , p} and t ∈ ℝ 󳨃→ ρ(t) ∈ D such that ψ(t) = exp(α1 (t)X1 ) ⋅ ⋅ ⋅ exp(αp (t)Xp ) ⋅ ρ(t). By (2.9), the functions αi (i ∈ {1, . . . , p}) vanish on V0 , and then on ℝ by analyticity, which completes the proof of (2.8). It follows in view of equations (2.4.2.3) and (2.8) that xn = yn for every n ∈ ℕ and that exp(xn X) = exp(xn X) exp(−xn X) exp(An ) exp(xn X) exp(xn U0 ) exp(Cn ) exp(−Bn ) for some Cn ∈ d. Thus, Adexp(−xn X) (An ) − Bn + xn U0 + Cn = 0.

(2.10)

xn U0 = Bn − An mod d.

(2.11)

Hence,

If U0 ∈ d, then there exists T0 ∈ g0 \{0} such that U0 = [T0 , X]. Whence, Adexp(T0 ) X = X + U0 ∈ k, which contradicts the (CI) property of the triple (G, H, K). Thus, we do assume U0 ∈ ̸ d, which means that xn is a bounded sequence by (2.11), and we get then an absurdity taking into account our assumption (2.4.2.2). This completes the proof of the theorem.

2.4 Proper actions on solvable homogeneous spaces | 67

2.4.2 Weak and finite proper actions on solvmanifolds In this section, we provide a criterion of weak and finite proper actions as defined in Subsection 2.1.3, for a solvable triplet (G, H, K). Our main result in this section is the following. Theorem 2.4.2. Let G be a connected, simply connected, solvable Lie group, and H and K be closed connected subgroups of G. Then the following assertions are equivalent: (i) The action of K on G/H is finitely proper. (ii) The action of K on G/H is weakly proper. (iii) The action of K on G/H is free. (iv) The triple (G, H, K) has the (CI) property. Proof. First of all, we remark that as G is connected, simply connected and solvable, it admits no compact nontrivial subgroups. Indeed, compact subgroups are central (and hence Abelian). It turns out that the action of K on G/H is free if and only if the triple (G, H, K) is (CI) as for every x ∈ X the isotropy group Kx is given by Kx = K ∩ gHg −1 ,

x = gH.

Whence, we have only to prove by means of Lemma 2.1.5 that (CI)-action implies both properties (i) and (ii). Assume that the triple (G, H, K) has the (CI) property and let us show that K acts finitely properly on G/H. We have only to show that for every s, g in G, the set H ∩ gKs is finite, and by replacing K by gKg −1 , that H ∩ Ks is finite for every s ∈ G. Let then s ∈ G and h1 , h2 be in H ∩ Ks, then there exist k1 , k2 in K such that h1 = k1 s and h2 = k2 s. Hence, k2 k1−1 = h2 h−1 1 ∈ H ∩ K = {e}. Finally, h1 = h2 , which entails that H ∩ Ks consists at most of one single point, which completes the proof in this case. We prove now that if the triple (G, H, K) has the (CI) property, then K acts weakly properly on G/H. Likewise, we have only to show that K ∩ SH is compact in G for every compact S in G. We use an induction on the dimension of G. The result is obviously true when dim G = 1. Let g, h and k be the Lie algebras of G, H and K, respectively. Assume in a first time that one of the subalgebras h or k is included in a proper ideal of g, and then by extension, in a one-codimensional ideal g′ of g. This is actually what typically happens in the nilpotent context. Let G′ be the Lie group associated to g′ and write g = g′ ⊕ℝX for some X ∈ g. Let S be a compact set of G, then there exist a compact set I of ℝ and a compact set S′ of G′ , such that S ⊂ exp(IX)S′ . Suppose in a first time that k ⊄ g′ , we can then pick the vector X in a way such that X ∈ k. We get therefore that K ∩ SH ⊂ K ∩ exp(IX)S′ H = exp(IX)(K ′ ∩ S′ H),

(2.12)

68 | 2 Proper actions on homogeneous spaces where K ′ = K ∩ G′ . We apply the induction hypothesis for the (CI)-triple (G′ , K ′ , H) to obtain the fact that K ∩ SH is compact, which implies that K acts weakly properly on G/H. Suppose now that k ⊂ g′ . Then H ∩ SK ⊂ H ∩ exp(IX)S′ K ⊆ H ∩ S′ K,

(2.13)

which is compact using again the induction procedure for the (CI)-triple (G′ , H, K). This completes the proof in this case. We suppose from now on that none of the subalgebras h and k is included in a proper ideal in g. So, there exists a maximal nonnormal subalgebra g1 containing h. As g is solvable, g1 is one- or two-codimensional. We keep the same notation as in Theorem 1.1.15 and we tackle separately these two situations. Suppose in a first time that g1 is one- codimensional. Let g0 be the one-codimensional ideal of g1 and let A and X be the vectors of g such that g = g1 ⊕ ℝX,

g1 = g0 ⊕ ℝA

and [A, X] = X mod g0 .

If k ⊂ g1 , then we argue similarly as to use the induction hypothesis on the triple (G1 , H, K) making use of the coexponential basis {X} of g1 in g. Here, G1 is the Lie group associated to g1 . Otherwise, there exists in k a nonzero vector of the form T = X + aA + u0 for some real number a and u0 ∈ g0 . If a = 0, then {T} is a coexponential basis of g1 in g. Then every compact set S of G is included by means of Proposition 2.1.12, in some compact of G of the form exp(IT)S1 where I is a compact set of ℝ and S1 is compact set of G1 and we argue as in (2.12). Assume then that a ≠ 0. If k1 = k ∩ g1 ⊄ g0 , then there exists in k a vector of the form A + v0 for some v0 ∈ g0 . It follows that T ′ = [A + v0 , T] = X + w0 ∈ k for some w0 ∈ g0 , and then we are led to the previous case. Assume now that k1 ⊂ g0 . Let b = a1 and consider the subalgebra k′ = Adexp(bX) k. Then k′ = ℝ(A + u′0 ) ⊕ (k′ ∩ g0 ) for some u′0 ∈ g0 . As k′ is included in g1 , the action of the subgroup K ′ associated to k′ on G/H is weakly proper as the triple (G, H, K ′ ) is (CI), which already entails that the action of K on G/H is weakly proper. We tackle now the case where g1 is two-codimensional. If k ⊂ g1 , then if {X ′ , Y ′ } is the coexponential basis of g1 in g, every compact set can be included by means of

2.4 Proper actions on solvable homogeneous spaces | 69

Proposition 2.1.12, in a compact set of the form exp(IX ′ ) exp(JY ′ )S1 for some compact sets I, J in ℝ and S1 in G1 . Then obviously, H ∩ SK ⊂ H ∩ exp(IX ′ ) exp(JY ′ )S1 K ⊂ H ∩ S1 K, and we can use the induction hypothesis for the (CI)-triplet (G1 , H, K). So, we assume from now on that k is not included in g1 . Let g0 be the ideal defined as in Theorem 1.1.15. We study first the case where g0 is three-codimensional. Let A, X, Y be the vectors of g and α ∈ ℝ given as in Theorem 1.1.15 such that g1 = g0 ⊕ ℝA,

[A, X + iY] = (α + i)(X + iY) mod g0

and [X, Y] = 0 mod g0 . If dim k/k1 = 1 (here as above k1 = k ∩ g1 and K1 the Lie group associated to k1 ), then there exists in k a vector of the form T = xX + yY + aA + u0 ,

u0 ∈ g0 ,

x, y, a ∈ ℝ

such that x2 + y2 ≠ 0. If a = 0, then {T} is a part of a coexponential basis of g1 in g. Let T ′ be another vector such that {T, T ′ } forms a coexponential basis of g1 in g. Then every compact set is included in a compact set of the form exp(IT) exp(JT ′ )S1 for some compact sets I, J in ℝ and S1 in G1 . Then obviously, K ∩ SH ⊂ K ∩ exp(IT) exp(JT ′ )S1 H = exp(IT)(K ∩ exp(JT ′ )S1 H) ⊂ exp(IT)(K ∩ S1 H) = exp(IT)(K1 ∩ S1 H).

We can use the induction hypothesis for the (CI)-triplet (G1 , H, K1 ), so we are done in this case. Assume then that a ≠ 0. If k1 ⊄ g0 , then a vector of the form A + v0 belongs to k for some v0 ∈ g0 , and then k contains consequently a vector of the form T ′ = x′ X + y′ Y + w0 , 2

2

for some w0 ∈ g0 and x′ , y′ ∈ ℝ such that x′ + y′ ≠ 0, which takes us back to the αx−y αy+x previous case. We look now at the case where k1 ⊂ g0 . Take u = a(1+α 2 ) and v = a(1+α2 ) ,

and consider the algebra k′ = Adexp(uX+vY) k, which is included in g1 , as k′ = ℝ(A + z0 ) ⊕ (k′ ∩ g0 ) for some z0 ∈ g0 making use of (2.16). We are also done in this case.

70 | 2 Proper actions on homogeneous spaces We treat now the case where dim(k/k1 ) = 2, then there exists two vectors in k of the form T1 = x1 X + y1 Y + a1 A + u0 ,

T2 = x2 X + y2 Y + a2 A + v0 ,

where u0 , v0 ∈ g0 , x1 , x2 , y1 , y2 , a1 , a2 ∈ ℝ such that the vectors x1 X + y1 Y and x2 X + y2 Y are linearly independent. If k1 ⊄ g0 , then as above, a vector A + v0 belongs to k for some v0 ∈ g0 and, therefore, Ti′ = [A + v0 , Ti ] = (αxi + yi )X + (αyi − xi )Y + t0i ∈ k for some t0i ∈ g0 , i = 1, 2. It is not hard to check that the vectors T1′ and T2′ are linearly independent in k, and obviously constitute a coexponential basis of g1 in g. As above, we write a compact set S of G as included in a compact of the form exp(IT1′ ) exp(JT2′ )S1 for some compact sets I, J in ℝ and S1 in G1 . Then K ∩ SH ⊂ K ∩ exp(IT1′ ) exp(JT2′ )S1 H = exp(IT1′ ) exp(JT2′ )(K ∩ S1 H) = exp(IT1′ ) exp(JT2′ )(K1 ∩ S1 H).

So, we are done by the induction hypothesis in this case. Suppose now that k1 ⊂ g0 . If a1 = a2 = 0, we are done as above. Otherwise, suppose for instance that a2 ≠ 0 and consider the Lie subalgebra k′ = Adexp(uX+vY) k where as αy2 +x2 2 −y2 above u = aαx(1+α 2 ) and v = a (1+α2 ) . By (2.16), one has that 2

2

k′ = ℝ(T1 −

a1 T + u′0 ) ⊕ ℝ(A + w0′ ) ⊕ (k′ ∩ g0 ) a2 2

for some u′0 and w0′ in g0 . So, we are led back to the case where dim(k/k1 ) = 1. We study now the situation where g0 is four-codimensional. Let A, B, X, Y be as in Theorem 1.1.15 such that g = g1 ⊕ ℝX ⊕ ℝY,

[A, X + iY] = X + iY mod g0 , [X, Y] = 0 mod g0

and

g1 = g0 ⊕ ℝA ⊕ ℝB,

[B, X + iY] = −Y + iX mod g0 , [A, B] = 0 mod g0 .

We can and do assume that k ⊄ g1 . We shall discuss as above several cases. Suppose first that dim(k/k1 ) = 1. There exists then in k a vector of the form T = xX + yY + aA + bB + u0 ,

u0 ∈ g 0 ,

x, y, a, b ∈ ℝ

such that x2 + y2 ≠ 0. We can also assume that a2 + b2 ≠ 0; otherwise {T} is a part of a coexponential basis of g1 in g and we can use the argument made earlier. Assume

2.4 Proper actions on solvable homogeneous spaces | 71

and v = for a while that k1 ⊂ g0 . Take u = ax−by a2 +b2 k′ = Adexp(uX+vY) k. Then by (2.17), we deduce that

ay+bx , a2 +b2

and consider the Lie algebra

k′ = ℝ(aA + bB + w0 ) ⊕ (k′ ∩ g0 ) ⊂ g1 , for some w0 ∈ g0 . So, we can argue as above to have the answer in this case. We are led then to study the case where k1 ⊄ g0 , then as above, a vector a′ A + b′ B + v0 belongs to k such that a′ 2 + b′ 2 ≠ 0 and v0 ∈ g0 . Therefore, T ′ = [a′ A + b′ B + v0 , T] = (a′ x + b′ y)X + (a′ y − b′ x)Y + t0 ∈ k\{0}, for some t0 ∈ g0 , which completes the proof in this case. We look now at the case where dim(k/k1 ) = 2, then there exists in k two vectors of the form: T1 = x1 X + y1 Y + a1 A + b1 B + u0 ,

T2 = x2 X + y2 Y + a2 A + b2 B + v0 ,

where u0 , v0 ∈ g0 , x1 , x2 , y1 , y2 , a1 , a2 , b1 , b2 ∈ ℝ such that the vectors x1 X + y1 Y and x2 X + y2 Y are linearly independent. Suppose first that k1 ⊄ g0 . As k is a subalgebra, there exists a vector aA + bB + w0 ∈ k1 such that a2 + b2 ≠ 0. This gives rise to the fact that the pair {Ti′ = [aA + bB + w0 , Ti ], i = 1, 2} is a coexponential basis of g1 in g contained in k, which takes us back to a previous situation. Suppose finally that k1 ⊂ g0 , if a21 + b21 = a22 + b22 = 0. Then as above, {T1 , T2 } is a coexponential basis of g1 in g contained in k and the result holds as drawn up earlier. Thus, we can assume a1 y1 +b1 x1 −b1 y1 and k′ = Adexp(uX+vY) k. for instance that a21 + b21 ≠ 0. Let u = a1ax21 +b 2 , v = a2 +b2 1

1

1

1

So, we are led back to the case where dim(k′ /k′1 ) ≤ 1. This completes the proof of the theorem.

Corollary 2.4.3. Assume the same hypotheses as in Theorem 2.4.2. Then the action of K on X = G/H is weakly proper if and only if the action of K on X is (CI) and all K-orbits of X are closed. Proof. We have only to prove by means of Theorem 2.4.2 that the weak proper action of K on X implies that all K-orbits are closed in X. Let x, y ∈ X such that x ∈ ̸ Ky and let U be a compact neighborhood of x in X such that Ky ∩ U ≠ 0.

(2.14)

So, there exist y′ ∈ U and k ∈ K such that y′ = ky, and then Ky = Ky′ . We can therefore admit that y ∈ U. As Ky,U = {k ∈ K : ky ∈ U} is a compact set of X, the set Ky,U ⋅ y turns out to be compact in X and x ∈ ̸ Ky,U ⋅ y ⊂ U. We can therefore use similar arguments developed in [74, Théorème 3, Section 1] to achieve the proof.

72 | 2 Proper actions on homogeneous spaces 2.4.3 Proper actions on maximal solvmanifolds 2.4.4 Connected subgroups acting properly on maximal solvmanifolds We now prove our first upshot on proper actions on maximal solvable homogeneous spaces, which basically makes use of the above classification result of maximal subalgebras in a solvable Lie algebra. Theorem 2.4.4. Let H, K be closed connected subgroups of a connected simply connected solvable Lie group G. Assume that one of the subgroups H or K is maximal in G (as in Definition 1.1.16), then K acts properly on G/H if and only if the triple (G, H, K) has the (CI) property. Proof. Recall the notation g, h and k for the Lie algebras associated respectively to G, H and K. We assume, for example, that h is maximal in g. If h is an ideal in g, then the result is not difficult to obtain. So, we assume that none of the subalgebras h and k is an ideal of g. We denote by g0 the ideal defined as in Theorem 1.1.15 and G0 its corresponding Lie group. We consider the quotient group G = G/G0 and we designate by f : G → G, the canonical projection homomorphism. Likewise, we denote H = f (H) and K = f (K). So, it appears clear that K ∩ ker f = K ∩ G0 ⊂ K ∩ H = {e}, and then K ∩ ker f is compact. On the other hand, it is not difficult to show that (G, H, K) has the (CI) property. In fact, −1 let h ∈ H, k ∈ K and t ∈ G such that h = tht . Then h = tkt −1 t0 for some t0 ∈ G0 , which implies that ht0−1 = tkt −1 ∈ H ∩ tKt −1 . Finally, h = t0 as (G, H, K) has the (CI) property and, therefore, h = e. To prove that K acts properly on G/H, it is then sufficient to prove that K acts properly on G/H according to Lemma 2.1.7. Suppose in a first time that h is onecodimensional. We denote by g the Lie algebra associated to G. It is clear that g is the 2-dimensional Lie algebra spanned by the vectors fields A, X satisfying the bracket relation [A, X] = X and that h = ℝA. If k is an ideal of g, we are already done as in the proof of Theorem 2.3.2. If not, there exists x ∈ ℝ∗ such that k = span(A + xX), but this gives Adexp(−xX) A = A + xX,

(2.15)

which contradicts the (CI) property of the triple (G, H, K). This completes the proof in this case. Suppose now that h is two-codimensional and we assume in a first time that codim g0 = 3. Then h = ℝA. We note g1 = span{X, Y}. If k ⊄ g1 , then there exists a nonzero (x, y) ∈ ℝ2 such that A + xX + yY ∈ k. It comes up by a simple computation

2.4 Proper actions on solvable homogeneous spaces | 73

that Adexp( −αx+y X− x+αy Y) (A) = A + xX + yY 1+α2

1+α2

(2.16)

which is impossible by the (CI) property. So, we are led to study the case where k ⊂ g1 . Let S be a compact set in G. Then there exists by Proposition 2.1.12 a compact set S1 in G1 , the analytic Lie group associated to g1 and a compact set I in ℝ such that S ⊂ exp(IA)S1 . Then H ∩ SKS−1 ⊂ H ∩ exp(IA)S1 KS1−1 exp(−IA) = exp(IA)(H ∩ S1 KS1−1 ) exp(−IA) = exp(IA) exp(−IA) which is compact in G. Suppose now that g0 is four-codimensional. Then h = ℝA⊕ℝB. We note as above g1 = span{X, Y} and G1 the Lie group associated to g1 . If k ⊄ g1 , then there exists a vector aA+bB+xX +yY ∈ k such that a2 +b2 ≠ 0 and x 2 +y2 ≠ 0. A routine computation shows that Adexp( −ax+by X− bx+ay Y) (aA + bB) = aA + bB + xX + yY a2 +b2

a2 +b2

(2.17)

which is impossible by the (CI) property. Suppose finally that k ⊂ g1 . Let S be a compact set in G. Then there exists by Proposition 2.1.12 a compact set S1 in G1 , two compact sets I and J in ℝ such that S ⊂ exp(IA) exp(JB)S1 . Then H ∩ SKS−1 ⊂ H ∩ exp(IA) exp(JB)S1 KS1−1 exp(−JB) exp(−IA) = exp(IA) exp(JB)(H ∩ S1 KS1−1 ) exp(−JB) exp(−IA) = exp(IA) exp(JB) exp(−JB) exp(−IA), which is compact in G. This completes the proof of the theorem. The following corollary stems directly from Propositions 2.1.14 and 2.4.4. Corollary 2.4.5. Assume that G is a connected and simply connected, solvable Lie group and one of the subgroups in question is normal or maximal. Then the following statements are equivalent: (i) K acts weakly properly on G/H. (ii) K acts properly on G/H. (iii) The triple (G, H, K) is (CI).

74 | 2 Proper actions on homogeneous spaces 2.4.5 From continuous to discrete actions Throughout this subsection, we consider a connected nonnormal and maximal subgroup H of a connected solvable Lie group G as in Definition 1.1.16. Let h = Lie(H) and g0 = Lie(G0 ) the ideal contained in h (accordingly, G0 ⊂ H) as in Theorem 1.1.15. In case 3, the family {A, B, X, Y} constitutes a coexponential basis of G0 in G. Thus, any element g in G can be written as g = (a, b, v, g0 ) := eaA ebB exX+yY g0 , where a, b ∈ ℝ, v = (x, y) ∈ ℝ2 and g0 ∈ G0 . As such, Theorem 1.1.15 gives that the multiplication law group of G is submitted to the following equation: ′

(a, b, v, g0 )(a′ , b′ , v′ , g0′ ) = (a + a′ , b + b′ , e−a r(−b′ )v + v′ ) mod(G0 ),

(2.18)

where for t ∈ ℝ, cos(t) sin(t)

r(t) = (

sin(−t) ) cos(t)

and r(t)v = exp(r(t)(xX + yY)). Accordingly, the law group of G in case 2 of Theorem 1.1.15 can be written down as ′

(a, v, g0 )(a′ , v′ , g0′ ) = (a + a′ , e−αa r(−a′ )v + v′ ) mod(G0 ),

(2.19)

where the coexponential basis of G0 in G here is {A, X, Y}. Finally in case 1, the law group of G is described as follows: ′

(a, x, g0 )(a′ , x′ , g0′ ) = (a + a′ , e−a x + x′ ) mod(G0 ),

(2.20)

where {A, X} is the coexponential basis of G0 in G as in Theorem 1.1.15. Our first result is the following. Theorem 2.4.6. Let G be a connected, simply connected, solvable Lie group and H a nonnormal connected maximal subgroup of G. Then any nontrivial discrete subgroup of G acting properly and freely on G/H is Abelian of rank 1 in case 1 and of rank ≤ 2 otherwise. Proof. We keep the same notation as before and we consider the quotient group G = G/G0 and π : G 󳨀→ G, the canonical projection. Let H = π(H) and Γ = π(Γ). We first need to prove the following lemmas.

2.4 Proper actions on solvable homogeneous spaces | 75

Lemma 2.4.7. Let Γ be a discrete subgroup of G and H a nonnormal maximal subgroup of G. Then: (𝚤) If Γ acts properly on G/H, then Γ acts properly on G/H. (𝚤𝚤) If Γ acts freely on G/H, then Γ acts freely on G/H and the restriction π|Γ is injective. Proof. (𝚤) Suppose that the action of Γ on G/H is not proper, then there exist an infinite sequence (γn )n ∈ Γ and a convergent sequence (sn )n in G such that (γ n sn H)n converges in G. On the other hand, γ n sn H = γn G0 sn G0 HG0 = γn sn H and G/H = G/G0 /H/G0 ≃ G/H, then (γn sn )n converges in G/H. As (γn )n is not convergent, this is a contradiction with the properness of the action of Γ. (𝚤𝚤) First remark that ker(π|Γ ) = Γ ∩ G0 ⊂ Γ ∩ H = {e}. Let h ∈ H, γ ∈ Γ and t ∈ G such that h = tγt . Then there is a t0 ∈ G0 such that h = tγt −1 t0 and ht0−1 = tγt −1 ∈ −1

H ∩ tΓt −1 = {e}, which implies that h = t0 so h = e. Then we obtain H ∩ tΓt all t ∈ G.

−1

= {e} for

Lemma 2.4.8. If Γ acts properly on G/H, then Γ is discrete. Proof. Suppose that Γ is not discrete, then there exist a nonstationary sequence (γ n )n ∈ Γ such that (γ n )n converges to e, which means that (γn G0 )n converges to G0 . Let S be a compact neighborhood of e in G, then S is a compact neighborhood of e in G. We have for sufficiently large n, γ n ∈ S, then γ n ∈ γ n S ∩ S and so γ n ∈ γ n SH ∩ SH, which implies that γn ∈ γn SHG0 ∩ SHG0 = γn SH ∩ SH. This is a contradiction with the fact that Γ acts properly on G/H. Lemma 2.4.9. Assume the situation of case 1 and let Γ be a subgroup of G. Then the action of Γ on G/H is free if and only if any element of Γ is written as γ = (0, x, g) with x ≠ 0. In particular, Γ ⊂ {0} × ℝ × G0 . Proof. Let γ = (a, x, g) be an element of Γ. From equation (2.20), for g = (a′ , x ′ , g0′ ) ∈ G we have gγg −1 = (a, ea ((e−a − 1)x′ + x)) mod (G0 ). ′

The action is free if and only if gγg −1 ∉ H for any g ∈ G whenever γ is nontrivial, which is equivalent to a = 0 and x ≠ 0. Lemma 2.4.10. Under the assumptions of case 2 and let Γ be a subgroup of G. Then the action of Γ on G/H is free if and only if: (𝚤) Any element of Γ is written as γ = (0, v, g) with v ∈ ℝ2 \ {(0, 0)} and g ∈ G0 , in the case where α ≠ 0. In such case, we have Γ ⊂ {0} × ℝ2 × G0 .

76 | 2 Proper actions on homogeneous spaces (𝚤𝚤) Any element of Γ is written as γ = (2kπ, v, g) with k ∈ ℤ, v ∈ ℝ2 \ {(0, 0)} and g ∈ G0 in the case where α = 0. We have in this case Γ ⊂ 2πℤ × ℝ2 × G0 . Proof. Let γ = (a, v, g) be an element of Γ. From equation (2.19), for g = (a′ , v′ , g0′ ) ∈ G, we have gγg −1 = (a, eαa r(a′ )((e−αa r(−a) − Id)v′ + v)) mod(G0 ) ′

= (0, eαa r(a′ )((I − eαa r(a))v′ + eαa r(a)v)) mod(H). ′

The action of Γ on G/H is free if and only if gγg −1 ∉ H for any g ∈ G whenever γ is nontrivial. Thus, if α ≠ 0, the free action of Γ on G/H is equivalent to a = 0 and v ≠ 0. If α = 0, the free action is equivalent to a ∈ 2πℤ and v ≠ 0. Lemma 2.4.11. Let G and H be as in case 3. For any subgroup Γ of G, the action of Γ on G/H is free if and only any element of Γ is written as (0, 2kπ, v, g) such that k ∈ ℤ, v ∈ ℝ2 \ {(0, 0)} and g ∈ G0 . In particular, Γ ⊂ {0} × 2πℤ × ℝ2 × G0 . Proof. Let γ = (a, b, v, g) be an element of Γ. For g = (a′ , b′ , v′ , g0′ ) ∈ G, we have gγg −1 = (a, b, ea r(b′ )((e−a r(−b) − Id)v′ + v)) mod(G0 ). ′

(2.21)

As before, the action of Γ on G/H is free if and only if gγg −1 ∉ H for any g ∈ G whenever γ is nontrivial, which equivalent to a = 0, b ∈ 2πℤ and v ≠ 0. We now go back to the proof of Theorem 2.4.6. By Lemma 2.4.7, the map π̃ = π|Γ : Γ 󳨀→ Γ is a group isomorphism. To prove therefore that Γ is Abelian, it is sufficient to show that Γ is. We take back the three cases enumerated in Theorem 1.1.15. Case 1: g = h ⊕ ℝX and h = g0 ⊕ ℝA, then g = Lie(G) = h ⊕ ℝX and h = ℝA. By Lemma 2.4.9, any element of Γ is written as (0, x). Thus, Γ is identified to a subgroup of ℝ. Furthermore, as Γ acts properly on G/H, Γ turns out to be discrete by Lemma 2.4.8. Then Γ is isomorphic to ℤ. Case 2: g = h ⊕ ℝX ⊕ ℝY and h = g0 ⊕ ℝA, then g = Lie(G) = h ⊕ ℝX ⊕ ℝY and h = ℝA. If α ≠ 0, then by Lemma 2.4.10, the free action of Γ on G/H implies that any element of Γ is written as (0, v). In particular, Γ is identified to a subgroup of ℝ2 . As the action of Γ is proper, it comes out that Γ is a discrete and in particular rk Γ ≤ 2. Now, if α = 0 then gγg −1 = (a′ , r(a)((r(−a′ ) − Id)v + v′ )).

2.4 Proper actions on solvable homogeneous spaces | 77

The action of Γ on G/H is free if and only if a′ ∈ 2πℤ and v′ ≠ 0. Thus, Γ is identified to a discrete subgroup of G1 = 2πℤ×ℝ2 ⊂ G thanks to Lemma 2.4.10. Now G1 can be viewed as a closed subgroup of the Abelian group ℝ3 . In particular, Γ is identified to a discrete subgroup of ℝ3 , so rk(Γ) ≤ 3. Assume that rk(Γ) = 3 and let γi = (ai , vi ), i = 1, 2, 3 be some generators of Γ. Then the vectors v1 , v2 and v3 are not linearly independent. Then we can choose a sequence (u1,n , u2,n , u3,n )n ⊂ ℤ3 such that lim u v n→∞ 1,n 1 u

u

+ u2,n v2 + u3,n v3 = 0.

u

It comes out that for γn = γ1 1,n γ2 2,n γ3 3,n we have γn = (0, u1,n v1 + u2,n v2 + u3,n v3 ) mod(H). In particular, the sequence of general term (γn H)n converges to e. Thus, the action of Γ on G/H is not proper. Case 3: g = Lie(G) = h ⊕ ℝX ⊕ ℝY and h = ℝA ⊕ ℝB. As before, from Lemma 2.4.11 the subgroup Γ is identified to a discrete subgroup of 2πℤ × ℝ2 , which is enough to conclude. We keep the same hypotheses and notation. Our next upshot consists in providing a simple criterion of the proper action of a rank one discontinuous group for a maximal solvable homogeneous space. This is actually a first step toward the study of the local rigidity problem. We prove the following. Theorem 2.4.12. Let G be a connected, simply connected, solvable Lie group, H a nonnormal maximal subgroup of G and Γ a rank one subgroup of G. Then the following properties are equivalent: (𝚤) The action of Γ on G/H is free. (𝚤𝚤) The action of Γ on G/H is CI. (𝚤𝚤𝚤) The action of Γ on G/H is proper. Proof. The group G is simply connected, then Γ is torsion-free and any compact subgroup of G is trivial. In this case, we have obviously (𝚤) ⇐⇒ (𝚤𝚤) and (𝚤𝚤𝚤) 󳨐⇒ (𝚤𝚤), then we have to prove (𝚤) 󳨐⇒ (𝚤𝚤𝚤). Suppose that the action of Γ on G/H is not proper, then there exist a sequence (γn )n ∈ Γ such that {γn , n ∈ ℕ} is not a compact set and a convergent sequence (sn )n in G such that (γn sn )n converges modulo H. Let γ be a generator of Γ and (λn ) a sequence of integers such that γn = γ λn . In case 1, for γ as in Lemma 2.4.9, we have γn = (0, λn x, 0) mod(G0 ). For sn = (tn , yn , 0) mod(G0 ), from equation (2.20) we get γn sn = (tn , e−tn λn x + yn , 0) mod(G0 ). Therefore, (γn sn )n converges modulo H if and only if x = 0, then the action is not free by Lemma 2.4.9.

78 | 2 Proper actions on homogeneous spaces We now look at case 2. For γ as in Lemma 2.4.10, we have λ

(0, λn v, 0) mod(G0 )

γ0n = {

(2kλn π, λn v, 0)

if α ≠ 0, if α = 0.

Thus, independently from α, for sn = (tn , wn , 0) mod(G0 ) we have γn sn = (0, λn v + eαtn r(tn )wn , 0) mod(H). As (tn )n is a convergent sequence, it comes out that (γn sn )n converges modulo H if and only if v = 0 and then the action is not free by Lemma 2.4.10. We finally tackle case 3. We have γn = (0, 2kλn π, λn v, 0) mod(G0 ) and for sn = (tn , fn , wn , 0) mod(G0 ), the sequence γn sn = (0, 0, λn v + etn r(fn )wn ) mod(H), converges modulo H if and only if v = 0. Proposition 2.4.13. Let Γ be a subgroup of G of rank ≤ 2 and φ a homomorphism from Γ to G. Then: (𝚤) If Γ is a rank one subgroup, then φ(Γ) is nontrivial if and only if φ is injective and φ(Γ) is discrete. (𝚤𝚤) If φ(Γ) is a rank two subgroup acting properly on G/H, then φ is injective and φ(Γ) is discrete. Proof. Note that any rank one subgroup of connected simply connected solvable Lie group is a torsion-free discrete subgroup. Thus, any nontrivial homomorphism is injective and its image is discrete. For the rank two case and keeping the same notation n n as before, if φ is noninjective, there exist two integers n1 , n2 , such that γ1 1 γ2 2 = e. If λ

μ

φ(Γ) is not discrete, then there exists an infinite sequence (γn = γ1 n γ2 n )n in φ(Γ), which converges to the unit element. In the first situation, we have n1 v1 + n2 v2 = 0 and in the second, one obtains that (λn v1 + μn v2 )n converges to zero. In both of the situations, we get det(v1 , v2 ) = 0.

2.5 Proper action for the compact extension K ⋉ ℝn Let K be a compact subgroup of GL(n, ℝ) and G := K ⋉ ℝn ⊂ I(n) the semidirect product group, where I(n) := On (ℝ) ⋉ ℝn denotes the semidirect product of the orthogonal group On (ℝ) as in Section 1.2. Let H be a closed subgroup of G and Γ a discontinuous group for the homogeneous space X = G/H. We establish a geometrical criterion of the proper action of Γ on X , which requires an accurate description of the structure

2.5 Proper action for the compact extension K ⋉ ℝn

| 79

of closed connected subgroups of Euclidean motion groups as developed in Subsection 1.2.4. We first prove the following, which concerns the case where K = On (ℝ). Proposition 2.5.1. Let G = I(n) be the Euclidean motion group, H is a closed subgroup of G and Γ an infinite discrete subgroup of G. Then Γ acts properly on G/H if and only if H is compact. Proof. We remark first that Γ acts properly on G/H if and only if gΓg −1 acts properly on G/g ′ Hg ′ −1 for any g, g ′ ∈ G. Suppose that H is not compact. We can then find a sequence {(hp , yp )}p∈ℕ of H such that limp→∞ ‖yp ‖ = ∞. As in Corollary 1.2.14, take γ = (A, x) ∈ Γ of infinite order as given in (1.24), then x ≠ 0 by Lemma 1.2.5 and up to conjugation, γ p = (Ap , px). For p ∈ ℕ, there exists α(p) ∈ ℕ such that α(p)‖x‖ ≤ ‖yp ‖ < (α(p) + 1)‖x‖. Let zp+ = λx ∈ S(0, ‖yp ‖) ∩ ℝx where α(p) ≤ λ < α(p) + 1. Consider tp = zp+ − α(p)x = (λ − α(p))x, and then ‖tp ‖ < ‖x‖. Furthermore, On (ℝ) acts transitively on S(0, ‖yp ‖); hence, there exists Op ∈ On (ℝ) such that Op yp = zp+ . Let B′ (0, ‖x‖) be the closed ball and K = On (ℝ)× B′ (0, ‖x‖). Then −1 α(p) (Op , −tp )(hp , yp )(h−1 , 0) = (Aα(p) , −tp + Op yp ) p Op A

= (Aα(p) , α(p)x) = γ α(p) ∈ Γ ∩ KHK −1 .

As the set {γ α(p) }p∈ℕ is infinite, we meet a contradiction and then H must be compact. The converse is trivial. Remark 2.5.2. In the setting where H, H ′ , L, L′ are closed subgroups of G, Fact 2.1.2 gives immediately that if L acts properly on G/H, then so does L on G/H ′ whenever H ∼ H ′ . Furthermore, if L acts properly on G/H, then so does L′ on G/H whenever L ∼ L′ . The following is then immediate. Proposition 2.5.3. Let Γ be a discrete subgroup of the Lie group NC (as in Subsection 1.2.4). Let also H be a closed subgroup of NC and R its solvable radical. Then the following are equivalent: (1) Γ acts properly on NC /H. (2) Γ acts properly on NC /R. (3) For any subgroup Γ′ of finite index in Γ, Γ′ acts properly on NC /R. Proof. Let H = S ⋅ R be a Levi decomposition of H, where S is semisimple, and hence compact. Clearly, R ⊂ SHS−1 = H and H = SRS−1 , then H ∼ R. This establishes the

80 | 2 Proper actions on homogeneous spaces equivalence between (1) and (2) since the fact that Γ ⋔ H in NC is equivalent to that Γ acts properly on NC /R, as described in Remark 2.5.2. Now Γ = Γ′ D where D is a finite set of Γ and Γ′ is a subgroup of Γ. As Γ = D−1 Γ′ D and Γ′ ⊂ D−1 ΓD = Γ, then Γ ∼ Γ′ . Hence, similar arguments allow to establish the equivalence between (2) and (3), which closes the proof.

2.5.1 Criterion for proper action Let H be a closed connected subgroup of G and R its solvable radical. We designate by B(0, r) the closed ball of radius r > 0 centered at the origin. For a subset X of ℝn and a subgroup L of GL(n, ℝ), set L ⋅ X = {A.x | A ∈ L, x ∈ X}. Note also LΓ = {I} × EΓ and LH = {I} × EH . We first prove the following results. Lemma 2.5.4. Let Γ be a discrete subgroup of G. There exists r > 0 such that, for any x ∈ EΓ , B(x, r) ∩ pr2 (Γ) ≠ 0. Proof. Keep the same notation as in Theorem 1.2.24. In virtue of Proposition 1.2.40, EΓg = EΓga . For some g := (I, t) ∈ G, one can take (A1 , y1 ), . . . , (Ak , yk ) ∈ Γga such that {y1 , . . . , yk } is a basis of EΓga and yi = PAi (yi ), for i = 1 . . . k, thanks to Proposi-

tion 1.2.18. For any x ∈ EΓg , there exist some reals a1 , . . . , ak such that x = ∑ki=1 ai yi . Put x = ∑ki=1 [ai ]yi , where [a] designates the floor of a real number a. This yields by (1.25): k

k

i=1

i=1

k

k

i=1

i=1

∏(Ai , yi )[ai ] = (∏ Ai i , ∑[ai ]yi ) = (∏ Ai i , x). [a ]

[a ]

This proves that x ∈ pr2 (Γg ) and satisfies k

‖x − x‖ ≤ r := ∑ ‖yi ‖. i=1

The following clear facts give some simple characterizations of the proper action that will be discussed later. Lemma 2.5.5. Let Γ and H be two closed subgroups of G. The following assertions are equivalent: (i) Γ acts properly on G/H. (ii) For any r > 0, [Cr ⋅ H ⋅ Cr ] ∩ Γ is compact, where Cr := K × B(0, r). (iii) For any compact sets C and C ′ of G, [C ⋅ H ⋅ C ′ ] ∩ Γ is compact. We now prove the following.

2.5 Proper action for the compact extension K ⋉ ℝn

| 81

Lemma 2.5.6. Let E and F be two linear subspaces of ℝn . Then E ∩ (K ⋅ F) = {0} if and only if E ∩ [K ⋅ F + B(0, r)] is compact for all r > 0. Proof. We denote by S(0, r) the sphere centered at the origin and of radius r > 0. It is clear that E ∩[K ⋅F +B(0, r)] is closed and that Sr := (K ⋅F)∩S(0, r) is a compact set of ℝn . For all x ∈ Sr , consider the continuous map ε(x) = d(x, E), where d(x, E) designates the distance from x to E. Set ε0 := min{ε(x) | x ∈ Sr }. There exists then x0 ∈ Sr such that ̃′ := r ′ a′ . ε0 = d(x0 , E) > 0, as E ∩(K ⋅F) = {0}. For a ∈ K ⋅F, set a′ ∈ ℝ+ a∩S(0, r), and a ε(a ) 2 ̃′ , E) = r and ‖a ̃′ ‖ ≤ r . Let then y = a + v ∈ E ∩ [K ⋅ F + B(0, r)] ≠ 0, with Note that d(a ε0

̃′ . If t > 1, then d(a, E) > r, a ∈ K ⋅ F and v ∈ B(0, r). There exists t ≥ 0 such that a = t ⋅ a and hence B(a, r) ∩ E = 0, which is absurd. Therefore, t ∈ [0, 1], r2 󵄩̃′ 󵄩󵄩 + r, ‖y‖ ≤ ‖a‖ + ‖v‖ ≤ 󵄩󵄩󵄩a 󵄩󵄩 + r ≤ cε0 ,r := ε0

and conclusively [K ⋅ F + B(0, r)] ∩ E ⊂ B(0, cε0 ,r ). Conversely, note that E ∩ (K ⋅ F) is compact as E ∩ (K ⋅ F) ⊂ E ∩ [K ⋅ F + B(0, r)]. If 0 ≠ x ∈ E ∩ (K ⋅ F), then ℝx ⊂ E ∩ (K ⋅ F), which is a contradiction. We now move to prove the main result of this section. Theorem 2.5.7. Let G = K ⋉ ℝn where K is a compact subgroup of GL(n, ℝ), Γ a discrete subgroup of G and H a closed connected subgroup of G. Then the following are equivalent: (i) Γ acts properly on G/H. (ii) [K ⋅ EH ] ∩ EΓ = {0}. Proof. We first prove that Γ acts properly on G/H if and only if [K ⋅ ER + B(0, r)] ∩ EΓ is compact, for any r > 0. The required result follows then from Lemma 2.5.6. Assume first that Γ acts properly on G/H and let r > 0 for which Wr := [K ⋅ ER + B(0, r)] ∩ EΓ is not compact. There exists a sequence {xp }p∈ℕ of Wr such that ‖xp ‖ > p, for sufficiently large p ∈ ℕ. Set xp = kp yp + up , where kp ∈ K, yp ∈ ER and up ∈ B(0, r). By Lemma 2.5.4, there exists r ′ > 0 such that for any p ∈ ℕ, there exists x p ∈ pr2 (Γ) verifying ‖xp − xp ‖ ≤ r ′ . This means that there exist vp ∈ B(0, r ′ ) such that xp = x p + vp . In addition, Corollary 1.2.37 and Lemma 1.2.38 show that there exists τ = (I, v) such that yp ∈ ER = pr2 (Rτ ) ⊂ pr2 (H τ ), for any p ∈ ℕ. Clearly, we get x p = kp yp + zp , with zp = up − vp satisfying ‖zp ‖ ≤ r + r ′ , for any p ∈ ℕ. Let Ap and Bp be two matrices of K such that (Ap , xp ) ∈ Γ and (Bp , yp ) ∈ H τ and set Cr+r′ = K × B(0, r + r ′ ). It is straightforward that −1 (kp , zp )(Bp , yp )(B−1 p kp Ap , 0) = (Ap , x p ),

which means that [Cr+r′ ⋅ H τ ⋅ Cr+r′ ] ∩ Γ is infinite, and thus the action of Γ on G/H τ is not proper by Lemma 2.5.5, which is enough to conclude.

82 | 2 Proper actions on homogeneous spaces Conversely, assume that [K ⋅ ER + B(0, r)] ∩ EΓ is compact for any r > 0. Let Γa be a normal Abelian subgroup of Γ of finite index. By means of Lemma 2.5.5 and Corollary 2.5.3, it is enough to show that [Cr ⋅ R ⋅ Cr ] ∩ Γa is finite, where Cr = K × B(0, r). From Lemma 1.2.40 and Corollary 1.2.37, there exist τ1 := (I, v1 ) and τ2 := (I, v2 ) τ such that pr2 (Γa1 ) ⊂ EΓa = EΓ and pr2 (Rτ2 ) = ER . In order to show that [Cr ⋅ R ⋅ Cr ] ∩ Γa τ is finite, it is enough to show that [Cr ⋅ Rτ2 ⋅ Cr ] ∩ Γa1 is finite. Let (A, x) ∈ Tr , there exist (M, t), (N, s) ∈ Cr and (B, y) ∈ R such that (A, x) = (M, t)(B, y)(N, s), τ

which gives x = My + u, with u := t + MBs. Clearly,we have x ∈ pr2 (Γa1 ) ⊂ EΓa = EΓ and y ∈ pr2 (Rτ2 ) = ER . This gives that x ∈ [K ⋅ ER + B(0, 2r)] ∩ EΓ , which is compact, and finally pr2 (Tr ) is compact, for any r > 0. It follows that Tr is compact as Tr ⊂ K × pr2 (Tr ), and hence finite, which shows that the action of Γ on G/R is proper, and hence the result by Proposition 2.5.3. The following is then immediate. Corollary 2.5.8. Let G = K ⋉ ℝn where K is a compact subgroup of GL(n, ℝ), Γ be a discrete subgroup and H a closed connected subgroup of G. Then the following are equivalent: (i) Γ acts properly on G/H. (ii) LΓ acts properly on G/LH . Proof. Assume first that Γ acts properly on G/H. For any r > 0, set Cr = K × B(0, r). We shall prove that LΓ ∩ [Cr ⋅ LH ⋅ Cr′ ] is compact, for any r, r ′ > 0. Let (I, x) ∈ LΓ ∩ [Cr ⋅ LH ⋅ Cr′ ]. There exist (A, u) ∈ Cr , (I, y) ∈ LH and (B, v) ∈ Cr′ , such that (I, x) = (A, u)(I, y)(B, v). An easy computation shows that B = A−1

and

x = Ay + u + Av.

This gives x ∈ [K ⋅ EH + B(0, 2r)] ∩ EΓ , which is compact by Theorem 2.5.7 and Lemma 2.5.6. This yields LΓ ∩ [Cr ⋅ LH ⋅ Cr′ ] is compact, and hence the action of LΓ on G/LH is proper. Conversely, assume that LΓ acts properly on G/LH . Then we get LΓ ∩ [K ⋅ LH ] is compact. This means that LΓ ∩ [K ⋅ LH ] = {0} and as a consequence EΓ ∩ [K ⋅ EH ] = {0}. By Theorem 2.5.7, we conclude that Γ acts properly on G/H. As a further consequence of Theorem 2.5.7, we can reestablish the following more general result than that of Proposition 2.5.1.

2.5 Proper action for the compact extension K ⋉ ℝn

| 83

Corollary 2.5.9. Let Γ be a discrete subgroup and H a closed connected subgroup of G, where K acts transitively on the sphere Sn−1 . Then Γ acts properly on G/H if and only if H is compact or Γ is finite. Proof. Assume that Γ acts properly on G/H. If EH = {0}, then H is compact by Corollary 1.2.39. Otherwise, let 0 ≠ x ∈ EH . For any 0 ≠ y ∈ ℝn , there exists A ∈ K such y x = A ‖x‖ , which gives that y ∈ [K ⋅ EH ], and hence [K ⋅ EH ] = ℝn . It follows that that ‖y‖ EΓ = {0} and that Γ is finite by Proposition 1.2.40. Example 2.5.10. Recall the unitary group Un (ℂ) = {u ∈ Mn (ℂ) | uu∗ = In }. One can identify Un with a compact real group through the homomorphism ϕn : Un A + iB

󳨅→

O2n (ℝ)

󳨃→

(

A B

−B ); A

here, A, B ∈ Mn (ℝ). Clearly, ϕn (Un ) is a compact subgroup, which acts transitively on the sphere S2n−1 of ℝ2n . Hence, for K = ϕn (Un ), the proper action of a discrete subgroup Γ on G/H is equivalent to the fact that Γ is finite or H is compact, thanks to Corollary 2.5.9. Remark 2.5.11. The problem of finding a criterion of proper action is studied by T. Kobabyashi and T. Yoshino (cf. [93]) in the context of Cartan motion groups. Let G be a linear reductive Lie group and K a maximal compact subgroup of G. Let θ be the corresponding involution and write G := K exp(p) a Cartan decomposition of G. The semidirect product Gθ := K ⋉ p is called the Cartan motion group associated to the pair (G, K). The multiplication law of Gθ is given by (k, X)(k ′ , X ′ ) := (kk ′ , X + Ad(k)X ′ ). Let a be maximal Abelian subspace of p. For a subset L of Gθ , define a(L) := KLK ∩ a. If L is a subgroup of Gθ set pL := L ∩ p and d(L) = dim pL . Finally, define an involutive automorphism θ : Gθ 󳨀→ Gθ ,

(k, X) 󳨃󳨀→ (k, −X).

Under the assumptions that L and H are θ-stable subgroups with at most finitely many connected components and that L acts properly on Gθ /H, it was shown that the Clifford–Klein form L\Gθ /H is compact if and only if d(L) + d(H) = d(Gθ ) (cf. [93, Lemma 5.3.5]). Further, a criterion of proper action has been established. It states that for any subsets L and H of Gθ , L acts properly on Gθ /H if and only if a(L) acts properly on a/a(H) (cf. [93, Lemma 5.3.6]).

84 | 2 Proper actions on homogeneous spaces

2.6 Proper actions for Heisenberg motion groups This section aims to generate a criterion of the proper action of a discontinuous group Γ ⊂ G = 𝕌n ⋉ ℍn , the Heisenberg motion group, acting on a homogeneous space G/H. We keep all the notation and definitions of Section 1.3. Remark first the following immediate result. Lemma 2.6.1. Let Γ be a discrete subgroup of G and H a closed subgroup of G. The following assertions are equivalent: (i) Γ acts properly on G/H. (ii) For any r > 0, [Cr ⋅ H ⋅ Cr ] ∩ Γ is finite, where Cr := 𝕌n × B(0, r) × [−r, r]. (iii) For any compact sets C and C ′ of G, [C ⋅ H ⋅ C ′ ] ∩ Γ is finite. We refer back to Proposition 1.3.17 for the classification of discrete subgroups of G. We first show the following. Proposition 2.6.2. Let H be a closed subgroup of G and Γ a discrete subgroup of G of type (A). Then Γ acts properly on G/H if and only if H ∩ (𝕌n × B(0, r) × ℝ) is compact for any r > 0. Proof. Assume that Γ is a discrete subgroup of G1 . For any γ ∈ [Cr ⋅ Γ ⋅ Cr ] ∩ H (r > 0), there exist λ := (Aλ , 0, tλ ) ∈ Γ and (S, u, x), (S′ , v, y) ∈ Cr such that γ = (S, u, s)λ(S′ , u′ , s′ ) = (SAλ S′ , u + SAλ u′ , s + s′ + tλ −

1 Im⟨u, SAλ u′ ⟩). 2

Hence, γ ∈ H ∩ (𝕌n × B(0, 2r) × ℝ). Therefore, [Cr ⋅ Γ ⋅ Cr ] ∩ H ⊂ H ∩ (𝕌n × B(0, 2r) × ℝ). Conversely, as Γ is infinite, it contains necessarily some γ = (A, 0, t) with t ≠ 0. Suppose that there exists r > 0 such that H ∩ (𝕌n × B(0, r) × ℝ) is not compact, we can find a sequence {(hp , wp , sp )}p∈ℕ of H ∩ (𝕌n × B(0, r) × ℝ) such that limp→∞ sp = ∞. For s s p ∈ ℕ, we have γ p = (Ap , 0, pt) and let α(p) be the floor of tp . As α(p) ≤ tp < (α(p) + 1). Then lp := sp − α(p)t satisfies |lp | ≤ |t|. Set the compact set K := 𝕌n × B(0, r) × [−|t|, |t|], and we have α(p) γ α(p) = (In , −wp , −lp )(hp , wp , sp )(h−1 , 0, 0) ∈ KHK −1 ∩ Γ. p A

As the set {γ α(p) }p∈ℕ is infinite, we meet a contradiction and this completes the proof.

We next show the following. Proposition 2.6.3. Let H be a closed subgroup of G and Γ a discrete subgroup of G of type (B).

2.6 Proper actions for Heisenberg motion groups | 85

(a) If pr(Γ) is discrete, then Γ acts properly on G/H if and only if H is conjugate to a subgroup of G1 . (b) Otherwise, Γ acts properly on G/H if and only if H is compact. Proof. We first prove that if a type (B) discrete subgroup acts properly on G/H, then H is conjugate to a subgroup of G1 . Indeed, assume that one can find some (A, z, 0) ∈ Γ such that Az = z ≠ 0. If there exists h = (B, w, s) ∈ H such that PB (w) ≠ 0, then by Lemma 1.3.1 there exists u ∈ ℂn such that τ−u hτu = (B, w′ , 0) ∈ H τu := τ−u Hτu . Here, ′ ‖ . Then Bw′ = w′ ≠ 0. For p ∈ ℕ, take α(p) to be the floor of ‖pw ‖z‖ 󵄩 󵄩 α(p)‖z‖ ≤ 󵄩󵄩󵄩pw′ 󵄩󵄩󵄩 < (α(p) + 1)‖z‖. Let zp = ρp z ∈ S(0, ‖pw′ ‖) ∩ ℝz, where α(p) ≤ ρp < α(p) + 1, and S(0, ‖pw′ ‖) designates the sphere of center 0 and radius ‖pw′ ‖. Then yp = zp − α(p)z = (ρp − α(p))z satisfies ‖yp ‖ < ‖z‖. Since 𝕌n acts transitively on S(0, ‖pw′ ‖), there exists Cp ∈ 𝕌n such that Cp (pw′ ) = zp . Let K = 𝕌n × B(0, ‖z‖) × {0}. Then we easily check that −1 α(p) (Cp , −yp , 0)(Bp , pw′ , 0)(B−1 , 0, 0) = (Aα(p) , α(p)z, 0) ∈ Γ ∩ KHK −1 . p Cp A

As the set {(Aα(p) , α(p)z, 0)}p∈ℕ is infinite. Then the proper action of Γ on G/H entails that any (B, w, s) ∈ H is such that PB (w) = 0. By Proposition 1.3.14, H is conjugate to a subgroup of G1 . Let us go back now to the proof of the proposition. For statement (a), assume that H is a subgroup of G1 . Let r > 0 and γ ∈ Γ∩[Cr ⋅H⋅Cr ], then there exist (S, u, s), (S′ , u′ , s′ ) ∈ Cr and (B, 0, t) ∈ H such that γ = (S, u, s)(B, 0, t)(S′ , u′ , s′ ) = (SBS′ , u + SBu′ , s + s′ + t −

1 Im⟨u, SBu′ ⟩). 2

Then pr(γ) ∈ 𝕌n × B(0, 2r). As pr(Γ) is discrete, then pr(Γ) ∩ 𝕌n × B(0, 2r) coincides with a finite set {(A1 , z1 ), . . . , (Ak , zk )}. Suppose in addition that Γ ∩ [Cr ⋅ H ⋅ Cr ] is infinite, then obviously there exists s0 ∈ {1, . . . , k} such as the set {(As0 , zs0 , t) ∈ Γ ∩ [Cr ⋅ H ⋅ Cr ]} is infinite. Hence, there exist (As0 , zs0 , t) and (As0 , zs0 , t ′ ) ∈ Γ such that t ≠ t ′ , which gives that (As0 , zs0 , t)(As0 , zs0 , t ′ )

−1

= (I, 0, t − t ′ ) ∈ Γ,

and this is absurd. For statement (b), assume that pr(Γ) is not discrete. There exists an infinite sequence {(Ap , zp )}p∈ℕ ⊂ (𝕌n × B(0, ρ)) ∩ pr(Γ) for some positive real ρ. For any p ∈ ℕ,

86 | 2 Proper actions on homogeneous spaces one can find some tp ∈ ℝ such as (Ap , zp , tp ) ∈ Γ. Since Γ is discrete, then necessarily the sequence {tp }p∈ℕ is not bounded. We can assume that |tp | tends to ∞. As H is conjugate to a subgroup of G1 , we may assume that there exists some (C, 0, t) ∈ H with t t ≠ 0, and let β(p) be the floor of tp . Remark that |tp − β(p)t| ≤ |t|, for R := max(ρ, |t|), Γ ∩ CR HCR contains the infinite set {(Aβ(p) , zβ(p) , tβ(p) )}, which is absurd. Hence, H is compact. The converse is trivial. Proposition 2.6.4. Let Γ be an infinite discrete subgroup of G of type (C). For a closed subgroup H of G, Γ acts properly on G/H if and only if H is compact. Proof. By Proposition 2.6.2, Γ acts properly on G/H, and thus H ∩ (𝕌n × B(0, r) × ℝ) is compact for any r > 0. Hence, H does not contain an element conjugate to some (B, 0, t) with t ≠ 0. Proposition 2.6.3 shows that H is conjugate to a subgroup of G1 . Conclusively, H is compact.

3 Deformation and moduli spaces Our attention in this chapter is focused on the explicit determination (to a certain extent) of the deformation and moduli spaces of discontinuous groups acting on homogeneous spaces. This issue is of major importance to understand the local geometric structures of these spaces as many examples reveal. Toward such a purpose, many people have been interested in studying the deformation spaces of some geometric structures of closed surfaces, such as projective structures, hyperbolic structures (the Teichmüller space), complex structures and so on. In some restrictive cases, these deformation spaces are explicitly computed and some of their topological features are studied. For broader details, the reader could consult [40, 57, 67, 73, 76, 77, 92, 103– 105] and some references therein. For instance, when G/H = SL(2, ℝ)/SO(2) is the Poincaré disk, the deformation space consists of the deformation of complex structures on a Riemann surface Mg of genus g ≥ 2 and when G/H = G′ × G′ / diag(G′ ) for G′ = SL(2, ℝ), the deformation space is nothing but the deformation of complex structures on a 3-dimensional manifold. We refer the reader to the above papers where many settings have been considered. Furthermore, the explicit determination of the deformation space helps to understand many of its geometrical features, such as the local rigidity, the stability and the Calabi–Markus phenomenon, which will be subject of study later on (cf. Chapters 5 and 6).

3.1 Deformation and moduli spaces of discontinuous actions 3.1.1 Parameter, deformation and moduli spaces The parameter space The problem of describing deformations is advocated by T. Kobayashi in [97] where he formalized the study of the deformation of Clifford–Klein forms from a theoretic point of view. (See Problem C in [95] for further perspectives and basic examples.) When it comes to the solvable context, the strategy basically consists in building up an accurate cross-section of the adjoint orbits of elements of the parameter space, which defines all of the possible deformations of the discrete subgroup in question, in such a way that the deformed subgroups preserve the property of being discontinuous groups for the homogeneous space. Let G be a Lie group and Γ be a finitely generated discrete group. We designate by Hom(Γ, G) of group homomorphisms from Γ to G endowed with the pointwise convergence topology. The same topology is obtained by taking generators γ1 , . . . , γk of Γ. Then using the injective map Hom(Γ, G) 󳨅→ G × ⋅ ⋅ ⋅ × G, https://doi.org/10.1515/9783110765304-003

φ 󳨃→ (φ(γ1 ), . . . , φ(γk ))

88 | 3 Deformation and moduli spaces to equip Hom(Γ, G) with the relative topology induced from the direct product G×⋅ ⋅ ⋅×G. Let H be a closed subgroup of G. If H is not compact, then the discrete subgroup Γ does not necessarily act properly discontinuously on G/H. We consider then the parameter space R (Γ, G, H) of Hom(Γ, G) defined by R(Γ, G, H) = {φ ∈ Hom(Γ, G) : φ is injective and φ(Γ) acts properly discontinuously and freely on G/H}.

(3.1)

This set plays an important role as we will see later. According to this definition, for each φ ∈ R (Γ, G, H), the space φ(Γ)\G/H is a Clifford–Klein form, which is a Hausdorff topological space and even equipped with a structure of a manifold for which the quotient canonical map is an open covering. The deformation and moduli space Let now φ ∈ R (Γ, G, H) and g ∈ G. We consider the element φg := g −1 ⋅φ⋅g of Hom(Γ, G) defined by φg (γ) = g −1 φ(γ)g,

γ ∈ Γ.

It is then clear that the element φg ∈ R (Γ, G, H) and that the map φ(Γ)\G/H 󳨀→ φg (Γ)\G/H,

φ(Γ)xH 󳨃→ φg (Γ)g −1 xH

is a natural diffeomorphism. We consider then the orbits space T (Γ, G, H) = R (Γ, G, H)/G

instead of R (Γ, G, H) in order to avoid the unessential part of deformations arising inner automorphisms and to be quite precise on parameters. We call the set T (Γ, G, H) as the space of the deformation of the action of Γ on the homogeneous space G/H. On the other hand, let the group Aut(Γ) act on Hom(Γ, G) by T ⋅ φ(γ) := φ(T −1 (γ)),

φ ∈ Hom(Γ, G),

T ∈ Aut(Γ),

γ ∈ Γ.

It is then easy to check that the group Aut(Γ) leaves the parameter space R (Γ, G, H) invariant and its action on it is G-equivariant. We define then (to avoid this unessential part, too) the moduli space as the double coset space M (Γ, G, H) := Aut(Γ)\R (Γ, G, H)/G.

With the above in mind, the following facts are immediate. Lemma 3.1.1. Let G be a locally compact group and let H and Γ be subgroups of G. Then the following assertions are equivalent:

3.1 Deformation and moduli spaces of discontinuous actions | 89

(𝚤) Γ is a discontinuous group for G/H. (𝚤𝚤) Γ is a discontinuous group for G/gHg −1 , for any g ∈ G. In particular, R (Γ, G, H) ⋍ R (Γ, G, gHg −1 ), for all g ∈ G. Lemma 3.1.2. Let Γ be a discontinuous group for G/H, assume that H is contained cocompactly in K and Γ is torsion-free. Then R (Γ, G, H) = R (Γ, G, K). Proof. As Γ is torsion-free, the discontinuous action of Γ on G/H is equivalent to its proper action. Then by definition of the parameter space, the result comes from Fact 2.1.7. Remark 3.1.3. Let Γ be generated by {γ1 , . . . , γk }. Let F(k) be the non-Abelian free group with a k generators x1 , . . . , xk and φ a group homomorphism from F(k) to G, then φ is completely determined by the image of the generators. As F(k) is a group without any nontrivial relation, the map Hom(F(k), G) → Gk ,

φ 󳨃→ (φ(x1 ), . . . , φ(xk )),

is a bijection and when Hom(F(k), G) is endowed with the pointwise convergence topology and Gk with the product topology. this map becomes a homeomorphism. Now the natural map πk : F(k) → Γ,

n

n

n

n

1

r

1

r

xi 1 ⋅ ⋅ ⋅ xi r 󳨃→ γi 1 ⋅ ⋅ ⋅ γi r ,

is a group homomorphism. This allows to conclude that Γ ≅ F(k)/ ker πk and Hom(Γ, G) = {φ ∈ Hom(F(k), G), ker φ ⊃ ker πk }. If we consider the identification of Hom(F(k), G) with Gk , then we can write Hom(Γ, G) = {(g1 , . . . , gk ) ∈ Gk , gi1 ⋅ ⋅ ⋅ gir = e for all xi1 ⋅ ⋅ ⋅ xir ∈ ker πk }, which is clearly a closed subset. As G is a real analytic manifold and the multiplication map is also analytic, it comes out that Hom(Γ, G) is an analytic subset of Gk whenever Γ is finitely presented, i. e., the kernel of πk is finitely generated; see [98, Section 5.2].

3.1.2 Case of effective actions We here refer to Subsection 2.1.2. We first introduce the Clifford–Klein space CK(Γ, G, H) = {Γ′ \G/H, Γ′ is isomorphic to Γ and Γ′ is discontinuous for G/H}.

90 | 3 Deformation and moduli spaces By a deformation of the Clifford–Klein form Γ\G/H, we mean any element of the related Clifford–Klein space CK(Γ, G, H). The next section is devoted to describe in the general context how to interpret these deformations and the space CK(Γ, G, H). We now introduce the kernel of effectiveness related to the action of G on G/H, which equals the normal subgroup N = ⋂g∈G gHg −1 . For any φ ∈ R (Γ, G, H), φ(Γ) ∩ N = {e}, which means that φ(Γ) acts effectively on G/H. There is a natural surjective map Ψ : R (Γ, G, H) → CK(Γ, G, H), φ 󳨃→ φ(Γ)\G/H. Now the group G itself acts on R (Γ, G, H) and CK(Γ, G, H) and the map Ψ is G-equivariant. In sum, Ψ(T (Γ, G, H)) = CK(Γ, G, H)/G and the deformation space determines therefore all possible deformations of the related Clifford–Klein form modulo the G-action. Let Γi , i = 1, 2 be discontinuous groups for G/H, then Γ1 \G/H = Γ2 \G/H if and only if Γ1 gHg −1 = Γ2 gHg −1 for all g ∈ G. We define an equivalence relation ⋏ on R (Γ, G, H) as follows: φ ⋏ φ′ if and only if φ(Γ)gHg −1 = φ′ (Γ)gHg −1 for any g ∈ G. We then introduce the space ̂(Γ, G, H) as being the quotient subsequent space. Clearly, the map Ψ factors to a biR ̂(Γ, G, H) to CK(Γ, G, H) and to a topological homeomorphism when these jection from R spaces are endowed with adequate topologies. Now the action of G on R (Γ, G, H) in̂(Γ, G, H), which commutes with the relation ⋏. The duces an associated action on R ̂(Γ, G, H)/G is called the refined deformation space and is identispace T̂(Γ, G, H) := R fied to the space CK(Γ, G, H)/G.

3.1.3 Deformation of (G, X )-structures Our interest to these spaces comes from the deformation theory of the (G, X)structures, where G is a Lie group and X is a homogeneous space. Let M be a smooth manifold such that dim X = dim M. A (G, X)-atlas on M is a collection (Uα , ϕα )α∈I , where {Uα , α ∈ I} is an open covering of M and {ϕα : Uα → X, α ∈ I} is a family of local coordinates charts such that, on a connected component C of Uα ∩ Uβ , there exists gC,α,β ∈ G satisfying gC,α,β ∘ ϕα = ϕβ . A (G, X)-structure on M is a maximal (G, X)-atlas on M and a (G, X)-manifold is a manifold endowed with a (G, X)-structure. Let Σ be a smooth manifold, a marked (G, X)structure on Σ is a pair (M, f ) where M is a (G, X)-manifold and f : Σ → M is a diffeomorphism. Let D(G,X) (Σ) be the space of the all marked (G, X)-structures on Σ. The group Diff0 (Σ) (the subgroup of the group of diffeomorphisms of Σ isotopic to the identity) acts on D(G,X) (Σ) through the law: ψ ⋆ (M, f ) = (M, f ∘ ψ−1 ),

ψ ∈ Diff0 (Σ).

3.2 Algebraic characterization of the deformation space

| 91

The deformation space of the (G, X)-structures on Σ is the quotient space Def(G,X) (Σ) = D(G,X) (Σ)/ Diff0 (Σ). Assume Σ is compact. By the deformation Theorem of Thurston, the holonomy map is a local homeomorphism between the deformation space of marked (G, X)-structures on Σ and the quotient space Hom(π1 (Σ), G)/G, (cf. [78]). If Γ is a discontinuous subgroup for X, the Clifford–Klein form Γ\X is a (G, X)-manifold. If there is a diffeomorphism f : Σ → Γ\X, then the marked (G, X)-structure (Γ\X, f ) is said to be complete. The set Dc(G,X) (Σ) of the complete (G, X)-structures on Σ, is invariant under the action of Diff0 (Σ). The deformation space of complete (G, X)-structures on Σ is defined as Defc(G,X) (Σ) = Dc(G,X) (Σ)/ Diff0 (Σ). Then the deformation space T (Γ, G, H) of the discontinuous actions of Γ on X = G/H contains the image of Defc(G,X) (Σ) by the holonomy map. Furthermore, if all the forms φ(Γ)\G/H are diffeomorphic for φ ∈ R (Γ, G, H), then the deformation space T (Γ, G, H) coincides with the image of Defc(G,X) (Σ). Remark 3.1.4. We close this section with the following important remark. Assume that the deformation space T (Γ, G, H) of the discontinuous actions of Γ on X = G/H coincides with the image of Defc(G,X) (Σ) by the holonomy map hol and that the stability holds. Then the restriction hol : Defc(G,X) (Σ) → T (Γ, G, H) is a local homeomorphism. Indeed, T (Γ, G, H) is an open set of Hom(Γ, G)/G, therefore, Defc(G,X) (Σ) is an open set of Def(G,X) (Σ). As hol is a local homeomorphism, we are done.

3.2 Algebraic characterization of the deformation space 3.2.1 The deformation and moduli spaces in the exponential setting Let g denote a n-dimensional real exponential solvable Lie algebra, G will be the associated connected and simply connected exponential Lie group. The Lie algebra g acts on g by the adjoint representation ad, and the group G acts on g by the adjoint representation Ad, defined by Adg = exp ∘adT ,

g = exp T ∈ G.

(3.2)

Let H = exp h be a closed, connected subgroup of G. Let Γ be an Abelian discrete subgroup of G of rank k and define the parameter space R (Γ, G, H) as given in (3.1). The

92 | 3 Deformation and moduli spaces aim now is to generate a characterization of the parameter and the deformation spaces is derived as follows. Let L be the syndetic hull of Γ, which is the smallest (and hence the unique) connected Lie subgroup of G, which contains Γ cocompactly (cf. Theorem 1.4.2). Recall that the Lie subalgebra l of L is the real span of the Abelian lattice log Γ, which is generated by {log γ1 , . . . , log γk } where {γ1 , . . . , γk } is a set of generators of Γ. The group G also acts on Hom(l, g) by ψ ⋅ g = Adg −1 ∘ψ.

(3.3)

Our first observation is that the parameter space defined above, only depends on the structure of the syndetic hull of Γ when the basis group G is completely solvable. Recall that any continuous homomorphism of a connected Lie groups is smooth and its derivative is a homomorphism of Lie algebras. We consider the smooth map d : Homc (L, G) 󳨀→ Hom(l, g), φ 󳨃→ dφ|e where l is the Lie algebras of L. In the case of exponential Lie groups, dφ|e (X) = log ∘φ ∘ exp(X) for any X ∈ g. The group G acts on the spaces Hom(Γ, G), Hom(L, G) and Hom(l, g), respectively, through the following laws: (g ⋅ φ)(γ) = gφ(γ)g −1 , g ⋅ ψ = Adg ∘ψ,

γ ∈ Γ (or L),

ψ ∈ Hom(l, g),

φ ∈ Hom(Γ, G) (or Hom(L, G)), g ∈ G.

g ∈ G,

As L is exponential, the differential map d from Aut(L) to Aut(l) is a topological isomorphism. Using the natural injection i of Aut(Γ) in Aut(L), we see that Aut(Γ) acts on Hom(L, G) and Hom(l, g) by a ⋅ φ = φ ∘ i(a)−1

and

−1

a ⋅ ψ = ψ ∘ d(i(a))| , e

(3.4)

for φ ∈ Hom(L, G), ψ ∈ Hom(l, g) and a ∈ Aut(Γ). Let R (l, g, h) = {ψ ∈ Hom(l, g) : dim ψ(l) = dim l, exp(ψ(l)) acts properly on G/H}. (3.5)

The following result provides a preliminary algebraic interpretation of the parameter, deformation and moduli spaces and of capital importance in the sequel. Theorem 3.2.1. Let G be a completely solvable Lie group, H a connected subgroup of G and Γ a discrete subgroup of G acting properly discontinuously on G/H. Then up to a homeomorphism, R (Γ, G, H) = R (l, g, h).

In particular, if Γ and Γ′ have the same syndetic hull, then R (Γ, G, H) and R (Γ′ , G, H) are homeomorphic. Furthermore, up to homeomorphism, the deformation space T (Γ, G, H) coincides with R (l, g, h)/G and the moduli space is identified to Aut(Γ)\R (l, g, h)/G.

3.2 Algebraic characterization of the deformation space

| 93

Proof. The composition map d ∘ ξ is a homeomorphism from {φ ∈ Hom(Γ, G), φ(Γ) isomorphic to Γ} to its image, where ξ is the inverse of the restriction map as in Proposition 1.4.7. We now show that this image is precisely the set {φ ∈ Hom(l, g), dim φ(l) = dim l}. Indeed, let φ ∈ Hom(Γ, G) such that Γ is isomorphic to Γ′ = φ(Γ). The homomorphism ξ (φ) is continuous and L is connected, then ξ (φ)(L) is a connected subgroup containing φ(Γ). This means that ξ (φ)(L) contains the syndetic hull of Γ′ denoted by L′ . In particular, dim L ≥ dim ξ (φ)(L) ≥ dim L′ . Let φ−1 be the inverse of φ : Γ → Γ′ ⊂ G. The composition of φ−1 and the natural injection of Γ in G is a homomorphism from Γ′ to G and Γ′ is isomorphic to Γ = φ−1 (Γ′ ). Then for ξ ′ the extension map from Hom(Γ′ , G) to Hom(L′ , G), we have ξ ′ (φ−1 )(L′ ) ⊃ L. Then dim L′ ≥ dim ξ ′ (φ−1 )(L′ ) ≥ dim L. This entails that dim L = dim ξ (φ)(L), which is equivalent to dim l = dim d ∘ ξ (φ)(l). Conversely, if ψ is a Lie algebras homomorphism, such that dim l = dim ψ(l), then exp ∘ψ ∘ log ∈ Hom(L, G), is an isomorphism from L to its image. Its restriction to Γ denoted by φ say, is an isomorphism from Γ to its image and satisfies d ∘ ξ (φ) = ψ. Now, from Fact 2.1.8, the proper action of φ(Γ) is equivalent to the proper action of its syndetic hull L′ = ξ (φ)(L) = exp ∘d ∘ ξ (φ)(l). We conclude therefore that d ∘ ξ (R (Γ, G, H)) = R (l, g, h). To achieve the proof of the theorem, we need the following. Lemma 3.2.2. With the same hypotheses as in Theorem 3.2.1. The maps d and ξ are G and Aut(Γ)-equivariant. Proof. For g ∈ G and φ ∈ Hom(Γ, G), we have g ⋅ξ (φ) = ξ (g ⋅φ) if and only if (g ⋅ξ (φ))|Γ = g ⋅ φ. Furthermore, for any γ ∈ Γ, we have (g ⋅ ξ (φ))|Γ (γ) = gξ (φ)(γ)g −1 = (g ⋅ φ)(γ). Let τg : G → G, t 󳨃→ gtg −1 be the conjugation. For any φ ∈ Hom(L, G) and any g ∈ G, we have d(g ⋅ φ)|e = d(τg ∘ φ)|e = d(τg )|e ∘ d(φ)|e = Adg ∘d(φ)|e = g ⋅ d(φ)|e ., where Adg is the adjoint representation defined as in formula (3.2). Likewise, for a ∈ Aut(Γ) and φ ∈ Hom(Γ, G), we have a ⋅ ξ (φ) = ξ (a ⋅ φ) if and only if (a ⋅ ξ (φ))|Γ = a ⋅ φ, and as before we have (a ⋅ ξ (φ))|Γ (γ) = ξ (φ) ∘ i(a)−1 (γ) = φ ∘ a−1 (γ) = (a ⋅ φ)(γ). For φ ∈ Hom(L, G), we have d(a ⋅ φ)|e = d(φ ∘ i(a)−1 )|e = d(φ)|e ∘ d(i(a)−1 )|e = a ⋅ d(φ)|e . Our theorem is thus proved. Using Lemma 3.2.2 and Theorem 3.2.1, the following is immediate.

94 | 3 Deformation and moduli spaces Corollary 3.2.3. Retain the same hypotheses as in Theorem 3.2.1. If Γ is uniform in G, then H is trivial and we have R (Γ, G, H) = Aut(g), T (Γ, G, H) = Aut(g)/ Ad(G)

and M (Γ, G, H) = Aut(Γ)\ Aut(g)/ Ad(G).

In the exponential setting, we can even get a similar result as in the preceding theorem under the condition that Γ is Abelian. Theorem 3.2.4. Let G = exp g be an exponential solvable Lie group, H = exp h a closed connected subgroup of G, Γ ≃ ℤk a discrete subgroup of G and L = exp l its syndetic hull. Then the parameter space R (Γ, G, H) is homeomorphic to R (l, g, h). Likewise, the deformation space can equivalently be described as T (Γ, G, H) = R (Γ, G, H)/ Ad, where the action Ad of G is given as in (3.3). Proof. The proof will be divided into some separate results. We start by proving the following. Proposition 3.2.5. Let G be an exponential solvable Lie group, Γ ≅ ℤk a discrete subgroup of G and L its syndetic hull and l = log L. Then the map ξ : Hom(l, g) 󳨀→ Hom(Γ, G)

ψ 󳨃󳨀→ exp ∘ψ ∘ log

is a G and Aut(Γ)-equivariant homeomorphism. Proof. The group G acts on gk by g ⋅ (X1 , . . . , Xk ) = (Adg (X1 ), . . . , Adg (Xk )) and on Gk by g ⋅ (x1 , . . . , xk ) = (gx1 g −1 , . . . , gxk g −1 ). The map exp gk → Gk ,

(X1 , . . . , Xk ) 󳨃→ (exp X1 , . . . , exp Xk )

is then a G-equivariant diffeomorphism. For a system of generators {γ1 , . . . , γk } of Γ, we define a natural G-equivariant injections as follows: Hom(l, g) → gk , k

Hom(Γ, G) → G ,

ψ 󳨃→ (ψ(log γ1 ), . . . ψ(log γk )), φ 󳨃→ (φ(γ1 , . . . , φ(γk )).

Thanks to these inclusions, we regard Hom(l, g) as a subset of gk and Hom(Γ, G) as a subset of Gk . We have then exp(Hom(l, g)) = Hom(Γ, G) and ξ = exp| Hom(l,g) , so ξ is a

3.2 Algebraic characterization of the deformation space

| 95

G-equivariant homeomorphism. To complete the proof, we only need to prove that ξ is Aut(Γ)-equivariant. Indeed, Let T ∈ Aut(Γ) and ψ ∈ Hom(l, g) then for any γ in L we have ξ (T ⋅ ψ)(γ) = exp ∘(T ⋅ ψ) ∘ log γ = exp ∘ψ ∘ log T −1 (γ) = ξ (ψ)(T −1 (γ))

= T ⋅ ξ (ψ)(γ). We now prove the following.

Lemma 3.2.6. Assume that G is an exponential solvable Lie group, Γ ≃ ℤk a discrete subgroup of G and l = log L, where L is the syndetic hull of Γ. Then for ψ ∈ Hom(l, g), the following two conditions are equivalent: (𝚤) ψ is injective. (𝚤𝚤) ξ (ψ) is injective and ξ (ψ)(Γ) is discrete. Proof. Let γ, γ ′ ∈ Γ such that ξ (ψ)(γ) = ξ (ψ)(γ ′ ), then clearly ψ(log γ) = ψ(log γ ′ ) suppose that ψ is injective then log γ = log γ ′ , which is equivalent to γ = γ ′ . Now the exponential map is an isomorphism from log Γ to Γ and ψ is an isomorphism between l and ψ(l). If we restrict ψ to log Γ, we obtain an isomorphism from log Γ to ψ(log Γ). But ψ(log Γ) = log ξ (ψ)(Γ), which is isomorphic to its exponential, then Γ is isomorphic to ξ (ψ)(Γ), which is enough to show the first part of the proposition. Conversely, if (𝚤𝚤) is satisfied then ξ (ψ) is a topological isomorphism between Γ and ξ (ψ)(Γ) and then log ∘ξ (ψ) ∘ exp is an isomorphism between the Abelian, discrete subgroups log Γ and ψ(log Γ). Then the linear subspaces l and ψ(l) are isomorphic, which means in particular that dim ψ(l) = dim l as was to be shown. Back now to the proof of Theorem 3.2.4. Using Proposition 3.2.5, we can identify

R (Γ, G, H) to its inverse image by ξ . We get then

󵄨󵄨 󵄨 ξ (ψ) injective and ξ (ψ)(Γ) acts }. 󵄨󵄨 properly discontinuously and freely on G/H 󵄨

󵄨 R (Γ, G, H) = {ψ ∈ Hom(l, g) 󵄨󵄨󵄨

Now the group Γ is torsion free, if therefore the action of Γ is proper, then it is free. Then the first equality is a direct consequence of Lemma 3.2.6 and Fact 2.1.8. We now consider the map ξ ̃ : Hom(l, g)/G [ψ]

󳨀→

Hom(Γ, G)/G

󳨃→

[ξ (ψ)],

which is clearly a homeomorphism. Then the result follows through the identification of T (Γ, G, H) with its inverse image ξ ̃ −1 . This achieves the proof of the theorem.

96 | 3 Deformation and moduli spaces 3.2.2 On pairs (G, H) having Lipsman’s property Any information concerning the spaces Hom(Γ, G) and R (Γ, G, H) may help to understand the properties and the structure of the deformation space T (Γ, G, H). The sets Hom(Γ, G) and R (Γ, G, H) may have some singularities and there is no clear reason, to say that the parameter space R (Γ, G, H) is an analytic or algebraic or smooth manifold. For instance, when the parameter space is a semialgebraic set (in the sense of Definition 3.2.7 below), it has certainly a finite number of connected components, which means in turn, that the deformation space itself enjoys this feature. To figure out such an issue, we treat here one situation when the basis group in question is nipotent. Definition 3.2.7 (cf. [41]). (1) A subset V of ℝn is called semialgebraic if it admits some representation of the form s

ri

V = ⋃ ⋂{x ∈ ℝn : Pi,j (x) i=1 j=1

sij

0},

where for each i = 1, . . . , s and j = 1, . . . , ri , Pi,j are some polynomials on ℝn and sij ∈ {>, =, n, and when k + s = n, it only consists of one single point. This will correspond later on, to the case of compact Clifford–Klein forms. Let now X be a subspace of ℝn of dimension s, write it for instance as x1 } { } { } { .. } { } { } { . } { } { } ( ) { } { } {(xs ) ) ( X = {( ) ; x1 , . . . , xs ∈ ℝ} . } { 0 ( ) } { } { } { } { . } { . } { } { . } { } { } {( 0 ) Lemma 3.3.5. Let W ∈ Gn,k (ℝ), W = M(ℝk ) say. Then W ∩ X = {0} if and only if η(M) ∈

⋃

α∈Is (n,k)

ηα (Uα ).

(3.10)

3.3 Case of Abelian discontinuous groups | 101

Proof. Write the matrix M as A M=( ) B for some A ∈ Ms,k (ℝ) and B ∈ Mn−s,k (ℝ). Then it is quite clear that W ∩ X = {0} if and only if rank(B) = k, which is also equivalent to the fact that there exists α ∈ Is (n, k) such that det(Mα ) ≠ 0, and then the existence of α ∈ Is (n, k) with the property that η(M) ∈ ηα (Uα ). We now consider some bilinear forms b1 , . . . , bl : ℝn × ℝn → ℝ. Having fixed a canonical basis for ℝn , we designate by Jb1 , . . . Jbl the matrices of b1 , . . . , bl written through. The set t

V = {M ∈ Mn,k (ℝ) : MJbi M = 0, i = 1, . . . , l}

is clearly GLk (ℝ)-invariant. Let also for α ∈ I(n, k), Vα = V ∩ Uα . We pose now the set 󵄨󵄨 W ∩ X = {0} and 󵄨 Uk (ℝn , X ) = {W ∈ Gn,k (ℝ) 󵄨󵄨󵄨 } ⊂ Gn,k (ℝ), 󵄨󵄨 b (W, W) = 0, i = 1, . . . , l i

(3.11)

endowed with the trace topology induced by the Grassmannian topology. The following lemma is then immediate. Lemma 3.3.6. We keep the previous notation and hypotheses. The sets ηα (Vα ) where α in Is (n, k), constitute an open covering of Uk (ℝn , X ). Proof. We prove first of all that the sets ηα (Vα ), α ∈ Is (n, k) form a covering of Uk (ℝn , X ). Let W be a subspace of Uk (ℝn , X ). As W ∩ X = {0}, we get by lemma (3.3.5) that W ∈ ⋃α∈Is (n,k) ηα (Uα ). Let α ∈ Is (n, k) such that W ∈ ηα (Uα ). Then from bi (W, W) = 0 for all i = 1, . . . , l, we obtain η−1 α (W) ∈ V and, therefore, W ∈ ηα (Vα ). To see that these sets are opens, it is sufficient to show the following set equality: ηα (Vα ) = ηα (Uα ) ∩ Uk (ℝn , X ). Indeed, the direct inclusion is immediate. Let W be in the intersection, then there exists β ∈ Is (n, k) such that W ∈ ηβ (Vβ ). This means that η−1 β (W) ∈ V , which is invariant by the action of GLk (ℝ). Now W ∈ ηα (Uα ) implies that η−1 α (W) ∈ Uα . Furthermore, −1 (W) are in the same GL (ℝ)-orbit, then we get η−1 η−1 (W) and η k α (W) ∈ V . We obtain α β conclusively that η−1 α (W) ∈ V ∩ Uα = Vα . Let Rk (ℝn , X ) := η−1 (Uk (ℝn , X )).

102 | 3 Deformation and moduli spaces Then it is easy to see that Rk (ℝn , X ) is the semialgebraic set given by { { Rk (ℝn , X ) = {M ∈ Mn,k (ℝ) { {

󵄨󵄨 2 󵄨󵄨 ∑ (det Mα ) ≠ 0 and } } 󵄨󵄨 󵄨󵄨 α∈Is (n,k) . 󵄨󵄨 } } 󵄨󵄨 t 󵄨󵄨 MJbi M = 0, i = 1, . . . , l }

(3.12)

From Lemma 3.11, the restriction of η to Rk (ℝn , X ) and the restrictions of the trivializations χα to GLk (ℝ) × ηα (Vα ), give a GLk (ℝ)-principal bundle, whose total space is Rk (ℝn , X ) and the base space is Uk (ℝn , X ). In particular, Rk (ℝn , X ) =

⋃

α∈Is (n,k)

χα (GLk (ℝ) × ηα (Vα )).

(3.13)

3.3.2 The parameter space for normal subgroups We assume from now on that H is a normal subgroup of the exponential solvable Lie group G. Let as earlier Γ be an Abelian discrete subgroup of G and L its syndetic hull. We shall identify our group G with ℝn , which allows us to make use of the results of the previous section, specially concerning Grassmannians. We begin by remarking the following fact. Proposition 3.3.7. Retain the same hypotheses and notation. For any Abelian discrete subgroup Γ acting on G/H properly discontinuously, we have 󵄨󵄨 󵄨 dim ψ(l) = dim l and }. 󵄨󵄨 ψ(l) ∩ h = {0} 󵄨

󵄨 R (Γ, G, H) = {ψ ∈ Hom(l, g) 󵄨󵄨󵄨

Proof. We know already that for a subalgebra l of g, exp l acts properly on G/H if and only if h ∩ l = {0}. Then the result follows from Theorem 3.2.4. We choose a good sequence of subalgebras of g passing through [g, g] and h assumed to contain [g, g], from which we extract a basis {X1 , . . . , Xn } of g such that {X1 , . . . , Xs } is a basis of h and {X1 , . . . , Xl } a basis of [g, g] with l ≤ s ≤ n = dim g. As earlier, write for X, Y ∈ g, l

[X, Y] = ∑ bi (X, Y)Xi i=1

for some l alternated bilinear forms b1 , . . . , bl . Let B = {e1 , . . . , en } be the canonical basis of ℝn . We identify g, h and the space of the linear maps L (l, g) with ℝn , the subspace X defined in (3.10) and the space of matrices Mn,k (ℝ), respectively, via the isomorphism ei 󳨃→ Xi . We designate by the way by Uk (g, h) the corresponding set once we implement the above identifications. We characterize first Hom(l, g) as a quadratic cone of Mn,k (ℝ).

3.3 Case of Abelian discontinuous groups | 103

Lemma 3.3.8. For any Abelian discrete subgroup Γ of G, we have Hom(l, g) = {M ∈ Mn,k (ℝ) : t MJbi M = 0, i = 1, . . . , l}, where k = dim l. Proof. Since Γ is Abelian, its syndetic hull algebra l is obviously so. Thus, ψ ∈ L (l, g) is an algebras homomorphism if and only if ψ is a linear map satisfying [ψ(X), ψ(Y)] = 0, for all X, Y ∈ l, which gives rise to the following writing: Hom(l, g) = {ψ ∈ L (l, g), bi (ψ(X), ψ(Y)) = 0, for all X, Y ∈ l, i = 1, . . . , l}. If now M is the matrix of ψ written through B , then bi (ψ(X), ψ(Y)) = 0 for all X, Y ∈ l if and only if t MJbi M = 0. Therefore, the following result is immediate. Lemma 3.3.9. With the same hypotheses and notation as above we have, R (Γ, G, H) = Rk (g, h) := η−1 (Uk (g, h)) where k designates the rank of Γ. Proof. Proposition 3.3.7 and Lemma 3.3.8 enable us to get that { { { R (Γ, G, H) = {M ∈ Mn,k (ℝ) { { {

󵄨󵄨 󵄨󵄨 dim M(ℝk ) = k, } 󵄨󵄨 } } 󵄨󵄨 󵄨󵄨 M(ℝk ) ∩ h = {0} and } 󵄨󵄨 } } 󵄨󵄨 t 󵄨󵄨 MJb M = 0, i = 1, . . . , l } 󵄨 i

as Γ is Abelian. We get therefore using the set equality given in equation (3.11) that R (Γ, G, H) = η−1 (Uk (g, h)). The following result stems directly from the equality (3.13) and the last lemma. Corollary 3.3.10. Assume that G is exponential solvable, H is a normal subgroup of G and Γ is an Abelian discrete subgroup of G. Let (Vα )α∈Is (n,k) be the net of layers defined by (3.7) and denote for every α ∈ Is (n, k) by χ̃α the restriction of the trivialization map given in (3.9) to the set GLk (ℝ) × ηα (Vα ). Then R (Γ, G, H) =

⋃

α∈Is (n,k)

χ̃α (GLk (ℝ) × ηα (Vα )).

3.3.3 The deformation space for normal subgroups We proceed in this section to the proof of our results. From now on, we assume that our group G is exponentially solvable and unless specific mention that the subgroup H contains [G, G]. We remark first of all of the following.

104 | 3 Deformation and moduli spaces Lemma 3.3.11. Every discrete subgroup Γ of G, acting on G/H properly discontinuously is isomorphic to ℤk for some integer k. Proof. As H is a normal subgroup of G and Γ acts on G/H properly discontinuously, we get that Γ ∩ H = {e}. As such, we also that [Γ, Γ] ⊂ [G, G] ∩ Γ ⊂ H ∩ Γ, as h contains [g, g]. This shows then that Γ is Abelian. We consider now the actions of G on Mn,k (ℝ) ≅ L (l, g) and Gn,k (ℝ) given by M ⋅ g = Adg −1 M and W ⋅ g = Adg −1 W. Then we have the following. Lemma 3.3.12. For every α ∈ Is (n, k), the set Vα is G-stable and ηα is G-equivariant. In particular, η−1 (ηα (Vα )) is an open G-stable subset of R (Γ, G, H). Proof. Let X ∈ g, using the hypothesis h ⊇ [g, g], and we can write Adexp X = (

C (0)

D ) In−s

for some C ∈ Ms (ℝ) and D ∈ Ms,n−s (ℝ). Let W ∈ ηα (Vα ), M = η−1 α (W) and write M as A M=( ) B for some A ∈ Ms,k (ℝ) and B ∈ Mn−s,k (ℝ) and Mα = Ik . We get therefore that CA + DB ), B

Adexp X M = (

and that (Adexp X M)α = Mα = Ik , because Mα depends only on B, for all α ∈ Is (n, k). Moreover, and with the above in mind, the hypothesis [W, W] = {0} is equivalent to M t Jbi M = 0, i = 1, . . . , l, which leads to the fact that [Adexp X (W), Adexp X (W)] = {0} as Adexp X is an automorphism of G. This is also equivalent to t

(Adexp X M)Jbi Adexp X M = 0,

i = 1, . . . , l.

We obtain finally that for every α ∈ Is (n, k) the set Vα is invariant through the right∘ side action of G on Mn,k (ℝ) by left multiplication. On the other hand, for g ∈ G and M ∈ Vα , we have that ηα (M ⋅ g) = ηα (Adg −1 M)

= (Adg −1 M)(ℝk )

= Adg −1 (M(ℝk )) = ηα (M) ⋅ g, which shows that the map ηα is G-equivariant.

3.3 Case of Abelian discontinuous groups | 105

For any g ∈ G and any (f , W) ∈ GLk (ℝ) × ηα (Vα ), define the product −1 (f , W) ⋅ g = (χα,W⋅g ∘ Adg −1 ∘χα,W ∘ f , W ⋅ g).

(3.14)

Proposition 3.3.13. For every α ∈ Is (n, k), the set GLk (ℝ) × ηα (Vα ) is a G-space through the right-side action defined as in (3.14). Furthermore, χα̃ : GLk (ℝ) × ηα (Vα ) (f , W)

󳨀→ 󳨃→

η−1 (ηα (Vα )) χα,W ∘ f

defines a G-equivariant homeomorphism. Proof. Note first of all that χα̃ (GLk (ℝ) × ηα (Vα )) = η−1 (ηα (Vα )). As the trivalization χα is a homeomorphism, so is its restriction χα̃ . For (f , W) ∈ GLk (ℝ) × ηα (Vα ) and g, g ′ ∈ G, we get easily the following: −1 ((f , W) ⋅ g) ⋅ g ′ = (χα,W⋅g ∘ Adg −1 ∘χα,W ∘ f , W ⋅ g) ⋅ g ′ −1 −1 ′ = (χα,(W⋅g)⋅g ′ ∘ Ad ′ −1 ∘χα,W⋅g ∘ χα,W⋅g ∘ Adg −1 ∘χα,W ∘ f , (W ⋅ g) ⋅ g ) g −1 ′ = (χα,W⋅gg ′ ∘ Ad ′ −1 ∘ Adg −1 ∘χW ∘ f , W ⋅ gg ) g

= (f , W) ⋅ gg ′ , which shows that G acts on the space GLk (ℝ) × ηα (Vα ). On the other hand, for g ∈ G and φ ∈ R (Γ, G, H), we have −1 χα ((f , W) ⋅ g) = χα (χα,W⋅g ∘ Adg −1 ∘χα,W ∘ f , W ⋅ g) −1 = χα,W⋅g ∘ (χα,W⋅g ∘ Adg −1 ∘χα,W ∘ f )

= Adg −1 ∘χα (f , W) = χα (f , W) ⋅ g,

which shows that χα̃ is G-equivariant. This completes the proof of the proposition. Proof of Theorem 3.3.1. From the continuity of η, Lemma 3.3.6 and Lemma 3.3.9, we get that the sets η−1 (ηα (Vα )), α ∈ Is (n, k) form an open covering of R (Γ, G, H). Using Proposition 3.3.13, we can see that, for any α ∈ Is (n, k) the set η−1 (ηα (Vα )) is G-stable. Denote by −1

Tα = η (ηα (Vα ))/G.

106 | 3 Deformation and moduli spaces It follows that T (Γ, G, H) =

⋃

α∈Is (n,k)

Tα .

Furthermore, Proposition 3.3.13 gives the identification Tα = (GLk (ℝ) × ηα (Vα ))/G.

For α ∈ Is (n, k), one has that −1 χα,W⋅g ∘ Adg ∘χα,W = (Adg ∘η−1 α (W))α

= (η−1 α (W))α = Ik .

The action of G on GLk (ℝ) × ηα (Vα ) does not affect therefore the factor GLk (ℝ). Furthermore, two pairs (f , W) and (f ′ , W ′ ) of the product GLk (ℝ) × ηα (Vα ) are in the same orbit if and only if W and W ′ are in the same orbit of the action on ηα (Vα ). Then up to a homeomorphism, and using the G-equivariance of the homeomorphism ηα , we obtain Tα = GLk (ℝ) × (ηα (Vα )/G)

= GLk (ℝ) × ηα (Vα /G)

= GLk (ℝ) × Vα /G.

This completes the proof of the assertion concerning the deformation space. Now the group Aut(Γ) can be identified to the subgroup of Aut(l) = GLk (ℝ), which leaves the Abelian subgroup log Γ = ℤk stable in l = ℝk . Then Aut(Γ) = GLk (ℤ) and acts on Mn,k (ℝ) by the same way as GLk (ℝ) does. This action leaves the sets η−1 (ηα (Vα )) stable and commutes with the action of G. Then, for −1

Mα := Aut(Γ)\η (ηα (Vα ))/G,

we have M (Γ, G, H) =

⋃

α∈Is (n,k)

Mα .

For a given α ∈ Is (n, k), the trivialization χα̃ transfers the action of Aut(Γ) on η−1 (ηα (Vα )) to the action of Aut(Γ) on GLk (ℝ)×ηα (Vα ) given by a⋅(f , W) = (f ∘a−1 , W). Let now (f , W) and (f ′ , W ′ ) belong to the same double coset Aut(Γ) ⋅ x ⋅ G, which means equivalently that W and W ′ are in the same G-orbit in ηα (Vα ) and f and f ′ are in the same orbit of the action of Aut(Γ) = GLk (ℤ) on GLk (ℝ), given by a ⋅ f = f ∘ a−1 . This means that the

3.3 Case of Abelian discontinuous groups | 107

canonical quotient map GLk (ℝ) × ηα (Vα ) → Aut(Γ)\(GLk (ℝ) × ηα (Vα ))/G factors to a homeomorphism from GLk (ℝ)/GLk (ℤ) × ηα (Vα /G) ≅ GLk (ℝ)/GLk (ℤ) × (Vα /G) to Aut(Γ)\(GLk (ℝ) × ηα (Vα ))/G = Mα , which is an open set of M (Γ, G, H). This completes the proof of the theorem.

3.3.4 Examples We study in the section some concrete examples of exponential and nilpotent Lie groups. We put the emphasis on the case where dim G ≤ 3. Example 3.3.14. Let G = Aff(ℝ) with the Lie algebra g = aff(ℝ) and a basis {X, Y} such that [X, Y] = Y. So, obviously [g, g] = ℝY. If H = exp(h) and Γ are as before, then h = ℝY and the Clifford–Klein form Γ\G/H is compact (otherwise, h = g and Γ is trivial). It turns out that the deformation space T (Γ, G, H) is homeomorphic to the space GL1 (ℝ) × (V1 /G), where V1 is the set X + ℝY. Let now g = exp(tX) exp(sY) for some s, t ∈ ℝ, then a routine computation shows that Ad(g)(X + αY) = X + (α − s)et Y,

α ∈ ℝ.

It turns out that the deformation space is homeomorphic to the space GL1 (ℝ) × {X}. Finally, we get that ×

T (Γ, G, H) ≃ ℝ := ℝ \ {0}

and M (Γ, G, H) ≃ ℝ>0 .

Example 3.3.15. Suppose now that G is exponential solvable of dimension 3. Then up to an isomorphism, one can assume that g admits a basis {A, X, Y} with nontrivial brackets: [A, X] = X − αY,

[A, Y] = αX + Y

for some α ∈ ℝ× (see [106]). In this situation, [g, g] = ℝX ⊕ ℝY and similar to the setting above, h = [g, g] and the Clifford–Klein form Γ\G/H is compact. We get then

108 | 3 Deformation and moduli spaces that T (Γ, G, H) is homeomorphic to the space GL1 (ℝ) × (V1 /G), where V1 is the set A + ℝY + ℝX. By an easy computation, we get Ad(g)(A + aX + bY) = A + es X((a − x − αy) cos(αs) + (b − y + αx) sin(αs))

+ es Y((b − y + αx) cos(αs) + ((−a + x + αy) sin(αs)).

Here, a, b ∈ ℝ and g = exp(sA) exp(xX) exp(yY). This gives rise to the fact that the deformation space is homeomorphic to the space GL1 (ℝ) × {A}. Finally, we have again that T (Γ, G, H) ≃ ℝ

×

and M (Γ, G, H) ≃ ℝ>0 .

Example 3.3.16. Assume now that G is nilpotent and non-Abelian. Then up to an isomorphism, G is the three-dimensional Heisenberg group whose Lie algebra admits a basis (X, Y, Z) such that [X, Y] = Z. According to our circumstances, we may assume for instance that h = ℝZ ⊕ ℝY and Γ is a rank one discrete subgroup generated, for example, by exp(X). Note here that this choice of h is irrelevant once we agree that it is two-dimensional and that the Clifford–Klein form in question is compact. We obtain as above that T (Γ, G, H) is homeomorphic to the space GL1 (ℝ) × ((X + ℝY + ℝZ)/G) ≃ ℝ× × {X + ℝY} ≃ ℝ× × ℝ and similarly that M (Γ, G, H) ≃ ℝ>0 × ℝ.

If h is one-dimensional, there does not exist any cocompact discrete subgroup for G/H. We are then submitted to look at the situation where the rank of Γ is one for which the Clifford–Klein form in question is not compact. As such, the deformation space T (Γ, G, H) is the union of the open sets GL1 (ℝ) × ((X + ℝY + ℝZ)/G) and GL1 (ℝ) × ((Y + ℝX + ℝZ)/G). Finally, ×

×

T (Γ, G, H) ≃ ℝ × (Y + ℝX) ∪ ℝ × (X + ℝY).

On the other hand, when we look at the trivialization corresponding to the layers Y + ℝX + ℝZ and X + ℝY + ℝZ, one easily gets that ℝ× × (Y + ℝX) is homeomorphic to

3.4 Non-Abelian discontinuous groups | 109

ℝ× Y + ℝX and likewise ℝ× × (X + ℝY) is homeomorphic to ℝ× X + ℝY. Finally, we get that T (Γ, G, H) ≃ ℝ2 \ {(0, 0)} and 2

M (Γ, G, H) ≃ {(x, y) ∈ ℝ : x > 0 or y > 0}

≃ ℝ2 \ (ℝ≤0 × ℝ≤0 ).

3.4 Non-Abelian discontinuous groups 3.4.1 Structure of a principal fiber bundle We retain all our hypotheses and notation. Let k = dim l, s = dim h and n = dim g. We fix a basis {X1 , . . . , Xn } of g passing through h. We identify the vector spaces g to ℝn , l to ℝk , h to the s-dimensional subspace ℝs × 0ℝn−s of ℝn , L (l, g) to Mn,k (ℝ) the real vector space of n × k matrices with real entries and Hom(l, g) to a closed subset of Mn,k (ℝ). ∘ Let Mn,k (ℝ) be the open set of Mn,k (ℝ) consisting of rank k matrices in Mn,k (ℝ), which is also identified to the set {φ ∈ L (l, g), φ injective}. We define the set I(n, k) = {(i1 , . . . , ik ) ∈ ℕk , 1 ≤ i1 < ⋅ ⋅ ⋅ < ik ≤ n}. For L1 M = ( ... ) ∈ Mn,k (ℝ) and α = (i1 , . . . , ik ) ∈ I(n, k), Ln we denote by Mα the k × k relative minor Li1 . ( .. ) . Lik Let ∘ Uα = {M ∈ Mn,k (ℝ) : Mα = Ik } ≅ Mn−k,k (ℝ)

and ∘

Uα = {M ∈ Mn,k (ℝ), det Mα ≠ 0} ≅ Mn−k,k (ℝ) × GLk (ℝ),

where Ik designates the identity element of Mk (ℝ). Then clearly ∘ Mn,k (ℝ) =

⋃ Uα .

α∈I(n,k)

(3.15)

110 | 3 Deformation and moduli spaces The group GLk (ℝ) acts on Mn,k (ℝ) through the right multiplication and the Grassmannian Gn,k (ℝ) of the k-dimensional subspaces of ℝn is identified to the quotient topolog∘ ∘ ical space Mn,k (ℝ)/GLk (ℝ). Let η : Mn,k (ℝ) → Gn,k (ℝ), M 󳨃→ M(ℝk ) be the canonical surjection. It is easy to see that the restriction ηα of η to the set Uα is a homeomorphism between Uα and its image. The group Aut(l) of automorphisms of l is a closed subgroup of GLk (ℝ), then the homogeneous space GLk (ℝ)/ Aut(l) is endowed with a manifold structure and the quotient map p : GLk (ℝ) → GLk (ℝ)/ Aut(l) admits local sections. Consider an open covering {Vβ }β∈I of GLk (ℝ)/ Aut(l) such that for any β ∈ I, there is a section sβ : Vβ → GLk (ℝ) satisfying p ∘ sβ = Id|Vβ . For every α ∈ I(n, k), we consider the map πα : Uα

󳨀→

M

󳨃→

GLk (ℝ)/ Aut(l) × η(Uα )

(p(Mα ), η(M))

and let Uαβ = πα−1 (Vβ × η(Uα )). Clearly, πα is continuous and surjective, Uαβ is open in Mn,k (ℝ) and the collection (Uαβ )β∈I constitutes an open covering of Uα . We finally consider the map ξαβ : Aut(l) × Vβ × η(Uα ) (A, x, W)

󳨀→

Uαβ

󳨃→

η−1 α (W)sβ (x)A.

This map is well-defined as for any (A, x, W) ∈ Aut(l) × Vβ × η(Uα ); the conclusion πα ∘ ξαβ (A, x, W) = (x, W) holds. Indeed, it is clear that the matrix M = η−1 α (W)sβ (x) is in πα−1 (x, W) and the set M Aut(l) is a subset of πα−1 (x, W). We further have the following useful properties. Lemma 3.4.1. For every α ∈ I(n, k), β, β′ ∈ I and (A, x, W) ∈ Aut(l) × Vβ × η(Uα ), we have: (1) ξαβ is a homeomorphism. (2) πα−1 (x, W) = η−1 α (W)sβ (x) Aut(l). (3) The map ξαβ,x,W : Aut(l) → πα−1 (x, W) given by ξαβ,x,W (A) = η−1 α (W)sβ (x)A is a homeomorphism. −1 (4) The map tαββ′ : (Vβ ∩ Vβ′ ) × η(Uα ) → Aut(l) given by tαββ′ (x, W) = ξαβ,x,W ∘ ξαβ′ ,x,W is continuous. Proof. For the first statement, it is easy to see that the map ξαβ is continuous. Note that for every M ∈ Uαβ , we have p(sβ (p(Mα ))) = p(Mα ) and then sβ (p(Mα ))−1 Mα ∈ Aut(l). Let ′ ξαβ : Uαβ

M

󳨀→

Aut(l) × Vβ × η(Uα )

󳨃→

(sβ (p(Mα )) Mα , p(Mα ), η(M)),

−1

(3.16)

3.4 Non-Abelian discontinuous groups | 111

′ ′ then ξαβ is continuous and ξαβ ∘ ξαβ = Id|Uαβ . Concerning the second statement, let

M ∈ πα−1 (x, W), then η(M) = W and p(Mα ) = x, which is equivalent to M = η−1 α (W)Mα

and sβ (x)−1 Mα ∈ Aut(l).

It follows that M = η−1 α (W)sβ (x)B for some B ∈ Aut(l). This proves the first direct inclusion and the converse is straight immediate. To prove (3), we consider the map ′ ′ ξαβ,x,W : πα−1 (x, W) → Aut(l) given by ξαβ,x,W (M) = sβ (x)−1 Mα . Then clearly ξαβ,x,W and ′ ′ ξαβ,x,W are continuous and ξαβ,x,W ∘ ξαβ,x,W is the identity map. For the last statement, it is pretty clear that −1 ξαβ,x,W ∘ ξαβ′ ,x,W (A) = sβ (x)−1 sβ′ (x)A. −1 The composition ξαβ,x,W ∘ ξαβ′ ,x,W is then the left translation identified to the element

sβ (x)−1 sβ′ (x) ∈ Aut(l). Furthermore, the continuity of tαββ′ is a direct consequence of the continuity of sβ and sβ′ . Consider the set Vα = Hom(l, g) ∩ Uα .

(3.17)

Note that for any α ∈ I, the set Vα is closed and stable by the action of Aut l on Uα , (regarded as a subgroup of GLk (ℝ)). Let us define the sets Wα = πα (Vα ),

Wαβ = Wα ∩ (Vβ × η(Uα )) and

−1

Wαβ = πα (Wαβ ) = Vα ∩ Uαβ . (3.18)

Then obviously Vα = ⋃ Wαβ β∈I

and Wα = ⋃ Wαβ . β∈I

(3.19)

The following lemma is immediate. Lemma 3.4.2. The set Wα is closed in (GLk (ℝ)/ Aut(l)) × η(Uα ), Wαβ is open in Wα and Wαβ is open in Vα . Proof. The set Vα is a closed in Uα and Aut(l)-stable, then πα (Vα ) = Wα is closed. The set Wαβ (resp., Wαβ ) is an intersection of Wα (resp., Vα ) with an open set, as it is to be shown. With the above in mind, we get the following. Corollary 3.4.3. For any α ∈ I(n, k) and any β ∈ I, the map ξαβ realizes a homeomorphism between Aut(l) × Wαβ and Wαβ .

112 | 3 Deformation and moduli spaces Proof. Toward the proof, it is sufficient to see that ξαβ (Aut(l) × Wαβ ) = Wαβ . Let (A, x, W) ∈ (Aut(l) × Wαβ ), then ξαβ (A, x, W) ∈ πα−1 (x, W) ⊂ πα−1 (Wαβ ) = Wαβ . Conversely, ′ ′ let M ∈ Wαβ , then ξαβ (M) ∈ Aut(l) × Wαβ and M = ξαβ (ξαβ (M)). With the above in mind, we get the following. Theorem 3.4.4. The collection (πα , Uα , (GLk (ℝ)/ Aut(l)) × η(Uα ), Aut(l)) defines a principal bundle; the maps (ξαβ )β∈I are the local trivializations and the maps (tαββ′ )β′ ∈I are the transition functions. Furthermore, the data (πα , Vα , Wα , Aut(l)) defines a principal bundle. We now make the following remark. For any α and β, the group Aut(Γ) acts on Aut(l) × Wαβ by −1

a ⋅ (A, x, W) = (A ∘ d(i(a))| , x, W) e

(3.20)

and the map ξαβ is Aut(Γ)-equivariant. If the set Wαβ is G-stable, then we define a map cW,α from G in GLk (ℝ) given by −1 Adg η−1 α (W) = ηα (g ⋅ W)cW,α (g).

(3.21)

−1 Using the identity Adg ′ g η−1 α (W) = Adg ′ Adg ηα (W) and the definition of cW,α , we get

cW,α (g ′ g) = cg⋅W,α (g ′ )cW,α (g).

(3.22)

As a direct consequence, we obtain the following. Proposition 3.4.5. For any α ∈ I(n, k) and any β ∈ I, if Wαβ is stable by the G-action, then G acts on Aut(l) × Wαβ through the following law: −1

g ⋅ (A, x, W) := (sβ (p(cW,α (g)sβ (x))) cW,α (g)sβ (x)A, p(cW,α (g)sβ (x)), g ⋅ W) for any (A, x, W) ∈ Aut(l) × Wαβ and any g ∈ G. Moreover, this action commutes with the action of Aut(Γ) and the trivialization map ξαβ is G-equivariant. Proof. Let M = η−1 α (W)sβ (x)A ∈ Wαβ . As Wαβ is G-stable, Adg M ∈ Wαβ and, therefore, ′ ′ (Adg M) = (Adg M) ∈ Aut(l) × Wαβ for any g ∈ G. It is then easy to see that ξαβ ξαβ ′ g ⋅(A, x, W), where ξαβ is given as in (3.16), which proves that the action is well-defined. Using (3.22), we directly get g ′ ⋅ (g ⋅ (A, x, W)) −1

= (sβ (p(cg⋅W,α (g ′ )sβ (p(cW,α (g)sβ (x))))) cg⋅W,α (g ′ )sβ (p(cW,α (g)sβ (x))) −1

× sβ (p(cW,α (g)sβ (x))) cW,α (g)sβ (x)A, p(cg⋅W,α (g ′ )sβ (p(cW,α (g)sβ (x)))), g ′ g ⋅ W),

3.4 Non-Abelian discontinuous groups | 113

for any g, g ′ ∈ G. Besides, p(cg⋅W,α (g ′ )sβ (p(cW,α (g)sβ (x)))) = p(cg⋅W,α (g ′ )cW,α (g)sβ (x)). This conclusively gives g ′ ⋅ (g ⋅ (A, x, W)) −1

= (sβ (p(cg⋅W,α (g ′ )cW,α (g)sβ (x))) cg⋅W,α (g ′ )cW,α (g)sβ (x)A, p(cg⋅W,α (g ′ )cW,α (g)sβ (x)), g ′ g ⋅ W) −1

= (sβ (p(cW,α (g ′ g)sβ (x))) cW,α (g ′ g)sβ (x)A, p(cW,α (g ′ g)sβ (x)), g ′ g ⋅ W) = g ′ g ⋅ (A, x, W). The commutation properties is a consequence of (3.20). For the G-equivariance equation, let g ∈ G and (A, x, W) ∈ Aut(l) × Wαβ . We have −1 g ⋅ ξαβ (A, x, W) = Adg η−1 α (W)sβ (x)A = ηα (g ⋅ W)cW,α (g)sβ (x)A

= ξαβ (g ⋅ (A, x, W)). 3.4.2 The context where [Γ, Γ] is uniform in [G, G] Throughout this section, H denotes a connected Lie subgroup of a completely solvable Lie group G and Γ a discontinuous group for the homogeneous space G/H whose syndetic hull L fulfills [L, L] = [G, G] (or equivalently [Γ, Γ] is uniform in [G, G]). The aim here is to give a comprehensive description of the associated deformation space. It is worth mentioning that the techniques used here also allow us to treat the context where [G, G] ⊂ H as it was the case in [31]. In such a situation and unlike the present setup, the discontinuous group is easily shown to be Abelian. The parameter space This subsection is devoted to produce a description of the subsequent parameter space R(Γ, G, H). This is important to a major extent to see that such a space smoothly splits into some G-invariant constituents. We first see the following. Proposition 3.4.6. Retain the same hypotheses and notation and assume that H is normal or [L, L] = [G, G]. Then 󵄨󵄨 󵄨󵄨 dim φ(l) = dim l R(Γ, G, H) = {φ ∈ Hom(l, g) 󵄨󵄨󵄨 }. 󵄨󵄨 φ(l) ∩ h = {0} 󵄨 In particular, R(Γ, G, H) is open in Hom(l, g) and semialgebraic. Proof. Our assumption says that [l, l] = [g, g]. Let φ ∈ R(l, g, h). Clearly, φ([g, g]) = [φ(l), φ(l)] ⊂ [g, g]. From the injectivity of φ, we deduce that [φ(l), φ(l)] = [g, g]. In

114 | 3 Deformation and moduli spaces particular, φ(l) ⊃ [g, g], which means that φ(l) is an ideal of g. Using Lemma 2.1.11, the proper action of exp φ(l) on G/H is equivalent to φ(l) ∩ h = {0}, and the result follows. With the above in mind, we now consider the decomposition, ∘ {φ ∈ Hom(l, g), dim φ(l) = dim l} = Hom(l, g) ∩ Mn,k (ℝ) =

⋃ Vα ,

α∈I(n,k)

(3.23)

which is open in Hom(l, g) and closed in Uα . Now, the following fact is proved in Lemma 3.3.5: For all φ ∈ L (l, g), such that dim φ(l) = dim l, we have φ(l) ∩ h = {0} if and only if there exists α in Is (n, k) such that φ ∈ Uα , where Is (n, k) = {(i1 , . . . , ik ), s < i1 < ⋅ ⋅ ⋅ < ik ≤ n}. Up to this step, Lemma 3.23 and the decomposition (3.19) of Vα , we get the following. Corollary 3.4.7. Suppose that H is normal or [L, L] = [G, G]. Then up to the homeomorphism we have R (Γ, G, H) =

⋃ Wαβ ,

α∈Is (n,k), β∈I

as a union of open sets in Hom(l, g). The deformation space Our main upshot in this section is the following. Theorem 3.4.8. Let G be a completely solvable Lie group, H a connected subgroup of G and Γ a discontinuous group for G/H such that [L, L] = [G, G]. Then for any α ∈ Is (n, k) and any β ∈ I, the set Wαβ is G-stable and the deformation and moduli spaces read T (Γ, G, H) =

⋃ Wαβ /G,

α∈Is (n,k), β∈I

M (Γ, G, H) =

⋃ Aut(Γ)\Wαβ /G,

α∈Is (n,k), β∈I

as a union of open sets. Furthermore, G acts on Aut(l)×Wαβ , the set Wαβ /G is homeomorphic to (Aut(l) × Wαβ )/G and the set Aut(Γ)\Wαβ /G is homeomorphic to Aut(Γ)\(Aut(l) × Wαβ )/G. Proof. Note first from Corollary 3.4.7 that R (Γ, G, H) =

⋃ Wαβ .

α∈Is (n,k), β∈I

From Corollary 3.4.3 and Proposition 3.4.5, we are done, provided that for any α ∈ Is (n, k) and β ∈ I, the set Wαβ is G-stable, which remains to be proved. Indeed, let

3.4 Non-Abelian discontinuous groups | 115

M ∈ Wαβ . For W = η(M) and x = p(Mα ), there is A ∈ Aut(l) such that M = η−1 α (W)sβ (x)A. When [l, l] = [g, g], for any M ∈ R(l, g, h), the subspace M(l) is an ideal of g. It follows therefore that g ⋅ W = W for any g ∈ G and from (3.21) we obtain Adg M = η−1 α (W)cW,α (g)sβ (x)A = McA,x,W,α (g),

(3.24)

where cA,x,W,α (g) = A−1 sβ (x)−1 cW,α (g)sβ (x)A. The set Wαβ is stable by the action of Aut(l), then to prove that Adg M ∈ Wαβ , it is sufficient to show the following lemma. Lemma 3.4.9. The map cA,x,W,α is a continuous homomorphism from G to Aut(l). Proof. The map cA,x,W,α : G → GLk (ℝ) is nothing but the composition of cW,α and the conjugation by the element A−1 sβ (x)−1 . From the fact that g ⋅ W = W for any g ∈ G and equation (3.22), we deduce that the map cW,α is a homomorphism and so is cA,x,W,α . The continuity is also a direct consequence from the continuity of the action of G on Mn,k (ℝ) and the expression cW,α (g) = (g ⋅ η−1 α (W))α ,

g ∈ G.

(3.25)

To conclude, we have to prove that cA,x,W (g) ∈ Aut(l). Let X, Y ∈ l, then M([cA,x,W,α (g)(X), cA,x,W,α (g)(Y)]) = [McA,x,W,α (g)(X), McA,x,W,α (g)(Y)] = Adg M[X, Y]

= McA,x,W,α (g)[X, Y]. This entails that M([cA,x,W,α (g)(X), cA,x,W,α (g)(Y)] − cA,x,W,α (g)[X, Y]) = 0. As M is of maximal rank, we are done. We close this section by the following remark, which will be of interest later on. Retain the same hypotheses and notation, especially that G is completely solvable and that [l, l] = [g, g]. Then as in Proposition 3.4.5, ξαβ is a G-equivariant homeomorphism from Wαβ and Aut(l)×Wαβ for any α ∈ Is (n, k) and any β ∈ I. Under these circumstances, the action of G is defined by g ⋅ (A, x, W) = (cx,W,α (g)A, x, W),

(A, x, W) ∈ Aut(l) × Wαβ ,

g ∈ G,

(3.26)

where cx,W,α (g) = sβ (x)−1 cW,α (g)sβ (x). Then Aut(l) × Wαβ splits into a union of fibers of the natural projection on Wαβ and each fiber is G-stable, or equivalently, the orbits of Aut(l) in Wαβ are G-stable. From Lemma 3.4.9, for (x, W) ∈ Wαβ we can see that the map cx,W,α : G → Aut(l) is a group homomorphism. This entails in fact that cx,W,α (G) is a Lie subgroup of Aut(l). To get that, it is sufficient to see the following. Lemma 3.4.10. Let c : G → G′ be a continuous Lie groups homomorphism. Suppose that G is connected, then the image of G is connected Lie subgroup of G′ .

116 | 3 Deformation and moduli spaces Proof. Let g be the Lie algebra of G and k = dc|e (g), then k is a Lie subalgebra of g′ , the Lie algebra of G′ . Now G is connected, then it is generated by exp(g) and c ∘ expG (X) = expG′ ∘dc|e (X), for any X ∈ g. This conclusively shows that c(G) is the connected Lie subgroup generated by expG′ (k). Remark 3.4.11. From (3.26), we can see that the natural projection of Aut(l) × Wαβ on Wαβ factors through the action of G to a continuous surjection from Wαβ /G to Wαβ such that the fiber of (x, W) is homeomorphic to the homogeneous space Aut(l)/cx,W,α (G).

4 The deformation space for nilpotent Lie groups The present chapter aims to provide a comprehensive description of the deformation and moduli spaces of the action of a discontinuous group Γ for a homogeneous space G/H, where H stands for an arbitrary, connected, closed subgroup of a connected and simply connected, nilpotent Lie group G. We shall figure out how complicated the explicit calculations could be, given the complexity of Lie brackets of Lie algebras in question and the lack of complete characterization of discrete subgroups of a given nilpotent Lie group. We treat first the setting of Heisenberg groups answering several questions. The case where the center of G sits inside the subgroup H plays a capital role in the computation. The case where the Clifford–Klein forms are compact presents itself major technical difficulties. We also consider the settings of general 2-step and 3-step nilpotent Lie groups and also the threadlike case (cf. Subsection 1.1.3). Let P := G×G the direct product Lie group and Δ := Δ(G) the diagonal Lie subgroup of P. The emphasis in also focused on the explicit determination of the deformation space of discontinuous groups acting on the nilpotent homogeneous space P/Δ for which the group G is the Heisenberg group. One motivation to seek such a setting comes up from the test case G = SL(2, ℝ), where all of the related objects are nicely determined, and specifically, the deformation space is nothing but the deformation of complex structures on 3-dimensional manifolds. Though this case does not overlap in nature with the classes of groups we are interested in, it arouses our interest to begin the study of such a problem in some nilpotent contexts.

4.1 Deformation and moduli spaces for Heisenberg groups 4.1.1 A criterion of the proper action, continued Let g be a Heisenberg Lie algebra and G its associated Lie group as defined in Subsection 1.1.2. Let Bh a symplectic basis of g adapted to h as in Proposition 1.1.12. Remark first that the matrix Jb of the bilinear form b defined by equation (1.6) written in Bh is as follows: 0 ̇ (0) (. Jb := M (b, Bh ) = ( (. . 0 (

https://doi.org/10.1515/9783110765304-004

̇

̇

0 (−In ) ) ). )

(In )

(0) )

118 | 4 The deformation space for nilpotent Lie groups Using Theorem 1.1.6, one can view G as the direct product of D = ℝ2n and ℝ with the following pointwise multiplication: 1 g1 g2 = (v + w, s + t + b(v, w)), 2

g1 = (v, s),

g2 = (w, t),

where b is explicitly given on D by b(v, w) = ⟨v1 , w2 ⟩ − ⟨v2 , w1 ⟩,

v = (v1 , v2 ),

w = (w1 , w2 ),

where v1 , w1 designate the coordinates of v and w, respectively, through the basis vectors (X1 , . . . , Xn ) and v2 , w2 their coordinates through the vectors (Y1 , . . . , Yn ). We get now the following characterization (which will be of multiple use later on) of the proper action in the Heisenberg setting. We have the following. Lemma 4.1.1. Let h, l be two subalgebras of g and H = exp h. Then exp l acts properly on G/H if and only if one of these two properties is satisfied: (𝚤) z ⊂ h and l ∩ h = {0}. (𝚤𝚤) z ⊄ h, l ∩ h = {0} and z ∩ (h ⊕ l) = l ∩ z. Proof. The Heisenberg Lie algebra is a 2-step nilpotent Lie algebra. Using Theorem 2.2.2, we get that the proper action is equivalent to the property Adg h ∩ l = {0} for all g ∈ G (or equivalently Adg l ∩ h = {0} for all g ∈ G). If z ⊂ h, it is then clear that Adg h = h and that the proper action is equivalent to l ∩ h = {0}. Assume that the action is proper and z ⊄ h, so obviously h ∩ l = {0}. Toward the equality z ∩ (h ⊕ l) = l ∩ z, it is sufficient to show that z ∩ (l ⊕ h) ⊂ (z ∩ l). Let x ∈ z ∩ (l ⊕ h). There exist then l ∈ l and h ∈ h such that x = l + h. We have to show that x = l. Suppose that l ∉ z. Then there exist X ∈ g such that [X, l] = −x as the center is one-dimensional. As such, the nontrivial element Adexp X l = l − x = −h belongs to the intersection Adexp X l ∩ h, which is impossible. This leads to the fact that x − l is a central element and belongs to h, which also means that it is trivial. Conversely, let t ∈ Adg l ∩ h. We have t = l + x with x ∈ z and l ∈ l. Then x ∈ z ∩ (l ⊕ h), which means that x ∈ l, and finally t ∈ h ∩ l = {0}. This consideration shows conclusively that exp l acts properly on G/H. 4.1.2 The deformation space for non-Abelian actions Let Γ be a non-Abelian discontinuous subgroup of G for G/H. Let L = exp(l) be the syndetic hull of Γ and Bl a symplectic basis of g adapted to l as in Proposition 1.1.12. We single out from Bl a new basis Bl′ of l as follows: Bl := {Z, Xp+1 , . . . , Xp+q , X1 , . . . , Xp , Y1 , . . . , Yp }. ′

We also fix a basis of g passing through h.

(4.1)

4.1 Deformation and moduli spaces for Heisenberg groups | 119

We refer back to Subsection 3.4 for notation and definitions. Let us denote by Is1 (2n + 1, k) the set of elements (as defined by equation (3.15)) of the form α = (s + 1, i2 , . . . , ik ). Pick first a finite open covering {Vβ }β∈I of GLk (ℝ)/ Aut(l) and its local sections sβ : Vβ → GLk (ℝ), β ∈ I. For α ∈ Is1 (2n + 1, k) and β ∈ I, we consider the set t t

Aα,β := {(A, M) ∈ sβ (Vβ ) × Uα : A MJg MA = Jl },

where Jg and Jl designates the matrices of b and of b|l , respectively, written in the basis Bl . Our main result in this section is the following. Theorem 4.1.2. Let G be the (2n + 1)-dimensional Heisenberg group, H a connected Lie subgroup of G and Γ a non-Abelian discontinuous subgroup of G for G/H. Let L = exp(l) be the syndetic hull of Γ. There exists a finite set of local sections (sβ )β∈I for the canonical surjection GLk (ℝ) → GLk (ℝ)/ Aut(l), such that the deformation space of Γ acting on G/H reads T (Γ, G, H) =

⋃

β∈I α∈Is1 (2n+1,k)

Tαβ ,

where for β ∈ I and α ∈ Is1 (2n + 1, k) the set Tαβ is open in T (Γ, G, H) and homeomorphic to the set ℝ∗ × ℝ2pq × Sp(2p) × GLq (ℝ) × Aα,β . Here, 1 + 2p + q = k and q + 1 = dim z(l), where z(l) is the center of l. Proof. We first start by proving a series of useful results on which the proof of the main upshot rests. Let f be an automorphism of l. Then clearly f (Z) ∈ z and f (z(l)) = z(l). Then with respect to the basis Bl′ , the matrix M of f is of the form a M = (0 0

c G 0

d F) , E

with a ∈ ℝ∗ ,

G ∈ GLq (ℝ),

c ∈ M1,q (ℝ),

F ∈ Mq,2p (ℝ),

d ∈ M1,2p (ℝ),

E ∈ GL2p (ℝ).

(4.2)

Let us denote by aM , the coefficient a of the matrix M. As f is a homomorphism, it comes out that t

MJl M = aM Jl ,

0ℝ where Jl = ( 0 0

0 0ℝq 0

0 0 ) ∈ Mk (ℝ), J2p

with J2p = (

By a direct calculation, this condition is equivalent to t EJ2p E = aM J2p .

0 −Ip

Ip ). 0

120 | 4 The deformation space for nilpotent Lie groups Lemma 4.1.3. Let l be a non-Abelian Lie subalgebra of g, 1 K = (0 0

M1,q (ℝ) GLq (ℝ) 0

M1,2p (ℝ) Mq,2p (ℝ)) Sp(2p)

and

a { { Q = {(0 { { 0

0 Iq+p 0

0 } } 0 ) , a ∈ ℝ∗ } . } aIp }

Then Aut(l) = KQ. Here, p and q are given as in Proposition 1.1.12. Proof. Clearly, any matrix in K or Q is an automorphism of l. Conversely, let f be an element of Aut(l) and M its matrix written as in (4.2). Let us write D E=( 1 D3

D2 ) D4

and EM = (

D1 D3

a−1 D2 ), a−1 D4

then t EJ2p E = aJ2p if and only if EM ∈ Sp(2p) and 1 M = (0 0

c G 0

d′ a F ′ ) (0 EM 0

0 Ip+q 0

0 0 ), aIp

where for d = (d1 , . . . , d2p ), d′ = (d1 , . . . , dp , a−1 dp+1 , . . . , a−1 d2p ) and for F = (F1 , F2 ) with F1 , F2 in Mq,p (ℝ), F ′ = (F1 , a−1 F2 ). The condition [l, l] = [g, g] implies that any homomorphism from l to g stabilizes the center of g. The following lemma then is immediate. Lemma 4.1.4. If l is a non-Abelian Lie subalgebra of g. Then l contains the center z, and any homomorphism from l to g stabilizes z. We consider a basis Bh = {Z, X1′ . . . , Xn′ , Y1′ , . . . , Yn′ } of g adapted to h as in Proposition 1.1.12 with the following ordering: ′ ′ X1′ . . . , Xs′ , Z, Y1′ , . . . , Ys′ , Xs+1 , . . . , Xn′ , Ys+1 , . . . , Yn′ .

and for l we keep fixing the basis (4.1). As a direct consequence, the set Hom(l, g) is identified with a subset of matrices in M2n+1,k (ℝ) written through the bases Bl′ and Bh as 0 M = (a 0

M1 d ), M2

(4.3)

where a ∈ ℝ, d ∈ ℝk−1 , M1 ∈ Ms,k−1 (ℝ), M2 ∈ M2n−s,k−1 (ℝ). As a direct consequence of (4.3), we have the following. Lemma 4.1.5. For any α ∈ Is (2n + 1, k), the set Vα is empty whenever α ∈ ̸ Is1 (2n + 1, k).

4.1 Deformation and moduli spaces for Heisenberg groups | 121

Proof. From (4.3), for any M in Hom(l, g) and α ∈ ̸ Is1 (2n + 1, k), we get det(Mα ) = 0. Then our result is a direct consequence of (3.17). Lemma 4.1.6. For W = ηα (M) and α ∈ Is1 (2n + 1, k), the image of the group homomorphism cW,α is independent from the choice of W and α. More precisely, cW,α (G) equals the matrix group 1 c(G) := ( 0

M1,k−1 (ℝ) ). Ik−1

Proof. As a direct consequence of (3.25), we get for X ∈ g: 1 . X cW,α (e ) = ( . . 0

b(X, c2 ) ⋅ ⋅ ⋅ b(X, ck ) ), (I2p+q )

where the ci are the column of η−1 α (W). Furthermore, the column vectors c2 , . . . , ck are linearly independent and the center of g is not contained in the space generated by these vectors. This means that the linear forms on g, b(⋅, c2 ), . . . , b(⋅, ck ) are independent. Then the system b(X, c2 ) = x1 , . . . , b(X, ck ) = xk−1 has a solution for any (x1 , . . . , xk−1 ) ∈ ℝk−1 and the result follows. The following result is immediate. Lemma 4.1.7. The normalizer of c(G) in GLk (ℝ) is the parabolic subgroup P of matrices of the form a ( 0

b ), B

where a ∈ ℝ∗ , b ∈ ℝk−1 and B ∈ GLk−1 (ℝ). Lemma 4.1.8. Let α ∈ Is (2n + 1, k) and β ∈ I. For any (x, W) ∈ Vβ × η(Uα ), we have η−1 α (W)sβ (x) ∈ Vα only if sβ (x) ∈ P. In other words, cx,W,α (G) = c(G). In particular, c(G) is normal in Aut(l). Proof. By Lemma 4.1.7, the parabolic subgroup P coincides with the normalizer N(c(G)) and it is clear that Aut(l) is a subgroup of P. Now sβ (x) = (η−1 α (W)sβ (x))α −1 and η−1 (W)s (x) ∈ V only if η (W)s (x) is written as in (4.3). Then s (x) ∈ P. β α β β α α

122 | 4 The deformation space for nilpotent Lie groups Lemma 4.1.9. The topological group Aut(l)/c(G) is isomorphic to K ′ Q, where 1 K ′ = (0 0

0 GLq (ℝ) 0

0 Mq,2p (ℝ)) . Sp(2p)

In particular, Aut(l)/c(G) is homeomorphic to ℝ∗ × ℝ2pq × Sp(2p) × GLq (ℝ). Proof. Consider the projection p : Aut(l) → K ′ Q, where p(A) is the matrix obtained from A by vanishing the k − 1 last coefficients of the first line of A. Then p is a surjective homomorphism and its kernel is c(G). Now K ′ is normal in K ′ Q and K ′ ∩ Q is trivial; then K ′ Q is isomorphic to the semidirect product K × Q. The question of the related topology is straight clear. Let us go back now to the proof of Theorem 4.1.2. Take a family of continuous local sections (sβ )β∈I of the canonical surjection GLk (ℝ) → GLk (ℝ)/ Aut(l). From Theorem 3.4.8, the set Tαβ is homeomorphic to the quotient space (Aut(l) × Wαβ )/G. By Lemmas 4.1.6 and 4.1.8, we have cx,W,α (G) = c(G) and then the map i : Aut(l)/c(G) × Wαβ (A, x, W)

→

(Aut(l) × Wαβ )/G

󳨃→

(A, x, W)

is a well-defined bijection and the canonical surjection p1 : Aut(l) × Wαβ → (Aut(l) × Wαβ )/G factors through the canonical surjection p2 : Aut(l) × Wαβ → Aut(l)/c(G) × Wαβ and i. Now p1 and p2 are continuous and open; then i is bicontinuous. We now focus attention on the set Wαβ . By (3.18) and Lemma 3.4.1, we can write Wαβ = {(x, W) ∈ Vβ × η(Uα ) : η−1 α (W)sβ (x) ∈ Vα }.

(4.4)

Using Lemma 4.1.5, we can see that Wαβ is empty for α ∈ ̸ Is1 (2n + 1, k). Let us denote by asβ (x) the coefficient of the first line and column in the matrix sβ (x). Then η−1 α (W)sβ (x)(Z) = asβ (x) Z and we can state that

−1 Wαβ = {(x, W) ∈ Vβ × η(Uα ) | t sβ (x)t η−1 α (W)Jg ηα (W)sβ (x) = asβ (x) Jl }.

(4.5)

To complete the proof, we have to replace the sections (sβ )β by new sections satisfying Wαβ is empty or asβ (x) = 1 for all (x, W) ∈ Wαβ . For i = 1, . . . , k, consider the open covering of GLk (ℝ), Gi = {A = (aij ) ∈ GLk (ℝ), ai1 ≠ 0}. Then all the Gi ’s are Aut(l)-stable and we can actually replace the sections (sβ )β and the covering (Vβ )β , by the covering (Vβ,i )β,i = (Vβ ∩Gi )β,i and the sections (sβ,i )β,i = (sβ|Vβ,i )β,i .

4.1 Deformation and moduli spaces for Heisenberg groups | 123

For i ≠ 1, sβ,i (x) ∈ ̸ P for all x ∈ Vβ,i and then Wα,β,i is empty by Lemma 4.1.8. For i = 1, let G1 = {A = (aij ) ∈ GLk (ℝ), a11 = 1} and consider the map δβ : sβ (Vβ,1 ) sβ (x)

→

G1

󳨃→

sβ (x)Q(asβ (x) ),

where a−1 sβ (x)

Q(asβ (x) ) = ( 0 0

0

Iq+p 0

0

0 ). a−1 sβ (x) Ip

Lemma 4.1.10. The map δβ is a homeomorphism from sβ (Vβ,1 ) on its image. Proof. Clearly, δβ is continuous. Note that sβ (x) and its image have the same coset class modulo Aut(l). Then δβ is injective and the map δβ′ : δβ (sβ (Vβ,1 )) → sβ (Vβ,1 ) given by δβ′ (M) = sβ (p(M)) is well-defined, continuous and turns out to be the inverse of δβ . We finally end up with new family of continuous local sections (s′β,i )β,i given by δβ ∘ sβ,1

s′β,i = {

sβ,i

if i = 1, if i ≠ 1

defined on the open sets Vβ,i such that as′ (x) = 1. This completes the proof of the β,i theorem.

4.1.3 Deformation and moduli spaces when H contains the center We assume in this section that the subalgebra h of g contains the center z = [g, g] and that l is a subalgebra of g such that l ∩ h = {0}. Then l is an Abelian subalgebra and if L (l, g) designates the vector space of the linear maps from l to g the set Hom(l, g) of Lie algebras homomorphisms can be regarded as the set Hom(l, g) := {ψ ∈ L (l, g), [ψ(x), ψ(y)] = 0 for all x, y ∈ l}. We fix by the way a symplectic basis Bh of g adapted to h as provided by Proposition 1.1.12. We identify g to ℝ2n+1 , h to a subspace of ℝ2n+1 and L (l, g) to a subset of

124 | 4 The deformation space for nilpotent Lie groups real matrices M2n+1,k (ℝ), where k = dim l. Let as usual s = dim h. For any α ∈ Is (n, k), we consider the set 0 A

t

Vα := {M = ( ) , A ∈ M2n,k (ℝ), Mα = Ik and MJb M = 0} ⊂ Vα , ′

(4.6)

where Jb is the matrix of b in Bh . The following theorem provides a description of the deformation and the moduli space in this context. Theorem 4.1.11. Let G be the Heisenberg Lie group of dimension 2n + 1, H a connected Lie subgroup of dimension s, which contains the center of G and Γ a discontinuous group for G/H of rank k. Then T (Γ, G, H) =

⋃

α∈Is (2n+1,k)

Tα

and

M (Γ, G, H) =

⋃

α∈Is (2n+1,k)

Mα ,

where for every α ∈ Is (2n+1, k), the set Tα is an open subset of T (Γ, G, H) homeomorphic to the product GLk (ℝ) × Vα′ and Mα is an open subset of M (Γ, G, H) homeomorphic to the product GLk (ℝ)/GLk (ℤ) × Vα′ . Proof. We will make use of Theorem 3.3.1 as h contains the first derivative group of g. As it stands there, we just have to prove that the quotient space Vα /G is homeomorphic to Vα′ for any α ∈ Is (2n + 1, k). We first fix all of the symplectic basis Bh = (Z, X1 , . . . , Xn , Y1 , . . . , Yn ) adapted to h. Take any α ∈ Is (2n + 1, k) and M ∈ Vα , we can then write a M=( ) A

with a = (a1 , . . . , ak ) ∈ ℝk and A ∈ M2n,k (ℝ).

Note that for X ∈ g we have

Adexp X

1 . = (. . 0

b(X, X1 ) ⋅ ⋅ ⋅ b(X, Yn ) ). (I2n )

Then the action of Adexp X on M affects only the first line. More precisely, if we identify the columns ci of A to a vector of g, then the first line of the product is (a1 + b(X, c1 ), . . . , ak + b(X, ck )). The following result, the subject of Lemma 3.3.5, will next be used. Let WM denote the subspace of g generated by the columns of M. If M ∈ Vα for α ∈ Is (2n + 1, k), then

4.1 Deformation and moduli spaces for Heisenberg groups | 125

∗ WM ∩ h = {0}. It turns out as the center z is not contained in WM that the map g → WM , x 󳨃→ b(x, .) is surjective and there exists therefore X ∈ g such that

b(X, c1 ) = −a1 , . . . , b(X, ck ) = −ak . It follows then that any M in Vα is G-equivalent to the matrix obtained from M by vanishing the first line of M. Conversely, if the first lines of M and M ′ are zero, then M = Adexp X M ′ only if M = M ′ . Let π be the continuous map from Vα to Vα′ , which sends the matrix a M=( ) A to the matrix 0 π(M) = ( ) , A where we consider the trace topology on Vα′ . Then the canonical surjection p : Vα → Vα /G factors through π to a continuous bijection f between Vα′ and Vα /G defined by p = f ∘ π. Now G acts continuously on Vα , then p is open and we can easily see that f −1 is continuous. This achieves the proof of the theorem.

4.1.4 The case when H does not meet the center We now tackle the case where the center of g does not meet h. In such a situation, H is an Abelian subgroup of G. We still need some other results. The following lemma describes the structure of the parameter space in this case. Lemma 4.1.12. Let G be the Heisenberg Lie group, H a connected subgroup, which does not contain the center, Γ a rank k Abelian discontinuous group for G/H and L = exp(l) its syndetic hull in G. Then the parameter space R (Γ, G, H) is the disjoint union of the two G-invariant sets { { { R1 (Γ, G, H) = {ψ ∈ Hom(l, g) { { {

󵄨󵄨 󵄨󵄨 dim ψ(l) = k, } } 󵄨󵄨 } 󵄨󵄨 󵄨󵄨 h ∩ ψ(l) = {0} } 󵄨󵄨 } } 󵄨󵄨 󵄨󵄨 and z ⊂ ψ(l) }

and 󵄨󵄨 󵄨󵄨 dim ψ(l) = k and R2 (Γ, G, H) = {ψ ∈ Hom(l, g) 󵄨󵄨󵄨 }. 󵄨󵄨 (h ⊕ z) ∩ ψ(l) = {0} 󵄨

126 | 4 The deformation space for nilpotent Lie groups Proof. From Theorem 3.2.4 and Lemma 4.1.1, we can easily see that R (Γ, G, H) is the union of the following sets: { { { { { { R1 (Γ, G, H) = {ψ ∈ Hom(l, g) { { { { { {

󵄨󵄨 dim ψ(l) = k, 󵄨󵄨 } } 󵄨󵄨 } } 󵄨󵄨 } } 󵄨󵄨 h ∩ ψ(l) = {0}, 󵄨󵄨󵄨 󵄨󵄨 z ∩ (h ⊕ ψ(l)) = ψ(l) ∩ z } } } } 󵄨󵄨󵄨 } } 󵄨󵄨 󵄨󵄨 and z ⊂ ψ(l) }

and 󵄨󵄨 dim ψ(l) = k, 󵄨󵄨 } } 󵄨󵄨 } } 󵄨󵄨 } } 󵄨󵄨 h ∩ ψ(l) = {0}, 󵄨󵄨 󵄨󵄨 } 󵄨󵄨 z ∩ (h ⊕ ψ(l)) = ψ(l) ∩ z } } } 󵄨󵄨 } } 󵄨󵄨 󵄨󵄨 and z ⊄ ψ(l) } 󵄨󵄨 󵄨󵄨 dim ψ(l) = k, { } { } 󵄨󵄨 { } 󵄨󵄨 = {ψ ∈ Hom(l, g) 󵄨󵄨 h ∩ ψ(l) = {0}, . } 󵄨 { } 󵄨󵄨 { 󵄨󵄨 z ∩ (h ⊕ ψ(l)) = {0} } 󵄨 { }

{ { { { { { R2 (Γ, G, H) = {ψ ∈ Hom(l, g) { { { { { {

To conclude, note that the third condition z ∩ (h ⊕ ψ(l)) = ψ(l) ∩ z involved in the set R1 (Γ, G, H) is trivial as z ⊂ ψ(l). Likewise, it is easily seen that z ∩ (h ⊕ ψ(l)) = {0} if and only if (z ⊕ h) ∩ ψ(l) = {0}, and then the three last set equations of R2 (Γ, G, H) together are equivalent to (h ⊕ z) ∩ ψ(l) = {0}. On the other hand, for any g ∈ G, z ⊂ Adg −1 ∘ψ(l) if and only if z ⊂ ψ(l), which proves the G-invariance of R1 (Γ, G, H). Furthermore, for any ψ ∈ R2 (Γ, G, H) and any g ∈ G, one has (z ⊕ h) ∩ Adg −1 (ψ(l)) = (z ⊕ h) ∩ (ψ(l)) = {0}, which shows the G-invariance of the set R2 (Γ, G, H). We now fix a basis Bh = {Z, X1 . . . , Xn , Y1 , . . . , Yn } of g adapted to h. We consider the decomposition g = z ⊕ h ⊕ h′ ⊕ k ⊕ k′ , where h = ⟨X1 , . . . , Xs ⟩,

h′ = ⟨Y1 , . . . , Ys ⟩,

k = ⟨Xs+1 , . . . , Xn ⟩ and

k′ = ⟨Ys+1 , . . . , Yn ⟩.

We identify as previously g to ℝ2n+1 = ℝ⊕ℝs ⊕ℝs ⊕ℝn−s ⊕ℝn−s and Hom(l, g) to the set of matrices given in (4.3), with l = 1 and b1 = b. Then with respect to this decomposition,

4.1 Deformation and moduli spaces for Heisenberg groups | 127

any element of g, n

n

i=1

i=1

x = a0 Z + ∑ ai Xi + ∑ bi Yi , is identified to the column vector t

(a0 a1 ⋅ ⋅ ⋅ as b1 ⋅ ⋅ ⋅ bs as+1 ⋅ ⋅ ⋅ an bs+1 ⋅ ⋅ ⋅ bn )

and every homomorphism ψ ∈ Hom(l, g), can be written as a matrix A0 A1 M = ( B1 ) , A2 B2 where A0 ∈ M1,k (ℝ), A1 , B1 ∈ Ms,k (ℝ) and A2 , B2 ∈ Mn−s,k (ℝ). Then from Lemma (4.1.12), we get { { { { { { R1 (Γ, G, H) = {M ∈ M2n+1,k (ℝ) { { { { { {

󵄨󵄨 dim M(ℝk ) = k, 󵄨󵄨 } 󵄨󵄨 } } 󵄨󵄨 } 󵄨󵄨 h ∩ M(ℝk ) = {0}, } } 󵄨󵄨 . 󵄨󵄨 } 󵄨󵄨 z ⊂ M(ℝk ) and } } } 󵄨󵄨 } } 󵄨󵄨 t 󵄨󵄨 MJ M = 0 } 󵄨 b

Up to this step, we consider the set Is1 (2n + 1, k) = {(i1 , . . . , ik ), i1 = 1 and i2 > s + 1}.

(4.7)

Now we can state the following. Lemma 4.1.13. The set R1 (Γ, G, H) is open in Hom(l, g) and the sets η−1 (ηα (Vα )), α ∈ Is1 (2n + 1, k) constitutes an open G-invariant covering of R1 (Γ, G, H). Proof. The condition z ⊂ M(ℝk ) equivalent to the existence of a matrix 1 0 M ′ = (0 0 0

0 A′1 B′1 ) , A′2 B′2

128 | 4 The deformation space for nilpotent Lie groups with A′1 , B′1 ∈ Ms,k−1 (ℝ), A′2 , B′2 ∈ Mn−s,k−1 (ℝ) and M(ℝk ) = M ′ (ℝk ). The conditions M(ℝk ) ∩ h = {0} and dim M(ℝk ) = k are equivalent to B′1 rank (A′2 ) = k − 1, B′2 which is also equivalent to the existence of α ∈ Is1 (2n + 1, k), such that M(ℝk ) ∈ ηα (Uα ). Now, if M(ℝk ) ∈ ηα (Uα ), then t MJb M = 0 if and only if t

k −1 k {η−1 α (M(ℝ ))}Jb {ηα (M(ℝ ))} = 0,

k or equivalently η−1 α (M(ℝ )) ∈ Vα . Then M ∈ R1 (Γ, G, H) if and only if

M(ℝk ) ∈

⋃

ηα (Vα ).

⋃

η−1 (ηα (Vα )).

α∈Is1 (2n+1,k)

This means that R1 (Γ, G, H) =

α∈Is1 (2n+1,k)

Furthermore, 󵄨󵄨 󵄨󵄨 (det Mα ) ≠ 0 η−1 (ηα (Vα )) = {M ∈ M2n+1,k (ℝ) 󵄨󵄨󵄨 }, 󵄨󵄨 and t MJ M = 0 b 󵄨 which is an open set of V , and then R1 (Γ, G, H) is also open in Hom(l, g). Let X ∈ g and M ∈ η−1 (ηα (Vα )). Then there exist A ∈ GLk (ℝ) and M ′ ∈ M2n+1,k (ℝ) such that

Adexp X

1 . = (. . 0

b(X, X1 ) ⋅ ⋅ ⋅ b(X, Yn ) ) (I2n )

and M = M ′ A, with 1 0 M ′ = (0 0 0

0 A′1 B′1 ) . A′2 B′2

4.1 Deformation and moduli spaces for Heisenberg groups | 129

Therefore, 1 . Adexp X M = M ′ ( . . 0

b(X, c2 ) ⋅ ⋅ ⋅ b(X, cn ) ) A, (Ik−1 )

where c2 , . . . , ck are the k − 1 last columns vectors of M ′ and we can see that η(M) = η(Adexp X M), which proves the G-invariance of η−1 (ηα (Vα )) for any α ∈ Is1 (2n + 1, k). Now we are ready to state our main result in this section concerning the deformation and the moduli space in the case where h does not meet the center of g. We have the following. Theorem 4.1.14. Let G be the Heisenberg Lie group, H a connected subgroup, which does not meet the center of G, Γ a rank k Abelian discontinuous group for G/H and L = exp(l) its syndetic hull in G. Then T (Γ, G, H) =

⋃

α∈Is+1 (2n+1,k)

Tα

⋃

Tα

⋃

Mα ,

α∈Is1 (2n+1,k)

and M (Γ, G, H) =

⋃

α∈Is+1 (2n+1,k)

Mα

α∈Is1 (2n+1,k)

where (1) For every α ∈ Is+1 (2n + 1, k), the set Tα is open in T (Γ, G, H) and homeomorphic to the product GLk (ℝ) × Vα′ and the set Mα is open in M (Γ, G, H) and homeomorphic to GLk (ℤ)\GLk (ℝ) × Vα′ . (2) For every α ∈ Is1 (2n + 1, k), the set Tα is open in T (Γ, G, H) and is homeomorphic to the product Ok × ℝk × Nk × Vα . Nk designates here the set of upper triangular unipotent matrices. Likewise, the set Mα is open in M (Γ, G, H) and homeomorphic to the product (GLk (ℤ)\GLk (ℝ)/ℝk−1 ) × Vα . Proof. We use Lemma 4.1.12 to write the following decomposition of the deformation space: R1 (Γ, G, H)/G ∪ R2 (Γ, G, H)/G. The set R2 (Γ, G, H) can be identified to the parameter space R(Γ, G, K), where K = Z(G)H and Z(G) is the center of G. Then by Theorem 4.1.11, we get the following

130 | 4 The deformation space for nilpotent Lie groups description of the quotient set: R2 (Γ, G, H)/G =

⋃

α∈Is+1 (2n+1,k)

Tα ,

where for any α ∈ Is+1 (2n + 1, k), the set Tα is open in T (Γ, G, H) and homeomorphic to the product GLk (ℝ) × Vα′ . On the other hand, thanks to Lemma 4.1.13, one can write R1 (Γ, G, H)/G =

⋃

α∈Is1 (2n+1,k)

η−1 (ηα (Vα ))/G

as union of open sets. Let then Tα = η−1 (ηα (Vα ))/G. Recall that the map χα is a homeomorphism between GLk (ℝ)×ηα (Vα ) and η−1 (ηα (Vα )). Consider the G-action on GLk (ℝ)× ηα (Vα ) given by −1 (A, W) ⋅ g = (χα,W Adg −1 χα,W A, W).

Then the map χα is G-equivariant. Indeed, −1 χα ((A, W) ⋅ g) = χα (χα,W ∘ Adg −1 ∘χα,W ∘ A, W) −1 = χα,W ∘ (χα,W ∘ Adg −1 ∘χα,W ∘ A)

= Adg −1 ∘χα (A, W) = χα (A, W) ⋅ g.

For every α ∈ Is1 (2n + 1, k) and W ∈ ηα (Vα ), we can easily see that there is A1 , B1 ∈ Ms,k (ℝ), A2 , B2 ∈ Mn−s,k (ℝ) such that 1 0 η−1 α (W) = (0 0 0

0 A1 B1 ) . A2 B2

Then, for g −1 = exp X we have 1 . −1 χα,W Adg −1 χα,W A = (Adg −1 η−1 (W)A) = ( . α α . 0

b(X, c2 ) ⋅ ⋅ ⋅ b(X, cn ) ) A, (Ik−1 )

4.1 Deformation and moduli spaces for Heisenberg groups | 131

where c2 , . . . , ck are the k − 1 last columns of η−1 α (W). We now consider the free action of ℝk−1 on GLk (ℝ) defined by 1 . (x1 , . . . , xk−1 ) ⋅ A = ( . . 0

x1 ⋅ ⋅ ⋅ xk−1 ) A. (Ik−1 )

The subspace W ′ of W generated by c2 , . . . , ck is an Abelian subalgebra of dimension ∗ k −1, that does not meet the center, which means that the map g → W ′ , X 󳨃→ b(X, ⋅)|W ′ is surjective. It follows therefore that for any (x1 , . . . , xk−1 ) ∈ ℝk−1 there is X ∈ g such that b(X, ci ) = xi−1 for all i = 2, . . . , k. Therefore, the quotient map π : GLk (ℝ) × ηα (Vα ) 󳨀→ (GLk (ℝ) × ηα (Vα ))/G, factors through the canonical surjection p : GLk (ℝ) × ηα (Vα ) 󳨀→ (GLk (ℝ)/ℝk−1 ) × ηα (Vα ) to give a continuous surjective map f : (GLk (ℝ)/ℝk−1 ) × ηα (Vα ) 󳨀→ (GLk (ℝ) × ηα (Vα ))/G defined by π = f ∘ p and we can easily see that f is injective. Now G acts continuously on GLk (ℝ) × ηα (Vα ), which entails that π is open and that f is a homeomorphism. The following lemma enables us to achieve the proof of the assertion concerning the deformation space. Lemma 4.1.15. Fix a positive integer p and regard ℝp as a subgroup of GLp+1 (ℝ) through the writing Rp := {(

1 0

t

x ) : x ∈ ℝp } . Ip

Then GLp+1 (ℝ)/ℝp ≃ Op+1 × ℝp+1 × Np , where Np denotes the totality of upper triangular unipotent matrices. Proof. Using the Iwasawa decomposition, we have GLp+1 (ℝ) ≃ Op+1 × Ap+1 × Np+1 ,

(4.8)

132 | 4 The deformation space for nilpotent Lie groups where Ap+1 (≃ ℝp+1 ) denotes the totality of diagonal matrices with positive entries. Thus, we obtain (4.8) because of the decomposition Np+1 ≃ Np × Rp . As for the moduli space, recall that Aut(Γ) = GLk (ℤ) and if we consider the action of Aut(Γ) on GLk (ℝ) × ηα (Vα ) given by T ⋅ (A, W) = (AT −1 , W), then χα is Aut(Γ)equivariant and the result follows immediately. The following result provides accurate layering of the deformation space in the context. Theorem 4.1.16. Let G be the Heisenberg Lie group, H a connected subgroup, which does not meet the center of G, Γ a rank k Abelian discontinuous group for G/H and L = exp(l) its syndetic hull in G. Then T (Γ, G, H) =

⋃

α∈Is+1 (2n+1,k)

Tα

k

⋃ Tα,j ,

⋃

α∈Is1 (2n+1,k) j=1

where for every α ∈ Is+1 (2n + 1, k), the set Tα is open in T (Γ, G, H) and homeomorphic to the product GLk (ℝ) × Vα′ . Furthermore, for any α ∈ Is1 (2n + 1, k) and j ∈ {1, . . . , k}, the set Tα,j is open in T (Γ, G, H) and is homeomorphic to the multiple direct product ℝ∗ × ℝk−1 × GLk−1 (ℝ) × Vα . Proof. We only need to show that k

GLk (ℝ)/ℝk−1 = ⋃ Ui , j=1

where for any j = 1, . . . , k, Uj is homeomorphic to ℝ∗ × ℝk−1 × GLk−1 (ℝ). Indeed, let A ∈ GLk (ℝ) and denote by Ai the matrix obtained from A by deleting the first line and the ith column of A. Then the union of the open sets Ui = {A ∈ GLk (ℝ), det Ai ≠ 0},

1≤i≤k

is equal to GLk (ℝ) and each of them is ℝk−1 -stable. Therefore, k

GLk (ℝ)/ℝk−1 = ⋃ Ui /ℝk−1 . i=1

The following lemma enables us to achieve the proof. Lemma 4.1.17. For 1 ≤ i ≤ k, we have Ui /ℝk−1 ≅ ℝ∗ × ℝk−1 × GLk−1 (ℝ). Proof. Let A ∈ Ui , write a A = ( 11 (a1 )

⋅⋅⋅ ⋅⋅⋅

a1k ), (ak )

4.1 Deformation and moduli spaces for Heisenberg groups | 133

where a11 , . . . , a1k ∈ ℝ and a1 , . . . , ak ∈ ℝk−1 . So for x ∈ ℝk−1 we have a11 + ⟨x, a1 ⟩ (a1 )

x⋅A=(

⋅⋅⋅ ⋅⋅⋅

a1k + ⟨x, ak ⟩ ), (ak )

where ⟨⋅, ⋅⟩ designates the natural scalar product on ℝk−1 . For x0 = −bi A−1 i , where bi ∈ k−1 ℝ obtained from (a11 , . . . , a1k ) by eliminating of the ith coordinate, we have a1j + ⟨x0 , aj ⟩ = 0

for all j ≠ i.

This means that A is equivalent (modulo ℝk−1 ) to a certain matrix in the set a Ki := {A = ( 11 (a1 )

⋅⋅⋅ ⋅⋅⋅

a1k ) ∈ GLk (ℝ), a1j = 0 for all j ≠ i} . (ak )

Note that Ki ≅ ℝ∗ × ℝk−1 × GLk−1 (ℝ). Let π : Ui 󳨀→ Ui /ℝk−1 be the canonical surjection and p : Ui 󳨀→ Ki the continuous surjection defined by p (A) p(A) = ( 1 (a1 )

⋅⋅⋅ ⋅⋅⋅

pk (A) ), (ak )

where pj (A) = 0, if j ≠ i and pi (A) = a1i − ⟨bi A−1 i , ai ⟩ ≠ 0. Then clearly the map f : Ki 󳨀→ Ui /ℝk−1 defined by f (p(A)) = π(A) is surjective. For the injectivity, let A, A′ ∈ Ki such that f (A) = f (A′ ), which means that there is x0 ∈ ℝk−1 such that x0 ⋅ A = A′ . But x ⋅ A ∈ Ki only if x = 0. Thus, x0 = 0 and A = A′ . Using the continuity of π and p with the fact that π is open, we obtain the bicontinuity of f . This achieves the proof of the lemma and also of the theorem. 4.1.5 Case of compact Clifford–Klein forms We finally describe the deformation and the moduli space for compact Clifford–Klein forms. We have the following. Theorem 4.1.18. Let G be the Heisenberg Lie group of dimension 2n + 1, H a connected Lie subgroup of dimension s and Γ a rank k discontinuous group for G/H. Assume in addition that the Clifford–Klein form Γ\G/H is compact. Then:

134 | 4 The deformation space for nilpotent Lie groups (1) If H contains the center of G, then k < s and 2

q

T (Γ, G, H) = GLk (ℝ) × Mp,q (ℝ) × Sym(ℝ ) × Sp(p, ℝ)/ Sp(p − r, ℝ)

and equivalently 2

q

M (Γ, G, H) = GLk (ℝ)/GLk (ℤ) × Mp,q (ℝ) × Sym(ℝ ) × Sp(p, ℝ)/ Sp(p − r, ℝ),

where for h = log H, q + 1 = dim(ker b|h ), 2p + q + 1 = dim h and p + q + r = n. (2) If H does not contain the center of G then n+1

T (Γ, G, H) = On+1 × ℝ

× Nn × Sym(ℝn )

and n

n

M (Γ, G, H) = (GLn+1 (ℤ)\GLn+1 (ℝ)/ℝ ) × Sym(ℝ ).

Proof. Note first of all that if Γ is a discontinuous group for G/H and H contains the center of G, then Γ is Abelian and so is its syndetic hull L. By Proposition 1.1.12, we get k < n + 1 where k designates the rank of Γ. If k = 2n + 1 − s and k > s, then obviously k > n, which means that either Γ is not a discontinuous group for G/H or Γ is not Abelian. So, if k > s and 2n + 1 − s = k, then the parameters space is empty. Assume now that k < s and 2n + 1 − s = k then the set Is (2n + 1, k) is reduced to the element α0 = (s + 1, . . . , 2n + 1). Using Theorem 4.1.11, we get T (Γ, G, H) = GLk (ℝ) × Vα0 . ′

To conclude, we just have to prove that 2

q

Vα0 = Mp,q (ℝ) × Sym(ℝ ) × Sp(p, ℝ)/ Sp(p − r, ℝ). ′

Having fixed an adapted basis Bh = {Z, X1 . . . , Xn , Y1 , . . . , Yn } of g adapted to h, we consider the vector subspaces: V1′ = ℝ-span{X1 , . . . , Xp },

V1′′ = ℝ-span{Y1 , . . . , Yp },

V0 = ℝ-span{Xp+1 , . . . , Xp+q }, N1′

= ℝ-span{Xp+q+1 , . . . , Xn }

N0 = ℝ-span{Yp+1 , . . . , Yp+q },

and N1′′ = ℝ-span{Yp+q+1 , . . . , Yn }.

So we have the following decompositions: g = z ⊕ V1′ ⊕ V1′′ ⊕ V0 ⊕ N0 ⊕ N1′ ⊕ N1′′

and

h = z ⊕ V1′ ⊕ V1′′ ⊕ V0 .

(4.9)

4.1 Deformation and moduli spaces for Heisenberg groups | 135

Any matrix M ∈ Vα′ can be written as 0 A1 (B1 ( M=( ( C1 (I 0 (0

0 A2 B2 C2 0 I 0

0 A3 B3 ) ) C3 ) ) 0) 0 I)

z V1′ V1′′ V0 , N0 N1′ N1′′

where A1 , B1 ∈ Mp,q (ℝ), C1 ∈ Mq,q (ℝ), A2 , A3 , B2 , B3 ∈ Mp,r (ℝ) and C2 , C3 ∈ Mq,r (ℝ), for r = n − p − q. The matrix of b is 0 0 (0 ( Jb = ( (0 (0 0 (0

0 0 I 0 0 0 0

0 −I 0 0 0 0 0

0 0 0 0 I 0 0

0 0 0 −I 0 0 0

0 0 0 0 0 0 I

0 0 0) ) 0) ) 0) −I 0)

and the condition t MJb M = 0 is equivalent to the following system: t

B1 A1 − t A1 B1 + C1 − t C1 ( t B2 A1 − t A2 B1 − t C2 t B3 A1 − t A3 B1 − t C3

t

B1 A2 − t A1 B2 + C2 t B2 A2 − t A2 B2 t B3 A2 − t A3 B2 + I

t

B1 A3 − t A1 B3 + C3 B2 A3 − t A2 B3 − I ) = 0. t B3 A3 − t A3 B3

t

This is in turn equivalent to C2 = t A1 B2 − t B1 A2 ,

C3 = t A1 B3 − t B1 A3 ,

1 C1 = (t A1 B1 − t B1 A1 ) + D 2

and t

(t

B2 A2 − t A2 B2 B3 A2 − t A3 B2

t t

B2 A3 − t A2 B3 0 )=( B3 A3 − t A3 B3 −I

where D ∈ Sym(ℝq ) and A1 , B1 ∈ Mp,q (ℝ). Let B2 A2

Y =(

B3 ) ∈ M2p,2r (ℝ) A3

and 0 Jm = ( −Im

Im ) ∈ M2m,2m (ℝ). 0

I ), 0

(4.10)

136 | 4 The deformation space for nilpotent Lie groups Then the condition (4.10) can be written as t YJp Y = Jr and for U = {Y ∈ M2p,2r (ℝ), t YJp Y = Jr } we easily see that 2

q

Vα ≅ Mp,q (ℝ) × Sym(ℝ ) × U. ′

To conclude, we finally prove the following lemma. Lemma 4.1.19. U ≅ Sp(p, ℝ)/ Sp(p − r, ℝ). Proof. Note first that the symplectic group Sp(p, ℝ) acts on U by multiplication on the left and its action is transitive. The matrix Ir 0 Y =( 0 0

0 0 ) Ir 0

belongs to U and with a direct verification, we get Ir { { { { 0 Stab(Y) = {P = ( { 0 { { 0 {

0 A 0 C

0 0 Ir 0

0 B A ), ( 0 C D

} } } } B ) ∈ Sp(p − r)} ≅ Sp(p − r). } D } } }

We pay attention finally to the case where H does not meet Z(G), the center of G. As we are dealing with compact Clifford–Klein forms, we are obviously submitted to write that s + k = 2n + 1, which entails that Is+1 (2n + 1, k) is empty and Is1 (2n + 1, k) is merely reduced to the single element α0 = (1, s + 2, . . . , 2n + 1). As it stands here, the subgroups H and Γ are Abelian, and we get by Proposition 1.1.12 that dim h = n and rank Γ = n + 1. Then Theorem 4.1.14 enables us to write that n

T (Γ, G, H) = GLn+1 (ℝ)/ℝ × Vα0 .

Now every matrix M in Vα0 can be written as 1 M = (0 0

0 A) In

for some A ∈ Mn (ℝ). The relation t MJb M = 0 is then equivalent to t A − A = 0 and the result follows from Lemma 4.1.15. A straight consequence of the last theorem, is the following.

4.1 Deformation and moduli spaces for Heisenberg groups | 137

Corollary 4.1.20. Let G be the Heisenberg Lie group, H a connected Lie subgroup of G and Γ an Abelian discontinuous group of G for G/H. Assume in addition that the Clifford– Klein form Γ\G/H is compact. Then the deformation space T (Γ, G, H) is endowed with a structure of a smooth manifold.

4.1.6 Examples To end this section, we present some enriching examples for which we carry out explicit computations of some chosen layers Tα and Mα involved in the description of the deformation and moduli space as we did in the case of compact Clifford–Klein forms where only one single strate occurs. We precise that our computations take into account the precise basis of g adapted to h and utterly rely on the position of h inside g. All of the matrices considered in the following examples are written in a basis Bh of g adapted to h. Example 4.1.21. We assume in this first example that h does not contain the center of g and that dim h = s according to Proposition 1.1.12. Take for instance k = s + 1 and α = (1, s + 2, . . . , 2s + 1). Then any M ∈ Vα can be written as 1 0 M = (0 0 0

0 A Is ) , B C

where A ∈ Ms (ℝ), B, C ∈ Mn−s,s (ℝ). The matrix Jb of the bilinear form b is then given by 0 0 Jb = (0 0 0

0 0 Is 0 0

0 −Is 0 0 0

0 0 0 0

In−s

0 0 0 ). −In−s 0

By a routine computation, we can easily see that the condition t MJb M = 0 gives rise to the following equation: −t A + A − t BC + t CB = 0.

138 | 4 The deformation space for nilpotent Lie groups So A = − 21 (−t BC + t CB) + D, for some D ∈ Sym(ℝs ), and we finally get that k−1

Tα = GLk (ℝ)/ℝ

2 × Mn−s,s (ℝ) × Sym(ℝs )

2 ≃ Ok × ℝk × Nk × Mn−s,s (ℝ) × Sym(ℝs )

and k−1

Mα = GLk (ℤ)\GLk (ℝ)/ℝ

2 × Mn−s,s (ℝ) × Sym(ℝs ).

We assume henceforth that h contains the center of g and, therefore, dim h = 1 + 2p + q according to the notation of Proposition 1.1.12. Example 4.1.22. Take for instance p + q + k = n and α = (1 + 2p + 2q + k + 1, . . . , 2n + 1). Let M ∈ Vα′ . Then 0 A1 ( A2 ) ( ) ) M=( ( A3 ) , (A ) 4 A5 ( Ik ) where A1 , A2 ∈ Mp,k (ℝ), A3 , A4 ∈ Mq,k (ℝ) and A5 ∈ Mk (ℝ). The matrix of the bilinear form b is 0 0 (0 ( Jb = ( (0 (0 0 0 (

0 0 Ip 0 0 0 0

0 −Ip 0 0 0 0 0

0 0 0 0 Iq 0 0

0 0 0 −Iq 0 0 0

0 0 0 0 0 0 Ik

0 0 0) ) 0) ). 0) −Ik 0)

So, the condition t MJb M = 0 is equivalent to the equation −t A1 A2 + t A2 A1 − t A3 A4 + t A4 A3 + A5 − t A5 = 0. Therefore, for A1 , A2 ∈ Mp,k (ℝ), A3 , A4 ∈ Mq,k (ℝ) we can take 1 A5 = − (−At1 A2 + t A2 A1 − t A3 A4 + t A4 A3 ) + D, 2

with D ∈ Sym(ℝk ).

4.1 Deformation and moduli spaces for Heisenberg groups | 139

and then 2

2

k

Tα ≅ GLk (ℝ) × Mp,k (ℝ) × Mq,k (ℝ) × Sym(ℝ ).

Example 4.1.23. We still take p+q+k = n and consider α = (2p+2q+2, . . . , 2p+2q+k+1) and let M ∈ Vα′ . Then 0 A1 ( A2 ) ( ) ) M=( ( A3 ) , (A ) 4 Ik (A5 ) where A1 , A2 ∈ Mp,k (ℝ), A3 , A4 ∈ Mq,k (ℝ) and A5 ∈ Mk (ℝ). Then the same calculation as in the first example gives 2

2

k

Tα ≅ GLk (ℝ) × Mp,k (ℝ) × Mq,k (ℝ) × Sym(ℝ ).

Example 4.1.24. Assume now that k = q and take α = (2p + q + 2, . . . , 2p + 2q + 1) and let M ∈ Vα′ . Then 0 A1 ( A2 ) ( ) ) M=( ( A3 ) , (I ) q

A5 A ( 6)

where A1 , A2 ∈ Mp,q (ℝ), A3 ∈ Mq (ℝ) and A5 , A6 ∈ Mr,q (ℝ) with r = n − p − q. So, the condition t MJb M = 0 is equivalent to the equation −t A1 A2 + t A2 A1 − t A3 + A3 − t A5 A6 + t A6 A5 = 0. Then as above we have 1 A3 = − (−t A1 A2 + t A2 A1 − t A5 A6 + t A6 A5 ) + D, 2

with D ∈ Sym(ℝq ).

We get then that 2

2

q

Tα ≅ GLq (ℝ) × Mp,q (ℝ) × Mr,q (ℝ) × Sym(ℝ ).

140 | 4 The deformation space for nilpotent Lie groups Example 4.1.25. Assume finally that k = q + r where p + q + r = n and take α = (2p + q + 2, . . . , 2p + 2q + r + 1). For M ∈ Vα′ , we have 0 A1 ( A2 ( M=( (A3 (I q 0 (A4

0 B1 B2 ) ) B3 ) ), 0) Ir B4 )

where A1 , A2 ∈ Mp,q (ℝ), B1 , B2 ∈ Mp,r (ℝ), A3 ∈ Mq (ℝ), B3 ∈ Mq,r (ℝ), A4 ∈ Mr,q (ℝ) and B4 ∈ Mr (ℝ). Then t

−t A1 A2 + t A2 A1 − t A3 + A3 −t B1 A2 + t B2 A1 − t B3 − A4

MJb M = (

−t A1 B2 + t A2 B1 + B3 + t A4 ). −t B1 B2 + t B2 B1 + t B4 − B4

Then the condition t MJb M = 0 is equivalent to 1 A3 = − (−t A1 A2 + t A2 A1 ) + D, D ∈ Sym(ℝq ), 2 1 B4 = − (t B1 B2 − t B2 B1 ) + D′ , D′ ∈ Sym(ℝr ), 2 −A4 = t B3 + t B1 A2 − t B2 A1 . We obtain therefore that 2

2

q

r

Tα = GLk (ℝ) × Mp,q (ℝ) × Mp,r (ℝ) × Mq,r (ℝ) × Sym(ℝ ) × Sym(ℝ ).

4.1.7 A smooth manifold structure on T (Γ, H2n+1 , H) Recall the matrix Jb of the bilinear form b written through the basis B . Having fixed a basis Bl of l, it appears clear that the map Ψ : Hom(l, g) 󳨀→ M2n+1,k (ℝ),

(4.11)

which associates to any element of Hom(l, g) its matrix written through the bases Bl and the basis B of g is a homeomorphism on its range. Throughout the whole text, the set Hom(l, g) is therefore homeomorphically identified to a set U of M2n+1,k (ℝ) and Hom∘ (l, g) of all injective homomorphisms to the subset U ∘ of U consisting of the totality of matrices in U of maximal rank. Obviously, the set U is closed and algebraic in M2n+1,k (ℝ). In addition, the set U ∘ is semialgebraic (a difference of two Zariski open sets) and open in U . More on that can be found in [24]. In general, the set Hom∘ (l, g)

4.1 Deformation and moduli spaces for Heisenberg groups |

141

fails in most of the cases to be equipped with a smooth manifold structure. (In the whole text, we only speak about C ∞ smooth structures). When restricted to the Heisenberg setup, the following result will be proved throughout the next coming sections. Theorem 4.1.26. Let g be the Heisenberg algebra and l a subalgebra of g. Then the semialgebraic set Hom∘ (l, g) is endowed with a smooth manifold structure of dimension k(4n−k+3) if l is Abelian and (k−1)(4n−k+2) + k otherwise. 2 2 One first pace toward that aim is the following. Let G act on M2n+1,k (ℝ) by g ⋅ M = Adg ⋅M,

M ∈ M2n+1,k (ℝ),

g ∈ G.

Here, we view Adg as a real valued matrix for any g ∈ G. More precisely, for X in g with coordinates t (γ, α, β), γ ∈ ℝ, α, β ∈ M1,n (ℝ) in the basis B and x M = M(x, A, B) = (A) ∈ M2n+1,k (ℝ), B x − βA + αB Adexp X ⋅M = ( ). A B

(4.12)

Taking into account the action of G on Hom(l, g) defined in (3.3), the following lemma is immediate. Lemma 4.1.27. The map Ψ defined in (4.11) is G-equivariant. That is, for any ψ ∈ Hom(l, g) and g ∈ G, we have Ψ(g ⋅ ψ) = g ⋅ Ψ(ψ). Let H = exp h be a connected subgroup of G of dimension s and Γ a discontinuous group of G for G/H of rank k with a syndetic hull L = exp l. For a given matrix M, we adopt henceforth and unless a specific mention Mi and M i to denote the rows and respectively the columns of M. Our first result concerns the case where Γ is not Abelian. We will prove the following. Theorem 4.1.28. Let H be a connected subgroup of the Heisenberg group G and Γ a non-Abelian discontinuous group of G for the homogeneous space G/H. Then the spaces Hom∘ (l, g) and R(l, g, h) (resp., Hom∘ (l, g)/G and T (l, g, h)) are endowed with a smooth manifold structure of dimension (k−1)(4n−k+2) + k (resp., of dimension (k−1)(4n−k+2) + 1). 2 2 Proof. It is clear that L contains the center of G as it is non-Abelian. Since Γ acts properly on G/H, h∩l = {0} and then h is Abelian. Thus, dim h ≤ n. We can therefore choose a symplectic basis B = {Z, X1 , . . . , Xn , Y1 , . . . , Yn } of g adapted to h (according to Proposition 1.1.12 above), being generated by Bh = {X1 , . . . , Xs }. Let also Bl = {e1 = Z, e2 , . . . , ek } be a basis of l. As l is non-Abelian, there exist X0 , Y0 ∈ l such that [X0 , Y0 ] = Z. So, for ψ in Hom(l, g), ψ(Z) = ψ([X0 , Y0 ]) = [ψ(X0 ), ψ(Y0 )] ∈ z(g). Its matrix written through

142 | 4 The deformation space for nilpotent Lie groups the bases Bl and B is therefore of the form a0 0

M(a0 , a, A, B) = (

a ), C

a0 ∈ ℝ,

a ∈ ℝk−1

(4.13)

and A C(A, B) = ( ) B

and

A, B ∈ Mn,k−1 (ℝ).

(4.14)

We opt also for the notation (4.14) for any pair of matrices having a common number of colums. Let E denote the subspace of M2n+1,k (ℝ) consisting of the totality of matrices M as in equation (4.13). For any i, j ∈ {1, . . . , k}, we have [ψ(ei ), ψ(ej )] = ψ([ei , ej ]) = b(ei , ej )ψ(Z) = a0 b(ei , ej )Z. Denote then by 0 0

L=(

0 ) L0

the matrix of the restriction of the bilinear form b to l. So, ψ ∈ Hom(l, g) if and only if t MJb M = a0 L. The set Hom(l, g) is therefore homeomorphic to the set U of all M(a0 , a, A, B) ∈ E for which t

AB − t BA = a0 L0 .

As such, the set Hom∘ (l, g) is homeomorphically identified to the set U = {M(a0 , a, A, B) ∈ U : a0 ≠ 0 and rk(C) = k − 1}, ∘

where the symbol rk merely designates the rank. Let A (p, ℝ) denote the subspace of skew-symmetric matrices. We now establish the following elementary result. Lemma 4.1.29. Let M ∈ Mm,p (ℝ) (p ≤ m) of maximal rank. Then the map φM : Mm,p (ℝ) → A (p, ℝ),

H 󳨃→ t MH − t HM

is surjective. Proof. Let Mi , i = 1, . . . , m be the rows of the matrix M. Since M is of rank p, there exist 1 ≤ i1 < ⋅ ⋅ ⋅ < ip ≤ m such that the matrix M̃ constituted of the rows Mi1 , . . . , Mip belongs to GLp (ℝ). Let M̂ denote the matrix in Mm−p,p (ℝ) obtained by subtracting from M the

rows Mi1 , . . . , Mip . Opting for the same notation for a given matrix H in Mm,p (ℝ), we get φM (H) = t M̃ H̃ − t H̃ M̃ + t M̂ Ĥ − t Ĥ M.̂

4.1 Deformation and moduli spaces for Heisenberg groups |

143

̃ − t K M̃ is surjective as ker(φ ̃ ) = The map φM̃ : Mp (ℝ) → A (p, ℝ), K 󳨃→ t MK M t ̃ −1 M S (p, ℝ), where S (p, ℝ) denotes the space of symmetric matrices. Finally, if S ∈ A (p, ℝ) and K ∈ Mp (ℝ) with φM̃ (K) = S, the matrix H ∈ Mm,p (ℝ) such that H̃ = K and Ĥ = 0 satisfies the equation φM (H) = S. The lemma is thus proved. Let V = {M(x, a, A, B) ∈ E : x ≠ 0, and rk(C) = k − 1} be the open subset of E and Ψ the smooth map Ψ : V → A (k − 1, ℝ),

M 󳨃→ t AB − t BA − xL0 .

Clearly, U ∘ = Ψ−1 ({0}). The goal now is to show that zero is a regular value of the map Ψ. That is, for all M = M(a0 , a, A, B) in U ∘ , the derivative dΨM : E → A (k − 1, ℝ),

X = M(h0 , h, H, K) 󳨃→ t HB − t BH + t AK − t KA − h0 L0

is surjective. But dΨM (X) + h0 L0 = t (

−K −K ) C − tC ( ) H H

which is enough to conclude thanks to Lemma 4.1.29. The following lemma conclusively enables us to conclude that Hom∘ (l, g) is endowed with a smooth manifold structure with the mentioned dimension. Lemma 4.1.30. Let X and Y be two Hausdorff topological spaces and h : X → Y a homeomorphism. Assume that one of these spaces is endowed with a smooth manifold structure, then so is the second. We now focus attention to the space Hom∘ (l, g)/G. For any X = t (γ, α, β) ∈ g and M(a0 , a, A, B) ∈ U ∘ , we have as in equation (4.12), Adexp X ⋅M(a0 , a, A, B) = M(a0 , a + βA − αB, A, B). Here, γ ∈ ℝ, α and β are in ℝn . Now we can easily see that the matrix through the canonical basis of ℝ2n and ℝk−1 of the map ΦA,B : ℝn × ℝn → ℝk−1 , (α, β) 󳨃→ βA − αB is M(ΦA,B ) = (−t B, t A), which means that rk(M(ΦA,B )) = k − 1 and that ΦA,B is surjective. ̃∘ = {M(a , a, A, B) ∈ U ∘ : a = 0}, then the mapping Let U 0 ̃∘ , π̃ : U ∘ /G → U

[M(a0 , a, A, B)] 󳨃→ M(a0 , 0, A, B)

is a continuous bijection. In addition, its inverse coincides with the restriction of ̃∘ regarded as a subset of U ∘ . This shows that the canonical quotient surjection to U Hom∘ (l, g)/G is also endowed with a smooth manifold structure using the same technique. To achieve the proof of the theorem, it is sufficient to show that R(l, g, h) is open in Hom∘ (l, g). Indeed, let K = t (0, Is , 0) ∈ M2n+1,s (ℝ) be the matrix of the canonical

144 | 4 The deformation space for nilpotent Lie groups injection from h to g through the bases Bh and B of h and g, respectively. As G is 2-step, it is immediate that the action of exp(ψ(l)) on G/H is proper if and only if rk(M(a0 , a, A, B) ⋒ K) = rk(M(a0 , a, C(0, A(n−s) ), B) ⋒ K) = k + s,

A=(

A(s) ), A(n−s)

where the symbol ⋒ merely means the concatenation of the matrices written through B . This leads to the fact that R(l, g, h) is homeomorphic to the set W = {M(a0 , a, A, B) ∈ U : rk(C(A(n−s) , B)) = k − 1}, ∘

which is semialgebraic and open in U ∘ as was to be shown. The following result is then immediate. Corollary 4.1.31. Retain the same assumptions as in Theorem 4.1.28. Then the deformation space T (Γ, G, H) is semialgebraic and homeomorphic to the set ̃∘ = {M(a , 0, A, B) ∈ U : rk(C(A W ∩U 0 (n−s) , B)) = k − 1}. ∘

We also get the following upshot making use of Theorem 4.1.18. As mentioned in the first section, such a result may fail in higher steps nilpotent Lie groups. Corollary 4.1.32. Let H be a connected subgroup of the Heisenberg group G and Γ a discontinuous group of G for the homogeneous space G/H such that the Clifford–Klein form Γ\G/H is compact. Then the deformation space T (l, g, h) is endowed with a smooth manifold structure. The case where H meets the center of G We assume henceforth that Γ is Abelian. Our first upshot concerns the case when h contains the center of g. More precisely, we prove the following. Theorem 4.1.33. Let H be a connected subgroup of the Heisenberg group G containing the center of G and Γ a discontinuous subgroup of G for the homogeneous space G/H. Then the parameter and the deformation spaces are semialgebraic sets and both of them and k(4n−k+1) , reare endowed with a smooth manifold structure of dimension k(4n−k+3) 2 2 spectively. Proof. We keep the same notation as in Theorem 4.1.28. Note first that l is Abelian as h contains the center of g. Let B = {Z, X1 , . . . , Xn , Y1 , . . . , Yn } be a basis of g such that the subalgebra h is generated by Bh = {Z, X1 , . . . , Xp , Y1 , . . . , Yq } with dim h = s = p + q + 1 =

4.1 Deformation and moduli spaces for Heisenberg groups | 145

r + 1. We also fix a basis Bl = {e1 , . . . , ek } of l. Let ψ be a linear map from l to g and x M(x, A, B) = ( ) ∈ M2n+1,k (ℝ); C

x ∈ ℝk ,

A C=( ) B

and

A, B ∈ Mn,k (ℝ)

its matrix through the basis Bl and B . The space of all matrices of this form is noted by E. Then Hom(l, g) is clearly homeomorphic to the set U = {M(x, A, B) ∈ E : t BA − t AB = 0}. This entails therefore that R(l, g, h) is open in Hom∘ (l, g) and homeomorphic to the semialgebraic set W = {M(x, A, B) ∈ U : rk(C(2n−r) ) = k},

according to the writing C(r) ), C(2n−r)

C=(

where C(i) ∈ Mi,k (ℝ) for i ∈ {r, 2n − r}. Looking at the action of G, which only affects the central variables and following the same arguments as in Theorem 4.1.28. One immediately gets that the deformation space T (l, g, h) is homeomorphic to the semialgebraic set {M(x, A, B) ∈ W : x = 0}. These spaces are endowed with smooth manifold structures making use of the same technique of the proof of Theorem 4.1.28. The case where H does not meet the center of G We now pay attention to the case where h does not meet the center of g. As we shall remark, here is the single case where the deformation space may fail to be endowed with a smooth manifold structure, even with a Hausdorff topology. Our first result in this direction is the following. Theorem 4.1.34. Let H be a connected subgroup of the Heisenberg group G, which does not meet the center of G and Γ an Abelian discontinuous group of G for the homogeneous space G/H. Then the parameter space is described by a union of finitely many semialgebraic and G-invariant smooth manifolds. Accordingly, the deformation space is described by a union of finitely many semialgebraic smooth manifolds. Proof. We fix a basis B = {Z, X1 , . . . , Xn , Y1 , . . . , Yn } of g such that h = ℝ-span{X1 , . . . , Xs }. Denote by K = t (0 Is 0) the matrix of the canonical injection from h to g. As earlier, if ψ denotes a linear map from l to g with a matrix M, the action of exp ψ(l) on G/H is proper if and only if rk(Adexp X ⋅M ⋒ K) = k + s

for any X ∈ g.

Keeping the same notation as previously, the set Hom(l, g) is clearly homeomorphic to the set U = {M(x, A, B) ∈ E : t BA − t AB = 0} and Hom∘ (l, g) to the set U ∘ of matrices

146 | 4 The deformation space for nilpotent Lie groups in U of maximal rank. For any X = t (γ, α, β) ∈ g and M(x, A, B) ∈ U , we have as in equation (4.12), Adexp X ⋅M(x, A, B) = M(x − βA + αB, A, B). Here, γ ∈ ℝ, α and β are in ℝn . We adopt for the notation A A ( ) = ( (s) ) , B C where A(s) ∈ Ms,k (ℝ) and C ∈ M2n−s,k (ℝ) and also M(x, A(s) , C) instead of M(x, A, B). Then the fact that rk(Adexp X ⋅M ⋒ K) = k + s for all X ∈ g is equivalent to say that rk(M(x − βA + αB, A(s) , C) ⋒ K) = k + s,

(4.15)

for all α and β in ℝn , which is also equivalent to the fact that rk(M(x + wA(s) , 0, C)) = k

(4.16)

for all w ∈ ℝs . Indeed, this follows from simple manipulations of the rows of the matrix in equation (4.15). Assume for a while that rk(M(0, A, B)) = k, then condition (4.16) is equivalently expressed to mean that rk(M(y, 0, C)) = k, for all y ∈ ℝk which in turn gives rk(C) = k. Otherwise, the matrix M(0, A, B) is found to be of rank k − 1 and so is C. Let k−1 I2n−s = {(i1 , . . . , ik−1 ) ∈ ℕk−1 : 1 ≤ i1 < ⋅ ⋅ ⋅ < ik−1 ≤ 2n − s}. k−1 There exists therefore θ ∈ I2n−s such that the matrix

x1 + wA1(s)

⋅⋅⋅ Cθ

(

xk + ωAk(s)

)

is regular for all w ∈ ℝs , where Ai(s) , i = 1, . . . , k designate the columns of A(s) and Cθ is the submatrix of C formed by the rows Ci , i ∈ {i1 , . . . , ik−1 }. This fact is also equivalent to s

k

i=1

j=1

k

∑ wi ∑(−1)j+1 aij det(Cθ ) + ∑(−1)j+1 xj det(Cθ ) ≠ 0, (j)

(j)

j=1

(4.17)

for any reals w1 , . . . , ws . Here, Cθ merely designates the resulting matrix after the substitution of the j-column. Obviously, (4.17) is equivalent to (j)

k

∑(−1)j+1 xj det(Cθ ) ≠ 0 j=1

(j)

and

k

∑(−1)j+1 aij det(Cθ ) = 0 j=1

(j)

(i ∈ {1, . . . , s}),

4.1 Deformation and moduli spaces for Heisenberg groups |

147

which gives in turn that x ) ≠ 0 Cθ

det (

and

Ai )=0 Cθ

det (

for any i ∈ {1, . . . , s}. According to our circumstances, the last condition obviously holds as rk(C) = k − 1. Let now R0 (l, g, h) = {M(x, A, B) ∈ U 0 : rk(M(0, A, B)) = k} k−1 and for θ ∈ I2n−s , j ∈ {1, . . . , k}, j

Rθ (l, g, h) = {M(x, A, B) ∈ U 0 : rk(M(x, 0, Cθ )) = rk(M(0, A, B)) + 1 = k and det(Cθ ) ≠ 0}. (j)

k−1 Let I(s, k) = I2n−s ∪ {0}. The parameter space splits as follows:

R (l, g, h) =

⋃

θ∈I(s,k) j∈{1,...,k}

j

Rθ (l, g, h).

j

When θ = 0, we also opt for the notation R0 (l, g, h) = R0 (l, g, h) for any j ∈ {1, . . . , k}. j Furthermore, the layers Rθ (l, g, h), θ ∈ I(s, k) are G-invariant and semialgebraic sets. j

Indeed, the point is obviously clear for R0 (l, g, h). The G-invariance of Rθ (l, g, h) comes from the conditions rk(C) = rk(Cθ ) = k − 1 and rk(M(x − βA + αB, 0, C)) = k, for all α, β ∈ ℝn . This gives in turn that (j)

T (l, g, h) =

j

⋃

θ∈I(s,k) j∈{1,...,k}

j

j

Tθ (l, g, h),

j

where Tθ (l, g, h) = Rθ (l, g, h)/G. We now show that Tθ (l, g, h) is homeomorphic to j

j

Tθ = {M(x, A, B) ∈ Rθ (l, g, h) : xi = 0 for all i ≠ j}

(4.18)

k−1 if θ ∈ I2n−s and j ∈ {1, . . . , k} and to T0 = {M(0, A, B) ∈ R0 (l, g, h)} when θ = 0. The last fact holds similarly as it was the situation in Theorems 4.1.28 and 4.1.33. Let now j k−1 θ = (i1 , . . . , ik−1 ) ∈ I2n−s , j ∈ {1, . . . , k} and M(x, A, B) ∈ Rθ . We pick

Y = (−y1 , . . . , −yn , yn+1 , . . . , y2n ) = (−u, v) ∈ ℝ2n regarded as a vector of g ≃ ℝ2n+1 such that Yθ = (yi1 , . . . , yik−1 ) = −(x1 , . . . , xǰ , . . . , xk )(Cθ )

(j) −1

148 | 4 The deformation space for nilpotent Lie groups and yi = 0 otherwise. Let also ξj = xj + vAj − uBj and ξ = (0, . . . , 0, ξj , 0, . . . , 0) ∈ ℝk . Clearly, Adexp(Y) M(x, A, B) = M(ξ , A, B) and the map j

j

j

ψθ : Rθ (l, g, h) → Tθ ;

M(x, A, B) 󳨃→ M(ξ , A, B)

j

is continuous. Furthermore, G ⋅ M ∩ Tθ consists of one single point. Indeed, let M = M(ξ , A, B) and M ′ = M(ξ ′ , A, B); ξ = (0, . . . , 0, ξj , 0, . . . , 0), ξ ′ = (0, . . . , 0, ξj′ , 0, . . . , 0) be j

points in G ⋅ M ∩ Tθ . There exists X = (u, v) ∈ ℝ2n regarded as earlier as a vector of g such that M = Adexp X M ′ . This entails that ξj′ = ξj + vAj − uBj and vAi − uBi = 0 for any i ≠ j. Since rk(M(0, A, B)) = rk(C) = rk(Cθ ) = k − 1, (j)

M j (0, A, B) turns out to be a linear combination of M i (0, A, B) (i ≠ j). So, ξj′ = ξj and, therefore, M = M ′ . It follows that the quotient map ̃j : T j (l, g, h) → T j ; ψ θ θ θ

[M] 󳨃→ M(ξ , A, B)

(4.19)

is a continuous bijection and its inverse coincides with the restriction of the canonical surjective map R (l, g, h) → T (l, g, h), as was to be shown. Using similar details as j j in the proof of Theorem 4.1.26, we get right away that Rθ (l, g, h) and Tθ are smooth k−1 manifolds for θ ∈ I2n−s and j ∈ {1, . . . , k}. Then Lemma 4.1.30 enables us to conclude. We next prove the following. Proposition 4.1.35. Let H be a connected subgroup of the Heisenberg group G, which does not meet the center of G and Γ a maximal discontinuous group of G for the homogeneous space G/H. Then the parameter space R (l, g, h) and the deformation space T (l, g, h) are endowed with smooth manifold structures of dimension (n+1)(3n+2) and 2 3n(n+1) + 1, respectively. 2 Proof. The assertion concerning the parameter space R (l, g, h) is a direct consequence from above, as it is open in Hom0 (l, g). As in Subsection 4.1.7, R0 (l, g, h) is empty and T (l, g, h) =

j

⋃

n θ∈I2n−s j∈{1,...,n+1}

j

Tθ (l, g, h)

j

with Tθ (l, g, h) = Rθ (l, g, h)/G. Note that j

Rθ (l, g, h) = {M = M(x, A, B) ∈ R(l, g, h) : Pj,θ (M) := det(Cθ ) ≠ 0}, (j)

n which is open in R(l, g, h) for any θ ∈ I2n−s and j ∈ {1, . . . , n+1}. We first need to show that T (l, g, h) is a Hausdorff space. Let then [M1 ] ≠ [M2 ] be points of T (l, g, h) representing

4.1 Deformation and moduli spaces for Heisenberg groups |

149

two orbits G ⋅ M1 and G ⋅ M1 of R (l, g, h). When both the points belong to the same j stratum Tθ (l, g, h), we are done as being open and endowed with a smooth manifold j

structure. Assume then [Mi ] ∈ Tθ i (l, g, h), i = 1, 2 and (θ1 , j1 ) ≠ (θ2 , j2 ) with Pj1 ,θ1 (M1 ) ≠ i 0, Pj1 ,θ1 (M2 ) = 0 and Pj2 ,θ2 (M2 ) ≠ 0. Let ε = min{|Pj1 ,θ1 (M1 )|, |Pj2 ,θ2 (M2 )|} > 0, U1 = {M = M(x, A, B) ∈ R(l, g, h) : 󵄨󵄨󵄨Pj1 ,θ1 (M) − Pj1 ,θ1 (M1 )󵄨󵄨󵄨


ε }. 2

󵄨

󵄨

and 󵄨

󵄨

Then U1 and U2 are open and G-invariant disjoint sets containing M1 and M2 , respecj tively, which is enough to conclude. Up to this step, Tθ (l, g, h) being open in T (l, g, h) n for any θ ∈ I2n−s and j ∈ {1, . . . , n + 1}, we also have to prove that T (l, g, h) is equipped ̃j defined as in equation with a manifold structure. As in Theorem 4.1.34, the map ψ θ n (4.19) is a homeomorphism for all θ and j. We need to show that for all θ, α in I2n−s and j l j, l in {1, . . . , n + 1} such that Tθ (l, g, h) ∩ Tα (l, g, h) ≠ 0, the transition map ̃j ∘ (ψ ̃l )−1 : U j,l = ψ ̃l (T j (l, g, h) ∩ T l (l, g, h)) → V j,l = ψ ̃j (T j (l, g, h) ∩ T l (l, g, h)) ψ α α α α θ θ,α θ θ,α θ θ j,l

j,l

j

is a C ∞ map. Indeed, when j = l, it is easy to see that Uθ,α = Vθ,α = Tθ ∩ Tαj and that ̃j ∘ (ψ ̃j )−1 is the identity map. Otherwise, if j ≠ l (j < l, e. g.), then ψ α

θ

j,l

j,l

j

Uθ,α = {M(ξ , A, B) ∈ Tαl : rk(Cθ ) = n} and Vθ,α = {M(ξ , A, B) ∈ Tθ : rk(Cα(l) ) = n} (j)

j,l

and for M = M(ξ , A, B) ∈ Uθ,α , ξ = (0, . . . , 0, ξl , 0, . . . , 0) ̃ j ∘ (ψ ̃l )−1 (M) = M(η, A, B), ψ α θ where η = (0, . . . , 0, ηj , 0, . . . , 0) with ηj = vAj − uBj and (−u, v) = Y = (−y1 , . . . , −yn , yn+1 , . . . , y2n ) j such that Yθ = (yi1 , . . . , yin ) = −(0, . . . , 0,̌ . . . , 0, ξl , 0, . . . , 0)(Cθ )−1 and yi = 0 otherwise. This achieves the proof of the proposition.

Remark 4.1.36. We close this section with the following example, which treats the case where Γ is of rank one. In this case, we clearly have R (l, g, h) = R1 (l, g, h) ∐ R2 (l, g, h),

150 | 4 The deformation space for nilpotent Lie groups where R1 (l, g, h) = {M(x, 0, 0) ∈ M2n+1,1 (ℝ) : x ∈ ℝ× } and n

R2 (l, g, h) = {M(x, a, b) ∈ M2n+1,1 : a, b ∈ M1,n (ℝ) and ‖b‖2 + ∑ a2i ≠ 0}. i=s+1

Here, ai , i = 1, . . . , n designate the coordinates of a and ‖ ⋅ ‖ the Euclidian norm. This description shows that the G-orbits in R (l, g, h) are not separated by open neighborhoods, which entails that T (l, g, h) fails to be a Hausdorff space. As in the proof of Theorem 4.1.34, T2 (l, g, h) = R2 (l, g, h)/G is homeomorphic to 2

n

2

T2 = {M(0, a, b) ∈ M2n+1,1 : a, b ∈ M1,n (ℝ) and ‖b‖ + ∑ ai ≠ 0} i=s+1

and T1 (l, g, h) to T1 = {M(x, 0, 0) ∈ M2n+1,1 (ℝ) : x ∈ ℝ× }. Clearly, Ti (l, g, h), i = 1, 2 are equipped with smooth manifold structures and that T2 (l, g, h) is open and dense in T (l, g, h). Case where Γ is Abelian and maximal We assume in this subsection that Γ is maximal. We first pose the following. Definition 4.1.37. Let g be a Lie algebra. A maximal Abelian subalgebra of g is an Abelian subalgebra of g of maximal dimension. Maximal subalgebras are not unique and obviously contain the center of g. Definition 4.1.38. Let G = exp g be an exponential solvable Lie group. An Abelian discrete subgroup Γ is said to be maximal, if its rank is maximal (hence equals n + 1 in the Heisenberg case). The Lie algebra of its syndetic hull is therefore maximal according to Definition 4.1.37, and thus contains the center of g. We now show the following elementary result which entails that R0 (l, g, h) is empty in this case. Lemma 4.1.39. Let A and B be matrices in Mn,n+1 (ℝ) such that t BA − t AB = 0. Then rk(M(0, A, B)) ≤ n. Proof. Equation t BA− t AB = 0 is equivalent to t Bi Aj − t Ai Bj = 0 for all i, j = 1, . . . , n+1 or equivalently the matrix product t M i (0, B, −A)M j (0, A, B) vanishes for all i, j = 1, . . . , n+1. Let a and b be the subspace of ℝ2n spanned with the vectors {M j (0, A, B), j = 1, . . . , n} and {M j (0, −B, A), j = 1, . . . , n + 1}, respectively. Then clearly b ⊂ a⊥ , where ⊥ means the orthogonality symbol with respect to the Euclidian scalar product of ℝ2n . Then

4.1 Deformation and moduli spaces for Heisenberg groups | 151

dim(b) = dim(a) ≤ dim(a⊥ ) = 2n − dim(a). This entails that dim(a) ≤ n as was to be shown. This gives in turn that T (l, g, h) =

j

Tθ (l, g, h)

⋃

n θ∈I2n−s

j∈{1,...,n+1} j

j

j

with Tθ (l, g, h) = Rθ (l, g, h)/G which is homeomorphic to the set Tθ as defined in equation (4.18). The following is a direct consequence of Theorem 4.1.34. Corollary 4.1.40. Retain the same assumptions as in Theorem 4.1.34. Then G acts on the parameter space R (l, g, h) with constant dimension orbits if and only if Γ is maximal. The following corollary stems from Theorems 4.1.28, 4.1.33 and 4.1.34. Corollary 4.1.41. Let H be a connected subgroup of the Heisenberg group G and Γ a discontinuous group of G for the homogeneous space G/H. Then the parameter space R (l, g, h) is semialgebraic (as in Definition 3.2.7). 4.1.8 Proof of Theorem 4.1.26 When Γ is non-Abelian, the theorem is already proved in Theorem 4.1.28. We now focus attention to the case where Γ is Abelian. Let U be the open subset of M2n+1,k (ℝ) of matrices of rank k and consider the smooth map φ : U → A (k, ℝ), M 󳨃→ t AB − t BA. One has Hom∘ (l, g) = φ−1 ({0}). So, it is sufficient to show that 0 is a regular value of φ. Let M = M(x, A, B) ∈ U. The derivative dφM : M2n+1,k (ℝ) 󳨀→ A (k, ℝ) can be easily written as dφM (M(a, H, K)) = t HB − t BH + t AK − t KA. Just in case the matrix M0 = M(0, A, B) is of maximal rank, we are already done thanks ̃0 be the resulting to Lemma 4.1.29. Assume otherwise that M0 has k−1 as its rank. Let M matrix from M0 after subtracting its first column. We also adopt here the same notation ̃0 is of rank k − 1. Then for any given matrix. We can and do admit that M t

0 −t u

AB − t BA = (

t

u ), ̃ ̃ AB − t B̃ Ã

where u = t A1 B̃ − t B1 A.̃ So, 0 −t v

dφM (M(a, H, K)) = (

t

v ), H̃ B̃ − t B̃ H̃ + t à K̃ − t K̃ Ã

152 | 4 The deformation space for nilpotent Lie groups where v = t H 1 B̃ − t B1 H̃ + t A1 K̃ − t K 1 A.̃ Let S=(

0 − S1 t

S1 ) ∈ A (k, ℝ), S̃

where S̃ belongs to A (k − 1, ℝ) and S1 to M1,k−1 (ℝ). As à rk ( ̃ ) = k − 1, B there exist by Lemma 4.1.29 some matrices H̃ and K̃ in Mn,k−1 (ℝ) such that S̃ = t H̃ B̃ − t ̃ ̃ BH + t à K̃ − t K̃ A.̃ On the other hand, as à rk ( ̃ ) = k − 1, B there exist H 1 and K 1 such that t

−K 1 Ã B1 H̃ t 1 ) ( ̃ ) = S1 + ( 1) ( ̃ ) . H B −A K

(

This proves that dφM is surjective. 4.1.9 From H2n+1 to the product group H2n+1 × H2n+1 Let p = h2n+1 × h2n+1 be the Lie algebra of P. For (X, Y), (X ′ , Y ′ ) ∈ p, the Lie bracket [(X, Y), (X ′ , Y ′ )] is given by [(X, Y), (X ′ , Y ′ )] = ([X, X ′ ], [Y, Y ′ ]). It is then obvious that the group P is a 2-step nilpotent Lie group. If exp : h2n+1 → H2n+1 is the exponential map, we denote also by exp the exponential map from p on P, which is simply defined by exp(X, Y) = (exp X, exp Y),

(X, Y) ∈ p.

The center z of the Lie algebra p is two-dimensional and is generated by the vectors {(Z, 0), (Z, Z)}. We denote by Δ the diagonal subgroup of P: Δ = {(x, x), x ∈ H2n+1 },

4.1 Deformation and moduli spaces for Heisenberg groups | 153

which is a (2n + 1)-dimensional subgroup of P. Its Lie algebra D = {(X, X), X ∈ h2n+1 } is generated by {(Z, Z), (Xi , Xi ), (Yi , Yi , ), 1 ≤ i ≤ n} so that p is decomposed as p = ℝ(Z, 0) ⊕ D ⊕ k, where k is the vector space spanned by {(Xi , 0), (Yi , 0), 1 ≤ i ≤ n}. The group P acts on p by the adjoint action AdP such that for all (X, X ′ ), (Y, Y ′ ) ∈ p, Adexp(X,X ′ ) (Y, Y ′ ) = ead(X,X′ ) (Y, Y ′ )

= (Y, Y ′ ) + ad(X,X ′ ) (Y, Y ′ )

= (Y, Y ′ ) + ([X, Y], [X ′ , Y ′ ]). From now on, we fix the following basis of p: B = {(Z, 0), (Z, Z), (Xi , Xi ), (Yi , Yi ), (Xi , 0), (Yi , 0), 1 ≤ i ≤ n}.

For (X, Y), (X ′ , Y ′ ) ∈ p, one has [(X, Y), (X ′ , Y ′ )] = ([X, X ′ ], [Y, Y ′ ])

= b1 ((X, Y), (X ′ , Y ′ ))(Z, 0) + b2 ((X, Y), (X ′ , Y ′ ))(Z, Z),

where b1 and b2 are the skew symmetric bilinear forms on p defining its Lie bracket. By a routine computation, the matrices Jb1 and Jb2 of b1 and b2 written through the basis B are given by Jb1

0M2 (ℝ) =( 0 0

0 0 J

0 J), J

Jb2

0M2 (ℝ) =( 0 0

0 J 0

0 0) 0

and

0 −In

J=(

In ). 0

Let now Γ be a discontinuous group for P/Δ and L = exp l its syndetic hull. Since l ∩ D = {0}, then (Z, Z) ∉ l that is dim (l ∩ z) ≤ 1. We fix then a basis B0 = {e1 , . . . , ek } of l where e1 is a generator of l ∩ z whenever this space is not trivial. For all 1 ≤ i, j ≤ k, we have [ei , ej ] = αij e1 . Denote by K the matrix (αij )i,j={1,...,k} . This matrix equals zero when l is Abelian. Otherwise, K has the form 0 0

K=(

0 ), K0

(4.20)

154 | 4 The deformation space for nilpotent Lie groups where K0 ∈ Mk−1 (ℝ) is a skew symmetric matrix. Let ψ be a linear map from l to p. Its matrix written through the bases B0 and B is x y M = M(x, y, A, B) = ( ) , A B

x, y ∈ M1,k (ℝ),

A=(

A1 ), A2

B B = ( 1) , B2

(4.21)

where A1 , A2 , B1 , B2 are in Mn,k (ℝ). If ψ ∈ Hom(l, p), we have [ψ(ei ), ψ(ej )] = ψ([ei , ej ])

for all 1 ≤ i, j ≤ k.

(4.22)

When l is not Abelian, there exist V1 , V2 ∈ l such that [V1 , V2 ] = e1 . So, ψ(e1 ) = ψ([V1 , V2 ]) = [ψ(V1 ), ψ(V2 )] ∈ z. There exist therefore x0 and y0 in ℝ such that ψ(e1 ) = x0 (Z, 0) + b0 (Z, Z). Hence, ψ([ei , ej ]) = αij ψ(e1 )

= αij (x0 (Z, 0) + y0 (Z, Z))

= x0 αij (Z, 0) + y0 αij (Z, Z).

Then equations (4.22) read in term of matrices: t

MJb1 M = x0 K

and

t

MJb2 M = y0 K,

which is equivalent to t

AJB + t BJA + t BJB = x0 K

and

t

AJA = y0 K,

or also t

{t

B2 A1 − t A1 B2 + t A2 B1 − t B1 A2 + t B2 B1 − t B1 B2 = x0 K,

A2 A1 − t A1 A2 = y0 K.

(4.23)

If l is Abelian, we have the same equations but with K = 0 as it was mentioned above. Let E = {M = M(x, y, A, B) ∈ M4n+2,k (ℝ) satisfying (4.23)}.

Since the spaces Hom(Γ, P) and Hom(l, p) are homeomorphic, the following proposition is straightforward.

4.1 Deformation and moduli spaces for Heisenberg groups | 155

Proposition 4.1.42. The space Hom(Γ, P) is homeomorphic to E . Recall that P acts on Hom(l, p) by ψg = Adg −1 ∘ψ,

g ∈ P,

ψ ∈ Hom(l, p)

M g = Adg −1 ⋅M,

g∈P

and M ∈ E .

and P acts on E by

The identification Φ : ψ 󳨃󳨀→ M(ψ, B0 , B ) is a homeomorphism, which is P-equivariant. That is, g

(Φ(ψ)) = Φ(ψg ),

g ∈ P.

Let X ∈ p with coordinates t (a, b, α, β, γ, δ) through the basis B , where a, b ∈ ℝ and α, β, γ, δ ∈ ℝ2n . For a matrix M = M(x, y, A, B) as in (4.21), we have Adexp X ⋅M = M(x − δA1 + γA2 − (β + δ)B1 + (α + γ)B2 , y − βA1 + αA2 , A, B). Putting u = (−(β + δ), α + γ) and v = (−β, α), we get Adexp X ⋅M = M(x + (u − v)A + uB, y + vA, A, B).

(4.24)

This allows us to give the following description of the parameter space. We then have the following. Proposition 4.1.43. The parameter space R (l, p, D) is homeomorphic to the space x − ωA ) = k for any ω ∈ ℝ2n } . B

{M ∈ E : rk (

(4.25)

Proof. Let ψ ∈ Hom(l, p) and M = M(x, y, A, B) its associated matrix. The action of exp ψ(l) on P/Δ is proper if and only if for all X ∈ p, Adexp X (ψ(l)) ∩ D = {0}. That is, Adexp X M(Y) ∉ D for all Y ∈ l\{0} and for all X ∈ p. According to (4.24), this means that for all u, v ∈ ℝ2n and Y ∈ l\{0}, x + (u − v)A + uB ) Y ≠ 0, B

(

156 | 4 The deformation space for nilpotent Lie groups which is in turn equivalent to x + (u − v)A + uB x + (u − v)A ) = rk ( )=k B B

rk ( for all u, v ∈ ℝ2n .

Assume now that the subgroup Γ is non-Abelian, which means that l∩z = ℝe1 . Note that if ψ ∈ Hom(l, p), ψ(e1 ) belongs to z and so its associated matrix is M = M(x, y, A, B) ̃ A = (0, A0 ) and B = (0, B0 ) where x0 , y0 ∈ ℝ, x,̃ ỹ ∈ ℝk−1 ̃ y = (y0 , y), such that x = (x0 , x), and A0 , B0 ∈ M2n,k−1 (ℝ). So, we have the following. Corollary 4.1.44. If the subgroup Γ is non-Abelian, the parameter space R (l, p, D) is homeomorphic to the space ̃ (0, A0 ), (0, B0 )) ∈ E : x0 ≠ 0 and rk(B0 ) = k − 1}. ̃ (y0 , y), {M = M((x0 , x),

(4.26)

Proof. It is sufficient to notice that x0 0

rk(M) = k ⇐⇒ rk (

⇐⇒ x0 ≠ 0

x − ωA0 )=k B0 and

for all ω ∈ ℝ2n

rk(B0 ) = k − 1.

4.2 Case of 2-step nilpotent Lie groups We assume henceforth that g is a 2-step nilpotent Lie algebra, l a subalgebra of g and z the (nontrivial) center of g. We consider the decompositions g = z ⊕ g′

and

l = [l, l] ⊕ l′ ,

(4.27)

where g′ (resp., l′ ) designates a subspace of g (of l, resp.). Any φ ∈ L (l, g) can be written as Aφ Cφ

φ=(

Bφ ), Dφ

(4.28)

where Aφ ∈ L ([l, l], z), Bφ ∈ L (l′ , z), Cφ ∈ L ([l, l], g′ ) and Dφ ∈ L (l′ , g′ ). Let Aφ 0

φ′ = (

0 ). Dφ

(4.29)

We first remark the following assertion. Lemma 4.2.1. Any element φ ∈ L (l, g) is a Lie algebra homomorphism if and only if Cφ = 0 and φ′ ∈ Hom(l, g).

4.2 Case of 2-step nilpotent Lie groups | 157

Proof. We point out first that if φ ∈ Hom(l, g), then φ([l, l]) = [φ(l), φ(l)] ⊂ z, in particular Cφ = 0 and A φ=( φ 0

Bφ ). Dφ

Let x = x1 + y1 and x′ = x2 + y2 ∈ l where xi ∈ [l, l] and yi ∈ l′ , then φ ∈ Hom(l, g) ⇔ [φ(x), φ(x′ )] = φ([x, x′ ])

⇔ [Aφ (x1 ) + Bφ (y1 ) + Dφ (y1 ), Aφ (x2 ) + Bφ (y2 ) + Dφ (y2 )] = φ([x1 + y1 , x2 + y2 ])

⇔ [Dφ (y1 ), Dφ (y2 )] = φ([y1 , y2 ]) = Aφ ([y1 , y2 ]) ⇔ [Dφ (y1 ), Dφ (y2 )] = Aφ ([y1 , y2 ]) ⇔ φ′ ∈ Hom(l, g). For any g ∈ G, let as earlier Adg designate the adjoint representation, which can be written making use the decomposition (4.37) as I Adg = ( z 0

σ(g) ) Ig′

(4.30)

for some map σ : G → L (g′ , z). Here, Iz and Ig′ denote the identity maps of z and g′ , respectively. Lemma 4.2.2. The map σ is a group homomorphism. In particular, the range of σ is a linear subspace of L (g′ , z). Proof. Clearly, Ad : G → GL(g) is a group homomorphism. Then for g and g ′ ∈ G, Iz 0

σ(gg ′ ) I )=(z Ig′ 0

σ(g) Iz )( Ig′ 0

Iz 0

σ(gg ′ ) I )=(z Ig′ 0

σ(g) + σ(g ′ ) ) Ig′

Adgg ′ = Adg Adg ′ ⇔ (

⇔(

σ(g ′ ) ) Ig′

⇔ σ(gg ′ ) = σ(g) + σ(g ′ ). In particular, σ is continuous and G is connected which means therefore that Im(σ) is a connected (linear) subgroup of the linear space L (g′ , z). Let h be a subalgebra of g and consider the decompositions g = (z ∩ h) ⊕ z′ ⊕ h′ ⊕ V

and l = [l, l] ⊕ l′ ,

(4.31)

158 | 4 The deformation space for nilpotent Lie groups where z = (z ∩ h) ⊕ z′ , h = (z ∩ h) ⊕ h′ and V is a linear subspace supplementary to (z ∩ h) ⊕ z′ ⊕ h′ in g. Then with respect to these decompositions, the adjoint representation Adg , g ∈ G can once again written down as I1 0 Adg = ( 0 0

0 I2 0 0

σ1 (g) σ2 (g) I3 0

δ1 (g) δ2 (g) ), 0 I4

where σ (g) σ(g) = ( 1 σ2 (g)

δ1 (g) ), δ2 (g)

σ1 (g) ∈ L (h′ , z ∩ h), δ1 (g) ∈ L (V, z ∩ h), σ2 (g) ∈ L (h′ , z′ ) and δ2 (g) ∈ L (V, z′ ) (here I1 , I2 , I3 and I4 designate the identity maps of z ∩ h, z′ , h′ and V, resp.). This leads to the fact that any element of Hom(l, g) can be written accordingly, as a matrix A1 A2 φ(A, B) = ( 0 0

B1 B2 ), B3 B4

where A A = ( 1) A2

B1 B2 B = ( ). B3 B4

and

(4.32)

Here, A1 ∈ L ([l, l], z ∩ h), A2 ∈ L ([l, l], z′ ), B1 ∈ L (l′ , z ∩ h), B2 ∈ L (l′ , z′ ), B3 ∈ L (l′ , h′ ) and B4 ∈ L (l′ , V). We can now state our first result. Theorem 4.2.3. Let G be a 2-step nilpotent Lie group, H a connected subgroup of G and Γ a discontinuous group for G/H. Then the parameter space R (l, g, h) splits into the disjoint union R1 ⊔ R2 , where 󵄨󵄨 ′ 󵄨 rk(B4 ) = dim(l ) } 󵄨󵄨 and rk(A ) = dim([l, l]) 2 󵄨

󵄨 R1 := {φ(A, B) ∈ Hom(l, g) 󵄨󵄨󵄨

4.2 Case of 2-step nilpotent Lie groups | 159

and { { R2 := {φ(A, B) ∈ Hom(l, g) { {

󵄨󵄨 rk(B ) < dim(l′ ) and for all g ∈ G, 󵄨󵄨 4 } 󵄨󵄨 } 󵄨󵄨 󵄨󵄨 A2 B2 + σ2 (g)B3 + δ2 (g)B4 }. 󵄨󵄨 rk ( ) = dim(l) } 󵄨󵄨 0 B4 } 󵄨

Proof. As the pair (G, H) has the Lipsman property (cf. Theorem 2.2.2), the equality set (3.5) enables us to state that 󵄨󵄨 󵄨 dim φ(A, B)(l) = dim(l), }. 󵄨󵄨 Ad φ(A, B)(l) ∩ h = {0} for all g ∈ G g 󵄨

󵄨 R (l, g, h) = {φ(A, B) ∈ Hom(l, g) 󵄨󵄨󵄨 Now A1 A2 Adg φ(A, B) = ( 0 0

B1 + σ1 (g)B3 + δ1 (g)B4 B2 + σ2 (g)B3 + δ2 (g)B4 ), B3 B4

(4.33)

which means that the condition [Adg φ(A, B)](l) ∩ h = {0} is equivalent to the fact that A rk ( 2 0

B2 + σ2 (g)B3 + δ2 (g)B4 ) = dim(l), B4

which is in turn equivalent to rk(B4 ) = dim(l′ ) and

rk(A2 ) = dim([l, l])

or rk(B4 ) < dim(l′ ) and

A2 0

rk (

B2 + σ2 (g)B3 + δ2 (g)B4 ) = dim(l). B4

4.2.1 Description of the deformation space T (l, g, h) We fix in this section our objects and we keep the same notation. Let g and l be as above and Hom1 (l, g) := {φ′ : φ ∈ Hom(l, g)}, where φ′ is as in (4.29). The group G acts on Hom1 (l, g) × L (l, g) through the following law: g ⋅ (φ′ , Bφ ) = (φ′ , Bφ + σ(g)Dφ ), where Bφ and Dφ are as in formula (4.28) and σ is as in (4.30).

160 | 4 The deformation space for nilpotent Lie groups Lemma 4.2.4. The map ψ : Hom(l, g) φ

Hom1 (l, g) × L (l′ , z)

󳨀→

(φ′ , Bφ )

󳨃󳨀→

is a G-equivariant homeomorphism. Proof. The fact that ψ is a well-defined homeomorphism comes directly from Lemma 4.2.1. Let g ∈ G and φ ∈ Hom(l, g), then ψ(g ⋅ φ) = ψ(Adg φ)

= (φ′ , Bφ + σ(g)Dφ )

= g ⋅ ψ(φ), which proves the lemma.

4.2.2 Decomposition of Hom1 (l, g) As in Lemma 4.2.2, the group σ(G) is a linear space. For any φ ∈ Hom(l, g), let lφ be the linear map defined by lφ : σ(G)

󳨀→

L (l , z)

σ(g)

󳨃󳨀→

σ(g)Dφ .

′

The range of lφ is a linear subspace of L (l′ , z) and the orbit G ⋅ (φ′ , Bφ ) = (φ′ , Bφ + Im(lφ )). Let m = dim(L (l′ , z)) and q = dim(σ(G)). For t = 0, 1, . . . , q, we define the sets Homt1 (l, g) := {φ′ ∈ Hom1 (l, g) : rk(lφ ) = t}. Then clearly, q

Hom1 (l, g) = ⨆ Homt1 (l, g). t=0

We fix a basis {e1 , . . . , em } of L (l′ , z) and let I(m, m − t) = {(i1 , . . . , im−t ); 1 ≤ i1 < ⋅ ⋅ ⋅ < im−t ≤ m}. For β = (i1 , . . . , im−t ) ∈ I(m, m − t), we consider the subspace Vβ := ⨁m−t j=1 ℝeij .

4.2 Case of 2-step nilpotent Lie groups | 161

Proposition 4.2.5. For any φ ∈ Homt1 (l, g), let Pφ : L (l′ , z) → L (l′ , z)/ Im(lφ ) and Homt1,β (l, g) := {φ ∈ Homt1 (l, g) : det(Pφ (ei1 ), . . . , Pφ (eim−t )) ≠ 0}. Then Homt1 (l, g) =

⋃

β∈I(m,m−t)

Homt1,β (l, g)

as a union of open subsets. Proof. We know that for all φ ∈ Homt1 (l, g), the set Im(lφ ) is a linear subspace of L (l′ , z) of dimension t. There exists therefore (i1 , . . . , im−t ) ∈ I(m, m − t) such that the family {Pφ (ei1 ), . . . , Pφ (eim−t )} forms a basis of L (l′ , z)/ Im(lφ ), and consequently det(Pφ (ei1 ), . . . , Pφ (eim−t )) ≠ 0. We are now ready to prove our main result in this section. Theorem 4.2.6. Let G, H and l be as before. The deformation space reads 2 q

T (l, g, h) = ⨆ ⨆

⋃

Tt,β,i ,

i=1 t=0β∈I(m,m−t)

where for β = (i1 , . . . , im−t ), the set Tt,β,1 is homeomorphic to the semialgebraic set

Tt,β,1

󵄨󵄨 B 󵄨󵄨 1 { } { } 󵄨󵄨 ( ) ∈ Vβ , rk(lφ(A,B) ) = t, { } { } 󵄨 B { } 󵄨 2 { } 󵄨 { } 󵄨󵄨 { } 󵄨󵄨 := {φ(A, B) ∈ Hom(l, g) 󵄨󵄨 det(Pφ(A,B) (ei1 ), . . . , Pφ(A,B) (eim−t )) ≠ 0, } 󵄨󵄨 { } { } 󵄨󵄨 { } { } 󵄨󵄨 { } A2 0 { } 󵄨 { } ) = dim(l) 󵄨󵄨 rk ( 󵄨 0 B4 󵄨 { }

and Tt,β,2 is homeomorphic to

Tt,β,2

󵄨󵄨 B } { 󵄨󵄨󵄨 ( 1 ) ∈ Vβ , rk(lφ(A,B) ) = t, } { } { 󵄨󵄨 B } { } { 󵄨󵄨 2 } { 󵄨 } { 󵄨 { } 󵄨 { } 󵄨 { } det(P (e ), . . . , P (e )) = ̸ 0, 󵄨 { } φ(A,B) i φ(A,B) i 󵄨 { } 1 m−t 󵄨 { } 󵄨󵄨 { } ′ 󵄨 , := {φ(A, B) ∈ Hom(l, g) 󵄨󵄨󵄨 rk(B4 ) < dim(l ), } { } 󵄨󵄨 } { { } 󵄨󵄨 { } { } A2 B2 + σ2 (g)B3 + δ2 (g)B4 󵄨󵄨 { { } ) = dim(l) } 󵄨󵄨󵄨 rk ( { } { } 0 B 󵄨󵄨 { } 4 { } 󵄨 { } 󵄨 { } 󵄨󵄨 󵄨 󵄨 for all g ∈ G { }

which is also semialgebraic.

162 | 4 The deformation space for nilpotent Lie groups Proof. It is easy to see that Dφ = Dg⋅φ , which means that lφ = lg.φ and Pφ′ = Pg⋅φ′ for all φ ∈ Hom(l, g) and g ∈ G. Then for all β ∈ I(m, m − t) and 0 ≤ t ≤ q, the set Homt1,β (l, g) × L (l′ , z) is G-stable. Let Homtβ (l, g) = ψ−1 (Homt1,β (l, g) × L (l′ , z)), then q

Hom(l, g) = ⨆ Homt (l, g), t=0

where Homt (l, g) :=

⋃

β∈I(m,m−t)

Homtβ (l, g).

(4.34)

Since ψ is G-equivariant, it is clear that for all t ∈ {0, . . . , q} and β ∈ I(m, m − t), the set Homtβ (l, g) is G-stable and by (4.34), we get Homt (l, g)/G =

⋃

β∈I(m,m−t)

Homtβ (l, g)/G,

(4.35)

and then q

Hom(l, g)/G = ⨆

⋃

t=0β∈I(m,m−t)

Homtβ (l, g)/G.

Whence, 2 q

R (l, g, h) = ⨆ ⨆

⋃

i=1 t=0β∈I(m,m−t)

Homtβ (l, g) ∩ Ri .

We now consider the maps ψ : Homtβ (l, g)/G πβt

G⋅φ :

→

(Homt1,β (l, g) × L (l′ , z))/G

󳨃→

G ⋅ ψ(φ),

(Homt1,β (l, g) × L (l′ , z))/G (φ′ , Bφ + Im(lφ ))

→

Homt1,β (l, g) × Vβ

󳨃→

−1 (φ′ , Pφ| (Bφ + Im(lφ ))) V β

and εβt = ψ−1 ∘ πβt ∘ ψ. We now show that for all t ∈ {0, . . . , q}, the collection (εβt , Homtβ (l, g)/G)β∈I(m,m−t) is a family of local sections of the canonical surjection π t : Homt (l, g) 󳨀→ Homt (l, g)/G. Indeed, we have to show that π t ∘ εβt = IdHomt (l,g)/G . Let φ ∈ Hom(l, g) such that the β

4.2 Case of 2-step nilpotent Lie groups | 163

orbit Aφ 0

(

Bφ + Im(lφ ) ) ∈ Homtβ (l, g)/G, Dφ

then Aφ 0

π t ∘ εβt (

Aφ Bφ + Im(lφ ) ) = πt ( Dφ 0

Pφ−1′ |V (Bφ + Im(lφ )) β

Dφ

)

Aφ

Pφ−1′ |V (Bφ + Im(lφ )) + Im(lφ )

Aφ 0

Bφ + Im(lφ ) ). Dφ

=(

0

=(

β

Dφ

)

In particular, εβt (Homtβ (l, g)/G)

󵄨󵄨 φ′ ∈ Homt (l, g), 󵄨󵄨 1,β }. = {φ ∈ Hom(l, g) 󵄨󵄨󵄨 󵄨󵄨 B ∈ V 󵄨 φ β

(4.36)

Now T (l, g, h) = {[φ] ∈ Hom(l, g)/G : exp(φ(l)) acts properly on G/H} q

=⨆

⋃

{[φ] ∈ Homtβ (l, g)/G} ∩ R (l, g, h).

t=0 β∈I(m,m−t)

4.2.3 Hausdorffness of the deformation space This section aims to study the Hausdorffness of the deformation space T (l, g, h) in the setting where g is 2-step nilpotent. We first prove the following. Lemma 4.2.7. For all φ ∈ Hom(l, g), we have dim(G ⋅ φ) = dim(g) − dim(φ(l)⊥ ), where φ(l)⊥ = {X ∈ g : [X, Y] = 0, ∀ Y ∈ φ(l)}. Proof. Recall that the map σ : G → L (l′ , z) is a group homomorphism and it is clear that ker(σ) = Z(G) where Z(G) is the center of G. Then σ factors via the projection map G → G/Z(G) to obtain an injective homomorphism σ̃ : G/Z(G) → L (l′ , z) such that ̃ σ(G) = σ(G).

164 | 4 The deformation space for nilpotent Lie groups ̃ X ) : [X, Y] = 0 ∀ Y ∈ φ(l)}. Thus, Let now φ ∈ Hom(l, g), then ker(lφ ) = {σ(e dim(G ⋅ φ) = dim(σ(G)) − dim(eφ(l) ) ⊥

= dim(g) − dim(z) − (dim(φ(l)⊥ ) − dim(z)) = dim(g) − dim(φ(l)⊥ ).

We now prove the main result of this section. Theorem 4.2.8. Let g be a 2-step nilpotent Lie algebra, if all G-orbits in R (l, g, h) have a common dimension, then the deformation space T (l, g, h) is a Hausdorff space. Proof. In such a situation, there is t ∈ {0, . . . , q} such that R (l, g, h) ⊂ Homt (l, g), where Homt (l, g) is as in equation (4.34) and q = dim σ(G). Indeed, let φ ∈ Homt (l, g) then rk lφ = t and so dim G ⋅ φ = dim ψ(G ⋅ φ) = dim G ⋅ ψ(φ)

= dim(φ′ , Bφ + σ(G)Dφ ) = dim(φ′ , Bφ + Im lφ ) = rk lφ = t. The deformation space T (l, g, h) is therefore contained in Homt (l, g)/G, and it is sufficient to show that Homt (l, g)/G is a Hausdorff space. Let [φ] ≠ [ξ ] be some points in Homt (l, g)/G. Suppose that [φ] and [ξ ] are not separated, then there exist (φn )n ⊂ Homt (l, g) and gn ∈ G such that φn converges to φ and gn ⋅ φn converges to ξ in Homt (l, g). Using the map ψ defined in Lemma 4.2.4, we can see that the sequence (φ′n , Bφn )n converges to (φ′ , Bφ ), (φ′n , Bφn + σ(gn )Dφn )n converges to (φ′ , Bφ ) and (φ′n , Bφn + σ(gn )Dφn )n converges to (ξ ′ , Bξ ). This means that φ′ = ξ ′ and in particular Dφ = Dξ and Pφ = Pξ . Finally, [φ] and [ξ ] belong to the open set Homtβ (l, g)/G for some β ∈ I(m, m − t). From (4.36), Homtβ (l, g)/G is homeomorphic to a semialgebraic set. Therefore, Homtβ (l, g)/G is a Hausdorff space. This leads thus to a contradiction.

Corollary 4.2.9. Let g be a 2-step nilpotent Lie algebra. If l is a maximal Abelian subalgebra of g, then the deformation space T (l, g, h) is a Hausdorff space. Proof. If l is a maximal Abelian subalgebra, then so is φ(l) and we have φ(l)⊥ = φ(l). Hence, from Lemma 4.2.7 dim(G ⋅ φ) = dim(g) − dim(φ(l)) = dim(g) − dim(l), which is constant. This achieves the proof. Proposition 4.2.10. Let l and h be two subalgebras of g and suppose that:

4.2 Case of 2-step nilpotent Lie groups | 165

(1) There is decomposition g = [l, l] ⊕ z1 ⊕ z2 ⊕ z3 ⊕ h′ ⊕ l′ ⊕ V, where z ∩ l = [l, l] ⊕ z1 , l = [l, l] ⊕ z1 ⊕ l′ , z ∩ h = z2 , h = h′ ⊕ z2 and z = [l, l] ⊕ z1 ⊕ z2 ⊕ z3 . (2) There is a codimension one subalgebra l1 of l such that [l⊥ 1 , l] ⊈ l ⊕ h and z(l) + l1 = l. Then the deformation space T (l, g, h) fails to be a Hausdorff space. Proof. Recall that dim([l, l]) = s and set dim(z1 ) = q, dim(z2 ) = p, dim(z3 ) = r and dim(z) = m. The Lie bracket of g is given by m

[X, Y] = ∑bi (X, Y)Zi , i=1

where {Z1 , . . . , Zm } is a basis of z passing through the decomposition z = [l, l]⊕z1 ⊕z2 ⊕z3 . There a basis {L1 , . . . , Lk−q−s } of l′ such that {Z1 , . . . , Zs+q , L2 , . . . , Lk−q−s } is a basis of the subalgebra l1 satisfying the second assumption in the proposition. The vector L1 ∈ z(l) and there is X0 ∈ l⊥ 1 such that the bracket [X0 , L1 ] ∉ l⊕h. In particular, there is p+q+s < i0 ≤ m such that bi0 (X0 , L1 ) = α ≠ 0. If we complete the vectors Z1 , . . . , Zm , L1 , . . . , Lk−q−s to a basis of g passing through the decomposition g = [l, l] ⊕ z1 ⊕ z2 ⊕ z3 ⊕ l′ ⊕ h′ ⊕ V, then the matrix Is 0 (0 ( M1 = ( (0 (0 0 (0

0 Iq 0 0 0 0 0

0 0 0) ) C1 ) ) ∈ R (l, g, h), D) 0 0)

with 0 0

D=(

0

Ik−q−s−1

) ∈ Mk−q−s (ℝ),

C1 = (u1 0) ∈ Mr,k−q−s (ℝ), where u1 = t (0, . . . , 0, x1 , 0, . . . , 0) ∈ ℝr and x1 is the i0 coordinate. Let Cs = (us 0) ∈ Ms,k−q−s (ℝ) with us = t (a1 , . . . , as ) ∈ ℝs , Cq = (uq 0) ∈ Mq,k−q−s (ℝ)

with uq = t (as+1 , . . . , as+q ) ∈ ℝq

and Cp = (up 0) ∈ Mp,k−q−s (ℝ) with up = t (aq+s+1 , . . . , aq+s+p ) ∈ ℝp .

166 | 4 The deformation space for nilpotent Lie groups Let C2 = (u2 0) ∈ Mm−q−s−p,k−q−s (ℝ), u2 = t (aq+s+p+1 , . . . , ai0 −1 , x2 , ai0 +1 , . . . , am ) ∈ 1 bi (X0 , L1 ). Then ℝm−q−s−p such that x2 ≠ x1 and ai = x2 −x α Is 0 (0 ( M2 = ( (0 (0 0 (0

0 Iq 0 0 0 0 0

Cs Cq Cp ) ) C2 ) ) ∈ R (l, g, h) D) 0 0)

and G ⋅ M1 ≠ G ⋅ M2 . Let Vi be a neighborhood of Mi for i = 1, 2. For ε > 0 small enough, the elements

M1,ε

Is 0 (0 ( =( (0 (0 0 (0

0 Iq 0 0 0 0 0

0 0 0) ) C1 ) ) ∈ V1 D)

and M2,ε

ε

0 0)

Is 0 (0 ( =( (0 (0 0 (0

0 Iq 0 0 0 0 0

Cs Cq Cp ) ) C2 ) ) ∈ V2 , D) ε

0 0)

where ε Dε = ( 0

0 ) Ik−q−s−1

and

Adexp x2 −x1 X M1,ε = M2,ε . αε

0

This means that T (l, g, h) is not a Hausdorff space.

4.3 The 3-step case 4.3.1 Some preliminary results We first prove the following series of lemmas, which will be of use in the section. Lemma 4.3.1. Let V be a vector space, E and F two subspaces of V such that V = E ⊕ F. Then for any v ∈ V, (v + E) ∩ F = P(v), where P is the projection of V on F parallel to E. Proof. Let v ∈ V and write v = v1 + v2 with v1 ∈ E and v2 ∈ F. Then P(v) = v2 and v + E = v2 + E. Let u ∈ (v + E) ∩ F, then there exists w ∈ E such that u = v2 + w and we

4.3 The 3-step case

| 167

have u ∈ F ⇒ v2 + w ∈ F ⇒ w ∈ F ⇒ w ∈ E ∩ F = {0}. Thus, u = v2 . Lemma 4.3.2. Let F, K be two finite-dimensional vector spaces and B = {e1 , . . . , em } a basis of F. (1) If φ ∈ L (F, K) of rank t > 0, then there exists {ej1 , . . . , ejt } a subset of B such that Im φ = ℝ-span{φ(ej1 ), . . . , φ(ejt )}. (2) Let S = {ej1 , . . . , ejt } be a subset of B , then the set A(S) = {φ ∈ L (F, K) | dim ℝ-span{φ(ej1 ), . . . , φ(ejt )} = t} is open in L (F, K). Proof. (1) As {φ(e1 ), . . . , φ(em )} is a generating family of Im(φ), we can extract from this family a basis of Im(φ). (2) Let r = dim(K), fix a basis B ′ of K and identify L (F, K) to the set of matrices Mr,m (ℝ) as a topological space. In this context, the set A(S) is identified to A′ (S) = {M ∈ Mr,m (ℝ) | rk(M ′ ) = t}, where M ′ ∈ Mr,t (ℝ) is the matrix obtained from M by deleting all the columns of index k ∉ {j1 , . . . , jt }. Let now J(t, r) = {(k1 , . . . , kt ) ∈ ℕt | 1 ≤ k1 < ⋅ ⋅ ⋅ < kt ≤ r}. For α = (k1 , . . . , kt ) and M ′ ∈ Mr,t (ℝ), we denote by Mα′ the square matrix obtained from M ′ by deleting all the lines of index k ∉ {k1 , . . . , kt }. Then the condition rk(M ′ ) = t is equivalent to 2

∑ [det(Mα′ )] ≠ 0,

α∈J(t,r)

which proves that A′ (S) is open and, therefore, A(S) is also open. Lemma 4.3.3. Let F be a finite-dimensional vector space, V a subspace of F and t = dim F −dim V. For all integer n, let Sn = {u1,n , . . . , ut,n } be a family of linearly independent vectors in F such that: (1) F = ℝ-span(Sn ) ⊕ V. (2) For all 1 ≤ i ≤ t, the sequence (ui,n )n converges to some vector ui . (3) S = {u1 , . . . , ut } is formed by linearly independent vectors and F = ℝ-span(S) ⊕ V.

168 | 4 The deformation space for nilpotent Lie groups Let Pn denote the projection of F on V parallel to ℝ-span(Sn ) and P the projection of F on V parallel to ℝ-span(S). Then the sequence (Pn )n converges in L (V) to P. Proof. Let m = dim F and B = {e1 , . . . , em } a basis of F such that {et+1 , . . . , em } is a basis of V. By the hypothesis (1), the set Bn = {u1,n , . . . , ut,n , et+1 , . . . , em } is a basis of F for all n and the matrix of Pn in the basis Bn is 0ℝ-span(Sn ) 0

Q=(

0 ). IdV

If PBBn is the transition base matrix, then the matrix of Pn in B is −1 Qn = PBBn QPBB . n

Now by (2) and (3), (PBBn )n converges to PBB′ , where B ′ = {u1 , . . . , ut , et+1 , . . . , en }. −1 Then (Qn )n converges to the matrix Q′ = PBB′ QPBB ′ , which is the matrix of P in B . Lemma 4.3.4. Let V, W be two finite-dimensional vector spaces, B = {e1 , . . . , en } a basis of V and f : V → W a linear map such that {f (e1 ), . . . , f (ek )} is a basis of Im f . Assume that f (ek+j ) = α1,j f (e1 ) + ⋅ ⋅ ⋅ + αk,j f (ek ),

1 ≤ j ≤ n − k.

Then the family of vectors uj = ek+j − α1,j e1 − ⋅ ⋅ ⋅ − αk,j ek ,

1≤j ≤n−k

is a basis of ker f . Proof. Clearly, the family {uj , 1 ≤ j ≤ n − k} is a family of linearly independent vectors and we have f (uj ) = 0 for all 1 ≤ j ≤ n − k. As dim ker f = n − k, the result follows. Lemma 4.3.5. Let V, W be two finite-dimensional vector spaces, B = {e1 , . . . , em } a basis of V and (fn : V → W)n a sequence of linear maps such that: (1) (fn )n converges to a linear map f : V → W. (2) The family {fn (e1 ), . . . , fn (ek )} is a basis of Im fn for all n. (3) The family {f (e1 ), . . . , f (ek )} is a basis of Im f . Assume that for all n ≥ 0 and 1 ≤ j ≤ m − k, we have n n fn (ek+j ) = α1,j fn (e1 ) + ⋅ ⋅ ⋅ + αk,j fn (ek )

4.3 The 3-step case

| 169

and f (ek+j ) = α1,j f (e1 ) + ⋅ ⋅ ⋅ + αk,j f (ek ). n Then for all 1 ≤ j ≤ n − k and 1 ≤ l ≤ k, the sequence (αl,j )n converges to αl,j .

Proof. As (fn )n converges to f , we have (fn (ek+j ) − f (ek+j ))n converges to zero. Let now dim W = r and let u1 , . . . , ur−k ∈ W be such that B ′ = {f (e1 ), . . . , f (ek ), u1 , . . . , ur−k } is a basis of W. As (fn (ei ))n converges to f (ei ) for all 1 ≤ i ≤ k, there exists N > 0 such that for all n > N the family {fn (e1 ), . . . , fn (ek ), u1 , . . . , ur−k } is also a basis of W. Let Sn = {fn (e2 ), . . . , fn (ek ), u1 , . . . , ur−k }, F = ℝ-span{f (e1 )} and qn the projection of W on F parallel to ℝ-span(Sn ). Then by Lemma 4.3.3, (qn )n converges to the projection q of W on F parallel to ℝ-span(S), where S = {f (e2 ), . . . , f (ek ), u1 , . . . , ur−k } n and (qn (fn (ek+j )))n converges to q(f (ek+j )). Note that qn (fn (ek+j )) = qn (α1,j fn (e1 )) = n α1,j qn (fn (e1 )) and q(f (ek+j )) = α1,j f (e1 ). As (α1,j qn (fn (e1 )))n converges to α1,j f (e1 ), the n sequence ((α1,j − α1,j )qn (fn (e1 )))n converges to zero. But (qn (fn (e1 )))n converges to the n nonzero vector f (e1 ), then (α1,j )n converges to α1,j . Using the same argument, we can n show that (αl,j )n converges to αl,j for all 1 ≤ l ≤ k and 1 ≤ j ≤ n − k. Lemma 4.3.6. Let (xn )n be a sequence in ℝq such that: (1) Any subsequence of (xn )n contains a convergent subsequence. (2) Two convergent subsequences of (xn )n converge to the same element. Then (xn )n is convergent. Proof. Suppose that (xn )n is not bounded, then for every integer k there exists (xnk )nk such that ‖xnk ‖ > k. Then obviously (xn )n contains a subsequence (xnk )nk such that lim ‖xnk ‖ = +∞, then (xnk )nk does not have convergent subsequence. Thus, by (1), (xn )n is bounded. Let (xnk )nk be a convergent subsequence of (xn )n , which converges to y and let A > 0 such that ‖xn − y‖ < A for all n that is (xn )n belongs to the closed ball B(y, A) of center y and radius A. Let U be a neighborhood of y. If B(y, A) \ U contains an infinite terms of (xn )n , then we can find a subsequence, which converges to y′ ≠ y and then by (2) only finite terms of (xn )n are not in U; thus, (xn )n converges to y. Lemma 4.3.7. Let (fn : ℝq → ℝm )n be a sequence of linear maps and (xn )n a sequence in ℝq such that: (1) (fn )n converges to a linear map f : ℝq → ℝm . (2) For all n, fn and f are injective. (3) There exists x ∈ ℝq such that (fn (xn ))n converges to f (x). Then (xn )n converges to x. Proof. If for all N > 0, there exists n > N such that xn = 0 then (xn )n contains a subsequence (xnk )nk such that xnk = 0 for all k and we have fnk (xnk ) = 0 then (fn (xn ))n

170 | 4 The deformation space for nilpotent Lie groups converges to zero. Let (xnk )nk be a subsequence of (xn )n such that xnk ≠ 0 for all k1 1 1 1 and we have to prove that (xnk )nk converges to zero. Note that 1

1

fnk (xnk ) = ‖xnk ‖fnk ( 1

1

1

1

xnk

1

‖xnk ‖

).

1

xn

As the sequence ( ‖x k1 ‖ )nk is bounded, we can assume that it converges to some y ≠ 0. nk

1

1

Now suppose that (‖xnk ‖)nk does not converge to zero, then up to the choice of a subse1 1 quence, we can assume that (‖xnk ‖)nk converges to some a ∈ ]0, +∞] then (fnk (xnk ))nk 1 1 1 1 1 converges to af (y) ≠ 0 because y ≠ 0 and f is injective, which is a contradiction. Now assume that there exists N > 0 such that xn ≠ 0 for all n > N. In this case, we have to show that (xn )n satisfies the conditions (1) and (2) of Lemma 4.3.6. Let (xn1 )n1 be a subsequence of (xn )n , then we can find a subsequence (xn2 )n2 of (xn1 )n1 such that x

x

( ‖xn2 ‖ )n2 converges to some y ≠ 0 and using (1), we see that (fn2 ( ‖xn2 ‖ ))n2 converges to n2

n2

f (y). As

(fn2 (xn2 ))n = (‖xn2 ‖fn2 ( 2

xn2 ‖xn2 ‖

))

n2

(x)‖ converges to f (x), we deduce that (‖xn2 ‖)n2 converges to ‖f . In particular, the se‖f (y)‖ quence (xn2 )n2 is bounded and contains a convergent subsequence. This shows that any subsequence of (xn )n contains a convergent subsequence. Let (xn,1 )n,1 and (xn,2 )n,2 be two convergent subsequences of (xn )n such that (xn,1 )n,1 converges to y1 and (xn,2 )n,2 converges to y2 . Then (fn,1 (xn,1 ))n,1 and (fn,2 (xn,2 ))n,2 converge to f (y1 ) and f (y2 ), respectively. By (3), f (y1 ) = f (y2 ). Then f (y1 − y2 ) = 0 and using (2) we deduce that y1 = y2 . Then two convergent subsequences of (xn )n converge to the same element. Using Lemma 4.3.6, we conclude that (xn )n is convergent.

4.3.2 On the quotient space Hom(l, g)/G Describing Hom(l, g) We assume henceforth that g is a 3-step nilpotent Lie algebra and l a subalgebra of g. Let g0 = [g, [g, g]] and l0 = [l, [l, l]]. We consider the decompositions g = g0 ⊕ g1 ⊕ g2

and

l = l0 ⊕ l1 ⊕ l2 ,

(4.37)

where g1 (resp., l1 ) designates a subspace of g (of l, resp.) such that g0 ⊕ g1 = [g, g] (resp., l0 ⊕ l1 = [l, l]). g2 (resp., l2 ) is a subspace of g (of l, resp.) supplementary to [g, g] (to [l, l], resp.) in g (in l, resp.).

4.3 The 3-step case |

171

Obviously, we can see that g0 (resp., l0 ) lies in the center of g (of l, resp.). Any φ ∈ L (l, g) can be written as Aφ φ = ( Iφ Jφ

Bφ Dφ Kφ

Cφ Eφ ) , Fφ

(4.38)

where Aφ ∈ L (l0 , g0 ), Bφ ∈ L (l1 , g0 ), Cφ ∈ L (l2 , g0 ), Iφ ∈ L (l0 , g1 ), Dφ ∈ L (l1 , g1 ), Eφ ∈ L (l2 , g1 ), Jφ ∈ L (l0 , g2 ), Kφ ∈ L (l1 , g2 ) and Fφ ∈ L (l2 , g2 ). For each φ ∈ L (l, g), we define φ1 ∈ L (l, g) by Aφ φ1 = ( 0 0

Bφ Dφ 0

0 Eφ ) . Fφ

(4.39)

We first remark the following assertion. Lemma 4.3.8. An element φ ∈ L (l, g) is a Lie algebra homomorphism if and only if Iφ = 0, Jφ = 0, Kφ = 0 and φ1 ∈ Hom(l, g). Proof. We point out first that if φ ∈ Hom(l, g), then φ(l0 ) = φ([l, [l, l]]) = [φ(l), [φ(l), φ(l)]] ⊂ g0 , in particular Iφ = 0 and Jφ = 0. Now φ(l1 ) ⊂ φ([l, l]) = [φ(l), φ(l)] ⊂ [g, g], so Kφ = 0 and Aφ φ=(0 0

Bφ Dφ 0

Cφ Eφ ) . Fφ

(4.40)

Let us take φ ∈ L (l, g) with Iφ = 0, Jφ = 0, Kφ = 0. Then for each x = x0 + x1 + x2 and x ′ = x0′ + x1′ + x2′ ∈ l where xi , xi′ ∈ li , i = 0, 1, 2, we have the following: [φ(x), φ(x′ )] = [Aφ (x0 ) + Bφ (x1 ) + Dφ (x1 ) + Cφ (x2 ) + Eφ (x2 ) + Fφ (x2 ), Aφ (x0′ ) + Bφ (x1′ ) + Dφ (x1′ ) + Cφ (x2′ ) + Eφ (x2′ ) + Fφ (x2′ )] = [Dφ (x1 ) + Eφ (x2 ) + Fφ (x2 ), Dφ (x1′ ) + Eφ (x2′ ) + Fφ (x2′ )] = [φ1 (x), φ1 (x′ )].

(4.41)

On the other hand, φ([x, x′ ]) = φ([x0 + x1 + x2 , x0′ + x1′ + x2′ ]) = φ([x1 + x2 , x1′ + x2′ ])

= φ([x1 , x2′ ]) + φ([x2 , x1′ ]) + φ([x2 , x2′ ])

= Aφ ([x1 , x2′ ]) + Aφ ([x2 , x1′ ]) + (Aφ + Bφ + Dφ )([x2 , x2′ ]) = φ1 ([x, x′ ]).

(4.42)

172 | 4 The deformation space for nilpotent Lie groups Conversely, let Iφ = 0, Jφ = 0, Kφ = 0 and φ1 ∈ Hom(l, g), then φ is as in (4.40). Hence, φ([x, x′ ]) = φ1 ([x, x ′ ])

by (4.42)

= [φ1 (x), φ1 (x )] ′

= [φ(x), φ(x′ )] by (4.41). Then for each φ ∈ L (l, g) with Iφ = 0, Jφ = 0, Kφ = 0, we have φ ∈ Hom(l, g) if and only if φ1 ∈ Hom(l, g). The G-action on Hom(l, g) For any X ∈ g, the adjoint representation adX can be written making use the decomposition (4.37) as 0 adX = (0 0

Σ1,2 (X) 0 0

Σ1,3 (X) Σ2,3 (X)) 0

(4.43)

for some maps Σ1,2 : g → L (g1 , g0 ), Σ1,3 : g → L (g2 , g0 ) and Σ2,3 : g → L (g2 , g1 ). The adjoint representation Adexp(X) reads therefore 1 Adexp(X) = 𝕀+ adX + ad2X 2 Ig0 Σ1,2 (X) =(0 Ig1 0 0

Σ1,3 (X) + 21 Σ1,2 (X)Σ2,3 (X) ). Σ2,3 (X) Ig2

Here, Ig0 , Ig1 and Ig2 denote the identity maps of g0 , g1 and g2 , respectively. The group G acts on Hom(l, g) through the following law: g ⋅ φ = Adg ∘φ Aφ =(0 0

Bφ + Σ1,2 (X)Dφ Dφ 0

Cφ + Σ1,2 (X)Eφ + Σ1,3 (X)Fφ + 21 Σ1,2 (X)Σ2,3 (X)Fφ ), Eφ + Σ2,3 (X)Fφ Fφ

where Aφ , Bφ , Cφ , Dφ , Eφ and Fφ are as in formula (4.38), g = exp(X) and Σ1,2 , Σ1,3 and Σ2,3 are as in (4.43). Let now Hom1 (l, g) := {φ ∈ Hom(l, g) | Cφ = 0}.

(4.44)

4.3 The 3-step case |

173

By Lemma 4.3.8, the correspondence φ 󳨃→ φ1 gives a map: Hom(l, g) → Hom1 (l, g). Then G also acts on Hom1 (l, g) as follows: Aφ g ∗ φ1 = ( 0 0

Bφ + Σ1,2 (X)Dφ Dφ 0

0 Eφ + Σ2,3 (X)Fφ ) . Fφ

(4.45)

In other words, g ∗ φ1 is defined by (g ⋅ φ1 )1 where (g ⋅ φ1 ) ∈ Hom(l, g). One can easily check that g∗φ1 defines a group action of G on Hom1 (l, g). Hence, G acts on Hom1 (l, g)× L (l2 , g0 ) as 1 g ⋅ (φ1 , Cφ ) = (g ∗ φ1 , Cφ + Σ1,2 (X)Eφ + Σ1,3 (X)Fφ + Σ1,2 (X)Σ2,3 (X)Fφ ). 2

(4.46)

We first have the following. Lemma 4.3.9. The map ψ : Hom(l, g) φ

󳨀→ 󳨃󳨀→

Hom1 (l, g) × L (l2 , g0 )

(φ1 , Cφ )

is a G-equivariant homeomorphism, where φ1 is as in (4.39). Proof. The fact that ψ is a well-defined homeomorphism comes directly from Lemma 4.3.8. Let g = exp(X) ∈ G and φ ∈ Hom(l, g), then ψ(g ⋅ φ) = ψ(Adg ∘φ)

1 = (g ∗ φ1 , Cφ + Σ1,2 (X)Eφ + Σ1,3 (X)Fφ + Σ1,2 (X)Σ2,3 (X)Fφ ) 2

= g ⋅ ψ(φ), which proves the lemma.

Decomposition of Hom1 (l, g) Now we consider the linear subspace Δ of L (l, g) defined by 󵄨󵄨 A = 0, I = 0, J = 0, D = 0, 󵄨󵄨 φ φ φ φ Δ = {φ ∈ L (l, g) 󵄨󵄨󵄨 } ≅ L (l1 , g0 ) × L (l2 , g1 ). 󵄨󵄨 K = 0, C = 0 and F = 0 󵄨 φ φ φ

174 | 4 The deformation space for nilpotent Lie groups For φ1 ∈ Hom1 (l, g), we consider the linear map lφ1 : g

󳨀→

Δ

X

󳨃󳨀→

0 (0 0

Σ1,2 (X)Dφ1 0 0

0 Σ2,3 (X)Fφ1 ) . 0

Then from equation (4.45) of the definition of the action of G on Hom1 (l, g), we obtain immediately the following description of the orbits in Hom1 (l, g). Lemma 4.3.10. The orbit G ∗ φ1 = φ0 + (Nφ1 + Im(lφ1 )), where Aφ1 φ0 = ( 0 0

0 Dφ1 0

0 0) Fφ1

and

0 Nφ1 = (0 0

Bφ1 0 0

0 Eφ1 ) . 0

Let m = dim Δ and q = dim g. For t = 0, . . . , q, we define the sets Homt1 (l, g) := {φ1 ∈ Hom1 (l, g) | rk(lφ1 ) = t}. Then clearly, q

Hom1 (l, g) = ⋃ Homt1 (l, g). t=0

(4.47)

We fix a basis {e1 , . . . , em } of Δ and let I(m, m − t) = {(i1 , . . . , im−t ) ∈ ℕm−t | 1 ≤ i1 < ⋅ ⋅ ⋅ < im−t ≤ m}. For β = (i1 , . . . , im−t ) ∈ I(m, m − t), we consider the subspace Vβ := ⨁m−t j=1 ℝeij and for

any φ1 ∈ Homt1 (l, g), let Pφ1 : Δ → Δ/ Im(lφ1 ) and

Homt1,β (l, g) := {φ1 ∈ Homt1 (l, g) | det(Pφ1 (ei1 ), . . . , Pφ1 (eim−t )) ≠ 0}. Then we have the following. Lemma 4.3.11. For each t = 0, . . . , q = dim g, the family {Homt1,β (l, g)}β∈I(m,m−t) of subsets in Homt1 (l, g) gives an open covering of Homt1 (l, g).

Proof. We know that for all φ1 ∈ Homt1 (l, g), the set Im(lφ1 ) is a linear subspace of Δ of dimension t. There exists therefore (i1 , . . . , im−t ) ∈ I(m, m − t) such that the family {Pφ1 (ei1 ), . . . , Pφ1 (eim−t )} forms a basis of Δ/ Im(lφ1 ), and consequently det(Pφ1 (ei1 ), . . . , Pφ1 (eim−t )) ≠ 0.

4.3 The 3-step case |

175

This shows that Homt1 (l, g) = ⋃β∈I(m,m−t) Homt1,β (l, g). Now det(Pφ1 (ei1 ), . . . , Pφ1 (eim−t )) ≠ 0 if and only if the family {Pφ1 (ei1 ), . . . , Pφ1 (eim−t )} is a basis of Δ/ Im lφ1 , which is equivalent to Δ = Im lφ1 ⊕ Vβ . As dim Im lφ1 = t, we get by [Lemma 4.3.2(1)] that there exists (j1 , . . . , jt ) ∈ I(q, t) such that the family {lφ1 (Yj1 ), . . . , lφ1 (Yjt ), ei1 , . . . , eim−t } forms a basis of Δ, or similarly 2

[det(lφ1 (Yj1 ), . . . , lφ1 (Yjt ), ei1 , . . . , eim−t )] ≠ 0.

∑ (j1 ,...,jt )∈I(k,t)

Then Homt1,β (l, g) 󵄨󵄨 2 󵄨 = {φ1 ∈ Homt1 (l, g) 󵄨󵄨󵄨 [det(lφ1 (Yj1 ), . . . , lφ1 (Yjt ), ei1 , . . . , eim−t )] ≠ 0}, ∑ 󵄨󵄨 (j1 ,...,jt )∈I(k,t) which is open by continuity of the determinant. Proposition 4.3.12. We have q

Hom1 (l, g) = ⋃

⋃

t=0 β∈I(m,m−t)

Homt1,β (l, g)

as a union of G-invariant subsets, where G acts on Hom1 (l, g) as in (4.45). Proof. The decomposition is given by Lemma 4.3.11 and equation (4.47). To see the G-invariance, observe that Dφ1 = Dg∗φ1 and Fφ1 = Fg∗φ1 , which means that lφ1 = lg∗φ1 and Pφ1 = Pg∗φ1 for all φ1 ∈ Hom1 (l, g) and g ∈ G. Then for all β ∈ I(m, m − t) and 0 ≤ t ≤ q the set Homt1,β (l, g) is G-invariant. Let us fix t = 0, . . . , q and β = (i1 , . . . , im−t ) ∈ I(m, m − t). Recall that Vβ is a subspace of Δ spanned by {eik }k=1,...,m−t . We define the subset Mβt (l, g) of Homt1,β (l, g) by t

t

Mβ (l, g) = {φ1 ∈ Hom1,β (l, g) | Nφ1 ∈ Vβ }

and we consider the map πβt : Homt1,β (l, g)/G G ∗ φ1

t

󳨀→

Mβ (l, g)

󳨃󳨀→

φ0 + Pφ−11 |Vβ (Nφ1 + Im(lφ1 )).

We next prove the following lemmas. Lemma 4.3.13. For each φ1 ∈ Homt1,β (l, g), the intersection of the G-orbit G ∗ φ1 and

Mβt (l, g) in Homt1,β (l, g) is the one set {φ0 + Pφ−11 |Vβ (Nφ1 + Im(lφ1 ))}. In particular, the

176 | 4 The deformation space for nilpotent Lie groups map πβt : Homt1,β (l, g)/G 󳨀→ Mβt (l, g);

G ∗ φ1 󳨃󳨀→ φ0 + Pφ−11 |Vβ (Nφ1 + Im(lφ1 ))

is well-defined. Proof. Let φ1 = φ0 + Nφ1 ∈ Homt1,β (l, g). Then from Lemma 4.3.10 the orbit G ∗ φ1 = φ0 + (Nφ1 + Im lφ1 ) and φ0 + Pφ−11 |Vβ (Nφ1 + Im lφ1 ) = φ0 + (Nφ1 + Im lφ1 ) ∩ Vβ ∈ Mβt (l, g). Thus, πβt is well-defined. Note now that the intersection of G ∗ φ1 and Mβt (l, g) in

Homt1,β (l, g) is not empty as it contains πβt (G∗φ1 ). Let φ1 , φ′1 be two elements in the intersection. Then there exist v, w ∈ Im(lφ1 ) such that φ1 = φ0 +Nφ1 +v and φ′1 = φ0 +Nφ1 +w with Nφ1 + v, Nφ1 + w ∈ Vβ . In particular, v − w ∈ Vβ ∩ Im(lφ1 ) = {0}, which means that φ1 = φ′1 . Lemma 4.3.14. The map h : Homt1,β (l, g) φ1

t

󳨀→

Mβ (l, g)

󳨃󳨀→

φ0 + Pφ−11 |Vβ (Nφ1 + Im lφ1 )

is continuous. Proof. To show this lemma, we prove the following fact. Fact 4.3.15. The map Hom1 (l, g) 󳨀→ L (g, Δ) φ1 󳨃󳨀→ lφ1

is continuous. Proof. Let (φ(n) 1 )n be a sequence, which converges to some element φ1 . Then obviously (Dφ(n) )n converges to Dφ1 and (Fφ(n) )n converges to Fφ1 . Then (lφ(n) )n converges to lφ1 . 1

1

1

Now to prove the lemma, let (φ1,n )n be a sequence in Homt1,β (l, g), which converges

to an element φ1 ∈ Homt1,β (l, g). We have to show that (h(φ1,n ))n converges to h(φ1 ).

Note first that for all φ1 ∈ Homt1,β (l, g), h(φ1 ) = φ0 +(Nφ1 +Im lφ1 )∩Vβ . As Δ = Im lφ1 ⊕Vβ , by Lemma 4.3.1, we see that h(φ1 ) = φ0 + qφ1 (Nφ1 ), where qφ1 is the projection on Vβ parallel to Im lφ1 . Let {X1 , . . . , Xn } be a basis of g, using [Lemma 4.3.2(1)], one can find Xj1 , . . . , Xjt ∈ {X1 , . . . , Xn } such that Im lφ1 = ℝ-span{lφ1 (Xj1 ), . . . , lφ1 (Xjt )}.

4.3 The 3-step case |

177

Now by Fact 4.3.15, we get the convergence of the sequence (lφ1,n )n to lφ1 . By [Lemma 4.3.2(2)], for S = {Xj1 , . . . , Xjt }, A(S) = {l ∈ L (g, Δ) | dim ℝ-span{l(Xj1 ), . . . , l(Xjt )} = t} is open in L (g, Δ). Then there exists N > 0 such that for all n > N, Im lφ1 = ℝ-span{lφ1,n (Xj1 ), . . . , lφ1,n (Xjt )}. As (lφ1,n )n converges to lφ1 , the sequence (lφ1,n (Xjk ))n converges to lφ1 (Xjk ) for all 1 ≤ k ≤ t. By Lemma 4.3.3, let qn be the projection of Δ on Vβ parallel to Im(lφ1,n ), the sequence (qn )n converges to the projection qφ1 of Δ on Vβ parallel to Im lφ1 . Finally, as (φ0,n )n converges to φ0 and (Nφ1,n )n converges to Nφ1 , we get (h(φ1,n ))n converges to h(φ1 ). Lemma 4.3.16. The map πβt : Homt1,β (l, g)/G 󳨀→ Mβt (l, g) defined above is a homeomorphism. Proof. To see that πβt is surjective, observe that Mβt (l, g) ⊂ Homt1,β (l, g) and for φ1 ∈

Mβt (l, g) we have πβt (G ∗ φ1 ) = φ1 . Let φ1 , ξ1 ∈ Homt1,β (l, g) such that πβt (G ∗ φ1 ) =

πβt (G ∗ ξ1 ). Then obviously φ0 = ξ0 and

Pφ−11 |Vβ (Nφ1 + Im lφ1 ) = Pφ−11 |Vβ (Nφ1 + Im lφ1 ). As lφ1 depends only on φ0 , we deduce that lφ1 = lξ1 , which implies that Pφ−11 |Vβ = Pξ−11 |Vβ .

Hence, Nφ1 + Im lφ1 = Nξ1 + Im lξ1 and in particular G ∗ φ1 = G ∗ ξ1 . Thus, πβt is injective. Now the following diagram commutes: Homt1,β (l, g) π

h

?

Homt1,β (l, g)/G

πβt

? ? M t (l, g) β

where h(φ1 ) = φ0 + Pφ−11 |Vβ (Nφ1 + Im lφ1 ). Since by Lemma 4.3.14, h is continuous and

since π is open then πβt is continuous. The quotient canonical map (πβt )−1 = π|M t (l,g) is continuous and then the map πβt is a homeomorphism.

β

Corollary 4.3.17. For all t = {0, . . . , q}, the collection Stβ = (πβt , Homt1,β (l, g)/G)β∈I(m,m−t) forms a family of local sections of the canonical surjection π t : Homt1 (l, g) → Homt1 (l, g)/G. In particular, 󵄨󵄨 φ ∈ Homt (l, g), 󵄨󵄨 1 1,β πβt (Homt1,β (l, g)/G) = {φ1 ∈ Hom1 (l, g) 󵄨󵄨󵄨 }. 󵄨󵄨 N ∈ V 󵄨 φ1 β

178 | 4 The deformation space for nilpotent Lie groups Decomposition of Hom(l, g) Let φ1 ∈ Homt1,β (l, g) and Gφ1 = {g ∈ G | g ∗ φ1 = φ1 } be the isotropy group of φ1 . The group Gφ1 acts on {φ1 }× L (l2 , g0 ) through the following law: exp(X) ⋅ (φ1 , C) = (φ1 , C + Σ1,2 (X)Eφ1 + Σ1,3 (X)Fφ1 ), where Σ1,2 and Σ1,3 are as in (4.43). Indeed, for any g = exp(X) ∈ Gφ1 , we have g ∗ φ1 = φ1 , then Σ1,2 (X)Dφ1 = 0 and Σ2,3 (X)Fφ1 = 0. We get therefore from (4.46), g ⋅ (φ1 , C) = (φ1 , C + Σ1,2 (X)Eφ1 + Σ1,3 (X)Fφ1 ). Let gπ t (G∗φ1 ) = log(Gπ t (G∗φ1 ) ) and fφ1 be the linear map defined by β

β

fφ1 : gπ t (G∗φ1 ) β

X

󳨀→

L (l2 , g0 )

󳨃󳨀→

Σ1,2 (X)Eπ t (G∗φ1 ) + Σ1,3 (X)Fπ t (G∗φ1 ) . β

β

Then the range of fφ1 is a linear subspace of L (l2 , g0 ) and we can see immediately that we have the following. Lemma 4.3.18. For any g ∈ G, we have fφ1 = fg∗φ1 . In addition, Gπ t (G∗φ1 ) ⋅ (πβt (G ∗ φ1 ), C) = (πβt (G ∗ φ1 ), C + Im(fφ1 )). β

sets

Let m′ = dim(L (l2 , g0 )) and q′ = dim(gπ t (G∗φ1 ) ). For t ′ = 0, . . . , q′ , we define the β

t,t Hom1,β (l, g) = {φ1 ∈ Homt1,β (l, g) | rk(fφ1 ) = t ′ }. ′

Then clearly Homt1,β (l, g)

q′

= ⋃ Homt,t 1,β (l, g). ′

(4.48)

t ′ =0

′ ′ ′ ′ ′ ′ ′ Let us fix a basis {e1′ , . . . , em ′ } of L (l2 , g0 ). For β = (i1 , . . . , im′ −t ′ ) ∈ I(m , m − t ) and

−t ′ φ1 ∈ Homt,t (l, g), we consider the subspace Vβ′ := ⨁m j=1 ℝei′ , the quotient map 1,β ′

′

′

j

Pφ′ 1 : L (l2 , g0 ) → L (l2 , g0 )/ Im(fφ1 )

4.3 The 3-step case |

179

and the set t,t Homt,t (l, g) = {φ1 ∈ Hom1,β (l, g) | det(Pφ′ 1 (ei′′ ), . . . , Pφ′ 1 (ei′′ 1,β,β′ ′

′

1

m′ −t ′

)) ≠ 0}.

Then we get the following. Lemma 4.3.19. For each t = 0, . . . , q = dim g, t ′ = 0, . . . , q′ = dim(gπ t (G∗φ1 ) ) and β ∈ β

t,t t,t I(m, m − t), the family {Hom1,β,β ′ (l, g)}β′ ∈I(m′ ,m′ −t ′ ) of subsets in Hom1,β (l, g) gives an open ′

′

covering of Homt,t (l, g). 1,β ′

(l, g), the set Im(fφ1 ) is a linear subspace of L (l2 , g0 ) of diProof. For all φ1 ∈ Homt,t 1,β ′

′ ′ ′ ′ mension t ′ . There exists therefore (i1′ , . . . , im ′ −t ′ ) ∈ I(m , m − t ) such that the family ′ ′ ′ ′ {Pφ1 (ei′ ), . . . , Pφ1 (ei′ )} forms a basis of L (l2 , g0 )/ Im(fφ1 ), and consequently 1

m′ −t ′

det(Pφ′ 1 (ei′′ ), . . . , Pφ′ 1 (ei′′ 1

m′ −t ′

)) ≠ 0.

(l, g) = ⋃β′ ∈I(m′ ,m′ −t ′ ) Homt,t (l, g). This shows that Homt,t 1,β 1,β,β′ ′

′

Now to prove that Homt,t (l, g) is open in Homt,t (l, g), we need the following 1,β 1,β,β′ facts. ′

′

Lemma 4.3.20. Let φ1 ∈ Homt1,β (l, g) and assume that πβt (G ∗ φ1 ) = ψ1 . Then gψ1 = ker lψ1 . Proof. gψ1 = {X ∈ g | exp(X) ∗ ψ1 = ψ1 } = {X ∈ g | ψ1 + lψ1 (X) = ψ1 } = {X ∈ g | lψ1 (X) = 0} = ker lψ1 .

Let B = {X1 , . . . , Xq } be a basis of g, φ1 ∈ Homt1,β (l, g) and ψ1 = πβt (G ∗ φ1 ). For γ = (j1 , . . . , jt ) ∈ I(q, t) such that Im lψ1 = ℝ-span{lψ1 (Xj1 ), . . . , lψ1 (Xjt )}, we define a linear map lφ1 ,γ : ℝq−t → g given by t

lφ1 ,γ (ui ) = Xsi − ∑ αr,si Xjr , r=1

where {u1 , . . . , uq−t } is the canonical basis of ℝq−t , {s1 < ⋅ ⋅ ⋅ < sq−t } = {1, . . . , q}\{j1 , . . . , jt } and t

lψ1 (Xsi ) = ∑ αr,si lψ1 (Xjr ) r=1

∀ 1 ≤ i ≤ q − r.

180 | 4 The deformation space for nilpotent Lie groups Fact 4.3.21. We have Im lφ1 ,γ = gψ1 . Proof. By Lemma 4.3.4 and Lemma 4.3.20, we have Im(lφ1 ,γ ) = ker lψ1 = gψ1 . Consider now the map lφ1 ,γ : ℝq−t → L (l2 , g0 ) defined by lφ1 ,γ = fφ1 ∘ lφ1 ,γ . Then we have the following result. Lemma 4.3.22. We have: (1) lφ1 ,γ is a linear map. (2) Im(lφ1 ,γ ) = Im fφ1 . Proof. As lφ1 ,γ and fφ1 are linear maps, the map lφ1 ,γ is also linear. Now by Fact 4.3.21, Im(lφ1 ,γ ) = fφ1 (Im lφ1 ,γ ) = fφ1 (gψ1 ) = Im(fφ1 ). Now for γ = (j1 , . . . , jt ) ∈ I(q, t), let t,t t,t Hom1,β,β ′ (γ) = {φ1 ∈ Hom1,β,β′ (l, g) | rk(lφ (Xj ), . . . , lφ (Xj )) = t}. 1 1 1 t ′

′

Then obviously, t,t Hom1,β,β (γ). ⋃ Homt,t ′ (l, g) = 1,β,β′ ′

′

γ∈I(q,t)

t,t t,t To conclude that Hom1,β,β ′ (l, g) is open in Hom1,β (l, g), we have to show that for all ′

′

t,t t,t γ ∈ I(q, t), the set Hom1,β,β ′ (γ) is open in Hom1,β (l, g). First, note that ′

′

{ t,t ′ t,t ′ Hom1,β,β ′ (γ) = {φ1 ∈ Hom1,β (l, g) {

󵄨󵄨 rk(l (X ), . . . , l (X )) = t, 󵄨󵄨 } φ1 j1 φ1 jt 󵄨󵄨 . 󵄨󵄨 󵄨󵄨 det(Pφ′ 1 (ei′′ ), . . . , Pφ′ 1 (ei′′ )) ≠ 0 } 󵄨 ′ ′ 1 } m −t

The set Aβt,t (γ) = {φ1 ∈ Homt,t 1,β (l, g) | rk(lφ1 (Xj1 ), . . . , lφ1 (Xjt )) = t} ′

′

t,t is open in Hom1,β (l, g) and ′

t,t t,t ′ ′ ′ ′ Hom1,β,β ′ (γ) = {φ1 ∈ Aβ (γ) | det(Pφ (ei′ ), . . . , Pφ (ei′ 1 1 ′

′

1

m′ −t ′

)) ≠ 0}.

Then to obtain our result, it is sufficient to prove that Homt,t (γ) is open in At,t (γ). Inβ 1,β,β′ ′

deed, the condition det(Pφ′ 1 (ei′′ ), . . . , Pφ′ 1 (ei′′ 1

m′ −t ′

′

)) ≠ 0 is equivalent to L (l2 , g0 ) = Im fφ1 ⊕

4.3 The 3-step case |

181

Vβ′ . By Lemma 4.3.22, we get det(Pφ′ 1 (ei′′ ), . . . , Pφ′ 1 (ei′′ 1

m′ −t ′

)) ≠ 0

⇔ Im(lφ1 ,γ ) ⊕ Vβ′ = L (l2 , g0 )

⇔ ∃θ ∈ I(q − t, t ′ ),

θ = (s1 , . . . , st ′ )/

det(lφ1 ,γ (us1 ), . . . , lφ1 ,γ (us ′ ), ei′′ , . . . , ei′′ t

⇔

∑

1

m′ −t ′

) ≠ 0

[det(lφ1 ,γ (us1 ), . . . , lφ1 ,γ (us ′ ), ei′′ , . . . , ei′′ t

θ∈I(q−t,t ′ )

1

m′ −t ′

2

)] ≠ 0.

Then Homt,t (γ) 1,β,β′ ′

󵄨󵄨 ′ 2 󵄨󵄨 = {φ1 ∈ At,t ∑ [det(lφ1 ,γ (us1 ), . . . , lφ1 ,γ (us ′ ), ei′′ , . . . , ei′′ )] ≠ 0}, β (γ) 󵄨󵄨󵄨 t ′ −t ′ 1 m 󵄨 θ∈I(q−t,t ′ ) which is clearly an open subset of Aβt,t (γ). ′

As a consequence, we get the following. Proposition 4.3.23. We have the following decomposition: q

q′

Hom1 (l, g) = ⋃ ⋃

⋃

⋃

t=0 t ′ =0 β∈I(m,m−t) β′ ∈I(m′ ,m′ −t ′ )

Homt,t (l, g) 1,β,β′ ′

as a union of G-invariant subsets, where G acts on Hom1 (l, g) as in (4.45). Proof. The decomposition is given by Proposition 4.3.12, Lemma 4.3.19 and equation (4.48). We showed already that for β ∈ I(m, m − t) and 0 ≤ t ≤ q, the set Homt1,β (l, g) is ′ G-invariant. Likewise, by Lemma 4.3.18 we have fφ1 = fg∗φ1 and then Pφ′ 1 = Pg∗φ for all 1 ′ g ∈ G and φ1 ∈ Hom1 (l, g). As a consequence, for all β ∈ I(m, m − t), β ∈ I(m′ , m′ − t ′ ), ′ 0 ≤ t ≤ q and 0 ≤ t ′ ≤ q′ the set Homt,t (l, g) is G-invariant. 1,β,β′ Using the map ψ defined as in Lemma 4.3.9, we will identify in the rest of this section the set Hom(l, g) to Hom1 (l, g) × L (l2 , g0 ). Let first for β ∈ I(m, m − t), β′ ∈ I(m′ , m′ − t ′ ), 0 ≤ t ≤ q and 0 ≤ t ′ ≤ q′ , t,t t,t Homβ,β ′ (l, g) = Hom1,β,β′ (l, g) × L (l2 , g0 ). ′

Then we have the following.

′

(4.49)

182 | 4 The deformation space for nilpotent Lie groups Proposition 4.3.24. We have q

q′

Hom(l, g) = ⋃ ⋃

⋃

Homt,t (l, g) β,β′ ′

⋃

t=0 t ′ =0 β∈I(m,m−t) β′ ∈I(m′ ,m′ −t ′ )

as a union of G-invariant subsets. Proof. As in Proposition 4.3.23, for all β ∈ I(m, m − t), β′ ∈ I(m′ , m′ − t ′ ), 0 ≤ t ≤ q and t,t ′ 0 ≤ t ′ ≤ q′ , the set Hom1,β,β ′ (l, g) is G-invariant where G acts on Hom1 (l, g) as in (4.45) t,t and then the set Homβ,β ′ (l, g) is also G-invariant. ′

Let now t,t ′

t

Mβ (l, g) = {φ1 ∈ Mβ (l, g) | rk(fφ1 ) = t } ′

= {φ1 ∈ Homt1,β (l, g) | Nφ1 ∈ Vβ ; rk(fφ1 ) = t ′ }

t,t = {φ1 ∈ Hom1,β (l, g) | Nφ1 ∈ Vβ }, ′

(4.50)

and t,t ′

t,t ′

Mβ,β′ (l, g) = {φ1 ∈ Mβ (l, g) | det(Pφ1 (ei′ ), . . . , Pφ1 (ei′ ′

′

′

1

′

m′ −t ′

)) ≠ 0}

t,t = {φ1 ∈ Hom1,β (l, g) | Nφ1 ∈ Vβ , det(Pφ′ 1 (ei′′ ), . . . , Pφ′ 1 (ei′′ ′

1

m′ −t ′

(by (4.50))

)) ≠ 0}

t,t = {φ1 ∈ Hom1,β,β ′ (l, g) | Nφ ∈ Vβ } 1 ′

t,t t = Hom1,β,β ′ (l, g) ∩ Mβ (l, g). ′

(4.51)

We show next the following lemmas. t,t t,t Lemma 4.3.25. Each G-orbit in Homβ,β ′ (l, g) intersects Mβ,β′ (l, g) × Vβ′ at exactly one ′

′

t,t t element in Homβ,β ′ (l, g), if we take an element g0 ∈ G such that g0 ∗ φ1 ∈ Mβ (l, g), then ′

t,t the element in the intersection of (G ⋅ (φ1 , C)) and Mβ,β ′ (l, g) × Vβ′ can be written as ′

(πβt (G ∗ φ1 ), (Pφ′ 1 |V ′ ) (C(g0 ) + Im fφ1 )), −1

β

where 1 C(g0 ) = C + Σ1,2 (X0 )Eφ1 + Σ1,3 (X0 )Fφ1 + Σ1,2 (X0 ).Σ2,3 (X0 )Fφ1 . 2 Here, Σ1,2 , Σ1,3 and Σ2,3 are as in formula (4.43).

4.3 The 3-step case |

183

Proof. From Lemma 4.3.13, the intersection of the G-orbit G ∗ φ1 and Mβt (l, g) in Homt1,β (l, g) is the one set {φ0 + Pφ−11 |Vβ (Nφ1 + Im(lφ1 ))}, then there exist g0 = exp(X0 ) ∈ G such that g0 ∗ φ1 = φ0 + Pφ−11 |Vβ (Nφ1 + Im(lφ1 )) = πβt (G ∗ φ1 ). Now g0 ⋅ (φ1 , C) = (g0 ∗ φ1 , C(g0 )) and Gg0 ∗φ1 ⋅ (g0 ∗ φ1 , C(g0 )) = (g0 ∗ φ1 , C(g0 ) + Im fφ1 ). Thus, t,t t,t (G ⋅ (φ1 , C)) ∩ (Mβ,β ′ (l, g) × Vβ′ ) = (G ∗ φ1 ∩ Mβ,β′ (l, g), (C(g0 ) + Im fφ ) ∩ Vβ′ ) 1 ′

′

(C(g0 ) + Im fφ1 )). = (g0 ∗ φ1 , Pφ′ −1 1 |V ′ β

Let (φ1 , A), (φ′1 , A′ ) be in the intersection. Then from Lemma 4.3.13, we have φ1 = φ′1 . Let now g = exp(X) and g ′ = exp(X ′ ) in G such that g ∗ φ1 = g ′ ∗ φ1 = πβt (G ∗ φ1 ). Then g ∗ φ1 = g ′ ∗ φ1 is equivalent to g −1 g ′ ∈ Gφ1 and there exists g1 = exp(X1 ) ∈ Gφ1 such that g ′ = gg1 . Let g2 = exp(X2 ) = gg1 g −1 , then g2 ∈ Gg∗φ1 and g ′ ⋅ (φ1 , C) = gg1 ⋅ (φ1 , C) = gg1 g −1 g ⋅ (φ1 , C) = g2 g ⋅ (φ1 , C) = g2 (g ∗ φ1 , C(g)) = (g ∗ φ1 , C(g) + fφ1 (X2 )).

Thus, C(g ′ ) − C(g) = fφ1 (X2 ) ∈ Im fφ1 which is equivalent to C(g) + Im fφ1 = C(g ′ ) + Im fφ1 , which means that A = A′ . Lemma 4.3.26. Let (φ1,n )n be a sequence of Homt1,β (l, g), which converges to φ1 , πβt (G ∗

φ1,n ) = ψ1,n and πβt (G ∗ φ1 ) = ψ1 . Then there exists a convergent sequence (Xn )n in g, which converges to some element X such that ψ1,n = exp(Xn ) ∗ φ1,n

and

ψ1 = exp(X) ∗ φ1 .

Proof. Let {X1 , . . . , Xi } be a basis of g and α = (i1 , . . . , it ) ∈ I(q, t) such that Im lφ1 = lφ1 (Uα ) where Uα = ℝ-span{lφ1 (Xi1 ), . . . , lφ1 (Xit )}. As G ∗ φ1 = φ1 + Im lφ1 and πβt (G ∗ φ1 ) ∈ G∗φ1 , there exists X ∈ Uα such that ψ1 = φ1 +lφ1 (X). By Lemma 4.3.2, there exists N > 0 such that for all n > N, Im lφ1,n = lφ1,n (Uα ). Thus, there exists (Xn )n in Uα such that ψ1,n = φ1,n + lφ1,n (Xn ) for all n > N. Now from the continuity of the map φ1 󳨃→ πβt (G ∗ φ1 ) and by the convergence of (φ1,n )n to φ1 , we deduce that (φ1,n + lφ1,n (Xn ))n converges to {φ1 + lφ1 (X)} and then (lφ1,n (Xn ))n converges to lφ1 (X) (because (φ1,n )n converges to φ1 ). Let

184 | 4 The deformation space for nilpotent Lie groups lφ′ 1,n be the restriction of lφ1,n to Uα and lφ′ 1 the restriction of lφ1 to Uα . Then all the maps

lφ′ 1,n , n > N are injective and lφ′ 1 is also injective. The sequence (lφ′ 1,n (Xn ))n converges

to lφ′ 1 (X), and obviously (lφ′ 1,n )n converges to lφ′ 1 . Using Lemma 4.3.7, we conclude that (Xn )n converges to X.

Lemma 4.3.27. Let (φ1,n )n be a sequence in Homt,t (l, g), which converges to φ1 ∈ 1,β,β′ ′

Homt,t (l, g). Then: 1,β,β′ (1) There exists γ = (j1 , . . . , jt ) ∈ I(q, t) such that ′

Im lψ1 = ℝ-span{lψ1 (Xj1 ), . . . , lψ1 (Xjt )},

ψ1 = πβt (G ∗ φ1 )

and Im lψ1,n = ℝ-span{lψ1,n (Xj1 ), . . . , lψ1,n (Xjt )},

ψ1,n = πβt (G ∗ φ1,n ).

(2) The sequence (lφ1,n ,γ )n converges to lφ1 ,γ . Proof. As the quotient map φ1 󳨃→ G ∗ φ1 and πβt are continuous, the sequence {ψ1,n } converges to ψ1 . Then (lψ1,n )n converges to lψ1 . Let γ = (j1 , . . . , jt ) ∈ I(q, t) be such that Im lψ1 = ℝ-span{lψ1 (Xj1 ), . . . , lψ1 (Xjt )}. By Lemma 4.3.2, we can assume that Im lψ1,n = ℝ-span{lψ1,n (Xj1 ), . . . , lψ1,n (Xjt )}. To prove the second result, let γ = (j1 , . . . , jt ) ∈ I(q, t) satisfying (1) and for n {s1 , . . . , sq−t } = {1, . . . , q} \ {j1 , . . . , jt }, lψ1,n (Xsi ) = ∑tr=1 αr,s l (Xjr ) and lψ1 (Xsi ) = i ψ1,n

n X and vsi = Xsi − ∑tr=1 αr,si Xjr . Then by ∑tr=1 αr,si lψ1 (Xjr ). Let vn,si = Xsi − ∑tr=1 αr,s i jr Lemma 4.3.5, (vn,si )n converges to vsi . This shows that (lφ1,n ,γ )n converges to lφ1 ,γ . Now, for all 1 ≤ i ≤ q − t, we have

lφ1,n ,γ (ui ) = Σ1,2 (lφ1,n ,γ (ui ))Eψ1,n + Σ1,3 (lφ1,n ,γ (ui ))Fψ1,n . As the matrix multiplication is continuous and the maps φ1 󳨃→ Eψ1 and φ1 󳨃→ Fψ1 are continuous, we deduce that (lφ1,n ,γ (ui ))n converges to lφ1 ,γ (ui ). Thus, (lφ1,n ,γ )n converges to lφ1 ,γ .

Let qφ1 ,γ be the projection of L (l2 , g0 ) on Vβ parallel to Im(lφ1 ,γ ). Then we have the following. Lemma 4.3.28. We have h′ (φ1 , C) = (πβt (G ∗ φ1 ), qφ1 ,γ (C(g0 ))), where C(g0 ) is given in Lemma 4.3.25.

4.3 The 3-step case

| 185

Proof. By [Lemma 4.3.22(2)], we have Im(lφ1 ,γ ) = Im(fφ1 ). Then Pφ′ −1 (C(g0 ) + Im fφ1 ) = (C(g0 ) + Im(fφ1 )) ∩ Vβ′ 1 |V ′ β

= (C(g0 ) + Im(lφ1 ,γ )) ∩ Vβ′

= qφ1 ,γ (C(g0 )) by Lemma 4.3.1.

Lemma 4.3.29. The map t,t h′ : Homβ,β ′ (l, g) ′

(φ1 , C)

→ 󳨃→

t,t ′

Mβ,β′ (l, g) × Vβ′

(C(g0 ) + Im fφ1 )) (πβt (G ∗ φ1 ), Pφ′ −1 1 |V ′ β

is continuous. t,t Proof. Let (φ1,n , Cn )n be a sequence in Homβ,β ′ (l, g), which converges to an element ′

t,t ′ ′ (φ1 , C) ∈ Homβ,β ′ (l, g). To see that h is continuous, we have to show that (h ((φ1,n , Cn )))n ′

converges to h′ ((φ1 , C)). As (φ1,n )n converges to φ1 , by Lemma 4.3.27 there exists γ ∈ I(q, t) such that the sequence (lφ1,n ,γ )n converges to lφ1 ,γ . As L (l2 , g0 ) = Im(lφ1,n ,γ ) ⊕ Vβ′ , L (l2 , g0 ) = Im(lφ1 ,γ ) ⊕ Vβ′ , Im(lφ1 ,γ ) = ℝ-span{lφ1 ,γ (Xi1 ), . . . , lφ1 ,γ (Xit )}

and Im(lφ1,n ,γ ) = ℝ-span{lφ1,n ,γ (Xi1 ), . . . , lφ1,n ,γ (Xit )}, where γ = (i1 , . . . , it ). Then by Lemma 4.3.3 the sequence (qφ1,n ,γ )n converges to qφ1 ,γ . Now (Cn )n converges to C and by Lemma 4.3.26 there exists a sequence (Xn )n in g such that (exp(Xn ))n converges to exp(X). Now by Lemma 4.3.28, h′ (φ1,n , Cn ) = (πβt (G ∗ φ1,n ), qφ1,n ,γ (Cn (exp(Xn )))) and it is clear that (Cn (exp(Xn )))n converges to C(exp(X)). Then (h′ ((φ1,n , Cn )))n converges to h′ ((φ1 , C)). By Lemma 4.3.25 above, for each G-orbit O in Homt,t (l, g), there uniquely exists β,β′ ′

t,t an element AO in O ∩ (Mβ,β ′ (l, g) × Vβ′ ). We define the map ′

t,t t,t εβ,β ′ : Homβ,β′ (l, g)/G ′

′

O

t,t ′

→

Mβ,β′ (l, g) × Vβ′

󳨃→

AO .

t,t t,t Then Mβ,β ′ (l, g) × Vβ′ is a fundamental domain of the G-action on Homβ,β′ (l, g) in the sense below. ′

′

186 | 4 The deformation space for nilpotent Lie groups t,t t,t t,t Lemma 4.3.30. The map εβ,β ′ : Homβ,β′ (l, g)/G → Mβ,β′ (l, g) × Vβ′ defined above is a homeomorphism. ′

′

′

t,t ′ ′ Proof. Let (φ1 , C) ≠ (φ′1 , C ′ ) ∈ Homβ,β ′ (l, g) such that G ⋅ (φ1 , C) = G ⋅ (φ1 , C ). Then there ′

exist g0 and g0′ in G such that

t,t ′ −1 εβ,β ′ (G ⋅ (φ1 , C)) = (g0 ∗ φ1 , Pφ |V ′ (C(g0 ) + Im(fφ ))) 1 1 ′

β

and t,t ′ ′ ′ ′ ′ ′ ′ −1 εβ,β ′ (G ⋅ (φ1 , C )) = (g0 ∗ φ1 , Pφ′ |V ′ (C (g0 ) + Im(fφ′ ))). 1 ′

1

β

Since G ⋅ (φ1 , C) = G ⋅ (φ′1 , C ′ ), there exists g ∈ G such that (φ1 , C) = g ⋅ (φ′1 , C ′ ) = (g ∗ φ′1 , C ′ (g)). We thus get the following: G ∗ φ1 = G ∗ (g ∗ φ′1 ) = G ∗ φ′1 ⇐⇒ πβt (G ∗ φ1 ) = πβt (G ∗ φ′1 ) ⇐⇒ g0 ∗ φ1 = g0′ ∗ φ′1 . This means in particular that fφ1 = fφ′1 . Besides, there exists g1′ = exp(X1′ ) ∈ Gg0′ ∗φ′1 such that g1′ g0′ ⋅ (φ′1 , C ′ ) = g0 ⋅ (φ1 , C). This entails that (g0 ∗ φ1 , C(g0 )) = (g0′ ∗ φ′1 , C ′ (g0′ ) + Σ1,2 (X1′ )Eg0′ ∗φ′1 + Σ1,3 (X1′ )Fg0′ ∗φ′1 ), which is equivalent to C(g0 ) + Im fφ1 = C ′ (g0′ ) + fφ′1 (X1′ ) + Im fφ1 = C ′ (g0′ ) + Im fφ′1 . t,t t,t t,t ′ ′ Thus, εβ,β ′ (G ⋅ (φ1 , C)) = εβ,β′ (G ⋅ (φ1 , C )) and εβ,β′ is a well-defined map. We now prove ′

′

′

t,t t,t that the map εβ,β ′ is a homeomorphism. In fact, we first show that εβ,β′ is a bijection. ′

′

t,t Let (φ1 , C) and (φ′1 , C ′ ) be in Homβ,β ′ (l, g) such that ′

t,t t,t ′ ′ εβ,β ′ (G ⋅ (φ1 , C)) = εβ,β′ (G ⋅ (φ1 , C )). ′

′

Then there exist g0 , g0′ ∈ G such that ′ ′ (g0 ∗ φ1 , Pφ′ −1 (C(g0 ) + Im fφ1 )) = (g0′ ∗ φ′1 , Pφ′ −1 ′ |V (C (g0 ) + Im fφ′ )). ′ 1 |V ′ 1 β

1

β

This implies that g0 ∗ φ1 = g0′ ∗ φ′1 , which means that fφ1 = fφ′1 and Pφ′ 1 = Pφ′ ′ . Then 1 from the equality ′ ′ Pφ′ −1 (C(g0 ) + Im fφ1 ) = Pφ′ −1 ′ |V (C (g0 ) + Im fφ′ ), ′ 1 |V ′ 1 β

1

β

4.3 The 3-step case |

187

we get C(g0 ) + Im fφ1 = C ′ (g0′ ) + Im fφ′1 . Then G ∗ φ1 = G ∗ φ′1 and Gg0 ∗φ1 ⋅ (g0 ∗ φ1 , C(g0 )) = Gg0′ ∗φ′1 ⋅ (g0′ ∗ φ′1 , C ′ (g0′ )), t,t which is equivalent to G ⋅ (φ1 , C) = G ⋅ (φ′1 , C ′ ) and the map εβ,β ′ is injective. Let now ′

t,t t (ψ1 , C1 ) ∈ Mβ,β ′ (l, g) × Vβ′ . Since the map πβ is a homeomorphism, there exist φ1 ∈ ′

Homt,t (l, g) and g ′ ∈ G such that 1,β,β′ ′

t,t ′ (G ∗ φ1 ) ∩ Mβ,β ′ (l, g) = ψ1 = g ∗ φ1 . ′

Hence, there exists C ∈ L (l2 , g0 ), such that g ′ ⋅ (φ1 , C) = (g ′ ∗ φ1 , C(g ′ )), C1 = Pψ′ −1 (C(g ′ ) + Im fψ1 ) 1 |V ′ β

t,t t,t and εβ,β ′ (G ⋅ (φ1 , C)) = (ψ1 , C1 ) and then the map εβ,β′ is surjective. Now the diagram below commutes ′

′

t,t Homβ,β ′ (l, g) ′

π′

h′

?

′

εt,t ′ β,β

t,t ′ Homβ,β ′ (l, g)/G

? ? M t,t ′′ (l, g) × Vβ′ β,β

(C(g0 ) + Im(fφ1 )). Since by Lemma 4.3.29, h′ is where h′ ((φ1 , C)) = (πβt (G ∗ φ1 ), Pφ′ −1 1 |V ′ β

t,t continuous and since π ′ is open then εβ,β ′ is continuous. Now the quotient canonical map t,t (εβ,β ′) ′

−1

′

= π′

′

|M t,t ′ (l,g)×Vβ′ β,β

t,t is continuous and then the map εβ,β ′ is a homeomorphism. ′

As an immediate consequence, we get the following. Proposition 4.3.31. We have the following: Hom(l, g)/G = ⋃

⋃

0≤t≤q β∈I(m,m−t) 0≤t ′ ≤q′ β′ ∈I(m′ ,m′ −t ′ )

Homt,t (l, g)/G, β,β′ ′

(4.52)

where for all t ∈ {0, . . . , q}, t ′ ∈ {0, . . . , q′ }, β ∈ I(m, m − t) and β′ ∈ I(m′ , m′ − t ′ ), ′ ′ (l, g) is defined as in formula (4.49) and the set Homt,t (l, g)/G is homeomorphic Homt,t β,β′ β,β′ t,t to Mβ,β ′ (l, g) × Vβ′ . ′

188 | 4 The deformation space for nilpotent Lie groups Let now Hom1t,t (l, g) = ′

Homt,t (l, g) 1,β,β′ ′

⋃

β∈I(m,m−t) β′ ∈I(m′ ,m′ −t ′ )

(4.53)

and Homt,t (l, g) = Homt,t 1 (l, g) × L (l2 , g2 ). ′

′

(4.54)

We then show the following. Lemma 4.3.32. Refer to Lemma (4.53) and Corollary (4.54). Then the collection t,t t,t St,t = (εβ,β ′ , Homβ,β′ (l, g)/G) β,β′ ′

′

′

(t = 0, . . . , q and t ′ = 0, . . . , q′ )

β∈I(m,m−t) β ∈I(m′ ,m′ −t ′ ) ′

constitutes a family of local sections of the canonical surjection π t,t : Homt,t (l, g) → Homt,t (l, g)/G. ′

′

′

t,t Proof. We have to show that π t,t ∘ εβ,β ′ = Id ′

′

′

Homt,t ′ (l,g)/G β,β

for all t, t ′ , β, β′ . Let φ =

(φ1 , Cφ ) ∈ Hom(l, g) be such that the orbit G ⋅ (φ1 , Cφ ) ∈ Homt,t (l, g)/G. Then β,β′ ′

t,t t,t π t,t ∘ εβ,β (g0 ∗ φ1 , Pφ′ −1 (Cφ (g0 ) + Im(fφ1 )). ′ (G ⋅ (φ1 , Cφ )) = π 1 |V ′ ′

′

′

β

There exists g1 ∈ Gg0 ∗φ1 such that g1 g0 ⋅ (φ1 , Cφ ) = (g0 ∗ φ1 , (Cφ (g0 ) + Im fφ1 ) ∩ Vβ′ ). Thus, π t,t (g0 ∗ φ1 , Pφ′ −1 (Cφ (g0 ) + Im(fφ1 )) = π t,t (g1 g0 ⋅ (φ1 , Cφ )) 1 |V ′ ′

′

β

= G ⋅ (g1 g0 ⋅ (φ1 , Cφ )) = G ⋅ (φ1 , Cφ ). In particular, t,t ′ t,t ′ εβ,β ′ (Homβ,β′ (l, g)/G)

Write

{ = {φ ∈ Hom(l, g) {

Homt,t (l, g) = ′

⋃

β∈I(m,m−t) β′ ∈I(m′ ,m′ −t ′ )

󵄨󵄨 t,t ′ 󵄨󵄨 φ1 ∈ Mβ,β ′ (l, g), } 󵄨󵄨 󵄨󵄨 }. 󵄨󵄨 C ∈ V ′ 󵄨 φ β }

Homt,t (l, g). β,β′ ′

189

4.3 The 3-step case |

Then from Proposition 4.3.31, the set Homt,t (l, g) is a G-invariant subset. More pre′ cisely all of the subsets of the union are G-invariant and open in Homt,t (l, g). Our main result is the following. ′

Theorem 4.3.33. The writing Hom(l, g)/G = ⋃ Homt,t (l, g)/G ′

0≤t≤q 0≤t ′ ≤q′

is a decomposition of Hom(l, g)/G as a union of Hausdorff subspaces. The sets ′ Homt,t (l, g)/G may fail to be open in Hom(l, g)/G. To prove this result, we need the following lemma. Lemma 4.3.34. Let φ = (φ1 , C) and ξ = (ξ1 , C ′ ) be two elements in Homt,t (l, g). If [φ1 ] ′ ′ and [ξ1 ] are separated in Hom1t,t (l, g)/G, then so are [φ] and [ξ ] in Homt,t (l, g)/G. ′

Proof. We consider the following diagram: Homt,t (l, g) ′

π1

? Homt,t ′ (l, g) 1 ?

?

Homt,t (l, g)/G ′

P1

̃1 P

π2

? Homt,t ′ (l, g)/G 1

̃1 (G ⋅ (φ1 , C)) = G ∗ φ1 . Then where π1 , π2 are the quotient maps, P1 (φ1 , C) = φ1 and P ̃1 are continuous. Assume obviously this diagram commutes and the maps P1 and P that [P1 (φ)] and [P1 (ξ )] are separated, then there exist a neighborhood U1 of π2 ∘ P1 (φ) ̃ 1 ∘ π1 (φ) ∈ U1 and and U2 of π2 ∘ P1 (ξ ) such that U1 ∩ U2 = ⌀. Now π2 ∘ P1 (φ) = P −1 ̃ ̃ ̃ π2 ∘ P1 (ξ ) = P1 ∘ π1 (ξ ) ∈ U2 . Thus, π1 (φ) ∈ P1 (U1 ), π1 (ξ ) ∈ P1−1 (U2 ) and we have ̃ −1 (U1 ) ∩ P ̃ −1 (U2 ) = ⌀. P 1 1 Proof of Theorem 4.3.33. Let φ = (φ1 , C) and ξ = (ξ1 , C ′ ) be two elements in Homt,t (l, g) and assume that [φ] and [ξ ] are not separated. From Lemma 4.3.34, [φ1 ] and [ξ1 ] ′ are not separated in Hom1t,t (l, g)/G. Then there exist (φ1,n )n ⊂ Homt1 (l, g) and gn = exp(Xn ) ∈ G such that φ1,n converges to φ1 and gn ∗ φ1,n converges to ξ1 in Homt1 (l, g). This means Aφ1 = Aξ1 , Dφ1 = Dξ1 and Fφ1 = Fξ1 . In particular, lφ1 = lξ1 and Pφ1 = Pξ1 . ′

t,t Thus [φ1 ] and [ξ1 ] belong to the open set Hom1,β (l, g)/G for some β ∈ I(m, m − t). Now ′

Homt,t (l, g)/G is a Hausdorff space as it is included in Homt1,β (l, g)/G, which is a Haus1,β ′

dorff space as being homeomorphic to Mβt (l, g) by Lemma 4.3.16. Then [φ1 ] = [ξ1 ] and this implies that fφ1 = fξ1 and Pφ′ 1 = Pξ′1 . Finally, [φ] and [ξ ] belong to the open set

190 | 4 The deformation space for nilpotent Lie groups Homt,t (l, g)/G for some β ∈ I(m, m − t) and β′ ∈ I(m′ , m′ − t ′ ). Now from Proposiβ,β′ ′

t,t t,t tion 4.3.31, Homβ,β ′ (l, g)/G is homeomorphic to the Hausdorff space Mβ,β′ (l, g) × Vβ′ , ′

′

t,t then Homβ,β ′ (l, g)/G is a Hausdorff space and [φ] = [ξ ]. ′

4.3.3 Description of the parameter and the deformation spaces We use the same setting and notation. Let us take a subalgebra h of g and consider the decompositions g = (g0 ∩ h) ⊕ g′0 ⊕ h′ ⊕ V ⊕ h′′ ⊕ W

and l = l0 ⊕ l1 ⊕ l2 ,

where g′0 , h′ and h′′ designate some subspaces of g such that g0 = (g0 ∩h)⊕g′0 , [g, g]∩h = (g0 ∩ h) ⊕ h′ , h = [g, g] ∩ h ⊕ h′′ , V is a linear subspace supplementary to (g0 ∩ h) ⊕ g′0 ⊕ h′ in [g, g] and W a linear supplementary subspace to (g0 ∩ h) ⊕ g′0 ⊕ h′ ⊕ V ⊕ h′′ in g. Then with respect to these decompositions, the adjoint representation Adg , g = exp(X) ∈ G can once again be written down as I1 0 (0 Adg = ( (0 0 (0

0 I2 0 0 0 0

σ1,3 (X) σ2,3 (X) I3 0 0 0

δ1,4 (X) δ2,4 (X) 0 I4 0 0

γ1,5 (X) γ2,5 (X) γ3,5 (X) γ4,5 (X) I5 0

ω1,6 (X) ω2,6 (X) ω3,6 (X)) ), ω4,6 (X)) 0 I6 )

where σ1,3 (X) σ2,3 (X)

δ1,4 (X) ), δ2,4 (X)

γ (X) 1 Σ1,3 (X) + Σ1,2 (X)Σ2,3 (X) = ( 1,5 γ2,5 (X) 2

ω1,6 (X) ) ω2,6 (X)

Σ1,2 (X) = (

and Σ2,3 (X) = (

γ3,5 (X) γ4,5 (X)

ω3,6 (X) ) ω4,6 (X)

with σ1,3 (X) ∈ L (h′ , g0 ∩ h), δ1,4 (X) ∈ L (V, g0 ∩ h), σ2,3 (X) ∈ L (h′ , g′0 ), δ2,4 (X) ∈ L (V, g′0 ), γ1,5 (X) ∈ L (h′′ , g0 ∩ h), ω1,6 (X) ∈ L (W, g0 ∩ h), γ2,5 (X) ∈ L (h′′ , g′0 ), ω2,6 (X) ∈ L (W, g′0 ), γ3,5 (X) ∈ L (h′′ , h′ ), ω3,6 (X) ∈ L (W, h′ ), γ4,5 (X) ∈ L (h′′ , V) and ω4,6 (X) ∈ L (W, V). Here, I1 , I2 , I3 , I4 , I5 and I6 designate the identity maps of g0 ∩ h, g′0 , h′ , V, h′′

4.3 The 3-step case |

191

and W, respectively. This leads to the fact that any element of Hom(l, g) can be written accordingly, as a matrix A1 A2 (0 φ := φ(A, B, C) = ( (0 0 0 (

B1 B2 B3 B4 0 0

C1 C2 C3 ) ), C4 ) C5 C6 )

where

A1 ), A2

A=(

B1 B B = ( 2) B3 B4

C1 C2 (C3 ) ) and C = ( (C ) . 4 C5 C ( 6)

Here, A1 ∈ L (l0 , g0 ∩ h), A2 ∈ L (l0 , g′0 ), B1 ∈ L (l1 , g0 ∩ h), B2 ∈ L (l1 , g′0 ), B3 ∈ L (l1 , h′ ), B4 ∈ L (l1 , V), C1 ∈ L (l2 , g0 ∩ h), C2 ∈ L (l2 , g′0 ), C3 ∈ L (l2 , h′ ), C4 ∈ L (l2 , V), C5 ∈ L (l2 , h′′ ) and C6 ∈ L (l2 , W). We can now state our first result. Theorem 4.3.35. Let G be a 3-step nilpotent Lie group, H a connected subgroup of G and Γ a discontinuous group for G/H. The syndetic hull of Γ in G and its Lie algebra are denoted by L and l, respectively. Then the parameter space R (l, g, h) writes as a disjoint union R1 ⊔ R2 , where R1 is the open set defined by 󵄨󵄨 󵄨󵄨 rk(C6 ) = dim(l2 ), } } 󵄨󵄨 } 󵄨󵄨 , 󵄨󵄨 rk(B4 ) = dim(l1 ) } 󵄨󵄨 } } 󵄨󵄨 󵄨󵄨 and rk(A2 ) = dim(l0 ) } 󵄨󵄨 󵄨󵄨 rk(B4 ) + rk(C6 ) < dim(l1 ⊕ l2 ) and R2 := {φ(A, B, C) ∈ Hom(l, g) 󵄨󵄨󵄨 } 󵄨󵄨 rk(M ) = dim(l) for all X ∈ g φ,X 󵄨 { { { R1 := {φ(A, B, C) ∈ Hom(l, g) { { {

which may fail to be open. Here, A2 Mφ,X = (

0 0

B2 + σ2,3 (X)B3 + δ2,4 (X)B4 B4 0

C2 + σ2,3 (X)C3 + δ2,4 (X)C4 +γ2,5 (X)C5 + ω2,6 (X)C6 ). C4 + γ4,5 (X)C5 + ω4,6 (X)C6 C6

192 | 4 The deformation space for nilpotent Lie groups Proof. As the pair (G, H) has the Lipsman property (cf. Definition 2.1.10), Theorem 2.2.2 enables us to state that 󵄨󵄨 󵄨 dim φ(l) = dim(l), }. 󵄨󵄨 Ad ∘φ(l) ∩ h = {0} for all g = exp(X) ∈ G g 󵄨

󵄨 R (l, g, h) = {φ ∈ Hom(l, g) 󵄨󵄨󵄨

(4.55)

Now A1

B1 + σ1,3 (X)B3 + δ1,4 (X)B4

(A2 ( ( ( Adg ∘φ = ( (0 ( (0 0 0 (

B2 + σ2,3 (X)B3 + δ2,4 (X)B4 B3 B4 0 0

C1 + σ1,3 (X)C3 + δ1,4 (X)C4 +γ1,5 (X)C5 + ω1,6 (X)C6 C2 + σ2,3 (x)C3 + δ2,4 (X)C4 ) ) +γ2,5 (X)C5 + ω2,6 (X)C6 ) ) ), C3 + γ3,5 (X)C5 + ω3,6 (X)C6 ) ) C4 + γ4,5 (X)C5 + ω4,6 (X)C6 ) C5 C6 )

which means that the condition Adg ∘φ(l) ∩ h = {0} is equivalent to the fact that rk(Mφ,X ) = dim(l), which is in turn equivalent to rk(C6 ) = dim(l2 ),

rk(B4 ) = dim(l1 )

and

rk(A2 ) = dim(l0 ),

or rk(C6 ) < dim(l2 ), { { { rk(B4 ) < dim(l1 ), { { { {rk(C6 ) < dim(l2 ),

rk(B4 ) = dim(l1 ) and rk(C6 ) = dim(l2 ) and

rk(B4 ) < dim(l1 ) and

rk(Mφ,X ) = dim(l)

rk(Mφ,X ) = dim(l)

rk(Mφ,X ) = dim(l).

The latter three cases are equivalent to say that rk(B4 ) + rk(C6 ) < dim(l1 ⊕ l2 ). We are now ready to present our main result in this section. Theorem 4.3.36. Let g, h and l be as before. The deformation space reads 2

T (l, g, h) = ⋃ ⋃

⋃

i=1 0≤t≤q β∈I(m,m−t) 0≤t ′ ≤q′ β′ ∈I(m′ ,m′ −t ′ )

Tt,t ′ ,β,β′ ,i (l, g, h),

or or

4.3 The 3-step case |

193

′ where for β = (i1 , . . . , im−t ) and β′ = (i1′ , . . . , im ′ −t ′ ) the set Tt,t ′ ,β,β′ ,1 is homeomorphic to the semialgebraic subset in L (l, g),

Tt,t ′ ,β,β′ ,1

{ { { { { { { { { { { ≃ {φ(A, B, C) ∈ Hom(l, g) { { { { { { { { { { {

′ 󵄨󵄨 󵄨󵄨 φ1 ∈ M t,t ′ (l, g), } } β,β 󵄨󵄨 } 󵄨󵄨 } } 󵄨󵄨 C } } } 󵄨󵄨󵄨 ( 1 ) ∈ V ′ and } } β 󵄨󵄨 C } 2 󵄨󵄨󵄨 } 󵄨󵄨 } } 󵄨󵄨 } A2 0 0 } 󵄨󵄨 } } 󵄨󵄨 󵄨󵄨 rk ( 0 B4 0 ) = dim(l) } } } 󵄨󵄨 } 0 0 C6 󵄨󵄨󵄨 }

and Tt,t ′ ,β,β′ ,2 is homeomorphic to

Tt,t ′ ,β,β′ ,2

{ { { { { { { { { ≃ {φ(A, B, C) ∈ Hom(l, g) { { { { { { { { {

′ 󵄨󵄨 󵄨󵄨 φ1 ∈ M t,t ′ (l, g), } β,β 󵄨󵄨 } } 󵄨󵄨 } } 󵄨󵄨 C } } } 󵄨󵄨󵄨 ( 1 ) ∈ Vβ′ , } 󵄨󵄨 C . 󵄨󵄨 2 } } 󵄨󵄨 } } 󵄨󵄨 } } 󵄨󵄨 rk(B4 ) + rk(C6 ) < dim(l1 ⊕ l2 ) } } 󵄨󵄨 } 󵄨󵄨 󵄨󵄨 and rk(Mφ,X ) = dim(l) for all X ∈ g }

Proof. Recall first that 2

R (l, g, h) = ⋃ ⋃

Homt,t (l, g) ∩ Ri . β,β′ ′

⋃

i=1 0≤t≤q β∈I(m,m−t) 0≤t ′ ≤q′ β′ ∈I(m′ ,m′ −t ′ )

t,t On the other hand, Homβ,β ′ (l, g) ∩ Ri is a G-invariant set as in formula (4.52). Hence, ′

2

T (l, g, h) = ⋃ ⋃

⋃

(Homt,t (l, g) ∩ Ri )/G, β,β′ ′

i=1 0≤t≤q β∈I(m,m−t) 0≤t ′ ≤q′ β′ ∈I(m′ ,m′ −t ′ )

and the result follows from Theorem 4.3.35. Now to see the semialgebraicness of Tt,t ′ ,β,β′ ,1 , we state first the following claim, which explains the semialgebraicness of Hom(l, g) in L (l, g). Lemma 4.3.37. The set Hom(l, g) is algebraic in L (l, g). Proof. Let {Y1 , . . . , Yk } be a basis of l and {X1 , . . . , Xq } a basis of g. Assume that the Lie

brackets of l are given by [Yi , Yj ] = ∑ku=1 ciju Yu for all 1 ≤ i, j ≤ k and the Lie brackets of g v ′ are given by [Xs , Xs′ ] = ∑qv=1 dss ′ Xv for all 1 ≤ s, s ≤ q. Let now φ ∈ L (l, g) and assume

194 | 4 The deformation space for nilpotent Lie groups that φ(Yi ) = ∑qs=1 asi Xs for all 1 ≤ i ≤ k. Now Hom(l, g) = {φ ∈ L (l, g) | φ([Y, T]) = [φ(Y), φ(T)], ∀Y, T ∈ l}

= {φ ∈ L (l, g) | φ([Yi , Yj ]) = [φ(Yi ), φ(Yj )], ∀1 ≤ i, j ≤ k}.

As k

k

q

q

k

u u u φ([Yi , Yj ]) = ∑ ci,j φ(Yu ) = ∑ ∑ ci,j avu Xv = ∑ ( ∑ ci,j avu )Xv , u=1

u=1 v=1

v=1 u=1

and q

q

q

s=1

s′ =1

s,s′ =1

[φ(Yi ), φ(Yj )] = [∑ asi Xs , ∑ as′ j Xs′ ] = ∑ asi as′ j [Xs , Xs′ ] q

q

q

q

v v = ∑ ∑ asi as′ j dss ′ Xv = ∑ ( ∑ asi as′ j dss′ )Xv , v=1 s,s′ =1

s,s′ =1 v=1

we get k

q

u=1

s,s′ =1

u v avu = ∑ asi as′ j dss [φ(Yi ), φ(Yj )] = φ([Yi , Yj ]) ⇔ ∑ ci,j ′

for any v = 1, . . . , q. Hence, if we identify L (l, g) to Mq,k (ℝ) via the map φ 󳨃→ Mφ = (φ(Y1 )| ⋅ ⋅ ⋅ |φ(Yk )), then q 󵄨󵄨 k 󵄨 u v Hom(l, g) = {(asi )1≤s≤q 󵄨󵄨󵄨 ∑ ci,j avu = ∑ asi as′ j dss ′ , 1 ≤ i, j ≤ k}, 󵄨 1≤i≤k 󵄨 u=1 s,s′ =1

which is an algebraic set. Now we see that the conditions C ( 1 ) ∈ Vβ′ C2

and

A2 rk ( 0 0

0 B4 0

0 0 ) = dim(l) C6

are semialgebraic conditions and as Homt,t (l, g) is semialgebraic and the condition 1,β,β′ ′

t,t Nφ1 ∈ Vβ is an algebraic condition. Then by (4.51), we conclude that Mβ,β ′ (l, g) is semialgebraic. ′

4.3 The 3-step case |

195

4.3.4 Hausdorffness of the deformation space This section aims to study the Hausdorfness of the deformation space T (l, g, h) in the setting where g is 3-step nilpotent. Let p : R (l, g, h) (φ1 , C)

󳨀→ 󳨃󳨀→

Hom1 (l, g)

φ1 ,

where Hom1 (l, g) is as in (4.44), φ1 is as in (4.39) and (φ1 , C) ∈ R (l, g, h) ⊂ Hom1 (l, g) × L (l2 , g0 ). Then p is a G-equivariant map and we can state the following. Theorem 4.3.38. Let G = exp g be a 3-step nilpotent Lie group, H = exp h a closed, connected subgroup of G, Γ a discontinuous group for the homogeneous space G/H and L = exp l its syndetic hull. If the dimensions of G-orbits in R (l, g, h) and those in p(R (l, g, h)) are constant, respectively, then T (l, g, h) is a Hausdorff space. Proof. In such a situation, there is t ∈ {0, . . . , q} and t ′ ∈ {0, . . . , q′ } such that R (l, g, h) ⊂ ′ Homt,t (l, g). Indeed, let φ = (φ1 , Cφ ) ∈ R (l, g, h) and assume that rk lφ1 = t and rk fφ1 = t ′ . As G ∗ φ1 = φ0 + Im(lφ1 ), we have dim G ∗ φ1 = t and dim G ⋅ φ = dim G ⋅ (φ1 , Cφ )

= dim G ∗ φ1 + dim(πβt (G ∗ φ1 ), Cφ + Im fφ1 )

= dim(φ0 + (Nφ1 + Im lφ1 )) + rk fφ1

= rk lφ1 + rk fφ1 = t + t ′ .

Since the dimensions of G-orbits of R (l, g, h) and of p(R (l, g, h)) are constant, then so ′ are t and t ′ . The deformation space is therefore contained in Homt,t (l, g)/G, which is a Hausdorff space by Theorem 4.3.33.

4.3.5 Illustrating examples For the convenience of the readers, we develop the following series of examples for which the hypotheses of Theorem 4.3.38 are met and then the corresponding deformation space turns out to be a Hausdorff space. Explicit computations are also developed in the paper [19]. Let g = ℝ-span{X0 , X1 , X2 , X3 } be the (3-step nilpotent) threadlike Lie algebra, whose pairwise brackets equal zero, except the following: [X0 , Xi ] = Xi+1 ,

i = 1, 2.

196 | 4 The deformation space for nilpotent Lie groups The center of g is the space ℝ-span{X3 }, g1 = ℝ-span{X2 }, g2 = ℝ-span{X0 , X1 } and for X = x0 X0 + x1 X1 + x2 X2 + x3 X3 ∈ g, we have adX (X0 ) = −x1 X2 − x2 X3 ,

adX (X1 ) = x0 X2

and

adX (X2 ) = x0 X3 .

On the other hand, through the basis B = {X3 , X2 , X1 , X0 }, the matrices of the endomorphisms adX and ad2X are written as 0 0 adX = ( 0 0

x0 0 0 0

0 x0 0 0

−x2 −x1 ), 0 0

0 0 ad2X = ( 0 0

0 0 0 0

x02 0 0 0

−x0 x1 0 ). 0 0

Hence, the matrix of the adjoint representation Adexp(X) can be expressed as

Adexp(X)

1 0 =( 0 0

x0 1 0 0

1 2 x 2 0

x0 1 0

−x2 − 21 x0 x1 −x1 ), 0 1

and finally a a + bx0 + 21 x02 c − d(x2 + 21 x0 x1 ) b b + cx0 − dx1 ), Adexp(X) ∘ ( ) = ( c c d d

(4.56)

where the vector t (a b c d) represents a vector of g through the basis B . Example 4.3.39. Let h = ℝ-span{X1 , X2 , X3 } and l = ℝ-span{X0 }. Then if G, H designate the Lie groups associated to g and h, respectively, and Γ = exp(ℤX0 ), then obviously the resulting Clifford–Klein form Γ\G/H turns out to be compact. Clearly, Γ ≃ ℤ, G/H ≃ ℝ and Γ\G/H ≃ S1 . As such, it is straightforward that through the basis B , any φ ∈ Hom(l, g) is given by φ : l → g;

λX0 󳨃→ λ(dX0 + cX1 + bX2 + aX3 )

4.3 The 3-step case |

197

and a b φ = ( ) ∈ R (l, g, h) ⇐⇒ d ≠ 0. c d

(4.57)

Indeed, from the set equality (4.55), 󵄨󵄨 a 󵄨󵄨 } { } { 󵄨󵄨 } { } { 󵄨󵄨 b R (l, g, h) = { φ = ( ) ∈ Hom(l, g) 󵄨󵄨󵄨 Adexp(X) ∘φ(l) ⊕ h = g} 󵄨󵄨 { } c { } 󵄨󵄨 } { 󵄨󵄨 d 󵄨 } { 󵄨󵄨 a 󵄨󵄨 { } { } 󵄨󵄨 { b } { } 󵄨󵄨 = { ( ) ∈ Hom(l, g) 󵄨󵄨󵄨 d ≠ 0} . { c } 󵄨󵄨󵄨 { } { } 󵄨󵄨 󵄨 d 󵄨 { } On the other hand, and according to our construction, for φ as in equation (4.57), 0 b φ1 = ( ) . c d By equation (4.56), we get bx0 + 21 x02 c − d(x2 + 21 x0 x1 ) cx0 − dx1 ) Adexp(X) ∘φ − φ = ( 0 0 and 0 cx0 − dx1 Adexp(X) ∗φ1 − φ1 = ( ), 0 0 which means that 󵄨󵄨 c b Gφ = {exp(x0 X0 + x1 X1 + x2 X2 + x3 X3 ) ∈ G 󵄨󵄨󵄨 x1 = x0 , x2 = x0 } 󵄨 d d 󵄨󵄨 c Gφ1 = {exp(x0 X0 + x1 X1 + x2 X2 + x3 X3 ) ∈ G 󵄨󵄨󵄨 x1 = x0 }. 󵄨 d

and

198 | 4 The deformation space for nilpotent Lie groups Finally, dim G ⋅ φ = 2 and dim G ∗ φ1 = 1 for any φ ∈ R (l, g, h). Then by Theorem 4.3.38, the deformation space T (l, g, h) is a Hausdorff space. Example 4.3.40. Let now h = ℝ-span{X0 } and l = ℝ-span{X1 , X2 , X3 }. Then again the resulting Clifford–Klein form Γ\G/H is compact. Clearly, Γ ≃ ℤ3 , G/H ≃ ℝ3 and Γ\G/H is homeomorphic to the 3-dimensional torus. As φ ∈ R (l, g, h) if and only if φ(l) = l, φ takes the following form: a1 b1 φ=( c1 0

a2 b2 c2 0

a3 b3 ). c3 0

Hence, a1 + b1 x0 + 21 x02 c1 b1 + x0 c1 Adexp(X) ∘φ = ( c1 0 b1 c = φ + x0 ( 1 0 0

b2 c2 0 0

a2 + b2 x0 + 21 x02 c2 b2 + x0 c2 c2 0

b3 c1 c3 0 1 2 ) + x0 ( 0 0 2 0 0

c2 0 0 0

a3 + b3 x0 + 21 x02 c3 b3 + x0 c3 ) c3 0 c3 0 ), 0 0

and then dim G ⋅ φ = 1. Besides, 0 b1 φ1 = ( c1 0

0 b2 c2 0

0 c1 Adexp(X) ∗φ1 = φ1 + x0 ( 0 0

0 b3 ), c3 0 0 c2 0 0

0 c3 ), 0 0

and likewise dim G ∗ φ1 = 1 for any φ ∈ R (l, g, h). Then by Theorem 4.3.38, the deformation space is a Hausdorff space. Example 4.3.41. The following example treats a noncompact Clifford–Klein form case. Let h = ℝ-span{X3 } and l = ℝ-span{X1 , X2 }. Clearly, Γ ≃ ℤ2 and G/H ≃ ℝ3 . Therefore, Γ\G/H is not compact. We first prove the following. Claim 4.3.42. For any φ ∈ R (l, g, h), we have φ(l) ⊂ ℝ-span{X1 , X2 , X3 }.

4.3 The 3-step case |

199

Proof. If not, there exists v = X0 + u ∈ φ(l) for some u ∈ ℝ-span{X1 , X2 , X3 }. Since dim(φ(l)) = 2, there exists w ≠ 0 such that w ∈ φ(l) ∩ ℝ-span{X1 , X2 , X3 }. Thus, ℝ-span{[v, [v, w]], [v, w], w} ∩ h ≠ {0}, which leads to a contradiction as φ(l) ∩ h = {0}. Now, any φ ∈ R (l, g, h) reads a1 b1 φ=( c1 0

a2 b2 ) c2 0

(4.58)

and we have the following. Claim 4.3.43. Let φ ∈ R (l, g, h) be as in equation (4.58), then (c1 , c2 ) ≠ (0, 0). Proof. From Claim 4.3.42, any φ ∈ R (l, g, h) reads a1 b1 φ=( c1 0

a2 b2 ). c2 0

Now, from the set equality (4.55), φ ∈ R (l, g, h) ⇔ dim h + dim φ(l) = 3, b1 c1

⇔ det (

φ ∈ R (l, g, h)

b2 ) ≠ 0. c2

Now for φ ∈ R (l, g, h) be as in equation (4.58), b1 c Adexp(X) ∘φ = φ + x0 ( 1 0 0

b2 c1 c2 0 1 2 ) + x0 ( 0 0 2 0 0

and then dim G ⋅ φ = 1. It is also obviously the case for 0 b1 φ1 = ( c1 0

0 b2 ) c2 0

c2 0 ) 0 0

200 | 4 The deformation space for nilpotent Lie groups as 0 c1 Adexp(X) ∗φ1 = φ1 + x0 ( 0 0

0 c2 ). 0 0

Then by Theorem 4.3.38, the deformation space is a Hausdorff space.

4.4 Deformation space of threadlike nilmanifolds Throughout the section, we note G = exp(g) a threadlike nilpotent Lie group as defined in Subsection 1.1.3. We fix a basis B = {X, Y1 , . . . , Yn } of g with nontrivial Lie brackets defined in (1.7). Recall the subspace g0 = ℝ-span{Y1 , . . . , Yn }, which is an Abelian ideal of g of codimension one and let G0 = exp(g0 ). The center z(g) of g is however one- dimensional and it is the space ℝ-span{Yn }. It turns out therefore that any threadlike Lie group belongs to the family of nilpotent Lie groups referred to as to be special, which admit one-codimensional normal Abelian subgroup, hence of the form ℝ ⋉ ℝn . The study of the deformation space of the action of discontinuous groups for special homogeneous spaces, especially its explicit determination seems to be subtle and problematic for some structural and technical reasons. However, we know already how to characterize the proper action of discontinuous groups on homogeneous spaces in this setup (cf. Chapter 2). 4.4.1 Description of Hom(l, g) Let Γ ≃ ℤk be a discrete finitely generated subgroup of G and L = exp l its syndetic hull. Our main result in this section consists in giving an explicit description of Hom(l, g), the set of all algebras homomorphisms from l to g. Toward such a purpose, recall first the result of Lemma 1.1.13 concerning the description of the structure of Lie subalgebras of threadlike algebras and we consider the sets: H0,k = {(

󳨀 t→

0 ) ∈ Mn+1,k (ℝ) : N ∈ Mn,k (ℝ)} ≃ Mn,k (ℝ), N

(4.59)

and for any j ∈ {1, . . . , k}: Hj,k = {(

λ1 T ⋅ ⋅ ⋅ λj−1 T z1 ⋅ ⋅ ⋅ zj−1

T zj

λj+1 T ⋅ ⋅ ⋅ λk T ) ∈ Mn+1,k (ℝ) : t T ∈ ℝ× × ℝn−1 , zj+1 ⋅ ⋅ ⋅ zk

(z1 , . . . , zk ) ∈ ℝk , (λ1 , . . . , λ̌j , . . . , λk ) ∈ ℝk−1 } ≃ ℝ× × ℝn−1 × ℝk−1 × ℝk .

(4.60)

4.4 Deformation space of threadlike nilmanifolds | 201

The following upshot accurately describes the structure of the set Hom(l, g), which is one of the main means to study the parameter and the deformation spaces. Theorem 4.4.1. With the same notation and hypotheses, we have k

Hom(l, g) = ⋃ Hj,k . j=0

Proof. We identify any element T = xX + ∑ni=1 yi Yi ∈ g by the column vector t ′ (x, y1 , . . . , yn ). We define now the subset Mn+1,k (ℝ) = {(T1 , . . . , Tk ) ∈ Mn+1,k (ℝ) : [Ts , Tr ] = 0, 1 ≤ r, s ≤ k}. Then having fixed a basis Bl of l, it appears clear that the map ′ (ℝ), Ψ : Hom(l, g) 󳨀→ Mn+1,k

(4.61)

which associates to any element of Hom(l, g) its matrix written through the bases Bl and B is a homeomorphism. Let now x1 y11 M=( . .. yn1

⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅

xk y1k ′ .. ) ∈ Mn+1,k (ℝ), . ykn

then for any 1 ≤ s, r ≤ k, [Ts , Tr ] = 0 where Ts = xs X +∑ni=1 yis Yi and Tr = xr X +∑ni=1 yir Yi . This gives rise to the following equation: xs yir − xr yis = 0

for all 1 ≤ r, s ≤ k and all 1 ≤ i ≤ n − 1.

(4.62)

In order to find the solutions of (4.62), we shall discuss the two following dichotomous cases. Assume in a first time that the first line of M is zero, that is, xj = 0, 1 ≤ j ≤ k. ′ (ℝ). Suppose now that In this case, M satisfies (4.62) and obviously belongs to Mn+1,k the first line of M is not zero. There exists then j ∈ {1, . . . , k} satisfying xj ≠ 0. So, ′ M ∈ Mn+1,k (ℝ) if and only if there exists Λj = (λ1 , . . . , λj−1 , λj+1 , . . . , λk ) ∈ ℝk−1 such that for s ≠ j, we have Ts′ = λs Tj′ , where Ts′ = t (xs , y1s , . . . , yn−1s ). According to this ′ discussion, we get that Mn+1,k (ℝ) = ⋃kj=0 Hj,k where Hj,k , 0 = 1, . . . , k are as determined by equations (4.59) and (4.60). This achieves the proof of the theorem.

Note that the set of all injective algebras homomorphisms from l to g denoted by Hom0 (l, g), rather than Hom(l, g) itself will be of interest in the next section, merely because it is involved in deformations. The following result accurately determines the stratification of such a set. Proposition 4.4.2. Let k ∈ {1, . . . , n + 1}, there exists a finite set Ik such that 0 ∈ Ik and Hom0 (l, g) = ∐j∈Ik Kj,k , where:

202 | 4 The deformation space for nilpotent Lie groups (i)

If k > 2, then Ik = {0} and for any k ∈ {1, . . . , n} we have K0,k = {(

(ii)

󳨀 t→

0 0 0 ) : N ∈ Mn,k (ℝ)} ≃ Mn,k (ℝ). N

(4.63)

0 Here, Mn,m (ℝ) denotes the set of all matrix of n rows, m columns and of maximal rank. If k = 2, then I2 = {0, 1, 2, 3} and

x 0 } { } { 󳨀 → 󳨀 K1,2 = {(→ y 0 ) ∈ Mn+1,2 (ℝ) : xz2 ≠ 0} , } { } { z1 z2 0 x } { } { → 󳨀 󳨀 K2,2 = {( 0 → y ) ∈ Mn+1,2 (ℝ) : xz1 ≠ 0} , } { } { z1 z2 x λx { } { 󳨀 } 󳨀y ) ∈ M K3,2 = {(→ y λ→ n+1,2 (ℝ) : λx ≠ 0, λz1 − z2 ≠ 0} . { } { z1 z2 } (iii) If k = 1, then I1 = {0, 1} and x × → n × n 󳨀 K1,1 = H1,1 = {(→ 󳨀y ) : x ∈ ℝ , y ∈ ℝ } ≃ ℝ × ℝ . Proof. Let k ∈ {1, . . . , n + 1}, j ∈ {1, . . . , k} and M ∈ Hj,k . It is not hard to check that rank(M) ≤ 2. In addition, M is of maximal rank if and only if rank(M) = k. So, it appears clear that if k > 2, we have Hom0 (l, g) ∩ Hj,k = 0 for all j = 1, . . . , k. This proves the first assertion of the proposition. Suppose now that k = 2 and choose in a first time x 󳨀y M = (→ z1

λx 󳨀y ) ∈ H , λ→ 1,2 z2

󳨀y ∈ ℝn−1 , λ ∈ ℝ and (z , z ) ∈ ℝ2 . The condition M is of maximal rank where x ∈ ℝ× , t → 1 2 is equivalent to λz1 − z2 ≠ 0. If furthermore, we choose λx x 󳨀y → 󳨀y ) ∈ H M = (λ→ 2,2 z1 z2

4.4 Deformation space of threadlike nilmanifolds | 203

󳨀y ∈ ℝn−1 and (z , z ) ∈ ℝ2 , we get M is of maximal rank if and only for some x ∈ ℝ× , t → 1 2 if λz2 − z1 ≠ 0. Therefore, 0

Hom (l, g) ∩ H1,2

x { { → 󳨀 = {M = ( y { z1 {

λx } 󳨀y ) ∈ H : λz − z ≠ 0} λ→ 1,2 1 2 } } z2 }

and 0

Hom (l, g) ∩ H2,2

λx x { } { 󳨀y → 󳨀y ) ∈ H : λz − z ≠ 0} . = {M = (λ→ 2,2 2 1 } { } z z 1 2 { }

It is then easy to see that Hom0 (l, g) ∩ (H1,2 ∪ H2,2 ) is equal to the disjoint union of the sets Kj,2 , j = 1, 2, 3 defined above. So we end up with the following decomposition Hom0 (l, g) = ∐3j=0 Kj,2 . Finally, if k = 1 then any homomorphism in H1,1 is injective, therefore, K1,1 = H1,1 and then Hom0 (l, g) = K1,1 ∐ K0,1 . Remark 4.4.3. It is worth noting at this step that the fact that Hom0 (l, g) ∩ Hj,k = 0 for k > 2 is justified by the fact that any discrete subgroup Γ ⊄ G0 must be of rank ≤ 2 as shows the following proposition. Proposition 4.4.4. Let Γ be an Abelian discrete subgroup of G such that Γ ⊄ G0 . Then rank(Γ) ∈ {1, 2}. Proof. Let L be the syndetic hull of Γ and l its corresponding Lie algebra. It is clear that L is Abelian and L ⊄ G0 . So, we can suppose that X ∈ l. Let now T = ∑ni=1 αi Yi ∈ l, for some αi ∈ ℝ, then [X, T] = 0. This gives T = αn Yn and then l ⊂ ℝ-span{X, Yn }. We conclude that rank(Γ) = dim L ≤ 2.

4.4.2 Description of the parameter space R (Γ, G, H) The most important problem in the study of the deformation space of discontinuous groups is the description of the parameter set R (Γ, G, H) given as in equation (3.1). Let ′ (ℝ) by G act on Mn+1,k g ⋅ M = Adg −1 ⋅M,

′ M ∈ Mn+1,k (ℝ),

g ∈ G.

Here, we view Adg −1 as a real valued matrix for any g ∈ G. Taking into account the action of G on Hom(l, g) defined in (3.3), the following lemma is immediate. Lemma 4.4.5. The map Ψ defined in (4.61) is G-equivariant. That is, for any ψ ∈ Hom(l, g) and g ∈ G, we have Ψ(g ⋅ ψ) = g ⋅ Ψ(ψ).

204 | 4 The deformation space for nilpotent Lie groups The following result is immediate. Lemma 4.4.6. Let G be a threadlike nilpotent Lie group, H = exp h be a closed connected subgroup of G and let Γ be an Abelian discrete subgroup of G. In light of Theorem 3.2.4, the set R (Γ, G, H) is homeomorphic to ′ {M ∈ Mn+1,k (ℝ) : rank(M ⋒ g ⋅ Mh,B ) = k + p for any g ∈ G},

(4.64)

where B = {X, Y1 , . . . , Yn } and the symbol ⋒ merely means the concatenation of the matrices written through B . 0 Proof. If M ∈ R (Γ, G, H), then M ∈ Mn+1,k (ℝ), which gives that rank(M) = k. Now using Theorem 2.3.2, the proper action of L on G/H is equivalent to the fact that l∩Adg h = {0} for any g ∈ G, which means that rank(M ⋒ g ⋅ Mh,B ) = k + p. Note finally that the condition rank(M) = k is irrelevant at this stage, as was to be shown.

Proposition 4.4.7. Let G be a threadlike Lie group and H a connected Lie subgroup of G. Then R (Γ, G, H) = ∐j∈Ik Rj,k where Rj,k = R (Γ, G, H) ∩ Kj,k . More precisely, one has: (i) If k > 2, then R (Γ, G, H) = R0,k . (ii) If k = 2, then R (Γ, G, H) = ∐3j=0 Rj,2 . (iii) If k = 1, then R (Γ, G, H) = R0,1 ∐ R1,1 . Proof. This result stems immediately from Proposition 4.4.2, which describes the structure of Hom0 (l, g). We now determine the parameter space according to the values of k = rank(Γ). Toward that purpose, let q denote the codimension of h in g and introduce the matrix A(t), t ∈ ℝ of (Adexp tX )|g0 written with respect to the strong Malcev basis B0 = {Y1 , . . . , Yn } of g0 . So, a routine computation shows that

( ( A(t) = ( (

1

0

t

1

⋅⋅⋅ .. .

2

t

1

t 2

.. .

t n−1 ( (n−1)!

..

.

⋅⋅⋅

..

t2 2

.

⋅⋅⋅ .. .. t

. .

0 .. . .. ) ) . .) ) 0

1)

The following proposition deals with the description of R0,k , which coincides with the parameter space in the case where 2 < k ≤ q. Proposition 4.4.8 (The parameter space for k > 2). We keep the same hypotheses and notation as before. Then:

4.4 Deformation space of threadlike nilmanifolds | 205

(i) If h ⊄ g0 , then R0,k

󳨀 t→ 0 { } { } 0 = {( N1 ) ∈ H0,k : N1 ∈ Mq,k (ℝ), N2 ∈ Mn−q,k (ℝ)} { } { N2 } 0 ≃ Mq,k (ℝ) × Mn−q,k (ℝ).

(4.65)

(ii) If h ⊂ g0 , then 󳨀 t→

R0,k = {(

0 ) ∈ H0,k : rank(A(t)N ⋒ Mh,B0 ) = k + p for any t ∈ ℝ} . N

Proof. Thanks to Lemma 4.4.7, R (Γ, G, H) = 0 for k > q. We can from now on suppose that k ≤ q. Suppose in first time that h ⊄ g0 . We note h0 = h ∩ g0 , which is an ideal of g. Let M=(

󳨀 t→

0 ) ∈ H0,k . N

So, equation (4.64) is equivalent to rank(M ⋒ Mh0 ,B ) = k + p − 1, which is in turn equivalent to the fact that rank(N ⋒ Mh0 ,B0 ) = k + p − 1.

(4.66)

According to our choice of the strong Malcev basis B , we get that (0) ) ∈ Mn,p−1 (ℝ), Ip−1

Mh0 ,B0 = (

where Ip−1 designates the identity matrix of Mp−1 (ℝ). We now write 󳨀 t→

0 M = ( N1 ) , N2 0 where N1 ∈ Mq,k (ℝ) and N2 ∈ Mn−q,k (ℝ), we get that equation (4.66) is equivalent to the fact rank(N1 ) = k. We hence end up with

R0,k = R (Γ, G, H) ∩ H0,k

󳨀 t→ 0 { } { } 0 = {( N1 ) : N1 ∈ Mq,k (ℝ), N2 ∈ Mn−q,k (ℝ)} . { } { N2 }

206 | 4 The deformation space for nilpotent Lie groups We now treat the case where h ⊂ g0 . So, it is not hard to see that AdG h = ⋃t∈ℝ Adexp tX h. We get therefore that M ∈ R (Γ, G, H) ⇔ rank(M ⋒ exp tX ⋅ Mh,B ) = k + p for all t ∈ ℝ ⇔ rank(exp tX ⋅ N ⋒ MhB0 ) = k + p

for all t ∈ ℝ

⇔ rank(A(t)N ⋒ Mh,B0 ) = k + p for all t ∈ ℝ, which completes the proof of the proposition. We assume henceforth that rank(Γ) ∈ {1, 2}. We will be dealing with these subsequent cases separately. The following upshot exhibits an accurate description of the parameter space when k = 2. Proposition 4.4.9 (The parameter space for k = 2). Assume that k = 2. Then R0,2 being described in Proposition 4.4.8, we have: (i) If h ⊄ g0 , then: i1 . If q = n, then

R1,2

R2,2

x 0 } { } { } { } { y1 0 ) M (ℝ) : y xz = ̸ 0 = {(→ → 󳨀 n+1,2 1 2 }, 󳨀y 0 } { } { } { } { z1 z2

0 x } { } { } { } { 0 y1 ) ∈ M (ℝ) : y xz = ̸ 0 = {(→ 󳨀 n+1,2 1 1 } → 󳨀 } { } { } { 0 y } { z1 z2

and

R3,2

x { { { { y1 = {(→ 󳨀y { { { { z1

λx } } } } λy1 ) ∈ Mn+1,2 (ℝ) : λx(λz1 − z2 )y1 ≠ 0} . → 󳨀 } λy } } z2 }

i2 . If q < n, then Rj,2 = 0, j = 1, 2, 3. (ii) If h ⊂ g0 , then R1,2 = R2,2 = R3,2 = 0 if Yn ∈ h. Otherwise, Ri,2 = Ki,2 , i = 1, 2, 3. Proof. Let x 󳨀y M = (→ z1

λx 󳨀y ) ∈ Hom0 (l, g), λ→ z2

4.4 Deformation space of threadlike nilmanifolds | 207

󳨀y = (y , . . . , y ) ∈ ℝn−1 and (z , z ) ∈ ℝ2 such that λz − z ∈ ℝ× . We where x ∈ ℝ× , t → 1 n−1 1 2 1 2 tackle first the case where h ⊄ g0 . In the case where h = ℝX, a simple computation shows that AdG h = ℝX + [X, g0 ]. Hence, the assertion rank(M ⋒ g ⋅ Mh,B ) = 3 for any g ∈ G, is equivalent to rank(M ⋒ t (1, 0, α2 , . . . , αn )) = 3, for all α2 , . . . , αn ∈ ℝ, which is in turn equivalent to y1 ∈ ℝ× . Therefore, M ∈ R1,2 ∪ R3,2 if and only if y1 ∈ ℝ× . Similar computations show that for 0 x → 󳨀 󳨀 M = (0 → y ) ∈ K2,2 , z1 z2 one gets that M ∈ R2,2 if and only if y1 ∈ ℝ× . Suppose now that ℝX ⊊ h, that is q < n, we have that the vector Yn = t (0, . . . , 0, 1) ∈ h and it is a linear combination of the columns of M. So rank(M ⋒ Mh,B ) < p + 2 and then R (Γ, G, H) ∩ Kj,2 = Rj,2 = 0, j = 1, 2, 3. Let finally h ⊂ g0 . If Yn ∈ h, then rank(M ⋒ Mh,B ) < p + 2, which gives R (Γ, G, H) ∩ Kj,2 = 0 (j = 1, 2, 3). Otherwise, M ∈ R (Γ, G, H) is equivalent to rank(M ⋒exp tX ⋅Mh,B ) = 2+p for all t ∈ ℝ, that is, rank(M) = 2 and then M ∈ Hom0 (l, g). Thus, we have Rj,2 = Kj,2 , j = 1, 2, 3, which completes the proof in this case. Similar arguments are used to prove the following. Proposition 4.4.10 (The parameter space for k = 1). Assume that k = 1. The layer R0,1 being described in Proposition 4.4.8, we get: (i) If h ⊄ g0 , then x } { { 󳨀y ∈ ℝn−1 } ≃ (ℝ× )2 × ℝn−1 . R1,1 = {(y1 ) : x ∈ ℝ× , y1 ∈ ℝ× ,→ } } { → 󳨀 } { y (ii) If h ⊂ g0 , then R1,1 = H1,1 = K1,1 . 4.4.3 Description of the deformation space T (Γ, G, H) This section aims to describe the deformation space of the action of an Abelian discrete subgroup Γ ⊂ G on a threadlike homogeneous space G/H. This description strongly relies on the comprehensive details about the parameter space provided in the previous section. Now we can state the following. Proposition 4.4.11. We keep the same hypotheses and notation. The disjoint components Rj,k involved through the description of the parameter space R (Γ, G, H) are G-invariant. More precisely, we have the following: (i) If k > 2, then T (Γ, G, H) = R0,k /G.

208 | 4 The deformation space for nilpotent Lie groups (ii) If k = 2, then T (Γ, G, H) = ∐3j=0 (Rj,2 /G). (iii) If k = 1, then T (Γ, G, H) = (R0,1 /G) ∐(R1,1 /G). Proof. Let us first prove that the set R0,k is G-stable. It is clear that the G-action on R0,k is reduced to the action of exp ℝX. Let M=(

󳨀 t→

0 ) ∈ R0,k N

and t ∈ ℝ, then 󳨀 t→ 0 exp tX ⋅ M = ( ). A(t)N From the G-invariance of R (Γ, G, H), we get that exp tX ⋅M ∈ R (Γ, G, H), so we are done in this case. Suppose now that k = 2. Let x y1 M = (→ 󳨀y z1

λx λy1 󳨀y ) ∈ R3,2 , λ→ z2

󳨀y = (y , . . . , y ) ∈ ℝn−2 and let g = exp(tX + a Y + ⋅ ⋅ ⋅ + a Y ) ∈ G for some where t → 2 n−1 1 1 n n t, a1 , . . . , an ∈ ℝ, then a routine computation shows that x y1 g ⋅ M = (→ 󳨀′ y z1′

λx λy1 → 󳨀), λy′ z2′

→ 󳨀 ′ where t y′ = (y2′ , . . . , yn−1 ) is such that i−1 j−1

yi′ = yi + ∑ j=1

t

j!

(tyi−j − xai−j ),

j−1

i = 2, . . . , n − 1

(4.67)

t ′ ′ and zi′ = zi + ∑n−1 j=1 j! (tyn−j − xan−j ), i = 1, 2. As λz1 − z2 = λz1 − z2 , we get that g ⋅ M ∈ R3,2 as was to be shown. We opt for the same arguments to show that R1,2 and R2,2 are G-invariant as well. For k = 1, M = t (x, y1 , . . . , yn−1 ) ∈ R1,1 and g ∈ G, we have

x y g ⋅ M = ( 1) , → 󳨀′ y

4.4 Deformation space of threadlike nilmanifolds | 209

→ 󳨀 ′ where t y′ = (y2′ , . . . , yn−1 ) is given as in equation (4.67). So, the same arguments allow us to conclude. We are now ready to give an explicit description of the deformation space T (Γ, G, H). Toward this purpose, we can divide the task into three parts as in the previous section. More precisely, we shall define a cross-section of Rj,k /G denoted by Tj,k for any j ∈ Ik and k ∈ {1, . . . , n}. Recall that if q < k and k ≥ 2, then we got R (Γ, G, H) = 0. We suppose then that k ≤ q. Let m, n ∈ ℕ and denote for any 1 ≤ r ≤ n and any 1 ≤ s ≤ m by Mn,m (r, s, ℝ) the subset of Mn,m (ℝ) defined by

0 { { { .. { { { . { { ( { { ( { 0 { { ( Mn,m (r, s, ℝ) = {(r) ( (0 { (∗ { { ( { { { { .. { { { . { { { (∗

⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅

0 .. . 0 0 ∗ .. . ∗

(s)

0 .. . 0 xrs ∗ .. . ∗

0 .. . 0 ∗ ∗ .. . ∗

⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅

0 } } } .. } } } . } } } ) } } 0) } } ) × ) ∗) ∈ Mn,m (ℝ) : xrs ∈ ℝ } , } } } ∗) } ) } } } .. } } } . } } ∗) }

and ′ Mn,m (r, s, ℝ) = {M ∈ Mn,m (r, s, ℝ) : x(r+1)s = 0}.

We now consider the set R0,k (r, s) = R0,k ∩ Mn+1,k (r, s, ℝ). For any k ∈ {1, . . . , n}, let Jk designate the set of all (r, s) ∈ {1 ≤ r ≤ n + 1} × {1 ≤ s ≤ k} for which R0,k (r, s) ≠ 0. Then (1, s) ∈ ̸ Jk and R0,k = ∐ R0,k (r, s) (r,s)∈Jk

whenever k > 2. Moreover, it is not hard to see that R0,k (r, s) is G-invariant. Then R0,k /G = ∐ (R0,k (r, s)/G). (r,s)∈Jk ′ ′ For (r, s) ∈ Jk , let T0,k (r, s) = R0,k (r, s) ∩ Mn+1,k (r, s, ℝ) = R0,k ∩ Mn+1,k (r, s, ℝ). We have the following.

Proposition 4.4.12. We keep all our notation as above. Then we have: (i) T0,k (r, s) is homeomorphic to R0,k (r, s)/G for any (r, s) ∈ Jk . (ii) T0,k = ∐(r,s)∈Jk T0,k (r, s). If in particular k > 2, then T (Γ, G, H) ≃ T0,k .

210 | 4 The deformation space for nilpotent Lie groups Proof. Let (r, s) ∈ Jk . We show that T0,k (r, s) is a cross-section of all adjoint orbits of R0,k (r, s). It is clear that the G-action on R0,k (r, s) is reduced to the action of exp ℝX. More precisely, let 󳨀 t→

M=(

0 ) ∈ R0,k (r, s), N

then 󳨀 t→ 0 G ⋅ M = [M] = {( ) : t ∈ ℝ} . A(t)N Noting N = {(ai,j ), 1 ≤ i ≤ n, 1 ≤ j ≤ k}, we get ar−1,s ≠ 0. Let a

{− r,s tM = { ar−1,s {0

if r < n + 1, if r = n + 1.

We can then show that 󳨀 t→ 0 {( )} = G ⋅ M ∩ T0,k (r, s). A(tM )N

(4.68)

Remark first that if r = n + 1 then k = 1 and G ⋅ M = M, so (4.68) holds. Suppose now that r ≤ n. It is then clear that 󳨀 t→ 0 ( ) ∈ G ⋅ M ∩ T0,k (r, s) A(tM )N using the G-invariance of the layer R0,k (r, s). Conversely, if 󳨀 t→ 0 ( ) ∈ T0,k (r, s), A(t)N then by an easy computation, we can see that ar,s + tar−1,s = 0, which gives that t = tM . The next step consists in showing that the map (Φ0,k )(r,s) : R0,k (r, s)/G [M]

→

T0,k (r, s)

󳨃→

(

󳨀 t→

0 ) A(tM )N

is bijective. First of all, it is clear that (Φ0,k )(r,s) is well-defined. In fact, let M1 , M2 ∈ R0,k (r, s) such that [M1 ] = [M2 ]. Then (Φ0,k )(r,s) ([M2 ]) = G ⋅ M2 ∩ T0,k (r, s) = G ⋅ M1 ∩

4.4 Deformation space of threadlike nilmanifolds | 211

T0,k (r, s) = (Φ0,k )(r,s) ([M1 ]). For

M1 = (

󳨀 t→

0 ) N1

and M2 = (

󳨀 t→

0 ) ∈ R0,k (r, s) N2

such that (Φ0,k )(r,s) ([M1 ]) = (Φ0,k )(r,s) ([M2 ]), we have (

󳨀 t→ 0 0 ) ∈ G ⋅ M1 ∩ G ⋅ M2 . )=( A(tM2 )N2 A(tM1 )N1 󳨀 t→

It follows therefore that [M1 ] = [M2 ], which leads to the injectivity of (Φ0,k )(r,s) . Now, to see that (Φ0,k )(r,s) is surjective, it is sufficient to verify that for all M ∈ T0,k (r, s), we have (Φ0,k )(r,s) ([M]) = M as G ⋅ M ∩ T0,k (r, s) = {M}. To achieve the proof, we prove that (Φ0,k )(r,s) is bicontinuous. Let (π0,k )(r,s) be the canonical surjection (π0,k )(r,s) : ̃ 0,k )(r,s) = (Φ0,k )(r,s) ∘ R0,k (r, s) → R0,k (r, s)/G. Thus, we can easily see the continuity of (Φ (π0,k )(r,s) , which is equivalent to the continuity of (Φ0,k )(r,s) . Finally, it is clear that ((Φ0,k )(r,s) )−1 = ((π0,k )(r,s) )|T0,k (r,s) , so the bicontinuity follows. Proposition 4.4.13. Assume that k = 2, then T0,2 is described in Proposition 4.4.12 and Rj,2 /G is homeomorphic to Tj,2 for j = 1, 2, 3 given as follows: (i) If h ⊄ g0 , then we have the following subcases: (i1 ) if q < n, then Tj,2 = 0, j = 1, 2, 3. (i2 ) if q = n, then

T1,2

T2,2

x 0 { } { } { } { y 0 × × ×} = {(→ ) ∈ M (ℝ) : x ∈ ℝ , β ∈ ℝ , y ∈ ℝ , 󳨀 → 󳨀 n+1,2 } { } { } { 0 0 } { 0 β }

0 x { } { } { } { 0 y × ×} ) ∈ Mn+1,2 (ℝ) : x ∈ ℝ , y ∈ ℝ, β ∈ ℝ } = {(→ 󳨀 → 󳨀 { } { } { 0 0 } } { β 0

and

T3,2

x λx { } } { { } { y λy × × ×} = {(→ ) ∈ M (ℝ) : x ∈ ℝ , y ∈ ℝ, β ∈ ℝ , λ ∈ ℝ . 󳨀 → 󳨀 n+1,2 } } { { 0 0 } } { { 0 β }

(ii) If h ⊂ g0 , then: (ii1 ) if Yn ∈ h, then Tj,2 = 0, j = 1, 2, 3.

(4.69)

212 | 4 The deformation space for nilpotent Lie groups (ii2 ) If Yn ∈ ̸ h, then Tj,2 , j = 1, 2, 3 are given by

T1,2

T2,2

x 0 { } { } { } { y 0 × ×} ) ∈ M (ℝ) : x ∈ ℝ , y ∈ ℝ, β ∈ ℝ , = {(→ → 󳨀 n+1,2 } { 󳨀 } { } { 0 0 } { 0 β }

0 x { } { } { } { 0 y × ×} = {(→ ) ∈ M (ℝ) : x ∈ ℝ , β ∈ ℝ 󳨀 → 󳨀 n+1,2 } { 0 0 } { } { } β 0 { }

and

T3,2

x λx } { } { } { { y λy × × ×} . ) ∈ M (ℝ) : x ∈ ℝ , β ∈ ℝ , λ ∈ ℝ = {(→ 󳨀 → 󳨀 n+1,2 } } { } { 0 0 } { } { 0 β

Proof. It is clear that whenever Rj,2 = 0, j = 1, 2, 3 we have Tj,2 = 0, j = 1, 2, 3 and therefore T (Γ, G, H) ≃ T0,2 . So, we only have to treat the case when h ⊄ g0 for q = n and the case when Yn ∈ ̸ h and h ⊂ g0 . Let in a first time x y1 M = (→ 󳨀y z

λx λy1 󳨀y ) ∈ R3,2 . λ→ λz + β

We get that x { { { { y1 G ⋅ M = {(→ 󳨀 { { a { { b

λx } } } λy1 t→ n−2 } 󳨀 ) : b ∈ ℝ, a ∈ ℝ → 󳨀 } } λa } } λb + β }

and then

G ⋅ M ∩ T3,2

x { { { { y1 = {(→ 󳨀 { { 0 { { 0

λx } } } λy1 } ) . → 󳨀 } } 0 } } β }

The other cases follow. This achieves the proof of the proposition. The same analysis gives us the following.

(4.70)

4.4 Deformation space of threadlike nilmanifolds | 213

Proposition 4.4.14. Assume that k = 1. The layer T0,1 being described in Proposition 4.4.12, we have that R1,1 /G is homeomorphic to T1,1 , where: (i) If h ⊄ g0 , then T1,1

x { } { } = {(y1 ) ∈ Mn+1,1 (ℝ) : x ∈ ℝ× , y1 ∈ ℝ× } . { → } 󳨀 { 0 }

(ii) If h ⊂ g0 , then { {

x

} }

T1,1 = {(y1 ) ∈ Mn+1,1 (ℝ) : x ∈ ℝ , y1 ∈ ℝ} . } {

{

×

→ 󳨀 0

}

Summarizing Sections 4.4.2 and 4.4.3, we get the following. Theorem 4.4.15. Let G be a threadlike nilpotent Lie group, H a closed, connected subgroup of G and Γ ≃ ℤk a discrete subgroup of G. Then the deformation space T (Γ, G, H) is described as follows: r,s

T (Γ, G, H) = ∐ T0,k (Γ, G, H) ∐ Tj,k (Γ, G, H), (r,s)∈Jk

j∈Ik \{0}

r,s where T0,k (Γ, G, H) is homeomorphic to T0,k (r, s) for any (r, s) ∈ Jk and Tj,k (Γ, G, H) to Tj,k for any j ∈ Ik \ {0}.

4.4.4 Case of non-Abelian discontinuous groups A first result on algebra homomorphisms Let G be a threadlike Lie group, Γ a discrete non-Abelian subgroup of G and L = exp l be its syndetic hull. The set of all injective homomorphisms from l to g denoted by Hom0 (l, g), which rather than Hom(l, g) itself will be of interest in the next section, merely because it is involved in deformations and is viewed as a starting means to study the parameter and the deformation spaces. Our main result in this section consists in giving an explicit description of its structure. In general, this set fails in most of the cases to be equipped with a smooth manifold structure. Clearly, Hom(l, g) is homeomorphically identified to a set U of Mn+1,k (ℝ) and Hom0 (l, g) of all injective homomorphisms to the subset U 0 of U consisting of the totality of matrices in U of maximal rank. Obviously, the set U is closed and algebraic in Mn+1,k (ℝ) and U 0 is semialgebraic and open in U . Our first result is the following.

214 | 4 The deformation space for nilpotent Lie groups Theorem 4.4.16. Let g be a threadlike Lie algebra and l a k-dimensional non-Abelian subalgebra of g. Then Hom0 (l, g) is endowed with a smooth manifold structure of dimension n + 4 for k = 3 and of dimension n + k whenever k > 3. For k = 3, Hom0 (l, g) is a disjoint union of an open dense smooth manifold and a closed smooth manifold of dimension n + 3. Proof. We fix a basis B = {X, Y1 , . . . , Yn } of g with nontrivial Lie brackets defined in (1.1.13) such that l = span{X, Yp , . . . , Yn }, where p = n − k + 2. We identify any element T = tX + ∑ni=1 ti Yi ∈ g to the column vector t (t, t1 , . . . , tn ). Let { { { U = {⌊T1 , . . . , Tk ⌋ ∈ Mn+1,k (ℝ) { { {

󵄨󵄨 󵄨󵄨 [T1 , Tj ] = Tj+1 , 2 ≤ j ≤ k − 1, } } 󵄨󵄨 } 󵄨󵄨 , 󵄨󵄨 [Tr , Ts ] = 0, 2 ≤ r, s ≤ k, } } 󵄨󵄨󵄨 } 󵄨󵄨 [T , T ] = 0 󵄨 1 k }

where the symbol ⌊T1 , . . . , Tk ⌋ merely designates the matrix constituted by means of the columns T1 , . . . , Tk . We first show that Hom(l, g) is homeomorphic to U . Any φ ∈ Hom(l, g) is determined by φ(X) and φ(Yj ) for j = p, . . . , n, where n

φ(X) = xX + ∑ yi Yi i=1

n

and φ(Yj ) = xj X + ∑ yij Yi , i=1

for some x, xj , yi , yij ∈ ℝ. Let Ψ : Hom(l, g) → Mn+1,k (ℝ) be the injection map defined by Ψ(φ) = ⌊φ(X), φ(Yp ), . . . , φ(Yn )⌋.

(4.71)

Now φ ∈ Hom(l, g) satisfies φ([X, Yj ]) = [φ(X), φ(Yj )]

and [φ(Yi ), φ(Yj )] = 0,

i, j = p, . . . , n.

(4.72)

This entails that Ψ(Hom(l, g)) ⊂ U . Let conversely ⌊T1 , . . . , Tk ⌋ ∈ U , we can define an algebras homomorphism: φ : l → g satisfying φ(X) = T1

and φ(Yn−k+j ) = Tj ,

j = 2, . . . , k

in such a way that Ψ(φ) = ⌊T1 , . . . , Tk ⌋. Thus, Ψ(Hom(l, g)) = U . By identifying gk = g × ⋅ ⋅ ⋅ × g to the space Mn+1,k (ℝ), we can easily see the bicontinuity of Ψ. We now identify any homomorphism φ ∈ Hom(l, g) through its corresponding matrix Ψ(φ) ∈ U subject of deal from now on. Let M = ⌊U, Vp , . . . , Vn ⌋ ∈ Mn+1,k (ℝ),

4.4 Deformation space of threadlike nilmanifolds | 215

where n

U = uX + ∑ ui Yi i=1

n

and Vj = vj X + ∑ vi,j Yi ,

p ≤ j ≤ n.

i=1

From equations (4.72), M ∈ U if and only if [U, Vj ] = Vj+1 for any p ≤ j ≤ n − 1, { { { [U, Vn ] = 0, { { { {[Vi , Vj ] = 0 for any p ≤ i, j ≤ n. This gives rise to the following equations: vj+1 = v1,j+1 = 0, p ≤ j ≤ n − 1, (1) { { { { { {uvi,j − ui vj = vi+1,j+1 , 1 ≤ i ≤ n − 1, p ≤ j ≤ n − 1, (2) { { (3) {uvi,n − ui vn = 0, 1 ≤ i ≤ n − 1, { { { (4) {vs vi,r − vr vi,s = 0, p ≤ r, s ≤ n, 1 ≤ i ≤ n − 1.

(4.73)

We now focus on the system (4.73). Thanks to equation (1), equation (2) gives that vi,p+j = 0 for all i = 2, . . . , n − p and j = 2, . . . , n − p satisfying i ≤ j. M thus shapes as u u1 ( ( u2 ( ( . ( .. ( M=( ( ( un−p ( (u ( n−p+1 .. . ( un

vp v1,p

0 0

0 0

v2,p .. .

v2,p+1 .. .

0 .. .

vn−p,p vn−p+1,p .. . vn,p

vn−p,p+1 vn−p+1,p+1 .. . vn,p+1

vn−p,p+2 vn−p+1,p+2 .. . vn,p+2

⋅⋅⋅ ⋅⋅⋅ .. . .. . .. . ⋅⋅⋅ ⋅⋅⋅

0 0 ) ) ) ) ) ) ). ) 0 ) ) vn−p+1,n ) ) .. . 0 .. .

vn,n )

We are conclusively led to the following discussions: Case 1: If u ≠ 0, then by equation (3), we have vi,n = 0

for all 1 ≤ i ≤ n − 1,

(4.74)

216 | 4 The deformation space for nilpotent Lie groups as vn = 0. Assume for a while that vp = 0. Then equation (2) gives u u1 u2 .. .

( ( ( ( ( M=( ( ( un−p ( (u ( n−p+1 .. . ( un

vp v1,p v2,p .. .

0 0 uv1,p .. .

vn−p,p vn−p+1,p .. . vn,p

uvn−p−1,p uvn−p,p .. . uvn−1,p

0 0 0 .. .

⋅⋅⋅ ⋅⋅⋅ .. ..

.

. ⋅⋅⋅

) ) ) ) ) ). ) ) 0 ) un−p vn−p,n−1 ) ) .. . un−p vn−1,n−1 )

⋅⋅⋅

By (4.74), we have vi,p = 0 for all i = 1, . . . , p − 1. This shows that M is of the form u u1 ( .. ( . ( (u ( p−1 M0 (U, V) := ( ( up ( ( (u ( p+1 .. . ( un

0 0 .. . 0

vp,p

0 0 .. . 0 0

vp+1,p .. . vn,p

uvp,p .. . uvn−1,p

0 0 .. . 0 0 .. .

⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ .. . .. . ⋅⋅⋅

) ) ) ) ) ). ) ) ) ) )

0 u vp,p ) n−p

Set H0,k = {M0 (U, V) ∈ Mn+1,k (ℝ) : u ∈ ℝ× }.

(4.75)

Let now vp ≠ 0. Using equations (2), (4) and (4.74), there exists λ ∈ ℝ× such that → 󳨀 Vp = t (λu, λu1 , . . . , λun−2 , vn−1,p , vn,p ). So, Vp+1 = t (0, . . . , 0, u(vn−1,p − λun−1 )) and Vi = 0 for all i = p + 2, . . . , n. We get finally that M belongs to the set defined as H1,k = {M1 (U, V, λ) ∈ Mn+1,k (ℝ) : λu ∈ ℝ× },

(4.76)

where u u1 ( .. ( . M1 (U, V, λ) := ( ( (u n−2 un−1 ( un

λu λu1 .. .

0 0 .. .

λun−2 vn−1 vn

0 0 u(vn−1 − λun−1 )

0 0

⋅⋅⋅

.. . 0

⋅⋅⋅

0 0 ) ) ). .. ) .) 0)

4.4 Deformation space of threadlike nilmanifolds | 217

→ 󳨀 Case 2: If u = 0, then Vp+1 = t (0, 0, −u1 vp , . . . , −un−1 vp ) and Vi = 0 for all i = p + 2, . . . , n by equation (2). When vp ≠ 0, we have by equation (4) that uj = 0 for all j = 1, . . . , n − 2 that is, U = t (0, . . . , 0, un−1 , un ) and then M reads 0 0 ( .. ( . M2 (U, V) := ( ( ( 0 un−1 ( un

v v1 .. .

0 0 .. .

vn−2 vn−1 vn

0 0 −vun−1

0 0 .. . .. . 0

⋅⋅⋅ ⋅⋅⋅

⋅⋅⋅

0 0 .. ) .) ). .. ) .) 0)

Set again H2,k = {M2 (U, V) ∈ Mn+1,k (ℝ) : v ∈ ℝ× }.

(4.77)

Otherwise, → 󳨀 → 󳨀 M ∈ H3,k = {⌊U, V, 0 , . . . , 0 ⌋ ∈ Mn+1,k (ℝ) : t U and t V ∈ {0} × ℝn }.

(4.78)

To determine a stratification of Hom0 (l, g), we note for all k ∈ {3, . . . , n + 1}, j ∈ {0, . . . , k} and Kj,k = Hom0 (l, g) ∩ Hj,k . So, it is not hard to check that for all M ∈ Hj,k , we have rank(M) ≤ 3 for j ∈ {1, 2} and that Hom0 (l, g) ∩ H3,k = 0 as l is not Abelian. In addition, M is of maximal rank if and only if rank(M) = k. So, Hom0 (l, g) ∩ Hj,k = 0 for j ∈ {1, 2, 3} whenever k > 3. With the above in mind, the following result is immediate. Proposition 4.4.17. Let g be a threadlike Lie algebra and l a k-dimensional non-Abelian subalgebra of g. We have the following: (1) If k > 3, then Hom0 (l, g) is homeomorphic to K0,k = {M ∈ H0,k : vp,p ≠ 0}. ̃ (2) If k = 3, then Hom0 (l, g) = K 1,3 ∐ K2,3 , where K1,3 = {M ∈ H1,3 : vn−1 − λun−1 ∈ ℝ× },

(4.79)

̃ K 1,3 = K1,3 ∐ K0,3

(4.80)

and K2,3 = {M ∈ H2,3 : un−1 ∈ ℝ× }.

(4.81)

We now look at the maps ϕ0 : ℝn+1 ×ℝk−1 ∋ (U, V) 󳨃→ M0 (U, V), ϕ1 : ℝn+1 ×ℝ2 ×ℝ× ∋ (U, V, λ) 󳨃→ M1 (U, V, λ) and ϕ2 : ℝ2 × ℝn+1 ∋ (U, V) 󳨃→ M2 (U, V), which are clearly C ∞ embeddings on their closed images regarded as subsets of Mn+1,k (ℝ). This shows ̃ that K0,k , K1,3 and K2,3 and also K 1,3 are endowed with smooth manifold structures

218 | 4 The deformation space for nilpotent Lie groups with the mentioned dimensions. Hence, Lemma 4.1.30 allows to close the proof. Let M2 (U, V) ∈ K2,3 . Then clearly it is the limit of the sequence 1 s v1 sv

( .. ( M s (U, V) = ( . ( vn−2 sv

un−1 ( un

v v1 .. . vn−2 vn−1 vn

0 0 .. ) ) . ), ) 0 0 1 v − vun−1 ) s n−1

0 ̃ which shows that K 1,3 is dense in Hom (l, g). On the other hand, it is not hard to check in this case that

󵄨󵄨 󵄨󵄨 uvn−1 − vun−1 ≠ 0, Hom0 (l, g) = {M(U, V) ∈ Mn+1,3 (ℝ) 󵄨󵄨󵄨 }, 󵄨󵄨 uv − vu = 0, 1 ≤ i ≤ n − 2 i 󵄨 i where keeping the same notation as above u u1 ( M(U, V) = ( ... un−1 ( un

v v1 .. . vn−1 vn

0 0 ) .. ). . 0 uvn−1 − vun−1 )

Let V = {M(U, V) ∈ Mn+1,3 (ℝ) : uvn−1 − vun−1 ≠ 0}, which is a smooth manifold. It is clear that Hom0 (l, g) = f −1 ({0}), where f : V 󳨀→ ℝn−2 , M(U, V) 󳨃󳨀→ (uv1 − vu1 , . . . , uvn−2 − vun−2 ). Clearly, the zero point of ℝn−2 is a regular value of the C ∞ − function f , which shows that Hom0 (l, g) is a manifold. This achieves the proof of the theorem. We get right away the following description of the set Hom(l, g). We have the following. Corollary 4.4.18. Let g be a threadlike Lie algebra and l a k-dimensional non-Abelian subalgebra of g. Then Hom(l, g) is homeomorphic to the disjoint union ∐3j=0 Hj,k . The following result is a direct consequence from above. Corollary 4.4.19. The Lie group Aut(g) of automorphisms of the n-step threadlike Lie algebra g is of dimension 2n + 1 whenever n > 2 and of dimension 6 for n = 2. Remark 4.4.20. The last result is a direct consequence of Theorem 4.4.16. For n = 2, ̃ g is the three-dimensional Heisenberg Lie algebra. It is then clear that K 1,3 ∐ K2,3 is

4.4 Deformation space of threadlike nilmanifolds | 219

nothing but the set u1 { { {(u2 { { u3

v1 v2 v3

0 } } ) ∈ M3 (ℝ) : u1 v2 − u2 v1 ≠ 0} , 0 } u1 v2 − u2 v1 }

and the result follows. An insight on G-orbits As above, the group G acts on Hom(l, g) through the law g ⋆ ψ = Adg ∘ψ as in equation (3.3). Let u M=( U0

→ 󳨀 0 ) ∈ K0,k , N

(4.82)

where t U0 = (u1 , . . . , un ), N = ⌊Wp , . . . , Wn ⌋ with t Wp = (0, . . . , 0, vp , . . . , vn ) ∈ ℝn , t Wp+j = (0, . . . , 0, uj vp , . . . , uj vn−j ), j = 1, . . . , n − p, and g = exp(xX + y1 Y1 + ⋅ ⋅ ⋅ + yn Yn ) ∈ G with x, y1 , . . . , yn ∈ ℝ. We have → 󳨀 0 ), A(x)N

u g⋆M =( ′ U0

(4.83)

where

( ( A(x) = ( (

1

0

x

1

2

x ..

x 2

.. .

xn−1 ( (n−1)!

⋅⋅⋅ .. . 1

.

⋅⋅⋅

..

.

x2 2

⋅⋅⋅ .. .. x

. .

0 .. . .. ) ) .) )

0

1)

denotes the matrix of (Adexp xX )|g0 written through the basis B0 = {Y1 , . . . , Yn } and U0′ is the column vector associated to g ⋆ U − uX for U = t (u, U0 ). Let on the other ̃ hand k = 3 and M = M1 (U, V, λ) ∈ K 1,3 . A routine computation shows that for g as above, u u1 ( u′2 ( ( g ⋆ M = ( ... ( (u′ n−2 u′n−1 ′ ( un

λu λu1 λu′2 .. . λu′n−2 ′ vn−1 vn′

0 0 ) 0 ) ) .. ), . ) ) 0 0 u(vn−1 − λun−1 ))

(4.84)

220 | 4 The deformation space for nilpotent Lie groups where i−1

u′i = ui + ∑ j=1

xj−1 (xui−j − uyi−j ), j! n−2

′ vn−1 = vn−1 + λ ∑ j=1

i = 2, . . . , n,

xj−1 (xun−j−1 − uyn−j−1 ) j!

(4.85) (4.86)

and n−2

vn′ = vn + (xvn−1 − λuyn−1 ) + λ ∑ j=2

x j−1 (xun−j−1 − uyn−j−1 ). j!

(4.87)

When submitted to the layer K2,3 , the action of G is likewise described as in equation (4.84) for λ = 0 and after having substituted the first two columns. The following upshot is a direct consequence from above. Corollary 4.4.21. Let g be a threadlike Lie algebra, and l a subalgebra of g. Then G = exp(g) acts on Hom0 (l, g) with constant dimension orbits if and only if dim l > 2. Proof. Let l be non-Abelian. Then the G-orbits in K0,k are uniformly n-dimensional. For k = 3, the orbits in K1,3 and in K2,3 are also n-dimensional as in equation (4.84). Suppose now that l is Abelian, then the G-orbits in Hom0 (l, g) are uniformly onedimensional whenever dim l > 2. For k = 1, 2, we have {dim(G ⋆ M) : M ∈ Hom0 (l, g)} = {1, n − 1}. We are now in measure to prove the following result. Theorem 4.4.22. Let G be a threadlike Lie group and Γ a non-Abelian discrete subgroup of G of rank k. Then: (1) For any H ∈⋔gp (Γ : G), the deformation space T (Γ, G, H) is a Hausdorff space. (2) For k > 3, T (Γ, G, H) is endowed with a smooth manifold structure. (3) For k = 3, T (Γ, G, H) is a disjoint union of an open dense smooth manifold and a closed, smooth manifold. Proof. The proof will be divided into several steps. With the above in mind, we first prove the following. Lemma 4.4.23. The disjoint components Rj,k , j = 0, 1, 2 are G-invariant. Proof. Let M ∈ R0,k be as defined in (4.82) and g = exp(xX + y1 Y1 + ⋅ ⋅ ⋅ + yn Yn ) ∈ G, where x, y1 , . . . , yn ∈ ℝ. By equation (4.83), g ⋆ M ∈ R0,k . Suppose now that k = 3. Let M = M1 (U, V, λ) ∈ R1,3 and g ∈ G, then g ⋆ M = M1 (U ′ , V ′ , λ) by equation (4.84), where ′ u′i , i = 2, . . . , n, vn−1 and vn′ are as in equations (4.85), (4.86) and (4.87), respectively. As ′ ′ (vn−1 − λun−1 ) = (vn−1 − λun−1 ), we get that g ⋆ M ∈ R1,3 . We opt for the same arguments to show that R2,3 is G-invariant.

4.4 Deformation space of threadlike nilmanifolds | 221

We now prove the following. Lemma 4.4.24. We keep all our notation as above, the set R0,k /G is homeomorphic to the set T0,k given by 󵄨󵄨 t 󵄨󵄨 U = (u, u1 , 0, . . . , 0), }. T0,k = {M0 (U, V) ∈ R0,k 󵄨󵄨󵄨 󵄨󵄨 V = t (0, . . . , 0, v , 0, v , . . . , v ) p p+2 n 󵄨

(4.88)

If k = 3, then Rj,3 /G is homeomorphic to Tj,3 for j = 1, 2 given as follows: (i) If h ⊄ g0 , then h = span{X + h1 Y1 + ⋅ ⋅ ⋅ + hn Yn } for some h1 , . . . , hn ∈ ℝ and 󵄨󵄨 t 󵄨󵄨 U = (u, u1 , 0, . . . , 0), 󵄨󵄨 } 󵄨󵄨 󵄨󵄨 V = t (0, . . . , 0, vn−1 , 0)

(4.89)

󵄨󵄨 t 󵄨󵄨 U = (0, . . . , 0, un−1 , 0), T2,3 = {M2 (U, V) ∈ R2,3 󵄨󵄨󵄨 }. 󵄨󵄨 V = t (v, v , 0, . . . , 0) 1 󵄨

(4.90)

T1,3 = {M1 (U, V, λ) ∈ R1,3 and

h (ii) If h ⊂ g0 , then IB ⊂ {1, . . . , n − 2} and T1,3 and T2,3 are defined respectively as in (4.89) and (4.90).

Proof. We show that T0,k is a cross-section of all adjoint orbits of R0,k . Let M ∈ R0,k as in (4.82), then by (4.83) we have G ⋆ M = [M] = {(

→ 󳨀 0 ) : t Y ∈ ℝn−1 , x ∈ ℝ} , A(x)N

u A(x)U0 − uB(x)Y

where

( ( ( B(x) = ( ( ( (

0 1

0 0

x 2

1

⋅⋅⋅ ⋅⋅⋅ .. .

x 3!

x 2

1

2

.. .

xn−2 ( (n−1)!

..

.

⋅⋅⋅

..

⋅⋅⋅ ⋅⋅⋅ ..

.

..

x2 3!

x 2

. .

Noting a ) A′ (x)

A(x) = (

0 0 .. ) .) ) .. ) ) ∈ Mn,n−1 (ℝ). .) ) 0 1)

(4.91)

222 | 4 The deformation space for nilpotent Lie groups with a = (1, 0, . . . , 0) and 0 ), B′ (x)

B(x) = ( we pose xM = −

vp+1 vp

and YM the unique solution of the linear equation uB′ (xM ) Y = A′ (xM ) U0 .

We then see that u G ⋆ M ∩ T0,k = {( A(xM )U0 − uB(xM )YM

0 )} = {XM }. A(xM )N

(4.92)

We now prove that the map: Φ0,k : R0,k /G [M]

→ 󳨃→

T0,k XM

(4.93)

is a homeomorphism. First of all, it is clear that Φ0,k is well-defined. The injectivity Φ0,k is obvious. To see that Φ0,k is surjective, it is sufficient to verify that for all M ∈ T0,k , we have Φ0,k ([M]) = M as G ⋆ M ∩ T0,k = {M}. Let π0,k be the canonical surjection π0,k : R0,k → R0,k /G. Thus, we can easily see the continuity of Φ0,k ∘ π0,k , which is equivalent to the continuity of Φ0,k . Finally, it is clear that Φ−1 0,k = (π0,k )|T0,k , so the bicontinuity follows. Assume now that k = 3. Let M = M1 (U, V, λ) ∈ R1,3 where t U = (u, u1 , . . . , un ) and V = t (0, . . . , 0, vn−1 , vn ). We get that { { { G ⋆ M = {M1 (U ′ , V ′ , λ) ∈ R1,3 { { {

󵄨󵄨 U ′ = t (u, u , u′ . . . , u′ ) ∈ ℝn+1 , 󵄨󵄨 1 2 n } } 󵄨󵄨 } 󵄨󵄨󵄨 V ′ = t (0, . . . , 0, v′ , v′ ) ∈ ℝn+1 , . 󵄨󵄨 } n−1 n } 󵄨󵄨 ′ } 󵄨󵄨 v − λu′ = v − λu 󵄨 n−1 n−1 n−1 } n−1

(4.94)

0 There exists then vn−1 ∈ ℝ× such that G ⋆ M ∩ T1,3 = {M1 (U 0 , V 0 , λ)} ⊂ R1,3 where 0 t 0 0 U = (u, u1 , 0, . . . , 0), V = t (0, . . . , 0, vn−1 , 0) and the map:

Φ1,3 : R1,3 /G [M]

→ 󳨃→

T1,3

M1 (U 0 , V 0 , λ)

(4.95)

is a homeomorphism. The other cases follow. This achieves the proof of the lemma. We finally prove that the deformation space is a Hausdorff space. When k > 3, T (Γ, G, H) = R0,k /G

4.4 Deformation space of threadlike nilmanifolds | 223

and the result is immediate by Lemma 4.4.24. Otherwise, T (Γ, G, H) = ∐2j=0 (Rj,3 /G). Let [M1 ] and [M2 ] be elements of T (Γ, G, H) such that [M1 ] ≠ [M2 ]. With the above in mind, we can choose the matrices Mj of the following form: uj u1,j → 󳨀 Mj = ( 0 αj (0

vj 0 v1,j 0 → 󳨀 → 󳨀 0 0) , ′ αj 0 0 βj )

j = 1, 2.

Now [M1 ] ≠ [M2 ] is equivalent to u1 u1,1

(β1 , (

v1 u )) ≠ (β2 , ( 2 v1,1 u1,2

v2 )) v1,2

which enough to conclude. As for the smooth structure of the deformation space, the result is immediate whenever k > 3. Suppose now that k = 3. Lemma 4.4.24 gives that R2,3 /G is a smooth manifold. Moreover, as R2,3 is closed in R (l, g, h) and G-invariant, we see that π(R2,3 ) is closed in T (Γ, G, H) where π : R (Γ, G, H) → T (Γ, G, H) is the ̃ canonical surjection. Now we denote by T 1,3 = T0,3 ∐ T1,3 . Using the same arguments of Lemma 4.4.24, we can prove that the map ̃ ̃ Φ 1,3 : R1,3 /G [M]

→

̃ T 1,3

󳨃→

̃ G⋆M∩T 1,3

̃ is a homeomorphism. We get therefore that R 1,3 /G is a smooth manifold, and which is clearly by Theorem 4.4.16, a dense subset of T (Γ, G, H).

5 Local and strong local rigidity Let G be a Lie group and Γ ⊂ G a finitely generated discrete subgroup. Let Hom(Γ, G) denote as earlier the set of deformation parameters of Γ in G, that is, the space of all homomorphisms Γ → G endowed with the topology of pointwise convergence. The subgroup Γ is said to be locally rigid if there is a neighborhood Ω of the inclusion map ρ0 : Γ → G in Hom(Γ, G) such that any ρ ∈ Ω is conjugate to ρ0 under the action of G (such ρ is called a trivial deformation). The study of local rigidity plays a crucial role in the deformation theory of discontinuous groups. For instance, in the case where G is a compact linear simple Lie group and H its maximal compact subgroup, there exists an equivalence between the existence of a uniform lattice φ : Γ → G such that φ ∈ R (Γ, G, H) is not locally rigid and the fact that G is locally isomorphic to SL2 (ℝ). On the hand, for an irreducible Riemannian symmetric space G/H of dimension ≥ 3 with a compact subgroup H and Γ a uniform lattice of G/H, there does not exist any essential deformation of Γ. This result, proved by Selberg and Weil (cf. [123]), claims that the deformation space is discrete in this context and can be regarded as the original model for various kinds of rigidity theorems in Riemannian geometry. Besides, the local rigidity does not hold in general in the non-Riemannian case as remarked first by T. Kobayashi. So once again referring to Chapter 4, it does make sense to determine explicitly the deformation space of a discontinuous action as it provides comprehensive information of its local structure, namely the Hausdorfness and the local rigidity. Throughout the chapter, we pay attention to the local rigidity property using some previous results. We substantiate the local rigidity conjecture in the nilpotent setting, which states that the local rigidity fails to hold for any nontrivial discontinuous group of a nilpotent homogeneous space. We further extend our study to many exponential and solvable settings. In this context, deformations of discontinuous groups means purely deformations of group homomorphisms unlike the semisimple setting, where the G-action on G/H is not always effective, and thus the space of group theoretic deformations (formal deformations) could be larger than geometric deformation spaces. We determine the corresponding deformation space and also its quotient modulo uneffective parts when rank Γ = 1. Unlike the context of the exponential solvable case, we prove the existence of formal colored discontinuous groups. That is, the parameter space admits a mixture of locally rigid and formally nonrigid deformations. We are also concerned with an analogue of the so-called Selberg–Weil–Kobayashi local rigidity theorem in the context of a real exponential group G and H a maximal subgroup of G, where the local rigidity property is shown to hold if and only if the group G is isomorphic to the group Aff(ℝ) of affine transformations of the real line. One substantial ingredient, which has made such an achievement possible, is that any Abelian discrete subgroup of G admits a syndetic hull in G, a unique connected analytic subgroup of G containing it cocompactly (cf. Chapter 1). https://doi.org/10.1515/9783110765304-005

226 | 5 Local and strong local rigidity

5.1 The local rigidity conjecture 5.1.1 The concept of (strong) local rigidity A. Weil [130] introduced the notion of local rigidity of homomorphisms in the case where the subgroup H is compact. T. Kobayashi [97] generalized it in the case where H is not compact. For the noncompact setting, the local rigidity does not hold in the general non-Riemannian case and has been studied in [31, 34, 93, 95, 98]. For comprehensible information, we further refer the readers to [57, 73, 77, 92, 92, 94, 97] and some references therein. For φ ∈ R (Γ, G, H), the discontinuous group φ(Γ) for the homogeneous space G/H is said to be locally rigid (resp., strongly locally rigid) (cf. [97]) as a discontinuous group of G/H if the orbit of φ through the inner conjugation is open in R (Γ, G, H) (resp., in Hom(Γ, G)). This means equivalently that any point sufficiently close to φ should be conjugate to φ under an inner automorphism of G. So, the homomorphisms, which are locally rigid are those which correspond to those which are isolated points in the deformation space T (Γ, G, H). When every point in R (Γ, G, H) is locally rigid, the deformation space turns out to be discrete and then we say that the Clifford–Klein form Γ\G/H cannot deform continuously through the deformation of Γ in G. If a given φ ∈ R (Γ, G, H) is not locally rigid, we say that it admits a continuous deformation and that the related Clifford–Klein form is continuously deformable.

5.1.2 The nilpotent setting We first restrict to the setting of nilpotent Lie groups. We substantiate the following local rigidity conjecture for nilpotent Lie groups. Conjecture 5.1.1 (A. Baklouti, cf. [11]). Let G be a connected simply connected nilpotent Lie group, H a connected subgroup of G and Γ a nontrivial discontinuous group for G/H. Then the local rigidity fails to hold. Note that we have a first element of an answer in the case of Abelian discontinuous groups. We have the following. Proposition 5.1.2. Conjecture 5.1.1 holds in the case of Abelian discontinuous groups. Proof. Let φ ∈ R (l, g, h) and Lφ = ℝ>0 ⋅ φ. Clearly, Lφ ⊂ R (l, g, h) and G ⋅ ψ ∩ Lφ = {ψ} for any ψ ∈ Lφ , as G acts unipotently on Hom(l, g). The projection π : Lφ → ℝ>0 ⋅ [G ⋅ φ] ⊂ T (l, g, h) is bijective, continuous and open and, therefore, bicontinuous. This means that G ⋅ φ cannot be open in ℝ>0 ⋅ [G ⋅ φ], which achieves the proof.

5.1 The local rigidity conjecture

| 227

Evidently, this proof is not adaptable to the case where the discontinuous group is no longer Abelian. We shall provide later on (cf. Remark 5.4.28) a counterexample showing its failure in the setting of general exponential Lie groups.

5.1.3 Case of 2-step nilpotent Lie groups As a direct consequence from Theorem 4.2.6, we prove that the rigidity property fails to hold. We will prove the following. Theorem 5.1.3. Let G be a connected and simply connected 2-step nilpotent Lie group. Then Conjecture 5.1.1 holds. Proof. Note first that ℝ×+ acts on Tt,β,i , i = 1, 2, by left multiplication ℝ×+ × Tt,β,i 󳨀→ Tt,β,i

(λ, φ(A, B)) 󳨃󳨀→ λ ⋅ φ = φ(λ2 A, λB). Then this action is well-defined. Indeed, we define an ℝ⋆+ -action ρ on l = [l, l] ⊕ l′ by λ2 X

ρ(λ)(X) := {

λX

for X ∈ [l, l], for X ∈ l′ .

Since l is 2-step nilpotent, this action preserves the Lie algebra structure of l. Therefore, this induces an ℝ⋆+ -action on L (l, g). Its restriction is nothing but the ℝ⋆+ -action on Tt,β,i . Suppose that there is a local rigid homomorphism φ(A, B) ∈ R (l, g, h), then its class [φ(A, B)] is an open point in T (l, g, h) and there exists β and t such that [φ(A, B)] ∈ Tt,β,i for some i ∈ {1, 2}. It follows that the image of [φ(A, B)] denoted also by [φ(A, B)], in the semialgebraic set Tt,β,i is an isolated point. Therefore, any continuous action of ℝ×+ on Tt,β,i fixes [φ(A, B)]. This means that for any λ ∈ ℝ×+ there is g(λ) ∈ G such that λ ⋅ φ(A, B) = g(λ) ⋅ φ(A, B) = Adg(λ) φ(A, B), for some g(λ) ∈ G where Adg(λ) is defined as in (4.33). This is equivalent to A1 A2 λ ⋅ φ(A, B) = ( 0 0

B1 + σ1 (g(λ))B3 + δ1 (g(λ))B4 B2 + σ2 (g(λ))B3 + δ2 (g(λ))B4 ) B3 B4

228 | 5 Local and strong local rigidity for any λ ∈ ℝ×+ , and this implies that B3 ) = 0, B4

(

A1 )=0 A2

(

and

B ( 1 ) = 0, B2

which is a contradiction with the injectivity of φ(A, B). Remark 5.1.4. Recall the context of Subsection 4.1.9. Let Γ be a nontrivial discontinuous group for the homogenous space P/Δ, where P = exp(p) and p = h2n+1 × h2n+1 . Then the following shows an alternative argument of the proof of local rigidity Conjecture 5.1.1 in this case. Indeed, we consider the action of ℝ∗+ on R(l, p, D) defined by ℝ∗+ × R(l, p, D) 󳨀→ R(l, p, D)

(t, M = M(x, y, A, B)) 󳨃󳨀→ t ⋆ M := M(t 2 x, t 2 y, tA, tB). The group ℝ∗+ acts on T (l, p, D) by the following: t ⋅ [M] = [t ⋆ M],

M ∈ R(l, p, D),

t ∈ ℝ∗+ .

According to (4.24), Adexp X ⋅M = M(x + (u − v)A + uB, y + vA, A, B) := Ad(u,v) ⋅M,

u, v ∈ ℝ2n .

Then t ⋆ Ad(u,v) ⋅M = M(t 2 x + t 2 (u − v)A + t 2 uB, t 2 y + t 2 vA, t 2 A, t 2 B) = Ad(tu,tv) ⋅(t ⋆ M). That is, the ℝ∗+ -action on T (l, p, D) is well-defined. Moreover, for s, t ∈ ℝ∗+ , t ⋅ [M] = s ⋅ [M] if and only if t ⋆ M = G ⋅ (s ⋆ M). Hence s = t by formula (4.24). This shows that [M] lies in a one-dimensional curve and, therefore, it cannot be an open point inside T (l, p, D). 5.1.4 The threadlike case As a direct consequence from Theorem 4.4.15, we get the following result concerning the property of local rigidity in the threadlike case. We have the following. Theorem 5.1.5. Conjecture 5.1.1 holds for threadlike nilpotent Lie groups.

5.2 Local rigidity for exponential Lie groups We now focus on the exponential setting. We first treat some general cases where the statement of Conjecture 5.1.1 holds. Let us start with the setting where H ⊂ G a nor-

5.2 Local rigidity for exponential Lie groups | 229

mal connected subgroup of an exponential solvable Lie group G, and Γ ≃ ℤk . A direct consequence from Theorem 3.3.1 is the following important fact concerning the topological features of Clifford–Klein forms. Theorem 5.2.1. Let G be an exponential solvable Lie group, H ⊂ G a normal connected subgroup and Γ ≃ ℤk a discrete subgroup of G. Then R (Γ, G, H) is an open set in Hom(Γ, G). If in addition H contains [G, G], then every Clifford–Klein form Γ\G/H is continuously deformable. Actually, the local rigidity property fails to hold. Proof. As G is an exponential solvable Lie group, the discrete subgroup Γ ≃ ℤk of G admits a syndetic hull L = exp(l). Then R (Γ, G, H) is homeomorphic to the set 󵄨󵄨 󵄨󵄨 dim ψ(l) = dim l and {ψ ∈ Hom(l, g) 󵄨󵄨󵄨 } 󵄨󵄨 ψ(l) ∩ h = {0} 󵄨 as provided by Proposition 3.3.7. It follows therefore that R (Γ, G, H) is homeomorphic to the set {ψ ∈ Hom(l, g) : ψ is injective and ψ(l) ∩ h = {0}} as h is an ideal of g. Let R ′ (Γ, G, H) = {ψ ∈ Hom(l, g) : ψ is injective}, then clearly R′ (Γ, G, H) is an open set of Hom(l, g) and that R (Γ, G, H) ⊂ R′ (Γ, G, H). It is then sufficient to see that R (Γ, G, H) is an open subset of R ′ (Γ, G, H). Let (X1 , . . . , Xs ) be the basis of h as indicated above and pick a basis (Y1 , . . . , Yk ) of l. Then (ψ(Y1 ), . . . , ψ(Yk )) is a basis of ψ(l), which is a complementary basis of h inside the subalgebra h ⊕ ψ(l). It follows therefore that the rank of the matrix (

Is (0)

ψ(Y1 ), . . . , ψ(Yk ) ) (∗)

is exactly k + s for every ψ ∈ R (Γ, G, H). Such a matter shows that R (Γ, G, H) appears to be the complementary set of the zeros of a polynomial function. This achieves the proof of the first part of the theorem. As for the local rigidity, referring back to Subsection 3.3.1 let [ψ] ∈ T (Γ, G, H), which is open; then [ψ] is open in Tα for some α ∈ Is (n, k). It follows that [ψ] is open in GLk (ℝ) × ηα (Vα /G). Now the projection on the first component is an open map, which means that the image of [ψ] by the first projection is open and consists of one single point in GLk (ℝ), which is absurd. This achieves the proof of the theorem. Remark 5.2.2. The first statement of Theorem 5.2.1 can be obtained differently. Indeed, if we consider the set equality given by equation (3.12), it comes out that the set Rk (g, h) appears to be an open subset of Hom(l, g), which is identified to the space V . As R (Γ, G, H) is homeomorphic to Rk (g, h) as provided by Lemma 3.3.9, we are done. It is somehow noteworthy to point out that the arguments given to prove the first statement of last theorem could run once we consider the case where G is nilpotent and

230 | 5 Local and strong local rigidity replace the hypothesis H normal by the assumption that the Clifford–Klein form in question is compact.

5.2.1 A local rigidity theorem where [L, L] = [G, G] Back now to our settings and notation of Section 3.4.2, in particular G is completely solvable. Consider first the natural continuous action of Aut(l) on Hom(l, g), which respects R (Γ, G, H). Then we have the following. Theorem 5.2.3. Let G be a completely solvable Lie group, H a connected subgroup of G and Γ a discontinuous group for G/H such that [L, L] = [G, G]. Then R (Γ, G, H) is an open set in Hom(Γ, G) and semialgebraic. Moreover, for φ ∈ R (Γ, G, H) the following assertions are equivalent: (𝚤) φ is strongly locally rigid. (𝚤𝚤) φ is locally rigid. (𝚤𝚤𝚤) The orbit φ Aut(l) is open in Hom(l, g) and dim Aut(l) + dim φ(l)⊥ = dim g,

(5.1)

where φ(l)⊥ = {Y ∈ g, [X, Y] = 0 for all X ∈ φ(l)}. Proof. Recall that R (Γ, G, H) is open by Proposition 3.4.6, then clearly (𝚤) and (𝚤𝚤) are ′ equivalent. Let φ ∈ R (Γ, G, H), M be the corresponding matrix to φ and ξαβ (M) = (A, x, W). Suppose that G ⋅ M is open. We have G ⋅ M = cA,x,W,α (G) ⋅ M

and

Aut(l) ⋅ M =

⋃

a∈Aut(l)

acA,x,W,α (G) ⋅ M.

(5.2)

Then Aut(l)M is a union of open subsets. The Aut(l)-orbit of M is identified via ξαβ with the set Aut(l) × {(x, W)}. The restriction of the canonical surjection from R (Γ, G, H) → T (Γ, G, H) to the Aut(l)-orbit of M is a continuous map, and from (3.26) its image is homeomorphic to the homogeneous space Aut(l)/cx,W,α (G). Now the strong local rigidity of φ implies that the image of φ in the homogeneous Hausdorff space Aut(l)/ cx,W,α (G) is open and closed. This means that this point is a connected component of Aut(l)/cx,W,α (G), in particular dim(Aut(l)/cx,W,α (G)) = 0. We now prove the following. Lemma 5.2.4. dim cx,W,α (G) = dim(g) − dim W ⊥ . Proof. By the definition (3.21) of cW,α , we have −1 ker(cW,α ) = {g ∈ G, Adg η−1 α (W) = ηα (W)}

= {exp(X), X ∈ g and Adexp(X) (Y) = Y, for any Y ∈ W}

= {exp(X), [X, Y] = 0, for any Y ∈ W} = exp(W ⊥ ).

5.3 Selberg–Weil–Kobayashi local rigidity theorem

| 231

Using Lemma 5.2.4, we get dim Aut(l) + dim φ(l)⊥ = dim g. Suppose now that (𝚤𝚤𝚤) holds. By the connectedness of G, we see that cx,W,α (G) is the connected component of the identity of Aut(l) and cx,W,α (G) = cA,x,W,α (G). To prove that G ⋅ M is open, it is sufficient using (5.2) to see that cA,x,W,α (G) ⋅ M is open. Let y ∈ cx,W,α (G) ⋅ M and pick an open neighborhood V0 of the identity in Aut(l) included in cx,W,α (G). Il follows therefore that V0 ⋅ y ⊂ cx,W,α (G) ⋅ M and it is enough to show that V0 ⋅ y is open in Hom(l, g). Recall that the map Aut(l) → Aut(l) ⋅ M, a 󳨃→ Ma−1 is a homeomorphism. Then V0 ⋅ y is open in Aut(l) ⋅ M, and thus in Hom(l, g). The following important consequence is therefore immediate. Corollary 5.2.5. We keep the same hypotheses and notation. If the group Aut(l) is not solvable, then the local rigidity fails to hold.

5.3 Selberg–Weil–Kobayashi local rigidity theorem In the case of semisimple Lie groups, local rigidity was first proved by Selberg [123] for uniform lattices in the case G = SLn (ℝ), n ≥ 3, and by Calabi [51] for uniform lattices in the case G = PO(n, 1) = Iso(ℍn ), n ≥ 3. Then Weil [129] generalized these results to any uniform irreducible lattice in any G, assuming that G is not locally isomorphic to SL2 (ℝ) in which case lattices have nontrivial deformations. More generally, these results may be formulated (cf. [130]). Theorem 5.3.1 (Local rigidity theorem—Selberg and Weil). Let G be a noncompact linear simple Lie group and H its maximal compact subgroup, then the following assertions on G are equivalent: (1) There exists a uniform lattice φ : Γ → G such that φ ∈ R (Γ, G, H) admits continuous deformations. (2) G is locally isomorphic to SL2 (ℝ). The following theorem (cf. [94]) produces some irreducible non-Riemannian symmetric spaces of arbitrary high dimension endowed with a uniform lattice for which the local rigidity theorem does not hold. Theorem 5.3.2. We keep the same assumptions as in Theorem 5.3.1 and let (G′ , H ′ ) := (G × G, ΔG ), where ΔG denotes the diagonal group. Then the following are equivalent: (1) There exists a uniform lattice φ : Γ → G such that φ × 1 ∈ R (Γ, G′ , H ′ ) admits continuous deformations. (2) G is locally isomorphic to SO(n, 1) or SU(n, 1). (3) G does not have Kazhdan’s property (T). Note that Theorem 5.3.1 was formulated so that these two rigidity theorems can be compared. As such, the result of Theorem 5.3.2 produces some irreducible non-

232 | 5 Local and strong local rigidity Riemannian symmetric spaces of arbitrary high dimension endowed with a uniform lattice for which local rigidity does not hold. For the Riemannian case, this is very rare as mentioned earlier. The aim of the present subsection is to derive an analogue to such results in the setting of exponential solvable Lie groups. More details could be found in the references [1, 14, 33]. Let us start by studying the case of maximal homogeneous spaces.

5.3.1 Maximal exponential homogeneous spaces Our intention in this section is to state some preliminary results with regard to the parameter and the deformation spaces of discontinuous groups acting on maximal homogenous spaces for which the underlying group in question is exponential. As in Theorem 1.1.15 and Remark 1.1.17, maximal subgroups are of codimension one or two and we have the following. Theorem 5.3.3. Let g be a exponential Lie algebra and h a maximal subalgebra of g, which is not an ideal. Then h is of codimension one or two in g and we have the following: (1) If h is of codimension one, then there exist a codimension one ideal g0 of h, which is a codimension two ideal in g, and two elements A, X in g such that g = h ⊕ ℝX,

h = g0 ⊕ ℝA

and [A, X] = X mod g0 .

(5.3)

(2) If h is of codimension two, then there exist a codimension one ideal g0 of h, which is an ideal of g of codimension 3, as well as three elements A, X, Y in g and a nonzero real number α such that g = h ⊕ ℝX ⊕ ℝY,

h = g0 ⊕ ℝA,

[A, X + iY] = (α + i)(X + iY) mod g0

(5.4)

and [X, Y] = 0 mod g0 .

(5.5)

A maximal basis of g adapted to h is any basis B constituted of a basis of g0 together with the vectors {A, X} (resp., {A, X, Y}) organized according to the order above. Making use of Theorem 2.4.4 dealing with the proper action of solvable maximal homogeneous spaces, the following turns out to be a direct consequence.

5.3 Selberg–Weil–Kobayashi local rigidity theorem

| 233

Corollary 5.3.4. Let G be an exponential Lie group, H a maximal subgroup of G and Γ a discontinuous group for the homogenous space G/H. Then Γ is isomorphic to ℤ or ℤ2 . Proof. As G is a exponential solvable, Γ admits a unique syndetic hull L = exp(l), which turns out to be Abelian as Γ does. Now thanks to Theorem 2.4.4, there is equivalence of the proper and free actions of L on G/H. If h denotes the Lie algebra associated to H, we get right away that l ∩ h = {0} and by the maximality of h, Γ is of rank dim l ≤ 2. The parameter space, revisited Let H = exp h be a connected subgroup of an exponential Lie group G = exp g, Γ a discontinuous group for the homogenous space G/H and L = exp l its syndetic hull. We designate by Hom(l, g) the set of all algebra homomorphisms from l to g. Let B be a maximal basis of g adapted to h. We identify g, l and the space of the linear maps L (l, g) with ℝn , ℝk and the space of matrices Mn,k (ℝ), respectively. Having fixed a basis Bl of l, it appears clear that the map Ψ : Hom(l, g) 󳨀→ Mn,k (ℝ),

(5.6)

which associates to any element of Hom(l, g) its matrix written in the bases Bl and the basis B of g is a homeomorphism on its range. Throughout the whole text, the set Hom(l, g) is therefore homeomorphically identified to a set U of Mn,k (ℝ) and Hom∘ (l, g) of all injective homomorphisms to the subset U ∘ of U consisting of all matrices in U of maximal rank. Obviously, the set U is closed and algebraic in Mn,k (ℝ). In addition, the set U ∘ is semialgebraic and open in U . Let G act on Mn,k (ℝ) by g ⋅ M = Adg ×M,

M ∈ Mn,k (ℝ) and

g ∈ G,

(5.7)

where Adg is viewed as a real valued matrix for any g ∈ G. In the setting where g = ℝX ⊕ h with h = ℝA ⊕ g0 and [A, X] = X mod(g0 ), for g = exp aA ⋅ exp xX ⋅ g0 ∈ G where a and x are in ℝ and g0 ∈ G0 , a routine computation shows that a0 a0 a g ⋅ ( x0 ) = ((x0 − xa0 )e ) , → 󳨀′ → 󳨀z z (g)

(5.8)

→ 󳨀 where z ′ (g) ∈ ℝn−2 is an analytic expression depending upon g (which means that → 󳨀 T 󳨃→ z ′ (exp T) is an analytic function on the vector space g). In the case where g = ℝX ⊕ ℝY ⊕ h with h = ℝA ⊕ g0 , [A, X + iY] = (α + i)(X + iY) mod(g0 ) and [X, Y] =

234 | 5 Local and strong local rigidity 0 mod(g0 ). For g = exp aA ⋅ exp xX ⋅ exp yY ⋅ g0 , where a, x and y ∈ ℝ and g0 ∈ G, a1 a1 αa e ((x − a (αx + y)) cos a + (y1 + a1 (x − αy)) sin a) x 1 1 g ⋅ ( 1 ) = ( αa ), e ((y + a (x − αy)) cos a − (x1 − a1 (αx + y)) sin a) y1 1 1 → 󳨀′ → 󳨀 z1 z1 (g)

(5.9)

→ 󳨀 where also z ′ (g) ∈ ℝn−3 is an analytic expression depending upon g. When Γ is of rank two, then a1 x g⋅( 1 y1 → 󳨀 z1

a1 a2 a2 x1′ x2′ x2 )=( ′ ), y1 y2′ y2 → 󳨀 → 󳨀 → 󳨀 z2 z1′ (g) z2′ (g)

(5.10)

→ 󳨀 where for i = 1, 2, zi′ (g) ∈ ℝn−3 depends analytically on g ∈ G, xi′ = eαa ((xi − ai (αx + y)) cos a + (yi + ai (x − αy)) sin a) and yi′ = eαa ((yi + ai (x − αy)) cos a − (xi − ai (αx + y)) sin a). Taking into account the action of G on Hom(l, g) defined in (3.3), the following lemma is immediate. Lemma 5.3.5. The map Ψ defined in (5.6) is G-equivariant with regard to the actions (3.3) and (5.7). That is, for any ψ ∈ Hom(l, g) and g ∈ G, we have Ψ(g ⋅ ψ) = g ⋅ Ψ(ψ). Assume now that l is Abelian. Thus, ψ ∈ L (l, g) is an algebra homomorphism if and only if ψ is a linear map satisfying [ψ(X), ψ(Y)] = 0 for all X and Y in l, which gives rise to the following expression: Hom(l, g) = {ψ ∈ L (l, g), [ψ(X), ψ(Y)] = 0 for all X and Y in l}. With the above in mind, for k = 2, the set U (resp., U ∘ ) is homeomorphic to {(M1 , M2 ) ∈ Mn,2 (ℝ), [M1 , M2 ] = 0} (resp., to the same set of matrices of maximal rank), with the convention that the Lie bracket [M1 , M2 ] represents the corresponding vectors of the Lie algebra defining the associated homomorphism. When k = 1, Hom(l, g) = Mn,1 (ℝ). We now prove the following propositions toward the determination of the parameter space R (l, g, h), which are the key points of many uses later. The first deals with the case where k = 1.

5.3 Selberg–Weil–Kobayashi local rigidity theorem

| 235

Proposition 5.3.6. Let G be an exponential Lie group of dimension n, H a nonnormal connected maximal subgroup of G and Γ a discontinuous subgroup for G/H of rank one. Then R (l, g, h) is a smooth manifold and described more precisely as follows: (i) If codim(h) = 1, R (l, g, h) is homeomorphic to n

t

n−2

󳨀z ) ∈ M (ℝ), x ∈ ℝ } ≃ ℝ × ℝ R1,1 = { (0 x → n,1 ×

×

(5.11)

.

(ii) If codim(h) = 2, then R (l, g, h) is homeomorphic to n

t

2

2

2

󳨀z ) ∈ M (ℝ), x + y ≠ 0} ≃ ℝ \ {(0, 0)} × ℝ R1,2 = { (0 x y → n,1

n−3

.

(5.12)

3 In particular, R1,2 ≃ ℝ2 \ {(0, 0)}.

󳨀z ) ∈ Proof. Let us assume first that h is of codimension one and let M = t (a0 x0 → Mn,1 (ℝ). From (5.8), M ∈ R (l, g, h) if and only if a0 a ((x0 − xa0 )e ) ∈ ̸ h → 󳨀′ z (g) for all reals x and a. This means x0 − xa0 ≠ 0 for all x ∈ ℝ, which is in turn equivalent to x0 ∈ ℝ× and a0 = 0, as was to be shown. Assume now that h is of codimension two. 󳨀 Then, for M = t (a1 x1 y1→ z1 ) ∈ Mn,1 (ℝ), we have by similar arguments: 2

2

M ∈ R (l, g, h) ⇔ (x1 − a1 (αx + y)) + (y1 + a1 (x − αy)) ≠ 0 ⇔

x12

+

y12

≠ 0

for all x, y ∈ ℝ

and a1 = 0,

which achieves the proof in this case. We now move to the case k = 2. The following proposition exhibits a description of the parameter space in this case. Proposition 5.3.7. Let G be an exponential Lie group of dimension n, H a nonnormal connected maximal subgroup of G and Γ a discontinuous group for G/H of rank two. Then: (𝚤) If codim(h) = 1, then R (l, g, h) = 0. (𝚤𝚤) If codim(h) = 2, then R (l, g, h) is homeomorphic to

n R2,2

0 0 { } { } { } { x1 x2 } = {( ) ∈ U , x1 y2 − x2 y1 ≠ 0} . { } y y { } 1 2 { } → 󳨀 → 󳨀 z z 1 2 { }

3 In particular, R2,2 = GL2 (ℝ).

(5.13)

236 | 5 Local and strong local rigidity Proof. When h is of codimension one, then h ∩ φ(l) is not trivial for any φ ∈ R (l, g, h), which is a contradiction as l is of dimension two. Hence R (l, g, h) = 0. Suppose now that h is of codimension two. Then any a1 x M=( 1 y1 → 󳨀 z1

a2 x2 )∈U y2 → 󳨀 z2

fulfills the quadratic equation a1 x2 − a2 x1 = a1 y2 − a2 y1 = 0. Then M ∈ R (l, g, h) if and only if the matrix x′ ( 1′ y1

x2′ ) y2′

is regular, where for j ∈ {1, 2}, zj′ = xj′ + iyj′ = e(α−i)a [(xj − aj (αx + y)) + i(yj + aj (x − αy))], as in equation (5.10). This gives in turn that a1 = a2 = 0 and x1 y2 − x2 y2 ≠ 0, as was to be shown.

5.3.2 Local rigidity for small-dimensional exponential Lie groups This subsection aims to study the local rigidity proprieties for which the underlying group in question is assumed to be exponential and of low dimension. We have the following. Lemma 5.3.8. (1) Let g be the Lie algebra of the group Aff(ℝ) (as in Example 1.1.4) and Γ a discontinuous group for exp(g)/ exp(h). Then the local rigidity property holds. (2) Let G = Gα (α ∈ ℝ× ) be the Lie group with the Lie algebra gα spanned by A, X, Y with [A, X + iY] = (α + i)(X + iY), h = ℝA and Γ a nontrivial discontinuous group for exp(g)/ exp(h). Then the local rigidity property fails to hold. Proof. (1) Assume that Γ is not trivial. Thanks to equation (5.8), it is clear that any 2 G-orbit in R1,1 ≃ ℝ× is either homeomorphic to (0, +∞) or to (−∞, 0), which is enough to conclude. (2) Now h is of codimension two. We argue separately according to the dimension 3 of l. Suppose first that l is of dimension two. For M ∈ R2,2 ≃ GL2 (ℝ) as in (5.13), equa-

5.3 Selberg–Weil–Kobayashi local rigidity theorem

| 237

tion (5.9) shows that the orbit of M through the action of G reads: 0 { { G ⋅ M = {( eαa (x1 cos a + y1 sin a) { αa { e (−x1 sin a + y1 cos a)

0 } } eαa (x2 cos a + y2 sin a) ) , a ∈ ℝ} , } eαa (−x2 sin a + y2 cos a) }

3 which is not an open set in R2,2 . Suppose now that l is of dimension one. It is clear 3 thanks to equation (5.10) that the G-orbit of M ∈ R1,2 as in (5.12),

G ⋅ M = {t (0 eαa (x cos a + y sin a) eαa (−x sin a + y cos a)), a ∈ ℝ}. So, the local rigidity property fails to hold globally on R (l, g, h) in this case. Now, our so-called analogue of Selberg–Weil–Kobayashi rigidity theorem in the context of maximal exponential homogeneous spaces is stated as follows. Theorem 5.3.9 (An analogue of Selberg–Weil–Kobayashi rigidity theorem). Let G be an exponential Lie group, H a nonnormal connected maximal subgroup of G and Γ a discontinuous group for G/H. Then the following assertions are equivalent: (𝚤) G is isomorphic to the group Aff(ℝ). (𝚤𝚤) Every homomorphism in R (Γ, G, H) is locally rigid. (𝚤𝚤𝚤) Some homomorphism in R (Γ, G, H) is locally rigid. The proof of Theorem 5.3.9 will be divided into different steps. First, Table 5.1 establishes the parallelism, which can be built to compare the two settings.

Table 5.1: Summarizing Selberg–Weil–Kobayashi rigidity results. G is a noncompact simple Lie group

G is an exponential Lie group

H ⊂ G is a maximal (compact) or H is the diagonal ΔG of G × G

H ⊂ G is a maximal (nonnormal)

The Clifford–Klein form Γ\G/H is compact

Γ\G/H is arbitrary

no continuous deformations

no continuous deformations

G is not locally isomorphic to SL2 (ℝ) for a Riemannian space G/H or G is not locally isomorphic to SO(n, 1) or SU(n, 1) for (G × G)/ΔG

G is isomorphic to Aff(ℝ)

238 | 5 Local and strong local rigidity 5.3.3 Passing through the quotients We keep all our assumptions and notation. G = exp g be an exponential Lie group, H = exp h a nonnormal connected maximal subgroup of G, Γ be a nontrivial discontinuous subgroup of G and L = exp l its syndetic hull. According to Lemma 5.3.11, l is an Abelian subalgebra of g of dimension one or two. We denote by g0 the ideal defined as in Theorem 5.3.3 and G0 its corresponding Lie group. Let K = exp k ⊂ H be a normal subgroup of G. We consider the quotient group G = G/K, π : G → G the canonical projection homomorphism and de π : g → g, X 󳨃→ X its derivative. Likewise, we denote H = π(H) := exp(h) and L = π(L) := exp(l). Since L is the syndetic hull of Γ, then L is a subgroup of G acting properly on G/H and then freely. So, L ∩ H is trivial and the homomorphism π̃ = π|L : L → L appears to be a groups isomorphism. Thus, the Lie algebra l = log L is isomorphic to l, and clearly L is isomorphic to L. Our intention now is to investigate the connection between the parameter spaces R (l, g, h) and R (l, g, h). It consists in fact in constructing an open surjective map between these spaces. We recall the notation Hom(l, g) (resp., Hom(l, g)) assigned to the set of all algebra homomorphisms from l to g (resp., from l to g). The group G (resp., G) acts on Hom(l, g) (resp., Hom(l, g)) as in (3.3). We consider the map ξ : Hom(l, g) φ

→

Hom(l, g)

󳨃→

φ = de π ∘ φ ∘ de π̃ −1 .

Lemma 5.3.10. The group G acts on ξ (Hom(l, g)) by the law g ⋅ ξ (φ) = g ⋅ ξ (φ)

(5.14)

making of ξ a G-equivariant map on its range. Proof. Clearly, the law (5.14) above defines a G-action on ξ (Hom(l, g)). Let, on the other hand, g ∈ G and φ ∈ Hom(l, g). Then, for X ∈ l, we have ξ (g ⋅ φ)(X) = ξ (Adg φ)(X) = Adg φ(X)

= Adg φ(X) + k

= Adg (φ(X) + k) = Adg φ(X) = Adg AdK φ(X) = AdgK φ(X)

= g ⋅ ξ (φ)(X), which shows that ξ is G-equivariant.

5.3 Selberg–Weil–Kobayashi local rigidity theorem

| 239

Weakening the assumptions of Theorem 2.4.6, we have the following. Lemma 5.3.11. Let G be an exponential Lie group and H be a maximal subgroup of G. Then any subgroup Γ of G acting on G/H freely is Abelian. Proof. Assume the same notation as in Subsection 5.3.3. One can easily see that Γ acts −1 freely on G/H. In fact, let h ∈ H, γ ∈ Γ and t ∈ G such that h = tγt . Then h = tγt −1 t0 −1 −1 −1 for some t0 ∈ G0 , which implies that ht0 = tγt ∈ H ∩ tΓt . Finally, h = t0 as Γ acts freely on G/H and, therefore, h = e. On the other hand, provided that Γ ∩ G0 is trivial, the homomorphism π̃ = π|Γ : Γ → Γ appears to be a group isomorphism. To prove that Γ is an Abelian subgroup of G, it is sufficient to prove that Γ is an Abelian subgroup of G. When H is normal, it is of codimension one, and by taking G0 = H, the result follows as G is of dimension one. Assume henceforth that H is not normal and denote by g the Lie algebra associated to G. We also denote by g0 the ideal defined as in Theorem 5.3.3 and G0 its corresponding Lie group. Suppose in a first time that h is of dimension one. It is clear that g is the Lie algebra of two dimension two spanned by the vectors A, X satisfying the bracket relation [A, X] = X and that h = ℝA. We identify G to ℝ2 with the multiplication law: ′

(a, x) ⋅ (a′ , x′ ) = (a + a′ , x′ + xe−a ). We note g1 = ℝX and G1 its Lie group. Suppose that Γ ⊄ G1 . Then there exist a ∈ ℝ× and x ∈ ℝ× such that g = (a, x) = exp aA ⋅ exp xX belongs to Γ. But this gives exp(

−x x )X ⋅ g ⋅ exp( −a )X ∈ H, e−a − 1 e −1

which contradicts the fact that Γ acts freely on G/H. This completes the proof in this case. Suppose now that h is of dimension two. Then g is the Lie algebra spanned by A, X, Y with [A, X + iY] = (α + i)(X + iY), α ∈ ℝ× and h = ℝA. We identify G to ℝ3 with the multiplication law: ′

′

(a, x, y) ⋅ (a′ , x′ , y′ ) = (a + a′ , x′ + e−αa (x cos a′ − y sin a′ ), y′ + e−αa (y cos a′ + x sin a′ )). We note g2 = ℝX ⊕ ℝY and G2 its Lie group. If Γ ⊄ G2 , then there exist a ∈ ℝ× and (x, y) ∈ ℝ2 \ {(0, 0)} such that g = (a, x, y) = exp aA ⋅ exp xX ⋅ exp yY is an element of Γ. It comes by a simple computation that exp x0 X ⋅ exp y0 Y ⋅ g ⋅ exp(−x0 X) ⋅ exp(−y0 Y) ∈ H, where x0 =

−eaα ((cos a − eaα )x + y sin a) (cos a − eaα )2 + sin2 a

240 | 5 Local and strong local rigidity and y0 =

eaα (x sin a − (cos a − eaα )y) (cos a − eaα )2 + sin2 a

.

This is impossible provided that Γ acts freely on the homogeneous space G/H. So, Γ ⊂ G2 and Γ is an Abelian subgroup of G. This completes the proof of the lemma. 5.3.4 Proof of Theorem 5.3.9 Thanks to Lemma 5.3.8, we only need to show the implication (𝚤𝚤𝚤) → (𝚤). We shall assume in what follows that the Lie group G is not isomorphic to Aff(ℝ) and show that there is no locally rigid homomorphisms inside the parameter space. We will separately tackle the following cases. Case where rank(Γ) = 2 We will take k = g0 in this case and the symbol “bar” designates along this subsection the related quotient objects. The following upshot is the starting means to study the connection between the parameter spaces R (l, g, h) and R (l, g, h). Lemma 5.3.12. We keep the same assumptions and notation as before and let Γ be of rank two. The restriction ξ ′ := ξ|R(l,g,h) of ξ to R (l, g, h) is open and onto R (l, g, h), and thus G-equivariant. Proof. Consider first the natural continuous action of Aut(l) on Hom(l, g), which ren spects R (λ, γ, h). This action induces a subsequent action of GL2 (ℝ) on the set R2,2 t n → 󳨀 → 󳨀 as defined in (5.13). More precisely, for (0 N0 z ) ∈ R2,2 with N0 ∈ GL2 (ℝ) and z ∈ Mn−3,2 (ℝ) and N ∈ GL2 (ℝ), a matrix-like expression of this action can be described as follows: MN

−1

→ 󳨀 0 = (N0 N −1 ) . → 󳨀z N −1

It is clear that g = gα for some α ∈ ℝ× and h = ℝA, where gα is as in Lemma 5.3.8. n 3 3 According to equation (5.13), we get ξ (R2,2 ) ⊂ R2,2 . Conversely, let M = t (0 N) ∈ R2,2 as t n −1 t → 󳨀 → 󳨀 in (5.13) with N ∈ GL2 (ℝ). Let M0 = (0 N0 z0 ) ∈ R2,2 . Then P = M0 N0 N = (0 N z ) ∈ n R2,2 satisfies ξ (P) = M. This shows that ξ ′ is onto and, therefore, G-equivariant by 󳨀 Lemma 5.3.10. Let A := {t (0 I2 → z0 ) ∈ Mn,2 (ℝ)} ∩ U and n ψ : R2,2 → GL2 (ℝ) × A , t 󳨀z ) 󳨃→ (N, MN −1 ) = (N, t (0 I → 󳨀 −1 M = (0 N → 2 z N )).

5.3 Selberg–Weil–Kobayashi local rigidity theorem

| 241

Clearly, ψ is well-defined and injective. On the other hand, let (N, M ′ ) ∈ GL2 (ℝ) × A , n then M = M ′ N ∈ R2,2 and ψ(M) = (N, M ′ ), which shows that ψ is onto. Finally, ψ is bijective and clearly a homeomorphism. On the other hand, it is immediate to see that 3 ξ = ι ∘ p1 ∘ ψ, where ι : GL2 (ℝ) → R2,2 , N 󳨃→ t (0 N) and p1 : GL2 (ℝ) × A → GL2 (ℝ) the natural projection map. As ι is a homeomorphism and p1 is open, ξ is an open map. Let now Ω be a G-orbit in R (l, g, h). Then ξ (Ω) is a G-orbit in R (l, g, h), which is not open by Lemma 5.3.8, nor is Ω by Lemma 5.3.12 as was to be shown. This achieves the proof in this case. Case where rank(Γ) = 1 We fix as above k an ideal of g included in g0 . The restriction ξ ′ := ξ|R(l,g,h) of ξ to R (l, g, h) is also open and onto R (l, g, h) as being a natural projection. It is immediate that in case where h is of codimension two. Lemma 5.3.8 gives us that the local rigidity fails at the level of R (l, g, h), upon the consideration of the case k = g0 , which is enough to conclude. Suppose now that h is of codimension one. In this case, we shall state the local rigidity property separately according to the dimension of the Lie algebra g0 . In the case where dim g0 = 1, g is the Lie algebra spanned by the vectors A, X, Y such that Y ∈ g0 and satisfying the brackets relations [A, X] = X + αY,

[A, Y] = βY

for some α, β ∈ ℝ. Thanks to Jacobi’s identity, one easily gets [X, Y] = 0. As R (l, g, h) ≃ ℝ× X +ℝY by Proposition 5.3.6. Then the one parameter subgroups exp ℝX and exp ℝY clearly belong to the stabilizer of any element of R (l, g, h), which in turn means that the orbit is of dimension one. So we are done in this case. We now pay attention to the case where dim g0 = 2. Then h is the Lie algebra of dimension three, which admits a basis B = {A, Y, T} with T, Y ∈ g0 , g = ℝX ⊕ h and [A, X] = X mod g0 . It comes out on the other hand that the parameter space R (l, g, h) is a smooth manifold of dimension three and is homeomorphic to ℝ× X + ℝY + ℝT by Proposition 5.3.6. Assume first that the nilpotent Lie algebra g′ = [g, g] is of dimension three. Then ′ g is either Abelian or isomorphic to the three-dimensional Heisenberg algebra. This means that for all v ∈ g′ , dim G′ ⋅ v ≤ 1 where G′ = exp(g′ ). But G = exp(ℝA)G′ and R (l, g, h) ⊂ g′ , which is enough to conclude. Assume now that dim g′ ≤ 2. If the vectors [Y, T], [X, T], [X, Y], [A, Y] and [A, T] are trivial, then obviously dim G ⋅ X0 ≤ 2 for any X0 ∈ R (l, g, h). Otherwise, we can and do assume that these vectors generate a one-dimensional subalgebra of g0 and ℝu0 . For X0 ∈ R (l, g, h), there exists a linear form fX0 ∈ g∗ , such that for any for Z =

242 | 5 Local and strong local rigidity aA + xX + yY + tT ∈ g, we have [Z, X0 ] = a[A, X0 ] + fX0 (Z)u0 . The Lie algebra of the stabilizer G(X0 ) reads g(X0 ) = {Z ∈ g; a = 0 and fX0 (Z) = 0}. Then dim g(X0 ) ≥ 2, or equivalently dim G ⋅ X0 ≤ 2, which achieves the proof in this case. We finally tackle the case where dim g0 > 2. There exists therefore an ideal k ⊂ g0 of g such that dim g0 /k = 1 or 2. Passing to the quotient with k, we are conclusively led to one of the situations above where the local rigidity fails to hold. To complete the proof, we point out that the restriction map is also open and onto in this case. Remark 5.3.13. When we remove the assumption on the maximal subgroup H to be nonnormal in G, we get the following. Corollary 5.3.14. Let G be an exponential Lie group, H a connected maximal subgroup of G and Γ a discontinuous group for G/H. If the parameter space R (Γ, G, H) admits a locally rigid homomorphism, then G is isomorphic to the group Aff(ℝ). Proof. If H is normal in G, then Γ stands to be Abelian as in the proof of Lemma 5.3.11. Looking at its syndetic hull l, the parameter space R (l, g, h) turns out to be semialgebraic and open in Hom(l, g) ≃ g. No G-orbit in R (l, g, h) is therefore open by a reason of dimensions. This means that H is not normal in G and the result follows by Theorem 5.3.9.

5.4 Criteria for local rigidity 5.4.1 Necessary condition for local rigidity using the automorphism group Aut(l) Let Gφ(l) be the stabilizer in G of φ(l) and ρφ the homomorphism defined as ρφ : Gφ(l) 󳨀→ Aut(l),

g 󳨃󳨀→ ψ ∘ Adg ∘φ,

(5.15)

φ

where ψ is the inverse of the isomorphism l → φ(l). Let Aut∘ (l) be the connected component of the identity of Aut(l). Our first result on rigidity is the following necessary condition for a locally rigid homomorphism. Theorem 5.4.1. Let G be an exponential Lie group, H a connected Lie subgroup of G and Γ a discontinuous subgroup for G/H. Let l be the Lie algebra of the syndetic hull of Γ. If φ ∈ R (l, g, h) is locally rigid, then Aut∘ (l) is contained in the range of the homomorphism ρφ defined in (5.15).

5.4 Criteria for local rigidity | 243

Proof. Note first that as a direct aftermath of Theorem 3.2.1, the group Aut(l) acts continuously on the parameter space via the law a ⋅ φ = φ ∘ a−1 ,

a ∈ Aut(l).

This action commutes with the action of G, thus Aut(l) acts on the deformation space and we can state the following result. Lemma 5.4.2. The action of Aut(l) on T (l, g, h), given by a⋅[φ] = [φ∘a−1 ], is well-defined and continuous. In particular, for any [φ] ∈ T (l, g, h), the map i[φ] : Aut(l) → O[φ] , a 󳨃→ [φ ∘ a−1 ] is continuous, where O[φ] is the orbit of [φ] under the action of Aut(l). Proof. The fact that the action of Aut(l) is well-defined is trivial. We now prove that the action is continuous. Indeed, remark first that the action of Aut(l) on R (l, g, h) is continuous. On the other hand, the quotient map R (l, g, h) → T (l, g, h) is an open continuous map. As the following diagram Aut(l) × R (l, g, h) π

?

Aut(l) × T (l, g, h)

a

a

? R (l, g, h) ?

π

? T (l, g, h)

commutes, we are done. Back now to the proof of the theorem. Let φ be locally rigid homomorphism then the class [φ] is an open point in T (l, g, h). As a direct matter of the continuity of the action of Aut(l) on T (l, g, h), we observe that the orbit O[φ] is a discrete topological space. By the continuity of the map i[φ] , we conclude that Aut∘ (l) fixes [φ]. Let a ∈ Aut∘ (l) then [φ ∘ a−1 ] = [φ], which is equivalent to the existence of g ∈ G such that Adg ∘φ = φ ∘ a−1 , in particular, g ∈ Gφ(l) and a−1 = ψ ∘ Adg ∘φ ∈ Im(ρφ ). We now introduce the following definitions. Definition 5.4.3. Let D (l) be the derivations algebra of l. It is the Lie algebra of Aut(l). A solvable (resp., a completely solvable, a nilpotent) Lie algebra for which D (l) is solvable (resp., a completely solvable, a nilpotent) Lie algebra, is said to be characteristically solvable (resp., completely solvable, nilpotent) Lie algebra. As a direct consequence of Theorem 5.4.1, we get the following. Theorem 5.4.4. The local rigidity fails to hold, in the context of the three following situations: (1) l is not characteristically solvable Lie algebra. (2) g is completely solvable and l is not characteristically completely solvable Lie algebra. (3) g is nilpotent and l is not characteristically nilpotent Lie algebra.

244 | 5 Local and strong local rigidity Proof. For any φ ∈ R (l, g, h), the subgroup Gφ(l) is a solvable Lie group and Im(ρφ ) is a quotient of Gφ(l) , then any subgroup of Im(ρφ ) is also solvable. In particular, if there is a locally rigid homomorphism φ say, by Theorem 5.4.1, we conclude that Aut∘ (l) is a subgroup of Im(ρφ ), and thus solvable. Now if G is completely solvable, the subgroups Gφ(l) and Ker(ρφ ) are connected closed subgroups of G and the quotient group Gφ(l) / ker(ρφ ) ≅ Im(ρφ ) is a completely solvable Lie group. If φ is locally rigid, then the same argument induces that Aut∘ (l) is completely solvable. When g is nilpotent, the same arguments work out.

5.4.2 Case of graded Lie subalgebras We will derive some direct consequences from Theorem 5.4.4, showing that the rigidity property fails to hold. This gives therefore a positive answer to Conjecture 5.1.1. Our first result is a direct consequence of the Theorem 5.4.4. Corollary 5.4.5. Let l = g or g is nilpotent of dimension less or than equal to 7, then Conjecture 5.1.1 holds. Proof. If l = g, then the deformation space is homeomorphic to Aut(g)/ AdG , which is smooth manifold of dimension greater than one. The deformation space contains therefore no open points. If dim(g) ≤ 7 and l ≠ g, then dim(l) ≤ 6 and it is well known that there is no characteristically nilpotent Lie algebra l for which dim(l) < 7 (cf. [109]). Definition 5.4.6. Recall that a graded Lie algebra is any Lie algebra l with a decomposition l = ⨁ ld , d∈ℤ

where ld , d ∈ ℤ are some subspaces of l, such that [li , lj ] ⊂ li+j ,

i, j ∈ ℤ.

For any t ∈ ℝ∗+ , the dilation λt defined on l by λt (v) = t d v for all v ∈ ld and extended on l by linearity, is a nonnilpotent automorphism of l and belongs to Aut∘ (l). Then l is not characteristically nilpotent. Corollary 5.4.7. Assume that g is nilpotent and l is a graded Lie algebra, then Conjecture 5.1.1 is positively answered. Remark 5.4.8. (1) Note that if g is a 2-step or a threadlike Lie algebra, then any subalgebra of g is either Abelian or a 2-step or a threadlike Lie algebra and, therefore, a graded Lie

5.4 Criteria for local rigidity | 245

algebra. It comes out that the nonlocal rigidity theorems obtained above, are direct consequences of the last corollary. (2) The necessary condition for the rigidity in Theorem 5.4.4 involves only the structure of g and l. (3) The following examples reveal that the additional assumption on l to be characteristically nilpotent might be irrelevant. Example 5.4.9. Let l be a characteristically nilpotent Lie algebra and G = ℝn × L. Then [g, g] = [l, l]. In this case, we know already that a homomorphism φ ∈ R (l, g, h) can be locally rigid only if dim φ(l)⊥ + dim(Aut(l)) = dim(g). But dim φ(l)⊥ ≥ dim(z(φ(l)) + n ≥ n + 1, and dim(Aut(l)) ≥ dim(l). So, Conjecture 5.1.1 has also a positive answer in this setup. Example 5.4.10. Suppose that g = h ⊕ l as a sum of two ideals of g. Then it is easy to see that the parameter space is identified to the product Hom(l, h) × Aut(l) and the deformation space can be written as T (l, g, h) = Hom(l, h)/ AdH × Aut(l)/ AdL .

Then if T (l, g, h) contains an open point, its image by the second projection must be open, which is impossible. So, a positive answer to question 5.1.1 holds also in this context. Remark finally that the condition on l to be characteristically nilpotent is not useful in this case. Let us derive first some immediate consequences from Theorem 5.4.4. Remark that if l is a Heisenberg algebra, then the automorphisms group of l contains a symplectic subgroup, which is simple. Then we get the following result. Corollary 5.4.11. If g is completely solvable and l is a Heisenberg algebra or more generally l is a direct sum of two ideals one of them is a Heisenberg algebra, then the local rigidity fails to hold. Proof. Indeed, let us write l = k ⊕ a, where k is a Heisenberg algebra and a is an ideal of g. Then Aut(k) is a subgroup of Aut(l), which therefore contains a symplectic group that is simple.

246 | 5 Local and strong local rigidity We now tackle the general situation of exponential Lie groups when Γ is Abelian. In this case, we can already note the following. Corollary 5.4.12. If g is exponential, l is Abelian and dim(l) ≥ 2, then the local rigidity fails to hold. Proof. If l is Abelian and dim(l) ≥ 2, then Aut(l) = GLk (ℝ), which is a reductive group. 5.4.3 Abelian discontinuous groups We now prove a main result, which deals with the local rigidity property in the case where Γ is Abelian. We have the following. Theorem 5.4.13. Let G be an exponential Lie group, H a connected Lie subgroup of G and Γ a Abelian discontinuous subgroup for G/H. Then R (Γ, G, H) admits a locally rigid homomorphism if and only if G is isomorphic to Aff(ℝ) and H is maximal and nonnormal in G. Proof. We proceed through different steps. When the rank of Γ is larger than 1, then Corollary 5.4.12 gives us the answer. We then assume in the rest of the proof that Γ is of rank one. In this case, there exists X0 ∈ g such that Γ = exp(ℤX0 ) and l = ℝX0 . The set Hom(l, g) can therefore be identified to g via the canonical map φ 󳨃→ φ(X0 ). Assume in a first time that H is contained in a connected normal subgroup of G. We will be proving some intermediary results, and the first is the following. Lemma 5.4.14. If H is a normal subgroup of G, then any homomorphism in R (Γ, G, H) is continuously deformable. Proof. If H is normal, then the parameter space reads R (Γ, G, H) = {φ ∈ g, φ ∈ ̸ h},

which is an open Zariski dense subset of g. Let φ ∈ R (Γ, G, H) be a locally rigid homomorphism, then the orbit AdG ⋅φ is open in R (Γ, G, H) and, therefore, in g. This means that the stabilizer of φ in g is trivial, which is impossible because it contains the onedimensional subgroup exp(φ(l)). Assume now that a normal connected subgroup contains H, then we have the following. Lemma 5.4.15. If H is contained in a normal connected Lie subgroup of G, then the local rigidity fails to hold. Proof. Let H1 be a connected normal subgroup containing H and let φ be a locally rigid homomorphism in R (Γ, G, H). As R (Γ, G, H1 ) ⊂ R (Γ, G, H) and R (Γ, G, H1 ) is dense in

5.4 Criteria for local rigidity | 247

g, then R (Γ, G, H1 ) turns out to be a dense subset of R (Γ, G, H). There exists therefore a sequence (φn )n in R (Γ, G, H1 ), which converges to φ. As the orbit of φ is open in R (Γ, G, H), the element φn belongs to the orbit of φ for n sufficiently large. It comes out that φn is also locally rigid. This means that the orbit AdG φn is open in R (Γ, G, H) and contained in R (Γ, G, H1 ). Then AdG φn is open in R (Γ, G, H1 ), which leads to a contradiction thanks to Lemma 5.4.14 as H1 is normal in G. As an immediate consequence, we have the following. Lemma 5.4.16. Suppose that φ ∈ R (Γ, G, H) is locally rigid, then φ(l) ⊂ [g, g]. Proof. Consider the Lie subgroup Gφ(l) and denote by gφ its Lie algebra. From Proposition 5.4.1, for all X ∈ gφ and Y in φ(l) we have Adexp(tX) (Y) = f (t)Y, for some analytic function f defined on ℝ. Then we get [X, Y] =

d Adexp(tX) (Y)|t=0 = f ′ (0)Y. dt

To get that Y ∈ [g, g], it is sufficient to see that there exists X ∈ gφ such that [X, Y] = αY for some nonzero α ∈ ℝ. Indeed, if for all X in gφ the bracket [X, Y] = 0 then ∘ the kernel of the homomorphism ρφ contains Gφ(l) , the connected component of the ∘ identity of Gφ(l) . This means that the quotient group Gφ(l) /Gφ(l) is a discrete group and ∘ ∘ × then Aut (l) is trivial. But this is impossible as Aut (l) = ℝ+ = {x ∈ ℝ : x > 0}. Assume henceforth that H is not contained in any normal connected subgroup of G. Let H1 be a maximal connected nonnormal subgroup containing H and denote by h1 its corresponding Lie subalgebra. Then by Theorem 5.3.3, we have the following: (1) H1 is of codimension 1 in G, and there exist A and X ∈ g such that g = ℝX ⊕ ℝA ⊕ g0

and

h1 = ℝA ⊕ g0 ,

where g0 is an ideal of g and [A, X] = X mod(g0 ). (2) H1 is of codimension 2 in G, and there exist A, X, Y ∈ g such that g = ℝX ⊕ ℝY ⊕ ℝA ⊕ g0

and

h1 = g0 ⊕ ℝA,

where g0 is an ideal of g and [A, X + iY] = (α + i)(X + iY) mod(g0 ) for some nonzero real number α. The following proposition stems directly from Proposition 5.3.6. Proposition 5.4.17. Let H1 be a connected maximal nonnormal Lie subgroup of G. Then we have:

248 | 5 Local and strong local rigidity (1) If the codimension of H1 is equal to 1, then R (Γ, G, H1 ) = g0 ⊕ ℝ∗ X. (2) If the codimension of H1 is equal to 2, then R (Γ, G, H1 ) = g0 ⊕ (ℝX ⊕ ℝY \ {(0, 0)}). Let V denote the subalgebra ℝX ⊕ g0 of g if H1 is of codimension 1 and ℝX ⊕ ℝY ⊕ g0 otherwise. Then it is clear that [g, g] ⊂ V and R (Γ, G, H1 ) is an open dense subset of V. Let now φ be a locally rigid homomorphism. By Lemma 5.4.16, φ ∈ [g, g]. There exists then a sequence (φn )n ⊂ R (Γ, G, H1 ), which converges to φ. In particular, for n sufficiently large, φn belongs to the orbit AdG φ. It follows that the orbit AdG φ is open in R (Γ, G, H1 ), which means that φ is locally rigid in R (Γ, G, H1 ). Using Theorem 5.3.9, we can conclude that G is isomorphic to Aff(ℝ). This achieves the proof of the theorem. As an immediate consequence, we get the following. Corollary 5.4.18. Let G be an exponential Lie group, H a connected Lie subgroup of G and Γ an Abelian discontinuous group for G/H. Then the local rigidity property fails to hold if and only if dim G ≠ 2 or otherwise H is normal in G. 5.4.4 Removing the assumption on Γ to admit a syndetic hull The result of this section is also a necessary condition for the local rigidity for a general Lie group, which is not necessarily exponential. Let G be a Lie group, H closed subgroup of G and Γ a discontinuous group for G/H. Let N(H) := {σ ∈ Aut(G), σ(H) ⊂ H} be the stabilizer of H, which is a closed subgroup of Aut(G) and denote by N ∘ (H) the connected component of the identity element of N(H). The automorphism group Aut(G) acts naturally on Hom(Γ, G) by composition on the left. For φ ∈ Hom(Γ, G), let Sφ := {σ ∈ Aut(G), σ ∘ φ = φ} denote the stabilizer of φ. Recall that the group of inner automorphisms I(G) forms a normal subgroup of Aut(G). Then the set Iφ (G) = Sφ I(G) is a subgroup of Aut(G) and we have the following. Theorem 5.4.19. Let G be a Lie group, H a connected Lie subgroup of G and Γ a discontinuous subgroup for G/H. If an infinitesimal deformation φ ∈ R (Γ, G, H) is locally rigid, then N ∘ (H) is contained in Iφ (G). Proof. We first need to prove the following lemmas.

5.4 Criteria for local rigidity | 249

Lemma 5.4.20. Let σ ∈ N(H) and φ ∈ Hom(Γ, G). Then: (𝚤) φ(Γ) acts properly on G/H if and only if σ ∘ φ(Γ) acts properly on G/H. (𝚤𝚤) φ(Γ) acts freely on G/H if and only if σ ∘ φ(Γ) acts freely on G/H. In particular, the action of the subgroup N(H) on Hom(Γ, G) leaves the parameter space R (Γ, G, H) stable. Proof. Let S be a compact set in G and γ ∈ φ(Γ), we have γSH ∩ SH ≠ 0

if and only if σ(γ)σ(S)H ∩ σ(S)H ≠ 0.

With the above in mind, we also get σ ∘ φ(Γ)σ(S) = σ(φ(Γ)S ), which gives (𝚤). For the second assumption, it is sufficient to see that for g ∈ G we have φ(Γ) ∩ gHg −1 = {e}

if and only if σ ∘ φ(Γ) ∩ σ(g)Hσ(g)−1 = {e}.

The next lemma describes the action of the connected subgroup N ∘ (H) on the deformation space T (Γ, G, H). Lemma 5.4.21. The connected subgroup N ∘ (H) acts continuously on T (Γ, G, H) through the law: σ ⋅ [φ] = [σ ∘ φ]. In particular, for any [φ] ∈ T (Γ, G, H), the map j[φ] : N ∘ (H) → F[φ] , σ 󳨃→ [σ ∘ φ] is continuous, where F[φ] is the orbit of [φ] under the action of N ∘ (H). Proof. As a direct consequence of the normality of I(G), the action is well-defined. The subgroup N ∘ (H) acts continuously on R (Γ, G, H) and the quotient map R (Γ, G, H) → T (Γ, G, H) is open and continuous. Then obviously the action of N ∘ (H) on the deformation space T (Γ, G, H) is continuous. Let [φ] be a point in T (Γ, G, H). From Lemma 5.4.21, the orbit F[φ] is connected. Assume that φ is a locally rigid homomorphism, or equivalently [φ] is an open point in T (Γ, G, H). Then the orbit F[φ] = {[φ]}. It follows that for any σ ∈ N ∘ (H) there exists g ∈ G such that σ ∘ φ = Tg ∘ φ, where Tg ∈ I(G) is the conjugation map by g. This implies that Tg −1 ∘ σ ∈ Sφ and, therefore, σ ∈ Iφ (G). This completes the proof of Theorem 5.4.19.

250 | 5 Local and strong local rigidity 5.4.5 Exponential Lie algebras of type T We now assume that G is an exponential Lie group and H a connected Lie subgroup of G. In this case, the map E : Aut(G) σ

󳨀→ 󳨃󳨀→

Aut(g) Log ∘σ ∘ exp

(5.16)

is a C ∞ isomorphism, with E(I(G)) = AdG . Let h = Log(H) be the Lie algebra associated to H and n(h) = {ψ ∈ Aut(g) and ψ(h) ⊂ h} the stabilizer of the Lie subalgebra h in Aut(g), then E(N(H)) = n(h). Similarly, for any φ ∈ Hom(Γ, G) let sφ = {ψ ∈ Aut(g), ψ(X) = X for all X ∈ Log ∘φ(Γ)}, then E(Sφ ) = sφ . Definition 5.4.22. A solvable Lie algebra is called a type T if for all X ∈ g we have sp(adX ) ∩ ℝ× = 0, where sp(adX ) denotes the set of all eigenvalues of the endomorphism adX . Here, ℝ× = {x ∈ ℝ : x ≠ 0}. Definition 5.4.23. Following Definition 5.4.6, a graded Lie algebra g = ⨁d∈ℤ gd is called positive if gd = {0} for any d < 0. A Lie subalgebra h of a graded Lie algebra g is called subgraded algebra if h = ⨁d∈ℤ (h ∩ gd ). Theorem 5.4.24. Let g be a positive graded exponential Lie algebra of type T , G the Lie group associated to g and H = exp(h) a connected subgroup of G. If the subalgebra g0 is Abelian and h is a subgraded subalgebra of g, then for any discontinuous group for G/H, the local rigidity property fails to hold. Proof. By Lie’s theorem, there exists a basis of the complexified Lie algebra gℂ = ℂ⊗ℝ g and some linear forms α1 , . . . , αn on gℂ such that the matrix of adX written in this basis is upper triangular with α1 (X), . . . , αn (X) on the diagonal. For any X ∈ g, let us write αj (X) = αj0 (X) + iαj1 (X),

1 ≤ j ≤ n.

Then clearly αj0 (X) and αj1 (X) are linear forms on g, and we have the following facts. Lemma 5.4.25. Assume that g is exponential of type T . Then for all 1 ≤ j ≤ n, there exists a real number aj ≠ 0 such that αj = aj αj1 + iαj1 . Proof. As g is exponential, any root of g is of the form (1 + iα)λ, where α denotes a real number and λ a linear form on g. Now the fact that g is of type T allows us to conclude.

5.4 Criteria for local rigidity | 251

Indeed, let αj = (1 + iα)λj for some αj ∈ g∗ If λj is trivial, we are done. Otherwise, α ≠ 0 and αj = α1 αj1 + iαj1 . For X ∈ g, let sp(adX ) = {α1 (X), . . . , αn (X)}. Then sp(Adexp(X) ) = {eα1 (X) , . . . , eαn (X) }. We denote by sp(AdG ) the subset of ℂ, obtained as the union of all the spectrum sp(Adg ), g ∈ G. Lemma 5.4.26. If g is an exponential Lie algebra of type T , then the set sp(AdG ) ∩ ℝ is a countable set. Proof. Suppose that eαj (X) ∈ ℝ. Then αj1 (X) ∈ 2πℤ and eαj (X) ∈ e2aj πℤ . In particular, n

sp(AdG ) ⊂ ⋃ e2aj πℤ . j=1

As g is a positive graded algebra, any element X ∈ g can be uniquely written as X = ∑ Xd , d≥0

Xd ∈ g d .

We have the following. Lemma 5.4.27. Let X, Y ∈ g and d0 = min{d ≥ 0, Yd ≠ 0}. Then we have Adexp(X) (Y) = Adexp(X0 ) (Yd0 ) mod(g≥d0 +1 ), where for k ≥ 0, g≥k = ⨁d≥k gd . Proof. The subspace g≥k is an ideal of g for all k. Then Adexp(X) (Y) = Adexp(X) (Yd0 ) mod(g≥d0 +1 ). For all n ≥ 0 and X = X0 + X ′ with X ′ ∈ g≥1 , we have (adX )n (Yd0 ) = (adX0 )n (Yd0 ) mod(g≥d0 +1 ). Then the result follows from the equality Adexp(X) = exp(adX ). We now come back to the proof of Theorem 5.4.24. By hypothesis, for any t > 0 the dilation λt defined as previously by λt (X) = t d X

for all X ∈ gd

is an automorphism of g and λt (h) = h. More precisely λt is an element of the connected component n0 (h) of the identity of n(h). Let Γ be a discontinuous group for G/H and

252 | 5 Local and strong local rigidity φ : Γ → G a locally rigid homomorphism. By Theorem 5.4.19, N ∘ (H) is included in Iφ (G). Using the isomorphism E given in equation (5.16), we can see that n∘ (h) is a subgroup of E(Iφ (G)) = AdG sφ . In particular, for all t > 0, there exist X ∈ g and σ ∈ sφ such that λt = Adexp(X) ∘σ. By Theorem 5.4.13, we can and do assume that the subgroup Γ is not Abelian, which means that φ(Γ)∩[G, G] ≠ {e}. There exists therefore Y ∈ [g, g] such that Y ∈ Log(φ(Γ)). In this case, we obtain σ(Y) = Y

and λt (Y) = Adexp(X) (Y).

Then by Lemma 5.4.27, we can write t d0 Yd0 = Adexp(X0 ) (Yd0 ) mod(g≥d0 +1 ), which gives the equality t d0 Yd0 = Adexp(X0 ) (Yd0 ) and then ℝ∗+ ⊂ sp(AdG ). By Lemma 5.4.26, g is not of type T . This completes the proof of the theorem. Remark 5.4.28. (1) The first example in Lemma 5.3.8 reveals that the rigidity holds everywhere despite the fact that Γ is Abelian. This shows that the result of Proposition 6.3.1 also fails in the context of exponential Lie groups. (2) For semisimple case G/H, if we drop the assumption that Γ\G/H is compact in Theorem 5.3.1, then the feature changes. Namely, there are more examples where Γ is not locally rigid. For instance, we see that this is the case if rankℝ H > 1 by the criterion of [94].

5.5 Local rigidity in the solvable case The present section deals with the general context when G is a connected solvable Lie group and H a maximal nonnormal subgroup of G. We discuss an analogue of the Selberg–Weil–Kobayashi local rigidity theorem (cf. Theorem 5.3.9) in this setting. In contrast to the semisimple case, the G-action on G/H is not always effective, and thus the space of group theoretic deformations T (Γ, G, G/H) could be larger than geometric deformation spaces. We determine T (Γ, G, G/H) and also its quotient modulo uneffective parts when rank Γ = 1. Unlike the context of exponential solvable case, we prove the existence of formal colored discontinuous groups. That is, the parameter space admits a mixture of locally rigid and formally nonrigid deformations.

5.5 Local rigidity in the solvable case

| 253

Therefore, our deformations of discontinuous groups may not give rise to deformations of actions of discontinuous groups. It should be noted that many of deformations studied here are such that nongeometric deformations of discontinuous groups, and thus we say them to be formal deformation. As in Chapter 2, we saw that any discontinuous group for the homogeneous space G/H appears to be Abelian and of rank ≤ 2 (cf. Theorem 2.4.6), but fails in general to admit a syndic hull. Consequently, the problems faced toward the study of the rigidity question are considerable. Unlike the setting of exponential Lie groups, a new phenomenon shows up. That is, the parameter space R (Γ, G, H) may admit a mixture of a locally rigid and formally nonrigid homomorphisms (cf. Theorem 5.5.2), in which case the discrete subgroup are said to be a formally colored discontinuous group for G/H. We are then submitted to study three dichotomous situations. The first one consists in proving that the local rigidity holds only when G is isomorphic to the group Aff(ℝ). The second says that no local rigidity phenomenon can occur. Our technique to tackle these cases consists in reducing the study of the problem to some lower-dimensional exponential Lie groups via an open descending map, where explicit computations are shown to be efficiently carried out. In the third situation, we explicitly build up an infinite family of solvable connected Lie groups Gn (where n is a positive integer) with a maximal nonnormal subgroup H; each of them admits a formally colored discontinuous group for Gn /H. In the case where Γ is of rank two, we give an explicit description of the parameter and provide some examples. We deeply believe that the rigidity property fails in this case, as the studied examples reveal. In addition, we conclusively provide an explicit description of the deformation space in the case where G acts effectively on G/H (cf. Proposition 5.5.13).

5.5.1 The notion of colored discrete subgroups Let G be a locally compact group and Γ a discrete subgroup of G. Recall first the set ⋔gp (Γ : G) as in Definition 2.1.1, consisting of all closed connected subgroups H for which SHS−1 ∩ Γ is compact for any compact set S in G. Definition 5.5.1. Let Γ be a discrete subgroup of G. (1) The subgroup Γ is said to be a colored subgroup of G if there exists H ∈⋔gp (Γ, G) such that R (Γ, G, H) contains at least a mixture of a locally rigid and a nonlocally rigid homomorphisms. (2) For a given connected subgroup H of G, a colored discontinuous group for G/H is abbreviated to be a H-colored discrete subgroup of G. (3) We shall adapt the terminology of formally colored (and formally H-colored) discrete subgroups.

254 | 5 Local and strong local rigidity 5.5.2 The rank-one solvable case The parameter space As a straightforward consequence of Theorem 2.4.12 and Proposition 2.4.13, we get the following simpler description of the parameter space: R (Γ, G, H) = {φ ∈ Hom(Γ, G) | φ(Γ) is nontrivial and acts freely on G/H}.

This therefore says that the parameter space is explicitly given as follows: In case 1, 0 { { { { g0

} } } }

(5.17)

R (Γ, G, H) = {( x ) ; x ∈ ℝ and g0 ∈ G0 } . ×

In case 2 and α ≠ 0, 0 { { { { g0

2

} } } }

(5.18)

R (Γ, G, H) = {( v ) ∈ G; v ∈ ℝ \ {(0, 0)} and g0 ∈ G0 } .

In case 2 and if α = 0, 2kπ } { } { 2 R (Γ, G, H) = {( v ) ∈ G; k ∈ ℤ, v ∈ ℝ \ {(0, 0)} and g0 ∈ G0 } . } { } { g0

(5.19)

In case 3, 󵄨󵄨 0 { } 󵄨󵄨󵄨 { } { } 󵄨󵄨 k ∈ ℤ, { 2kπ } 󵄨󵄨 R (Γ, G, H) = { ( ) ∈ G 󵄨󵄨 . 2 󵄨󵄨 v ∈ ℝ \ {(0, 0)} and g ∈ G } { } v { 󵄨󵄨 0 0} { } 󵄨󵄨 g0 󵄨 { }

(5.20)

We now treat the question of local rigidity of discontinuous action on a maximal homogeneous space for the rank-one case. We will proceed through case by case attack according to Theorem 1.1.15. We now prove the following. Theorem 5.5.2. Assume the context of Theorem 1.1.15, then: (1) In case 1, the local rigidity holds if and only if G is isomorphic to Aff(ℝ). (2) In case 2, the local rigidity fails to hold. In particular, G admits no formally colored discrete subgroups in these two cases.

5.5 Local rigidity in the solvable case

| 255

(3) In case 3 and if dim G = 4, then the local rigidity holds. Otherwise, for any integer n > 4, there exists a solvable nonexponential connected and simply connected Lie group Gn of dimension n, which admits at least a formally H-colored discrete subgroup, where H stands for a maximal nonnormal subgroup of Gn . Proof. (1) If g0 is trivial, then G is the group Aff(ℝ) for which the Lie algebra is given by g = ℝ-span{X, Y} such that [X, Y] = Y with h is isomorphic to ℝX. Let Γ for any discontinuous group for exp(g)/ exp(h), and the local rigidity property holds. Indeed, if Γ is nontrivial, it is isomorphic to exp(ℤY). The corresponding parameter space is then homeomorphic to ℝ× Y. For φ = aY ∈ R (Γ, G, H) with a ∈ ℝ× , we have G ⋅ φ = {aeb Y, b ∈ ℝ}. This means that R (Γ, G, H) only admits two open orbits. In other words, the parameter space reads 0 y

R (Γ, G, H) = {( ) , y ∈ ℝ } ×

as in Theorem 5.5.2 and the deformation space T (Γ, G, H) turns out to be a union of two open orbits. Now, if g0 is not trivial then dim(g) ≥ 3, then there is a ideal g1 , which is included in h such that codim g1 = 3 or 4 in g. Let G = G/G1 , H = H/G1 and Γ = ΓG1 /G1 , then the canonical surjection π : R (Γ, G, H) 󳨀→ R (Γ, G, H) is an open map. If G ⋅ φ is an open orbit in R (Γ, G, H), then π(G ⋅ φ) = G ⋅ φ is an open orbit in R (Γ, G, H) and the result follows from the following lemma. Lemma 5.5.3. If dim(G) = 3 or 4, then the parameter space admits no open orbits. Proof. If G is exponential, then the result follows from Theorem 5.3.9. Assume therefore that G is solvable and nonexponential. By Theorem 1.1.15, g is isomorphic to one of the following algebras: (𝚤) g = ℝ-span{A, X, Y} where [A, X] = αX − Y and [A, Y] = X − αY. (𝚤𝚤) g = ℝ-span{A, B, X, Y} such that [A, X + iY] = X + iY and [B, X + iY] = i(X + iY). (𝚤𝚤𝚤) g = ℝ-span{A, B, X, Y} such that [B, X] = Y, [A, B] = αB − X, [A, X] = B + αX and [A, Y] = 2αY. (𝚤v) g = ℝ-span{A, X, Y, Z} where [A, X] = αX − Y and [A, Y] = X − αY. (v) g = ℝ-span{A, B, X, Y} such that [A, B] = αB, [A, X] = βX −Y and [A, Y] = X +βY. The four first algebras do not admit any maximal subalgebras of codimension one for which, the associated Lie subgroup is not normal. We only have to treat the fifth case. In this case, the unique ideal of g of codimension 2 is g0 = ℝX ⊕ ℝY. It follows that h is generated by the basis {X, Y, A + λB} for some λ ∈ ℝ, and finally h is isomorphic to

256 | 5 Local and strong local rigidity ℝ-span{X, Y, A′ } such that for some B′ ∈ g \ h, we have [A′ , B′ ] = αB′ , [A′ , X] = βX − Y and [A′ , Y] = X + βY. From Theorem 5.5.2, the parameter space reads { {

0

}

2}

2

R (Γ, G, H) = {(b′ ) , b ∈ ℝ and v ∈ ℝ } ≃ ℝ × ℝ . { } ′

×

′

′ { v

×

}

Let φ = t (0, b′ , v′ ) ∈ R (Γ, G, H) and g = (a, b, v) ∈ G, gφg −1 = t (0, b′ eαa , eβa r(a)v′ ). Clearly, dim G ⋅ φ = 1 < dim R (Γ, G, H). Then the property of rigidity fails to hold. This ends the proof of the lemma. (2) If α ≠ 0, from Theorem 5.5.2, 2

R (Γ, G, H) ≃ ℝ \ {(0, 0)} × G0 .

Let φ = t (0, v, g0 ) ∈ R (Γ, G, H) as in equation (5.18) and g = (a0 , v0 , g0′ ) ∈ G, then gφg −1 = t (0, eαa0 r(a0 )v) mod(G0 ).

(5.21)

Then the orbit of φ is not open in R (Γ, G, H). Now, if α = 0, 2

R (Γ, G, H) ≃ 2πℤ × ℝ \ {(0, 0)} × G0 .

Let φ = t (2kπ, v, g0 ) ∈ R (Γ, G, H) as in equation (5.19) and g = (a0 , v0 , g0′ ) ∈ G, then gφg −1 = t (2kπ, r(a0 )v) mod(G0 ),

(5.22)

which gives that G⋅φ is not open in R (Γ, G, H). This achieves the proof of Theorem 5.5.2. (3) We now look at case 3. If dim G = 4, then g0 is trivial, and from equation (5.20) the parameter space reads 󵄨󵄨 0 󵄨󵄨 k ∈ ℤ, { } { } 󵄨 R (Γ, G, H) = { (2kπ ) ∈ G 󵄨󵄨󵄨󵄨 . { } 󵄨󵄨 v ∈ ℝ2 \ {(0, 0)}} 󵄨 v 󵄨 { } Let φ = t (0, 2kπ, v) ∈ R (Γ, G, H) and g = (a′ , b′ , v′ ) ∈ G. Then gφg −1 = t (0, 2kπ, ea r(b′ )v), ′

which means that G ⋅ φ = t (0, 2kπ, ℝ2 \ {(0, 0)}), and consequently G ⋅ φ is an open orbit of R (Γ, G, H).

5.5 Local rigidity in the solvable case

| 257

We consider next the following infinite family of solvable Lie algebras gn := Lie(Gn ), where gn = ℝ-span{A, B, X, Y, C1 , . . . , Cn } such that [A, X + iY] = X + iY,

[B, X + iY] = i(X + iY)

and

[A, Cj ] = [B, Cj ] = Cj

for all 1 ≤ j ≤ n and h = ℝ-span{A, B, C1 , . . . , Cn }. Let g = (a, b, v, c1 , . . . , cn ) and g ′ = (a′ , b′ , v′ , c1′ , . . . , cn′ ) ∈ G, then the multiplication group is given by ′

′

′

′

′

gg ′ = (a + a′ , b + b′ , e−a r(−b′ )v + v′ , c1 e−a −b + c1′ , . . . , cn e−a −b + cn′ ). As before, a routine computation shows that 󵄨󵄨 0 󵄨󵄨 { } { } 󵄨󵄨 { } 󵄨󵄨 { } 2kπ { } 󵄨 { } 󵄨 k ∈ ℤ, { } 󵄨 ′ { } 󵄨 { ( ) } v 󵄨 {( } 󵄨󵄨 ′ ) 2 n 2 󵄨 ′ R (Γ, Gn , H) = { ( c ) 󵄨󵄨 v ∈ ℝ \ {(0, 0)}, } ≃ ℤ × ℝ \ {(0, 0)} × ℝ . ) { ( } 1 󵄨 { } 󵄨󵄨 ′ { { } .. 󵄨󵄨 cj ∈ ℝ for 1 ≤ j ≤ n} { } { } 󵄨 { } 󵄨 . 󵄨 { } 󵄨 { } 󵄨 ′ 󵄨 { ( cn ) 󵄨󵄨 } Let φ = t (0, 2kπ, v′ , c1′ , . . . , cn′ ) ∈ R (Γ, Gn , H) and g = (a, b, v, c1 , . . . , cn ) ∈ Gn . Then gφg −1 = t (0, 2kπ, ea r(b)v′ , ea+b (c1 (e−2kπ − 1) + c1′ ), . . . , ea+b (cn (e−2kπ − 1) + cn′ )). If k = 0, then gφg −1 = t (0, 0, ea r(b)v′ , ea+b c1′ , . . . , ea+b cn′ ). We have exp(ℝX) ⋅ exp(ℝY) ⋅ exp(ℝC1 ) ⋅ ⋅ ⋅ exp(ℝCn ) = Stabφ := {t ∈ Gn : t ⋅ φ = φ}, then dim Gn ⋅ φ = 2 < dim R (Γ, Gn , H), and finally the local rigidity fails to hold. Now, if k ≠ 0, Gn ⋅ φ = t (0, 2kπ, ℝ2 \ {(0, 0)}ℝn ), which is an open orbit sitting inside R (Γ, Gn , H). This achieves the proof of Theorem 5.5.2. Remark 5.5.4. Up to this point, the Selberg–Weil–Kobayashi local rigidity theorem for the rank-one case of solvable Lie groups can be schematized, with respect to the simple noncompact linear case, as in Table 5.2

258 | 5 Local and strong local rigidity Table 5.2: Summarizing local rigidity results. (A): G is a noncompact simple linear Lie group, H a maximal compact subgroup and Γ is a uniform lattice

R(Γ, G, H) admits a continuous deformation ⇐⇒ G is locally isomorphic to SL2 (ℝ)

(B): G′ = G × G, H′ = ΔG′ and Γ is a uniform lattice of G (G as in (A))

R(Γ, G′ , H′ ) admits a continuous deformation ⇐⇒ G is locally isomorphic to SO(n, 1) or SU(n, 1)

(C): G is exponential solvable, H a maximal nonnormal subgroup of G and Γ a discontinuous group for G/H

R(Γ, G, H) admits a locally rigid deformation ⇐⇒ G is locally isomorphic to Aff(ℝ)

(D): G is solvable connected and simply connected Lie group, H a maximal nonnormal subgroup of G and Γ a discontinuous group for G/H of rank one

Case 1 Local rigidity ⇐⇒ G ≃ Aff(ℝ)

Case 2 The local rigidity fails

Case 3 if dim G ≤ 4, then the local rigidity holds. For any n > 4, there exists a Lie group Gn as in (D), which admits at least a formally H-colored discontinuous group

5.5.3 The setting where the action of G on G/H is effective Let G0 denote the kernel of the action of G on G/H. It is the maximal normal subgroup of G included in H, G0 = ⋂g∈G gHg −1 . Let us opt for the notation R (Γ, G, X) and T (Γ, G, X) (instead of R (Γ, G, H) and T (Γ, G, H)), in the setting where the action of G on X is effective. In such a case, G0 is trivial and then obviously dim G ≤ 4. As an immediate consequence from Theorem 5.5.2, the following is then immediate. Corollary 5.5.5. Let us keep the same hypotheses and notation. For a discontinuous group Γ ≃ ℤ for X = G/H, the following holds: (1) If (G, H) is as in case 1, T (ℤ, G, X) consists of two isolated points (i. e., the topology is discrete). (2) If (G, H) is as in case 2, T (ℤ, G, X) has no isolated points. (3) If (G, H) is as in case 3, T (ℤ, G, X) ≃ ℤ consists only of isolated points (i. e., the topology is discrete). To seek the kind of perturbations of Γ in G with no contribution to the action on X, one can consider the following equivalence relation on Hom(Γ, G): φ1 ∼G0 φ2 ,

φ1 , φ2 ∈ Hom(Γ, G)

if there exists g ∈ G such that π ∘ φ1 = (π ∘ φ2 )g in Hom(Γ, G).

(5.23)

5.5 Local rigidity in the solvable case

| 259

We then get the following. Corollary 5.5.6. Let us keep the same hypotheses and notation. For a discontinuous group Γ ≃ ℤ for G/H, the following holds: (1) If (G, H) is as in case 1, R (ℤ, G, H)/ ∼G0 consists of isolated points (i. e., the topology is discrete). (2) If (G, H) is as in case 2, R (ℤ, G, H)/ ∼G0 has no isolated points. (3) If (G, H) is as in case 3, R (ℤ, G, H)/ ∼G0 consists only of isolated points (i. e., the topology is discrete). Proof. Let Γ ≅ ℤ be a discontinuous group for G/H. In this situation, the spaces G/H and G/H are diffeomorphic, where as before G = G/G0 and H = H/G0 . As in Lemma 2.4.7, the action of Γ on G/H is discontinuous and obviously the projection π : R (ℤ, G, H) 󳨀→ R (ℤ, G, H) is open. As Hom(ℤ, G) is identified to G as a set, the group G0 acts by left multiplication on R (ℤ, G, H) by (g0 ⋅ φ)(γ) = g0 φ(γ),

∀ g0 ∈ G, γ ∈ Γ.

Then the projection map π factors through the quotient map p : R (ℤ, G, H) 󳨀→ R (ℤ, G, H)/G0 , which also an open map. Then the induced map i : R (ℤ, G, H)/G0 󳨀→ R (ℤ, G, H) defined by π = i ∘ p is a homeomorphism. The quotient group G acts also on

R (ℤ, G, H)/G0 by g ⋅ p(φ) = p(g ⋅ φ) and the double cosets space G0 \R (ℤ, G, H)/G is identified to the set of the equivalence classes R (ℤ, G, H)/ ∼G0 . Now

̇ g ⋅ i(p(φ)) = g ⋅ π(φ) = π(g ⋅ φ) = i ∘ p(g ⋅ φ) = i(g p(φ)). Thus, i induces a homeomorphism i : R (ℤ, G, H)/ ∼G0 󳨀→ T (ℤ, G, H). But the spaces T (ℤ, G, H) and R (ℤ, G, X) are homeomorphic, then Corollary 5.5.5 allows to conclude.

260 | 5 Local and strong local rigidity 5.5.4 The rank-two case This section aims to put the emphasis on the case where Γ is of rank 2. Unlike the context of exponential solvable Lie groups, the authors do not know so far how to deal with this case. However, we can get an accurate description of the parameter space R (ℤ2 , G, H). We first prove the following. Proposition 5.5.7. Let G be a connected simply connected solvable Lie group, H a nonnormal maximal subgroup of G and Γ a subgroup of G of rank 2. Then: (1) In case 2, Γ is a discrete subgroup acting properly on G/H if and only if Γ is generated by two elements γi = (ai , vi , gi ), i = 1, 2 satisfying: (𝚤) ai ∈ 2πℤ and det(v1 , v2 ) ≠ 0 if α = 0. (𝚤𝚤) ai = 0 and det(v1 , v2 ) ≠ 0 if α ≠ 0. (2) In case 3, Γ is a discrete subgroup acting properly on G/H if and only if Γ is generated by γi = (ai , bi, vi , gi ), i = 1, 2 such that ai = 0, bi ∈ 2πℤ and det(v1 , v2 ) ≠ 0. As proper action is also free in our context, by Lemmas 2.4.10 and 2.4.11 the subgroup Γ acts properly on G/H only if the two elements γ1 , γ2 satisfy (0, vi , gi ) { { { γi = {(2ki π, vi , gi ) { { {(0, 2ki π, vi , gi )

if α ≠ 0 in case 2,

(5.24)

if α = 0 in case 2, in case 3.

Then the proposition is a direct consequence of the following result. Lemma 5.5.8. Let Γ be a subgroup of G generated by two elements γ1 , γ2 satisfying (5.24). Then the action of Γ on G/H is proper if and only if det(v1 , v2 ) ≠ 0. Proof. As before, the nonproperness of the action of Γ on G/H is equivalent to the existence of a convergent sequence (sn )n in G and a sequence (γn )n ∈ Γ with {γn , n ∈ ℕ} λ μ is noncompact such that (γn sn )n converges modulo H. For γn = γ1 n γ2 n and (tn , wn , gn )

sn = {

(tn , fn , wn , gn )

in case 2, in case 3,

we obtain (0, λn v1 + μn v2 + eαtn r(tn )wn , 0) mod(H)

γn sn = {

tn

in case 2,

(0, 0, λn v1 + μn v2 + e r(fn )wn , 0) mod(H) in case 3.

Now (tn )n , (fn )n and (wn )n are convergent, then the nonproperness of the action is equivalent to the existence of a sequence (λn , μn ) ⊂ ℤ2 such that (λn v1 + μn v2 )n converges, which is also equivalent to the condition det(v1 , v2 ) = 0.

5.5 Local rigidity in the solvable case

| 261

Remark 5.5.9. In the rank-two case, there no equivalence between free action and proper action. In fact, in Proposition 5.5.7, if we replace the word properly by freely, and the statement det(v1 , v2 ) ≠ 0 by the fact that v1 , v2 are ℚ-linearly independent, we obtain a characterization of the free action. Now as a direct consequence of Proposition 5.5.7 and Proposition 2.4.13, we get the following. Theorem 5.5.10 (The rank-two case). Let G be a connected simply connected solvable Lie group, H a connected nonnormal maximal subgroup of G and Γ ≃ ℤ2 a discontinuous group for G/H of rank two. Then the parameter is given in the following cases by: (𝚤) In case 2, we have 0 { { R (ℤ , G, H) = { (v1 { { g1 2

󵄨󵄨 0 󵄨󵄨 det(v , v ) ≠ 0,} } 󵄨 1 2 , v2 ) ∈ Hom(Γ, G) 󵄨󵄨󵄨󵄨 } } 󵄨󵄨 g1 , g2 ∈ G0 󵄨 g2 󵄨 }

if α ≠ 0

and 2k1 π { { R (ℤ , G, H) = { ( v1 { g1 { 2

󵄨󵄨 2k2 π 󵄨󵄨 det(v , v ) ≠ 0,} } 󵄨 1 2 , v2 ) ∈ Hom(Γ, G) 󵄨󵄨󵄨󵄨 } } 󵄨󵄨 g1 , g2 ∈ G0 g2 󵄨󵄨 }

if α = 0.

(𝚤𝚤) In case 3, we have 0 { { { { 2k1 π 2 R (ℤ , G, H) = { ( { v1 { { g 1 {

󵄨󵄨 0 󵄨󵄨 k1 , k2 ∈ ℤ, } } 󵄨󵄨 } } 󵄨󵄨 2k2 π ) ∈ Hom(Γ, G) 󵄨󵄨󵄨 det(v1 , v2 ) ≠ 0,} . 󵄨󵄨 } v2 } 󵄨󵄨 } 󵄨󵄨 g1 , g2 ∈ G0 g2 󵄨 }

Finally, the following examples show that the local rigidity property fails to hold in the rank-two case. Example 5.5.11. Let g = ℝ-span{A, X, Y} be such that [A, X + iY] = (α + i)(X + iY) for some α ∈ ℝ. Let g = (a, v) and g ′ = (a′ , v′ ) ∈ G, then by equation (2.19), we have ′

gg ′ = (a + a′ , e−αa r(−a′ )v + v′ ). Let h = ℝA ⊕ ℝB and Γ = ⟨exp X, exp Y⟩ the discrete subgroup generated by exp X and exp Y. If α ≠ 0, then G is exponential solvable and Theorem 5.5.2 shows that the local rigidity fails to hold.

262 | 5 Local and strong local rigidity Now, if α = 0, which means that G is no longer exponential solvable. Then from Theorem 5.5.10, 2kπ v

2k ′ π ) ∈ Hom(Γ, G) v′

R (Γ, G, H) = {(

󵄨󵄨 ′ 󵄨󵄨 k, k ∈ ℤ, 󵄨󵄨 }. 󵄨󵄨 󵄨󵄨 det(v, v′ ) ≠ 0

Let 2kπ φ=( v

2k ′ π ) ∈ R (Γ, G, H) v′

and g = (a0 , v0 ) ∈ G. Then 2kπ e r(a0 )v

gφg −1 = (

αa0

2k ′ π ), e r(a0 )v′ αa0

which means that G ⋅ φ is not an open orbit and the local rigidity fails to hold. Example 5.5.12. g = ℝ-span{A, B, X, Y} such that [A, X + iY] = X + iY and [B, X + iY] = i(X + iY). Let g = (a, b, v) and g ′ = (a′ , b′ , v′ ) ∈ G. Then by equation (2.18), we have ′

gg ′ = (a + a′ , b + b′ , e−a r(−b′ )v + v′ ). Let as earlier h = ℝA ⊕ ℝB and Γ = ⟨exp X, exp Y⟩. Then as in Theorem 5.5.10, 0 { { R (Γ, G, H) = { (2kπ { v {

󵄨󵄨 0 󵄨󵄨 k, k ′ ∈ ℤ, } } 󵄨 ′ . 2k π ) ∈ Hom(Γ, G) 󵄨󵄨󵄨󵄨 ′ } 󵄨󵄨 det(v, v ) ≠ 0} 󵄨󵄨 v′ }

Let 0 φ = (2kπ v

0 2k ′ π ) ∈ R (Γ, G, H) v′

and g = (a0 , b0 , v0 ) ∈ G. Then gφg

−1

0 = ( 2kπ ea0 r(b0 )v

0 2k ′ π ) , a0 e r(b0 )v′

which shows that G ⋅ φ is not an open orbit. So, the local rigidity fails to hold. Theorem 5.5.2 and Examples 5.5.11 and 5.5.12 above allow us to write down explicitly the deformation space T (Γ, G, X) in the setting where G acts effectively on X. Indeed, the following holds.

5.6 The case of Diamond groups |

263

Proposition 5.5.13. Assume that G acts effectively on X and Γ a discontinuous group for G/H, then: (1) If dim(G) = 2, then T (Γ, G, X) consists of two isolated points. (2) If dim(G) = 3, then we are in case 2. In this case, we have: (a) If Γ is of rank one, then T (Γ, G, X) is homeomorphic to: (𝚤) ℝ×+ × ℤ, if α = 0. (𝚤𝚤) S1 , if α ≠ 0. (b) If the rank of Γ is two, then T (Γ, G, X) is homeomorphic to: (𝚤) ℤ2 × GL2 (ℝ)/SO2 , if α = 0. (𝚤𝚤) GL2 (ℝ)/ℝ, if α ≠ 0, where ℝ is embedded as a subgroup via t 󳨃→ eαt r(t). (3) If dim(G) = 4, then for the rank-one case T (Γ, G, X) is homeomorphic to ℤ for the rank-one case and to ℤ2 × GL2 (ℝ)/P for the rank-two case, where P is the twodimensional connected Lie subgroup of GL2 (ℝ) image of ℝ2 by the homomorphism (a, b) 󳨃→ ea r(b). Proof. (1) comes directly from Theorem 5.5.2. For (2), if Γ is of rank one, then equations (5.18) and (5.19) give us the result provided the expressions of the corresponding actions as in equations (5.21) and (5.22). If Γ is of rank two or dim(G) = 4, then the result comes from Examples 5.5.11 and 5.5.12 and Corollary 5.5.5.

5.6 The case of Diamond groups The diamond algebra g is defined as the direct sum of the Heisenberg Lie algebra h2n+1 as defined in Subsection 1.1.2 and an n-dimensional Abelian Lie algebra a = ⨁nl=1 ℝAi with the additional nontrivial brackets [Al , Xl ] = Yl

and [Al , Yl ] = −Xl ,

for l = 1, . . . , n.

(5.25)

A Lie algebra l is called a graded Lie algebra if there is a decomposition l = ⨁ ld , d∈ℤ

where ld , d ∈ ℤ are a subspaces of l such that [li , lj ] ⊂ li+j ,

i, j ∈ ℤ.

From the brackets relations (5.25) and (1.5), the diamond algebra g is a graded Lie algebra with the decomposition g = g0 ⊕ g1 ⊕ g2 ,

264 | 5 Local and strong local rigidity where n

g0 = a,

g1 = ⨁(ℝXi ⊕ ℝYi ), i=1

and g2 = ℝZ denotes the center of g. As a consequence, for all t ∈ ℝ∗+ := {t ∈ ℝ, t > 0}, the dilation μt : g → g defined for v ∈ gd by μt (v) = t d v is an automorphism of g. Definition 5.6.1. A subalgebra h of a graded Lie algebra l is called subgraded subalgebra if h = ⨁(h ∩ ld ). d∈ℤ

The subgraded subalgebras of g are the subalgebras of g, which are stable by the one parameter family of dilations μt , t ∈ ℝ∗+ . Compatible subalgebras A subalgebra h of g is said to be compatible with a given Levi decomposition if it is a direct sum u ⊕ h0 , where u = a ∩ h and h0 = h2n+1 ∩ h. Any subgraded subalgebra is compatible with a given Levi decomposition, and conversely a subalgebra, which is compatible with a given Levi decomposition and contains the center of g is subgraded subalgebra. In general, a subalgebra, which is compatible with a given Levi decomposition, is not subgraded. The subalgebra generated by X1 + Z is compatible with a given Levi decomposition but not subgraded subalgebra of g. Nevertheless, we have the following statement. Proposition 5.6.2. Let h be a subalgebra of g, which is compatible with a given Levidecomposition. Then there exists g ∈ G such that Adg (h) is a subgraded subalgebra. Assume for the rest of this paragraph that h = u ⊕ h0 , with u and h0 as before. To prove Proposition 5.6.2, we need some preliminary remarks. Let A be an element of u. We define the support of A as the unique subset IA ⊆ {1, . . . , n} such that A = ∑ αi Ai i∈IA

with αi ≠ 0, ∀ i ∈ IA .

(5.26)

We define also the support of h as the set I := ⋃A∈u IA . Obviously, there exists A ∈ u such that I = IA ; such a vector is called generic element of u. Denote by UI the linear subspace of g generated by Xi , Yi , i ∈ I and let hI = ℝZ ⊕ UI . Note that h2n+1 = UI ⊕ hI C and [hI , hI C ] = [u, hI C ] = {0}.

5.6 The case of Diamond groups |

265

Lemma 5.6.3. Let I be the support of h. Then h = u ⊕ (h0 ∩ UI ) ⊕ (h0 ∩ hI C ), as a direct sum of linear subspaces. Proof. Let v ∈ h0 such that v = vI + vI C , where vI ∈ UI and vI C ∈ hI C . To conclude, we have to prove that vI ∈ h. Let A be a generic element of u. Then we have ad(A)m v = ad(A)m vI and ad(A)m vI ∈ h for all m > 0, where ad(A)m vI = [A, ad(A)m−1 vI ]. Let A be written as in (5.26) and vI = ∑ vi ,

with vi ∈ U{i} .

i∈I

Consider the set of strictly positive integers {β1 , < ⋅ ⋅ ⋅ , < βq } := {|αi |, i ∈ I}. Then from the expressions q

αj

A = ∑ βi ∑ i=1

|αj |=βi

|αj |

q

Aj

and vI = ∑ ∑ vj , i=1 |αj |=βi

we deduce that q

ad(A)m vI = ∑ βim ∑ ( i=1

|αj |=βi

m

αj |αj |

) ad(Aj )m vj .

Now observe that ad(Aj )4 vj = vj . Then q

ad(A)4m vI = ∑ βi4m ∑ vj . i=1

|αj |=βi

For ui = ∑|αj |=βi vj , the sequence wm = ∑qi=1 βi4m ui = ad(A)4m vI is contained in the subspace L generated by the free family {u1 , . . . , uq }. The matrix associated to the coordinates of w1 , . . . , wq via the basis {u1 , . . . , uq } is the Vandermonde matrix β14 . ( .. βq4

⋅⋅⋅ ⋅⋅⋅

β14q .. . ), 4q βq

which is nonsingular, because βi4 ≠ βj4 for all i ≠ j. This means that the vectors w1 , . . . , wq form a basis of L. As wm ∈ h for all m > 0, the subspace L ⊂ h, in particular, the vector vI = ∑qi=1 ui ∈ h. Recall now the following result, object of Proposition 1.1.12, which asserts the following. Let h be a Lie subalgebra of h2n+1 , such that z(h2n+1 ) ⊄ h. Then dim h ≤ n and

266 | 5 Local and strong local rigidity there exists a basis Bh = {Z, X1′ . . . , Xn′ , Y1′ , . . . , Yn′ } of h2n+1 with the Lie commutation relations [Xi′ , Yj′ ] = δi,j Z,

i, j = 1, . . . , n,

(5.27)

such that h is generated by X1′ , . . . , Xp′ , where p = dim h. The symbol δi,j designates here the Kronecker index. Proof of Proposition 5.6.2. If h contains the center, then obviously h is subgraded. Assume that h does not contain the center. Using Lemma 5.6.3, where we replace h2n+1 ′ by hI C and h by h ∩ hI C , we deduce the existence of a basis {Z, X1′ . . . , Xm , Y1′ , . . . , Ym′ } ′ ′ of hI C satisfying the relation (5.27) and such that X1 . . . , Xp is a basis of h ∩ hI C , where p = dim(h ∩ hI C ) and 2m + 1 = dim hI C . For Xi′ = Xi′′ + αi Z, with Xi′′ ∈ g1 , let X = ∑pi=1 αi Yi′ . Then a direct calculation shows that Adexp(X) (h ∩ hI C ) is the linear span of X1′′ , . . . , Xp′′ , in particular, Adexp(X) (h ∩ hI C ) ⊂ g1 . Observe that for all X ∈ hI C , we have [X, u] = 0 and [X, UI ] = 0. Thus, Adexp(X) (u) = u and Adexp(X) (UI ∩ h) = UI ∩ h. Using Lemma 5.6.3, we conclude that Adexp(X) (h) = u ⊕ (UI ∩ h) ⊕ Adexp(X) (h ∩ hI C ), which is graded since UI ∩ h ⊕ Adexp(X) (h ∩ hI C ) ⊆ g1 . 5.6.1 Dilation invariant subgroups The Heisenberg group H2n+1 is the connected simply connected Lie group of dimension 2n + 1 associated to h2n+1 . As h2n+1 is nilpotent, H2n+1 is an exponential Lie group. We identify H2n+1 with the affine space ℝ2n+1 = (ℝ2 )n × ℝ. Any element x of H2n+1 can be written as x = (x1 , . . . , xn , z),

α where xi = ( i ) ∈ ℝ2 . βi

Thanks to the Campbell–Backer–Hausdorff formula for 2-step nilpotent Lie groups, 1 C(X, Y) = X + Y + [X, Y], 2 the multiplication is obtained by the formula xx′ = (x1 + x1′ , . . . , xn + xn′ , z + z ′ +

1 n ∑ b(xl , xl′ )), 2 l=1

where b is the standard nondegenerate skew-symmetric bilinear form on ℝ2 , given by b(xl , xl′ ) = det(xl , xl′ ).

5.6 The case of Diamond groups |

267

As [g, g] = h2n+1 , the family {A1 , . . . , An } is a coexponential basis to h2n+1 in g. The diamond group is identified to the product space ℝn × H2n+1 and the map a × h2n+1 → G,

(A, X) 󳨃→ exp(A) exp(X)

is a diffeomorphism. The Abelian group exp(a) = ℝn acts on h2n+1 via the adjoint representation αj Xj + βj Yj Adexp(ti Ai ) (αj Xj + βj Yj ) = { ′ αj Xj + βj′ Yj α′

if i ≠ j,

if i = j,

αj

where ( β′j ) = r(tj )( βj ) and r(ti ) is the rotation transformation j

cos(ti ) sin(ti )

r(ti ) = (

− sin(ti ) ), cos(ti )

through the basis {Xi , Yi }. Thus, the diamond group is the semidirect product G = ℝn ⋉ H2n+1 , where ℝn acts on H2n+1 as follows: (t1 , . . . , tn ) ⋅ x = (r(t1 )x1 , . . . , r(tn )xn , z). More precisely, let us consider the projections: ξl : G → ℝ2 ,

τl : G → ℝ and

ζ :G→ℝ

defined by ξl (g) = xl , τl (g) = tl for l = 1, . . . , n and ζ (g) = z, where g = (t1 , . . . , tn , x1 , . . . , xn , z). The product of two elements g, g ′ ∈ G is determined by the formulas ξl (gg ′ ) = rl (g ′ )ξl (g) + ξl (g ′ ),

τl (gg ′ ) = τl (g) + τl (g ′ ),

(5.28)

and ζ (gg ′ ) = ζ (g) + ζ (g ′ ) +

1 n ∑ b(r (g ′ )ξl (g), ξl (g ′ )), 2 l=1 l

(5.29)

where rl (g) = r(τl (g))−1 . As a consequence, the action of G on itself by conjugation is given by δl (gg ′ g −1 ) = δl (g ′ ),

ξl (gg ′ g −1 ) = rl (g)−1 ((rl (g ′ ) − I)ξl (g) + ξl (g ′ )),

(5.30)

268 | 5 Local and strong local rigidity

ζ (gg ′ g −1 ) = ζ (g ′ ) + −

1 n ∑ b(r (g ′ )ξl (g), ξl (g ′ )) 2 l=1 l

1 n ∑ b(r (g ′ )ξl (g), ξl (g)) + b(ξl (g ′ ), ξl (g)). 2 l=1 l

(5.31)

Compatible subgroups Let H0 be a subgroup of H2n+1 and U a subgroup of exp(a), such that U normalizes H0 . Then the semidirect product U ⋉ H0 is a well-defined subgroup of G. Definition 5.6.4 (cf. [89]). A closed subgroup H of G is said to be compatible with a given Levi decomposition if it is a semidirect product U ⋉ H0 , where U = exp(a) ∩ H and H0 = H ∩ H2n+1 . Proposition 5.6.5. Let H = U ⋉ H0 be a subgroup of G compatible with a given Levi decomposition and L(H0 ) the syndetic hull of H0 . Then U normalizes L(H0 ) and the subgroup U ⋉ L(H0 ) contains U ⋉ H0 cocompactly. In particular, if U is connected and U ⋉ L(H0 ) is a syndetic hull of H. Note that since a compatible subgroup with a given Levi decomposition is closed, H0 is closed subgroup of H2n+1 . Thus, the syndetic hull of H0 exists and it is unique. Similarly, any closed subgroup of the Abelian group exp(a) has a unique syndetic hull. More precisely, we can state the following. Lemma 5.6.6. Let K be a closed subgroup of a completely solvable Lie group. Then the syndetic hull of K is the connected Lie subgroup K ′ generated by the one parameter subgroups {exp(tX) | t ∈ ℝ}, for all exp(X) ∈ K. Proof. As G is a completely solvable, K admits a unique syndetic hull L(K); cf. Lemma 1.4.2. Clearly, K ′ is connected subgroup containing K and K ′ ⊂ L(K). The homogeneous space L(K)/K ′ is homeomorphic to ℝd , where d = dim L(K) − dim K ′ and the map L(K)/K → L(K)/K ′ ,

xK 󳨃→ xK ′

is a surjective and continuous. As L(K)/K is compact, d = 0 and L(K) = K ′ . Proof of Proposition 5.6.5. Using Lemma 5.6.6, it is sufficient to show that U ⋉ H0′ is a subgroup of G. Or equivalently, U normalizes H0′ . Let exp(tX) be a generator of H0′ and exp(A) an element of U, then exp(A) exp(tX) exp(−A) = exp(t Adexp(A) (X)) and

Adexp(A) (X) ∈ H0 .

Thus, the one parameter subgroup exp(t Adexp(A) (X)) is also a generator of H0′ . For the second statement, let S be a compact set in L(H0 ) such that L(H0 ) = H0 S. Then U ⋉ L(H0 ) = U ⋉ H0 L = (U ⋉ H0 )(0, S) and (0, S) is a compact set in U ⋉ L(H0 ).

5.6 The case of Diamond groups |

269

Lemma 5.6.7. Let H be a subgroup of G, which is compatible with a given Levi decomposition. Assume that H2n+1 is the syndetic hull of H0 . If L(U) is the syndetic hull of U, then the connected subgroup L(U) ⋉ H2n+1 is a syndetic hull of H. Proof. If L(U) = SU for a compact set S in exp(a) and H2n+1 = H0 S′ for S′ a compact set of H2n+1 . Then L(U) ⋉ H2n+1 = (S, 0)(U ⋉ H0 )(0, S′ ) = (S, 0)(0, S′ −1 )(U ⋉ H0 ) = (S, S′ −1 )H. Subgraded and dilation-invariant subgroups Recall that, if L is a connected simply connected Lie groups, then any automorphism μ of the Lie algebra l of L lifts to an automorphism μ̃ of G, by the formula ̃ μ(exp(X)) = exp(μ(X)),

for all X ∈ l.

In particular, if l is graded Lie algebra, for any t ∈ ℝ∗+ the dilation automorphism μt induces an automorphism of L. This leads us to define an action of ℝ∗+ on L. Definition 5.6.8. Assume that L is a connected simply connected Lie group with a graded Lie algebra l. A closed subgroup H of L is said to be: (1) dilation-invariant, if H is stable under the action of ℝ∗+ on L. (2) subgraded, if H is connected and its Lie subalgebra is a subgraded subalgebra of l. If H is subgraded, then obviously H is a dilation-invariant subgroup. Conversely, a dilation-invariant subgroup is not necessarily connected; and a closed subgroup with a subgraded Lie subalgebra may or not be dilation-invariant. As an example, in our situation, for g ∈ G the dilation action is given by τl (μ̃ t (g)) = τl (g),

ξl (μ̃ t (g)) = tξl (g)

and ζ (μ̃ t (g)) = t 2 ζ (g).

(5.32)

The center of G, Z(G) = {g ∈ G, ξl (g) = 0 and τl (g) ∈ 2πℤ for all l = 1, . . . , n} and the subgroup H = exp(ℤX) exp(ℝZ) for X ∈ g1 are both nonconnected and have a same Lie subalgebra the center g2 , which is subgraded. The center Z(G) is dilationinvariant but H is not. The following lemma is a characterization of dilation-invariant subgroups in G. Lemma 5.6.9. A closed subgroup H of G is a dilation-invariant subgroup if and only if H is compatible with a given Levi decomposition such that H ∩ H2n+1 is a subgraded subgroup.

270 | 5 Local and strong local rigidity Proof. Suppose that H is dilation-invariant and let U and H0 be as in Definition 5.6.4. As H2n+1 is a dilation-invariant subgroup, then so is H0 . Clearly, U normalizes H0 and U ⋉ H0 is a closed subgroup of H. Let h = exp(A) exp(X) ∈ H. As H is closed lim μ̃ t (exp(A) exp(X)) = lim exp(A) exp(μt (X)) = exp(A) ∈ U,

t→0

t→0

thus exp(X) ∈ H0 and H = U ⋉ H0 . As limt→0 exp(μt (X)) = e for all exp(X) in H0 , the dilation invariance of H0 leads to conclude that H0 is connected and the Lie subalgebra of H0 is h0 = {X, exp(X) ∈ H0 }. Let X ∈ h0 , and write X = X ′ + X ′′ with X ′ ∈ g1 and X ′′ ∈ g2 . For t ≠ 0, we have X − 1t μt (X) = tX ′′ ∈ h0 . Then X ′ , X ′′ ∈ h0 and h0 = h0 ∩ g1 ⊕ h0 ∩ g2 . Conversely, if H0 is subgraded then H0 is dilation-invariant and we have μ̃ t (U ⋉ H0 ) = U ⋉ μ̃ t (H0 ) = U ⋉ H0 . 5.6.2 Strong local rigidity results The main results in this section are as follows. Theorem 5.6.10. Let G be the diamond group and Γ a nontrivial finitely generated subgroup of G (not necessarily discrete). Then there is no open G-orbit in Hom(Γ, G). In particular, if Γ is a discontinuous group for a homogeneous space G/H, then the strong local rigidity property fails to hold. Theorem 5.6.11. Assume that H is a dilation-invariant subgroup of G and Γ is a nontrivial discontinuous group for G/H. Then for every φ ∈ R (Γ, G, H), local rigidity property fails to hold. Corollary 5.6.12. Let H = U ⋉ H0 be a subgroup of G compatible with a given Levi decomposition. Assume that one of the following statements is satisfied: (𝚤) The subgroup H0 ∩ Z(H2n+1 ) is nontrivial. (𝚤𝚤) The subgroup U is connected, which is the case when H is connected. Then for any nontrivial discontinuous group for G/H, local rigidity property fails to hold. To prove these results, we need some preliminary results. The following observation is an important tool. The action of ℝ∗+ on G defined by (5.32) induces a natural action of ℝ∗+ on Hom(Γ, G) by the formula t ⋅ φ = μ̃ t ∘ φ. Lemma 5.6.13. The map ℝ∗+ × Hom(Γ, G)/G → Hom(Γ, G)/G, (t, [φ]) 󳨃→ [t ⋅ φ] gives a well-defined continuous action of ℝ∗+ on Hom(Γ, G)/G. In particular, for all φ ∈ Hom(Γ, G), the map ℝ∗+ → Hom(Γ, G)/G, t 󳨃→ t ⋅ [φ] is continuous. Proof. The fact that the map is a well-defined action is derived from the equality t ⋅ (g ⋅ φ) = (t ⋅ g) ⋅ (t ⋅ φ).

5.6 The case of Diamond groups | 271

Since the quotient map Hom(Γ, G) → Hom(Γ, G)/G is open and the action of ℝ∗+ on Hom(Γ, G) is continuous, the action of ℝ∗+ on Hom(Γ, G)/G is also continuous. The aim now is to prove Theorem 5.6.10. Let γ1 , . . . , γk be the generators of Γ and identify Hom(Γ, G) to a subspace of Gk via the injection φ 󳨃→ (φ(γ1 ), . . . , φ(γk )). To simplify the notation, we write φi instead of φ(γi ). Let Fix(Γ, G) be the set of fixed points in Hom(Γ, G)/G under the action of ℝ∗+ . Then as a direct consequence of Lemma 5.6.13, we obtain the following. Lemma 5.6.14. Let φ ∈ Hom(Γ, G). If the G-orbit of φ is open in Hom(Γ, G), then [φ] ∈ Fix(Γ, G). Proof. The action of ℝ∗+ is continuous. If [φ] is an open point in Hom(Γ, G)/G, then the stabilizer of [φ] is an open subgroup of ℝ∗+ . We consider the subset F ′ (Γ, G) := {(φ1 , . . . , φk ) ∈ Hom(Γ, G), ξl (φi ) = 0 for all i and l},

(5.33)

and F ′′ (Γ, G) its image by the quotient map: Hom(Γ, G) → Hom(Γ, G)/G. Lemma 5.6.15. Fix(Γ, G) ⊂ F ′′ (Γ, G). Proof. Let [φ] ∈ Fix(Γ, G), then for all t ∈ ℝ∗+ there exists an element gt ∈ G, such that t ⋅ φ = gt ⋅ φ. Thus, for all l = 1, . . . , n and i = 1, . . . , k, we have tξl (φi ) = ξl (gt φi gt−1 ), which is equivalent by (5.30) to tξl (φi ) = rl (gt )−1 rl (φi )ξl (gt ) + rl (gt )−1 ξl (φi ) − rl (gt )−1 ξl (gt ).

(5.34)

If τl (φi ) ∈ 2πℤ, then tξl (φi ) = rl (gt )−1 ξl (φi ) for all t, which implies that ξl (φi ) = 0. For τl (φi ) ∈ ̸ 2πℤ, let xφ,l,i := (I − rl (φi )) ξl (φi ). −1

(5.35)

Then from (5.34), we get ξl (gt ) = (I − trl (gt ))xφ,l,i . In particular, for all i, j such that τl (φi ), τl (φi ) ∈ ̸ 2πℤ, we have the relation xφ,l,i = xφ,l,j . Let g ∈ G be an element of G satisfying xφ,l,i ξl (g) = { 0

if there exist i such that τl (φi ) ∈ ̸ 2πℤ,

otherwise.

272 | 5 Local and strong local rigidity By a direct calculation, using (5.30), we show that ξl (gφi g −1 ) = 0 for all l and i. Therefore, g ⋅ φ ∈ F ′ (Γ, G) and [g ⋅ φ] = [φ]. Lemma 5.6.16. The restriction of the quotient map π : Hom(Γ, G) → Hom(Γ, G)/G to F ′ (Γ, G) is a continuous bijection from F ′ (Γ, G) to F ′′ (Γ, G). Proof. The restriction is continuous and by definition of F ′′ (Γ, G) is surjective. To see that the restriction is injective, suppose that there exists φ ∈ F ′ (Γ, G) and g ∈ G such that g ⋅ φ ∈ F ′ (Γ, G), we have to prove that φ = g ⋅ φ. By definition of F ′ (Γ, G), ξl (φi ) = ξl (gφi g −1 ) = 0

for all l and i.

(5.36)

If there exists i such that τl (φi ) ∈ ̸ 2πℤ, the last equality implies that ξl (g) = 0. Then from equality (5.31), we conclude that ζ (gφi g −1 ) = ζ (φi ). Therefore, φ = g ⋅ φ. Observe that for all φ ∈ F ′ (Γ, G), the image φ(Γ) is an Abelian subgroup of the Abelian subgroup ℝn × Z(H2n+1 ) = {g ∈ G, ξl (g) = 0, l = 1, . . . , n} ≅ ℝn+1 . Then φ factors through the quotient map π :, Γ → Γ′ = Γ/[Γ, Γ], where [Γ, Γ] is the derivative group. We consider the map ξ : F ′ (Γ, G) φ

󳨀→ 󳨃󳨀→

Hom(Γ′ , ℝn+1 )

φ,̃

̃ where φ(γ[Γ, Γ]) = φ(γ). Lemma 5.6.17. The map ξ is a homeomorphism. Proof. The inverse of ξ is the map φ̃ 󳨃→ φ̃ ∘ π, and the bicontinuity is clear. The Abelian group Γ′ is finitely generated. Let T(Γ′ ) be the subgroup of the torsion elements in Γ′ . Then the quotient group Γ′′ = Γ′ /T(Γ′ ) is an Abelian torsion-free finitely generated group. Thus, Γ′′ is a free Abelian group isomorphic to ℤl , where l = rk(Γ′′ ). Lemma 5.6.18. The topological spaces Hom(Γ′ , ℝn+1 ) and Hom(Γ′′ , ℝn+1 ) are homeomorphic. Proof. Note that T(Γ′ ) ⊂ ker(ψ) for all ψ ∈ Hom(Γ′ , ℝn+1 ). Then as before, the map ξ ′ : Hom(Γ′ , ℝn+1 ) ψ

󳨀→ 󳨃󳨀→

Hom(Γ′′ , ℝn+1 ) ψ,̃

′ ̃ defined by ψ(γT(Γ )) = ψ(γ), is a continuous bijection, its inverse is ψ̃ 󳨃→ ψ̃ ∘ π ′ , where ′ π is the quotient map Γ′ → Γ′′ .

5.6 The case of Diamond groups | 273

Proof of Theorem 5.6.10. Suppose that [φ] is open in Hom(Γ, G)/G, then by Lemmas 5.6.14 and 5.6.15, [φ] is an open point in F ′′ (Γ, G). Using Lemma 5.6.16, we see that the inverse image of [φ] in F ′ (Γ, G) is an isolated point. As a consequence of Lemmas 5.6.17 and 5.6.18, the space Hom(Γ′′ , Rn+1 ) contains an isolated point. If the rank l of Γ′′ is nonzero, then Hom(Γ′′ , Rn+1 ) is homeomorphic to Mn,l (ℝ). Thus, l = 0 and the set Hom(Γ′′ , Rn+1 ) is reduced to the trivial homomorphism. Using again Lemmas 5.6.18 and 5.6.17, we see that F ′ (Γ, G) = {0}. In particular, φ is the trivial homomorphism. In this case, φ is a fixed point by the action of G on Hom(Γ, G), and thus φ is an isolated point in Hom(Γ, G). But Hom(Γ, G) is closed, then for all ψ ∈ Hom(Γ, G), limt→0 t ⋅ ψ ∈ F ′ (Γ, G) = {φ}. It comes out that φ is an isolated point if only if Hom(Γ, G) is reduced to the trivial homomorphism, which is impossible because the natural injection of Γ in G is not trivial. 5.6.3 Proofs of Theorem 5.6.11 and Corollary 5.6.12 Let us prove first the following results. Lemma 5.6.19. If H is a dilation-invariant subgroup, then R (Γ, G, H) is an ℝ∗+ -stable subspace of Hom(Γ, G). In particular, ℝ∗+ acts continuously on T (Γ, G, H). Proof. As ℝ∗+ acts by a continuous automorphism, φ is injective and φ(Γ) is discrete if and only if (t ⋅ φ) is also injective and (t ⋅ φ)(Γ) is discrete. Recall that G is simply connected, then the subgroup Γ is torsion-free. Thus, the action of Γ on G/H is proper only if it is free. By definition of the parameter space (3.1), we have to prove that the action of Γ on G/H via φ is proper if and only if the action of Γ on G/H via t ⋅ φ is proper. Let S be a compact set in G. Then (t ⋅ φ)(Γ)S = {γ ∈ (t ⋅ φ)(Γ), γSH ∩ SH ≠ 0}

= t{γ ∈ φ(Γ), (t ⋅ γ)SH ∩ SH ≠ 0} = t{γ ∈ φ(Γ), t −1 ((t ⋅ γ)SH ∩ SH) ≠ 0} = t{γ ∈ φ(Γ), γ(t ⋅ S)H ∩ (t ⋅ S)H ≠ 0} = tφ(Γ)t⋅S .

Therefore, the sets (t ⋅ φ)(Γ)S and φ(Γ)t⋅S are homeomorphic. Lemma 5.6.20. Let φ ∈ R (Γ, G, H). If φ is a locally rigid, then G ⋅ φ ∩ F ′ (Γ, G) ≠ 0. In particular, Γ is an Abelian subgroup. Proof. Assume that φ is locally rigid, then [φ] is an open point in T (Γ, G, H). By continuity of the action of ℝ∗+ , we deduce that [φ] ∈ Fix(Γ, G) and we conclude by Lemma 5.6.15 that G ⋅ φ ∩ F ′ (Γ, G) ≠ 0. Let φ′ be an element of the intersection, then φ′ (Γ) is a subgroup of the Abelian subgroup ℝn × Z(H2n+1 ). As φ′ is injective, Γ is Abelian.

274 | 5 Local and strong local rigidity Lemma 5.6.21. Let φ ∈ F ′ (Γ, G)∩R (Γ, G, H). Then for any real number t ≠ 0, the element φt ∈ F ′ (Γ, G) defined by ζ ((φt )i ) = tζ (φi )

and

τl ((φt )i ) = tτl (φi ) for all l = 1, . . . , n,

is also an element of R (Γ, G, H). Proof. The subgroup φ(Γ) is a discrete subgroup of the connected simply connected Lie subgroup ℝn × Z(H2n+1 ). Thus, φ(Γ) has a syndetic hull Lφ , which is the linear span of the generators φ1 , . . . , φk . Similarly, if t ≠ 0 the subgroup φt (Γ) is a discrete subgroup of ℝn × Z(H2n+1 ) and its syndetic hull is equal to Lφ . By Fact 2.1.8, the action of φ(Γ) is proper if and only if the action of φt (Γ) is proper. Let ℝ∗ := ℝ \ {0}. From Lemma 5.6.21, for φ ∈ F ′ (Γ, G) ∩ R (Γ, G, H), the map cφ : ℝ∗ t

󳨀→

R (Γ, G, H)

󳨃󳨀→

φt

is a well-defined and continuous injection. As a consequence, we can state the following. Lemma 5.6.22. The map cφ : ℝ∗ → T (Γ, G, H), t 󳨃→ [φt ] is a continuous injection. Proof. The map cφ is the composition of two continuous injective maps, cφ and the restriction of the quotient map to F ′ (Γ, G) ∩ R (Γ, G, H). Proof of the Theorem 5.6.11. Suppose that φ is locally rigid. By Lemma 5.6.20, we can assume that φ ∈ F ′ (Γ, G). Then the map cφ is well-defined and using Lemma 5.6.22, we conclude that cφ −1 ([φ]) is an open point in ℝ∗ , which is impossible. Proof of Corollary 5.6.12. Assume that U is connected. By Proposition 5.6.5, the subgroup K = U ⋉ L(H0 ) is a syndetic hull of H, where L(H0 ) is the syndetic hull of H0 . As φ(Γ) is a torsion-free, R (Γ, G, H) = R (Γ, G, L(H)) by Lemma 3.1.2. Proposition 5.6.2 induces the existence of g ∈ G, such that Adg (k) is a subgraded subalgebra, where k is the Lie subalgebra of K. Then gKg −1 is a dilation-invariant subgroup and by Lemma 3.1.1 we conclude that R (Γ, G, H) = R (Γ, G, gKg −1 ). Assume now H ∩ Z(H2n+1 ) is nontrivial. As before, we have R (Γ, G, H) = R (Γ, G, K). By hypothesis, L(H0 ) ∩ Z(H2n+1 ) is a nontrivial connected subgroup of Z(H2n+1 ), which is one-dimensional, thus Z(H2n+1 ) ⊂ L(H0 ). Then the Lie subalgebra of L(H0 ) is graded and from Lemma 5.6.9 we conclude that K is dilation-invariant.

5.7 A local rigidity theorem for finite actions Let G be a Lie group and Γ a finite group. We show here that the space Hom(Γ, G)/G is discrete and in addition finite if G has finitely many connected components. This

5.7 A local rigidity theorem for finite actions | 275

means that in the case where Γ is a discontinuous group for the homogeneous space G/H, where H is a closed subgroup of G. All of the elements of the parameter space are locally rigid. Equivalently, any Clifford–Klein form of the finite fundamental group does not admit nontrivial continuous deformations. We have the following. Theorem 5.7.1. Let G be a Lie group and Γ a finite discontinuous group for a homogeneous space G/H, where H is a closed subgroup of G. Then any homomorphism in R (Γ, G, H) is strongly locally rigid. Theorem 5.7.1 is obtained as a direct consequence from a more general result (cf. Theorem 5.7.4 below). The first goal is to show that the quotient topology on the space Hom(Γ, G)/G turns out to be discrete for any Lie group G and for any finite group Γ. In this setting, we show that Hom(Γ, G)/G and G/G0 share some common properties, where G0 denotes the connected component of the identity. Namely, Hom(Γ, G)/G is discrete and countable as it is the case for G/G0 . Likewise Hom(Γ, G)/G is finite whenever G/G0 is. Compact subgroups of G are of paramount importance, and we first quote the following result. Fact 5.7.2 ([82, Theorem 14.1.3], [83, Theorem 3.1] and [17]). Any Lie group G with finitely many connected components contains a compact subgroup C with the property that for every other compact subgroup U of G, there exists g ∈ G such that gUg −1 ⊂ C. Let G0 designate as above the identity connected component of G. This subgroup has the following properties: (i) C is a maximal compact subgroup of G. (ii) C ∩ G0 is connected and C intersects each connected component of G. (iii) Any other maximal compact subgroup of G is conjugate to C by an element of G0 . (iv) C0 := C ∩ G0 is a maximal compact subgroup of G0 . (v) If U ⊂ G is a compact subgroup intersecting with each connected component of G and for which U ∩ G0 is a maximal compact subgroup of G0 , then U is a maximal compact subgroup of G. Our second objective is to generate a first-step solution to a conjecture of Montgomery: If C is a compact Lie group operating on a compact manifold M, then there exist at most a finite number of inequivalent orbits. For a finite group H, let ord(H) denote its cardinality. We record the following. Fact 5.7.3 ([112, Lemma 1]). Let G be a compact Lie group and B an integer. Then there are at most a finite number of mutually nonconjugate subgroups H with ord(H) ⩽ B. Recall that the group G acts on Hom(Γ, G) through inner conjugation, i. e. for φ ∈ Hom(Γ, G) and g ∈ G, we consider the element φg of Hom(Γ, G) defined by φg (γ) =

276 | 5 Local and strong local rigidity gφ(γ)g −1 , γ ∈ Γ. In this respect, we define an equivalence relation on Hom(Γ, G) by φ1 ∼ φ2

mod (G)

if there exists g ∈ G such that φg1 = φ2 .

(5.37)

To summarize, we prove the following. Theorem 5.7.4. Let G be a Lie group and Γ a finite group. Then the space Hom(Γ, G)/G is discrete and at most countable. This space is finite if in addition G has finitely many connected components. Proof. Assume first that G has finitely many connected components. By Fact 5.7.2, G contains a maximal compact subgroup C. Put Γ := {γ1 , . . . , γp }. We denote by Fp (C) (resp., Fp (G)) the set of subgroups of C (resp., of G) of orders less or equal to p. By Fact 5.7.2, the natural map Fp (C)/C 󳨀→ Fp (G)/G is surjective and by Fact 5.7.3, Fp (C)/C is finite. Hence Fp (G)/G is a finite set, say, {[Δ1 ], . . . , [ΔM ]}, where Δj ⊂ G with #Δj ⩽ p. Then #(Hom(Γ, G)/G) ⩽ ∑M j=1 #(Hom(Γ, Δj )). This shows that Hom(Γ, G)/G is finite. We now move to the general case. Let φ ∈ Hom(Γ, G) and we define a subgroup ̃ G(φ) := π −1 (π(φ(Γ))) of G, where π : G 󳨀→ G/G0 is the natural quotient map. As ̃ φ(Γ) ⊂ G(φ) and ̃ G(φ) = ⋃ φ(γ) G0 = ⋃ φ(γi ) G0 , γ∈Γ

1⩽i⩽p

̃ ̃ then Hom(Γ, G(φ))/ G(φ) is finite. Since G/G0 is countable, there exist I ⊂ ℕ and some class representatives {gi }i∈I such that G = ⋃i∈I gi G0 . Therefore, 󵄨󵄨 Λ := {⋃ gi G0 󵄨󵄨󵄨 F ⊂ I, F is finite} 󵄨 i∈F

̃ is a countable set (as a partition of sets) and so is the set Σ := {G(φ) | φ ∈ Hom(Γ, G)}. ̃ , where G ̃ := There exists a sequence {φk }k∈ℕ of Hom(Γ, G) such that Σ = ⋃k∈ℕ G k k −1 π (π(φk (Γ))). As the natural map, ̃ )/G ̃ 󳨀→ Hom(Γ, G)/G, ⋃ Hom(Γ, G k k

k∈ℕ

is surjective, it follows that Hom(Γ, G)/G is countable. Let {φk }k∈ℕ be a sequence of Hom(Γ, G) that converges to φ ∈ Hom(Γ, G). We ̃ prove that φk ∼ φ mod(G(φ)), for k large enough, where the relation ∼ is as defined above in (5.37). Note that there exists N ∈ ℕ such that for any k ⩾ N and ̃ ) = G(φ). ̃ i ∈ {1, . . . , p}, we have φk (γi )G0 = φ(γi )G0 and then G(φ Hence {φk }k∈ℕ conk ̃ ̃ verges to φ in Hom(Γ, G(φ)). As G(φ) has finitely many connected components, then ̃ φk ∼ φ mod(G(φ)), for k large enough. This shows that Hom(Γ, G)/G is discrete.

5.7 A local rigidity theorem for finite actions | 277

5.7.1 (Strong) local rigidity for K ⋉ ℝn Let as in Section 2.5, K be a compact subgroup of GLn (ℝ) and G := K ⋉ℝn the semidirect product of K (with respect to the Euclidean product ⟨⋅, ⋅⟩ on ℝn ) and ℝn . The goal here is to study the rigidity proprieties of deformation parameters of the natural action of a discontinuous subgroup Γ ⊂ G, on a homogeneous space G/H, where H stands for a closed subgroup of G. That is, we prove the following local (and global) rigidity theorem: The parameter space admits a locally rigid (equivalently a strongly locally rigid) point if and only if Γ is finite. In this last situation, we also show that R (Γ, G, H) is a finite union of G-orbits for which the corresponding subgroups act fixed point freely on G/H and the deformation space T (Γ, G, H) is a finite set. Theorem 5.7.5. Let G = K ⋉ ℝn be a compact extension of ℝn , H is a closed subgroup of G and Γ a discontinuous group for G/H. Then the following assertions are equivalent: (1) R (Γ, G, H) contains a locally rigid element. (2) Γ is finite. (3) Any element of R (Γ, G, H) is a strongly, locally, rigid element. (4) Any element of R (Γ, G, H) is locally rigid. Proof. Let Γ be a discrete subgroup of G and as above Γ∞ a subgroup of Γ fulfilling Theorem 1.2.15. As Γ/Γ∞ is finite, Γ∞ is cocompact in Γ and, therefore, Γ acts properly on G/H if and only if Γ∞ does as in Fact 2.1.8. For μ ∈ ℝ∗ and Γ a discrete subgroup of G, we set Γμ = {(A, μx) | (A, x) ∈ Γ}. It is readily seen that Γμ is also a discrete subgroup of G. We first prove the following. Lemma 5.7.6. For any φ ∈ Hom(Γ, G), there exist φ1 ∈ Hom(Γ, K) and a map φ2 : Γ → ℝn such that φ(γ) = (φ1 (γ), φ2 (γ)) and φ2 (γγ ′ ) = φ2 (γ)+φ1 (γ)φ2 (γ ′ ), for any γ and γ ′ ∈ Γ. Proof. The homomorphism condition says that (φ1 (γγ ′ ), φ2 (γγ ′ )) = (φ1 (γ), φ2 (γ))(φ1 (γ ′ ), φ2 (γ ′ )), for γ and γ ′ ∈ Γ. This gives in turn that φ1 ∈ Hom(Γ, K)

and φ2 (γγ ′ ) = φ2 (γ) + φ1 (γ)φ2 (γ ′ ).

We next remark the following. Lemma 5.7.7. Let Γ be a discrete subgroup of G and H a closed one. Then Γ acts properly on G/H if and only if Γμ does, for any μ ∈ ℝ∗ . Proof. As K is conjugate to a subgroup of On (ℝ), we can and do assume that K ⊂ On (ℝ). The statement is obvious whenever Γ is finite. For this, we will consider the case where

278 | 5 Local and strong local rigidity Γ is infinite. Thanks to Lemma 1.2.5, Γ contains an element of infinite order. It is then enough to show the result when Γ is Abelian. Going through a conjugation, we can assume, as in Lemma 1.2.18, that any element (A, x) of Γ satisfies the condition x = PA (x).

(5.38)

Let (x1 , . . . , xp ) be a family of maximal rank in p2 (Γ), where p2 : G 󳨀→ ℝn stands for the second projection. Fix r > 0 and set Tr := [Cr ⋅ H ⋅ Cr ] ∩ Γμ , where Cr = K × B(0, r). Clearly, Tr is a subset of K × p2 (Tr ). Consider (A, μx) ∈ Tr . There exist (S, u), (S′ , u′ ) elements of Cr and (B, y) ∈ H such that (A, μx) = (S, u)(B, y)(S′ , u′ ). Then μx = Sy + v with v := u + SBu′ ∈ B(0, 2r). Moreover, there exist α1 , . . . , αp ∈ ℝ such that x = ∑pi=1 αi xi . Consider now x = ∑pi=1 [μαi ]xi , where [a] designates the floor of a given real number a. It is easy to observe that x ∈ p2 (Γ). Indeed, for any 1 ⩽ i ⩽ p, as xi ∈ p2 (Γ), there exists Ai ∈ K such that (Ai , xi ) ∈ Γ. As Γ is Abelian satisfying (5.38), then thanks to Lemma 1.2.18, p

p

∏(Ai , xi )[μαi ] = (∏ Ai i=1

i=1

[μαi ]

p

, ∑[μαi ]xi ). i=1

Besides, we have p

p

i=1

i=1

‖μx − x‖ ⩽ ∑(μαi − [μαi ]) ⋅ ‖xi ‖ ⩽ c := ∑ ‖xi ‖. The latter inequality means that there exists v′ ∈ B(0, c) such that μx = x + v′ . This gives in turn that x = Sy + w where w := v − v′ ∈ B(0, 2r + c). Since x ∈ p2 (Γ), there exists Ω ∈ K such that (Ω, x) ∈ Γ. Hence we get (Ω, x) = (S, w)(B, y)(B−1 S−1 Ω, 0). This in turn yields (Ω, x) ∈ [C2r+c ⋅ H ⋅ C2r+c ] ∩ Γ. As [C2r+c ⋅ H ⋅ C2r+c ] ∩ Γ is finite, there exists c′ such that ‖x‖ ⩽ c′ and, therefore, ‖μx‖ ⩽ ‖μx − x‖ + ‖x‖ ≤ 2r + c + c′ . This proves that p2 (Tr ) is bounded, which implies that Tr is finite, and hence the action is proper as in Lemma 4.3.23. Back to the proof of Theorem 5.7.5. The assertions (3) 󳨐⇒ (4) and (4) 󳨐⇒ (1) are obvious. The assertion (2) 󳨐⇒ (3) comes from Theorem 5.7.1. We only prove (1) 󳨐⇒ (2). It suffices to show that local rigidity fails whenever Γ is infinite. Assume that Γ is

5.7 A local rigidity theorem for finite actions | 279

infinite. Take φ ∈ R (Γ, G, H) and μ ∈ ℝ∗ . For any γ ∈ Γ, set φ(γ) = (φ1 (γ), φ2 (γ)) and φμ (γ) = (φ1 (γ), μφ2 (γ)). Notice that φμ (Γ) is also discrete since φμ is a homomorphism and φμ (Γ) = (φ(Γ))μ . As φ(Γ) acts properly on G/H, we get as in Lemma 5.7.7, φμ (Γ) does for any μ ∈ ℝ∗ . We shall now prove that φμ (Γ) acts freely on G/H. Assume that for some (S, t) ∈ G and μ ∈ ℝ∗ , we have (S, t)(B, y)(S, t)−1 = (A, μx), where (A, x) ∈ φ(Γ) and (B, y) ∈ H. This means that (A, μx), (A, x) and (B, y) are of finite orders because PA (μx) = μPA (x) = 0, as in Lemma 1.2.5. Hence A and B are similar and thanks to Lemma 1.2.6, (A, x) and (B, y) are conjugate, then φ(Γ) does not act freely on G/H, which is absurd. Therefore, φμ ∈ R (Γ, G, H) for any μ ∈ ℝ∗ . Assume now that the orbit of φ is open in R (Γ, G, H), and there exists ε > 0 such that for any μ ∈ ]1 − ε, 1 + ε[, there exists (Mμ , tμ ) ∈ G such that for any γ ∈ Γ, φμ (γ) = (Mμ , tμ )φ(γ)(Mμ , tμ )−1 . This means already that Mμ commutes with φ1 (γ) and Mμ φ2 (γ) = (φ1 (γ) − I)tμ + μφ2 (γ).

(5.39)

As Γ is infinite, there exists an element γ0 ∈ Γ of infinite order thanks to Lemma 1.2.5. Let (φ1 (γ0 ), φ2 (γ0 )) = (A0 , x0 ), then equation (5.39) gives Mμ x0 = (A0 − In )tμ + μx0 . Since (A0 − I)tμ ∈ {ker A0 − I}⊥ , then P(A0 )Mμ x0 = μPA0 (x0 ). As PA0 is a polynomial of A0 , it commutes with Mμ . At the end, Mμ PA0 (x0 ) = μPA0 (x0 ), and PA0 (x0 ) is not zero by Lemma 1.2.5. This is absurd as Mμ is orthogonal. 5.7.2 (Strong) local rigidity for Heisenberg motion groups This section aims to study the local rigidity proprieties of deformation parameters of the natural action of a discontinuous group Γ ⊂ G = Gn acting on a homogeneous space Gn /H, where H stands for a closed subgroup of the Heisenberg motion group Gn := 𝕌n ⋉ ℍn . That is, the parameter space admits a locally rigid (equivalently a strongly locally rigid) point if and only if Γ is finite. The results are mainly obtained thanks to basic upshots concerning closed subgroups of the group in question as in Section 1.3. We keep all the notation and definitions there. For any μ ∈ ℝ∗+ , define the map Θμ : G → G, (A, z, t) 󳨃→ (A, μz, μ2 t). Clearly, Θμ is an automorphism of G. For any φ ∈ Hom(Γ, G), set φμ := Θμ ∘ φ. Proposition 5.7.8. For any μ ∈ ℝ∗+ and any φ ∈ R (Γ, G, H), φμ ∈ R (Γ, G, H).

280 | 5 Local and strong local rigidity Proof. Remark first that for any μ ∈ ℝ∗+ and any φ ∈ R (Γ, G, H), φμ (Γ) = Θμ ∘ φ(Γ). Then φμ is injective and φμ (Γ) is discrete. Now for γ := (Aγ , zγ , tγ ) ∈ φ(Γ) of infinite order, γμ = (Aγ , μzγ , μ2 tγ ) is also of infinite order. By Lemma 1.3.1, there exists uγ ∈ ℂn such that τuγ γτu−1γ = (Aγ , zγ′ , tγ′ ) where Aγ zγ′ = zγ′ , tγ′ zγ′ = 0 and ‖zγ′ ‖ + |tγ′ | ≠ 0. A direct computation shows that −1 τμuγ γμ τμu = (Aγ , μzγ′ , μ2 tγ′ ). γ

(5.40)

Equation (5.40) shows that for any μ ∈ ℝ∗+ , φμ (Γ) and φ(Γ) are of the same type. Moreover, pr(φμ (Γ)) is discrete if and only if pr(φ(Γ)) is. Indeed, for any positive real ρ, Card[pr(φμ (Γ)) ∩ (𝕌n × B(0, ρ))] = Card{(A, μz) ∈ pr(φμ (Γ)), ‖μz‖ ≤ ρ} = Card{(A, μz) ∈ pr(φμ (Γ)), ‖z‖ ≤ = Card{(A, z) ∈ pr(φ(Γ)), ‖z‖ ≤

ρ } μ

ρ }, μ

where the symbol “Card” means the cardinality. Hence φμ (Γ) acts properly on G/H whenever φ(Γ) does. It remains to see that φμ (Γ) acts freely on G/H. Assume that one can find λμ ∈ gφμ (Γ)g −1 ∩ H. Since the Γ-action on G/H is proper, λμ is of finite order and so is δμ = g −1 λμ g. Note that δμ = (B, μv, μ2 s) for some δ = (B, v, s) ∈ φ(Γ). By the arguments of the proofs of Lemma 1.3.1 and Proposition 1.3.3, this leads us to find u1 , u2 ∈ ℂn such that τu1 δτu−11 = τu2 δμ τu−12 = (B, 0, 0). In particular, δ and δμ are G-conjugate to each other. Hence δμ is in g ′ φ(Γ)(g ′ )−1 ∩ H for some g ′ ∈ G. This implies that δμ is the identity element of G since φ(Γ)-action on G/H is free. Therefore, gφμ (Γ)g −1 ∩ H = {(I, 0, 0)} for any g ∈ G, and thus the φμ (Γ)-action on G/H is free. This completes the proof. The main result in this section is the following. Theorem 5.7.9. Let G be the Heisenberg motion group, H is a closed subgroup of G and Γ a discontinuous group for G/H. Then the statement of Theorem 5.7.5 holds. Proof. Thanks to Theorem 5.7.4, it suffices to show the assertion (3) ⇒ (4), which means that the local rigidity fails whenever Γ is infinite. Take φ ∈ R (Γ, G, H) and μ ∈ ℝ∗+ . Thanks to Proposition 5.7.8, φμ ∈ R (Γ, G, H). Now assume that the G-orbit of φ is an open set in R (Γ, G, H). There exists ε > 0 such that for any μ ∈ ]1 − ε, 1 + ε[, there exists (Mμ , zμ , 0) ∈ G such that for any γ ∈ Γ, φμ (γ) = (Mμ , zμ , 0)φ(γ)(Mμ , zμ , 0)−1 .

(5.41)

5.7 A local rigidity theorem for finite actions | 281

For any γ ∈ Γ, write φ(γ) = (A(γ), z(γ), t(γ)) and then equation (5.41) reads Mμ ∈ ⋂ C (A(γ)), γ∈Γ

μz(γ) = Mμ (I − A(γ))Mμ−1 zμ + Mμ z(γ)

(5.42) (5.43)

and μ2 t(γ) = t(γ) −

1 Im[⟨zμ , Mμ z(γ))⟩ − ⟨zμ + Mμ z(γ), Mμ A(γ)Mμ−1 zμ ⟩]. 2

(5.44)

Here, C (A(γ)) denotes the set of commutators of A(γ). By Proposition 1.3.8, as Γ is infinite, there exists γ0 ∈ Γ of infinite order. Let φ(γ0 ) = (A0 , z0 , t0 ). One can assume thanks to Lemma 1.3.1 that A0 z0 = z0 , t0 z0 = 0 and ‖z0 ‖ + |t0 | ≠ 0. If z0 = 0 and t0 ≠ 0, then (5.44) leads to μ2 t0 = t0 for any μ ∈ ]1 − ε, 1 + ε[, which is absurd. If z0 ≠ 0 and t0 = 0, then μz0

= = by (5.43)

=

by (5.42)

μPA0 (z0 )

PA0 (μz0 )

PA0 Mμ (I − A0 )Mμ−1 zμ + PA0 Mμ z0

=

P A0 (I − A0 ) zμ + Mμ PA0 (z0 ) ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟

=

Mμ z0 ,

=0

which is also absurd since Mμ ∈ 𝕌n and |μ| ≠ 1. 5.7.3 A variant of the local rigidity conjecture As a direct consequence of Theorem 5.7.4, for any finite subgroup Γ of a connected nilpotent Lie group G, any φ ∈ R (Γ, G, H) is locally rigid and, therefore, the deformation space T (Γ, G, H) is discrete. This gives some evidence to the indirect statement of the following conjecture. Conjecture 5.7.10. Let G be a connected nilpotent Lie group, H a connected subgroup of G and Γ a nontrivial discontinuous group for G/H. Then the local rigidity holds if and only if Γ is a finite group. In this case, finite subgroups of G turn out to be central and, therefore, trivial when G is in addition simply connected. This finally shows that Conjectures 5.1.1 and 5.7.10 are equivalent in this setup. On the other hand, it will be interesting to figure out whether Conjecture 5.1.1 may hold for compact extensions of nilpotent Lie groups as already shown in Theorems 5.7.5 and 5.7.9.

6 Stability concepts and Calabi–Markus phenomenon Let G be a Lie group, H a closed subgroup of G and Γ a discrete subgroup of G. The concept of stability of an element φ ∈ Hom(Γ, G) measures in general the fact that in a neighborhood of φ, the properness property of the action on G/H is preserved. This concept may be one fundamental genesis to understand the local structure of the deformation space. Many open problems and related issues can be found in [32, 96, 99] and [100]. The stability fails for the ℤ-action on ℝ in Aff(ℝ) [98, Example 1.1.1], whereas stability holds for the standard compact Lorentzian space form (cf. [95]). When the set R (Γ, G, H) is an open subset of Hom(Γ, G), then obviously each of its elements is stable, which is the case for any irreducible Riemannian symmetric space with the assumption that Γ is a torsion-free uniform lattice of G ([98] and [130]). The determination of stable points is a very difficult problem in general, which is our subject of deal in this chapter. This reduces in fact to describe explicitly the interior of the subset of Hom(Γ, G) of injective homomorphisms with discrete image. This set fails to be open in Hom(Γ, G) but nevertheless the stability property may hold as reveals Example 6.4.19 below. This arouses our attention to consider some other variants of stability, such as geometric and near stability. These variants may help us in understanding the local geometric features of the deformations in question. We are then led to investigate several kinds of questions of geometric nature related to the structure of the deformation space and as a result, many stability theorems will be established in the nilpotent and exponential cases and also in the context of some compact extensions. On the other hand, it may then happen that there does not exist an infinite discrete subgroup Γ of G, which acts properly discontinuously on G/H. This phenomenon was first discovered by E. Calabi and L. Markus [52] for (G, H) = (SO(n, 1), SO(n − 1, 1)), and is called the Calabi–Markus phenomenon. For instance, T. Kobayashi proved in [94] that the Calabi–Markus phenomenon occurs if and only if rankℝ G = rankℝ H when G is a reductive linear Lie group, and H a closed reductive subgroup of G. Besides, he showed in [97] that for a proper closed subgroup H of a solvable Lie group G, there exists a discontinuous group Γ for G/H such that the fundamental group π1 (Γ\G/H) is infinite, showing that the Calabi–Markus phenomenon does not occur in this context. We here deal with Calabi–Markus’s phenomenon and the question of existence of compact Clifford–Klein forms in the context of some compact extensions of nilpotent Lie groups. The obtained results are mainly based on several upshots proved in previous chapters.

https://doi.org/10.1515/9783110765304-006

284 | 6 Stability concepts and Calabi–Markus phenomenon

6.1 Stability concepts We keep all our notation and settings. For a discontinuous group Γ for a homogeneous space G/H, we pose Hom0 (Γ, G) = {φ ∈ Hom(Γ, G) : φ is injective} and Hom0d (Γ, G) = {φ ∈ Hom0 (Γ, G) : φ(Γ) is discrete}. Then clearly Hom0 (Γ, G) and Hom0d (Γ, G) coincide whenever Γ is finite. Definition 6.1.1 (The concept of stability). Let us come back for a while to a general locally compact group G. Let Γ be a discrete subgroup of G. We focus attention here on the Γ-action on G/H when H is a noncompact closed subgroup of G. The homomorphism φ is said to be topologically stable or merely stable in the sense of Kobayashi– Nasrin [98], if there is an open set in Hom(Γ, G), which contains φ and is contained in R (Γ, G, H). When the set R (Γ, G, H) is an open subset of Hom(Γ, G), then obviously each of its elements is stable, which is the case for any irreducible Riemannian symmetric spaces with the assumption that Γ is a torsion-free uniform lattice of G (cf. [98] and [130]). The following general fact allows to establish a relationship between local rigidity and stability properties. Proposition 6.1.2. A point in R (Γ, G, H) is rigid if and only if it is locally rigid and stable. Proof. Let U be a neighborhood in Hom(Γ, G) of a locally rigid point u ∈ R (Γ, G, H) and let it be contained in R (Γ, G, H). Then G ⋅ u is an open set of R (Γ, G, H) and is contained in G ⋅ U, which is also open in Hom(Γ, G). Thus, G ⋅ u is an turn open in Hom(Γ, G); this conclusively shows that u is rigid. The converse is trivial. Definition 6.1.3 (Geometric stability, cf. [15]). A homomorphism φ ∈ R (Γ, G, H) is said geometrically stable if there exist a neighborhood Vφ ⊂ Hom(Γ, G) such that for any ψ ∈ Vφ , ψ(Γ), the closure of ψ(Γ) in G acts properly on G/H. The following remark is then immediate. Remark 6.1.4. Any point in R (Γ, G, H) is geometrically stable if Γ is finite or H is compact. Definition 6.1.5 (Near stability, cf. [15]). A homomorphism φ ∈ R (Γ, G, H) is said to be nearly stable if there is an open set in Hom0d (Γ, G), which contains φ and is contained in R (Γ, G, H). The following is also immediate.

6.2 Stability of nilmanifold actions | 285

Fact 6.1.6. Assume the Γ is torsion-free and H is compact. Then any homomorphism in R (Γ, G, H) is nearly stable. Recall now the set ⋔ (Γ : G) as in Definition 2.1.1, consisting of subsets H for which SHS−1 ∩ Γ is compact for any compact set S in G and ⋔gp (Γ : G) the set of all closed connected subgroups belonging to ⋔ (Γ : G). Definition 6.1.7 (Stability of discrete subgroups). (1) Let Γ be a discrete subgroup of G. We set Stab(Γ : G) the set of all H ∈ ⋔gp (Γ : G) for which the parameter space R (Γ, G, H) is open. (2) A discrete subgroup Γ of G is said to be topologically stable (or merely stable), if Stab(Γ : G) = ⋔gp (Γ : G). Remark 6.1.8. The notion of stability is defined for discrete subgroups. For a discrete subgroup Γ ⊂ G, Γ becomes a discontinuous group for G/H for any H ∈ ⋔gp (Γ : G).

6.2 Stability of nilmanifold actions 6.2.1 Case of Heisenberg groups We now come back to the Heisenberg case. We refer to Section 4.1 for notation and definitions. The aim is to provide a necessary and sufficient condition for the stability property. Such a result is fundamental when we treat the question whether the deformation space is (or is not) equipped with a smooth manifold structure. We first record the following direct consequence. Proposition 6.2.1. Retain the same assumptions as in Theorem 4.1.28 (or also in Theorem 4.1.33). Then the stability property holds and G acts on the parameter space R (l, g, h) with constant dimension orbits. We now look at the case where Γ is not maximal in G according to Definition 4.1.38. Proposition 6.2.2. Let H be a connected subgroup of the Heisenberg group G, which does not meet the center of G and Γ a nonmaximal discontinuous group of G for the homogeneous space G/H. Then: (1) T0 (l, g, h) is an open dense smooth manifold of T (l, g, h) and the set of stable points precisely coincides with R0 (l, g, h), which in turn consists of points whose orbits are of maximal dimension among orbits in R(l, g, h). (2) T (l, g, h) fails to be a Hausdorff space. (3) The G-orbits of R(l, g, h) are not of constant dimensions.

286 | 6 Stability concepts and Calabi–Markus phenomenon Proof. Let i ∈ {1, . . . , k} and let M = M(x, A, B) ∈ Ri (l, g, h) = ⋃θ∈I k−1 Riθ (l, g, h). Then 2n−s

A A1 ⋅ ⋅ ⋅ Ai−1 Ai+1 ⋅ ⋅ ⋅ Ak rk ( ) = rk ( 1 )=k−1 B B ⋅ ⋅ ⋅ Bi−1 Bi+1 ⋅ ⋅ ⋅ Bk and t Bp Aq − t Ap Bq = 0 for all p, q = 1, . . . , k. We stick to the notation a and b to designate the subspaces of ℝ2n spanned with the column vectors {M j (0, A, B), j = 1, . . . , k} and {M j (0, −B, A), j = 1, . . . , k}, respectively. Then dim(b) = dim(a) = k − 1 and as above, the homomorphism condition is being equivalent to b ⊂ a⊥ but in turn b ⊊ a⊥ as Γ is not maximal. Fix T = t (−T 2 , T 1 ) ∈ a⊥ \ b and let Mn (x, A, B) be the matrix obtained when replacing in M(x, A, B), the columns t (Ai , Bi ) by t

(Ai +

1 1 i 1 2 T , B + T ). n n

Then clearly Mn ∈ R0 (l, g, h) and (Mn )n converges to M, and finally no point inside the k−1 layers Riθ (l, g, h) (θ ∈ I2n−s and i ∈ {1, . . . , k}) is stable. This also shows that T0 (l, g, h) is an open, dense, smooth manifold of T (l, g, h) as the canonical surjection is open and continuous. This completes the proof of the first point. We now show that T (Γ, G, H) fails to be a Hausdorff space in this case. In fact, let Mi = M(xi , 0, C), i = 1, 2 where xi = (ti , 0, . . . , 0) and t1 ≠ t2 , As = 0 and 0 0

C=(

0 ). Ik−1

So, obviously, Mi ∈ R1θ (l, g, h) where θ = (2n − s − k + 2, . . . , 2n − s) and [M1 ] ≠ [M2 ] in T (Γ, G, H). Given any open neighborhoods Vi of Mi in R(l, g, h), for ε small enough, the matrix Miε = M(xi , 0, Cε ) belongs to Vi for i = 1, 2 where 0 Cε = ( ε 0

0 0 ).

Ik−1

This leads to the existence of gε ∈ G such that Adgε M1ε = M2ε , which is enough to conclude. This proves the second point. The proof of point 3 is obvious. We now state our main result in this section. Theorem 6.2.3. Let H = exp(h) be a connected subgroup of the Heisenberg group G = exp(g) and Γ a discontinuous group of G for the homogeneous space G/H with a syndetic hull L = exp(l). Then the following assertions are equivalent: (1) The space T (l, g, h) is equipped with a smooth manifold structure. (2) The space T (l, g, h) is a Hausdorff space. (3) dim G ⋅ ψ is constant for any ψ ∈ R (l, g, h).

6.2 Stability of nilmanifold actions | 287

(4) The stability holds. (5) z(g) ⊂ ψ(l) + h for any ψ ∈ R (l, g, h). More generally, the space T (l, g, h) admits a smooth manifold as its dense open subset whose preimage consists of topologically stable and maximal-dimensional orbit points. Proof. The proof will be divided into several steps. Under the condition z(g) ⊂ [l, l] + h, which means that either l is non-Abelian or h meets the center z(g), any deformation space T (l, g, h) is endowed with a smooth manifold structure as a direct consequence of Theorems 4.1.28 and 4.1.33. Beyond these cases, Γ turns out to be Abelian. With Proposition 6.2.1 in hand, we only have to prove that 5 implies 1. We first remark that Lemma 4.1.1 asserts that exp l acts properly on G/H if and only if z(g) ⊄ h, l∩h = {0} and z(g)∩(h⊕l) = l∩z(g). This entails that this condition means that either z(g) ⊂ h or z(g) ⊂ ψ(l) for any ψ ∈ R(l, g, h). This last fact means then that l is maximal and, therefore, R(l, g, h) is open in Hom∘ (l, g), so that the stability property holds. Proposition 4.1.35 allows to conclude. Remark 6.2.4. We showed in other words that the statements of Theorem 6.2.3 only hold in the cases where Γ is not Abelian, Γ is Abelian and maximal (according to Definition 4.1.38) or Γ is Abelian and h meets the center of g. This is also equivalent to the fact that z(g) ⊂ [l, l] + h or Γ is Abelian and maximal, which in turn encompasses the setting of compact Clifford–Klein forms.

6.2.2 From H2n+1 to H2n+1 × H2n+1 /Δ We study in this section the context of Subsection 4.1.9. Let Γ be a nontrivial discontinuous group for the homogenous space P/Δ, where P = exp(p) and p = h2n+1 × h2n+1 . In this contest, as Γ is a discontinuous group for the homogeneous space P/Δ, l ∩ D = {0}, then dim l ≤ 2n + 1. On the other hand, the vector space spanned by {(Z, 0), (Xi , 0), (0, Xi ), i = 1, . . . , n} is a (2n + 1)-dimensional Abelian subalgebra of p. That is, l is maximal Abelian (as in Definition 4.1.37) if and only if dim l = 2n + 1. Theorem 6.2.5. Let Γ be a discontinuous group for the homogeneous space P/Δ with the syndetic hull L = exp(l). Then the following assertions are equivalent: (i) z(p) ⊂ D ⊕ ψ(l) for any ψ ∈ R(l, p, D). (ii) The subalgebra l is non-Abelian or maximal Abelian. (iii) The stability holds. Namely, R(Γ, P; P/Δ) is open in Hom(Γ, P). Proof. Let ψ be an element of R(l, p, D) and M = M(x, y, A, B) its matrix as above. Then z(p) ⊂ D ⊕ ψ(l) if and only if there exist two vectors S ∈ l and T ∈ D satisfying MS = (Z, 0) + T.

(6.1)

288 | 6 Stability concepts and Calabi–Markus phenomenon We prove first that (i) ⇔ (ii). If l is non-Abelian, the matrix M equals to ̃ (0, A0 ), (0, B0 )), ̃ (y0 , y), M((x0 , x),

where x0 ≠ 0 and rank(B0 ) = k − 1.

Equality (6.1) holds by taking S = x1 (Z, 0) and T = 0 Abelian. Then dim(l) = 2n + 1 and the matrix

y0 (Z, Z). x0

Suppose that l is maximal

x ( ) ∈ M2n+1 (ℝ) B is invertible according to Proposition 4.1.43. Equality (6.1) holds for x S = ( ) (Z, 0) and B −1

y T = ( ) S. A

Conversely, if l is Abelian and nonmaximal, let ψ ∈ R(l, p, D) such that its matrix M = M(x, y, 0, B) with I B = ( k) . 0 Equation (6.1) gives that BS = 0 and so S = 0, which is impossible as (Z, 0) ∉ D. We prove now that (ii) ⇔ (iii). When l is non-Abelian, Corollary 4.1.44 enables us to conclude that any element of R(l, p, D) is stable. In case l is maximal Abelian, dim(l) = 2n+1. Therefore, the Clifford–Klein form Γ\G/H is compact and the parameter space is open in Hom(l, p). Conversely, if l is Abelian and nonmaximal to see that R(l, p, D) is not open in Hom(l, p), choose M0 = M(x0 , y0 , 0, B0 ) ∈ R(l, p, D) such that x0 = (1, 0, . . . , 0) and Ik−1 ). 0

B0 = ( Let Mε = M(x0 , y0 , Aε , B0 ), with

A1,ε ) 0

Aε = (

such that A1,ε = (ai,j ) ∈ Mn,k (ℝ), a1,1 = ε and ai,j = 0 elsewhere. It is easy to check that the matrix Mε satisfies the homomorphism conditions as here K = 0, and hence Mε is in E . For wε = ( ε1 , 0, . . . , 0) ∈ ℝ2n , we have x − wε Aε rk ( 0 ) = k − 1. B So, for all ε > 0, Mε ∉ R(l, p, D) according to Proposition 4.1.43.

6.2 Stability of nilmanifold actions | 289

Hausdorffness of the deformation space Theorem 6.2.6. Let Γ be a discontinuous group for the homogeneous space P/Δ. Then the deformation space T (Γ, P; P/Δ) is a Hausdorff space if and only if Γ is maximal Abelian. Proof. When the subgroup Γ is maximal Abelian, then by Proposition 6.2.8, the deformation space is a Hausdorff space. We first study the case where l is Abelian and nonmaximal. In this case, dim(l) ≤ 2n. Let M = M(x, y, A, B) and M ′ = M(x, y′ , A, B) with y = (y1 , 0, . . . , 0), y′ = (y1′ , 0, . . . , 0), y1 ≠ y1′ , B and B = ( 1 ) . B2

A1 ) A2

A=(

First, assume that rk(Γ) = k ≤ n and take here A = 0 and I B = ( k) . 0 For ε > 0, let also Mε = M(x, y, Aε , B) and Mε′ = M(x, y′ , Aε , B) be such that Aε = (ai,j ) where a2n,1 = ε and ai,j = 0 otherwise. It is clear that M, M ′ , Mε and Mε′ are in R(l, p, D) and that [M] ≠ [M ′ ]. For uε = (

y1′ − y1 , 0, . . . , 0), ε

vε = (0, . . . , 0,

y1′ − y1 ) ε

and gε ∈ P parameterized by (uε , vε ), one has gε ⋅ Mε = Mε′ and so [Mε ] = [Mε′ ]. Let V[M] and V[M ′ ] be two open sets in T (l, p, D) containing [M] and [M ′ ], respectively. If σ denotes the projection map from R(l, p, D) to T (l, p, D), then σ −1 (V[M] ) = VM and σ −1 (V[M ′ ] ) = VM ′ are two open sets in R(l, p, D) containing M and M ′ , respectively. There exists therefore ε > 0 such that Mε ∈ VM and Mε′ ∈ VM ′ , which shows that V[M] ∩ V[M ′ ] ≠ 0. Thus, the space T (l, p, D) fails to be a Hausdorff space. Suppose now that n < k < 2n and choose A and B in M2n,k (ℝ) such that B1 = (In

0) ,

0 B2 = ( 0

Ik−n ), 0

A1 = 0,

A2 = −B2 .

Let Mε = M(x, y, Aε , Bε ) and Mε′ = M(x, y′ , Aε , Bε ) such that A1ε = 0, B1,ε = B1 , 0 ∇ε

B2,ε = −A2ε = (

Ik−n ), 0

where ∇ε = (μi,j ) ∈ M2n−k,n (ℝ) with μ1,1 = ε and μi,j = 0 otherwise. Then the same conclusion holds when we take uε = (u1 , . . . , u2n ) and vε = (v1 , . . . , v2n ) in ℝ2n such that u1 = vk+1 =

y1 −y1′ ε

and ui = vj = 0 otherwise.

290 | 6 Stability concepts and Calabi–Markus phenomenon Finally, if k = 2n, let B1 = (In B2 = (

0),

0Mn−1,1 (ℝ) 0

In−1 0M1,n−1 (ℝ)

0Mn−1,n (ℝ) ), 0M1,n (ℝ)

A1 = 0 and A2 = −B2 . Choose Mε = M(x, y, Aε , B) and Mε′ = M(x, y′ , Aε , B) such that B1,ε = B1 ,

B2,ε = (

In−1

0Mn−1,1 (ℝ) ε

0M1,n−1 (ℝ)

0Mn−1,n (ℝ) ), 0M1,n (ℝ)

A1ε = 0

and A2ε = −B2ε .

Suppose here that x = (0, . . . , 0, 1). This gives that M, M ′ , Mε , Mε′ are all in R(l, p, D) and [M] ≠ [M ′ ] as rank(B) = k − 1 in these circumstances. However, for uε = (u1 , . . . , u2n ) y −y′

and vε = (v1 , . . . , v2n ) in ℝ2n such that u1 = vk+1 = 1 ε 1 and ui = vj = 0 otherwise. We get the result. Let now l be a non-Abelian subalgebra of p of dimension k spanned by {e1 , . . . , ek } such that l ∩ z = ℝe1 . Suppose first that k ≤ n + 1. For i = 1, 2, let ̃ (0, ỹi ), (0, 0), (0, B)) Mi = M((1, x), such that x,̃ ỹi ∈ ℝk−1 , ỹ1 ≠ ỹ2 and B B = ( 1) , B2

Ik−1 ), 0

B1 = (

− 21 K0 ). 0

B2 = (

So, M1 and M2 are in R(l, p, D) and [M1 ] ≠ [M2 ]. Moreover, for ε > 0, let ̃ (0, ỹi ), (0, Aε ), (0, B)) Miε = M((1 + ε, x), such that A1ε ) A2ε

Aε = (

εIk−1 ) and A2ε = 0. 0

with A1ε = (

It is clear that Miε are in R(l, p, D) and as rk(Aε ) = rk(B) = rk(Aε + B) = k − 1, [M1ε ] = [M2ε ], which is enough to conclude that the deformation space fails to be a Hausdorff space. Assume now that l is non-Abelian and n + 2 ≤ dim(l) ≤ 2n + 1 and take a basis {U, e2 , . . . , en , en+1 , . . . , ek } of l. We can and do assume that [ei , ej ] = 0 for all i, j ≥ n + 1. In these circumstances, the matrix K0 can be written in this basis as K K0 = ( t 1 − K2

K2 ), 0Mk−1−n (ℝ)

6.2 Stability of nilmanifold actions | 291

where K1 ∈ Mn (ℝ) is a skew symmetric matrix and K2 is in Mn,k−1−n (ℝ). Choose then two matrices ̃ (0, ỹi ), (0, A), (0, B)), Mi = M((1, x),

i = 1, 2,

such that x,̃ ỹi ∈ ℝk−1 , ỹ1 = (a, 0, . . . , 0), ỹ2 = (b, 0, . . . , 0), a ≠ b, B I B = ( 1 ) = ( k−1 ) B2 0 with I B1 = ( n ) 0

0 ), A2

and A = (

A2 ∈ Mn,k−1 (ℝ).

Let E = K0 − t B2 B1 + t B1 B2 . Then E E =( t1 − E2

E2

0Mn,k−1−n (ℝ)

),

where E1 ∈ Mn (ℝ) is a skew symmetric matrix and E2 is in Mn,k−1−n (ℝ). Then the homomorphism conditions (4.23) hold when t

A2 B1 − t B1 A2 = E.

Putting A2 = (A′ A′′ ) for A′ ∈ Mn (ℝ) and A′′ ∈ Mn,k−1−n (ℝ), the last equation holds when A′′ = E2 and A′ is an upper triangular matrix such that t A′ − A′ = E1 . So, M1 and M2 are in R(l, p, D) and as the first column of A vanishes, for all v ∈ ℝ2n such that vA ≠ ỹ1 − ỹ2 , that is, [M1 ] ≠ [M2 ]. Moreover, for ε > 0, let ̃ (0, ỹi ), (0, Aε ), (0, B)), Miε = M((1, x),

i = 1, 2,

where A1ε ) A2ε

Aε = (

and A1ε = (aij ) ∈ Mn,k−1 (ℝ)

such that an1 = ε and aij = 0 elsewhere. We can easily check as earlier that Miε are in R(l, p, D) and [M1ε ] = [M2ε ], which is enough to conclude that the deformation space fails to be a Hausdorff space. Remark 6.2.7. (1) In [39], S. Barmeier, studied the case where n = 1 and Γ is the discrete Heisenberg group where the parameter space is defined modulo the kernel G → Diff(G/H). He proved that in these circumstances the deformation space is homeomorphic to GL2 (ℝ) × ℝ× × ℝ3 .

292 | 6 Stability concepts and Calabi–Markus phenomenon (2) In the case of the Heisenberg group H2n+1 and H and Γ are arbitrary, Theorem 6.2.3 says that there is equivalence between the fact that T (Γ, H2n+1 ; H2n+1 /H) is a Hausdorff space and the fact that the stability holds. We remark that here it is no longer the case P = H2n+1 × H2n+1 . The reader can consult the reference [60] for more details.

6.2.3 Case of 2-step nilpotent Lie groups Coming back to our setting where G is nilpotent and 2-step, we refer to Section 4.2 for all of the materials. From the fact that the pair (G, H) have the Lipsman property (cf. Definition 2.1.10), if l is an ideal of g, then exp(l) acts properly on G/H if and only if l ∩ h = {0} (cf. Lemma 2.1.11). As a consequence of this observation, we get the following result concerning the case where l is a maximal Abelian subalgebra (as in Definition 4.1.37) of g. Proposition 6.2.8. If l is a maximal Abelian subalgebra of g, then the stability property holds. Proof. As l is maximal and Abelian, then φ(l) is also maximal and Abelian for any φ ∈ Hom0 (l, g). It follows that z ⊂ φ(l) for all φ ∈ Hom0 (l, g) and, therefore, 0

R (l, g, h) = {φ ∈ Hom (l, g) : Adg φ(l) ∩ h = {0}}

= {φ ∈ Hom0 (l, g) : φ(l) ∩ h = {0}},

which is an open set of Hom(l, g). As a direct consequence of Theorem 4.2.3, we get the following. Proposition 6.2.9. If dim z′ = dim[l, l], then the stability property holds. Proof. Put s = dim([l, l]), then we have A2 ∈ Ms (ℝ). Thus, the inequation rk(B4 ) < dim l′ implies A2 0

rk (

⋆ ) ≤ s + rk(B4 ) < dim l. B4

This means R2 is empty and R (l, g, h) = R1 , which is open in Hom(l, g). Remark 6.2.10. Note that when l is Abelian, the hypothesis of Proposition 6.2.9 holds if and only if z ⊂ h, which means in particular that h is an ideal of g. So the proposition is a consequence from a more general result, easily derived from Lemma 2.1.11: If h is an ideal of g, then the parameter space is open in Hom(l, g).

6.2 Stability of nilmanifold actions | 293

A stability theorem In the rest of this section, we give a new sufficient criterion of stability, which is useful when Γ is Abelian as we show in some examples. Definition 6.2.11. Let l, g and h be as above, the subalgebra l is said to satisfy (⋆) for g/h if there is a decomposition of h = (z∩h)⊕h′ ⊕h′′ and g = (z∩h)⊕z′ ⊕h′ ⊕h′′ ⊕V = h⊕z′ ⊕V. Here, h′′ and V are some subspaces of g such that: (⋆1)

(z ∩ h) ⊕ h′ is an ideal of g.

(⋆2)

rk (

B′′ 3 ) = rk(B4 ) B4

or

B4 = 0,

where B3 = (

B′3 ). B′′ 3

Example 6.2.12. Let g = ℝ-span{X, Y1 , . . . , Yn , Z1 , . . . , Zn } such that [X, Yi ] = Zi for 1 ≤ i ≤ n. For all r > 0 and t ≥ 0, let h = hr,t = ℝ-span{X, Y1 , . . . , Yr−t−1 , Z1 , . . . , Zr−1 } and l = lr,t,q = ℝ-span{Yr+t+q , . . . , Yn , Zr , . . . , Zn } with 0 ≤ q ≤ n − r − t. Then hr,t and lr,t,q are subalgebras of g and the subalgebras (z ∩ h) ⊕ h′ = ℝ-span{Y1 , . . . , Yr+t−1 , Z1 , . . . , Zr−1 } of h is an ideal of g and h′′ = ℝ-span{X}. B1 = (ui,j )1≤i≤r−1 , 1≤j≤k

B2 = (ui,j )r≤i≤n , 1≤j≤k

B′3 = (vi,j )1≤i≤r+t−1 , 1≤j≤k

B′′ 3 = (x1 , . . . , xk )

and B4 = (vi,j )r+t≤i≤n . Then 1≤j≤k

󵄨󵄨 (ui,j )1≤i≤n,1≤j≤k 󵄨󵄨 { } { } 󵄨 { { (vi,j )1≤i≤r+t−1,1≤j≤k 󵄨󵄨󵄨 xi vl,j − xj vl,i = 0, 1 ≤ l, i ≤ n} } 󵄨 Hom(l, g) = { ( ) 󵄨󵄨 } 󵄨 { } (xj )1≤j≤k 󵄨󵄨 and 1 ≤ j ≤ k { } { } 󵄨󵄨 󵄨󵄨 (vi,j )r+t≤i≤n,1≤j≤k { } = {φ(B) ∈ M2n+1,k (ℝ) : B′′ 3 = 0}

B′′ ∪ {φ(B) ∈ M2n+1,k (ℝ) : rk ( 3 ) ≤ 1} , B4 and l satisfies (⋆) for g/h. Example 6.2.13. Let g = ℝ-span{X1 , . . . , Xn , (Zi,j )1≤i 2, then for any φ ∈ R (Γ, G, H) we have that φ(Γ) ⊂ G0 . Moreover, since the Clifford–Klein form φ(Γ)\G/H is compact as mentioned above, we can deduce that h ⊄ g0 for dimension reasons, and then according to Proposition 6.2.19, the parameter space is stable and has a structure of a semialgebraic set. Suppose now that k = 2 and h ⊄ g0 , then q = 2 < n. It follows, in view of Proposition 4.4.9 that R (Γ, G, H) = R0,2 , which is stable according to Proposition 6.2.19 and semialgebraic. As for the case where k = 2 and h ⊂ g0 , we have φ(Γ) ⊄ G0 for any φ ∈ R (Γ, G, H) and we get that R0,2 = 0. One gets then R (Γ, G, H) = ∐3i=1 Ri,2 , which is stable according to Proposition 6.2.21, and on the other hand semialgebraic as being a finite union of semialgebraic sets.

6.2 Stability of nilmanifold actions | 303

We now study the situation where k = 1 and h ⊄ g0 . In this case, q = 1. Then by Proposition 6.2.19, the parameter space is stable and semialgebraic. Finally, if h ⊂ g0 , then R0,1 = 0 and R (Γ, G, H) = R1,1 , which is stable as shown in Proposition 6.2.21 and also semialgebraic. From the characterization of the parameter space given in Propositions 4.4.8 and 4.4.9, it is easy to check that in the cases where k ≥ 2 and h ⊄ g0 , the space T (Γ, G, H) is a Hausdorff space. The case 4 is also immediate. We now pay attention to the case when k = 2 and h ⊂ g0 . Let [M1 ], [M2 ] ∈ T (Γ, G, H) such that [M1 ] ≠ [M2 ]. We designate by xj xj′ yj yj′ Mj = (→ 󳨀 → 󳨀) 0 0 βj βj′

(j = 1, 2).

Provided that x ( j yj

xj′ xj ′ ) ≠ ( yj yj

xj′ ), yj′

we can obviously separate G ⋅ M1 and G ⋅ M2 by open neighborhoods. In addition, elements in R1,2 and R2,2 are immediately separated by disjoint open sets. We only have to treat the case where xj yj Mj = (→ 󳨀 0 zj

λxj λyj → 󳨀) 0 zj′

(j = 1, 2)

and x ( 1 y1

λx1 x )=( 2 λy1 y2

λx2 ) λy2

for λ ≠ 0. Remark that G ⋅ M1 ≠ G ⋅ M2 if and only if z1′ − λz1 ≠ z2′ − λz2 , and that xj { { { { yj G ⋅ Mi = {(→ 󳨀 { { { uj { v

λxj } } } } λyj t→ n−1 󳨀 ) : uj ∈ ℝ , v ∈ ℝ} . 󳨀󳨀→ } λuj } } ′ λv + zj − λzj }

When we project the orbit through the partial coordinates system (Yn , Yn ), we get the closed line (v, λv) + (0, z1′ − λz1 ) ≠ (v, λv) + (0, z2′ − λz2 ). So, one can obviously separate

304 | 6 Stability concepts and Calabi–Markus phenomenon the orbits in question at this level. Finally, we show that the deformation space is not a Hausdorff space in case 4. Indeed, let [M1 ], [M2 ] ∈ T0,1 such that [M1 ] ≠ [M2 ]. Noting 0 yj Mj = ( ) 0 → 󳨀 aj

(j = 1, 2),

(6.4)

where t aj = (aj,3 , . . . , aj,n ) ∈ ℝn−2 and we suppose that y1 = y2 . Let now Vj be an open neighborhood of [Mj ] for j = 1, 2. We get that π −1 (Vj ) is an open neighborhood of Mj in R (Γ, G, H). There exists then ε > 0 such that

Mjε

ε yj = ( ) ∈ π −1 (Vj ) ε → 󳨀 aj

for j = 1, 2, and on the other hand gε ∈ G such that gε ⋅ M1ε = M2ε , which conclusively shows that V1 ∩ V2 ≠ 0. This proves that the deformation space is not a Hausdorff space in this case. Finally, it is clear that two points [Mj ], j = 1, 2 in T (Γ, G, H), which are not separated belong to T0,1 and of the form (6.4) with y1 = y2 . Let φj be the Lie algebras homomorphisms associated to Mj , j = 1, 2. Now any g ∈ G is written as g = exp(xY1 ) ⋅ g ′ for some x ∈ ℝ and g ′ ∈ H as G is solvable and H is simply connected (cf. Theorem 1.1.6). Then clearly exp(ℤφ1 (T))gH = exp(ℤφ2 (T))gH, where exp(T) is any generator of Γ. This achieves the proof of the theorem. Remark 6.2.23. In case 4 of Theorem 6.2.22, the refined deformation space defined in Section 3.1.2 also fails to be a Hausdorff space. Indeed, thanks to Lemma 2.1.11 and the fact that H is normal, one easily sees that { {

x

}

×}

̂(Γ, G, H) = (y1 ) ∈ Mn+1,1 (ℝ) : x ∈ {0, 1}, y1 ∈ ℝ , R { } { } {

→ 󳨀 0

}

̂(Γ, G, H) trivially. endowed with the quotient topology. The result follows as G acts on R Smoothness of the deformation space It is an interesting question to know whether the deformation space can be endowed with a manifold structure in the case of compact Clifford–Klein forms provided that it is a Hausdorff space. In [98], it is shown that the deformation space contains a smooth manifold as its open dense subset. Using Theorem 6.2.22, we prove the following.

6.2 Stability of nilmanifold actions | 305

Theorem 6.2.24. Let G be a threadlike Lie group, H a closed connected subgroup of G, Γ an Abelian discontinuous group for the homogeneous space G/H such that the Clifford– Klein form Γ\G/H is compact. Then the associated deformation space contains a smooth manifold as its open dense subset, which is contained in the set of maximal-dimensional orbits. Proof. We make use of Theorem 6.2.22. Denote as before π : R (Γ, G, H) → T (Γ, G, H) the canonical projection, then the image of any open dense subset of R (Γ, G, H) is open and dense in T (Γ, G, H). When k > 2 (or k = 2 and H ⊄ G0 ), take R′ = {(

󳨀 t→

0 ) ∈ R0,k , N = (xi,j )1≤i≤n,1≤j≤k ; x1,1 ≠ 0} , N

which is open and dense in R (Γ, G, H). In addition, no point in R′ is fixed through the action of G. So all G-orbits of R′ are of maximal dimension. We note T ′ = π(R′ ) which, by Proposition 4.4.12, is homeomorphic to the set T0,k (1, 1). We now look at the situation where k = 2 and H ⊂ G0 . We prove that R′ = R3,2 is an open and dense subset in R (Γ, G, H). Let M ∈ R (Γ, G, H) \ R′ . So, we can find a sequence (Mi )i∈ℕ in R (Γ, G, H) \ R′ = R1,2 ∪ R2,2 , which converges to M. This leads immediately to the fact that M ∈ R1,2 ∪ R2,2 which shows that R (Γ, G, H) \ R′ is closed in R (Γ, G, H). We now prove that R′ is dense in R (Γ, G, H). Let 0 x → 󳨀 → M = ( 0 󳨀y ) ∈ R2,2 , z1 z2 there exists then n0 ∈ ℕ such that the sequence Mn =

x n 1→ ( n 󳨀y

z1

x → 󳨀y ) , z2

n > n0 ,

belongs to R′ and converges to M. Suppose now that x 0 󳨀 󳨀y → M = (→ 0 ) ∈ R1,2 , z1 z2 then obviously the sequence x 󳨀y Mn = (→ z1

x n 1→ 󳨀y ) , n z2 + zn1

n>0

306 | 6 Stability concepts and Calabi–Markus phenomenon belongs to R′ and converges to M. Finally, by Proposition 4.4.13 we have T ′ = π(R′ ) is homeomorphic to T3,2 and G-orbits of elements of R′ are of maximal dimension as reveals equation (4.70). This achieves the proof in this case. We now treat the case where k = 1. If H ⊂ G0 , then the result is immediate by Proposition 4.4.14. Otherwise, we note R′ = R1,1 and we argue similarly as previously to see that R′ is open and dense in R (Γ, G, H). Clearly, all G-orbits inside R′ are of maximal dimension. More significantly, the phenomenon of Hausdorffness of the deformation space is strongly linked to the feature of adjoint orbits of the basis group G on R (Γ, G, H), specifically to their dimensions. We have the following. Theorem 6.2.25. Let G be a threadlike Lie group, H a closed connected subgroup of G and Γ a discontinuous subgroup for G/H. If G acts on the parameter space R (Γ, G, H) with constant dimension orbits, then the deformation space T (Γ, G, H) is a Hausdorff space. When the Clifford–Klein form Γ\G/H is compact, this implication becomes an equivalence. Proof. When Γ is non-Abelian, we can immediately see that dim G ⋆ M = n for any M ∈ R (l, g, h) and Theorem 4.4.22 allows us to conclude. Suppose now that Γ is Abelian. Recall that in this situation we have an analogue decomposition of the parameter space as in the non-Abelian case, and accordingly the dimension of all orbits in the parameter space is constant if and only if k > 2. Proposition 4.4.12 is then sufficient to conclude. Suppose now that the Clifford–Klein form Γ\G/H is compact. Recall from Theorem 6.2.22 that if k = 1 and h ⊄ g0 , the deformation space is not a Hausdorff space and the dimension of the G-orbits is not constant. Except for this previous case, the deformation space is a Hausdorff space and the dimension of all G-orbits is constant. This achieves the proof of the theorem. Remark 6.2.26. (1) The deformation space is not connected, in general, in the nilpotent context. Though it is not clear that R (Γ, G, H) is a semialgebraic set in this case, it appears that it is so in most of the cases when restricted to the threadlike setting (or to the case of compact nilpotent Clifford–Klein forms). This means that the parameter space splits to a finite number of connected components with respect to the Euclidean topology, which gives in turn that the deformation space does, when endowed with the quotient topology. The number of the connected components is actually respected through the G-action. Looking at case 4 in Theorem 6.2.22 above for the simple setting when k = 1 and h ⊂ g0 , the deformation space T (Γ, G, H) turns out to be homeomorphic to T1,1 , which admits two connected components. In [98], it is shown that the parameter space R (Γ, G, H) of the action of ℤk on ℝk+1 consists of two disconnected semialgebraic sets of different dimensions, but topologically connected.

6.2 Stability of nilmanifold actions | 307

(2) The points in the deformation space, which cannot be separated correspond to a faithful translation action on G/H ≃ ℝ in the compact case. Beyond the compact case, this phenomenon fails to hold. Indeed, we can for instance consider the case where h = ℝ-span{X, Y3 , . . . , Yn } and Γ = exp(ℤY1 ).

6.2.5 Stability of discrete subgroups Stable discrete subgroups as in Definition 6.1.7 have important impact on local geometric and differential structures of the related parameter and deformation spaces as reveal many studied cases (cf. Chapter 4). For more details, the readers can consult the reference [29]. Our next upshot about stable discrete subgroups of threadlike Lie groups is as follows. Theorem 6.2.27. Let G be a threadlike Lie group and Γ a non-Abelian discrete subgroup of G. Then Γ is stable. In this case, for any H ∈ ⋔gp (Γ : G), the deformation parameter space R (Γ, G, H) is a semialgebraic smooth manifold of dimension n+k if k > 3 and n+4 otherwise. Proof. We take throughout the proof, a non-Abelian discrete subgroup Γ and let H be a fixed subgroup of G, which belongs to ⋔gp (Γ : G). We will show that the parameter space R (Γ, G, H) is open and semialgebraic. In light of Theorem 3.2.4, this space is homeomorphic to R (l, g, h) = {M ∈ U : rank(g ⋆ M ⋒ Mh,B ) = k + dim h for all g ∈ G},

where B = {X, Y1 , . . . , Yn } and the symbol ⋒ merely means the concatenation of matrices written through B . We also assume that 3 ≤ k ≤ n + 1 − dim h, otherwise R (l, g, h) is empty. We note for all j ∈ {0, . . . , k}, Rj,k = R (l, g, h) ∩ Kj,k . Then according to Proposition 4.4.17, we have that R (l, g, h) = R0,k whenever k > 3. Otherwise, 2 R (l, g, h) = ∐j=0 Rj,3 . We denote by IB = {i1 < ⋅ ⋅ ⋅ < iq } h

(q = dim h)

the set of indices i (1 ≤ i ≤ n + 1) such that h ∩ gi ≠ h ∩ gi−1 , where gi = {Yn , . . . , Yi }, i = 1, . . . , n, g0 = g, and gn+1 = {0}. We first prove the following. Proposition 6.2.28. We keep the same hypotheses and notation as before. Then: (i) If h ⊄ g0 , then h = ℝ-span{X + h1 Y1 + ⋅ ⋅ ⋅ + hn Yn } for some h1 , . . . , hn ∈ ℝ and R0,k = {M0 (U, V) ∈ K0,k : (u1 − h1 u) ∈ ℝ× }. h (ii) If h ⊂ g0 , then IB ⊂ {1, . . . , p − 1} and R0,k = K0,k .

(6.5)

308 | 6 Stability concepts and Calabi–Markus phenomenon ̃ for some X ̃ = X + h1 Y1 + Proof. Suppose first that h ⊄ g0 , then obviously h = ℝ-span{X} ⋅ ⋅ ⋅ + hn Yn where h1 , . . . , hn ∈ ℝ. Let M = M0 (U, V) ∈ K0,k . Then clearly, M ∈ R (l, g, h) ⇔ rank(M ⋒ g ⋆ Mh,B ) = k + 1 t

t

for all g ∈ G

⇔ rank⌊ (u, u1 , . . . , un ), (1, h1 , α2 , . . . , αn )⌋ = 2 ⇔ u1 − uh1 ∈ ℝ .

for all α2 , . . . , αn ∈ ℝ

×

h Suppose now that h ⊂ g0 . If IB ∩ {p, . . . , n} ≠ 0, then there exists i0 ∈ {p, . . . , n} and ̃ = Yi + hi +1 Yi +1 + ⋅ ⋅ ⋅ + hn Yn ∈ h for some hi +1 , . . . , hn ∈ ℝ, which is impossible Y 0 0 0 0 Ad (h)

as L acts on G/H properly. So, IB g ∩ {p, . . . , n} = 0 for all g ∈ G, which gives that rank(g ⋆ M ⋒ Mh,B ) = k + q for all M ∈ K0,k . We argue similarly as in the previous proposition to treat the case where k = 3. Proposition 6.2.29. We keep the same hypotheses and notation as before. Then: (i) If h ⊄ g0 , then h = ℝ-span{X + h1 Y1 + ⋅ ⋅ ⋅ + hn Yn }, for some h1 , . . . , hn ∈ ℝ, then R1,3 = {M1 (U, V, λ) ∈ K1,3 : (u1 − h1 u) ∈ ℝ× }

(6.6)

R2,3 = {M2 (U, V) ∈ K2,3 : (v1 − h1 v) ∈ ℝ× }.

(6.7)

and

h (ii) If h ⊂ g0 , then IB ⊂ {1, . . . , n − 2} and Rj,3 = Kj,3 , j = 0, 1, 2.

Now according to Propositions 6.2.28 and 6.2.29, the parameter space has a structure of a semialgebraic set. It remains therefore to show that it is open in Hom(l, g). As Hom0 (l, g) is an open set in Hom(l, g), M ∈ R (l, g, h) is stable if and only if there exists an open subset V of Hom0 (l, g), such that M ∈ V ⊂ R (l, g, h). In the case when k > 3, the result stems directly from Proposition 6.2.28 and equation (6.5). Assume now that k = 3. The result is immediate when h ⊂ g0 . Suppose now that h ⊄ g0 . It suffices then to see that Hom0 (l, g) \ R (l, g, h) = ∐2j=0 (Kj,3 \ Rj,3 ) is closed in Hom0 (l, g). Let then Hom0 (l,g)

M ∈ Hom0 (l, g) \ R (l, g, h)

,

there exists therefore a sequence (Mi )i∈ℕ assumed to belong to Hom0 (l, g) \ R (l, g, h), which converges to M. So we can extract from (Mi )i∈ℕ a subsequence (Mis )s∈ℕ of elements in Kj,3 \ Rj,3 for some j ∈ {0, 1, 2}. If Mis ∈ Kj,3 \ Rj,3 for j ∈ {0, 2}, then obviously its limit M belongs to Kj,3 \ Rj,3 .

6.3 Stability in the exponential setting

| 309

Suppose now that us us1 ( .. ( Mis = ( . (us n−2 usn−1 s ( un

λs us λs us1 .. . λs usn−2 s vn−1 vns

0 0 .. ) ) . ) ∈ K1,3 \ R1,3 ) 0 0 s s u (vn−1 − λs usn−1 ))

for some real sequence (λs )s∈ℕ . If (λs )s goes to infinity as s goes to +∞, then we can easily check that M ∈ K2,3 \ R2,3 . Otherwise, M ∈ (K0,3 \ R0,3 ) ∐(K1,3 \ R1,3 ). We get therefore that M ∈ Hom0 (l, g) \ R (l, g, h). The rest of the proof follows from Theorem 4.4.16. Corollary 6.2.30. Let Γ be a discrete subgroup and H ∈ ⋔gp (Γ : G). The parameter space

R (Γ, G, H) is endowed with a smooth manifold structure of dimension n + k whenever

k > 3. For k = 3, it is a disjoint union of an open, dense, smooth manifold of dimension n + 4 and a closed, smooth manifold of dimension n + 3.

̃ Proof. The result is immediate when k > 3. If k = 3, it is not hard to see that R 1,3 = ̃ ̃ R0,3 ∐ R1,3 and R2,3 are respectively open sets in K1,3 and K2,3 and that R1,3 is an open dense set of R (l, g, h).

6.3 Stability in the exponential setting We first look back at the case of connected, simply connected, nilpotent Lie groups. We prove the following result and provide in Remark 6.3.4 below a counterexample showing the failure of its validity in the general context of exponential Lie groups.

6.3.1 Case of general compact Clifford–Klein forms Proposition 6.3.1. Let G = exp g be a connected, simply connected, nilpotent Lie group, H = exp h a connected subgroup of G and Γ a discontinuous group for G/H of syndetic hull L = exp l. If the Clifford–Klein form Γ\G/H is compact, then the stability holds everywhere. That is, the parameter space R (l, g, h) is semialgebraic and open. Proof. The following lemma dealing with proper actions when the Clifford–Klein form is compact is proved in [135]. Lemma 6.3.2. Let G be a connected, simply connected, nilpotent Lie group, L and H be its connected, closed subgroups. If L\G/H is compact, then the following three conditions are equivalent: (i) The L-action on G/H is proper.

310 | 6 Stability concepts and Calabi–Markus phenomenon (ii) The L-action on G/H is free. (iii) L ∩ H = {0}. This result enables us to see that R (l, g, h) is homeomorphic to the set 󵄨󵄨 󵄨󵄨 dim ψ(l) = dim l and }, {ψ ∈ Hom(l, g) 󵄨󵄨󵄨 󵄨󵄨 ψ(l) ∩ h = {0} 󵄨 which is open in Hom(l, g) and semialgebraic. We now find back our settings. G designates an exponential solvable Lie group, H a nonnormal connected maximal subgroup of G and Γ a discontinuous group for G/H. The most important problem in the study of the deformation space of discontinuous groups is the description of the parameter space R (Γ, G, H) as in equation (3.1), which is homeomorphic to R (l, g, h) as in Theorem 3.2.4. Our main result in this section is the following. Theorem 6.3.3. Let G be an exponential Lie group, H a nonnormal, connected maximal subgroup of G and Γ a discontinuous group for G/H. Then the parameter space R (Γ, G, H) is semialgebraic. Furthermore, the stability property holds if and only if Γ is of rank two. Proof. We make use of Propositions 5.3.6 and 5.3.7 concerning the determination of the parameter space R (l, g, h), which are the key points in the proof. Clearly, R (l, g, h) is semialgebraic. For k = 1, the parameter space is a smooth manifold of dimension n − 1 as reveals Proposition 5.3.6, so no stable points can be found. For k = 2, the parameter space is the intersection of Hom(l, g) and the Zariski open set defined by a1 { { { { x1 R = {( { y1 { { → 󳨀 { z1

a2 } } } } x2 ) ∈ Mn,2 (ℝ), x1 y2 − x2 y1 ≠ 0} , } y2 } } → 󳨀 z2 }

which is enough to conclude. Remark 6.3.4. We now provide a counterexample to the result of Proposition 6.3.1 in the context of exponential Lie groups. Remark that the first example in Lemma 5.3.8 2 produces a parameter space R1,1 homeomorphic to ℝ× , which is not open in 2 Hom(l, g) ≃ ℝ . This case corresponds indeed to a situation where the Clifford–Klein form in question is compact.

6.4 Stability for Euclidean motion groups | 311

6.4 Stability for Euclidean motion groups We first prove some elementary results, which will be of use throughout the section. The following is a first observation in linear algebra; it states that there are at most finitely many On (ℝ)-conjugacy classes [X] for which X n = I for fixed N ∈ ℕ∗ . Fact 6.4.1. Fix N ∈ ℕ∗ . The set of solutions of the equation (E ) : X N = I in On (ℝ) coincides with the set SN =

⨆ {S−1 Ai S | S ∈ On (ℝ)},

1≤i≤MN

where MN ∈ ℕ∗ and for 1 ≤ i ≤ MN , ANi = I and Ai and Aj belong to different classes of conjugacy for 1 ≤ i ≠ j ≤ MN . Corollary 6.4.2. Fix N ∈ ℕ∗ and let {Ap }p∈ℕ be a sequence of orthogonal matrices satisfying for any p ∈ ℕ, ANp = I. If {Ap }p∈ℕ converges to an orthogonal matrix A, say, there exist p0 ∈ ℕ such that for any p ≥ p0 , Ap = Sp−1 ASp for some Sp ∈ On (ℝ). Proof. Thanks to Fact 6.4.1, {Ap }p∈ℕ ⊂ SN . We remark that there exists j ∈ {1, . . . , MN } such that the orbit Θj = {S−1 Bj S | S ∈ On (ℝ)}, being a closed subset of On (ℝ), contains an infinite term of the sequence {Ap }p∈ℕ . Then there exists p0 ∈ ℕ such that for any p ≥ p0 , Ap and A are sitting inside Θj . This justifies the statement. We now recall a classical structural result for finitely generated Abelian groups. Fact 6.4.3 ([2, Theorem 7.22]). Any finitely generated Abelian group M is isomorphic to ℤr ⊕ M1 where |M1 | < ∞. The integer r is an invariant of M. Any finite Abelian group is a direct sum of cyclic groups of prime power order and these prime power orders, counted with multiplicity, completely characterize the finite Abelian group up to isomorphism. Also, any finite Abelian group is uniquely isomorphic to a group ℤ/(s1 ) × ⋅ ⋅ ⋅ × ℤ/(sm ) where si divides si+1 for any i ∈ {1, . . . , m − 1}. As a direct consequence, we get the following. Proposition 6.4.4. Let L be a discrete subgroup of ℝn . For any subgroup L ’ of L , there exists a unique sequence of positive integers s1 , . . . , sm satisfying si divides si+1 for any 1 ≤ i ≤ m − 1, and such that k−k ′

L /L ’ ≃ ℤ

× ℤ/(s1 ) × ⋅ ⋅ ⋅ × ℤ/(sm ),

where si divides si+1 for any i ∈ {1, . . . , m − 1} and k, k ′ designate the rank of L and L ’, respectively. Let us mention that for a given group K with an Abelian normal and finitely generated subgroup Ka of finite index in K, there exists by Fact 6.4.3 an integer k and a

312 | 6 Stability concepts and Calabi–Markus phenomenon unique sequence of positive integers s1 , . . . , sm satisfying si divides si+1 for any 1 ≤ i ≤ m − 1, and such that Ka ≃ ℤk × ℤ/(s1 ) × ⋅ ⋅ ⋅ × ℤ/(sm ).

(6.8)

We easily check that the integer k is an invariant of K. Definition 6.4.5. Let K be a group with an Abelian normal and finitely generated subgroup Ka of finite index in K. The integer k given in equation (6.8) is said to be the effective rank of K and denoted by re (K). We now come back to our group G. For a subgroup K of G, set n

FK := {PA (x) : (A, x) ∈ K} ⊂ ℝ .

(6.9)

We have the following. Lemma 6.4.6. For any discrete subgroup Γ of G, the rank of the family FΓ coincides with the effective rank of Γ. Proof. Remark first that for any g = (Sg , tg ) ∈ G, Γg = {(Sg−1 ASg , Sg−1 [x + (A − I)tg ]) : (A, x) ∈ Γ}. As for any (A, x) ∈ Γ, PSg−1 ASg (Sg−1 [x + (A − I)tg ]) = Sg−1 PA Sg (Sg−1 [x + (A − I)tg ]) = Sg−1 PA (x) + Sg−1 P A (A − I)(tg ), ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ 0

we get FΓg = Sg−1 FΓ . Take g as described in Theorem 1.2.24 and keep the same notation. Obviously, the rank of FΓg is exactly the rank of the family {yk,i }1≤i≤k . Since {γi }1≤i≤k0 generates an Abelian normal subgroup Γga of finite index in Γg , satisfying ⟨γ1 , . . . , γk ⟩ ≃ ℤk and ⟨γk+1 , . . . , γk0 ⟩ is finite, we get re (Γ) = k. The following upshot, will be of use later. Proposition 6.4.7. Let Γ be a discrete subgroup of G and KΓ the set of elements of Γ of finite order. Then pr1 (KΓ ) is finite. Proof. We keep the same notation as in Theorem 1.2.24. Let Γ0 = {a0 , . . . , aN−1 } as a finite set. Any γ ∈ Γ is written as γ = bγ aiγ δjγ where bγ ∈ Γ∞ , iγ ∈ {0, . . . , N − 1} and jγ ∈ {0, . . . , q − 1} with a0 = γ0 = e. Let γ ∈ KΓ , then γ q ∈ Γa and, therefore, γ q ∈ Γ0 , which entails that γ qN = e. Assume that KΓ is infinite. Then KΓ =

⋃

0≤i≤N−1 0≤j≤q−1

Ki,j ,

6.4 Stability for Euclidean motion groups | 313

where Ki,j = {γ ∈ KΓ : γ = bγ ai δj }. Then any γ ∈ Ki,j can be written as γ = (Aγ , xγ ) = (Bγ , yγ )(Ci , 0)(Sj , zj ) = (Bγ Ci Sj , yγ + Bγ Ci zj ), where ai = (Ci , 0), bγ = (Bγ , yγ ) and δj = (Sj , zj ); hence CiN = I and Bγ Ci zj = zj . Assume that for a given j ∈ {0, . . . , q − 1} and i ∈ {0, . . . , N − 1}, pr1 (Ki,j ) is infinite. Then one can find an infinite sequence {Ap }p∈ℕ of p1 (Ki,j ) converging to some A ∈ On (ℝ). Then for p ∈ ℕ, we can write Ap = Bp Ci Sj . Besides, let (δj(s,γ) )s∈ℕ be the sequence such that bγ ai δj = δj(1,γ) bγ ai

and bγ ai δj(s,γ) = δj(s+1,γ) bγ ai .

This gives likewise a sequence (Sj(s,p) )s∈ℕ such that Bp Ci Sj = Sj(1,p) Bp Ci

and

Bp Ci Sj(s,p) = Sj(s+1,p) Bp Ci .

As the sequence {Bp }p∈ℕ converges, the sequences {Sj(s,p) }p∈ℕ also do and as Sj(s,p) is of finite values, there exists p′′ ≥ p′ such that for any p ≥ p′′ , Sj(s,p) = Sj

(s,p′′ )

we deduce that (Bp Ci Sj )qN = I and then

(Bp )qN ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ (Ci )qN Sj I

(qN−1,p′′ )

⋅ ⋅ ⋅ Sj

(1,p′′ )

. As AqN p = I,

Sj = I.

This gives that (Bp )qN = (Sj ⋅ ⋅ ⋅ Sj ′′ Sj )−1 , which does not depend upon p. Fur(qN−1,p′′ ) (1,p ) thermore, the matrix form of Bp reads Im+ ( ( Bp = ( (

εp Im−

dp (−1, 1)

r(θ1 (p))

(

..

.

) ) ), ) r(θl (p)))

where εp ∈ {−1, 1} and dp (−1, 1) is a diagonal matrix of diagonal values in {−1, 1}. Since (Bp )qN does not depend upon p, the sequence {Bp }p∈ℕ turns out to be stationary and, therefore, {Ap }p∈ℕ must be finite, which is absurd and, therefore, pr1 (Ki,j ) cannot be infinite. Thanks to Proposition 6.4.7, one can write KΓ = ⨆ Ki , 1≤i≤MΓ

where Ki = {(Ai , x) ∈ KΓ } and MΓ designates the cardinality of pr1 (KΓ ). Fix i ∈ {1, . . . , MΓ } and (Ai , x0 ) ∈ Ki . Remark that Ki (Ai , x0 )−1 = {(I, x − x0 ), x ∈ pr2 (Ki )} := Λi

314 | 6 Stability concepts and Calabi–Markus phenomenon is a discrete subgroup because x ∈ ker(Ai − I)⊥ for any (Ai , x) ∈ Ki . This entails that Fi := pr2 (Λi ) is a discrete subgroup of ker(Ai − I)⊥ . Let Λi act by conjugation on the set Ki . We finally prove the following. Proposition 6.4.8. Ki /Λi is finite. Proof. Since Ai − I induces an isomorphism on ker(Ai − I)⊥ , its restriction on Fi is injective. Furthermore, (Ai , x)2 (Ai , x0 )−1 = (Ai , x + Ai (x − x0 )) ∈ Ki , which entails that Ai (x − x0 ) ∈ Fi and also (Ai − I)(x − x0 ) ∈ Fi . Then (Ai − I)(Fi ) ⊂ Fi . Obviously, (Ai − I)(Fi ) and Fi are of the same rank. Thanks to Proposition 6.4.4 there exist s1 , . . . , sm ∈ ℤ satisfying si divides si+1 for any 1 ≤ i ≤ m − 1, and such that Fi /(Ai − I)(Fi ) ≃ ℤ/(s1 ) × ⋅ ⋅ ⋅ × ℤ/(sm ).

Hence Fi /(Ai − I)(Fi ) is a finite group of cardinality s = ∏m j=1 sj . Denote Fi /(Ai − I)(Fi ) = {0, x1 − x0 , . . . , xs−1 − x0 }.

Fix (Ai , x) ∈ Ki , then x − x0 ∈ Fi , and so there exists y ∈ Fi , j ∈ {0, . . . , s − 1} such that x −x0 = xj −x0 +(Ai −I)(y), which is in turn equivalent to the fact that x = xj +(Ai −I)(y), and finally (Ai , x) = (I, −y)(Ai , xj )(I, y). At the end, Ki = ⨆0≤j≤d−1 Θi,j where Θi,j is the orbit of (Ai , xj ) under the action of Λi by conjugation.

6.4.1 Geometric stability We now study the above variants of stability in the context of Euclidean motion groups. As an immediate and important consequence of Proposition 2.5.1, we get the following description of the parameter space of the action of any discontinuous group acting on a homogeneous space G/H, where G stands for the Euclidean motion group. Corollary 6.4.9. Let Γ be a discrete subgroup of the Euclidean motion group G acting properly on G/H, then 0

R (Γ, G, H) = {φ ∈ Homd (Γ, G) : φ(Γ) acts freely on G/H}.

One immediate aftermath concerning geometric properties of Clifford–Klein forms is when the fundamental group is torsion-free for which we have the following.

6.4 Stability for Euclidean motion groups | 315

Proposition 6.4.10. R (Γ, G, H) and Hom0d (Γ, G) coincide whenever Γ is torsion-free. Proof. It suffices to remark that thanks to Proposition 2.5.1, H is compact whenever Γ is infinite. As Γ is torsion-free, φ(Γ)∩gHg −1 is trivial for any g ∈ G and any φ ∈ Hom0d (Γ, G), and then Corollary 6.4.9 completes the proof. The following is then an immediate consequence of Remarks 6.1.4 and Proposition 2.5.1. Proposition 6.4.11. Let G be the Euclidean motion group, H a closed subgroup of G and Γ a discontinuous group for G/H. Then any point in R (Γ, G, H) is geometrically stable. 6.4.2 Near stability Our main result in this section is to characterize the set of nearly stable points in the context of Euclidean motion groups. We will prove the following. Theorem 6.4.12. Let G be the Euclidean motion group, H a closed subgroup of G and Γ a discontinuous group for G/H. Then any point in R (Γ, G, H) is nearly stable. That is, R (Γ, G, H) is an open set of Hom0d (Γ, G). Proof. Thanks to Theorem 5.7.4, the parameter space R (Γ, G, H) is a finite union of open G-orbits whenever Γ is finite, which proves the statement in this case. Assume then that Γ is infinite, then H is compact by Proposition 2.5.1. Thanks to Fact 6.1.6, we assume right away that Γ is not torsion-free. Let {φp }p∈ℕ be a sequence of Hom0d (Γ, G) converging to φ ∈ Hom0d (Γ, G) for which φp (Γ) does not act freely on G/H. There exists then γp ∈ Γ\{e} such that φp (γp ) ∈ gp−1 Hgp \{e} for some gp ∈ G. As φp is injective, φp (γp ) and γp have the same finite order. Then γp ∈ KΓ , and since KΓ = ⨆ Ki , 1≤i≤MΓ

one can find i ∈ {1, . . . , MΓ } and a subsequence {γα(p) }p∈ℕ in such a way that γα(p) ∈ Ki for any α(p). Furthermore, thanks to Proposition 6.4.8, Ki =

⨆ Θi,j

0≤j≤d−1

and we can therefore assume that there exists j ∈ {0, . . . , d − 1} such that γα(p) ∈ Θi,j . Hence γα(p) = (I, −yα(p) )(Ai , xj )(I, yα(p) ). Define Bα(p) ∈ On (ℝ), zα(p) ∈ ℝn for p ∈ ℕ by the identity (Bα(p) , zα(p) ) = φα(p) (γα(p) ). Thanks to Corollary 6.4.2, we can extract a convergent subsequence {Bβ(p) }p∈ℕ sitting

316 | 6 Stability concepts and Calabi–Markus phenomenon −1 in the orbit of a matrix B ≠ I, say, in such a way that (Bβ(p) , zβ(p) ) ∈ gβ(p) Hgβ(p) \{e} for some γβ(p) ∈ G. This gives in turn that

φβ(p) ((Ai , xj )) = φβ(p) ((I, yβ(p) ))(Bβ(p) , zβ(p) )φβ(p) ((I, yβ(p) ))

−1

′ =: (Bβ(p) , zβ(p) ).

′ ′ Finally, φβ(p) ((Ai , xj )) ∈ (g ′ β(p) )−1 Hgβ(p) for gβ(p) = gβ(p) φβ(p) ((I, yβ(p) ))−1 . Furthermore, the convergence of the sequence (φβ(p) ((Ai , xj )))p to φ((Ai , xj )) entails the convergence ′ of ((Bβ(p) , zβ(p) ))p to some (B, z). Thanks to Corollary 6.4.2, Bβ(p) = Sp BSp−1 for a given sequence {Sp }p∈ℕ ⊂ On (ℝ). We can hence assume that {Sp }p∈ℕ converges to a matrix S and then B ∈ S−1 pr1 (H)S. Thanks to Lemma 1.2.6, there exists g ∈ G such that (B, z) ∈ g −1 Hg, which gives that φ ∈ ̸ R (Γ, G, H) and the proof is complete.

6.4.3 Case of crystallographic discontinuous groups We study in this section the set of stable points. This simply reduces to describe the set of interior points of Hom0d (Γ, G) inside Hom(Γ, G). We first prove some preliminary useful results, which will be fundamental ingredients to reach our objectives. Let us start with the following. Proposition 6.4.13. Let Γ be a discrete subgroup of G and Γa a finitely generated Abelian normal subgroup of finite index in Γ. Take any φ ∈ Hom0d (Γ, G). If φ|Γa is an interior point of Hom0d (Γa , G) in Hom(Γa , G), then φ is an interior point of Hom0d (Γ, G) in Hom(Γ, G). Proof. We first prove the following. Lemma 6.4.14. Assume the same setting of Proposition 6.4.13. For φ ∈ Hom(Γ, G), the condition φ|Γa ∈ Homd (Γa , G) implies φ ∈ Homd (Γ, G). Let φ ∈ Hom0d (Γ, G) be such that φ|Γa is an interior point of Hom0d (Γa , G) in Hom(Γa , G), and φ is not an interior point of Hom0d (Γ, G) in Hom(Γ, G). Take a sequence {φp }p∈ℕ of Hom(Γ, G) converging to φ such that φp ∈ ̸ Hom0d (Γ, G). Since {φp|Γa }p∈ℕ converges to φ|Γa in Hom(Γa , G), there exists p0 ∈ ℕ such that for any p ≥ p0 , φp|Γa ∈ Hom0d (Γa , G). By Lemma 6.4.14, for any p ≥ p0 , φp ∈ Homd (Γ, G). This

leads us to take a sequence {gp }p≥p0 in Γ\{e} such as φp (gp ) = e. Take δ0 , δ1 , . . . , δq−1 for the complete representative of the finite set Γ/Γa , where q = ♯Γ/Γa . Then we can take αp ∈ Γa such that gp = αp δjp , where jp ∈ {0, 1, . . . , q − 1}, p ≥ p0 . As φp|Γa is injective, then δjp ∈ ̸ Γa . Hence φp (αp ) = φp (δj−1 ), which entails that φp (αpq ) = φp (δj−q ). Therefore, p p

αpq = (δj−q ). p

6.4 Stability for Euclidean motion groups | 317

This means that the set {αpq }p≥p0 is finite. Therefore, the set {αp }p≥p0 is also finite because Γa is finitely generated Abelian group. Then the set {gp }p≥p0 is finite in Γ. This entails that there exists g̃ ∈ {gp }p≥p0 such that for p large enough, φp (g̃ ) = e, and then φ(g̃ ) = e, which is impossible. This completes the proof. Lemma 6.4.15. Let {γp }p∈ℕ and {γp′ }p∈ℕ be two sequences of SO2 (ℝ) ⋉ ℝ2 converging to (I2 , y) and (I2 , y′ ), respectively. Assume that both {γp }p∈ℕ and {γp′ }p∈ℕ are not translations of ℝ2 and commute. Then y and y′ are linearly dependent. Proof. Consider the following equation: 0 1 dr lim (r(θ) − I2 ) = (0) = ( 1 θ dθ

θ→0

−1 ). 0

(6.10)

Suppose that γp = (r(θp ), yp ) and γp′ = (r(θp′ ), yp′ ). The commutativity condition implies (r(θp ) − I2 )yp′ = (r(θp′ ) − I2 )yp .

(6.11)

Since γp and γp′ are not translations, then for any p ∈ ℕ, θp , θp′ ≠ 0. Then we have 1 1 (r(θp ) − I2 )yp′ = ρp ′ (r(θp′ ) − I2 )yp , θp θp where ρp =

θp′ . θp

By the convergence of {yp′ }p∈ℕ to y′ , {yp }p∈ℕ to y and equation (6.10),

{ρp }p∈ℕ converges to some real number ρ and we obtain y′ = ρy.

Remark 6.4.16. The last lemma allows us to see that the converse of Proposition 6.4.13 is not true in general, as reveals the following example. Example 6.4.17. Set G = I(3) and Γ the discrete subgroup generated by e γ = (I3 , ( 1 )) 0 and r( 2π ) δ = (( 3

0 ) , ( 2 )) , 0 1

where e1 = ( 01 ). The subgroup Γa generated by γ and )e1 r( 2π 3 )) 0

γ ′ = (I3 , (

is an Abelian normal subgroup of index 3 in Γ and such that Hom0d (Γa , G) is not open in Hom(Γa , G), but Hom0d (Γ, G) is open in Hom(Γ, G). Indeed, for any positive integer p,

318 | 6 Stability concepts and Calabi–Markus phenomenon we define φp ∈ Hom(Γa , G) as follows: φp (γ) = γ

1 and φp (γ ′ ) = ((

−1 2 1 ) , ( √3 )) . r( p ) e 2 1

Then {φp } converges to idΓa and φp ∈ ̸ Hom0d (Γa , G); otherwise, re (φp (Γa )) = 1 ≠ re (Γ) = 2, which is absurd. Furthermore, let {ψp }p∈ℕ be a sequence of homomorphisms, which converges to some ψ ∈ Hom0d (Γ, G) and such that for any p ∈ ℕ, ψp ∈ ̸ Hom0d (Γ, G). Due to Corollary 6.4.2, for p large enough, there exists Sp ∈ O3 (ℝ) such that (Sp , 03 )−1 ψp (δ)(Sp , 03 ) = ((

r( mπ ) 3

y (p) ) , ( 2 )) , 0 1

where m ∈ {2, 4} and thanks to Lemma 1.2.1, (Sp , 03 )−1 ψp (γ)(Sp , 03 ) = ((

r(θ(p))

ε1

x2 (p) )) αp

),(

and r(θ′ (p))

x ′ (p) ) , ( 2 )) , ε2 βp

(Sp , 03 )−1 ψp (γ ′ )(Sp , 03 ) = ((

where εi ∈ {−1, 1}. We can assume that the sequence {(Sp , 03 )}p∈ℕ converges to some g ∈ G. By Lemma 6.4.15, I g −1 ψ(γ)g = (( 2

ε1

x ) , ( 2 )) α

and I g −1 ψ(γ ′ )g = (( 2

ε2

ρx2 )) . β

),(

It happens therefore that the subgroup of ℝ2 generated by x2 and ρx2 is not stable by the subgroup generated by r( mπ ), which contradicts Theorem 1.2.24. Hence for p large 3 enough, ψp ∈ Hom0d (Γ, G) and then Hom0d (Γ, G) is open in Hom(Γ, G). We now prove the following the main result. Theorem 6.4.18. Let Γ be a discontinuous crystallographic group for a homogenous space G/H, then any φ ∈ R (Γ, G, H) is stable. That is, R (Γ, G, H) = Hom0d (Γ, G) is open in Hom(Γ, G). Proof. Let Γ be a crystallographic subgroup of G acting discontinuously on G/H, then H is compact due to Proposition 2.5.1. Let ψ ∈ Hom0d (Γ, G) such that ψ(Γ) does not act

6.4 Stability for Euclidean motion groups | 319

freely on G/H. There exists therefore γ = (Cγ , xγ ) ∈ Γ of finite order such that ψ(γ) ∈ sHs−1 for some s = (S, ts ) ∈ G. Since ψ(γ) = gγg −1 for some g = (A, t) ∈ GLn (ℝ) ⋉ ℝn by Lemma 1.2.20, we get gγg −1 = shs−1 for some h = (Ch , th ) ∈ H. This implies that Cγ and Ch being of finite order, are conjugate in GLn (ℝ) and then conjugate in On (ℝ). Thanks to Lemma 1.2.6, γ and h are conjugate in G, which is absurd. Therefore, R (Γ, G, H) = Hom0d (Γ, G). On the other hand, Γ contains an Abelian normal subgroup Γa , say, which is generated by n free translations and of finite index in Γ. Hence Γa admits generators {γ1 , . . . , γn } such that for any i ∈ {1, . . . , n}, γi = (I, xi ) where the family {x1 , . . . , xn } is of rank n. For any φ ∈ Hom0d (Γ, G), φ|Γa is an isomorphism from Γa into φ(Γa ), by Lemma 1.2.20, there exists gφ := (Aφ , tφ ) ∈ GLn (ℝ) ⋉ ℝn such that φ(γi ) = (I, Aφ xi )(1 ≤ i ≤ n). Let {φp }p∈ℕ be a sequence of homomorphisms, which converges to φ. For any i ∈ {1, . . . , n} and any p ∈ ℕ, set φp (γi ) = (Ai (p), xi (p)) ∈ G. Making use of Fact 1.2.1, there exists gp := (Sp , 0) ∈ G for which ϕp := gp−1 φp gp is such that for any i ∈ {1, . . . , n}, Di (p) ϕp (γi ) = ((

r(θ1,i (p))

..

.

yi′ (p) x1,i (p) ),( )) , .. . r(θl(p),i (p)) xl(p),i (p)

(6.12)

where for any i ∈ {1, . . . , n}, Di (p) is a diagonal matrix of Om(p) (ℝ) where m(p) = n−2l(p), and where yi′ (p) x1,i (p) ( ) = Sp−1 xi (p) .. . xl(p),i (p) for some xs,i (p) ∈ ℝ2 and yi′ (p) ∈ ℝm(p) . Since for any i ∈ {1, . . . , n}, the sequence {Ai (p)}p∈ℕ converges to I, then for p large enough Di (p) = Im(p) . On the other hand, we can find a subsequence {ϕα(p) }p∈ℕ such that l(α(p)) = l. If for instance l ≥ 1, then equation (6.12) reads Im ϕα(p) (γi ) = ((

r(θ1,i (α(p)))

..

.

yi′ (α(p)) x1,i (α(p)) ),( )) . .. . r(θl,i (α(p))) xl,i (α(p))

(6.13)

For any i ∈ {1, . . . , n} and any s ∈ {1, . . . , l}, the sequence {r(θs,i (α(p)))}p∈ℕ tends to I2 . If there exists i0 ∈ {1, . . . , n} and s0 ∈ {1, . . . , l} such that the set Ji0 ,s0 := {p ∈ ℕ |

320 | 6 Stability concepts and Calabi–Markus phenomenon r(θs0 ,i0 (α(p))) ≠ I2 } is infinite, then by Lemma 6.4.15, the sequence {ϕα(p) }p∈ℕ converges to ϕ such that Im ϕ(γi ) = ((

I2

..

.

yi′ x1,i ) , ( . )) , .. I2 xl,i

(6.14)

and for which the family {xs0 ,i }1≤i≤n is of rank 1. Therefore, the family yi } { } { } { { } { x1,i } ) ( . } { .. } { } { } { } { x }1≤i≤n { l,i ′

is of rank less or than equal to n − 1, which is absurd. This leads us to conclude that for sufficiently large p, l = 0. Equivalently, for any i ∈ {1, . . . , n} Ai (p) = I, and hence φp (γi ) = (I, xi (p)) for p large enough. Now the convergence of the sequence {φp }p∈ℕ to φ is equivalent to the convergence of the sequence of matrices Mp , say, of column xi (p) (1 ≤ i ≤ n) to the invertible matrix M(φ), say, of column Aφ xi (1 ≤ i ≤ n). As GLn (ℝ) is an open set of Mn (ℝ), there exists p0 such that for p ≥ p0 Mp is invertible, then the family {xi (p)}1≤i≤n is of rank n, and thus φp|Γa ∈ Hom0d (Γa , G). This justifies that φ|Γa is an interior point of Hom0d (Γa , G) in Hom(Γa , G). By Proposition 6.4.13, we get that Hom0d (Γ, G) is open in Hom(Γ, G) and this completes the proof of the theorem. 6.4.4 Further remarks I. In the setting where G = On (ℝ) ⋉ ℝn , the geometric stability is evidently verified. Indeed whenever Γ acts properly on G/H, the pair (Γ, H) is such that H is compact or Γ is finite, and hence any φ ∈ Hom(Γ, G) is such that φ(Γ) acts properly on G/H. When Γ is finite, the parameter space R (Γ, G, H) turns out to be a finite union of G-orbits for which the corresponding subgroups act fixed point freely on G/H as shown in [13]. Then R (Γ, G, H) is open in Hom(Γ, G) (thus any homomorphism is stable). The following example shows that the parameter space R (Γ, G, H) may be open in Hom(Γ, G), but Hom0d (Γ, G) fails in general to be. Example 6.4.19. Let Γ be the discrete subgroup of I(5) generated by 1 γ1 = ((

I2

1 ) , (02 )) I2 02

and

1

γ2 = ((

r( 2π ) 3

0 ) , (02 )) r( 2π ) 02 3

6.4 Stability for Euclidean motion groups | 321

and let I H = {(( 3

󵄨󵄨 󵄨 ) , 05 ) 󵄨󵄨󵄨 A ∈ SO2 (ℝ)} . 󵄨󵄨 A

Notice that Hom0d (Γ, I(5)) is not open in Hom(Γ, I(5)). Indeed, for any p ∈ ℕ∗ define the homomorphism φp by r( p1 ) φp (γ1 ) = ((

e1 ) , ( 0 )) 03 I2

1

and 1 φp (γ2 ) = ((

I2

0 ) , (02 )) , r( 2π ) 02 3

where 1 e1 = ( ) . 0 If φp ∈ Hom0d (Γ, I(5)), then re (φ(Γ)) = 0, which is absurd. We easily check that the limit φ of the sequence {φp }p∈ℕ satisfies φ ∈ Hom0d (Γ, I(5)). On the other hand, R (Γ, I(5), H) is open in Hom(Γ, I(5)). In fact, any homomorphism ψ ∈ R (Γ, I(5), H) is such that ψ(γ2 ) ∈ ̸ g −1 Hg for any g ∈ G, and thus there exists gψ ∈ G such that 1 { { ̃ ψ(γ2 ) ∈ { (( { {

r( 2iπ ) 3

󵄨󵄨 0 󵄨󵄨 } } 󵄨󵄨 ) , (02 )) 󵄨󵄨󵄨 i, j ∈ {1, 2}} , 󵄨 } 󵄨󵄨 02 r( 2jπ ) 󵄨󵄨 } 3

̃ := g −1 ψg . Let {ψ } where ψ ψ p p∈ℕ be a sequence of homomorphism, which converges to ψ ̃ (γ ) = ψ(γ ̃ ), thanks to Corollary 6.4.2. some ψ ∈ R (Γ, I(5), H). For p large enough ψ p

Then

1 ̃ ψp (γ1 ) = ((

r(θp )

2

2

αp ) , (02 )) . r(θp′ ) 02

Necessarily, {αp }p∈ℕ converges to some α ∈ ℝ∗ ; otherwise re (ψ(Γ)) = 0, which is absurd. Therefore, for sufficiently large p, αp ≠ 0 and then ψp ∈ R (Γ, I(5), H). For an infinite discrete subgroup Γ of G, the following example shows that the parameter space may admits both stable and nonstable points.

322 | 6 Stability concepts and Calabi–Markus phenomenon Example 6.4.20. Let Γ be the discrete subgroup of I(5) generated by 1 γ1 = ((

1 ) , (02 )) I2 02

I2

1

and γ2 = ((

I2

0 ) , (02 )) r( 2π ) 02 3

and let 1 { { H = { (( { {

A

󵄨󵄨 󵄨󵄨 } } 󵄨 ) , 05 ) 󵄨󵄨󵄨󵄨 A ∈ SO2 (ℝ)} . } 󵄨󵄨 󵄨󵄨 A }

Notice that idΓ is not stable. Indeed, for any p ∈ ℕ∗ define the homomorphism φp by r( p1 ) φp (γ1 ) = ((

1

e1 ) , ( 0 )) 03 I2

and 1 φp (γ2 ) = ((

0 ) , (02 )) , r( 2π ) 02 3

I2

where 1 e1 = ( ) . 0 If φp ∈ Hom0d (Γ, I(5)), then re (φ(Γ)) = 0, which is absurd. We easily check that the sequence (φp )p∈ℕ converges to idΓ , and thus idΓ is not stable point of R (Γ, I(5), H). On the other hand, let φ be the homomorphism such that φ(γ1 ) = γ1

1

and φ(γ2 ) = ((

) r( 4π 3

0 ) , (02 )) . r( 2π ) 02 3

Then φ is a stable point. Indeed, as described in Example 6.4.19, If a sequence of homomorphisms (φp )p∈ℕ converges to φ, then for p large enough φp belongs to Hom0d (Γ, I(5)) and by Theorem 6.4.12 we deduce that for sufficiently large p, φp ∈ R (Γ, I(5), H). II. For general compact extensions of ℝn , the geometric and near stability do not hold as it is shown in Example 6.4.22 below. The tangential homogeneous spaces of reductive homogeneous spaces are typical examples. Remark first the following immediate fact.

6.4 Stability for Euclidean motion groups | 323

Fact 6.4.21. Suppose that G := G1 × G2 is a Lie group, Γ ⊂ G1 × {e′ } is a discrete subgroup and H ⊂ {e} × G2 is a closed subgroup where e and e′ designate the unity elements of G1 and G2 , respectively. Then Γ acts on G/H properly discontinuously and freely. Example 6.4.22. Let G = K ⋉ ℝ4 , where K = O2 (ℝ) × O2 (ℝ). Any g ∈ G is written as A g = (Ag , xg ) = (( 1,g

A2,g

x ) , ( 1,g )) , x2,g

(6.15)

where A1,g , A2,g ∈ O2 (ℝ) and x1,g , x2,g ∈ ℝ2 . Let (ei )1≤i≤4 be the standard basis of ℝ4 . Let Γ be generated by γ1 = (I4 , e1 ) and 󵄨󵄨 0 󵄨 H = { (I4 , ( 2 )) 󵄨󵄨󵄨󵄨 x ∈ ℝ2 } . x 󵄨󵄨 Thanks to Fact 6.4.21, Γ acts properly and freely on G/H. We show next that idΓ is neither geometrically nor nearly stable. Set Cr := K × B(0, ρ), where B(0, ρ) is the closed ball of center 0 and radius ρ. For any p ∈ ℕ∗ , the homomorphism φp defined as r( p1 )

φp (γ1 ) = ((

I2

x ) , ( 1 0 )) =: sp , x p 0

where 1 x0 = ( ) . 0 By Theorem 1.2.24, φp (Γ) is discrete, and hence φp ∈ Hom0d (Γ, G). Obviously, {φp }p∈ℕ∗ converges to the identity of Γ. On the other hand, any γ ∈ φp (Γ) is such that γ = σpm for some m ∈ ℤ, so it is of the matrix form r( mp )

γ = σpm = ((

I2

v ) , ( m m )) , x p 0

where vm = (I2 − r( p1 ))−1 (I2 − r( mp ))x0 as in equation (6.11). Remark that ‖vm ‖ ≤

2

1 | cos( 2p )|

=: ρp

and limm→∞ ‖ mp x0 ‖ = +∞; hence φp (Γ) is not compact and we have 0 r( m ) 0 I v I ) , ( 2 )), σpm = (( 2 ) , ( m )) (( 2 ) , ( m 2 )) (( p x 02 I 0 I I 0 2 2 2 2 p ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ ∈Cρp

∈H

∈Cρp

324 | 6 Stability concepts and Calabi–Markus phenomenon and φp (Γ) ⊂ Cρp HCρp so that φp ∈ ̸ R (Γ, G, H). This allows us to conclude that for any neighborhood V of idΓ , there exists p ∈ ℕ such that φp ∈ V ∩ Hom0d (Γ, G) and, therefore, idΓ is not geometrically stable and not nearly stable.

III. Table 6.1 summarizes the obtained results concerning the different stability variants in the setting of Euclidean motion groups. Table 6.1: Summarizing various stability results.

Γ finite Γ infinite Γ crystallographic

Near Stability

Geometric Stability

Stability

holds holds holds

holds holds holds

holds fails in general holds

6.5 The Calabi–Markus phenomenon Let (G, H) be in full generality. The problem of finding discontinuous groups for G/H is trivial when H is compact. So, the interesting cases consist of constructing infinite discontinuous groups for G/H when H and G/H are noncompact and are the subject of our inquiry in the present subsection. When H is noncompact, it may happen that there does not exist an infinite discrete subgroup Γ of G, which acts properly discontinuously on G/H. This phenomenon is called the Calabi–Markus phenomenon. We here focus attention on this issue in the context of some compact extensions of nilpotent Lie groups.

6.5.1 Case of Euclidean motion groups In the context of Euclidean motion groups, we proved that when Γ is infinite (this is for instance the case for crystallographic groups), the proper action obliges the subgroup H not supposed to be connected, to be compact (cf. Proposition 2.5.1). This phenomenon makes it easier the study of the parameter and the deformation spaces and their topological and geometrical features. As such, this important fact allows us to prove that the Calabi–Markus occurs in the setting of Euclidean motion groups. That is, if H is a closed noncompact subgroup of G, then G/H does not admit a compact Clifford–Klein form, unless G/H itself is compact. Let us record first the following questions posed by T. Kobayashi in [95] for general Lie groups. Question 1. Does G/H admit a Clifford–Klein form of infinite fundamental group? Question 2. Does G/H admit a noncommutative-free group as a discontinuous group?

6.5 The Calabi–Markus phenomenon

| 325

We first give complete answers to these questions in the context of Euclidean motion based on prior results. More precisely, we prove the following. Theorem 6.5.1. Let G be the Euclidean motion group. Then: (1) The Calabi–Markus phenomenon occurs for G. That is, if H is a closed, noncompact subgroup of G, then G/H does not admit a compact Clifford–Klein form, unless G/H itself is compact. (2) If H is a closed subgroup of G, then G/H admits a Clifford–Klein form of infinite fundamental group if and only if H is compact. (3) G/H admits no noncommutative-free group as a discontinuous group. More precisely, G itself admits no noncommutative-free discrete subgroups. Proof. The first point of the theorem immediately follows from Proposition 2.5.1. This proposition also shows that it is not possible to get a Clifford–Klein form Γ\G/H of infinite fundamental group unless H is compact, which proves the second point. The third point is an immediate consequence of Lemma 1.2.22. Remark 6.5.2. (1) Given a closed subgroup H of G, one important question is to find a nontrivial discrete subgroup Γ of G in such a way that Γ\G/H is a Clifford–Klein form. We show hereafter that this is not true in general in the setting of Euclidean motion groups. When for instance G/H is compact, it fails in general to admit a compact Clifford– Klein form Γ\G/H with a nontrivial discontinuous group Γ for G/H. Indeed, take n = 4, G = I(4) and H = { ((

A 0

󵄨󵄨 0 x 󵄨 ) , ( ′ )) 󵄨󵄨󵄨󵄨 A, B ∈ O2 (ℝ) and x, x ′ ∈ ℝ2 } . B x 󵄨󵄨

Then H is not compact and, therefore, the proper action holds if and only if Γ is finite as in Proposition 2.5.1. Let g = (A, x) be an element of G of finite order. Then A itself is of finite order and according to equation (1.15), A is conjugate to an element of the form A1 0

(

0 ), A2

where Ai belongs to O2 (ℝ), i = 1, 2. Using Lemma 1.2.6, the element g is conjugate to an element of H. Hence any element of G of finite order is conjugate to an element of H. This entails that the unique Clifford–Klein form is G/H since the fixed-point free action holds uniquely when Γ is trivial.

326 | 6 Stability concepts and Calabi–Markus phenomenon (2) For the same context where G = I(4), we show that one can find nontrivial discontinuous groups for a compact homogeneous space G/H. Let H = { ((

I2 0

󵄨󵄨 0 x 󵄨 ) , ( 1 )) 󵄨󵄨󵄨󵄨 A ∈ O2 (ℝ), x1 , x2 ∈ ℝ2 } . A x2 󵄨󵄨

Then G/H is compact and the proper action holds if and only if Γ is finite. Let r( 2π ) γ = (( 3 0

0

) , 0)

r( 2π ) 3

and Γ = ⟨γ⟩. Then clearly Γ\G/H is a compact Clifford–Klein form. In several contexts (G is nilpotent connected and simply connected, G is exponential and Γ is Abelian, G is the reduced Heisenberg group, etc.), it is shown that whenever Γ is infinite, Hom0d (Γ, G) is an open set in Hom(Γ, G) (cf. [25, 31, 34]). In the context of Euclidean motion groups, we shall show in contrast that it is no longer the case. Let us start by studying an example, which is analogue to [98, Example 1.1.1] for the action of Aff(ℝ) on ℝ. Example 6.5.3. Let G = I(2), (ei )1≤i≤2 the standard basis of ℝ2 and Γ = ⟨γ⟩, where γ = (I2 , e1 ). Γ is a discrete subgroup of G. Consider the sequence of noninjective homomorphisms {φp }p∈ℕ∗ such that φp (γ) = (r( πp ), e1 ). Clearly, {φp } converges to idΓ which is in Hom0d (Γ, G). This shows that this set is not open in Hom(Γ, G). Else, let ψp such that ψp (γ) = (r( p1 ), e1 ), ψp is not of discrete image, and converges to idΓ . We now proceed to prove our first rigidity result in this context. Proposition 6.5.4. Let G be the Euclidean motion group and Γ a finite subgroup of G. Then Hom0 (Γ, G)/G is finite. Proof. Let {φs }s∈ℕ be a sequence in Hom0 (Γ, G) converging to φ. Denote φs = (φ1,s , φ2,s ) and φ = (φ1 , φ2 ). Remark that by Lemma 5.7.6, the sequence {φ1,s }s∈ℕ belongs to Hom(Γ, On (ℝ)). Let γ ∈ Γ such that φ1,s (γ) = I, then φs (γ) is an element of infinite order whenever φ2,s (γ) ≠ 0. This means conclusively that {φ1,s }s∈ℕ is in Hom0 (Γ, On (ℝ)) and converges to φ1 . Therefore, one of the orbits [ψi ]’s contains an infinite number of elements of this sequence and, therefore, due to its convergence there exists i0 ∈ {1, . . . , m}, s0 ∈ ℕ such that for any s ≥ s0 , φ1,s ∈ [ψi0 ] and so φ1 ∈ [ψi0 ]. Equivalently, for s ≥ s0 there exists As ∈ On (ℝ) such that φ1,s = As φ1 A−1 s . Hence p1 (φs (Γ)) = As p1 (φ(Γ))A−1 . On the other hand, by Fact 1.2.10, there exist t and t ∈ ℝn s s such that (I, t)φ(Γ)(I, −t) = p1 (φ(Γ)) × {0} and

(I, ts )φs (Γ)(I, −ts ) = p1 (φp (Γ)) × {0},

6.5 The Calabi–Markus phenomenon

| 327

and then one can easily check that φs = (As , t − ts )(φ1 , 0)(As , t − ts )−1 , which completes the proof. Remark 6.5.5. Corollary 6.5.4 shows immediately that for any finite subgroup Γ, the parameter space R (Γ, G, H) is a finite union of G-orbits for which the corresponding subgroups act fixed point freely on G/H. Then R (Γ, G, H) is open in Hom(Γ, G) (thus any homomorphism is stable) and the deformation space T (Γ, G, H) is a finite set. 6.5.2 The case of SOn (ℝ) ⋉ ℝn Let M(n) := SOn (ℝ) ⋉ ℝn be the semidirect product of the rotation group SOn (ℝ) (with respect to the canonical Euclidean product on ℝn ) and ℝn . Then, obviously, one can remark that the statement of Proposition 2.5.1 holds for M(n) and so does that of Corollary 6.4.9. This allows us to affirm the following. Theorem 6.5.6. The conclusions of Theorems 5.1.5 and 6.5.1 hold for the group M(n) for any n ≥ 2. When more generally the compact component SOn (ℝ) is replaced by a general compact subgroup of Aut(ℝn ), the statement of Proposition 2.5.1 clearly fails to hold as many examples reveal. So, there is no hope that the statements of Theorem 6.5.1 hold for general compact extensions of ℝn as in the coming subsection. 6.5.3 Case of the semidirect product K ⋉ ℝn We now prove the following. Theorem 6.5.7. Let H be a closed, connected subgroup of G := K ⋉ℝn . Then the Calabi– Markus phenomenon occurs if and only if, for any linear subspace V ≠ {0} of ℝn , there exists k ∈ K such that V ∩ [k ⋅ EH ] ≠ {0}. Proof. If there exists some nontrivial linear subspace V of ℝn such that K ⋅ V ∩ EH ≠ {0}, then for a nonzero vector u of V the discrete subgroup Γ = {(I, pu), p ∈ ℤ} acts properly on G/LH thanks to Theorem 2.5.7. Note here that Γ is torsion-free and then it acts freely on G/LH whenever it acts properly on G/LH . Conversely, if there exists an infinite discrete subgroup Γ, which acts discontinuously on G/LH , then LG acts properly G/LH , and hence K ⋅ EΓ ∩ EH ≠ {0}, which completes the proof. 6.5.4 Case of Heisenberg motion groups We here refer back to Section 1.3 with all the notation and results, and prove the following.

328 | 6 Stability concepts and Calabi–Markus phenomenon Theorem 6.5.8. Let G be a Heisenberg motion group and H a closed subgroup of G. Then G/H admits an infinite discontinuous group if and only if H is conjugate to a subgroup of G1 or for any r > 0, H ∩ (𝕌n × B(0, r) × ℝ) is compact. Proof. As in Proposition 1.3.8, an infinite discrete subgroup contains an element of infinite order γ, say, and by Lemma 1.3.1, there exists g ∈ G such that gγg −1 = (A, z, t) where Az = z, tz = 0 and ‖z‖ + |t| ≠ 0. If t = 0, then by Proposition 2.6.3 H is conjugate to a subgroup of G1 . Otherwise, H ∩ (𝕌n × B(0, r) × ℝ) is compact for any r > 0 thanks to Proposition 2.6.2. Conversely, if H is conjugate to a subgroup of G1 , take Γ the subgroup of type (B) generated by (I, z, 0) for some z ∈ ℂn \{0}. As pr(Γ) is discrete, Proposition 2.6.3 asserts that Γ acts properly on G/H and, therefore, discontinuously on G/H. Finally, if H ∩ (𝕌n × B(0, r) × ℝ) is compact for any r > 0 one can consider Γ to be the subgroup of type (A) generated by (I, 0, 1) and the result follows from Proposition 2.6.2. Remark 6.5.9. When H is conjugate to a subgroup of G1 and satisfies H ∩(𝕌n ×B(0, r)× ℝ) is compact for any r > 0, then obviously H is compact and, therefore, any infinite torsion- free discrete subgroup of G acts discontinuously on G/H. The readers can consult the reference [16] for more comprehensive details. The following is a direct consequence of Theorem 6.5.8. Corollary 6.5.10. The Calabi–Markus phenomenon occurs for a homogenous space G/H if and only if H contains an infinite discrete subgroup conjugate to a subgroup of 𝕌n × ℂn × {0} and H ∩ (𝕌n × B(0, r) × ℝ) is not compact for some r > 0. Indeed, if H contains an infinite discrete subgroup conjugate to a subgroup of 𝕌n × ℂn × {0}, it then contains an element (B, z, t) for which PB (z) = z ≠ 0, and Proposition 1.3.14 allows to conclude.

6.5.5 Existence of compact Clifford–Klein forms This section aims to study the existence of a compact Clifford–Klein form Γ\G/H, where H stands for a closed, connected subgroup of a Lie group G and Γ is an infinite discontinuous group for G/H. Our first result is the following. Proposition 6.5.11. Let H be a closed connected subgroup of G and R its solvable radical. The following are equivalent: (i) G/H admits a compact Clifford–Klein form. (ii) G/R admits a compact Clifford–Klein form. Proof. Suppose that G/H admits a compact Clifford–Klein form Γ\G/H, where Γ designates a discontinuous group for G/H. Equivalently, there exists a compact set C of G

6.5 The Calabi–Markus phenomenon

| 329

such as G = ΓCH. Clearly, Γ acts discontinuously on G/R. ΓCH = Γ(CS)R, which entails that Γ\G/R is compact. Conversely, assume that there exists a discontinuous group Γ for G/R such that Γ\G/R is compact. Through Theorem 1.2.24, we easily get that Γ contains an Abelian torsion-free group Γa of finite index in Γ, which entails that Γa \G/R is compact. Write H = SR for some Levi complement S of R. For any compact set K of G, Γa ∩ KHK −1 = Γa ∩ (KS)RK −1 is compact by means of Lemma 2.5.5. Therefore, Γa acts properly on G/H and gives in turn that for any g ∈ G, Γa ∩ gHg −1 is a finite group and then trivial. Hence Γa acts freely on G/H. It suffices now to consider a compact set C such that G = Γa CR. Then G = Γa CSR, as C ⊂ CS, and hence Γa \G/H is compact. We next prove the following. Theorem 6.5.12. Let G := K ⋉ ℝn and H a closed, connected subgroup of G. Then G/H admits a compact Clifford–Klein form, if and only if, there exists a linear subspace V of ℝn such that for all k ∈ K, ℝn = V ⊕ [k ⋅ EH ]. Proof. By Proposition 6.5.11 and Proposition 1.2.38, we can assume that H is a solvable subgroup of G. Assume first that for any linear subspace V of ℝn , there exists k ∈ K such that V ∩ [k ⋅ EH ] ≠ {0} or V + [k ⋅ EH ] ≠ ℝn . Let now Γ be a discontinuous group for G/H and take V = EΓ . By Theorem 2.5.7, there exists k ∈ K such that EΓ ⊕ [k ⋅ EH ] ≠ ℝn , which is in turn equivalent to the fact that EΓ ⊕ EH ≠ ℝn , as dim(EH ) = dim(k ⋅ EH ). Let S be the orthogonal complement of EΓ ⊕ EH in ℝn . From Corollary 1.2.41, there exist τ := (I, t) ∈ G and τ′ := (I, t ′ ) ∈ G ′ such that EH = pr2 (H τ ) and pr2 (Γτ ) ⊂ EΓ . Let (I, s) and (I, s′ ) be two elements of S ′ ′ ′ such that Γτ (I, s)H τ = Γτ (I, s′ )H τ . There exist (A, v) ∈ Γτ and (B, y) ∈ H τ such that (I, s′ ) = (A, v)(I, s)(B, y). It follows that B = A−1 and s′ = v +As+Ay. From Lemma 1.2.28, EΓ and EH are stable by A and so is S. Hence As ∈ S, v + Ay = 0 and s′ = As. This entails ′ ′ that the set {Γτ (I, αs)H τ | α ∈ ℝ} is not compact and lies in Γτ \G/H τ . Conversely, let g = (A, x) ∈ G. As V ⊕ [A ⋅ EH ] = ℝn , there exist x ′ ∈ V and x′′ ∈ EH such that x = x′ + Ax ′′ . Let now (v1 , . . . , vp ) be a basis of V and Γ the discrete subgroup of G spanned by the set (v1 , . . . , vp ). Then Γ acts properly on G/H, as EΓ = V by Theorem 2.5.7. In addition, it is clear that Γ acts freely on G/H. Further by Corollary 1.2.37, there exists τ := (I, t) ∈ G such that EH = pr2 (H τ ). Hence there exists A′′ ∈ K such that (A′′ , x′′ ) ∈ H τ . From Proposition 1.2.34, we get x′′ ∈ ker(A′′ − I). Indeed, it is enough to observe that PA′′ (x′′ ) = x′′ , as x′′ ∈ EH . Therefore, ΓgH τ = Γ(A, x′ + x′′ )H τ = Γ(A, x ′ )(I, x ′′ )H τ = Γ(A, x′ )(A′′ −1 , 0)(A′′ , x ′′ )H τ = Γ(AA′′ −1 , x′ )H τ .

330 | 6 Stability concepts and Calabi–Markus phenomenon As V/Γ is compact, there exists a compact subset C of ℝn such that V = Γ ⋅ C. Write x′ = xΓ′ + xC′ , where xΓ′ ∈ Γ and xC′ ∈ C. As a consequence, we get ΓgH τ = Γ(AA′′ −1 , xΓ′ + xC′ )H τ = Γ(I, xΓ′ )(AA′′ −1 , xC′ )H τ = Γ(AA′′ −1 , xC′ )H τ .

(6.16)

Consider now the natural continuous mapping π : G 󳨀→ Γ\G/H τ . It follows from equation (6.16) that Γ\G/H τ = π(G) ⊂ π(K × C). This shows that Γ\G/H is compact. Remark 6.5.13. The relation between the existence of compact Clifford–Klein forms and the Calabi–Markus phenomenon depends upon the choice of the closed subgroup H. Note that in general, the Calabi–Markus phenomenon (i. e., the nonexistence of infinite discontinuous group for the homogenous space G/H) entails the nonexistence of compact Clifford–Klein forms unless the homogenous space G/H itself is compact. As proved in [13], both properties are equivalent in the setting of Euclidean motion groups. However, this is no longer the case as revealed in the coming examples. Example 6.5.14. Set K = {(

A

󵄨󵄨 󵄨 ) 󵄨󵄨󵄨󵄨 A, B ∈ SO2 } . B 󵄨󵄨

x

For any vector x = ( x21 ) ∈ ℝ4 , where x1 , x2 ∈ ℝ2 and any k = ( A B ) ∈ K, one has Ax1 ). Bx2

k⋅x =( Put G = K ⋉ ℝ4 and

0 󵄨󵄨󵄨󵄨 { } { } 󵄨󵄨 { } { t } 󵄨󵄨 ′ 󵄨 H1 = { ( ) 󵄨󵄨 t, t ∈ ℝ} . 󵄨 { } 0 󵄨󵄨 { } { } 󵄨󵄨 ′ 󵄨 󵄨 { t } Then G/H1 is not compact and there exist no infinite discrete discontinuous group for u G/H. Indeed, for any nonzero vector u = ( u21 ), u1 , u2 ∈ ℝ2 there exists A1 , B1 ∈ SO2 such that 0 A1 u1 = ( ) ‖u1 ‖ Hence (

A1

B1 )

and

B1 u2 = (

0 ). ‖u2 ‖

⋅ u ∈ H1 , which is enough to conclude thanks to Theorem 6.5.7.

6.5 The Calabi–Markus phenomenon

| 331

Take now 0 󵄨󵄨󵄨󵄨 } { } { 󵄨󵄨 } { } { t 󵄨󵄨 󵄨 H2 = { ( ) 󵄨󵄨 t ∈ ℝ} . 󵄨 } { 0 󵄨󵄨 } { } { 󵄨󵄨 󵄨 󵄨 } { t There exists an infinite discontinuous group Γ for the homogenous space G/H2 . It suffices to consider the discrete subgroup 󵄨󵄨 p 󵄨󵄨 { } { } 󵄨󵄨 { } { } 󵄨󵄨 0 Γ = { (I4 , ( )) 󵄨󵄨󵄨 p ∈ ℤ} . 󵄨 { } 0 󵄨󵄨 { } { } 󵄨󵄨 󵄨 0 󵄨 { } Indeed, for k ∈ K and u ∈ EH such that ku ∈ EΓ , we easily get that u = 0, which entails that Γ acts properly on G/H2 thanks to Theorem 2.5.7. As Γ is torsion-free, then evidently Γ is discontinuous for G/H2 . Moreover, for a given linear subspace V such that for any k ∈ K, V ∩ k ⋅ EH2 = {0}, V does not contain vectors of the form ( ba ) for a, b ∈ ℝ2 such that ‖a‖ = ‖b‖. Hence 1 0 } { { } { { 0 } 1 } V ∩ ℝ-span {( ) , ( )} = {0}, { 0 } { 1 } { } 0 1 } { and then the dimension of V is less than 2, which gives in turn that for any k ∈ K, V ⊕ k ⋅ EH2 ≠ ℝ4 . Thanks to Theorem 6.5.12, G/H2 does not admit a compact Clifford– Klein form. Case of Heisenberg motion groups The following theorem provides a criterion for the existence of compact Clifford–Klein forms as an important consequence from Theorem 6.5.8 in the case of Heisenberg motion groups. Theorem 6.5.15. Let H be a closed subgroup of G. The homogenous space G/H admits a compact Clifford–Klein form if and only if either H or G/H is compact. Proof. There is nothing to do in the setting where G/H is compact. Assume now that H is compact. Then any torsion-free group is a discontinuous group for G/H. Consider {es }1≤s≤n the standard basis of ℝn and define for any s ∈ {1, . . . , n} us = es and un+s = ies . Consider the subgroup L of ℂn of generator {us }1≤s≤2n . Take Γ to be the subgroup of G

332 | 6 Stability concepts and Calabi–Markus phenomenon generated by {γs }1≤s≤2n where, for any s ∈ {1, . . . , 2n}, γs = (I, us , 0). A direct computation shows that [γs+n , γs ] = (I, 0, 1). Then Γ is not Abelian and for any v ∈ L and any p ∈ ℤ, (I, v, p2 ) ∈ Γ. Hence Γ coincides with the set {I} × L × 21 ℤ and, therefore, Γ is a discrete subgroup of G. On the other hand, we easily check that G = Γ ⋅ C1 , for some compact set C1 of G. Thus, G = Γ ⋅ C1 ⋅ H, which gives in turn that Γ\G/H is compact. Conversely, assume that both H and G/H are not compact. Since Γ\G/H is compact but not for G/H, the discrete group Γ should be infinite. Let Γ be an infinite discontinuous group for G/H such that Γ\G/H is compact. Therefore, Γ is not of the type (C), by Proposition 2.6.4. Assume first that Γ is of the type (A), then Γ is conjugate to a subgroup of G1 . By Fact 1.3.6, Γ contains some torsion-free subgroup Γ∗ of Γ of finite index in Γ and isomorphic to a central subgroup of ℍn . Then Γ∗ is of rank 1 and generated by some (A0 , 0, t0 ). Write Γ = Γ∗ ⋅ Λ, for some finite set Λ, say, and remark that the compactness property of Γ\G/H is equivalent to that of Γ∗ \G/H. On the other hand, Γ∗ ⊂ {(I, 0, pt0 ) : p ∈ ℤ}⋅ C where C designates the closure of the set {(Ap0 , 0, 0) : p ∈ ℤ}. If for some compact set C of G, G = Γ ⋅ C ⋅ H then G = {(I, 0, pt0 ) : p ∈ ℤ} ⋅ C ⋅ C ′ ⋅ H, for some other compact set C ′ of G. Therefore, {(I, 0, pt0 ) : p ∈ ℤ}\G/H is compact. Hence we can take (A0 , 0, t0 ) = (I, 0, 1). By Proposition 2.6.2, for any r > 0, H ∩ (𝕌n × B(0, r) × ℝ) is compact, which says that pr(H) is a closed subgroup of 𝕌n ⋉ ℂn thanks to Lemma 1.3.15. Assume for a while that 𝕌n ⋉ ℂn / pr(H) is not compact, then there exists some noncompact cross-section X for the pr(H)-cosets of 𝕌n ⋉ ℂn for which 𝕌n ⋉ ℂn / pr(H) is homeomorphic to X . Write 𝕌n ⋉ ℂn = X ⋅ pr(H), then G = (X ⋅ pr(H)) × ℝ.

(6.17)

Remark further that if there exist (B, z, t), (B, z, t ′ ) ∈ H, then necessarily t = t ′ . Otherwise, (I, 0, t − t ′ ) ∈ H, which contradicts the hypothesis. Hence one can consider the mapping s1 : pr(H) → ℝ, (B, u) 󳨃→ s, for which (B, u, s) is the unique element of H such that (B, u) ∈ pr(H). With respect to decomposition (6.17), any g ∈ G can be written as g = (A, z, t) = ((A ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ X , zX ) ⋅ (AH , zH ), t) ∈𝕌n ⋉ℂn

6.5 The Calabi–Markus phenomenon

| 333

and then g = (A ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ X , zX , t − s1 ((AH , zH )) + ∈X

1 Im⟨zX , AX zH ⟩) ⋅ (A ⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟⏟ H , zH , s1 ((AH , zH ))). 2

(6.18)

∈pr(H)

Besides, for any m := (AX , zX ) ∈ X and for any p ∈ ℤ, there exists some tp ∈ ℝ such that tp − s1 ((AH , zH )) +

1 Im⟨zH , AH zX ⟩ = p. 2

Thus, by equation (6.18), G/H contains a subset Σ = {(m, p) : m ∈ X , p ∈ ℤ}, where (m, p) designates the class of (m, p) for the H-coset in G. The action of Γ on G/H induces an action on Σ given as (I, 0, q) ⋅ (m, p) = (m, p + q) for each q ∈ ℤ. Recall the covering map π : G/H → Γ\G/H, then Σ̃ := π(Σ) = π(Σ′ ), where Σ′ = {(m, 0) : m ∈ X } is the noncompact set of G/H. Now π|Σ′ is injective and π(Σ′ ) is a noncompact set lying in Γ\G/H, and hence a contradiction. Suppose now that 𝕌n ⋉ ℂn / pr(H) is compact. Necessarily, there exists (A, z) ∈ pr(H) such that PA (z) ≠ 0, we can assume that Az = z ≠ 0. If for some α ∈ ℝ, (A, αiz) ∈ pr(H) then for t, t ′ ∈ ℝ such that h = (A, z, t) ∈ H and h′ = (A, αiz, t ′ ) ∈ H, a direct computation shows that [h, h′ ] = (I, 0, −α) ∈ H, which is absurd. This already gives that for any α, α′ ∈ ℝ, the fact that α ≠ α′ is equivalent to the fact (A, α′ iz) ∈ ̸ (A, αiz) pr(H). Then {(A, αiz) : α ∈ ℝ} is a noncompact, closed subset sitting inside 𝕌n ⋉ ℂn / pr(H), which is also absurd. Therefore, Γ is necessarily of type (B) and pr(Γ) is discrete thanks to Proposition 2.6.3. Moreover, for any (B, u, s) ∈ H, PB (u) = 0. Hence H is conjugate to a subgroup of G1 . As we may assume that H ⊂ G1 , and clearly H contains some (A, 0, t) with t ≠ 0. Therefore, the subgroup ΓH generated by (A, 0, t) is a discrete cocompact subgroup of H. Remark that the compactness of Γ\G/H is equivalent to that of ΓH \G/Γ. Since ΓH is of type (A), then it suffices to substitute ΓH for Γ and Γ for H (as neither Γ nor G/Γ is compact) and use the same arguments as above to meet a contradiction.

6.5.6 Concluding remarks As in Theorem 6.5.1, there is a relationship between the existence of a compact Clifford–Klein forms and the Calabi–Markus phenomenon in the context of Euclidean motion groups. That is, there is no infinite discontinuous groups for a noncompact homogenous space I(n)/H whenever H is a closed subgroup, which is not compact.

334 | 6 Stability concepts and Calabi–Markus phenomenon However, this is not longer the case for the Heisenberg motion groups. We present the following examples to illustrate this issue. Example 6.5.16. Let H be the subgroup of G generated by h1 = (I, z, 0) and h2 = (I, √2z, 1), for some z ∈ ℂn with ‖z‖ = 1. Remark that for any h ∈ H there exist p, q ∈ ℤ such that h = (I, (p + q√2)z, q), and subsequently H is discrete. For any (p, q) ∈ ℤ2 \{(0, 0)}, we have τup,q hτu−1p,q = (I, (p + q√2)z, 0) for up,q =

−iz . |p+q√2|2

On the other hand, there exist some infinite sequences {α(p)}p∈ℕ ⊂

ℤ and {β(p)}p∈ℕ ⊂ ℤ such that {α(p) + √2β(p)}p∈ℕ tends to 0. For p large enough, |α(p) + √2β(p)| < 1 and then (I, (α(p) + β(p)√2)z, β(p)) ∈ 𝕌n × B(0, 1) × ℝ. This allows us to conclude that there is no infinite discontinuous groups for G/H. Subsequently, there is no compact Clifford–Klein forms for G/H. For a noncompact homogenous space G/H, it is clear that in order to obtain a compact Clifford–Klein form, we necessarily need to find an infinite discontinuous group Γ for G/H, which is not the case as in Theorem 6.5.10. On the other hand, one can find a homogenous space admitting infinite discontinuous groups but without compact Clifford–Klein forms as shown in the following example. Example 6.5.17. Set n = 2 and fix a vector z ∈ ℂ2 \{0}, H = {(I2 , tz, 0) : t ∈ ℝ}. The discrete subgroup Γ of generator (I2 , 0, 1) is discontinuous for the homogenous space G/H and then the Calabi–Markus’s phenomenon does not hold. On the other hand, G/H does not admit a compact Clifford–Klein forms by Theorem 6.5.15.

7 Discontinuous actions on reduced nilmanifolds In the present chapter, the point is to remove the assumption on the groups in question to be simply connected. We refer to the background from Chapter 1 for the general thẽ the universal covering ory. That is, G denotes an exponential connected Lie group, G ̃ This means that g of G and g the real exponential solvable Lie algebra of both G and G. is solvable, the exponential mapping of G is surjective and the associated exponential ̃ is a C ∞ -diffeomorphism. map expG̃ : g → G r The attention is first focused on the reduced Heisenberg group H2n+1 for which the universal covering is H2n+1 , defined in Subsection 1.1.2. Unlike the context of H2n+1 (cf. Theorem 6.2.3), we show here that the deformation space T (Γ, G, H) is a Hausdorff space and even endowed with a smooth manifold structure for any arbitrary connected subgroup H of G and any arbitrary discontinuous group Γ for G/H. Indeed, we will provide a disjoint decomposition of T (Γ, G, H) into open smooth manifolds of a common dimension. On the other hand, we show that the stability property holds for any deformation parameter, giving evidence that in some small neighborhood Vφ of any element φ of the parameter space. The proper action of the discrete subgroup ψ(Γ), ψ ∈ Vφ on G/H is preserved. r r r Moving to the product Lie group G = H2n+1 × H2n+1 and ΔG = {(x, x) ∈ G : x ∈ H2n+1 } the diagonal subgroup of G, we provide given any discontinuous group Γ ⊂ G for G/ΔG , a layering of the parameter space R (Γ, G, ΔG ), which is shown to be endowed with a smooth manifold structure. We also show that the stability property holds. On the r r other hand, a (strong) local rigidity theorem is obtained for both H2n+1 and H2n+1 × r H2n+1 . That is, the parameter space admits a locally rigid point if and only if Γ is finite (giving an affirmative answer to Conjecture 5.7.10) and this is also equivalent to the fact that the deformation space is a Hausdorff space. We also consider the setting of reduced threadlike groups and tackle similar questions. We show that Conjecture 5.7.10 holds for Abelian discontinuous groups and that non-Abelian discontinuous groups are stable. We also single out the notion of stability on layers and show that any Abelian discontinuous group is stable on layers. More detailed results on this subject could be found in [27] and [28].

7.1 Reduced Heisenberg groups 7.1.1 Backgrounds Let g := h2n+1 designate the Heisenberg Lie algebra of dimension 2n + 1 as defined in Section 1.1.2 and H2n+1 the corresponding connected and simply connected Lie group. r Let G := H2n+1 be the reduced Heisenberg Lie group, which is defined as the quotient of H2n+1 by the central discrete subgroup expG̃ (ℤZ), where Z designates a nonzero generator of the center of g as in equation (1.6). The center Z(G) of G is compact and is https://doi.org/10.1515/9783110765304-007

336 | 7 Discontinuous actions on reduced nilmanifolds identified to the torus 𝕋, the group of complex numbers of modulus 1. The group G is therefore identified to ℝ2n ⋉ 𝕋. As the exponential mapping exp := expG is given by exp(U + λZ) = (U, e2iπλ ),

U ∈ ℝ2n and λ ∈ ℝ,

G can be equipped with the following law: 1

(X, Y, e2iπt ) ∗ (X ′ , Y ′ , e2iπs ) = (X + X ′ , Y + Y ′ , e2iπ(t+s+ 2 (≺X ,Y≻−≺X,Y ′

′

≻))

),

where X, Y, X ′ , Y ′ ∈ ℝn , t, s ∈ ℝ and ≺, ≻ denotes the usual Euclidian scalar product. According to Proposition 1.4.11, the exponential map exp : g → G is surjective. Recall that the Lie algebra g acts on itself by the adjoint representation ad, that is, adT (Y) = [T, Y],

T, Y ∈ g.

The group G also acts on g by the adjoint representation Ad, defined by Adg = exp ∘ adT ,

g = exp T ∈ G.

󳨀 For → w ∈ ℝ2n and c ∈ ℝ, we adopt the notation t→ 󳨀 w 󳨀 exp(→ w + cZ) = ( 2iπc ) . e

r 7.1.2 Discrete subgroups of H2n+1

Let Γ be a discrete subgroup of G. We first pose the following. Definition 7.1.1. Let εΓ be the integer given by εΓ = 0 if Γ is torsion-free and εΓ = 1 otherwise. Let also rΓ be the rank of Γ, which is the cardinality of a minimal generating set. The nonnegative integer lΓ := rΓ − εΓ is called the length of the subgroup Γ. We next prove the following structure result. Proposition 7.1.2. For a discrete subgroup Γ of G, there exist a unique nonnega󳨀 󳨀 tive integer lΓ and a linearly independent family of vectors {→ w 1 , . . . ,→ w lΓ } of ℝ2n such that (1) If Γ is torsion-free, then nlΓ n1 t→ t→ 󳨀 󳨀 w w Γ = {( 2iπc11 ) ⋅ ⋅ ⋅ ( 2iπclΓl ) ; n1 , . . . , nlΓ ∈ ℤ} , e e Γ

for some c1 , . . . , clΓ ∈ ℝ.

7.1 Reduced Heisenberg groups | 337

(2) Otherwise, let q ∈ ℕ∗ be the order of Γ ∩ Z(G). Then Γ=

n1 t→ 󳨀 w {( 2iπc11 )

e

⋅⋅⋅(

nl t→ 󳨀 w lΓ Γ ) 2iπclΓ

e

s

󳨀 t→ 0

( 2iπ 1 ) ; n1 , . . . , nlΓ , s ∈ ℤ} , e q

for some c1 , . . . , clΓ ∈ ℝ. Proof. We first consider the surjective projection π1 : G

G/Z(G)

󳨀→ t→ 󳨀

w ( 2iπc ) e

t→ 󳨀

w.

󳨃󳨀→

Then π1 (Γ) is a discrete subgroup of ℝ2n . This gives that there exist a nonnegative 󳨀 󳨀 integer lΓ and a family {→ w 1 , . . . ,→ w lΓ } of linearly independent vectors of ℝ2n such that 󳨀 󳨀 π1 (Γ) = {n1 t → w 1 + ⋅ ⋅ ⋅ + nlΓ t → w lΓ ; n1 , . . . , nlΓ ∈ ℤ} 󳨀 is a discrete subgroup of ℝ2n . As for all j ∈ {1, . . . , lΓ }, → w j ∈ π1 (Γ), there exists cj ∈ ℝ 󳨀 t→ t→ w 󳨀 j such that γ = ( ) ∈ Γ and π (γ ) = w . Then j

e

1

2iπcj

j

j

π1 (Γ) = {n1 π1|Γ (γ1 ) + ⋅ ⋅ ⋅ + nlΓ π1|Γ (γlΓ ); n1 , . . . , nlΓ ∈ ℤ} nl

n

= {π1|Γ (γ1 1 ) + ⋅ ⋅ ⋅ + π1|Γ (γl Γ ); n1 , . . . , nlΓ ∈ ℤ} = =

Γ

nl ⋅ ⋅ ⋅ γl Γ ); n1 , . . . , nlΓ ∈ ℤ} Γ nl n π1|Γ ({γ1 1 ⋅ ⋅ ⋅ γl Γ ; n1 , . . . , nlΓ ∈ ℤ}). Γ

n {π1|Γ (γ1 1

Hence nl

n

Γ = {γ1 1 ⋅ ⋅ ⋅ γl Γ ; n1 , . . . , nlΓ ∈ ℤ} ⋅ (Γ ∩ Z(G)). Γ

We now show that lΓ is unique. Indeed, if lΓ and lΓ′ are two distinct such integers with 󳨀 󳨀 lΓ < lΓ′ , say, there exist two linearly independent families of vectors {→ w 1 , . . . ,→ w lΓ } and ′ ′ 2n → 󳨀 → 󳨀 { w , . . . , w ′ } of ℝ such that 1

lΓ

Γ = {(

n1 t→ 󳨀 w1 ) 2iπc1

e

⋅⋅⋅(

nl t→ 󳨀 w lΓ Γ ) 2iπclΓ

e

m1

′ 󳨀 { t→ w = {( 2iπc1′ ) e 1 {

⋅⋅⋅(

′ t→ 󳨀 w l′ Γ 2iπcl′′

e

Γ

; n1 , . . . , nlΓ ∈ ℤ} ⋅ (Γ ∩ Z(G)) ml′

Γ

)

} ; m1 , . . . , ml′ ∈ ℤ} ⋅ (Γ ∩ Z(G)) Γ }

338 | 7 Discontinuous actions on reduced nilmanifolds for some c1 , . . . , clΓ , c1′ , . . . , cl′′ ∈ ℝ. There exist then for all j ∈ {1, . . . , lΓ′ } some inΓ ′ lΓ j j󳨀 󳨀 tegers (ni )1⩽i⩽lΓ ∈ ℤ such that → w j = ∑i=1 ni→ w i . This is impossible given lΓ < lΓ′ . 1⩽j⩽lΓ′

The following is an immediate consequence of the last proposition. Corollary 7.1.3. Any discrete subgroup of G is finitely generated. Remark 7.1.4. The integer lΓ is indeed the length of Γ. 7.1.3 A matrix-like writing of elements of Hom(Γ, G) This subsection aims to describe the set Hom(Γ, G) of homomorphisms from Γ to G. Let

Mr,s (ℂ) be the vector space of matrices of r rows and s columns. When r = s, we adopt the notation Mr (ℂ) instead of Mr,r (ℂ). Let now {γ1 , . . . , γk } be a set of generators of Γ.

Thanks to the injective map

Hom(Γ, G) → G × ⋅ ⋅ ⋅ × G,

φ 󳨃→ (φ(γ1 ), . . . , φ(γk ))

to equip Hom(Γ, G) with the relative topology induced from the direct product G×⋅ ⋅ ⋅×G and the identification of G × ⋅ ⋅ ⋅ × G to the space M2n+1,k (ℂ), it appears clear that the map Ψ : Hom(Γ, G) 󳨀→ M2n+1,k (ℂ),

(7.1)

which associates to any element φ ∈ Hom(Γ, G), its matrix A C Mφ (A, B, z) = ( B ) = ( 2iπz ) ∈ M2n+1,k (ℂ), e 2iπz e

A C = ( ), B

(7.2)

where A and B ∈ Mn,k (ℝ) and z := (z1 , . . . , zk ) ∈ ℝk , with e2iπz := (e2iπz1

⋅⋅⋅

e2iπzk ) ∈ M1,k (ℂ)

is a homeomorphism on its range. Let us write C = ⌊C 1 , . . . , C k ⌋, where this symbol merely designs the matrix constituted of the columns C 1 , . . . , C k . This means indeed that φ(γj ) := exp(C j + zj Z)

7.1 Reduced Heisenberg groups | 339

for any 1 ≤ j ≤ k. Let E denote the subset of M2n+1,k (ℂ) consisting of the totality of matrices as in (7.2), which is homeomorphic to the set M2n,k (ℝ) × 𝕋k . Through the next coming sections, Γ will serve as a discontinuous group for a homogeneous space G/H. Recall the definitions: Hom0 (Γ, G) = {φ ∈ Hom(Γ, G) : φ is injective} and Hom0d (Γ, G) = {φ ∈ Hom0 (Γ, G) : φ(Γ) is discrete}. The set Hom(Γ, G) is homeomorphically identified to a subset U of E and Hom0d (Γ, G) to a subset Ud0 of U . The group G acts on E through the law: For g = exp X, with X ∈ g with coordinates t (α, β, γ), α, β ∈ M1,n (ℝ), γ ∈ ℝ, g⋆(

⌊g ⋅ C 1 ⋅ g −1 , . . . , g ⋅ C k ⋅ g −1 ⌋ C = ⌊C 1 , . . . , C k ⌋ ) = ( 1 1 k k ), e2iπz e2iπ(z1 +αC1 −βC2 ) ⋅ ⋅ ⋅ e2iπ(zk +αC1 −βC2 )

Ci C i = ( 1i ) , C2

where C1i , C2i ∈ Mn,1 (ℝ), i ∈ {1, . . . , k}. The map Ψ : Hom(Γ, G) 󳨀→ E given in equation (7.1) turns out to be G-equivariant. For A M = M(A, B, z) = ( B ) ∈ M2n+1,k (ℂ), e2iπz A g ⋆ M = Adexp X ⋅M = ( ). B e2iπ(z−βA+αB) For all j ∈ {1, . . . , l = lΓ }, we consider the notation t→ 󳨀 wj

1 t→ 󳨀 wj 2n = (t → 2) ∈ ℝ , 󳨀 wj

1 󳨀2 󳨀 where → w j ,→ w j ∈ ℝn . Let pij = 0 if Γ is torsion-free and 1 󳨀2 1 󳨀2 󳨀 󳨀 pij = q(≺ → w j ,→ wi ≻ − ≺ → w i ,→ w j ≻)

(7.3)

340 | 7 Discontinuous actions on reduced nilmanifolds otherwise. Let also 0 −p ( 12 ( .. P(Γ) = ( . ( .. . −p ( 1l

p12 .. . .. . ⋅⋅⋅

⋅⋅⋅ .. . .. . .. . ⋅⋅⋅

p1l .. . .. .

⋅⋅⋅ .. ..

.

. −pl−1 l

) ) ) ∈ Ml (ℝ), )

pl−1 l 0 )

A (l, ℝ) the subspace of Ml (ℝ) of skew-symmetric matrices and A (l, ℤ) the subset of A (l, ℝ) with entries in ℤ. We now prove the following.

Proposition 7.1.5. We keep the same notation and hypotheses. Let M(A, B, z) be as in (7.2), where A, B ∈ Mn,k (ℝ). We have: (1) If Γ is torsion-free, then k = l and t

t

U = {M(A, B, z) ∈ E : AB − BA ∈ A (l, ℤ)}.

(2) Otherwise, k = l + 1 and { { { { { { U = {M(A, B, z) ∈ E { { { { { {

󳨀 → 󳨀 ′ ′ 󵄨󵄨 A = ( ′ t → A 0 ) , B = (B′ t 0 ) , A , B ∈ Mn,l (ℝ), 󵄨󵄨 } 󵄨󵄨 } } 󵄨󵄨 } } 󵄨󵄨 p 󵄨󵄨 z = (z1 , . . . , zl , ), p ∈ {0, . . . , q − 1}, z1 , . . . , zl ∈ ℝ and } . 󵄨󵄨 } q } 󵄨󵄨 } } 󵄨󵄨 t ′ ′ t ′ ′ p } } 󵄨󵄨 A B − B A ∈ P(Γ) + A (l, ℤ) 󵄨󵄨 } q 󵄨

Proof. It is sufficient to prove the proposition when Γ is not torsion-free. Indeed otherwise, P(Γ) = 0 and the same arguments work. For φ ∈ Hom(Γ, G), Mφ (A, B, z) ∈ U and 󳨀 t→ 0 γl+1 = ( 2iπ 1 ) , e q p we have φ(γl+1 ) = γl+1 for some p ∈ {0, . . . , q − 1}. Now, let r, j ∈ {1, . . . , l}. Then

φ(γr γj γr−1 γj−1 )

−1

= φ(γr )φ(γj )φ(γr ) φ(γj )

−1

󳨀 t→

0 = ( 2iπ(t Aj Br −t Bj Ar ) ) . e

On the other hand, we have γr γj γr−1 γj−1 = (

󳨀 󳨀 t→ t→ 0 0 prj 1 2 1 2 . ) = ( 2iπ prj ) = γl+1 → 󳨀 → 󳨀 → 󳨀 → 󳨀 2iπ(≺ w j , w r ≻−≺ w r , w j ≻) e q e

7.1 Reduced Heisenberg groups |

341

As φ ∈ Hom(Γ, G), then prj

p p + ℤ for some q rj such that gj = t (C j , e2iπzj ) for j

This gives t Aj Br − t Bj Ar ∈

󳨀 t→ 0

pp

φ(γr γj γr−1 γj−1 ) = (φ(γl+1 ))

= γl+1rj = (

e

2iπ

pprj q

).

p ∈ {0, . . . , q − 1}. Let now M(A, B, z) = 2iπ p

⌊g1 , . . . , gl+1 ⌋ ∈ E ∈ {1, . . . , l} and gl+1 = t (0, e q ) for some p ∈ {0, . . . , q − 1} with the convention that gl+1 = e if Γ is torsion-free, which satisfies the required conditions. Let φ be the map defined by φ:Γ

n γ1 1

n nl+1 ⋅ ⋅ ⋅ γl l γl+1

→

G

󳨃→

l+1 g1 1 ⋅ ⋅ ⋅ gl l gl+1 .

n

n

n

n

n

n

m

m

m

l+1 We need to show that φ ∈ Hom(Γ, G). Let γ = γ1 1 ⋅ ⋅ ⋅ γl l γl+1 and γ ′ = γ1 1 ⋅ ⋅ ⋅ γl l γl+1l+1 in Γ. Therefore,

n

n

n

m

m

m

l+1 φ(γγ ′ ) = φ(γ1 1 ⋅ ⋅ ⋅ γl l γl+1 γ1 1 ⋅ ⋅ ⋅ γl l γl+1l+1 )

n +m1

= φ(γ1 1

n +ml m γl+1 ),

⋅ ⋅ ⋅ γl l

where m = nl+1 + ml+1 − ∑ ni mj pij . 1⩽j 0 such that Mjε

Cε

2iπzj

= (e ) ∈ π −1 (Vj ), ′ e2iπz

j ∈ {1, 2},

where 0 C2ε C =( ) Il 0 ε

and C2ε = (

0 ∇ε

0 ), 0

and ∇ε = (μij )i,j ∈ M2n−l,n (ℝ) with μ11 = ε and μij = 0 otherwise. Then the same conclusion holds when we take gε = t (0l−n ,

′ z − z2 z2 − z1 , 02n−1 , 1 , 02n−l−1 , z1 − z2 , 0n−1 , e2iπd , e2iπd ) ∈ G. ε ε

Finally, suppose that l = 2n. There exists then ε > 0 such that Cε

Mjε = (e2iπzj ) ∈ π −1 (Vj ), ′ e2iπz

j ∈ {1, 2},

where 0 C ε = (C2ε ) Il

0 ε

and C2ε = (

0 ). 0

Then the same conclusion holds when we take gε = t (0n−1 ,

′ z − z2 z2 − z1 , 02n−1 , 1 , z1 − z2 , 0n−1 , e2iπd , e2iπd ) ∈ G. ε ε

7.3 Reduced threadlike groups From now on and unless a specific mention g designates the threadlike Lie algebra of dimension n + 1, n ≥ 2 as defined in Subsection 1.1.3. Recall the subspace g0 = ℝ-span{Y1 , . . . , Yn }, which is a one-codimensional Abelian ideal of g. The center z(g) of g is however one-dimensional and it is the space ℝ-span{Yn }. The reduced thread̃ by the central discrete subgroup exp ̃ (ℤY ). The like Lie group G is the quotient of G n G

7.3 Reduced threadlike groups | 367

corresponding exponential map exp := expG is given by exp(xX + y1 Y1 + ⋅ ⋅ ⋅ + yn Yn ) = (x, y1 , . . . , yn−1 , e2iπyn ),

x, yj ∈ ℝ, 1 ≤ j ≤ n.

For the sake of simplicity, we identify from now on this element by the associated column vector.

7.3.1 Proper action of closed subgroups of G We look first at the case of closed connected subgroup of G. Thus, we only have to study the structure of the associated Lie subalgebra. Let then h be a p-dimensional subalgebra of g. We are going to construct a strong Malcev basis Bh of h extracted from B . Recall that a family of vectors {Z1 , . . . , Zm } is said to be a strong Malcev basis of a Lie algebra l (m = dim l) if ls = ℝ-span{Z1 , . . . , Zs } is an ideal of l for all s ∈ {1, . . . , m}. h We denote by h0 = h ∩ g0 and Ig00 = {i1 < ⋅ ⋅ ⋅ < ip0 } the set of indices i ∈ {1, . . . , n} such that h0 + gi = h0 + gi+1 , where (gi )1≤i≤n+1 is the decreasing sequence of ideals of g given by gi = ℝ-span{Yi , . . . , Yn },

i = 1, . . . , n

and

gn+1 = {0}.

(7.10)

h We note for all is ∈ Ig00 , Ỹ s = Yis + ∑nr=is +1 αr,s Yr ∈ h. We get therefore that, if h ⊂ g0 , then Bh = {Ỹ 1 , . . . , Ỹ p0 }. Otherwise, there exists X̃ = X + ∑nr=1 xr Yr ∈ h. Hence Bh = {X,̃ Ỹ 1 , . . . , Ỹ p0 }. We denote by Mh,B ∈ Mn+1,p (ℝ) the matrix of Bh written in the basis B . Then Lemma 1.1.13 asserts that there exists a basis B = {X, X1 , . . . , Xn } of g such that

[X, Xi ] = Xi+1 ,

[Xi , Xj ] = 0,

i = 1, . . . , n − 1, i, j = 1, . . . , n

and h = ℝ-span{X, Xn−p+2 , . . . , Xn }. This will be of use in the sequel and permits to have a particular form of the matrix Mh,B . We also opt for the notation G0 = exp(g0 ) and Z(G) = exp(z(g)). The following immediate result will be of several uses. Lemma 7.3.1. Let Γ be a closed Abelian subgroup of G. Then for any exp T1 , exp T2 ∈ Γ, we have [T1 , T2 ] ∈ ℤYn . ̃ of G, we can easily check that Proof. Using the universal covering G expG̃ T1 expG̃ T2 = expG̃ T2 expG̃ T1 expG̃ (sYn )

368 | 7 Discontinuous actions on reduced nilmanifolds for some s ∈ ℤ. We get therefore that T1 − eadT2 (T1 ) = −([T2 , T1 ] + ⋅ ⋅ ⋅ +

1 [T , [. . . , [T2 , T1 ], . . .]]) = sYn . (n − 1)! 2

Fixing a sequence of ideals as in equation (7.10) and suppose that [T2 , T1 ] ∈ gi \ gi+1 , we get that T1 − eadT2 (T1 ) ∈ gi \ gi+1 . This is enough to conclude. We now look at the point to establish a criterion on the triple (G, H, L) such that the action of L on G/H is proper or free. Here, H and L designate closed connected subgroups of G. The main result of this subsection is the following. Theorem 7.3.2. Let G = exp g be a reduced threadlike Lie group, H = exp h and L = exp l some closed, connected subgroups of G. We have the following: (1) If the action of L on G/H is free, then Adg h ∩ l = {0} for any g ∈ G. (2) Assume that one of the subgroups H or L contains the center of G. Then: (i) The action of L on G/H is free if and only if Adg h ∩ l = {0} for any g ∈ G. (ii) The action of L on G/H is proper if and only if l ∩ Adg h ⊆ z(g) for any g ∈ G. Proof. (1) Assume that the action of L on G/H is free and suppose that there exist V ∈ l \ {0}, U ∈ h and g ∈ G such that V = Adg U. Then exp V = g(exp U)g −1 = e. This leads to the fact that U = V = pYn for some p ∈ ℤ∗ . Then Z(G) ⊂ H ∩ L, which contradicts the free action of L on G/H. (2) Suppose now that Z(G) ⊂ L. To prove assertion (i), it suffices to show that for all g ∈ G, we have exp(l ∩ Adg h) = L ∩ gHg −1 . Let g ∈ G. It is clear that exp(l ∩ Adg h) ⊂ L ∩ gHg −1 . Let now a ∈ L ∩ gHg −1 . Then there exist T ∈ g, V ∈ l and U ∈ h such that a = exp V = exp T exp U exp(−T). This implies that V = Adexp T U mod(z(g)). As z(g) ⊂ l, then Adexp T U ∈ Adexp T h ∩ l, which implies that a ∈ exp(l ∩ Adexp T h). The proof of assertion (ii) will be divided through the following lemmas. Lemma 7.3.3. Let G be a reduced threadlike Lie group. The surjective projection defined by πG : G t

󳨀→

(w, e

2iπc

)

󳨃󳨀→

G/Z(G) = G t

w

is closed and proper. Proof. Let F be a closed subset of G, then πG−1 (πG (F)) = FZ(G).

(7.11)

7.3 Reduced threadlike groups |

369

This gives that πG (F) is closed in G. Let now S be a compact set of G and t (uj , e2iπzj )j∈ℕ a sequence of πG−1 (S). Then (uj )j∈ℕ is a sequence of S. Therefore, we can extract a subsequence (ujs )s∈ℕ , which converges to u ∈ S. Moreover, we can extract from t (0n , e2iπzjs )s∈ℕ a sequence, which converges to t (0n , e2iπz ) and which we denote t (0n , e2iπzjs )s∈ℕ . The resulting sequence t (ujs , e2iπzjs )s∈ℕ converges to t (u, e2iπz ) ∈ πG−1 (S). Lemma 7.3.4. Let G be a reduced threadlike Lie group, H and L some closed, connected subgroups of G. Assume that one of the subgroups H or L contains the center of G, then L acts properly on G/H if and only if L acts properly on G/H, where L = πG (L) and H = πG (H). Proof. Assume, for example, that Z(G) ⊂ L. Suppose first that L acts properly on G/H and let S be a compact set of G. Then πG−1 (S) is a compact set of G and then

πG−1 (S)H(πG−1 (S))−1 ∩ L is compact in G. As Z(G) ⊂ L, we can easily see that SHS compact in G. Conversely, let S be a compact set of G. It is not hard to see that

−1

∩ L is

−1

SHS−1 ∩ L ⊂ SHS−1 Z(G) ∩ L = πG−1 (πG (S)H(πG (S)) ) ∩ πG−1 (L) = πG−1 (πG (S)H(πG (S))

−1

∩ L),

which is compact in G. Hence SHS−1 ∩ L is compact. To conclude, it suffices to use Proposition 2.3.1.

7.3.2 Hom(Γ, G) for an Abelian discrete subgroup Γ Let G be the reduced threadlike Lie group and Γ be a discrete Abelian nontrivial subgroup of rank k ∈ ℕ of G. Recall the set of all injective group homomorphisms Hom0d (Γ, G) from Γ to G, which will be of interest in the next section, merely because it is involved in deformations and it is viewed as a starting means to study the parameter and the deformation spaces. We will next construct a family of infinite order generators of Γ, which we denote by {γ1 , . . . , γk } whenever k ≥ 1, and γk+1 a finite order (central) element of Γ if Γ is not torsion-free. More precisely, if Γ is torsion-free, then Γ = ⟨γ1 , . . . , γk ⟩. n

n

This notation means that γ ∈ Γ if and only if γ = γ1 1 ⋅ ⋅ ⋅ γk k for some n1 , . . . , nk ∈ ℤ. In the case where Γ is not torsion-free, we get Γ = ⟨γ1 , . . . , γk ⟩ ⊕ ⟨γk+1 ⟩.

370 | 7 Discontinuous actions on reduced nilmanifolds Let us first give in the following two lemmas the explicit the expression of the torsionfree part of Γ. Lemma 7.3.5. If Γ ⊂ G0 , then there exist a linearly independent family {w1 , . . . , wk } of ℝn−1 and v1 , . . . , vk ∈ ℝ such that for any j ∈ {1, . . . , k}, γj = t (0, wj , e2iπvj ). Proof. We consider the surjective projection πG0 : G0 t

G0 /Z(G)

󳨀→

(0, w, e

2iπv

)

t

󳨃󳨀→

(0, w).

Then πG0 (Γ) is a discrete subgroup of G0 /Z(G) ≃ ℝn−1 . This gives that there exist a unique nonnegative integer kΓ and a family {w1 , . . . , wkΓ } of linearly independent vectors of ℝn−1 such that n1

nkΓ

0 0 ) πG0 (Γ) = {(t ) ⋅ ⋅ ⋅ (t wkΓ w1

: n1 , . . . , nkΓ ∈ ℤ} .

The integer kΓ is nothing but the rank k of Γ. Then, for all j ∈ {1, . . . , k}, there exists vj ∈ ℝ such that γj = t (0, wj , e2iπvj ) ∈ Γ. Hence we have that n

n

Γ = {γ1 1 ⋅ ⋅ ⋅ γk k : n1 , . . . , nk ∈ ℤ} ⋅ (Γ ∩ Z(G)). Lemma 7.3.6. If Γ ⊄ G0 , then k ≤ 2 and we can take γ1 as γ1 = t (a, w, e2iπv ), where w ∈ ℝn−1 , a ∈ ℝ∗ and v ∈ ℝ. Moreover, if k = 2, we can take γ2 as γ2 = t (0n−1 , α, e2iπv ), ′

with 0n−1 = (0, . . . , 0) ∈ ℝn−1 , α ∈ ℝ∗ and v′ ∈ ℝ. Proof. We consider the surjective projection defined in equation (7.11). First of all, it is clear that G is a connected and simply connected threadlike Lie group and its associated Lie algebra is given by g := g/z(g). We denote by ρg : g → g/z(g) the canonical surjection, then clearly expG ∘ρg = πG ∘ expG . On the other hand, we denote by Γ = πG (Γ) which is a discrete Abelian subgroup of G and L = expG l its corresponding syndetic hull. Since Γ ⊄ G0 , there exists T ∈ g \ g0 such that exp T ∈ Γ. We get that T := ρg (T) ∈ l. As l is Abelian, there exists α ∈ ℝ such that l = ℝT ⊕ ℝ(αY n−1 ).

7.3 Reduced threadlike groups | 371

We get therefore that Γ = expG (ℤT) expG (ℤ(αY n−1 )) and then that Γ = exp(ℤT) exp(ℤ(αYn−1 + vYn )) ⋅ (Γ ∩ Z(G)) for some v ∈ ℝ. We obtain kΓ = dim l, which is unique according to Theorem 1.4.12. Now using the fact that Γ ∩ Z(G) is a finite central subgroup of G, we get immediately the following. Lemma 7.3.7. If Γ is not torsion-free, then γk+1 = t (0n , e

2iπ q

),

where q ∈ ℕ∗ is the cardinality of Γ ∩ Z(G). Suppose from now on that Γ is an Abelian discrete subgroup of rank k and having a family of generators γ1 , . . . , γl defined as in Lemmas 7.3.5, 7.3.6 and 7.3.7. Here, k

if Γ is torsion-free,

k+1

otherwise.

l={

Now, we are ready to give an explicit description of Hom(Γ, G) and Hom0d (Γ, G). Recall first that Hom(Γ, G) is endowed with the pointwise convergence topology. The same topology is obtained by using the injective map Hom(Γ, G) → G × ⋅ ⋅ ⋅ × G,

φ 󳨃→ (φ(γ1 ), . . . , φ(γl ))

to equip Hom(Γ, G) with the relative topology induced from the direct product Gl = G × ⋅ ⋅ ⋅ × G. Having in mind the identification of any element exp(xX + ∑ni=1 yi Yi ) ∈ G by the column vector t (x, y1 , . . . , yn−1 , e2iπyn ), we identify the product set Gl by the set of matrices in Mn+1,l (ℂ) defined by C ) ∈ Mn+1,l (ℂ) : C ∈ Mn,l (ℝ), z ∈ ℝl } e2iπz

Mn+1,l = {(

for z = (z1 , . . . , zl ), e2iπz := (e2iπz1 , . . . , e2iπzl ) ∈ 𝕋l . Then it appears clear that the following map: Ψ : Hom(Γ, G) φ

→

Mn+1,l

󳨃→

(φ(γ1 )

⋅⋅⋅

φ(γl ))

(7.12)

372 | 7 Discontinuous actions on reduced nilmanifolds is a homeomorphism on its image. Our task is reduced to give the description of Ψ(Hom(Γ, G)) and Ψ(Homd0 (Γ, G)), which will be denoted respectively by H and K . Toward such a purpose, we consider the following discussion. The torsion-free case Throughout the present subsection, Γ denotes a torsion-free Abelian discrete subgroup of G. A preliminary algebraic interpretation of H is given by the following: { { { C H = {( 2iπz ) ∈ Mn+1,k { { e {

󵄨󵄨 C = (C ⋅ ⋅ ⋅ C ) , 󵄨󵄨 } 1 k } 󵄨󵄨 } 󵄨󵄨 . 󵄨󵄨 C C C C u v v u } 󵄨󵄨 ( 2iπz ) ( 2iπz ) = ( 2iπz ) ( 2iπz ) , 1 ≤ u, v ≤ k } } 󵄨󵄨 e u e v e v e u 󵄨 }

Indeed, for φ ∈ Hom(Γ, G) we have φ(γu )φ(γv ) = φ(γv )φ(γu ),

1 ≤ u, v ≤ k.

Thus, we get Ψ(Hom(Γ, G)) ⊂ H . Let conversely M=(

C1 e2iπz1

Ck )∈H, e2iπzk

⋅⋅⋅ ⋅⋅⋅

we can define a group homomorphism: φ : Γ → G satisfying Cj ). e2iπzj

φ(γj ) = (

The next step consists in giving an explicit description of H . Toward such a purpose, we define the sets Hj,k , j ∈ {0, . . . , k} as follows: 0k { } { } H0,k = {( N ) ∈ Mn+1,k : N ∈ Mn−1,k (ℝ), z ∈ ℝk } { 2iπz } { e } and for any j ∈ {1, . . . , k},

Hj,k

{ λ1 t c { { = {( y1 { { e2iπz1 {

⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅

λk t c yk ) ∈ Mn+1,k 2iπzk e

󵄨󵄨 ∗ n−2 󵄨󵄨 λj = 1, c ∈ ℝ × ℝ , } } 󵄨󵄨 } 󵄨󵄨 . 󵄨󵄨 λs , ys , zs ∈ ℝ, s ∈ {1, . . . , k}, } } 󵄨󵄨󵄨 } 󵄨󵄨 (λ y − λ y )ρ (c) ∈ ℤ, u, v ∈ {1, . . . , k} v u 1 󵄨 u v }

Here, ρj designates the projection map from ℝn−1 on ℝ given by ρj (c1 , . . . , cn−1 ) = cj ,

1 ≤ j ≤ n − 1.

7.3 Reduced threadlike groups | 373

Proposition 7.3.8. With the same notation and hypotheses, we have k

H = ⋃ Hj,k . j=0

Proof. First, it is clear that ⋃kj=0 Hj,k ⊂ H . Conversely, let C1 ⋅ ⋅ ⋅ Ck )∈H. e2iπz

M=(

Let for all 1 ≤ j ≤ k, t Cj = (cj , c1,j , . . . , cn−1,j ). Then for any u, v ∈ {1, . . . , k}, we get by Lemma 7.3.1, n−1

n−1

j=1

j=1

[cu X + ∑ cj,u Yj , cv X + ∑ cj,v Yj ] ∈ ℤYn , which gives rise to the following equations: cu cj,v − cv cj,u = 0, u, v ∈ {1, . . . , k}, j ∈ {1, . . . , n − 2}, { cu cn−1,v − cv cn−1,u ∈ ℤ, u, v ∈ {1, . . . , k}.

(7.13)

If (c1 , . . . , ck ) = 0k , then M ∈ H0,k . Otherwise, there exists j0 ∈ {1, . . . , k} such that cj0 ≠ 0. Then by equations (7.13), we have that M ∈ Hj0 ,k . Next, we give the description of K according to the rank of Γ. We denote for all j ∈ {0, . . . , k} by Kj,k := Ψ(Hom0d (Γ, G)) ∩ Hj,k . Then k

K = ⋃ Kj,k . j=0

Our task is then reduced to the determination of those layers according to the value of k. Case 1: k ≥ 3 Proposition 7.3.9. Assume that Γ is a torsion-free Abelian discrete subgroup of G of rank k ≥ 3. Under the above notation, we have K = K0,k

0k { } { } 0 = {( N ) ∈ H0,k : N ∈ Mn−1,k (ℝ)} , { 2iπz } { e }

0 with Mn,m (ℝ) denotes the set of all (n, m) matrices of maximal rank.

374 | 7 Discontinuous actions on reduced nilmanifolds Proof. Let {u1 , . . . , uk } be a linearly independent family of ℝn and v1 , . . . , vk ∈ ℝ such that γj = t (uj , e2iπvj ). Let 0k ( N ) = Ψ(φ) ∈ K0,k e2iπz such that N = (N1 ⋅ ⋅ ⋅ Nk ). Suppose that rk N < k, as φ(Γ) is discrete, there exist a set of distinct integers J = {j0 , j1 , . . . , jk′ } ⊊ {1, . . . , k} and η1 , . . . , ηk′ ∈ ℚ∗ such that ′ Nj0 = ∑ks=1 ηs Njs . We denote for s ∈ {1, . . . , k ′ }, α ηs = s , βs

k′

b = ∏ βs ,

bs =

s=1

b , βs

and bα

b ′ αk′

γ = γj−b γj 1 1 ⋅ ⋅ ⋅ γj k′ 0 1

As rk(uj0

⋅⋅⋅

k

.

uj ′ ) = k ′ + 1, we have γ ≠ e. Moreover, it is not hard to see that k φ(γ) = t (0n , e2iπb(η1 zj1 +⋅⋅⋅+ηs zjs −zj0 ) ) ∈ φ(Γ) ∩ Z(G).

This gives that η1 zj1 + ⋅ ⋅ ⋅ + ηs zjs − zj0 ∈ ℚ. Let (u, v) ∈ ℤ × ℤ∗ such that η1 zj1 + ⋅ ⋅ ⋅ + ηs zjs − zj0 =

u , v

we have φ(γ v ) = e, which contradicts the fact that φ is injective. Suppose now that there exists j ∈ {1, . . . , k} such that Kj,k ≠ 0 and let λ1 t c ( y1 e2iπz1

⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅

λk t c yk ) = Ψ(φ) ∈ Kj,k . e2iπzk

Suppose that there exist s such that ys − λs yj = 0. If λs ∈ ̸ ℚ, then there exists a ∈ ℝ \ (ℤ + λs ℤ) and a sequence (mu + λs m′u )u∈ℕ in ℤ + λs ℤ, which converges to a. Thus, we m

m′

can extract from (φ(γj u γs u ))u∈ℕ a convergent (hence a stationary as φ(Γ) is discrete) subsequence. This shows that a = mu0 + λs m′u0 , for some u0 ∈ ℕ, which contradicts

7.3 Reduced threadlike groups | 375

the fact that a ∈ ̸ ℤ + λs ℤ. It follows therefore that λs ∈ ℚ. Let (α, β) ∈ ℤ × ℤ∗ be such that λs = αβ . By an easy computation, we get φ(γjα γsβ ) = t (0n , e2iπβ(λzj +zs ) ) ∈ φ(Γ) ∩ Z(G) and conclusively λs zj − zs ∈ ℚ. Let (α′ , β′ ) ∈ ℤ × ℤ∗ such that λs zj − zs =

m1 = −αβ and m2 = ββ ≠ 0, we have ′

′

α′ , β′

then for

m

e ≠ γj 1 γsm2 ∈ Ker φ, which contradicts the injectivity of φ. This gives that λs yj − ys ≠ 0, s ∈ {1, . . . , k} \ {j}. We denote by ps = (ys − λs yj )ρ1 (c) ∈ ℤ∗ , then we get (λs ys′ − λs′ ys )ρ1 (c) = λs ps′ − λs′ ps ∈ ℤ∗ . A routine computation shows that for s, s′ ∈ {1, . . . , k} \ {j} such that s ≠ s′ , we have −ps′ (λs ps′ −λs′ ps ) ) γj

p

φ(γs′s γs

∈ φ(Γ) ∩ Z(G)

this gives rise to ps zs′ − ps′ zs + (λs ps′ − λs′ ps )zj =

α β

for some (α, β) ∈ ℤ × ℤ∗ . Hence p

−ps′ (λs ps′ −λs′ ps ) β ) γj

(γs′s γs

∈ Ker φ.

Thus, we have Kj,k = 0 for all j ∈ {1, . . . , k}. Case 2: k = 2 We argue similarly as in the previous proposition to prove the following result. Proposition 7.3.10. Let G be a reduced threadlike Lie group and Γ a torsion-free Abelian discrete subgroup of G of rank 2. The sets Kj,2 , j = 0, 1, 2, are given by 02 { } { } 0 K0,2 = {( N ) ∈ H0,2 : N ∈ Mn−1,2 (ℝ)} { 2iπz } { e } and for all j ∈ {1, 2}, Kj,2

λ1 t c { { = {( y1 { 2iπz1 { e

λ2 t c } } y2 ) ∈ Hj,2 : (λ1 y2 − λ2 y1 ) ∈ ℝ∗ } . } e2iπz2 }

376 | 7 Discontinuous actions on reduced nilmanifolds Case 3: k = 1 The description of Kj,1 is similar to the previous case. Using the same analysis as in Proposition 7.3.9, we obtain the following result. Proposition 7.3.11. We keep the same notation. The sets Kj,1 , j = 0, 1, are given by 0 { } { t ∗} = {( n ) ∈ H0,1 : n ∈ (ℝn−1 ) } { 2iπz } { e }

K0,1 and

K1,1 = H1,1 . The torsion case Suppose that Γ is an Abelian discrete subgroup of G, which admits a finite order element. Using the injective map Ψ defined in equation (7.12), the representative matrix of any element of Hom(Γ, G) admits an additional column, which represents the finite order element. It appears immediate that when restricted to this case, the same analysis as in Subsection 7.3.2 takes place to describe the sets Hom(Γ, G) just by adding the aforementioned column. In order to avoid the redundance, let us define the sets ̃j,k , j = 0, . . . , k, as follows: H 0k { { ̃ H0,k = {( N { 2iπz { e

t

0 } } 0n−1 ) ∈ Mn+1,k+1 : N ∈ Mn−1,k (ℝ), z ∈ ℝk , r ∈ ℤ} } 2iπr e q }

and for any j ∈ {1, . . . , k},

̃j,k H

{ { { λ1 t c { { { = {( y1 { { 2iπz1 { { { e {

⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅

λk t c yk

e2iπzk

t

0n−1 0 ) ∈ Mn+1,k+1

e

2iπr q

󵄨󵄨 λ = 1, c ∈ ℝ∗ × ℝn−2 , 󵄨󵄨 j } } 󵄨󵄨 } } 󵄨󵄨 } } 󵄨󵄨 λs , ys , zs ∈ ℝ, r ∈ ℤ, 󵄨󵄨 󵄨󵄨 }. 󵄨󵄨 (λu yv − λv yu )ρ1 (c) ∈ ℤ, } } } 󵄨󵄨 } } 󵄨󵄨 󵄨󵄨 s, u, v ∈ {1, . . . , k} }

The following proposition stems immediately from Proposition 7.3.8, which describes the structure of Hom(Γ, G). Indeed, the last added vector of Ψ(φ), φ ∈ Hom(Γ, G), takes the following form: t 0n r ∈ Z(G), φ(γk+1 ) = ( 2iπr ) = γk+1 e q

for some r ∈ ℤ, being the image by a group homomorphism of the finite order element γk+1 of Γ ∩ Z(G).

7.3 Reduced threadlike groups | 377

Proposition 7.3.12. Let G be a reduced threadlike Lie group and Γ an Abelian discrete subgroup of G of rank k with a finite order element. The set Hom(Γ, G) is homeomorphic ̃j,k . to ⋃kj=0 H Take now φ ∈ Hom0d (Γ, G), we get that q ∧ r = 1 where the symbol ∧ means the greatest common divisor. It appears clear that when k = 0, Γ is a central and cyclic subgroup and then Hom(Γ, G) is homeomorphic to Γ. Furthermore, Hom0d (Γ, G) = Aut(Γ),

(7.14)

where the last means the automorphism group of Γ, which is a finite group. Suppose now that k > 0. First, it is easy to see that the description of K is similar to the torsionfree case, using the same analysis as in previous subsection. The analogues of the ̃j,k := Ψ(Hom0 (Γ, G)) ∩ H ̃j,k are as follows. For j = 0, objects Kj,k , j ∈ {0, . . . , k}, given by K d we have 0k { { ̃0,k = ( N K { { 2iπz { e

t

0 } } ̃0,k : N ∈ M 0 (ℝ), 0 < r < q, r ∧ q = 1 . 0n−1 ) ∈ H n−1,k } } 2iπr e q }

Let now j ∈ {1, . . . , k}. If k ≥ 3, then ̃j,k = 0. K Otherwise, we get the following discussion. When k = 2, we have for j ∈ {1, 2}: λ1 t c { { ̃j,2 = ( y1 K { { 2iπz 1 { e

λ2 t c y2

t

e2iπz2

0n−1 } } ̃j,2 : (λ1 y2 − λ2 y1 ) ∈ ℝ∗ , 0 < r < q, r ∧ q = 1 . 0 )∈H } } 2iπr e q }

Finally, when k = 1, we have ̃1,1 = {( K

t

e

c

2iπz

t

0n

e

2iπr q

̃1,1 : 0 < r < q, r ∧ q = 1} . )∈H

Now we can state the following proposition, which will be used later. Proposition 7.3.13. Let G be a reduced threadlike Lie group and Γ a discrete Abelian subgroup of G. The set Hom0d (Γ, G) is open in Hom(Γ, G). Proof. We only treat the case where Γ is torsion-free, the other case is handled similarly. Whenever k ≤ 2, the result is immediate. In fact, it suffices to see that the set 0

M = {(

C 0 ) ∈ Mn+1,k : C ∈ Mn,k (ℝ)} e2iπz

378 | 7 Discontinuous actions on reduced nilmanifolds is open in M = {(

C ) ∈ Mn+1,k : C ∈ Mn,k (ℝ)} e2iπz

and, therefore, K = H ∩ M 0 is open in H ∩ M = H . Thus, we only have to treat H

the case when k > 2. We prove that H \ K is closed in H . Let M ∈ H \ K , there exists therefore a sequence (Ms )s∈ℕ belongs to H \ K and converges to M. Then we can extract from (Ms )s∈ℕ a subsequence (Mσ(s) )s∈ℕ of elements in Hj,k \ Kj,k for some j ∈ {0, . . . , k}. Suppose first that j = 0, writing 0k Mσ(s) = ( Ns ) , e2iπzs we get that 0k M = ( N ) ∈ H0,k \ K0,k e2iπz as rk(N) ≤ rk Ns < k. Suppose now that j ∈ {1, . . . , k}. We get therefore Mσ(s)

λ1,s t cs = ( y1,s e2iπz1,s

⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅

λk,s t cs yk,s ) . 2iπzk,s e

If there exists j0 ∈ {1, . . . , k} such that lims→+∞ ρ1 (λj0 ,s cs ) is not zero, then M ∈ Hj0 ,k . Otherwise, M ∈ H0,k . Let then N = (N1 ⋅ ⋅ ⋅ Nk ) ∈ Mn−1,k (ℝ) and z ∈ ℝk such that 0k M = ( N ). e2iπz If there exists j0 ∈ {1, . . . , k} such that λj0 ,s goes to infinity as s goes to +∞, then we can easily check that lims→+∞ cs = 0n−1 . Moreover, we can easily check that the sequence ρ1 (cs )(λj0 ,s yj,s − yj0 ,s ) of ℤ is stationary and equals to zero. We get therefore that lims→+∞ yj,s = 0 and then Nj = t 0n−1 , which is enough to conclude. Suppose finally that for all j′ ∈ {1, . . . , k}, λj′ ,s goes to λj′ ∈ ℝ as s goes to +∞. Thus, λj′ ,s yj,s = yj′ ,s . It follows therefore that rk(N) ≤ 1.

7.3 Reduced threadlike groups | 379

7.3.3 Parameter and deformation spaces Let H = exp h be a closed connected subgroup of G and Γ an Abelian discontinuous subgroup for G/H. As shows the title, the purpose of this section is to give an explicit description of the parameter and the deformation spaces of the action of Γ on G/H. We start by the following result. Proposition 7.3.14. Let G be a reduced threadlike Lie group, H = exp h a closed, connected group of G and Γ an Abelian discontinuous subgroup for G/H. We have 0

R (Γ, G, H) = {φ ∈ Homd (Γ, G) : lφ ∩ Adg h ⊆ z(g) for any g ∈ G},

where lφ is the Lie subalgebra associated to the synthetic hull of φ(Γ). Proof. We first show that the proper action of φ(Γ) on G/H implies its free action. It is clear that the proper action implies that the triplet (G, H, φ(Γ)) is (CI), which gives that for all g ∈ G, the subgroup K := φ(Γ) ∩ gHg −1 is central and then finite as φ(Γ) is discrete. As the map φ : Γ → φ(Γ) is a group isomorphism and K is finite and cyclic, we get that φ−1 (K) = K. Therefore, K ⊂ Γ ∩ gHg −1 = {e}, for all g ∈ G. Thus, the action of φ(Γ) on G/H is free. As Lφ = exp lφ contains φ(Γ) cocompactly, we get that 0

R (Γ, G, H) = {φ ∈ Homd (Γ, G) : Lφ acts properly on G/H}.

Now, Theorem 7.3.2 allows us to conclude. Thanks to Theorem 7.3.2, one sees that if Γ acts properly on G/H then p ≤ n − k + dim(h ∩ z(g)). We assume therefore that this previous condition is satisfied. Moreover, we fix from now on a basis B = {X, Y1 , . . . , Yn } of g with nontrivial Lie brackets defined in (1.7) and supposed to be adapted to h when h ⊄ g0 as in Lemma 1.1.13. Consider the following notation: Mh,B = (

1 Mh,B ), zh

zh ∈ ℝp

x 1 Mh,B = ( 0h ) , Mh,B

xh ∈ ℝp .

and

Now, we consider the action of G on Mn+1,l defined as follows. For g ∈ G, t Cj ∈ ℝn , zj ∈ ℝ, j ∈ {1, . . . , l}: C1 g ⋆ ( 2iπz e 1

⋅⋅⋅ ⋅⋅⋅

Cl C1 ) = (g ( 2iπz ) g −1 e2iπzl e 1

⋅⋅⋅

Cl g ( 2iπz ) g −1 ) . e l

(7.15)

380 | 7 Discontinuous actions on reduced nilmanifolds Lemma 7.3.15. The map Ψ defined in (7.12) is G-equivariant. That is, for any φ ∈ Hom(Γ, G) and g ∈ G, we have Ψ(g ⋅ φ) = g ⋆ Ψ(φ). Proof. It is sufficient to see that for any φ ∈ Hom(Γ, G) and g ∈ G: g ⋆ Ψ(φ) = (gφ(γ1 )g −1 , . . . , gφ(γl )g −1 ) = (g ⋅ φ(γ1 ), . . . , g ⋅ φ(γl )) = Ψ(g ⋅ φ).

We consider then the orbit space T = R /G.

In light of Proposition 7.3.14 and Lemma 7.3.15, the following result is immediate. Proposition 7.3.16. Let G be a reduced threadlike Lie group, H = exp h a closed, connected group of G and Γ an Abelian discontinuous subgroup for G/H. The parameter space is homeomorphic to R = {(

C 1 ) ∈ K : rk(C ⋒ MAd ) = k + p − dim(h ∩ z(g)) for any g ∈ G} , g h,B e2iπz

where the symbol ⋒ merely means the concatenation of matrices. Moreover, the deformation space T (Γ, G, H) is homeomorphic to T . In the sequel, we fix some notation and we define the necessary ingredients, which will be used in the description of T . First, we give the explicit expression of the action of G on Mn+1,l . Let then g = t (x, x1 , . . . , xn−1 , e2iπxn ) ∈ G, C ∈ Mn,l (ℝ) and z ∈ ℝl , we have Bn (g)C C ) = ( 2iπ(z+P ), n (g)C) e2iπz e

g⋆(

(7.16)

where Pn (g) = (Pn,1 (g) ⋅ ⋅ ⋅ Pn,n (g)) satisfies n−1 xs−1 x , s! n−s

{Pn,1 (g) = − ∑s=1 { xn−j+1 {Pn,j (g) = (n−j+1)! ,

j = 2, . . . , n

(7.17)

and 1 Qn (g)

Bn (g) = (

0 ) ∈ Mn,n (ℝ). An−1 (x)

(7.18)

7.3 Reduced threadlike groups | 381

Here, Qn (g) = t (Q1,n (g) ⋅ ⋅ ⋅ Qn−1,n (g)) ∈ Mn−1,1 (ℝ) is such that Q1,n (g) = 0,

{

j−1 xs−1 x , s! j−s

j = 2, . . . , n − 1

Qj,n (g) = − ∑s=1

(7.19)

and

( ( An−1 (x) = ( (

1

0

x

1

x2 2

x ..

.. .

xn−2 ( (n−2)!

⋅⋅⋅ .. . 1

.

⋅⋅⋅

..

.

x2 2

.

0 .. . .. ) ) ∈ Mn−1,n−1 (ℝ). .) )

.

0

⋅⋅⋅ .. .. x

(7.20)

1)

Now, we define the following sets. For u, v ∈ ℕ∗ satisfying 1 ≤ u ≤ n and 1 ≤ v ≤ l, { { { { { { C Mn+1,l (u, v) = {( 2iπz ) ∈ Mn+1,l (ℂ) { e { { { { {

󵄨󵄨 l 󵄨󵄨 C = (cr,s ) ∈ Mn,l (ℝ), z ∈ ℝ , } } 󵄨󵄨 } } 󵄨󵄨 } 󵄨󵄨 cr,s = 0, 1 ≤ r ≤ u − 1, 1 ≤ s ≤ l, } 󵄨󵄨 . 󵄨󵄨 } } 󵄨󵄨 cu,s = 0, 1 ≤ s ≤ v − 1, } } 󵄨󵄨 } } 󵄨󵄨 󵄨󵄨 cu,v ≠ 0 }

If u ≤ n − 1, we put C ) ∈ Mn+1,l (u, v) : c(u+1),v = 0} . e2iπz

′

Mn+1,l (u, v) = {(

Otherwise, ′

Mn+1,l (n, v) = {(

e

C

2iπz )

∈ Mn+1,l (n, v) : ρv (z) = 0} .

The next step consists in giving an explicit description of R and T in order to have comprehensive details about the parameter and the deformation spaces. For such a purpose, we can divide the task into two parts as in the previous section. The torsion-free case We consider the following notation. For all k ∈ {1, . . . , n} and j ∈ {0, . . . , k}, Rj,k := Ψ(R (Γ, G, H)) ∩ Kj,k , R0,k (u, v) := R0,k ∩ Mn+1,k (u, v)

382 | 7 Discontinuous actions on reduced nilmanifolds and Jk = {(u, v) ∈ {1, . . . , n} × {1, . . . , k} : R0,k (u, v) ≠ 0}. Case 1: k ≥ 3 Proposition 7.3.17. Assume that Γ is a torsion-free Abelian discrete subgroup of G of rank k ≥ 3. Under the above notation, we have: (1) The parameter space is homeomorphic to R = R0,k where: (i) If H ⊄ G0 , then R0,k = K0,k if p = 1. Otherwise, 0k } { } { N 0 (ℝ)} . R0,k = {( N ) ∈ K0,k : N = ( 1 ) , N1 ∈ Mn−p+1,k } { 2iπz N2 } { e (ii) If H ⊂ G0 , then R0,k

󵄨󵄨 0k 󵄨󵄨 rk(A (x)N ⋒ M 0 ) = k + p − dim(h ∩ z(g))} { { } 󵄨 n−1 h,B = { ( N ) ∈ K0,k 󵄨󵄨󵄨󵄨 . } { } 󵄨 for any x ∈ ℝ 2iπz 󵄨 󵄨󵄨 { e }

(2) For any (u, v) ∈ Jk , the set R0,k (u, v) is G-invariant and R0,k (u, v)/G is homeomorphic to ′

T0,k (u, v) := R0,k (u, v) ∩ Mn+1,k (u, v).

(7.21)

Proof. (1) As a direct consequence from Proposition 7.3.9, we have Rj,k = 0 for all j ∈ {1, . . . , k}. Now, in order to find the description of R0,k , we shall discuss the two following cases. Assume first that H ⊄ G0 . If p = 1, then H = exp(ℝX) and AdG h = ℝ(X + g2 ), where (gj )1≤j≤n+1 is the sequence defined as in equation (7.10). Hence, for any 0k ( N ) ∈ K0,k , e2iπz we have 0k N

rk (

1 )=k+1 a

t

7.3 Reduced threadlike groups | 383

for all a ∈ {0} × ℝn−2 . Then R0,k = K0,k . Suppose now that p ≥ 2, then for all g ∈ G, the 1 matrix MAd is of the form g h,B 1 0 ( (∗ ( ( ( (∗ ( ( .. (. ( .. . ∗ (

0 0 t

0n−p

0 0 .. .

1

0

∗ .. . ∗

1 ..

0 0 .. . .. .

. ⋅⋅⋅

0 .. . ∗

0 0 ) ) ) .. ) ) . ) ∈ Mn,p (ℝ). ) ) ) ) .. . 0)

⋅⋅⋅ ⋅⋅⋅

.. 1

.

(7.22)

Now, let 0k M = ( N ) ∈ K0,k e2iπz and write N N = ( 1) , N2

(7.23)

where N1 ∈ Mn−p+1,k (ℝ) and N2 ∈ Mp−2,k (ℝ). We get then 0 M ∈ R ⇔ N1 ∈ Mn−p+1,k (ℝ).

Finally, when H ⊂ G0 , the result follows immediately from Proposition 7.3.16. (2) It suffices to prove that R0,k is G-stable. Thus, the G-stability of R0,k (u, v) follows. First, it is not hard to see that the action of G on R0,k is reduced to the action of exp(ℝX). Indeed, 0k 0k { } { } G ⋆ ( N ) = {( An−1 (x)N ) : x ∈ ℝ} . { } 2iπ(z+Tn (x)N) e2iπz { e } Here, An−1 (x) ∈ Mn−1,n−1 (ℝ) is given by equation (7.20) and 1 xn−1 Tn (x) = ( (n−1)!

⋅⋅⋅

x) ∈ M1,n−1 (ℝ).

The fact that N is of maximal rank is equivalent to An−1 (x)N is also of maximal rank for any x ∈ ℝ. If furthermore we write N as in (7.23), where N1 ∈ Mn−p+1,k (ℝ) whenever

384 | 7 Discontinuous actions on reduced nilmanifolds p > 1, we get N A (x)N1 An−1 (x) ( 1 ) = ( n−p+1 ), N2 N2 (x) for some N2 (x) ∈ Mp−2,k (ℝ). Therefore, rk(An−p+1 (x)N1 ) = rk N1 ,

x ∈ ℝ.

To conclude, it suffices to see that An−1 (x)An−1 (x′ ) = An−1 (x + x ′ ),

x, x ′ ∈ ℝ.

Now, let (u, v) ∈ Jk , we show that T0,k (u, v) is a cross-section of all adjoint orbits of R0,k (u, v). Let 0k M = ( N ) ∈ R0,k (u, v). e2iπz Noting N = {(ar,s ), 1 ≤ r ≤ n − 1, 1 ≤ s ≤ k}, we get au−1,v ≠ 0. Let a

{− u,v tN,z = { aρu−1,v (z) − v { an−1,v

if u < n, if u = n.

By an easy computation, we show that G ⋆ M ∩ T0,k (u, v) = exp(tN,z X) ⋆ M. The next step consists in proving that the following map: (Φ0,k )(u,v) : R0,k (u, v)/G [M]

→ 󳨃→

T0,k (u, v)

exp(tN,z X) ⋆ M

is an homeomorphism. First, it is not hard to see that (Φ0,k )(u,v) is well-defined. In fact, let Mj ∈ R0,k (u, v), j = 1, 2 such that [M1 ] = [M2 ]. Then (Φ0,k )(u,v) ([M1 ]) = G ⋆ M1 ∩ T0,k (u, v) = G ⋆ M2 ∩ T0,k (u, v) = (Φ0,k )(u,v) ([M2 ]).

7.3 Reduced threadlike groups | 385

The injectivity of (Φ0,k )(u,v) is immediate. Indeed, for Mj ∈ R0,k (u, v), j = 1, 2 such that (Φ0,k )(u,v) ([M1 ]) = (Φ0,k )(u,v) ([M2 ]), we have G ⋆ M1 ∩ T0,k (u, v) = G ⋆ M2 ∩ T0,k (u, v) ∈ G ⋆ M1 ∩ G ⋆ M2 , which gives that [M1 ] = [M2 ]. Now, to see that (Φ0,k )(u,v) is surjective, it suffices to verify that for all M ∈ T0,k (u, v), we have (Φ0,k )(u,v) ([M]) = M. To achieve the proof, we prove that (Φ0,k )(u,v) is bicontinuous. Let (π0,k )(r,s) be the canonical surjection (π0,k )(r,s) : R0,k (r, s) → R0,k (r, s)/G. ̃ 0,k )(u,v) = (Φ0,k )(u,v) ∘ (π0,k )(u,v) , which Thus, we can easily see the continuity of (Φ is equivalent to the continuity of (Φ0,k )(u,v) . Finally, it is clear that ((Φ0,k )(u,v) )−1 = ((π0,k )(u,v) )|T0,k (u,v) , then the bicontinuity follows. Case 2: k = 2 Proposition 7.3.18. Let G be a reduced threadlike Lie group and Γ a torsion-free Abelian discrete subgroup of G. (1) The parameter space is homeomorphic to R = ⋃2j=0 Rj,2 where: (i) If H ⊄ G0 , then R0,2 = K0,2 if p = 1. Otherwise, 02 { } { } N 0 R0,2 = {( N ) ∈ K0,2 : N = ( 1 ) , N1 ∈ Mn−p+1,2 (ℝ)} { 2iπz } N2 { e } and for j ∈ {1, 2}: Rj,2

λ1 t c { { = {( y1 { 2iπz1 { e

λ2 t c } } y2 ) ∈ Kj,2 : ρ2 (c) ≠ 0} . } e2iπz2 }

(ii) If H ⊂ G0 , then R0,2

󵄨󵄨 02 󵄨󵄨 rk(A (x)N ⋒ M 0 ) = p + 2 − dim(h ∩ z(g))} { { } 󵄨 n−1 h,B = { ( N ) ∈ K0,2 󵄨󵄨󵄨󵄨 . } { } 󵄨󵄨 for any x ∈ ℝ 2iπz 󵄨 e 󵄨 { }

386 | 7 Discontinuous actions on reduced nilmanifolds Moreover, for j ∈ {1, 2}: if n − 1 ∈ Ig00 , h

0

Rj,2 = {

Kj,2

otherwise.

(2) For any (u, v) ∈ J2 , the set R0,2 (u, v) is G-invariant and R0,2 (u, v)/G is homeomorphic to T0,2 (u, v) given as in equation (7.21) and the components Rj,2 , j = 1, 2 are G-invariant and homeomorphic to t

Tj,2

λ c { { 1 = {( 0 { 1 {

λ2 t c } } 0 ) ∈ Rj,2 : ρs (c) = 0, s = 3, . . . , n − 1} . } 1 }

Proof. We opt for the same arguments as in Proposition 7.3.17 to have the description of R0,k . Now, let us give the description of Rj,2 , j = 1, 2, according to the position of H inside G. Let λ1 t c M = ( y1 e2iπz1

λ2 t c y2 ) ∈ Kj,2 . e2iπz2

In the case where H ⊄ G0 , we have ℝ(X + g2 ) ⊂ AdG h and then for all g ∈ G, the 1 matrix MAd is of the form t (1, 0, ∗, . . . , ∗) if p = 1. Otherwise, such a matrix is given g h,B as equation (7.22). Hence the assertion λ tc rk (( 1 y1

λ2 t c 1 ) ⋒ Mh,B ) = 2 + p − dim(h ∩ z(g)) y2

for any g ∈ G, is equivalent to λ tc rk (( 1 y1

1 λ2 t c ) ⋒ ( 0 )) = 3, y2 t a

for a ∈ ℝn−2 , which is in turn equivalent to ρ2 (c) ∈ ℝ∗ . h We now treat the case where H ⊂ G0 . Suppose first that n − 1 ∈ Ig00 , then there exists αn ∈ ℝ such that Yn−1 + αn Yn ∈ h. Then Mh,B is of the form A (a b

t

0n−1 1 ). αn

7.3 Reduced threadlike groups | 387

As λ1 t c y1

rk (

λ2 t c y2

t

0n−1 ) = 2, 1

we get λ tc rk (( 1 y1

λ2 t c 1 ) ⋒ Mh,B ) < p + 2 − dim(h ∩ z(g)). y2

Suppose finally that n − 1 ∈ ̸ Ig00 . Then it is not hard to see that h

λ tc rk (( 1 y1

λ2 t c 1 ) ⋒ Mh,B ) = p + 2 − dim(h ∩ z(g)). y2

This gives that Rj,2 = Kj,2 , j = 1, 2. (2) We only have to show the G-invariance of Rj,k , j = 1, 2, as for the cross-sections, we argue as in previous proposition. Using equation (7.16), we have λ1 t c G ⋆ ( y1 e2iπz1

λ1 Bn−1 (g)t c λ2 t c { { t y2 ) = {( y1 + λ1 Pn−1 (g) c { t 2iπ(z1 +Pn (g) (λ1 c,y1 )) e2iπz2 { e

λ2 Bn−1 (g)t c } } y2 + λ2 Pn−1 (g)t c ) : g ∈ G} , } t e2iπ(z2 +Pn (g) (λ2 c,y2 )) }

where Bj (g) and Pj (g), j ∈ {n−1, n} are defined in equations (7.17) and (7.18), respectively. Thus, we can easily see that for j = 1, 2, ρj (c) = ρj (t (Bn−1 (g)t c)). Case 3: k = 1 The description of the parameter and the deformation spaces is similar to the previous case. Using the same analysis as in Propositions 7.3.17 and 7.3.18, we obtain the following result. Proposition 7.3.19. We keep the same notation and hypothesis and suppose that k = 1. (1) The parameter space is homeomorphic to R = R0,1 ∪ R1,1 where: (i) If H ⊄ G0 , then R0,1 = K0,1 if p = 1. Otherwise,

R0,1

0 { } { } { } { t n1 t n−p+1 ∗ } = {( t ) ∈ K0,1 : n1 ∈ (ℝ ) }. { } n2 } { { } 2iπz e { }

388 | 7 Discontinuous actions on reduced nilmanifolds Moreover, R1,1 = {(

t

e

c

2iπz )

∈ K1,1 : ρ2 (c) ≠ 0} .

(ii) If H ⊂ G0 , then R0,1

󵄨󵄨 0 󵄨󵄨 rk(A (x)t n ⋒ M 0 ) = p + 1 − dim(h ∩ z(g))} { } { 󵄨 n−1 h,B t = { ( n ) ∈ K0,1 󵄨󵄨󵄨󵄨 } } { 󵄨󵄨 for any x ∈ ℝ 2iπz 󵄨󵄨 } { e

and R1,1 = K1,1 . (2) The components R0,1 (u, v) and R1,1 are G-invariant. Their corresponding G-orbit sets are homeomorphic to T0,1 (u, v), given as in equation (7.21) and t

c 1

T1,1 = {( ) ∈ R1,1 : ρs (c) = 0, s = 3, . . . , n} ,

respectively. The torsion case Suppose first that k = 0, then we have 0

R (Γ, G, H) = Homd (Γ, G).

(7.24)

Moreover, as Γ = ⟨γ1 ⟩ ⊂ Z(G), we get for all g ∈ G and φ ∈ R (Γ, G, H), g ⋅ φ(γ1 ) = φ(γ1 ) and then T (Γ, G, H) = R (Γ, G, H).

(7.25)

We now treat the case where k > 0. First, using the same analysis as in Propositions 7.3.9, 7.3.10 and 7.3.11, it is easy to see that the description of R and T is similar to ̃ j,k := Ψ(R (Γ, G, H)) ∩ the torsion-free case. The analogues of the objects Rj,k , given by R ̃j,k , j ∈ {0, . . . , k}, are as follows. For j = 0, if H ⊄ G0 , then K ̃ 0,k = K ̃0,k . R Otherwise, ̃ 0,k R

0k { { = {( N { 2iπz { e

󵄨󵄨 0 󵄨󵄨 rk(A (x)N ⋒ M 0 ) = k + p − dim(h ∩ z(g))} } 󵄨 n−1 h,B t ̃0,k 󵄨󵄨󵄨 0n−1 ) ∈ K . } 󵄨󵄨󵄨 for any x ∈ ℝ } 2iπr 󵄨󵄨󵄨 e q }

7.3 Reduced threadlike groups | 389

Let now j ∈ {1, . . . , k}. If k ≥ 3, then ̃ j,k = 0. R Otherwise, we get the following discussion. When k = 2, we have for j ∈ {1, 2}, λ1 t c { { = {( y1 { 2iπz 1 { e

̃ j,2 R

λ2 t c y2

e2iπz2

t

0n−1 } } ̃j,2 : ρ2 (c) ≠ 0 0 )∈K } } 2iπr e q }

if H ⊄ G0 . Otherwise, ̃ j,2 = {0 R ̃j,2 K

if n − 1 ∈ Ig00 , h

otherwise.

Finally, for k = 1, if H ⊄ G0 then ̃ 1,1 R

t c { { y = {( { 2iπz { e

t

0n−1 } } ̃1,1 : ρ2 (c) ≠ 0 . 0 )∈K } } 2iπr e q }

Otherwise, ̃ 1,1 = K ̃1,1 . R ̃ j,k )0≤j≤k as in torsion-free We remark that we have the same description of the sets (R ̃ 0,k admits a slightly different subcases except for the case where H ⊄ G0 for which R expression. In fact, using the free action of Γ on G/H, we get that H = exp(ℝX). That ̃ 0,k = K ̃0,k . Last but not least, using the same analysis of the is p = 1. We get then R previous subsection, if we denote by Jk+1 be the set of all (u, v) ∈ {1, . . . , n} × {1, . . . , k + 1} ̃ 0,k (u, v) := R ̃ 0,k ∩ Mn+1,k+1 (u, v) ≠ 0, then the G-invariance of the compofor which R ̃ ̃ ̃ 0,k (u, v)/G is nents R0,k (u, v) and (Rj,k ), j = 1, . . . , k is also required. We get then that R homeomorphic to ′

̃ 0,k (u, v) ∩ M T̃0,k (u, v) := R n+1,k+1 (u, v). ̃ j,2 /G is homeomorphic to Moreover, for k = 2, the set R λ tc { { 1 T̃j,2 = {( 0 { 1 {

λ2 t c 0 1

t

0 } } ̃ j,2 : ρs (c) = 0, s = 3, . . . , n − 1 . 0 )∈R } } 2iπr e q }

(7.26)

390 | 7 Discontinuous actions on reduced nilmanifolds ̃ 1,1 /G is homeomorphic to Finally, when k = 1, the set R t

T̃1,1 = {(

c

t

1

e

0n 2iπr q

̃ 1,1 : ρs (c) = 0, s = 3, . . . , n} . )∈R

7.3.4 The local rigidity problem Theorem 7.3.20. Let G be a threadlike Lie group, H a connected Lie subgroup of G and Γ an Abelian discontinuous group for G/H. Then Conjecture 5.7.10 holds. Proof. When G is simply connected, a positive answer to Conjecture 5.1.1 is the subject of Theorem 5.1.5. Indeed, G admits no nontrivial finite discrete subgroups. Suppose now that G is a reduced threadlike Lie group. The result is immediate when Γ is finite by equations (7.14), (7.24) and (7.25). Suppose now that Γ is infinite, that is, k ≥ 1. We have then to show that the G-orbit of any M ∈ R is not open in the parameter space. Assume first that k ≥ 3 and let 0k M = ( N ) ∈ R0,k . e2iπz The sequence (Ms )s∈ℕ∗ given by 0k Ms = (An−1 ( s1 )N ) e2iπz belongs to R \ G ⋆ M and converges to M. Now for k = 2, thus either we are in the context where M belongs to R0,2 and then the last arguments apply or λ1 t c M = ( y1 e2iπz1

λ2 t c y2 ) ∈ Rj,2 . e2iπz2

Then it suffices to consider the sequence (Ms )s∈ℕ∗ given by λ1 t cs Ms = ( y1 e2iπz1

λ2 t c y2 ) , e2iπz2

7.3 Reduced threadlike groups | 391

where cs = (c1,s , . . . , cn−1,s ) satisfies ρj (cs ) = ρj (c) for all j ∈ {1, . . . , n − 1} \ {2} and ρ2 (c) + ρ2 (cs ) = { ρ2 (c) −

1 s 1 s

if ρ2 (c) > 0, otherwise.

Finally, for k = 1, the same argument as in case k = 2 applies.

7.3.5 The stability problem As shown in the previous section, the description of the parameter space in the case when H ⊂ G0 is not so explicit. Then we are led to treat separately the cases H ⊄ G0 and H ⊂ G0 . Assume first that H ⊄ G0 and let us move on to the following discussion. Case 1: k ≥ 2 Proposition 7.3.21. Let G be a reduced threadlike Lie group, H a closed connected subgroup of G and Γ an Abelian discontinuous group for the homogeneous space G/H of rank k ≥ 2. Assume that H ⊄ G0 , then the stability holds. Proof. According to Proposition 7.3.13, it suffices to show that R (Γ, G, H) is an open set in Hom0d (Γ, G), which is equivalent to prove that R is open in K . By Propositions 7.3.9 and 7.3.17, this result is immediate when k > 2. We tackle now the case where k = 2. K Let M ∈ K \ R . Assume first that M ∈ K0,2 . Then we can find a sequence (Ms )s∈ℕ in K \ R , which converges to M. Suppose that we can extract a subsequence (Mσ(s) )s∈ℕ lying in (Kj,2 \ Rj,2 ) for some j ∈ {1, 2}. Without loss of generality, we can suppose that j = 1. Note that t

Mσ(s)

cs = ( y1,s e2iπz1,s

λs t cs y2,s ) . e2iπz2,s

We get then that the sequence ρ1 (cs )(y2,s − λs y1,s ) of ℤ converges to 0 and is stationary, which contradicts the fact that Ms ∈ K for all s ∈ ℕ. We get therefore that for all subsequence (Mσ(s) )s∈ℕ of (Ms )s∈ℕ there exists s0 ∈ ℕ such that for all s ≥ s0 we have Mσ(s) ∈ K0,2 \R0,2 , which gives that M ∈ K0,2 \R0,2 . Assume now that there exists j ∈ {1, 2} such that M ∈ Kj,2 . Then we can find a sequence in K \ R , which converges to M and for which we can extract a subsequence lying in (Kj′ ,2 \ Rj′ ,2 ) for some j′ ∈ {1, 2}, which leads to the fact that M ∈ (K1,2 ∪ K2,2 ) \ Rj,2 .

392 | 7 Discontinuous actions on reduced nilmanifolds Case 2: k = 1 Proposition 7.3.22. Let G be a reduced threadlike Lie group, H a closed, connected subgroup of G such that H ⊄ G0 and Γ an Abelian discontinuous group for the homogeneous space G/H of rank 1. If n ∈ {2, p}, then the stability holds in the parameter space. Otherwise, the set of nonstable parameters is given by 0l

{ {

} 0 }

R = {( N ) ∈ R : N = ( l′ )} . { 2iπz N } ′

{ e

}

Proof. First of all, it is not hard to see that when n = p or n = 2, the result is immediate as K = {(

t

e

a

2iπz )

∈ H : a ∈ (ℝn ) } ∗

and R = {(

e

t

a

2iπz )

∈ K : a = (a1 , . . . , an ) ∈ ℝn , a2 ∈ ℝ∗ } .

Suppose finally that n > 2 and p < n. First of all, it is not hard to see that R1,1 is open in R . Then let 0 M = ( tn ) ∈ R e2iπz K

be a nonstable point. This is equivalent to M ∈ K \ R . There exists therefore a sequence (Ms )s∈ℕ ⊂ K \ R , which converges to M. Then we can extract a subsequence lying in (K1,1 \ R1,1 ), which leads to the fact that M ∈ R ′ . Conversely, it is clear that any element 0 M = ( t n ) ∈ R′ e2iπz is a limit of a sequence (Ms )s∈ℕ∗

7.3 Reduced threadlike groups | 393

given by t

1 s

Ms = ( n ) , e2iπz which gives that M ∈ K \ R

K

.

We tackle now the case where H ⊂ G0 and k ∈ {1, 2}. Proposition 7.3.23. Let G be a reduced threadlike Lie group, H a closed, connected subgroup of G such that H ⊂ G0 and Γ an Abelian discontinuous group for the homogeneous space G/H. Then the set R \ R0,k is stable for any k ∈ {1, 2}. Proof. Suppose first that k = 2. If n − 1 ∈ Ig00 , then R \ R0,2 = 0. Otherwise, it is not hard to see that R1,2 ∪ R2,2 = K1,2 ∪ K2,2 is open in K . Now, if k = 1, then K0,1 is closed in K , which gives that K1,1 = R1,1 is open in K . h

We finally study the stability of elements of the parameter space when H ⊂ G0 and k ≥ 3. For such a purpose, we consider the following notation. Let p1 := p−dim(h∩z(g)) and for any k ∈ {1, . . . , n − p1 − 1}, we set Sk = {(i1 , . . . , ip1 +k ) ∈ ℕp1 +k : 1 ≤ i1 < ⋅ ⋅ ⋅ < ip1 +k ≤ n − 1}. We denote for any N ∈ Mn−1,k (ℝ), x ∈ ℝ, by Δ(i1 ,...,ip +k ) (N, x) the relative minor of order 1

0 k + p1 obtained by considering the rows i1 , . . . , ip1 +k of the matrix An−1 (x)N ⋒ Mh,B and

PN (x) = ∑ Δ2a (N, x), a∈Sk

we get that 0k { } { } R0,k = {( N ) ∈ K0,k : PN (x) ≠ 0 for any x ∈ ℝ} . { 2iπz } { e } We denote by d(N) the degree of PN and 0 d = max{d(N) ∈ ℕ : N ∈ Mn−1,k (ℝ)}.

Let then PN (x) = ad(N) (N)xd(N) + ⋅ ⋅ ⋅ + a0 (N),

394 | 7 Discontinuous actions on reduced nilmanifolds where (aj )0≤j≤d(N) are polynomial functions on the coefficients of N and ad(N) (N) ≠ 0. We define the following subset of R0,k by 0k { } { } (R0,k )d = {( N ) ∈ K0,k : ad (N) ≠ 0, PN (x) ≠ 0 for any x ∈ ℝ} . { 2iπz } { e } We prove that (R0,k )d is open in K . Let 0k ( N ) ∈ K0,k e2iπz such that ad (N) ≠ 0. We decompose PN as d 2

PN (x) = ad (N) ∏(x2 + αj (N)x + βj (N)) j=1

for some nontrivial functions αj , βj , which depend continuously upon the coefficients of N, when restricted if needed to a smaller set, still denoted by (R0,k )d . We get then that 0k { { } d} (R0,k )d = {( N ) ∈ K0,k : ad (N) ≠ 0, αj2 (N) − 4βj (N) < 0, 1 ≤ j ≤ } , { 2iπz 2} { e } is an open set of K0,k . In the case where k ≥ 3, K0,k = K . This means therefore that the subset of nonstable elements of R = R0,k is included in R0,k \ (R0,k )d . As we saw, it is in general difficult to characterize the set of nonstable parameters. Hence it would be very interesting to study the local stability relatively to the different layers. Definition 7.3.24. Let L = (Hi )i∈I be a finite covering of sets of Hom(Γ, G). A homomorphism φ ∈ Ri := Hi ∩ R (Γ, G, H) is said to be stable on layers with respect to the layering L , if there is an open set in Hi (which is not necessarily open in Hom(Γ, G)), which contains φ and is contained in R (Γ, G, H). The parameter space R (Γ, G, H) is said to be stable on layers with respect to the layering L if each of its elements is stable on layers with respect to L (cf. [90] for more details). Remark that the stability of the parameter space implies the stability on layers, but the converse fails to hold in general. Note also that the stability on layers holds in the case of Heisenberg groups (cf. [20]). We here provide an explicit decomposition of Hom(Γ, G) and we prove that the parameter space is stable on layers with respect to such decomposition. We now prove the following main result.

7.3 Reduced threadlike groups | 395

Theorem 7.3.25. Let G be a threadlike Lie group, H a closed, connected subgroup of G and Γ an Abelian discontinuous subgroup for G/H. There exists a covering of sets of Hom(Γ, G), which permits the parameter space to be stable on layers. Proof. Recall first that when G is simply connected, we have an analogue decomposition Hom(Γ, G) and R (Γ, G, H) (cf. Theorem 4.4.1 and Proposition 4.4.7). We can argue exactly with the same way as in reduced situation to prove that the stability on layers holds on the parameter space with respect to a adequate layering. Suppose then that G is a reduced threadlike Lie group and Γ a torsion-free Abelian discontinuous subgroup of G for G/H. From Propositions 7.3.17, 7.3.18 and 7.3.19, it is easy to check that in the cases where H ⊄ G0 the parameter space is stable on layers with respect to the layering (Hj,k )0≤j≤k given in Proposition 7.3.8. We now pay attention to the case when H ⊂ G0 . We are going to construct a refined covering of Hom(Γ, G) obtained from the decomposition given in Proposition 7.3.8. For such purpose, we keep all our notation as above and we put 0k { } { } Hj = {( N ) ∈ H0,k : d(N) = j} . { 2iπz } { e } We prove then that the parameter space is stable on layers with respect to the layering given by d

k

j=0

j=1

Hom(Γ, G) = (⋃ Hj ) ∪ (⋃ Hj,k ).

(7.27)

Remark first that Hj is G-invariant. Indeed, it suffices to see that PA(x′ )N (x) = PN (x + x ′ ) for all N ∈ Mn−1,k (ℝ) and x, x′ ∈ ℝ and then d(A(x′ )N) = d(N). Let now d(N) 2

PN (x) = ad(N) (N) ∏ (x 2 + αd(N),s (N)x + βd(N),s (N)) s=0

be the decomposition of PN where ad(N) , αd(N),s and βd(N),s are some polynomial functions depending upon the coefficients of N. We get therefore that for any j ∈ {1, . . . , m}, 0k { { } j} Rj = R ∩ Hj = {( N ) ∈ Hj : (αd(N),s )2 − 4βd(N),s < 0, s = 1, . . . , } . { 2iπz 2} { e } It follows therefore that Rj is open in Hj . Finally, it is not hard to see that Rj,k is open in Kj,k for any j ∈ {1, . . . , k}.

396 | 7 Discontinuous actions on reduced nilmanifolds

7.4 A stability theorem for non-Abelian actions We still consider in this section the setting of a reduced threadlike Lie group G. Let H be an arbitrary closed subgroup of G and Γ ⊂ G a non-Abelian discontinuous group for G/H. Unlike the setting where Γ is Abelian, we show that the stability property holds. Our main target now is the following. Theorem 7.4.1. Let G be a threadlike group, then any non-Abelian discrete subgroup of G is stable. In the case where G is simply connected, the proof of Theorem 7.4.1 is subject of Theorem 6.2.27. We thus only treat the nonsimply connected case. In the case where Γ is Abelian, the property of stability fails to hold in general and depends upon the structure and the position of H and Γ inside G (cf. Subsection 7.3.5). The proof of Theorem 7.4.1 will be provided later after introducing several other results. The first is the following. Proposition 7.4.2. Let G be a reduced threadlike Lie group and Γ a discrete non-Abelian subgroup of G such that rank(Γ) = k = p + ε with ε = 0 if Γ is torsion-free and ε = 1 otherwise. Then p ⩾ 3 and there exist γ, γq , . . . , γn−1 ∈ G such that m

m

n−1 Γ = {γ m γq q ⋅ ⋅ ⋅ γn−1 , m, mj ∈ ℤ}(Γ ∩ Z(G))

−1 with q = n − p + 1, γγj γ −1 γj−1 = γj+1 ; q ⩽ j ⩽ n − 2, γγn−1 γ −1 γn−1 = e and γi γj γi−1 γj−1 = e; q ⩽ i, j ⩽ n − 1.

Proof. We consider the surjective projection π:G t

(ω, e2iπv )

󳨀→ 󳨃󳨀→

G/Z(G) = G′ t

ω.

First of all, it is clear that G′ is a connected and simply connected threadlike Lie group and its associated Lie algebra is given by g′ := g/z(g). We denote by expG′ : g′ → G′ the associated exponential map and ρ : g → g/z(g) the canonical surjection, then clearly expG′ ∘ρ = π ∘ exp. On the other hand, we denote by Γ′ = π(Γ), which is a discrete subgroup of G′ and L′ = expG′ l′ its corresponding syndetic hull. Since Γ ⊈ G0 , there exists T ∈ g\g0 such that exp T ∈ Γ. This gives that ρ(T) ∈ l′ \ρ(g0 ). Therefore, l′ ⊈ ρ(g0 ). Let W = ρ(W) for W ∈ g. According to Lemma 1.1.13, there exists a strong Malcev basis B ′ = {X, Y 1 , . . . , Y n−1 } of g′ such that [X, Y i ] = Y i+1 ,

[Y i , Y j ] = 0,

i = 1, . . . , n − 2, i, j = 1, . . . , n − 1

and l′ = ℝ-span{X, Y n−p+1 , . . . , Y n−1 } with p = dim l′ . We now need the following lemma.

7.4 A stability theorem for non-Abelian actions |

397

Lemma 7.4.3. Γ is non-Abelian if and only if Γ′ is. Proof. Assume that Γ′ is Abelian, then so is l′ . Therefore, l′ = ℝ-span {X} or l′ = ℝ-span {X, Y n−1 }. In the case where l′ = ℝ-span {X}, it is clear that Γ is Abelian. Otherwise, Γ′ = expG′ (ℤX) expG′ (ℤY n−1 ), which implies that Γ = exp(ℤ(X + αYn )) exp(ℤ(Yn−1 + αn−1 Yn ))(Γ∩Z(G)) for some α, αn−1 ∈ ℝ, and finally Γ is Abelian. The inverse implication is trivial. According to Lemma 7.4.3, we have that p ⩾ 3. We can assume that l′ = ℝ-span{X, Z n−p+1 , Z n−p+2 , . . . , Z n−2 , Z n−1 }, where (i) Zn−p+1 = Yn−p+1 , (ii) Zj = Yj + ∑n−1 i=j+1 ui,j Yi , j = n − p + 2, . . . , n − 2, (iii) Zn−1 = Yn−1 such that ui,j ∈ ℝ and satisfying expG′ (X) expG′ (Z j ) expG′ (−X) expG′ (−Z j ) = expG′ (Z j+1 ),

j = n − p + 1, . . . , n − 2.

This gives that L′ = expG′ (ℝX) expG′ (ℝZ n−p+1 ) ⋅ ⋅ ⋅ expG′ (ℝZ n−1 ). Then Γ′ = expG′ (ℤX) expG′ (ℤZ n−p+1 ) ⋅ ⋅ ⋅ expG′ (ℤZ n−1 ), and there exist α, αj ∈ ℝ for n − p + 1 ⩽ j ⩽ n − 1 such that Γ = exp(ℤ(X + αYn )) exp(ℤ(Zn−p+1 + αn−p+1 Yn )) ⋅ ⋅ ⋅ exp(ℤ(Zn−1 + αn−1 Yn ))(Γ ∩ Z(G)).

7.4.1 Description of Hom(Γ, G) Our main result in this subsection consists in giving an explicit description of Hom(Γ, G). We regard the product set Gk = G × ⋅ ⋅ ⋅ × G as a set of matrices in Mn+1,k (ℂ) defined by { { { C Ek = {( 2iπz ) ∈ Mn+1,k (ℂ) { { e {

󵄨󵄨 C ∈ M (ℝ), 󵄨󵄨 n,k } 󵄨󵄨 } } 󵄨󵄨 󵄨󵄨 z := (z1 , . . . , zk ) ∈ ℝk , . } 󵄨󵄨 } } 󵄨󵄨 2iπz 2iπz 2iπz 󵄨󵄨 e := (e 1 ⋅ ⋅ ⋅ e k ) } 󵄨

398 | 7 Discontinuous actions on reduced nilmanifolds Recall that 0

if Γ is torsion-free,

1

otherwise.

ε={

In the case where Γ is not torsion-free, we assume that s ∈ ℕ∗ is the cardinality of Γ ∩ Z(G) and 0 γn = ( 2iπ 1 ) e s one generator. Let {γ, γq , . . . , γn−1+ε } be a family of generators of Γ, we consider the injective map Ψ : Hom(Γ, G) φ

󳨀→ 󳨃󳨀→

Mn+1,k (ℂ)

(7.28)

⌊φ(γ), φ(γq ), . . . , φ(γn−1+ε )⌋.

It is not hard to check that Ψ is a homeomorphism on its range. So, our task is reduced to give an explicit description of Ψ(Hom(Γ, G)). Toward such a purpose, we define some matrices in Ep : u u1 ( .. ( . ( (u ( q−1 ( ( uq ( Mu = ( (u ( q+1 ( . ( . ( . ( ( .. ( . un−1 2iπun e (

0 0 .. . 0 vq,q

0 0 .. . 0 0

vq+1,q .. . .. . vn−1,q e2iπvn,q

uvq,q vq+2,q+1 .. . vn−1,q+1 e2iπvn,q+1

u u1 ( .. ( . ( = (u ( n−3 (u n−2 un−1 2iπu (e n

λu λu1 .. . λun−3 vn−2,q vn−1,q e2iπvn,q

0 0 .. . 0 0 u(vn−2,q − λun−2 ) e2iπvn,q+1

Mu,λ

⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ .. . .. . ⋅⋅⋅ ⋅⋅⋅

0 0 .. . 0 0 .. . .. .

) ) ) ) ) ) ) ) ), ) ) ) ) ) ) ) )

0 un−q−1 vq,q e2iπvn,n−1 ) 0 0 .. . 0 0 0

e2iπvn,q+2

0 0 .. . 0 0 0 1

⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅

0 0 .. ) .) ) , 0) ) ) 0 0 1)

7.4 A stability theorem for non-Abelian actions | 399

Mvq

( ( ( =( ( (u

0 0 .. . 0

vq v1,q .. . vn−3,q vn−2,q vn−1,q e2iπvn,q

n−2

un−1 2iπu (e n

0 0 .. . 0 0

0 0 .. . 0 0 0 1

−un−2 vq e2iπvn,q+1

⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅

0 0 .. ) .) ) 0) ) 0) 0 1)

and 0 u1 ( . M0 = ( .. un−1 2iπun (e

0 v1,q .. . vn−1,q e2iπvn,q

0 0 .. . 0 1

⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅

0 0 .. ) .). 0 1)

Finally, we define the sets: 0

∗

E0,p = {Mu ∈ Ep : u ∈ ℝ }, 0

∗

E1,p = {Mu,λ ∈ Ep : λu ∈ ℝ }, 0

∗

E2,p = {Mvq ∈ Ep : vq ∈ ℝ }, 0

E3,p = {M0 ∈ Ep }, 1

t

E0,p = {Mu ⋒ (0n , e 1

2iπ sr

t

E1,p = {Mu,λ ⋒ (0n , e 1

t

E2,p = {Mvq ⋒ (0n , e 1

t

E3,p = {M0 ⋒ (0n , e

0 ) ∈ Ep+1 : Mu ∈ E0,p , r ∈ {0, . . . , s − 1}},

2iπ sr

2iπ

2iπ sr

r s

0 ) ∈ Ep+1 : Mu,λ ∈ E1,p , r ∈ {0, . . . , s − 1}},

0 ) ∈ Ep+1 : Mvq ∈ E2,p , r ∈ {0, . . . , s − 1}},

) ∈ Ep+1 : r ∈ {0, . . . , s − 1}}

and ε

ε

Hj,p = Ej,p ∩ Ψ(Hom(Γ, G)),

j ∈ {0, 1, 2, 3},

where ⋒ merely means the concatenation of matrices. The following proposition describes the structure of the set Hom(Γ, G). Proposition 7.4.4. Let G be a reduced threadlike Lie group and Γ a non-Abelian discrete 0 subgroup of G. The set Hom(Γ, G) is homeomorphic to ∐3j=0 Hj,p if Γ is torsion-free and 1 to ∐3j=0 Hj,p otherwise.

Proof. According to Proposition 7.4.2, when Γ is torsion-free we have m

m

n−1 Γ = {γ m γq q ⋅ ⋅ ⋅ γn−1 , m, mj ∈ ℤ}

400 | 7 Discontinuous actions on reduced nilmanifolds −1 such that q = n − p + 1, γγn−1 γ −1 γn−1 = e, γγj γ −1 γj−1 = γj+1 ; q ⩽ j ⩽ n − 2 and γi γj γi−1 γj−1 = e; q ⩽ i, j ⩽ n − 1. Let φ ∈ Hom(Γ, G) and

u u1 ( . Ψ(φ) := Mφ = ( .. un−1 2iπun e (

vq v1,q .. . vn−1,q e2iπvn,q

⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅

vn−1 v1,n−1 .. ) . ). vn−1,n−1 e2iπvn,n−1 )

−1 As γγn−1 γ −1 γn−1 = e, we get φ(γ)φ(γn−1 )φ(γ)−1 φ(γn−1 )−1 = e and, therefore,

uv1,n−1 − u1 vn−1 = 0. Let j ∈ {q, . . . , n − 2}. We have γγj γ −1 γj−1 = γj+1 and then φ(γ)φ(γj )φ(γ)−1 φ(γj )−1 = φ(γj+1 ). This gives that vj+1 = v1,j+1 = 0, { v2,j+1 = uv1,j − u1 vj . Likewise, for i, j ∈ {q, . . . , n − 1}, γi γj γi−1 γj−1 = e and then φ(γi )φ(γj )φ(γi )−1 φ(γj )−1 = e. This also gives vi v1,j − v1,i vj = 0. Finally, we obtain the following: vj+1 = v1,j+1 = 0,

{

q ⩽ j ⩽ n − 2,

v2,j+1 = uv1,j − u1 vj ,

(1)

(7.29)

q ⩽ j ⩽ n − 2. (2)

Equations (1) and (2) in (7.29) give v2,j = 0 for all j ∈ {q + 2, . . . , n − 1}. Therefore, u u1 ( u2 ( ( Mφ = ( u3 ( . ( .. un−1 2iπu (e n

vq v1,q v2,q v3,q .. . vn−1,q e2iπvn,q

0 0 uv1,q − u1 vq v3,q+1 .. . vn−1,q+1 e2iπvn,q+1

0 0 0

v3,q+2 .. . vn−1,q+2 e2iπvn,q+2

We are conclusively led to the following discussions:

⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅

0 0 0

) ) ) ). ) )

v3,n−1 .. . vn−1,n−1 e2iπvn,n−1 )

7.4 A stability theorem for non-Abelian actions |

401

Case 1: If u ≠ 0, assume for a while that vq = 0. For all j ∈ {q, . . . , n − 2}, we have γγj γ −1 γj−1 = γj+1 and then φ(γ)φ(γj )φ(γ)−1 φ(γj )−1 = φ(γj+1 ). Hence we get that u u1

( ( u2 ( ( ( ( u3 ( ( .. Mφ = ( ( . (u ( n−q ( (un−q+1 ( ( .. ( . un−1 2iπun (e

0 v1,q

0 0

v2,q

uv1,q

v3,q .. . vn−q,q vn−q+1,q .. . vn−1,q e2iπvn,q

v3,q+1 .. . vn−q,q+1 vn−q+1,q+1 .. . vn−1,q+1 e2iπvn,q+1

⋅⋅⋅ ⋅⋅⋅ .. . .. .

⋅⋅⋅ ⋅⋅⋅ ..

⋅⋅⋅ ⋅⋅⋅

. ⋅⋅⋅ ⋅⋅⋅

⋅⋅⋅ ⋅⋅⋅

⋅⋅⋅ ⋅⋅⋅

..

.

0 0 ) ) ) ) ) ) ) ) ) 0 ) n−q−1 u v1,q ) ) ) vn−q+1,n−1 ) ) ) .. ) . vn−1,n−1 e2iπvn,n−1 ) 0 .. .

with k

i−1 u {vi,j = ∑k=1 k! vi−k,j−1 , q + 1 ⩽ j ⩽ n − 1, 2 ⩽ i ⩽ n − 1, { n−1 uk {vnj − ∑k=1 k! vn−k,j−1 ∈ ℤ, q + 1 ⩽ j ⩽ n − 1. −1 As γγn−1 γ −1 γn−1 = e, we get φ(γ)φ(γn−1 )φ(γ)−1 φ(γn−1 )−1 = e, which gives in turn that

{vi,q = 0, 1 ⩽ i ⩽ q − 1, { { vi,n−1 = 0, 1 ⩽ i ⩽ n − 2, { { { n−q {u vq,q ∈ ℤ. Therefore, u u1 ( .. ( . ( (u ( q−1 ( ( uq ( Mφ = ( (u ( q+1 ( . ( . ( . ( ( .. ( . un−1 2iπun e (

0 0 .. . 0 vq,q

0 0 .. . 0 0

vq+1,q .. . .. . vn−1,q e2iπvn,q

uvq,q vq+2,q+1 .. . vn−1,q+1 e2iπvn,q+1

⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ .. . .. . ⋅⋅⋅ ⋅⋅⋅

0 0 .. . 0 0 .. . .. .

) ) ) ) ) ) ) ) ) ∈ E0 . 0,p ) ) ) ) ) ) ) )

0 un−q−1 vq,q e2iπvn,n−1 )

402 | 7 Discontinuous actions on reduced nilmanifolds Let now vq ≠ 0. There exists λ ∈ ℝ∗ such that vq = λu. For all j ∈ {q, . . . , n − 2}, we have γγj γ −1 γj−1 = γj+1 and then φ(γ)φ(γj )φ(γ)−1 φ(γj )−1 = φ(γj+1 ). This entails in turn that u u1

( ( u2 ( ( ( ( u3 ( ( .. Mφ = ( ( . (u ( n−q ( (un−q+1 ( ( .. ( . un−1 2iπun e (

λu v1,q

0 0

v2,q

u(v1,q − λu1 )

v3,q .. . vn−q,q vn−q+1,q .. . vn−1,q e2iπvn,q

v3,q+1 .. . vn−q,q+1 vn−q+1,q+1 .. . vn−1,q+1 e2iπvn,q+1

⋅⋅⋅ ⋅⋅⋅ .. . .. .

..

⋅⋅⋅ ⋅⋅⋅

. ⋅⋅⋅ ⋅⋅⋅

⋅⋅⋅ ⋅⋅⋅

⋅⋅⋅ ⋅⋅⋅

0 0

⋅⋅⋅ ⋅⋅⋅

..

.

) ) ) ) ) ) ) ) ) 0 ) n−q−1 u (v1,q − λu1 )) ) ) ) vn−q+1,n−1 ) ) .. ) . vn−1,n−1 e2iπvn,n−1 ) 0 .. .

with k

i−1 u {vi,j = ∑k=1 k! vi−k,j−1 , q + 2 ⩽ j ⩽ n − 1, 3 ⩽ i ⩽ n − 1, { n−1 uk {vnj − ∑k=1 k! vn−k,j−1 ∈ ℤ, q + 2 ⩽ j ⩽ n − 1.

Besides, φ(γq )φ(γq+1 )φ(γq )−1 φ(γq+1 )−1 = e and φ(γ)φ(γq )φ(γ)−1 φ(γq )−1 = φ(γq+1 ) and this allows us to write vj,q = λuj , 1 ⩽ j ⩽ n − 3, { { { { { {vi,q+1 = 0, 1 ⩽ i ⩽ n − 2, { {vn−1,q+1 = u(vn−2,q − λun−2 ), { { { { 2 {λu (vn−2,q − λun−2 ) ∈ ℤ. 0 We get finally that Mφ ∈ E1,p . Case 2: If u = 0 and vq ≠ 0, we have for all j ∈ {q, . . . , n − 2}, γγj γ −1 γj−1 = γj+1 and then φ(γ)φ(γj )φ(γ)−1 φ(γj )−1 = φ(γj+1 ). Hence

0 u1 ( u2 ( ( . ( .. Mφ = ( ( ( un−3 ( (u n−2 un−1 2iπun (e

vq v1,q v2,q .. . vn−3,q vn−2,q vn−1,q e2iπvn,q

0 0

v2,q+1 .. . vn−3,q+1 vn−2,q+1 vn−1,q+1 e2iπvn,q+1

0 0 0 .. . 0 0 0 1

⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅ ⋅⋅⋅

0 0 0) ) .. ) .) ) ) 0) ) 0) 0 1)

7.4 A stability theorem for non-Abelian actions | 403

with v

k

i−1 q {vi,q+1 = − ∑k=1 k! ui−k , 2 ⩽ i ⩽ n − 1, k { n−1 vq {vn,q+1 + ∑k=1 k! un−k ∈ ℤ.

We have φ(γq )φ(γq+1 )φ(γq )−1 φ(γq+1 )−1 = e and then ui = 0 for all i ∈ {1, . . . , n − 3}, 0 0 un−2 vq2 ∈ ℤ and Mφ ∈ E2,p ; otherwise, Mφ ∈ E3,p . Now, when Γ is not torsion-free, we get an additional column at the representative matrices of homomorphisms, which reads as φ(γn ) = γnr such that r ∈ {0, . . . , s − 1}. From now on, we identify any homomorphism φ ∈ Hom(Γ, G) with its correspondε ing matrix Ψ(φ) ∈ ∐3j=0 Hj,p . Let ε

0

ε

Kj,p = Ψ(Homd (Γ, G)) ∩ Hj,p ,

j ∈ {0, 1, 2, 3}.

The following two propositions accurately determine the structure of Hom0d (Γ, G). Proposition 7.4.5. Keep the same notation, and assume that Γ is torsion-free. Then Hom0d (Γ, G) is homeomorphic to K such that: 0 0 (1) If p ⩾ 4, then K = K0,p = {M ∈ H0,p : vq,q ≠ 0}. (2) If p = 3, then K = ∐2j=0 Kj,30 , where 0

0

0

0

K0,3 = {M ∈ H0,3 : vn−2,n−2 ≠ 0}, K1,3 = {M ∈ H1,3 : vn−2,n−2 − λun−2 ≠ 0}

and 0

0

K2,3 = {M ∈ H2,3 : un−2 ≠ 0}.

Proof. We have 0 Ψ(Hom0d (Γ, G)) ∩ H3,p = 0. 0 In fact, let φ ∈ H3,p , then we have φ(γ)φ(γq )(φ(γ))−1 (φ(γq ))−1 = e. This implies that −1 −1 φ(γγq γ γq ) = e. If φ is injective, then γγq γ −1 γq−1 = e and, therefore, Γ is Abelian, which is impossible. We begin by the case where p ⩾ 4, we have 0 0 0 Ψ(Hom0d (Γ, G)) ∩ (H1,p ) = 0. ∐ H2,p ∐ H3,p 0 In fact, let φ ∈ Hj,p , j = 1. Then φ(γn−p+3 ) = exp(vn,n−p+3 Yn ) ∈ φ(Γ) ∩ Z(G). Suppose

b that φ(Γ) is discrete, then vn,n−p+3 = ba ∈ ℚ for some b ≠ 0. Therefore, φ(γn−p+3 )=e 0 and φ is not injective. Let now φ ∈ Hom(Γ, G) and Mφ ∈ K0,p . It is not hard to show

404 | 7 Discontinuous actions on reduced nilmanifolds 0 that φ is injective and φ(Γ) is discrete. Assume that Mφ ∈ H0,p with vq,q = 0, then φ(γn−1 ) = exp(vn,n−1 Yn ) ∈ φ(Γ) ∩ Z(G), which is discrete if φ(Γ) is discrete and then b ) = e and φ is not injective. We treat similarly vn,n−1 = ba ∈ ℚ, b ≠ 0. Therefore, φ(γn−1 the case where p = 3.

The case where Γ is not torsion-free is straightforward. Proposition 7.4.6. With the same notation, when Γ is not torsion-free, Hom0d (Γ, G) is homeomorphic to K such that: (1) If p ⩾ 4, then 1

1

K = K0,p = {M ∈ H0,p : vq,q ≠ 0, r ∈ {1, . . . , s − 1} and r ∧ s = 1}.

(2) If p = 3, then K = ∐2j=0 Kj,31 , where 1

1

1

1

K0,3 = {M ∈ H0,3 : vn−2,n−2 ≠ 0, r ∈ {1, . . . , s − 1} and r ∧ s = 1}, K1,3 = {M ∈ H1,3 : vn−2,n−2 − λun−2 ≠ 0, r ∈ {1, . . . , s − 1} and r ∧ s = 1}

and 1

1

K2,3 = {M ∈ H2,3 : un−2 ≠ 0, r ∈ {1, . . . , s − 1} and r ∧ s = 1}.

7.4.2 Explicit determination of the parameter space The following result follows from the Theorem 7.3.2. Proposition 7.4.7. Let G be a reduced threadlike Lie group, H = exp h a closed, connected subgroup of G and Γ a non-Abelian discontinuous group for G/H. We have 0

R (Γ, G, H) = {φ ∈ Homd (Γ, G) : Adg h ∩ lφ ⊆ z(g) for any g ∈ G},

where lφ is the Lie subalgebra associated to the syndetic hull of φ(Γ) (as in Theorem 1.4.12). Proof. Let φ ∈ Hom0d (Γ, G). We first show that the proper action of φ(Γ) on G/H implies its free action. It is clear that the proper action implies that the triplet (G, H, φ(Γ)) is (CI), which gives that for all g ∈ G, the subgroup K := φ(Γ) ∩ gHg −1 is central and then finite as φ(Γ) is discrete. As the map φ : Γ → φ(Γ) is a group isomorphism and K is finite and cyclic, we get that φ−1 (K) = K. Therefore, K ⊂ Γ ∩ H = {e}. Thus, the action of φ(Γ) on G/H is free. As Lφ contains φ(Γ) cocompactly, 0

R (Γ, G, H) = {φ ∈ Homd (Γ, G) : Lφ acts properly on G/H}.

Now, Theorem 7.9 allows us to conclude.

7.4 A stability theorem for non-Abelian actions |

405

Using Theorem 7.9, the proper action of Γ on G/H implies that r ⩽ n−p+dim(h∩z(g)) where r = dim h. We assume from now on that this previous condition is satisfied. Let g ∈ G, BAdg h a strong Malcev basis of Adg h extracted from B and MAdg h,B the matrix of Adg h written in the basis B = {X, Y1 , . . . , Yn }. We put M1 MAdg h,B = ( Adg h,B ) , zAdg h

zAdg h ∈ ℝr .

In light of Proposition 7.4.7, the following result is immediate. Lemma 7.4.8. Let G be a reduced threadlike Lie group, H = exp h a nontrivial, closed, connected subgroup of G and Γ a non-Abelian discontinuous group for G/H. The parameter space is homeomorphic to R = {(

C 1 ) ∈ K : rk(C ⋒ MAd ) = p + r − dim(h ∩ z(g)) for any g ∈ G} . g h,B e2iπz

ε We denote by Rj,p = Ψ(R (Γ, G, H)) ∩ Kj,pε . Then according to Propositions 7.4.5

ε ε and 7.4.6, we have that R = R0,p whenever p ⩾ 4. Otherwise, R = ∐2j=0 Rj,3 . Let first p ⩾ 4. We denote by

IB = {i1 < ⋅ ⋅ ⋅ < ir } h

(r = dim h)

the set of indices i ∈ {1, . . . , n} such that h∩gi ≠ h∩gi+1 , where gi = ℝ-span{Yi , . . . , Yn }, i = 1, . . . , n, g0 = g, and gn+1 = {0}. Proposition 7.4.9. We keep the same hypotheses and notation as before. Suppose that p ⩾ 4 and h ⊄ g0 . Then there exist h1 , . . . , hn ∈ ℝ such that h = ℝ-span{X + h1 Y1 + ⋅ ⋅ ⋅ + hn Yn } ⊕ h ∩ z(g) and ε

ε

∗

R0,p = {M ∈ K0,p : (u1 − h1 u) ∈ ℝ }.

̃ for some X ̃ = X + h1 Y1 + ⋅ ⋅ ⋅ + hn Yn Proof. If z(g) ⊄ h, then it is clear that h = ℝ-span{X} where h1 , . . . , hn ∈ ℝ. Otherwise, as Γ acts freely on G/H, we have h = ℝ-span{X +h1 Y1 + ⋅ ⋅ ⋅ + hn Yn , Yn } for some h1 , . . . , hn ∈ ℝ. ε Let M ∈ K0,p . Clearly, we have 1 M ∈ R ⇔ rk(C ⋒ MAd ) = p + r − dim(h ∩ z(g)) g h,B t

t

for all g ∈ G

⇔ rk⌊ (u, u1 , . . . , un−1 ), (1, h1 , α2 , . . . , αn−1 )⌋ = 2 ∗

⇔ u1 − uh1 ∈ ℝ .

for all α2 , . . . , αn−1 ∈ ℝ

406 | 7 Discontinuous actions on reduced nilmanifolds Proposition 7.4.10. We keep the same hypotheses and notation as before. If p ⩾ 4 and h ε ε h ⊂ g0 , then IB ⊂ {1, . . . , q − 1, n} and R0,p = K0,p . h Proof. Suppose that IB ∩ {q, . . . , n − 1} ≠ 0, then there exists i0 ∈ {q, . . . , n − 1} and ̃ = Yi + hi +1 Yi +1 + ⋅ ⋅ ⋅ + hn Yn ∈ h for some hi +1 , . . . , hn ∈ ℝ, which is impossible Y 0 0 0 0 Ad h

as L acts on G/H properly. So, IB g ∩ {q, . . . , n − 1} = 0 for all g ∈ G. This gives that 1 rk(C ⋒ MAd ) = r + p − dim(h ∩ z(g)) for all g h,B M=(

e

C

2iπz )

ε ∈ K0,p ,

g ∈ G.

We argue similarly as in the previous propositions to treat the case where p = 3. Proposition 7.4.11. We keep the same hypotheses and notation as before. Suppose that p = 3 and h ⊄ g0 . Then there exist h1 , . . . , hn ∈ ℝ such that h = ℝ-span{X + h1 Y1 + ⋅ ⋅ ⋅ + hn Yn } ⊕ h ∩ z(g). Moreover, we have ε

ε

∗

Rj,3 = {M ∈ Kj,3 : (u1 − h1 u) ∈ ℝ },

j ∈ {0, 1}

and ε

ε

∗

R2,3 = {M ∈ K2,3 : (v1,q − h1 vq ) ∈ ℝ }.

Proposition 7.4.12. We keep the same hypotheses and notation as before. If p = 3 and h ε ⊂ {1, . . . , q − 1, n} and Rj,3 = Kj,3ε , j ∈ {0, 1, 2}. h ⊂ g0 , then IB Proposition 7.4.13. Let G = 𝔾r3 be the 4-dimensional reduced threadlike Lie group, H = exp h a nontrivial closed connected Lie subgroup of G and Γ a non-Abelian discontinuous 0 group for G/H. We have p = 3, h = z(g) and Rj,3 = Kj,30 , j ∈ {0, 1, 2}. Proof. As r ⩽ 3 − p + dim(h ∩ z(g)), we have that r ⩽ dim(h ∩ z(g)). If z(g) ⊄ h, then H is trivial, which is impossible. Now, when z(g) ⊂ h, we get r = 1 and h = z(g). Hence, 0 Rj,3 = Kj,30 , j ∈ {0, 1, 2}.

7.4.3 Proof of Theorem 7.4.1 We start by proving this lemma that will be used later. ε Lemma 7.4.14. The disjoint components Rj,p , j ∈ {0, 1, 2} are G-invariant.

7.4 A stability theorem for non-Abelian actions |

Proof. The group G acts on Hom(l, g) through the law g ⋆ φ = Adg ∘φ. Let → 󳨀 0

u u1 M=( U2 e2iπun

N ⋅⋅⋅

e2iπvn,q

0 ) ∈ R0,p ,

e2iπvn,n−1

where U2 = t (u2 , . . . , un−1 ) ∈ ℝn−2 and N = ⌊Wq , . . . , Wn−1 ⌋ ∈ Mn−2,n−q (ℝ) with Wq = t (vq,q , . . . , vn−1,q ) ∈ ℝn−2 , { { { { { Wq+1 = t (0, uvq,q , vq+2,q+1 , . . . , vn−1,q+1 ) ∈ ℝn−2 , { { { { { {Wq+2 = t (0, 0, u2 vq,q , vq+3,q+2 , . . . , vn−1,q+2 ) ∈ ℝn−2 , { .. { { { . { { { { { { Wn−2 = t (0, . . . , 0, un−q−2 vq,q , vn−1,n−2 ) ∈ ℝn−2 , { { { t n−q−1 vq,q ) ∈ ℝn−2 . {Wn−1 = (0, . . . , 0, u Let g = exp(xX + y1 Y1 + ⋅ ⋅ ⋅ + yn Yn ) ∈ G with x, y1 , . . . , yn ∈ ℝ. We have u u1 g⋆M =( ′ U2

→ 󳨀 0

e2iπun

An−3 (x)N

e2iπvn,q

′

′

⋅⋅⋅

), e2iπvn,n−1 ′

where

( ( An (x) = ( (

1

0

x

1

2

x ..

x 2

.. .

xn ( (n)! t

(U2′ , e

2iπu′n

⋅⋅⋅ .. . 1

.

⋅⋅⋅

) = g ⋆ U1 − u1 Y1 ,

..

.

x2 2

.

0 .. . .. ) ) ∈ Mn+1 (ℝ), .) )

.

0

⋅⋅⋅ .. .. x

1)

with U1 = t (u1 , U2 , e2iπun )

and j

x ′ vn,q = vn,q + ∑n−q v , { j=1 j! n−j,q { { { j { n−q−2 x xn−q−1 ′ {v { = vn,q+1 + ∑j=1 j! vn−j,q+1 + (n−q−1)! uvq,q , { n,q+1 { {. . { {. { { 2 { ′ { { vn,n−2 = vn,n−2 + xvn−1,n−2 + x2 un−q−2 vq,q , { { { ′ n−q−1 vq,q . {vn,n−1 = vn,n−1 + xu

407

408 | 7 Discontinuous actions on reduced nilmanifolds 0 1 It is then clear that g ∗ M ∈ R0,p . Likewise, we obtain that R0,p is G-invariant. Suppose now that p = 3. Let

u u1 ( .. ( . ( M=( ( un−3 ( ( un−2 un−1 2iπun (e

λu λu1 .. . λun−3 vn−2,q vn−1,q e2iπvn,q

0 0 ) .. ) . ) ) ∈ R0 . 0 1,3 ) ) ) 0 u(vn−2,q − λun−2 ) e2iπvn,q+1 )

We have u u1 ( u′2 ( ( . ( .. g⋆M =( ( ′ ( un−3 ( ′ (u n−2 u′n−1

2iπun (e ′

λu λu1 λu′2 .. . λu′n−3 ′ vn−2,q ′ vn−1,q

e2iπvn,q ′

0 0 ) 0 ) ) .. ) . ), ) ) 0 ) ) 0 u(vn−2,q − λun−2 ) e2iπvn,q+1 ′

)

where j−1

x u′i = ui + ∑i−1 i ∈ {2, . . . , n}, { j=1 j! (xui−j − uyi−j ), { { { n−3 xj−1 ′ { { {vn−2,q = vn−2,q + λ ∑j=1 j! (xun−j−2 − uyn−j−2 ), { { ′ xj−1 vn−1,q = vn−1,q + (xvn−2,q − λuyn−2 ) + λ ∑n−2 (xun−j−1 − uyn−j−1 ), { j=2 j! { { { x xj−1 ′ { vn,q = vn,q + (xvn−1,q − λuyn−1 ) + 2 (xvn−2,q − λuyn−2 ) + λ ∑n−1 (xun−j−1 − uyn−j−1 ), { j=3 { j! { { ′ {vn,q+1 = vn,q+1 + xu(vn−2,q − λun−2 ). 1 The result follows in this case. Similarly, we obtain that R1,3 is G-invariant. Let

( ( ( M=( ( (u

0 0 .. . 0

n−2

un−1 2iπun e (

vq v1,q .. . vn−3,q vn−2,q vn−1,q e2iπvn,q

0 0 .. . 0 0

) ) ) 0 ) ∈ R2,3 . ) )

−un−2 vq e2iπvn,q+1 )

7.4 A stability theorem for non-Abelian actions | 409

We have 0 0 0 .. . 0

vq v1,q ′ v2,q .. . ′ vn−3,q ′ vn−2,q ′ vn−1,q

( ( ( ( g∗M =( ( ( ( ( un−2 u′n−1

2iπu (e n

0 0 0 .. . 0 0

−un−2 vq

e2iπvn,q

′

) ) ) ) ), ) ) ) )

e2iπvn,q+1 )

′

′

where u′n−1 = un−1 + xun−2 , { { { 2 { { {u′n = un + xun−1 + x2 un−2 , { xj−1 ′ { { vi,q = vi,q + ∑i−1 { j=1 j! (xvi−j,q − vq yi−j ), { { ′ {vn,q+1 = vn,q+1 − xun−2 vq .

i ∈ {2, . . . , n},

The result also follows in this case. It is sufficient to prove Theorem 7.4.1 when Γ is torsion-free. We first prove that R is open in K . In the case when p ⩾ 4, the result stems directly from Propositions 7.4.9 and 7.4.10. Suppose now that p = 3. It suffices then to see that 2

0

0

K \ R = ∐(Kj,3 \ Rj,3 ) j=0

K

is closed in K . Let then M ∈ K \ R . There exists therefore a sequence (Mm )m∈ℕ assumed to belong to K \ R , which converges to M. So we can extract from (Mm )m∈ℕ a 0 0 subsequence (Mms )s∈ℕ of elements in Kj,30 \ Rj,3 for some j ∈ {0, 1, 2}. If Mms ∈ Kj,30 \ Rj,3 0 for j ∈ {0, 2}, then obviously its limit M belongs to Kj,30 \ Rj,3 . Suppose now that

Mms

us us1 ( .. ( . ( s =( ( un−3 ( s ( un−2 usn−1 (e

2iπusn

λs us λs us1 .. . λs usn−3 s vn−2,n−2 s vn−1,n−2

e

s 2iπvn,n−2

0 0 .. . 0 0

) ) ) ) ∈ K 0 \ R0 1,3 1,3 ) ) )

s us (vn−2,n−2 − λs usn−2 ) s

e2iπvn,n−1

)

for some real sequence (λs )s∈ℕ . If (λs )s goes to infinity as s goes to +∞, then we can 0 0 0 0 0 0 easily check that M ∈ K2,3 \ R2,3 . Otherwise, M ∈ (K0,3 \ R0,3 ) ∐(K1,3 \ R1,3 ). We get

410 | 7 Discontinuous actions on reduced nilmanifolds 0 therefore that M ∈ K \ R . We argue similarly to obtain that K is open in ∐3j=0 Hj,p . This completes the proof of our theorem.

7.4.4 A concluding remark Table 7.1 summarizes the results concerning local rigidity, stability and Hausdorfness in the setting of (double) Heisenberg and threadlike Lie groups. Table 7.1: Summarizing the rigidity and stability results. Rigidity

Stability

Hausdorffness

G = H2n+1 , H and Γ are arbitrary

Local rigidity fails to hold

Stability holds ⇐⇒ G-orbits have a common dimension

The deformation space is equipped with a smooth manifold structure ⇐⇒ The deformation space is a Hausdorff space ⇐⇒ G-orbits have a constant dimension

r , G = H2n+1 H and Γ are arbitrary

Local rigidity holds ⇐⇒ Strong local rigidity holds ⇐⇒ Γ is finite

Stability holds

The parameter and the deformation spaces are endowed with smooth manifold structures

G = H2n+1 × H2n+1 , H = ΔG and Γ are arbitrary

Local rigidity fails to hold

Stability holds ⇐⇒ Γ is non-Abelian or maximal Abelian

The deformation space is a Hausdorff space ⇐⇒ Γ is maximal Abelian

r r G = H2n+1 × H2n+1 , H = ΔG and Γ are arbitrary

Local rigidity holds ⇐⇒ Strong local rigidity holds ⇐⇒ Γ is finite

Stability holds

The deformation space is a Hausdorff space ⇐⇒ Γ is finite ⇐⇒ The parameter space admits a rigid homomorphism ⇐⇒ The G-orbits have a common dimension

G = 𝔾rn , H and Γ are arbitrary

Local rigidity holds ⇐⇒ Γ is finite (Γ is Abelian)

Stability holds for Abelian Γ

The deformation space is a Hausdorff space for non-Abelian Γ

8 Deformation of topological modules The purpose here is to describe a dequantization procedure for topological modules over a deformed algebra. We first define the characteristic variety of a topological module as the common zeroes of the annihilator of the representation obtained by setting the deformation parameter to zero. On the other hand, the Poisson characteristic variety is defined as the common zeroes of the ideal obtained by considering the annihilator of the deformed representation, and then setting the deformation parameter to zero. An involutive (or coisotropic) submanifold of a Poisson manifold V is a submanifold W of V such that the ideal of functions that vanish out on W is a Poisson subalgebra of C ∞ (V). In the context of the quantization by deformation, several authors [46, 54] have proposed some methods to associate to an involutive submanifold, a left ideal of the deformed algebra (C ∞ (V)⟦ν⟧, ∗), where the star-product ∗ on V comes from the constructions of M. Kontsevich [102] or of D. Tamarkin [126]. The aim here is to propose a reverse step. It consists in describing a method for dequantizing some modules on a deformed algebra, which means to associate to such module, an involutive submanifold and a Poisson submanifold of the underlying Poisson manifold. The C ∞ case is poorly adapted, but the analytic or algebraic cases turn to be appropriate. We first consider the complex field and then propose a natural involution on the deformed algebra where the objects thus define hold on the real field. These are (strongly pseudo-unitary) modules, which means modules endowed with bilinear Hermitian nondegenerate forms compatible with the involution, taking values in ℂ⟦ν⟧ (where ν is the parameter of deformation), and such that the quotient bilinear form obtained where ν = 0 is still nondegenerate. We here exclusively limit to the algebraic framework. We introduce the notion of divisible ideals of a deformed algebra A . Inspired by [86], we shall also define the notion of the characteristic and Poisson characteristic manifolds for a topologically free A -module. These manifolds are not necessarily conic (unlike what happens in [86] or [79]). Thanks to the introduction of the deformation parameter, the notion of strongly pseudo-unitarity allows to define these objects on the real field. Using Gabber’s theorem, we show the involutivity of the characteristic variety. The Poisson characteristic variety is indeed a Poisson subvariety of the underlying Poisson manifold. We compute explicitly the characteristic variety in several examples in the Poisson linear case, including the dual of any exponential solvable Lie algebra. In the nilpotent case, we show that any coadjoint orbit appears as the Poisson characteristic variety of a well-chosen topological module. In the case of an exponential solvable Lie group G = exp g, we substantiate the Zariski closure conjecture claiming that for an irreducible unitary representation π of G, associated to a coadjoint orbit Ω via the Kirillov orbit method, the Poisson characteristic variety associated to a topological module with an adequate way coincides with the Zariski closure in g∗ of the orbit Ω. We compute the characteristic varieties in https://doi.org/10.1515/9783110765304-008

412 | 8 Deformation of topological modules some fundamental examples where we show that the conjecture holds. We also prove the conjecture in some restrictive cases.

8.1 Geometric objects associated with deformed algebras Let (V, {, }) be a real analytic (resp., algebraic) Poisson manifold, which means a real analytic (resp., algebraic) manifold endowed with a 2-tensor P with analytic (resp., regular) coefficients such that the Schouten bracket [P, P] vanishes. The Poisson bracket endows the structural sheaf O of analytic (resp., regular) functions germs with a Poisson structure. We will limit to the flat case V = ℝd . So by complexification, one obtains a complex analytic (resp., algebraic) Poisson manifold structure on V ℂ = ℂd . Let A be the Poisson algebra OV of analytic (resp., polynomials) functions on ℂd . M. Kontsevich (cf. [102]) built a star-product # on A: f # g = ∑ νk Ck (f , g), k≥0

(8.1)

where the coefficients Ck are the bidifferential operators described using completely explicit formulas only involving partial differentials of constants of the Poisson 2-tensor structure (see again [8] and [110]). In particular, if the Poisson tensor is of analytic coefficients (resp., polynomials), then so is the star-product. In other words, the star-product (8.1) endows A = A⟦ν⟧

(8.2)

with an associative topologically free algebra structure on ℂ⟦ν⟧. This algebra is naturally filtered by An = νn A , and its associated graduation is naturally isomorphic to the algebra of polynomials A[ν], endowed with the commutative product of A extended through ℂ[ν]-linearity. Note that the star-product (8.1) is of real coefficients: if f and g are of real values on V, then so is f # g. The star-product (8.1) is equivalent to another star-product ∗ (called of Duflo– Kontsevich [6]) (cf. [55, 87] and [110]) having the same properties, and satisfying furthermore the following property: for all f , g in the center of (A , ∗), we have f ∗ g = fg. In addition, the two algebras (A , #) and (A , ∗) are isomorphic. Let us restrict to the algebraic frame. We shall define the characteristic manifold V(M ) ⊂ V ℂ of a topologically free A -module M as well as its Poisson characteristic manifold VA(M ) by adapting the definitions of [86] as follows: Consider the annihilator Ann M of the A -module M and define V(M ) as the common zeroes set of the

8.1 Geometric objects associated with deformed algebras | 413

annihilator of the A-module M = M /νM , and VA(M ) as the common zeroes set of Ann M /(Ann M ∩ νA ). These two objects are affine submanifolds of ℂn (i. e., defined by the annihilator of finite number of polynomials). We show (using [72]) that V(M ) is an involutive submanifold of V ℂ , and VA(M ) is a Poisson submanifold of V ℂ (what justifies the name), and that we always have the inclusion V(M ) ⊂ VA(M ). After showing (using [53]) that the involution f 󳨃󳨀→ f ∗ defined by f ∗ (ξ ) = f (ξ ) is an antiautomorphism of the algebra (A , ∗), we introduce the notion of a strongly pseudo-unitary A -module, and we show that the associated characteristic manifold is defined on the real field, as well as its Poisson characteristic manifold. More precisely, we first extend the complex conjugation to an automorphism of ℂ⟦ν⟧ when fixing ν = iℏ is purely imaginary, which means ν = −ν. The module M is said to be pseudounitary (or equivalently that the associate representation is ∗-representation [50]), if there exists a sesquilinear nondegenerated form ⟨−, −⟩ν on M , taking value in ℂ⟦ν⟧, which is Hermitian, that is, ⟨m, n⟩ν = ⟨n, m⟩ν ,

m, n ∈ M

and compatible with the involution, that is, satisfying for all a ∈ A : ⟨am, n⟩ν = ⟨m, a∗ n⟩ν . The form ⟨−, −⟩ν induced by passage to the quotient reveals an Hermitian form ⟨−, −⟩0 on M. If this form is nondegenerate, we say that the module M is strongly pseudounitary (or equivalently, the associated representation is a strongly nondegenerate ∗-representation). Unlike [50], we do not necessarily assume positivity on the sesquilinear form. A strongly pseudo-unitary module endowed with a well-defined positive form will said to be (strongly unitary). 8.1.1 Topological modules on the ring of formal series We refer in this subsection to [53, 87] and [69]. Let k ⟦ν⟧ be the ring of formal series on any field k endowed with the ν-adic topology, defined by the ultrametric distance: d(a, b) = 2− val(a−b) , with val a = sup{j, a ∈ νj k ⟦ν⟧}. This distance makes of k ⟦ν⟧ a complete topological ring. Further, on a k ⟦ν⟧-module M , we put an invariant topology by translation by

414 | 8 Deformation of topological modules deciding that the family νj M , j ∈ ℕ forms a neighborhood basis of zero. This topology is separated if and only if the intersection of all νj M is reduced to {0}. In this case, we can define the valuation: val m = sup{j, m ∈ νj M } and also the topology defined by the ultrametric distance d(m, m′ ) = 2− val(m−m ) . ′

One says that a k ⟦ν⟧-module M is torsion-free, if the action of ν is an injection of M . A topologically free k ⟦ν⟧-module M is isomorphic to M ⟦ν⟧ for some vector space M.

Proposition 8.1.1 ([87, Proposition XVI.2.4], [47] and [50, Lemma A1]). A k ⟦ν⟧-module M is topologically free if and only if it is separated, complete and torsion-free. Definition 8.1.2. Let M be a k ⟦ν⟧-module, and let N be a sub-k ⟦ν⟧-module of M . We consider the k-vector spaces M = M /νM and N = N /νN . The inclusion i : N ↺ M induces a k-linear map: i0 : N 󳨀→ M.

We say (cf. [69]) that the sub-k ⟦ν⟧-module N is divisible if the map i0 is injective.

Remark 8.1.3. (1) For a topologically free module M = M ⟦ν⟧, the vector space M can also be seen as the quotient M /νM . (2) The sub-k ⟦ν⟧-module N is divisible if and only if νN = N ∩ νM . A simple example of nondivisible sub-k ⟦ν⟧-module is νM ⟦ν⟧ ⊂ M ⟦ν⟧. (3) For any k ⟦ν⟧-module M , we write again “m = O(νj ) in M ” for m ∈ νj M . A sub-k ⟦ν⟧module N of M is then divisible if and only if, the fact that for all m ∈ N , m = O(ν) in M implies that m = O(ν) in N . The associative algebra A defined in (8.2) is by construction a ℂ⟦ν⟧-module topologically free. Definition 8.1.4. (1) A topological A -module is a ℂ⟦ν⟧-module M endowed with a ℂ⟦ν⟧-bilinear map: Φ:A ×M (φ, m)

󳨀→ 󳨃󳨀→

M

πν (φ)m,

(8.3)

making of M an A -module. (2) Let M1 and M2 be two topological A -modules. A morphism of topological A -modules (or an intertwining operator) of M1 on M2 is a ℂ⟦ν⟧-linear continuous map commuting with the actions of elements of A . We say that M1 and M2

8.1 Geometric objects associated with deformed algebras | 415

are equivalent, if there exists an intertwining bijective bicontinuous operator of M1 on M2 . Proposition 8.1.5. Let M be a topological A -module. Then the ℂ⟦ν⟧-bilinear map Φ defined by equation (8.3) is continuous for the ν-adic topologies of A and of M . Proof. Let m ∈ M and a ∈ A. The family {Wj = Φ(a, m) + νj M }j forms a neighborhood basis of Φ(a, m) = πν (a)m. Consider the neighborhood Uj = a + νj A and Vj = m + νj M of a and of m, respectively. It is then clear that the image by Φ of the product Ui × Vj is included in Wj , which shows the continuity. In particular, the multiplication of A × A in A is ℂ⟦ν⟧-bilinear and continuous for the ν-adic topology, which makes of A a topological algebra.

8.1.2 Divisible ideals Let A = A⟦ν⟧ be a free topologically module, where we identify A with A /νA . Let J be a left ideal of A . It is immediate to see that J = J /(J ∩ νA ) is an ideal of the commutative algebra A. The same happens in the context of right ideals. Proposition 8.1.6. Let J be a divisible bilateral ideal of A . Then J = J /(J ∩ νA ) is a Poisson ideal of A. Proof. For a0 ∈ J and b0 ∈ A, let a = a0 + νa1 + ⋅ ⋅ ⋅ ∈ J and b = b0 + νb1 + ⋅ ⋅ ⋅ ∈ A . By divisibility of J , we have 1 (a ∗ b − b ∗ a) ∈ J , ν which means by considering the constant term that {a0 , b0 } belongs to J. We also remark that as J is divisible, we can also identify J to J /νJ . 8.1.3 Cancelations Let M be a topological A -module. We define the annihilator Ann M of the module M as the set of φ ∈ A such that πν (φ)m = 0 for all m ∈ M . We see immediately that Ann M is a bilateral ideal of A , and that Ann M is divisible if M is torsion-free. The following is then immediate. Proposition 8.1.7. Let M be an A -module. Then M = M /νM is a module on the commutative algebra A = A /νA . We note by π0 the representation of A on the associated module M.

416 | 8 Deformation of topological modules Proposition 8.1.8. Let M be a topological A -module. Then the annihilator of the A-module M = M /νM is stable by the Poisson bracket of A. Proof. We note by πν the representation of A in the module M . The annihilator of M can be seen as the set of f ∈ A such that πν (f )u = O(ν) for all u ∈ M . For all f , g ∈ Ann M, we have πν (f ∗ g)u = πν (f )πν (g)u = O(ν2 ), which gives 1 π (f ∗ g − g ∗ f ).u + O(ν) ν ν = O(ν),

πν ({f , g})u =

and hence the result. Remark 8.1.9. The ring A being Noetherian (cf. [47]), the annihilator Ann M is finitely generated.

8.1.4 Involutivity Definition 8.1.10. Let V = ℝd be a real algebraic flat Poisson variety, and let W ⊂ V ℂ be an affine subvariety. Let I(W) be the ideal of A consisting of functions that vanish on W. We say that W is involutive or coisotropic if the ideal I(W) is stable by the Poisson bracket. This is equivalent to the cancellation of the Poisson 2-tensor outside the conormal fiber of W (cf. [46, 54]). Proposition 8.1.11. Let W be an affine submanifold of V ℂ . Let W r be the nonsingular part of W. If W is involutive, then for all symplectic leaf S of V ℂ meeting W r , the intersection W r ∩ S is a coisotropic subvariety of S, which means that for all x ∈ W r ∩ S we have ω

(Tx (W r ∩ S)) ⊂ Tx (W r ∩ S),

(8.4)

where the exponent ω designates the orthogonal in Tx S with respect to the symplectic form. Proof. Let x ∈ W r ∩S and let Px be the Poisson 2-tensor in x. Let also P̃ x : Tx∗ V ℂ → Tx V ℂ be the skew-symmetric associated linear map, defined by ⟨η, P̃ x (ξ )⟩ = Px (ξ , η). Then the image of P̃ x is precisely Tx S.

(8.5)

8.1 Geometric objects associated with deformed algebras | 417

Lemma 8.1.12. We have the equality, ω

Tx (W r ∩ S) = P̃ x (Tx (W r ∩ S) ), ⊥

(8.6)

where the symbol ⊥ designates the orthogonal of a subspace in the dual. Proof. Let ξ ∈ Tx (W r ∩ S)⊥ . Then for all Y ∈ Tx (W r ∩ S), we have ω(P̃ x (ξ ), Y) = ⟨ξ , Y⟩ = 0,

(8.7)

which shows the inclusion ⊥ ω P̃ x (Tx (W r ∩ S) ) ⊂ Tx (W r ∩ S) .

To show the opposite inclusion, we consider X in Tx (W r ∩ S)ω , image by P̃ x of an element ξ of Tx∗ V, and it is immediate from (8.7) that ξ belongs to Tx (W r ∩ S)⊥ . Proof. Let us go back to the proof of Proposition 8.1.11. Let X ∈ Tx S, by definition of a symplectic leaf, there exists φ ∈ A such that X coincides with the Hamiltonian field Hφ (x). Then X belongs to (Tx (W r ∩ S))ω if and only if for all Y ∈ Tx (W r ∩ S) we have ω(X, Y) = Y ⋅ φ(x) = dφ(x)(Y) = 0. Now, the space Bx of dφ(x) where φ ∈ I(W) is the orthogonal of Tx W r . Thanks to the transversality condition, we can therefore write Tx (W r ∩ S) = (Tx W r ∩ Tx S) = Bx + (Tx S)⊥ . ⊥

⊥

(8.8)

Let then X, Y ∈ Tx (W r ∩ S)ω . From (8.8) and Lemma 8.1.12, there exists φ, ψ ∈ I(W) such that X = P̃ x (dφ(x)) = Hφ (x) and Y = Hψ (x). We have then thanks to the involutivity, ω(X, Y) = {φ, ψ}(x) = 0, which ends the demonstration. Remark 8.1.13. We have the equality ω ⊥ Tx (W r ∩ S) = P̃ x ((Tx W r ) ).

(8.9)

The following complements Proposition 8.1.11. Proposition 8.1.14. Let W be an affine submanifold of V ℂ . Let W r be the nonsingular part of W. Suppose that there exists a Zariski dense set U of W r such that:

418 | 8 Deformation of topological modules (1) For all x ∈ U , the intersection of W r with the symplectic leaf Sx going through x is transverse. (2) W r ∩ Sx is coisotropic in Sx . Then W is involutive. Proof. It is easy to see that under the hypotheses of the proposition, if f and g belong to I(W), then {f , g} vanishes on the dense Zariski open set U of W r , and so on W in whole. Recall [127] that even in the algebraic case, the symplectic leaves are not generally algebraic subvarieties of V ℂ . An example of this situation is given later in Subsection 8.3.4.

8.1.5 Characteristic manifolds Let M be a topological A -module. The characteristic variety V(M ) of M is defined as the set of common zeroes of the annihilator of the A-module M = M /νM . If M is torsion-free, it is called the Poisson characteristic variety of M , and as in [86] will be noted VA(M ), being the set of common zeroes of the ideal Ann M /(Ann M ∩νA ) of A. This terminology is justified by Proposition 8.1.8 below. As A is a commutative Noetherian ring, the associated graduation A[ν] of A is also Noetherian (cf. [47]). We record the following theorem (cf. [72, Theorem I]). Theorem 8.1.15 (Integrability of characteristic varieties: O. Gabber). Suppose that M is a finitely generated A -module and let M = M /νM . Then the radical J(M ) of Ann M is stable by the Poisson bracket. Remark 8.1.16. J(M ) is the ideal of elements of A that vanishes on the characteristic variety V(M ). Theorem 8.1.15 therefore says that V(M ) is involutive. Theorem 8.1.17. The set of common zeroes of a Poisson ideal is a Poisson submanifold of V ℂ . Proof. Let J be a Poisson ideal of A, and let f ∈ J. Let x be canceling all elements of J and let y = ϕt (x), where (ϕt )|t| 0 for all nonzero x ∈ M , the order on ℝ⟦ν⟧ being the lexicography order. This sesquilinear form defined by passing to the quotient is a complex sesquilinear form on M = M /νM . We will assume that the form ⟨−, −⟩ν is strongly nondegenerate, that is to also assume that the quotient form is nondegenerate on M. In this case, we say that the ∗-representation is strongly nondegenerate. A unitary strongly nondegenerate representation is said to be strongly unitary. A unitary A -module (resp., pseudo-unitary) is by definition a topological module endowed with a unitary representation (resp., a ∗-representation) of A . A strongly unitary A -module (resp., strongly pseudo-unitary) is defined to be a topological module equipped with a strongly unitary representation (resp., with a strongly nondegenerate ∗-representation) of A . Proposition 8.1.23. Let πν be a strongly nondegenerate ∗-representation of (A , ∗) in a topological module M . Then the annihilator of M = M /νM is generated by a finite number of self-adjoint elements of A. The same happens for Ann M /(Ann M ∩ νA ). Proof. Any element f of A is written in a unique way: f = f + + if − ,

(8.21)

where f + and f − are self-adjoint. We have 1 f + = (f + f ∗ ), 2

f− =

1 (f − f ∗ ). 2i

(8.22)

Due to the fact that the scalar product on M is nondegenerate, we see that if f belongs to Ann M, then so is f ∗ . Let {f1 , . . . , fk } be a system of generators of Ann M. It is then clear that Ann M is generated by {f1+ , . . . , fk+ , f1− , . . . , fk− }. The same applies to Ann M /(Ann M ∩ νA ), as we can see using the second equality (8.12) and the strong pseudo-unitarity of M . Corollary 8.1.24. Under the hypotheses below, the characteristic varieties V(M ) and VA(M ) of a strongly pseudo-unitary module are real, when M is torsion-free. Proof. The characteristic variety V(M ) is defined by the 2k equations: fj+ (ξ ) = fj− (ξ ) = 0,

j = 1, . . . , k.

(8.23)

Now the polynomials fj± are well of real coefficients by definition of the involution. We have the same justification for VA(M ). Remark 8.1.25. Proposition 8.1.23 and Corollary 8.1.24 do not use the hypothesis of positivity on the scalar product. It is necessary to note that the notions of nondegen-

422 | 8 Deformation of topological modules eracy introduced here are different from those introduced in [50], in which the unitary case is considered. For the sake of brevity, note respectively V(M ) = V(πν ) and VA(M ) = VA(πν ) for the characteristic and the Poisson characteristic varieties. 8.1.7 Topologically convergent modules We need later to specialize the parameter of deformation ν to be a nonzero complex number for the sake of convergence of the formal series. Let M = M ⟦ν⟧ be a topological free module on the deformed algebra A = A⟦ν⟧, and let πν be the associated representation. We further assume that M is a locally convex topological space. Let A0 be the sub-ℂ[ν]-algebra of A generated by A. This is the set of sums: N

∑ νj αj ,

j=0

where N ∈ ℕ and each αj is a sum of terms of type a1 ∗ ⋅ ⋅ ⋅ ∗ ar , with a1 , . . . , ar ∈ A. We say that M = M ⟦ν⟧ is weakly convergent if there is R > 0 such that for all a ∈ A0 and all m ∈ M, the entire series πν (a)m weakly converges for ν = ν0 in the disk of radius R to a vector of M, which will be noted πν̃ 0 (a)m. The uniqueness of this weak limit is ensured by the Hahn–Banach theorem. The radius of convergence RM of the module is then defined as the upper limit of the radius R above. Proposition 8.1.26. A topological free weekly convergent module M = M ⟦ν⟧ of radius RM induces a family of representations (πν̃ 0 )ν0 ∈D(0,RM ) of A0 in M. Proof. For all a, b ∈ A, m ∈ M and for all m′ in the topological dual M ′ , we have equality between the formal series: ⟨m′ , πν (a ∗ b)m⟩ = ⟨m′ , πν (a)πν (b)m⟩.

(8.24)

The equality still holds when a and b are in A0 . We have convergence of these two integer series at ν = ν0 ∈ D(0, RM ), and the right member is also the limit of the entire series ⟨m′ , πν (a)πν̃ 0 (b)m⟩ at ν = ν0 . We have then for all ν0 ∈ D(0, RM ) the equality: πν̃ 0 (a ∗ b)m = πν̃ 0 (a)πν̃ 0 (b)m.

(8.25)

8.2 Case of linear Poisson manifolds We now apply the previous results to the case of representations of Lie algebra. Let V = ℝd be a linear Poisson manifold. Then the dual V ∗ of linear forms on V form

8.2 Case of linear Poisson manifolds | 423

a Lie subalgebra g of the algebra A = S(gℂ ) of polynomials on V endowed with the Poisson bracket. Let A = A⟦ν⟧ be the associated deformed algebra. We then regard the Poisson manifold V as the dual g∗ of the Lie algebra g. The Poisson bracket of Kirillov–Kostant–Souriau is given for φ, ψ ∈ A and ℓ ∈ g by the formula (cf. [61]): {φ, ψ}(ℓ) := ⟨ξ , [dφ(ℓ), dψ(ℓ)]⟩.

(8.26)

Note that is possible to specialize the indeterminate ν to a nonnull value in the expression of the star-product (cf. [68]): we introduce the family of complex parameter of Lie ℂ algebras gℂ ν0 , of the underlying same space g with the Lie bracket: [X, Y]ν0 = ν0 [X, Y]. The evaluation at ν = ν0 provides a noncommutative law ∗ν0 on S(gℂ ), which is the multiplication of the enveloping algebra of gℂ ν0 transported by Duflo isomorphism. We shall also introduce the notion of weakly convergent topologically free module, which allows us to specialize the indeterminate ν to a nonzero value for representations. In a precise way, a weakly convergent topologically free module on A is a topologically free A -module on M = M ⟦ν⟧, where M is a locally convex separated topological space, such that there exists R > 0 meeting the property that for all a ∈ A0 and all m ∈ M, the whole series πν (a)m is weakly convergent of radius R. Here, A0 designates the sub-ℂ[ν]-algebra of A generated by A, and πν the associate representation. The essential ingredient here is the specialization of the deformation parameter ν, being any purely imaginary, which allows (cf. Proposition 8.1.26) to associate to a weakly convergent strongly unitary topologically free module on the deformed algebra, a family of unitary representations (ρℏ ) of Lie algebras labeled by a real parameter ℏ. Let then (gℏ )ℏ∈ℝ be the family of a real parameter of real Lie algebras of same underlying space g with the bracket: [X, Y]ℏ = ℏ[X, Y]. We prove the equivalence between the notion of strongly unitary weakly convergent A -module of radius R and the notion of one parameter family (ρℏ )ℏ∈]−R,R[ of unitary representations of gℏ such that the associated family of representations of U (gℏ ) ≃ (S(g), ∗ℏ ) depends weakly analytically of the parameter ℏ. The transition form is given by the formula: ρℏ (X) = −iπν (X)|ν=iℏ , for all X ∈ g. The factor i is explained by the fact that ℏ is real while the indeterminate ν is formally pure imaginary.

424 | 8 Deformation of topological modules We then develop some examples in the linear Poisson setting, more precisely in the nilpotent and solvable cases, placed away from Verma modules treated in conclusion. In the last part, using the method of Kirillov orbits [91] and results of N. V. Pedersen [118, 119], we determine, in the case of exponential solvable Lie algebras the characteristic manifold of a strongly unitary module obtained by unitary induction of any real polarization. We also show that when V is the dual of a nilpotent Lie algebra, any symplectic leaf (i. e., any coadjoint orbit) can be seen as the characteristic Poisson manifold of a well-chosen strongly unitary A -module M . 8.2.1 The algebra (A , ∗) It results from the works of B. Shoikhet (cf. [70] and [72]), related to the annihilator of the associated weights on the “wheels” that the two star-products # and ∗ coincide. The algebra (A , ∗) is isomorphic to the enveloping formal complexified algebra: Uν (g ) = T(g )⟦ν⟧/⟨x ⊗ y − y ⊗ x − ν[x, y]⟩, ℂ

ℂ

(8.27)

and we have precisely f ∗ g = τ−1 (τf .τg).

(8.28)

Here, τ : A → Uν (gℂ ) is the Duflo isomorphism (cf. [64]): τ = σ ∘ J(D)1/2 , where σ denotes the symmetrization map and J(D)1/2 is the differential operator of infinite order with constant coefficients corresponding to the formal series J(x)1/2 = (det

sh ad ν2 x ad ν2 x

1/2

) .

(8.29)

Note that (Sn (gℂ ))n≥0 is the increasing usual filtration of the symmetric algebra. We can then specialize the value of the deformation parameter ν. Indeed, for all f , g in A, the series at ν defining f ∗ g is polynomial in ν, and thus can be evaluated in any complex number ν. The star-product thus generates a family of noncommutative associative laws (∗ν ) on A, the parameter ν varying through the field of complex numbers. Each of these algebras is identified via the Duflo isomorphism τν to the enveloping Lie algebra ℂ gℂ ν of the underlying vector space g but with the bracket defined by [x, y]ν = ν[x, y]. For a real parameter ℏ, it is worth to notice that the real Lie algebra gℏ is nothing but the underlying vector space g endowed with the bracket defined by [x, y]ℏ = ℏ[x, y].

8.2 Case of linear Poisson manifolds | 425

8.2.2 Converging topological free modules on (A , ∗) Let as above A0 designate the ℂ[ν]-subalgebra of A generated by A. As the entire series a ∗ b is a polynomial at ν for all a, b ∈ A, we have A0 = A[ν].

For all ν0 ∈ ℂ, the evaluation in ν0 , evν0 : A0 n

∑ νk ak

k=0

󳨀→

(A, ∗ν0 )

󳨃󳨀→

∑ ν0k ak

n

k=0

is a morphism of ℂ-algebras. We have the following. Proposition 8.2.1. Let R > 0, and let for all ν0 ∈ D(0, R), πν0 be a representation of the algebra (A, ∗ν0 ) on a separated locally convex topological vector space M. We assume that for all a ∈ A, m ∈ M and ν0 ∈ D(0, R), the vector πν0 (a)m is given by the evaluation at ν0 of a weakly convergent entire series of radius ≥ R. Then for ν0 in the disk of radius R is induced a representation πν̃ 0 = πν0 ∘ evν0 of A0 in M, and M = M ⟦ν⟧ is then a topological weakly convergent-free module of radius ≥ R. Conversely, a weakly convergent topological-free module of radius ≥ R induces for all ν0 ∈ D(0, R) a representation πν0 of (A, ∗ν0 ) in M. Proof. Let a, b ∈ A and m ∈ M. The equality, πν0 (a ∗ν0 b)m = πν0 (a)πν0 (b)m, for all ν0 ∈ D(0, R) implies the equality between the formal series: πν (a ∗ b)m = πν (a)πν (b)m, which makes of M = M ⟦ν⟧ a topological free module. It is by construction weakly convergent of radius ≥ R. Conversely, if M is a separated locally convex topological vector space and if M = M ⟦ν⟧ is a weakly convergent topological free A -module of radius ≥ R, consider for all ν0 ∈ D(0, R) the representation πν̃ 0 of A0 in M as given in Proposition 8.1.26. If we note πν0 the restriction of πν̃ 0 to A ⊂ A0 , we have immediately for all m ∈ M: πν0 (a)πν0 (b)m = πν̃ 0 (a)πν̃ 0 (b)m = πν̃ 0 (a ∗ b)m

= πν0 (a ∗ν0 b)m.

426 | 8 Deformation of topological modules 8.2.3 Unitarity A unitary representation of a Lie algebra g is a representation ρ realized on a preHilbertian space such that the operators ρ(X) are anti-Hermitian for all X ∈ g. The following theorem shows how to connect a strongly unitary topological free module on A to a family of unitary representations ρℏ of the Lie algebra gℏ , when ℏ takes real values. Theorem 8.2.2. Let R > 0, and let M be a separated pre-Hilbertian locally convex topological vector space M. Assume that M = M ⟦ν⟧ is a weakly convergent topological free module of radius ≥ R. For all ν0 ∈ D(0, R), let πν0 be the representation of the algebra (A, ∗ν0 ) in M associated to M by Proposition 8.1.26. Then: (1) The identity ρℏ (X) = −iπiℏ (X) for X ∈ gℏ , defines a representation ρℏ of the Lie algebra gℏ in M for all ℏ ∈ D(0, R). (2) The following two assertions are equivalent: (a) For all ℏ ∈ ]−R, R[, the representation ρℏ is unitary. (b) The topological free module M = M ⟦ν⟧, endowed with the scalar product taking values in ℂ⟦ν⟧ naturally extending that of M, is strongly unitary. Proof. We have for X, Y ∈ g: [ρℏ (X), ρℏ (Y)] = −[πiℏ (X), πiℏ (Y)] = −iℏπiℏ ([X, Y]) = ℏρℏ ([X, Y])

= ρℏ ([X, Y]ℏ ), hence the first part of the theorem. The unitarity of the topological module M = M ⟦ν⟧ allows to claim that for all a ∈ A and u, v ∈ M , the equality between the formal series holds: ⟨πν (a)u, v⟩ = ⟨u, πν (a∗ )v⟩. As we have assumed that the indeterminate ν is purely imaginary, this equality is specialized (in view of Proposition 8.1.26) in all parameters ν0 ∈ i]−R, R[: for all a ∈ A and u, v ∈ M, we have ⟨πν0 (a)u, v⟩ = ⟨u, πν0 (a∗ )v⟩. As X ∗ = X for all X ∈ g, we see immediately that the operators πiℏ (X) are Hermitian for all ℏ ∈ ]−R, R[. The operator ρℏ (X) is thus anti-Hermitian, hence the implication (b) ⇒ (a). The converse is immediate.

8.3 The Zariski closure conjecture |

427

8.3 The Zariski closure conjecture We focus in this section on the setting of exponential solvable Lie groups. We first determine the characteristic variety of a representation πν associated in a natural way to an arbitrary unitary monomial representation induced by a real polarization (using [119]). and then substantiate the Zariski closure conjecture.

8.3.1 Kirillov–Bernat theory for exponential groups Let G be an exponential solvable real Lie group G with Lie algebra g. Any ξ ∈ g∗ admits a Pukánszky polarization, that is, a maximal vector subspace h of g such that ⟨ξ , [h, h]⟩ = {0}, which is moreover a Lie subalgebra of g such that Pukánszky’s condition: Ad∗ H.ξ = ξ + h⊥

(8.30)

holds, where H = exp h is the analytic subgroup of G with Lie algebra h. Consider the unitary induced representation: ̃G χ , π ξ ,h = Ind H ξ

(8.31)

where χξ denotes the unitary character of H defined by χξ (exp X) = e−i⟨ξ ,X⟩ .

(8.32)

The representation π ξ ,h is unitary and irreducible; its class does not depend on the ′ choice of the Pukánszky polarization h, and two such representations π ξ ,h and π η,h are equivalent if and only if ξ and η are in the same coadjoint orbit. Moreover, any unitary irreducible representation of G arises this way. The Kirillov map, κ : g∗ /G Ad∗ G.ξ

󳨀→ 󳨃󳨀→

̂ G [π ξ ,h ],

thus obtained is then a bijection. There are natural topologies on both sides of the Kirillov map: the quotient topology on the space g∗ /G of coadjoint orbits, and the Fell ̂ topology on the unitary dual G. Theorem 8.3.1. The Kirillov map κ is a homeomorphism. This theorem has been proved by I. D. Brown in 1973 (cf. [49]) for nilpotent, simply connected groups, and by H. Leptin and J. Ludwig in 1994 (cf. [106]) for general exponential groups (see also [26] for several details). Let Of be the coadjoint orbit of

428 | 8 Deformation of topological modules f ∈ g∗ and d = dim Of . Let pr : Of → G/H be the surjection defined by: pr(g.f ) = gH.

(8.33)

This is well-defined because H contains the stabilizer Gf of f . We consider the restriction of the Poisson bracket from g∗ to the symplectic leaf Of . According to [119], we designate by ℰ 0 (Of ) the space of functions φ = ψ ∘ pr, where ψ ∈ C ∞ (G/H), and we designate by ℰ 1 (Of ) the normalizer de ℰ 0 (Of ) in C ∞ (Of ) with respect to the Poisson bracket. Any X ∈ g can be considered as a linear function on g∗ . It induces by restriction a C ∞ function on Of that we denote always X. Lemma 8.3.2. For all X ∈ g, the function X belongs to ℰ 1 (Of ). Proof. The Hamiltonian field HX associated to the function X is equal to the fundamental field given by the coadjoint action: HX φ(η) =

d 󵄨󵄨󵄨󵄨 󵄨 φ(exp(−tX).η), dt 󵄨󵄨󵄨t=0

η ∈ Of .

(8.34)

Note that φ ∈ ℰ 0 (Of ) if and only if φ(gh.f ) = φ(g.f ) for all g ∈ G and h ∈ H, or if HX φ = 0 for all X ∈ h. Hence it is clear that for all φ ∈ ℰ 0 (Of ) and all t ∈ ℝ, η 󳨃→ φ(exp −tX.η) belongs also to ℰ 0 (Of ). We deduce that HX .φ = {X, φ} belongs also to ℰ 0 (Of ). Let τ = (τ1 , . . . , τd/2 ) : G/H 󳨀→ Ω ⊂ ℝd/2 a global chart. We will suppose here that the open set Ω is equal to the whole ℝd/2 . It is always possible in the case of a solvable exponential group choosing a coexponential basis (X1 , . . . , Xd/2 ) to h in g as in Theorem 1.1.5, and considering ∼

τ−1 (x1 , . . . , xd/2 ) = exp x1 X1 ⋅ ⋅ ⋅ exp xd/2 Xd/2 H.

(8.35)

Assume that the global chart τ is defined as above. Let (q1 , . . . , qd/2 ) be the functions in ℰ 0 (Of ) defined by qj = τj ∘ pr. We first record the following two results [119, Theorems 2.2.2 and 3.2.3]. Theorem 8.3.3. There exists a family (p1 , . . . , pd/2 ) in ℰ 1 (Of ) such that (p1 , . . . , pd/2 , q1 , . . . , qd/2 ) forms a Darboux global chart, Φ : Of 󳨀→ Wf × ℝd/2 , ∼

(8.36)

8.3 The Zariski closure conjecture

| 429

where Wf is an open set in ℝd/2 , which means that {pi , pj } = {qi , qj } = 0,

j

{pi , qj } = δi .

(8.37)

This open set is the whole ℝd/2 if and only if the polarization h satisfies the Pukanszky condition (8.30). Theorem 8.3.4. Let (p, q) be a global Darboux coordinate system: Of 󳨀→ Wf × ℝd/2 ⊂ ℝd as in Theorem 8.3.3. Then: (1) With respect to these coordinates, the space ℰ 0 (Of ) is identified to the space of functions depending only on q, and the space ℰ 1 (Of ) is identified to the space of functions: ∼

d/2

φ(p, q) = ∑ au (q)pu + a0 (q), u=1

(8.38)

where a0 , a1 , . . . , ad/2 ∈ C ∞ (ℝd/2 ). In particular, for X ∈ g one has d/2

X(p, q) = ∑ aX,u (q)pu + aX,0 (q), u=1

(8.39)

where the aX,u , u = 0, . . . , d/2 are functions in C ∞ (ℝd/2 ). (2) There exists a unique unitary strongly continuous representation ρ of G in L2 (ℝd/2 ) such that the space ℋρ∞ of its C ∞ -vectors contains Cc∞ (ℝd/2 ), and such that for all ξ ∈ C ∞ (ℝd/2 ) we have d/2

ρ(X)ξ (t) = ∑ aX,u (t) u=1

d/2 𝜕aX,u 𝜕ξ (t) 1 − iaX,0 (t)ξ (t) + ( ∑ (t))ξ (t). 𝜕tu 2 u=1 𝜕tu

(8.40)

Remark 8.3.5. This representation is equivalent to the induced representation IndGH χf . The expression of the functions X(p, q) follows immediately from Lemma 8.3.2. In addition, the functions aX,u are entire with at most an exponential growth (cf. [7, Theorem 1.6]). We can also deduce an information about the value at q = 0 of the functions aX,u . Lemma 8.3.6. (1) For any X ∈ g, we have aX,0 (0) = ⟨f , X⟩. (2) For any X ∈ h and any u = 1, . . . , d/2, we have aX,u (0) = 0. (3) aXj ,u (0) = −δju , for any j = 1, . . . , d/2 and any u = 1, . . . , d/2. Proof. The first statement follows immediately from the fact that the point of Of of coordinates (0, 0) is f . The second follows from the fact that the set of points of coor-

430 | 8 Deformation of topological modules dinates (p, q) with q = 0 and H.f , is also included in f + h⊥ . As to the third assertion, it comes from a direct computation: aXj ,u (0) =

𝜕 X (p, 0) 𝜕pu j

= −{qu , Xj }(p, 0)

= {Xj , qu }(p, 0) d 󵄨󵄨󵄨 = 󵄨󵄨󵄨 qu (exp −tXj .(p, 0)) dt 󵄨󵄨t=0 d 󵄨󵄨󵄨 = 󵄨󵄨󵄨 qu (p, (0, . . . , −t, . . . , 0)) (−t in the jth position) dt 󵄨󵄨t=0 = −δju .

8.3.2 Construction of a representation πν of 𝒜 We apply the previous construction to a family (Gℏ )ℏ∈ℝ−{0} of exponential groups. More precisely, let (gℏ )ℏ∈ℝ be the family of solvable Lie algebras defined by the same underlying vector-space g, with the bracket [X, Y]ℏ = ℏ[X, Y]. We will denote by expℏ the exponential of gℏ in Gℏ . Let f ∈ g∗ , the coadjoint orbit Of ,ℏ = Gℏ .f ⊂ g∗ is the same for all ℏ ≠ 0, but the Poisson structure depends on ℏ: for all η ∈ g and φ, ψ ∈ C ∞ (g∗ ), we have {φ, ψ}ℏ (η) = ℏ{φ, ψ}1 (η) = ℏ⟨η, [dφ(η), dψ(η)]⟩.

(8.41)

We denote always Of this common orbit, when it is not necessary to precise the Poisson structure. However, the orbit Of ,0 degenerates and is reduced to the point f , since the group G0 is Abelian. There exists a subspace h of g, which is a real polarization of f in gℏ for all ℏ ≠ 0. Let Hℏ = expℏ h ⊂ Gℏ . Let χf ,ℏ be the character of Hℏ defined by χf ,ℏ (expℏ X) = e−i⟨f ,X⟩ .

(8.42)

We use the results recorded in Subsection 8.3.1 to construct a simultaneous realizaG tion of all the induced IndHℏ χf ,ℏ in the same L2 (ℝd/2 ). In fact, if (p, q) is the global ℏ Darboux coordinate system of Theorem 8.3.4 for Of ,1 (corresponding to ℏ = 1), then for all ℏ ≠ 0, a global Darboux coordinate system for Of ,ℏ is given by (p, q′ ) = (p, ℏ−1 q) = (p1 , . . . , pd/2 , ℏ−1 q1 , . . . , ℏ−1 qd/2 ). For all X ∈ g, the corresponding function

8.3 The Zariski closure conjecture |

431

writes d/2

X(p, q′ ) = ∑ aX,u (q)pu + aX,0 (q) u=1 d/2

= ∑ aX,u (ℏq′ )pu + aX,0 (ℏq′ ). u=1

Thus, by Theorem 8.3.4, there exists for all ℏ ≠ 0 a unique unitary strongly continuous representation ρℏ from Gℏ to L2 (ℝd/2 ) such that the space ℋρ∞ℏ of its C ∞ vectors

contains Cc∞ (ℝd/2 ), and verifying d/2

ρℏ (X)ξ (t) = ∑ aX,u (ℏt) u=1

d/2 𝜕aX,u 𝜕ξ (t) ℏ − iaX,0 (ℏt)ξ (t) + ( ∑ (ℏt))ξ (t). 𝜕tu 2 u=1 𝜕tu

(8.43)

G

This representation is equivalent to the induced representation IndHℏ χf ,ℏ . This repℏ resentation is irreducible if and only if the polarization h verifies the Pukanszky condition Hℏ .f = f + h⊥ for an arbitrary ℏ ≠ 0. Otherwise, one can easily realize the G induced representation IndH0 χf ,0 in L2 (ℝd/2 ), and we obtain that 0

ρ0 (exp tXj ).φ(t1 , . . . , td/2 ) = φ(t1 , . . . , tj−1 , tj − t, tj+1 , . . . , td/2 ), and for any X ∈ hh, ρ0 (exp tX).φ(t1 , . . . , td/2 ) = e−it⟨f ,X⟩ φ(t1 , . . . , td/2 ). By differentiating in t = 0, we obtain ρ0 (Xj ).φ(t1 , . . . , td/2 ) = −

𝜕 φ(t1 , . . . , td/2 ) 𝜕tj

and for all X ∈ h, ρ0 (X).φ(t1 , . . . , td/2 ) = −i⟨f , X⟩.φ(t1 , . . . , td/2 ).

(8.44)

Thanks to Lemma 8.3.6, the representation ρ0 of G0 in Cc∞ (ℝd/2 ) is obtained by extending to ℏ = 0 the formula (8.43). Let now K be a fixed compact in ℝd/2 with a nonempty interior, and let M = CK∞ (ℝd/2 ) be the space of C ∞ -functions supported in K. This space is for all real numbers ℏ, a submodule of the C ∞ -vectors module of ρℏ . M is endowed with the Fréchet topology defined by the seminorms: 󵄨 󵄨 Nk (φ) = sup sup󵄨󵄨󵄨Dα φ(t)󵄨󵄨󵄨. |α|≤k t∈K

The topological dual of M is the space of distributions supported in K. Let φ ∈ M and T ∈ M ′ . Since the functions aX,u are entire functions, applying Lemma 8.3.6,

432 | 8 Deformation of topological modules we see that the expression ⟨T, ρℏ (w)φ⟩ is entire as a function of the variable ℏ for all w ∈ S(g), by identifying S(g) and 𝒰 (gℏ ) via the Duflo isomorphism. We consider πν0 (X) = iρ−iν0 (X),

(8.45)

and we obtain a family (πν0 ) of representations (A, ∗ν0 ) in M satisfying the conditions of Proposition 8.2.1 (with R = +∞). As above, one obtains a structure of weakly convergent and strongly unitary topologically free module ℳ = M ⟦ν⟧. The expression of the representation πν of the deformed algebra 𝒜 in ℳ is obtained as follows: for all X ∈ g, one has d

d 2

𝜕ξ ν 2 𝜕 (t) + aX,0 (−iνt)ξ (t) + ∑ a (−iνt)ξ (t), πν (X)ξ (t) = ∑ iaX,u (−iνt) 𝜕tu 2 u=1 𝜕tu X,u u=1

(8.46)

and the action of an element of 𝒜 is obtained via the identification 𝒜 = 𝒰ν (gℂ ) given by the Duflo isomorphism.

8.3.3 The characteristic variety Theorem 8.3.7. Let g be a exponential solvable Lie algebra, f an element of g∗ and h a real polarization at f . Let d be the dimension of the coadjoint orbit of f , and let K be a compact in ℝd/2 with a nonempty interior. Let πν be the representation of the deformed algebra 𝒜 on the convergent topologically free module M = CK∞ (ℝd/2 )⟦ν⟧ constructed as in Subsection 8.3.2. Then V(πν ) = f + h⊥ .

(8.47)

Particularly, if h satisfies the Pukanszky condition (8.30), then we get that V(πν ) = H.f . Proof. Taking ν = 0 in (8.46), we obtain d 2

π0 (X) = ∑ iaX,u (0) u=1

𝜕 + aX,0 (0). 𝜕tu

By Lemma 8.3.6, we have 𝜕 π0 (Xj ) = −i , j = 1, . . . , d/2, 𝜕t j ( ) π0 (X) = ⟨f , X⟩ for any X ∈ h.

8.3 The Zariski closure conjecture

| 433

The annihilator of π0 is the ideal of S(g) generated by X−⟨f , X, X ∈ h. As a consequence, we obtain the equality V(πν ) = f + h⊥ . 8.3.4 Fundamental examples We compute in this section the characteristic and the Poisson characteristic varieties for some examples of Poisson linear manifolds. The following lemma will be of use along the section. Lemma 8.3.8. Let K be a compact of ℝn of nonempty interior, m a nonzero integer, R > 0, and let (Pν )ν∈D(0,R) be a family of differential operators of order ≤ m whose coefficients restricted to K analytically depends upon ν. Then for all φ ∈ Cc∞ (ℝn ) and all distribution T with supports included in K, the function ν 󳨃→ ⟨T, Pν (φ)⟩ is analytic on D(0, R). Proof. Let CK∞ (ℝn ) be the space of smooth functions on ℝn of support included in K. We endow this space with the Frechet topology defined by seminorms: 󵄨 󵄨 Nk (φ) = sup sup󵄨󵄨󵄨Dα φ(x)󵄨󵄨󵄨. |α|≤k x∈K

We consider the Laplacian Δ on ℝn . There exists an integer L such that the distribution (1 − Δ)−L T is a continuous function on K, since T is of finite order. We have then by integration by parts: ⟨T, Pν (φ)⟩ = ∫(1 − Δ)−L T(x)(1 − Δ)L Pν (φ)(x)dx. K

On the other hand, (1 − Δ)L Pν (φ)(x) = Pν′ (φ)(x) = ∑ νk Qk (φ)(x), k≥0

where Qk are differential operators of order ≤ m + 2L. The coefficients of Pν′ being given on K by convergent entire series on D(0, R), there exists for all r < R a constant C such that the absolute values of the coefficients of Qk are all dominated on K by Cr −k . We have then 󵄨 󵄨 sup󵄨󵄨󵄨Qk (φ)(x)󵄨󵄨󵄨 ≤ C ′ r −k Nm+2L (φ), x∈K

from where the domination 󵄨󵄨 󵄨󵄨 󵄨󵄨 󵄨 󵄨 󵄨 −k −L ′ −L 󵄨󵄨∫(1 − Δ) T(x)Qk φ(x) dx󵄨󵄨󵄨 ≤ C (Vol K)Nm+2L (φ) sup󵄨󵄨󵄨(1 − Δ) T(x)󵄨󵄨󵄨r 󵄨󵄨 󵄨󵄨 x∈K K

434 | 8 Deformation of topological modules holds on the disk of radius r. The expression ⟨T, Pν (φ)⟩ is thus given by the convergent entire series ⟨T, Pν (φ)⟩ = ∑ νk ∫(1 − Δ)−L T(x)Qk φ(x) dx. k≥0

K

This being true for all r < R, we have convergence of this entire series on D(0, R). 8.3.5 The Heisenberg groups We here refer to Subsection 1.1.2 and consider the Heisenberg group H2n+1 = ℝn ×ℝn ×ℝ with the product 1 (x, y, z) ⋅ (x′ , y′ , z ′ ) = (x + x′ , y + y′ , z + z ′ + (xy′ − x ′ y)), 2 its Lie algebra h2n+1 is generated by the family of vector fields {X1 , . . . , Xn , Y1 , . . . , Yn , Z} whose Lie brackets are given by [Xi , Xj ] = δij Z,

i, j = 1, . . . , n,

δij being the Kronecker symbol. For n

n

i=1

i=1

fλ = λZ ∗ + ∑ ai Xi∗ + ∑ bi Yi∗ ∈ h∗2n+1 , consider the representation ρλ associated to fλ by the orbit method. We now discuss the following two cases. Case 1: If λ ≠ 0, we can take fλ = λZ ∗ . Indeed, the dimension of ρλ is infinite and the associated orbit of fλ under the coadjoint action is Ωfλ = {(λ, u, v), u, v ∈ ℝn }. If one realises ρλ using the polarization b = ℝ-span{Y1 , Y2 , . . . , Yn , Z}, then ρλ acts on the space L2 (ℝn ). Also note ρλ its differential, which is realized on the Schwartz space S (ℝn ), the Frechet space of the C ∞ -vectors of ρλ (cf. [91]). It is given by λ

ρ (Z) = −iλ, { { { λ ρ (Xi ) = − 𝜕t𝜕 , i = 1, . . . , n, { i { { λ {ρ (Yj ) = itj , j = 1, . . . , n.

8.3 The Zariski closure conjecture |

435

Thus, the representation ρλℏ is defined by the relations: λ

ρ (Z) = −iλ, { { { ℏλ ρℏ (Xi ) = − 𝜕t𝜕 , i = 1, . . . , n, { i { { λ {ρℏ (Yj ) = iℏtj , j = 1, . . . , n. These expressions depend polynomially upon ℏ; this allows to use Proposition 8.1.26 and Theorem 8.2.2, which makes of M = S (ℝn )⟦ν⟧ a weakly convergent strongly unitary topological free module (of infinite radius). The associated representation of A is given by λ

πν (Z) = λ, { { { λ πν (Xi ) = −i 𝜕t𝜕 , { i { { λ {πν (Yj ) = −iνtj ,

i = 1, . . . , n,

j = 1, . . . , n.

The annihilator of πνλ is therefore generated by Z − λ, and thus VA(πνλ ) = Ωfλ . It immediately results that λ

π (Z) = λ, { { { 0λ π (X ) = −i 𝜕t𝜕 , i = 1, . . . , n, { i { 0 i { λ {π0 (Yj ) = 0, j = 1, . . . , n. We then deduce that Ann(π0λ ) is generated by the family {Z −λ, Yj , j = 1, . . . , n}. It comes then V(πνλ ) = {l ∈ h∗2n+1 : l(Z − λ) = 0 and l(Yj ) = 0, j = 1, . . . , n} n

= λZ ∗ ⨁ℝXi∗ = fλ + b⊥ . i=1

A similar calculation shows that V(πνλ ) = fλ + b′ if we realize ρλ using the polarization b′ = ℝ-span{X1 , . . . , Xn , Z}. Therefore, we can clearly observe that V(πνλ ) depends upon the realization of π λ . ⊥

Case 2: If λ = 0, then the orbit Ωf0 reduces to {f0 } and the associated polarization to f0 is the Lie algebra h2n+1 . Hence, ρ0 (X) = −if0 (X) for all X ∈ h2n+1 .

436 | 8 Deformation of topological modules In this case, we have πν0 (X) = π00 (X), for all X ∈ h2n+1 . We then deduce that the annihilator of πν0 is the ideal generated by the family {X − f0 (X), X ∈ h2n+1 }. It follows that V(πν0 ) = {l ∈ h∗2n+1 /l(X − f0 (X)) = 0, X ∈ h2n+1 } = {f0 }, and the same, VA(πν0 ) = {f0 }. 8.3.6 The n-step threadlike Lie algebras We now refer to Subsection 1.1.3. Consider the n-step threadlike Lie algebras, Gn of Lie algebra gn of dimension n + 1 and endowed with a basis {X1 , . . . , Xn+1 } with the nonvanishing brackets: [Xn+1 , Xj ] = Xj−1 ,

j = 2, . . . , n.

∗ Let {X1∗ , . . . , Xn+1 } be the dual basis of g∗ . The center of g is generated by the vector X1 . ∗ ∗ Let l = l1 X1 + ⋅ ⋅ ⋅ + ln+1 Xn+1 ∈ g∗ with l1 ≠ 0. Then b(l) = ℝ-span{X1 , . . . , Xn } is an Abelian ideal of gn , which polarizes l. As the Pukanszky indices set is {2, n + 1}, it can be assumed without loss of generalities, that l2 = ln+1 = 0 (cf. [18]). The irreducible unitary representation ρ = ρl associated to l is then realized on L2 (ℝ). Its differential is given by

ρ(Xn+1 ) = − 𝜕t , { { { { { {ρ(X1 ) = −il1 , { { { {ρ(X ) = itl , { { 2 1 1 2 { { ρ(X ) = −i(l 3 3 + 2 t l1 ), { { { { .. { { { . { { { l1 1 2 n−3 l3 n−3 + (−1)n−1 (n−1)! t n−1 ). {ρ(Xn ) = −i(ln − ln−1 t + 2 ln−2 t + ⋅ ⋅ ⋅ + (−1) (n−3)! t 𝜕

It follows therefore that the representation ρℏ is determined by ρℏ (Xn+1 ) = − 𝜕t , { { { { { ρ { ℏ (X1 ) = −il1 , { { { { { {ρℏ (X2 ) = +iℏtl1 , { { ρℏ (X3 ) = −i(l3 + 21 ℏ2 t 2 l1 ), { { { { .. { { { . { { { 1 2 2 {ρℏ (Xn ) = −i(ln − ln−1 ℏt + 2 ln−2 ℏ t + ⋅ ⋅ ⋅ + 𝜕

l3 (−ℏt)n−3 (n−3)!

+

l1 (−ℏt)n−1 ). (n−1)!

8.3 The Zariski closure conjecture |

437

These expressions being polynomials upon ℏ, we can also apply Proposition 8.1.26 and Theorem 8.2.2, which makes of M = S (ℝ)⟦ν⟧ a strongly unitary weakly convergent module. We get πν (Xn+1 ) = −i 𝜕t𝜕 , { { { { { πν (X1 ) = l1 , { { { { { {πν (X2 ) = +iνtl1 , { { { πν (X3 ) = l3 − 21 ν2 t 2 l1 , { { { {. { .. { { { { { l1 1 2 2 n−3 l3 n−3 + (iν)n−1 (n−1)! t n−1 . {πν (Xn ) = ln + iνln−1 t − 2 ν ln−2 t + ⋅ ⋅ ⋅ + (iν) (n−3)! t Making ν = 0, we get π0 (Xn+1 ) = −i 𝜕t , { { { { { {π0 (X1 ) = l1 , { { { {π (X ) = 0, { { 0 2 { { π0 (X3 ) = l3 , { { { { .. { { { . { { { {π0 (Xn ) = ln . 𝜕

Noticing that Ann(π0 ) is generated by the family {X1 −l1 , X2 , X3 −l3 , . . . , Xn −ln }, it comes that for f = (f1 , . . . , fn+1 ) ∈ g∗ , f ∈ V(πν ) if and only if f ∈ l + b(l)⊥ . On the other hand, we see that the annihilator of πν is generated by the family vk , k = 1, . . . , n, with vk = Xk − lk −

l3 lk−1 l 1 X − k−2 X 2 − ⋅ ⋅ ⋅ − X k−3 − X2k−1 . k−3 2 l1 2 2l12 2 (k − 1)!l1k−2 (k − 3)!l1

The Poisson characteristic variety VA(πν ) is therefore equal to the coadjoint orbit Ωl .

8.3.7 The affine group of the real line Let G = Aff(ℝ), its Lie algebra admits two generators X and Y such that [X, Y] = Y. This group admits two unitary irreducible representations ρ+ and ρ− associated respectively to the linear functionals Y ∗ and −Y ∗ . The differentials of these representations are defined by d ρ+ (X) = − dx , { −x ρ+ (Y) = −ie

and

d ρ− (X) = − dx , { −x ρ− (Y) = −ie .

438 | 8 Deformation of topological modules Hence, the expressions of ρ+,ℏ and ρ−,ℏ are respectively given by d , ρ+ , ℏ(X) = − dx

{

ρ+ , ℏ(Y) = −ie−ℏx

and

d , ρ− , ℏ(X) = − dx { −ℏx ρ− , ℏ(Y) = −ie .

We restrict these differentials to M = CK∞ (ℝ) where K is a compact set of nonempty interior. Applying first Lemma 8.3.8 then Proposition 8.2.1 and Theorem 8.2.2, we make of ℳ = M ⟦ν⟧ a weakly convergent unitary strongly module (of infinite radius). The annihilators of the representations π+ν and π−ν are trivial, and consequently, the sets VA(π+,ν ) and VA(π−,ν ) are equal to g∗ . Taking ν = 0, it follows that the annihilators of π+,0 and π−,0 are respectively generated by ⟨Y − 1⟩ and ⟨Y + 1⟩. We then obtain that V(π+,ν ) = {l = xX ∗ + yY ∗ ∈ g∗ /∀φ ∈ Ann π+,0 , φ(l) = 0} = {l = xX ∗ + yY ∗ ∈ g∗ /y − 1 = 0} = Y ∗ + b⊥ ,

where b = ℝY is the polarization associated to the functionals Y ∗ and −Y ∗ . Similarly, V(π−,ν ) = {l = xX ∗ + yY ∗ ∈ g∗ /y + 1 = 0} = −Y ∗ + b⊥ . Remark 8.3.9. For these two representations, the Poisson characteristic manifold coincides with the Zariski closure of the associated coadjoint orbit. 8.3.8 A 3-dimensional exponential solvable Lie group Let g be the Lie algebra generated by the three vectors {A, X, Y} whose Lie brackets are given by [A, X] = X − Y, [A, Y] = X + Y, and let G = exp g. Hence, G is an exponential noncompletely solvable Lie group. Let f = xX ∗ + yY ∗ + aA∗ ∈ g∗ . If x2 + y2 = 0, then the orbit of f is reduced to the unit set {f }. Thereby, the computation done in Example 8.3.7 shows that in this case, V(πf ,ν ) = {f }. In the case where x 2 + y2 ≠ 0, the subalgebra b = ℝ-span{X, Y} is a polarization of f satisfying the Pukanszky condition. Then let χf be the character defined on B = exp b by χf (exp U) = e−if (U) and ρf = IndGB χf . We know (cf. [5]) that there exists a unique θ ∈ [0, 2π[ such that ρ = ρθ = ρfθ où fθ = cos θX ∗ + sin θY ∗ . The orbit Ω associated to ρ is parameterized by Ω = {sA∗ + e−t cos(t + θ)X ∗ + e−t sin(t + θ)Y ∗ , s, t ∈ ℝ}. On the other hand, we have that d ρ(A) = − dt , { { { t ρ(X) = −ie cos(θ + t), { { { −t {ρ(Y) = −ie sin(θ + t)

and

d ρℏ (A) = − dt , { { { −ℏt ρℏ (X) = −ie cos(θ + ℏt), { { { −ℏt {ρℏ (Y) = −ie sin(θ + ℏt).

8.3 The Zariski closure conjecture

| 439

We restrict our study to M = CK∞ (ℝ), where K is a compact set with a nonempty interior. With the same arguments as above, ℳ = M ⟦ν⟧ stands for a unitary weakly convergent module (of infinite radius). The representation πν of 𝒜 writes d , πν (A) = −i dt { { { iνt πν (X) = e cos(θ − iνt), { { { iνt {πν (Y) = e sin(θ − iνt)

and

d π0 (A) = −i dt , { { { π0 (X) = cos θ, { { { {π0 (Y) = sin θ.

It follows that the annihilator of πν is reduced to {0}, therefore, VA(πν ) = g∗ . The characteristic Poisson manifold coincides here also with the Zariski closure of the associated coadjoint orbit. To see this, it suffices to be convinced that the logarithmic spiral is Zariski dense in the 2-dimensional plan, remarking that every line passing through the origin intersects this spiral infinitely many times. Otherwise, the annihilator of π0 is the ideal generated by the two generators {X − cos θ, Y − sin θ}. The representation ρ acts on the space L2 (G/B), which is isomorphic to L2 (exp ℝA). Then we have V(πν ) = {l ∈ g∗ /l(X − cos θ) = 0 and l(Y − sin θ) = 0} = fθ + ℝA∗ = fθ + b⊥ .

8.3.9 The Zariski closure conjecture For any exponential solvable Lie algebra g and any coadjoint orbit Ω ⊂ g∗ , we constructed above a topological module (πν , ℳ) over the formal enveloping algebra: ̂ν (gℂ ) = T(g)⟦ν⟧/⟨x ⊗ y − y ⊗ x − ν[x, y]⟩, 𝒰 identified to a deformed algebra 𝒜 = (S(gℂ )⟦ν⟧, ∗) via a ℂ⟦ν⟧-module isomorphism (e. g., the symmetrization map or the Duflo isomorphism). This module is obtained by considering the differential of the associated irreducible unitary representation ρ of the group G = exp g. This representation is constructed by inducing a unitary character of a polarization h of ξ ∈ Ω, that is, a maximal isotropic Lie subalgebra of g with respect to the bilinear form Bℓ = ⟨ℓ, [−, −]⟩. Any polarization obviously contains the radical g(ℓ) of the bilinear form Bℓ . The polarization h satisfies Pukanszky’s condition (8.30). Namely, this construction gives rise for any real number ℏ to a unitary irreducible representation ρℏ of the group Gℏ = exp gℏ , which is irreducible for ℏ ≠ 0. Here, gℏ is the Lie algebra obtained from g by multiplying the Lie bracket by ℏ. Differentiating each of these representations and setting ν = iℏ, we get the requested topological module by considering ν as an indeterminate πν = iρ−iν (cf. [21]).

440 | 8 Deformation of topological modules Examples 8.3.7 and 8.3.8 show that Theorem 8.3.11 does not hold in the case of a nonnilpotent exponential solvable group. In these two examples, the characteristic Poisson manifold is equal to the Zariski closure of the orbit. More precisely, we state the following conjecture. Conjecture 8.3.10 (cf. [23]). Let G be an exponential solvable Lie group, with Lie algebra g, and let π be an irreducible unitary representation of G, associated to a coadjoint orbit (Ad∗ G)ℓ = Ω via the Kirillov orbit method. Then the Poisson characteristic variety VA(πν ) coincides with the Zariski closure in g∗ of the orbit Ω. 8.3.10 The nilpotent case In Examples 8.3.5 and 8.3.6, the characteristic Poisson variety coincides with the coadjoint orbit. This property can be generalized for all unitary irreducible representation of a nilpotent group. We have the following. Theorem 8.3.11. Under the same hypotheses, Conjecture 8.3.10 holds for nilpotent Lie groups. Proof. Let g be an n-dimensional nilpotent real Lie algebra. Let f ∈ g∗ and h a real polarization at f and let ρℏ be the unitary representation of the simply connected group Gℏ = exp gℏ given by the construction of Subsection 8.3.2. This representation is irreducible since h satisfies the Pukanszky condition. The coadjoint orbit Of ⊂ g∗ is an algebraic submanifold, given by the annulation of real valued polynomials (Qj )j=1,...,n−d where d designates the dimension of the orbit. Let τ be the Duflo isomorphism, which is here reduced to the symmetrization. Based on Theorem 2.3.2 of [118] (adapted here to our sign conventions in the definition of the character χf ), the annihilator of ρℏ in 𝒰 (gℏ ) is the ideal generated by the uj ’s with τ−1 (uj )(η) = Qj (iη). Hence, we see that the annihilator of the representation πν of the deformed algebra 𝒜 is generated by the Qj ’s. Thus, the ideal Ann πΩν /(Ann πΩν ∩ ν𝒜) of S(g) is also generated by the Qj ’s, and consequently the orbit Ω and the characteristic Poisson variety coincide. We prove Conjecture 8.3.10 in several other situations: in the case g = [g, g] + g(ℓ), the proof is similar to the one in the nilpotent case. The case when the chosen Pukanszky polarization is normal is also possible to handle. We also treat several interesting low-dimensional examples. We have not yet been able to prove the conjecture in the general case.

8.3.11 First approach to Conjecture 8.3.10 We keep the notation of the introduction. Let d be the dimension of the coadjoint orbit (Ad∗ G)ℓ. We choose a coexponential basis (X1 , . . . , Xd/2 ) of the polarization h in g. The

8.3 The Zariski closure conjecture

| 441

Hilbert space of the representation ρℏ is thus identified to L2 (ℝd/2 ) via the diffeomorphism Φ : ℝd/2

󳨀→

ℏt = (ℏt1 , . . . , ℏtd/2 )

󳨃󳨀→

Gℏ /Hℏ

exp t1 X1 ⋅ ⋅ ⋅ exp td/2 Xd/2 H.

Recall from [21] that I is the ideal of those polynomials Q0 such that πν (Q0 ) = O(ν), whereas J is the ideal of the polynomials Q0 such that there exists a sequence (Qj )j≥1 of polynomials such that Q = Q0 + νQ1 + ν2 Q2 + ⋅ ⋅ ⋅ + νk Qk + ⋅ ⋅ ⋅ ∈ Ker πν . In other words, I is the set of polynomials Q0 such that πν (Q0 ) is “small” (i. e., vanishes for ν = 0), and J is the set of polynomials Q0 , which can be deformed into an element of the annihilator. One clearly has the inclusion I ⊂ J. Conjecture 8.3.10 can be reformulated as follows. Conjecture 8.3.12. Any polynomial vanishing on the orbit can be deformed into an element of the annihilator of the associated topological module πν . Now let Q0 ∈ S(g) such that Q0 vanishes on the orbit Ω = (Ad∗ G)ℓ. Recall that we look for Q1 , Q2 , . . . ∈ S(g) such that πν (Q0 + νQ1 + ν2 Q2 + ⋅ ⋅ ⋅) = 0. We have Q0 ∈ Ker π0 , hence πν (Q0 ) = O(ν). Moreover, ad X ⋅ Q0 vanishes on Ω for any X ∈ g, hence πν (ad X ⋅ Q0 ) = O(ν), that is, [πν (X), πν (Q0 )] = νπν ([X, Q0 ]) = O(ν2 ). Applying this to X = X1 , . . . , Xd/2 and using πν (Xj ) = −𝜕j + O(ν) [21, Lemme 5.1.4], we end up with the fact that π̇ 0 (Q0 ) =

d 󵄨󵄨󵄨󵄨 󵄨 π (Q ) dν 󵄨󵄨󵄨ν=0 ν 0

is a partial differential operator with constant coefficients. Using the same lemma from [21] again, there exists an element Q1 ∈ S(g) such that π̇ 0 (Q0 ) = −π0 (Q1 ). Hence we get πν (Q0 + νQ1 ) = O(ν2 ),

(8.48)

thus making a first step toward deforming Q0 as we would like. Unfortunately, the next steps are much more harder, and we have been able to carry out the process only in the nilpotent case. Lemma 8.3.13. Let G be an exponential solvable Lie group with lie algebra g. Let Q0 ∈ JΩ . Then, the rth derivative π0(r) (Q0 ) is a partial differential operator with polynomial coefficients of degree at most r − 1.

442 | 8 Deformation of topological modules Proof. The result has been just proved for r = 1. We now detail the case r = 2. Let j, k ∈ {1, . . . , d/2}. Starting from the fact that ad Xj ad Xk Q0 vanishes on Ω, we get [πν (Xj ), [πν (Xk ), πν (Q0 )]] = O(ν3 ).

(8.49)

Vanishing of the coefficient of ν2 in (8.49) yields [π0 (Xj ), [π0 (Xk ), π̈ 0 (Q0 )]] + [π̇ 0 (Xj ), [π0 (Xk ), π̇ 0 (Q0 )]] + [π0 (Xj ), [π̇ 0 (Xk ), π̇ 0 (Q0 )]] = 0, hence [−𝜕j , [−𝜕k , π̈ 0 (Q0 )]] + [π̇ 0 (Xj ), [−𝜕k , π̇ 0 (Q0 )]] + [−𝜕j , [π̇ 0 (Xk ), π̇ 0 (Q0 )]] = 0.

(8.50)

The second term of the right-hand side vanishes because π̇ 0 (Q0 ) is a constant coefficient partial differential operator. The last term also vanishes because of the following computation: [−𝜕j , [π̇ 0 (Xk ), π̇ 0 (Q0 )]] = [[−𝜕j , π̇ 0 (Xk )], π̇ 0 (Q0 )] + [π̇ 0 (Xk ), [−𝜕j , π̇ 0 (Q0 )]]. The second term of this sum vanishes as π̇ 0 (Q0 ) is a constant coefficient partial differential operator. The first term also vanishes because π̇ 0 (Xk ) is a partial differential operator with coefficients of degree at most one: this comes from the formula d 2

d

ν 2 𝜕aX,u (νt) 𝜕 πν (Xk ) = ∑ aX,u (νt) + aX,0 (νt) + ∑ , 𝜕tu 2 u=1 𝜕tu u=1

(8.51)

where aX,0 , . . . , aX,d/2 are analytic functions on ℝd/2 (cf. [119]). Thus the first term of the sum (8.50) vanishes, which is equivalent to the fact that π̈ 0 (Q0 ) is a differential operator with affine coefficients. The case r ≥ 3 is treated similarly, by induction on r. From [πν (Xj1 ), [⋅ ⋅ ⋅ [πν (Xjr ), πν (Q0 )] ⋅ ⋅ ⋅]],

(8.52)

the vanishing of the coefficient of νr in (8.52) implies that [𝜕j1 , [⋅ ⋅ ⋅ [𝜕jr , π0(r) (Q0 )] ⋅ ⋅ ⋅]] = O(νr+1 ) is a finite sum of Lie brackets involving only the operators π0(u) (Q0 ), u = 1, . . . , r − 1 and π0(s) (Xj ), s = 1, . . . , r − 1, j = 1, . . . , d/2. The operators of the first family are partial differential operators with coefficients of degree at most u − 1 by the induction hypothesis; those of the second family are partial differential operators with coefficients of degree

8.3 The Zariski closure conjecture

| 443

at most s by (8.51). More precisely, we have (−1)r [𝜕j1 , [⋅ ⋅ ⋅ [𝜕jr , π0(r) (Q0 )] ⋅ ⋅ ⋅]] =−

∑

[π0 1 (X1 ), [⋅ ⋅ ⋅ [π0 r (Xr ), π0(u) (Q0 )] ⋅ ⋅ ⋅]]. (s )

(s )

u+s1 +⋅⋅⋅+sr =r, sj ≥0, 0 0 of axe H ∗ and radius r, defined by the equation ξ22 + ξ32 = r 2 . The Duflo isomorphism is once again given by τ=

sin(ν𝜕1 /2) ∘ σ, ν𝜕1 /2

where 𝜕1 designates the partial derivative operator

𝜕 𝜕ξ1

in g∗ , and σ designates the sym-

metrization. The algebra of invariants has a unique generator C : ξ 󳨃→ ξ22 + ξ32 , and the Duflo isomorphism still consists in applying the identity on this generator. The Casimir τ(C), still denoted by C, writes C = P 2 + Q2 .

We consider for r > 0, ℏ ≠ 0 and 0 ≤ λ < 1, the family ρr,λ;ℏ of the following unitary irreducible representations of Gℏ = exp gℏ : The representation ρr,λ;ℏ acts on the space ℋλ of the 2π-pseudo-periodic functions classes φ, satisfying φ(t + 2π) = e

2iπλ

2π

φ(t),

󵄨 󵄨2 ∫ 󵄨󵄨󵄨φ(t)󵄨󵄨󵄨 dt < +∞. 0

Its differential is defined by ρr,λ;ℏ (H) = −ℏ

d , dθ

8.4 Some nonexponential restrictive cases | 451

ρr,λ;ℏ (P) = ir sin(θ),

ρr,λ;ℏ (Q) = ir cos(θ). On the space ℋλ∞ of C ∞ -vectors, which is here the space of C ∞ - functions of ℋλ . These expressions are polynomials of variable ℏ. Considering ν = iℏ and applying the techniques above, one gets therefore a family of unitary representations of 𝒜 in the topologically free weakly convergent and strongly unitary module ℋλ∞ ⟦ν⟧: d , dθ πr,λ;ν (P) = −r sin(θ),

πr,λ;ν (H) = −ν

πr,λ;ν (Q) = −r cos(θ). Assuming that ν is null, we see that the annihilator Ann πr,λ;0 is generated by H and by P 2 + Q2 − r 2 . The characteristic variety V(πr,λ;ν ) turns out to be the circle in Ωr defined by the equations ξ22 + ξ32 = r 2 and ξ1 = 0. Then it is independent of the parameter λ. The annihilator Ann πr,λ;ν is generated in 𝒜 by C − r 2 . The ideal Ann πr,λ;ν /ν Ann πr,λ;ν is also generated by C−r 2 in S(g). The characteristic Poisson variety VA(πr,λ;ν ) is therefore equal to the coadjoint orbit Ωr . We consider now the space ℋλω of analytic vectors. These are the entire functions φ on ℝ such that φ(t + 2π) = e2iπλ φ(t). Let R be the opertaor on C ω (ℝ)⟦ν⟧ defined by Rφ(θ) = νφ(νθ). ̃ = R(ℋω ⟦ν⟧), and let π̃ This operator is injective. Let C λ r,λ;ν be the representation of 𝒜 λ ̃ on Cλ defined by π̃r,λ;ν (X) = R ∘ πr,λ;ν ∘ R−1 . The two representations are obviously equivalent, and we have d , dθ π̃r,λ;ν (P) = −r sin(νθ),

π̃r,λ;ν (H) = −

π̃r,λ;ν (Q) = −r cos(νθ). The annulator of π̃r,λ;0 is generated by P and Q + r. This time the characteristic variety V(π̃r,λ;ν ) is therefore the generator of the cylinder Ωr defined by the equations ξ3 = −r et ξ2 = 0.

452 | 8 Deformation of topological modules 8.4.2 The diamond groups It is the 4-dimensional real Lie algebra g with the basis (H, P, Q, E) and the brackets [H, P] = −Q,

[H, Q] = P,

[P, Q] = E,

the other brackets are null. W refer to M. Vergne in [42]. If X = aH + bP + cQ + dE, we easily compute the matrix of ad X in this basis 0 −c ad X = ( b 0

0 0 −a −c

0 a 0 b

0 0 ). 0 0

We also have 0 ab (ad X) = ( ac b2 + c2 2

0 −a2 0 −ab

0 0 −a2 −ac

0 0 ), 0 0

as well as the equality (ad X)3 = −a2 (ad X). We deduce the explicit expression exp(ad X) = I +

1 − cos a sin a ad X + (ad X)2 . a a2

After transposition and inversion, the matrix of exp(ad∗ X) in the dual basis reads 1 0 exp(ad∗ X) = ( 0 0

a c sina a + b 1−cos a cos a − sin a 0

a −b sina a + c 1−cos a sin a cos a 0

a (b2 + c2 ) 1−cos a2 sin a 1−cos a c a −b a a) . −b sina a − c 1−cos a 1

The coadjoint action of exp X on an element ξ = μH ∗ + βP ∗ + γQ∗ + λE ∗ writes sin a 1 − cos a sin a 1 − cos a +b )β + (−b +c )γ a a a a 1 − cos a + (b2 + c2 ) λ)H ∗ a2 sin a 1 − cos a + ((cos a)β + (sin a)γ + (c −b )λ)P ∗ a a

Ad∗ (exp X).ξ = (μ + (c

8.4 Some nonexponential restrictive cases | 453

+ ((− sin a)β + (cos a)γ + (−b + λE ∗ .

1 − cos a sin a −c )λ)Q∗ a a

Let ξi denote the ith component of Ad∗ (exp X).ξ . The last component ξ4 is invariant under the coadjoint action. Two cases are to be considered here: – First case: ξ4 = λ = 0. The expression ξ22 + ξ32 = β2 + γ 2 is invariant on the subspace defined by ξ4 = 0. Then we see that the corresponding coadjoint orbits are the cylinders of axe H ∗ and every point of this axe, depending on whether ξ22 + ξ32 is positive or null. They are exactly the coadjoint orbits of the quotient of γ by its center (the Lie algebra of the 2-dimensional motion group): we treat this example later in Subsection 8.4.1. – Second case: ξ4 = λ ≠ 0. Making exp Y act on ξ with Y = bP + cQ, we obtain Ad∗ (exp Y).ξ = (μ + cβ + bγ + (b2 + c2 )λ)H ∗ + (β + cλ)P ∗

+ (γ − bλ)Q∗ + λE ∗ .

By an adequate choice of b and c, it brings us back to the case where β = γ = 0, which we will assume. The explicit formula giving Ad∗ (exp X).ξ can be simplified: 1 − cos a λ)H ∗ a2 sin a 1 − cos a + (c −b )λP ∗ a a sin a 1 − cos a + (−b −c )λQ∗ a a

Ad∗ (exp X).ξ = (μ + (b2 + c2 )

+ λE ∗ . We notice that we have ξ1 −

ξ2 2 + ξ3 2 = μ and 2ξ4

ξ4 = λ.

(8.70)

The coadjoint orbits in this case are the paraboloids of revolution Ωλ,μ and axe H ∗ given by the equations (8.70). Let φ be the entire function defined by φ(z) =

sh z/2 . z/2

454 | 8 Deformation of topological modules It is clear that the matrix of φ1/2 (ad X) is of the form: 1 ∗ ( ∗ ∗

0 φ1/2 (ia) ∗ ∗

0 0 φ1/2 (ia) ∗

0 0 ), 0 1

which gives, with the notation of Subsection 8.2.1 that J(X)1/2 =

sin(νa/2) . νa/2

Hence, the Duflo isomorphism is given by τ=

sin(ν𝜕1 /2) ∘ σ, ν𝜕1 /2

where 𝜕1 designates the partial derivative operator 𝜕ξ𝜕 in g∗ , and σ designates the 1 symmetrization. Taking into account the equations (8.70), we easily see that the algebra S(g)g of invariant polynomials on g∗ is generated by C1 and C2 , where C1 (ξ ) = ξ4 and C2 (ξ ) = ξ22 + ξ32 − 2ξ1 ξ4 . The coadjoint orbit Ωλ,μ defined by (8.70) is given similarly by the equations: C1 (ξ ) = λ,

C2 (ξ ) = −2λμ.

(8.71)

The operator J(D)1/2 acts on these two generators by the identity, thus the Duflo isomorphism τ goes back to apply the symmetrization on these generators. We abusively denote C1 and C2 , the two Casimirs τ(C1 ) and τ(C2 ). We have explicitly C1 = E,

C2 = P 2 + Q2 − 2EH.

We use now the notation of Section 8.2. For all real number ℏ, there exists a family of unitary irreducible representations ρλ,μ;ℏ de Gℏ = exp gℏ , whose differentials are defined on the Schwartz space 𝒮 (ℝ) (provided with a scalar product of L2 (ℝ)) by ρλ,μ;ℏ (H) = −i(− d , dx ρλ,μ;ℏ (Q) = iλℏx,

1 d2 1 + λℏ2 x 2 + μ), 2λ dx2 2

ρλ,μ;ℏ (P) = −

ρλ,μ;ℏ (E) = −iλ. These representations are unitarily equivalent to those obtained by holomorphic induction from the point μH ∗ + λE ∗ and the complex polarization h generated by

8.4 Some nonexponential restrictive cases | 455

H, E, P + iQ [42, Sections V.4 and VIII.1.4.4]. The expressions here are polynomials of the variable ℏ. Considering ν = iℏ and applying the techniques above, we get a unitary representation πλ,μ;ν of the deformed algebra 𝒜 in the topologically free, weakly convergent and strongly unitary module 𝒮 (ℝ)⟦ν⟧: 1 1 d2 − λν2 x 2 + μ, 2λ dx 2 2 d πλ,μ;ν (P) = −i , dx πλ,μ;ν (Q) = iλνx,

πλ,μ;ν (H) = −

πλ,μ;ν (E) = λ. Admitting that the parameter ν is null, we see that the annihilators Ann πλ,μ;0 are the ideal of S(g) generated by par E − λ, Q and C2 + 2λμ. We deduce that the characteristic variety V(πλ,μ;ν ) is defined by the equations: ξ22 + ξ32 − 2ξ1 ξ4 = −2λμ,

ξ4 = λ,

ξ3 = 0.

The characteristic variety V(πλ,μ;ν ) generates therefore the paraboloid of revolution Ωλ,μ . The nonexistence of real polarizations can be seen by the fact that the characteristic variety is not an affine subspace (cf. Subsection 8.3.2). The annihilator Ann πλ,μ;ν is generated in 𝒜 by C1 − λ and C2 + 2λμ. The ideal Ann πλ,μ;ν /ν Ann πλ,μ;ν is therefore generated in S(g) by C1 − λ and C2 + 2λμ. We deduce that the characteristic Poisson manifold VA(πλ,μ;ν ) coincides with the coadjoint orbit Ωλ,μ .

8.4.3 The semisimple case: Verma modules Let g1 be a complex semisimple Lie algebra. The Killing form is defined by (X, Y) = Tr(ad X ∘ ad Y) is nondegenerate and invariant under the adjoint action. Thus, it induces a linear isomorphism κ from g1 to its dual g∗1 , defined by κ(X) = (X, −), which intertwines the adjoint and the coadjoint representations. Let H be a semisimple element of g1 , h1 be a Cartan subalgebra containing H, Δ the associated root system and let Δ+ designate the set of positive roots coming from the choice of an order h∗1 . Let also W denote the Weyl group associated in this context. The Killing form restricted

456 | 8 Deformation of topological modules to h1 is nondegenerate. Let λ = κ−1 (H) ∈ g∗1 . The radical decomposition of g1 writes g1 = n1− ⊕ h1 ⊕ n1+ , where n1± = ⨁α∈±Δ+ gα1 . Let δ ∈ h∗1 be the half-sum of positive roots. The Cartan subalgebra h1 is orthogonal to n1+ and n1− with respect to the Killing form, which allows us to prove that κ −1 (H) ∈ g∗1 vanishes on n1± . Thus, we will identify h∗1 to the orthogonal of n1− ⊕ n1+ in g∗1 by taking the trivial prolongation to n1− ⊕ n1+ of the linear forms on h1 . If H is regular, then the stabilizer of λ under the coadjoint action equals to h1 , and the solvable subalgebra b1 = h1 ⊕ n1+ is a solvable polarization on λ. The context above applies to the Lie algebras (gν )ν∈ℂ−{0} (with the notation of Section 8.2), that have all the same underlying vector space, which will be denoted by g. We will always identify g and its dual thanks to the Killing form of g = g1 (independently of ν), not with the Killing form of gν . The radical decomposition gν = nν− ⊕ hν ⊕ nν+ is independent from ν in what concerns the underlying vector spaces, which will be denoted by n− , h and n+ . Similarly, we denote b+ = h ⊕ n+ and gα = {X ∈ g, ∀A ∈ h, [A, X]ν = να(A)X}. The root system associated gν is νΔ, and gα coincides for all ν with the radical subspace of gν corresponding to να. Let Mλν be the Verma module associated to λ for gν . We have Mλν = 𝒰 (gν ) ⨂ ℂ, 𝒰 (bν+ )

where an element A + Y of bν+ = hν ⊕ nν+ acts on ℂ via multiplication by (λ − νδ)(A), and where the action of 𝒰 (gν ) is given by left multiplication. When we see ν as an indeterminate, this module is topologically free on 𝒜 = 𝒰ν (g). In fact, we can identify it to 𝒰ν (n− ), and to S(n− )⟦ν⟧ via symmetrization. We have the decomposition on weight subspaces under the action of hν : Mλν = ⨁ (Mλν )λ−νδ−νβ , β∈Q+

where Q+ designates the set of linear combinations with positive integer coefficients of elements of Δ+ . Supposing ν = 0, we see that all A ∈ h0 acts on Mλ0 by multiplication by λ(A). We see also that action de n0+ is trivial on Mλ0 , and the action de n0− is faithful. The annulator of Mλ0 in S(g) is therefore the ideal generated by n+ and by A−λ(A), A ∈ h. We deduce V(Mλν ) = λ + b⊥ +.

8.4 Some nonexponential restrictive cases | 457

Let χλν : Z(𝒰 (gν )) → ℂ be the central character of Mλν . Let also γν : Z(𝒰 (gν ))̃ 󳨀→S(h)W W be the Harish–Chandra isomorphism of gν (cf. [63]). Considering S(h) as the set of W-invariant polynomials on h∗ , we get (cf. [63]): χλν (u) = γν (u)(λ). The Duflo isomorphism τν : S(g)gν → Z(𝒰 (gν )) is obtained by left composition of γν−1 by the restriction on h∗ (Chevalley isomorphism). For all v ∈ S(g)gν , we get finally χλν (τν (v)) = v(λ). The annihilator of the Verma module Mλν is the bilateral ideal of 𝒰 (gν ) generated by Ker χλν [63, Theorem 8.4.3]. Then it is the bilateral ideal of (S(g), ∗ν ) generated by {v − v(λ), v ∈ S(g)gν } via the Duflo isomorphism. Consider now ν as an indeterminate and Mλν as topologically free on 𝒜. The ideal Ann Mλν /(Ann Mλν ∩ ν𝒜) of S(g) is therefore the ideal generated by {v − v(λ), v ∈ S(g)gν }. We deduce the characteristic Poisson variety: VA(Mλν ) = {ξ ∈ g∗ , v(ξ ) = v(λ) for any v ∈ S(g)gν }. When λ (which means H) is regular, this turns out to be the coadjoint orbit of λ. When λ = 0, this is precisely the nilpotent cone. Verma modules and real forms The Verma module Mλν has a natural symmetric bilinear form with good covariance properties, the Shapovalov form (cf. [65, 124]). To transform it in an Hermitian sesquilinear form, we can apply the results of Section 8.1.6. We keep the notation of Subsection 8.4.3. Let (X−α , Hα , Xα )α∈Δ+ be a Chevalley basis of g1 . Then {−iX−α , −iHα , −iXα }α∈Δ+ is a Chevalley basis of gi (where i = √−1). Let gi,ℝ be the associated deployed real form of gi . It is defined as the real vector space generated by −iX−α , −iHα , −iXα . This defines a real form gℝ of the underlying vector space g, which is automatically a deployed real form gν,ℝ of the Lie algebra gν for all imaginary number ν. The conjugation respects the radical subspaces, then necessarily the three decomposition components gν = nν− ⊕ hν ⊕ nν+ . The conjugation on g (resp., on h) with respect to this real form induces a conjugation on the dual g∗ (resp., h∗ ) thanks to the formula ξ (X) := ξ (X).

458 | 8 Deformation of topological modules We extend the conjugation by multiplicativity to the enveloping algebra 𝒰 (gν ). This enveloping algebra has (thanks to the Poincaré–Birkhof-f-Witt theorem) the decomposition 𝒰 (gν ) = 𝒰 (hν ) ⊕ (nν− 𝒰 (gν ) + 𝒰 (gν )nν+ ).

Let Pν : 𝒰 (gν ) → 𝒰 (hν ) be the projection corresponding to this decomposition. From the above, this projection is compatible with the conjugation, which means that P(u) = P(u)

for any u ∈ 𝒰 (gν ).

We define the transposition on 𝒰 (gν ) as follows: t

Xα = X−α ,

t

X−α = Xα ,

t

Hα = Hα ,

and we extend this transposition to an antiautomorphism of 𝒰 (gν ). We see immediately that the transposition commutes with the conjugation. We consider on 𝒰 (gν ) the involution (semilinear) defined by a∗ = t a. This involution (restricted to g) is a new conjugation on g and, therefore, a new real form gℝ , which is a compact real form of the Lie algebra gℏ where ℏ = −iν is a nonzero real number (cf. [81, Section III.6]). We notice that h ∩ gℝ = ih ∩ gℝ . Denote by hℝ this intersection and denote by τ the conjugation (in g or h or in their duals, resp.) with respect to this new real form. Suppose then that λ ∈ h∗ is real, which means that τ(λ) = λ. The involution ∗ on 𝒰 (gν ) coincides, via the Duflo isomorphism, with the involution given in Subsection 8.1.6, the conjugation being τ. Consider the Verma module Mνλ+νδ (instead of considering Mνλ as in Subsection 8.4.3). Consider also ν as an indeterminate. As modules on the deformed algebra 𝒜, they coincide up to O(ν) and, therefore, their characteristic and Poisson characterν ν . istic varieties coincide. Let eλ+νδ be the highest weight vector of Mλ+νδ Suppose now that λ is a real number. Define then a sesquilinear form on the Verma ν by the formula module Mλ+νδ ν ν ⟨m, n⟩ν = ⟨a.eλ+νδ , b.eλ+νδ ⟩ν = Pν (a∗ b)(λ). ν This is an Hermitian version of the Shapovalov form of Mλ+νδ .

8.5 A deformation approach of the Kirillov map |

459

ν Proposition 8.4.1. The representation πν of 𝒜 = 𝒰 (gν ) in Mλ+νδ satisfies for all m, n ∈ ν Mλ and all u ∈ 𝒰 (gν ). We have the following:

⟨um, n⟩ν = ⟨m, u∗ n⟩ν . Proof. Let a, b ∈ 𝒰 (gν ) verify m = a.eλν and n = b.eλν . The proposition is an immediate consequence of the equality ν ν ν ν ⟨ua.eλ+νδ , b.eλ+νδ ⟩ = ⟨a.eλ+νδ , u∗ b.eλ+νδ ⟩.

The associated representation is therefore a ∗-representation. Moreover, the quotient form defined par ⟨−, −⟩ν in ν = 0 is Hermitian nondegenerate. In other words, ν the Verma module Mλ+νδ is strongly pseudo-unitary. Hence the characteristic varieties V(πν ) and VA(πν ) are defined on the real field, and they are given by the same explicit formulas as in Subsection 8.4.3. Remark 8.4.2. Let λ ∈ h∗ . Using Theorem 7.6.24 of [65], it is easy to see that the Verma ν module Mλ+νδ is simple (consequently, the Shapovalov form is nondegenerate) except the possibly where the set of ν’s is countable discrete. Otherwise, on the question of unitarizability of some modules of highest weight, see [68] and [85].

8.5 A deformation approach of the Kirillov map We now outline an approach of the Kirillov map for exponential groups based on deformations, and suggest a possible way toward an alternative proof of the Leptin– Ludwig bicontinuity theorem [106] along these lines. 8.5.1 Poisson ideals General setting Let A be a commutative C ∗ -algebra, which identifies itself via the Gel’fand transform ̂ of continuous functions on the spectrum of A, which vanish at with the algebra C0 (A) ̂ via F 󳨃→ infinity. The closed ideals of A are in bijection with the closed subsets of A IF = {a ∈ A, a|F = 0}. ̂ of A is endowed with a Poisson manifold Suppose now that the spectrum X = A ∞ structure, that is, a C manifold structure together with a Lie bracket {−, −} on C ∞ (X) (the Poisson bracket), which is a derivation with respect to both arguments (Leibniz rule). A Poisson ideal of A is a closed ideal J = IF such that the set F of its common zeroes is invariant under the flows of Hamiltonian vector fields. For any f ∈ J ∩ C ∞ (X) and any g ∈ C ∞ (X)c , {f , g} ∈ J, but this property does not characterize Poisson ideals. A Poisson ideal J = IF is primitive if F is the closure of a symplectic leaf in X. Let us denote by Prim𝒫 A the set of primitive Poisson ideals of A.

460 | 8 Deformation of topological modules Proposition 8.5.1. Any maximal ideal J of A contains a unique Poisson ideal p(J), which is maximal for the inclusion in J, and it is primitive in the above sense. The map p : X = ̂ → Prim𝒫 A thus defined is surjective. A Proof. Let J be a maximal ideal of A, which corresponds to a point ξ ∈ X. Let Sξ be the symplectic leaf through ξ , and let p(J) be the ideal of elements of A, which vanish on Sξ . The primitive Poisson ideal thus obtained is clearly maximal among the Poisson ideals contained in J. Uniqueness is clear, as the sum of two Poisson ideals is a Poisson ideal. Surjectivity of p follows from the fact that any point of X is contained in a symplectic leaf. Set now a “Jacobson-like” topology on Prim𝒫 A: for a subset T of Prim𝒫 A one defines T by T = {J ∈ Prim𝒫 A, I(T) ⊂ J}, where the Poisson ideal I(T) ie defined by I(0) = A and I(T) = ⋂ I I∈T

for T ≠ 0.

Proposition 8.5.2. There exists a unique topology on Prim𝒫 A such that T 󳨃→ T is the closure map for this topology, which makes Prim𝒫 A a T0 -space. Proof. We clearly have 0 = 0, T ⊂ T and T = T. We define the closed subsets as those writing T, T ⊂ Prim𝒫 A. Let (Tj )j∈Λ be a collection of closed subsets. We have then ⋂ Tj = {I ∈ Prim𝒫 A, I(Tj ) ⊂ I for any j ∈ Λ}

j∈Λ

= {I ∈ Prim𝒫 A, ∑ I(Tj ) ⊂ I} j∈Λ

whereas ⋂ Tj = {I ∈ Prim𝒫 A,

j∈Λ

⋂

K∈⋂j∈Λ Tj

K ⊂ I}.

But the inclusion I(Tj ) = ⋂K∈Tj K ⊂ ⋂K∈⋂r∈Λ Tr K, verified for any j ∈ Λ, yields the

inclusion ∑j∈Λ I(Tj ) ⊂ ⋂K∈⋂j∈Λ Tj K. Hence ⋂j∈Λ Tj ⊂ ⋂j∈Λ Tj , which shows that the intersection ⋂j∈Λ Tj is closed. In order to show that the union of two closed subsets is closed, we need the following lemma. Lemma 8.5.3. Let I1 and I2 be two Poisson ideals. Then a primitive Poisson ideal, which contains I1 ∩ I2 contains I1 or I2 .

8.5 A deformation approach of the Kirillov map |

461

Proof. Let VI1 (resp., VI2 ) be the set of common zeroes of the elements of I1 (resp., I2 ). These are two closed sets of X, which are invariant under Hamiltonian flows. We have I1 I2 ⊂ I1 ∩ I2 , hence VI1 ∩I2 ⊂ VI1 I2 , and VI1 ∪ VI2 ⊂ VI1 ∩I2 . On the other hand, VI1 I2 = {ξ ∈ X, ∀(a, b) ∈ I1 × I2 , a(ξ )b(ξ ) = 0} = VI1 ∪ VI2 .

We have then VI1 ∩I2 ⊂ VI1 I2 = VI1 ∪ VI2 ⊂ VI1 ∩I2 , which finally yields VI1 ∩I2 = VI1 I2 = VI1 ∪ VI2 .

(8.72)

Let I be a primitive Poisson ideal containing I1 ∩ I2 . Its set of common zeroes VI is the closure of a symplectic leaf contained in VI1 ∪VI2 , hence contained in VI1 or VI2 , whence the result. End of proof of the proposition. Let T1 and T2 be two closed subsets of Prim𝒫 A, let I1 = I(T1 ) and I2 = I(T2 ). Then T1 ∪ T2 = {J ∈ Prim𝒫 A, I(T1 ∪ T2 ) ⊂ J} = {J ∈ Prim𝒫 A, I1 ∩ I2 ⊂ J}

= {J ∈ Prim𝒫 A, I1 ⊂ J or I2 ⊂ J}

(according to Lemma 8.5.3)

= T1 ∪ T2 .

Finally, Prim𝒫 A is a T0 -space: in fact (see [62, Paragraph 3.1.3] for a similar argument), if I1 and I2 are two distinct elements of Prim𝒫 A, we have for instance I1 ⊄ I2 . Let then F be the closed subset of the elements I ∈ Prim𝒫 A, which contain I1 . We have I1 ∈ F and I2 ∉ F, hence the complement of F inside Prim𝒫 A is a neighborhood of I2 , which does not contain I1 . ̂ defined by Consider the equivalence relation on X = A I ∼ J ⇔ p(I) = p(J). Two points of X are equivalent if and only if the closures of their symplectic leaves ̂ ∼ with the quotient topology. coincide. We endow A/ ̂ ∼󳨀→ Prim𝒫 A derived from p is an homeomorphism. ̃ : A/ Proposition 8.5.4. The map p ̃ is clearly bijective. Let us prove continuity first: Let T be a closed Proof. The map p subset of Prim𝒫 A, let I(T) = ⋂J∈T J be the associated Poisson ideal. We have then ̂ I(T) ⊂ p(K)} p−1 (T) = {K ∈ A, ̂ I(T) ⊂ K}. = {K ∈ A,

462 | 8 Deformation of topological modules ̂ Hence the map Hence p−1 (T) is the set of common zeroes of I(T), which is closed in A. ̃ as well by definition of the quotient topology. Let us now prove p is continuous, and p ̃ be a closed subset of A/∼, ̂ ̂ be the inverse image of ̃ is closed: Let U that p and let U ⊂ A ̃ ̂ J(U) ⊂ I}, and U by the canonical projection. Let J(U) = ⋂K∈U K. Then U = {I ∈ A, ̃ = p(U) = {p(I), I ∈ A, ̂ J(U) ⊂ I}. ̃ (U) p But we have J(U) = ⋂ K K∈U

= ⋂ ( ⋂ K) ̃ p(U) ̃ p(K)=K ̃ K∈̃

̃ = ⋂ K ̃ p(U) ̃ K∈̃

̃ ̃ (U)). = I(p We have then ̃ = {p(I), I ∈ A, ̂ J(U) ⊂ I} ̃ (U) p ̂ I(p ̃ ⊂ I} ̃ (U)) = {p(I), I ∈ A, ̂ I(p ̃ ⊂ p(I)} ̃ (U)) = {p(I), I ∈ A, ̃ ⊂ J} ̃ (U)) = {J ∈ Prim𝒫 A, I(p ̃ ̃ (U), =p ̃ is closed. Hence the map p ̃ (U) ̃ is bicontinuous. which proves that p

8.6 Type-I-ness and consequences O. Takenouchi proved in 1957 that exponential groups are type I groups [125]. It means that any unitary representation π of G is quasi-equivalent to a representation ρ, which is multiplicity-free, that is, such that the commutant ρ(G)′ is commutative. This has nice consequences on both domain and range of the Kirillov map.

8.6.1 Linear Poisson manifolds and primitive Poisson ideals Recall that a linear Poisson manifold is nothing but the dual g∗ of a Lie algebra g with the Kirillov–Kostant–Souriau bracket (8.26) The symplectic leaves coincide with the coadjoint orbits under the action of a connected Lie group G with Lie algebra g [131].

8.6 Type-I-ness and consequences | 463

Theorem 8.6.1. Let X = g∗ be a linear Poisson manifold, and let A = C0 (X). Suppose that g is solvable and that the associated connected simply connected Lie group G is of type I. Then Prim𝒫 A is homeomorphic to the space of coadjoint orbits g∗ /G endowed with the quotient topology. Proof. It is enough to show, according to Proposition 8.5.4, that the equivalence ̂ = g∗ . Two points ξ and η of g∗ verify classes for ∼ are the coadjoint orbits in A ∗ ∗ ξ ∼ η if and only if Ad G.ξ = Ad G.η. But G is supposed to be solvable and of type one; hence each coadjoint orbit is locally closed, that is, open in its closure [10]. Hence ξ ∼ η if and only if Ad∗ G.ξ = Ad∗ G.η. 8.6.2 The unitary dual and the orbit space (1) Any strongly continuous unitary representation of a type I group G is uniquely dê is then homeomortermined, up to isomorphism, by its kernel: the unitary dual G ∗ phic to the primitive spectrum Prim C (G) endowed with the Jacobson topology defined as follows: the closure of any subset T is given by T := {J ∈ Prim C ∗ (G), ⋂ K ⊂ J}. K∈T

(8.73)

This topology is T0 : for any J, K ∈ Prim C ∗ (G) there is a neighborhood of J or K, which does not contain the other element (see [62]). (2) According to Theorem 8.6.1, the orbit space g∗ /G of an exponential group is homeomorphic to the primitive Poisson ideal space Prim𝒫 C ∗ (g) endowed with the Jacobson-like topology defined in paragraph 8.5.1. ̂ have very similar structure when G is exponential. As a consequence, g∗ /G and G 8.6.3 The Tangent groupoid and Rieffel’s strict quantization Let G be an exponential Lie group. The idea consists in “interpolating” between g∗ /G ̂ more precisely between Prim C ∗ (G) and Prim C ∗ (g). In order to do this, we inand G, 𝒫 terpolate between G and its Lie algebra g by means of the continuous family (Gt )t∈[0,1] , where Gt has Lie algebra gt = (g, t[−, −]). The tangent groupoid of G is the union of these groups. More precisely, it is given by 𝒢 = g × [0, 1]

(8.74)

464 | 8 Deformation of topological modules with the law (X, t)(Y, t) = (X . Y, t),

(8.75)

t

where X . Y = t

Hausdorff series:

1 t

log(exp tX. exp tY) is given near (0, 0) by the Baker–Campbell–

t t2 X . Y = X + Y + [X, Y] + ([X, [X, Y]] + [Y, [Y, X]]) + ⋅ ⋅ ⋅ . t 2 12

(8.76)

The groupoid C ∗ -algebra 𝒜 := C ∗ (𝒢 ) is the C ∗ -algebra associated with the continuous field of C ∗ -algebras 𝒜t := C ∗ (Gt ). In particular, we have a surjective evaluation morphism Et : 𝒜 󳨀→ 𝒜t a 󳨃󳨀→ at

for any t ∈ [0, 1]. The 𝒜t ’s are all isomorphic except for t = 0: 𝒜0 is commutative and isomorphic to C0 (g∗ ) via the Fourier–Gel’fand transformation. Denote by ∗ the convot

lution on Cc∞ (Gt ) ∼ Cc∞ (g), and by # the corresponding Fourier transform convolution t ̃ := 𝒞 1 ([0, 1], C ∞ (g)) is a dense subalgebra of 𝒜, on 𝒜𝒮 (g∗ ) = ℱ (Cc∞ (g)). The algebra 𝒜 c ̃ isomorphic via Fourier transform to ℬ := 𝒞 1 ([0, 1], 𝒜𝒮 (g∗ )). Theorem 8.6.2 (M. A. Rieffel, N. P. Landsman, B. Ramazan). For any f , g ∈ ℬ̃, the following holds in C0 (g∗ ): i lim (f (t) # g(t) − g(t) # f (t)) = {f (0), g(0)}. t t t

t→0

(8.77)

Here, {−, −} is the usual Kirillov–Kostant–Poisson bracket on g∗ . 8.6.4 One-parameter families of representations Let ξ ∈ g∗ and let h be a Pukánszky polarization at ξ . For any t ∈ [0, 1], consider as above the unitary induced representation: ξ ,h

πt

̃ Gt χ . = Ind Ht ξ

(8.78)

The coadjoint orbits of ξ under the action of Gt are all the same 𝒪 except for t = 0. The ξ ,h representation πt is a model for the image of 𝒪 by the Kirillov map of Gt when t ≠ 0, but is not irreducible in general for t = 0.

8.6 Type-I-ness and consequences | 465

ξ ,h

Theorem 8.6.3. The family (πt )t∈]0,1] is a continuous field of representations, that is, ξ ,h |||πt (at )||| is continuous on ]0, 1] for any a 𝒪

the map t 󳨃→ at t = 0 to a limit L (a).

∈ 𝒜, and extends by continuity

ξ ,h

Proof. The representations πt admit a common realization in L2 (ℝd/2 ). The derivatives are given for any X ∈ g by ξ ,h

d/2

dπt (X)f (y) = ∑ aX,u (ty) u=1

𝜕f (y) 𝜕yu d/2

𝜕aX,u t − iaX,0 (ty)f (y) + ( ∑ (ty))f (y). 2 u=1 𝜕yu

(8.79)

Here, the aX,u ’s are analytic [7, 21, 119]. The expression above is hence analytic w. r. t. ξ ,h

the parameter t, and the functions t 󳨃→ dπt (X) are strongly continuous in the generalized sense [88]. Exponentiating we get that for any X ∈ g the operator-valued function ξ ,h ξ ,h t 󳨃→ πt (exp X) is strongly continuous. Strong continuity of t 󳨃→ πt (at ) follows from integration and Lebesgue’s dominated convergence theorem. ξ ,h Now consider for t ∈ ]0, 1] the ideal It𝒪 := Ker πt ∈ Prim 𝒜t , with Ad∗ Gt .ξ = 𝒪, and the following closed ideal in the C ∗ -algebra 𝒜: J 𝒪 := ⋂ Jt𝒪 , t∈]0,1]

(8.80)

where Jt𝒪 = Et−1 (It𝒪 ) ∈ Prim 𝒜. The quotient 𝒜/J 𝒪 is then a continuous field of C ∗ algebras (with (𝒜/J 𝒪 )t = 𝒜t /Jt𝒪 ∼ πt (𝒜t )) (cf. [45]), whence the result. The value at t = 0 is the norm of the class of a0 in the quotient 𝒜0 /E0 (J 𝒪 ). We try to compute it by means of the semicharacter formula [66, 117], which generalizes the Kirillov character formula for simply connected nilpotent Lie groups: let us formulate it directly in our “variable” context. For any t ∈ ]0, 1], there are a finite number of Ad∗ Gt -invariant subsets Ωj ⊂ g∗ , j = 1, . . . , k (independent of t), a family (χjt )j=1,...,k of characters of Gt with positive real values, a family (Qj )j=1,...,k of polynomials on g∗ (independent of t) and a family (αjt )j=1,...,k of positive Ad∗ Gt -invariant analytic functions on g such that Ωj = {ξ ∈ g∗ , Qj (ξ ) ≠ 0 and Qr (ξ ) = 0 for r < j},

(8.81)

the dual g∗ is the disjoint union of the Ωj ’s, and for any φ ∈ Cc∞ (Gt ) and ξ ∈ Ωj we have (with 𝒪 = Ad∗ G1 .ξ and dim 𝒪 = d): ξ ,h t φ(η) Q (η)dβ (η). Tr πt (utj ∗ φ) = t −d/2 ∫ α̂ j 𝒪 j t

𝒪

(8.82)

466 | 8 Deformation of topological modules ξ ,h

Here, utj is the symmetrization of Qj in 𝒰 (gt ) and dπt (uj ) is positive self-adjoint χjt -semiinvariant. The measure Qj dβ𝒪 is tempered.1 In view of the semicharacter for̃: mula, we introduce “twisted Schatten seminorms” on 𝒜 d

ξ ,h

ξ ,h

ξ ,h

p

1

𝒪,t S2p (a) := (t 2 Tr dπt (utj )(πt (at )∗ πt (at )) ) 2p .

(8.83)

Theorem 8.6.4. We have the following diagram, which suggests a good candidate for the limit: 𝒪,t S2p (a)

? 󵄨󵄨󵄨󵄨󵄨󵄨󵄨󵄨󵄨π ξ ,h (at )󵄨󵄨󵄨󵄨󵄨󵄨󵄨󵄨󵄨 󵄨󵄨󵄨 t 󵄨󵄨󵄨

p→+∞

t→0

? L𝒪 (a)

t→0

?

???

(∫ |â0 |2p (η) Qj dβ𝒪 (η))

1 2p

? supη∈𝒪 󵄨󵄨󵄨â0 (η)󵄨󵄨󵄨 󵄨 󵄨

p→+∞

𝒪

Proof. The right arrow is a direct consequence of Theorem 8.6.3, the left arrow comes from the semicharacter formula and the properties of the # product and the bottom arrow is a standard property of Lp norms. The upper arrow will show up as a consequence of the following lemma. Lemma 8.6.5. Let T be a bounded operator on a Hilbert space ℋ and 𝒟 a dense subspace of ℋ containing the image of ℋ by T and by its adjoint T ∗ . Let A be a nonnegative self-adjoint operator, which can be unbounded, defined on a domain containing 𝒟, and such that A|𝒟 is an isomorphism from 𝒟 onto 𝒟. Suppose moreover that A(T ∗ T)p A and A2 (T ∗ T)p are trace-class operators for any integer p ≥ 1, and that the image of A(T ∗ T)p A is contained in 𝒟 for any p ≥ 1. Then: (1) The following equality holds: p p p󵄨 󵄨 Tr A2 (T ∗ T) = Tr A(T ∗ T) A = Tr󵄨󵄨󵄨A2 (T ∗ T) 󵄨󵄨󵄨,

(8.84)

where the notation |U| stands for the operator (U ∗ U)1/2 . (2) We have p

1

|||T||| = lim (Tr A(T ∗ T) A) 2p . p→∞

1 Notice the t −d/2 factor, due to the normalization of the Liouville measure.

(8.85)

8.6 Type-I-ness and consequences | 467

Proof. Let us denote by V the nonnegative self-adjoint trace-class operator A(T ∗ T)p A (with a fixed integer p ≥ 1). Let (λi ) be the sequence of positive eigenvalues of V (without multiplicities) decreasingly ordered. Consider the decomposition: (Ker V)⊥ = ⨁ Eλi ,

(8.86)

i

where Eλi is the eigenspace of V of eigenvalue λi (the direct sum is orthogonal). Every Eλi is finite-dimensional, V being a compact operator. The inclusion Eλi ⊂ 𝒟 holds thanks to the hypotheses of the lemma. The subspace Fλi := AEλi then contains only eigenvectors of U := A2 (T ∗ T)p = AVA−1 for the eigenvalue λi . Reciprocally, if μ is a nonzero eigenvalue of U, the corresponding eigenspace Fμ′ is in 𝒟 (thanks again to the hypotheses of the lemma), and Eμ′ := A−1 Fμ′ consists of eigenvectors of V for the eigenvalue μ. In particular, μ is one of the eigenvalues λi and we have Eμ′ = Eλi , Fμ′ = Fλi . The nonzero eigenvalues of U and of V are then positive, equal and occur with the same multiplicity, which proves the first assertion of the lemma. The second assertion will be proved with the help of the first one: on the one hand, we have p

1

1

p

(Tr A(T ∗ T) A) 2p = (Tr A2 (T ∗ T) ) 2p

1

p−1 󵄨󵄨󵄨 󵄨󵄨󵄨 ≤ (Tr(A2 T ∗ T)󵄨󵄨󵄨󵄨󵄨󵄨󵄨󵄨󵄨(T ∗ T) 󵄨󵄨󵄨󵄨󵄨󵄨󵄨󵄨󵄨) 2p 1

≤ (Tr(A2 T ∗ T)) 2p |||T|||

p−1 p

(8.87)

,

and using Assertion 1, we have on the other hand, Tr V = Tr U = Tr |U| ≥ |||U|||.

(8.88)

We have then for any ε > 0, p

1

(Tr A(T ∗ T) A) 2p ≥ ≥ ≥ ≥

1

p

sup ⟨A(T ∗ T) A(v), v⟩ 2p

v∈𝒟, ‖v‖=1

1

p

sup ⟨(T ∗ T) A(v), A(v)⟩ 2p

v∈𝒟, ‖v‖=1

sup

v∈𝒟, ‖v‖=1, ‖Av‖≥ε

sup

1

p

⟨(T ∗ T) A(v), A(v)⟩ 2p

w∈𝒟, ‖A−1 w‖=1, ‖w‖≥ε

p

1

⟨(T ∗ T) w, w⟩ 2p 1

ε p ) ≥ sup ⟨(T T) w, w⟩ ( ‖w‖ w∈𝒟, ‖A−1 w‖=1, ‖w‖≥ε ∗

p

1 2p

468 | 8 Deformation of topological modules hence finally, p

(Tr A(T T) A) ∗

1 2p

1

≥

sup

w∈𝒟, w=0, ̸

‖w‖ ‖A−1 w‖

⟨(T ∗ T)p (w), w⟩ 2p p1 ( ) ε . ‖w‖2 ≥ε

(8.89)

Let us now apply the spectral theorem for the nonnegative bounded self-adjoint operator T ∗ T: |||T|||2

p

(T T) = ∫ x p dE(x) ∗

(8.90)

0

for any positive integer p. According to the spectral theorem, one can write for any w ∈ ℋ − {0} and for any integer p ≥ 1: p

⟨(T ∗ T) w, w⟩ = ‖w‖2 ‖x‖pp , where x stands for the identity map x 󳨃→ x on ℝ, and where ‖−‖p stands for the Lp norm with respect to the probability measure λw = ‖w‖−2 d⟨Ex w, w⟩. Inequality ‖x‖p ≥ ‖x‖1 writes p

⟨(T ∗ T)p w, w⟩ ⟨T ∗ Tw, w⟩ ≥ ( ) . ‖w‖2 ‖w‖2

(8.91)

For any α ∈ ]0, 1[, let us denote by Cα the (open nonempty) cone consisting of vectors w ∈ ℋ − {0} such that ⟨T ∗ Tw, w⟩ > (1 − α)2 |||T|||2 . ‖w‖2 According to (8.91), we have for any w ∈ Cα , ⟨(T ∗ T)p w, w⟩ 2p > ((1 − α)|||T|||) . ‖w‖2

(8.92)

Let us choose some ε(α) > 0 such that ε(α) < sup

w∈Cα ∩𝒟

‖w‖ . ‖A−1 w‖

(8.93)

Starting from inequality (8.89) with ε = ε(α), one can then write using (8.93): p

1

(Tr A(T ∗ T) A) 2p ≥

1

sup

w∈Cα ∩𝒟,

‖w‖ ≥ε(α) ‖A−1 w‖

(

1 ⟨(T ∗ T)p w, w⟩ 2p ) ε(α) p , 2 ‖w‖

(8.94)

8.6 Type-I-ness and consequences | 469

which yields, using (8.87) and (8.92), 1

p

1

1

(1 − α)|||T||| ε(α) p ≤ (Tr A(T ∗ T) A) 2p ≤ (Tr(A2 T ∗ T)) 2p |||T|||

p−1 p

.

(8.95)

The second assertion of the lemma follows by taking some arbitrarily small α > 0 and by considering both sides of inequality (8.95) for p → +∞. Proof of Theorem 8.6.4 (continued). Fix any t ∈ ]0, 1], and apply Lemma 8.6.5 to the d ̃ := (t 2 dπ ξ ,h (ut ))1/2 and T̃ := π ξ ,h (at ), which verify the hypotheses of operators A j t t

Lemma 8.6.5 once we have identified all the representation spaces with ℋ = L2 (ℝd/2 ) as indicated above. The dense domain 𝒟 can be taken as the intersection of the smooth vector spaces ℋ∞ξ ,h for t ∈ ]0, 1]. πt

The limit L𝒪 (a) is indeed what we would expect. Theorem 8.6.6. The following equality holds: 󵄨󵄨󵄨 ξ ,h 󵄨󵄨󵄨 lim 󵄨󵄨󵄨󵄨󵄨󵄨π (at )󵄨󵄨󵄨󵄨󵄨󵄨󵄨󵄨󵄨 t→0 󵄨󵄨󵄨 t

󵄨 󵄨 = sup󵄨󵄨󵄨â0 (η)󵄨󵄨󵄨. η∈𝒪

(8.96)

Proof. This is actually a direct consequence of the general Leptin–Ludwig theorem for variable exponential groups. Indeed, if we see the groupoid 𝒢 as a variable group with parameter space [0, 1], the orbit space Ω is g∗ ∐(]0, 1] × g∗ /G) endowed with the following topology: a point (t, λ) ∈ Ω is limit of the sequence (tn , λn ) if and only if tn → t ∈ [0, 1] and: – either t ≠ 0 and λ is limit of λn in g∗ /G, – or t = 0 and for any n there is ξn ∈ λn such that λ ∈ g∗ is the limit of ξn . In particular, the closure of ]0, 1] × {𝒪} in Ω is given by ]0, 1] × {𝒪} = 𝒪 ∐(]0, 1] × {𝒪}),

(8.97)

expressing the fact that a coadjoint orbit splits in the collection of all of its points when t reaches 0. The Leptin–Ludwig theorem then says that the closure of the set ξ ,h

E𝒪 := {πt

̂ ∘ Et , t ∈ ]0, 1]} ⊂ 𝒜

(8.98)

is {χξ ∘ Et , ξ ∈ 𝒪} ∐ E𝒪 . The conclusion follows then from [71, Lemma I.9]. It would be of course very desirable to have an independent proof of this result: This could be reached by showing that the convergence indicated by the upper arrow of the diagram is uniform with respect to t (or, alternatively, by showing that the convergence indicated by left arrow of the diagram is uniform with respect to p), what we have been unable to prove. This in turn could then be a first step in an alternative

470 | 8 Deformation of topological modules proof of the Leptin–Ludwig bicontinuity theorem, the following conjecture being the second main ingredient. Conjecture 8.6.7. For any compact subset K of g∗ and for any a ∈ 𝒜, the maps t 󳨃→ ξ ,h |||πt (at )||| are equicontinuous at t = 0 with respect to ξ ∈ K. Equivalently, for any quasi-compact subset T of g∗ /G and for any a ∈ 𝒜, the maps t 󳨃→ |||[πt ]𝒪 (at )||| are equicontinuous at t = 0 with respect to 𝒪 ∈ T. Remark that Conjecture 8.6.7 cannot be proved by simply looking at formulae (8.79): This comes from the fact that the analytic functions aX,u , which depend on the initial point ξ ∈ g∗ , do not always behave smoothly and can explode when ξ moves inside a stratum Ωj and reaches its boudary. This phenomenon already occurs on the nilpotent Lie algebra g5,4 , defined by the basis (X1 , X2 , X3 , X4 , X5 ) and nonvanishing brackets [X5 , X3 ] = X1 , [X4 , X3 ] = X2 and [X5 , X4 ] = X3 .

8.6.5 Kirillov map revisited ξ ,h

We keep the previous notation: For t ∈ ]0, 1] recall It𝒪 := Ker πt Ad∗ Gt .ξ = 𝒪, and J 𝒪 := ⋂ Jt𝒪 , t∈]0,1]

∈ Prim 𝒜t , with (8.99)

where Jt𝒪 = Et−1 (It𝒪 ) ∈ Prim 𝒜. For any closed ideal I ⊂ 𝒜 and for any t ∈ [0, 1], we denote by evt (I) the closed ideal Et (I) of 𝒜t . Consider the set of closed ideals: 𝒪

∗

𝒬 := {J , 𝒪 ∈ g /G}.

(8.100)

Theorem 8.6.8. (1) For any t ∈ ]0, 1] the evaluation evt is a bijection from 𝒬 onto Prim 𝒜t . (2) There is a unique topology on 𝒬 such that evt is a homeomorphism for any t ∈ ]0, 1]. (3) The evaluation ev0 is a bijection from 𝒬 onto Prim𝒫 𝒜0 , and the Kirillov map writes κ = ev1 ∘ ev−1 0 .

(8.101)

Proof. The first assertion comes from the fact that Et : J 𝒪 → It𝒪 is surjective and (2) is rather easy. It is enough to prove κ(T) = κ(T) for any quasi-compact T of g∗ /G (by local quasi-compactness of both spaces involved). Let 𝒯 ⊂ g∗ be the union of the orbits in T. Set J 𝒯 := ⋂ J 𝒪 . 𝒪∈𝒯

(8.102)

8.6 Type-I-ness and consequences | 471

If a ∈ J 𝒯 , then â0 |𝒯 = 0 as a consequence of Theorem 8.3.1. Hence we have 𝒯

ℱ (ev0 (J )) ⊂ ℐ𝒯 .

(8.103)

We want the reverse inclusion. Suppose that Conjecture 8.6.7 is true. Then for any a ∈ 𝒜 the map 󵄨󵄨󵄨 󵄨󵄨󵄨 t 󳨃→ sup 󵄨󵄨󵄨󵄨󵄨󵄨󵄨󵄨󵄨[πt ]𝒪 (at )󵄨󵄨󵄨󵄨󵄨󵄨󵄨󵄨󵄨 𝒪∈T

extends by continuity at t = 0 by 󵄨 󵄨 sup󵄨󵄨󵄨â0 (η)󵄨󵄨󵄨 η∈𝒯

(using Theorems 8.6.3 and 8.6.6). Hence 𝒜/J 𝒯 is a field of C ∗ -algebras, which is coñ ∈ 𝒜/J 𝒯 such that tinuous at t = 0. Now let a ∈ 𝒜 such that â0 |𝒯 = 0. It defines a ̃ t ‖ → 0 when t → 0. Hence we have, using Kasparov’s notation,2 ‖a ̃ ∈ (𝒜/J 𝒯 )|]0,1] = 𝒜|]0,1] /(J 𝒯 ∩ A|]0,1] ). a

(8.104)

̃ in the quotient. We obviously have Now choose some element a′ ∈ 𝒜|]0,1] with image a a′ − a ∈ J 𝒯 and (a′ − a)0 = a0 ,

(8.105)

which immediately yields the reverse inclusion ℐ𝒯 ⊂ ℱ (ev0 (J 𝒯 )). We have then the equality 𝒯

ℐ𝒯 = ℱ (ev0 (J ))

(8.106)

for any quasi-compact T ⊂ g∗ /G, which means that ev0 : 𝒬 → Prim𝒫 (𝒜0 ) is a homeomorphism. This proves Theorem 8.6.8. With Theorem 8.6.6 only, we can only repeat the proof for T containing a single orbit, which proves that ev0 is a bijection. The first inclusion for quasi-compact T only shows that this bijection is open. Remark 8.6.9. Theorem 8.6.6 and Conjecture 8.6.7 together yield a proof of the bicontinuity of the Kirillov map. The pertinence of Theorem 8.6.8 is subordinated to finding a proof of Theorem 8.6.6 independent of the Leptin–Ludwig bicontinuity theorem, and a proof of Conjecture 8.6.7 would moreover yield an alternative proof of the latter.

2 According to which, for any open subset U ⊂ [0, 1], 𝒜|U stands for the set of a ∈ 𝒜 such that Et (a) = 0 for any t ∉ U.

Bibliography [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23]

Abdelmoula. L, Baklouti. A, and Kédim. I, The Selberg–Weil–Kobayashi local rigidity theorem for exponential Lie groups, Int. Math. Res. Not. 2012, No. 17, 4062–4084 (2012). Adkins. W. A and Weintraub. S. H, Algebra: An Approach via Module Theory, Graduate Texts in Mathematics, Springer-Verlag, New York, Berlin, Heidelberg (1992). ISBN 0-387-97839-9. Andler. M and Manchon. D, Opérateurs aux différences finies, calcul pseudo-différentiel et représentations des groupes de Lie, J. Geom. Phys. 27, 1–29 (1998). Apanasov. B and Xie. X, Discrete actions on nilpotent Lie groups and negatively curved spaces, Differ. Geom. Appl. 20, 11–29 (2004). Arnal. D, Baklouti. A, Ludwig. J, and Selmi. M, Separation of unitary representations of exponential Lie groups, J. Lie Theory 10, 399–410 (2000). Arnal. D, Ben Amar. N, and Masmoudi. M, Cohomology of good graphs and Kontsevich linear star products, Lett. Math. Phys. 48, 291–306 (1999). Arnal. D and Cortet. J-C, Représentations-∗ des groupes exponentiels, J. Funct. Anal. 92, No. 1, 103–135 (1990). Arnal. D, Manchon. D, and Masmoudi. M, Choix des signes pour la formalité de M. Kontsevich, Pac. J. Math. 203, 23–66 (2002). arXiv:math.QA/0003003. Auslander. L, Bieberbach’s theorem on space groups and discrete uniform subgroups of Lie groups II, Amer. J. Math. 83, No. 2, 276–280 (1961). Auslander. L and Kostant. B, Polarization and unitary representations of solvable Lie groups, Invent. Math. 14, 255–354 (1971). Baklouti. A, Deformation of discontinuous groups acting on some nilpotent homogeneous spaces, Proc. Jpn. Acad., Ser. A, Math. Sci. 85, No. 4, 41–45 (2009). Baklouti. A, On discontinuous subgroups acting on solvable homogeneous spaces, Proc. Jpn. Acad., Ser. A, Math. Sci. 87, 173–177 (2011). Baklouti. A and Bejar. S, On the Calabi–Markus phenomenon and a rigidity theorem for Euclidean motion groups, Kyoto J. Math. 56, No. 2, 325–346 (2016). Baklouti. A and Bejar. S, On the Calabi–Markus phenomenon and a rigidity theorem for Euclidean motion groups. Kyoto. J. Math. 56, No. 2, 325–346 (2016). Baklouti. A and Bejar. S, Variants of stability of discontinuous groups for Euclidean motion groups, Int. J. Math. 28, No. 6, 26p (2017). Baklouti. A, Bejar. S and Dhahri. K, Deforming discontinuous groups for Heisenberg motion groups, Int. J. Math. 30, No. 9, (2019). Baklouti. A, Bejar. S, and Fendri. R, A local rigidity theorem for finite actions on Lie groups and application to compact extensions of ℝn , Kyoto J. Math. 59, No. 3, 607–618 (2019). Baklouti. A, Benson. C, and Ratcliff. G, Moment sets and the unitary dual of a nilpotent Lie group, J. Lie Theory 11, 135–154 (2001). Baklouti. A, Bossofora. M, and Kedim. I, Deformation problems on three-step nilpotent Lie groups. Hiroshima Math. J. 49, No. 2, 195–233 (2019). Baklouti. A, Boussoffara. M, and Kedim. I, Stability of Discontinuous Groups Acting on Homogeneous Spaces, Mathematical Notes. 103, No. 4, 9–22 (2018). Baklouti. A, Dhieb. S, and Manchon. D, Orbites coadjointes et variétés caractéristiques, J. Geom. Phys. 54, 1–41 (2005). Baklouti. A, Dhieb. S, and Manchon. D, A deformation approach of the Kirillov map for exponential groups, Adv. Pure Appl. Math. 2, 421–436 (2011). Baklouti. A, Dhieb. S, and Manchon. D, The Poisson characteristic variety of unitary irreducible representations of exponential Lie groups, Springer Proceedings in Mathematics and Statistics. 290, 207–217 (2019).

https://doi.org/10.1515/9783110765304-009

474 | Bibliography

[24] [25] [26]

[27] [28] [29]

[30] [31] [32] [33] [34]

[35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47]

Baklouti. A, Dhieb. S, and Tounsi. K, When is the deformation space T (Γ, H2n+1 , H) a smooth manifold?, Int. J. Math. 22, No. 11, 1661–1681 (2011). Baklouti. A, ElAloui. N, and Kédim. I, A rigidity theorem and a stability theorem for two-step nilpotent Lie groups, J. Math. Sci. Univ. Tokyo 19, 281–307 (2012). Baklouti. A, Fujiwara. H, and Ludwig. J, Representation Theory of Solvable Lie Groups and Related Topics, Springer Monographs in Mathematics (2021). eBook ISBN 978-3-030-82044-2. Baklouti. A, Ghaour. S, and Khlif. F, Deforming discontinuous subgroups of reduced Heisenberg groups, Kyoto J. Math. 55, No. 1, 219–242 (2015). Baklouti. A, Ghaour. S, and Khlif. F, On discontinuous groups acting on (ℍr2n+1 × ℍr2n+1 )/Δ, Adv. Pure Appl. Math. 6, No. 2, 63–79 (2015). Baklouti. A, Ghaour. S, and Khlif. F, A stability theorem for non-abelian actions on threadlike homogeneous spaces, Springer Proceedings in Mathematics and Statistics. 207, 117–135 (2017). Baklouti. A and Kédim. I, On the deformation space of Clifford–Klein forms of some exponential homogeneous spaces, Int. J. Math. 20, No. 7, 817–839 (2009). Baklouti. A and Kédim. I, On non-Abelian discontinuous subgroups acting on exponential solvable homogeneous spaces, Int. Math. Res. Not. 2010, No. 7, 1315–1345 (2010). Baklouti. A and Kédim. I, Open problems in deformation theory of discontinuous groups acting on homogeneous spaces, Int. J. Open Problems Comput. Math. 6, No. 1, 115–131 (2013). Baklouti. A and Kédim. I, On the local rigidity of discontinuous groups for exponential solvable Lie groups, Adv. Pure. Appl. Maths. 4, No. 1, 3–20 (2013). Baklouti. A, Kédim. I, and Yoshino. T, On the deformation space of Clifford–Klein forms of Heisenberg groups, Int. Math. Res. Not. 2008, No. 16, Art. ID rnn066, 35p (2008). doi:10.1093/imrn/rnn066. Baklouti. A and Khlif. F, Proper actions on some exponential solvable homogeneous spaces, Int. J. Math. 16, No. 9, 941–955 (2005). Baklouti. A and Khlif. F, Criterion of weak proper actions on solvable homogeneous spaces, Int. J. Math. 18 No. 8, 903–918 (2007). Baklouti. A, and Khlif. F, Deforming discontinuous subgroups for threadlike homogeneous spaces, Mathematical Notes. 146, 117–140 (2010). Baklouti. A, Khlif. F, and Kooba. H, On the geometry of stable discontinuous subgroups acting on threadlike homogeneous spaces, Mathematical Notes. 89, No. 5-6, 761–776 (2011). Barmeier. S, Deformations of the discrete Heisenberg group, Proc. Jpn. Acad., Ser. A, Math. Sci. 89, No. 4, 55–59 (2013). Baues. O and Goldman. W, Is the deformation space of complete affine structures on the 2-torus smooth?, Geom. Dyn., Contemp. Math. 389, 69–89 (2005). Benedetti. R and Risler. J. J, Real Algebraic and Semi-Algebraic Sets, Herman, 340p (1990). Bernat. P, Conze. N, Duflo. M, et al., Représentations des Groupes de Lie Résolubles, Monographies de la Société Mathématique de France, Vol. 4, Dunod, Paris (1972). Bieberbach. L, Über die Bewegungsgruppen der Euklidischen Rame I, Math. Ann. 70, 297–336 (1911). Bieberbach. L, Über die Bewegungsgruppen der Euklidischen Rame II, Math. Ann. 72, 400–412 (1912). Blanchard. E, Déformations de C ∗ -algèbres de Hopf, Bull. Soc. Math. Fr. 124, No. 1, 141–215 (1996). Bordemann. M, Ginot. G, Halbout. G, Herbig. H-C, and Waldmann. S, Star-représentations sur des sous-variétés co-isotropes, arXiv:math.QA/0309321 (2003). Bourbaki. N, Algèbre Commutative, Hermann, Paris (1961).

Bibliography | 475

[48] [49] [50] [51] [52] [53] [54] [55] [56] [57] [58] [59] [60] [61] [62] [63] [64] [65] [66] [67] [68] [69] [70] [71] [72] [73] [74]

Bourbaki. N, Éléments de Mathématiques Topologie Générale Chapitre 5 á 10, Springer (2007). Brown. I. D, Dual topology of a nilpotent Lie group, Ann. Sci. Éc. Norm. Supér. 6, No. 3, 407–411 (1973). Bursztyn. H and Waldmann. S, ∗-ideals and formal Morita equivalence of ∗-algebras, Int. J. Math. 12 No 5, 555–577 (2001). Calabi. E, On compact, Riemannian manifolds with constant curvature, I, in Proceedings of Symposia in Pure Mathematics, Vol. III, Amer. Math. Soc., Providence, RI, 155–180 (1961). Calabi. E and Markus. L, Relativistic space forms, Ann. Math. 75, 63–76 (1962). Cattaneo. A and Felder. G, A path integral approach to the Kontsevic quantization formula, Commun. Math. Phys. 212, No. 3, 591–611 (2000). Cattaneo. A and Felder. G, Coisotropic submanifolds in Poisson geometry and branes in the Poisson sigma model, arXiv:math.QA/0309180 (2003). Cattaneo. A, Felder. G and Tomassini. L, From local to global deformation quantization of Poisson manifolds, arXiv:math.QA/0012228 (2000). Chevallier. D, Introduction à la Théorie des Groupes de Lie Réels, Ellipses (2006). Choi. S and Goldman. W, The classification of real projective structures on compact surfaces, Bull. Amer. Math. Soc. (N.S.) 34, No. 2, 161–171 (1997). Corwin. L and Greenleaf. F, Representations of Nilpotent Lie Groups and Their Applications. Part 1, Cambridge Studies in Advanced Mathematics, Vol. 18, Cambridge University Press (2004). Corwin. L, Greenleaf. F. P, and Grelaud. G, Direct integral decompositions and multiplicities for induced representations of nilpotent Lie groups, Trans. Amer. Math. Soc. 304, 549–583 (1987). Dhieb. S, Deformation of discontinuous groups acting on (H2n+1 × H2n+1 )/Δ, J. Lie Theory 26, No. 2, 371–382 (2016). Dito. G, Kontsevich star product on the dual of a Lie algebra, Lett. Math. Phys. 48, 307–322 (1999). Dixmier. J, Les C ∗ -Algèbres et Leurs Représentations, Gauthier-Villars, Paris (1964). Dixmier. J, Algèbres Enveloppantes, Gauthier-Villars, Paris (1974). Duflo. M, Opérateurs différentiels bi-invariants sur un groupe de Lie, Ann. Sci. Éc. Norm. Supér. (4) 10, 265–288 (1977). Duflo. M, Sur la classification des idéaux primitifs dans l’algèbre enveloppante d’une algèbre de Lie semi-simple, Ann. Math. (2) 105, No. 1, 107–120 (1977). Duflo. M and Raïs. M, Sur l’analyse harmonique sur les groupes de Lie résolubles, Ann. Sci. Éc. Norm. Supér. 9, 107–144 (1976). Dunfield. N. M and Thurston. W. P, The virtual Haken conjecture: experiments and examples, Geom. Topol. 7, 399–441 (2003). Enright. T, Howe. R, and Wallach. N, A classification of unitary highest weight modules, Prog. Math. 40, 97–143 (1983). Etingof. P and Kazhdan. D, Quantization of Lie bialgebras, I, Sel. Math. 2, 1–41 (1996). Felder. G and Shoikhet. B, Deformation quantization with traces, arXiv:math.QA/0002057 (2000). Fell. J. M. G, The structure of algebras of operator fields, Acta Math. 106, 233–280 (1961). Gabber. O, The integrability of the characteristic variety, Am. J. Math. 103, No. 3, 445–468 (1981). Ghys. É, Déformations des structures complexes sur les espaces homogènes de SL(2, ℂ), J. Reine Angew. Math. 468, 113–138 (1995). Godement. R, Introduction à la Théorie des Groupes de Lie, Springer-Verlag, Berlin, Heidelberg (2004).

476 | Bibliography

[75] [76] [77] [78] [79] [80] [81] [82] [83] [84] [85] [86] [87] [88] [89] [90] [91] [92] [93]

[94] [95] [96] [97] [98] [99]

Godfrey. C, Ideals of coadjoint orbits of nilpotent Lie algebras, Trans. Amer. Math. Soc. 233, 295–307 (1977). Goldman. W, Non-standard Lorentz space forms, J. Differ. Geom. 21, 301–308 (1985). Goldman. W and Millson. J. J, Local rigidity of discrete groups acting on complex hyperbolic space, Invent. Math. 88, 495–520 (1987). Goldman. W. M, Locally homogeneous geometric manifolds, in Proceedings of the International Congress of Mathematicians, Hyderabad, India (2010). Granger. M and Maisonobe. Ph, A basic course on differential modules, in D-Modules Cohérents et Holonomes, Les Cours du CIMPA, Hermann, Paris (1993). Hall. B. C, Lie Groups, Lie Algebras, and Representations, Springer (2004). Helgason. S, Differential Geometry and Symmetric Spaces, Academic Press (1962). Hilgert. J and Neeb. K-H, Structure and Geometry of Lie Groups, Springer Monographs in Mathematics, Springer Science+Business Media, LLC (2012). doi:10.1007/978-0-387-84794-8. Hochschild. G. P, The Structure of Lie Groups, Holden-Day Series in Mathematics, Holden-Day, Inc., San Francisco (1965). Hofmann. K. H and Neeb. K-H, The compact generation of closed subgroups of locally compact groups, J. Group Theory 12, No. 4, 555–559 (2009). Jakobsen. H. P, Hermitian symmetric spaces and their unitary highest weight modules, J. Funct. Anal. 52 No. 3, 385–412 (1983). Joseph. A, On the classification of primitive ideals in the enveloping algebra of a semi-simple Lie algebra, in Lie Group Representations I, Lecture Notes in Mathematics, Vol. 1024, Springer, 30–78 (1983). Kassel. Ch, Quantum Groups, Springer (1995). Kato. T, Perturbation Theory for Linear Operators, Springer (1966). Kédim. I, Rigidity of discontinuous actions on diamond homogeneous spaces. J. Math. Sci. Univ. Tokyo 23, No. 2, 381–403 (2016). Khlif. F, Stability of discontinuous groups for reduced threadlike Lie groups, Int. J. Math. 26, No. 08 (2015). Kirillov. A. A, Elements of the Theory of Representations, Springer (1976). Kobayashi. T, Proper action on homogeneous space of reductive type, Math. Ann. 285, 249–263 (1989). Kobayashi. T, Discontinuous groups acting on homogeneous spaces of reductive type, in Proceeding of the Conference on Representation Theorie of Lie Groups and Lie Algebras held in 1990 August–September at Fuji-Kawaguchiko (ICM-90 Satellite Conference), Word Scientific, 59–75 (1992). Kobayashi. T, On discontinuous groups on homogeneous space with noncompact isotropy subgroups, J. Geom. Phys. 12, 133–144 (1993). Kobayashi. T, Criterion of proper action on homogeneous space of reductive type, J. Lie Theory 6, 147–163 (1996). Kobayashi. T, Discontinuous groups and Clifford–Klein forms of pseudo-Riemannian homogeneous manifolds, in Algebraic and Analytic Methods in Representation Theory, Perspectives in Mathematics, Vol. 17, Academic Press, 99–165 (1996). Kobayashi. T, Deformation of compact Clifford–Klein forms of indefinite Riemannian homogeneous manifolds, Math. Ann. 310, 394–408 (1998). Kobayashi. T, Discontinuous groups for non Riemannian homogeneous spaces, in Mathematics Unlimited—2001 and Beyond (B. Engquist and W. Schmid, eds), Springer-Verlag, 723–747 (2001). Kobayashi. T, On discontinuous group actions on non Riemannian homogeneous spaces, Sūgaku Expo. 22, 1–19 (2009). arXiv:math.DG/0603319.

Bibliography | 477

[100] Kobayashi. T and Nasrin. S, Deformation of properly discontinuous action of ℤk on ℝk+1 , Int. J. Math. 17, 1175–1193 (2006). [101] Kobayashi. T and Yoshino. T, Compact Clifford–Klein forms of symmetric spaces-revisited, Pure Appl. Math. Q. 1, No. 3, 591–663 (2005), special issue: in memory of Armand Borel, part 2 of 3. [102] Kontsevich. M, Deformation quantization of Poisson manifolds I, arXiv:math.QA/9709040 (1997). [103] Kulkarni. R. S, Proper actions and pseudo-Riemannian space forms, Adv. Math. 40, No. 1, 10–51 (1981). [104] Kulkarni. R. S, Lee. K. B, and Raymond. F, Deformation spaces for Seifert manifolds, in Geometry and Topology, Lecture Notes in Mathematics, Vol. 1167, Springer, Berlin, 180–216 (1985). [105] Labourie. F, Mozes. S, and Zimmer. R. J, On manifolds locally modelled on non-Riemannian homogeneous spaces, Geom. Funct. Anal. 5, No. 6, 955–965 (1995). [106] Leptin. H and Ludwig. J, Unitary Representation Theory of Exponential Lie Groups, De Gruyter Expositions in Mathematics, Vol. 18 (1994). [107] Lipsman. R, Representations of exponential solvable Lie groups induced from maximal subgroups, Mich. Math. J. 40, 299–320 (1993). [108] Lipsman. R, Proper action and a compactness condition, J. Lie Theory 5, 25–39 (1995). [109] Magnin. L, Determination of 7-dimensional indecomposable nilpotent complex Lie algebras by adjoining a derivation to 6-dimensional Lie algebras, Algebr. Represent. Theory 13, No. 6, 723–753 (2010). [110] Manchon. D and Torossian. Ch, Cohomologie tangente et cup-produit pour la quantification de Kontsevich, arXiv:math.QA/0106205 (2001). [111] Mathieu. O, Bicontinuity of the Dixmier map, J. Am. Math. Soc. 4, No. 4, 837–863 (1991). [112] Mostow. G. D, On a conjecture of Montgomery, Ann. Math. (2) 65, No. 3, 513–516 (1957). [113] Motzkin. T. S and Taussky. O, Pairs of matrices with property L, Trans. Amer. Math. Soc. 73, 108–114 (1952). [114] Nasrin. S, Criterion of proper actions for 2-step nilpotent Lie groups, Tokyo J. Math. 24, No. 2, 535–543 (2001). [115] Oliver. R. K, On Bieberbach’s analysis of discrete Euclidean groups, Proc. Amer. Math. Soc. 80, 15–21 (1980). [116] Palais. R. S, On the existence of slices for actions of non-compact Lie groups, Ann. Math. 73, 295–323 (1961). [117] Pedersen. N. V, On the characters of exponential solvable Lie groups, Ann. Sci. Éc. Norm. Supér. 17, No. 1, 1–29 (1984). [118] Pedersen. N. V, On the infinitesimal kernel of irreducible representations of nilpotent Lie groups, Bull. Soc. Math. Fr. 112, No. 42, 3–467 (1984). [119] Pedersen. N. V, On the symplectic structure of coadjoint orbits of (solvable) Lie groups and applications. I, Math. Ann. 281, 633–669 (1988). [120] Pedersen. N. V, Orbits and primitive ideals of solvable Lie algebras, Math. Ann. 298, No. 2, 275–326 (1994). [121] Robinson. D. J, A Course in the Theory of Groups, 2nd ed., Springer-Verlag (1996). [122] Saitô. M, Sur certain groupes de Lie résolubles I; II, Sci. Pap. Coll. Gen. Educ. Univ. Tokyo 7, 1–11 (1957). [123] Selberg. A, On discontinuous groups in higher-dimension symmetric spaces, in Contributions to Functional Theory, Tata Institute, Bombay, 147–164 (1960). [124] Shapovalov. N. N, A certain bilinear form on the universal enveloping algebra of a complex semi-simple Lie algebra, Funct. Anal. Appl. 6, No. 4, 307–312 (1972).

478 | Bibliography

[125] Takenouchi. O, Sur la facteur-représentation d’un groupe de Lie résoluble de type (E), Math. J. Okayama Univ. 7, 151–161 (1957). [126] Tamarkin. D, Another proof of M. Kontsevich formality theorem, arXiv:math.QA/9803025 (1998). [127] Vanhaecke. P, Integrable Systems in the Realm of Algebraic Geometry, Lecture Notes in Mathematics, Vol. 1638, Springer (1996). [128] Varadarajan. V. S, Lie Groups, Lie Algebras and Their Representations, Graduate Texts in Mathematics, Vol. 102, Springer, New York, NY (1984). [129] Weil. A, On discrete subgroups of Lie groups II, Ann. Math. 75, 578–602 (1962). [130] Weil. A, Remarks on the cohomology of groups, Ann. Math. 80, 149–157 (1964). [131] Weinstein. A, The local structure of Poisson manifolds, J. Differ. Geom. 18, 523–557 (1983). [132] Wüstner. M, A connected Lie group equals the square of the exponential image, J. Lie Theory 13, 307–309 (2003). [133] Yoshino. T, A counterexample to Lipsman’s conjecture, Int. J. Math. 16, 561–566 (2005). [134] Yoshino. T, Criterion of proper actions for 3-step nilpotent Lie groups, Int. J. Math. 85, 887–893 (2007). [135] Yoshino. T, Deformation spaces of compact Clifford–Klein forms of homogeneous spaces of Heisenberg groups, in Representation Theory and Analysis on Homogeneous Spaces, RIMS Kôkyûroku Bessatsu B7, Res. Inst. Math. Sci. (RIMS), Kyoto, 45–55 (2008). [136] Yoshino. T, A solution to Lipsman’s conjecture for ℝ4 , Preprint.

Index C ∞ -vector 429 G-equivariant map 203 G-principal bundle 100 ν-adic topology 413 m-step nilpotent Lie algebra, group 2 (CI) triple 51 affine group of the real line 3 almost connected group 20 antiautomorphism 419 bidifferential operator 412 Calabi–Markus phenomenon 324 Campbell–Baker–Hausdorff formula 56 canonical Euclidean scalar product 10 Cartan decomposition 83 Cartan motion group 83 Cartan subalgebra 455 characteristic variety 418 characteristically completely solvable algebra 243 characteristically nilpotent algebra 243 characteristically solvable algebra 243 Chevalley basis 457 Clifford–Klein form 52 Clifford–Klein space 89 co-exponential basis 3 coadjoint orbit 427 colored discrete subgroup 253 compact Clifford–Klein form 133, 302 compact group of automorphisms 37 compact intersection property 51 compact Lie algebra 26 compactly generated group 20 compatible affine charts 99 compatible subalgebra 264 completely solvable Lie algebra 2 complex semisimple Lie algebra 455 complex unitary matrix 35 concatenation of matrices 204 connected component of the identity 20 continuous deformation 226 continuous group homomorphism 45 continuously deformable form 226 converging topological free module 425 crystallographic discontinuous group 316 https://doi.org/10.1515/9783110765304-010

crystallographic subgroup 21, 22 cyclic group 22 Darboux coordinate system 429 deformation space 88 derivations algebra 243 diagonal subgroup 152 diamond group 263, 452 dilation 244, 251, 264 dilation-invariant subgroup 269 discontinuous group 52 discrete subgroup of I(n) 15 divisible ideal 415 divisible submodule 414 Dixmier map 448 Duflo isomorphism 424 Duflo–Kontsevich star-product 412 effective action 89 element of finite order 18 element of infinite order 20 equivalent topological modules 415 equivariant map 90 Euclidean motion groups 10 exponential mapping 2 exponential solvable Lie algebra of type T 250 exponential solvable Lie group 2 Fell topology 427 filtered algebra 412 finite index subgroup 37 finite proper action 53 finitely generated Abelian group 311 finitely generated group 37 fixed point free action 51 formal differential operator 420 free group 22 free triple 51 generation family of a group 18 geometric stability 284 graded Lie algebra 263 graded Lie subalgebra 244 Grassmannian structure 98 Hahn–Banach theorem 422 Hausdorff space 164

480 | Index

Heisenberg group 5 Heisenberg motion group 35 Hermitian nondegenerate sesquilinear form 421 homogeneous space 51 infinite fundamental group 324 intertwining operator 415 invariant geometric structure 52 invariant symmetric bilinear form 26 involution of a Lie group 83 involutive automorphism 83 involutive ideal 416 irreducible Riemannian symmetric space 284 isotropy group 51 Jordan–Hölder basis 4 Jordan–Hölder sequence 4 Kazhdan’s property (T) 231 kernel of effectiveness 90 Killing form 455 Kirillov map 427 Kirillov orbits method 424 Kirillov–Bernat theory 427 Kirillov–Kostant–Souriau bracket 423 Kontsevich star-product 412 Levi decomposition 33, 264, 269, 270 Levi factor 26 Lie bracket 1 linear Poisson manifold 422 linear reductive Lie group 83 Lipsman’s conjecture 55 local diffeomorphism 52 local rigidity 226 local rigidity conjecture 226 local section 110 local trivialization 99 locally convex topological space 422 locally isomorphic groups 231 Malcev basis 4 manifold structure 52 matrix of maximal rank 115, 140, 142, 203, 213, 217, 233 maximal Abelian subalgebra 164 maximal compact subgroup 43 maximal ideal 460 maximal solvable homogeneous space 72

maximal solvable subalgebra 72 maximal solvmanifold 72 maximal subgroup of a solvable Lie group 7 maximal torus 40 moduli space 88 Montgomery conjecture 275 multiplicity-free representation 462 near stability 284 non-Riemannian symmetric space 231 noncommutative free group 324 open covering 52 orthogonal group 10 orthogonal projection 13 pair having Lipsman property 55 parameter space 87 point wise convergence topology 47 Poisson algebra 412 Poisson characteristic variety 418 Poisson ideal 459 Poisson manifold 412 positive root 455 pre-Abelian subgroup 50 prime power order 311 primitive Poisson ideal 459 principal bundle 112 principal fiber bundle 109 proper action 51 proper map 46 proper triple 51 properly discontinuous action 52 Pukánszky polarization 427 quasi-equivalent representations 462 quotient canonical surjection 52 quotient space 170 radical decomposition 456 rank one solvable case 254 rank two solvable case 260 reduced exponential Lie group 48 reduced Heisenberg group 335 reduced homogeneous space 346 reduced nilmanifold 335 reduced threadlike group 366 refined deformation space 90 representation of a topological module 420

Index |

restriction natural map 47 Rieffel’s strict quantization 463 ring of formal series 413 root system 455 Selberg–Weil–Kobayashi local rigidity theorem 231 semialgebraic map 96 semialgebraic subset 96 semicharacter formula 466 semidirect product Lie group 19 semisimple Lie subgroup 26 sequence of linear maps 169 solvable Lie algebra, group 2 solvable radical 33 special nilpotent Lie group 62 special solvmanifold 65 stability 284 stable discrete subgroup 285 stable homomorphism 284 stable on layers homomorphism 394 strong local rigidity 226 strong Malcev basis 4 strong Malcev sequence 4 strongly continuous representation 429 strongly nondegenerate representation 421 strongly pseudo-unitary module 421 strongly unitary module 421 strongly unitary representation 421 structural sheaf 412 subgraded subalgebra 264 subgraded subgroup 269 subgroup of finite index 20 symmetrization map 424 symplectic leaf 416–419, 424, 459, 461 syndetic hull 43

tangent groupoid 463 threadlike Lie group 6 threadlike nilmanifold 200 topological fiber bundle 99 topological module 414 torsion-free 39 torsion-free subgroup 37 torsion-free uniform lattice 284 total space 99 transition functions 112 transition matrix 23 transitive action 100 trivial deformation 225 twisted Schatten seminorm 466 type I group 448, 462, 463 ultrametric distance 413 uniform lattice 225 unitary character 427, 439 unitary dual 427 unitary induced representation 427 unitary irreducible representation 411, 427 unitary module 421 universal covering 48 Verma module 455 weak proper action 53 weakly convergent module 423 weakly proper triple 54 Weyl’s theorem 26 Zariski closure conjecture 427 Zariski dense set 417

481

De Gruyter Expositions in Mathematics Volume 71 Victor P. Maslov, Yurievich Oleg Shvedov† The Canonical Operator in Multipartiсle Problems and Quantum Field Theory, 2022 ISBN 978-3-11-076238-9, e-ISBN 978-3-11-076270-9 Volume 70 Aron Simis, Zaqueu Ramos Graded Algebras in Algebraic Geometry, 2022 ISBN 978-3-11-063754-0, e-ISBN 978-3-11-064069-4 Volume 69/2 Mariusz Urbański, Mario Roy, Sara Munday Non-Invertible Dynamical Systems. Volume 2: Finer Thermodynamic Formalism – Distance Expanding Maps and Countable State Subshifts of Finite Type, Conformal GDMSs, Lasota-Yorke Maps and Fractal Geometry, 2022 ISBN 978-3-11-070061-9, e-ISBN 978-3-11-070269-9 Volume 69/1 Mariusz Urbański, Mario Roy, Sara Munday Non-Invertible Dynamical Systems. Volume 1: Ergodic Theory – Finite and Infinite, Thermodynamic Formalism, Symbolic Dynamics and Distance Expanding Maps, 2021 ISBN 978-3-11-070264-4, e-ISBN 978-3-11-070268-2 Volume 68 Yuri A. Bahturin Identical Relations in Lie Algebras, 2021 ISBN 978-3-11-056557-7, e-ISBN 978-3-11-056665-9 Volume 34 Marek Jarnicki, Peter Pflug Extension of Holomorphic Functions, 2nd Edition, 2020 ISBN 978-3-11-062766-4, e-ISBN 978-3-11-063027-5 Volume 67 Alcides Lins Neto, Bruno Scárdua Complex Algebraic Foliations, 2020 ISBN 978-3-11-060107-7, e-ISBN 978-3-11-060205-0

www.degruyter.com