N-Geometry of the generalized geodesic flow and inverse spectral problems [2 ed.] 1119107660, 9781119107668, 9781119107699, 1119107695

Citation preview

Geometry of the Generalized Geodesic Flow and Inverse Spectral Problems

Geometry of the Generalized Geodesic Flow and Inverse Spectral Problems Second Edition Vesselin M. Petkov University of Bordeaux, France

Luchezar N. Stoyanov University of Western Australia, Australia

This edition first published 2017 © 2017 John Wiley & Sons, Ltd First Edition published in 1992 Registered office John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, United Kingdom For details of our global editorial offices, for customer services and for information about how to apply for permission to reuse the copyright material in this book please see our website at www.wiley.com. The right of the author to be identified as the author of this work has been asserted in accordance with the Copyright, Designs and Patents Act 1988. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK Copyright, Designs and Patents Act 1988, without the prior permission of the publisher. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The publisher is not associated with any product or vendor mentioned in this book Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. It is sold on the understanding that the publisher is not engaged in rendering professional services and neither the publisher nor the author shall be liable for damages arising herefrom. If professional advice or other expert assistance is required, the services of a competent professional should be sought.

Library of Congress Cataloging-in-Publication Data Names: Petkov, Vesselin. | Stoyanov, Luchezar N., 1954- | Petkov, Vesselin. Geometry of reflecting rays and inverse spectral problems. Title: Geometry of the generalized geodesic flow and inverse spectral problems / Vesselin M. Petkov, Luchezar N. Stoyanov. Other titles: Geometry of reflecting rays and inverse spectral problems Description: Second edition. | Chichester, West Sussex : John Wiley & Sons, Inc., 2017. | Revised and expanded edition of: Geometry of reflecting rays and inverse spectral problems (Chichester, England ; New York : Wiley, c1992). | Includes bibliographical references and index. Identifiers: LCCN 2016022663| ISBN 9781119107668 (cloth) | ISBN 9781119107699 (epub) Subjects: LCSH: Spectral theory (Mathematics) | Inverse problems (Differential equations) | Geometry, Differential. | Geodesic flows. | Flows (Differentiable dynamical systems) Classification: LCC QA320 .P435 2017 | DDC 515/.7222–dc23 LC record available at https://lccn.loc.gov/2016022663 A catalogue record for this book is available from the British Library. ISBN: 9781119107668 Typeset in 10/12pt TimesLTStd by SPi Global, Chennai, India

10 9 8 7 6 5 4 3 2 1

Contents Preface

ix

1 Preliminaries from differential topology and microlocal analysis 1.1 Spaces of jets and transversality theorems 1.2 Generalized bicharacteristics 1.3 Wave front sets of distributions 1.4 Boundary problems for the wave operator 1.5 Notes

1 1 5 15 23 25

2 Reflecting rays 2.1 Billiard ball map 2.2 Periodic rays for several convex bodies 2.3 The Poincar´e map 2.4 Scattering rays 2.5 Notes

26 26 31 40 49 56

3 Poisson relation for manifolds with boundary 3.1 Traces of the fundamental solutions of  and 2 3.2 The distribution σ(t) 3.3 Poisson relation for convex domains 3.4 Poisson relation for arbitrary domains 3.5 Notes

57 58 62 64 71 81

4 Poisson summation formula for manifolds with boundary 4.1 Global parametrix for mixed problems 4.2 Principal symbol of FˆB 4.3 Poisson summation formula 4.4 Notes

82 82 94 103 117

5 Poisson relation for the scattering kernel 5.1 Representation of the scattering kernel 5.2 Location of the singularities of s(t, θ, ω) 5.3 Poisson relation for the scattering kernel 5.4 Notes

118 118 127 130 137

vi

CONTENTS

6 Generic properties of reflecting rays 6.1 Generic properties of smooth embeddings 6.2 Elementary generic properties of reflecting rays 6.3 Absence of tangent segments 6.4 Non-degeneracy of reflecting rays 6.5 Notes

139 139 145 155 160 172

7 Bumpy surfaces 7.1 Poincar´e maps for closed geodesics 7.2 Local perturbations of smooth surfaces 7.3 Non-degeneracy and transversality 7.4 Global perturbations of smooth surfaces 7.5 Notes

173 173 182 191 199 202

8 Inverse spectral results for generic bounded domains 8.1 Planar domains 8.2 Interpolating Hamiltonians 8.3 Approximations of closed geodesics by periodic reflecting rays 8.4 The Poisson relation for generic strictly convex domains 8.5 Notes

204 204 214 221 235 241

9 Singularities of the scattering kernel 9.1 Singularity of the scattering kernel for a non-degenerate (ω, θ)-ray 9.2 Singularities of the scattering kernel for generic domains 9.3 Glancing ω-rays 9.4 Generic domains in R3 9.5 Notes

242 242 252 253 258 263

10 Scattering invariants for several strictly convex domains 10.1 Singularities of the scattering kernel for generic θ 10.2 Hyperbolicity of scattering trajectories 10.3 Existence of scattering rays and asymptotic of their sojourn times 10.4 Asymptotic of the coefficients of the main singularity 10.5 Notes

264 264 273 281 287 296

11 Poisson relation for the scattering kernel for generic directions 11.1 The Poisson relation for the scattering kernel 11.2 Generalized Hamiltonian flow 11.3 Invariance of the Hausdorff dimension 11.4 Further regularity of the generalized Hamiltonian flow 11.5 Proof of Proposition 11.1.2 11.6 Notes

298 298 303 309 320 325 336

12 Scattering kernel for trapping obstacles 12.1 Scattering rays with sojourn times tending to infinity 12.2 Scattering amplitude and the cut-off resolvent

337 337 343

CONTENTS

12.3 Estimates for the scattering amplitude 12.4 Notes 13 Inverse scattering by obstacles 13.1 The scattering length spectrum and the generalized geodesic flow 13.2 Proof of Theorem 13.1.2 13.3 An example: star-shaped obstacles 13.4 Tangential singularities of scattering rays I 13.5 Tangential singularities of scattering rays II 13.6 Reflection points of scattering rays and winding numbers 13.7 Recovering the accessible part of an obstacle 13.8 Proof of Proposition 13.4.2 13.9 Notes

vii

347 350 351 351 356 363 365 368 374 380 385 394

References

396

Topic Index

405

Symbol Index

409

Preface This monograph is devoted to the analysis of some inverse problems concerning the spectrum of the Laplace operator in a bounded domain Ω ⊂ Rn , n ≥ 2, and of the scattering length spectrum (SLS) (the set of sojourn times of reflecting rays) of the scattering kernel associated with scattering in the exterior Ω of a bounded obstacle K ⊂ Rn , n ≥ 2. In both cases our aim is to obtain some geometric information about Ω (resp. K) from spectral (resp. scattering) data. We treat both inverse problems by using similar techniques based on properties of the generalized geodesic flow in Ω and on microlocal analysis of the corresponding mixed problems. Let Ω ⊂ Rn , n ≥ 2, be a closed bounded domain with C ∞ smooth boundary ∂Ω, and let A be the self-adjoint operator in L2 (Ω) related to the Laplacian −Δ = −

n 

∂x2 j

j=1

in Ω with Dirichlet boundary condition on ∂Ω. The spectrum of A is given by a sequence (0.1) 0 ≤ λ21 ≤ λ22 ≤ · · · ≤ λ2m ≤ · · · of eigenvalues λ2j for which the problem  −Δϕj = λ2j ϕj in Ω, ϕj = 0 on ∂Ω has a non-trivial solution ϕj ∈ C ∞ (Ω). The counting function N (λ) = #{j : λ2j ≤ λ2 }, where every eigenvalue is counted with its multiplicity, admits a polynomial bound N (λ) ≤ Cλn ,

λ → +∞.

(0.2)

Moreover, it is known (see [Se], [H4], [SaV]) that N (λ) has a Weyl type asymptotic N (λ) =

(4π)−n/2 Voln (Ω)λn + O(λn−1 ) Γ(n/2 + 1)

(0.3)

x

PREFACE

as λ → ∞. Thus, from the spectrum (0.1) we can recover the volume of Ω. In 1911, Weyl [W] conjectured that for every bounded domain Ω in Rn with smooth boundary ∂Ω we have (4π)−(n−1)/2 (4π)−n/2 Voln (Ω)λn − Voln−1 (∂Ω) + o(λn−1 ) Γ(n/2 + 1) 4Γ(n − 1/2 + 1) (0.4) as λ → ∞. Ivrii [Ivl] proved that if the points (x, v) ∈ ∂Ω × Sn−1 for which there exists a periodic billiard trajectory in Ω issued from x in direction v form a subset of Lebesgue measure zero in the space ∂Ω × Sn−1 , then the asymptotic (0.4) holds. Therefore, for such domain Voln−1 (∂Ω) becomes another spectral invariant. It is not known so far if the assumption in Ivrii’s result is always satisfied. To obtain more information from the knowledge of the spectrum {λ2j }, it is convenient to examine some distributions determined by the sequence (0.1). The distribution  2 ¯ ) e−λj t ∈ D (R τ (t) = + N (λ) =

j

has the asymptotic τ (t) ∼

∞ 

cj t−(n/2)+j/2 as t 0,

(0.5)

j=1

and the constants cj are spectral invariants. Moreover, one can recover Voln (Ω) and Voln−1 (∂Ω) from c0 and c1 . In his classical work Kac [Kac] posed the problem of recovering the shape of a strictly convex domain Ω ⊂ R2 from the spectrum (0.1). This article has had a big influence on the investigations of various inverse spectral problems for manifolds with and without boundary as well as on the analysis of the so-called isospectral manifolds, that is manifolds for which the spectra of the corresponding Laplace–Beltrami operators coincide. To determine a strictly convex planar domain Ω, modulo Euclidean transformations, it suffices to know the curvature K(x) of ∂Ω at each point x ∈ ∂Ω. In general, the spectral data {cj }∞ j=0 , given by (0.5), is not sufficient to determine the function K(x). Let us mention that the distribution τ (t) is singular only at t = 0. A distribution related to {λ2j } having a larger singular set is σ(t) =

∞ 

cos(λj t) ∈ S  (R).

(0.6)

j=1

This distribution is singular at 0 and σ(t) ∼

∞ 

dj t−n+j

j=0

(see [Me3], [Iv2]). The constants dj provide other spectral invariants, and the first two determine again Voln (Ω) and Voln−1 (∂Ω).

PREFACE

xi

It turns out that the set of singularities of σ(t) is related to the so-called length spectrum LΩ of Ω. By definition, LΩ is the set of periods (lengths) of all periodic generalized geodesics in Ω. Let us mention that the generalized geodesics are the projections in Ω of the generalized bicharacteristics of the wave operator  = ∂t2 − Δx in T ∗ (R × Ω) defined by Melrose and Sjöstrand ([MS1], [MS2]). We refer to Chapter 1 for the precise definitions. The so-called Poisson relation for manifolds with boundary has the form sing supp σ(t) ⊂ {0} ∪ {T ∈ R : |T | ∈ LΩ }.

(0.7)

For strictly convex (concave) domains this relation has been established by Anderson and Melrose [AM]. Its proof for general domains is based on the results in [MS2] on the propagation of C ∞ singularities. A relation similar to (0.6) was first established for Riemannian manifolds without boundary. This was achieved independently by Chazarain [Ch2] and Duistermaat and Guillemin [DG]. Moreover, under certain assumptions on the periodic geodesics with period T , the leading singularity at T was examined in [DG]. It is natural to investigate the inverse inclusion in (0.7), however in the general case, very little is known so far. For certain strictly convex planar domains Ω Marvizi and Melrose [MM] found a sequence of closed billiard trajectories in Ω whose lengths belong to sing supp σ(t). It was expected ([Cl], [GM3]) that for generic strictly convex domains in R2 the inclusion (0.7) could become an equality. Such a result was established in [PS2] (see also [PSl]) for all generic domains (not necessarily convex). Its analogue in the case n > 2 is proved only for strictly convex domains [S3]. The results, just mentioned, form one of the main topics in this book. If the equality sing supp σ(t) = {0} ∪ {T : |T | ∈ LΩ }

(0.8)

holds for some domain Ω, then the lengths of the periodic geodesics in Ω can be considered as spectral invariants. From them one can determine various spectral invariants. The reader may consult [MM], [Cl], [Pol], [Po2], [Po3], [PoT], [HeZ] and [Z] for more information and further results in this direction. Let LΩ be the set of all periodic geodesics in Ω. For γ ∈ LΩ we denote by Tγ the period (length) of γ. There are three types of elements of LΩ : periodic reflecting rays (i.e. closed billiard trajectories in Ω), closed geodesics on ∂Ω and periodic geodesics of mixed type, containing both linear segments in Ω and geodesic segments on ∂Ω. Amongst the periodic reflecting rays we will distinguish those without segments tangent to the boundary ∂Ω; such rays will be called ordinary. Similarly to the case of closed geodesics on ∂Ω, for each ordinary periodic reflecting ray γ one can naturally define a Poincaré map Pγ such that the spectrum spec (Pγ ) of the linearization Pγ of Pγ contains certain information about the behaviour of billiard flow along γ. Such a ray γ will be called non-degenerate if 1 ∈ / spec Pγ . Poincaré maps for periodic reflecting rays are defined in Chapter 2.

xii

PREFACE

Given a smooth submanifold X of Rn , we denote by C ∞ (X, Rn ) the space of all smooth maps f : X → Rn , endowed with the Whitney C ∞ topology ∞ (see Chapter 1). Let C(X) = Cemb (X, Rn ) be its subspace consisting of all smooth n embedding of X into R . Being open in C ∞ (X, Rn ), C(X) is a Baire space, so every residual (countable intersection of open dense subsets) subset of C(X) is dense in it. Throughout the book we will consider very often the situation when Ω is a compact domain with smooth boundary ∂Ω and X = ∂Ω. Then for every f ∈ C(X) there exists a unique compact domain Ωf in Rn with boundary ∂Ωf = f (X) = f (∂Ω). Let us note that if Ω is strictly convex, the set O(Ω) of those f ∈ C(X) such that Ωf is strictly convex, is open in C(X), and so it is a Baire topological space, too. If Ω is a domain in Rn with bounded complement, for f ∈ C(X) we denote by Ωf the unbounded domain in Rn with ∂Ωf = f (X). In the following we sometimes say that a property is generically satisfied (briefly a generic property) in some classes of objects, say for the compact domains in Rn with smooth boundaries. By this we mean a property S such that for every bounded domain with smooth boundary X = ∂Ω there exists a residual subset R of C(X) such that Ωf has the property S for every f ∈ R. In the same way considering residual subsets of O(Ω), one can talk about generic properties of the strictly convex domains, etc. Let us note that in the whole book ‘smooth’ means C ∞ (although many separate arguments work replacing C ∞ by C k for some k ≥ 1). By a domain we always mean a domain with smooth boundary. Exploiting the Multijet Transversality Theorem (see Section 1.1), we establish that the following properties of the compact domains in Rn are generic: (I) Tγ /Tδ ∈ / Q for all periodic ordinary reflecting rays γ and δ such that neither of them is a multiple of the other. (II) Every periodic reflecting ray in Ω is ordinary and non-generate. As a consequence of this, it is established that the asymptotic (0.4) holds for generic domains Ω ⊂ Rn . Using (i) and (ii), we prove (0.8) for generic strictly convex domains in the plane. In fact, if Ω has the properties (i) and (ii), then each periodic reflecting ray in Ω has a period √ Tγ which is an isolated point in LΩ . The kernel E(t, x, y) of the operator cos(t A) satisfies the equality  E(t, x, x)dx.

σ(t) = Ω

One can compute the leading singularity of σ(t) for t close to Tγ by the Poisson summation formula discussed in Chapter 4. This leads to (0.8), since by (i) the singularities, related to different periodic rays, cannot be cancelled. In general, a domain Ω ⊂ R2 might admit periodic geodesics of mixed type. The analysis of the singularities of σ(t), related to the periods of such geodesics, leads to some rather difficult problems. We overcome this difficulty by showing that the following property is generic for domains Ω ⊂ R2 : (III) There are no periodic geodesics of mixed type in Ω.

PREFACE

xiii

The analysis of the generic properties, such as (i)–(iii), is the second main topic of this book. To establish (0.8) for generic convex domains in Rn , n ≥ 3, in Chapter 7 we prove an analogue of the classical bumpy metric theorem of Abraham–Klingenberg–Takens–Anosov, considering Riemannian metrics on X ⊂ Rn , induced by smooth embeddings of X into Rn . Our third topic concerns the kernel s(t − t , θ, ω) of the operator S − Id : L2 (R × Sn−1 ) → L2 (R × Sn−1 ). Here θ, ω ∈ Sn−1 , t, t ∈ R, and S is the scattering operator related to the Dirichlet problem for the wave operator  = ∂t2 − Δx in the exterior of a bounded obstacle K with smooth boundary ∂Ω = ∂K (see [LP1]). For fixed θ, ω ∈ Sn−1 the scattering kernel s(t, θ, ω) is a tempered distribution in S  (R). The Fourier transform Ft→λ s(t, θ, ω) with respect to t yields the scattering amplitude  a(λ, θ, ω) =

2π iλ

(n−1)/2 Ft→λ s(t, θ, ω).

It is well known that the scattering amplitude a(λ, θ, ω) determines uniquely the obstacle K (see for instance [LP1]). On the other hand, in the applications for given directions ω, θ is difficult to measure a(λ, θ, ω) for all λ ∈ R and we can measure only the singularities of s(t, θ, ω). It turns out that these singularities are related to sojourn times of generalized (ω, θ)-rays in Ω. These rays are generalized geodesics in Ω, incoming with direction ω and outgoing with direction θ. For such a ray γ the sojourn time was defined by Guillemin [G1] as an analogue of the notion of a period of a periodic geodesic; this notion appears also in the physical literature. The sojourn time measures the time which a point, moving along γ with a unit speed, spends near the obstacle K. For strictly convex obstacles K and fixed θ = ω one has sing supp t s(t, θ, ω) = { − Tγ }, γ being the unique (ω, θ)-ordinary reflecting ray in Ω (see [Ma2]). In general, the set L(ω,θ) (Ω) of all (ω, θ)-generalized rays in Ω could contain more than one element. Assuming that for (ω, θ) ∈ Sn−1 × Sn−1 every (ω, θ) ray γ in Ω is the projection of a uniquely extendible generalized bicharacteristic γ˜ of , we prove the inclusion sing supp t s(t, θ, ω) ⊂ { − Tγ : γ ∈ L(ω,θ) (Ω)},

(0.9)

which is called the Poisson relation for the scattering kernel. The above assumption for the (ω, θ) rays is fulfilled for generic obstacles as well as for generic directions, that is for (ω, θ) in a subset R of Sn−1 × Sn−1 whose complement has Lebesgue measure zero. We prove that the relation (0.9) becomes an equality for (θ, ω) ∈ R and also for generic obstacles in R3 and all directions θ = ω. For this purpose we study generic properties of (ω, θ)-rays, similar to (i)–(iii). Here the analogue of a periodic reflecting ray is an ordinary reflecting (ω, θ)-ray and that of Poincaré map is the so-called differential cross section dJγ of an ordinary reflecting (ω, θ)-ray.

xiv

PREFACE

The non-degeneracy of such a ray γ means that det dJγ = 0. The analogue of (iii) says that, given (θ, ω) ∈ R ⊂ Sn−1 × Sn−1 , there are no (ω, θ)-rays of mixed type in Ω. For an ordinary reflecting non-degenerate (ω, θ)-ray γ whose sojourn time Tγ is an isolated point in L(ω,θ) (Ω), we find the leading singularity of s(t, θ, ω) for t sufficiently close to −Tγ . To do this, as in the analysis of the singularities of σ(t) for t close to a period Tγ , we construct a global parametrix for the mixed problem by a global Fourier integral operator and we obtain a precise information about the principal symbol of this operator after multiple reflections. In this way the calculation of the singularity is reduced to the asymptotic of an oscillatory integral for which we apply the stationary phase argument. It turns out that the leading singularity of σ(t), as well as that of s(t, θ, ω), is given by some global geometric characteristics. This is the third main topic of this book. Similar to the length spectrum for bounded domains, the right-hand side of (0.9) contains certain information about the geometry of the obstacle K; we call it the scattering length spectrum (SLS) with respect to ω, θ. The sojourn times of the (ω, θ)-rays are easy to be observed and they form scattering data for the inverse scattering problems. The fourth main topic in this book concerns inverse scattering results. First, in Chapter 10 we study inverse scattering problems for obstacles K that are finite disjoint unions of several strictly convex domains. Under a geometric condition (H), introduced by M. Ikawa, a hyperbolic property of the billiard trajectories in the exterior Ω of the obstacles is established. This allows us to show that all periodic reflecting rays in Ω can be approximated by (ω, θ)-rays for appropriately fixed directions ω and θ and that their periods can be determined from the sojourn times of these rays. Also we find the asymptotic of the coefficients in front of the leading singularities of the scattering kernel, corresponding to the sojourn times of the approximating (ω, θ)-rays. A more general approach to the inverse problem of recovering information about an obstacle from the SLS is discussed in Chapter 13. It turns out that if two obstacles K and L have (almost) the same scattering length spectra, then the generalized geodesic flows in their exteriors are naturally conjugated on the non-trapping parts of their phase spaces via a time-preserving conjugacy. We use this result to show that certain properties of obstacles are recoverable from the SLS and also that some classes of obstacles can be uniquely recovered from their SLS. In this book we assume some knowledge of differential geometry, including basic facts in symplectic geometry, as well as some knowledge of differential topology. The analysis of the generalized bicharacteristics is based on several deep and important results from microlocal analysis and the calculus of global Fourier integral operators. We present a summary of known results in this area proving for convenience some of them in Chapter 1. On the other hand, in Chapter 11 we present detailed proofs of some new properties of the generalized bicharacteristics that are essentially used in Chapters 12 and 13. The main references for these results are the monographs of Hörmander [Hl]–[H4]. The reader might read these results informally, omitting their proofs, and then proceed to Chapters 2, 7–10. The first edition of this monograph was published in 1992 (see [PS7]). The present (second) edition is an improved version of the first. Various misprints and arguments

PREFACE

xv

have been corrected and several details added to the exposition. Apart from that, in the present edition Chapters 11–13 are entirely new. These chapters contain several results established after 1992 which could be also of independent interest. Most of the publications cited in the References concern inverse spectral results for manifolds with boundary and inverse scattering results related to the singularities of the scattering kernel. It was not possible and we have not even attempted, to cover the immense range of works devoted to inverse spectral and inverse scattering results.

Acknowledgements We are indebted to numerous colleagues for their assistance and for the discussions during the work of this book. Special thanks are due to V. Kovachev, G. Popov, Yu. Safarov, J. Sjöstrand and Pl. Stefanov. We are grateful to the following publishing companies and societies for permitting us to include parts of papers of ours in this book : the American Mathematical Society for [PS5]; Cambridge University Press for [PS4] and [S1], as well as for Figures 2.1, 2.2, 2.3 and 2.4 which appeared as Figures 1, 2, 3 and 4 in [S1]; the John Hopkins University Press for [PS2]; Springer-Verlag for [PS3], Elsevier and Annales Scientiques de l Ecole Normale Supèrieure for [PS9] and [S6], and Elsevier and Journal of Functional Analysis for [S7].

1

Preliminaries from differential topology and microlocal analysis Here we collect some facts concerning manifolds of jets, spaces of smooth maps and transversality, as well as some material from microlocal analysis. A special emphasis is given to the definition and main properties of the generalized bicharacteristics of the wave operator and the corresponding generalized geodesics.

1.1

Spaces of jets and transversality theorems

We begin with the notion of transversality, manifolds of jets and spaces of smooth maps. The reader is referred to Golubitsky and Guillemin [GG] or Hirsch [Hir] for a detailed presentation of this material. In this book smooth means C ∞ . Let X and Y be smooth manifolds and let f : X −→ Y be a smooth map. Given x ∈ X, we will denote by Tx f the tangent map of f at x. Sometimes we will use the notation dx f = Tx f . If rank(Tx f ) = dim(X) ≤ dim(Y ) (resp. rank(Tx f ) = dim(Y ) ≤ dim(X)), then f is called an immersion (resp. a submersion) at x. Let W be a smooth submanifold of Y . We will say that f is transversal to W at x ∈ X,  x W , if either f (x) ∈ and will denote this by f  / W or f (x) ∈ W and Im(Tx f ) + Tf (x) W = Tf (x) Y . Here for every y ∈ W we identify Ty W with its image under the map Ty i : Ty W −→ Ty W , where i : W −→ Y is the inclusion. Clearly, if f is a  for every submanifold W of Y . If Z ⊂ X and f W  submersion at x, then f W Geometry of the Generalized Geodesic Flow and Inverse Spectral Problems, Second Edition. Vesselin M. Petkov and Luchezar N. Stoyanov. © 2017 John Wiley & Sons, Ltd. Published 2017 by John Wiley & Sons, Ltd.

2

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

for every x ∈ Z, we will say that f is transversal to W on Z. Finally, if f is transversal  . to W on the whole X, we will say that f is transversal to W and write f W The next proposition contains a basic property of transversality that will be used several times throughout. Proposition 1.1.1: Let f : X −→ Y be a smooth map, and let W be a smooth  . Then f −1 (W ) is a smooth submanifold of X submanifold of Y such that f W with (1.1) codim (f −1 (W )) = codim (W ). In particular: (a) if dim(X) < codim (W ), then f −1 (W ) = ∅, that is f (X) ∩ W = ∅. (b) if dim(X) = codim (W ), then f −1 (W ) consists of isolated points in X. Consequently, if f is a submersion, then for every submanifold W of Y , f −1 (W ) is a submanifold of X with (1.1). Thus, in this case, f −1 (y) is a submanifold of X of codimension equal to dim(Y ) for every y ∈ Y . Let again X and Y be smooth manifolds and let x ∈ X. Given two smooth maps f, g : X −→ Y , we will write f ∼x g if dx f = dx g. For an integer k ≥ 2, we will dg for write f ∼kx g if for the smooth maps df, dg : T X −→ T Y , we have df ∼k−1 ξ every ξ ∈ Tx X. In this way by induction one defines the relation f ∼kx g for all integers k ≥ 1. Fix for a moment x ∈ X and y ∈ Y . Denote by Jk (X, Y )x,y the family of all equivalence classes of smooth maps f : X −→ Y with f (x) = y with respect to the equivalence relation ∼kx . Define the space of k-jets by  J k (X, Y ) = J k (X, Y )x,y . (x,y)∈X×Y

So, for each k-jet σ ∈ J k (X, Y ), there exist x ∈ X and y ∈ Y σ ∈ J k (X, Y )x,y . We set α(σ) = x and β(σ) = y, thus obtaining maps α : J k (X, Y ) −→ X,

β : J k (X, Y ) −→ Y ,

with

(1.2)

called the source and the target map, respectively. Given an arbitrary smooth f : X −→ Y , let (1.3) j k f : X −→ J k (X, Y ) be the map assigning to every x ∈ X the equivalence class j k f (x) of f in J k (X, Y )x,f (x) . There is a natural structure of a smooth manifold on J k (X, Y ) for every k. We refer the reader to [GG] or [Hir] for its description and main properties. Let us only mention that with respect to this structure for every smooth map f the maps (1.2) and (1.3) are also smooth.

DIFFERENTIAL TOPOLOGY AND MICROLOCAL ANALYSIS

3

For a non-empty set A and an integer s ≥ 1, define A(s) = {(a1 , . . . , as ) ∈ As : ai = aj , 1 ≤ i < j ≤ s}. Note that if A is a topological space, then A(s) is an open (dense) subset of the product space As . If f : A −→ B is an arbitrary map, define f s : As −→ B s by f s (a1 , . . . , as ) = (f (a1 ), . . . , f (as )), Let X and Y be smooth manifolds, let s and k be natural numbers and let αs : (J k (X, Y ))s −→ X s . The open submanifold Jsk (X, Y ) = (αs )−1 (X (s) ) of (J k (X, Y ))s is called an s-fold k-jet bundle. For a smooth f : X −→ Y , define the smooth map jsk f : X (s) −→ Jsk (X, Y ) by jsk f (x1 , . . . , xs ) = (j k f (x1 ), . . . , j k f (xs )). We will now define the Whitney C k topology on the space C ∞ (X, Y ) of all smooth maps from X into Y . Let k ≥ 0 be an integer and let U be an open subset of J k (X, Y ). Set M (U ) = {f ∈ C ∞ (X, Y ) : j k f (X) ⊂ U }. The family {M (U )}U , where U runs over the open subsets of J k (X, Y ), is the basis for a topology on C ∞ (X, Y ), called the Whitney C k topology. The supremum of all Whitney C k topologies for k ≥ 0 is called the Whitney C ∞ topology. It follows from this definition that fn → f as n → ∞ in the C ∞ topology if fn → f in the C k topology for all k ≥ 0. Note that if X is not compact (and dim(Y ) > 0), then any of the C k topologies (including the case k = ∞) does not satisfy the first axiom of countability, and therefore is not metrizable. On the other hand, if X is compact, then all C k topologies on C ∞ (X, Y ) are metrizable with complete metrics. In this book we always consider C ∞ (X, Y ) with the Whitney C ∞ topology. An important fact about these spaces, which will be often used in what follows, is that whenever X and Y are smooth manifolds, the space C ∞ (X, Y ) is a Baire topological space. Recall that a subset R of a topological space Z is called residual in Z if R contains a countable intersection of open dense subsets of Z. If every residual subset of Z is dense in it, then Z is called a Baire space. In some of the next chapters we will consider spaces of the form C ∞ (X, Rn ), X being a smooth submanifold of Rn for some n ≥ 2. Let us note that these spaces have a natural structure of Frechet spaces. Moreover, if X is compact, then C ∞ (X, Rn ) has a natural structure of a Banach space. Denote by ∞ (X, Rn ) C(X) = Cemb

4

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

the subset of C ∞ (X, Rn ) consisting of all smooth embeddings X −→ Rn . Then C(X) is open in C ∞ (X, Rn ) (cf. Chapter II in [Hir]), and therefore it is a Baire topological space with respect to the topology induced by C ∞ (X, Rn ). Finally, notice that for compact X the space C(X) has a natural structure of a Banach manifold. We refer the reader to [Lang] for the definition of Banach manifolds and their main properties. The following theorem is known as the multijet transversality theorem and will be used many times later in this book. Theorem 1.1.2: Let X and Y be smooth manifolds, let k and s be natural numbers and let W be a smooth submanifold of Jsk (X, Y ). Then  } TW = {F ∈ C ∞ (X, Y ) : jsk F W is a residual subset of C ∞ (X, Y ). Moreover, if W is compact, then TW is open in C ∞ (X, Y ). For s = 1, this theorem coincides with Thom’s transversality theorem. We conclude this section with a special case of the Abraham transversality theorem which will be used in Chapter 6. Now by a smooth manifold we mean a smooth Banach manifold of finite or infinite dimension (cf. [Lang]). Let A, X and Y be smooth manifolds, and let ρ : A −→ C ∞ (X, Y )

(1.4)

evρ : A × X −→ Y

(1.5)

be a map, A a → ρa . Define

by evρ (a, x) = ρ + a(x). The next theorem is a special case of Abraham’s transversality theorem (see [AbR]). Theorem 1.1.3: Let ρ have the form (1.4) and let W be a smooth submanifold of Y . (a) If the map (1.5) is C 1 and K is a compact subset of X, then  x W, x ∈ K} AK,W = {a ∈ A : ρa  is an open subset of A. (b) Let dim(X) = n < ∞, codim (W ) = q < ∞ and let r be a natural number with r > n − q. Suppose that the manifolds A, X and Y satisfy the second axiom of  . Then countability, the map (1.5) is C r and evρ W  } AW = {a ∈ A : ρa W is a residual subset of A.

DIFFERENTIAL TOPOLOGY AND MICROLOCAL ANALYSIS

1.2

5

Generalized bicharacteristics

Our aim in this section is to define the generalized bicharacteristics of the wave operator  = ∂t2 − Δx and to present their main properties which will be used throughout the book. Here we use the notation from Section 24 in [H3]. In what follows Ω is a closed domain in Rn+1 with a smooth boundary ∂Ω. Given a point on ∂Ω, we choose local normal coordinates x = (x1 , . . . , xn+1 ),

ξ = (ξ1 , . . . , ξn+1 )

in T ∗ (Rn+1 ) about it such that the boundary ∂Ω is given by x1 = 0 and Ω is locally defined by x1 ≥ 0. We assume that the coordinates ξi are those dual to xi . The coordinates x, ξ can be chosen so that the principal symbol of  has the form p(x, ξ) = ξ12 − r(x, ξ  ), where x = (x2 , . . . , xn+1 ),

ξ  = (ξ2 , . . . , ξn+1 ),

and r(x, ξ  ) is homogeneous of order 2 in ξ  . Introduce the sets Σ = {(x, ξ) ∈ T ∗ Rn+1 \ {0} : p(x, ξ) = 0}, Σ0 = {(x, ξ) ∈ T ∗ Rn+1 : x1 > 0}, H = {(x, ξ) ∈ Σ : x1 = 0, r(0, x , ξ  ) > 0}, G = {(x, ξ) ∈ Σ : x1 = 0, r(0, x , ξ  ) = 0}. The sets Σ, H and G are called the characteristic set, the hyperbolic set and the glancing set, respectively. Let r0 (x , ξ  ) = r(0, x , ξ  ),

r1 (x , ξ  ) =

∂r (0, x , ξ  ). ∂x1

The diffractive and the gliding sets are defined by Gd = {(x, ξ) ∈ G : r1 (x , ξ  ) > 0}, Gg = {(x, ξ) ∈ G : r1 (x , ξ  ) < 0}, respectively.

6

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Next, consider the Hamiltonian vector fields  n+1   ∂p ∂ ∂p ∂ Hp = · − ∂x · ∂ξ , ∂ξj ∂xj j j j=1 Hr0 =

n+1   ∂r

∂r ∂ ∂ · − ∂x0 · ∂ξ ∂ξj ∂xj j j 0

j=1

 .

Notice that dξ p(x, ξ) = 0 on Σ and dξ r0 (x, ξ  ) = 0 on G, so Hp and Hr0 are not radial on Σ and G, respectively. Next, introduce the sets (r1 ) = 0}, Gk = {(x, ξ) ∈ G : r1 = Hr0 (r1 ) = . . . = Hrk−3 0 G∞ =

∞ 

k ≥ 3,

Gk .

k=3

The above definitions are independent of the choice of local coordinates. Let us mention that if ∂Ω is given locally by ϕ = 0 and Ω by ϕ > 0, ϕ being a smooth function, then H = {(x, ξ) ∈ T ∗ (R × Ω) : p(x, ξ) = 0, Hp ϕ(x, ξ) = 0}, G = {(x, ξ) ∈ T ∗ (R × Ω) : p(x, ξ) = 0, Hp ϕ(x, ξ) = 0}, Gd = {(x, ξ) ∈ G : Hp2 ϕ(x, ξ) > 0}, Gg = {(x, ξ) ∈ G : Hp2 ϕ(x, ξ) < 0}, Gk = {(x, ξ) ∈ G : Hpj ϕ(x, ξ) = 0, 0 ≤ j < k}. We define the generalized bicharacteristics of  using the special coordinates (x, ξ) chosen above. Definition 1.2.1: Let I be an open interval in R. A curve γ : I −→ Σ

(1.6)

is called a generalized bicharacteristic of  if there exists a discrete subset B of I such that the following conditions hold: (i) If γ(t0 ) ∈ Σ0 ∪ Gd for some t0 ∈ I \ B, then γ is differentiable at t0 and d γ(t ) = Hp (γ(t0 )). dt 0 (ii) If γ(t0 ) ∈ G \ Gd for some t0 ∈ I \ B, then γ(t) = (x1 (t), x (t), ξ1 (t), ξ  (t)) is differentiable at t0 and dx1 dξ (t ) = 1 (t0 ) = 0, dt 0 dt

d  (x (t), ξ  (t))|t=t0 = Hr0 (γ(t0 )). dt

DIFFERENTIAL TOPOLOGY AND MICROLOCAL ANALYSIS

7

(iii) If t0 ∈ B, then γ(t0 ) ∈ Σ0 for all t = t0 , t ∈ I, with |t − t0 | sufficiently small. Moreover, for ξ1± (x , ξ  ) = ± r0 (x , ξ  ), we have lim

t→t0 ,±(t−t0 )>0

γ(t) = (0, x (t0 ), ξ1± (x (t0 )), ξ  (t0 )) ∈ H.

This definition does not depend on the choice of the local coordinates. Note that when ∂Ω is given by ϕ = 0 and Ω by ϕ > 0, then the condition (ii) means that if γ(t0 ) ∈ G \ Gd , then dγ (γ(t0 )) = HpG (γ(t0 )), dt where HpG = Hp +

Hp2 ϕ H Hϕ2 p ϕ

is the so-called glancing vector field on G. It follows from the above definition that if (1.6) is a generalized bicharacteristic, the functions x(t), ξ  (t), |ξ1 (t)| are continuous on I, while ξ1 (t) has jump discontinuities at any t ∈ B. The functions x (t) and ξ  (t) are continuously differentiable on I and ∂r ∂r dξ  dx = − , = . (1.7) dt ∂ξ dt ∂x Moreover, for t ∈ B, x1 (t) admits left and right derivatives x (t ± ) − x1 (t) d± x1 (t) = lim ± 1 = 2ξ1 (t ± 0). →+0 dt 

(1.8)

The function ξ1 (t) also has a left derivative and a right derivative. For γ(t) ∈ / Gg , we have ∂r ξ (t ± ) − ξ1 (t) d± ξ1 (t) = lim ± 1 = (x(t), ξ  (t)), →+0 dt  ∂x1

(1.9)

±

while ddtξ1 (t) = 0 for γ(t) ∈ Gg . Thus, if γ(t) remains in a compact set, then the functions x(t), ξ  (t), ξ12 (t) and x1 (t)ξ1 (t) satisfy a uniform Lipschitz condition. For the left and right derivatives of |ξ1 (t)|, one gets  ±     d |ξ1 (t)|   ∂r    ≤ . (x(t), ξ (t)) (1.10)     dt ∂x1 Melrose and Sjöstrand [MS2] (see also Section 24 in [H3]) showed that for each z0 ∈ Σ, there exists a generalized bicharacteristic (1.6) of  with γ(t0 ) = z0 for some t0 ∈ I. Since the vector fields Hp and HpG are not radial on Σ and G, respectively, such a bicharacteristic γ can be extended for all t ∈ R. However, in general, γ is not unique. We refer the reader to [Tay] or [H3] for examples demonstrating this.

8

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

For ρ ∈ Σ, denote by Ct (ρ) the set of those μ ∈ Σ such that there exists a generalized bicharacteristic (1.6) with 0, t ∈ I, γ(0) = ρ and γ(t) = μ. In many cases Ct (ρ) is related to a uniquely determined bicharacteristic γ. In the general case it is convenient to introduce the following. Definition 1.2.2: A generalized bicharacteristic γ : R −→ Σ of  is called uniquely extendible if for each t ∈ R, the only generalized bicharacteristics (up to a change of parameter) passing through γ(t) is γ. That is, for ρ = γ(0), we have Ct (ρ) = {γ(t)} for all t ∈ R. It was proved by Melrose and Sjöstrand [MS1] that if Im(γ) ⊂ Σ \ G∞ , then γ is uniquely extendible. If z0 = γ(t0 ) ∈ H for some t0 ∈ B, then γ(t) meets ∂Ω transversally at x(t0 ) and (iii) holds. For z0 ∈ Σ0 ∪ Gd we have γ(t) ∈ Σ0 for |t − t0 | small enough, while in the case z0 ∈ Gg for small |t − t0 |, γ(t) coincides with the gliding ray (1.11) γ0 (t) = (0, x (t), 0, ξ  (t)), where (x (t), ξ(t)) is a null bicharacterstic of the Hamiltonian vector field Hr0 . To discuss the local uniqueness of generalized bicharacteristics, let γ(t) = (x(t), ξ(t)) be such a bicharacteristic and let y  (t), η  (t)) be the solution of the problem ⎧  ∂r dy ⎪ ⎪ (t) = 0 (y  (t), η  (t)), ⎪ ⎪ dt ∂ξ ⎪ ⎨  ∂r dη (1.12) (t) = − 0 (y  (t), η  (t)), ⎪ ⎪ ∂x ⎪ dt ⎪ ⎪ ⎩  y (0) = x (0), η  (0) = ξ  (0). Then setting e(t) = r1 (y  (t), η  (t)), we have the following local description of γ. Proposition 1.2.3: Let γ(0) ∈ G3 . If e(t) increases for small t > 0, then for such t the bicharateristic γ(t) is a trajectory of Hp . If e(t) decreases for 0 ≤ t ≤ T , then for such t, γ(t) is a gliding ray of the form (1.11). A proof of this proposition and some other properties of generalized bicharacteristics can be found in Section 24.3 in [H3]. It should be mentioned that for k ≥ 3 and γ(0) ∈ Gk \ Gk+1 , we have e(t) =

1 H k ϕ(γ(0)) tk−2 + O(tk−1 ), 2(k − 2)! p

therefore the sign of Hpk ϕ(γ(0)) determines the local behaviour of e(t). Corollary 1.2.4: In each of the following cases, every generalized bicharacteristic of  is uniquely extendible: (a) the boundary ∂Ω is a real analytic manifold;

DIFFERENTIAL TOPOLOGY AND MICROLOCAL ANALYSIS

9

(b) there are no points y ∈ ∂Ω at which the normal curvature of ∂Ω vanished of infinite order in some direction ξ ∈ Ty (∂Ω); (c) ∂Ω is given locally by ϕ = 0 and Hp2 ϕ(z) ≤ 0

(1.13)

for every z ∈ G. If ∂Ω is locally convex in the domain of ϕ, then (1.13) holds. Proof: In the case (a) the symbols r0 (x , ξ  ) and r1 (x , ξ  ) are real analytic, so the solution (y  (t), η  (t)) of (1.12) is analytic in t. Consequently, the function e(t) is analytic and we can use its Taylor expansion in order to apply Proposition 1.2.3. In the case (c), using the special coordinates x, ξ, and combining (1.13) with ± (1.9), we get ddtξ1 (t) ≥ 0. On the other hand, if ξ1 (t) has a jump at γ(t) ∈ H, then this jump is equal to 2r0 (x (t), ξ  (t)) > 0. Thus, the function ξ1 (t) is increasing. If e(t) = 0 for 0 ≤ t ≤ t0 , we get x1 (t) = ξ1 (t) = 0 for such t, so {γ(t) : 0 ≤ t ≤ t1 } is a gliding ray. Assume that there exists a sequence tk  0 such that e(tk ) = 0 for all k ≥ 1. Then ξ1 (t) > 0 for all sufficiently small t > 0. Now (1.8) shows that x1 (t) is increasing for such t, therefore there is t > 0 such that {γ(t) : 0 ≤ t ≤ t } coincides with a trajectory

of Hp . 2 and let ϕ depend on x1 , . . . , xn only. Then Let p = nj=1 ξj2 − ξn+1 (Hp2 ϕ)(x, ξ) = 4

n 

∂2ϕ (x)ξi ξj , ∂xi ∂xj i,j=1

and if the boundary ∂Ω is locally convex, we obtain (1.13). Finally, in the case (b), for each x ∈ ∂Ω there exists a multi-index α, depending on x, such that (∂ α ϕ)(x) = 0. This implies G∞ = ∅, which completes the proof.  According to Lemma 6.1.2, in the generic case discussed in Chapter 6 the assumption (b) is always satisfied. Let Q = Ω × R. We will again use the coordinates x = (x1 , . . . , xn+1 ), this time denoting the last coordinate by t, that is t = xn+1 . For x ∈ ∂Q = ∂Ω × R, let Nx (∂Q) be the space of covectors ξ ∈ Tx∗ Q vanishing on Tx (∂Q). Define the equivalence relation ∼ on T ∗ Q by (x, ξ) ∼ (y, η) if and only if either x = y ∈ Q \ ∂Q and ξ = η, or x = y ∈ ∂Q and ξ − η ∈ Nx (∂Q). Then T ∗ Q/ ∼ can be naturally identified over ∂Q with T ∗ (∂Q). Consider the map ∼: T ∗ Q (x, ξ) → (x, ξ|Tx (∂Q) ) ∈ T ∗ (∂Q),  = Σ is called the compressed chardefined as the identity on T ∗ (Q \ ∂Q). Then Σ b acteristic set, while the image γ˜ of a bicharacteristic γ under ∼ is called a compressed generalized bicharacteristic. Clearly γ˜ is a continuous curve in Σb .

10

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Given ρ = (x, ξ), μ = (y, η) ∈ T ∗ Q, denote by d(ρ, μ) the standard Euclidean distance between ρ and μ. For ρ, μ ∈ Σ define D(ρ, μ) =

inf

ν  ,ν  ∈Σ,ν∼ν 

(min{d(ρ, μ), d(ρ, ν  ) + d(ν  , μ)}).

Clearly, D(ρ, μ) = 0 if and only if ρ ∼ μ, and D(ρ, μ) = D(ρ , μ ) provided ρ ∼ ρ and μ ∼ μ . It is easy to check that D is symmetric and satisfies the triangle inequality. Thus, D is a pseudo-metric on Σ, which induces a metric on Σb . For the next lemma we assume that I is a closed non-trivial interval in R, (y0 , η0 ) ∈ Σ and Γ is a neighbourhood of (y0 , η0 ) in Q. Lemma 1.2.5: There exists a constant C0 > 0 depending only on Γ and I such that for every generalized bicharacteristic γ : I −→ Σ ∩ γ we have D(γ(t), γ(s)) ≤ C0 |t − s| for all t, s ∈ I. Proof: It is enough to consider the case when |t − s| is small. Then we can use the local coordinates introduced earlier. From (1.7), (1.8) and (1.10), we get |x(t) − x(s)| + |ξ  (t) − ξ  (s)| ≤ C1 |t − s|,

||ξ1 (t)| − |ξ1 (s)|| ≤ C1 |t − s|,

where C1 > 0 is a constant independent of t and s. Thus, if ξ1 (t) = 0 or ξ1 (s) = 0 we / ∂Ω get |ξ1 (t) − ξ1 (s)| ≤ C1 |t − s|. The latter holds also in the case when γ(t ) ∈ for all t ∈ (t, s). Consequently, D(γ(t), γ(s)) ≤ C2 |t − s| whenever either ξ1 (t)ξ1 (s) = 0 or γ(t ) ∈ ∂Ω only for finitely many t ∈ (t, s). Assume that there are infinitely many t ∈ (t, s) such that γ(t ) is a reflection point of γ. Then there exists u ∈ [t, s] with γ(u) ∈ G. Hence, D(γ(t), γ(u)) ≤ C2 |t − u|,

D(γ(u), γ(s)) ≤ C2 |u − s|,

and using the triangle inequality for D, we complete the proof of the assertion.



The next lemma shows that any sequence of generalized bicharacteristics has a subsequence that is convergent on a given compact interval. Lemma 1.2.6: Let I = [a, b] be a compact interval in R, let K be a compact subset of Σ and let γ (k) (t) = (x(k) (t), ξ (k) (t)) : I −→ K ⊂ Σ be a generalized bicharacteristic of  for every natural number k. Then there exists an infinite sequence k1 < k2 < . . . of natural numbers and a generalized bicharacteristic γ(t) = (x(t), ξ(t)) : I −→ Σ such that (1.14) lim D(γ (km ) (t), γ(t)) = 0 m→∞

for all t ∈ I.

DIFFERENTIAL TOPOLOGY AND MICROLOCAL ANALYSIS

11

Proof: Using local coordinates, we see that the derivatives of (x(k) ) (t) and (ξ (k) ) (t) (k) (k) and the left and right derivatives of x1 (t) and ξ1 (t) are uniformly bounded for (k) (k) (k) t ∈ I and k ≥ 1. Hence the maps x(k) (t), (ξ (k) ) (t), x1 (t)ξ1 (t) and (ξ1 (t))2 are uniformly Lipschitz, which implies that there exists an infinite sequence k1 < k2 < (k ) . . . of natural numbers such that the sequences x(km ) (t), (ξ (km ) ) (t), (ξ1 m (t))2 and (km ) (km ) x1 (t), ξ1 (t) are uniformly convergent for t ∈ I. It now follows from Proposition 24.3.12 in [H3] that there exists a generalized bicharacteristic γ(t) : I → Σ of  such hat (1.15) lim γ (km ) (t) = γ(t) m−→∞

for all t ∈ I with γ(t) ∈ / H. Let t ∈ I be such that γ(t ) is a reflection point of γ. Then there exists a sequence tj → t with γ(tj ) ∈ Σ0 ∪ G for all j. Thus, D(γ (km ) (t ), γ(t )) ≤ D(γ (km ) (t ), γ (km ) (tj )) + D(γ(tj ), γ(t )) + D(γ (km ) (tj ), γ(tj )). By Lemma 1.2.5, the first two terms in the right-hand side can be estimated uniformly with respect to m, while for the third term we can use (1.15). Taking j and m  sufficiently large, we obtain (1.14), which proves the lemma. In what follows we will use local coordinates (t, x) ∈ R × Ω and the corresponding local coordinates (t, x; τ, ξ) ∈ T ∗ (R × Ω). In these coordinates the principal symbol p of  has the form p(x, τ, ξ) = ξ12 − q2 (x, ξ  ) − τ 2 , where ξ  = (ξ2 , . . . , ξn ) and q2 (x, ξ  ) is homogeneous of order 2 in ξ  . Consequently, the vector fields Hp and HpG do not involve derivatives with respect to τ , so by Definition 1.2.1, the variable τ remains constant along each generalized bicharacteristic. This makes it possible to parametrize every generalized bicharacteristic by the time t. Given (y, η) ∈ T ∗ (Ω) \ {0}, consider the points μ± = (0, y, ∓|η|, η) ∈ Σ. Assume that locally ∂Ω is given by x1 = 0 and Ω by x1 ≥ 0. Let μ+ be a hyperbolic point and let ξ1± (y  , η) be the different real roots of the equation p(0, y  , |η|, z, η  ) = 0 with respect to z. Denote by γ the generalized bicharacteristic parameterized by a parameter s such that lim γ(s) = μ+ . s0

12

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Then τ = −|η| < 0 along γ and the time t increases when s increases. Such a bicharacteristic will be called forward. For the right derivative of x1 (t) we get d+ x1 /ds ξ (+0) d+ x1 = = 1 > 0, dt dt/ds −τ since for small t > 0, γ(t) enters the interior of Ω and x1 (t) > 0. Therefore, setting  ξ1± (y  , η) = ± |η|2 + q2 (0, y  , η  ), we find

lim ξ1 (s) = ξ1+ (y  , η).

s0

In the case μ+ ∈ G it may happen that there exist several forward bicharacteristic passing through μ+ . Denote by C+ the set of those (t, x, y; τ, ξ, η) ∈ T ∗ (R × Ω × Ω) \ {0} such that τ = −|ξ| = −|η| and (t, x, τ, ξ) and (0, y, τ, η) lie on forward generalized bicharacteristics of . In a similar way we define C− using a backward bicharacteristic, determined as the forward ones replacing μ+ by μ− . The set C = C+ ∪ C− is called the bicharacteristic relation of . If μ = (0, y, τ, η) ∈ H and τ < 0 (resp. τ > 0), we will say that μ is a reflection point of a forward (resp. backward) bicharacteristic. Similarly, if ρ = (t, x, τ, ξ) ∈ H, then ρ is a reflection point of a generalized bicharacteristic passing through (0, y, τ, η), and, working in local coordinates as before, the sign of τ determines uniquely ξ1 (t + 0). The sets C± and C are homogeneous with respect to (τ, ξ, η), that is (t, x, y, τ, ξ, η) ∈ C± implies (t, x, y, sτ, sξ, sη) ∈ C± for all s ∈ R+ . Lemma 1.2.7: The sets C± are closed in T ∗ (R × Ω × Ω) \ {0}. Proof: Since C+ is homogeneous, it is sufficient to show that if C+ zk = (tk , xk , yk , −1, ξk , ηk ),

|ξk | = |ηk | = 1

for all k ≥ 1 and there exists lim zk = z0 = (t0 , x0 , y0 , −1, ξ0 , η0 ),

k→∞

then z0 ∈ C+ . Let γ (k) (t) be a generalized bicharacteristic of  such that (tk , xk , −1, ξk ) and (0, yk , −1, ηk ) lie on Im(γ (k) ). If one of these points belongs to H, we consider it as a reflection point of γ (k) , according to the above-mentioned (k) convention by suitably choosing ξ1 (t). Assume |tk | ≤ T . Then there exists a (k) compact set K ⊂ Σ such that γ (t) ∈ K for all |t| ≤ T , so we can apply the argument in the proof of Lemma 1.2.6. Consequently, there exists an infinite

DIFFERENTIAL TOPOLOGY AND MICROLOCAL ANALYSIS

13

sequence k1 < k2 < . . . of natural numbers and a generalized bicharacteristic γ satisfying (1.14) and (1.15). Then for the Euclidean distance d we find d(γ (km ) (tkm ), γ(t0 )) ≤ d(γ (km ) (tkm ), γ km ) (t0 )) + d(γ (km ) (t0 ), γ(t0 )). If γ(t0 ) ∈ Σ0 ∪ G, according to (1.15) and the continuity of x(t), ξ  (t) and |ξ1 (t)|, we get (1.16) d(γ (km ) (tkm ), γ(t0 )) → 0 as m → ∞, which shows that z0 ∈ C+ . If γ(t0 ) ∈ H, then by our convention, (k ) ξ1 (t + 0) and ξ1 m (t + 0) have the same sign for large m, which implies z0 ∈ C+ . Therefore, C+ is closed. In the same way one proves that C− is closed as  well. Using C+ we now define the so-called generalized Hamiltonian flow Ft of ; it is sometimes called the broken Hamiltonian flow. Given (y, η) ∈ T ∗ Ω \ {0}, set Ft (y, η) = {(x, ξ) ∈ T ∗ Ω \ {0} : (t, x, y, −|η|, ξ, η) ∈ C+ }. In general, Ft (y, η) is not a one-point set. Nevertheless, setting Ft (V ) = {Ft (y, η) : (y, η) ∈ V } for V ⊂ T ∗ Ω \ {0}, we have the group property Ft+s (y, η) = Ft (Fs (y, η)). The flow generated by C− is Ft (y, −η). Let ∂Ω be locally given by x1 = 0 and let p(x, τ, ξ) = ξ12 − q2 (x, ξ  ) − τ 2 be the principal symbol of . A point σ = (t, x , τ, ξ  ) ∈ T ∗ (R × ∂Ω) \ {0} is called hyperbolic (resp. glancing) for  if the equation p(0, x , τ, ξ1 , ξ  ) = 0

(1.17)

with respect to ξ1 has two different real roots (resp. a double real root). These definitions are invariant with respect to the choice of the local coordinates. If (1.17) has no real roots, then σ is called an elliptic point. Clearly, the set of hyperbolic points is open in T ∗ (R × ∂Ω), while that of the glancing points is closed. Let π : T ∗ (R × Ω) −→ Ω be the natural projection, π(t, x, τ, ξ) = x.

14

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Definition 1.2.8: A continuous curve g : [a, b] −→ Ω is called a generalized geodesic in Ω if there exists a generalized bicharacteristic γ : [a, b] −→ Σ such that g(t) = π(γ(t)),

t ∈ [a, b].

(1.18)

Notice that, in general, a generalized geodesic is not uniquely determined by a point on it and the corresponding direction. If the generalized bicharacteristic γ with (1.18) satisfies γ(t) ∈ Σ0 ∪ H, t ∈ [a, b], we will say that g (or Im(g)) is a reflecting ray in Ω. Two special kinds of such rays will be studied in detail in Chapter 2. One of them is defined as follows. Definition 1.2.9: A point (x, ξ) ∈ T ∗ Ω \ {0} is called periodic with period T = 0 if (T, x, x, ±|ξ|, ξ, ξ) ∈ C. A generalized bicharacteristic γ(t) = (t, x(t), τ, ξ(t)) ∈ Σ, t ∈ R, will be called periodic with period T = 0 if for each t ∈ R the point (x(t), ξ(t)) is periodic with period T . The projections on Ω of the periodic generalized bicharacteristics of  are called periodic generalized geodesics. Notice that if (T, x, x, −|ξ|, ξ, ξ) ∈ C+ , then (T, x, x, |ξ|, −ξ, −ξ) ∈ C− , since we can change the orientation on the bicharacteristic passing through (0, x, −|ξ|, ξ). A uniquely extendible bicharacteristic γ is periodic provided Im(γ) contains a periodic point. If T is the period of a generalized geodesic g, then |T | coincides with the standard length of the curve Im(g). Let LΩ be the set of all periodic generalized geodesics in Ω. For g ∈ LΩ we denote by Tg the length of Im(g). We call length spectrum the following set LΩ = {Tg : g ∈ LΩ }. Lemma 1.2.10: The set LΩ is closed in R and 0 ∈ / LΩ . Proof: Consider a convergent sequence {Tk } of elements of LΩ converging to some T0 ∈ R as k → ∞. Then for every k ≥ 1 there exists a generalized bicharacteristic γ (k) of  with period Tk passing through a point of the form (0, xk , −1, ξk ). If T0 = 0, choosing a subsequence as in the proof of Lemma 1.2.7, we obtain T0 ∈ LΩ . It remains to show that the case T0 = 0 is impossible. Assume T0 = 0. Passing to an appropriate subsequence, we may assume that there exists limk→∞ (xk , ξk ) = (x0 , ξ0 ) and for every t there exists lim γ (k) (t) = lim (t, x(k) (t), −1, ξ (k) (t)) = γ0 (t) = (t, x0 (t), −1, ξ0 (t)),

k→∞

k→∞

DIFFERENTIAL TOPOLOGY AND MICROLOCAL ANALYSIS

15

provided γ0 (t) ∈ / H and |t| ≤ T . If x0 is in the interior of Ω, then xk is also in the interior of Ω for large k. Then for such k, x(k) (t) is in the interior of Ω for sufficiently small t > 0, which is a contradiction. If there exists t with |t | ≤ T and x0 (t ) in the interior of Ω, then we get a contradiction by the same argument. It remains to consider the case when γ0 (t) ∈ G for all t ∈ [−T, T ]. Then for such t, γ0 (t) = (x0 (t), ξ0 (t)) is an integral curve of the glancing vector field HpG . Since the latter is not radial, γ0 (t) has no stationary points for t ∈ [−T, T ]. Given a small neigh/U bourhood U of x0 in ∂Ω, there exist δ0 , δ1 such that 0 < δ0 < δ1 ≤ T and x0 (t) ∈ for δ0 ≤ |t| ≤ δ1 . Since x(k) (t) → x0 (t) as k → ∞ uniformly for |t| ≤ T , for sufficiently large k there exists a natural number mk with δ0 ≤ mk Tk ≤ δ1 ,

x(k) (Tk ) = x(k) (mk Tk ).

/ U , which is a contradiction. This Then x0 = limk→∞ xk = limk→∞ x(k) (Tk ) ∈  proves that T0 = 0 and this completes the proof of the proposition.

1.3

Wave front sets of distributions

In this section we collect some basic facts concerning wave fronts of distributions. For more details, we refer the reader to the books of Hörmander [Hl], [H3]. Let X be an open subset of Rn and let D (X) be the space of all distributions on X. The singular support sing supp(u) of u ∈ D (X) is a closed subset of X such / sing supp(u) there exists an open neighbourhood U of x0 in X and a that if x0 ∈ smooth function f ∈ C ∞ (U ) such that  u, ϕ =

f (x)ϕ(x) dx,

ϕ ∈ C0∞ (U ).

For a more precise analysis of sing supp(u), it is useful to consider the directions ξ ∈ Rn \ {0} along which the Fourier transform ϕu(ξ)  of the distribution ϕu ∈ E  (X) is not rapidly decreasing, provided ϕ ∈ C0∞ (X) and supp (ϕ) ∩ sing supp(u) = ∅. Definition 1.3.1: Let u ∈ D (X) and let O be the set of all (x0 , ξ0 ) ∈ X × Rn \ {0} for which there exists an open neighbourhood U of x0 in X and an open conic neighbourhood V of ξ0 in Rn so that for ϕ ∈ C0∞ (U ) and ξ ∈ V we have |ϕu(ξ)|  ≤ Cm (1 + |ξ|)−m ,

m ∈ N.

The closed subset W F (u) = (X × Rn ) \ {0} of X × Rn \ {0} is called the wave front set of u.

16

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

It is easy to see that W F (u) is a conic subset of X × Rn \ {0} with the property π(W F (u)) = sing supp(u), where π : X × Rn −→ X is the natural projection. For our aims in Chapter 3 we will describe the wave front sets of distributions given by oscillatory integrals. Such integrals have the form  (1.19) eiϕ(x,θ) a(x, θ) dθ. Here the phase ϕ(x, θ) is a C ∞ real-valued function, defined for (x, θ) ∈ Γ ⊂ X × (RN \ {0}), and Γ is an open conic set, i.e. (x, θ) ∈ Γ implies (x, tθ) ∈ Γ for all t > 0. We assume that ϕ has the properties: (x, θ) ∈ Γ, t > 0,

ϕ(x, tθ) = t ϕ(x, θ), dx,θ ϕ(x, θ) = 0,

(x, θ) ∈ Γ.

The amplitude a(x, θ) belongs to the class of symbols S m (X × RN ), formed by C ∞ functions on X × RN such that for each compact K ⊂ X and all multi-indices α, β, we have |∂ α ∂ β a(x, θ)| ≤ Cα,β,K (1 + |θ|)m−|β| ,

x ∈ K,

θ ∈ RN .

(1.20)

We endow S m (X × RN ) with the topology defined by the semi-norms pα,β,j (a) =

sup x∈Kj

(1 + |θ|)−m+|β| |∂ α ∂ β a(x, θ)|,

,θ∈RN

where {Kj } is an increasing sequence of compact sets with ∪∞ j=1 Kj = X. Let F ⊂ Γ ∪ (X × {0}) be a closed cone and let supp (a) ⊂ F . For ψ ∈ C0∞ (X) we will now define the integral  eiϕ(x,θ) a(x, θ) ψ(x) dx dθ to obtain a distribution in D (X). To do this, we need a regularization, since the integral in θ is not convergent for m > −N . Choose a function χ ∈ C0∞ (RN ) such that χ(θ) = 1 for |θ| ≤ 1 and χ(θ) = 0 for |θ| ≥ 2. For 0 <  ≤ 1, the functions χ(θ) form a bounded set in S 0 (X × RN ). Then the functions a = a(x, θ)χ(θ) also form a bounded set in S 0 (X × RN ) and 

a → a ∈ S m (X × RN ) as  → 0 for each m > m.

DIFFERENTIAL TOPOLOGY AND MICROLOCAL ANALYSIS

17

Consider the operator L=

n  j=1

with

 ∂ ∂ + bj +χ ∂xj ∂θ j j=1 N

aj

aj = −i(1 − χ)κ−1 ϕxj ,

bj = −i(1 − χ)κ−1 |θ|2 ϕθj ,

and κ = |ϕx |2 + |θ|2 |ϕθ |2 . For each compact set K ⊂ X we have κ(x, θ) ≥ δK |θ|2 ,

x ∈ K,

(x, θ) ∈ Γ,

where δK > 0 depends on K only. Clearly L(eiϕ ) = eiϕ , and the operator t L formally adjoint to L has the form t

L=−

n  j=1

with

 ∂ ∂ − bj +c ∂xj ∂θ j j=1 N

aj

aj ∈ S −1 (X × RN ), bj ∈ S 0 (X × RN ), c ∈ S −1 (X × RN ).

The operator t (L)k is a continuous map of S m onto S m−k . Define the linear map Iϕ,a : C0∞ (X) → R by   eiϕ(x,θ) a(x, θ)χ(θ)ψ(x) dx dθ Iϕ,a (ψ) = lim →0   eiϕ(x,θ) (t L)k [a(x, θ)χ(θ)ψ(x)] dx dθ. = lim (1.21) →0

For m − k < −N the integral on the right-hand side of (1.21) is absolutely convergent, and it is easy to see that Iϕ,a becomes a distribution in D (X). Thus, we obtain the following. Proposition 1.3.2: Let ϕ(x, θ) and a(x, θ) be as above. Then the oscillatory integral (1.19) defines a distribution Iϕ,a given by (1.21). We are now going to describe the set W F (Iϕ,a ). Theorem 1.3.3: We have W F (Iϕ,a ) ⊂ {(x, ϕx (x, θ)) : (x, θ) ∈ F, ϕθ (x, θ) = 0}.

(1.22)

18

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Proof: Let f ∈ C0∞ (X). Then the Fourier transform    ei(ϕ(x,θ)−x,ξ) a(x, θ)f (x) dx dθ f Iϕ,a (ξ) = is expressed by an oscillatory integral. Let V be a closed cone in RN such that V ∩ {ϕx (x, θ) : (x, θ) ∈ F, x ∈ supp (f ), ϕθ (x, θ) = 0} = ∅. By compactness, there exists δ > 0 such that μ = |ξ − ϕx (x, θ)|2 + |θ|2 |ϕθ (x, θ)|2 ≥ δ(|θ| + |ξ|)2

(1.23)

for (x, θ) ∈ F , x ∈ supp (f ) and ξ ∈ V . To obtain (1.23) it suffices to observe that if the latter conditions are satisfied, then the left-hand side of (1.23) is positive and then use the homogeneity with respect to (θ, ξ). As above, consider the operator L=

n  j=1

with aj = −

 ∂ ∂ + bj +χ ∂xj ∂θj j=1 N

aj

i(1 − χ) (ϕxj − ξj ), μ

Then f Iϕ,a (ξ) = lim →0

bj = −

i(1 − χ) 2 |θ| ϕθj . μ

  ei(ϕ(x,θ)−x,ξ) (t L)k [a(x, θ)χ(θ)f (x)] dx dθ,

and applying (1.23), we conclude that |f Iϕ,a (ξ)| ≤ CN (1 + |ξ|)−N ,

ξ ∈ V. 

This implies (1.22).

For asymptotics of oscillatory integrals depending on a parameter λ ∈ R we have the following. Lemma 1.3.4: Let u ∈ D (X), f ∈ C0∞ (X) and let ϕ ∈ C0∞ (X) be a real-valued function. Assume W F (u) ∩ {(x, ϕx ) : x ∈ supp (f )} = ∅. Then for each m ∈ N we have |u, f (x)eiλϕ(x) | ≤ Cm (1 + |λ|)−m ,

λ ∈ R.

DIFFERENTIAL TOPOLOGY AND MICROLOCAL ANALYSIS

19

Proof: Choosing a finite partition of unity, we can restrict our attention to the case u ∈ E  (X). Set Σf = {ξ ∈ Rn \ {0} : ∃x ∈ supp (f ) with (x, ξ) ∈ W F (u)}. Then

  ei(x,ξ−λϕ(x)) f (x)ˆ u, f (x)eiλϕ(x)  = (2π)−n u(ξ) dx dξ     = + = I1 (λ) + I2 (λ). X

W

X

Rn \W

Here W is a closed conic set such that Σf ⊂ W , W ∩ {ϕx (x) : x ∈ supp (f )} = ∅, and I1 (λ) is interpreted as an oscillatory integral. For x ∈ supp (f ) and ξ ∈ W we have |ξ − λϕx (x)| ≥ δ(|ξ| + |λ|), λ ∈ R, with δ > 0. Using the same argument as in the proof of Theorem 1.3.3, we see that I1 (λ) = O(|λ|−m ) for all m ∈ N. For I2 (λ) we use the fact that if ξ ∈ Rn \ W and supp (u) ∩ supp (f ) = ∅, then u ˆ(ξ) is rapidly decreasing. This proves the  assertion. Now let Γ ⊂ X × Rn \ {0} be a closed conic set. Set DΓ (X) = {u ∈ D (X) : W F (u) ⊂ Γ}. Using an argument similar to that in the proof of Lemma 1.3.4, it is easy to see that u ∈ DΓ (X) if and only if for each ϕ ∈ C0∞ (X) and each closed cone V ⊂ Rn with (supp (ϕ) × V ) ∩ Γ = ∅

(1.24)

we have  < ∞, sup |ξ|m |ϕu(ξ)|

m ∈ N.

ξ∈V

This makes it possible to introduce the following. Definition 1.3.5: Let {uj }j ⊂ DΓ (X) and let u ∈ DΓ (X). We will say that the sequence {uj } converges to u in DΓ (X) if: (a) uj → u weakly in D (X) , (b) supj∈N supξ∈V |ξ|m |ϕu j (ξ)| < ∞ for every m ∈ N, every ϕ ∈ C0∞ (X) and every closed cone V satisfying (1.24).

20

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

For every u ∈ Dγ (X) there exists a sequence {uj } ⊂ C0∞ (X) converging to u ∞ in DΓ (X). To prove  this, consider two sequences χj , ϕj ∈ C0 (X) such that χj = 1 on Kj , ϕj ≥ 0, ϕj (x) dx = 1 and supp (χj ) + supp (ϕj ) ⊂ X. Then uj = ϕj ∗ χj u ∈ C0∞ (X) and uj → u in D (X). Moreover, the condition (b) also holds, so uj → u in DΓ (X). For our aims in Chapter 3 we need to justify some operations on distributions (see [Hl] for more details). For convenience of the reader we list these properties, including only one proof of these – namely that of the existence of the pull-back f ∗ . We use the notation from [Hl]. Let X ⊂ Rn and Y ⊂ Rm be open sets and let f : X −→ Y be a smooth map. Consider a closed cone Γ ⊂ Y × Rm \ {0} and set Nf = {(f (x), η) ∈ Y × Rn :t f  (x)η = 0}, f ∗ (Γ) = {(x,t f  (x)η : (f (x), η) ∈ Γ}. For u ∈ C0∞ (Y ), consider the map (f ∗ u)(x) = u(f (x)). Theorem 1.3.6: Let Nf ∩ Γ = ∅. Then the map f ∗ u can be extended uniquely on the space DΓ (Y ) such that W F (f ∗ u) ⊂ f ∗ Γ. (1.25) Proof: Using a partition of unity, we may consider only the case when X and Y are small open neighbourhoods of x0 ∈ X and y0 ∈ Y , respectively. Set Γy = {η : (y, η) ∈ Γ}. Choose a small compact neighbourhood X0 of x0 and a closed conic neighbourhood V of Γy0 so that t  f (x)η = 0 for x ∈ X0 , η ∈ V. Next, choose a small compact neighbourhood Y0 of y0 with Γy ⊂ V for all y ∈ Y0 . Now let χ ∈ C0∞ (X0 ) and let {uj }j ⊂ C0∞ (Y ) be a sequence such that uj → u in DΓ (Y ). Choosing ϕ ∈ C0∞ (Y0 ) with ϕ = 1 on f (X0 ), we have    j (η)Iχ dη = + = I1 + I2 , f ∗ uj , χ = f ∗ (ϕuj ), χ = (2π)−m ϕu V



where Iχ (η) =

eif (x),η χ(x) dx.

For x ∈ supp (χ) and η ∈ V we obtain |∇x f (x), η| ≥ δ|η|,

δ > 0.

Rm \V

DIFFERENTIAL TOPOLOGY AND MICROLOCAL ANALYSIS

21

Using the operator L=

n  ∂ −i ∂xj (f (x), η) , 2 |∇x f (x), η| j=1 ∂xj

we integrate by parts in Iχ (η) and get |Iχ (η)| ≤ Cp (1 + |η|)−p ,

η ∈V,

for all p ∈ N. On the other hand, there exists M > 0 such that |ϕu j (η)| ≤ C(1 + |η|)−M , j ∈ N. Thus, I1 is absolutely convergent, and we can consider the limit as j → ∞. / V , (b) yields the To deal with I2 , notice that (suppϕ \ V ) ∩ Γ = ∅. For η ∈ estimates (1.26) |ϕu j (η)| ≤ Cp (1 + |η|)−p , p ∈ N, uniformly with respect to j. Thus, we can let j → ∞ in I2 . To establish (1.25), replace χ(x) by χ(x)e−ix,ξ and write  Iχ (η, ) = (2π)−n eif (x),η−ix,ξ χ(x) dx. Choose a small open conic neighbourhood W of the set {ξ = t f  (x0 )η : (f (x0 ), η) ∈ Γ} so that x ∈ X0 and η ∈ V imply t f  (x)η ∈ W . As above, for x ∈ X0 , η ∈ V and ξ∈ / W we deduce the estimate |ξ − t f  (x)η| ≥ δ(|ξ| + |η|),

δ > 0.

For such ξ and η we integrate by parts in Iχ (η, ) and obtain |Iχ (η, )| ≤ Cp (1 + |ξ| + |η|)−p ,

p ∈ N.

C0∞ (R)

with ψ(ξ) = 1 for |ξ| ≤ 1, For η ∈ / V,ξ ∈ / W we choose a function ψ(ξ) ∈ and consider the operator   ∂ + ψ(x). L = −i(1 − ψ(ξ))|ξ|−2 ξ, ∂x Then L(eix,ξ ) = eix,ξ , and, as in the previous case, for η ∈ / V and ξ ∈ / W , we get the estimates |Iχ (η, )| ≤ Cp (1 + |η|)p (1 + |ξ|)−p , p ∈ N. Combining these estimates with (1.26), we obtain ∗ u )(ξ)| ≤ C (1 + |ξ|)−N |χ(f j N

for ξ ∈ / W , where the constant CN does not depend on j. Letting j → ∞ proves  (1.25).

22

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

By an easy modification of the above-mentioned argument, one proves the following modification of Theorem 1.3.6 for distributions depending on a parameter. Corollary 1.3.7: Let Z be a compact subset of Rp and let Z z → (u, ·, z) ∈ DΓ (Y ) be a continuous map. Under the assumptions of Theorem 1.3.6, the map Z z → f ∗ (u, ·, z) ∈ Df ∗ Γ (X) is continuous. Next, consider a linear continuous map K : C0∞ (Y ) −→ D (X). By Schwartz’s theorem (cf. Theorem 5.2.1 in [Hl]), there exists a distribution K ∈ D (X × Y ), called the kernel of K, such that K, ϕ(x) ⊗ ψ(y) = (Kψ)(x), ϕ(x) for all ϕ ∈ C0∞ (X) and ψ ∈ C0∞ (Y ). W F (K) will be called the wave front set of K. Set W F  (K) = {(x, y, ξ, η) : (x, y, ξ, −η) ∈ W F (K)}, W F (K)X = {(x, ξ) : (x, y, ξ, 0) ∈ W F (K) for some y ∈ Y }, W F  (K)Y = {(y, η) : (x, y, 0, η) ∈ W F  (K) for some x ∈ X}, and consider the composition W F  (K) ◦ W F (u) = {(x, ξ) : ∃(y, η) ∈ W F (u) with (x, y, ξ, η) ∈ W F  (K)}. The following two results will also be necessary for Chapter 3. Their proofs can be found in Section 8.2 of [Hl]. Theorem 1.3.8: For ψ ∈ C0∞ (Y ) we have W F (Kψ) ⊂ {(x, ξ) : (x, y, ξ, 0) ∈ W F (K) for some y ∈ supp (ψ)}. Theorem 1.3.9: There exists a unique extension of K on the set {u ∈ E  (Y ) : W F (u) ∩ W F  (K)Y = ∅}

DIFFERENTIAL TOPOLOGY AND MICROLOCAL ANALYSIS

23

such that for each compact M ⊂ Y and each closed conic set Γ with Γ ∩ W F  (K)Y = ∅ the map E  (M ) ∩ DΓ (Y ) u → Ku ∈ D (X) is continuous. Moreover, the inclusion W F (Ku) ⊂ W F (K)X ∪ W F  (K) ◦ W F (u) holds. The wave front of u ∈ D (X) can be described by means of the characteristic set of pseudo-differential operators on X. Denote by Lm (X) the class of all pseudo-differential operators (PDO) in X of order m. If x(x, ξ) ∈ S m (X × Rn ) is the symbol of A ∈ Lm (X), then the oscillatory integral  −n eix−y,ξ a(x, ξ) dξ KA (x, η) = (2π) determines the kernel of A and W F (A) = W F (KA ). The operator A ∈ Lm (X) is called properly supported if for each compact K ⊂ X there exists another compact K  ⊂ X so that supp (u) ⊂ K implies supp (Au) ⊂ K  , and if u = 0 on K  , then Au = 0 on K. A point (x0 , ξ0 ) ∈ T ∗ X \ {0} is called non-characteristic for a properly supported PDO A ∈ Lm (X) if there exists a properly supported PDO B ∈ L−m (X) so that (x0 , ξ0 ) ∈ / W F (AB − Id) ∪ W F (BA − Id). In this case A is called elliptic at (x0 , ξ0 ). Proposition 1.3.10: If there exists a properly supported PDO A ∈ Lm (X), elliptic / W F (u). at (x0 , ξ0 ), such that Au ∈ C ∞ (X), then (x0 , ξ0 ) ∈ The reader may consult Section 18 in [Hl] for the main properties of PDOs and for a proof of the above-mentioned proposition.

1.4

Boundary problems for the wave operator

Let Ω ⊂ Rn be a domain in Rn , n ≥ 2 with C ∞ smooth compact boundary ∂Ω. Consider the problem ⎧ 2 ◦ ⎪ ⎨(∂t − Δx )u = f in R × Ω , (1.27) u = u0 on R × ∂Ω, ⎪ ⎩ u|t t0 }. Then zˆ is either a characteristic point in Σ0 or a point in T ∗ (∂Ω) ∩ Σ = H ∪ G, and there exists a maximal compressed generalized bicharacteristics γ˜ (σ) = (x(σ), ξ(σ)) of , passing through zˆ and staying in W Fb (u) as long as t(σ) = xn+1 (σ) > t0 . One can also describe the singularities of a boundary problem with nonhomogeneous boundary condition  (∂t2 − Δx )u = f in R × Ω◦ , (1.29) u = g on R × ∂Ω, with f = 0, g = 0 for t < t0 . In this situation we have the following result established in [MS2] (see Theorem 6.14). Theorem 1.4.3: Let u be a solution of (1.29) and let f ∈ C ∞ . Then W Fb (u) is a complete union of the generalized half-bicharacteristics issued from W F (g). Here half-bicharacteristics means that we consider these bicharacteristics γ for which the time increases when we move along γ. The same results hold for the boundary problem  (∂t2 − Δx )u = f in R × Ω◦ , (1.30) (∂ν + α(x))u = u0 on R × ∂Ω, where ∂ν is the derivative with respect to a normal field of ∂Ω and α(x) is a C ∞ function on ∂Ω. For α(x) = 0 we have the Neumann problem, while for α(x) = 0 we obtain the Robin problem.

1.5 Notes The results in Section 1.1 can be found with detailed proofs in [GG] and [Hir]. In Section 1.2 we follow [MS1], [MS2] and [H3]. Lemma 1.2.5 is proved in [MS1], while Lemmas 1.2.6, 1.2.7 and 1.2.10 can be found in [H3]. The results in Section 1.3 concerning wave front sets of distributions and operators are due to Hörmander [Hl], [H3]. The definition of generalized wave front set W Fb (u) was introduced by Melrose and Sjöstrand [MS1]. Theorem 1.3.11 was established in [MS1], [MS2]. We refer the reader to Section 24 in [H3] for more details concerning the generalized bicharacteristics and the propagation of singularities for the Dirichlet problem.

2

Reflecting rays In this chapter we begin with some elementary properties of periodic reflecting rays in a domain Ω, relating these rays to critical points of certain length functions. In a similar way in Section 2.4 we deal with scattering rays. For both kinds of rays the special case is considered when the complement of Ω is a finite disjoint union of strictly convex bounded domains. The results obtained in this case will be useful for our considerations in subsequent chapters. The linear Poincaré map Pγ of a periodic reflecting ray γ is frequently used in this book. It is defined in Section 2.3, where a useful matrix representation of it is also described. The analogue of a Poincaré map for a reflecting scattering ray γ, the so-called differential cross section dJγ , is defined and studied in Section 2.4. This section contains also the main definitions concerning scattering rays which will be frequently used in the next chapters.

2.1

Billiard ball map

Let Ω be a domain in Rn , n ≥ 2, with a smooth boundary X = ∂Ω. The billiard flow in Ω is the dynamical system generated by the motion of a material point in Ω. The point is moving with constant velocity in the interior of Ω making reflections at ∂Ω according to the usual law of geometrical optics ‘the angle of incidence is equal to the angle of reflection’. The successive positions of the point at ∂Ω are described by the so-called billiard ball map, which will be denoted hereafter by B. This map is defined on a subset M  of the set M = {(x, v) ∈ X × Sn−1 : ν(x), v ≥ 0}, ν(x) being the unit normal to ∂Ω at x pointing into the interior of Ω, as follows. Let q = (x, v) ∈ M be such that the straight-line ray γ issued from x in direction Geometry of the Generalized Geodesic Flow and Inverse Spectral Problems, Second Edition. Vesselin M. Petkov and Luchezar N. Stoyanov. © 2017 John Wiley & Sons, Ltd. Published 2017 by John Wiley & Sons, Ltd.

REFLECTING RAYS

27

v has a common point with X and let y be the first such point, that is y ∈ X and the open segment (x, y) does not contain any points of X. By definition, the subset M  consists of those q so that γ intersects X transversally at y. For such q define w = v − 2v, ν(y) ν(y) and set B(q) = (y, w). Thus, one obtains a map B : M  −→ M . Clearly M  and M are open subsets of X × Sn−1 and so they have natural structures of smooth manifolds. It is a standard exercise to show that B is a smooth map. From a dynamical point of view it is more convenient to consider B as a map B : M0 −→ M0 , where n  M0 = ∩∞ n=−∞ B (M )

and B n = B ◦ B ◦ · · · ◦ B (n times). The points q ∈ M0 with B k (q) = q for some integer k ≥ 2 will be called periodic points of period k of B. Clearly B has no fixed points in M  , that is points q with B(q) = q. Notice that in general M  is a proper subset of M . For unbounded domains Ω this is clear, while for bounded but non-convex Ω this is due to the existence of rays tangent to ∂Ω. In the latter case it is more convenient to deal with the generalized geodesic flow on T ∗ Ω introduced in Chapter 1. Using the standard identification of T Ω with T ∗ Ω, one can consider the billiard flow as a subsystem of the generalized geodesic flow, and respectively, the billiard ball map B as a map on some subset of the cosphere bundle S ∗ (∂Ω) (see Section 4.2). Example 2.1.1: Let Ω be a strictly convex-bounded domain in Rn with smooth boundary ∂Ω. Clearly in this case, M0 = M = M  . Moreover, B can be naturally extended to a diffeomorphism B : M −→ M by B(q) = q for every q ∈ M \ M , where M = {(x, v) ∈ X × Sn−1 : ν(x), v ≥ 0} is the closure of M in X × Sn−1 . In fact, M is a manifold with boundary ∂M = M \ M . We refer the reader to [Ko] for a proof of the fact that B is smooth on M . It can be easily shown that for any integer s ≥ 2, there exists a periodic point q ∈ M of period s. Indeed, fix an arbitrary s and consider the function F = Fs : Ωs −→ R given by F (x1 , . . . , xs ) =

s 

xi − xi+1 ,

(2.1)

i=1

where xs+1 = x1 by definition. Since Ωs is compact and F is continuous, there exists x = (x1 , . . . , xs ) ∈ Ωs such that F has a maximum at x. A trivial argument shows that xi ∈ ∂Ω and xi = xi+1 for every i = 1, . . . , s. Then the restriction G of F to (∂Ω)s has a maximum at x. Since G is smooth on (∂Ω)s , x is a critical point of G. It then follows that x1 , x2 , . . . , xs are the successive reflection points of a periodic billiard trajectory (see Proposition 2.1.3 for a rigorous proof), that is

28

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

B(xi , vi ) = (xi+1 , vi+1 ), where vi = (xi+1 − xi )/ xi+1 − xi . Therefore, B s (xi , vi ) = (xi , vi ), which shows that B has at least s distinct periodic points of period s. In the case n = 2, that is Ω ⊂ R2 , one can modify the above argument to prove the existence of periodic points of B of arbitrary period s and a given rotation number k ≤ s/2. Given x = (x1 , . . . , xs ) ∈ Ωs , define the rotation (winding) number r(x) as follows. Set xs+1 = x1 , and for any i = 1, . . . , s, denote by i the length of the segment [xi , xi+1 ] on ∂Ω with respect to the positive (counter-clockwise) orientation the negative orientation of ∂Ω. Denote by of ∂Ω, and by i its length with respect to L the length of ∂Ω. The integer part of L1 si=1 i will be denoted by r+ (x) and that of L1 si=1 i by r− (x). Since i + i = L, we have r+ (x) + r− (x) = s, therefore s r(x) = min{r+ (x), r− (x)} ≤ . 2 The number r(x) is called the rotation number of x = (x1 , . . . , xs ). Given two integers s ≥ 2 and k with 1 ≤ k ≤ s/2, applying the above argument to the function F = Fs on the set of those x ∈ Ωs with r(x) = k, one gets that there exists x ∈ Ωs with r(x) = k such that x is generated by a periodic point (x1 , v1 ) of B of period s. For our needs in subsequent chapters it will be convenient to put the notion of a periodic orbit of the billiard in a more general setting. This will allow us to consider such orbits in arbitrary domains Ω with smooth boundaries ∂Ω. Let X be a smooth (n − 1)-dimensional submanifold of Rn , n ≥ 2. Given two linear segments 1 and 2 with a common end x ∈ X, we will say that these segments satisfy the law of reflection at x with respect to X if 1 and 2 make equal acute angles with one of the unit normals ν(x) to X at x , and 1 , 2 and ν(x) lie in a common two-dimensional plane. Definition 2.1.2: Let X be as above and let γ be a curve in Rn of the form γ = ∪ki=1 i , where i = [xi , xi+1 ] is a straight-line segment, xi ∈ X for each i = 1, . . . , k, k ≥ 2, and we set for convenience xk+1 = x1 and k+1 = 1 . We will say that γ is a periodic reflecting ray for X if the following conditions are satisfied: (a) for each i = 1, . . . , k the open segments ◦i do not intersect transversally X (but may have common tangent points with X); (b) for each i = 1, . . . , k the segments i and i+1 satisfy the law of reflection at xi+1 with respect to X. The points x1 , . . . , xs will be called reflection points of γ, while γ =

k  i=1

will be called the length of γ.

xi − xi+1

REFLECTING RAYS

29

Notice that if Ω is a domain with boundary X, then a periodic reflecting ray for X may not be entirely in Ω (see Figure 2.1). If γ contains a segment orthogonal to X at some of its end points, then γ will be called symmetric, otherwise it will be called non-symmetric. In general a periodic reflecting ray γ may have segments tangent to X at some of its interior points (see Figure 2.2). If γ has no such segments, we will say that γ is ordinary. Let us mention that in what follows, with the exception of Section 2.2, points like z1 in Figure 2.2 are not considered as reflection points. In general a periodic reflecting ray can pass two or more times through some of its reflection points, and two different periodic reflecting rays could have some common reflection points. Given a periodic reflecting ray γ and an integer k ≥ 2, one defines naturally the k-multiple δ of γ. Clearly, as a subset of Rn , δ coincides with γ; however, the number of reflection points of δ is ks, where s is the number of reflection points of γ. We will say that γ is primitive if γ is not a multiple of any periodic reflecting ray (Figures 2.3, 2.4). We conclude this section with an elementary fact, which however will be important later on. Namely, we will show that there is a natural one-to-one correspondence between the periodic reflecting rays with s reflection points for a given submanifold X and some kind of critical points of the restriction of the map F = Fs defined by (2.1) on X s . Notice that F is well defined and continuous on (Rn )s and F is smooth on the set Us of those y = (y1 , . . . , ys ) ∈ (Rn )s such that yi = yi+1 for all i = 1, . . . , s (as before ys+1 = y1 by definition).

Figure 2.1 Periodic reflecting rays.

z1

z2

Figure 2.2

Tangent rays.

30

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS x1 = x4

x6

x2

x3 x5

Figure 2.3

Multiple reflections through a point.

Figure 2.4 Periodic rays with common reflection points. Proposition 2.1.3: Let γ be a curve in Rn of the form γ = ∪si=1 i , where i = [xi , xi+1 ] is a straight-line segment, xi ∈ X for each i = 1, . . . , s, s ≥ 2, xs+1 = x1 , and such that the open segments ◦i do not intersect X transversally. Then γ is a periodic reflecting ray for X if and only if x = (x1 , . . . , xs ) is a critical point of the map F|(X s ∩Us ) . Proof: Take arbitrary smooth charts ϕj : Rn−1 −→ Uj ⊂ X, such that ϕj (0) = xj . Then {

∂ϕj (t)

∂uj

j = 1, . . . , s,

(0)}n−1 t=0 is a basis of the tangent space to X at xj .

(1)

(n−1)

Here we use the notation uj = (uj , . . . , uj

) ∈ Rn−1 . Consider the function

G : (Rn−1 )s −→ R

REFLECTING RAYS

31

defined by G(u1 , . . . , us ) = F (ϕ1 (u1 ), . . . , ϕs (us )). (1)

(n−1)

). Clearly for any u ∈ (Rn−1 )s sufficiently close to 0, G Let ϕj = (ϕj , . . . , ϕj is differentiable at u and   ϕj (uj ) − ϕj+1 (uj+1 ) ∂ϕj ϕj (uj ) − ϕj−1 (uj−1 ) ∂G + , (u) = (u ) . (t) ϕj (uj ) − ϕj−1 (uj−1 ) ϕj (uj ) − ϕj+1 (uj+1 ) ∂u(t) j ∂u j

j

Setting vi,j = we get



∂G (t)

∂uj

xi − xj , xi − xj

(0) =

vj,j−1 + vj,j+1 ,

∂ϕj (t)

 (0) .

∂uj

Notice that the segments j and j+1 satisfy the law of reflection at xj+1 with respect to X if and only if the vector vj,j−1 + vj,j+1 is orthogonal to X at xj+1 . According to the above argument, this is equivalent to the fact that ∂G(t) (0) = 0 for all ∂uj

t = 1, . . . , n − 1. Hence, γ is a periodic reflecting ray for X if and only if 0 is a  critical point of G. This proves the proposition.

2.2

Periodic rays for several convex bodies

In this section we study periodic reflecting rays in a domain Ω in Rn such that the complement K = Rn \ Ω has the form K = K1 ∪ · · · ∪ K s ,

(2.2)

s ≥ 3. Here each Ki is a compact convex domain in Rn with C 2 -smooth boundary Γi = ∂Ki and Ki ∩ Kj = ∅ whenever i = j. Our aim in what follows is to provide a coding for the periodic reflecting rays in Ω. Namely, we associate with any periodic reflecting ray γ a finite sequence αγ = (i1 , . . . , ik ) ∈ {1, . . . , s}k , where k is the number of reflections of γ such that the jth successive reflection point belongs to Kij for any j = 1, . . . , k. Clearly, for such a sequence we have ij = ij+1 for j = 1, . . . , k − 1and ik = i1 . Every α = (i1 , . . . , ik ) ∈ {1, . . . , s}k

(2.3)

with the latter property will be called a configuration of length |α| = k. Denote by Ak the set of all configurations of length k. We will show that if all Ki are strictly

32

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

convex, then the correspondence γ −→ αγ ∈ Ak is invertible, and under some additional assumption, it is moreover bijective. It is easy to construct examples showing that in general this map is not surjective. Clearly Γ = ∂Ω = Γ1 ∪ · · · ∪ Γs . For q ∈ Γ, the unit normal vector to Γ at q pointing into the interior of Ω will be denoted by ν(q). The second fundamental form of Γ is non-positive definite at any q ∈ Γ with respect to this choice of the normal field. Definition 2.2.1: We will say that K satisfies the condition (H) if for any i = j the / {i, j}. convex hull of Ki ∪ Kj has no common points with Kr for any r ∈ In what follows the reflecting rays for Γ contained in Ω will be called briefly ¯ reflecting rays in Ω. Proposition 2.2.2: Let K satisfy the condition (H). Then for every integer k ≥ 2 and ¯ such that α = α. every α ∈ Ak there exists a periodic reflecting ray γ in Ω γ Proof: Fix an arbitrary α of the form (2.3) and consider the function F = Fα : Kα = Ki1 × · · · × Kik −→ R, defined by F (q1 , . . . , qk ) =

k 

qj − qj+1 ,

(2.4)

(2.5)

j=1

where we use the notation qk+1 = q1 . Since F is continuous and Kα is compact, there exists q = (q, . . . , qk ) ∈ Kα such that F has an absolute minimum at q. A simple geometric argument shows that qj ∈ Γij for all j. It follows from the condition (H) that each of the open segments (qj , qj+1 ) has no common points with K. Now applying Proposition 2.1.3 one gets that q1 , . . . , qk are the successive ¯ and α = α. This reflection points of a periodic reflecting ray for Γ. Clearly γ ⊂ Ω γ  proves the assertion. In what follows up to the end of this section we deal with the general case, that is we do not assume the condition (H) to be satisfied. Now it is more convenient to consider the points of tangency of periodic reflecting rays as reflection points (we will do the same in Section 2.4). To this end we need an extension of the billiard ball map B similar to that in Example 2.1.1. Let Lq Γ be the tangent hyperplane to Γ at q ∈ Γ. An element x = (q, v) ∈ Γ × Sn−1 will be called regular if either ν(q), v > 0 or ν(q), v = 0 and there exists a neighbourhood U of q in Γ such that U ∩ Lq Γ = {q}. If all Ki are strictly convex, then each point x with ν(q), v ≥ 0 is regular; however in the general case this condition is not enough.

REFLECTING RAYS

33

Denote by M  the set of those regular elements x such that the straight-line ray δ starting at q with direction v has a common point with Γ and if p is the first common point (i.e. the open segment (q, p) has no common points with Γ), then y = (p, w) is a regular element of Γ × Sn−1 , where w = v − 2ν(p), vν(p). We set B(x) = y, thus extending B to a map B : M  −→ Γ × Sn−1 . We will be interested in the restriction B : M1 −→ M1 of B, where −m M1 = ∩∞ (M  ). m=0 B

More specifically, we will study the periodic points x ∈ M1 of B, that is the points for which there exists k ∈ N with B k (x) = x. Let π : Γ × Sn−1 −→ Γ be the natural projection and let α ∈ Ak . A point x = (q, v) ∈ M1 will be called a periodic point of type α for B if B k (x) = x and qj = π ◦ B j−1 (x) ∈ Γij

(2.6)

for all j = 1, . . . , k. If the segment [qj , qj+1 ] is tangent to Γ at qj , we will say that qj is a tangential reflection point of the corresponding periodic billiard trajectory γ(x); otherwise qj will be called a proper reflection point. In general there could exist distinct periodic points of B having the same type α. One can construct examples involving periodic reflecting rays with parallel corresponding segments arranging suitably obstacles with flat parts of their boundaries (see Figure 2.5). As we see from the next theorem these are in fact the only possibilities to construct such examples.

Figure 2.5

Parallel periodic rays.

34

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Theorem 2.2.3: Let α ∈ Ak , k ≥ 2, and suppose there exist two distinct periodic points (q, v) and (p, w) of type α for B and let qj = π ◦ B j−1 (q, v) and pj = π ◦ B j−1 (p, w) for all j = 1, 2, . . . . Then v = w and for any j ≥ 1 the segments [qj , qj+1 ] and [pj , pj+1 ] are parallel. If qj is a proper reflection point, then tqj + (1 − t)pj ∈ Γij

(2.7)

for all t ∈ [0, 1]. If all qj are proper reflection points, then for t ∈ (0, 1) sufficiently close to 1 the points (tq + (1 − t)p, v) are periodic reflection points of type α for B, ¯ with equal lengths and parallel corregenerating periodic billiard trajectories in Ω sponding segments. In other words for any α ∈ Ak , there are three possibilities: (i) there are no periodic points of type α for B; (ii) there is exactly one periodic point of type α for B; (iii) the periodic points of type α generate a (continuous or discrete) family of ¯ of equal lengths and with parallel corresponding periodic billiard trajectories in Ω segments. Before proceeding with the proof of the above theorem, we consider some consequences of it. Corollary 2.2.4: Let α have the form (2.3) and let Γij be strictly convex for all j = 1, . . . , k. Then there exists at most one periodic point of type α for B. For k ≥ 2 set ak = Ak . Clearly a2 = s(s − 1),

a3 = s(s − 1)(s − 2).

On the other hand, it is easy to show that for k ≥ 4 we have ak = (s − 2)ak−1 + (s − 1)ak−2 . Thus, ak = (s − 1)k + (−1)k (s − 1) for any k ≥ 2. Combining the latter with Corollary 2.2.4 and Proposition 2.2.2, we deduce the following. Corollary 2.2.5: Let Ki be strictly convex for all i = 1, . . . , s and let Pk be the number of periodic points of B of period k. Then Pk ≤ ak = (s − 1)k + (−1)k (s − 1),

(2.8)

and therefore lim sup k→∞

log Pk ≤ log(s − 1). k

(2.9)

REFLECTING RAYS

35

If an addition Ω satisfies the condition (H), then there are equalities in both (2.8) and (2.9). Here log = logc with an arbitrary constant c > 1. The rest of this section is devoted to the proof of Theorem 2.2.3. Fix an α of the form (2.3) and consider the function (2.4) defined by (2.5). Set Γα = Γi1 × · · · × Γik . Clearly, Kα is a compact convex subset of (Rn )k ; however, Γα is not its boundary. In fact, Γα is a ‘very thin’ subset of ∂Kα . Lemma 2.2.6: Let x = (q, v) be a periodic point of type α of B and let the points qj be defined by (2.6) for all j = 1, . . . , k. Then: (a) The map F : Kα −→ R has a local minimum at q˜ = (q1 , . . . , qk ); (b) If there exists at least one j such that Γ is strictly convex at qj and qj+1 is a proper reflection point, then F has a strict local minimum at q˜. Proof: Since the case k = 2 is trivial, we assume k ≥ 3. There exist C 2 -smooth charts ϕj : Rn−1 −→ Uj ⊂ Γij such that ϕj (0) = qj . Consider the function G : (Rn−1 )k −→ R defined by G(u1 , . . . , uk ) = F (ϕ1 (u1 ), . . . , ϕk (uk )). First we will show that G has a local minimum at 0; this would imply that F|Γα has a local minimum at q˜. (1) (n) Let ϕj = (ϕj , . . . , ϕj ) and let u = (u1 , . . . , uk ) ∈ (Rn−1 )k . In what follows we also use the following notation: Ij = {j − 1, j + 1}, aij = 1/ qi − qj ,

(1)

(n−1)

uj = (uj , . . . , uj

) ∈ Rn−1 ,

vij = aij (qi − qj ).

Clearly, aij = aji > 0 and vji = −vij ∈ Sn−1 . For j = 1, . . . , k, t = 1, . . . , n − 1 and u sufficiently close to 0 we have    ϕj (uj ) − ϕi (ui ) ∂ϕj ∂G , (u) = (u ) . (2.10) (t) ϕj (uj ) − ϕi (ui ) ∂u(t) j ∂u j

i∈Ij

j

36

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

By Proposition 2.1.3, 0 is a critical point of G. We will prove that the second fundamental form of G at 0 is non-negative defined. We now need the derivatives ∂2G (t)

(m)

(0)

(2.11)

∂uj ∂ui

for i, j = 1, . . . , k and t, m = 1, . . . , n − 1. Having fixed j, there are three possibilities for i. Case 1. i ∈ / Ij ∪ {j}. Then clearly the derivative (2.11) is 0. Case 2. i ∈ Ij . In this case (2.10) implies 

∂2G (t)

(m)

∂uj ∂ui

(0) = −aij

+aji

∂ϕi

∂ϕi

(0) (m) ∂ui     ∂ϕj ∂ϕi (0), vji (0), vji . (t) (m) ∂uj ∂ui (t)

(0),



∂uj

Case 3. i = j. Then ∂2G (t)

(m)

(0) =

∂uj ∂uj



 vji ,

i∈Ij

+



aji

i∈Ij

−



aji

i∈Ij



∂ 2 ϕj

(0) (t) (m) ∂uj ∂uj   ∂ϕj ∂ϕj (0), (m) (0) (t) ∂uj ∂uj     ∂ϕj ∂ϕj (0), vji (0), vji . (t) (m) ∂uj ∂uj

(t)

Fix an arbitrary vector ξ = (ξj )1≤j≤k,1≤t≤n−1 ∈ (Rn−1 )k . We have to establish that σ=

k n−1  

∂2G (t)

(t) (m)

(m)

i,j=1 t,m=1 ∂uj ∂ui

(0) ξj ξi

Set (1)

(n−1)

ξj = (ξj , . . . , ξj

) ∈ Rn−1

and zj =

n−1  t=1

(t)

ξj

∂ϕj (t)

∂uj

(0).

≥ 0.

REFLECTING RAYS

37

Notice that for νj = ν(qj ), there exists λj > 0 such that vjj−1 + vjj+1 = −λj vj . Since the hypersurface Uj = ϕj (Rn−1 ) ⊂ Γ is convex at qj , the choice of the normal field ν shows that the second fundamental form Bj of Uj at qj is a non-positive definite. That is,   n−1  ∂ 2 ϕj (t) (m) νj , (t) (m) (0) ξj ξj ≤ 0 Bj (ξj , ξj ) = ∂uj ∂uj t,m=1 for all ξj ∈ Rn−1 . Using the expressions for the second derivatives of G at 0 in the three possible cases, we get σ=

k  n−1 

∂2G (t)

(t) (m)

(m)

j=1 t,m=1 ∂uj ∂uj

+ ⎛

k   n−1 

∂2G

k 

λj

j=1



n−1 

k   n−1 

 aji

+ ⎝−

∂ϕj (t) ∂uj

 aji

(t)



k   n−1 

aji

+

=−

aji

∂ϕj

∂ϕj (t)

k 

k  

k   j=1 i∈Ij

ξj ξj

k   j=1 i∈Ij

⎞



∂ϕj (m)

(t) (m) ⎠

ξj ξj

(0), vji



∂ϕi (m) ∂ui

(0)

∂ϕi (m)

∂ui

aji zj , zj  −

j=1 i∈Ij

aji zj , zi  +

(t) (m)

(0)

∂uj

(0),

(0), vji

j=1 i∈Ij t,m=1

λj Bj (ξj , ξj ) +

(m) ∂uj



 

∂uj

j=1

−



∂ϕj

ξj ξi

(0), vji

(t) ∂uj

j=1 i∈Ij t,m=1 k   n−1 

(0),

(t) (m)

(0)

 

∂ϕj ∂uj

j=1 i∈Ij t,m=1

⎛

(t) (m) ∂uj ∂ui

t,m=1

k   n−1 



∂ 2 ϕj

νj ,

j=1 i∈Ij t,m=1

−

(t) (m)

(0) ξj ξi

(t) (m) j=1 i∈Ij t,m=1 ∂uj ∂ui

= ⎝−

+

(0) ξj ξj

(t) (m)

ξj ξi

⎞

 (0), vji

k   j=1 i∈Ij

aji zj , vji zi , vji .

(t) (m) ⎠

ξj ξi

aji zj , vji 2

38

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Since i ∈ Ij is equivalent to j ∈ Ii , aji = aij and vji = −vij , it now follows that σ=−

k 

λj Bj (ξj , ξj ) +

j=1

k 

ajj+1 zj 2 − zj , vjj+1 2

j=1

−zj , zj+1  + zj , vjj+1 zj+1 , vjj+1 + zj+1 2



−zj+1 , vj+1j 2 − zj+1 , zj  + zj+1 , vj+1j zj , vj+1j  =−

k 

λj Bj (ξj , ξj ) +

j=1

k 

ajj+1 ( zj − zj+1 2 − zj − zj+1 , vjj+1 2 ).

j=1

Since vjj+1 = 1, we have zj − zj+1 , vjj+1 2 ≤ zj − zj+1 2 , which yields σ ≥ 0. Next, assume that ξ = 0 and σ = 0, and let there exist j such that Γ is strictly convex at qj and qj+1 is a proper reflection point. It follows from σ = 0 that Bj (ξj , ξj ) = 0 and the vector zj − zj+1 is parallel to vjj+1 , that is to the segment [qj , qj+1 ]. Since Bj is definite, one gets ξj = 0, that is zj = 0. On the other hand, the vector zj+1 lies in the tangent hyperplane to Γ at qj+1 ; therefore, [qj , qj+1 ] is tangent to Γ at qj+1 , a contradiction with the assumption that qj+1 is a proper reflection point. Thus, the assumptions in (b) imply σ > 0 for any choice of ξ = 0. In this way we have established that G has a local minimum at 0, hence F|Γα has a local minimum at q˜. Moreover, if the assumptions in (b) are satisfied, then F|Γα has a strict local minimum at q˜. q ) ≤ F (˜ p) Next, for every j, fix a neighbourhood Vj of qj in Kij such that F (˜ whenever p˜ ∈ V ∩ Γα , where V = V1 × · · · × Vk . Since the points B j−1 (q, v) are regular, we may assume that neighbourhoods Vj are chosen so that for each p˜ ∈ V and each j = 1, . . . , k the straight line determined by the segment [pj , pj+1 ] intersects Γij and Γij+1 at points in Vj and Vj+1 , respectively. Indeed, if qj is a tangential reflection point, we may define Vj by Vj = {pj ∈ Kij : pj − qj , ν(qj ) > −j } for some sufficiently small j > 0. If qj is a proper reflection point, consider an open ball Dj in Rn centred at qj and having a sufficiently small radius j > 0 and set Vj = Kij ∩ Dj . Let p˜ = (p1 , . . . , pk ) ∈ V . Denote by p1 the intersection point of Γi1 with the segment [p1 , p2 ]. Then p1 ∈ V1 and the triangle inequality implies F (p1 , . . . , pk ) ≥ F (p1 , p2 , . . . , pk ). Next, for the intersection point p2 of Γi2 with the segment [p1 , p2 ] we get F (p1 , p2 , p3 , . . . , pk ) ≥ F (p1 , p2 , p3 , . . . , pk ),

REFLECTING RAYS

39

etc. Continuing in this way, for any j we find a point pj ∈ Γij ∩ Vj such that F (˜ p) ≥ F (˜ p ) holds for p˜ = (p1 , p2 , . . . , pk ) ∈ Γα ∩ V . It now follows from the q ) and therefore F (˜ p) ≥ F (˜ q ). This concludes the choice of V that F (˜ p ) ≥ F (˜ proof of part (a). The proof of (b) follows easily from the above arguments. We leave the details to  the reader. Proof of Theorem 2.2.3: Fix α of the form (2.3) and let F : Kα −→ R be defined as above. Clearly F is a convex function, that is F (˜ q + (1 − t)˜ p) ≤ tF (˜ q ) + (1 − t)F (˜ p) for all t ∈ [0, 1] and all q˜, p˜ ∈ Kα . Assume that there exist two different periodic points (q, v) and (p, w) of type α for B. Set q˜ = (q1 , . . . , qk ), p˜ = (p1 , . . . , pk ). Then q˜, p˜ ∈ Kα and Lemma 2.2.6 implies that F has local minima at both these points. For t ∈ [0, 1] set (t)

qj = tqj + (1 − t)pj ,

(t)

(t)

q˜(t) = (q1 , . . . , qk ).

Clearly q˜(t) = t˜ q + (1 − t)˜ p ∈ Kα . We will show that F (˜ q ) = F (˜ p). Suppose that F (˜ q ) > F (˜ p). Then for every t ∈ (0, 1) we have q + (1 − t)˜ p) ≤ tF (˜ q ) + (1 − t)F (˜ p) < F (˜ q ). F (˜ q (t) ) = F (t˜

(2.12)

Since q˜(t) −→ q˜ as t −→ 1, we get a contradiction with the fact that F has a local minimum at q˜. Therefore, F (˜ q ) ≤ F (˜ p). Similarly, we obtain F (˜ q ) ≥ F (˜ p), so F (˜ q ) = F (˜ p). Combining the latter with (2.12), we get more, namely that q ) = F (˜ p) F (˜ q (t) ) = F (˜

(2.13)

for all t sufficiently close to 0 or 1. Now the convexity of F implies that (2.13) holds for all t ∈ [0, 1]. Let us recall that for p = p , q = q  and t ∈ (0, 1) the equality (tq + (1 − t)p) − (tq  + (1 − t)p ) = t q − q  +(1 − t) p − p holds if and only if the segments [p, p ] and [q, q  ] are parallel. In our situation this yields that the segments [qj , qj+1 ] and [pj , pj+1 ] are parallel for every j. In particular, v = w.

40

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Choose the neighbourhoods Vj of the points qj as in the proof of Lemma 2.2.5. (t) There exists t0 ∈ (0, 1) such that qj ∈ Vj for any t ∈ (t0 , 1]. Clearly F has a mini(t) mum at qj in V = V1 × · · · × Vk for every t ∈ (t0 , 1]. Let qj be a proper reflection (t) / Γij for some t ∈ (t0 , 1). Set point for some j and assume that qj ∈ r˜ = (q1 , . . . , qj−1 , qj , qj+1 , . . . , qk ), (t)

(t)

(t)

(t)

where qj is the intersection point of Γij with the segment [qj , qj+1 ]. Since qj is a proper reflection point, it follows from the above remark that (t)

(t)

qj−1 − qj + qj − qj+1 > qj−1 − qj + qj − qj+1 , (t)

(t)

(t)

(t)

(t)

(t)

r). This is a contradiction with the minimality of F (˜ q (t) ). therefore F (˜ q (t) ) > F (˜ (t) Thus, qj ∈ γij for all t ∈ (t0 , 1] sufficiently close to 1. Finally, if all qj are proper reflection points, the last argument shows that for all t ∈ (0, 1) sufficiently close to 1 the points (tq + (1 − t)p, v) are periodic points of type α of B. Clearly, these generate periodic billiard trajectories in Ω with lengths  F (˜ q ) = F (˜ p) and parallel corresponding segments.

2.3

The Poincaré map

Throughout this section Ω will be a closed domain in Rn with smooth boundary X = ∂Ω and γ will be an ordinary periodic reflecting ray in Ω with successive reflection points q1 , q2 , . . . , qm , qm+1 = q1 and period (length) T > 0. Here we define the linear Poincaré map Pγ of γ and present a useful representation of it. There are different ways to define this map, however all of them are equivalent in the sense that the spectrum spec(Pγ ) is the same. γ ) = γ, Let γ˜ be a generalized bicharacteristic of  in T ∗ (R × Ω) such that πx (˜ where πx : T ∗ (R × Ω) −→ Ω is the composition of the natural projections T ∗ (R × Ω) −→ R × Ω −→ Ω. Given ρ ∈ γ˜ with πx (ρ) = qi for all i = 1, . . . , m, there exists a small conic neighbourhood V of ρ in T ∗ (Ω◦ ) such that for every (y, η) ∈ V the generalized bicharacteristic of , parameterized by the time and issued from (y, η) has exactly m reflections at ∂Ω for all t ∈ [0, T ]. The Hamiltonian flow FT , introduced in Section 1.2, maps V into a conic neighbourhood W of ρ, and we can define the map (dFT )(ρ) : Tρ (T ∗ (Ω)) −→ Tρ (T ∗ (Ω)). Clearly the tangent vectors e to γ˜ at ρ and the direction f of the cone axis at ρ are invariant with respect to (dFT )(ρ). Let Eρ be the two-dimensional space generated

REFLECTING RAYS

by e and f , and let

41

Σρ = Tρ (T ∗ (Ω))/Eρ

be the corresponding quotient space. The linear map Pγ (ρ) = dFT (ρ)|Σρ will be called the (linear) Poincaré map of γ at ρ. Clearly Pγ preserves the natural symplectic structure of Σρ (cf. e.g. [AbM]). / ∂Ω and πx (μ) ∈ / ∂Ω, If ρ and μ are two different points on γ˜ such that πx (ρ) ∈ then for some τ ∈ R we have Φτ (ρ) = μ and therefore Fτ ◦ FT (σ) = FT ◦ Fτ (σ), Thus,

σ ∈ V.

dFτ (ρ) ◦ dFT (μ) = dFT (μ) ◦ dFτ (ρ),

so the Poincaré map Pγ (ρ) is conjugated to the Poincaré map Pγ (μ). Therefore, the spectrum spec(Pγ ) of Pγ (ρ) is independent of the choice of ρ. We will say that γ is non-degenerate if 1 ∈ / spec(Pγ ). Denote by Πi the hyperplane in Rn passing through the point qi and orthogonal −−→ q− to the line qi qi+1 and by ωi the unit vector determined by the vector − i , qi+1 . In what follows we assume for convenience that for j ≡ i (mod m) we have Πj = Πi , qj = qi , etc. We also assume that for each i the hyperplane Πi is endowed with a linear basis such that qi = 0. Πi+1

l'(u, u)

X

υ'

qi+1

ωi+1 p

l(u, u) ωi u

υ

qi Πi

Figure 2.6

The billiard ball map.

u'

42

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

For a pair (u, v) ∈ Πi × Πi sufficiently close to (0, 0) let (u, v) be the oriented line passing through u and having direction ωi + v. Here we identify the point v with the vector v. If (u, v) is sufficiently close to (0, 0), then (u, v) intersects transversally X at some point p = p(u, v) close to qi+1 . Let  (u, v) be the oriented line symmetric to (u, v) with respect to the tangent hyperplane to X at p, and let u be the intersection point of  (u, v) with Πi+1 (clearly such a point exists for (u, v) close to (0, 0)). There is a unique v  ∈ Πi+1 such that ωi+1 + v  has the direction of the line  (u, v) (see Figure 2.6). Thus, we obtain a map Φi+1 : Πi × Πi  (u, v) → (u , v  ) ∈ Πi+1 × Πi+1 , defined for (u, v) in a small neighbourhood of (0, 0). The smoothness of this map follows from the smoothness of the billiard ball map. Consider the composition Pγ = Φm ◦ · · · ◦ Φ1 : Πm × Πm −→ Πm × Πm , and the linear map dPγ (0, 0) : Πm × Πm −→ Πm × Πm . Let ρ ∈ γ˜ be a fixed point such that z = πx (ρ) lies on the open segment (qm , q1 ). −→ qm q1 . Then we Consider the hyperplane Πz passing through z and orthogonal to − ∗ can identify Πz × Πz with the space Tρ (T (Ω))/Eρ . Given (u, v) ∈ Πm × Πm sufficiently close to (0, 0), consider the oriented line (u, v) passing through u with ˜. Write the unit vector ω(u, v) with the direction ωm + v. Let (u, v) intersect Πz at u direction of (u, v) in the form ω(u, v) = ωm + v˜, where v˜ ∈ Πz . Thus, we obtain a map u, v˜) ∈ Πz × Πz , Φz : Πm × Πm  (u, v) → (˜ defined for (u, v) sufficiently close to (0, 0) such that Φz (0, 0) = (0, 0). u, v˜) be the minimal positive number such that For (˜ u, v˜) ∈ Πz × Πz let t(˜ u, ωm + v˜) = (p, ωm + q) Ft(u,˜ ˜ v ) (˜ with (p, q) ∈ Πz × Πz . Setting u, v˜) = (p, q) ∈ Πz × Πz , Qz (˜ we obtain a map defined in a small neighbourhood of (0, 0) in Πz × Πz . Clearly, t(0, 0) = T , Qz (0, 0) = (0, 0), and for (u, v) close to (0, 0) we have (Φz ◦ Pγ )(u, v) = (Qz ◦ Φz )(u, v). By using the local smoothness of Ft (σ) with respect to t and σ we get dΦz (0, 0) ◦ dPγ (0, 0) = (dFT )|Πz ×Πz (0, 0) ◦ dΦz (0, 0). Therefore, dPγ (0, 0) is conjugated to the Poincaré map Pγ of γ.

REFLECTING RAYS

43

In what follows very often we will use the notation Pγ for dPγ (0, 0) and call it the Poincaré map of γ. Next, we proceed to describe a useful representation of Pγ = dPγ (0, 0). We need some additional notation. Set λi = qi−1 − qi ,

(2.14)

and denote by αi the tangent hyperplane to X at qi , by σi the symmetry with respect to αi , and by Πi the hyperplane passing through qi and orthogonal to ωi−1 . Clearly Πi is parallel to Πi−1 ; see Figure 2.6. Moreover, σi (ωi−1 ) = ωi , σi (Πi ) = Πi . Choose a continuous unit normal field νi (q) to X for q ∈ X near qi such that νi (qi ), ωi  > 0. For u ∈ Πi close to 0 (i.e. to qi ) denote by (u, 0) the line through u orthogonal to Πi . The intersection points of (u, 0) with X and αi will be denoted by fi (u) and πi (u), respectively (we choose fi (u) close to qi ). Then πi : Πi −→ αi is the natural projection along the vector ωi−1 , while fi is a local diffeomorphism fi : (Πi , qi ) −→ (X, qi ), which can be considered as a parameterization of X about qi . The second fundamental form S(ξ, η) of X at qi is defined for ξ, η ∈ αi by S(ξ, η) = Gi (ξ), η, where Gi = dνi (qi ) : αi −→ αi . Since S is symmetric and bilinear (cf. e.g. [GKM]), there exists a unique symmetric linear map (2.15) ψ˜i : Πi −→ Πi such that ψ˜i σi (ξ), σi (η) = −2ωi−1 , νi (qi ) Gi (πi (ξ)), πi (η)

(2.16)

for all ξ, η ∈ Πi (Figure 2.7). Since σi (Πi ) = Πi with respect to the linear basis fixed in Πi and that in Πi obtained by identifying the latter with Πi−1 using the translation along the line qi−1 qi we may regard (σi )|Πi as a real symmetric (n − 1) × (n − 1) matrix. For the sake of brevity we will denote this matrix again by σi . Next, we write the linear maps

44

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Πi–1

qi–1

ωi–1

qi+1

νi ωi X

αi

qi πi

Π'i

Figure 2.7

Πi

The reflection operator.

between products of the type Πj × Πj as 2(n − 1) × 2(n − 1) block matrices

A C

B , D

where A, B, C, D are real (n − 1) × (n − 1) matrices. By I we denote the identity matrix on any Πj . After these preparations, fix an arbitrary i = 1, . . . , m and write the map Φi in the form (r) (t) Φi = Φi ◦ Φi , where Φi : Πi−1 × Πi−1 −→ Πi × Πi , (t)

Φi : Πi × Πi −→ Πi × Πi (r)

are defined in small neighbourhoods of (0, 0) as follows. For u, v ∈ Πi−1 let (u, v) be the oriented line defined as above. Denote by u the intersection point of (u, v) (t) with Πi and set Φi (u, v) = (u , v  ), where v  ∈ Πi is such that the vector v  + ωi−1 has the direction of (u, v). Finally, set Φi = Φi ◦ (Φi )−1 . (r)

(t)

(t)

Clearly Φi is a linear map and (t)

Φi =

I 0

λi I . I

(2.17)

REFLECTING RAYS

45

(r)

Next, write the linear map Ri = dΦi (0, 0) in the form

A B . Ri = C D We will now determine the matrices A, B, C, D in terms of λi , σi and ψ˜i . (r) Take u = 0 and v ∈ Πi close to 0. Then clearly Φi (0, v) = (0, σi (v)), which yields



A B 0 0 , = C D v σi (v) and therefore B = 0, D = σi . Now take u ∈ Πi close to 0 and v = 0 and set (u , v  ) = Φi (u, 0) ∈ Πi × Πi . (r)

(r)

It follows from the definition of Φi that u = fi (u) + tω, v  = ω  − ω  , ωi ωi

(2.18)

for some t ∈ R, where ω  is the vector symmetric to ωi−1 with respect to the tangent hyperplane to X at fi (u), that is ω  = ωi−1 − 2ωi−1 , νi (fi (u)) νi (fi (u)). Setting νi = νi (qi ), we have νi (fi (u)) = νi + Gi (πi (u)) + O( u 2 ), and, taking into account that σi (ωi−1 ) = ωi = ωi−1 − 2ωi−1 , νi  νi , we find ω  = ωi − 2ωi−1 , Gi (πi (u)) νi − 2ωi−1 , νi  Gi (πi (u)) + O( u 2 ).

(2.19)

Since ωi−1 , νi  = ωi , νi  and Gi (πi (u)) ∈ αi , (2.19) implies ω  , ωi  = 1−2ωi−1 , Gi (πi (u)) νi , ωi  − 2ωi−1 , νi Gi (πi (u)), ωi +O( u 2 ) = 1 + O( u 2 ). It now follows from (2.18) and (2.19) that v  = −2ωi , Gi (πi (u)) νi + 2ωi , νi  Gi (πi (u)) + O( u 2 ).

(2.20)

To compute u , first combine (2.18)–(2.20) to get u = fi (u) + t(ωi − v  ) + O( u 2 ) = πi (u) + t(ωi − v  ) + O( u 2 ).

(2.21)

46

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Since u , ωi  = v  , ωi  = 0, (2.21) yields t = −πi (u), ωi  + O( u 2 ), so u = πi (u) − πi (u), ωi  ωi + O( u 2 ) = σi (πi (u) − πi (u), ωi  ωi−1 ) + O( u 2 ). On the other hand, it is easily seen that u = πi (u) − πi (u), ωi ωi−1 . Combining this with the latter expression for u gives u = σi (u) + O( u 2 ).

(2.22)

Now it follows from (2.22) and (2.20) that the components A and C of the matrix (r) R = Φi (0, 0) have the form A = σi and Cu = −2ωi , Gi (πi (u)) νi + 2ωi , νi  Gi (πi (u)).

(2.23)

We will show now that C = ψ˜i σi . Indeed, since σi (v) = πi (v) − πi (v), ωi  ωi for v ∈ Πi and Cu, ωi  = 0, it follows from (2.23) that Cu, σi (v) = Cu, πi (v) = 2ωi , νi  Gi (πi (u)), πi (v) = −2ωi−1 , νi  Gi (πi (u)), πi (v). Combining the latter with (2.16), one gets (C − ψ˜i σi )(u), σi (v) = 0 for all u, v ∈ Πi . Therefore, C = ψ˜i σi which shows that

σi 0 (r) . Φi (0, 0) = ˜ ψi σi σi Finally, the latter and (2.17) imply



σ 0 σ I λi I = ˜i dΦi (0, 0) = ˜ i 0 I ψi σi σi ψi σ i



I λi I σi 0 = ˜ . 0 σi ψi I + λi ψ˜i

λi σ i (I + λi ψ˜i )σi

REFLECTING RAYS

47

Next, we identify the hyperplanes Πi−1 and Πi using the translation along the line qi−1 qi (which is orthogonal to both hyperplanes). Then we can write σi (Πi−1 ) = Πi and for the composition (2.24) si = σi ◦ σi−1 ◦ · · · ◦ σ1 one has si (Πm ) = Πi . Consider the symmetric linear map ˜ ψi = s−1 i ψi si : Πm −→ Πm . Now we can view the matrix

−1 s I λi I = i ψ i I + λi ψi 0

0 s−1 i



I ˜ ψi

λi I I + λi ψ˜i

(2.25)



si 0

0 si

as a linear map Πm × Πm −→ Πm × Πm . Combining this with the representation of dΦi (0, 0), we obtain the following. Theorem 2.3.1: Under the assumptions and conventions above, the Poincaré map Pγ : Πm × Πm −→ Πm × Πm is a linear symplectic map that has the following matrix representation





I λm I I λ1 I sm 0 Pγ = ··· , 0 sm ψ m I + λ m ψm ψ 1 I + λ 1 ψ1

(2.26)

where sm , ψj and λj are given by (2.24), (2.25), (2.16) and (2.14). Let us recall that X = ∂Ω is called strictly convex (convex) at qi with respect to the unit normal field νi (q) provided the linear operator Gi (qi ), defined above, is positive definite (resp. non-negatively semi-definite). It follows from (2.16) that this condition is equivalent to the fact that the linear symmetric map ψ˜i is positive definite (resp. non-negatively semi-definite). Proposition 2.3.2: Let Ω and γ be as in the beginning of this section, and assume that X = ∂Ω is strictly convex at qi with respect to the normal field νi (q) for every i = 1, . . . , m. Then the Poincaré map Pγ is hyperbolic, that is spec(Pγ ) has no common points with the unit circle. Proof: For any k = 1, . . . , m denote by Mk the space of all linear symmetric maps M : Πk −→ Πk . Recall that Π0 = Πm according to our notation above. Let M0 ∈ M0 be non-negatively semi-definite (we will denote this by M0 ≥ 0). Consider the linear subspace L0 = {(u, M0 u) : u ∈ Π0 } of Π0 × Π0 . Then L0 is a Lagrangian subspace of Π0 × Π0 with respect to the natural symplectic structure of that space, and dΦ1 (0, 0)(L0 ) coincides with the linear

48

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

subspace L1 = {σ1 (I + λ1 M0 )u, σ1 ((I + λ1 ψ1 )M0 + ψ1 )u : u ∈ Π0 } of Π1 × Π1 . It is convenient to introduce the operators

defined by

Ai : Mi−1 −→ Mi

(2.27)

Ai (M ) = σi M (I + λi M )−1 + ψ˜i .

(2.28)

Then for M1 = A1 (M0 ) we have L1 = {(v, M1 v) : v ∈ Π1 }. Define inductively Mk = Ak (Mk−1 ),

Lk = {(u, Mk u) : u ∈ Πk }.

Then Mk ∈ Mk and Lk is a linear subspace of Πk × Πk . Choose an arbitrary  > 0 such that ψ˜i ≥ I for every i = 1, . . . , m and denote by Mk () the subspace of Mk consisting of all M ∈ Mk such that M ≥ I. Notice that Ak (Mk−1 ()) ⊂ Mk () and for any A, B ∈ Mk−1 () we have Ak (A) − Ak (B) = σk ((I + λk A)−1 (A − B)(I + λk B)−1 )σk . Therefore, Ak (A) − Ak (B) ≤ (1 + λk )−2 A − B ≤

A − B , (1 + λ)2

where λ = min λk . This shows that for any k the map Ak is a contraction from Mk−1 () to Mk (). Then the map A = Am ◦ Am−1 ◦ · · · ◦ A1 is a contraction from M0 () into M0 (). Consequently, there exists a (unique) fixed point M0 ∈ M0 () of A. Now taking into account (2.26) and (2.27), we see that for any u ∈ Πm = Π0 we have



u Su = , Pγ M0 u M0 Su where the linear map S : Πm −→ Πm is defined by S = σm (I + λm Am−1 (M0 )) ◦ σm−1 (I + λm−1 Am−2 (M0 )) ◦ · · · ◦σ2 (I + λ2 A1 (M0 )) ◦ σ1 (I + λ1 M0 ),

REFLECTING RAYS

49

and Ak = Ak ◦ Ak−1 ◦ · · · ◦ A1 . Moreover, it follows from the expression for S that Sx ≥

m 

(1 + λi ) x

i=1

for any x ∈ Πm . Consequently, spec(S) ⊂ {z ∈ C : |z| > 1}. The eigenvalues of S are clearly eigenvalues of Pγ . Hence Pγ has n − 1 eigenvalues zj with |zj | > 1. Since Pγ is symplectic, 1/zj are eigenvalues, too. This proves the  proposition. Corollary 2.3.3: Let Ω = Rn \ (K1 ∪ K2 ), where K1 and K2 are compact disjoint strictly convex domains in Rn with smooth boundaries ∂K1 and ∂K2 . Let γ be the unique periodic reflecting ray in Ω with two reflection points q1 ∈ K1 and q2 ∈ K2 . Then spec(Pγ ) ⊂ (0, 1) ∪ (1, ∞). Proof: We use the argument from the proof of Proposition 2.3.2. In the present case, we have λ1 = λ2 = λ, and moreover Π1 and Π2 are parallel and can be identified. Thus, S = σ2 (I + λA1 (M0 )) ◦ σ1 (I + λM0 ), and setting M2 = M0 (I + λM0 )−1 + ψ1 , we obtain (I + λM2 )−1/2 S(I + λM2 )1/2 = (I + λM2 )−1/2 (I + λM0 )(I + λM2 )1/2 . Therefore, the eigenvalues of S are real and greater than 1 which proves the assertion. 

2.4

Scattering rays

Let Ω be a closed connected domain in Rn , n ≥ 2, with smooth boundary X = ∂Ω and bounded complement. Set K = Rn \ Ω.

(2.29)

Clearly K is a compact domain with smooth boundary ∂K = X and the complement of Ω is K \ X. Let ω and θ be two fixed unit vectors in Rn . Definition 2.4.1: Let γ be a curve in Ω of the form γ = ∪kj=0 i for some k ≥ 1, where i = [xi , xi+1 ] are finite segments for i = 1, . . . , k − 1, xi ∈ X for all i, and 0 (resp. k ) is the infinite segment starting at x0 (resp. at xk ) and having direction −ω (resp. θ). The curve γ is called a reflecting (ω, θ)-ray in Ω if for every

50

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

i = 0, 1, . . . , k − 1 the segments i and i+1 satisfy the law of reflection at xi+1 with respect to X. For such γ the points x1 , . . . , xk will be called reflection points of γ. Clearly, for every generalized (ω, θ)-geodesic c : R −→ Ω, which has no gliding segments on ∂Ω and only finitely many reflection points, the curve γ = Image (c) is a reflecting (ω, θ)-ray in Ω (cf. Section 1.2). It is easy to construct examples showing that the converse is not true. By a scattering ray in Ω we mean a reflecting (ω, θ)-ray for some unit vectors ω and θ. Such a ray γ will be called symmetric if some segment of γ is orthogonal to X at some of its end points; otherwise γ will be called non-symmetric. Clearly, a symmetric reflecting (ω, θ)-ray may exist only if θ = −ω, and for such a ray γ we have k = 2m + 1 and m−i = m+i−1 for all i = 0, 1, . . . , m. A scattering ray without segments tangent to X will be called ordinary. Next, we define two important notions related to a scattering ray. Fix an arbitrary open ball U0 with radius a > 0 containing K. For any ξ ∈ Sn−1 denote by Zξ the hyperplane orthogonal to ξ and tangent to U0 such that ξ is pointing into the interior of the open half-space Hξ with boundary Zξ and containing U0 . Let πξ : Rn −→ Zξ be the orthogonal projection. For a reflecting (ω, θ)-ray γ in Ω with successive reflection points x1 , . . . , xk the sojourn time Tγ of γ is defined by Tγ = πω (x1 ) − x1 +

k−1 

xi − xi+1 + xk − π−θ (xk ) −2a.

(2.30)

i=1

Clearly, Tγ + 2a coincides with the length of the part of γ that lies in Hω ∩ H−θ (see Figure 2.8). It is easy to see that Tγ does not depend on the choice of the ball U0 . Indeed, we have πω (x1 ) − x1 = a + ω, x1 , xk − π−θ (xk ) = a − θ, xk , and (2.30) implies Tγ = ω, x1  +

k−1 

xi − xi+1 −θ, xk .

(2.31)

i=1

This proves that Tγ does not depend on the choice of U0 . Let γ be a reflecting (ω, θ)-ray as above. Set uγ = πω (xi ) and assume that γ is ordinary, that is it has no segments tangent to X = ∂Ω. Then there exists a neighbourhood W = Wγ of uγ in Zω such that for every u ∈ W there are unique θ(u) ∈ Sn−1 and points x1 (u), . . . , xk (u) ∈ X which are the successive reflection points of a reflecting (ω, θ(u))-ray in Ω with πω (x1 (u)) = u. We set Jγ (u) = θ(u), thus obtaining a map Jγ : Wγ −→ Sn−1 . It follows immediately from the smoothness of the billiard ball map related to an appropriately chosen domain Ω ⊂ Ω ∩ Hω ∩ H−θ , that the map Jγ is smooth, too. It is an easy exercise to check the latter fact directly. A scattering ray γ will be called non-degenerate if rank(dJγ ) = n − 1. Next, applying Theorem 2.3.1, we will obtain a matrix representation for dJγ (uγ ). Set m = k + 2, qi = xi for i = 1, . . . , k, q0 = πω (q1 ), qk+1 = π−θ (qk ),

REFLECTING RAYS

51

θ

uγ

u

θ(u)

Zω ω Z−θ

x1(u)

xm(u) xm

x1

H−θ

∂K

U0

Figure 2.8

The map Jγ .

λi = qi−1 − qi , Π0 = Zω , Πk+1 = Z−θ . For i = 1, . . . , k define Πi , σi , ψ˜i as in Section 2.3. We assume again that in every Πi a linear basis is fixed with qi = 0. Define the maps Φi+1 : Πi × Πi −→ Πi+1 × Πi+1 , i = 0, 1, . . . , k, as in Section 2.3. Then by the same argument one gets

σ λi σ i , i = 0, 1, . . . , k + 1. dΦi (0, 0) = ˜ i ψi σi σi + λi ψ˜i σi On the other hand, dJγ (q0 )u = pr2





u dΦk+1 (0, 0) ◦ · · · ◦ dΦ1 (0, 0) 0

u = v. Therefore, v

σk σ1 λk σ k × · · · × dJγ (q0 )u = pr2 σk ψ˜k σk + λk σk ψ˜k ψ˜1 σ1 for u ∈ Π0 , where pr2

The next proposition treats a special case of scattering rays.

λ1 σ 1 σ1 + λ1 ψ˜1 σ1

u . 0 (2.32)

52

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Proposition 2.4.2: Under the notation and conventions above, assume in addition that for every i = 1, . . . , k, X = ∂K is (strictly) convex at qi = xi with respect to the unit normal νi pointing into the interior of Ω. Then for every u ∈ Π0 = Zω we have dJγ (q0 )u = Mk σk (I + λk Mk−1 )σk−1 (I + λk−1 Mk−2 ) · · · σ2 (I + λ2 M1 )σ1 u, (2.33) where (2.34) Mi : Πi −→ Πi , i = 1, . . . , k, are non-negative semi-definite (resp. positive definite) symmetric linear maps defined inductively by M1 = ψ˜1 ,

Mi = σi Mi−1 (I + λi Mi−1 )−1 σi + ψ˜i ,

(2.35)

i = 2, 3, . . . , k + 1. In particular, det dJγ (q0 ) = 0. Proof: Since ∂K is (strictly) convex at qi , it follows from the definitions of ψ˜i (cf. Section 2.3) that they are non-negatively semi-definite (resp. positive definite) symmetric linear maps. Now we can use (2.34) to define the maps Mi inductively; the definition is correct and Mi ≥ 0 (resp. Mi > 0) for all i = 1, . . . , k. Set u ui = dΦi (0, 0) ◦ · · · ◦ dΦ1 (0, 0) . vi 0 Clearly, u1 = σ1 u, v1 = ψ˜1 σ1 u. We will prove by induction that ui = σi (I + λi Mi−1 )ui−1 , vi = Mi ui

(2.36)

for every i = 2, 3, . . . , k + 1. From this the equality (2.33) follows immediately. Assume that vi−1 = Mi−1 ui−1 for some i > 1. Then





I λi I ui σi ui−1 u = dΦi (0, 0) i−1 = ˜ = vi vi−1 σi Mi−1 ui−1 ψi I + λi ψ˜i   σi (I + λi Mi−1 )ui−1 = . (ψ˜ σ + σ M + λ ψ˜ σ M )u i i

i

i−1

i

i i

i−1

i−1

Thus, ui = σi (I + λi Mi−1 )ui−1 and vi = (σi Mi−1 + ψ˜i σi (I + λi Mi−1 ))ui−1 = (σi Mi−1 (I + λi Mi−1 )−1 σi + ψ˜i )σi (I + λi Mi−1 )ui−1 = Mi ui . Therefore, (2.36) holds which proves the assertion.



REFLECTING RAYS

53

From now on until the end of this section we will assume that K has the form (2.2), where s ≥ 2 and Ki are disjoint strictly convex compact domains in Rn with smooth boundaries ∂Ki = Γi . As before ω and θ will be two fixed unit vectors and U0 an open ball containing K. We will also use the notation Zξ , Hξ and πξ introduced above. For any u ∈ Zω consider the billiard semi-trajectory γ(u) = {St (u) : t ≥ 0} such that S0 (u) = u and N0 (u) = ω, where Nt (u) is the velocity vector of the trajectory St (u) at time t, that is the unit vector determining the direction of the trajectory at that point. For y = St (u) ∈ X we set N−t (u) = lim Nt− (u),

 > 0,

→0

and N+t (u) = σy (N−t (u)), where σy is the symmetry with respect to the tangent hyperplane to X at y. By x1 (u), x2 (u), . . . we denote the successive reflection points of γ(u) and by t1 (u), t2 (u), . . . the corresponding times (moments) of reflection. Notice that the points xj (u) include not only the proper (transversal) reflection points of γ(u) but its tangent (reflection) points as well. For convenience set x0 (u) = u,

t0 (u) = 0,

and denote by r(u) the number of reflection points of γ(u). Thus, r(u) is a non-negative integer or ∞. Denote by Ak the set of all symbols of the form (2.3) such that ij = ij+1 for all j = 1, . . . , k − 1. Here we do not assume that i1 = ik , so for k > 1 and s > 1 the set Ak introduced in Section 2.2 is a proper subset of Ak . The elements of the latter set will be called configurations of length k. Let α ∈ Ak be a fixed configuration of the form (2.3). Set Fα = {u ∈ Zω : r(u) ≥ k ,

xj (u) ∈ Kij for all j = 1, . . . , k},

(2.37)

and denote by Uα the set of those u ∈ Fα such that xj (u) is a proper reflection point of γ(u) for all j = 1, . . . , k. Clearly, Fα is a bounded subset of Zω (it is contained in πω (K)) consisting of those u ∈ Zω for which the trajectory γ(u) has at least k reflection points and the first k reflections ‘follow’ the configuration α. In general Fα is not a closed subset of Zω . However, it is easy to see that Uα is open in Zω . For every u ∈ Fα set Jα (u) = Nt+ (u) ∈ Sn−1 ,

t = tk (u).

Thus, we obtain a continuous map Jα : Fα −→ Sn−1 . Clearly Jα is smooth on Uα .

(2.38)

54

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

The rest of this section is devoted to the study of the map Jα . First, we consider some immediate consequences of the definition of this map. Lemma 2.4.3: (a) For every u ∈ Fα , there exists β = (i1 , . . . , ip ) ∈ Ap for some p ≥ k such that u ∈ Fβ and there exists a sequence p1 < p2 < · · · < pk = p with ipj = ij for any j = 1, . . . , k. Moreover, if xr (u) is a proper reflection point of γ(u) for some r = 1, . . . , k, then ir = ipj for some j = 1, . . . , k. (b) Jα can be extended to a continuous map Jα : Fα −→ Sn−1 . Before continuing let us introduce some additional notation. Set Lα = {u ∈ Fα : Nt (u) = ω, Mα = Fα \ Lα ,

t ≥ 0},

Eα = Jα (Fα ).

(2.39) (2.40)

Note that Lα is a compact subset of Zω contained in the boundary (in Zω ) of the convex subset πω (Ki1 ). Hence Lα has Lebesgue measure zero and empty interior in Zω . In fact, it is a smooth compact (n − 2)-dimensional submanifold. It is clear from the above definition that Lα is non-empty only for special K and special configurations α. Since Fα is compact, Eα is a compact subset of Sn−1 . Finally, since Jα (u) = ω for any u ∈ Lα , the set Jα (Mα ) coincides either with Eα (in most cases) or with Eα \ {ω}. We will use the main result in Section 2.2 to prove the following property of the map Jα . Proposition 2.4.4: For every configuration α the map Jα : Mα −→ Jα (Mα )

(2.41)

is a homeomorphism. Proof: It is enough to prove that if u ∈ Mα , v ∈ Fα and Jα (u) = Jα (v), then u = v. Indeed, assume this is true. Then (2.41) is a continuous bijection and we have to show that its inverse is continuous. Let {uk } ⊂ Mα and u ∈ Mα be such that Jα (uk ) −→ Jα (u) as k −→ ∞, and let v be an arbitrary limit point of the sequence {uk }. Then v ∈ Fα and clearly Jα (u) = Jα (v). Then we must have u = v. The compactness of Fα now shows that uk −→ u as k −→ ∞. Thus, the map (2.41) is a homeomorphism. Let α have the form (2.3) and let u ∈ Mα and v ∈ Fα , u = v. It follows from the definition of Mα that γ(u) has at least one proper reflection point. Let j1 < · · · < jm (m ≤ k) be all natural numbers not greater than k such that y = xij (u) are proper  reflection points of γ(u). It follows from Lemma 2.4.3(a) that there exists a configuration β with the properties listed in the lemma such that v ∈ Fβ . Moreover, there exist reflection points z ∈ Kij ( = 1, . . . , m) of γ(v) such that the successive proper  reflection points of γ(v) form a subsequence of z1 , . . . , zm .

REFLECTING RAYS

55

Assume that Jα (u) = Jα (v) = η ∈ Sn−1 . Consider arbitrary convex domains K0 and Ks+1 with smooth boundaries in Rn such that K0 ⊂ Rn \ Hω , Ks+1 ⊂ Hη , πω (K) ⊂ ∂K0 and π−η (K) ⊂ ∂Ks+1 . Now we are in a position to apply Lemma 2.2.5 to K  = K0 ∪ K1 ∪ · · · ∪ Ks ∪ Ks+1 and the configuration λ = (0, ij1 , . . . , ijm , s + 1, ijm , . . . , ij1 , 0). Consider the convex domain Kλ = K0 × Kij × · · · × Kijm × Ks+1 × Kijm × · · · × Kij × K0 1

1

in (Rn )2m+3 and the corresponding length function F = Fλ : Kλ −→ R (see (2.4) and (2.5)). Then by Lemma 2.2.6(b), F has a strict local minimum at the point q˜ = (q0 , q1 , . . . , qm , qm+1 , qm , . . . , q1 , q0 ), where qi = xi (u) for i = 0, 1, . . . , m and qm+1 = πη (qm ). It then follows from the convexity of F that it has no other local minima in Kλ . On the other hand, by Lemma 2.2.6(a), F has also a local minimum at the point p˜ = (p0 , p1 , . . . , pm , pm+1 , pm , . . . , p1 , p0 ), where pi = xi (v) for i = 0, 1, . . . , m and pm+1 = πη (pm ). Therefore, q˜ = p˜ which implies u = q0 = p0 = v, in contradiction with u = v. This shows that Jα (u) = Jα (v) and thus completes the proof of the proposition. 

Combining Propositions 2.4.2 and 2.4.4, we get the following. Corollary 2.4.5: For every configuration α the map Jα : Uα −→ Jα (Uα ) is a diffeomorphism. Let α be a configuration of the form (2.3) and let γ be a reflecting (ω, θ)-ray in Ω with successive reflection points x1 , . . . , xk . We will say that γ is of type α if xj ∈ Kij for all j = 1, . . . , k. The following is another direct consequence of Proposition 2.4.4.

56

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Corollary 2.4.6: If ω = θ, then for every configuration α there is at most one reflecting (ω, θ)-ray of type α in Ω. Under some conditions on α (resp. K), ω and θ, it is shown in Section 10.3 that there exists a (unique) reflecting (ω, θ)-ray of type α in Ω.

2.5 Notes The material discussed in Section 2.1 was already known to Birkhoff [Bir]. The reader is referred to [KozT] for more details on this and related topics. The results in Section 2.2 are taken from [S2], where the more general case of semi-dispersing billiards is considered. In the particular case when the condition (H) is satisfied the statement of Corollary 2.2.4 was proved by Ikawa [I4] using a different technique. The representation of the linear Poincaré map Pγ described in Theorem 2.3.1 was obtained in [PV]. Proposition 2.3.2 generalizes a result from [BGR] concerning two disjoint strictly convex domains. Most of the material in Section 2.4 is taken from [PS5], however the proof of Proposition 2.4.4 is different from that in [PS5]. A general definition of a billiard on a Riemann manifold with boundary can be found in [CFS] (see also [Sin2]). A more general type of dynamical systems is studied in [KS]. Ergodic properties of billiards related to certain problems in statistical mechanics have been studied very intensively in the 1970s, 1980s and 1990s by Sinai, Bunimovich, Chernov and many others (see [Sinl], [BunS], [BCS], [Cher1], [KSS], [Sim] and the references there). One should mention in particular the solutions of the Boltzmann–Sinai Ergodic Hypothesis by N. Simányi [Sim]. For dispersing billiards in the plane the results in [BCS] imply an exponential estimate from below for the number Pk of periodic points of period k (cf. also Corollary 2.2.5), and the set of periodic points is dense in the phase space of such billiards. Further results concerning periodic points of billiards are contained in [Cher2]. As mentioned in Section 3.4 in [Cher2], according to some arguments in [BCS] the estimate (2.8) in Corollary 2.2.5 does not hold for dispersing billiards on the flat torus; for the latter it might even happen that Pk = ∞ for some k. It follows from [Cher2] that for dispersing billiards log Pk lim inf k can be estimated from below by the metric entropy of the billiard ball map (see e.g. [CFS] or [Wa] for the definition of entropy). Let us mention that for the geodesic flow ϕt on a compact manifold with constant negative curvature Margulis [Marg] proved the same formula. There has been a significant activity in the last 20 years or so in studying dynamical systems involving billiards of various kinds. Apart from what was mentioned above, we refer the reader to the monographs [ChM], [KozT], [Pla], [Tab] and the articles [BCST], [ChD], [Gut], [Katl], [Kat2], [Kat3], [KP], [Mor], [PlaR], [S5] and the references there for general information. We should note however that our list of references covers only a small percentage of the large number of works in this area.

3

Poisson relation for manifolds with boundary This chapter is devoted to the analysis of the singularities of the distribution  cos λj t, σ(t) = j

where {λ2j }∞ j=1 are the eigenvalues of the Laplacian in a bounded domain Ω with Dirichlet boundary condition on ∂Ω. The main purpose is to establish the so-called Poisson relation for manifolds with boundary, namely that sing supp σ(t) ⊂ {0} ∪ { ± Tγ : γ ∈ LΩ }, where LΩ is the set of all generalized periodic bicharacteristics of  in Ω and Tγ denotes the period of Tγ . In Section 3.1, the fundamental solutions e0 (t, x − y) and h0 (t, x − y) of  and 2 , respectively, are studied. We describe the singularities of e0 and h0 on the diagonal of ∂Ω × ∂Ω. In Section 3.2 we introduce the distribution σ(t) and show that it coincides with the trace of the fundamental solution E(t, x, y) of the Dirichlet problem for . In Section 3.3 the proof of the Poisson relation is reduced to the analysis of the trace of a distribution B(t, x, y), defined by (3.13), on the manifold without boundary ∂Ω. For convex domains, we examine the singularities of B(t, x, y) and those of the trace B(t, x, x), x ∈ ∂Ω. The analysis of the singularities of B(t, x, y) for non-convex Ω leads to some difficulties. For this reason for general domains we study in Section 3.4 separately the singularities of E(t, x, y) for x, y ∈ Ω◦ and those for x, y close to ∂Ω, provided t∈ / { ± Tγ : γ ∈ LΩ }. For this purpose we apply a localization that will be exploited in the next chapter. The advantage of the proof in Section 3.4 is that it works without Geometry of the Generalized Geodesic Flow and Inverse Spectral Problems, Second Edition. Vesselin M. Petkov and Luchezar N. Stoyanov. © 2017 John Wiley & Sons, Ltd. Published 2017 by John Wiley & Sons, Ltd.

58

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

any change for Neumann and Robin boundary conditions, according to the results on propagation of singularities in [MS1] and [MS2].

3.1

Traces of the fundamental solutions of  and 2

Let e0 (t, x − y) be the solution of the problem  (∂t2 − Δx )e0 (t, x − y) = δ(t)δ(x − y), supp e0 (t, x − y) ⊂ {(t, x, y) ∈ Rt × R2n : t ≥ 0}. The second condition implies that the Fourier transform eˆ0 (τ, x − y) of e0 (t, x − y) with respect to t admits an analytic continuation in {τ ∈ C : Im τ < 0}. This implies  −n eix−y,ξ [ξ 2 − (τ − i0)2 ]−1 dξ, eˆ0 (τ, x − y) = (2π) with ξ 2 = ξ, ξ and −n−1

e0 (t, x − y) = (2π)

  lim

→+0

eiτ |ξ| [1 − (τ − i )2 ]−1 eix−y,ξ |ξ|−1 dτ dξ.

For > 0 the theorem of residues yields  ∞ eiτ |ξ| [1 − (τ − i )2 ]−1 dτ = −iπ(eit|ξ|(1+i) − eit|ξ|(i−1) ). −∞

Letting → +0, for t ≥ 0 we obtain  (2π)−n dξ (ei(t|ξ|+x−y,ξ ) − ei(−t|ξ|+x−y,ξ ) ) . e0 (t, x − y) = 2i |ξ| The integral for |ξ| ≥ 1 can be considered as a sum of oscillatory integrals with phase functions ϕ± (t, x, y, ξ) = ±t|ξ| + x − y, ξ (see Proposition 1.3.2). Moreover, by using Theorem 1.3.3, it is easy to find the wave front set W F (e0 (t, x − y)) ⊂ {(t, x, y, τ, ξ, η) ∈ T ∗ (Rt × Rnx × Rny ) \ {0} : t > 0, x = y ∓ t

ξ , τ = ±|ξ|, ξ + η = 0} |ξ|

∪{(t, x, y, τ, ξ, η)} ∈ T ∗ (Rt × Rnx × Rny ) \ {0} : t = 0, x = y, ξ + η = 0}. (3.1) Now let Ω ⊂ Rn be a bounded closed domain with smooth boundary ∂Ω. We wish to define the trace f0 (t, x − y) = e0 (t, x − y)|Rt ×∂Ω×∂Ω ,

POISSON RELATION FOR MANIFOLDS WITH BOUNDARY

59

and to describe W F (f0 (t, x − y)). To do this, consider the inclusion map j : Rt × ∂Ω × ∂Ω → Rt × Rnx × Rny , j(t, x, y) = (t, x, y). The normal set Nj of j has the form Nj = {(t, x, y, τ, ξ, η) ∈ T ∗ (Rt × Rnx × Rny ) : x ∈ ∂Ω, y ∈ ∂Ω, τ = 0, ξ|Tx (∂Ω) = η|Ty (∂Ω) = 0}. Obviously, for R+ t = {t ∈ R : t > 0} we have   W F e0 (t, x − y)|R+t ×∂Ω×∂Ω ∩ Nj = ∅. According to Theorem 1.3.6, we can define f0 (t, x − y) for t > 0 by setting f0 (t, x − y) = j ∗ e0 (t, x − y), j ∗ being the pull-back of j. The same procedure works for t = 0, x = y. The definition of f0 (t, x − y) for t = 0, x = y, x ∈ ∂Ω is more difficult, since the set Nj contains some points lying over the set {(0, x, x) ∈ Rt × ∂Ω × ∂Ω}. To cover this case we make a more precise analysis of f0 for (t, x, y) close to (0, x0 , x0 ), x0 ∈ ∂Ω. For x ∈ Ω◦ , y ∈ ∂Ω, we have  −n eix−y,ξ (ξ 2 − τ 2 )−1 dξ, Im τ < 0. eˆ0 (τ, x − y) = (2π) To find the limit of the right-hand side as x → x0 , Im τ → −0, we introduce near x0 ∈ ∂Ω local coordinates (x1 , x ), x = (x2 , . . . , xn ) so that in a neighbourhood of x0 , the domain Ω is given by x1 ≥ g(x ), while ∂Ω has the form x1 = g(x ), g being a smooth function. Let H(x , y  ) ∈ C ∞ (Rn−1 , Rn−1 ) be a smooth vector-valued function such that g(x ) − g(y  ) = x − y  , H(x , y  ) with H(x , x ) = dg(x ). Here and below ·, · denotes the standard inner product in Rn−1 . The metric on T (∂Ω), inherited from the standard Euclidean one in Rn , has the form ν  , H(x , y  ) 2 . m2 (x , y  ; ν  ) = ν  , ν  − 1 + |H(x , y  )|2 In the local coordinates (x1 , x ), for x ∈ Ω◦ , y ∈ ∂Ω, we get   1/2     eˆ0 (τ, x, y  ) = (2π)−n eix −y ,ξ eiξ1 (x1 −g(x )) 1 + |dg(y  )|2 (ξ 2 − τ 2 )−1 dξ. (3.2)

60

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Here the factor (1 + |dg(y  )|2 )1/2 appears, since in the coordinates (z1 , y  ) with z1 = y1 − g(y  ) we have δ∂Ω = (1 + |dg(y  )|2 )1/2 ⊗ δ(z1 ). Next, change the variables  ζ1 = ξ1 , ζ  = ξ  + ξ1 H(x , y  ). Then (3.2) becomes 

−n



eˆ0 (τ, x, y ) = (2π)







eix −y ,ξ +ρξ1 (1 + |dg(y  )|)1/2 (ξ 2 − τ 2 )dζ1 dζ ,

with ρ = x1 − g(y  ) and ξ 2 − τ 2 = (ζ  )2 − 2ζ1 ζ  , H + (1 + |H|2 )ζ12 − τ 2 . The roots z± of the equation ξ 2 − τ 2 = 0 with respect to ζ1 have the form z± =

1/2  2 ζ  , H m − τ2 ± i . 1 + |H|2 1 + |H|2

Here we choose the square root so that Re (m2 − τ 2 )1/2 > 0. Hence Im z+ > 0, and by the theorem of residues we get  ∞  −1 eiρζ1 (1 + |H|2 )(ζ1 − z+ )(ζ1 − z− ) dζ1 −∞

 −1/2 = πeiρζ+ (1 + |H|2 )(m2 − τ 2 ) . Letting ρ → +0, for x ∈ ∂Ω, y  ∈ ∂Ω, we obtain     (2π)−n−1 eix −y ,ζ (1 + |dg(y  )|2 )1/2 eˆ0 (τ, x , y  ) = dζ  . 2 [(1 + |H(x , y  )|2 )(m2 (x , y  ; ζ  ) − τ 2 )]1/2 The form m2 (x , y  ; ζ  ) is positively definite, so there exists a positively definite symmetric matrix Q(x , y  ) such that m2 (x , y  ; ζ  ) = |Q(x , y  )−1 ζ  |2 . Changing the variables ζ  → Q(x , y  )ξ  , we deduce       2     eˆ0 (τ, x , y ) = F (x , y ) eiQ(x ,y )(x −y ),ξ (ξ  − (τ − i0)−1/2 dξ  with F (x , y  ) ∈ C ∞ .

POISSON RELATION FOR MANIFOLDS WITH BOUNDARY

61

Taking the inverse Fourier transform in τ , we have to examine the integrals        eitτ +Q(x ,y )(x −y ),ξ [ξ  − (τ − i0)2 ]−1/2 dτ dξ  I1 + I2 = |τ |≤|ξ  |+1

 



+ |τ |>|ξ  |+1









eitτ +Q(x ,y )(x −y ),ξ [ξ  − (τ − i0)2 ]−1/2 dτ dξ  .

Setting τ = μ|ξ  |, we may consider I1 , modulo smooth terms, as an oscillatory integral with a phase function Ψ1 = tμ|ξ  | + Q(x , y  )(x − y  ), ξ  , since Ψ1,x = 0 for x = y  , |ξ  | ≥ 1. On the other hand, we can treat I2 as an oscillatory integral with a phase function Ψ2 = tτ + Q(x , y  )(x − y  ), ξ  , since |Ψ2,t | + |Ψ2,x | = 0 for x = y  . Consequently, we may define e0 (t, x − y)|Rt ×∂Ω×∂Ω for t = 0, and the analysis of the wave front sets of the integrals I1 , I2 yields W F (e0 (t, x − y)|Rt ×∂Ω×∂Ω ) ∩ {t = 0} ⊂ {(t, x , y  , τ, ξ  , ξ  ) ∈ T ∗ (Rt × ∂Ω × ∂Ω) \ {0} : t = 0, x = y  , ξ  + η  = 0}.

(3.3)

In a similar way one examines the distribution h0 (t, x − y) determined as the solution of the problem  (∂t2 − Δx )2 h0 (t, x − y) = δ(t)δ(x − y), supp h0 (t, x − y) ⊂ {(t, x, y) ∈ Rt × R2n : t ≥ 0}. ˆ (τ, x − y) as Write the Fourier transform h 0  ˆ (τ, x − y) = (2π)−n eix−y,ξ [ξ 2 − (τ − i0)2 ]−2 dξ. h 0 Then for t ≥ 0 we obtain the oscillatory integral h0 (t, x − y) =

i(2π)−n 2



[eiϕ+ (it|ξ| − 1) + eiϕ− (it|ξ| + 1)]|ξ|−3 dξ.

This implies a relation completely analogous to (3.1). Thus for t > 0 and for t = 0, x = y, we can define F1 (t, x − y) = j ∗ h0 (t, x − y).

62

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

For the trace on t = 0 and x = y, we repeat the above procedure. For Im τ < 0 and x , y  ∈ ∂Ω, one obtains  ∞ π eiρξ1 (ξ 2 − τ 2 )−2 dξ1 = (1 + |H|2 )−1/2 (m2 − τ 2 )−3/2 lim ρ→+0 −∞ 2 and we deduce fˆ1 (τ, x , y  ) = F1 (x , y  )













eiQ(x ,y )(x −y ),ξ (ξ  − (τ − i0)2 )−3/2 dξ  2

with F1 (x , y  ) ∈ C ∞ . It remains to study two oscillatory integrals, similar to I1 and I2 . The analysis is completely analogous to the previous one and we leave the details to the reader. Finally, combining the action of the pull-back j ∗ with (3.3), we obtain the following. Theorem 3.1.1: The wave front sets of the distributions fk (t, x − y) ∈ D (Rt × ∂Ω × ∂Ω), k = 0, 1, are contained in the set of points ˜ η˜) ∈ T ∗ (R × ∂Ω × ∂Ω) \ {0} (t, x , y  , τ, ξ, t satisfying the following conditions: (i) t > 0 and there exists ξ ∈ Rn \ {0} such that x = y  ∓ tξ/|ξ|, τ = ±|ξ|, ξ˜ = px (ξ), ξ˜ + η˜ = 0; (ii) t = 0, x = y  , ξ˜ + η˜ = 0. Here px : Tx∗ (Rn ) −→ Tx∗ (∂Ω) is the canonical projection.

3.2

The distribution σ (t)

Let Ω ⊂ Rn , n ≥ 2, be a bounded closed domain with C ∞ smooth boundary ∂Ω and Ω◦ = ∅. Let H01 (Ω) be the closure of the space C0∞ (Ω) with respect to the norm u21 =



∂ α u2 ,

|α|≤1

. being the norm in L2 (Ω). Introduce the operator A by Au = −Δu for u ∈ C0∞ (Ω), and extend it to the domain DA = {u ∈ H01 (Ω) : Δu ∈ L2 (Ω)}, where Δu is interpreted in the sense of distributions. Let (·, ·) be the inner product in L2 (Ω). Clearly (Au, u) = (u, Au) for u, v ∈ DA .

POISSON RELATION FOR MANIFOLDS WITH BOUNDARY

63

Thus, A is symmetric and closed operator in L2 (Ω). Moreover, ((A + 1)u, u) =u2 , u ∈ DA , and the form A(u, u) = ((A + 1)u, u) is continuous and coercive in H01 (Ω). Consequently, by Lax–Milgram theorem for each f ∈ L2 (Ω), there exists an unique solution u ∈ H01 (Ω) of the problem −Δu + u = f in D (Ω). This implies that A + 1 is a bijection from DA into L2 (Ω) and (A + 1)−1 f H 1 (Ω) ≤ C f, ∀f ∈ L2 (Ω).

(3.4)

Hence, −1 is not in the spectrum of A, and, by the well-known criteria for self-adjointness, A is a self-adjoint operator in L2 (Ω), related to the Laplacian −Δ with Dirichlet boundary condition on ∂Ω. The estimate (3.4) shows that the resolvent (A + 1)−1 is compact in L2 (Ω) and we deduce that the spectrum of A is formed by an infinite number of eigenvalues 0 < λ21 ≤ λ22 ≤ . . . ≤ λ2m ≤ . . . with finite multiplicities. Let {ϕj (x)}∞ j=1 be an orthonormal set of eigenfunctions of A so that  −Δϕj (x) = λ2j ϕj (x), x ∈ Ω, ϕj (x) = 0, x ∈ ∂Ω. Therefore, ϕj (x) ∈ C ∞ (Ω), and we can introduce the spectral function 

e(x, y, λ) =

ϕj (x)ϕj (y),

λ2j ≤λ2

which is the kernel of the spectral projection Eλ of A. Moreover, we have the estimate sup {|e(x, y, λ)| : x, y ∈ Ω} ≤ Cλn , λ ≥ 1

(3.5)

with a constant C > 0 independent of λ. We refer to Hörmander [[H3], Section 17.5] for the proof of (3.5). Let N (λ) = #{j : λ2j ≤ λ2 } be the counting function of eigenvalues. Clearly,  e(x, x, λ2 )dx,

N (λ) = Ω

and (3.5) yields N (λ) ≤ C1 λn , λ −→ +∞.

64

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Introduce the tempered distribution σ(t) =

∞ 

cos λj t = Re

j=1

∞ 

exp (λj t) ∈ S  (R).

j=1

Since A is a non-negative self-adjoint operator, by the spectral calculus we may define the operator cos(A1/2 t). Let E(t, x, y) ∈ D (R × Ω × Ω) be the kernel of cos(tA1/2 ). Therefore,  ∞   cos(λt) e(x, y, λ2 )f (x)f (y)dx dy, (cos(tA1/2 )f, f ) = Ω

0

Ω

and a simple calculus implies E(t, x, y) =

∞ 

(cos λj t)ϕj (x)ϕ(y).

j=1



Hence

E(t, x, y)dx

σ(t) =

(3.6)

Ω

and E(t, x, y) is a solution of the problem ⎧ 2 ◦ ◦ ⎪ ⎨(∂t − Δx )E(t, x, y) = 0, t ∈ R, x ∈ Ω , y ∈ Ω , E(t, x, y)|x∈∂Ω = E(t, x, y)|y∈∂Ω = 0, ⎪ ⎩ E(0, x, y) = δ(x − y), ∂t E(0, x, y) = 0.

(3.7)

Following the results for the propagation of singularities in [[MS2], [H3]] (see Theorem 1.4.2), we can describe the singularities of E(t, x, y). Since the trace defining σ(t) is taken over a manifold with boundary, Theorems 1.3.8 and 1.3.9, concerning the calculus with wave front sets, cannot be applied immediately. To describe the singularities of σ(t), in the next section we find another representation of σ(t) involving integration over ∂Ω.

3.3

Poisson relation for convex domains

As in the previous section, Ω is a compact domain in Rn , n ≥ 2. Our first argument concerns the general case and we do not assume that Ω is convex. Let ∞  λ−1 E(t, x, y) = j sin(λj t)ϕj (x)ϕj (y) j=1

POISSON RELATION FOR MANIFOLDS WITH BOUNDARY

65

be the kernel of the operator A−1/2 sin(tA1/2 ). Clearly, E(t, x, y) = ∂t E(t, x, y) and E(t, x, y) is the solution of the problem ⎧ 2 ◦ ◦ ⎪ ⎨(∂t − Δx )E(t, x, y) = 0, t ∈ R, x ∈ Ω , y ∈ Ω , E(t, x, y)|x∈∂Ω = 0, ⎪ ⎩ E(0, x, y) = 0, ∂t E(0, x, y) = δ(x − y). Next, we examine the trace

 E(t, x, x)dx.

For y ∈ ∂Ω introduce the distribution K(t, x, y) as the solution of the problem ⎧ 2 ◦ ◦ ⎪ ⎨(∂t − Δx )K(t, x, y) = 0, t ∈ R, x ∈ Ω , y ∈ Ω , (3.8) K(t, x, y) = δ(t) ⊗ δ(x − y), x ∈ ∂Ω, ⎪ ⎩ supp K(t, x, y) ⊂ {t, x, y) ∈ Rt × Ω × ∂Ω : t ≥ 0}. Notice that K(t − s, x, y) is the kernel of a continuous operator ¯  (R × Ω). K : C0∞ (R × ∂Ω)  f (s, y) → (Kf )(t, x) ∈ D Here (Kf )(t, x) is the solution of the problem ⎧ 2 ◦ ⎪ ⎨(∂t − Δx )Kf = 0 in R × Ω , Kf − f = 0, on R × ∂Ω, ⎪ ⎩ Kf |t 0. First, notice that sing supp e0 (t, x − y) ⊂ {(t, x, y) ∈ Rt × Rnx × Rny : |x − y| = t}.

(3.11)

POISSON RELATION FOR MANIFOLDS WITH BOUNDARY

67

This follows from the propagation of singularities of the solution of Cauchy problem satisfied by e0 (t, x − y). Consequently, for t > 0 we have  e0 (t, 0)dx ∈ C ∞ (R+ ). Ω

Thus, for t > 0, modulo C ∞ terms, we obtain    ∞

x, x)dx = − E(t, x, x)dx = − E(t, Ω

∞

Ω

=−

 t 0

K(t − s, x, z)e0 (s, z − x)dsdz

∂Ω

K(t − s, x, z)(∂s2 − Δx )h0 (s, z − x)dsdz,

∂Ω

where h0 (t, x − y) is the fundamental solution of 2 , introduced in Section 3.1 and extended as 0 for t < 0. Integrating by parts with respect to s and x, we find   t  

x, x)dx = ds dz [∂νx K(t − s, x, z)h0 (s, z − x) E(t, 0

Ω

∂Ω

∂Ω

−K(t − s, x, z)∂νx h0 (s, z − x)]dx.

(3.12)

Here, ∂νx denotes the derivative with respect to the exterior normal νx at x ∈ ∂Ω. On the other hand, sing supp h0 (t, x − y) satisfies the relation (3.11) since h0 (t, x − y) is a solution of the Cauchy problem ⎧ + 2 2 n ⎪ ⎨(∂t − Δx ) h0 (t, x − y) = 0, t ∈ Rt , x, y ∈ R , j ∂t h0 (0, x − y) = 0, j = 0, 1, 2, ⎪ ⎩ 3 ∂t h0 (0, x − y) = δ(x − y), where h0 (t, x − y) is extended as 0 for t < 0. Therefore, the second term in the right-hand side of (3.12) becomes   t  ds dz (∂νx h0 (t, 0)dx ∈ C ∞ (R+ t ). 0

∂Ω

∂Ω

The boundary ∂Ω is not characteristic for the operator  and K(t, x, y) satisfies (3.8). We can apply the partial hypoellipticity of K(t, x, y) with respect to the normal direction νx to introduce the trace k(t, x, y) = ∂νx K(t, x, z)|(x,z)∈∂Ω×∂Ω . Applying Theorem 3.1.1, the wave front W F (h0 (t − s, z − y)|(y,z)∈∂Ω×∂Ω satisfies a relation analogous to (3.10) and we can define the distribution  ∞ B(t, x, y) = − k(s, x, z)h0 (t − s, z − y)dsdz, ∞

(3.13)

∂Ω

¯  (R × ∂Ω × ∂Ω). interpreted as the action of h0 (t − s, z − y) on k(s, x, z) ∈ D s

68

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Consider B(t, x, y) for t > 0, x, y ∈ ∂Ω, |x − y| < t. A finite speed of propagation argument yields sing supp k(t, x, z) ⊂ {(t, x, z) ∈ R × ∂Ω × ∂Ω : |x − z| ≤ t}. Combining this with the information for sing supp h0 (t − s, z − y), we conclude that the integrand in (3.13) is singular only if (t, s, x, y, z) satisfies t > |x − y| ≥ |z − y| − |x − z| ≥ t − 2s. To describe W F (B(t, x, y)|t>0, |x−y| 0. Recall that the relation C, introduced in Section 1.2, is the set of points (t, x, y, τ, ξ, η) ∈ T ∗ (R × Ω × Ω) \ {0}, such that τ 2 = |ξ|2 , and (t, x, τ, ξ) and (0, y, τ, η) lie on a generalized bicharacteristic of . According to Lemma 1.2.7, the relation C is closed. Denote by Cb the set of those ˜ η˜) ∈ T ∗ (R × ∂Ω × ∂Ω) \ {0} (t, x, y, τ, ξ, such that there exist ξ ∈ Tx∗ (Ω), η ∈ Ty∗ (Ω) with ξ˜ = px (ξ), η˜ = py (η) and (t, x, y, τ, ξ, η) ∈ C. Here, px is the projection, introduced at the end of Section 3.1. Repeating the proof of Lemma 1.2.7, we conclude that Cb is closed. Proposition 3.3.1: We have   W F  k(t, x, y)|t>0 ⊂ Cb .

(3.14)

Proof: It is convenient to introduce the set C˜b of those ˜ η˜) ∈ T ∗ (R × R × ∂Ω × ∂Ω) \ {0} (t, s, x, y, τ, σ, ξ, t s ˜ p (η) such that τ = σ, and there exist ξ ∈ Tx∗ (Ω), η ∈ Ty∗ (Ω) with px (ξ) = ξ, y 2 2 2 = η˜, τ = |ξ| = |η| such that (t, x, τ, ξ) and (s, y, τ, η) lie on a generalized bicharacteristic of . We will prove that   (3.15) W F  k(t − s, x, y)|t>s ⊂ C˜b and (3.14) follows from (3.15) taking s = 0. / C˜b with t0 > s0 . Since Consider an arbitrary ρ0 = (t0 , s0 , x0 , y0 , τ0 , σ0 , ξ˜0 , η˜) ∈ τ = σ = 0 on W F  (k(t − s, ·, ·)), we may assume τ0 = σ0 = 0. The relation C˜b is closed, so there exist open conic neighbourhoods Γ1 of (s0 , y0 , τ0 , ξ˜0 )

POISSON RELATION FOR MANIFOLDS WITH BOUNDARY

69

˜ ∈ Γ imply and Γ2 of (t0 , x0 , τ0 , ξ˜0 ) so that (s, y, σ, η˜) ∈ Γ1 and (t, x, τ, ξ) 2 ˜ ˜ (t, s, x, y, τ, σ, ξ, η˜) ∈ / Cb . Let π : T ∗ (R × Ω) −→ R be the natural projection. We may choose Γ1 and Γ2 in a such a way that π(Γ1 ) = (α, β), π(Γ2 ) = (γ, δ), β < γ. Let A1 ∈ L0 (R × ∂Ω) be a pseudo-differential operator on R × ∂Ω with full symbol equal to 1 in a small conic neighbourhood of (s0 , y0 , τ0 , η˜0 ), and with wave front set W F (A1 ) ⊂ Γ1 . We choose A1 so that the kernel of A1 has a compact support, contained in (α, β) × ∂Ω × (α, β) × ∂Ω. In a similar way we choose a pseudo-differential operator A2 ∈ L0 (R × ∂Ω) with full symbol equal to 1 in a small conic neighbourhood of (t0 , x0 , τ0 , ξ˜0 ) such that W F (A2 ) ⊂ Γ2 and the kernel of A2 has a compact support, contained in R × ∂Ω × R × ∂Ω. Fix p ≥ 2. Then for each f ∈ Hploc (R × ∂Ω) we have A1 f ∈ Hp (R × ∂Ω), supp A1 f ⊂ (α, β) × ∂Ω. By Theorem 1.4.1, there exists a solution (SA1 f )(t, x) of the problem ⎧ 2 ◦ ⎪ ⎨(∂t − Δx )SA1 f = 0 in R × Ω , SA1 f − A1 f = 0 on R × ∂Ω, ⎪ ⎩ (SA1 f )|t0, |x−y|0, x∈∂Ω ) ˜ η˜) ∈ C }. ⊂ {(t, x, τ, τ, ξ˜ − η˜) ∈ T ∗ (R × ∂Ω) \ {0} : (t, x, x, τ, ξ, b Finally, we apply Theorem 1.3.8 for the integral over small open sets ωj covering ∂Ω and deduce that   B(t, x, x)dx|t>0 WF ∂Ω

is contained in the set ∗ ˜ ˜ ˜ {(t, τ ) ∈ T ∗ (R+ t ) \ {0} : ∃(x, ξ) ∈ T (∂Ω) with (t, x, x, τ, ξ, ξ) ∈ Cb }.

On the other hand, as we have mentioned in Section 1.2, (T, x, x, τ, ξ, ξ) ∈ C means that there exists a periodic generalized bicharacteristic of  with period T passing through (x, ξ). Thus, we obtain the following. Theorem 3.3.2: For every compact convex domain Ω in Rn , n ≥ 2, with C ∞ smooth boundary, we have sing supp σ(t) ⊂ {0} ∪ { ± Tγ : γ ∈ LΩ },

(3.18)

where LΩ is the set of all periodic generalized bicharacteristics of  in Ω.

3.4

Poisson relation for arbitrary domains

We consider the same problem as in the previous section treating arbitrary domains Ω. If we follow the proof in the convex case, we must establish the inclusion (3.17). Since the intersection of straight lines, issued from y and Ω, could be non-convex, we are going to consider the generalized bicharacteristics passing through all points (s, z) ∈ R × ∂Ω such that (t − s, z, y) ∈ sing supp h0 (t − s, z − y)|y∈∂Ω,z∈∂Ω . This leads to some singularities that are not described by the relation Cb . Throughout this section we apply the notation from the previous ones. The following assertion can be established repeating the proof of Proposition 3.3.1. We leave the details to the reader. ◦ Proposition 3.4.1: The intersection W F  (K(t, x, z) ∩ T ∗ (R+ t × Ω × ∂Ω) is contained in the set of all points

˜ ∈ T ∗ (R+ × Ω◦ × ∂Ω) \ {0} (t, x, z, τ, ξ, ζ) t

72

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

˜ τ 2 = |ξ|2 = |ζ|2 and (t, x, τ, ξ) and such that there exists ζ ∈ Tz∗ (Ω) with pz (ζ) = ζ, (0, z, τ, ζ) lie on a common generalized bicharacteristic of . In the following we examine E(t, x, y) for t > 0 and |x − y| < t. First, we deal with the case x, y ∈ Ω◦ . Proposition 3.4.2: We have the inclusion W F  (E(t, x, y)|t>0,|x−y| 0, x0 , y0 ∈ Ω◦ , |x0 − y0 | < t0 . Notice that for t > 0, x, y ∈ Ω◦ we have (∂t2 − Δx )E(t, x, y) = 0, (∂t2 − Δy )E(t, x, y) = 0, consequently τ 2 = |ξ|2 = |η|2 . In particular, τ02 = |ξ0 |2 = |η0 |2 if q0 ∈ W F  (E(t, x, y)). We assume that τ0 < 0, the case τ0 > 0 is treated similarly. Choose small conic neighbourhoods Γ1 = V1 × Σ1 of (y0 , η0 ) and Γ2 = V2 × Σ2 of (x0 , ξ0 ) and δ0 > 0 such that (y, η) ∈ Γ1 , (x, ξ) ∈ Γ2 , |t − t0 | < δ0 imply (t, x, y, τ, ξ, η) ∈ / C for τ 2 = |ξ|2 = |η|2 . We may take Vi so that V¯i ⊂ Ω◦ , i = 1.2. Let A1 (y, Dy ) ∈ L0 (Ω◦ ) be a pseudo-differential operator, the full symbol a1 (y, η) of which equals to 1 in some conic neighbourhood of (y0 , η0 ), the kernel of A1 has a compact support in Ω◦ × Ω◦ and W F (A1 ) ⊂ γ1 . Consider the straight line L0 = {z ∈ Rn : z = y0 + ση0 , σ ∈ R}, and let L0 ∩ Ω = ∪j∈P lj , where lj = [xj , xj+1 ], xj ∈ ∂Ω, lj ∩ lk = ∅ for k = j and P ⊂ Z ∩ [−p, q], p, q being non-negative integers or ∞. We may assume that y0 ∈ l0 . Notice that in general l0 ∩ ∂Ω could contain non-trivial segments lying on ∂Ω. Let j ∈ P and xj = y0 + σj η0 /|η0 |. Assume that −2, 2 ∈ P . Then 0 ≤ σ1 < σ2 , σ−1 < σ0 ≤ 0. Choose  < (σ2 − σ1 )/2,  < (σ0 − σ−1 )/2 and two functions κ(t), χ(t) ∈ C ∞ (R) such that  1 for t ≥ σ0 −  , κ(t) = 0 for t ≤ σ−1 +  ,  1 for t ≤ σ1 +  , χ(t) = 0 for t ≥ σ2 −  .

POISSON RELATION FOR MANIFOLDS WITH BOUNDARY

73

Now set e1 (t, x, y) = A1 (y, Dy )e0 (t, x − y), e2 (t, x, y) = e0 (t, x, y) − e1 (t, x, y), e˜1 (t, x, y) = κ(t)χ(t)A1 (y, Dy )e0 (t, x − y). Notice that (∂t2 − Δx )(e1 − e˜1 )(t, x, y) = 0 implies σ1 +  ≤ t ≤ σ2 −  or σ−1 +

 ≤ t ≤ σ0 −  . For sufficiently small Γ1 and  we obtain e1 + e2 ) ∈ C ∞ (R × Ω × Ω), (∂t2 − Δx )(˜ (˜ e1 + e2 )(0, x, y) = 0, ∂t (˜ e1 + e2 )(0, x, y) − δ(x − y) ∈ C ∞ (Ω × Ω). This implies  E(t, x, y) − (˜ e1 + e2 )(t, x, y) +

∞

∞

 K(s, x, z)(˜ e1 + e2 ) ∂Ω

×(t − s, z, y)dsdz ∈ C ∞ (R × Ω × Ω).

(3.20)

For |x − y| < t the sum (˜ e1 + e2 )(t, x, y) is C ∞ smooth, so we have to examine the integral  ∞

(t, x, y) = K(s, x, z)˜ e (t − s, z, y)dsdz (3.21) E 1

∞

1

∂Ω

and a similar one with e2 instead of e˜1 . Applying Theorem 1.3.9, we deduce

(t, x, y)) ∩ {(t, x, y, τ, ξ, η) ∈ T ∗ (R+ × V × V ) : |x − y| < t} W F  (E 1 2 1     ⊂ W F  K(s, x, z)|x∈V2 ,z∈∂Ω ◦ W F  e˜1 (t − s, z, y)|y∈V1 ,z∈∂Ω , (3.22) where the composition of wave front sets is taken with respect to s and z. Now, it is ˆ ∈

(t, x, y)). Indeed, assume that for some (ˆ / W F  (E s, zˆ, σ ˆ , ζ) easy to see that q0 ∈ 1 T ∗ (R × ∂Ω), we have ˆ η ) ∈ W F  (˜ ˆ , ζ, e1 (t − s, z, y)), (t0 , sˆ, zˆ, y0 , τ0 , σ 0 ˆ ∈ W F  (K(s, x, z)). ˆ , ξ0 , ζ) (ˆ s, x0 , zˆ, σ First, assume that t − sˆ > 0. Then σ ˆ = τ0 , zˆ = y0 + (t0 − sˆ)

η0 ˆ , ζ = pzˆ (η0 ) |η0 |

and the construction of e˜1 yields zˆ ∈ l0 . Thus, the generalized bicharacteristic issued ˆ |ζ|2 = τ 2 is a part of a generalized bicharfrom (t0 − sˆ, zˆ, τ0 , ζ) with pzˆ (ζ) = ζ, 0 acteristic issued from (0, y0 , τ0 , η0 ) and passing through (t0 , x0 , τ0 , ξ0 ). This is a contradiction with the choice of q0 . The case t0 = sˆ is handled by a similar argument.

74

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

The analysis of the integral involving e2 is trivial and we conclude that / W F  (E(t, x, y)), q0 ∈ which completes the proof of the proposition.



Now we pass to the analysis of the singularities of E(t, x, y) for x and y close to some point z0 ∈ ∂Ω. In an open neighbourhood U of z0 , we choose local coordinates x = (x1 , . . . , xn−1 , xn ) = (x1 , x ), x1 = dist(x, ∂Ω) such that ∂Ω ∩ U is given by x1 = 0 and Ω ∩ U lies in the half-space x1 ≥ 0. As in Section 1.2, in these local coordinates the principal symbol of ∂t2 − Δx has the form q(x, τ, ξ) = ξ12 − q2 (x1 , x , ξ  ) − τ 2 , where ξ  = (ξ2 , . . . , ξn ) and −q2 (x, ξ  ) is a positively definite quadratic form in ξ  . For t > 0 we have (∂t2 − Δx )E(t, x, y) = 0 and the boundary ∂Ω is non-characteristic for ∂t2 − Δx . Therefore, the partial hypoellipticity of E(t, x, y) implies that E(t, x, y)|t>0 is a C ∞ smooth function of n−1 × Ω). Since the same x1 with values in the space of distributions D (R+ t × Rx  argument works for the variables y = (y1 , y ), for sufficiently small > 0 we obtain a C ∞ function H : [0, ] × [0, ] −→ D (R+ × U  × U  ), H(x1 , y1 ) = E(t, x1 , x , y1 , y  ), where U  ⊂ Rn−1 is a small neighbourhood of 0. For this purpose we exploit the smoothness of E with respect to y1 and we apply Theorem B.2.9 in [H3], considering y1 as a parameter. The proof of the later theorem can be trivially extended to cover a smooth dependence on a parameter and we deduce the above assertion. Let LΩ be the set of periods of all periodic generalized bicharacteristics of  in Ω. According to Lemma 1.2.10, the set LΩ is closed in R. / LΩ . Then for sufficiently small > 0 we have Proposition 3.4.3: Let t0 > 0, t0 ∈ / W F  (H(x1 , y1 )), (t0 , x , x , τ, ξ  , ξ  ) ∈ whenever (x1 , y1 ) ∈ [0, ] × [0, ], τ ∈ R and (x , ξ  ) ∈ U × Rn−1 . The proof of this proposition is long, and we begin with some preparations. The main difficulty is to provide uniformity with respect to x1 and y1 . We follow the idea of the proof of Proposition 3.4.2, by studying the singularities of a distribution similar

(t, x, y). If to E 1 (t, x , y  , τ, ξ  , η  ) ∈ W F  (H(x1 , y1 )), t > 0, then the point (t, x , τ, ξ  ) belongs to the compressed characteristic set Σb of ∂t2 − Δx , with respect to the surface x1 = const, introduced in Section 1.2. This means that the equation q(x, τ, ξ) = ξ12 − q2 (x1 , x , ξ  ) − τ 2 = 0 with respect to ξ1 has only real roots and this implies −q2 (x1 , x , ξ  ) ≤ τ 2 . Therefore, τ = 0 leads to ξ  = 0 and η  = 0. Hence we may assume τ = 0.

POISSON RELATION FOR MANIFOLDS WITH BOUNDARY

75

Consider a point ρ0 = (t0 , x0 , x0 , τ0 , ξ0 , ξ0 ) with τ0 < 0, −q2 (0, x0 , ξ0 ) ≤ τ02 , (x0 , ξ0 ) ∈ U  × Rn−1 . The case τ0 > 0 can be treated by a similar argument. Below we choose 0 < < t0 /4 and obtain (t0 , x1 , x0 , y1 , y0 ) ∈ / sing supp e0 (t, x − y) for x1 , y1 ∈ [0, ]. For technical reasons it is more convenient to study the distribution Ξ(t − t , x, y) = E(t − t , x, y)Y (t − t ),  1, t ≥ 0, Y (t) = 0, t < 0,

where

is the Heaviside function, and to take the trace t = 0. Let A1 (t , y, Dt , Dy ) ∈ L0 (Rt × Rn−1 y ) be a zero order pseudo-differential operator depending smoothly on y1 ∈ [0, ] such that the full symbol a1 (t , y, τ  , η  ) of A1 is equal to 1 in some conic neighbourhood ˜ be another conic neighbourhood of ˜ of (0, x , τ , ξ  ) for y ∈ [0, ]. Let Γ ⊃ Γ Γ 1 1 1 1 0 0 0 the same point and let W F (A1 (·, y1 , ·)) ⊂ Γ1 . Moreover, we assume that the kernel of A1 (·, y1 , ·) has a compact support in T × U  × T × U  , T being a small neighbourhood of 0 in R. Consider the equation (3.23) q(0, x0 , τ0 , ξ1 , ξ0 ) = 0 with respect to ξ1 . First, we treat the case when this equation has a double real root ξ10 . Set y0 = (0, x0 ), η0 = (ξ10 , ξ0 ) and consider the line L0 and the linear segments lj introduced in the proof of Proposition 3.4.2. Next, define e1 (t − t , x, y) = A1 (t , y, Dt , Dy )e0 (t − t , x, t), e2 (t − t , x, y) = e0 (t − t , x, y) − e1 (t − t , x, y), e˜1 (t − t , x, y) = κ(t − t )χ(t − t )e0 (t − t , x, y), where κ(t) and χ(t) are the functions introduced in the proof of Proposition 3.4.2. Thus, for Γ1 , T and sufficiently small we are going to study ˜ (t − t , x, y) = Ξ 1



∞

∞

 ∂Ω

K(s, z, z)˜ e1 (t − t − s, z, y)dsdz.

76

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

˜ (t − t , x, y) we need an inclusion similar to that in PropoFor the distribution Ξ 1 sition 3.4.1 for x close to ∂Ω. For this purpose by using the fact that ∂Ω is not characteristic for , for t > 0 and for small > 0, one introduces the C ∞ function   K : [0, ] −→ D (R+ t × U × ∂Ω), K(x1 ) = K(t, x1 , x , z).

We have the following. Proposition 3.4.4: For sufficiency small > 0 and x1 ∈ [0, ], W F  (K(x1 )) is con˜ ∈ T ∗ (R+ × U  × ∂Ω) \ {0} such that tained in the set of all points (t, x , z, τ, ξ  , ζ) t ∗ ˜ τ 2 = |ζ|2 , q(x , x , τ, ξ , ξ  ) = 0, there exist ζ ∈ Tz (Ω) and ξ1 ∈ R with pz (ζ) = ζ, 1 1 and (0, z, τ, ζ) and (t, x1 , x , τ, ξ1 , ξ  ) lie on a common generalized bicharacteristic of . Proof: For x ˆ1 > 0 the result follows from Proposition 3.4.1 taking the trace on x1 = ˆ1 = 0 the proof is a modification of that of Proposition 3.3.1. By using the x ˆ1 . For x notation of this proof, it suffices to apply the results in [MS2] or [H3], Section 24, concerning the propagation of the generalized wave front set W Fb (SAf ) introduced in Section 1.4. Then for some pseudo-differential operator B0 (t, x1 , x , Dt , Dx ) ∈ L0 (Rt × Rn−1 x ), depending smoothly on x1 , we obtain a linear map Hploc (R × ∂Ω)  f −→ B0 SAf ∈ C ∞ (Rt × Rn ), and we apply the argument of the proof of Proposition 3.3.1.



Consider the inclusion   ˜ (t − t, x , x , y , y  ) ∩ T ∗ (R+ × R  × U  × U  ) WF Ξ t 1 1 1 t     ⊂ W F  K(s, x , xn , z)|x ∈U  ,z∈∂Ω ◦ W F  e˜1 (t − t − s, z, y1 , y  )|y ∈U  ,z∈∂Ω , (3.24) where x1 , y1 are parameters. Observe that for Γ1 , T and sufficiently small, the e1 (t − t − s, z, y1 , y  )) has a projection on T ∗ (Rt × Rn ), which wave front set W F  (˜ is related to the straight lines issued from y with direction η, (y, η) being close to (y0 , η0 ). The precise choice of will be discussed below. Introduce    η0 : 0 ≤ σ ≤ σ1 . σ, y0 + σ l0 (σ) = |η0 | Recall that the set Ct (μ) is formed by points ν ∈ T ∗ (R × Ω) such that there exists a generalized bicharacteristic Γ(t) of  with γ(0) = μ, γ(t) = ν. Consider the metric D(ρ, μ) defined in Section 1.2 and recall that D(ρ, μ) = 0 implies ρ = μ or ρ = (x, ξ), μ = (x, η) with x ∈ ∂Ω, px (ξ) = px (η). We denote by γ(t; μ) one of the

POISSON RELATION FOR MANIFOLDS WITH BOUNDARY

77

generalized bicharacteristics of  parameterized by the time t and passing through μ for t = 0. Thus, we have Ct (μ) = ∪γ(t; μ), where the union is taken over all bicharacteristics issued from μ. Set ν0 = (0, 0, x0 , τ0 , ξ10 , ξ0 ) = (0, y0 , τ0 , η0 ). Lemma 3.4.5: For each δ > 0, there exists (δ) > 0 such that if D(μ, l0 (σ)) < (δ) for some σ ∈ [0, σ1 ], then for each ν ∈ Ct0 −σ (μ) we have D(ν, Ct0 (ν0 )) =

inf

D(ν, ρ) < δ.

ρ∈Ct0 (ν0 )

Remark 3.4.6: This lemma says that if γ¯ (σ; ν) is a curve that coincides with a linear segment, passing through ν for some 0 ≤ σ ≤ σ ˆ and with the generalized bicharacteristic issued from μ = γ¯ (ˆ σ , ν) for σ ˆ ≤ σ ≤ t0 , then γ¯ (t0 ; ν) is close to Ct0 (ν0 ), provided ν is close to ν0 . We need this property because in general γ¯ (σ; ν), 0 ≤ σ ≤ t0 , is not a generalized bicharacteristic of . Proof of Lemma 3.4.5: Assume that there exist sequences {σk }, 0 ≤ σk ≤ σ1 , {μk } and νk = γ(t0 − σk ; μk ) so that D(μk , l0 (σk ))
0 are propagating along the outgoing and incoming bicharacteristics entering the exterior or interior of Ω, respectively. For 0 < 2δ < introduce a function κδ (t) ∈ C ∞ (R) such that  1, t ≤ δ, κδ (t) = 0, t ≥ 2δ.

POISSON RELATION FOR MANIFOLDS WITH BOUNDARY

79

For sufficiently small δ consider the distributions e2 (t − t , x, y)(1 − A)e0 (t − t , x − y), e˜1 (t − t , x, y) = e1 (t − t , x, y)κδ (t − t )1Ω (x), 1Ω (x) being the characteristic function of Ω. For (t, x) ∈ R × Ω◦ we have e1 (t − t , x, y) = fδ (t − t , x, y), (∂t2 − Δx )˜ and for sufficiently small δ > 0, we deduce sing supp fδ (t − t , x, y) ⊂ {(t, x, t , y) : δ ≤ t − t ≤ 2δ, x ∈ Oδ },

(3.26)

¯ ⊂ Ω◦ . Hence the singularities of f for t small enough are bounded away where O δ δ from the boundary ∂Ω. We extend E(t, x, y) as 0 for t < 0, and for t ≥ 0 we write e1 + e2 )(t − t , x, y) Ξ(t − t , x, y) = (˜  ∞ − K(s, x, z)(˜ e1 + e2 )(t − t − s, z, y)dsdz ∞ ∞

 −

∞

∂Ω



E(t − t − s, x, z)fδ (s, z, y)dsdz.

∂Ω

The integrals are interpreted in the sense of distributions. The smoothness of e0 (t, x − y) with respect to t ∈ R+ and (3.26) make this possible. The analysis of the term involving e˜1 is easy since the singularities of e˜1 are concentrated around the bicharacteristics entering Ω◦ . For the term involving fδ we apply a similar argument. To do this, we prove an assertion similar to Proposition 3.4.4 with ∂Ω replaced by an open domain O, O ⊂ Ω◦ . After this, we establish an analogue of (3.19) for  W F  (E(t, x1 , x , y)) ∩ T ∗ (R+ t × U × O)

uniformly with respect to x1 ∈ [0, ]. This can be done applying the arguments from the proofs of Propositions 3.4.2 and 3.4.4 with slight modifications. We leave the details to the reader. Combining these results and taking the trace t = 0, we obtain (3.25). In the above argument the choice of has been related to that of the conic neighbourhood Γ1 . To obtain uniformity with respect to U  , consider a covering T × U  × (Rn \ {0}) ⊂ ∪M j=1 Γj with conic neighbourhoods Γj for which our argument works with = j > 0. Taking = min j , we obtain > 0 that depends on U  and T0 , only. This completes the proof of Proposition 3.4.3, since the relation (3.25) holds for > 0 chosen  above.

80

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Now it is easy to establish the inclusion (3.18). Let Ω ⊂ ∪m k=1 Uk

(3.27)

be a covering with open sets Uk ⊂ Rn . Assume that for k = 1, . . . , m0 , we have Uk ⊂ Ω◦ , while Uk ∩ ∂Ω = ∅ for k = m0 + 1, . . . , m. Let t0 = LΩ , t0 > 0. Fix a neighbourhood Uk ⊂ Ω◦ . Applying (3.19), as at the end of Section 3.3, we deduce that   E(t, x, x)dx|t>0 WF Uk

is contained in the set of all (t, τ ) ∈ T ∗ (R) \ {0} such that there exists (x, ξ) ∈ T ∗ (Uk ) with (t, x, x, τ, ξ, ξ) ∈ C. We pass to the case Uk ∩ ∂Ω = ∅. Introduce in Uk local coordinates (x1 , x ) so that Uk ∩ ∂Ω has the form x1 = 0, while Uk ∩ Ω is given by x ∈ U  , 0 ≤ x1 ≤ . For sufficiently small δ0 > 0 we have t0 − δ0 > 0 and (t0 − δ0 , t0 + δ0 ) ∩ LΩ = ∅. Set Δ0 = (t0 − δ0 , t0 + δ0 ). Taking δ0 and small enough, Proposition 3.4.3 implies (t, x , x , τ, ξ  , ξ  ) ∈ / W F (H(x1 , y1 ))

(3.28)

for t ∈ Δ0 , (x , ξ  ) ∈ U  × Rn−1 , (x1 , y1 ) ∈ [0, ] × [0, ]. We consider t, x1 , y1 as parameters and apply the argument at the end of Section 3.3 for the trace x = y  and the integral over U  . Therefore, the relation (3.28) yields  E(t, x1 , x , x1 , x )dx |t∈Δ0 ,x1 ∈[0,] ∈ C ∞ , U

and integrating with respect to x1 , we get   E(t, x1 , x , x1 , x )dx1 dx ∈ C ∞ (Δ0 ). 0

U

Consequently, we can find a sufficiently fine covering (3.27) such that / sing supp t0 ∈

m   k=1

E(t, x, x)dx

Uk ∩Ω

and we obtain the inclusion (3.18) for an arbitrary domain Ω with smooth boundary ∂Ω. The above argument, with some trivial modifications, can be applied to the analysis of the distribution σN (t) =

∞  j=1

cos λj t ∈ S  (R),

POISSON RELATION FOR MANIFOLDS WITH BOUNDARY

81

where 0 ≤ λ21 ≤ λ22 ≤ · · · ≤ λ2m ≤ · · · are the eigenvalues of the self-adjoint operator AN in L2 (Ω) related to the Laplacian in Ω with Neumann or Robin boundary conditions on ∂Ω. The corresponding eigenfunctions {ϕj (x)}∞ j=1 satisfy  −Δϕj (x) = λ2j ϕ(x) in Ω, (∂ν + α(x))ϕj (x) = 0 on ∂Ω. Here ∂ν denotes the derivative with respect to a continuous normal field ν(x) to ∂Ω and α(x) ∈ C ∞ (∂Ω). We define E(t, x, y) and E(t, x, y) in the same way as in Section 3.2. The proof of (3.18) for arbitrary domains goes without any change, by using the results for propagation of singularities in [MS2] concerning the Neumann and Robin boundary problems. For example, we define K(t, x, y) as the solution of the problem ⎧ 2 ◦ ⎪ ⎨(∂t − Δx )K(t, x, y) = 0 in R × Ω , (∂ν + α(x))K(t, x, y) = δ(t) ⊗ δ(x − y) for x ∈ ∂Ω, ⎪ ⎩ supp K(t, x, y) ⊂ {(t, x, y) ∈ Rt × Ω × ∂Ω : t ≥ 0}. The assertions of Propositions 3.4.1 and 3.4.4 are true for K. Summing up the above results, we get the following. Theorem 3.4.7: Let Ω be a compact domain in Rn , n ≥ 2, with C ∞ smooth boundary ∂Ω. Then (3.29) sing supp σ(t) ⊂ {0} ∪ { ± Tγ : γ ∈ LΩ }. The same is true for sing supp σN (t), where σN (t) is related to Neumann or Robin boundary problem for the Laplacian.

3.5 Notes The analysis of the fundamental solution in Section 3.1 is taken from [BLR]. The idea of the proof in Section 3.3 of the Poisson relation for convex domains was proposed in [BLR]. For strictly convex (concave) domains this relation was obtained previously by Anderson and Melrose [AM]. The argument of [AM] can be generalized for general domains by using the results of propagation of singularities established in [MS1] and [MS2]. We followed the approach in [AM] exploiting the continuity properties of the generalized bicharacteristics described by Lemmas 1.2.6 and 3.4.5. A detailed investigation of N (λ), e(x, y, λ) and the distribution E(t, x, y) is contained in [H3] and [H4]. In particular, in [H4], Proposition 29.3.2, it was proved that W F  (E(t, x, y)|Rt ×Ω◦ ×Ω◦ ) ⊂ C.

4

Poisson summation formula for manifolds with boundary  In this chapter the leading singularity of σ(t) = j cos λj t near the period Tγ of a periodic ordinary reflecting bicharacteristic γ of  is examined. We assume that if δ is another period bicharacteristic of  in Ω with the same period Tγ , then the projections of γ and δ on Ω coincide. Moreover, we suppose that the Poincaré map Pγ of γ has no eigenvalues equal to 1. In Section 4.1 a global parametrix for the mixed problem characterizing E(t, x, y) is constructed. For this purpose we apply global Fourier integral distributions to express the successive reflections. The principal symbol of the parametrix is investigated in Section 4.2. The singularity of σ(t) is studied in Section 4.3 and a Poisson summation formula for manifolds with boundary is obtained in Theorem 4.3.1.

4.1

Global parametrix for mixed problems

In this section we use the notation of Chapter 3. Our aim is to construct a global parametrix for the operator EB = cos(tA)B(y, Dy ), where B(y, Dy ) is a zero order pseudo-differential operator with W F (B) ⊂ Γ, Γ = U × V being a small conic neighbourhood of a fixed point (y0 , η0 ) ∈ T ∗ (Ω◦ ) \ {0}. First, we assume that ¯ ⊂ Ω◦ and that the kernel of B has compact support in U × U . U Let T0 > 0 be fixed. We consider the generalized bicharacteristics γ(t; μ± ) of  issued from μ± = (0, y, ∓|η|, η) with (y, η) ∈ Γ and parameterized by the time t. In this section we treat the case when γ(t; μ± ) is reflecting at ∂Ω and without tangent segments for |t| ≤ T0 . We will treat γ+ (t; μ) = γ(t; μ+ ) and for simplicity of the notation we will omit the sign + in μ+ . Geometry of the Generalized Geodesic Flow and Inverse Spectral Problems, Second Edition. Vesselin M. Petkov and Luchezar N. Stoyanov. © 2017 John Wiley & Sons, Ltd. Published 2017 by John Wiley & Sons, Ltd.

POISSON SUMMATION FORMULA FOR MANIFOLDS WITH BOUNDARY

83

Let FB (t, x, y) be the kernel of EB . Our purpose is to construct a global Fourier integral distribution FˆB (t, x, y) so that (FB (t, x, y) − FˆB (t, x, y))|[0,T0 ]×Ω×U ∈ C ∞ . The distribution FˆB will be obtained as a sum of global Fourier integral distributions related to the reflections of γ+ (t; μ). For reader’s convenience, we recall some basic facts concerning global Fourier integral distributions and we refer to Sections 21 and 25 in [H4] for more details. Let W = X × (RN \ {0}), dim X = n, be an open conic set and let ϕ(x, θ) ∈ C ∞ (W ) be a real-valued phase function homogeneous of order 1 in θ. The phase ϕ is ∂ϕ ), j = 1, . . . , N , are linearly independent on non-degenerate if dϕ = 0 and d( ∂θ j Cϕ = {(x, θ) ∈ W : dθ ϕ = 0} which is a smooth manifold of dimension N . Consider the immersion iϕ : Cϕ (x, θ) → {(x, dx ϕ) ∈ T ∗ (X) \ {0}} = Λϕ . Then Λϕ is Lagrangian manifold in T ∗ (X) \ {0}. Consider the Fourier integral distribution  eiϕ(x,θ) a(x, θ)dθ ∈ I m (X, Λϕ ), I(x) = RN

n−2N

where a ∈ S m+ 4 (X × RN ) is a classical symbol. Following Hörmander 1/2 (Section 25, [H4]), it is convenient to consider I(x) as a distribution in D (X, ΩX ) 1/2 with values the half-density bundle ΩX of X. The corresponding class of Fourier 1/2 distributions is denoted by I m (X, Λϕ ; ΩX ). A Fourier integral operator V : C0∞ (Y, ΩY ) → D (X, ΩX ) 1/2

1/2

has a kernel given by a Fourier integral distribution  1/2 eiϕ(x,y,θ) b(x, y, θ)dθ ∈ I m (X × Y, C ; ΩX×Y ). I(x, y) = RN

Here C = {(x, dx ϕ, y, dy ϕ) ∈ T ∗ (X) \ {0} × T ∗ (Y ) \ {0} : ϕθ (x, y, θ) = 0} is called a canonical relation of V or I(x, y) and Λ = C = {(x, dx ϕ, y, −dy ϕ) ∈ T ∗ (X) \ {0} × T ∗ (Y ) \ {0} : ϕθ (x, y, θ) = 0} is a Lagrangian manifold with respect to the form σX + σY , where σX and σY are the symplectic forms on T ∗ (X) and T ∗ (Y ) trivially lifted to T ∗ (X) × T ∗ (Y ).

84

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

The principal symbol v0 of V belongs to the class   1/2 S m+(dim X+dim Y −2N )/4 X × Y, MΛ ⊗ ΩΛ , 1/2

where MΛ is the Maslov bundle of Λ, while ΩΛ is the bundle of half-densities on 1/2 Λ. We refer to [H4], Section 25, for the precise definitions of MΛ , ΩΛ and S m (X × 1/2 1/2 Y, MΛ ⊗ ΩΛ ). The Maslov bundle MΛ is locally trivial and the bundle ΩΛ of half-densities can be trivialized by choosing a canonical half-density on Λ. The situation is more simple if X and Y are n-dimensional manifolds, and C = graph r is the graph of a homogeneous canonical transformation r : T ∗ (Y ) \ {0} → T ∗ (X) \ {0}. Then v0 is a product of a half-density on C and a Maslov factor. Since C is diffeomorphic to T ∗ (Y ) \ {0}, we may consider v0 as an object on T ∗ (Y ) \ {0} parameterized by the symplectic coordinates (y, η) ∈ T ∗ (Y ). Next on T ∗ (Y ) \ {0} there exists a canonical half-density dcan = |dy ∧ dη|1/2 related to the coordinates (y, η), and the symbol v0 , modulo Maslov factors, becomes f0 dcan with a classical symbol f0 . Moreover, in this case the operator V ∈ I m (X × Y, C ) defines a continuous map from the 1/2 1/2 comp loc Sobolev spaces H(s) (Y, ΩY ) to H(s−m) (X, ΩX ) for every s ∈ R (see Section 25.3 in [H4]). In particular, we may define V G for distribution with compact support G ∈ E (Y ). We start by the fundamental solution    (2π)−n eit|η|+ix−y,η dη + e−it|η|+ix−y,η dη R0 (t, x − y) = 2 Rn Rn = satisfying

1 − (R + R+ 0 ), 2 0

 2 (∂t − Δx )R0 = 0, R0 (0, x − y) = δ(x − y), ∂t R0 (0, x − y) = 0.

We consider R± 0 as a Fourier integral distribution −1/4 R± (Rt × Rnx × Rny , (C0± ) ), 0 ∈I

where

 C0±

=

(t, x, y, τ, ξ, η) ∈ T ∗ (R × Rn × Rn ) \ {0}; x = y ± t ξ = η, τ = ∓|η|

η , |η|

POISSON SUMMATION FORMULA FOR MANIFOLDS WITH BOUNDARY

85

is parameterized by (y, η) ∈ Γ ⊂ T ∗ (Rn ) \ {0} and C0 = {(t, x, y, τ, ξ, −η) : (t, x, y, τ, ξ, η) ∈ C0 }. + In the following we deal with R+ 0 related to C0 and given by the Fourier integral with phase function ϕ− = −t|η| + x − y, η. Denote by t1 (y, η) the time of the first reflection of γ+ (t; μ) and set

t1 := inf t1 (y, η), T1 := sup t1 (y, η), I1 = [t1 , T1 ]. (y,η)∈Γ

(y,η)∈Γ

We need to examine the trace on ∂Ω of the distribution  −n eiϕ− β(y, η)dη (t, x, y) = (2π) R+ B for t ∈ I1 , provided Γ is sufficiently small. Here β(y, η) ∈ S 0 (Rn × Rn ) is the symbol of a zero order pseudo-differential operator B(y, Dy ) with W F (B) ⊂ Γ. Consider the inclusion map i : R × ∂Ω (t, x) −→ (t, x) ∈ Ω and denote by i∗ the operator of the trace on R × ∂Ω. The kernel K of i∗ is a Fourier integral distribution in I −1/4 (R × ∂Ω × R × Ω, N ), where the canonical relation N has the form 

˜ = ξ| ˜ t, x, τ, ξ) ∈ T ∗ (R × ∂Ω) × T ∗ | (R × Ω); x ∈ ∂Ω, ξ N = (t, x, τ, ξ, R×∂Ω Tx (∂Ω) . If ∂Ω is given by x1 = 0 in normal geodesic coordinates (x1 , x ), then  K(t, x , t, y) = (2π)−n eix −y ,ξ −iy1 ξ1 dξ and this explains the order −1/4 of K. We wish to prove that for any fixed t ∈ I1 the trace i∗ R+ B is a Fourier integral distribution, and for this purpose it is necessary to compose the canonical relation N and CΓ = {(t, x, y, τ, ξ, η) ∈ C0+ , (y, η) ∈ Γ}. Introduce Z = T ∗ (R × ∂Ω) × Δ(T ∗ (R × Ω) \ {0}) × T ∗ (U ) \ {0}, where Δ(U) = {(m, m) : m ∈ U} denotes the diagonal of U. Let γ+ (t; μ) hit (transversally) ∂Ω at x1 (y, η) for t = t1 (y, η). Assume that 1 x (y, η) ∈ ω1 ⊂ ∂Ω for (y, η) ∈ Γ, and let ω1 have in local normal coordinates (x1 , x ) the form x1 = 0. Let the principal symbol q of  have the form q(x, τ, ξ) = ξ12 + m(x, ξ ) − τ 2

86

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

in these local coordinates. A simple calculus shows that codim (N ×CΓ ) = 4n + 2, codim Z = 2n + 2. On the other hand, the transversality of γ+ (t; μ) to ∂Ω yields ∂q = 2ξ1 = τ 2 − m(0, x , ξ ) = 0 ∂ξ1

(4.1)

for x = x (y, η), ξ = ξ (y, η). This implies easily that codim ((N × CΓ ) ∩ Z) = 6n + 3. The manifold N × CΓ intersects Z cleanly, which means that for each u ∈ (N × CΓ ) ∩ Z we have Tu ((N × CΓ ) ∩ Z) = Tu (N × CΓ ) ∩ Tu (Z). The excess e of this clean intersection is 1 and the composition of the canonical relations ρ0 = N ◦ CΓ will be a canonical relation (see [H3], Section 21). Since the projection ˜ η) ∈ ρ −→ (y, η) ∈ Γ π0 : (t1 (y, η), x1 (y, η), y, −|η|, ξ, is a diffeomorphism, the relation ρ0 is locally the graph of a homogeneous canonical transformation r0 : Γ → T ∗ (R × ∂Ω). By the calculus for Fourier integral operators with clean intersection (see [H4], Section 25), we deduce that i∗ R+ B is a Fourier integral distribution 0 i∗ R+ B (t, x, y) ∈ I (R × ∂Ω × U, ρ0 ).

(4.2)

Notice that we obtain a Fourier integral distribution of order −1/4 − 1/4 + e/2 = 0. Next, we modify R+ B (t, x, y) for t > T1 + 1 and small 1 > 0, so that ∞ R+ B ∈ C , for t > T1 + 1

and for t ≥ 0 the trace i∗ R+ B , modulo smooth functions, coincides with (4.2). This continuation is possible since the singularities of R+ B (t, x, y) are propagating along the bicharacteristics of  lying in the exterior of Ω for T1 +  ≤ t ≤ T1 + 2, provided  small enough. To construct the parametrix for t > t1 , we need to satisfy the boundary condition on ∂Ω. Let ν(x) be the exterior unit normal at x ∈ ∂Ω. Consider the set

˜ ∈ T ∗ (R × ∂Ω) \ {0} : z = y + t (t, η) η , Σ1 = (t1 (y, η), z, τ, ξ) 1 |η|  ˜ ν(z), η > 0, (y, η) ∈ Γ . τ = −|η|, η|Tz (∂Ω) = ξ, which depends on Γ but for the brevity of the notation we will omit this later. Introduce the map ˜ → (exp tH )(s, z, τ, ξ), Φ(t, ρ) : Σ1 ρ = (s, z, τ, ξ) q

POISSON SUMMATION FORMULA FOR MANIFOLDS WITH BOUNDARY

87

where ξ = η − 2ν(z), ην(z) is the reflected direction determined from the incoming direction η. Notice also that |ξ| = |η| and the principal symbol q of  vanishes on Φ(t, ρ). Moreover, τ remains constant along the action of Φ, and Φ(t, ·) is one-parameter group along the trajectories of the Hamiltonian field Hq of the principal symbol q. Consider the map ˜ → (s, z, τ, ξ) ∈ l (Σ ) = U ⊂ H, l1 : Σ1 (s, z, τ, ξ) 1 1 1 where ξ is the reflected direction introduced above and H is the set of hyperbolic points introduced in Section 1.2. Thus, exp(tHq )

Σ1 −−−1−→ U1 −−−−→ T ∗ (R × Ω). l

It is easy to see that Φ(t, ρ) is an immersion. Indeed, for t = 0, Φ(0, ρ) maps Σ1 diffeomorphically into T ∗ (R × Ω) since ξ is uniquely determined from η|Tz (∂Ω) . ∂ Moreover, dΦ|t=0 maps ∂t into the Hamiltonian field Hq . The condition (4.1) implies that Hq is transversal to ∂Ω, and this shows that dΦ|t=0 is injective. Finally, the group property Φ(t1 + t2 , ·) = Φ(t1 , Φ(t2 , .)) implies that dΦ(t, ρ) is injective for all t and Φ(t, ρ) is an immersion. Introduce C1 = {(Φ(t, ρ), ρ) ∈ T ∗ (R × Ω) \ {0} × T ∗ (R × ∂Ω) \ {0} : t ≥ 0, ρ ∈ Σ1 }. To prove that C1 is a canonical relation on T ∗ (R × Ω) \ {0} × T ∗ (R × ∂Ω) \ {0}, consider the symplectic forms dα and dαb on T ∗ (R × Ω) and T ∗ (R × ∂Ω), respectively, where α and αb are canonical one-forms. For example, if w ∈ T(s,σ) (T ∗ (R × ∂Ω)) with s ∈ R × ∂Ω, σ ∈ Ts∗ (R × ∂Ω), we have αb , w = σ, w , w ∈ Ts (R × ∂Ω) being the projection of w on Ts (R × ∂Ω). By using a trivial lifting, consider α and αb as one-forms on W = T ∗ (R × Ω) × ∗ T (R × ∂Ω) and denote them by the same notation. Our purpose is to show that α − αb vanishes on the image of dΦ. This will imply that C1 is a Lagrangian submanifold with respect to the symplectic structure defined by α − αb . First consider ∂ ) = (Hq , 0), the case t = 0 and set Λ0 = {(Φ(0, ρ), ρ) : ρ ∈ Σ1 }. Since (dΦ|t=0 )( ∂t we get    ∂ = α, Hq (Φ(0, ρ)) = q(Φ(0, ρ)) = 0, α − αb , (dΦ|t=0 ) ∂t because q vanishes on Φ(0, ρ). Now consider w ∈ T(s,σ) (T ∗ (R × ∂Ω)) and let dΦ(0, ρ)w = v ∈ T(s,ζ) (T ∗ (R × Ω)). Then ζ|Ts (R×Ω) = σ and the projection v of v on Ts (R × ∂Ω) coincides with w defined above. Consequently, α, dΦ(0, ρ)w = ζ, w  = σ, w 

88

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

and α − αb , (dΦ(0, ρ)w, w) = 0. Thus, Λ0 is the embedded Lagrangian submanifold. Next Λt = {(Φ(t, ρ), ρ) : t > 0, ρ ∈ Σ1 } is also an embedded Lagrangian submanifold with respect to dα − dαb because Φ(t, .) is the Hamiltonian flow of q. Thus, we have proved the following. Lemma 4.1.1: C1 is a canonical relation. For every fixed t > 0 the map U1 u → Φ(t, u) ∈ H preserves the 2-form on H induced by the canonical symplectic form in T ∗ (R × Ω). The second statement follows from the fact that for fixed t, the set {Φ(t, l1 (ρ)) : ρ ∈ Σ1 } is diffeomorphic to U1 . The fact that for fixed time the shift along the billiard trajectories issued from Γ and reflecting on R × ∂Ω preserves the fundamental one-form. η, dy has been proved by a direct calculus in Proposition 2.3.5 in [SaV]. Clearly, a transformation preserving η, dy is a canonical one. ˜ → x is the projection on ∂Ω. To Now, let ω1 = π(Σ1 ), where π : (t, x, τ, ξ) arrange the Dirichlet boundary condition on ω1 , we introduce a Fourier integral operator R1+ ∈ I −1/4 (R × ∂Ω × R × ∂Ω, C1 ) satisfying the conditions

 2 (∂t − Δx )R1+ ≡ 0, i∗ω1 R1+ f ≡ f,

(4.3)

for every distribution f ∈ E (R × ∂Ω) with W F (f ) ⊂ Σ1 . Here ≡ means an equality modulo operators with smooth kernels or equality modulo C ∞ terms, while i∗ω1 is the trace on ω1 . Notice that i∗ω1 does not coincide with the identity on ∂Ω and i∗ω1 is a Fourier integral operator. Moreover, for sufficiently small ω1 , hence for small enough Γ, we have Φ(t, ρ)|∂Ω ∈ U1 only for t = 0. For the composition i∗ω1 R1+ , we apply the above argument based on a clean intersection and i∗ω1 R1+ is related to the canonical relation {(ρ, ρ) : ρ ∈ Σ1 }, hence it is a zero order pseudo-differential operator. To construct R1+ , we must solve the transport equations for the principal and lower-order symbols of R1+ . To do this, recall the notion of the principal symbol discussed in the beginning of this section. For brevity of the notations, put Y = R × Ω × ∂Ω, Λ = C1 . Then the kernel of R1+ is given by a distribution   1/2 −1/4 R+ Y, Λ; ΩY 1 ∈I

POISSON SUMMATION FORMULA FOR MANIFOLDS WITH BOUNDARY

89

 1/2  and its principal symbol a0 belongs to the class S 0 Y, MΛ ⊗ ΩΛ . Denote by q the principal symbol of  and consider Hq as a vector field on Y by using a trivial lifting. Since the subprincipal symbol of  vanishes, we obtain the transport equation LHq a0 = 0,

(4.4)

where LHq denotes the Lie derivative along the Hamiltonian field Hq (see Theorem 25.2.4 in [H4]). Since Maslov factors are locally constant, it is sufficient to solve equation (4.4) for the half-density f0 dΛ part of a0 , where dΛ is a half-density of Λ. In the next section we prove that we can choose dΛ invariant with the action of Hq and the problem is reduced to the equation Hq f0 = 0. Since Hq is transversal to ∂Ω, we solve the last equation with initial condition f0 |ω1 = 1, so that the principal symbol of i∗ω1 R1∗ , modulo Maslov factors, is dΛ . Then we get  1/2  (∂t2 − Δx )R1+ ∈ I −1/4+1 Y, Λ; ΩY . Next for the lower-order symbols aj = fj dΛ of R1+ , we obtain the equations Hq fj = gj (f0 , . . . , fj−1 ), j ≥ 1 which we solve with zero initial conditions on ω1 . Thus, the full symbol of (∂t2 − Δx )R1+ is 0, that of i∗ω1 R1+ is equal to 1 and we arrange (4.3). Since Φ is an immersion, it follows easily that the set C1 × (N ◦ CΓ ) intersects T ∗ (R × Ω) \ {0} × Δ(T ∗ (R × ∂Ω) \ {0}) × T ∗ (U ) \ {0} transversally. Then the composition Γ1+ = C1 ◦ (N ◦ CΓ ) is a canonical relation, and by the calculus of Fourier integral operators we deduce   −1/4 R × Ω × U, (Γ1+ ) . R1+ i∗ R+ B ∈I Notice that Γ1+ ⊂ C+ , where C+ is the relation introduced in Section 1.2 and Γ1+ = {(t, −|η|, Ft−t1 (y,η) (x1 (y, η), ξ 1 (y, η)), y, η) ∈ T ∗ (R × Ω × ∂Ω) \ {0}} is parameterized by (y, η). Here, the reflected direction ξ1 (y, η) is determined as in the definition of Σ1 , and Ft denotes the generalized Hamiltonian flow. Let tk (y, η) be the time of the kth reflection of γ+ (t; μ). Let γ+ (t, μ) hit T ∗ (∂Ω) at points λ1 (y, η), . . . , λk (y, η), . . . with λk (y, η) = (xk (y, η), η k (y, η)) ∈ T ∗ (∂Ω) ∗ and let λ k (y, η) ∈ T (∂Ω) be obtained from λk (y, η) by changing the incoming direction η k (y, η) with the reflecting one ξ k (y, η) so that η k , ν(xk )(y, η) = −ξ k , ν(xk )(y, η). Set tk = inf tk (y, η), Tk = sup tk (y, η). (y,η)∈Γ

(y,η)∈Γ

90

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

+ + Our analysis in Section 3.1 shows that W F (i∗ R+ B ) ⊂ Σ1 . Setting V0 = RB , we get

i∗ω1 R1+ i∗ V0+ ≡ i∗ V0+ and we define

V1+ = R1+ i∗ V0+ .

(4.5)

Then for 0 < t < t2 the trace of W1+ = −V1+ + V0+ on ∂Ω vanishes modulo smooth functions. This completes the construction related to the first reflection. Now we repeat the above procedure for other reflections. For t ∈ [t2 , T2 ], the generalized bicharacteristics γ+ (t; μ) hit transversally ∂Ω. To prove that ρ1 = N ◦ C1 ⊂ T ∗ (R × ∂Ω) \ {0} × T ∗ (R × ∂Ω) \ {0} is a canonical relation, we apply the above argument showing that N × C1 intersects cleanly T ∗ (R × ∂Ω) × Δ(T ∗ (R × Ω)) \ {0}) × T ∗ (R × ∂Ω). Moreover, the map U1 u → Φ(t2 (y, η), u) ∈ U2 ⊂ H is a diffeomorphism. Taking the projection on Σ1 , we conclude that ρ1 is locally the graph of a homogeneous canonical transformation r1 : Σ1 → Σ2 = r1 (Σ1 ) ⊂ T ∗ (R × ∂Ω). Next, we modify R1+ for t ∈ / [t1 − 2 , T2 + 2 ] and small 2 > 0, so that R1+ Qf ∈ / [t1 − 2 , T2 + 2 ] and Q is a pseudo-differential operators with W F (Q) ⊂ C if t ∈ + Σ1 . The singularities of the kernel R+ 1 of R1 and the form of C1 make this possible. Set Σ2 = r1 (Σ1 ), ω2 = π(Σ2 ) ⊂ ∂Ω (see the diagram), and consider the Fourier integral operator i∗ω2 R1+ ∈ I 0 (R × ∂Ω × R × ∂Ω, ρ 1 ). ∞

Φ(t2 (y,η),.)

U1 −−−−−−−→ U2   ⏐ ⏐ l1 ⏐ l2 ⏐ Σ1 ⏐ ⏐ π ω1

r

−−−1−→

Σ2 ⏐ ⏐ π ω2

Following this procedure, define Σk = rk−1 (Σk−1 ), π(Σk ) = ωk , lk (Σk ) = Uk ⊂ H,

POISSON SUMMATION FORMULA FOR MANIFOLDS WITH BOUNDARY

91

the canonical relation Ck = {(Φ(t, lk (ρ)) ∈ T ∗ (R × Ω × R × ∂Ω) \ {0} : t ≥ 0, ρ ∈ Σk } related to the kth reflection of γ+ (t; μ). For small Γ the sets Uk belong to H and the inverse map lk−1 is diffeomorphism. Applying Lemma 4.1.1, we conclude that rk : Σk → Σk+1 ⊂ T ∗ (R × ∂Ω), is a homogeneous canonical transformation . Denote by i∗ωk the trace on ωk . Repeating the construction of R1+ , we can find a Fourier integral operator Rk+ ∈ I −1/4 (R × Ω × R × ∂Ω, Ck ) satisfying the conditions

 2 (∂t − Δx )Rk+ ≡ 0, i∗ωk Rk+ f ≡ f,

for each f ∈ E (R × ∂Ω) with W F (f ) ⊂ Σk . As above, we modify the kernel R+ k of Rk+ for t ∈ / [tk − k , Tk+1 + k ], k > 0 being sufficiently small. Our construction shows that the trace i∗ Rk+ , modulo smoothing operators, coincides with the sum of the traces i∗ωk Rk+ and i∗ωk+1 Rk+ . To satisfy the boundary conditions, introduce a zero order pseudo-differential operator Mk ∈ L0 (R × Ω) such that W F (Mk ) ∩ Uk+1 = ∅, W F (Id − Mk ) ∩ Uk = ∅, k ≥ 1. This is possible since Uk ∩ Uk+1 = ∅, provided Γ small enough. Consequently, + + + + = i∗ωk−1 (Id − Mk−1 )Rk−1 + i∗ωk (I − Mk−1 )Rk−1 ≡ i∗ωk Rk−1 i∗ (Id − Mk−1 )Rk−1 + + and i∗ωk Rk+ i∗ (Id − Mk−1 )Rk−1 ≡ i∗ (Id − Mk−1 )Rk−1 . After this preparation define + ,k ≥ 2 Vk+ = Rk+ i∗ (Id − Mk−1 )Vk−1

and set Wp+ =

p 

(−1)k Vk+ .

k=0

Here the coefficients (−1)k are added to arrange the boundary conditions on ∂Ω for time 0 ≤ t < tk+1 . Notice that Vk+ ∈ I −1/4 (R × Ω × U, (Γk+ ) ), where the canonical relation Γk+ has the form Γk+ =



  ∗ t, −|η|, Ft−tk (y,η) λ k (y, η), y, η ∈ T (R × Ω × U ) ⊂ C+ .

(4.6)

92

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Therefore, for 0 ≤ t < tp+1 we have  2 (∂t − Δx )Wp+ ∈ C ∞ , i∗ Wp+ ∈ C ∞ . Notice that the construction of Wp+ works if γ+ (t; μ) have at least p reflections for 0 ≤ t ≤ T0 . In the same way we treat the relation C0− and the generalized bicharacteristics γ− (t; μ) of  issued from ν(0, y, |η|, η), (y, η) ∈ Γ. Let Wp− =

p 

−n (−1)k Vk− , V0 = R− B = (2π)

 eiϕ+ β(y, η)dη

k=0

be the corresponding distribution. Here Vk− ∈ I −1/4 (R × Ω × U, (Γk− ) ), where

    Γk− = t, |η|, F−t−tk (y,η) λk,− (y, η), y, η ∈ T ∗ (R × Ω × U ) \ {0} ⊂ C−  with λk,− (y, η) related to γ− (y, μ) and λk,− (y, η) obtained from λk,− (y, η) changing the incoming direction by the reflecting one. Therefore, 1 (−1)k (Vk+ + Vk− ) Wp = 2 p

k=0

will be the solution of the problem ⎧ 2 (∂ − Δx )Wp ∈ C ∞ , ⎪ ⎪ ⎨ t i∗ Wp ∈ C ∞ , ⎪ ⎪ ⎩ Wp |t=0 = B ∗ (y, Dy )δ(x − y), ∂t Wp |t=0 = 0 for 0 ≤ t ≤ tˆp+1 , where tˆp+1 depends on the time of the (p + 1)th reflection of γ± (y; μ) and B ∗ (y, Dy ) is the operator adjoint to B(y, Dy ). For large p we obtain the distribution FˆB (t, x, y) for 0 ≤ t ≤ T0 . Now let γ be a periodic ordinary reflecting bicharacteristic of  with period T > 0 passing through (y0 , η0 ). Let mγ be the number of reflections of γ. The operator Vk+ is related to the relation Γ+ k . Let U and  > 0 be sufficiently small and let I = (T − , T + ). Then for t ∈ I modulo smooth terms, we obtain + ≡ (−1)mγ Vm+γ ∈ I −1/4 (I × U × U, (Γ+ γ ) ). Wm γ m

Finally, we take 1 1 FˆB (t, x, y)|I×U ×U = (−1)mγ (Vm+γ + Vm−γ ) = (FˆB+ + FˆB− ). 2 2 This proves the following.

(4.7)

POISSON SUMMATION FORMULA FOR MANIFOLDS WITH BOUNDARY

93

Proposition 4.1.2: For 0 ≤ t ≤ T0 we have (FB (t, x, y) − FˆB (t, x, y))|Ω×U ∈ C ∞ . Now we will discuss briefly the case when U is a small open neighbourhood of a point y0 ∈ ∂Ω. Let (y1 , y ), y1 = dist(x, ∂Ω) be local normal coordinates. Then ∂Ω ∩ U has the form y1 = 0 and let Ω ∩ U = {(y1 , y ) : y ∈ U , 0 ≤ y1 ≤ α}. Let

q(y, τ, η) = η12 − τ 2 + m(y, η ), m(y, η ) ≥ c0 |η |2 , c0 > 0,

be the principal symbol of  in the local coordinates and let μ0 = (0, y0 , τ0 , η0 ) ∈ T ∗ (Rn ) \ {0} be a hyperbolic point of . This means that the equation q(0, y0 , τ0 , η1 , η0 ) = 0 with respect to η1 has two distinct real roots. Let O = J × U , J being a small neighbourhood of 0, and let Γ = O × V ⊂ T ∗ (Rt × Rn−1 y ) \ {0} be an open conic neighbourhood of μ0 such that {(t , y1 , y , τ , η ) : (t , y , τ , η ) ∈ Γ, 0 ≤ y1 ≤ α} are hyperbolic points for . The latter means that τ 2 − m(y, η ) > 0. Consider a zero order pseudo-differential operator B(t , y, Dt , Dy ), depending smoothly on y1 , so that W F (B(·, y1 , ·)) ⊂ Γ and the kernel of B(·, y1 , ·) has a compact support in O × O. As in Section 3.4, it is more convenient to study EB = cos((t − t )A)Y (t − t )B(t , y, Dt , Dy ), where 0 ≤ y1 ≤ α is considered as a parameter and α > 0 is sufficiently small. Let η1± (y1 , y , τ, η ) be the real roots of q(y1 , y , τ , η1 , η ) = 0 with respect to η1 . Assume that the bicharacteristics γ± (t; μ) issued from μ = (t , y1 , y , τ , η1± , η ) are reflecting for |t| ≤ T0 . We treat the case with η1 = η1+ . Similarly to the situation examined above, introduce ˜ ∈ T ∗ (R × ∂Ω) \ {0} : t = t (γ (t; μ)), z = y + t (γ (t; μ)) Σ 1 = {(t, z, τ, ξ) 1 + 1 +

η , |η|

˜ ν(z), η > 0}. τ = −τ , η|Tz (∂Ω) = ξ, Here t1 (γ+ (t; μ)) is the time of the first reflection of γ+ (t; μ). Clearly, for μ with y1 = 0 we have t1 (γ(t; μ)) = 0 and z = y ∈ ∂Ω. On the other hand, we have (∂t + ∂t )EB = 0 and on the wave front set of EB one has τ + τ = 0. We consider y1 as a parameter, and for fixed y1 , the set Σ 1 is parameterized by the symplectic

94

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

coordinates (t , y , τ , η ). Next, we define the flow Φ(t, l1 (ρ)), ρ ∈ Σ 1 and repeat the construction of Vk+ . As it was mentioned in Section 3.4, the distribution E(t − t , x1 , x , y1 , y ) for 0 ≤ x1 ≤ α, 0 ≤ y1 ≤ α, depends smoothly on x1 , y1 . For small α > 0 a periodic bicharacteristic issued from y1 = yˆ1 and transversal to this hyperplane after reflections will return to this hyperplane. Thus, we construct a Fourier integral distribution Wmγ (t − t , x, y) =

 1 + − FB (t − t , x, y) + FB (t − t , x, y) 2

such that (EB (t, t , x1 , x , y1 , y) − Wmγ (t − t , x1 , x , y1 , y ))|I×J ×U ×U ∈ C ∞ .

(4.8)

Here J ⊂ R is a small neighbourhood of 0, I = (T − , T + ),  > 0, EB (t, t , x, y) is the kernel of EB and 0 ≤ x1 , y1 ≤ α . The Fourier integral distributions ± (t − t , x, y) are related to the canonical relations FB MΓ,± = {(t, x, τ, ξ, t , y, τ , η) ∈ T ∗ (I × U × J × U ) \ {0} : τ = τ , η1 = η1± (y, τ, η ), (t, x, τ, ξ) and (t , y, τ, η) lie on a generalized bicharacteristic of  and (t , y , τ , η ) ∈ Γ}.

4.2

Principal symbol of FˆB

In this section we shall study the principal symbol of the distribution FˆB (t, x, y) given by (4.7). To do this, we must examine the principal symbol of Vm±γ following the rules for composition of Fourier integral operators (see [H4]). + + Consider the kernel R+ 1 of the operator R1 . The principal symbol a0 of R1 satisfies equation (4.5). Setting Λ = C1 , the symbol a0 is a section of the bundle 1/2 MΛ ⊗ ΩΛ , where MΛ is the Maslov bundle over Λ. Ignoring the Maslov factors in MΛ , which are locally constant, consider the half-density part of a0 . It is convenient 1/2 to trivialize the bundle of half-densities ΩΛ by using a half-density dΛ on Λ , which is invariant with respect to the action of Hq . We choose a canonical half-density dcan = (|dy| ∧ |dη|)1/2 on T ∗ (U ) related to the symplectic coordinates (y, η). As it was proved in the previous section, Σ1 can be parameterized by the symplectic coordinates (y, η) ∈ Γ and we can define a half-density ((r0 )−1 )∗ (|dy| ∧ |dη|)1/2 = dcan on Σ1 by using the projection π0 and the canonical transformation r0 . In the same way we choose a half-density δcan = (|dt| ∧ |dy| ∧ |dη|)1/2 on R × l1 (Σ1 ). Therefore, since C1 is parameterized by (t, ρ) ∈ R × Σ1 ,  ∗ (Φ(t, .)−1 δcan = δcan

POISSON SUMMATION FORMULA FOR MANIFOLDS WITH BOUNDARY

95

is a half-density on Λ and (Γ1+ ) . Here we used the fact that for every fixed t0 , the map Φ(t0 , .) is a canonical transformation from l1 (Σ1 ) to {Φ(t0 , u) : u ∈ l1 (Σ1 )}, hence (Φ(t0 , .)−1 )∗ leaves dcan invariant. Now it is clear that δcan is invariant under the action of Φ(t, ·)∗ and this implies LHq (δcan ) =

∂ (Φ∗ (t, .)δcan )|t=0 = 0. ∂t

The half-density part of a0 has the form a0 = f0 δcan with a function f0 homogeneous of order 0. The transport equation LHq a0 = 0 yields Hq f0 = 0. Thus, f0 is constant along the orbits of Hq and the initial condition on ω1 for a0 implies f0 = 1. Next, consider the operator (Id − M1 )i∗ R1+ related to the canonical relation ρ1 = graph r1 . The condition W F (M1 ) ∩ Σ2 = ∅ shows that the symbol of M1 vanishes on Σ2 . As we mentioned in the previous section, we can parameterize Σ2 by (y, η) exploiting the homogeneous canonical transformation r1 . Therefore, on ρ1 , we have a canonical half-density ((r1 )−1 )∗ dcan = dcan . Repeating this procedure for the operator (Id − Mk )i∗ Rk+ , we are in a position to apply the rule for the computation of the principal symbol of a product of Fourier integral operators associated with homogeneous canonical transformations (see [H4], Section 25). Setting σk = rk ◦ · · · ◦ r1 : Σ1 → Σk+1 , k ≥ 2, introduce a half-density dcan on ρk = graph σk and a half-density δcan on (Γk+ ) parameterized by (t, y, η). Then the principal symbol of the operator + . . . (Id − M1 )i∗ R1+ , (−1)k Rk+ i∗ (Id − Mk−1 )i∗ Rk−1

modulo Maslov factor, is (−1)k δcan . To obtain a local representation of FˆB+ , m consider a small conic neighbourhood Z0 ⊂ (Γ+ γ ) of ν0 = (T, y0 , y0 , −|η0 |, η0 , ηo ) mγ ∈ (Γ+ ) related to a periodic reflecting ray γ = γ+ (t; μ0 ) with period T issued from μ0 = (y0 , η0 ). For fixed t ∈ I the projection Z0 (t, x, y, τ, ξ, η) → (y, η) ∈ U × V is a diffeomorphism. Let Ft (y, η) be the generalized Hamiltonian flow of  introduced in Section 1.2. Then for fixed t, the generalized bicharacteristics hit the surface {Ft (y, η); (y, η) ∈ Γ} transversally and the map (x, ξ) = Ft (y, η) is a homogeneous canonical transformation according to the argument in the previous section. Therefore, it is possible to find a phase function ϕ(t, x, η), determined in ˜ × V˜ of (T, y , η ) and homogeneous of a small conic neighbourhood Y = I˜ × U 0 0 order 1 in η, so that (see Proposition 25.3.3 in [H4]) (Ft (y, η), (y, η)) = (x, ϕη , ϕx , η), det ϕx,η (t, x, η) = 0, (t, x, η) ∈ Y. This implies Z0 = {(t, x, ϕη , −|η|, ϕx , η) : (t, x, η) ∈ Y }.

96

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

For I, U, V small enough we arrange I × U × V ⊂ Y and for t ∈ I, x, y ∈ U , modulo C ∞ terms, we have the representation  FˆB+ (t, x, y) = (2π)−n eiϕ(t,x,η)−iy,η b(t, x, y, η)dη. (4.9) |η|≥1

Here t is a parameter, b(t, x, y, η) ∼

∞ 

bj (t, x, y, η)

j=0

and bj (t, x, y, η) are homogeneous of order (−j) with respect to η. For fixed t ∈ I we may consider (x, η) as local coordinates on Z0 and (|dx| ∧ |dη|)1/2 becomes a half-density on Z0 . With respect to this localization, the principal symbol of (4.9), modulo Maslov factors, has the form b0 (t, x, ϕx , η)(|dx| ∧ |dη|)1/2 . On the other hand, for fixed t0 it is possible to express the principal symbol of FˆB+ (t0 , x, y) by using the symplectic coordinates (y, η) on T ∗ (U ) and the half-density dcan on Z0 |(Φt0 (y,η),(y,η)) related to (y, η). The principal symbol of (−1)mγ Vm+γ is (−1)mγ δcan , and we must take the restriction on t = t0 . Following the rules of composition of half-densities, it is easy to see that the half-density part of the restriction on t = t0 is (−1)mγ dcan . We will explain a similar argument below in a more difficult case when we take the trace of δcan on ∂Ω × ∂Ω. Since y = ϕx (x, η) on Z0 , we obtain |dy|1/2 ∧ |dη|1/2 = |det ϕxη (t, x, η)|1/2 |dx|1/2 ∧ |dη|1/2 . Comparing this with the form of this symbol in the coordinates (x, η), we deduce π

b0 (t, x, ϕx , η) = (−1)mγ ei 2 σ |det ϕxη (t, x, η)|1/2 .

(4.10)

π

Here σ ∈ N and ei 2 σ is a Maslov factor. The integer σ depends on γ, only. To see this, we will express σ by the signatures of the matrices d2ηη ϕj , where ϕj are phase functions parameterizing C+ in small neighbourhoods Zj along γ. Let γ = γ(t) be parameterized by the time, and let 0 = t0 ≤ t1 ≤ · · · ≤ tl = T be a sequence of times along γ with tj = tj+1 only if γ(tj ) is a reflection point of γ. Assume tj < tj+1 and let γ(t) have no reflections for tj < t < tj+1 . Suppose γ(tj ) ∈ Zj ∩ Zj+1 and let Zk for k = j, j + 1 be expressed by ϕk as earlier Z0 has been expressed by ϕ. Then passing from the representation of FˆB+ by ϕj to that related to ϕj+1 , we must add the Maslov factor i1/2(sign dηη ϕj −sign dηη ϕj+1 ) . 2

2

(4.11)

POISSON SUMMATION FORMULA FOR MANIFOLDS WITH BOUNDARY

97

Now suppose γ(tj ) = (tj , x0 , τ0 , ξo ) is a reflection point of γ. Let x0 ∈ ωk ⊂ ∂Ω and let ωk have locally the form x1 = 0, x0 = (0, x 0 ). Denote as above by q(x, τ, ξ) the principal symbol of  in these local coordinates. For (x , τ, ξ ) close to (x 0 , τ0 , ξ0 ), denote by ξ1± (x, τ, ξ ) the roots of the equation q(x, τ, ξ1 , ξ ) = 0 with respect to ξ1 . For t close to tj , the distribution FˆB+ has the form FˆB+ = (−1)k (Rk+ (Id − Mk−1 )i∗ Lk−1 − Lk−1 )i∗ R+ B. The operator i∗ωk Lk−1 has a canonical relation that is the graph of σk . Let χ(t, x , τ, ξ ) be the generating function of σk , that is     χτ χξ t x , . graph σk−1 = χt χx τ ξ By convention, assume that (ξ1± , ξ ) is close to the direction of γ(t) for ∓(t − tj ) > 0 and |t − tj | sufficiently small. In other words, ξ1− (resp. ξ1+ ) corresponds to incoming (resp. outgoing) segments reflecting on ωk . Introduce the phase functions ϕ∓ (t, x, τ, ξ ) as solutions of the Cauchy problems  ∂ϕ∓ ∓ ∓ ∓ ∂x1 = ξ1 (x, ϕt , ϕx ), ϕ∓ |x1 =0 = χ(t, x , τ, ξ ). Then the kernel of Lk−1 for t close to tj admits the representation  − eiϕ (t,x,τ,ξ )−it τ −ix ,ξ  b(k) (t, x, τ, ξ )dτ dξ ,

(4.12)

while the kernel of Rk+ (Id − Mk−1 )i∗ Lk−1 has a similar representation by ϕ+ (t, x, τ, ξ ). Hence putting ζ = (τ, ξ ), by the initial conditions for ϕ∓ , we deduce sign d2ζζ ϕ− = sign d2ζζ ϕ+ .

(4.13)

Thus, the reflection at γ(tj ) does not involve a Maslov factor. Denote by M the set of j ∈ R such that tj = tj+1 . Then, according to (4.11) and (4.13), the Maslov factor σ in (4.10) has the form σγ =

1 2

L−1 



 sign d2ηη ϕj − sign d2ηη ϕj+1 .

(4.14)

j=0,j ∈ /M

Clearly, σγ depends on γ, only, since the choice of γ(0) is not important for the sum in (4.14). For FˆB− we follow a completely similar argument. For t ∈ I, x, y ∈ U we get a representation  − −n ˆ eiψ(t,x,η)+iy,η c(t, x, η)dη (4.15) FB (t, x, y) = (2π) |η|≥1

98

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS m

with phase function ψ(t, s, η) representing locally (Γ− γ ) and  cj (t, x, η). c(t, x, η) ∼ j

The principal symbol has the form c0 (t, x, η) = (−1)mγ e−i 2 σγ |det ψxη (t, x, η)|1/2 . π

(4.16)

In fact, we repeat the construction of FˆB+ following a covering of γ(t), where we m change the orientation because τ > 0 on (Γ− γ ) and the time t decreases when we move along γ(t). Now we pass to the case when U is a neighbourhood of a point in ∂Ω, and we use the notation of the previous section. The purpose is to describe the principal symbol of EB |t =0 for t close to a period T of a periodic bicharacteristic. We introduce normal coordinates (y1 , y , η1 , η ) and (x1 , x , ξ1 , ξ ) so that the boundary ∂Ω is given locally, respectively, by y1 = 0 or x1 = 0. Let the principal symbol q of  have the forms η12 − τ 2 + m(y, η ) and ξ12 − τ 2 + m1 (x, ξ ), respectively, and assume that on Ω◦ locally we have y1 > 0, x1 > 0, respectively. Let i∂Ω be the restriction on ∂Ω and let i∂Ω × i∂Ω : R × ∂Ω × ∂Ω → R × Ω × Ω be the inclusion map. Here we use i∂Ω for the restriction on ∂Ω, while in Section 4.1, the operator i was the restriction on R × ∂Ω. Clearly, i∂Ω × i∂Ω ∈ I 1/4 (R × ∂Ω × ∂Ω × R × Ω × Ω, N ∂Ω×∂Ω ) is a Fourier integral operator with canonical relation ˜ η˜, t, x, y, τ, ξ, η) ∈ T ∗ (R × ∂Ω × ∂Ω) × T ∗ (R × Ω × Ω) : N∂Ω×∂Ω = {(t, x, y, τ, ξ, x ∈ ∂Ω, y ∈ ∂Ω, ξ˜ = ξ|Tx (∂Ω) , η˜ = η|Ty (∂Ω) }. As in the previous section, it is easy to check that N∂Ω×∂Ω × Γk+ intersects transversally T ∗ (R × ∂Ω) \ {0} × Δ(T ∗ (R × Ω) \ {0}) × T ∗ (R × Ω) \ {0} and this explains the order 1/4 of the operator i∂Ω × i∂Ω . Therefore, following [H4] for the composition of canonical relations with transversal intersection, we deduce that Γk∂,+ = N∂Ω×∂Ω ◦ Γk+ ⊂ T ∗ (R × ∂Ω × ∂Ω) \ {0} is a canonical relation. We need to introduce a suitable parameterization of Γk∂,+ . For this purpose, we introduce the billiard map. Consider the unit ball bundle B ∗ (∂Ω) = {(y, η) ∈ T ∗ (∂Ω) : h0 (y, η) < 1},

POISSON SUMMATION FORMULA FOR MANIFOLDS WITH BOUNDARY

99

where h0 (y, η) is the induced Riemannian metric on ∂Ω. In local coordinates, given above, a point (0, y , ητ ) is in B ∗ (∂Ω) if m(0, y , η ) < τ 2 , since m(0, y , η ) is the induced Riemannian metric on ∂Ω in local coordinates. This is just the condition that (0, y , τ, η ) is a hyperbolic point for the operator  (see Section 1.2). Let S ∗ (∂Ω) = {(y, η) ∈ T ∗ (Ω) : y ∈ ∂Ω, |η| = 1} be the cosphere bundle and let Σ± = {(y, η) ∈ S ∗ (∂Ω) : ±η, ν(y) > 0}. Consider the projection πΣ+ : Σ+ (y, η) → (y, η|T (∂Ω) ) ∈ B ∗ (∂Ω). Then the billiard map has the form β = πΣ+ ◦ B ◦ πΣ−1+ : B ∗ (∂Ω) → B ∗ (∂Ω), where B : Σ+ −→ Σ+ is the billiard ball map defined in Section 2.1. Thus, β is defined in a neighbourhood of a hyperbolic point if the bicharacteristic issued from this point hits transversally T ∗ (∂Ω). √ −Δ. In local coordinates, Consider the principal symbol g = |η| of g = (|η1 |2 + m(y, η )|)1/2 . Let W1 = {(t, 1, x, ξ ) ∈ T ∗ (R × ∂Ω) ∩ H}, H being the hyperbolic set introduced in Section 1.2. Then W1 is a submanifold of codimension 1 of the set {(t, τ, x, ξ ) ∈ T ∗ (R × ∂Ω) ∩ H} and Ft induces a map ϕ on W1 . Let ωW1 be the restriction on W1 of the canonical symplectic form of T ∗ (R × Ω) and consider the projection π : W1

(t, 1, x, ξ ) → (x, ξ ) ∈ Σ+ . Therefore, there exists a unique two-form ωS on Σ+ of maximal rank such that (π)∗ ωS = ωW1 and Σ+ has a symplectic structure with canonical symplectic form ωS . Now consider the diagram

W1 ⏐ ⏐ π

ϕ

−−−−→

W1 ⏐ ⏐ π

B

Σ+ Σ+ −−−−→ ⏐ ⏐ ⏐ ⏐ πΣ+  πΣ +  β

B ∗ (∂Ω) −−−−→ B ∗ (∂Ω)

100

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Here ϕ is the restriction of the generalized flow on W1 . It is clear that ϕ preserves W1 and the fibration W1 −→ Σ+ , and we can define the billiard map B as B = π ◦ ϕ ◦ π −1 . On the other hand, (π −1 ◦ πΣ−1+ )B ∗ (∂Ω) ⊂ W1 ∩ H. Applying Lemma 4.1.1, we deduce that ϕ preserves the form ωW1 on W1 ∩ H, and this implies that B preserves ωS on πΣ−1+ (B ∗ (∂Ω)). Next, we define a symplectic preserving ωB . form ωB = (πΣ−1+ )∗ ωS on B ∗ (∂Ω) and  β becomes a symplectic map −1 ∗ In local coordinates, we have ωB = n−1 dy ∧ dη . Notice that π (B (∂Ω)) does j j j=1 Σ+ ∗ not contain directions in S (∂Ω), which are tangent to ∂Ω and πΣ+ is a diffeomorphism on π(W1 ). By using the map β k , we introduce a parameterization of Γk∂,+ given by     

  η η , τ, τ β k 0, y , , y , η ∈ Γk∂,+ . Γ (0, y , τ, η ) → Tk (πΣ+ )−1 0, y , τ τ (4.17) Here Tk ((πΣ+ )−1 (0, y , ητ )) is the length of the reflecting bicharacteristic issued from πΣ−1+ (0, y , ητ ) and hitting the boundary after k reflections at β k (0, y , η k τ ), while τ (x, ξ) = (x, τ ξ). Obviously, on Γ∂,+ , we have a half-density (|dy | ∧ |dη | ∧ |dτ |)1/2 . Recall that the principal symbol of FˆB (t, x, y) over Ω × Ω for t close to T has the form i π (−1)mγ e σγ δcan . We ignore the coefficient in front of δcan , and for simplicity we write k instead of mγ . After the application of i∗∂Ω×∂Ω , we must find the composition of the half-density δcan = (|dt| ∧ |dy| ∧ |dη|)1/2 on Γk+ with the half-density on N∂Ω×∂Ω . According to the rules of composition of densities in the case of transversal intersection (see [H4] and [DG]), first we take the exterior tensor product of the half-density |dt ∧ dτ ∧ dx ∧ dξ1 ∧ dξ ∧ dy ∧ dη1 ∧ dη |1/2 on N∂Ω×∂Ω and the half-density |ds ∧ dz ∧ dζ|1/2 on Γk+ at the points of the fiber product F , where the components in T ∗ (R) × T ∗ (Ω) × T ∗ (Ω) of these half-densities are equal (see the following diagram). Γk+ ⏐ ⏐ i

π

←−−−−

π

F ⏐ ⏐ π

Ω −− N∂Ω×∂Ω T ∗ (ℝ) × T ∗ (Ω) × T ∗ (Ω) ←−−

α

−−−−→ N∂Ω×∂Ω ◦ Γk+ = Γk∂,+

POISSON SUMMATION FORMULA FOR MANIFOLDS WITH BOUNDARY

101

Second, we divide the tensor product by the canonical half-density |ds ∧ dτ ∧ dx1 ∧ dx ∧ dξ1 ∧ dξ ∧ dz ∧ dζ|1/2 , on the common T ∗ (R) × T ∗ (Ω) × T ∗ (Ω) component. Thus, we obtain the half-density |dt ∧ dy ∧ dη1 ∧ dη |1/2 |dx1 |1/2 on Γk∂,+ . ∗ The numerator above is a half-density on R × T∂Ω (Rn ). The problem is to compare this half-density on Γk∂,+ with the half-density |dy ∧ dη ∧ dτ |1/2 in the parameterization (4.17). Consider a parameterization ∗ ∗ (Rn ) → T ∗ (R) × T ∗ (Ω) × T∂Ω (Ω), Φ+ : R × T∂Ω

of the graph of the billiard flow given by Φ+ (t, 0, y , η1 , η ) ∗ = (t, −|η|, exp (tHg )(0, y , η1 , η ), 0, y , η1 , η ) ∈ T ∗ (R) × T ∗ (Ω) × T∂Ω (Ω)

and denote by ωT ∗ (Rn ) the canonical 2n symplectic form on T ∗ (Rn ). This form can be ∗ (Ω) by a trivial lifting. Therefore, considered as a 2n form on T ∗ (R) × T ∗ (Ω) × T∂Ω ∗ 1/2 ∗ n |Φ+ ωT ∗ (Rn ) | is a half-density on R × T∂Ω (R ). Set 



γ1 (y , η , τ ) =

1−

m(0, y , η ) . τ2

Then have the following. Lemma 4.2.1: We have the equality

1/2

|dt ∧ dy ∧ dη1 ∧ dη |

 −1/2   η1   =  2 |Φ∗+ ωT ∗ (Rn ) |1/2   η1 + m(y, η ) 

∗ as half-densities on R × T∂Ω (Rn ). On the set Γk∂,+ |y1 =0,t=0 , the factor on the right-hand side becomes γ1 (y , η , τ )−1/2 .

∂ ∂ Proof: We have dΦ+ ∂t = ∂t Φ+ (0, y , η1 , η ) = Hg , where Hg is the Hamiltonian vector field of g = (η12 + m(y, η ))1/2 . Thus,

102

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS



Φ∗+ ωT ∗ (Rn )

∂ ∂ ∂ ∂ Φ (0, y , η1 , η ), dΦ+ , dΦ+ = ωT ∗ (Rn ) , dΦ+ dt ∧ dy ∧ dη1 ∧ dη ∂t + ∂y ∂η1 ∂η   ∂ ∂ ∂ = ωT ∗ (Rn ) Hg , dΦ+ , dΦ+ , dΦ+ ∂y ∂η1 ∂η   ∂ ∂ ∂ = ωT ∗ (Rn ) Hg , , . , ∂y ∂η1 ∂η



Here we have used the fact that dΦ+ |T (Ω) is a symplectic diffeomorphism. On the ∂ other hand, Hg = √ 2 η1 + · · ·, where · · · denotes vector fields spanned by ∂y1 η1 +m(y,η ) ∂ ∂ ∂ , , . Evaluating the form ωT ∗ (Rn ) ∂y ∂η1 ∂η

by using the coordinates (y1 , y , η1 , η ), we   ∂ ∂ ∂ ∂ = 1. , , , ωT ∗ (Rn ) ∂y1 ∂y ∂η1 ∂η

have

Now consider the restriction of Φ+ on

(0, y , η1 , η ) : η1 =



 τ2 − m(0, y, η ), (0, y , τ, η ) ∈ Γ ,

and the parameterization (4.17) of Γk∂,+ over the domain points. Then η12 + −1/2 m(0, y , η ) = τ 2 , and the factor ( √ 2 η1 )−1/2 becomes γ1 .  η +m(0,y ,η ) 1

Next we are going to consider the points Γk∂,+ |x1 =0,t=T k in the image of Φ+ . ∗ Lemma 4.2.2: On the set Γk∂,+ |x1 =0,t=T k ⊂ Φ+ (R × T∂Ω (Rn )) we have

|Φ∗+ ωT ∗ (Rn ) |1/2 |dx1 |1/2

−1/2

= γ1

   η , τ |dτ ∧ dy ∧ dη |1/2 . τ β k y , τ



Proof: We are going to examine the half-density |Φ∗+ ωT ∗ (Rn ) |1/2 |dx1 |−1/2 . Notice that Φ∗+ dτ = dτ and Φ+ (y1 ) = x1 yields Φ∗+ (dy1 ) = dx1 . Thus,    ωT ∗ (Rn ) 1/2  = |Φ∗+ (dy ∧ dη ∧ dη1 )|1/2 . |Φ∗+ ωT ∗ (Rn ) |1/2 |dx1 |−1/2 = Φ∗+ |dy |  1

Next we have η1 =



τ 2 − m(0, y , η ), hence

|Φ∗+ (dy ∧ dη ∧ dη1 )| = |Φ∗+ (dy ∧ dη ) ∧ Φ∗+ (dη1 )|      = (dy ∧ dη ) ∧ Φ∗+ d τ 2 − m(0, y , η ) 

POISSON SUMMATION FORMULA FOR MANIFOLDS WITH BOUNDARY

and

103

       τ dτ   ∗   τ 2 − m(0, y, η )  = Φ∗+  ∧ ··· Φ+ d 2  τ − m(0, y , η )   ⎞−1/2  ⎛ m 0, τ (β k )∗ (y , ητ ) ⎠ |dτ | ∧ · · · , = ⎝1 − τ2

where . . . denotes forms which multiplied by (dy ∧ dη ) vanish. Here we have used the equality     η η ∗ ∗k = τβ dy ∧ d τ Φ+ dy ∧ d τ τ = (dy ∧ dη )|τ β k (y ,η /τ ) which follows from the fact that β preserves the symplectic form on B ∗ (∂Ω). This  completes the proof. Combining these two lemmas, we obtain the following. mγ

Proposition 4.2.3: The principal symbol of (i∂Ω × i∂Ω )∗ V+ nates (4.17), modulo Maslov factors, has the form

in the local coordi-

  −1/2 |dτ ∧ dy ∧ dη |−1/2 . (−1)mγ γ1 (τ, y , η )γ1 τ, τ β k (y , η /τ )

(4.18)

Remark 4.2.4: The same argument works if we consider the symbol (ix1 =xˆ 1 × iy1 =yˆ1 )∗ V+ γ , m

ˆ1 . Then the coefficients γ1 where the restriction is on the surfaces y1 = yˆ1 and x1 = x is defined by m(y1 , y , η ).

4.3

Poisson summation formula

In this section we use the notation of the previous sections. Let γ be a periodic ordinary reflecting bicharacteristic of  in Ω with period T = Tγ > 0. Let Pγ be the Poincaré map of γ introduced in Section 2.3. Denote by π : T ∗ (R × Ω) −→ Ω the usual projection. Then π(γ) = γ˜ will be a (generalized) periodic geodesic in Ω. Throughout this section we make the following assumptions: (i) if δ is a periodic bicharacteristic of  in Ω with period T , then π(δ) = π(γ), (ii) det(Pγ − Id) = 0.

104

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Since by Lemma 1.2.10 the set LΩ of periods of periodic bicharacteristics in Ω is closed, T is an isolated point in sing supp σ(t). Indeed, if there exist bicharacteristics with periods Tk → T , passing over (xk , ξk ) ∈ T ∗ (Ω), we can find subsequences xk → x0 , ξk → ξ0 , and a bicharacteristic δ with period T passing over (x0 , ξ0 ). If γ does not pass through (x0 , ξ0 ), we obtain a contradiction with (i). If γ passes through (x0 , ξ0 ), we obtain a contradiction with (ii). Choose  > 0 and I = (T − , T + ) ⊂ R+ so that sing supp σ(t) ∩ I = {T }. Let Oγ be a sufficiently small open neighbourhood of γ˜ . Then (i) and the choice of I yield {(t, x, x, τ, ξ, ξ) ∈ C : t ∈ I, x ∈ Ω◦ \ Oγ } = ∅. Applying the argument at the end of Section 3.4, with I instead of Δ0 , for W ⊂ Ω◦ \ Oγ we obtain  E(t, x, x)dx ∈ C ∞ (I). W

For W ∩ ∂Ω = ∅ and W ∩ Oγ = ∅ we obtain the same result by using Proposition 3.4.3. To study the leading singularity of σ(t) for t close to T , we must examine the traces M   E(t, x, x)ds, j=1

Ω∩Uj

where Oγ ⊂

M !

Uj

(4.19)

j=1

is a covering and Uj ⊂ Ω◦ for j = 1, . . . , M0 , Uj ∩ ∂Ω = ∅ for M0 + 1 ≤ j ≤ M . First we study the trace on Uj ⊂ Ω◦ , and for simplicity we write U instead of Uj . To microlocalize the problem, introduced a covering T ∗ (U ) \ {0} ⊂

N !

(U × Vk ),

k=1

Vk being small conic neighbourhoods. As in Section 4.1, suppose that γ pass over / Vk , (y0 , η0 ) ∈ U × Vk0 . Choose the above covering sufficiently fine to arrange η0 ∈ whereas k = j0 , {(t, x, x, τ, ξ, ξ) ∈ C : t ∈ I, (x, ξ) ∈ U × Vk } = ∅.

(4.20)

POISSON SUMMATION FORMULA FOR MANIFOLDS WITH BOUNDARY

105

˜ ∈ L0 (Ω◦ ) with principal symbols ˜b (y, η) Choose pseudo-differential operators B j j and full symbols β˜j (y, η) so that N 

˜b = 1 on T ∗ (U ) \ {0}, j

j=1

conesupp β˜j (y, η) ⊂ U × Vj .  ˜ The operator j B j is elliptic on U , and there exists a pseudo-differential operator P ∈ L0 (Ω◦ ) such that  ˜ = Id + R(y, D ), PB j y j ∞ ˜ , we obtain a partition R(y, Dy ) being an operator with C kernel. Setting Bj = P B j ∗ of unity on T (U ) \ {0} given by j Bj = Id + R.

Bk∗

The kernel of the operator cos(A1/2 t)Bk has the form Bk∗ (y, Dy )E(t, x, y), where is the operator adjoint to Bk . By using Proposition 3.4.2, it is easy to see that W F (Bk∗ (y, Dy )E(t, x, y))|I×U ×U ⊂ {(t, x, y, τ, ξ, η) ∈ C : (y, η) ∈ W F (Bk )}.

As in the proof of Proposition 3.3.1, we can choose a pseudo-differential operator Ck , elliptic at ρ = (t, x, yˆ, τ, ξ, ηˆ) ∈ C and a pseudo-differential operator Ak with (ˆ y , ηˆ) ∈ / W F (Ak ), so that Ck A∗k Bk∗ E(t, x, y) ∈ C ∞ . Thus, ρ∈ / W F (Bk∗ (y, Dy )E(t, x, y)|I×U ×U ). In the case ρ ∈ / C the result follows immediately and (4.20) implies for k = k0 that  (Bk∗ (y, Dy )E)(t.x, x)dx ∈ C ∞ (I). U

We are going to study exp(∓itA)Bk0 , and we omit the index k0 writing B and V instead of Bk0 and Vk0 . Set Γ = U × V and apply the construction of Section 4.1 for B with W F (B) ⊂ Γ. If F± (t, x, y) are the kernels of exp (∓itA), we have B ∗ (y, Dy )F± (t, x, y) − FˆB± (t, x, y) ∈ C ∞ (I × U × U ). For the distribution FˆB+ = (−1)mγ Vm+γ , given by (4.9), we obtain modulo terms in C ∞ (I) the representation    ∞  + −n ˆ dx dr eir(ϕ(t,x,ω)−x,ω) FB (t, x, x)dx = (2π) U

U

×

 j

1

V ×Sn−1

bj (t, x, ω)rn−1−j dω,

106

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

where the integral is interpreted in the sense of distributions. We can assume that the coordinates are chosen so that U = {(x1 , x ) ∈ Rn : x ∈ U , 0 < α < x1 < β}, η0 = (1, 0, · · · , 0), η1 > 0 on V , where {(x1 , 0) : α ≤ x1 ≤ β} ⊂ γ. Here U ⊂ Rn−1 is an open neighbourhood of y0 = 0 and α < y0,1 < β. Introduce the integral Iα,β = (2π) ×

−n







β

dx1 α

 

∞

eir(ϕ(t,x,ω)−x,ω)

dr 1

U V ∩Sn−1

bj (t, x, ω)rn−1−j dx dω.

(4.21)

j

Our aim is to apply a stationary phase argument for the integral with respect to x and ω, considering x1 as a parameter. The critical points satisfy the equalities ϕx = ω , ϕω = x. The representation of C+ by the phase ϕ, given in Section 4.2, implies ϕt (t, x, ω) = −|ϕx (t, x, ω)| = −1. Moreover, ω1 > 0 and ϕx1 > 0 on V ∩ Sn−1 . Thus, at the critical points (ˆ x , ω ˆ ), we get ϕt = −1, ϕx = ω and the form of C+ yields (x1 , x ˆ , ω ˆ ) = Φt (x1 , x ˆ , ω ˆ ). This equality is possible for t = T, x ˆ = 0, ω ˆ = η0 , only. Next, introduce local coordinates   1 − |η |2 , η ∈ V ∩ Sn−1 Rn−1 ⊃ W η → and write the phase in the form   ϕ t, x1 , x , 1 − |η |2 , η − x1 1 − |η |2 − x , η . To apply a stationary phase argument, we need to examine the matrix   ϕx x ϕx η − In−1 (ρ(x1 )), Δ(ρ(x1 )) = ϕη x − In−1 ϕη η where ρ(x1 ) = (T, x1 , 0, η0 ) and Im denotes the (m × m) identity matrix. Here we have used the equality ϕω1 (ρ(x1 )) = x1 for α < x1 < β.

POISSON SUMMATION FORMULA FOR MANIFOLDS WITH BOUNDARY

107

Consider the generalized Hamiltonian flow Ft : (ϕη , η) → (x, ϕx ) defined in Section 1.2. For the differential dFT , we get " (dFT )

ϕηx δx + ϕηη δη

#

δη

" =

#

δx ϕxx δx + ϕxη δη

,

hence " dFT − In = Qϕ =

=

ϕ−1 ηx − In

−ϕ−1 ηx ϕηη

#

−1 ϕxx ϕ−1 ηx −ϕxx ϕηx ϕηη + ϕxη − In " #" # " −1 # ϕxx ϕxη − In ϕηx −ϕ−1 0 −In ηx ϕηη

In

0

ϕηx − In

ϕηη

0

In

.

Clearly, ϕxη1 (ρ(x1 )) = η0 , ϕηη1 (ρ(x1 )) = 0, detϕxη (ρ(x1 )) = detϕx η (ρ(x1 )). According to the definition of Pγ in Section 2.3, the Poincaré map Pγ , modulo conjugations, is the restriction of dΦT on the linear space L = {(0, δx , 0, δη ) : δx ∈ Rn−1 , δη ∈ Rn−1 }. Let Q ϕ be the (n − 1) × (n − 1) matrix obtained from Qϕ replacing In by In−1 and making only the derivatives of ϕ with respect to x and η . Therefore, for each l = (0, δx , 0, δη ) ∈ L, we have (dFT − In )l , l  =



Q ϕ



   δx δx , , . δη δη

This implies (detϕx η )−1 (detΔ)(ρ(x1 )) = det(Pγ − I). We may apply the representation (4.10) at the points ρ(x1 ) and we get  π  b0 (ρ(x1 )) = (−1)mγ exp i σγ |detϕx η |1/2 (ρ(x1 )), 2 σγ ∈ N being the Maslov index of γ. By the Euler equality for the phase ϕ, we deduce ϕ(t, x1 , 0, η0 ) = T − t + x1 .

108

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Applying the stationary phase method, we get    (2π)−n eir(ϕ(t,x,ω)−x,ω) bj (t, x, ω)rn−1−j dx dω U V ∩ Sn−1

= eir(T −t)

"N −1 

j

#

ck r−k + O(r−N

,

k=0

where c0 =

 π  i exp i βγ |det(Pγ − I)|−1/2 2π 2

and βγ = 2mγ + σγ +

sign Δ − 1. 2

(4.22)

The number βγ is locally constant since for small β − α, we have sign Δ(ρ(x1 )) = const and the integers mγ ∈ N, σ ∈ N depend on γ, only. Thus, c0 is independent of x1 and r. To complete the computation of the leading term of Iα,β , take the Fourier transform of the Heaviside function Y (r). The integral in r can be interpreted in the sense of distributions, hence  ∞ ei(T −t) dr == −i(t − T − i0)−1 + L1loc (R). 1

Finally, for t ∈ I we have Iα,β =

 π  β−α exp i βγ |det(Pγ − I)|−1/2 (t − T − i0)−1 + L1loc (R). (4.23) 2π 2

A similar argument with trivial modifications works for R− B (t, x, y). On the other hand, we may use the equality F− (t, x, y) = F+ (t, x, y). Then, modulo C ∞ (I), we get   FˆB− (t, x, x)dx = F − (t, x, x)dx U U   = F + (t, x, x)dx = FˆB+ (t, x, x)dx = Iα,β + L1loc (R). U

U

Thus, 



β−α |det(Pγ − I)|−1/2 Fˆ + (t, x, x)dx = 2π U % $  π  ×Re exp i βγ (t − T − i0)−1 + L1loc (R). (4.24) 2

E(t, x, x)dx = Re U

POISSON SUMMATION FORMULA FOR MANIFOLDS WITH BOUNDARY

109

Now we turn to the case ∂Ω ∩ U = ∅. Let π(γ) = γ˜ =

pγ !

lj , lj = [qj , qj+1 ]

j=1

with qj ∈ ∂Ω, qpγ +1 = q1 and mγ = mpγ . Let qj+1 ∈ U and let (y1 , y ), y ∈ U ⊂ Rn+1 be local coordinates chosen at the end of Section 4.1. Choose a small neighbourhood J of 0 ∈ R and set O = J × U , J , U being the same as in Section 4.1. Introduce a covering N ! (O × Vk ), T ∗ (O) \ {0} ⊂ k=1

\ {0}. Choose zero where Vk are small conic neighbourhoods in Rt \ {0} × Rn−1 y order pseudo-differential operators B(t , y, Dt , Dy ) depending smoothly on 0 ≤ y1 ≤ α and satisfying W F ((Bk (·, y1 , ·)) ⊂ O × Vk . As earlier, construct a microlocal partition of unity on T ∗ (O) \ {0} given by  Bk (t , y, Dt , Dy ) = Id + R , k

R being an operator with C ∞ smooth kernel. Recall that Ω ∩ U = {(y1 , y ) : y ∈ U , 0 ≤ y1 ≤ α} and denote by q(y, τ, η) = η12 − τ 2 + m(y, η ) the principal symbol of  in the local coordinates, where m(y, η ) ≥ c0 |η |2 , c0 > 0. Let qj+1 = (0, y0 ), and let η1± be the roots of the equation q(y, τ, η1 , η ) = 0 with respect to η1 . Assume that γ passes over (0, y0 , τ0 , η0 ) and set μ± = (0, y0 , ±τ0 , η0 ), η0± = η1± (0, y0 , τ0 , η0 ). Since τ02 − m(y0 , η0 ) > 0, we have τ0 = 0 and without loss of generality we suppose that τ0 < 0. By convention , choose η1± so that ±

∂q (0, y0 , τ0 , η0± , η0 ) > 0. ∂η1

Then (η0+ , η0 ) (resp. (η0− , η0 )) is collinear with the direction of lj+1 (resp. lj ) at qj+1 . Suppose Γ± = O × V± are small open conic neighbourhoods of μ± . By the / Γ+ ∪ Γ− there assumptions (i), (ii), we may choose Γ± small enough, so that for μ ∈ are no periodic bicharacteristics of  in Ω passing over μ and having periods in I = (T − , T + ), where  > 0 is small enough. Taking a suitable partition of unity on T ∗ (O) \ {0}, assume that for k = k0 and 0 ≤ y1 ≤ α we have W F (Bk (·, y1 , ·)) ∩ Γ± = ∅.

110

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

± Let FB (t − t , x, y) be the kernels of the operators k

exp(∓i(t − t )A)Y (t − t )Bk (t , y, Dt , Dy ). According to the results of Section 3.4 for the kernels of exp(∓i(t − t )A)Y (t − t ), for t ∈ I, t ∈ J we get ± (t, t , x , x , τ, τ, ξ , ξ ) ∈ / W F (FB (t − t , x1 , x , y1 , y ), k

whenever (x1 , y1 ) ∈ [0, α] × [0, α], τ ∈ R, (t , y , τ, η ) ∈ W F (Bk (·, y1 , ·)). Then for k = k± we have  ω∩U

± FB (t − t , x, x)dx ∈ C ∞ (I × J ). k

± ± In the following we treat FB and we omit k± in the notation of FB and B. Let Γ = k± O × V be open conic neighbourhood of μ+ = μ and suppose W F (B(·, y1 , ·)) ⊂ Γ. As we have mentioned in Section 4.1, we can construct Fourier integral distribution + (t − t , x, y) related to the canonical relation M+ = MΓ,+ defined at the end of FB Section 4.1. In the following we consider (x1 , y1 ) ∈ [0, α] × [0, α] as parameters. For fixed x1 , y1 and for t ∈ I, t ∈ J , the projection

M+ (t, t , x, y, τ, τ, ξ, η) → (t , y , τ, η ) ∈ J × U × V is locally a diffeomorphism. In fact for fixed x ˆ1 , yˆ1 , the generalized flow induces a homogeneous canonical transformation from {(t , yˆ1 , y , τ, η ) : (t , y , τ, η ) ∈ J × U × V } into {t, x ˆ1 , x , τ, η ) : (t, x , τ, ξ ) ∈ I × U × V } Then there exists a generating function ϕ(t, x1 , x , y1 , τ, η ), depending smoothly on x1 , y1 and homogeneous of order 1 in (τ, η ), such that M+ has locally the form {(t, ϕτ , x1 , x , y1 , ϕη , ϕt , τ, ϕx , η1+ , η ) : (t, x , τ, η ) ∈ I × U × V }. Here η1+ (t, x1 , x , y1 , τ, η ) is determined from the equation q(y1 , ϕη , τ, η1 , η ) = 0 with respect to η1 so that η1+ (T, 0, y0 , 0, τ0 , η0 ) = η0+ . Moreover,  det

ϕt,τ ϕt,η ϕx ,τ ϕx ,η

 = 0.

POISSON SUMMATION FORMULA FOR MANIFOLDS WITH BOUNDARY

111

Now we write + FB (t − t , x, y) = (2π)−n

×

∞ 













eiϕ(t,x1 ,x ,y1 ,τ,η )−it τ −iy ,η 

˜b (t, t , x, y, τ, η )dτ dη j

j=0

with ˜bj homogeneous of order (−j) with respect to (τ, η ). Notice that τ < 0 and |η | ≤ C0 |τ | on M+ with C0 > 0 independent of η . We may suppose that bj vanish for |η | ≥ C0 |τ |. Thus, taking t = 0 and setting τ = −r, r > 0, modulo C ∞ terms, we get  α  ∞  + FB (t, x, x)dx = (2π)−n dx1 dr Iα = 0 1 Ω∩U    + × eirΨ (t,x,η ) bj (t, x, x1 , −1, η )rn−1−j dx dη U |η |≤C0

j

with Ψ+ (t, x, η ) = ϕ(t, x1 , x , x1 , −1, η ) − x , η , bj (t, x, x1 , −1, η ) = ˜bj (t, 0, x1 , x , x1 , x , −1, η ). For the integral with respect to x and η , we wish to apply a stationary phase argument. The critical points x ˆ , ηˆ satisfy the equalities ˆ , ϕx = ηˆ , ϕη = x ˆ , −1, ϕx1 (t, x1 , x ˆ , x1 , −1, ηˆ ), ηˆ ) = q(x1 , x ˆ , −1, η1+ , ηˆ ) = 0. q(x1 , x Since ϕx1 (T, 0, y0 , 0, −1, η0 ) = η0+ (0, y0 , −1, η0 ), we deduce ϕx1 = η1+ at the critical points and ˆ , η1+ (· · ·), ηˆ ) = Φt−ϕτ (x1 , x ˆ , η1+ (· · ·), ηˆ ). (x1 , x This is possible only for x ˆ = y0 , ηˆ = η0 , ϕτ (T, x1 , y0 , x1 , −1, η0 ) = 0, t = T. Since on M+ we have ϕt ( . . . ) = τ , we get ϕt ( . . . ) = −1, ϕtt ( . . . ) = 0 and deduce Ψ+ (t, x1 , x , η ) = Ψ+ (T, x1 , x , η ) + (T − t). Therefore, we reduce the analysis of the integral with respect to x , η to that with phase function Ψ+ (T, x1 , x , η ).

112

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Set "



Δ+ (T, x1 , x , η ) = " =

+ Ψ+ x x Ψx η

# (T, x1 , x , η )

+ Ψ+ η x Ψη η

ϕx x

ϕx η − In−1

ϕη x − In−1

ϕη η

# (T, x1 , x , x1 , −1, η ).

Since det ϕx η = 0, as in the previous case of a neighbourhood included in Ω◦ , we conclude by using the Poincaré map Pγ and the transversality of lj+1 to ∂Ω that det Δ+ (T, 0, y0 , η0 ) = det ϕx η (T, 0, y0 , 0, −1, η0 ) det(Pγ − I) = 0. By the implicit function theorem, there exist functions x (x1 ), η (x1 ) determined for 0 ≤ x1 ≤  so that gradx η Ψ+ (T, x1 , x (x1 ), η (x1 )) = 0, x (0) = y0 , η (0) = η0 . Moreover, by the Euler equality for the function Ψ+ (T, x1 , x , η ) homogeneous with respect to η , we deduce Ψ+ (T, x1 , x1 (x ), η (x1 )) = 0. Assuming α ≤  and Γ+ small enough, we apply the stationary phase argument with parameter x1 for the integral with respect to (x , η ) (see, for example Theorem 7.7.6 in [Hl]) and obtain    + −n eirΦ (T,x1 ,x ,η ) bj (t, x1 , x , x1 , τ, η )rn−1−j dx dη (2π) U η |≤C0

j

= eiπs/4 |Δ+ (x1 )|−1/2 b0 (t, x1 , x (x1 ), x1 , −1, η (x1 )) + O(r−1 ), "

where Δ+ (x1 ) =

Ψ+ x x Ψx η Ψ+ η x

Ψ+ η η

# (T, x1 , x (x1 ), η (x1 ))

and s = sign Δ+ (0). The term O(r−1 ) yields lower order singularity after the integration with respect to r. On the other hand, writing b0 (t, x1 , x (x1 ), x1 , −1, η (x1 )) = b0 (T, x1 , x (x1 ), x1 , −1, η (x1 )) + (t − T )c0 (t, x1 ), we conclude that the term with coefficient (t − T ) yields lower order singularity since the Fourier transform of the Heaviside function Fr→t Y (r) is the distribution πδ(t) − iv.p. 1t and  ∞ 1 1 i ∈ L1loc (R). e−ir(t−T ) (t − T )dr = − (t − T )v.p. 2π 1 2π t−T

POISSON SUMMATION FORMULA FOR MANIFOLDS WITH BOUNDARY

113

Now we apply Proposition 4.2.3 for the principal symbol b0 at ρ(x1 ) = (T, x1 , x (x1 ), x1 , −1, η (x1 )). The point (x1 , x (x1 ), η (x1 )) is periodic and exploiting Proposition 4.2.3 and the remark at the end of the previous section, the factors in (4.18) for τ = 1 become (1 − m (x1 , x (x1 ), η (x1 )))−1/2 = β0 (x1 ). Consequently, since τ = −1, the symbol b0 (ρ(x1 )), modulo the half-density |dτ ∧ dx ∧ dη |1/2 , has the form π

b0 (ρ(x1 )) = (−1)mγ ei 2 σγ |det ϕx η (ρ(x1 ))|1/2 β0 (x1 ). For the integral with respect to x1 , we get  α  π |Δ+ (x1 )|−1/2 b0 (ρ(x1 ))dx1 = (−1)mγ ei 2 σγ |det(Pγ − I)|−1/2 0

α

β0 (x1 )dx1 .

0

&α On the other hand, 0 β0 (x1 )dx1 = lj,0 , where lj,0 is the length of the segment lj lying in U ∩ Ω. In fact, lj,0 is parameterized by (x1 , x (x1 )), 0 ≤ x1 ≤ α, and   dx (x1 ) = 1 + m(x1 , x (x1 ), η (x1 ))β02 (x1 ) = β02 (x1 ). 1 + m x1 , x (x1 ), dx1 Thus, modulo smooth terms, we get ( '   sign Δ+ (0) π 2mγ + σγ + −1 F + (t, x, x)dx = lj,0 exp i 2 2 Ω∩U ×|det(Pγ − I)|−1/2 (t − T − i0)−1 + L1loc (R). (4.25) To find the trace of FBk− (t − t , x, y), we use once more the argument based on the equality F − (t − t , x, y) = F + (t − t , x, y), F − (t − t , x, y) being the kernel of the operator exp (i(t − t )A). Then for t ∈ I, t ∈ J , modulo C ∞ terms, we have   − F − (t − t , x, x)dx = FB (t − t , x, x)dx k− Ω∩U Ω∩U   + = F + (t − t , x, x)dx = FB (t − t , x, x)dx k+ Ω∩U

Ω∩U

and taking t = 0, we deduce  ( '  sign Δ+ (0) π − 2mγ + σγ + −1 FBk (t, x, x)dx = lj,0 exp −i − 2 2 Ω∩U × |det(Pγ − I)|−1/2 (t − T + i0)−1 + L1loc (R).

(4.26)

114

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

On the other hand, we may exploit a local representation of the canonical relation M− by a phase function ψ(t, x1 , x , y1 , τ, η ). Setting Ψ− (t, x, η ) = ψ(t, x1 , x , x1 , 1, η ) − x , η , # " − Ψx x Ψ− x η Δ− (T, x1 , x , η ) = (T, x1 , x , η ), − − Ψη x Ψη η − we repeat the above argument to compute the trace of FB over U . Comparing the k− arguments of the leading terms, and taking into account that for the leading term of − FB we have Maslov index −σγ , we get k −

−

sign Δ− (0) sign Δ+ (0) = (mod 4). 2 2

(4.27)

Below we consider βγ , given by (4.21), as an element of Z4 . Our aim is to show that βγ modulo 4 depends only on γ. If for some segment lj of γ we have Ω◦ ⊃ Uk ∩ lj = ∅, Ω◦ ⊃ Um ∩ lj = ∅, covering lj by a chain of neighbourhoods connecting Uk and Um and using the leading terms in (4.23), we conclude that βγ does not depend on the choice of Uk . To prove that βγ does not depend on the choice of a segment lj , take two sufficiently small neighbourhoods Uk ⊂ Ω◦ , k = j, j + 1, so that Uj ∩ lj = ∅, Uj+1 ∩ lj+1 = ∅, Uk ⊂ {(x1 , x ), x ∈ U , 0 <  ≤ x1 ≤ α}, k = j, j + 1. For the analysis of the contribution of the outgoing segment lj+1 we study the trace + over Uj+1 , and from (4.23) and (4.25) for the number βγ,j+1 related to lj+1 of FB we deduce (j+1) sign Δ+ − 1. βγ,j+1 = 2mγ + σγ + 2 (j+1)

Here Δ+ is related to a phase function ϕj+1 parameterizing M+ . Next, for the analysis of the contribution coming from the incoming segment lj we must study the + − trace of FB = FB over Uj , and for the corresponding number βγ,j we get (j)

βγ,j = 2mγ + σγ −

sign Δ− − 1, 2

(j)

where Δ− is related to a phase function ψj parameterizing M− . In the Fourier inte+ , we take the phase ψj with sign – and this explains the gral operator related to FB sign – in the expression of βγ,j . Finally, by (4.27), we have βγ,j+1 = βγ,j (mod 4), so βγ ∈ Z4 depends only on γ.

POISSON SUMMATION FORMULA FOR MANIFOLDS WITH BOUNDARY

115

Summing up the contributions of lj in the leading terms in (4.24), (4.25), we obtain the following. Theorem 4.3.1: Let γ be a periodic ordinary reflecting bicharacteristic of  in Ω with period Tγ > 0 and primitive period Tγ# . Assume the conditions (i) and (ii) fulfilled. Then the distribution σ(t) near Tγ has the form σ(t) =

Tγ# π Re [exp (i βγ )(t − Tγ − i0)−1 ]|det(Pγ − I)|−1/2 + L1loc (R). (4.28) 2π 2

Remark 4.3.2: For the proof of (4.28), we need a weaker result concerning the behaviour of term (4.25). In fact, it suffices to show that  F + (t, x, x)dx = O(α)|det(Pγ − I)|−1/2 (t − T − i0)−1 + L1loc (R) Ω∩U

with O(α) → 0 as α → 0. To obtain this, it is not necessary to know the precise form of b0 (T, x1 , x (x1 ), x1 , −1, η (x1 )). Now we discuss briefly the Neumann and Robin boundary problems. For these problems we repeat the construction from Section 4.1. Let Vk+ be Fourier integral operators related to the same canonical relations as in the case of Dirichlet problem. We consider p  Vk+ . (4.29) Wp = k=0

We must satisfy for 0 ≤ t ≤ tˆp the boundary conditions   ∂ ∗ + + α(x) (Vk+ + Vk−1 ) ∈ C ∞ , k = 1, · · · , p. i ∂ν

(4.30)

Here ν(x) is the unit normal to ∂Ω, pointing into the exterior of Ω, while α(x) ∈ C ∞ (∂Ω). By using the notations of Section 4.2, consider the term iωk

∂ (R+ (I − Mk−1 )i∗ Lk−1 + Lk−1 )i∗ R+ B. ∂ν k

(4.31)

Introduce local normal coordinates (y1 , y ), where y1 = dist(x, ∂Ω) and let ∂Ω have locally the form y1 = 0. Let q(y, τ, η) = η12 + m(x, η ) − τ 2 be the principal symbol of  in the local coordinates (y1 , y , η1 , η ), where m(y, η ) is homogeneous of order 2 in η and m(y, η ) ≥ c0 |η |2 , c0 > 0. The derivative with respect to the normal vector ∂ . The roots ξ1± of the equation field in the new coordinates is transformed into ∂y 1 q(x, τ, ξ1 , ξ ) = 0 with respect to ξ1 become ξ1± = ∓ τ 2 − m(x, ξ ) = ∓ μ(x, τ, ξ ).

116

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Now let ϕ± (t, x, τ, ξ ) be the phase functions introduced in Section 4.2. The kernel ∂ Lk−1 becomes of Lk−1 has the form (4.12), and the principal symbol of iωk ∂ν ) − iξ1− (x, ϕt , ϕx |x1 =0 ) = i μ(x, ϕ− t , ϕx )|x1 =0 . Similarly, the principal symbol of iωk

∂ (R+ (I − Mk−1 )i∗ Lk−1 ) ∂ν k

− + − is equal to −i μ(x, ϕ− t , ϕx )|x1 =0 . On the other hand, on x1 = 0, we have ϕ = ϕ , hence the principal symbol of (4.31) vanishes. Choosing suitably the lower order symbols of Rk+ , we arrange (4.30). To find the leading singularity of σ(t) neat Tγ , we need only to know the principal symbol of Vm+γ in Ω◦ × Ω◦ , which differs from that in the case of Dirichlet boundary conditions by the absence of the factor (−1)mγ . For the analysis close to the boundary according to the Remark 4.3.2, we can obtain a O(α) term. Thus, repeating the argument of this section, we get the following. Theorem 4.3.3: Under the assumptions of Theorem 4.3.1, the distributions σ(t), related to the eigenvalues of Neumann and Robin boundary problems in Ω, near Tγ has the form σ(t) =

Tγ# π Re [exp(i βγ )(t − Tγ − i0)−1 ]|det(Pγ − I)|−1/2 + L1loc (R), (4.32) 2π 2

where δγ = σγ +

sign Δ 2

− 1 ∈ N depends only on γ.

In the special case when βγ = −1 or δγ = −1, the formulae (4.28) and (4.32) can be simplified. Indeed, notice that 2Re (−i(t − Tγ − i0)−1 ) = i(t − Tγ + i0)−1 − i(t − Tγ − i0)−1 = 2πδ(t − Tγ ). hence we have the following. Corollary 4.3.4: Under the assumptions of Theorems 4.3.1, let βγ = 2mγ − 1 (resp. δγ = −1 for Robin boundary problem ). Then for t near Tγ , we have σ(t) =

1 (−1)mγ Tγ# |det(Pγ − I)|−1/2 δ(t − Tγ ) + L1loc (R), 2

(4.33)

where for Robin problem the factor (−1)mγ is omitted. The analysis of the singularity of σ(t) can be applied if we study a periodic reflecting non-generated ray γ in an unbounded domain. Therefore, such a ray lies in a compact set and we may repeat the above argument. In the case when the unbounded domain is the exterior of a finite union of strictly convex disjoint obstacles in R3 , the Maslov index σγ is zero. In fact, we may construct a global phase functions

POISSON SUMMATION FORMULA FOR MANIFOLDS WITH BOUNDARY

117

(see [I4], [Bu1]) related to every segment lj of γ, hence σγ is zero along lj . Since σγ does not change after reflections, we get σγ = 0. Moreover, sign Δ = 0 (see [16]) and we may apply Corollary 4.3.4. Finally, notice that the classical Poisson summation formula has the form   e−ikt = 2π δ(t − 2πk), (4.34) k∈Z

k∈Z

where the equality is interpreted in the sense of distributions in D (R). This means that for every function ϕ ∈ C0∞ (R), we have   ϕ(k) ˆ = 2π ϕ(2πk). k∈Z

k∈Z 2

d 1 2 2 The Laplace–Beltrami operator − ds 2 on S has eigenvalues λk = k , k ∈ N, and all 1 # periodic geodesics on S have primitive period Tγ = 2π. Then (4.33) can be considd2 ered as a Poisson summation formula for − ds 2.

4.4 Notes The construction of the global parametrix in Section 4.1 follows the work of Guillemin and Melrose [GM1] (see also [Chl], [Ch2] and [HeZ] for similar constructions). The analysis of the principal symbol of FˆB in the interior of Ω is based on [GM1] and [DG]. The form of the principal symbol on the boundary was established in [HeZ] (see also [PoT] and [SaV] for similar investigations). Lemmas 4.2.1, 4.2.2 and Proposition 4.2.3 are due to [HeZ]. Theorems 4.3.1 and 4.3.3 have been proved in [GM1]. In Section 4.3 we present a more detailed proof concerning the leading term in (4.27) and (4.31). An application of these results is considered in [GM2], where a simple inverse problem for the ellipse is studied. For more fine inverse spectral results we refer to [PoT], [Z] and [HeZ]. For manifolds without boundary the singularity of σ(t) related to the set of periodic bicharacteristics has been examined in [Ch3], [C2] and [DG]. For the tools related to symplectic geometry, calculus with half-densities and global theory of Fourier integral operators the reader may find more details in [H3], [H4], [GS].

5

Poisson relation for the scattering kernel This chapter is devoted to the proof of a relation analogous to that obtained in Section 3.4. We introduce the scattering kernel s(t, θ, ω), which is also given by the Fourier transform of the scattering amplitude well known in the physical literature. In Section 5.2, the contributions of the rays incoming with direction ω are localized. The Poisson relation for the scattering kernel has the form sing supp s(t, θ, ω) ⊂ { − Tγ : γ ∈ L(ω,θ) (Ω)}. Here L(ω,θ) (Ω) is the set of all (ω, θ)-rays in Ω and Tγ is the sojourn time of γ. The above relation is established in Section 5.3 under the assumption that each (ω, θ)-ray is the projection of a uniquely extendible generalized bicharacteristic. The results for propagation of singularities in [MS2] are not sufficient to eliminate all contributions, which must be cancelled from physical point of view in order to obtain singularities related only to the sojourn times of (ω, θ)-rays. To overcome this difficulty, we apply an argument based on the (iλ)-solutions of the reduced wave equation (Δ + λ2 )u = 0.

5.1

Representation of the scattering kernel

Let K be a compact subset of Rn , n ≥ 2, with non-empty interior and C ∞ smooth boundary ∂K. Assume that Ω = Rn \ K is connected. Clearly, ∂Ω = ∂K. In this section we introduce and study the scattering kernel related to the scattering operator

Geometry of the Generalized Geodesic Flow and Inverse Spectral Problems, Second Edition. Vesselin M. Petkov and Luchezar N. Stoyanov. © 2017 John Wiley & Sons, Ltd. Published 2017 by John Wiley & Sons, Ltd.

POISSON RELATION FOR THE SCATTERING KERNEL

119

for the wave operator  = ∂t2 − Δx with Dirichlet boundary condition in the exterior of K. Fix ρ0 > 0 so that K ⊂ {x ∈ Rn : |x| ≤ ρ0 }. First we will treat the case when n is odd, and at the end of this section we will discuss the case n even. Consider the Dirichlet problem ⎧ 2 ⎨(∂t − Δx )u = 0 in R × Ω◦ , u = 0 on R × ∂Ω, (5.1) ⎩ u|t=0 = f1 , ut |t=0 = f2 . Denote by HD (Ω) the completion of the space C0∞ (Ω◦ ) with respect to the norm 

1/2 ∇x ϕ dx

ϕD =

2

Ω

and introduce the energy space H = HD (Ω) ⊕ L2 (Ω). There exists a unitary group U (t) = eitG of H with generator iG such that for f = (f1 , f2 ) ∈ H we have U (t)f = (u(t, ·), ut (t, ·)), u(t, x) being the solution of (5.1) in the sense of distributions. Here the operator G has form   0 1 G = −i ΔD 0 and domain D(G) = {(u, v) : u ∈ H01 (Ω) ∩ H 2 (Ω), v ∈ HD (Ω)} ⊂ H. Moreover, G is self-adjoint in H and ΔD is the Dirichlet Laplacian in L2 (Ω) with domain D(ΔD ) = H01 (Ω) ∩ H 2 (Ω). Similarly, define the space H0 (Rn ) as above, replacing Ω by Rn , and consider the energy space H0 = HD (Rn ) ⊕ L2 (Rn ). For the solution u0 (t, x) of the Cauchy problem for  in Rt × Rnx with initial date f = (f1 , f2 ) ∈ H0 we have U0 (t)f = (u0 (t, x), ∂t u0 (t, x)), where U0 (t) = eitG0 is a unitary group in H0 with generator iG0 , where   0 1 G0 = −i Δ 0 is self-adjoint in H0 , Δ is the Laplacian in L2 (Rn ) with domain D(Δ) = H 2 (Rn ) and D(G0 ) = {(u, v) : u ∈ H 2 (Rn ), v ∈ H 1 (Rn )} ⊂ H0 . The space H can be considered as a subspace of H0 , extending f ∈ H as 0 in K. Let J : H0 → H

120

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

be the orthogonal projection. Consider the wave operators W± f = lim U (t)JU0 (−t)f, f ∈ H0 . t→∓∞

These operators exist for each f ∈ H0 and, moreover, they are isometrics from H0 onto H0 (see [LP1]). The operator W+ (resp. W− ) is related to the evolution when the time t → +∞ (resp. t → −∞). The operators W± are complete, that is Image W+ = Image W− , and this makes possible to define the scattering operator S = (W+ )−1 ◦ W− as a unitary operator from H0 onto H0 . We refer to [LP1] for the existence of W± and the main properties of S. Notice that for all t ∈ R we have SU0 (t) = U0 (t)S. By using the Radon transform, we can construct an isometric isomorphism (see [LP1]) R : H0 −→ L2 (R × Sn−1 ) so that RU0 (t) = Tt R, where Tt is the translation operator in L2 (R × Sn−1 ) having the form Tt f (σ, ω) = f (σ − t, ω), f ∈ L2 (R × Sn−1 ). Therefore, S˜ = R ◦ S ◦ R−1 : L2 (Rt × Sn−1 ) −→ L2 (Rt × Sn−1 ) becomes a unitary operator commuting with the translations in t and S˜ − Id is a linear continuous map from C0∞ (Rt × Sn−1 ) into D (Rt × Sn−1 ). By the Schwartz theorem, the operator S˜ − Id has a kernel s(t − t , θ, ω) ∈ D (Rt × Sn−1 × Rt × Sn−1 ), where θ, ω ∈ Sn−1 . Since for fixed ω and θ, the kernel s depends only on t − t , we can consider the distribution s(t, θ, ω) ∈ D (R × Sn−1 × Sn−1 ), called the scattering kernel. To obtain a representation of s(t, θ, ω), introduce the solution w(t, x; ω) of the problem ⎧ 2 ⎨(∂t − Δx )w(t, x; ω) = 0 in R × Ω◦ , w = 0 on R × ∂Ω, (5.2) ⎩ w|t t, so for |x − y| < A we have  A+1 e−iλt w0 (t, x − y)dt

R0 (λ, x − y) =

0 2 (Rn ). and R0 (λ) admits an analytic continuation in C from L2comp (Rn ) to Hloc

POISSON RELATION FOR THE SCATTERING KERNEL

123

Definition 5.1.1: A function u(λ, x) is called (iλ)-outgoing if there exist R1 > 0 and f (x) ∈ L2comp (Rn ) such that u(λ, x) = R0 (λ)f (x) for |x| ≥ R1 . The Fourier transform of w+ will be (iλ)-outgoing solution of the reduced wave equation (Δ + λ2 )u = 0 (see [LP1] and [P5]) and it is sufficient to prove that the Fourier transform of w2 is (iλ)-outgoing. To see this, let χ(x) ∈ C0∞ (Rn ) be a function such that χ(x) = 0 for |x| ≤ ρ0 + 1, χ(x) = 1 for |x| ≥ ρ0 + 2. Therefore, (χw2 ) = 2∇χ, ∇w2  − (Δχ)w2 = h2 (t, x), and for each t ∈ R we have h(t, ·) = (0, h2 (t, ·)) ∈ H0 . Moreover, h(t, x) has a compact support with respect to x. Since χw2 is an outgoing solution of the wave equation vanishing for t < −ρ0 , we can write  χw2 (t, x; ω) =



t

−∞

(U0 (t − τ )h((τ, x))1 dτ =

∞

(U0 (σ)h(t − σ, x))1 dσ.

0

Here (g)1 denotes the first component of g = (g1 , g2 ) ∈ H0 . Let v2 (λ, x; ω) be the Fourier transform of χw2 (t, x; ω) with respect to t. For each ϕ(λ) ∈ S(R) we have

  −iλt ϕ(λ)dλ v2 (λ, x; ω), ϕ(λ) = χw2 (t, x; ω), e 

∞



∞

= −∞ 0 ∞



 (U0 (σ)(0, h2 (t − σ, x)))1 dσ

−iλσ

e

=

 (U0 (σ)(0, g2 (λ, x))1 dσ, ϕ(λ) ,

0

where g2 (λ, x) = Ft→λ h2 (t, x). Thus,  v2 (λ, x; ω) = e−iλσ (U0 (σ)(0, g2 (λ, x))1 dσ. On the other hand, −1

i(G0 − λ)



∞

=

e−iλσ U0 (σ)dσ, Im λ < 0

0

and (G0 − λ)−1 =

 e−iλ ϕ(λ)dλ dt



λR0 (λ) −iΔR0 (λ)

 −iR0 (λ) . λR0 (λ)

124

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Using the analytic continuation of R0 (λ), this yields i((G0 − λ)(0, g2 ))1 = R0 (λ)g2 , λ ∈ R. Consequently, v2 (λ, x; ω) is (iλ)-outgoing and the same is true for the Fourier transform of w2 (t, x; ω). This implies that vsc (λ, x; ω) is (iλ)-outgoing. 2 −1 is called outgoing Green funcThe kernel G+ iλ (x − y) of −R0 (λ) = (Δ + λ ) + tion. A precise formula for Giλ is given in [LP1] (see also Chapter 2 in [P5]). It is possible to find G+ iλ from the Fourier transform with respect to λ of the fundamental solution of the wave operator e0 (t, x − y). The following representation of e0 (t, x) is obtained in Section 6.2 in [Hl]. For a function ψ(x) ∈ C0∞ (Rn ) we have e0 (t, x), ψ(x) =



1 4π (n−1)/2

1 ∂ 2t ∂t

(n−3)/2

 tn−2

ψ(tω)dω, t > 0. Sn−1

Setting t = r = |x|, x = rω, we get G+ iλ (x), ψ(x) = −

1 4π (n−1)/2

 0

∞

 Sn−1

e−iλr



1 ∂ 2r ∂r

(n−3)/2 rn−2 ψ(rω)dr dω.

Since rn−1 dr dω = dx, after an integration by parts, we deduce    −iλr  e (−1)(n−3)/2 1 + (n−3)/2 ∂r . Giλ (x) = − (n−1)/2 r r 2(2π) Notice that this implies easily the following asymptotic  −iλr  (iλ)(n−3)/2 e−iλr e G+ , r → +∞. (x) = − + O iλ 2(2π)(n−1)/2 r(n−1)/2 r(n+1)/2

(5.7)

It is easy to see that a (iλ)-outgoing solution u(λ, x) satisfies for |x| → ∞ the condition   −iλr   ⎧ e x e−iλr ⎪ ⎪ , ⎨u(λ, x) = (n−1)/2 b λ, |x| + O (n+1)/2 r r (5.8)  −iλr  ⎪ e ∂u ⎪ ⎩ (λ, x) + iλu(λ, x) = O , ∂r r(n+1)/2 called (iλ)-outgoing Sommerfeld radiation condition (see [LP1]). Applying (5.8) to vsc (λ, x; ω), we get  −iλr  e−iλr e , x = rθ, r → ∞. ˜(λ, θ, ω) + O vsc (λ, x; ω) = (n−1)/2 a r r(n+1)/2

(5.9)

On the other hand, exploiting (5.8) and applying the Green formula, it is easy to see that    ∂G+ ∂vsc iλ (λ, y; ω) − (x − y)v G+ (x − y) (λ, y; ω) dSy . vsc (λ, x; ω) = sc iλ ∂ν ∂ν ∂Ω

POISSON RELATION FOR THE SCATTERING KERNEL

125

Multiplying both sides of this equality by eiλr r(n−1)/2 and setting x = rθ, r = |x|, we find a ˜(λ, θ, ω) = lim eiλr r(n−1)/2 vsc (λ, rθ; ω) r→∞    (iλ)(n−3)/2 iλx,θ ∂vsc (λ, x; ω) − iλν, θv e (λ, x; ω) dSx =− sc ∂ν 2(2π)(n−1)/2 ∂Ω    (iλ)(n−3)/2 iλx,θ ∂vsc iλx,θ−ω (λ, x; ω) − iλν, ωe e dSx . =− ∂ν 2(2π)(n−1)/2 ∂Ω Here we have used the equality   iλx,θ−ω e ν, θ + ωdSx = ∂Ω

K

∂ (eiy,θ−ω )dy = 0. ∂(θ + ω)

Thus, a(λ, θ, ω) = a ˜(λ, θ, ω) and the scattering amplitude can be considered as the asymptotic profile of the outgoing solution vsc (λ, x; ω) of the problem (5.6). The scattering amplitude determines uniquely the obstacle K (see for instance [LP1]). On the other hand, in the application it is possible to measure only the singularities of s(t, θ, ω) and their leading terms. Hence, in general, we cannot measure the Fourier transform of s(t, θ, ω), and for this reason it is more important to investigate the inverse scattering problems related to the singularities of s(t, θ, ω). The above analysis can be applied to other boundary problems for the wave equation in R × Ω. For example, to study the Neumann problem ⎧ 2 ◦ ⎪ ⎨(∂t − Δx )u = 0 in R × Ω , ∂ν u = 0 on R × ∂Ω, (5.10) ⎪ ⎩ u|t=0 = f1 , ut |t=0 = f2 , we introduce the energy space HN = H 1 (Ω) ⊕ L2 (Ω) and the unitary group UN (t) = eiGN t . The operator GN has the form   0 1 , GN = −i ΔN 0 where DN is the Laplace operator in L2 (Ω) with Neumann boundary condition with domain D(DN ) = {f ∈ H 2 (Ω) : ∂ν f |∂Ω = 0}. We can define the wave operators W± and the scattering operator S, and for the scattering kernel sN (t, θ, ω) related to Neumann problem we obtain the representation  ν, θ∂tn−2 wN (x, θ − σ, x; ω)dSx , (5.11) sN (σ, θ, ω) = Cn ∂Ω

126

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

where wN (t, x; ω) is the solution of the problem ⎧ 2 ⎨(∂t − Δx )wN (t, x; ω) = 0 in R × Ω◦ , ∂ w = 0 on R × ∂Ω, ⎩ ν N wN |t 0. Under (1) this condition, the outgoing resolvent has the form (5.13) with H n−2 (u) instead of 2

(2)

H n−2 (u). In particular for n = 3 one has 2

R0 (λ, x) =

5.2

eiλ|x| . 4π|x|

Location of the singularities of s(t, θ, ω )

In this section we begin by the analysis of the singularities of s(t, θ, ω). Let θ = ω be fixed. Recall that Lω,θ (Ω) denotes the set of all (ω, θ)-rays in Ω. As usual, π is the natural projection T ∗ (R × Ω) → Ω. We fix ρ0 > 0 so that Ω ⊂ {x ∈ Rn : |x| ≤ ρ0 }. Recall that γ is an (ω, θ)-ray if γ = π ◦ γ˜ , where γ˜ (t) = (t, x(t), ±1, ξ(t)) ∈ T ∗ (R × Ω) is a generalized bicharacteristic of  in Ω such that there exist real numbers t1 > t2 with (5.15) ξ(t) = −ω for t ≤ t1 , ξ(t) = −θ for t ≥ t2 . This means that the curve x(t) has direction ω for t ≤ t1 and direction θ for t ≥ t2 . We assume that the time t increases when we move along γ˜ (t). Denote by Tγ the sojourn time of γ introduced in Section 2.4. In the following we consider a fixed t0 such that / { − Tγ : γ ∈ Lω,θ (Ω)}. −t0 ∈ Take T > 0 with |t0 | < T and consider the set ΓT = {Tγ : |Tγ | ≤ T, γ ∈ Lω,θ (Ω)}. To check that this set is closed, take a sequence {γk } ⊂ Lω,θ (Ω) with Tγk → T0 . There exists a compact set M such that γk (t) ∈ M for |t| ≤ T + 3ρ0 and all k. Let

128

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

γ˜k be a generalized bicharacteristic of  in Ω such that γk = π ◦ γ˜k . There exist t1 < t2 such that |t1 | < T + 3ρ0 , i = 1, 2, and for each k (5.15) holds replacing ξ(t) by ξk (t). It follows by Lemma 1.2.6 that there exists a generalized bicharacteristic γ˜ of  such that π ◦ γ˜ is an (ω, θ)-ray with sojourn time T0 . Hence ΓT is closed. Choose 0 > 0 so that Tγ ∈ / [t0 − 0 , t0 + 0 ], γ ∈ Lω,θ (Ω).

(5.16)

Let ρ(t) ∈ C0∞ (R), ρ(t) = 1 for |t| ≤ 12 , ρ(t) = 0 for |t| ≥ 1. Set ρδ (t) = ρ(t/δ) for 0 < δ ≤ 0 /2 and consider the integral J(λ) = s(t, θ, ω), ρδ (t + t0 )e−iλt    n−2  ∂w (k) n−2−k (t, x; ω)dt dSx . eiλ(t−x,θ) ρδ (x, θ − t + t0 ) = ck (−iλ) ∂ν R ∂Ω k=0

(k)

k

Here w(t, x; ω) is the solution of (5.2), ck = const, c0 = Cn and ρδ = ddtρkδ . Our aim is to show that for sufficiently small δ, the integral J(λ) is rapidly decreasing with respect to λ. In the following we study the term with k = 0. The analysis of the other terms is completely analogous. Without loss of the generality, we may assume that ω = (0, . . . , 0, 1). Consider the hyperplane Z(τ ) = {x ∈ Rn : xn = τ }, where τ < −ρ0 will be fixed below. Let R+ τ = {t ∈ R : t > τ } and let ϕj (x ) ∈ C0∞ (Rn−1 ), x = (x1 , . . . , xn−1 ). Consider the problems ⎧ n vj = 0 in R+ ⎪ τ × Rx , ⎪ ⎪ ⎨ vj (τ, x) = ϕj (x )δ(τ − xn ), ⎪ ⎪ ∂vj ⎪ ⎩ (τ, x) = ϕj (x )δ  (t − xn ), ∂t

(5.17)

⎧ Wj = 0 in R × Ω◦ , ⎪ ⎪ ⎪ ⎪ ⎪ ⎨Wj = on R × ∂Ω, Wj (τ, x) = ϕj (x )δ(τ − xn ), ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ ∂Wj (τ, x) = ϕ (x )δ  (τ − x ). j n ∂t

(5.18)

POISSON RELATION FOR THE SCATTERING KERNEL

129

There exists a compact set F0 ⊂ Rn−1 such that if supp ϕj ∩ F0 = ∅, then the straight lines issued from (x , τ ), x ∈ supp ϕj , with direction ω do not meet ∂Ω. Hence for such j we obtain      ∂Wj   ∩ (t, x, 1 − θ| ) : |t| ≤ T + ρ + 1, x ∈ ∂Ω = ∅. WF T (∂Ω) 0 x ∂ν R×∂Ω (5.19) Covering ∂Ω by small open neighbourhoods ωk , we can apply Theorem 1.3.4 to the integrals over R × ωk . Thus, we obtain   ∂Wj dt dSx = O(|λ|−m ), ∀m ∈ N. (5.20) eiλ(t−x,θ) ρδ (x, θ − y + t0 ) ∂ν R ∂Ω Set

F0 = {x ∈ Rn : x ∈ F0 , xn = τ }

and denote by l(u0 ) the straight line passing through u0 ∈ F0 with direction ω. First, consider the case ¯ ⊂ ∂Ω, ∅ = l(u0 ) ∩ K that is l(u0 ) could meet ∂Ω only at points, where it is tangent to ∂Ω. Let γ0 (t) be a generalized bicharacteristic of  in Ω with Im (π ◦ γ0 ) = l(u0 ). Then γ0 is uniquely extendible in the sense of Definition 1.2.2. To prove this, assume that ∂Ω is locally given by ϕ(x) = 0 and Ω by ϕ(x) ≥ 0. If x ˆ ∈ l(u0 ) ∩ ∂Ω, the above-mentioned x) ≥ 0, ϕxn (ˆ x) = 0. Now we can apply the geometric assumption implies ϕxn ,xn (ˆ argument of the proof of Corollary 1.2.4 based on condition (c) to conclude that γ0 is uniquely extendible. Next, we use the sets Ct (μ) and the metric D(ρ, μ) introduced in Section 1.2. Set μu = (τ, u, 1 − ω). Then we have Ct (μu0 ) = γ0 (t). Applying the argument of the proof of Lemma 1.2.6 for fixed  > 0, we find a small neighbourhood O(u0 ) ⊂ F0 of u0 such that for |t| ≤ T + ρ0 + 1 and u ∈ O(u0 ) we have D(Ct (μu ), γ0 (t)) =

inf

D(ν, γ0 (t)) < .

ν∈Ct (μu )

Now, take ϕj with supp ϕj ⊂ O(u0 ) and consider the solution Wj of (5.18). The singularities of Wj are contained in the set {Ct (μu ) : u ∈ F0 ∩ supp ϕj }. By the above inequality with  small enough, we arrange (5.19) and hence (5.20) holds. Secondly, consider the case when l(u0 ) has common points with the interior of K. Choose x1 (u0 ) ∈ l(u0 ) such that the linear segment [u0 , x1 (u0 )] is the maximal one that has no common points with the interior of K. There are two possibilities: (i) l(u0 ) meets transversally ∂Ω at x1 (u0 ); (ii) l(u0 ) is tangent to ∂Ω at x1 (u0 ) and ω is an asymptotic direction for ∂Ω at x1 (u0 ).

130

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Notice that in the case (ii) for each neighbourhood V of x1 (u0 ), we have l(u0 ) ∩ V ∩ K ◦ = ∅. Set t1 (u0 ) = |u0 − x1 (u0 )|. As in Section 3.1, it is easy to write the solution vj of (5.17) as an oscillatory integral and to find that W F (vj ) is contained in the set of all points (t, x, ±σ, ∓σω) ∈ T ∗ (Rn=1 ) \ {0} such that σ > 0 and there exist ˆ ± sω. In the case (i) we modx ˆ ∈ Z(1), x ˆ ∈ supp ϕj , s ≥ 0 with t = τ ± s, x = x ify vj to v˜j on the intersection of the interior of K with a small neighbourhood of x1 (u0 ) so that v˜j = vj for t < T1 + , v˜j = 0 for t > t1 + 2. Here t1 = max{t1 (u) : u ∈ O(u0 )}, O(u0 ) and  > 0 are chosen sufficiently small and supp ϕj ⊂ O(u0 ). Thus, we preserve the condition ˜ vj = 0 in R+ τ × Ω.

(5.21)

In the case (ii) we repeat the same procedure, modifying vj in the interior of K so that v˜j = 0 for t > t1 + 2. For this choose z ∈ l(u0 ) ∩ K ◦ sufficiently close to x1 (u0 ) and modify vj in a small neighbourhood W ⊂ K ◦ of z so that (5.21) remains valid. Set hj = vj |R+τ ×∂Ω and notice that hj = 0 for t sufficiently close to τ . Extending hj as 0 for t < τ , consider the solution wj of the problem ⎧ ⎨wj = 0 in R × Ω◦ , w + hj = 0 on R × ∂Ω, (5.22) ⎩ j wj |t T1 with Cσ (u0 ) ∩ {(σ, x, 1, −θ) ∈ T ∗ (R × Ω) : ρ0 ≤ |x| ≤ τ1 + σ + 1} = ∅. Then there exists a generalized bicharacteristic γ of  issued from μu0 and passing for t = σ over some point y, |y| ≥ ρ0 with direction θ. This means that π ◦ γ is an (ω, θ)-ray. The assumption (U(ω,θ) ) implies that γ is uniquely extendible, hence Ct (u0 ) = γ(t). Let Tγ be the sojourn time of γ and let Im γ = {(t, x(t), 1, −ξ(t)) ∈ T ∗ (R × Ω) : |ξ(t)| = 1, t ≥ τ }, where for γ(t) ∈ H instead of ξ(t), we determine ξ(t + 0) and ξ(t − 0). Recall that H is the set of hyperbolic points introduced in Section 1.2. Introduce the numbers T2 = inf {σ : σ ≥ τ, ξ(t) = θ for t > σ}, T3 = inf {σ : σ ≥ τ, x(t) ∈ / ∂Ω for t > σ}. Clearly, T2 ≤ T3 . It is easy to see that t − x(t), θ = Tγ for T2 ≤ t ≤ T3 . For t = T3 this follows from the definition of Tγ and the parameterization of γ(t) by t. For T2 ≤ t ≤ T3 we use the equality x(T3 ) − x(t), θ = T3 − t. Taking into account (5.16), we obtain |x(t), θ − t + t0 | ≥ 0 , T2 ≤ t ≤ T3 . Now choose O(u0 ) small enough and assume supp ϕj ⊂ O(u0 ). Then for τ ≤ t ≤ T3 + 1, the singularities of wj will be contained in a small neighbourhood

136

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

of γ(t). This makes it possible to choose s < T2 sufficiently close to T2 and to arrange γ(s) ∈ / H, ξ(s) = θ,  |x, θ − t + t0 | ≥ 0 2 for t ≥ s and

(5.32) (5.33) 

(t, x) ∈ sing supp (ωj |R×∂Ω ) ∪ sing supp

∂wj ∂ν

   R×∂Ω .

It is necessary to satisfy (5.33) for s ≤ t ≤ T3 + 1 , where 1 > 0 is chosen so that the singularities of wj for t ≥ T3 + 1 lie in the interior of Ω. Moreover, if γ(t) is a glancing point, (5.32) yields ξ(s) = θ|Tx(s) (∂Ω)

(5.34)

because |ξ(s)| = |θ| = 1. For O(u0 ) small enough (5.32) and (5.34) imply x, 1 − θ)} = ∅. W Fb (wj ) ∩ {μ ∈ T˜∗ (R × Ω) : μ = (s,  Since W Fb (wj ) is closed, for small  > 0 we have  1 − θ), s ≤ t ≤ s + } = ∅. W Fb (wj ) ∩ {μ ∈ T˜∗ (R × Ω) : μ = (t, x,

(5.35)

Choose a function α2 (t) ∈ C0∞ (R) such that  1 for t ≤ s, α2 (t) = 0 for t ≥ s + . Write

  (λ) + Ij,δ (λ), Ij,δ (λ) = Ij,δ

  (λ) (resp. Ij,δ (λ)) is obtained from Ij,δ (λ) replacing wj by α2 wj (resp. where Ij,δ by (1 − α2 )(wj )). We may assume that the inequality (5.33) holds for (t, x) in some neighbourhood of    ∂ ∂ − ν, θ (1 − α2 )ωj |R×∂Ω . sing supp ∂ν ∂t

Then δ ≤ 0 /2 implies ρδ (x, θ − t + t0 ) = 0 for such (t, x) and  (λ) = O(|λ|−m ), ∀m ∈ N. Ij,δ  (λ), set Fj = (α2 wj ). It follows from (5.35) that To examine Ij,δ

 W Fb (Fj ) ∩ {μ ∈ T˜∗ (R × Ω) : μ = (t, x, 1, −θ)} = ∅.

POISSON RELATION FOR THE SCATTERING KERNEL

137

 Then for Ij,δ (λ), we can apply the argument from the proof of Proposition 5.3.1.  (λ) in the case B. Therefore, (5.31) holds for Ij,δ In both cases A and B we have chosen a neighbourhood O(u0 ) of each u0 ∈ F0 . (j) There exists a finite set {u0 : 1 ≤ j ≤ M } ⊂ F0 such that (j)

F0 ⊂ ∪M j=1 O(u0 ). (j)

Suppose that the points u0 , j ≤ N, N ≤ M , satisfy condition (i) or (ii) of (j) Section 5.2. Choose a partition of unity {ϕj (x )}∞ j=1 so that supp ϕj ⊂ O(u0 ),  j = 1, . . . , N, supp ϕj ∩ F0 = ∅ for j > M . Set w ˜=

N  j=1

(wj + v˜j ) +



Wj ,

j>N

where wj , Wj , v˜j are introduced in the previous section. Clearly ⎧ ◦ ˜ = 0 in R+ ⎪ τ ×Ω , ⎨w + w ˜ = 0 on Rτ × ∂Ω, ∂w ˜ ⎪ ⎩w| | = δ  (τ − xn ). ˜ t=τ = δ(τ − xn ), ∂t t=τ ˜ in J(λ). This implies w = w ˜ in R+ τ × Ω. Choosing τ < −T1 , we can replace w by w Finally, (5.20), (5.28) and (5.31) show that J(λ) is rapidly decreasing, and we con/ sing supp s(t, θ, ω). clude that −t0 ∈ Thus, we have proved the following. Theorem 5.3.2: Let θ = ω be fixed and let the condition (U(ω,θ) ) be fulfilled. Then sing supp s(t, θ, ω) ⊂ { − Tγ : γ ∈ L(ω,θ) (Ω)}.

(5.36)

The inclusion (5.36) is called the Poisson relation for the scattering kernel by analogy with the relation (3.29) in Chapter 3. The condition (U(ω,θ) ) has been used only for the analysis of Ct (u) in the case B. If Ct (u) contains many generalized bicharacteristics, scattering with different directions, a localization of Ct (u) might be done to eliminate the contributions related to the rays having outgoing directions η = θ.

5.4 Notes The representation (5.4) of s(t, θ, ω) was obtained in [Ma2] (see also [LP1] and Chapter 8 in [P5] for related results). The outgoing solutions of the reduced wave equation and the outgoing Green function are examined in more detail in [LP1]. The Poisson relation (5.36) for non-convex domains K has been studied in [Pl] under

138

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

some geometric restrictions concerning the rays incoming with directions ±ω. For several convex obstacles, (5.36) has been proved in [PS5] (see also [Nal], [Na2] and [NS] for partial results). Theorem 5.3.2 was obtained in [CPS] (see also [Me4] for similar result). For the description of the uniquely extendible bicharacteristics we may apply Corollary 1.2.4. For generic domains Ω the condition (U(ω,θ) ) is satisfied for all ω, θ ∈ Sn−1 . Moreover, according to the results in Chapter 11, (U(ω,θ) ) is satisfied for all (ω, θ) ∈ Sn−1 × Sn−1 \ R, where the residual set R has measure zero. For the scattering theory for the wave equation the reader may find more references in [LP1], [LP2], [P5], [Me4], [Me2], [PP].

6

Generic properties of reflecting rays In this chapter we establish several properties of periodic reflecting rays in bounded domains and scattering rays in domains with bounded complements. These properties will be used in Chapters 7 and 11 to investigate certain inverse spectral problems for generic domains. In Section 6.1 we prove a general theorem that provides existence of residual sets of smooth embeddings of a given submanifold of Rn satisfying certain conditions. As a consequence of it, some elementary generic properties are derived for the two kinds of reflecting rays under consideration. In particular, we show that Herman Weyl’s conjecture is true for generic bounded domains. The main result in Section 6.1, as well as the scheme of its proof, will be used several times in this book. Following this scheme, we prove in the present chapter that for generic domains the reflecting rays under consideration are ordinary and non-degenerate.

6.1

Generic properties of smooth embeddings

Let X be a smooth (n − 1)-dimensional submanifold of Rn , n ≥ 2. In this section we prove a general theorem establishing the existence of a residual set of smooth embeddings f of X into Rn satisfying some particular properties. This theorem will be applied several times in the present chapter. Theorem 6.1.1: Let n ≥ 2, s ≥ 2, p and q be natural numbers, let U be an open subset of (Rn )(s) and let X be a smooth (n − 1)-dimensional submanifold of Rn . Let H = (H1 , . . . , Hp ) : U −→ Rp Geometry of the Generalized Geodesic Flow and Inverse Spectral Problems, Second Edition. Vesselin M. Petkov and Luchezar N. Stoyanov. © 2017 John Wiley & Sons, Ltd. Published 2017 by John Wiley & Sons, Ltd.

140

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

be a smooth map such that for every y ∈ U and every i = 1, . . . , s there exists an integer ri = 1, . . . , p with (6.1) gradyi Hri (y) = 0, where y = (y1 , . . . , ys ). (A) Let p = 1 and let T1 be the set of those f ∈ C(X) for which the critical points x of H ◦ f s with f s (x) ∈ U form a discrete subset of X (s) . Then T1 contains a residual subset of C(X). (B) Let p be arbitrary and let L = (L1 , . . . , Lq ) : U −→ Rq be a smooth map such that dL(y) = 0 for any y ∈ U with L(y) = 0. Let T2 be the set of all f ∈ C(X) such that if x is a critical point of H ◦ f s with f s (x) ∈ U , then L(f s (x)) = 0. Then T2 contains a residual subset of C(X). Here we use the notation



gradyi Hri (y) =

∂Hri (j) ∂yi

n ∈ Rn .

(y) j=1

For the proof of the theorem we need some preparation. First, notice that it is enough to consider the case q = 1 in part (B). Indeed, assume that the assertion in (B) is true for q = 1. Let q > 1. For m = 1, . . . , q set Um = {y ∈ U : dLm (y) = 0}. Then U1 , . . . , Uq are open subsets of U and U = ∪qm=1 Um . Moreover, we have (m)

∩qm=1 T2 (m)

⊂ T2 ,

where T2 is the set of all f ∈ C(X) such that if x is a critical point of H ◦ f s (m) with f s (x) ∈ Um , then Lm (f s (x)) = 0. It follows from our assumption that T2 contains a residual subset of C(X) for every m = 1, . . . , q. Hence T2 has the same property. Thus, assume q = 1. We may assume in addition that for every i = 1, . . . , s there exists ri = 1, . . . , p such that (6.1) holds whenever y ∈ U . Indeed, for any r = (r1 , . . . , rs ), let Ur be the set of those y ∈ U such that (6.1) holds for i = 1, . . . , s. Then U = ∪Ur , where r runs over all s-tuples of the considered type (r) (r) (there are only finitely many of them), and therefore ∩r T2 ⊂ T2 , where T2 is the s s set of those f ∈ C(X) such that if x is a critical point of H ◦ f with f (x) ∈ Ur , (r) then L(f s (x)) = 0. Thus, if every T2 contains a residual subset of C(X), then T2 has the same property. From now on we assume that q = 1 and for every i = 1, . . . , s there exists ri = 1, . . . , p such that (6.1) holds whenever y ∈ U . We will consider (A) and (B) simultaneously.

GENERIC PROPERTIES OF REFLECTING RAYS

141

Let Js1 (X, Rn ) be the s-fold bundle of 1-jets, and let α and β be the corresponding source and target maps (cf. Section 1.1). Set M = (αs )−1 (X (s) ) ∩ (β s )−1 (U ).

(6.2)

Clearly, M is an open subset (and therefore a submanifold) of Js1 (X, Rn ). Let Σ1 be the set of those (6.3) σ = (j 1 f1 (x1 ), . . . , j 1 fs (xs )) ∈ M such that x = (x1 , . . . , xs ) is a critical point of the map H ◦ (f1 × · · · × fs ). For a given f ∈ C(X) set Af = {x ∈ X (s) : js1 f (x) ∈ Σ1 }. It then follows that T1 = {f ∈ C(X) : Af is a discrete subset of X (s) }.

(6.4)

To describe T2 in a similar way, consider the set Σ2 of those σ ∈ Σ1 of the form (6.3) such that L ◦ (f1 × · · · × fs ) = 0. Then T2 = {f ∈ C(X) : js1 f (X (s) ) ∩ Σ2 = ∅}.

(6.5)

The central point in the proof of Theorem 6.1.1 is in obtaining relevant information about the sets Σ1 and Σ2 . Our aim is to show that each of these sets can be covered by a countable family of smooth submanifolds of M of sufficiently large codimension. Lemma 6.1.2: When p = 1, Σ1 is a smooth submanifold of M with codim (Σ1 ) = (n − 1)s.

(6.6)

For every p ∈ N, there exists a finite or countable family {Wm } of smooth submanifolds of M with (6.7) codim (Wm ) = (n − 1)s + 1 for all m such that Σ2 ⊂ ∪m Wm .

(6.8)

Proof: Consider an arbitrary σ0 ∈ M . We will construct a chart on M defined on a neighbourhood of σ0 . There exist coordinate neighbourhoods V1 , . . . , Vs of elements of X such that Vi ∩ Vj = ∅ for i = j and σ0 ∈

s  i=1

J 1 (Vi , Rn ) ⊂ Js1 (X, Rn ).

142

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Set D=M∩

 s 

 J (Vi , R ) . 1

n

(6.9)

i=1

Clearly, D is an open neighbourhood of σ0 in M (see Section 1.1). Consider arbitrary charts ϕi : Vi −→ Rn−1 and define the chart ϕ : D −→ (Rn−1 )(s) × (Rn )(s) × Rn(n−1)s

(6.10)

ϕ(σ) = (u; v; a),

(6.11)

by where σ has the form (6.3) and u = (u1 , . . . , us ),

v = (v1 , . . . , vs ),

(t)

a = (aij )1≤i≤s,1≤j≤n−1,1≤t≤n ,

ui = ϕi (xi ), vi = fi (xi ),   (t) ∂ fi ◦ ϕ−1 i (t) aij = (ui ) (j) ∂ui

(6.12) (6.13) (6.14)

for i = 1, . . . , s, j = 1, . . . , n − 1 and t = 1, . . . , n. Here we use the notation (1)

(n)

(1)

(n−1)

fi = (fi , . . . , fi ), ui = (ui , . . . , ui (1)

) ∈ Rn−1 ,

(n)

vi = (vi , . . . , vi ) ∈ Rn . Let us mention that if F : U −→ R is a smooth function, then    n −1 ∂ F ◦ (f1 ◦ ϕ−1 ∂F (t) 1 ) × · · · × (fs ◦ ϕs ) (u) = (v)aij , (j) (t) ∂ui ∂y t=1 i (1)

(6.15)

(n)

where yi = (yi , . . . , yi ). Since each of the sets Σ1 and Σ2 can be covered by a countable number of coordinate neighbourhoods, it is sufficient to prove that if D is an arbitrary coordinate neighbourhood of the form described earlier, then for p = 1, ϕ(D ∩ Σ1 ) is a smooth submanifold of ϕ(D) of codimension (n − 1)s, while for arbitrary p ∈ N, ϕ(D ∩ Σ2 ) is contained in a smooth submanifold of ϕ(D) of codimension (n − 1)s + 1. We will write the elements ξ of ϕ(D) of the form ξ = (u; v; a) ∈ (Rn−1 )(s) × (Rn )(s) × Rn(n−1)s , where u, v and a are determined by (6.12)–(6.14). It follows from our assumptions that for every i = 1, . . . , s there exists ri = 1, . . . , p such that (6.1) holds for all y ∈ U . For i = 1, . . . , s and j = 1, . . . , n − 1 set bij (ξ) =

n ∂Hr

i

t) i=1 ∂yi

(t)

(v)aij .

(6.16)

GENERIC PROPERTIES OF REFLECTING RAYS

143

Define the maps R1 : ϕ(D) −→ R(n−1)s ,

R2 : ϕ(D) −→ R(n−1)s × R

by R1 (ξ) = (bij (ξ))1≤i≤s,1≤j≤n−1 ,

˜ R2 (ξ) = (R1 (ξ), L(ξ)),

˜ where L(ξ) = L(v) by definition. Note that for p = 1 we have ri = 1 for any i, so ϕ(D ∩ Σ1 ) = R1−1 (0).

(6.17)

For an arbitrary p ∈ N the definitions of Σ2 and R2 yield ϕ(D ∩ Σ2 ) ⊂ R2−1 (0).

(6.18)

We will now show that R2 is a submersion on R2−1 (0). Let ξ = (u; v; a) ∈ Assume that

R2−1 (0).

s n−1

˜ Bij grad bij (ξ) + C grad L(ξ) =0

(6.19)

i=1 j=1

for some constants C and Bij (1 ≤ i ≤ s, 1 ≤ j ≤ n − 1). Here we consider ˜ grad bij (ξ) and grad L(ξ) as vectors in (Rn−1 )(s) × (Rn )(s) × Rn(n−1)s . It follows from (6.11)–(6.14), D ⊂ M and ξ ∈ ϕ(D) that v ∈ U . Fix arbitrary i = 1, . . . , s and j = 1, . . . , n − 1. There exists t = 1, . . . , n such that ∂Hri (t)

∂yi

(v) = 0.

Then, according to (6.16), we get ∂bij (t) ∂aij

Moreover, deduce

(ξ) =

˜ ∂L (t) (ξ) ∂aij

∂Hri (t) ∂yi

∂bi j

(v) = 0,

(t) ∂aij

(ξ) = 0 for i = i, j = j. (t)

= 0. Considering the derivatives with respect to aij in (6.19), we

Bij

∂Hri (t)

(v) = 0,

∂yi

hence Bij = 0. The latter holds for all i = 1, . . . , s and j = 1, . . . , n − 1. Now (6.19) ˜ becomes C grad L(ξ) = 0. Since ξ ∈ R2−1 (0), we have R2 (ξ) = 0, so in particular

144

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

˜ L(v) = L(ξ) = 0. The assumption on L and v ∈ U now implies dL(v) = 0. Thus, C = 0, and therefore, R2 is a submersion at ξ. Hence R2 is a submersion on R2−1 (0), and therefore (see Section 1.1) R2−1 (0) is a smooth submanifold of ϕ(D) with codim (R2−1 (0)) = dim(R(n−1)s × R) = (n − 1)s + 1. Applying the above argument in the case p = 1, we derive that R1 is a submersion on R1−1 (0). Therefore, R1−1 (0) is a submanifold of ϕ(D) with codim (R1−1 (0)) = dim(Rn−1)s ) = (n − 1)s. Now the assertion follows from (6.17) and (6.18).



Proof of Theorem 6.1.1: (A) Let p = 1. According to the above lemma, Σ1 is a smooth submanifold of M with (6.6). Since M is open in Js1 (X, Rn ), Σ1 is a smooth submanifold of Js1 (X, Rn ) of the same codimension. Applying the Multijet Transversality Theorem (see Section 1.1), we derive that the set  1} T1 = {f ∈ C ∞ (X, Rn ) : js1 f Σ is residual in C ∞ (X, Rn ). Since C(X) is open in C ∞ (X, Rn ), the set T1 = T1 ∩ C(X) is residual in C(X).  1. Now to prove (A) it is enough to establish T1 ⊂ T1 . Let f ∈ T1 ; then js1 f Σ Since js1 f : X (s) −→ Js1 (X, Rn ) and dim(X (s) ) = (n − 1)s = codim (Σ1 ), it follows that Af is a discrete subset of Σ1 , that is f ∈ T1 . Thus, T1 ⊂ T1 , which concludes the proof of part (A). (B) It follows from Lemma 6.1.2 that there exists a finite or countable family {Wm } of smooth submanifolds of M (and therefore of Js1 (X, Rn )) satisfying (6.7) and (6.8). By the Multijet Transversality Theorem, for any m the set  m} Sm = {f ∈ C(X) : js1 f W is residual in C(X). It remains to check that ∩m Sm ⊂ T2 .

(6.20)

 m . By (6.16), we Let f ∈ ∩m Sm . Then for every m we have f ∈ Sm , so js1 f W have dim(X (s) ) = (n − 1)s < codim Wm ,

GENERIC PROPERTIES OF REFLECTING RAYS

145

so js1 f (X (s) ) ∩ Wm = ∅. This holds for all m, so by (6.8), js1 (X (s) ) ∩ Σ2 = ∅. This and (6.5) imply f ∈ T2 which proves (6.20). Hence, T2 contains a residual subset of  C(X).

6.2

Elementary generic properties of reflecting rays

In this section we apply Theorem 6.1.1 to obtain some properties of generic domains Ω in Rn concerning the behaviour of periodic reflecting rays and scattering rays in Ω. In particular, we establish that for generic Ω the following properties are satisfied: (a) The lengths of any two distinct primitive periodic reflecting rays are independent over the rationals; (b) For every x ∈ ∂Ω there exists at most one direction ξ ∈ Sn−1 (up to the symmetry with respect to the normal to ∂Ω at x) such that (x, ξ) generates a periodic reflecting ray in Ω. Similar properties are considered for scattering rays. Clearly in the general case neither (a) nor (b) are satisfied. A simple example is a disk Ω in the plane. Let us mention that property (b) is equivalent to the following: any two distinct primitive periodic reflecting rays in Ω have no common reflection point, and any primitive periodic reflecting ray passes only once through each of its reflection points. More precisely, the latter is true only for non-symmetric rays (see Section 2.1); for symmetric rays this property can be formulated in a similar way. We begin with a simple combinatorial classification of the periodic reflecting rays. Let k ≥ s ≥ 2 be natural numbers, and let α : {1, . . . , k} −→ {1, . . . , s}

(6.21)

α(i) = α(i + 1)

(6.22)

be a map with for all i = 1, . . . , k. Here for convenience we set α(m) = α(i) for m = i + pk, 1 ≤ i ≤ k, p being an integer. If {α(i), α(i + 1)} = {α(j), α(j + 1)}

(6.23)

holds for all 1 ≤ i < j ≤ k, we will say that α is a non-symmetric map. If k = 2m and there is i0 = 1, . . . , k such that (6.22) holds for i0 ≤ i < j ≤ i0 + m and α(i0 + m + j) = α(i0 + m − j),

j = 1, . . . , m − 1,

(6.24)

then we will say that α is a symmetric map. By an admissible map we mean a map (6.21) that is either a non-symmetric or a symmetric map. Next, assume that an admissible map of the form (6.21) is fixed. Consider the sets Ii = Ii (α) = {j : ∃t = 1, . . . , k with {i, j} = {α(t), α(t + 1)}}

(6.25)

146

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

for i = 1, . . . , s. Denote by Uα the set of all y = (y1 , . . . , ys ) ∈ (Rn )(s) such that yi ∈ / conv{yj : j ∈ Ii },

i = 1, . . . , s,

and for any i = 1, . . . , s, if q, j, r and t are distinct elements of Ii , then at least one of the triples yi , yq , yj and yi , yr , yt consists of non-collinear points. It is easily seen that Uα is open in (Rn )(s) and the function F = Fα : Uα −→ R, given by F (y) =

k

yα(i) − yα(i+1)  ,

(6.26)

(6.27)

i=1

is smooth. Notice that if γ is a periodic reflecting ray with reflection points y1 , . . . , ys (their ordering does not matter in this case), there exist k ≥ s and an admissible map (6.21) such that (6.28) yα(1) , . . . , yα(k) are the successive reflection points of γ. In such a case we will say that γ is of type α. Then we have y = (y1 , . . . , ys ) ∈ Uα and F (y) is equal to the length Tγ of γ. Moreover, for any i = 1, . . . , s the relation j ∈ Ii is equivalent to the fact that there exists a segment of γ connecting yi and yj . Clearly, the type α of a periodic reflecting ray γ is not uniquely determined. In what follows X will be an arbitrary fixed smooth (n − 1)-dimensional submanifold of Rn . The following proposition is a consequence of Proposition 2.1.3. Proposition 6.2.1: Let α be an admissible map of the form (6.21) and let x1 , . . . , xs be all distinct reflection points of a periodic reflecting ray γ of type α for X, that is xα(1) , . . . , xα(s) are the successive reflection points of γ. Then x = (x1 , . . . , xs ) is a critical point of the map (Fα )|X (s) . Proof: Define the maps g : Uα −→ (Rn )k ,

G : (Rn )k −→ R

by g(y) = (yα(1) , . . . , yα(s) ), and G(z1 , . . . , zk ) =

k i=1

zi − zi+1  .

GENERIC PROPERTIES OF REFLECTING RAYS

147

Then Fα = G ◦ g. Moreover, g(X (s) ) ⊂ X k . It follows from our assumptions and Proposition 2.1.3 that g(x) is a critical point of G|X k . Therefore, x is a critical point  of (Fα )|X (s) . In order to apply the above proposition we need another property of the map F = Fα . Lemma 6.2.2: Let α be an admissible map of the form (6.21). For every i = 1, . . . , s and every y ∈ Uα there exists j = 1, . . . , n such that ∂F (j)

∂yi

(y) = 0.

(6.29)

Proof: Fix arbitrary i = 1, . . . , s and y ∈ Uα . For any j = 1, . . . , n we have ∂F

y (j) − y (j)

∂yi

yi − yt 

(y) = a (j)

i

t∈Ii

t

,

(6.30)

where a = 1 for a non-symmetric map α and a = 2 if α is symmetric. Assume that the derivative (6.30) is 0 for all j. Then y (j) − y (j) t

i

t∈Ii

yi − yt 

= 0,

which can be re-written as yi =



at yt ,

t∈Ii

where at = Since

t∈Ii

1

. yi − yt  ( j∈Ii 1/ yi − yj )

at = 1, we get yi ∈ conv{yt : t ∈ Ii },

which is a contradiction with y ∈ Uα .



Denote by R the set of those f ∈ C(X) such that every two primitive periodic reflecting rays for f (X) have rationally independent lengths. Consider also the set A of those f ∈ C(X) such that for every y ∈ f (X) there exists at most one direction ξ ∈ Sn−1 (up to symmetry with respect to the normal to f (X) at y) such that (y, ξ) generates a periodic reflecting ray for f (X).

148

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Theorem 6.2.3: Each of the sets R and A contains a residual subset of C(X). Proof: Fix an arbitrary surjective non-symmetric map (6.21) and suppose that k > s. Without loss of generality we will assume that α(1) = 1,

|α−1 (1)| > 1.

Denote by Aα the set of those f ∈ C(X) such that there is no periodic reflecting ray of type α for f (X). We will show that Aα contains a residual subset of C(X). To do this we are going to apply Theorem 6.1.1(B) with U = Uα , p = 1 and H = Fα . Choose arbitrary distinct elements j1 , j2 of α−1 (1). Then q = α(j1 − 1),

j = α(j1 + 1),

r = α(j2 − 1),

t = α(j2 + 1)

are distinct elements of I1 = I1 (α). Set  yq − y1 yj − y1 yr − y 1 u u yt − y1 + (−1) , − (−1) Lu (y) = yq − y1  yj − y1  yr − y1  yt − y1  for u = 1, 2, y ∈ U = Uα , and define L : Uα −→ R2 by L(y) = (L1 (y), L2 (y)). We will check that if L(y) = 0 for some y ∈ Uα , then dL(y) = 0. Let y ∈ Uα be such that L(y) = 0. If ∂L1 ()

(y) = 0,

= 1, . . . , n,

∂ym

then a simple calculation implies that yq − y1 is collinear with the vector v=

y − y1 yr − y1 + t . yr − y1  yt − y1 

Notice that y ∈ U implies v = 0. Since L1 (y) = 0 and (yq − y1 )/ yq − y1  and (yj − y1 )/ yj − y1  are unit vectors, we deduce that yj − y1 is collinear with v, too. Therefore, y1 , yq , yj are collinear. Next, assume that ∂L2 ()

(y) = 0,

= 1, . . . , n.

∂ym

In the same way one obtains that y1 , yr , yt are collinear, which is a contradiction with y = Uα and the definition of Uα . Hence dL(y) = 0. Finally, note that if y1 , . . . , ys are the reflection points of a periodic reflecting ray of type α, then for y = (y1 , . . . , ys ) we have y ∈ Uα and L(y) = 0. Now applying Theorem 6.1.1(B) we obtain that Aα contains a residual subset of C(X).

GENERIC PROPERTIES OF REFLECTING RAYS

149

Next, we consider the case when α is a surjective symmetric map of the form (6.21). Let k = 2m and let y1 , . . . , ys be all distinct reflection points of a periodic reflecting ray γ for X such that (6.28) are the successive reflection points of γ (i.e. γ is of type α). Then y is a critical point of the map F|X (s) . Moreover, F (y) = 2

m

yα(i) − yα(i+1)  .

i=1

Assume that k > 2s − 2 and define Aα as in the non-symmetric case. Using a slight modification of the above argument, replacing F by G and applying Theorem 6.1.1(B) again, we deduce that Aα contains a residual subset of C(X). Hence the set A1 = ∩α Aα , where α runs over the surjective maps (6.21) with k > s for non-symmetric α and with k > 2s − 2 for symmetric α, contains a residual subset of C(X). Let Z = f (X) for some f ∈ A1 . Suppose that there exist two different primitive periodic reflecting rays γ and δ for Z which have a common reflection point. We may assume that z1 , . . . , zs are the successive reflection points of γ, u1 , . . . , ut those of δ and z1 = u1 . For k ∈ N set αk = id : {1, . . . , k} −→ {1, . . . , k} and Fk = Fαk : Uk = Uαk −→ R. Let zi = f (xi ), uj = f (yj ) for i = 1, . . . , s and j = 1, . . . , t. Then for x = (x1 , . . . , xs ) and y = (y1 , . . . , yt ) we have z = f s (x) ∈ Us , u = f t (y) ∈ Ut , x is a critical point of Fs ◦ f s and y is a critical point of Ft ◦ f t . Hence (x, y) ∈ Us × Ut and (x, y) is a critical point of the map H ◦ f s+t , where H : Us × Ut −→ R2

(6.31)

H(z, u) = (Fs (z), Ft (u)).

(6.32)

is defined by

The fact that z1 = u1 can be expressed by L(f s (x), f t (y)) = 0, where L : Us × Ut −→ Rn is given by L(z, u) = z1 − u1 . It is easily seen that U = Us × Ut , H and L satisfy the assumptions of Theorem 6.1.1(B). Consequently, there exists a residual subset A2 (s, t) of C(X) such that for any f ∈ A2 (s, t), we have L(f s (x), f t (y)) = 0 for any critical point

150

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

(x, y) of H with (f s (x), f t (y)) ∈ Us × Ut . Then A2 = ∩{A2 (s, t) : s, t ≥ 2} is also a residual subset of C(X). It follows from the definitions of A1 and A2 that if f ∈ A1 ∩ A2 , then any two different primitive periodic reflecting rays for f (X) have no common reflection point. Therefore, A1 ∩ A2 ⊂ A, so A contains a residual subset of C(X). Next, consider the subset R of C(X). Let Z = f (X) for some f ∈ A. Suppose that there exist two different primitive periodic reflecting rays γ and δ for Z such that pTγ = qTδ for some p, q ∈ N. Let z1 , . . . , zs and u1 , . . . , ut be the successive reflection points of γ and δ, respectively. Set xi = f −1 (zi ), yj = f −1 (uj ), x = (x1 , . . . , xs ),

y = (y1 , . . . , yt ).

Consider the map H given by (6.31) and (6.32). As above one shows that (x, y) is a critical point of H ◦ f s+t . Moreover, for the map K : Us × Ut −→ Rn , defined by K(z, u) = pFs (z) − qFt (u), we have K ◦ f s+t (x, y) = 0. It follows from Lemma 6.2.2 that dK(z, u) = 0 for all (z, u) ∈ Us × Ut . Now applying Theorem 6.1.1(B) we deduce that there exists a residual subset R(p, q, s, t) of C(X) such that if f ∈ R(p, q, s, t), then whenever (x, y) ∈ X s × X t , (f s (x), f t (y)) ∈ Us × Ut and (x, y) is a critical point of H ◦ f s+t , we have K(f s (x), f t (y)) = 0. This yields the inclusion ∩{R(p, q, s, t) : p, q, s, t ∈ N,

s, t ≥ 2} ∩ A ⊂ R,

which shows that R contains a residual subset of C(X). This completes the proof of  the theorem. Let us mention that the property (b) from the beginning of this section implies that the measure of the set of periodic points in the phase space of the billiard system related to Ω is zero. More precisely, let Σ be the set of those (x, ξ) ∈ X × Sn−1 such that there exists a periodic reflecting ray for X passing through x with direction ξ. If μ denotes the standard Lebesgue measure on X × Sn−1 , then for generic X we have μ(Σ) = 0. In Section 6.4 we will establish stronger results, namely we will show that for generic X in Rn there exist at most countably many periodic reflecting rays for X. However, the weaker property proved in this section is already sufficient to make an application. As a direct consequence of the result of Ivrii in [Ivl] concerning the asymptotic (0.4) for the counting function N (λ) related to the point spectrum of the Laplacian, and Theorem 6.2.3 we obtain the following. Corollary 6.2.4: For every bounded domain Ω in Rn , n ≥ 2, with smooth boundary X = ∂Ω there exists a residual subset T of C(X) such that for every f ∈ T the asymptotic (0.4) holds replacing Ω by Ωf .

GENERIC PROPERTIES OF REFLECTING RAYS

151

In other words the Herman Weyl conjecture (see the Preface) is true for generic domains in Rn . The second part of this section is devoted to some properties of generic compact domains K in Rn with smooth boundaries X = ∂K concerning reflecting (ω, θ)-rays in Ω = Rn \ K. These properties are analogous to the properties of periodic reflecting rays considered in the first part of this section, and we will use the same technique in their proofs. That is why we will omit some of the details in the following considerations. In what follows X will be a fixed compact smooth (n − 1)-dimensional submanifold of Rn , n ≥ 2, and ω and θ will be fixed unit vectors in Rn . It is convenient to introduce a notion slightly more general than that of an (ω, θ)-ray. Let γ be a curve of the form γ = ∪ki=0 i ⊂ Rn such that i = [xi , xi+1 ] are finite straight-line segments for i = 1, . . . , k − 1, xi ∈ X for all i, and 0 (resp. k ) is the infinite straight-line segment starting at x1 (resp. xk ) and having direction −ω (resp. θ). Then γ will be called an (ω, θ)-trajectory for X if for any i = 0, 1, . . . , k − 1 the segments i and i+1 satisfy the law of reflection at xi+1 with respect to X. The points x1 , . . . , xk will be called reflection points of γ and the sojourn time Tγ of γ is defined by (2.31). As for reflecting (ω, θ)-rays (see Section 2.4), we distinguish two types of (ω, θ)-trajectories–symmetric and non-symmetric ones. The definitions are the same. We should note that in general the segments of an (ω, θ)-trajectory for X may intersect transversally X. According to Definition 2.4.1 every reflecting (ω, θ)-ray is an (ω, θ)-trajectory but the converse is not true in general. Define the subsets B, P, S of C(X) as the sets of those f ∈ C(X) such that: (a) B: for every x ∈ f (X) there exists at most one direction ξ ∈ Sn−1 (up to the symmetry with respect to the normal ν(x) to f (X)) so that there exists an (ω, θ)-trajectory for f (X) passing through x with direction ξ; (b) P: there is no (ω, θ)-trajectory for f (X) having different parallel segments; (c) S: Tγ = Tδ for any two different (ω, θ)-trajectories for f (X). Before going on, let us make the following remark. Let T be a subset of C(X), and let {Uk } be a sequence of open subset of Rn such that ∪k Uk = Rn and X ⊂ Uk ∞ for any k. Then T is a residual subset of C(X) if and only if T ∩ Cemb (X, Uk ) ∞ is residual in Cemb (X, Uk ) for each k. This follows easily from the fact that ∞ Cemb (X, Uk ) is open in C(X) for all k and ∞ C(X) = ∪k Cemb (X, Uk ).

Theorem 6.2.5: Each of the sets B, P and S defined above is a residual subset of C(X).

152

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Proof: Fix an arbitrary open ball U0 with radius a in Rn such that X ⊂ U0 . For the sake of brevity, set Z1 = Zω ,

Z2 = Z−θ ,

π1 = πω ,

π2 = π−θ ,

where we use the notation Zξ and πξ from Section 2.4. According to the above remark, it is enough to establish that the intersection of each of the sets B, P and S with ∞ (X, U0 ) C(X, U0 ) = Cemb

is a residual subset of C(X, U0 ). We need a modification of the notion of an admissible map introduced in the present section. Let k ≥ s ≥ 2 be natural numbers, and let α be a map of the form (6.21) such that (6.22) holds for i = 1, . . . , k − 1. We will say that α is a weakly non-symmetric map if (6.22) holds for 1 ≤ i < j ≤ k − 1. If k = 2m + 1, α(m − i + 1) = α(m + i + 1) for i = 0, 1, . . . , m and (6.22) holds for 1 ≤ i < j ≤ m, we will say that α is a weakly symmetric map. A map of the form (6.21) that is either weakly non-symmetric or weakly symmetric will be called a weakly admissible map. Next, consider a fixed weakly admissible map (6.21) and set α(0) = 0,

α(k + 1) = s + 1.

Thus, we regard α as a map α : {0, 1, . . . , k, k + 1} −→ {0, 1, . . . , s, s + 1}. Define the sets Ii = Ii (α) by (6.25) for i = 1, . . . , s. In this proof we will also use the notation y0 = π1 (y1 ) and ys+1 = π2 (yα(k) ) for any y = (y1 , . . . , ys ) ∈ (Rn )(s) . (s) Set Uα∗ = Uα ∩ U0 , where Uα is as in the beginning of this section. Then Uα∗ is an (s) open subset of U0 and the function F ∗ = Fα∗ : Uα∗ −→ R, given by F ∗ (y) =

k

y(αi ) − yα(i+1)  ,

(6.33)

(6.34)

i=0

is smooth. If y1 , . . . , ys are the distinct reflection points of an (ω, θ)-trajectory γ for X such that (6.28) are the successive reflection points of γ, we will say that γ is of type α. In this case we have y = (y1 , . . . , ys ) ∈ Uα∗ and F ∗ (y) = Tγ . Moreover, y is a critical point of the map (F ∗ )|X s : X s −→ R.

GENERIC PROPERTIES OF REFLECTING RAYS

153

It is also clear that for every (ω, θ)-trajectory γ there exists a surjective weakly admissible map α such that γ is of type α. The following lemma can be proved in the same way as Lemma 6.2.2. We leave  the details to the reader. Lemma 6.2.6: For every i = 1, . . . , s and every y ∈ Uα∗ there exists j = 1, . . . , n ∂Fα∗ such that (j) (y) = 0. ∂yi

Next, assume that (6.21) is a surjective weakly non-symmetric map such that k > s. Denote by Bα the set of those f ∈ C(X, U0 ) such that there does not exist an (ω, θ)-trajectory of type α for f (X). To show that Bα contains a residual subset of C(X, U0 ) we will use again Theorem 6.1.1(B), this time with U = Uα∗ , p = 1 and H = F ∗. Since k > s, there exists i = 1, . . . , s such that α−1 (i) contains more than one element. Take two arbitrary distinct j1 , j2 ∈ α−1 (i). Then q = α(j1 − 1), j = α(j1 + 1), r = α(j2 − 1), t = α(j2 + 1) are distinct elements of Ii . Since {q, j} = {0, s + 1}, either q or j is not in {0, s + 1}. We may assume that q∈ / {0, s + 1}; otherwise we will simply change our notation by setting / {0, s + 1}. q = α(j1 + 1) and j = α(j1 − 1). Similarly, we may assume that r ∈ For u = 1, 2 and y ∈ Uα∗ set  yj − yi yq − yi yr − y i u u yt − yi + (−1) , − (−1) , Lu (y) = yq − yi  yj − yi  yr − yi  yt − yi  and define L : Uα∗ −→ R2 by L(y) = (L1 (y), L2 (y)). Now repeating the corresponding part in the proof of Theorem 6.2.3 we deduce that Bα contains a residual subset of C(X, U0 ). Next, suppose that θ = −ω and α is a surjective weakly symmetric map of the form (6.21) with k > 2s − 1. Let k = 2m + 1 and let y1 , . . . , ys be the distinct reflection points of an (ω, θ)-trajectory γ for X of type α, that is (6.28) are the successive reflection points of γ. Then one gets easily that y = (y1 , . . . , ym , ym+1 ) is a critical point of the map G∗|X (m+1) , where G∗ : (Rn )(m+1) −→ R is defined by G∗ (y) =

m

yα(i) − yα(i+1)  .

i=0

Define Bα as in the previous case. Now, using an argument similar to the above, replacing F ∗ by G∗ and applying again Theorem 6.1.1(B), we see that Bα contains a residual subset of C(X, U0 ).

154

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

In this way we have established that the set B1 = ∩α Bα , where α runs over the set of surjective weakly admissible maps (6.21) with k > s in the non-symmetric case and with k > 2s − 1 in the symmetric case, containing a residual subset of C(X, U0 ). Next, we have to prove that the set B2 of those f ∈ C(X, U0 ) such that any two distinct (ω, θ)-trajectories for f (X) have no common reflection points contains a residual subset of C(X, U0 ). This can be established using a modification of the corresponding argument in the proof of Theorem 6.2.3. We leave the details to the reader. Since B1 ∩ B2 ⊂ B, it follows that B contains a residual subset of C(X, U0 ). Next, we proceed with the set S. Let f ∈ B ∩ C(X, U0 ), and let γ and δ be distinct non-symmetric (ω, θ)-trajectories for Y = f (X) such that Tγ = Tδ . Let yi = f (xi ) be all distinct reflection points of γ and δ taken together. Since f ∈ B, we may assume that y1 , . . . , yr are the successive reflection points of γ for some r < s, while yr+1 , . . . , ys are those of δ. Consider the functions F ∗∗ , G∗∗ : U −→ R defined by F ∗∗ (z) = π1 (z1 ) − z1  +

r−1

zi − zi+1  + zr − π2 (zr ) ,

(6.35)

i=1

G∗∗ (z) = π1 (zr+1 ) − zr+1  +

s−1

zi − zi+1  + zs − π2 (zs ) .

(6.36)

i=r+1

/ [zi−1 , zi+1 ] for all i = 2, 3, . . . , Here U is the set of all z ∈ (Rn )(s) such that zi ∈ / [π1 (z1 ), z2 ], zr ∈ / [zr−1 , π2 (zr )], zr+1 ∈ / [π1 (zr+1 ), zr+2 ], s − 1 with i = r, z1 ∈ / [zs−1 , π2 (zs )]. Clearly, F ∗∗ (y) = G∗∗ (y). Applying Theorem 6.1.1(B) with zs ∈ H = (F ∗∗ , G∗∗ ) and L : U −→ R, defined by L(z) = F ∗∗ (z) − G∗∗ (z), we derive that the set S1 (r, s) = {f ∈ C(X, U0 ) : gradx (H ◦ f s )(x) = 0 ⇒ L(f s (x)) = 0} contains a residual subset of C(X, U0 ). Hence S1 = ∩r dim(X s+1 ). On the other hand, 1 1 f : X s+1 −→ Js+1 (X, Rn ), js+1

therefore 1 1  m. f (X s+1 ) ∩ Wm = ∅ ⇐⇒ js+1 f W js+1 (m)

It then follows from the Multijet Transversality Theorem (see Section 1.1) that Ts is a residual subset of C(X) for every m, and by (6.44), Ts contains a residual subset of C(X). Since ∩∞ s=2 Ts ∩ A ⊂ T and, according to Theorem 6.2.3, A contains a residual  subset of C(X), it follows that T has the same property. The next theorem can be proved applying the previous argument with some small modifications. We leave the details to the reader. Theorem 6.3.3: Let ω and θ be two fixed unit vectors in Rn , and let X be a compact smooth (n − 1)-dimensional submanifold of Rn . Let T (ω, θ) be the set of those f ∈ C(X) such that every reflecting (ω, θ)-ray for f (X) is ordinary. Then T (ω, θ) contains a residual subset of C(X).

6.4

Non-degeneracy of reflecting rays

In this section it is proved that for generic Ω in Rn all periodic reflecting rays are non-degenerate. It is also established that, given two fixed unit vectors ω and θ, for generic Ω with bounded complements, every reflecting (ω, θ)-ray γ in Ω is non-degenerate, that is det(Jγ ) = 0. In what follows X will be a fixed compact smooth (n − 1)-dimensional submanifold of Rn , n ≥ 2. Theorem 6.4.1: Let Λ be an arbitrary countable set of complex numbers and let TΛ be the set of those f ∈ C(X) such that every periodic reflecting ray γ for f (X) is ordinary and spec(Pγ ) does not contain elements of Λ. Then TΛ contains a residual subset of C(X). Since the definition of the Poincaré map involves second derivatives, there is no way to prove the above theorem applying somehow Theorem 6.1.1. However, as in Section 6.3, an appropriate modification of the scheme of proof of Theorem 6.1.1 will be useful again. We begin with some preliminary considerations.

GENERIC PROPERTIES OF REFLECTING RAYS

161

Let γ be an ordinary primitive non-symmetric reflecting ray for X with successive reflection points q1 , . . . , qm , qm=1 = q1 . We will assume that the points q1 , . . . , qm are all different. Recall the notation Πi , σi , λi , νi , Gi ψ˜i , ψi , si from Section 2.3. Then Pγ has the representation (2.26). For brevity set   I λj I (6.45) Aj = ψj I + λ j ψ j for j = 1, . . . , m. Since Pγ is a symplectic matrix, we have 0 ∈ / spec(Pγ ). Let 1/μ ∈ C be an eigenvalue of Pγ . Then by (2.26) we get     sm 0 1 I− Am · · · A1 0 = det μ 0 sm    μsm 0 . = det I − Am · · · A1 0 μsm 

Set

E = det I − Am · · · A1

 μsm 0

0 μsm

 ,

(t)

(6.46)

and denote by dij the elements of the matrix ψt , t = 1, . . . , m, i, j = 1, . . . , n − 1. (t) Then clearly E can be expressed as a polynomial of the elements dij . The terms in (1) E involving only products of elements dij (i, j = 1, . . . , n − 1) are contained in the determinant        μsm I λ2 I 0 I λm I ··· A1 D = det I − 0 I 0 I 0 μsm   

m μsm μ( i=2 λi )sm . = det A−1 1 − 0 μsm Since A−1 1 we find D = det

=

 I + λ 1 ψ1 −ψ1

 I + λ1 ψ1 − μsm

−λ1 I I

 ,



−μ( m i=2 λi )sm − λ1 I

−ψ1 I − μsm  

m I − μsm −μ( i=1 λi )sm . = det −ψ1 I − μsm (1) (1)

(1)

It is now clear that the product d11 d22 · · · dn−1n−1 has a non-zero coefficient m  μ λi det(sm ), i=1

162

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS (t)

where  = ±1. Consequently, E is a non-trivial polynomial of dij (1 ≤ t ≤ m, 1 ≤ i, j ≤ n − 1) with coefficients depending on μ, λi and σi . Consider the multiindices τ = ((i1 , j1 , t1 ), . . . , (i , j , t )),

(6.47)

consisting of triples (is , js , ts ) of integers such that 1 ≤ is , js ≤ n − 1,

1 ≤ ts ≤ m,

t1 ≥ t2 ≥ · · · ≥ t ≥ 1.

(6.48)

For i ≤ denote by pi (τ ) the number of those triples (is , js , ts ) in τ such that ts = ti . Set |τ | =



ti ,

(t ) (t )

(t )

dτ = di1 1j1 di2 2j2 · · · dij ,

i=1 τ

and define the function ∂ E by ∂τ E =

∂ |τ | E (t ) (t ) ∂di1 1j1 ∂di2 2j2

(t )

· · · ∂dij

.

It follows from our arguments above that cτ dτ , E= τ

where τ runs over the set of the multiindices (6.47) satisfying (6.48) such that |τ | ≤ m(n − 1) and pi (τ ) ≤ n − 1. Here cτ = cτ (μ, λ1 , . . . , λm , σ1 , . . . , σm ) are real coefficients. 2 (X, Rn ). Next, we will define an open subset M of the m-fold bundle of 2-jets Jm First, consider the open subset V = {j 2 f (x) ∈ J 2 (X, Rn ): rank(df (x)) = n − 1} of J 2 (X, Rn ). Denote by Um the set of those y = (y1 , . . . , ym ) ∈ (Rn )(m) such that for every i = 1, . . . , m, the point yi does not belong to the segment [yi , yi+1 ]. As before we set for convenience y0 = ym and ym+1 = y1 . We will also need the function F = Fm : Um −→ R, given by F (y) =

m i=1

yi − yi+1  .

GENERIC PROPERTIES OF REFLECTING RAYS

163

Finally, set M = (αm )−1 (X (m) ) ∩ (β m )−1 (Um ) ∩ V m , where α and β are the source and the target maps, respectively, defined in Section 1.1. An atlas for M can be described as in Sections 6.1 and 6.3. Namely, consider arbitrary coordinate neighbourhoods V1 , . . . , Vm of distinct elements of X such that Vi ∩ Vj = ∅ for i = j, and let ϕj : Vi −→ Rn−1 be arbitrary smooth charts. Set m 

D=M∩

J 2 (Vi , Rn ),

(6.49)

i=1

and define the chart ϕ : D −→ (Rn−1 )(s) × (Rn )(s) × Rs(n−1)n × Rs(n−1)(n−2)n/2 by ϕ(σ) = (u; v; a; b) for every σ = (j 2 f1 (x1 ), . . . , j 2 fm (xm )) ∈ D,

(6.50)

where u, v, a are defined by (6.12) (replacing s by m), (6.13) and (6.14), while (t)

b = (bij )1≤i≤m,1≤j,≤n−1,1≤t≤n

(6.51)

is given by ∂ 2 (fi ◦ ϕ−1 i ) (t)

(t)

bij =

(j)

()

∂ui ∂ui

(ui ).

(6.52)

As mentioned in Section 6.3, the vector (1)

(n)

Ni = (Ni , . . . , Ni ),

(6.53)

determined by (6.42), is orthogonal to fi (X) at the point fi (xi ). Let σ ∈ D have the form (6.50) and let qi = fi (xi ), i = 1, . . . , m. Now we define the numbers λi and the maps σi as in the case of a periodic reflecting ray for a given submanifold f (X). Namely, we first define λi by (2.14) and Ni by (6.53) and (6.42). Next, denote by Πi the hyperplane passing through fi (xi ) and orthogonal to −−−−−−−−−−→ fi (xi )fi+1 (xi+1 ) and by αi the hyperplane passing through fi (xi ) and orthogonal to

164

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Ni . Now the symmetry σi is defined as in Section 2.4. Till now we have only used the data j 1 f1 (x1 ), . . . , j 1 fm (xm ). Next, define the differential of the Gauss map Gi : αi −→ αi by means of j 2 fi (xi ) and determine ψi and ψ˜i by (2.16), (2.24) and (2.25). Finally, define the matrices Ai = Ai (σ) by (6.45) and the function E = E(σ) by (6.46), where μ is a fixed complex number. Clearly, E is completely determined by σ, therefore E can be viewed as a function E : D −→ R. Denote by Σ the set of all elements σ of M of the form (6.50) such that x = (x1 , . . . , xm ) is a critical point of Fm ◦ f m , f m (x) ∈ Um and E(σ) = 0. Lemma 6.4.2: Σ is contained in the union of a countable family of smooth submanifolds of M of codimension s(n − 1) + 1. Proof: Consider a coordinate neighbourhood D of the form (6.49) of an element of Σ, and let the chart ϕ on D be defined as earlier. To prove the lemma, it is enough to show that ϕ(D ∩ Σ) is contained in a finite union of smooth submanifolds of ϕ(D) of codimension s(n − 1) + 1. The elements ξ of ϕ(D) have the form ξ = (u; v; a; b), where u, v, a, b are given by (6.12) with s replaced by m, (6.13), (6.14), (6.51) and (6.52). Set G(ξ) = E(ϕ−1 (ξ)) and cij (ξ) =

n ∂F

(t) m (v)aij (t)

t=1

∂yi

for 1 ≤ i ≤ m, 1 ≤ j ≤ n − 1. Consider the multiindices δp = ((1, 1, 1), (2, 2, 1), . . . , (p, p, 1)). For any p = 0, 1, . . . , n − 2 denote by Mp the set of those ξ ∈ ϕ(D) such that cij (ξ) = 0 for all i = 1, . . . , m and j = 1, . . . , n − 1, and ∂ δp G(ξ) = 0,

∂ δp+1 G(ξ) = 0.

Here we set for convenience ∂ δ0 G = G. Given two multiindices τ and τ of the form (6.47) with (6.48), we will write τ < τ if |τ | < |τ | and τ contains all triples in τ . Denote by M the set of all multiindices τ of the form (6.47) with (6.48) such that |τ | ≤ m(n − 1), τ > δn−1 and τ contains

GENERIC PROPERTIES OF REFLECTING RAYS

165

exactly n − 1 triples (i, j, t) with t = 1. Let M2 be the set of all pairs (τ, τ ) ∈ M2 such that τ < τ and |τ | = |τ | + 1. Given (τ, τ ) ∈ M2 , set M (τ, τ ) = {ξ ∈ ϕ(D) : cij (ξ) = 0 ∀i = 1, . . . , m,

∀j = 1, . . . , n − 1, ∂ τ G(ξ) = 0 , ∂ τ G(ξ) = 0}. It then follows from above that ϕ(D ∩ Σ) ⊂ ∪n−2 p=0 Mp ∪ ∪(τ,τ )∈M2 M (τ, τ ).

Therefore, the lemma will be proved if we establish that each of the sets Mp and M (τ, τ ) is a smooth submanifold of ϕ(D) of codimension s(n − 1) + 1. Before going on let us make the following remark. Let A = (Aij ) and B = (Bij ) be n × n symmetric matrices such that U AV = B

(6.54)

for some invertible matrices U and V . Let ψ(B11 , B12 , . . . , Bnn ) be a smooth function of the elements Bij of the matrix B. If the elements of the matrices U and V do not depend on (Aij ) and (Bij ), then ∂ψ/∂Bij = 0 for all i, j = 1, . . . , n is equivalent to ∂ψ/∂Aij = 0 for all i, j = 1, . . . , n. Here ψ is considered as a function of (Aij ) replacing each Bij by the corresponding function of (Aij ) according to (6.54). For every k = 1, . . . , m we have ψk = σ1 · · · σk πk∗ Gk πk σk · · · σ1 , (k)

where the projection πk is defined as in Section 2.4. Let Gk = (gij )n−1 i,j=1 . It then follows from the above remark that if ∂ (k)

∂dij

(∂ τ G)(ξ) = 0

(6.55)

for some (i, j, k), then there exist i and j such that ∂ (k)

∂gi j

(∂ τ G)(ξ) = 0.

Notice that (k) gj

=−

n

(t)

(t)

bkj Nk ,

t=1 (t)

(t)

(t)

where bkj are given by (6.52) and Nk by (6.42). Thus, G(ξ) is a polynomial of bkj (1 ≤ k ≤ m, 1 ≤ j, ≤ n − 1, 1 ≤ t ≤ n). Therefore, if ∂ (k)

∂gj

(∂ τ G)(ξ) = 0

166

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

for some (k, j, ), then there exists t = 1, . . . , n such that ∂ (t) ∂bkj

(∂ τ G)(ξ) = 0.

(6.56)

Fix an arbitrary (τ, τ ) ∈ M2 and denote by Oτ the set of those ξ ∈ ϕ(D) such that ∂ τ G(ξ) = 0. Define the map L : Oτ −→ Rs(n−1)+1 by L(ξ) = ((cij (ξ))1≤i≤m,1≤j≤n−1 ; ∂ τ G(ξ)). Clearly, M (τ, τ ) = L−1 (0) ⊂ Oτ . We are going to show that L is a submersion on Oτ ; this will imply that M (τ, τ ) is a smooth submanifold of ϕ(D) of codimension s(n − 1) + 1. Let ξ ∈ Oτ and assume that m n−1

Cij grad(cij )(ξ) + A grad(∂ τ G)(ξ) = 0

(6.57)

i=1 j=1

for some constants Cij and A. Since ξ ∈ M (τ, τ ), there exists (i0 , j0 , k0 ) such that (6.55) holds with i = i0 , j = j0 and k = k0 . It then follows from our above reasoning that there exist i, j and t such that (6.56) holds with k = k0 . Since the functions cij (ξ) (t) do not depend on the variables bkj , we get A = 0. Next, fix arbitrary i and j and (t) consider the derivatives with respect to aij in (6.57). Since v ∈ Um , according to Lemma 6.2.2 and using the same idea as that in the proof of Lemma 6.1.2, we find Cij = 0. Therefore, L is a submersion at ξ. This shows that M (τ, τ ) is a smooth submanifold of ϕ(D) of codimension s(n − 1) + 1. Applying the same argument with a slight modification, we also see that Mp has the same property for any p = 0, 1, . . . , n − 2. This completes the proof  of the lemma. Proof of Theorem 6.4.1: Given a complex number μ and an integer m ≥ 2, set 2 f (X (m) ) ∩ Σ = ∅}. T (μ, m) = {f ∈ C(X) : jm

Then T (μ, m) contains a residual subset of C(X). This follows easily from Lemma 6.4.2, applying the Multijet Transversality Theorem and an argument similar to that in the proofs of Theorems 6.1.1 and 6.3.1. Let f ∈ T (μ, m). If x = (x1 , . . . , xm ) ∈ X (m) is a critical point of the map Fm ◦ m f with f m (x) ∈ Um , then we have E(σ) = 0, where σ is defined by (6.50) with

GENERIC PROPERTIES OF REFLECTING RAYS

167

f1 = · · · = fm = f . According to our considerations at the beginning of this section, this implies that for any ordinary primitive non-symmetric periodic reflecting ray γ for f (X), we have μ ∈ / spec(Pγ ). Set  = {z ∈ C : ∃k ∈ N with z k ∈ Λ}. Λ  is also countable. Let A and T be the subsets of C(X) from Since Λ is countable, Λ Theorems 6.2.3 and 6.3.1, respectively. It follows from the considerations above and the theorem just cited that T (μ, m) T = ∩μ∈Λ,m≥2 ˜ contains a residual subset of C(X). Moreover, the properties of the sets T (μ, m) mentioned earlier yield that for every f ∈ T and every ordinary non-symmetric periodic reflecting ray γ for f (X), spec(Pγ ) does not contain elements of Λ. In a similar way one constructs T ⊂ C(X) containing a residual subset of C(X) such that for every f ∈ T and every ordinary symmetric periodic reflecting ray γ for f (X) we have spec(Pγ ) ∩ Λ = ∅. To do this one can repeat most of the considerations in this section with minor modifications. Let us mention that if qi = f (xi ), i = 1, . . . , m, are the successive reflection points of a symmetric primitive periodic reflecting ray for f (X) with f ∈ A ∩ T , then we can always assume that the segment [q1 , q2 ] is orthogonal to f (X) at q1 . Moreover, f ∈ A ∩ T implies that γ is ordinary and the points q1 , . . . , qk with k = (m − 2)/2 are all different. Finally, x = (x1 , . . . , xk ) is a critical point of the function Gk ◦ f k , where Gk : Uk −→ R is given by Gk (y) =

k−1

yi − yi+1 ,

i=1

and Uk ⊂ (Rn )(k) is defined appropriately. After these remarks, one can define the function E in the same way as above and repeat the argument from the non-symmetric case. The necessary modifications are very easy and we leave them to the reader. Since T ∩ T ⊂ TΛ , it follows that TΛ contains a residual subset of C(X).  Combining Theorems 6.2.3, 6.3.1 and 6.4.1, we deduce that the set F = A ∩ T ∩ TQ/Z contains a residual subset of C(X). Here Q/Z = {z ∈ C : z k = 1 for some k ∈ N}, and, as before, X is a compact (n − 1)-dimensional submanifold of Rn , n ≥ 2.

168

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

The next theorem shows that for generic domains Ω in Rn there exist at most countably many periodic reflecting rays in Ω. Theorem 6.4.3: For every f ∈ F and every integer s ≥ 2 there exist only finitely many periodic reflecting rays for f (X) with exactly s reflection points. Proof: Fix arbitrary f ∈ F and s ≥ 2. Without loss of generality we may assume that f = id; otherwise we will replace X by f (X). Denote by Ks the set of all x = (x1 , . . . , xs ) ∈ X s such that x1 , . . . , xs are the successive reflection points of a periodic reflecting ray for X. We will prove that Ks is finite. Assume that Ks is infinite. It then follows by the compactness of X that there exists a sequence {(x1,m , . . . , xs,m )}∞ m=1 of different elements of Ks such that xi = limm→∞ xi,m exists for every i = 1, . . . , s. As before, we set for convenience xs+1,m = x1,m and xs+1 = x1 . Lemma 6.4.4: There exist i = j with xi = xj . Proof of Lemma 6.4.4: Set xi+1,m − xi,m xi+1,m − xi,m  , ai,m = s . xi+1,m − xi,m  j=1 xj+1,m − xj,m 

Then ei,m = 1 and si=1 ai,m = 1. Without loss of generality we may assume that there exist ei,m =

lim ei,m = ei ,

lim ai,m = ai .

m→∞

m→∞

Then ei = 1 for all i = 1, . . . , s and si=1 ai = 1. Assume that x1 = · · · = xs . Then using lim x1,m = lim x2,m = lim x3,m ,

m→∞

m→∞

m→∞

we find that e2 = e1 . In the same way we get ei+1 = ei for all i, so e1 = e2 = · · · = es . Then s

implies

s i=1

(xi+1,m − xi,m ) = 0

i=1

ai,m ei,m = 0. Letting m → ∞, gives si=1 ai ei = 0. However,   s s ai ei = ai e1 = e1 , i=1

i=1

so we must have e1 = 0, a contradiction. This proves the lemma.



GENERIC PROPERTIES OF REFLECTING RAYS

169

We now continue with the proof of Theorem 6.4.3. According to the above lemma, without loss of generality we may assume that x1 = x2 . There exists a uniquely determined sequence i1 = 1 < i2 < · · · < ik ≤ s,

ik+1 = s + 1

of integers such that for every j = 2, . . . , k, ij is the maximal integer i > ij−1 so that the points xij−1 , xij−1 +1 , . . . , xi are collinear. It then follows that the points xi1 , xi2 , . . . , xik are the successive reflection points of a periodic reflecting ray γ for X. Lemma 6.4.5: We have k = s and ij = j for every j = 1, . . . , s. Proof of Lemma 6.4.5: Suppose that i2 > 2; then i2 ≥ 3. Case 1. There exists i with 1 < i < i2 and xi = xi2 . In this case the segment [x1 , xi2 ] of γ is tangent to X at the point xi , which is a contradiction with id ∈ F ⊂ T (γ must be ordinary). Case 2. x2 = x3 = · · · = xi2 . Denote by θm the measure of the angle between the vector x3,m − x2,m and the tangent hyperplane to X at x2,m . Since lim x3,m = lim x2,m = x2 ,

m→∞

m→∞

we have limm→∞ θm = 0. On the other hand, θm coincides with the measure of the angle between the vector x1,m − x2,m and the tangent hyperplane to X at x2,m . Thus, the vector x1 − x2 = lim (x1,m − x2,m ) m→∞

must be tangent to X at x2 = xi2 . The latter implies that xi2 is contained in the segment [x1 , xi3 ], which is a contradiction with the choice of i2 . In this way we have shown that i2 = 2. Proceeding in the same way, we prove that i3 = 3, . . . , is = s. In particular, k = s.



It follows from the last two lemmas that xi = xi+1 for all i = 1, . . . , s. Moreover, it is clear that x1 , . . . , xs are the successive reflection points of a periodic reflecting ray γ for X. Let us remark that in general some of the reflection points of γ could coincide even though x1,m , . . . , xs,m are different for all m. For example,

170

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

γ could be a symmetric periodic reflecting ray with 1 + s/2 different reflection points. Denote by γm the periodic reflecting ray for X with reflection points x1,m , . . . , xs,m . Set ηi =

xi+1 − xi , xi+1 − xi 

ηi,m =

xi+1,m − xi,m . xi+1,m − xi,m 

For every i we can define the billiard ball map B in a neighbourhood of (xi , ηi ) in X × Sn−1 so that it takes values in a neighbourhood of (xi+1 , ηi+1 ). Then B s (x1 , η1 ) = (x1 , η1 ) and B s (x1,m , η1,m ) = (x1,m , η1,m ) for every m ≥ 1. On the other hand, it is easy to see that the linear map L = dB s (x1 , η1 ) is conjugate to / spec(L). This the linear Poincaré map Pγ . Now id ∈ F ⊂ TQ/Z implies that 1 ∈ is a contradiction with the fact that B s has an infinite sequence of fixed points (x1,m , η1,m ) approaching (x1 , η1 ).  Thus, the set Ks is finite which concludes the proof of the theorem. Next, we consider (ω, θ)-trajectories for X, where ω and θ are two fixed unit vectors in Rn . Theorem 6.4.6: Let T (ω, θ) be the set of those f ∈ C(X) such that every reflecting (ω, θ)-trajectory for f (X) is ordinary and det(dJγ ) = 0. Then T (ω, θ) contains a residual subset of C(X). Proof of Theorem 6.4.6: Fix an open ball U0 containing X and set Z1 = Zω ,

Z2 = Z−θ ,

π1 = πω ,

π2 = π−θ ,

where Zξ and πξ are defined in Section 2.4. According to the remark before Theorem 6.2.5, it is sufficient to establish that T (ω, θ) ∩ C(X, U0 ) contains a residual subset of C(X, U0 ). Let f ∈ B ∩ T (ω, θ) ∩ C(X, U0 ), where B and T (ω, θ) are the sets from Theorems 6.2.6 and 6.3.3, respectively. Let γ be a reflecting (ω, θ)-trajectory for f (X) with successive reflection points q1 , . . . , qk . Set uγ = q0 = π1 (q1 ) and qk+1 = π2 (qk ) and define Πi , λi , σi , ψi , etc., as in Section 2.4. Then for dJγ (uγ ), we have the representation (2.32). Next, consider the function E = det(dJγ )(uγ ). Clearly, E is a polynomial of the (t) elements d˜ij (1 ≤ i, j ≤ n − 1) of the matrices ψ˜t , t = 1, . . . , k. First, assume that γ is non-symmetric. Since f ∈ B ∩ T (ω, θ), γ is ordinary and the reflection points q1 , . . . , qk of γ are all different. We will now show that the coefficient in front of (1) (1) (1) (6.58) d˜11 d˜22 · · · d˜n−1n−1

GENERIC PROPERTIES OF REFLECTING RAYS

171

in E is non-zero. Let si be given by (2.24). Then we have      σ1 λ1 σ 1 σ k λk σ k σ 2 λ2 σ 2 ··· 0 σk 0 σ2 ψ˜1 σ1 σ1 + λ1 ψ˜1 σ1  

  k I λ1 I sk 0 I i=2 λi I . = 0 sk ψ1 I + λ 1 ψ1 0 I Therefore

   k    sk 0 u I λ1 I λi I I i=2 = sk ψ1 (u), pr2 0 I + λ ψ 0 sk ψ 0 I 1 1 1

where

  u pr2 = v. v

This clearly implies that the coefficient in front of the product (6.58) in E is 1. Thus, (t) E is a non-trivial polynomial of the variables d˜ij with coefficients depending on λi , σi (i = 1, . . . , k). Next, repeating most of the considerations in the proof of Theorem 6.4.1 and replacing F by the function F ∗ determined by (6.33) and (6.34), we prove that there exists a residual subset R of C(X) with R ⊂ B ∩ T (ω, θ) such that f ∈ R yields dJγ (uγ ) = 0 for any non-symmetric (ω, θ)-trajectory γ for f (X). To deal with the symmetric case, assume θ = −ω, and let again f ∈ B ∩ T (ω, θ) ∩ C(X, U0 ). Let q1 , . . . , qk be the successive reflection points of a symmetric (ω, θ)-trajectory γ for f (X). Then γ is ordinary, k = 2m + 1 and the points q1 , . . . , qm are different. Clearly qm+i+1 = qm−i+1 for i = 0, 1, . . . , m. We have      σm λm σ m σm−1 λm−1 σm−1 σ 1 λ1 σ 1 ··· M= 0 σ1 0 σm−1 ψ˜m σm σm + λm ψ˜m σm     σ 1 λ1 σ 1 σm−1 λm−1 σm−1 ··· × 0 σm−1 0 σ1  

  m−1 I λm I s−1 0 I m−1 i=1 λi I = ψ m I + λm ψm 0 s−1 0 I m−1  

 m−1 sm 0 I i=1 λi I . × 0 sm 0 I Therefore

   u pr2 M = s−1 m ψm sm (u), 0

172

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

(m) (m) (m) which shows that the coefficient in front of the product d˜11 d˜22 · · · d˜n−1n−1 in E is non-zero. Thus, E is a non-trivial polynomial, and we can apply the argument from the proof of Theorem 6.4.1, replacing F by the map ∗ −→ R, G∗ : Um

defined by G∗ (y) =y1 − π1 (y1 ) +

m−1

yi − yi+1 ,

i=1 ∗ where Um is an appropriately defined open subset of (Rn )(m) . In this way we prove that there exists a residual subset R of C(X) with R ⊂ B ∩ T (ω, θ) such that f ∈ R implies det(dJγ )(uγ ) = 0 for any symmetric (ω, θ)-trajectory γ for f (X). Finally, notice that R ∩ R ⊂ T (ω, θ). Since each of the sets R and R contains a residual subset of C(X), the same is true for T (ω, θ). This completes the proof of  the theorem.

Consider the subset F(ω, θ) = B ∩ T (ω, θ) ∩ T (ω, θ) of C(X), where B and T (ω, θ) are the sets from Theorems 6.2.6 and 6.3.3, respectively. It follows from Theorems 6.2.6, 6.3.3 and 6.4.6 that F(ω, θ) contains a residual subset of C(X). Applying the argument from the proof of Theorem 6.4.3, we get the following. Corollary 6.4.7: For every f ∈ F(ω, θ) and every integer m ≥ 1 there exist only finitely many (ω, θ)-trajectories for f (X) with m reflection points.

6.5 Notes Theorem 6.1.1 was proved in a slightly different form in [PS2] (see also [PSl]). The material in Section 6.2 is a modification of parts of [PS2], [S1] and [CPS], while that of Section 6.3 is taken from [PS4]. The case n = 2 of Theorem 6.4.1 was proved in [PS2] (see also [PSl]). In its present form this theorem as well as Theorem 6.4.6 was established in [PS3]. Finally, Theorem 6.4.3 and Corollary 6.4.7 are taken from [PS4]. Generic properties of reflecting rays were first considered by Lazutkin [L2], [Ll] who proved an analogue of the Kupka–Smale theorem for billiards in strictly convex planar domains. It should be noted that Theorem 6.4.1 can be considered as a first part of a Kupka–Smale theorem for general billiards.

7

Bumpy surfaces In this chapter we study some particular properties of generic compact submanifolds M of Rn with 1 ≤ dim(M ) < n. These concern the behaviour of the geodesic flow on M determined by the standard Riemannian metric on M inherited from the Euclidean structure of Rn . Combined with the generic properties concerning periodic reflecting rays, established in Chapter 6, these properties will be essentially used when we study the Poisson relation for generic domains in Rn in Chapter 8. Our aim in the present chapter is to prove the existence of a residual set of smooth embeddings F of M into Rn such that the standard Riemannian metric on M  = F (M ) is a bumpy metric, that is all closed geodesics on M  are non-degenerate. As a consequence we obtain the classical Bumpy Metric Theorem of Abraham–Klingenberg–Takens–Anosov: for every compact smooth manifold M there exists a residual set in the space of all smooth Riemannian metrics on M consisting of bumpy metrics.

7.1

Poincaré maps for closed geodesics

In this section we recall some standard facts from the theory of ordinary differential equations which will be used in subsequent sections. Let Δ = [0, a] ⊂ R for some a > 0, and let

X = (X (1) , . . . , X (k) ) : Δ × U −→ Rk

Geometry of the Generalized Geodesic Flow and Inverse Spectral Problems, Second Edition. Vesselin M. Petkov and Luchezar N. Stoyanov. © 2017 John Wiley & Sons, Ltd. Published 2017 by John Wiley & Sons, Ltd.

174

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

be a C 1 map, where U is an open neighbourhood of 0 in Rk . For u ∈ U close to 0 let x(t; u) be the solution to the differential equation

.

x(t; u) = X(t; x(t; u)).

(7.1)

.

Here x denotes the derivative with respect to t. We will assume that x(t; u) exists for all t ∈ Δ whenever u belongs to a small neighbourhood V of 0 in Rk with V ⊂ U . Define the map Pt : V −→ Rk by Pt (u) = x(t, u). The proof of the following proposition can be found in standard texts on ordinary differential equations (see e.g. [Pon]). Proposition 7.1.1: Under the above assumptions, the map Pt is differentiable at 0, and for any t ∈ Δ the matrix Pt = dPt (0) is a solution to the problem . P t = dx X(t; x(t; 0))·Pt , t ∈ Δ, (7.2) P0 = I, where

⎛

∂X (1) ⎜ ∂x 1 ⎜ ⎜ dx X = ⎜ · · · ⎜ ⎝ ∂X (k) ∂x1

··· ··· ···

⎞ ∂X (1) ∂xk ⎟ ⎟ ⎟ ··· ⎟, ⎟ ∂X (k) ⎠ ∂xk

and I is the identity matrix. Corollary 7.1.2: Under the assumptions of Proposition 7.1.1, assume also that Y : Δ × U −→ Rk is another C 1 map. Consider the differential equation

.

y(t; u) = Y (t; y(t; u)),

(7.3)

and assume that x(t; 0), t ∈ Δ, is a solution to both (7.1) and (7.3), and ∂Y ∂X (t; x(t; 0)) = (t; x(t; 0)) ∂xi ∂xi for all i = 1, . . . , k and all t ∈ Δ. Define the map Qt : V −→ Rk (possibly with a smaller neighbourhood V of 0 in Rk ) by Qt (u) = y(t; u). Then dPt (0) = dQt (0) for all t ∈ Δ. Proof: Since dx X(t; x(t; 0)) = dx Y (t; x(t; 0)) for all t ∈ Δ, it follows that dPt (0)  and dQt (0) are both solutions to (7.2). Thus, they must coincide.

BUMPY SURFACES

175

Let M be a smooth manifold and let π : T ∗ M −→ M be the natural projection on the cotangent bundle T ∗M of M . Let m = dim(M ) − 1, then dim(M ) = m + 1. We assume that m ≥ 1. Let ω be the canonical symplectic form on T ∗M (see e.g. [AbM]), and let g be a fixed smooth Riemannian metric on M . Define the energy function H = Hg : T ∗ M −→ R by H(q) =

1 q, q g , 2

where ·, · is the inner product on T ∗ M determined by the Riemannian metric g. The Hamiltonian vector field determined by H (and so by g) is the unique vector field X = Xg on T ∗ M such that ω(X, Y ) = dH ·Y for any smooth vector field Y on T ∗ M . The flow on T ∗ M determined by X is called the geodesic flow. A curve c on T ∗ M is an integral curve of X if and only if the curve π ◦ c in M is a geodesic with respect to the metric g. We refer the reader to Chapter 3 in [AbM] for the basic facts concerning Hamiltonian dynamics. Next, consider a closed integral curve c : [0, θ] −→ T ∗ M,

θ > 0,

(7.4)

.

of X, that is a curve with c(0) = c(θ) and c(t) = X(c(t)) for all t ∈ [0, θ]. Then θ is called the period of c. If c(t) = c(0) for all t ∈ (0, θ), we will say that θ is the minimal period of c and that c is primitive. Similar terminology will be used for the closed geodesic γ : [0, θ] −→ M, γ = π ◦ c, (7.5) on (M, g). To define the Poincaré map of c, set q = c(0), p = π(q) and consider a smooth . m-dimensional submanifold Σ∗ of M containing p such that c(0) = X(q)is transversal to the (2m + 1)-dimensional submanifold Σ = π −1 (Σ∗ ) of T ∗ M at q. For q  ∈ Σ close to q the integral curve of X passing through q  at t = 0, after time t close to θ, intersects Σ at some point q  ∈ Σ. In this way we obtain a map P = P (g) : Σ q  → q  ∈ Σ,

176

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

defined in a small neighbourhood of q in Σ. This map leaves the 2m-dimensional submanifold = {q  ∈ Σ : H(q  ) = H(q)} Σ induced by ω. In this way invariant and preserves the natural symplectic form on Σ P induces a local symplectic diffeomorphism q) −→ (Σ, q). P : (Σ, The linear map −→ T Σ P = P (g) = dP (g) (q) : Tq Σ q is called the linear Poincaré map of the integral curve c (or the closed geodesic γ). The reader may refer Chapters 7 and 8 in [AbM] for more details concerning the definition of the Poincaré map, as well as the proof of the fact that, up to conjugacy, it does not depend on the choice of the initial point q and the submanifold Σ∗ . The latter shows that the spectrum spec(P ) of P does not depend on the choice of q and Σ∗ . We will say that c (resp. γ) is non-degenerate as an integral curve (resp. closed geodesic) of period θ if 1 ∈ / spec(P ). If all closed geodesics on (M, g) are non-degenerate, then g is called a bumpy metric on M . In what follows we assume that g is a fixed smooth Riemannian metric on M and (7.1) is a fixed closed integral curve of X with minimal period θ > 0. Define γ by (7.2). There exist Fermi coordinates in a neighbourhood of (γ) in M (see e.g. Section 1.12 in [Kl]). This means that there exist an open neighbourhood U of (γ) in M and a local diffeomorphism r : V = (−α, θ + α) × Bα (0) −→ U ,

(7.6)

for some constant α > 0, where Bα (0) = {x ∈ Rm : x < α}, such that the following conditions are satisfied: (i) γ(t) = r(t, 0, . . . , 0) for every t ∈ [0, θ]; (ii) the 1-jet of g00 coincides with the 1-jet of the constant 1 at all points of γ0 = {(t, 0, . . . , 0) ∈ Rm+1 : 0 ≤ t ≤ θ}; (iii) g0i = 0 on γ0 for all i = 1, . . . , m. Here gij (i, j = 0, 1, . . . , m) are the components of the metric g with respect to the coordinates x0 , x1 , . . . , xm provided by r. To be more precise, xi : U −→ R are smooth functions such that r(x0 (ξ), x1 (ξ), . . . , xm (ξ)) = ξ,

ξ ∈ U.

BUMPY SURFACES

177

Since r is only a local diffeomorphism, xi provide coordinates only locally, that is every point in U has an open neighbourhood W ⊂ U such that the restriction of r to r−1 (W ) is a diffeomorphism between r−1 (W ) and W . In what follows this is already sufficient in order to treat the xi ’s in the same way as if they were coordinates on the whole of U . Let y0 , y1 , . . . , ym be the coordinates dual to x0 , x1 , . . . , xm ; then x0 , x1 , . . . , xm ; y0 ,

y 1 , . . . , ym

are (local) coordinates in T ∗ M in a neighbourhood of (c). With respect to these coordinates we have ω=

m

dxi ∧ dyi

i=0

and H(x0 , x1 , . . . , xm ; y0 , y1 , . . . , ym ) =

m 1

g (x , . . . , xm ) yi yj 2 i,j=0 ij 0

(see e.g. Chapter 3 in [AbM]). For convenience we will use the following abbreviations: x = (x0 ; x ),

x = (x1 , . . . , xm ),

y = (y0 ; y  ),

y  = (y1 , . . . , ym ).

For 0 ≤ t ≤ θ define Σ(t) = {(x; y) : x0 = t},

= {(x; y) : x = t, y = 1}. Σ(t) 0 0

Given t ≥ 0, let Pt : Σ(0) −→ Σ(t) be the map defined in a small neighbourhood of q = c(0) which assigns to each q  ∈ Σ(0) the first intersection point of the positive integral curve of X through q  with Σ(t). Next, consider a perturbation g˜ = g + g  of g with a small g  which is another smooth Riemannian metric on M and g  satisfies the following conditions:  is zero on γ0 ; (ii ) the 1-jet of g00  = 0 on γ0 for all i = 1, . . . , m. (iii ) g0i

= X can be written in the form X = X + X  , where X  is the Then X g˜ ∗ Hamiltonian vector field on T M (defined only locally near (c)) determined by

178

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

the Hamiltonian function H  (x; y) =

m 1  g (x) yi yj . 2 i,j=0 ij

as well, that is γ(t) is a Notice that c(t) is an integral curve for not only X but X geodesic on (M, g˜). This follows immediately from the conditions (ii ) and (iii ), writing down the corresponding Hamiltonian system of differential equations for an integral curve of X. Define the maps Pt : Σ(0) −→ Σ(t) instead of X. in the same way as Pt using the vector field X In a neighbourhood of (c) the vector fields X and X  have the form X=

m 

∂H i=0



X =

∂H ∂ ∂ · − ∂x · ∂y ∂yi ∂xi i i

m 

∂H  i=0

∂H  ∂ ∂ − · · ∂yi ∂xi ∂xi ∂yi

, .

Now define the time-dependent vector fields Xt and Xt on Σ(0) by Xt (ζ) = 

X =

m 

∂H



i=1

∂ ∂H ∂ (t; ζ) − (t; ζ) ∂yi ∂xi ∂xi ∂yi

i=1

∂ ∂H  ∂ (t; ζ) − (t; ζ) ∂yi ∂xi ∂xi ∂yi

m 

∂H 

,

for t ∈ [0, θ] and ζ = (x1 , . . . , xm ; y0 , y1 , . . . , ym ) ∈ R2m+1 close to 0. The corresponding local diffeomorphisms , P˜  : (Σ(0), c(0)) −→ (Σ(t), c(t)) P t t are determined in the same way as Pt and Pt , replacing X by Xt and X  by Xt . (ζ) is the first intersection point of More precisely, given ζ ∈ Σ(0) close to c(0), P t the positive integral curve of the time-dependent vector field Xt starting at ζ with Σ(t). The definition of P˜t is similar. Lemma 7.1.3: For every t ∈ [0, θ] we have (0) = dP (0), dP t t

dP˜t (0) = dPt (0).

BUMPY SURFACES

179

Proof: We will prove the first of the above equalities; the proof of the second is very similar. Notice that the zero component of X corresponding to the coordinate x0 has the form

∂H (x; y) = gi0 (x)yi . ∂y0 i=0 m

X (0) (x; y) = Therefore,

g ∂X (0) (c(t)) = 00 (t; 0) = 0 ∂xk ∂xk and ∂X (0) (c(t)) = gk0 (t; 0) = 0 ∂yk for every k = 0, 1, . . . , m. Here we used the fact that the metric g satisfies the conditions (i), (ii) and (iii). Similarly, for the (m + 1)st component X (m+1) of X corresponding to the coordinate y0 , we find X (m+1) (x; y) = −

m ∂H 1 ∂gij (x) (x; y) = − yi yj . ∂x0 2 i,j=0 ∂x0

Therefore, 1 ∂ 2 g00 ∂X (m+1) (c(t)) = − (t; 0) = 0, ∂xk 2 ∂xk ∂x0 and ∂g ∂X (m+1) (c(t)) = − k0 (t; 0) = 0 ∂yk ∂x0 for every k = 0, 1, . . . , m. Now the assertion follows from Corollary 7.1.2.



The definition of Xt implies that for every integral curve ξ(t) of Xt we have y0 (t) = const. The same is true for the integral curves of Xt , so (Σ(0)) ⊂ Σ(t), P t

P˜t (Σ(0)) ⊂ Σ(t)

(7.7)

for every t ∈ [0, θ]. Notice that similar inclusions are in general not satisfied for Pt and Pt . However, according to Lemma 7.1.3 and (7.7), we have ⊂ Tc(t) Σ(t), dPt (c(0))(Tc(0) Σ(0)) for every t ∈ [0, θ].

dPt (c(0))(Tc(0) Σ(0)) ⊂ Tc(t) Σ(t)

180

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Let Pt and Pt be the matrices of the restrictions of the linear maps dPt (c(0)) Here using the coordinates x1 , . . . , xm , and dPt (c(0)), respectively, on Tc(0) Σ(0). for all t with an open neighbourhood of 0 in R2m . Cory1 , . . . , ym , we identify Σ(t) are identified with R2m × R2m , so P and respondingly, the tangent spaces Tc(t) Σ(t) t  Pt are 2m × 2m symplectic matrices smoothly depending on t. In particular, Rt = Pt−1 Pt

(7.8)

is also a symplectic matrix smoothly depending on t. We are going to show that the matrix function Rt is the solution of a certain matrix differential equation. To do this we need the following simple fact. Lemma 7.1.4: Let Δ be an interval in R and let Qt , Qt , Yt , Yt (t ∈ Δ) be k × k real matrices, differentiable with respect to t in Δ, such that Qt is invertible for all t ∈ Δ. If

.

.

Qt = (Yt + Yt )Qt ,

Qt = Yt Qt ,

t ∈ Δ,

 then for St = Q−1 t Qt we have

.

 S t = (Q−1 t Yt Qt )St ,

t ∈ Δ.

Proof: It follows from Q−1 t Qt = I that

.

−1 · (Q−1 t ) Qt + Qt Qt = 0,

and therefore

.

−1 −1 · (Q−1 t ) = −Qt Qt Qt

for all t ∈ Δ. Then

.

.

.

−1 −1  −1   ·  −1 S t = (Q−1 t ) Qt + Qt Qt = −Qt Qt Qt Qt + Qt (Yt + Yt )Qt  −1  −1  −1  −1  = −Q−1 t Yt Qt + Qt Yt Qt + (Qt Yt Qt )Qt Qt = (Qt Yt Qt )St

for every t ∈ Δ.



(ξ) as a time-dependent vector field defined for Next, consider X t ξ = (x ; y  ) = (x1 , . . . , xm ; y1 , . . . , ym ) ∈ Σ(0). Then (ξ) = J dξ X t



d2x x H(t; ξ) d2x y H(t; ξ) d2x y H(t; ξ)

d2y y H(t; ξ)

,

BUMPY SURFACES

where

 J=

0 −I

I 0

181



is the canonical 2m × 2m symplectic matrix, and  2 m  2 m ∂ H ∂ H (t; ξ) , d2x y H(t; ξ) = (t; ξ) , d2x x H(t; ξ) = ∂xi ∂xj ∂xi ∂yj i,j=1 i,j=1 etc. In the same way one obtains

d2x x H  (t; ξ)  (ξ) = J dξ X t d2x y H  (t; ξ)

d2x y H  (t; ξ) d2y y H  (t; ξ)

 .

(7.9)

Now applying Proposition 7.1.1 and Lemma 7.1.3, we deduce that Pt and Pt are solutions of the following problems: . (0)P , t ∈ [0, θ], P t = dξ X t t P0 = I,  .  (0) + d X P t = dξ (X t ∈ [0, θ], t ξ t (0))Pt ,  P0 = I. Now it follows from Lemma 7.1.4 that the matrix function Rt determined by (7.8) is a solution of the problem .  (0)P )R , t ∈ [0, θ], Rt = (Pt−1 dξ X t t t (7.10) R0 = I. Next, observe that m  1 ∂gkj ∂H  = (x) yk yj , ∂xi 2 ∂xi j,k=0

∂H   = gij (x) yj . ∂yi j=0 m

Since x0 = t, y0 = 1 and x = y  = 0 on c(t), we get  1 ∂ 2 g00 ∂2H  (c(t)) = (t; 0), ∂xi ∂xj 2 ∂xi ∂xj

 ∂g0j ∂2H  (c(t)) = (t; 0), ∂xi ∂yj ∂xi

∂2H   (c(t)) = gij (t; 0). ∂yi ∂yj Consider the homogeneous polynomial m 

1 1   a (t)xi xj + bij (t)xi yj + cij (t)yi yj , Ht (x ; y ) = 2 ij 2 i,j=1

(7.11)

182

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

where aij (t) =

 ∂ 2 g00 (t; 0), ∂xi xj

bij (t) =

 ∂g0j (t; 0), ∂xi

cij (t) =

1  g (t; 0). 2 ij

(7.12)

(x ; y  ) is the sum of those terms in the Taylor series of the function H  Clearly, H t  |Σ(t) that involve second derivatives. It follows from (7.9) and the expression for the second derivatives of H  along  (0) = J ·D(t), where c(t) that dξ X t

 A(t) B(t) D(t) = , (7.13) B(t)T C(t) and A(t) = (aij (t)),

B(t) = (bij (t)),

C(t) = (cij (t)).

(7.14)

Clearly, J ·D(t) belongs to the Lie algebra sp(2m) of the symplectic Lie group Sp(2m) for all t (see e.g. [AbM]). Finally, from (7.10) and the last expression for  (0), we derive the following. dξ X t Proposition 7.1.5: The matrix function Rt = Pt−1 Pt is a solution of the problem . Rt = Pt−1 JD(t)Pt Rt , t ∈ [0, θ], (7.15) R0 = I, where D(t) is given by (7.13), (7.14) and (7.12).

7.2

Local perturbations of smooth surfaces

Let M be a smooth submanifold of Rn , n ≥ 3. As before, set dim(M ) = m + 1 and assume that 1 ≤ m ≤ n − 2, that is dim(M ) ≤ n − 1. On M we consider the standard Riemannian metric g inherited from the Euclidean structure of Rn . Set H = Hg and X = Xg (cf. the notation in Section 7.1). Given F ∈ C(M ), denote by gF the Riemannian metric on M such that F : (M, gF ) −→ (F (M ), g) is an isometry. In what follows we assume that (7.4) is a fixed integral curve of X with minimal period θ > 0 and (7.5) is a corresponding closed geodesic on M . Theorem 7.2.1: Let Λ be an arbitrary countable set of complex numbers. Under the above assumptions, there exists t0 ∈ (0, θ) such that for every neighbourhood U of 0

BUMPY SURFACES

183

in C ∞ (M, Rn ) and every neighbourhood W of γ(t0 ) in M there exists f ∈ U with supp (f ) ⊂ W such that F = id + f ∈ C(M ), γ is a geodesic on (M, gF ) and the spectrum of the Poincaré map related to γ with respect to the matrix gF does not contain elements of the set Λ. The proof of this theorem is rather lengthy and is broken into several lemmas. As in Section 7.1, consider a local diffeomorphism (7.6) satisfying the conditions (i), (ii) and (iii). We will use again the coordinates x0 , x1 , . . . , xm ; y0 , y1 , . . . , ym in a neighbourhood of (c) in T ∗ M . Notice that the components gij of the metric g have the form   ∂r ∂r (x), (x) , x ∈ V, i, j = 0, 1, . . . , m, gij (x) = ∂xi ∂xj where r is considered as a map r : V −→ Rn and ·, · is the standard inner product in Rn . Given an embedding F ∈ C(M ), we can write it in the form F = id + f for some f ∈ C ∞ (M, Rn ). The corresponding perturbed metric on M has the form g˜ = gF = g + g  for some smooth two-form g  . Clearly, in the special case under consideration, we have     ∂r ∂(f ◦ r) ∂r ∂(f ◦ r)  (x), (x) + (x), (x) gij (x) = ∂xi ∂xi ∂xj ∂xi   ∂(f ◦ r) ∂(f ◦ r) + (x), (x) (7.16) ∂xi ∂xj for all x ∈ V and all i, j = 0, 1, . . . , m. Next, we will use the symplectic matrices Pt , Pt and Rt and the homogeneous from Section 7.1. We will only consider smooth two-forms g  with polynomial H t supports in a small neighbourhood of γ(t) for some t ∈ (0, θ), so that the matrix functions D(t), defined by (7.12)–(7.14), have compact supports in (0, θ). Now the problem is to find a perturbation F = id + f with a small f such that if Rt is the solution of the problem (7.15) for the corresponding matrix function D(t), then the spectrum of the matrix Pθ = Pθ Rθ does not contain elements of the set Λ. Since γ is a closed curve, the vector ∂r/∂x0 is not constant along γ0 , so there exists t0 ∈ (0, θ) such that (∂ 2 r/∂ 2 x0 )(t; 0) = 0. Fix t0 with this property and an arbitrary small neighbourhood W of γ(t0 ) in M . There exist real numbers a and b with ⎧ ⎪ a < t0 < b < θ0 , ⎪ ⎪0 < ⎨ ∂2r (7.17) (t; 0) = 0 ∀t ∈ [a, b], ⎪ ∂ 2 x0 ⎪ ⎪ ⎩r(t; 0) ∈ W ∀t ∈ [a, b].

184

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Choose an arbitrary β with 0 < β < min{a, θ − b, α}, and consider an arbitrary smooth function ρ : Rm+1 −→ [0, 1] such that supp (ρ) ⊂ (0, θ) × (−β, β)m and ρ(x) = 1 for all x ∈ [a, b] × [−β/2, β/2]m . In what follows the numbers t0 , a, b, β and the function ρ with the above properties will be fixed. Define the map h : V −→ Rn by h(x) =

m 1

v (x )x x , 2 i,j=1 ij 0 i j

(7.18)

where vij : [0, θ] −→ Rn are smooth maps with vji = vij ,

supp (vij ) ⊂ [a, b]

(i, j = 1, . . . , m)

which will be constructed later. Set  ρ(r−1 (ξ))h(r−1 (ξ)), f (ξ) = 0, ξ ∈ M \ U.

ξ ∈ U,

(7.19)

(7.20)

Then f : M −→ Rn is a smooth map with supp (f ) ⊂ r([a, b] × [−β, β]m ). Clearly, supp (f ) ⊂ W, provided β is chosen sufficiently small. Moreover, if the maps vij ∈ C ∞ ([0, θ], Rn ) are sufficiently close to 0 in the C ∞ topology, then f ∈ U and F = id + f ∈ C(M ). Set g  = g − gF as above. Notice that for x close to γ0 we have f (r(x)) = h(x), so (7.16) implies     ∂h ∂h ∂r ∂r  (x), (x) + (x), (x) gij (x) = ∂xi ∂xj ∂xj ∂xi   ∂h ∂h + (x), (x) . (7.21) ∂xi ∂xj Using the form (7.18) of h, by a direct calculation one checks that g  satisfies the conditions (ii ) and (iii ) in Section 7.1; therefore, γ is a geodesic in (M, gF ). We will now show that choosing the maps vij with (7.19) in a special way, we obtain all perturbations g  of g of a certain kind.

BUMPY SURFACES

Lemma 7.2.2: Let aij , bij : [0, θ] −→ Rn be smooth maps such that  aij = aji , bij = bji , supp (aij ) ⊂ [a, b], supp (bij ) ⊂ [a, b],

185

(7.22)

for all i, j = 1, . . . , m. Then there exists a neighbourhood V of 0 in C ∞ ([0, θ], Rn ) such that if all aij , bij are in V, then there exists a smooth map h of the form (7.18) with (7.19) for which the map f , defined by (7.20), belongs to U, and for g  = g − gF , has the form F = id + f , the corresponding polynomial H t m 

1   a (t)xi xj + bij (t)xi yj . Ht (x ; y ) = (7.23) 2 ij i,j=1 Proof: We have to choose the maps vij so that  ∂ 2 g00 (t; 0) = aij (t), ∂xi ∂xj

∂g0 (t; 0) = bij (t), ∂xj

 gij (t; 0) = 0,

(7.24)

for all i, j = 1, . . . , n and all t ∈ [0, θ]. Let vij : [0, θ] −→ Rn be arbitrary smooth maps satisfying (7.19). Then for ∂h  i = 1, . . . , n and t ∈ [0, θ] we have ∂x (t; 0) = 0, so gij = 0 by (7.21). Then for i x ∈ V close to γ0 and any i, j ≥ 1 we find

∂vij ∂2h (x) = (x )x , ∂x0 ∂xi ∂x0 0 j j=1 n

∂vij ∂3h (x) = (x ). ∂x0 ∂xi ∂xj ∂x0 0

Next, differentiating (7.21) we get    ∂r ∂ 2 g00  (t; 0) = 2 (t; 0), vij (t) , ∂xi ∂xj ∂x0

 ∂g0i (t; 0) = ∂xj





∂r (t; 0), vij (t) ∂x0

for all i, j ≥ 1 and t ∈ [0, θ]. ∂r (t; 0). Then by (7.17), Set w(t) = ∂x 0  2   ∂ r    w(t)=  2 (t; 0) > 0 ∂ x0 whenever t ∈ [a, b]. Fix arbitrary i, j = 1, . . . , n. According to (7.24), we have to choose the maps vij in such a way that  w(t), vij (t) = bij (t) (7.25) . 2 w(t), v ij (t) = aij (t) / [a, b] and for all t ∈ [0, θ]. Define vij : [0, θ] −→ Rn by vij (t) = 0 for t ∈

.

bij (t) − 12 aij (t) w(t) vij (t) = bij (t)w(t) + w(t)2

186

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

for t ∈ [a, b]. It follows from above that vij are well-defined smooth maps with supp (vij ) ⊂ [a, b]. A straightforward verification shows that (7.25) and (7.19) hold. Moreover, if all aij and bij are taken in a small neighbourhood V of 0 in C ∞ ([0, θ], Rn ), then h is C ∞ close to 0 and the map f , defined by (7.20), belongs  to U. This proves the assertion. Our interest to Hamiltonians of the form (7.23) with cij = 0 for all i, j and symmetric matrices A = (aij ) and B = (bij ) leads naturally to the consideration of a special subset of sp(2m). Let a be the linear subspace of sp(2m) consisting of all matrices of the form  B 0 , A −B where A and B are symmetric m × m real matrices. The following lemma shows that a is sufficiently large for our aims. Lemma 7.2.3: Let P =

 X Z

Y T



be an arbitrary 2m × 2m real matrix. For every > 0 there exists N ∈ a such that N< and det(P − exp N ) = 0. Proof: Fix an arbitrary > 0. Take δ > 0 so small that if Ni ∈ a, Ni< δ for i = 1, 2 and exp N = (exp N1 )(exp N2 ), then N< . Denote by a the set of all 2m × 2m matrices of the form  0 0 , A 0 where A is a symmetric m × m real matrix. It is easy to see that a is a Lie subalgebra of sp(2m), a ⊂ a, and the commutator [a, a ] is contained in a . Therefore, for every N1 ∈ a the set a(N1 ) = {tN1 + X : t ∈ R,

X ∈ a }

is a Lie subalgebra of sp(2m). Hence if N1 ∈ a and N2 ∈ a are sufficiently close to 0 and exp N = (exp N1 )(exp N2 ), then N ∈ a(N1 ) and in particular N ∈ a. Next, consider matrices N1 , N2 of the form   −B 0 0 0 N1 = , N2 = , 0 B A 0 where A and B are symmetric m × m real matrices. Then Ni ∈ a and there exists δ  > 0 such that if A< δ  and B< δ  , then Ni< δ for i = 1, 2. Define N by exp N = (exp N1 )(exp N2 ). Then N ∈ a, and setting D = exp B, we obtain

BUMPY SURFACES

 exp N =

D−1 0

0 D



I 0 A I



 −1 D = DA

187

0 . D

Choose τ > 0 such that if D is a symmetric positive definite m × m matrix with D − I< τ , then D = exp B for some symmetric m × m matrix B with B< δ  . Assume that det(P − exp N ) = 0 (7.26) for every choice of the symmetric matrices A and B with A< δ  and B< δ  . Consider an arbitrary symmetric positive definite matrix D with D − I< τ . We can write D in the form D = E −1 D1 E, where E is an orthogonal matrix and ⎛ ⎞ 0 ··· 0 y1 ⎜ 0 ··· 0⎟ y2 ⎟ , |yi − 1| < τ, i = 1, . . . , m. D1 = ⎜ ⎝· · · · · · ··· · · ·⎠ 0 0 0 · · · ym Fix E, A with A< δ  and y2 , . . . , ym with |yi − 1| < τ for i = 2, . . . , m. Then (7.26) implies that 

Y X − E −1 D1−1 E =0 (7.27) det Z − E −1 D1 EA T − E −1 D1 E holds for every y1 ∈ R with |y1 − 1| < τ . The left-hand side of (7.27) is a rational function of y1 determined for all y1 = 0. Since it vanishes for infinitely many values of y1 , it must be zero for all y1 = 0. Therefore for fixed E, A, y2 , . . . , ym , (7.27) holds for all y1 = 0. Using the same argument, we get by induction that for fixed E and A, (7.27) holds for all y1 = 0, . . . , ym = 0. This is true for any choice of the orthogonal matrix E, hence  Y X − D−1 =0 (7.28) Z − DA T − D holds for every non-singular symmetric matrix D. Next, for any non-singular symmetric matrix D, we can apply an argument similar to the previous one to show that (7.28) holds for all symmetric matrices A. In this way we have established that (7.28) holds for all symmetric matrices A and D with det(D) = 0. On the other hand, it is well known from linear algebra that any square matrix Z can be written in the form Z = D0 A0 , where A0 and D0 are symmetric matrices and det(D0 ) = 0. Let A0 and D0 be such matrices. Set D = yD0 , A = y1 A0 for an arbitrary real y = 0. Then Z = DA and (7.28) implies   1 X − D−1 Y det(X − yD0−1 ) det T − D0 = det = 0. 0 T −D y Since each of the equalities det(X − yD0−1 ) = 0 and det(T − y1 D0 ) = 0 is satisfied  only for finitely many y ∈ R, we get a contradiction. This proves the lemma.

188

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

The next lemma concerns analytical dependence on parameters of solutions of a special kind of systems of ordinary differential equations. Lemma 7.2.4: Let L be a smooth map from [0, θ] into the space of k × k real matrices such that L(t)< c ψ(t) for every t ∈ [0, θ], where c ∈ (0, 1) is a constant and ψ : [0, θ] −→ [0, N ] for some N > 0 is an integrable function with  θ ψ(t) dt ≤ 1. 0

Let Y (t) ( ∈ R) be the k × k matrix function which is the solution of the problem: . Y  (t) = L(t)Y (t), t ∈ [0, θ] , (7.29) Y (0) = I. Then there exist smooth matrix functions Bp (t), p = 0, 1, . . . , such that Bp (t) ≤ cp ,

.

B p (t) ≤ N cp

and Y (t) =

(p = 0, 1, . . . ;

∞

t ∈ [0, θ]),

p Bp (t)

(7.30)

(7.31)

p=0

for all t ∈ [0, θ] and |e| < c1 . Proof: Set Z (p) (t, ) =

dp Y (t). d p 

Writing down the variational equation of (7.29), by induction we get  . (p) Z (t, ) = L(t)Z (p) (t, ) + pL(t)Z (p−1) (t, ), Z (p) (0, ) = 0, for all t ∈ [0, θ], ∈ R and p = 1, 2, . . . . Then for = 0 one finds

. (p)

Z

(t, 0) = pL(t)Z (p−1) (t, 0),

Z (p) (0, 0) = 0

for all t ∈ [0, θ], p = 1, 2, . . . . Clearly Z (0) (t, 0) = Y0 (t) = 1. Suppose that for some integer p > 0 we have Z (p−1) (t, 0) ≤ (p − 1)!cp−1

BUMPY SURFACES

for all t ∈ [0, θ]. It then follows from above that   t     (p) (p−1) p−1     L(s)Z (s, 0) ds ≤ p!c Z (t, 0) = p 0

t

189

L(s) ds ≤ p!cp .

0

Moreover, Z (0) (t, 0) = 0 and for p ≥ 1 we get

. (p)

Z

(t, 0) ≤ p L(t) Z (p−1) (t, 0) ≤ pcn(p − 1)!cp−1 = N p!cp .

Therefore, the maps Bp (t) =

1 (p) 1 dp Z (t, 0) = Y (t) p! p! d p  |=0

satisfy the conditions (7.30). Hence the series G(t, ) =

∞

p Bp (t)

p=0

is uniformly and absolutely convergent for t ∈ [0, θ], provided | | ≤ const < c1 . Then for satisfying the latter condition we have

.

G(t, ) =

∞

.

p B p (t) =

p=0

= L(t)

p=0 ∞

p=1

= L(t)

∞

p

∞

p!

.

Z(t, 0)

p−1

Z (p−1) (t, 0) (p − 1)! p−1 Bp−1 (t) = L(t)G(t, ).

p=1

Moreover, G(0, ) = B0 (0) = Y (0) = I. Therefore, G(t, ) coincides with the solution Y (t) of (7.29). This is true for every  with | | < 1/c, hence (7.31) is satisfied. We are now ready to prove the main result of this section. Proof of Theorem 7.2.1: We will use the map r, the coordinates xi , yi and the numbers a, b and t0 satisfying (7.17). Take an arbitrary μ > 0 such that c = μ max{ Pt 2 : t ∈ [0, θ]} < 1. In what follows the numbers a, b, t0 , μ and c will be fixed.

(7.32)

190

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Fix an arbitrary neighbourhood W of γ(t0 ) in M and an arbitrary neighbourhood U of 0 in C ∞ (M, Rn ). Given a neighbourhood O of 0 in C ∞ ([0, θ], a), denote by A the set of all N ∈ O with supp (N ) ⊂ [a, b]. We will consider A with the topology induced by O ⊂ C ∞ ([0, θ], a). Then, being an open subset of a closed linear subspace of C ∞ ([0, θ], a), A is a Baire space. For N ∈ A we have  (bij (t)) 0 N (t) = , (7.33) (aij (t)) −(bij (t)) where the functions aij (t) and bij (t) satisfy (7.22). Let V be a neighbourhood of 0 in C ∞ ([0, θ], Rn ) as in Lemma 7.2.2. Fix O in such a way that aij , bij ∈ V for all i, j = 1, . . . , m. Then for every N ∈ A, we construct h = hN and f = fN as in Lemma 7.2.2 and set FN = id + fN . Denote by RN (t) the smooth 2m × 2m matrix function which is the solution of the problem . RN (t) = Pt−1 N (t)Pt RN (t), t ∈ [0, θ], (7.34) RN (0) = I. For a given λ ∈ C set Aλ = {N ∈ A : det(Pθ RN (θ) − λ) = 0}. It is easy to see that Aλ is open in A; we leave this as an exercise for the reader. We will now show that Aλ is dense in A. It is sufficient to show that Aλ contains elements arbitrarily close to 0. Indeed, for N ∈ A we can apply this to the submanifold M  = FN (M ) to show that there exist elements of Aλ arbitrarily close to N . By Lemma 7.2.3 there exists A ∈ a such that A< μ and ) = 0. det(exp A − λPt0 Pθ−1 Pt−1 0 Fix a matrix A with these properties and take an arbitrary smooth function ϕ : R −→ [0, 1]  such that supp (ϕ) ⊂ [−1, 1] and R ϕ(t) dt = 1. For δ > 0 set  t − t0 1 . ϕδ (t) = ϕ δ δ  Then supp (ϕδ ) ⊂ [t0 − δ, t0 + δ], 0 ≤ ϕδ ≤ 1/δ and R ϕδ (t) dt = 1. Let Rδ (t) be / A for sufficiently small the solution of (7.34) for N (t) = ϕδ (t)A. (Notice that ϕδ A ∈ δ > 0.) Then for t > t0 we have APt0 ) = Pt−1 (exp A)Pt0 Rδ (t) → exp(Pt−1 0 0 as δ −→ 0. The choice of A now implies the existence of δ > 0 with   det Rδ (θ) − λPθ−1 = 0.

(7.35)

BUMPY SURFACES

191

Fix an arbitrary δ > 0 with this property and set L(t) = ϕδ (t) Pt−1 APt ,

t ∈ [0, θ].

Then L(t)= ϕδ (t) JPtT JAPt ≤ μ ϕδ (t) max Ps 2 = c ϕδ (t), s

which yields that the assumptions of Lemma 7.2.4 are satisfied for L(t), c, N = 1/δ and ψ = ϕδ . By the lemma, the solution Y (t) of (7.29) satisfies (7.31) and (7.30). In particular, χλ ( ) = det(Y (θ) − λPθ−1 ) (7.36) is an analytic function of ∈ R for | | < 1/c. Notice that for = 1 the solution of (7.29) coincides with the solution of (7.34) for N = ϕδ A. Hence Y1 (t) = Rδ (t) for every t, and (7.36) and (7.35) imply χλ (1) = det(Rδ (θ) − λPθ−1 ) = 0. On the other hand, c < 1 by (7.33), so 1 < 1/c. This shows that χλ ( ) is an analytic function for | | < 1/c which is not trivially zero. In particular, 0 is not a limit point of the set Eλ = { ∈ (0, 1/c) : χλ ( ) = 0}. Hence for all sufficiently small > 0 the map N , defined by N (t) = ϕ(t)A, belongs to A and χλ ( ) = 0, i.e. N ∈ Aλ . Since N → 0 as → 0, it follows that Aλ contains elements arbitrarily close to 0. In this way we have established that Aλ is open and dense in A for each complex number λ. Since Λ is countable, AΛ = ∩λ∈Λ Aλ is a residual subset of A. In particular, there exists N ∈ AΛ such that f = fN ∈ U  and F = FN satisfy the requirements of the theorem.

7.3

Non-degeneracy and transversality

Our aim in this section is to describe the notion of non-degeneracy of a closed geodesic by means of some transversality condition. This will allow us to apply the Transversality Theorem of Abraham in the next section and establish the existence of a residual set of embeddings inducing bumpy metrics on M . Throughout M will be a compact smooth (m + 1)-dimensional submanifold of Rn , 1 ≤ m ≤ n − 2. Let σ : S ∗ M −→ M

192

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

be the co-sphere bundle of M . That is, S ∗ M = ∪x∈M Sx∗ M, where for every x ∈ M , Sx∗ M is the space of straight lines through 0 in Tx∗ M endowed with the quotient topology, and σ(v) = x for every v ∈ Sx∗ M . Let p : T ∗ M −→ S ∗ M be the canonical projection. Consider the map H : T ∗ M × R+ × C(M ) −→ T ∗ M, which assigns to (X, t, F ) ∈ T ∗ M × R+ × C(M ) the shift of X after time t under the action of the geodesic flow on T ∗ M generated by the metric gF . There exists a unique map F : S ∗ M × R+ × C(M ) −→ S ∗ M such that F ◦ (p × id × id) = p ◦ H. In what follows we will use not only the topological structure of C(M ) but its structure of a Banach manifold as well. The latter is inherited naturally from the Banach space structure of C ∞ (M, Rn ). We will now describe it briefly. As a general reference on infinite dimensional manifolds the reader may consult [Lang]. Let 2 ≤ s ≤ ∞. Fix a finite number of charts ϕi : Br −→ Ui ,

i = 1, . . . , k,

where Ui are open subsets of M , Br is the open ball with centre 0 and radius r > 0 in Rm+1 and ∪ki=1 Ki = M , where Ki = ϕi (Br/2 ). For f ∈ C s (M, Rn ) define f s =

s

1 max sup Dj (f ◦ ϕ−1 i )(x) . 1≤i≤k x∈Ki j! j=0

Then  ·s is a norm in C s (M, Rn ) with respect to which the latter is a Banach space. This norm generates the Whitney C s topology (see Section 2.1 in [Hir]). Denote by Cs (M ) the subset of C s (M, Rn ) consisting of all C s embeddings of M into Rn . Then Cs (M ) is an open subset of C s (M, Rn ) (see e.g. Section 2.1 in [Hir]) and therefore has a natural structure of a Banach manifold with a model space C ∞ (M, Rn ). In what follows we consider Cs (M ) with this structure. As before, the space C∞ (M ) will be denoted by C(M ) for brevity. The following lemma is a consequence of Chapter 5 in [AbR], and in fact can be derived easily from some standard facts about smooth dependence of solutions to systems of differential equations on parameters. We omit the proof. Lemma 7.3.1: The maps F and H are smooth. The concept of non-degeneracy of a closed geodesic introduced in the previous section has a local character and at a first glance seems to be inconvenient when we try

BUMPY SURFACES

193

to perturb the whole manifold M in order to make all geodesics on M non-degenerate. However, this concept can be described in terms of some global characteristics of the corresponding geodesic flow. Given a fixed smooth Riemannian metric g on M , for t ∈ R and Y ∈ T ∗ M denote by ϕt (Y ) the shift of Y after time t under the action of the geodesic flow on T ∗ M determined by g. Let ψt be the projection of ϕt on S ∗ M , that is ψt : S ∗ M −→ S ∗ M such that ψt ◦ p = p ◦ ϕt for all t ∈ R. Let Y ∈ T ∗ M \ {0} generate a periodic trajectory with period θ > 0, that is ϕθ (Y ) = Y . Consider the linear map T ϕθ : TY (T ∗ M ) −→ TY (T ∗ M ). The following proposition is a special case of a general fact in the theory of dynamical systems (see e.g. Sections 7.1 and 8.1 in [AbM]). We omit the proof. Proposition 7.3.2: Under the above assumptions, let P be the linear Poincaré map related to the closed integral curve c = {ϕt (Y ) : t ∈ [0, θ]} determined by Y . Then spec(T ϕθ ) = {1} ∪ spec(P ), and the multiplicity of 1 in spec(T ϕθ ) is k + 2, where k is the multiplicity of 1 in spec(P ). According to the definition in Section 7.1, the curve c (and therefore the corresponding closed geodesic on M ) is non-degenerate as a curve of period θ if k = 0 in the above proposition. This is equivalent to the fact that 1 occurs with multiplicity exactly 2 in spec(T ϕθ ). Setting X = p(Y ), we see that c is non-degenerate if and only if 1 occurs with multiplicity exactly 1 in spec(T ψθ ). Denote by D the diagonal of S ∗ M × S ∗ M , that is D = {(X, X) : X ∈ S ∗ M }. The following lemma is the central point in this section. Lemma 7.3.3: Let (Y0 , θ, F0 ) ∈ (T ∗ M \ {0}) × R+ × C(M ) be such that c : [0, θ] −→ T ∗ M,

c(t) = H(Y0 , t, F0 ),

is a closed geodesic with respect to the metric gF0 of period θ, and let X0 = p(Y0 ).

194

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

(a) If c is non-degenerate as a curve of period θ, then the map F1 : S ∗ M × R+ −→ S ∗ M × S ∗ M, defined by F1 (X, t) = (X, F(X, t, F0 )), is transversal to D at (X0 , θ). (b) If θ is the minimal period of c, then the map F2 : R+ × C(M ) −→ S ∗ M × S ∗ M, defined by F2 (t, F ) = (X0 , F(X0 , t, F )), is transversal to D at (θ, F0 ). Proof: It is sufficient to consider the case F0 = id; the general case follows by replacing M by F0 (M ) and g by gF0 . So, assume F0 = id. Consider coordinates x0 , . . . , xm ; y0 , . . . , ym in a neighbourhood of (c) in T ∗ M by means of a map (7.6) as in Section 7.1 such that Y0 = (0, . . . , 0; 1, 0, . . . , 0). Using these coordinates, we will identify TY0 (T ∗ M ) with R2m+2 = (R × Rm ) × (R × Rm ). Notice that we can use x0 , . . . , xm ; y1 , . . . , ym as coordinates in a neighbourhood of p ◦ c in S ∗ M . (a) Let c be non-degenerate as an integral curve of period θ. Define H1 : T ∗ M × R+ −→ T ∗ M × T ∗ M by H1 (Y, t) = (Y, H(Y, t, id)). For s, t ∈ R+ and Y = (s, 0; 1, 0) we have H1 (Y, t) = (Y, (s + t, 0; 1, 0)). Therefore, (T H1 (Y0 , θ)) ⊃ (R × {0}) × (R × {0}).

(7.37)

Fix t = θ and consider Y ∈ T ∗ M of the form Y = (0, x; 0, y), where x, y ∈ R . Then Y ∈ Σ(0) (see the notation in Section 7.1) and H1 (Y, θ) = (Y, Y  ), where / spec(dPθ (0, 0)), the Y  = (0, x ; 0, y  ) ∈ Σ(θ) and (x , y  ) = Pθ (x, y). Since 1 ∈ map 2m

(x, y) → ((x, y), Pθ (x, y))

BUMPY SURFACES

195

of R2m into R2m is transversal to the diagonal at (0, 0). Letting D denote the diagonal in R2m+2 × R2m+2 = (TY0 (T ∗ M ))2 , we get D + (T H1 (Y0 , θ)) ⊃ ({0} × Rm )4 . Combining this with (7.37), we find D + (T H1 (Y0 , θ)) ⊃ (Rm+1 × ({0} × Rm ))2 , which proves part (a). (b) Let θ be the minimal period of c. Fix the numbers a, b, t0 with (7.17) and a smooth map ρ : Rm+1 −→ [0, 1] as in Section 7.2. Let ϕ : R −→ [0, 1] be a smooth function with  ϕ(t) dt = 1. supp (ϕ) ⊂ [−1, 1], R

For , δ > 0 set 1 ϕδ (t) = ϕ δ



t − t0 δ

,

χ(t) = ϕδ (t).

Later we will determine how small and δ should be. Clearly, if δ is sufficiently small, then supp (χ) ⊂ [a, b]. . ∂r It follows from (7.17) that for λ(t) = ∂x (t; 0) we have λ(t) = 0 whenever t ∈ 0 [a, b]. As in the proof of Lemma 7.2.2 we construct smooth maps e, d : [0, θ] −→ Rn with supports in [a, b] such that  . λ(t), e(t) = χ(t), λ(t), e(t) = 0, (7.38) . λ(t), d(t) = 0, 2 λ(t), e(t) = −χ(t) for all t ∈ [0, θ]. Namely, we set e(t) = χ(t) λ(t) +

. χ(t) . . 2 λ(t),

λ(t)

and d(t) =

χ(t)

.

2 λ(t)2

.

λ(t)

for all t ∈ [0, θ]. Next, define W = {ω = (u1 , . . . , um ; v1 , . . . , vm ) ∈ R2m :ω< 1},

196

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

and consider the map Ω : W −→ C(M ),

Ω(ω) = F = id + f,

where f is determined by (7.20) by means of the map h given by  

m

m



ui xi e(x0 ) + vi xi d(x0 ) h(x) = i=1

(7.39)

i=1

for x = (x0 , x1 , . . . , xm ) ∈ V . If e and d are sufficiently close to 0 in the C ∞ topology (and so χ is C ∞ close to 0), then Ω is well defined. In fact, Ω coincides with the restriction of a linear map R2m −→ C ∞ (M, Rn ). To prove part (b), it is sufficient to show that the map (t, F ) → F(X0 , t, F ) from R+ × C(M ) into S ∗ M is a submersion at (θ, id). Consider the map H2 : R+ × W −→ T ∗ M defined by H2 (t, w) = H(Y0 , t, Ω(w)). The assertion will be proved if we show that (T H2 (θ, 0)) = (R × Rm ) × ({0} × Rm ).

(7.40)

Let ω = 0; then H2 (t, 0) = (t, 0; 1, 0) and therefore (T H2 (θ, 0)) ⊃ (R × {0}) × ({0} × {0}).

(7.41)

Let ω = (u; v) ∈ W . Define h by (7.39) and set g˜ = gF , F = Ω(ω), g  = g˜ − g.   , where gij are determined by (7.21). In a small neighbourhood Then g˜ij = gij + gij ∗ of (c) in T M the Hamiltonian function corresponding to g˜ has the form m 1

˜ g˜ (x)yi yj , H(x; y) = 2 i,j=0 ij

where x = (x0 , . . . , xm ), y = (y0 , . . . , ym ). Therefore, the coordinate functions x ˜(t; ω) and y˜(t; ω) of H(Y0 , t, Ω(ω)) with t ∈ [0, θ] satisfy the Hamiltonian equations . ∂ H . ∂H x ˜k = (˜ x; y˜; ω), y˜k = − (˜ x; y˜; ω). (7.42) ∂yk ∂xk Set x(t) = x ˜(t; 0), y(t) = y˜(t; 0); then x(t) = (t, 0, . . . , 0) and y(t) = (1, 0, . . . , 0) are the coordinates of c(t).

BUMPY SURFACES

197

Next, consider x ˜(t; ω) and y˜(t; ω) as column vectors. For q = 1, . . . , m and t ∈ [0, θ] set     d d x ˜(t; ω) x ˜(t; ω) , ηq (t) = . ξq (t) = duq |ω=0 y˜(t; ω) dvq |ω=0 y˜(t; ω) Writing the variational equations of (7.42), we find ξq (0) = 0 and

.

t ∈ [0, θ],

ξ q (t) = S(t)ξq (t) + Rq (t), where

⎛

S1 (t)

S(t) = ⎝ S3 (t) 

S2 (t)

(7.43)

⎞ ⎠,

S4 (t)

 ∂2H ∂2H S1 (t) = (x(t); y(t); 0) , S2 (t) = (x(t); y(t); 0) , ∂yk ∂xi ∂yk ∂yi



 ∂2H ∂2H (x(t); y(t); 0) , S4 (t) = − (x(t); y(t); 0) , S3 (t) = − ∂xk ∂xi ∂xk ∂yi and

Rq (t) =

  T ∂2H ∂2H − (x(t); y(t); 0) ; − (x(t); y(t); 0) . ∂yk ∂uq ∂xk ∂uq k

k

Since y(t) = (1, 0, . . . , 0), it follows that

∂˜ gkj ∂˜ g ∂2H (x(t); y(t); 0) = (x(t); 0)yj (t) = k0 (x(t); 0). ∂yk ∂xi ∂x ∂x i i j=0 m

On the other hand, (7.21) and (7.39) imply     ∂r ∂h ∂r ∂h + + O( ω2 ). g˜k0 = gk0 + , , ∂x0 ∂xk ∂xk ∂x0 Differentiating the latter equality with respect to xi and evaluating at x0 = t, x1 = · · · = xm = 0, we find ∂g ∂˜ gk0 (x(t); 0) = k0 (t; 0). ∂xi ∂xi Therefore,  S1 (t) =

∂gk0 (t; 0) . ∂xi k,i

198

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

By similar calculations one obtains S2 (t) = (gki (t; 0))k,i ,   1 ∂ 2 g00 ∂gi0 S3 (t) = − (t; 0) , S4 (t) = − (t; 0) . 2 ∂xk ∂xi ∂xk k,i k,i Moreover, (7.38) yields ∂˜ g ∂2H (x(t); y(t); 0) = k0 (x(t); 0) = ∂yk ∂uq ∂uq



0, k = q, λ(t), e(t) = χ(t),

k = q,

and ∂ 2 g˜00 ∂2H (x(t); y(t); 0) = (x(t); 0) = 0 ∂xk ∂uq ∂xk ∂uq for k = 0, 1, . . . , m and q = 1, . . . , m. Thus, the column vector Rq (t) has the form Rq (t) = (0, . . . , 0, χ(t), 0, . . . , 0; 0, . . . , 0)T , where χ(t) is the qth component. Since g00 (t; 0) = 1, ∂ 2 g00 ∂g ∂g00 (t; 0) = (t; 0) = i0 (t; 0) = 0 ∂xi ∂x0 ∂xi ∂x0 for i = 0, 1, . . . , m and gi0 (t; 0) = 0 for i = 1, . . . , m, it follows that the 0th row of S(t) has the form (0, . . . , 0; 1, 0, . . . , 0), while the (m + 1)st one consists of zeros only. Combining this with (7.43), we obtain

.(0)

ξ q (t) = ξq(m+1) (t),

.(m+1)

ξq

(t) = 0,

t ∈ [0, θ].

(i)

(0)

Here, ξq is the ith component of the vector ξq . Consequently, we get ξq (t) = (m+1) (t) = 0 for all t ∈ [0, θ] and all q = 1, . . . , m. In particular, ξq ξq (θ) ∈ ({0} × Rm ) × ({0} × Rm ),

q = 1, . . . , m.

In the same way one gets similar inclusions for the vectors ηq (θ), q = 1, . . . , m. Next, notice that the subspace T H2 (θ, 0)({0} × T0 W ) is generated by the vectors ξ1 (θ), . . . , ξm (θ),

η1 (θ), . . . , ηm (θ)

(7.44)

(see the beginning of the proof of part (b)). Therefore, T H2 (θ, 0)({0} × T0 W ) ⊂ ({0} × Rm ) × ({0} × Rm ).

(7.45)

Now we will show that if δ and are sufficiently small, then (7.45) becomes an equality. In fact, it is sufficient to choose δ so small that the vectors (7.44) are linearly independent. To see that this is possible, consider the fundamental solution Z(t)of (7.43). That is, Z(t) is the (2m + 2) × (2m + 2) smooth matrix function with Z(0) = I

BUMPY SURFACES

199

.

and Z(t) = S(t)Z(t) for t ∈ R. Since S(t) ∈ sp(2m), we have Z(t) ∈ Sp(2m) for every t. Now (7.43) implies  t Z(s)−1 Rq (s) ds, t ∈ R. (7.46) ξq (t) = Z(t) 0

Let λ0 (t), . . . , λm (t), μ0 (t), . . . , μm (t) be the successive column vectors of Z(t)−1 . Then  θ ϕδ (s)λq (s) ds → λq (t0 ) 0

and 

θ

ϕδ (s)μq (s) ds → μq (t0 )

0

as δ → 0. Hence if δ is sufficiently small, the vectors λ 1 , . . . , λm ,

μ1 , . . . , μm ,

given by  λq =



θ

ϕδ (s)λq (s) ds, 0

θ

μq =

ϕδ (s)μq (s) ds, 0

are linearly independent. Fix such a δ > 0 and choose > 0 so small that Ω(ω) ∈ C(M ) for every ω ∈ W (see (7.38) and (7.39)). Using (7.46), we find  θ  θ Z(s)−1 Rq (s) ds = Z(θ) ϕδ (s)λq (s) ds = Z(θ)λq . ξq (θ) = Z(θ) 0

0

In the same way one gets ηq (θ) = Z(θ)μq . Thus, the vectors (7.44) are linearly independent, and therefore (7.45) becomes an equality with this choice of δ. This  and (7.41) imply (7.40), which concludes the proof of the lemma.

7.4

Global perturbations of smooth surfaces

Throughout we use the notation from the previous section. Our aim in this section is to prove the following Theorem 7.4.1: Let M be a smooth compact submanifold of Rn , n ≥ 3, with dim(M ) < n. There exists a residual subset B of C(M ) such that for every F ∈ B the standard metric on F (M ) is a bumpy metric. Proof: Given F ∈ C(M ), X ∈ S ∗ M and t ∈ R, set ψtF (X) = F(X, t, F ).

200

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

For 0 < a ≤ b denote by C(a, b) the set of those F ∈ C(M ) such that if c˜ = {ψtF (v) : 0 ≤ t ≤ ω} is a periodic trajectory of the flow ψtF of period ω ≤ b and having minimal period θ ≤ a, then c˜ is non-degenerate as a curve of period ω. Notice that C(a , b ) ⊂ C(a, b) if a ≤ a , b ≤ b and a ≤ b . Consider the set B = ∩∞ k=1 C(k, k). Then clearly for every F ∈ B the standard metric on F (M ) is a bumpy metric. Thus, the theorem will be proved if we show that C(k, k) is an open and dense subset of C(M ) for every k = 1, 2 . . . . Next, notice that for every F ∈ C(M ) there exist a neighbourhood U of F in C(M ) and α > 0 such that for every G ∈ U the flow ψtG has no periodic trajectories with period ≤ α. Indeed, suppose this is not true. Then there exist sequences {Fk } ⊂ C(M ), {tk } ⊂ R+ , {vk } ⊂ S ∗ M such that Fk → F , tk → 0 and ψtFkk vk = vk for every k ≥ 1. Due to the compactness of S ∗ M we may assume that vk → v ∈ S ∗ M . Fix an arbitrary t > 0. For each k write t = mk tk + sk with mk ∈ N and 0 ≤ sk < tk . Then sk → 0, so we have ψtFk vk = ψsFkk vk → v as k → ∞. On the other hand, clearly ψtFk vk → ψtF v. Thus, ψtF v = v for all t > 0, which is a contradiction with the well-known fact that the geodesic flow ψtF has no fixed points. Now let 0 < a ≤ b be fixed numbers. We will show that C(a, b) is open in / C(a, b) for all k ≥ 1. Then for every k ≥ 1 C(M ). Assume that Fk → F with Fk ∈ there exist vk ∈ S ∗ M , tk ∈ (0, a] and k ∈ N such that ψtFkk vk = vk , k tk ≤ b and spec(T ψ Fkktk (vk )) contains 1 with multiplicity at least 2. According to the above argument, there exist a neighbourhood U of F in C(M ) and α > 0 such that for every G ∈ U the flow ψtG has no periodic trajectories of period ≤ α. Without loss of generality we may assume that Fk ∈ U for all k and tk → t, vk → v ∈ S ∗ M as k → ∞. Now we see that the sequence {k } must be bounded, so we may assume F (v)) contains k =  for all k. It then follows that ψtF (v) = v, t ≤ b and spec(T ψ t 1 with multiplicity at least 2. Thus, F ∈ / C(a, b), which proves that C(a, b) is open. Given a > 0, consider the map F a : S ∗ M × (0, 2a) × C(a, 2a) −→ S ∗ M × S ∗ M, defined by F a (X, t, F ) = (X, F(X, t, F )). It follows from Lemma 7.3.3 that  D. Indeed, let F a (X0 , θ, F0 ) ∈ D. Then θ < 2a and c˜ = {ψtF0 (X) : 0 ≤ Fa  t ≤ θ} is a periodic trajectory of period θ. If θ is the minimal period, then  D at (X0 , θ, F0 ). F a (X0 , t, F ) = F2 (t, F ) and Lemma 7.3.3(b) imply that F a  Next, assume that the minimal period ω of c˜ is < θ; then clearly ω ≤ θ/2 ≤ a. Since F0 ∈ C(a, 2a), c˜ is non-degenerate as a trajectory with period θ. By Lemma 7.3.3(a)

BUMPY SURFACES

201

 D at (X0 , θ, F0 ) again. Hence and F a (X, t, F0 ) = F1 (X, t), we see that F a   D. Fa  Next, for F ∈ C(M ) define FFa : S ∗ M × (0, 2a) −→ S ∗ M × S ∗ M by FFa (X, t) = F a (X, t, F ). Set  D}. U = {F ∈ C(a, 2a) : FFa  It follows from Abraham’s Transversality Theorem (see Section 1.1) that U is open  D, and dense in C(a, 2a). On the other hand, U ⊂ C(3a/2, 3a/2). Indeed, if F a  then every periodic trajectory of ψtF of period θ < 2a is non-degenerate as a trajectory of period θ, so F ∈ C(3a/2, 3a/2). Thus, C(3a/2, 3a/2) ∩ C(a, 2a) is an open and dense subset of C(a, 2a). Next, fix an arbitrary a > 0. We will now show that C(a, 2a) is dense in C(a, a). Consider an arbitrary F0 ∈ C(a, a) and an arbitrary open neighbourhood U of F0 in C(a, a). Notice that if X ∈ S ∗ M generates a periodic trajectory c˜ of ψtF0 with period not greater than a, then c˜ is non-degenerate as a trajectory with period a. Therefore, there exists a neighbourhood V of X in S ∗ M such that there is no Y ∈ V \ {X} generating a periodic trajectory of period ≤ a. Now the compactness of S ∗ M shows that there exist only finitely many periodic trajectories c˜i = {ψtF0 Xi : 0 ≤ t ≤ θi },

i = 1, . . . , k,

of the flow ψtF0 with minimal periods θi ≤ a. A standard result from the theory of differential equations implies that there exist a neighbourhood V ⊂ U of F0 and continuous maps Zi : V −→ S ∗ M,

ωi : V −→ R+ ,

i = 1, . . . , k,

with Zi (F0 ) = Xi , ωi (F0 ) = θi for all i and such that for every F ∈ V c˜i (F ) = {ψtF Zi (F ) : 0 ≤ t ≤ ωi (F )},

i = 1, . . . , k,

(7.47)

are periodic trajectories of the flow ψtF with minimal periods ωi (F ), respectively. Moreover, using the non-degeneracy of the trajectories c˜i and a compactness argument very similar to one of those already applied in this proof, we see that if V is sufficiently small, then for every F ∈ V if c˜ = {ψtF (X)} is a periodic trajectory of period ≤ a, then it coincides with some of the trajectories (7.47). Then by Theorem 7.2.1 there exists F ∈ V such that the trajectories (7.47) are non-degenerate as trajectories with periods kωi (F ), respectively, for every integer k ≥ 1. However, as we have already mentioned, the trajectories (7.47) are the only periodic trajectories of ψtF having minimal periods ≤ a. Therefore, F ∈ C(a, 2a). In this way we have proved that C(a, 2a) is dense in C(a, a). Hence C(3a/2, 3a/2) is dense in C(a, a). The latter implies that C((3a/2)k , (3a/2)k ) is

202

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

dense in C(a, a) for every integer k ≥ 1, and therefore C(b, b) is dense in C(a, a) for every b ≥ a. Let again a > 0 be a fixed number. To show that C(a, a) is dense in C(M ), fix an arbitrary F ∈ C(M ) and an arbitrary neighbourhood U of F in C(M ). We may assume that U is so small that there exists α ∈ (0, a) such that for every G ∈ U the flow ψtG has no periodic trajectories with periods ≤ α. Then obviously U ⊂ C(α, α). Now the density of C(a, a) in C(α, α) yields that U ∩ C(a, a) is non-empty. Thus, C(a, a) is dense in C(M ). The above shows that each of the sets C(k, k) is open and dense in C(M ). Thus,  B is a residual subset of C(M ), which proves the theorem. The Classical Bumpy Metric Theorem concerns the space GM of all smooth Riemannian metrics on a given smooth compact manifold M . More precisely, GM is the set of all smooth symmetric and positive definite tensors g ∈ T20 (M ) (see e.g. Sections 1.7 and 2.5 in [AbM]), endowed with the Whitney C ∞ topology. Using the well-known fact that for every g ∈ GM , the Riemannian manifold (M, g) can be isometrically embedded in some Rn (with the standard metric), and applying Theorems 7.2.1 and 7.4.1 we deduce the following. Corollary 7.4.2 (Bumpy Metric Theorem): There exists a residual subset of GM consisting of bumpy metrics on M .

7.5 Notes The Classical Bumpy Metric Theorem (Corollary 7.4.2) was announced by Abraham [Ab] giving an idea of a proof. More general results were established by Klingenberg and Takens [KT]. Given a smooth manifold M with dim(M ) = m + 1, an integer k > 0 and open, dense and invariant subset Q of the Lie group of k-jets of smooth local symplectic maps (R2m , 0) −→ (R2m , 0), it was proved in [KT] that there exists a residual subset R of GM such that for g ∈ R the k-jet of the Poincaré map of every closed geodesic on (M, g) belongs to Q. This global result was derived as a consequence of the following local result in [KT]: if g ∈ GM and γ is a closed geodesic on (M, g), then there exists g  ∈ GM arbitrarily close to g such that γ is a geodesic on (M, g  ) and the k-jet of the Poincaré map of γ with respect to g  belongs to Q. Different proofs of these results for k = 1 and k = 3 were given in [K2]. In fact, both [KT] and [K2] do not contain detailed proofs of the global theorem in the case k = 1 which includes the Bumpy Metric Theorem. The latter theorem was proved in details by Anosov [An] using the local theorem in [KT]. The main results in this chapter, which are analogous to special cases of the results in [KT], are taken from [S3]. The material in Section 7.1 is a mixture of parts of [KT] and [S3], presented in a different form here. Sections 7.2 and 7.3 follow [S3] with

BUMPY SURFACES

203

some modifications, while the globalization argument in Section 7.4 is a modification of Section 4 in [An]. Lemma 7.3.3 is an analogue of the main Lemma 1 in [An]. Complete analogues of the results in [KT] for hypersurfaces in Rn endowed with the standard metric were established in [ST]. Generic properties of more general Hamiltonian systems were studied by Robinson [R], Meyer and Palmore [MeyP], Takens [T], Rademacher [Ra], Contreras [Con] and many others (see also [AbM] and the historical remarks and the references in the articles just cited). It is worth mentioning that the analogue of the Bumpy Metric Theorem for general Hamiltonian systems is not true, as an example in [MeyP] shows.

8

Inverse spectral results for generic bounded domains In this chapter we study the Poisson relation for generic bounded domains Ω in Rn , n ≥ 2, with smooth boundaries ∂Ω. We begin with the simplest case n = 2 and show that for generic Ω ⊂ R2 , without any conditions concerning convexity, the Poisson relation becomes an equality. A similar result for n ≥ 3 is only known in the case of strictly convex domains. The main tools used in the proof of this are the ones developed in Chapters 3 and 4. Here we also have to deal with the lengths Tγ of the closed geodesics γ on ∂Ω. To do this, one needs to know that non-degenerate closed geodesics on ∂Ω can be approximated arbitrarily well by periodic reflecting rays in Ω. A proof of this approximation result is given in Section 8.3 using Melrose’ interpolating Hamiltonians studied in Section 8.2. Throughout this chapter (c) denotes Image (c).

8.1

Planar domains

In this section we establish that for generic bounded domains Ω in R2 with smooth boundaries X = ∂Ω the Poisson relation becomes an equality, that is we have sing supp σΩ = {0} ∪ { ± Tγ : γ ∈ LΩ }.

(8.1)

In the proof of this we use results from Chapters 3, 4 and 6. The main point here is to show that for generic Ω, LΩ consists only of periodic reflecting rays and, possibly, multiples of the boundary ∂Ω.

Geometry of the Generalized Geodesic Flow and Inverse Spectral Problems, Second Edition. Vesselin M. Petkov and Luchezar N. Stoyanov. © 2017 John Wiley & Sons, Ltd. Published 2017 by John Wiley & Sons, Ltd.

INVERSE SPECTRAL RESULTS FOR GENERIC BOUNDED DOMAINS

205

Let X be an arbitrary fixed smooth curve in R2 . A curve γ in R2 of the form γ = ∪k−1 i=1 i will be called a degenerate broken ray for X if the following conditions are satisfied: (i) there exist points x1 , . . . , xk ∈ X such that i = [xi , xi+1 ] and the interior of the segment i does not intersect X transversally for every i = 1, . . . , k − 1; (ii) i and i+1 satisfy the law of reflection at xi+1 with respect to X for every i = 1, . . . , k − 2; (iii) the curvature of X vanishes at x1 and xk , and 1 and k−1 are tangent to X at x1 and xk , respectively. The points x1 , . . . , xk will be called vertices of γ. Clearly when k = 2 the condition (ii) can be dropped. Examples of degenerate broken rays are given in Figures 8.1 and 8.2. If γ contains a segment orthogonal to X at some of its end points, we will say that γ is symmetric (see Figure 8.2); otherwise γ will be called non-symmetric. Notice that 1 = k−1 for symmetric γ.

Figure 8.1

A degenerate trajectory.

Figure 8.2 A symmetric degenerate trajectory.

206

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Proposition 8.1.1: There exists a residual subset D of C(X) such that for every f ∈ D there are no degenerate broken rays for f (X). Proof of Proposition 8.1.1: We will use the technique from Sections 6.2–6.4, slightly changing the definitions of the main objects. Fix arbitrary integers k ≥ s ≥ 2 and consider a non-symmetric map α : {1, . . . , k} −→ {1, . . . , s}

(8.2)

(see the definition in Section 6.2) such that α(1) = 1,

α(2) = 2,

α(k) = s.

(8.3)

We will use the notation Ii = Ii (α) given by (6.25). As in Chapter 6, denote by Uα the set of all y = (y1 , . . . , ys ) ∈ (R2 )(s) such that yi ∈ / conv{yj : j ∈ Ii } for all i = 1, . . . , s − 1. Define H = Hα : Uα −→ R by H(y) =

s−1 

||yα(i) − yα(i+1) ||.

i=1

Notice that, given a non-symmetric degenerate broken ray γ for Y = f (X) with f ∈ C(X), there exist integers k ≥ s ≥ 2, a non-symmetric map (8.2) with (8.3) and distinct points y1 = f (x1 ), . . . , ys = f (xs ) ∈ Y such that yα(1) , . . . , yα(k) are the successive vertices of γ. Then we will say that γ has type α. In this case we have x = (x1 , . . . , xs ) ∈ X (s) , f s (x) ∈ Uα and grad x (H ◦ f s )(x) = 0, where

(8.4)

x = (x2 , . . . , xs−1 ) ∈ X (s−2) .

The proof of (8.4) is almost the same as that of Proposition 6.2.1, and we leave it as an exercise to the reader. Notice that the curvature of Y = f (X) vanishes at the points f (x1 ) and f (xs ) and the segment [f (x1 ), f (x2 )] is tangent to f (X) at f (x2 ). Similar conditions are satisfied at f (xs ); however, we do not need them here. Next, consider the open subset V = {j 2 f (x) ∈ J 2 (X, R2 ): rank df (x) = 1} of J 2 (X, R2 ). Denote by M the set of all σ = (j 2 f1 (x1 ), . . . , j 2 fs (xs ))

(8.5)

INVERSE SPECTRAL RESULTS FOR GENERIC BOUNDED DOMAINS

207

such that σ ∈ V s , x = (x1 , . . . , xs ) ∈ X (s) , f s (x) ∈ Uα . Clearly M is an open subset of the smooth manifold Js2 (X, R2 ). Let Σ be the set of those elements (8.5) of M such that the curvature of fi (X) vanishes at fi (xi ) for i = 1 and i = s, grad x (H ◦ (f1 × · · · × fs ))(x) = 0, and

f2 (x2 ) − f1 (x1 ), N1  = 0, N1 being a non-zero normal vector to f1 (X) at f1 (x1 ). It then follows from the above that for any f in the set Dα = {f ∈ C(X) : js2 f (X (s) ) ∩ Σ = ∅},

(8.6)

there are no degenerate broken rays of type α for f (X). We will now show that Dα is a residual subset of C(X). To do this, we will first prove that Σ is a smooth submanifold of M of codimension s + 1. Consider a coordinate neighbourhood D of an element of Σ in M . We may assume that D=M∩

s 

J 2 (Vi , R2 ),

i=1

where V1 , . . . , Vs are coordinate neighbourhoods of different elements of X such that Vi ∩ Vj = ∅ whenever i = j. Fix arbitrary smooth charts ϕi : Vi −→ R and define the chart ϕ : D −→ R(s) × (R2 )(s) × R2s × R2s by ϕ(σ) = (u; v; a; b) for every σ ∈ D of the form (8.5), where u and v are determined by (6.12) and (6.13), (t)

a = (ai )1≤i≤s,1≤t≤2 , ∂(fi ◦ ϕ−1 i ) (ui ), ∂ui

(t)

b = (bi )1≤i≤s,1≤t≤2 ,

(t)

(t)

ai =

∂ 2 (fi ◦ ϕ−1 i ) (ui ) ∂u2i (t)

(t)

bi =

(1)

(2)

(8.7)

for all i = 1, . . . , s and t = 1, 2. Recall that the vector Ni = (Ni , Ni ) determined by (6.42) for n = 2 is orthogonal to fi (X) at the point fi (xi ). It is easy to see that (1) (2) (2) (1) the curvature of fi (X) vanishes at fi (xi ) if and only if ai bi − ai bi = 0. To see that Σ is a smooth submanifold of M of codimension s + 1, it is sufficient to establish that ϕ(D ∩ Σ) is a smooth submanifold of ϕ(D) of the same codimension. We will use the procedure applied several times in Chapter 6. Define the map R : ϕ(D) −→ Rs+1 by R(ξ) = ( (ci (ξ))2≤i≤s−1 ; L1 (ξ) ; Ls (ξ) ; K(ξ) ),

208

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

where the elements ξ of ϕ(D) are written in the form ξ = (u; v; a; b) with u, v, a, b given by (6.12), (6.13) and (8.7), and the functions ci , L1 , Ls and K are defined as follows: ci (ξ) =

∂H (1) ∂yi

(1)

(v)ai +

(1) (2)

∂H

(2) (1)

Lj (ξ) = aj bj − aj bj , (1)

(1)

(2)

(2)

(2)

∂yi

(v)ai ,

i = 2, . . . , s − 1,

j = 1, s, (2)

(2)

(1)

K(ξ) = (v2 − v1 )a1 − (v2 − v1 )a1 . It is now clear that ϕ(D ∩ Σ) = R−1 (0), so showing that R is a submersion on ϕ(D) will prove that ϕ(D ∩ Σ) is a smooth submanifold of ϕ(D) of codimension s + 1. Let ξ ∈ ϕ(D) and assume that s−1 

Ci grad ci (ξ) + A1 grad L1 (ξ) + As grad Ls (ξ) + Bgrad K(ξ) = 0

i=2 (1)

for some constants Ci , Aj and B. For a given j = 1, s we have either aj = 0 or (2) (t) aj = 0. Thus, considering the derivatives with respect to bj in the above equality, we get A1 = As = 0. In a similar way, using the fact that the functions ci (ξ) do (t) (1) (2) not depend on a1 and v1 = (v1 , v1 ) = 0, one gets B = 0. Finally, using an argument very similar to that in the proof of Lemma 6.1.2 and an obvious analogue of Lemma 6.2.2 for the present function H, we obtain Ci = 0 for all i = 2, . . . , s − 1. Hence, R is a submersion on ϕ(D), so ϕ(D ∩ Σ) is a smooth submanifold of ϕ(D) of codimension s + 1. In this way we have established that Σ is a smooth submanifold of M of codimension s + 1. By the definition Dα we have  Dα = {f ∈ C(X) : js2 f Σ} (see e.g. the end of the proof of Theorem 6.1.1), and now Theorem 1.1.2 implies that Dα is a residual subset of C(X). Set D = ∩α Dα , where α runs over the set of all non-symmetric maps (8.2) with (8.3). Then D is a residual subset of C(X). It follows from the above that for any f ∈ D , there are no non-symmetric degenerate broken rays for f (X). As in Chapter 6, the treatment of the symmetric case is very similar to the non-symmetric case. We leave it to the reader to prove in details that there exists a residual subset D of C(X) so that for any f ∈ D there are no symmetric degenerate broken rays for f (X). Then the set D = D ∩ D has the required  properties.

INVERSE SPECTRAL RESULTS FOR GENERIC BOUNDED DOMAINS

209

Using the above proposition, we intend to show that for generic Ω in R2 every generalized periodic geodesic in Ω, which is not contained in ∂Ω, is a periodic reflecting ray. To do so, we will apply results of Melrose and Sjöstrand on properties of generalized geodesics; however, we need to know that the curvature of ∂Ω does not vanish of infinite order. The following proposition shows that a much stronger property is satisfied by generic domains. We prove it in a more general form, having in mind another application in Chapter 11. Proposition 8.1.2: Let X be a smooth (n − 1)-dimensional submanifold of Rn , n ≥ 2, and let K be the set of those f ∈ C(X) for which there are no points y ∈ f (X) and directions v ∈ Ty F (X) \ {0} such that the curvature of f (X) at y vanishes of order 2n − 3 in the direction of v. Then K is a residual subset of C(X). In the proof of this proposition we will use the following lemma that can be established using the argument from the proof of Thom’s Transversality Theorem (see [GG]). We omit the details. Lemma 8.1.3: Let Σ, X and Y be smooth manifolds, let k ≥ 1 be an integer and let g : Σ −→ J k (X, Y ) be a smooth map. Then  kf } R = {f ∈ C ∞ (X, Y ) : g j is a residual subset of C ∞ (X, Y ). Proof of Proposition 8.1.2: Set k = 2n − 1 and denote by PRn the projective space of all lines through 0 in Rn . Let M be the set of all (w, j k f (x)) ∈ PRn × J k (X, Rn ) such that rank(df (x)) = n − 1 and w is tangent to f (X) at f (x). It is easily seen that M is a smooth submanifold of PRn × J k (X, Rn ). Let Σ be the set of those (w, j k f (x)) ∈ M such that the curvature of f (X) vanishes of order 2n − 3 in the direction of w. In the following we will describe this last condition analytically. We claim that Σ is a smooth submanifold of M of codimension 2n − 2. To prove this, consider an open covering {Oj }nj=1 of PRn , where Oj is determined by all vectors w ∈ Rn with w(j) = 0. It is sufficient to show that for every j, Σj = {(w, j k f (x)) ∈ Σ : w ∈ Oj } is a submanifold of M of codimension 2n − 2. We will do this for j = 1; the other cases are similar. Consider an arbitrary smooth chart ϕ : U −→ V ⊂ X, where U is an open subset of Rn−1 and V is an open neighbourhood of some element of X. Define the chart ψ : W = M ∩ (O1 × J k (X, Rn )) −→ Rn−2 × Rn−1 × Rn × Rn

210

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

by ψ(w, j k f (x)) = (λ; u; v; a). Here λ = (λ2 , . . . , λn−1 ) ∈ Rn−2 \ {0}, w is determined by the vector  ∂ϕ ∂ϕ (u) + λj (u), ∂u1 ∂uj j=2 n−1

ϕ(u) = x,

(t)

v = f (x) and a = (ai ) is given by (t)

ai =

∂ i (f (t) ◦ ϕ) (u) ∂ui

for every t = 1, . . . , n and every multiindex i = (i1 , . . . , in−1 ) with |i| = i1 + · · · + ik ≤ k. As before, we use the notation f = (f (1) , . . . , f (n) ). The assertion will be proved if we show that ψ(Σ ∩ W ) is a smooth submanifold of ψ(W ) of codimension 2n − 2. For each u ∈ U choose a unit normal vector ν(u) to f (X) at f (ϕ(u)) so that the map u → ν(u) is continuous. Then the coefficients bij (u) of the second fundamental form of f (X) at f (x), x = ϕ(u), are determined by  2  ∂ ϕ (u) , ν(u) , bij (u) = ∂ui ∂uj for i, j = 1, . . . , n − 1. Now set λ1 = 1, and for j = 0, 1, 2, . . . , consider the functions j  n−1 n−1    ∂ λi λp λq bpq (u) . Kj (λ; u) = ∂ui p,q=1 i=1 The curvature of f (X) at f (x) vanishes of order m in the direction of w(λ) =

n−1  i=1

λi

∂ϕ (u) ∂ui

if and only if Kj (λ; u) = 0 for all j = 0, 1, . . . , m (see e.g. [GKM]). Clearly, there exist smooth functions ˜ : ψ(W ) −→ R ν˜, K j ˜ (ξ) = K (λ; u) whenever ξ = (λ; u; v; a) = such that ν˜(ξ) = ν(u) and K j j k ψ(w; j f (x)). Define the map K : ψ(W ) −→ R2n−2

INVERSE SPECTRAL RESULTS FOR GENERIC BOUNDED DOMAINS

211

˜ (ξ))2n−3 . Then ψ(W ∩ Σ) = K −1 (0), and it is sufficient to show that by K(ξ) = (K j j=0 K is a submersion at any point of ψ(W ∩ Σ). Let ξ = (λ; u; v; a) ∈ ψ(W ∩ Σ), and let 2n−3 

˜ (ξ) = 0 Aj grad K j

(8.8)

j=0

for some real constants Aj . Since W ⊂ M and ξ ∈ ψ(W ), there exists t = 1, . . . , n with ν˜(t) (ξ) = 0. Fix such a t and set i = (2n − 1, 0, . . . , 0). Next, considering the (t) derivatives with respect to ai in (8.7), we get A2n−3 = 0. In the same way one obtains A2n−4 = 0, etc. Thus, Aj = 0 for all j, so K is a submersion. This shows that Σ is a smooth submanifold of M of codimension 2n − 2. Now we can apply Lemma 8.1.3 for Σ, X, Y = Rn and g = π ◦ i, where i : Σ −→ M is the inclusion and π : M −→ J k (X, Rn ) is the natural projection. Since codim ((Tσ π)) ≥ (2n − 2) − (n − 2) = n  is equivalent to the relation and dim (X) = n − 1 < n, the condition j k f g j k f (X) ∩ π(Σ) = ∅. On the other hand, K coincides with the set of those f ∈ C(X) satisfying the latter relation. Therefore, by Lemma 8.1.3, K is a residual subset of  C(X). In the rest of this section we consider the case X ⊂ R2 , that is n = 2. We will also assume that X is compact and connected; then X = ∂Ω for some bounded domain Ω in R2 . Given f ∈ C(X), we will denote by Ωf the bounded domain in R2 with boundary f (X) and by Lf the length of the curve f (X). Using the notation in Propositions 8.1.1 and 8.1.2, we have the following. Corollary 8.1.4: Let f ∈ D ∩ K. Then every periodic generalized geodesic in Ωf which is not contained in f (X) is a periodic reflecting ray in Ωf . Proof: Consider an arbitrary periodic generalized geodesic γ in Ωf which is not entirely contained in f (X). Since f ∈ K, the curvature of ∂Ωf = f (X) can only simply vanish. Therefore, γ is the projection of a uniquely extendible bicharacteristic of , so γ consists of a finite number of linear segments and segments (arcs) of ∂Ωf (see Section 1.2). Clearly, γ has at least one linear segment with points in the interior of Ωf . If γ contains at least one non-trivial segment (arc) of ∂Ωf , then it would contain a whole degenerate broken ray for f (X), which is a contradiction with f ∈ D. Thus, γ consists of linear segments only, so it is a periodic reflecting ray  in Ωf . The next lemma will be useful when we consider generic strictly convex domains Ω in R2 and try to prove that the length of ∂Ω is contained in sing suppσΩ .

212

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Lemma 8.1.5: There exists a residual subset W of C(X) such that every f ∈ W has the following properties: (a) every periodic reflecting ray in Ωf is ordinary and non-degenerate; / Q for every two distinct primitive periodic reflecting rays γ and δ (b) Tγ /Tδ ∈ in Ωf ; (c) for every integer s ≥ 2 there are only finitely many periodic reflecting rays in Ωf with s reflection points; / Q for every periodic reflecting ray γ in Ωf . (d) Tγ /Lf ∈ Proof of Lemma 8.1.5: It follows from Theorems 6.2.3, 6.3.1, 6.4.1 and 6.4.3 that there exists a residual subset W  such that every f ∈ W  has the properties (a)–(c). Next, fix arbitrary p, q, s ∈ N, s ≥ 2, and denote by W(p, q, s) the set of those f ∈ W  such that pTγ = qLf for every periodic reflecting ray γ in Ωf having s reflection points. We will show that W(p, q, s) is open and dense in W  . To establish the density, we may assume that id ∈ W  . Then we have to prove that W(p, q, s) contains elements arbitrarily close to id in the C ∞ topology. Let γ1 , . . . , γm be all the periodic reflecting rays in Ω = Ωid with s reflection points; since id ∈ W  , there are only finitely many of them. There exists a non-trivial closed segment (arc) Δ of X which does not contain reflection points of γi for all i = 1, . . . , m. We claim that for every f ∈ W  which is sufficiently close to id and f (x) = x for all x ∈ X \ Δ, the only periodic reflecting rays in Ωf with s reflection points are γ1 , . . . , γm . If not, there would exist a sequence {fk } ⊂ W  converging to id such that for every k there exists a periodic reflecting ray δk in Ωfk with s reflection points y1,k , . . . , ym,k and δk = γi for all i = 1, . . . , m. The latter implies that at least one reflection point of δk is in fk (Δ). We may assume that y1,k ∈ fk (Δ) for all k. Using the compactness of X, we may assume that there exists limk→∞ yi,k = yi for all i = 1, . . . , m. Then y1 ∈ Δ, and a simple continuity argument shows that y1 , . . . , ym are the reflection points of a periodic reflection ray γ for X. Now y1 ∈ Δ shows that γ = γi for all i, which is a contradiction. Thus, for every f ∈ W  sufficiently close to id and f = id on X \ Δ, γ1 , . . . , γm are the only periodic reflecting rays in Ωf with s reflection points. Clearly we can choose such f ∈ W  so that pTγi = qLf for all i = 1, . . . , m; then we will have f ∈ W(p, q, s). This shows that W(p, q, s) is dense in W  . To prove that W(p, q, s) is open in W  , consider an arbitrary sequence {fk } ⊂  / W(p, q, s). Without W \ W(p, q, s) with fk → f ∈ W  . We have to show that f ∈ loss of generality, we may assume that f = id. By the assumptions, for every k there exists a periodic reflecting ray δk in Ωfk with s reflection points y1,k , . . . , ys,k such that pTδk = qLfk . Using again the compactness of X, we may assume that there exist limk→∞ yi,k = yi for all i = 1, . . . , s. Then y1 , . . . , ys are the reflection points of a / W(p, q, s). periodic reflecting ray δ in Ω with pTδ = qLf , which implies f = id ∈ This proves that W(p, q, s) is open in W  . Setting  W= W(p, q, s), p,q,s∈N,s≥2

we obtain a residual subset of C(X) having the required properties.



INVERSE SPECTRAL RESULTS FOR GENERIC BOUNDED DOMAINS

213

What follows now is the main point in this section. Denote by Ξ the family of all bounded domains Ω in R2 with smooth connected boundaries ∂Ω with the following properties: (ND) every periodic reflecting ray in Ω is ordinary and non-degenerate; (R)

Tγ /Tδ ∈ / Q for every two distinct primitive trajectories γ and δ in LΩ ;

(K)

the curvature of ∂Ω does not vanish of order k ≥ 1, and there are no degenerate broken rays in Ω.

Notice that for strictly convex Ω the condition (K) is trivially satisfied. If X = ∂Ω = (γ) is (smoothly) parameterized by a parameter t ∈ R and κ(t) is the curvature of X at γ(t), then (K) implies that whenever κ(t) = 0 for some t, then κ (t) = 0. As in Chapters 3 and 4, given a domain Ω, we denote by {λ2j } the spectrum of the Laplace operator in Ω and by σ(t) = σΩ (t) the corresponding distribution defined in Section 3.2. Theorem 8.1.6: (a) For every bounded domain Ω in R2 with smooth connected boundary X = ∂Ω, there exists a residual subset W of C(X) such that Ωf ∈ Ξ for all f ∈ W. (b) The equality (8.1) holds for every Ω ∈ Ξ. Moreover, if Ω ∈ Ξ, then for every periodic reflecting ray γ in Ω, spec(Pγ ) is uniquely determined from the spectrum {λ2j }. Before proceeding with the proof of the theorem, let us mention that the second part in (b) means the following: If for Ω1 , Ω2 ∈ Ξ, the corresponding spectra {λ2j } of the Dirichlet problems for the Laplacians are the same, then there exists a bijection μ : LΩ1 −→ LΩ2 , μ(γ) = γ  , such that Tγ = Tγ  and spec(Pγ ) = spec(Pγ  ) for every γ ∈ LΩ1 . Proof of Theorem 8.1.6: (a) Given Ω with X = ∂Ω, consider the residual set W from Lemma 8.1.5. Then each f ∈ W has the properties (a)–(d) from Lemma 8.1.5. Hence, Ωf ∈ Ξ. (b) Let Ω ∈ Ξ. Then for every periodic reflecting ray γ in Ω, the conditions (ND) and (R) are satisfied. Thus, for each γ ∈ LΩ the assumptions of Theorem 4.3.1 are fulfilled, so Tγ ∈ sing suppσΩ . First, suppose that Ω is strictly convex. Parameterizing ∂Ω by arc length, we obtain a primitive generalized geodesic δ. It follows from Example 2.1.1 that there exists a sequence {γk } of periodic reflecting rays in Ω such that Tγk → Tδ as k → ∞. Since sing suppσΩ is a closed subset of R, it follows that Tδ ∈ sing suppσΩ . Now Theorems 3.4.7 and 4.3.1 show that (8.1) holds. Next, assume that Ω ∈ Ξ is not strictly convex. We will show that all elements of LΩ are periodic reflecting rays. Assume that there exists δ ∈ LΩ which is not a

214

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

periodic reflecting ray. Then the condition (K) and the connectedness of X = ∂Ω yield that (δ) = X. On the other hand, since Ω is not strictly convex, there exists a point x ∈ X at which the curvature of X vanishes. Let δ(t0 ) = x, and let κ(t) be the curvature of X at δ(t). Since κ(t0 ) = 0, the condition (K) implies κ (t0 ) = 0, so κ(t) changes its sign at t = t0 . Then by the properties of generalized bicharacteristics discussed in Proposition 1.2.3, it follows that there exists  > 0 such that either {δ(t) . : t0 −  < t ≤ t0 } or {δ(t) : t0 ≤ t < t0 + } is a linear segment parallel to δ(t0 ) and having an end x. Now (δ) = X implies that X contains a non-trivial linear segment which is a contradiction with (K). Thus, in this case LΩ contains only periodic reflecting rays, and as above we conclude that the equality (8.1) holds. Finally, let Ω ∈ Ξ and let γ be a periodic reflecting ray in Ω. We have to show that {λ2j } determines spec(Pγ ). Since γ is non-degenerate, we have spec(Pγ ) = {a, 1/a} for some a = ±1. It follows from Theorem 4.3.1 that for t close to Tγ we have (2π)σΩ (t) = Re(cTγ |det(I − Pγ )|−1/2 (t − Tγ − i0)−1 ) + L1loc , where c ∈ { ± 1, ±i} is a constant. Since (t − Tγ − i0)−1 − (t − Tγ + i0)−1 = 2πi δ(t − Tγ ), from σΩ (t), we determine Tγ |det(I − Pγ )|−1/2 . Therefore, Tγ |det(I − Pγ )|−1/2 can be determined from {λ2j }. On the other hand, since Ω ∈ Ξ, Tγ can be determined by Tγ . In fact, Tγ is the smallest positive number u ∈ sing suppσ such that Tγ /u is an integer. Hence we can determine the number d = |det(I − Pγ )| = 2 − (a + 1/a). Since the elements of spec(Pγ ) are the roots of the equation x2 − (2 − d)x + 1 = 0,  this completes the proof of the assertion.

8.2

Interpolating Hamiltonians

The present and the next sections are devoted to the study of the billiard ball map B in a small neighbourhood of a closed geodesic on the boundary of a strictly convex compact domain in Rn . A very useful tool here will be the so-called interpolating Hamiltonians. Their existence was established by Melrose [Me1] in a more general situation which we are now going to present briefly. Let (S, ω) be a smooth symplectic manifold with boundary ∂S and dim (S) = 2m + 2, and let F and G be two hypersurfaces of S, that is F and G are smooth

INVERSE SPECTRAL RESULTS FOR GENERIC BOUNDED DOMAINS

215

submanifolds of codimension 1 of S. Let s0 ∈ F ∩ G. There exists an open neighbourhood V of s0 in S and C ∞ functions f, g : V −→ R+ such that F ∩ V = f −1 (0),

G ∩ V = g −1 (0),

df = 0,

dg = 0 on V.

Then f and g are called defining functions in V for F and G, respectively. The considerations that follow are only local near s0 , so we will assume that V = S, that is f and g are globally defined on S. Assume that F and G have a transversal intersection at s0 , that is df (s0 ) and dg(s0 ) are linearly independent. Shrinking S = V if necessary, we may assume that F and G have a transversal intersection at any point of J = F ∩ G. Then J is a smooth submanifold of S of codimension 2. A point s ∈ J will be called a glancing point of F and G if the Poisson bracket {f, g} of f and g (see e.g. [AbM] or [H3]) vanishes at s. Thus, K = {s ∈ J : {f, g}(s) = 0} is the set of all glancing points of F and G. If s ∈ K and {f, {f, g}}(s) = 0,

{g, {f, g}}(s) = 0,

(8.9)

then s will be called a non-degenerate glancing point. From now on we will assume that (8.9) holds for every s ∈ K. Then K is a smooth submanifold of S of codimension 3. To prove this, first recall that {f, g} = Xf g, where Xf is the Hamiltonian vector field on S determined by the function f , and that for every s ∈ J the condition {f, g}(s) = 0 is equivalent to the fact that the integral curve of Xf (which is contained in F ) is tangent to G at s. Given s ∈ K, we have Xf f (s) = Xf g(s) = 0, and if d{f, g}(s) is a linear combination of df (s) and dg(s), then {f, {f, g}}(s) = (Xf {f, g})(s) = 0, in contradiction with (8.9). Thus, df (s), dg(s) and d{f, g}(s) are linearly independent for every s ∈ K, which shows that K is a smooth submanifold of S of codimension 3. Next, fix an arbitrary smooth submanifold MF of F of codimension 1 intersecting transversally ∂S at s0 ∈ ∂S ∩ F . For s ∈ F denote by πF (s) the intersection point of the integral curve of Xf through s with MF . Assuming that S = V is sufficiently small, the map πF : F −→ MF

216

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

is a well-defined smooth submersion. In fact, MF can be identified with the quotient space F/ ∼, where s1 ∼ s2 if s1 and s2 lie on the same integral curve of Xf in F . Then πF : F −→ F/ ∼ is just the canonical projection. In a similar way one defines πG : G −→ MG . Finally, set JF = πF (J) ⊂ MF ,

JG = πG (J) ⊂ MG .

We can now state the result of Melrose [Me1] which is crucial for the approximation theorem in the next section. Theorem 8.2.1: Under the assumptions and notation above, we have the following: (a) JF has a natural structure of a smooth symplectic manifold inherited from (S, ω) such that πF : J −→ JF is a smooth symplectic map and ∂JF = πF (K). There exist two uniquely determined continuous maps α± : JF −→ J such that πF ◦ α± = id,

(α+ ) ∪ (α− ) = J,

and α± are smooth on JF \ ∂JF . A similar statement holds for JG . Let β± : JG −→ J be the corresponding continuous inverses of πG . Then the maps δ± : JF −→ JF defined by δ± = πF ◦ β± ◦ πG ◦ α± , are continuous on JF and smooth and symplectic on JF \ ∂JF , δ± (∂JF ) ⊂ ∂JF , and locally δ± ◦ δ∓ = id. (b) There exist smooth symplectic coordinates (x; ξ) = (x0 , x1 , . . . , xm ; ξ0 , ξ1 , . . . , ξm ) in a neighbourhood of s0 in S such that s0 = (0, 0), F = {(x; ξ) ∈ S : x0 = 0},

G = {(x, ξ) ∈ S : ξ02 − x0 − ξ1 = 0}.

INVERSE SPECTRAL RESULTS FOR GENERIC BOUNDED DOMAINS

217

These coordinates induce canonical coordinates (x1 , . . . , xm ; ξ1 , . . . , ξm ) in JF such that ξ1 ≥ 0 in JF , ∂JF = {ξ1 = 0}, with respect to which the maps δ± have the form δ± (x1 , . . . , xm ; ξ1 , . . . , ξm ) = (x1 ± 2 ξ1 , x2 , . . . , xm ; ξ1 , . . . , ξm ). For a proof of this theorem we refer the reader to [Me1] (see also Section 21.4 in [H3]). As an immediate consequence of the above one gets the following. Corollary 8.2.2: Let s0 ∈ ∂S be a non-degenerate glancing point of F and G. There exists a neighbourhood V of s0 in S and a smooth function h : V −→ R+ , which is a defining function for ∂S in V , such that δ± have the following form in V : √ (8.10) δ± (s) = exp(± h Xh )(s). If h1 : V1 −→ R+ is another smooth function with these properties, then h − h1 vanishes of infinite order on ∂S ∩ V ∩ V1 . A smooth function h : V −→ R+ , which is a defining function for ∂S in V and satisfies (8.10), will be called a local interpolating Hamiltonian for the maps δ± . The following elementary lemma concerns the canonical form of the maps δ± from Theorem 8.2.1(b) and will be useful later. Lemma 8.2.3: Let a < b, ω > 0 and  > 0 be real numbers such that ω ≤ 2 . Define the map u : R × R+ −→ R × R+ by u(x, ξ) = (x +

√ ξ, ξ). Then for V = [a, a + ] × (0, ω],

we have j [a, ∞) × (0, ω] ⊂ ∪∞ j=0 u (V ).

Proof of Lemma 8.2.3: Fix arbitrary y > a and η ∈ (0, ω]. Notice that √ √ uj ([a, a + ] × {η}) = [a + j η, a +  + j η] × {η}. √ √ √ √ On the other hand, a +  + j η ≥ a + (j + 1) η, since  ≥ ω ≥ η. Therefore, j ∪∞ j=0 u ([a, a + ] × {η}) = [a, ∞) × {η},

which proves the assertion.



218

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Consider again a non-degenerate glancing point s0 of F and G, and let U  be a neighbourhood of s0 in S on which there exist smooth symplectic coordinates x1 , . . . , xm ; ξ1 , . . . , ξm with the properties listed in Theorem 8.2.1(b). Notice that if for some a < b, c : [a, b] −→ U = πF (U  ) ⊂ JF is the projection under πF of an integral curve of Xg through s0 , then c has the form c(t) = (A + t, 0, . . . , 0; 0, . . . , 0) for some real constant A > 0. Let  and ω be such that 0 <  < b − a, 0 < 4ω < 2 , and let U (a, b, ω) = {(x; ξ) : x1 ∈ [a, b], 0 < ξ1 ≤ ω, |xi | ≤ ω, |ξi | ≤ ω, i = 2, . . . , m} ⊂ U. For every integer j ≥ 0 set −i (U ). Uj (a, a + , ω) = U (a, a + , ω) ∩ ∩ji=0 δ+

It follows from Lemma 8.2.3 and the form of the map δ+ that j U (a, b, ω) ⊂ ∪∞ j=0 δ+ (Uj (a, a + , ω)).

In particular, j ∪∞ j=0 δ+ (Uj (a, a + , ω))

contains an open neighbourhood of c((a, b)) in JF . We will now consider the case when (c) is no longer contained in a small coordinate neighbourhood of a point. For λ > 0 denote by Vλ the set of all x1 , . . . , xm ; ξ1 , . . . , ξm ∈ R2m such that x1 ∈ (a − λ, b + λ), ξ1 ∈ [0, λ), |xi | < λ and |ξi | < λ for all i = 2, . . . , m. Then Vλ is a smooth manifold with boundary ∂Vλ = {ξ1 = 0}. Lemma 8.2.4: Let a < b be real numbers and let c˜ : [a, b] −→ K ⊂ G c) be an integral curve of the Hamiltonian vector field Xg such that all points of (˜ are non-degenerate glancing points of F and G. Let c = πF ◦ c˜ be its projection in JF , and suppose that δ+ is a well-defined and continuous invertible map in a neighbourhood U of (c) having the properties listed in Theorem 8.2.1(b). Assume that for some λ > 0 Ψ : Vλ −→ U

INVERSE SPECTRAL RESULTS FOR GENERIC BOUNDED DOMAINS

219

is a smooth map that is a local diffeomorphism and Φ(t, 0, . . . , 0; 0, . . . , 0) = c(t),

t ∈ (a − λ, b + λ).

Then there exist μ ∈ (0, λ), a continuous invertible map δ˜+ : Vμ −→ Vλ

(8.11)

with Φ ◦ δ˜+ (v) = δ+ ◦ Φ(v),

v ∈ Vμ ,

(8.12)

˜ : V −→ R+ which is a defining function of ∂V and and a smooth function h μ μ 

˜ X ˜ (v) (8.13) δ˜+ (v) = exp h h holds for all v ∈ Vμ , where Vμ is considered as a symplectic manifold with respect to the pull-back by Φ of the symplectic structure of JF inherited from S. Proof of Lemma 8.2.4: Clearly (˜ c) can be covered by a finite family of coordinate neighbourhoods U1 , . . . , Uk , each of them possessing smooth symplecting coordinates with the properties listed in Theorem 8.2.1(b). We will consider only the case k = 2; the general case follows in the same way using a simple induction on k. So, assume that (˜ c) ⊂ U1 ∪ U2 . Then there exist open subsets U1 , U2 of JF such that (c) ⊂ U1 ∪ U2 ,

Ui ⊂ πF (Ui ) ⊂ U

(i = 1, 2).

We may assume that c(a) ∈ U1 and c(b) ∈ U2 . Set V = U1 ∩ U2 ,

−i Vj = V ∩ ∩ji=0 δ+ (U ),

j = 0, 1, 2, . . . .

According to the choice of the neighbourhoods Ui , there exist smooth symplectic coordinates x1 , . . . , xm ; ξ1 , . . . , ξm in U2 with the properties listed in Theorem 8.2.1(b). There exists a ∈ (a, b) such that c([a, a ]) ⊂ U1 ,

c([a , b]) ⊂ U2 .

With respect to the coordinates in U2 we have c(t) = (A + t, 0, . . . , 0; 0, . . . , 0) for some constant A. Shifting the coordinate x1 , we may assume that A = 0. Next, fix an arbitrary  > 0 such that c([a , a + ]) ⊂ U1 ∩ U2 and take ω > 0 with 4ω < 2 . We take  and ω so small that W = {(x; ξ) ∈ U2 : a − ω ≤ x1 ≤ b + ω, 0 ≤ ξ1 ≤ ω, |xi | ≤ ω, |ξi | ≤ ω, i = 2, . . . , m}

220

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

−1 is contained in U2 ∩ δ+ (U2 ), and

V = {(x; ξ) ∈ W : a ≤ x1 ≤ a + } ⊂ U1 ∩ U2 . Using the reasoning just after Lemma 8.2.3, it follows that j W \ ∂JF ⊂ ∪∞ j=0 δ+ (V ),

j W ⊂ ∪∞ j=0 δ+ (V ).

(8.14)

˜ First, choose μ ∈ (0, λ) so that We now begin the construction of μ, δ˜+ and h.  every (x; ξ) = Φ(v) with v ∈ Vμ and x1 ∈ [a , b] is contained in W . Since Φ is a local ˜ fix diffeomorphism, there is a unique continuous map (8.11) with (8.12). To define h, an arbitrary local interpolating Hamiltonian h for δ+ in U1 ; its existence follows from the choice of U1 . Set Uμ = Vμ ∩ Φ−1 (U1 ), and define ˜ h(v) = h ◦ Φ(v),

v ∈ Uμ .

(8.15)

Then (8.12) and the fact that h is an interpolating Hamiltonian for δ+ in U1 imply that ˜ as follows. For v ∈ ∂V we simply set h(v) ˜ (8.13) holds for v ∈ Uμ . We extend h = μ 0; then (8.13) is trivially satisfied on ∂Vμ . Next, assume v ∈ Vμ \ ∂(Vμ ∪ Uμ ). Then j (s) for some j ≥ 0 and s ∈ V . Set Φ(v) ∈ W \ ∂JF , hence (8.14) implies Φ(v) = δ+  ˜ h(v) = h(s). To check the correctness of this definition, assume that Φ(v) = δ+ (s ) j−  for some  ≥ 0 and s ∈ V . Let j ≥ ; the other case is similar. Then δ+ (s) = s ∈ V ⊂ U1 , and since h is constant along the orbits of δ+ , one gets h(s) = h(s ). This j ˜ is correct. Moreover, for Φ(v) = δ+ shows that the definition of h (s), we can find a j small neighbourhood Q of s in V \ JF such that δ+ (Q) is a neighbourhood of Φ(v) j j ˜ on Φ−1 (δ+ ˜ −1 (δ+ (Q)) by h(Φ (s ))) = h(s ) for in U2 \ JF , and then we can define h ˜ is smooth in a neighbourhood of v and (8.13) holds. all s ∈ Q. This shows that h ˜ is smooth on V \ ∂V and (8.13) holds for all v ∈ V . It remains to show Thus, h μ μ μ ˜ is smooth on ∂V . that h μ Since the function (x; ξ) → ξ1 is a local interpolating Hamiltonian for δ+ in U2 , it follows from the second part of Corollary 8.2.2 that the function h − ξ1 vanishes of infinite order on V ∩ ∂JF . This implies that for every integer p > 0 there exists a constant Cp > 0 such that |h(x1 , . . . , xm ; ξ1 , . . . , ξm ) − ξ1 | ≤ Cp ξ1p for all (x; ξ) ∈ V . Let v ∈ ∂Vμ \ Uμ . Then s = Φ(v) = (x1 , . . . , xm ; 0, ξ2 , . . . , ξm ) ∈ W. For every ξ1 ∈ (0, ω) we have s = (x1 , . . . , xm ; ξ1 , ξ2 , . . . , ξm ) ∈ W ⊂ U2 ,

(8.16)

INVERSE SPECTRAL RESULTS FOR GENERIC BOUNDED DOMAINS

and there exist y1 ∈ [a , a + ] and j ∈ N such that y1 + j = s , where

221

j ξ1 = x1 , that is δ+ (s )

s = (y1 , x2 , . . . , xm ; ξ1 , ξ2 , . . . , ξm ) ∈ V. Defining h1 (s ) = h(s ) and h1 = 0 on ∂JF , one gets a function h1 on a neighbour˜  ) = h ◦ Φ(v  ) for v  in a small neighbourhood of v hood of s in U2 such that h(v 1 in Vμ . Although the choice of j and y1 above depends on s , according to (8.16) we always have |h1 (s ) − ξ1 | = |h(s ) − ξ1 | ≤ Cp ξ1p for all p ≥ 0. This shows that h1 is smooth in a neighbourhood of s in U2 ; in fact, all derivatives of h1 on ∂JF coincide with the corresponding derivatives of the func˜ is smooth in a neighbourhood of tion ξ1 . Now the smoothness of Φ implies that h  v in Vμ .

8.3

Approximations of closed geodesics by periodic reflecting rays

Let Ω be a compact strictly convex domain in Rn , n ≥ 2, with smooth boundary ∂Ω, and let γ : [0, L] −→ ∂Ω be a closed geodesic on ∂Ω. Consider the corresponding integral curve c : [0, L] −→ T ∗ (∂Ω) of the Hamiltonian vector field determined by the standard Riemannian metric on ∂Ω (see the beginning of Section 7.1). We will assume that L > 0 is the primitive period of γ (and c) and that γ (resp. c) is non-degenerate as a curve of period mL for every m = 1, 2, . . . . The latter means that spec(Pγ ) does not contain any roots of unity. There is a natural way to define a winding number for any finite sequence of points lying in a small neighbourhood U of (c) in T ∗ Ω. We take U so that (c) ⊂ ∪s∈(c) Ns () for some small  > 0, where Ns () is the -neighbourhood of s in the orthogonal complement of Ts∗ ((c)) in Ts∗ Ω. If  > 0 is sufficiently small, then the projection μ : U −→ (c), defined by μ(s ) = s for s ∈ Ns (), is a well-defined smooth submersion. As a closed oriented curve without self-intersections with the direction . determined by c(0), (c) is homeomorphic to the unit circle S1 with the counterclockwise orientation. Now for every sequence s1 , . . . , sk in U define the winding number wn({si }) to be the winding number of the sequence μ(s1 ), . . . , μ(sk ) in (c) (see Section 2.1). We define the winding number of a closed billiard trajectory as the winding number of the sequence of its successive reflection points. Let N be an arbitrary positive integer. It will stay fixed until the end of this section. Our aim is to prove the following theorem. Theorem 8.3.1: Every neighbourhood of (γ) in Ω contains a closed billiard trajectory in Ω with winding number N .

222

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

For the proof of this theorem we need to consider multiples of the closed curves c and γ. It is convenient to extend c and γ periodically with period L, that is so that c(t + L) = c(t) for all t ∈ R and similarly for γ. In fact, we will need these extensions only on a sufficiently large compact interval I containing [0, N L]. Next, we are going to use the previous section to study the billiard ball map in a neighbourhood of (γ). Consider the symplectic manifold S = T ∗ Rn endowed with the canonical symplectic form ω=

n 

dpi ∧ dqi ,

i=1

where p1 , . . . , pn ; q1 , . . . , qn are the standard coordinates in T ∗ Rn . Consider the hypersurfaces ∗ Rn = {(p, q) ∈ T ∗ Rn : p ∈ ∂Ω}, F = T∂Ω

G = S ∗ Rn = {(p, q) ∈ T ∗ Rn : |q| = 1}. Clearly g : T ∗ Rn −→ R, g(p, q) = |q|2 − 1 is a defining function for G. To get a similar function for F , fix an arbitrary function ϕ, defined and smooth on a neighbourhood of ∂Ω in Rn such that ∂Ω = ϕ−1 (0) and dϕ = 0 on ∂Ω. Then f (p, q) = ϕ(p) provides a defining function f for F . Since f depends on p only and g on q, it is clear that F and G intersect transversally at any point of ∗ Rn . J = F ∩ G = S∂Ω

To describe the set K of glancing points of F and G, notice that {f, g}(p, q) =

n  ∂f ∂g ∂f ∂g (p, q) − (p, q) ∂pi ∂qi ∂qi ∂pi i=1

=2

n  i=1

qi

∂ϕ (p) = 2 q, ∇ϕ(p), ∂pi

where ∇ϕ(p) is the gradient of ϕ at p. Since ∇ϕ(p) is parallel to the unit normal vector ν(p) to ∂Ω, pointing into the interior of Ω, the condition {f, g, }(p, q) = 0 is equivalent to q, ν(p) = 0. Therefore, the set K of glancing points coincides with S ∗ (∂Ω). To apply Theorem 8.2.1, we need to know that the points of K are non-degenerate. For (p, q) ∈ K we have {f, {f, g}}(p, q) = 2

n  ∂ϕ i=1

∂pi

2 (p)

= 0,

INVERSE SPECTRAL RESULTS FOR GENERIC BOUNDED DOMAINS

223

since dϕ(p) = 0. On the other hand, the strict convexity of ∂Ω at p implies {g, {f, g}}(p, q) = 4

n 

qi qj

i,j=1

∂2ϕ (p) = 0 ∂pi ∂pj

whenever q = 0. Therefore, every point of K is non-degenerate, so Theorem 8.2.1 is applicable in the present situation. To describe the maps δ± , first we give a geometric interpretation of the spaces MF and MG . Notice that Xf (p, q) = (0; −∇ϕ(p)),

Xg (p, q) = 2(q; 0).

This shows that every integral curve of Xf has the form p(t) = p(0),

q(t) = q(0) − t∇ϕ(p(0)).

To such a trajectory we assign the point (p(0), q  (0)), where q  (0) is the orthogonal ∗ (∂Ω). In other words, we will identify MF with T ∗ (∂Ω). projection of q(0) onto Tp(0) ∗ Then πF : F −→ MF = T (∂Ω) is the projection just defined. Now we have JF = πF (J) = B ∗ (∂Ω) = {(p, q) ∈ T ∗ (∂Ω) : |q| ≤ 1}, and clearly ∂JF = S ∗ (∂Ω) = K. Here B ∗ (∂Ω) is the closure of the set B ∗ (∂Ω) introduced in Section 4.2. In fact, πF = id on K. The maps ∗ α± : B ∗ (∂Ω) −→ J = S∂Ω Rn

are now defined as follows. For (p, q) ∈ B ∗ (∂Ω), let q ± ∈ Sn−1 be such that q ± = q ± λν(p) for some λ ≥ 0. Then q ± , ν(p) = ±λ. Set α± (p, q) = (p, q ± ). It is easy to see that these are exactly the maps from Theorem 8.2.1(a). To deal with G, notice that the integral curves of Xg have the form p(t) = p(0) + 2tq(0),

q(t) = q(0).

Thus, MG can be naturally identified with the space of all oriented lines in Rn : to the integral curve (p(t), q(t)) we assign the line trough p(0) with direction q(0). The projection πG : G = S ∗ Rn −→ MG is similarly defined: πG (p, q) is the line through p in the direction of q. In this setting, clearly JG is the subset of MG consisting of those lines that have a common point with ∂Ω. Then ∂JG = πG (K) is the set of all oriented lines tangent to ∂Ω, and πG induces a diffeomorphism between K = S ∗ (∂Ω) and ∂JG . It is now easy to describe the inverses ∗ Rn . β± : JG −→ J = S∂Ω

224

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Given an oriented line  ∈ JG with direction q ∈ Sn−1 , denote by p± the intersection points of  with ∂Ω (which may coincide) in such a way that p+ = p− + λq, λ ≥ 0 (see Figure 8.3(a)). Set β± () = (p± , q). These maps clearly have the properties described in Theorem 8.2.1(a). We can now describe the map δ+ = πF ◦ β+ ◦ πG ◦ α+ : B ∗ (∂Ω) −→ B ∗ (∂Ω). ∗ Rn . Then  = πG (p, q + ) is the line Given (p, q) ∈ B ∗ (∂Ω), we have (p, q + ) ∈ S∂Ω + + + through p with direction q . Thus, β+ () = (p , q ), where p+ is the other intersection point of  and ∂Ω (clearly we will have p+ = p if q + = q). Finally, πF (p+ , q + ) = (p+ , r) ∈ B ∗ (∂Ω), where r is the orthogonal projection of q + on Tp∗+ (∂Ω), and then δ+ (p, q) = (p+ , r) (see Figure 8.3(b)). Thus, δ+ is globally defined and is naturally equivalent to the billiard ball map defined in Section 2.1. In this section we will simply set B = δ+ and call it the billiard ball map. This map is the extension of the billiard map β defined in Section 4.2. Thus,

B = πF ◦ β+ ◦ πG ◦ α+ : B ∗ (∂Ω) −→ B ∗ (∂Ω), and B −1 = δ− . Setting m = n − 1, we have dim (∂Ω) = m. Consider the closed interval I = [−L, N L + L] ⊂ R. As in Section 7.1, there exist an open neighbourhood O of (γ) in ∂Ω and a local diffeomorphism r : Ou = (−u − L, u + N L + L) × Bu (0) −→ O, where u > 0 and Bu (0) is the open u-ball about 0 in Rm−1 with the following properties: (i) γ(t) = r(t, 0, . . . , 0) for every t ∈ (−u − L, u + N L + L); (ii) the 1-jet of g11 coincides with the 1-jet of the constant 1 at all points of γ0 = {(t, 0, . . . , 0) ∈ Rm : t ∈ (−u − L, u + N L + L)}; (iii) g1i = 0 on γ0 for all i = 2, . . . , m. p+

p+

r

l q+

q p–

𝜕Ω (a)

Figure 8.3

q+

l

p

q (b)

The billiard ball map B = δ+ .

𝜕Ω

INVERSE SPECTRAL RESULTS FOR GENERIC BOUNDED DOMAINS

225

Here gij (i, j = 1, . . . , m) are the components of the standard metric on ∂Ω in the coordinates x1 , . . . , xm provided by r. Let ξ1 , . . . , ξm be the corresponding dual coordinates in T ∗ (∂Ω). Then, as before, in a sufficiently small neighbourhood of any point of (c), x1 , . . . , xm ; ξ1 , . . . , ξm can be used as coordinates. In such coordinates ω=

m 

dxi ∧ dξi

i=1

is the canonical symplectic form on T ∗ (∂Ω), while H(x; ξ) =

m 1  g (x)ξi ξj 2 i,j=1 ij

is the function whose Hamiltonian vector field XH determines the geodesic flow on T ∗ (∂Ω). That is, the geodesics on ∂Ω are exactly the projections of the integral curves of XH on T ∗ (∂Ω). In what follows it will be convenient to make the correspondence between the coordinates (x; ξ) and points in T ∗ (∂Ω) more clear. For u > λ > 0 set  = {(x; ξ) : x ∈ O , 1 − λ < 2H(x; ξ) ≤ 1}. W λ u  is a submanifold of O with boundary Clearly W λ u  = {(x; ξ) ∈ O : 2H(x; ξ) = 1}. ∂W λ u  the unique point in T ∗ (∂Ω) with Let Ψ be the map that assigns to any (x; ξ) ∈ W λ coordinates (x; ξ) as explained above. If λ > 0 is sufficiently small, this map is well defined. Notice that  ) Wλ = Ψ(W λ is an open subset of B ∗ (∂Ω) and  ). ∂Wλ = Wλ ∩ S ∗ (∂Ω) = Ψ(∂ W λ Moreover,  −→ W Ψ:W λ λ is a local diffeomorphism. In order to apply Lemma 8.2.4, first we will slightly change the map Ψ to get a map Φ : Vλ −→ T ∗ (∂Ω)

226

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

satisfying the corresponding requirements. Denote the points in Vλ (see the text just before Lemma 8.2.4 for this notation) by (y; η) = (y1 , . . . , ym ; η1 , . . . , ηm ). Define the map  −→ V ψ:W λ λ by ψ(x; ξ) = (y; η), where y = x, ηi = ξi for i = 2, . . . , m and η1 = 1 − 2H(x; ξ) = 1 −

m 

gij (x)ξi ξj .

i,j=1

Since ξ1 = 1, ξ2 = · · · = ξm = 0 along (c), we have  ∂η1 = −2 g1i (x1 , 0, . . . , 0)ξi = −2 ∂ξ1 i=1 m

at all points of (c). Therefore, if λ > 0 is sufficiently small, the map ψ is a diffeomorphism. Assume now that λ > 0 is fixed small enough so that it satisfies the above requirements. Then Φ = Ψ ◦ ψ −1 is a local diffeomorphism between Vλ and Φ(Vλ ) = Wλ ⊂ T ∗ (∂Ω). To get the situation in Lemma 8.2.4, we endow Vλ with  via the the symplectic structure induced by the canonical symplectic form ω on W λ diffeomorphism ψ. Then Φ becomes a smooth symplectic map. Applying Lemma  ) using the diffeomorphism ψ, we 8.2.4 and replacing the pair (Φ, Vλ ) by (Ψ, W λ get the following. Lemma 8.3.2: There exist μ ∈ (0, λ), a continuous invertible map  ˜:W  −→ W B λ λ with ˜ Ψ ◦ B(v) = B ◦ Ψ(v),

(8.17)  , v∈W λ

(8.18)

˜:W  and  −→ R , which is a defining function for ∂ W and a smooth function h λ + λ 

˜ X ˜ (v) ˜ (8.19) B(v) = exp h h  . for all v ∈ W λ ˜ is a covering map for the billiard ball map B by means of the In other words, B ˜ is an interpolating Hamiltonian for B ˜ on the whole domain covering map Ψ, and h  . W λ

INVERSE SPECTRAL RESULTS FOR GENERIC BOUNDED DOMAINS

227

Using the canonical local normal fibration of B ∗ (∂Ω) in the neighbourhood Wλ of (c), we get a smooth family Σ(t), t ∈ R, of smooth submanifolds of Wλ of codimension one such that for every t, Σ(t) contains c(t) and is transversal (in fact, orthogonal) to (c) at c(t). Each Σ(t) is a manifold with boundary ∂Σ(t) = Σ(t) ∩ S ∗ (∂Ω), and Σ(t + L) = Σ(t)

(8.20)

for all t. Notice that if (p, q) ∈ Σ(t), with pi , qi being the standard coordinates in T ∗ Rn , then p = c(t) and (p, q/|q|) ∈ ∂Σ(t). Next, consider the curve  , d(t) = (t, 0, . . . , 0; 0, . . . , 0) ∈ W λ which is the preimage of the integral curve c with respect to Ψ. For each t ∈ (−u − ˜  , passing through L, u + N L + L) there exists a smooth submanifold Σ(t) of W λ d(t) and transversal to (d) at d(t), such that ˜ Ψ(Σ(t)) = Σ(t). ˜  corresponding to Σ(t) in W . For t ∈ I Thus, Σ(t) is the smooth fibration of W λ λ define the map ˜ ˜ + N L) −→ Σ(t P˜t : Σ(t) ˜ locally around d(t) as follows. Given v ∈ Σ(t), consider the integral curve v(t) of the Hamiltonian vector field Xh˜ through v with v(0) = v, and denote by P˜t (v) the ˜ + N L). Then P˜ (v) = v(T ) for some T = intersection point of this curve with Σ(t t Tv . Setting Pt (Ψ(v)) = Ψ(v(T )), we get another map Pt : Σ(t) −→ Σ(t) = Σ(t + N L). Clearly, P˜t is a smooth local symplectic map of a small neighbourhood of d(t) in ˜ ˜ + N L). P has similar properties Σ(t) onto a neighbourhood of d(t + N L) in Σ(t t and moreover Pt ◦ Ψ = Ψ ◦ P˜t ˜ locally around d(t) in Σ(t) for each t ∈ I. Notice that the notation Pt , P˜t , Σ(t) and ˜ Σ(t) differs from the corresponding one in Section 7.1. However, the restriction of Pt on ∂Σ(t) = Σ(t) ∩ S ∗ (∂Ω) is exactly the Poincaré map of the integral curve c(t) (t ∈ [0, N L]) of XH , that is defined by means of the geodesic flow on S ∗ (∂Ω). Consider the map τ : Wλ −→ ∂Wλ = Wλ ∩ S ∗ (∂Ω)

(8.21)

which is the restriction of the orthogonal projection of B ∗ (∂Ω) \ {0} on S ∗ (∂Ω) along the normal fibres. More precisely, using again the standard coordinates pi , qi

228

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

in T ∗ Rn from the beginning of this section, we have τ (p, q) = (p, q/|q|). We assume that λ > 0 is taken so small that q = 0 for all (p, q) ∈ Wλ . Then (8.21) is a well-defined smooth submersion. Notice that the corresponding map  ,  −→ ∂ W τ˜ : W λ λ for which Φ ◦ τ˜ = τ ◦ Φ, has the form τ˜(x; ξ) = (x; ξ/ 2H(x; ξ)), and is also a smooth submersion. Next, we are going to exploit the non-degeneracy of the curves c and γ. Lemma 8.3.3: For all sufficiently small  ∈ (0, μ) there exists a unique family of smooth maps ct : [1 − , 1] −→ Σ(t),

t ∈ I,

such that τ ◦ Pt ◦ ct (s) = τ ◦ ct (s),

s ∈ [1 − , 1],

(8.22)

and the map I × (1 − , 1] −→ Wλ ,

(t, s) → ct (s),

is smooth. The uniqueness means the following. If t : [1 − δ, 1] −→ Σ(t),

t ∈ I,

is another smooth family with the same properties, then ct (s) = t (s) for all t ∈ I and all s ∈ [1 − min{, δ}, 1]. Proof of Lemma 8.3.3: For all sufficiently small δ > 0, Ψ induces a diffeomorphism between  : t − δ < x < t + δ, |x | < δ, |ξ | < δ, i = 2, . . . , m} ⊂ W  ˜ = {(x; ξ) ∈ W U t δ 1 i i δ ˜ ) ⊂ W for all t ∈ I. Then we can use (x; ξ) as coordinates in each and Ut = Ψ(U t δ ˜ ). Ut by means of the chart (Ψ, U Fix an arbitrary t0 ∈ I and introduce new coordinates (x; η) in U = Ut0 setting η1 = ξ1 , η  = ξ  / 2H(x; ξ).

INVERSE SPECTRAL RESULTS FOR GENERIC BOUNDED DOMAINS

229

Here we use the notation η  = (η2 , . . . , ηm ),

ξ  = (ξ2 , . . . , ξm ).

Clearly, (x; ξ) = (y; η) on U ∩ S ∗ (∂Ω), and in the new coordinates the map τ has the form τ (x; η1 , η  ) = (x; ηˆ1 , η  ). Fix an arbitrary t ∈ Δ = (t0 − δ, t0 + δ). Then the points in Σ(t) have the form (t, x ; s, η  ), where s ∈ (1 − δ, 1] and in these coordinates the map Pt : Σ(t) −→ Σ(t) ⊂ U takes the form

Pt (t, x ; s, η  ) = (t, z  ; χ, ζ  )

(8.23)

for some χ ∈ R, z  , ζ  ∈ Rm−1 . Fix s ∈ [1 − δ, 1] and consider the map Qs : R2m−2 −→ R2m−2 defined in a small neighbourhood of 0 by Qs (x ; η  ) = (z  ; ζ  ), where z  and ζ  are determined by (8.23). As we have already mentioned above, for s = 1, Pt coincides with the Poincaré map of c; therefore, the non-degeneracy of c implies that id − dQ1 is invertible at 0. Since dQs depends smoothly on s, this yields that id − dQs (0) is an invertible map for all s ≤ 1 sufficiently close to 1. Applying the Implicit Function Theorem to the system   x = z  (x ; s, η  ) η  = ζ  (x ; s, η  ), we find 0 <  < μ < λ and maps s → x (s),

s → η  (s),

defined and smooth for s ∈ [1 − , 1] such that Pt (t, x (s); s, η  (s)) = (t, x (s); χ(s), η  (s)),

s ∈ [1 − , 1].

(8.24)

Now we define the curve ct : [1 − , 1] −→ Σ(t) by

ct (s) = (t, x (s); s, η  (s)).

(8.25)

230

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

The smoothness and the uniqueness of the map (t, s) → ct (s) for t ∈ Δ, s ∈ [1 − , 1], follow from the Implicit Function Theorem, while (8.22) for t ∈ Δ is a consequence of the definition of ct (s), (8.24) and the form of the map τ in the coordinates (x; η). Finally, take t1 , . . . , tk ∈ I such that the intervals Δi = (ti − δ, ti + δ), (i) i = 1, . . . , k, cover I and choose  > 0 so small that the maps ct (s) determined as above replacing Δ by Δi are all defined for s ∈ [1 − , 1]. By the uniqueness of (i) (j) these functions, it follows that ct (s) = ct (s) for s ∈ Δi ∩ Δj . Therefore, setting (i) ct (s) = ct (s) whenever t ∈ Δi and s ∈ [1 − , 1], we obtain a smooth family of  maps having the desired properties. For t ∈ I there exists an unique smooth map ˜ ˜ ⊂U c˜t : [1 − , 1] −→ Σ(t) such that ct = Ψ ◦ c˜t . ˜ to flow along the correDenote by Tt (s) the time required for the points c˜t (s) ∈ Σ(t) ˜ + N L). ct (s)) ∈ Σ(t sponding integral curve of the Hamiltonian Xh˜ to the point P˜t (˜ Then, assuming that  > 0 is sufficiently small, Tt (s) is a continuous bounded function of (t, s) with Tt (1) = N L > 0,

t ∈ I.

Lemma 8.3.4: For all sufficiently large  ∈ (0, μ) there exists an integer k0 > 0 such that if 0 <  ≤ 0 , then for every k ≥ k0 there is a unique smooth function sk : I −→ [1 − , 1] with k

 ˜ c (s (t))) = T (s (t)) h(˜ t k t k

(8.26)

for all t ∈ I. ˜ is a defining function for Proof of Lemma 8.3.4: Recall that h  ∩ S ∗ (∂Ω)  =W ∂W μ μ  (see Lemma 8.3.2). On the other hand, from the proof of Lemma 8.3.3, the in W μ curve ct (s) has the form (8.25) in the coordinates (x; η). We may choose  ∈ (0, μ) with the properties in Lemma 8.3.3 such that ˜ ∂h (x; η) = 0 ∂η1 for all (x; η) ∈ Wμ with η ∈ [1 − , 1].

INVERSE SPECTRAL RESULTS FOR GENERIC BOUNDED DOMAINS

231

˜ except that with respect to η , are zero for Next, notice that all derivatives of h, 1 η1 = 1. Thus, the absolute values of these derivatives will be arbitrarily small for all η1 ∈ [1 − , 1], provided  is chosen sufficiently small. By (8.25) for such  we have d˜ h(˜ ct (s)) = 0, s ∈ [1 − , 1]. ds This implies that for any sufficiently large integer k > 0 the function  ˜ c (s)) − T (s) gt (s) = k h(˜ t t is strictly monotone in [1 − , 1] and takes values with distinct signs at 1 −  and 1. Therefore, there exists a unique sk (t) ∈ [1 − , 1] such that gt (sk (t)) = 0, that is (8.26) holds. Now the Implicit Function Theorem implies that sk (t) depends  smoothly on t. For later use, let us just mention that if  > 0 is chosen sufficiently small as in the proof above, then there exist constants C2 > C1 > 0 such that ˜ η)| ≤ C (1 − η ), C1 (1 − η1 ) ≤ |h(x; 2 1

η1 ∈ [1 − , 1].

(8.27)

From now on we assume that  ∈ (0, μ) is fixed so small that the corresponding requirements of Lemmas 8.3.3 and 8.3.4 are satisfied and (8.27) holds whenever  . (x; η) ∈ W μ For k ≥ k0 setting ρk (t) = ct (sk (t)),

t ∈ I,

we obtain smooth functions ρk : I −→ Wμ ⊂ B ∗ (∂Ω) such that ρk (t) ∈ Σ(t),

t ∈ I.

Using (8.18) and (8.19) for the covering map (8.17) of B, we will now show that B k (ρk (t)) ∈ Σ(t) for all t ∈ I. Indeed, for v = c˜t (sk (t)), (8.19) implies 

˜ ˜ (v). ˜ k (v) = exp k hX B h Combining this with (8.26) and the definition of Tt (s), we find ˜ k (v) = P˜ (v). B t

(8.28)

232

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Using the map Ψ and taking (8.18) into account, we get B k (Ψ(v)) = Pt (Ψ(v)). Finally, ct = Ψ ◦ c˜t and the definition of ρk (t) yield (8.28). Next, considering the functions ρk (t − L) on an appropriate subinterval of I and using the uniqueness from Lemmas 8.3.3 and 8.3.4 (and decreasing  once again if necessary), we see that ρk (t) is periodic in t with period L, that is ρk (t + L) = ρk (t)

(8.29)

for all t ∈ I with t + L ∈ I and all k ≥ k0 . Thus, each ρk can be considered as a smooth map ρk : S1 −→ B ∗ (∂Ω). Lemma 8.3.5: For every k ≥ k0 and every t ∈ [0, L] the sequence {B j (ρk (t))}k−1 j=0 has winding number N and ρk (t) → c(t) as k → ∞ uniformly for t ∈ [0, L]. Proof of Lemma 8.3.5: The first assertion follows immediately from the construction of ρk (t). To prove the second assertion, it is sufficient to show that sk (t) → 1 as k → ∞ uniformly for t ∈ [0, L], which will follow trivially from the inequalities K22 K12 < 1 − s (t) < k k 2 C2 k 2 C1

(8.30)

for t ∈ [0, L], k ≥ k0 . Here C1 , C2 are the constants from (8.27), while Ki > 0 can be chosen so that (8.31) K1 ≤ Tt (s) ≤ K2 for all t ∈ I, s ∈ (1 − , 1]. To prove (8.30), first combine (8.26) and (8.31) to get 2 K12 ˜ c (s (t))) ≤ K2 . ≤ h(˜ t k 2 2 k k

(8.32)

Recall that using the coordinates (x; η) from Lemma 8.3.2, the η1 component of c˜t (s) is exactly s. Thus, applying (8.27), we find ˜ c (s (t))) ≤ C (1 − s (t)). C1 (1 − sk (t)) ≤ h(˜ t k 2 k

(8.33)

Combining (8.32) and (8.33), one gets (8.30) immediately, which proves the lemma. 

We now turn to the final step in the proof of the main theorem. Proof of Theorem 8.3.1: According to Lemma 8.3.5, the theorem will follow if we show that for every integer k ≥ k0 there exists t ∈ [0, L] with B k (ρk (t)) = ρk (t).

(8.34)

INVERSE SPECTRAL RESULTS FOR GENERIC BOUNDED DOMAINS

233

Fix an arbitrary k ≥ k0 . Let π : T ∗ Rn −→ Rn be the natural projection on the first component, that is π(x; ξ) = x, and let ||·|| be the standard norm in Rn . Define the function Fk : B ∗ (∂Ω) −→ R by Fk (v) =

k−1 

||π(B j (v)) − π(B j+1 (v))||.

j=0

Clearly Fk is continuous. It is convenient to write it as a composition Fk = R ◦ Q, where Q : B ∗ (∂Ω) −→ (∂Ω)k ,

R : (∂Ω)k −→ R

are given by Q(v) = (π(v), π(B(v)), . . . , π(B k−1 (v))) and R(x1 , . . . , xk ) =

k 

||xj − xj+1 ||.

j=1

These maps are continuous, Q is smooth for v ∈ B ∗ (∂Ω) \ S ∗ (∂Ω) with dQ(v) = 0 and R is smooth on (∂Ω)k . Since the map ρk (t) is periodic, the function Fk ◦ ρk (t) has a minimum and a maximum on [0, L]. Therefore, there exists at least one t ∈ [0, L] with d(Fk ◦ ρk ) (t) = 0. dt

(8.35)

Fix an arbitrary t with this property. We will now prove that (8.34) holds for this choice of t. It follows from ρk (t) ∈ Σ(t) that ρk (t) = (γ(t); ξ), γ being the initial geodesic on ∂Ω (see the beginning of this section). Then by (8.28), we have B k (ρk (t)) = (γ(t); ζ) ∈ Σ(t), which implies ζ = Cξ,

C > 0.

It is now sufficient to show that C = 1; this would clearly imply (8.34). Set Q(ρk (t)) = (x1 , . . . , xk ) and v1 =

x2 − x1 , ||x2 − x1 ||

vk =

x1 − xk . ||x1 − xk ||

234

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Then x1 = γ(t) and, identifying Tx (∂Ω) and Tx∗ (∂Ω) via the natural duality, we have that the orthogonal projections of v1 and vk on Tγ(t) (∂Ω) coincide with ξ and ζ, respectively. As in the proof of Proposition 2.1.3, take arbitrary smooth charts ϕj : Rn−1 −→ Uj ⊂ ∂Ω,

j = 1, . . . , k,

with ϕj (0) = xj . Consider the function G : (Rn−1 )k −→ R defined by G(u1 , . . . , uk ) = R(ϕ1 (u1 ), . . . , ϕk (uk )). Since the segments [xj−1 , xj ] and [xj , xj+1 ] satisfy the law of reflection at xj with respect to ∂Ω for every j = 2, 3, . . . , k, using the calculations from the proof of Proposition 2.1.3, we find ∂G (i)

(0) = 0

∂uj

for all j = 2, . . . , k, i = 1, . . . , n − 1. Moreover,     ∂ϕ1 ∂ϕ1 ∂G (0) = v1 + vk , (i) (0) = ζ − ξ, (i) (0) (i) ∂u1 ∂u1 ∂u1   ∂ϕ1 (0) , = (C − 1) ξ, (i) ∂u1

(8.36)

according to the above remark about v1 and vk and taking into account the fact that ∂ϕ1 (i) (0) is tangent to ∂Ω at x1 . There exists a smooth map ∂u1

χ = (χ1 , . . . , χk ) : Δ −→ (Rn−1 )k defined on a small open interval Δ around t in R such that χ(0) = 0 and G(χ(s)) = Fk (ρk (s)). It now follows from (8.35) and (8.36) that 0 = d(G ◦ χ)(t) =

ξ,

(i)

(χ(t)) dχj (t) = (i)

j=1 i=1

 = (C − 1)

k  n−1  ∂G

n−1 

∂ϕ1

(i) i=1 ∂u1

∂uj

 (i)

(χ(t)) dχ1 (t)

n−1  ∂G (i)

i=1

∂u1

(i)

(χ(t)) dχ1 (t)

= (C − 1) ξ, d(ϕ1 ◦ χ1 )(t).

Therefore to prove C = 1, it is sufficient to show that ξ, d(ϕ1 ◦ χ1 )(t) = 0. Notice that ϕ1 ◦ χ1 (s) coincides with the first component of Q ◦ ρk (s), which is γ(s). Thus,

INVERSE SPECTRAL RESULTS FOR GENERIC BOUNDED DOMAINS

235

.

identifying γ(t) with dγ(t), using the natural identification of T (∂Ω) with T ∗ (∂Ω), we have

.

ξ, d(ϕ1 ◦ χ1 )(t) = ξ, γ(t), which is non-zero for sufficiently large k, since

.

ρk (t) → c(t) = (γ(t), bγ(t)) for some b > 0. This completes the proof of the theorem.

8.4



The Poisson relation for generic strictly convex domains

Here we prove that the equality (8.1) holds for generic strictly convex domains Ω in Rn , n ≥ 3, with smooth boundaries ∂Ω. As we have already mentioned, for a strictly convex Ω ⊂ Rn every γ ∈ LΩ is either a closed geodesic on ∂Ω or a periodic reflecting ray in Ω. If γ is not a multiple of another element δ of LΩ , then γ is called primitive. We will say that γ is a non-degenerate element of LΩ if the Poincaré map Pγ of γ has no eigenvalues that are roots of unity. Denote by Ξ = Ξ(n) the family of all strictly convex compact domains Ω in Rn with C ∞ boundaries ∂Ω satisfying the following two conditions: (R)

/ Q for every two different primitive elements γ and δ of LΩ ; Tγ /Tδ ∈

(ND) every element of LΩ is non-degenerate. Given a strictly convex domain Ω (with a smooth boundary, which will be assumed throughout this section), denote by OΩ the set of all F ∈ C(∂Ω) such that ΩF is strictly convex. Recall that ΩF is the domain with boundary F (∂Ω). Clearly, OΩ is an open subset of C(∂Ω) containing id. In particular, OΩ is a Baire topological space with respect to the topology inherited from C(∂Ω), therefore every residual subset of OΩ is dense in it. The first main result in this section shows that the family Ξ is ‘very large’ in some topological sense. In particular, for every strictly convex domain Ω there exist smooth perturbations of Ω arbitrarily close to id with respect to the C ∞ topology such that the perturbed domain is in Ξ. Moreover, almost all perturbations in OΩ have this property. Theorem 8.4.1: Let Ω be an arbitrarily strictly convex domain in Rn with smooth boundary ∂Ω. There exists a residual subset R(Ω) of OΩ such that ΩF ∈ Ξ for every F ∈ R(Ω). The proof of this theorem is postponed to the end of this section. We now proceed to prove that the Poisson relation becomes an equality for all Ω ∈ Ξ.

236

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Define the length spectrum LΩ of Ω by LΩ = {Tγ : γ ∈ LΩ }. Applying the approximation theorem from the previous section, we obtain the following characterization of the isolated points in LΩ for Ω ∈ Ξ. Proposition 8.4.2: Let Ω ∈ Ξ and let γ ∈ LΩ . Then γ is a periodic reflecting ray in Ω if and only if Tγ is an isolated point in LΩ . Proof of Proposition 8.4.2: Let γ be a closed geodesic on ∂Ω. Since Ω is strictly convex and γ is non-degenerate by (ND), it follows from Theorem 8.3.1 that γ can be approximated by periodic reflecting rays in Ω. In particular, there exists a sequence {γk } of periodic reflecting rays in Ω with Tγk → Tγ as k → ∞. Condition (R) implies Tγk = Tγ for all k. Thus, Tγ is not isolated in LΩ . Suppose now that γ is a periodic reflecting ray in Ω. We will show that Tγ is isolated in LΩ . Assume that there exists a sequence {γk } ⊂ LΩ with Tγk → Tγ and Tγk = Tγ for all k. For any k choose an arbitrary point xk ∈ γk ∩ ∂Ω and denote by vk ∈ Sn−1 the outgoing direction of γk at xk . We may assume that there exist the limits xk → x ∈ ∂Ω and vk → v ∈ Sn−1 . Using the continuity of the generalized geodesic flow (see Section 1.2), there exists δ ∈ LΩ passing through x with direction v such that Tγk → Tδ . Thus, Tγ = Tδ and (R) implies δ = γ, that is δ is a periodic reflecting ray in Ω. In particular, v is transversal to ∂Ω at x, and therefore vk is transversal to ∂Ω at xk for sufficiently large k. Thus, γk is a periodic reflecting ray with the same number of reflection points as δ = γ for large k. This now implies 1 ∈ spec(Pγ ), which is a contradiction with the non-degeneracy of γ. Hence Tγ is  an isolated point in LΩ . The central moment in this section is the following. Theorem 8.4.3: For every Ω ∈ Ξ, we have sing suppσΩ (t) = {0} ∪ { ± Tγ : γ ∈ LΩ }.

(8.37)

Proof of Theorem 8.4.3: The inclusion ⊂ follows from Theorem 3.3.2. To check the converse inclusion, consider an arbitrary periodic reflecting ray γ in Ω. Since (R) and (ND) hold, it follows from Theorem 4.3.1 that Tγ ∈ sing suppσΩ . Finally, let γ be a closed geodesic on ∂Ω. Since there exists a sequence {γk } of periodic reflecting rays in Ω with Tγk → Tγ as k → ∞ (see the first part of the proof of Proposition 8.4.2) and Tγk ∈ sing suppσΩ for all k, it follows that Tγ ∈ sing suppσΩ . This proves (8.37). 

The above theorem shows that knowing the point spectrum of the Laplacian for a domain Ω ∈ Ξ, we can recover the length spectrum of Ω, which clearly contains some geometric information about Ω. It turns out that one can also recover some part of spec(Pγ ) for every γ ∈ LΩ . To see this, the following result of Stark will be useful.

INVERSE SPECTRAL RESULTS FOR GENERIC BOUNDED DOMAINS

237

Lemma 8.4.4: Let P and Q be 2n × 2n matrices such that |det(I − P k )| = |det(I − Qk )| for all k = 1, 2, . . . . Then spec(P ) \ S1 = spec(Q) \ S1 , and there exists an integer N = N (P, Q) > 0 such that spec(P N ) = spec(QN ). For a proof of this lemma we refer the reader to the Appendix in [DG]. Combining it with Theorem 8.4.3 and the main result of Chapter 4, we obtain the following. Corollary 8.4.5: Let Ω1 , Ω2 ∈ Ξ be such that the Dirichlet problem for the Laplacian has the same spectrum for Ω1 and Ω2 . Then there exists a bijection LΩ1 → LΩ2 ,

γ → γ  ,

such that for every γ ∈ LΩ1 we have Tγ  = Tγ ; γ is a periodic reflecting ray in Ω1 if and only if γ  is a periodic reflecting ray in Ω2 ; spec(Pγ  ) \ S1 = spec(Pγ ) \ S1 and there exists an integer N = N (γ) > 0 with spec(PγN ) = spec(PγN ). Proof: It follows from our assumptions and (8.37) for Ω = Ω1 and Ω = Ω2 that {Tγ : γ ∈ LΩ1 } = {Tγ  : γ  ∈ LΩ2 }. Since (R) holds for both Ω1 and Ω2 , for every γ ∈ LΩ1 there exists a unique γ  ∈ Ω2 with Tγ  = Tγ . Moreover, the map γ → γ  is a bijection. Let γ be a periodic reflecting ray in Ω1 . By Proposition 8.4.2, Tγ is an isolated point in LΩ1 . Thus, Tγ  is an isolated point in LΩ2 , so γ  is a periodic reflecting ray. Applying Theorem 4.3.1 to the k-multiples of γ and γ  , we get |det(I − Pγk )| = |det(I − Pγk )| for all k = 1, 2, . . . . Lemma 8.4.4 now implies that the third property listed in the Corollary is fulfilled. The same property in the case of a closed geodesic γ on ∂Ω1 follows from the classical result of Duistermaat and Guillemin [DG] concerning the Laplace–Beltrami operator on manifolds without boundary, the condition (ND) and  Lemma 8.4.4. The following lemma will be used in the proof of Theorem 8.4.1 below. Lemma 8.4.6: Let M be a compact smooth (n − 1)-dimensional submanifold of Rn , let γ be a primitive closed geodesic on M and let K be a finite subset of (γ).

238

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

There exists a point q0 ∈ (γ) \ K such that for every neighbourhood U of q0 in M , there exist λ > 0 and a continuous family Fμ = id + fμ , μ ∈ (−λ, λ), of elements of C(M ) such that f0 = 0, and for every μ ∈ (−λ, λ) we have supp (fμ ) ⊂ U ,

(8.38)

the curve γμ = Fμ ◦ γ is a closed geodesic on Fμ (M ) with period θ(μ) depending . smoothly on μ and θ(0) = 0. Proof of Lemma 8.4.6: Let γ : [0, θ] −→ M , where θ > 0 is the minimal period of / K. Fix t0 γ. Clearly there exists t0 ∈ (0, θ) such that γ  (t0 ) = 0 and q0 = γ(t0 ) ∈ and q0 with these properties and consider an arbitrary neighbourhood U of q0 in M with U ∩ K = ∅. We may assume that U is so small that there exist semi-geodesic coordinates along (γ) in U . Namely, U is the image of a chart r : V = (t0 − a, t0 + a) × Ba (0) −→ U ⊂ M, where a > 0 and Ba (0) is the open ball of radius a and centre 0 in Rm , m = n − 2, and in the coordinates x0 , x1 , . . . , xm in U provided by r, for all y = (y1 , . . . , ym ) ∈ Ba (0) the curves {(t; y) : t ∈ (t0 − a, t0 + a)} are geodesic lines orthogonal to any surface {(s; y) : y ∈ Ba (0)},

s ∈ (t0 − a, t0 + a).

Then for x ∈ V we have g00 (x) = 1, g0i (x) = 0 for all i ≥ 1. Here g is the standard metric on M . We will assume that a > 0 is chosen so small that w(t) =

∂r (t; 0) = 0 ∂x0

for all t ∈ [t0 − a, t0 + a]. Fix arbitrary smooth functions ρ : Rm −→ [0, 1] with compact supp (ρ) ⊂ Ba (0), ρ(y) = 1 for all y ∈ Ba/2 (0), and ϕ : R −→ [0, 1] such that ϕ(t0 ) > 0 and supp (ϕ) ⊂ [t0 − a/2, t0 + a/2]. Define the map v : V −→ Rn by v(t; y) = −

ϕ(t) ρ(y) . w(t). . ||w(t)||2

Then v is smooth with compact supp (v) ⊂ [t0 − a/2, t0 + a/2] × Ba (0) and   ∂v (t; 0) = ϕ(t) w(t), ∂x0

INVERSE SPECTRAL RESULTS FOR GENERIC BOUNDED DOMAINS

239

for all t ∈ (t0 − a, t0 + a). For μ close to 0 define f = fμ : M −→ Rn by  z, z∈ / U, f (z) = r(x) + μ v(x), z = r(x) ∈ U. Clearly, there exists a sufficiently small λ > 0 such that for all μ ∈ (−λ, λ) we have Fμ = id + fμ ∈ C(M ). Fix an arbitrary μ ∈ (−λ, λ). The map r˜ = r + μ v : V −→ Rn ˜ = F (M ), where γ = F ◦ provides coordinates x0 , x1 , . . . , xm near (γμ ) on M μ μ μ ˜ γ. Let g˜ be the standard metric on M . Then for s ∈ (t0 − a, t0 + a) and y ∈ Ba/2 (0) we have (8.39) g˜00 (s; y) = 1 + 2μ ϕ(s) + O(μ2 ), where the last term depends only on s. Moreover, g˜0i (s; y) = 0 for all i > 0. It is now ˜ (although clear that the curve γμ (t) = r˜(t; 0), t ∈ [0, θ], is a closed geodesic on M t is not a natural parameter for it). Let θ(μ) be the minimal period of γμ . It follows from the construction of Fμ that θ(μ) depends smoothly on μ. . It remains to show that θ(0) = 0. It follows from (8.39) that  θ  θ θ(μ) = g˜00 (t; 0) dt = 1 + 2μ ϕ(t) + O(μ2 ) dt. 0

0

Differentiating this equality with respect to μ and evaluating at μ = 0, gives  θ ϕ(t) dt > 0, θ (0) = 0



which completes the proof of the lemma.

Proof of Theorem 8.4.1: Set M = ∂Ω and fix an arbitrary integer q > 0. Denote by LΩ (q) the set of all γ ∈ LΩ such that Tγ ≤ q, and if γ is a periodic reflecting ray, then it has not more than q reflection points. It follows from the results in Chapters 6 and 7 that there exists a residual subset Rq of OΩ such that every F ∈ Rq has the following properties: every element of LΩF (q) is non-degenerate; Tγ = Tδ for any two different periodic reflecting rays γ, δ ∈ LΩF (q). Notice that LΩF (q) is finite whenever F ∈ Rq (see Theorem 6.4.3). Let Rq be the set of those F ∈ Rq such that Tγ = Tδ for any two different elements γ and δ of LΩF (q). We will show that Rq is open and dense in Rq . To prove the openness, we will show that Rq \ Rq is closed in Rq . Let {Fk } ⊂ Rq \ Rq ,

Fk −→ F ∈ Rq ,

in the C ∞ topology. We have to check that F ∈ / Rq . Without loss of generality, we / Rq , we can find two elements γk = δk of LΩk (q), may assume F = id. Since Fk ∈

240

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

where Ωk = ΩFk , with Tγk = Tδk , γk being a closed geodesic passing through some point Fk (xk ), xk ∈ M , with direction dFk (uk ), uk ∈ Sxk M . Considering an appropriate subsequence, we may assume that either δk is a periodic reflecting ray with at most q reflection points for all k, or δk is a closed geodesic passing through some point Fk (yk ), yk ∈ M , with direction dFk (vk ), vk ∈ Syk M , for all k. In the first case, using a standard continuity argument and taking subsequences, one finds a closed geodesic γ on M and a periodic reflecting ray δ in Ω with at most q reflection / Rq . We leave the details to the points such that Tγ = Tδ ≤ q, which implies id ∈ reader. Consider the second case when all δk are closed geodesics. We may assume xk → x and yk → y in M , uk → u and vk → v in Sn−1 , and also Tγk = Tδk → T as k → ∞. Then there are closed geodesics γ and δ on M determined by (x, u) and (y, v), respectively, such that Tγ = Tδ = T . We claim that γ = δ. Assume that γ = δ. Let g be the standard metric on M and let gk be the Riemannian metric on M so that Fk : (M, gk ) −→ Fk (M ) is an isometry. If t ∈ [0, T ) is the time for which the geodesic flow on (M, g) shifts (y, v) to (x, u), then for the same time t the geodesic flow on (M, gk ) shifts (yk , vk ) along δk to (yk , vk ) = (xk , uk ) such that (yk , vk ) → (x, u). Therefore, without loss of generality, we may assume y = x and v = u. Denote by Σ the hyperplane in Rn passing through x and orthogonal to u. Shifting xk and yk along the geodesic γk , we may also assume xk , yk ∈ Σ for all sufficiently large k. Using the natural identification of T ∗ M with T M , we may consider the Poincaré map Pγ as a local symplectic map Pγ : SΣ∩M M −→ SΣ∩M M defined in a small neighbourhood U of (x, u). For (z, w) ∈ U consider the geodesic on (M, gk ) passing through z in direction w. After some time close to T this curve intersects Σ at some point z  with direction w ∈ Sn−1 . Define Pk (z, w) = (z  , w ). Since gk → g, there exists a neighbourhood V of (x, u) in U and k0 > 0 such that for k ≥ k0 the map Pk is well defined and smooth on V and (xk , vk ) ∈ V , (yk , vk ) ∈ V . So, Pk can be viewed as a Poincaré map for γk in the neighbourhood V of both (xk , uk ) and (yk , vk ). Since (xk , uk ) = (yk , vk ) are fixed points for Pk , for large k there exists (zk , wk ) ∈ V such that (dPk − id)(zk , wk ) is not invertible. Now Pk → Pγ , zk → x and wk → u imply that (dPγ − id)(x, u) is not invertible. Thus, 1 ∈ spec(Pγ ) in contradiction with the non-degeneracy of γ. This proves that γ = δ which implies id ∈ / Rq . Hence Rq is open in Rq . To establish the density, it is enough to assume id ∈ Rq and prove that there are elements of Rq arbitrarily close to id. So, assume id ∈ Rq and let W be an arbitrary neighbourhood of id in C(M ). Since id ∈ Rq , there are only finitely many closed geodesics γ1 , . . . , γs on M with periods ≤ q, and finitely many periodic reflecting rays δ1 , . . . , δk with periods ≤ q and not more than q reflection points. Applying Lemma 8.4.6 to γ = γ1 and K = (γ1 ) ∩ (∪i=2 (γi ) ∪ ∪kj=1 (δj )),

INVERSE SPECTRAL RESULTS FOR GENERIC BOUNDED DOMAINS

241

we find F = id + f ∈ W such that supp (f ) is contained in a small neighbourhood U of a point in (γ1 ) with U ∩ (∪i=2 (γi ) ∪ ∪kj=1 (δj )) = ∅, for which γ˜1 = F ◦ γ1 is a closed geodesic on F (M ) and / Q·{Tδ1 , . . . , Tδk }. Tγ˜1 ∈ We choose F in such a way that Tγ˜1 < q if Tγ1 < q and Tγ˜1 > q if Tγ1 = q. Notice that if U is sufficiently small and f is sufficiently close to 0 in the C ∞ topology, then δ1 , . . . , δk are the only periodic reflecting rays in ΩF with periods ≤ q and not more than q reflection points. Moreover, the closed geodesics on F (M ) with periods ≤ q are γ2 , . . . , γs and possibly γ˜1 . Thus, choosing U small enough and f sufficiently close to 0, we have F ∈ Rq . Repeating this procedure s − 1 times, we find G ∈ W ∩ Rq which proves the density of Rq in Rq . Therefore, Rq is a residual subset of OΩ . Finally, setting R(Ω) = ∩∞ q=1 Rq , we obtain a residual subset of OΩ which has all the desired properties. This proves  the theorem.

8.5 Notes A slightly different version of Proposition 8.1.2 was proved by Landis [Lan]. Lemma 8.1.3 is also contained in [Lan]. The rest of Section 8.1 is taken from [PS2] (see also [PSl]). Theorem 8.2.1 and Corollary 8.2.2 are due to Melrose [Me1]. The other material in Sections 8.2 and 8.3 is a modification of a part of Magnuson’s thesis [Mag]. The results in Section 8.4 are taken from [S3] and, as one can see, rely heavily on previous results of Anderson and Melrose [AM], Duistermaat and Guillemin [DG], Guillemin and Melrose [GM1], Magnuson [Mag], Melrose [Me1], Petkov and Stoyanov [PS2], [PS3], [PS4], Stoyanov [S1], [S3] and others. For other inverse spectral results see [Ber], [Prot], [MSi], [Vi].

9

Singularities of the scattering kernel In this chapter a formula is proved for the leading singularity of the scattering kernel at −Tγ , where Tγ is the sojourn time of an ordinary non-degenerate reflecting (ω, θ)-ray satisfying some additional assumptions. As a consequence, applying the results from Chapter 5, certain information on the singular set of the scattering kernel is obtained. A special emphasis is given to three-dimensional generic domains. We prove that for such domains, the (ω, θ)-rays of mixed type disappear and any singularity of the scattering kernel has the form −Tγ for some reflecting (ω, θ)-ray.

9.1

Singularity of the scattering kernel for a non-degenerate (ω, θ )-ray

Let Ω = Rn \ K be a connected closed domain in Rn , n ≥ 2, with bounded complement and smooth boundary ∂Ω, introduced in Chapter 5. Let ω = θ be fixed unit vectors. Throughout this chapter we use the notation from Chapter 5 (see also Section 2.4). In this section γ will be a fixed ordinary reflecting non-degenerate (ω, θ)-ray with sojourn time Tγ satisfying the following assumption: (I) Tδ = Tγ for every δ ∈ L(ω,θ) (Ω) \ {γ}. As in Section 5.3, it is easy to check that for sufficiently small  > 0 we have (Tγ − , Tγ + ) ∩ sing supp s(t, θ, ω) = {Tγ }.

Geometry of the Generalized Geodesic Flow and Inverse Spectral Problems, Second Edition. Vesselin M. Petkov and Luchezar N. Stoyanov. © 2017 John Wiley & Sons, Ltd. Published 2017 by John Wiley & Sons, Ltd.

SINGULARITIES OF THE SCATTERING KERNEL

243

Fix a such  > 0 and let ρδ (t) ∈ C0∞ (R), 0 < δ ≤ , be the function defined in Section 5.2. Let q1 ,. . ., qm be the successive reflection points of γ, and let γ˜ be the generalized bicharacteristic of  such that π(˜ γ ) = γ. We suppose that γ˜ is issued from x0 , |x0 | > ρ0 , with (incoming) direction ω. Following the localization argument in Section 5.2, introduce the distribution u(t, x; ω) as the solution of the problem: ⎧ 2 ◦ ⎪ ⎨(∂t − Δx )u = 0 in R × Ω , (9.1) u + ϕ(x)δ(t − x, ω ) = 0 on R × ∂Ω, ⎪ ⎩ u|t g(x ) in O1 ∩ Ω◦ . For η ∈ Rn consider the hyperplane Zη,a ⊂ Rn with normal η such that distance of q1 to Zη,a is a > 0. Let πη : ∂Ω → Zη,a be the projection introduced in Section 2.4 along the direction η. Applying the construction of Section 4.1, consider the Fourier integral operators + associated with the canonical relations C1 ,. . ., Cm . We modify the R1+ , R2+ ,. . ., Rm + The symbol of operator R1 = V1+ to satisfy the inhomogeneous boundary condition. f . R1+ , modulo Maslov factors, is f |dt ∧ dy ∧ dη|1/2 with f ∼ ∞ j=0 j In Section 4.1 we have choose f0 = 1 and the principal symbol of i∗ω1 R1+ , modulo Maslov factors, in local coordinates (y  , η  ) becomes γ1 (y  , η  , 1)−1/2 α1 (y  , η  )|dτ ∧ dy  ∧ dη  |1/2 , where γ1 is given by Lemma 4.2.1 and α1 (y  , η  ) > 0 is related to the expression of the half-density |dτ ∧ dy  ∧ dη  |1/2 in normal geodesic coordinates used in Lemma 4.2.1. Notice that α1 (y0 , η  ) = 1. 1/2 To obtain the identity operator in local coordinates (y  , η  ), we choose γ1 α1−1 as initial condition for f0 on ω1 and next we determine f0 from the transport equation Hq f0 = 0 taking f0 constant along the orbits of Hq . Therefore, the principal symbol 1/2 of R1+ will be γ1 α1−1 |dt ∧ dy ∧ dη|−1/2 . With this choice, we guarantee that i∗ω1 V1+ u0 ≡ u0 , provided that W F (u0 ) ⊂ Σ1 .

244

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

To satisfy the last condition, we must define Σ1 suitably. First, choose a small conic neighbourhood V of ω. Next for η ∈ V consider Uϕ,η,a = {πη (z) ∈ Zη,a : z ∈ supp ϕ ∩ ∂Ω}. The vector ω is transversal to ∂Ω for z close to q1 , hence for small V and supp ϕ the map πη is a diffeomorphism. We introduce U = {x ∈ Uϕ,η,a : |a − a0 | ≤ , η ∈ W }, where a0 > 0 is fixed and  > 0 is small enough. With this choice, we define Γ = U × V and the set Σ1 as in Section 4.1. It is easy to see that W F (u0 ) ⊂ Σ1 . By using the notation of Section 4.1, introduce + Vk+ = Rk+ (Id − Mk−1 )i∗ Vk−1 , k ≥ 2,

and consider Um =

m

(−1)k−1 Vk+ u0 .

k=1 k

Notice that the factors (−1) are chosen to obtain vanishing boundary condition on ωk , k ≥ 2. Then u − Um ∈ C ∞ (R × Ω) for t < tm+1 , where tm+1 is related to the mth reflection of the rays issued from Σ1 . Consequently, we may study (9.2) replacing u by Um . To do this, we need a representation of   ∂ ∂ m−1 ∗ + − ν, θ

Rm iωm (Id − Mm−1 )i∗ Vm−1 u0 . Lm = (−1) ∂ν ∂t + ∗ + Consider the operator Rm = Rm i Vm−1 and neglect the term involving Mm−1 , which yields a smooth term when we take the trace on ωm . This operator is related to the canonical relation     G+m = {t + s, τ, Φt+s−Tm (s,τ,y ,η ) λ m−1 (s, τ, y , η ), s, τ, y , η ) 



∈ T ∗ (R × Ω × R × ω1 )}. Similar to the canonical relation (4.6), here λm−1 (s, τ, y  , η  ) is the point on ∗ (R × Ω) obtained after (m − 1) reflections of the generalized bicharacteristics T∂Ω  issued from (s, τ, g(y  ), y  , η1 , η  ), η1 = τ 2 − |η  |2 , while λ m−1 ( . . . ) denotes the point obtained from λm−1 ( . . . ) after mth reflection and Tm (s, τ, y  , η  ) is the length of the projection of these bicharacteristics on Ω. Let Om ⊂ Rn be a small neighbourhood of qm and let in local coordinates (h(x ), x ), ωm = Om ∩ ∂Ω be given by x1 = h(x ) with qm = (0, x0 ) and x1 > h(x ) in Om ∩ Ω◦ . In local coordinates (s, y  , τ, η  ) and (t, x , σ, ξ  ) in + + ∗ iωm Vm−1 is related T ∗ (R × ω1 ) and T ∗ (R × ωm ), respectively, the operator i∗ωm Rm to the graph of a canonical transformation σm−1 and it can be expressed by a homogeneous of order 1 with respect to (τ, η  ) generating function χ(t, x , τ, η  ) (see Section 4.1) such that   χtτ χtη = 0, det χx τ χx η

SINGULARITIES OF THE SCATTERING KERNEL

245

and σm−1 : (χτ , χη , τ, η  ) → (t, x , χt , χx ). Clearly, τ is constant with respect to the action of σm−1 , and we get χtτ = 1, χtt = χtη = χtx = 0.

(9.3)

This yields det χx η = 0. We have the representation + ∗ + + iωm Vm−1 ≡ (−1)m−1 i∗ωm Vm−1 Jm = (−1)m−1 i∗ωm Rm      = (2π)−n (−1)m−1 eiχ(t,x ,τ,η )−isτ −i y ,η b(t, x , s, y  , τ, η  )ds dy  dτ dη  ,

   where b ∼ ∞ k=0 bk (t, x , s, y , τ, η ) with bk homogeneous of order −k with respect  to (τ, η ). Applying Jm to u0 , one obtains         Jm u0 = (2π)−n (−1)m eiχ(t,x ,τ,η )−i( y ,ω +g(y )ω1 )τ −i y ,η

× b(t, x , y  , ω  + g(y  )ω1 , y  , τ, η  )(1+|dg(y  )|2 )1/2 ϕ(g(y  ), y  )dy  dτ dη  . Now we pass to Lm . Let pm (x, τ, ξ) be the principal symbol of  in local coordinates (x1 , x ) and let ξ1+ (x, τ, ξ  ) be the outgoing root of the equation −ξ + pm (x, τ, ξ1 , ξ  ) = 0 with respect to ξ1 satisfying the condition τ 1 > 0. Consider the phase function ϕ+ (t, x, τ, ξ  ) determined as the solution of the problem ⎧ + ⎨ ∂ϕ = ξ + (x, ϕ+ , ϕ+ ), t 1 x ∂x ⎩ +1 ϕ |x1 =h(x ) = tτ + x , ξ  . Therefore, for x close to ωm we have  +    + −n eiϕ (t,x,τ,ξ )−isτ −i y ,ξ a(t, x, τ, ξ  )g(s, y  )ds dy  dτ dξ  (Rm g)(t, x) = (2π)  with a ∼ ∞ k=0 ak (t, x, τ, ξ ), ak being homogeneous of order −k with respect to (τ, ξ  ). Moreover, a0 |x1 =h(x ) = 1, ak |x1 =h(x ) = 0, k ≥ 1. + g are propagating along the bicharacteristics of  entering in The singularities of Rm Ω. Therefore,   ∂ ∂ + + − ν, θ

Rm g = Bm i∗ω Rm g, i∗ωm ∂ν ∂t

246

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

where Bm is a first order pseudo-differential operator. The principal symbol of Bm has the form

 β1 (x , τ, ξ  ) = i (1 + |dh(x )|2 )−1/2 ξ1+ (h(x ), x , τ, ξ  ) − ξ  , hx

 (9.4) − ν(h(x ), x ), θ τ , and modulo smooth terms, we have Lm = Bm Jm u0 . On the other hand, Bm Jm u0 is Fourier integral operator with the same phase as Jm u0 and principal symbol ˜b (t, x , y  , τ, η  ) = β (x , χ , χ  )b (t, x , y  , ω  + g(y  )ω , y  , τ, η  ). 1 1 t x 0 1 We set ψ = t − x, θ and later we study the asymptotic of (Bm Jm u0 , ρeiλψ ). The leading term is the integral  n  λ    m eiλΦ(t,x ,y ,τ,η )˜b1 (t, x , y  , τ, η  ) I0 (λ) = (−1) 2π × ρ( x , θ − t + Tγ )(1 + |dg(y  )|2 )1/2 dt dx dy  dτ dη  + lower order terms. Here the phase function Φ has the form Φ(t, x , y  , τ, η  ) = t − h(x )θ1 − x , θ + χ(t, x , τ, η  ) −( y  , ω  + g(y  )ω1 )τ − y  , η  . The critical points of Φ are determined from the equations: ⎧ ⎪ χt (t, x , τ, η  ) = −1, ⎪ ⎪ ⎪     ⎪ ⎪ ⎨χx (t, x , τ, η ) = θ + dh(x )θ1 ,    η = −(ω + dg(y )ω1 )τ, ⎪ ⎪ ⎪ χτ (t, x , τ, η  ) = y  , ω  + g(y  )ω1 , ⎪ ⎪ ⎪ ⎩χ (t, x , τ, η  ) = y  . η Let (tˆ, x ˆ , yˆ , τˆ, ηˆ ) be a critical point of Φ. Since τ is constant along the generalized bicharacteristics of , we get τˆ = 1 and we may parameterize the bicharacteristics by the time t. This implies y  )ω1 . −ˆ η  = ω  + dg(ˆ Set yˆ = (g(ˆ y  ), yˆ ) and let p(y, τ, η) be the principal symbol of . Denote by −   ηˆ1 (y , τ, η ) the incoming root of the equation p(g(y  ), y  , 1, η1 , η  ) = 0 with η1− , ν < 0. Then for yˆ close to y0 , we conclude that respect to η1 , that is ˆ −    y , 1, ηˆ ), ηˆ ) will be close to the reflected direction of γ at q1 having −ˆ η = −(ˆ η1 (ˆ the form ω − 2 ν(q1 ), ω ν(q1 ).

SINGULARITIES OF THE SCATTERING KERNEL

247

Let Tˆ be the length of the generalized geodesics ˆl issued from yˆ with direction −ˆ η ˆ ). Then and joining yˆ and x ˆ = (h(ˆ x ), x ˆ y , ω = χτ (tˆ, x ˆ , 1, ηˆ ) = tˆ − Tˆ, x, θ − since χτ = s and tˆ = sˆ + Tˆ. The reflecting direction of ˆl is close to θ and ˆ tˆ + Tγ ∈ supp ρδ yields tˆ = ˆ x, θ + Tγ + O(δ). Thus if supp ϕ and δ are small enough, the sojourn time of the ray issued from (ˆ y , −ˆ η ) is close to Tγ . Exploiting the condition (I) and the non-degeneracy of γ, we get x ˆ = x0 , yˆ = y0 , tˆ − ˆ x0 , θ = Tγ , ηˆ = −ω  . Moreover, on the critical points of Φ we have Φtt = Φtx = Φty = Φtη = 0, Φtτ = 1. Put G = ν(y0 ), ω gy y (y0 ), H = ν(x0 ), θ hx x (x0 ), and consider the matrix

⎛

G Δ=⎝ 0 I We have

0 −H − χx x −χη x ⎛

G 2 ⎝ 0 det Δ = (det χx η )det −I

I

⎞

χx η ⎠ . −χη η

0 χx η (χx x + H)χη x I

⎞ I I ⎠. χη η

To find det Δ, we apply the following. Lemma 9.1.1: Let F, M, L be n × n matrices. Then ⎛ ⎞ F 0 I det ⎝ 0 M I ⎠ = det (F LM + M − F ). −I I L Proof: First assume that F is invertible. Then ⎛ ⎞   F 0 I M I ⎝ ⎠ 0 M I = (det F ) det . det I L + F −1 −I I L It is easy to see that det



M I I L + F −1



= det ((L + F −1 )M − I),

(9.5)

248

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

from which (9.5) follows directly. In the general case replace F by F = F + I, where  > 0 is taken small so that F is invertible. Applying (9.6) for F and letting   → 0, we complete the proof. The symmetry of the matrices χy y , χη η , H, G and Lemma 9.1.1 yield   −1 det χ−1 det Δ = det (H + χ )χ (I + χ G) − χ G .           xx ηη ηη xη xη

(9.6)

By exploiting the non-degeneracy of γ, we will show that the right-hand side of (9.6) is not zero. To do this, we are going to use the hyperplane Zω , orthogonal to ω and the orthogonal projection πω onto Zω (cf. Section 2.4 for the notation). Consider the diagram π

μ1

r

μ2

μ ˜1

r˜

μ˜2

px

ω Zω ←−− −− ∂Ω −−−−→ B ∗ (∂Ω) −−−−→ B ∗ (∂Ω) −−−−→ Bx∗ (∂Ω) −−−−→ 𝕊n−1  ⏐  ⏐ ⏐ ⏐ ⏐ ⏐ j1 ⏐ j1 ∗  j2 ∗ ⏐ j2 

ℝn−1 −−−−→ T ∗ (ℝn−1 ) −−−−→ T ∗ (ℝn−1 ) −−−−→

ℝn−1

the maps in it being defined as follows. First, μ1 has the form μ1 (y) = (y, ν(y), ω ν(y) − ω) ∈ B ∗ (∂Ω). Here η(ω) = −ω + 2 ν(y), ω ν(y) ∈ Sn−1 is the reflected direction associated with −ω. To define r, we use the canonical transformation σm−1 . Applying (9.3), from the ˜ =σ ˜ η˜) equality (t, x, 1, ξ) ˜) we determine two functions x(y, η˜), ξ(y, m−1 (s, y, 1, η and define ˜ η˜)) ∈ B ∗ (∂Ω). r : B ∗ (∂Ω)  (y, η˜) −→ (x(y, η˜), ξ(y, The map μ2 is related to the reflection on ∂Ω in a neighbourhood of qm and has the form ˜ = −ξ + ν(x), ξ ν(x) ∈ T ∗ (∂Ω), μ2 (x, ξ) x where ξ ∈ Tx∗ (Ω) is the unique vector with ν(x), ξ < 0 the orthogonal projection ˜ The map p : B ∗ (∂Ω) −→ Sn−1 is defined by of which on Tx (∂Ω) coincides with ξ. x x px : Bx∗ (∂Ω)  ζ − ν(x), ζ ν(x) → ζ ∈ Sn−1 . The maps j1 , j2 , j1∗ , j2∗ are diffeomorphisms associated with the local coordinates x , y  chosen earlier. For example, j1 (y  ) = (g(y  ), y  ), j2 ( dh(x ), ξ  , ξ  ) = ξ  . Finally, the maps μ ˜1 , r˜, μ ˜2 are defined in such a way that the considered diagram is commutative.

SINGULARITIES OF THE SCATTERING KERNEL

249

It is clear that dj1 (y0 ) = I, dj2 (0, θ ) = I and μ2 ◦ r˜ ◦ μ ˜1 )(y0 ). d(μ2 ◦ r ◦ μ1 )(q1 ) = d(˜ In local coordinates y  we have μ ˜1 (y  ) = (y  , −(1 + |dg(y  )|2 )−1 (ω1 − dg(y  ), ω  )(−1, dg(y  )) − ω), hence d˜ μ1 (y0 ) =



 I . −G

Similarly, d˜ μ2 (0, x0 ) = (H, I). To obtain d˜ r, observe that the map r˜ is given by the generating function χ(tˆ, x , 1, η  ), that is      χη x r˜ : . → χx η A simple calculation shows that  

d˜ r(χη , η ) =

χ−1 η  x

−χ−1 η  x χη  η 

χx x χ−1 η  x

χx η − χx x χ−1 η  x χη  η 

 .

μ2 (x0 , θ ), we find Therefore, by using the form of d˜ μ1 (y0 ) and d˜ ˜1 )(y0 ) = −(H + χx x χ−1 d(˜ μ2 ◦ r˜ ◦ μ η  x (I + χη  η  G) + χx η  G. In the same way consider the local inverse πω−1 : Zω → ∂Ω around q1 . It is easy to see that det d(πω−1 )(uγ ) = − ν(q1 ), ω −1 , det d(px )(θ ) = ν(qm ), θ −1 . Since γ is non-degenerate, there exists a neighbourhood Wγ of uγ = πω (q1 ) in Zω such that the map Jγ : Wγ → Sn−1 is well defined and smooth (see Section 2.4). Moreover, we have Jγ = px ◦ μ2 ◦ r ◦ μ1 ◦ πω−1 and det dJγ (uγ ) = 0. On the other hand, (9.6) implies  −1 |det dJγ (uγ )| = | ν(q1 ), ω || ν(qm ), θ |     × det(H + χx x )χ−1 η  x (I + χη  η  G) − χx η  G = | ν(q1 ), ω |−1 | ν(qm ), θ |−1 |(det χx η )−1 ||det Δ|. Therefore, det Δ = 0 and we are in position to apply the stationary phase argument to the integral I0 (λ). It follows from the Euler equality for χ(t, x , τ, η  ) that Φ( x0 , θ + Tγ , x0 , y0 , 1 − ω  ) = Tγ .

250

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

On the other hand, by (9.4) for tˆ = x0 , θ + Tγ , we have d1 = β1 (x0 , χt (tˆ, x0 , 1, η0 ), χx (tˆ, x0 , 1, η0 )) = i(ξ1+ (0, x0 , 1, θ ) + ν(qm ), θ ) = 2i ν(qm ), θ , since the outgoing condition implies ξ1+ (0, x0 , 1, θ ) =

 1 − |θ |2 = θ1 .

It remains to find the form of d0 = b0 (tˆ, x0 , χη (tˆ, x0 , 1, η0 ), ω , χη (tˆ, x0 , 1, η0 ), 1, η0 ). Recall that this symbol is attached to the half-density |dτ ∧ dx ∧ dη  |1/2 . As in Section 4.2, we have |dτ ∧ dy  ∧ dη  |1/2 = |det χx η |1/2 |dτ ∧ dx ∧ dη  |1/2 . Next the principal symbol b0 (tˆ, x , sˆ, y  , τ, η  ) of Jm with respect to the half-density |dτ ∧ dy  ∧ dη  |1/2 according to Lemma 4.2.2 and our choice of V1+ , modulo Maslov factor, is equal to −1/2

γ1 (y  , η  , 1)γ1 1/2

(β m (y  , η  ), 1)α1−1 (y  , η  )αm (x , ξ  ).

Here the coefficient γ1 (y  , η  , 1) is related to the choice of initial conditions, the coef−1/2 ficient γ1 (β m (y  , η  ), 1) comes from Lemma 4.2.2, while a1 and am are related to the change of coordinates passing to normal geodesic coordinates used in Section 4.2 to local coordinates introduced in the beginning of this section. It is easy to see that α1 = αm = 1 at the points (y0 , η0 ) and (x0 , ξ0 ), respectively, since the unit normal vectors at (0, y0 ) and (0, x0 ) have the form (1, 0, . . . 0). Therefore, d0 =



1 − |θ |2

−1/2 

1 − |ω  |2

1/2

= ν(qm ), θ −1/2 | ν(q1 ), ω |1/2 ,

and we obtain ˜b (tˆ, x , χ  (tˆ, x , 1, η  ), ω , χ  (tˆ, x , 1, η  ), 1, η  ) 0 0 0 0 0 0 1 η η π

π

= ei 2 σγ d1 d0 |det χx η |1/2 = 2iei 2 σγ ( ν(qm ), θ )1/2 | ν(q1 ), ω |1/2 |det χx η |1/2 , σγ being the Maslov index. By the stationary phase argument, we get 

2π I0 (λ) = 2i λ

(n−1)/2

 π  (−1)m exp i βγ eiλTγ 2

×|det dJγ (uγ )|−1/2 + O(λ−(n+1)/2) ),

SINGULARITIES OF THE SCATTERING KERNEL

251

where βγ = σγ +

1 sign Δ ∈ N. 2

Recall that (s(t, θ, ω), ρ(t + Tγ )e−iλt ) =

ck (−iλ)n−2−k

k=0

  ×

n−2

R ∂Ω



eiλ(t− x,θ ) ρ(k) ( x, θ − t + Tγ )

∂ ∂ − ν, θ

∂ν ∂t

 u dt dSx

with c0 = 12 (−1)(n−1)/2 (2π)1−n . Thus, we obtain the following. Theorem 9.1.2: Let γ be an ordinary non-degenerate reflecting (ω, θ)-rays with mγ reflection points satisfying assumption (I). Then −Tγ ∈ sing supp s(t, θ, ω), and for t sufficiently close to −Tγ the scattering kernel has the form  π  s(t, θ, ω) = (2πi)(1−n)/2 (−1)mγ −1 exp i βγ 2 × |det dJγ (uγ )|−1/2 δ (n−1)/2 (t + Tγ ) + lower order singularities.

(9.7)

In the particular case when mγ = 1, the integral I0 (λ) can be written with a phase function ψ(x ) = x , ω  − θ + h(x )(ω1 − θ1 ). Then det Δ(x0 ) = ν(q1 ), ω − θ n−1 det hx x (x0 ), and ν(q1 ), ω + θ = 0. Thus, |det dJγ (uγ )|−1/2 = 2(1−n)/2 | ν(q1 ), ω |(3−n)/2 |det hx x (x0 )|−1/2 . Since ν(q1 ) = (θ − ω)/|θ − ω|, we get ν(q1 ), ω =

1 θ, ω − 1 = − |θ − ω|, |θ − ω| 2

and therefore |det dJγ (uγ )|−1/2 =

1 |θ − ω|(3−n)/2 |K(q1 )|−1/2 , 2

252

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

where K(q1 ) is the Gauss curvature of ∂Ω at q1 . In this way we have established the following. Corollary 9.1.3: Let γ be as in Theorem 9.1.2 and assume in addition that γ has exactly one reflection point q1 . Then for t sufficiently close to −Tγ we have s(t, θ, ω) =

 π  1 (2πi)(1−n)/2 exp i sign Δ(q1 ) |θ − ω|(3−n)/2 2 4 ×|K(q1 )|−1/2 δ (n−1)/2 (t + ω − θ, q1 ) + lower order singularities.

Moreover, if ∂Ω is strictly convex at q1 with respect to the normal field ν, then sign Δ(q1 ) = (n − 1) and s(t, θ, ω) =

1 (2π)(1−n)/2 |θ − ω|(3−n)/2 2 ×|K(q1 )|−1/2 δ (n−1)/2 (t + ω − θ, q1 ) + lower order singularities.

These results imply an asymptotic of the scattering amplitude a(λ, θ, ω) given in Section 5.1 by  a(λ, θ, ω) =

2π iλ

(n−1)/2 Ft→λ s(t, θ, ω),

where Ft→λ is the Fourier transform with respect to t.

9.2

Singularities of the scattering kernel for generic domains

Let Ω, ω, θ be as in the previous section. In this section we consider an application of Theorem 9.1.2 exploiting some results in Chapter 6. Denote by Lm ω,θ (Ω) the set of all (ω, θ)-rays of mixed type in Ω. Then Lω,θ = Lω,θ (Ω) \ Lm ω,θ (Ω) is exactly the set of all reflecting (ω, θ)-rays in Ω. Set G(Ω) = { − Tγ : γ ∈ Lm ω,θ (Ω)}. Theorem 9.2.1: There exists a residual subset R of C(X) such that for every f ∈ R we have { − Tγ : γ ∈ Lω,θ (Ωf )} \ G(Ωf ) ⊂ sing supp sΩf (t, θ, ω),

(9.8)

and for t close to −Tγ , (9.7) holds with Ω replaced by Ωf , provided −Tγ belongs to the left-hand side of (9.8).

SINGULARITIES OF THE SCATTERING KERNEL

253

Proof: It follows by Theorems 6.2.3, 6.3.3, 6.4.6 and 8.1.2 that there exists a residual subset R of C(X) such that every f in it has the following properties: (i) every reflecting (ω, θ)-ray in Ωf is ordinary and non-degenerate; (ii) Tγ = Tδ for every two different reflecting (ω, θ)-rays γ and δ in Ωf ; (iii) the normal curvature of f (X) does not vanish of infinite order. To check that R has the desired properties, fix an arbitrary f ∈ R. We claim that k−→∞ G(Ωf ) is closed in R. Indeed, let {γk } ⊂ Lm ω,θ and let Tγk −→ T . We may assume that γk hits ∂Ω for first time at xk and choosing a subsequence, which we denote again by {xk }, we have xk → x ∈ ∂Ω for k → ∞. By using Lemma 1.2.6, we may assume that {γk } converges to some γ ∈ Lω,θ (Ωg ) and Tγ = T . If γ is a reflecting (ω, θ)-ray, then it would be ordinary by f ∈ R, and therefore for all sufficiently large k, γk would be a reflecting (ω, θ)-ray, which is a contradiction with the non-degeneracy of γ (cf. (i)). Thus, γ is an (ω, θ)-ray of mixed type, which shows that G(Ωf ) is closed in R. Now the desired properties of f follows from Theorem 9.1.2 and conditions (i)  and (ii). As an immediate consequence of the above-mentioned theorem, one gets the following. Corollary 9.2.2: Assume in addition that K = R \ Ω is a finite disjoint union of strictly convex domains Ki . Let O be the set of those f ∈ C(X) such that f (Ki ) is strictly convex for every i. Then there exists a residual subset S of O such that for every f ∈ S the relation (5.34) becomes an equality with Ω replaced by Ωf and for each γ ∈ Lω,θ (Ωf ) and t close to −Tγ we have (9.7).

9.3

Glancing ω -rays

From now till the end of this chapter we consider domains Ω in R3 . As in Section 9.2, Ω will be a connected close domain with smooth boundary ∂Ω and bounded complement. Let Ω be fixed and let Zω and πω be as in Section 2.4. A curve γ = ∪ki=0 lk in Ω consisting of linear segments li = [xi , xi+1 ], xi ∈ ∂Ω, i = 1,. . ., k, and an infinite ray l0 starting from xi with direction −ω, will be called a glancing ω-ray in Ω if it has the following properties: (i) li and li+1 satisfy the law of reflection at xi+1 with respect to ∂Ω for every i = 0, 1,. . ., k − 1; (ii) lk is tangent to ∂Ω and the normal curvature of ∂Ω at xk vanishes in direction lk . The points x1 ,. . ., xk will be called vertices of γ. Our aim in this section is to show that for generic domains Ω, for any k ≥ 1 the glancing ω-rays with k vertices in Ω form a discrete subset of a certain manifold. This fact will be applied in the next section.

254

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Let γ be a glancing ω-ray in Ω and let x1 ,. . ., xs be all different vertices of it such that xs is the last one. Then there exists a surjective ns-map (cf. Section 6.2) α : {1,. . ., k} → {1,. . ., s}

(9.9)

α(k) = s,

(9.10)

such that

and xα(1) ,. . ., xα(k) are the successive vertices of γ. If s > 1, we may assume that α(k − 1) = s − 1.

(9.11)

Fix arbitrary integers k ≥ s ≥ 1 and a surjective ns-map (9.9) with (9.10) and (9.11), the latter provided s > 1. Set X = ∂Ω. Recall that for f ∈ C(X) by Ωf we denote the unbounded domain in R3 with boundary f (X). Denote by G(ω, α) the set of those f ∈ C(X) such that there does not exist y = (y1 ,. . ., ys ) ∈ f (X)(s) such that yα(1) ,. . ., yα(k) are successive vertices of a glancing ω-ray in Ωf . Lemma 9.3.1: Let α be non-invertible, that is k > s. Then G(ω, α) contains a residual subset of C(X). Proof: We assume s > 1. The case s = 1 can be considered using some part of the following reasonings. Given i − 1,. . ., s − 1, we determine Ii (α) by (6.25). Denote by Uα the set of those y = (y1 ,. . ., ys ) ∈ (R3 )(s) such that yi does not belong to the convex hull of the set {yj : j ∈ Ii (α)} for every i = 1,. . ., s − 1. It is convenient to set y0 = πω (y1 ).

(9.12)

Define F : Uα → R by F (y) =

s−1  

  yα(i) − yα(i+1)  . i=0

Let γ be a glancing ω-ray of type α in Ωf , f ∈ C(X), that is there exists an ordering y1 ,. . ., ys of different vertices of γ such that yα(1) , . . . yα(k) are the successive vertices of γ. Then t = (y1 ,. . ., ys ) ∈ Uα . Moreover, for x = (x1 ,. . ., xs ), f (xi ) = yi , x = (x1 ,. . ., xs−1 ), we have gradx F (f s (x)) = 0 and ys − ys−1 , ν(ys ) = 0 (the fact that the normal curvature of f (X) at yi vanishes in direction ys − ys−1 is not needed here). Assuming k > s, we use almost the same argument as in the proof of Theorem 6.2.3. After the above preparation, the details are rather standard and we leave them  to the reader.

SINGULARITIES OF THE SCATTERING KERNEL

255

In view of Lemma 9.3.1, we can restrict our attention to glancing ω-rays that pass only once through each of their vertices. Fix k ∈ N and denote by D(ω, k) the set of those f ∈ C(X) such that the elements y = (y1 ,. . ., yk ) of f (X)(k) for which y1 ,. . ., yk are the successive vertices of a glancing ω-ray in Ωf form a discrete subset of f (X)(k) . Lemma 9.3.2: D(ω, k) contains a residual subset of C(X). Proof: We assume again k > 1, the case k = 1 can be proved by using a part of the following argument. / [yi−1 , yi+1 ] Let Uk be the set of those y = (y1 ,. . ., yk ) ∈ (R3 )(k) such that yi ∈ for every i = 1,. . ., s − 1. Define H : Uk → R by H(y) =

k−1

||yi − yi+1 ||.

i=0

As earlier, if y1 ,. . ., yk are the successive vertices of a glancing ω-ray in Ωf , then y = (y1 ,. . ., yk ) ∈ Uk , and for x = (x1 ,. . ., xk ), f (xi ) = yi , x = (x1 ,. . ., xk−1 ) we have gradx H(f (k (x)) = 0, yk − yk−1 , ν(yk ) = 0, and the normal curvature of Y = f (X) at yk vanishes in direction ω = (yk − yk−1 )/||yk − yk−1 ||. The later condition can be expressed analytically as follows. Let r : V → Y be a smooth chart, V being an open neighbourhood of 0 in R2 and r(V ) an open neighbourhood of yk in Y , r(0) = yk . Writing the standard coordinates in V by v = (v1 , v2 ), we have w=λ

∂r ∂r (0) + μ (0) ∂v1 ∂v2

for real λ, μ. Recall the coefficients of the second fundamental form Y at yk :   2   2   2 ∂ r ∂ r ∂ r (0), ν(yk ) , M = (0), ν(yk ) , N = (0), ν(yk ) . L= ∂v12 ∂v1 ∂v1 ∂v12 Then the fact that the normal curvature of Y at yk is zero in direction w is equivalent to (9.13) Lλ2 + 2M λμ + N μ2 = 0. Indeed,    ∂r  ∂r (EG − F 2 ) , −F w, G ∂v1 ∂v2     ∂r ∂r  2 (EG − F ) , μ = w, E −F ∂v2 ∂v1 

λ=

256

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

where E = ||∂r/∂v1 ||2 , F = ∂r/∂v1 , ∂r/∂v2 , G = ||∂r/∂v2 ||2 are the coefficients of the first fundamental form. Therefore, (9.14) is equivalent to       ∂r ∂r ∂r ∂r + 2M w, G L w, G −F −F ∂v1 ∂v2 ∂v1 ∂v2    2   ∂r ∂r ∂r ∂r + N w, E −F −F = 0. × w, E ∂v2 ∂v1 ∂v2 ∂v1 Next, we proceed as in the proofs of Theorems 6.3.1 and 6.4.1. To do this, we need the bundle J 2 (X, R3 ) of 2-jets. Let M be the set of those τ = (j 2 f1 (x1 ),. . ., j 2 fk (xk )) ∈ Jk2 (X, R3 ) such that (x1 ,. . ., xk ) ∈ X (k) , (f1 (x1 ),. . ., fk (xk )) ∈ Uk , rank dfi (xi ) = 2 for all i = 1,. . ., k. Clearly, M is an open submanifold of Jk2 (X, R3 ). The singular set Σ is now defined as the set of all τ ∈ M such that gradx F ◦ (f1 × · · · × fk )(x) = 0, fk (xk ) − fk1 (xk−1 ), ν = 0, where x = (x1 ,. . ., xk ), ν is a non-zero normal vector to fk (X) at fk (xk ), and the normal curvature of fk (X) at fK (xk ) vanishes in direction w = fk (xk ) − fk−1 (xk−1 ). We shall show that Σ is a smooth manifold of M of codimension 2k. Fix coordinate neighbourhoods Vi , i = 1,. . ., k with Vi ∩ Vj = ∅ whenever i = j and set 



k

V =M∩

J (Vi , R ) . 2

3

i=1

Consider arbitrary charts ϕi : Wi → Vi , Wi ⊂ R2 and define the chart ϕ : V → (R2 )(k) × (R2 )(k) × (R2 )(2k) × (R2 )(4k) by ϕ(j 2 f1 (x1 ),. . ., j 2 fk (xk )) = (u; v; a; b), where u = (u1 ,. . ., u2 ), v = (v1 ,. . ., vk ), (t)

(t)

a = (aij ), b = (bkjl ), ϕi (ui ) = xi , vi = f (xi ), (t)

(t)

aij =

∂(fi ◦ ϕi ) (j)

∂ui

(t)

(t)

(ui ), bkjl =

∂ 2 (fi ◦ ϕi ) (j)

(l)

∂ui ∂ui

(ui )

for i = 1,. . ., k, j, l = 1, 2, t = 1, 2, 3. As before, we use the (1) (2) (1) (2) (3) (1) (2) (3) ui = (ui , ui ) ∈ R2 , vi = (v1 , vi , vi ) ∈ R3 , fi = (fi , fi , fi ).

(9.14) notation

SINGULARITIES OF THE SCATTERING KERNEL

257

In what follows we write the elements of ϕ(v) in the form ξ = (u; v; a; b), where u, v, a, b are determined by (9.14). For such a ξ set ⎛

e1

⎜ (1) ν(ξ) = det ⎜ ⎝ ak1 (1)

ak2

e2

e3

(2)

ak1

(2)

ak2

⎞

(3) ⎟ ak1 ⎟ ⎠, (3)

ak2

where e1 = (1, 0, 0), e2 = (0, 1, 0), e3 = (0, 0, 1). Then ν(ξ) = 0 is a normal vector to fk (X) at fk (xk ). Our aim is to show that ϕ(V ∩ Σ) is a smooth manifold of ϕ(V ) of codimension 2k. For ξ ∈ ϕ(V ) define E(ξ) = ||ak1 ||2 , F (ξ) = ak1 , ak2 , G(ξ) = ||ak2 ||2 , L(ξ) = bk11 , ν(ξ) , M (ξ) = bk12 , ν(ξ) , N (ξ) = bk22 , ν(ξ) , λ(ξ) = vk − vk−1 , G(ξ)ak1 − F (ξ)ak2 , μ(ξ) = vk − vk−1 , E(ξ)ak2 − F (ξ)ak1 , where akj (t) (t) and bijl are the vectors in R3 with components aij and bijl , respectively. For m = 1, 2, 3 set Om = {ξ ∈ ϕ(V ) : ν (m) (ξ) = 0}. Clearly, Om is an open subset of ϕ(V ) that covers ϕ(V ). Thus, it is sufficient to show that Om ∩ ϕ(V ∩ Σ) is a smooth submanifold of Om of codimension 2k. We shall do this for m = k − 1, the other cases are the same. Consider the map K : O1 → R2k , defined by K(ξ) = ((dp,q )(ξ))p=1,...,k−1;q=1,2 ; (Ki (ξ))i=1,2 ), where dpq (ξ) =

3

∂H (l) t=1 ∂yp

(v)a(l) pq , K(ξ) = vk − vk−1 , ν(ξ) ,

K2 (ξ) = L(ξ)λ(ξ)2 + 2M (ξ)λ(ξ)μ(ξ) + N (ξ)μ(ξ)2 . The map K is so defined that O1 ∩ ϕ(V ∩ Σ) = K −1 (0). Therefore, it is sufficient to establish that K is a submersion on O1 ∩ ϕ(V ∩ Σ). Fix an arbitrary ξ ∈ O1 ∩ ϕ(V ∩ Σ) and assume that 2 k−1

p=1 q=1

Dpq grad dpq (ξ) +

2

Ai grad K(ξ) = 0

(9.15)

i=1

for some real constants Dpq and Ai . Since ξ = (u; v; a; b) ∈ ϕ(V ) and V ⊂ M, we have vk − vk−1 = 0. It follows by the definitions of ν(ξ), λ(ξ), μ(ξ) and K1 (ξ) = 0 that vk − vk−1 = C(λ(ξ)ak1 + μ(ξ)ak2 )

258

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

with some coefficient C = 0. Consequently, either λ(ξ) = 0 or μ(ξ) = 0. Let λ(ξ) = 0, the other case is similar. Consider in (9.16) the derivatives with respect to (1) bkl1 . The only non-zero derivative is ∂K2 (1) ∂bkl1

= λ(ξ)2 ν (1) (ξ) = 0

because ξ ∈ O1 . Hence A2 = 0. Next, considering the derivatives with respect to (t) apq , as in the proofs of Theorems 6.3.1 and 6.4.1, we obtain Dpq = 0 for all p, q. Now A1 = 0 follows trivially and K is a submersion at ξ. In this way we established that Σ is a smooth submanifold of M of codimen is equivalent to the fact that sion 2k. Then for f ∈ G(ω, α) the condition jk2 f Σ {x ∈ X (k) : jk2 f (x) ∈ Σ} is a discrete subset of X (k) , that is f ∈ D(ω, k). According to the Multijet Transversality Theorem, we see that D(ω, k) contains a residual  subset of C(X).

9.4

Generic domains in R3

Let Ω be an arbitrary domain with smooth compact boundary X = ∂Ω and bounded complement in R3 , and let ω = θ be two unit vectors in R3 . Our aim in this section is to establish the following result. Theorem 9.4.1: There exists a residual subset R of C(X) such that for every f ∈ R there are no (ω, θ)-rays of mixed type in Ωf . As an immediate consequence of this theorem and the results in the previous sections, we obtain the following. Corollary 9.4.2: The generic connected domains Ω in R3 with smooth boundaries and bounded complements have the following properties: (a) every (ω, θ)-ray in Ω is an ordinary non-degenerate reflecting (ω, θ)-ray; (b) Tγ = Tδ for every two different γ, δ ∈ Lω,θ (Ω); (c) the relation (5.30) becomes an equality; (d) for every γ ∈ Lω,θ (Ω) and every t close to −Tγ , we have (9.7). The rest of this section is devoted to the proof of Theorem 9.4.1. We begin with a technical lemma. Lemma 9.4.3: Let X be a smooth surface in R3 and let c : [a, b] −→ X, b > a, be a geodesic on X. Let t0 ∈ (a, b) be such that c(t0 ) is not a point of self-intersection of c. For every sufficiently small interval (α, β) containing t0 and every sufficiently small open neighbourhood U of c(t0 ) in X with c(α, β) ⊂ U, U ∩ Im c = c([α, β]),

(9.16)

SINGULARITIES OF THE SCATTERING KERNEL

259

there exists f ∈ C(X), arbitrarily close to id with respect to the C ∞ topology such ˜ = f (X) is the geodesic on that f (x) = x for all x ∈ X \ U , and if c˜ : [a, b] −→ X ˜ X with c˜(t) = c(t) for t ∈ [a, α], then c˜((α, β]) ∩ c((α, β]) = ∅.

(9.17)

Proof: We take U and (α, β) so small that there exist local coordinates x0 , x1 in U , determined by a chart r : V = (α, β) × (−δ, δ) → U ⊂ X with δ > 0, such that the components gij of the standard metric on X have the form g00 (x) = 1, g01 (x) = 0, g11 (x) = G(x) > 0 for all x = (x0 , x1 ) ∈ V . Moreover, we may assume that G(x) < 1 for every x ∈ V.

(9.18)

Otherwise we can replace r by another chart r˜ : V → X, given by r˜(x0 , x1 ) = r(x0 , x1 ). If  > 0 is chosen sufficiently small, then g˜11 (x0 , x1 ) = 2 g11 (x0 , x1 ) < 1 for all x ∈ V . Clearly, (9.18) holds for sufficiently small intervals (α, β) and δ > 0. Next, we assume that α, β and δ are fixed with these properties. Finally, mention that r(t, 0) = c(t) for all t ∈ [α, β]. Fix two smooth functions λ, μ : R → [0, 1] such that supp λ = [α, β], p = p(0) > 0, q = μ (0) > 0.

(9.19)

For  > 0 define f (y) = y for y ∈ X \ U , and f (y) = r(x) + λ(x0 )μ(x1 )

∂r (x) ∂x0

for y = r(x) ∈ U, x = (x0 , x1 ) ∈ V . It is easy to see that for all sufficiently small  we have f ∈ C(X), therefore X = f (X) is a smooth surface. Moreover, ψ(x) = r(x) + λ(x0 )μ(x1 )

∂r (x) ∂x0

determines a chart ψ : V → ψ(V ) ⊂ X . Denote by gij (; x) the components of the standard metric on ψ(V ) ⊂ X . We have g00 (; x) = 1 + 2λ (x0 )μ(x1 ) + O(2 ), g01 (; x) = λ(x0 )μ (x1 ) + O(2 )   ∂r ∂2r (x), (x) + O(2 ), g11 (; x) = G(x0 , x1 ) + 2λ(x0 )μ(x1 ) ∂x1 ∂x0 ∂x1 as  → 0.

260

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Using the coordinates x0 , x1 , introduce the canonical coordinates x0 , x1 , y0 , y1 in T ∗ (X ), and consider the Hamiltonian vector field generated by the Hamiltonian H(; x, y) =

1 1 g00 (; x)y02 + g01 (; x)y0 y1 + g11 (; x)y12 , 2 2

where x = x0 , x1 , y = (y0 , y1 ). Denote by c(; x) the geodesic on X such that c(; x) = c(t) for t ∈ [a, α], and let (x(; t), y(; t) be the corresponding integral curve on T ∗ (X ). Writing the Hamiltonian equations for this curve and then the corresponding variational equations for Xi (t) =

d d xi (; x)|=0 , Yi (t) = yi (; t)|=0 , dt dt

according to (9.19), we obtain ⎧ ⎪ X0 (t) = Y0 (t) + 2pλ (t), ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ X  (t) = G(t, 0)Y1 (t) + qλ(t), ⎪ ⎨ 1 Y0 (t) = −pλ (t), ⎪ ⎪ ⎪ ⎪ Y1 (t) = −qλ (t), ⎪ ⎪ ⎪ ⎪ ⎩X (α) = X (α) = Y (α) = Y (α) = 0, 0 1 0 1 for t ∈ [α, β]. Consequently, Y1 (t) = −qλ(t), and so X1 (t) = qλ(t)(1 − G(t, 0)). This and (9.18) imply X1 (t) > 0 for all t ∈ (α, β). Therefore, for all sufficiently small  > 0 we have d x (; x) > 0, t ∈ (α, β). dt 1 Fix such an . Then x1 (; t) > 0 for all t ∈ (α, β), and f = f has the desired  properties. This proves the assertion. It follows by Lemma 8.1.2 that there exists a residual subset K of C(X) such that for f ∈ K, the normal curvature of f (X) can vanish only of finite order. Then by the properties of the generalized bicharacteristics of  (cf. Section 1.2), we have that if f ∈ K and γ : R → Ωf is an (ω, θ)-ray in Ωf , then Im γ is a finite union of linear segments (two of them are infinite) and geodesic segments on ∂Ω. The ends of these (linear or geodesic) segments will be called vertices of γ (or of Im γ). Proof of Theorem 9.4.1: We are going to construct by induction a sequence V1 ⊃ V2 ⊃ . . . ⊃ Vk ⊃ . . . of residual subsets of K such that for every k and every f ∈ Vk there are no (ω, θ)-rays of mixed type in Ωf with not more than k + 1 different (linear or geodesic) segments.

SINGULARITIES OF THE SCATTERING KERNEL

261

Set V1 = K. Clearly, for every f ∈ V1 there are no (ω, θ)-rays of mixed type in Ωf having exactly two segments and therefore only one vertex. Let k > 1 and assume that V1 ⊃ · · · ⊃ Vk−1 are already constructed and have the desired properties. To construct Vk , we need some technical preparation. A map of the form κ : {1, 2,. . ., k} → {0, 1} will be called k-design, if κ is not identically zero, κ(k) = 0 and κ(i)κ(i + 1) = 0 for all i = 1,. . ., k − 2. An (ω, θ)-ray γ will be called (ω, θ)-ray with design κ if Im γ has k + 1 segments l0 , l1 ,. . ., lk and for every i = 1,. . ., k − 1, li is a linear segment if and only if κ(i) = 0. Fix an arbitrary k-design κ and set q = max{i : 1 ≤ i ≤ k − 1, κ(i) = 1}, p = min{i : 1 ≤ i ≤ k − 1, κ(i) = 1}. It follows from Lemma 9.3.2 that for every η ∈ S2 and every m ∈ N there exists a residual subset D(η, m) of C(X) such that for f ∈ D(η, m) the elements y = (y1 ,. . ., ym ) of f (X)(m) such that y1 ,. . ., ym are the successive vertices of a glancing η-ray in Ωf form a discrete subset of f (X)(m) . On the other hand, Theorem 6.3.3 and Corollary 6.4.7 imply the existence of a residual subset Tk of C(X) such that for every f ∈ Tk there are only finitely many reflecting (ω, θ)-rays in Ωf with not more than k reflection points, and all of them are ordinary. We set W = Vk−1 ∩ D(ω, p) ∩ D(−θ, q) ∩ Tk . Then W is a residual subset of C(X) which is clearly contained in K. Denote by V(k, κ) the set of those f ∈ W such that there are no (ω, θ)-rays of mixed type with design κ in Ωf . We will first show that V(k, κ) is dense in W. To this end, we may assume that id ∈ W and then prove that there are f ∈ V(k, κ) arbitrarily close to id. Suppose id ∈ W. We claim that there exist only finitely many (ω, θ)-rays of mixed type with design κ in Ω. Assume the contrary, and let γ1 ,. . ., γm , . . . be an infinite sequence of (m) (m) different (ω, θ)-rays of mixed type with design κ in Ω. Denote by x1 ,. . ., xk the (m) successive vertices of Im γm . We may assume that for all m, one has γm (0) = x1 . and γ m (t) = ω for t < 0. Moreover, considering an appropriate subsequence, we may assume that there exists (m)

lim xi

m→∞

= xi ∈ ∂Ω

for all i = 1,. . ., k. It then follows by the continuity of the broken Hamiltonian flow that for every t ∈ R there exists lim γm (t) = γ(t) ∈ Ω,

m→∞

262

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

and γ is an (ω, θ)-ray in Ω. Roughly speaking, the ith finite segment of Im γ has end points xi and xi+1 . However, this is not precise, since in general some of these (m) (m) segments can vanish. Denote by l0 ,. . ., lk the successive segments of Im γm . Set (m)

li = lim li m→∞

, i = 0, 1,. . ., k.

Then each li is either the one-point set {xi }, or a linear segment, or a geodesic segment on ∂Ω. Clearly, the first segments l0 is the infinite ray starting from x1 and having direction −ω. Let s + 1 be the number of different segments of γ, then s ≤ k. There are two cases. Case 1. s = k. Then each li is a non-degenerate linear or geodesic segment, and clearly γ is an (ω, θ)-ray of mixed type with design κ in Ω. Consequently, x1 ,. . ., xp are the successive vertices of a glancing ω-ray in Ω. Moreover, for every (m) (m) m ∈ N, x1 ,. . ., xp are the successive vertices of a glancing ω-ray in Ω. Since (m) limm→∞ xi = xi for every i, this is a contradiction with id ∈ W ⊂ D(ω, p). Case 2. s < k. Then s ≤ k − 1, and id ∈ W ⊂ Vk−1 implies that γ cannot be an (ω, θ)-ray of mixed type. Therefore, li is one point for all i with κ(i) = 1, and γ is a reflecting (ω, θ)-ray with s reflection points. Since li vanishes for at least one i, some segment of γ is tangent to ∂Ω, which is a contradiction with id ∈ W ⊂ Tk . In both possible cases we got a contradiction. This shows that there are only finitely many (ω, θ)-rays of mixed type with design κ in Ω. Let γ, γ1 ,. . ., γs be all of them. Denote by x1 ,. . ., xk the successive vertices of Im γ and by l0 , l1 ,. . ., lk its successive segments. Since id ∈ D(−θ, q), there exists a neighbourhood V of xk in X such that V ∩ Im γi = ∅ for all i = 1,. . ., s, and if δ is a glancing (ω, θ)-ray in Ω with k − q vertices and the first y1 of them belongs to V , then y1 = xk (and therefore δ is a part of Im γ). Let 0 = t1 < · · · , tk be the times with γ(ti ) = xi . Since lq is a geodesic segment on X, we can find t0 ∈ (α, β) ⊂ (tq , tq+1 ) and a small coordinate neighbourhood U of γ(t0 ) in X such that 

U ∩ (∪si=1 Im γi ) ∪ V¯ ∪ ∪kj=0,j=q lj = ∅, and (9.16) holds for c(t) = γ(t), a = tq , b = tq+q , α, β. By Lemma 9.4.1, there exists f ∈ C(X), arbitrary close to id, such that f = id on X \ U and (9.18) is satisfied for ˜ = f (X) with c˜(t) = c(t) for t ∈ [t , α]. We claim that if f is the geodesic c˜ on X q chosen in this way and it is sufficiently close to id, then the only (ω, θ)-rays of mixed type with design κ in Ωf are γ1 ,. . ., γs . Indeed, assume this is not true. Then we can find a sequence fm −→m id of such fm so that for every m there exists an (ω, θ)-ray of mixed type δm with design κ in Ωfm , different from γ1 ,. . ., γs . Since id ∈ D(ω, p), for large m, the first vertex of Im δm is necessarily x1 . We may assume δm (0) = x1 , and then we obtain δm (t) = γ(t) for all t ≤ α. Now the construction of fm shows that (m) the last vertex yk of δm cannot be xk . Otherwise, we would have δm (β) = γ(β), (m) which would imply c˜(β) = c(β) in contradiction with (9.17). So yk = xk , and the (m) choice of V yields tk ∈ / V.

SINGULARITIES OF THE SCATTERING KERNEL

263

Considering appropriate convergence subsequences, we may assume that there exists δ(t) = limm→∞ δm (t) for every t. Then δ is clearly an (ω, θ)-ray in Ω with first reflection point of δ x1 , therefore δ coincides with γ. On the other hand, the (m) last reflection point of δ is yk = limm yk ∈ / V , so δ cannot coincide with γ. This contradiction shows that γ1 ,. . ., γs are the only (ω, θ)-rays of mixed type with design κ in Ωf , provided f is constructed as above and is sufficiently close to id. Moreover, if f is sufficiently close to id, then f ∈ W. Repeating this procedure s times, we find g ∈ W, arbitrarily close to id, such that there are no (ω, θ)-rays of mixed type with design κ in Ωg . Then g ∈ V(k, κ), which shows that V(k, κ) is dense in W. To establish that V(k, κ) is open in W, we may assume that fm →m id ∈ W for some sequence {fm } ⊂ W \ V(k, κ). Then we have to prove that there exists an (ω, θ)-ray of mixed type with design κ in Ω. For every m there exists an (ω, θ)-ray of mixed type δm with design κ in Ωfm . Choosing again appropriate subsequence and repeating a part of the above argument, we see that there exists an (ω, θ)-ray of mixed type in Ω. Thus, V(k, κ) is open in W. Set Vk = ∩k V(k, κ), where κ runs over the set of all k-designs. Since the later is finite, Vk is open and dense in W, so it is residual in C(X). Clearly, Vk has all desired properties. This completes the construction of the sequence {Vk }. Finally, define V = ∩∞ k=1 Vk . Then V is a residual subset of C(X), and for every f ∈ V there are no (ω, θ)-rays of mixed type in Ωf . This concludes the proof of the  theorem.

9.5 Notes Under more restrictive assumptions on γ and Ω, Theorem 9.1.2 was proved in [[Pl]]. The case of strictly convex obstacles (cf. Corollary 9.1.3) was studied sometime before by Majda [[Mal]] (see also [[Ma2], [MaT], [So2], [Y]]). In Section 9.1, we follow the analysis in [[Pl]] based on the construction of a global parametrix in [[GM1]]. Lemmas 4.2.1 and 4.2.2 make possible to precise the form of the coefficient in (9.7). The results of Section 9.2 were proved in [[CPS]]. The material in Sections 9.3 and 9.4 is taken from [[S4]]. Note that to prove a result similar to Corollary 9.4.2 for domains in Rn , n > 3, it is desirable to have an analogue of Theorem 9.4.1 for such domains. Except Lemma 9.3.2, all arguments in Sections 9.3 and 9.4 can be modified to cover the general case. We do not know whether their analogues are true for n > 3 for either Lemma 9.3.2 or Theorem 9.4.1.

10

Scattering invariants for several strictly convex domains In this chapter we study scattering rays and related singularities of the scattering kernel in the exterior Ω of an obstacle K which is a disjoint union of a finite number of strictly convex compact domains in Rn . It is shown first that if ω ∈ Sn−1 is fixed, then for most of the vectors θ ∈ Sn−1 all (ω, θ)-rays γ are ordinary with distinct sojourn times providing singularities of the scattering kernel s(t, θ, ω). Starting from the second section, we assume that the convex hull of every two connected components of K does not contain points of any other connected component of K. Under this condition, we prove that for fixed configurations α of connected components of K, one can choose appropriate ω and θ so that for every integer q ≤ 1 there exists a unique (ω, θ)-ray γq with q reflection points following the given configurations in a certain way. It turns out that the reflection points of these rays approximate the corresponding reflection points of the unique periodic reflecting ray γα in Ω having type α. Moreover, the period of γα is an invariant which can be determined from the asymptotic of the sojourn times Tγ as q → ∞. Another geometric invariant, related to the Poincaré map of γα , can be determined by the asymptotic of the coefficients cq in front of the main singularity of s(t, θ, ω) for t ∼ −Tγq .

10.1

Singularities of the scattering kernel for generic θ

Let K ⊂ Rn , n ≥ 2, have the form K = K1 ∪ · · · ∪ K s ,

Geometry of the Generalized Geodesic Flow and Inverse Spectral Problems, Second Edition. Vesselin M. Petkov and Luchezar N. Stoyanov. © 2017 John Wiley & Sons, Ltd. Published 2017 by John Wiley & Sons, Ltd.

(10.1)

SCATTERING INVARIANTS FOR SEVERAL STRICTLY CONVEX DOMAINS

265

where Ki are strictly convex compact domains in Rn with smooth boundaries Γi = ∂Ki . As in Chapters 5 and 9, we shall use the notation Ω = Rn \ K. Fix arbitrary vector, ω ∈ Sn−1 . Our aim in this section is to prove the following. Theorem 10.1.1: There exists a residual subset R(ω) of Sn−1 such that for every θ ∈ R(ω) one has the following properties: (a) each (ω, θ)-ray in Ω is ordinary and every two different (ω, θ)-rays in Ω have distinct sojourn times; (b) if n is odd we have sing supp sΩ (t, θ, ω) = { − Tγ : γ ∈ Lω,θ (Ω)}, and for every γ ∈ Lω,θ (Ω) and t close to −Tγ , we have (9.7). Notice that property (b) follows immediately from (a) and Theorems 5.3.2 and 9.1.2. Thus, we have to construct R(ω) in a such way that (a) is satisfied. Let us recall some notation from Section 2.4, which will be used in the whole chapter. Given u ∈ Zω , denote by St (u) the shift of u after time t along the billiard semi-trajectory γ(u) in Ω, starting at u in direction ω. Then γ(u) = {St (u) : t ≥ 0}. Let Nt (u) be the velocity vector (i.e. the direction) of γ(u) at St (u). If t is such that x = St (u) ∈ ∂Ω, we shall use the notation N−t (u) = lim Nt− (u),  0

and N+t (u) = σx (N−t (u)), σx being the symmetry with respect to the tangent hyperplane Tx (∂Ω). Note that S0 (u) = u and N0 (u) = ω. By x1 (u), x2 (u), . . . , we denote the successive reflections points of γ if any, and by t1 (u), t2 (u), . . . the corresponding times (moments) of reflection. It is convenient to set x0 (u) = u and t0 (u) = 0. Finally, denote by r(u) the number of reflections of γ(u), 0 ≤ r(u) ≤ ∞. (0) Furthermore, let Zω be the set of those u ∈ Zω such that r(u) < ∞. Define (0) (0) the map T : Zω → R as follows. Given u ∈ Zω , the vector Nt (u) is constant for sufficiently large t, so there exists η = limt→+∞ Nt (u). Clearly, there is exactly one t > 0 such that St (u) ∈ Z−η . We set T (u) = t. Let us mention that under the latter notation, if γ is the unique (ω, θ)-ray in Ω, passing through u, then Tγ = T (u) + 2a. Recall the notion of a configuration from Section 2.4. Given an integer m ≥ 1, by a configuration of length m we mean a sequence α = (i1 ,. . ., im ) ∈ {1, 2,. . ., s}m

(10.2)

such that ij = ij+1 for j = 1,. . ., m − 1. For such an α we set |α| = m and define the sets Uα ⊂ Fα ⊂ Zω and the map Jα : F¯α → R as in Section 2.4.

266

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Lemma 10.1.2: (a) There exists a sequence L1 ⊃ L2 ⊃ · · · ⊃ Lm ⊃ · · · of open dense subsets of Sn−1 such that for every m ∈ N, if α is a configuration of length m, u ∈ Fα and Jα (u) ∈ Lm , then u ∈ Uα ; (b) For every two configurations α and β there exists a residual subset L(α, β) of Sn−1 such that for θ ∈ L(α, β) the conditions u ∈ Uα , v ∈ Uβ , r(u) = |α|, r(v) = |β|, Jα (u) = Jβ (v) = θ,

(10.3)

imply T (u) = T (v). For the proof of this lemma we need a general fact concerning the propagation of convex wave fronts in Ω. Let X be a smooth hypersurface lying in the interior of Ω and let e(x), x ∈ X, be a continuous field of unit normal vectors to X. We assume that X is convex at every x ∈ X with respect to the normal field e(x), that is the corresponding second fundamental form of X is non-negative semi-definite everywhere in X. Consider an open coordinate neighbourhood V of some point x0 in X and a smooth chart U u → x(u) ∈ V ⊂ X of an open neighbourhood U of 0 in Rn−1 onto V with x0 = x(0). Set e(u) = e(x(u)). With respect to this parameterization, the second fundamental form II(X) u of X at u (i.e. at x(u)) is given by II(X) u (ξ, η) =

n−1 

(X)

bij (u)ξi ηj ,

i,j=1

where

 (X)

bij (u) =

e(u),

 ∂2x (u) , i, j = 1,. . ., n − 1 ∂ui ∂uj (X)

are the coefficients of II(X) at u. The first fundamental form Iu u by n−1  (X) (ξ, η) = gij (u)ξi ξj , I(X) u

of X at u is given

i,j=1



with (X) gij (u)

=

 ∂x ∂x (u), (u) , ∂ui ∂uj

i, j = 1,. . ., n − 1.

SCATTERING INVARIANTS FOR SEVERAL STRICTLY CONVEX DOMAINS

267

The sectional normal curvature of X at x(u) in direction ξ (more precisely, in direc ∂x tion n−1 i=1 ξi ∂ui (u) ∈ Tx(u) X) is defined by κ(X) u (ξ) = −

II(X) u (ξ, ξ) (X)

.

Iu (ξ, ξ)

For x ∈ X and t ∈ R let Qt (x) be the shift of x along the billiard semi-trajectory χ(x) in Ω starting at x in direction e(x) and let Mt (x) be the direction of the trajectory at Qt (x). Then χ(x) = {Qt (x) : t ≥ 0}. Assume that the trajectory χ(x0 ) hits ∂Ω = ∂K at some point y0 , reflecting transversally on ∂K at y0 . Then for the angle ϕ0 between ν(y0 ) and the direction Nt0 (x0 ) at γ(x0 ) at y0 , we have cos ϕ0 > 0. Here t0 = ||x0 − y0 || > 0. Furthermore, take T > t0 such that Qt (x0 ) makes only one reflection when t runs from 0 to T . Restricting our considerations to a small neighbourhood of x0 in X, we may assume that for every x ∈ X the trajectory Qt (x) makes exactly one transversal reflection on ∂K when t runs from 0 to T . Then Z = Z (T ) = {QT (x) : x ∈ X} is a smooth hypersurface in Rn . This follows easily, for example, from the next considerations and the implicit function theorem. Set z0 = QT (x(0)), and denote by κ(T ) the minimum of the sectional curvatures of Z at z0 and by κ0 the same for ∂K at y0 . Finally, set κ+ (t0 ) = lim κ(T ). T t0

Before going on, let us mention that for 0 < t < t0 , we have (X)

κ(X) u (ξ) =

κu (ξ) (X)

.

1 + tκu (ξ)

This formula can be easily derived from a part of the argument in the proof of the following lemma. The latter shows that the wave front Z = Z (T ) is always strictly convex and gives an estimate for its minimal normal curvature. Actually, we need only the strict convexity, the estimate of the curvature will not be used in our next analysis. Lemma 10.1.3: Under the above assumptions, the hypersurface Z is strictly convex at z0 and 2κ0 cos ϕ0 . (10.4) κ(T ) ≥ 1 + 2(T − t0 )κ0 cos ϕ0 In particular, κ+ (t0 ) ≥ 2κ0 cos ϕ0 .

(10.5)

268

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Proof: Note that (10.4) follows from (10.5) and the above result. Thus, it is sufficient to prove only (10.5). For u ∈ U denote by y(u) the reflection point of χ(x(u)) on ∂K, and set Y = ∂K, z(u) = z (T ) (u) = ST (x(u)), f (u) =

z(u) − y(u) . ||z(u) − y(u)||

We shall see later that f (u) is the unit normal to Z at z(u). It follows by our assumptions that u → y(u), u → z(u) provide smooth parameterization for ∂K around y0 and for Z around z0 . Setting t(u) = ||y(u) − x(u)||, we have y(u) = x(u) + t(u)e(u)

(10.6)

z(u) = y(u) + (T − t(u))f (u).

(10.7)

and

Differentiating (10.6) and (10.7) with respect to ui , i = 1,. . ., n − 1, one finds ∂x ∂t ∂e ∂y = + e+t , ∂ui ∂ui ∂ui ∂ui

(10.8)

∂z ∂y ∂t ∂f = − f + (T − t(u)) . ∂ui ∂ui ∂ui ∂ui

(10.9)

Taking the inner product of (10.8) with e(u), we get   ∂y ∂t = e(u), (u) . ∂ui ∂ui Then (10.9) implies ∂y ∂z (u) = (u) − ∂ui ∂ui



 ∂y (u), e(u) f (u) + O(T − t0 ), ∂ui

(10.10)

where O(T − t0 ) denotes a term that tends to 0 as T t0 . Notice that f (u) does not depend on T , provided T − t0 > 0 is sufficiently small. By (10.10), we first obtain       ∂z ∂y ∂y (u), f (u) = (u), f (u) − (u), e(u) = 0, ∂ui ∂ui ∂ui since e(u) and f (u) are symmetric with respect to the tangent hyperplane to ∂K at y(u). This shows that f (u) is the unit normal to Z at z(u). Next, (10.10) implies    ∂y ∂y (Z) (Y ) (u) e(u), (u) . gij (u) = gij (u) − e(u), ∂ui ∂uj

SCATTERING INVARIANTS FOR SEVERAL STRICTLY CONVEX DOMAINS

269

Consider an arbitrary ξ = (ξ1 ,. . ., ξn−1 ) ∈ Rn−1 and set η=

n−1  i=1

ξi

∂y (u) ∈ Ty(u) (∂K). ∂ui

Then I(Z) u (ξ, ξ) =

n−1 

(Z)

) 2 ξi ξj gij (u) = I(Y u (ξ, ξ) − e(u), η + O(T − y0 )

i,j=1 ) ≤ I(Y u (ξ, ξ) + O(T − t0 ).

(10.11)

We are going to deduce similar relations between the second fundamental forms. Differentiating (10.8) and (10.9) with respect to uj , we get ∂2y ∂2x ∂t ∂t ∂e ∂2e = + e+ +t , ∂ui ∂uj ∂ui ∂uj ∂ui ∂uj ∂ui ∂uj ∂ui ∂uj

(10.12)

∂2z ∂2y ∂2t ∂t ∂f ∂t ∂f ∂2f = − f− − + (T − t(u)) . ∂ui ∂uj ∂ui ∂uj ∂ui ∂uj ∂ui ∂uj ∂uj ∂ui ∂ui ∂uj (10.13) To find the term ∂ 2 t/∂ui ∂uj , we take the inner product of (10.10) with e(u) and get  2      ∂2t ∂ y ∂2x ∂2e = ,e − ,e − t ,e . ∂ui ∂uj ∂ui ∂uj ∂ui ∂uj ∂ui ∂uj Replacing this term in (10.13), we obtain   2   2 ∂ y ∂2t ∂ z (0), e(0) = (0), f (0) − (0) + O(T − t0 ) ∂ui ∂uj ∂ui ∂uj ∂ui ∂uj   2   2 ∂ x ∂ y (0), f (0) − e(0) + (0), e(0) = ∂ui ∂uj ∂ui ∂uj   ∂2e (0), e(0) + O(T − t0 ) + t0 ∂ui ∂uj  2   2  ∂ y ∂ x (0), ν(0) + (0), e(0) = 2 cos ϕ0 ∂ui ∂uj ∂ui ∂uj   ∂e ∂e − t0 (0), (0) + O(T − t0 ). (10.14) ∂ui ∂uj Here we have taken into account that f (0) − e(0) = 2(cos ϕ0 )ν(0) and     ∂2 ∂e ∂e . ,e = − , ∂ui ∂uj ∂ui ∂uj

270

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

It follows immediately from (10.14) that for u = 0 and ζ =

n−1 i=1

∂e ξi ∂u (0) we have i

(Y ) (Y ) 2 II(Z) u (ξ, ξ) = 2 cos ϕ0 IIu (ξ, ξ) + IIu (ξ, ξ) − t0 ||ζ|| + O(T − t0 ).

The convexity of X implies II(X) ≤ 0, so u (Y ) II(Z) u (ξ, ξ) ≤ 2 cos ϕ0 IIu (ξ, ξ) + O(T − t0 ).

Finally, combining the latter with (10.11), one gets (Z)

−

II0 (ξ, ξ) (Z)

I0 (ξ, ξ)

≥−

) 2 cos ϕ0 II(Y u (ξ, ξ) + O(T − t0 ) (Y )

I0 (ξ, ξ)

≥ 2 cos ϕ0 + O(T − t0 ), 

which clearly yields (10.5).

Proof of Lemma 10.1.2: (a) Set L1 = Sn−1 \ {ω}. Assume that the sets L1 ⊃ · · · ⊃ Lm are already constructed and have the desired properties. We are going to construct Lm+1 . Fix an arbitrary configuration α = (i1 ,. . ., im , im+1 ) of length m + 1, and for k ≤ m set αk = (i1 ,. . ., ik ). Since Lm is open and dense in Sn−1 , F = S \ Lm is a compact subset of Sn−1 with empty interior. Fix for a moment an arbitrary k ≤ m and set β = αk . Next, we use the notations (2.39) and (2.40) from Section 2.4, as well as Proposition 2.4.4. First consider the map Jβ : F¯β → Eβ . Since F ∩ Eβ is a compact subset of Eβ with empty interior, and Jβ−1 (F ) = Jβ−1 (F ∩ Eβ ) ⊂ (Jβ−1 (F ∩ Eβ ) ∩ Mα ) ∪ Lα , it follows by Proposition 2.4.4 that Jβ−1 (F ) is a compact subset of Zω with empty interior. By the definition of β, we have Fα ⊂ Fβ and Lα ⊂ Lβ . On the other hand, ω ∈ / L1 ⊃ Lm , and the definitions of Jβ and F imply Jβ−1 (F ) ∩ Lβ = ∅. Consequently, Jβ−1 (F ) ∩ Lα = ∅, which shows that Jβ−1 (F )F¯α is contained in Mα . Applying again Proposition 2.4.4, we deduce that Jα (Jβ−1 (F ) ∩ F¯α ) is a compact subset of Sn−1 with empty interior. In this way, we have established that −1 V = Lm \ ∪m k=1 Jα (F ∩ Jαk (F ))

is an open dense subset of Sn−1 . Note that if u ∈ Fα and Jα (u) ∈ V , then the first m reflection points of γ(u) are proper, that is transversal ones. (α) Denote by Lm+1 the set of those θ ∈ V such that if Jα = θ for some u ∈ Fα , (α) then the first m + 1 reflection points of γ(u) are proper ones. We claim that Lm+1 is open and dense in V . The openness is clear. To establish the density, fix an arbitrary

SCATTERING INVARIANTS FOR SEVERAL STRICTLY CONVEX DOMAINS

271

(α)

θ ∈ V \ Lm+1 . Then θ = Jα (u0 ) for some u0 ∈ Fα such that the first m reflection points x1 (u0 ),. . ., xm (u0 ) of γ(u0 ) are proper ones and γ(u0 ) is tangent to ∂K at xm+1 (U0 ). Take an arbitrary t with tm (u0 ) < t < tm+1 (u0 ), and choose an open (n − 2)-dimensional ball U in Zω with centre u0 so small that for every u ∈ U the trajectory {Ss (u) : 0 ≤ s ≤ t} has exactly m reflections, all of them being proper ones. Then, according to Lemma 10.1.3, Y = St (U ) is a smooth strictly convex hypersurface in Rn (Figure 10.1). Note that {Nt (u) : u ∈ U } is a normal field for Y . It is clear now that there exists v ∈ U such that the ray, starting at St (v) in direction (α) (α) Nt (v), intersects transversally Γim+1 , which means that v ∈ Lm+1 . Therefore, Lm+1 (α) is dense in V . Since V is open and dense in Sn−1 , we deduce that Lm+1 is also open n−1 and dense in S . U

Zw

u0

xm+1(u0)

x1(u0)

Ki

1

Figure 10.1

xm(u0)

Kim

Kim+1

St(U)

Strictly convex hypersurface Si (U ).

(α)

Set Lm+1 = ∩α Lm+1 , where α runs over the finite set of all configurations of length m + 1. Then Lm+1 is open and dense in Sn−1 and has the desired properties. This concludes the proof of (a). (b) Let {Lm } be the sequence constructed in (a), and set R (Ω) = ∩∞ m=1 Lm . Then R (ω) is a residual subset of Sn−1 such that for every configuration α we have Jα−1 (R (ω)) ∩ Fα ⊂ Uα . Fix two arbitrary configurations α and β, and set m = max{|α|, |β|}. Denote by L(α, β) the set of all θ ∈ R (ω) such that (10.3) implies T (u) = T (v). We are going to show that L(α, β) is open and dense in R (ω).

272

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

First, we prove that R (ω) \ L(α, β) is closed in R (ω). Consider an arbitrary sequence R (ω) \ L(α, β) θk → θ ∈ R (ω). Then for every k there exist uk ∈ Uα ∩ Jα−1 (θk ) and vk ∈ Uβ ∩ Jβ−1 (θk ) with rk (uk ) = |α|, r(vk ) = |β|, T (uk ) = T (vk ). Taking appropriate subsequences, we / Fα . Then u ∈ Fα for may assume that uk → u ∈ F¯α , vk → v ∈ F¯β . Assume u ∈ some α , and γ(u) is tangent to ∂K at some of its points. On the other hand, Jα (u) = Jα (u) = lim Jα (uk ) = lim θk = θ ∈ R (ω), k

k

which is a contradiction with the properties of R (ω). Hence u ∈ Fα , and now Jα (u) = θ ∈ R (ω) implies u ∈ Uα . In the same way, one finds Jβ (v) = θ and v ∈ Uβ . Moreover, we have r(u) = |α|, r(v) = |β|. Thus, θ ∈ R (ω) \ L(α, β), which shows that L(α, β) is open in R (ω). To establish the density, fix an arbitrary θ ∈ R (ω) \ L(α, β) and set η = −θ. Then there exist u and v having the properties (10.3) with T (u) = T (v). Applying again Lemma 10.1.3, we find open (n − 2)-dimensional balls U  ⊂ Uα and V  ⊂ Uβ centred at u and v, respectively, such that for t = T (u) = T (v), St (U  ) and St (V  ) are smooth strictly convex hypersurfaces in Rn (Figure 10.2). Note that these hypersurfaces are tangent to Zη at St (u) and Sy (v), respectively, so these are contained in the closed half-space, determined by Zη and η. It follows by the strict convexity that Zw

γ(u) St(U´)

γ(u´)

St(V´)

U0

γ(υ)

γ(υ´)

Zη ´

A

B t1

Figure 10.2

t2

Zη

Strictly convex hypersurfaces St (U ) and St (V ).

for every θ ∈ Sn−1 close to θ, there are unique u ∈ U  and v  ∈ V  with Jα (u ) = Jβ (v  ) = θ and St (U  ) and St (V  ) have a common tangent hyperplane at St (u ) and

SCATTERING INVARIANTS FOR SEVERAL STRICTLY CONVEX DOMAINS

273

St (v  ), respectively. It is easy to see that D has empty interior in Sn−1 ; in fact, it is contained in a smooth submanifold of Sn−1 with positive codimension. Since R (ω) is dense in Sn−1 , there exists θ ∈ R (ω) \ D arbitrary close to θ. Take such a θ and denote by A (resp. B) the hyperplane tangent to St (U  ) at St (u ) (resp. tangent to St (V  ) at St (v  )). Then for η  = −θ , the three hyperplanes A, B and Zη are parallel. Setting t1 = T (u ) − t, t2 = T (v  ) − t, we see that |t1 | is the distance between A and Zη , while |t2 | is the distance between B and Zη . Since A = B, we have t1 = t2 ; therefore, T (u ) = t + t1 = t + t2 = T (v  ). Moreover, it follows by the properties of the maps Jα and Jβ that (10.3) holds if we replace u by u , v by v  and θ by θ . Thus, θ ∈ L(α, β). This proves the density of L(α, β) in R (ω). Since R (ω) is residual in Sn−1 and L(α, β) is open and dense in R (ω), we get  that L(α, β) is residual in Sn−1 , too. This concludes the proof of (b). Proof of Theorem 10.1.1: Let L(α, β) be the residual subsets of Sn−1 from Lemma 10.1.2, chosen as subsets of R (ω), the latter being defined in the proof of Lemma 10.1.2 (b). Set R(ω) = ∩α,β L(α, β), where α and β run over the set of all configurations. Then R(ω) is residual in Sn−1 and has the property (a) from Theorem 10.1.1. As we have already mentioned, the  property (b) follows immediately from (a) and Theorems 8.3.2 and 9.1.2.

10.2

Hyperbolicity of scattering trajectories

Let Ω and K be as in the previous section. From now on, we assume that K satisfies the following condition: ⎧ ⎪ ⎨for any three distinct connected components, (H) L, M, N of K the convex hull of ⎪ ⎩ L ∪ M does not contain points of N. In what follows, we use the notation from the beginning of the previous section. Recall that κ0 > 0 denotes the minimum of the normal curvatures of ∂K. Let m ≥ 2 be an arbitrary integer, and let L , L1 , L2 ,. . ., Lm , L

(10.15)

be an arbitrary sequence of connected components of K such that Li = Li+1 for all i = 1,. . ., m − 1, L = L1 , Lm = L . Note that some of these connected components may coincide. This is clearly the case when m is larger than the number of connected components of K.

274

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Before going on, let us mention that the condition (H) implies the existence of ϕ0 ∈ (0, π/2) with the following property: if x, y, z belong to the boundaries of connected components L, M and N of K, L = M, M = N , the open segments (x, y) and (y, z) have no common points with K and [x, y] and [y, z] satisfy the law of reflection at y with respect to ∂K, then ϕ < ϕ0 , where ϕ ∈ (0, π/2] is the angle between [y, z] and the normal ν(y) to ∂K at y. Furthermore, it is easy to see that there exists ψ0 ∈ (0, π/2) with the following property: for every two distinct connected components M and L of K, there is a tangent hyperplane H to M such that M and L are contained in one and a same half-space with respect to H, and the minimal angle between H and a straight line, having common point with both M and L, is greater than or equal to ϕ0 . Clearly, ψ0 depends only on K. Let Z  be a hyperplane, tangent to L and having the properties of H with respect to the pair M = L , L = L1 , and Z  be a hyperplane tangent to L and having similar properties with respect to the pair M = L , L = Lm . Denote by V the set of those x ∈ Z  such that if y ∈ ∂L1 , z ∈ ∂L2 , the open segments (x, y) and (y, z) have no common points with K \ L1 and [x, y] and [y, w] satisfy the law of reflection at y with respect to ∂K, then for the angle ϕ ∈ (0, π/2] between [y, w] and the normal ν(y) we have ϕ < ϕ0 . Clearly, V is an open subset of Z  . In a similar way we define an open subset W of Z  , this time the corresponding property concerns points x ∈ Lm1 , y ∈ Lm , z ∈ W . The central moment in this section is the following. Lemma 10.2.1: There exist constants C > 0 and δ ∈ (0, 1), depending only on K, with the following property: if   ∈ ∂Lm , ym+1 ∈ W, y0 ∈ V, y1 ∈ ∂L1 ,. . ., ym

and

  y0 ∈ V, y1 ∈ ∂L1 ,. . ., ym ∈ ∂Lm , ym+1 ∈W

 are two sequences of points such that for every j = 1,. . ., m the segments [yj−1 , yj ]    and [yj , yj+1 ] satisfy the law of reflection at yj with respect to ∂Lj and the segments   , yj ] and [yj , yj+1 ] satisfy the law of reflection at yj with respect to ∂Lj , then [yj−1

||yi − yi || ≤ C(δ i + δ m−i ) for all i = 1,. . ., m. To prove this lemma, we need some preparation. Define the function F : V × ∂L1 × · · · × ∂Lm × W → R by F (v; y1 ,. . ., ym ; w) = ||v − y1 || +

m−1  i=1

||yi − yi+1 || + ||ym − w||.

(10.16)

SCATTERING INVARIANTS FOR SEVERAL STRICTLY CONVEX DOMAINS

275

Applying a standard argument and using the choice of V and W , we see that for every v ∈ V and every w ∈ W there exist y1 (v, w) ∈ ∂L1 ,. . ., ym (v, w) ∈ ∂Lm such that F (v; y1 (v, w),. . ., ym (v, w); w) = min{F (v; y1 ,. . ., ym ; w) : (y1 ,. . ., ym ) ∈ ∂L1 × . . . × ∂Lm } (cf., e.g. the proof of Proposition 10.3.2). Moreover, by an argument, very similar to that in the proof of Proposition 2.4.4, one gets that yi (v, w) are unique with this property. In fact, yi (v, w) are successive reflection points of a billiard trajectory in Ω connecting v and w. This shows that the maps yi (v, w) depend smoothly on (v, w). The latter can be derived also by the implicit function theorem. Next, we identify every tangent hyperplane Tx (∂K), x ∈ ∂K (including Z  and Z  ) with the (n − 1)-dimensional linear subspace of Rn parallel to it, and we measure the lengths of the vectors ξ ∈ Tx (∂K) using the standard norm in Rn . In this way we define also the norm of a linear operator between two tangent spaces. That is, we use the standard Riemannian metric on ∂K. We also assume that some basis in Z  is fixed and denote the elements of Z  by v = (v (1) ,. . ., v (n−1) ). In the same way, we fix a basis in Z  and set w = (w(1) ,. . ., w(n−1) ) ∈ Z  . For fixed w, let ∂v yi (v, w) : Z  → Tyi (v, w)∂Li be the tangential map of the map V v → yi (v, w) ∈ ∂Li at v. In the same way we define the tangential map ∂w yi (v, w) : Z  → Tyi (v,w) ∂Li of the map W w → yi (v, w) ∈ ∂Li . We are going to estimate the norms of the linear operators ∂v yi (v, w) and ∂w yi (v, w). Lemma 10.2.2: For all v ∈ V, w ∈ W and every j = 1,. . ., m we have

and

||∂v yj (v, w)|| ≤ C  e−j

(10.17)

||∂w yj (v, w)|| ≤ C  e−(m−j) ,

(10.18)

where C  > 0 and  > 0 are constants depending only on K. Proof: Fix arbitrary v0 ∈ V and w0 ∈ W . For every j = 1,. . ., m take a smooth chart ϕj : Rn−1 → Uj ⊂ ∂Lj such that ϕj (0) = yj (v0 , w0 ) and {

∂ϕj (p)

∂uj

(0)}n−1 p=1 is an orthonormal basis in

Tyj (v0 ,w0 ) ∂Lj . As in the proof of Lemma 2.2.6, define the function G : V × (Rn−1 )m × W → R

276

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

by G(v; u1 ,. . ., um ; w) = F (v; ϕ1 (u1 ),. . ., ϕm (um ); w). Given i, j = 1,. . ., m, consider the (n − 1) × (n − 1) symmetric matrix

n−1 ∂2G Gij (v, w) = (u(v, w)) . (p) (q) ∂ui ∂uj p,q=1 Then

⎛

G11 ⎜ G21 Guu (v, w) = ⎜ ⎝ ··· Gm1

⎞ · · · G1m · · · G2m ⎟ ⎟ ··· ··· ⎠ · · · Gmm

G12 G22 ··· Gm2

is a symmetric m × m block matrix. According to our computations in the proof of Lemma 2.2.6, we have Gij (v, w) = 0 whenever |i − j| > 1.

(10.19)

Consequently, there exists a constant c0 > 0, depending only on K such that ||Gij (v, w)|| ≤ c0 for all i, j = 0, 1,. . ., m − 1. It then follows by (10.19) that for C0 = 6c0 we have ||Guu (v, w)|| ≤ C0 .

(10.20)

Moreover, using again the computation in the proof of Lemma 2.2.6, we see that Guu (v, w) is positive definite and 2κ0 cos ϕ0 I ≤ Guu (v, w) ≤ C0 I,

(10.21)

I being the identity matrix. In what follows denote the matrices Gij (v0 , w0 ), Guu (v0 , w0 ), etc., briefly by Gij , Guu , etc. We should note that Gij is considered as the matrix of a symmetric linear operator Tyi (v0 ,w0 ) ∂Li → Tyj (v0 ,w0 ) ∂Lj , so Guu is the matrix of a symmetric positive definite linear operator Guu : T =

m  i=1

Tyi (v0 ,w0 ) ∂Li →

m 

Tyi (v0 ,w0 ) ∂Li .

i=1

For (v, w) ∈ V × W close to (v0 , w0 ) and j = 1,. . ., m there is a uniquely determined smooth map (v, w) → uj (v, w) ∈ Rn−1 such that yj (v, w) = ϕj (uj (v, w)). Next, consider the map w → uj (v0 , w).

SCATTERING INVARIANTS FOR SEVERAL STRICTLY CONVEX DOMAINS

277

For l = 1,. . ., n − 1 and i = 1,. . ., m define the column-vectors

n−1

n−1 (p) ∂ui ∂G (v , w) , Gi (ui ) = (ui ) . ∂l ui (w) = (p) ∂w(l) 0 ∂u i

p=1

p=1

For the sake of brevity, we set ∂l ui = ∂l ui (v0 , w0 ) and Gi = Gi (0). We shall also consider the elements ξ of T as column-vectors consisting of m blocks so that the ith block of ξ corresponds to a column-vectors from Tyi (v0 ,w0 ) ∂Li , i = 1,. . ., m. Consider the linear operator of T into itself with diagonal block matrix ⎞ ⎛ 0 ··· 0 D1 ⎜ 0 0 ⎟ D2 · · · ⎟, D=⎜ ⎝· · · ··· ··· ···⎠ 0 0 ··· Dm the ith block Di being a diagonal (n − 1) × (n − 1) matrix ⎛ ⎞ 0 ··· 0 di ⎜ 0 ··· 0⎟ di ⎟, Di = ⎜ ⎝· · · ··· ··· · · ·⎠ 0 0 ··· di with di = e(m−i) . The constant  > 0 will be chosen in a special way later. Note that dm = 1, so Dm is the identity (n − 1) × (n − 1) matrix. For c = e − 1 > 0 we have  d   d     i − 1 ≤ c,  i+1 − 1 ≤ c  di+1 di for all i = 1,. . ., m − 1. Using these inequalities and according to the choice of C0 and c0 , by a direct computation we get ||Guu − DGuu D−1 || ≤ 2cC0 . Therefore, for ξ ∈ T we have ||Guu ξ|| ≤ ||DGuu D−1 ξ|| + 2cC0 ||ξ||. On the other hand, (10.21) implies 2κ0 cos ϕ0 ||ξ|| ≤ ||Guu ξ||, which combined with the previous inequality gives ||ξ|| ≤

1 (||DGuu D−1 ξ|| + 2cC0 ||ξ||). 2κ0 cos ϕ0

We now choose  > 0 such that c = e − 1 =

κ0 cos ϕ0 , 2C0

(10.22)

278

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

then cC0 /κ0 cos ϕ0 = 12 , and therefore ||ξ|| ≤

1 ||DGuu D−1 ξ||. κ0 cos ϕ0

Setting b0 =

1 >0 κ0 cos ϕ0

and η = D−1 ξ ∈ T in the latter inequality, we find ||η|| ≤ b0 ||DGuu η||, η ∈ T .

(10.23)

Fix an arbitrary i = 1,. . ., m. Clearly, for w ∈ W close to w0 we have Gi (ui (v0 , w)) = 0. Differentiating this equality with respect to w(l) and evaluating at w = w0 , one gets m  Gij ∂i uj + ∂i Gi = 0. (10.24) j=1



Here ∂i Gi =

∂2G (p)

∂ui ∂v (l)

n−1 (0) p=1

is a column-vector in Tyi (v0 ,w0 ) ∂Li . Again according to the computations in the proof of Lemma 2.2.6, we observe that ∂i Gi = 0 for all i < m, while ||∂l Gm || ≤ c0 , with a constant c0 > 0, depending only on K. Exchanging c0 , we may assume that c0 = c0 . Let ∂i G ∈ T be the vector, consisting of m blocks each of them having n − 1 entries, such that the first m − 1 blocks are zero, while the mth block coincides with the column-vector ∂i Gm . Define ξ ∈ T so that its ith block coincides with the column-vector ∂i ui . Using (10.24) for all i = 1,. . ., m, we obtain Guu η + ∂i G = 0, and therefore DGuu η + D∂l G = 0. Since Dm = I and all blocks of ∂l G, expect the last one, are zero, we deduce that DGuu η = −∂l G, hence ||DGuu η|| ≤ c0 . This and (10.23) imply ||Dη|| ≤ b0 c0 . The ith block of the vector Dη ∈ T has the form di ∂l ii , and according to di = e(m−i) , we obtain ||∂l ui || ≤ b0 c0 e−(m−i) .

SCATTERING INVARIANTS FOR SEVERAL STRICTLY CONVEX DOMAINS

279

This is true for all i = 1,. . ., m and all l = 1,. . ., n − 1. Consequently, for the map ∂w ui (v0 , w0 ), we have √ ||∂w ui (v0 , w0 )|| ≤ b0 c0 n − 1e−(m−1) . Since yi (v, w) = ϕi (ui (v, w)) and ϕi was chosen in such a way that ∂ui ϕi (0) = I, the same estimate holds for ∂w yi (v0 , w0 ). Therefore, (10.18) is satisfied with   √ κ0 cos ϕ0  , C = b0 c0 n − 1,  = log 1 + 2C0 which clearly depends only on K. To establish (10.17), we proceed in the same way, considering the maps v → yi (v, w0 ) and determining the matrix D by dj = ej , where  is defined as above.  This concludes the proof of the assertion. Proof of Lemma 10.2.1: Let  > 0 be defined as above and let δ = e− . Then 0 < δ < 1 and δ depends only on K. According to our assumptions, we have yi (v  , w ) = yi , yi (v  , w ) = yi ,   for all i = 1,. . ., m, where v  = y0 , v  = y0 ∈ V and w = ym+1 , w = ym+1 ∈ W.   For s ∈ [0, 1] set ws = sω + (1 − s)ω . Given i = 1,. . ., m, consider the smooth curve c(s) = yi (v  , ws ) on ∂Li . Since diam W ≤ C1 , we have ||w0 − w1 || ≤ C1 with some constant C1 > 0 depending only on K. Combining this with (10.18), we obtain . ||c(s)|| ≤ C  C1 δ m−i .

Integrating this inequality for s from 0 to 1, we see that the length of the curve c is not greater than Cδ m−1 , where C = C  C1 > 0. Hence there exists a curve with length ≤ Cδ m−i on ∂Li , joining yi (y  , w ) and yi (v  , w ). Consequently, ||yi (v  , w ) − yi (v  , w )|| ≤ Cδ m−i .

(10.25)

Now set vs = sv  + (1 − s)v  and consider the curve d(s) = yi (vs , w ) on ∂Li . Applying the same argument and according to (10.17), we get ||yi (v  , w ) − yi (v  , w )|| ≤ Cδ m−i . Combining this with (10.25), one gets ||yi (v  , w ) − yi (v  , w )|| ≤ C(δ i + δ m−i ), which proves (10.16).



We conclude this section with another lemma, the proof of which is very similar to that of Lemma 10.2.1.

280

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Fix an arbitrary w ∈ Sn−1 and set Z = Zω . Let M1 , M2 ,. . ., Mr , r ≥ 2,

(10.26)

be a sequence of connected components of K such that Mi = Mi+1 for all i = 1,. . ., r − 1. We assume that πω (M1 ) ∩ πω (M2 ) = ∅.

(10.27)

Using this assumption, we may choose ϕ0 from the beginning of the section in such a way that if x ∈ Z, y ∈ M1 , z ∈ M2 , the open segment (x, y) has direction ω, (x, y) and (y, z) have no common points with K and [x, y] and [y, z] satisfy the law of reflection at y with respect to ∂K, then ϕ < ϕ0 , where ϕ ∈ [0, π/2] is the angle between [y, z] and the normal ν(y) to ∂K at y. Hereafter, we assume that ϕ0 is chosen in this way. In this case ϕ0 depends on K and the choice of Z (i.e. on the choice of ω). We are going to study trajectories γ(u) issued from u ∈ Z with direction ω having r(u) ≥ r reflection points x1 (u), x2 (u),. . ., xr(u) (u) and xi (u) ∈ Γi for every i = 1,. . ., r. For j ≤ r(u) denote by ϕj (u) ∈ [0, π/2] the angle between the normal ν(xj (u)) and the vector N+tj (u) (u), where tj (u) is the time of the jth reflection of γ(u). Then it follows from the choice of ϕ0 that ϕj (u) < ϕ0 , j < r(u),

(10.28)

for every u ∈ Z. Denote by Ur the set of all u ∈ Z with these properties. Lemma 10.2.3: For all u , u ∈ Ur and every i = 0, 1,. . ., r we have ||xi (u ) − xi (u )|| ≤ Bδ r−i ,

(10.29)

where x0 (u) = u, B > 0 and δ ∈ (0, 1) are constants depending only on K and Z. Proof: We use almost the same argument as in the proof of Lemma 10.2.1. Set Z  = Z and choose Z  as above, considering the pair M = Mr , L = Mr−1 . Define V, W and F in the same way. This time is more convenient to denote the elements of (1) (n−1) ). Given w ∈ W , there exist uniquely determined z0 (w) ∈ V by u0 = (u0 ,. . ., u0 Z, z1 (w) ∈ ∂M1 ,. . ., zr−1 (w) ∈ ∂Mr−1 such that F (z0 (w), z1 (w),. . ., zr−1 (w); w) = min{F (z0 , z1 ,. . ., zr−1 ; w) : (z0 , z1 ,. . ., zr−1 ) ∈ Z × ∂M1 × . . . × ∂Mr−1 }. Then for u = z0 (w) we have zi (w) = xi (u), i = 1,. . ., r − 1 and w is a point on the ray starting at xr−1 (u) in the direction N+tr−1 (u) (u). We claim that for every w ∈ W and every j = 0, 1,. . ., r − 1 we have ||∂w zj (w)|| ≤ C  e−(r−j−1) , where C  > 0 and  > 0 are the same constants as in Lemma 10.2.2.

(10.30)

SCATTERING INVARIANTS FOR SEVERAL STRICTLY CONVEX DOMAINS

281

Fix w0 ∈ W and take a smooth chart ϕj : Rn−1 → Uj ⊂ ∂Mj , j = 1,. . ., r − 1, such that ϕj (0) = zj (w0 ) and {

∂ϕj (p)

∂uj

(0)}n−1 p=1 is an orthonormal basis in Tzj (w0 ) ∂Mj .

Take ϕ0 (u0 ) = u0 + z0 (w0 ). Define the function G : V × (Rn−1 )r−1 × W → R by G(u0 , u1 ,. . ., ur−1 ; w) = F (ϕ0 (u0 ), ϕ1 (u1 ),. . ., ϕr−1 (ur−1 ); w). Next, we repeat the argument from the proof of (10.18) by slightly exchanging the notation. After the change of ϕ0 , the constants c0 , C0 , , etc., are the same. In this way we establish the inequalities (10.30). Let u , u ∈ Ur , then u = z0 (w ) and u = z0 (w ) for some w , w ∈ W . By using an integration, as in the proof of Lemma 10.2.1, we see that for every i = 0, 1,. . ., r − 1 we have ||z( w ) − zi (w )|| ≤ Bδ r−i , where δ = e− is the same as above, and B = C = C  C1 e. Since xi (u ) =  zi (w ), xi (u ) = zi (w ), this implies (10.29).

10.3

Existence of scattering rays and asymptotic of their sojourn times

Let the obstacle K have the form (10.1) and let Ω be the closure of its complements. We assume again the condition (H) is satisfied. In this section we show that for every configuration α, under a special choice of ω, θ ∈ Sn−1 , there exists an infinite sequence γq of (ω, θ)-rays in Ω, following α in a certain way, and we study the asymptotics of the sojourn times Tγq as q → ∞. In what follows, we use the notation from Section 10.1. Given x ∈ Rn , η ∈ Sn−1 , denote by l(x, η) the linear ray starting at x with direction η. Sometimes it will be convenient to use the notation dist(x, y) = ||x − y||. Fix an arbitrary configuration α of the form (10.2). We say that the pair (ω, θ) of element of Sn−1 is α-admissible if the following conditions are satisfied: (i) every (ω, θ)-ray in Ω is ordinary and any two different (ω, θ)-rays in Ω have distinct sojourn times; (ii) for every x ∈ Γi1 , the ray l(x, −ω)(resp. l(x, ω)) has no common points with K \ Ki1 (resp. with Ki2 ); (iii) for every x ∈ Γim , the ray l(x, θ) (resp. l(x, −θ)) has no common points with K \ Kim (resp. Kim−1 ).

282

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Lemma 10.3.1: For every configuration α there exist unit vectors ω = θ such that (ω, θ) is α-admissible. Proof: Set D1 = K1 , D2 = Ki2 . Take an arbitrary hyperplane A separating D1 and D2 such that A is tangent to D1 at some point x and to D2 at another point y. Set ω  = (x − y)/||x − y|| and consider the convex cone C = {y + t(u − y) : u ∈ D1 , t ≥ 0}. It is easy to check that the orthogonal projection of D1 on the hyperplane Zω is contained in C. We claim that l(u, ω  ) ∩ (K \ D1 ) = ∅, u ∈ D1 .

(10.31)

To prove this, assume that there exist u ∈ D1 , t > 0, j = i1 such that v = u − tω  ∈ Kj . Then u and v have a common orthogonal projection u on Zω . Since u, u ∈ C, we have v ∈ C. On the other hand, the definition of C implies that the segment [y, v] contains points of D1 which is a contradiction of the condition (H). Thus, (10.31) holds. Then by the compactness of K \ D1 , there exists  > 0 such that l(u, −ω) ∩ (K \ D1 ) = ∅, u ∈ D1 , holds, provided ω ∈ Sn−1 and ||ω − ω  || < . Take an arbitrary ω ∈ Sn−1 , satisfying the latter inequality and such that −ω, ν(x) > 0. Then clearly the condition (ii) is satisfied. Fix an ω with property (ii) and denote by R(ω) the residual subset of Sn−1 from Theorem 10.1.1. Using the density of this set, and applying the above argument for D1 = Kik and D2 = Kik−1 , we find θ ∈ R(ω) satisfying (iii). Now θ ∈ R(ω) implies  that (i) is also satisfied. This proves the assertion. From now on till the end of the chapter, α = (i1 ,. . ., ik ) will be a fixed configuration with k ≥ 2 and i1 = ik and l will be a a fixed integer with 1 ≤ l ≤ k. For every integer q ≥ 0 set αq,l = (i1 ,. . ., ik ; . . . ; i1 ,. . ., ik ; i1 ,. . ., il ),

(10.32)

where the block (i1 ,. . ., ik ) is repeated q times. Clearly, αq,l is a configuration of length qk + l. Now we fix two arbitrary unit vectors ω = θ such that the pair (ω, θ) is α1,l -admissible. The existence of such vectors is guaranteed by Lemma 10.3.1. The pair (ω, θ) will also be fixed till the end of the chapter. As earlier, we shall use the notation Z = Zω . Proposition 10.3.2: For every integer q ≥ 0 there exists a unique (ω, θ)-ray γq of type αq,l in Ω.

SCATTERING INVARIANTS FOR SEVERAL STRICTLY CONVEX DOMAINS

283

Proof: Set for convenience η = −θ. Fix an arbitrary integer q ≥ 0 and set m = qk + l and D = Z × Γi1 × · · · × Γim × Zη , where ij are the successive components of αq,l . Define the function F : D → R by F (ζ) = ||z1 − x1 || +

m−1 

||xj − xj+1 || + ||xm − z2 ||

j=1

for every ζ = (z1 ; x1 ,. . ., xm ; z2 ) ∈ D. Clearly, F is continuous and, considering its restriction on an appropriate compact subset of D, we see that there exists ζ  = (z1 ; x1 ,. . ., xm ; z2 ) ∈ D with F (ζ  ) = minF . Clearly, z1 is the orthogonal projection of x1 on Z, while z2 is the projection of xm on Zη . Since (ω, θ) satisfies the condition (ii), the segment [z1 , x2 ] has no common points with Ki1 . For c > 0 consider the rotative ellipsoid Ec = {x ∈ Rn : ||z1 − x|| + ||x − x2 || ≤ c}. Let c > 0 be the minimal number with Ec ∩ Ki1 = ∅. Then Ec is tangential to Ki1 at some of its points y1 . It is now clear that y1 = x1 , since F has total minimum at ζ  . Therefore, the segments [z1 , x1 ] and [x1 , x2 ] satisfy the law of reflection at x1 with respect to Γi1 . Repeating this argument several times and using the condition (H), we see that x1 ,. . ., xm are the successive reflection points of a (ω, θ)-ray of type of αq,l in Ω.  The uniqueness of this ray follows from Corollary 2.4.6. For every integer q ≥ 0 set Uq = Uαq,l . Then Uq is an open subset of Z = Zω , and the above proposition implies Uq = ∅. More precisely, there exists a unique uq ∈ Uq such that γq = γ(uq ) is an (ω, θ)-ray in Ω of type αq,l . As in Section 10.2, the condition (H) and the choice of Z imply the existence of ϕ0 ∈ (0, π/2) and using the notation of this section, we have ϕj (u) < ϕ0 , j = 1,. . ., r(u),

(10.33)

for every u ∈ Z. Then as an immediate consequence of Lemma 10.2.3, we get the following. Lemma 10.3.3: There exist constants C > 0 and δ > 0, depending only on K and Z, such that dist (xi (u), xi (v)) ≤ Cδ qk+l−i , i = 0, 1,. . ., qk + l

(10.34)

for every integer q ≥ 0 and all u, v ∈ Uq . We can also apply Lemma 10.2.3, considering the hyperplane Z−θ instead of Z = Zω . Note that γq is a (ω, θ)-ray; therefore, taking its reflection points in the

284

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

opposite order, we get a sequence xql+k (uq ), xqk+l−1 (uq ),. . ., x2 (uq ), x1 (uq ), which is the sequence of the successive reflection points of a (−θ, −ω)-ray in Ω. In a similar way we can consider also the ray γq+1 . Now, eventually exchanging the constants C and δ (making them depending on K, Zω and Z−θ ) to get an analogue of Lemma 10.3.3 with Z replaced by Z−θ , we obtain the inequalities dist(xqk+l−i (uq ), x(q+1)k+l−i (uq+1 )) ≤ Cδ qk+l−i−1 for all i = 0, 1,. . ., qk + l. Setting j = qk + l − i, the latter implies dist(xj (uq ), xj+k (uq+1 )) ≤ Cδ j−1 , j = 0, 1,. . ., qk + l.

(10.35)

Next, we assume that C and δ are fixed having the above properties. Using (10.34) for i = 0, we see that diam U¯q ≤ Cδ qk+l for every q. Since U1 ⊃ · · · ⊃ Uq ⊃ · · · , we deduce that the intersection of the closures of these sets consists of exactly one point u∞ , that is ∞ ¯ ∩∞ q=0 Uq = {u }. It is clear that the trajectory γ ∞ = γ(u∞ ) has infinitely many reflection points. ∞ Moreover, for x∞ i = xi (u ) we have x∞ qk+l ∈ Γij , q ≥ 0, 1 ≤ j ≤ k. Fix for a moment q ≥ 0, r ≥ 0 and j = 1,. . ., k. Take an arbitrary integer p > q and apply Lemma 10.2.1 for m = pk + j − 2, the connected components L = K1 , L1 = Ki2 ,. . ., Lm = Kij−1 , L = Kij of K and the sequences of points ∞ ∞ ∞ x∞ 1 , x2 ,. . ., xpk+j−1 , xpk+j

and

∞ ∞ ∞ x∞ rk+1 , xrk+2 ,. . ., x(p+r)k+j−1 , x(p+r)k+j .

Then, we obtain ∞ qk+j dist(x∞ + δ (p−q)k−2 ). qk+j , x(q+r)k+j ) ≤ C(δ

(10.36)

Here we have used the inequality (10.16) for i = qk + j. Since (10.36) holds for all p ≥ q, letting p → q, we get ∞ qk+j dist(x∞ qk+j , x(q+r)k+j ) ≤ Cδ

(10.37)

SCATTERING INVARIANTS FOR SEVERAL STRICTLY CONVEX DOMAINS

285

for all q and r, which shows that the sequence {x∞ qk+j }q is convergent. Denote zj by its limit. Then zj ∈ Γij and (10.37) implies qk+j dist (x∞ , q ≥ 0, 1 ≤ j ≤ k. qk+j , zj ) ≤ Cδ

(10.38)

Moreover, it follows from ∞ zj+k = lim x∞ (q+1)k+j = lim xqk+j = zj q

q

that z1 , z2 ,. . ., zk are the successive reflection points of a periodic reflecting ray γα of type α in Ω. Note that by Corollary 2.2.4, there exists only one such ray. Remark 10.3.4: One can avoid the application of Lemma 10.2.1 in the above argument, according to the inequalities (10.35), which follows in fact from Lemma 10.2.3. In other words, for our considerations in this section and the following one as well, only Lemma 10.2.3 from Section 10.2 is necessary. However, Lemma 10.2.1 presents an important property of the billiard trajectories, and its proof is almost the same as that of Lemma 10.2.3, we include it in this book. Set j  ||zp − zp=1 ||, 1 ≤ j ≤ k, dj = p=1

dα = dk , and ∞ L∞ m = x1 , ω +

m 

∞ ||x∞ p − xp+1 ||.

p=1

Clearly, dα is the period (length) of γα . Using (10.38) for q ≥ 0 and r ≥ 0, we find ∞ |(L∞ q+r)k+j − (q + r)dα − dj ) − (Lqk+j − qdα − dj )|

≤ 2C

rk+1 

δ qk+t+j ≤ C1 δ q ,

p=1

where the constant C1 > 0 is determined by C and δ. Hence for every j ≤ k, there exists Lj = Lα,ω,j = lim (L∞ qk+j − qdα − dj ). q

Moreover, we have the asymptotic q L∞ qk+j = qdα + dj + Lj + O(δ ) as q → ∞.

(10.39)

In the same way as above for u∞ ∈ Z = Zω , we find a unique v ∞ ∈ Zη such that the billiard trajectory γ˜ ∞ , starting from η ∞ in direction η = −θ has infinitely many ∞ ∈ Γjr , for all q ≥ 0, 1 ≤ r ≤ k, where reflection points yi∞ such that yqk+r (j1 ,. . ., jk ) = (il , il−1 ,. . ., i1 ; ik , ik−1 ,. . ., il+2 , il+1 ).

286

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Now for ∞ G∞ m = − y1 , θ +

m 

∞ ||yp∞ − yp+1 ||

p=1

we get the asymptotic q G∞ qk = qdα + Lα,θ + O(δ ) as q → ∞.

(10.40)

where Lα,θ is a constant, depending only on K, α and θ. This can be proved by the same argument as above, and we omit the details. Set Tq = Tγq , L(q) p





= x1 (uq ), ω +

p 

||xr (uq ) − xr+1 (uq )||,

r=1

and G(q) p

= −xqk+l(uq ) , θ +

qk+l−1 

||xr (uq ) − xr+1 (uq )||.

r=p+1

  Given q ≥ 0, we define p = p(q) by p = k q2 + l − 1. Using the choice of the constants C and δ (cf. Lemma 10.3.3), we get the following: ∞ (q) |L(q) p − Lp | + |Gp − G(q−[q/2])k |

≤2

p+1  r=1



(q−[q/2])k+1

||xr (uq ) − x∞ r || + 2

||xqk+l−r+1 (uq ) − yr∞ || ≤ C2 δ q ,

r=1

where C1 > 0 is a constant, depending only on K, α, ω and θ. Combining this with the asymptotics (10.39) and (10.40), and using the fact that for every q we have Tq = (q)

(q)

Lp + Gp , one obtains the following. Theorem 10.3.5: The sojourn times Tq have the asymptotic Tq = qdα + Lα,ω,θ + O(δ q ) as q → ∞,

(10.41)

where Lα,ω,θ = Ll−1 + Lα,θ + dl−1 , and the constants dj , Lj and Lα,θ , d0 = 0, are determined as above. In particular, from the sojourn times Tq one can recover the period dα of the periodic reflecting ray γα . Corollary 10.3.6: Let s = 2, that is K = K1 ∪ K2 , and let d be the distance between K1 and K2 . Let (ω, θ) be α-admissible for α = (1, 2). Then for every

SCATTERING INVARIANTS FOR SEVERAL STRICTLY CONVEX DOMAINS

287

i = 1, 2 and every integer m ≥ 1 there exists a unique (ω, θ)-ray in Ω with m (i) reflection points, the first of which belongs to Ki . Let Tm be the sojourn time of this (ω, θ)-ray. There exists a constant δ ∈ (0, 1) and for i = 1, 2, j = 0, 1 a constant (i,j) Lω,θ , such that (i) (i,j) T2q+l = 2qd + Lω,θ + O(δ q ) as q → ∞. Example 10.3.7: Under the notation in Corollary 10.3.6, assume in addition that n = 2 and K1 and K2 are discs in R2 = Oxy, having one and the same radius r and centers (−a, 0) and (a, 0), respectively. We suppose that a > r > 0, so K1 and K2 are disjoint. Set ω = (0, −1), θ = (cos θ0 , sin θ0 ), where θ0 ∈ (0, π/2) is taken close to π/2. By Lemma 10.3.1 we can choose θ0 in such a way that the pair (ω, θ) is α-admissible for α = (1, 2). In fact, under certain assumptions on r and α, it can be shown that this is true for all θ in a small neighbour(i) hood of −ω in Sn−1 (cf., e.g. [NS]). Using the notation Tm from the above corollary, (i) (j) we then have Tm = Tn whenever (m, i) = (n, j). Next, consider the two-dimensional torus K  in R3 = Oxyz, obtained by rotating K (or K1 only) about the axis Oy. Let Ω be the closure of the complement of K  in R3 . We shall consider ω and θ as vectors in R3 having third component 0. Then it is easy to see that the scattering length spectrum of Ω coincides with (i) : m ∈ N, i = 1.2}. {Tm

Moreover, for m ∈ N and i = 1, 2 if γ is (ω, θ)-ray in Ω with sojourn time Tm , then (i) close to −Tγ = Tm we have (9.7). (i)

10.4

Asymptotic of the coefficients of the main singularity

We continue with the notation and assumptions from the previous section. For the sake of brevity, set κ = cos ϕ0 . Given q ≥ 0, denote by cq the coefficient in front of the main singularity of the scattering kernel sΩ (t, θ, ω) for t close to −Tq . In other words, cq is the coefficient in front of δ (n−1)/2 (t + Tq ) in the formula (9.7) for the (ω, θ)-ray γ = γq and t close to −Tq . Our aim in this section is to find the asymptotic of |cq | as q → ∞. Set D = diam K, d = min dist (Ki , Kj ), d = i=j

1 . d

Since the domains Ki are compact and strictly convex, there exist constants μ2 > μ1 > 0 such that μ1 (v, v) ≤ Gx v, v ≤ μ2 (v, v), v ∈ Tx (∂K),

288

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Gx : Tx (∂K) → Tx (∂K) being the differential of the Gauss map of ∂K at x. Let x ∈ Γi , y ∈ Γj , i = j, and assume that the segment [x, y] is contained in Ω and is transversal to both Γi and Γj . Denote by Π the hyperplan passing through x and orthogonal to [x, y]. Set e = (y − x)/||y − x||, and denote by π the projection Π → Tx (∂K) along the vector e. As in Section 2.3, define the symmetric linear map ψ˜ : Π → Π by   ˜ (10.42) ψ(u), u = 2 e, ν(x) Gx (π(u)), π(u) , u ∈ Π. We shall say that ψ˜ is the operator determined by the segment [x, y]. A standard exercise shows that spec ψ˜ ⊂ [2μ1 ν(x), e, 2μ2 ν(x), e−1 ].

(10.43)

We shall prove two technical lemmas which will be used several times later. Lemma 10.4.1: Let x, x ∈ Γi , y, y  ∈ Γj , i = j, and let e > 0 be such that dist (x, x ) <  and dist (y, y  ) < . Introduce the vectors e=

y−x , ||y − x||

e =

y  − x , ||y  − x ||

and assume that e, ν(x) ≥ κ, e , ν(x ) ≥ κ. Let ψ˜ : Π → Π, ψ˜ : Π → Π be the operators determined by the segments [x, y] and [x , y  ], respectively. Then there exist a linear isometry A : Rn → Rn and a constant C  > 0, depending only on K and κ, such that A(Π ) = Π, ||A − I|| < C   and ||ψ˜ − Aψ˜ A−1 || < C  . Proof: Let A1 : Rn → Rn be the translation determined by the vectors x − x. Set ν  = A1 (ν(x )) and denote by A2 the rotation around a line in Rn in a angle ϕ = arccosν(x), ν(x ), for which A2 (ν  ) = ν(x) and such that A2 = id on {ν(x), ν  }⊥ . Finally, let e = A2 ◦ A1 (e ) and let A3 be the rotation with A3 (e ) = e, which is identical on {e, e }⊥ . We set A = A3 ◦ A2 ◦ A1 . Clearly, A is a linear isometry. It is easy to check that ||Ai − I|| < const  for every i = 1, 2, 3. For example,    1 − cos ϕ sin ϕ    = 2(1 − cos ϕ) = ||ν(x) − ν(x )||, ||A2 − I|| =   − sin ϕ 1 − cos ϕ  and the smoothness of Gx and the compactness of K imply ||ν(x) − ν(x )|| < const ||x − x || < const . Similar simple estimates can be written for A1 and A3 .

SCATTERING INVARIANTS FOR SEVERAL STRICTLY CONVEX DOMAINS

289

Thus, ||A − I|| < const . Next, set χ ˜ = Aψ˜ A−1 , G = Gx , G = Gx . Take an  arbitrary u ∈ Π with ||u|| = 1 and set u = A−1 u. Then u ∈ Π and ||u || = 1. Let π : Π → Tx (∂K), π  = Π → Tx (∂K) be the projections along the vectors e and e , respectively, and let v = π(u), v  = π  (u ) (see the diagram). Π

˜ ψ

A

Π

Π

˜ ψ

Π

π

π Tx (∂K)

Tx(∂K)

We have ˜ ˜ |ψ(u), u − ˜ χ(u), u|= |ψ(u), u − ψ˜ (u ), u | = |2e, ν(x)G(π(u)), π(u) − 2e , ν(x )G (π  (u )), π  (u )| ≤ 2|e, ν(x) − e , ν(x )|G(v), v + 2e , ν(x )|G(v), v − G (v  ), v  | < const  + 2|G(v), v − G (v  ), v  |. It follows by the smoothness of the Riemannian metric on ∂K that |G(v), v − G (v  ), v  | < const ||v − v  ||. Now taking into account that ||π|| ≤ 1/κ, ||π  || ≤ 1/κ, we find ||v − v  || = ||π(u) − π  (A−1 (u))|| ≤ ||π − π  ◦ A−1 || ≤ (||π|| + ||π  ||)||(A − I)|| < const . Therefore, |(ψ˜ − χ ˜ )(u), u| < const , which implies ||ψ˜ − χ ˜ || < const . Thus  proves the assertion. Furthermore, we are going to apply the above lemma for two sequences of points. Let x1 ,. . ., xp and x1 ,. . ., xp be points of K such that for every j = 1,. . ., p, the points xj and xj belong to Γi for one and the same i = i(j). Assume that dist (xj , xj ) ≤ Daj ,

j = 1,. . ., p

for some constants D > 0 and a > 0. Introduce the unit vectors ej =

xj+1 − xj , ||xj+1 − xj ||

ej =

xj+1 − xj , ||xj+1 − xj ||

and assume that ej , ν(xj ) ≥ κ,

ej , ν(xj ) ≥ κ,

j = 1,. . ., p.

(10.44)

290

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

It follows from the above lemma that for every j ≤ p there exists a linear isometry Aj in Rn such that Aj (Π) = Πj and ||Aj − I|| < C  D(1 + a)aj ,

 j ||ψ˜j − Aj ψ˜j A−1 j || < C D(1 + a)a .

(10.45)

Here ψ˜j : Πj → Πj and ψ˜ : Πj → Πj are the operators determined by the segments [xj , xj+1 ] and [xj , xj+1 ], respectively. Let M1 : Π1 → Π1 and let M1 : Π1 → Π1 be arbitrary symmetric non-negative definite linear operators. Define recursively Mi = σi Mi−1 (I + λi Mi−1 )−1 σi + ψ˜i ,

i = 2,. . ., p,

(10.46)

where λi = dist (xi−1 , xi ) and σi is the symmetry with respect to Πi . We define the maps Mi , i = 2,. . ., p, in the same way, replacing ψ˜i , σi , λi and xi by ψ˜i , σi , λi and xi , respectively. Finally, set  a if a ≥ 1,  −1 b = (1 + 2μ1 κd ) , a1 = (10.47) max{a, b} if a < 1. Next, we use the notation log = loge . Lemma 10.4.2: Under the above assumptions, there exist constants E > 0, E  > 0, depending only on K, κ and a, such that j 2(j−r) ||Mr − Ar Mr A−1 Mj − Aj Mj A−1 r || j < DEa1 + b

(10.48)

and  )−1 | | log det((I + λi+1 Mj )(I + λj+1 Mj+1

< DE  aj1 + (n − 1)db2(j−r)+1 ||Mr − Ar Mr A−1 r ||

(10.49)

for all 1 ≤ r ≤ j ≤ p. Proof: First, note that |λi − λi | < D(1 + a)ai . Moreover, for every symmetric non-negative definite linear operator M we have ||(I + λM )−1 || ≤ (1 + λσ)−1 ,

||(M (1 + λM )−1 || ≤

1 , λ

where σ = min (spec M ). It follows by (10.46) that  min (spec Mi−1 ) ≥ min (spec ψ˜i−1 ),

i ≥ 2,

therefore ||(I + λi Mi−1 )−1 || ≤ b, ||Mi−1 (I + λi Mi−1 )−1 || ≤

1 ≤ d . λi

SCATTERING INVARIANTS FOR SEVERAL STRICTLY CONVEX DOMAINS

Introduce the operator

291

Li = Ai Mi A−1 i : Πi → Πi .

Since σi = A−1 I σi Ai , we find Li = σi Bi Li−1 (I + λi Li−1 )−1 Bi−1 σi + Ai ψ˜i A−1 i for Bi = Ai ◦ A−1 i−1 . Using (10.46) and the trivial inequality X − Bi Y Bi−1  ≤ 2||X||||I − Bi || + ||X − Y ||, we have ||Mi − Li || ≤ Mi−1 (I + λi Mi−1 )−1 − Bi Li−1 (I + λi Li−1 )−1 Bi−1  i   2 i−1 +||ψ˜i − Ai ψ˜i A−1 i || < CD(1 + a)a + 2C d (1 + a) a

+||Mi−1 (I + λi Mi−1 )−1 − Li−1 (I + λi Li−1 )−1 ||. The last term can be estimated as follows: ||Mi−1 (I + λi Mi−1 )−1 − Li−1 (I + λi Li−1 )−1 || ≤ |λi − λi |||(I + λi Mi−1 )−1 ||||Li−1 (I + λi Li−1 )−1 || +b2 ||Mi−1 − Li−1 || < Dd (1 + a)ai + b2 ||Mi−1 − Li−1 ||. 2

Therefore, ||Mi − Li || < DE  ai + b2 ||Mi−1 − Li−1 ||, i = 2,. . ., p, where E  = (1 + a)(C  + 2d (1 + a)a−1 C  + d 2 ) > 0. Repeating this procedure j − r times, one gets ||Mj − Lj || < DE 

j−r−1 

aj−i b2t + b2(j−r) ||Mr − Lr ||,

t=0

for all 1 ≤ r ≤ j ≤ p. There are two cases. Case 1. a ≥ 1. Then a > b2 , and (10.50) implies ||Mj − Lj || < DE  aj

j−r−1  t=0

(b2 /a)t + b2(j−r) ||Mr − Lr ||

−1  b2 + b2(j−r) ||Mr − Lr ||. < DE  aj 1 − a In this case, we set E = E  a(a − b2 )−1 .

(10.50)

292

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Case 2. 0 < a < 1. Then a1 = max{a, b} < 1, and (10.50) implies ||Mj − Lj || < DE



j−r−1 

aj+t + b2(j−r) ||Mr − Lr || 1

t=0

< DE  (1 − a1 )−1 aj1 + b2(j−r) ||Mr − Lr ||. Now set E = E  (1 − a1 )−1 . It follows by the choice of E in both cases that (10.48) holds for 1 ≤ r ≤ j ≤ p. Before going on, let us note that if A is an arbitrary linear operator in Rk , then |det A| ≤ (1 + ||A − I||)k . Using this, we find the following estimates: det((I + λi+1 Mi )(I + λi Mi )−1 )

 n−1 ≤ 1 + ||I − (I + λi+1 Mi )(I + λi+1 Li )−1 ||  = 1 + λi+1 Li − λi+1 Mi )(I + λi+1 Li )−1  n−1  n−1 |λ − λ | ≤ 1 + i+1  i+1 + bλi+1 ||Mi − Li || λi+1 < (1 + D(1 + a)d ai+1 + bd||Mi − Li ||)n−1 . In the same way one gets a similar inequality for det((I + λi+1 Mi )−1 (I + λi+1 Mi )), therefore | log det(I + λi+1 Mi ) − log det(I + λi+1 Mi )| < (n − 1) log(1 + Dd (1 + a)ai+1 + bd||Mi − Li ||) < (n − 1)(Dd (1 + a)ai+1 + bd||Mi − Li ||). Finally, applying (10.48) for j = i, we get (10.49) with E  = (n − 1)(a21 + a1 )d +  bdE) for all 1 ≤ r ≤ j ≤ p. This completes the proof of the lemma. For the reflection points zi of the unique periodic reflecting ray of type α in Ω (cf. Section 10.3), we define zm = zj , whenever m has the form m = qk + j, 1 ≤ j ≤ k. Let ψ˜j : Πj → Πj be the operator determined by [zj , zj+1 ], and let M(Πj ) be the space of all symplectic positively definite linear maps M : Πj → Πj . Define Fj : M(Πj ) → M(Πj ) by    M (I + λj+1 M )−1 σj+1 + ψ˜j+1 , Fj (M ) = σj+1

SCATTERING INVARIANTS FOR SEVERAL STRICTLY CONVEX DOMAINS

293

where λj = dist (zj , zj−1 ) and σj is the symmetry with respect to the tangent hyperplane to ∂K at zj . Using the argument from the proof of Proposition 2.3.2, we deduce that the map Fk ◦ Fk−1 ◦ . . . ◦ F1 : M(Π1 ) → M(Π1 ) has a unique fixed point M1 . Then M2 = F1 (M1 ) is the unique fixed point of Fk ◦ Fk−1 ◦ . . . ◦ F2 , etc. Let q ≥ 0 be an arbitrary integer. Consider the configuration αq,l , and set Jq = Jαq ,l : Fαq ,l → Sn−1 . Recall that uq ∈ U ⊂ Z is the unique point in Z for which there exists a (ω, θ)-ray of type αq,l intersecting Z at uq . We denote this (ω, θ)-ray by γq . Set m = qk + l and xi = xi (uq ) for i = 1,. . ., m and xi = xi (uq+1 ) for i = 1,. . ., m + k. Define the operators ψ˜i and ψ˜ as in the text before Lemma 10.4.2, and set M1 = ψ˜1 , M1 = ψ˜1 . Next, define M , recursively by (10.46), and Mi in a similar way. Finally, set !p" !m" , t= . (10.51) p= 2 2 Clearly, 2p ≤ m < 2p + 1, 2t ≤ p < 2t + 1, which implies 4t ≤ m < 4p + 3. Applying the inequalities (10.34), we get dist (xi , xi ) < Cδ p−i , dist (xi , xi ) < Cδ i ,

i = 1,. . ., t,

i = t + 1,. . ., p.

(10.52) (10.53)

Next, (10.35) implies dist (xp+i , xp+k+i ) < Cδ p−i−1 , dist (xp+i , xp+k+i ) < Cδ  ,

i = 1,. . ., t,

i = t + 1,. . ., m − p.

(10.54)

Finally, apply Lemma 10.2.1 to the sequences x1 , x2 ,. . ., xm and z1 , z2 ,. . ., zm . Instead of referring to Lemma 10.2.1, one can use the inequalities (10.35) only (cf. Remark 10.3.4). Then we find dist (xrk+j , zj ) ≤ C(δ rk+j + δ m+k−(rk+j) ) for rk + j ≤ p + k. Since m ≥ 2p, this implies dist (xrk+j , zj ) < C  δ rk+j , rk + j ≤ p + k, where C  = C(1 + δ −k ) > C. Set D = C  δ p , a = 1/δ > 1, then (10.44) takes the form dist (xi , xi ) < Dai , i = 1,. . ., t. Now we can apply Lemma 10.4.2 to the sequences x1 ,. . ., xt and x1 ,. . ., xt . Since   p−2 ˜ ˜ −1 , ||M1 − A1 M1 A−1 1 || = ||ψ1 − A1 ψ1 A1 || < C D(1 + a)a < 2CC δ

294

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

it follows by (10.49) for r = 1 and a1 = 1/δ that | log det(I + λi+1 Mi ) − log det(I + λi+1 Mi )| < DEδ −i + 2(n − 1)db2i−1 C  Cδ p−2 < CE  δ1p−i + 2(n − 1)dCC  δ1p+2i−3 < F1 δ1p−i for all i = 1,. . ., t, where F1 = CE  + 2(n − 1)dCC  and δ1 = max {b, δ} ∈ (0, 1). Furthermore, observe that (10.52) implies dist (xi , xi ) < Cδ i , i = 1,. . ., t. This and (10.53) show that Lemma 10.4.2 is applicable with D = C, a = δ = a1 , r = 1. Then we get | log det(I + λi+1 Mi ) − log det(I + λi+1 Mi )| < F1 δ1i , i = 1,. . ., p.

(10.55)

Next, we apply Lemma 10.4.2 to the sequences z1 ,. . ., zp+k and x1 ,. . ., xp+k . It follows by (10.43) that 2μ ||M1 || = ||ψ˜1 || ≤ 2 . κ Therefore, (10.49) implies  | log det(I + λp+j+1 Mp+j ) − log det(I + λp+j+1 Mp+j )| < F2 δ1p+j , j = 1,. . ., κ, (10.56) where F2 > 0 is a constant. Consider the sequences x1 ,. . ., xp and xk+1 ,. . ., xk+p , and the corresponding isometries Aj : Πj+k → Πj (cf. Lemma 10.4.1). Applying again Lemma 10.4.2 and the inequalities (10.35), we find a constant F3 > 0 such that p  ||Mp − Ap Mp+k A−1 p || < F3 δ1 .

(10.57)

Now we turn to the sequences xp+1 ,. . ., xp+t and xp+k+1 ,. . ., xr+k+t . It follows by (10.54), (10.57) and Lemma 10.4.2 for D = cδ p , a = 1/δ = a1 , r = p, that  | log det(I + λp+j+1 Mp+j ) − log det(I + λp+k+j+1 Mp+k+j )|

< F4 δ1p−j , j = 1,. . ., t, where F4 > 0 is a constant.

(10.58)

SCATTERING INVARIANTS FOR SEVERAL STRICTLY CONVEX DOMAINS

295

Finally, applying Lemma 10.4.2 twice more, we find constants F5 > 0, F6 > 0 such that  )| | log det(I + λp+j+1 Mp+j ) − log det(I + λp+k+j+1 Mp+k+j < F5 δ1j , j = t + 1,. . ., m − p, | log |det Mm | − log |det

 Mm+k ||


0. Therefore, by (10.63), |δq,l | < F0 δ0kq (1 − δ0k )−1 + F0 δ0kq = F0 δ0kq , where F0 = F0 (1 − δ0k )−1 + F0 > 0. Finally, set cα = c2˜ . As a consequence of the considerations in this section, we obtain the following. Theorem 10.4.3: There exist constant Qα and δ0 , 0 < δ0 < 1, depending only on K, α, ω and θ, such that log |cq | = qcα + Qα + O(δ0q ) as q → ∞.

(10.64)

Let us note that the constant cα has a certain geometrical meaning. Namely, we have

n−1  n−1  1 1 (10.65) log |μj | = − log μj , cα = − 2 j=1 2 j=1 μ1 ,. . ., μn−1 being the eigenvalues of the linear Poincaré map Pγα with modulus greater than 1. Indeed, the latter eigenvalues are precisely the eigenvalues of the operator S from the proof of Proposition 2.3.2. Using the representation of S found there, we get k  det(I + λi+1 Mi ) = −˜ c = −2cα , log det S = log i=1

which proves (10.65). Notice that in the case of two strictly convex disjoint obstacles K1 , K2 ⊂ R3 , the constants d = dist (K1 , K2 ) and c0 = − 12 log(μ1 μ2 ) determine a sequences of resonances of the scattering matrix (see [I3], [Ger]) given by zm =

10.5

c mπ + i 0 , m ∈ Z. d 2d

Notes

Theorem 10.1.1 and Lemma 10.1.2, as well as the whole of Sections 10.3 and 10.4, are taken from [PS5]. Some results related to Theorem 10.3.5 are proved by Nakamura and Soga (cf. [Nal], [Na2], [NS]). Lemma 10.3.4 is well known in the theory of dispersing billiards. It seems that it was first proved for curves in the plane by

SCATTERING INVARIANTS FOR SEVERAL STRICTLY CONVEX DOMAINS

297

Sinai [Sinl], see also [Sin2]. The material in Section 10.2 is an adaptation of a part of Appendix 9(b) in [Sjo], perhaps it might be derived also from Section 3 in [I4]. As we have already mentioned in Remark 10.3.4, Lemma 10.2.3 is sufficient for our aims in this chapter. Namely, one can slightly exchange the arguments in Sections 10.3 and 10.4 to prove the same results using only Lemma 10.2.3 from Section 10.2. For two strictly convex obstacles, it is proved in [S9] a more precise result than Corollary 10.3.5. Namely, it was shown that we have an asymptotic with remainder O(Λq ), where Λ is one of the Lyapunov exponent of the billiard ball map related to the shortest periodic ray. For the scattering poles for several strictly convex disjoint obstacles and the behaviour of the dynamical zeta function see [P3], [I3], [I4], [S10].

11

Poisson relation for the scattering kernel for generic directions In this chapter we prove that the Poisson relation for the scattering kernel s(t, ω, θ) becomes an equality for almost all pairs of unit vectors (ω, θ). Apart from the main results in Chapters 5 and 9, this requires a certain regularity property of the related generalized Hamiltonian flow. In Section 11.3 we prove that for each T > 0 the phase space of the generalized Hamiltonian flow can be represented as a countable union of compact subsets Si such that on each Si , {Ft }0≤t≤T coincides with the restriction (i) of a one-parameter family Gt of uniformly Lipschitz maps defined in a neighbour(i) hood of Si such that for all but finitely many t, Gt is smooth and its restriction to transversal sections is symplectic. For ‘Sard’s theorem-type’ applications, this regularity property is as good as smoothness, and in particular implies that the generalized Hamiltonian flow preserves the Hausdorff dimension dimH of Borel subsets of its phase space.

11.1

The Poisson relation for the scattering kernel

Let K be a compact subset of Rn , n ≥ 2, with C ∞ boundary ∂K such that ΩK = Rn \ K is connected. As before, such a set K is called an obstacle in Rn . Denote by K the class of obstacles K such that for (x, ξ) ∈ T ∗ (∂ΩK ), if the curvature of ∂K at x vanishes of infinite order in direction ξ, then ∂K is convex at x

Geometry of the Generalized Geodesic Flow and Inverse Spectral Problems, Second Edition. Vesselin M. Petkov and Luchezar N. Stoyanov. © 2017 John Wiley & Sons, Ltd. Published 2017 by John Wiley & Sons, Ltd.

SCATTERING KERNEL FOR GENERIC DIRECTIONS

299

in direction ξ. It should be mentioned that K is of second Baire category in the space of all obstacles with smooth boundaries endowed with the Whitney C ∞ topology. The main result in this chapter is the following. Theorem 11.1.1: Let K ∈ K. There exists a subset R of full Lebesgue measure in Sn−1 × Sn−1 such that sing supp sK (t, θ, ω) = { − Tγ : γ ∈ Lω,θ } holds for all (ω, θ) ∈ R. It can be seen from the proof that the set R is also of second Baire category in Sn−1 × Sn−1 . This theorem is related to some problems in inverse scattering theory. It shows that for most obstacles K the singularities of the scattering kernel are completely determined by some geometric objects – the scattering length spectrum of K. In general these objects are not enough to recover the obstacle. One may conjecture that for obstacles K satisfying a natural accessibility condition the scattering length spectrum completely determines K. This is indeed so for some special classes of obstacles (see Chapter 13). To prove Theorem 11.1.1 we need some preparation. Consider again an obstacle K in Rn and let Ω = ΩK . It follows from results of Melrose and Sjöstrand [MS2] (see also [[H3], Theorem 24.3.9]) that every (ω, θ)-ray γ in ΩK that does not contain gliding segments is a reflecting (ω, θ)-ray, that is it consists of finitely many straight line segments in Ω (two of them are in fact infinite rays). For such a ray γ we have (cf. Section 2.4 and [G1]) Tγ = ω, x1  +

s−1 

||xi − xi+1 || − θ, xs ,

i=1

where x1 , . . . , xs are the successive reflection points of γ and denotes the standard inner product in Rn . Recall from Chapter 2 that a reflecting (ω, θ)-ray γ in Ω is called ordinary if γ has no tangencies to ∂Ω, and an ordinary (ω, θ)-ray γ is called non-degenerate if det(dJγ ) = 0. Proposition 11.1.2: (a) There exists a set R ⊂ Sn−1 × Sn−1 the complement of which is a countable union of compact subsets of measure zero in Sn−1 × Sn−1 such that for every pair (ω, θ) ∈ R all (ω, θ)-trajectories for X = ∂K are ordinary. (b) For every ω ∈ Sn−1 there exists a set S(ω) ⊂ Sn−1 the complement of which is a countable union of compact subsets of Sn−1 of measure zero such that if θ ∈ S(ω), then any ordinary reflecting (ω, θ)-ray in Ω is non-degenerate and any two different ordinary reflecting (ω, θ)-rays in Ω have distinct sojourn times.

300

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

(c) There exists a set S ⊂ Sn−1 × Sn−1 the complement of which is a countable union of compact subsets of measure zero in Sn−1 × Sn−1 such that for every pair (ω, θ) ∈ S all (ω, θ)-trajectories for X = ∂K are ordinary and non-degenerate, and any two different ordinary reflecting (ω, θ)-rays in Ω have distinct sojourn times. We prove this proposition in Section 11.5. Fix a hyperplane Z in Rn such that K is contained in one of the open half-spaces determined by Z. As a simple corollary of our argument in Section 11.5 we get also the following result, which is in fact a consequence of a result of Melrose and Sjöstrand [MS2] (see also Chapter 24 in [H3]). Proposition 11.1.3: There exists T ⊂ Z × Sn−1 , the complement of which is a countable union of compact subsets of measure zero in Z × Sn−1 , such that for every (x, ω) ∈ T the trajectory of the generalized geodesic flow in Ω starting at x in direction ω has no tangencies to ∂K. For convenience of the reader we will now briefly recall some definitions from previous chapters that will be used here. Let Ω be a domain in Rn . Consider the symplectic manifold Sˆ = T ∗ (Ω × R) and n  ξi2 − τ 2 . Since both vector fields Hpˆ and HpG the smooth function pˆ(x, t; ξ, τ ) = ˆ i=1

do not depend on t, we have τ = const along each generalized integral curve of pˆ. The change of τ can only affect the parameterization along a generalized integral curve which is not important for our aim. Thus, we may assume that τ = ±1. There is a natural correspondence between the generalized integral curves of pˆ with this property and the generalized integral curves of the Hamiltonian function p(x, ξ) =

n 

ξi2 − 1

(11.1)

i=1

on the symplectic manifold S = T ∗ (Ω); the correspondence being given by (x(t), ξ(t)) → (x(t), t; ξ(t), ±1). Let Ft be the generalized Hamiltonian flow on T ∗ (Ω) \ {0} generated by the function (11.1). Notice that the submanifold Σ = p−1 (0) of S coincides with the cosphere bundle S ∗ (Ω). The restriction of Ft to S ∗ (Ω) will be called the generalized geodesic flow (K) (11.2) Ft = Ft : S ∗ (Ω) −→ S ∗ (Ω), t ∈ R. Let pr1 : T ∗ (Ω) −→ Ω and pr2 : T ∗ (Ω) −→ Rn be the natural projections. A curve γ in Ω is called a generalized geodesic in Ω if there exist an interval I and

SCATTERING KERNEL FOR GENERIC DIRECTIONS

301

ρ ∈ T ∗ (Ω) such that γ = {pr1 (Ft (ρ)) : t ∈ I}. We will say that γ is a gliding segment on ∂Ω (resp. reflecting ray in Ω) if {Ft (ρ) : t ∈ I} is a gliding trajectory (resp. reflecting trajectory) of p. Next, assume that Ω = ΩK for some obstacle K in Rn and ω, θ ∈ Sn−1 . Let γ = {pr1 (Γ(t)) : t ∈ R}, where Γ : R −→ S ∗ (Ω) is a generalized integral curve of p. The curve γ is called an (ω, θ)-ray in Ω if there exist real numbers a < b such that pr2 (Γ(t)) = ω for t ≤ a and pr2 (Γ(t)) = θ for t ≥ b. If γ is a reflecting ray, that is it does not contain gliding segments on ∂Ω and has only finitely many reflection points, it is called a reflecting (ω, θ)-ray in Ω. By Lω,θ (K) we denote the set of all (ω, θ)-rays in ΩK . Fix an open ball O that contains K. Given ξ ∈ Sn−1 denote by Zξ the hyperplane in Rn orthogonal to ξ and tangent to O such that O is contained in the open half-space Rξ determined by Zξ and having ξ as an inner normal. Given an (ω, θ)-ray γ in Ω, the sojourn time Tγ of γ is defined by Tγ = Tγ − 2a, where Tγ is the length of that part of γ which is contained in Rω ∩ R−θ and a is the radius of the ball O. It is known (cf. [G1] and Section 2.4) that this definition does not depend on the choice of the ball O. Given T > 0, denote by TT the set of those ρ ∈ Σ such that  {Ft (ρ) : 0 ≤ t ≤ T } Gg = ∅, that is the trajectory {Ft (ρ) : 0 ≤ t ≤ T } contains a non-trivial gliding part on ∂S. Recall that if (S, ω) is a symplectic manifold with a two-form ω, a submanifold L of S is called isotropic if Tx L is isotropic for every x ∈ L, that is ωx (u, v) = 0 for all u, v ∈ Tx L. If L is isotropic and has maximal possible dimension, that is dim(L) = 1 2 dim(S), then L is called Lagrangian (see e.g. Chapter 21 in [H3]). For any metric space X denote by dimH (X) the Hausdorff dimension of X (see e.g. [E]). The following proposition is a special case of Theorem 11.4.1 proved below. Proposition 11.1.4: Let S = T ∗ (Ω) and let L0 be an isotropic submanifold of S \ ∂S of dimension n − 1 such that Hp (ρ) is not tangent to L0 at each ρ ∈ L0 . Then for every T > 0 we have dimH FT (TT ∩ L0 ) ≤ n − 2. Moreover, if for a given smooth T we have FT (L0 ) ⊂ S \ ∂S, then there exists a countable family {Im } of  (n − 2)-dimensional isotropic submanifolds of S such that FT (TT ∩ L0 ) ⊂ Im . m

Proof of Theorem 11.1.1: Let K be an obstacle in R of the class K. We are going to show that there exists a subset R of full Lebesgue measure in Sn−1 × Sn−1 such that for each (ω, θ) ∈ R the only (ω, θ)-rays in ΩK are reflecting (ω, θ)-rays. Consider the domain Ω = ΩK , the symplectic manifold S = T ∗ (Ω) and the corresponding generalized geodesic flow (11.2) of the function (11.1). As above, denote by O an open ball in Rn containing the obstacle K and by C the boundary sphere of O. Fix ω ∈ Sn−1 , x0 ∈ C and consider the generalized geodesic (x(t), ξ(t)) = Ft (x0 , ω). Let T > 0 be such that x(T ) ∈ C. Set T = TT and n

S0 = {(x, ξ) ∈ S : x ∈ C, ξ is transversal to C}.

302

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Since Σ = p−1 (0) = S ∗ (Ω), using the notation ∗ (Ω) = {(x, ξ) ∈ S ∗ (Ω) : x ∈ C}, SC

we have ∗ (Ω) : ξ is transversal to C}. S0 = S0 ∩ Σ = {(x, ξ) ∈ SC

Then S0 is a symplectic submanifold of S. Let P : S0 −→ S0 be the local map defined in a neighbourhood of (x0 , ω) using the shift along the flow Ft ; then P(S0 ) ⊂ S0 . Consider the Lagrangian submanifold L0 = {(x, ξ) ∈ S0 : ξ = ω} of S0 . Then L0 is an isotropic submanifold of S. Applying Proposition 11.1.4 to L0 gives that FT (L0 ∩ T ) is contained in a countable union of isotropic (n − 2)-dimensional submanifolds of S. Since locally near (x0 , ω) the map FT : S0 −→ FT (S0 ) is smooth, FT (S0 ) is a (2n − 1)-dimensional submanifold of S transversal to the flow Ft at FT (x0 , ω). Consequently, locally near FT (x0 , ω) ∈ FT (S0 ) ∩ S0 , the shift Q along Ft from FT (S0 ) to S0 is a smooth map. Moreover, Q maps FT (S0 ) into S0 (since p−1 (0) is invariant under the flow Ft ), the restriction Q : FT (S0 ) −→ S0 is a local symplectic map, and P = Q ◦ FT . Hence the set P(L0 ∩ T ) = Q(FT (L0 ∩ T )) is contained in a countable union of isotropic (n − 2)-dimensional submanifolds of S. The projection j : S0 −→ Sn−1 , j(x, ξ) = ξ, is smooth, so Sard’s theorem now gives that the set j(P(L0 ∩ T )) has Lebesgue measure zero in Sn−1 . Hence there exists a neighbourhood U of x0 in C and a subset Rω (U ) = Sn−1 \ j(P(L ∩ T )) of full Lebesgue measure in Sn−1 such that for x ∈ U every generalized (ω, θ)-ray in Ω passing through x with θ ∈ Rω (U ) is a reflecting (ω, θ)-ray. Covering C by a countable family of neighbourhoods Ui , we find a subset Rω = ∩∞ i=1 Rω (Ui ) of full Lebesgue measure in Sn−1 such that every (ω, θ)-ray in Ω with θ ∈ Rω is a reflecting (ω, θ)-ray. It now follows from Fubini’s theorem that  = {(ω, θ) ∈ Sn−1 × Sn−1 : θ ∈ R } R ω is a subset of full Lebesgue measure in Sn−1 × Sn−1 . Moreover, it is clear that for  all (ω, θ)-rays in Ω are reflecting ones. (ω, θ) ∈ R According to Proposition 11.1.2, there exists a subset R of Sn−1 of full Lebesgue measure such that for (ω, θ) ∈ R every reflecting (ω, θ)-ray in ΩK is ordinary and non-degenerate and Tγ = Tδ whenever γ and δ are different reflecting (ω, θ)-rays  we may simply assume that R ⊂ R. ˜ Then, given in ΩK . Intersecting R with R, (ω, θ) ∈ R, it follows from Theorem 9.1.2 that −Tγ ∈ sing supp sK (t, θ, ω) for all γ ∈ Lω,θ (ΩK ). Combining this with Theorem 5.3.2 proves the theorem.



SCATTERING KERNEL FOR GENERIC DIRECTIONS

11.2

303

Generalized Hamiltonian flow

We begin with the definition of Melrose and Sjöstrand [MS1], [MS2] of the generalized Hamiltonian flow (GHF) in the symplectic invariant form given by Hörmander [H3] (see Section 24.3). This is rather similar to what we did in Section 1.2 in the case S = T ∗ (Ω), where Ω ⊂ Rn for some n ≥ 2. Here we deal with the general case of an arbitrary symplectic manifold. Let S be a symplectic manifold with boundary ∂S, dim S = 2n, n ≥ 2, and let p : S −→ R be a smooth (C ∞ ) function. Let ϕ ∈ C ∞ (S) be a defining function of ∂S, that is ϕ > 0 in S \ ∂S, ϕ = 0 on ∂S and dϕ = 0 on ∂S (ϕ might be only locally defined near a compact part of ∂S). The first assumption that we make about S and p is the following. A1. dp|∂S = 0 and {ϕ, {ϕ, p}}(ρ) = 0 whenever ρ ∈ ∂S and {ϕ, p}(ρ) = 0. Here {f, g} denotes the Poisson bracket of f and g. Denote by Hp the Hamiltonian vector field determined by the function p and consider the following subsets of S: G = {σ ∈ S : ϕ(σ) = Hp ϕ(σ) = 0} (glancing set), Gd = {σ ∈ G : Hp2 ϕ(σ) > 0} (diffractive set), Gg = {σ ∈ G : Hp2 ϕ(σ) < 0} (gliding set), Gk = {σ ∈ G : Hpj ϕ(σ) = 0, j = 0, 1, . . . , k − 1} and k G∞ = ∩∞ k=2 G . The gliding vector field HpG on G is defined by HpG = Hp +

Hp2 ϕ H . Hϕ2 p ϕ

In fact, HpG is a well-defined smooth vector field in a neighbourhood of G in S. In order to properly define the GHF, one should be able to define a ‘reflected trajectory’ at a point ρ ∈ ∂S where the flow of Hp hits transversally ∂S. This requires some sort of hyperbolic structure of Hp near such points ρ ∈ ∂S. In what follows we make the following assumption about the symplectic manifold S and the function p: A2. For every point ρ0 ∈ ∂S∩ p−1 (0) there exists an open neighbourhood O of ρ0 in S and symplectic coordinates (x, ξ) = (x1 , . . . , xn ; ξ1 , . . . , ξn ) in O such that ϕ(x, ξ) = x1 in O, that is S ∩ O = {(x, ξ) : x1 ≥ 0}, and such that

∂S ∩ O = {(x, ξ) : x1 = 0},

p(x, ξ) = g(x, ξ)[ξ12 − r(x, ξ )],

(x, ξ) ∈ O,

(11.3)

for some smooth functions g(x, ξ) and r(x, ξ ) with |g(x, ξ)| ≥ a > 0 in O for some constant a > 0.

304

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Here we use the notation x = (x2 , . . . , xn ), ξ = (ξ2 , . . . , ξn ). In all cases known to the authors where the generalized Hamiltonian flow has been involved (e.g. propagation of singularities for second-order linear differential operators; cf. [MS1], [MS2], [H3], [Tay] and the references there), the condition A2 has been satisfied. Notice also that when ρ0 ∈ G, then A2 follows from A1 and the Malgrange preparation theorem (see [MS1] or Section 24.3 in [H3]). In this case we may also assume that the coordinates (x, ξ) are centred at ρ0 , that is ρ0 = (0, 0). The following definition is due to Melrose and Sjöstrand [MS1], [MS2]. Here we consider it in the form given by Hörmander (cf. Definition 24.3.6 in [H3]). Definition 11.2.1: Let I ⊂ R be an interval. A curve Γ : I −→ S is called a generalized integral curve of p if there exists a discrete subset B of I such that: (i) if t ∈ I \ B and Γ(t) ∈ (S \ ∂S) ∪ Gd , then there exists Γ (t) = Hp (Γ(t)); (ii) if t ∈ I \ B and Γ(t) ∈ G \ Gd , then there exists Γ (t) = HpG (Γ(t)); (iii) for each t ∈ B, Γ(t + s) ∈ S \ ∂S for all small s = 0 and there exist the limits Γ(t − 0) = Γ(t + 0), which are points of the same integral curve of ϕ on ∂S. We will only consider integral curves on the zero bicharacteristic set Σ = p−1 (0). For k = 2, 3, . . . denote Gk− = {σ ∈ Gk : Hpk (σ) < 0},

Gk+ = {σ ∈ Gk : Hpk (σ) > 0}.

The third assumption that we make about S and p is the following: A3. Σ



G∞

∞ 

Gk− = ∅.

k=2

In this case one can define a (local) flow (K)

Ft = Ft

: Σ −→ Σ,

t ∈ R,

such that {Ft (σ) : t ∈ R} is an integral curve of p for each σ ∈ Σ. This flow is called the generalized Hamiltonian flow (GHF) generated by p. Let S˜ = S/ ∼ be the quotient space with respect to the following equivalence relation on S: ρ ∼ σ iff either ρ = σ or ρ ∈ S ∩ ∂S, σ ∈ S ∩ ∂S and ρ and σ lie on the same integral curve of ϕ on ∂S. S˜ carries a natural structure of a manifold ˜ the flow F gives rise to another with boundary. Using the natural map π : S −→ S, t ˜ called the compressed Hamiltonian flow. We will consider this flow Ft : S˜ −→ S, ˜ = π(Σ). It follows from Theorem 3.22 in [MS2] that F flow on the invariant set Σ t ˜ is continuous on Σ.

SCATTERING KERNEL FOR GENERIC DIRECTIONS

305

Let Γ : I −→ S be a generalized integral curve of p. We say that Γ is gliding on ∂S if the set of those t ∈ I such that Γ(t) ∈ Gg is dense in I. In this case the trajectory {Γ(t) : t ∈ I} is called a gliding trajectory of p on ∂S. If Γ(I) ∩ Gg = ∅, then Γ is called a reflecting integral curve of p and {Γ(t) : t ∈ I} a reflecting trajectory. Remark 11.2.2: The maps Ft depend on ϕ and in general ϕ is only locally defined near ∂S. However the integral curves of Ft are globally defined and do not depend on the choice of ϕ. Since the behaviour of Ft away from ∂S is trivial (a smooth Hamiltonian flow on a symplectic manifold without boundary), the emphasis here is on the study of Ft near ∂S. The main result in this section is the following. Theorem 11.2.3: The generalized Hamiltonian flow Ft preserves the Hausdorff dimension of Borel subsets of the phase space Σ. We derive this from a technical result whose statement involves Lipschitz maps on subsets of Σ. Without loss of generality, we may assume that S is part of a symplectic manifold V of the same dimension and without boundary and that p is a smooth function on V. Denote by Hp the Hamiltonian vector field on V determined by p and by Φt the corresponding smooth Hamiltonian flow on V. Clearly, if ρ ∈ S \ ∂S and Ft (ρ) ∈ S \ ∂S for all t ∈ I = (a, b), then Ft (ρ) = Φt (ρ) for all t ∈ I. The apparent difference between Ft and Φt is that the latter is smooth and has no reflections at ∂S; in fact the trajectories of Φt can cross ∂S and enter V \ S. Set1 V˜ = V/ ∼, ˜ where ∼ is the same equivalence relation by means of which we defined S. For every point ρ ∈ ∂S there is a symplectic chart O in V with the properties described in A2. (In fact, V can be constructed by gluing such charts around ∂S.) There exists a metric d0 on V that is equivalent to the standard metric ||x − y|| + ||ξ − η|| on each chart O. In what follows d0 will denote a fixed metric on V with this property. There exists a pseudometric d on V such that c min{d0 (ρ , σ ) : π(ρ ) = π(ρ), π(σ ) = π(σ)} ≤ d(ρ, σ) ≤ C min{d0 (ρ , σ ) : π(ρ ) = π(ρ), π(σ ) = π(σ)}

(11.4)

for all ρ, σ ∈ V, where C > c > 0 are constants. Given a coordinate open subset O of S defined by a Darboux chart as in A2 with d0 equivalent to d 0 ((x, ξ), (y, η)) = ||x − y|| + ||ξ − η|| 1 This notation is introduced just for convenience; the set V ˜ does not have any geometric importance and will not be used significantly later.

306

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

on O, we can define a pseudometric d on O, say by d ((x, ξ), (y, η)) = ||x − y|| + ||x1 ξ − y1 η|| + min{||ξ − η||, ||ξ − η||}, where η = (−η1 , η2 , . . . , ηn ). Being the sum of three pseudometrics on O, d is a pseudometric, too. Moreover, (11.4) holds with d0 and d replaced by d 0 and d , respectively. Gluing appropriately the above locally defined pseudometrics, one gets a globally defined pseudometric d on V satisfying the condition (11.4). Then the pro˜ is continuous with jection d˜ of d to V˜ is a metric. Since the projection of Ft to Σ ˜ respect to the metric d ([MS2]), the flow Ft on S is continuous with respect to the pseudometric d. Remark 11.2.4: One can easily see that for any Borel subset B of Σ, dimH (B) calculated with respect to the metric d0 is the same as dimH (B) calculated with respect to the pseudometric d. To check this, it is enough to consider separately three cases: B ⊂ S \ ∂S (trivial since d0 is equivalent to d locally in S \ ∂S), B ⊂ G (trivial since d0 is equivalent to d on G) and B ⊂ ∂S \ G. Consider the last case. Then B = B− ∪ B+ , where B− = {σ ∈ B : σ = limt0 Ft (σ)} and B+ is defined similarly. Then B± are Borel subsets of Σ and it is enough to show that dimH (B± ) is the same with respect to d0 and d. However this follows trivially since d is a metric on each of the sets B± equivalent to d0 . From the last case one also obtains that for any Borel subset B of Σ we have dimH (B) = dimH (π(B)). Given σ ∈ S, denote (σ) = {Ft (σ) : 0 ≤ t ≤ T }. Theorem 11.2.5: Let T > 0. There exists a representation of Σ as a countable union Σ = ∪i∈I Si of Borel subsets Si such that for each i ∈ I there exist an open neighbourhood Vi of Si in V and a family of maps (i)

Gt : Vi −→ V,

0 ≤ t ≤ T,

with the following properties: (i) (a) Gt (σ) = Ft (σ) for all σ ∈ Si and all t ∈ [0, T ]; (b) For every σ ∈ Si and every t ∈ (0, T ], there exists an open neighbourhood (i) W = W (σ, i, t) of σ in Vi such that Gt : (W, d1 ) −→ (V, d2 ) is Lipschitz, where d1 = d0 if σ ∈ (S \ ∂S) ∪ G and d1 = d if σ ∈ ∂S \ G, and similarly d2 = d0 if Ft (σ) ∈ (S \ ∂S) ∪ G and d2 = d if Ft (σ) ∈ ∂S \ G; (c) If σ ∈ Si ∩ [(S \ ∂S) ∪ G] and t ∈ (0, T ] is such that Ft (σ) ∈ (S \ ∂S) ∪ G, then there exists an open neighbourhood W = W (σ, i, t) of σ in Vi such that the (i) map Gt : W −→ V is smooth. If moreover both σ and Ft (σ) are not ends of gliding segments of {Fs (σ) : s ∈ [−, T + ] } for any small  > 0, then W can be chosen (i) in such a way that the restriction of Gt to any smooth cross section in W at σ is a contact transformation.

SCATTERING KERNEL FOR GENERIC DIRECTIONS

307

The latter means that if M is a smooth local submanifolds of W of codimen(i) (i) sion 1 containing σ and transversal to (σ), then Gt |M : M −→ Gt (M) is a contact (canonical) transformation with respect to the standard contact structures on M (i) and Gt (M) inherited from the symplectic structure of V (cf. e.g. Section 5.2 and (i) Proposition 8.1.3 in [AbM]). In particular, M ∩ Σ and Gt (M ∩ Σ) are symplectic submanifolds of V of codimension 2 and the restriction (i)

(i)

Gt : M ∩ Σ −→ Gt (M ∩ Σ) (i)

is a symplectic map. However, in general, Σ is not invariant under Gt , that is (i) Gt (Vi ∩ Σ) is not necessarily a subset of Σ. It follows from [MS2] that for any given σ ∈ Σ the trajectory (σ) has only finitely many transversal reflection points and finitely many gliding segments, so part (c) in the above theorem concerns all but finitely many t ∈ [0, T ]. It can be seen (i) from the proof of Theorem 11.2.5 that Gt is actually a ‘finite combination’ of local Hamiltonian flows in V. Clearly Theorem 11.2.3 would have been trivial (and Theorem 11.2.5 would have been unnecessary for its proof) if the maps Ft were Lipschitz. However, it is well known and easy to see that this is not the case. Locally near a point ρ ∈ S˜ the map Ft is Lipschitz on a neighbourhood of ρ for small |t| when ρ ∈ / ∂S or ρ is a transversal reflection point. Whenever ρ ∈ G, the map Ft is not Lipschitz (cf. [MS1] or [H3]). For example, in the simplest case of a diffractive tangent point ρ ∈ Gd , the map Ft has a singularity of ‘square root type’ at ρ, so it is clearly not Lipschitz. Theorem 11.2.5 is proved in Section 11.3. Proof of Theorem 11.2.3: It is enough to show that for each t the map Ft : Σ −→ Σ does not increase the Hausdorff dimension of Borel subsets; then using the same property for F−t , one concludes that Ft actually preserves dimH . For a similar reason it is enough to consider the case t > 0. Let B be a Borel subset of Σ and let t > 0 be a fixed number. We have to show that dimH (Ft (B)) ≤ dimH (B). From the properties of Hausdorff dimension (cf. e.g. [E]) we have dimH (B) = max1≤i≤3 dimH (Bi ), where B = B1 ∪ B2 ∪ B3 with B1 = B \ ∂S, B2 = B ∩ G, B3 = (B ∩ ∂S) \ G. So, it is enough to prove that dimH (Ft (Bi )) ≤ dimH (Bi ) for i = 1, 2, 3. This essentially means that we have to consider separately three cases: B ⊂ S \ ∂S, B ⊂ G and B ⊂ ∂S \ G. First, assume that B ⊂ S \ ∂S. Take an arbitrary T > t and let Σ = ∪i∈I Si be a representation of Σ as a countable union of Borel subsets Si of S with the properties listed in Theorem 11.2.5. To prove dimH (Ft (B)) ≤ dimH (B), it is enough to show that dimH (Ft (B ∩ Si )) ≤ dimH (B ∩ Si ) for each i, for which in turn it is enough for each σ ∈ B ∩ Si to find an open neighbourhood U of σ in S such that dimH (Ft (B ∩ Si ∩ U )) ≤ dimH (B ∩ Si ∩ U ). Fix for a moment i and σ ∈ B ∩ Si . Then by Theorem 11.2.5(b) there exists a neigh(i) bourhood W of σ in Vi such that Gt : (W, d0 ) −→ (V, d2 ) is Lipschitz, where d2 =

308

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

d or d0 depending on where Ft (σ) belongs. Since d ≤ const d0 , it follows that (i) Gt : (W, d0 ) −→ (V, d) is Lipschitz. Moreover, we can take W such that W is compact and has no common points with ∂S. Then d0 is equivalent to d on W and so (i) (i) Gt : (W, d) −→ (V, d) is Lipschitz. Using this and the fact that Gt = Ft on Si (see condition (a) in Theorem 11.2.5), we get (i)

dimH (Ft (B ∩ Si ∩ W )) = dimH (Gt (B ∩ Si ∩ W )) ≤ dimH (B ∩ Si ∩ W ). Set U = W ∩ Σ; then U is a neighbourhood of σ in Σ having the desired property. This completes the proof in the case B ⊂ S \ ∂S. The case B ⊂ G is very similar to the first one – since d0 is equivalent to d on G, one can use Theorem 11.2.5(b) again as above. Finally, consider the case B ⊂ ∂S \ G. It is enough for each σ ∈ B ∩ Si to find an open neighbourhood U of σ in Σ such that dimH (Ft (B ∩ Si ∩ U )) ≤ dimH (B ∩ Si ∩ U ). By Theorem 11.2.5(b), there exists an open neighbourhood W of σ in Vi (i) (i) (the domain of Gt ) such that Gt : (W, d) −→ (V, d2 ) is Lipschitz, where again (i) d2 = d0 or d2 = d. As in the first case, one concludes that Gt : (W, d) −→ (V, d) is Lipschitz and that dimH (Ft (B ∩ Si ∩ U )) ≤ dimH (B ∩ Si ∩ U ), where U = W ∩ Σ. This completes the proof of dimH (Ft (B)) ≤ dimH (B).



Using Theorem 11.2.3, we will now show that most rays incoming from infinity are not trapped by the obstacle K. ˆ of K. In Denote by ΩKˆ the closure of the complement of the convex hull K the following it is important that we consider points (x, ξ) ∈ S ∗ (ΩKˆ ). In general, ˆ that generate it is not clear at all whether the points (x, ξ) ∈ S ∗ (ΩK ) (with x ∈ K) ∗ bounded trajectories form a set of Lebesgue measure zero in S (ΩK ). The example of M. Livshitz (see Chapter 13) shows that for some obstacles K, there is a non-trivial open set of elements (x, ξ) in S ∗ (ΩK ) that generate bounded (trapped) trajectories. Proposition 11.2.6: If K ∈ K, then the set of those (x, ξ) ∈ S ∗ (ΩKˆ ) such that the trajectory {Ft (x, ξ) : t ≥ 0} is bounded has Lebesgue measure zero in S ∗ (ΩKˆ ). Proof of Proposition 11.2.6: Let K ∈ K, let O be an open ball containing K and C ∗ (Ω), let δ(x, ω) be the be the boundary sphere of O. Set Ω = ΩK . For (x, ω) ∈ SC generalized geodesic in ΩK issued from x in direction ω. Assume that there exists ∗ (Ω) such that δ(x, ω) ⊂ O for all a subset W of positive Lebesgue measure in SC (x, ω) ∈ W . According to Theorem 11.2.5 and Proposition 11.1.2, we may assume that for all (x, ω) ∈ W , the generalized geodesic δ(x, ω) does not contain gliding segments on ∂Ω and has only transversal reflections at ∂K. Given (x, ω) ∈ W , denote by x the first common point of δ(x, ω) with ∂K and by ω the reflected direction of δ(x, ω) at x , that is ω = ω − 2ω, NK (x )NK (x ).

SCATTERING KERNEL FOR GENERIC DIRECTIONS

309

Then the set W = {(x , ω ) : (x, ω) ∈ W } is a subset of positive Lebesgue measure ∗ (Ω). in S∂K ∗ (Ω) for which the billiard ball map B Denote by M the set of those (y, η) ∈ S∂K is well defined (see the general definition of a billiard in Section 4.2; see also Ch. 6 of [CFS]). Moreover, B preserves the so-called Liouville’s measure μ on M which is equivalent to the Lebesgue measure. Next, we use the argument from the proof of the Poincaré recurrence theorem (see [CFS]). It follows from the definition of W that B k (W ) ⊂ M and μ(B k (W )) = μ(W ) > 0 for all k = 0, 1, 2, . . . . On the other hand, in the k

situation under consideration, we clearly have μ(∪∞ k=0 B (W )) < ∞. Therefore, k

there exist non-negative integers k < m with B (W ) ∩ B m (W ) = ∅. Since B is invertible, this means that there exists (x , ω ) ∈ W ∩ B m−k (W ). Then (x , ω ) = B(y, η) for some (y, η) ∈ B m−k−1 (W ) ⊂ M . Now the choice of W and the definition of W show that W has no common points with B(M ). This is  a contradiction that proves the proposition.

11.3 Invariance of the Hausdorff dimension This section is devoted to the proof of Theorem 11.2.5. Throughout this section S will be a 2n-dimensional symplectic manifold and p a smooth function of S satisfying the conditions A1, A2 and A3, and T > 0 will be a fixed real number. Let ρ0 ∈ ∂S \ G. There exist a neighbourhood U of ρ0 in S and T > 0 such that for every ρ ∈ U the trajectory {Ft (ρ) : 0 ≤ t ≤ T } has at most one common point with ∂S that is a transversal reflection point. In fact, taking T > 0 and U sufficiently small, for every ρ ∈ U there exists a unique real number t(ρ) with |t(ρ)| < T such that Ft(ρ) (ρ) ∈ ∂S. Lemma 11.3.1: Under the assumptions above, if the neighbourhood U is taken sufficiently small, then the family of maps Ft : U −→ S,

0 ≤ t ≤ T ,

is uniformly Lipschitz with respect to the pseudometric d on S. That is, there exists a constant C > 0 such that d(Ft (ρ), Ft (σ)) ≤ Cd(ρ, σ) for all ρ, σ ∈ U and all t ∈ [0, T ]. Proof: It is enough to show that the map U  ρ → t(ρ) is uniformly Lipschitz with respect to the pseudometric d. The rest follows from the smoothness of the Hamiltonian flow of Hp , its transversality to ∂S at ρ0 , and the fact that the pseudometric d is equivalent to the metric d0 on any subset W of S such that σ = limt0 Ft (σ) for any σ ∈ W (or σ = limt0 Ft (σ) for any σ ∈ W ). Let O be a coordinate neighbourhood of ρ0 of the type described in A2. Then (0) 2|ξ1 | (0) for every for ρ0 = (x(0) , ξ (0) ) we have ξ1 = 0. Take U so small that |ξ1 | > 3 ρ = (x, ξ) ∈ U . Notice that since |ξ1 | is uniformly bounded from below, we have

310

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

|t(ρ)| ≤ const x1 for all ρ = (x, ξ) ∈ U , where const means a positive constant that does not depend on ρ (and σ later on). Given ρ = (x, ξ) ∈ U and σ = (y, η) ∈ U , we have to show that |t(ρ) − t(σ)| ≤ const d(ρ, σ).

(11.5)

If both t(ρ) and t(σ) are non-negative or non-positive, this follows again from the smoothness of the flow of Hp . Assume t(ρ) > 0 and t(σ) < 0 (the other remaining case is similar). Then ξ1 < 0 and η1 > 0. It follows from the main property of d that |x1 ξ1 − y1 η1 | ≤ const d(ρ, σ). Hence x1 + y1 ≤ const d(ρ, σ), so x1 ≤ const d(ρ, σ) and y1 ≤ const d(ρ, σ). This implies |t(ρ)| ≤ const x1 ≤ const d(ρ, σ), ˜ σ). Thus, (11.5) holds in all possible cases for t(ρ) and similarly |t(σ)| ≤ const d(ρ, and t(σ).  To every ρ ∈ Σ, we will now associate a string α = (k0 , k1 , . . . , km , km+1 ; l0 , l1 , . . . , lm , lm+1 ; q0 , q1 , . . . , qm ; q),

(11.6)

of integers that roughly describes the geometry of the trajectory (ρ). For example, m will be the number of different gliding segments contained in the interior of (ρ), ki and li will be the orders of tangency of (ρ) to ∂S at the initial and terminal point of the ith gliding segment, and qi will be the number of transversal reflections of (ρ) between the ith and the (i + 1)st gliding segments. The numbers k0 , l0 , km+1 , lm+1 will describe the combinatorial type of (ρ) at its initial and terminal points. For example, if ρ ∈ / ∂S, we will have k0 = l0 = −1; if ρ ∈ ∂S \ G, then k0 = l0 = 0; if (ρ) begins with a gliding segment, then k0 and l0 will be the orders of tangency of this segment to ∂S at its initial and terminal points, etc. The pair km+1 , lm+1 will play a similar role at the end of the trajectory (ρ). Finally, 1/q will be (roughly speaking) a lower bound of the distance to the set G at any transversal reflection of (ρ) at ∂S. For the precise definition it is better to start with a given α and define the set of points ρ ∈ Σ whose type is represented by α. Notice that a point ρ ∈ S belongs to a gliding segment if there exist a < b such that 0 ∈ [a, b] and {Ft (ρ) : a ≤ t ≤ b} is a gliding segment on ∂S (cf. Section 11.2). Then ρ ∈ G but in general we do not necessarily have ρ ∈ Gg . However, according to condition A3, we do have ρ ∈ / G∞ , hence ρ ∈ Gk \ Gk+1 for some k ≥ 2. Let (11.6) be a string of integers, where m = m(α) ≥ 0, ki ≥ 3, li ≥ 3 (1 ≤ i ≤ m), k0 , l0 , km+1 , lm+1 ≥ −1, qi ≥ 0 (0 ≤ i ≤ m), and q ≥ 1. We will say that α is admissible if whenever k0 ≤ 1 (resp. lm+1 ≤ 1) we have l0 = k0 (resp. km+1 = lm+1 ) and when k0 ≥ 2 (resp. lm+1 ≥ 2) we have l0 ≥ 2 (resp. km+1 ≥ 2). Definition 11.3.2: Let α be an admissible string of the form (11.6). Denote by Sα the set of those ρ ∈ Σ for which there exist a sequence of real numbers 0 = t0 (ρ) ≤ s0 (ρ) < t1 (ρ) < s1 (ρ) < . . . < tm (ρ) < sm (ρ) < tm+1 (ρ) ≤ sm+1 (ρ) = T (11.7)

SCATTERING KERNEL FOR GENERIC DIRECTIONS

311

with the following properties: (i) For every i = 0, 1, . . . , m the curve {Ft (ρ) : t ∈ [si (ρ), ti+1 (ρ)]} has exactly qi transversal reflections at ∂S and no common points with Gg ; (ii) For all i = 0, 1, . . . , m, m + 1, {Ft (ρ) : t ∈ [ti (ρ), si (ρ)]} is an integral curve of the vector field HpG on G and Ft (ρ) ∈ Gg for almost all t ∈ [ti (ρ), si (ρ)]; (iii) For every i = 1, . . . , m we have Fti (ρ) (ρ) ∈ Gki \ Gki +1 and Fsi (ρ) (ρ) ∈ li G \ Gli +1 ; / ∂S for k0 = −1, ρ ∈ ∂S \ G for (iv) If k0 ≤ 1, then t0 (ρ) = s0 (ρ) = 0 and: ρ ∈ k0 = 0, ρ ∈ G but ρ does not belong to a gliding segment for k0 = 1. If k0 ≥ 2, then ρ belongs to a gliding segment, ρ ∈ Gk0 \ Gk0 +1 and Fs0 (ρ) (ρ) ∈ Gl0 \ Gl0 +1 ; / ∂S for lm+1 = −1, (v) If lm+1 ≤ 1, then tm+1 (ρ) = sm+1 (ρ) = T and: FT (ρ) ∈ FT (ρ) ∈ ∂S \ G for lm+1 = 0, FT (ρ) ∈ G but FT (ρ) does not belong to a gliding segment for lm+1 = 1. If lm+1 ≥ 2, then FT (ρ) belongs to a gliding segment, Ftm+1 (ρ) (ρ) ∈ Gkm+1 \ Gkm+1 +1 and FT (ρ) ∈ Glm+1 \ Glm+1 +1 . (vi) For every t ∈ [0, T ] such that Ft (ρ) ∈ ∂S \ G we have d(Ft (ρ), G) ≥ q1 . One can check that each Sα is a Borel subset of Σ (this can be derived using arguments from the proof of Lemma 11.3.5). Notice that some of the sets Sα may be empty and any ρ ∈ Σ belongs to many (in fact, infinitely many) Sα . Remark 11.3.3: Notice that condition (i) does not exclude the possibility that {Ft (ρ) : t ∈ (si (ρ), ti+1 (ρ))} has some other common points with ∂S apart from the qi transversal reflections. In general ∂S ∩ {Ft (ρ) : t ∈ (si (ρ), ti+1 (ρ))} may be a very complicated set (e.g. a Cantor set). Most of the points in this set (in fact all except the qi transversal reflections) will be points from the set k G∞ ∪ ∪∞ k=2 G+ , which according to condition A3 is far from Gg . Because of this (i) possibility the construction of the maps Gt is a bit more complicated than perhaps anticipated. Lemma 11.3.4: We have Σ = ∪α Sα , where α runs over all admissible strings. Proof: Let ρ ∈ Σ. It follows from [MS1] that {Ft (ρ) : 0 ≤ t ≤ T } has only finitely many transversal reflections at ∂S and finitely many gliding segments on ∂S. Take a small  > 0 and let E = {t ∈ [−, T + ] : Ft (ρ) ∈ Gg } ∩ [0, T ]. Then E is a finite disjoint union of closed subintervals of the interval [0, T ]. If 0 ∈ / E, / ∂S; k0 = l0 = 0 if ρ ∈ ∂S \ G; k0 = l0 = 1 set s0 (ρ) = 0 and k0 = l0 = −1 if ρ ∈ if ρ ∈ G. If 0 ∈ E, then [0, s0 (ρ)] is a connected component of E for some s0 (ρ) ≥ 0 (i.e. ρ belongs to a gliding segment). Consequently, there exist k0 ≥ 2 and l0 ≥ 2 such that

312

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

ρ ∈ Gk0 \ Gk0 +1 and Fs0 (ρ) (ρ) ∈ Gl0 \ Gl0 +1 . Notice that s0 (ρ) > 0 implies l0 ≥ 3 (cf. Section 24.3 in [H3]), while 0 = s0 (ρ) yields k0 = l0 ≥ 3. This defines completely the pair of integers k0 , l0 . In a similar way one defines the pair km+1 , lm+1 . Since (ρ) has only finitely many transversal reflections, there exists an integer q ≥ 1 such that d(Ft (ρ), G) ≥ q1 whenever Ft (ρ) is a transversal reflection point (0 ≤ t ≤ T ). If every connected component of E contains either 0 or T , set m = 0 and α = (k0 , k1 ; l0 , l1 ; q0 ; q), with k0 , l0 and k1 , l1 already defined and q0 being the number of reflections of (ρ). Assume that the union of connected components of E that do not contain 0 or T is not empty; then it has the form ∪m i=1 [ti (ρ), si (ρ)]. For each i = 1, . . . , m, according to assumption A3 again, there exist integers li ≥ 3, ki ≥ 3 such that condition (iii) in Definition 11.3.2 holds. Finally, denote by qi the number of transversal reflections of {Ft (ρ) : si (ρ) < t < ti+1 (ρ)} at ∂S and define α by (11.6). Then ρ ∈ Sα which  proves the assertion. Theorem 11.2.5 follows immediately from the following. Lemma 11.3.5: Let α be an admissible string of the form (11.6). For every ρ ∈ Sα there exist an open neighbourhood V (α, ρ) of ρ in V and a family of maps (α,V ) Gt : V (α, ρ) −→ V, 0 ≤ t ≤ T , such that: (α,V ) (a) Gt (σ) = Ft (σ) for all σ ∈ Sα ∩ V (α, ρ) and all t ∈ [0, T ]; (b) For every σ ∈ Sα ∩ V (α, ρ) and every t ∈ (0, T ] there exists an open neigh(α,V ) bourhood W = W (σ, α, t) of σ in V (α, ρ) such that Gt : (W, d1 ) −→ (V, d2 ) is Lipschitz, where d1 and d2 are as in Theorem 11.2.5(b); (c) If σ ∈ Sα ∩ V (α, ρ) ∩ [(S \ ∂S) ∪ G] and t ∈ (0, T ] are such that Ft (σ) ∈ (S \ ∂S) ∪ G, then there exists an open neighbourhood W = W (σ, α, t) of σ in (α,V ) V (α, ρ) such that the map Gt : W −→ V is smooth. If moreover both σ and Ft (σ) are not ends of gliding segments of {Fs (σ) : s ∈ [−, T + ] } for any  > 0, (α,V ) then W can be chosen in such a way that the restriction of Gt to any smooth local cross section at σ in W is a contact transformation. Proof of Theorem 11.2.5: For each ρ ∈ Sα fix a neighbourhood V (α, ρ) and a family of maps G (α,V ) as in the above lemma. For each α there exists a countable open cover of Sα consisting of sets of the form V (α, ρj (α)), j = 1, 2, . . . . Then the set I of pairs i = (α, j) with α an admissible string of the form (11.6) and j a positive integer is countable. For each i = (α, j) set Vi = V (α, ρj (α)), Si = Sα ∩ Vi and (α,ρ (α)) (i) Gt = Gt j (0 ≤ t ≤ T ). According to Lemma 11.3.5, these objects have all the properties required in Theorem 11.2.5.  The rest of this section is devoted to the proof of Lemma 11.3.5.

SCATTERING KERNEL FOR GENERIC DIRECTIONS

313

Before we go on, let us briefly describe the idea of the construction of the maps . Recall the gliding vector field HpG from Section 11.2. We will slightly change Hp2 ϕ it to make a Hamiltonian vector field in V. The function 2 is well defined and Hϕ p Hp2 ϕ smooth near ∂S. Fix an arbitrary smooth extension f of 2 to V and denote Hϕ p (α,V )

Gt

pˆ = p + f ϕ, thus obtaining another smooth function on V. Notice that pˆ = p on ∂S, so p−1 (0) ∩ ∂S = pˆ−1 (0) ∩ ∂S. Moreover, Hpˆ = Hp + f Hϕ = HpG on ∂S. Denote by Ψt the flow of the Hamiltonian vector field Hpˆ on V. Since the flows Φt and Ψt are smooth on V, the families of maps {Φt }0≤t≤T and {Ψt }0≤t≤T are uniformly Lipschitz on any subset U of V with U compact. (α,V ) . For simplicity consider the case Idea of the construction of the maps Gt α = (0, k1 , 0; 0, l1 , 0; 1, 0; q). Given ρ ∈ Sα , we have 0 = s0 (ρ) < t1 (ρ) < s1 (ρ) < t2 (ρ) = T , and (ρ) has exactly one transversal reflection at time a ∈ (0, t1 (ρ)). Take b and c very close to t1 (ρ) such that 0 < b < a < c < t1 (ρ) and Ft (ρ) ∈ S \ ∂S for all t ∈ [b, a) and t ∈ (a, c]. Consider arbitrary smooth local cross sections B and C in S to Ft containing the points Fb (ρ) and Fc (ρ), respectively. Choosing appropriately small neighbourhoods U1 and W1 of Ft1 (ρ) (ρ) and Fs1 (ρ) (ρ) in V, set M1 = {σ ∈ U1 : Hpk1 −1 ϕ(σ) = 0},

N1 = {σ ∈ W1 : Hpl1 −1 ϕ(σ) = 0};

these are then smooth local cross sections to Ft at Ft1 (ρ) (ρ) and Fs1 (ρ) (ρ), respec(α,V ) tively. On a small neighbourhood V = V (α, ρ) of ρ in V ‘the flow’ Gt = Gt is defined as follows: it carries σ ∈ V along the trajectory Φt (σ) until it hits the hypersurface B; between the hypersurfaces B and C ‘the flow’ Gt coincides with Ft ; between C and M1 , Gt acts as Φt again; between M1 and N1 , Gt coincides with the flow Ψt of Hpˆ; and finally from N1 ‘onwards’ Gt coincides with Φt . As one can see, the idea is quite simple – the action of Gt between any two consecutive distinguished cross-sections (B, C, M1 , N1 ) coincides with the action of one of the flows Φt , Ψt and Ft (the first two being smooth Hamiltonian flows in V). Notice that Φt = Ft near B and C, so there is no loss of smoothness there. The places where we can (and actually do) loose smoothness are the transversal reflections and the cross-sections at the ends of gliding segments (M1 and N1 in our example). One can easily observe that if V is chosen sufficiently small, then Gt (σ) = Ft (σ) whenever σ ∈ Sα ∩ V . We will now give a detailed proof of Lemma 11.3.5. Fix ρ ∈ Σ and an admissible string α of the form (11.6) such that ρ ∈ Sα . We are going to define the neighbourhood V = V (α, ρ) and the family of maps (α,V ) Gt = Gt required in Lemma 11.3.5.

314

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

There are several possible cases for the pairs k0 , l0 and km+1 , lm+1 described in (iv) and (v) in Definition 11.3.2. We will consider in details one of these; the others can be dealt with in the same way with minor modifications at the ends of the trajectory (ρ) (see the end of this section for some details). We will assume that (11.8) k0 = −1, lm+1 ≥ 2. Let ti = ti (ρ), si = si (ρ) be the corresponding numbers from (11.7). The assumption (11.8) implies that (see (iv) and (v) in Definition 11.3.2) 0 = t0 (ρ) = / ∂S and FT (ρ) belongs to a gliding segment. Thus, {Ft (ρ) : tm+1 ≤ t ≤ T } s0 (ρ), ρ ∈ is a gliding segment on ∂S if tm+1 < sm+1 = T , and {Ft (ρ) : T ≤ t ≤ T + } is a gliding segment on ∂S for some  > 0 if tm+1 = sm+1 = T . For every i = 1, 2, . . . , m + 1, ρ ∈ Sα gives Fti (ρ) ∈ Gki \ Gki +1 , thus ki Hp ϕ(Fti (ρ)) = 0. From Definition 11.3.2 it also follows that Ft (ρ) ∈ S \ ∂S for t < ti sufficiently close to ti . This is only possible if Hpki ϕ(Fti (ρ)) < 0 (cf. Section 24.3 in [H3]). In the same way one gets Hpli ϕ(Fsi (ρ)) > 0. Fix small open neighbourhoods Ui of Fti (ρ) (1 ≤ i ≤ m + 1) and Wi of Fsi (ρ) (1 ≤ i ≤ m) in V such that Hpki ϕ(σ) < 0 for σ ∈ Ui (1 ≤ i ≤ m + 1) and Hpli ϕ(σ) > 0 for σ ∈ Wi (1 ≤ i ≤ m) and Hplm+1 ϕ(σ) < 0 for σ ∈ Wm+1 . Define Mi = {ρ ∈ Ui : Hpki −1 ϕ(ρ) = 0},

Ni = {ρ ∈ Wi : Hpli −1 ϕ(ρ) = 0}

for 1 ≤ i ≤ m + 1 and 1 ≤ i ≤ m, respectively. Since {p, Hpki −1 ϕ}(Fti (ρ)) = Hpki (Fti (ρ)) = 0, shrinking the neighbourhood Ui if necessary, we have that Mi is a smooth (2n − 1)-dimensional submanifold of V containing Fti (ρ0 ) and transversal to the flow Ft at this point. Similarly, Ni is a smooth (2n − 1)-dimensional submanifold of V containing Fsi (ρ0 ) and transversal to Ft at Fsi (ρ0 ). It follows from the definition of the numbers ti , si that the part {Ft (ρ) : si ≤ t ≤ ti+1 } of the trajectory of ρ does not contain gliding segments to ∂S and has exactly qi transversal reflections at ∂S. However, it may have some other common points with ∂S (cf. Remark 11.3.3). Our plan is to isolate the times of transversal reflections in small open intervals; then on the rest of [si , ti+1 ], which we denote by Ii (ρ), the trajectory of ρ will be an integral curve of Hp in S and therefore of Φt in V. The latter is smooth and we can use it to define the orbit of Gt over Ii (ρ) for any point σ ∈ V sufficiently close to ρ. (1) (q ) Let i = 0, 1, . . . , m be such that qi > 0 and let ai < . . . < ai i be the times of the transversal reflections of {Ft (ρ) : si ≤ t ≤ ti+1 }. For each j = 1, . . . , qi fix (j) (j) (j) arbitrary numbers bi and ci close to ai such that (1)

(1)

(1)

(2)

(2)

(2)

(qi )

ti < bi < ai < ci < bi < ai < ci < . . . < bi (j)

(j)

(j)

(j)

(qi )

< ai

(qi )

< ci

and Ft (ρ) ∈ S \ ∂S for t ∈ [bi , ai ) ∪ (ai , ci ], j = 1, 2, . . . , qi .

< si+1 ,

SCATTERING KERNEL FOR GENERIC DIRECTIONS

315 (j)

Next, choose arbitrary smooth local (2n − 1)-dimensional submanifolds Bi and (j) (j) (j) Ci of S so that Bi (resp. Ci ) contains Fb(j) (ρ) (resp. Fc(j) (ρ)) and is transversal i i to Hp at Fb(j) (ρ) (resp. at Fc(j) (ρ)). We take these submanifolds in such a way that i

(j)

i

(j)

Bi ∩ ∂S = Ci ∩ ∂S = ∅. Using the continuity of the flows Ft , Φt and Ψt and a simple (backward) induction, we may assume that these local cross-sections are such that: (q

)

(C) the shift along the flow Φt maps Ci i−1 to Mi (1 ≤ i ≤ m with qi > 0); (M) the shift along the flow Ψt maps Mi to Ni (1 ≤ i ≤ m); (1) (N) the shift along the flow Φt maps Ni to Bi if qi > 0 and to Mi+1 if qi = 0 (1 ≤ i ≤ m); (j) (j) (B) the shift along the flow Ft maps Bi to Ci (0 ≤ i ≤ m with qi > 0, 1 ≤ j ≤ qi ). Finally, using again the continuity of Ft , choose an open neighbourhood V = V (α, ρ) of ρ in S (hence in V) such that the shift along the flow Φt maps V (1) onto B0 if q0 > 0. If q0 = 0, we choose V so small that Φt1 (σ) (σ) ∈ M1 for all σ ∈V. Definition 11.3.6: Given σ ∈ V , consider the curve {Gt (σ) : 0 ≤ t ≤ T } in V with the following properties: (i) There exist numbers ti (σ) (1 ≤ i ≤ m + 1) and si (σ) (1 ≤ i ≤ m) with 0 < t1 (σ) < s1 (σ) < . . . < tm (σ) < sm (σ) < . . . < tm+1 (σ) such that Gti (σ) (σ) ∈ Mi for i = 1, . . . , m, m + 1, and Gsi (σ) (σ) ∈ Ni for i = 1, . . . , m. (ii) For every i = 0, 1, . . . , m with qi > 0 and every j = 1, . . . , qi there exist (j) (j) (j) numbers ai (σ), bi (σ) and ci (σ) such that (1)

(1)

(1)

(q )

si (σ) < bi (σ) < ai (σ) < ci (σ) < . . . < bi i (σ) (q )

(q )

< ai i (σ) < ci i (σ) < ti+1 (σ), (j)

(j)

Gb(j) (σ) (σ) ∈ Bi , Gc(j) (σ) (σ) ∈ Ci and Ga(j) (σ) (σ) ∈ ∂S; i i i (iii) {Gt (σ) : t ∈ I} is a trajectory of the following: • Φt for any interval I contained in (j)

(j)

i Ii (σ) = [si (σ), ti+1 (σ)] \ ∪qj=1 (bi (σ), ci (σ))

for some i = 0, 1, . . . , m; • Ψt for I = [ti (σ), si (σ)], i = 1, . . . , m, m + 1; (j)

(j)

(j)

(j)

• Ft for I = [bi (σ), ai (σ)) or I = (ai (σ), ci (σ)] for any i = 0, 1, . . . , m with qi > 0 and j = 1, . . . , qi .

316

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Clearly the definition of Gt can be carried out step by step – first on the interval (1) (1) (1) (1) (2) [0, b0 (σ)] (assuming q0 > 0), then on [b0 (σ), c0 (σ)], [c0 (σ), b0 (σ)], etc. The (j) (j) (j) numbers b0 (σ), a0 (σ), c0 (σ), t1 (σ), s1 (σ), etc., are also defined step by step following the inductive construction. From this procedure, which is effectively described by Definition 11.3.6, one can see that if the neighbourhood V = V (α, ρ) of ρ in V is taken sufficiently small, then the curve {Gt (σ) : 0 ≤ t ≤ T } is well defined for all σ ∈ V . Moreover, we can choose V in such a way that d(Ga(j) (σ) (σ), G) ≥ i

1 , σ∈V 2q

for all i = 0, 1, . . . , m with qi > 0 and j = 1, . . . , qi . Notice that in (i) it may happen that tm+1 (σ) > T . For such σ in the corresponding parts in (iii) the last interval (q ) involved will be [sm (σ), T ] if qm = 0 and [cmm (σ), T  ] if qm > 0.  (j) (j) (j) In what follows we will use the notation Ii (σ) = bi (σ), ci (σ) . Clearly, it makes sense only when qi > 0. Proof of Lemma 11.3.5: We will show that V = V (α, ρ) and the maps Gt = G (α,V ) have the properties listed in Lemma 11.3.5. We are still considering the case (11.8). As promised earlier, at the end of the proof we will say how to deal with the other possible cases. (j) (j) Step 1. We will show that the real-valued functions ti (σ), si (σ), ai (σ), bi (σ), (j) ci (σ) (i ≤ m) and the corresponding points Gti (σ) (σ), Gsi (σ) (σ), Ga(j) (σ) (σ), i Gb(j) (σ) (σ), Gc(j) (σ) (σ) depend smoothly on σ ∈ V . If q0 = 0, then t1 (σ) is just i i the (first) time when the trajectory {Φt (σ) : 0 ≤ t ≤ T } hits the cross-section M1 . Since Φt is a smooth (Hamiltonian) flow in V, it follows that both t1 (σ) and Gt1 (σ) (σ) depend smoothly on σ ∈ V . If q0 > 0, the first number we have to define (1) (1) is b0 (σ). This is the time when {Φt (σ) : 0 ≤ t ≤ T } hits the cross-section B0 , (1) so for the same reason as above, b0 (σ) and Gb(1) (σ) (σ) depend smoothly on σ. 0

(1)

From B0 to ∂S our trajectory follows Ft which in S \ ∂S is a smooth Hamiltonian (1) (1) flow (= Φt in S \ ∂S) transversal to ∂S at Ga(1) (σ) (σ). Hence a0 (σ) − b0 (σ) and 0

(1)

therefore a0 (σ) depend smoothly on σ. This also implies that Ga(1) (σ) (σ) is smooth. (1)

0

The corresponding statement for c0 (σ) follows similarly. Next, suppose we have shown that t1 (σ) and Gt1 (σ) (σ) ∈ M1 depend smoothly on σ ∈ V . From the cross-section M1 to the cross-section N1 , Gt acts as the smooth Hamiltonian flow Ψt in V. Thus, s1 (σ) − t1 (σ) (and therefore s1 (σ)) and Gs1 (σ) (σ) depend smoothly on σ. Proceeding in this way inductively, one completes Step 1. By the same procedure, it follows that tm+1 (σ) is a smooth function of σ ∈ V (α, ρ). However, as mentioned earlier if tm+1 (ρ) = T , then we may have tm+1 (σ) > T for some σ ∈ V (α, ρ) arbitrarily close to ρ. Step 2. We are going to show that Gt = Ft on Sα ∩ V ; this will prove part (a) of Lemma 11.3.5. Let σ ∈ Sα ∩ V . It follows from the choice of the neighbourhood V and the definition of the numbers ti (σ) and si (σ) that

SCATTERING KERNEL FOR GENERIC DIRECTIONS

317

Fti (σ) (σ) ∈ Mi , Fsi (σ) (σ) ∈ Ni . Moreover, the definition of Gt gives that on each interval contained in I0 (σ), Gt acts as the flow Φt . However, σ ∈ Sα implies that Ft (σ) has no transversal reflections or gliding segments on I0 (σ); so on any time interval contained in I0 (σ) the action of the flow F is the same as that of Φ. On (j) the intervals I0 (σ) containing the times of transversal reflections Gt acts as Ft by definition. Therefore, Gt (σ) = Ft (σ) for all t ∈ [0, t1 (σ)]. Next, σ ∈ Sα implies that {Ft (σ) : t ∈ [t1 (σ), s1 (σ)]} is an integral curve of the vector field HpG contained in G ⊂ ∂S. Since Hpˆ = HpG on ∂S, it follows that {Ft (σ) : t ∈ [t1 (σ), s1 (σ)]} is an integral curve of the vector field Hpˆ, too. This agrees with the definition of Gt , so Ft (σ) = Gt (σ) for all t ∈ [0, s1 (σ)]. This and the definition of Gt yield Ft (σ) = Gt (σ) for t ∈ [0, s2 (σ)], etc. Applying the above procedure inductively, we get Ft (σ) = Gt (σ) for all t ∈ [0, T ]. Step 3. Next, we check condition (c) of Lemma 11.3.5. Let σ ∈ Sα ∩ V ∩ [(S \ ∂S) ∪ G] and t ∈ (0, T ] be such that Ft (σ) ∈ (S \ ∂S) ∪ G, and let M be an arbitrary smooth cross-section to the flow Ft with σ ∈ M ⊂ V . First, consider the case 0 < t < t1 (σ). If q0 = 0, then I0 (σ) = [0, t1 (σ)], and so Gs (σ) = Φs (σ) for all s ∈ [0, t1 (σ)]. Moreover for σ in a small open neighbourhood W of σ in V , we have Gs (σ ) = Φs (σ ) for all s ∈ [0, t]. Since Φs is a smooth Hamiltonian flow in V, it follows that Gt : W −→ V is smooth and Gt : M ∩ W −→ Gt (M ∩ W ) is a contact transformation. (j) Let q0 ≥ 1. Then {Fs (σ) : 0 ≤ s ≤ t1 (σ)} has transversal reflections for s = a0 (1) (j = 1, . . . , q0 ) and possibly some other common points with ∂S. For s ∈ [0, b0 (σ)], (1) we have Gs (σ) = Φs (σ), so for t ≤ b0 (σ) the map Gt : V −→ V is smooth and Gt : M −→ Gt (M) is a contact transformation. Also notice that Gb(1) (σ ) (σ ) = Φb(1) (σ ) (σ ) ∈ B0

(1)

0

0

depends smoothly on σ ∈ V and defines a contact transformation from M to B0 . (1) (1) Let b0 (σ) < t ≤ c0 (σ); then the definition of G implies (1)

Gt (σ ) = Ft−b(1) (σ ) ◦ Φb(1) (σ ) (σ ) 0

0

on a small neighbourhood W of σ in V . The only s ∈ [b0 (σ ), c0 (σ )] with (1) Fs (σ ) = Gs (σ ) ∈ ∂S is s = a0 (σ ), the time of the corresponding transversal (1) reflection. Assuming t = a0 (σ), we can take the neighbourhood W so small that Gt (W ) ∩ ∂S = ∅; then Gt is smooth on W and Gt : M ∩ W −→ Gt (M ∩ W ) is a contact transformation. Moreover, (1)

(1)

Gc(1) (σ ) (σ ) = Fc(1) (σ )−b(1) (σ ) ◦ Φb(1) (σ ) (σ ) 0

0

0

0

(1)

is smooth on the whole V and defines a contact transformation between M and C0 . Continuing in this way by induction, one checks that condition (ii) in Lemma 11.3.5

318

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

holds for t < t1 (σ). Apart from that, we get that the map V  σ → Gt1 (σ ) (σ ) ∈ M1 (which is smooth by Step 1) defines a contact transformation between M and M1 . Next, consider the case t1 (σ) < t < s1 (σ). On this time interval Gt acts as the smooth Hamiltonian flow Ψt , so condition (ii) is again trivially satisfied. More precisely, we have Gt (σ ) = Ψt−t1 (σ ) ◦ Gt1 (σ ) (σ ) on a small neighbourhood W of σ in V . Moreover, V  σ → Gs1 (σ ) (σ ) ∈ N1 is smooth and its restriction to M defines a contact transformation. Proceeding in this way, we show that for every t which (in the case under consid(j) eration) is different from ti (σ) (i = 1, . . . , m + 1), si (σ) (i = 1, . . . , m) and ai (σ) (i = 0, 1, . . . , m with qi > 0 and j = 1, . . . , qi ), there exists an open neighbourhood W of σ in V such that Gt : W −→ V is smooth and Gt : M ∩ W −→ Gt (M ∩ W ) is a contact transformation. Step 4. Let us now prove Lemma 11.3.5(b). Let σ ∈ Sα ∩ V and t ∈ (0, T ]. (j) If t is different from all ti (σ), si (σ) and ai (σ), then it follows from the previous step that Gt : W −→ V is smooth for some neighbourhood W of σ in V , thus (possibly shrinking W so that W is contained in the domain of smoothness of Gt ), Gt : (W, d0 ) −→ (V, d0 ) is Lipschitz. (j) Next, assume that t = ai (σ) for some i = 0, 1, . . . , m with qi > 0 and some j = 1, . . . , qi . Then by Definition 11.3.6, Gt (σ ) = Ft−b(j) (σ ) ◦ Gb(j) (σ ) (σ ) i

i

for all σ ∈ V . Since V  σ → Gb(j) (σ ) (σ ) ∈ Bi

(j)

i

is smooth, it is Lipschitz with respect to the metric d0 on every neighbourhood W (j) of σ in V with W compact and contained in V . On the other hand, Bi ∩ ∂S = ∅ (j) shows that d and d0 are equivalent on Bi . Thus, taking W sufficiently small, Lemma 11.3.1 gives that Gt : (W, d0 ) −→ (V, d) is Lipschitz. Finally, assume that t = ti (σ) for some i = 1, . . . , m, m + 1 (the case t = si (σ) is almost identical). Take some τ < t close to t so that Gτ (σ) ∈ S \ ∂S (such τ exists according to Proposition 24.3.8 in [H3]). Then by Step 3, Gτ : W −→ V is smooth for some small neighbourhood W of σ in V . For σ ∈ W , we have Gt (σ ) = Φt−τ ◦ Gτ (σ ) if ti (σ ) ≥ t and Gt (σ ) = Ψt−ti (σ ) ◦ Φti (σ )−τ ◦ Gτ (σ ) if ti (σ ) < t. From this it follows easily that Gt : (W, d0 ) −→ (V, d0 ) is Lipschitz. With this the proof of Lemma 11.3.5 in the case k0 = −1 and lm+1 ≥ 2 is complete. Step 5. Let us now explain how to deal with the other possible cases for k0 and lm+1 . Case 2. k0 = 0, lm+1 ≥ 2. Given ρ ∈ Sα , we have that ρ ∈ ∂S \ G. Take an arbitrary c0 > 0 close to 0 such that {Ft (ρ) : 0 < t ≤ c0 } ⊂ S \ ∂S and a smooth local

SCATTERING KERNEL FOR GENERIC DIRECTIONS

319

cross section C0 at Fc0 (ρ). We take C0 such that C0 ∩ ∂S = ∅. Then d0 is equivalent to d on C0 . Taking a sufficiently small neighbourhood V of ρ in V, we now define Gt slightly changing Definition 11.3.6 in the following way: Gt (σ) = Ft (σ) for t ≤ c0 (σ), where Fc0 (σ) (σ) ∈ C0 . From the cross-section C0 ‘onwards’, we define the action of Gt as in Definition 11.3.6. One proves (a) of Lemma 11.3.5 as in Step 1. To prove (b), consider arbitrary σ ∈ V and t ∈ (0, T ]. Using Lemma 11.3.1 as in Step 4, one shows that if t ≤ c0 (σ), then Gt : (W, d) −→ (V, d) is Lipschitz. Let t > c0 (σ). Then for t (σ ) = t − c0 (σ ) we have Gt (σ ) = Gt (σ ) ◦ Gc0 (σ ) (σ ). For the map σ → Gt (σ ) we can apply the arguments in the previous steps. Since d is equivalent to d0 on C0 , condition (b) of Lemma 11.3.5 follows. Condition (c) does not apply to the case under consideration. Case 3. k0 ≥ 2, lm+1 ≥ 2. Then ρ ∈ Sα implies that ρ belongs to a gliding segment. Taking a small open neighbourhood V of ρ in V, set M0 = {ρ ∈ V : Hpl0 −1 ϕ(ρ ) = 0}. Change Definition 11.3.6 in the following way: for σ ∈ V there exists s0 (σ) (which may be negative if s0 (ρ) = 0) such that Fs0 (σ) (σ) ∈ M0 ; if s0 (σ) > 0, then Gt (σ) = Ψt (σ) for 0 ≤ t ≤ s0 (σ), while for t > s0 (σ) the orbit Gt (σ) is defined as in Definition 11.3.6; if s0 (σ) ≤ 0, then the orbit Gt (σ) is defined as in Definition 11.3.6. One proves (a) and (b) as in the first case with minor modifications. The only difference comes when one deals with condition (b) of Lemma 11.3.5. Now W has to be considered with the metric d0 . The rest is the same. Case 4. k0 = −1, lm+1 = −1. This is in fact the easiest case to deal with. Now ρ ∈ Sα implies that both ρ and FT (ρ) are in S \ ∂S, and we can take V = V (α, ρ) in such a way that V ∩ ∂S = ∅ and FT (V ) ∩ ∂S = ∅. The rest is the same. Case 5. k0 = −1, lm+1 = 0. Similar to Case 2, take a smooth local cross section Bm+1 at some point Fbm+1 (ρ) (ρ), where bm+1 (ρ) is less than but very close to T . We take Bm+1 such that Bm+1 ∩ ∂S = ∅, then d0 is equivalent to d on Bm+1 . We change Definition 11.3.6 so that for any σ ∈ V , Gt acts as Ft on the interval [bm+1 (σ), T ], where Gbm+1 (σ) (σ) ∈ Bm+1 . Given σ ∈ V and t ∈ (0, bm+1 (σ)], the corresponding statements in Lemma 11.3.5 follow immediately from the first case considered. For t > bm+1 (σ) we have Gt (σ ) = Ft−bm+1 (σ ) ◦ Gbm+1 (σ ) (σ ) on a sufficiently small neighbourhood W of σ in V . Moreover, as in the first case, one shows that Gbm+1 (σ ) (σ ) is smooth on V (provided the latter is small enough). Combining this with Lemma 11.3.1, one derives that Gt : (W, d0 ) −→ (S, d) is Lipschitz for a sufficiently small neighbourhood W of σ. If σ ∈ Sα ∩ V and t > bm+1 (σ) are such that Ft (σ) = Gt (σ) ∈ (S \ ∂S) ∪ G, then as in the first case we derive that Gt is smooth on a small neighbourhood W . Cases 6–9. The remaining cases can be easily dealt with combining arguments from the previous cases considered. We leave the details to the reader. 

320

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

11.4

Further regularity of the generalized Hamiltonian flow

Given T > 0, denote by TT the set of those ρ ∈ Σ such that  {Ft (ρ) : 0 ≤ t ≤ T } Gg = ∅, that is the trajectory {Ft (ρ) : 0 ≤ t ≤ T } contains a non-trivial gliding segment on ∂S. In this section we prove the following additional regularity property of the generalized Hamiltonian flow, which implies Proposition 11.1.4 immediately. Theorem 11.4.1: Let L0 be an isotropic submanifold of Σ \ ∂S of dimension n − 1 such that Hp (ρ) is not tangent to L0 at any ρ ∈ L0 . Then for every T > 0 we have dimH (FT (TT ∩ L0 )) ≤ n − 2. Moreover, if for a given T we have FT (L0 ) ⊂ S \ ∂S, then there exists a countable (n − 2)-dimensional isotropic submanifolds of S such that family {Im } of smooth  FT (TT ∩ L0 ) ⊂ Im . m

Remark 11.4.2: The above statement is not true if we replace TT by the set T˜T of those ρ ∈ Σ such that {Ft (ρ) : 0 ≤ t ≤ T } G = ∅. Using simple caustics in the plane, one can easily construct examples when dimH (FT (T˜T ∩ L0 )) = n − 1. Let L0 be an isotropic submanifold of Σ \ ∂S = p−1 (0) \ ∂S of dimension n − 1 such that Hp (ρ) is not tangent to L0 at each ρ ∈ L0 and let T > 0. It is sufficient to consider the case when L0 is contained in a small open neighbourhood of some of its points. That is why we may assume that there exists a (2n − 1)-dimensional submanifold S0 of S which is transversal to Hp and such that S0 = S0 ∩ p−1 (0) is a (2n − 2)-dimensional symplectic submanifold of S containing L0 . The main point is to prove the following local version of Theorem 11.4.1. Lemma 11.4.3: For every admissible string α of the form (11.6) and every ρ ∈ TT ∩ L0 ∩ Sα , there exists an open neighbourhood W = W (α, ρ) of ρ in S such that dimH (FT (TT ∩ L0 ∩ Sα ∩ W )) ≤ n − 2. Moreover, if FT (ρ) ∈ / ∂S, then W can be chosen in such a way that FT (TT ∩ L0 ∩ Sα ∩ W ) is contained in an (n − 2)-dimensional isotropic submanifold of S. We will now use Lemma 11.4.3 to prove Theorem 11.4.1.

SCATTERING KERNEL FOR GENERIC DIRECTIONS

321

Proof of Theorem 11.4.1: Assume that for each string α of the form (11.6) and each ρ ∈ TT ∩ L0 ∩ Sα there exists a neighbourhood W (α, ρ) as stated in Lemma 11.4.3. Since TT ∩ L0 ∩ Sα is a separable metric space, there exists a sequence ρ1 (α), . . . , ρm (α), . . . of elements of TT ∩ L0 ∩ Sα such that TT ∩ L0 ∩ Sα ⊂ ∪∞ m=1 W (α, ρm (α)). Thus, we have FT (TT ∩ L0 ∩ Sα ) ⊂ ∪∞ m=1 FT (TT ∩ L0 ∩ Sα ∩ W (α, ρm (α))), which implies dimH FT (TT ∩ L0 ∩ Sα ) ≤ n − 2. Since TT ∩ L0 ⊂ ∪α (TT ∩ L0 ∩ Sα ), where α runs over the countable set of all configurations of the form (11.6), it now follows that dimH (TT ∩ L0 ) ≤ n − 2. In the case FT (L0 ) ⊂ S \ ∂S, we may assume (according to Lemma 11.4.3) that each FT (TT ∩ L0 ∩ Sα ∩ W (α, ρm (α))) is contained in an (n − 2)-dimensional isotropic submanifold of S. Then FT (TT ∩ L0 ∩ Sα ) is contained in a countable union of (n − 2)-dimensional isotropic sub manifolds of S, and so FT (TT ∩ L0 ) has the same property. For the proof of Lemma 11.4.3 we need the following fact. Proposition 11.4.4: Let N be a symplectic manifold without boundary with dim N = 2k, k ≥ 2, and let E be a symplectic submanifold of N with dim E = 2k − 2. For every Lagrangian submanifold L of N and every ρ0 ∈ L ∩ E there exist an open neighbourhood U of ρ0 in N and a Lagrangian submanifold L of E such that ρ0 ∈ L ∩ E ∩ U ⊂ L . Proof of Proposition 11.4.4: Since the statement is of a local nature, we may assume N = Rk × Rk with the standard symplectic form ω and ρ0 = 0. Given L with 0 ∈ L ∩ E, denote E = T0 E and L = T0 L. Then E is a symplectic linear subspace of N = T0 N = Rk × Rk with dim E = 2k − 2, while L is a Lagrangian subspace of N . For A ⊂ N set A⊥ = {v ∈ N : ω(v, u) = 0 ∀ u ∈ A}. Now the assumptions on L and E imply L = L⊥ and E ∩ E ⊥ = {0}. It then follows that E ⊥ is not contained in L. Indeed, if E ⊥ ⊂ L, then L = L⊥ ⊂ E and therefore E ⊥ ⊂ L ⊂ E which is a contradiction since dim E ⊥ = 2 and E ∩ E ⊥ = {0}. Hence the linear subspace E ⊥ ∩ L is either zero or one dimensional.

322

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Case 1. dim(E ⊥ ∩ L) = 0. Then N = E + L, so E and L are transversal at 0. Hence there exists a neighbourhood U of 0 in N such that L

= E ∩ L ∩ U is a smooth submanifold of L with codimension 2 in L, that is dim L

= k − 2. Being a submanifold of L, L

is isotropic, so (possibly shrinking U) it is contained in a Lagrangian submanifold L of E. Case 2. dim(E ⊥ ∩ L) = 1. Locally near 0, we may assume that E = f −1 (0) ∩ −1 g (0), where f and g are smooth functions such that df (ρ) and dg(ρ) are linearly independent and {f, g}(ρ) = 1 for each ρ in an open ball U with centre 0 in N . Then E ⊥ = span {Xf (0), Xg (0)} and therefore there exist a, b ∈ R, (a, b) = (0, 0), with aXf (0) + bXg (0) ∈ L. We may assume that Xg (0) ∈ L; otherwise one can replace g by an appropriate linear combination of f and g. With this assumption, we have Xg (0) ∈ E ⊥ ∩ L. Since (E + L)⊥ = E ⊥ ∩ L is one dimensional, dim(E + L) = 2k − 1 and therefore dim(E ∩ L) = k − 1. Fix an arbitrary basis v2 , . . . , vk in E ∩ L and set v1 = Xg (0). Then v1 ∈ E ⊥ ∩ L, and E ∩ E ⊥ = {0} implies that v1 , v2 , . . . , vk is a basis in L. Set u1 = Xf (0). Then u1 ∈ E ⊥ gives ω(u1 , vi ) = 0 for all i = 2, . . . , k. Moreover, ω(u1 , v1 ) = ω(Xf (0), Xg (0)) = {f, g}(0) = 1. There exists u2 , . . . , uk ∈ E such that u2 , . . . , uk , v2 , . . . , vk form a symplectic basis in E. From u1 , v1 ∈ E ⊥ we get ω(u1 , ui ) = ω(v1 , ui ) = 0 for all i = 2, . . . , k which shows that u1 , u2 , . . . , uk , v1 , v2 , . . . , vk is a symplectic basis in N . Then shrinking U again if necessary, there exist symplectic coordinates x1 = g, x2 , . . . , xk , ξ1 = f, ξ2 , . . . , ξk such that ui = Xxi (0) = −

∂ , ∂ξi

vi = Xξi (0) =

∂ ∂xi

for all i = 1, . . . , k. In these coordinates we have E ∩ U = {ρ = (x, ξ) ∈ U : x1 = ξ1 = 0}. Moreover,



L = span{v1 , . . . , vk } = span

∂ ∂ ,. . ., ∂x1 ∂xk

 = Rk × {0}.

Therefore, taking U small enough, the Lagrangian submanifold L ∩ U can be written as a graph of a smooth map: L ∩ U = {(x, h(x)) : x ∈ W }, where W is a neighbourhood of 0 in Rk and h(x) = (h1 (x), . . . , hk (x)) is smooth in W . Then ∂hj ∂hi (x) = (x), i, j = 1, . . . , k, x ∈ W. (11.9) ∂xj ∂xi

SCATTERING KERNEL FOR GENERIC DIRECTIONS

323

This follows, for example, from the fact that L ∩ U is the graph of the 1-form β(x) =

k 

hi (x)dxi .

i=1

It is known (cf. e.g. [AbM]) that in such a case, L ∩ U is Lagrangian iff β is closed, that is dβ = 0 on U. Since dβ =

k  i=1

k  n  ∂h  ∂hj ∂hi i dxj ∧ dxi , dhi ∧ dxi = dx ∧ dxi = − ∂xj j ∂xj ∂xi i=1 j=1 j 0 so that the open ball B = BR in Rn with centre 0 and radius R contains X. Given a vector ω ∈ Sn−1 , we will denote by Zω the hyperplane tangent to B and orthogonal to ω such that the half-space determined by Zω and having ω as an inner normal contains X. For a string of the form x = (x1 , . . . , xs ) we will denote by x0 = x0 (x) (resp. xs+1 = xs+1 (x)) the orthogonal projection of x1 on Zω (resp. of xs on Z−θ ). Fix two integers k and s with s ≥ 1 and 0 ≤ k ≤ s. Set Ξs = Sn−1 × X (s) × X × Sn−1 . Consider the set Ξ(s, k) of those ξ = (ω; x; y; θ) with ω, θ ∈ Sn−1 , x = (x1 , . . . , xs ) ∈ X (s) and y ∈ X for which there exists an (ω, θ)-trajectory2 for X with successive (transversal) reflection points x1 , . . . , xs such that the segment [xk , xk+1 ] is tangent to X at the point y ∈ (xk , xk+1 ). Lemma 11.5.1: Ξ(s, k) is a smooth submanifold of Ξs of dimension 2n − 3. Proof of Lemma 11.5.1: We will use arguments similar to some of those used in Chapter 6 – see for example the proof of Theorem 6.3.1. 2

See Section 6.2 for the definition of an (ω, θ)-trajectory.

326

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Consider the open subsets (r)

(r)

Vr (s, k) = {(ω; x; y; θ) ∈ Ξs : xk = xk+1 },

r = 1, . . . , n

of Ξs . Clearly they cover Ξs , that is Ξs = ∪nr=1 Vr (s, k). To prove the lemma it is therefore enough to show that for each r = 1, . . . , n the set Ξr (s, k) = Ξ(s, k) ∩ Vr (s, k) is a smooth submanifold of Ξs of dimension 2n − 3. We will deal in details with the case r = n; the other cases are similar. Consider an arbitrary η = (ω (0) ; x(0) ; y (0) ; θ(0) ) ∈ Ξn (s, k). (0)

Choose smooth charts ϕi : Ui −→ X of X around xi and ψ : V −→ X of X around y (0) such that ϕi (Ui ) ∩ ϕi+1 (Ui+1 ) = ∅, i = 1, . . . , s − 1, ϕk (Uk ) ∩ ψ(V ) = ∅, ϕk+1 (Uk+1 ) ∩ ψ(V ) = ∅ (for k = 0 or s the corre(0) sponding condition is to be deleted). Fix an integer p with ωp = 0. Then on a small neighbourhood of ω (0) , we may parameterize Sn−1 by D1  ω = (ω1 , . . . , ωp−1 , ωp+1 , . . . , ωn ) → ω(ω ) ∈ Sn−1 , where ωp = (1 − |ω |2 )1/2 for some constant  = ±1 and D1 is an open subset of Rn−1 . Similarly, we may assume that Sn−1 is parameterized around θ(0) by θ

= (θ1 , . . . , θq−1 , θq+1 , . . . , θn ) ∈ D2 for some q = 1, . . . , n and some open subset D2 of Rn−1 . In this way we get a chart χ : U = D1 × U1 × . . . × Us × V × D2 −→ D ⊂ Ξs , defined by χ(ξ) = (ω; ϕ1 (u1 ), . . . , ϕs (us ); ψ(v); θ) for ξ = (ω ; u; v; θ

) ∈ U . Here ω = ω(ω ) and θ = θ(θ

), (1) (n−1) ui = (ui , . . . , ui ) ∈ Ui . As we have done several times in Chapter 6, we will now use the length function F : U −→ R defined by F (ξ) =

s−1  i=1

||ϕi (ui ) − ϕi+1 (ui+1 )||.

SCATTERING KERNEL FOR GENERIC DIRECTIONS

327

First, consider the case 0 < k < s. Let ξ = (ω ; u; v; θ

) ∈ U be such that χ(ξ) ∈ Ξn (s, k). Then we have gradui F (ξ) = 0, i = 2, . . . , s − 1,  ∂ϕ1 ϕ2 (u2 ) − ϕ1 (u1 ) − ω, (j) (u1 ) = 0, j = 1, . . . , n − 1, ||ϕ2 (u2 ) − ϕ1 (u1 )|| ∂u1   ϕs (us ) − ϕs−1 (us−1 ) ∂ϕs − θ, (j) (us ) = 0, j = 1, . . . , n − 1, ||ϕs (us ) − ϕs−1 (us−1 )|| ∂us ψ(v) − ϕk+1 (uk+1 ) ψ(v) − ϕk (uk ) + = 0, ||ψ(v) − ϕk (uk )|| ||ψ(v) − ϕk+1 (uk+1 )|| 

and ϕk+1 (uk+1 ) − ϕk (uk ), N (ξ) = 0, ⎛

where

f1

f2

...

⎜ ⎜ ∂ψ (1) ∂ψ (2) ⎜ ⎜ ∂v (1) (v) ∂v (1) (v) . . . ⎜ N (ξ) = det ⎜ .. .. .. ⎜ . . . ⎜ ⎜ ⎝ ∂ψ (2) ∂ψ (1) (v) (v) . . . ∂v (n−1) ∂v (n−1)

fn

⎞

⎟ ∂ψ (n) ⎟ ⎟ (v) ⎟ ∂v (1) ⎟ ⎟ .. ⎟ . ⎟ ⎟ ⎠ ∂ψ (n) ∂v (n−1)

is a normal vector to X at ψ(v). Here f1 , . . . , fn are the standard basis vectors in Rn . To use the above conditions, introduce the functions (j)

Ki (ξ) = 

∂F (j)

∂ui

(ξ),

i = 2, . . . , s − 1, j = 1, . . . , n − 1,

 ∂ϕ1 ϕ2 (u2 ) − ϕ1 (u1 ) − ω, (j) (u1 ) , j = 1, . . . , n − 1, Lj (ξ) = ||ϕ2 (u2 ) − ϕ1 (u1 )|| ∂u1   ϕs (us ) − ϕs−1 (us−1 ) ∂ϕs − θ, (j) (us ) , j = 1, . . . , n − 1, Mj (ξ) = ||ϕs (us ) − ϕs−1 (us−1 )|| ∂us (j)

Pj (ξ) =

(j) ψ (j) (v) − ϕk (uk ) ψ (j) (v) − ϕk+1 (uk+1 ) + , j = 1, . . . , n − 1, ||ψ(v) − ϕk (uk )|| ||ψ(v) − ϕk+1 (uk+1 )||

and Q(ξ) = ϕk+1 (uk+1 ) − ϕk (uk ), N (ξ). Finally, define the map G : U −→ (Rn−1 )s−2 × Rn−1 × Rn−1 × Rn−1 × R

328

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

by (j)

G(ξ) = ((Ki (ξ))1≤j≤n−1 2≤i≤s−1 ; (Lj (ξ))1≤j≤n−1 ; (Mj (ξ))1≤j≤n−1 ; (Pj (ξ))1≤j≤n−1 ; Q(ξ)). Clearly G is smooth and it follows from the above that χ−1 (D ∩ Ξn (s, k)) = G−1 (0). To prove the lemma, we have to establish that G−1 (0) is a smooth submanifold of U of dimension 2n − 3, and, as we have done several times in Chapter 6, to do this it is sufficient to show that G is submersion at any point of G−1 (0). This would then imply that G−1 (0) is a smooth submanifold with dim G−1 (0) = (s + 3)(n − 1) − [(s + 1)(n − 1) + 1] = 2n − 3. Fix ξ ∈ G−1 (0) and assume that s−1  n−1 

(j)

n−1 

(j)

Ai grad Ki (ξ) +

i=2 j=1

+

n−1 

Bj grad Lj (ξ)

j=1

Cj grad Mj (ξ) +

j=1

n−1 

pj grad Pj (ξ) + q grad Q(ξ) = 0 (11.10)

j=1 (j)

for some real coefficients Ai , Bj , Cj , pj , q. We will prove that these constants are zero. Considering in (11.10) the derivatives with respect to ωr , r = p, and using 2 2 − ωp+1 − . . . − ωn2 )1/2 ωp = (1 − ω12 − . . . − ωp−1

for some constant  = ±1, we get   n−1 (r) (p)  ∂ϕ1 ωr ∂ϕ1 Bj − (j) (u1 ) + (u ) = 0 · ωp ∂u(j) 1 ∂u1 j=1 1 for r = p. Clearly (11.11) holds for r = p, as well. Setting c=

n−1 (p) 1  ∂ϕ1 Bj (j) (u1 ), ωp j=1 ∂u1

(11.11) gives cωr =

n−1  j=1

(r)

Bj

∂ϕ1

(j)

∂u1

(u1 ),

r = 1, . . . , n,

(11.11)

SCATTERING KERNEL FOR GENERIC DIRECTIONS

which implies

n−1 

cω =

Bj

j=1

∂ϕ1 (j)

∂u1

(u1 ).

329

(11.12)

So, cω is a tangent vector to X at ϕ1 (u1 ). On the other hand, ξ = (ω; u; v; θ) ∈ G−1 (0) implies χ(ξ) ∈ Ξn (s, k) ⊂ Ξ(s, k), and so ϕ1 (u1 ) is the first (transversal) reflection point of an (ω, θ)-trajectory for X. Hence ω is not tangent to X at ϕ1 (u1 ). Thus, c = 0 and now (11.12) gives B1 = . . . = Bn−1 = 0. Similarly, using the derivatives with respect to θ2 , . . . , θn in (11.10), we find C1 = . . . = Cn−1 = 0. (1) (n−1) To prove Am = . . . = Am = 0 for all m = 2, . . . , k, we will assume k ≥ 2; (j) the case k = 1 is trivial. Now k ≥ 2 implies that the functions Pj , Q and Ki for (r) i ≥ 3 do not depend on the variables u1 . On the other hand,   ϕ2 (u2 ) − ϕ3 (u3 ) ∂ϕ2 ∂F ϕ2 (u2 ) − ϕ1 (u1 ) (j) + , K2 (ξ) = (ξ) = (u ) , (j) ||ϕ2 (u2 ) − ϕ1 (u1 )|| ||ϕ2 (u2 ) − ϕ3 (u3 )|| ∂u(j) 2 ∂u 2

2

so



(j)

∂K2

(r)

∂u1

(ξ) = −a1

∂ϕ1 (r)

∂u1

(u1 ),



∂ϕ2 (j)

∂u2



(u2 ) −

e1 ,

∂ϕ1 (r)

∂u1

 (u1 )

e1 ,

∂ϕ2 (j)

∂u2

 (u2 )

where ai =

ϕ (u ) − ϕi+1 (ui+1 ) 1 , ei = i i . ||ϕi (ui ) − ϕi+1 (ui+1 )|| ||ϕi (ui ) − ϕi+1 (ui+1 )|| (r)

Considering the derivatives with respect to u1 in (11.10), we obtain n−1 

(j) (j) ∂K2 (ξ) (r) ∂u1

A2

j=1

Set w=

n−1 

(j)

A2

j=1

∂ϕ2 (j)

∂u2

= 0.

(11.13)

(u2 ) ∈ Tϕ2 (u2 ) X.

(11.14)

(j)

The expression for 

∂K2

(r) ∂u1

∂ϕ1 (r)

∂u1 Thus,

found above implies that (11.13) can be written in the form 

(u1 ), w



∂ϕ1 (r)

∂u1

 − e1 , w e1 ,



∂ϕ1 (r)

∂u1

(u1 )

 (u1 ), w − e1 , we1

= 0.

= 0.

,

330

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

This is true for all r = 1, . . . , n − 1, so w − e1 , we1 = λN1

(11.15)

for some λ ∈ R, where N1 is an unit normal vector to X at ϕ1 (u1 ). Note that e1 , N1  = 0. Taking inner product of (11.15) with e1 gives 0 = λN1 , e1 , and so λ = 0. Using (11.15) again, w = e1 , we1 . Since ξ ∈ G−1 (0), we cannot have e1 ∈ Tϕ2 (u2 ) X, therefore e1 , w = 0 and so w = 0. Now (11.14) gives (1) (n−1) = 0. A2 = . . . = A2 (j) Using the above procedure several times, by induction one obtains Am = 0 for all m = 2, . . . , k and j = 1, . . . , n − 1. The case k < s − 1 is dealt with similarly – repeating the above argument, con(r) (j) sider the derivatives with respect to us and show that As−1 = 0 for j = 1, . . . , n − 1. (j) In a similar way, by induction one gets Am = 0 for all m = s − 1, s − 2, . . . , k + 1 (j) and j = 1, . . . , n − 1. Therefore, all coefficients Am in (11.10) are zero. We can now write (11.10) in the simpler form n−1 

pj grad Pj (ξ) + q grad Q(ξ) = 0.

(11.16)

j=1

Notice that ψ(v) − ϕk+1 (uk+1 ) ψ(v) − ϕk (uk ) =− = ek . ||ψ(v) − ϕk (uk )|| ||ψ(v) − ϕk+1 (uk+1 )|| Setting b1 =

1 , ||ψ(v) − ϕk (uk )||

we calculate ∂Pj (r)

∂uk

 (ξ) = −b1

b2 =



(j)

∂ϕk

(r)

∂uk

1 , ||ψ(v) − ϕk+1 (uk+1 )||

(uk ) −

ek ,

∂ϕk (r)

∂uk



 (uk )

(j) ek

and ∂Pj (r)

∂uk+1

 (ξ) = −b2



(j)

∂ϕk+1 (r)

∂uk+1

(uk+1 ) −

ek ,

∂ϕk+1 (r)

∂uk+1



 (uk+1 )

(j) ek+1

.

Next, set pn = 0 and p = (p1 , . . . , pn ) ∈ Rn . Considering the derivatives with (r) respect to uk in (11.16), we get  (j)      n−1  ∂ϕk ∂ϕk ∂ϕk (j) pj (uk ) − ek , (r) (uk ) ek − q (uk ), N (ξ) = 0. −b1 (r) (r) ∂uk ∂uk ∂uk j=1

SCATTERING KERNEL FOR GENERIC DIRECTIONS

331

In vector form this is       ∂ϕk ∂ϕk ∂ϕk −b1 p, (r) (uk ) + b1 ek , p ek , (r) (uk ) − q N (ξ), (r) (uk ) = 0, ∂uk ∂uk ∂uk which can be written as  b1 p − ek , b1 pek + qN (ξ),

∂ϕk (r)

∂uk

 (uk )

= 0.

This is true for all r = 1, . . . , n − 1, so b1 p − ek , b1 pek + qN (ξ) = μNk

(11.17)

for some μ ∈ R, where Nk is an unit normal vector to X at ϕk (uk ). Taking the inner product of (11.17) with ek , gives μNk , ek  = qN (ξ), ek  = 0, since the segment [ϕk (uk ), ϕk+1 (uk+1 )] is tangent to X at ψ(v) and N (ξ) is a normal vector to X at ψ(v). Now Nk , ek  = 0 implies μ = 0. Using this back in (11.17), gives b1 p − ek , b1 pek + qN (ξ) = 0.

(11.18)

(r)

Similarly, using the derivatives with respect to uk+1 in (11.16), we get b2 p − ek , b2 pek − qN (ξ) = 0.

(11.19)

Set b = b1 + b2 > 0 and combine (11.18) and (11.19) to get p = ek , pek .

(11.20)

So, p is parallel to ek which combined with (11.18) gives q = 0. Since ξ ∈ G−1 (0), (n) (n) (n) we have χ(ξ) ∈ Ξn (s, k), so ϕk (uk ) = ϕk+1 (uk+1 ) and therefore ek = 0. On the other hand, pn = 0 by definition, so (11.20) implies ek , p = 0. Now (11.20) shows that p = 0. Hence all coefficients in (11.10) are zero. This completes the proof in the case 0 < k < s. The cases k = 0 and k = s can be dealt with in a similar way. We leave the details as an exercise to the reader.  Proof of Proposition 11.1.2(a): Given integers 0 ≤ k ≤ s, the projection πs : Ξs = Sn−1 × X (s) × X × Sn−1 −→ Sn−1 × Sn−1 , πs (ω; x; y; θ) = (ω, θ), is clearly smooth. Since Ξr (s, k) is a smooth submanifold of dimension 2n − 3 < dim(Sn−1 × Sn−1 ), by Sard’s theorem, the set Λr (s, k) = πs (Ξr (s, k)) ⊂ Sn−1 × Sn−1 has measure zero. Clearly, Ξr (s, k) can be written as an union of a finite or countable number of compact sets, so Λr (s, k) is a finite or countable union of compact subsets of Sn−1 × Sn−1 of measure zero. Setting Λ = ∪0≤k≤s ∪nr=1 Λr (s, k), it follows immediately that R = (Sn−1 × Sn−1 ) \ Λ  has the desired properties.

332

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Proof of Proposition 11.1.3: Consider the projections π ˜s : Ξs −→ Sn−1 × X, defined by π ˜s (ω; x1 , . . . , xs ; y; θ) = (ω, x1 ). Using the argument from the proof of Proposition 11.1.2(a), one shows that ˜s (Ξr (s, k)) T = (Sn−1 × X) \ ∪0≤k≤s ∪nr=1 π 

has the desired properties.

We now turn to the proof of Proposition 11.1.2(b). We begin with some additional notation and technical preparation. For any x ∈ X we will denote by N (y) the unit normal to X at x pointing into Ω. Fix an arbitrary ω ∈ Sn−1 and let Z = Zω . For a given integer k ≥ 1 denote by Uk the open subset of Z consisting of the points x ∈ Z generating a trajectory γ(u) of the generalized geodesic flow in Ω which is an ordinary reflecting ray with exactly k reflection points. Let Jk (u) ∈ Sn−1 be the direction of γ(u) after the last reflection. Let k ≥ 1 and s ≥ 1 be two arbitrary integers. For u ∈ Uk denote by fk (u) the sojourn time of the scattering ray determined by the generalized geodesic γ(u). The definition of the set Uk shows that fk : Uk −→ R is a smooth function. Given u ∈ Uk , denote by x1 (u), . . . , xk (u) ∈ X the successive reflection points of γ(u). Clearly,   xk (u) − xk−1 (u) x (u) − xk−1 (u) −2 , N (xk (u)) N (xk (u)) Jk (u) = k ||xk (u) − xk−1 (u)|| ||xk (u) − xk−1 (u)|| and fk (u) =

k−1 

||xi+1 (u) − xi (u)|| + t − 2R,

i=0

where x0 (u) = u and xk+1 (u) denotes the orthogonal projection of xk (u)) on Z−θ with θ = Jk (u). Set t = ||xk (u) − xk+1 (u)||. Since xk − (R − t)θ⊥θ, we have θ, xk + tθ − Rθ = 0. Therefore t = R − θ, xk , and so fk (u) =

k−1 

||xi+1 (u) − xi (u)|| − xk (u), Jk (u) − R.

i=0

Given v ∈ Us , we denote the successive reflection points of γ(v) by y1 (v) , . . . , ys (v); then set y0 (v) = v and denote by ys+1 (v) the orthogonal projection of ys (v) on Z−θ with θ = Js (v). Let Δ(k, s) be the set of those (u, v) ∈ Uk × Us such that Jk (u) = Js (v), fk (u) = fs (v) and rank dJk (u) = rank dJs (v) = n − 1.

SCATTERING KERNEL FOR GENERIC DIRECTIONS

333

Lemma 11.5.2: Δ(k, s) is a smooth (n − 2)-dimensional submanifold of Uk × Us . Proof: The argument here is very similar to that in the proof of Lemma 11.5.1. Fix an arbitrary w(0) = (u(0) , v (0 ) in Δ(k, s). Since rank (dJk (u(0) )) = rank (dJs (v (0 )) = n − 1, there exists a neighbourhood V of w(0) in Uk × Us such that we have rank (dJk (u)) = rank (dJs (v)) = n − 1, (u, v) ∈ V. The map needed here is H : U −→ Rn defined by H(u, v) = (λ(u, v); (χ(j) (u, v))1≤j≤n−1 ), where λ(u, v) = fk (u) − fs (v),

χ(u, v) = Jk (u) − Js (v).

We then have Δ(k, s) ∩ U ⊂ H −1 (0), so we have to show that H is submersion at any point of H −1 (0). To keep the notation simple, we will just show that H is submersion at w(0) ; for all other points in H −1 (0) the argument is the same. Set θ = Jk (u0 ). Without loss of generality we may assume θ(n) = 0. Let n−1 

Aj grad χ(j) (w0 ) + Cgrad λ(w0 ) = 0

(11.21)

j=1

for some constants Aj , C. Set An = 0, A = (A1 , . . . , An ) ∈ Rn and ei =

xi+1 (u0 ) − xi (u0 ) , ||xi+1 (u0 ) − xi (u0 )||

for i = 1, . . . , k − 1. Then for all p = 1, . . . , n − 1 and i = 1, . . . , k − 1 we have   ∂xi+1 ∂ 1 ∂xi xi+1 − xi , ||x − xi ||(u0 ) = (u0 ) − (u ) ∂up i+1 ||xi+1 − xi || ∂up ∂up 0   ∂xi+1 ∂xi = ei , (u0 ) − (u ) . ∂up ∂up 0

334

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Since

∂xi (u ) is tangent to X at xi (u0 ), we have ∂up 0     ∂xi ∂xi ei−1 , (u0 ) = ei , (u0 ) , ∂up ∂up

so      k−1  ∂xi+1 ∂xi ∂xk ∂Jk ∂f ei , (u ) = (u0 ) − (u ) − (u ), J (u ) − xk , (u ) . ∂up 0 ∂up ∂up 0 ∂up 0 k 0 ∂up 0 i=0 This and



   ∂xk ∂xk (u ), J (u ) = ek , (u ) ∂up 0 k 0 ∂up 0

imply      k  k−1   ∂xi ∂xi ∂xk ∂f ei−1 , ei , (u ) = (u ) − (u ) − ek , (u ) ∂up 0 ∂up 0 ∂up 0 ∂up 0 i=1 i=0   ∂Jk (u ) , − xk (u0 ), ∂up 0 and using e0 = ω, we get       ∂x0 ∂Jk ∂Jk ∂f (u ) = − e0 , (u ) − xk (u0 ), (u ) = − xk (u0 ), (u ) . ∂up 0 ∂up 0 ∂up 0 ∂up 0 Since xk (u0 ), θ = 0, it follows that   ∂f ∂Jk ∂λ (u ) = (u ) = − xk (u0 ), (u ) = 0. ∂up 0 ∂up 0 ∂up 0 (j)

On the other hand, clearly ∂χ ∂up (u0 ) = respect to up in (11.21) now gives

(j)

∂Jk ∂up

(u0 ). Considering the derivatives with

  (j) (j) ∂Jk ∂Jk 0= Aj (u ) − C xk (u0 ), (u ) ∂up 0 ∂up 0 j=1       (j) (j) ∂Jk ∂Jk ∂Jk = A, (u ) − C xk (u0 ), (u ) = A − Cxk (u0 ), (u ) ∂up 0 ∂up 0 ∂up 0 n−1 

for all p = 1, . . . , n − 1. Since rank dJk (u0 ) = n − 1, this yields A − Cxk (u0 ) = aθ

(11.22)

SCATTERING KERNEL FOR GENERIC DIRECTIONS

335

for some a ∈ R. Similarly, considering the derivatives with respect to vp in (11.21), one derives A − Cys (v0 ) = bθ (11.23) for some b ∈ R. Combining (11.22) and (11.23), we get C(xk (u0 ) − ys (v0 )) = 0. On the other hand, xk (u0 ) = ys (v0 ), so we must have C = 0, and (11.2) becomes A = aθ. However An = 0 by definition, so aθ(n) = 0 and the assumption θ(n) = 0 implies a = 0. Thus A = aθ = 0, that is A1 = . . . = An−1 = 0. This proves that H  is submersion at w(0) . Proof of Proposition 11.1.2(b): Let again ω ∈ Sn−1 be a fixed vector and let Z = Zω . We will use the sets Uk ⊂ Z, the functions fk and Jk , and Lemma 11.5.2. Given integers k and s, the map hk,s : Uk × Us −→ Sn−1 ,

hk,s (u, v) = Jk (u)

is smooth and by Lemma 11.5.2, Δ(k, s) is a smooth submanifold of Uk × Us with dim Δ(s, k) = n − 2. By Sard’s theorem, hk,s (Δ(s, k)) is a countable union of compact subsets of Sn−1 of measure zero. For any k ≥ 1 consider the closed subset Fk = {u ∈ Uk : rank (dJk (u)) ≤ n − 2} of Uk . Clearly, Fk can be represented as a countable union of compact subsets, Fk = ∪i Fk,i . Using Sard’s theorem again, Jk (Fk ) is a countable union of compact sets of measure zero in Sn−1 . Thus, S (ω) = Sn−1 \ (∪k Jk (Fk ) ∪ ∪k,s hk,s (Δ(k, s))), is a subset of Sn−1 the complement of which is a countable union of compact subsets of Sn−1 of measure zero. So, S (ω) has full measure in Sn−1 . Denote by Gk the set of regular values of the map Jk : Uk −→ Sn−1 . By Sard’s theorem again, Gk has full measure in Sn−1 , so G(ω) = ∩∞ k=0 Gk has full measure in Sn−1 , as well. Hence S(ω) = G(ω) ∩ S (Ω) has full measure in Sn−1 . We claim that S(ω) has the desired properties. First, for any θ ∈ S(ω), we have θ ∈ G(ω). Given an ordinary (ω, θ)-ray γ in Ω, it is incoming through a point u ∈ Uk for some k, and then θ = Jk (u) ∈ Gk shows that rank(dJk (u)) = n − 1, that is γ is non-degenerate. Next, for any θ ∈ S (ω) any two distinct (ω, θ)-rays in Ω have different sojourn times. Indeed, let θ ∈ S (ω). Assume that there exist u = v in Z which determines ordinary reflecting (ω, θ)-rays in Ω with coinciding sojourn times. Then we have / Jk (Fk ), we must have u ∈ Uk and v ∈ Us for some k and s. Since θ = Jk (u) ∈ rank (dJk (u)) = n − 1. Similarly, rank (dJs (v)) = n − 1, so (u, v) ∈ Δ(k, s). Thus, θ = Jk (u) = hk,s (u, v) which is again a contradiction with θ ∈ S (ω). Hence any two different ordinary reflecting (ω, θ)-rays in Ω have distinct sojourn  times.

336

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Proof of Proposition 11.1.2(c): Define S = {(ω, θ) ∈ R : θ ∈ S(ω)}. It follows by the properties of the sets R and S(ω) that the complement of S in Sn−1 × Sn−1 has measure zero and for any (ω, θ) ∈ S all (ω, θ)-trajectories for X are ordinary and non-degenerate and any two different ordinary reflecting (ω, θ)-rays  in Ω have distinct sojourn times.

11.6

Notes

The exposition of Sections 11.1–11.4 follows mostly [S6]. The main Theorem 11.1.1 was proved in [S6] using results from [Pl] (see Theorems 5.3.2 and 9.1.2). In the special case when K is a finite disjoint union of convex bodies Theorem 11.1.1 was proved in [PS7]. Proposition 11.2.6 first appeared in [LP1]. Here we give a different (and more rigorous) proof. Proposition 11.1.2 was proved in [PS5].

12

Scattering kernel for trapping obstacles Let Ω be a closed domain in Rn , n ≥ 2, with bounded complement K and smooth boundary ∂Ω. In this chapter we show that if the obstacle K = Rn \ Ω◦ is trapping and K ∈ K, then there exists a sequence of ordinary reflecting non-degenerate (ωm , θm )-rays γm in Ω with sojourn times Tγm → ∞ such that −Tγm ∈ sing supp s(t, θm , ωm ). Here K is the class of obstacles introduced in the beginning of Section 11.1. We obtain a representation of the scattering amplitude a(λ, θ, ω), introduced in Section 5.1 by the cut-off outgoing resolvent of the Dirichlet Laplacian. From this we deduce a meromorphic continuation of a(λ, θ, ω) in C for n odd and in the logarithmic covering of C for n even. Finally, we introduce weakly non-degenerate trapping rays and examine the estimate of the scattering amplitude if there exists at least one such ray.

12.1

Scattering rays with sojourn times tending to infinity

Let K be a compact subset of Rn , n ≥ 2, with smooth boundary and Ω = Rn \ K. Set Q = R × Ω and denote by (τ, ξ) the variables dual to (t, x) in T ∗ (Q). The characteristic set of the wave operator  has the form Σ = {(t, x, τ, ξ) ∈ T ∗ (Q) \ {0} : τ 2 =| ξ|2 }. Geometry of the Generalized Geodesic Flow and Inverse Spectral Problems, Second Edition. Vesselin M. Petkov and Luchezar N. Stoyanov. © 2017 John Wiley & Sons, Ltd. Published 2017 by John Wiley & Sons, Ltd.

338

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Consider the generalized bicharacteristics of the operator  introduced in Section 1.2. In general, the generalized flow Ft : (0, x0 , 1, ξ0 ) → (x(t), ξ(t)) is not smooth, and in some cases there may exist two different integral curves issued from the same point in the phase space (see [T] for an example). To avoid this situation throughout this chapter, recall that an obstacle K belongs to the class K introduced in Section 11.1, if at any point (x, ξ) ∈ T ∗ (∂K), where the curvature of ∂K vanishes of infinite order in direction ξ, the boundary ∂K is convex in direction ξ. Then every generalized bicharacteristic of  is uniquely extendible and every generalized ray with finite length can be uniformly approximated by ordinary reflecting ones (see Section 7 in [MS2]). Consider the compressed cotangent bundle T˜∗ (Q) introduced in Section 1.4 and the map ∼: T ∗ (Q) (t, x, τ, ξ) → (t, x, τ, ξ|Tx (∂Ω) ) ∈ T ∗ (∂Q) ˜ = Σ is the comdefined as identity on T ∗ (Q \ ∂Q) (see Section 1.2). Recall that Σ b pressed characteristic set, and the image γ˜ =∼ (γ) of a generalized bicharacteristic γ of  is a compressed generalized bicharacteristic. Let ρ0 > 0 be fixed so that K ⊂ B0 = {x ∈ Rn : ||x|| ≤ ρ0 }. Given a point ν = (0, x, 1, ξ) ∈ Σb , (x, ξ) ∈ T ∗ (∂Ω), consider the compressed generalized bicharacteristic γν (t) = (t, x(t), 1, ξ(t)) ∈ T ∗ (Q) of , parameterized by the time t and passing through ν for t = 0. Denote by T (ν) ∈ R+ ∪ ∞ the maximal T > 0 such that x(t) ∈ B0 for 0 ≤ t ≤ T (ν), and denote by Σ∞ the set of those ν = (0, x, 1, ξ) ∈ Σb , (x, ξ) ∈ T ∗ (∂Ω) such that T (ν) = ∞. By using the continuity of the generalized Hamiltonian flow of  (see Section 1.2 and Theorem 3.22 in [MS2]), it is easy to see that Σ∞ is closed in Σb . On the other hand, Σ∞ = Σb . Indeed, take a hyperplane Π tangent to ∂Ω such that K is contained in a half-space determined by Π. Consider an arbitrary ν0 = (0, x0 , 1, ξ0 ) ∈ Σb with (x0 , ξ0 ) ∈ T ∗ (∂Ω), x0 ∈ ∂Ω ∩ Π and ξ0 tangent to ∂Ω. Then we have T (ν0 ) < ∞, since γν0 (t) leaves B0 for t > 2ρ0 . Definition 12.1.1: The obstacle K is trapping if Σ∞ = ∅. For trapping obstacles the boundary ∂Σ∞ of Σ∞ in Σb is not empty. Choose an arbitrary νˆ ∈ ∂Σ∞ . Since Σb \ Σ∞ = ∅, there exists a sequence of points / Σ∞ for all m νm = (0, xm , 1, ξm ) ∈ Σb with (xm , ξm ) ∈ T ∗ (∂Ω) such that νm ∈ and νm → νˆ ∈ Σ∞ . Consider the compressed generalized bicharacteristics γνm (t) = (t, xm (t), 1, ξm (t)) passing through νm for t = 0 and such that T (νm ) < ∞. If the sequence {T (νm )} is bounded, this would imply T (ˆ ν ) < ∞ in contradiction with νˆ ∈ Σ∞ . Therefore, {T (νm )} is unbounded and we may assume T (νm )−→m−→∞ + ∞. Set ym = xm (T (νm )) ∈ ∂B0 , ωm = ξm (T (νm )) ∈ Sn−1 .

SCATTERING KERNEL FOR TRAPPING OBSTACLES

339

Passing to a subsequence, we may assume that ym → zˆ ∈ ∂B0 and ωm → ω ˆ ∈ Sn−1 . Consider the generalized bicharacteristic γμ (t) = (t, y(t), 1, ξ(t)) of  issued from μ = (0, zˆ, 1, ω ˆ ). Then by continuity one obtains T (γμ ) = ∞ and y(t) ∈ B0 for t ≥ 0. As in Section 2.4, let Zωˆ be the hyperplane passing through zˆ and orthogonal to ω ˆ . Denote by Z∞ the set of those points y ∈ Zωˆ such that the generalized bicharacˆ ) has the property T (μy ) = ∞. A simple teristic γμy passing through μy = (0, y, 1, ω argument shows that Z∞ is closed in Zωˆ and clearly Z∞ = Zωˆ . Consequently, there exists a sequence zm → y0 with zm ∈ Zωˆ \ Z∞ for all m such that T (γzm ) < ∞ for all m and T (γzm ) → ∞. In general the bicharacteristic γμzm could contain gliding or glancing segments. According to the results in [MS2] mentioned above and the fact that K ∈ K, every generalized bicharacteristic γμzm can be approximated by multiple ordinary reflecting rays δm with Tδm → +∞. Moreover, applying Proposition 11.2.6, we may assume that δm are unbounded in both directions, that is δm is a   , ωm ) sufficiently close to (zm , ω ˆ ), (ˆ ω , θm )-ray for some θ ∈ Sn−1 . Thus, taking (zm we obtain the following result. Proposition 12.1.2: Let the obstacle K ∈ K be trapping. Then there exists a   , θm )-rays γm such that Tγm −→ ∞. sequence of ordinary reflecting (ωm Now consider an obstacle K having the form K = ∪N j=1 Kj , Ki ∩ Kj = ∅ for i = j,

(12.1)

where Kj is convex for all j = 1, . . . , N . Obviously, K ∈ K and Σ∞ = ∅. Thus, we obtain the following corollary. Corollary 12.1.3: Let the obstacle K have the form (12.1). Then there exists a   , θm )-rays γm such that Tγm −→ ∞. sequence of ordinary reflecting (ωm Notice that Theorem 11.1.1 cannot be applied directly since it is not known   , θm } is in the subset R ⊂ Sn−1 × Sn−1 . Recall that, whether the sequence {ωm following our analysis in Section 9.1, to obtain the singularity of s(t, θ, ω), produced by the sojourn time of an ordinary reflection (ω, θ)-ray γ, we need to know that γ is a non − degenerate ordinary reflecting ray

(12.2)

Tγ = Tδ for every δ ∈ L(ω,θ) (Ω) \ {γ}.

(12.3)

and First we will examine the case when K has the form (12.1). For z ∈ ∂K denote by K(z) the Gauss curvature of ∂K at z. Lemma 12.1.4: Let K have the form (12.1) and let γ be an ordinary reflecting (ω, θ)-ray with reflection points x1 , . . . , xk . Suppose that there exists 1 ≤ j ≤ k such that K(xj ) > 0. Then the ray γ is non-degenerate.

340

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Proof: Using the notation of Section 2.4, for the map Jγ (uγ ) one has the representation dJγ (uγ )u = Mk σk (I + λk Mk−1 )σk−1 (I + λk−1 Mk−2 ) · · · σ2 (I + λ2 M1 )σ1 u given in Proposition 2.4.2. Recall that λi = ||xi−1 − xi ||, i = 1, . . . , k, x0 = uγ , σi is a linear map associated with the symmetry with respect to the tangent plane to ∂K at xi and Mi are symmetric linear maps having the form M1 = ψ˜1 , Mi = σi Mi−1 (I + λi Mi−1 )−1 σi + ψ˜i , i = 2, . . . , k, where ψ˜i is a linear symmetric map depending on the second fundamental form of ∂K at xi . The obstacles Kj are convex, so the maps ψ˜i ≥ 0 are definitive non-negative for every i = 1, . . . , k. By induction this implies that Mi ≥ 0 for i = 1, . . . , k. By assumption we have K(xj ) > 0, hence ψ˜j > 0. One deduces that Mi > 0 for i = j, j + 1, . . . , k.  Therefore, if dJγ (uγ )u = 0, we get u = 0 and the map dJγ (uγ ) is invertible. Next, we consider a fixed ordinary reflecting (ωm , θm )-ray γm which is   , θm ) sufficiently close non-degenerate. We wish to replace (ωm , θm ) by a pair (ωm   , θm )-rays to (ωm , θm ) for which there exist ordinary reflecting non-degenerate (ωm δm such that Tδm satisfy (12.3). By Proposition 11.1.2, we know that we may find   , θm )-rays for which (12.3) holds. an approximation by ordinary reflecting rays (ωm We wish to show that the property of an ordinary ray to be non-degenerate is locally preserved. To do this, we use a corollary of the inverse mapping theorem (cf. [Hl], Theorem 1.1.7). Let U and V be open subsets of Rm and let F : U x → f (x) ∈ V be a C ∞ map. Suppose that x0 ∈ U is such that detdf (x0 ) = 0. Then α = 1/||df (x0 )−1 || > 0, ||·|| being the standard norm in the space of linear maps L(Rm , Rm ). Set y0 = f (x0 ) and choose δ > 0 small enough so that Uδ = {x ∈ Rm : ||x − x0 || < δ} ⊂ U , ||df (x) − df (x0 )|| ≤ α2 for x ∈ Uδ , Vδ = {y ∈ Rm : ||y − y0 || < δα 2 }⊂V. Then it follows by the inverse mapping theorem that the map f is injective on Uδ and surjective on Vδ . Now let x0 = zm and ω0 = ωm . Consider the hyperplane Z = Zω0 . For ω sufficiently close to ω0 , the (ω, θ)-rays issued from y ∈ Zω in direction ω can be considered as suitable (ω, θ)-rays issued from a point x ∈ Z, provided y is close enough to x0 . Thus, we obtain a C ∞ map U = O × Γ (x, ω) → f (x, ω) ∈ Sn−1 , where O ⊂ Z is a small neighbourhood of x0 , Γ ⊂ Sn−1 is a small neighbourhood of ω0 and f (x, ω) is the outgoing direction of the ray issued from x in direction ω. Since γm is non-degenerate by assumption, we have detdf x (x0 , ω0 ) = 0. We may assume that U is chosen so small that detdf x (x, ω) = 0 holds for all (x, ω) ∈ U . Let max ||(df x (x, ω))−1 || =

(x,ω)∈U

1 . α

SCATTERING KERNEL FOR TRAPPING OBSTACLES

341

Then there exists δ > 0 such that for (x, ω) ∈ U with ||x − x0 || < δ, ||ω − ω0 || < δ, we have ||dfx (x, ω) − dfx (x0 , ω0 )|| ≤ 14 α. We may assume that δ is so small that Oδ = {x ∈ Z : ||x − x0 || < δ} ⊂ O, Γδ = {ω ∈ Sn−1 : ||ω − ω0 || < δ} ⊂ Γ. Clearly, for ω ∈ Γδ fixed the map Oδ x → f (x, ω) ∈ Sn−1 is injective. Denote  θ0 = f (x0 , ω0 ) and consider the set Wδ = {θ ∈ Sn−1 : ||θ − θ0 || < δα 4 }. Choose δ ∈ δα (0, δ) so small that ||f (x0 , ω) − θ0 || < 4 for ω ∈ Γδ . Then for ω ∈ Γδ and θ ∈ Wδ , we deduce ||θ − f (x0 , ω)|| < δα 2 . Consequently, applying the version of inverse mapping theorem mentioned above, for each fixed ω ∈ Γδ and each fixed θ ∈ Wδ we can find x(ω,θ) ∈ Oδ with f (x(ω,θ) , ω) = θ. Thus, locally we obtain non-degenerate (ω, θ) ordinary reflecting rays. Combining this with the properties (a) and (b) of Proposition 11.1.2, we obtain the following. Theorem 12.1.5: Let K have the form (12.1). Assume that there are no points z ∈ ∂K such that the Gauss curvature K(u) of ∂K vanishes for every u in some neighbourhood Uz of z in ∂K. Then there exists a sequence of ordinary reflecting ¯ with sojourn times T → ∞ such that for t non-degenerate (ωm , θm )-rays in Ω m near −Tm , we have s(t, θm , ωm ) = Am δ (n−1)/2 (t + Tm ) + lower order singularities with Am = 0. In the case n = 3 the assumption of Theorem 12.1.5 means that there are no points z ∈ ∂K such that the standard metric on ∂K is locally flat around z. To satisfy the condition (12.2), we need to construct a sequence of ordinary reflecting non-degenerate rays. If we have a degenerate ray γm , we must replace it by non-degenerate one with sojourn time sufficiently close to Tγ . Let C = {x ∈ Rn : ||x|| = ρ0 } be the boundary of B0 . We may study the rays issued z, ω ˆ ) ∈ C × Sn−1 from a small neighbourhood W ⊂ C × Sn−1 of the point (ˆ introduced in proof of Proposition 12.1.2. Let O(W ) be the set of all pairs of directions (ω, θ) ∈ Sn−1 × Sn−1 for which there exists an ordinary reflecting (ω, θ)-ray issued from (x, ω) ∈ W with outgoing direction θ ∈ Sn−1 . To establish an approximation with (ω, θ)-rays issued from W , it is useful to know that O(W ) has a positive measure in Sn−1 × Sn−1 for all sufficiently small neighbourhoods z, ω ˆ ). For this purpose, one introduces the following. W ⊂ C × Sn−1 of (ˆ Definition 12.1.6: A generalized bicharacteristic γ issued from (y, η) ∈ C × Sn−1 is called weakly non-degenerate if for every neighbourhood W ⊂ C × Sn−1 of (y, η) the set O(W ) has a positive measure in Sn−1 × Sn−1 . This definition generalizes that of non-degenerate ordinary reflecting ray given in Section 2.4. This follows from the above argument based on the implicit function theorem.

342

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Remark 12.1.7: In general a weakly non-degenerate ordinary reflecting ray does not need to be non-degenerate. In fact, the set of points (y, η) ∈ C × Sn−1 that generate weakly non-degenerate bicharacteristics is closed in C × Sn−1 . As an example, consider the special case when K is convex with vanishing Gauss curvature at some point x0 ∈ ∂K and strictly positive Gauss curvature at any other point of ∂K. Consider a reflecting ray γ in Rn with a single reflection point at x0 . Therefore, it is easy to see that γ is degenerate following the definition in Section 2.4. However, arbitrarily close to γ, we can find an ordinary reflecting ray δm with a single reflection point xm = x0 . Then δm is non-degenerate and hence it is weakly non-degenerate. Thus, γ can be approximated arbitrarily well with weakly non-degenerate rays, and since the set of points generating weekly non-degenerate rays is close, the ray γ itself is weakly non-degenerate. For obstacles with weekly non-degenerate generalized bicharacteristic we have the following. Theorem 12.1.8: Let the obstacle K ∈ K have at least one trapping weakly non-degenerate bicharacteristic δ issued from (y, η) ∈ C × Sn−1 . Then there exists a sequence of ordinary reflecting non-degenerate (ωm , θm )-rays γm with sojourn times Tγm −→ ∞ so that − Tγm ∈ sing supp s(t, θm , ωm ), ∀m ∈ N.

(12.4)

Proof: Let Wm ⊂ C × Sn−1 be a neighbourhood of (y, η) such that for every z ∈ Wm , the generalized bicharacteristic γz issued from z satisfies the condition T (γz ) > m. For every m ∈ N the continuity of the compressed generalized flow guarantees the existence of Wm and one has Wm+1 ⊂ Wm . Consider the open subset Fm of C × Sn−1 × C × Sn−1 consisting of the points (x, ω, z, θ) such that (x, ω) ∈ Wm , and there exists an ordinary reflecting (ω, θ)-ray issued from (x, ω) ∈ Wm and passing through z with direction θ. The projection Fm (x, ω, z, θ) −→ (ω, θ) is smooth and Sard’s theorem implies the existence of a set Dm ⊂ Sn−1 × Sn−1 with measure zero so that if (ω, θ) ∈ / Dm the corresponding (ω, θ)-ray issued from (x, ω) ∈ Wm is non-degenerate. Then the set O(Wm ) \ Dm has a positive measure and taking (ωm , θm ) ∈ O(Wm ) \ Dm , we obtain an ordinary reflecting non-degenerate (ωm , θm )-ray δm with sojourn time Tm issued from zm ∈ Wm . Next we choose q(m) > max{m + 1, Tm }, q(m) ∈ N and repeat the same argument for Wq(m) and Fq(m) . This proves the existence of a sequence of ordinary reflecting non-degenerate (ωm , θm )-rays γm with sojourn times Tγm → ∞. Now let γm be an ordinary reflecting non-degenerate (ωm , θm )-ray issued from C × Sn−1 . Applying (a) and (b) of Proposition 11.1.2, we deduce that for almost all directions (ω, θ) ∈ Sn−1 × Sn−1 , the sojourn times of ordinary reflecting rays

SCATTERING KERNEL FOR TRAPPING OBSTACLES

343

  (ω, θ)-ray are different. Hence we can approximate (ωm , θm ) by directions (ωm , θm ) so that the sojourn times of the corresponding ordinary reflecting non-degenerate   , θm )-rays are different. Next, since γm is non-degenerate, by using the (ωm inverse mapping theorem, it is possible to find ordinary reflecting non-degenerate   , θm )-rays γ  with sojourn time Tγ sufficiently close to Tγm so that (12.2) and (ωm  . This completes the proof.  (12.3) hold for γm

12.2 Scattering amplitude and the cut-off resolvent We start with the representation (5.3) of the scattering kernel given in Section 5.1. Consider the solution of the problem ⎧ 2 ◦ ⎪ ⎨(∂t − Δx )w(t, x; ω) = 0 in R × Ω , w = 0 on R × ∂Ω, (12.5) ⎪ ⎩ w|t 0. We will express vsc (x, λ; ω)|∂Ω by using the operator R(λ). Consider a function ϕ1 ∈ C0∞ (Rn ) such that ϕ1 = 1 on a neighbourhood of K. We get

vsc (x, λ; ω) + ϕ1 (x)e−iλx,ω = −R(λ) (Δ + λ2 )(ϕ1 e−iλx,ω ) = −R(λ)([Δ, ϕ1 ]e−iλx,ω ). Next, choose a function ϕ2 (x) ∈ C0∞ (Rn ) such that ϕ2 (x) = 1 on a neighbourhood of K and ϕ1 (x) = 1 on supp ϕ2 . Then the normal derivative becomes 

∂ ∂vsc  −iλx,ω −iλx,ω ϕ |∂Ω = iλe ν, ω| − R(λ)([Δ, ϕ ]e ∂Ω 2 1 ∂ν ∂Ω ∂ν

SCATTERING KERNEL FOR TRAPPING OBSTACLES

345

and the term iλeiλx,θ−ω ν, ω|∂Ω cancels the same term with sign (−) in (12.8). On the other hand, by using Green’s formula, we obtain 

eiλx,θ (Δ + λ2 ) ϕ2 R(λ)([Δ, ϕ1 ]e−iλx,ω ) dx Ω 

=− eiλx,θ ∂ν ϕ2 R(λ)([Δ, ϕ1 ]e−iλx,ω ) dSx ∂Ω  = eiλx,θ [Δ, ϕ2 ]R(λ)([Δ, ϕ1 ]e−iλx,ω )dx. Rn

To see that the last integral is independent of the choice of ϕ2 , assume that ϕ1 ∈ C0∞ (Rn ) is equal to 1 on the support of χi ∈ C0∞ (Rn ), i = 1, 2, while χi are equal to 1 on K. Let v1 = eiλx,θ , v2 = eiλx,ω . Then the last integral has the form

R(λ)[Δ, χ1 − χ2 ]v1 , [Δ, ϕ1 ]v2 2 L (Ω)

and



R(λ)((Δ + λ2 )(χ1 − χ2 ) − (χ1 − χ2 )(Δ + λ2 ))v1 , [Δ, ϕ1 ]v2 =

− (χ1 − χ2 )v1 , [Δ, ϕ1 ]v2



L2 (Ω)

L2 (Ω)

= 0.

In the same way we can assume that ϕ2 = 1 on the support of ϕ1 so we may switch the conditions on ϕ2 and ϕ1 . Indeed, let ϕ1 = 1 on the support ϕ˜1 , ϕ˜1 = 1 on a neighbourhood of K and let ϕ2 = 1 on the supports of ϕ1 and ϕ˜1 . Then by the same argument, we get v1 [Δ, ϕ2 − ϕ˜1 ]R(λ)[Δ, ϕ1 ]v¯2 = 0 since [Δ, ϕ˜1 ]ϕ1 = 0 and ϕ2 [Δ, ϕ1 ] = 0. Thus, we obtain the following. ¯ and let Proposition 12.2.1: Let ϕi ∈ C0∞ (Rn ), i = 1, 2, be such that ϕi = 1 on K ϕ1 = 1 on the support of ϕ2 or ϕ2 = 1 on the support of ϕ1 . Then the scattering amplitude has the representation  (iλ)(n−3)/2 eiλx,θ [Δ, ϕ1 ]R(λ)[Δ, ϕ2 ]e−iλx,ω dx, λ ∈ R, a(λ, θ, ω) = − 2(2π)(n−1)/2 Ω (12.10) and this representation is independent of the choice of ϕ1 and ϕ2 . Remark 12.2.2: It is easy to see that if we replace in (12.10) the resolvent of the Dirichlet Laplacian R(λ) by the resolvent R0 (λ) = (−Δ − λ2 )−1 of the Laplacian in Rn , then the integral (12.10) vanishes. This follows from the fact that R0 (λ) and Δ commute. Let ψ(x) ∈ C0∞ (Rn ) be a cut-off function such that ψ(x) = 1 on supp ϕi , i = 1, 2. Then we obtain a representation of a(λ, θ, ω) by the cut-off resolvent

346

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Rψ (λ) = ψ(x)R(λ)ψ(x). Since the cut-off resolvent has a meromorphic continuation in C for n odd and in the logarithmic covering of C for n even, we obtain a meromorphic continuation of the scattering amplitude a(λ, θ, ω) with poles in Im z > 0. For ω ∈ Sn−1 introduce the operators  L2comp (Rn ) f → (E± (z, ω)f )(ω) = e±izx,ω f (x)dx ∈ L2 (Sn−1 ), Rn

 L2 (Sn−1 ) g → (t E± (z, ω)g)(x) =

Sn−1

e±izx,ω g(ω)dω ∈ L2loc (Rn ).



The operator (K(λ)f )(θ) =

Sn−1

a(λ, θ, ω)f (ω)dω, λ ∈ R

is called far-field operator. Choosing χ ∈ C0∞ (Rn ) equal to 1 on the supports of ϕi , i = 1, 2, we get the representation (K(λ)f )(θ) = cn (λ(n−3)/2 )t E+ (λ, θ)[Δ, ϕ1 ]R(λ)[Δ, ϕ2 ]E− (λ, ω)f.

(12.11)

The operator S(λ) = Id + K(λ), λ ∈ R is called scattering matrix (see [LP1] for odd dimensions and [PZ], [Z1] for even dimension). For odd dimensions the operator S(λ) is analytic in {z ∈ C : Im z ≤ 0} and has a meromorphic continuation for Im z > 0. For even dimensions one obtains a meromorphic continuation on the logarithmic covering of C. It is possible to show that the scattering matrix S(λ) has the same poles as the operator 2 (Ω) R(λ) : L2comp (Ω) → Hloc and the multiplicities of these poles coincide (see [LP1] for n odd and [Z1], [PZ] for n even). In the following we discuss only the generic case of simple poles and refer to [PZ], [Z1] for multiple poles and their multiplicities. Let z0 ∈ C be a simple pole of R(λ) in a neighbourhood U ⊂ C of z0 and let R(λ) =

A + B(z), z − z0

where A is a rank one operator and the operator-valued function B(z) is analytic in U. Therefore, there exist Φ, Ψ ∈ L2loc (Ω) so that Af = (f, Ψ)L2 (Ω) Φ, ∀f ∈ L2comp (Ω). It is clear that (Δ + z02 )A = 0 and choosing f so that C = (f, Ψ)L2 (Ω) = 0, we get (Δ + z02 )Φ = 0. On the other hand, applying Cauchy integral formula, we have  Af 1 Φ= = R(z)f dz, C 2πiC |z−z0 |=

SCATTERING KERNEL FOR TRAPPING OBSTACLES

347

provided  > 0 small enough. To prove that Φ is z0 -outgoing, we take R > ρ0 large enough and consider |x|≥R R(z)f . Choosing a function χ ∈ C ∞ (Ω) such that χ = 0 for |x| ≤ ρ0 , χ = 0 on supp f and χ = 1 for |x| ≥ R > ρ0 , we deduce |x|≥R R(z)f

=

|x|≥R χR(z)f

=−

|x|>R R0 (z)[Δ, χ]R(z)f ,

provided Im z < 0. By analytic continuation, the above equality will be true for |z − z0 | =  and this yields |x|≥R Φ = |x|≥R R0 (z0 )g(x) 1 with g(x) = − 2πiC [Δ, χ] |z−z0 |= R(z)f dz = [Δ, χ]g1 (x) ∈ L2comp (Ω).

Next, if z is not pole of R(z), the operator R(z) is symmetric with respect to the ¯ and we bilinear form (v, w) ¯ L2 (Ω) , so A must be symmetric, too. This implies Ψ = Φ ¯ have Au = (u, Φ)L2 (Ω) Φ, hence R(z) =

Φ⊗Φ + B(z), z ∈ U. λ − z0

Let g ∈ L2comp (Ω), supp g ⊂ {x ∈ Rn : ρ0 < a ≤ ||x|| ≤ b} and let χ ∈ C0∞ (Ω), χ = 1 on supp g, Then E± (z, ω)g = E± (z, ω)[Δ, χ]R0 (z)g. In fact, for every f ∈ L2 (Sn−1 ) by integration by parts, we get ([Δ, χ]R0 (z)g,t E± (z, ω)f )L2 (Ω) = ((Δ + z 2 )χR0 (z)g,t E± (z, ω)f )L2 (Ω) +(g,t E± (z, ω)f )L2 (Ω) = (E± (z, ω)g, f )L2 (Sn−1 ) . Therefore, K(z) has kernel cn z (n−3)/2

(E+ (z0 , θ)g)(E− (z0 , ω)g) + B1 (z, θ, ω), cn = 0 z − z0

with B1 (z, θ, ω) analytic in U, and for a dense set of directions θ, ω ∈ Sn−1 we have E− (z0 , ω)g = 0, E+ (z0 , θ)g = 0. This implies that K(z) as well as S(z) have simple pole at z0 .

12.3

Estimates for the scattering amplitude

Consider the cut-off resolvent Rψ (λ) with supp ψ ⊂ Ca,b = {x ∈ Rn : 0 < a ≤ ||x|| < b}.

348

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

For λ ∈ R and sufficiently large a N. Burq [Bu3] and Cardoso–Vodev [CV] established the estimate ||Rψ (λ)||L2 (Ω)−→L2 (Ω) ≤

C2 , λ∈R 1 + |λ|

(12.12)

without any geometrical restriction of K. Clearly, this implies |a(λ, θ, ω)| ≤ C0 (1 + |λ|)(n−3)/2 , ∀(θ, ω) ∈ Sn−1 × Sn−1 , λ ∈ R.

(12.13)

On the other hand, if we have poles λj of Rψ (λ) converging sufficiently fast to the real axis, the norm ||χR(λ)χ||L2 (Ω)−→L2 (Ω) with an arbitrary cut-off function χ ∈ C0∞ (Rn ), χ = 1 on K may increase like O(eC|λ| ) for λ ∈ R, |λ| −→ ∞. (see [Bu2]). This happens if there exists an elliptic periodic ordinary reflecting ray with Poincaré map satisfying some technical conditions (see [Po3] for the existence of quasi-modes and [SV] for the existence of poles). Consequently, the existence of trapping rays influences the estimates of Rχ (λ) = χR(λ)χ with χ(x) equal to 1 on a neighbourhood of the obstacle, and the behaviours of the scattering amplitude a(λ, θ, ω) and Rχ (λ) with arbitrary χ are rather different for λ ∈ R if we have trapping rays. By using the notation of Section 12.1, an obstacle K ∈ K is called non-trapping if Σ∞ = ∅. From the results on propagation of singularities given in [MS1], [MS2], it follows that if K ∈ K is non-trapping, there exist  > 0 and d > 0 so that Rχ (λ) has no poles in the domain U ,d = {λ ∈ C : 0 ≤ Im λ ≤  log (1 + |λ|) − d}. Moreover, for non-trapping obstacles we have the estimate (see [Va]) ||Rχ (λ)||L2 (Ω)−→L2 (Ω) ≤

C eC|Im λ| , ∀λ ∈ U ,d . 1 + |λ|

We conjecture that the existence of singularities tm −→ −∞ of the scattering kernel s(t, θm , ωm ) for suitable directions (θm , ωm ) ∈ Sn−1 × Sn−1 implies that for every  > 0 and d > 0 we have poles of the cut-off resolvent in U ,d . In general, without any information of the geometry of trapping rays, this is a difficult problem. Here we prove a weaker result assuming an estimate of the scattering amplitude. Theorem 12.3.1: Let K ∈ K and let n be odd. Suppose that there exist m ∈ N, α ≥ 0,  > 0, d > 0 and C > 0 so that a(λ, θ, ω) is analytic in U ,d and |a(λ, θ, ω)| ≤ C(1 + |λ|)m eα|Im λ| , ∀(ω, θ) ∈ Sn−1 × Sn−1 , ∀λ ∈ U ,d . (12.14) ¯ Then there are no trapping weakly non-degenerate bicharacteristics in T ∗ (Ω). For the proof we need the following.

SCATTERING KERNEL FOR TRAPPING OBSTACLES

349

Lemma 12.3.2: Let u ∈ S  (R) be a distribution with supp u ⊂ {t ∈ R : t ≤ τ }. Assume that the Fourier transform u ˆ(λ) = u, eitλ D (R) , Im λ < 0 admits an analytic continuation in the domain U ,d such that for all ζ ∈ U ,d we have |u ˆ(ζ) |≤ C(1+ | ζ |)N eα|Imζ| , α ≥ 0.

(12.15)

Then for each q ∈ N there exist tq < τ and vq ∈ C q (R) such that u = vq for t ≤ tq . Proof: Choose a function ϕ ∈ C0∞ (R) such that  supp ϕ ⊂ (−1, 1), ϕ(t)dt = 1. R

Set ϕδ (t) =

1 t δ ϕ( δ ),

0 < δ ≤ 1, and consider u ∗ ϕδ . Introduce the path

Γ : R \ [−γ, γ] ξ → ζ = ξ + i(d −  log (1+ | ξ |) ∈ U ,d , where γ = exp ( d ) − 1 and d is given in the definition of U ,d . Clearly, (12.15) implies |u ˆ(ζ)ϕ(δζ) ˆ |≤ CM (1+ | ζ |)N (1+ | δζ |)−M e(a+δ)|Im ζ| , ζ ∈ U ,d . By using the analyticity of u ˆ(ζ) in U ,d combined with the estimate obtained above for fixed δ, we write the integral  ˆ(ξ)ϕ(δξ)dξ ˆ (u ∗ ϕδ )(t) = (2π)−1 eitξ u R

as a sum of two integrals   −1 itζ −1 e u ˆ(ζ)ϕ(δζ)dζ ˆ + (2π) (2π)

eitζ u ˆ(ξ)ϕ(δξ)dξ. ˆ

|ξ|≤γ

Γ

Since the second integral is over a compact interval, passing to a limit as δ → 0, we get a C ∞ function. Next, for ζ ∈ Γ , 0 < δ ≤ 1 we have the estimate itζ |u ˆ(ζ)ϕ(δζ)e ˆ |≤ C(1+ | ζ |)N e−(a+1+t)Im ζ ≤ C  (1+ | Re ζ |)N + (a+1+t) ,

provided a + 1 + t < 0. Given q ∈ N, choose tq = Then t ≤ tq implies

1 (−N − n − 1 − q − (a + 1)). 

(a + 1 + t) ≤ −N − n − 1 − q

350

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

and, since dζ = F (ξ)dξ → dξ as | ξ |→ ∞, for t ≤ tq the integral on Γ is uniformly convergent for 0 ≤ δ ≤ 1. The same is true if we take the derivatives with respect to t up to order q. Taking the limit δ → 0 and exploiting the fact that ϕ(δζ) ˆ → 1, by Lebesgue theorem we deduce that for t ≤ tq we have u ∗ ϕδ −→δ→0 f , f being a C q function. This completes the proof of the lemma.  Proof of Theorem 12.3.1: If K ∈ K is trapping and has a weakly non-degenerate trapping bicharacteristic, we can apply Theorem 12.1.8. Then we have delta-type singularities tm → −∞ of um (t) = s(t, θm , ωm ). From (12.15), we obtain a uniform estimate with respect to (θm , ωm ) for the Fourier transform of um (t). For q = 0 we choose t0 = 1 (−N − n − 1 − (a + 1)). Next, we take m large enough so that  tm < t0 and fix m. Applying Lemma 12.3.2, we obtain a contradiction. Since S(λ) − Id is a Hilbert–Schmidt operator with kernel a(λ, θ, ω), a suitable estimate for the scattering amplitude in a domain U in C, where we have no poles of the cut-off resolvent, will imply an estimate for the scattering matrix for λ ∈ U. On the other hand, for the estimate of the scattering amplitude we need an estimate for the norm || Ca,b R(λ) Ca,b ||L2 (Ω)−→L2 (Ω) , λ ∈ U, provided a > ρ0 large enough (see [BP2] for the estimates of the above norm). If an obstacle K is trapping and there are no sequences of poles λj with limj→∞ Im λj = 0, then there exists δ > 0 such that in Vδ = {z ∈ C : 0 ≤ Im z ≤ δ : |z| ≥ a0 > 0} there are no poles. One expects that under this condition the scattering amplitude admits an exponential estimate, that is for 0 < μ < δ there exists constants Cμ > 0, C ≥ 0 so that |a(λ, θ, ω)| ≤ Cμ eC|λ| , ∀(θ, ω) ∈ Sn−1 × Sn−1 , ∀λ ∈ Vμ .

(12.16)

For dimension n = 2 the estimate (12.16) has been established in [BP1] without any condition of trapping rays. For dimensions n ≥ 3 this is an interesting open problem.

12.4

Notes

The results in Section 12.1 have been obtained in [PS6], [PS8], [PS9] and [PS10], and the notion of weakly non-degenerate trapping bicharacteristic has been introduced in [PS9]. The results in Section 12.2 concerning the maximal singularity of s(t, −ω, ω) are obtained by [Ma2] (see [P5] for a similar result). The Proposition 12.2.1 has been proved in [PZ]. The link between the poles of the cut-off resolvent and those of the operator S(λ) has been studied in many papers. We refer to the classical book [LP1] for odd dimension and to [Z2] for an analysis covering all dimensions n ≥ 2. Lemma 12.3.2 was proved in [PS8] and our argument follows the proof of Theorem 7.3.8 in [Hl]. Theorem 12.3.1 was established in [PS10].

13

Inverse scattering by obstacles In this chapter we discuss the inverse problem of recovering information about an obstacle from the singularities of the scattering kernel. As we already know, this scattering data is closely related to the set of sojourn times of scattering rays in the exterior of the obstacle, the so-called scattering length spectrum (SLS). We will in fact try to recover information about the obstacle from its SLS. The first observation that we make, and it is rather important, is that if two obstacles K and L have (almost) the same scattering length spectra, then the generalized geodesic flows in their exteriors are naturally conjugated on the non-trapping parts of their phase spaces via a time-preserving conjugacy. This is explained in Section 13.1 and the proof of the main result is given in Section 13.2. In subsequent sections we use this result to show that certain properties of obstacles are recoverable from the SLS, and also that some classes of obstacles can be uniquely recovered from their SLS.

13.1

The scattering length spectrum and the generalized geodesic flow

Let K be an obstacle in Rn , n ≥ 2, that is a compact subset of Rn with C ∞ boundary ∂K such that ΩK = Rn \ K is connected. Given (ω, θ) ∈ Sn−1 × Sn−1 , let SLK (ω, θ) be the set of sojourn times Tγ of all (ω, θ)-rays γ in ΩK . The SLS of K is by definition the map Sn−1 × Sn−1  (ω, θ) → SLK (ω, θ) ⊂ [0, ∞). As we know from Chapter 11, for ‘most’ obstacles K we have SLK (ω, θ) = sing supp sK (t, θ, ω) Geometry of the Generalized Geodesic Flow and Inverse Spectral Problems, Second Edition. Vesselin M. Petkov and Luchezar N. Stoyanov. © 2017 John Wiley & Sons, Ltd. Published 2017 by John Wiley & Sons, Ltd.

352

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

for almost all (ω, θ), where sK is the scattering kernel related to the scattering operator for the wave equation in R × ΩK with Dirichlet boundary condition on R × ∂ΩK . It is a rather important problem in inverse scattering by obstacles to get information about the geometry of the obstacle K from its SLS. It is well known  of K can be recovered from SL ; this has and easy to see that the convex hull K K been noted by various people –Majda [Ma2] (see also Majda and Ralston [MaR]) and Lax and Phillips [LP2]. Indeed, one can recover all supporting hyperplanes to K by using back-scattering only, that is pairs (ω, θ) with θ = −ω (see our comments in Section 12.2). In fact, since we have to work with generic (ω, θ) ∈ Sn−1 × Sn−1 , we need to use pairs (ω, θ) with θ very close to −ω. An obvious consequence from this is that convex obstacles K are completely determined by its SLS. Similarly, connected obstacles with real analytic boundaries are also uniquely determined by their SLS (in the class of obstacles with real analytic boundaries). However, as the next example shows, in general SLK does not determine K uniquely. Example 13.1.1: (Livshits’ Example, adapted from Chapter 5 of [Me4]) Consider the obstacle K in R2 bounded by the closed curve in Figure 13.1. Here the curve E is half an ellipse with end points A and B, while F1 and F2 are the foci of the ellipse. According to a well-known property of the ellipse, if a ray enters the area inside the ellipse between the foci F1 and F2 , after reflection at E, it will go out again between the foci. Thus, no scattering ray ‘coming from infinity’ has a common point with the bold lines from A to F1 and from B to F2 . As one can see in Livshits’ example, there are points in ΩK that cannot be reached by scattering rays ‘incoming from infinity’ and ‘outgoing to infinity’ after a finite amount of time spent ‘near the obstacle’. In fact, there is a whole open subset of S ∗ (ΩK ) consisting of points σ = (x, ξ) which do not belong to an (ω, θ)-ray for any (ω, θ). Such points are called trapped points.

E

A

F1

Figure 13.1

F2

Livshits’ example.

B

INVERSE SCATTERING BY OBSTACLES

353

More precisely, considering again an arbitrary obstacle K, a point σ = (x, ξ) ∈ S ∗ (ΩK ) is called non-trapped if both curves (K)

and

+ (σ) = {pr1 (Ft γK

(σ)) : t ≤ 0}

− γK (σ) = {pr1 (Ft

(σ)) : t ≥ 0}

(K)

in ΩK are unbounded in Rn . Otherwise σ is called a trapped point and σ belongs to the set Σ∞ introduced in Section 12.1. As before, here we use the notation pr1 (y, η) = y,

pr2 (y, η) = η.

Denote by Trap(ΩK ) the set of all trapped points. As we have already mentioned, Livshits’ example shows that in general Trap(ΩK ) may have positive Lebesgue measure and a non-empty interior in S ∗ (ΩK ). K is called a non-trapping obstacle if Trap(ΩK ) = ∅. For example, as we will see in Section 13.3, all star-shaped obstacles the curvature of whose boundaries does not vanish of infinite order are non-trapping. For any σ = (x, ξ) ∈ S ∗ (ΩK ) we will also use the notation (K)

γK (σ) = {pr1 (Ft Here (K)

Ft

+ − (σ)) : t ∈ R} = γK (σ) ∪ γK (σ).

.

.

: T ∗ (ΩK ) −→ T ∗ (ΩK )

is the generalized geodesic flow in ΩK (see Section 1.2), where

.

T (Ω) = {(x, ξ) ∈ T ∗ (Ω) : ξ = 0}. Fix a large open ball O containing K in its interior and set Ω0 = Rn \ O. In what follows, for convenience, all obstacles K considered will be contained in O, therefore we will always have Ω0 ⊂ ΩK . Denote by K the class of obstacles K in Rn such that for each (x, ξ) ∈ S ∗ (∂K) if the curvature of ∂K at x vanishes of infinite order in direction ξ, then all points (y, η) sufficiently close to (x, ξ) are diffractive points (see Section 1.2). It follows (K) from a result in [MS2] that for K ∈ K, the flow Ft is well defined and continuous. Let K0 the class of all obstacles K ∈ K satisfying the following non-degeneracy conditions: γK (σ) is a non-degenerate ordinary reflecting ray for almost all σ ∈ S ∗ (Ω0 ) such that γK (σ) ∩ ∂K = ∅, and ∂K does not contain non-trivial open flat subsets. Using arguments from Chapter 6, it is not difficult to show that obstacles in the class K0 are ‘generic’. This means that for every obstacle K in Rn , applying suitable arbitrarily small C ∞ deformations to ∂K, one gets obstacles from the class K0 . ‘Most’ deformations in topological sense have this property.

354

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

We will say that two obstacles K and L have almost the same SLS if there exists a subset R of full Lebesgue measure in Sn−1 × Sn−1 such that SLK (ω, θ) = SLL (ω, θ)

(13.1)

for all (ω, θ) ∈ R. We will now state the main result in this section – briefly, it says that if two obstacles K, L ∈ K0 have almost the same SLS, then their generalized geodesic flows are conjugate with a time-preserving conjugacy on the non-trapping parts of their phase spaces. This will be used several times in subsequent sections. Theorem 13.1.2: Assume that two obstacles K, L ∈ K0 have almost the same SLS. Then there exists a homeomorphism

.∗

.∗

Φ : T (ΩK ) \ Trap(ΩK ) −→ T (ΩL ) \ Trap(ΩL ), which has the following properties:

.∗

(i) Φ defines a symplectic map on an open dense subset of T (ΩK ) \ Trap(ΩK ), (ii) Φ maps S ∗ (ΩK ) \ Trap(ΩK ) onto S ∗ (ΩL ) \ Trap(ΩL ), (L)

(iii) Ft

(K)

◦ Φ = Φ ◦ Ft

.∗

for all t ∈ R,

.∗

(iv) Φ = id on T (Ω0 ) \ Trap(ΩK ) = T (Ω0 ) \ Trap(ΩL ). Conversely, if K, L ∈ K0 are two obstacles for which there exists a homeomorphism (L) (K) Φ : S ∗ (ΩK ) \ Trap(ΩK ) −→ S ∗ (ΩL ) \ Trap(ΩL ) such that Ft ◦ Φ = Φ ◦ Ft for all t ∈ R and Φ = id on S ∗ (Ω0 ) \ Trap(ΩK ), then K and L have the same SLS. We prove Theorem 13.1.2 in Section 13.2, and, as an easy consequence of it, there we also derive the following. Corollary 13.1.3: If two obstacles K, L ∈ K0 have almost the same SLS and the sets of trapped points of both K and L have Lebesgue measure zero, then Vol(K) = Vol(L). It is clear from Livshits’ example that the above conclusion is not true without any assumption about the sets of trapped points. Recall that, according to Proposition 11.2.6, for any obstacle K the set S ∗ (Ω0 ) ∩ Trap(ΩK ) is relatively small, that is it has zero measure. On the other hand, Livshits’ example demonstrates that in some cases, we may have dimH (Trap(ΩK )) = 2n − 1 = dimH (S ∗ (ΩK )). So, from the SLS, we cannot ‘see’ whether the trapped set Trap(ΩK ) is significant or not – the trapped set that we ‘see’ far from the obstacle is always small. Apart from this, the local version of the above problem appears to be even less informative about the obstacle. The following example shows that in the corresponding local problem there is no uniqueness. It concerns obstacles containing flat spots on their boundaries, so it is natural to ask whether similar examples exist where the Gauss curvatures of ∂K and ∂L are non-zero. At present we do not know the answer to this question.

INVERSE SCATTERING BY OBSTACLES

355

L2

L1

L3

K

Figure 13.2

Local non-uniqueness.

Example 13.1.4: Let K and L = L1 ∪ L2 ∪ L3 be two obstacles in Rn (n can be any integer ≥ 2), as in Figure 13.2. Here K and L2 are strictly convex domains in Rn , which are symmetric with respect to a hyperplane H containing the ‘top parts’ of ∂L1 and ∂L3 . L1 and L3 are convex domains in Rn contained in the ‘lower’ half-space with respect to H. Both contain non-trivial open subsets of H. The rays on the figure are generated by some σ0 in S ∗ (Ω0 ). Clearly, for σ close to σ0 , we (K) (L) have Ft (σ) = Ft (σ) for t  0. Moreover, both trajectories γK (σ) and γL (σ) have common points with the corresponding obstacles and are non-degenerate. At the same time, the obstacles K and L are completely different, both globally and locally. In fact, K ∩ L = ∅. We conclude this section with two easy consequences of Theorem 13.1.2. Given an obstacle K, recall that for any x ∈ ∂K, νK (x) denotes the exterior unit normal to ∂K at x. Points of the form (x, νK (x)) ∈ S ∗ (ΩK ) which are non-trapped clearly define back-scattering rays. Indeed, if for some t > 0 we have (K) Ft (x, νK (x)) = (y, η) ∈ S ∗ (Ω0 ), then the scattering ray γ issued from y in direction −η will hit ∂K orthogonally at x after time t, and so after time 2t it will return to its ‘initial’ position y. Denote by Trap(n) (∂K) the set of those x ∈ ∂K such that (x, νK (x)) ∈ Trap(ΩK ). Proposition 13.1.5: Assume that two obstacles K and L from the class K0 have almost the same SLS. Then there exists a homeomorphism ϕ : ∂K \ Trap(n) (∂K) −→ ∂L \ Trap(n) (∂L) such that for all x ∈ ∂K \ Trap(n) (∂K), setting y = ϕ(x), we have Φ(x, νK (x)) = (y, νL (y)). Proof: Let K, L ∈ K0 have almost the same SLS, and let Φ be the conjugacy from Theorem 13.1.2. Given x ∈ ∂K \ Trap(n) (∂K), there exists t > 0 (K) so large that (z, ζ) = Ft (x, νK (x)) ∈ S ∗ (Ω0 ). This simply means that

356

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

(K)

Ft (z, −ζ) = (x, νK (x)) and F (K) 2t (z, −ζ) = (z, ζ). Since Φ(z, ζ) = (z, ζ), (L) (L) it follows that F2t (z, −ζ) = (z, ζ). Setting (y, η) = Ft (z, −ζ), we must have y ∈ ∂L and η⊥∂L at y. Using the identification of directions at y, we may assume  η = νL (y). So, Φ(x, νK (x)) = (y, νL (y)) for some y ∈ ∂L \ Trap(n) (∂L). If under the assumptions of Proposition 13.1.5 we have the additional information that the trapped sets Trap(n) (∂K) and Trap(n) (∂L) are small, then we can conclude that K and L have the same number of connected components. Corollary 13.1.6: Assume that two obstacles K, L ∈ K0 in Rn , n ≥ 3, have almost the same SLS, and moreover dimH (Trap(n) (∂K)) < n − 2 and dimH (Trap(n) (∂L)) < n − 2. Then K and L have the same number of connected components. Proof: It is known that for compact subsets of Rn the topological dimension dim does not exceed the Hausdorff dimension dimH (see e.g. [E]). So, we have dim(Trap(n) (∂K)) < n − 2 and dim(Trap(n) (∂L)) < n − 2. Let K1 , . . . , Kp be the connected components of K. Since dim(Trap(n) (∂K)) < n − 2, it follows that ∂Ki \ Trap(n) (∂K) is a connected open subset of ∂Ki for any i = 1, . . . , p (see e.g. Theorem IV.4 in [HW]). Therefore, the sets ∂Ki \ Trap(n) (∂K) are the open connected components of ∂K \ Trap(n) (∂K). The homeomorphism ϕ from Proposition 13.1.5 has to map these into the open connected components of ∂L \ Trap(n) (∂L). Thus, the number of connected components of ∂L \ Trap(n) (∂L) is the same as the number of connected components of ∂K \ Trap(n) (∂K). We know already that the latter is the same as the number of  connected components of K. A similar argument applies to L.

13.2

Proof of Theorem 13.1.2

Throughout we assume that K, L ∈ K0 are two obstacles in Rn , n ≥ 2, with almost the same SLS. As before we will denote by O a large open ball in Rn that contains K and L, and Ω0 will be the closure of the complement of O in Rn . As in Chapter 11, given ξ ∈ Sn−1 , we will denote by Zξ the hyperplane in Rn orthogonal to ξ and tangent to O such that O is contained in the open half-space determined by Zξ and having ξ as an inner normal. The following lemma is the main step in the proof of Theorem 13.1.2. Lemma 13.2.1: If two obstacles K, L ∈ K0 have almost the same SLS, then for every (K) (K) (L) σ ∈ S ∗ (Ω0 ) and every t ∈ R with Ft (σ) ∈ S ∗ (Ω0 ) we have Ft (σ) = Ft (σ). Proof of Lemma 13.2.1: Assume that K, L ∈ K0 have almost the same SLS, and let R be a subset of Sn−1 × Sn−1 of full Lebesgue measure in Sn−1 × Sn−1 such that SLK (ω, θ) = SLL (ω, θ),

(ω, θ) ∈ R.

(13.2)

INVERSE SCATTERING BY OBSTACLES

357

Using Proposition 11.1.2, we may assume that R has the following additional properties: (i) for (ω, θ) ∈ R all (ω, θ)-rays in ΩK and ΩL are non-degenerate simply reflecting (ω, θ)-rays; (ii) if (ω, θ) ∈ R and γ and δ are (ω, θ)-rays in ΩK (or ΩL ), then Tγ = Tδ . Moreover, shrinking R a bit if necessary, we may assume that (ω, ω) ∈ / R for any ω ∈ Sn−1 . Then for (ω, θ) ∈ R any (ω, θ)-ray in ΩK or ΩL must have at least one reflection point.  We now need the following technical lemma. Lemma 13.2.2: Assume that γ is a non-degenerate simply reflecting (ω0 , θ0 )-ray in ΩK for some (ω0 , θ0 ) ∈ Sn−1 × Sn−1 with successive reflection points x1 , . . . , xk (k ≥ 1). Then there exist: (a) a neighbourhood U of (ω0 , θ0 ) in Sn−1 × Sn−1 , (b) for each i = 1, . . . , k a neighbourhood Ui of xi in ∂K, (c) for every (ω, θ) ∈ U unique x1 (ω, θ) ∈ U1 , . . . , xk (ω, θ) ∈ Uk which are the successive reflection points of an (ω, θ)-ray in ΩK , (d) a sufficiently large T0 > 0, such that xi (ω, θ) depends smoothly on (ω, θ) ∈ U for each i = 1, . . . , k, and for every T ≥ T0 there exists an open neighbourhood W of (x1 , ω0 ) in ∂K × Sn−1 such (K) that the map H(y, ω) = (ω, pr2 (Ft (y, ω))) is a diffeomorphism H : W −→ U . Proof of Lemma 13.2.2: There exist σ0 = (x0 , ξ0 ) ∈ S ∗ (Ω0 ) and T0 > 0 such that (K) γ = γK (σ0 ) and Ft (σ0 ) ∈ S ∗ (Ω0 ) for all t ∈ (−∞, 0] ∪ [T0 , ∞). Fix such σ0 and T0 . Then each xi = pr1 (F (K) ti (σ0 )) for some ti > 0. Let Π and Π be the hyperplanes in Rn through x0 and y0 , respectively, perpendicular to the trajectory γK (σ0 ) at these two points. The cross-sectional map (K) PK : S ∗ (Π) −→ S ∗ (Π ) defined by the shift along the flow Ft is smooth on a small open neighbourhood V of σ0 in S ∗ (Π). Choose V so small that for all points σ ∈ V the trajectory γK (σ) is simply reflecting and has exactly k reflection points x1 (σ), . . . , xk (σ). Then xj (σ) are smooth maps of σ ∈ V into ∂K. Another smooth map is given by V  (x, ξ) → J(x, ξ) = pr2 (PK (x, ξ)) ∈ Sn−1 . For ξ close to ξ0 consider the map Jξ = J(.,ξ). Clearly, det dJξ (x) depends smoothly on (x, ξ) ∈ V , and moreover det dJξ0 (x0 ) = 0 since γ is non-degenerate. Taking V small enough, we may assume that det dJξ (x) = 0 for all (x, ξ) ∈ V . We will now use the Inverse Function Theorem for the map F : V −→ Sn−1 × Sn−1

358

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

defined by F (x, ξ) = (ξ, pr2 (PK (x, ξ))) = (ξ, J(x, ξ)). The Jacobian matrix of F at (x, ξ) ∈ V has the form   0 I dF (x, ξ) = , dJξ (x) ∗ where I is the identity (n − 1) × (n − 1) matrix. Hence dF (x, ξ) is non-singular and by the Inverse Function Theorem, F has a local inverse which is a smooth map as well. Assuming again that V is taken small enough, F is a diffeomorphism between V and an open neighbourhood U of (ω0 , θ0 ) in Sn−1 × Sn−1 . Now for G = F −1 we have G(ω, θ) = (x(ω, θ), ξ) for (ω, θ) ∈ U , and the smoothness of G implies that xj (ω, θ) = xj (x(ω, θ), ω) depends smoothly on (ω, θ) ∈ U for all j = 1, . . . , k. Moreover, the definition of F shows that x1 (ω, θ), . . . , xk (ω, θ) are the successive reflection points of a reflecting (ω, θ)-ray in ΩK , and if y1 , . . . , yk are the successive reflection points of an (ω, θ)-ray in ΩK and for each j, yj is sufficiently close to xj (ω0 , θ0 ), then yj = xj (ω, θ) for any j = 1, . . . , k. Next, given (x, ξ) ∈ V , let x1 (x, ξ) ∈ ∂K be the first reflection point of γK (x, ξ). Setting L(x, ξ) = (x1 (x, ξ), ξ), we get a diffeomorphism between V and an open neighbourhood W of (x1 , ξ) in ∂K × Sn−1 . It is easy to see now that the diffeomorphism H = F ◦ L−1 : W −→ U has the required properties. Indeed, if T ≥ T0 , the definitions of F and L yield (K)

H(y, ω) = (ω, pr2 (Ft

(y, ω)))

for all (y, ω) ∈ W .



We now continue with the proof of Lemma 13.2.1. Fix an arbitrary σ0 = (q0 , u0 ) ∈ S ∗ (Ω0 ) and t0 > 0 such that F (K) t0 (σ0 ) ∈ (K) (L) ∗ S (Ω0 ). We have to prove that Ft0 (σ0 ) = Ft0 (σ0 ). It follows from ∗ Proposition 11.2.6 that S (Ω0 ) ∩ Trap(ΩK ) has measure zero in S ∗ (Ω0 ). Similarly for S ∗ (Ω0 ) ∩ Trap(ΩL ), so S ∗ (Ω0 ) \ (Trap(ΩK ) ∪ Trap(ΩL )) is dense (K) (L) in S ∗ (Ω0 ). Since both Ft and Ft are continuous flows, it is enough to consider the case σ0 ∈ S ∗ (Ω0 ) \ (Trap(ΩK ) ∪ Trap(ΩL )). So, we assume from now on that σ0 ∈ S ∗ (Ω0 ) \ (Trap(ΩK ) ∪ Trap(ΩL )). Then γK (σ0 ) is an (ω0 , θ0 )-ray for some (ω0 , θ0 ) ∈ Sn−1 × Sn−1 . Reversing directions along the trajectory γK (σ0 ), we may assume that u0 = ω0 . Thus, if γK (σ0 ) ∩ ∂K = ∅, then this intersection happens forwards, that is − + (σ0 ) ∩ ∂K = ∅ and γK (σ0 ) ∩ ∂K = ∅. γK

INVERSE SCATTERING BY OBSTACLES

359

Case 1. γK (σ0 ) ∩ ∂K = ∅. Then, according to the above, the forward tra+ (σ0 ) has a common point with ∂K. So, there exists s0 > 0 with jectory γK (K) Fs0 (σ0 ) = (x0 , ξ0 ) and x0 ∈ ∂K. Take the minimal s0 > 0 with this property; (K) then pr1 (Fs (σ0 )) ∈ / ∂K for s ∈ [0, s0 ). It follows from Proposition 11.1.3 that there exist points σ0 ∈ S ∗ (Ω0 ) arbitrarily close to σ0 such that γK (σ0 ) and γL (σ0 ) are simply reflecting rays. Moreover, using the assumption that K ∈ K0 and γK (σ0 ) ∩ ∂K = ∅, we may take σ0 so that γK (σ0 ) is a non-degenerate (ω0 , θ0 )-ray for some (ω0 , θ0 ) close to (ω0 , θ0 ). (K) (L) Using again the continuity of the flows Ft and Ft , we may simple assume this for the trajectories generated by σ0 . Thus, we will assume from now on that γK (σ0 ) is a simply reflecting non-degenerate ray. Moreover, since R is dense in Sn−1 × Sn−1 , using the diffeomorphism H from Lemma 13.2.2 for γ = γk (σ0 ) and perturbing slightly (x0 , ξ0 ) ∈ ∂K × Sn−1 if necessary, we may assume that (ω0 , θ0 ) ∈ R. Let x1 , . . . , xk ∈ ∂K (k ≥ 1) be the successive reflection points of δ = γK (σ0 ). It follows from (13.2) and (ω0 , θ0 ) ∈ R that there exists (at least one) (ω0 , θ0 )-ray δ  in ΩL with Tδ = Tδ  .

(13.3)

Fix one δ  with this property. Since (ω0 , θ0 ) ∈ R, δ  is a simply reflecting non-degenerate (ω0 , θ0 )-ray in ΩL . Let y1 , . . . , ym ∈ ∂L (m ≥ 1) be the reflection points of δ  . Using Lemma 13.2.2, choose a neighbourhood U of (ω0 , θ0 ) in Sn−1 × Sn−1 such that for each (ω, θ) ∈ U there are a unique reflecting (ω, θ)-ray δ(ω, θ) in ΩK with reflection points x1 (ω, θ), . . . , xk (ω, θ) close to x1 , . . . , xk and a unique reflecting (ω, θ)-ray δ  (ω, θ) in ΩL with reflection points y1 (ω, θ), . . . , ym (ω, θ) close to y1 , . . . , ym . We now need the following lemma. Lemma 13.2.3: Assume that Tδ(ω,θ) = Tδ (ω,θ) for all (ω, θ) ∈ U . Then for every (ω, θ) ∈ U the vector y1 (ω, θ) − x1 (ω, θ) is parallel to ω (unless it is the zero vector), and similarly (ym (ω, θ) − xk (ω, θ)) is parallel to θ. Proof of Lemma 13.2.3: Choose an arbitrary smooth parameterization of U : Rn−1 × Rn−1 ⊃ W  (u, v) → (ω(u), θ(v)). Set xj (u, v) = xj (ω(u), θ(v)),

yj (u, v) = yj (ω(u), θ(v)).

The assumptions of the lemma show that the time functions T (u, v) = ω(u), x1 (u, v) +

k−1  i=1

||xi (u, v) − xi+1 (u, v)|| − xk (u, v), θ(v),

360

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS m−1 

S(u, v) = ω(u), y1 (u, v) +

||yi (u, v) − yi+1 (u, v)|| − ym (u, v), θ(v)

i=1

coincide, that is T (u, v) = S(u, v) for all (u, v) ∈ W . Calculating the derivatives of these functions (as we have done before; see e.g. Chapter 2), we get     ∂x ∂ω ∂T (u, v) = , x1 + ω, 1 ∂uj ∂uj ∂uj     k−1  xi+1 − xi ∂xi+1 ∂xk ∂xi , − − ,θ . + ||xi+1 − xi || ∂uj ∂uj ∂uj i=1 Using the vectors pi = ∂T (u, v) = ∂uj

xi+1 −xi ||xi+1 −xi || ,

the reflection law implies

     ∂ω ∂x1 ∂x2 + p1 − p 2 , + ··· , x + ω − p1 , ∂uj 1 ∂uj ∂uj       ∂x ∂ω ∂x + pk−2 − pk−1 , k−1 + pk−1 − θ, k = , x1 . ∂uj ∂uj ∂uj 

Similarly, ∂S (u, v) = ∂uj



 ∂ω ,y . ∂uj 1

∂T Since T (u, v) = S(u, v) on W , we must have ∂u (u, v) = j (u, v) ∈ W , so     ∂ω ∂ω , x1 = , y1 ∂uj ∂uj

for all j = 1, . . . , n − 1. That is, 

∂ω , y − x1 ∂uj 1

∂S ∂uj (u, v)

for all w =

 =0

∂ω for all j. Since the vectors ∂u , j = 1, . . . , n − 1, span a plane perpendicular to ω, it j follows now that (y1 − x1 ) || ω.  Similarly, we derive (ym − xk ) || θ.

Next, we continue with the proof of Lemma 13.2.1. We will use the notation introduced just before the statement of Lemma 13.2.3. We will now use again the assumption (13.2). For every (ω, θ) ∈ R ∩ U there exists a unique reflecting (ω, θ)-ray δ  (ω, θ) in ΩL with Tδ (ω,θ) = Tδ(ω,θ) .

(13.4)

INVERSE SCATTERING BY OBSTACLES

361

The uniqueness follows from the choice of the set R. Moreover, choosing the neighbourhood U of (ω0 , θ0 ) in Sn−1 × Sn−1 sufficiently small, we have δ  (ω, θ) = δ  (ω, θ) for all (ω, θ) ∈ R ∩ U . Indeed, if this is not true, then there exists a sequence {(ωp , θp )}∞ p=1 ⊂ R ∩ U with (ωp , θp ) → (ω0 , θ0 ) as p → ∞ such that δ  (ωp , θp ) = δ  (ωp , θp ),

p ≥ 1.

Consider the hyperplane Z = Zω0 , and for each p ≥ 1 let yp be the (incoming) intersection point of δ  (ωp , θp ) with Z, so that δ  (ωp , θp ) = γL (yp , ωp ). Choosing a subsequence, we may assume that yp → y ∈ Z as p → ∞. Since ωp → ω0 , it follows that δ  = γL (y, ω0 ) is an (ω0 , θ0 )-ray in ΩL with Tδ = lim Tδ (ωp ,θp ) = lim Tδ(ωp ,θp ) = Tδ . p→∞

p→∞

Here we used (13.4). Now (13.3) implies Tδ = Tδ = Tδ . From this, (ω0 , θ0 ) ∈ R and the choice of R, it follows that δ  = δ  . Hence u belongs to δ  = δ  (ω0 , θ0 ); therefore, for large p the ray δ  (ωp , θp ) has m reflection points belonging to the neighbourhoods Uj , respectively. From the choice of U and the uniqueness of the (ω, θ)-rays δ  (ω, θ) for (ω, θ) ∈ U , it now follows that δ  (ωp , θp ) = δ  (ωp , θp ). This is a contradiction which proves that δ  (ω, θ) = δ  (ω, θ) for all (ω, θ) ∈ R ∩ U . Hence Tδ (ω,θ) = Tδ(ω,θ)

(13.5)

for all (ω, θ) ∈ R ∩ U , and the density of R ∩ U in U and the continuity of Tδ(ω,θ) and Tδ (ω,θ) with respect to (ω, θ) imply that (13.5) holds for all (ω, θ) ∈ U . Now Lemma 13.2.3 gives (y1 (ω, θ) − x1 (ω, θ)) || ω and (ym (ω, θ) − xk (ω, θ)) || θ0 . So, y1 (ω, θ) = x1 (ω, θ) + λ(ω, θ) ω,

ym (ω, θ) = xk (ω, θ) + μ(ω, θ) ω

(13.6)

for some real numbers λ(ω, θ), μ(ω, θ). (K) By assumption we have (y, η) = Ft0 (σ0 ) ∈ S ∗ (Ω0 ). Thus, either η = ω0 and y = x1 + sω0 for some s, or η = θ0 and y = xk + sθ0 for some s > 0. The (L) same holds for Ft0 (σ0 ) = (y  , η  ). Now (13.5) and (13.6) imply (y, η) = (y  , η  ). Indeed, in the case η = ω0 and y = x1 + sω0 for some s this is trivial. Assume that y = xk + sθ0 for some s > 0. Then η = θ0 and (13.6) shows that η  = θ0 , too. It remains to see that y = y  . Recall the hyperplane Zω0 tangent to the ball O and perpendicular to ω0 and so that ω0 points into the half-space determined by Zω0 and containing O. Shifting points along the trajectory γK (σ0 ), it is enough to consider the case when q0 ∈ Zω0 and y ∈ Z−θ0 . If R is the radius of the ball O, we then have Tδ = t0 − 2R (see [G1]). Now (13.5) gives Tδ = t0 − 2R which implies that the point y  must (K) (L) lie in the hyperplane Z−θ0 . This and (13.6) yield y  = y. Thus, Ft0 (σ0 ) = Ft0 (σ0 ). This completes the proof in Case 1.

362

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Case 2.γK (σ0 ) ∩ ∂K = ∅. Then we must have γL (σ0 ) ∩ ∂L = ∅. Otherwise the above argument (swapping the roles of K and L) implies γK (σ0 ) ∩ ∂K = ∅, which is a contradiction. So, both γK (σ0 ) and γL (σ0 ) are free rays in Rn and therefore (K) (L) Ft (σ0 ) = Ft (σ0 ) for all t ∈ R. 

.∗

Proof of Theorem 13.1.2: For any σ ∈ T (Ω) \ Trap(ΩK ) take t = t(σ) ∈ R so (K) (L) (K) large that pr1 (Ft (σ)) ∈ S ∗ (Ω0 ) and set Φ(σ) = F−t ◦ Ft (σ). The definition (L) (K) of Φ is correct by Lemma 13.2.1, and . ∗ clearly Ft ◦ Φ = Φ ◦ Ft for all t ∈ R. Apart from that Φ(σ) = σ for σ ∈ T (Ω0 ) \ Trap(ΩK ). The general properties of (K) (L) the generalized geodesic flows Ft and Ft (see [MS2] or Sect. 24.3 in [H3]) imply that. Φ is a homeomorphism . ∗ and it is a symplectic map on an open dense ∗ subset of T (Ω0 ) \ Trap(ΩK ) = T (Ω0 ) \ Trap(ΩK ). Conversely, assume that for two obstacles K, L ∈ K0 there exists a homeomorphism Φ : S ∗ (ΩK ) \ Trap(ΩK ) −→ S ∗ (ΩL ) \ Trap(ΩL ) with Ft ◦ Φ = Φ ◦ Ft for all t ∈ R and such that Φ = id on S ∗ (Ω0 ) \ Trap(ΩK ). Consider an arbitrary (ω, θ) ∈ Sn−1 × Sn−1 . Given an (ω, θ)-ray γ in ΩK , there (K) exist σ ∈ S ∗ (Ω0 ) and T > 0 with γ = γK (σ) and Ft (σ) ∈ S ∗ (Ω0 ) for all t ≤ 0 and t ≥ T . Thus, for all t ∈ (−∞, 0] ∪ [T, ∞) we have (L)

(K)

(L)

(L)

Ft (σ) = Ft

(K)

◦ Φ(σ) = Φ ◦ Ft

(K)

(σ) = Ft

(σ),

so γL (σ) is an (ω, θ)-ray in ΩL having the same sojourn time as γK (σ). In particular, SLK (ω, θ) ⊂ SLL (ω, θ) for all (ω, θ). By symmetry, we must have SLK (ω, θ) =  SLL (ω, θ) for all (ω, θ). Proof of Corollary 13.1.3: By the assumptions, Φ : S ∗ (O ∩ ΩK ) −→ S ∗ (O ∩ ΩL ) is defined almost everywhere and is measure-preserving with respect to the Liouville measures μK on S ∗ (ΩK ) and μL on S ∗ (ΩL ). So, we must have μK (S ∗ (O ∩ ΩK )) = μL (Sb∗ (O ∩ ΩL )). By Fubini’s theorem, μK (S ∗ (O ∩ ΩK )) = Voln−1 (Sn−1 )Voln (O ∩ ΩK ) = Voln−1 (Sn−1 )[Voln (O) − Voln (K)], and similarly μL (S ∗ (O ∩ ΩL )) = Voln−1 (Sn−1 )Voln (O ∩ ΩL ) = Voln−1 (Sn−1 )[Voln (O) − Voln (L)]. Hence, Voln (K) = Voln (L).



INVERSE SCATTERING BY OBSTACLES

13.3

363

An example: star-shaped obstacles

An obstacle K in Rn is called star-shaped if there exists a point x0 in the interior of K such that the segment [x0 , x] is entirely in K for any x ∈ ∂K. Proposition 13.3.1: If K is a star-shaped obstacle and the curvature of K does not vanish of infinite order, then K is non-trapping, that is Trap(ΩK ) = ∅. Proof: Assume that K is star shaped with respect to 0, that is for every x ∈ ∂K the segment [0, x] lies in K. Consider the function f : S ∗ (ΩK ) −→ R given by f (x, ξ) = x, ξ. We will show that f is increasing along reflecting rays in ΩK from one reflection point to the next. Let x ∈ ∂K and ξ ∈ Sn−1 be such that νK (x), ξ > 0. Let y = x + tξ for some t > 0 be the first intersection point of the straight-line ray issued from x in direction ξ with ∂K, and let η be the reflected direction of that ray, that is η = ξ − 2νK (y), ξ νK (y). Then f (y, η) = y, η = x + tξ, ξ − 2νK (y), ξ νK (y) = x, ξ + t − 2νK (y), ξ νK (y), y = x, ξ + t + 2νK (y), η νK (y), y ≥ f (x, ξ) + t. Let R be the radius of a large ball O with centre 0 containing K, and let T = 2R. ∗ (ΩK ) such that Assume that Trap(ΩK ) = ∅; then there exists σ = (x, ξ) ∈ S∂K (K) ||pr1 (Ft (σ))|| ≤ R for all t ≥ 0. Since the curvature of K does not vanish of infinite order, σ can be approximated arbitrarily well with points σ  such that the (K) trajectory {Ft (σ  ) : t ∈ [0, T ]} has only simple (transversal) reflections (see Theorem 7.16 in [MS2] or [H3]), so without loss of generality we will assume that (K)

γ = {Ft

(σ) : t ∈ [0, T ]}

has only simple (transversal) reflections and (K)

||pr1 (Ft

(σ))|| ≤ R,

0 ≤ t ≤ T.

Let (xi , ξi ), i = 1, 2, . . . , be the successive reflection points (with reflected (K) directions) of γ. Set (x0 , ξ0 ) = (x, ξ) and let (xi+1 , ξi+1 ) = Fti (xi , ξi ) for all i ≥ 0. Then for any m ≥ 1 we have ||xm || ≥ xm , ξm  = f (xm , ξm ) ≥ f (xm−1 , ξm−1 ) + tm−1 ≥ tm−1 + tm−2 + · · · + t1 + x0 , ξ0  ≥ tm−1 + tm−2 + · · · + t1 .

364

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS (K)

So, for any time t ∈ [0, T ] we have ||pr1 (Ft (σ))|| ≥ t. In particular, (K) ||pr1 (Ft (σ))|| ≥ T > R, which is impossible. Hence Trap(ΩK ) = ∅.  Remark 13.3.2: Notice that in the calculation above involving (x, ξ) and y = x + t ξ the inequality ||y|| < ||x|| is only possible if f (x, ξ) = x, ξ < 0. Indeed, ||y||2 = ||x + tξ||2 = ||x||2 + t2 + 2t x, ξ, so clearly ||y||2 < ||x||2 implies x, ξ < 0. As an application of Theorem 13.1.2 we will derive the following. Proposition 13.3.3: Let K, L ∈ K0 have almost the same SLS. If K is star shaped, then ∂K ⊂ ∂L. If moreover L is non-trapped or ∂L is connected, then K = L. Proof: Given x ∈ ∂K, as before we will denote by νK (x) the outward unit normal to K at x. In the following for brevity we will use the notation γK (x) = γK (x, νK (x)). Assume K, L ∈ K0 have almost the same SLS and K is star shaped. Then the conclusion of Theorem 13.1.2 holds for the generalized geodesic flows in ΩK and ΩL . We will assume that in the above definition of star-shaped x0 = 0, that is for every x ∈ ∂K the segment [0, x] lies in K. Set Wr = {y ∈ Rn : ||y|| > r} for r > 0, and let a = inf {R > 0 : Wr ∩ ∂K ⊂ ∂L ∀r ≥ R}. Obviously, Wa ∩ ∂K ⊂ ∂L. We will prove that a = 0. We claim that Wa ∩ ∂K = Wa ∩ ∂L.

(13.7)

Indeed, if Wa ∩ ∂K is a proper subset of Wa ∩ ∂L, then there exists x ∈ ∂L \ ∂K with ||x|| > a, so b = sup {||x|| : x ∈ ∂L \ ∂K} ∈ (a, ∞) is well defined. By the definition of b we can find a sequence {xm } of points in ∂L \ ∂K such that ||xm ||  b as m → ∞. Since ∂L is compact, taking a subsequence, we may assume that xm → x ∈ ∂L. Then ||x|| = b > a, so x ∈ ∂K is impossible; otherwise by the choice of a we would have ∂K = ∂L near x, a contradiction. Thus, x ∈ ∂L \ ∂K. This shows that b is not just a supremum, it is a maximum, that is b = max{||x|| : x ∈ ∂L \ ∂K}.

INVERSE SCATTERING BY OBSTACLES

365

Take x ∈ ∂L \ ∂K such that ||x|| = b. Then the normal to ∂L at x must be perpendicular to ∂L, that is νL (x) = x/||x||. Moreover, the choice of b shows that for r ≥ b we have Wr ∩ ∂K ⊂ ∂L. Thus, the ray  = {x + tνL (x) : t > 0} has no common points with ∂L \ ∂K, so  ∩ ∂L =  ∩ ∂K. If there exists a point y ∈  ∩ ∂K, since K is star shaped, we must have [0, y] ⊂ K. However, clearly x ∈ [0, y], so we must have x ∈ K, which is a contradiction. Thus,  ∩ ∂L =  ∩ ∂K = ∅. Thus, γL (x) defines a back-scattering ray in ΩL , that is a scattering ray with just one reflection point x where the ray is perpendicular to ∂L. Now using the conjugacy Φ from Theorem 13.1.2, it follows immediately that γL (x) coincides with a corresponding ray in ΩK , that is we must have x ∈ ∂K. This is a contradiction that proves (13.7). We are now ready to prove that a = 0. Assume a > 0. Then there exists x ∈ ∂K with ||x|| = a. Fix for a moment an arbitrary x with this property. If {x + tνk (x) : t > 0} ∩ ∂K = ∅, then there exists an open neighbourhood Ox of x in Rn such that {y + tνk (y) : t > 0} ∩ ∂K = ∅ for all y ∈ Ox ∩ ∂K. In this case it follows easily, using the conjugacy Φ from Theorem 13.1.2, that ∂K = ∂L in a neighbourhood of x, and we may take Ox so that Ox ∩ ∂K = Ox ∩ ∂L. If {x + tνk (x) : t > 0} ∩ ∂K = ∅, then there exist  > 0 and an open neighbourhood Ox of x in Rn such that {y + tνk (y) : t > 0} ∩ ∂K = ∅ and ty = min{t > 0 : y + tνK (y) ∈ ∂K} >  for all y ∈ Ox ∩ ∂K. Shrinking Ox if necessary, we may assume that ||y|| + ty > a + (y, νK (y)). Let for all y ∈ Ox . Given y ∈ Ox , consider the trajectory δ(y) = γK f be the function from the proof of Proposition 13.3.1. Since f (y, νK (y)) > 0, it follows from the argument in the proof of Proposition 13.3.1 that for any reflection point (z, ζ) of δ(y) we have f (z, ζ) > f (y, νK (y)) > 0. Combining this with Remark 13.3.2, shows that ||z|| > ||y||. Thus, whenever y ∈ Ox , we have that all reflection points of δ(y) belong to Wa . Now (13.7) implies (y, νK (y)) = (y, νL (y) for any y ∈ Ox , so in particular Ox ∩ ∂K = Ox ∩ ∂L. Thus, for any x ∈ Γa = {y ∈ ∂K : ||y|| = a} we can choose an open neighbourhood Ox of x in Rn with Ox ∩ ∂K = Ox ∩ ∂L. Since Γa is compact, there exist x1 , . . . , xp ∈ Γa with Γa ⊂ ∪pi=1 Oxi . The latter is then an open neighbourhood of Γa in Rn , so for r < a, r sufficiently close to a, we will have Γr ⊂ ∪pi=1 Oxi . This immediately implies Wr ∩ ∂K ⊂ ∂L for such r, which is a contradiction with the choice of a. Thus, we must have a = 0. This proves that ∂K ⊂ ∂L. The second part of the statement in Proposition 13.3.3  follows trivially from the first.

13.4

Tangential singularities of scattering rays I

As one may expect the behaviour of the generalized geodesic flow near a scattering ray γK (σ) tells us whether the ray has some kind of tangency to ∂K. A simple way to look at this is by using Poincaré maps between appropriate cross-sections of γK (σ). As before, throughout O denotes a large open ball in Rn and Ω0 = Rn \ O.

366

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Let K be an obstacle in Rn and let σ0 = (x0 , ξ0 ) ∈ S ∗ (Ω0 ) generate a scattering (K) ray, i.e. there exists T > 0 such that Ft (σ0 ) ∈ S ∗ (Ω0 ) for all t ≤ 0 and all t ≥ T . Consider arbitrary smooth (n − 1)-dimensional local submanifolds (e.g. (K) hyperplanes) X and Y of Rn with x0 ∈ X, y0 = pr1 (Ft (σ0 )) ∈ Y and such that X and Y are transversal to γK (σ0 ) at x0 and y0 , respectively. Define the Poincaré map (or cross-sectional map) PK : S ∗ X −→ S ∗ Y by using the shift along the flow Ft from S ∗ X near σ0 to S ∗ Y near ρ0 = Ft0 (σ0 ). If L is another obstacle contained in O, K, L ∈ K0 and K and L have almost the same SLS, then according to Theorem 13.1.2 we have PK = PL . So whatever singularities we observe for PK , exactly the same we have for PL . We discuss simple tangencies of scattering rays in this section. In typical situations we can also distinguish between diffractive and gliding behaviour, however the proof of this is a bit more involved and will be done in the next section. First, let us consider the local cross-sectional map near a tangent point of a generalized geodesic. Let ρ0 = (y, ξ) ∈ S ∗ (∂K) be a diffractive point (i.e. ρ0 ∈ Gd ; see Section 1.2). Take  > 0 so small that (K)

(K)

 = {pr1 (Ft

(K)

(ρ0 )) : − ≤ t ≤ }

is a straight-line segment in ΩK . Denote by Π and Π the hyperplanes in Rn through (K) (K) x = pr1 (F− (ρ0 )) and z = pr1 (F (ρ0 )), respectively, and perpendicular to . Let σ = (x, ξ) and let P : S ∗ Π −→ S ∗ Π (K)

be the (local) cross-sectional map defined by the shift along the flow Ft trajectory . Consider the Gauss map

near the

G = dν(y) : Ty (∂K) −→ Ty (∂K) of ∂K at y. Then κ = G(ξ), ξ > 0, since (y, ξ) is a diffractive point. As we have done in Sections 2.3 and 2.4, for ρ ∈ S ∗ (Π) close to σ one can identify the tangent space Tρ (S ∗ Π) with Π × Π in a natural way. The following lemma is an easy consequence of Proposition 2.4.2. ∗ Lemma 13.4.1: Assume that {σm }∞ m=1 ⊂ S (Π) is a sequence converging to σ such that for each m the generalized geodesic (K)

m = {pr1 (Ft

(σm )) : − ≤ t ≤ }

has a transversal reflection point ym near y. (This is then its only reflection point.) If ρ0 = (y, ξ) is a diffractive point, that is ρ0 ∈ Gd , then for each m ≥ 1 there exists

INVERSE SCATTERING BY OBSTACLES

367

um ∈ Π with ||um || = 1 such that ||dP(σm )(um , 0)|| ≥

2κ , ξm , ν(ym )

where ξm is the reflected direction of m at ym . In particular, ||dP(σm )|| → ∞ as  m → ∞. In what follows and also in the next few sections we will work with obstacles K so that the normal curvature of K does not vanish of infinite order. Denote by K(fin) the class of obstacles with this property. Clearly K(fin) ⊂ K. Set (fin)

K0

= K(fin) ∩ K0 .

As shown in the following proposition, the points in S ∗ (Ω0 ) generating scattering rays having multiple tangencies can be well approximated by points generating rays with just one tangency to ∂K. Proposition 13.4.2: For any obstacle K ∈ K(fin) and any point σ = (y, η) ∈ S ∗ (∂K) \ Trap(ΩK ) such that the Gauss curvature of ∂K at y is non-zero, there exists σ  = (y  , η  ) ∈ S ∗ (∂K) arbitrarily close to σ such that y  is the only tangent point of the scattering ray γK (σ  ) to ∂K. We prove this proposition in Section 13.8. Now we will use to derive an important consequence. (fin)

Proposition 13.4.3: Assume that two obstacles K, L ∈ K0 have almost the same SLS. For any point σ0 = (x0 , ξ0 ) ∈ S ∗ (Ω0 ) \ Trap(ΩK ) the scattering ray γK (σ0 ) contains a point of tangency to ∂K if and only if γL (σ0 ) contains a point of tangency to ∂L. Proof of Proposition 13.4.3: Replacing ξ0 by −ξ0 if necessary, we may assume (K) that there exists T > 0 such that Ft (σ0 ) ∈ S ∗ (Ω0 ) for all t ≤ 0 and all t ≥ T . (K) Consider the hyperplanes X and Y of Rn with x0 ∈ X, y0 = pr1 (Ft (σ0 )) ∈ Y and such that X and Y are perpendicular to γK (σ0 ) at x0 and y0 , respectively. Define the cross-sectional maps PK , PL : S ∗ X −→ S ∗ Y as before. Then PK = PL . Assume γK (σ0 ) ∩ S ∗ (∂K) = ∅, that is γK (σ0 ) contains a tangent point to ∂K. (K) If for some t0 > 0 the point ρ0 = Ft0 (σ0 ) ∈ S ∗ (∂K) is a single tangent point, that is it does not belong to a gliding segment on ∂K, then there exists a diffractive point ρ ∈ S ∗ (∂K) arbitrarily close to ρ0 (see e.g. Section 24.3 in [H3]). By Proposition 13.4.2, we can take ρ so that it is the only tangent point of γK (ρ) to ∂K. (K) Take such a point ρ and let σ ∈ S ∗ X be so that Ft (σ) = ρ for some t > 0 close to t0 . Then γK (σ) = γK (ρ) has only one tangent point to ∂K which is the projection

368

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

of a diffractive point in S ∗ (∂K). Now Lemma 13.4.1 and PK = PL imply that the map PL has a singularity at σ, so γL (σ) must contain a tangent point to ∂L. Letting ρ → ρ0 , we have σ → σ0 and therefore γL (σ0 ) contains a tangent point to ∂L. Next, consider the case when γK (σ0 ) contains a gliding segment on ∂K, (K) however it has no simple tangencies to ∂K. Let ρ0 = Ft0 (σ0 ) ∈ S ∗ (∂K) be one 3 end of such a gliding segment; then ρ0 ∈ G (see Section 1.2), and nearby ρ0 there (K) are points from the set Gg on the trajectory {Ft (σ0 ) : t ∈ R}. Since K ∈ K(fin) , setting ρ0 = (x0 , ξ0 ), the curvature of ∂K in the direction of ξ0 must change sign at x0 , so there exist diffractive points ρ ∈ S ∗ (∂K) arbitrarily close to ρ0 (see Section 1.2). Now repeating the argument from the previous case, we derive again that the trajectory γL (ρ) must have a tangency to ∂L. Letting ρ → ρ0 shows that γL (σ0 )  contains a tangent point to ∂L.

13.5

Tangential singularities of scattering rays II

This section deals with a more comprehensive approach in studying tangential singularities of scattering rays. It is rather technical and might be skipped at first reading. It will be used in subsequent sections, and the reader may then need to look into this section. It turns out that the behaviour of the sojourn time function near σ tells us, for example, whether γK (σ) has a simple tangency at ∂K or it contains a whole gliding segment on ∂K. Some other information can be obtained as well. We begin with some local considerations. Let again K be an obstacle in Rn , n ≥ 2, such that the normal curvature of K does not vanish of infinite order. In this section for brevity we will use the notation (K)

Ft = F t

,

Ω = ΩK .

Let ϕ be a defining function for ∂K in a small neighbourhood V0 of ∂K. That is, ϕ : V0 −→ R is smooth, dϕ = 0 on ∂K, and ϕ−1 (0) = ∂K. Consider the Hamiltonian function 1 p : T ∗ (Rn ) −→ R, p(x, ξ) = (|ξ|2 − 1). 2 The corresponding Hamiltonian vector field is Hp = (ξ1 , . . . , ξn ; 0, . . . , 0) =

n 

ξi

i=1

∂ . ∂xi

As before, O will denote a fixed open ball in Rn containing K and Ω0 = Rn \ O. Set S0 = ∂O. In this section we will first study the sets Tk = {σ ∈ S ∗ (V0 ) : Hpj ϕ(σ) = 0

for j = 0, 1, . . . , k

and

Hpk+1 ϕ(σ) = 0 },

INVERSE SCATTERING BY OBSTACLES

369

where k is a positive integer. In general these sets are not manifolds, however locally each of them is contained in a submanifold of codimension 2 in S ∗ (V0 ). What is more important, it turns out that for k ≥ 2 the set Tk is locally contained in a submanifold of codimension 3 (see Proposition 13.5.1). An important consequence of this is that the set of those σ ∈ S ∗ (Ω0 ) that generate trajectories containing gliding segments on ∂K can be covered by a countable family of submanifolds of codimension 2 in S ∗ (Ω0 ), so topologically it does not divide S ∗ (Ω0 ) (see e.g. [F]). Proposition 13.5.1: For each k ≥ 1 and each σ ∈ Tk there exists an open neighbourhood V (σ) of σ in T ∗ (V0 ) and a smooth submanifold Γ(σ) of V (σ) such that Tk ∩ V (σ) ⊂ Γ(σ) ⊂ S ∗ (V0 ) and the codimension of Γ(σ) in T ∗ (V0 ) is 3 for k = 1 and 4 for k ≥ 2. Consequently, as a submanifold of S ∗ (V0 ), the codimension of Γ(σ) is 2 for k = 1 and 3 for k ≥ 2. Proof of Proposition 13.5.1: Denote by V  the set of those ρ ∈ T ∗ (V0 ) such that ξ = 0 and Hpk+1 ϕ(ρ) = 0, and define g : V  −→ R by g(ρ) = Hp ϕ(ρ) =

n  i=1

ξi

∂ϕ (x), ∂xi

where ρ = (x, ξ). First, consider the case k = 1. We claim that T1 = {ρ ∈ V  : p(ρ) = ϕ(ρ) = g(ρ) = 0} is a submanifold of V (σ). For this of course one has to show that dp(ρ), dϕ(ρ) and dg(ρ) are linearly independent on T1 . Let ρ = (x, ξ) ∈ T1 and assume that u dp(ρ) + a dϕ(ρ) + b dg(ρ) = 0

(13.8)

for some u, a, b ∈ R. Here d = d(x,ξ) . Considering derivatives with respect to xj , (13.8) implies n  ∂ϕ ∂2ϕ (x) + b ξi (x) = 0. a ∂xj ∂xi ∂xj i=1 Multiplying the latter by ξj and summing up, gives aHp ϕ(ρ) + bHp2 ϕ(ρ) = 0. Since Hp ϕ(ρ) = 0 and Hp2 ϕ(ρ) = 0, the above implies b = 0. This and (13.8) yield u = a = 0. Hence T1 is a submanifold of V (σ) of codimension 3. Next, consider the case k ≥ 2. Define another function f on V  by f (ρ) = Hpk ϕ(ρ) =

n  i1 ,i2 ,...,ik =1

Set

ξi1 ξi2 · · · ξik

∂kϕ (x) . ∂xi1 ∂xi2 · · · ∂xik

Γ = {ρ ∈ V  : p(ρ) = ϕ(ρ) = g(ρ) = f (ρ) = 0}.

370

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Clearly, Tk ∩ V  ⊂ Γ ⊂ S ∗ (V0 ). We will show that dp, dϕ, dg and df are linearly independent at any point ρ ∈ Tk ∩ V  (in particular at σ). Let ρ = (x, ξ) ∈ Tk ∩ V  and let u dp(ρ) + a dϕ(ρ) + b dg(ρ) + c df (ρ) = 0 (13.9) for some u, a, b, c ∈ R. Considering derivatives with respect to xm , (13.9) implies a

n  ∂ϕ ∂2ϕ (x) + b ξi (x) ∂xm ∂xi ∂xm i=1

+c

n 

ξi1 ξi2 · · · ξik

i1 ,i2 ,...,ik =1

∂ k+1 ϕ (x) = 0 ∂xi1 ∂xi2 · · · ∂xik ∂xm

for all m = 1, . . . , n. Multiplying the latter by ξm and summing up, we get aHp ϕ(ρ) + bHp2 ϕ(ρ) + cHpk+1 ϕ(ρ) = 0. Since ρ ∈ Tk and k ≥ 2, we have Hp ϕ(ρ) = Hp2 ϕ(ρ) = 0 and Hpk+1 ϕ(ρ) = 0. Hence c = 0. Next, considering the terms in (13.9) corresponding to derivatives with respect to ξi , we get ∂ϕ (x) = 0 u ξi + b ∂xi for all i. Multiplying this by ξi and summing up, gives 0 = u |ξ|2 + b Hp ϕ(ρ) = u, so u = 0. Returning to the previous equality and using the fact that dϕ(ρ) = 0, one gets b = 0. Then (13.9) yields a = 0 as well. This shows that dp, dϕ, dg and df are linearly independent at σ. By continuity, there exists an open neighbourhood V (σ) of σ in V  such that dp, dϕ, dg and df are linearly independent on V (σ). Then Γ ∩ V (σ)  is a submanifold of codimension 4 in V (σ) and Tk ∩ V (σ) ⊂ Γ ∩ V (σ). Next, we are going to show that for k ≥ 2 the set Tk can be covered by a countable family of codimension 2 submanifolds of S ∗ (S0 ). (m) Fix for a moment k ≥ 2 and m ≥ 0. Denote by Tk the set of those σ ∈ Tk such that there exists t > 0 with Ft (σ) ∈ S ∗ (S0 ) and the trajectory {Fs (σ) : 0 < s ≤ t} has no common points with ∪∞ r=2 Tr and has exactly m transversal reflection points (m) at ∂K (and possibly some tangent points that belong to T1 ). For σ ∈ Tk denote by (K) s(σ) the minimal number t > 0 with Ft (σ) ∈ S ∗ (S0 ). (m)

Lemma 13.5.2: For every σ0 ∈ Tk of σ0 in S ∗ (V0 ) such that the set (m)

(m)

there exists an open neighbourhood Uk (σ0 ) (m)

Nk (σ0 ) = {Fs(σ) (σ) : σ ∈ Tk

(m)

∩ Uk (σ)}

is contained in a smooth codimension 2 submanifold of S ∗ (S0 ). Proof: We will use the above proposition and an argument from Section 11.3.

INVERSE SCATTERING BY OBSTACLES

371

(m)

Fix an arbitrary σ0 ∈ Tk and let s1 < s2 < · · · < sm be the times of the transversal reflections of {Fs (σ0 ) : 0 < s ≤ s(σ0 )}. Clearly 0 < s1 and sm < s(σ). For each j = 1, . . . , m fix two numbers aj and bj close to sj and such that b0 = 0 < a1 < s1 < b1 < a2 < s2 < b2 < · · · < am < sm < bm < am+1 = s(σ0 ). For each j = 1, . . . , m choose arbitrary smooth cross sections (i.e. submanifolds of S ∗ (Ω) of codimension 1 transversal to the flow Ft ) Aj and Bj to the trajectory {Fs (σ0 ) : 0 < s ≤ s(σ0 )} such that Faj (σ0 ) ∈ Aj and Fbj (σ0 ) ∈ Bj . We assume that Aj and Bj are so small and so close to the reflection point Fsj (σ0 ) that for any ρ ∈ Aj the trajectory of ρ under Ft makes exactly one (transversal) reflection at ∂K before intersecting transversally Bj . Let V (σ0 ) be an open neighbourhood of σ0 in S ∗ (V0 ) with the properties described in Proposition 13.5.1. For ρ in a small neighbourhood W of σ0 in V (σ0 ) we denote by ψt (ρ) the unique curve in S ∗ (Rn ) for which there exists a sequence of numbers b0 (ρ) = 0 < a1 (ρ) < s1 < b1 (ρ) < a2 (ρ) < · · · < am (ρ) < sm < bm (ρ) < am+1 (ρ) with the following properties: (i) ψaj (ρ) (ρ) ∈ Aj and ψbj (ρ) (ρ) ∈ Bj for all j = 1, . . . , m, ψam+1 (ρ) (ρ) ∈ S ∗ (S0 ), and ψs (ρ) ∈ S ∗ (O) for s ∈ (0, am+1 (ρ)); (ii) for each j = 0, 1, . . . , m the curve {ψt (ρ) : bj (ρ) ≤ t ≤ aj+1 (ρ)} is a trajectory of the vector field Hp in Rn (this curve could have common points with the interior of K); (iii) for each j = 1, . . . , m the curve {ψt (ρ) : aj (ρ) ≤ t ≤ bj (ρ)} is a trajectory of the GHF Ft in Ω. It is clear that if the neighbourhood W of σ0 in V (σ0 ) is sufficiently small, then the curve ψt (ρ) is well defined for all ρ ∈ W . Set Λ(σ0 ) = {ρ ∈ W : Hpk ϕ(ρ) = 0 }. We assume that Hpk+1 ϕ = 0 on V (σ0 ) (which follows from the construction of V (σ0 ) in the proof of Proposition 13.5.1). Then Λ(σ0 ) is a codimension 1 submanifold of W transversal to the vector field Hp . Consequently, the map Λ(σ0 )  ρ → ψa1 (ρ) (ρ) ∈ A1 is smooth and a local bijection, so it defines a local diffeomorphism. Dealing in the same way with the shift along the curve ψt (ρ) between successive cross sections, (m) one derives that the map Ψk (ρ) = ψam+1 (ρ) (ρ) from Λ(σ0 ) to S ∗ (S0 ) is a local diffeomorphism.

372

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS (m)

(m)

Set Uk (σ0 ) = W . Let Γ(σ) be as in Proposition 13.5.1. Then Γ(σ0 ) ∩ Uk (m) (m) is a codimension 2 submanifold of Λ(σ0 ). Hence Ψk (Γ(σ0 ) ∩ Uk (σ0 )) is a ∗ codimension 2 submanifold of S (S0 ). It remains to show that (m)

(m)

(m)

Nk (σ0 ) = Ψk (Γ(σ0 ) ∩ Uk (σ0 )). (m)

(m)

To check this, observe that for any ρ ∈ Uk (σ0 ) ∩ Tk we have ψs (ρ) = Fs (ρ) for all s ∈ [0, am+1 (ρ)]. Indeed, for such ρ the trajectory {Ft (ρ) : t ≥ 0} has exactly m transversal reflection points Fsi (ρ) (ρ), i = 1, . . . , m, where si (ρ) is close to si for each i. There exist real numbers ai (ρ) close to ai and bi (ρ) close to bi such that Fai (ρ) (ρ) ∈ Ai and Fbi (ρ) (ρ) ∈ Bi for all i = 1, . . . , m and Fam+1 (ρ) (ρ) ∈ S ∗ (S0 ). Hence the curve ψs (ρ) = Fs (ρ), s ∈ [0, am+1 (ρ)], satisfies the conditions (i)–(iii). In particular, s(σ) = am+1 (ρ) and therefore (m) (m) (m) Nk (σ0 ) = Ψk (Γ(σ0 ) ∩ Uk (σ0 )). This proves the lemma.  Denote by GK the set of those σ ∈ S ∗ (S0 ) such that Ft (σ) ∈ Tk for some t ∈ R and some k ≥ 2. Then GK contains any σ ∈ S ∗ (S0 ) that generates a trajectory containing a gliding segment on ∂K (see Section 1.2). As an immediate consequence of the above lemma we get the following. Proposition 13.5.3: There exists a countable family {Ni } of codimension 2 submanifolds of S ∗ (S0 ) such that GK ⊂ ∪i Ni . (m)

(m)

(k,m)

Proof: Cover the set Tk with a countable family Uk (σj S ∗ (V0 ) with the properties listed in Lemma 13.5.2. Then ∞ ∞ GK ⊂ ∪∞ j=1 ∪m=0 ∪k=1 Nk (σj (m)

(k,m)

) of open subsets of

),

so the statement follows from Lemma 13.5.2.



Finally, it remains to deal with the case k = 1. Lemma 13.5.4: For every σ0 ∈ T1 there exists an open neighbourhood U (σ0 ) of σ0 such that the set N (σ0 ) = {Fs(σ) (σ) : σ ∈ T1 ∩ U (σ0 )} is contained in a smooth codimension 1 submanifold of S ∗ (S0 ). Proof: This is essentially a repetition of the proof of Lemma 13.5.2 with minor modifications. It is actually simpler since there are no further tangencies that have  to be avoided. We leave the details to the reader.

INVERSE SCATTERING BY OBSTACLES

373

An important consequence of the last lemma is the following. Proposition 13.5.5: There exists a countable family {Mi } of codimension 1 submanifolds of S ∗ (S0 ) \ (Trap(ΩK ) ∪ GK ) such that every σ ∈ S ∗ (S0 ) \ (Trap(ΩK ) ∪ GK ∪ ∪i Mi ) generates a simply reflecting trajectory in Ω. Moreover, the family {Mi } is locally finite in S ∗ (S0 ) \ (Trap(ΩK ) ∪ GK ), that is any compact subset of S ∗ (S0 ) \ (Trap(ΩK ) ∪ GK ) has common points with only finitely many of the submanifolds Mi . Proof of Proposition 13.5.5: For every σ ∈ T1 \ Trap(ΩK ) choose an open neighbourhood U (σ) as in Lemma 13.5.4. Shrinking U (σ) if necessary, we may assume that Hp2 ϕ = 0 on the closure U (σ) of U (σ), and therefore it does not have common points with Tk for any k ≥ 2. Also, we choose U (σ) such that U (σ) ∩ Trap(ΩK ) = ∅. Consequently, for the set N (σ) from Lemma 13.5.4 we have dist(N (σ), Trap(ΩK ) ∪ GK ) > 0, and so there exists a smooth codimension 1 submanifold M (σ) of S ∗ (S0 ) such that N (σ) ⊂ M (σ) and M (σ) has no common points with Trap(ΩK ) ∪ GK . Choose a countable set of elements σi ∈ T1 \ Trap(ΩK ) such that T1 \ Trap(ΩK ) ⊂ ∪i U (σi ) and {U (σi ) ∩ T1 } is a locally finite family in T1 \ Trap(ΩK ). Denote Mi = M (σi ). It is now clear that any σ ∈ S ∗ (S0 ) \ (Trap(ΩK ) ∪ GK ∪ ∪i Mi ) generates a simply reflecting trajectory in Ω. It remains to show that {Mi } is locally finite in S ∗ (S0 ) \ (Trap(ΩK ) ∪ GK ). Let L be a compact subset of S ∗ (S0 ) \ (Trap(ΩK ) ∪ GK ). Then the sojourn (travelling) times of trajectories generated by elements of L are uniformly bounded. Assume that there exists ρm ∈ Mim ∩ L for infinitely many im . Choosing a subsequence, we may assume that ρm → ρ ∈ L as m → ∞. Since ρm ∈ Mim , there exists tm ∈ R with Ftm (ρm ) ∈ U (σim ) ∩ T1 . Since the sequence {tm } is bounded, we may assume that tm → t ∈ R as t → ∞. Then Ftm (ρm ) → Ft (ρ), so for σ = Ft (ρ) we must / Trap(ΩK ) ∪ GK , have Hp ϕ(σ) = 0. On the other hand, ρ ∈ L shows that ρ ∈ so the trajectory generated by ρ is not trapped and does not have common points with Tk for any k ≥ 2. Hence σ ∈ T1 \ Trap(ΩK ), and therefore there exists a compact neighbourhood W of σ in S ∗ (Rn ) such that W ∩ T1 ∩ Trap(ΩK ) = ∅. Then Ftm (ρm ) → σ implies that W has common points with U (σim ) ∩ T1 for all sufficiently large m, a contradiction with the local finiteness of the family {U (σi ) ∩ T1 } in T1 \ Trap(ΩK ). Hence L can have common points with only  finitely many Mi ’s. Remark 13.5.6: In general different submanifolds Mi and Mj may have common points ρ and they are not necessarily transversal at such points. Clearly for such a point ρ the trajectory γK (ρ) will have more than one tangent point to ∂K. However, it

374

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

follows from the construction in the proof of Proposition 13.5.5 that if Mi = Mj and σ ∈ Mi ∩ Mj , then there exist points in Mi \ Mj arbitrarily close to σ. Moreover, the construction shows that ∪i Mi is a closed subset of S ∗ (S0 ) \ (Trap(ΩK ) ∪ ∪q Nq ). Indeed, by the local finiteness of the family {Mi }, if σ ∈ Mi \ (Trap(ΩK ) ∪ ∪q Nq ) for some i, then σ ∈ Mj for some j.

13.6

Reflection points of scattering rays and winding numbers

Let again K ∈ K be an obstacle in Rn . We already know (see Proposition 11.1.3) that the set of points σ ∈ S ∗ (ΩK ) generating rays tangent to ∂K has Lebesgue measure zero in S ∗ (ΩK ). The question that we will discuss at some stage here is how to distinguish1 between rays containing simple tangencies only and rays containing non-trivial segments on ∂K. It turns out that this is possible – near points generating rays with gliding segments there is a large variety of points generating rays with simple tangencies. To demonstrate this we will use winding numbers. We did use winding numbers in Section 8.3, however here we will proceed in a different way using an idea of Melrose and Sjöstrand [MS2]. As in the previous section, consider a defining function ϕ ∈ C ∞ (Rn ) of ∂K. We choose ϕ with the following properties: (i) ϕ ≥ 0 in ΩK and ϕ−1 (0) = ∂K; (ii) ||∇ϕ|| = 1 in a neighbourhood V of ∂K; (iii) ϕ = 1 on ΩK \ V0 , where V0 is a neighbourhood of ∂K with V ⊂ V0 . Then ∇ϕ(x) = νK (x) for x ∈ ∂K. Consider the continuous function λ : S ∗ (ΩK ) × R −→ C defined by λ(σ, t) = [ϕ(x(t)) + iξ(t), ∇ϕ(x(t))]2 , (K)

where Ft (σ) = (x(t), ξ(t)), t ∈ R. Now observe that when x(t) ∈ ΩK \ V0 , then ϕ(x(t)) = 1 and ∇ϕ(x(t)) = 0, so λ(σ, t) = 1. So, in particular, if σ is a non-trapped point, then it will generate an (ω, θ)-ray for some (ω, θ) ∈ Sn−1 × Sn−1 , and then most of the points on the incoming ray and those on the outgoing ray will be outside V0 . Thus, for such σ, t → λ(σ, t) defines a closed curve in C beginning and ending at 1. Also, λ(σ, t) = 0 is equivalent to ϕ(x(t)) = 0 and ξ(t), ∇ϕ(x(t)) = 0. This means that the ray γK (σ) is tangent to ∂K at x(t). 1

For example, by using cross-sectional maps, as in Section 13.4.

INVERSE SCATTERING BY OBSTACLES

375

So, if σ is a non-trapped point such that γ(σ) is a simply reflecting ray, then t → λ(σ, t) is a closed continuous curve in C \ {0}, so it has a well-defined winding number WN (σ) with respect to 0. This is the degree of the map R  t →

λ(σ, t) ∈ S1 |λ(σ, t)|

(see e.g. [Hir]). We can now prove the following simple but rather important fact. Proposition 13.6.1: Assume that K ∈ K and σ ∈ S ∗ (ΩK ) \ Trap(ΩK ) generates a simply reflecting ray γK (σ) in ΩK . Then the number of reflection points of γK (σ) equals WN (σ). Proof: Notice that for the number k of reflections of γ(σ) we have k = {t : λ(σ, t) ∈ R− }. Indeed, λ(σ, t) = ϕ(x(t))2 − ξ(t), ∇ϕ(x(t))2 + 2iϕ(x(t))ξ(t), ∇ϕ(x(t)) ∈ R− if and only if ϕ(x(t))ξ(t), ∇ϕ(x(t)) = 0 and ϕ(x(t))2 − ξ(t), ∇ϕ(x(t))2 < 0. These two relations can happen only when ϕ(x(t)) = 0 and ξ(t), ∇ϕ(x(t)) = 0, that is when γK (σ) has a simple (transversal) reflection at x(t). Moreover, if γK (σ) has a reflection at x(t0 ) for some t0 ∈ R, then for t near t0 we have Im(λ(σ, t)) = ϕ(x(t))ξ(t), ∇ϕ(x(t)) < 0 for t < t0 and Im(λ(σ, t)) > 0 for t > t0 . So, the curve λ(σ, t) intersects R− at t = t0 from below to above.2 Thus, WN (σ) = {t : λ(σ, t) ∈ R− }, so WN (σ) = k.



Corollary 13.6.2: Let K ∈ K and let σ(s), s ∈ [a, b], be a continuous curve in S ∗ (ΩK ) consisting of non-trapped points such that γK (σ(s)) is simply reflecting for all s ∈ [a, b]. Then the number of reflection points of γK (σ(s)) is the same for all s ∈ [a, b]. Proof: It is well known that winding numbers are homotopy invariant (see e.g. [Hir]). For any a ≤ s1 < s2 ≤ b the map Λ : R × [s1 , s2 ] −→ C \ {0}, defined by Λ(s, t) = λ(σ(s), t), is a continuous homotopy between the curves λ(σ(s1 ), t) and  λ(σ(s2 ), t). Now the assertion follows from Proposition 13.6.1. 2 That is, for t near t , when t increases, the point λ(σ, t) moves from the lower half of the complex 0 plane to the upper one.

376

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

We will now characterize the non-trapped points σ generating rays γK (s) containing a gliding segment on ∂K. Proposition 13.6.3: Assume that K ∈ K and σ ∈ S ∗ (Ω0 ) \ Trap(ΩK ). (a) If γK (σ) contains a gliding segment on ∂K, then for every continuous curve σ(s) (0 ≤ s ≤ a for some a > 0) in S ∗ (Ω0 ) with σ(0) = σ there exist infinitely many s ∈ (0, a] for which the trajectory γK (σ(s)) is tangent to ∂K at some of its points. (b) Suppose K ∈ K(fin) and γK (σ) does not contain a gliding segment on ∂K. Then there exists a continuous curve σ(s) (0 ≤ s ≤ a for some a > 0) in S ∗ (Ω0 ) with σ(0) = σ such that γK (σ(s)) is a simply reflecting ray for all s ∈ (0, a]. Proof: (a) Let σ(s), 0 ≤ s ≤ a, be a continuous curve in S ∗ (Ω0 ) with σ(0) = σ. Assume that for some  ∈ (0, a] the trajectory γK (σ(s)) is simply reflecting for all s ∈ (0, ]. Then by Corollary 13.6.2, the number k(s) of reflection points of γK (σ(s)) is constant: k(s) = k for all s ∈ (0, ]. However it follows from results in [MS2] and the assumption that γK (σ) contains a gliding segment on ∂K that we must have k(s) → ∞ as s  0, a contradiction. This proves the assertion. (b) In this case γK (σ) has only finitely many common points with S ∗ (∂K); some of them might be tangent points but these will be simple tangencies. Since σ∈ / Trap(ΩK ) and γK (σ) do not contain a gliding segment, it follows from Propositions 13.5.3 and 13.5.5 that there exist an open neighbourhood V of σ in S ∗ (Ω0 ) and finitely many smooth submanifolds S1 , . . . , Sk of V of positive codimension such that σ ∈ ∪ki=1 Si and any point ρ ∈ V \ ∪ki=1 Si generates a simply reflecting scattering ray in ΩK . As one can easily observe, we can find a continuous curve σ(s), 0 ≤ s ≤ a, so that σ(s) ∈ / ∪ki=1 Si for all s ∈ (0, a]. Then γK (σ(s)) will be a simply reflecting  ray in ΩK for all s ∈ (0, a]. We can apply the developments so far in this section to get a consequence concerning recovering information about an obstacle from its SLS. (fin)

Corollary 13.6.4: Assume that K, L ∈ K0 have almost the same SLS and that for some σ ∈ S ∗ (Ω0 ), γK (σ) is a scattering ray in ΩK containing a gliding segment on ∂K. Then γL (σ) is a scattering ray in ΩL containing a gliding segment on ∂L. Proof: Let σ = (x, ξ). Replacing ξ0 by −ξ0 if necessary, we may assume that there (K) exists T > 0 such that Ft (σ) ∈ S ∗ (Ω0 ) for all t ≤ 0 and all t ≥ T . Consider the (K) hyperplanes X and Y of Rn with x ∈ X, y = pr1 (Ft (σ)) ∈ Y and such that X and Y are perpendicular to γK (σ) at x and y, respectively. Define the cross-sectional maps PK , PL : S ∗ X −→ S ∗ Y as before. Then, by Theorem 13.1.2, PK = PL . It follows from Proposition 13.4.1 that γL (σ) contains a tangent point to ∂L. If γL (σ) does not contain a gliding ray on ∂L, it follows from Proposition 13.6.3(b) that we can find a continuous curve σ(s), 0 ≤ s ≤ a, in S ∗ (Ω0 ) with σ(0) = σ such that γL (σ(s)) is a simply reflecting ray for all s ∈ (0, a]. Since PK = PL , it follows that

INVERSE SCATTERING BY OBSTACLES

377

γK (σ(s)) is also a simply reflecting ray for all s ∈ (0, a]. This contradicts Proposition  13.6.3(a). Thus, γL (σ) must contain a gliding segment on ∂L. For an obstacle K in Rn denote by U (K) the set of these non-trapped points σ ∈ S ∗ (Ω0 ) such that γK (σ) is a simply reflecting ray. We know from Propositions 11.1.3 and 11.2.6 that U (K) is open and dense and has full Lebesgue measure in (fin) S ∗ (Ω0 ). Moreover, by Proposition 13.4.3, if K, L ∈ K0 have almost the same SLS, (K) (L) =U . then U Since S ∗ (S0 ) \ Trap(ΩK ) is an open subset of S ∗ (S0 ), it is a union of (possibly infinitely many) disjoint connected open subsets of S ∗ (S0 ). Clearly some of these connected components contain points ρ ∈ S ∗ (S0 ) generating γK (ρ) in ΩK , that is rays with no common points with ∂K. Definition 13.6.5: A point σ ∈ S ∗ (S0 ) will be called accessible if it belongs to a connected component of S ∗ (S0 ) \ Trap(ΩK ) containing a point that generates a free ray. Similarly, a connected component of S ∗ (S0 ) \ Trap(ΩK ) will be called accessible if it contains a point generating a free ray. Denote by A(K) the set of all accessible points of S ∗ (S0 ) \ Trap(ΩK ). Clearly, is a union of (open) connected components of S ∗ (S0 ) \ Trap(ΩK ), so this is A an open subset of S ∗ (S0 ). (K) Denote by U0 the set of all accessible points ρ ∈ U (K) , that is (K)

(K)

U0

= A(K) ∩ U (K) .

(K)

Clearly U0 is dense in A(K) . The central point in this section is the following. (fin)

Theorem 13.6.6: Assume that two obstacles K, L ∈ K0 have almost the same SLS. (a) For every connected component W of S ∗ (S0 ) \ Trap(ΩK ) there exists an integer m = m(K, L, W ) such that #(γK (σ) ∩ ∂K) = #(γL (σ) ∩ ∂L) + m

(13.10)

for all σ ∈ W ∩ U (K) . (b) If W is an accessible connected component of S ∗ (S0 ) \ Trap(ΩK ), then m(K, L, W ) = 0, that is #(γK (σ) ∩ ∂K) = #(γL (σ) ∩ ∂L)

(13.11)

for all σ ∈ W ∩ U (K) . The first step in the proof of the above theorem uses the submanifolds Mi of S ∗ (S0 ) \ (Trap(ΩK ) ∪ GK ) constructed in the proof of Proposition 13.5.5.

378

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS (K)

Lemma 13.6.7: Let σ0 ∈ Mi0 \ ∪i =i0 Mi for some i0 be such that ρ0 = Ft0 (σ0 ) ∈ Gd for some (unique) t0 > 0. Then for every smooth (C 1 ) curve σ(s), |s| < 0 , in S ∗ (S0 ) transversal to Mi0 at σ(0) = σ0 there exists  ∈ (0, 0 ] such that # [γK (σ(s )) ∩ ∂K] = # [γK (σ(s )) ∩ ∂K] for all − < s < 0 < s < . Proof of Lemma 13.6.7: Assume that σ(s), |s| < 0 , is a smooth curve in S ∗ (S0 ) transversal to Mi0 at σ0 = σ(0). By assumption γK (σ0 ) has only one tangent point (K) ρ0 to ∂K, that is {Ft (σ0 ) : t ∈ R} ∩ S ∗ (∂K) consists of one single point ρ0 . There exists an open neighbourhood V of ρ0 in S ∗ (Rn ) such that Hp2 ϕ > 0 on V . Let k ≥ 0 be the number of transversal reflections of γK (σ0 ) at ∂K. Taking the neighbourhood (K) V sufficiently small, we may assume that ρ0 is the only point of {Ft (σ0 ) : t ∈ R} ∩ S ∗ (∂K) contained in V . Now take  ∈ (0, 0 ] so small that {σ(s) : 0 < |s| < } ∩ (∪i Mi ) = ∅. (K)

Then for any 0 < |s| < , the ray {Ft (σ(s)) : t ∈ R} has only transversal reflection points (no tangencies) and their number is either k or k + 1, depending on whether the ray has a common point with ∂K near ρ0 or not. Consider the subsets G2 ∩ V ⊂ Γ = {ρ ∈ V : Hp ϕ(ρ) = 0} of V . It is easy to see that both are submanifolds of V , Γ has codimension 1 in V , while (K) G2 ∩ V has codimension 1 in Γ. Moreover, Γ is transversal to the flow Ft at ρ0 . Let (K)  ∗  V be an open neighbourhood of σ0 in S (S0 ) such that Ft0 (V ) ⊂ V . Shrinking V  if necessary and taking δ > 0 sufficiently small, for every σ ∈ V  we can find a real (K) number t(σ) ∈ (0, t0 − δ] close to t0 − δ with Ft(σ) (σ) ∈ V ∩ S ∗ (Rn \ K). Given σ ∈ V  , the forward trajectory of Hp in S ∗ (Rn ) starting at Ft(σ) (σ) (its projection in Rn is just a straight line) intersects Γ transversally at some point ψ(σ). This defines a smooth map ψ : V −→ Γ, and so ρ(s) = ψ(σ(s)) is a smooth curve in Γ. We now need to recall the construction of the submanifolds Mi from the proof of Proposition 13.5.5. In particular, since σ0 ∈ Mi0 and ρ0 = ψ(σ0 ) ∈ G2 ∩ V , the definition of the submanifold Mi0 shows that Mi0 = ψ −1 (G2 ∩ V ) locally near σ0 . In particular, the curve ρ(s) in Γ is transversal to G2 ∩ V at ρ0 . Recalling the definition of the set G2 from Section 1.2, we have (K)

G2 ∩ V = {ρ ∈ Γ : ϕ(ρ) = 0}. Since Hp ϕ is a defining function Hp ϕ for Γ and the differentials of Hp ϕ and ϕ are linearly independent of V , we have d . ϕ(ρ(s))|s=0 = ∇ϕ(ρ0 ), ρ(0) = 0, ds

INVERSE SCATTERING BY OBSTACLES

379

so ϕ(ρ(s)) has a different sign for s < 0 and for s > 0 (s close to 0). Then for s in one (K) of the intervals [−δ, 0) and (0, δ], the trajectory {Ft (σ(s)) : t ∈ R} has a common ∗ n point with S∂K (R ) near ρ0 , while for s in the other interval, it does not. This proves  the assertion. Proof of Theorem 13.6.6: Let {Ni } and {Mi } be the submanifolds of S ∗ (S0 ) (L) constructed in Propositions 13.5.3 and 13.5.5 for the obstacle K, and let {Ni } and (L) {Mi } be corresponding submanifolds of S ∗ (S0 ) for the obstacle L. We will also (K) (L) use the particular construction of the submanifolds Mi and Mi in the proof of Proposition 13.5.5. Consider an arbitrary connected component W of S ∗ (S0 ) \ Trap(ΩK ) = ∗ S (S0 ) \ Trap(ΩL ), an arbitrary point σ0 ∈ W ∩ U (K) = W ∩ U (L) and set (K)

(K)

m = #(γK (σ0 ) ∩ ∂K) − #(γL (σ0 ) ∩ ∂L). Fix an arbitrary σ1 ∈ W ∩ U . We will show that (13.10) holds for σ = σ1 and the number m defined above. Since σ0 ∈ W ⊂ S ∗ (S0 ) \ Trap(ΩK ), we can take a smooth curve σ(s), 0 ≤ s ≤ a, in S ∗ (S0 ) \ Trap(ΩK ) such that σ(0) = σ0 , σ(a) = σ1 . Then take a compact neighbourhood W0 of the curve λ = {σ(s) : s ∈ [0, a]} in S ∗ (S0 ) \ Trap(ΩK ). Slightly perturbing λ in the interior of W0 , we can (K) (L) (K) (L) make it transversal to any of the submanifolds Mi , Mi , Ni and Ni (K) (L) (see e.g. [Hir]). However, Ni and Ni have codimension 2 in S ∗ (S0 ), so the (K) (L) transversality to these submanifolds simply means that λ ∩ Ni = λ ∩ Ni = ∅ for all i. Recalling Proposition 13.5.3, we now obtain λ ∩ GK = λ ∩ GL = ∅. On the other hand, GK ∩ W0 and GL ∩ W0 are closed in W0 since W0 is a compact set without common points with Trap(ΩK ) = Trap(ΩL ). So, shrinking W0 if necessary, we may assume W0 ∩ GK = W0 ∩ GL = ∅. (K) Next, since the family {Mi } is locally finite in S ∗ (S0 ) \ (Trap(ΩK ) ∪ GK ), (K) we have that λ ∩ Mi = ∅ only for finitely many i. So, there are only finitely many s1 , . . . , sp ∈ [0, a] such that γK (σ(sj )) contains a tangent point to ∂K. According to Proposition 13.4.3, γL (σ(s)) has a tangency to ∂L for exactly the same values of s, that is for s = s1 , . . . , sp . Now by Proposition 13.4.2 (or rather, its proof in Section 13.8; see also Remark 13.5.6), we can perturb the curve σ(s) near each sj , so that each of the rays γK (σ(si )) and γL (σ(si )) has a single point of tangency to ∂K and ∂L, respectively, for any i = 1, . . . , p. Fix for a moment s ∈ [0, a]. Proceeding for example as in the proof of Proposition (K) 13.4.3, we may assume that there exists T > 0 such that Ft (σ(s)) ∈ S ∗ (Ω0 ) for all t ≤ 0 and all t ≥ T . Consider the hyperplanes X = X(s) and Y = Y (s) of Rn (K) with x = x(s) ∈ X, y = y(s) = pr1 (Ft (σ(s))) ∈ Y and such that X and Y are perpendicular to γK (σ(s)) at x and y, respectively. Define the cross-sectional map PK (s) : S ∗ X −→ S ∗ Y as before, and observe (as before) that PK (s) = PL (s) for all s.

380

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Set s0 = 0, sp+1 = a. It follows from Corollary 13.6.2 that for every j = 0, 1, . . . , p there exists a constant kj (resp. j ) such that for every s ∈ (sj , sj+1 ), the number of reflection points of γK (σ(s)) (resp. γL (σ(s))) is equal to kj (resp. j ). We claim that kj = j + m for all j. We will prove this by induction. By the definition of m we have k0 = 0 + m. Assume kj = j + m for some j ≤ p. By construction, σ(sj ) ∈ Mi for some i and γK (σ(sj )) has a single tangency to ∂K which is the projection of a point in G2 \ G3 . Moreover, the curve σ(s) is transversal to Mi at σ(sj ). Applying Lemma 13.6.7, it follows now that either kj+1 = kj + 1 or kj+1 = kj − 1. By the same argument, lj+1 = lj ± 1. The difference between the cases kj+1 = kj + 1 and kj+1 = kj − 1 is that in the former we have lim supssj ||dPK (s)|| < ∞, while in the latter lim supssj ||dPK (s)|| = ∞. Since PK (s) = PL (s), we now see that in the former case we must have j+1 = j + 1, while in the latter, j+1 = j − 1. Hence kj+1 = j+1 + m. By induction, kj = j + m for all j. This proves the relationship (13.10) for σ = σ1 , and thus completes the proof of part (a). (b) This follows trivially from part (a), since if W is accessible, we can choose σ0 so that it generates a free ray in ΩK . By Theorem 13.1.1, γL (σ0 ) is also a free ray,  so we must have m = 0 in (13.10). That is, (13.11) holds.

13.7

Recovering the accessible part of an obstacle

It is natural to expect that the accessible points on the boundary ∂K of an obstacle K will be easier to recover using the SLS of the generalized geodesic flow in ΩK . In this section we will demonstrate that to a big extend this is indeed so (see Definition 13.6.5 for the definition of an accessible point). Throughout K ∈ K(fin) will be a fixed obstacle in Rn , n ≥ 2. As before S0 will denote the boundary sphere of a large ball O in Rn containing the obstacle K. Fix (K) a countable set {Mi } = {Mi } of submanifolds of S ∗ (S0 ) with the properties described in Proposition 13.5.5 (see also Remark 13.5.6). We will also use the set (K) U (K) and U0 from Section 13.5. Definition 13.7.1: Let σ(s), 0 ≤ s ≤ a, be a smooth curve in S ∗ (S0 ) for some a > 0, and let the family {Mi } be as above. The curve σ(s) will be called regular if it satisfies the following conditions: (i) γK (σ(0)) is a free ray in ΩK , (ii) σ(a) ∈ / ∪i Mi ; (iii) σ(s) ∈ / Trap(ΩK ) ∪ GK for all s ∈ [0, a];  i for all i and σ(s) ∈ (iv) σ M / Mi ∩ Mj for any i = j and any s ∈ [0, a]. Clearly, by (ii) and (iii), γK (σ(a)) is a simply reflecting ray, while (iii) and (iv) show that all trajectories γK (σ(s)) are scattering rays with at most one tangent point

INVERSE SCATTERING BY OBSTACLES

381

to ∂K. Notice that if σ(s) is a regular curve as above and ρ(s) (0 ≤ s ≤ a) is uniformly close to σ(s) in the C 1 topology, then ρ(s) is also regular. The following is easy to derive using the argument in the first part of the proof of Theorem 13.6.6. We leave the details to the reader. (K)

Lemma 13.7.2: For every σ0 ∈ U0 with σ(a) = σ0 .

there exists a regular curve σ(s), 0 ≤ s ≤ a,

We will now define a sequence of subsets of the boundary ∂K of K. Definition 13.7.3: The sequence ∂K (1) ⊂ ∂K (2) ⊂ · · · ⊂ ∂K (m) ⊂ · · · ⊂ ∂K is defined recursively as follows: (a) Set ∂K (0) = ∅. (b) Let ∂K (1) be the set of those x ∈ ∂K such that there exists a regular curve σ(s) (0 ≤ s ≤ a) in S ∗ (S0 ) with γK (σ(s)) having at most one common point with ∂K for all s ∈ [0, a] and x ∈ γK (σ(a)). (c) Assume that the subsets ∂K (1) ⊂ · · · ⊂ ∂K (m) of ∂K have been constructed already for some m ≥ 1. Denote by ∂K (m+1) the set of those x ∈ ∂K such that there exists a regular curve σ(s) (0 ≤ s ≤ a) in S ∗ (S0 ) with γK (σ(s)) having at most one common point with ∂K that is not in ∂K (m) for all s ∈ [0, a] and x ∈ γK (σ(a)). Clearly, each of the sets ∂K (m) is open in ∂K. The strongly accessible part of ∂K is by definition (m) . ∂K (∞) = ∪∞ m=1 ∂K

The obstacle K will be called strongly accessible if ∂K (∞) = ∂K. Simple examples show that in many cases the strongly accessible part of the obstacle is significant, in fact even the set ∂K (1) is already substantial. For example, it is easy to see that the star-shaped obstacles from Section 13.3 are strongly accessible. It seems natural to conjecture that all non-trapping obstacles are strongly accessible, however no proof of this is known to the authors. As is natural to expect, for ‘relatively regular’ obstacles, the strongly accessible part of the boundary can be completely recovered from the SLS of the obstacle. This is the main result in this section. (fin)

Theorem 13.7.4: Let K, L ∈ K0 be two obstacles in Rn , n ≥ 2, with almost the same SLS. (a) We have ∂K (m) = ∂L(m) for all m ≥ 0, and therefore ∂K (∞) = ∂L(∞) . (b) If in addition it is known that K is strongly accessible, then L = K ∪ L for some connected component L of L with L ∩ K = ∅.

382

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

(c) If in addition it is known that K is strongly accessible and any connected component of L can be reached by a trajectory γL (ρ) generated by an accessible point ρ ∈ S ∗ (S0 ) \ Trap(ΩL ), then K = L. Proof: (a) We will use induction on m. Assume ∂K (m−1) = ∂L(m−1) for some m ≥ 1. We have to prove ∂K (m) = ∂L(m) . By symmetry, it is enough to show that ∂K (m) ⊂ ∂L(m) . (K)

Given x ∈ ∂K (m) , there exist a regular (with respect to the submanifolds Mi of S ∗ (S0 )) curve σ(s), 0 ≤ s ≤ a, with γK (σ(s)) having at most one common point with ∂K that is not in ∂K (m−1) for all s ∈ [0, a] and x ∈ γK (σ(a)). Thus, (K) σ(a) = Ft (x, ξ) for some t > 0 and ξ ∈ Sn−1 with ξ, νK (x) > 0. Applying an arbitrarily small in the C 1 Whitney topology (see e.g. [Hir]) perturbation of the curve σ(s) in S ∗ (S0 ), we may assume that the curve σ is transversal to each of the subman(L) (L) ifolds Ni , Mi , it has the same end point σ(a), and that σ(s) belongs to at most (L) one of the submanifolds Mi for all s ∈ [0, a] (see the proof of Theorem 13.6.6). As (L) (L) before, since the codimension of each Ni in S ∗ (S0 ) is 2, one gets σ(s) ∈ / ∪i Ni , and therefore σ(s) ∈ / GL for all s ∈ [0, a]. Finally, Proposition 13.4.3 and (K) (L) σ(a) ∈ / GK ∪ ∪i Mi imply σ(a) ∈ / GL ∪ ∪i Mi . All this shows that the perturbed (L) curve σ(s) (0 ≤ s ≤ a) is a regular curve with respect to the family {Mi }. Lemma 13.7.5: We have ∂K = ∂L in a neighbourhood of the point x and γK (σ(s)) = γL (σ(s))

(13.12)

for all s ∈ [0, a]. Proof of Lemma 13.7.5: Since γK (σ(0)) is a free ray in Rn , and so γK (σ(0)) = γL (σ(0)), we have s0 = sup {s ∈ [0, a] : γK (σ(t)) = γL (σ(t)),

0 ≤ t ≤ s } > 0.

We will prove that s0 = a. Assume s0 < a. By continuity, ρ0 = σ(s0 ) ∈ S ∗ (S0 ) satisfies (13.13) γK (ρ0 ) = γL (ρ0 ). For brevity set (K)

U0 = U0

(L)

= U0

⊂ U = U (K) = U (L) .

By Theorem 13.6.6(b), #(γK (ρ) ∩ ∂K) = #(γL (ρ) ∩ ∂L)

(13.14)

for all ρ ∈ U0 . Denote by x1 , . . . , xk the successive common points of γK (ρ0 ) with ∂K. Since ρ0 = σ(s0 ) and σ is a regular curve, there is at most one i0 = 1, . . . , k such that

INVERSE SCATTERING BY OBSTACLES

383

xi0 ∈ / ∂K (m−1) . If there is no such i0 , there is nothing to prove. Assume that x = xi0 ∈ / ∂K (m−1) for some i0 , and so xi ∈ ∂K (m−1) = ∂L(m−1) for all i = i0 . There are two possible cases to consider. Case 1. xi is a transversal reflection point of γK (ρ0 ) for all i = 1, . . . , k. Then by Proposition 13.4.3, γL (ρ0 ) also has only transversal reflections at ∂L, that is no tangencies to ∂L. By (13.13), the reflection points of γL (ρ0 ) coincide with those of γK (ρ0 ), that is they are x1 , . . . , xk . Clearly this implies that every ρ ∈ S ∗ (S0 ) sufficiently close to ρ0 generates a scattering ray γK (ρ) with exactly k common points x1 , . . . , xk with ∂K and also a scattering ray γL (ρ) with exactly k common points x1 , . . . , xk with ∂L, where for all i the points xi and xi are close to xi . By assumption ∂K (m−1) = ∂L(m−1) . Since this is open in both ∂K and ∂L, assuming ρ is sufficiently close to ρ0 , we have xi , xi ∈ ∂K (m−1) = ∂L(m−1) for all (K) (L) i = i0 . By Theorem 13.1.2, Ft (ρ) = Ft (ρ) for |t|  0, so we must have xi = xi for i < i0 and also for i > i0 . The only other reflection points of γK (ρ) and γL (ρ) are xi0 and xi0 , respectively, and now a simple geometric argument implies that we must have xi0 = xi0 . That is, γK (ρ) = γL (ρ) for all ρ ∈ S ∗ (S0 ) sufficiently close to ρ0 = σ(s0 ). This is a contradiction with the choice of s0 which proves that we must have s0 = a. Thus, (13.12) holds for all s ∈ [0, a]. Moreover, the above argument, replacing ρ0 by σ(a), shows that ∂K = ∂L in a neighbourhood of the point x. Case 2. γK (ρ0 ) has a tangency to ∂K at some point xq ∈ ∂K. This tangent point is then unique by the definition of a regular curve. Clearly, there are s ∈ (0, a] arbi(K) trarily close to a for which σ(s) ∈ U0 , so (13.14) holds with ρ = σ(s) for such s. Hence (13.14) holds for ρ = ρ0 as well. Take an open connected neighbourhood W of ρ0 in S ∗ (S0 ) \ (Trap(ΩK ) ∪ GK ) so small that both γK (ρ) and γL (ρ) have transversal reflections near xi for any / GK , we may assume W is taken i = q and any ρ ∈ W . Using the fact that ρ0 ∈ so small that for any ρ ∈ W the trajectory γK (ρ) has at most k common points with ∂K. Indeed, suppose this is not the case. Since W ∩ Trap(ΩK ) = ∅, we can find a sequence {ρp } ⊂ S ∗ (S0 ) such that ρp → ρ0 as p → ∞ and for every p there (K) (K) exist real numbers tp = tp with pr1 (Ftp (ρp )) ∈ ∂K and pr1 (Ftp (ρp )) ∈ ∂K for  all p. Choosing an appropriate subsequence, we may assume tp → t and tp → t as (K) p → ∞ for some t ∈ R. This gives Ft (ρ0 ) ∈ S ∗ (∂K) and also it follows from (K) known facts (cf. e.g. Section 24.3 in [H3]) that Ft (ρ0 ) ∈ G3 . This is a contradic/ GK . Thus, shrinking the open neighbourhood W of ρ0 , we may assume tion with ρ0 ∈ that for any ρ ∈ W , the trajectory γK (ρ) has at most k common points with ∂K. Fix a neighbourhood W with this property. We now claim that (K)

Ft

(L)

(ρ) = Ft (ρ),

ρ ∈ W, t ∈ R.

(13.15)

Let xi = pr1 (Fti (ρ0 )) for some 0 < t1 < · · · < tk , and let Ft (ρ0 ) ∈ S ∗ (S0 ) (K) for some T > tk . Then pr1 (Ft (ρ0 )) will be the last common point of γK (ρ0 ) with O before it ‘escapes to ∞’. Taking a sufficiently small  > 0 with ti0 −1 + 2 < ti0 , ti0 + 2 < ti0 +1 and tk + 2 < T , and shrinking W if necessary, we may assume (K)

(K)

384

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

that Ft (ρ) = Ft (ρ) for all ρ ∈ W and all t < ti0 − . Set t0 = ti0 , t = t0 −  (K) (L) and t = t0 + . By Theorem 13.1.2, Ft (ρ) = Ft (ρ) for ρ ∈ W and all t  0 and t  T . Since ∂K = ∂L near xi for all i = i0 , it then follows that (K)

(L)

(K)

Ft

(L)

(ρ) = Ft (ρ),

ρ ∈ W, t ≤ t , t ≥ t . (K)

Moreover, the ‘short trajectory’ δK = {pr1 (Ft

(13.16)

(ρ)) : t ∈ [t , t ]} is either:

(i) a straight-line segment (this can only happen if q = i0 ), which may have a common point with ∂K near xi0 , or (ii) δK has just one transversal reflection at some point x (ρ) ∈ ∂K near xi0 (this will necessarily happen if q = i0 and it may also happen if q = i0 ). Similarly, the ‘short trajectory’ δL = {pr1 (Ft (ρ)) : t ∈ [t , t ]} has either one or zero common points with ∂L. It is now easy to see that in case (i), δL must also be a straight-line segment, and moreover (13.16) implies δL = δK . In the case (ii), a similar argument shows that δL = δK again. This proves (13.15). (K) Let ρ0 = (y0 , η0 ) = Ft0 (ρ0 ); then xi0 = y0 . Using the continuity of the flow (K) (K) Ft , there exists an open neighbourhood V of ρ0 in S ∗ (ΩK ) such that F−t(ω) (ω) ∈ W for all ω ∈ V , where t(ω) is a number close to t0 = ti0 . Then there exists an open neighbourhood V  of y0 in ∂K such that for every y ∈ V  , we can find a vector η ∈ Sy (∂K) close to η0 such that ω = (y, η) ∈ V . Setting t = t(ω) and using ρ = (K) F−t (ω) ∈ W and (13.15), we get (L)

(K)

y = pr1 (Ft

(K)

(K)

(F−t (ω)) = pr1 (Ft

(L)

(ρ)) = pr1 (Ft (ρ)) ∈ ∂L.

Thus, V  ⊂ ∂L which shows that ∂K = ∂L near xi0 . So, by (13.15) we have γK (ρ) = γL (ρ) for all ρ ∈ S ∗ (S0 ) sufficiently close to ρ0 = σ(s0 ). This a contradiction with the choice of s0 which proves s0 = a. Finally, as in Case 1, repeating the latest argument above, replacing ρ0 by σ(a), shows that ∂K = ∂L in a neighbourhood of the point x. This completes the proof in Case 2,  thus proving the lemma. We now continue with the proof of the theorem. Lemma 13.7.5 gives x ∈ ∂L(m) , and so this proves ∂K (m) ⊂ ∂L(m) . By symmetry ∂K (m) = ∂L(m) . Hence by induction ∂K (m) = ∂L(m) for all m ≥ 0, and therefore ∂K (∞) = ∂L(∞) . (b) Assume that K is strongly accessible, that is ∂K (∞) = ∂K. Then the above relation implies ∂K ⊂ ∂L. This is only possible when L = K ∪ L for some (obstacle) L with L ∩ K = ∅. (c) Assume that K is strongly accessible and any connected component of L can be reached by a scattering ray γL (ρ) for some accessible ρ ∈ S ∗ (S0 ) \ Trap(ΩL ). By part (b), K = L ∪ L for some obstacle L . Assume L = ∅, otherwise there is nothing to prove. Let L be a connected component of L . By assumption, there exists an accessible ρ ∈ S ∗ (S0 ) \ Trap(ΩL ) such that γL (ρ) ∩ ∂L = ∅.

INVERSE SCATTERING BY OBSTACLES (L)

385

(L)

We can choose ρ so that ρ ∈ / ∪i Ni ∪ ∪i Mi . Repeating an argument from the proof of Theorem 13.6.6, we find a smooth curve σ(s), 0 ≤ s ≤ 1, in (L) S ∗ (S0 ) \ (Trap(ΩL ) ∪ GL ) transversal to each of the submanifolds Mi such that σ(0) generates a free ray in ΩL and σ(1) = ρ. Set s = min{s ∈ [0, 1] : γL (σ(s)) ∩ ∂L = ∅} and ρ = σ(s ). Then the trajectory γL (ρ ) must have a tangency to ∂L at some point y ∈ ∂L ; otherwise we get a contradiction with the choice of s . It now follows from the choice of the curve σ that γL (ρ ) has no other tangencies to L, and the choice of s shows that γL (ρ ) has no other common points with L . Thus, all other common points of γL (ρ ) and L belong to L \ L = K, so γK (ρ ) = γL (ρ ). However this is a contradiction with Proposition 13.4.3 since γK (ρ ) has no tangency to ∂K, while γL (ρ ) has a tangent  point to ∂L. Hence L = ∅ and therefore K = L.

13.8

Proof of Proposition 13.4.2

Most of this section is taken by the proofs of two technical lemmas. These proofs are relatively elementary however rather lengthy. Lemma 13.8.1: Let X be a C ∞ smooth submanifold of codimension 1 in Rn , and let x0 ∈ X and ξ0 ∈ Tx X, ||ξ0 || = 1, be such that the normal curvature of X at x0 in the direction ξ0 is non-zero. Then for every  > 0 there exist an open neighbourhood V of x0 in X, a smooth map V  x → ξ(x) ∈ Tx X and a smooth positive function t(x) ∈ [δ, ] on V for some δ ∈ (0, ) such that Y = {y(x) = x + t(x)ξ(x) : x ∈ V } is a smooth strictly convex surface with unit normal field μ(y(x)) = ξ(x), x ∈ V . That is, the normal field of Y consists of vectors tangent to X at the corresponding points of V . Proof of Lemma 13.8.1: Considering X with the Riemannian metric induced by Rn , there exists a local smooth codimension 1 submanifold X  of X containing x and perpendicular to ξ0 at x0 such that the second fundamental form of X  in X with respect to the normal ξ0 is negative definite at x0 . For example, we can take an appropriate strictly convex (n − 1)-dimensional submanifold Z of Rn containing x0 and having ‘outward’ unit normal ξ0 at x0 , and set X  = X ∩ Z. Parameterize X  by h(u ), u = (u2 , . . . , un−1 ), h(0) = x0 , and let ξ(u ) be a continuous unit normal field to X  with ξ(u ) ∈ Th(u ) X for all u and ξ(0) = ξ0 . For any u let c(t; u ) be the geodesic on X . parameterized by arc length t such that c(0; u ) = h(u ) and c(0; u ) = ξ(u ). Define r(u1 , u2 , . . . , un−1 ) = c(u1 ; u ).

386

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

It then follows that for |u1 | and ||u || small enough r(u) is a smooth parameterization ∂r (0) and of an open neighbourhood V of x0 in X such that x0 = r(0), ξ0 = ∂u 1      ∂r  ∂r ∂r   (u), (u) = 0, i > 1, u ∈ U. (13.17)  ∂u (u) = 1, ∂u1 ∂ui 1 Shrinking the neighbourhood V of x0 if necessary, we may assume that u runs over some open ball U in Rn−1 . Notice that the second fundamental form of X  in X at x0 has the form   n−1  ∂2r vi vj ξ0 , (0) , II  (v  ) = ∂ui ∂uj i,j=2 where v  = (v2 , . . . , vn−1 ) ∈ Rn−2 . So the choice of X  implies II  (v  ) < 0 whenever v  = 0. Fix a small  > 0 and set y(u) = r(u) + ( − u1 )

∂r (u), ∂u1

u ∈ U.

Shrinking the ball U if necessary, we may assume that |u1 | < /2 for all u ∈ U . It will become clear later how small  should be. We claim that Y = {y(u) : u ∈ U } is a smooth submanifold of Rn (provided  > 0 and U are small enough), y(u) is a smooth parameterization of Y and ∂r (u) is a normal vector to Y at y(u). μ(u) = ∂u 1 First, notice that differentiating (13.17) with respect to uj , implies   ∂2r ∂r (u), (u) = 0, 1 ≤ j ≤ n − 1, (13.18) ∂u1 ∂u1 ∂uj 

   ∂2r ∂r ∂r ∂2r (u), (u) + (u), (u) = 0, ∂u1 ∂uj ∂ui ∂u1 ∂ui ∂uj 2 ≤ i ≤ n − 1, 1 ≤ j ≤ n − 1.

From these two equalities it follows in particular that ∂2r (u)is ∂u21

(13.19) ∂2r ∂r (u)⊥ ∂u (u) ∂u21 i

for any

i = 1, . . . , n − 1, and therefore a normal vector to X at r(u). On the other hand, the assumption that the normal curvature of X at x0 = r(0) in the direction of ∂r ∂2r ξ0 = ∂u (0) is non-zero implies that ∂u 2 (0) = 0. In particular, the vectors 1 1

2

∂r ∂r ∂ r (0), (0), . . . , (0) ∂u21 ∂u2 ∂un−1 in Rn are linearly independent. Without loss of generality, we may assume that the matrix formed by the first n − 1 coordinates of these vectors has a non-zero determinant.

INVERSE SCATTERING BY OBSTACLES

Since ∂2r ∂y (u) = ( − u1 ) 2 (u), ∂u1 ∂u1

∂y ∂r ∂2r (u) = (u) + ( − u1 ) (u), ∂ui ∂ui ∂u1 ∂ui

387

i > 1,

(13.20) ∂r (u) is a normal vector to Y at y(u), (13.17) and (13.18) imply that μ(u) = ∂u 1 provided Y is a smooth submanifold with parameterization y(u). To prove that Y is locally a smooth (n − 1)-dimensional submanifold of ∂y (0) Rn , it is enough to show that if  > 0 is small enough, then the vectors ∂u i (1 ≤ i ≤ n − 1) are linearly independent. Assume the contrary, that is there exists ∂y (0) are linearly a > 0 such that for any  ∈ (0, a] the corresponding vectors ∂u i dependent. Let y(u) = (y1 (u), . . . , yn (u)) and r(u) = (r1 (u), . . . , rn (u)). Define z(u) = (y1 (u), . . . , yn−1 (u)) and h(u) = (r1 (u), . . . , rn−1 (u)). Then the vectors ∂z ∂ui (0) (1 ≤ i ≤ n − 1) are also linearly dependent, so we must have ⎞ ⎛ ⎛ ∂z ⎞ ∂2h  ∂u 2 (0) ∂u1 (0) 1 ⎟ ⎜ ⎜ ∂z ⎟ ⎜ ∂h (0) +  ∂ 2 h (0) ⎟ ⎜ ∂u2 (0) ⎟ ∂z ⎟ ⎜ ∂u2 ∂u1 ∂u2 ⎟ 0 = det (0) = det ⎜ ⎟, ⎜ . . . ⎟ = det ⎜ ⎟ ⎜ ∂u · · · ⎝ ⎠ ⎠ ⎝ 2 ∂z ∂h ∂ h (0) +  (0) ∂un−1 (0) ∂un−1 ∂u1 ∂un−1 where the rows in the matrices above are vectors in Rn−1 . Dividing by  the first row of the determinant in the right-hand side and then letting  → 0, we obtain that the matrix formed by the first n − 1 coordinates of the vectors ∂r ∂r ∂2r (0), (0), . . . , (0) ∂u21 ∂u2 ∂un−1 has a zero determinant – contradiction with our assumption above. ∂y (0) (1 ≤ i ≤ Thus, there exists arbitrarily small  > 0 such that the vectors ∂u i n − 1) are linearly independent. Given such an , shrinking U if necessary, we may ∂y (u) (1 ≤ i ≤ n − 1) are linearly independent of any u ∈ U . Then Y assume that ∂u i is a smooth (n − 1)-dimensional submanifold of Rn and y(u) is a smooth parameteri∂r (u) is then a unit normal to Y at y(u). zation of Y . As we observed above, μ(u) = ∂u 1 Moreover, by the definition of y(u), the segment [r(u), y(u)] is tangent to X at r(u). It remains to show that the normal curvature of Y with respect to the normal field μ(u) is negative. For this we need the second derivatives of y(u) which we get from (13.20): ∂2r ∂3r ∂2y (0) = − (0) +  (0), ∂u21 ∂u21 ∂u31

∂2y ∂3r (0) =  2 (0), 2 ≤ i ≤ n − 1, ∂u1 ∂ui ∂u1 ∂ui

∂2y ∂2r ∂3r (0) = (0) +  (0), 2 ≤ i, j ≤ n − 1. ∂ui ∂uj ∂ui ∂uj ∂u1 ∂ui ∂uj 

Hence the coefficients cij =

 ∂2y μ(0), (0) ∂ui ∂uj

388

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

of the second fundamental form of Y at y(0) have the form:  c1j =   cij =

 ∂r ∂3r (0), (0) , ∂u1 ∂u1 ∂u21 ∂uj

2≤j ≤n−1,

   ∂2r ∂3r ∂r ∂r (0), (0) +  (0), (0) , 2 ≤ i, j ≤ n − 1. ∂u1 ∂ui ∂uj ∂u1 ∂u1 ∂ui ∂uj

On the other hand, differentiating (13.18) with respect to ui , one gets 

  2  ∂r ∂3r ∂ r ∂2r (0), (0) = − (0), (0) . ∂u1 ∂u1 ∂ui ∂uj ∂u1 ∂ui ∂u1 ∂uj

Thus, the second fundamental form II(v), v = (v1 , . . . , vn−1 ) ∈ Rn−1 , of Y at y(0) has the form II(v) = 

n−1 

vi vj cij =

i,j=1

+

n−1 

 vi vj

i,j=1

 vi vj

i,j=2

= −

n−1 

n−1 

 ∂r ∂2r (0), (0) ∂u1 ∂ui ∂uj

 vi vj

i,j=1

 ∂r ∂3r (0), (0) ∂u1 ∂u1 ∂ui ∂uj

 ∂2r ∂2r (0), (0) + II  (v  ) ∂u1 ∂ui ∂u1 ∂uj

= − ||w||2 + II  (v  ), where v  = (v2 , . . . , vn−1 ) and w=

n−1  i=1

vi

∂2r (0). ∂u1 ∂ui

Since by the choice of the submanifold X  we have II  (v  ) < 0 for v  = 0, it now follows that II(v) < 0 whenever v = 0. Thus, Y is strictly convex at y(0) with ∂r (0), and therefore the same conclusion holds for respect to the normal μ(0) = ∂u 1 any y(u) close enough to y(0). So, shrinking U (and therefore V ), we get that the  whole submanifold Y is strictly convex in Rn . Let again X be a smooth bounded (n − 1)-dimensional submanifold of Rn , n ≥ 2. Recall the definition of an (ω, θ)-trajectory for X from Section 6.2. Let O be an open ball containing X. Given ω ∈ Sn−1 , define the hyperplane Zω as in Section 13.2. For an integer p ≥ 1 consider the smooth manifolds X (p) = {(x1 , . . . , xp ) ∈ X p : xi = xj , i = j },

Mp = Sn−1 × X (p) × Sn−1 .

INVERSE SCATTERING BY OBSTACLES

389

Fix integers k, m and s ≥ 1 and 0 ≤ k < m ≤ s. Denote by M (s, k, m) the set of those η = (ω; x; y; z; θ) ∈ Ms+2 with x = (x1 , . . . , xs ), y, z ∈ X, such that there exists an (ω, θ)-trajectory for X with successive (transversal) reflection points x1 , . . . , xs , the segment [xk , xk+1 ] of which is tangent to X at the point y ∈ (xk , xk+1 ), the segment [xm , xm+1 ] is tangent to X at z ∈ (xm , xm+1 ) and the Gauss curvature of X either at y or at z is non-zero. Here by x0 (resp. xs+1 ) we denote the orthogonal projection of x1 on Zω (resp. of xs on Z−θ ). Lemma 13.8.2: M (s, k, m) is a smooth submanifold of Ms+2 of dimension 2n − 4. Proof of Lemma 13.8.2: We will use an argument similar to that in the proof of Lemma 11.5.1. Assume 0 < k and m < s; the cases k = 0 and/or m = s are similar. ˆ ∈ M (s, k, m), choose smooth charts ϕ : U −→ X Given ηˆ = (ˆ ω; x ˆ; yˆ; zˆ; θ) i i of X around x ˆi , ψ : V −→ X of X around yˆ and χ : W −→ X of X around zˆ such that ϕi (Ui ) ∩ ϕi+1 (Ui+1 ) = ∅, i = 1, . . . , s − 1, ϕk (Uk ) ∩ ψ(V ) = ∅, ϕk+1 (Uk+1 ) ∩ ψ(V ) = ∅, ϕm (Um ) ∩ χ(W ) = ∅ and ϕm+1 (Um+1 ) ∩ χ(W ) = ∅. ˆ (say, ω  Let ω(ω  ), ω  ∈ D1 ⊂ Rn−1 , be a smooth parameterization of Sn−1 near ω   is an appropriate choice of n − 1 coordinates of ω), and let θ(θ ), θ ∈ D2 ⊂ Rn−1 ˆ Consider the chart be a similar parameterization of Sn−1 near θ. Φ : U = D1 × U1 × · · · × Us × V × W × D2 −→ D ⊂ Ms+2 , defined by Φ(ξ) = (ω(ω  ); ϕ1 (u1 ), . . . , ϕs (us ); ψ(v); χ(w); θ(θ )) for ξ = (ω  ; u; v; w; θ ) ∈ U . Because of the symmetry of the roles of k and m, we may assume that the Gauss ˆ = ηˆ. ˆ; vˆ; w; ˆ θˆ ) ∈ U be such that Φ(ξ) curvature of X at yˆ is non-zero. Let ξˆ = (ωˆ ; u v )⊥ˆ xk+1 − x ˆk , where ν1 (ˆ v ) is the naturally Notice that ηˆ ∈ M (s, k, m) implies ν1 (ˆ defined normal to X at ψ(ˆ v ) (see below). Choosing an appropriate coordinate system in Rn , we may assume that x ˆk+1 − x ˆk = (0, . . . , 0, a),

ν1 (ˆ v ) = (1, 0, . . . , 0)

(13.21)

for some a > 0. Shrinking Um and Um+1 if necessary, we can find i0 = 1, . . . , n so that (i0 ) 0) (um+1 ) − ϕ(i um ∈ Um , um+1 ∈ Um+1 . ϕm+1 m (um ) = 0, Fix i0 with this property and set B = {1, . . . , n} \ {i0 }. Define F : U −→ R by F (ξ) =

s−1  i=1

||ϕi (ui ) − ϕi+1 (ui+1 )||.

390

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS (1)

(n−1)

Here ui = (ui , . . . , ui

) ∈ Ui . Let

f1 = (1, 0, . . . , 0), . . . , fn = (0, . . . , 0, 1). As in the proof of Lemma 11.5.1, to express the condition ξ = (ω  ; u; v; w; θ ) ∈ Φ−1 (M (s, k, m)), we will use the naturally defined normals ⎛ ··· f1 ⎜ ∂ψ(1) ⎜ ∂v(1) (v) · · · ⎜ ν1 (v) = det ⎜ .. .. ⎜ . . ⎝ ∂ψ (1) (v) ∂v (n−1)

⎛

and

⎜ ⎜ ⎜ ν2 (w) = det ⎜ ⎜ ⎝

fn

⎞

⎟ ∂ψ (n) (v)⎟ ∂v (1) ⎟ ⎟ ⎟ ⎠

.. .

∂ψ (n) ∂v (n−1)

···

⎞

f1

···

fn

∂χ(1) (w) ∂w(1)

∂χ(n) (w) ∂w(1)

.. .

··· .. .

∂χ(1) (w) ∂w(n−1)

···

∂χ(n) (w) ∂w(n−1)

⎟ ⎟ ⎟ ⎟ ⎟ ⎠

.. .

to X at ψ(v) and χ(w), respectively, and the functions (j)

Ki (ξ) =  Lj (ξ) =

∂F (j)

∂ui

(ξ),

i = 2, . . . , s − 1,

j = 1, . . . , n − 1,

 ∂ϕ1 ϕ2 (u2 ) − ϕ1 (u1 )  − ω(ω ), (j) (u1 ) , ||ϕ2 (u2 ) − ϕ1 (u1 )|| ∂u

j = 1, . . . , n − 1,

1

 Mj (ξ) =

 ϕs (us ) − ϕs−1 (us−1 ) ∂ϕs − θ(θ ), (j) (us ) , ||ϕs (us ) − ϕs−1 (us−1 )|| ∂us

j = 1, . . . , n − 1,

(j)

Pj (ξ) =

(j) ψ (j) (v) − ϕk (uk ) ψ (j) (v) − ϕk+1 (uk+1 ) + , ||ψ(v) − ϕk (uk )|| ||ψ(v) − ϕk+1 (uk+1 )||

j = 1, . . . , n − 1,

(j)

Qj (ξ) =

(j) χ(j) (w) − ϕm (um ) χ(j) (w) − ϕm+1 (um+1 ) + , ||χ(w) − ϕm (um )|| ||χ(w) − ϕm+1 (um+1 )||

j ∈ B,

R(ξ) = ϕk+1 (uk+1 ) − ϕk (uk ), ν1 (v), T (ξ) = ϕm+1 (um+1 ) − ϕm (um ), ν2 (w). Consider the map G : U −→ (Rn−1 )s−2 × Rn−1 × Rn−1 × Rn−1 × Rn−1 × R × R

INVERSE SCATTERING BY OBSTACLES

391

defined by (j)

G(ξ) = ( (Ki (ξ))1≤j≤n−1 2≤i≤s−1 ; (Lj (ξ))1≤j≤n−1 ; (Mj (ξ))1≤j≤n−1 ; (Pj (ξ))1≤j≤n−1 ; (Qj (ξ))j∈B ; R(ξ); T (ξ) ). Then G is smooth and we have Φ−1 (D ∩ M (s, k, m)) = G−1 (0). We will show that ˆ Then shrinking U (and therefore D), G is a submersion on the G is submersion at ξ. −1 whole U , so G (0) is a smooth submanifold of U with dim(G−1 (0)) = (s + 4)(n − 1) − [(s + 2)(n − 1) + 2] = 2n − 4 . Assume that s−1  n−1 

(j)

(j)

ˆ + Ai ∇Ki (ξ)

i=2 j=1

+

n−1 

n−1 

ˆ + Bj ∇Lj (ξ)

j=1

ˆ + pj ∇Pj (ξ)

j=1



n−1 

ˆ Cj ∇Mj (ξ)

j=1

ˆ + r∇R(ξ) ˆ + t∇T (ξ) ˆ =0 qj ∇Qj (ξ)

(13.22)

j∈B (j)

for some real numbers Ai , Bj , Cj , pj , qj , r, t. We will show that all these are zero. Here ∇ means ∇ξ , the gradient with respect to ξ = (ω  ; u; v; w; θ ). First, exactly as in the proof of Lemma 11.5.1, considering the derivatives with respect to ω  and θ in (13.22), one shows that B1 = · · · = Bn−1 = C1 = · · · = Cn−1 = 0. Then, repeating again the corresponding argument from the proof of Lemma 11.5.1, (j) it follows that Ai = 0 for all j = 1, . . . , n − 1, 1 ≤ i ≤ k and m ≤ i ≤ s. Now (13.22) gets the form m n−1   i=k+1 j=1

(j)

(j)

ˆ + Ai ∇Ki (ξ)

n−1  j=1

ˆ + pj ∇ Pj (ξ)



ˆ qj ∇ Qj (ξ)

j∈B

ˆ + t∇ T (ξ) ˆ = 0. + r ∇ R(ξ) Set for convenience b1 =

1 , ||ψ(ˆ v ) − ϕk (ˆ uk )|| b = b 1 + b2 ,

b2 = e=

1 , ||ψ(ˆ v ) − ϕk+1 (ˆ uk+1 )||

ˆk x ˆk+1 − x . ||ˆ xk+1 − x ˆk ||

Since ξˆ ∈ M (s, k, m), we have ψ(ˆ v ) ∈ [ˆ xk , x ˆk+1 ]. This and (13.21) imply ψ(ˆ v ) − ϕk+1 (ˆ uk+1 ) ψ(ˆ v ) − ϕk (ˆ uk ) =− = e = (0, . . . , 0, 1) . ||ψ(ˆ v ) − ϕk (ˆ uk )|| ||ψ(ˆ v ) − ϕk+1 (ˆ uk+1 )||

(13.23)

392

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Setting pn = 0 and p = (p1 , . . . , pn ) ∈ Rn , as in the proof of Lemma 11.5.1, one derives from (13.23) that v) = 0 . b1 p − e, b1 pe + rν1 (ˆ On the other hand, e = (0, . . . , 0, 1) and the definition of p give p⊥e, so v ) = 0. That is, b1 p + rν1 (ˆ r p = − ν1 (ˆ v ). (13.24) b1 Next, we have ∂R (ξ) = ∂vi



 ∂ν1 ϕk+1 (uk+1 ) − ϕk (uk ), (v) , ∂vi

and 

 ∂ψ ψ(v) − ϕk (u), (v) ∂vi   (j) ψ (j) (v) − ϕk+1 (u) ∂ψ ∂ψ (j) +b2 ψ(v) − ϕk+1 (u), (v) − (v) ∂vi ||ψ(v) − ϕk+1 (u)||3 ∂vi     ∂ψ (j) ∂ψ ∂ψ (j) (j) =b e, e, (v) − b1 e (v) − b2 e (v) ∂vi ∂vi ∂vi   ∂ψ ∂ψ (j) (j) e, (v) − be (v) . =b ∂vi ∂vi

(j) ∂Pj ψ (j) (v) − ϕk (u) ∂ψ (j) (ξ) = b1 (v) − ∂vi ∂vi ||ψ(v) − ϕk (u)||3

Considering the derivatives with respect to vi in (13.23), we get     ∂ψ (j) ∂ψ ∂ν1 (j) e, bpj (ˆ v) − e (ˆ v) + r x ˆk+1 − x ˆk , (ˆ v) 0= ∂vi ∂vi ∂vi j=1       ∂ψ ∂ψ ∂ν = b p, (ˆ v ) − bp , e e , (ˆ v ) + ra e , 1 (ˆ v) ∂vi ∂vi ∂vi n 



∂ψ v ) ⊥ ∂v (ˆ v ) for all i = 1, . . . , n − 1 and (13.24), this implies 0 = Using e⊥ p, ν1 (ˆ i  ∂ν1 v ) for all i = 1, . . . , n − 1. Since the Gauss curvature of X at ψ(ˆ v ) is e , ∂vi (ˆ non-zero and a = 0, it follows that r = 0. Now (13.24) implies p = 0, so (13.23) takes the form n−1 m   i=k+1 j=1

(j)

(j)

ˆ + Ai ∇Ki (ξ)



ˆ + t∇ T (ξ) ˆ =0. qj ∇ Qj (ξ)

(13.25)

j∈B (j)

Then, applying again an argument from the proof of Lemma 11.5.1, one gets Ai = 0 ˆ  for all i and j, qj = 0 for all j ∈ B and t = 0. Thus, G is a submersion at ξ.

INVERSE SCATTERING BY OBSTACLES

393

We will also need the following lemma. Lemma 13.8.3: Let X and Z be smooth local (n − 1)-dimensional submanifolds of Rn (n ≥ 2) with X ∩ Z = ∅, and let (x0 , ξ0 ) ∈ S ∗ (X) be such that (z0 , ξ0 ) ∈ S ∗ (Z), where z0 = x0 + t0 ξ0 for some t0 > 0. If the curvature of X in the direction of ξ0 is non-zero, then there exists (x, ξ) ∈ S ∗ (X) arbitrarily close to (x0 , ξ0 ) such that (x + tξ, ξ) ∈ / S ∗ (Z) for any t > 0. Proof of Lemma 13.8.3: It follows from Lemma 13.8.1 that there exist  > 0, an open neighbourhood V of x0 in X, a smooth positive function t(x), x ∈ V , with t(x) ∈ [δ, ] for all x ∈ V , and a smooth map V  x → ξ(x) ∈ Tx (X) such that Y = {y(x) = x + t(x)ξ(x) : x ∈ V } is a smooth strictly convex surface with unit normal field ξ(x). Given z  ∈ Z with z  = x + tξ(x) for some t > 0 and a sufficiently small open neighbourhood W of z  in Rn , the orthogonal projection ϕ : W −→ Y is well defined and smooth. Thus, ϕ : W ∩ Z −→ Y is a smooth map. If z ∈ W ∩ Z is such that the ray {y(x) + tξ(x) : t > 0} is tangent to Z at z, then z is a critical point and x is a critical value of ϕ. By Sard’s theorem (see e.g. [Hir]), the set of critical values of ϕ has Lebesgue measure zero in Y . Covering Z by a finite or countable family of neighbourhoods W , one shows that the set of those y(x) ∈ Y such that  {y(x) + tξ(x) : t > 0} is tangent to Z has measure zero in Y . Proof of Proposition 13.4.2: Let σ = (y, η) ∈ S ∗ (∂K) be such that γK (σ) is a scattering ray in ΩK . Since the curvature of ∂K does not vanish of infinite order, γK (σ) contains only finitely many gliding segments (if any) and finitely many diffractive tangent points to ∂K [MS1]. Using Proposition 13.5.1 and perturbing slightly σ in S ∗ (∂K) if necessary, we may assume that γK (σ) does not contain gliding segments on ∂K. Then γK (σ) has only finitely many tangent points to ∂K, all of them being diffractive tangent points. We will assume that γK has  ≥ 2 different tangent points to ∂K; otherwise there is nothing to prove. One of these is y; denote one of the others (if there are more than two) by z. The case when γK (σ) has no transversal reflections follows immediately from Lemma 13.8.3. Assume that γK (σ) has s ≥ 1 transversal reflection points x1 , . . . , xs at ∂K. As we have done before, denote by x0 (resp. xs+1 ) the orthogonal projection of x1 on the hyperplane Zω (resp. of xs on Z−θ ). We claim that there exists σ  ∈ S ∗ (∂K) arbitrarily close to σ such that γK (σ  ) has at most  − 1 tangent points to ∂K. To prove this, consider some small open neighbourhood Ui of xi (i = 1, . . . , s), V of y and W of z in ∂K such that V ∩ Ui = W ∩ Ui = ∅ for all i. We take these so small that X = V ∪ W ∪ (∪si=1 Ui ) does not contain any other tangent points of γK (σ) to ∂K. We will now apply Lemma 13.8.2 to the smooth submanifold X of Rn .

394

GENERALIZED GEODESIC FLOW AND INVERSE SPECTRAL PROBLEMS

Let k, m = 0, 1, . . . , s be such that y and z belong to the segments [xk , xk+1 ] and [xm , xm+1 ], respectively. We may assume that k ≤ m; otherwise, we can replace σ by (y, −η). If k = m, then the claim stated above follows immediately from Lemma 13.8.3. Assume k < m. Then, setting x = (x1 , . . . , xs ) and denoting by ω and θ the incoming and outgoing directions of γK (σ), we have (ω; x, y, z; θ) ∈ M (s, k, m) ⊂ Ms+2 . The natural projection p : Ms+2 −→ Ms+1 defined by ˜ = (˜ ˜ p(˜ ω; x ˜; y˜; z˜; θ) ω; x ˜; y˜; θ) determines a smooth map p : M (s, k, m) −→ M (s, k). It follows from Lemmas 11.5.1 and 13.8.2 that dim(M (s, k)) = 2n − 3 > 2n − 4 = dim(M (s, k, m)). Now Sard’s theorem gives that M (s, k) \ p(M (s, k, m)) contains points (ω  ; x ; y  ; θ ) arbitrarily close to (ω; x, y, θ). Setting η =

xk+1 − xk , ||xk+1 − xk ||

we get points σ  = (y  , η  ) ∈ S ∗ (∂K) arbitrarily close to σ such that ∂K(σ  ) does not have a tangency to ∂K in W , that is γK (σ  ) has at most  − 1 tangent points to ∂K. Proceeding by induction, one finds σ  ∈ S ∗ (∂K) arbitrarily close to σ such that  γK (σ  ) has only one tangency to ∂K.

13.9

Notes

The inverse scattering problem discussed in this chapter resembles the problem concerning the so-called lens equivalence of geodesic flows on Riemannian manifolds [Cr]. Various other problems related to recovering information from the length spectrum of certain types of geodesics have been considered in Riemannian geometry – see for example [SU], [SUV], [CULV] for general information and many references. The main results in this chapter were proved in [S8] and [S7]. More precisely, the main Theorem 13.1.2 and its consequences in Sections 13.1 and 13.3 were established in [S8]. The main result Proposition 13.4.3 of Section 13.4 was also obtained in [S8], however the main tool for it, Proposition 13.4.2, was proved in [S7]. The exposition of Section 13.5 also follows [S7]. The results of Sections 13.6 and 13.7 were established in [S8]. Section 13.8 devoted to the proof of Proposition 13.4.2 follows [S7]. There are some very recent results (not covered in this book) concerning problems related to recovering information about obstacles from scattering rays in their exteriors – see [NS1] and [NS2]. For example, it was proved in [NS1] that if K and L are two obstacles in Rn , n ≥ 2, such that each of them is a finite disjoint

INVERSE SCATTERING BY OBSTACLES

395

union of strictly convex domains with smooth boundaries and K and L have almost the same SLS, then K = L. For obstacles with real analytic boundaries this was established earlier in [S7]. A higher-dimensional version of Livshits’ example was constructed recently in [NS3]. Various other problems related to billiards in the exterior of certain subsets of Euclidean spaces have been considered as well – see for example [Pla] and[PlaR] for some information and references in this direction.

Topic Index accessible, 377 admissible map, 145 almost the same SLS, 354 assumption (I), 242 bicharacteristic relation, 12 billiard ball map, 26, 33, 99 billiard flow, 26 billiard map, 99 billiard semi-trajectory, 53 bumpy metric, 176 canonical relation, 83 characteristic set compressed, 9 characteristic set Σ, 5 classical bumpy metric theorem, 202 condition (H), 32 configuration, 31 cosphere bundle, 27, 99 cotangent bundle compressed, 24 counting function of eigenvalues, 63 cut-off resolvent, 346 degenerate broken ray, 205 diffractive point, 6 set, 5 Dirichlet Laplacian, 119

Dirichlet problem, 25, 119 existence of solution, 24 propagation of singularities, 25 elliptic operator, 23 energy function, 175 equivalence relation ∼, 9 Fermi coordinates, 176 first fundamental form, 266 Fourier integral distibution, 83 operator, 83 free rays, 377 fundamental solution, 58, 61 wave front set, 62 generalized bicharacteristic, 6 backward, 12 compressed, 9 forward, 12 half, 25 periodic, 14 uniquely extendible, 8 weakly non-degenerate, 341 generalized geodesic, 14 periodic, 14 generalized geodesic flow, 353 generic domain, 145 property, 145 geodesic flow, 175

Geometry of the Generalized Geodesic Flow and Inverse Spectral Problems, Second Edition. Vesselin M. Petkov and Luchezar N. Stoyanov. © 2017 John Wiley & Sons, Ltd. Published 2017 by John Wiley & Sons, Ltd.

406

TOPIC INDEX

glancing set, 5 vector field, 7 glancing ω-ray, 253 glancing point, 215 non-degenerate, 215 gliding set, 5 half-density bundle, 83 canonical, 94 Hamiltonian flow broken, 13 generalized, 13, 89 Hamiltonian vector field, 6 hyperbolic point, 6, 99 set, 5 immersion, 1 inclusion map, 85 incoming direction, 87 interpolating Hamiltonian, 217 inverse scattering problems, 125 Lagrangian manifold, 83 law of reflection, 28 length spectrum, 14 Lie derivative, 89 Livshits’ example, 352 local normal coordinates, 5 Maslov bundle, 94 Maslov factor, 96 Neumann problem, 25, 125 non-characteristic point, 23 non-symmetric map, 145 non-trapped, 353 non-trapping obstacle, 353 oscillatory integral, 16 outgoing Green function, 124

radiation condition, 124 resolvent R0 (λ), 122 solution, 123 periodic points, 27 type α, 33 phase function, 83 non-degenerate, 83 Poincar´e map closed geodesic, 176 Poincar´e map, 40 hyperbolic, 47 matrix representation, 47 spectrum, 40 Poisson relation, 71 distribution σ(t), 71, 81 scattering kernel, 137 Poisson summation formula, 115 classical, 117 primitive periodic ray, 29 principal symbol, 94 representation, 98, 103 properly supported operator, 23 reduced wave equation, 123 reflected direction, 87 reflecting ray, 29 (ω, θ), 49 non-symmetric, 29 ordinary, 29 periodic, 28, 29 symmetric, 29 reflection point proper, 33 tangential, 33 reflection points, 28, 50 Robin problem, 25 scattering amplitude, 121 estimate, 348 meromorphic continuation, 346 representation, 121, 345 scattering kernel, 120 representation, 121 scattering length spectrum (SLS), 351 scattering operator, 120, 126

TOPIC INDEX

scattering ray, 50 non-degenerate, 50 ordinary, 50 second fundamental form, 266 sectional normal curvature, 267 self-adjoint operator A, 63 singular support, 15 sojourn time, 50 source map, 2 space C ∞ maps, 3 Baire, 3 residual, 3 spectral function, 63 star-shaped obstacle, 363 strongly accessible, 381 part, 381 submersion, 1 support function, 343 symmetric map, 145 symplectic coordinates, 84, 94

target map, 1, 2 transport equation, 95 transversal, 2 transversality theorem Abraham, 4 multijet, 4 Thom, 4 trapped point, 353 trapping obstacle, 338 unit ball bundle, 98 wave front set, 15 generalized, 24 propagation, 24 wave operator, 5 wave operators W± , 120 winding number, 28, 221 Withney topology, 3

407

Symbol Index (−ΔD − λ2 )−1 , 344 (J k (X, Y ))s s-fold k-jet bundle, 3 (U(ω,θ) ), 133 (iλ)–outgoing, 123 B billiard ball map, 26 B ∗ (∂Ω), 98 C relation, 12, 68 C ∞ (X, Y ) space of all smooth maps, 3 C+ , 12 C− , 12 D(ρ, μ) pseudo-metric, 10 FB (t, x, y) kernel of EB , 83 G+ iλ , 124 H hyperbolic set, 5 1/2 I m (X, Λϕ ; ΩX ), 83 k J (X, Y ) space of k-jets, 2 Jα , 53 Js1 (X, Rn ) s-fold bundle of 1-jets, 141 Lm (X), 23 LΩ , 14 Pγ Poincar´e map of γ, 40 S ∗ (∂Ω), 27, 99 Tγ sojourn time of γ, 50 Tγ period of γ, 103 U (K) , 377 WF(K) wave front of K, 22 WF(u) wave front, 15 WFb (u), 24 Zξ , 301, 356

A(K) , 377 E(t, x, y), 64 Γk+ canonical relation, 91 Γk− canonical relation, 92 Ω0 , 356, 365 Σ characteristic set, 5 Ak , 32 β billiard map, 99 C(X), 3 cos(A1/2 t), 64 ∂K (∞) , 381 γ(t; μ± ), 82 FˆB (t, x, y), 83 K(fin) , 367 (fin) K0 , 367 K, 353 K0 , 353 λk (y, η), 89 LΩ set of periodic generalized geodesics, 14 (Ω), 252 Lm ω,θ ν(x) exterior unit normal, 86 νK , 355 O, 365 π projection, 33 PK , 366 σ(t), 64 σγ , 107  wave operator, 5 Sn−1 × Sn−1 , 351

Geometry of the Generalized Geodesic Flow and Inverse Spectral Problems, Second Edition. Vesselin M. Petkov and Luchezar N. Stoyanov. © 2017 John Wiley & Sons, Ltd. Published 2017 by John Wiley & Sons, Ltd.

410 (m)

SYMBOL INDEX

Tk , 370 Trap(n) (∂K), 355 Trap(ΩK ), 353 Tk , 368 T∗ (Q), 24 a(λ, θ, ω), 121 dΛ , 94 f  W, 2 mγ , 92 r(x) winding number, 28 r0 , 86

rk , 91 s(t, θ, ω), 120 tk (y, η), 89 EB , 82 LHq , 89 LΩ , 14, 71 MΓ,± canonical relations, 94 N canonical relation, 85 (X) IIu second fundamental form, 266 (X) Iu first fundamental form, 266

References [AbM] R. Abraham, J. Marsden: Foundations of Mechanics. New York: Benjamin, 1967. [Ab] R. Abraham: Bumpy metrics. In: Proceedings of Symposia in Pure Mathematics, Vol. XIV. Providence, RI: American Mathematical Society, 1970. [AbR] R. Abraham, J. Robbin: Transversal Mappings and Flows. New York: Benjamin, 1967. [AM] K. Anderson, R. Melrose: The propagation of singularities along gliding rays. Invent. Math. 41 (1977), 197–232. [An] D.V. Anosov: On generic properties of closed geodesics. Izv. Acad. Nauk SSSR 4 (1982), 675–703 (in Russian); English transl.: Math. USSR Izv. 21 (1983), 1–29. [BCST] P. Bálint, N. Chernov, D. Szász, I.P. Tóth: Geometry of multi-dimensional dispersing billiards. Asterisque 286 (2003), 119–150. [BGR] C. Bardos, J.C. Guillot, J. Ralston: Le relation de Poisson pour l’équation des ondes dans un ouvert non-borné. Commun. Partial Diff. Equ. 7 (1982), 905–958. [BLR] C. Bardos, G. Lebeau, J. Rauch: Scattering frequences and Gevrey 3 singularities. Invent. Math. 90 (1987), 77–114. [Ber] P.H. Berard: Spectral Geometry: Direct and Inverse Problems, Lecture Notes in Mathematics, vol. 1207. Berlin: Springer-Verlag, 1986. [Bir] G.D. Birkhoff: Dynamical Systems. New !York: AMS, Revised Eds., 1966. [BP1] J. F. Bony, V. Petkov: Resolvent estimates and local energy decay of hyperbolic equations. Ann. Univ. Ferrara, Sec. VII - Sci. Math. 52 (2) (2006), 233–246. [BP2] J. F. Bony, V. Petkov: Semiclasssical estimates of the cut-off resolvent for trapping perturbations. J. Spectral Theory 3 (2013), 399–422. [BunS] L. Bunimovich, Ya. Sinai: Markov partitions for dispersed billiards. Commun. Math. Phys. 73 (1980), 247–280. [BCS] L. Bunimovich, N. Chemov, Ya. Sinai: Markov partitions for two-dimensional hyperbolic billiards. Uspehi Mat. Nauk 45 (3) (1990), 97–134 (in Russian); English transl. Russ. Math. Surv. 45 (1990), 105–152. [Bu1] N. Burq: Controle de l’équation des plaques en présence d’obstacles strictement convexes. Mém. Soc. Math. France (N.S.) 55 (1993), 126pp. [Bu2] N. Burq: Décroissance de l’énergie locale de l’équation des ondes pour le problème extérieur et absence de résonance au voisinage du réel. Acta Math. 180 (1) (1998), 1–29. Geometry of the Generalized Geodesic Flow and Inverse Spectral Problems, Second Edition. Vesselin M. Petkov and Luchezar N. Stoyanov. © 2017 John Wiley & Sons, Ltd. Published 2017 by John Wiley & Sons, Ltd.

REFERENCES

397

[Bu3] N. Burq: Lower bounds for shape resonances widths of long range Schrödinger operators. Amer. J. Math. 124 (2002), 677–735. [CPS] F. Cardoso, V. Petkov, L. Stoyanov: Singularities of the scattering kernel for generic obstacles. Ann. Inst. Henri Poincaré, (Phys. Théor.) 53 (4) (1990), 445–466. [CV] F. Cardoso, G. Vodev: Uniformes estimates of the resolvent of Beltrami-Laplace operator on infinite volume Riemannian manifolds. Ann. Henri Poincaré 3 (2002), 673–691. [Chl] J. Chazarain: Construction de la paramétrix du problème mixte hyperbolique pour l’équation des ondes. C.R. Acad. Sci. Paris, Sér. A 276 (1973), 1213–1215. [Ch2] J. Chazarain: Construction de la paramétrix du problème mixte hyperbolique pour l’équation des ondes à l’intérieur d’un domaine convexe pour les bicaractéristiques. (French) Journées Équations aux Dérivées Partielles de Rennes (1975), 165–181. Asterisque, No. 34–35, Soc. Math. France, Paris, 1976. [Ch3] J. Chazarain: Formule de Poisson pour les variétés riemanniennes. Invent. Math. 24 (1) (1974), 65–82. [Cher1] N. Chernov: Construction of transversal foliations for multidimensional semi-scattering billiards. Funct. Anal. Appl. 16 (1982), 33–46 (in Russian); English transl. 16 (1983), 270–280. [Cher2] N. Chernov: Topological entropy and periodic points of two-dimensional hyperbolic billiards. Funct. Anal. Appl. 25 (1991), 50–57 (in Russian); English transl. 25 (1991), 39–45. [ChD] N. Chernov, D. Dolgopyat: Brownian Brownian motion - I. Mem. Amer. Math. Soc. 198 (927) (2009), viii + 193 pp. [ChM] N. Chernov, R. Markarian: Chaotic Billiards, Mathematical Surveys and Monographs, vol. 127. Providence, RI: American Mathematical Society, 2006. [Cl] Y. Colin de Verdière: Sur les longueurs des trajectoires périodiques d’ un billiard. In: South Rhone seminar on geometry, III. Travaux en cours, Paris: Hermann, 1984, 122–139. [C2] Y. Colin de Verdière: Spectre du laplacien et longueurs des géodésiques périodiques II. Compos. Math. 27 (1973), 159–184. [Con] G. Contreras: Generic dynamics of geodesic flows. Proceedings of the International Congress of Mathematics, Hyderabad, India, 2010. [CFS] I.P. Cornfeld, S.V. Fomin, Ya.G. Sinai: Ergodic Theory. Berlin: Springer-Verlag, 1982. [Cr] C. Croke: Rigidity and the distance between boundary points. J. Differ. Geom. 36 (1991) 445–464. [CULV] Ch. Croke, G. Uhlmann, I. Lasiecka, M. Vogelius: Geometric Methods in Inverse Problems and PDE Control. Springer Science & Business Media, 2012. [DG] J.J. Duistermaat, V. Guillemin: The spectrum of positive elliptic operators and periodic geodesics. Invent. Math. 29 (1975), 39–79. [E] G. Edgar: Measure, Topology, and Fractal Geometry. New York: Springer-Verlag, 1990. [F] V. V. Fedorchuk: The fundamentals of dimension theory. In: General Topology I, Encyclopedia of Mathematical Sciences, vol. 17, A.V. Arhagel’skii, L.S. Pontryagin (Eds). Springer-Verlag, 1990. [Ger] C. Gérard: Asymptotique des poles de la matrice de scattering pour deux obstacles strictement convexes. Mém. Soc. Math. France (N.S.) 31(1988), 146 pp.

398

REFERENCES

[GS] V. Giuillemin, S. Sternberg: Semi-Classical Analysis. Boston, MA: International Press, 2013. [GG] M. Golubitskii, V. Guillemin: Stable Mappings and their Singularities. Berlin: Springer-Verlag, 1973. [GKM] D. Gromoll, W. Klingenberg, W. Meyer: Riemanische Geometrie im Grossen, Lecture Notes in Mathematics, vol. 55. Berlin: Springer-Verlag, 1975. [G1] V. Guillemin: Sojourn time and asymptotic properties of the scattering matrix. Publ. RIMS Kyoto Univ. 12 (1977), 69–88. [GM1] V. Guillemin, R. Melrose: The Poisson sumation formula for manifolds with boundary. Adv. Math. 32 (1979), 204–232. [GM2] V. Guillemin, R. Melrose: An inverse spectral result for elliptic regions in R2 . Adv. Math. 32 (1979), 128–148. [GM3] V. Guillemin, R. Melrose: A cohomological invariant of discrete dynamical systems. In: Christoffel Centennial Volume, P.I. Putzer, F. Feher (Eds). Basel: Birkhauser, 1981, 672–679. [Gut] E. Gutkin: Billiard dynamics: a survey with the emphasis on open problems. Regul. Chaotic Dyn. 8 (2003), 1–13. [HeZ] H. Hezari, S. Zelditch, C ∞ spectral rigidity of the ellipse. Anal. & PDE 5 (2012), 1105–1132. [Hir] M. Hirsch: Differential Topology. Berlin: Springer-Verlag, 1976. [Hl] L. Hörmander: The Analysis of Linear Partial Differential Operators, Vol. I, 2nd Edn. Berlin: Springer-Verlag, 1983. [H2] L. Hörmander: The Analysis of Linear Partial Differential Operators, Vol. II. Berlin: Springer-Verlag, 1983. [H3] L. Hörmander: The Analysis of Linear Partial Differential Operators: PseudoDifferential Operators, Vol. III. Berlin: Springer-Verlag, 1985. [H4] L. Hörmander: The Analysis of Linear Partial Differential Operators, Vol. IV. Berlin: Springer-Verlag, 1985. [HW] W. Hurewicz, H. Wallman: Dimension Theory. Princeton, NJ: Princeton University Press, 1948. [I3] M. Ikawa: Precise information on the poles of the scattering matrix for two strictly convex obstacles. J. Math. Kyoto Univ. 27 (1987), 69–102. [I4] M. Ikawa: Decay of solutions of the wave equation in the exterior of several strictly convex bodies. Ann. Inst. Fourier (Grenoble) 38 (1988), 113–146. [16] M. Ikawa: Singular perturbation of symbolic flows and poles of the zeta functions. Osaka J. Math. 27 (1990), 281–300. [I7] M. Ikawa: Singular perturbations of symbolic flows and poles of the zeta functions. Addendum, Osaka J. Math. 29 (1992), 161–174. [Ivl] V.Ia. Ivrii: Second term of the spectral asymptotic expansion of the Laplace-Beltrami operator on manifolds with boundary. Funct. Anal. Appl. 14 (2) (1980), 25–34 (in Russian); English transl: 14 (1980), 98–106. [Iv2] V.Ia. Ivrii: Sharp spectral asymptotics for the Laplace-Beltrami operator under geneeral elliptic boundary conditions. Funct. Anal. Appl. 15 (1981), 74–75.

REFERENCES

399

[Kac] M. Kac: Can one hear the shape of a drum? Amer. Math. Soc. Mon. 73 (1966), 1–23. [Katl] A. Katok: Lyapunov exponents, entropy and periodic orbits for diffeomorphisms. Publ. Math. lHES 51 (1980), 137–173. [Kat2] A. Katok: Entropy and closed geodesics. Ergod. Theor. Dyn. Syst. 2 (1982), 339–367. [Kat3] A. Katok: The growth rate of the number of singular and periodic orbits for a polygonal billiard. Commun. Math. Phys. 111 (1987), 151–160. [KS] A. Katok, J.M. Strelcyn, collaboration with F. Ledrappier, F. Przytycki: Smooth Maps with Singularities: Invariant Manifolds, Entropy and Billiards, Lecture Notes in Mathematics, Vol. 1222. Berlin: Springer-Verlag, 1986. [K2] W. Klingenberg: Lectures in Closed Geodesics. Berlin: Springer-Verlag, 1978. [Kl] W. Klingenberg: Riemannian Geometry. Berlin: Walter de Gruyter, 1982. [KT] W. Klingenberg, F. Takens: Generic properties of geodesic flows. Math. Ann. 197 (1972), 323–334. [Ko] V. Kovachev: Smoothness of the billiard ball map for strictly convex domains near the boundary. Proc. Amer. Math. Soc. 103 (1988), 856–860. [KP] V. Kovachev, G. Popov: Invariant tori for the billiard ball map. Trans. Amer. Math. Soc. 317 (1990), 45–81. [KozT] V. Kozlov, D. Treshchev: Billiards: A Genetic Introduction to the Dynamics of Systems with Impact, Translations of Mathematics Monographs, Vol. 89. Providence, RI: American Mathematical Society, 1991. [KSS] A. Kràmli, N. Simànyi, D. Szàsz: Dispersing billiards without focal points on surfaces are ergodic. Commun. Math. Phys. 125 (1989), 439–457. [Lan] E.E. Landis: Tangential singularities. Funct. Anal. Appl. 15 (1981), 36–49 (in Russian); English transl.: 15 (1981), 103–114. [L2] V.F. Lazutkin: The existence of caustics for the billiard problem in a convex domain. Izv. Akad. Nauk SSSR, Ser. Mat. 37 (1973), 186–216 (in Russian); English transl.: Math. USSR Izv. 7 (1974), 185–214. [Ll] V.F. Lazutkin: Convex Billiard and Eigenfunctions of the Laplace Operator. Leningrad: Leningrad University, 1981 (in Russian). [Lang] S. Lang: Differential Manifolds. Reading, MA: Addison-Wesley, 1972. [LP1] P. Lax, R. Phillips: Scattering Theory, 2nd Edn. New York: Academic Press, 1989. [LP2] P. Lax, R. Phillips: Scattering theory for the acoustic equation in an even number of space dimensions. Indiana Univ. Math. J. 22 (1972), 101–134. [MSi] H.P. McKean, I.M. Singer: Curvature and the eigenvalues of the Laplacian. J. Differ. Geom. 1 (1967), 43–69. [Mag] A. Magnuson: Symplectic singularities, periodic orbits of the billiard ball map and the obstacle problem. Thesis, Massachusetts Institute of Technology, 1984. [Mal] A. Majda: High frequency asymptotics for the scattering matrix and the inverse problem of acoustical scattering. Commun. Pure Appl. Math. 29 (1976), 261–291. [Ma2] A. Majda: A representation formula for the scattering operator and the inverse problem for arbitrary bodies. Commun. Pure Appl. Math. 30 (1977), 165–194. [MaR] A. Majda, J. Ralston: An analogue of Weyl’s formula for unbounded domains. Duke Math. J. 45 (1978),183–196.

400

REFERENCES

[MaT] A. Majda, M. Taylor: The asymptotic behaviour of the diffraction peak in classical scattering. Commun. Pure Appl. Math. 30 (1977), 639–669. [Marg] G. Margulis: Applications of ergodic theory to the investigation of manifolds of negative curvature. Funct Anal. Appl. 3 (1969), 335–336 (in Russian). [MM] S. Marvizi, R. Melrose: Spectral invariants of convex planar regions. J. Differ. Geom. 17 (1982), 475–502. [Me1] R. Melrose: Equivalence of glancing hypersurfaces. Invent. Math. 37 (1975), 165–192. [Me2] R. Melrose: Singularities and energy decay of acoustical scattering. Duke Math. J. 46 (1979),43–59. [Me3] R. Melrose: Forward scattering by a convex obstacle. Commun. Pure Appl. Math. 33 (1980), 461–499. [Me4] R. Melrose: Geometric Scattering Theory. Cambridge University Press, 1994. [MS1] R. Melrose, J. Sjöstrand: Singularities in boundary value problems. I. Commun. Pure Appl. Math. 31 (1978), 593–617. [MS2] R. Melrose, J. Sjöstrand: Singularities in boundary value problems. II. Commun. Pure Appl. Math. 35 (1982), 129–168. [MeyP] K. Meyer, J. Palmore: A generic phenomenon in conservative. Proceedings of Symposia in Pure Mathematics. Providence, RI: American Mathematical Society, 1970, 185–189. [Mor] T. Morita: The symbolic representation of billiards without boundary conditions. Trans. Amer. Math. Soc. 325 (2) (1991), 819–828. [Nal] S. Nakamura: Singularities of the scattering kernel for two convex obstacles. Publ. RIMS Kyoto Univ. 25 (1989), 223–238. [Na2] S. Nakamura: Singularities of the scattering kernel for several convex obstacles. Hokkaido Math. J. 18 (1989), 487–496. [NS] S. Nakamura, H. Soga: Singularities of the scattering kernel for two balls. J. Math. Soc. Japan 40 (1988), 205–220. [NS1] L. Noakes, L. Stoyanov: Rigidity of scattering lengths and traveling times for disjoint unions of convex bodies. Proc. Amer. Math. Soc. 143 (2015), 3879–3893. [NS2] L. Noakes, L. Stoyanov: Traveling times in scattering by obstacles. J. Math. Anal. Appl. 430 (2015), 703–717. [NS3] L. Noakes, L. Stoyanov: Obstacles with non-trivial trapping sets in higher dimensions. Arch. Math. 107 (2016), 73–80. [Pl] V. Petkov: High frequency asymptotics of the scattering amplitude for non-convex bodies. Commun. Partial Differ. Equ. 5 (1980), 293–329. [P3] V. Petkov: Singularities of the scattering kernel. Nonlinear partial differential equations and their applications. In: College de France Seminar VI, 1984, 288–296. [P5] V. Petkov: Scattering Theory for Hyperbolic Operators. Amsterdam: North-Holland, 1989. [P6] V. Petkov: Analytic singularities of the dynamical zeta function. Nonlinearity, 12 (1999), 1663–1681. [PP] V. Petkov, G. Popov: Asymptotic behavior of the scattering phase for non-trapping obstacles. Ann. Inst. Fourier (Grenoble) 32 (1982), 114–149. [PSl] V. Petkov, L. Stoyanov: Periodic geodesics of generic nonconvex domains in R2 and the Poisson relation. Bull. Amer. Math. Soc. 15 (1986), 88–90.

REFERENCES

401

[PS2] V. Petkov, L. Stoyanov: Periods of multiple reflecting geodesics and inverse spectral results. Amer. J. Math. 109 (1987), 619–668. [PS3] V. Petkov, L. Stoyanov: Spectrum of the Poincare map for periodic reflecting rays in generic domains. Math. Z. 194 (1987), 505–518. [PS4] V. Petkov, L. Stoyanov: On the number of periodic reflecting rays in generic domains. Ergod. Theor. Dynam. Syst. 8 (1988), 81–91. [PS5] V. Petkov, L. Stoyanov: Singularities of the scattering kernel and scattering invariants for several strictly convex obstacles. Trans. Amer. Math. Soc. 312 (1989), 203–235. [PS6] V. Petkov, L. Stoyanov: Singularities of the scattering kernel for a class of star-shaped non-convex obstacles. Math. Appl. Comput. 8 (1989), 167–176. [PS7] V. Petkov, L. Stoyanov: Geometry of Reflecting Rays and Inverse Spectral Problems. Chichester: John Wiley & Sons, Ltd, 1992. [PS8] V. Petkov, L. Stoyanov: Sojourn times of trapping rays and the behaviour of the modified resolvent of the Laplacian. Ann. Inst. Henri Poincaré (Phys. Théor.) 62 (1995), 17–45. [PS9] V. Petkov, L. Stoyanov: Singularities of the scattering kernel for trapping obstacles. Ann. Sci. Ec. Norm. Super. 29 (1996), 737–756. [PS10] V. Petkov, L. Stoyanov: Singularities of the scattering kernel related to trapping rays. Progress in Nonlinear Differential Equations and their Applications. In: Advances in Space Phase Analysis of Partial Differential Equations, vol. 78. Boston, MA: Birkhauser, 2009, 235–251. [PV] V. Petkov, P. Vogel: La représentation de l’application de Poincaré correspondant aux rayons périodiques réflechissants. C. R. Acad. Sci. Paris, Sér. A 296 (1983), 633–635. [PZ] V. Petkov, M. Zworski: Semi-classical estimates on the scattering determinant. Ann. Henri Poincaré 2 (2001), 675–711. [Pla] A. Plakhov: Exterior Billiards: Systems with Impacts Outside Bounded Domains. New York: Springer-Verlag, 2012. [PlaR] A. Plakhov, V. Roshchina: Invisibility in billiards. Nonlinearity 24 (2011), 847–854. [Pon] L. Pontriagin: Ordinary Differential Equations. Reading, MA: Addison-Wesley, 1962. [Pol] G. Popov: Quasimodes for the Laplace operator and glancing hypersurfaces. In: Proceedings of the Conference on Microlocal Analysis and Nonlinear Waves, M. Beals, R. Melrose, J. Rauch (Eds). Berlin: Springer-Verlag, 1990, 167–178. [Po2] G. Popov: Length spectrum invariants of Riemannian manifolds. Math. Z. 213 (1990), 311–351. [Po3] G. Popov: Quasi-modes for the Laplace operator and Glancing hypersurfaces. In: Proceedings of Conference on Microlocal Analysis and Nonlinear Waves, Minnesota 1989, M. Beals, R. Melrose, J. Rauch (Eds). Berlin: Springer-Verlag; 1991. [PoT] G. Popov, P. Toplalov: Invariants of isospectral deformations and spectral rigidity. Commun. Partial Differ. Equ. 37 (2012), 369–466. [Prot] M.H. Protter: Can one hear the shape of a drum? Revisited. SIAM Rev. 29 (2) (1987), 185–197. [Ra] H.-B. Rademacher: On a generic property of geodesic flows. Math. Ann. 298 (1994), 101–116. [R] R.C. Robinson: Generic properties of conservative systems. I, II. Amer. J. Math. 92 (1970), 562–603 and 897–906.

402

REFERENCES

[SaV] Y. Safarov, D. Vassiliev: The Asymptotic Distribution of Eigenvalues of Partial Differential Operators, Translations of Mathematical Monographs, Vol. 155. Providence, RI: American Mathematical Society, 1997. [Se] R. Seeley: A sharp asymptotic remainder estimate for the eigenvalues of the Laplacian in a domain in R3 . Adv. Math. 29 (1978), 244–269. [Sim] N. Simányi: Conditional Proof of the Boltzmann-Sinai Ergodic Hypothesis. Invent. Math. 177 (2009), 381–413. [Sinl] Ya. Sinai: Dynamical systems with elastic reflections. Ergodic properties of dispersing billiards. Uspehi Mat. Nauk 25 (1970), 141–192 (in Russian); English transl.: Russ. Math. Surveys 25 (1970), 137–189. [Sin2] Ya. Sinai: Development of Krylov ideas. An addendum to the book: N. S. Krylov “Works on the Foundations of Statistical Physics”. Princeton, NJ: Princeton University Press, 1979. [Sjo] J. Sjostrand: Geometric bounds on the density of resonances for semiclassical problems. Duke Math. J. 60 (1990), 1–57. [So2] H. Soga: Singularities of the scattering kernel for convex obstacles. J. Math. Kyoto Univ. 22 (1983), 729–765. [SU] P. Stefanov, G. Uhlmann: Boundary rigidity and stability for generics simple metrics. J. Amer. Math. Soc. 18 (2005), 975–1003. [SUV] P. Stefanov, G. Uhlmann, A. Vasy: Boundary rigidity with partial data. J. Amer. Math. Soc. 29 (2) (2015), 299–332. [SV] P. Stefanov, G. Vodev: Neumann resonances in linear elasticity for an arbitrary body. Commun. Math. Phys. 176 (1996), 645–659. [S1] L. Stoyanov: Generic properties of periodic reflecting rays. Ergod. Theor. Dynam. Syst. 7 (1987), 597–609. [S2] L. Stoyanov: An estimate from above of the number of periodic orbits for semi-dispersed billiards. Commun. Math. Phys. 124 (1989), 217–227. [S3] L. Stoyanov: A bumpy metric theorem and the Poisson relation for generic strictly convex domains. Math. Ann. 287 (1990), 675–696. [S4] L. Stoyanov: Nonexistence of generalized scattering rays and singularities of the scattering kernel for generic domains in R3 . Proc. Amer. Math. Soc. 113 (1991), 847–856. [S5] L. Stoyanov: Exponential instability for a class of dispersing billiards. Ergod. Theor. Dynam. Syst. 19 (1999), 210–226. [S6] L. Stoyanov: Generalized Hamilton flow and Poisson relation for the scattering kernel. Ann. Sci. Ecole Norm. Sup., 33 (2000), 361–382. [S7] L. Stoyanov: On the scattering length spectrum for real analytic obstacles. J. Funct. Anal., 177 (2000), 459–488. [S8] L. Stoyanov: Rigidity of the scattering length spectrum. Math. Ann. 324 (2002), 743–771. [S9] L. Stoyanov: A sharp asymptotic for billiard traveling times in the exterior of two convex domains. Asymptotic Anal. 35 (2003), 235–255. [S10] L. Stoyanov: Scattering resonances for several small convex bodies and the Lax-Phillips conjecture. Mem. Amer. Math. Soc. 199 (933) (2009), vi+76 pp. [ST] L. Stoyanov, F. Takens: Generic properties of closed geodesics on smooth hypersurfaces. Math. Ann. 296 (1993), 385–402.

REFERENCES

403

[Tab] S. Tabachnikov: Billiards, Société Mathématique de France, Paris, 1995. [T] F. Takens: Hamiltonian systems: Generic properties of closed orbits and local perturbations. Math. Ann. 188 (1970), 304–312. [Tay] M. Taylor: Grazing rays and reflections of singularities of solutions to wave equations. Commun. Pure Appl. Math. 29 (1976), 1–38. [Va] B.R. Vainberg: Asymptotic Methods in Equations of Mathematical Physics. New York: Gordon & Breach Science Publishers, 1989. [Vi] M.F. Vigneras: Variétés riemanniennes isospectrales et non isométriques. Ann. Math. 91 (1980), 21–32. [Wa] P. Walters: An Introduction to Ergodic Theory. Berlin: Springer-Verlag, 1981. [W] H. Weyl: Uber die Asymptotische Verteilung der Eigenwerte. Nachr. K. Ges. Wiss. Göttingen (1911), 110–117. [Y] K. Yamamoto: Characterization of a convex obstacle by singularities of the scattering kernel. J. Differ. Equ. 64 (1986), 283–293. [Z] S. Zelditch: The inverse spectral problem. Surveys in Differential Geometry, Vol. IX, 401–467. Somerville, MA: International Press, 2004. [Z1] M. Zworski: Singulat part of the scattering matrix determine the obstacle. Osaka J. Math. 38 (1) (2001), 13–20. [Z2] M. Zworski. Quantum resonances and partial differential equations. ICM 2002, Vol. III, 1–10.