STABILITY OF THE CONJUGATE INDEX, DEGENERATE CONJUGATE POINTS AND THE MASLOV INDEX IN SEMI-RIEMANNIAN GEOMETRY

FRANCESCO MERCURI, PAOLO PICCIONE, AND DANIEL V. TAUSK

Abstract. We investigate the problem of the stability of the number of conjugate or focal points (counted with multiplicity) along a semi-Riemannian geodesic γ. For a Riemannian or a nonspacelike Lorentzian geodesic, such number is equal to the intersection number (Maslov index) of a continuous curve with a subvariety of codimension one of the Lagrangian Grassmannian of a symplectic space. In the general semi-Riemannian case, under a certain nondegeneracy assumption on the conjugate points, this number is equal to an algebraic count of their multiplicities. In this paper we reprove some results that were incorrectly stated by Helfer in [14], where the occurrence of degeneracies was overlooked; in particular, a counterexample to one of Helfer’s results, which is essential for the theory, is given. In the last part of the paper we discuss a general technique for the construction of examples and counterexamples in the index theory for semi-Riemannian geodesics, in which some new phenomena appear.

1. Introduction

The original motivation for writing this paper was given by the following problem. Given a sequence γ_{j} of geodesics in a semi-Riemannian manifold (M, 𝔤) converging to a geodesic γ_{∞}, what can be said about the convergence of the conjugate index of γ_{j} to that of γ_{∞}? The question arose in the context of Lorentzian geometry, where the problem originated in an attempt to develop a Morse theory for lightlike geodesics as a limit of the theory for timelike geodesics (see [11]). Recall that the conjugate index of a geodesic γ : [a, b] → M is the number of points that are conjugate to γ(a) along γ, counted with multiplicity. Conjugate points along a geodesic correspond to the zeroes of (nontrivial) Jacobi fields along γ, which are vector fields in the kernel of the second variation of the action functional z ↦ ∫_{a}^{b} 𝔤(z′, z′) dt, called the index form I_{γ}. The celebrated Morse Index Theorem (see for instance [2, 3, 6, 7, 9, 16, 17] for versions of this theorem in different contexts) states that the conjugate index of a Riemannian or nonspacelike Lorentzian geodesic γ is equal to the index of I_{γ}, provided that the final point γ(b) is not considered in the count of conjugate points. It is not too hard to prove that the convergence of a sequence of geodesics to a geodesic implies the norm convergence of the correspondent index forms, seen as bilinear forms on the Hilbert space of H^{1} variational vector fields. Hence, the problem of convergence of the conjugate index can be reformulated in terms of convergence of the index on a sequence of self-adjoint operators converging in the operator norm. It follows from the spectral theorem that the index of a self-adjoint operator is indeed stable by small perturbations in the norm topology, provided that the operator be invertible. If (M, 𝔤) is Riemannian, i.e., 𝔤 is positive definite, then the self-adjoint operator associated to the index form along the limit geodesic γ_{∞} is indeed invertible, provided that the point γ_{∞}(b) is not conjugate to γ_{∞}(a) along γ_{∞}; thus, the Morse index theorem and the stability of the index of an invertible self-adjoint operator on a Hilbert space imply that the conjugate index of γ_{j} is eventually constant, and equal to the conjugate index of γ_{∞}. The same conclusion holds for timelike Lorentzian geodesics, provided that the limit be also timelike. When one considers the case of lightlike geodesics, though, the situation is complicated by the fact that the index form always has a nontrivial kernel, even when the final endpoint is nonconjugate to the initial one. Thus, using functional-analytical arguments one only proves the semi-continuity of the index for Lorentzian nonspacelike geodesics.

Date: revised version of March 2002.

2000 Mathematics Subject Classification. 34B24, 53C22, 53C50.

The authors are partially sponsored by CNPq (Brazil).

A different technique to study the stability of the index is suggested by an analogy with the classical Sturm problem in the theory of ordinary differential equations. The Sturm oscillation theorem ([5, Chapter 8]) gives the equality between the index of a certain symmetric bilinear form B and the number of zeroes of a solution of the Sturm equation. Its proof is obtained by showing that the two quantities involved in the thesis can be obtained as the winding number of two homotopic closed curves in the real projective line. As a side effect of this theory, one obtains immediately that, since the winding number is stable by small C^{0}-perturbations, the index of B is stable by small perturbations. So, the stability of the index for a Sturm system is proven by relating the index form B to some homotopical invariants of the problem. In this paper we exploit this idea to obtain results on the stability of the conjugate index for semi-Riemannian geodesics. More precisely, we investigate the notion of Maslov index for a Jacobi type system of ordinary differential equations, which is obtained as the intersection number between a curve ℓ determined by the flow of the system and a subvariety of codimension one of the Lagrangian Grassmannian of a symplectic space. The zeroes of the solutions of the Jacobi type system correspond to the intersections of the curve ℓ with the subvariety; moreover, the Maslov index of the system arising from the Jacobi equation along a Riemannian or a nonspacelike Lorentzian geodesic is proven to be equal to the conjugate index of the geodesic. In the general semi-Riemannian case, under a suitable nondegeneracy assumption on the conjugate points, the Maslov index corresponding to a geodesic is proven to be equal to an algebraic count of the conjugate points, which we also call the conjugate index of the semi-Riemannian geodesic.

The idea of studying conjugate points in differential equations via intersection theory is not new. In [4] and [8] it is developed an approach to the index problem through topological methods. For instance, in [8], it is employed a similar technique to prove a generalization of Sturm’s Theorems in the case of an arbitrary self-adjoint system of even order subject to arbitrary self-adjoint boundary conditions. In [14], the author carries out a similar analysis for Morse–Sturm–Liouville systems which are obtained from the Jacobi equation along a semi-Riemannian geodesic. The intersection theory employed is based on the notion of Maslov index of a curve which is defined in [14] using homology/cohomology duality on the Lagrangian Grassmannian. This approach does not seem to deal properly with the lack of orientability of the Lagrangian Grassmannian (see [10]). Furthermore, several results of Helfer’s paper are incorrectly stated, due mainly to the lack of an essential assumption of nondegeneracy for the restrictions of certain symmetric bilinear forms. More precisely, in [14, Proposition 5.1 (b)] it is claimed the equality between the Maslov index and the sum of the signatures of the conjugate points along a semi-Riemannian geodesic; we give a counterexample to such equality in Subsection 5.4. For these reasons, we have opted to provide an alternative, self-contained, presentation of the definition and some basic properties of the Maslov index. It is important to observe that the kind of degeneracy discussed above, which do not occur in Riemannian or nonspacelike Lorentzian geodesics, is responsible for many of the new phenomena which arise in the general semi-Riemannian case, like for instance, the possibility of accumulation of the conjugate points along a geodesic. Curiously enough, also in the book by O’Neill [15, Exercise 8, page 299], the nondegeneracy assumption is missing, and the author claims incorrectly that the set of focal points along a geodesic is discrete.

The paper is organized as follows. In Section 2 we introduce the basic notation and we discuss some preliminary facts. Section 3 is devoted to the intersection theory used for the definition and the properties of the Maslov index. We remark that the geometry of the Lagrangian Grassmannian has already been studied by several authors (see for instance [1, 13, 18]). The techniques of computation of the Maslov index in terms of change of signatures of symmetric bilinear forms presented in Section 3 are very practical for our purposes of relating the Maslov index and the conjugate index of a geodesic (see Theorem 4.3 and the following corollary).

In Section 4 we apply the results of the previous sections to the problem of the stability of the conjugate and focal indexes in semi-Riemannian geometry. A special attention is given to the case of the approximation of a lightlike geodesic by timelike geodesics in a Lorentzian manifold. If γ_{∞} is a lightlike geodesic in a Lorentzian manifold whose endpoints are not conjugate and γ_{j} is a sequence of timelike geodesics converging to γ_{∞}, then the conjugate index of γ_{j} is eventually constant and equal to the conjugate index of γ_{∞}. We also study the problem of stability in the case of geodesics γ_{j} starting orthogonally to submanifolds P_{j}, assuming a suitable notion of convergence for the initial data. It is a curious fact that, even in the Riemannian case, stability does not hold under the most general natural notion of convergence of the initial data (see Example 4.9).

In Section 5, we study the problem of determining which curves of Lagrangians are associated to solutions of Jacobi systems. We give some necessary and sufficient conditions for this occurrence, and we use these conditions to find an example in which the equality between the Maslov index and the conjugate index of a Lorentzian spacelike geodesic γ fails, due to the degeneracy of the metric on the space 𝔍[t] consisting of the evaluations at t of the Jacobi fields along γ that vanish at the initial instant.

The authors wish to express their gratitude to Professor Daniel Baumann Henry, Professor Daciberg Lima Gonçalves and Professor Volker Perlick who have given many fruitful suggestions.

2. Preliminaries

Given vector spaces V, W, we denote by Lin(V, W) the space of linear maps from V to W; all vector spaces will be assumed to be real and finite-dimensional. We also denote by Bil(V, W) the space of bilinear forms B : V × W → IR; when V = W we write simply Lin(V) and Bil(V). We will always implicitly identify a bilinear map B : V × W → IR with the linear map from V to the dual space W^{∗} given by v ↦ B(v, ·). The space of symmetric bilinear forms B : V × V → IR will be denoted by Bil_{sym}(V); observe that B ∈ Bil(V) ≅ Lin(V, V^{∗}) is symmetric if and only if the transpose linear map B^{∗} : V^{∗∗} ≅ V → V^{∗} is equal to B. If S is a subspace of V then we denote by S^{o} ⊂ V^{∗} the annihilator of S in V. Whenever B ∈ Bil_{sym}(V) is fixed by the context, we denote by S^{⊥} = B^{−1}(S^{o}) ⊂ V the orthogonal complement of S with respect to B. The index of a symmetric bilinear form B ∈ Bil_{sym}(V), denoted by n_{−}(B), is defined as the maximum of the dimensions of the subspaces of V on which B is negative definite; the coindex of B, denoted by n_{+}(B), is defined by n_{+}(B) = n_{−}(−B) and the signature of B is defined by sgn(B) = n_{+}(B) − n_{−}(B). The kernel of B, denoted by Ker(B), is the set V^{⊥} of vectors v ∈ V such that B(v, w) = 0 for all w ∈ V; when Ker(B) = {0} we say that B is nondegenerate.

2.1. The geometrical problem

Let (M, 𝔤) be a semi-Riemannian manifold, i.e., M is a finite dimensional real differentiable manifold and 𝔤 is a smooth nondegenerate metric tensor on M. We set n = dim(M). A nonzero vector v ∈ T M is called spacelike, timelike or lightlike according to 𝔤(v, v) being positive, negative or null. The semi-Riemannian manifold (M, 𝔤) is said to be Lorentzian if the index of 𝔤 at every point of M is equal to 1. Let ∇ denote the Levi-Civita connection of 𝔤 and let R denote the curvature tensor of ∇ chosen with sign convention R(X, Y) = ∇_{X}∇_{Y} − ∇_{Y} ∇_{X} − ∇_{[X,Y]}. If V : [a, b] → T M is a vector field along a given curve γ : [a, b] → M, we will write V′ for the covariant derivative of V along γ. Let P be a smooth submanifold of M, p ∈ P and 𝔫 ∈ T_{p}P^{⊥}; by ⊥ we mean the orthogonal complement with respect to 𝔤. The second fundamental form of P at p in the normal direction 𝔫 is the symmetric bilinear form S_{𝔫} ∈ Bil_{sym}(T_{p}P) given by S_{𝔫}(v_{1}, v_{2}) = 𝔤(∇_{v_{1}}V_{2}, 𝔫), where V_{2} is any extension of v_{2} to a vector field on P. If P is nondegenerate at p, i.e., if the restriction of 𝔤 to T_{p}P is nondegenerate, then there exists a unique linear endomorphism A_{𝔫} of T_{p}P such that S_{𝔫}(v_{1}, v_{2}) = 𝔤(A_{𝔫}(v_{1}), v_{2}) for all v_{1}, v_{2} ∈ T_{p}P. Let γ : [a, b] → M be a geodesic orthogonal to P at the initial point, i.e., γ(a) ∈ P and γ′(a) ∈ T_{γ(a)}P^{⊥}. We say that γ is spacelike, timelike or lightlike according to γ′(t) being spacelike, timelike or lightlike for all t. Let’s assume that P is nondegenerate at γ(a); we will say that the family of objects (M, 𝔤, γ, P) is an admissible quadruple for the geometrical problem. A Jacobi field along γ is a smooth vector field J along γ that satisfies the Jacobi equation:

(2.1)  J′′(t) = R(γ′(t), J (t))γ′(t);

we say that J is a P-Jacobi field if, in addition, J (a) satisfies:

(2.2)  J (a) ∈ T_{γ(a)}P and J′(a) + A_{γ′(a)}(J (a)) ∈ T_{γ′(a)}P^{⊥}.

If P is a single point of M, then (2.2) reduces to J (a) = 0. Geometrically, equation (2.1) means that J is the variational vector field corresponding to a variation of γ by geodesics; condition (2.2) says that these geodesics are orthogonal to P at their initial points. We set:

𝔍 = {J : J is a P-Jacobi field along γ}, 𝔍[t] = {J (t) : J ∈ 𝔍}.

Observe that 𝔍 is an n-dimensional subspace of the space of all vector fields along γ because the pairs (J (a), J′(a)) ∈ T_{γ(a)}M ⊕ T_{γ(a)}M satisfying (2.2) form an n-dimen-sional subspace. A point γ(t), t ∈ ]a, b] is said to be P-focal along γ if there exists a nonzero J ∈ 𝔍 with J (t) = 0; the multiplicity of γ(t), denoted by mul(t), is defined to be the dimension of the space of those J ∈ 𝔍 with J (t) = 0. If γ(t) is not P-focal we set mul(t) = 0. Since dim(𝔍) = dim(T_{γ(t)}M) = n, then:

(2.3)  mul(t) = codim(𝔍[t]) = dim(𝔍[t]^{⊥}).

When P reduces to a point then the P-focal points are also called conjugate points.

Definition 2.1. The signature of a P-focal point γ(t), denoted by sgn(t), is defined to be the signature of the restriction of the metric 𝔤 to the space 𝔍[t]^{⊥}; if γ(t) is not a P-focal point, we set sgn(t) = 0. A P-focal point γ(t) is called nondegenerate if the restriction of 𝔤 to 𝔍[t] (or, equivalently, the restriction of 𝔤 to 𝔍[t]^{⊥}) is nondegenerate. The focal index of the geodesic γ relative to P is defined by i_{foc}(γ, P) = ∑_{t∈]a,b]} sgn(t), provided that the number of P-focal points along γ is finite (see Proposition 2.2 and Remark 2.3). When the submanifold P reduces to a point, the focal index is also called the conjugate index.

If 𝔤 is Riemannian or if 𝔤 is Lorentzian and γ is nonspacelike then 𝔤 is positive definite on 𝔍[t]^{⊥} for all t ∈ ]a, b], so that all P-focal points are nondegenerate and, by (2.3), the signature of a P-focal point is equal to its multiplicity.

2.2. The differential problem

We consider the following setup. Let g be a nondegenerate symmetric bilinear form in IR^{n} and let R : [a, b] → Lin(IR^{n}) be a continuous curve of linear endomorphisms of IR^{n} such that R(t) is g-symmetric for all t, i.e., g(R(t)·, ·) is a symmetric bilinear form on IR^{n}. Let P ⊂ IR^{n} be a subspace on which g is nondegenerate and let S ∈ Bil_{sym}(P) be a symmetric bilinear form on P. We will say that (g, R, P, S) is an admissible quadruple for the differential problem in IR^{n}. We consider the linear differential equation in IR^{n}:

(2.4)  J′′(t) = R(t)J(t), t ∈ [a, b];

we will consider solutions J of (2.4)that satisfy in addition the following initial conditions:

(2.5)  J(a) ∈ P, g(J′(a))|_{P} + S(J(a)) = 0 ∈ P^{∗},

where as usual we identify bilinear forms with linear maps taking values in the dual space. A solution J of (2.4) satisfying the initial conditions (2.5) will be called a (P, S)-solution. Note that, if P = {0} (and thus S = 0), a (P, S)-solution is simply a solution of (2.4) vanishing at t = a. Let 𝕁 denote the space of all (P, S)-solutions:

(2.6)  𝕁 = {J : J satisfies (2.4) and (2.5)};

for t ∈ [a, b], we set 𝕁[t] = {J(t) : J ∈ 𝕁}. Observe that dim(𝕁) = n because the pairs (J(a), J′(a)) ∈ IR^{n} ⊕ IR^{n} satisfying (2.5) form an n-dimensional subspace. We say that an instant t ∈ ]a, b] is (P, S)-focal if there exists a nonzero J ∈ 𝕁 with J(t) = 0; the dimension of the space of those J ∈ 𝕁 with J(t) = 0 is called the multiplicity of t and is denoted by mul(t). We have mul(t) = codim(𝕁[t]) = dim(𝕁[t]^{⊥}), where ⊥ denotes the orthogonal complement with respect to g. If P = {0} then the (P, S)-focal instants are also called conjugate instants. The signature of a (P, S)-focal instant t ∈ ]a, b], denoted by sgn(t), is defined to be the signature of the restriction of g to the space 𝕁[t]^{⊥}; the (P, S)- focal instant t is called nondegenerate if the restriction of g to 𝕁[t] is nondegenerate. If the number of (P, S)-focal instants is finite we also define the focal index of the quadruple (g, R, P, S) by i_{foc}(g, R, P, S) = ∑_{t∈]a,b]} sgn(t). If P = {0} then the focal index is also called the conjugate index.

Let (M, 𝔤, γ, P) be an admissible quadruple for the geometrical problem and denote by n the dimension of M. Consider a parallel referential along γ; such referential induces an isomorphism ϕ_{t} : T_{γ(t)}M → IR^{n} for all t ∈ [a, b]. Let g be the nondegenerate symmetric bilinear form on IR^{n} which equals the push-forward of 𝔤_{γ(t)} by ϕ_{t}; observe that, since 𝔤 is parallel, g is independent of t. For each t ∈ [a, b], let R(t) denote the g-symmetric linear endomorphism of IR^{n} such that ϕ_{t}^{−1} ∘ R(t) ∘ ϕ_{t} equals the linear endomorphism R(γ′(t), ·)γ′(t) of T_{γ(t)}M. Set P = ϕ_{a}(T_{γ(a)}P) and let S : P × P → IR be the pushforward of the second fundamental form S_{γ′(a)} : T_{γ(a)}P × T_{γ(a)}P → IR by ϕ_{a}. Then (g, R, P, S) is an admissible quadruple for the differential problem in IR^{n}; we say that (g, R, P, S) is obtained from (M, 𝔤, γ, P) by a parallel trivialization of the tangent bundle along γ. Obviously, the P-focal points along γ correspond to the (P, S)-focal instants of (g, R, P, S); moreover, the multiplicity, signature and degeneracy of a P-focal point along γ are equal respectively to the multiplicity, signature and degeneracy of the corresponding (P, S)-focal instant. In particular, the quadruples (M, 𝔤, γ, P) and (g, R, P, S) have the same focal indexes.

2.3. On the discreteness of the set of focal instants

We now give some sufficient conditions for the number of focal points along a semi-Riemannian geodesic to be finite. We state our main results in terms of differential problems. Let (g, R, P, S) be an admissible quadruple for the differential problem. A simple computation using the g-symmetry of R shows that:

(2.7)  g(J′_{1}(t), J_{2}(t)) − g(J_{1}(t), J′_{2}(t)) ≡ constant,

for all solutions J_{1}, J_{2} of (2.4). The initial conditions (2.5) imply that the lefthand side of (2.7) vanish at t = a; hence:

(2.8)  g(J′_{1}(t), J_{2}(t)) = g(J_{1}(t), J′_{2}(t)), t ∈ [a, b],

for all J_{1}, J_{2} ∈ 𝕁. Observe that (2.8) and a dimension counting argument show that:

(2.9)  𝕁[t]^{⊥} = {J′(t) : J ∈ 𝕁, J(t) = 0}, t ∈ [a, b].

Proposition 2.2. If (g, R, P, S) is an admissible quadruple for the differential problem then the nondegenerate (P, S)-focal instants are isolated. Moreover, there are no (P, S)- focal instants near t = a.

Proof. Let t_{0} ∈ ]a, b] be a nondegenerate (P, S)-focal instant or let t_{0} = a. Since 𝕁[a] = P is nondegenerate for g, we have that, in any case, 𝕁[t_{0}] is nondegenerate for g. Denote by e_{t} : 𝕁 → IR^{n} the evaluation map e_{t}(J) = J(t). Choose a basis (J_{i})_{i=1}^{n} for 𝕁 such that (J_{i})_{i=1}^{k} is a basis for the kernel of e_{t_{0}}; then (J_{i}(t_{0}))_{i=k+1}^{n} is a basis for 𝕁[t_{0}]. Observe that an instant t ∈ ]a, b] is (P, S)-focal if and only if e_{t} is not invertible, i.e., if and only if the determinant det (J_{i}(t))_{i=1}^{n} is zero. Since a solution J of (2.4) is uniquely determined by the values of J(t_{0}) and J′(t_{0}), equality (2.9) implies that the map J ↦ J′(t_{0}) takes the kernel of e_{t_{0}} isomorphically onto 𝕁[t_{0}]^{⊥}. Thus (J′_{i}(t_{0}))_{i=1}^{k} is a basis for 𝕁[t_{0}]^{⊥}. For i = 1, …, k, define J̃_{i} : [a, b] → IR^{n} by setting J̃_{i}(t) = \frac{1}{t−t_{0}} J_{i}(t) if t ≠ t_{0} and J̃_{i}(t_{0}) = J′_{i}(t_{0}); since J_{i} is of class C^{2} and J_{i}(t_{0}) = 0, it follows that J̃_{i} is of class C^{1}. For i = k + 1, …, n we set J̃_{i} = J_{i} and then (J̃_{i}(t_{0}))_{i=1}^{n} is a basis for IR^{n}, because the nondegeneracy of g on 𝕁[t_{0}] implies IR^{n} = 𝕁[t_{0}] ⊕ 𝕁[t_{0}]^{⊥}. It follows that det (J_{i}(t))_{i=1}^{n} = (t − t_{0})^{k} det (J̃_{i}(t))_{i=1}^{n} is nonzero for t ≠ t_{0} near t_{0}. □

Remark 2.3. It is clear from the proof of Proposition 2.2 that the number of focal instants of a quadruple (g, R, P, S) with R real-analytic (or, similarly, the number of focal points corresponding to a quadruple (M, 𝔤, γ, P) with (M, 𝔤) real-analytic) is finite.

3. The Lagrangian Grassmannian and the Maslov Index

Let (V, ω) be a symplectic space, i.e., V is a finite-dimensional real vector space and ω is a symplectic form on V, i.e., a skew-symmetric nondegenerate symmetric bilinear form on V. A linear map T : V → V′ between symplectic spaces (V, ω), (V′, ω′) is called a symplectomorphism if T is an isomorphism and ω is the pull-back of ω′ by T. The set of all symplectomorphisms of (V, ω) is a closed Lie subgroup of the general linear group of V and we will denote it by Sp(V, ω). The Lie algebra of Sp(V, ω), denoted by sp(V, ω), consists of all linear endomorphisms X : V → V such that ω(X·, ·) is a symmetric bilinear form on V. A symplectic space is necessarily even dimensional; we set dim(V) = 2n. By a symplectic basis for V we mean a basis (b_{i})_{i=1}^{2n} of V with ω(b_{i}, b_{n+i}) = −ω(b_{n+i}, b_{i}) = 1, i = 1, …, n and ω(b_{i}, b_{j}) = 0 otherwise. Every symplectic space admits a symplectic basis; in other words, every 2n-dimensional symplectic space is symplectomorphic to the space IR^{n} ⊕ IR^{n}^{∗} endowed with its canonical symplectic form:

(3.1)  ω((v_{1}, α_{1}), (v_{2}, α_{2})) = α_{2}(v_{1}) − α_{1}(v_{2}), v_{1}, v_{2} ∈ IR^{n}, α_{1}, α_{2} ∈ IR^{n}^{∗}.

The symplectic group of IR^{n} ⊕ IR^{n}^{∗} endowed with the symplectic form (3.1) will be denoted by Sp(2n, IR); its Lie algebra will be denoted by sp(2n, IR). If (b_{i})_{i=1}^{2n} is a symplectic basis for V and if a linear endomorphism J : V → V is defined by J(b_{i}) = b_{n+i}, J(b_{n+i}) = −b_{i}, i = 1, …, n then J is a complex structure for V and the bilinear form ω(·, J·) is a positive definite inner product for V; we say that J is compatible with ω. Observe that J is a symplectomorphism of V. If we identify the complex vector space (V, J) with ℂ^{n} by means of the complex basis (b_{i})_{i=1}^{n} then the symplectic form ω corresponds to minus the imaginary part of the canonical Hermitian product of ℂ^{n}. We will call the latter the canonical symplectic form of the space ℂ^{n}. Observe that the unitary group is contained in the symplectic group. A subspace S ⊂ V is called isotropic if ω vanishes on S; an n-dimensional isotropic subspace of V is called a Lagrangian subspace. We are interested in studying the Lagrangian Grassmannian of (V, ω), i.e., the set Λ = Λ(V, ω) of all Lagrangian subspaces of V. Obviously, symplectomorphisms map Lagrangian subspaces to Lagrangian subspaces, and therefore we get an action of Sp(V, ω) on Λ; for every T ∈ Sp(V, ω) we will still denote by T the corresponding map T : Λ → Λ. If ℂ^{n} is endowed with its canonical symplectic form then a subspace L ⊂ ℂ^{n} is Lagrangian if and only if it is a real form (i.e., ℂ^{n} = L ⊕ iL) on which the canonical Hermitian product is real. It follows that the unitary group U(n) acts transitively in the Lagrangian Grassmannian of ℂ^{n}. The isotropy subgroup of the Lagrangian subspace IR^{n} ⊂ ℂ^{n} is the orthogonal group O(n) ⊂ U(n); in particular, the Lagrangian Grassmannian of ℂ^{n} (and thus of any other 2n-dimensional symplectic space) is diffeomorphic to the homogeneous space U(n)/O(n). Hence Λ has the structure of a compact, connected, real-analytic \frac{1}{2}n(n + 1)- dimensional manifold (it is indeed an embedded submanifold of the Grassmannian of all n-dimensional subspaces of V).

If J : V → V is a complex structure compatible with ω and L ⊂ V is a Lagrangian subspace then J(L) is also Lagrangian and V = L⊕J(L). A pair of Lagrangians (L_{0}, L_{1}) with V = L_{0}⊕L_{1} is called a Lagrangian decomposition for V. If (L_{0}, L_{1}) is a Lagrangian decomposition for V then the symplectic form ω induces a natural isomorphism between L_{1} and the dual space of L_{0}. Namely, we define ρ_{L_{0},L_{1}} : L_{1} → L_{0}^{∗} by ρ_{L_{0},L_{1}}(v) = ω(v, ·)|_{L_{0}}, for all v ∈ L_{1}. The anti-symmetry of ω implies that ρ_{L_{0},L_{1}} = −ρ_{L_{1},L_{0}}^{∗}.

Remark 3.1. Observe that if (b_{i})_{i=1}^{n} is an arbitrary basis for L_{0} and (b_{i})_{i=n+1}^{2n} is the basis of L_{1} which is mapped by ρ_{L_{0},L_{1}} to the dual basis of (b_{i})_{i=1}^{n} then (b_{i})_{i=1}^{2n} is a symplectic basis of V. From this observation it follows that if (L_{0}, L_{1}) and (L′_{0}, L′_{1}) are Lagrangian decompositions then every isomorphism from L_{0} to L′_{0} extends (uniquely) to a symplectomorphism of V that carries L_{1} to L′_{1}.

Given a Lagrangian subspace L_{0} ⊂ V, we consider the following subsets of Λ:

Λ_{k}(L_{0}) = {L ∈ Λ : dim(L ∩ L_{0}) = k}, k = 0, 1, …, n;

Λ_{≥k}(L_{0}) = ⋃_{j=k}^{n} Λ_{j}(L_{0}), Λ_{≤k}(L_{0}) = ⋃_{j=0}^{k} Λ_{j}(L_{0}).

For T ∈ Sp(V, ω), we have that T (L_{0}) is in Λ_{≤k}(L_{0}) if and only if the linear map L_{0} ∋ v ↦ T (v) + L_{0} ∈ V/L_{0} has rank greater than or equal to n − k; since this last condition is open, the set Λ_{≤k}(L_{0}) (and in particular the set Λ_{0}(L_{0})) is open in Λ. Obviously the set Λ_{≥k}(L_{0}) is closed in Λ and therefore Λ_{k}(L_{0}) = Λ_{≥k}(L_{0}) ∩ Λ_{≤k}(L_{0}) is locally closed, i.e., it is an open subset of its closure. This observation is used below to show that Λ_{k}(L_{0}) is an embedded submanifold of Λ. We first need the following:

Lemma 3.2. Given Lagrangian subspaces L_{0}, L_{1}, L′_{0}, L′_{1} ⊂ V with dim(L_{0} ∩ L_{1}) = dim(L′_{0} ∩ L′_{1}), there exists T ∈ Sp(V, ω) with T (L_{0}) = L′_{0} and T (L_{1}) = L′_{1}.

Proof. By Remark 3.1 we may assume that L_{0} = L′_{0} and that L_{0} ∩ L_{1} = L′_{0} ∩ L_{1} = S. If S^{⊥} denotes the ω-orthogonal complement of the isotropic space S then S^{⊥}/S has a natural symplectic structure induced by ω and (L_{0}/S, L_{1}/S), (L_{0}/S, L′_{1}/S) are both Lagrangian decompositions of S^{⊥}/S. Again by Remark 3.1 we can find a symplectomorphism \overline{T} of S^{⊥}/S which is the identity on L_{0}/S and such that \overline{T} (L_{1}/S) = L′_{1}/S. Using a complementary Lagrangian to L_{0} in V it is easy to construct a symplectomorphism T of V that is the identity on L_{0} and that induces \overline{T} on the quotient S^{⊥}/S. Obviously T (L_{1}) = L′_{1}. □

Corollary 3.3. The set Λ_{k}(L_{0}) is a locally closed orbit in Λ of the Lie group Sp(V, ω, L_{0}) of symplectomorphisms of V that preserve L_{0}. In particular, Λ_{k}(L_{0}) is an embedded submanifold of Λ.

Proof. Lemma 3.2 implies that Λ_{k}(L_{0}) is an orbit of Sp(V, ω, L_{0}) and a locally closed orbit of a Lie group is an embedded submanifold (see [19, Theorem 2.9.7]). □

Let us now describe an explicit differentiable atlas for Λ. Let (L_{0}, L_{1}) be a Lagrangian decomposition for V; we define a map ϕ_{L_{0},L_{1}} : Λ_{0}(L_{1}) ⟶ Bil_{sym}(L_{0}) by setting ϕ_{L_{0},L_{1}}(L)(v, w) = ω(Zv, w), for all L ∈ Λ_{0}(L_{1}) and all v, w ∈ L_{0}, where Z : L_{0} → L_{1} is the unique linear map whose graph in V = L_{0}⊕L_{1} equals L. By identifying Bil_{sym}(L_{0}) with a subspace of Lin(L_{0}, L_{0}^{∗}), we have ϕ_{L_{0},L_{1}}(L) = ρ_{L_{0},L_{1}} ∘ Z. The map ϕ_{L_{0},L_{1}} is a diffeomorphism onto Bil_{sym}(L_{0}), i.e., a chart on the manifold Λ. If (L_{0}, L_{1}) and (L_{0}, L′_{1}) are Lagrangian decompositions for V then an explicit computation of the transition map from ϕ_{L_{0},L_{1}} to ϕ_{L_{0},L′_{1}} shows that dϕ_{L_{0},L_{1}}(L_{0}) = dϕ_{L_{0},L′_{1}}(L_{0}), i.e., the differential dϕ_{L_{0},L_{1}}(L_{0}) : T_{L_{0}}Λ → Bil_{sym}(L_{0}) does not depend on the choice of L_{1}. We shall henceforth identify the tangent space T_{L}Λ with the space Bil_{sym}(L) by means of the isomorphism obtained by the differential of a chart of the form ϕ_{L,L_{1}}. If ℓ : I → Λ is a curve of class C^{1} defined in an interval I then the derivative ℓ′(t_{0}) ∈ Bil_{sym}(ℓ(t_{0})), t_{0} ∈ I, can be computed as follows; choose v, w ∈ ℓ(t_{0}) and a map 𝔳 : (t_{0} − ε, t_{0} + ε) → V of class C^{1} with 𝔳(t_{0}) = v and 𝔳(t) ∈ ℓ(t) for all t. Then ℓ′(t_{0})(v, w) = ω(𝔳′(t_{0}), w). Using this formula one can easily prove the following:

Lemma 3.4. Given a Lagrangian L_{0} ∈ Λ, denote by β_{L_{0}} : Sp(V, ω) → Λ the evaluation map β_{L_{0}}(T) = T (L_{0}), T ∈ Sp(V, ω); its differential is given by:

(3.2)  dβ_{L_{0}}(T) · (XT) = ω(X·, ·)|_{T (L_{0})} ∈ Bil_{sym}(T (L_{0})),

for all X ∈ sp(V, ω). If T ∈ Sp(V, ω) is a symplectomorphism then the differential of the induced diffeomorphism T : Λ → Λ is given by:

dT (L) · B = B(T^{−1}·, T^{−1}·)|_{T (L)} ∈ Bil_{sym}(T (L)),

for all L ∈ Λ, B ∈ Bil_{sym}(L). Given a Lagrangian decomposition (L_{0}, L_{1}) then the differential of the chart ϕ_{L_{0},L_{1}} at a point L ∈ Λ_{0}(L_{1}) is given by:

(3.3)  dϕ_{L_{0},L_{1}}(L) · B = B(η·, η·)|_{L_{0}} ∈ Bil_{sym}(L_{0}), B ∈ Bil_{sym}(L),

where the isomorphism η : L_{0} → L is the restriction to L_{0} of the projection onto L relative to the decomposition V = L ⊕ L_{1}. □

Using formula (3.2) it is easy to compute the tangent space of the manifold Λ_{k}(L_{0}) at a point L ∈ Λ_{k}(L_{0}). Namely, if sp(V, ω, L_{0}) denotes the Lie algebra of Sp(V, ω, L_{0}) then Corollary 3.3 implies that the tangent space T_{L}Λ_{k}(L_{0}) is equal to the image of sp(V, ω, L_{0}) by dβ_{L}(Id); keeping in mind that sp(V, ω, L_{0}) consists of those linear endomorphisms X : V → V such that ω(X·, ·) is a symmetric bilinear form on V vanishing on L_{0}, we obtain from (3.2) that T_{L}Λ_{k}(L_{0}) equals the space of bilinear forms B ∈ Bil_{sym}(L) that vanish on L ∩ L_{0}. Thus, the restriction map B ↦ B|_{L∩L_{0}} induces an isomorphism between the quotient T_{L}Λ/T_{L}Λ_{k}(L_{0}) and Bil_{sym}(L ∩ L_{0}); in particular, Λ_{k}(L_{0}) has codimension \frac{1}{2}k(k + 1) in Λ and Λ_{1}(L_{0}) is a hypersurface on Λ. Such hypersurface has a canonical transverse orientation which is defined by declaring a vector B ∈ T_{L}Λ, B ∉ T_{L}Λ_{1}(L_{0}), positively oriented if the nonzero symmetric bilinear form B|_{L∩L_{0}} on the line L ∩ L_{0} is positive definite.

Remark 3.5. If {L_{j} : j = 1, 2, … } is a countable subset of Λ then we can find a Lagrangian L ∈ Λ with L ∩ L_{j} = {0} for all j. Namely, we have shown above that Λ_{k}(L_{j}) is a submanifold of Λ of positive codimension for k = 1, …, n; hence, the union ⋃_{j=1}^{+∞} ⋃_{k=1}^{n} Λ_{k}(L_{j}) has null measure in Λ.

Let L ∈ Λ in the domain of ϕ_{L_{0},L_{1}} be fixed and set β = ϕ_{L_{0},L_{1}}(L) ∈ Bil_{sym}(L_{0}). Then:

(3.4)  L_{0} ∩ L = Ker(β);

then L ∈ Λ_{k}(L_{0}) iff the kernel of β has dimension equal to k and in particular L ∈ Λ_{≥1}(L_{0}) iff β is degenerate. Assume now that ℓ : I ⊂ IR → Λ is a curve of class C^{1} in Λ with image in the domain of ϕ_{L_{0},L_{1}}; set β = ϕ_{L_{0},L_{1}} ∘ ℓ. Using formula (3.3) and keeping in mind that the isomorphism η in (3.3) is the identity on L_{0} ∩ L we conclude that:

(3.5)  ℓ′(t_{0})|_{ℓ(t_{0})∩L_{0}} = β′(t_{0})|_{Ker(β(t_{0}))},

for all t_{0} ∈ I. We are now going to define the Maslov index of a curve of Lagrangians. We start by recalling from [1] that the fundamental group of the Lagrangian Grassmannian is infinite cyclic. More explicitly, we have Λ ≅ U(n)/O(n); the square of the determinant map induces a fibration det^{2} : U(n)/O(n) → S^{1}. The long exact homotopy sequence of such fibration shows that the map det^{2} induces an isomorphism between fundamental groups; then, by Hurewicz theorem, π_{1}(Λ) ≅ H_{1}(Λ) ≅ ℤ. Let L_{0} ∈ Λ be fixed. The open set Λ_{0}(L_{0}) ⊂ Λ is contractible (because it is diffeomorphic to a vector space) and therefore the inclusion of (Λ, ∅) in (Λ, Λ_{0}(L_{0})) induces an isomorphism in singular homology. So we have an isomorphism:

(3.6)  μ_{L_{0}} : H_{1}(Λ, Λ_{0}(L_{0})) ⟶^{≅} ℤ.

The Maslov index (relative to L_{0}) of a continuous curve ℓ : [a, b] → Λ with endpoints in Λ_{0}(L_{0}) is defined as the integer number corresponding to the homology class determined by ℓ in H_{1}(Λ, Λ_{0}(L_{0})) via the isomorphism (3.6). We denote the Maslov index of ℓ by μ_{L_{0}}(ℓ). Obviously the Maslov index is additive by concatenation and invariant by homotopies of curves with endpoints in Λ_{0}(L_{0}). Actually, in order to complete the definition of Maslov index we have to choose a sign for the isomorphism (3.6) (Lemma 3.8 below).

Lemma 3.6. Let (L_{0}, L_{1}) be a Lagrangian decomposition for V and let ℓ_{1}, ℓ_{2} : [a, b] → Λ be continuous curves with endpoints in Λ_{0}(L_{0}) and image contained in the domain of the chart ϕ_{L_{0},L_{1}}; set β_{i} = ϕ_{L_{0},L_{1}} ∘ ℓ_{i} : [a, b] → Bil_{sym}(L_{0}), i = 1, 2. If n_{+}(β_{1}(a)) = n_{+}(β_{2}(a)) and n_{+}(β_{1}(b)) = n_{+}(β_{2}(b)) then the curves ℓ_{1} and ℓ_{2} are homologous in H_{1}(Λ, Λ_{0}(L_{0})).

Proof. Since the set of nondegenerate symmetric bilinear forms on L_{0} with a fixed coindex is connected, we can find a homotopy from β_{1} to β_{2} through curves in Bil_{sym}(L_{0}) whose endpoints are nondegenerate symmetric bilinear forms. Such homotopy lifts via ϕ_{L_{0},L_{1}} to a homotopy from ℓ_{1} to ℓ_{2} through curves with endpoints in Λ_{0}(L_{0}) (recall (3.4)). □

Lemma 3.7. Let β : [t_{0}, t_{1}[ → Bil_{sym}(IR^{n}) be a C^{1} curve such that the restriction of β′(t_{0}) to the kernel of β(t_{0}) is nondegenerate. Then, for ε > 0 sufficiently small, the bilinear form β(t_{0} + ε) is nondegenerate and its coindex is given by:

n_{+}(β(t_{0} + ε)) = n_{+}(β(t_{0})) + n_{+}(β′(t_{0})|_{Ker(β(t_{0}))}).

Thus, if β is of class C^{1} in a neighborhood of t_{0} then β(t) is nondegenerate for t ≠ t_{0} sufficiently near t_{0} and:

n_{+}(β(t_{0} + ε)) − n_{+}(β(t_{0} − ε)) = sgn(β′(t_{0})|_{Ker(β(t_{0}))}),

for ε > 0 sufficiently small.

Proof. See [12, Proposition 2.5]. □

The lemma below gives a canonical sign for the isomorphism (3.6).

Lemma 3.8. Let ℓ : [a, b] → Λ be a curve of class C^{1} such that:

(a) ℓ(t) ∈ Λ_{≥1}(L_{0}) at a unique instant t = t_{0} ∈ ]a, b[;

(b) ℓ(t_{0}) ∈ Λ_{1}(L_{0});
(c) ℓ′(t_{0}) is positively oriented in T_{ℓ(t_{0})}Λ/T_{ℓ(t_{0})}Λ_{1}(L_{0}).

Then ℓ defines a generator of H_{1}(Λ, Λ_{0}(L_{0})). Moreover, any two curves of class C^{1} in Λ satisfying conditions (a), (b), (c) are homologous in H_{1}(Λ, Λ_{0}(L_{0})).

Proof. Recalling that det^{2} : Λ ≅ U(n)/O(n) → S^{1} induces an isomorphism between fundamental groups it is easy to exhibit a generator of the fundamental group of Λ (and hence of H_{1}(Λ, Λ_{0}(L_{0}))) that satisfies (a), (b) and (c). We now have to show that if ℓ_{1}, ℓ_{2} : [t_{0} − ε, t_{0} + ε] → Λ are curves of class C^{1} in Λ satisfying (a), (b) and (c) then ℓ_{1} and ℓ_{2} are homologous in H_{1}(Λ, Λ_{0}(L_{0})). The action of Sp(V, ω, L_{0}) preserves the transverse orientation of Λ_{1}(L_{0}) in Λ and it induces the identity^{1} on H_{1}(Λ, Λ_{0}(L_{0})). Since Sp(V, ω, L_{0}) acts transitively on Λ_{1}(L_{0}) (Corollary 3.3), we may assume that ℓ_{1}(t_{0}) = ℓ_{2}(t_{0}) = L. The conclusion follows from Lemmas 3.6, 3.7 and formulas (3.4) and (3.5), using a local chart ϕ_{L_{0},L_{1}} around L (the existence of such chart follows from Remark 3.5).

□

From now on, we fix the sign of the isomorphism (3.6) by choosing any C^{1} curve in Λ satisfying (a), (b) and (c) as a generator of H_{1}(Λ, Λ_{0}(L_{0})).

Proposition 3.9. Let ℓ : [a, b] → Λ be a continuous curve with endpoints outside Λ_{≥1}(L_{0}). If there exists a Lagrangian subspace L_{1} complementary to L_{0} and such that the image of ℓ is entirely contained in the domain Λ_{0}(L_{1}) of the chart ϕ_{L_{0},L_{1}} then, setting β = ϕ_{L_{0},L_{1}} ∘ℓ, the Maslov index of ℓ with respect to L_{0} is given by:

1Since Sp(V, ω) is connected, for T ∈ Sp(V, ω), the diffeomorphism T : Λ → Λ is homotopic to the identity of Λ and therefore induces the identity of H_{1}(Λ). It follows from functoriality of homology that every T ∈ Sp(V, ω, L_{0}) induces the identity on H_{1}(Λ, Λ_{0}(L_{0})).

(3.7)  μ_{L_{0}}(ℓ) = n_{+}(β(b)) − n_{+}(β(a)).

Proof. By Lemma 3.6 it suffices to find for every i, j = 0, …, n a continuous curve β̃ : [0, 1] → Bil_{sym}(L_{0}) such that the Maslov index of ℓ̃= ϕ_{L_{0},L_{1}}^{−1} ∘ β̃ with respect to L_{0} is equal to j − i and such that β̃(0), β̃(1) are both nondegenerate and have coindexes respectively equal to i and j. It is easy to provide examples of such a curve β̃ by considering, for instance, diagonal matrices. □

Corollary 3.10. Let ℓ : [a, b] → Λ be a curve of class C^{1} having endpoints outside Λ_{≥1}(L_{0}). If for all t ∈ ]a, b[ such that ℓ(t) ∈ Λ_{≥1}(L_{0}) we have that ℓ′(t) is nondegenerate on ℓ(t) ∩ L_{0}, then the number of intersections of ℓ with Λ_{≥1}(L_{0}) is finite, and:

(3.8)  μ_{L_{0}}(ℓ) = ∑_{t∈]a,b[} sgn(ℓ′(t)|_{ℓ(t)∩L_{0}}).

Proof. If ℓ(t) ∈ Λ_{≥1}(L_{0}) then, by Remark 3.5, we can find a Lagrangian L_{1} ∈ Λ with L_{1}∩ L_{0} = L_{1} ∩ ℓ(t) = {0}; thus we can consider the local chart ϕ_{L_{0},L_{1}} around ℓ(t). Keeping in mind formulas (3.4) and (3.5), it follows directly from Proposition 3.9 and Lemma 3.7 that μ_{L_{0}}(ℓ|_{[t−ε,t+ε]}) = sgn(ℓ′(t)|_{ℓ(t)∩L_{0}}) for ε > 0 small enough. The conclusion now follows from the additivity of the Maslov index by concatenation of curves. □

4. Applications of the Maslov Index: Stability of the Focal Index

4.1. The Maslov index of a differential problem

Let (g, R, P, S) be an admissible quadruple for the differential problem. By setting α(t) = gJ′(t) ∈ IR^{n}^{∗}, the second order differential equation (2.4) can be rewritten as:

(4.1)  \frac{d}{dt} (\begin{matrix}J(t) \\ α(t) \end{matrix}) = X(t) (\begin{matrix}J(t) \\ α(t) \end{matrix}) ,

where X(t) is the linear endomorphism of IR^{n} ⊕ IR^{n}^{∗} given by:

(4.2)  X(t) = (\begin{matrix}0 & g^{−1} \\ gR(t) & 0 \end{matrix}) , t ∈ [a, b].

Let Φ denote the fundamental matrix of the system (4.1), i.e., for each t ∈ [a, b], Φ(t) is the isomorphism of IR^{n} ⊕ IR^{n}^{∗} such that:

(4.3)  Φ(t)(J(a), gJ′(a)) = (J(t), gJ′(t)),

for all solutions J of (2.4). Then Φ satisfies the following matrix differential equation:

(4.4)  Φ′(t) = X(t)Φ(t), t ∈ [a, b],

with initial condition Φ(a) = Id. The conservation law (2.7) implies that Φ(t) is indeed a symplectomorphism of the symplectic space IR^{n} ⊕ IR^{n}^{∗} endowed with its canonical symplectic form (3.1); alternatively, the fact that Φ(t) belongs to Sp(2n, IR) follows from (4.4) by observing that X(t) belongs to the Lie algebra sp(2n, IR).

Consider the following two Lagrangian subspaces of IR^{n} ⊕ IR^{n}^{∗}:

ℓ_{0} = {(v, α) ∈ IR^{n} ⊕ IR^{n}^{∗} : v ∈ P, α|_{P} + S(v) = 0 ∈ P^{∗}}, (4.5)

L_{0} = {0} ⊕ IR^{n}^{∗}; (4.6) observe that the initial condition (2.5) is equivalent to (J(a), gJ′(a)) ∈ ℓ_{0}. Since Φ(t) is a symplectomorphism, the space ℓ(t) = Φ(t)(ℓ_{0}) is Lagrangian; more explicitly, ℓ(t) is given by:

(4.7)  ℓ(t) = {(J(t), gJ′(t)) : J ∈ 𝕁},

for all t ∈ [a, b]. From (2.9) it follows that:

(4.8)  ℓ(t) ∩ L_{0} = {0} ⊕ 𝕁[t]^{o} ⊂ {0} ⊕ IR^{n}^{∗}.

In particular, an instant t ∈ ]a, b] is (P, S)-focal if and only if ℓ(t) ∩ L_{0} ≠ {0}; moreover, t has multiplicity k if and only if ℓ(t) ∈ Λ_{k}(L_{0}). In the notation of Lemma 3.4, we have:

(4.9)  ℓ = β_{ℓ_{0}} ∘ Φ,

and in particular ℓ is a curve of class C^{1} on the Lagrangian Grassmannian Λ of IR^{n} ⊕IR^{n}^{∗}. Recalling Proposition 2.2, we give the following:

Definition 4.1. Let (g, R, P, S) be an admissible quadruple for the differential problem and assume that t = b is not a (P, S)-focal instant. The Maslov index of (g, R, P, S) is defined by i_{Maslov}(g, R, P, S) = μ_{L_{0}}(ℓ|_{[a+ε,b]}), where ε > 0 is chosen in such a way that there are no (P, S)-focal instants in ]a, a + ε].

Obviously the Maslov index of (g, R, P, S) does not depend on the choice of ε.

We now want to compute the Maslov index of a quadruple (g, R, P, S) using Corollary 3.10. We start by computing the derivative ℓ′(t) in the lemma below.

Lemma 4.2. For every t ∈ [a, b] the isomorphism 𝕁[t]^{⊥} ∋ v ↦ (0, g(v)) ∈ ℓ(t) ∩ L_{0} carries the restriction of g to the restriction of ℓ′(t).

Proof. From (3.2), (4.4) and (4.9) it follows that:

(4.10)  ℓ′(t) = ω(X(t)·, ·)|_{ℓ(t)}.

The conclusion then follows from (4.2) and (4.8). □

Theorem 4.3. Let (g, R, P, S) be an admissible quadruple for the differential problem in IR^{n}. If all the (P, S)-focal instants are nondegenerate and if t = b is not (P, S)-focal then the focal index of (g, R, P, S) is (well defined and) equal to the Maslov index of (g, R, P, S), i.e., i_{foc}(g, R, P, S) = i_{Maslov}(g, R, P, S).

Proof. Follows directly from Corollary 3.10 and Lemma 4.2. □

Corollary 4.4. Let (M, 𝔤, P, γ) be an admissible quadruple for the geometric problem such that γ(b) is not a P-focal point. Assume that (M, 𝔤) is Riemannian or Lorentzian, and in the latter case, that γ is nonspacelike. If (g, R, P, S) corresponds to (M, 𝔤, P, γ) by a parallel trivialization of the tangent bundle then the Maslov index of (g, R, P, S) is equal to the sum of the multiplicities of the P-focal points, i.e., i_{Maslov}(g, R, P, S) = ∑_{t∈]a,b]} mul(t). □

Remark 4.5. It is easy to see that every Lagrangian subspace ℓ_{0} ⊂ IR^{n}⊕IR^{n}^{∗} is of the form (4.5) for a unique pair (P, S), with P ⊂ IR^{n} a subspace and S : P × P → IR a symmetric bilinear form. This means that (2.5)is the most general form of initial conditions that yield a Lagrangian subspace.

4.2. Stability of the indexes

Let G_{r}(n) ≅ O(n)/(O(r) × O(n − r)) denote the Grassmannian of all r-dimensional subspaces of IR^{n}. Let E_{r,n} denote the tautological vector bundle over G_{r}(n), i.e., the set of pairs (P, v) ∈ G_{r}(n) × IR^{n} with v ∈ P. We denote by ⋁_{2} E_{r,n}^{∗} the second symmetric power of the dual bundle of E_{r,n}, i.e., ⋁_{2} E_{r,n}^{∗} is a vector bundle over the Grassmannian G_{r}(n) whose fiber over P ∈ G_{r}(n) is the space Bil_{sym}(P) of symmetric bilinear forms on P. The elements of ⋁_{2} E_{r,n}^{∗} are thus pairs (P, S) with P ⊂ IR^{n} an r-dimensional subspace and S : P × P → IR a symmetric bilinear form. We have a bijective map (see Remark 4.5):

(4.11)  ⋁_{2} E_{r,n}^{∗} ∋ (P, S) ⟼ ℓ_{0} ∈ Λ_{n−r}(L_{0}),

where ℓ_{0} is defined as in (4.5). The map (4.11) is in fact a smooth diffeomorphism. This can be proven, for instance, by observing that the Lie group Sp(2n, IR, L_{0}) of symplectomorphisms of IR^{n} ⊕ IR^{n}^{∗} that preserve L_{0} acts smoothly and transitively in both ⋁_{2} E_{r,n}^{∗} and Λ_{n−r}(L_{0}); moreover, the map (4.11) intertwines such actions.

We can now prove the following result about the stability of the Maslov and the focal indexes for quadruples (g, R, P, S).

Theorem 4.6. For each j ∈ IN ∪ {∞}, let (g_{j}, R_{j}, P_{j}, S_{j}) be an admissible quadruple for the differential problem in IR^{n}.

Assume that (g_{j}, R_{j}, P_{j}, S_{j}) converges to (g_{∞}, R_{∞}, P_{∞}, S_{∞}) in the following sense:

(1) dim(P_{j}) = r for all j ∈ IN ∪ {∞} and some r = 0, …, n;

(2) (P_{j}, S_{j}) → (P_{∞}, S_{∞}) in ⋁_{2} E_{r,n}^{∗} as j → ∞;

(3) g_{j} → g_{∞} in Bil_{sym}(IR^{n}) as j → ∞;

(4) R_{j} → R_{∞} uniformly on [a, b] as j → ∞. If t_{0} = b is not (P_{∞}, S_{∞})-focal for (g_{∞}, R_{∞}, P_{∞}, S_{∞}) then, for j ∈ IN sufficiently large, t_{0} = b is not (P_{j}, S_{j})-focal for (g_{j}, R_{j}, P_{j}, S_{j}), and:

(4.12)  i_{Maslov}(g_{j}, R_{j}, P_{j}, S_{j}) = i_{Maslov}(g_{∞}, R_{∞}, P_{∞}, S_{∞});

in particular, if all the (P_{j}, S_{j})-focal instants of (g_{j}, R_{j}, P_{j}, S_{j}) are nondegenerate for all j ∈ IN ∪ {∞} then, for j sufficiently large:

(4.13)  i_{foc}(g_{j}, R_{j}, P_{j}, S_{j}) = i_{foc}(g_{∞}, R_{∞}, P_{∞}, S_{∞}),

Proof. We define objects (ℓ_{0})_{j}, Φ_{j} and ℓ_{j} corresponding to the quadruple (g_{j}, R_{j}, P_{j}, S_{j}) in analogy respectively with (4.5), (4.3) and (4.7). Since (P_{j}, S_{j}) → (P, S), the fact that (4.11) is a diffeomorphism implies that (ℓ_{0})_{j} → ℓ_{0} in Λ. Standard results on the continuous dependence of solutions of a differential equation with respect to the data show that Φ_{j} → Φ uniformly on [a, b] (and even in the C^{1}-topology). The smoothness of the action of Sp(2n, IR) on Λ and (4.9) imply then that ℓ_{j} → ℓ on the compact-open topology; in particular, ℓ_{j}(b) → ℓ(b) in Λ. The fact that t_{0} = b is not (P_{∞}, S_{∞})-focal for (g_{∞}, R_{∞}, P_{∞}, S_{∞}) means that ℓ(b) belongs to the open set Λ_{0}(L_{0}); thus ℓ_{j}(b) ∈ Λ_{0}(L_{0}) for large j and therefore t_{0} = b is not (P_{j}, S_{j})-focal for (g_{j}, R_{j}, P_{j}, S_{j}) as well. We will see in Lemma 4.10 below that there exists ε > 0 such that none of the quadruples (g_{j}, R_{j}, P_{j}, S_{j}) has focal instants in ]a, a + ε]; thus:

i_{Maslov}(g_{j}, R_{j}, P_{j}, S_{j}) = μ_{L_{0}}(ℓ_{j}|_{[a+ε,b]}), i_{Maslov}(g, R, P, S) = μ_{L_{0}}(ℓ|_{[a+ε,b]}).

Since Λ is semi-locally simply-connected and Λ_{0}(L_{0}) is locally arc-connected, the fact that ℓ_{j} → ℓ in the compact-open topology implies that ℓ_{j}|_{[a+ε,b]} is homologous to ℓ|_{[a+ε,b]} in H_{1}(Λ, Λ_{0}(L_{0})) for j sufficiently large; thus μ_{L_{0}}(ℓ_{j}|_{[a+ε,b]}) = μ_{L_{0}}(ℓ|_{[a+ε,b]}) for such j. This proves equality (4.12); equality (4.13) now follows from Theorem 4.3. □

If (M, 𝔤) is an n-dimensional semi-Riemannian manifold and P ⊂ M is a smooth submanifold then the normal bundle T P^{⊥} = ⋃_{p∈P} T_{p}P^{⊥} is a smooth n-dimensional submanifold of the tangent bundle T M. Thus, for every 𝔫 ∈ T_{p}P^{⊥}, p ∈ P, the tangent space T_{𝔫}(T P^{⊥}) is an n-dimensional subspace of the 2n-dimensional space T_{𝔫}T M. Moreover, if P is nondegenerate at p and if one identifies T_{𝔫}T M with the direct sum T_{p}M ⊕ T_{p}M using the horizontal space determined by the Levi-Civita connection then T_{𝔫}(T P^{⊥}) is given by (compare with (2.2)):

(4.14) T_{𝔫}(T P^{⊥}) = {(v, w) ∈ T_{p}M ⊕ T_{p}M : v ∈ T_{p}P, w + A_{𝔫}(v) ∈ T_{p}P^{⊥}}.

In the corollary below we denote by G_{n}(T M) the fiber bundle over T M whose fiber at v ∈ T M is the Grassmannian of all n-dimensional subspaces of the tangent space T_{v}T M. Observe that T_{𝔫}(T P^{⊥}) is then a point in G_{n}(T M).

Corollary 4.7. Let (M, 𝔤) be an n-dimensional semi-Riemannian manifold. Assume that we are given a family of pairs (P_{j}, γ_{j}), j ∈ IN ∪ {∞}, where each P_{j} is a submanifold of M and each γ_{j} : [a, b] → M is a geodesic starting orthogonally at P_{j}. Assume that:

(1) P_{j} is nondegenerate at γ_{j}(a) for all j ∈ IN ∪ {∞};

(2) dim(P_{j}) = dim(P_{∞}) for all j ∈ IN;

(3) all P_{j}-focal points along γ_{j} are nondegenerate, for all j ∈ IN ∪ {∞};

(4) γ_{∞}(b) is not P_{∞}-focal along γ_{∞};

(5) γ′_{j}(a) → γ′_{∞}(a) in T M as j → ∞;

(6) T_{γ′_{j}(a)}(T P_{j}^{⊥}) → T_{γ′_{∞}(a)}(T P_{∞}^{⊥}) in G_{n}(T M) as j → ∞.

Then, for j large enough, γ_{j}(b) is not P_{j}-focal along γ_{j} and i_{foc}(γ_{j}, P_{j}) = i_{foc}(γ_{∞}, P_{∞}).

Proof. Choose a smooth referential around γ_{∞}(a) and parallel transport such referential along each γ_{j}. The conclusion follows by applying Theorem 4.6 to the quadruples (g_{j}, R_{j}, P_{j}, S_{j}) obtained from (M, 𝔤, P_{j}, γ_{j}). □

Corollary 4.8. Under the hypothesis of Corollary 4.7, if (M, 𝔤) is Riemannian or if (M, 𝔤) is Lorentzian and γ_{j} is nonspacelike for all j ∈ IN ∪ {∞}, then for j sufficiently large, the sum of the multiplicities of the P_{j}-focal points along γ_{j} is equal to the sum of the multiplicities of the P_{∞}-focal points along γ_{∞}. □

Hypothesis (2) is essential for Corollary 4.7 as the following example shows.

Example 4.9. Consider the plane M = IR^{2} endowed with the standard Euclidean metric 𝔤. For j ∈ IN, let P_{j} ⊂ IR^{2} be the circle centered at (0, \frac{1}{j}) and passing through the origin; then the segment [0, 1] ∋ t ↦ γ(t) = (0, t) is a geodesic starting orthogonally to P_{j} and containing a unique P_{j}-focal point at (0, \frac{1}{j}) whose multiplicity (and signature) is equal to 1. Hence i_{foc}(γ, P_{j}) = 1. We have T_{γ′(0)}(T P_{j}^{⊥}) = span\{(\frac{1}{j}, 0, −1, 0), (0, 0, 0, 1)\}; thus T_{γ′(0)}(T P_{j}^{⊥}) → {0} × IR^{2} = T_{γ′(0)}(T P_{∞}^{⊥}), where P_{∞} = {0}. But γ has no P_{∞}-focal (i.e., conjugate) points and thus i_{foc}(γ, P_{∞}) = 0.

Lemma 4.10. Let X be a topological space. For each λ ∈ X, let (g_{λ}, R_{λ}, P_{λ}, S_{λ}) be an admissible quadruple for the differential problem in IR^{n}. Assume that dim(P_{λ}) = r for all λ ∈ X and assume that the maps:

X ∋ λ ⟼ g_{λ} ∈ Bil_{sym}(IR^{n}), X × [a, b] ∋ (λ, t) ⟼ R_{λ}(t) ∈ Lin(IR^{n}),
X ∋ λ ⟼ (P_{λ}, S_{λ}) ∈ ⋁_{2} E_{r,n}^{∗},

are continuous. Then, given λ_{0} ∈ X, there exists ε > 0 and a neighborhood V of λ_{0} in X such that (g_{λ}, R_{λ}, P_{λ}, S_{λ}) has no (P_{λ}, S_{λ})-focal instants in ]a, a + ε] for all λ ∈ V.

Proof. Arguing as in the proof of Proposition 2.2, we can find continuous maps (t, λ) ↦ J̃_{i}(t, λ), i = 1, …, n, such that (J̃_{i}(a, λ_{0}))_{i=1}^{n} is a basis of IR^{n} and such that t ∈ ]a, b] is (P_{λ}, S_{λ})-focal for (g_{λ}, R_{λ}, P_{λ}, S_{λ}) iff det (J̃_{i}(t, λ))_{i=1}^{n} = 0. The conclusion follows. □

5. Curves of Lagrangians originating from differential problems

Given an admissible quadruple for the differentiable problem (g, R, P, S), one can construct objects Φ, ℓ_{0} and ℓ as in Subsection 4.1. In Subsections 5.1, 5.2 and 5.3 we investigate the inverse problem: when does a curve ℓ : [a, b] → Λ arises from a quadruple (g, R, P, S)? In Subsection 5.4 we use the theory developed in the earlier subsections to provide a counterexample to the equality between the Maslov and the focal index of a quadruple (g, R, P, S) having degenerate focal instants. More specifically, we give an example of an admissible quadruple (g, R, P, S) in IR^{2}, with R real-analytic, n_{+}(g) = 1, P = {0}, having a unique (P, S)-focal instant, and whose Maslov index is equal to −1, while the focal index is equal to zero.

It is proven in [14, Section 3] that every admissible quadruple for the differential problem in IR^{n} with P = {0} and R smooth can be obtained from an admissible quadruple for the geometric problem (M, 𝔤, γ, P), with P = {γ(a)}, dim(M) = n + 1, by a parallel trivialization of the normal bundle along γ. Moreover, (M, 𝔤) can be chosen to be conformally flat and γ can be chosen to be either timelike or spacelike. In particular, the counter-example given in Subsection 5.4 actually admits a geometric realization in terms of a spacelike geodesic γ on a real-analytic conformally flat three-dimensional Lorentzian manifold (M, 𝔤).

5.1. Differential problems generate curves of Lagrangians tangent to affine distributions

Let g ∈ Bil_{sym}(IR^{n}) be a fixed nondegenerate symmetric bilinear form in IR^{n}. Let D̂ denote the right-invariant distribution of affine spaces in the Lie group Sp(2n, IR) whose value at the identity is the affine subspace of the Lie algebra sp(2n, IR) consisting of those linear endomorphisms of IR^{n} ⊕ IR^{n}^{∗} that can be written in block-matrix form as X = ( \begin{matrix}0 & g^{−1} \\ C & 0 \end{matrix} ), with C ∈ Bil_{sym}(IR^{n}) symmetric. From (4.2)and (4.4)it follows directly that

a C^{1} curve Φ : [a, b] → Sp(2n, IR) with Φ(a) = Id arises from an admissible quadruple for the differential problem (g, R, P, S) if and only if Φ is tangent to the distribution D̂, i.e., if Φ′(t) ∈ D̂_{Φ(t)} for all t ∈ [a, b].

Now let ℓ_{0} ∈ Λ be fixed and consider the fibration β_{ℓ_{0}} : Sp(2n, IR) → Λ given by evaluation at ℓ_{0}. The curve of Lagrangians ℓ arising from (g, R, P, S) is simply the projection on Λ of the curve Φ (recall (4.9)). Since the Lie group Sp(2n, IR) acts on Λ on the left, the right-invariant distribution D̂ on Sp(2n, IR) is projectable on Λ and such projection does not depend on the choice of ℓ_{0}. More explicitly, we define a distribution of affine spaces D on Λ by setting D_{L} = dβ_{L}(Id)(D̂_{Id}) ⊂ T_{L}Λ, for all L ∈ Λ. It is easy to see that if Φ : [a, b] → Sp(2n, IR) is tangent to D̂ and if ℓ = β_{ℓ_{0}} ∘ Φ for some ℓ_{0} then ℓ is tangent to D. Hence, tangency to D is a necessary condition for a C^{1} curve ℓ on Λ to arise from some quadruple (g, R, P, S). We remark that the distribution D on Λ does not have constant rank. Using formula (3.2) for dβ_{L} one computes easily:

D_{L} = {(0 ⊕ g^{−1})|_{L} + B : B ∈ Bil_{sym}(L), L_{0} ∩ L ⊂ Ker(B)},

where L_{0} = {0} ⊕ IR^{n}^{∗} and 0 ⊕ g^{−1} ∈ Bil_{sym}(IR^{n} ⊕ IR^{n}^{∗}) denotes the symmetric bilinear form (0 ⊕ g^{−1})((v_{1}, α_{1}), (v_{2}, α_{2})) = g^{−1}(α_{1}, α_{2}). Thus, for L ∈ Λ_{k}(L_{0}), we have dim(D_{L}) = \frac{1}{2}(n−k)(n−k +1). In particular D has constant rank on each manifold Λ_{k}(L_{0}); in the open dense set Λ_{0}(L_{0}), the distribution D is equal to the entire tangent bundle of Λ. Hence, the condition ℓ′ ∈ D is vacuous outside the variety Λ_{≥1}(L_{0}), but it gives some information on how ℓ should intersect Λ_{≥1}(L_{0}).

5.2. A study of curves of Lagrangians in local coordinates

Let L_{1} ⊂ IR^{n} ⊕ IR^{n}^{∗} be a Lagrangian complementary to L_{0} = {0} ⊕ IR^{n}^{∗}. We study the curve ℓ : [a, b] → Λ arising from a quadruple (g, R, P, S) in the chart ϕ_{L_{0},L_{1}}. The Lagrangian L_{1} can be written in the form:

(5.1)  L_{1} = {(v, Zv) : v ∈ IR^{n}},

where Z : IR^{n} → IR^{n}^{∗} is symmetric. We set β(t) = ϕ_{L_{0},L_{1}}(ℓ(t)) ∈ Bil_{sym}(L_{0}), for t in some subinterval of [a, b]. We identify β(t) with a symmetric bilinear form on IR^{n}^{∗} and with a linear map β(t) : IR^{n}^{∗} → IR^{n}. Then ℓ(t) is the graph of L_{0} ∋ (0, α) ⟼ (β(t)α, Zβ(t)α) ∈ L_{1}. Obviously β′(t) = dϕ_{L_{0},L_{1}}(ℓ(t)) · ℓ′(t); from (4.10) and (3.3) we get:

(5.2)  β′(t) = ω(X(t)η(t)·, η(t) · ),

where η(t) : L_{0} → ℓ(t) is the isomorphism given by the restriction to L_{0} of the projection onto ℓ(t) corresponding to the decomposition IR^{n} ⊕ IR^{n}^{∗} = ℓ(t) ⊕ L_{1}. One computes:

(5.3)  η(t) : L_{0} ∋ (0, α) ⟼ (β(t)α, α + Zβ(t)α) ∈ ℓ(t).

From (4.2), (5.2) and (5.3) we get:

(5.4) β′(t) = g^{−1} + β(t)Zg^{−1} + g^{−1}Zβ(t) + β(t)(Zg^{−1}Z − gR(t))β(t).

We call (5.4) the generalized Riccati equation.

Our goal is, given a fixed nondegenerate symmetric bilinear form g ∈ Bil_{sym}(IR^{n}), to determine conditions for a curve β on Bil_{sym}(IR^{n}^{∗}) to be the solution of (5.4) for some curve R : [a, b] → Lin(IR^{n}) of g-symmetric endomorphisms of IR^{n}. Observe that if β(t) is invertible for some t then R(t) must be given by:

(5.5) R(t) = g^{−1}[β(t)^{−1}(g^{−1} + β(t)Zg^{−1} + g^{−1}Zβ(t) − β′(t))β(t)^{−1} + Zg^{−1}Z];

obviously the righthand side of (5.5) is g-symmetric, so that we indeed get a curve R of g-symmetric endomorphisms of IR^{n} in every interval on which β is invertible. Observe that this is consistent with the results of Subsection 5.1. We now study the derivatives of β at those instants t where β(t) is degenerate. First, observe that (5.4) implies that:

(5.6) β′(t)(α_{1}, α_{2}) = (g^{−1} + β(t)Zg^{−1} + g^{−1}Zβ(t))(α_{1}, α_{2}),

if α_{1} ∈ Ker(β(t)) or if α_{2} ∈ Ker(β(t)).

It is easy to see that (5.6) is just the coordinate version of the condition that ℓ be tangent to D. By taking another derivative of (5.4), we obtain:

(5.7) β′′(t)(α_{1}, α_{2}) = (β′(t)Zg^{−1} + g^{−1}Zβ′(t))(α_{1}, α_{2}), α_{1}, α_{2} ∈ Ker(β(t));

finally, from (5.6) and (5.7) we obtain the following second order condition for β:

(5.8) β′′(t)(α_{1}, α_{2}) = 2(g^{−1}Zg^{−1} + g^{−1}Zβ(t)Zg^{−1})(α_{1}, α_{2}), α_{1}, α_{2} ∈ Ker(β(t)).

In Subsection 5.3 below we will show that conditions (5.6) and (5.8) are sufficient for β to arise from some R in a nondegenerate case.

5.3. The case where β′(t) is nondegenerate on Ker(β(t))

Consider a smooth curve β : [a, b] → Bil_{sym}(IR^{n}^{∗}). Assume that, for every t ∈ [a, b] for which β(t) is degenerate, the derivative β′(t) is nondegenerate on Ker(β(t)) and conditions (5.6) and (5.8) hold. We will prove that there exists a unique smooth curve R : [a, b] → Lin(IR^{n}) of g-symmetric endomorphisms of IR^{n} such that (5.4) holds. Then, if P ⊂ IR^{n} is a nondegenerate subspace for g, S ∈ Bil_{sym}(P) is a symmetric bilinear form, ℓ_{0} is defined as in (4.5) and β(a) = ϕ_{L_{0},L_{1}}(ℓ(a)), it follows that the curve of Lagrangians ℓ = ϕ_{L_{0},L_{1}}^{−1} ∘ β arises from (g, R, P, S), where L_{1} is defined in (5.1).

In order to achieve our goal, we start by observing that Lemma 3.7 implies that there are only a finite number of instants t ∈ [a, b] with β(t) degenerate. Formula (5.5) can be used to define R at those t where β(t) is invertible. We now fix t_{0} ∈ [a, b] with β(t_{0}) degenerate and we show that R admits a smooth extension to t_{0}.

Let (b_{i})_{i=1}^{n} be a basis of IR^{n}^{∗} such that (b_{i})_{i=1}^{k} is a basis of Ker(β(t_{0})); the restriction of β(t_{0}) to the span of (b_{i})_{i=k+1}^{n} is clearly nondegenerate. We will think of the bilinear forms on IR^{n}^{∗} as matrices relative to the basis (b_{i})_{i=1}^{n}. Observe that β(t_{0}) has zeroes outside the lower-right (n − k) × (n − k) block.

For t ≠ t_{0}, let β̃(t) be the matrix obtained by dividing the first k columns of β(t) by t − t_{0}; we define β̃(t_{0}) by replacing the first k columns of β(t_{0}) with the first k columns of β′(t_{0}). It is easy to see that t ↦ β̃(t) is smooth around t_{0}. For a ∈ IR, we set D_{k}(a) = ( \begin{matrix}aI_{k} & 0 \\ 0 & I_{n−k} \end{matrix} ), where I_{j} denotes the j × j identity matrix. Then β(t) = β̃(t)D_{k}(t − t_{0}) =

D_{k}(t − t_{0})β̃(t)^{∗}, for all t ≠ t_{0}. Hence:

(5.9)  β(t)^{−1} = D_{k}(\frac{1}{t−t_{0}} )β̃(t)^{−1} = (β̃(t)^{−1})^{∗}D_{k}(\frac{1}{t−t_{0}} ),

for all t ≠ t_{0}. Since β′(t_{0}) is nondegenerate on Ker(β(t_{0})), the matrix β̃(t_{0}) is invertible and so t ↦ β̃(t)^{−1} is smooth around t_{0}. From (5.5), in order to prove that R has a smooth extension to t_{0}, it suffices to show that the map t ↦ β(t)^{−1}Q(t)β(t)^{−1} has a smooth extension to t_{0}, where Q is defined by:

(5.10)  Q(t) = g^{−1} + β(t)Zg^{−1} + g^{−1}Zβ(t) − β′(t).

We claim that Q(t_{0}) is zero outside the lower-right (n − k) × (n − k)-block and that Q′(t_{0}) is zero on the upper-left k ×k-block. Namely, it is easy to check that (5.6)and (5.8) imply (5.7). Then (5.7) and (5.10) imply that Q(t_{0})(α_{1}, α_{2}) = 0 if either α_{1} or α_{2} is in Ker(β(t_{0})) and that Q′(t_{0})(α_{1}, α_{2}) = 0 if both α_{1} and α_{2} are in Ker(β(t_{0})). This proves the claim. We can now write:

(5.11)  Q(t) = D_{k}(t − t_{0})Q̃(t)D_{k}(t − t_{0}),

with t ↦ Q̃(t) smooth around t_{0}. From (5.9), (5.11) and the fact that t ↦ β̃(t)^{−1} is smooth around t_{0}, it follows that t ↦ β(t)^{−1}Q(t)β(t)^{−1} (and thus t ↦ R(t)) is smooth around t_{0}.

5.4. A counterexample to the equality i_{Maslov}(g, R, P, S) = i_{foc}(g, R, P, S)

In Subsection 5.3 we have shown that, if β′(t) is nondegenerate on Ker(β(t)), then conditions (5.6) and (5.8) are sufficient for the existence of a smooth map R for which β is a solution of (5.4). It will become clear below that the hypothesis that β′(t) be nondegenerate on Ker(β(t)) is indeed essential here; namely, if β′(t_{0}) is degenerate on Ker(β(t_{0})) for some t_{0} then one has to impose conditions on the higher order derivatives of β at t_{0} in order to guarantee that the map t ↦ R(t) defined by (5.5) has a smooth extension to t_{0}.

We consider the following setup. Set n = 2 and define the objects g, Z and P by g = ( \begin{matrix}0 & 1 \\ 1 & 0 \end{matrix} ), Z = ( \begin{matrix}0 & 0 \\ 0 & 1 \end{matrix} ), P = \{0\}. We will describe a curve β : [a, b] → Bil_{sym}(IR^{2}^{∗}),

(5.12)  β(t) = (\begin{matrix}x(t) & z(t) \\ z(t) & y(t) \end{matrix}) ,

where x, y and z are real-analytic scalar functions on [a, b], a < 0 < b, such that the following properties are satisfied:

(1) x(a) = y(a) = z(a) = 0;

(2) x′(a) = y′(a) = 0, z′(a) = 1, x′′(a) = 2 and y′′(a) = z′′(a) = 0;

(3) x(t)y(t) − z(t)^{2} vanishes precisely for t = a and for t = 0;

(4) x(0) = z(0) = 0 and y(0) = −1;

(5) x′(0) = x′′(0) = 0 and z′(0) = 0;

(6) x′′′(0) < 0;

(7) the map t ↦ R(t) defined by (5.5)has an extension to t = 0 which is real-analytic around t = 0.

Once a curve β satisfying properties (1)—(7) above is determined, the desired counterexample can be easily constructed. Namely, define R by (5.5). From condition (3), R is real-analytic on [a, b] \ {a, 0}; condition (7) says that R has a real-analytic extension to 0. A simple computation shows that, under condition (1), condition (2) is equivalent to (5.6) and (5.8) at t = a. Thus, the result proved in Subsection 5.3 guarantees that R has a realanalytic extension to t = a. Now define L_{1} by (5.1), L_{0} by (4.6) and set ℓ = ϕ_{L_{0},L_{1}}^{−1} ∘ β. Condition (1) implies that ℓ(a) = L_{0} and therefore ℓ is the curve of Lagrangians corresponding to the quadruple (g, R, P, S). By condition (3), t = 0 is the unique (P, S)-focal instant; condition (4) implies that the kernel of β(0) is spanned by the first vector of the canonical basis of IR^{2}^{∗} and thus the signature of the (P, S)-focal instant t = 0 is zero (recall (3.4) and (4.8)). Hence i_{foc}(g, R, P, S) = 0. On the other hand:

(5.13)  i_{Maslov}(g, R, P, S) = −1;

namely, recall from Proposition 3.9 that i_{Maslov}(g, R, P, S) = μ_{L_{0}}(ℓ|_{[a+ε,b]}) equals the variation of the coindex of β(t) near t = 0. Conditions (4), (5), (6) imply n_{+}(β(t)) = 1 for t < 0 near zero and n_{+}(β(t)) = 0 for t > 0 near zero; this yields (5.13).

We will now show that it is possible to determine polynomial functions x, y and z satisfying conditions (1)—(7). We start by analyzing condition (7). As in Subsection 5.3, t ↦ R(t) has a real-analytic extension to t = 0 if and only if t ↦ β(t)^{−1}Q(t)β^{−1}Q(t) does, where Q is given by (5.10). Consider the matrix β̂(t) = ( \begin{matrix}y(t) & −z(t) \\ −z(t) & x(t) \end{matrix} ), so that

β(t)^{−1}Q(t)β(t)^{−1} = _{\frac{1}{det (β(t))^{2}}} β̂(t)Q(t)β̂(t), when β(t) is invertible. We have:

β̂(t)Q(t)β̂(t) = (\begin{matrix}p_{1}(t) & p_{2}(t) \\ p_{2}(t) & p_{3}(t) \end{matrix}) ,

where p_{1} = −x′y^{2} − 2yz + 2yzz′ − y′z^{2}, p_{2} = x′yz − yz^{2} + xy − xyz′ + xy^{2} + z^{2} − z^{2}z′ + xy′z and p_{3} = −x′z^{2} + 2z^{3} − 2xz + 2xzz′ − 2xyz − x^{2}y′. Conditions (4), (5) and (6) imply that t ↦ det (β(t)) has a zero of order 3 at t = 0; thus condition (7) is equivalent to the maps p_{i}(t), i = 1, 2, 3, having zeroes of order at least 6 at t = 0. This can be achieved, for instance, by setting:

(5.14)  x(t) = −2t^{3} − \frac{54}{5}t^{5}, y(t) = −1 − 6t + 18t^{2} − 54t^{3}, z(t) = −3t^{2},

for t near zero. Observe that the polynomials above also satisfy conditions (4)—(6). For t near a, we take x, y, z to be arbitrary smooth maps satisfying (1) and (2). It is easy to see that n_{+}(β(t)) = 1 for t > a near a and for t < 0 near zero. Since the set of invertible symmetric matrices of coindex 1 is connected and open in Bil_{sym}(IR^{2}^{∗}), we may then extend x, y, z smoothly to [a, 0] in such a way that β(t) is invertible for all t ∈ ]a, 0[; we then extend x, y, z smoothly to a slightly larger interval [a, b], b > 0, so that condition (3) is satisfied.

The construction above yields the desired counterexample in the smooth case. We now use a density argument to show the existence of a real-analytic counterexample.

Let x, y and z be given smooth functions such that conditions (1)—(7) are satisfied; we start by observing that if x̃, ỹ and z̃ are smooth functions having the first six derivatives at t = 0 and the first two derivatives at t = a equal to the corresponding derivatives of x, y and z, then, by replacing x, y and z with x̃, ỹ and z̃, only condition (3) may fail to hold. If such a replacement is made in such a way that x̃, ỹ and z̃ are sufficiently close to x, y and z in the C^{4}-topology, then also condition (3) will remain true. To prove this, we apply the following elementary lemma to the function f(t) = x(t)y(t) − z(t)^{2} on the interval [a, 0] with k = 3:

Lemma 5.1. Let k ∈ IN and let f : [c, d] → IR be a function of class C^{k+1}. Assume that f has zeroes precisely at the endpoints c, d and that these zeroes are of order at the most k. Then, there exists a neighborhood U of f in the C^{k+1}-topology such that, every g ∈ U having the same order of zeroes as f at c and d has no zeroes in ]c, d[. □

The proof of the existence of polynomial functions x, y, z : [a, b] → IR satisfying conditions (1)—(7) now follows by applying the elementary density lemma below in the interval [a, 0], with k = 6.

Lemma 5.2. Let k ∈ IN and p_{i}, q_{i} ∈ IR, i = 0, …, k, be fixed. Let A denote the affine subspace of C^{k}([a, b], IR) consisting of maps f with f^{(i)}(c) = p_{i}, f^{(i)}(d) = q_{i}, i = 0, …, k. Then the subset of A consisting of polynomials maps is dense in A with respect to the C^{k}-topology. □

References

[1] V. I. Arnol’d, Characteristic Class Entering in Quantization Conditions, Funct. Anal. Appl. 1 (1967), 1—13.

[2] J. Beem, P. Ehrlich, A Morse Index Theorem for Null Geodesics, Duke Math. J. 46 (1979), 561—569.

[3] J. K. Beem, P. E. Ehrlich, K. L. Easley, Global Lorentzian Geometry, Marcel Dekker, Inc., New York and Basel, 1996.

[4] R. Bott, On the Iteration of Closed Geodesics and the Sturm Intersection Theory, Commun. Pure Appl. Math. 9 (1956), 171—206.

[5] E. A. Coddington, N. Levinson, Theory of Ordinary Differential Equations, McGraw–Hill Book Company, New York, Toronto, London, 1955.

[6] J. C. Corrêa Eidam, A. L. Pereira, P. Piccione, D. V. Tausk, On the Equality between the Maslov Index and the Spectral Index for the semi-Riemannian Jacobi Operator, to appear in J. Math. Anal. Appl.

[7] M. do Carmo, Riemannian Geometry, Birkhäuser, Boston, 1992.

[8] H. M. Edwards, A Generalized Sturm Theorem, Ann. of Math. 80 (1964), 22—57.

[9] P. E. Ehrlich, S. Kim, A Focal Index Theorem for Null Geodesics, J. Geom. Phys. 6, n. 4 (1989), 657—670.

[10] D. B. Fuks, Maslov–Arnol’d Characteristic Classes, Soviet Math. Dokl. 9, n. 1 (1968), 96—99.

[11] F. Giannoni, A. Masiello, P. Piccione, A Morse Theory for Massive Particles and Photons by Fermat Principles in General Relativity, Journal of Geometry and Physics 35, vol. 1 (2000), 1–35.

[12] F. Giannoni, A. Masiello, P. Piccione, D. Tausk, A Generalized Index Theorem for Morse–Sturm Systems and Applications to semi-Riemannian Geometry, Asian Journal of Mathematics Vol. 5, no. 3 (2001).

[13] V. Guillemin, S. Sternberg, Geometric Asymptotics, Mathematical Surveys and Monographs n. 14, AMS, Providence RI, 1990.

[14] A. D. Helfer, Conjugate Points on Spacelike Geodesics or Pseudo-Self-Adjoint Morse-Sturm-Liouville Systems, Pacific J. Math. 164, n. 2 (1994), 321—340.

[15] B. O’Neill, Semi-Riemannian Geometry with Applications to Relativity, Academic Press, New York, 1983.

[16] P. Piccione, D. V. Tausk, A Note on the Morse Index Theorem for Geodesics between Submanifolds in semi-Riemannian Geometry, J. Math. Phys. 40, vol. 12 (1999), 6682—6688.

[17] P. Piccione, D. V. Tausk, An Index Theorem for Non Periodic Solutions of Hamiltonian Systems, Proceedings of the London Mathematical Society Society (3) 83 (2001), 351–389.

[18] F. Treves, Introduction to Pseudodifferential Operators, Plenum, New York, 1982.

[19] V. S. Varadarajan, Lie Groups, Lie Algebras and Their Representations, Prentice-Hall series in Modern Analysis, 1974, New Jersey.

Departamento de matemática, Departamento de Matemática Universidade Estadual de Campinas, SP, Brazil Universidade de São Paulo, Brazil

   E-mail address: mercuri@ime.unicam.it            piccione,tausk   @ime.usp.br