Zero-sum state constrained differential games: Existence of value for Bolza problem^{∗}

Bettiol Piernicola^{†}, Cardaliaguet Pierre ^{‡}& Quincampoix Marc^{§}

November 5, 2004

Abstract

We prove the existence of a lower semicontinuous value function for Bolza problem in differential games with state-constraints. As a byproduct, we obtain a new estimation of trajectories of a control system by trajectories with state constraints. This result which could be interesting by itself enables us to build a suitable strategy for constrained differential games. We also characterize the value function by means of viscosity solutions and give conditions under which the value function is locally Lipschitz continuous.

Key words: differential games, Bolza problem, state constraints, Viability Theory.

AMS Classification: 49N70, 90D25.

Introduction

Our aim is to prove that Bolza problem for two-players differential games with state-constraints admits a value.

In the differential game we treat, the first player, playing with u, controls a first system

(1)  \{ \begin{matrix}y′(t) & = g(y(t), u(t)), u(t) ∈ U \\ y(t_{0}) & = y_{0} ∈ K_{U} \end{matrix}

∗Work supported by the European Community’s Human Potential Program under contract HPRN-CT-2002-00281, [Evolution Equations].

†SISSA/ISAS via Beirut, 2-4 - 34013 Trieste Italy, bettiol@sissa.it

‡Département de Mathématiques, Université de Bretagne Occidentale, 6 Avenue Victor Le Gorgeu F-29200 Brest France, Pierre.Cardaliaguet@univ-brest.fr

§Département de Mathématiques, Université de Bretagne Occidentale, 6 Avenue Victor Le Gorgeu F-29200 Brest France, Marc.Quincampoix@univ-brest.fr

where t_{0} ∈ [0, T] and has to ensure the state constraint y(t) ∈ K_{U} to be fulfilled for any t ∈ [t_{0}, T]. On the other hand the second player, playing with v, controls a second system

(2)  \{ \begin{matrix}z′(t) & = h(z(t), v(t)), v(t) ∈ V \\ z(t_{0}) & = z_{0} ∈ K_{V} \end{matrix}

where t_{0} ∈ [0, T] and has to ensure the state constraint z(t) ∈ K_{V} for any t ∈ [t_{0}, T]. Associated with any initial data (t_{0}, x_{0}) = (t_{0}, y_{0}, z_{0}) and with any couple of controls (u(·), v(·)) we consider the following payoff functional:

(3)  J(t_{0}, x_{0}; u(·), v(·)) = ∫_{t_{0}}^{T} L(t, x(t), u(·), v(·)) dt + Ψ(x(T)),

where x(t) = x[t_{0}, x_{0}; u(·), v(·)](t) = (y[t_{0}, y_{0}; u(·)](t), z[t_{0}, z_{0}; v(·)](t)) denotes the solution of the systems (1) and (2). The function L is called running cost and Ψ is the final cost. The first player wants to maximize the functional J, while the second player’s goal is to minimize J.

As is usual in differential game theory, one can define two value functions for the game, the upper one v^{♯} and the lower one v^{♭} (see the definitions (7) and (8) below). Here the value functions are defined through nonanticipative strategies (or Varayia-Roxin-Elliot-Kalton strategies, cf. [12] or preliminaries below).

The purpose of this paper is to prove that this game admits a value, i.e. to obtain the following equality:

∀(t, x) ∈ [0, T] × K_{U} × K_{V} v^{♭}(t, x) = v^{♯}(t, x).

Related works and different approaches to the problem can be found in [1, 5, 7, 21], see also the discussion in [12]. Here we follow closely the techniques developed in [12, 11, 13], which consider the problem of existence of a value for pursuit-evasion games. The main idea amounts to reduce the qualitative game (here the Bolza problem) to a quantitative one, for which we prove an alternative result (see Proposition 2.3). This in turns gives the existence and a characterization of the value. Although this paper is strongly inspired by [12], there are important differences, that we explain now.

As is usual in state-constraint problems, the main difficulty comes from the fact that the players have to use admissible controls and strategies, and in particular that the set of controls allowed to a player strongly depends on its position. To overcome this difficulty, one has to be able, for two fixed initial positions of a player, to approximate a given admissible control at one position by some admissible control at the other position. This problem has been successfully handled for control problems by several authors, under various assumptions in [2, 16, 19]. However, in the case of differential games, it is very important to built the approximating control in a nonanticipative way. Unfortunately, the constructions of the above quoted papers are all anticipative. We note also that this problem was overlooked in [12], and that, in particular, the proof of Lemma 4.4 of [12] was not completely correct because of this difficulty. The main technical result of this paper (Proposition 3.1) is the construction of such nonanticipative approximating control.

Another new aspect of the present paper is the regularity property and the characterization of the value function. We prove, under suitable regularity conditions on the running cost and the final cost, that the value function is Lipschitz continuous and that it is the unique viscosity solution of some Hamilton-Jacobi equation with a discontinuous Hamiltonian.

The paper is organized in the following way. Section 1 is devoted to the statement of the main result (existence of a value) and to its proof, based on the characterization of the value functions in terms of the so-called discriminating kernel, which is a victory domain in the qualitative game. This characterization is based on a result usually called “alternative” (Proposition 2.3), which is proved in section 4. As explained above, such a proof requires a technical tool allowing to compare the sets of admissible controls at different points. This is the aim of section 3. Finally, section 5 is devoted to the regularity and characterization of the value function as a viscosity solutions of some Hamilton-Jacobi equation.

1 Preliminaries

1.1 Notations and assumptions

We first introduce some notations. Throughout this paper, | · | denotes the euclidean norm of IR^{N}. If K is a subset of IR^{N}, d_{K}(x) denotes the distance of x to K, i.e., d_{K}(x) = inf_{y∈K} |y − x|. We also denote by B_{N} the closed unit ball of IR^{N} (and we write simply B when the dimension of the ball is understood). If K is a subset of IR^{N} and r > 0, we denote by K + rB the set of points x ∈ IR^{N} such that d_{K}(x) ≤ r.

In the following lines we summarize all the assumptions concerning with the vector fields of the dynamics:

(4)  \{ \begin{matrix}(i) & U and V are compact subsets of some finite \\ & dimensional spaces \\ (ii) & f : IR^{n} × U × V → IR^{n} is continuous and \\ & Lipschitz continuous (with Lipschitz constant M) \\ & with respect to x ∈ IR^{n}; \\ (iii) & ⋃_{u} f(x, u, v) and ⋃_{v} f(x, u, v) are convex for any x. \\ (iv) & K_{U} = \{y ∈ IR^{l}, ϕ_{U}(y) ≤ 0\} with ϕ_{U} ∈ C^{2}(IR^{l}; IR), \\ & ∇ϕ_{U}(y) ≠ 0 if ϕ_{U}(y) = 0 \\ (v) & K_{V} = \{z ∈ IR^{m}, ϕ_{V} (z) ≤ 0\} with ϕ_{V} ∈ C^{2}(IR^{m}; IR), \\ & ∇ϕ_{V} (z) ≠ 0 if ϕ_{V} (z) = 0 \\ (vi) & ∀y ∈ ∂K_{U}, ∃u ∈ U such that < ∇ϕ_{U}(y), g(y, u) > < 0 \\ (vii) & ∀z ∈ ∂K_{V} , ∃v ∈ V such that < ∇ϕ_{V} (z), h(z, v) > < 0 \end{matrix}

For any x = (y, z) ∈ IR^{n}, we set

U(y) := {u ∈ U| g(y, u) ∈ T_{K_{U}} (y)}

where T_{K_{U}} (y) is the tangent half-space to the set K_{U}. Notice that under assumptions (4) the set-valued map y ⇝ g(y, U(y)) is lower semicontinuous with convex compact values (for definitions and properties, see [4]).

For any starting point x_{0} = (y_{0}, z_{0}) ∈ K_{U} × K_{V}, for any initial time t_{0} ∈ [0, T] and for any measurable controls (u(·), v(·)) : [t_{0}, T] → U × V, we denote by x[t_{0}, x_{0}; u(·), v(·)](t) = (y[t_{0}, y_{0}; u(·)](t), z[t_{0}, z_{0}; v(·)](t)) the solution of system (1-2).

The first player, controlling u(·), has to ensure that y(t) ∈ K_{U} for any t ≥ 0, while the second player, by using v(·), has to ensure the state constraint z(t) ∈ K_{V} for any t ≥ 0.

We need to introduce the notions of admissible controls: ∀y_{0} ∈ K_{U}, ∀z_{0} ∈ K_{V} and ∀t_{0} ∈ [0, T] we define

U(t_{0}, y_{0}) := {u(·) : [t_{0}, +∞) → U measurable | y[t_{0}, y_{0}; u(·)](t) ∈ K_{U} ∀t ≥ t_{0}};
V(t_{0}, z_{0}) := {v(·) : [t_{0}, +∞) → V measurable | z[t_{0}, z_{0}; v(·)](t) ∈ K_{V} ∀t ≥ t_{0}} .

Under the assumptions (4), it is well known that ∀x_{0} = (y_{0}, z_{0}) ∈ K_{U} × K_{V}

U(t_{0}, y_{0}) ≠ ∅ and V(t_{0}, z_{0}) ≠ ∅.

Both players play non-anticipative strategies. A map α : V(t_{0}, z_{0}) → U(t_{0}, y_{0}) is a non-anticipative strategy (for the first player Ursula and for the point (t_{0}, x_{0}) := (t_{0}, y_{0}, z_{0}) ∈ IR^{+}×K_{U} ×K_{V}) if, for any τ > 0, for all controls v_{1}(·) and v_{2}(·) belonging to V(t_{0}, z_{0}), which coincide a.e. on [t_{0}, t_{0} + τ], α(v_{1}(·)) and α(v_{2}(·)) coincide almost everywhere on [t_{0}, t_{0} + τ]. The non-anticipative strategies β for the second player Victor are symmetrically defined. For any point x_{0} ∈ K_{U} × K_{V} and ∀t_{0} ∈ [0, T] we denote by S_{U}(t_{0}, x_{0}) and by S_{V} (t_{0}, x_{0}) the sets of the non-anticipative strategies for Ursula and Victor respectively.

We consider costs with the following assumptions:

(5)  \{ \begin{matrix}i) & L : [0, T] × IR^{n} × U × V ⟶ IR is a bounded and Lipschitz \\ & continuous with respect to all the variables of constant M; \\ ii) & Ψ : IR^{n} ⟶ IR is bounded and lower semicontinuous. \end{matrix}

We also have to assume some structure condition on g, h and L; namely,

(6) one of the following conditions holds:

condition 1- L := L(t, x) (L does not depend on u and v)
condition 2- L := L(t, x, u, v) = L_{0}(t, x) + L_{1}(t, x)u + L_{2}(t, x)v,

g(y, u) = g_{1}(y)u + g_{2}(y), h(z, v) = h_{1}(z)v + h_{2}(z) where g_{1}(y) and h_{1}(z) are invertible bounded matrices with inverse Lipschitz continuous w.r. to x.

We are now ready to define the value functions of the game. The lower value v^{♭} is defined by:

(7)  v^{♭}(t_{0}, x_{0}) := inf_{β∈S_{V} (t_{0},x_{0})} sup_{u(·)∈U(t_{0},y_{0})} J(t_{0}, x_{0}; u(·), β(u(·))) ,

where J is defined by (3). On the other hand we define the upper value function as follows:

(8)  v^{♯}(t_{0}, x_{0}) := lim_{ε→0^{+}} sup_{α∈S_{U} (t_{0},x_{0})} inf_{v(·)∈V(t_{0},z_{0})} J_{ε}(t_{0}, x_{0}; α(v(·)), v(·))

where

J_{ε}(t_{0}, x_{0}; u(·), v(·)) := ∫_{t_{0}}^{T} L(t, x[t_{0}, x_{0}; u(·), v(·)](t)) dt + Ψ_{ε}(x[t_{0}, x_{0}; u(·), v(·)](T)),

and Ψ_{ε} is the lower semicontinuous function defined by

Ψ_{ε}(x) = inf {ρ ∈ IR | ∃y ∈ IR^{n} with |(y, ρ) − (x, Ψ(x))| = ε} .

Remark 1.1 When the function Ψ is continuous then it turns out that the upper value function is also given by:

(9)  v^{♯}(t_{0}, x_{0}) = sup_{α∈S_{U} (t_{0},x_{0})} inf_{v(·)∈V(t_{0},z_{0})} J(t_{0}, x_{0}; α(v(·)), v(·)).

However when Ψ is not continuous, we cannot expect that the game has a value when v^{♯} is defined by the above formula (see for instance the counterexample given in [12]). The correct formulation for v^{♯} is (8).

Proof of Remark 1.1. Let v^{♯} be the value function given by (8) and w the function given by (9). Fix (t_{0}, x_{0}) and ε′ ∈ (0, 1). From (4), there exists M_{1} > 0 such that for each control u(·) and v(·), x[t_{0}, x_{0}; u(·), v(·)]([t_{0}, T]) ⊂ M_{1}B_{n}. Since Ψ is uniformly continuous on (M + 1)B_{n}, there exists ε > 0 such that

∀x ∈ (M + 1)B, ∀a ∈ B, |Ψ(x + εa) − Ψ(x)| ≤ ε′.

Thus

Ψ(x) − 2ε′ ≤ inf_{a∈B_{n}} Ψ(x + εa) − ε′ ≤
≤ inf_{(a,b)∈B_{n+1}} Ψ(x + εa) + εb ≤
≤ Ψ_{ε}(x)

Consequently, for any control u(·) and v(·), we have

|J(t_{0}, x_{0}; α(v(·)), v(·)) − J_{ε}(t_{0}, x_{0}; α(v(·)), v(·))| ≤ 2ε′

which entails that |v^{♯}(t_{0}, x_{0}) − w(t_{0}, x_{0})| ≤ 2ε′. Whence the desired result, ε′ being arbitrary.

1.2 Discriminating domains and kernels

We now introduce the notion of discriminating domain and kernel for a Hamiltonian function H (the original definition is given in [3], cf. also [11]). This notion is fundamental in our approach because we will prove that the epigraphs of the values are discriminating kernels of suitably defined Hamiltonians.

Definition 1.2 A closed subset D of IR^{N} is a discriminating domain for H : IR^{N} × IR^{N} → IR if and only if

∀x ∈ D, ∀p ∈ NP_{D}(x), H(x; p) ≤ 0,

where NP_{D}(x) denotes the set of proximal normal to D at the point x, i.e., the set of p ∈ IR^{N} such that the distance of x + p to D is equal to |p|.

Note that, if K is the closure of an open set with C^{2} boundary, then the above definition reduces to the condition:

∀x ∈ ∂K, H(x; ν_{x}) ≤ 0 ,

where ν_{x} denotes the outward unit normal to K at x. Next we introduce the notion of discriminating kernel: For a given closed set K—which is not a discriminating domain in general—it is possible to define a largest discriminating domain contained in K. This is the discriminating kernel of K.

Proposition 1.3 ([10]) Let H : IR^{N} × IR^{N} → IR be a lower semicontinuous map. If K is a subset of IR^{N}, then K contains a largest (for the inclusion) closed discriminating domain for H. This set, denoted by Disc_{H}(K), is called the discriminating kernel of K for H.

2 Value functions

In this section we state the main result of this paper, namely the existence of a value for our game. For this we have to consider separately the case of a running cost independent of the control, i.e., L = L(t, x), and the more difficult situation where L = L(t, x, u, v).

Theorem 2.1 Assume that (4,5,6) hold. Then the game has a value, and this value is lower semi-continuous:

∀(t, x) ∈ [0, T] × K_{U} × K_{V} v^{♭}(t, x) = v^{♯}(t, x).

The proof of this result is obtained by identifying the epigraph of the value function as the discriminating kernel of some closed set for a suitable dynamics. We now explain this identification.

Let us define an “extended” dynamics function

(10)  f̃((t, x, ρ), u, v) := {1} × {f(x, u, v)} × {−L(t, x, u, v)}

and a corresponding Hamiltonian:

(11) H̃(t, x, ρ; p_{t}, p_{x}, p_{ρ}) := \{ \begin{matrix}sup_{u∈U(y)} inf_{v∈V} ⟨f̃, p⟩ & if (t, x, ρ) ∉ E \\ min \{0; sup_{u∈U(y)} inf_{v∈V} ⟨f̃, p⟩\} & if (t, x, ρ) ∈ E, \end{matrix}

where

E := {(t, x, ρ) ∈ [0, T] × K_{U} × K_{V} × IR| t = T, ρ ≥ Ψ(x)} = {T } × EpiΨ.

We also define the set K := [0, T] × K_{U} × K_{V} × IR ⊂ IR^{n+2}. Writing explicitly the scalar product term, we get

⟨f̃, p⟩ = p_{t} + ⟨f(x, u, v), p_{x}⟩ − p_{ρ}L(t, x, u, v) ∀p = (p_{t}, p_{x}, p_{ρ}) ∈ IR^{n+2}.

Hereafter, in order to simplify the notations, we will write X for the triple (t, x, ρ), i.e. X = (t, x, ρ) ∈ IR^{N} where N := n + 2. In particular, once X_{0} = (t_{0}, x_{0}, ρ_{0}) and (u(·), v(·)) ∈ U(t_{0}, y_{0}) × V(t_{0}, z_{0}) are fixed, we denote by X[X_{0}; u(·), v(·)](s) the solution of the Cauchy problem

(12)  X′(s) = f̃(X(s), u(s), v(s)), X(t_{0}) = X_{0}.

Let us point out that the t-component of X, denoted t(s) is always equal to s.

Theorem 2.2 Let H̃ be defined as in (11) and assume (4,5,6) hold. Then

Epi(v^{♭}) = DiscH̃(K)  and  Epi(v^{♯}) = DiscH̃(K).

From Theorem 2.2 we can easily deduce the proof of Theorem 2.1.

Proof of Theorem 2.1: Since the epigraph of v^{♭}(t, x) and v^{♯}(t, x) coincide, both functions are equal. Furthermore, since the discriminating kernel is a closed set, their epigraph is closed, and therefore they are lower semicontinuous.

We now turn to the proof of Theorem 2.2, which is crucially related to the following interpretation result of discriminating kernels:

Proposition 2.3 Suppose that assumptions of Theorem 2.2 hold true. Then the Discriminating Kernel DiscH̃(K) can be characterized as follows

i) DiscH̃(K) is the set of all the starting points X_{0} = (t_{0}, x_{0}, ρ_{0}) ∈ [0, T]×K_{U} ×K_{V} ×IR for which there exists an admissible nonanticipative strategy β : U(t_{0}, y_{0}) ⟶ V(t_{0}, z_{0}) such that, for any admissible control u(·) ∈ U(t_{0}, y_{0}), the solution X[X_{0}; u(·), β(u(·))] remains in K as long as it does not reach E; ii) The set ([0, T] × K_{U} × K_{V} × IR) \ DiscH̃(K) is the set of all the starting points X_{0} = (t_{0}, x_{0}, ρ_{0}) ∈ [0, T] × K_{U} × K_{V} × IR for which there exists ε > 0 and an admissible nonanticipative strategy α : V(t_{0}, z_{0}) ⟶ U(t_{0}, y_{0}) such that, for any admissible control v(·) ∈ V(t_{0}, z_{0}), the solution X[X_{0}; α(v(·)), v(·)](s) avoids E +εB for any time s.

This result can be viewed as an alternative Theorem [9, 18]. After some estimation results in section 3, the fourth section will be devoted to the proof of Proposition 2.3.

Proof of Theorem 2.2. Let us first check that Epi(v^{♭}) = DiscH̃(K). By i) of Proposition 2.3 we get

DiscH̃(K) ={(t_{0}, x_{0}, ρ_{0}) ∈ K for which ∃β ∈ S_{V} (t_{0}, x_{0}) s. t. ∀u(·) ∈ U(t_{0}, y_{0})

the trajectory X[X_{0}, u(·), β(u(·))](·) remains in K until it reaches E}.

Suppose that (t_{0}, x_{0}, ρ_{0}) ∈ DiscH̃(K). By the above characterization of DiscH̃(K) we can choose a strategy β ∈ S_{V} (t_{0}, x_{0}) such that, for any u(·) ∈ U(t_{0}, y_{0}), the solution X(s) = (t(s), x(s), ρ(s)) of the system

(13)  \{ \begin{matrix}t′(s) & = 1 \\ x′(s) & = f(x(s), u(s), β(u(·))(s)), \\ ρ′(s) & = −L(s, x(s), u(s), β(u(·))(s)), \\ t(t_{0}) & = t_{0}, x(t_{0}) = x_{0}, ρ(t_{0}) = ρ_{0} \end{matrix}

satisfies

ρ(T) = ρ_{0} − ∫_{t_{0}}^{T} L(r, x(r), u(r), β(u(·))(r))dr ≥ Ψ(x(T)).

Hence, we have

v^{♭}(t_{0}, x_{0}) ≤ sup_{u(·)∈U(t_{0},y_{0})} J(t_{0}, x_{0}; u(·), β(u(·))) ≤ ρ_{0}

and so (t_{0}, x_{0}, ρ_{0}) ∈ Epi(v^{♭}).

On the other hand suppose that the point (t_{0}, x_{0}) belongs to the domain of v^{♭}, and let ρ_{0} > v^{♭}(t_{0}, x_{0}). There exists a nonanticipative strategy β ∈ S_{V} (t_{0}, x_{0}) such that

v^{♭}(t_{0}, x_{0}) ≤ sup_{u(·)∈U(t_{0},y_{0})} J(t_{0}, x_{0}; u(·), β(u(·))) ≤ ρ_{0} .

Hence for any u(·) ∈ U(t_{0}, y_{0}) we have

v^{♭}(t_{0}, x_{0}) ≤ ∫_{t_{0}}^{T} L(r, x(r), u(r), β(u(·))(r)) dr + Ψ(x(T)) ≤ ρ_{0} ,

and the ρ-component of X[X_{0}, u(·), β(u(·))](·)—denoted by ρ(·)—satisfies

ρ(T) = ρ_{0} − ∫_{t_{0}}^{T} L(r, x(r), u(r), β(u(·))(r)) dr ≥ Ψ(x(T)).

Therefore the trajectory X[X_{0}, u(·), β(u(·))](·) remains in K until it reaches E. Since this holds true for all u(·) ∈ U(t_{0}, y_{0}), thanks to i) of Proposition 2.3, we have (t_{0}, x_{0}, ρ_{0}) ∈ DiscH̃(K) for any ρ_{0} > v^{♭}(t_{0}, x_{0}) . DiscH̃(K) being closed, we obtain

Epi(v^{♭}) ⊂ DiscH̃(K).

Now, we prove that DiscH̃(K) = Epi(v^{♯}). Suppose that X_{0} = (t_{0}, x_{0}, ρ_{0}) ∈ DiscH̃(K). Then, by ii) of Proposition 2.3, for any ε > 0 and for any nonanticipative strategy α(·) ∈ S_{U}(t_{0}, x_{0}) there exists a control of Victor v(·) ∈ V(t_{0}, z_{0}) such that for some τ ∈ [t_{0}, T] X(τ) = X[X_{0}, α(v(·))(·), v(·)](τ) ∈ E + εB. Therefore we get the following two facts: T − τ ≤ ε and (x(τ), ρ(τ)) ∈ Epi(Ψ_{ε}). On the other hand we have: |ρ(T) − ρ(τ)| ≤ M_{L}ε, |x(T) − x(τ)| ≤ M_{f}ε where M_{L} and M_{f} are the upper bounds of the function L and f respectively. So, since d_{Epi(Ψ)}(x(τ), ρ(τ)) ≤ ε, we obtain:

\begin{aligned}d_{Epi(Ψ)}((x(T), ρ(T))) & ≤ d_{Epi(Ψ)}((x(τ), ρ(τ))) + |x(T) − x(τ)| + |ρ(T) − ρ(τ)| ≤ \\ & ≤ (M_{L} + M_{f} + 1)ε \end{aligned}

which implies, calling ε′ := (M_{L} + M_{f} + 1)ε,

ρ(T) ≥ Ψ_{ε′}(x(T)) .

Hence, we obtain

(14)  \begin{aligned}ρ_{0} & = ∫_{t_{0}}^{T} L(s, x(s), α(v(·))(s), v(s)) ds + ρ(T) ≥ \\ & ≥ ∫_{t_{0}}^{T} L(s, x(s), α(v(·))(s), v(s)) ds + Ψ_{ε′}(x(T)) ≥ \\ & ≥ J_{ε′}(t_{0}, x_{0}, α(v(·)), v(·)) ≥ \\ & ≥ inf_{v(·)∈V(t_{0},z_{0})} J_{ε′}(t_{0}, x_{0}, α(v(·)), v(·)). \end{aligned}

Recalling that (14) holds true for any α(·) ∈ S_{U}(t_{0}, x_{0}), we get

ρ_{0} ≥ sup_{α(·)∈S_{U} (t_{0},x_{0})} inf_{v(·)∈V(t_{0},z_{0})} J_{ε′}(t_{0}, x_{0}, α(v(·)), v(·))

Since the right hand side of the inequality above is bounded by ρ_{0} and increases as ε′ ↘ 0^{+}, we can pass to the limit:

ρ_{0} ≥ lim_{ε′→0} sup_{α(·)∈S_{U} (t_{0},x_{0})} inf_{v(·)∈V(t_{0},z_{0})} J_{ε′}(t_{0}, x_{0}, α(v(·)), v(·)) = v^{♯}(t_{0}, x_{0}).

This implies DiscH̃(K) ⊂ Epi(v^{♯}).

Conversely, let us consider ρ_{0} > v^{♯}(t_{0}, x_{0}). Then for any ε > 0 we have ρ_{0} > sup_{α(·)∈S_{U} (t_{0},x_{0})} inf_{v(·)∈V(t_{0},z_{0})} J_{ε}(t_{0}, x_{0}, α(v(·)), v(·)) and, so, for any strategy α ∈ S_{U}(t_{0}, x_{0})

ρ_{0} > inf_{v(·)∈V(t_{0},z_{0})} J_{ε}(t_{0}, x_{0}; α(v(·)), v(·)).

Now we take a control v(·) ∈ V(t_{0}, z_{0}) such that ρ_{0} ≥ J_{ε}(t_{0}, x_{0}; α(v(·)), v(·)). The ρ-component of the trajectory X[X_{0}, α(v(·))(·), v(·)](·) satisfies the condition ρ(T) = ρ_{0}−∫_{t_{0}}^{T} L(s, x(s), α(v(·))(s), v(s)) ds. Hence, we obtain ρ(T) ≥ J_{ε}(t_{0}, x_{0}; α(v(·)), v(·))−

∫_{t_{0}}^{T} L(s, x(s), α(v(·))(s), v(s)) ds = Ψ_{ε}(x(T)) and, finally, X(τ) ∈ E + εB for a time τ ≤ T.

3 Approximation by Constrained Trajectories

An important problem in order to get suitable estimations on constrained trajectories, is to obtain a kind of Filippov Theorem with constraints. Namely a result which allows to approach - in a suitable sense - a given trajectory of the dynamics by a constrained trajectory. Note that similar results exists in the literature (cf [2], [16], [19]) but in the present paper we need a construction of the constrained trajectory in a nonanticipative way. This result also contains, as a particular case the correct proof of Lemma 4.4 in [12], and can be seen as a Corrigendum of this proof. The present section is concerned with the proof of the following

Proposition 3.1 Assume that conditions (4) are satisfied. For any R > 0 there exist C_{0} = C_{0}(R) > 0 such that for any initial time t_{0} ∈ [0, T], for any y_{0}, y_{1} ∈ K_{U} with |y_{0}|, |y_{1}| ≤ R, there is a nonanticipative strategy σ : U(t_{0}, y_{0}) ⟶ U(t_{0}, y_{1}) with the following property: for any u_{0}(·) ∈ U(t_{0}, y_{0}) and for t ∈ [t_{0}, T] we have

(15)  \begin{matrix}|y_{0}(t) − y_{1}(t)| + ∫_{t_{0}}^{t} |g(y_{0}(s), u_{0}(s)) − g(y_{1}(s), σ(u_{0}(·))(s))| ds \\ ≤ C_{0}|y_{0} − y_{1}|e^{C_{0}(t−t_{0})} \end{matrix}

where we have set for simplicity y_{0} = y[t_{0}, y_{0}; u_{0}(·)] and y_{1} = y[t_{0}, y_{1}; σ(u_{0}(·))](t). In particular if g is affine with respect to the control u, namely

g(y, u) = g_{1}(y)u + g_{2}(y)

where g_{1}(y) is an invertible matrix with a Lipschitz continuous inverse, then we have

(16)  |y_{0}(t) − y_{1}(t)| + ∫_{t_{0}}^{t} |u_{0}(s) − σ(u_{0}(·))(s)| ds ≤ C_{1}|y_{0} − y_{1}|e^{C_{1}(t−t_{0})}.

for some constant C_{1} = C_{1}(R) > 0.

The rest of this section is devoted to the proof of Proposition 3.1. For this, let us fix an admissible control u_{0}(·) ∈ U(t_{0}, y_{0}), and let us set y_{0}(·) = y[t_{0}, y_{0}; u_{0}(·)](·), take a new starting point ȳ ∈ K_{U}. We wish to build in a nonanticipative way a control ū(·) satisfying:

(17)  \begin{matrix}∫_{t_{0}}^{t} |g(y[t_{0}, y_{0}; u_{0}(·)](s), u_{0}(s)) − g(y[t_{0}, ȳ; ū](s), ū(s))| ds+ \\ |y[t_{0}, y_{0}; u_{0}(·)](t) − y[t_{0}, ȳ; ū](t)| ≤ C_{0}|y_{0} − ȳ|e^{C_{0}(t−t_{0})}. \end{matrix}

To this end, we consider the system:

(18)  \{ \begin{matrix}y′(t) & = π_{G(y(t))∩T_{K_{U}} (y(t))}(g(y(t), u_{0}(t))) \\ y(t_{0}) & = ȳ ∈ K_{U}, \end{matrix}

where G(y) := g(y, U) and π_{G(y(t))∩T_{K_{U}} (y(t))}(g(y(t), u_{0}(t))) denotes the projection of g(y(t), u_{0}(t)) onto G(y(t)) ∩ T_{K_{U}} (y(t)). Notice that π_{G(y)∩T_{K_{U}} (y)}(g(y, u_{0}(t))) = g(y, u_{0}(t)) whenever y belongs to the interior of K_{U} or if y is on the boundary of K_{U} and

⟨g(y(t), u_{0}(t)), ∇ϕ_{U}(y(t))⟩ ≤ 0 .

Let us also underline that, since the set G(y) ∩ T_{K_{U}} (y) is convex, the projection onto G(y) ∩ T_{K_{U}} (y) is unique. We denote it by g(y, ū(y, u)) and we note that the control ū(y, u) ∈ U is not necessarily unique. Our goal is to show that ū(y, u) is a suitable feedback, which enables us to build the control ū in a nonanticipative way. First we show that there is a solution to (18).

Lemma 3.2 System (18) admits at least one solution.

Proof of Lemma 3.2. We claim that the set of solutions of system (18) is the same as the set of solutions of the following system

(19)  \{ \begin{matrix}y′(t) ∈ G̃(t, y(t)), y(t) ∈ K_{U} \\ y(t_{0}) = ȳ ∈ K_{U}, \end{matrix}

where

G̃(t, y) := \{ \begin{matrix}g(y, u_{0}(t)) if y ∈ Int(K_{U}) \\ \overline{co} \{g(y, u_{0}(t)); g(y, ū(y, u_{0}(t)))\} if y ∈ ∂K_{U} \end{matrix}

Before proving the claim, let us note that, since the set-valued function G̃ is clearly Lebesgue-Borel measurable in (t, y) and upper semicontinuous with respect to y, by the measurable viability Theorem of [15], we obtain that system (19) and, so, according to the claim, also system (18), has a solution for any starting point ȳ ∈ K_{U} at any initial time t_{0}.

We now prove the claim. Since

π_{G(y)∩T_{K_{U}} (y)}(g(y, u_{0}(t))) ⊂ G̃(t, y)

then any solution of (18) is also a solution of (19). Conversely, suppose that y(·) is a solution of (19) and consider the set

D := {t | ∃y′(t) with y′(t) ∉ π_{G(y(t))∩T_{K_{U}} (y(t))}(g(y(t), u_{0}(t)))}

We have D = D_{1} ∪ D_{2} where D_{1} = {t ∈ D | y(t) ∈ ∂K_{U}} and D_{2} = {t ∈ D | y(t) ∈ Int(K_{U})}. The measure of D_{2} is zero because

G̃(t, y)|_{Int(K_{U})} ≡ π_{G(y)∩T_{K_{U}} (y)}(g(y, u_{0}(t))).

On the other hand, the measure of D_{1} is also zero; indeed, for almost every t such that y(t) ∈ ∂K_{U} the derivative y′(t) exists, and, since t is a local maximum for the function s ↦ ϕ_{U}(y(s)), the derivative of ϕ_{U}(y(s)) with respect to s at time t vanishes:

0 = \frac{d}{ds}ϕ_{U}(y(s))|_{s=t} = ⟨y′(t), ∇ϕ_{U}(y(t))⟩.

If ⟨g(y(t), u_{0}(t)), ∇ϕ_{U}(y(t))⟩ > 0 then we obtain y′(t) = g(y(t), ū(y(t), u_{0}(t))); otherwise g(y(t), ū(y(t), u_{0}(t))) = g(y(t), u_{0}(t)). In any case we get

y′(t) ∈ π_{G(y)∩T_{K_{U}} (y)}(g(y(t), u_{0}(t))).

Next Lemma allows to compare g(y, u) and g(y, ū(y, u)):

Lemma 3.3 Under assumption (4), for any R > 0, there is a constant C > 0 such that for any y ∈ ∂K_{U} with |y| ≤ R and any u ∈ U, we have

(20)  |g(y, ū(y, u)) − g(y, u)| ≤ C (⟨g(y, u), ∇ϕ_{U}(y)⟩)_{+} ,

where (x)_{+} = max{x, 0}.

Proof of Lemma 3.3. From (4-vi), we can choose η > 0 such that:

sup_{y∈∂K_{U} , |y|≤R} inf_{u∈U}⟨g(y, u), ∇ϕ_{U}(y)⟩ < −η < 0.

Fix y ∈ ∂K_{U} and consider u_{1} ∈ U such that

< g(y, u_{1}), ∇ϕ_{U}(y) > < −η < 0 .

Let us set

λ = \frac{(⟨g(y, u), ∇ϕ_{U}(y)⟩)_{+}}{η + (⟨g(y, u), ∇ϕ_{U}(y)⟩)_{+}} .

Note that λ ∈ [0, 1]. From the convexity of g(y, U), we can find some u_{λ} ∈ U such that

g(y, u_{λ}) = (1 − λ)g(y, u) + λg(y, u_{1}) .

Then

⟨g(y, u_{λ}), ∇ϕ_{U}(y)⟩ ≤ −ηλ + (1 − λ) (⟨g(y, u), ∇ϕ_{U}(y)⟩)_{+} = 0.

Hence, observing that λ ≤ \frac{1}{η} (⟨g(y, u), ∇ϕ_{U}(y)⟩)_{+}, we obtain

\begin{aligned}|g(y, ū(y, u)) − g(y, u)| & ≤ |g(y, u_{λ}) − g(y, u)| ≤ \\ ≤ λ|g(y, u_{1}) − g(y, u)| & ≤ \frac{M}{η} (⟨g(y, u), ∇ϕ_{U}(y)⟩)_{+} , \end{aligned}

for some constant M = M(R).

If ȳ(·) is a solution of (18), by the measurable selection theorem there exists an admissible control ū(·) such that

(21)  \{ \begin{matrix}ȳ′(t) & = g(ȳ(t), ū(t)) = π_{G(ȳ(t))∩T_{K_{U}} (ȳ(t))}(g(ȳ(t), u_{0}(t))) \\ y(t_{0}) & = ȳ ∈ K_{U}, \end{matrix}

Lemma 3.4 Assume that conditions (4) hold. For any positive constant R there exists a positive C̃ = C̃(R) such that for any y_{0}, ȳ ∈ K_{U} with |y_{0}|, |ȳ| ≤ R and for any admissible control u_{0}(·) ∈ U(t_{0}, y_{0}), the admissible control ū(·) ∈ U(t_{0}, ȳ) is such that

(22) ∫_{t_{0}}^{t} |g(ȳ(s), ū(s)) − g(y_{0}(s), u_{0}(s))| ds ≤ C̃ (|ȳ − y_{0}| + ∫_{t_{0}}^{t} |ȳ(s) − y_{0}(s)| ds) ,

where ȳ(s) := y[t_{0}, ȳ; ū(·)](s) and y_{0}(s) := y[t_{0}, y_{0}; u_{0}(·)](s).

Proof. Recall that (ū(·), ȳ(·)) denotes the couple control-trajectory which satisfies system (21). In order to fix the ideas, let us assume that ȳ ∈ Int(K_{U}). The case in which ȳ belongs to the boundary of K_{U} can be treated similarly. Let us define the following set:

O := {s ∈ (t_{0}, t) | ȳ(s) ∈ Int(K_{U})} = {s ∈ (t_{0}, t) | ϕ_{U}(ȳ(s)) < 0}.

The set O is open in [t_{0}, t] and it is an enumerable union of open disjoint intervals, I_{n},

O = ⋃_{n∈I N} I_{n}.

For any ε > 0 we can choose a finite number of these intervals, say I_{i} for i = 1, …, k, such that

|O \ ⋃_{i=1}^{k} I_{i}| ≤ ε.

Let us call O_{k} := Int (\overline{⋃_{i=1}^{k} I_{i}}); notice that O_{k} = ⋃_{j=0}^{h} J_{j}, where J_{j} are open intervals: J_{j} =]t_{2j}, t_{2j+1}[ with t_{2j+1} ≤ t_{2j+2}. Observe that

|O_{k} △ (⋃_{i=1}^{k} I_{i})| = 0

and that ϕ_{U}(ȳ(t_{2j})) = ϕ_{U}(ȳ(t_{2j+1})) = 0 for any j. Moreover we have

O_{k}^{c} = [t_{0}, t] \ O_{k} = [t_{0}, t] \ ⋃_{j=0}^{h} J_{j} = ⋃_{j=0}^{h}[t_{2j+1}, t_{2j+2}],

where t_{2h+2} = t. We claim that there is a constant C, independent of the control u_{0} and of the initial positions y_{0} and ȳ, such that for almost every s ∈ O^{c} we have

(23)  |g(ȳ(s), ū(s)) − g(ȳ(s), u_{0}(s))| ≤ C⟨g(ȳ(s), u_{0}(s)), ∇ϕ_{U}(ȳ(s))⟩ .

For this we apply Lemma 3.3, for the constant R̃ such that any solution staring from a point y ∈ RB ∩ K_{U} remains in R̃B on the time interval [0, T]. We have now to explain how to remove the “plus” in the inequality (20) of Lemma 3.3. Let E be the set where the derivative of ȳ(·) exists. For any s ∈ E ∩ O^{c}, we obtain

(24)  0 = \frac{d}{dτ} ϕ_{U}(ȳ(τ))|_{τ=s} = ⟨ȳ′(s), ∇ϕ_{U}(ȳ(s))⟩

because s is a local maximum for τ ⟼ ϕ_{U}(ȳ(τ)). Since |E ∩ O^{c}| = |O^{c}|, by (24) we obtain that for almost every s ∈ O^{c} either ū(s) = u_{0}(s) and, hence,

⟨g(ȳ(s), u_{0}(s)), ∇ϕ_{U}(ȳ(s))⟩ = ⟨g(ȳ(s), ū(s)), ∇ϕ_{U}(ȳ(s))⟩ = 0,

or ū(s) ≠ u_{0}(s) and, so, ū(s) = ū(ȳ(s), u_{0}(s)) and

⟨g(ȳ(s), u_{0}(s)), ∇ϕ_{U}(ȳ(s))⟩ > ⟨g(ȳ(s), ū(s)), ∇ϕ_{U}(ȳ(s))⟩ = 0.

Thanks to Lemma 3.3 we have (23).

Now, by using (23), we obtain

∫_{t_{0}}^{t}|g(ȳ(s), ū(s)) − g(y_{0}(s), u_{0}(s))| ds ≤

≤ ∫_{O^{c}} |g(ȳ(s), ū(s)) − g(ȳ(s), u_{0}(s))| ds + ∫_{t_{0}}^{t} |g(ȳ(s), u_{0}(s)) − g(y_{0}(s), u_{0}(s))| ds ≤

≤ C ∫_{O^{c}}⟨g(ȳ(s), u_{0}(s)), ∇ϕ_{U}(ȳ(s))⟩ ds + M ∫_{t_{0}}^{t} |ȳ(s) − y_{0}(s)| ds =

= C [∫_{O_{k}^{c}}⟨g(ȳ(s), u_{0}(s)), ∇ϕ_{U}(ȳ(s))⟩ ds − ∫_{O_{k}^{c}\O^{c}}⟨g(ȳ(s), u_{0}(s)), ∇ϕ_{U}(ȳ(s))⟩ ds] +

+ M ∫_{t_{0}}^{t} |ȳ(s) − y_{0}(s)| ds ≤

≤ C [∫_{O_{k}^{c}}⟨g(ȳ(s), u_{0}(s)), ∇ϕ_{U}(ȳ(s))⟩ ds + ε∥⟨g(ȳ, u_{0}), ∇ϕ_{U}(ȳ)⟩∥_{∞}] +

+ M ∫_{t_{0}}^{t} |ȳ(s) − y_{0}(s)| ds =

(because |O_{k}^{c} \ O^{c}| ≤ ε)

= C [∑_{j=0}^{h} ∫_{t_{2j+1}}^{t_{2j+2}}⟨g(ȳ(s), u_{0}(s)), ∇ϕ_{U}(ȳ(s))⟩ ds + εM̄] + M ∫_{t_{0}}^{t} |ȳ(s) − y_{0}(s)| ds ≤

≤ C[ ∑_{j=0}^{h} ∫_{t_{2j+1}}^{t_{2j+2}}⟨g(y_{0}(s), u_{0}(s)), ∇ϕ_{U}(y_{0}(s))⟩ ds

+ K_{0} ∑_{j=0}^{h} ∫_{t_{2j+1}}^{t_{2j+2}} |ȳ(s) − y_{0}(s)| ds + εM̄] + M ∫_{t_{0}}^{t} |ȳ(s) − y_{0}(s)| ds,

where K_{0} and M̄ are suitable constants depending on R. Observe that

∑_{j=0}^{h} ∫_{t_{2j+1}}^{t_{2j+2}}⟨g(y_{0}(s), u_{0}(s)), ∇ϕ_{U}(y_{0}(s))⟩ ds = ∑_{j=0}^{h} (ϕ_{U}(y_{0}(t_{2j+2})) − ϕ_{U}(y_{0}(t_{2j+1})))

= ϕ_{U}(y_{0}(t_{2h+2})) − ϕ_{U}(y_{0}(t_{1})) − ∑_{j=1}^{h} (ϕ_{U}(y_{0}(t_{2j+1})) − ϕ_{U}(y_{0}(t_{2j})))

= ϕ_{U}(y_{0}(t_{2h+2})) − ϕ_{U}(y_{0}(t_{1})) − ∑_{j=1}^{h} ∫_{t_{2j}}^{t_{2j+1}}⟨g(y_{0}(s), u_{0}(s)), ∇ϕ_{U}(y_{0}(s))⟩ ds

≤ −ϕ_{U}(y_{0}(t_{1})) − ∑_{j=1}^{h} ∫_{t_{2j}}^{t_{2j+1}}⟨g(ȳ(s), u_{0}(s)), ∇ϕ_{U}(ȳ(s))⟩ ds

+K_{0} ∑_{j=1}^{h} ∫_{t_{2j}}^{t_{2j+1}} |ȳ(s) − y_{0}(s)| ds =

= −ϕ_{U}(y_{0}(t_{1})) − ∑_{j=1}^{h} (ϕ_{U}(ȳ(t_{2j+1})) − ϕ_{U}(ȳ(t_{2j})))

+K_{0} ∑_{j=1}^{h} ∫_{t_{2j}}^{t_{2j+1}} |ȳ(s) − y_{0}(s)| ds

= −ϕ_{U}(y_{0}(t_{1})) + K_{0} ∑_{j=1}^{h} ∫_{t_{2j}}^{t_{2j+1}} |ȳ(s) − y_{0}(s)| ds,

because the points ȳ(t_{2j}) and ȳ(t_{2j+1}) for each j = 1, …, h belong to the boundary. Moreover, we have:

−ϕ_{U}(y_{0}(t_{1})) ≤ −ϕ_{U}(ȳ(t_{1}))+K_{0}|ȳ(t_{1})−y_{0}(t_{1})| ≤ K_{0} [|ȳ − y_{0}| + ∫_{t_{0}}^{t_{1}} |ȳ(s) − y_{0}(s)| ds]

because −ϕ_{U}(ȳ(t_{1})) = 0 and applying Gronwall Lemma. Then, finally, we obtain:

∫_{t_{0}}^{t}|g(ȳ(s), ū(s)) − g(y_{0}(s), u_{0}(s))| ds ≤

≤ C{K_{0} [|ȳ − y_{0}| + ∫_{t_{0}}^{t_{1}} |ȳ(s) − y_{0}(s)| ds] +

+ K_{0} ∑_{j=1}^{h} ∫_{t_{2j}}^{t_{2j+1}} |ȳ(s) − y_{0}(s)| ds + K_{0} ∑_{j=0}^{h} ∫_{t_{2j+1}}^{t_{2j+2}} |ȳ(s) − y_{0}(s)| ds + εM̄}+

+ M ∫_{t_{0}}^{t} |ȳ(s) − y_{0}(s)| ds ≤

≤ C {K_{1} [|ȳ − y_{0}| + ∫_{t_{0}}^{t} |ȳ(s) − y_{0}(s)| ds] + εM̄} ,

for some constant K_{1} > 0. This gives (22) because ε is arbitrary.

Proof of Proposition 3.1. For any admissible control u_{0}(·) ∈ U(t_{0}, y_{0}), we claim that it is possible to construct (in a nonanticipative way) an admissible control u_{1}(·) ∈ U(t_{0}, y_{1}) such that ∀t ∈ [t_{0}, T]

(25) |y_{1}(t) − y_{0}(t)| + ∫_{t_{0}}^{t} |g(y_{1}(s), u_{1}(s)) − g(y_{0}(s), u_{0}(s))| ds ≤ C_{0}|y_{1} − y_{0}|e^{C_{0}(t−t_{0})}

where y_{0}(t) = y[t_{0}, y_{0}; u_{0}(·)](t) and y_{1}(t) = y[t_{0}, y_{1}; u_{1}(·)](t). Indeed, let (u_{1}(·), y_{1}(·)) the couple control-trajectory which satisfies system (21) with the starting point y_{1} = ȳ. We get

(26)  \begin{aligned}|y_{1}(t) & − y_{0}(t)| \\ & ≤ |y_{1} − y_{0}| + ∫_{t_{0}}^{t} |g(y_{1}(s), u_{1}(s)) − g(y_{0}(s), u_{0}(s))| ds \\ & ≤ (1 + C̃)|y_{1} − y_{0}| + C̃ ∫_{t_{0}}^{t} |y_{1}(s) − y_{0}(s)| ds, \end{aligned}

invoking Lemma 3.4. Thus

\begin{aligned}|y_{1}(t) − y_{0}(t)| & + ∫_{t_{0}}^{t} |g(y_{1}(s), u_{1}(s)) − g(y_{0}(s), u_{0}(s)| ds \\ & ≤ C_{0}|y_{1} − y_{0}| + C_{0} ∫_{t_{0}}^{t} |y_{1}(s) − y_{0}(s)| ds \end{aligned}

for some positive constant C_{0} and, thanks to the Gronwall’s Lemma, we obtain (25).

Finally, it is easy to check that the set-valued map Σ : U(t_{0}, y_{0}) ⇝ U(t_{0}, y_{1}) defined by:

Σ(u_{0}(·)) := {u(·) ∈ U(t_{0}, y_{1}) | (u(·), y(·)) solves (21) }

is nonexpansive with nonempty (*)-closed values (in the sense of [13]). Hence, by Plaskacz Lemma (see Lemma 2.7 of [13]) there exists a nonanticipative selection σ with the following property: σ(u_{0}(·)) ∈ Σ(u_{0}(·)) for any u_{0}(·) ∈ U(t_{0}, y_{0}).

The proof of (16) is a direct consequence of the assumptions and of (15).

We now apply the result just proved to system (12).

Corollary 3.5 Under the assumptions (4,5,6), for any terminal time T > 0, for any R > 0, there is a constant C_{2} such that, for any X_{0}, X_{1} ∈ [0, T] × K_{U} × K_{V} × IR, |X_{0}| ≤ R, |X_{1}| ≤ R, for any nonanticipative strategy α ∈ S_{U}(t_{0}, x_{0}), there is a nonanticipative strategy α′ ∈ S_{U}(t_{1}, x_{1}) and a nonanticipative strategy τ : V(t_{1}, z_{1}) → V(t_{0}, z_{0}), such that: ∀t ∈ [t_{1}, T] and ∀v ∈ V(t_{1}, z_{1})

|X[X_{1}, α′(v), v](t) − X[X_{0}, α ∘ τ(v), τ(v)](t − t_{1} + t_{0})| ≤ C_{2}|X_{1} − X_{0}| .

Remarks 3.6 1) The strategy τ : V(t_{1}, z_{1}) → V(t_{0}, z_{0}) is nonanticipative in the following sense: if two control v_{1}, v_{2} of V(t_{1}, z_{1}) coincide on the time interval [t_{1}, t_{1}+θ] for some θ > 0, then the images τ(v_{1}) and τ(v_{2}) coincide on [t_{0}, t_{0} + θ].

2) Of course a symmetric result holds true for a nonanticipative strategy β ∈ S_{V} (t_{0}, x_{0}).

3) If z_{0} = z_{1} and t_{0} = t_{1}, then we can take τ(v) = v for any v ∈ V(t_{1}, z_{1}).

Proof of Corollary 3.5: To fix the ideas, we suppose to be under assumption 2 of (6). Set T′ = T + |t_{1} − t_{0}|. From Proposition 3.1 (see estimate (16)) there is some constant C_{1} > 0 and some nonanticipative strategy σ_{1} : V(t_{0}, z_{1}) → V(t_{0}, z_{0}) such that: ∀v ∈ V(t_{0}, z_{1}), ∀t ∈ [t_{0}, T′],

(27) |z[t_{0}, z_{1}, v](t)−z[t_{0}, z_{0}, σ_{1}(v)](t)|+∫_{t_{0}}^{t} |v(s)−σ_{1}(v)(s)|ds ≤ C_{1}e^{C_{1}(t−t_{0})}|z_{0}−z_{1}| .

Let us define τ_{1} and τ_{2} as the shifts τ_{1}(ϕ)(s) = ϕ(s − t_{0} + t_{1}) and τ_{2} = τ_{1}^{−1}. We also set τ = σ_{1} ∘ τ_{1}. Then we get: ∀v ∈ V(t_{1}, z_{1}), ∀t ∈ [t_{1}, T],

(28) \begin{matrix}|z[t_{1}, z_{1}, v](t) − z[t_{0}, z_{0}, τ(v)](t + t_{0} − t_{1})| + ∫_{t_{1}}^{t} |v(s) − τ_{2} ∘ σ_{1} ∘ τ_{1}(v)(s)|ds \\ ≤ C_{1}e^{C_{1}(t−t_{1})}|z_{0} − z_{1}| , \end{matrix}

by applying inequality (27) at time t + t_{0} − t_{1} to the control τ_{1}(v). In the same way, there is a nonanticipative strategy σ_{2} : U(t_{0}, y_{0}) → U(t_{0}, y_{1}) such that: ∀u ∈ U(t_{0}, y_{0}), ∀t ∈ [t_{0}, T′],

|y[t_{0}, y_{1}, σ_{2}(u)](t) − y[t_{0}, y_{0}, u](t)| + ∫_{t_{0}}^{t} |σ_{2}(u)(s) − u(s)|ds ≤ C_{1}e^{C_{1}(t−t_{0})}|y_{0} − y_{1}| .

Hence we have: ∀u ∈ U(t_{0}, y_{0}), ∀t ∈ [t_{1}, T],

(29)  \begin{matrix}|y[t_{1}, y_{1}, τ_{2} ∘ σ_{2}(u)](t) − y[t_{0}, y_{0}, u](t − t_{1} + t_{0})| \\ + ∫_{t_{1}}^{t} |τ_{2} ∘ σ_{2}(u)(s) − τ_{2}(u)(s)|ds ≤ C_{1}e^{C_{1}(t−t_{1})}|y_{0} − y_{1}| . \end{matrix}

Let us set α′ = τ_{2} ∘ σ_{2} ∘ α ∘ σ_{1} ∘ τ_{1}. Then we have: ∀v ∈ V(t_{1}, z_{1}), ∀t ∈ [t_{1}, T],

|X[X_{1}, α′(v), v](t) − X[X_{0}, α ∘ τ(v), τ(v)](t − t_{1} + t_{0})|
≤ |y[t_{1}, y_{1}, α′(v)](t) − y[t_{0}, y_{0}, α ∘ τ(v)](t − t_{1} + t_{0})| +
+ |z[t_{1}, z_{1}, v](t) − z[t_{0}, z_{0}, τ(v)](t − t_{1} + t_{0})| +
+ |ρ_{1} + ∫_{t_{1}}^{t} L(s, x_{1}(s), α′(v)(s), v(s)]ds − ρ_{0}
− ∫_{t_{0}}^{t−t_{1}+t_{0}} L(s, x_{0}(s), α ∘ τ(v)(s), τ(v)(s))ds|
(where x_{0}(·) = (y_{0}(·), z_{0}(·)) and x_{1}(·) = (y_{1}(·), z_{1}(·)) are the x−component
of X[X_{0}, α ∘ τ(v), τ(v)] and X[X_{1}, α′(v), v] respectively)
≤ |y_{1}(t) − y_{0}(t − t_{1} + t_{0})| + |z_{1}(t) − z_{0}(t − t_{1} + t_{0})| + |ρ_{1} − ρ_{0}|+
+ ∫_{t_{1}}^{t} |L(s, x_{1}(s), α′(v)(s), v(s))−
L(s − t_{1} + t_{0}, x_{0}(s − t_{1} + t_{0}), τ_{2} ∘ α ∘ τ(v)(s), τ_{2} ∘ τ(v)(s))| ds
≤ |y_{1}(t) − y_{0}(t − t_{1} + t_{0})| + |z_{1}(t) − z_{0}(t − t_{1} + t_{0})| + |ρ_{1} − ρ_{0}|
+M ∫_{t_{1}}^{t} (|t_{1} − t_{0}| + |x_{1}(s) − x_{0}(s + t_{1} − t_{0})| + |α′(v)(s) − τ_{2} ∘ α ∘ τ(v)(s)|
+|v(s) − τ_{2} ∘ τ(v)(s))|) ds
≤ C̄_{1}e^{C_{1}(t−t_{1})}|X_{0} − X_{1}|

for some constant C̄_{1} ≥ C_{1}, thanks to (28) and (29) applied to u := α ∘ τ(v).

4 Proof of Proposition 2.3

Throughout this section we consider the dynamic (12). It is worth pointing out that this new dynamic do satisfy assumptions (4) (when K_{U} is replaced by IR^{+} × K_{u} × IR). Note also that under the assumptions 1- or 2- of (6) the Hamiltonian H̃ (defined by (11)) satisfies the Isaacs condition

inf_{u∈U(y)} sup_{v∈V (y)}⟨f̃, p⟩ = sup_{v∈V (y)} inf_{u∈U(y)}⟨f̃, p⟩, ∀y, p

The following result expresses a property of discriminating domains in terms of trajectories. Since this is a rather direct adaptation of results of [9], we omit its proof.

Proposition 4.1 Assume that the conditions (4), (5) and (6) are satisfied. Then a closed set D ⊂ [0, T]×K_{U} ×K_{V} ×IR is a discriminating domain for H̃ if and only if for any X_{0} = (t_{0}, x_{0}, ρ_{0}) ∈ D there is some nonanticipative strategy β ∈ S_{V} (t_{0}, x_{0}) such that, for any control u(·) ∈ U(t_{0}, y_{0}), the trajectory X_{0}[X_{0}; u(·), β(u(·))](s) remains in D until it reaches E.

Proposition 4.2 Assume that the conditions (4), (5) and (6) are satisfied. A closed set D ⊂ [0, T]×K_{U} ×K_{V} ×IR is a discriminating domain for H̃ if and only if for any X_{0} = (t_{0}, x_{0}, ρ_{0}) ∈ D, for any nonanticipative strategy α ∈ S_{U}(t_{0}, x_{0}), for any ε > 0, there is some control v(·) ∈ V(t_{0}, z_{0}) such that the trajectory X[X_{0}; α(v(·)), v(·)](s) remains in D + εB on [t_{0}, T], at least as long as it does not reach E + εB: namely, there is a time T′ ∈ [t_{0}, T] such that

X[X_{0}; α(v(·)), v(·)](s) ∈ D + εB ∀s ∈ [t_{0}, T′]

with either T′ = T, or X[X_{0}; α(v(·)), v(·)](T′) ∈ E + εB.

Proof. The condition is sufficient. We claim that for any X_{0} ∈ D, for any admissible strategy α ∈ S_{U}(t_{0}, x_{0}) and for any ε > 0, there is a control v ∈ V(t_{0}, z_{0}) and a time T′ ∈ [t_{0}, T] such that

(30)  d_{D}(X[X_{0}, α(v), v)](t)) ≤ ε ∀t ∈ [0, T′]

with either T′ = T, or X[X_{0}; α(v(·)), v(·)](T′) ∈ E + εB.

Let X_{0} ∈ D, α ∈ S_{U}(t_{0}, x_{0}) and ε > 0. Since D is a discriminating domain, Proposition 4.1 states that there is some nonanticipative strategy β_{0} ∈ S_{V} (t_{0}, x_{0}) such that, for any control u ∈ U(t_{0}, y_{0}), the solution X[X_{0}, u, β_{0}(u)] remains in D as long as it does not reach the evasion set E.

Let us also fix v̄ ∈ V(t_{0}, z_{0}) and let us set z_{1} = z[t_{0}, z_{0}, v̄](τ), for some τ > 0 to be defined later. Note for later use that |z_{0} − z_{1}| ≤ M_{g}τ, where M_{g} is a bound on g. We set X_{1} = (t_{0}, y_{0}, z_{1}, ρ_{0}). By Corollary 3.5 and Remark 3.6-3), (for any β ∈ S_{V} (t_{0}, x_{0})) there is a nonanticipative strategy β_{1} ∈ S_{V} (t_{1}, x_{1}) and a constant C_{2} such that, for any u ∈ U(t_{0}, y_{0}) (recall that (t_{0}, y_{0}) = (t_{1}, y_{1})), ∀t ∈ [t_{0}, T],

|X[X_{1}, u, β_{1}(u)](t) − X[X_{0}, u, β(u)](t)| ≤ C_{2}|z_{1} − z_{0}| ≤ C_{2}M_{g}τ .

We then define β ∈ S_{V} (t_{0}, x_{0}) by setting

β(u)(t) = \{ \begin{matrix}v̄(t) & if t ∈ [t_{0}, t_{0} + τ) \\ β_{1}(u)(t − τ) & if t ≥ t_{0} + τ \end{matrix}

We note that β is a nonanticipative strategy with delay, i.e.,

if u_{1} = u_{2} on [t_{0}, t] then β(u_{1}) = β(u_{2}) on [t_{0}, t + τ] .

The next step of the proof consists in comparing X[X_{0}, u, β(u)] and X[X_{0}, u, β_{0}(u)] for any control u ∈ U(t_{0}, y_{0}). Let us recall that

X[X_{0}, u, β(u)](t) = (t, y[y_{0}, u](t), z[t_{0}, z_{0}, β(u)](t), ρ[X_{0}, u, β(u)](t))

and

X[X_{0}, u, β_{0}(u)](t) = (t, y[y_{0}, u](t), z[t_{0}, z_{0}, β_{0}(u)](t), ρ[x_{0}, u, β_{0}(u)](t)) .

We note that the y-component of X[X_{0}, u, β(u)] and X[X_{0}, u, β_{0}(u)] is the same. It remains to compare the z and ρ components. We first compare z_{1}(·) := z[t_{0}, z_{0}, β(u)](·) and z_{0}(·) := z[t_{0}, z_{0}, β_{0}(u)](·). For any t ≥ t_{0}+τ, we have z_{1}(t) = z[t_{0}, z_{1}, β_{1}(u)](t−τ) and so

\begin{matrix}|z_{1}(t) − z_{0}(t)| ≤ & |z[t_{0}, z_{1}, β_{1}(u)](t − τ) − z_{0}(t − τ)| + |z_{0}(t − τ) − z_{0}(t)| \\ ≤ & C_{2}M_{g}τ + M_{g}τ . \end{matrix}

Next we compare ρ_{1}(·) := ρ[X_{0}, u, β(u)](·) and ρ_{0}(·) := ρ[X_{0}, u, β_{0}(u)](·). To fix the ideas we work in the case 2 of assumption (6). For any t ≥ t_{0}, we have

ρ_{1}(t) = ρ_{0} − ∫_{t_{0}}^{t} L(s, y_{0}(s), z_{1}(s), u(s), β(u)(s))ds .

So, setting x_{0}(·) = (y[t_{0}, y_{0}, u](·), z_{0}(·)), we get

|ρ_{1}(t) − ρ_{0} + ∫_{t_{0}}^{t} L(s, y_{0}(s), z_{0}(s), u(s), β(u)(s))ds| ≤ M ∫_{t_{0}}^{t} |z_{0}(s) − z_{1}(s)|ds ≤ Cτ

for some constant C. Then, from the structure condition (6-condition 2), we have

|ρ_{0}(t) − ρ_{1}(t)| ≤ |ρ_{0}(t) − ρ_{0} + ∫_{t_{0}}^{t} L(s, x_{0}(s), u(s), β(u)(s))ds| + Cτ
≤ |∫_{t_{0}}^{t} (L_{2}(s, x_{0}(s))β_{0}(u)(s) − L_{2}(s, x_{0}(s))β(u)(s)) ds| + Cτ
≤ |∫_{t_{0}}^{t_{0}+τ} L_{2}(s, x_{0}(s))β(u)(s)ds| + |∫_{t−τ}^{t} L_{2}(s, x_{0}(s))β_{0}(u)(s)ds|

+ |∫_{t_{0}}^{t−τ} (L_{2}(s, x_{0}(s))β_{0}(u)(s) − L_{2}(s + τ, x_{0}(s + τ))β_{1}(u)(s)) ds| + Cτ

≤ C′τ

for some new constant C′ ≥ C which does not depend on u and β_{0}.

From now on, we choose τ such that C′τ ≤ ε. We have proved that, for any t ∈ [t_{0}, T],

|X[X_{0}, u, β(u)](t) − X[X_{0}, u, β_{0}(u)](t)| ≤ C′τ ≤ ε .

Since X[X_{0}, u, β_{0}(u)] remains in D until it reaches the target E at some time T′, we have

d_{D}(X[X_{0}, u, β(u)](t)) ≤ |X[X_{0}, u, β(u)](t) − X[X_{0}, u, β_{0}(u)](t)| ≤ ε

for all t ∈ [0, T′] and

d_{E}(X[X_{0}, u, β(u)](T′)) ≤ |X[X_{0}, u, β(u)](T′) − X[X_{0}, u, β_{0}(u)](T′)| ≤ ε .

Hence X[X_{0}, u, β(u)] remains in D + εB until it reaches the target E + εB.

Since α is nonanticipative and β is nonanticipative with delay, it can be easily proved that there is a unique pair (u, v) ∈ U(t_{0}, y_{0}) × V(t_{0}, z_{0}) such that

α(v) = u and β(u) = v .

Then, for this control v we have

X[X_{0}, α(v), v] = X[X_{0}, u, β(u)]

which shows that for this choice of v, the trajectory X[X_{0}, α(v), v] remains in D + εB until it reaches the target E + εB. This completes the proof of the sufficiency part of the Proposition.

Now we show that the condition is necessary. We suppose to be under condition 1- of (6) for simplicity of notations (case 2- of (6) is quite similar). Assume that D is not discriminating for H̃. Let X_{0} = (t_{0}, x_{0}, ρ_{0}) ∈ ∂D \ E be such that ∃p ∈ NP_{D}(X_{0}) and ∃γ > 0 with the property:

H̃(X_{0}, p) ≥ 2γ.

There exists ū ∈ U(y_{0}) such that

p_{t} + inf_{V} {⟨h(z_{0}, v), p_{z}⟩ − p_{ρ}L(t, x_{0})} + ⟨g(y_{0}, ū), p_{z}⟩ ≥ γ.

The set valued map y ↦ g(y, U(y)) is lower semicontinuous with convex compact values, so by Michael’s selection Theorem (see for instance [4]) there exists ũ : K_{U} ⟶ U such that

• y ↦ g(y, ũ(y)) is continuous

• g(y_{0}, ũ(y_{0})) = g(y_{0}, ū) .

Hence, there exists a neighborhood of (t_{0}, x_{0}), say I_{(t_{0},x_{0})}, such that ∀(t, x) ∈ I_{(t_{0},x_{0})}

inf_{V} \{⟨f̃(X, ũ(y), v), p⟩\} ≥ \frac{γ}{2} .

So, ∀v(·) ∈ V(t_{0}, z_{0}), there exists a control u(·) = ũ(y(·)) such that for all s ∈ (0, τ), there exists τ > 0 such that:

|X[X_{0}, u(·), v(·)](s) − X_{0} − p|^{2} ≤
≤ |p|^{2} − 2 ∫_{t_{0}}^{t_{0}+s}⟨f̃(X(σ), ũ(y(σ)), v(σ)), p⟩ dσ + Cs^{2} ≤
≤ |p|^{2} − γs + Cs^{2}.

Hence, we obtain

d_{D}(X[X_{0}, u(·), v(·)](s)) ≥ |p|^{2} − [|p|^{2} − γs + Cs^{2}]^{\frac{1}{2}}

which is positive ∀s ∈ (0, τ) for τ > 0 sufficiently small. Therefore, there exists a nonanticipative strategy is α : V(t_{0}, z_{0}) ⟶ U(t_{0}, y_{0}), defined by the constant map α(v(·)) = u(·) ∀v(·) ∈ V(t_{0}, z_{0}), such that the trajectory X[X_{0}, α(v(·)), v(·)](s) ∀v(·) ∈ V(t_{0}, z_{0}) leaves D + εB before the time τ for ε > 0 suitably small.

Recall now the notation K = [0, T] × IR^{N} × IR. The next step toward the proof of Proposition 2.3 is the characterization of the set

\begin{aligned}L := \{X_{0} ∈ K | & ∀α ∈ S_{U}(t_{0}, x_{0}), ∀ε > 0, ∃v(·) ∈ V(t_{0}, z_{0}), ∃T′ ∈ [t_{0}, T] s.t. \\ & X[X_{0}, α(v(·)), v(·)](T′) ∈ E + εB\} \end{aligned}

Lemma 4.3 Under the assumptions (4,5,6), suppose that X_{0} ∈ K \ L. Then there is η_{T} > 0, ε > 0 and, for any X_{1} ∈ {X_{0} + η_{T} B}, a nonanticipative strategy α ∈ S_{V} (t_{1}, x_{1}) such that, for any control v(·) ∈ V(t_{1}, z_{1}), the trajectory X[X_{1}, α(v(·)), v(·)](·) never reaches E + εB before T.

Proof. By the very definition of L, there exists a strategy α_{0} ∈ S_{U}(t_{0}, x_{0}) and a real number ε_{0} > 0 such that for any v_{0}(·) ∈ V(t_{0}, z_{0}), the solution X[X_{0}, α_{0}(v_{0}(·)), v_{0}(·)] never reaches E + ε_{0}B between t_{0} and T.

Let us set η_{T} := ε_{0}/(2(M_{0} + C_{2})), where C_{2} is the constant which appears in Corollary 3.5 for the time T + 1, R = |X_{0}| + 1 and M_{0} is a bound on the dynamics on a sufficiently large ball. From Corollary 3.5, for any X_{1} ∈ X_{0} + η_{T} B, there is a nonanticipative strategy α ∈ S_{U}(t_{1}, x_{1}) and a nonanticipative strategy τ : V(t_{1}, z_{1}) → V(t_{0}, z_{0}), such that: ∀v ∈ V(t_{1}, z_{1}), ∀t ∈ [t_{1}, T + 1],

|X[X_{1}, α(v), v](t) − X[X_{0}, α_{0}(τ(v)), τ(v)](t − t_{1} + t_{0})| ≤ C_{2}|X_{1} − X_{0}| .

Let us fix v ∈ V(t_{1}, z_{1}) and let us set X_{1} := X[X_{1}, α(v), v], X_{0} := X[X_{0}, α_{0}(τ(v)), τ(v)] and T_{1} = min{T, T − t_{1} + t_{0}}. Then, for any t ∈ [t_{1}, T_{1}],

d_{E}(X_{1}(t)) ≥ d_{E}(X_{0}(t − t_{1} + t_{0})) − |X_{0}(t − t_{1} + t_{0}) − X_{1}(t)| ≥ ε_{0} − C_{2}|X_{1} − X_{0}| ≥ ε_{0}/2

since d_{E}(X_{0}(t − t_{1} + t_{0})) ≥ ε on [t_{1}, T_{1}]. In conclusion, there is a nonanticipative strategy α ∈ S_{U}(t_{1}, x_{1}) such that, for any v ∈ V(t_{1}, z_{1}), the solution X[X_{1}, α(v), v](t) never reaches E + εB on [t_{0}, T], where ε := ε_{0}/2.

We now complete the proof of Proposition 2.3.

• Part ii) of Proposition 2.3 can be easily deduced from the previous results. We refer the reader to [12] for the complete proof, recalling only its scheme: Proposition 4.2 implies DiscH̃(K) ⊂ L. On the other hand, DiscH̃(K) ⊃ L is given by Lemma 4.3. So ii) is proved.

• Let us prove i) of Proposition 2.3. From Proposition 4.2, we obtain that DiscH̃(K) is contained in the set of points X_{0} = (t_{0}, x_{0}, ρ_{0}) ∈ [0, T] × K_{U} × K_{V} × IR, for which there is some β ∈ S_{V} (t_{0}, x_{0}), such that, for any control u(·) ∈ U(t_{0}, y_{0}), the solution X[X_{0}; u, β(u)] remains in K as long as it does not reach E.

Let us prove the converse inclusion. Consider X_{0} ∈ K\DiscH̃(K). For any β : U(t_{0}, y_{0}) ↦ V(t_{0}, z_{0}), we want to construct u(·) ∈ U(t_{0}, y_{0}) such that X[X_{0}; u(·), β(u(·))] leaves K before reaching E. For doing this, we adopt a method described in [20, 12]. We claim that ⋂_{n} K_{n} = DiscH̃(K) where K_{n} is defined as follows:

\{ \begin{matrix}K_{0} = K \\ K_{n+1} := \{X_{0} ∈ K_{n} | \begin{matrix}∀u(·) ∈ U(t_{0}, y_{0}), ∃v(·) ∈ V(t_{0}, z_{0}), ∃τ ∈ [0, +∞] \\ s.t. X[X_{0}, u(·), v(·)](τ) ∈ E if τ < +∞, and \\ ∀t ∈ [0, τ), X[X_{0}, u(·), v(·)](t) ∈ K_{n} \end{matrix} \} \end{matrix}

Note that DiscH̃(K) ⊂ K_{n} for any n. One can also easily check that ⋂_{n} K_{n} is a discriminating domain. Hence ⋂_{n} K_{n} ⊂ DiscH̃(K) which proves our claim.

Now let us fix some β ∈ S_{V} (t_{0}, x_{0}). Since X_{0} ∈ K\DiscH̃(K), there exists some i_{0} > 0 with X_{0} ∉ K_{i_{0}+1}. Thus there exists u_{0}(·) ∈ U(t_{0}, y_{0}) such that for any v(·) ∈ V(t_{0}, z_{0}), the solution X[X_{0}, u_{0}, v] leaves K_{i_{0}} before reaching E. In particular, this is the case for v = β(u_{0}). Let τ_{0} be such that X[X_{0}, u_{0}, v](τ_{0}) ∉ K_{i_{0}} and X[X_{0}, u_{0}, β(u_{0})] ∉ E on [t_{0}, τ_{0}]. Then we can apply the same procedure for X_{1} := X[X_{0}, u_{0}, β(u_{0})](τ_{0}) which does not belong to K_{i_{0}}.

This leads to construct recursively a control u ∈ U(t_{0}, y_{0}) and a increasing sequence of times (τ_{i})_{i=0,...,i_{0}+1} such that

X[X_{0}, u, β(u)](τ_{i}) ∉ K_{i_{0}−i}, X[X_{0}, u, β(u)](s) ∉ E ∀s ∈ [t_{0}, τ_{i_{0}+1}] .

In particular, X[X_{0}, u, β(u)](τ_{i_{0}+1}) ∉ K_{0} = K.

This proves that for any nonanticipative strategy β ∈ S_{V} (t_{0}, x_{0}), there is a control u ∈ U(t_{0}, y_{0}) such that the solution X[X_{0}, u, β(u)] leaves K before reaching E.

5 The Hamilton-Jacobi formulation of the problem

In this Section our aim is to characterize the value function studied before as the unique viscosity solution to the Hamilton-Jacobi-Isaacs equation:

(31)  \{ \begin{matrix}−∂_{t}W (t, x) + H(t, x, ∂_{x}W (t, x)) & = 0  on (0, T) × K_{U} × K_{V} \\ W (T, x) & = Ψ(x)  on K_{U} × K_{V} , \end{matrix}

where the Hamiltonian function, H, is given by:

H(t, x, p) := max_{v∈V (z)} min_{u∈U(y)}{−⟨f(x, u, v), p⟩ − L},

and where L is the running cost function. Since the function H is not continuous in general, in order to consider the notion of solution in viscosity sense we have to use the upper and lower semicontinuous envelope of H, denoted by H^{∗} and H_{∗} respectively (see for instance [8] or [6] ). We remind that by definition these functions are:

H^{∗}(t, x, p) := lim sup_{(t′,x′,p′)→(t,x,p)} H(t′, x′, p′)

and

H_{∗}(t, x, p) := lim inf_{(t′,x′,p′)→(t,x,p)} H(t′, x′, p′).

Remark 5.1 Suppose that assumption (6) holds true. In this case, since the setvalued maps y ⇝ U(y) and z ⇝ V (z) are lower semicontinuous and the functions f and L are Lipschitz continuous, it is straightforward to see that

H^{∗}(t, x, p) = max_{v∈V} min_{u∈U(y)}{−⟨f(x, u, v), p⟩ − L}

and

H_{∗}(t, x, p) = max_{v∈V (z)} min_{u∈U}{−⟨f(x, u, v), p⟩ − L}.

Definition 5.2 (Viscosity solution) A viscosity super-solution for the Hamilton-Jacobi-Isaacs equation (31) is a lower semicontinuous function w : [0, T) × K_{U} × K_{V} ⟶ IR with the following property: for any test function φ ∈ C^{1} such that w − φ has a local minimum at (t_{0}, x_{0}) then

−∂_{t}φ(t_{0}, x_{0}) + H^{∗}(t_{0}, x_{0}, ∂_{x}φ(t_{0}, x_{0})) ≥ 0.

An upper semicontinuous function w : [0, T) × K_{U} × K_{V} ⟶ IR is called viscosity subsolution of (31) if ∀φ ∈ C^{1} such that w − φ has a local maximum at (t_{0}, x_{0}) then

−∂_{t}φ(t_{0}, x_{0}) + H_{∗}(t_{0}, x_{0}, ∂_{x}φ(t_{0}, x_{0})) ≤ 0.

We say that a continuous function is a viscosity solution of (31) if it is both supersolution and subsolution of (31) at the same time.

5.1 Characterization of supersolution

In this section we show that the value function is the smallest supersolution of the Hamilton-Jacobi-Isaacs equation (31). For this we give an equivalent formulation of the supersolution, which involves the notion of proximal normal.

Define the set Q := [0, T) × K_{U} × K_{V} ⊂ IR^{n+1}. Let w : Q ⟶ IR be a lower semicontinuous function. Then any proximal normal to the epigraph of w at the point (q, w(q)) belongs to IR^{n+1} × IR. If we take an element (p_{q}, p_{ρ}) ∈ NP_{Epi(w)}(q, w(q)) with p_{q} ∈ IR^{n+1} and p_{ρ} ∈ IR, it is easy to check that p_{ρ} ≤ 0.

Theorem 5.3 (See Theorem 7.2 in [11]) Assume that the Hamiltonian H : Q × IR × IR^{n+1} ⟶ IR is a lower semicontinuous and that w : Q ⟶ IR ∪ {∞} is an extended lower semicontinuous map. Then w is a viscosity super-solution to

H(q, w(q), Dw(q)) = 0

if and only if w satisfies

∀q ∈ Q, ∀(p_{q}, p_{ρ}) ∈ NP_{Epi(w)}(q, w(q)), p_{ρ} ≠ 0 then H(q, w(q), \frac{p_{q}}{p_{ρ}}) ≥ 0.

Besides this theorem, we need an another result due to Rockafellar, in order to explain what happens for the proximal normals of the form (p_{q}, 0) ∈ NP_{Epi(w)}(q, w(q)) and (p_{q}, p_{ρ}) ∈ NP_{Epi(w)}(q, ρ) with ρ > w(q) (See for instance Lemma 7.3 and Lemma 7.4 of [11]).

Lemma 5.4 Let w : Q ⟶ IR ∪ {∞} be a lower semicontinuous map and q̄ a point of the domain of w, Dom(w).

i) If (p_{q}, 0) ∈ NP_{Epi(w)}(q̄, w(q̄)), then there exists a sequence q^{n} ∈ Dom(w) and (p_{q}^{n}, p_{ρ}^{n}) ∈ NP_{Epi(w)}(q^{n}, w(q^{n})) such that

q^{n} → q̄, w(q^{n}) → w(q̄), p_{ρ}^{n} < 0 and \frac{(p_{q}^{n}, p_{ρ}^{n})}{|(p_{q}^{n}, p_{ρ}^{n})|} → \frac{(p_{q}, 0)}{|p_{q}|}.

ii) If (p_{q}, p_{ρ}) ∈ NP_{Epi(w)}(q̄, ρ) with ρ > w(q̄), then we have p_{ρ} = 0 and (p_{q}, 0) ∈ NP_{Epi(w)}(q̄, w(q̄)).

Theorem 5.5 Under the assumptions (4,5,6), the function ϑ := v^{♭} = v^{♯} is the smallest lower semicontinuous super-solution to (31).

Proof of Theorem 5.5. By Theorem 2.2, the epigraph of ϑ is the discriminating kernel of K for the Hamiltonian function H̃, DiscH̃(K). So, for any q = (t, x) ∉ E and for any (p_{q}, p_{ρ}) = (p_{t}, p_{x}, p_{ρ}) ∈ NP_{Epi(ϑ)}((t, x), ϑ(t, x)), by the definition of H̃ and proximal normal, we get

p_{ρ} < 0 ⟹ H̃(t, x, ρ; p_{t}, p_{x}, p_{ρ}) ≤ 0

that is:

−\frac{p_{t}}{|p_{ρ}|} + H(t, x, \frac{p_{x}}{|p_{ρ}|}) ≥ 0.

So applying Theorem 5.3 with H(t, x; p_{t}; p_{x}) = p_{t} + H_{∗}(t, x, p_{x}), we have that ϑ is a lower semicontinuous super-solution of the Hamilton-Jacobi-Isaacs equation (31).

On the other hand, we claim that the epigraph Epi(w) of any lower semicontinuous viscosity super-solution of (31) w is a discriminating domain for the Hamiltonian H̃. Since Epi(w) is contained in K, the discriminating domain of Epi(w) is contained in DiscH̃(K) which is precisely equal to the epigraph of ϑ.

Indeed, choose a point (t, x) with t ≠ T and take any (p_{t}, p_{x}, p_{ρ}) ∈ NP_{Epi(w)}(t, x, w(t, x)). If p_{ρ} < 0, just applying Theorem 5.3 we obtain that

p_{ρ} < 0 ⟹ −\frac{p_{t}}{|p_{ρ}|} + H(t, x, \frac{p_{x}}{|p_{ρ}|}) ≥ 0

and, so,

H̃(t, x, ρ; p_{t}, p_{x}, p_{ρ}) ≤ 0.

If p_{ρ} = 0, then thanks to i) of Lemma 5.4, it is possible to find sequences

(t^{n}, x^{n}) and (p_{t}^{n}, p_{x}^{n}, p_{ρ}^{n}) ∈ NP_{Epi(w)}(t^{n}, x^{n}, w(t^{n}, x^{n}))

such that

(t^{n}, x^{n}) → (t, x), w(t^{n}, x^{n}) → w(t, x), p_{ρ}^{n} < 0 and \frac{(p_{t}^{n}, p_{x}^{n}, p_{ρ}^{n})}{|(p_{t}^{n}, p_{x}^{n}, p_{ρ}^{n})|} → \frac{(p_{t}, p_{x}, 0)}{|(p_{t}, p_{x})|}.

Since H̃(t^{n}, x^{n}, ρ^{n}; p_{t}^{n}, p_{x}^{n}, p_{ρ}^{n}) ≤ 0, H̃ is lower semicontinuous and positively homogeneous, we conclude that H̃(t, x, ρ; p_{t}, p_{x}, 0) ≤ 0. If (p_{t}, p_{x}, p_{ρ}) ∈ NP_{Epi(ϑ)}((t, x), ρ) for some ρ > w(t, x), then ii) of Lemma 5.4 states that p_{ρ} = 0 and (p_{t}, p_{x}, 0) ∈ NP_{Epi(w)}(t, x, w(t, x)). Hence, we obtain H̃(t, x, ρ; p_{t}, p_{x}, 0) ≤ 0

5.2 Lipschitz continuity of the value function

The aim of this part is to prove the following result:

Proposition 5.6 Under the assumptions (4,5,6), if the final cost Ψ = Ψ(x) is locally Lipschitz continuous, then the value function ϑ := v^{♭} = v^{♯} is also locally Lipschitz continuous.

Proof : Since the dynamics have a linear growth, and that we work locally in space, we can assume that f̃ defined by (10) and Ψ are bounded by some constant M_{0}, and that Ψ is Lipschitz continuous with Lispchitz constant M_{0}.

Let (t_{0}, x_{0}), (t_{1}, x_{1}) ∈ [0, T]×K_{U} ×K_{V} and ε > 0. From the definition of v^{♭}, there is a nonanticipative strategy β such that

v^{♭}(t_{0}, x_{0}) + ε ≥ sup_{u∈U(t_{0},y_{0})} J(t_{0}, x_{0}, u, β(u)) .

Let us set ρ_{0} = ρ_{1} = 0, X_{0} = (t_{0}, x_{0}, ρ_{0}) and X_{1} = (t_{1}, x_{1}, ρ_{1}). From Corollary 3.5, there is a nonanticipative strategy β′ ∈ S_{V} (t_{1}, x_{1}) and a nonanticipative strategy τ : U(t_{1}, y_{1}) → U(t_{0}, y_{0}), such that: ∀t ∈ [t_{1}, T + 1],

|X[X_{1}, u, β′(u)](t) − X[X_{0}, τ(u), β(τ(u))](t − t_{1} + t_{0})| ≤ C_{2}|X_{1} − X_{0}| .

Let us denote by x_{0}(·) and x_{1}(·), and by ρ_{0}(·) and ρ_{1}(·) the x− and ρ−components of X[X_{0}, τ(u), β(τ(u))] and X[X_{1}, u, β′(u)], respectively. Then

ρ_{0}(t) = − ∫_{t_{0}}^{t} L(s, x_{0}(s), τ(u)(s), β(τ(u))(s))ds ∀t ≥ t_{0}

and

ρ_{1}(t) = − ∫_{t_{1}}^{t} L(s, x_{1}(s), u(s), β′(u)(s))ds ∀t ≥ t_{1} .

From the choice of β, we have that

Ψ(x_{0}(T)) − ρ_{0}(T) = J(t_{0}, x_{0}, τ(u), β(τ(u)) ≤ v^{♭}(t_{0}, x_{0}) + ε .

Hence

J(t_{1}, x_{1}, u, β′(u)) = Ψ(x_{1}(T)) − ρ_{1}(T) ≤ Ψ(x_{0}(T)) − ρ_{0}(T) + M_{0}|x_{0}(T) − x_{0}(T − t_{1} + t_{0})| + M_{0}|x_{0}(T − t_{1} + t_{0}) − x_{1}(T)|

+|ρ_{0}(T) − ρ_{0}(T − t_{1} + t_{0})| + |ρ_{0}(T − t_{1} + t_{0}) − ρ_{1}(T)|

≤ v^{♭}(t_{0}, x_{0}) + ε + 2M_{0}^{2}|t_{1} − t_{0}| + M_{0}C_{2}|X_{1} − X_{0}| .

Taking in the above inequality the supremum over u ∈ U(t_{1}, y_{1}), then the infinimum over β′ ∈ S_{V} (t_{1}, x_{1}) and letting ε → 0 finally gives:

v^{♭}(t_{1}, x_{1}) ≤ v^{♭}(t_{0}, x_{0}) + C(|t_{1} − t_{0}| + |x_{1} − x_{0}|)

for some constant C.

5.3 Uniqueness of continuous viscosity solution

The last result of this paper amounts to characterize the value function as unique viscosity solution of the Hamilton-Jacobi-Isaacs equation (31). For this we assume from now on that the data satisfy the assumptions (4,5,6) and that the final cost Ψ is locally Lipschitz continuous. Under these assumptions we have proved in the previous subsection that the value function ϑ is Lipschitz continuous.

Let us start by recalling a well-known result:

Lemma 5.7 An upper semicontinuous function w : [0, T) × K_{U} × K_{V} ⟶ IR is a viscosity subsolution of (31) if and only if −w is a viscosity supersolution of

(32)  −∂_{t}W (t, x) − H(t, x, −∂_{x}W (t, x)) = 0

Let us also recall that in our case the Hamiltonian function is given by

H(t, x, p) := max_{v∈V (z)} min_{u∈U(y)}{−⟨f(x, u, v), p⟩ − L},

by Remark 5.1 we deduce that

(−H)^{∗}(t, x, −p) = min_{v∈V (z)} max_{u∈U}{−⟨f(x, u, v), p⟩ + L}.

Lemma 5.8 The function ϑ is the largest viscosity subsolution of (31).

Proof. First observe that

−ϑ(t_{0}, x_{0}) = sup_{β∈S_{V} (t_{0},x_{0})} inf_{u(·)∈U(t_{0},y_{0})} −J(t_{0}, x_{0}; u(·), β(u(·)))

and recall that −Ψ is continuous by the assumptions. We just apply the result given by Theorem 5.5 to the function −ϑ with the Hamiltonian H̄(t, x, p) = (−H)^{∗}(t, x, −p). Finally, Lemma 5.7 allows us to conclude.

We conclude with the following uniqueness result.

Proposition 5.9 (Uniqueness) The Hamilton-Jacobi-Isaacs equation (31) admits a unique continuous viscosity solution, which is given by the value function ϑ.

Proof. We have to show that if w is a continuous viscosity solution of (31) then w = ϑ. Indeed, by Theorem 5.5 we get that w ≥ ϑ. On the other hand, thanks to Lemma 5.8 −w ≥ −ϑ and, so, w = ϑ.

References

[1] ALZIARY de ROQUEFORT (1991) Jeux différentiels et approximation numérique de fonctions valeur, RAIRO Math. Model. Numer. Anal., 25, pp. 517-560.

[2] ARISAWA M. & LIONS P.L. (1996) Continuity of admissible trajectories for state constraints control problems, Discrete Contin. Dynam. Systems, 2, no. 3, 297–305.

[3] AUBIN J.-P. (1991) Viability Theory. Birkhäuser.

[4] AUBIN J.-P.& FRANKOWSKA H. (1990) Set-valued analysis. Birkhäuser.

[5] BARDI M., BOTTACIN S. & FALCONE M. (1995) Convergence of discrete schemes for discontinuous value functions of pursuit-evasion games. New trends in dynamic games and applications, 273–304, Ann. Internat. Soc. Dynam. Games, 3, Birkhpp. 273–304.

[6] BARDI, M., CAPUZZO-DOLCETTA I. (1997) Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations. Systems and Control: Foundations and Applications. Boston, Birkhäuser. xvii, 570 pp.

[7] BARDI M., KOIKE S. & SORAVIA P. (2000) Pursuit-evasion game with state constraints: dynamic programming and discrete-time approximations, Discrete Contin. Dynam. Systems, Vol. 6, no. 2, 361–380.

[8] BARLES G. (1994) Solutions de viscosité des équations de Hamilton-Jacobi. (Viscosity solutions of Hamilton-Jacobi equations). Mathématiques & Applications (Paris). 17. Paris: Springer-Verlag. ix, 194 p.

[9] CARDALIAGUET P. (1996) A differential game with two players and one target, SIAM J. Control and Optimization, Vol. 34, N. 4, pp. 1441-1460.

[10] CARDALIAGUET P. (1997) Non smooth semi-permeable barriers, Isaacs equation and application to a differential game with one target and two players, Appl. Math. Opti., 36, pp. 125-146.

[11] CARDALIAGUET P., QUINCAMPOIX M. & SAINT-PIERRE P. (1999) Numerical methods for differential games, in “Stochastic and differential games : Theory and numerical methods”, pp. 177-247. Annals of the international Society of Dynamic Games, M. Bardi, T.E.S. Raghavan, T. Parthasarathy Eds. Birkhäuser.

[12] CARDALIAGUET P., QUINCAMPOIX M. & SAINT-PIERRE P. (2001) Pursuit differential games with state constraints., SIAM J. Control and Optimization Vol. 39, N. 5, pp.1615-1632

[13] CARDALIAGUET P. & PLASKACZ S. (2000) Invariant solutions of differential games and Hamilton-Jacobi equations for time-measurable hamiltonians, SIAM Journal on Control and Optim. Vol. 38, N. 5, pp.1501-1520

[14] EVANS L.C. & SOUGANIDIS P.E. (1984) Differential games and representation formulas for solutions of Hamilton-Jacobi Equations Indiana Univ. Math. J., 282, pp. 487-502.

[15] FRANKOWSKA H., PLASCASZ M. & RZEZUCHOWSKI T. (1995) Measurable Viability Theorem and Hamilton-Jacobi-Bellman Equations, J. Diff. Eqs., Vol. 116, No 2, pp. 265-305.

[16] FRANKOWSKA H & RAMPAZZO F. (2000) Filippov’s and Filippov-Wazewski’s theorems on closed domains, J. Differential Equations 161, no. 2, 449–478.

[17] ISAACS R. (1965) Differential Games Wiley, New York.

[18] KRASOVSKII N.N. & SUBBOTIN A.I. (1988) Game-Theorical Control Problems Springer-Verlag, New-York.

[19] LORETI P. & TESSITORE M.E. (1994) Approximation and regularity results on constrained viscosity solutions of Hamilton-Jacobi-Bellman equations. J. Math. Systems Estim. Control 4, no. 4, pp. 467–483.

[20] OSIPOV Ju. S. (1971) Alternative in a Differential-Difference Game, Soviet Math. Dokl., Vol. 12, pp. 619-624.

[21] ROZYEV I. & SUBBOTIN A.I. (1988) Semicontinuous solutions of Hamilton-Jacobi Equations. PMM U.S.S.R., Vol. 52, N. 2, pp. 141-146.

[22] SONER M.H. (1986) Optimal control problems with state-space constraints. SIAM J. on Control and Optimization, 24, pp. 552-562 and pp.1110-1122.