Part 1. Let’s first look at the case when w is bounded by a constant w1 and define for (z, ) 2
⌥⇥ R
W (z, ) = Z T h
0 s(z)
✓Z
Z
w( s(z), d, h + s, u) (s)(du)
◆ ds
Now take (z, ) 2 ⌥ ⇥ R and suppose (zn, n) ! (z, ). Let’s write z = (v, d, h) and zn = (vn, dn, hn) for n 2 N. For s 2 [0, T ], let wn(s, u) := w( sn(zn), dn, hn+ s, u) and w(s, u) :=
w( s(z), d, h + s, u). Let also an= min(T h, T hn)and bn = max(T h, T hn).
Then
The first term on the right-hand side converges to zero for n ! 1 since the integrand is bounded.
Z T h
by dominated convergence and the continuity of w and of proved in Lemma 3.5.
Z T h
again by dominated convergence, provided that for s 2 [0, T ], the convergence
sn(zn) n!1! s(z)holds. For this convergence to hold it is enough that for t 2 [0, T ], By the local Lipschitz property of d,
Z t term converges to zero by the definition of the weakly* convergence in L1(M (Z)).
Part 2. In the general case where |w| wcB⇤, let wB(z, u) = w(z, u) cwB⇤(z) 0 for (z, u) 2 ⌥ ⇥ U. wB is a continuous function and there exists a nonincreasing sequence (wnB)of bounded continuous functions such that wBn n!1!wB. By the first part of the proof we know that
Wn(z, ) = Z T h
0 s(z) Z
Z
wnB( s(z), d, h + s, u)µs(du)ds is bounded, continuous, decreasing and converges to
W (z, ) cw
Z T h
0 s(z)b( s(z))e⇣⇤(T h s)ds
which is thus upper semicontinuous. Since b is a continuous bounding function it is easy to show that
(z, )! Z T h
0 s(z)b( s(z))e⇣⇤(T h s)ds
is continuous so that in fact W is upper semicontinuous. Now considering the function wB(z, u) = w(z, u) cwB⇤(z) 0 we easily show that W is also lower semicontinuous so that finally W is continuous.
Now the continuity of the applications (z, ) ! c0(z, ) and (z, ) ! (Q0w)(z, ) comes from the previous result applied to the continuous functions defined for (z, u) 2 ⌥ ⇥ U by w1(z, u) :=
c(v, u) and w2(z, u) := d(v, u) Z
D
w(v, r, h)Q(dr|v, d, u) with z = (v, d, h). Here the different assumptions of continuity (H( ))2.3., (H(c))1. and (H(Q)) are needed.
References
[1] N.U. Ahmed. Properties of relaxed trajectories for a class of nonlinear evolution equations on a Banach space. SIAM J. Control Optim., 21(6):953–967, 1983.
[2] N.U. Ahmed and K.L. Teo. Optimal control of systems governed by a class of nonlinear evolu-tion equaevolu-tions in a reflexive Banach space. Journal of Optimizaevolu-tion Theory and Applicaevolu-tions, 25(1):57–81, 1978.
[3] N.U. Ahmed and X. Xiang. Properties of relaxed trajectories of evolution equations and optimal control. SIAM J. Control Optim., 31(5):1135–1142, 1993.
[4] T.D. Austin. The emergence of the deterministic Hodgkin-Huxley equations as a limit from the underlying stochastic ion-channel mechanism. Ann. Appl. Probab, 18:1279–1325, 2008.
[5] E.J. Balder. A general denseness result for relaxed control theory. Bull. Austral. Math. Soc., 30:463–475, 1984.
[6] D. Bertsekas and S. Shreve. Stochastic optimal control: the discrete-time case. Academic Press, 1978.
[7] P. Billingsley. Convergence of probability measures. John Wiley & sons, New York, 1968.
[8] E.S. Boyden, F. Zhang, E. Bamberg, G. Nagel, and K. Deisseroth. Millisecond-timescale, genetically targeted optical control of neural activity. Nature Neuroscience, 8(9):1263–1268, September 2005.
[9] A. Brandejsky, B. de Saporta, and F. Dufour. Numerical methods for the exit time of a Piecewise Deterministic Markov Process. Adv. in Appl. Probab., 44(1):196–225, 2012.
[10] E. Buckwar and M. Riedler. An exact stochastic hybrid model of excitable membranes in-cluding spatio-temporal evolution. J. Math. Biol., 63(6):1051–1093, 2011.
[11] N. Bäuerle and U. Rieder. MDP algorithms for portfolio optimization problems in pure jump markets. Finance Stoch, 13:591–611, 2009.
[12] N. Bäuerle and U. Rieder. Optimal control of Piecewise Deterministic Markov Processes with finite time horizon. Modern trends of controlled stochastic processes: Theory and Applications, pages 144–160, 2010.
[13] N. Bäuerle and U. Rieder. Markov Decision Processes with Applications to finance. Springer, Heidelberg, 2011.
[14] O. Costa and F. Dufour. Stability and Ergodicity of Piecewise Deterministic Markov Pro-cesses. SIAM J. of Control and Opt., 47:1053–1077, 2008.
[15] O. Costa and F. Dufour. Singular perturbation for the discounted continuous control of Piecewise Deterministic Markov Processes. Appl. Math. and Opt., 63:357–384, 2011.
[16] O.L.V. Costa, C.A.B Raymundo, F. Dufour, and K. Gonzalez. Optimal stopping with contin-uous control of piecewise deterministic markov processes. Stoch. Stoch. Rep., 70(1-2):41–73, 2000.
[17] A. Crudu, A. Debussche, A. Muller, and O. Radulescu. Convergence of Stochastic Gene Networks to Hybrid Piecewise Deterministic Processes. Ann. Appl. Prob., 22(5):1822–1859, 2012.
[18] M.H.A. Davis. Piecewise-Deterministic Markov Processes: a general class of non-diffusion stochastic models. J. R. Statist. Soc., 46(3):353–388, 1984.
[19] M.H.A Davis. Markov Models and Optimization. Chapman and Hall, 1993.
[20] J. Diestel and J.J. Uhl. Vector measures. American Mathematical Society, Providence, 1977.
[21] J. Dieudonné. Sur le théorème de Lebesgue-Nikodym (iii). Ann. Université de Grenoble, 23:25–53, 1947-48.
[22] J. Dieudonné. Sur le théorème de Lebesgue-Nikodym (iv). J. Indian Math. Soc., 22:77–86, 1951.
[23] V. Dumas, F. Guillemin, and Ph. Robert. A markovian analysis of additive-increase multiplicative-decrease algorithms. Adv. in Appl. Probab., 34(1):85–111, 2002.
[24] N. Dunford and J.T. Schwartz. Linear operators. Part I: General theory. Academic Press, New York, 1988.
[25] K-J. Engel and R. Nagel. One parameter semigroups for linear evolution equations. Springer-Verlag New York, 2000.
[26] H.O. Fattorini. Existence theory and the maximum principle for relaxed infinite-dimensional optimal control problems. SIAM J. Control Optim., 32(2):311–331, 1994.
[27] H.O. Fattorini. Relaxation theorems, differential inclusions, and Filippov’s theorem for re-laxed controls in semi linear infinite dimensional systems. Journal of Differential Equations, 112:131–153, 1994.
[28] L. Forwick, M. Schäl, and M. Schmitz. Piecewise deterministic markov control processes with feedback controls and unbounded costs. Acta Applicandae Mathematicae, 82(3):239–267, 2004.
[29] R. Gamkrelidze. Principle of optimal control theory. Plenum, New York, 1987.
[30] A. Genadot. A multiscale study of stochastic spatially-extended conductance-based models for excitable systems. PhD thesis, Université Pierre et Marie Curie - Paris VI, 2013.
[31] A. Genadot and M. Thieullen. Averaging for a fully coupled Piecewise Deterministic Markov Process in infinite dimensions. Adv. in Appl. Probab., 44(3):749–773, 2012.
[32] D. Goreac and M. Martinez. Algebraic invariance conditions in the study of approximate (null-)controllability of markov switch processes. Mathematics of Control, Signals, and Systems, 27(4):551–578, 2015.
[33] A.L. Hodgkin and A.F. Huxley. A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Physiol., 117:500–544, 1952.
[34] Q. Hu and W. Yue. Markov Decision Processes with their applications. Springer US, 2008.
[35] J. Jacod. Multivariate point processes: predictable projections, Radon-Nikodym derivatives, representation of martingales. Z. Wahrsag. Verw. Gebiete, 34:235–253, 1975.
[36] C. Morris and H. Lecar. Voltage oscillations in the barnacle giant muscle fiber. Biophysics Journal, 35:193–213, 1981.
[37] K. Nikolic, N. Grossman, M.S. Grubb, J. Burrone, C. Toumazou, and P. Degenaar. Photo-cycles of channelrhodopsin-2. Photochemistry and Photobiology, 85:400–411, 2009.
[38] K. Nikolic, S. Jarvis, N. Grossman, and S. Schultz. Computational models of Optogenetic tools for controlling neural circuits with light. Conf Proc IEEE Eng Med Biol Soc, pages 5934–5937, 2013.
[39] K. Pakdaman, M. Thieullen, and G. Wainrib. Fluid limit theorems for stochastic hybrid systems and applications to neuron models. Adv. in Appl. Probab., 42(3):761–794, 2010.
[40] K. Pakdaman, M. Thieullen, and G. Wainrib. Reduction of stochastic conductance-based neuron models with time-sacles separation. J. Comput. Neurosci., 32(2):327–346, 2012.
[41] N.S. Papageorgiou. Properties of the relaxed trajectories of evolution equations and optimal control. SIAM J. Control Optim., 27(2):267–288, 1989.
[42] V. Renault, M. Thieullen, and E. Trélat. Minimal time spiking in various chr2-controlled neuron models. hal-01320492, 2016.
[43] M. Riedler, M. Thieullen, and G. Wainrib. Limit theorems for infinite-dimensional Piece-wise Deterministic Markov Processes. Applications to stochastic excitable membrane models.
Electron. J. Probab., 17(55):1–48, 2012.
[44] D. Vermes. Optimal control of Piecewise Deterministic Markov Processes. Stochastics. An International Journal of Probability and Stochastic Processes, 14(3):165–207, 1985.
[45] G. Wainrib. Randomness in neurons: a multi scale probabilistic analysis. PhD thesis, École Polytechnique, 2010.
[46] J. Warga. Necessary conditions for minimum in relaxed variational problems. J. Math. Anal.
Appl., 4:129–145, 1962.
[47] J. Warga. Relaxed variational problems. J. Math. Anal. Appl., 4:111–128, 1962.
[48] J. Warga. Optimal Control of Differential and Functional Equations. Wiley-Interscience, New York, 1972.
[49] J.C. Williams, J. Xu, and al. Computational Optogenetics: empirically-derived voltage- and light-sensitive Channelrhodopsin-2 model. PLoS Comput Biol, 9(9):1–19, 2013.
[50] L.C. Young. Lectures on the Calculus of Variations and Optimal Control Theory. W.B.
Saunders, Philadelphia, PA, 1969.
[51] A.A. Yushkevich. On reducing a jump controllable Markov model to a model with discrete time. Theory Probab. Appl., 25:58–69, 1980.
[52] E. Zeidler. Nonlinear functional analysis and its applications. Springer, New York, 1990.