Numerical Methods and Deep Learning for Stochastic Control Problems and Partial Differential Equations

Numerical Methods and Deep Learning for Stochastic Control Problems and Partial Differential Equations
Author: Come Huré
Publisher:
Total Pages: 0
Release: 2019
Genre:
ISBN:

The present thesis deals with numerical schemes to solve Markov Decision Problems (MDPs), partial differential equations (PDEs), quasi-variational inequalities (QVIs), backward stochastic differential equations (BSDEs) and reflected backward stochastic differential equations (RBSDEs). The thesis is divided into three parts.The first part focuses on methods based on quantization, local regression and global regression to solve MDPs. Firstly, we present a new algorithm, named Qknn, and study its consistency. A time-continuous control problem of market-making is then presented, which is theoretically solved by reducing the problem to a MDP, and whose optimal control is accurately approximated by Qknn. Then, a method based on Markovian embedding is presented to reduce McKean-Vlasov control prob- lem with partial information to standard MDP. This method is applied to three different McKean- Vlasov control problems with partial information. The method and high accuracy of Qknn is validated by comparing the performance of the latter with some finite difference-based algorithms and some global regression-based algorithm such as regress-now and regress-later.In the second part of the thesis, we propose new algorithms to solve MDPs in high-dimension. Neural networks, combined with gradient-descent methods, have been empirically proved to be the best at learning complex functions in high-dimension, thus, leading us to base our new algorithms on them. We derived the theoretical rates of convergence of the proposed new algorithms, and tested them on several relevant applications.In the third part of the thesis, we propose a numerical scheme for PDEs, QVIs, BSDEs, and RBSDEs. We analyze the performance of our new algorithms, and compare them to other ones available in the literature (including the recent one proposed in [EHJ17]) on several tests, which illustrates the efficiency of our methods to estimate complex solutions in high-dimension.Keywords: Deep learning, neural networks, Stochastic control, Markov Decision Process, non- linear PDEs, QVIs, optimal stopping problem BSDEs, RBSDEs, McKean-Vlasov control, perfor- mance iteration, value iteration, hybrid iteration, global regression, local regression, regress-later, quantization, limit order book, pure-jump controlled process, algorithmic-trading, market-making, high-dimension.

Numerical Control: Part A

Numerical Control: Part A
Author:
Publisher: Elsevier
Total Pages: 596
Release: 2022-02-15
Genre: Mathematics
ISBN: 0323853390

Numerical Control: Part A, Volume 23 in the Handbook of Numerical Analysis series, highlights new advances in the field, with this new volume presenting interesting chapters written by an international board of authors. Chapters in this volume include Numerics for finite-dimensional control systems, Moments and convex optimization for analysis and control of nonlinear PDEs, The turnpike property in optimal control, Structure-Preserving Numerical Schemes for Hamiltonian Dynamics, Optimal Control of PDEs and FE-Approximation, Filtration techniques for the uniform controllability of semi-discrete hyperbolic equations, Numerical controllability properties of fractional partial differential equations, Optimal Control, Numerics, and Applications of Fractional PDEs, and much more. Provides the authority and expertise of leading contributors from an international board of authors Presents the latest release in the Handbook of Numerical Analysis series Updated release includes the latest information on Numerical Control

Numerical Methods for Stochastic Control Problems in Continuous Time

Numerical Methods for Stochastic Control Problems in Continuous Time
Author: Harold Kushner
Publisher: Springer Science & Business Media
Total Pages: 480
Release: 2013-11-27
Genre: Mathematics
ISBN: 146130007X

Stochastic control is a very active area of research. This monograph, written by two leading authorities in the field, has been updated to reflect the latest developments. It covers effective numerical methods for stochastic control problems in continuous time on two levels, that of practice and that of mathematical development. It is broadly accessible for graduate students and researchers.

Numerical Methods for Stochastic Partial Differential Equations with White Noise

Numerical Methods for Stochastic Partial Differential Equations with White Noise
Author: Zhongqiang Zhang
Publisher: Springer
Total Pages: 391
Release: 2017-09-01
Genre: Mathematics
ISBN: 3319575112

This book covers numerical methods for stochastic partial differential equations with white noise using the framework of Wong-Zakai approximation. The book begins with some motivational and background material in the introductory chapters and is divided into three parts. Part I covers numerical stochastic ordinary differential equations. Here the authors start with numerical methods for SDEs with delay using the Wong-Zakai approximation and finite difference in time. Part II covers temporal white noise. Here the authors consider SPDEs as PDEs driven by white noise, where discretization of white noise (Brownian motion) leads to PDEs with smooth noise, which can then be treated by numerical methods for PDEs. In this part, recursive algorithms based on Wiener chaos expansion and stochastic collocation methods are presented for linear stochastic advection-diffusion-reaction equations. In addition, stochastic Euler equations are exploited as an application of stochastic collocation methods, where a numerical comparison with other integration methods in random space is made. Part III covers spatial white noise. Here the authors discuss numerical methods for nonlinear elliptic equations as well as other equations with additive noise. Numerical methods for SPDEs with multiplicative noise are also discussed using the Wiener chaos expansion method. In addition, some SPDEs driven by non-Gaussian white noise are discussed and some model reduction methods (based on Wick-Malliavin calculus) are presented for generalized polynomial chaos expansion methods. Powerful techniques are provided for solving stochastic partial differential equations. This book can be considered as self-contained. Necessary background knowledge is presented in the appendices. Basic knowledge of probability theory and stochastic calculus is presented in Appendix A. In Appendix B some semi-analytical methods for SPDEs are presented. In Appendix C an introduction to Gauss quadrature is provided. In Appendix D, all the conclusions which are needed for proofs are presented, and in Appendix E a method to compute the convergence rate empirically is included. In addition, the authors provide a thorough review of the topics, both theoretical and computational exercises in the book with practical discussion of the effectiveness of the methods. Supporting Matlab files are made available to help illustrate some of the concepts further. Bibliographic notes are included at the end of each chapter. This book serves as a reference for graduate students and researchers in the mathematical sciences who would like to understand state-of-the-art numerical methods for stochastic partial differential equations with white noise.

Backward Stochastic Differential Equations

Backward Stochastic Differential Equations
Author: N El Karoui
Publisher: CRC Press
Total Pages: 236
Release: 1997-01-17
Genre: Mathematics
ISBN: 9780582307339

This book presents the texts of seminars presented during the years 1995 and 1996 at the Université Paris VI and is the first attempt to present a survey on this subject. Starting from the classical conditions for existence and unicity of a solution in the most simple case-which requires more than basic stochartic calculus-several refinements on the hypotheses are introduced to obtain more general results.

Estimation and Control Problems for Stochastic Partial Differential Equations

Estimation and Control Problems for Stochastic Partial Differential Equations
Author: Pavel S. Knopov
Publisher: Springer Science & Business Media
Total Pages: 191
Release: 2013-09-17
Genre: Mathematics
ISBN: 1461482860

Focusing on research surrounding aspects of insufficiently studied problems of estimation and optimal control of random fields, this book exposes some important aspects of those fields for systems modeled by stochastic partial differential equations. It contains many results of interest to specialists in both the theory of random fields and optimal control theory who use modern mathematical tools for resolving specific applied problems, and presents research that has not previously been covered. More generally, this book is intended for scientists, graduate, and post-graduates specializing in probability theory and mathematical statistics. The models presented describe many processes in turbulence theory, fluid mechanics, hydrology, astronomy, and meteorology, and are widely used in pattern recognition theory and parameter identification of stochastic systems. Therefore, this book may also be useful to applied mathematicians who use probability and statistical methods in the selection of useful signals subject to noise, hypothesis distinguishing, distributed parameter systems optimal control, and more. Material presented in this monograph can be used for education courses on the estimation and control theory of random fields.

Numerical Methods for Optimal Stochastic Control in Finance

Numerical Methods for Optimal Stochastic Control in Finance
Author: Zhuliang Chen
Publisher:
Total Pages: 231
Release: 2008
Genre:
ISBN: 9780494432501

In this thesis, we develop partial differential equation (PDE) based numerical methods to solve certain optimal stochastic control problems in finance. The value of a stochastic control problem is normally identical to the viscosity solution of a Hamilton-Jacobi-Bellman (HJB) equation or an HJB variational inequality. The HJB equation corresponds to the case when the controls are bounded while the HJB variational inequality corresponds to the unbounded control case. As a result, the solution to the stochastic control problem can be computed by solving the corresponding HJB equation/variational inequality as long as the convergence to the viscosity solution is guaranteed. We develop a unified numerical scheme based on a semi-Lagrangian timestepping for solving both the bounded and unbounded stochastic control problems as well as the discrete cases where the controls are allowed only at discrete times. Our scheme has the following useful properties: it is unconditionally stable; it can be shown rigorously to converge to the viscosity solution; it can easily handle various stochastic models such as jump diffusion and regime-switching models; it avoids Policy type iterations at each mesh node at each timestep which is required by the standard implicit finite difference methods. In this thesis, we demonstrate the properties of our scheme by valuing natural gas storage facilities---a bounded stochastic control problem, and pricing variable annuities with guaranteed minimum withdrawal benefits (GMWBs)---an unbounded stochastic control problem. In particular, we use an impulse control formulation for the unbounded stochastic control problem and show that the impulse control formulation is more general than the singular control formulation previously used to price GMWB contracts.

Numerical Approximations of Stochastic Maxwell Equations

Numerical Approximations of Stochastic Maxwell Equations
Author: Chuchu Chen
Publisher: Springer Nature
Total Pages: 293
Release: 2024-01-04
Genre: Mathematics
ISBN: 9819966868

The stochastic Maxwell equations play an essential role in many fields, including fluctuational electrodynamics, statistical radiophysics, integrated circuits, and stochastic inverse problems. This book provides some recent advances in the investigation of numerical approximations of the stochastic Maxwell equations via structure-preserving algorithms. It presents an accessible overview of the construction and analysis of structure-preserving algorithms with an emphasis on the preservation of geometric structures, physical properties, and asymptotic behaviors of the stochastic Maxwell equations. A friendly introduction to the simulation of the stochastic Maxwell equations with some structure-preserving algorithms is provided using MATLAB for the reader’s convenience. The objects considered in this book are related to several fascinating mathematical fields: numerical analysis, stochastic analysis, (multi-)symplectic geometry, large deviations principle, ergodic theory, partial differential equation, probability theory, etc. This book will appeal to researchers who are interested in these topics.

Numerical Methods for Stochastic Partial Differential Equations

Numerical Methods for Stochastic Partial Differential Equations
Author:
Publisher:
Total Pages: 5
Release: 1999
Genre:
ISBN:

This is the final report of a Laboratory Directed Research and Development (LDRD) project at the Los Alamos National laboratory (LANL). The objectives of this proposal were (1) the development of methods for understanding and control of spacetime discretization errors in nonlinear stochastic partial differential equations, and (2) the development of new and improved practical numerical methods for the solutions of these equations. The authors have succeeded in establishing two methods for error control: the functional Fokker-Planck equation for calculating the time discretization error and the transfer integral method for calculating the spatial discretization error. In addition they have developed a new second-order stochastic algorithm for multiplicative noise applicable to the case of colored noises, and which requires only a single random sequence generation per time step. All of these results have been verified via high-resolution numerical simulations and have been successfully applied to physical test cases. They have also made substantial progress on a longstanding problem in the dynamics of unstable fluid interfaces in porous media. This work has lead to highly accurate quasi-analytic solutions of idealized versions of this problem. These may be of use in benchmarking numerical solutions of the full stochastic PDEs that govern real-world problems.

Stochastic Numerics for Mathematical Physics

Stochastic Numerics for Mathematical Physics
Author: Grigori Noah Milstein
Publisher: Springer
Total Pages: 0
Release: 2010-12-01
Genre: Science
ISBN: 9783642059308

Stochastic differential equations have many applications in the natural sciences. Besides, the employment of probabilistic representations together with the Monte Carlo technique allows us to reduce solution of multi-dimensional problems for partial differential equations to integration of stochastic equations. This approach leads to powerful computational mathematics that is presented in the treatise. The authors propose many new special schemes, some published here for the first time. In the second part of the book they construct numerical methods for solving complicated problems for partial differential equations occurring in practical applications, both linear and nonlinear. All the methods are presented with proofs and hence founded on rigorous reasoning, thus giving the book textbook potential. An overwhelming majority of the methods are accompanied by the corresponding numerical algorithms which are ready for implementation in practice. The book addresses researchers and graduate students in numerical analysis, physics, chemistry, and engineering as well as mathematical biology and financial mathematics.