Optimal Control of Continuous-time Stochastic Systems

Optimal Control of Continuous-time Stochastic Systems
Author: Richard Edgar Mortensen
Publisher:
Total Pages: 194
Release: 1966
Genre:
ISBN:

This report is concerned with determining the optimal feedback control for continuous-time, continuous-state, stochastic, nonlinear, dynamic systems when only noisy observations of the state are available. At each instant of time, the current value of the control is a functional of the entire past history of the observations. The principal mathematical apparatus used in this investigation is the following: (1) the theory of probability measures and integration on infinite dimensional function spaces, (2) the Ito stochastic calculus for differentiation and integration of random functions, (3) the Frechet derivative of a functional on an infinite dimensional function space, and (4) dynamic programming. In Sections I and II, items (1) and (2) above are used to establish rigorously sufficient conditions for the existence of a conditional probability density for the current state of the system given the entire past history of the observations. A rigorous derivation is then given of a stochastic integral equation which is obeyed by an unnormalized version of the desired conditional density. In Section III, items (3) and (4) above are used heuristically to obtain a stochastic Hamilton-Jacobi equation in function space. It is shown that the solution of this equation would yield the desired feedback control. (Author).

Numerical Methods for Stochastic Control Problems in Continuous Time

Numerical Methods for Stochastic Control Problems in Continuous Time
Author: Harold Kushner
Publisher: Springer Science & Business Media
Total Pages: 480
Release: 2013-11-27
Genre: Mathematics
ISBN: 146130007X

Stochastic control is a very active area of research. This monograph, written by two leading authorities in the field, has been updated to reflect the latest developments. It covers effective numerical methods for stochastic control problems in continuous time on two levels, that of practice and that of mathematical development. It is broadly accessible for graduate students and researchers.

Advanced Topics in Control and Estimation of State-Multiplicative Noisy Systems

Advanced Topics in Control and Estimation of State-Multiplicative Noisy Systems
Author: Eli Gershon
Publisher: Springer
Total Pages: 256
Release: 2013-03-21
Genre: Technology & Engineering
ISBN: 1447150708

Advanced Topics in Control and Estimation of State-Multiplicative Noisy Systems begins with an introduction and extensive literature survey. The text proceeds to cover the field of H∞ time-delay linear systems where the issues of stability and L2−gain are presented and solved for nominal and uncertain stochastic systems, via the input-output approach. It presents solutions to the problems of state-feedback, filtering, and measurement-feedback control for these systems, for both the continuous- and the discrete-time settings. In the continuous-time domain, the problems of reduced-order and preview tracking control are also presented and solved. The second part of the monograph concerns non-linear stochastic state- multiplicative systems and covers the issues of stability, control and estimation of the systems in the H∞ sense, for both continuous-time and discrete-time cases. The book also describes special topics such as stochastic switched systems with dwell time and peak-to-peak filtering of nonlinear stochastic systems. The reader is introduced to six practical engineering- oriented examples of noisy state-multiplicative control and filtering problems for linear and nonlinear systems. The book is rounded out by a three-part appendix containing stochastic tools necessary for a proper appreciation of the text: a basic introduction to stochastic control processes, aspects of linear matrix inequality optimization, and MATLAB codes for solving the L2-gain and state-feedback control problems of stochastic switched systems with dwell-time. Advanced Topics in Control and Estimation of State-Multiplicative Noisy Systems will be of interest to engineers engaged in control systems research and development, to graduate students specializing in stochastic control theory, and to applied mathematicians interested in control problems. The reader is expected to have some acquaintance with stochastic control theory and state-space-based optimal control theory and methods for linear and nonlinear systems.

Relative Optimization of Continuous-Time and Continuous-State Stochastic Systems

Relative Optimization of Continuous-Time and Continuous-State Stochastic Systems
Author: Xi-Ren Cao
Publisher: Springer Nature
Total Pages: 376
Release: 2020-05-13
Genre: Technology & Engineering
ISBN: 3030418464

This monograph applies the relative optimization approach to time nonhomogeneous continuous-time and continuous-state dynamic systems. The approach is intuitively clear and does not require deep knowledge of the mathematics of partial differential equations. The topics covered have the following distinguishing features: long-run average with no under-selectivity, non-smooth value functions with no viscosity solutions, diffusion processes with degenerate points, multi-class optimization with state classification, and optimization with no dynamic programming. The book begins with an introduction to relative optimization, including a comparison with the traditional approach of dynamic programming. The text then studies the Markov process, focusing on infinite-horizon optimization problems, and moves on to discuss optimal control of diffusion processes with semi-smooth value functions and degenerate points, and optimization of multi-dimensional diffusion processes. The book concludes with a brief overview of performance derivative-based optimization. Among the more important novel considerations presented are: the extension of the Hamilton–Jacobi–Bellman optimality condition from smooth to semi-smooth value functions by derivation of explicit optimality conditions at semi-smooth points and application of this result to degenerate and reflected processes; proof of semi-smoothness of the value function at degenerate points; attention to the under-selectivity issue for the long-run average and bias optimality; discussion of state classification for time nonhomogeneous continuous processes and multi-class optimization; and development of the multi-dimensional Tanaka formula for semi-smooth functions and application of this formula to stochastic control of multi-dimensional systems with degenerate points. The book will be of interest to researchers and students in the field of stochastic control and performance optimization alike.

Optimal Control and Estimation

Optimal Control and Estimation
Author: Robert F. Stengel
Publisher: Courier Corporation
Total Pages: 716
Release: 1994-09-20
Genre: Mathematics
ISBN: 9780486682006

"An excellent introduction to optimal control and estimation theory and its relationship with LQG design. . . . invaluable as a reference for those already familiar with the subject." — Automatica. This highly regarded graduate-level text provides a comprehensive introduction to optimal control theory for stochastic systems, emphasizing application of its basic concepts to real problems. The first two chapters introduce optimal control and review the mathematics of control and estimation. Chapter 3 addresses optimal control of systems that may be nonlinear and time-varying, but whose inputs and parameters are known without error. Chapter 4 of the book presents methods for estimating the dynamic states of a system that is driven by uncertain forces and is observed with random measurement error. Chapter 5 discusses the general problem of stochastic optimal control, and the concluding chapter covers linear time-invariant systems. Robert F. Stengel is Professor of Mechanical and Aerospace Engineering at Princeton University, where he directs the Topical Program on Robotics and Intelligent Systems and the Laboratory for Control and Automation. He was a principal designer of the Project Apollo Lunar Module control system. "An excellent teaching book with many examples and worked problems which would be ideal for self-study or for use in the classroom. . . . The book also has a practical orientation and would be of considerable use to people applying these techniques in practice." — Short Book Reviews, Publication of the International Statistical Institute. "An excellent book which guides the reader through most of the important concepts and techniques. . . . A useful book for students (and their teachers) and for those practicing engineers who require a comprehensive reference to the subject." — Library Reviews, The Royal Aeronautical Society.

Optimization of Stochastic Systems

Optimization of Stochastic Systems
Author: Masanao Aoki
Publisher: Academic Press
Total Pages: 374
Release: 1967-01-01
Genre: Computers
ISBN: 0080955398

Optimization of Stochastic Systems is an outgrowth of class notes of a graduate level seminar on optimization of stochastic systems. Most of the material in the book was taught for the first time during the 1965 Spring Semester while the author was visiting the Department of Electrical Engineering, University of California, Berkeley. The revised and expanded material was presented at the Department of Engineering, University of California, Los Angeles during the 1965 Fall Semester. The systems discussed in the book are mostly assumed to be of discrete-time type with continuous state variables taking values in some subsets of Euclidean spaces. There is another class of systems in which state variables are assumed to take on at most a denumerable number of values, i.e., these systems are of discrete-time discrete-space type. Although the problems associated with the latter class of systems are many and interesting, andalthough they are amenable to deep analysis on such topics as the limiting behaviors of state variables as time indexes increase to infinity, this class of systems is not included here, partly because there are many excellent books on the subjects and partly because inclusion of these materials would easily double the size of the book.

Discrete-time Stochastic Systems

Discrete-time Stochastic Systems
Author: Torsten Söderström
Publisher: Springer Science & Business Media
Total Pages: 387
Release: 2012-12-06
Genre: Mathematics
ISBN: 1447101014

This comprehensive introduction to the estimation and control of dynamic stochastic systems provides complete derivations of key results. The second edition includes improved and updated material, and a new presentation of polynomial control and new derivation of linear-quadratic-Gaussian control.

Adaptive Control Tutorial

Adaptive Control Tutorial
Author: Petros Ioannou
Publisher: SIAM
Total Pages: 401
Release: 2006-01-01
Genre: Mathematics
ISBN: 0898716152

Designed to meet the needs of a wide audience without sacrificing mathematical depth and rigor, Adaptive Control Tutorial presents the design, analysis, and application of a wide variety of algorithms that can be used to manage dynamical systems with unknown parameters. Its tutorial-style presentation of the fundamental techniques and algorithms in adaptive control make it suitable as a textbook. Adaptive Control Tutorial is designed to serve the needs of three distinct groups of readers: engineers and students interested in learning how to design, simulate, and implement parameter estimators and adaptive control schemes without having to fully understand the analytical and technical proofs; graduate students who, in addition to attaining the aforementioned objectives, also want to understand the analysis of simple schemes and get an idea of the steps involved in more complex proofs; and advanced students and researchers who want to study and understand the details of long and technical proofs with an eye toward pursuing research in adaptive control or related topics. The authors achieve these multiple objectives by enriching the book with examples demonstrating the design procedures and basic analysis steps and by detailing their proofs in both an appendix and electronically available supplementary material; online examples are also available. A solution manual for instructors can be obtained by contacting SIAM or the authors. Preface; Acknowledgements; List of Acronyms; Chapter 1: Introduction; Chapter 2: Parametric Models; Chapter 3: Parameter Identification: Continuous Time; Chapter 4: Parameter Identification: Discrete Time; Chapter 5: Continuous-Time Model Reference Adaptive Control; Chapter 6: Continuous-Time Adaptive Pole Placement Control; Chapter 7: Adaptive Control for Discrete-Time Systems; Chapter 8: Adaptive Control of Nonlinear Systems; Appendix; Bibliography; Index