Sequential Control with Incomplete Information

Sequential Control with Incomplete Information
Author: E. L. Presman
Publisher:
Total Pages: 304
Release: 1990
Genre: Mathematics
ISBN:

This book is devoted to a specific problem in the general theory of optimal control--sequential control under conditions of incomplete information. The main results concern the case in which at each moment of (continuous or discrete) time only a finite number of controls are admissible and the results of control action conducted in a Bayesian framework are represented by realizations of random variables whose distributions for a given control correspond to one of several alternative hypotheses.**This situation is related to problems in the sequential distribution of resources with incomplete information, problems in the sequential setting of prices in the face of random demand, search problems, and so on. Similar problems are found in the general theory of statistical decisions and in the theory of planning experiments--under the name of multi-armed bandit problems and in the theory of automatic control--as problems of dual control.

Iterative Learning Control with Passive Incomplete Information

Iterative Learning Control with Passive Incomplete Information
Author: Dong Shen
Publisher: Springer
Total Pages: 298
Release: 2018-04-16
Genre: Technology & Engineering
ISBN: 9811082677

This book presents an in-depth discussion of iterative learning control (ILC) with passive incomplete information, highlighting the incomplete input and output data resulting from practical factors such as data dropout, transmission disorder, communication delay, etc.—a cutting-edge topic in connection with the practical applications of ILC. It describes in detail three data dropout models: the random sequence model, Bernoulli variable model, and Markov chain model—for both linear and nonlinear stochastic systems. Further, it proposes and analyzes two major compensation algorithms for the incomplete data, namely, the intermittent update algorithm and successive update algorithm. Incomplete information environments include random data dropout, random communication delay, random iteration-varying lengths, and other communication constraints. With numerous intuitive figures to make the content more accessible, the book explores several potential solutions to this topic, ensuring that readers are not only introduced to the latest advances in ILC for systems with random factors, but also gain an in-depth understanding of the intrinsic relationship between incomplete information environments and essential tracking performance. It is a valuable resource for academics and engineers, as well as graduate students who are interested in learning about control, data-driven control, networked control systems, and related fields.

Strategies for Sequential Search and Selection in Real Time

Strategies for Sequential Search and Selection in Real Time
Author: F. Thomas Bruss
Publisher: American Mathematical Soc.
Total Pages: 258
Release: 1992
Genre: Mathematics
ISBN: 0821851330

This volume contains the proceedings of the MS-IMS-SIAM Joint Summer Research Conference on Strategies for Sequential Search and Selection in Real Time, held in June 1990 at the University of Massachusetts at Amherst. The conference focused on problems related to sequential observation of random variables and selection of actions in real time. This book will provide readers with a feeling for the breadth and depth of contemporary research in these areas.

Sequential Stochastic Optimization

Sequential Stochastic Optimization
Author: R. Cairoli
Publisher: John Wiley & Sons
Total Pages: 348
Release: 2011-07-26
Genre: Mathematics
ISBN: 1118164407

Sequential Stochastic Optimization provides mathematicians andapplied researchers with a well-developed framework in whichstochastic optimization problems can be formulated and solved.Offering much material that is either new or has never beforeappeared in book form, it lucidly presents a unified theory ofoptimal stopping and optimal sequential control of stochasticprocesses. This book has been carefully organized so that littleprior knowledge of the subject is assumed; its only prerequisitesare a standard graduate course in probability theory and somefamiliarity with discrete-parameter martingales. Major topics covered in Sequential Stochastic Optimization include: * Fundamental notions, such as essential supremum, stopping points,accessibility, martingales and supermartingales indexed by INd * Conditions which ensure the integrability of certain suprema ofpartial sums of arrays of independent random variables * The general theory of optimal stopping for processes indexed byInd * Structural properties of information flows * Sequential sampling and the theory of optimal sequential control * Multi-armed bandits, Markov chains and optimal switching betweenrandom walks

Optimal Control of Random Sequences in Problems with Constraints

Optimal Control of Random Sequences in Problems with Constraints
Author: A.B. Piunovskiy
Publisher: Springer Science & Business Media
Total Pages: 355
Release: 2012-12-06
Genre: Mathematics
ISBN: 9401155089

Controlled stochastic processes with discrete time form a very interest ing and meaningful field of research which attracts widespread attention. At the same time these processes are used for solving of many applied problems in the queueing theory, in mathematical economics. in the theory of controlled technical systems, etc. . In this connection, methods of the theory of controlled processes constitute the every day instrument of many specialists working in the areas mentioned. The present book is devoted to the rather new area, that is, to the optimal control theory with functional constraints. This theory is close to the theory of multicriteria optimization. The compromise between the mathematical rigor and the big number of meaningful examples makes the book attractive for professional mathematicians and for specialists who ap ply mathematical methods in different specific problems. Besides. the book contains setting of many new interesting problems for further invf'stigatioll. The book can form the basis of special courses in the theory of controlled stochastic processes for students and post-graduates specializing in the ap plied mathematics and in the control theory of complex systf'ms. The grounding of graduating students of mathematical department is sufficient for the perfect understanding of all the material. The book con tains the extensive Appendix where the necessary knowledge ill Borel spaces and in convex analysis is collected. All the meaningful examples can be also understood by readers who are not deeply grounded in mathematics.

Dynamic Optimization

Dynamic Optimization
Author: Karl Hinderer
Publisher: Springer
Total Pages: 530
Release: 2017-01-12
Genre: Business & Economics
ISBN: 3319488147

This book explores discrete-time dynamic optimization and provides a detailed introduction to both deterministic and stochastic models. Covering problems with finite and infinite horizon, as well as Markov renewal programs, Bayesian control models and partially observable processes, the book focuses on the precise modelling of applications in a variety of areas, including operations research, computer science, mathematics, statistics, engineering, economics and finance. Dynamic Optimization is a carefully presented textbook which starts with discrete-time deterministic dynamic optimization problems, providing readers with the tools for sequential decision-making, before proceeding to the more complicated stochastic models. The authors present complete and simple proofs and illustrate the main results with numerous examples and exercises (without solutions). With relevant material covered in four appendices, this book is completely self-contained.

Multi-armed Bandit Allocation Indices

Multi-armed Bandit Allocation Indices
Author: John Gittins
Publisher: John Wiley & Sons
Total Pages: 233
Release: 2011-02-18
Genre: Mathematics
ISBN: 1119990211

In 1989 the first edition of this book set out Gittins' pioneering index solution to the multi-armed bandit problem and his subsequent investigation of a wide of sequential resource allocation and stochastic scheduling problems. Since then there has been a remarkable flowering of new insights, generalizations and applications, to which Glazebrook and Weber have made major contributions. This second edition brings the story up to date. There are new chapters on the achievable region approach to stochastic optimization problems, the construction of performance bounds for suboptimal policies, Whittle's restless bandits, and the use of Lagrangian relaxation in the construction and evaluation of index policies. Some of the many varied proofs of the index theorem are discussed along with the insights that they provide. Many contemporary applications are surveyed, and over 150 new references are included. Over the past 40 years the Gittins index has helped theoreticians and practitioners to address a huge variety of problems within chemometrics, economics, engineering, numerical analysis, operational research, probability, statistics and website design. This new edition will be an important resource for others wishing to use this approach.

Bandit Algorithms

Bandit Algorithms
Author: Tor Lattimore
Publisher: Cambridge University Press
Total Pages: 538
Release: 2020-07-16
Genre: Computers
ISBN: 1108687490

Decision-making in the face of uncertainty is a significant challenge in machine learning, and the multi-armed bandit model is a commonly used framework to address it. This comprehensive and rigorous introduction to the multi-armed bandit problem examines all the major settings, including stochastic, adversarial, and Bayesian frameworks. A focus on both mathematical intuition and carefully worked proofs makes this an excellent reference for established researchers and a helpful resource for graduate students in computer science, engineering, statistics, applied mathematics and economics. Linear bandits receive special attention as one of the most useful models in applications, while other chapters are dedicated to combinatorial bandits, ranking, non-stationary problems, Thompson sampling and pure exploration. The book ends with a peek into the world beyond bandits with an introduction to partial monitoring and learning in Markov decision processes.