Mathematical Programming Methods for Decentralized POMDPs

Mathematical Programming Methods for Decentralized POMDPs
Author: Raghav Aras
Publisher:
Total Pages: 0
Release: 2008
Genre:
ISBN:

In this thesis, we study the problem of the optimal decentralized control of a partially observed Markov process over a finite horizon. The mathematical model corresponding to the problem is a decentralized POMDP (DEC-POMDP). Many problems in practice from the domains of artificial intelligence and operations research can be modeled as DEC-POMDPs. However, solving a DEC-POMDP exactly is intractable (NEXP-hard). The development of exact algorithms is necessary in order to guide the development of approximate algorithms that can scale to practical sized problems. Existing algorithms are mainly inspired from POMDP research (dynamic programming and forward search) and require an inordinate amount of time for even very small DEC-POMDPs. In this thesis, we develop a new mathematical programming based approach for exactly solving a finite horizon DEC-POMDP. We use the sequence form of a control policy in this approach. Using the sequence form, we show how the problem can be formulated as a mathematical progam with a nonlinear object and linear constraints. We thereby show how this nonlinear program can be linearized to a 0-1 mixed integer linear program (MIP). We present two different 0-1 MIPs based on two different properties of a DEC-POMDP. The computational experience of the mathematical programs presented in the thesis on four benchmark problems (MABC, MA-Tiger, Grid Meeting, Fire Fighting) shows that the time taken to find an optimal joint policy is one or two orders or magnitude lesser than the exact existing algorithms. In the problems tested, the time taken drops from several hours to a few seconds or minutes.

A Concise Introduction to Decentralized POMDPs

A Concise Introduction to Decentralized POMDPs
Author: Frans A. Oliehoek
Publisher: Springer
Total Pages: 146
Release: 2016-06-03
Genre: Computers
ISBN: 3319289292

This book introduces multiagent planning under uncertainty as formalized by decentralized partially observable Markov decision processes (Dec-POMDPs). The intended audience is researchers and graduate students working in the fields of artificial intelligence related to sequential decision making: reinforcement learning, decision-theoretic planning for single agents, classical multiagent planning, decentralized control, and operations research.

A Paradigm for Decentralized Process Modeling

A Paradigm for Decentralized Process Modeling
Author: I. Ben-Shaul
Publisher: Springer
Total Pages: 328
Release: 1995-10-31
Genre: Computers
ISBN:

A Paradigm for Decentralized Process Modeling presents a novel approach to decentralized process modeling that combines both trends and suggests a paradigm for decentralized PCEs, supporting concerted efforts among geographically-dispersed teams - each local individual or team with its own autonomous process - with emphasis on flexible control over the degree of collaboration versus autonomy provided. A key guideline in this approach is to supply abstraction mechanisms whereby pre-existing processes (or workflows) can be encapsulated and retain security of their internal artifacts and status data, while agreeing with other processes on formal interfaces through which all their interactions are conducted on intentionally shared information. This book is primarily intended to provide an in-depth discussion of decentralized process modeling and enactment technology, covering both high-level concepts and a full-blown realization of these concepts in a concrete system. Either the whole book or selected chapters could be used in a graduate course on software engineering, software process, or software development environments, or even for a course on workflow systems outside computer science (e.g., in a classical engineering department for engineering design, or in a business school for business practices or enterprise-wide management, or in the medical informatics department of a health science institution concerned with computer-assistance for managed care). Selected portions of the book, such as section 2.2 on Marvel, could also be employed as a case study in advanced undergraduate software engineering courses. A Paradigm for Decentralized Process Modeling is a valuable resource for both researchers and practitioners, particularly in software engineering, software development environments, and software process and workflow management, but also in electrical, mechanical, civil and other areas of engineering which have analogous needs for design processes, environmental support and concurrent engineering, and beyond to private and public sector workflow management and control, groupware support, and heterogeneous distributed systems in general.

Reinforcement Learning

Reinforcement Learning
Author: Marco Wiering
Publisher: Springer Science & Business Media
Total Pages: 653
Release: 2012-03-05
Genre: Technology & Engineering
ISBN: 3642276458

Reinforcement learning encompasses both a science of adaptive behavior of rational beings in uncertain environments and a computational methodology for finding optimal behaviors for challenging problems in control, optimization and adaptive behavior of intelligent agents. As a field, reinforcement learning has progressed tremendously in the past decade. The main goal of this book is to present an up-to-date series of survey articles on the main contemporary sub-fields of reinforcement learning. This includes surveys on partially observable environments, hierarchical task decompositions, relational knowledge representation and predictive state representations. Furthermore, topics such as transfer, evolutionary methods and continuous spaces in reinforcement learning are surveyed. In addition, several chapters review reinforcement learning methods in robotics, in games, and in computational neuroscience. In total seventeen different subfields are presented by mostly young experts in those areas, and together they truly represent a state-of-the-art of current reinforcement learning research. Marco Wiering works at the artificial intelligence department of the University of Groningen in the Netherlands. He has published extensively on various reinforcement learning topics. Martijn van Otterlo works in the cognitive artificial intelligence group at the Radboud University Nijmegen in The Netherlands. He has mainly focused on expressive knowledge representation in reinforcement learning settings.