Sparse Optimization Theory and Methods

Sparse Optimization Theory and Methods
Author: Yun-Bin Zhao
Publisher: CRC Press
Total Pages: 222
Release: 2018-07-04
Genre: Business & Economics
ISBN: 1351624148

Seeking sparse solutions of underdetermined linear systems is required in many areas of engineering and science such as signal and image processing. The efficient sparse representation becomes central in various big or high-dimensional data processing, yielding fruitful theoretical and realistic results in these fields. The mathematical optimization plays a fundamentally important role in the development of these results and acts as the mainstream numerical algorithms for the sparsity-seeking problems arising from big-data processing, compressed sensing, statistical learning, computer vision, and so on. This has attracted the interest of many researchers at the interface of engineering, mathematics and computer science. Sparse Optimization Theory and Methods presents the state of the art in theory and algorithms for signal recovery under the sparsity assumption. The up-to-date uniqueness conditions for the sparsest solution of underdertemined linear systems are described. The results for sparse signal recovery under the matrix property called range space property (RSP) are introduced, which is a deep and mild condition for the sparse signal to be recovered by convex optimization methods. This framework is generalized to 1-bit compressed sensing, leading to a novel sign recovery theory in this area. Two efficient sparsity-seeking algorithms, reweighted l1-minimization in primal space and the algorithm based on complementary slackness property, are presented. The theoretical efficiency of these algorithms is rigorously analysed in this book. Under the RSP assumption, the author also provides a novel and unified stability analysis for several popular optimization methods for sparse signal recovery, including l1-mininization, Dantzig selector and LASSO. This book incorporates recent development and the author’s latest research in the field that have not appeared in other books.

Parameterized and Exact Computation

Parameterized and Exact Computation
Author: Dániel Marx
Publisher: Springer Science & Business Media
Total Pages: 280
Release: 2012-03-16
Genre: Computers
ISBN: 3642280498

This book constitutes the thoroughly refereed post-conference proceedings of the 6th International Symposium on Parameterized and Exact Computation, IPEC 2011, in Saarbrücken, Germany, in September 2011. The 21 revised full papers presented were carefully reviewed and selected from 40 submissions. The topics addressed cover research in all aspects of parameterized and exact computation and complexity, including but not limited to new techniques for the design and analysis of parameterized and exact algorithms, fixed-parameter tractability results, parameterized complexity theory, relationship between parameterized complexity and traditional complexity classifications, applications of parameterized and exact computation, and implementation issues of parameterized and exact algorithms.

Image Understanding using Sparse Representations

Image Understanding using Sparse Representations
Author: Jayaraman J. Thiagarajan
Publisher: Springer Nature
Total Pages: 115
Release: 2022-06-01
Genre: Technology & Engineering
ISBN: 3031022505

Image understanding has been playing an increasingly crucial role in several inverse problems and computer vision. Sparse models form an important component in image understanding, since they emulate the activity of neural receptors in the primary visual cortex of the human brain. Sparse methods have been utilized in several learning problems because of their ability to provide parsimonious, interpretable, and efficient models. Exploiting the sparsity of natural signals has led to advances in several application areas including image compression, denoising, inpainting, compressed sensing, blind source separation, super-resolution, and classification. The primary goal of this book is to present the theory and algorithmic considerations in using sparse models for image understanding and computer vision applications. To this end, algorithms for obtaining sparse representations and their performance guarantees are discussed in the initial chapters. Furthermore, approaches for designing overcomplete, data-adapted dictionaries to model natural images are described. The development of theory behind dictionary learning involves exploring its connection to unsupervised clustering and analyzing its generalization characteristics using principles from statistical learning theory. An exciting application area that has benefited extensively from the theory of sparse representations is compressed sensing of image and video data. Theory and algorithms pertinent to measurement design, recovery, and model-based compressed sensing are presented. The paradigm of sparse models, when suitably integrated with powerful machine learning frameworks, can lead to advances in computer vision applications such as object recognition, clustering, segmentation, and activity recognition. Frameworks that enhance the performance of sparse models in such applications by imposing constraints based on the prior discriminatory information and the underlying geometrical structure, and kernelizing the sparse coding and dictionary learning methods are presented. In addition to presenting theoretical fundamentals in sparse learning, this book provides a platform for interested readers to explore the vastly growing application domains of sparse representations.

Convex Optimization & Euclidean Distance Geometry

Convex Optimization & Euclidean Distance Geometry
Author: Jon Dattorro
Publisher: Meboo Publishing USA
Total Pages: 776
Release: 2005
Genre: Mathematics
ISBN: 0976401304

The study of Euclidean distance matrices (EDMs) fundamentally asks what can be known geometrically given onlydistance information between points in Euclidean space. Each point may represent simply locationor, abstractly, any entity expressible as a vector in finite-dimensional Euclidean space.The answer to the question posed is that very much can be known about the points;the mathematics of this combined study of geometry and optimization is rich and deep.Throughout we cite beacons of historical accomplishment.The application of EDMs has already proven invaluable in discerning biological molecular conformation.The emerging practice of localization in wireless sensor networks, the global positioning system (GPS), and distance-based pattern recognitionwill certainly simplify and benefit from this theory.We study the pervasive convex Euclidean bodies and their various representations.In particular, we make convex polyhedra, cones, and dual cones more visceral through illustration, andwe study the geometric relation of polyhedral cones to nonorthogonal bases biorthogonal expansion.We explain conversion between halfspace- and vertex-descriptions of convex cones,we provide formulae for determining dual cones,and we show how classic alternative systems of linear inequalities or linear matrix inequalities and optimality conditions can be explained by generalized inequalities in terms of convex cones and their duals.The conic analogue to linear independence, called conic independence, is introducedas a new tool in the study of classical cone theory; the logical next step in the progression:linear, affine, conic.Any convex optimization problem has geometric interpretation.This is a powerful attraction: the ability to visualize geometry of an optimization problem.We provide tools to make visualization easier.The concept of faces, extreme points, and extreme directions of convex Euclidean bodiesis explained here, crucial to understanding convex optimization.The convex cone of positive semidefinite matrices, in particular, is studied in depth.We mathematically interpret, for example,its inverse image under affine transformation, and we explainhow higher-rank subsets of its boundary united with its interior are convex.The Chapter on "Geometry of convex functions",observes analogies between convex sets and functions:The set of all vector-valued convex functions is a closed convex cone.Included among the examples in this chapter, we show how the real affinefunction relates to convex functions as the hyperplane relates to convex sets.Here, also, pertinent results formultidimensional convex functions are presented that are largely ignored in the literature;tricks and tips for determining their convexityand discerning their geometry, particularly with regard to matrix calculus which remains largely unsystematizedwhen compared with the traditional practice of ordinary calculus.Consequently, we collect some results of matrix differentiation in the appendices.The Euclidean distance matrix (EDM) is studied,its properties and relationship to both positive semidefinite and Gram matrices.We relate the EDM to the four classical axioms of the Euclidean metric;thereby, observing the existence of an infinity of axioms of the Euclidean metric beyondthe triangle inequality. We proceed byderiving the fifth Euclidean axiom and then explain why furthering this endeavoris inefficient because the ensuing criteria (while describing polyhedra)grow linearly in complexity and number.Some geometrical problems solvable via EDMs,EDM problems posed as convex optimization, and methods of solution arepresented;\eg, we generate a recognizable isotonic map of the United States usingonly comparative distance information (no distance information, only distance inequalities).We offer a new proof of the classic Schoenberg criterion, that determines whether a candidate matrix is an EDM. Our proofrelies on fundamental geometry; assuming, any EDM must correspond to a list of points contained in some polyhedron(possibly at its vertices) and vice versa.It is not widely known that the Schoenberg criterion implies nonnegativity of the EDM entries; proved here.We characterize the eigenvalues of an EDM matrix and then devisea polyhedral cone required for determining membership of a candidate matrix(in Cayley-Menger form) to the convex cone of Euclidean distance matrices (EDM cone); \ie,a candidate is an EDM if and only if its eigenspectrum belongs to a spectral cone for EDM^N.We will see spectral cones are not unique.In the chapter "EDM cone", we explain the geometric relationship betweenthe EDM cone, two positive semidefinite cones, and the elliptope.We illustrate geometric requirements, in particular, for projection of a candidate matrixon a positive semidefinite cone that establish its membership to the EDM cone. The faces of the EDM cone are described,but still open is the question whether all its faces are exposed as they are for the positive semidefinite cone.The classic Schoenberg criterion, relating EDM and positive semidefinite cones, isrevealed to be a discretized membership relation (a generalized inequality, a new Farkas''''''''-like lemma)between the EDM cone and its ordinary dual. A matrix criterion for membership to the dual EDM cone is derived thatis simpler than the Schoenberg criterion.We derive a new concise expression for the EDM cone and its dual involvingtwo subspaces and a positive semidefinite cone."Semidefinite programming" is reviewedwith particular attention to optimality conditionsof prototypical primal and dual conic programs,their interplay, and the perturbation method of rank reduction of optimal solutions(extant but not well-known).We show how to solve a ubiquitous platonic combinatorial optimization problem from linear algebra(the optimal Boolean solution x to Ax=b)via semidefinite program relaxation.A three-dimensional polyhedral analogue for the positive semidefinite cone of 3X3 symmetricmatrices is introduced; a tool for visualizing in 6 dimensions.In "EDM proximity"we explore methods of solution to a few fundamental and prevalentEuclidean distance matrix proximity problems; the problem of finding that Euclidean distance matrix closestto a given matrix in the Euclidean sense.We pay particular attention to the problem when compounded with rank minimization.We offer a new geometrical proof of a famous result discovered by Eckart \& Young in 1936 regarding Euclideanprojection of a point on a subset of the positive semidefinite cone comprising all positive semidefinite matriceshaving rank not exceeding a prescribed limit rho.We explain how this problem is transformed to a convex optimization for any rank rho.

Blind Source Separation

Blind Source Separation
Author: Yong Xiang
Publisher: Springer
Total Pages: 101
Release: 2014-09-16
Genre: Technology & Engineering
ISBN: 9812872272

This book provides readers a complete and self-contained set of knowledge about dependent source separation, including the latest development in this field. The book gives an overview on blind source separation where three promising blind separation techniques that can tackle mutually correlated sources are presented. The book further focuses on the non-negativity based methods, the time-frequency analysis based methods, and the pre-coding based methods, respectively.

Coping with Complexity: Model Reduction and Data Analysis

Coping with Complexity: Model Reduction and Data Analysis
Author: Alexander N. Gorban
Publisher: Springer Science & Business Media
Total Pages: 356
Release: 2010-10-21
Genre: Mathematics
ISBN: 3642149413

This volume contains the extended version of selected talks given at the international research workshop "Coping with Complexity: Model Reduction and Data Analysis", Ambleside, UK, August 31 – September 4, 2009. The book is deliberately broad in scope and aims at promoting new ideas and methodological perspectives. The topics of the chapters range from theoretical analysis of complex and multiscale mathematical models to applications in e.g., fluid dynamics and chemical kinetics.

Compressed Sensing

Compressed Sensing
Author: Yonina C. Eldar
Publisher: Cambridge University Press
Total Pages: 557
Release: 2012-05-17
Genre: Technology & Engineering
ISBN: 1107394392

Compressed sensing is an exciting, rapidly growing field, attracting considerable attention in electrical engineering, applied mathematics, statistics and computer science. This book provides the first detailed introduction to the subject, highlighting theoretical advances and a range of applications, as well as outlining numerous remaining research challenges. After a thorough review of the basic theory, many cutting-edge techniques are presented, including advanced signal modeling, sub-Nyquist sampling of analog signals, non-asymptotic analysis of random matrices, adaptive sensing, greedy algorithms and use of graphical models. All chapters are written by leading researchers in the field, and consistent style and notation are utilized throughout. Key background information and clear definitions make this an ideal resource for researchers, graduate students and practitioners wanting to join this exciting research area. It can also serve as a supplementary textbook for courses on computer vision, coding theory, signal processing, image processing and algorithms for efficient data processing.

Convex Optimization in Normed Spaces

Convex Optimization in Normed Spaces
Author: Juan Peypouquet
Publisher: Springer
Total Pages: 132
Release: 2015-03-18
Genre: Mathematics
ISBN: 3319137107

This work is intended to serve as a guide for graduate students and researchers who wish to get acquainted with the main theoretical and practical tools for the numerical minimization of convex functions on Hilbert spaces. Therefore, it contains the main tools that are necessary to conduct independent research on the topic. It is also a concise, easy-to-follow and self-contained textbook, which may be useful for any researcher working on related fields, as well as teachers giving graduate-level courses on the topic. It will contain a thorough revision of the extant literature including both classical and state-of-the-art references.

Statistical Physics, Optimization, Inference, and Message-Passing Algorithms

Statistical Physics, Optimization, Inference, and Message-Passing Algorithms
Author: Florent Krzakala
Publisher: Oxford University Press
Total Pages: 319
Release: 2015-12-17
Genre: Science
ISBN: 0191061425

In the last decade, there have been an increasing convergence of interest and methods between theoretical physics and fields as diverse as probability, machine learning, optimization and compressed sensing. In particular, many theoretical and applied works in statistical physics and computer science have relied on the use of message passing algorithms and their connection to statistical physics of spin glasses. The aim of this book, especially adapted to PhD students, post-docs, and young researchers, is to present the background necessary for entering this fast developing field.

Handbook on Semidefinite, Conic and Polynomial Optimization

Handbook on Semidefinite, Conic and Polynomial Optimization
Author: Miguel F. Anjos
Publisher: Springer Science & Business Media
Total Pages: 955
Release: 2011-11-19
Genre: Business & Economics
ISBN: 1461407699

Semidefinite and conic optimization is a major and thriving research area within the optimization community. Although semidefinite optimization has been studied (under different names) since at least the 1940s, its importance grew immensely during the 1990s after polynomial-time interior-point methods for linear optimization were extended to solve semidefinite optimization problems. Since the beginning of the 21st century, not only has research into semidefinite and conic optimization continued unabated, but also a fruitful interaction has developed with algebraic geometry through the close connections between semidefinite matrices and polynomial optimization. This has brought about important new results and led to an even higher level of research activity. This Handbook on Semidefinite, Conic and Polynomial Optimization provides the reader with a snapshot of the state-of-the-art in the growing and mutually enriching areas of semidefinite optimization, conic optimization, and polynomial optimization. It contains a compendium of the recent research activity that has taken place in these thrilling areas, and will appeal to doctoral students, young graduates, and experienced researchers alike. The Handbook’s thirty-one chapters are organized into four parts: Theory, covering significant theoretical developments as well as the interactions between conic optimization and polynomial optimization; Algorithms, documenting the directions of current algorithmic development; Software, providing an overview of the state-of-the-art; Applications, dealing with the application areas where semidefinite and conic optimization has made a significant impact in recent years.