Non-convex Optimization for Machine Learning

Non-convex Optimization for Machine Learning
Author: Prateek Jain
Publisher: Foundations and Trends in Machine Learning
Total Pages: 218
Release: 2017-12-04
Genre: Machine learning
ISBN: 9781680833683

Non-convex Optimization for Machine Learning takes an in-depth look at the basics of non-convex optimization with applications to machine learning. It introduces the rich literature in this area, as well as equips the reader with the tools and techniques needed to apply and analyze simple but powerful procedures for non-convex problems. Non-convex Optimization for Machine Learning is as self-contained as possible while not losing focus of the main topic of non-convex optimization techniques. The monograph initiates the discussion with entire chapters devoted to presenting a tutorial-like treatment of basic concepts in convex analysis and optimization, as well as their non-convex counterparts. The monograph concludes with a look at four interesting applications in the areas of machine learning and signal processing, and exploring how the non-convex optimization techniques introduced earlier can be used to solve these problems. The monograph also contains, for each of the topics discussed, exercises and figures designed to engage the reader, as well as extensive bibliographic notes pointing towards classical works and recent advances. Non-convex Optimization for Machine Learning can be used for a semester-length course on the basics of non-convex optimization with applications to machine learning. On the other hand, it is also possible to cherry pick individual portions, such the chapter on sparse recovery, or the EM algorithm, for inclusion in a broader course. Several courses such as those in machine learning, optimization, and signal processing may benefit from the inclusion of such topics.

Variational and Non-variational Methods in Nonlinear Analysis and Boundary Value Problems

Variational and Non-variational Methods in Nonlinear Analysis and Boundary Value Problems
Author: Dumitru Motreanu
Publisher: Springer Science & Business Media
Total Pages: 400
Release: 2003-05-31
Genre: Mathematics
ISBN: 9781402013850

This book reflects a significant part of authors' research activity dur ing the last ten years. The present monograph is constructed on the results obtained by the authors through their direct cooperation or due to the authors separately or in cooperation with other mathematicians. All these results fit in a unitary scheme giving the structure of this work. The book is mainly addressed to researchers and scholars in Pure and Applied Mathematics, Mechanics, Physics and Engineering. We are greatly indebted to Viorica Venera Motreanu for the careful reading of the manuscript and helpful comments on important issues. We are also grateful to our Editors of Kluwer Academic Publishers for their professional assistance. Our deepest thanks go to our numerous scientific collaborators and friends, whose work was so important for us. D. Motreanu and V. Radulescu IX Introduction The present monograph is based on original results obtained by the authors in the last decade. This book provides a comprehensive expo sition of some modern topics in nonlinear analysis with applications to the study of several classes of boundary value problems. Our framework includes multivalued elliptic problems with discontinuities, variational inequalities, hemivariational inequalities and evolution problems. The treatment relies on variational methods, monotonicity principles, topo logical arguments and optimization techniques. Excepting Sections 1 and 3 in Chapter 1 and Sections 1 and 3 in Chapter 2, the material is new in comparison with any other book, representing research topics where the authors contributed. The outline of our work is the following.

Modern Nonconvex Nondifferentiable Optimization

Modern Nonconvex Nondifferentiable Optimization
Author: Ying Cui
Publisher: Society for Industrial and Applied Mathematics (SIAM)
Total Pages: 0
Release: 2022
Genre: Convex functions
ISBN: 9781611976731

"This monograph serves present and future needs where nonconvexity and nondifferentiability are inevitably present in the faithful modeling of real-world applications of optimization"--

Duality Principles in Nonconvex Systems

Duality Principles in Nonconvex Systems
Author: David Yang Gao
Publisher: Springer Science & Business Media
Total Pages: 476
Release: 2000-01-31
Genre: Mathematics
ISBN: 9780792361459

Motivated by practical problems in engineering and physics, drawing on a wide range of applied mathematical disciplines, this book is the first to provide, within a unified framework, a self-contained comprehensive mathematical theory of duality for general non-convex, non-smooth systems, with emphasis on methods and applications in engineering mechanics. Topics covered include the classical (minimax) mono-duality of convex static equilibria, the beautiful bi-duality in dynamical systems, the interesting tri-duality in non-convex problems and the complicated multi-duality in general canonical systems. A potentially powerful sequential canonical dual transformation method for solving fully nonlinear problems is developed heuristically and illustrated by use of many interesting examples as well as extensive applications in a wide variety of nonlinear systems, including differential equations, variational problems and inequalities, constrained global optimization, multi-well phase transitions, non-smooth post-bifurcation, large deformation mechanics, structural limit analysis, differential geometry and non-convex dynamical systems. With exceptionally coherent and lucid exposition, the work fills a big gap between the mathematical and engineering sciences. It shows how to use formal language and duality methods to model natural phenomena, to construct intrinsic frameworks in different fields and to provide ideas, concepts and powerful methods for solving non-convex, non-smooth problems arising naturally in engineering and science. Much of the book contains material that is new, both in its manner of presentation and in its research development. A self-contained appendix provides some necessary background from elementary functional analysis. Audience: The book will be a valuable resource for students and researchers in applied mathematics, physics, mechanics and engineering. The whole volume or selected chapters can also be recommended as a text for both senior undergraduate and graduate courses in applied mathematics, mechanics, general engineering science and other areas in which the notions of optimization and variational methods are employed.

Conjugate Gradient Algorithms in Nonconvex Optimization

Conjugate Gradient Algorithms in Nonconvex Optimization
Author: Radoslaw Pytlak
Publisher: Springer Science & Business Media
Total Pages: 493
Release: 2008-11-18
Genre: Mathematics
ISBN: 354085634X

This book details algorithms for large-scale unconstrained and bound constrained optimization. It shows optimization techniques from a conjugate gradient algorithm perspective as well as methods of shortest residuals, which have been developed by the author.

Nonsmooth Equations in Optimization

Nonsmooth Equations in Optimization
Author: Diethard Klatte
Publisher: Springer Science & Business Media
Total Pages: 351
Release: 2005-12-17
Genre: Mathematics
ISBN: 0306476169

Many questions dealing with solvability, stability and solution methods for va- ational inequalities or equilibrium, optimization and complementarity problems lead to the analysis of certain (perturbed) equations. This often requires a - formulation of the initial model being under consideration. Due to the specific of the original problem, the resulting equation is usually either not differ- tiable (even if the data of the original model are smooth), or it does not satisfy the assumptions of the classical implicit function theorem. This phenomenon is the main reason why a considerable analytical inst- ment dealing with generalized equations (i.e., with finding zeros of multivalued mappings) and nonsmooth equations (i.e., the defining functions are not c- tinuously differentiable) has been developed during the last 20 years, and that under very different viewpoints and assumptions. In this theory, the classical hypotheses of convex analysis, in particular, monotonicity and convexity, have been weakened or dropped, and the scope of possible applications seems to be quite large. Briefly, this discipline is often called nonsmooth analysis, sometimes also variational analysis. Our book fits into this discipline, however, our main intention is to develop the analytical theory in close connection with the needs of applications in optimization and related subjects. Main Topics of the Book 1. Extended analysis of Lipschitz functions and their generalized derivatives, including ”Newton maps” and regularity of multivalued mappings. 2. Principle of successive approximation under metric regularity and its - plication to implicit functions.

Nonsmooth/Nonconvex Mechanics

Nonsmooth/Nonconvex Mechanics
Author: David Yang Gao
Publisher: Springer Science & Business Media
Total Pages: 505
Release: 2013-12-01
Genre: Mathematics
ISBN: 1461302757

Nonsmooth and nonconvex models arise in several important applications of mechanics and engineering. The interest in this field is growing from both mathematicians and engineers. The study of numerous industrial applications, including contact phenomena in statics and dynamics or delamination effects in composites, require the consideration of nonsmoothness and nonconvexity. The mathematical topics discussed in this book include variational and hemivariational inequalities, duality, complementarity, variational principles, sensitivity analysis, eigenvalue and resonance problems, and minimax problems. Applications are considered in the following areas among others: nonsmooth statics and dynamics, stability of quasi- static evolution processes, friction problems, adhesive contact and debonding, inverse problems, pseudoelastic modeling of phase transitions, chaotic behavior in nonlinear beams, and nonholonomic mechanical systems. This volume contains 22 chapters written by various leading researchers and presents a cohesive and authoritative overview of recent results and applications in the area of nonsmooth and nonconvex mechanics. Audience: Faculty, graduate students, and researchers in applied mathematics, optimization, control and engineering.

Global Optimization with Non-Convex Constraints

Global Optimization with Non-Convex Constraints
Author: Roman G. Strongin
Publisher: Springer Science & Business Media
Total Pages: 717
Release: 2013-11-09
Genre: Mathematics
ISBN: 146154677X

Everything should be made as simple as possible, but not simpler. (Albert Einstein, Readers Digest, 1977) The modern practice of creating technical systems and technological processes of high effi.ciency besides the employment of new principles, new materials, new physical effects and other new solutions ( which is very traditional and plays the key role in the selection of the general structure of the object to be designed) also includes the choice of the best combination for the set of parameters (geometrical sizes, electrical and strength characteristics, etc.) concretizing this general structure, because the Variation of these parameters ( with the structure or linkage being already set defined) can essentially affect the objective performance indexes. The mathematical tools for choosing these best combinations are exactly what is this book about. With the advent of computers and the computer-aided design the pro bations of the selected variants are usually performed not for the real examples ( this may require some very expensive building of sample op tions and of the special installations to test them ), but by the analysis of the corresponding mathematical models. The sophistication of the mathematical models for the objects to be designed, which is the natu ral consequence of the raising complexity of these objects, greatly com plicates the objective performance analysis. Today, the main (and very often the only) available instrument for such an analysis is computer aided simulation of an object's behavior, based on numerical experiments with its mathematical model.