Sparse Polynomial Approximation of High-Dimensional Functions

Sparse Polynomial Approximation of High-Dimensional Functions
Author: Ben Adcock
Publisher: SIAM
Total Pages: 310
Release: 2022-02-16
Genre: Mathematics
ISBN: 161197688X

Over seventy years ago, Richard Bellman coined the term “the curse of dimensionality” to describe phenomena and computational challenges that arise in high dimensions. These challenges, in tandem with the ubiquity of high-dimensional functions in real-world applications, have led to a lengthy, focused research effort on high-dimensional approximation—that is, the development of methods for approximating functions of many variables accurately and efficiently from data. This book provides an in-depth treatment of one of the latest installments in this long and ongoing story: sparse polynomial approximation methods. These methods have emerged as useful tools for various high-dimensional approximation tasks arising in a range of applications in computational science and engineering. It begins with a comprehensive overview of best s-term polynomial approximation theory for holomorphic, high-dimensional functions, as well as a detailed survey of applications to parametric differential equations. It then describes methods for computing sparse polynomial approximations, focusing on least squares and compressed sensing techniques. Sparse Polynomial Approximation of High-Dimensional Functions presents the first comprehensive and unified treatment of polynomial approximation techniques that can mitigate the curse of dimensionality in high-dimensional approximation, including least squares and compressed sensing. It develops main concepts in a mathematically rigorous manner, with full proofs given wherever possible, and it contains many numerical examples, each accompanied by downloadable code. The authors provide an extensive bibliography of over 350 relevant references, with an additional annotated bibliography available on the book’s companion website (www.sparse-hd-book.com). This text is aimed at graduate students, postdoctoral fellows, and researchers in mathematics, computer science, and engineering who are interested in high-dimensional polynomial approximation techniques.

Statistical Foundations of Data Science

Statistical Foundations of Data Science
Author: Jianqing Fan
Publisher: CRC Press
Total Pages: 752
Release: 2020-09-21
Genre: Mathematics
ISBN: 1466510854

Statistical Foundations of Data Science gives a thorough introduction to commonly used statistical models, contemporary statistical machine learning techniques and algorithms, along with their mathematical insights and statistical theories. It aims to serve as a graduate-level textbook and a research monograph on high-dimensional statistics, sparsity and covariance learning, machine learning, and statistical inference. It includes ample exercises that involve both theoretical studies as well as empirical applications. The book begins with an introduction to the stylized features of big data and their impacts on statistical analysis. It then introduces multiple linear regression and expands the techniques of model building via nonparametric regression and kernel tricks. It provides a comprehensive account on sparsity explorations and model selections for multiple regression, generalized linear models, quantile regression, robust regression, hazards regression, among others. High-dimensional inference is also thoroughly addressed and so is feature screening. The book also provides a comprehensive account on high-dimensional covariance estimation, learning latent factors and hidden structures, as well as their applications to statistical estimation, inference, prediction and machine learning problems. It also introduces thoroughly statistical machine learning theory and methods for classification, clustering, and prediction. These include CART, random forests, boosting, support vector machines, clustering algorithms, sparse PCA, and deep learning.

Handbook of Quantile Regression

Handbook of Quantile Regression
Author: Roger Koenker
Publisher: CRC Press
Total Pages: 463
Release: 2017-10-12
Genre: Mathematics
ISBN: 1498725295

Quantile regression constitutes an ensemble of statistical techniques intended to estimate and draw inferences about conditional quantile functions. Median regression, as introduced in the 18th century by Boscovich and Laplace, is a special case. In contrast to conventional mean regression that minimizes sums of squared residuals, median regression minimizes sums of absolute residuals; quantile regression simply replaces symmetric absolute loss by asymmetric linear loss. Since its introduction in the 1970's by Koenker and Bassett, quantile regression has been gradually extended to a wide variety of data analytic settings including time series, survival analysis, and longitudinal data. By focusing attention on local slices of the conditional distribution of response variables it is capable of providing a more complete, more nuanced view of heterogeneous covariate effects. Applications of quantile regression can now be found throughout the sciences, including astrophysics, chemistry, ecology, economics, finance, genomics, medicine, and meteorology. Software for quantile regression is now widely available in all the major statistical computing environments. The objective of this volume is to provide a comprehensive review of recent developments of quantile regression methodology illustrating its applicability in a wide range of scientific settings. The intended audience of the volume is researchers and graduate students across a diverse set of disciplines.

Compressive Imaging: Structure, Sampling, Learning

Compressive Imaging: Structure, Sampling, Learning
Author: Ben Adcock
Publisher: Cambridge University Press
Total Pages: 620
Release: 2021-09-16
Genre: Computers
ISBN: 1108383912

Accurate, robust and fast image reconstruction is a critical task in many scientific, industrial and medical applications. Over the last decade, image reconstruction has been revolutionized by the rise of compressive imaging. It has fundamentally changed the way modern image reconstruction is performed. This in-depth treatment of the subject commences with a practical introduction to compressive imaging, supplemented with examples and downloadable code, intended for readers without extensive background in the subject. Next, it introduces core topics in compressive imaging – including compressed sensing, wavelets and optimization – in a concise yet rigorous way, before providing a detailed treatment of the mathematics of compressive imaging. The final part is devoted to recent trends in compressive imaging: deep learning and neural networks. With an eye to the next decade of imaging research, and using both empirical and mathematical insights, it examines the potential benefits and the pitfalls of these latest approaches.

Post-Shrinkage Strategies in Statistical and Machine Learning for High Dimensional Data

Post-Shrinkage Strategies in Statistical and Machine Learning for High Dimensional Data
Author: Syed Ejaz Ahmed
Publisher: CRC Press
Total Pages: 409
Release: 2023-05-25
Genre: Business & Economics
ISBN: 1000876659

This book presents some post-estimation and predictions strategies for the host of useful statistical models with applications in data science. It combines statistical learning and machine learning techniques in a unique and optimal way. It is well-known that machine learning methods are subject to many issues relating to bias, and consequently the mean squared error and prediction error may explode. For this reason, we suggest shrinkage strategies to control the bias by combining a submodel selected by a penalized method with a model with many features. Further, the suggested shrinkage methodology can be successfully implemented for high dimensional data analysis. Many researchers in statistics and medical sciences work with big data. They need to analyse this data through statistical modelling. Estimating the model parameters accurately is an important part of the data analysis. This book may be a repository for developing improve estimation strategies for statisticians. This book will help researchers and practitioners for their teaching and advanced research, and is an excellent textbook for advanced undergraduate and graduate courses involving shrinkage, statistical, and machine learning. The book succinctly reveals the bias inherited in machine learning method and successfully provides tools, tricks and tips to deal with the bias issue. Expertly sheds light on the fundamental reasoning for model selection and post estimation using shrinkage and related strategies. This presentation is fundamental, because shrinkage and other methods appropriate for model selection and estimation problems and there is a growing interest in this area to fill the gap between competitive strategies. Application of these strategies to real life data set from many walks of life. Analytical results are fully corroborated by numerical work and numerous worked examples are included in each chapter with numerous graphs for data visualization. The presentation and style of the book clearly makes it accessible to a broad audience. It offers rich, concise expositions of each strategy and clearly describes how to use each estimation strategy for the problem at hand. This book emphasizes that statistics/statisticians can play a dominant role in solving Big Data problems, and will put them on the precipice of scientific discovery. The book contributes novel methodologies for HDDA and will open a door for continued research in this hot area. The practical impact of the proposed work stems from wide applications. The developed computational packages will aid in analyzing a broad range of applications in many walks of life.

Log-Linear Models, Extensions, and Applications

Log-Linear Models, Extensions, and Applications
Author: Aleksandr Aravkin
Publisher: MIT Press
Total Pages: 215
Release: 2018-11-27
Genre: Computers
ISBN: 0262039508

Advances in training models with log-linear structures, with topics including variable selection, the geometry of neural nets, and applications. Log-linear models play a key role in modern big data and machine learning applications. From simple binary classification models through partition functions, conditional random fields, and neural nets, log-linear structure is closely related to performance in certain applications and influences fitting techniques used to train models. This volume covers recent advances in training models with log-linear structures, covering the underlying geometry, optimization techniques, and multiple applications. The first chapter shows readers the inner workings of machine learning, providing insights into the geometry of log-linear and neural net models. The other chapters range from introductory material to optimization techniques to involved use cases. The book, which grew out of a NIPS workshop, is suitable for graduate students doing research in machine learning, in particular deep learning, variable selection, and applications to speech recognition. The contributors come from academia and industry, allowing readers to view the field from both perspectives. Contributors Aleksandr Aravkin, Avishy Carmi, Guillermo A. Cecchi, Anna Choromanska, Li Deng, Xinwei Deng, Jean Honorio, Tony Jebara, Huijing Jiang, Dimitri Kanevsky, Brian Kingsbury, Fabrice Lambert, Aurélie C. Lozano, Daniel Moskovich, Yuriy S. Polyakov, Bhuvana Ramabhadran, Irina Rish, Dimitris Samaras, Tara N. Sainath, Hagen Soltau, Serge F. Timashev, Ewout van den Berg

Proceedings of the Thirteenth International Conference on Management Science and Engineering Management

Proceedings of the Thirteenth International Conference on Management Science and Engineering Management
Author: Jiuping Xu
Publisher: Springer
Total Pages: 837
Release: 2019-06-19
Genre: Technology & Engineering
ISBN: 3030212483

This book gathers the proceedings of the 13th International Conference on Management Science and Engineering Management (ICMSEM 2019), which was held at Brock University, Ontario, Canada on August 5–8, 2019. Exploring the latest ideas and pioneering research achievements in management science and engineering management, the respective contributions highlight both theoretical and practical studies on management science and computing methodologies, and present advanced management concepts and computing technologies for decision-making problems involving large, uncertain and unstructured data. Accordingly, the proceedings offer researchers and practitioners in related fields an essential update, as well as a source of new research directions.

Statistics for High-Dimensional Data

Statistics for High-Dimensional Data
Author: Peter Bühlmann
Publisher: Springer Science & Business Media
Total Pages: 568
Release: 2011-06-08
Genre: Mathematics
ISBN: 364220192X

Modern statistics deals with large and complex data sets, and consequently with models containing a large number of parameters. This book presents a detailed account of recently developed approaches, including the Lasso and versions of it for various models, boosting methods, undirected graphical modeling, and procedures controlling false positive selections. A special characteristic of the book is that it contains comprehensive mathematical theory on high-dimensional statistics combined with methodology, algorithms and illustrations with real data examples. This in-depth approach highlights the methods’ great potential and practical applicability in a variety of settings. As such, it is a valuable resource for researchers, graduate students and experts in statistics, applied mathematics and computer science.

Neural Information Processing

Neural Information Processing
Author: Akira Hirose
Publisher: Springer
Total Pages: 646
Release: 2016-09-30
Genre: Computers
ISBN: 3319466879

The four volume set LNCS 9947, LNCS 9948, LNCS 9949, and LNCS 9950 constitutes the proceedings of the 23rd International Conference on Neural Information Processing, ICONIP 2016, held in Kyoto, Japan, in October 2016. The 296 full papers presented were carefully reviewed and selected from 431 submissions. The 4 volumes are organized in topical sections on deep and reinforcement learning; big data analysis; neural data analysis; robotics and control; bio-inspired/energy efficient information processing; whole brain architecture; neurodynamics; bioinformatics; biomedical engineering; data mining and cybersecurity workshop; machine learning; neuromorphic hardware; sensory perception; pattern recognition; social networks; brain-machine interface; computer vision; time series analysis; data-driven approach for extracting latent features; topological and graph based clustering methods; computational intelligence; data mining; deep neural networks; computational and cognitive neurosciences; theory and algorithms.