A unified methodology for categorizing various complex objects is presented in this book. Through probability theory, novel asymptotically minimax criteria suitable for practical applications in imaging and data analysis are examined including the special cases such as the Jensen-Shannon divergence and the probabilistic neural network. An optimal approximate nearest neighbor search algorithm, which allows faster classification of databases is featured. Rough set theory, sequential analysis and granular computing are used to improve performance of the hierarchical classifiers. Practical examples in face identification (including deep neural networks), isolated commands recognition in voice control system and classification of visemes captured by the Kinect depth camera are included. This approach creates fast and accurate search procedures by using exact probability densities of applied dissimilarity measures. This book can be used as a guide for independent study and as supplementary material for a technically oriented graduate course in intelligent systems and data mining. Students and researchers interested in the theoretical and practical aspects of intelligent classification systems will find answers to: - Why conventional implementation of the naive Bayesian approach does not work well in image classification? - How to deal with insufficient performance of hierarchical classification systems? - Is it possible to prevent an exhaustive search of the nearest neighbor in a database?