Several branches of modern computer vision research make heavy use of machine learning techniques. Machine learning for computer vision generally deals with Euclidean data. However, with the advances in the field, mathematical objects lying in non-Euclidean spaces that can be naturally modeled as Riemannian manifolds are now commonly encountered in computer vision. Therefore, machine learning methods on Riemannian manifolds has become an interesting area of computer vision research. Many Euclidean machine learning methods cannot be directly utilized on data lying in a Riemannian manifold. Generalizing Euclidean methods to Riemannian manifolds is not straightforward either due to differences in geometries. This thesis targets at solving this problem of learning on manifold-valued data, by performing kernel methods on Riemannian manifolds. More specifically, we aim to introduce a superior class of learning algorithms on manifold-valued data by proposing positive definite kernels on Riemannian manifolds and by designing improved kernel learning algorithms on them. We work on a number of Riemannian manifolds encountered in computer vision research, namely, the unit n-sphere, the Riemannian manifold of symmetric positive definite matrices, the Grassmann manifold and the shape manifold. A key component in kernel methods is the positive definite kernel employed. In the earlier chapters of the thesis we introduce positive definite kernels on these manifolds while giving rigorous proofs for their positive definiteness. Being able to define positive definite kernels on these manifolds enables us to use powerful Euclidean algorithms such as support vector machines and principle component analysis on manifold-valued data. This approach significantly reduces the complexities associated with learning on manifold-valued data while simultaneously yielding much better results. In the later chapters, we tackle a more advanced problem: automatically learning the optimal kernel on a manifold for a given computer vision task. The ability to learn the optimal kernel automatically eliminates the need to manually select kernels and the risk of using a sub-optimal kernel, which can significantly degrade the performance of kernel methods. We demonstrate applications of our algorithms on a variety of computer vision tasks such as pedestrian detection, object recognition, image-set recognition, segmentation, clustering, shape recognition and shape retrieval. Experimental evaluations towards the end of each chapter provide evidence that using kernel methods on manifolds achieves superior performance compared to state-of-the-art learning methods on manifold-valued data.