A persistent central challenge in computational science and engineering (CSE), with both national and global security implications, is the efficient solution of large-scale Bayesian inverse problems. These problems range from estimating material parameters in subsurface simulations to estimating phenomenological parameters in climate models. Despite recent progress, our ability to quantify uncertainties and solve large-scale inverse problems lags well behind our ability to develop the governing forward simulations. Inverse problems present unique computational challenges that are only magnified as we include larger observational data sets and demand higher-resolution parameter estimates. Even with the current state-of-the-art, solving deterministic large-scale inverse problems is prohibitively expensive. Large-scale uncertainty quantification (UQ), cast in the Bayesian inversion framework, is thus rendered intractable. To conquer these challenges, new methods that target the root causes of computational complexity are needed. In this dissertation, we propose data driven strategies for overcoming this "curse of di- mensionality." First, we address the computational complexity induced in large-scale inverse problems by high-dimensional observational data. We propose a randomized misfit approach (RMA), which uses random projections--quasi-orthogonal, information-preserving transformations--to map the high-dimensional data-misfit vector to a low dimensional space. We provide the first theoretical explanation for why randomized misfit methods are successful in practice with a small reduced data-misfit dimension (n = O(1)). Next, we develop the randomized geostatistical approach (RGA) for Bayesian sub- surface inverse problems with high-dimensional data. We show that the RGA is able to resolve transient groundwater inverse problems with noisy observed data dimensions up to 107, whereas a comparison method fails due to out-of-memory errors. Finally, we address the solution of Bayesian inverse problems with spatially localized data. The motivation is CSE applications that would gain from high-fidelity estimation over a smaller data-local domain, versus expensive and uncertain estimation over the full simulation domain. We propose several truncated domain inversion methods using domain decomposition theory to build model-informed artificial boundary conditions. Numerical investigations of MAP estimation and sampling demonstrate improved fidelity and fewer partial differential equation (PDE) solves with our truncated methods.