Uncertainty quantification has become an important task and an emerging topic in many engineering fields. Uncertainties can be caused by many factors, including inaccurate component models, the stochastic nature of some design parameters, external environmental fluctuations (e.g., temperature variation), measurement noise, and so forth. In order to enable robust engineering design and optimal decision making, efficient stochastic solvers are highly desired to quantify the effects of uncertainties on the performance of complex engineering designs. Process variations have become increasingly important in the semiconductor industry due to the shrinking of micro- and nano-scale devices. Such uncertainties have led to remarkable performance variations at both circuit and system levels, and they cannot be ignored any more in the design of nano-scale integrated circuits and microelectromechanical systems (MEMS). In order to simulate the resulting stochastic behaviors, Monte Carlo techniques have been employed in SPICE-like simulators for decades, and they still remain the mainstream techniques in this community. Despite of their ease of implementation, Monte Carlo simulators are often too time-consuming due to the huge number of repeated simulations. This thesis reports the development of several stochastic spectral methods to accelerate the uncertainty quantification of integrated circuits and MEMS. Stochastic spectral methods have emerged as a promising alternative to Monte Carlo in many engineering applications, but their performance may degrade significantly as the parameter dimensionality increases. In this work, we develop several efficient stochastic simulation algorithms for various integrated circuits and MEMS designs, including problems with both low-dimensional and high-dimensional random parameters, as well as complex systems with hierarchical design structures. The first part of this thesis reports a novel stochastic-testing circuit/MEMS simulator as well as its advanced simulation engine for radio-frequency (RF) circuits. The proposed stochastic testing can be regarded as a hybrid variant of stochastic Galerkin and stochastic collocation: it is an intrusive simulator with decoupled computation and adaptive time stepping inside the solver. As a result, our simulator gains remarkable speedup over standard stochastic spectral methods and Monte Carlo in the DC, transient and AC simulation of various analog, digital and RF integrated circuits. An advanced uncertainty quantification algorithm for the periodic steady states (or limit cycles) of analog/RF circuits is further developed by combining stochastic testing and shooting Newton. Our simulator is verified by various integrated circuits, showing 102 x to 103 x speedup over Monte Carlo when a similar level of accuracy is required. The second part of this thesis presents two approaches for hierarchical uncertainty quantification. In hierarchical uncertainty quantification, we propose to employ stochastic spectral methods at different design hierarchies to simulate efficiently complex systems. The key idea is to ignore the multiple random parameters inside each subsystem and to treat each subsystem as a single random parameter. The main difficulty is to recompute the basis functions and quadrature rules that are required for the high-level uncertainty quantification, since the density function of an obtained low-level surrogate model is generally unknown. In order to address this issue, the first proposed algorithm computes new basis functions and quadrature points in the low-level (and typically high-dimensional) parameter space. This approach is very accurate; however it may suffer from the curse of dimensionality. In order to handle high-dimensional problems, a sparse stochastic testing simulator based on analysis of variance (ANOVA) is developed to accelerate the low-level simulation. At the high-level, a fast algorithm based on tensor decompositions is proposed to compute the basis functions and Gauss quadrature points. Our algorithm is verified by some MEMS/IC co-design examples with both low-dimensional and high-dimensional (up to 184) random parameters, showing about 102 x speedup over the state-of-the-art techniques. The second proposed hierarchical uncertainty quantification technique instead constructs a density function for each subsystem by some monotonic interpolation schemes. This approach is capable of handling general low-level possibly non-smooth surrogate models, and it allows computing new basis functions and quadrature points in an analytical way. The computational techniques developed in this thesis are based on stochastic differential algebraic equations, but the results can also be applied to many other engineering problems (e.g., silicon photonics, heat transfer problems, fluid dynamics, electromagnetics and power systems). There exist lots of research opportunities in this direction. Important open problems include how to solve high-dimensional problems (by both deterministic and randomized algorithms), how to deal with discontinuous response surfaces, how to handle correlated non-Gaussian random variables, how to couple noise and random parameters in uncertainty quantification, how to deal with correlated and time-dependent subsystems in hierarchical uncertainty quantification, and so forth.