In this thesis, we tackle various problems in strongly correlated electron systems, which can be addressed properly via the non-perturbative dynamical mean field theory (DMFT) approach using the continuous time quantum Monte Carlo method as an impurity solver. First, we revisit the old Nagaoka ferromagnetism problem in the U=oo Hubbard model and study the stability of the ferromagnetic state as a function of the temperature, the doping level, and the next-nearest-neighbor hopping t'. We then address the nature of the Mott transition in the two-dimensional Hubbard model at half-filling using cluster DMFT. Cluster DMFT can incorporate the short-range correlations beyond DMFT by extending the spatial range in which correlations are treated exactly to a finite cluster size. The non-local correlations reduce substantially the critical interaction U and modify the shape of the transition lines in the phase diagram. We then concentrate on the calculation of two-particle response functions from the ab initio perspective by means of computing the one-particle excitation spectrum using the combination of the density functional theory (DFT) and DMFT and extracting the two-particle irreducible vertex function from a local two-particle Green's function computed within DMFT. In particular, we derive the equations for calculating the magnetic/charge susceptibility and the pairing susceptibility in superconductivity. This approach is applied to the Hubbard model and the periodic Anderson model and we determine the phase diagram of magnetism and superconductivity in these models. We show that the superconducting phase is indeed stable near the magnetic phase where the pairing interaction mediated by spin fluctuations is dominantly enhanced. The non-local correlation effect to superconductivity is also discussed using the dual fermion approach and the dynamical vertex approximation. We finally apply the vertex function approach within DFT+DMFT to a Fe-based superconductor, BaFe2As2, and compute the dynamical magnetic susceptibility in this material. Our calculation results show a good agreement with the magnetic excitation spectra observed in a neutron scattering experiment. The response function calculation method derived in this thesis can capture both a localized and an itinerant nature of collective excitations in strongly correlated electron systems.