Emerging and existing applications in wireless communications, queueing networks, biological models, financial engineering, and social networks demand the mathematical modeling and analysis of hybrid models in which continuous dynamics and discrete events coexist. Assuming that the systems are in continuous times, stemming from stochastic-differential-equation-based models and random discrete events, switching diffusions come into being. In such systems, continuous states and discrete events (discrete states) coexist and interact. A switching diffusion is a two-component process $(X(t),\alpha(t))$, a continuous component and a discrete component taking values in a discrete set (a set consisting of isolated points). When the discrete component takes a value $i$ (i.e., $\alpha(t)=i$), the continuous component $X(t)$ evolves according to the diffusion process whose drift and diffusion coefficients depend on $i$. Until very recently, in most of the literature $\alpha(t)$ was assumed to be a process taking values in a finite set, and that the switching rates of $\alpha(t)$ are either independent or depend only on the current state of $X(t)$. To be able to treat more realistic models and to broaden the applicability, this dissertation undertakes the task of investigating the dynamics of $(X(t),\alpha(t))$ in a much more general setting in which $\alpha(t)$ has a countable state space and its switching intensities depend on the history of the continuous component $X(t)$. We systematically established important properties of this system: well-posedness, the Markov Feller property, and the recurrence and ergodicity of the associated function-valued process. We have also studied several types of stability for the system.