The U.S. electrical grid is in a transformation from centralized generation sources and unidirectional flow of power, to distributed networks of utility-scale and on-site renewable generation, energy storage, and flexible demand. As the electrical grid adopts more intermittent renewable energy sources, the challenges to maintaining grid stability and meeting electricity demand will only increase. The variable generation of intermittent sources combined with the existing variations in daily and seasonal electricity demand could create situations where maintaining sufficient capacity and managing distribution is often infeasible. With renewable energy aside, the grid can still struggle to meet and manage peak loads, often resorting to quick-acting, dirty “peaker” plants to compensate for supply. These peak loads are not only a challenge for supply, but also require infrastructure to be sized for such capacity. Demand-side management, or demand response, incorporate the objectives and incentives for consumers to manage their own electricity demand throughout the day so as to reduce peak loads and support grid stability. The incentives for demand response participation are often provide through the dynamic pricing of electricity. By targeting cheaper prices throughout the day, consumers can minimize their energy expense while simultaneously satisfying demand response objectives. However, this coordinated use of electricity requires flexible loads, and heating, ventilation, and air conditioning systems is one such load of particular interest. Thermal inertia of buildings and favorable weather conditions allow for its flexible use, and its energy intensiveness and rising usage around the world make it an important load to consider. Although, coordinating such loads as to maintain comfortable indoor climate and satisfy demand response objectives is not so easily done, and it is a contradictory task. In this thesis we employ a deep reinforcement learning approach to thermostatic control of HVAC to maintain thermal comfort and maximize demand response participation. We utilize EnergyPlus building energy simulation as a testbed for experimentation of reinforcement learning control. However, we see that for a number of reasons this problem and environment is challenging for the reinforcement learning framework. We address and characterize these challenges encountered from experimentation. We also present a reinforcement learning framework that utilizes a native tool of EnergyPlus which allows the implementation of custom control on running simulations. This framework allows reinforcement learning researchers and practitioners to easily interface with any configured EnergyPlus building model for the experimentation of building energy control. This platform, along with the characterization of reinforcement learning in this environment, provide a baseline for accelerating further research in this space of building energy control for dynamically priced demand response participation.