Individual driving behavior, such as anticipation, risk-taking and cooperative lane change, has significant impact on overall traffic flow characteristics and highway performance. It contributes to various traffic flow phenomena, including platooning, capacity drop and traffic oscillations. A good understanding of driving behavior under different driving environments, such as curved roads, lane-drops, merges and diverges, and platooning enabled by vehicle to vehicle communication, can help us design safer roads, and safer and more efficient autonomous or semi-autonomous driving vehicles. New car following models have been developed to capture the empirical observed anticipation and risk-taking driving behavior, and applied to investigate how anticipation and risk-taking may lead to different traffic flow phenomena and influence highway capacity and safety. Considering gap anticipation, full range traffic conditions can be reproduced, including free-flow, congestion and traffic jam under fixed and moving bottleneck, realistic flow capacities and fundamental diagrams with different levels of anticipation, as well as platoon driving when gap anticipation dependents on the gap. The effect of risk-taking on traffic safety is studied with a collision-possible car following model considering driver anticipation. Risk-taking leads to traffic oscillations and potential collision hazards when traffic is not stable. Longer length of view field can improve traffic safety, and large numbers of vehicle crashes happen when view field length is shorter than given threshold. Merge traffic dynamic has been studied by simulating of cooperative lane change, and drivers' merge location choice is studied to show its impact on traffic oscillations near merging junction. A simplified lane change cooperation strategy is developed and integrated with optimal speed car following logic to capture cooperative lane change behavior in merge junctions. This model can reproduce reasonable merge ratio, capacity drop, turn taking merging behavior and stop and go traffic at merge bottleneck. Lane change incentive and main lane traffic condition affect drivers' lane change behavior and leads to different merge location choice. Microscopic and macroscopic traffic simulation show merge location choice contributes to the formation of stop-and-go waves near merging junctions and the period of these waves are closely related to the distance between the two dominant merging locations. Theoretical and data analysis are used to reveal the correlation between drivers' anticipation, relaxation behavior and traffic hysteresis. Through an analysis of the trajectory data from NGSIM and a theoretical analysis of car-following models, it is revealed that traffic hysteresis is generated by an imbalance in driver relaxation and anticipation. By changing the strength of relaxation and anticipation, we are able to reproduce positive, negative and double hysteresis loops, as well as aggressive and timid driving behavior. It is further shown that the relative positions of acceleration and deceleration phase with respect to the equilibrium state is not unique and are determined by the comparative strength of relaxation and anticipation in different traffic conditions. This study suggests that traffic hysteresis can be suppressed by balancing driver relaxation and anticipation, and stop-and-go traffic can be smoothed by eliminating aggressive driving in congested traffic. A three-mode vehicle control law is proposed for ACC (Adaptive Cruise Control) and CACC (Cooperative Adaptive Cruise Control) and implemented in VENTOS (VEhicular NeTwork Open source Simulator). Traffic hysteresis and stability of studied both analytically and using VENTOS simulation. The ability of ACC/CACC to improve highway safety and eliminating traffic hysteresis is verified by traffic simulation under critical traffic conditions, including realistic stop-and-go traffic and worst case stopping. Through analytical approaches and simulation, we have demonstrated the stability and robustness of our proposed ACC/CACC control system against sensor measurement errors and lossy wireless communication links which is required to implement the CACC control logic. The benefit of wireless communication, even with some lossy links, is significant in ensuring stream stability and performance.