The development of economics and technologies has promoted urbanization worldwide. Urbanization has brought great convenience to daily life. The fast construction of transportation facilities provides various means of transportation for everyday commuting. However, the growing traffic volume has threatened the existing transportation system by raising more traffic safety and congestion issues. Therefore, it is urgent and necessary to implement ITS with dynamic sensing and adjustment abilities. ITS shows great potential to improve traffic safety and efficiency, empowered by advanced IoT and AI. Within this system, the urban sensing and data analysis modules play an essential role in providing primary traffic information for follow-up works, including traffic prediction, operation optimization, and urban planning. Cameras and computer vision algorithms are the most popular toolkit in traffic sensing and analysis tasks. Deep learning-based computer vision algorithms have succeeded in multiple traffic sensing and analysis tasks, e.g., vehicle counting and crowd motion detection. The large-scale deployment of the sensor network and applications of deep learning algorithms significantly magnify previous methods' flaws, which hinder the further expansion of ITS. Firstly, the large-scale sensors and various tasks bring massive data and high workloads for data analysis on central servers. In contrast, annotated data for deep learning training in different tasks is insufficient, which leads to poor generalization when transferring to another application scenario. Additionally, traffic sensing faces adverse conditions with insufficient data and analysis qualities. This dissertation works on proposing efficient and robust machine learning methods for challenging traffic video sensing applications by presenting a systematic and practical workflow to optimize algorithm accuracy and efficiency. This dissertation first considers the high data volume challenge by designing a compression and knowledge distillation pipeline to reduce the model complexity and maintain accuracy. After applying the proposed pipeline, it is possible to further use the optimized algorithm on edge devices. This pipeline also works as the optimization foundation in the remaining works of this dissertation. Besides high data volume for analysis, insufficient training data is a considerable problem when deploying deep learning in practice. This dissertation has focused on two representative scenarios related to public safety – detecting and tracking small-scale persons in crowds and detecting rare objects in autonomous driving. Data augmentation and FSL strategies have been applied to increase the robustness of the machine learning system with limited training data. Finally, traffic sensing targets 24/7 stable operation, even in adverse conditions that reduce visibility and increase image noise with the RGB camera. Sensor fusion by combining RGB and infrared cameras is studied to improve accuracy in all light conditions. In conclusion, urbanization has simultaneously brought opportunities and challenges to the transportation system. ITS shows great potential to take this development chance and handle these challenges. This dissertation works on three data-oriented challenges and improves the accuracy and efficiency of vision-based traffic sensing algorithms. Several ITS applications are explored to demonstrate the effectiveness of the proposed methods, which achieve state-of-the-art accuracy and are far more efficient. In the future, additional research works can be explored based on this dissertation. With the continuing expansion of the sensor network, edge computing will be a more suitable system framework than cloud computing. Binary quantization and hardware-specific operator optimization can contribute to edge computing. Since data insufficiency is common in other transportation applications besides traffic detection, FSL will elevate traffic pattern forecasting and event analysis with a sequence model. For crowd monitoring, the next step will be motion prediction in bird's-eye view based on motion detection results.