Action Recognition in Continuous Data Streams Using Fusion of Depth and Inertial Sensing

Action Recognition in Continuous Data Streams Using Fusion of Depth and Inertial Sensing
Author: Neha Dawar
Publisher:
Total Pages:
Release: 2018
Genre: Human activity recognition
ISBN:

Human action or gesture recognition has been extensively studied in the literature spanning a wide variety of human-computer interaction applications including gaming, surveillance, healthcare monitoring, and assistive living. Sensors used for action or gesture recognition are primarily either vision-based sensors or inertial sensors. Compared to the great majority of previous works where a single modality sensor is used for action or gesture recognition, the simultaneous utilization of a depth camera and a wearable inertial sensor is considered in this dissertation. Furthermore, compared to the great majority of previous works in which actions are assumed to be segmented actions, this dissertation addresses a more realistic and practical scenario in which actions of interest occur continuously and randomly amongst arbitrary actions of non-interest. In this dissertation, computationally efficient solutions are presented to recognize actions of interest from continuous data streams captured simultaneously by a depth camera and a wearable inertial sensor. These solutions comprise three main steps of segmentation, detection, and classification. In the segmentation step, all motion segments are extracted from continuous action streams. In the detection step, the segmented actions are separated into actions of interest and actions of non- interest. In the classification step, the detected actions of interest are classified. The features considered include skeleton joint positions, depth motion maps, and statistical attributes of acceleration and angular velocity inertial signals. The classifiers considered include maximum entropy Markov model, support vector data description, collaborative representation classifier, convolutional neural network, and long short-term memory network. These solutions are applied to the two applications of smart TV hand gestures and transition movements for home healthcare monitoring. The results obtained indicate the effectiveness of the developed solutions in detecting and recognizing actions of interest in continuous data streams. It is shown that higher recognition rates are achieved when fusing the decisions from the two sensing modalities as compared to when each sensing modality is used individually. The results also indicate that the deep learning-based solution provides the best outcome among the solutions developed.

Fusion of Depth and Inertial Sensing for Human Action Recognition

Fusion of Depth and Inertial Sensing for Human Action Recognition
Author: Chen Chen
Publisher:
Total Pages: 260
Release: 2016
Genre: Human activity recognition
ISBN:

Human action recognition is an active research area benefitting many applications. Example applications include human-computer interaction, assistive-living, rehabilitation, and gaming. Action recognition can be broadly categorized into vision-based and inertial sensor-based. Under realistic operating conditions, it is well known that there are recognition rate limitations when using a single modality sensor due to the fact that no single sensor modality can cope with various situations that occur in practice. The hypothesis addressed in this dissertation is that by using and fusing the information from two differing modality sensors that provide 3D data (a Microsoft Kinect depth camera and a wearable inertial sensor), a more robust human action recognition is achievable. More specifically, effective and computationally efficient features have been devised and extracted from depth images. Both feature-level fusion and decision-level fusion approaches have been investigated for a dual-modality sensing incorporating a depth camera and an inertial sensor. Experimental results obtained indicate that the developed fusion approaches generate higher recognition rates compared to the situations when an individual sensor is used. Moreover, an actual working action recognition system using depth and inertial sensing has been devised which runs in real-time on laptop platforms. In addition, the developed fusion framework has been applied to a medical application.

Human Action Recognition with Depth Cameras

Human Action Recognition with Depth Cameras
Author: Jiang Wang
Publisher: Springer Science & Business Media
Total Pages: 65
Release: 2014-01-25
Genre: Computers
ISBN: 331904561X

Action recognition technology has many real-world applications in human-computer interaction, surveillance, video retrieval, retirement home monitoring, and robotics. The commoditization of depth sensors has also opened up further applications that were not feasible before. This text focuses on feature representation and machine learning algorithms for action recognition from depth sensors. After presenting a comprehensive overview of the state of the art, the authors then provide in-depth descriptions of their recently developed feature representations and machine learning techniques, including lower-level depth and skeleton features, higher-level representations to model the temporal structure and human-object interactions, and feature selection techniques for occlusion handling. This work enables the reader to quickly familiarize themselves with the latest research, and to gain a deeper understanding of recently developed techniques. It will be of great use for both researchers and practitioners.

Intelligent Data Engineering and Analytics

Intelligent Data Engineering and Analytics
Author: Vikrant Bhateja
Publisher: Springer Nature
Total Pages: 633
Release: 2023-11-25
Genre: Technology & Engineering
ISBN: 9819967066

The book presents the proceedings of the 11th International Conference on Frontiers of Intelligent Computing: Theory and Applications (FICTA 2023), held at Cardiff School of Technologies, Cardiff Metropolitan University, Cardiff, Wales, UK, during April 11–12, 2023. Researchers, scientists, engineers, and practitioners exchange new ideas and experiences in the domain of intelligent computing theories with prospective applications in various engineering disciplines in the book. This book is divided into two volumes. It covers broad areas of information and decision sciences, with papers exploring both the theoretical and practical aspects of data-intensive computing, data mining, evolutionary computation, knowledge management and networks, sensor networks, signal processing, wireless networks, protocols, and architectures. This book is a valuable resource for postgraduate students in various engineering disciplines.

Body Sensor Networks

Body Sensor Networks
Author: Guang-Zhong Yang
Publisher: Springer
Total Pages: 572
Release: 2014-04-16
Genre: Computers
ISBN: 1447163745

The last decade has witnessed a rapid surge of interest in new sensing and monitoring devices for wellbeing and healthcare. One key development in this area is wireless, wearable and implantable in vivo monitoring and intervention. A myriad of platforms are now available from both academic institutions and commercial organisations. They permit the management of patients with both acute and chronic symptoms, including diabetes, cardiovascular diseases, treatment of epilepsy and other debilitating neurological disorders. Despite extensive developments in sensing technologies, there are significant research issues related to system integration, sensor miniaturisation, low-power sensor interface, wireless telemetry and signal processing. In the 2nd edition of this popular and authoritative reference on Body Sensor Networks (BSN), major topics related to the latest technological developments and potential clinical applications are discussed, with contents covering. Biosensor Design, Interfacing and Nanotechnology Wireless Communication and Network Topologies Communication Protocols and Standards Energy Harvesting and Power Delivery Ultra-low Power Bio-inspired Processing Multi-sensor Fusion and Context Aware Sensing Autonomic Sensing Wearable, Ingestible Sensor Integration and Exemplar Applications System Integration and Wireless Sensor Microsystems The book also provides a comprehensive review of the current wireless sensor development platforms and a step-by-step guide to developing your own BSN applications through the use of the BSN development kit.

Description Logics in Multimedia Reasoning

Description Logics in Multimedia Reasoning
Author: Leslie F. Sikos
Publisher: Springer
Total Pages: 215
Release: 2017-06-28
Genre: Computers
ISBN: 3319540661

This book illustrates how to use description logic-based formalisms to their full potential in the creation, indexing, and reuse of multimedia semantics. To do so, it introduces researchers to multimedia semantics by providing an in-depth review of state-of-the-art standards, technologies, ontologies, and software tools. It draws attention to the importance of formal grounding in the knowledge representation of multimedia objects, the potential of multimedia reasoning in intelligent multimedia applications, and presents both theoretical discussions and best practices in multimedia ontology engineering. Readers already familiar with mathematical logic, Internet, and multimedia fundamentals will learn to develop formally grounded multimedia ontologies, and map concept definitions to high-level descriptors. The core reasoning tasks, reasoning algorithms, and industry-leading reasoners are presented, while scene interpretation via reasoning is also demonstrated. Overall, this book offers readers an essential introduction to the formal grounding of web ontologies, as well as a comprehensive collection and review of description logics (DLs) from the perspectives of expressivity and reasoning complexity. It covers best practices for developing multimedia ontologies with formal grounding to guarantee decidability and obtain the desired level of expressivity while maximizing the reasoning potential. The capabilities of such multimedia ontologies are demonstrated by DL implementations with an emphasis on multimedia reasoning applications.

Adaptive Activity Recognition Techniques with Evolving Data Streams

Adaptive Activity Recognition Techniques with Evolving Data Streams
Author: Zahraa Said Emam Ammar Abdallah
Publisher:
Total Pages: 480
Release: 2015
Genre:
ISBN:

Activity recognition aims to provide accurate and opportune information on people's activities by leveraging sensory data available in today's sensory rich environments. Activity recognition has become an emerging field in the areas of pervasive and ubiquitous computing. The process of recognising activities flows through three key steps: sensing, modelling, and recognition. A typical activity recognition technique processes data streams that evolve from sensing platforms such as mobile sensors, on body sensors, and/or ambient sensors. Learning models in activity recognition are built from historical data and rely strongly on prior knowledge of activities. The learning model in this scenario is static and thus unable to cope with the evolving nature of activities in data streams.The evolving nature of activities arises for many reasons. Intuitively, people perform activities in different ways. "Walking" for one person could be "jogging" for another. Therefore, there is no model that fits all in activity recognition. To attain an accurate recognition, a learning model has to be tuned to suit a user's personalised way of performing activities. Moreover, it is unrealistic to assume that the number of activities is static along the stream. While the learning model is built from historical data, novel activities may emerge and abandoned ones may disappear over time.This thesis develops adaptive techniques for activity recognition that dynamically change the learning model while activities evolve. These techniques apply an incremental and continuous learning approach for both personalisation and adaptation of the learning model. As a strategy to harness the potential of activity for pervasive environments, our techniques are capable of recognising activities that evolve from data streams. The first contribution of this thesis is to build a flexible, efficient, robust, and accurate learning model that enables personalisation and adaptation with evolving data streams. This learning model is the core for all our techniques developed in this thesis.Based on the developed learning model, we propose a technique for recognising activities efficiently. The recognition technique is an ensemble classifier that integrates with the learning model to recognise activities based on a hybrid similarity measure approach. The merit of this approach is to bring different perspectives together for more accurate recognition, especially across users. The ensemble classifier is evaluated on benchmarked datasets for activity recognition. The evaluation demonstrated the robustness, efficiency, and accurate recognition of activities. Our technique shows its best performance when applied across users and with noisy data. The accuracy is improved by more than 10% in these cases compared to other state-of-the-art techniques in activity recognition using benchmarked multidimensional datasets.The above activity recognition technique is extended to include incremental learning for personalisation with evolving data streams. This technique leverages the flexibility of the learning model for personalisation in real time to achieve an accurate recognition with the evolving activities. Furthermore, we deploy our technique on a mobile device to demonstrate its efficiency. Although the streaming environment imposes more constraints on the recognition process, the proposed recognition technique outperforms other benchmarked incremental techniques in activity recognition. Our technique shows its best performance when applied to data that contains noise with accuracy enhancement of about 15%.The last contribution is a technique that enables continuous learning to adapt the learning model. To fulfil this goal, our technique detects the arrival of new activities in data streams and/or the disappearance of abandoned ones. Moreover, it dynamically adapts the learning model with the detected changes for a future recognition. The developed technique is evaluated on benchmarked datasets to demonstrate its efficiency in recognising changes in activities and adaptation of the learning model accordingly. The recognition of novel activities varies depending on the characteristics of the datasets and the nature of the detected activity. This technique, as well as all techniques in this thesis, incorporates active learning to address the scarcity of labelled data especially in streaming environment by annotating only small amounts of the most informative data. Thus, this thesis takes a step forward in activity recognition dynamics in pervasive and ubiquitous computing by building efficient and adaptive techniques for recognising evolving activities.

International Conference on Innovative Computing and Communications

International Conference on Innovative Computing and Communications
Author: Deepak Gupta
Publisher: Springer Nature
Total Pages: 772
Release: 2022-11-07
Genre: Technology & Engineering
ISBN: 981193679X

This book includes high-quality research papers presented at the Fifth International Conference on Innovative Computing and Communication (ICICC 2022), which is held at the Shaheed Sukhdev College of Business Studies, University of Delhi, Delhi, India, on February 19–20, 2022. Introducing the innovative works of scientists, professors, research scholars, students and industrial experts in the field of computing and communication, the book promotes the transformation of fundamental research into institutional and industrialized research and the conversion of applied exploration into real-time applications.

A Case Study on Robustness of Dynamic Time Warping for Activity Recognition Using Wearable Computers

A Case Study on Robustness of Dynamic Time Warping for Activity Recognition Using Wearable Computers
Author: Nimish Rajiv Kale
Publisher:
Total Pages: 172
Release: 2012
Genre: Dynamic programming
ISBN:

We describe a body sensor system that detects human activities in real-time. The system consists of wearable computers known as sensor nodes (motes) that can sense information, process them and transmit the results to a Personal Device like Smart phone, PDA or Personal Computer. The motes are attached to different parts of the human body, namely waist and right thigh. Daily living activity monitoring is important in improving quality of life especially in elderly. A wireless wearable network of inertial sensor nodes can be used to observe daily motions. Continuous stream of data generated by these sensor networks can be used to recognize the movements of interest. Dynamic Time Warping (DTW) is a widely used signal processing for time-series pattern matching because of its robustness to variations in time domain and speed as opposed to other template matching methods such as Euclidean Distance. Despite of this flexibility, for the application of activity recognition, DTW can only find the similarity between template of a movement and the incoming samples, when the location and orientation of sensor remains unchanged. Due to this restriction, small sensor misplacements can lead to false classifications. In this work, we adopt DTW distance as a feature for real-time detection of human daily activities like sit to stand. To measure this performance of DTW, we need infinite closely spaced sensors which are impractical. To deal with this problem, we use the marker based optical motion capture system and generate inertial sensor data for different location and orientation on the body. We study the performance of the DTW under these conditions and determine the worst-case sensor location variations, the algorithm can accommodate.