Prosody Based Audio-visual Co-analysis for Coverbal Gesture Recognition

Prosody Based Audio-visual Co-analysis for Coverbal Gesture Recognition
Author: Sanshzar Kettebekov
Publisher:
Total Pages: 30
Release: 2002
Genre: Computer vision
ISBN:

Abstract: "Although recognition of natural speech and gestures have been studied extensively, previous attempts of combining them for multimodal Human-Computer Interaction (HCI) were mostly semantically motivated, e.g., keyword-gesture co-analysis. Such top-down co-analysis for improving gesture recognition is associated with the inherent complexity of natural language processing and is not always suitable for real-time HCI. This paper explores prosodic phenomena of spontaneous gesture and speech production and presents a computational framework for improving the recognition of continuous gestures. Prosody based co-analysis of audio and visual signal is investigated at two different levels, namely, physiological and articulation. Physiological constraints are defined in a feature-based integration framework using Hidden Markov Models (HMMs). Co-articulation is analyzed using a Bayesian belief network of naïve classifiers to explore alignment of intonationally [sic] prominent speech segments and hand velocity. A weighted fusion scheme is applied for combining the decisions of the two co-analysis models. It was found that both levels of co-analyses uniquely contribute in detection and disambiguation of kinematically defined gesture primitives, which subsequently improves the performance of continuous gesture recognition. The efficacy of the proposed approach was demonstrated on a large database collected from the Weather Channel broadcast. This formulation opens new avenues for bottom-up frameworks for inclusion of natural gesticulation for HCI."

Data-Driven 3D Facial Animation

Data-Driven 3D Facial Animation
Author: Zhigang Deng
Publisher: Springer Science & Business Media
Total Pages: 303
Release: 2008
Genre: Computers
ISBN: 1846289068

Data-Driven 3D Facial Animation systematically describes the important techniques developed over the last ten years or so. Comprehensive in scope, the book provides an up-to-date reference source for those working in the facial animation field.

Body - Language - Communication. Volume 2

Body - Language - Communication. Volume 2
Author: Cornelia Müller
Publisher: Walter de Gruyter GmbH & Co KG
Total Pages: 1084
Release: 2014-10-29
Genre: Language Arts & Disciplines
ISBN: 3110302020

Volume II of the handbook offers a unique collection of exemplary case studies. In five chapters and 99 articles it presents the state of the art on how body movements are used for communication around the world. Topics include the functions of body movements, their contexts of occurrence, their forms and meanings, their integration with speech, and how bodily motion can function as language. By including an interdisciplinary chapter on ‘embodiment’, volume II explores the body and its role in the grounding of language and communication from one of the most widely discussed current theoretical perspectives. Volume II of the handbook thus entails the following chapters: VI. Gestures across cultures, VII. Body movements: functions, contexts and interactions, VIII. Gesture and language, IX. Embodiment: the body and its role for cognition, emotion, and communication, X. Sign Language: Visible body movements as language. Authors include: Mats Andrèn, Richard Asheley, Benjamin Bergen, Ulrike Bohle, Dominique Boutet, Heather Brookes, Penelope Brown, Kensy Cooperrider, Onno Crasborn, Seana Coulson, James Essegby, Maria Graziano, Marianne Gullberg, Simon Harrison, Hermann Kappelhoff, Mardi Kidwell, Irene Kimbara, Stefan Kopp, Grigoriy Kreidlin, Dan Loehr, Irene Mittelberg, Aliyah Morgenstern, Rafael Nuñez, Isabella Poggi, David Quinto-Pozos, Monica Rector, Pio Enrico Ricci-Bitti, Göran Sonesson, Timo Sowa, Gale Stam, Eve Sweetser, Mark Tutton, Ipke Wachsmuth, Linda Waugh, Sherman Wilcox.

Intelligent Virtual Agents

Intelligent Virtual Agents
Author: Jonathan Gratch
Publisher: Springer Science & Business Media
Total Pages: 485
Release: 2006-08-10
Genre: Computers
ISBN: 3540375937

This book constitutes the refereed proceedings of the 6th International Workshop on Intelligent Virtual Agents, IVA 2006. The book presents 24 revised full papers and 11 revised short papers together with 3 invited talks and the abstracts of 19 poster papers. The papers are organized in topical sections on social impact of IVAs, IVAs recognizing human behavior, human interpretation of IVA behavior, embodied conversational agents, characteristics of nonverbal behavior and more.

Dual Bayesian and Morphology-based Approach for Markerless Human Motion Capture in Natural Interaction Environments

Dual Bayesian and Morphology-based Approach for Markerless Human Motion Capture in Natural Interaction Environments
Author: Pedro Correa Hernandez
Publisher: Presses univ. de Louvain
Total Pages: 152
Release: 2006
Genre: Science
ISBN: 9782874630293

The goal of this work has been to tackle the problem of gestural human-computer interfaces in its most natural form, i.e. without markers or invasive devices. In that sense a complete system is proposed in order to classify and track in real-time a sufficient number of human features that allow novel forms of gestural man-machine interaction. The algorithm is basically composed of an intra-image phase and an inter-image phase. The first one takes advantage of several mathematical morphology tools in order to analyze the user silhouette and robustly extract head, hands and feet. The second phase works in an inter-image Bayesian framework in order to achieve the classification and tracking of the previously extracted features. Due to its low computational complexity, the system can run at real-time paces on standard Personal Computers, with an average error rate range between 2% and 7% in realistic situations, depending on the context and segmentation quality.

Modeling Communication with Robots and Virtual Humans

Modeling Communication with Robots and Virtual Humans
Author: Ipke Wachsmuth
Publisher: Springer Science & Business Media
Total Pages: 344
Release: 2008-04-03
Genre: Computers
ISBN: 3540790365

Embodied agents play an increasingly important role in cognitive interaction technology. The two main types of embodied agents are virtual humans inhabiting simulated environments and humanoid robots inhabiting the real world. So far research on embodied communicative agents has mainly explored their potential for practical applications. However, the design of communicative artificial agents can also be of great heuristic value for the scientific study of communication. It allows researchers to isolate, implement, and test essential properties of inter-agent communications in operational models. Modeling communication with robots and virtual humans thus involves the vision of using communicative machines as research tools. Artificial systems that reproduce certain aspects of natural, multimodal communication help to elucidate the internal mechanisms that give rise to different aspects of communication. In short, constructing embodied agents who are able to communicate may help us to understand the principles of human communication. As a comprehensive theme, “Embodied Communication in Humans and Machines” was taken up by an international research group hosted by Bielefeld University’s Center for Interdisciplinary Research (ZiF – Zentrum für interdisziplinäre Forschung) from October 2005 through September 2006. The overarching goal of this research year was to develop an integrated perspective of embodiment in communication, establishing bridges between lower-level, sensorimotor functions and a range of higher-level, communicative functions involving language and bodily action. The present volume grew out of a workshop that took place during April 5–8, 2006 at the ZiF as a part of the research year on embodied communication.

PARTICIPANT LIST ENTERFACE'05

PARTICIPANT LIST ENTERFACE'05
Author: Thierry Dutoit
Publisher: Presses univ. de Louvain
Total Pages: 98
Release: 2005-12
Genre: Computers
ISBN: 9782874630033

What are eNTERFACE workshops?The eNTERFACE summer workshops ( www.enterface.net ), organized by the SIMILAR European Network of Excellence, are a new type of European workshops. They aim at establishing a tradition of collaborative, localized research...

Prosody and Kinesics Based Co-analysis Towards Continuous Gesture Recognition

Prosody and Kinesics Based Co-analysis Towards Continuous Gesture Recognition
Author: Pratiksha Subedi
Publisher:
Total Pages:
Release: 2012
Genre:
ISBN:

The aim of this study is to develop a multimodal co-analysis framework for continuous gesture recognition by exploiting prosodic and kinesics manifestation of natural communication. Using this framework, a co-analysis pattern between correlating components is obtained. The co-analysis pattern is clustered using K-means clustering to determine how well the pattern distinguishes the gestures. Features of the proposed approach that differentiate it from the other models are its less susceptibility to idiosyncrasies, its scalability, and simplicity. The experiment was performed on Multimodal Annotated Gesture Corpus (MAGEC) that we created for research on understanding non-verbal communication community, particularly the gestures.

Creativity and Robotics

Creativity and Robotics
Author: Patricia Alves-Oliveira
Publisher: Frontiers Media SA
Total Pages: 232
Release: 2022-11-03
Genre: Technology & Engineering
ISBN: 2832503845

Symbols and Embodiment

Symbols and Embodiment
Author: Manuel de Vega
Publisher:
Total Pages: 472
Release: 2008
Genre: Computers
ISBN:

Cognitive scientists have a variety of approaches to studying cognition: experimental psychology, computer science, robotics, neuroscience, educational psychology, philosophy of mind, and psycholinguistics, to name but a few. In addition, they also differ in their approaches to cognition - some of them consider that the mind works basically like a computer, involving programs composed of abstract, amodal, and arbitrary symbols. Others claim that cognition is embodied - that is, symbols must be grounded on perceptual, motoric, and emotional experience. The existence of such different approaches has consequences when dealing with practical issues such as understanding brain disorders, designing artificial intelligence programs and robots, improving psychotherapy, or designing instructional programs. The symbolist and embodiment camps seldom engage in any kind of debate to clarify their differences. This book is the first attempt to do so. It brings together a team of outstanding scientists, adopting symbolist and embodied viewpoints, in an attempt to understand how the mind works and the nature of linguistic meaning. As well as being interdisciplinary, all authors have made an attempt to find solutions to substantial issues beyond specific vocabularies and techniques.