A Framework for Interactive Three-dimensional Sound and Spatial Audio Processing in a Virtual Environment

A Framework for Interactive Three-dimensional Sound and Spatial Audio Processing in a Virtual Environment
Author: Michael Wozniewski
Publisher:
Total Pages: 106
Release: 2007
Genre: Shared virtual environments
ISBN:

"Immersive virtual environments offer the possibility of natural interaction within a virtual world that is familiar to users because it is based on everyday activity. The use of such environments for the representation and control of 3-D sound remains largely unexplored. We propose a novel paradigm for interacting with sound and using virtual space as the medium for spatial audio processing. A supporting software framework has been developed that provides functionality not available in other 3-D audio systems, including powerful control over the directivity of sound and the ability to bend the rules of physics for musical purposes. These features provide the necessary tools to create virtual scenes for audio engineering, musical creation, listening, and performance. Tracking technology allows the use of gesture-based interaction techniques to control the environment, resulting in many possibilities for novel applications." --

Spatial Audio in Graphical Applications

Spatial Audio in Graphical Applications
Author: Jurriaan Derk Mulder
Publisher:
Total Pages: 17
Release: 1994
Genre: User interfaces (Computer systems)
ISBN:

Abstract: "The use of audio in human-computer interfaces is gaining much attention nowadays. A neglected issue in this regard is the fact that human beings perceive sounds in a spatial context. Simulated partial sound presented over headphones has many potential uses in a number of fields, such as virtual reality and visualization. In this paper a simple experimental system for spatial sound simulation is presented, along with the results of several experiments that were conducted with this system. The system called SAGA (Spatial Audio in Graphical Applications), was built to investigate the usability of spatial sound in three- dimensional graphical applications such as visualization. Two primary uses of spatial sound in such applications are envisioned: (1) the active localization of virtual sound sources in a three-dimensional virtual environment in order to be able to move the point of view or a three- dimensional cursor towards this sound source, and (2) as an aid in the position determination of a cursor in a three-dimensional virtual environment. The results of the experiments conducted with the SAGA system show that the first use is very feasible, but the second will be more difficult to accomplish."

Ambisonics

Ambisonics
Author: Franz Zotter
Publisher: Springer
Total Pages: 210
Release: 2019-04-30
Genre: Technology & Engineering
ISBN: 3030172074

This open access book provides a concise explanation of the fundamentals and background of the surround sound recording and playback technology Ambisonics. It equips readers with the psychoacoustical, signal processing, acoustical, and mathematical knowledge needed to understand the inner workings of modern processing utilities, special equipment for recording, manipulation, and reproduction in the higher-order Ambisonic format. The book comes with various practical examples based on free software tools and open scientific data for reproducible research. The book’s introductory section offers a perspective on Ambisonics spanning from the origins of coincident recordings in the 1930s to the Ambisonic concepts of the 1970s, as well as classical ways of applying Ambisonics in first-order coincident sound scene recording and reproduction that have been practiced since the 1980s. As, from time to time, the underlying mathematics become quite involved, but should be comprehensive without sacrificing readability, the book includes an extensive mathematical appendix. The book offers readers a deeper understanding of Ambisonic technologies, and will especially benefit scientists, audio-system and audio-recording engineers. In the advanced sections of the book, fundamentals and modern techniques as higher-order Ambisonic decoding, 3D audio effects, and higher-order recording are explained. Those techniques are shown to be suitable to supply audience areas ranging from studio-sized to hundreds of listeners, or headphone-based playback, regardless whether it is live, interactive, or studio-produced 3D audio material.

3D Audio

3D Audio
Author: Justin Paterson
Publisher: Routledge
Total Pages: 321
Release: 2021-07-25
Genre: Technology & Engineering
ISBN: 0429957238

3D Audio offers a detailed perspective of this rapidly developing arena. Written by many of the world’s leading researchers and practitioners, it draws from science, technologies, and creative practice to provide insight into cutting-edge research in 3D audio. Through exploring the intersection of these fields, the reader will gain insight into a number of research areas and professional practice in 3D sonic space. As such, the book acts both as a primer that enables readers to gain an understanding of various aspects of 3D audio, and can inform students and audio enthusiasts, but its deep treatment of a diverse range of topics will also inform professional practitioners and academics beyond their core specialisms. The chapters cover areas such as an Ambisonics, binaural technologies and approaches, psychoacoustics, 3D audio recording, composition for 3D space, 3D audio in live sound, broadcast, and movies – and more. Overall, this book offers a definitive insight into an emerging sound world that is increasingly becoming part of our everyday lives.

Scalable Immersive Audio for Virtual Environments

Scalable Immersive Audio for Virtual Environments
Author: Mirza Beig
Publisher:
Total Pages: 0
Release: 2018
Genre:
ISBN:

This thesis discusses the history of game sound and how it developed to become increasingly more immersive and realistic by accounting for the spatial aspects of sound. It then compares several 3D sound technologies before continuing on to present a custom spatial sound system that uses the highly parallel nature of modern graphics processing units (GPUs) to process large amounts of data for simulated sound propagation. In a real-time virtual reality game environment, the system is able to handle dynamic geometry and movable sound sources in full 3D at interactive rates. To test the implementation, I developed a VR-compatible video game for desktop and mobile platforms with an emphasis on play-by-sound gameplay mechanics. Results demonstrated low overhead on all devices tested when used with the appropriate pathfinding backend (CPU/GPU). Additional testing yielded that the GPU part of the system is capable of processing several thousand 3D paths at interactive rates.

Binaural Technology for Virtual Reality

Binaural Technology for Virtual Reality
Author: Tobias Lentz
Publisher: Logos Verlag Berlin GmbH
Total Pages: 166
Release: 2008
Genre: Computers
ISBN: 3832519351

The use of non-intrusive virtual environments is gaining more and more importance but was focused mainly on addressing the visual sense. However, the human perception consists not only of visual input and thus it would be worthwhile to create multi-modal and interactive virtual environments. This thesis describes the techniques required to include the acoustic component into a virtual environment and furthermore the implementation of a software system, which takes advantage of these techniques to create complex acoustical scenes in real time. The system is based on the binaural technology. It features spatially distributed sound sources which are utilized to create an environment that is as authentic as possible. This comprises a description of the source, including its relevant angle-, distance- and time- dependent radiation, the sound distribution in the virtual scene (room acoustics), the perception-related consideration of all sound field components, as well as the exact reproduction of the artificial sound at the ears of the user. The focus of this thesis is put on the reproduction technology. In this context, an approach for dynamic crosstalk cancellation is presented, which enables a loudspeaker-based reproduction. The required filters are processed in real time on the basis of the position data and measured transfer functions of the outer ear. Furthermore the integration of this spatial audio system into a five-sided Virtual Reality display system is described and evaluated.

Sound - Perception - Performance

Sound - Perception - Performance
Author: Rolf Bader
Publisher: Springer Science & Business Media
Total Pages: 393
Release: 2013-05-23
Genre: Technology & Engineering
ISBN: 3319001078

Musical Performance covers many aspects like Musical Acoustics, Music Psychology, or motor and prosodic actions. It deals with basic concepts of the origin or music and its evolution, ranges over neurocognitive foundations, and covers computational, technological, or simulation solutions. This volume gives an overview about current research in the foundation of musical performance studies on all these levels. Recent concepts of synchronized systems, evolutionary concepts, basic understanding of performance as Gestalt patterns, theories of chill as performance goals or historical aspects are covered. The neurocognitive basis of motor action in terms of music, musical syntax, as well as therapeutic aspects are discussed. State-of-the-art applications in performance realizations, like virtual room acoustics, virtual musicians, new concepts of real-time physical modeling using complex performance data as input or sensor and gesture studies with soft- and hardware solutions are presented. So although the field is still much larger, this volume presents current trends in terms of understanding, implementing, and perceiving performance.

Immersive Sound

Immersive Sound
Author: Agnieszka Roginska
Publisher: Taylor & Francis
Total Pages: 676
Release: 2017-10-17
Genre: Technology & Engineering
ISBN: 1317480104

Immersive Sound: The Art and Science of Binaural and Multi-Channel Audio provides a comprehensive guide to multi-channel sound. With contributions from leading recording engineers, researchers, and industry experts, Immersive Sound includes an in-depth description of the physics and psychoacoustics of spatial audio as well as practical applications. Chapters include the history of 3D sound, binaural reproduction over headphones and loudspeakers, stereo, surround sound, height channels, object-based audio, soundfield (ambisonics), wavefield synthesis, and multi-channel mixing techniques. Knowledge of the development, theory, and practice of spatial and multi-channel sound is essential to those advancing the research and applications in the rapidly evolving fields of 3D sound recording, augmented and virtual reality, gaming, film sound, music production, and post-production.

Computational Framework for the Analysis of Spatial Audio

Computational Framework for the Analysis of Spatial Audio
Author: Steven E. Crawford
Publisher:
Total Pages: 144
Release: 2020
Genre:
ISBN:

"In this thesis, the extent to which the coherence properties of binaural signals can be used in the computation of cues that aid in the prediction of perceived location and apparent auditory source width of sound events is explored. To this end, a computational, modular framework for spatial audio analysis is developed and applied as a prediction tool on synthetically rendered spatial sound fields. The primary focus is to develop a binaural fusion model, and more generally, a model of spatial hearing with immediate practical application to objective, spatial sound-localization predictions for arbitrary multi-channel and/or headphone-based spatial audio synthesis schemes. The binaural model and overall framework may serve as useful tools for the analysis and design of spatial audio experiences for virtual and augmented reality systems. The framework also may be employed as a convenient alternative to the direct use of human participants in listening experiments, and as such may serve as a tool in the development of spatial audio rendering systems. Utilizing a computational binaural auditory model as its front-end, the framework is composed of a series of modular signal processing blocks designed to simulate the peripheral and central stages of the human auditory system. The present implementation of the binaural fusion model builds upon the Meddis hair cell model for the peripheral stages and coincidence detection of delay-line activity patterns (the Jeffress model) for the central processing stage. However, the overall framework is intended to be modular and not model specific. Additionally, signals are processed using a gammatone filter bank to produce an interaural coherence function; a short-time windowed cross-correlation of the binaural signals in each gammatone filter band. These interaural coherence functions collectively produce a binaural activity representation, called a 'correlogram', from which the perceived auditory image location and its spatial extent may be inferred. A dictionary of basis correlograms corresponding to measured sound source locations in three-dimensional space is generated and then used to compare the rendering precision and accuracy of virtual auditory images produced using first order and higher order Ambisonics (FOA and HOA). A form of regularized regression, called an elastic net, is used to infer the spatial and psychoacoustic properties of the virtual acoustic source. The final output of the framework is a predictive metric representing the perceived location and ?width' of an acoustic source in 3D space"--Pages viii-ix.