Motion Estimation from Image and Inertial Measurements

Motion Estimation from Image and Inertial Measurements
Author: Dennis W. Strelow
Publisher:
Total Pages: 154
Release: 2004
Genre: Computer vision
ISBN:

Abstract: "Robust motion estimation from image measurements would be an enabling technology for Mars rover, micro air vehicle, and search and rescue robot navigation; modeling complex environments from video; and other applications. While algorithms exist for estimating six degree of freedom motion from image measurements, motion from image measurements suffers from inherent problems. These include sensitivity to incorrect or insufficient image feature tracking; sensitivity to camera modeling and calibration errors; and long-term drift in scenarios with missing observations, i.e., where image features enter and leave the field of view. The integration of image and inertial measurements is an attractive solution to some of these problems. Among other advantages, adding inertial measurements to image-based motion estimation can reduce the sensitivity to incorrect image feature tracking and camera modeling errors. On the other hand, image measurements can be exploited to reduce the drift that results from integrating noisy inertial measurements, and allows the additional unknowns needed to interpret inertial measurements, such as the gravity direction and magnitude, to be estimated. This work has developed both batch and recursive algorithms for estimating camera motion, sparse scene structure, and other unknowns from image, gyro, and accelerometer measurements. A large suite of experiments uses these algorithms to investigate the accuracy, convergence, and sensitivity of motion from image and inertial measurements. Among other results, these experiments show that the correct sensor motion can be recovered even in some cases where estimates from image or inertial estimates alone are grossly wrong, and explore the relative advantages of image and inertial measurements and of omnidirectional images for motion estimation. To eliminate gross errors and reduce drift in motion estimates from real image sequences, this work has also developed a new robust image feature tracker that exploits the rigid scene assumption and eliminates the heuristics required by previous trackers for handling large motions, detecting mistracking, and extracting features. A proof of concept system is also presented that exploits this tracker to estimate six degrees of freedom motion from long image sequences, and limits drift in the estimates by recognizing previously visited locations."

Continuous Models for Cameras and Inertial Sensors

Continuous Models for Cameras and Inertial Sensors
Author: Hannes Ovrén
Publisher: Linköping University Electronic Press
Total Pages: 67
Release: 2018-07-25
Genre:
ISBN: 917685244X

Using images to reconstruct the world in three dimensions is a classical computer vision task. Some examples of applications where this is useful are autonomous mapping and navigation, urban planning, and special effects in movies. One common approach to 3D reconstruction is ”structure from motion” where a scene is imaged multiple times from different positions, e.g. by moving the camera. However, in a twist of irony, many structure from motion methods work best when the camera is stationary while the image is captured. This is because the motion of the camera can cause distortions in the image that lead to worse image measurements, and thus a worse reconstruction. One such distortion common to all cameras is motion blur, while another is connected to the use of an electronic rolling shutter. Instead of capturing all pixels of the image at once, a camera with a rolling shutter captures the image row by row. If the camera is moving while the image is captured the rolling shutter causes non-rigid distortions in the image that, unless handled, can severely impact the reconstruction quality. This thesis studies methods to robustly perform 3D reconstruction in the case of a moving camera. To do so, the proposed methods make use of an inertial measurement unit (IMU). The IMU measures the angular velocities and linear accelerations of the camera, and these can be used to estimate the trajectory of the camera over time. Knowledge of the camera motion can then be used to correct for the distortions caused by the rolling shutter. Another benefit of an IMU is that it can provide measurements also in situations when a camera can not, e.g. because of excessive motion blur, or absence of scene structure. To use a camera together with an IMU, the camera-IMU system must be jointly calibrated. The relationship between their respective coordinate frames need to be established, and their timings need to be synchronized. This thesis shows how to automatically perform this calibration and synchronization, without requiring e.g. calibration objects or special motion patterns. In standard structure from motion, the camera trajectory is modeled as discrete poses, with one pose per image. Switching instead to a formulation with a continuous-time camera trajectory provides a natural way to handle rolling shutter distortions, and also to incorporate inertial measurements. To model the continuous-time trajectory, many authors have used splines. The ability for a spline-based trajectory to model the real motion depends on the density of its spline knots. Choosing a too smooth spline results in approximation errors. This thesis proposes a method to estimate the spline approximation error, and use it to better balance camera and IMU measurements, when used in a sensor fusion framework. Also proposed is a way to automatically decide how dense the spline needs to be to achieve a good reconstruction. Another approach to reconstruct a 3D scene is to use a camera that directly measures depth. Some depth cameras, like the well-known Microsoft Kinect, are susceptible to the same rolling shutter effects as normal cameras. This thesis quantifies the effect of the rolling shutter distortion on 3D reconstruction, depending on the amount of motion. It is also shown that a better 3D model is obtained if the depth images are corrected using inertial measurements. Att använda bilder för att återskapa världen omkring oss i tre dimensioner är ett klassiskt problem inom datorseende. Några exempel på användningsområden är inom navigering och kartering för autonoma system, stadsplanering och specialeffekter för film och spel. En vanlig metod för 3D-rekonstruktion är det som kallas ”struktur från rörelse”. Namnet kommer sig av att man avbildar (fotograferar) en miljö från flera olika platser, till exempel genom att flytta kameran. Det är därför något ironiskt att många struktur-från-rörelse-algoritmer får problem om kameran inte är stilla när bilderna tas, exempelvis genom att använda sig av ett stativ. Anledningen är att en kamera i rörelse ger upphov till störningar i bilden vilket ger sämre bildmätningar, och därmed en sämre 3D-rekonstruktion. Ett välkänt exempel är rörelseoskärpa, medan ett annat är kopplat till användandet av en elektronisk rullande slutare. I en kamera med rullande slutare avbildas inte alla pixlar i bilden samtidigt, utan istället rad för rad. Om kameran rör på sig medan bilden tas uppstår därför störningar i bilden som måste tas om hand om för att få en bra rekonstruktion. Den här avhandlingen berör robusta metoder för 3D-rekonstruktion med rörliga kameror. En röd tråd inom arbetet är användandet av en tröghetssensor (IMU). En IMU mäter vinkelhastigheter och accelerationer, och dessa mätningar kan användas för att bestämma hur kameran har rört sig över tid. Kunskap om kamerans rörelse ger möjlighet att korrigera för störningar på grund av den rullande slutaren. Ytterligare en fördel med en IMU är att den ger mätningar även i de fall då en kamera inte kan göra det. Exempel på sådana fall är vid extrem rörelseoskärpa, starkt motljus, eller om det saknas struktur i bilden. Om man vill använda en kamera tillsammans med en IMU så måste dessa kalibreras och synkroniseras: relationen mellan deras respektive koordinatsystem måste bestämmas, och de måste vara överens om vad klockan är. I den här avhandlingen presenteras en metod för att automatiskt kalibrera och synkronisera ett kamera-IMU-system utan krav på exempelvis kalibreringsobjekt eller speciella rörelsemönster. I klassisk struktur från rörelse representeras kamerans rörelse av att varje bild beskrivs med en kamera-pose. Om man istället representerar kamerarörelsen som en tidskontinuerlig trajektoria kan man på ett naturligt sätt hantera problematiken kring rullande slutare. Det gör det också enkelt att införa tröghetsmätningar från en IMU. En tidskontinuerlig kameratrajektoria kan skapas på flera sätt, men en vanlig metod är att använda sig av så kallade splines. Förmågan hos en spline att representera den faktiska kamerarörelsen beror på hur tätt dess knutar placeras. Den här avhandlingen presenterar en metod för att uppskatta det approximationsfel som uppkommer vid valet av en för gles spline. Det uppskattade approximationsfelet kan sedan användas för att balansera mätningar från kameran och IMU:n när dessa används för sensorfusion. Avhandlingen innehåller också en metod för att bestämma hur tät en spline behöver vara för att ge ett gott resultat. En annan metod för 3D-rekonstruktion är att använda en kamera som också mäter djup, eller avstånd. Vissa djupkameror, till exempel Microsoft Kinect, har samma problematik med rullande slutare som vanliga kameror. I den här avhandlingen visas hur den rullande slutaren i kombination med olika typer och storlekar av rörelser påverkar den återskapade 3D-modellen. Genom att använda tröghetsmätningar från en IMU kan djupbilderna korrigeras, vilket visar sig ge en bättre 3D-modell.

Multisensor Attitude Estimation

Multisensor Attitude Estimation
Author: Hassen Fourati
Publisher: CRC Press
Total Pages: 607
Release: 2016-11-03
Genre: Technology & Engineering
ISBN: 1498745806

There has been an increasing interest in multi-disciplinary research on multisensor attitude estimation technology driven by its versatility and diverse areas of application, such as sensor networks, robotics, navigation, video, biomedicine, etc. Attitude estimation consists of the determination of rigid bodies’ orientation in 3D space. This research area is a multilevel, multifaceted process handling the automatic association, correlation, estimation, and combination of data and information from several sources. Data fusion for attitude estimation is motivated by several issues and problems, such as data imperfection, data multi-modality, data dimensionality, processing framework, etc. While many of these problems have been identified and heavily investigated, no single data fusion algorithm is capable of addressing all the aforementioned challenges. The variety of methods in the literature focus on a subset of these issues to solve, which would be determined based on the application in hand. Historically, the problem of attitude estimation has been introduced by Grace Wahba in 1965 within the estimate of satellite attitude and aerospace applications. This book intends to provide the reader with both a generic and comprehensive view of contemporary data fusion methodologies for attitude estimation, as well as the most recent researches and novel advances on multisensor attitude estimation task. It explores the design of algorithms and architectures, benefits, and challenging aspects, as well as a broad array of disciplines, including: navigation, robotics, biomedicine, motion analysis, etc. A number of issues that make data fusion for attitude estimation a challenging task, and which will be discussed through the different chapters of the book, are related to: 1) The nature of sensors and information sources (accelerometer, gyroscope, magnetometer, GPS, inclinometer, etc.); 2) The computational ability at the sensors; 3) The theoretical developments and convergence proofs; 4) The system architecture, computational resources, fusion level.

Camera Motion Estimation Using Monocular Image Sequences and Inertial Data

Camera Motion Estimation Using Monocular Image Sequences and Inertial Data
Author:
Publisher:
Total Pages: 28
Release: 1999
Genre: Image processing
ISBN:

This paper presents a robust model-based algorithm for camera motion estimation using a monocular image sequence and inertial data. Conventional algorithms using only video information encounter difficulties in robustly estimating irregular camera motion. In our approach, we use both video and inertial information to estimate the camera motion. The inertial data are acquired by a set of on-board Micro-Electro-Mechanical-System (MEMS) based sensors. The key features of our algorithm are (1) utilization of inertial data and (2) full exploitation of Directions of Location (DOLs) of feature points. By using inertial data and DOLs of feature points, we track and recover more complex platform motion than conventional algorithms. An Iterated Extended Kalman Filter (IEKF) is used to estimate the motion parameters. This algorithm has been tested on synthetic and real image sequences and the results in both cases show the efficacy of our approach.

Using Inertial Sensors for Position and Orientation Estimation

Using Inertial Sensors for Position and Orientation Estimation
Author: Manon Kok
Publisher:
Total Pages: 174
Release: 2018-01-31
Genre: Technology & Engineering
ISBN: 9781680833560

Microelectromechanical system (MEMS) inertial sensors have become ubiquitous in modern society. Built into mobile telephones, gaming consoles, virtual reality headsets, we use such sensors on a daily basis. They also have applications in medical therapy devices, motion-capture filming, traffic monitoring systems, and drones. While providing accurate measurements over short time scales, this diminishes over longer periods. To date, this problem has been resolved by combining them with additional sensors and models. This adds both expense and size to the devices. This tutorial focuses on the signal processing aspects of position and orientation estimation using inertial sensors. It discusses different modelling choices and a selected number of important algorithms that engineers can use to select the best options for their designs. The algorithms include optimization-based smoothing and filtering as well as computationally cheaper extended Kalman filter and complementary filter implementations. Engineers, researchers, and students deploying MEMS inertial sensors will find that this tutorial is an essential monograph on how to optimize their designs.

Robust and Large-scale Human Motion Estimation with Low-cost Sensors

Robust and Large-scale Human Motion Estimation with Low-cost Sensors
Author: Hua-I Chang
Publisher:
Total Pages: 117
Release: 2016
Genre:
ISBN:

Enabling large-scale monitoring and classification of a range of motion activities is of primary importance due to the need by healthcare and fitness professionals to monitor exercises for quality and compliance. Video based motion capturing systems (e.g., VICON cameras) provide a partial solution. However, these expensive and fixed systems are not suitable for patients' at-home daily motion monitoring. Wireless motion sensors, including accelerometers and gyroscopes, can provide a low-cost, small-size, and highly-mobile option. However, acquiring robust inference of human motion trajectory via low-cost inertial sensors remains challenging. Sensor noise and drift, sensor placement errors and variation of activity over the population all lead to the necessity of a large amount of data collection. Unfortunately, such a large amount of data collection is prohibitively costly. In observance of these issues, a series of solutions for robust human motion monitoring and activity classification will be presented. The implementation of a real-time context-guided activity classification system will be discussed. To facilitate ground truth data acquisition, we proposed a virtual inertial measurements platform to convert the currently available MoCap database into a noiseless and error-free inertial measurements database. An opportunistic calibration system which deals with sensor placement errors will be discussed. In addition, a sensor fusion approach for robust upper limb motion tracking will also be presented.

The Essential Guide to Video Processing

The Essential Guide to Video Processing
Author: Alan C. Bovik
Publisher: Academic Press
Total Pages: 777
Release: 2009-07-07
Genre: Technology & Engineering
ISBN: 0080922503

This comprehensive and state-of-the art approach to video processing gives engineers and students a comprehensive introduction and includes full coverage of key applications: wireless video, video networks, video indexing and retrieval and use of video in speech processing. Containing all the essential methods in video processing alongside the latest standards, it is a complete resource for the professional engineer, researcher and graduate student. Numerous conceptual and numerical examples All the latest standards are thoroughly covered: MPEG-1, MPEG-2, MPEG-4, H.264 and AVC Coverage of the latest techniques in video security "Like its sister volume "The Essential Guide to Image Processing," Professor Bovik’s Essential Guide to Video Processing provides a timely and comprehensive survey, with contributions from leading researchers in the area. Highly recommended for everyone with an interest in this fascinating and fast-moving field." —Prof. Bernd Girod, Stanford University, USA Edited by a leading person in the field who created the IEEE International Conference on Image Processing, with contributions from experts in their fields Numerous conceptual and numerical examples All the latest standards are thoroughly covered: MPEG-1, MPEG-2, MPEG-4, H.264 and AVC Coverage of the latest techniques in video security

Robotics Research

Robotics Research
Author: Antonio Bicchi
Publisher: Springer
Total Pages: 525
Release: 2017-07-25
Genre: Technology & Engineering
ISBN: 3319515322

ISRR, the "International Symposium on Robotics Research", is one of robotics pioneering Symposia, which has established over the past two decades some of the field's most fundamental and lasting contributions. This book presents the results of the seventeenth edition of "Robotics Research" ISRR15, offering a collection of a broad range of topics in robotics. The content of the contributions provides a wide coverage of the current state of robotics research.: the advances and challenges in its theoretical foundation and technology basis, and the developments in its traditional and new emerging areas of applications. The diversity, novelty, and span of the work unfolding in these areas reveal the field's increased maturity and expanded scope and define the state of the art of robotics and its future direction.

Experimental Robotics VIII

Experimental Robotics VIII
Author: Bruno Siciliano
Publisher: Springer Science & Business Media
Total Pages: 671
Release: 2003-01-21
Genre: Technology & Engineering
ISBN: 3540003053

This book is a collection of papers on the state of the art in experimental robotics. Experimental Robotics is at the core of validating robotics research for both its systems science and theoretical foundations. Because robotics experiments are carried out on physical, complex machines, of which its controllers are subject to uncertainty, devising meaningful experiments and collecting statistically significant results, pose important and unique challenges in robotics. Robotics experiments serve as a unifying theme for robotics system science and algorithmic foundations. These observations have led to the creation of the International Symposia on Experimental Robotics. The papers in this book were presented at the 2002 International Symposium on Experimental Robotics.

Image and Graphics

Image and Graphics
Author: Yao Zhao
Publisher: Springer
Total Pages: 733
Release: 2017-12-29
Genre: Computers
ISBN: 3319716077

This three-volume set LNCS 10666, 10667, and 10668 constitutes the refereed conference proceedings of the 9th International Conference on Image and Graphics, ICIG 2017, held in Shanghai, China, in September 2017. The 172 full papers were selected from 370 submissions and focus on advances of theory, techniques and algorithms as well as innovative technologies of image, video and graphics processing and fostering innovation, entrepreneurship, and networking.