Studies in Computational Intelligence

(1)

Studies in Computational Intelligence

Volume 481

Series Editor

J. Kacprzyk, Warsaw, Poland

For further volumes:

(2)

Aleksander Nawrat

· Zygmunt Ku´s

Editors

Vision Based Systems

for UAV Applications

(3)

Editors

Prof. Aleksander Nawrat Silesian University of Technology Gliwice

Poland

Dr. Zygmunt Ku´s

Silesian University of Technology Gliwice

Poland

ISSN 1860-949X ISSN 1860-9503 (electronic)

ISBN 978-3-319-00368-9 ISBN 978-3-319-00369-6 (eBook) DOI 10.1007/978-3-319-00369-6

Springer Cham Heidelberg New York Dordrecht London Library of Congress Control Number: 2013934960

c

Springer International Publishing Switzerland 2013

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broad-casting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its cur-rent version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein.

Printed on acid-free paper

(4)

“In God We Trust, All others we observe and keep very close. . . ”

(5)

Preface

The book was motivated by a significant number of vision based algorithms for Un-manned Aerial Vehicles (UAV) that are developed during research and development projects. Nowadays, it can be observed a significant increase in a number of UAV applications. One of the main factors in the development of UAV is rapid progress of development of advanced sensory and considerable increase of computing power. Aerial observation is currently used for both military and civilian purposes. The va-riety of drones and their parameters gives almost unlimited possibilities in areas like vision surveillance, cartography, geodesy, cinematography or meteorology. Most if not all of these applications utilize some kind of vision based sensor like video camera, thermal imaging camera or laser range finders. Therefore it is essential to improve the existing general video acquisition devices and design and construct innovative dedicated for special purposes cameras. The amount of high and low fre-quencies disruptions is significantly greater than for traditional handheld cameras. It is a matter of high importance to design devices to reduce the impact in order to deliver proper video stream stabilization. Additionally bearing capacity of quick response class UAV is highly limited. Therefore video acquisition devices are re-quired to be as light as possible but at the same time rugged and reliable. However, there is also a considerable group of vision based control algorithms that are used e.g. for target detection or collision-free navigation. It is worth mentioning about the task of controlling a group of mobile robots within line of sight. There is also an active field of classification of actions and patterns based on information often fused from multiple video stream sources. Finally the video stream has to be com-pressed and transmitted wirelessly from UAV to ground station over heterogeneous networks with different dynamically changing topologies.

Main idea of visual surveillance is to detect any specified targets or abnormal sit-uations in the line of sight. Targets can be detected due to their characteristic features e.g. face, character of movement, size, color or shape. Those features are detected, tracked and recognized by a dedicated set of algorithms processing images acquired from live video stream. Detection and recognition are usually a time consuming thus there is object tracking phase in the middle. The main challenge of such algorithms is to reconcile the tracking quality with simplicity required for real time process-ing. Regardless the application object recognition is an essential task for complex systems where a certain amount of system autonomy is required. The book contains

(6)

VIII Preface valuable information in the field of detection, tracking and recognition of objects. Presented challenges and solutions are the results of research conducted by the team. Nowadays, image acquisition devices are mainly used by human operators or large tactical or strategic class UAVs. Therefore there is a need for innovative video sources that are characterized by small weight, low energy consumption and at the same time high resistance to vibrations, shock and temperature. Traditionally used visible light cameras cover only a small part of the magnetic spectrum and are easily covered by dust, smoke, fog, etc. In order to continue aerial observation thermal imaging and omnidirectional cameras have to be used.

It is commonly accepted that over 80% of all perceptual information being re-ceived by human brain is perre-ceived through eyes. It is natural for control algorithms developers to try to utilize the information as it is done by both human and ani-mal brains. Vision information can be utilized not only for control a single device but also for controlling a group of e.g. mobile robots. An alternative approach is to use aerial vision information for controlling detected and recognized objects on the ground.

Main aim of design and construction of UAVs is their practical application. How-ever, their design is a lengthy process and requires intensive evaluation using both simulation and experimental data. Even then, it is often necessary to apply amend-ments based on the actual real life experience of the application. Misrecognition can sometimes lead to irreversible consequences therefore in order to improve the quality of recognition frequently fusion of data from multiple sources of informa-tion is used. However it is worth meninforma-tioning that before the video stream from a camera mounted on an UAV it is necessary to wirelessly transmit to human operator using ground base station. In addition, video data transmission requires a significant network bandwidth. Communication over long distances is charged a considerable limitation in this regard. Therefore it is necessary to use a dedicated image and sound compression algorithms.

The topics covered in this book also includes practical applications of classifi-cation algorithms used in video streams for UAV. This book is not only intended for researchers involved in image processing field. Aim of the book was to create a valuable source of information for researchers and constructors of solutions utiliz-ing vision from UAV. Scientists and researchers involved in computer vision, image processing, data fusion, control algorithms, mechanics, data mining, navigation and IC can find many valuable, useful and practical suggestions and solutions. The latest challenges for vision based systems are also presented. Furthermore the information included in the book can benefit greatly students of higher years of study in the fields of computer science, automation, robotics, mechanical engineering or data mining. The author strongly encourage all readers to explore the following chapters of the book and invite to open and honest discussion of any questions or concerns encoun-tered during the lecture.

Gliwice Aleksander Nawrat

(7)

Acknowledgments

This work has been supported by Applied Research Programme of the National Centre for Research and Development as a project ID 178438 path A - Costume for

acquisition of human movement based on IMU sensors with collection, visualization and data analysis software.

(8)

Part I: Design of Object Detection, Recognition and

Tracking Algorithms

Machine Vision in Autonomous Systems of Detection and Location of

Objects in Digital Images. . . . 3

Artur Babiarz, Robert Bieda, Karol J˛edrasiak, Aleksander Nawrat

Recognition and Location of Objects in the Visual Field of a UAV

Vision System. . . . 27

Robert Bieda, Krzysztof Jaskot, Karol J˛edrasiak, Aleksander Nawrat

Automatic Targeting Sentry Turret for Distributed Systems . . . . 47

Piotr Demski, Tomasz Grzejszczak, Karol J˛edrasiak, Michał Mikulski

Object Tracking for Rapid Camera Movements in 3D Space . . . . 57

Zygmunt Ku´s, Aleksander Nawrat

Object Tracking in a Picture during Rapid Camera Movements . . . . 77

Zygmunt Ku´s, Aleksander Nawrat

Part II: Construction of Image Acquisition Devices Used

in UAV Applications

Thermal Camera for Autonomous Mobile Platforms. . . . 95

Grzegorz Bieszczad, Michał Krupi´nski, Henryk Madura, Tomasz Sosnowski

Gyro-Stabilized Platform for Multispectral Image Acquisition. . . . 115

Damian Bereska, Karol J˛edrasiak, Krzysztof Daniec, Aleksander Nawrat

Omnidirectional Video Acquisition Device (OVAD) . . . . 123

Sławomir Fra´s, Karol J˛edrasiak, Jan Kwiatkowski, Aleksander Nawrat, Dawid Sobel

(9)

XII Contents

Part III: Design of Vision Based Control Algorithms

Vision System for Group of Mobile Robots . . . . 139

Artur Babiarz, Robert Bieda, Krzysztof Jaskot

A Distributed Control Group of Mobile Robots in a Limited Area

with a Vision System. . . . 157

Artur Babiarz, Robert Bieda, Krzysztof Jaskot

System for Multi-axial Mechanical Stabilization of Digital Camera. . . . . 177

Damian Bereska, Krzysztof Daniec, Sławomir Fra´s, Karol J˛edrasiak, Mateusz Malinowski, Aleksander Nawrat

Probabilistic Approach to Planning Collision Free Path of UAV. . . . 191

Dawid Cedrych, Adam Gałuszka, Marcin Pacholczyk, Krzysztof Skrzypczyk, Aleksander Nawrat

Part IV: Practical Applications of Classification

Algorithms Used in Video Streams for UAV

Information Fusion in Multi-agent System Based on Reliability

Criterion. . . . 207

Martin Mellado, Krzysztof Skrzypczyk

Prototyping the Autonomous Flight Algorithms Using the Prepar3DR

Simulator. . . . 219

Krzysztof Daniec, Paweł Iwaneczko, Karol J˛edrasiak, Aleksander Nawrat

Feature Extraction and HMM-Based Classification of Gait Video

Sequences for the Purpose of Human Identification. . . . 233

Henryk Josi´nski, Daniel Kostrzewa, Agnieszka Michalczuk, Adam ´Swito´nski, Konrad Wojciechowski

The Dedicated Wireless Communication Device for Group of

Unmanned Vehicles. . . . 247

Krzysztof Daniec, Karol J˛edrasiak, Roman Koteras, Aleksander Nawrat, Tadeusz Topór-Kami´nski

Selection of Individual Gait Features Extracted by MPCA Applied to

Video Recordings Data. . . . 257

Henryk Josiński, Agnieszka Michalczuk, Andrzej Polański, Adam ´Switoński, Konrad Wojciechowski

Comparative Analysis of Power Loses in Selected Measurement

Devices Used in SMART GRID Systems . . . . 273

Aleksander Nawrat, Anna Piaskowy, Artur Skórkowski, Tadeusz Topór-Kami´nski

(10)

Contents XIII

Technology Development of Military Applications of Unmanned

Ground Vehicles . . . . 293

Tomasz Czapla, Józef Wrona

Estimation of Solution of Discrete Linear Time-Varying System . . . . 311

Adam Czornik, Aleksander Nawrat, Michał Niezabitowski

Virtual Reality Technology for Military and Industry Training

Programs . . . . 327

Marek Ko´zlak, Antoni Kurzeja, Aleksander Nawrat

Verifying Unmanned Land Vehicle Vision System Configuration

Using Serious Gaming Solution. . . . 335

Marek Ko´zlak, Antoni Kurzeja, Aleksander Nawrat

Conclusions . . . 343 Author Index . . . 345

(11)

List of Contributors

Artur Babiarz

Silesian University of Technology, Institute of Automatic Control, Akademicka 16, 44-101 Gliwice, Poland

e-mail:[email protected]

Damian Bereska

Robert Bieda

Grzegorz Bieszczad

Military University of Technology, Institute of Optoelectronics, Warsaw, Poland e-mail:[email protected]

Dawid Cedrych

Tomasz Czapla

Silesian University of Technology, Faculty of Mechanical Engineering, Division of Theoretical and Applied Mechanics, ul. Konarskiego 18A, 44-101 Gliwice, Poland e-mail:[email protected]

(12)

XVI List of Contributors Adam Czornik

Krzysztof Daniec

Piotr Demski

Sławomir Fra´s

Adam Gałuszka

Tomasz Grzejszczak

Paweł Iwaneczko

Krzysztof Jaskot

Karol J˛edrasiak

(13)

List of Contributors XVII Henryk Josi´nski

Silesian University of Technology, Institute of Computer Science, Akademicka 16, 44-101 Gliwice, Poland

Daniel Kostrzewa

Roman Koteras

Marek Ko´zlak

O´srodek Badawczo-Rozwojowy Urz ˛adze´n Mechanicznych “OBRUM” sp. z o.o. ul. Toszecka 102, 44-117 Gliwice, Poland

email:[email protected]

Silesian University of Technology, Technopark, ul. Konarskiego 18C, 44-100 Gli-wice, Poland

Michał Krupi´nski

Antoni Kurzeja

Zygmunt Ku´s

Jan Kwiatkowski

(14)

XVIII List of Contributors Henryk Madura

Mateusz Malinowski

Martin Mellado

Instituto de Automatica e Informatica Industrial, Universidad Plitecnica de Valencia e-mail:[email protected]

Agnieszka Michalczuk

Michał Mikulski

Aleksander Nawrat

Michał Niezabitowski

Marcin Pacholczyk

(15)

List of Contributors XIX Anna Piaskowy

Silesian University of Technology, 44-101 Gliwice, Poland

Andrzej Pola´nski

Artur Skórkowski

Silesian University of Technology, 44-101 Gliwice, Poland

Krzysztof Skrzypczyk

Dawid Sobel

Tomasz Sosnowski

Adam ´Swito´nski

Tadeusz Topór-Kami´nski

Konrad Wojciechowski

(16)

XX List of Contributors Józef Wrona

Military University of Technology, ul. gen. Sylwestra Kaliskiego 2, Warsaw, Poland, Industrial Research Institute of Automation and Measurements, Aleje Jerozolimskie 2022, 02-486 Warsaw, Poland