Motion Tracking System Using Computer Vision With OpenCV Source Software

UNIVERSITI TEKNIKAL MALAYSIA MELAKA

MOTION TRACKING SYSTEM USING

COMPUTER VISION WITH OPENCV SOURCE SOFTWARE

This report submitted in accordance with requirement of the Universiti Teknikal Malaysia Melaka (UTeM) for the Bachelor Degree of Engineering Technology

Telecommunication (Hons.)

by

MUHAMMAD DANIAL BIN ZAINAL ABIDEEN

B071310936

920420075343

UNIVERSITI TEKNIKAL MALAYSIA MELAKA

BORANG PENGESAHAN STATUS LAPORAN PROJEK SARJANA MUDA

TAJUK: MOTION TRACKING SYSTEM USING COMPUTER VISION WITH OPENCV SOURCE SOFTWARE

SESI PENGAJIAN: 2016/17 Semester 2

Saya MUHAMMAD DANIAL BIN ZAINAL ABIDEEN

mengaku membenarkan Laporan PSM ini disimpan di Perpustakaan Universiti Teknikal Malaysia Melaka (UTeM) dengan syarat-syarat kegunaan seperti berikut:

1. Laporan PSM adalah hak milik Universiti Teknikal Malaysia Melaka dan penulis. 2. Perpustakaan Universiti Teknikal Malaysia Melaka dibenarkan membuat salinan untuk

tujuan pengajian sahaja dengan izin penulis.

3. Perpustakaan dibenarkan membuat salinan laporan PSM ini sebagai bahan pertukaran antara institusi pengajian tinggi.

4. **Sila tandakan ( )

SULIT

TERHAD

TIDAK TERHAD

(Mengandungi maklumat yang berdarjah keselamatan atau kepentingan Malaysia sebagaimana yang termaktub dalam AKTA RAHSIA RASMI 1972)

(Mengandungi maklumat TERHAD yang telah ditentukan oleh organisasi/badan di mana penyelidikan dijalankan)

(TANDATANGAN PENULIS)

Alamat Tetap:

6-13-8 Villa Sri Kenanga

11700 Batu Uban

Pulau Pinang

Tarikh: ________________________

Disahkan oleh:

(TANDATANGAN PENYELIA)

Cop Rasmi:

iv

DECLARATION

I hereby, declared this report entitled “Motion Tracking System Using Computer Vision With Opencv Source Software” is the results of my own research

except as cited in references

Signature : ……….

Author’s Name : ………

v

APPROVAL

This report is submitted to the Faculty of Engineering Technology of UTeM as a partial fulfillment of the requirements for the degree of Bachelor of Electronic Engineering Technology (Telecommunications) with Honours. The member of the supervisory is as

follow:

………

vi

ABSTRAK

vii

ABSTRACT

viii

DEDICATION

ix

ACKNOWLEDGEMENT

x TABLE OF CONTENT

Declaration iv

Approval v

Abstrak vi

Abstract vii

Dedication vii

Acknowledgement ix

Table of Content x

List of Figures xiii

List of Table xv

1 INTRODUCTION 1 1.1 Introduction 1

1.2 Project Briefing 1

1.3 Motion Tracking 2

1.4 Open Computer Vision 2

1.5 Problem Statement 3

1.6 Objective 4

1.7 Scopes 4

2 LITERATURE REVIEW 5 2.1 Introduction 5

2.2 Motion Tracking 5

2.2.1 Definition to Motion Tracking 6

2.3 Motion Tracking Techniques 9

2.3.1 Optical Flow Based 9

xi 2.3.2 Color Based Motion Tracking Technique 13

2.3.2.1 Color Based Motion Tracking Technique (The RGB Space)

14

2.3.2.2 Color Based Motion Tracking Technique (The Greyscale Space)

14

2.3.2.3 Color Based Motion Tracking Technique (The Hue, Saturation and Lightness Space)

15

2.4 Feature Extraction and Image Processing 17

2.4.1 Image Filtering 17

2.5 Previous Related Work 18

2.5.1 Tracking a Tennis Ball Using Image Processing Techniques 18 2.5.2 Detection and Tracking of Human Subjects 23

2.6 Open Computer Vision 26

2.6.1 Open Computer Vision Implementation 26

3 METHODOLOGY 28

3.0 Introduction 28

3.1 Motion Tracking Methodology 29

3.2 Optical Flow 31

3.2.1 Background Subtraction 32

3.2.1.1. Edge Detection 34

3.3 Color Based 35

3.3.1 Red, Green, Blue (RGB) 35

3.3.1.1 Segmentation 36

3.4 Mean Shift (camshift) 38

xii 4 4.0 4.1 4.2 4.3 4.4 4.5 4.6 4.7 5.0 5.1 5.2

RESULT AND DISCUSSION Background

Camshift Technique

Movement based on each object

4.2.1 Moving Slowly Toward and Away From the Camera 4.2.2 Swaying Side to Side

4.2.3 Rotational

Object with yellow color (Tennis ball) Object with grey color (powerbank)

Object with green and red color (oil bottle) Hue,Saturation,Value (HSV) of each object Tracking object in different brightness condition

CONCLUSION AND RECOMMENDATION Background Conclusion Recommendation 40 40 41 42 43 44 45 46 48 51 53 55 55 56

REFERENCES 58

xiii LIST OF FIGURES

2.1 The Class of the Motion Tracking 7

2.2 The Motion Tracking of Face Detection 8

2.3 The Object (face) Has Been Identified and Recognized 8

2.4 Motion form recognition 10

2.5 The RGB Cube 14

2.6 The Grayscale Space 14

2.7 The HSV Cone 16

2.8 The HSL Cone Inside the RGB Cube 16

2.9 Background Image Done the Edges 19

2.10 The Sequence of the Object Detection 20

2.11 The Sequence Image Tennis Ball Detection using the Colour Segmentation Approach

21

2.12 The Image of Tennis Ball with Shape Recognition 22

2.13 The Result Image of Tennis Ball detection using the Classifier Creation

23

2.14 The Image of Walking Woman change to Gray Scale 24

2.15 Binary Image Generated with a Threshold of 8%. 25

2.16 Example of the Threshold Processing 26

3.1 General Methodology Motion Tracking Flow Chart 29

xiv 3.3 Moving Object Detection using Optical Flow 32

3.4 Example Background Subtraction 33

3.5 3.6 3.7 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 4.10 4.11 4.12 4.13 4.14 4.25

Result of Edge Detection

The RGB technique

Image use the Segmentation Technique

Object Moving Away from Camera

Object Moving Toward to Camera

Object Swaying Side to Side

Object Rotational

Threshold Value of Tennis Ball

Tennis ball tracking

Threshold Value of Powerbank

Powerbank tracking

Threshold of the Bottle

Threshold of Bottle Cap

Oil Bottle Tracking with Two Different Color

RGB to HSV color table

HSV Cylinder

Tracking in The Bright Room and Value Of HSV

Tracking in the dark room and value of HS

xv 4.1

4.2

4.3

4.4

4.5

LIST OF TABLE

Hue,saturation,value result of tennis ball

Hue,saturation,value result of powerbank

Hue,saturation,value result of oil bottle

Hue,saturation,value result in bright room

Hue,saturation,value result in dark room 51

51

51

53

1 CHAPTER 1

INTRODUCTION

1.1 Introduction

In this chapter will describes and simple briefly about the motion tracking and OpenCV definition. This chapter discusses about the problem statement of the motion tracking, objective and scope of the project

1.2 Project Briefing

2 an adaptive region-sizing step. The camshift is a simple step function applied to a color probability map. This main project will include the research on the motion tracking method to be done by detection by a camera. The movement on the camera vision range will be tracking on.

1.3 Motion tracking

Motion tracking is track the movement of an object and then apply the tracking data for that movement to another object such as another layer or an effect control point to create compositions in which images and effects follow the motion. Besides, motion tracking is the process where the vision of image or videos detected. The movement on the camera vision range will be tracking. The information of the tracking can be as simple as the position of the body in space or as complex as the deformations of the face and muscle masses. Motion tracking for computer character animation involves the mapping of human motion onto the motion of a computer character. The mapping can be direct, such as human arm motion controlling a character's arm motion, or indirect, such as human hand and finger patterns controlling a character's skin color or emotional state.

1.4 Open Computer Vision

3 program, and to run with a device to process an image or videos. OpenCV can be viewed as signal processing applied to higher dimensions of image and videos. OpenCV library usually used the C and C++ to programming the database. The OpenCV is optimized and intend for real time applications.

1.5 Problem Statement

4 1.6 Objective

Main objective of this project is to:

1) To study technique on how motion tracking worked

2) Develop a vision based system for detect motion or movement of the object 3) To created and implement a programming using opencv source software for

motion tracking

1.7 Scopes

5 CHAPTER 2

LITERATURE REVIEW

2.1 Introduction

In this chapter will review and discuss available literature on Motion Tracking using OpenCV software. This section will be including the motion tracking technique and related previous projects. This reviews also explain about motion tracking. Then, the OpenCV will be discussed in this chapter.

2.2 Motion Tracking

Motion tracking definition will be discuss on this title. The motion tracking technique also will be describe in this topic. The technique will be divide into a few sub topic depend on the scope of the motion tracking research and many more will be discuss in this topic.

6 Zhang and Payandeh 2002) or to select an optimal grasp for picking-up object (Han and Kuc 1998).Over the years, a vast number of algorithms have been proposed for object tracking, and there are large numbers of applications that require such algorithms to track different target in different conditions (Maurin et al. 2005). For example, to guide an autonomous vehicle in a simple or complex environments (Kia and Arshad 2005, and Asif et al. 2005) or it may be used to track vehicle for collecting the traffic data from highway scenes (Kastrinaki et al. 2003) or even to detect human in a surveillance system (Collins et al. 2000a).

2.2.1 Definition to Motion Tracking

7

Figure 2.1: The class of the motion tracking

8

Figure 2.2: Show that the motion tracking of face detection. The red box is the box

of reorganization progress where the face have been detect and not identified yet. The green box shows that the object has been identified and recognized

Figure 2.3: Show the object (face) has been identified and recognized. From red the

boxes change to green. The box will follow and track the face everywhere as long as in range

9 unusual object movements or coordinated activities. With motion tracking, you can track the movement of an object and then apply the tracking data for that movement to another object such as another layer or an effect control point to create compositions in which images and effects follow the motion

2.3 Motion Tracking Techniques

The science of motion tracking is fascinating because of its highly interdisciplinary nature and wide range of applications. The technique of motion tracking is the big impact of the way motion will be tracking and detecting. The motion tracking technique usually being used is color based, feature extracting, and optical flow based for the tracking method. Beside there have many more of motion tracking technique. However the technique named above is related with this project and most popular usage nowdays. The technique of the motion tracking choose also depend on the task of motion tracking. Every technique has their own algorithm and own characteristics. The algorithm of motion tracking will make the result of the motion tracking output implement.

2.3.1 Optical Flow Based