FEEESRC2014 Proceedings Rev2

(1)

Faculty of Electrical and Electronics Engineering

Ho Chi

Minh ci

ty Univer

sity

of

T

echnology

Student

Research

Conference

2014

PROCEEDINGS OF

FEEE

Ho Chi Minh city, January 18, 2014

(2)

(3)

Pr

Proce

ocee

edi

din

ngs

gs of

of

FEEE

FEEE STUDENT

STUDENT RESEARCH

RESEARCH

CONFERENCE

CONFERENCE 2014

2014

January 18, 2014, Ho Chi Minh City, Vietnam

Faculty of Electrical and Electronics Engineering

Ho Chi Minh City University of Technology

(4)

Message from the FEEE Dean

...

ii

3-D Positioning using Stereo Camera ...

22

Hai-Ho Dac, Quan

Hai-Ho Dac, Quang-Anh Le

g-Anh Le

Indoo

r

Mobi

le

Navi

gatio

n

using ROS

...

88

Thien-Minh

Thien-Minh Nguyen-P

Nguyen-Pham

ham

Controlling an Inverted Pendulum by a Regulator LQR with Feedback Information from Camera....

1616

Tan-Khoa Mai, Duy-Thanh Dang

Design

the O

ptimal Ro

bust PID

Controlle

r for

a Ball

and

Beam

System

...

....

2222

Nguyen Quang

Nguyen Quang Chanh, Do Cong P

Chanh, Do Cong Pham

ham

Autopilot Multicopter using Embedde

d Image Processing System: Design and Implem

entation

...

2828

Gia-Bao Nguyen Vu, Dang-Khoa Phan

Design Swing-Up and Balance Controllers for a Pendubot ...

3434

Van-Khoa Le

Design Video Door Bell Systems ...

4242

Van-Thang Vuong

Application

of W

ireless Se

nsor Ne

twork

And TC

P Soc

ket Se

rver in Sm

art Home

...

....

4848

Thanh-Tan Pham, Nhut-Huy O, Hoang-Phi Le-Nguyen

Display video on Led

Matrix RGB 64x128 using k

it De0-nano and BeagleBone

Black

...

5252

Thanh-Phong Do

I/O Minim

izing by

Multiplexing To

uch Fee

dback

on Capaci

tive Sen

sor

...

5858

Tuan-Vu Ho

Moving Object Detection in Traffic Scene ...

6262

Thanh-Hue Nguyen-Thi

Object Surface Reconstruction ...

68

Thanh-Hai Tran-Truong

Modelling and Designing PID Controller for BLDC Motor ...

74

Quang-Vu Nong, Anh-Quan Nguyen

Research on Application of Singer Wire Earth Return Distribution Systems in Vietnam ...

80

Duc-Toan Nguyen, Huu-Thanh Nguyen

Power Quality Analysis for Distribution Systems in Ho Chi Minh City ...

84

Minh-Khanh Lam, Dinh-Truc Pham, Huu-Phuc Nguyen

An Approach Designing SCADA Developer with Kernel Structure and XML Technology on iOS ...

90

Pham Hoang Hai Quan, Nguyen Van Phu, Le Hong Hai, Truong Dinh Chau

Design a Self-Tuning-Regulator for DC Motor's Velocity and Position Control ...

96

Tan-Khoa Nguyen

A Minutiae-Based Matching Algorithm in Fingerprint Recognition System ...

104

Hai Bui-Thanh, Hong-Nhat Thai-Xuan

High-speed Moving Object Tracking System for Inverted Pendulum ...

108

Hong-Hiep Nghiem

3-D Mouse using Inertial Measurement Unit Sensor ...

112

An Nguyen

Design and Implementation of Fuzzy-PID Controller for DC Motor Speed Control ...

114

Khanh-Cuong Mai-Manh, Thai-Cong Pham

WiFi Controlled Tracked-Car ...

118

Huynh Trung Bac, Tran Le Duc, Truong Nguyen Minh Trung

Design and Implementation of Music-Glove ...

122

Quoc-Duong Giang-Hoang

Neural-Network Control for a Mobile-robot ...

128

Thanh-Hoan Nguyen

A Three-phase Grid-connected Photovoltaic System with Power Factor Regulation ...

134

Tien-Manh Nguyen, Minh-Huy Nguyen, Minh-Phuong Le

ORAL SESSION C

(6)

A Modified Flood Fill Algorithm for Multi-destination Maze Solving Problem ...

140

Dinh-Huan Nguyen, Hong-Hiep Nghiem

SMS Registration using Digi Connect WAN Via TCP Socket ...

144

(7)

G

ENERAL

P

ROGRAM

D

ATE

: J

ANUARY

18

TH

, 2014

V

ENUE

: H

O

C

HI

M

INH

C

ITY

U

NIVERSITY OF

T

ECHNOLOGY

T

IME

A

RRANGEMENT

V

ENUE

12:00

_–

13:00

Registration

306B1

13:00

_–

13:10

Plenary session

306B1

13:10

_–

14:10

Keynote Speeches

14:10

_–

14:20

Coffee break

B1 Ground Floor

14:20

_–

15:00

Poster and Exhibition Sessions

B1 Ground Floor

15:00

_–

17:00

Oral Session A

308B1

Oral Session B

309B1

Oral Session C

104B1

17:00

_–

17:30

Closing ceremony & Awards

Announcement

104B1

18:00-21:00

Dinner

B1 Ground Floor

T

ECHNICAL

P

ROGRAM

K

EYNOTE

S

PEECHES

Time:

13:10

_–

14:10, January 18

th

, 2014

V

enue:

Room 306B1

T

IME

K

EYNOTE

S

PEAKER

13:10

_–

13:30

Keynote 1

Dr. Nguyen Quang Nam

13:30

_–

13:50

Keynote 2

Dr. Huynh Phu Minh Cuong

13:50

_–

14:10

Keynote 3

Dr. Nguyen Vinh Hao

P

OSTER

S

ESSION

Time:

14:20

_–

15:00, January 18

t

, 2014

Chair:

MEng. Ho Thanh Phuong

V

enue:

B1 Ground Floor

Co-chair:

BEng. Nguyen Tan Sy

I

NDEX

P

APER

T

ITLE

A

UTHOR

1 A Minutiae-Based Matching Algorithm in

Fingerprint Recognition System

(Paper ID: 24)

Hai Bui Thanh

Nhat Thai Xuan Hong

2 High-speed Moving Object Tracking System

for Inverted Pendulum (Paper ID: 20)

Hiep Nghiem

3 3-D Mouse using Inertial Measurement Unit

(8)

4 Design and Implementation of Fuzzy-PID

Controller for DC Motor Speed Control

(Paper ID: 09)

Mai Khanh Cuong

Cong Pham

5 WiFi Controlled Tracked-Car

(Paper ID: 12)

Duc Tran, Huynh Bac

6 Design and Implementation of Music-Glove

(Paper ID: 04)

Giang Hoang Quoc Duong

7 Neural-Network Control for a Mobile-robot

(Paper ID: 26)

Hoan Nguyen

8 A Three-Phase Grid-Connected Photovoltaic

System With Power Factor Regulation

(Paper ID: 25)

Manh Nguyen, Huy Nguyen,

Phuong Le

9 A Modified Flood Fill Algorithm for

Multi-destination Maze Solving Problem

(Paper ID: 28)

Huan Dinh Nguyen,

Hiep Nghiem

10 SMS Registration using Digi Connect WAN

via TCP Socket (Paper ID: 27)

Pham Ngoc Hoa,

Truong Thanh Hien

O

RAL

S

ESSION

A

Time:

15:00

_–

17:00, January 18

t

, 2014

Chair:

Dr. Nguyen Vinh Hao

V

enue:

Room 308B1

Co-chair:

Dr. Nguyen Trong Tai

T

IME

P

APER

T

ITLE

A

UTHOR

15:00

_–

15:20

3-D Positioning using Stereo Camera

(Paper ID: 21)

Hai Ho Dac

15:20

_–

15:40

Indoor Mobile Navigation using ROS

(Paper ID: 8)

Thien-Minh Nguyen-Pham

15:40

_–

16:00

Controlling an Inverted Pendulum by a

Regulator LQR with Feedback Information

from Camera (Paper ID: 23)

Khoa Mai, Thanh Dang

16:00

_–

16:20

Design the Optimal Robust PID Controller

for a Ball and Beam System (Paper ID: 16)

Do Cong Pham

Nguyen Quang Chanh

16:20

_–

16:40

Autopilot Multicopter using Embedded

_{Image Processing System: Design and}

Implementation (Paper ID: 24)

Khoa Phan, Gia-Bao Nguyen

16:40

_–

17:00

Design Swing-Up and Balance Controllers

for a Pendubot (Paper ID: 7)

Khoa Le

O

RAL

S

ESSION

B

Time:

15:00

_–

17:00, January 18

t

, 2014

Chair:

Dr. Vo Que Son

V

enue:

Room 309B1

Co-chair:

Dr. Che Viet Nhat Anh

T

IME

P

APER

T

ITLE

A

UTHOR

15:00

_–

15:20

Design Video Door Bell Systems

(Paper ID: 18)

Thang Vuong

15:20

_–

15:40

Application of Wireless Sensor Network

And TCP Socket Server in Smart Home

(Paper ID: 19)

Tan Pham, Phi Le Nguyen,

Huy O

(9)

using kit De0-nano and BeagleBone Black

(Paper ID: 14)

16:00

_–

16:20

I/O Minimizing by Multiplexing Touch

Feedback on Capacitive Sensor

(Paper ID: 17)

Ho Tuan Vu

16:20

_–

16:40

Moving Object Detection in Traffic Scene

(Paper ID: 10)

Hue Nguyen

16:40

_–

17:00

Object Surface Reconstruction

(Paper ID: 5)

Tran Hai

O

RAL

S

ESSION

C

Time:

15:00

_–

17:00, January 18

t

, 2014

Chair:

Dr. Ho Pham Huy Anh

V

enue:

Room 104B1

Co-chair:

Dr. Pham Dinh Truc

T

IME

P

APER

T

ITLE

A

UTHOR

15:00

_–

15:20

Modelling and Designing PID Controller for

BLDC Motor (Paper ID: 6)

Vu Nong Quang

Quan Nguyen Anh

15:20

_–

15:40

Research on Application of Singer Wire

Earth Return Distribution Systems in

Vietnam (Paper ID: 11)

Nguyen Huu Thanh

Nguyen Duc Toan

15:40

_–

16:00

Power Quality Analysis for Distribution

Systems in Ho Chi Minh City (Paper ID: 15)

Minh-Khanh Lam

Dinh-Truc Pham

Huu-Phuc Nguyen

16:00

_–

16:20

An Approach Designing SCADA Developer

with Kernel Structure and XML Technology

on iOS (Paper ID: 13)

Quan Pham

Phu Nguyen, Hai Le

16:20

_–

16:40

Design a Self-Tuning-Regulator for DC

Motor's Velocity and Position Control

(Paper ID: 3)

(10)

(11)

MESSAGE FROM THE FEEE DEAN

Dear all participants,

It is our great pleasure and honor to warmly welcome you to the 2014 FEEE

Student Research Conference (FEEE-SRC 2014) organized firstly by the

Pay-It-Forward club - a research club for student in the Faculty of Electrical and

Electronics Engineering, Ho Chi Minh City University of Technology, held on

January 18, 2014.

The 2014 FEEE Student Research Conference provides an excellent forum in

electrical and electronics engineering for sharing knowledge, encouraging

creativity and scientific research among students and creating an environment

to exchange, learn and share experiences and develop their talents, in three

tracks of Electronics and Telecommunications Engineering, Automation and

Control, and Electrical Engineering. The conference provides an opportunity

for students to improve communication skill and English skill, as well.

For the conference we have assembled 3 keynote speeches, high quality

technical sessions including 26 papers from junior and senior students, as

well as an exhibition of 20 excellent research projects.

With your active participation, we are confident that the 2014 FEEE Student

Research Conference will be successful as a major event for students in the

areas of electrical and electronics engineering. On behalf of the conference

organization committee, I would like to express our sincere thanks to the

faculty members for their valuable support to the conference, as well as

thanks to members of organization committee and members of the

Pay-It-Forward club for their very hard working to make the success of the

conference.

Dr.-Ing. Do Hong Tuan

Dean, Faculty of Electrical and Electronics Engineering

(12)

(13)

3-D Positioning using Stereo Camera

Dac-Hai Ho

Department of Automatic Control

Faculty of Electrical and Electronics Engineering Ho Chi Minh City University of Technology

[email protected]

Quang-Anh Le

Department of Automatic Control

Faculty of Electrical and Electronics Engineering Ho Chi Minh City University of Technology

[email protected]

Abstract _—This paper presents a standalone robot vision system, a stereo camera device, which is used for studying and applying localization and navigation technique in robotics and autonomous vehicles. By analyzing visual image captured from the stereo camera, motion estimation is computed to record and redraw its trajectory.

Keywords _—stereo image processing; robot vision system; motion estim ation; simul taneous localization and mapping

I. I NTRODUCTION

Trajectory estimation and object tracking nowadays has so much potential in reality implementations. Many works have been experimented to achieve accurate trajectory. In comparison to other measuring techniques such as using

encoder, GPS, laser scan…

though considered as a new approach, visual technology has demonstrated a dramatic development in recent years. Being aware of the importance of this application in artificial intelligence and high-tech products, we decided to carry out a research in computer vision.

In previous works, stereo camera is connected to a computer as laptop because it can provide sufficient computing power. Somehow, this setup makes the system looks big and overwhelming. However, the power of computer still proves irreplaceable for a fast speed system. Some practical experiments had been done to test the system in comparison to other measuring techniques. In Vietnam, there were just a few researches and applications related to stereo image processing as well as applying visual technology into trajectory tracking and navigation.

This paper is about a new approaching technique in visual tracking. In order to build a standalone system which is easy to implement and connect to other system, a FPGA core and an ARM core were implemented in this project. The FPGA core is used to capture stereo images and the ARM processing unit is used to handle all the algorithms and controls. FPGA advances in parallel processing which is suitable for handling and capturing stereo images at the same time but it has weakness in CPU frequency for high-speed processing. ARM advances in serial processing with high speed frequency processing but it causes a delay time in parallel processes. Therefore, to eliminate weaknesses of FPGA and ARM, we combined them as one system to take full advantages of each core. Furthermore, system can be further developed in the future as both the ARM board and FPGA board is so functional and powerful with many external IOs.

II. HARDWAREMODELING

A. Overview

For a practical experimental model, basically, there are two camera modules, a FPGA board, an ARM board and some external actuators and communication devices. Two cameras are connected to FPGA board. FPGA board can configure and interface with cameras and as well as ARM board. ARM board acts as a main processor of the whole system which can give commands to FPGA board, communicate with PC and process all the data. Besides, FPGA board and ARM board are functional to be programmed to communicate with external devices. For a standalone system, ARM board is programmed to interface with higher level of administrator like PC. PC can login into ARM board to get data, give command and control the whole stereo system. Power source is a matter to be concerned for a standalone system so a lifelong battery and a regulator power circuit are also necessary.

Fig. 1. Hardware modeling

For a simulation experiment model, there are only two webcams and a laptop in need. Algorithms can be programmed and developed directly on laptop which helps users more visually aware of how the system works and debugs.

B. FPGA Configurations

This project used two TRDB-D5M cameras comes along with the FPGA DE2-115 board. Camera parameters can easily be configured by I2C protocol. Bayer color filter was used to

capture RGB raw images then a small transformation was carried out to convert RGB images to gray scale images. Firstly, gray scale images will be stored in DDRAM then they can be stored in SDCard or transmitted directly to ARM board or to monitor through VGA output. This process needs a little help of a virtual NIOS core created within FPGA core. NIOS core will handle storing images progression on SDCard

(14)

and manage image address in DDRAM which can give ARM core an access into DDRAM on FPGA to get images. Secondly, FPGA board is configured a communication interface with ARM board. This procedure is quite complicated as it depends on the hardware itself. All the hardware configurations as well as applications on FPGA board were developed and programmed by Quartus II and Nios II software.

Fig. 2. FPGA configuration for capturing stereo images C. ARM Configurations

As a main processor of the system, ARM board is configured and programmed a bit more complicated rather than FPGA board. First of all, an embedded operating system is downloaded on ARM board. In this case, Linux is a free and open source with a various versions which is an ideal choice for the system. Moreover, Linux community is quite enormous so it is easy to get supports from people around the globe. Another thing needs to be taken into consideration is that the hardware configuration of ARM board usually goes with its appropriate version of Linux. It is because the producer had already declared hardware configurations (as known as drivers) on their Linux version. Moving to another point, the original Linux version needs to be updated and upgraded so it is important to get these things work before going to further steps. By the way, installing necessary packages is as important as getting the system updated because the embedded OpenCV library which is about to be installed on ARM board will need these packages to get it work properly. Nevertheless, OpenCV is quite hard to be compiled and installed on an embedded system as it uses different compilers in comparing with Linux running on PC. To get external IOs work, users should know a little bit about device tree and overlay knowledge. It is more like a hardware configuration layer covering on the board to get all the external hardware function work. Finally, users can develop their own applications on their hardware. This work needs a lot of testing and experiments to get the final application works correctly. Besides, all the single modules will also be checked carefully among all the device connections.

Fig. 3. ARM configuration

III. VISUALODOMETRY

Visual odometry is a new way of practical localization and navigation technique used in robotics or autonomous vehicles. By analyzing sequence image data captured from stereo cameras, we can extract information to localize the system. In modern robotics nowadays, this technology is commonly used to increase the artificial intelligence of robots or any smart system integrations.

Fig. 4. Visual odometry algorithm overview [1] A. Stereo Camera

Stereo camera is simply a device made of two cameras. Camera can be a webcam or a camera module. So how is a stereo camera that meets specification requirements? There are many specification criteria such as frame rate, resolution, baseline, communication protocol, type of output imag

e…

In general, some hardware specifications need to be taken into consideration such as baseline, field of view (FOV), image sensor specifications (size of sensor) and focal length. Besides, some aspects to be concerned are output image resolution, interface protocol, frame rate, type of compression and

decompression supported, hardware compatible…

Camera selection is very important as it affects directly to the quality of the system. As regards to recommendations, higher resolution and higher frame rate are definitely better. Moreover, hardware compatible and connection type are issues to be taken care of because it is related to the input of the boards we are using. Baseline is also a curious specification as there is no rule for it. It really depends on how far your view would be so the farther the view the wider the baseline.

B. Image Rectification

Qualified camera gives quality output image. Input images captured from camera are called raw images and they cannot be putted into processing at once. They need to be modified over a

few steps before getting a reliable usable pair of stereo images.

(15)

So, here comes the necessary of pre-processing and post- processing procedures. For the pre-processing procedure, stereo camera needs to be calibrated to get its hardware specifications. This procedure is very important as it would return valuable parameters for a precise calculation in upcoming steps. A chessboard was used as a calibration tool. Chessboard corners are key points and examined factors for the calibration procedure. The more pictures of chessboard we took, the more precise in parameter values returned by the calculation. OpenCV provides stereoCalibrate function [3] which returns camera parameters, distortion coefficients and relationship of rotation matrix and translation vector between 1st and 2nd camera coordinate systems. All the computed parameters will be used to adjust the distortion and rectify

images to get final reliable useful stereo images.

Fig. 6. Stereo image rectification process [4] C. Feature detection

For feature detection process, there are many feature

detectors to find edge, corner, blob… In this paper, corner

detectors were needed because the purpose of the project is to base on movements of these key points in sequenced images to estimate distance among objects in environment and redraw

system’s

trajectory. Thanks to OpenCV for providing useful feature detection functions. A simple comparison was made to check the quality of each corner feature detectors supported by OpenCV [5]

Fig. 7. Corner detectors compare in average detection time [5]

Fig. 8. Corner detectors compare in number features found [5] In this work, SURF and FAST feature detectors were examined to identify which function is better and faster than others.

SURF FAST Feature Points 400 ~ 1000 5000 ~ 10000 Processing Time ~70ms ~ 5ms

SURF and FAST detectors comparison

These are very important factors for a real-time standalone application and embedded devices.

Let’s see how they affects

to the whole processing time in the Experimental Setup and Result section.

D. Disparity and Triangulating 3-D points

Having had all the key points presented on left and right screens, they will be matched coordinately to calculate their disparity. For the matching procedure, there are two ways to do this which are BruteForceMatcher function and FLANNBasedMatcher function [6]. The quality of two functions is the same but as regards to properties and computing methods, processing time of each method experiments a vast difference. BruteForceMatcher is a simple and basic matching function which analyzes descriptor of key point in the 1st image to find the approximately closest descriptor of each key point in the 2nd image. It takes comparison from pairs to pairs which are really a serious time consuming process. On the other hand, FLANNBasedMatcher performs a quick and efficient matching by using Fast

Approximate Nearest Neighbor Search Library. This function is optimized to work with a huge and multi-dimensions database which sounds like an ideal matcher that we are looking for.

(16)

Each feature points will have different position on the image planes as presented as xl, xr on left and right image plane. Based on the difference on these two positions, disparity

value is calculated by the following equation:

  



_{ }



_



 (1)

Where:

 xl, xr is coordinate position of object projected on

image plane in pixel

 B is a baseline in meter  f is a focal length in pixel  d is disparity in pixel

From the above equation, focal length of camera is in pixel unit so a small transformation was made to convert focal length in meter into pixel.

    

( )  ( )_{( )}



[][]

[] (2)

    

( )  ()

() (3)

From now on, 3-D position of key points can be calculated easily according to similar triangles in Fig. 9 and return final equations:

 

 

 

 

 

  (4)

Shortly, depth of a point in reality is a reverse of the variance between corresponding points in left and right image comparing to center of the camera. Another way of saying is that the smaller the disparity value is, the farther the Z position value of the point will be correspondingly.

Fig. 10. Distance is reversely proportional with disparity [4] E. Feature Tracking

In the feature tracking algorithm, calcOpticalFlowPyrLK function in OpenCV [3] was used to track the 3-D feature points above. We assumed that previous image and present image stood still in a very short transition time, so we can track how these 3-D points drifted between 2 frames based on the difference in the complex optical flow. In computer vision, this method is commonly used in tracking and estimating motion of points and objects. This method is less sensitive with light distortion rather than other method which is a plus criterion for users to put this method onto their project.

Fig. 11. Estimated position of tracked features [2] F. Motion Estimation using RANSAC & SVD algorithm

To calculate the movement of a point cloud, a minimum of three points and their positions before and after the move is required. We now formalize the idea of movement as follows



_

⃗  

⃗  

_ (5)

where pi is any point from the point cloud in the past frame, xi

is its corresponding point from the point cloud in the current frame, R is a rotation matrix, and t is a translation vector. To obtain the new location xi of point pi, location is rotated about

some points by R and then translated by t.

Fig. 12. Sample movement estimation calculation [1]

Assuming an ideal point set, Besl’s method

[8] would be solution to the motion problem: pick any three points, and calculate R and t. However, the data is highly error-prone

(especially along the dimension of the camera’s optical ray), so

just using this method would result in gross error that would render the implementation unusable. Therefore, RANSAC algorithm is introduced, a random-sample consensus algorithm, is able to eliminate gross outliers and perform least-squares estimation on the valid data points.

RANAC is immune to gross outliers (also known as poisoned data points). Applied to this particular problem, the

RANSAC algorithm can be presented as the following sequence of steps [1]:

1. Pick three points and use the 3-point problem solution presented above to calculate the R matrix and t vector.

(17)

2. Apply R and t to the entire past point cloud. If the transformation is perfect, the two sets should now overlap completely. That will not be the case, so we identify the points that are within a distance e of their positions in the current point cloud, and call them the support set for this particular

hypothesis.

a. If the support set has t or more members, then we call it the optimal set and move to step 3.

b. If the support set has less than t members, we go back to step 1 and pick another hypothesis. We repeat this up to k times. If we cannot find a hypothesis with more than t members, then we pick the hypothesis with the largest support set to be optimal.

3. Once the optimal hypothesis is determined, we re-solve the model with the entire support set for that hypothesis. If we have more than 3 points in the support set, the system will be over-constrained, and we use a least-squares technique (described later) to come up with the polished Rand t.

In the last step of RANSAC algorithm, a least-squares solution which is a singular-value decomposition (SVD) method is found to an over-constrained system of equations.

Fig. 13. SVD implementation [1] IV. EXPERIMENTALSETUP ANDR ESULTS

In the simulation model, 2 standard 5MP webcams are connected to laptop to develop the whole stereo image processing from the raw images to the system localization.

Fig. 14. Simulation stereo camera model

The hardware model was made of two TRDB-D5M cameras connected to ALTERA FPGA Board DE2-115 through GPIO-HSMC External Card THDB-H2G. FPGA is programmed in Verilog language and configured to capture and

save stereo images on DDRAM as well as SDCard at the same time. Frame rate performs at 5 frames per second. Resolution was 640x480 pixels. So far, captured images were store in SDCard and they would be transmitted to ARM board later. At present, the system cannot process stereo images stream online.

Fig. 15. FPGA DE2-115 and stereo camera TRDB-D5M Process for implementing the application on BBB can be described as following steps

Fig. 16. Application process development

Applications will be programmed and developed by Visual Studio on Window operating system. Some codes were developed by MATLAB. Then, applications will be modified to adapt to Linux operating system environment and embedded Linux operating system on BBB after that. On the other hand, applications can be developed directly on Linux as well as embedded Linux.

Data after being transmitted to BeagleBone Black (BBB) board will be calculated and redraw trajectory of the system. Besides, PC can connect to BBB to control all the process, give command and read data. BBB and DE2-115 board have many external I/O functions, so they can be expanded and developed

to control motors, LCD, sensors, web server…

Calibration procedure captured 15 pairs of chessboard stereo images from different angles and result:

Fig. 17. Stereo camera calibration result

To test the algorithms on computer, a sample stereo image database was used to test the system. The database belongs to Bumblebee stereo camera of Pointgrey Company. Trajectory was recorded in meter unit as all the computed 3-D points were in meter unit.

(18)

Fig. 18. Algorithm procedure

First of all, feature key points found by SURF detector were matched correspondingly by FLANNBasedMatcher function and triangulated 3-D position. After having 3-D position of key points at time t, these key points were tracked in separated sequence of images in each camera to check how these points drifted in frame captured at time (t+1) by using calcOpticalFlowPyrLK function. At this time, it is capable to triangulate 3-D position of key points at time (t+1). As result, all the 3-D key points were found in previous stereo images and present stereo images which were input parameters for running the RANSAC algorithm. Trajectory record can be displayed on Google map according to appropriate system ratio.

Fig. 19. Trajectory record V. CONCLUSION ANDFUTURE WORKS

The final results recorded were not as good as expectation. Currently, both hardware and software have errors found and being checked all the procedures from the beginning till the

end as soon as possible. Moreover, an important issue needs to be solved is the communication interface between FPGA and ARM boards. Hardware replacement would be made if necessary, in case the problem cannot be solved at once.

In the near future, the system will be improved by undertaking more practical experiments and optimizing algorithms. Some comparisons with other type of

measurements such as encoders, GPS, laser scanner… would

be made to test the quality of the system. Furthermore, combination among these methods would be made to improve the result as well. By taking advantage of multi-functional and various external I/O pins, both FPGA and ARM have its own potential for users to implement or develop their own system.

ACKNOWLEDGMENT

Thanks for the sponsor and financial support of the honor program funds of Faculty of Electrical and Electronics Engineering, Ho Chi Minh City University of Technology, for students in scientific research in 2013. This work was first presented in December 19th 2013 as a scientific project and was

developed to the final thesis presented in January 7th 2014. We would like to acknowledge Mr.

Lê Văn Thạ

nh from class 2008 for providing us sample image database [7] and Dr.Nguy

ễn Vĩnh Hả

o

–

our supervisor

–

for enthusiast guidance and support.

R EFERENCES

[1] Yavor Georgiev, “E90 Project: Stereo Visual Odometry,” 2006

[2] Sergio A. Rodriguez F., Vincent Fremont, Philippe Bonnifait, “An

Experiment of a 3D Real-Time Robust Visual Odometry for Intelligent

Vehicles,” Universitede Technologie de Compiegn, France, Oct 2009 [3] OpenCV documentation [online] http://opencv.org/documentation.html [4] Gary Bradski & Adrian Kaehler, Learning OpenCV. Computer Vision

with the OpenCV Library. O’Reilly Media, Inc., 2008

[5] Levgen Khvedchenia. (January 4th_{2011). Comparison of the OpenCV’s}

feature detection algorithms [online]. Available: http://computer-vision- talks.com/2011/01/comparison-of-the-opencvs-feature-detection-algorithms-2/

[6] Robert Laganière, OpenCV 2 Computer Vision Application Programming Cookbook .

[7] Robot pose estimation using Stereo visual odometry – Van-Thanh Le, Tam-Trung Dang, Vinh-Hao Nguyen, Ho Chi Minh City University of Technology, Vietnam, 2012

[8] Besl, P., McKay, N., "A Method for Registration of 3-D Shapes," IEEE Trans. PAMI, Vol. 14, No. 2, February, 1992, pp. 239-256

(19)

Indoor Mobile Navigation Using ROS

Thien-Minh Nhat Nguyen-Pham

Department of Automatic Control Department of Automatic Control Faculty of Electrical and

Faculty of Electrical and Electronics EngineeringElectronics Engineering Ho Chi Minh City University of Technology Ho Chi Minh City University of Technology

[email protected] [email protected]

Abstract

Abstract _—_—Developing the ability to navigate and travelDeveloping the ability to navigate and travel through an dynamic environment for a mobile robot is a complex through an dynamic environment for a mobile robot is a complex problem. This article presents the implementation of an problem. This article presents the implementation of an intelligent indoor mobile robot using several sophisticated intelligent indoor mobile robot using several sophisticated navigation algorithms and applications from the Robot navigation algorithms and applications from the Robot Operating System framework. The robot is able to localize itself, Operating System framework. The robot is able to localize itself, update the information of the environment to make a path plan update the information of the environment to make a path plan and follow it to reach an assigned goal. This motivates the future and follow it to reach an assigned goal. This motivates the future

research on similar navigation problems for outdoor

environment as well as developing several indoor robot environment as well as developing several indoor robot applications such as building guide, household assistant or applications such as building guide, household assistant or warehouse deliverer.

warehouse deliverer.

Keywords

Keywords _—_—Navigation; SLAMNavigation; SLAM ; M; M onte onte Carlo meCarlo methodsthods; L; L asaseerr bea

beams; Path pms; Path planlan nini ng.ng.

I.

II

NTRODUCTI NTRODUCTIONON

Mobile Navigation can be seen as a set of three problems. Mobile Navigation can be seen as a set of three problems. When a robot is given a goal to reach, it should firstly ask the When a robot is given a goal to reach, it should firstly ask the question

question where am I?where am I? This pThis problem is roblem is calledcalled self-loca self-localizationlization.. The next question is

The next question is what is the world likewhat is the world like? or the? or the mapping mapping problem. Finally,

problem. Finally, when the when the robot knrobot kn ows where ows where it it is is and and whatwhat the world is like, it should ask the question

the world is like, it should ask the question how should I travelhow should I travel through this?

through this? This is theThis is the path-plann path-planning ing problem. Of three, the problem. Of three, the two problems of

two problems of self-localiza self-localizationtion andand mapping mapping are oftenare often inseparable and relate to each other in a

inseparable and relate to each other in a chicken and eggchicken and egg loop.loop. When the robot knows where it is, it will base on this intuition When the robot knows where it is, it will base on this intuition to add the newly detected features to the map and then use this to add the newly detected features to the map and then use this map to know how much displacement it has made since the last map to know how much displacement it has made since the last update in the n

update in the n ext loop. These two problems are often groupedext loop. These two problems are often grouped together as the Simultaneous Localization and Mapping together as the Simultaneous Localization and Mapping problem, o

problem, or SLAMr SLAM..

Previous implementations of autonomous robot [1] and Previous implementations of autonomous robot [1] and path

path planner [2] planner [2] have have achieved a achieved a level of level of navigation navigation with with thethe ability to avoid obstacle to reach the assigned goal. However, ability to avoid obstacle to reach the assigned goal. However, these approaches used a deterministic model for the these approaches used a deterministic model for the localization and mapping process. In this approach, the localization and mapping process. In this approach, the mapping has to start from scratch and the process only mapping has to start from scratch and the process only observes the boundaries of the space the robot can travel in observes the boundaries of the space the robot can travel in along

along with the assumption that the sensor’s measurement waswith the assumption that the sensor’s measurement was performe

performed d with with almost almost absolute absolute accuracyaccuracy. Also, the robot’s. Also, the robot’s self-loc

self-localization did not alization did not base on the map base on the map but the confidence onbut the confidence on the robot’s ability to perform accurate motion

the robot’s ability to perform accurate motion, which is, which is separate

separated into d into translattranslational and ional and in-place rotational movesin-place rotational moves, , thus,thus, the robot has low flexibility and the error in estimation will the robot has low flexibility and the error in estimation will accumulate through time.

accumulate through time.

In this project, we provided the robot with a prior In this project, we provided the robot with a prior knowledge, or a static map, about the environment and an knowledge, or a static map, about the environment and an initial location inside this map. The relation between the initial location inside this map. The relation between the

robot

robot’s’s motion and i motion and its position-ots position-orientation is also observed in arientation is also observed in a probabilis

probabilistic mannetic manner. Thus the error. Thus the error will permear will permeate in the modelte in the model but still

but still under control and under control and will preserve the basis will preserve the basis for the robotfor the robot to reconsider the former locations at later stage. This method is to reconsider the former locations at later stage. This method is based

based on on the the Monte Monte Carlo Carlo LocaliLocalization zation (MCL) (MCL) algorithm algorithm foforr robot localization and will be explained more in the next part. robot localization and will be explained more in the next part. In general, we have implemented some interactive methods for In general, we have implemented some interactive methods for mobile navigation in this paper:

mobile navigation in this paper:



 We provided the robot with a map from an explorationWe provided the robot with a map from an exploration

SLAM process and even a simple hand-drawn map. SLAM process and even a simple hand-drawn map. The map needs not to explicitly describe the robot The map needs not to explicitly describe the robot’s’s operating environment, but only static features as the operating environment, but only static features as the landmarks for the robot to localize itself in it.

landmarks for the robot to localize itself in it.



 During operation the robot could update the map withDuring operation the robot could update the map with

new obstacles or free space from its observation. new obstacles or free space from its observation.



 We applied a path planner to come up with anWe applied a path planner to come up with an

immediate plan to follow and reach the assigned goal. immediate plan to follow and reach the assigned goal. During the navigation proces

During the navigation process, the s, the plan can be plan can be adaptedadapted to newly acknowledged features in the map.

to newly acknowledged features in the map.



 We applied a method to control the We applied a method to control the robot in a robot in a velocity

velocity--oriented way rather than a sequence of translation and oriented way rather than a sequence of translation and rotation, thus the robot could move flexibly on a path rotation, thus the robot could move flexibly on a path optimized for its shape and kinematic constraints. optimized for its shape and kinematic constraints.

II.

II. ROSROS N NAVIGATIONAVIGATIONSSTACK TACK

A.

A. Overview of ROS Navigation StackOverview of ROS Navigation Stack

The ROS Navigation Stack is an ROS framework to solve The ROS Navigation Stack is an ROS framework to solve the navigation problem for an indoor robot. The libraries and the navigation problem for an indoor robot. The libraries and applications of ROS are supported in

applications of ROS are supported in packages packages. In the scope of. In the scope of an operating system, a task in ROS is implemented by a node an operating system, a task in ROS is implemented by a node and the set of nodes in an application is called the and the set of nodes in an application is called the computational graph. The communication between nodes computational graph. The communication between nodes follows the

follows the publication publication/subscrip/subscriptiontion mechanism. The core of mechanism. The core of Navigatio

Navigation Stack is a nn Stack is a nodeode move_basemove_base that will update a static that will update a static map with the data received from the sensor sources. It then map with the data received from the sensor sources. It then combines this map with the kinematic constraints of the robot combines this map with the kinematic constraints of the robot to calculate an appropriate path to avoid collision with to calculate an appropriate path to avoid collision with obstacles in the world. This node will then output the velocity obstacles in the world. This node will then output the velocity commands via the

commands via the cmd_velcmd_vel topic to drive the mobile base to topic to drive the mobile base to follow this path plan. The velocity command in ROS follow this path plan. The velocity command in ROS Navigatio

Navigation n Stack Stack is is called called aa twist twist . It differs from the simple. It differs from the simple rot-trans-rot

rot-trans-rot procedure (rotate, then translate and then rotate procedure (rotate, then translate and then rotate again).

again). TheThe base_controllerbase_controller node is created by user to suit thenode is created by user to suit the mobile base in different robot platform. ROS Navigation Stack mobile base in different robot platform. ROS Navigation Stack supports both holonomic and non-holonomic robot. User supports both holonomic and non-holonomic robot. User should consider the format of the

(20)

contains the linear and angular velocities on each direction of contains the linear and angular velocities on each direction of the robot’s coordinate frame to develop a control loop for the the robot’s coordinate frame to develop a control loop for the driving actuators. An important setup diagram for ROS driving actuators. An important setup diagram for ROS Navigatio

Navigation Stack can be n Stack can be found at [3]found at [3]..

Figure 1. Intelligent and interactive approach for mobile navigation using Figure 1. Intelligent and interactive approach for mobile navigation using Navigation Stack.

Navigation Stack.

The remaining problem when applying Navigation Stack is The remaining problem when applying Navigation Stack is localization. To implement the ROS Navigation Stack. One can localization. To implement the ROS Navigation Stack. One can simply count on the odometry of the robot. Many kinds of simply count on the odometry of the robot. Many kinds of sensors can be fused togeth

sensors can be fused together to determine the robot’s locationer to determine the robot’s location with great accuracy. However, as mentioned above, the with great accuracy. However, as mentioned above, the problem of determining a

problem of determining a model is model is that we that we can ncan n eveever r have have anan exact model due to noise and the error will also accumulate exact model due to noise and the error will also accumulate through time. The Navigation Stack provides an optional node through time. The Navigation Stack provides an optional node amcl

amcl that implements the Adaptive Monte Carlo Localization that implements the Adaptive Monte Carlo Localization (AMCL) algorithm with adjustable parameters for a model of (AMCL) algorithm with adjustable parameters for a model of robot moving in a

robot moving in a 2D environment with 2D laser scan sensor as2D environment with 2D laser scan sensor as the observing source. The MCL is a very popular method for the observing source. The MCL is a very popular method for tthe robot’s global localization.he robot’s global localization. In general, the MCL approach In general, the MCL approach describes the robot motion in a probabilistic manner and takes describes the robot motion in a probabilistic manner and takes account of the

account of the robot’s motionrobot’s motion, the sensor data and a prior, the sensor data and a prior knowledge about the environment to find the belief on the knowledge about the environment to find the belief on the current position-orientation of the robot

current position-orientation of the robot.. The adjectiveThe adjective adaptiveadaptive added

added to the MCL to the MCL method used in method used in the ROS the ROS Navigation StackNavigation Stack relates to an advanced

relates to an advanced resampling resampling algorithm for the weighted algorithm for the weighted particle

particle filter filter .. The theory to utilize these parameters will beThe theory to utilize these parameters will be described further in the next part.

described further in the next part. B.

B. Implementation the ROS Implementation the ROS Navigation Stack.Navigation Stack.

This part describes the practical work to implement the This part describes the practical work to implement the Navigation Stack o

Navigation Stack on a mobile robot platfon a mobile robot platform as well as therm as well as the underlying theorem for localization to utilize the libraries from underlying theorem for localization to utilize the libraries from ROS.

ROS. 1.

1. OdometryOdometry

Odometry is the use of data from moving sensors to Odometry is the use of data from moving sensors to estimate change in position over time. These sensors can be estimate change in position over time. These sensors can be wheel encoders and IMU. To supply Navigation Stack with an wheel encoders and IMU. To supply Navigation Stack with an odometry source we will assess our robot’s motion model and odometry source we will assess our robot’s motion model and use the estimated

use the estimated pose pose from this model as an input for the from this model as an input for the Monte Carlo Localization.

Monte Carlo Localization.

Figure 2. The motion of differential steered mobile. Figure 2. The motion of differential steered mobile.

Let us consider a differential steered mobile platform as Let us consider a differential steered mobile platform as shown in Fig. 2. At a time instance

shown in Fig. 2. At a time instance



the ideal, noise-free robot the ideal, noise-free robot has a

has a pose pose









where where



,,



are the coordinates of the are the coordinates of the robot in the world frame and

robot in the world frame and



is called the is called the heading heading angleangle.. Also the robot has a pair of linear and angular velocities Also the robot has a pair of linear and angular velocities





. Suppose that we keep the velocities fixed during a time. Suppose that we keep the velocities fixed during a time interval

interval

 



. The robot will have a circular movement. The robot will have a circular movement around the instantaneous center

around the instantaneous center



with radius with radius







(2.1) (2.1) andand (2.2) are the relation between the linear coordinates in the pose (2.2) are the relation between the linear coordinates in the pose





and the coordinates of and the coordinates of



..











))



(2.1)(2.1)











))



(2.2)(2.2)

Using the same relation of (2.1) and (2.2) for the Using the same relation of (2.1) and (2.2) for the instantaneous center

instantaneous center



and the new pose and the new pose





















atat



we have: we have:











(2.3)(2.3)











(2.4)(2.4)

Combine (2.1), (2.2), (2.3), (2.4) we have the relation Combine (2.1), (2.2), (2.3), (2.4) we have the relation betw

betweeneen





and and





 





 



_{}







(2.5)(2.5) Denote

Denote





 



to be the velocities of the right and left to be the velocities of the right and left wheels respectively. Also, we have

wheels respectively. Also, we have



to be the distance to be the distance betw

between the two ween the two wheels. From theels. From the circular he circular movememovement innt in



, we, we have the relation:

have the relation:





 



))

(2.7)(2.7)





 



))

(2.8)(2.8)

 

(2.9)(2.9)

Manipulating the equations from (2.7) to (2.9) we have Manipulating the equations from (2.7) to (2.9) we have













_







(21)

(2.5) and then substitute











,,











wewe have: have:

 



_[[

















































_









_

_





_



_





_{}













]]

(2.10) (2.10) Where

Where





and and





are the changes on the encoder count are the changes on the encoder count after each time interval.

after each time interval.



is the number of counts per is the number of counts per revolutio

revolution n andand



is the wheel is the wheel’s’s diameter. The above relation is diameter. The above relation is indefinite when

indefinite when









, this is the case when the robot moves, this is the case when the robot moves in a

in a straigstraight path. ht path. We can overcome this by finding the limit ofWe can overcome this by finding the limit of (2.10) when

(2.10) when











, which is:, which is:

 







_



_

_







_

_{}

_{ }

(2.11)(2.11) Also when the change in the heading is not so large, this Also when the change in the heading is not so large, this limit can also be applied to other values of

limit can also be applied to other values of











.. Since the motion of the robot follows a

Since the motion of the robot follows a twist twist manner, rather manner, rather than the

than the rot-trans-rot rot-trans-rot , it is useful to find, it is useful to find





andand





from from



and and



to feed to the control loop of each wheel. From (2.7) to (2.9), to feed to the control loop of each wheel. From (2.7) to (2.9), we have:

we have:











(2.12)(2.12)











(2.13)(2.13) 2.

2. Building a m Building a mapap

To make a map, one has to rely on the self-localization To make a map, one has to rely on the self-localization proces

process. In this s. In this paper, we applied the grid mappipaper, we applied the grid mapping applicationng application from the ROS community to build up simple maps. We from the ROS community to build up simple maps. We assumed that the level of accuracy from our equations for assumed that the level of accuracy from our equations for odometry is sufficient so that the self-localization process can odometry is sufficient so that the self-localization process can use the output from this as the actual location of the robot. Fig. use the output from this as the actual location of the robot. Fig. 2(a) is a map of an area of 4.6x3.5m

2(a) is a map of an area of 4.6x3.5m22 with only plain with only plain walls andwalls and doors. In this area, the accumulated error was low enough so doors. In this area, the accumulated error was low enough so that the ROS mapping process gave a good map with each that the ROS mapping process gave a good map with each pixel covers an

pixel covers an area of 1cmarea of 1cm22. In Fig. 2(b), each pixel covers a. In Fig. 2(b), each pixel covers a larger area of 5cm

larger area of 5cm22 in a room which is in a room which is 45m45m22 large. In this case, large. In this case, the scenery of the area contains many

the scenery of the area contains many features features so an so an interestiinterestingng combination of visual and encoder odometry as introduced in combination of visual and encoder odometry as introduced in [4] can be applied for a better result.

[4] can be applied for a better result.

Figure 2. Scanned maps using odometry data and Kinect sensor and ROS grid Figure 2. Scanned maps using odometry data and Kinect sensor and ROS grid mapping package.

mapping package.

Prior knowledge needs not to be acquired only from Prior knowledge needs not to be acquired only from exploration. As long as it is accurate and sufficient, both exploration. As long as it is accurate and sufficient, both methods would help the robot localize itself accurately. Fig. 3 methods would help the robot localize itself accurately. Fig. 3

is a map of an area of 20x12m

is a map of an area of 20x12m22 with each pixel covering an with each pixel covering an area of 1cm

area of 1cm22 and the robot could travel from one side to and the robot could travel from one side to another with great accuracy.

another with great accuracy.

Figure 3. Map of a floor in a building drawn and given as prior knowledge to Figure 3. Map of a floor in a building drawn and given as prior knowledge to the robot.

the robot.

3.

3. Monte Carlo LocalizationMonte Carlo Localization

According to [6] the Monte Carlo Localization is arguably According to [6] the Monte Carlo Localization is arguably one of the most popular method for the

one of the most popular method for the robot’s globalrobot’s global localization. Within the terminological expression of localization. Within the terminological expression of probabilis

probabilistic tic robotics, robotics, this this approacapproach h hypothesizes hypothesizes the the roborobott motion in a probabilistic manner and takes in account the motion in a probabilistic manner and takes in account the odometry

odometry value, the value, the measurement measurement and a pre-known and a pre-known mapmap toto describe the

describe the belief belief on its current pose by a on its current pose by a particle particle filter.filter. InIn ROS we can adjust the parameters of the

ROS we can adjust the parameters of the amclamcl package by package by giving values to the

giving values to the node’snode’s paramet parametersers when launching it to the when launching it to the computational graph.

computational graph.

To understand the basics of MCL algorithm, let us denote To understand the basics of MCL algorithm, let us denote





to be a pair of odometry values to be a pair of odometry values

̅̅



̅

̅  ̅ ̅

and and

̅̅



̅  ̅ ̅

̅

which are the estimated poses from the which are the estimated poses from the odometer in the last time interval. One can obtain

odometer in the last time interval. One can obtain





using the using the method that we have applied to come up with the equation method that we have applied to come up with the equation (2.10).

(2.10). Denote

Denote





{{



 







}}

to be a measurement consisting to be a measurement consisting of

of



individual range values from a popular laser scan model individual range values from a popular laser scan model Denote

Denote





 









to be a map consisting of to be a map consisting of



points that

points that will describe the will describe the envienvironment in ronment in a a featufeature-based orre-based or location-based structure. In feature-based map, the member location-based structure. In feature-based map, the member





will consist of the coordinates of an obstacle in the will consist of the coordinates of an obstacle in the environment. In location-based map,

environment. In location-based map,





will contain a number will contain a number expressing the certainty a cell of the map being occupied by an expressing the certainty a cell of the map being occupied by an obstacle.

obstacle. The

The beliefbelief of the robot can be modeled by a possibilityof the robot can be modeled by a possibility distribution function

distribution function

((



||



 





that conveys the that conveys the meaning of the possibility of a pose

meaning of the possibility of a pose





when the odometer when the odometer returns

returns





, the sensor gives us a set of measurement, the sensor gives us a set of measurement





and theand the prior know

prior knowledge wledge was given as as given as a mapa map



.. Also we will denote

Also we will denote

 



 



 



 



to be the set to be the set of M particles, where a particle is a specific pose of the robot. of M particles, where a particle is a specific pose of the robot. The intuition behind the particle filter is that though one can The intuition behind the particle filter is that though one can find

find

((



||



 





from the basic componential distribution from the basic componential distribution functions, such as the Gaussian or uniform ones, and the functions, such as the Gaussian or uniform ones, and the relation between then such as superposition or convolution, this relation between then such as superposition or convolution, this combination is not a convenient way to describe a combination is not a convenient way to describe a

(a) (b)