• No results found

TRENTINO - The research, training and mobility programme in Trentino - PCOFUND-GA

N/A
N/A
Protected

Academic year: 2021

Share "TRENTINO - The research, training and mobility programme in Trentino - PCOFUND-GA"

Copied!
7
0
0

Loading.... (view fulltext now)

Full text

(1)

Ricercatore: Ilya Afanasyev Soggetto

ospitante: UNIVERSITA' DEGLI STUDI DI TRENTO

Bando: Incoming post doc 2009

Soggetto partner (solo per outgoing):

e-mail: ilya.afanasyev@unitn.it, ilya.afanasyev@gmail.com

Ai sensi della Decreto Leg. n. 196 del 30/06/2003 – Codice in materia del trattamento dei dati personali - autorizzo la pubblicazione sul Sito internet della PAT dell’abstract e delle immagini (foto o filmati) relativi al mio progetto.

Area di ricerca: Mechathronics / Meccatronica Acronimo Laser-Camera systems

Titolo Laser-Camera systems and algorithms to sense the environment for robotic applications Data inizio 12 maggio, 2010 Durata 36 Finanziamento: € 149.000,00

(2)

Provincia Autonoma di Trento European Union – 7th research framework programme Marie Curie Actions – COFUND

“TRENTINO”

- The research, training and mobility programme in Trentino -

PCOFUND-GA-2008-226070

The final goal of the project is to build and mount a system of a laser rangefinder (LRF) and a camera (oriented and calibrated together with LRF) on a robotic vehicle, using them together with other 3D sensors and concentrating on Robotic and Human-Machine Interface (HMI)

applications by creating algorithms to sense the environment. The general purposes of the project are:

1. Building a sensor system, including the Laser-Camera (LCS), for robotic applications. 2. Mounting LCS on the robotic vehicle (initially – on a prototype of a robotic vehicle, then - on the robot Pioneer P3-DX).

3. Development of objects recognition and localization algorithms, based on processing 3D point cloud obtained from sensor system (LCS, Kinect, camera, etc.).

4. Verification of the algorithms with hardware facilities, including tests with different 3D sensors and the robot “Pioneer P3-DX”.

   

The sensor system includes the laser rangefinder (LRF), RGB camera and also an additional sensor (Microsoft Kinect for data validation). This sensor system captures a target object as 3D point cloud, which is structured and processed by object localization and recognition algorithms together with simultaneous processing of images from 2D camera. For sensor fusion the sensor system was mutually calibrated, using the standard camera calibration technique (Bouguet method). Microsoft Kinect sensor was applied to (a) validate LCS measurements, and (b) control accuracy and precision of object location. The joint 3D and 2D data are processed by the state-of-the-art object recognition and localization algorithms, which can be used for path planning /

mapping the indoor robots (based on sensor integration of RGB camera and a 2D laser rangefinder) and also for different scientific and industrial applications. For these purposes:

- The Sensor System has been investigated by indoor environmental conditions of the

Mechatronics lab., UniTN, both independently and together with the robot “Pioneer P3-DX” (after on-board installation).

- The object recognition and localization algorithms have been created.

- The sensor data has been validated and the measurement accuracy and precision have been controlled with Microsoft Kinect.

  The main accomplishments obtained by the project are:

- Building the Laser-Camera system for robotic applications (Fig.1).

- Mounting laser-camera system on the robot “Pioneer P3-DX” (see Fig. 2).

- Development and experimental verification of 3D objects recognition and localization algorithms (with different sensors and hardware facilities, Fig.3,4).

- Improvements of the object localization technique, achieving the measurement accuracy and precision comparable with Microsoft Kinect (Fig.5,6).

- The implementation of the algorithms with hrdwre facilities to enhance the motion control with object and environment mapping (Fig.7,8).

(3)

Figure 1. A laser and a camera oriented/calibrated together form the Laser-Camera system.

Figure 2. Robotic vehicle with Laser-Camera System (LCS).Left -the prototype of a robotic vehicle with installed LCS (before purchasing the robot in 2012). Right – LCS installed on the robot “Pioneer P3-DX”.

(4)

Provincia Autonoma di Trento European Union – 7th research framework programme Marie Curie Actions – COFUND

“TRENTINO”

- The research, training and mobility programme in Trentino -

PCOFUND-GA-2008-226070

Figure 3. Human pose recognition: the block diagram of 3D Human Body Pose algorithm (left), the result pose (right): photo, 3D point cloud, simulation in superquadrics.

 

Figure 4. 3D gesture recognition. Data acquisition (left) and the result of gesture pose estimation by Superquadrics (right).

(5)

Figure 5: Experiment setup layout, data were acquired both from a camera and Kinect. Chessboard is used to estimate camera and Kinect positions to provide the reference frame for the localization.

Figure 6: The comparison between the ellipses of uncertainty (green - for Kinect, blue - for new system) for the box (left) and the cylinder (central) localizations according to the reference grid. SuperQuadrics fitting with depth data of a cylinder and a box (right).

Possible impacts. 3D Laser-Camera System can provide 2D/3D measurements capturing 2D/3D data of an object (with distance and color information) for indoor applications.

Object recognition and pose estimation algorithms can be applied for:

- detection and reconstruction by math. models of different real objects (including complex objects, like human body, gesture recognition, etc.).

-detection, classification and selection a definite thing in a heap of different objects for robotic manufacturing (such as a search an object in mechanical details laying in mess).

(6)

Provincia Autonoma di Trento European Union – 7th research framework programme Marie Curie Actions – COFUND

“TRENTINO”

- The research, training and mobility programme in Trentino -

PCOFUND-GA-2008-226070

Figure 7: The set of some training images (for a cylinder and a box) and the final results of detection (the red rectangle surrounding the objects for test images).

(7)

Publications.

1. Afanasyev I. and De Cecco M. 3D Gesture Recognition by Superquadrics. // Proc. VISAPP conf. (Barcelona, Spain), V.2, 429-433 (February, 2013). ISBN: 978-989-8565-48-8.

(www.ing.unitn.it/~afanasye/Publications/VISAPP_2013/)

2. Enrico Zappia, Ilya Afanasyev, Nicolo' Biasi, Mattia Tavernini, Alberto Fornaser, Antonio Selmo, and Mariolino De Cecco. Monocular object localization by superquadrics curvature reprojection and matching. // Proc. Graphicon-2012 conf. (Moscow, Russia), 33-38 (October, 2012). (http://www.ing.unitn.it/~afanasye/Publications/Graphicon_2012/)

3. Afanasyev I., Lunardelli M, De Cecco M., et al. 3D Human Body Pose Estimation by

Superquadrics. // Proc. VISAPP conf. (Rome, Italy), V.2, 294-302 (February, 2012). ISBN: 978-989-8565-04-4. (http://visapp.visigrapp.org/Abstracts/2012/VISAPP_2012_Abstracts.htm) 4. Afanasyev I. Laser-Camera systems and algorithms to sense the environment for robotic applications. // Abstract book of Marie Curie Researchers Symposium (Warsaw, Poland), P. 16 (2011). (www.mariecurie2011.pl)

Impacts on career. The project has strong impacts on career development, helping in:

-developing the professional skills in computer vision, human-machine interaction, robotics and algorithm developing;

- getting an experience in resource, time and money management;

-supervising students, getting experience in teaching with lectures and practical exercises; -making collaboration with leading research groups and centers in computer vision and

robotics both in Italy and abroad, incl. for searching possibilities of funding; -publishing papers, participating in international conferences with oral presentations. -strengthening the researcher reputation.

The experience obtained in during the project must be valuable for future work at a research center or an industrial company.

Figure

Figure 2. Robotic vehicle with Laser-Camera System (LCS). Left - the prototype of a robotic  vehicle with installed LCS (before purchasing the robot in 2012)
Figure 3. Human pose recognition: the block diagram of 3D Human Body Pose algorithm (left),  the result pose (right): photo, 3D point cloud, simulation in superquadrics
Figure 6: The comparison between the ellipses of uncertainty (green - for Kinect, blue - for new  system) for the box (left) and the cylinder (central) localizations according to the reference grid
Figure 8: The results of correct object localization detection for a cylinder and a box in general

References

Related documents

from a technical mixture with higher degree of chlorination to a technical mixture with lower degree of chlorination over this time period, (2) early, industrial, or combustion

Data related to student interaction with course content, including time spent reviewing online course materials, such as module PowerPoint presentations and course videos and

It was decided that with the presence of such significant red flag signs that she should undergo advanced imaging, in this case an MRI, that revealed an underlying malignancy, which

and instruments in demographic censuses to best capture the resident foreigners and those temporarily present, in the last two censuses the attention being mainly focused on

There are four main structural criticisms of the current regulatory structure: that consumer protection is a so-called “orphan” mission; that consumer protection conflicts with,

(2) In determining the amount of Monthly Tax Deduction based on Computerised Calculation , the employer shall allow the employee to claim allowable deductions and

A token describes an item which is checked for by the application software (here the sequencer software) upon installation or launch. If the application software cannot find

(d) Candidates who have not studied in any school / college in the state of A.P., but resided in Andhra Pradesh with private study in Andhra Pradesh & Telangana seven