Intelligent System Synthesis for Dynamic Locomotion Behavior in Multi-legged Robots

Intelligent

System

Synthesis

for Dynamic

Locomotion

Behavior

in Multi-legged

Robots

Thesis submitted for acquiring the degree

of

Master

of Engineering

by

Azhar

Aulia Saputra

Under the guidance of

Professor

Naoyuki

Kubota

Department of Intelligent Mechanical System Graduate School of System Design

Tokyo Metropolitan University

Tokyo, Japan

February, 6th 2018

Abstract

Robot technology has been implemented in many fields of our life, such as entertain-ment, security, rescue, rehabilitation, social life, the military, and etc. Multi-legged robot always exist in many fields, therefore it is important to be developed. Motion capabilities of the robot will be a main focus to be developed. Current development or conventional model of motion capabilities have several issues in saturation of development. There are some lim-itation in dynamic factors such as, locomotion generator, flexibility of motion planning, and smoothness of movement. Therefore, in this research, natural based computation are imple-mented as the basic model. There are three subsystems to be developed and integrated, (1) locomotion behavior model, (2) stability behavior model, and (3) motion planning model.

Since individual people has different walking behavior in each walking direction and walking speed, locomotion behavior learning model of omni-directional bio-inspired loco-motion which is generating different walking behavior in different walking provision are re-quired to be developed. Step length in sagital and coronal direction, and degree of turning are considered parameters in walking provision. In proposed omni-directional walking model, interconnection structures composed by 16 neurons where 1 leg is represented by 4 joints and 1 joint is represented by 2 motor neurons. In order to acquire walking behavior in certain walking provision, the interconnection structure is optimized by multi-objectives evolution-ary algorithm. For acquiring the diversity of references, several optimized interconnection structures are generated in optimization processes in different walking provisions. Learn-ing models are proposed for solvLearn-ing non-linearity of relationship between walkLearn-ing input and walking output representing the synaptic weight of interconnection structure, where one learning model representing one walking parameter. Furthermore, by using optimized model, walking behavior can be generated with unsealed walking provision. Smooth walking

tran-sition with low error of desired walking provision was proved based on several numerical experiments in physical computer simulation.

In stability behavior model, neuro-based push recovery controller is applied in multi-legged robot in order to keep the stability with minimum energy required. There are three motion patterns in individual people behavior when it gets external perturbation, those are

11

ankle behavior, hip behavior, and step behavior. We propose a new model of Modular Re-current Neural Network (MRNN) for performing online learning system in each motion be-havior. MRNN consists of several recurrent neural networks (RNNs) working alternatively depending on the condition. MRNN performs online learning process of each motion behav-ior controller independently. The aim of push recovery controller is to manage the motion behavior controller by minimizing the energy required for responding to the external per-turbation. This controller selects the appropriate motion behavior and adjusts the gain that represent the influence of the motion behavior to certain push disturbance based on behavior graphs which is generated by adaptive regression spline. We applied the proposed controller to the humanoid robot that has small footprint in open dynamics engine. Experimental re-sult shows the effectiveness of the push controller stabilizing the external perturbation with minimum energy required.

Proposed motion planning model presents a natural mechanism of the human brain for generating a dynamic path planning in 3-D rough terrain. The proposed model not only em-phasizes the inner state process of the neuron but also the development process of the neurons in the brain. There are two information transmission processes in this proposed model, the forward transmission activity for constructing the neuron connections to find the possible way and the synaptic pruning activity with backward neuron transmission for finding the best pathway from current position to target position and reducing inefficient neuron with its synaptic connections. In order to respond and avoid the unpredictable obstacle, dynamic path planning is also considered in this proposed model. An integrated system for applying the proposed model in the actual experiments is also presented. In order to confirm the effective-ness of the proposed model, we applied the integrated system in the pathway of a four-legged robot on rough terrain in computer simulation. For analyzing and proving the flexibility of proposed model, unpredictable collision is also performed in those experiments. The model can find the best pathway and facilitate the safe movement of the robot. When the robot found an unpredictable collision, the path planner dynamically changed the pathway. The proposed path planning model is capable to be applied in further advance implementation.

In order to implement the motion capabilities in real cases, all subsystem should be inte-grated into one interconnected motion capabilities model. We applied small quadruped robot equipped with IMU, touch sensor, and dual ultrasonic sensor for performing motion plan-ning in real terrain from starting point to goal point. Before implemented, topological map is generated by Kinect camera. In this implementation, all subsystem were analyzed and per-formed well and the robot able to stop in the goal point. These implementation proved the effectiveness of the system integration, the motion planning model is able to generate safe path planning, the locomotion model is able to generate flexible movement depending on the

111

walking provision from motion planing model, and the stability model can stabilize the robot on rough terrain. Generally, the proposed model can be expected to bring a great contribution to the motion capabilities development and can be used as alternative model for acquiring

the dynamism and efficient model in the future instead of conventional model usage.

In the future, the proposed model can be applied into any legged robot as navigation, supporter, or rescue robot in unstable environmental condition. In addition, we will realize a cognitive locomotion that generates multiple gaits depending on the 3 aspects, embodiment,

locomotion generator, and cognition model. A dynamic neuro-locomotion integrated with internal and external sensory information for correlating with the environmental condition will be designed.

平 成29年

度

修 士 論 文

IntelligentSystemSynthesisfbr

DynamicLocomotionBehaviorin

Multi-leggedRobots

首都 大学 東 京 大 学 院

シス テ ムデ ザ イ ン研 究科

博 士 前 期 課 程

知能 機 械 シ ステ ム学 域

学 修 番 号16889503

AzharAuliaSaputra

指 導教員

久保 田

直行

教授

平 成30年2月

概要

ロ ボ ッ ト技 術 は、エ ン ター テ イメン ト、 セキ ュ リテ ィ、救 助 、 リハ ビ リ、社会 生 活 、軍 事 な どの様 々な生 活 分 野 に実 現 さてい る。 多脚 ロポ ッ トは常 に多 くの 分 野 に存 在 す る ため 開 発す る こ とが重 要 で あ る。 ロ ボ ッ トの 運動 能 力 が開 発 の主 要 とな って い る。現 状 の開 発 されて い る動 作 能力 は,飽 和 状 態 にあ る。 い くつ か の 動 的 な要 因 に よ り、歩 行 生成 器 、動 作 計 画 の柔 軟性 、 お よび動 作 の滑 らか さ等 に 制 限 があ る。 そ こで 、本 研究 で は、基 本 的 なモ デ ル と して 自然 計算 に基 づ く方 法 論 を実 装 す る、 また 、本 研究 で は、歩 行 動 作 モ デ ル、安 定 動 作 モ デ ル 、や運 動 計 画 モ デル か ら なる3っ の サ ブシ ステムを 開発 し統 合す る。 人 間 は歩 行 方 向 と速 度 に応 じて歩 行動 作 が異 な る ため 、異 な る歩 行 軸 で は異 な る歩 行 動 作 を生 成 す る とい う全 方位 生 物 的な運 動 の歩 行 動 作学 習 モ デ ルが 開 発 には要 求 され る.球 欠お よ び制 御 方 向の ステ ップ長 や旋 回 の 度 合 い は,歩 行軸 の パ ラ メー タ と して考 慮 され る 。提 案 した全 方 位歩 行 モ デ'レで は,1肢 につ き16個 の ニ ュー一ロ ン に よ って構 成 され る相 互 接 続 構 造 を4つ の 関節 に よ って 表 現 す る. また 、1つ の 関 節 は,2個 の モー タニ ュー ロン に よ って 表 現 す る.一 定 の歩 行軸 で の歩 行動 作 を獲 得 す るた め に,本 研究 で は,多 目的進 化 ア ル ゴ リズム に よ うて 最 適 化 を行 う。 提案 手 法 で は、参 照点 の多 様性 を獲 得 す るため に,異 な る歩 行軸 に おい てい くつ かの最 適 な相互 接 続 構造 が生 成 され る。 相互 接 続構 造 の シ ≠ プ ス重 みを表 現 して い る歩 行 入 力 と出力 間 の非 線形 な 関係 を 解 くた め の学 習 モデ ルを 構 築 す る.本 手 法 で は,1つ の学 習 モデ ル が1つ の 歩 行 パ ラ メー タで表 現 され 、最 適 化 され た モ デ ルを 用 い る こ とに よ り,歩 行 動作 は,ス ケー リン グされて い ない 歩 行 軸 を生 成 す る こ とが可 能 とな る,物 理 演算 シ ミュレー シ ョンを 用 い た実 験 に よ り,誤 差 の少 ない歩 行軸 の 滑 らかな歩 行遷 移 を本実 験 で は示 して い る。 安 定動 作 モ デ ルで は、必 要 最小 限の エ ネルギ ーで安 定性 を維持 す るため 多足歩 行 ロ ボ ッ トに ニ ュー ロベ ー スプ ッシ ュ リカ バ リ制 御 器 を適 用 した 。 外 力を を受 け た とき,人 間 の行動 には足 首 の動 作 ・股 関節 の動 作 ・踏 み動 作の3つ の 動 作 パ ター ン が 存 在 す る 。 本 研 究 で は,各 運 動 動 作 に お け る オ ン ラ イン 学 習 シ ス テ ム を 実 現 す るた め に 、 モ ジ ュ ラー りカ レン トニ ュー ラル ネ ッ トワー ク(MRNN)を 用 い た 新 た な学 習 モ デ ルを提 案 す る。MRNNは 状況 に応 じて選 択 され る複数 のilカ レン V

トニ ユー ラル ネ ッ トワー ク(RNN)に よ って構 成 され る。MRNNは 各運 動動 作 コ ン トロー ラの オ ン ラ イン学 習 プ ロセ スを独立 して実 行 す る。 プ ッシ ュ リカバ リ制 御 器 の 目的 は、外 乱 に応 じて エ ネ ルギ ー最 小 化 を行 うこ とに よ って運 動 動作 制 御 器 を管 理 す る こ とで あ る。 この 制御 器 は適 切 な運 動 動 作 を選 択 し,適 応 回帰 ス プ ラ イン に よ り生 成 され た動 作 グ ラフに基 づ き押 し動 作 に対 して最 も影 響 を及 ぼす 運動 動作 の ゲ インの調 整 を行 う.提 案 した制 御器 をOpenDynamicsEngine(ODE) 上で小 さな足 の長 さを持 っ ヒ ュー マ ノイ ドロ ボ ッ トに適 用 し,必 要 最 小 限の エ ネ ル ギ ーで 外 力 に対 して安 定 させ る プ ッシ ュ リカ バ リ制 御 器 の 有 効 性 を 示 して い る。 3次 元 の 不 整地 に お け る動 的 な経 路 計画 を生 成 す るた めに,人 間 の 自然 な脳 機 能 に基 づい た 動作 計 画 手法 を提 案 す る。本 モ デル は 、 ニ ユー ロ ンの 内部 状態 過 程 だ けで な く、 脳 内の ニ ュー ロンの 発達 過 程 も重 視 してい る。 本 モデ ル は二 っ の ア ル ゴ リズム に構成 され る。1つ は、通過 可 能 な道 を見 つけ るため に構 築 され る接 続 的 なニ ュー ロ ン活 動 で あ る順 方 向 伝達 活 動 で あ り,も う1っ は、現在 位 置 か ら最 適 経 路 を 見 つけ るた め に 、 シナ プ ス結 合 を用 い て非 効 率 的な ニ ュー ロン を減 少 させ る逆 方 向 に ニ ュー ロン伝達 を行 うシナ プ スプ ルー ニ ン グ活 動 で あ る。 また,予 測 不可 能 な衝 突 を 回避 す るた め に,動 的 な経 路 計 画 も実 行 され る。 さ らに 、実 環 境 に おい て提 案 され た モ デル を実 現 す るため の 統合 シ ステム も提示 され る 。提 案 モ デ ルの有 効 性 を 検証 す るた め に,コ ン ピ ュー タシ ミュ レー シ ョン上 で 、不整 地 環 境 の4足 歩 行 ロ ボ ッ トに 関す る シ ミュ レー シ ョン環境 を実 装 した。 これ らの実験 で は,予 測不 能 な衝突 に 関す る実 験 も行 った。 本 モ デ ル は、最 適経 路 を見 つけ 出 し ロ ボ ッ トの 安 全 な移 動 を実 現 で き た。 さ らに 、 ロボ ッ トが予 測 で きな い衝突 を検 出 した場 合,経 路 計 画 ア ル ゴ リズム が経 路 を 動 的に変 更可 能で あ る こ とを示 して い る。 これ らの こ とか ら、提 案 された経 路 計 画 モ デ ル は さ らな る先進 的 な展 開 が 実現可 能で あ る と考 え られ る。 実 環 境 にお け る運 動 能力 を実 装 す るため に は、 す べて の サ ブシ ステ ムを1っ の 運 動 能 力 モ デ ル に統 合 す る必 要 が あ る。 そ こで本 研 究 で は 、IMU、 タ ッチ セ ン サ 、2つ の超 音 波 セ ン サを搭 載 した小型 の4足 歩 行pポ ッ トを 用 い た実 環 境 にお い て 出発地 点 か ら 目的地点 まで の運 動計 画 を行 った、本 実 装で は 、3次 元距 離計 測 セ ン サで あ るKinecを 用 い3次 元 空 間 の位 相 構造 ーを生成 す る。 また 、本 実 装 で は、 す べての サ ブシ ステム が分析 され 、 ロボ ッ トは 目的地 点 で 停止 す る こ とがで きた 。 さらに 、安 全 な経 路 計 画を 生成 す る こ とがで きた こ とか ら シス テ ム統 合 の有 効性 が確認 で きた 。 また 、歩 行 モ デ ルに よ り歩 行 軸 に応 じた柔軟 な動 きが生 成 され る こ とで 、 この 安 定 性 モ デ ル は 不 整 地 環 撹 で も ロ ボ ッ トの 歩 行 を 安 定 さ せ る こ と が で き た。 これ らの こ とか ら 、本 提 案 モデ ル は運 動 能 力 へ の 多 大 な貢 献 が期 待 さ れ 、 ダイナ ミ クスを 獲 得す る ため の 代替 モ デ ル と して使 用 す る こ とがで き,現 在

vii よ く使 用 され てい るモ デルに代 わ る効 率 的 なモ デル となる こ とが考え られ る. 今 後 の 課 題 と して は,不 安 定 な環 境 下 に お け る ナ ビゲ ー シ ョン ・支 援 ・レ ス キ ュー ロ ボ ッ ト とい った任意 の 肢 の数 を持 っ 多足歩 行 ロ ボ ッ トへ の本提 案 モ デ ル の適 用 があ げ られ る.さ らに,身 体性,歩 行生成,認 知 モ デ ルの3っ の 観点 か ら複 数 の歩 容 を生 成 す る認 知 的歩 行 を実 現 す る こ とを 考之 て い る.環 境 と相 互 作用 す るた め の モ デ ル と して 、 内界 セ ン サ と外 界 セン サ情 報 を統 合 した動 的 ニ ュー ロ歩 行 を実現 す る予 定 であ る.

Acknowledgement

I would like to give my humble thanks to Allah, because of His blessing and benevolence I could live until now and reach .

I would like also to present this thesis to my wife, Ashri Ratnasari and our parents. Their struggle to raise and support me, have taken me until this point. And also my little brother, many thanks for his supports and spirit during my life in Japan.

I would like to extend my special thanks to my advisor, Prof. Naoyuki Kubota for his kind and strong supports, inspiring advises, and wonderful guidance. I could learn and experience a lot of things from him in the past three years and half.

I would like to thank to Dr. Janos Botzheim, his advices and supports gave me the oppor-tunity to learn about metodology and paper writing. For Mr. Indra Adji Sulistijono, I would

like to thank for his advices and recomendations for studying in Japan

I would also like to thank to Mr. Achmad Subhan Khalilullah, Mr. Ali Husein Alasiry, and All of my supervisors in Politeknik Elektronika Negeri Surabaya for his advices and supervision during my undergraduate so that I could receive great achievements as consider-ation study in Japan.

Thanks to all the members of the laboratory for their support and advice. Ms. M. Shimada and Ms. T. Suzuki kindly supported my daily life in the laboratory. Dr. Yuichiro Toda, Dr. Noel Tay, Dr. Jinseok Woo, Dr. Takahiro Takeda and Dr. Takenori Obo for their advices and support during conducting researches and projects. Mr. Kaif Jaden Chin, Mr Iwasa for being good friend, partner, with nice advices. Mr Sun Siqi, Mr Shao Shuai, and Ms. Chiaki Kasuya for helping me during lecture, sharing information, report writing, and daily support. Mr. Shuo, Ms. Noriko, and Mr. Quan for their support. Mr. Shimazaki, Mr. Yoshida, Mr. Matsuo, and Mr. Kusaka for their friendly support. Mr. Kaku, Ms. Yuri, and Ms. Ono for helping me in doing projects. Mr. Miki, Mr. Mabuchi, Mr. Miyake, Mr. Zheng Wei, Mr. Tamatsu, Mr. Sato, and Mr. Joji Sato for their daily support in the laboratory. Mr. Kato, Mr. Sasuga, Mr. Tonomura, Mr. Kanei, Mr. Arakawa, Mr. Arai, Mr. Nakamura, Mr. Shimizu, Mr. Tanaka, Mr. seaside, and Mr. Wu Zepei for having good conversation. Lastly, Also many thanks to Mr. Yusuf Bakhtiar for his helps in Japan life, private life, Japanese translation.

Contents

Abstract

Abstract in Japanese (o)

Acknowledgement Contents List of Figures List of Tables 1 Introduction 1.1 Background ... 1.1.1 Social background ... 1.1.2 Technical background . . . . 1.1.2.1 Locomotion behavior 1.1.2.2 Stability behavior . 1.1.2.3 Motion planning . . 1.2 Motivation and objectives ...

1.3 Thesis structure ...

2 Multi-legged Robot System 2.1 Multi-Legged Robot ...

2.1.1 A Brief History of Legged robot 2.1.2 Type of legged robot ...

2.1.2.1 One-legged robot . . 2.1.2.2 Two-legged robot . . 2.1.2.3 Four-legged robot . . xi i V ix xi xix xxix 1 1 I 2 3 5 6 8 9 13 13 13 14 14 14 15

xii CONTENTS 2.2 2.3 2.1.2.4 Six-legged robot ... 2.1.2.5 Eight-legged robot ... 2.1.2.6 Hybrid robot ... Kinematic Model ... 2.2.1 Forward Kinematics ... 2.2.1.1 Denavit-Hartenberg convention 2.2.2 Inverse Kinematics ... Motion capabilities ... 2.3.1 2.3.2 2.3.3 Locomotion Generator ...

2.3.1.1 Trajectory based Locomotion ...

2.3.1.1.1 Basic Orbital Function ...

2.3.1.1.2 Design of Orbital Function Using Dimensional Prenominal ...

2.3.1.2 Limit Cycle Locomotion [61] ... 2.3.1.2.1 Matsuoka Model ... 2.3.1.2.2 Rowat-Selverston Model ... 2.3.1.3 Passive Locomotion [61] ... Stability Model ...

2.3.2.1 Zero Moment Point ... 2.3.2.2 Center of Gravity ... Motion Planning Model [1] ...

2.3.3.1 Concept ... 2.3.3.1.1 Configuration space ... 2.3.3.1.2 Free space ... 2.3.3.1.3 Target space ... 2.3.3.2 Algorithm ... 2.3.3.3 2.3.3.2.1 2.3.3.2.2 2.3.3.2.3 2.3.3.2.4 2.3.3.2.5 Grid-based Search ... Interval-based search ... Reward-based algorithms ... Artificial potential fields ... Sampling-based algorithm ... Completeness and performance

15 16 16 16 16 17 20 22 22 22 24 24 25 27 28 30 31 31 33 35 35 35 36 36 36 37 37 38 38 38 39 3 Computational Intelligence 3.1 Fuzzy Computing ... 3.2 Neuro Computing ... 43 43 47

CONTENTS xiii 3.2.1 3.2.2 3.2.3 3.2.4 3.2.5 3.2.6 3.2.7 3.3 Evolutionary 3.3.1 3.3.2 3.3.3

Artificial neuron model... Network architectures ... Single-layer perceptron ... Multilayer perceptron... Spiking Neurons ... Self Organizing Map ... Growing Neural Gas ... mary Computing ... Simple genetic algorithm ...

Steady State Genetic Algorithm (SSGA)... NSGA-II ... 3.3.3.1 Nondominated sorting ... 3.3.3.2 Diversity preservation ... 47 49 50 50 51 55 56 58 59 60 60 61 61 4 Robot Development

4.1 Humanoid Biped Robot ...

4.1.1 Kubota's Simple Humanoid Robot (K-SHuBot) 4.1.2 EEPIS Humanoid Robot Soccer (EROS) . . . .

4.1.2.1 Mechanical Structure ...

4.1.2.2 Electronic Hardware Structure . 4.2 Quadruped Robot ...

4.2.1 Kubota's Simple Quadruped Robot (K-SQuBot) 4.3 Conventional Model Development ...

4.3.1

4.3.2

Motion Pattern Generation ...

4.3.1.1 Design of System Control ...

4.3.1.2 Design of Motion Pattern Generation ... 4.3.1.3 Experiments ...

Stability Model ... 4.3.2.1 Introduction ...

4.3.2.2 Modeling and Controlling System...

4.3.2.2.1 Modeling System ... 4.3.2.2.2 Combination Control ...

4.3.2.2.3 Motion modeling in proposed biped robot . . . 4.3.2.3 Experimental result ... 63 63 64 66 66 67 69 69 71 71 72 75 79 82 82 84 84 86 87 90

xiv CONTENTS

5 Locomotion Generator Model

5.1

5.2

5.3

5.4

Neural Oscillator based Locomotion 5.1.1 Locomotion 5.1. 5.1. 5.1. 5.1. 5.1. 1.1 1.2 1.3 1.4 1.5 5.1.2 Experimental 5.1.3 Experimental 5.1.3.1 Simulation Experiment ... 5.1synapse 5.1.3.1.2

5.1.3.2 Application in Real Robot ... 5.1.3.2.1

5.1.4 Discussion ... Neural Oscillator based

5.2.1 5.2.2 5.2.3 5.2.4 Locomotion 5.2.1.1 Neural Oscillator ... 5.2.1.2 Locomotion Generator Synapse Optimization ... Experimental Conclusion Bezier Curve Model

5.3.1 5.3.2 5.3.3 5.3.4 5.3.5 Introduction ...

Neuro-based Locomotion System ... Bezier Curve

Experimental

5.3.4.1 Neural Oscillator Optimization ... 5.3.4.2 Bezier Curve Optimization ... Discussion ...

Walking Speed Control 5.4.1 Introduction ... 5.4.2 Locomotion 5.4.2.1 Walking Model ... 5.4.2.1.1 )n Model ... Neural Oscillator ... Inter-connection Model ... Synapse Optimization ...

Single Objective Evolutionary Algorithm . . Multi-objective Evolutionary Algorithm . . ital Result using Single Objective Model ...

Simulation Experiment ...

.3.1.1 Experiment for optimizing weight sy .3.1.2 Experiment for variant locomotion

Application in Real Robot ...

.3.2.1 Experiment in slope terrain ...

Dased Locomotion (Quadruped robot) ... )n Generator ... Neural Oscillator Locomotion Generator... ital Result ...

lel for Efficient Locomotion Model ...

ed Locomotion System ...

rve Optimization for Passive Neural Control ltal Result and Discussion ...

Neural Oscillator Optimization ...

Bezier Curve Optimization ...

nitro' Behavior ... Dn... m System ... Walking Model ... .2.1.1 Locomotion Generator ... 93 ... 93 ... 95 ... 96 96 ... 101 ... 101 ... 103 ... 104 ... 106 ... 106 apse ... 107 ... 110 ... 112 ... 112 ... 112 ... 117 ... 118 ... 119 ... 120 ... 122 ... 124 ... 127 ... 129 ... 129 ... 130 ... 132 ... 135 ... 135 ... 136 ... 139 ... 143 ... 144 ... 145 ... 146 ... 147

CONTENTS xV

5.4.3

5.4.4

5.4.2.1.2 Synaptic Connectivity Optimization ...

5.4.2.1.3 Calculation ... 5.4.2.2 Gait Generator ... Experimental Results ...

5.4.3.1 Walking Patterns Formation Optimization ... 5.4.3.2 Gait Generator Learning ...

5.4.3.3 Walking Speed Control ...

5.4.3.3.1 Stability Test ... Discussion ...

5.5 Omni-directional Locomotion Behavior 5.5.1 5.5.2 5.5.3 5.5.4 5.5.5 5.5.6 Introduction...

Centered Learning Model ... Neuro-based Locomotion Generator ... Behaviors Learning System ... 5.5.4.1 Behavior Building ...

5.5.4.1.1 Sagittal direction behaviors optimization . . . . 5.5.4.1.2 Coronal direction behaviors optimization . . . . 5.5.4.1.3 Turning direction behaviors optimization . . . . Experimental Result and Discussion ...

5.5.5.1 Center Interconnection Structure Optimization ... 5.5.5.2 Branch Interconnection Structure Optimization ...

5.5.5.2.1 Corona' Movement Behavior ...

5.5.5.2.2 Turning Movement Behavior ... 5.5.5.3 Learning of Omni-directional Walking Behavior ... Conclusion and Discussion ...

150 151 152 154 154 158 160 162 164 164 164 165 167 168 170 170 172 172 176 176 177 177 177 177 183 6 Stability Behavior

6.1 Local effect approach interconnection model ... 6.1.1 Synapse Structure ... 6.1.2 Design of RNN Model ... 6.1.3 Experimental Result ... 6.1.4 Conclusion ...

6.2 Dynamic Synaptic Value for Walking Stability ...

6.2.1 Synapse Structure between Sensoric and Motoric Neurons 6.2.2 Structure Model of Recurrent Neural Network (RNN)

6.2.3 Experimental Result ... 185 ... 185 ... 185 ... 188 ... 190 ... 192 ... 193 ... 193 ... 195 ... 197

xvi CONTENTS

6.2.4 Stability analysis and Discussion ...

6.3 Design of Multimodal Recurrent Neural Network (MRN] ...

6.4 6.3.1 6.3.2 6.3.3 6.3.4 6.3.5 6.3.6

Neuro-based Controller for Stability Behavior 6.4.1 6.4.2 6.4.3 6.4.4 6.4.5 Introduction ...

Overview Implemented Robot Design ... RNN model...

MRNN model ... Experimental Result ... Discussing ...

lased Controller for Stability Behavior ... Introduction...

Motion Behavior Controller ... 6.4.2.1 RNN ... 6.4.2.2 MRNN model ... 6.4.2.3 Motion behaviors ... 6.4.2.3.1 Ankle behavior • 6.4.2.3.2 Hip behavior ... 6.4.2.3.3 Step behavior ... Push recovery controller ... Experimental result ... Conclusion ... ... 200 ... 202 ... 202 ... 204 ... 205 ... 207 ... 209 ... 212 ... 213 ... 214 ... 215 ... 216 ... 217 ... 220 ... 220 ... 221 ... 221 ... 222 ... 224 ... 228

7 Neural Based Path Planning 7.1 Introduction ... 7.2 The Proposed Model ... 7.2.1 Neural Forward Transmission Based Model ... 7.2.2 Synaptic Pruning with Backward Transmission Model 7.3 Application design ... 7.3.1 Robot Design ... 7.3.2 Integrated System... 7.4 Experimental Results ... 7.4.1 Neuron Transmission and Synaptic Pruning Experiment 7.4.1.1 Grid map model ... 7.4.1.2 Topological map model ... 7.4.1.3 Performed with unpredictable collisions 7.5 System Integration ... 7.5.1 Simulation case ... 229 ... 229 ... 233 ... 233 ... 239 ... 242 ... 242 ... 244 ... 244 ... 246 ... 246 ... 248 ... 248 ... 248 ... 254

CONTENTS ,

7.5.2 Real case . . . . 7.6 Discussion...

8 Conclusion and Future Works 8.1 Conclusion ... 8.2 Future Works ... References xvii 254 259 263 263 267 271

List

of Figures

1.1 1.2

(a) Rescue robots (b) Social robots (c) Millitary robots (d) Entertainment robots The organization of the thesis ...

2 10

2.1 The four parameters of classic DH convention are shown in red text, which are With those four parameters, we can translate the coordinates from to O XiYZi [3] ...

2.2 Nature based locomotion principles [5] ... 2.3 Diagram of motion control system ... 2.4 Van Der Poi oscillator model ... 2.5 Limit Cycle ...

2.6 x behavior of Fig.2.5 in the x — i plain. (Limit Cycle) ... 2.7 Limit cycle stability ...

2.8 The block diagram of Matsuoka's neural oscillator ...

2.9 An oscillation generated by the neural oscillator. a) xi (t) (thin line) and yx (t)

(thick line), in which xi (t) = y1(t) for yi (t) > 0; b) x(t) (thin line) and (t)

(thick line); c) X(t) (thin line) and V(t) (thick line) ...

2.10 IV curves and nullclines in the cell model. (a) The curve i = F (V, o f ) for

three values of o-

f. (b) The standard slow current IV curve. ES is the reversal

potential. (c) The split slow current IV curve with different inward and

ward conductances. (d) The V-nullcline for a1= 2, 1, 0. For o f= 2, three

values of the injected current produce three positions of the V-nullcline. (e)

The q-nullcline. (f) The q-nullcline when using a split slow current. A fast

current null cline is obtained by reflecting the IV curve of the fast current

in the V-axis and then moving it up or down by the amount of the injected

current. A slow current nullcline is identical with the IV curve of the slow

current. [161] ...

2.11 Example representation of Center of Gravity model ...

19 23 24 26 26 27 27 28 29 41 42 xix

xx LIST OF FIGURES

2.12 _{(a) Example of a valid path (b) example of invalid path (c) example of a road}

map ... 42 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 4.10 4.11 4.12 4.13 4.14 4.15 4.16 4.17 4.18 4.19 4.20 4.21 4.22 4.23 Membership function ... Fuzzy computing ...

Mamdani inference algorithm... Biological neuron ...

McCulloh-Pitts model of neuron ... Activation functions ...

Neural network architecture ... Multilayer perceptron ...

Geometric interpretation of hidden layer ...

(a) Supposed points in a set with equal rank (b) Crowding distance for point i a) Robot design from front side. b) Robot design from side ...

Design of the real robot a) from the front; b) from the left side ... Hardware structure ...

Design of ground sensor ...

EEPIS Robot Soccer 3rd Generation ... Hardware construction ...

Joint structure of the robot ...

Design of the real robot a) from the front; b) from the left side ... Hardware structure ...

Overall Control System Diagram ... Time scheduling model ... Proposed RNN strcuture ...

(a) Changing movement pattern x--direction (b) Inverted pendulum model . Trajectory planning in x-direction ...

Trajectory plan in z-direction ... Trajectory plan in z-direction ...

Trajectory plan in z--direction when get disturbance ... ZMP Trajectory normally movement ...

ZMP Trajectory with disturbance ... Trajectory z-direction in flat surfaces ... Trajectory z-direction in uneven surfaces ... Robot moves on uneven terrain ... Robot moves on artificial grass ...

44 45 46 49 50 51 53 54 55 62 64 65 65 66 67 68 69 70 70 72 73 74 75 76 77 79 79 80 80 81 81 81 82

LIST OF FIGURES xxi 4.24 4.25 4.26 4.27 4.28 4.29 4.30 4.31 4.32 4.33 4.34

Dynamic model of proposed Robot... Stability system of proposed Robot... Combining control diagram... PID Control Diagram of Pose Control ...

PID Control Diagram of Angular Velocity Control... Motion trajectory ...

Speed changing motion ... Fuzzy membership functions

Robot body's tilt oscillation Graphics ... Robot angular velocity oscillation Graphics ... Oscillation graphics when got disruption ...

83 85 86 87 88 88 89 89 90 91 92

5.1 Neuron model for locomotion ... 5.2 Neural oscillator diagram ...

5.3 Inter-connection model of the motoric neurons ... 5.4 Chromosome of an individual...

5.5 Fitness evolution ...

5.6 Length of walking comparison ... 5.7 The robot walking in simulation ...

5.8 a) Pareto front with 32 individuals; b) Pareto front with 64 individuals . . . 5.9 The changing output of neural oscillator of robot while walking, using

ferent gain parameters in certain time sampling...

5.10 Two-dimensional tracking from the center of mass in robot with different gain parameters ...

5.11 The changing output of neural oscillator in hip--z joint while walking, using different direction parameters in certain time sampling ...

5.12 Two-dimensional tracking from the center of mass in robot with different direction parameters ...

5.13 Robot walking with different lengths of step. Direction parameter = 0 a) gain = 0.4; b) gain = 0.6; c) gain = 0.8; d) gain = 1.0; e) gain = 1.2 ...

5.14 Robot walking with different walking directions. Gain parameter = 0 a) rection = -0.15; b) direction = -0.09; c) direction = 0.15; d) direction = 0.09 5.15 Robot while walking on a slope: a) with 10° tilt angle; b) with 14° tilt angle 5.16 The output of joint angle with different slope terrains ...

5.17 Pattern of supporting limbs and numerical formula [10] ... 5.18 Mechanical structure of the robot ...

95 97 100 102 105 105 107 109 111 111 111 113 114 115 116 116 118 119

xxii LIST OF FIGURES 5.19 5.20 5.21 5.22 5.23 5.24 5.25 5.26 5.27 5.28 5.29 5.30 5.31 5.32 5.33 5.34 5.35

(a) General coupled neuron (b) Coupled neuron with sensor connection . . Diagram of Neuron Structure ...

Sensor connection ... Signal Angle of Joint ... Simulation Result ...

Pareto front a) 12 individuals b) 32 individuals c) 64 individuals ... Fitness evolution ...

Signal Angle of Joint ...

Locomotion model in each leg composed of one neuron. One motor neuron generates 3 joints based on the Bezier optimization model. Sensory neurons are providing the feedback signal for dynamic locomotion. ...

The previous structure of neural oscillator that has 24 motor neurons con-nected to each other is decreased into 4 motor neurons ...

The proposed Bezier curve model. There are 3 quadratic Bezier curves, where 4 points are required in each curve, P0 — P3, P4 - P7, P8 - P11 rep-resent 1st curve, 2nd curve, 3rd curve, respectively. P3, P7, P11 are equal to P4, P8, P0, respectively. P0and P11 are defined as thefirst point of the joint trajectory. Therefore, there are 8 genes to be optimized which consist of 2-D parameters. ...

Neural oscillator signal as the input of the passive neural system ... Fitness evolution ...

A sample of Bezier curve evolution in certain generations ...

Reference and Bezier output of relationship between neuron signal and joint trajectories (hip-y, hip-x, and knee) (a) first leg generated by Neuron 1 (b) second leg generated by Neuron 2 (c) third leg generated by Neuron 4 (d) fourth leg generated by Neuron 3 ...

Comparison between joint reference and Bezier output of angle joint ex-tracted during robot walking with neural signal as the input generator in time cycle based graphic (a) first leg generated by Neuron 1 (b) second leg gener-ated by Neuron 2 (c) third leg genergener-ated by Neuron 4(d) fourth leg genergener-ated by Neuron 3 ...

Four legged robot walking experiment (a) rough terrain (b) slope terrain with 16° slope ... 120 121 122 124 125 126 127 128 131 132 134 136 137 137 140 141 142

LIST OF FIGURES xxiii 5.36 5.37 5.38 5.39 5.40 5.41 5.42 5.43 5.44 5.45 5.46 5.47 5.48 5.49 5.50 5.51 5.52 5.53 5.54 5.55 5.56 5.57

Body's tilt recorded during robot walking in rough terrain and flat terrain shows the stability can be reached by using the proposed model. In rough terrain experiment, the body tilt angle in pitch direction has big oscillation because the robot walking is uphill and downhill on the rough terrain. . . . The effect of ground sensor. The data are taken from first leg during robot walking. The graphic shows the comparison between the joint angle without influenced by ground sensor and the joint angle influenced by ground sensor. Diagram of the locomotion system ...

Neuro-locomotion structure ...

Diagram of MLP with back-propagation model ...

Comparison diagram of fitness evolution between BMA and SSGA a) Fit-ness evolution of BMA b) FitFit-ness evolution of SSGA c) FitFit-ness average of BMA and SSGA ...

142 143 146 149 153 156

Neuron connectivity structures a) 7th walking pattern; b) 14th walking pattern 158 Tracking of walking movements a) 7th walking pattern; b) 14th walking pattern 159 Error evolution in different MLP structures ...

Proposed locomotion implemented for humanoid robot in simulation level . The output of neuron response to the changing walking speed...

The joint angle signal of the robot with the change in walking speed . . . . A change of robot's torso walking speed ...

Proposed locomotion implemented in a simple humanoid robot ... Oscillation of angular velocity recorded from robot's body ... Poincare diagram represents the stability of the robot ...

Online model of centered interconnection structure of omni-directional neuro based locomotion model ...

Whole structure of centered based interconnection model of neural based locomotion ...

(a) Robot design from front side (b) Robot design from side. ...

Sample of pareto front in different generation (a) 3 m/s of desire walking speed (b) 5 m/s of desire walking speed (c) 7 m/s of desire walking speed (d) 9 m/s of desire walking speed. ...

Sample of fitness evolution of population in every generation (a) with 5 m/s of desire walking speed (b) with 7 m/s of desire walking speed ...

Generated sagittal movement behavior in different speed (a) normal walking behavior in low speed (b) fast walking behavior in medium speed (c) jogging behavior in medium speed (d) running behavior in high speed movement . .

159 160 161 161 162 162 163 163 166 167 171 173 174 175

xxiv LIST OF FIGURES 5.58 5.59 5.60 5.61 5.62 5.63

Generated coronal movement behavior in different speed (a) slow coronal walking behavior in right direction (b) left direction (c) fast coronal walking in right direction (d) left direction ...

Generated turning movement in different behavior (a) turning right on a place (b) turning left on a place (c) walking with turning right (d) walking with turning left ...

Proposed MLP structure ...

Generated dynamic signal movement in different behavior (a) signal output of the motor neurons (b) the signal output converted in joint angle level . . Average error evolution with 4 samples of MLP's cross validation errors . . Interconnection structure generated from different movement provision (a) structure of coronal walking behavior (b) structure of turning walking be-havior (c) interconnection structure during combined coronal and turning provision ... 178 179 180 181 182 184

6.1 Sensor connection structure in joint angle level ...

6.2 Recurrent neural network structure ...

6.3 Comparison angular velocity data between locomotion system without SLS

and with SLS (a) Roll direction (b) Pitch direction ...

6.4 Comparison angular velocity data between locomotion system without SLS

and with SLS (a) Roll direction (b) Pitch direction ...

6.5 Signal output of coupled neuron ...

6.6 Running process in ODE simulation ...

6.7 a) Sensor connection in hand area; b) Sensor connection in soles of feet; c)

Sensor connection in foot ...

6.8 Process of weight transfer ...

6.9 The changes in joint angle signal, angular velocity, and body tilt angle from

using stability system to without using stability system ...

6.10 ZMP trajectory during straight walking without stability learning system.

Gain parameter (ugai.,,,

) = 1.0 and time cycle = 0.01 ...

6.11 ZMP trajectory during straight walking by using stability learning system

with gain parameter (,ugain)

= 1.2 and time cycle = 0.01 ...

6.12 6.13 6.14 186 189 190 191 191 192 194 196 198 199 199 Phase diagram of robot tilt angle and stability analysis, based on Poincare map201 Cobweb diagram representation of Figure 6.12 ...201

LIST OF FIGURES xxv 6.15 6.16 6.17 6.18 6.19 6.20 6.21 6.22 6.23 6.24 6.25 6.26 6.27 6.28 6.29

Humanoid biped robot design representation of proposed robot design. The body speed is controlled by controlling the walking step behavior in further model ...

Illustration model of MRNN ... Illustration of MRNN process ...

The illustration RNN bather a) desire angle in the RNN bather b) desire angle inside the RNN range ...

The result of the learning process with different number of RNNs (desire point is inside RNN range) ...

The result of the learning process with different number of RNNs (desire point is in RNN barrier) ...

The result of the learning process with given some disturbances...

Sample of simulations represent the experimental result in Fig. 7.10 with 11 RNNs (a) when robot was falling down (100 < time sampling < 150) (b)

205 207 209 210 211 211 212

when robot was trying stabilize the disturbance (300 < time sampling < 400)213

Design of MRNN model, RNNs have one sharing context layer ... 218

Model of push recovery controller ...222

The prediction of graphics relationship between force input and energy required223

Training process of MRNN...

Robot performed the stability respond (a) ankle motion behavior (b) hip mo-tion behavior (c) step momo-tion behavior ...

(a) Initial behavior graph with initial reference data (b) the changing of be-havior graph with 20 data reference (c) 25 data reference (d) 30 data refer-ence (e) 50 data referrefer-ence (f) 70 data referrefer-ence (g) optimized graph behavior is formed in a hundred data reference...

Push recovery testing under external perturbation when walking at speed of 50 cm/second ... 224 225 227 228 7.1 7.2 7.3 7.4 7.5 7.6 7.7

Design of the proposed model ...

Neuro-transmission model ...

Environmental influence represented by surrounding neurons, where each

neuron has 3-D coordinate information (a).from top view (b) side view . . . Illustration of the components that influence the synaptic pruning...

Simple spiking neuron model for backward transmission ...

232 234

237 240 241 Grand design of the integrated system (a) connection model (b) system model 243 Joint structure of the robot ...243

xxvi LIST OF FIGURES

7.8 Design of the real robot a) from the front; b) from the left side ...

7.9 Experimental terrain (a) perspective view (b) top view (c) Grid map model of experimental terrain ...

7.10 Result of neuron transmission model in grid map model from top view

(a) first experiment (81 = { 0.5, 11} and T1 = { 11.0,1.0

}) without

BT model (b) with SP model (c) second experiment (S2 = {0.2, 7} and

T2 = {6.5, 11.5}) without SP-BT model (d) with SP-BT model (e) third

experiment

(S3 = {11.2, 9} and T3 = {7.0, 1.0}) without SP-BT model (f)

with SP-BT model ...

7.11 Topological map model of experimental terrain (Fig. 7.9a) (a) top view (b) perspective view ...

7.12 Result of neuron transmission model in topological map model from top view (a) first experiment (b) first experiment with SP-BT model (c) second

experiment (d) third experiment ...

7.13 Firing signal of neurons in FT and SP-BT processes, where the red circle represented pruned neuron and blue circle represented the neurons which

constructed the pathway trajectory ...

7.14 Result of neuron transmission model in grid map model (a) initial pathway (b) changing pathway when the robot found the first obstacle (c) second

stacle (d) third obstacle. The black point represents the obstacle position and the gray point represents the influence of the existence of the obstacle . . . 7.15 Design of system integration ...

7.16 Robot is performing the movement on rough terrain (first experiment) from perspective view (a) without unpredictable obstacle (b) with unpredictable obstacle ...

7.17 Result of the robot movement based on the generated pathways (a) pathway generated in the first experiment (b) second experiment (c) first experiment with unpredictable obstacle (gray color as the obstacle and the black one

is a shadow of the obstacle) (d) without performing the generated pathway (fallen down) ...

7.18 Artificial map represents real cases (a) real terrain (b) feature extraction by using Kinect camera (c) reconstruction into topological map (d) topological map from top view ...

7.19 Result of path planning model in topological map model (real terrain) (a) initial pathway (b) changing pathway when the robot found first obstacle (c) final pathway ... 244 245 247 249 250 251 252 253 255 256 257 258

LIST OF FIGURES xxvii 7.20

7.21

8.1

4-legged robot is performing the movement on real rough terrain (a) without unpredictable obstacle (b) with unpredictable obstacle ...

Result of the 4-legged robot movement based on the generated pathways on real rough terrain (a) without unpredictable obstacle (b) with unpredictable

obstacle ...

Diagram of Internal and external sensory information ...

259

260

List

of Tables

3.1 Modern parallel computer versus biological neural system ...

4.1 Denavit-Haternberg parameters ... 4.2 Walking Condition ...

5.1 Clusters of Weights in the Optimization ... 5.2 Robot specification ...

5.3 Parameters of SSGA ...

5.4 Real Robot Properties Corresponding to the Robot Simulation ... 5.5 NSGA-II Parameters ...

5.6 NS GA-II Parameters '... 5.7 Result of Weight Parameters ... 5.8 Parameter Values of Optimization ... 5.9 Representation Parameters and Results ...

5.10 Optimized Parameters in Bezier based Optimization ... 5.11 Interconnection Maps Represented by Genes... 5.12 BMA Parameters ...

5.13 Walking Pattern Parameters ...

5.14 Interconnection Maps Represented by Genes ... 5.15 NS GA-II Parameters ...

6.1 Connection structure ...

6.2 Configuration of Input and Output Neuron ... 6.3 Sensoric and Motoric Connection ...

6.4 Clustering Sensoric Weights ...

6.5 Comparison With Stability System and Without Stability System ... 6.6 Constant parameter of maximum and minimum value ...

7.1 Parameter Values ... 48 63 92 99 106 106 107 108 124 125 135 136 138 151 155 157 169 176 187 190 195 197 200 220 246 xxix

xxxLIST OF TABLES

Chapter

1

Introduction

1.1

Background

1.1.1 Social background

Nowadays, robots are no longer utilized as amere technological supplementary tools for labor-intensive or hazardous tasks such as, factory automation, military operation, and even space or seabed exploration). Now, robot has become a part of our daily lives. Robot technol-ogy has been implemented in many fields of our life, such as entertainment, security, rescue, rehabilitation, social life, industry, and the military. Most researchers build robots for partic-ular purposes. Some researchers use tank model robots for disaster problems and navigation in dangerous areas [228], while other researchers build a robot partner to support elderly people [233]. Furthermore, some researchers use humanoid robots for dangerous areas and rescuing humans [207]. Honda produced the humanoid robot "AS MO" that can serve people in their social life. DARPA developed a humanoid robot for military service. Nevertheless, the humanoid biped robot is a suitable robot in many fields: it can be applied for social life [167], rescue [228], [207], military purposes, or entertainment (Soccer Robot, Dancing Robot) [176], [173]. Therefore, it is important to improve the development of legged robots. Although the cognitive capabilities of humanoid robots are important, but their motion ca-pabilities are also important to support its activity. Therefore, in this thesis, we developed motion capabilities of the robot. In our motion capabilities model, there are three subsys-tems should be developed, which are, (1) locomotion behavior model, (2) stability behavior

model, and (3) motion planning model.

2 CHAPTER 1. INTRODUCTION

(a)

of

(b)

(c)(d)

Figure 1.1: (a) Rescue robots (b) Social robots (c) Millitar-y robots (d) Entertainment robots

1.1.2 Technical background

In motion capabilities model, most of the researcher using conventional model to de-velop trajectory generation that used zero moment point and the inverted pendulum ap-proach [176], [173], [234]. Yoshida et al. implemented the double inverted pendulum for the balancing system in their humanoid robot [234]. Most of them develop the certain trajectory generator for certain motion behavior [96, 28, 132, 193, 32, 121]. Kim et al. also used a con-ventional approach for development of the HUBO robot: they applied a mathematical method to develop the trajectory generator in the robot [96]. They developed difference mathemati-cal model for developing 3 difference walking behavior (Sagital, Corona', and Turning). Fur-thermore, Matsuzawa developed independent crawling motion behavior for moving on rough terrain [132]. Brandao et al proposed footstep planning for slippery and slanted terrain [28]. In WAREC-1 robot, trajectory based locomotion generator also implemented, they devel-oped different motion behavior such as: bipedal/quadrupedal walking, crawling, and ladder climbing [70]. In order to the footstep movement, A* algorithm is implemented. However, those conventional model have some limitation and inefficient in dynamic locomotion plan-ning. They have to model certain locomotion planning in order to generate certain motion

1.1. BACKGROUND 3

behavior. These motion strategy will receive no problems when implements small number of behavior and implemented in non-global area. When we implement in global area, number of motion behavior will be increased. These implies requirement to model many number of behavior and requires a high number of memory and computational cost. These kind of prob-lems is still becoming a big issues and makes saturation development in conventional model. Therefore, we tried to implemented neural based model as alternate way instead of conven-tional model. This model uses natural process and has higher expectation in the future. In order to simplify the technical background of neural based motion capabilities model, we

separated into three part, locomotion behavior, stability model, and motion planning model.

1.1.2.1 Locomotion behavior

From explanation above, the conventional model does not represent well natural behavior during locomotion. Since the trajectory based locomotion model generate in cartesian level, it requires inverse kinematic model for converting into joint angle level. It also needs high mathematical complexity to realize the dynamic walking pattern. In other hand, bio-inspired model may be the alternate approach for considering the dynamical system, non-linear sys-tem, and natural process. Bio-inspired locomotion provides bottom-up process, where the joint angles are firstly generated before trajectory in Cartesian level. In neural oscillator model provide simple solution which can be applied in complex problem, because its non-linearity. Therefore, this model can decrease the computational cost. Since 2000, bio-inspired locomotion developments are exponentially increased. Although the performance results are not better than conventional approach, bio-inspired model have higher expectation perfor-mance than conventional approach because it proved non linearity and dynamism.

The controller system in the brain has been proposed by Roy [164]; this is the basis of how the brain controls locomotion. Locomotion models based on a biological approach have been proposed by several researchers [86, 84, 196, 136, 142]. Before we applied locomotion in a multi-legged robot based on the neuro-biological approach, we studied the locomotion system as adjusted by animal morphologies [51]. Four-legged animal locomotion has been proposed by Ijspeert et al. They control animal locomotion by using a neural oscillator and also design the transition mechanism from swimming to walking [86], [84], [85]. Locomo-tion based on the central pattern generaLocomo-tion (CPG) approach in four-legged animal robots has been applied by several researchers [51]. Furthermore, CPG can also be applied for mal-function compensation in six-legged robots [158].

A neural oscillator is also implemented for legged robot locomotion, as proposed by sev-eral researchers [196, 136, 142, 88, 14]. Taga et al. used coupled neurons for generating the

4 CHAPTER 1. INTRODUCTION oscillated signal to drive the joint. They dealt with a sensory feedback system to adapt to the environmental conditions and created a mathematical model in order to acquire the feed-back calculation. Their proposed method is applied in computer simulation [196]. Another neuro-model of locomotion is presented by Matos et al., who proposed a CPG approach based on phase oscillators for bipedal locomotion [127]. However, the ability to recover the disturbance is required. Ishiguro et al. also proposed the concept of a neural oscillator to realize two-legged robot locomotion. This model is applied to control a three-dimensional biped robot that is intrinsically unstable. They applied a feedback sensor to form dynamic locomotion; however, the robot has a limited degree of freedom and is applied at simulation level only [88]. In 2014, Nassour et al. proposed the locomotion model in the humanoid biped robot: they extended the mathematical model of CPG and designed a multi-layered neuron connection in order to control various models of walking [142], [141]. Nassour's re-search has good stability; however, the aim of our rere-search is to improve the stability level of walking. In 2010, Park et al. designed a locomotion using an evolutionary optimized CPG. They also proposed sensory feedback to support the walking model; however, they did not consider a learning system for stability [148]. Other researchers have considered center of mass (COM) for their locomotion, based on central pattern generation [79], [149]. They have not considered however the stability of the learning system, or control to get various walking patterns.

In this current issue of bio-inspired model, stability, walking provision (omni-directional

walking), and learning process are obstacle in this locomotion development.

Omni-directional walking are solved in this proposed research. Omni Omni-directional locomotion model provides dynamic movement and ability to modify its motion quickly so that support the robot to move in dynamic environment [12].

Most of the limit cycle development only consider- speed in unidirectional walking [77, 52, 68]. Endo et al developed adjusted walking velocity for neural oscillator based lo-comotion. However, range of adjusted velocity is small [52]. In 2008, Manoonpong et al developed neural based locomotion in order to generate omnidirectional movement in any types of legged robot. The proposed model can easily be adapted to control other kinds of walking machine without changing the internal network structure and its parameters. This model also can produce at least 11 different walking patterns and a self-protective reflex by using five input neurons [124]. Therefore in this thesis, we proposed dynamic structure model in neural oscillator based locomotion and learning model in order to generate un-scaled omnidirectional movement in biped robot for solving current issues in neural based locomotion model.

1.1. BACKGROUND 5

1.1.2.2 Stability behavior

Stability is the important part in bipedal humanoid robot. Robots have to walk stably in any condition and have to be ready getting external disturbance or internal disturbance. Following the humanoid robot development, the size of its footprint is getting smaller. It causes the walking of humanoid robot mainly not stable and its stability is difficult to be obtained. In the current state of the stability development, open loop based control is the most famous stability model applied in humanoid robot, especially in small humanoid robot. It results a walking model with good stability. However, pre-learning processes on well-defined surface are required. By using this technique, robot will be unstable when it finds undefined surface and undefined disturbance. Therefore, online stability learning model may be a great model in order to acquire a good stability [230].

In online based stability model, most researchers implemented physical method for the stabilization. They implemented inertial sensor or physical sensor and calculated the torques required in each joint for responding to any external disturbances. [82, 81] We also im-plemented physical approach in previous research. We applied inverted pendulum model

and zero moment point in humanoid robot locomotion [176], [169]. On the other hand, re-searchers use biomechanical approach, applied the human behavior in order to recover and reach the stability level. Human shows the three different motion behaviors for

respond-ing sudden external disturbances, which are ankle recovery, hip recovery, and step recovery strategy [9, 186]. This algorithm is claimed that it has lower computational cost than phys-ical based model because of its simplicity. Yi et al applied push recovery controller inte-grated with three motion behaviors. However, these algorithms required a lot of training data [230, 231]. They also did not consider the energy required in stability activity that considered in this proposed research. Pre-defining the strategy classification is required to be performed before learning process, it is seemed as unnatural process in human behavior. [230] also has not been proved yet to be applied in humanoid robot that has human-like footprint. Our proposed model is therefore applied in human-like footprint or small footprint.

Furthermore, bio-inspired control system may be able to become a new innovation in push recovery controller. This algorithm shows the natural process of human model. Its ef-fectiveness has been proved by several researchers [236], [62], [95]. Zhang et al shows the effectiveness of Recurrent neural network (RNN) as predictive control [236]. RNN was ap-plied to control the stability of biped robot locomotion [115]. Neural network is also imple-mented in order to process sensory feedback information in central pattern generation (CPG) based locomotion [62]. Fukuda et al. combine RNN with evolutionary algorithm to achieve the stable locomotion in humanoid robot [59].

6 CHAPTER 1. INTRODUCTION

Depending on the current issues, in this proposed model, we applied bio-inspired stability controller for humanoid robot locomotion. This proposed stability model used multimodal learning system which can assume different condition. It assumes that robot performs dif-ferent behavior in different condition. In the humanoid robot case, if the robot gets a small disturbance such as the push or uneven terrain, then the robot only gives hands response for protecting its stable condition. Three motion behaviors in biomechanical approach is also

considered in this proposed model.

1.1.2.3 Motion planning

_{During a disaster, the terrain, route, area, and also building are becoming unstructured ,}

diverse, and challenging. It makes the rescuing process difficult and challenging for human

or robot who is in charge in disaster area. The route in disaster area becomes unstructured, making it difficult for the rescuer whether human or robot to find the best and possible route

in the rough and unstructured terrain. Since it is very dangerous for human to rescue in

unstable environment, robot is the best choice for exploring and investigating the disaster area. Most researchers proposed wheeled mobile robot for rescuing in disaster areas [139 , 159, 182], but legged robot is more effective in rough terrain [11]. Therefore, we applied this proposed algorithm in a four-legged robot which is equipped with several supporting

sensors. Four-legged robots are more efficient than biped robots in the stabilization cases

and are able to achieve the maximum movement speed compared to robots with more legs.

Furthermore, 3-D path planning model is required in order to support and facilitate those

robots while moving on rough terrain. Therefore, in this research, we propose an online 3-D path planning optimization based on neural activity.

Three dimensional path planning studies have used images for path planning and also applied multiclass support vector machines for obstacle avoidance [137]. By using this idea , the robot can generate the safe pathway. In [179], 3-D based path planning was proposed , and D* algorithm was modified in order to estimate the distance in sloping terrain. Be-side that, Dogru et al proposed genetic algorithm for optimizing pathway with minimum

energy required [43, 44]. However, [179, 43, 44] did not consider the unpredictable

obsta-cle. Beside that, Kroumov et al solved the obstacle problem but there is still problem with concave 3-D obstacles [103]. 3-D path planning was also proposed for UAV movement algo-rithm [145, 166, 2111 These algoalgo-rithms were applied in computer simulation. In the integra-tion system, UAV or drone is used for generating the map. Some researchers used 3-D map

reconstruction in order to acquire the 3-D map model and to find the best pathway based on

1.1. BACKGROUND 7

Kinect sensor in order to support the reconstruction algorithm. [26] also used LRF to gener-ate the 2-D maps. This idea deals with static environment. Henry et al used Kinect camera in order to model 3-D indoor environment [74]. In the previous research, multi-resolution map was also used for decreasing the computational cost existing in high resolution map of path planning [201]. After that, a real-time feature extraction and segmentation method for a 3-D map [202] was proposed in order to increase the efficiency of the topological map. This map model can decrease the memory usage for reconstructing the map. Therefore, [202] is

suitable to be combined for the proposed 3-D path planning model in the next stage.

In further cases of the environment model generated by LRF and Kinect sensor, the un-predictable surface such as friction, unstable terrain, and unun-predictable collision sometimes become the problem. Those sensors are used as initial data in path planning, therefore those ones only deal with static path planning. In the real cases, the robot deals with dynamic environment, and it should autonomously work through dynamic environment. When the algorithm deals only with static environment, the robot will get problem when it finds an unpredictable collision. Therefore, the dynamic path planning is important in order to deal with dynamic environment. Dynamic path planning will regenerate the path planning when the algorithm finds unpredictable condition or changing environmental condition. In order to

support the dynamic path planning, either additional sensors in mobile robot or measurement tools for human are required to measure and detect unpredictable collisions (friction, obsta-cle, and unstable of terrain) which cannot be considered in initial map generated by LRF or

Kinect sensor.

Most mobile robots (wheeled and legged) are supplemented with the capability for find-ing the pathway. Some people used traditional algorithm such as: A*, D*, and Dijkstra algo-rithm in order to find the best pathway [114], [53], [143], [42], [113]. Ferguson et al proposed a modified D* algorithm in order to reduce the path cost in non-uniform environment. An interpolation equation was proposed in order to cut the inefficient pathway [53]. However, in [53]'s algorithm more computational cost was required. In common cases, D* path planning algorithm is faster than A*, but [75] modified the adaptive A*, therefore it could be faster than D* in some cases. These algorithms [75, 53] required predefined travel cost, therefore these algorithms seem difficult to cover unpredictable obstacles.

However, the traditional algorithm of path planning requires the determination of the path planning rule and it uses a recursive algorithm. Therefore, these algorithms require high computational cost. Bio-inspired algorithm provides a natural process that is able to dynam-ically generate the best pathway [160, 154]. Some researchers have proved the effectiveness of neural based model for path planning problems [64], [224], [227], [154]. In [64], Hopfield method was applied in a neural network to generate the pathway with obstacle avoidance.