Decision Making Method Using Fuzzy Logic for Autonomous Agent Navigation

(1)

electronic Journal of Computer Science and Information Technology (eJCSIT), Vol. 3, No. 1, 2011

J. Jaafar and E. McKenzie, Decision Making Method using Fuzzy Logic for Autonomous Agent Navigation 8

Abstract— In this paper, we present a solution for autonomous agent navigation in unknown virtual environments. A new decision making method using fuzzy logic is proposed. The objective is to solve behaviour conflicts in behaviour-based architectures. Two main problems have been identified: how to decide which behaviour should be activated at each instant; and how to combine the results from different behaviours into one action. The method uses fuzzy α-levels to compute behaviour weights for each behaviour and the final action is selected using the Hurwicz criterion. The experimental results show that the autonomous virtual agent had successfully navigated various virtual environments. The results show the virtual agent required less decision making which means more reliable decisions had been generated and redundant decisions had been reduced. The qualities of the path produced are reasonably smooth, short and clear of collision.

Keywords – Decision Making; Fuzzy Logic; Navigation; Agent

I. INTRODUCTION

Autonomous virtual agent navigation in a virtual environment can be described as the ability of a virtual agent to move purposefully without user intervention. The navigation task may be decomposed into three sub-tasks: mapping and modelling the environment, path planning and selection, path following, and collision avoidance [1]. Virtual agent navigation can occur in known and unknown environments. For a known environment, the virtual agent will have knowledge about the environment and can generate the navigation path. The methods used are based on optimization and computational intelligence. In contrast, an unknown environment in which the virtual agent does not have any knowledge about the environment, the navigation path is generated according to user specifications; and the virtual agent cannot be prepared ahead of time [2].

The control of a virtual agent is shared between multiple behaviours with different and possibly incompatible goals. Each behaviour is responsible for controlling the virtual agent to achieve or maintain a particular objective. The goal of one behaviour might be in conflict with the goals of the others. Therefore, the main problem is to decide what action to select next. Action selection is the means by which a virtual agent (either an animal or an autonomous artificial system) determines at any instant what to do next. The questions are what and how is it being selected. Thus, an important consideration is the formulation of effective mechanisms for coordination of the behaviour activities into strategies for rational and coherent behaviour.

Although there are successful implementations of fuzzy logic in robot navigation, the technique has not commonly been used in virtual environments. In order to maximize the fuzzy logic concept, a cell decomposition strategy was used [3]. The fuzzy logic improves the description of the virtual agent’s environment. Fuzzy logic does not modify a virtual agent’s environment but instead describes the environment in greater detail. This allows virtual agents to navigate better among obstacles. An intelligent virtual agents system has also been developed [4]. The systems are incorporated into the decision-making process of the Virtual Robot Servant to allow it to understand and evaluate the fuzzy value requirements and enhance its performance. In agent-based game design, fuzzy logic has provided a natural way of modelling the game’s creatures with high flexibility and low coupling, allowing verity of behaviour. This has improved the quality of the interaction [5]. Other work such as [6], developed the fuzzy reinforcement function using the fuzzy goal and the fuzzy state with extended fuzzyQ-learning. The limitation of this method is the requirement of exacting analysis to generate the proper fuzzy sets to the vague goal and the uncertain environments.

The aim of this research is to incorporate a virtual agent with behaviour-based control using only fuzzy logic (such as fuzzy rules and fuzzy reasoning) for coordinating conflicts and competition among different operations of reactive behaviour. The basic structure consists of all behaviours, taking input from the sensors and sending output to the actuators. A new behaviour based fuzzy controller is established to optimize the fuzzy behaviour rules using Fuzzy Associative Memory (FAM). FAM maps the complete input space to the outputs. The fuzzy

α−level technique, as a behaviour selection method, has

been developed to decide which behaviour task needs to be executed. The local minima algorithm has also been used to help the virtual agent escape from traps or dead-ends.

II. PROBLEM IN AGENT DECISION MAKING

The problem of decision making is central each time autonomous entities such as robots, virtual characters, or humans are designed. The system should decide what to do next according to its internal and external information without outside interventions. Decision making is a control structure for an autonomous agent and can be considered as the ‘mind’ of the agent. The continuing task of mind is to produce the agent’s next action to answer the only really significant question there is: what shall I do next?

Decision Making Method Using Fuzzy Logic for

Autonomous Agent Navigation

Jafreezal Jaafar

Computer and Information Sciences Department Universiti Teknologi PETRONAS 31750 Tronoh, Perak, MALAYSIA

[email protected]

Eric McKenzie

School of Informatics University of Edinburgh

(2)

In the decision process, multiple conflicting objectives are considered simultaneously, subject to certain constraints dictated by the agent’s limitations [7]. The constraints are based on the complexity of the environment, unpredictability, and an agent’s limited resources. This implies that the decision making cannot be completely optimal. The decision making should be fast, robust and good enough for satisfying a decision [8]. The decisionmaking process searches for ‘good enough’ options, rather than an optimum solution. Satisfying decision making becomes something which is carried out in a limited time, though with some limits on the individuals concerned [9].

Some of these requirements might conflict with each other; for example, planning is in conflict with reactivity [10]. This is not a major problem since it depends where and when the decision is to be applied. Also, there are some agent architectures and decision making methods that use both planning and reactive approaches, such as in hybrid systems. The main concern is how this method can fulfil the autonomous agent goal of ‘good enough’ action.

At every instant the agent should choose the actions which can achieve its objective, given its internal state (e.g. food and water needs), its perception of its environment, and its repertoire of possible actions. Moreover, the temporal pattern of its behaviour should make sense as well. If it is working on a given goal, it should continue working on that goal until either the goal is satisfied or something better comes along. That is, it should be able to balance persistence with opportunism and have a sense of whether it is making progress, i.e., it should not get stuck in mindless loops [11]. Many problems are linked with decision making, such as action persistence, evaluation of the action choice, chaining actions to obtain coherent behaviours, authorizing an opportunist, and in compromising behaviours [12].

Fuzzy logic is inspired by the approximate type of reasoning that allows humans to make decisions under uncertain and incomplete information. In the context of the aforementioned trade-offs imposed on virtual agent learning, fuzzy techniques offer a means to sacrifice optimal performance for a reduction in complexity, elimination of unnecessary details and increased robustness of solutions.

In the past, several works relating to virtual agents have been done which describe mathematical models [13] and fuzzy logic systems [14] for behaviour selection. However, the limitations are the insufficient knowledge based on perception of the environment and the absence of a decision making capabilities similar to that of a human driver.

III. BACKGROUND ON AGENT DECISION MAKING Many decision making methods have been proposed, yet there is still no clear classification of the different techniques in the literature. A global classification, accepted by the majority of scientists [15], lists systems according to their adaptability and might depend on when and where it is being used, and also on how the decision making has been accomplished.

One of the earlier classifications was introduced by [10, 16, 17]. The classifications have some similarity, in that arbitration (competitive behaviour) and command fusion (cooperative behaviour) corresponded to state-based and continuous approaches, respectively. Both also focus on the problem of behaviour coordination and command fusion.

Behaviour coordination is concerned with how to decide which behaviour to activate at each moment or state, and command fusion is concerned with how to combine the results from different behaviours into one command [7]. Alternatively, classified decision making into a symbol-based system (classical planning), distributed solution and reactive planning (dynamic planning) has also been attempted [18]. Even though there is some similarity with others [10, 16, 17], the decision making has been classified based on agent architecture.

Arbitration and command fusion decision making mechanisms are mutually exclusive in that the same set of behaviours cannot use both mechanisms at the same time. However, it is still possible to use them together in the same architecture as long as there is a way to decide which selection mechanism gets to select behaviours at any given time; for example, in hybrid architecture [19] that furthermore expanded arbitration and command fusion into implicit behaviour and explicit behaviour.

Implicit behaviour selection uses structural features of the architecture to select behaviours. This can be seen, for example, in the relative strengths of inhibitory and excitatory connections among components as in the cooperative example of Braitenberg vehicles [20]. In contrast, explicit behaviour selection uses specialized components. Implicit and explicit behaviour selection mechanisms are also mutually exclusive analogous to competitive and cooperative mechanisms. Similar to arbitration and command fusion, implicit and explicit behaviour can also coexist in a single architecture.

This is similar to voting, but uses fuzzy inferencing methods to formalize the voting approach. The result of inferencing is represented in a fuzzy [21] variable and a ‘defuzzified’ to get a crisp value that can be directly used to control the agent. Two main advantages of using fuzzy logic compared to other methods are that it deals with various situations without needing an analytical model of the environment; and it is easier to merge different strategies by means of the fuzzy rules, depending on different situations [22].

(3)

J. Jaafar and E. McKenzie, Decision Making Method using Fuzzy Logic for Autonomous Agent Navigation 10 Figure 1. The navigation system

We have distinguished between arbitration (competitive behaviour [16]) and command fusion (cooperative behaviour [16]) decision making methods. Although both methods can be used in reactive and deliberative architectures, the distinction between reactive and deliberative behaviour selection is pertinent to the run-time instance of an architecture. They can also be used together, for example in a hybrid architecture. Hybrid architectures may consist of a command fusion decision making mechanism in the reactive layer, and an arbitration action-selection mechanism in the deliberative layer.

IV. THE METHOD A. The Virtual Environment

A simple virtual agent and virtual environment was developed. The virtual agent’s task is to navigate in an unknown environment; the only pieces of information known by the virtual agent are the coordinates of the start and target/goal points. The navigation task requires the virtual agent to activate each of the components separately in its own context of applicability.

A. The Navigation System

The navigation system can be divided into three main components, which are the fuzzy navigator, virtual agent,

and the environment (see Figure 1). The main component of the navigation system is the virtual agent itself. The virtual agent should be able to make its own decisions, and does not require any information about the virtual environment. It also does not require any training or learning before the navigation task.

The fuzzy navigator is the main engine for the virtual agent. It comprises three main components:

1. Fuzzy Logic Controller (FLC) - using a behaviour-based architecture which comprises of Path-Planning Behaviour (PP), Goal-Seeking Behaviour (GS) and Obstacle-Avoidance Behaviour (OA).

2. Local Minima Solver (LMS) - responsible for helping the virtual agent escape from dead-ends.

3. Fuzzy Action Selection (Decision Making) Mechanism (Fuzzy-ASM) - to make the final decision in selecting the possible action required by the virtual agent to reach the goal.

The fuzzy navigator receives input from the visual sensor and produces the final action needed to be executed by the virtual agent. Each component in the fuzzy navigator is integrated and works independently.

(4)

B. The Visual Sensor

The information between the environment and the virtual agent are retrieved using a visual sensor. This visual sensor differs from vision systems in robotics, since all information about pattern recognition and noisy images can be ignored [24]. The visual sensor captures the information about the virtual environment or identifies which part of an obstacle can be seen from the position of the virtual agent as in Figure 3. Also, the visual sensor only identifies whether a square (cell) in the vision range is occupied by an obstacle or not. The assumption has been made that all objects are opaque.

Figure 3. Example of vision field and sensor’s region based on location The visual sensor field of the vision range is 180o. The vision field is divided into eight main sectors which are represented as S0 through S7. Hence, there is a probability

that the cells located in the proximity may be occupied. Cells firmly inside the vision field sector are likely to be empty. An occupancy grid is essentially a data structure that indicates the certainty that a specific part of space is occupied by an obstacle. It is a representation of an environment as a two-dimensional array. Each element of the array corresponds to a specific square on the surface of the actual world, and its value shows the certainty that there is some obstacle there.

C. Modelling of the Control System

The architecture of the fuzzy controller is comprised of three behaviours. The behaviours operate at three different ranges, with path-planning (PP) and goal-seeking (GS) behaviours in global path planning, and the obstacle-avoidance (OA) behaviour in local path planning, as in Figure 2.

A Fuzzy Associative Memory (FAM) is used as a process of encoding and mapping the input fuzzy sets to the output fuzzy set [25]. The fuzzy controller is based on our proposed method [26, 27]. Consider a set of fuzzy rules, R={R1,R2KRi,KRk}where is the

rule of the fuzzy controller. The rule is given as follows:

… (1)

The following fuzzy relation will implementRi:

) ( )

( ) ( ) ( ) , , , ,

( 1 1 2 2

2

1 C Z

X A

X A X A Z X X X

R nm

n m n

m m

n

m →

  

 

∧

∧ ∧

=

K K

(2)

Where X1,X2,K,X_nare input variables which are the

sensor data of the virtual agent, m n m m

A A

A₁, ₂,K, are the input fuzzy sets,

C

_nmis the output fuzzy set,

Z

is the output variable,

n

is the dimension of the input vector and

m

is the number of fuzzy sets.

The weighted sum Cfor each individual membership can be defined by using minmax aggregation [28] operators as given below:

[

]

(

1 ( 1) 2 ( 2) ( ) ( )

)

1 '

1

Z C X A X

A X A U

C U C

m n m n m

m k

i m k

i m

⇒

∧ ∧ ∧

= =

∑

= =

K

(3)

The non-negative weight ௜ summaries the strength of the ௧௛

FAM entry and is the number of rules in the system. In order to relate the ௧௛

fuzzy set of the ௧௛ fuzzy rule, the fuzzy implication model using the Mamdani min operator [29] interprets the logical rules for rule firing. We obtain the final defuzzification response for a output membership function ௖ is defined as:

௖ ௠ୀଵ ௑ୀ௎ ௞

௠ୀଵ ௡

஺_೙೘_௡,

ோ_೘ଵ, ଶ, … , ௡,

(4)

Equations (4) and (2) are used to derive the FAM model and the output fuzzy system respectively.

D. Fuzzy Associative Memory (FAM)

The virtual agent is supported by eight visual sensors which provide object detection and range information for recognition of features, as well as navigation around obstacles. Figure 3 shows the visual sensors array with sensing angles. The input and output fuzzy sets and their ranges are shown in Table 1. These values are used to establish the FAM Table and fuzzy logic controller.

Table 1. Linguistic input fuzzy set and their ranges

(5)

J. Jaafar and E. McKenzie, Decision Making Method using Fuzzy Logic for Autonomous Agent Navigation 12 1. It is assumed that the virtual agent can only move in

the forward direction by using the front vision sensor.

2. The rule combinations should have a tendency to select the direction that is closest to the forward direction, so that the virtual agent does not make unnecessary rotations.

3. Rule combinations that yield empty sets should be eliminated.

In the first step of the FAM methodology, the most traversable sector in the right and left regions are found independently with a preference toward the forward direction. In the second step, the best sector among the preferred-right (PR), preferred-left (PL), and front sectors is determined. FAM uses a fuzzy inference system to derive the behaviour rules.

F. Path-Planning Behaviour

The path-planning (PP) behaviour is used to develop simple fuzzy rules for determination of the virtual agent turn angle as shown in Figure 4. This behaviour will monitor sensor information and identify a local minima situation while the regular fuzzy controller is working.

In general, the fuzzy controller for PP behaviour contains two main parts, i.e. turn rules and the weight rule. The local minima algorithm identifies if the virtual agent is trapped in a local minima or not. Here, the fuzzy controller will generate the potential turning angle and the weight rule used for the virtual agent in its behaviour selection.

Figure 4. PP Behaviour with FLC and local minima algorithm 1) Turn Rules and Weight Rules

The α_{value of PP behaviour is represented by three} linguistic fuzzy sets (LOW, MEDIUM, HIGH), and is derived directly from both the obstacle distance dObs and

obstacle direction θ_Obs_{, using the rule set.}

Figure 5. Membership function for turn angle

The virtual agent turn angle is represented by five linguistic fuzzy sets {NL, NS, ZE, PS, PL}, with the membership functions shown in Figure 5, where NL is negative-large, NS negative-small, ZE zero, PS positive-small, and PL positive-large.

Table 2. Weight rules for path-planning behaviour

Negative and positive mean that the virtual agent turns to the left and right, respectively. The weighting factor represents the strength by which the PP behaviour recommendation is taken into account to calculate the final motion command. The weight of PP behaviour is represented by three linguistic fuzzy sets {SMALL, MEDIUM, LARGE}, and is derived directly from both the obstacle distance and obstacle direction, using the rule sets in Table 2.

E. Goal-Seeking Behaviour

The goal-seeking behaviour generates a turning angle based on the location of the goal. A simple analytical model rather than a set of fuzzy logic navigation rules have been used. Two assumptions have been made, i.e. the GS behaviour:

(a) does not influence the speed of the virtual agent, and contributes only to the turning angle;

(b) the turn angle recommended by the GS behaviour is the heading error between the current virtual agent heading and goal direction - thus, the value domain of this turn angle is (-180°, 180°).

Once the goal direction has been identified, the GS behaviour will produce a potential turning angle towards the goal. The weight of GS behaviour, wgs has also been

(6)

Figure 6: Weight determination of OA, PP and GS behaviours. Figure 6 shows the weight determination of three behaviours. The simple weight rules are as follows:

• IF WPP is Large OR WOA is Large, THEN WGS is

Small

• IF WPP is Small AND WOA is Small, THEN WGS is

Large

Importantly, the weight of GS behaviour is suppressed and small when any one weight of the OA and PS behaviours is not SMALL. When the weights of both OA and PS are SMALL, the GS behaviour makes a dominant contribution to the final control command. Although the GS behaviour is often suppressed, the GOAL factor is reflected in the turn rules of both OA and PP behaviours.

F. Obstacle Avoidance Behaviour

The local obstacle-avoidance (OA) behaviour is actually a sensor-based behaviour which implements a control strategy based on external sensing. OA behaviour is effective if obstacles are close. The visual sensors of the virtual agent are grouped into five sectors (Left, LeftFront, Front, RightFront and Right). If the virtual agent has a ring of eight forward vision fields, these will produce a set of obstacle distances (d0 through d7). From these, we obtain

three groups of obstacle distances using the following equations:

• dleft = min(d0,d1);

• dfront = min(d2,d3,d4,d5);

• dright= min(d6,d7).

The obstacle distance of each sector is represented by three linguistic fuzzy sets {NEAR, MEDIUM, FAR}. Observe that the rules exhibit this behaviour characteristic: if the obstacle distance in any sector is NEAR, the virtual agent should turn away to find a safer direction. For instance, it can be written out as the rule:

• IF dfront is Far AND dleft is Far AND dright is Near,

THEN θ_oa_{is PS}

Note that, when the virtual agent needs to turn, but the left and right sectors have the same obstacle distance, then the recommended turn angle is GOAL, where GOAL implies

that the recommended turn angle should be toward the direction closest to the goal location. This is similar to the turn rules for PP behaviour. One last important note: when the three sectors have the same NEAR obstacle distance, a large left turn (PL) angle is recommended. This turn rule enables the virtual agent to escape from its current dead-end situation.

Similarly to the weights of PP behaviour, the weights of OA behaviour Woa are represented by three linguistic fuzzy sets {SMALL, MEDIUM, LARGE}, and are derived directly from obstacle distances in the three sectors.

V. THE DECISION MAKING METHOD

This research is inspired by the ranking method of Huang [30], Mabuchi [31] and Yuan [32] and uses α_-level and fuzzy subtraction operations to calculate the area of a new fuzzy number, which is produced by the comparison of two fuzzy numbers. If there are mfuzzy numbers, then

2 / ) 1 (m−

m pairs of fuzzy numbers must be compared to determine the overall rank. Our proposed method will reduce the redundancy of calculating m(m−1)/2 pair-wise comparisons to m pair-wise comparisons by the fuzzy subtraction operation.

In general, when comparing m different fuzzy numbers produced by each behaviour (OA, GS, PP) the height and common maximizing and minimizing barriers are used. Let ~(x)

X

µ be the membership function of a fuzzy

number,

X

~

(behaviour output), defined on

R

. Unlike convexity, n assumptions about the normality of ~(x)

X µ

are made.

Based on [33], the loci of the left or right spreads and the maximum and minimum barriers of the α_{-cut of the} fuzzy number,

X

~

, are ~ (x)

L Xα

µ and ~ (x)

R Xα

µ , and 0≤α≤h_X~,

respectively, where h_X~is the height. If X_α

~

is denumerable or connected, then:

. 0 }, ~ | { max ) ( and , 0 }, ~ | { min ) ( ~ ~ ~ ~ X R X X L X h X x x x h X x x x ≤ ≤ ∈ = ≤ ≤ ∈ = α µ α µ α α α α (5)

The height, maximizing and minimizing barriers are set to:

}. 0 ; , , 2 , 1 | ) ( { max }, 0 ; , , 2 , 1 | ) ( { min }, , , 2 , 1 | { max ) ( ~ ~ ~ ~ ~ ~ X R X X L X X X h m i x d h m i x c m i x h i i i ≤ ≤ = = ≤ ≤ = = = = α µ α µ µ α α α α K K K (6)

Based on [30] and Figure 7, that is, h~(x)

X is the

(7)

J. Jaafar and E. McKenzie, The variables

left spread and the minimum right spread of all fuzzy numbers, respectively. To simplify the fuzzy subtraction between the fuzzy number

at αilevel, interval subtraction is used:

R X i ~ ~ 〈−〉 α

then, the behaviour weight,

=

R X

W j )

~ , ~ (

where n is the number of the approaches to

measurement. In equation (

value and

∑

i

denominator represents the total area as

In addition, if all of the aggregated fuzzy numbers are normal and within the unit interval,

, 0 , 1

~ = c=

h_x

R

X

W

j

~

,

~

(

In our case, the behaviour weight value (9) will be used. For

(maximin) criterion, which selects the lowest value from each behaviour as

from each behaviour as

σ

, is used to represent the level of uncertainty of the

J. Jaafar and E. McKenzie, Decision Making Method using Fuzzy Logic

Figure 7: Trapezoidal fuzzy number

The variables c and d are at the minimum value of the left spread and the minimum right spread of all fuzzy numbers, respectively. To simplify the fuzzy subtraction between the fuzzy number

level, interval subtraction is used:

r d r R i i i i , [ ][ ][ , [ ~ l l − = − =

hen, the behaviour weight,

− − − =

∑

= = = c r r n i n i i n i i ) ( ( 0 0

is the number of the

approaches to ∞, the summation becomes the area

measurement. In equation (

∑

= − n i i 0 ) 1

(l _{is a negative value. Here, the}

1

d = , and equation (

−

=

∑

= = =

r

R

_n i n i i n i

)

~

0 0 0

In our case, the behaviour weight value ) will be used. For every

(maximin) criterion, which selects the lowest value from each behaviour as δ1; and then selects the highest value

from each behaviour as δ

, is used to represent the level of uncertainty of the

Decision Making Method using Fuzzy Logic

: Trapezoidal fuzzy number

are at the minimum value of the left spread and the minimum right spread of all fuzzy numbers, respectively. To simplify the fuzzy subtraction between the fuzzy number

X

~

and referential rectangle

level, interval subtraction is used:

i c d c 1 , 0 ], ] , ][ = −

hen, the behaviour weight, W of equation (12) becomes:

− −

∑

= n c n i and , ) 1 ( ) 0 l

is the number of the

α

−

, the summation becomes the area

measurement. In equation (8),

∑

= − n i i r 0 (

is a negative value. Here, the

quation (8) becomes:

−

∑

r

i i

as

,

)

1 (

0

l

In our case, the behaviour weight value

every W , we use the minimax (maximin) criterion, which selects the lowest value from ; and then selects the highest value

2

δ . The index of optimism , is used to represent the level of uncertainty of the

Decision Making Method using Fuzzy Logic

: Trapezoidal fuzzy number.

are at the minimum value of the left spread and the minimum right spread of all fuzzy numbers, respectively. To simplify the fuzzy subtraction

and referential rectangle

n , , 2 , 1 K (7)

of equation (12) becomes:

∞ →

n

(

level

− and as , the summation becomes the area

−c) is a positive

denominator represents the total area as n approaches In addition, if all of the aggregated fuzzy numbers are normal and within the unit interval,

) becomes:

∞

→

n

(

In our case, the behaviour weight value W from equation , we use the minimax (maximin) criterion, which selects the lowest value from ; and then selects the highest value

. The index of optimism , is used to represent the level of uncertainty of the

Decision Making Method using Fuzzy Logic for Autonomous Agent Navigation

are at the minimum value of the left spread and the minimum right spread of all fuzzy numbers, respectively. To simplify the fuzzy subtraction and referential rectangle

R

~

,

)

of equation (12) becomes:

(8)

and as n , the summation becomes the area

is a positive

approaches∞. In addition, if all of the aggregated fuzzy numbers are then

(9)

from equation , we use the minimax (maximin) criterion, which selects the lowest value from ; and then selects the highest value [34], , is used to represent the level of uncertainty of the

virtual environment. When selecting one particular action from a range of possible actions, the selection is based on the Hurwicz criterion

where

Based on the above discussion been used

Figure

controller and the weight calculated in Weight

arbitrary bo behaviour. Step 1 Step 2 Step 3 Step 4 Step 5 Step 6

The aim of the

performance of the methods for autonomous navigation in an unknown environment. It is assumed that

agent has no information about the virtual environment. The only information known by the virtual agent are the coordinates of the start and target/goal points. The navigation task requires the virtual agent to activate each of the behaviours s

applicability. The shapes of obstacles are not used as parameters.

for Autonomous Agent Navigation

virtual environment. When selecting one particular action from a range of possible actions, the selection is based on the Hurwicz criterion

     = < = = =    ⋅ = ∈ = 1 0 0 where max min σ σ σ η σ η J j m i

Based on the above discussion been used in the ‘

Figure 2. Behaviour

controller and the weight calculated in Weight’ sub-module.

arbitrary bounded fuzzy numbers produced by behaviour.

Step 1: Set the height barrier

referential rectangle

Step 2: Determine the subtracted interval

[

l_i−d,r_i

level −

α

X~_i <−>

Step 3: Determine the behaviour weight, , using equation (

Step 4: Repeat Steps 2 and 3, for

j

=

0 ,

1 ,

2

for m fuzzy numbers are obtained.

Step 5: For every

criterion, which selects the lowest value from each behaviour as

value from each behaviour as

Step 6: Determine the index of

action is selected based on the Hurw using equation (10

VI. EXPERIMENTAL RESULTS The aim of the

performance of the methods for autonomous navigation in unknown environment. It is assumed that

agent has no information about the virtual environment. The only information known by the virtual agent are the coordinates of the start and target/goal points. The navigation task requires the virtual agent to activate each of the behaviours s

applicability. The shapes of obstacles are not used as parameters. The evaluation

for Autonomous Agent Navigation

virtual environment. When selecting one particular action from a range of possible actions, the selection is based on the Hurwicz criterion [35] which is defined as:

→ → < →          ⋅ − +    = = criterion Maximax 1 Compromise 1 criterion Maximin 0 min ) 1 ( min 1 1 0 σ σ σ ij m i J ij m W W

Based on the above discussion the ‘Decision Maker Behaviours are retrieve controller and the weight calculated in

module. Let X~₁

unded fuzzy numbers produced by

Set the height hg(

barrier d and minimizing barrier

referential rectangle

R

~

Determine the subtracted interval

]

c

− , i=0,1,2

levels for each fuzzy j

R~, =0,1,2,K

>

Determine the behaviour weight, equation (9).

Repeat Steps 2 and 3, for

m

,

2 K

and the

fuzzy numbers are obtained.

every W , use

criterion, which selects the lowest value from each behaviour as δ

value from each behaviour as

Determine the index of

action is selected based on the Hurw using equation (10).

XPERIMENTAL RESULTS The aim of the experiment

performance of the methods for autonomous navigation in unknown environment. It is assumed that

agent has no information about the virtual environment. The only information known by the virtual agent are the coordinates of the start and target/goal points. The navigation task requires the virtual agent to activate each of the behaviours separately in its own context of applicability. The shapes of obstacles are not used as

The evaluation was electronic Journal of Computer Science and Information Technology (eJCSIT), Vol. 3, No. 1, 2011

virtual environment. When selecting one particular action from a range of possible actions, the selection is based on

which is defined as:

   ⋅ − +       ⋅ = = criterion opinion Compromise criterion max ) 1 ( max ) 1 1 0 σ m i ij m i W

Based on the above discussion, the following

Decision Maker’ sub-module as in are retrieved from

controller and the weight calculated in the ‘

j

X X~ , ,~ , , 2

1 K K

unded fuzzy numbers produced by

)

x , common maximizing

and minimizing barrier

~

.

Determine the subtracted interval n

, ,

2K by calculating the s for each fuzzy

m , K .

Determine the behaviour weight, W

Repeat Steps 2 and 3, for

and the m behaviour weights fuzzy numbers are obtained.

, use the minimax (maximin) criterion, which selects the lowest value from

1

δ and selects the highest value from each behaviour as δ2.

Determine the index of optimism action is selected based on the Hurw

XPERIMENTAL RESULTS AND COMPARISON experiment is to measure performance of the methods for autonomous navigation in

unknown environment. It is assumed that

agent has no information about the virtual environment. The only information known by the virtual agent are the coordinates of the start and target/goal points. The navigation task requires the virtual agent to activate each eparately in its own context of applicability. The shapes of obstacles are not used as was divided into four which

14 virtual environment. When selecting one particular action from a range of possible actions, the selection is based on

which is defined as:

     

maxWij

(10)

the following steps have module as in from the fuzzy the ‘Behaviour

m

X~ ,

K be m unded fuzzy numbers produced by each

, common maximizing and minimizing barrier c for

Determine the subtracted interval numbers by calculating the n s for each fuzzy number

W for each Xj

~

Repeat Steps 2 and 3, for every

j

behaviour weights

the minimax (maximin) criterion, which selects the lowest value from and selects the highest

σ

. The final action is selected based on the Hurwicz criterion

MPARISON is to measure the performance of the methods for autonomous navigation in unknown environment. It is assumed that the virtual agent has no information about the virtual environment. The only information known by the virtual agent are the coordinates of the start and target/goal points. The navigation task requires the virtual agent to activate each eparately in its own context of applicability. The shapes of obstacles are not used as divided into four which

14 virtual environment. When selecting one particular action from a range of possible actions, the selection is based on

ve module as in fuzzy Behaviour m

each

, common maximizing for numbers n number j ,

behaviour weights

the minimax (maximin) criterion, which selects the lowest value from and selects the highest

final icz criterion

(8)

are basic navigation, escape from local minima, navigate in a cluttered and maze environment, and comparing with other methods.

A. Basic Skill

The aim is to evaluate basic skill required by virtual agent for navigating in unknown environment. The virtual agent requires reaching the goal and avoiding the obstacle.

Figure 8. Basic navigation skill for two wall, narrow door and edge The experiments had demonstrated that our virtual agent has the basic skills to navigate in an unknown virtual environment. The skills are avoiding an obstacle, reaching the goal, and navigating a narrow path (two wall, narrow door and edge) as in Figure 8. The virtual agent had produced consistent results (time, number of decisions and path length) for each 25 test run with a mean error less than 5%. The path produced can be considered smooth since there is minimal sharp turn.

Figure 9: Escaping from bench, corner and dead-end. We verify the performance of our approach by applying it to three types of basic obstacle shapes which are a bench corner and a dead-end as shown in Figure 9. The simulation shows that the virtual agent avoids the obstacle, then follows the wall and heads for the target. The path generated by the virtual agent can be considered smooth even though it is not a shortest path.

A. Navigate in cluttered and Maze Environment

A central issue in the design of reactive control architectures for autonomous virtual agents is the formulation of effective decision making mechanisms (ASMs) to coordinate the behaviours. Experiments will evaluate the Fuzzy-ASM method and compare the results with the behaviour fusion method (FBF) by [36]. Four test cases have been used which are the virtual agent being moved from the same starting point to different target points as in Figure 10 through 13 (Test cases 1 through 4); (a) is above (b).

Figure 13 shows the example of the path produced by the virtual agent in Test case 4. Figure 13(a) is the path

produced by Cang's Method and Figure 13(b) shows the path produced by our Fuzzy-ASM. The path produced by the Fuzzy-ASM is shorter than Cang's method even though the smoothness of the path is similar.

Figure 10. Test case 1 Figure 11. Test case 2

Figure 12. Test case 3. Figure 13. Test case 4. Table 3. The Performance of Fuzzy-ASM vs. FBF

Pa(m)

s s a

P P P E==== −−−−

Test

Case Ps(m)

Fuzzy-ASM FBF

1 21.67 22.62 24.12 4.38 11.31

2 21.47 27.73 na 29.16 na

3 13.55 19.64 na 44.94 na

4 22.16 22.78 24.34 2.80 9.39

Notes:

Ps - the shortest path length.

Pa - the actual path length.

(9)

J. Jaafar and E. McKenzie, Decision Making Method using Fuzzy Logic for Autonomous Agent Navigation 16 Moreover, Table 3 shows that Fuzzy-ASM had a better

performance with a small performance factor , which was 4.38% in Test Case 1 and 2.80% in Test Case 4 compared to a FBF of 11.31% and 9.39%, respectively. The difference in ܧ paths produced between the Fuzzy-ASM and FBF are 1.50 in Test Case 1 and 1.56 in Test Case 4. The difference in and path produced shows that the Fuzzy-ASM has produced better results compared to the FBF. Unfortunately, Test Case 2 and Test Case 3 showed that FBF was trapped in local minima and failed to complete the task. Fuzzy-ASM had a successful escape form the trap and reached the target point.

A. Comparison with other method

We evaluated our proposed fuzzy ASM and compared the result with behaviour fusion method propose by [37]. Figure 14 shows how the virtual agent navigates in a cluttered environment. Figure 14(a) is the path produce by proposed fuzzy ASM and Figure 14(b) shows the path produce by [37]. The path produce by fuzzy ASM shorter than Wang’s method even though the smoothness of the path almost the same.

Nine testing had been conducted in nine different environments. Figure 17 show the result of time, path length and number of decision taken by virtual agent for all nine environments. The result show proposed fuzzy method had produced less time and distance to complete the task. The average percentage of ∆_{time and}∆_length are 16% and 17.4% respectively.

(a) (b)

Figure 14. Example of path produced by (a) propose fuzzy ASM, and (b) Wang and Liu method

When we compare number of decision made by each method, it shows that the proposed fuzzy ASM has made fewer decisions in comparison to Wang’s method as in Figure 15 and16. This can be seen by the average percentage of ∆_{; the number of decisions is 8.04% less} than Wang’s method. Fewer decisions mean faster and more accurate decisions had been made during the process; they also lead to a faster and more reliable decision making process.

Figure 15. Comparisons of success rates

Figure 16. Comparisons of number of decisions

VII.CONCLUSIONS

Our method shows that behaviour rules containing

α

intervals of input and output spaces are easily integrated with a virtual agent. The evaluation results show that the virtual agent has deviated with minimum distance when avoiding the obstacles. The results also clearly demonstrate the mapping of inputs to output with a near-optimum path in a navigation task. This presents a natural way of dealing with a virtual environment using linguistic logic rules. The agent’s behaviours are easy to comprehend, because it captures the behaviour in linguistics form by a simple

α

−level technique. For future work, Type-2 Fuzzy Logic Controller can be explored. The main challenge is how our method can be used with Type-2 FLCs to reduce the computational burden by providing faster type-reduction methods and a simpler architecture.

0.00 20.00 40.00 60.00 80.00 100.00 120.00

1 2 3 4 5 6 7 8 9

Test

N

o

f

d

e

c

is

io

n

0.00 10.00 20.00 30.00 40.00 50.00 60.00 70.00 80.00 90.00 100.00

P

e

rc

e

n

t

Fuzzy ASM Wang Method ∆ Decision 0

20 40 60 80 100

1 2 3 4 5 6 7 8 9

Test

Fuzzy ASM Wang

(10)

Figure 17. Fuzzy ASM vs. Wang and Liu method

REFERENCES

[1] T. R. Wan, H. Chen, and R. Earnshaw, "Real-time path planning for navigation in unknown environment," in Theory and Practice of Computer Graphics 2003 (TPCG 03), Birmingham, UK, 2003, p. 138.

[2] T.-Y. Li, J.-M. Lien, S.-Y. Chiu, and T.-H. Yu, "Automatically generating virtual guided tours," in '99 Computer Animation Conference, Geneva, Switz, 1999, p. 99.

[3] K. Walker and A. C. Esterline, "Fuzzy motion planning using the Takagi-Sugeno method," in IEEE Southeastcon 2000, Nashville, TN, USA, 2000, p. 56.

[4] C. Gatzoulis, W. Tang, and T. R. Wan, "Fuzzy Reinforcement Learning for an evolving virtual servant robot," in 13th IEEE International Workshop on Robot and Human Interactive Communication, 2004 (ROMAN 2004), Okayama, Japan, 2004, p. 685.

[5] L. Yifan, P. Musilek, and L. Wyard-Scott, "Fuzzy logic in agent-based game design," in IEEE Annual Meeting of the Fuzzy Information, NAFIPS '04, Banff, Alta, Canada, 2004, p. 734. [6] S. Ho-Sub, Y. So-Joeng, and O. Kyung-Whan, "Fuzzy reinforcement

function for the intelligent agent to process vague goals," in Annual Conference of the North American Fuzzy Information Processing Society - NAFIPS, Atlanta, GA, USA, 2000, p. 29.

[7] P. Pirjanian and H. I. Christensen, "Behavior Coordination using Multiple-Objective Decision Making," in SPIE Conference on Intelligent Systems and Advanced Manufacturing, Pittsburgh, USA, 1997.

[8] H. A. Simon, The New Science of Management Decision: Prentice Hall, 1977.

[9] B. Brown, "Choice and mobility: decision making on the move,"

University of Glasgow, 2005.

http://www.dcs.gla.ac.uk/~barry/papers/decisionmaking.pdfAccess [10] P. Pirjanian and M. Mataric, "A decision-theoretic approach to fuzzy

behavior coordination," in 1999 IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA '99), Monterey, Canada, 1999, p. 101.

[11] P. Maes, "Situated agents can have goals," Special Issue of Journal of Robotics and Autonomous Vehicle Control, pp. 49-70, 1990. [12] B. Blumberg, "Action-selection in hamsterdam: lessons from

ethology," in Proceedings of the third international conference on Simulation of adaptive behavior : from animals to animats 3: from animals to animats 3 Brighton, United Kingdom: MIT Press, 1994. [13] K. Lerman, "Design and Mathematical Analysis of Agent-Based

Systems," in 1st. International Workshop on Formal Approaches to Agent-Based Systems-Revised Papers: Springer-Verlag, 2001. [14] J. Yen, M. S. El-Nasr, and T. R. Ioerger, "Fuzzy logic and intelligent

agents," in 1999 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE '99), Seoul, 1999, p. 342.

[15] T. Ziemke, "Adaptive Behavior in Autonomous Agents," Presence: Teleoper. Virtual Environ., vol. 7, pp. 564-587, 1998.

[16] D. C. MacKenzie, R. Arkin, and J. M. Cameron, "Multiagent Mission Specification and Execution," Autonomous Robots, vol. 4, p. 29, 1997.

[17] A. Saffiotti, "Fuzzy logic in autonomous robotics: Behavior coordination," in 6th IEEE International Conference on Fuzzy Systems, FUZZ-IEEE'97, Barcelona, Spain, 1997, p. 573.

[18] C. Brom and J. Bryson, "Action Selection for Intelligent Systems." vol. 13 February 2008: European Network for the Advancement of Artificial Cognitive System, 2006. http://www.eucognition.org/white_papers.htmAccess

[19] M. Scheutz and V. Andronache, "Architectural mechanisms for dynamic changes of behavior selection strategies in behavior-based systems," Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, vol. 34, pp. 2377-2395, 2004.

[20] V. Braitenberg, Vehicles: Experiments in Synthetic Psychology. Massachusetts, USA: The MIT Press, 1984.

[21] J. A. Dickerson and B. Kosko, "Virtual Worlds as Fuzzy Dynamical Systems," in Technology for Multimedia, B. Sheu, Ed.: IEEE Press, 1996.

[22] B.-Y. Chee, S. Y. T. Lang, and P. W. T. Tse, "Fuzzy mobile robot navigation and sensor integration," New Orleans, LA, USA, 1996, p. 7.

[23] S. Vosinakis and T. Panayiotopoulos, "A tool for constructing 3D environments with virtual agents," Multimedia Tools and Applications, vol. 25, p. 253, 2005.

[24] J. J. Kuffner and J.-C. Latombe, "Fast synthetic vision, memory, and learning models for virtual humans," Computer Animation, Conference Proceedings, p. 118, 1999.

[25] B. Kosko, Neural Networks and Fuzzy Systems: A Dynamical Systems Approach to Machine Intelligence. New Jersey: Prentice-Hall inc., 1992.

[26] J. Jaafar and E. McKenzie, "Behaviour Coordination of Virtual Agent Navigation using Fuzzy Logic," in 2006 IEEE International Conference on Fuzzy Systems, 2006, pp. 1139-1145.

[27] J. Jaafar and E. Mckenzie, "A reactive architecture for autonomous agent navigation using fuzzy logic," in Artificial Intelligence and Soft Computing 2007, Palma de Mallorca, Spain, 2007.

[28] T. J. Ross, Fuzzy Logic with Engineering Application, 2 ed. West Sussex: John Wiley & Sons, 2004.

[29] L.-X. Wang, A Course on Fuzzy System and Control. London: Prentice-Hall International Inc., 1997.

[30] C. C. Huang, "A Study on The Fuzzy Ranking and Its Application on The Decision Support System." vol. Phd Taiwan: Tamkang University, 1989.

[31] S. Mabuchi, "An approach to the comparison of fuzzy subsets with an alpha-cut dependent index," Systems, Man and Cybernetics, IEEE

0.00 2.00 4.00 6.00 8.00 10.00 12.00 14.00

1 2 3 4 5 6 7 8 9

Test

T

im

e

0.00 10.00 20.00 30.00 40.00 50.00 60.00 70.00 80.00

P

e

rc

e

n

t

Propose Fuzzy ASM Wang Method ∆ Time

0.00 20.00 40.00 60.00 80.00 100.00 120.00 140.00

1 2 3 4 5 6 7 8 9

Test

L

e

n

g

th

0.00 10.00 20.00 30.00 40.00 50.00 60.00 70.00 80.00

P

e

rc

e

n

t

(11)

J. Jaafar and E. McKenzie, Decision Making Method using Fuzzy Logic for Autonomous Agent Navigation 18 [32] Y. Yuan, "Criteria for evaluating fuzzy ranking methods," Fuzzy

Sets and Systems, vol. 43, pp. 139-157, 25 September 1991 1991. [33] F. Choobineh and H. Li, "An Index for Ordering Fuzzy Numbers,"

Fuzzy Sets and Systems, vol. 54, p. 287, 1993.

[34] T. Y. Chen and Y. T. Yu, "On the Criteria of Allocating Test Cases under Uncertainty," in Fourth Asia-Pacific Software Engineering and International Computer Science Conference Hong Kong: IEEE Computer Society, 1997.

[35] B. F. Arnold, I. Grossl, and P. Stahlecker, "The minimax, the minimin, and the Hurwicz adjustment principle," Theory and Decision, vol. 52, p. 233, 2002.

[36] Y. Cang and W. Danwei, "Novel Behavior Fusion method for the navigation of mobile robots," in 2000 IEEE International Conference on Systems, Man and Cybernetics, Nashville, USA, 2000, p. 3526. [37] M. Wang and J. N. K. Liu, "Autonomous robot navigation using