Deliberation to Control Interfaces

This section presents past and current research of systems and techniques that try to interface deliberative plans and behavior execution. Researchers have identified that behavior selection in behavior-based architectures is usually hand-coded to fit an architecture and is amenable to change [2, 43]. Therefore, some techniques have been developed in attempt to automate this process to make the system more robust.

3.3.1 Universal Plans. Universal plans are plans that compactly represent every classical plan [44]. The Universal plan is synthesized automatically and executes the classical plan that corresponds to the environment that the agent encounters at runtime. This approach requires that each state follows a Markov assumption and that the programmer knows all of the states that the robot encounters. This leads to a hard-coded Sequencer that requires recoding and testing whenever the system or the working environment changes. System changes range from adding or removing a behavioral capability to slightly modifying an existing behavior to function differently. This technique is not appropriate for creating a robust and modular system and is representative of the common hand-coded behavior selection techniques.

3.3.2 Behavior Coordination. Behavior coordination is the mechanism of executing the behaviors that are selected for activation accordingly to accomplish the high-level goals. In [43], the authors propose a generic architecture for dynamic behavior selection, which can integrate existing behavior selection mechanisms in a unified way. Although similar to the UBF in that this mechanism addresses the prob- lem of selecting behaviors to accomplish high-level plans, it differs from the UBF in that it dynamically changes arbitration techniques rather than dynamically selecting the behaviors and the arbitration technique. Thus, it functions more as an arbitra- tion technique switch. This functionality is accomplished in the Controller, rather than the Sequencer, through the UBF using the arbitrated behavior hierarchy that the Sequencer provides. In [43], the hierarchies are created and implemented for spe-

achieve in this thesis. Although the framework is modular for easily changing the behavior selection strategies, the interface that links the decisions to change the be- havior selection technique is still hand-coded to the specific behaviors that the system employs.

Another approach for behavior coordination is to perform a dynamic struc- turing of a hybrid behavior group coordinator by using priority and Finite State Automata (FSA) [49]. Although this approach is dynamic, the majority of the work is hand-coded in the FSAs before execution. Additionally, the technique presented in [49] requires predefined FSAs as behavior group selectors (behavior hierarchies) for each task that the system accomplishes. Lastly, in [39] they use a degenerative sequencing layer that is merely a switch for activating different behavior sets (or “navigation modes”). Therefore, the deliberative layer is really controlling the robot, which is not always active.

3.3.3 Learning. Argall, et al [2], state that hierarchical state machines are powerful tools for controlling autonomous robots, but the control hierarchy is often hand-coded and error prone. To combat this, they use an expert learning approach to learn to select the best state machine expert to execute for a particular task and state belief. This method is beneficial for determining the best skill (or behavior) created for a specific task. However, defining a reward in a dynamic environment and ensuring that the system has learned the best expert for all cases is difficult. Additionally, the behavior hierarchies are still hand-coded and this technique only determines the best of the hand-coded behaviors.

3.3.4 Abstraction. Abstraction is a term that denotes some strategic form of problem simplification that involves a representational transformation [36]. By abstracting the problem space, highly complex environments can be distilled into more manageable and relevant environments suitable for a planning domain. The use of an abstraction hierarchy recursively abstracts the problem space from the original space to the most abstract problem space. This is useful for eliminating complex

details so the planning system can focus on the goal to achieve regardless of the vast initial conditions. By distilling an abstraction to a base level, high-level plans can map to base level behavior abstractions. Nourbakhsh [36] uses a state-based abstraction hierarchy and assumes action models are expressed as arbitrary mappings between knowledge states. However, the actual translation from abstract description to concrete behavior connection is not clear.

The idea that “plans should abstractly describe” the intended behavior of the system instead of strictly “dictating” it was presented in [37]. This idea contradicts the traditional model of plans as executable programs that identify precise behaviors to accomplish intended goals. In [38], Pfleger explores this concept by describing a system that produces plans that accomplish tasks but not the actual method. The method to be used is determined at “run-time”. If a plan abstractly describes the intended behavior, then the abstract behavior representation can be a placeholder that is filled at run-time with the actual behavior implementation. This allows for interchangeable behaviors for dynamic updates. Pfleger refers to these plans as plans- as-intentions and identifies that they are not well defined and need more information for effective use in the plan-following phase. This thesis aims to abstract the behaviors in a well defined manner so that a plan is generated and subsequently followed. The inner workings of the behavior (or explicit implementation) are not known to the “planner” and therefore, the “planner” plans tasks to be accomplished and the actual method is determined at run-time of the behavior. This is ideal so the behavior can determine (through sensor readings) the appropriate method to accomplish its intended behavior. Actual “abstract plan” to behavior assignment is not addressed, but is suggested to occur in the plan following phase of an agent architecture [22].

A weakness of abstraction is the potential of generating less optimal execution plans because some underlying details are ignored. Therefore, the level of abstraction must be effective enough to facilitate planning but not broad enough to overlook crucial information. When abstracting behaviors, the higher-level planners assume that the abstract representations map directly to output with minimal uncertainty.

3.3.5 Plan Libraries. In [34], the authors argue that the performance of plan-based control critically depends on the design and implementation of plan li- braries. The planning system employs the use of low-level and high-level hierarchical planning of a library of predefined low-level and high-level plans. Low-level plans are “black-boxes” that the system does not have to reason out to employ. They achieve certain goals by containing specific steps to achieving the designated goal. This poses possible problems if the abstraction of the “black box” is too high or too low. If the abstraction is too high, it may limit the behavioral functionality of the system. Conversely, if the abstraction is too low, then the time spent in the search space may outweigh the increased behavior functionality. A low-level plan’s main characteriza- tion is that it substantiates the execution parameters that are abstracted away by the high-level plans. High-level plans are the “plans that the planning mechanisms have to reason about.” The high-level plan contains the mechanisms for goal monitoring, failure detection and failure handling/recovery. At this high level of abstraction, the high-level goals are achieved by reasoning/decomposing the goals into sub-goals that the low-level plans can achieve. In this system, plans are expressed in RPL and thus have the same limitations described in Section 3.1.2.

3.4 Summary

When translating plans into actual robot actions and behaviors, a number of methods have been employed. Some architectures hardcode this mechanism like uni- versal plans and some create the mechanism a posteriori to augment the current architecture to perform more effectively (e.g. IXTET −E XEC). However different, many of the techniques share common concerns and considerations, which include translating desired goal to behavior execution, monitoring behavior performance, and quickly reacting to environment changes. The constructs of plan execution languages identify that using a standardized behavior representation enables robust and mod- ular sequencing of plans to active behaviors. Additionally, abstraction simplifies the search and allows the translation of high-level goals to low-level actions. The abstrac-

tion of behavior functionality through its representation appears to be the limiting factor for most of the techniques. By creating a behavior representation that is ro- bust and modular, an appropriate sequencing mechanism can be developed to link the decomposed plans of the Deliberator to the behavior execution of the Controller.