Contextual Knowledge in Autonomous Robot Agents

Commonly, robot plans in cognitivist architectures such as CRAM are structured as hierarchical trees, allowing encapsulation and modular reuse. An underrepresented aspect in plan languages

Generalized Plan Design and Representation for Robots

is implicit context awareness. Tasks ought to perform dierently based on their explicit parame- terization, but also on implicit circumstances. These depend on dynamic factors, such as volatile perception knowledge, but also on static knowledge available through external knowledge bases. Abstract generalized, modular plans are not supposed to cover all possible situations ex- plicitly, but represent a generalist strategy that changes its structure and resulting behavior based on available knowledge. As briey introduced in Section 4.2, I therefore introduce the with-context environment:

(with-context (c1 ... cn) code )

Within plans, a contextual parametrization for other plan building blocks they are using can be dened. Any layer can therefore either add new contextual parameters ci, or alter old ones. The behaviour of single plan blocks in code can thus be inuenced by a semantically higher hierarchy. One simple example that benets from this approach is a context that describes the transport of liquid lled mugs. The result would produce dierent maximum tilting angles during motion planning than when transporting empty ones, given the assumption that nothing should be spilled.

In the following subsections, I will start with an example explaining how the with-context environment is used, going into detail about its implementation. I will then briey discuss static and dynamic knowledge, and give an overview of where the dierent types of knowledge commonly originate from in robot systems.

4.4.1 Contextually Constraining Generalized Plans

This section motivates the usage of with-context, and its eect on lower-level plans. Here, the fetch-object task from Section 4.3.6 is constrained when transporting liquid-lled objects (e.g. coee cups). In this plan, the non-lazy variant of the search-object plan is used. For the sake of clarity, if search-object is called with the description of an object already found, it just returns that instance.

(def-plan constrained-fetch-object (obj) (let* ((obj (search-object obj))

(obj-state (perceive-state obj)) ;; Identifies facts of `obj'

(obj (or (when (has-fact obj-state :liquid-level) (update-object

obj `((:liquid-level

,(get-fact obj-state :liquid-level))))) obj)))

(with-context `((:max-tilt-angle ,(kb-constraint :max-tilt-angle obj)) (:max-velocity ,(kb-constraint :max-velocity obj))) (fetch-object obj))))

The above plan is overly explicit. It does, however, show the implications of using contex- tual constraints: fetch-object is implicitly re-parametrized without mentioning those changes in any arguments to the function. fetch-object retrieves required constraints by calling context-constraints (again, see Section 4.3.6). In the above plan, kb-constraint returns the value of a specic constraint as reported by the knowledge base, based on a complete object description. A more general version of constrained-fetch-object incorporating all possible modiers from an external knowledge base is shown below:

(def-plan constrained-fetch-object (obj) (let* ((obj (search-object obj))

4. Contextual Knowledge in Autonomous Robot Agents (facts (mapcar (lambda (name) `(,name ,(get-fact obj-state name)))

(fact-names obj-state))) (obj (update-object obj facts))

(constraints (mapcar (lambda (name) `(,name ,(kb-constraint name obj))) (fact-names obj-state))))

(with-context constraints (fetch-object obj))))

In this version, rst all facts are extracted from the object state and inserted into the object description. Using this updated description, all constraints available from the knowledge base are collected. These constraints then parametrize with-context, and further implicitly constrain fetch-object. A possible implementation of with-context is seemingly simple:

(defvar *contextual-constraints* (make-hash-table)) (defun get-contextual-constraints ()

*contextual-constraints*)

(defun set-contextual-constraints (constraints) (setf *contextual-constraints* constraints)) (defun context-constraint (constraint)

(gethash constraint *contextual-constraints*)) (defun context-constraints (constraints)

(mapcar (lambda (constraint) `(,constraint ,(context-constraint constraint))) constraints))

(defmacro with-context (constraints &body code)

`(let* ((old-constraints (get-contextual-constraints)) (new-constraints (make-hash-table)))

;; Make a copy of the existing constraints table

(loop for h being the hash-keys in old-constraints

do (setf (gethash h new-constraints) (gethash h old-constraints))) (dolist (constraint ,constraints) ;; Update inner scope

(destructuring-bind (name value) constraint (setf (gethash name new-constraints) value))) (set-contextual-constraints new-constraints) (unwind-protect (progn ,@code)

(set-contextual-constraints old-constraints)))) ;; Return to outer scope

4.4.2 Static and Dynamic Knowledge

I will briey explain two types of knowledge a robot has at its disposal, what their origins are, and what they are used for: (1) Static and (2) dynamic (volatile) knowledge.

Static knowledge is all information known a priori that does not change during the course of a single robot task episode. Task episodes are enclosed sets of actions. If a piece of information, such as the opening degree of a drawer, changes in between two task episodes  opening the drawer and picking an object from it  the information about its collision model becomes quasi-static for the governing task, such as setting a table.

Generalized Plan Design and Representation for Robots

Information Static Dynamic Source

Floor Maps Map les

Collision Environment Environment description

Task Description Parent task / human operator

Person's Meal Preferences Knowledge Base / Experience

Perception Results Perception system / sensors

Kinematic Robot State Joint sensors / self model

Object Pose in Hand Object tracking

Milk Availability Check kitchen

Table 4.2: Common types of either static or dynamic information and where they originate from

Figure 4.10: Decision ow of grasping an object based on both, static and dynamic information: Available arms, known grasps for the object, their current feasibility, xed (pre-)grasp osets, and the volatile success/failure of all intermediate steps.

Dynamic knowledge is volatile information unavailable at design time that becomes avail- able at, and vanishes again during runtime. Dynamic knowledge is either information recorded by sensors, acquired from other external input channels, or inferred from available information (dynamic or static).

Table 4.2 shows common types of information from both classes and where they originate from. Figure 4.10 shows the decision ow of grasping an object based on both, static and dynamic information.

A special type of quasi-static knowledge are Episodic Memories. They specically change only after complete episodes, and contain all relevant dynamic knowledge that was deemed volatile during the execution of the task episode. This knowledge is now available as an a posteriori source for knowledge inference, resulting in new static knowledge for the next task episode. Chapter 5 details this topic further in great detail.