Cognition s Interface to the World and to Metacognition in MIDCA

(1)

Cognition’s Interface to the World

and to Metacognition in MIDCA

MICHAEL T. COX

D E PA R T M E N T O F C O M P U T E R S C I E N C E & E N G I N E E R I N G W R I G H T S TAT E U N I V E R S I T Y, D AY TO N , O H

Virtual International Symposium on Cognitive

Architecture 2021 ♠ 8 May 2021

(2)

Levels of Cognitive Computation

Ground Level

◦

Action

◦

Perception

Object Level

◦

Problem Solving

◦

Comprehension

Meta-Level

◦

Meta-level Control

◦

Introspective Monitoring

_{Cox, M. T., & Raja, A. (2011). Metareasoning: An introduction.}

(3)

The

Metacognitive

Integrated

Dual-Cycle

Architecture

(MIDCA)

Cox, M. T., Alavi, Z., Dannenhauer, D., Eyorokon, V., Munoz-Avila,

Object Level

Meta-Level

Metacognition

Cognition

Memory Mission & Goals( ) World Model (MΨ) Episodic Memory Semantic Memory Problem Solving Comprehension

goal change goal input goal insertion Intend Plan Evaluate Interpret Goals subgoal Goal Actions MΨ MΨ MΨ State Hypotheses goal change Intend Controller Plan Evaluate Monitor Interpret Meta Goals goal insertion subgoal goal input Goal Hypotheses Algorithms _Trace Meta-Level

Control Introspective _Monitoring

Memory Reasoning Trace ( ) Strategies (△ ) Episodic Memory Metaknowledge Self Model ( ) Mental Domain = Ω Goal Management

(4)

Executive Metacognition

Memory

Mission & Goals( ) World Model (M_Ψ) Episodic Memory Semantic Memory ( ) & Ontology Plans( ) & States( ) Problem Solving Comprehension

goal change goal input

(5)

Introduction

Cognition’s Interfaces

Cognition’s Interface to the Environment

Cognition’s Interface to Metacognition

Conclusion

(6)

Cognition’s Interfaces

(7)

Dannenhauer, Z. A., & Cox, M. T. (2018). Rationale-based perceptual monitors. AI Communications, 31(2), 197-212.

Cognition’s

Interfaces in

MIDCA

MIDCA Cognitive cycle

MIDCA Metacognitive cycle

Memory T rac e C ont rol

Standard Simulator

Buffers

Feedback Audio Image

(8)

Cognition’s Interface to

the Environment

(9)

Domain Structure Definition

Object Detection

◦

Type hierarchy

◦

Object attributes and values

◦

Relationships between objects

Action Execution

◦

Operators mapped to low-level

communication channels

Configuration File parsed for new domain

◦

Generates object detection stubs

◦

Creates handlers for MIDCA

Act and Perceive phases

(10)

MIDCA / ROS API

on(block1,block2)

( { block1 color:“red”; z: “11”; y: “2”; x: “2”}, { block2 color:“green”; z: “6”; y: “2”; x: “2”} ) Command Nodes

unstack(block1,block2)

loc_cmd(11,2,2) grab_cmd() raise_cmd() Object Detection Nodes

MIDCA’s Knowledge

of states and actions is

in terms of

◦

Logical predicates

◦

PDDL operators

Mapped to and from

◦

ROS nodes through

◦

ROS topics

Topics:

(11)

Perception

Sensor Handler

◦

Detects objects in environment

◦

Places their IDs and attributes in

perceptual buffer

Perceive Phase

◦

Infers predicate representations of objects

and their relationships

(12)

Action

Execution Handler

◦

Packages command arguments into strings

◦

Sends message to command nodes through

associated topics to control effectors

Act Phase

◦

Watches for feedback from ROS

Problem: Too much unnecessary

(13)

Plan Monitoring Demo

Neo

◦

The Baxter humanoid robot

Plan and Goal Monitors

◦

Manage perception

(14)

Cognition’s Interface to

Metacognition

(15)

The

Metacognitive

Cycle of

MIDCA

goal change

Intend

Controller

Plan

Evaluate

Monitor

Interpret

Meta Goals

goal insertion subgoal goal input

Goal

Hypotheses

Algorithms

_Trace

Meta-Level

Control Introspective _Monitoring

(16)

Cognitive Phases as mental actions

Memory as mental states

Memory

Snapshot 1

Intend

Plan

Introspective Monitoring

Memory

Snapshot 2

Snapshot 3

Memory

_{(& Speak)}

Act

(17)

Metacognitive Expectations

A Mental State is a vector of variables ⃑" = "

₁

, … , "'

(

)

= (+

_,

, -

_{., /, 0Y, 1, χ, 3}

_,

)

◦

Current goal set

◦

Goal agenda

◦

Current plan

◦

World state

Metacognitive Expectation as Boolean function over segment of mental trace

56 )

(

₇

)

, 3

₇

)

, (

₇₈₉

)

→ {⊤, ⊥}

◦

Discrepancies

◦

Explanation

(18)

Metacognitive Expectation for ‘Explanation’ Mental Action

("

_#

$

, &'()+,-.+, "

_#/0

$

)

Memory

Snapshot 1

Problems

Detect

Explanation

Metacognitive Expectation for Interpret

Memory

Snapshot 2

Snapshot 3

Memory

_Formulation

Goal

Prior mental

state

Mental

action

Post mental

state

(19)

Meta-Level Control

Goal Change

◦

Goal transformation

◦

Goal priorities

Execution Control

◦

Over cognitive phases

Learning

◦

Action models

◦

Other?

Executive Metacognition

World =

Ψ

Memory

Mission & Goals( ) World Model (MΨ) Episodic Memory Semantic Memory ( ) & Ontology Plans( ) & States( ) Problem Solving Comprehension goal change goal input

(20)

Metacognition Example

Plant Protection Domain with faulty spray op

◦

Goals: all invasive plants; all native plants alive

◦

Spray herbicide in one cell but kills native plant

in adjacent cell

Goal Expectation Failure detected

by Interpret at object level

◦

Goal to preserve native plant but now dead

◦

MIDCA cannot explain the failure

Metacognitive Expectation Failure detected

by meta-level Interpret at meta-level

(21)

Experimental Results in Two Domains

Average performance as a function of problem complexity

in the Plant Protection domain.

(22)

Open-source Code available at https://github.com/COLAB2/midca

Integrating Behavior, Cognition and Metacognition is hard for any architecture

◦

MIDCA has explored metacognition from the beginning but only lightly

◦

MIDCA has ignored grounded cognition for a long time

◦

But both have started to take off as of late

The Future is interesting

(23)

Backup Slides

(24)

Origins of MIDCA: Don Norman (1986)

Norman, D. 1986. Cognitive Engineering. In D. Norman and S. Draper eds., User-centered system design: New perspectives on

human-computer interaction. Hillsdale, NJ: Lawrence Erlbaum.

Norman’s HCI model

MIDCA’s Cognition model

Cox, M. T.; Oates, T.; and Perlis, D. 2011. Toward an integrated metacognitive architecture. In P. Langley ed., Advances in

(25)