From Memory to Problem Solving: Mechanism Reuse in a Graphi - PowerPoint Presentation

Slide1

From Memory to Problem Solving: Mechanism Reuse in a Graphical Cognitive Architecture

Paul S. Rosenbloom | 8/5/2011

The projects or efforts depicted were or are sponsored by the U.S. Army Research, Development, and Engineering Command (RDECOM) Simulation Training and Technology Center (STTC) and the Air Force Office of Scientific Research, Asian Office of Aerospace Research and Development (AFOSR/AOARD). The content or information presented does not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred.Slide2

Cognitive Architecture

Symbolic working memory

Long-term memory of rules

Decide what to do next based on preferences generated by rules

Reflect when can’t decide

Learn results of reflection

Interact with world

Soar 3-8

Cognitive architecture

: hypothesis about fixed structure underlying intelligent behavior

Defines core memories, reasoning processes, learning mechanisms, external interfaces, etc.

Yields intelligent behavior when add knowledge and skills

May serve as

a

Unified Theory of Cognition

the core of virtual humans and intelligent agents or robotsthe basis for artificial general intelligence

ICT 2010Slide3

Hybrid Short-Term Memory

Prediction-Based Learning

Hybrid Mixed Long-Term Memory

Graphical Architecture

Decision

How to

build architectures that

combine

:

Theoretical elegance, simplicity, maintainability, extendibility

Broad scope of capability and applicability

Embodying a superset of

existing architectural capabilities

Cognitive

, perceptuomotor, emotive, social, adaptive,

…Diversity Dilemma

Soar 9

Soar

3-8Slide4

Goals of This Work

Extend graphical memory architecture to (Soar-like) problem solvingOperator generation, evaluation, selection and applicationReuse existing memory mechanisms, based on graphical models, as much as possibleEvaluate ability to extend architectural functionality while retaining simplicity and eleganceEvidence for ability of approach to resolve diversity dilemmaSlide5

Problem Solving in Soar

Base levelGenerate, evaluate, select and apply operatorsGeneration: Retractable rule firing – LTM(WM)  WMEvaluation: Retractable rule firing – LTM(WM)  PM (Preferences)Selection: Decision procedure – PM(WM)  WMApplication: Latched rule firing – LTM(WM)  WMMeta level (not focus here)

LTM

PM

WM

Selection

Application

Generation

Evaluation

Decision Cycle

Elaboration Cycle

Match Cycle

Elaboration cycles + decision

Parallel rule match + firing

Pass token within Rete rule-match network

DSlide6

Enable efficient computation over multivariate functions by decomposing them into products of subfunctions

Bayesian/Markov networks, Markov/conditional random fields, factor graphsYield broad capability from a uniform baseState of the art performance across symbols, probabilities and signals via uniform representation and reasoning algorithm(Loopy) belief propagation, forward-backward algorithm, Kalman filters, Viterbi algorithm, FFT, turbo decoding, arc-consistency and production match, …

Support mixed and hybrid processingSeveral neural network models map onto themGraphical Models

w

y

x

z

u

p

(

u

,

w

,

x

,

y,z) = p(u)p(w)p(x

|u,w

)p(y|

x)p(z|x)

f

1

w

f

3

f

2

y

x

z

u

f

(

u

,

w

,

x

,

y

,

z

) =

f

1

(

u

,

w

,

x

)

f

2

(

x

,

y

,

z

)

f

3

(

z

)Slide7

The Graphical ArchitectureFactor Graphs

and the Summary Product AlgorithmSummary product processes messages on linksMessages are distributions over domains of variables on linkAt variable nodes messages are combined via pointwise productAt factor nodes input product is multiplied with factor function and then all variables not in output are summarized out

f

1

w

f

3

f

2

y

x

z

u

f

(

u

,

w

,

x

,

y

,

z

) =

f

1

(

u

,

w

,

x

)

f

2

(

x

,

y

,

z

)

f

3

(

z

)

.2

.4

.1

.3

.2

.1

.06

.08

.01

A single settling of the graph can efficiently compute:

Variable marginals

Maximum a posterior (MAP) probs

.Slide8

A Hybrid Mixed Function/Message Representation

Represent both messages and factor functions as multidimensional continuous functions Approximated as piecewise linear over rectilinear regionsDiscretize domain for discrete distributions & symbols[1,2>=.2, [2,3>=.5, [3,4>=.3, … Booleanize

range (and add symbol table) for symbols[0,1>=1  Color(x

,

Red

)=

True, [1,2>=0  Color(x, Green)=False

y

\x[0,10>[10,25>

[25,50>[0,5>

0.2y

0[5,15>.5x

1

.1+.2x+.4ySlide9

Graphical Memory Architecture

Developed general knowledge representation layer on top of factor graphs and summary productDifferentiates long-term and working memoriesLong-term memory defines a graphWorking memory specifies peripheral factor nodesWorking memory consists of instances of predicates(Next ob1:O1 ob2:O2), (weight object:O1 value

:10)Provides fixed evidence for a single settling of the graphLong-term memory consists of conditionalsGeneralized rules defined via predicate patterns

and

functions

Patterns define

conditions, actions and condacts (a neologism)Functions are mixed hybrid over pattern variables in conditionalsEach predicate induces own working memory node

WMSlide10

Conditionals

CONDITIONAL

Transitive

c

onditions

: (Next ob1:a ob2:

b

)

(Next ob1:b

ob2:c)

a

ctions

: (Next ob1:a

ob2:c

)

WM

Pattern

Join

w

\

c

Walker

Table

…

[1,10>

.01

w

.001

w

…

[10,20>

.2-.01

w

“

…

[20,50>

0

.025-.00025

w

…

[50,100>

“

“

…

CONDITIONAL

Concept-Weight

c

ondacts

: (concept

object:

O1

c

lass:

c

)

(weight

o

bject:

O1

v

alue:

w

)

function:

WM

Pattern

Join

Function

Conditions

test WM

Actions

propose changes to WM

Condacts

test

and

change WM

Functions

modulate variables

All four can be freely mixedSlide11

A rule-based procedural memorySemantic and episodic declarative memories

Semantic: Based on cued object features, statistically predict object’s concept plus all uncued featuresA constraint memoryBeginnings of an imagery memoryMemory Capabilities Implemented

CONDITIONAL Transitive

Conditions:

Next(

a,b)

Next(

b,c

) Actions:

Next(a

,c)

WM

Pattern

Join

w

\

c

Walker

Table

…

[1,10>

.01

w

.001

w

…

[10,20>

.2-.01

w

“

…

[20,50>

0

.025-.00025

w

…

[50,100>

“

“

…

Function:

CONDITIONAL

ConceptWeight

Condacts: Concept(O1,

c

)

Weight

(O1,

w

)

Concept (S)

Legs (D)

Mobile (B)

Weight (C)

Color (S)

Alive (B)Slide12

Additional Aspects Relevant to Problem SolvingOpen World versus

Closed World PredicatesPredicates may be open world or closed worldDo unspecified WM regions default to false (0) or unknown (1)?A key distinction between declarative and procedural memoryOpen world allows changes within a graph cyclePredicts unknown values within a graph cycleChains within a graph cycleRetracts when WM basis changesClosed world only changes across cycles

Chains only across graph cyclesLatches results in WMSlide13

Predicate variables may be universal

or uniqueUniversal act like rule variablesDetermine all matching valuesActions insert all (non-negated) results into WMAnd delete all negated results from WMUnique act like random variablesDetermine distribution over best valueActions insert only a single best value into WMNegations clamp values to 0Additional Aspects Relevant to Problem SolvingUniversal versus Unique Variables

Join

Negate

WM

Changes

+

–

Action combination subgraph:Slide14

The last message sent along each link in the graph is cached on the link

Forms a set of link memories that last until messages changeSubsume alpha & beta memories in Rete-like rule match cycleAdditional Aspects Relevant to Problem SolvingLink MemorySlide15

Problem Solving in the

Graphical ArchitectureBase levelGenerate, evaluate, select and apply operatorsGeneration: (Retractable) Open world actions – LTM(WM)  WMEvaluation: (Retractable) Actions + functions – LTM(WM)  LMSelection: Unique variables – LM(WM)  WMApplication: (Latched) Closed world actions – LTM(WM)

 WMMeta level (not focus here)

LTM

L

M

WM

Selection

Application

Generation

Evaluation

Graph Cycle

Message Cycle

Message cycles + WM change

Process message within factor graphSlide16

Eight Puzzle Results

Preferences encoded via functions and negationsTotal of 19 conditionals* to solve simple problems in a Soar-like fashion (without reflection)747 nodes (404 variable, 343 factor) and 829 linksSample problem takes 6220 messages over 9 decisions (13 sec) CONDITIONAL

goal-best ; Prefer operator that moves a tile into its desired location :conditions (blank

state:

s

cell:cb) (acceptable state:s operator:ct) (location cell:ct tile:

t) (goal cell:

cb tile:t)

:actions (selected states operator:ct)

:function 10

CONDITIONAL previous-reject ; Reject previously moved operator

:conditions

(acceptable state:s operator:ct

) (previous state:s operator:

ct)

:actions (selected - state:s operator:ct)Slide17

Conclusion

Soar-like base-level problem solving grounds directly in mechanisms in graphical memory architectureFactor graphs and conditionals  knowledge in problem solvingSummary product algorithm  processingMixed functions  symbolic and numeric preferencesLink memories

 preference memoryOpen world vs. closed world 

generation vs. application

Universal

vs. unique  generation vs. selectionAlmost total reuse augurs well for diversity dilemmaOnly added architectural selected predicate for operatorsAlso progressing on other forms of problem solvingSoar-like reflective processing (e.g., search in problem spaces)POMDP-based operator evaluation (decision-theoretic lookahead)