Painting an experience? - PowerPoint Presentation

Slide1

Painting an experience?How aesthetics might assist a neuroscience of sensory experience

Ron ChrisleyCentre for Research in Cognitive Science andSchool of Informatics, University of SussexNeuroesthetics: Where Art and the Brain CollideESF Workshop, IULM, Milan 24-25 September 2009Slide2

5 relevant areas of my research

Embodied creativityEnactive models of experienceSynthetic phenomenologyInteractive empiricism

Art works/installations

(The title of my talk concerns area 3)Slide3

My research and neuroestheticsNot a neuroscientist

Much of the work I am reporting on is trying to provide a bridge from art/aesthetics into the cognitive sciences, which then permits a connection with neuroscienceRe: this workshop:How can the models of aesthetic processes I am investigating be informed by neuroscience/aesthetics?Can the models suggest future directions for neuroscience? Aesthetics/art?Slide4

1. Embodied creativity

Goal: Design a robot/environment system likely to exhibit creative behaviour:Novel (at least for the robot)Of (aesthetic) value (for humans, if possible)Engineering approach:No direct modelling of human creativityBut exploit what is known about creativity in humans (and animals?), when expedient

Allow for possibility that insights into the human case may accrue anywayManifesto only: No implementation yet

Set of "axioms"

Assume case of musical output for examplesSlide5

Principles of embodied (aesthetic) creativity

If you make your robot pleasure-seeking, and make creativity pleasurable, you'll make your robot creativeTo be a good creator, it helps to be an appreciatorLet the robot experience output in the real world, as we do

We won’t like what the robot likes unless it likes what we like

An important motivator is the approval or attention of others

Novelty can be achieved by trying to produce outputs on the subjective edge of chaos (that lie just beyond the robot’s ability to explain/predict)

Let dynamics play a role in appreciation

Patterns in one's own states can be the objects of appreciation

The best way to make outputs in the real world is to be embodied in the real worldSlide6

Underlying architecture

CNM:Recurrent neural networkForward model of environmentLearns to anticipate/predict the sensory input it will receive if it performs a given action in a given contextIn conjunction with motivators can enable the robot to select actions that carry an expectation of "pleasure"Slide7

Underlying architecture

Key:

Recurrent Connection (Copy)

Full Inter-Connection Between Layers Of Units

Action

Expected

Sensations

Predicted

State

Previous Predicted

State

(Context Units)

D-map

T-mapSlide8

Generalisation to aesthetic experience

Perhaps the content of aesthetic experience can be understood using the same framework, with two extensions:Affective expectationsAesthetic expectationsSlide9

Aesthetic experience: affect

Just as a system can have expectations concerning its actions and resulting sensory inputs, it can also have expectations concerning its actions and resulting desirable or undesirable statesThus, affect, and in particular the affective character of aesthetic experience, may be accommodated in EBASlide10

Aesthetic experience: artifice

When we experience a visual work of art we do not just experience its visual properties (explicable in terms of expectations to receive input x if we move our eyes thus)We also experience it as a work of art: we possess expectations for how the work would change if we were to make (or the artist were to have made) this or that brushstrokeSet of all such expectations is the content of the aesthetic experience of the workSlide11

3. Synthetic phenomenology

A science of consciousness needs a way to refer to or specify the content of conscious experiencesStandard means: e.g., "Mary is having a visual experience of a red bike leaning against a white fence"Problem: Can only specify experiences with linguistic, conceptual, content

Yet several good reasons to believe that some of content of experience is non-conceptualSlide12

Do I have to draw you a picture?

An obvious alternative is to use non-linguistic, non-symbolic specificationsE.g., for the case of visual experiences, use imagesCan't just take a picture of the scene the subject is seeing (literalism)Even in the case of a robot model of experience,

can't just use the raw video camera outputFor example; the current "output" of a human retina contains gaps or

blindspots

that are not part of experience.

Furthermore, our visual experience, as opposed to our retinal output, at any given time is

stable

, encompassing

more

than the current region of foveation, and is

coloured

to the periphery

But what

alternatives

are there?Slide13

Depictive specifications of the content of visual experience

If the set of expectations determines experiential content, then displaying those expectations (in the right way) will count as a specification of that content"Filled-in" areas specify what input the robot would expect to receive if it moved its head so that it is looking in that location

Grey areas do not indicate an expectation to receive grey input; they indicate to you

the

absence

of any expectation for that location

"Absence of expectation is not an expectation of absence"

Alternative architectures (e.g., generalising neural networks) would have no such undefined regions of state spaceSlide14

DepictionsNeed not explicitly occur anywhere in the agent; are of experience corresponding to current

expected inputs, not actual inputsGenerated off-line by the theorist:Sample the action spaceFeed actions into forward model

Arrange the resulting expected sensory patterns in a spatial array according to the spatial relations of the sampled actionsTheorist's experience of the depiction (plus interpretation/bracketing instructions) is the same as (or shares crucial properties with) the experience to be specifiedSlide15

Depiction of anexpectational stateSlide16

Depictive specification and art

In a sense, content specification is what (at least some) artists have been trying to do for milleniaThus, a clear role for artistsgraphic designerssound engineersdirectors

etc.Slide17

Depictive specification and art

E.g., creating a film of a car chase in the desertNot just a matter of "objectively recording"Requires artistic insight into how film will affect viewers, e.g.:Pacing

timing, number, kinds of cuts and editswhat is in the framesteadiness

Cf Picasso's

portraitsSlide18

Problem: Subjectivity of Art?

Science strives to be objectiveBut content of a work of art is highly subjectiveYes, but artisitic insights can be objectively investigated; e.g.:

PerspectiveMoving images: “persistence of vision”Effect of colour on mood\

Et alSlide19

Problem: Subjectivity of Art?

Is subjectivity always at odds with science?Objective science is not the elimination of the scientist's subjectivityRather the negotiation of itPerhaps subjectivity of theorist can be

exploitedLessons from interactive Art?Slide20

A two-way interaction

Often, it has been thought that the interaction was one-way:cogntive scientists informing the work of designers, artists, and other "creative" typesBut the converse interaction is needed as well:Artistic input into radically different means of specifying the contents of experienceSlide21

4. Interactive empiricism

Some problems concerning (aesthetic) experience are philosophical; require conceptual breakthroughs/progressSuch breakthroughs (e.g. new concepts) may not be achievable by reason alone, but require experiential activity“…Modelling of consciousness… requires some clarifications and refinements of our concept of consciousness. Design of, construction of, and interaction with artificial systems can itself assist in this conceptual development.” (Sloman and Chrisley 2003)Sensory augmentation (e.g., work with Froese and Spiers on how using the “Enactive Torch” might alter concepts of perception)

Art works are also “artificial systems”: creation of and interaction withSlide22

5. Art works/installationsSlide23

5. Art works/installationsSlide24

Recap

Embodied (artistic) creativityEnactive models of (aesthetic) experienceSynthetic phenomenology: specifying the content of the sensory, affective and aesthetic components of experiencing art worksInteractive empiricism: changing our concepts of experience through creating or interacting with art

Art works/installations based on the aboveSlide25

5. Art works/installationsSlide26

Thank you.

Comments welcome:

ronc@sussex.ac.uk