ArtisticallySkilledEmbodiedAgentsPatrickTresset1OliverDeussen2Inthispa - PDF document

ArtisticallySkilledEmbodiedAgentsPatrickTresset1OliverDeussen2Inthispaperwereportonourresearchintothedevelop-mentofcomputationalembodiedsystemsdedicatedtotheproduc-tionofartworks.Wepresentaconceptualframeworkthatintroducesthenotionofstyle-spaceinrelationtothevisualarts.Thisframe-workunderpinsandguidesourgeneralapproachtothedevelopmentofautonomousagentscapableofproducingobjectsthathaveartisticvalue.Inadditionweintroducetheimportanceofintentionalityandembodiment,twointerrelatedelementsthatarefundamentalintheappreciationofvisualartworkssuchaspaintingsanddrawings.Our 1Goldsmiths,UniversityofLondon,UnitedKingdom,email:2UniversityofKonstanz,Germany,email:oliver.deussen@uni-konstanz.de 3IntheBeaux-Artsschools3to4years Figure1.PaintingsfromtheseriesPaul'sMemories,2013theartworkwasanimportantfeature.Later,inthe1980's,thead-ventofevolutionaryartwiththeworkofKarlSims[25]andWilliamLatham[28],andlaterworkslinkedtoarticiallifesuchasthoseofPenousalMachado[15],LeonelMoura[18]orRuiAntunes[1].Anotherbranchofcomputationalartisdrivenbyaninvestigationintohowanartistproducesartworks,withtwomainstrands:workdrivenbyindividualswhoareoriginallyartpractitioners:thepio-neeringworkofHaroldCohenwithAAron[17],inthelate1980'sEdBurton'sworkwithROSE[5],morerecentlywiththeworkofSteveDiPaola[8]andalsoourwork[31];workdrivenbyresearcherswhoareoriginallycomputationalscientists:suchasSimonColton'sworkonthePaintingFool[7],OliverDeussenandThomasLinde-meier'sworkone-David[14].Thecomputerartmovementisarguablyasignicantartstrandthathassincethebeginningattractedcollectors,heldimportantfoundingexhibitions,andisnowthesubjectofarthistorypublications.WeseededicatedcollectionsinmajormuseumssuchastheVictoriaandAlbertmuseumandyetstillcomputerartisapartfromacademiccontemporaryarthistory.Thismightnaturallyevolveasthedigitalnativestakeprogressivelymoreimportanceintheart-world.3Art,style,intentionalityandembodiment3.1Artandstyle-spacesFrameworkssuchastheonedescribedin[32]whichformaliseanddetailBoden'sviewsoncreativity[2],offeraperspectivewherecre-ativityisconsideredrelatedtoaconceptualspace,thesetofallcon-cepts,anotionthatcoverseverythingthatistheresultofcreativity,concreteorabstract.ToformaliseourperspectiveonvisualartweconsiderwhatwenameassubspacesoftheWorksofartdisplayandhaveembeddedinthemanensembleofcharacteristics.Thecombination,theexpressionofthesecharacter-isticsiswhatwecallstyle.Greatandminorartistsproduceartworksthathaveanindividualidentiablestyle.Stylesarenotisolatedorautonomous,theycontaintracesofinuencesfrompastandcon-temporaryarthistory.Theseinuencesarenotexpresseddirectlyasakindofpatchworkorcollage,butrathertheyarecombinedandinacertainmanner“digested”bytheartist.Artistsmakethesein-uencestheirown,transformedbytheir“personality”orwhatwe 4Inthemodernistperiod:thesuprematists,DeStiljltandincontemporaryart:minimalism,conceptual,op-art,processart.Motivationsfortheremovalofthehumanaremultiplesuchasspiritual,purity,theremovalofsentimen-talism,anti-individualism,artforartandpoliticalmotivation.5Peoplebornafterthedemocratisationofthepersonalcomputercouldconsidertheartistscharateristics,suchastheirstrengthsandweaknesses(psychological,motor,perceptual,cognitive).Thispro-cessenablestheartworkproducedtodisplayrichcoherentindividualstyle,inwhicharthistoryandcontextualinuencesareembedded.Thisfacilitatestheperceptionandappreciationofartworksasworksofart.Wecandescribetheartist'smajorcreativeeventastheestab-lishmentoftheirnovel–original–personalstyle-space.Typically,thistypeofcreativeeventoccursalimitedamountoftimesinanartistscareer.Generally,therstofsucheventsoccursonlyafteryearsoflearning,practiceandexploration.Forexample,anartistsuchasPabloPicassowhohasthereputationofbeinghighlycreative,had6distinctmajorcreativeeventsoveracareerthatspanned70yearswhilstanartistsuchasFrancisBaconhadonlyonesignicantsuchevent.Thefound–developed–createdartist-style-spacecanbeenseenasasubspaceofamovement-style-space,itselfasubspaceoftheperiod-style-space,whichisasubspaceoftheart-space.Theartist-style-spacecontainssubspacesorregionsrelatedtoperiods,thatincludesubspacesrelatedtoserieswhicharepopulatedbyindividualart-works.Eachstyle-spaceandsubspaceischaracterisedbyanum-berofdimensions,andgovernedbyspecicrulesthatdeneele-mentssuchasconceptualstances,colourharmonies,typeofcom-position.Associatedtoeachofthesestyle-spacesandsubspacesarestrategiesandtechniquesusedforthedevelopmentandproductionofartworks.Meta-rules,strategiesandtechniquesassociatedtopar-entspaceswouldalsoapplytosubspaces.Fromthisperspective,art-workscanbeseenasconstructedsystemsthatarefunctionalintheArtistsdevelop–create–inventtheirpersonal–original–identiablestyle-space,thesestyle-spaceshavecommonalitieswithexistingstyle-spaces,thisoccursthroughtheunderstandingofrules,tech-niquesandstrategiesrelatedtostyle-spacesestablishedbyotherartists,achievedforexamplebycopyingexistingartworks.Thesesetsofinheritedstrategies,rulesandknowledge(schemata)aredeveloped–adapted–personalisedthroughpractice,explorationandexperimentation.Asinotherdomainsofexpertise,schematacanbedescribedasthesetsofknowledgeandstrategiesfor:action,informa-tiongathering,evaluation,planningrequiredtoachieveatask,withtheschemataforhigh-leveltasksrelyingonlower-levelschemata.Worksofartcanbeconsideredassystemswithavisualappear-ancethathasbeeninvented–developed–executedtoactontheob- 6Thenumbervariesdependingonthehistoriansandsourcesbutinourview:BluePeriod,PinkPeriod,AfricanPeriod,Cubism,Surrealism,Classicism server'sperceptual/cognitivesystemtoproduceacertainaesthetic–artistic–emotionalexperience.Wecanconsidertheestablishmentofastyle-space(artist,period,series)astheresultoftransformatorycreativity,andthedevelopment/executionofartworksastheresultofexploratorycreativity[2].Althoughlevelofcreativityisgenerallyconsideredameasureofartisticvalue,itisdifculttoexamineitwithoutconsideringmasteryandexpertise,andinthecontextoftheartstheseelementsareinti-matelylinked,onebeinguselesswithouttheother.Withoutalevelofexpertise,majorcreativeeventsinanartistscareerwouldbeunlikelytooccurorberecognisedassuchbytheartist,andwithoutmastery,style-spaceexplorationwouldnotbepossible.Thelooseformalisationdescribedaboveenablestheattributionoflevelsofcreative/artisticautonomytocomputationalsystemsthatproduceartworks.Thehighestlevelofautonomybeingattributedtosystemsthatarecapableofcreating–developing–ndingtheirownstyle-space,withthelevelofautonomydecreasingwhenthesystemisonlycapableofestablishingperiod-style-space,decreasingfurtherstillfortheabilitytoestablishseries-style-space,whichisconsideredatahigherlevelofartisticautonomythantheproductionofindivid-ualartworks.Theestablishmentandexistenceoftheserichcoherentstyle-spacesisessentialfortheconsiderationoftheproducedartifactsasworksofart,astheyproviderecoverabletracesofhistoricalcon-textandartisticcoherence.Developingsystemsthatonlymimicthesupercialappearanceofartworks,wouldbelikelookingatamathe-matician'sblackboardandbasedonlyonsupercialobservation,de-velopingasystemtoproduceimagesusingthesamesignsarrangedinthesamemanner,andthenexpectingthisimagetoexpressamath-ematicaltruth.Thesystemsdescribedinthispaperdisplayonlythelowestlevelofartisticautonomy,thatistosaythesystemsarenotyetabletodeveloptheirownstyle-spaces.Asartist–researchers–developerswehavebeenresponsiblefortheestablishmentoftheartist,periodandseriesstyle-spaces.Atpresenttheagentsareresponsibleforthecre-ationofartworksthatarefunctionalinthesestyle-spaces.Itisimpor-tanttonotethatthestylisticappearanceoftheartworksproducedisnotthatofapastiche.Theartworksareinterpretationsnaturallyin-uencedbytheagent'scharacteristicsandcapabilities(physicaland3.2IntentionalityAlthoughitisdifculttodenewhatartis,wecandenetheartistasanexpertwhoconceivesandproducesartifactsthatareexhibitedingalleries,museumsandotherpublicspaces,withtheseartworksbeingexhibited,appreciated,acquiredandcollectedbyactorsorin-stitutionsbelongingtotheart-worldOurgoalisnotforthesystemstomimicpreciselytheappear-anceofhuman-madeartworkssoastocreatepastiches,buttopro-duceartworksthattriggersimilaraestheticemotionsintheviewerasahuman-madedrawingsorpaintings.Withthisinmindweshouldidentifyanddenearangeofcharacteristicsthatenablethepercep-tionandcategorisationofhuman-madeobjectsasbeingworksofartbytheart-world.Therearetwoperspectivesonvisualartsperception/appreciation: 7Inthesamemannerasscientist'sresearchpublicationsarereviewedbyspe-cialists/expertswebelievethatitshouldbethesameforart.Expertsintheartsinclude:curators,collectors,artcritics,amateurs,patrons,arthistoriansandpractitioners.Inthispaperthetermart-worldcanbedenedasarangeofpeoplethatareabletoformaninformedopinionaboutartworks. Figure2.PaulsketchingAmy,2012,photobyStephHorakthecognitiveapproachthatgenerallypostulatesthatartworkshaveintrinsicartisticvaluewhichcanbeunderstoodthroughstudyingtheeffectoftheappreciationofartworksattheneurologicallevel;thehumanitiesapproachthatarguesthatanartwork'sappreciationisstronglyinuencedbycontextualknowledgerelatedtoitsproduc-Inthecognitiveapproach,scientistsfromempiricalaesthetics[21,16,24],neuroaesthetics[6,33,26]usetoolsandmethodsfrompsychologyandneurosciencestofurthertheunderstandingofthementalandneuralprocessesinvolvedinartappreciation.Thisfo-cusonperceptionandobservablecognitiveprocessesdoesnottakeintoaccountthehumanitiesapproachandevenoftenrejectsit,look-ingforartisticuniversalsthataresufcientinaccountingforartisticvalue.Thehumanitiesapproach,withoutrejectingtheintrinsicaestheticvalueofartworks,considerstheroleofcontext(artistic,historical,sociological),aswellastheartist'sintentionsasbothhavinganeffectontheproductionandappreciationofartifactsasworksofart.Inthecontextofcomputationally-producedartworks,itiseasiertotakeintoaccountthecognitiveapproachratherthanthehumani-tiesapproach,thereforedevelopingsystemsthatproduceimagesthatsuperciallylooklikeartworks,andfocusingonthisaim.Butcon-sideringthatactorsoftheart-worldinthemajoritytakethehuman-itiesperspective,wehavetoacceptthisframeworkifwewanttheartifactsproducedbyoursystemstobeappreciatedasworkofarts.Asaconsequencewemustconsiderthatintention-richartifactsthatcanbeassociatedtocoherentartisticandhistoricalcontextsaremorelikelytobeconsideredworksofart.Pignocchiin[23]uniesthecognitiveandhumanitiesapproachesusingamodeloftheexperienceofartworksbasedonthemecha-nismsofintentionattribution.Pignocchiclaimsthatthetraditionalnotionof“intention”istoorestrictivewhenconsideredinthehu-manitiesapproach,andarguesthatanobservercannotonlyre-covertheartist'shigh-levelovertintentions,butalso“allthementalstates—consciousornot,propositionalornot—thathaveplayedacausalroleduringtheproductionofthework”.Pignocchi'sframe-workpostulatesthat:“theperceptionofanartworkactivatesrichintentionat- tributions,implicitandexplicit,consciousandunconscious,wheretheintentionsattributedarepotentiallyanykindofmen-talstatesthatcouldhaveplayedacausalroleduringthepro-ductionofthework.”Anartworkcontainsthehistoryoftheintentionsthatledtoitscre-ation,andsomeofthishistoryisrecoverablebytheobserverandcontributestotheartwork'sappreciation.Atlowlevel,thebrush-strokes,marksorpen'stracesonthepaper,theiroverlappingandinteractions,areadirectmemoryoftheartist'sdecisionandactionsovertimeandarerecoverablebytheobserver.BasedonPignocchi'smodelwecanhypothesisethatwhenpro-ducedusingtherulesandstrategiesrelatedtocoherentstyles-spaces,tracesandcuesareembeddedintheartworksthatenabletheirasso-ciationtorichandcomplexartisticandhistoricalcontexts.3.3EmbodimentThereareanumberoffundamentaldifferencesbetweenacomputa-tionalsystemdesignedtoproduceimagesthatlooksuperciallylikedrawingsorpaintingsandanembodiedcomputationalsystemthatphysicallyproducespaintingsordrawings.Thesedifferencesarees-sentiallydueto:thecharacteristicsoftherobotandthecharacter-isticsofthemedium;thephysicalityoftheproducedoutput;perceivedagencyofthesystem.Thecharacteristicsoftherobot,themovementsandtasksitcanachieve,arehighlyconstrainedandrequirecarefulconsiderationandcomplexcontrolinanembodiedsystem.Inthecaseofpainting:thecontrolofthepressureofthebrushontothecanvastheangleatwhichthebrushtouchesthecanvasthepathofthebrushstrokethevelocityofthemovementscontrollingthequantityofpaintloadedontothebrushthecontrolofthepaintviscosity,texture,transparencymixingpaintstoobtainspeciccolourthetimeittakestomovefromoneareatoanotherareathetimeittakestoreloadthepainttimetakentocleanthebrushesthenumberofbrushesitcanusethecharacteristicsofpaintsandbrushestherangeofpigmentsavailableTheseelementsimposestrongconstraintsoraretoocomplextocon-trolwhendevelopinganembodiedsystemthatactsintherealworld.Inapurelycomputationalsystem,theyarecontrolledeffortlesslyandwithoutconstraints,bysimplychangingsomeparameters.Inthissituationonecouldthinkthatitismorelogicaltodevelopanon-embodiedsystemcapableofproducingworksofart,butthisisnotthecase.Thecomplexityandlimitsencounteredwhendevelopinganembodiedsystemforcesthedevelopertosimplifyandadapttotheconstraints,andasaconsequencebringaformofstylisationthatdenesandcharateriseswithprecisionalocalstyle-space,basedonboththerobotsandthemediums'characteristics.Withregardstothephysicalityoftheoutput,paintingsanddraw-ingsarephysicalobjectsthathavespecicqualitiesthatcanonlybeappreciatedwhenintheirpresence.Artworksareobjectswithaes-theticqualitiesthatarevalued,notonlyforwhattheyrepresentandhowtheydepictbutalsofortheirmaterialpropertiessuchasscale,textures,howthesurfacesreectlight.Reproductionsinbooksorprintedpostersinnowayenabletheexperienceofappreciatingtheartworkindirectphysicalcontact;adigitalprintofanimagepro-ducedbyanon-embodiedsystem,evenofthebestpossiblequality,isnotcapableofencapsulatingtheaestheticrichnessofadrawingorpainting.Moreprosaically,apaintingoradrawinghasmorevaluethanaprintcommercially,duetoitsuniquenessanditsmaterialqual-ities.Furthermore,physicallyproducedobjectsencapsulaterecover-ablememoriesofthesuccessionofprocessesthathavecreatedthem[22,13,9],enablinginturnaperceptionofthesuccessionoftheartist'sintentionsovertime.Althoughnotoftenconsideredorknownbythepublicatlarge,mostartistsofacertainstatushad/haveassistantsandcollaboratorswhotakechargeofsomeofthetasksinvolvedintheproductionandcommercialisationofartworks.Inthepast(pre-nineteenthcen-tury)youngartistswouldlearntheircraftandacquirethenecessaryskillsbyworkingforMasters,someofthembecomingMastersintheirownrights,otherspursuingrolesasworkersintheMaster'sstudio/workshop.Inregardstoartist'sassistants,thecontemporarysituationisnottotallydissimilarinthatartstudentsandyoungartistsoftenworkforpracticingartistsasawaytoearnanincomeandlearncertainaspectsoftheirprofession.Thisuseofassistantstoaidtheconceptionandfabricationofartworksdoesnotaffecttheworksca-pabilityofbeingperceivedandconsideredasart.Withthisinmind,theuseofcomputationalsystemsshouldnotpreventtheirproductionbeingconsideredasartaslongastheyareperceivedassuchbytheAsfortheperceivedagencyofthesystem,studieshaveshownthathumanshaveatendencytoconsiderrobotsassocialagents,ashavingpersonalitiesandagency[3,4].Itislikelythatwiththeknowledgethatanartworkhasbeenproducedbyarobot,theobserverwillbeabletoconsidertherobotastheoriginatoroftheintentionsthatledtotheartworks'creation,astheartist. Figure3.SketchbyPaul,2011 4PaulThissectiongivesabriefdescriptionofarobotnamedPaul,anart-workandobsessivearticialdrawingentitythatwascreatedusingtechnologiesandideasdevelopedinthecontextofAIkonII,aprojectconductedatGoldsmiths,UniversityofLondonbytherstauthorincollaborationwithProf.FredericFolLeymarie.Adetaileddescrip-tionofPaulandthealgorithmsdrivingitcanbefoundin[31].TheAIkonIIsystemsweinitiallydevelopedtoinvestigatetheob-servationaldrawingpractice,werenotembodiedinarobot.Yetitrapidlybecameapparentthatembodimentwasnecessary.Thereareanumberoffactorsthatmotivatethisdecision:thephysicalityofdrawing,havingasystemthatdrawsbyphysicallymovingapenincontactwithamedium:wehaveseenpreviouslythatthewayartworksaremadehasastronginuenceonthewaytheyareper-ceived;themannerinwhichrobotsarecontrolled:contemporarysoftwareframeworkarchitecturetocontrolrobotsiscomposedofconcurrentlyrunningprocessesthatcommunicate.Thisarchitectureforcesonetoapproachproblemsfromananglewhichismoreappro-priateforunderstandinghowtoimplementcomplexsystems,suchasdrawingfromobservation;theadvantagesfordissemination:theappealthatrobotshaveonanaudienceattractsinterestandattention.Thestrategies,techniquesandaestheticsimplementedinPaularederivedandinaccordwiththosecharacterisingTresset'sdrawing-style-space.ThisspacewasdevelopedandinuencedbyyearsofpracticeandbystudyinginparticularthedrawingsofLeonardoDaVinci,EgonSchieleandAlbertoGiacometti.Paulisdesignedtolimitcomplexity,andonlytofulllitsfunction,drawing.Assuchitiscomposedofathree-jointedplanararmwithanextrajointtoraiseandlowerthepenincontactwiththepaperandanactuatedpanandtiltcamerausedasitseye.Botharmandeyeareboltedtoasingleschooldeskwherethedrawingpaperisattachedusingpins(g.2). Figure4.Paintingfromtheseries:Paul'sMemories,2013InourexperienceofexhibitingPaul'sproductiontoawideaudi-ence,wehavenoticedthatoneoftheinterestingpropertiesoftheproducedportraitsisthattheyareperceived,consideredandappre-ciatedasdrawings.Whenobservingaseries,Paul'sdrawingsarerecognisedasdrawnbythesameauthormeaningthattheydisplayanautographicstyle.ContrarytoothercomputationalsystemsthatproducedrawingsfromphotographssuchasAIkon-I[30,29],draw-ingsproducedbyPauldonotdisplaythesameserialuniformityoftreatment(g.5). Figure5.SketchesbyPaul,2011AnumberoffactorscanaccountfortheperceivedqualityofPaul'ssketches,suchasthechoiceofpaper,layoutandcomposition.WhenPauldrawslines,theirpathsareextractedfromtheresponsesofGa-borlters[20].Suchltersareknowntobegoodmodelsofsimplecellsintheearlyvisualcortex(V1)[12]andasaconsequenceac-centuatewhatwouldbeperceivedassalientfeatures[11].Theuseofsimulatedvisualfeedbacktoconstrainandevaluatetherandomex-plorationduringtheshadingprocessissufcienttoproducepatternsthatareperceivedasbeingtheresultofanintentionalprocess.Paul'sdrawingsaretheresultofasequenceofmovementsandassuchtheyaretherecordofaprocess.EvidencethatthetracesformingpartofadrawingbyPaularetheresultsofmovementscanbefoundinthelinesirregularities.Althoughtheseirregularitiesarenotakintotheimperfectionsahumanmightproduce,theyhavecharacteristicsthatcouldonlybetheresultofapeninmotiondrivenbyanarticulatedarm.Furthermore,thelayeringofsuccessivelinesandofsuccessiveshadingpatternsaddstothedrawingbeingperceivedastheresultorconsequenceofasequenceofintentionalmovements/processesPaulanditsdrawingshavebeenwidelyexhibitedincludinginma-jorartmuseums.Since2011,thePaulshavedrawnthousandsofpeopleandhundredsofdrawingshavebeenacquiredbythepublicingalleries,museumsandart-fairs.OneofPaul'sdrawingsisintheVictoriaandAlbertMuseum'scollections.5PaintingPaul'sMemoriesThissectionreportsontherstauthor'sresearchandtheartisticout-comeofa9-monthresidencywiththeresearchteaminKonstanzled 8TherearetodatetenPaulsinexistence. byProf.OliverDeussen.TheaimoftheresidencywasforTressettoexperimentwiththerobot[14]anddevelopalgorithmstodriveitwiththeviewofcreatingseriesofpaintings.TheaimsoftheteaminKonstanzforthee-Davidsystemare:ingthemanualpaintingprocessesbyamachine;ndingouttowhatextentthesystemisabletoproduceartisticlookingpaintings;lookingfornewformsofvisualrepresentationthatareespeciallysuitedtopaintingmachines;ndingouthowtointroducehigh-levelsemanticinformationintotheprocess.Inrecentyears,methodsforimageunderstandinghavedevelopedalot,soitisplausiblethatpaintingmachinesofthefuturecould`know'whattheydrawandautomaticallyadapttheirpaintingstrat-egy[14].Inthesubsection5.1e-David'stechnicalsetupisdescribed,followedinsubsection5.2byadescriptionoftheideasandthetech-niquesusedtopaintthePaul'sMemories5.1e-David Figure6.e-Davide-Davidconsistsofanindustrialrobot(ReisRV20-6)equippedwithaspecializedpickingdeviceforgraspinguptovedifferentbrushesthatcanbeusedduringthepaintingprocess(g.6).Paintisstoredinarepository;sofarthesystemallowsfortheuseofupto24differentcolors.Therobotaccessesapaintcontainerbymechani-callyopeningthecoverplateanddippingthebrushintothecolor.Thesetupisamendedbyacleaninganddryingfacility.Togetherwiththebrushgraspingtool,adistancesensorismountedontherobotarmwhichallowsthesystemtocompensateforcurvedcanvasesandeventhemechanicaltolerancesoftherobotwhilemovingthebrushoverthecanvas.Suchtolerancesareinthesub-millimeterrangebutstillvisibleforimagestyleswithlongstrokesandnebrushes.ThevisualfeedbackofthesystemiscreatedusingaCanonEOS5DMarkIISLRcamerawitha21MPixelsensorandaxed50mmlens.Thisprovidesaresolutionofabout1mmonthecanvas.Twospecializeduorescenttubeswithpolarisationltersilluminatethecanvas,andthecameraalsohasapolarisationlter.Therobotiscontrolledbyanassembler-likelanguage,andaserverapplicationthatcontrolsthemachineandacceptsXMLcommandswasbuilttoeaseitsuse.Mostcommandsareplotter-likeinstructionssuchaspenselectionanddrawing,butspecializedcommandsformeasuringthecanvasandthehandlingofbrushesandcolorshavealsobeenimplemented.Asecondserverisusedforthecamera.Thecameraallowsforthecontrolofallnecessaryfunctionsfromthecomputer.ImagesarecreatedinXYZcolorspaceandarecalibratedusinggeometriccalibrationpatternsandcolorsheets.Thegeometriccalibrationiswithintherangeofasinglepixel,andtodatethecolorcalibrationisintherangeof5%,whichissufcientforcurrentappli-cationsbutmustbeenhancedinthefuture.Therationaleofbuildinganindustrialrobotratherthanjustusingapenplotter,wastoremovethelimitationoftheuseofonlyonetypeofphysicalinstrumentwhencreatingpaintingsinthismanner.Thecurrentimplementationofe-Davidcansuccessfullyuseanybrushorpencil.Itcanpaintwithanumberofpaintingmediumssuchasink,acrylicandoilpaints.Theinaccuraciesofbrush-strokeren-deringandtheunpredictabilityofcolorinteractionsonacanvasarereducedwiththeuseofthefeedbackloop.Testshaveshownthatthisbehaviorhasprovedtobeveryimportantsinceitnotonlyallowstheuseofrelativelysimplesimulationtechniques,butcanalsoeasilybeadaptedtodifferentstrategiesforplacingstrokes.5.2Paul'sMemoriesTheaimoftheresearchandworkwithe-Davidconducteddur-ingtherstauthor'sresidencywastoproduceseriesofpaintingsthatcouldpotentiallybeappreciatedasworkofarts.Asourap-proachpostulates,thereisaneedforartifactstobelocatedinaseries-style-space,period-style-spaceandtheindividual/originalartist-style-space,andatpresentitisnotpossibleforoursystemstodeveloptheirownstyle-spaces.Asaconsequenceitisnecessarythattheresearcher/developertakesresponsibilityforestablishingtherules,parameters,strategiesandtechniquesrelatedtothesespacesinorderfortheproducedartifactstobeperceivedasworksofart.Whenartistsareintheresearch-experimentationphasethatwillleadtotheestablishmentoftheartist-style-space,anumberofpos-sibilitiesareexplored,techniquesdevelopedtobelaterabandonedduetovariousfactorssuchastechnicallimitationormorepromisingavenuesbeingfound.Fortheworkdevelopedhere,suchanaban-donedspaceisexploredandreused.Afteracoupleofyearsofexplo-rationthispotentialstyle-spacewasabandonedessentiallyduetotheartist'sskillslimitations.Thestyle-spaceexploredherewasestablishedastheresultofex-perimentationsbytherstauthorwithtechniquesandstylesofpre-impressionistperiods,combiningtheuseofchiaroscuro,impastoandglazes.ThisresearchwasaimedatndingwaystodepicthumanfacesandwasinuencedbyRembrandt,FranciscoGoyaandalsobytwentiethcenturypainterssuchasLeonKossoff,HowardHodgkinandYanPei-Ming.Thestyle-spaceisdenedandcharacterisedby:alightingakintochiaroscuro,achievedbyrelativelyfewthick,well-denedbrush-strokes(impasto),withtonalvariationsachievedfromasuccessionofthinmonochromaticuniformtransparentlayers(glazes).Theseries-style-spaceestablishedforPaul'sMemoriesseriesisdenedandcharacterisedby:astylisationreminiscentofearlycom-putationalimagesachievedbyusingonlystraightbrushstrokes,withalimitednumberoforientations,alimiteddiscretisedtonalrange(8levels),andthreedistinctbrushstrokewidths.Thechoiceofthese-riessubjectmatter:theindividualsportrayedinthisseriesaretakenfromPaul'smemories(Paulstoresadigitalimageofeachpersonithasdrawn). 5.2.1ThepaintingprocessWeprovidehereabriefandsupercialdescriptionofthepaintingprocessandofthealgorithmsused.Therobotonlypaintswiththickwhiteacrylicpaintonapre-preparedcanvasboardofamid-greycolour,withthreedifferentsizedbrushes.Ninepaintingsarepaintedatatime,arrangedona3x3grid.Thefullpaintingprocessforeachsettakesbetween24and36hoursTheuniformmonochromeglazesareappliedbyaskilledhumanoperator.Althoughthisoperationisseeminglysimplewhenexe-cutedbyahumanwithabrush,itrequiresconstantadaptationofspeedandpressure,andremovalofexcesspaintbasedonvisualfeedback.Thisadaptationisnecessarytoachieveextremelythinanduniformlayers.Itwouldhavebeenacomplexresearchprojectinitselftogetarobottoperformthisphaseofthepaintingpro-cess.Itcouldperhapshavebeenachievedautomaticallybyhavingtherobotspraytheglazes,butwedidnothavetheresources,com-petencesandtimetodevelopandimplementthistechnology.Fur-thermore,thedelegationofanon-creativerepetitivelow-leveltasktoahumanaddsaninterestingelementtothecontextinwhichthepaintingswerecreated.Ateachiterationtheglazes'colourationalternatesbetweenadarkbluishpurpleandadarkreddishpurple.Ateachiterationtheglazes'transparencyisincreasedby50%.Thereare8cycles,oneforeachofthetonalvalues.Eachcycle:therobottakesapictureofthecanvas(PCI)brushstrokes,length,orientationsandlocationsarecomputedtherobotpaintsasetofwhitebrushstrokeswithalargebrushtherobottakesapictureofthecanvas(PCL)thebrushstrokes'length,orientationsandlocationsarecom-therobotpaintsasetofwhitebrushstrokeswithamediumtherobottakesapictureofthecanvas(PCM)thebrushstrokes',length,orientationsandlocationsarecom-therobotpaintsasetofwhitebrushstrokeswithasmallbrushwhenthepaintisdrythehumanoperatorappliesaglazeForeachcyclealevelofthegraylevelimage,discretisedusingak-meansclusteringalgorithm,isusedasabinarymap(GM)Foreachofthebrushsizes,abrushstrokemap(BM)iscom-puted.BMisalow-resolutiontexture-owmapofthesubjectimage,theresolutionofeachmapbeingrelatedtothebrushsize(largerbrushsize,lowerresolution).Theorientationofeachcellofthemapislimitedto8anglesForthelargerbrushstrokes,GMisusedasamasktoselectfromBM,thesetofbrushstrokestobepainted.Toselectthemediumandsmallbrushstrokesset,usingPCIandPCLorPCMasvi-sualfeedback,anewbinarymapiscomputedthatrepresentswhichareaofGMhasnotyetbeenpaintedThepaintingsproducedinthePaul'sMemoriesseriesarearguablyinacoherentidentiablepersonalstyle,andhavebeenevaluatedby 95setswerepainted,2ofthemwerefailuresanumberofartists,acuratorfromamajorcontemporaryartmuseumandotheramateursasbeingofartisticinterest(g.1,4).Fromdis-cussionswithpeoplewhohaveseenthepaintingsitisclearthattheyareevaluatedwiththesamestandardsasiftheywerepaintedbyahu-manartist.Althoughthesepaintingsdonotresembleworkwehavepaintedourselves,wewouldbeverysatisedtoachievesuchwork.6ConclusionsInthispaperwehavedescribedviews,considerationsandstrategiesthatshapeandmotivateourdevelopmentofartistically-skilledem-bodiedagentsusedtoproduceartworks.Thenumerousdiscussionswehavehadwithactorsoftheart-worldandtheart-lovingpublic,aroundthedrawingsbyPaulandthePaul'sMemoriesseriesofpaint-ings,haveconvincedusthattheseartifactsareconsideredasworksofart.Thequestionofevaluatingthelevelofcreativityofacomputa-tionalsystemisnotasimpleone.Ifweconsidertheperceivedcre-ativityinbothofthesystemsthatwehavepresentedinthispaper,theyarebothperceivedasbeingcreativeandartisticbythepublicatlargeandartspecialists.Evenwe,thecreatorsofthesystems'areveryoftensurprisedbytheartifactsproducedbyPaulande-David.Thereisaleveloforiginalityineachartifactandyettheycanbeperceivedasdisplayinganautographicstyle.Weareawarethatevenwiththeuseofvisualfeedbackandotherlow-levelevalua-tions,bothsystemscannotatpresentbeevaluatedasbeingcreative.Althoughshapedbyourpersonalstyle-spaces,thesystemshavedif-ferentstrengthsandlimitationstoourownandasaconsequencetheworktheyproducedoesnotbelongtous.Itisaninterpretationofourstyleshapedbytheircharacteristics,andforanumberofreasons,ob-jectiveorsubjective,weevaluatetheirartworksasmoreinterestingandartisticallyvaluablethanwhatwehaveproducedbyhand.Nowthatwehavedevelopedskilledagents,thenextstageswillaimatprogressivelyincreasingthelevelofautonomouscreativitywithoutdecreasingthequalityoftheartworksproduced.Theframe-workpresentedinthispaperencouragesadevelopmentapproachfromthegroundup.Inthecontextofdrawingandpainting,thelow-estlevelbehaviorismark-making.Thenextstepinourresearchistousereinforcementlearning(RL)toallowtherobottolearntogainmorecontrolwhentracinglineswithoutlosingitsapparentspontane-ityandwithoutremovingtracesofitsintrinsiccharacteristics.RLisatechniqueusingtrials,rewardandpunishmenttolearn,ortomodifybehaviours.Asweprogressinenablingthesystemstogainmorecreativeau-tonomybydevelopingtheirownstyle-spaces,wewillhavetocon-siderthedifcultissueofthesystem'sabilitytoassessthequalityofitsownwork.TheobjectiveisforthePaulande-Davidsystemstobeabletobeinuencedbytheirartisticenvironment,arthistory,con-temporaryartsandtheirownpractice,toinventtheirownstyle-spacewhereourinuenceisonlyoneamongstmanyothers.ACKNOWLEDGEMENTSPaulisbasedontechnologiesdevelopedinthecontextoftheAIkon-IIprojectwhichwasco-directedwithProf.FredericFolLeymarieandwasinpartsupportedbyaLeverhulmeTrustgrant.PatrickTres-set'sninemonthresidencyintheinformatikdepartmentoftheUni-versityofKonstanzwasnancedthroughaseniorfellowshipattheUniversity'sZukunftskolleg. [1]R.F.AntunesandF.FolLeymarie,`xtnz-anevolutionarythree-dimensionalecosystem',inProceedingsofthe4thInternationalCon-ferenceonDigitalArts,Artech,pp.201–204,(2008).[2]M.A.Boden,Thecreativemind:Mythsandmechanisms,Routledge,[3]C.BreazealandB.Scassellati,`Howtobuildrobotsthatmakefriendsandinuencepeople',inIntelligentRobotsandSystems,1999.IROS'99.Proceedings.1999IEEE/RSJInternationalConferenceonvolume2,pp.858–863.IEEE,(1999).[4]C.L.Breazeal,DesigningSociableRobotswithCDROM,MITpress,[5]E.Burton,`Thoughtfuldrawings:Acomputationalmodelofthecogni-tivenatureofchildren'sdrawing',inComputerGraphicsForum,vol-ume14,pp.159–170.WileyOnlineLibrary,(1995).[6]A.Chatterjee,`Neuroaesthetics:acomingofagestory',JournalofCog-nitiveNeuroscience(1),53–62,(2011).[7]S.Colton,`Thepaintingfool:Storiesfrombuildinganautomatedpainter',inComputersandcreativity,3–38,Springer,(2012).[8]S.DiPaola,`Exploringaparameterisedportraitpaintingspace',Inter-nationalJournalofArtsandTechnology(1),82–93,(2009).[9]D.FreedbergandV.Gallese,`Motion,emotionandempathyinestheticexperience',Trendsincognitivesciences(5),197–203,(2007).[10]E.J.Ilfeld,`Contemporaryartandcybernetics:Wavesofcyberneticdis-coursewithinconceptual,videoandnewmediaart',Leonardo57–63,(2012).[11]L.Itti,C.Koch,andE.Niebur,`Amodelofsaliency-basedvisualat-tentionforrapidsceneanalysis',PatternAnalysisandMachineIntelli-gence,IEEETransactionson(11),1254–1259,(1998).[12]J.P.JonesandL.A.Palmer,`Anevaluationofthetwo-dimensionalga-borltermodelofsimplereceptiveeldsincatstriatecortex',JournalofNeurophysiology(6),1233–1258,(1987).[13]M.Leyton,Thestructureofpaintings,Springer,2006.[14]T.Lindemeier,S.Pirk,andO.Deussen,`Imagestylizationwithapaint-ingmachineusingsemantichints',Computers&Graphics,(2013).[15]P.MachadoandA.Cardoso,`Allthetruthaboutnevar',AppliedIntel-ligence(2),101–118,(2002).[16]C.Martindale,Theclockworkmuse:Thepredictabilityofartisticchange.,BasicBooks,1990.[17]P.McCorduck,Aaron'scode,WHFreeman&Co.,1991.[18]L.Moura,`Anewkindofart:Theroboticactionpainter',XGenerativeArtConferece,PolitecnicodiMilanoUniversity,(2007).[19]A.Pease,D.Winterstein,andS.Colton,`Evaluatingmachinecreativ-ity',inWorkshoponCreativeSystems,4thInternationalConferenceonCaseBasedReasoning,pp.129–137,(2001).[20]N.Petkov,`Biologicallymotivatedcomputationallyintensiveap-proachestoimagepatternrecognition',FutureGenerationComputer(4),451–465,(1995).[21]R.W.Pickford,`Psychologyandvisualaesthetics.',(1972).[22]A.Pignocchi,`Howtheintentionsofthedraftsmanshapeperceptionofadrawing',Consciousnessandcognition(4),887–898,(2010).[23]A.Pignocchi,`Historyandintentionsintheexperienceofartworks',Topoi,1–10,(2012).[24]A.P.ShimamuraandS.E.Palmer,Aestheticscience:Connectingminds,brains,andexperience,OxfordUniversityPress,2012.[25]K.Sims,`Evolving3dmorphologyandbehaviorbycompetition',Ar-ticiallife(4),353–372,(1994).[26]R.L.Solso,Cognitionandthevisualarts,MITpress,1996.[27]A.M.Spalter,Thecomputerinthevisualarts,Addison-WesleyLong-manPublishingCo.,Inc.,1998.[28]S.ToddandW.Latham,`Evolutionaryartandcomputers',(1994).[29]P.TressetandF.FolLeymarie,`Generativeportraitsketching',inPro-ceedingsofVSMM,ed.,HalTwaites,11thInternationalConferenceonVirtualSystemsandMultiMedia,pp.739–748,Ghent,Belgium,(2005).Archaeolinguapublishers.[30]P.TressetandF.FolLeymarie,`Aikon:theartistic/automaticikono-graph',inACMSIGGRAPH2006Researchposters,p.37.ACM,[31]P.TressetandF.FolLeymarie,`Portraitdrawingbypaultherobot',Computers&Graphics,(2013).[32]G.A.Wiggins,`Apreliminaryframeworkfordescription,analysisandcomparisonofcreativesystems',Knowledge-BasedSystems449–458,(2006).[33]S.Zeki,`Artandthebrain',JournalofConsciousnessStudies6–7,(1999).