SIGGRAPHCourse2014—Skinning:Real-timeShapeDeformationPartI:Direct - PDF document

SIGGRAPHCourse2014—Skinning:Real-timeShapeDeformationPartI:DirectSkinningMethodsandDeformationPrimitivesLadislavKavan EE,G.S.,LIN,A.,SCHILLER,M.,PETERS,S.,MAUGHLIN,M.,ANDANNER,F.2013.Enhanceddualquaternionskinningforproductionuse.InACMSIGGRAPH2013Talks,ACM,9.EWIS,J.P.,CORDNER,M.,ANDONG,N.2000.Posespacedeformation:auniedapproachtoshapeinterpolationandskeleton-drivendeformation.InProceedingsofACMSIGGRAPH,165–172.AGNENAT-THALMANN,N.,LAPERRIÈRE,R.,ANDHALMANN,D.1988.Joint-dependentlocaldeformationsforhandanimationandobjectgrasping.InGraphicsInterface,26–33.ARTHY,J.M.1990.Introductiontotheoreticalkinematics.MITpress.ERRY,B.,MARAIS,P.,ANDAIN,J.2006.Animationspace:Atrulylinearframeworkforcharacteranimation.ACMTrans.Graph.25,4,1400–1423.ERRY,B.,MARAIS,P.,ANDAIN,J.2006.Normaltransformationsforarticulatedmodels.InACMSIGGRAPH2006SketchesACM,134.OHR,A.,ANDLEICHER,M.2003.Buildingefcient,accuratecharacterskinsfromexamples.ACMTrans.Graph.22,3(July),ÜLLER,M.,HEIDELBERGER,B.,TESCHNER,M.,ANDROSS,M.2005.Meshlessdeformationsbasedonshapematching.InACMTransactionsonGraphics(TOG),vol.24,ACM,471–478.EUMANN,T.,VARANASI,K.,WENGER,S.,WACKER,M.,MAGNOR,M.,ANDHEOBALT,C.2013.Sparselocalizeddeformationcomponents.ACMTransactionsonGraphics(TOG)32,6,179.ZTIRELI,A.C.,BARAN,I.,POPA,T.,DALSTEIN,B.,SUMNER,R.W.,ANDROSS,M.2013.Differentialblendingforexpressivesketch-basedposing.InProc.SCAARK,S.I.,ANDODGINS,J.K.2006.Capturingandanimatingskindeformationinhumanmotion.InACMTransactionsonGraphics(TOG),vol.25,ACM,881–889.ODOLPHE,V.,BARTHE,L.,GUENNEBAUD,G.,CANI,M.-P.,ROHMER,D.,WYVILL,B.,GOURMEL,O.,ANDAULIN,M.2013.Implicitskinning:Real-timeskindeformationwithcontactmodeling.ACMTransactiononGraphics(TOG).ProceedingsofACMSIGGRAPHOSSIGNAC,J.,ANDINACUA,Á.2011.Steadyafnemotionsandmorphs.ACMTransactionsonGraphics(TOG)30,5,116.ATTLER,M.,SARLETTE,R.,ANDLEIN,R.2005.Simpleandefcientcompressionofanimationsequences.InProceedingsofthe2005ACMSIGGRAPH/EurographicssymposiumonComputeranimation,ACM,209–217.HOEMAKE,K.1985.Animatingrotationwithquaternioncurves.InACMSIGGRAPHcomputergraphics,vol.19,ACM,245–254.INGH,K.,ANDIUME,E.1998.Wires:ageometricdeformationtechnique.InProceedingsofthe25thannualconferenceonComputergraphicsandinteractivetechniques,ACM,405–414.ARINI,M.,PANOZZO,D.,ANDORKINE-HORNUNG,O.2014.Accurateandefcientlightingforskinnedmodels.InGraphicsForum,vol.33,WileyOnlineLibrary,421–428.ANG,X.C.,ANDHILLIPS,C.2002.Multi-weightenveloping:least-squaresapproximationtechniquesforskinanimation.InProc.SCA,129–138.EBER,J.2000.Run-timeskindeformation.InProceedingsofgamedevelopersconference11of11 SIGGRAPHCourse2014—Skinning:Real-timeShapeDeformationPartI:DirectSkinningMethodsandDeformationPrimitivesLadislavKavan 7UpcomingtrendsandopenproblemsIntheprevioussections,weattemptedtocoverthemaintrendsindirectskinningmethods.Itisimportanttonotethatthislistisnotexhaustiveandalmostcertainlywillchangeinthefuture.AnexcellentexampleofarecentlyhatchedtechniqueisImplicitSkinning[Rodolpheetal.2013].8AcknowledgementsManythankstoMarcAlexa,DanielePanozzo,SvenForstman,CengizOztireli,JarekRossignac,IlyaBaran,andBruceMerryforsharingtheirresultsandexpertise.ReferencesLEXA,M.,ANDÜLLER,W.2000.Representinganimationsbyprincipalcomponents.InComputerGraphicsForum,vol.19,WileyOnlineLibrary,411–418.LEXA,M.2002.Linearcombinationoftransformations.InACMTransactionsonGraphics(TOG),vol.21,ACM,380–387.ADLER,N.I.,ANDORRIS,M.1982.Modellingexiblearticulatedobjects.InProc.ComputerGraphics'82,OnlineConfLOOM,C.,BLOW,J.,ANDURATORI,C.,2004.ErrorsandomissionsinMarcAlexa's“Linearcombinationoftransformations”.OTSCH,M.,KOBBELT,L.,PAULY,M.,ALLIEZ,P.,ANDÉVY,B.2010.PolygonMeshProcessing.AKPeters.USS,S.R.,ANDILLMORE,J.P.2001.Sphericalaveragesandapplicationstosphericalsplinesandinterpolation.ACMTransactionsonGraphics(TOG)20,2,95–126.ORDIER,F.,ANDAGNENAT-THALMANN,N.2005.Adata-drivenapproachforreal-timeclothessimulation.InGraphicsForum,vol.24,WileyOnlineLibrary,173–183.AURE,F.,GILLES,B.,BOUSQUET,G.,ANDAI,D.K.2011.Sparsemeshlessmodelsofcomplexdeformablesolids.ACMTrans.Graph.30(August),73:1–73:10.ORSTMANN,S.,ANDHYA,J.2006.Fastskeletalanimationbyskinnedarc-splinebaseddeformation.InProc.Eurographics,shortpapersvolumeORSTMANN,S.,OHYA,J.,KROHN-GRIMBERGHE,A.,ANDOUGALL,R.2007.Deformationstylesforspline-basedskeletalanimation.InProc.SCA,141–150.OVINDU,V.M.2004.Lie-algebraicaveragingforgloballyconsistentmotionestimation.InComputerVisionandPatternRecognition,2004.CVPR2004.Proceedingsofthe2004IEEEComputerSocietyConferenceon,vol.1,IEEE,I–684.REGORY,A.,ANDESTON,D.2008.Offsetcurvedeformationfromskeletalanimation.InACMSIGGRAPH2008talks,ACM,ANSON,A.J.2005.Visualizingquaternions.InACMSIGGRAPH2005Courses,ACM.EJL,J.2004.Hardwareskinningwithquaternions.GameProgrammingGems4,487–495.YUN,D.-E.,YOON,S.-H.,CHANG,J.-W.,SEONG,J.-K.,KIM,M.-S.,ANDÜTTLER,B.2005.Sweep-basedhumanTheVisualComputer21,8-10,542–550.ACOBSON,A.,ANDORKINE,O.2011.Stretchableandtwistablebonesforskeletalshapedeformation.ACMTrans.Graph.30,6,ALRA,P.,MAGNENAT-THALMANN,N.,MOCCOZET,L.,SANNIER,G.,AUBEL,A.,ANDHALMANN,D.1998.Real-timeanimationofrealisticvirtualhumans.ComputerGraphicsandApplications,IEEE18,5,42–56.AVAN,L.,ANDORKINE,O.2012.Elasticity-inspireddeformersforcharacterarticulation.ACMTransactionsonGraphics(proceedingsofACMSIGGRAPHASIA)31,6,196:1–196:8.AVAN,L.,ANDÁRA,J.2005.Sphericalblendskinning:areal-timedeformationofarticulatedmodels.InProceedingsofthe2005symposiumonInteractive3Dgraphicsandgames,ACM,9–16.AVAN,L.,COLLINS,S.,ZARA,J.,ANDO'SULLIVAN,C.2008.Geometricskinningwithapproximatedualquaternionblending.ACMTrans.Graph.27,4,105:1–105:23.AVAN,L.,COLLINS,S.,ANDO'SULLIVAN,C.2009.Automaticlinearizationofnonlinearskinning.InProc.I3D,49–56.IM,Y.,ANDAN,J.2014.Bulging-freedualquaternionskinning.ComputerAnimationandVirtualWorlds25,3-4,323–331.10of11 SIGGRAPHCourse2014—Skinning:Real-timeShapeDeformationPartI:DirectSkinningMethodsandDeformationPrimitivesLadislavKavan Figure9:Unlikelinearanddualquaternionskinning,stretchableandtwistablebones[JacobsonandSorkine2011]allowustospreaddeformationsalongthelengthofabone.6ComputingnormalsofskinnedsurfacesIntheprevioussections,wediscusseddeformationsoftheactualshape,representedusingapolygonmesh.Forrendering,itisnecessarytocalculatenotonlydeformedvertexpositions,butalsotheircorrespondingnormals.Ofcourse,oncethedeformedvertexpositionshavebeencomputed,thecorrespondingnormalscanbeestimatedbyaveragingnormalsofadjacenttriangleswithappropriateweights[Botschetal2010].Whileeasytoimplement,thisapproachisnotwellsuitedforparallelprocessing,becausethenormalscomputationstepwouldhavetowaituntilall;:::;havebeencomputed.EspeciallyinGPUimplementationsofdirectskinningmethods,itisadvantageoustocalculatethedeformednormalsalongwiththevertexIfourmodelisdeformedbyagloballineartransformation(translationsobviouslydonotaffectthenormals),thenormalstransformbyitsinversetranspose:.Withlinearblendskinning,wecanusethistechniquewith.Fordualquaternionskinningwecancalculatethematrixsimilarly,byblendingunitdualquaternions.Ineithercase,thenormalsarecomputedas:Whilethismethodisstraightforwardandverycommon,thenormalscomputedthiswayarenotalwaysagoodapproximationofthetruenormalswhichwewouldobtainbyaveragingnormalsofadjacenttriangles.Thisisbecauseforskinnedmodels(eitherlinearordual-quat),thetransformationmatrixchangesfromonevertextoanother;Equation(16)iscorrectonlyinpartsofthemeshwheretheskinningweightsareconstant,i.e.,isconstant.Inareaswhereskinningweightshaveanon-trivialgradient,Equation(16)leadstobiasednormals,illustratedinFigure10.Inthisgure,theshapeistransformedbytwoskinningtransformations.Theproblemisthattheentiredeformationhasbeenachievedonlybytranslating;thelinearpartsofboth Figure10:A2Dcapsuleobjectdemonstratingthatskinnednormalscanbeapoorapproximationoftrue,geometricnormals.ThechallengeofcalculatingmoreaccuratenormalsofskinnedsurfaceshasbeenopenedbyMerryetal.[2006b]andfurtherrenedbyTarinietal.[2014].Thekeyideaistoassumethatskinningweightsarecontinuousfunctions,whichallowsustodenethegradientofaweightinaparticularvertex:.WeightgradientsallowustoobtainamoreaccurateapproximationoftheJacobianoftheskinningtransformation:Ifwecomputeournormalsas,weobtainskinnednormalsthatcorrectlyaccountforeffectssuchasthoseshowninFigure10.[Tarinietal2014]discussesmanypracticalimprovementsofthisidea,suchashowtheinversionoftheJacobiancanbeavoided,andpresentsadetailedimplementationrecipe.9of11 Rest pose SIGGRAPHCourse2014—Skinning:Real-timeShapeDeformationPartI:DirectSkinningMethodsandDeformationPrimitivesLadislavKavan hand.Insomecases,however,thebulgingisunwanted,andseveralstrategiestoeliminateithavebeenproposed.Observingthatlinearblendskinningdoesnotproducebulgingwhilebending,AutodeskMayaallowsuserstoblendtheresultoflinearanddualquaternionskinning.Thisrequiresanadditionalblendingweight.Theproblemwiththisapproachisthatevenasmallamountoflinearblendskinningre-introducesthecandy-wrapperartifacts,soacompromisemustbesought.Theswing-twistdeformerpresentedin[KavanandSorkine2012]isbasedonthesameobservation(linearblendingworkswellwhilebending),butcombineslinearanddualquaternionblendinginanon-linearway.Specically,therotationofajointisdecomposedintoaswingandtwistcomponents;thetwistisblendedspherically(usingSLERP)andtheswingisblendedusinglinearmatrixinterpolation,composingtheresultusingmatrixmultiplication.Thiseliminatesthebulgingartifactswithoutre-introducingthecandy-wrapperartifactswhiletwisting.Theswing-twistdeformerleadsustotheideaofgeneralizeddeformationprimitives,discussedinthenextsection.5DeformationprimitivesBydeformationprimitiveswemeantheelementarybuildingblocksofadeformation.Inlinearanddualquaternionskinning,thedeformationprimitivesaretheindividualskinningtransformations;:::;.Intuitively,thesetransformationsspecifyhowpartsoftheinputobjectwilldeform.Thisisveryobviouswithbinaryskinningweights,whichsimplypartitiontheinputobjectintoseveralparts.Smoothskinningweightsthenmeanthattherewillbesmoothblendingbetweentheindividualparts.Incharacteranimation,thesepartstypicallycorrespondtobones,e.g.,theforearm,upperarm,etc.Whileafnetransformationsarecertainlythemostcommondeformationprimitives,thereareotherpossibilitieswhichpresentcertainbenets,pioneeredbyworkssuchas[SinghandFiume1998;Kalraetal1998;Hyunetal2005].Forbrevity,werefertodeformationprimitivesasdeformers.TheworkofForstmannandcolleagues[ForstmannandOhya2006;Forstmannetalintroducesspline-baseddeformers,seeFigure8.Theideaistobindrest-poseverticestoasplinecurve.Specically,forvertexwendtheclosestpointonthespline,andbindtotheFrenetframeatthispoint.Atrun-time,weevaluatethedeformedsplineincludingitsFrenetframes,whichdeterminethetransformationof.Eachsplinecorrespondstoonedeformer.Multiplespline-baseddeformersareblendedtogetherlinearly,similarlytolinearblendskinning. Figure8:Splineskinningoffersmoreaccuratecontrolofdeformationsthenlinearordualquaternionskinning.[GregoryandWeston2008]presentanadvancedversionofthepreviousidea,calledOffsetCurveDeformations(OCD).OCDsupportssplineandothersmoothcurves;forconcreteness,wewillassumeB-splinecurves.Similarlyto[ForstmannetalOCDconnectsa“master”B-splinetothebonesofourobject,smoothingouttheirpiecewise-linearnature.ThemainideaofOCDistoassociateeachvertexwithanindividualB-spline,whichisanoffsetcurveofthe“master”B-spline.Incontrasttospline-basedskinning[Forstmannetal2007],thisallowsustobettercontrolthedistributionofthedeformations,especiallyontheinsideandoutsideofabend.BecauseB-splinesarelinear,theresultingskinningtechniqueisalsolinear.ItturnsoutthatOCDisequivalenttoAnimationSpace,discussedinSection3.AgreatbenetoftheOCDtechniqueisthatitsparameterscanbeadjustedinauser-friendlyway.JacobsonandSorkine[]observedthatlinearordualquaternionskinningisunabletocontroldeformationsalongthebone;theblendingisconstrainedtoatypicallyrathersmallregionnearthejoint.Aroundthebone,theweightsaretypicallybinary(1fortheclosestboneand0fortheothers),whichdoesnotallowustocontroldeformationsoftheboneitself.Stretchingandtwistingofbones,highlydesirablee.g.withstylizedorcartooncharacters,canbesupportedbyusinganadditionalweightfunction,calledendpointweight[JacobsonandSorkine2011].Theendpointweightvariesfrom0to1aswemovefromoneboneendpointtotheother;thereisoneendpointweightforeachbone.Theendpointweightprovidesuswiththeinformationwhereavertexislocatedwithrespecttothebone,whichallowsustoimplementeffectssuchasstretchingandtwistingofthebones,seeFigure9.Focusingondeformationsinthevicinityofajoint,KavanandSorkine[2012]proposeageneralconceptofjoint-baseddeformersAjoint-baseddeformerisafunction�:SOwhichtakesajointrotationandarest-posevertexpositionasinputandproducesadeformedvertexpositionasoutput.Theswing-twistdeformer,discussedinSection4,isaspecicexampleofajoint-baseddeformer.Individualdeformersareblendedlinearly,similarlytolinearblendskinning.Theideaisthatartifactssuchascandy-wrapperswillbeavoidedbythedeformersthemselves;theblendingservesonlytoeasethetransitionsbetweentheindividualdeformers,assumingtherearenolargediscrepanciesbetweentheblendeddeformers.8of11 Linear blend skinningDual quaternion skinningSpline skinning SIGGRAPHCourse2014—Skinning:Real-timeShapeDeformationPartI:DirectSkinningMethodsandDeformationPrimitivesLadislavKavan matrixexponential:=explog(FromtheviewpointofLie-algebraicaveraging,theproblemofthismethodisthatthelogarithmscorrespondtousingtheLiealgebraattheidentity,eveniftheinputtransformationsareallfarawayfromtheidentity.Thisleadstonon-shortest-pathinterpolations,whichhasbeencriticizedinaratherharshwaybygamedevelopers[Bloometal2004].Alexa'smethodhasbeenappliedinskinning[CordierandMagnenat-Thalmann2005],butitsnon-shortest-pathnaturecanleadtoartifacts,asshownin[Kavanetal.2008].Animprovedmethodtoblendafnetransformationsispresentedby[RossignacandVinacua2011],however,theproblemthatlog(maynotexistintherealdomainpersists.Evenifwerestrictourselvesonlytomatriceswithapositivedeterminant,thelogarithmcanstillbeill-denedifthematrixhastwo(real)negativeeigenvaluesofdifferentmagnitude.Onepossibilitytoavoidthisproblemisbyintroducinganintermediarytransformation,essentiallysubdividingthemotionintotwosmallerones[RossignacandVinacua2011].Toourknowledge,thismethodisyettobetestedinskinning.Whiletheshortest-interpolation-pathpropertymakessensewheninterpolatingtheelementsofSO,realelasticmaterialscanbetwistedmultipletimes.Inthiscase,bothrotationsmatricesandquaternionsfallshortinrepresentingsuchdeformations(quaternionsdorememberonefullrevolution,buttwofullrevolutionsreturnusbacktothestartingpoint,i.e.,are“forgotten”,seeFigure4).Thislimitationofbothlinearanddual-quaternion-basedmethodsinspiredtheworkofOztirelietal.[2013],calledDifferentialBlendingDifferentialblendingassumesaconnected,rootedskeleton.Skinningtransformationsarebrokenintosmallerpiecesj;kthepathfromtheroottothetargetbone:j;:::j;l.Themainideaisthattheblendingofthesesmallerpiecescanbedonewithpreviousmethods,andtheblendedpiecesaresubsequentlymultiplied(composed)together:)=j;kInthisformula,isastandardblendingoperator(linearordualquaternion)whichblendsinputtransformationsusingtheprovidedweights.Theresultingtakescorrectlyintoaccountmultiplerevolutions,i.e.,longerinterpolationpaths,seeFigure6. Figure6:Creatingacorkscrew-typedeformationfailswithlinearanddualquaternionblending.Multiplerevolutionsarecorrectlyhandledbydifferentialblending.Anotherproblemwithdualquaternionskinningisknownasa“bulgingartifact”[KavanandSorkine2012;KimandHan2014].Theproblemisbestexplainedonasimplecylinderskinnedusingtwobones,connectedwithonejoint(crudeapproximationofthehumanarm,withthejointcorrespondingtotheelbow).Inthiscase,dualquaternionskinningreducestosphericalblendingaroundthejoint.Sphericalblendingcanbeintuitivelyunderstoodasa“pointonastick”approach:eachvertexisattachedtoanimaginaryrigidsticklinkingthevertexto.Inotherwords,eachvertexisconstrainedtoaxedspherecenteredat;theradiusofthissphereis.Thisworksverywellwhenwearetwistingthecylinder,butbendingproducessomewhatunnaturalbulgingeffects,seeFigure7.Notethatthisbulgingeffectisnotalwaysundesired:forexample,whenmodelingknucklesofthe Figure7:Demonstrationofdualquaternionbulgingeffectswhilebendingacylinder.Elasticity-inspireddeformers[KavanandSorkine2012]avoidthisproblem.7of11 Rest pose Linear blendingDual quaternion blendingDifferential blending SIGGRAPHCourse2014—Skinning:Real-timeShapeDeformationPartI:DirectSkinningMethodsandDeformationPrimitivesLadislavKavan Skinningtransformations;:::;typicallycontainanon-trivialtranslationvector.Assumingthattheleftsubmatricesof;:::;arerotations,astraightforwardsolutionistoblendtherotationsandtranslationsindependently.Linearblendingoftranslationsisperfectlyjustied,becausetranslationvectorsformalinearspace,notacurvedmanifold.Ifweimplementthisapproachwendoutthat,evenwithperfectlyintrinsicrotationblending[BussandFillmore2001],theresultsarenotacceptable–oftenmuchworsethanwithlinearblendskinning,seeFigure5.Thereasonsforthisbehaviorarediscussedindetailin[Kavanetal2008].Inshort,bysplittingarigidtransformationintoarotationandtranslationpair,wearecommittingtoaspecicpivotpoint(centerofrotation),aroundwhichtherotationswillbeinterpolated.Bydefault,thiscenterofrotationcorrespondstotheoriginofmaterial-spacecoordinates,whichistypicallylocatedneartheobject'scenterofmass–thisexplainstheunusualresultinFigure5(left). Figure5:Separateblendingofrotationandtranslationcomponentsleadstounacceptableresults(left),worsethanstandardlinearblendskinning(right).Itisofcoursepossibletodesignamoresuitablecenterofrotation,e.g.,coincidingwiththejoint[Hejl2004],orcomputedusingleastsquaresoptimization[Kavanandára2005].However,itispossibletondsituationswhereeachofthesestrategiesresultsinartifacts[Kavanetal2008].Thelatterpapershowsthatthecomplicationswiththechoiceofthecenterofrotationdisappearifweuseunitdualquaternionstorepresentrigidbodytransformations.Thereasonisthatunitdualquaternionsrepresentrigidtransformationsusingtheirintrinsicparameters,thusavoidinganexplicitchoiceofacenterofrotation.Dualquaternionsarecloselyrelatedtoscrewmotionsstudiedintheoreticalkinematics[McCarthy1990].Whiletheunderlyingmathematicsmaynotbetrivial,anactualimplementationofdualquaternionskinningisquitestraightforward.First,thetransformationmatrices;:::;areconvertedtounitdualquaternions.Theseunitdualquaternionsareblendedlinearly,similarlytolinearblendskinning(Equation(1)).Becauselinearcombinationofunitdualquaternionsdoesnotingeneralproduceaunitdualquaternion,anormalization(projection)operationisperformed.Theresultingunitdualquaternioncanbeconvertedtoamatrixwhichtransformsarest-posevertex.Thedoublecoverpropertyofregularquaternionsoccursalsoindualquaternions;specically,unitdualquaternionsfromadoublecoverofSE.Itisthereforeimperativetocarefullychoosesignsofthedualquaternionsobtainedbyconvertingmatrices.Notethatallstepsinthisalgorithmaresimpleclosed-formoperations.Whilelinearityislost(duetotheprojectiononunitdualquaternions),theresultingalgorithmdoesnotrequireanyiterationsandcanbeimplementedveryefciently.Dualquaternionskinningsuccessfullyeliminatesthecandy-wrapperartifacts(seeFigure3),buthasanumberoflimitations,whichwediscussinthefollowing.Arelativelybenignissueisthatlinearblendingofunitdualquaternionsfollowedbynormalization(Dual-quaternionLinearBlending,DLB)isnotperfectmanifold-intrinsicaveraging,similarlytotheSOcaseillustratedinFigure4.AcompletelyintrinsicblendingcanbeachievedbyapplyingLie-algebraicaveraging[Govindu2004],whichisageneralizationofsphericalaverages[BussandFillmore2001].AdualquaternionversionofLie-algebraicaveraginghasbeendiscussedin[Kavanetal2008](AlgorithmDIB).Forshortestpathinterpolations,thedifferencesbetweenDLBandDIBareinsignicant;inapplicationssuchasskinningthedifferencesbetweenDLBandDIBarebarelynoticeable.Amoreseriousconcernespeciallyinaproductionenvironmentarenon-rigidtransformations.Otherthanuniformscale,dualquaternionsareunabletorepresentnon-rigidtransformations,suchasnon-uniformscaleandshear.Theseeffectsareespeciallyimportantwithstylizedandcartooncharacters.Onepossibilitytoside-stepthislimitationistobreaktheskinningprocessintwosteps:intherststep,therest-pose(includingtheskeleton)isrescaled.Inthesecondstep,dualquaternionskinningisappliedtoproducethedesiredpose(articulation)[Kavanetal2008].Whileasimilarapproachhasbeenappliedsuccessfullyinaproductionsetting(Disney'sFrozen[Leeetal2013]),thequestionofoptimalblendingofgeneralafnetransformationsremainsopen.AnearlyinvestigationofthisproblemhasbeencarriedoutbyAlexa[2002],whoproposeslinearblendingofmatrixlogarithms,followedby6of11 SIGGRAPHCourse2014—Skinning:Real-timeShapeDeformationPartI:DirectSkinningMethodsandDeformationPrimitivesLadislavKavan 4NonlinearskinningmethodsLinearskinningmethodsarepopularduetotheirefcientimplementationsandwellunderstoodmathematicalproperties,makingthemwellsuitedforuseasbuildingblocksinmorecomplexalgorithms,e.g.,inphysics-basedsimulation[FaureetalHowever,aswehaveseenintheprevioussection,candy-wrapperartifactscanonlybeavoidedusingmoreparameters,whichhavetobelearned,stored,andretrievedatruntime.Eventhen,someamountofundesiredshrinkingcanstillremain.Thefundamentalproblemisthatlinearblendingofrotations,i.e.,elementsofSO,doesnotrespectthefactthatSOisacurvedmanifold.Theideaofreplacinglinearblendingwithmanifold-intrinsicaveragesleadsustononlinearskinningmethods.Amanifold-intrinsicinterpolationmethodbetweentwo3DrotationsisSLERP(SphericalLinearInterpolation),introducedbyKenShoemake[1985].EventhoughSLERPcanbeformulatedpurelywithmatrices,Shoemakerecognizedtheadvantagesofquaternions:matricescontain9degreesoffreedomandthereforerequire6constraintstorepresentrotations.Quaternions,ontheotherhand,featureonly4degreesoffreedomandthereforerequireonlyoneconstrainttorepresentrotations.Specically,rotationscorrespondtounitquaternions,i.e.,quaternionswithlengthone.Formally,wedenotethesetofunitquaternionsas=1Geometrically,unitquaternionsforma3Dunithyper-spherein4DEuclideanspace.Whilethisisdifculttovisualize,itisoftensufcienttoappealtotheintuitionofthefamiliar2Dspherein3DEuclideanspace(orevenaunitcirclein2D).Mathematiciansarehardlyimpressedbythefactthatquaternionsrequireonly4scalarsasopposedto9;worse,quaternionsaresometimesincorrectlyassumedtobejustadifferentsemanticstodescribethegroupof3Drotations.ThisisincorrectbecausetheSOarenothomomorphic.Thecatchisthefactthatthetwoquaternionsexactlythesamerotation,i.e.,thegroupiscoveringSOtwice.Thisiscalledthe“doublecover”property[Hanson2005]anditisthemainreasonwhyquaternionsareverypracticalinskinning;letusexplainthisinmoredetail.Consideruniformrotationofarigidbodyaboutthez-axis(forexample).Startingfromtheidentity,i.e.,norotation,weeventuallyperformonefullrevolution,i.e.,a360degreesrotation.Rotationmatricesdonotdistinguishbetween0and360degreesrotation.Interestingly,quaternionsdo:a360degreesrotationwillcorrespondtoquaternion.Thesigndistinguishesbetween0degreesrotation(correspondingto)and360degreesrotation(correspondingto).Notethattheformulasforconvertingfromquaterniontomatrix[Shoemake1985]arequadraticandcancelthesign,i.e.,afterconversiontoamatrix,thedistinctionbetweendisappears.Ifwekeeprotatingourobjectbeyond360degrees,wewilleventuallyreachtwofullrevolutions,i.e.,720degreesrotation.Twofullrevolutionsarenotrememberedbyquaternions,i.e.,a720degreesrotationisequivalentto0degreesrotationinbothrotationmatricesandquaternions.Thisinterestingfeatureofquaternionscanbeobservedintherealworld;see,forexample,the“Dirac'sbelttrick”[Hanson2005].Intermsofblendingrotationsand,consequently,skinning,theimportantconsequenceisthatthemanifoldis“lessnon-linear”SO.ThisisillustratedinFigure4.Intheleftpartofthegure,SOdifferbya180degreesrotationSOistheidentity).Alinearaverageofresultsinasingularmatrix.Intherightpartofthegure,thesamesituationisdepictedonthemanifold.Inparticular,areunitquaternionscorrespondingto,andisone,interpretedasaquaternion.Notethataremuchcloserthan.Thisisbecauseonefullloopcorrespondstoa720 Figure4:Illustrationofthegeometryofrotations(SO,left),comparedtounitquaternions(,right).degreesrevolutionin,butonlyto360degreesinSO.Inotherwords,distancesbetweenrotationsinaretwiceassmallasthecorrespondingdistancesinSO.Asaresult,linearaveragingofisnowmuchmoreaccurate,asisobviousfromFigure4.Thelineconnectingismuchclosertothemanifoldanddoesnotpassthroughasingularity.Thisbenetdoesnotcomeforfree,however,becauseweneedtobecarefulaboutpickingthesignsduringmatrixtoquaternionconversion:wrongsignscanleadtoatypicallyundesiredlong-arcinterpolation.Notethatprojectionofanon-unitquaterniononthemanifoldisextremelysimple:TheSLERPalgorithmhasbeengeneralizedtomorethan2rotationsbyBussandFillmore[2001],however,resultinginaniterativeprocedure.Becausefullyaccurateblendingofrotationsisnotnecessaryinskinning,itispossibletoapproximatethecorrectmanifold-intrinsicaveragesbylinearcombinationofunitquaternions,followedbynormalization(projectionon).Thishasbeenutilizedinearlyquaternion-basedskinningmethodssuchas[Hejl2004]and[Kavanandára2005].Thepracticalimpactofthesetechniquesislimitedduetotheirhandlingofthetranslationalcomponentoftheskinningtransformations.5of11 SO(3) SIGGRAPHCourse2014—Skinning:Real-timeShapeDeformationPartI:DirectSkinningMethodsandDeformationPrimitivesLadislavKavan AnotherlinearskinningmodelisAnimationSpace[Merryetal2006a].Merryandcolleaguesobservedthatnotallofthe12weightsofMWEareusefulandproposedalinearmodelwhichusesonly4weightspervertex-bonepair.WhilethismakesAnimationSpaceseeminglylesspowerfulthanMWE,itturnsoutthatAnimationSpaceenforcesworld-spacerotationinvariance,i.e.,itdeliberatelyloosestheabilitytotreatindividualworldcoordinatesdifferently.Thisfeaturedoesnotseemtobeverypractical,becauseitisnaturaltoassumethataworld-spacerotationofalltransformations;:::;willrotatebutnototherwisedeformtheresultingvertices.Thisworld-spacerotationinvariancecorrespondstotheclassicalrequirementfromelasticity,i.e.,thetotalelasticenergyshouldnotdependonglobalworld-spacerotations.Notethatitisperfectlyreasonableforelasticdeformationstodependonmaterial-spacerotations;onlyisotropicmaterialsareinvariantalsotomaterial-spacerotations.Inthefollowing,wederivethegeneralformofrotationinvariantlinearskinning.Ifwedenote=[;:::;,wecanexpresstherequirementofworld-spacerotationinvarianceas:SO(3):vecRT)=vecApplyingEquation(4)transformsthisequationinto:vecasbefore.BecauseEquation(8)mustbesatisedforarbitrary,itfollowsthat:)=Ifwelookatasacollectionofblocks,Equation(9)requiresthateachsuchblockcommuteswith,i.e.,RY.BecausethismustbetrueforallSO,itisnotdifculttoprovethatthisimpliesthatisascaledidentity,(sketchofaproof:chooseafewspecicmatricesandmassagetheresultingsystemoflinearequations).Therefore,ifagenerallinearskinningmethod(Equation(2))isworld-spacerotationinvariant,itscorrespondingblocksmusthavethefollowingstructure:=[i;j;i;j;i;j;i;j;i.e.,theoriginal36degreesoffreedomreducetofour:i;j;;yi;j;;yi;j;;yi;j;.Anadditionalrestrictionwillfollowfromconsideringtranslationinvariance:globalworld-spacetranslationscanonlytranslatebutnototherwisedeformtheresultingvertices.Formally,ifisaglobaltranslation,i.e.,stackofvectors;x;y;z,werequire:)=CombinedwithEquation(10),itfollowsthatrotationandtranslationinvariancerequiresi;j;=1Thetotalnumberofdegreesoffreedompervertexistherefore.AlinearskinningmodelsatisfyingEquation(10)andEquation(12)isexactlytheAnimationSpaceintroducedbyMerryandcolleagues[Merryetal2006a].FromthediscussionaboveitfollowsthatAnimationSpaceisthemostgenerallinearskinningmethodinvarianttoworld-spacerotationsandtranslations.Similarlytomulti-weightenveloping,AnimationSpaceweightsarealsolearnedfromasetofexamplesandareabletosuppresscandy-wrapperartifacts.SomeintuitioncanbegainedbyconsideringthisformulationwhichispracticallyequivalenttoAnimationSpace:Theonlydifferencetotheclassicallinearblendskinning(Equation(1))isthatweallowadifferentrest-poseforeachtransformationi.e.,wehaveinsteadofjust.Classicallinearblendskinning(Equation(1))canbeinterpretedastransformingtherest-posemeshbyeachtransformation,andblendingtheresultsusingvertexweights.InAnimationSpace,wealloweachtransformationtohaveadifferentrestpose.Theindividual“imaginary”restposesareneverrevealedtotheuserdirectly,becausethealwaysblendsthemtogether,evenifallofthetransformations;:::;areidentities.ThisindicatesthatAnimationSpaceisinfactaspecialcaseofexample-basedskinningmethods,featuringaparticularlyelegantmathematicalformulation.Furthergeneralizationsoflinearskinningmethods,goingbeyondEquation(2),arepossiblebyconsideringmoregeneraltransfor-mationsthantheclassicalafnetransformations;:::;.OnespecicexampleispresentedbyParkandHodgins[2006],whoconsiderquadratictransformationsrepresentedbymatrices(seealso[Mülleretal2005]).Withthismoreliberalmindset,clusteredPCAappliedtoanimatedsequences[Sattleretal2005]canbealsoviewedasaspecialtypeofalinearskinningmethod.Theideaofclustered(local)PCAistosplittheinputmeshintoseveraldisjointcomponents,eachofwhichissubjectedtoclassical(global)PCA[AlexaandMüller2000].Evenclosertoskinningarelineartechniqueswhichallowforoverlapsbetweentheindividualcomponents,suchastheSPLOCsmodel[Neumannetal2013].Data-drivenskinningtechniquesarediscussedinmoredetailinthenalpartofthiscoursebyZhigangDeng.4of11 SIGGRAPHCourse2014—Skinning:Real-timeShapeDeformationPartI:DirectSkinningMethodsandDeformationPrimitivesLadislavKavan producesthedeformedvertexpositions(scalars).Aconvenienttoolhereis,i.e.,anoperatorvec)=convertsamatrixintoavectorbystackingtheindividualcolumns.Formoredetails,werecommendtheWikipediapage.Subsequently,westackallofthevectors.Followingthesameconvention,wecanstackallofthedeformedvertexpositionsintoonelongvector.Now,wecandeneagenerallinearskinningmodel:isamatrixofparameters.Itwillbeusefultosplitthismatrixintoblocks=1;:::;n=1;:::m.Eachcorrespondstoavertex-bonepairandhas,initsmostgeneralform,36degreesoffreedom.Itisinterestingtothinkhowlinearblendskinningrelatestothisform.Specically,whatisthematrixcorrespondingtolinearblendskinning(Equation(1))?ThisquestioncanbeelegantlyansweredusingKroneckerproducts–aspecialtypeofmatrixmultiplicationwhichiscloselyrelatedtothevectorizationoperator.Ingeneral,theKroneckerproductofmatrices





qsKroneckerproductsexhibitanumberofinterestingproperties,summarizedat.Here,wewillneedthefollowingproperty,whichassumesthatmatricescanbemultipliedtogether,i.e.,Inthatcase,vec)=(vec)=(vecareidentitymatrices.Letusapplythistothelinearblendskinningformula(Equation(1)):vec)=vec))=LBSLBSLBSLBS=[LBS;:::;LBS.WecanwriteouteachLBSblockevenmoreexplicitly:ifwedenotethecoordinatesof=(;v;v,wehaveLBS=[Thisrevealsthattheclassicallinearblendskinning(Equation(1))isaratherspecialformofgenerallinearskinning.Wearenotawareofanyworkexploringthefullygeneral36-parameterlinearskinningmodel(Equation(2)).TheclosestinstanceisMulti-WeightEnveloping(MWE)[WangandPhillips2002],whichadvocatesalinearmodelwithtwelveweightspereachvertex-bonepair:MWE00 00 00 00 0w5i;jvi;20 0w8i;jvi;30 00 00 00 00Noticethatoneweightofclassicallinearblendskinning,isreplacedby;:::;w.AsdemonstratedbyWangandPhillips[2002],thisresultsinamorepowerfullinearmethodwhichcanavoidcandy-wrapperartifacts.Theproblemisthattherearetwelvetimesasmanyweights.Theirdesignismorecomplicated,becausethesemoregeneralweightsnolongerhaveasimpleintuitiveinterpretationasinclassicallinearblendskinning(seeFigure2).Instead,theMWEweightsareoptimizedusingasetofexampleshapes[WangandPhillips2002].Thisassumesthattheseexampleshapeshavebeenpreparedbyartists,whichmakesthismethodrelatedtoexample-basedskinningtechniques,coveredinmoredetailinthethirdsectionofthiscoursebyJ.P.Lewis. Figure3:Anelasticbarintherestpose(left)withtwobones.A180degreestwistoftherightbone(lightergray)resultsinacandy-wrapperartifact(middle).Thecentralcross-sectionistransformedbyaprojectionmatrix,whichcollapsestheentireedgelooptoasinglepoint.Thisbehaviorcanbeavoidedwithdualquaternionskinning(right).3of11 Rest pose SIGGRAPHCourse2014—Skinning:Real-timeShapeDeformationPartI:DirectSkinningMethodsandDeformationPrimitivesLadislavKavan isnotalinear(“at”)space,butacurvedmanifold.JustlikethecurvatureoftheEarthrarelybothersusineverydaylife,thisisnotabigproblemiftheblendedtransformationsareclose.However,considerwhathappensifweneedtoblendthesetworotations:10001000110010001istheidentityandisrotationaboutthez-axisby180degrees(translationsarenotrelevantinthisexample).Linear+0resultsinarank-1matrix,projectingthe3Dspaceontothez-axis.Transformationofashapebythismatrixtypicallyresultsinundesirabledeformations.Largerelativerotationsarenotuncommoninskinning,becausejointssuchasshoulders,wrists,orevenelbowsoftenexhibitaratherlargerangeofmotion.Thisproblemissocommonitearnedaname:acandy-wrapperartifact,seeFigure3.Inadditiontothisproblem,itisobviousthatlinearblendskinninghasonlyalimitednumberofparameters.Onepossibilitytoexpandtheexpressivepoweroflinearblendskinningisbyusingmoretransformations,typicallycalculatedfromtheoriginalbonetransformationsbynon-linearprocedures[Weber2000;MohrandGleicher2003;Kavanetal2009].Alternatively,itispossibletoenrichthespaceofskinningweights,leadingtomethodswhicharestilllinear,butfeaturemoreparametersthanlinearblendskinning.Wecallthesetechniquesandweexploretheminthefollowingsection.3Multi-linearskinningmethodsWhilethecandy-wrapperartifactsoflinearblendskinning(Figure3)cannotbexedbychangingweights,itturnsoutthatintroducingmoreweightshelps,atleasttosomeextent.Thisindicatesthatlinearblendskinningisnotthemostgenerallinearmodel–so,whatis?Weassumethattherest-poseshapeandtheskinningweightsarepropertiesoftheobjectandarenotchangingduringthecourseofananimation.Theonlyquantitywhichchangesduringanimationarethetransformations;:::;.Therefore,themostgenerallinearskinningmodelissimplyalinearfunctionwhichconsumesallofthetransformations(scalars)asinput,and Figure1:Bonetransformations(lowerandupperarmbones)foroneexampledeformedpose. Figure2:Inuenceweightscorrespondingtolowerandupperarmbones.2of11 M1M2 SIGGRAPHCourse2014—Skinning:Real-timeShapeDeformationPartI:DirectSkinningMethodsandDeformationPrimitivesLadislavKavanUniversityofPennsylvania1IntroductionSkinningistheprocessofcontrollingdeformationsofagivenobjectusingasetofdeformationprimitives.Atypicalexampleofanobjectoftensubjectedtoskinningisthebodyofavirtualcharacter.Inthiscase,thedeformationprimitivesarerigidtransformationsassociatedwithbonesofananimationskeleton.Laterwewillseethatthisisnottheonlypossibility.Also,skinningcanbewelldenedevenwithoutanyskeleton.Atahighlevel,thelandscapeofskinningalgorithmscanbedividedintotwomainstreams:directandvariationalmethods.Variationalmethodsposethetaskasanoptimizationproblem,minimizinganobjectivefunction(deformationenergy).Variationalmethodsarerootedincontinuummechanicsandthetheoryofelasticityandtypicallyrequireiterativesolvers.Thefocusofthispartisondirectskinningmethods,whichhavebeendevelopedinordertosidestepthehighcomputationalrequirementsofvariationalmethods.Directmethodscomputetheresultingdeformationsusingclosed-formexpressions,i.e.,withoutanynumericaloptimization.Directmethodsareoftenveryfastandembarrassinglyparallel,whichmakesthemparticularlyattractiveforinteractive,real-timeapplicationsandGPUimplementations.2LinearblendskinningLinearblendskinning,alsoknownasskeleton-subspacedeformation,(single-weight-)enveloping,ormatrix-paletteskinning,isthebasicandmostwellknownalgorithmfordirectskeletalshapedeformation.Itisdifculttotracetherootsoflinearblendskinning.Someoftheearlyideasappearedinthepioneeringworks[BadlerandMorris1982]and[Magnenat-Thalmannetal1988].PerhapstherstpaperthatgivesanexactmathematicaldescriptionoflinearblendskinningisduetoLewis[2000],butitmentionsthealgorithmiswell-knownandimplementedincommercialsoftwarepackages.Linearskinningassumesthefollowinginputdata:Restposeshape,typicallyrepresentedasapolygonmesh.Themeshconnectivityisassumedtobeconstant,i.e.,onlyvertexpositionswillchangeduringdeformations.Wedenotetherest-poseverticesas;:::;.Itisoftenconvenienttoassumethatareinfactvectorswiththelastcoordinateequaltoone,accordingtothecommonconventionofhomogeneouscoordinates.Bonetransformations,representedusingalistofmatrices;:::;.Thematricescanbeconvenientlydenedusingananimationskeleton;inthiscasetheycorrespondstospatialtransformationsaligningtherestposeofbonewithitscurrent(animated)pose.Bonetransformationsaretypicallytheonlyquantitythatisallowedtovaryduringthecourseofananimation.Skinningweights.Forvertex,wehaveweights;:::;w.Eachweightdescribestheamountofinuenceofboneonvertex.Acommonrequirementisthat


=1(partitionofunity).Theseconceptsarebestillustratedwithanexample;seeFigure1foranexampleofbonetransformationsandFigure2forinuenceweights.Linearblendskinningcomputesdeformedvertexpositionsaccordingtothefollowingformula:Thelatterformhighlightsthefactthattherestposevertexistransformedbyalinearcombination(blend)ofbonetransformation.Thesematricesarethedeformationprimitivesoflinearblendskinning,i.e.,elementarybuildingblocksofdeformations.Whilearbitraryafnetransformationsareallowed,sometimesitisconvenienttoassumethatarerigidbodytransformations,i.e.,SE.Notethatmanyimplementationsassumethatmostoftheweights;:::;warezero.Duetographicshardwareconsiderations,itiscommontoassumethereareatmostfournon-zeroweightsforeveryvertex;differentlimitscanbefoundinsomesystems.Someoldergamesusedavariantdubbedrigidskinningwhichcorrespondstoallowingonlyoneinuencingbonepervertex.Withincreasingpolygonbudgets,linearblendskinningquicklyreplacedrigidskinningbecauseitallowedforsmoothtransitionsbetweenindividualtransformations(somesystemsusedthetermsmoothskinning).ThedesignofhighqualityskinningweightsisfarfromtrivialandwillbecoveredinthesecondpartofthesenotesbyAlecJacobson.Linearblendskinningworksverywellwhentheblendedtransformationsarenotverydifferent.Issuesariseifweneedtoblendtransformationswhichdiffersignicantlyintheirrotationcomponents.Itisawellknownfactthatalinearcombinationofrotationsisnolongerarotation[Alexa2002].Geometrically,thisisaconsequenceofthefactthattheLiegroupof3Dtransformations,SO