Interactive performances with style and substance David Wu Microsoft Game Studios GDC 2009 Takeaway Survey of recent research in the field of interactive animation from the point of view of a game developer ID: 164513
Download Presentation The PPT/PDF document "Animation With Momentum" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Animation With Momentum
Interactive performances with style and substance.
David Wu
Microsoft Game Studios
GDC 2009Slide2
Takeaway
Survey of recent research in the field of interactive animation from the point of view of a game developer.
Assessments of the effectiveness and overall applicability of proposed techniques
A pragmatic distillation of subsets of ideas from the large, fully designed systems presented.
Emphasis on techniques and insights applicable now.Slide3
Straw Man Motivation:
“Animation in Games is not perfect”Da
Silva et al, Siggraph 08
[Video]
Geometric blending used to adapt existing character animations to novel situations (i.e. walk cycle on steps) does not look physically sound.
Characters do not react to external stimulation that has not been planned for (i.e. throwing boxes at people you are talking to)
They remain kinematic, which – from the point of view of physical interactions – is equivalent to having infinite inertiaSlide4
What does an Interactive Animation System provide?
Characters that preserve the personality and style of their animations, seamlessly interact with a physically simulated world and take orders from the gamepad of abusive players and confused AIs.
Figure 1: Physical feasibility is one of many challenges faced by otherwise content virtual people.Slide5
Will Compelling Characters and Physics make my game Sell?
* figures beyond 2009 are estimatesSlide6
SIMBICON: Simple Biped Locomotion Control
SIGGRAPH ‘07Kang Kang
Yin, Kevin
Loken
,
Michiel
van de Panne
Goal is to create a robust control system for bipeds that can:
Balance
Walk with various gaits
Handle external perturbations such as varying terrain and being pushed
Transition between animations
Approximate animations from motion capture
Last goal seems to be treated as an afterthought
Quality and results are not mentioned
Not quantifiable?
This is a first order concern for games.
Robustness as measured by physical simulation results is treated as the highest priority result.Slide7
Van de Panne’s Animation Pipeline
Each animation is a stored as the gains and parameters of one physical controller.
Various controllers exist, each defined by a finite state machine with various parameters and gains.
Each animation corresponds to one controller that has been trained to emulate the animation using physical actuation.
In the absence of external disturbances, the resulting motion mimics the source animation.
In addition to feeding the Mixer, closed loop feedback is required to determine FSM transitions.
(i.e. Heel Strike)
Mixer linearly blends control parameters rather than poses.
Similar controllers are synchronized such that they are in the same state.
A Balance post process is applied. (see next slide for details)
Integrator receives forces and torques, forward dynamics equations are solved and the world state accordingly
Walk Finite State MachineSlide8
SIMBICON: Simple Biped Locomotion Control
SIGGRAPH ’07 (continued)Kang Kang
Yin, Kevin
Loken
,
Michiel
van de Panne
System is based on PD controllers acting at joints
Each animation has its own individual control system that operates the PD gains and targets with values specified by a simple finite state machine.
While this is not treated as a first order concern, the resulting controllers require very little memory.
Compression ratios of 100:1 are typical.Slide9
Balance control system is an adaption of the original
hopper control system developed by Raibert in the ’80s.
Main considerations are foot placement upon landing and inertial forces due to acceleration of legs in flight
SIMBICON: Simple Biped Locomotion Control
SIGGRAPH ’07 (continued)
Kang
Kang
Yin, Kevin
Loken
,
Michiel
van de PanneSlide10
Raiberts
Hopper is Simple and effective. Unfortunately it cannot balance when stationary.Slide11
[Video]
SIMBICON: Simple Biped Locomotion Control
SIGGRAPH ’07 (continued)
Kang
Kang
Yin, Kevin
Loken
,
Michiel
van de PanneSlide12
Continuation Methods forAdapting Simulated Skills
Kangkang
Yin,
Stelian
Coros
Philippe
Beadoin
,
Michiel
van de Panne SIGGRAPH’08
“Simulated characters in simulated worlds require simulated skills.”
An extension of SIMBICON
Machine learning is used to create new animation controllers based on
Existing controllers
New challenges such as variations in terrain, new goals, etc
Learning framework based on numerical optimization employing continuation methods
i.e. Creating an animation controller for pushing a table from a walking motion capture.Slide13
Continuation Methods for
Adapting Simulated Skills
Kangkang
Yin,
Stelian
Coros
Philippe
Beadoin
,
Michiel
van de Panne SIGGRAPH’08
Continuation Methods:Slide14
Continuation Methods forAdapting Simulated Skills
Kangkang
Yin,
Stelian
Coros
Philippe
Beadoin
,
Michiel
van de Panne SIGGRAPH’08
[Video]Slide15
Ad Hoc Meta-analysis of perceived error in physically simulated characters
David Wu, GDC 09
A number of studies have looked at the relative sensitivity of viewers towards various types of error in physical simulation of humans:
Anna
Majkowska
,
Petros
Faloutsos
, Flipping with physics: motion editing for acrobatics,
Eurographics
animation 2007
Yeuhi Abe , C. Karen Liu ,
Zoran
Popovic
, Momentum-based parameterization of dynamic character motion, Graphical Models 2006
Carol O'Sullivan , John
Dingliana , Thanh Giang , Mary K. Kaiser, Evaluating the visual fidelity of physically based animations, SIGGRAPH 2003Paul S. A. Reitsma , Nancy S. Pollard, Perceptual metrics for character animation: sensitivity to errors in ballistic motion, SIGGRAPH 2003Slide16
Ad Hoc Meta-analysis of perceived error in physically simulated characters
Meta findings regarding sensitivity to error:
Linear Momentum: High
Linear Velocity: Moderate
Angular Momentum: Moderate
Angular Velocity: Low
Conservation of Energy: Low
Gravity Magnitude: Inconclusive
Static Balance: High
Dynamic Balance: High
These findings have been cited in various papers, they are considered valid despite the relatively low sample sizes and relatively poor control procedures of the studies.Slide17
Synthesis of Constrained Walking Skills
Stelian Coros
, Philippe
Beaudoin
, Kang
Kang
Yin,
Michiel
van de Panne Siggraph Asia 08
Given a source walk cycle and model, offline optimization is used to generate many single step walk cycle variations
Each variation is generated to with the optimization goal of a random foot placement offset.Slide18
Synthesis of Constrained Walking Skills
Stelian Coros
, Philippe
Beaudoin
, Kang
Kang
Yin,
Michiel
van de Panne Siggraph Asia 08
Variations are assembled to create what the authors call a “
Step-to-Step Dynamics Model”
or an
“SSDM”
.
The step-to-step dynamics model (SSDM). The nonparametric (example-based) model makes predictions using the results of the offline synthesis. The given dimensions for the state and actions spaces are for the 2D bipeds.Slide19
Synthesis of Constrained Walking Skills
Stelian Coros, Philippe
Beaudoin
, Kang
Kang
Yin,
Michiel
van de Panne Siggraph Asia 08
At run time, the controller uses the SSDM with its reference walk cycle to generate a continuous walk cycle that satisfies constraints discovered in real time.
The SSDM itself stores its data as samples compressed via principle component analysis on the differences between learned steps and the reference walk cycle.
Given the strong correlation between variations the authors discover the only 2-4 principle components are required.
Sampling consists of applying an N dimension blend on reference samples and then combining the result with the reference walk cycle.Slide20
Synthesis of Constrained Walking Skills
Stelian Coros, Philippe
Beaudoin
, Kang
Kang
Yin,
Michiel
van de Panne Siggraph Asia 08
Takeaways
The overall framework provides:
A potential alternative to traditional foot placement IK.
Higher quality results
A potential alternative to animator provided walk cycle transitions or variations
Saves animator timeSlide21
Synthesis of Constrained Walking Skills
Stelian Coros, Philippe
Beaudoin
, Kang
Kang
Yin,
Michiel
van de Panne Siggraph Asia 08
Take-Away: PCA Hypothesis
Principle Component Analysis (PCA) is a technique used for the
lossy
compression of vector data sets.
A 3 DOF analogy might be DXT1 compression of a single quad of pixels. In this case only the first Principle component is maintained and this 3D vector is the is the difference between the two end point colors.
The early PCA vectors resulting from all poses of a character are strongly correlated with the signature style and physical characteristics of that character.Slide22
Not Mickey Mouse
Synthesis of Constrained Walking Skills
Stelian
Coros
, Philippe
Beaudoin
, Kang
Kang
Yin,
Michiel
van de Panne Siggraph Asia 08
Take-Away: PCA Hypothesis
When performing inverse kinematics, using PCA vectors as the degrees of freedom and weighting each in a manner inversely proportional to it’s
Eigenvalue
, a least squares IK solution is likely to represent a pose variation that correlates highly with the preferred poses of that character.
Figure 3: 1
st
Principle component vector consists of DOF values that produce a characteristic pose.Slide23
Synthesis of Constrained Walking Skills
Stelian Coros, Philippe
Beaudoin
, Kang
Kang
Yin,
Michiel
van de Panne Siggraph Asia 08
Take-Away: PCA Hypothesis (cont)
There are a few conditions that the dataset used to generate the Principle Component Vectors must meet for this hypothesis to hold:
Should be representative of the entire range of motions preferred poses
Should be representative in a quantitative, statistical sense
Appropriate DOF should form the initial basis
Forces or Poses?
May require doctoring of the input i.e. weighting the std deviation of underrepresented poses.
Figure 4: Animation set has been over-trained. The 1
st
Principle Component vector is not ideal.
Not Mickey MouseSlide24
Flexible Registration of Human Motion Data with Parameterized Motion Models
Yen-Lin Chen, Jian Yuan
Miny
,
Jinxiang
Chaiz
An initial set of animations is collected
Animations are time-warped such that each “cycle” takes 2 seconds
PCA is performed on these animations and the 20 top
Eigenmodes
are maintained
These are run through a “physics filter”, which iteratively modified them until they are physically valid
Figure 9:
First Principle ComponentSlide25
Flexible Registration of Human … (cont)
All animations are projected onto the resulting Principle Components
The authors claim that 20 Eigen modes is enough for mostly lossless representation
New animations are “registered” into the motion database and stored as linear combinations of the components
The authors state the following applications:
Effective motion retargeting
Kinematics, dynamics
and
style
Physics improves believability of motions
Implicitly enforced by the Components
Consistent, realistic motion generation from sparse data.
Registration enforces the constraints implied by the sparse data and fills in the blanks with motion from existing motions.Slide26
Animating responsive characters with dynamic constraints in near-
unactuated coordinates
Yuting
Ye, C. Karen Liu
Combines kinematic motion with dynamic reactions that are constrained to minimize interference with muscles essential to the base motion.
To accomplish this, the system does the following:
Projects motions Slide27
Animating responsive characters… (cont)
Specifically, the system does the following:
For each frame of animation, the actuator torques required to drive the motion are computed
Principle Component Analysis (PCA) is performed on all frames of all animations.
Eigenvectors with the
lowest
8
Eigenvalues
are collected, these form the basis for dynamic reactions.
The biped’s dynamics are represented using 8 DOF, each being a scalar multiple of one of these vectors.
For each frame of animation the forward dynamics of the system are projected onto this basis (J)
f =
Mx
’’ +…
Jf
= (
JMJ
t
)(
Jx
’’) Jtq’’ = x’’Slide28
Animating responsive characters… (cont)
Due to the
orthogonality
guarantees of PCA, these Components act in the null space of the most significant Components.
They correspond to motions requiring little or no muscle action
Hence the term “
unactuated
”
Furthermore the composite effect of these components tend to cancel each other out with respect to side effects at the primary, well actuated DOF.Slide29
Animating responsive characters… (cont)
[video]Slide30
Animating responsive characters… (cont)
Extremely efficient
Fairly convincing
Characters display motion consistent with their style
Primary issues is that for relatively strong perturbations the response is not convincing – you would expect perturbation to effect the walk cycle as a whole, unfortunately the problem statement forbids this.
Changing the problem statement would invalidate this technique.Slide31
Simulating Biped Behaviors from Human Motion Data
Kwang Won, Sok
Manmyung
Kim,
Jehee
Lee
ACM Transactions on Graphics, July 2007.
Uses machine learning to construct physical controllers that can emulate motion capture data while maintaining balance
Controllers operate on an animator specified biped.
The system behaves best when the virtual biped is similar to captured subject.Slide32
Simulating Biped Behaviors … (cont)
Animators construct a finite state machine that describes potential transitions between animations.
Further controllers are generated and trained for each transition
All animations can transition to the balance/ recovery states.Slide33
Controllers consist of parameterized PD servos.
Training consists of finding feedback/time varying parameterization for the servos using a numerical optimization technique with the objective function of minimizing error between the motion capture and the simulated character.
The optimization problem is highly nonlinear
Simulating Biped Behaviors … (cont)Slide34
Optimization strategy:
Initialize solution to {0}
Scatter random points about the current solution
Optimize each point using a gradient descent technique
Select best point(s) and repeat
Slow, brute force method.
Given optimization landscape, faster search methods such as Conjugate Gradients and Quasi Newton fail to converge, so the others use a downhill simplex descent
In the second phase of learning (transitions) the optimization parameterizations consist of a linear combination of existing controllers
Speeds up the optimization considerably
Maintains style of source animations
Key takeaway.
Simulating Biped Behaviors … (cont)Slide35
Key Takeaways
PD servo based physical controllers with very few biped/locomotion specific heuristics are feasible
Blending controllers trained on animation with a specific style seems yield new controllers that maintain this style.
Effectiveness demonstrated for transitions, will blending work with more general extrapolations?
Simulating Biped Behaviors … (cont)Slide36
Interactive Simulation of Stylized
Human LocomotionMarco da
Silva, Yeuhi Abe, Jovan Popović - SIGGRAPH 08
Animating natural human motion in dynamic environments is difficult for various reasons, including complex geometric and physical interactions.
Simulation combined with physical controllers has been demonstrated to provide an automatic solution to parts of this problem by the robotics community.
In the field of video game animation, we require style and personality in addition to competent locomotion.Slide37
Describes the systematic synthesis of controllers that can reproduce a range of locomotion styles in interactive simulations.
Given a reference motion that describes the desired style, a control system is derived that can reproduce that style in simulation and in new environments.
Figure 1
Numerical optimization provides a framework for computing a solution that is optimal in the context of any number of prioritized goals.
In this case the goals are balance and style, the optimization technique is quadratic programming.
Simulation of Stylized … (cont)Slide38
A balance strategy is pre-computed for the given style using automated analysis of linear time varying approximations.
By tailoring the balance strategy in this manner, a controller preserves the style better than a more cautious strategy
Simulation of Stylized … (cont)Slide39
The Style feedback loop tracks individual joint angles to compute the accelerations needed to preserve the given style.
Reference motions guide both the style and balance feedback.
The style feedback aims to preserve the nuances of the motion
The balance feedback seeks to adapt the motion of three balance critical segments
The control algorithm computes a final set of forces by maintaining a desired tradeoff between the balance and style feedback.
Simulation of Stylized … (cont)Slide40
Dynamic balance is a hard problem
Many
dof
-> infinite solutions
Numerically unstable -> most states invalid
Valid states may be balanced, but awkward
i.e. if you have a tendency to lean to the left, cranking your head to the right may help you to balance but it is not the most natural way to address the issue.
Simulation of Stylized … (cont)Slide41
To mitigate these problems, the authors:
Project the higher detail (60 DOF) biped onto a low detail (9 DOF, 6 Actuator DOF) model
Use a piecewise linear model to approximate the non-linear dynamics of the reduced detail model.
i.e
F=MA, where M is a constant matrix.
Use a Linear Quadratic Regulator (LQR) that tracks the animation data
Assumes that poses are dynamically stable
Eliminates the need to explicitly search for
any
dynamically stable state.
Simulation of Stylized … (cont)Slide42
The 3 link model’s root is the contact foot and it has two 3 DOF actuators acting at the hips
One for the torso
One for the swing leg
The linearalized dynamic equations are
precomputed
, rolled together with the equations necessary for the LQR, factored and stored for each time step.
This amounts to about 20k per second of animation
The full character would require about 2 meg per second.
Aside: Is 60hz necessary?
Simulation of Stylized … (cont)Slide43
[Video]
Interactive Simulation of Stylized
Human Locomotion
Marco
da
Silva, Yeuhi Abe, Jovan Popović - SIGGRAPH 08Slide44
The results are promising:
Maintains lifelike human motion in dynamic environments, which is difficult to accomplish with kinematics or dynamics aloneTransforms a single recorded motion, valid for one environment only, into a general purpose action that can be used in many other settings or even composed with other actions to create versatile characters.
Performance reasonable in worst case, plenty of room for optimizations in normal situations.
Simulation of Stylized … (cont)Slide45
Combines blended animation with the results of a physical simulation
6 DOF springs are attached to the root of the dynamic model to keep it from falling over.
Dynamo: dynamic, data-driven character control with adjustable balance
ACM Siggraph Video Game Symposium
Pawel
Wrotek
,
Odest
Chadwicke
Jenkins, Morgan McGuire Slide46
While not mentioned explicitly by the authors, blending in world space helps preserve the momentum characteristics of the source animations, thus making the result more physically plausible (if the source animations is physically plausible)
Dynamo: dynamic, data-driven character control with adjustable balance
ACM Siggraph Video Game Symposium
Pawel
Wrotek
,
Odest
Chadwicke
Jenkins, Morgan McGuire Slide47
The reason for the improved preservation of momentum in world space is the fact that the Hessian describing the system dynamics in world space coordinates varies less across poses than the corresponding Hessian describing the system dynamics in joint space.
This insight is arguably the primary take-away from the paper.
A natural corollary is that physical controllers will usually be more robust when operating in world space.
The authors point out this observation in the diagram below.
Dynamo: dynamic, data-driven character control with adjustable balance
ACM Siggraph Video Game Symposium
Pawel
Wrotek
,
Odest
Chadwicke
Jenkins, Morgan McGuire Slide48
Animating reactive motion using Momentum-based Inverse Kinematics
Taku Komura, Edmond S. L. Ho and Rynson
W. H. Lau
Virtual Worlds 2005Slide49
Animating reactive motion … (cont)
System designed for dynamic reactions that allow for recovery.
Intended to bridge the gap between Ragdolls and layered hit reactions for mild impacts that do not affect the characters momentum or trajectory
Animations with different stepping strategies form the basis for reactions
Foot placement of animations is modified counteract the characters momentum.
This balance strategy is basis for
Asimo’s
control system
The ideal foot placement point is called the Zero Moment PointSlide50
Animating reactive motion … (cont)
Momentum Based IK consists of:
Constraining a kinematic motion to conserve momentum each frame
This is achieved by running a physical simulation in which only the non-essential DOF have finite inertia.
Initial DOF accelerations are computed using backwards differencing between the prior state and the animation sample.
Playing back the new, modified animation.
System must be able to maintain velocity/momentum, along with state across framesSlide51
Animating reactive motion … (cont)
Seems quite effective in theory
Unfortunately I was unable to find a video
Fits in nicely with existing systems
Performance cost is low
Not too far from what some people do currentlySlide52
Interactive Dynamic Response for Games
Victor B. Zordan, Adriano Macchietto, Jose Medina,
Maarc
Soriano, Chun-Chih Wu, I3D 07Slide53
Assumes many hit reactions (paper cited 500 in their tests)
Finding the appropriate reaction becomes a bottleneckUses an optimized search tree based on a Support Vector Machine (SVM)
Quality of SVM is highly dependent on the DOF used in classification
Tried many combinations
Optimal set of DOF was 20 in total:
Momentum (linear and angular) at various time steps surrounding the impact
Center of mass at same time steps
Orientation at time of impact
Once a good SVM is constructed, real time search is feasible for a large number of motions
Time to find animation was cited as ~2 ms
vs
2 seconds prior to the SVM.
Interactive Dynamic Response … (cont)Slide54
[video]
Interactive Dynamic Response … (cont)Slide55
Anticipation from Example
Victor Zordan, Chun-Chih Wu, et al.
Time
q(t)
Impact
Reaction Animation
Blend
Find Reaction
Simulation
Detect Impact
Defensive MotionSlide56
Simple Steps for Simply Stepping
Victor Zordan, Chun-Chih Wu, et al.Visual Computing 2008
“Inspired” by the paper Animating reactive motion using Momentum-based Inverse Kinematics
Uses Momentum
based IK to drop animation
Uses principles of Dynamic Balance/Zero moment point to synthesize steps from exampleSlide57
Wrap: Research, Animation and Physics
Figure 3: When shipping a game, elegance and simplicity occasionally slip.
When publishing a paper, elegance and simplicity occasionally slip.
There is a lot of great research and many ideas with strong potential.
When evaluating ideas, it is important to remember that the whole is not always greater than the sum of its parts.Slide58
Characters that preserve the personality and style of their animations
, seamlessly interact with a physically simulated world and take orders from the gamepad of abusive players and confused AIs.
Just about every successful entertainment medium features compelling characters – personality and style are essential.
Animators are often better at creating compelling characters than random algorithms/equations.
In games we have Animators (hopefully) and many algorithms/equations.
Presenting
optimal
relevant algorithms that preserve the personality and style of characters/animators is the focus of this lecture.Slide59
Interactivity defines a video game.
A non-interactive game is a movie.Physics is a powerful tool that can enable Interactivity through simulation.There are other ways but
they suck
They are:
BEYOND THE SCOPE
OF THIS LECTURE
Characters that preserve the personality and style of their animations,
seamlessly interact with a physically simulated world
and take orders from the gamepad of abusive players and confused AIs.Slide60
Characters that preserve the personality and style of their animations, seamlessly interact with a physically simulated world and
take orders from the gamepad of abusive players and confused AIs.
I found two papers addressing this concern
Ideas and or results are depressing
Not worth mentioning
They are:
BEYOND THE SCOPE
OF THIS LECTURESlide61
Questions & Answers
Figure 1: Physical feasibility is one of many challenges faced by otherwise content virtual people.Slide62
Past End of File.
You should not be here.
Please shut down current application.
Reboot if necessary.Slide63
Problem: Game is Interactive.
Animation Database is not.
Modeling the large space of animations that may be required at any moment due to open ended nature of player actions is non-trivial
Rule based controllers handle this via animation selection guided by heuristics.
i.e. pre-authored decision tree populated with animation clips.
Many problems and failure cases
Still the basis for many animation systems
Apparently academia sees this as a problem - over the last five years, the amount of research dedicated to this problem has grown exponentially.Slide64
The Arsenal
Meta techniques
Generalizing from examples
Interpolate and/or extrapolate from source animations
Weighted blend of source (convex interpolation)
Transform source to a new basis using direct techniques (i.e. PCA)
Use source to train/derive more complex models.
Splice to micro motion snippets and transition rapidly
Synthesis
Physical Simulation
Rag dolls
Secondary dynamics
Kinematic and/or Dynamic Constraints
Inverse Kinematics
“Physics Filter”
Physically based controllers
General Human biometric data
The best solutions tend to be hybrid implementations of these techniques, as each has its limitations and strengths.
Hence the term “Arsenal”: Use, understand, build, augment, combine, patentSlide65
Generation and Visualization of Emotional States in Virtual Characters
Diana Arellano , Javier
Varona
, Francisco
Perales
CASA 08
The aim of this paper is to model characters that have PERSONALITY, feel EMOTIONS and can MANIFEST EMOTIONAL STATES
The motive is to create unique, real, distinguishable individuals depending on their
personalities and emotional
states