Keith Dalbey PhD Sandia National Labs Dept 1441 Optimization and Uncertainty Quantification Michael Levy PhD Sandia National Labs Dept 1442 Numerical Analysis and Applications Sandia is a multiprogram laboratory operated by Sandia Corporation a Lockheed Martin Company for the Unit ID: 688899
Download Presentation The PPT/PDF document "Generation of Pareto Optimal Ensemble..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Generation of Pareto Optimal Ensembles of Calibrated Parameter Sets for Climate Models
Keith Dalbey, Ph.D.Sandia National Labs, Dept 1441, Optimization and Uncertainty QuantificationMichael Levy, Ph.D.Sandia National Labs, Dept 1442, Numerical Analysis and Applications
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy’s National Nuclear Security Administration under Contract DE-AC04-94AL85000.
December 12-17, 2010Slide2
Outline
MotivationApproach: Pareto EnsembleWhat Does “Pareto Optimal” Mean?Finding a “Pareto Optimal” EnsembleResults of Tuning Climate ModelSummary & Future WorkReferencesJackson et al, “Error reduction and convergence in climate prediction,” Journal of Climate, 2008.Eddy & Lewis, “Effective generation of pareto sets using genetic programming,” Proc. of ASME Design Engineering Technical Conference, 2001.Dalbey & Karystinos, “Fast generation of space-filling latin hypercube sample designs,” Proc. of 13th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, 2010.Slide3
Motivation
Calibrating (tuning) climate models choosing values of model parameters to predict wellIs difficult becauseThey have many inputs and outputsDiverse parameters sets can match observations similarly wellErrors can compensate: “2 wrongs can make a right” under historical conditionsClimate change (new conditions) might expose a previously hidden mis-calibration, so…History matching is necessary but not sufficient for good predictions.The future is uncertain, but we can quantify the uncertainty (estimate statistics) for possible future climates.Slide4
Approach: Pareto Ensemble
How can we make good statistical predictions? Use a diverse ensemble of “good” parameter sets to determine the range/spread of possible future climatesQUESTION: What’s the definition of a “good” parameter set? There are multiple outputs and what’s good for one output can be bad for another.(AN) ANSWER: It’s Pareto optimal. A point (parameter set) is Pareto optimal if there is no other point that is as good or better than it in ALL outputs.What does the “Pareto” mean? It’s just the name of the person who discovered it… Vilfredo Federico Damaso Pareto was an Italian engineer, sociologist, economist, and philosopher.Slide5
What Does “Pareto Optimal” Mean?
2D Pareto front schematicsSlide6
What Does “Pareto Optimal” Mean?
Usually, the current approx. of the true Pareto front.The Pareto front defines the “zero sum game” of all optimal compromises you could make.Unlike a weighted combination of objective functions, it lets you choose a specific compromise/ combination AFTER the optimization is complete.It does NOT say which compromise/combination is best, just what all the “optimal” choices are.It says “Don’t choose anything Pareto non-optimal because there’s something better in all criteria.”Slide7
Finding a “Pareto Optimal” Ensemble
Used the Multi Objective Genetic Algorithm (MOGA) in DAKOTA’s (Design Analysis Kit for Optimization and Terascale Applications) JEGA (John Eddy’s Genetic Algorithm) sub-packageGA’s typically need 1000’s of simulations, I could only afford 1000… Used test problem (find surface of radius=1 6D hyper-sphere in input space, 10 outputs) to tune MOGA settings and initial population (space-filling, specifically Binning Optimal, Symmetric Latin Hypercube Sampling, or BOSLHS), for:Large Pareto EnsembleMean radius close to 1Uniform spreadSmall radius varianceSlide8
Finding a “Pareto Optimal” Ensemble
Use DAKOTA’s MOGA on a test problem with 6 inputs and 10 outputs; true solution is a radius 1 hypersphereDefault Monte Carlo seedPDF’s of the Pareto Ensemble’s# of pointsPoint spreadMean radiusStandard deviation of radius
1
2
3
4Slide9
Finding a “Pareto Optimal” Ensemble
Use DAKOTA’s Multi Objective Genetic Algorithm on a test problem with 6 inputs and 10 outputs true solution is a radius 1 hypersphereBOSLHS seedDefault Monte Carlo seedSlide10
Results of Tuning Climate ModelSlide11
Summary & Future Work
Climate model parameters that match history well might not predict well (climate change might expose a previously hidden mis-calibration of parameters).Plan: Use a diverse ensemble of “good” (Pareto optimal) parameter sets to determine the range/spread of possible future climates.Used MOGA to find a (very large) Pareto optimal ensemble of calibrated parameter sets.Next steps: down select Pareto optimal ensemble, andsimulate smaller ensemble out to 2100.Slide12
Some “Good” Parameter Sets