Bill Spotz Sandia National Laboratories SIAM Conference on UQ April 5 2016 Thanks Jeff Fike SNL New Mexico Andy Salinger SNL New Mexico Irina Tezaur SNL California Eric Phipps SNL New Mexico ID: 675838
Download Presentation The PPT/PDF document "UQ for the Global Atmosphere Using Concu..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
UQ for the Global Atmosphere Using Concurrent Samples
Bill Spotz, Sandia National LaboratoriesSIAM Conference on UQ, April 5, 2016Slide2
Thanks
Jeff Fike, SNL New MexicoAndy Salinger, SNL New MexicoIrina Tezaur
, SNL CaliforniaEric Phipps, SNL New Mexico2Slide3
Outline
Aeras, a next-generation global atmosphere modelNext generation capabilitiesSoftware infrastructureConcurrent samples
Leveraging embedded UQ technologiesSoftware infrastructureNumerical experimentSummary and Future Work
3Slide4
Aeras is a
Sandia LDRD project whose goal is to develop a “Next Generation Global Atmosphere Model”Utilize state-of-the-art librariesImplement proven dycore
numerics: spectral elements w/hyperviscosityDemonstrate physics parameterization with simple cloud physicsAeras leverages leading-edge, massively parallel, C++ computational technologies from Sandia:
Albany (multiphysics application code base)Trilinos (solvers, discretizations, meshing, coupling, perf portability, …)Dakota (optimization, uncertainty quantification)Next generation capabilities
Performance portability, enabled by the Kokkos (Trilinos) programming modelUncertainty quantification, enabled by Stokhos (Trilinos) intrusive stochastic UQ package, and Dakota4Slide5
5
Element Level Fill
Material Models
Sensitivities
Field Manager
Discretization Library
Remeshing
UQ Solver
Nonlinear Solver
Time Integration
Optimization
Objective Function
Local Fill
Mesh Database
Mesh Tools
I/O Management
Input File Parser
Utilities
UQ (sampling)
Parameter Studies
Mesh I/O
Optimization
Geometry Database
Discretizations
Derivative Tools
Adjoints
UQ / PCE
Propagation
Constraints
Error Estimates
Continuation
Constrained Solves
Sensitivity Analysis
Stability Analysis
V&V, Calibration
Parameter List
Verification
Visualization
PostProcessing
Adaptivity
Model Reduction
Memory Management
System Models
MultiPhysics Coupling
OUU, Reliability
Communicators
Partitioning
Load Balancing
Analysis Tools
(
black-box)
Physics Fill
Composite Physics
Data Structures
Direct Solvers
Linear Algebra
Architecture-
Dependent Kernels
Preconditioners
Iterative Solvers
Eigen Solver
System UQ
Analysis Tools
(
embedded)
Matrix Partitioning
Inline Meshing
MMS Source Terms
Grid Transfers
Quality Improvement
Mesh Database
Solution Database
Derivatives
Regression Testing
Bug Tracking
Version Control
Software Quality
Porting
Performance Testing
Code Coverage
Mailing Lists
Release Process
Unit Testing
Web Pages
Build System
Backups
Verification Tests
DOF map
Multi-Core
Accelerators
Linear Programming
Graph Algorithms
Data-Centric Algs
SVDs
Map-Reduce
Network ModelsSlide6
Software Infrastructure
6
Tpetra
Kokkos
Epetra
Trikota
Stokhos
Evaluators
Note:
global atmosphere models have specializations and optimizations not originally supported by Albany:
horizontal vs. vertical bias
need for shell elements
spectral elements
explicit time stepping
transient embedded UQSlide7
Albany Generic Physics Interface
7Slide8
Concurrent Samples
There is another generic template type that determines the mathematical scalar typeThis would typically be
double in practiceWe can specialize to be an Array<double>
Length of array equals number of samples to computeInitialization…Overload operatorsWe can now leverage common objectsMeshMatricesEtc…In theory, hardware that requires more data to perform efficiently, will
8Slide9
Governing Equations
Shallow Water Equations3D Hydrostatic Equations
9
Legend
Horizontal velocity
Vorticity
Coriolis parameter
Radial unit vector
Horizontal gradient operator
Geopotential
Atmospheric thickness
Vertical coordinate
Vertical velocity
Gas constant
Temperature
Virtual temperature
Pressure
Specific heat, constant pressure
Total pressure time derivative
Legend
Horizontal velocity
Vorticity
Coriolis parameter
Radial unit vector
Horizontal gradient operator
Geopotential
Atmospheric thickness
Vertical coordinate
Vertical velocity
Gas constant
Temperature
Virtual temperature
Pressure
Specific heat, constant pressure
Total pressure time derivativeSlide10
Numerical Approach
Quadrilateral spectral elements tile the surface of the sphereNode points = Gauss-Lobato points = quadrature points → diagonal mass matrixContinuous
GalerkinRunge-Kutta 4th order time stepping
Finite difference formulation in verticalHyperviscosity for stabilization10Slide11
Numerical Experiment
Shallow Water EquationsTest Case 5 [Williamson]Zonal Flow Over an Isolated MountainMountain height used as random input variable
For each experimentWe run concurrent samples of size nWe repeat, as necessary, to achieve an ensemble of size 32
Sensitivities of latitudinal velocity with respect to mountain heightSlide12
Numerical Experiment
12Slide13
Aside: Performance Portability
13Slide14
Summary
We have developed Aeras, a next-generation global atmosphere model, that leverages leading-edge computational technologies from Sandia National LaboratoriesAeras is built on top of Albany, a multi-physics finite element code that takes advantage of
Trilinos for high-performance computing solver technologies, and Dakota for optimization and uncertainty quantificationThrough a creative use of embedded UQ, we have achieve a
doubling of the efficiency of our shallow water solver by running ensemble samples concurrently14Slide15
Future Work
Upgrade from an older set of linear algebra classes (Epetra) to a newer package that supports C++ templates (Tpetra)This will allow using the latest version of
Aeras/Albany, which in turn will allow concurrent samples to be used in conjunction with:Our performance portability
effortsSpectral elements3D hydrostatic equations15