Paul Ruth RENCI UNC Chapel Hill pruthrenciorg Motivation Motivation Are the current GENI technologies capable of effectively supporting the computing needs of domain science Motivation Are the current GENI technologies capable of effectively supporting the computing needs of domain sci ID: 399964
Download Presentation The PPT/PDF document "GENI Science Shakedown" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
GENI Science Shakedown
Paul Ruth
RENCI / UNC Chapel Hill
pruth@renci.orgSlide2
MotivationSlide3
Motivation
Are the current GENI technologies capable of effectively supporting the computing needs of domain science? Slide4
Motivation
Are the current GENI technologies capable of effectively supporting the computing needs of domain science?
What are the benefits and limitations of using GENI for domain science?Slide5
ObservationsSlide6
Observations
GENI has matured to the point where it is now time to
evaluate
its capabilities with respect to
domain science applications
“at scale” and use the results of this evaluation to
inform
the direction of
future GENI development
.Slide7
Observations
GENI has matured to the point where it is now time to
evaluate
its capabilities with respect to
domain science applications
“at scale” and use the results of this evaluation to
inform
the direction of
future GENI development
.
There is an opportunity
to introduce domain scientists
to the
GENI-way-of-thinking
about networking and distributed systems in order to seed the dissemination of GENI technologies to existing and future high throughput and high-performance computing centersSlide8
Personnel
Computer Scientists:
Paul Ruth,
pruth@
renci.org
Anirban
Mandal
,
anirban@
renci.org
Domain Scientists:
Jeffery L. Tilson
,
jtilson@
renci.org
Brian Blanton,
bblanton@renci.orgSlide9
Project Goals
Apply
the GENI experiment workflow
to domain science applications and evaluate GENI’s ability to
run domain science experiments
P
erformance
Ease
-of-
use.
Provide
feedback to GENI rack developers
on the current capabilities with respect to science applications as well as target areas for improvement
.
Build a
tool
for domain scientists to create
RSpec
from high-level descriptions of high-throughput and high-performance applications.Slide10
Project Goals
Apply
the GENI experiment workflow
to domain science applications and evaluate GENI’s ability to
run domain science experiments
(performance and ease-of-use).
Build a
tool
for domain scientists to create
RSpec
from high-level descriptions of high-throughput and high-performance applications.
Provide
feedback to GENI rack developers on the current capabilities with respect to science applications as well as target areas for improvement.
Bonus Goal
: Attract domain scientists to GENISlide11
Domain Science Applications
MotifNetwork
Jeffery
L.
Tilson
(RENCI)
High
Throughput Computational
Genomics
Identification
of functional domains and the identification of conserved functional relationships across large numbers of genomes
.
Workflow ensemble (Pegasus/HTCondor
)
Task data sets on the order of 50-100 GB.
Implicit iteration
Scales to 1000s of tasksSlide12
Domain Science Applications
ADCIRC
Brian Blanton (RENCI)
Storm
surge and tide model
Finite element model
MPI tightly coupled
Urgent Computing
Approved by FEMA for computing storm surge flood hazard simulations
Used for Digital Flood Insurance Rate Maps (DFIRMs)
Scales to 10000+ MPI processesSlide13
Experiments
Provisioning o
verhead
creating
, destroying, and modifying slices.
GENI for big-data.
Large
workflow inputs/
output
GENI for MPI.
Tightly
coupled Slide14
Outreach to Domain Science
Tool for deploying domain science applications easily
.
RSpec
generation of a Pegasus/
HTCondor
cluster and MPI cluster.
Extendable to other science slice types.
Introducing domain scientists to GENI.
Feedback from domain scientists to inform future GENI
developmentSlide15
Final thoughts on what we will find
Most problems will be on the edges
Disk I/O
Local disk
Network storage
Network virtualization mechanism
Tradeoffs of:
virtio
, OVS,
linux
bridges, SR-IOV
More overhead than we realize
Performance isolationSomething unexpected
15Slide16
First year goals
GEC 19
Applications working on
InstaGENI
and
ExoGENI
Initial performance profile
GEC 20
Deeper performance profile
Initial suggestions for rack developers
16Slide17
ThanksQuestions?
17