National Resource for Biomedical Supercomputing HPC User Forum Houston TX April 6 2011 Contributors Joel Stiles PhD MD Director NRBSC Markus Dittrich Senior Scientific Specialist ID: 717498
Download Presentation The PPT/PDF document "The Anton Project at the" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
The Anton Project at theNational Resource forBiomedical Supercomputing
HPC User Forum
Houston, TX
April 6, 2011Slide2
ContributorsJoel Stiles, Ph.D, M.D. Director, NRBSCMarkus Dittrich, Senior Scientific Specialist NRBSCRalph Roskies, Scientific Director, PSCJim Kasdorf, Director of Special Projects, PSCSlide3
DisclaimerNothing in this presentation represents an official view or position of the U.S. National Institutes of Health, nor of D.E. Shaw Research, nor of the Pittsburgh Supercomputing Center nor of Carnegie Mellon University. All material is from public sources. All errors are my own (JK). Slide4
The PSC Biomedical InitiativeFounded in 1987Funded by U.S. National Institutes of HealthFirst extramural NIH supercomputing programRenewed every five yearsWith 2006 grant, renamed: National Resource for Biomedical SuperComputing Slide5
National Resource for Biomedical SupercomputingPerforms leading-edge research in high performance computing and the life sciencesFosters exchange between PSC expertise in computational science and biomedical researchers nationwideSlide6
National Resource for Biomedical Supercomputing Computational biomedical research Outreach to the national biomedical research communityUser trainingSoftware distributionPublicationsOnline courses and workshop webcasts.Slide7
NRBSC Research AreasMicrophysiologyVolumetric visualization and analysisComputational structural biologySlide8
AntonAnton van Leeuwenhoek(October 24, 1632 – August 26, 1723)Perfected the microscope“Father of microbiology”Slide9
Anton Massively parallel supercomputer Designed and built by D.E. Shaw Research (DESRES) Special-purpose system for Molecular Dynamics (MD) simulations of proteins and other biological macromoleculesSlide10
David E. Shaw B.S. / M.S. UCSDPh.D. Stanford, 1980Columbia CS Department until 1986Formed D.E. Shaw Group in 1988“King Quant” (Fortune, Feb. 5,1996)Assets Under Management ~$20BNet worth: $2.2B (March 2011)Slide11
David E. Shaw Founded DESRES, 2002Senior Research Fellow, Columbia University, 2005PCAST appointments, 1994 and 2009Council of Competitiveness Executive CommitteeFellow of AAASAmerican Academy of Arts and SciencesAmerican Association for the Advancement of ScienceSlide12
D.E. Shaw Research(www.deshawresearch.com)Independent research laboratory Conducts basic scientific research in computational biochemistry Involved primarily in:The design of novel algorithms and machine architectures for high-speed molecular dynamics (MD) simulations of proteins and other biological macromoleculesThe application of these simulations to basic scientific research in structural biology and biochemistry, and to the process of computer-aided drug designSlide13
The MD ChallengeMolecular dynamics (MD) simulation allows scientists to study the microscopic dynamics of biomolecular systems and thus gain crucial insight into processes such as:Protein foldingHow drug molecules interact with proteins and nucleic
acidsThe mode of operation of many important cellular proteins such as ion channels or other transport
proteins
Importantly, most relevant biological events take place on a timescale of
milliseconds
(10
-3
s) or
longerSlide14
AntonAnton is a special-purpose supercomputer designed to specifically address the need of MD simulations to access the biologically-relevant realm of millisecond timescales.Anton was designed to dramatically increase the speed of MD simulations.Anton runs MD simulations fully in hardware (as opposed to running MD software on general purpose hardware
).Slide15
AntonCompared to the previous state-of-the-art Anton provides a speedup of ~100x, rendering millisecond timescales attainableAnton uses custom-designed ASICs and novel simulation algorithmsSlide16
Anton Processing NodeSlide17
Anton ASICSlide18
A 512-Node Anton MachineSlide19
Anton Benchmarks
Computational platform
Speed (ns/day)
single CPU codes
~1
Parallel supercomputers
~100
Anton (512 node machine)
17,400
DHFR benchmark, simulated biological time)Slide20
MD Simulations on AntonStandard, unbiased MD runs in NVE, NVT, and NPT ensembles are possible.Simulate proteins, lipids, water, RNA, DNA.On a 512-processor Anton, simulation systems with up to 120,000 atoms can be simulated effectively.
Supports standard MD forcefields
(CHARMM, AMBER, OPLSAA, ...).
Input is in Maestro format (similar to what is used in
the DESRES
Desmond
code).Slide21
Anton BenchmarksChemical system (PDB ID)System Size (# atoms)
Speed (
μ
s/machine day)
DHFR (5dfr)
23,558
17.4
aSFP (1sfp)
48,423
11.7
FtsZ (1fsz)
98,236
5.7
T7Lig (1a01)
116,650
5.5
All simulations used 2.5-femtosecond time steps with long-range interactions evaluated at every other time step and a Berendsen thermostat applied every 100 time stepsSlide22
Anton @NRBSC/PSCThe availability of Anton at the National Resource for Biomedical Supercomputing (NRBSC) at PSC was made possible by a partnership with DESRES.DESRES made available an Anton machine without cost for non-commercial research use by not-for-profit institutions. The Anton at NRBSC is the only one available outside DESRES.Slide23
Anton @NRBSC/PSCOperations are funded by a two-year $2.7 million grant from the National Institute of General Medical Sciences (NIGMS), part of NIH.This is the first time NIH has provided funds to make available a supercomputing system for national use. Grand Opportunities (GO) grant mechanism
from ARRA funds One of only 14 GO awards from NIGMSSlide24Slide25
Anton Hardware @NRBSC512-node Anton production machineFour cabinets, each with 128 special-purpose ASICs
Operational since October 2010
64-node
Anton test machine
One cabinet
with 64
special-purpose
ASICs
Operational since May 2010
Access to both Anton back-ends is via a Linux front-end running a standard PBS queuing system.Slide26
Anton Hardware @NRBSCAntonFS storage clusterDedicated to Anton MD trajectory storageLustre-based file system with ~450 TB of storage
Kollman analysis cluster
Dedicated to analysis of MD trajectories generated on Anton
Four
nodes each with 12
cores (2 * 3.33GHz Intel Xeon
Westmere
EP X5680 6-core), 96
GB of RAM and one
NVIDIA
Tesla C2050
card, QDR IBSlide27
Anton AllocationsIn spring of 2010 NRBSC solicited proposals from the scientific community; 67 proposals were received.Proposal review and assignment of allocations was conducted by a committee convened by the National Research Council at the National Academies of Science.
15 proposals were awarded 100,000 node-hours each during the first phase of the project (October 2010 through March 2011). Slide28
Anton Allocations32 proposals were awarded ~50,000 node hours each during the second phase of the project (April 2011 through September 2011). Slide29
Science Projects on AntonThe range of science topics proposed for simulation on Anton is quite diverse:Studies of membrane channels and transporters
Protein folding
Design and evaluation of novel enzyme catalysts
DNA sequencing
Protein dynamics
Signal transduction
Force field development
…. Slide30
Science Projects on AntonCurrently (as of March 2011), the investigators of phase 1 are wrapping up their simulations and are conducting preliminary analyses of their generated data. Slide31
References“New Architectures for a New Biology”, Martin Deneroff, February 22, 2007 www.nersc.gov/projects/SDSA/meetings/SIAM_CSE07/SIAM_2-07-desres.pdf“Anton, a special-purpose machine for molecular dynamics simulation”, D.E. Shaw, et al., Communications of the ACM, Volume 51, Issue 7, July 2008 “Herculean Device for Molecular Mysteries”, John Markoff, The New York Times, July 8, 2008Slide32
References“Millisecond-scale molecular dynamics simulations on Anton”, D.E. Shaw, et al., SC’09 Proceedings“Exploiting 162-Nanosecond End-to-End Communication Latency on Anton”, Ron O. Dror, et al., SC10 ProceedingsSlide33
References“Atomic-Level Characterization of the Structural Dynamics of Proteins”, D.E. Shaw, et al. Science 15 October 2010: Vol. 330 no. 6002 pp. 341-346“Custom-Built Supercomputer Brings Protein Folding Into View”, Robert F. Service, Science 15 October 2010: 308-309.Slide34
References“Anton Supercomputer Proves Its Mettle”, Chemical & Engineering News, October 14, 2010 http://pubs.acs.org/cen/news/88/i42/8842notw1.htm“Supercomputer sets protein-folding record”, Nature, October 14, 2010, http://www.nature.com/news/2010/101014/full/news.2010.541.htmlSlide35
Jim Kasdorfkasdorf@psc.eduwww.psc.eduwww.nrbsc.org