BeStMan Berkeley Storage Manager BeStMan Current version 2211r6 Oct 9 2008 Full implementation of SRM v22 Support for WLCG SRM MoU For diskbased storage and file systems ID: 459788
Download Presentation The PPT/PDF document "Berkeley Storage Manager" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Berkeley Storage Manager
(BeStMan)Slide2
Berkeley Storage Manager (BeStMan
)Current version 2.2.1.1.r6 (Oct. 9, 2008)Full implementation of SRM v2.2Support for WLCG SRM MoUFor disk-based storage and file systemsNFS, GPFS, GFS, NGFS, PNFS, HFS+, PVFS, Lustre, XrootdFSSupport for multiple disk partitionsDynamic space reservation and support for space token based operationsSupport for accessibility to user managed disk space through SRM interfaceMass Storage supportCurrently, for HPSS and NCAR-MSS
Plug-in capability for easy customization for special underlying storageE.g. SRM/L-Store, SRM/XrootDUse of existing file transfer services
GSIFTP, FTP, BBFTP, HTTP, HTTPS
Support for multiple transfer servers
Easy adaptability and integration to special project environments
Simple installation and easy maintenanceSlide3
Who is BeStMan
for?Smaller sites with shared and limited resourcesSites where minimal administrative effort is neededMain use is for disks with POSIX file systemsSites/users that need dynamic invocation of an SRMCan be customized for different file systems and MSSSites that need great supportSlide4
BeStMan Gateway
Light-weight implementation of SRM v2.2 for POSIX file systemsDesigned to work with any posix file systemsNFS, GPFS, GFS, NGFS, PNFS, HFS+, PVFS, AFS, Lustre, XrootdFS, HadoopGeneric SRM v2.2 load balancing front-end for GridFTP serversWorks with Grid-mapfile or GUMS serverSupport for subset of SRM v2.2 interfaces
srmPing, srmLs, srmRm,
srmMkdir
,
srmRmdir
,
srmPrepareToPut
(Status,
PutDone
),
srmPrepareToGet
(Status,
ReleaseFiles
)
Use of existing
gridftp
servers
Support for multiple file transfer servers
Can be used with any storage systems
that
have
one or more
GridFTP
servers
and a POSIX file system interface
Same packaging with
BeStMan
full management mode
Simple installation and maintenanceSlide5
Who is BeStMan Gateway mode
for?Sites with POSIX file systemsSites without needs for request queuing or disk space managementSites that need great performanceSites that need SRM interface to their local resources Sites where minimal administrative effort is neededSites that need great supportSlide6
A few hints on what BeStMan can do
If the target host does not have an SRM, BeStMan can be installed, configured, and started with a few commands.Configurable support for accessibility to user’s own disk path as source url or target url in the same SURL format.User’s own disk file pathsrm://host:port/srm/v2/server?SFN=/home/john/myfileSRM cache pathsrm://host:port/srm/v2/server?SFN=/srmcache/john/myfile2Works with PhEDEx and FTS file transfersBetween BeStMan and other SRM v2.2 servers
Between BeStMan and SRM v1.1 serversSlide7
SRB
(iRODS)SDSCSINICALBNLEGEE
Interoperability with other SRM v2.2
Client
User/application
CASTOR
Disk
BeStMan
xrootd
BNL
SLAC
LBNL
dCache
DPM
mySQL DBSlide8
Some Use Cases
US ATLASBeStMan Gateway on XrootdFS/XrootdBeStMan Gateway on GPFSSTARData replication between BNL and LBNLHPSS access at BNL and NERSCSRMs in production for over 4 yearsPart of analysis scenario to move job-generated data files from PDSF/NERSC to remote BNL storageEarth System GridServing about 6000 users Over a million files and 170TB of climate data from 5 storage sites with SRMs (LANL, LLNL, NCAR, NERSC, ORNL)Uses an adapted BeStMan
for NCAR’s own MSSSlide9
Testing and Monitoring
OSG provides daily SRM v2.2 testing and monitoring for registered SRM endpointsResults on http://datagrid.lbl.gov/osgSite registration on http://datagrid.lbl.gov/siteregSite testingAvailable through srm-tester-2Instructions can be found onhttps://twiki.grid.iu.edu/twiki/bin/view/Storage/BeStManSlide10
Summary
BeStMan is an implementation of SRM v2.2.Great for small disk-based storage and file systemsBeStMan Gateway mode gives an SRM v2.2 access to POSIX compliant file systemsInstall/maintain through VDTWorks with other SRM v2.2 implementationsServers: CASTOR, dCache, DPM, StoRM, SRM/SRB, …Clients: PhEDEx, FTS, glite-url-copy, srm-copy, srmcp, …In OSG, WLCG/EGEE, ESGSlide11
Documents and Support
BeStManhttp://datagrid.lbl.gov/bestmanhttps://twiki.grid.iu.edu/twiki/bin/view/Storage/BeStManhttp://hep-t3.physics.umd.edu/HowToForAdmins.html#osgBestmanBeStMan Gatewayhttp://datagrid.lbl.gov/bestmanhttp://wt2.slac.stanford.edu/xrootdfs/bestman-gateway.htmlSRM testing and monitoringhttp://datagrid.lbl.gov/SRM Collaboration and SRM Specifications
http://sdm.lbl.gov/srm-wgContact and support : srm@lbl.gov