Oliver Keeble DOMA The WLCG DOMA project consists of several activities in the area of Data Organization Management and Access with a focus on the mediumlong term evolution https twikicernchtwikibinviewLCGDomaActivities ID: 790754
Download The PPT/PDF document "Archival Storage WG DOMA Update" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Slide2Archival Storage WG DOMA Update
Oliver Keeble
Slide3DOMA
“The WLCG DOMA project consists of several activities in the area of Data Organization, Management and Access, with a focus on the medium/long term
evolution”
https://twiki.cern.ch/twiki/bin/view/LCG/DomaActivitiesA coordinating layer which tracks targeted activities, currently:DOMA WGsThird party copyData AccessQuality of Service (QoS)Other activitiesArchival storage (WLCG WG)Data and networks Auth* (WLCG TF)…
18/01/2019
WLCG Archival Storage Group
3
Slide4DOMA Nov 2018
November general meeting concentrated on Archival storage
https
://indico.cern.ch/event/767209/CarouselsArchival Storage WG reportExperiment input18/01/2019WLCG Archival Storage Group4
Slide5Today’s meeting
Focuses on archive storage:
Processing not-so-frequently-accessed-data (a.k.a. “cold data”) from high latency and low cost media (today a.k.a. TAPE) => “Tape Carousels”
Archive storage Working Group and the ongoing activities. Technology evolution. Synergy with the work ongoing in QoS DOMA activity. Inputs from experiments: plans, prospects, strategies, concerns, ongoing efforts Name one.We should then try to understand how those initiatives will proceed coherentlyFinally, a quick roundtable of other DOMA activities in the AOB DOMA Meeting Intro Slide
18/01/2019
WLCG Archival Storage Group
5
Slide6Future WG activities
Long term evolution of experiment use of archival storage
Vlado’s
talk todaySee experiment contributions at DOMA meetingCost modellingRenaud’s talk todayShould we pursue the following?QoS, non-tape archive, pledgesPost-SRM interfaces18/01/2019WLCG Archival Storage Group6
Slide7post-SRM interfaces
18/01/2019
WLCG Archival Storage Group
7
Slide8Why are we considering this?
Two providers (CERN & RAL) are developing new systems
CERN has no
plans to provide an SRM interfaceCERN’s EOS/CTA is using an xrootd interfaceTweaked to handle tape operationsFTS can already use thisIs there an argument for a new standard/convention?What should the WG’s role be in answering this question?If we do nothing?CERN and RAL will have their own interfaces18/01/2019
WLCG Archival Storage Group
8
Slide9Is an agreement required?
We use only a small part of SRM
A new interface could allow simpler, cheaper systems
We could add new functions to a new interfaceTagging datasetsExpressing localityDataset operationsA more generic QoS interface could be usedCandidates existMore systems will add new interfaces and there will be proliferationSRM is not being obsoletedThe number of systems needing a new interface may be very small (Just CERN & RAL in the end)FTS can be used and its interface will not
changeIt can absorb differences between systems
Many smaller communities already use storage-specific commandsCastor “stager_get
”
This works OK
SRM can be extended if new operations are required
18/01/2019
WLCG Archival Storage Group
9
We need a new common interface definition because…
We
don’t
need a new common interface definition because…