Samba: lessons learned Online access to EOS for Windows-based users EOS Workshop 2021 1 Aritz Brosa Iartza, Giuseppe Lo Presti CERN IT-ST Status EOS Workshop 2020 Windows DFS at CERN phase-out started, users being migrated to CERNBox
"Samba: lessons learned Online access to EOS for" is the property of its rightful owner. Permission is granted to
download and print the materials on this website for personal, non-commercial use only, and to display it
on your personal computer provided you do not modify the materials and that you retain all copyright
notices contained in the materials. By downloading content from our website, you accept the terms of this
agreement.
Presentation Transcript
01
Samba: lessons learned Online access to EOS for Windows-based users EOS Workshop 2021 1 Aritz Brosa Iartza, Giuseppe Lo Presti
CERN IT-ST<br>
02
Status @ EOS Workshop 2020 Windows DFS at CERN phase-out started, users being migrated to CERNBox
Samba gateways from prototype to production
High Available cluster in place since Sep 2019
Starting to gain some operational experience
Growing user base, production service at its inception
Still dealing with functionality issues, e.g. concurrent Microsoft Office editing was broken
See [EOS Workshop 2020] EOS Workshop 2021 2<br>
03
Today: significant usage growth Close to peaks of 3K connections, average has doubled compared to one year ago.
Usage is significant also in terms of I/O ops sustained by FUSEX
~10 kHz of IO ops on a regular basis EOS Workshop 2021 3 2020 <|> 2021 EOS Workshop 2020<br>
04
Monitoring and Alerting EOS Workshop 2021 4 Developed from scratch in Grafana
Aim to address issues before users are impacted This plot shows long run trend in terms of FUSEX latency. Faulty behavior in FUSEX logs can be alerted to administrators.<br>
05
Experience and issues EOS Workshop 2021 5 Samba over FUSEX stack is extremely latency-sensitive
Many issues fixed in EOS to address latency
Latency to stat the Samba mount from a Windows client (note the log scale) Windows SMB mounts latency<br>
06
Experience and issues: a few examples Windows client's inefficiencies => fetch of all xattrs of all target files
In POSIX jargon, fstatat, listxattr, and getxattr called for each file and each extended attribute of the folder
Aggressive caching + disabling of eos-specific system-level xattrs
"Surprisingly", smbclient on Linux does NOT query all xattrs EOS Workshop 2021 6<br>
07
Experience and issues: a few examples Windows client's inefficiencies => fetch of all xattrs of all target files
In POSIX jargon, fstatat, listxattr, and getxattr called for each file and each extended attribute of the folder
Aggressive caching + disabling of eos-specific system-level xattrs
"Surprisingly", smbclient on Linux does NOT query all xattrs
Most recurrent filenames: Thumbs.db, desktop.ini
Representing ~40-50% of the Samba traffic in terms of requests count!
This is because of our strategy to provide home dirs through Samba
Considering caching them in a separate SSD-based pool EOS Workshop 2021 7<br>
08
Coming next Usage growth expected ahead: about to migrate more Windows-based use cases
Possibly need to isolate the most demanding use cases in a separate EOS backend, optimized for low latency operations
Looking forward to the EOS Windows native client from Comtrade! EOS Workshop 2021 8<br>
09
Conclusions: the bigger picture Multiple access paths . . .
Windows Desktop/Documents/… system folders
Either synchronized, or mapped to Samba
Samba became a first-class citizen among the available access methods to CERNBox
Significant usage, critical service for many workflows in the user community
. . . aiming at a coherent cross-platform UX EOS Workshop 2021 9<br>
10
Thank you for your attention!
Time for questions<br>
