Keep appetite for the stats, it costs nothing
Description: Keep appetite for the stats, it costs nothing Presentation of the statistics consumption model in VPP from the costless low-level design to their exploitation in user space 6 February 2022 FOSDEM 2022 1 Arthur de Kerhor
Related Topics
Download Presentation
"Keep appetite for the stats, it costs nothing" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
slide1. Keep appetite for the stats, it costs nothing Presentation of the statistics consumption model in VPP from the costless low-level design to their exploitation in user space 6 February 2022 FOSDEM 2022 1 Arthur de Kerhor
arthurdekerhor@gmail.com Hedi Bouattour<br>
slide2. VPP intro af-packet-input ethernet-input ip4-input ip4-lookup ip4-receive ip4-drop ip4-midchain host-ens5-output Open-source networking dataplane in user space
Multi-threaded
Highly optimized for performance, speed and scale
Designed as a graph of nodes
Supports tons of features in L2-4 such as crypto, NAT, LB, ACL, tunnels, … 6 February 2022 FOSDEM 2022 2<br>
slide3. Stats segment in VPP Shared memory
Vector based
Single writer, multiple readers
Optimistic locking
Hypothesis: number of writes << number of reads
Reading the stats in shared memory costs nothing for the data plane
What is costly however is to update the stats counters in the data plane: it sometimes induces some additional cacheline misses. 6 February 2022 FOSDEM 2022 3<br>
slide4. Shared memory layout 6 February 2022 FOSDEM 2022 4<br>
slide5. Optimistic locking Take the lock = set in_progress boolean
Release the lock = unset in_progress + increment epoch counter
WRITE: Counter added/extended by the main thread (e.g. new interface):
lock taken directory vector update (pointers may change) counters vectors updates (e.g. new row in /if/tx) lock released
READ: A stats client wants to access shared memory:
wait until in_progress == 0 store epoch value read the stats of interests check that (epoch == stored epoch and not in_progress) If false, retry the read
No concurrency issues when adding/extending counters because it is always done under a barrier no counters updates during writes. 6 February 2022 FOSDEM 2022 5<br>
slide6. Various clients for different usages A simple client to access raw statistics
A filesystem in user space leveraging go-fuse
A Prometheus exporter to gather statistics for applications 6 February 2022 FOSDEM 2022 6<br>
slide7. Simple client to access raw statistics Small executable
Connects to the stats socket of VPP to learn about the shared memory location
Allows to list and/or dump the statistics available
Takes a list of regexp as input and returns the matching counters
Usage: quick access to the statistics, easy to launch 6 February 2022 FOSDEM 2022 7<br>
slide8. Fuse filesystem in Golang Directory Node
Path
Epoch File Node (leaf)
Name
Counter index Root node
Path = “/”
Epoch = 1 Path = “interfaces/”
Epoch = 2 Path = “sys/”
Epoch = 3 Path = “net/”
Epoch = 3 Path = “eth0/”
Epoch = 3 Path = “eth1/”
Epoch = 2 Name = “last_update”
Counter index = 134 Name = “rx”
Counter index = 46 Name = “tx”
Counter index = 76 Name = “num_threads”
Counter index = 21 … OpenDir Open 5 Name = “punt”
Counter index = 163 Epoch = 4 Epoch = 4 Epoch = 4 6 February 2022 FOSDEM 2022 8<br>
slide9. Demo! 6 February 2022 FOSDEM 2022 9<br>
slide10. Calico/VPP context 6 February 2022 FOSDEM 2022 10 Calico: open-source Kubernetes networking solution and network policy
Standard Linux networking pipeline mode dataplane
Integrating VPP dataplane: faster networking<br>
slide11. 6 February 2022 FOSDEM 2022 11 Calico/VPP agent Architecture<br>
slide12. 6 February 2022 FOSDEM 2022 12 Calico/VPP agent Architecture Stat segment shared memory Prometheus Server<br>
slide13. VPP stats in Prometheus 6 February 2022 FOSDEM 2022 13 Prometheus Server PULL Stat segment shared memory Kubernetes cluster Node exporter VPP<br>
slide14. Demo! 6 February 2022 FOSDEM 2022 14<br>
slide15. Q&A session 6 February 2022 FOSDEM 2022 15<br>
arthurdekerhor@gmail.com Hedi Bouattour<br>
slide2. VPP intro af-packet-input ethernet-input ip4-input ip4-lookup ip4-receive ip4-drop ip4-midchain host-ens5-output Open-source networking dataplane in user space
Multi-threaded
Highly optimized for performance, speed and scale
Designed as a graph of nodes
Supports tons of features in L2-4 such as crypto, NAT, LB, ACL, tunnels, … 6 February 2022 FOSDEM 2022 2<br>
slide3. Stats segment in VPP Shared memory
Vector based
Single writer, multiple readers
Optimistic locking
Hypothesis: number of writes << number of reads
Reading the stats in shared memory costs nothing for the data plane
What is costly however is to update the stats counters in the data plane: it sometimes induces some additional cacheline misses. 6 February 2022 FOSDEM 2022 3<br>
slide4. Shared memory layout 6 February 2022 FOSDEM 2022 4<br>
slide5. Optimistic locking Take the lock = set in_progress boolean
Release the lock = unset in_progress + increment epoch counter
WRITE: Counter added/extended by the main thread (e.g. new interface):
lock taken directory vector update (pointers may change) counters vectors updates (e.g. new row in /if/tx) lock released
READ: A stats client wants to access shared memory:
wait until in_progress == 0 store epoch value read the stats of interests check that (epoch == stored epoch and not in_progress) If false, retry the read
No concurrency issues when adding/extending counters because it is always done under a barrier no counters updates during writes. 6 February 2022 FOSDEM 2022 5<br>
slide6. Various clients for different usages A simple client to access raw statistics
A filesystem in user space leveraging go-fuse
A Prometheus exporter to gather statistics for applications 6 February 2022 FOSDEM 2022 6<br>
slide7. Simple client to access raw statistics Small executable
Connects to the stats socket of VPP to learn about the shared memory location
Allows to list and/or dump the statistics available
Takes a list of regexp as input and returns the matching counters
Usage: quick access to the statistics, easy to launch 6 February 2022 FOSDEM 2022 7<br>
slide8. Fuse filesystem in Golang Directory Node
Path
Epoch File Node (leaf)
Name
Counter index Root node
Path = “/”
Epoch = 1 Path = “interfaces/”
Epoch = 2 Path = “sys/”
Epoch = 3 Path = “net/”
Epoch = 3 Path = “eth0/”
Epoch = 3 Path = “eth1/”
Epoch = 2 Name = “last_update”
Counter index = 134 Name = “rx”
Counter index = 46 Name = “tx”
Counter index = 76 Name = “num_threads”
Counter index = 21 … OpenDir Open 5 Name = “punt”
Counter index = 163 Epoch = 4 Epoch = 4 Epoch = 4 6 February 2022 FOSDEM 2022 8<br>
slide9. Demo! 6 February 2022 FOSDEM 2022 9<br>
slide10. Calico/VPP context 6 February 2022 FOSDEM 2022 10 Calico: open-source Kubernetes networking solution and network policy
Standard Linux networking pipeline mode dataplane
Integrating VPP dataplane: faster networking<br>
slide11. 6 February 2022 FOSDEM 2022 11 Calico/VPP agent Architecture<br>
slide12. 6 February 2022 FOSDEM 2022 12 Calico/VPP agent Architecture Stat segment shared memory Prometheus Server<br>
slide13. VPP stats in Prometheus 6 February 2022 FOSDEM 2022 13 Prometheus Server PULL Stat segment shared memory Kubernetes cluster Node exporter VPP<br>
slide14. Demo! 6 February 2022 FOSDEM 2022 14<br>
slide15. Q&A session 6 February 2022 FOSDEM 2022 15<br>