Part 3 Block Components Ken Cantrell NetApp Mark Rogov EMC David Fair SNIA ESF Chair Intel March 8 2016 SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA unless otherwise noted ID: 816237
Download The PPT/PDF document "Storage Performance Benchmarking:" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Storage Performance Benchmarking:Part 3 – Block Components
Ken Cantrell, NetAppMark Rogov, EMCDavid Fair, SNIA ESF Chair, Intel
March 8, 2016
Slide2SNIA Legal NoticeThe material contained in this tutorial is copyrighted by the SNIA unless otherwise noted.
Member companies and individual members may use this material in presentations and literature under the following conditions:Any slide or slides used must be reproduced in their entirety without modificationThe SNIA must be acknowledged as the source of any material used in the body of any document containing material from these presentations.This presentation is a project of the SNIA Education Committee.Neither the author nor the presenter is an attorney and nothing in this presentation is intended to be, or should be construed as legal advice or an opinion of counsel. If you need legal advice or a legal opinion please contact your attorney.The information presented herein represents the author's personal opinion and current understanding of the relevant issues involved. The author, the presenter, and the SNIA do not assume any responsibility or liability for damages arising out of any reliance on or use of this information.
NO WARRANTIES, EXPRESS OR IMPLIED. USE AT YOUR OWN RISK
.
2
Slide3About The Speakers
3
Mark Rogov
EMC
Advisory Systems
Engineer
@
rogovmark
Dr.
David Fair
SNIA ESF Chair& IntelEthernet Networking Marketing Manager
Ken CantrellNetApp Manager Perf Engineering@kencantrelljr
Slide4Storage Performance Benchmarking
4
Metrics and
terminology
File
Components
Workload
definitions
Today
FUTURE WEBCASTS
July 30, 2015
Solution
Under Test
r/w
intro
tech
raid
fun
end
block
Components
OCT 20, 2015
Slide5Session 1 – Terminology and Context5
GRAPH FUN
CONTEXT
MAKES
METRICS
MATTER
OPS
Count every
protocol
operation
per secondMB/SPAYLOAD sumOf every operationper secondTERMINOLOGY
IOPSCount every IO operationper secondResponsetimeTime targetTakes to replyTo an io
r/w
intro
tech
raid
fun
end
Slide6Session 2 – The Slowest Component Matters Most
6
DISK BOUND
CLIENT
BOUND
SLOW COMPONE
N
T
MATTERS MOST
BOTTLENECKS
ALWAYS EXIST3 PERFORMANCEPRINCIPLESIncrease Parallelism
Do Less
Work
Do Work
Faster
r/w
intro
tech
raid
fun
end
Slide7Split Equally
Enterprise Storage Capacity Shipped In 3Q’15
7
33.1
exabytes
World Population
4.5
Gigabytes
per
individual
7.4
billion
Metrics and
terminology
1541
copies of our first webcast
PowerPoint
OR
r/w
intro
tech
raid
fun
end
Slide8Eventually, All Data Goes To Block Storage
8
Block storage
Solutions UNDER TEST
HYPERVISOR
VM
VM
HYPERVISOR
VM
VM
workloads
r/w
intro
tech
raid
fun
end
Slide9Agenda9
Reading, Writing; What is the Difference?
How does this tech work anyway?
What if you need more than one?
Performance?
Summary
Introduction
END
FUN
RAID
TECH
R/WINTRO
Slide10Let’s Take A Drive… And Test It!
10
r/w
intro
tech
raid
fun
end
IOPS
Slide11Detour! What Does “Random” Mean?11
A QUICK BROWN
FOX JUMPED
OVER A LAZY DOG
KEYS ARE ALL OVER
r/w
intro
tech
raid
fun
end
Imagine that the Keyboardis a disk driveAQ
U
I
C
K
Slide12What Does “Sequential” Mean?12
1 2 3 4 5 6
EVERY KEY IS NEXT TO PREVIOUS
1
2
3
4
5
6
r/w
intro
tech
raid
fun
end
Imagine that the Keyboard
is a disk drive
Slide13“Sequential Read” Example13
“SEQUENTIAL READ”
r/w
intro
tech
raid
fun
end
Slide14Let’s Take A Drive… And Test It!
14
r/w
intro
tech
raid
fun
end
IOPS
Slide15Let’s Take Two Drives… And Test Them!
15
r/w
intro
tech
raid
fun
end
IOPS
HDD
Slide16And Add More SSDs16
r/w
intro
tech
raid
fun
end
SINGLE DRIVE
IOPS
HDD
Slide17Agenda17
Reading, Writing; What is the Difference?
How does this tech work anyway?
What if you need more than one?
Performance?
Summary
Introduction
END
FUN
RAID
TECH
R/WINTRO
Slide18How Does This Tech Work?18
Flash
HDD or Disk Drive
r/w
intro
tech
raid
fun
end
Slide19Spinning Drives And Sectors19
SPIN
SEEK
READ
WRITE
Seek the Track
Spin to the Sector
r/w
intro
tech
raid
fun
end
track
sector
Slide20Flash And NAND Gates20
1
1
0
1
WRITE 0
Write
input
Every NAND can be set to 0 individually
1
1
1
1
WRITE 1
Write
input
To set back to 1, an entire group
needs to be reset
Erase
input
Erase
input
Common
inputCommoninputr/wintrotechraid
funend
Slide21Page
Flash Construction
21
2KiB
4KiB
8KiB
…
Flash BLOCK
Write—
1 page
at a time
Pages per Block (
dep
on model)
128, 256,
…, Etc.
Block
Block
Block
Block
Block
Block
Block
Block
Flash Device
number of blocks defines
Capacity
clean
data
dirty
data
data
dirty
dirty
dirty
Redirect On Over-Write
Most
Flash
r/w
intro
tech
raid
fun
end
An IO is redirected to a
clean
block/page
Leaving old block/page
dirty
0x00
0x01
0x02
0xAA
Logical To Physical Redirection Map
Slide22Garbage Collection22
Block
Block
Block
Block
Block
Block
Block
Block
Flash Device
number of blocks defines
Capacity
clean
data
dirty
data
data
dirtydirtydirtyGarbage Collectionr/wintrotechraid
funendErase—1 dirty block at a time(when number of Clean Blocks is low)
0x00
0x010x020xAALogical To Physical Redirection MapBlockBlockBlock
Blockdirtydirtydirty
dirty
BlockBlockBlockBlockdirtydirtydirtycleanErase
Slide23Sequential Vs. Random23
SPIN
SEEK
SSD or Flash
HDD or Disk Drive
Write
Read
everything
is RANDOM IO
for Flash
Erase + Write
Read
Write
Read
Sequential
Write
Read
Random
Seek/Spin
+ Write
Seek/Spin+ Read
SLOWER PERFORMANCE
r/w
introtechraidfunend
Slide24Agenda24
Reading, Writing; What is the Difference?
How does this tech work anyway?
What if you need more than one?
Performance?
Summary
Introduction
END
FUN
RAID
TECH
R/WINTRO
Slide25Just One?25
protection
capacity
performance
r/w
intro
tech
raid
fun
end
Slide26RAID—
R
edundant
A
rray Of
I
nexpensive
D
isks
26
clients /
hosts
s
torage
controller
p
hysical
storagefront-endconnectback-endconnect
RAID
r/w
introtechraidfunend
Slide27RAID-0 (Striping Without Parity)
27
FRONT END
BACK
END
clients /
hosts
p
hysical
storage
11
01
virtual/
logical
01
11
r/w
intro
tech
raid
fun
end
protection
capacity
performance
100%
none
100%
100%
RAID-0 (Striping Without Parity)
Slide28RAID-1 (Mirroring)28
FRONT END
BACK
END
clients /
hosts
p
hysical
storage
virtual/
logical
11
01
11
01
11
01
r/w
intro
tech
raid
fun
end
protection
capacity
50
%
1 Drive
performance
100%50%
Slide29RAID-3, -4, -5 [-6, -DP]*Striping
With Parity*29
FRONT END
BACK
END
p
hysical
storage
virtual/
logical
01
11
P*
* RAID-6
/-DP
requires
more than one parity
N
is number of data drives
P
is number of parity drives
11
01
r/w
intro
tech
raid
funend
protection
capacity
N
P
performance
N
“It Depends”
Slide30RAID Partial WritesAll Single Parity RAID: RAID-3, -4, -5, and etc.
30
r/w
intro
tech
raid
fun
end
clients /
hosts
virtual/
logical
100
Random
70R/30W
IOs
FRONT END
BACK
END
100 IOs 70
R
/30
W
= 70
Read
+ 30
Write IOs Backend = (70
R + 30 * (2W + 2R)) = 190 IOs190 DiskIOsSingle Partial Write:Read Old DataRead Old ParityCalculate New ParityWrite New DataWrite New Parityclients /hosts
virtual/
logical
FRONT END
BACK
END
write
n
ew
data
old
data
old
parity
new
parity
n
ew
data
2 Reads
2 Writes
performance
2 reads
2 writes
1 read
RAID Penalty
Slide31RAID Implementation
31
clients /
hosts
s
torage
controller
p
hysical
storage
front-end
connect
back-end
connect
RAID
r/w
intro
tech
raid
fun
end
Slide32Erasure Coding Implementation
32
clients /
hosts
s
torage
controllers
p
hysical
storage
front-end
connect
back-end
connect
SCALE OUT
ERASURE CODING
r/w
intro
tech
raid
funend
Slide33Erasure CodingN + M = 2 + 1
33
FRONT END
BACK
END
p
hysical
storage
11
01
01
11
22
P2
22
02
02
P1
r/w
intro
tech
raid
fun
end
protection
capacity
performance
N
M
“different”
“different”
N
is number of data blocks
M
is number of protection blocks
Slide34Agenda34
Reading, Writing; What is the Difference?
How does this tech work anyway?
What if you need more than one?
Performance?
Summary
Introduction
END
FUN
RAID
TECH
R/WINTRO
Slide35What “Really” Happens With RAID-5?
35
r/w
intro
tech
raid
fun
end
IOPS
Slide36What “Really” Happens With RAID-5?
36
r/w
intro
tech
raid
fun
end
IOPS
Slide37What “Really” Happens With RAID-5?
37
r/w
intro
tech
raid
fun
end
IOPS
Slide38What “Really” Happens With RAID-5?
38
r/w
intro
tech
raid
fun
end
IOPS
Slide39What “Really” Happens With RAID-5?
39
r/w
intro
tech
raid
fun
end
IOPS
318
Slide40What “Really” Happens With RAID-5?
40
r/w
intro
tech
raid
fun
end
IOPS
5399
97369736318
Slide41What “Really” Happens With RAID-5?
41
r/w
intro
tech
raid
fun
end
MiB
/s
385938.0338.0321.09
179
Slide42What “Really” Happens With RAID-5?
42
r/w
intro
tech
raid
fun
end
MB/s
Slide43Flash In The Real World
43
clients /
hosts
s
torage
controller
p
hysical
storage
front-end
connect
back-end
connect
MB/s
Requires
10x
6Gb/s SAS5685 MB/s
Requires 6x PCIe 3.0 LanesRequires 4x 16Gb FCHow
many
hosts?1064r/wintrotechraidfunend??
Slide44Flash In The Real World
44
clients /
hosts
s
torage
controller
p
hysical
storage
front-end
connect
back-end
connect
MB/s
Requires
10x
6Gb/s SAS5685 MB/sRequires
6x PCIe 3.0 LanesRequires 4x 16Gb FCHowmany
hosts?
1064r/wintrotechraidfunendNew Tech
Slide45Flash
I
n The Real World
45
clients /
hosts
s
torage
controller
p
hysical
storage
front-end
connect
back-end
connect
MB/s
Requires 10x 6Gb/s SAS5685 MB/s
Requires 6x PCIe 3.0 LanesRequires 4x 16Gb FCHowmany
hosts?
1064r/wintrotechraidfunendBLOCK is the foundationof Storage Performance
Slide46Agenda46
Reading, Writing; What is the Difference?
How does this tech work anyway?
What if you need more than one?
Performance?
Summary
Introduction
END
FUN
RAID
TECH
R/WINTRO
Slide47Storage Performance Benchmarking
47
Metrics and
terminology
File
Components
Workload
definitions
Today
FUTURE WEBCASTS
July 30, 2015
Solution
Under Test
block
Components
OCT 20, 2015
r/w
intro
tech
raid
fun
end
Slide48After This Webcast
48
A
PDF
and a PPT of
the
slides for this and all previous parts of this Webcast series
will be posted to the SNIA Ethernet Storage Forum (ESF) website and available on-demand
PPT and PDF:
http://www.snia.org/forums/esf/knowledge/webcasts Presentation Recording: https://www.brighttalk.com/webcast/663/189797 A full Q&A from this webcast, including answers to questions we couldn't
get to today, will be posted to the SNIA-ESF bloghttp://sniaesfblog.org/ Follow us on Twitter @SNIAESF, @RogovMark, @KenCantrellJr, @DrJMetzNext Webcast – Second Half of 2016“Storage Performance Benchmarking: Part 4”
Slide49QUESTIONS?
Slide50THANK YOU!
Slide51Appendix – Additional Reading
51
Slide52Appendix – More ReadingSNIA S3
TWG Guide to SSD Performance: http://www.snia.org/sites/default/files/UnderstandingSSDPerformance.Jan12.web_.pdfSNIA S3 TWG SSD Performance Primer, 2013: http://www.snia.org/sites/default/files/SNIASSSI.SSDPerformance-APrimer2013.pdf
Benchmarking methods for randomly sampling from a file, and why random seeks can (usually)
hurt performance: http://simpsonlab.github.io/2015/05/19/io-performance
/
Excellent hard drive overview:
https://
www.backblaze.com/hard-drive.html
SSD Performance results: http://www.tomshardware.com/charts/ssd-charts-2014/benchmarks,129.htmlSSD Performance results: http://www.anandtech.com/show/6433/intel-ssd-dc-s3700-200gb-review/3Intel Performance Benchmarking for PCIe* and NVMe* Enterprise Solid-State Drives: http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/performance-pcie-nvme-enterprise-ssds-white-paper.pdfSSD M.2 Interface: http://arstechnica.com/gadgets/2015/02/understanding-m-2-the-interface-that-will-speed-up-your-next-ssd/More complete SSD interface article, covering NVMe, U.2 and M.2: http://blog.ocz.com/ssd-interfaces-sata-m2-u2-nvme/SSD vs HDD performance characteristics: http://www.tomshardware.com/reviews/ssd-gaming-performance,2991-3.html52
Slide53RAIDhttp://
www.raid-recovery-guide.com/raid5-parity.aspxhttp://rickardnobel.se/how-raid5-works/http://igoro.com/archive/how-raid-6-dual-parity-calculation-works/RAID Perf Calculator: http://wintelguy.com/raidperf.pl
RAID
Reliability Calculator:
http://wintelguy.com/raidmttdl.plRAID
Failure Calculator:
http://
raid-failure.com/raid10-50-60-failure.aspx
RAID Survival
Rate Simulation: https://linustechtips.com/main/topic/103179-lets-talk-about-raid-survival-rates/53