Vasily Tarasov SBU Dean Hildebrand IBM Almaden Renu Tewari IBM Almaden Erez Zadok SBU File system and Storage Lab F S L and a bit more Outline How all that started ID: 649230
Download Presentation The PPT/PDF document "Virtual Machine Disk Images Introspectio..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Virtual Machine Disk Images Introspection
Vasily Tarasov
(SBU)
Dean Hildebrand (IBM Almaden)Renu Tewari (IBM Almaden)Erez Zadok (SBU)
File system and Storage Lab (FSL)
and a bit more...Slide2
Outline
How all that started The idea of introspection
A couple of results from a 1st prototype
Future work
Benchmarking, FilebenchSlide3
Virtual Machines
(VMs)
- Computational resources consolidation
- Flexible, efficient and scalable
- Hardware support- Multiple solutions: VMWare, KVM, Xen, ...- Cloud-way of delivering services
Network Attached Storage (NAS)
- Storage consolidation
- Scalable, manageable and efficient
- NFS/CIFS available on majority of Operating Systems
- NAS sales jumped from $540M in 1998 to $5.1B in 2003
- IBM SONAS
Two
important technologiesSlide4
VM
NAS
Two technologies…
DeanSlide5
VM
NAS
…and they grow
DeanSlide6
VM
NAS
IBM
SO
How do VM & NAS work together?
Can we make them work better?Slide7
Typical Setup
VM 2-1
VM 2-2
Virtual
Machines
Host2
NFS CLIENT
Storage 2-1
Storage 2-2
GPFS
Node 1
Storage 3-1
Storage 3-2
GPFS
Node 3
GPFS
Node 1
Storage 4-1
Storage 4-2
GPFS
Node 4
VM 3-2
Virtual
Machines
Host
3
NFS CLIENT
VM 3-1
NFS
SERVER
2
VM 1-1
VM 1-2
Virtual
Machines
Host
1
NFS CLIENT
NFS
SERVER
1
Storage 1-1
Storage 1-2
GPFS
Node 1
GPFS
Node 2
VMWare
, KVM, XEN, ...Slide8
Datapath Decomposed
Applications
Virtual File System
On-Disk File System
Block Layer
Controller Driver
Controller Emulator
NFS Client
VM Guest
NETWORK
Host
NFS Server
Virtual File System
On-Disk File System
Block Layer
Controller Driver
NAS
CA
RA
RA
RM
CA
RA
RM
RM
CA
RA
RM
RM
CA
RA
RA
RM
CA
RA
RM
RM
–
CA
ching
CA
–
R
ead-
A
head
RA
–
R
equest
Mangling and
Scheduling
RMSlide9
Collecting traces: setup
1Gbps
VMWare ESX4
NFS Server
VSCSI
Layer
Trace
Network
Trace
Block
Layer
Trace
Within
VM
trace
Rand/Seq Read
Rand/Seq Write
Various I/O sizes
Multi-file workloads
Multi-process workloads
Meta-data intensiveSlide10
Collecting traces: setup
VSCSI Layer
Trace
Network
TraceApplications
Virtual File System
On-Disk File System
Block Layer
Controller Driver
Controller Emulator
NFS Client
VM Guest
NETWORK
Host
NFS Server
Virtual File System
On-Disk File System
Block Layer
Controller Driver
NAS
Block
Layer
Trace
User-Space
Workload
Rand/
Seq
Read
Rand/
Seq
Write
Various I/O sizes
Multi-file workloads
Multi-process workloads
Meta-data intensiveSlide11
Some interesting results
I/O sizes change
4MB
4KB
128KB256KB
Applications
Virtual File System
On-Disk File System
Block Layer
Controller Driver
Controller Emulator
NFS Client
VM Guest
NETWORK
Host
NFS Server
Virtual File System
On-Disk File System
Block Layer
Controller Driver
NAS
1MB
32KB
WIOV’11 -
Revisiting the Storage Stack in Virtualized NAS EnvironmentsSlide12
Meta-data Ops Data Ops
# stat /foo/bar
sys_stat(/foo/bar)
NFS_
GETATTR
(foobar
_fh)
# stat /foo/bar
sys_stat(/foo/bar)
NFS_
READ
(
dskimg
_fh)
Non-VM case
VM case
Update attributes
List directories
Creation/deletion
Lookup
Access permissions
Link/Symlink operations
NFS_
WRITE
(
dskimg
_fh)Slide13
Come up with an idea
READ
(dskfh, offset, len)
Disk Image File
NFS
Server
Offset
Size
What is located in
this region?
Ext, NTFS,
UFS, ...
READ from:
Inode
Directory entry
Data of specific file
...
Do intelligent things!Slide14
Prototype Results: Find
find
80% improvementSlide15
Prototype Results
: Startup
130 sec
50 sec
2.6x times fasterSlide16
Future work
Solid implementation
More efficient cache policies
Optimizations on the write path
Analysis of more complex workloadsSlide17
Virtual
Machine
Disk Images Introspection
a
bit more...Slide18
A Recent Study Concluded that…
Much of what researchers conclude in their studies is misleading, exaggerated,or flat-out
wrongA
new claim about a research findings is more likely to be
false than trueResearchers tend to publish positive results more often than negative findingsChances to be accepted to a conference are higher if the results are “more exciting”5/4/201118
Medicine
A
Sociology
B
Computer Science
C
Biology
D
Physics
E
2005-2008 study
by J. Ioannidis
HotOS’11: Benchmarking FS Benchmarking: It is Rocket ScienceSlide19
FilebenchOriginally created by
SUN Microsystem (RIP )Maintained by FS
LUsed in many papersFlexible: Workload Model Language –
WMLPortable: Linux, FreeBSD, Solaris,
MacOS, Windows *Slide20
Filebench WML
define fileset name=myfileset,size=16kb,entries=1000define process
name=reader,instances=1
{ thread name=
readerthread,memsize=10m,instances=10 { flowop read name=myread,filesetname=myfileset,iosize=2kb } }Slide21
Filebench for Cloud Services
ReadsWritesCreatesDeletes+20 moresophisticated
POSIX
NFS RPC
AFS RPC
Cloud
flowops:Slide22
Filebench for Virtualized Environments
define process name=reader,instances=1 {
thread name=readerthread,memsize=10m
,instances=10 {
flowop read name=myread1,filesetname=myfileset,… } }define hypervisor name=hpv,type=esx3.1,instances=1{define vm
name=hpv,type=windows,instances=5{
}
}Slide23
Virtual Machine Disk Images Introspection
Vasily
Tarasov
(SBU)Dean Hildebrand (IBM Almaden)Renu
Tewari (IBM Almaden)
Erez Zadok (SBU)
File system and Storage Lab (
F
S
L
)
and a bit more...Thank you!