Virtualized Cloud Infrastructure without the Virtualization Eric Keller Jakub Szefer Jennifer Rexford Ruby Lee ISCA 2010 Princeton University Virtualized Cloud Infrastructure Run virtual machines on a hosted infrastructure ID: 476177
Download Presentation The PPT/PDF document "NoHype" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
NoHype: Virtualized Cloud Infrastructure without the Virtualization
Eric Keller, Jakub Szefer, Jennifer Rexford, Ruby Lee
ISCA 2010
Princeton UniversitySlide2
Virtualized Cloud InfrastructureRun virtual machines on a hosted infrastructure
Benefits…Economies of scaleDynamically scale (pay for what you use)Slide3
Without the VirtualizationVirtualization used to share servers
Software layer running under each virtual machine3
Physical Hardware
Hypervisor
OS
OS
Apps
Apps
Guest VM1
Guest VM2
serversSlide4
Without the VirtualizationVirtualization used to share servers
Software layer running under each virtual machineMalicious software can run on the same serverAttack hypervisorAccess/Obstruct other VMs4
Physical Hardware
Hypervisor
OS
OS
Apps
Apps
Guest VM1
Guest VM2
serversSlide5
Are these vulnerabilities imagined?No headlines… doesn’t mean it’s not real
Not enticing enough to hackers yet?(small market size, lack of confidential data)Virtualization layer huge and growing100 Thousand lines of code in hypervisor1 Million lines in privileged virtual machineDerived from existing operating systems Which have security holes
5Slide6
NoHypeNoHype removes the hypervisor
There’s nothing to attackComplete systems solutionStill retains the needs of a virtualized cloud infrastructure6
Physical Hardware
OS
OS
Apps
Apps
Guest VM1
Guest VM2
No hypervisorSlide7
Virtualization in the CloudWhy does a cloud infrastructure use virtualization?
To support dynamically starting/stopping VMsTo allow servers to be shared (multi-tenancy)Do not need full power of modern hypervisorsEmulating diverse (potentially older) hardwareMaximizing server consolidation
7Slide8
Roles of the HypervisorIsolating/Emulating resources
CPU: Scheduling virtual machinesMemory: Managing memoryI/O: Emulating I/O devicesNetworkingManaging virtual machines
8Slide9
Roles of the HypervisorIsolating/Emulating resources
CPU: Scheduling virtual machinesMemory: Managing memoryI/O: Emulating I/O devicesNetworkingManaging virtual machines
9
Push to HW /
Pre-allocationSlide10
Roles of the HypervisorIsolating/Emulating resources
CPU: Scheduling virtual machinesMemory: Managing memoryI/O: Emulating I/O devicesNetworkingManaging virtual machines
10
Push to HW /
Pre-allocation
RemoveSlide11
Roles of the HypervisorIsolating/Emulating resources
CPU: Scheduling virtual machinesMemory: Managing memoryI/O: Emulating I/O devicesNetworkingManaging virtual machines
11
Push to HW /
Pre-allocation
Remove
Push to sideSlide12
Roles of the HypervisorIsolating/Emulating resources
CPU: Scheduling virtual machinesMemory: Managing memoryI/O: Emulating I/O devicesNetworkingManaging virtual machines
12
Push to HW /
Pre-allocation
Remove
Push to side
NoHype
has a double meaning… “no hype”Slide13
Scheduling Virtual MachinesScheduler called each time hypervisor runs
(periodically, I/O events, etc.)Chooses what to run next on given coreBalances load across cores13
hypervisor
timer
switch
I/O
switch
timer
switch
VMs
time
TodaySlide14
Dedicate a core to a single VM
Ride the multi-core trend1 core on 128-core device is ~0.8% of the processorCloud computing is pay-per-useDuring high demand, spawn more VMsDuring low demand, kill some VMsCustomer maximizing each VMs work, which minimizes opportunity for over-subscription
14
NoHypeSlide15
Managing MemoryGoal: system-wide optimal usage
i.e., maximize server consolidationHypervisor controls allocation of physical memory
15
TodaySlide16
Pre-allocate MemoryIn cloud computing: charged per unit
e.g., VM with 2GB memoryPre-allocate a fixed amount of memoryMemory is fixed and guaranteedGuest VM manages its own physical memory(deciding what pages to swap to disk)Processor support for enforcing:allocation and bus utilization
16
NoHypeSlide17
Emulate I/O Devices
Guest sees virtual devices
Access to a device’s memory range traps to hypervisor
Hypervisor handles interrupts
Privileged VM emulates devices and performs I/O
17
Physical Hardware
Hypervisor
OS
OS
Apps
Apps
Guest VM1
Guest VM2
Real
Drivers
Priv. VM
Device
Emulation
trap
trap
hypercall
TodaySlide18
Guest sees virtual devicesAccess to a device’s memory range traps to hypervisorHypervisor handles interruptsPrivileged VM emulates devices and performs I/O
Emulate I/O Devices
18
Physical Hardware
Hypervisor
OS
OS
Apps
Apps
Guest VM1
Guest VM2
Real
Drivers
Priv. VM
Device
Emulation
trap
trap
hypercall
TodaySlide19
Dedicate Devices to a VMIn cloud computing, only networking and storage
Static memory partitioning for enforcing accessProcessor (for to device), IOMMU (for from device)19
Physical Hardware
OS
OS
Apps
Apps
Guest VM1
Guest VM2
NoHypeSlide20
Virtualize the DevicesPer-VM physical device doesn’t scale
Multiple queues on deviceMultiple memory ranges mapping to different queues20
Processor
Chipset
Memory
Classify
MUX
MAC/PHY
Network Card
Peripheral
bus
NoHypeSlide21
Ethernet switches connect serversNetworking
21
server
server
TodaySlide22
Software Ethernet switches connect VMs
Networking (in virtualized server)
22
Virtual server
Virtual server
Software
Virtual switch
TodaySlide23
Software Ethernet switches connect VMs
Networking (in virtualized server)
23
OS
Apps
Guest VM1
Hypervisor
OS
Apps
Guest VM2
hypervisor
TodaySlide24
Software Ethernet switches connect VMs
Networking (in virtualized server)
24
OS
Apps
Guest VM1
Hypervisor
OS
Apps
Guest VM2
Software
Switch
Priv. VM
TodaySlide25
Do Networking in the NetworkCo-located VMs communicate through software
Performance penalty for not co-located VMsSpecial case in cloud computingArtifact of going through hypervisor anywayInstead: utilize hardware switches in the networkModification to support hairpin turnaround
25
NoHypeSlide26
Managing Virtual MachinesAllowing a customer to start and stop VMs
26
Wide Area Network
Request:
Start VM
Cloud
Customer
Cloud
Provider
TodaySlide27
Managing Virtual MachinesAllowing a customer to start and stop VMs
27
Wide Area Network
Servers
Request:
Start VM
Cloud
Customer
Cloud
Provider
.
.
.
VM images
Cloud
Manager
Request:
Start VM
TodaySlide28
Hypervisor’s Role in ManagementRun as application in privileged VM
28
Physical Hardware
Hypervisor
Priv. VM
VM
Mgmt.
TodaySlide29
Hypervisor’s Role in ManagementReceive request from cloud manager
29
Physical Hardware
Hypervisor
Priv. VM
VM
Mgmt.
TodaySlide30
Hypervisor’s Role in ManagementForm request to hypervisor
30
Physical Hardware
Hypervisor
Priv. VM
VM
Mgmt.
TodaySlide31
Hypervisor’s Role in ManagementLaunch VM
31
Physical Hardware
Hypervisor
Priv. VM
VM
Mgmt.
OS
Apps
Guest VM1
TodaySlide32
Decouple Management And Operation
System manager runs on its own core32
Core 0
System
Manager
Core 1
NoHypeSlide33
Decouple Management And Operation
System manager runs on its own coreSends an IPI to start/stop a VM33
Core 0
System
Manager
Core 1
IPI
NoHypeSlide34
Decouple Management And Operation
System manager runs on its own coreSends an IPI to start/stop a VMCore manager sets up core, launches VMNot run again until VM is killed34
Core 0
System
Manager
Core 1
Core
Manager
OS
Apps
Guest VM2
IPI
NoHypeSlide35
Removing the Hypervisor SummaryScheduling virtual machines
One VM per coreManaging memoryPre-allocate memory with processor supportEmulating I/O devicesDirect access to virtualized devicesNetworkingUtilize hardware Ethernet switches
Managing virtual machinesDecouple the management from operation
35Slide36
Security BenefitsConfidentiality/Integrity of data
AvailabilitySide channels36Slide37
Security BenefitsConfidentiality/Integrity of data
AvailabilitySide channels37Slide38
Confidentiality/Integrity of Data38
Requires access to the data
System manager can alter memory access rules
But, guest VMs do not interact with the system manager
With
hypervisor
NoHype
Registers upon
VM exit
No scheduling
Packets sent
through software switch
No software switch
Memory
accessible by hypervisor
No hypervisorSlide39
NoHype Double Meaning Means
no hypervisor, also means “no hype”Multi-core processorsAvailable nowExtended (Nested) Page TablesAvailable now
SR-IOV and Directed I/O (VT-d)Network cards now, Storage devices near futureVirtual Ethernet Port Aggregator (VEPA)
Next-generation switches39Slide40
Conclusions and Future WorkTrend towards hosted and shared infrastructures
Significant security issue threatens adoptionNoHype solves this by removing the hypervisorPerformance improvement is a side benefitFuture work:Implement on current hardwareAssess needs for future processors
40Slide41
Questions?
Contact info:ekeller@princeton.eduhttp://www.princeton.edu/~ekellerszefer@princeton.edu
http://www.princeton.edu/~szefer
41