Qingyang Wang Simon Malkowski Yasuhiko Kanemasa Deepal Jayasinghe Pengcheng Xiong Motoyuki Kawaba Lilian Harada Calton Pu 25th IEEE International Parallel amp Distributed Processing Symposium ID: 598469
Download Presentation The PPT/PDF document "The Impact of Soft Resource Allocation o..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
The Impact of Soft Resource Allocation on n-tier Application Scalability
Qingyang Wang, Simon Malkowski, Yasuhiko Kanemasa, Deepal Jayasinghe, Pengcheng Xiong, Motoyuki Kawaba, Lilian Harada, Calton PuSlide2
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011OutlineBackground & MotivationBackgroundMotivational experimentPerformance Impact of Soft Resource Allocation Over-allocation of soft resourcesUnder-allocation of soft resourcesSpecial case of under-allocation SolutionA practical algorithm for good soft resource allocationConclusion & Future WorksSlide3
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Cloud Computing Environment
Good performance
+
Cost efficiency
Scaling applications on demand
High throughput + low response time
High resource utilizationSlide4
Soft resources
in n-tier systemsThreads, database connections, TCP connections, locks, etc. Soft Resource Allocation25th IEEE International Parallel & Distributed Processing Symposium19 May 2011
Is it okay to duplicate the same configuration of soft resource allocation?Slide5
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011OutlineBackground & MotivationBackgroundMotivational experimentPerformance Impact of Soft Resource Allocation Over-allocation of soft resourcesUnder-allocation of soft resourcesSpecial case of under-allocation SolutionA practical algorithm for good soft resource allocationConclusion & Future WorksSlide6
RUBBoS
benchmarkBulletin board system like Slashdot (www.slashdot.org)Typical 3-tier or 4-tier architectureTwo types of workloadBrowsing onlyRead/Write mix24 web interactionsC-JDBCMiddleware for database scale-outRead: act as a load-balancerWrite: send a request to all databases to keep consistencyExperimental Environment (1)
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011Slide7
Emulab
(http://www.emulab.net)Relatively modest testbed originally for network researchVirtual network & physical machines (not VM)Experimental Environment (2)25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Hardware
Specifications
Server type
PC3000 in
Emulab
Processor
Xeon 3GHz 64bit
Memory
2GB
Network
1Gbps
Disk
2
x 146GB 10,000rpmSlide8
Software setups
Experimental Environment (3)25th IEEE International Parallel & Distributed Processing Symposium19 May 2011FunctionSoftware
Web server
Apache 2.0.54
Application server
Apache
Tomcat 5.5.17
DB clustering
middleware
C-JDBC 2.0.2
Database server
MySQL
5.0.51a
Java
Sun jdk1.6.0_14
Operating
system
Redhat
FC4
System Monitor
Sysstat
7.0.2Slide9
Notation
Experimental Environment (4)25th IEEE International Parallel & Distributed Processing Symposium19 May 2011400-6-200
1 / 3 / 1 / 2 configurationSlide10
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011Result of Motivational Experiment
Scale outSlide11
Hardware configuration 2
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Challenge
Hardware configuration
Soft resource allocation
Hardware configuration
1
Soft resource allocation 1
Thread pool,
DB connection pool
Thread pool,
DB connection pool
Soft resource allocation 2
Good performance
Bad performance
Good performance
Scale out
How to choose a reasonable soft resource allocation to match the hardware configuration?Slide12
Evaluate two important soft resources
ThreadsDatabase connectionsShow their performance impact on n-tier applicationsOver-allocation & under-allocation casesSpecial case of under-allocationIntroduce a practical way to choose a reasonable allocation of soft resourcesFocus of This Paper
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011Slide13
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011OutlineBackground & MotivationBackgroundMotivational experimentPerformance Impact of Soft Resource Allocation Over-allocation of soft resourcesUnder-allocation of soft resourcesSpecial case of under-allocation SolutionA practical algorithm for good soft resource allocationConclusion & Future WorksSlide14
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011Performance Loss due to Soft Resource Over-allocation (1)400-200-6
Sensitivity
analysis
: change
DB Connection pool
size
in TomcatSlide15
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
50
6
100
200
Performance degradation
Goodput
1/4/1/4
CPU utilization in CJDBC
6
200
Throughput with response time boundary
Performance Loss due to Soft Resource Over-allocation (2)
High allocation of DB connections in Tomcat degrades the system performanceSlide16
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011JVM Garbage Collection CostsOver-allocation of soft resources causes waste of critical hardware resources
200
6
8% more time used for GC over experimental time
JVM Garbage Collection in CJDBC Slide17
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011OutlineBackground & MotivationBackgroundMotivational experimentPerformance Impact of Soft Resource Allocation Over-allocation of soft resourcesUnder-allocation of soft resourcesSpecial case of under-allocation SolutionA practical algorithm for good soft resource allocationConclusion & Future WorksSlide18
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011Performance Loss due to Soft Resource Under-allocation (1)400-6-200
Sensitivity
analysis
: change
thread
pool
size
in TomcatSlide19
6
200
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Performance Loss due to Soft Resource Under-allocation (2)
Goodput
1/2/1/2
20
10
CPU utilization of Tomcat
20
6
Under-allocation of soft resources causes inefficient utilization of hardware resourcesSlide20
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011OutlineBackground & MotivationBackgroundMotivational experimentPerformance Impact of Soft Resource Allocation Over-allocation of soft resourcesUnder-allocation of soft resourcesSpecial case of under-allocation SolutionA practical algorithm for good soft resource allocationConclusion & Future WorksSlide21
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011Special Case of Soft Resource Under-allocation30-6-100
Sensitivity
analysis
: change
thread
pool
size
in ApacheSlide22
30
-6-100400-6-100
50-6-100
100
-6-10025th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Performance Loss due to Under-allocation of Apache Threads
Goodput
1/4/1/4
CPU utilization in CJDBC
30
-6-100
400
-6-100
Low allocation of Apache threads degrades
the system performanceSlide23
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011Non-Trivial Correlation between Apache and Tomcat Threads (1)30-6-100
30 > 6 * 4
Why are 30 threads in Apache not enough? Slide24
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011Non-Trivial Correlation between Apache and Tomcat Threads (2)
Waiting for TCP FIN reply from the clientSlide25
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011Non-Trivial Correlation between Apache and Tomcat Threads (3)
The long waiting time for FIN reply from clients is unpredictable and frequently happens under high workload
Waiting for TCP FIN reply from clientSlide26
Concurrency for
30-6-100Concurrency for 400
-6-100
Large number of soft resources in front tier acts as a buffer
providing stable workload for lower tiers25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Concurrency of Apache Threads
Connecting to TomcatSlide27
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011Summary of ExperimentsOver-allocation of soft resources causes waste of critical hardware resourcesUnder-allocation of soft resources causes inefficient utilization of hardware resources
Large number of soft resources in front tier acts as a
buffer
providing stable workload for downstream tiersSlide28
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011OutlineBackground & MotivationBackgroundMotivational experimentPerformance Impact of Soft Resource Allocation Over-allocation of soft resourcesUnder-allocation of soft resourcesSpecial case of under-allocation SolutionA practical algorithm for good soft resource allocationConclusion & Future WorksSlide29
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011Soft Resource Allocation AlgorithmKey idea: allocating soft resources globally to utilize the critical hardware resource as efficiently as possible Slide30
Identifying the critical hardware resource first
Soft Resource Allocation Algorithm (1)25th IEEE International Parallel & Distributed Processing Symposium19 May 2011
(1) Bottleneck server Slide31
Identifying the critical hardware resource first
Allocating proper soft resources for the bottleneck serverSoft Resource Allocation Algorithm (2)25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
Throughput knee
Minimum Saturation workload
(2) Ave. number of active threads
(1) Bottleneck server Slide32
(3) Dependency between current tier and bottleneck tier
Identifying the critical hardware resource firstAllocating proper soft resources for the bottleneck serverAllocating proper amount soft resources in other tiers
Soft Resource Allocation Algorithm (3)
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
(2) Ave. number of active threads
(1) Bottleneck server Slide33
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011OutlineBackground & MotivationBackgroundMotivational experimentPerformance Impact of Soft Resource Allocation Over-allocation of soft resourcesUnder-allocation of soft resourcesSpecial case of under-allocation SolutionA practical algorithm for good soft resource allocationConclusion & Future WorksSlide34
Achieving
good performance by scaling n-tier applications in Cloud requires a unified exploration of both hardware and softwareContributions: We showed allocation of soft resources has a big impact on the total performance of an N-tier system We showed to decide a proper soft-resources allocation is a complex problem, especially in cloud environments which requires dynamic scaling-out/inWe gave a practical algorithm for proper soft resource allocation
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011
ConclusionSlide35
25th IEEE International Parallel & Distributed Processing Symposium
19 May 2011Future WorkExplore more soft resources such as locks, buffer/queueExplore more efficient ways to find proper soft resource allocationExplore the impact of soft resource allocation in virtualized environment Slide36
Thank You. Any Questions?
Qingyang
Wang
qywang
@cc.gatech.edu